Verified errors in high-impact papers: write-ups
2026-10-28 05:09 UTC. 272 findings, each documented from the paper's own text and numbers (and, for trials, the registry's version history), with a runnable script and figures. Wording is factual and makes no claim about intent; none of these has yet been raised with the authors or journals.
How each one was checked. Numerical errors were found by a cheap full-paper screen (Claude Haiku 5.5 or GPT-6 Luna), confirmed by two cheap verifiers and then by Opus 5.5 and GPT-6.1 Sol independently. Registry discrepancies were flagged by GPT-6 Luna and Claude Haiku 5.5 comparing each trial report with its ClinicalTrials.gov record and confirmed by Opus. For this page each was checked again by agents with Python and web access: the publisher's version of record and supplements; Crossref, Europe PMC, PubPeer and Retraction Watch for corrections; for trials, the registry's version history (what was registered before enrollment and at publication), the paper's role and any disclosure; and a blind re-check by an agent that never saw the earlier verdicts. A separate agent then fact-checked each write-up, re-running the script, verifying every quote and date, and inspecting every figure; write-ups it rejected were revised up to twice.
Severity (from a separate consequence assessment): 5 = a headline conclusion or decision-relevant estimate is wrong (for trials: the conclusion rests on an outcome that was not the registered primary); 4 = a key result is wrong but the main conclusion survives; 3 = a secondary result; 2 = a misstatement a careful reader would catch; 1 = cosmetic.
Index
Numerical and internal-consistency errors (261)
Undisclosed differences from the trial registration (11)
2 verified finding(s) are held back because their write-up did not pass the fact-check after two revisions: PMC6878715, PMC12757054.
Interim primary result is called significant, but p = 0.039 exceeds the trial's stated 0.0054 threshold
Cell (2020) · doi:10.1016/j.cell.2020.08.051 · PMC7462457
Summary
The paper reports an interim analysis of the ACTIVATE trial, which randomized hospitalized elderly patients to BCG or placebo. Its Abstract and Results say that BCG "significantly increased the time to first infection", and the Abstract concludes that BCG "can protect the elderly against infections". The STAR Methods and the supplementary statistical plan set the significance level for this interim analysis at α = 0.0054 (O'Brien-Fleming adjustment), but the primary-outcome hazard ratio is 0.55 (95% CI 0.31–0.97) with p = 0.039, about seven times that level. At the stated interim level the interval is about 0.24–1.24, which includes 1. The stepwise Cox result and the Abstract's viral-respiratory result also fail to meet the threshold. The paper's headline efficacy claim is therefore not supported under the significance level the paper itself states for the interim analysis, and the paper does not say so.
What the paper reports
Abstract:
At interim analysis, BCG vaccination significantly increased the time to first infection (median 16 weeks compared to 11 weeks after placebo). The incidence of new infections was 42.3% (95% CIs 31.9%–53.4%) after placebo vaccination and 25.0% (95% CIs 16.4%–36.1%) after BCG vaccination; most of the protection was against respiratory tract infections of probable viral origin (hazard ratio 0.21, p = 0.013).
Data show that BCG vaccination is safe and can protect the elderly against infections.
Introduction, paragraph 4:
Results of this interim analysis clearly showed protection of the elderly from new infections with major effect on the prevention of respiratory infections.
Results, "BCG Vaccination Decreases the Incidence of New Infections with a Major…", paragraph 1:
Regarding the primary endpoint of the study, BCG vaccination significantly increased the time to first infection: median 16 weeks after BCG vaccine compared to 11 weeks after placebo administration. The incidence of a new infection during the 12-month period of follow-up after vaccination was also significantly decreased; the statistically significant hazard ratio (HR) of 0.55 corresponds to 45% reduction in the risk of a new infection in the BCG group compared to the placebo group (Figure 2 A).
Same section, paragraph 3:
Stepwise Cox regression analysis showed that BCG vaccination was an independent protective factor from the incidence of new infection until month 12 (HR 0.56; 95% CI, 0.32–0.99; p = 0.048) (Table 3 ).
Table 2 ("Primary and Secondary Study Outcomes"), rows 1 and 2 (columns: placebo N = 78, BCG N = 72, HR (95% CI), p):
Incidence at least one new infection until month 12, no. (%)a | 33 (42.3) | 18 (25.0) | 0.55 (0.31–0.97) | 0.039
Respiratory infections of probable viral origin necessitating medical treatment, no. (%) | 14 (17.9) | 3 (4.2) | 0.21 (0.06–0.72) | 0.013
STAR Methods, "Quantification and Statistical Analysis", paragraph 4 (repeated with the same values and only minor wording differences in the supplementary Data S1/Methods S1, "Statistical plan for interim analysis of the ACTIVATE trial"):
The interim analysis included only patients with completed 12-month of follow-up. In order to preserve the overall Type I error rate at 5%, an adjustment of the level of significance of the interim and final analyses was done by O’Brien-Fleming strict alpha adjustment. This adjustment provides significance a = 0.0054 at interim and a = 0.0492 at final (DeMets and Lan, 1994). The purpose of using this seemingly unattainable level of significance at interim analysis was to allow the study to conclude, at the same time providing evidence that the required level of significance will be attained at the final stage.
STAR Methods, same section, later paragraph:
All p values were two-sided and any p value < 0.05 was considered as statistically significant.
Discussion, paragraphs 2 and 5:
In line with this, the incidence of infection in the ACTIVATE trial was significantly lower in the elderly individuals vaccinated with BCG, compared to the non-vaccinated participants.
It needs, however, to be mentioned that despite the small sample size significant differences were found.
What the paper's own numbers show
Method. For a Cox model a 95% Wald interval is exp(b ± 1.96·SE), where b = ln HR. The standard error is therefore SE = (ln upper − ln lower)/(2 × 1.96), the Wald statistic is z = ln HR / SE, and the two-sided p is 2·(1 − Φ(|z|)). An interval at the stated interim level α = 0.0054 uses the critical value z = 2.782 (a 99.46% interval). I applied this to every hazard ratio the paper uses for its efficacy claims, checked the crude counts with 2×2 tests, recomputed O'Brien-Fleming-type interim levels (Lan-DeMets spending, two looks, overall two-sided α = 0.05) for the stated and the actual information fractions, and solved for the information fraction at which the stated pair of levels (0.0054 interim, 0.0492 final) keeps the overall α at 0.05 (bivariate-normal probabilities, checked by direct numerical integration). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Primary outcome, any new infection to month 12 | Table 2 row 1; Results; Fig. 2A | HR 0.55 (0.31–0.97), p = 0.039, "statistically significant" | SE 0.291, z = −2.05, p = 0.040 (consistent); 99.46% CI 0.24–1.24, includes 1 |
| Primary p against the stated interim level | Table 2; STAR Methods | p = 0.039; interim α = 0.0054 | 0.039 > 0.0054 (7.2 times); one-sided p = 0.020, also above |
| Primary p, rounding range | derived | HR 0.55, CI 0.31–0.97 | p = 0.034–0.047 over all rounding of HR and limits |
| Same HR and CI in Table 3 (univariate) | Table 3 | p = 0.035 | also above 0.0054 (Table 2 gives 0.039 for the identical estimate) |
| Stepwise (multivariate) Cox, BCG | Results; Table 3 | HR 0.56 (0.32–0.99), p = 0.048, "independent protective factor" | p = 0.044; 99.46% CI 0.25–1.25, includes 1 |
| Respiratory infections of probable viral origin | Abstract; Table 2 row 2 | HR 0.21 (0.06–0.72), p = 0.013 | p = 0.014; 99.46% CI 0.04–1.23, includes 1 |
| All respiratory infections | Table 2 row 5 | HR 0.25 (0.10–0.60), p = 0.002 | p = 0.0024; 99.46% CI 0.07–0.89, excludes 1 (meets the threshold) |
| Patient-infections per year | Table 2 | p = 0.003 | below 0.0054 (no HR or CI to recompute) |
| Crude 2×2, 33/78 vs 18/72 | Table 2 row 1 counts | 42.3% vs 25.0% | Wilson 95% CIs 32.0–53.4% and 16.4–36.1% (match within 0.1 point); χ² p = 0.025, Yates p = 0.039, Fisher p = 0.038, all above 0.0054 |
| Sensitivity analysis, all 198 patients | Table S2 | HR 0.59 (0.36–0.95), p = 0.029 | p = 0.033; 99.46% CI 0.30–1.17, includes 1 |
| Interim level 0.0054 alone | STAR Methods; Data S1 | α = 0.0054 interim | Matches an O'Brien-Fleming-type boundary at about half the information: Lan-DeMets 0.0056, classical 0.0052 at t = 0.5 |
| Stated pair 0.0054 interim / 0.0492 final | STAR Methods; Data S1 | α = 0.0054 interim, 0.0492 final (t not stated) | Final level that keeps overall α at 0.05 after an interim look at exactly 0.0054: 0.0479 at t = 0.5, 0.0485 at t = 0.6, 0.0492 at t = 0.70, 0.0495 at t = 0.75–0.76. The stated pair is consistent at t = 0.70 (0.69–0.71 allowing for rounding of 0.0492), close to the look actually taken. An O'Brien-Fleming-type interim level at t = 0.70 would be about 0.02 (Lan-DeMets 0.019, classical 0.017), so 0.0054 is stricter than O'Brien-Fleming at that point |
| Interim level at the actual fraction of patients | Results (150 of 198) | not reported | t = 150/198 = 0.758: Lan-DeMets interim α = 0.024 (classical 0.021); p = 0.039 still above |
| Interim level at the actual fraction of events | Table 2; Table S2 | not reported | 51 interim events of at least 68 (41 + 27 in all 198 at the same lock): t ≤ 0.75, α ≤ 0.024 |
| Information fraction at which the interim level would reach 0.039 | derived | n/a | t = 0.90 |
Of the results the paper uses to support efficacy, only "all respiratory infections" (p = 0.002) and patient-infections per year (p = 0.003) meet the interim level the paper states. The primary outcome, the adjusted Cox model, and the viral-respiratory result cited in the Abstract do not. Recalculating the threshold for the fraction of patients or events actually analysed does not change this. The paper's numbers are internally consistent. The problem lies in the verdict drawn from them, not in the arithmetic.
Figure
Figure 1. Hazard ratios (BCG vs placebo, log scale) for the five efficacy results the paper relies on. For each one, the orange circle and line show the hazard ratio and 95% CI as printed in Table 2, Table 3 or Table S2. The blue square and line show the same estimate with the 99.46% interval that corresponds to the interim significance level α = 0.0054 stated in the STAR Methods, computed from the printed CI. The vertical line, labelled in the figure, marks HR = 1, no effect. The p value printed by the paper is given at the right, in bold where it is below 0.0054. Under the paper's own interim rule, the intervals for the primary outcome, the stepwise Cox model, the viral-respiratory endpoint cited in the Abstract, and the full-cohort sensitivity analysis all include 1. Only "all respiratory infections" excludes it.
Figure 2. The blue curve is the recomputed two-sided nominal significance level for an interim look under Lan-DeMets O'Brien-Fleming-type spending (overall α = 0.05), as a function of the information fraction at that look. The orange dotted line is the paper's stated interim level, 0.0054 (the paper does not state the information fraction it assumed); the curve crosses it near t = 0.5 (0.0056 there, blue circle). The grey band (t = 0.69–0.71) is where the stated pair of levels, 0.0054 interim and 0.0492 final, keeps the overall α at exactly 0.05; there the O'Brien-Fleming-type level would be 0.019 (blue diamond), so 0.0054 is the stricter of the two. The orange dashed line is the reported primary-outcome p = 0.039. The filled blue square is the level recomputed for the fraction of patients actually analysed (150 of 198, t = 0.76): 0.024. The open blue square is where the curve reaches 0.039, at t = 0.90. Whichever reading of the stated levels is taken, and even if the threshold is recalculated for the interim analysis as actually performed, the primary result does not meet it. (For a time-to-event endpoint, information is measured in events. The event fraction is at most 51/68 = 0.75, which gives a level of at most 0.024.)
Why it matters
The trial is the paper's only efficacy evidence. Its primary endpoint is the time to first infection, and the paper's central claim, stated in the Abstract, Introduction, Results and Discussion, is that BCG significantly reduced infections. The Methods say that, to keep the overall type I error at 5%, the interim analysis would be judged at α = 0.0054. The primary result (p = 0.039) does not meet that level, and neither does the adjusted Cox analysis (p = 0.048). The Abstract's main secondary figure, protection against respiratory infections of probable viral origin (HR 0.21, p = 0.013), does not meet it either. Read against the paper's own stated interim level, the interim analysis shows a favourable point estimate (HR 0.55) that is compatible with no effect at that level (99.46% CI 0.24–1.24). It does not show a statistically significant effect. The Abstract's conclusion that the data "show that BCG vaccination … can protect the elderly against infections" is therefore stated with more certainty than the paper's stated interim level allows. The paper gives the context of the interim analysis: the Introduction says it was prompted by "the pressure rising from the need of protection of the elderly" during the pandemic; Discussion paragraph 3 says "The reason for this interim analysis that has been approved by National Ethics Committee and by the National Organization for Medicines of Greece was the emergence of the COVID-19 pandemic and the initiation of several major studies on the effect of BCG on the infection with SARS-CoV-2"; and the STAR Methods say the sponsor's initiative for the analysis was submitted to those bodies on 14 April 2020, before the database lock on 21 April 2020. It does not say when the 0.0054 level itself was set, and the point does not depend on that: the paper states 0.0054 as the interim level and then reports results that do not meet it as significant.
The paper was published in September 2020, early in the COVID-19 pandemic. Its Introduction and Discussion place the result in the context of BCG trials against SARS-CoV-2, and press coverage reported it as showing that BCG protects elderly people against respiratory infections (for example News-Medical, 1 September 2020, "BCG vaccine offers protection for elderly people against respiratory infections"; Drug Topics, 9 October 2020, "Study: BCG Vaccine Protects Elderly Against Respiratory Infections"). The News-Medical story reports the infection rates but does not mention the interim significance level. Readers of the Abstract have no way to tell that the headline result missed the trial's own interim threshold.
Two results are not affected. "All respiratory infections" (HR 0.25, p = 0.002) and patient-infections per year (p = 0.003) meet the 0.0054 level. Both are secondary endpoints, and the paper does not designate either as a confirmatory test. The safety findings are also unaffected. ClinicalTrials.gov lists the trial (NCT03296423) as completed in November 2020, with no results posted. I did not find a published final analysis that would show whether the final threshold (0.0492) was met.
Explanations considered
- The paper's general rule "any p value < 0.05 was considered as statistically significant". This explains the wording but does not resolve the conflict. It is a general statement in a short general paragraph near the end of the section, after the cytokine analyses. The interim-specific paragraph, and the supplementary statistical plan, state that the interim analysis is judged at 0.0054 "in order to preserve the overall Type I error rate at 5%". Judging the interim analysis at 0.05 discards that adjustment, and the paper nowhere says the adjustment was dropped.
- The "seemingly unattainable" sentence means the 0.0054 level was never intended as the decision rule. This does not account for the numbers. The sentence says the strict level was used "to allow the study to conclude" while "providing evidence that the required level of significance will be attained at the final stage". Whatever its intended meaning, it does not state that the primary endpoint met the interim level, and the Abstract and Results still call the result significant without qualification.
- The 0.0054 level was computed for a different information fraction, or is not a standard O'Brien-Fleming boundary. This does not change the conclusion. Two readings of the stated levels fit the numbers. Taken alone, 0.0054 matches an O'Brien-Fleming-type boundary at about half the information (Lan-DeMets 0.0056, classical 0.0052 at t = 0.5). Taken as a pair, 0.0054 interim and 0.0492 final keep the overall α at exactly 0.05 for a single interim look at t = 0.70 (0.69–0.71 allowing for rounding of 0.0492), close to the look actually taken; at that fraction an O'Brien-Fleming-type boundary would be about 0.02 (Lan-DeMets 0.019, classical 0.017), so the stated interim level is stricter than O'Brien-Fleming there. The stated levels are therefore internally consistent with a valid two-look design, and the paper's label "O'Brien-Fleming" fits the interim value only at t = 0.5. Under either reading 0.0054 is the level the paper states for the interim analysis, and p = 0.039 does not meet it. The analysis itself used 150 of 198 patients (t = 0.76). Recalculated at t = 0.76, the interim level is 0.024 (classical O'Brien-Fleming: 0.021). Measured by events it is at most 0.024. The reported p = 0.039 exceeds all of these. It would meet an O'Brien-Fleming-type interim level only at t = 0.90 or later.
- One-sided testing. This does not account for the numbers. The paper states that all p values were two-sided, and the one-sided p (0.020) also exceeds 0.0054.
- A different test or rounding. This does not account for the numbers. The Wald p recomputed from the printed HR and CI (0.040) matches the reported 0.039. All rounding of the inputs gives 0.034–0.047. Table 3 reports p = 0.035 for the identical HR and CI, and crude χ², Yates and Fisher tests on the counts give 0.025–0.039. None is below 0.0054.
- The full-cohort sensitivity analysis supersedes the interim result. This does not account for the numbers. Table S2's analysis of all 198 patients (HR 0.59, 0.36–0.95, p = 0.029) is a second look at incompletely followed data, not the final analysis, and it is also above 0.0054. The paper presents it as confirming the interim result, not as a test at a different level.
- The interim analysis was for safety or for planning, so efficacy significance was not claimed. This does not account for the text. The Abstract, Introduction, Results and Discussion all state the efficacy finding as significant or "clearly" shown.
- Conversion or typesetting error in the archived text. This is excluded. The publisher-supplied JATS XML in Europe PMC and the publisher's supplementary PDF carry the same values: p = 0.039 in a separate table cell, and α = 0.0054 in both the STAR Methods and Data S1/Methods S1.
Status
- Version of record. The publisher pages (cell.com, sciencedirect.com) returned HTTP 403 to automated access, and doi.org resolves to an Elsevier redirect stub. I therefore compared the PMC-archived text with the publisher-supplied JATS XML in Europe PMC (PMC7462457) and with the publisher's Supplemental Information PDF (mmc1.pdf, "Cell, Volume 183 Supplemental Information"). All quoted sentences and table values agree. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. Document S1 (mmc1.pdf), the only supplementary item, holds Tables S1–S4 and Data S1/Methods S1. Data S1/Methods S1, the statistical plan for the interim analysis (supplement p. 3), repeats "a=0.0054 at interim and a=0.0492 at final". Table S2 gives the interim analysis (33/78 vs 18/72, HR 0.55, 0.31–0.97, p = 0.035) and all 198 patients (41/98 vs 27/100, HR 0.59, 0.36–0.95, p = 0.029). Nothing in the supplement states that the interim threshold was met, revised or abandoned.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI (its only relation is a Faculty Opinions review), and PMC and Europe PMC list no correction notices. A web search for comments on the trial's interim threshold found none. The PubPeer API refused automated access (HTTP 403), so PubPeer could not be checked directly. The ClinicalTrials.gov record (NCT03296423) lists the trial as completed, has no posted results, and lists no results publication other than this paper.
- Sources. PMC article PMC7462457; Europe PMC full-text XML and supplementary-files bundle for PMC7462457; Crossref; ClinicalTrials.gov API record NCT03296423.
Suggested correction
- State in the Abstract and Results that the interim primary-outcome result (HR 0.55, 95% CI 0.31–0.97, p = 0.039) did not meet the interim significance level of α = 0.0054 that the STAR Methods state. Replace "significantly increased the time to first infection" and "statistically significant hazard ratio" with wording that reports the estimate as nominally significant at 0.05 but not at the interim boundary.
- Report the interval at the interim level (about 0.24–1.24) alongside the 95% CI for the primary outcome, or state the boundary explicitly next to each efficacy p value in Table 2 and Table 3.
- Qualify the Abstract's conclusion ("can protect the elderly against infections") and the Introduction's "clearly showed protection" accordingly. Note that the viral-respiratory result (p = 0.013) and the stepwise Cox result (p = 0.048) also do not meet the interim level, and that only "all respiratory infections" (p = 0.002) and patient-infections per year (p = 0.003) do.
- State the information fraction and spending rule behind the interim level. The stated pair (0.0054 / 0.0492) preserves the overall α for a look at about 70% of the information, but an O'Brien-Fleming-type boundary at that fraction would be about 0.02, and 0.0054 alone matches O'Brien-Fleming only at about 50%. Give the boundary that applies to the look actually performed (150 of 198 patients, at most 75% of events); no boundary recomputed here for that look (0.021–0.024) is met by p = 0.039.
- Minor: reconcile the p value for the identical primary estimate (0.039 in Table 2, 0.035 in Table 3 and Table S2) or name the test behind each.
- Report the final analysis of all 198 patients against the stated final level (α = 0.0492), which is consistent with an interim look at 0.0054 at about 70% of the information.
Recomputation script (python recompute.py.txt)
"""Recompute the interim significance checks of PMC7462457
(Cell 2020, doi:10.1016/j.cell.2020.08.051, ACTIVATE trial of BCG vaccination in the elderly).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (and its Supplemental Information, Table S2):
Table 2, row 1 (primary): any new infection to month 12, 33/78 vs 18/72, HR 0.55 (0.31-0.97), p = 0.039
Table 2, row 2: respiratory infections of probable viral origin, HR 0.21 (0.06-0.72), p = 0.013
Table 2, row 3: community-acquired pneumonia, HR 0.38 (0.10-1.43), p = 0.153
Table 2, row 5: all respiratory infections, HR 0.25 (0.10-0.60), p = 0.002
Table 2: patient-infections per year, p = 0.003 (no HR or CI given)
Table 3: BCG, univariate HR 0.55 (0.31-0.97) p = 0.035; multivariate HR 0.56 (0.32-0.99) p = 0.048
Table S2: interim 150, 33/78 vs 18/72, HR 0.55 (0.31-0.97), p = 0.035;
all 198, 41/98 vs 27/100, HR 0.59 (0.36-0.95), p = 0.029
STAR Methods: O'Brien-Fleming adjustment, alpha = 0.0054 at interim and 0.0492 at final;
interim = 150 of 198 patients (78 + 72) with completed 12-month follow-up.
Method: for a Cox model a 95% Wald CI is exp(b +/- 1.96 SE), so SE = (ln hi - ln lo)/(2*1.96),
z = ln(HR)/SE, two-sided p = 2*(1 - Phi(|z|)), and a 100*(1 - a)% CI is exp(b +/- z_{1-a/2} SE).
"""
import math
import os
import numpy as np
from scipy import stats, optimize
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
import matplotlib.ticker as mticker # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
N = stats.norm
Z95 = N.isf(0.025)
A_INT, A_FIN = 0.0054, 0.0492 # stated interim / final significance levels
Z_INT = N.isf(A_INT / 2) # two-sided critical z at the interim level
# key, label, HR, lo, hi, reported p, location
ROWS = [
("primary", "Any new infection (primary)", 0.55, 0.31, 0.97, 0.039, "Table 2 row 1; Fig. 2A"),
("cox", "Any new infection, stepwise Cox", 0.56, 0.32, 0.99, 0.048, "Table 3 (multivariate); Results"),
("viral", "Resp. infection, probable viral", 0.21, 0.06, 0.72, 0.013, "Table 2 row 2; Abstract"),
("cap", "Community-acquired pneumonia", 0.38, 0.10, 1.43, 0.153, "Table 2 row 3"),
("allresp", "All respiratory infections", 0.25, 0.10, 0.60, 0.002, "Table 2 row 5"),
("s2all", "Any new infection, all 198 (Table S2)", 0.59, 0.36, 0.95, 0.029, "Table S2"),
]
def wald(hr, lo, hi):
se = (math.log(hi) - math.log(lo)) / (2 * Z95)
z = math.log(hr) / se
return se, z, 2 * N.sf(abs(z))
R = {}
print("=== 1. Wald p and interim-level (99.46%) CI from each reported HR and 95% CI ===")
print(f"interim alpha {A_INT} -> two-sided critical z = {Z_INT:.3f}; CI level = {100*(1-A_INT):.2f}%")
for key, lab, hr, lo, hi, prep, loc in ROWS:
se, z, p = wald(hr, lo, hi)
mid = math.exp((math.log(lo) + math.log(hi)) / 2)
l99, h99 = math.exp(math.log(hr) - Z_INT * se), math.exp(math.log(hr) + Z_INT * se)
R[key] = dict(lab=lab, hr=hr, lo=lo, hi=hi, prep=prep, se=se, z=z, p=p, l99=l99, h99=h99)
print(f" {lab:40s} [{loc}] HR {hr} ({lo}-{hi}) reported p={prep}: SE={se:.3f} z={z:.2f} "
f"Wald p={p:.4f}; CI log-midpoint {mid:.3f}; 99.46% CI {l99:.2f}-{h99:.2f}; "
f"p<{A_INT}? reported {prep < A_INT}, recomputed {p < A_INT}; 99.46% CI excludes 1? {h99 < 1}")
print(" Patient-infections per year (Table 2): reported p = 0.003 < 0.0054? ", 0.003 < A_INT)
print("\n=== 2. Primary outcome: rounding and sidedness ===")
se = R["primary"]["se"]
for h in (0.545, 0.555):
print(f" HR {h} with SE {se:.3f}: p = {2*N.sf(abs(math.log(h)/se)):.4f}")
ps = [wald(h, l, u)[2] for h in (0.545, 0.555) for l in (0.305, 0.315) for u in (0.965, 0.975)]
print(f" over HR 0.545-0.555 and each limit +/-0.005: p = {min(ps):.4f}-{max(ps):.4f}")
print(f" one-sided p = {R['primary']['p']/2:.4f} (still > {A_INT})")
print(f" reported p 0.039 is {0.039/A_INT:.1f} times the interim alpha")
print(f" Table 3 univariate p for the same HR and CI: 0.035 (Table 2: 0.039); both > {A_INT}")
print("\n=== 3. Crude 2x2 check of the primary counts (33/78 vs 18/72) ===")
tab = [[33, 45], [18, 54]]
for k, n in ((33, 78), (18, 72)):
c = k / n
zz = Z95
den = 1 + zz**2 / n
cen = (c + zz**2 / (2 * n)) / den
half = zz * math.sqrt(c * (1 - c) / n + zz**2 / (4 * n * n)) / den
print(f" {k}/{n} = {100*c:.1f}% Wilson 95% CI {100*(cen-half):.1f}-{100*(cen+half):.1f}%")
_, pc, _, _ = stats.chi2_contingency(tab, correction=False)
_, py, _, _ = stats.chi2_contingency(tab, correction=True)
pf = stats.fisher_exact(tab)[1]
print(f" chi-square p = {pc:.4f}; Yates p = {py:.4f}; Fisher exact p = {pf:.4f}; all > {A_INT}: {max(pc,py,pf) > A_INT and min(pc,py,pf) > A_INT}")
print(f" risk ratio 25.0/42.3 = {(18/72)/(33/78):.3f}")
print("\n=== 4. Two-look group-sequential boundaries (two-sided overall alpha 0.05) ===")
def final_nominal(t, a1):
"""Final nominal two-sided alpha given interim nominal alpha a1 at information fraction t."""
c1 = N.isf(a1 / 2)
rho = math.sqrt(t)
mvn = stats.multivariate_normal(mean=[0, 0], cov=[[1, rho], [rho, 1]])
def f(c2):
cont = mvn.cdf([c1, c2]) - mvn.cdf([-c1, c2]) - mvn.cdf([c1, -c2]) + mvn.cdf([-c1, -c2])
return cont - 0.95
return 2 * N.sf(optimize.brentq(f, 1.5, 3.5))
def ld_obf(t):
"""Lan-DeMets O'Brien-Fleming-type spending: interim nominal alpha at fraction t."""
return 2 * N.sf(Z95 / math.sqrt(t))
def classic_obf(t):
"""Classical O'Brien-Fleming: boundaries C/sqrt(t_k); returns (interim, final) nominal alpha."""
rho = math.sqrt(t)
mvn = stats.multivariate_normal(mean=[0, 0], cov=[[1, rho], [rho, 1]])
def f(C):
c1, c2 = C / math.sqrt(t), C
cont = mvn.cdf([c1, c2]) - mvn.cdf([-c1, c2]) - mvn.cdf([c1, -c2]) + mvn.cdf([-c1, -c2])
return cont - 0.95
C = optimize.brentq(f, 1.5, 3.5)
return 2 * N.sf(C / math.sqrt(t)), 2 * N.sf(C)
T_ACT = 150 / 198
B = {}
for t in (0.5, T_ACT):
a1 = ld_obf(t)
B[t] = a1
ci, cf = classic_obf(t)
print(f" t = {t:.3f}: Lan-DeMets OBF interim {a1:.4f}, final {final_nominal(t, a1):.4f};"
f" classical OBF interim {ci:.4f}, final {cf:.4f}")
print(f" stated: interim {A_INT}, final {A_FIN}")
print(f" stated interim {A_INT} lies between the t = 0.5 values above (classical / Lan-DeMets),"
f" i.e. it corresponds to a look at about half the information")
print(f" if the look were at t = 0.5, the stated final {A_FIN} would be higher than the recomputed finals "
f"({final_nominal(0.5, ld_obf(0.5)):.4f} / {classic_obf(0.5)[1]:.4f})")
print(f" final level that keeps overall two-sided alpha at 0.05 after an interim look at {A_INT}"
f" with t = 0.5: {final_nominal(0.5, A_INT):.4f} (stated: {A_FIN}; see 4b for other t)")
print(f" (the two nominal levels may add to more than 0.05 because the two tests are correlated:"
f" Lan-DeMets t = 0.5 sum = {ld_obf(0.5) + final_nominal(0.5, ld_obf(0.5)):.4f})")
print("\n=== 4b. Which information fraction does the stated pair (0.0054 interim, 0.0492 final) fit? ===")
print(" final level that keeps overall two-sided alpha at 0.05 after an interim look at exactly 0.0054:")
for t in (0.5, 0.6, 0.65, 0.7, 0.75, 0.758, 0.8):
print(f" t = {t:.3f}: final {final_nominal(t, A_INT):.4f}; OBF-type interim at this t:"
f" Lan-DeMets {ld_obf(t):.4f}, classical {classic_obf(t)[0]:.4f}")
def t_for_final(target):
return optimize.brentq(lambda t: final_nominal(t, A_INT) - target, 0.3, 0.95)
T_PAIR = t_for_final(A_FIN)
T_PAIR_LO, T_PAIR_HI = t_for_final(0.04915), t_for_final(0.04925)
print(f" stated final {A_FIN} is reached at t = {T_PAIR:.3f} "
f"(t = {T_PAIR_LO:.3f}-{T_PAIR_HI:.3f} for a final level rounded from 0.04915-0.04925)")
print(f" at t = {T_PAIR:.2f}, an O'Brien-Fleming-type interim level would be: Lan-DeMets {ld_obf(T_PAIR):.4f},"
f" classical {classic_obf(T_PAIR)[0]:.4f}; stated 0.0054 is stricter")
print(f" check with direct quadrature (independent of the multivariate-normal cdf):")
from scipy import integrate
def cont_quad(t, c1, c2):
# P(|Z1| < c1, |Z2| < c2), Z2 = sqrt(t) Z1 + sqrt(1-t) W
r, s = math.sqrt(t), math.sqrt(1 - t)
g = lambda z1: N.pdf(z1) * (N.cdf((c2 - r * z1) / s) - N.cdf((-c2 - r * z1) / s))
return integrate.quad(g, -c1, c1, epsabs=1e-12)[0]
c1q, c2q = N.isf(A_INT / 2), N.isf(A_FIN / 2)
print(f" overall alpha with 0.0054 / 0.0492 at t = 0.5: {1 - cont_quad(0.5, c1q, c2q):.5f};"
f" at t = {T_PAIR:.2f}: {1 - cont_quad(T_PAIR, c1q, c2q):.5f}")
print(f" whatever t is assumed, primary p 0.039 > interim {A_INT}: {0.039 > A_INT}")
print(f"\n interim patient fraction 150/198 = {T_ACT:.3f}")
t_star = (Z95 / N.isf(0.039 / 2)) ** 2
T_EV = 51 / 68 # interim events (33 + 18) / events among all 198 at the same lock (41 + 27, Table S2)
print(f" event fraction: interim events 33+18 = 51; all 198 at the same lock 41+27 = 68 (Table S2);"
f" 51/68 = {T_EV:.3f} is an upper bound on the event-based fraction (final events >= 68)")
print(f" Lan-DeMets OBF interim alpha at t = {T_EV:.3f}: {ld_obf(T_EV):.4f} (upper bound)")
print(f" Lan-DeMets OBF interim alpha equals 0.039 only at t = {t_star:.3f}")
print(f" with the interim alpha recomputed at t = {T_ACT:.3f} ({B[T_ACT]:.4f}), primary p 0.039 still exceeds it: {0.039 > B[T_ACT]}")
Z_ACT = N.isf(B[T_ACT] / 2)
s = R["primary"]["se"]
print(f" primary-outcome CI at that level ({100*(1-B[T_ACT]):.1f}%): "
f"{math.exp(math.log(0.55)-Z_ACT*s):.2f}-{math.exp(math.log(0.55)+Z_ACT*s):.2f}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
LW, MS = 2.0, 8.0
# Figure 1: reported 95% CI vs 99.46% CI implied at the stated interim alpha
order = ["primary", "cox", "viral", "allresp", "s2all"]
fig, ax = plt.subplots(figsize=(7.2, 4.1), facecolor=BG)
ax.set_facecolor(BG)
for i, k in enumerate(order):
r = R[k]
y = len(order) - 1 - i
ax.plot([r["lo"], r["hi"]], [y + 0.14] * 2, color=ORANGE, lw=LW, solid_capstyle="butt")
ax.plot(r["hr"], y + 0.14, "o", color=ORANGE, ms=MS)
ax.plot([r["l99"], r["h99"]], [y - 0.14] * 2, color=BLUE, lw=LW, solid_capstyle="butt")
ax.plot(r["hr"], y - 0.14, "s", color=BLUE, ms=MS - 0.5)
ax.text(0.99, y, f"p = {r['prep']:g}", va="center", ha="right", fontsize=10,
transform=ax.get_yaxis_transform(),
color=INK if r["prep"] < A_INT else INK2,
fontweight="bold" if r["prep"] < A_INT else "normal")
ax.axvline(1, color=INK2, lw=0.9)
ax.text(0.96, len(order) - 0.25, "HR = 1 (no effect)", fontsize=9.5, color=INK2, va="top", ha="right")
ax.set_xscale("log")
ax.set_xlim(0.03, 3.6)
ax.xaxis.set_major_locator(mticker.FixedLocator([0.05, 0.1, 0.2, 0.5, 1]))
ax.xaxis.set_major_formatter(mticker.FixedFormatter(["0.05", "0.1", "0.2", "0.5", "1"]))
ax.xaxis.set_minor_locator(mticker.NullLocator())
ax.set_yticks(range(len(order)))
ax.set_yticklabels([R[k]["lab"] for k in order][::-1])
ax.set_ylim(-0.6, len(order) - 0.2)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
ax.set_xlabel("Hazard ratio, BCG vs placebo (log scale; < 1 favours BCG)")
ax.text(0.99, 1.0, "reported p\n(< 0.0054 needed)", fontsize=9.5, color=INK2, va="bottom", ha="right",
transform=ax.transAxes)
leg = [Line2D([], [], color=ORANGE, lw=LW, marker="o", ms=MS, label="Reported: 95% CI (paper)"),
Line2D([], [], color=BLUE, lw=LW, marker="s", ms=MS - 0.5,
label="Recomputed: 99.46% CI at the stated interim alpha 0.0054")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.42, -0.17), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 2: the interim significance level as a function of information fraction
ts = np.linspace(0.3, 1.0, 300)
fig, ax = plt.subplots(figsize=(7.2, 4.9), facecolor=BG)
ax.set_facecolor(BG)
ax.axvspan(T_PAIR_LO, T_PAIR_HI, color=GRID, alpha=0.9, lw=0, zorder=0)
ax.text(T_PAIR_LO - 0.008, 0.00085,
f"grey band: the stated pair\n0.0054 / 0.0492 keeps overall\nalpha at 0.05 at t = {T_PAIR:.2f}",
fontsize=10, color=INK2, va="center", ha="right", multialignment="left")
ax.plot(ts, [ld_obf(t) for t in ts], color=BLUE, lw=LW,
label="Recomputed: O'Brien-Fleming-type interim alpha (Lan-DeMets)")
ax.axhline(0.039, color=ORANGE, lw=LW, ls="--", label="Reported primary-outcome p = 0.039")
ax.axhline(A_INT, color=ORANGE, lw=LW, ls=":", label="Stated interim alpha 0.0054 (no t given)")
ax.plot(0.5, ld_obf(0.5), "o", color=BLUE, ms=MS, zorder=5)
ax.annotate(f"t = 0.5: {ld_obf(0.5):.4f}", (0.5, ld_obf(0.5)), (0.535, 0.0027), fontsize=10,
color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.plot(T_PAIR, ld_obf(T_PAIR), "D", color=BLUE, ms=MS, zorder=5)
ax.annotate(f"t = {T_PAIR:.2f}: {ld_obf(T_PAIR):.3f}", (T_PAIR, ld_obf(T_PAIR)), (0.50, 0.020), fontsize=10,
color=INK, ha="left", arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.plot(T_ACT, B[T_ACT], "s", color=BLUE, ms=MS, zorder=5)
ax.annotate(f"at 150/198 = {T_ACT:.2f}: {B[T_ACT]:.3f}", (T_ACT, B[T_ACT]), (0.80, 0.0105), fontsize=10,
color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.plot(t_star, 0.039, "s", color=BLUE, ms=MS, mfc=BG, mew=2.0, zorder=5)
ax.annotate(f"would reach 0.039 only at t = {t_star:.2f}", (t_star, 0.039), (0.62, 0.068), fontsize=10,
color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.set_yscale("log")
ax.set_ylim(0.0005, 0.12)
ax.yaxis.set_major_locator(mticker.FixedLocator([0.001, 0.002, 0.005, 0.01, 0.02, 0.05, 0.1]))
ax.yaxis.set_major_formatter(mticker.FixedFormatter(["0.001", "0.002", "0.005", "0.01", "0.02", "0.05", "0.1"]))
ax.yaxis.set_minor_locator(mticker.NullLocator())
ax.set_xlim(0.3, 1.0)
ax.set_xlabel("Information fraction at the interim look")
ax.set_ylabel("Two-sided significance level")
ax.grid(color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.17), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Reported correlations of |r| = 0.26–0.37 cannot reach P < 0.01 with 27 participants
Cereb Cortex (2023) · doi:10.1093/cercor/bhad233 · PMC10431748
Summary
The paper reports eight Spearman correlations between the tone-deviant mismatch negativity (MMN) and cognitive scores in adults with Down syndrome, with |r| between 0.26 and 0.37, and labels them significant at P < 0.01, P < 0.001 or P < 0.0001. The stated analysis sample is 27 participants, which the paper's Supplementary Table 1 confirms. At n = 27 these coefficients give two-sided P values of 0.057 to 0.19, so none meets the paper's own alpha of 0.01, and none reaches 0.05. These eight correlations are the only evidence for the title claim that the MMN is "an index of cognitive abilities", and for the associations with memory, verbal ability and attention stated in the Abstract and Conclusions.
What the paper reports
Results, "Spearman’s rank correlation coefficients" (the whole paragraph):
The amplitude of the Tone deviant MMN (i.e. the difference between Tone deviant and Frequent sounds) significantly negatively correlated with the number of PAL stages completed (r = -0.26, P < 0.01). Finally, Tone deviant MMN latency significantly negatively correlated with the KBIT-2 composite IQ score (r = -0.37, P < 0.0001), KBIT-2 verbal IQ score (r = -0.31, P < 0.0001), KBIT-2 verbal raw score (r = -0.35, P < 0.0001), KBIT-2 non verbal IQ score (r = -0.27, P < 0.001), PAL first trial memory score (r = -0.35, P < 0.0001), SRT total score (r = -0.30, P < 0.01), and significantly positively correlated with the SRT standard deviation score (r = 0.30, P < 0.001), as shown in Fig. 5.
Results, "Sample description":
The final sample for analyses consisted of 27 individuals (M = 32.4 years, SD = 11.5; age range = 17–51; 14 females), of which 12 had mild intellectual disability, 14 moderate, and 1 severe.
Materials and methods, "Statistical analyses":
Correlations between mean amplitude and latency of both Pitch and Tone deviant MMN and neuropsychological scores were conducted using Spearman’s rank correlation coefficients. This exploratory approach enabled the examination of potential associations between the auditory response and cognitive function without a priori hypotheses. To minimize type I error likelihood arising from multiple correlations, the alpha significance level was set to P < 0.01.
Fig. 5 caption (beginning):
Spearman’s correlation between MMN parameters and cognitive task scores (P < 0.01).
Fig. 5 itself is a triangular heat map in which only some cells carry a coefficient. The MMN rows show the eight coefficients above and one more, MMN latency vs K-BIT 2 non-verbal raw, −0.14. Among the cognitive-cognitive cells, the smallest shown include −0.11 (K-BIT 2 verbal IQ vs SRT mean) and −0.15 (K-BIT 2 composite IQ vs SRT mean). I read these values from the figure image (bhad233f5.jpg).
Abstract:
Reduced MMN amplitude was associated with lower memory scores, while longer MMN latencies were associated with poorer memory, verbal abilities, and attention. Therefore, the MMN may represent a valuable index of cognitive abilities in DS.
Conclusions:
The prolonged MMN latencies appear to be associated with memory, verbal IQ, and general IQ, suggesting that delays in speech-sound processing may underlie a more global detrimental effect on cognition in this population that precedes the development of AD; though this might be further worsened by AD pathology.
Discussion, "Strengths, limitations, and future directions":
Additionally, whilst the correlational analyses provide insight into the relationship between MMN characteristics and cognitive function, it should be noted that they were conducted in an exploratory manner without correction for multiple comparisons, which may increase the risk of Type I errors.
What the paper's own numbers show
Method. For a two-sided test of a Spearman coefficient r with n observations, the usual approximation is t = r·√((n − 2)/(1 − r²)) on n − 2 degrees of freedom. I applied it at the stated n = 27 (df = 25). Because the t approximation can be imprecise for rank correlations in small samples, I also computed each P value from a Monte Carlo approximation of the exact permutation null of Spearman's rho for n = 27 without ties (200,000 random permutations). I then checked whether rounding of the printed coefficients matters, what n each coefficient would need to meet its reported threshold, and whether the larger counts in the paper (all 47 recruited participants, or 54 = 27 × 2 electrodes) would reconcile the numbers. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Analysis sample | Results, Sample description; Table 3 title; Abstract | n = 27 | Supplementary Table 1: 47 rows, 27 marked as included (14 female; 12 mild, 14 moderate, 1 severe; age mean 32.4, SD 11.5, range 17–51) |
| Tone MMN amplitude vs PAL stages completed | Results; Fig. 5 | r = −0.26, P < 0.01 | t = −1.35, P = 0.19 (permutation 0.19); needs n ≥ 98 for P < 0.01 |
| Tone MMN latency vs KBIT-2 composite IQ | Results; Fig. 5 | r = −0.37, P < 0.0001 | t = −1.99, P = 0.057 (permutation 0.058); needs n ≥ 106 |
| Tone MMN latency vs KBIT-2 verbal IQ | Results; Fig. 5 | r = −0.31, P < 0.0001 | t = −1.63, P = 0.12; needs n ≥ 153 |
| Tone MMN latency vs KBIT-2 verbal raw | Results; Fig. 5 | r = −0.35, P < 0.0001 | t = −1.87, P = 0.074; needs n ≥ 119 |
| Tone MMN latency vs KBIT-2 non-verbal IQ | Results; Fig. 5 | r = −0.27, P < 0.001 | t = −1.40, P = 0.17; needs n ≥ 146 |
| Tone MMN latency vs PAL first trial memory | Results; Fig. 5 | r = −0.35, P < 0.0001 | t = −1.87, P = 0.074; needs n ≥ 119 |
| Tone MMN latency vs SRT total | Results; Fig. 5 | r = −0.30, P < 0.01 | t = −1.57, P = 0.13; needs n ≥ 73 |
| Tone MMN latency vs SRT SD | Results; Fig. 5 | r = 0.30, P < 0.001 | t = 1.57, P = 0.13; needs n ≥ 118 |
| Effect of rounding r by ±0.005 | derived | two decimals | P ranges stay within 0.054–0.199; none crosses 0.05 |
| Critical |r|, two-sided, n = 27 | derived | — | 0.381 (P < 0.05), 0.487 (P < 0.01), 0.597 (P < 0.001), 0.679 (P < 0.0001); permutation null agrees (tail shares 0.0107, 0.0013, 0.00016) |
| Correlations meeting alpha = 0.01 at n = 27 | derived | 8 of 8 ("significantly") | 0 of 8; 0 of 8 also at P < 0.05 |
| Correlations meeting their reported threshold at n = 47 or n = 54 | derived | 8 of 8 | 0 of 8 at either n (smallest: r = −0.37 gives P = 0.010 at n = 47, 0.0059 at n = 54) |
| Fig. 5 cell, MMN latency vs K-BIT 2 non-verbal raw | Fig. 5 (caption: P < 0.01) | −0.14, shown | P = 0.49 at n = 27; needs n ≥ 338 |
| Fig. 5 cell, K-BIT 2 verbal IQ vs SRT mean | Fig. 5 (caption: P < 0.01) | −0.11, shown | P = 0.58 at n = 27 (0.46 at n = 47); needs n ≥ 548 |
| Ordering of reported thresholds | Results | |r| = 0.30 at P < 0.01 and at P < 0.001; |r| = 0.27 at P < 0.001 | With a common n, equal |r| gives equal P and a smaller |r| gives a larger P; at n = 27, P(0.27) = 0.17 > P(0.30) = 0.13 |
| Repeated-measures df | Results, Linear mixed-effect models | F(2, 52) | (3 − 1) × (27 − 1) = 52, consistent with 27 participants contributing to the analyses |
Under the paper's own n, the eight correlations have P values between 0.057 and 0.19. The reported thresholds are smaller by factors of 10 to over 1,000. The exact permutation null gives the same answer as the t approximation to two decimals. The ordering check does not depend on n at all: the same |r| = 0.30 is reported once at P < 0.01 and once at P < 0.001, and |r| = 0.27 is given a smaller P than |r| = 0.30. If all correlations used the same participants, this ordering cannot occur.
Figure
Figure 1. For each correlation the paper reports as significant, the orange bar ending in an arrowhead covers the range the paper claims ("P below this threshold"), and the blue dot is the two-sided P value recomputed from the printed coefficient at the stated n = 27 (t approximation; the permutation null gives the same values to two decimals). The axis is logarithmic. The dashed line marks the paper's alpha, P = 0.01, and the dotted line marks P = 0.05. The two lowest rows are cells displayed in Fig. 5, whose caption states P < 0.01. Every blue dot lies to the right of P = 0.05.
Figure 2. For each coefficient, the orange diamond marks the stated analysis sample (n = 27) and the blue dot the smallest sample at which that coefficient would meet the threshold printed next to it (two-sided t approximation). The dashed line marks the 47 recruited participants. The eight Results correlations would need 73 to 153 participants, between 2.7 and 5.7 times the analysed sample; the two Fig. 5 cells would need 338 and 548. Even all 47 recruited participants fall well short of every requirement.
Why it matters
The title ("The mismatch negativity as an index of cognitive abilities") and the Abstract's conclusion ("the MMN may represent a valuable index of cognitive abilities in DS") rest on these correlations. Apart from a mixed model of ERP amplitude by intellectual-disability level, which the paper reports as non-significant (F(4, 52) = 0.19, P = 0.94), no other analysis links the MMN to cognitive scores. In the Discussion, the correlations support three lines of argument: that MMN amplitude reflects memory (via CANTAB PAL), that MMN latency reflects speech-signal processing (via KBIT-2 verbal scores), and that latency reflects frontal white-matter integrity (via SRT standard deviation). The Conclusions sentence on "a more global detrimental effect on cognition" also rests on them.
At the paper's stated n and alpha, none of the eight correlations is statistically significant, and none would be significant at the conventional 0.05 level either. The coefficients themselves (|r| = 0.26–0.37) may be correct, but with 27 participants they are compatible with no association. The paper's limitation that the correlations were "conducted in an exploratory manner without correction for multiple comparisons" understates the problem: the issue is not a lack of multiplicity correction but that each uncorrected P value is above both thresholds.
Findings that do not depend on the correlations are not affected. These include the mixed-model result that the tone-deviant vs frequent contrast is significant at ages 26 and 31 but not 41, and the observation that MMN latencies are longer than canonical values.
Explanations considered
- Exact rather than approximate Spearman test. This does not account for the numbers. The Monte Carlo exact null for n = 27 gives P values within 0.003 of the t approximation (for example 0.058 vs 0.057 for r = −0.37), and its tail shares at the critical values match the nominal levels.
- Rounding of the printed coefficients. This does not account for the numbers. Moving each |r| by ±0.005 keeps every P between 0.054 and 0.199.
- One-sided tests. This does not account for the numbers. Halving the P values gives 0.029 to 0.095, still above 0.01 in every case, and the Methods do not mention one-sided tests.
- A larger n than 27 (all 47 recruited, or both electrodes pooled as 54 observations). This does not account for the numbers. The stated n of 27 is confirmed by Supplementary Table 1 and fits the mixed-model df of 52. Even at n = 47 or n = 54, none of the eight coefficients meets its reported threshold. The strongest, r = −0.37, gives P = 0.010 and 0.0059, against a reported P < 0.0001. Treating two electrodes from the same person as independent observations would not be valid in any case. The two Fig. 5 cells (−0.14 and −0.11) would need 338 and 548 observations.
- Pairwise missing data. This makes the discrepancy larger, not smaller. Any missing cognitive score lowers n below 27 and raises the P value.
- The printed values are a different coefficient (Kendall's tau) or a squared coefficient (r²). Neither accounts for the numbers. Read as Kendall's tau at n = 27, the printed values give P = 0.0068 to 0.057, and 0 of 8 meet their reported thresholds. Read as r² (so |r| = 0.51–0.61), they give P = 0.00076 to 0.0066, and 2 of 8 meet their thresholds. The Methods and the Fig. 5 caption both say Spearman's correlation.
- The P labels are mis-assigned between rows. This does not account for the numbers. Reassigning labels cannot help, because even the largest coefficient fails the paper's alpha. Under any single n, the reported thresholds are also not ordered by |r|: equal coefficients (0.30) carry different thresholds, and a smaller coefficient (0.27) carries a smaller threshold than a larger one (0.30).
- The Fig. 5 cells are filtered by some criterion other than P < 0.01. This is possible for the figure but does not change the text. The figure displays coefficients as small as −0.11, which no P < 0.01 criterion at n = 27 or n = 47 would pass, so the displayed cells were not selected by the criterion the caption states. The Results paragraph gives its own P values independently of the figure.
- Conversion or typesetting error in the archived text. This is excluded. The version of record, the PMC text and the Europe PMC full-text XML carry the same coefficients and P values, and the Fig. 5 image shows the same coefficients (see Status).
- The values are explained in a supplement. This is excluded. The only supplementary file is the participant table, which contains no MMN or cognitive scores and no alternative analysis.
The archived material does not show how the P values were produced, and the participant-level data needed to recompute the correlations are not published.
Status
- Version of record. The version of record at Oxford Academic (doi.org/10.1093/cercor/bhad233, which resolves to academic.oup.com/cercor/article/33/16/9639/7217126) was compared with the PMC archive. Its Results paragraph, sample description, alpha statement and Fig. 5 caption match the PMC text exactly, as does the Europe PMC full-text XML. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. The Fig. 5 coefficients were read from the figure image in the Europe PMC supplementary bundle (bhad233f5.jpg). - Supplements. The single supplementary file, Supplementary_Table_1_bhad233.docx (from the Europe PMC supplementaryFiles bundle), lists 47 participants with ID, study inclusion, age, sex, intellectual-disability level, Whisper hearing test, assessment gap and CAMDEX-DS decline. Of these, 27 are marked as included, with the same sex, intellectual-disability and age summary as the Results. The table contains no MMN or cognitive scores, so the correlations cannot be recomputed from it.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no notices (
src/known.py). A web search for corrections or comments on the article found none. The PubPeer API and search page refused automated access (HTTP 404 on the API query and HTTP 403 on the search page), so PubPeer could not be checked directly. - Sources. PMC article PMC10431748; Europe PMC full-text XML and supplementary-files bundle for PMC10431748; Oxford Academic version of record; Crossref; Europe PMC.
Suggested correction
- Recompute the eight Spearman correlations in the Results from the participant-level data and report, for each, the coefficient, the number of participants contributing, and the exact two-sided P value. At n = 27, the printed coefficients imply P ≈ 0.06–0.19.
- If the recomputed P values do not meet the stated alpha of 0.01, remove "significantly" from the Results paragraph. Revise the Abstract ("Reduced MMN amplitude was associated with lower memory scores, while longer MMN latencies were associated with poorer memory, verbal abilities, and attention"), the Conclusions and the corresponding Discussion passages to describe these as non-significant exploratory associations. Reconsider the title claim that the MMN is "an index of cognitive abilities".
- Redraw Fig. 5 so that the displayed cells match the stated criterion, or state the criterion actually used. Report the n for each cell.
- State in the Methods how the correlation P values were computed (software, exact or approximate test, one- or two-sided, unit of analysis) so the results can be reproduced.
Recomputation script (python recompute.py.txt)
"""Recompute the Spearman correlation P values of PMC10431748
(Cereb Cortex 2023, doi:10.1093/cercor/bhad233,
"The mismatch negativity as an index of cognitive abilities in adults with Down syndrome").
Run: python recompute.py.txt [path/to/supplementary_table_1_bhad233.docx]
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file. If the path to the paper's
Supplementary Table 1 (.docx, from the Europe PMC supplementaryFiles bundle) is given, it
also counts the included participants from that table.
Inputs are copied from the paper as archived in PMC (identical in the version of record):
Results, Sample description: "The final sample for analyses consisted of 27 individuals"
Methods, Statistical analyses: Spearman's rank correlations, alpha "set to P < 0.01"
Results, Spearman's rank correlation coefficients: the eight r / P pairs in CLAIMS below
Fig. 5 (heat map, caption "(P < 0.01)"): two further displayed cells in FIG5_CELLS below
Method: two-sided test of a Spearman coefficient with the usual t approximation,
t = r * sqrt((n - 2) / (1 - r^2)) on n - 2 df, checked against a Monte Carlo approximation
of the exact permutation null of Spearman's rho for n = 27 (no ties).
"""
import math
import os
import re
import sys
import zipfile
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
N = 27 # stated analysis sample
N_ALL = 47 # all recruited participants (Methods, Participants)
N_POOL = 54 # 27 participants x 2 fronto-midline electrodes, if treated as independent
# (label, short label, r, reported threshold as printed, threshold value, location)
CLAIMS = [
("Tone MMN amplitude vs PAL stages completed", "Amplitude vs PAL stages", -0.26, "P < 0.01", 1e-2),
("Tone MMN latency vs KBIT-2 composite IQ", "Latency vs KBIT-2 composite IQ", -0.37, "P < 0.0001", 1e-4),
("Tone MMN latency vs KBIT-2 verbal IQ", "Latency vs KBIT-2 verbal IQ", -0.31, "P < 0.0001", 1e-4),
("Tone MMN latency vs KBIT-2 verbal raw", "Latency vs KBIT-2 verbal raw", -0.35, "P < 0.0001", 1e-4),
("Tone MMN latency vs KBIT-2 non-verbal IQ", "Latency vs KBIT-2 non-verbal IQ", -0.27, "P < 0.001", 1e-3),
("Tone MMN latency vs PAL first trial memory", "Latency vs PAL first-trial memory", -0.35, "P < 0.0001", 1e-4),
("Tone MMN latency vs SRT total", "Latency vs SRT total", -0.30, "P < 0.01", 1e-2),
("Tone MMN latency vs SRT SD", "Latency vs SRT SD", 0.30, "P < 0.001", 1e-3),
]
FIG5_CELLS = [
("Fig. 5 cell: MMN latency vs KBIT-2 non-verbal raw", "Fig. 5: latency vs non-verbal raw", -0.14, "P < 0.01 (caption)", 1e-2),
("Fig. 5 cell: KBIT-2 verbal IQ vs SRT mean", "Fig. 5: verbal IQ vs SRT mean", -0.11, "P < 0.01 (caption)", 1e-2),
]
ALL = CLAIMS + FIG5_CELLS
def p_t(r, n):
t = r * math.sqrt((n - 2) / (1 - r * r))
return t, 2 * stats.t.sf(abs(t), n - 2)
def min_n(r, alpha):
for m in range(4, 100000):
if p_t(r, m)[1] < alpha:
return m
def crit_r(alpha, n):
tc = stats.t.isf(alpha / 2, n - 2)
return tc / math.sqrt(n - 2 + tc * tc)
def fmt_p(p):
return f"{p:.3f}" if p >= 0.001 else f"{p:.1e}"
# ---------------------------------------------------------------- Monte Carlo exact null, n = 27
rng = np.random.default_rng(20231024)
B = 200_000
ranks = np.arange(N)
rho_null = np.empty(B)
for i in range(0, B, 20_000):
perm = np.argsort(rng.random((20_000, N)), axis=1)
rho_null[i:i + 20_000] = 1 - 6 * ((perm - ranks) ** 2).sum(1) / (N * (N * N - 1))
R = {}
print(f"=== 1. Two-sided P for each reported coefficient at the stated n = {N} (df = {N - 2}) ===")
print(f"{'comparison':50s} {'r':>6s} {'reported':>19s} {'t':>7s} {'P(t)':>7s} {'P(perm)':>8s}"
f" {'P range, r+-0.005':>18s} {'1-sided':>8s} {'P n=47':>7s} {'P n=54':>7s} {'n needed':>8s}")
for lab, short, r, rep, a in ALL:
t, p = p_t(r, N)
pperm = (np.abs(rho_null) >= abs(r) - 1e-12).mean()
lo = p_t(abs(r) + 0.005, N)[1]
hi = p_t(abs(r) - 0.005, N)[1]
p47 = p_t(r, N_ALL)[1]
p54 = p_t(r, N_POOL)[1]
need = min_n(r, a)
R[lab] = dict(short=short, r=r, rep=rep, a=a, t=t, p=p, pperm=pperm, lo=lo, hi=hi, p47=p47, p54=p54, need=need)
print(f"{lab:50s} {r:6.2f} {rep:>19s} {t:7.2f} {p:7.3f} {pperm:8.3f} {lo:8.3f}-{hi:.3f} "
f" {p / 2:8.3f} {p47:7.3f} {p54:7.4f} {need:8d}")
print(f"\n range of P(t) over the eight Results correlations: "
f"{min(R[c[0]]['p'] for c in CLAIMS):.3f}-{max(R[c[0]]['p'] for c in CLAIMS):.3f}")
print(f" smallest one-sided P among the eight: {min(R[c[0]]['p'] for c in CLAIMS) / 2:.3f}")
print(f" n needed for the reported threshold, eight Results correlations: "
f"{min(R[c[0]]['need'] for c in CLAIMS)}-{max(R[c[0]]['need'] for c in CLAIMS)}")
print(f" correlations meeting the paper's alpha 0.01 at n = {N}: "
f"{sum(R[c[0]]['p'] < 0.01 for c in CLAIMS)} of {len(CLAIMS)}; meeting 0.05: "
f"{sum(R[c[0]]['p'] < 0.05 for c in CLAIMS)} of {len(CLAIMS)}")
print(f" meeting the reported threshold at n = {N_ALL}: {sum(R[c[0]]['p47'] < c[4] for c in CLAIMS)} of 8; "
f"at n = {N_POOL}: {sum(R[c[0]]['p54'] < c[4] for c in CLAIMS)} of 8")
print(f"\n=== 2. Critical |r| (two-sided) ===")
for n in (N, N_ALL, N_POOL):
print(f" n = {n}: " + ", ".join(f"P < {a:g}: {crit_r(a, n):.3f}" for a in (0.05, 0.01, 1e-3, 1e-4)))
for a in (0.01, 1e-3, 1e-4):
q = (np.abs(rho_null) >= crit_r(a, N)).mean()
print(f" Monte Carlo check: share of exact-null |rho| >= {crit_r(a, N):.3f} at n = {N}: {q:.5f} (nominal {a:g})")
print("\n=== 3. Internal ordering of the reported thresholds (holds for any common n) ===")
print(" |r| = 0.30 is reported as P < 0.01 (SRT total) and as P < 0.001 (SRT SD)")
print(" |r| = 0.27 (non-verbal IQ) is reported as P < 0.001, |r| = 0.30 (SRT total) as P < 0.01")
print(" |r| = 0.31 (verbal IQ) is reported as P < 0.0001, |r| = 0.30 (SRT SD) as P < 0.001")
for n in (N, N_ALL, N_POOL):
print(f" n = {n}: P(|r|=0.27) = {p_t(0.27, n)[1]:.4f} > P(|r|=0.30) = {p_t(0.30, n)[1]:.4f}")
print("\n=== 4. Alternative readings of the printed coefficients (n = 27) ===")
print(" (a) printed values are Kendall's tau (normal approximation, no ties):")
for lab, short, r, rep, a in CLAIMS:
z = 3 * abs(r) * math.sqrt(N * (N - 1)) / math.sqrt(2 * (2 * N + 5))
p = 2 * stats.norm.sf(z)
R[lab]["tau_p"] = p
print(f" {short:36s} tau = {r:5.2f}: z = {z:.2f}, P = {p:.4f} reported {rep:10s} -> {'meets' if p < a else 'fails'}")
print(f" meeting the reported threshold: {sum(R[c[0]]['tau_p'] < c[4] for c in CLAIMS)} of 8")
print(" (b) printed values are r^2 (so |r| = sqrt of the printed value):")
for lab, short, r, rep, a in CLAIMS:
p = p_t(math.sqrt(abs(r)), N)[1]
R[lab]["sq_p"] = p
print(f" {short:36s} |r| = {math.sqrt(abs(r)):.3f}: P = {p:.5f} reported {rep:10s} -> {'meets' if p < a else 'fails'}")
print(f" meeting the reported threshold: {sum(R[c[0]]['sq_p'] < c[4] for c in CLAIMS)} of 8")
print("\n=== 5. Consistency of the stated n with the mixed-model df ===")
print(f" repeated-measures error df for 3 stimuli x {N} participants: (3 - 1) x ({N} - 1) = {(3 - 1) * (N - 1)}"
f" (paper: F(2, 52))")
# ---------------------------------------------------------------- optional: supplement count
if len(sys.argv) > 1 and os.path.exists(sys.argv[1]):
x = zipfile.ZipFile(sys.argv[1]).read("word/document.xml").decode("utf8")
rows = [[''.join(re.findall(r'<w:t[^>]*>([^<]*)</w:t>', c)) for c in re.findall(r'<w:tc>.*?</w:tc>', tr, re.S)]
for tr in re.findall(r'<w:tr[ >].*?</w:tr>', x, re.S)]
inc = [c for c in rows[1:] if c[1].strip().lower().startswith("yes")]
exc = [c for c in rows[1:] if c[1].strip().lower().startswith("no")]
ages = np.array([float(c[2]) for c in inc])
print("\n=== 6. Supplementary Table 1 ===")
print(f" rows: {len(rows) - 1}; inclusion = Yes: {len(inc)}; No: {len(exc)}")
print(f" included: age mean {ages.mean():.1f}, SD {ages.std(ddof=1):.1f}, range {ages.min():.0f}-{ages.max():.0f}; "
f"females {sum(c[3].strip().upper().startswith('F') for c in inc)}; "
f"ID level " + ", ".join(f"{k} {sum(c[4].strip().lower() == k for c in inc)}" for k in ("mild", "moderate", "severe")))
# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": TXT, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 1.5,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def style(ax):
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
TICK = ["PAL stages completed", "KBIT-2 composite IQ", "KBIT-2 verbal IQ", "KBIT-2 verbal raw",
"KBIT-2 non-verbal IQ", "PAL first-trial memory", "SRT total", "SRT SD",
"latency vs non-verbal raw", "verbal IQ vs SRT mean"]
labels = [f"{t} (r = {c[2]:.2f})".replace("= -", "= \u2212") for t, c in zip(TICK, ALL)]
HEADS = [(0, "Tone MMN amplitude vs"), (1, "Tone MMN latency vs"), (8, "Fig. 5 cells (caption: P < 0.01)")]
ys, heads, y = [], [], 0.0
for i in range(len(ALL)):
for j, h in HEADS:
if j == i:
y -= 0.35 if i == 0 else 1.0
heads.append((y, h))
y -= 0.85
ys.append(y)
y -= 1.0
ys = np.array(ys)
def rowlabels(ax):
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10)
for yh, h in heads:
ax.text(-0.02, yh, h, transform=ax.get_yaxis_transform(), ha="right", va="center",
fontsize=10.5, fontweight="bold", color=TXT)
ax.set_ylim(ys[-1] - 0.7, heads[0][0] + 0.9)
# Figure 1: reported P threshold vs recomputed P at n = 27
fig = plt.figure(figsize=(7.2, 6.0))
ax = fig.add_axes([0.40, 0.15, 0.57, 0.76])
XMIN, XMAX = 3e-5, 1.0
for (lab, short, r, rep, a), y in zip(ALL, ys):
d = R[lab]
ax.plot([XMIN, a], [y + 0.16] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot([a], [y + 0.16], marker="<", ms=9, color=ORANGE, mec=ORANGE)
ax.plot([d["lo"], d["hi"]], [y - 0.16] * 2, color=BLUE, lw=2)
ax.plot([d["p"]], [y - 0.16], "o", ms=8, color=BLUE)
ax.text(d["hi"] * 1.3, y - 0.16, f"{d['p']:.2f}", va="center", ha="left", fontsize=10, color=BLUE)
top = heads[0][0] + 0.9
ax.axvline(0.01, color=SEC, lw=0.9, ls=(0, (4, 3)), zorder=0)
ax.text(0.0085, top, "P = 0.01\n(paper's alpha)", ha="right", va="bottom", fontsize=10, color=SEC)
ax.axvline(0.05, color=SEC, lw=0.9, ls=(0, (1, 2)), zorder=0)
ax.text(0.058, top, "P = 0.05", ha="left", va="bottom", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "1"])
ax.set_xlabel("Two-sided P value (log scale)")
rowlabels(ax)
style(ax)
fig.legend(handles=[
Line2D([], [], color=ORANGE, lw=2, marker="<", ms=9, label="reported: P below this threshold"),
Line2D([], [], color=BLUE, lw=0, marker="o", ms=8, label=f"recomputed from r at n = {N}")],
loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: participants needed for each reported threshold vs the stated n
fig = plt.figure(figsize=(7.2, 6.0))
ax = fig.add_axes([0.40, 0.17, 0.57, 0.74])
for (lab, short, r, rep, a), y in zip(ALL, ys):
d = R[lab]
ax.plot([N, d["need"]], [y, y], color="#cfcec9", lw=2, zorder=1)
ax.plot([N], [y], "D", ms=8, color=ORANGE, zorder=3)
ax.plot([d["need"]], [y], "o", ms=8, color=BLUE, zorder=3)
ax.text(d["need"] * 1.15, y, f"{d['need']} (for {rep.split(' (')[0]})", va="center", ha="left",
fontsize=10, color=BLUE)
ax.axvline(N_ALL, color=SEC, lw=0.9, ls=(0, (4, 3)), zorder=0)
ax.text(N_ALL, heads[0][0] + 0.9, f"all {N_ALL} recruited", ha="center", va="bottom", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(15, 3000)
ax.set_xticks([20, 27, 50, 100, 200, 500, 1000])
ax.set_xticklabels(["20", "27", "50", "100", "200", "500", "1000"])
ax.set_xlabel("Number of participants (log scale)")
rowlabels(ax)
style(ax)
fig.legend(handles=[
Line2D([], [], color=ORANGE, lw=0, marker="D", ms=8, label=f"reported analysis sample, n = {N}"),
Line2D([], [], color=BLUE, lw=0, marker="o", ms=8,
label="recomputed: smallest n at which this r\nmeets the reported threshold (two-sided)")],
loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Reported p<0.001 for lower heart-failure hospitalisation and mortality does not follow from the paper's counts
Eur Respir J (2022) · doi:10.1183/13993003.02276-2021 · PMC9535117
Summary
The abstract and Results report that heart-failure hospitalisation (1.2% vs 1.9%) and non-COVID-19 mortality (0.3% vs 0.5%) among patients with pulmonary arterial hypertension (PAH) were lower in March–May 2020 than in the same period of 2019, with p<0.001. The underlying counts are printed in the paper: 23 of 1922 vs 36 of 1967 hospitalisations, and 6 of 1922 vs 10 of 1967 deaths. With the tests the Methods name (chi-squared or Fisher's exact), these give p = 0.11 to 0.12 for hospitalisation and p = 0.34 to 0.45 for death. The same p<0.001 is printed for add-on therapy among hospitalised patients (15/23 vs 15/36), which recomputes to p = 0.08 to 0.11. These are the comparisons behind the "paradox" in the title, the abstract's results sentence and the Conclusions. None of them reaches conventional significance, let alone p<0.001.
What the paper reports
All quotations are verbatim from the PMC-archived text and match the typeset PDF.
Abstract, Results:
Less systematic activities were converted into more effective remote interfacing between clinicians and PAH patients, resulting in lower rates of hospitalisation (1.2% versus 1.9%) and related death (0.3% versus 0.5%) compared with 2019 (p<0.001).
Results, "Downsizing of systematic activities in PAH", first paragraph (the two cohorts):
Systematic activities for 1922 PAH patients between March 1 and May 1, 2020, were compared with those for 1967 PAH patients managed in the same period in 2019.
Results, "Withdrawal of in-person-directed medical management and outcomes", first paragraph:
The hospitalisation rate for heart failure in patients treated between March 1 and May 1, 2020, was 1.2%, which was significantly lower than that in the equivalent period in 2019 (1.9%; p<0.001). This corresponds to a 36.1% reduction in hospitalisation for heart failure, from 36 patients in 2019 to 23 patients in 2020. Additionally, patients during the COVID-19 outbreak were less likely to receive elective sequential add-on therapy compared with the equivalent period in 2019 (n=24, 1.2% versus n=97, 5.0%; p<0.001), while higher rates were observed for sequential add-on therapies in patients hospitalised for heart failure (n=15, 65.2% versus n=15, 41.6%; p<0.001) (figures 3 and 4).
Same section, second paragraph:
Finally, all cause mortality (not including COVID-19) was lower among PAH patients between March 1 and May 1, 2020, compared with the previous year (n=6, 0.3% versus n=10, 0.5%; p<0.001) (figure 3).
Methods, "Statistical analysis":
Two-group comparisons were done with unpaired, two-tailed t-tests for means if the data were normally distributed or with Wilcoxon's rank-sum tests if the data were not normally distributed. Chi-squared or Fisher's exact tests were used to analyse the categorical data.
Results, "Incidence of SARS-CoV-2 infection and outcomes":
while nine patients had a diagnosis of COVID-19, corresponding to an incidence of 0.46%. The latter patients were admitted to hospital and received noninvasive or invasive ventilation at any time during hospitalisation and all of them died
Discussion, second and fifth paragraphs:
The second unexpected finding to emerge from our study, and shown for the first time, is the low morbi-mortality rates in the PAH cohort despite the apparent toned-down level of care from the previous year.
However, this was paradoxically not the case for PAH, as evidenced by lower mortality rates compared with the same period of the previous year and even lower hospitalisation rates for right heart failure.
Conclusions:
Equally important, paradoxically less systematic activities encouraged more effective remote interfacing between clinicians and patients, allowing lower rates of hospitalisation for heart failure and mortality compared with the previous year, and potentially reducing the risk of patient infection.
What the paper's own numbers show
Method. For each comparison I built the 2 × 2 table from the counts printed in the Results, with the cohort sizes 1922 (2020) and 1967 (2019) as denominators, and applied the tests the Methods name. These are Fisher's exact test (two-sided) and Pearson's chi-squared test, with and without Yates' correction. I also ran a one-sided Fisher test in the direction the paper reports and a conditional Poisson test that compares the event counts directly. Every number below is printed by recompute.py.txt (Python, scipy).
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Heart-failure hospitalisation, 2020 vs 2019 | Abstract; Results, "Withdrawal…" ¶1 | 1.2% vs 1.9%, p<0.001 | 23/1922 = 1.20% vs 36/1967 = 1.83%; Fisher p = 0.116, chi-squared p = 0.106 (Yates 0.138); one-sided Fisher p = 0.068 |
| Non-COVID-19 all-cause death, 2020 vs 2019 | Abstract; Results, "Withdrawal…" ¶2 | n=6, 0.3% vs n=10, 0.5%, p<0.001 | 6/1922 = 0.31% vs 10/1967 = 0.51%; Fisher p = 0.454, chi-squared p = 0.339 (Yates 0.481); one-sided Fisher p = 0.241 |
| Add-on therapy among patients hospitalised for heart failure | Results, "Withdrawal…" ¶1 | n=15, 65.2% vs n=15, 41.6%, p<0.001 | 15/23 = 65.2% vs 15/36 = 41.7%; Fisher p = 0.110, chi-squared p = 0.078 (Yates 0.134) |
| Elective sequential add-on therapy (for contrast) | Results, "Withdrawal…" ¶1 | n=24, 1.2% vs n=97, 5.0%, p<0.001 | 1.25% vs 4.93%; Fisher p = 1.4 × 10-11; consistent with p<0.001 |
| Risk ratio 2020/2019, heart-failure hospitalisation | derived | not reported | 0.65 (95% CI 0.39 to 1.10) |
| Risk ratio 2020/2019, non-COVID-19 death | derived | not reported | 0.61 (95% CI 0.22 to 1.69) |
| 2019 count needed for Fisher p<0.001 against the 2020 count | derived | — | ≥ 25 deaths (reported 10) against 6; ≥ 53 hospitalisations (reported 36) against 23 |
| Deaths including the 9 COVID-19 deaths | derived (Results counts) | not reported | 15/1922 = 0.78% vs 10/1967 = 0.51%; risk ratio 1.54 (0.69 to 3.41); Fisher p = 0.321 |
| Reduction in heart-failure hospitalisation | Results, "Withdrawal…" ¶1 | 36.1% | (36 − 23)/36 = 36.1%; consistent |
| 2019 heart-failure hospitalisation rate | Abstract; Results | 1.9% | 36/1967 = 1.83%, which rounds to 1.8%; 36/1922 = 1.87% would round to 1.9% |
The descriptive figures are reproduced: the counts, the 36.1% reduction and the 2020 rates all agree with the text. What does not reproduce is the inference. Under every version of the stated tests, the hospitalisation and mortality differences have p-values between about 0.07 (one-sided) and 0.48. For deaths, a p<0.001 against 6 deaths in 2020 would need at least 25 deaths in 2019 rather than 10. For hospitalisation, it would need at least 53 hospitalisations rather than 36. The elective add-on comparison, which has larger counts and a fourfold difference, does give p<0.001. This shows the problem is confined to the comparisons with few events.
Figure
Figure 1. Two-sided p-values for the four 2020-vs-2019 comparisons that the Results report as p<0.001, on a log scale. Orange arrows show the reported value; p<0.001 is an upper bound, so each arrow starts at 0.001 and points towards smaller values. Blue markers show the p-values recomputed from the paper's printed counts with Fisher's exact test (filled circles) and Pearson's chi-squared test (open diamonds). Dashed reference lines mark 0.001 and 0.05. For heart-failure hospitalisation, non-COVID-19 death and add-on therapy among hospitalised patients, the recomputed p-values (0.08 to 0.45) lie above 0.05. The bottom row, elective add-on therapy (24/1922 vs 97/1967), is included for contrast: there the reported p<0.001 is consistent with the counts.
Figure 2. Risk ratios (2020 / 2019) on a log scale, with 95% confidence intervals (Katz log method) recomputed from the printed counts (blue). Orange squares are the ratios of the paper's printed percentages (1.2/1.9 and 0.3/0.5), which agree with the recomputed point estimates. Both intervals include 1 (thin vertical line, no difference). The bottom row counts the nine COVID-19 deaths reported in the Results together with the six non-COVID deaths. The paper does not report this comparison. Counted this way, deaths were more frequent in 2020 than in 2019 (15/1922 vs 10/1967), though not significantly so.
Why it matters
- The title's "paradox" rests on these comparisons. The paper presents lower hospitalisation and mortality despite reduced in-person care as "the second unexpected finding" (Discussion) and as being "paradoxically not the case for PAH" that reduced care worsened outcomes. Its only quantitative support is the pair of comparisons reported at p<0.001. With the paper's own counts, both are compatible with no change: the 95% intervals run from 0.39 to 1.10 for hospitalisation and from 0.22 to 1.69 for death.
- A causal and policy claim builds on them. The abstract and Conclusions attribute the lower rates to "more effective remote interfacing", and the Conclusions recommend that PAH patients be "prioritised within the Recovery Plan policy document for remote monitoring by dedicated tools". The case for remote monitoring may stand on other grounds. The statistical evidence of an outcome benefit cited for it, however, is not present in the data.
- The mortality comparison excludes COVID-19 deaths. This exclusion is disclosed. Counting all deaths among the 2020 cohort reverses the direction of the point estimate (0.78% vs 0.51%). A reader who took "related death (0.3% versus 0.5%) ... (p<0.001)" from the abstract could come away with a stronger and more favourable impression than the counts support.
- The finding has been repeated. An editorial commentary in the same journal (doi:10.1183/13993003.00796-2022) restates the 36.1% reduction in hospitalisations and the lower proportion of deaths without referring to their uncertainty.
- What is not affected. The large reductions in visits, tests, right heart catheterisations, new diagnoses and elective add-on therapy are not in question here. Those comparisons involve large counts, and where I checked one (elective add-on therapy) the reported p<0.001 is consistent.
Explanations considered
- A different denominator for 2020. Not able to explain it. Figure 3 describes the outcomes as those of "non-infected" patients. Using 1902 (excluding the 20 SARS-CoV-2-positive patients) or 1913 (excluding the 9 COVID-19 patients) as the 2020 denominator gives Fisher p = 0.455 for deaths and 0.117 to 0.118 for hospitalisation. Using 1922 for both years gives 0.454 and 0.115.
- A one-sided test. Not able to explain it. The Methods state two-tailed tests. Even one-sided in the reported direction, Fisher p is 0.241 (deaths) and 0.068 (hospitalisation).
- A test on event counts or rates rather than proportions. Not able to explain it. A conditional Poisson test of 6 vs 10 events gives p = 0.455, and of 23 vs 36 gives p = 0.119. A goodness-of-fit test on the counts alone gives 0.454 and 0.117.
- Treating the 2019 rate as a fixed reference. Not able to explain it. This is not one of the tests the paper names. A one-sample binomial test of the 2020 count against the 2019 rate gives p = 0.331 for deaths and p = 0.040 for hospitalisation, neither below 0.001.
- A paired (McNemar) analysis of overlapping cohorts. Not the stated analysis: the Methods describe unpaired comparisons, and the cohorts differ in size. For deaths, pairing cannot help, because a patient who died in 2019 cannot die in 2020. The discordant counts are therefore 6 and 10, and exact McNemar p = 0.454. For hospitalisation, McNemar p would fall below 0.001 only if at least 22 of the 23 patients hospitalised in 2020 had also been hospitalised in the same weeks of 2019. The paper reports nothing to suggest such an overlap.
- The p-value refers to a different comparison in the same sentence. Not supported. In the Results, each p<0.001 is attached to its own comparison in parentheses ("(1.9%; p<0.001)", "(n=6, 0.3% versus n=10, 0.5%; p<0.001)"). No grouping of these counts into a joint test is described, and no p-value for such a test is reported.
- A text-conversion artifact. Ruled out. The typeset ERJ PDF (11 pages) carries the same "p<0.001" in the abstract and at each Results location, as does the publisher's one-page shareable PDF. The counts in the text agree with the bar heights in Figure 3.
- A correction already issued. None found (see Status).
- Side observations, not part of the main finding. The 2019 hospitalisation rate printed as 1.9% matches 36/1922 (1.87%) rather than 36/1967 (1.83%, which rounds to 1.8%). "n=15, 41.6%" is 15/36 = 41.67%, which rounds to 41.7%. The COVID-19 incidence of 0.46% is 9/1922 = 0.468%, which rounds to 0.47%. "The majority of the in-person visits were registered for patients in WHO functional classes III–IV (n=158, 66.7%)": 158/250 is 63.2%, while 66.7% corresponds to 158/237. None of these changes a conclusion.
Status
- Version of record. The publisher's page (publications.ersnet.org, reached through doi.org) returned HTTP 403 to automated requests. I checked the typeset ERJ PDF instead (
2102276.full.pdf, 11 pages, the copy deposited in a university repository, arts.units.it). Its abstract, Results and Methods carry the passages quoted above word for word, including each "p<0.001". The publisher's one-page shareable abstract PDF, the Europe PMC full-text XML and the PMC HTML give the same text. Every quotation was checked verbatim against the PMC-archived text withsrc/show.py. - Supplements. The Europe PMC supplementary bundle holds an appendix listing the network members and their affiliations (4 pages), the shareable abstract (which repeats p<0.001) and the figure images. None contains outcome data or statistical output that could account for the reported p-values.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations and no PMC or Europe PMC notices. The Crossref record has empty update-to, updated-by and relation fields. The PubMed record (PMID 35301247) lists only a "Comment in" link to the editorial commentary (PMID 36202406), which does not discuss the p-values. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC9535117; typeset ERJ PDF; Europe PMC full text and supplementary files; editorial commentary PMC9924355 (doi:10.1183/13993003.00796-2022); Crossref; PubMed; PubPeer; https://doi.org/10.1183/13993003.02276-2021.
Suggested correction
In the abstract, replace
resulting in lower rates of hospitalisation (1.2% versus 1.9%) and related death (0.3% versus 0.5%) compared with 2019 (p<0.001).
with wording that reports the comparisons as not statistically significant, for example
with numerically lower rates of hospitalisation for heart failure (23/1922, 1.2% versus 36/1967, 1.8%; p=0.12) and of non-COVID-19 death (6/1922, 0.3% versus 10/1967, 0.5%; p=0.45) compared with 2019.
In the Results, replace "(1.9%; p<0.001)" with "(1.8%; p=0.12)" and drop "significantly". Replace "(n=6, 0.3% versus n=10, 0.5%; p<0.001)" with "(n=6, 0.3% versus n=10, 0.5%; p=0.45)" and "(n=15, 65.2% versus n=15, 41.6%; p<0.001)" with "(n=15, 65.2% versus n=15, 41.7%; p=0.11)" (Fisher's exact test, two-sided). The Discussion and Conclusions statements that hospitalisation and mortality were lower, and the attribution of this to remote interfacing, would need to be reworded to say that no significant difference was observed. The authors may also wish to report all-cause mortality including the nine COVID-19 deaths (15/1922, 0.8% versus 10/1967, 0.5%) next to the non-COVID figure, and to check the 66.7% given for the share of in-person visits in WHO functional classes III–IV (158/250 = 63.2%).
Recomputation script (python recompute.py.txt)
"""Recompute the 2020-vs-2019 outcome comparisons of PMC9535117
(Eur Respir J 2022, doi:10.1183/13993003.02276-2021,
"The paradox of pulmonary arterial hypertension in Italy in the COVID-19 era:
is risk of disease progression around the corner?").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (Results and Abstract):
Cohorts: 1922 PAH patients in 2020, 1967 in 2019.
Heart-failure hospitalisation: 23 (2020) vs 36 (2019); printed 1.2% vs 1.9%, p<0.001.
All-cause mortality not including COVID-19: n=6, 0.3% vs n=10, 0.5%; p<0.001.
Elective sequential add-on therapy: n=24, 1.2% vs n=97, 5.0%; p<0.001.
Add-on therapy in patients hospitalised for heart failure: n=15, 65.2% vs n=15, 41.6%; p<0.001.
SARS-CoV-2 infection: 20 patients (1.0%); COVID-19: 9 patients (0.46%), all of whom died.
Methods: "Chi-squared or Fisher's exact tests were used to analyse the categorical data."
"""
import os
import numpy as np
from scipy.stats import fisher_exact, chi2_contingency, binomtest, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
import matplotlib.ticker # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
N20, N19 = 1922, 1967
COMPARISONS = [
# label, a (2020), n1, b (2019), n2, printed % 2020, printed % 2019, printed p
("Heart-failure hospitalisation", 23, N20, 36, N19, 1.2, 1.9, "<0.001"),
("Death, not incl. COVID-19", 6, N20, 10, N19, 0.3, 0.5, "<0.001"),
("Add-on therapy among\nHF-hospitalised patients", 15, 23, 15, 36, 65.2, 41.6, "<0.001"),
("Elective add-on therapy\n(for contrast: consistent)", 24, N20, 97, N19, 1.2, 5.0, "<0.001"),
]
COVID_CASES, COVID_DEATHS, SARS_COV_2 = 9, 9, 20
def tests(a, n1, b, n2):
t = np.array([[a, n1 - a], [b, n2 - b]])
return dict(
fisher=fisher_exact(t)[1],
fisher_less=fisher_exact(t, alternative="less")[1],
fisher_greater=fisher_exact(t, alternative="greater")[1],
chi2=chi2_contingency(t, correction=False)[1],
yates=chi2_contingency(t, correction=True)[1],
poisson=binomtest(a, a + b, n1 / (n1 + n2)).pvalue,
)
def rr_ci(a, n1, b, n2):
"""Risk ratio 2020/2019 with a 95% Katz log interval."""
rr = (a / n1) / (b / n2)
se = np.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n2)
z = norm.ppf(0.975)
return rr, rr * np.exp(-z * se), rr * np.exp(z * se)
# ------------------------------------------------------------ 1. the four reported tests
print("=== 1. Reported comparisons, recomputed with the tests named in Methods ===")
R = {}
for lab, a, n1, b, n2, p20, p19, prep in COMPARISONS:
r = tests(a, n1, b, n2)
R[lab] = r
name = lab.replace("\n", " ")
print(f"{name}: {a}/{n1} = {100*a/n1:.2f}% (printed {p20}%) vs {b}/{n2} = {100*b/n2:.2f}% "
f"(printed {p19}%); printed p{prep}")
print(f" Fisher 2-sided p = {r['fisher']:.3g}; Pearson chi-squared p = {r['chi2']:.3g}; "
f"Yates p = {r['yates']:.3g}; Fisher 1-sided (2020 lower) p = {r['fisher_less']:.3g}; "
f"Fisher 1-sided (2020 higher) p = {r['fisher_greater']:.3g}; "
f"conditional Poisson (count) p = {r['poisson']:.3g}")
# ------------------------------------------------------------ 2. derived quantities
print("\n=== 2. Derived quantities ===")
print(f"HF hospitalisation reduction (36-23)/36 = {100*(36-23)/36:.1f}%")
print(f"36/1922 = {100*36/N20:.2f}% (rounds to 1.9%); 36/1967 = {100*36/N19:.2f}% (rounds to 1.8%)")
print(f"15/36 = {100*15/36:.2f}% (printed 41.6%); 15/23 = {100*15/23:.2f}%")
print(f"COVID-19 incidence 9/1922 = {100*COVID_CASES/N20:.3f}% (printed 0.46%); "
f"SARS-CoV-2 20/1922 = {100*SARS_COV_2/N20:.2f}%")
print(f"WHO FC III-IV share of in-person visits 158/250 = {100*158/250:.1f}% (printed 66.7%); "
f"158/237 = {100*158/237:.1f}%")
for lab, a, n1, b, n2, *_ in COMPARISONS[:3]:
rr, lo, hi = rr_ci(a, n1, b, n2)
print(f"Risk ratio 2020/2019, {lab.replace(chr(10), ' ')}: {rr:.2f} (95% CI {lo:.2f} to {hi:.2f})")
d_all = 6 + COVID_DEATHS
rr, lo, hi = rr_ci(d_all, N20, 10, N19)
r = tests(d_all, N20, 10, N19)
print(f"Deaths including COVID-19: {d_all}/{N20} = {100*d_all/N20:.2f}% vs 10/{N19} = {100*10/N19:.2f}%; "
f"RR {rr:.2f} (95% CI {lo:.2f} to {hi:.2f}); Fisher p = {r['fisher']:.3g}; chi-squared p = {r['chi2']:.3g}")
# smallest 2019 count that would have given Fisher p<0.001 against the 2020 count
for a, lab in [(6, "deaths"), (23, "HF hospitalisations")]:
for b in range(a, 400):
if fisher_exact([[a, N20 - a], [b, N19 - b]])[1] < 0.001:
print(f"2019 {lab} needed for Fisher p<0.001 against {a}/{N20}: {b} "
f"(reported {10 if a == 6 else 36})")
break
# ------------------------------------------------------------ 3. alternative explanations
print("\n=== 3. Alternative explanations ===")
for lab, a, b in [("Deaths", 6, 10), ("HF hospitalisation", 23, 36)]:
for den, dl in [(N20 - SARS_COV_2, "1902 (excluding 20 SARS-CoV-2-positive)"),
(N20 - COVID_CASES, "1913 (excluding 9 COVID-19)"),
(N19, "1967 (2019 size)")]:
r = tests(a, den, b, N19)
print(f"{lab}, 2020 denominator {dl}: Fisher p = {r['fisher']:.3g}, chi-squared p = {r['chi2']:.3g}")
r = tests(a, N20, b, N20)
print(f"{lab}, 1922 in both years: Fisher p = {r['fisher']:.3g}, chi-squared p = {r['chi2']:.3g}")
print(f"{lab}, one-sample binomial treating the 2019 rate as fixed: "
f"p = {binomtest(a, N20, b / N19).pvalue:.3g}")
# paired cohorts: if k patients had the event in both years, discordant pairs are (a-k, b-k)
# (the methods describe unpaired tests; this checks a paired reading anyway)
pk = {k: binomtest(a - k, a + b - 2 * k, 0.5).pvalue for k in range(0, a + 1)}
print(f"{lab}, McNemar exact if no patient had the event in both years (k=0): p = {pk[0]:.3g}")
if lab == "Deaths":
print(" (a patient who died in 2019 cannot die in 2020, so k=0 is the only possible overlap)")
else:
kmin = min(k for k, p in pk.items() if p < 0.001)
print(f" McNemar p<0.001 needs at least k={kmin} of the {a} patients hospitalised in 2020 "
f"to have been hospitalised in 2019 too (p = {pk[kmin]:.3g}); "
f"all {a} (k={a}): p = {pk[a]:.3g}")
print(f"{lab}, goodness-of-fit on counts alone ({a} vs {b}, equal expected): "
f"p = {binomtest(a, a + b, 0.5).pvalue:.3g}")
# ------------------------------------------------------------ figure style
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
# ------------------------------------------------------------ figure 1: p-values
fig, ax = plt.subplots(figsize=(7.2, 4.1))
XMIN, XMAX = 1e-12, 1.0
ys = np.arange(len(COMPARISONS))[::-1]
for y, (lab, *_rest) in zip(ys, COMPARISONS):
r = R[lab]
# reported p<0.001: an upper bound, drawn as a bar from the axis edge to 0.001
ax.annotate("", xy=(3e-12, y + 0.17), xytext=(1e-3, y + 0.17),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=12))
ax.plot([1e-3], [y + 0.17], marker="|", ms=12, mew=2.5, color=ORANGE)
# recomputed (Fisher and chi-squared on two sub-rows so neither hides the other)
ax.plot([r["fisher"]], [y - 0.08], "o", ms=9, color=BLUE)
ax.plot([r["chi2"]], [y - 0.32], "D", ms=8, mfc=BG, mec=BLUE, mew=2)
txt = f"Fisher p = {r['fisher']:.2g} chi-squared p = {r['chi2']:.2g}"
if r["fisher"] > 1e-3:
ax.text(4e-12, y - 0.2, txt, color=BLUE, va="center", fontsize=10)
else:
ax.text(r["chi2"] * 12, y - 0.2, txt, color=BLUE, va="center", fontsize=10)
for xv, t in [(0.05, "0.05"), (0.001, "0.001")]:
ax.axvline(xv, color=SEC, lw=0.8, ls=(0, (3, 3)), zorder=0)
ax.text(xv, len(COMPARISONS) - 0.45, t, color=SEC, fontsize=10, ha="center", va="bottom")
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX * 1.0)
ax.set_ylim(-0.6, len(COMPARISONS) - 0.35)
ax.set_yticks(ys)
ax.set_yticklabels([c[0] for c in COMPARISONS])
ax.set_xticks([1e-12, 1e-9, 1e-6, 1e-3, 1])
ax.set_xticklabels(["10⁻¹²", "10⁻⁹", "10⁻⁶", "10⁻³", "1"])
ax.set_xlabel("Two-sided p-value, 2020 vs 2019 (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right"]:
ax.spines[s].set_visible(False)
handles = [
Line2D([], [], color=ORANGE, lw=2, marker="<", ms=8, label="Reported in paper: p<0.001"),
Line2D([], [], color=BLUE, marker="o", ms=9, ls="", label="Recomputed: Fisher's exact"),
Line2D([], [], color=BLUE, marker="D", ms=8, mfc=BG, mew=2, ls="", label="Recomputed: Pearson chi-squared"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.45, -0.17), ncol=2,
frameon=False, fontsize=10)
save(fig, "fig1")
# ------------------------------------------------------------ figure 2: risk ratios
rows = [
("Heart-failure hospitalisation", 23, N20, 36, N19, 1.2 / 1.9),
("Death, not incl. COVID-19", 6, N20, 10, N19, 0.3 / 0.5),
("Death, incl. 9 COVID-19 deaths\n(not reported by the paper)", 15, N20, 10, N19, None),
]
fig, ax = plt.subplots(figsize=(7.2, 3.0))
ys = np.arange(len(rows))[::-1]
for y, (lab, a, n1, b, n2, rep) in zip(ys, rows):
rr, lo, hi = rr_ci(a, n1, b, n2)
ax.plot([lo, hi], [y - 0.1, y - 0.1], color=BLUE, lw=2)
ax.plot([rr], [y - 0.1], "o", ms=9, color=BLUE)
ax.text(hi * 1.08, y - 0.1, f"{rr:.2f} ({lo:.2f}–{hi:.2f})", color=BLUE, va="center", fontsize=10)
if rep is not None:
ax.plot([rep], [y + 0.2], "s", ms=8, color=ORANGE)
ax.text(rep * 1.08, y + 0.2, f"{rep:.2f}", color=ORANGE, va="center", fontsize=10)
ax.axvline(1, color=SEC, lw=0.8)
ax.text(1, len(rows) - 0.4, "no difference", color=SEC, fontsize=10, ha="center", va="bottom")
ax.set_xscale("log")
ax.set_xlim(0.15, 6)
ax.set_ylim(-0.55, len(rows) - 0.4)
ax.set_xticks([0.2, 0.5, 1, 2, 5])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xticklabels(["0.2", "0.5", "1", "2", "5"])
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.set_xlabel("Risk ratio, 2020 / 2019 (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right"]:
ax.spines[s].set_visible(False)
handles = [
Line2D([], [], color=ORANGE, marker="s", ms=8, ls="", label="Ratio of the paper's printed percentages"),
Line2D([], [], color=BLUE, marker="o", ms=9, lw=2, label="Recomputed from counts, with 95% CI"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.4, -0.25), ncol=2,
frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
SIR and SMR confidence intervals are 34–42 times too narrow for the observed counts
J Hepatol (2020) · doi:10.1016/j.jhep.2020.07.040 · PMC7395653
Summary
The paper compares 111 COVID-19 cases and 20 deaths among 13,255 Spanish liver transplant (LT) recipients with the age- and gender-matched general population. It reports a standardised incidence ratio (SIR, ×100) of 191.22 (95% CI 190.28–192.16) and a standardised mortality ratio (SMR, ×100) of 95.55 (95% CI 94.25–96.85), with intervals of the same width for the over-60 subgroup. The Methods say the intervals treat the observed count as Poisson. With 111 cases and 20 deaths that method gives 157.3–230.3 for the SIR and 58.4–147.6 for the SMR. Every other standard method computed from the paper's own strata gives a similarly wide interval. The published widths instead correspond to Poisson counts of about 153,000–159,000 and 20,750 events. Supplementary Table S2 shows where the SMR width comes from: a projected count of 20,741 deaths, obtained by applying the LT stratum case-fatality to the general population's cases. The SIR still exceeds 100 once corrected. The SMR interval, however, includes 100. The paper's repeated statement that LT recipients' mortality is lower than in the matched general population, and its suggestion that immunosuppression may be protective, therefore lack statistical support in its own data.
What the paper reports
Patients and methods, "Data sources, calculation of standardised ratios, and statistical analysis":
Expected incidence and outcomes in LT patients were obtained from the numbers observed in the general population, after balancing for age and gender. Ninety-five percent confidence intervals (95% CI) were calculated under the assumption that the number of cases observed followed a Poisson distribution.
Abstract, Results:
A total of 111 liver transplant patients were diagnosed with COVID-19 (SIR = 191.2 [95% CI 190.3–192.2]).
The mortality rate was 18%, which was lower than in the matched general population (SMR = 95.5 [95% CI 94.2–96.8]).
Results, "Standardised incidence of COVID-19 in LT patients":
After adjusting for age and gender, the number of observed cases almost doubled the expected number of cases (Table S1) thus resulting in a standardised incidence ratio of 191.22 (95% CI 190.28–192.16). In the subset of patients older than 60 years, the standardised incidence ratio was even higher (206.48; 95% CI 205.45–207.52).
Results, "Standardised mortality ratio of COVID-19 in LT patients":
After adjusting for age and gender, the number of observed deaths among LT patients was slightly lower than the expected number of deaths (Table S2), thus resulting in a standardised mortality ratio of 95.55 (95% CI 94.25–96.85).
Adjusted mortality rates in patients older than 60 were similar in LT recipients and in the general population (standardised mortality ratio = 100.29; 95% CI 98.93–101.66).
Supplementary Tables S1 and S2 (mmc1.pdf) print the following at the foot of each table:
Observed 280,501; Expected 146,690 … 191.22 (95%CI 190.28-192.16)
Observed 20,741; Expected 21,707 … 95.55 (95%CI 94.25-96.85)
The conclusions that rest on the SMR interval. From the Abstract, Conclusions:
Being chronically immunosuppressed, liver transplant patients have an increased risk of acquiring COVID-19 but their mortality rates are lower than the matched general population.
From the Lay summary:
In liver transplant patients, chronic immunosuppression increases the risk of acquiring COVID-19 but it could reduce disease severity.
From the Highlights:
Mortality rates are lower than those observed in the matched general population.
From the Discussion:
we demonstrated that standardised mortality rates were actually lower in LT patients as compared with the general population
From the Discussion, concluding paragraph:
Mortality rates were lower than those observed in the age- and gender-matched general population, thereby suggesting that chronic immunosuppression could exert a certain protective effect against the most severe forms of COVID-19.
What the paper's own numbers show
Method. All ratios are written ×100, as in the paper. I recomputed each interval in three ways.
- The stated method. I took the observed count O (111 cases, 92 cases over 60 from Table S1, and 20 deaths, all over 60). The expected count E is implied by the reported ratio, E = O / (ratio/100). I then computed the exact (Garwood) Poisson 95% interval for O and divided it by E. Byar's approximation and the log-normal approximation agree to within about 3 units.
- Direct standardisation from the strata. I entered the 12 age × gender strata of Tables S1–S2 and checked how the point estimates were built. The published ratios are reproduced by direct standardisation: LT stratum rates are applied to the general-population structure, and the resulting projected count (the tables' "Observed" 280,501 and 20,741) is divided by the general-population count. For that estimator, the variance comes from the LT stratum counts, Σ(wᵢ²·xᵢ). I give log-scale intervals, plus Fay–Feuer gamma intervals as a sensitivity check.
- Indirect standardisation from the same strata, i.e. the conventional SIR/SMR (O/E, with E taken from the general-population stratum rates), with exact Poisson intervals.
Finally, I asked which Poisson count would produce each published width: N = (1.96 × ratio / half-width)².
Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Crude LT incidence | Results | 837.41 per 10⁵ (111 of 13,255) | 837.42 per 10⁵ |
| SIR point estimate | Abstract; Results; Table S1 | 191.22 ("Observed 280,501; Expected 146,690") | Direct standardisation from the strata: projected 280,556 cases / 146,690 = 191.26 |
| SIR, 95% CI | Abstract; Results; Table S1 | 190.28–192.16 (half-width 0.49% of the estimate) | Stated method (O = 111, E = 58.05): 157.3–230.3, 39 times as wide. Direct standardisation: 142.0–257.7 (Fay–Feuer 138.5–271.9). Indirect: 177.9 (146.3–214.2) |
| SIR >60, point and 95% CI | Results | 206.48 (205.45–207.52) | Stated method (O = 92, E = 44.56): 166.5–253.2, 42 times as wide. Direct: 206.61 (160.1–266.6; Fay–Feuer 157.3–269.7). Indirect: 194.9 (157.1–239.0) |
| SMR point estimate | Abstract; Results; Table S2 | 95.55 ("Observed 20,741; Expected 21,707") | Projected deaths 20,740.75 / 21,707 = 95.55 |
| SMR, 95% CI | Abstract; Results; Table S2 | 94.25–96.85 | Stated method (O = 20, E = 20.93): 58.4–147.6, 34 times as wide, includes 100. Direct: 57.8–158.0 (Fay–Feuer 53.7–310.3). Indirect: 97.3 (59.5–150.3) |
| SMR >60, point and 95% CI | Results | 100.29 (98.93–101.66) | 20,740.75 / 20,681 = 100.29. Stated method (O = 20, E = 19.94): 61.3–154.9. Direct: 60.7–165.8. Indirect: 99.3 (60.6–153.3) |
| Poisson count implied by each published width | derived | — | SIR 158,967; SIR >60 152,887; SMR 20,753; SMR >60 20,737 (the observed counts are 111, 92, 20, 20) |
| Published SMR intervals, reproduced | Table S2 | 94.25–96.85; 98.93–101.66 | Poisson on the projected 20,741 deaths: 94.25–96.85 and 98.93–101.65 |
| Published SIR >60 interval, reproduced | Results | 205.45–207.52 | Poisson on the projected >60 count (152,770): 205.44–207.52 |
| Published overall SIR interval | Table S1 | 190.28–192.16 | Not reproduced exactly. Poisson on 280,501 gives 190.51–191.93; on the 146,690 general-population cases, 190.24–192.20 |
Three of the four published intervals are reproduced, to within rounding, by treating a projected population-scale count as if it were the observed Poisson count. The fourth (overall SIR) has the same order of width. Its exact source could not be identified, but it corresponds to roughly 159,000 events. In every case the actual information is 111 cases or 20 deaths.
Two smaller inconsistencies in the supplement, neither of which changes any conclusion:
- In Table S1 the general-population strata sum to 48,640,775, while the printed overall population is 47,026,208. The printed crude incidence of 311.93 per 10⁵ uses the latter.
- In Table S2 the general-population deaths sum to 21,717, which matches the Results text. The SMR, however, uses "Expected 21,707": 20,741/21,707 = 95.55, while 20,741/21,717 = 95.50.
Figure
Figure 1. Reported and recomputed 95% confidence intervals for the four standardised ratios (×100), on one log-scale axis. The thin vertical line marks a ratio of 100, meaning no difference from the matched general population. Orange shows the values printed in the paper; their intervals are narrower than the marker. Filled blue shows the paper's stated method: an exact Poisson interval for the observed count (111 cases, 92 cases over 60, 20 deaths), scaled by the expected count implied by the reported ratio. Open blue squares show the interval for the directly standardised ratio, computed from the 12 strata in Supplementary Tables S1–S2 with the variance of the LT stratum counts. Both recomputations of the SIR stay above 100. Both recomputations of the SMR, overall and over 60, include 100.
Figure 2. Where the published widths come from. The gray line is the relation between a Poisson count N and the relative half-width of its 95% interval, 1.96/√N. Each reported interval (orange) is placed at the count its width implies: about 20,750 events for both SMR intervals and 152,887–158,967 events for the SIR intervals. Those values match the projected counts printed or implied in Tables S1–S2 (20,741 deaths; 152,770 cases over 60). The counts actually observed (blue) are 111 and 92 cases and 20 deaths. For these, the relative half-width is about 18–20% for the SIRs and about 44% for the SMR.
Why it matters
The incidence finding survives correction. The corrected SIR is about 1.9, with a 95% CI of about 1.57–2.30 under the stated method and about 1.42–2.58 for the directly standardised estimator. That still shows a higher incidence among LT recipients. The paper overstates its precision by a factor of about 40, but the direction and significance of the result do not change.
The mortality finding does not survive. Under every method computed here, the SMR interval runs from about 54–60 to 148–310, so it is consistent with mortality anywhere from roughly 40% lower to substantially higher than in the matched general population. The over-60 SMR, which the paper itself describes as "similar", has an interval of about 61–155. In short, 20 deaths against about 21 expected cannot distinguish LT recipients from the matched population.
The statement that mortality is "lower than in the matched general population" appears in five places: the Abstract Results, the Abstract Conclusions, a Highlight, the Discussion ("we demonstrated that standardised mortality rates were actually lower") and the concluding paragraph. The concluding paragraph uses it to suggest that "chronic immunosuppression could exert a certain protective effect", and the Lay summary says that immunosuppression "could reduce disease severity". These statements rest on an interval (94.25–96.85) that excludes 100 only because its variance comes from a projected count of 20,741 rather than from the 20 deaths observed. The paper is a widely cited early report on COVID-19 in transplant recipients (Europe PMC lists 260 citing records), so the unsupported claim about mortality and immunosuppression could have reached clinical readers.
Explanations considered
- Ratios written ×100. 191.22 means 1.9122. Scaling changes the point estimate and the interval in proportion, so it leaves the relative width (0.49%) unchanged. It does not explain the narrow intervals.
- A typographical or rounding slip. All four intervals are affected. The pattern is consistent across them, and three of the four are reproduced to within 0.01 by a single calculation (Poisson on a projected count). This is a systematic method choice, not a slip.
- Text conversion error. The same values appear in the PMC text, the Europe PMC XML and the publisher's typeset supplement PDF (mmc1.pdf), so a conversion error is ruled out.
- The interval reflects only the uncertainty of the general-population reference rates. That uncertainty is small, because the general population had 146,690 cases. But the Methods say the interval is based on the observed count being Poisson. Also, the published SMR width matches the projected LT count (20,741), not the general-population deaths (21,717). An interval that ignores the variability of the 111 or 20 LT events does not describe uncertainty in the LT ratio in any case.
- Direct rather than indirect standardisation. The point estimates are directly standardised, although the Methods' wording and the labels "Observed/Expected" suggest an indirect SIR/SMR. A directly standardised ratio has a valid variance, Σ(wᵢ²·xᵢ) over the LT strata. Computed from Tables S1–S2, that variance gives intervals wider than the stated-method intervals (Figure 1). The indirect version gives similar intervals (SIR 146.3–214.2, SMR 59.5–150.3). Neither choice yields intervals of ±0.5%–1.4%.
- A different confidence level. At these counts, the published widths correspond to confidence levels of about 4–5% (3.8%–4.9%), and the paper says 95% throughout.
None of these explanations accounts for the numbers.
Status
As of 2026-10-23:
- Version of record. The publisher's site (sciencedirect.com and journal-of-hepatology.eu) returned HTTP 403, and doi.org redirects only to a linkinghub stub. I therefore checked the publisher-deposited text in PubMed Central and Europe PMC. Both carry the values quoted above. Crossref lists the article as J Hepatol 2021;74(1):148–155; it was published online in 2020.
- Supplement. The publisher's supplement mmc1.pdf (md5 05e647ab36152f45688c49925b4052af, from the Europe PMC supplementary-files archive) prints the same SIR and SMR intervals under Tables S1 and S2. It also gives the stratum data used for every recomputation here.
- Corrections. Crossref update records, PMC notices and Europe PMC notices list no correction, erratum or retraction (
src/known.py, run 2026-10-23). - Published discussion. The 260 citing records listed in Europe PMC were screened by title. One letter in J Hepatol 2021 ("Age and comorbidity are central to the risk of death from COVID-19 in liver transplant recipients", PMC7863751) and the authors' reply (doi:10.1016/j.jhep.2021.04.005, PMC8056821) were found. An earlier check of both texts in this pipeline found that they discuss age, comorbidity and mycophenolate, not the SIR or SMR intervals.
- PubPeer. pubpeer.com returned HTTP 403 (Cloudflare) on 2026-10-23. Web searches for the DOI and title together with PubPeer, erratum or correction, run earlier in this pipeline, found no thread. A further web search could not be run today.
Sources: the PMC article (PMC7395653); the Europe PMC full-text XML and supplementary files for PMC7395653; Crossref metadata for doi:10.1016/j.jhep.2020.07.040; the Europe PMC citations list for PMID 32750442.
Suggested correction
- Recompute the four intervals from the observed counts, as the Methods describe: SIR 191.22 (95% CI about 157–230); SIR >60 206.48 (about 167–253); SMR 95.55 (about 58–148); SMR >60 100.29 (about 61–155). An alternative is to report the directly standardised ratios with intervals based on the LT stratum counts (about 142–258, 160–267, 58–158 and 61–166).
- State that LT recipients' COVID-19 mortality did not differ significantly from that of the age- and gender-matched general population. Revise the Abstract Results and Conclusions, the second Highlight, the Lay summary and the Discussion to match, and remove or clearly label as speculative the inference that chronic immunosuppression may be protective.
- Describe the standardisation actually used (direct, with LT stratum rates applied to the general-population structure) and relabel the "Observed" and "Expected" entries in Tables S1–S2 as projected and reference counts.
- Reconcile the Table S1 population total (strata sum 48,640,775 against 47,026,208 printed) and the Table S2 denominator (21,707 against 21,717 deaths).
Recomputation script (python recompute.py.txt)
# Recomputation for PMC7395653 (J Hepatol 2020, doi:10.1016/j.jhep.2020.07.040)
# Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
# Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png)
# into the folder that holds this script.
#
# Inputs are copied from the paper:
# - Results text: 111 LT cases, SIR 191.22 (190.28-192.16); >60 SIR 206.48 (205.45-207.52);
# 20 deaths, SMR 95.55 (94.25-96.85); >60 SMR 100.29 (98.93-101.66).
# - Supplementary Tables S1 and S2 (mmc1.pdf): 12 age x gender strata.
# Ratios are written x100, as in the paper (191.22 means 1.9122).
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs
strata = ["0-19 M", "0-19 W", "20-39 M", "20-39 W", "40-49 M", "40-49 W",
"50-59 M", "50-59 W", "60-69 M", "60-69 W", "70+ M", "70+ W"]
# Table S1, general population (GP): population N and COVID-19 cases C
N_gp = np.array([4772317, 4481988, 5540648, 5474878, 3943490, 3869686,
3457353, 3516656, 2543236, 2738641, 3357068, 4944814])
C_gp = np.array([602, 625, 8497, 13307, 10105, 12173,
12818, 14620, 13398, 11387, 25693, 23465])
# Table S1, liver transplant (LT) population: patients n and COVID-19 cases c
n_lt = np.array([313, 285, 310, 280, 479, 333, 2373, 693, 3601, 1029, 2432, 1127])
c_lt = np.array([1, 0, 0, 1, 3, 4, 8, 2, 40, 13, 27, 12])
# Table S2: deaths among GP cases and among LT cases
D_gp = np.array([3, 5, 69, 55, 153, 84, 461, 206, 1355, 569, 10864, 7893])
D_lt = np.array([0, 0, 0, 0, 0, 0, 0, 0, 5, 3, 7, 5])
old = np.array([s.startswith(("60", "70")) for s in strata])
# Reported (point, lower, upper) as printed in Results
REP = {
"SIR overall": (191.22, 190.28, 192.16),
"SIR >60": (206.48, 205.45, 207.52),
"SMR overall": (95.55, 94.25, 96.85),
"SMR >60": (100.29, 98.93, 101.66),
}
# Abstract (rounded) versions
REP_ABS = {"SIR overall": (191.2, 190.3, 192.2), "SMR overall": (95.5, 94.2, 96.8)}
def garwood(o, a=0.05):
"""Exact Poisson CI for an observed count o."""
lo = stats.chi2.ppf(a / 2, 2 * o) / 2 if o > 0 else 0.0
hi = stats.chi2.ppf(1 - a / 2, 2 * (o + 1)) / 2
return lo, hi
def byar(o):
lo = o * (1 - 1 / (9 * o) - Z / (3 * np.sqrt(o))) ** 3
hi = (o + 1) * (1 - 1 / (9 * (o + 1)) + Z / (3 * np.sqrt(o + 1))) ** 3
return lo, hi
def fay_feuer(w, x, wmax):
"""Fay & Feuer (1997) gamma CI for a weighted sum of Poisson counts sum(w*x)."""
y, v = np.sum(w * x), np.sum(w ** 2 * x)
lo = stats.gamma.ppf(0.025, y ** 2 / v, scale=v / y)
hi = stats.gamma.ppf(0.975, (y + wmax) ** 2 / (v + wmax ** 2),
scale=(v + wmax ** 2) / (y + wmax))
return lo, hi
def log_delta(est, rel_se):
return est * np.exp(-Z * rel_se), est * np.exp(Z * rel_se)
def h(t):
print("\n" + "=" * 78 + "\n" + t + "\n" + "=" * 78)
# ---------------------------------------------------------------- 0. table checks
h("0. Totals of Supplementary Tables S1-S2")
print(f"GP cases (sum of strata) {C_gp.sum():,} (printed overall 146,690)")
print(f"GP population (sum of strata) {N_gp.sum():,} (printed overall 47,026,208)")
print(f"LT patients / LT cases {n_lt.sum():,} / {c_lt.sum()} (printed 13,255 / 111)")
print(f"GP deaths / LT deaths {D_gp.sum():,} / {D_lt.sum()} (printed 21,717 / 20)")
print(f"LT cases >60: {c_lt[old].sum()}; LT deaths >60: {D_lt[old].sum()}; "
f"GP cases >60: {C_gp[old].sum():,}; GP deaths >60: {D_gp[old].sum():,}")
print(f"Crude LT incidence 111/13,255 = {111 / 13255 * 1e5:.2f} per 10^5 (printed 837.41)")
print(f"Crude LT mortality 20/111 = {100 * 20 / 111:.2f}% (printed 18%, 18.01)")
# ---------------------------------------------------------------- 1. how the ratios were built
h("1. Reproducing the published point estimates from the strata")
proj_cases = np.sum(N_gp * c_lt / n_lt) # LT stratum rates applied to GP population
proj_cases_old = np.sum((N_gp * c_lt / n_lt)[old])
cfr_lt = np.divide(D_lt, c_lt, out=np.zeros(12), where=c_lt > 0)
proj_deaths = np.sum(C_gp * cfr_lt) # LT stratum case fatality applied to GP cases
print(f"Projected cases = sum(N_gp * c/n) = {proj_cases:,.1f} (Table S1 prints 'Observed 280,501')")
print(f" ratio to 146,690 GP cases = {100 * proj_cases / C_gp.sum():.2f} (reported 191.22)")
print(f" >60: projected {proj_cases_old:,.1f} / {C_gp[old].sum():,} = "
f"{100 * proj_cases_old / C_gp[old].sum():.2f} (reported 206.48)")
print(f"Projected deaths = sum(C_gp * d/c) = {proj_deaths:,.2f} (Table S2 prints 'Observed 20,741')")
print(f" ratio to 21,707 ('Expected' in S2) = {100 * proj_deaths / 21707:.2f} (reported 95.55)")
print(f" ratio to 21,717 (GP deaths in S2/text) = {100 * proj_deaths / 21717:.2f}")
print(f" >60: {proj_deaths:,.2f} / {D_gp[old].sum():,} = {100 * proj_deaths / D_gp[old].sum():.2f} (reported 100.29)")
print("=> the published ratios are directly standardised: LT stratum rates applied to the")
print(" GP structure. The numerator 'Observed' is a projected count, not an observed one.")
# ---------------------------------------------------------------- 2. stated method
h("2. The paper's stated method: CI from a Poisson observed count")
O = {"SIR overall": 111, "SIR >60": int(c_lt[old].sum()),
"SMR overall": 20, "SMR >60": int(D_lt[old].sum())}
stated = {}
for k, (r, lo, hi) in REP.items():
o = O[k]
E = o / (r / 100)
gl, gh = garwood(o)
bl, bh = byar(o)
stated[k] = (100 * gl / E, 100 * gh / E)
hw = (hi - lo) / 2
exact_hw = (stated[k][1] - stated[k][0]) / 2
print(f"{k}: O = {o}, E implied by reported ratio = {E:.2f}")
print(f" reported {r} ({lo}-{hi}); half-width {hw:.2f} = {100 * hw / r:.2f}% of estimate")
print(f" exact Poisson O CI {gl:.2f}-{gh:.2f} -> ratio CI {stated[k][0]:.1f}-{stated[k][1]:.1f}"
f" (Byar {100 * bl / E:.1f}-{100 * bh / E:.1f}; log-normal "
f"{r * np.exp(-Z / np.sqrt(o)):.1f}-{r * np.exp(Z / np.sqrt(o)):.1f})")
print(f" exact CI is {(stated[k][1] - stated[k][0]) / (hi - lo):.0f} times as wide as the reported CI")
for k, (r, lo, hi) in REP_ABS.items():
print(f"Abstract {k}: {r} ({lo}-{hi})")
# ---------------------------------------------------------------- 3. strata-based alternatives
h("3. Alternatives computed from the strata (both standardisation directions)")
# 3a indirect standardisation (conventional SIR/SMR)
E_ind = np.sum(n_lt * C_gp / N_gp)
E_ind_old = np.sum((n_lt * C_gp / N_gp)[old])
cfr_gp = D_gp / C_gp
Ed_ind = np.sum(c_lt * cfr_gp)
Ed_ind_old = np.sum((c_lt * cfr_gp)[old])
ind = {}
for k, o, E in [("SIR overall", 111, E_ind), ("SIR >60", O["SIR >60"], E_ind_old),
("SMR overall", 20, Ed_ind), ("SMR >60", 20, Ed_ind_old)]:
gl, gh = garwood(o)
ind[k] = (100 * o / E, 100 * gl / E, 100 * gh / E)
print(f"Indirect {k}: O = {o}, E = {E:.2f}, ratio {ind[k][0]:.1f} (exact 95% CI {ind[k][1]:.1f}-{ind[k][2]:.1f})")
# 3b direct standardisation with the variance of the LT stratum counts
dirci = {}
specs = [
("SIR overall", N_gp / n_lt / C_gp.sum(), c_lt, np.ones(12, bool)),
("SIR >60", N_gp / n_lt / C_gp[old].sum(), c_lt, old),
("SMR overall", np.where(c_lt > 0, C_gp / np.maximum(c_lt, 1), 0) / 21707, D_lt, c_lt > 0),
("SMR >60", np.where(c_lt > 0, C_gp / np.maximum(c_lt, 1), 0) / D_gp[old].sum(), D_lt, (c_lt > 0) & old),
]
for k, w, x, use in specs:
w = np.where(use, w, 0.0)
est = np.sum(w * x)
rel = np.sqrt(np.sum(w ** 2 * x)) / est
dl, dh = log_delta(100 * est, rel)
fl, fh = fay_feuer(w, x, w[use].max())
dirci[k] = (100 * est, dl, dh)
print(f"Direct {k}: ratio {100 * est:.2f}, relative SE {rel:.3f}; log-scale 95% CI {dl:.1f}-{dh:.1f};"
f" Fay-Feuer gamma 95% CI {100 * fl:.1f}-{100 * fh:.1f}")
# ---------------------------------------------------------------- 4. where the width comes from
h("4. Count implied by each reported interval width (Poisson, N = (z*ratio/half-width)^2)")
implied = {}
for k, (r, lo, hi) in REP.items():
hw = (hi - lo) / 2
implied[k] = (Z * r / hw) ** 2
print(f"{k}: implied N = {implied[k]:,.0f} (observed count {O[k]})")
zeq = hw / (r / np.sqrt(O[k])) # z that the observed-count SE would need
print(f" relative half-width for the observed count: {100 * Z / np.sqrt(O[k]):.1f}%;"
f" the reported width equals z = {zeq:.3f}, i.e. a {100 * (2 * stats.norm.cdf(zeq) - 1):.1f}% confidence level")
for lab, cnt, r in [("SIR overall, Poisson on 280,501 (S1 'Observed')", 280501, 191.22),
("SIR overall, Poisson on 146,690 (GP cases)", 146690, 191.22),
("SIR >60, Poisson on projected >60 count", proj_cases_old, 206.48),
("SMR overall, Poisson on 20,741 (S2 'Observed')", 20741, 95.55),
("SMR >60, Poisson on 20,741", 20741, 100.29)]:
se = r / np.sqrt(cnt)
print(f"{lab}: normal {r - Z * se:.2f}-{r + Z * se:.2f}")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
# Figure 1: reported vs recomputed intervals on one log axis
groups = [("SIR, all LT patients", "SIR overall", "111 cases"),
("SIR, age >60", "SIR >60", "92 cases"),
("SMR, all LT patients", "SMR overall", "20 deaths"),
("SMR, age >60", "SMR >60", "20 deaths")]
fig, ax = plt.subplots(figsize=(7.2, 6.0), facecolor=BG)
style(ax)
y, yt, ylab = 0, [], []
for title, k, cnt in groups:
r, lo, hi = REP[k]
rows = [("Reported", r, lo, hi, ORANGE, "o", True),
("Exact Poisson, " + cnt, r, *stated[k], BLUE, "o", True),
("Strata, direct std.", *dirci[k], BLUE, "s", False)]
ax.text(30.5, y - 0.62, title, fontsize=11, fontweight="bold", color=INK, va="center")
for lab, p, a, b, col, mk, filled in rows:
ax.plot([a, b], [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot([a, a], [y - 0.12, y + 0.12], color=col, lw=2)
ax.plot([b, b], [y - 0.12, y + 0.12], color=col, lw=2)
ax.plot(p, y, marker=mk, ms=8, color=col, mfc=col if filled else BG, mew=2, zorder=3)
fmt = "{:.2f}" if lab == "Reported" else "{:.1f}"
ax.text(b * 1.04, y, f"{fmt.format(p)} ({fmt.format(a)}–{fmt.format(b)})",
va="center", fontsize=10, color=INK2)
yt.append(y)
ylab.append(lab)
y += 0.55
y += 0.75
ax.axvline(100, color=INK2, lw=0.9, zorder=1)
ax.text(100, -1.05, "ratio = 100\n(no difference)", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(30, 720)
ax.set_xticks([40, 60, 100, 150, 200, 300])
ax.set_xticklabels(["40", "60", "100", "150", "200", "300"])
ax.minorticks_off()
ax.set_yticks(yt)
ax.set_yticklabels(ylab, fontsize=10)
for t, lab in zip(ax.get_yticklabels(), ylab):
t.set_color(INK)
ax.set_ylim(y - 0.5, -1.25)
ax.set_xlabel("Standardised ratio ×100 with 95% CI (log scale)")
handles = [plt.Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper"),
plt.Line2D([], [], color=BLUE, marker="o", ms=8, lw=2,
label="Recomputed: stated method, exact Poisson on the observed count"),
plt.Line2D([], [], color=BLUE, marker="s", ms=8, mfc=BG, mew=2, lw=2,
label="Recomputed: Tables S1–S2 strata, direct standardisation variance")]
ax.legend(handles=handles, loc="upper left", bbox_to_anchor=(-0.42, -0.11), frameon=False,
fontsize=10, ncol=1)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 2: relative half-width vs the Poisson count it corresponds to
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
style(ax)
ax.grid(axis="y", which="major", color=GRID, lw=0.8)
xs = np.logspace(1, 6, 200)
ax.plot(xs, 100 * Z / np.sqrt(xs), color=INK2, lw=1.2, zorder=1)
ax.text(1500, 100 * Z / np.sqrt(1500) * 1.5, "Poisson: half-width = 1.96/√N", color=INK2, fontsize=10, ha="left")
pts_rep = [("SIR overall", "SIR, all"), ("SIR >60", "SIR, >60"), ("SMR overall", "SMR, all and >60")]
for k, lab in pts_rep:
r, lo, hi = REP[k]
ax.plot(implied[k], 100 * (hi - lo) / 2 / r, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(implied["SMR >60"], 100 * (REP["SMR >60"][2] - REP["SMR >60"][1]) / 2 / REP["SMR >60"][0],
"o", ms=9, color=ORANGE, zorder=3)
ax.annotate(f"Reported SIR intervals: ≈ {implied['SIR >60']:,.0f}–{implied['SIR overall']:,.0f} events\n"
f"(projected >60 count {proj_cases_old:,.0f};\n Table S1 'Observed' 280,501)",
(implied["SIR overall"], 100 * 0.94 / 191.22), xytext=(250, 0.115), fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.annotate(f"Reported SMR intervals: ≈ {implied['SMR >60']:,.0f}–{implied['SMR overall']:,.0f} events\n"
f"(Table S2 'Observed' 20,741)",
(implied["SMR overall"], 100 * 1.30 / 95.55), xytext=(250, 0.55), fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
for o in (111, 92, 20):
ax.plot(o, 100 * Z / np.sqrt(o), "o", ms=9, color=BLUE, zorder=3)
ax.text(150, 100 * Z / np.sqrt(111) * 1.05, "111 cases (all) and 92 (>60)", color=INK, fontsize=10)
ax.text(27, 100 * Z / np.sqrt(20) * 1.05, "20 deaths", color=INK, fontsize=10)
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_xlim(10, 1e6)
ax.set_ylim(0.1, 100)
ax.set_yticks([0.1, 0.2, 0.5, 1, 2, 5, 10, 20, 50])
ax.set_yticklabels(["0.1%", "0.2%", "0.5%", "1%", "2%", "5%", "10%", "20%", "50%"])
ax.set_xticks([10, 100, 1e3, 1e4, 1e5, 1e6])
ax.set_xticklabels(["10", "100", "1,000", "10,000", "100,000", "1,000,000"])
ax.minorticks_off()
ax.set_xlabel("Poisson count N that the interval corresponds to (log scale)")
ax.set_ylabel("95% CI half-width, % of estimate")
handles = [plt.Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0,
label="Reported intervals, at the count their width implies"),
plt.Line2D([], [], color=BLUE, marker="o", ms=9, lw=0,
label="Recomputed: width for the observed counts")]
ax.legend(handles=handles, loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nfigures written to", HERE)
Reported risk ratios are ratios of mean 1/2 codes, understating relative risk reductions for perpetration 9- to 28-fold
JAMA Network Open (2018) · doi:10.1001/jamanetworkopen.2018.1213 · PMC6324287
Summary
This secondary analysis of a cluster randomized trial reports that a school-based HIV/STD risk-reduction intervention reduced self-reported perpetration of forced sex. The reported "risk ratios" for perpetration are 0.978, 0.964, 0.959, 0.967 and 0.964 at 3–54 months, and the paper sums up the effect as "about a 3% reduction in risk". The paper's own counts give risk ratios of 0.39 to 0.67, which are relative reductions of 33% to 61%. Every reported value instead matches (1 + p₁)/(1 + p₀), the ratio of group means of an outcome coded 1 (no) and 2 (yes), which is the coding the Methods describe. After approximate adjustment for the arms' different sex mix, all 10 cumulative-incidence ratios in Table 3 are reproduced to three decimals. The direction of the effect and the significance tests are not affected. What is wrong is the published effect size, its confidence intervals, the sex-comparison ratios in Table 4 and the "3%" statement, and these are the numbers a reader, meta-analyst or guideline would take from the paper.
What the paper reports
Methods, "Outcomes":
Binary variables were used to assess history of ever perpetrating and experiencing forced sexual intercourse, with responses coded 1 for respondents not reporting ever experiencing the event and 2 for those reporting ever experiencing the event.
Methods, "Statistical Analysis" (the superscript is a reference number):
The efficacy of the HIV/STD intervention compared with the control intervention on the forced sex outcomes was tested using generalized estimating equation Poisson regression models, adjusting for gender and students clustered within schools.
Risk ratios42 and corresponding 95% confidence intervals are reported.
Abstract, "Results" (repeated word for word in the Results text):
Fewer intervention than control participants reported forced sex perpetration postintervention compared with the control group at 3 months (9 of 561 [2%] vs 20 of 491 [4%]; risk ratio [RR], 0.978; 95% CI, 0.959-0.997), 6 months (17 of 561 [3%] vs 35 of 491 [7%]; RR, 0.964; 95% CI, 0.941-0.988), 12 months (21 of 561 [4%] vs 42 of 491 [9%]; RR, 0.959; 95% CI, 0.934-0.985), 42 months (41 of 561 [7%] vs 56 of 491 [11%]; RR, 0.967; 95% CI, 0.937-0.998), and 54 months (52 of 561 [9%] vs 68 of 491 [14%]; RR, 0.964; 95% CI, 0.932-0.997).
Table 3 (columns: perpetrating RR (95% CI) | P | experiencing RR (95% CI) | P), first and last rows:
3 | 0.978 (0.959-0.997) | .02 | 0.983 (0.963-1.004) | .11
3-54 | 0.990 (0.982-0.999) | .02 | 0.992 (0.982-1.001) | .09
Table 4 (first five columns: follow-up | boys | girls | "Boys vs Girls, RR (95% CI)" | P), perpetration at 12 months:
12 | 58/495 (12) | 5/557 (1) | 0.904 (0.881-0.928) | <.001
Results text:
Table 4 shows reports of perpetrating and experiencing forced sex were significantly higher among boys compared with girls at each of the postintervention assessments
Among boys, the effect on perpetration of forced sex was significant at 3-month follow-up (RR, 0.952; 95% CI, 0.917-0.988), 6-month follow-up (RR, 0.934; 95% CI, 0.892-0.979), and 12-month follow-up (RR, 0.924; 95% CI, 0.879-0.972)
Discussion, "Limitations":
The magnitude of the intervention effects in this trial was modest, about a 3% reduction in risk.
What the paper's own numbers show
Method. With a log-link Poisson model, the exponentiated arm coefficient estimates the ratio of the outcome's means in the two arms. For a 0/1 outcome that ratio is the risk ratio p₁/p₀. For the 1/2 coding the Methods describe it is (1 + p₁)/(1 + p₀), which stays close to 1 whatever the risks are. For each reported ratio I computed, from the counts printed next to it (Table 2 cumulative and time-specific incidence; Table 4 counts by sex): the risk ratio with an unadjusted Wald CI; the ratio of 1/2-code means with an unadjusted CI; and, as alternative readings, the inverted "no event" ratio (1 − p₀)/(1 − p₁), 1 + risk difference, and the odds ratio. The paper's model is adjusted for sex, and the arms differ in sex mix (Table 1: 306 of 561 vs 251 of 491 girls), while boys report forced sex several times as often as girls (Table 4). I therefore also applied an approximate sex standardization: each arm's expected value under no intervention effect, using the pooled sex-specific risks from Table 4, divides out the confounding. All CIs computed here ignore clustering. They show the scale of each quantity and do not replace the trial's own clustered analysis. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Perpetration, 3 mo (9/561 vs 20/491) | Abstract; Results; Table 3 | RR 0.978 (0.959–0.997) | risk ratio 0.394 (0.18–0.86); ratio of 1/2-code means 0.976 (0.957–0.996); sex-standardized 0.978 |
| Perpetration, 6 mo (17/561 vs 35/491) | Abstract; Results; Table 3 | RR 0.964 (0.941–0.988) | risk ratio 0.425 (0.24–0.75); code-mean ratio 0.962 (0.938–0.986); sex-standardized 0.964 |
| Perpetration, 12 mo (21/561 vs 42/491) | Abstract; Results; Table 3 | RR 0.959 (0.934–0.985) | risk ratio 0.438 (0.26–0.73); code-mean ratio 0.956 (0.930–0.982); sex-standardized 0.959 |
| Perpetration, 42 mo (41/561 vs 56/491) | Abstract; Results; Table 3 | RR 0.967 (0.937–0.998) | risk ratio 0.641 (0.44–0.94); code-mean ratio 0.963 (0.933–0.995); sex-standardized 0.967 |
| Perpetration, 54 mo (52/561 vs 68/491) | Abstract; Results; Table 3 | RR 0.964 (0.932–0.997) | risk ratio 0.669 (0.48–0.94); code-mean ratio 0.960 (0.927–0.994); sex-standardized 0.964 |
| Experience, 3/6/12/42/54 mo | Table 3 | 0.983 / 0.977 / 0.966 / 0.963 / 0.972 | risk ratios 0.544 / 0.586 / 0.561 / 0.655 / 0.771; code-mean ratios 0.981 / 0.974 / 0.962 / 0.959 / 0.967; sex-standardized 0.983 / 0.977 / 0.966 / 0.963 / 0.972 |
| Time-specific incidence pooled over visits, perpetration (52/2717 vs 68/2302) | Table 3, row 3-54 | 0.990 | risk ratio 0.648; code-mean ratio 0.990 |
| Time-specific incidence pooled over visits, experience (72/2680 vs 82/2287) | Table 3, row 3-54 | 0.992 | risk ratio 0.749; code-mean ratio 0.991 |
| Width of reported CIs | Table 3 | SE(log) 0.0099–0.0186 | matches the unadjusted SE of the code-mean ratio (ratio 0.96–0.99 at all 10 points); the unadjusted SE of the log risk ratio is 8 to 40 times larger |
| Boys vs girls, perpetration 3 mo (26/495 vs 3/557) | Table 4 | 0.956 | boys/girls risk ratio 9.75; girls/boys code-mean ratio 0.955 |
| Boys vs girls, perpetration 54 mo (95/495 vs 25/557) | Table 4 | 0.878 | boys/girls risk ratio 4.28; girls/boys code-mean ratio 0.877 |
| Boys vs girls, experience 12 mo (72/494 vs 5/556) | Table 4 | 0.882 | boys/girls risk ratio 16.21; girls/boys code-mean ratio 0.881 |
| The 12 Table 4 "Boys vs Girls" values that have counts beside them | Table 4 | 0.875–0.994 | girls/boys code-mean ratio within 0.0015 of each reported value; boys/girls risk ratios 2.80–18.01. The two pooled 3–54 month values in this column (0.968, 0.965) have no counts beside them and are also below 1 |
| Size of effect | Discussion, Limitations | "about a 3% reduction in risk" | mean of (1 − reported perpetration RR) = 3.4%; relative reductions from the counts 61%, 57%, 56%, 36%, 33% (perpetration) and 46%, 41%, 44%, 34%, 23% (experience) |
| Significance | Table 3 | P = .02, .004, .002, .04, .03 (perpetration) | unadjusted two-proportion tests P = 0.015, 0.002, 0.001, 0.022, 0.020; the reported P values are somewhat larger, as clustering would make them |
Across the 24 reported ratios with counts beside them (Table 3 cumulative and pooled rows, Table 4 sex comparison), the crude ratio of 1/2-code means differs from the reported value by at most 0.0047 (RMS 0.0024). The small remaining gap in Table 3 is in the direction sex adjustment predicts: the intervention arm has more girls, so the crude comparison slightly favors it. The approximate sex standardization closes the gap at all 10 cumulative time points (largest difference 0.0004).
Figure
Figure 1. Intervention vs control on one log-scale axis, with the line at 1 marking no effect. Orange circles: the "risk ratios" and 95% CIs reported in Table 3 (and, for perpetration, in the Abstract and Results). The reported CIs are so narrow that they barely extend past the markers. Blue diamonds: the risk ratio p₁/p₀ computed from the cumulative-incidence counts printed beside them (Table 2 and Abstract), with an unadjusted 95% CI that ignores clustering, so the true intervals would be somewhat wider. The reported values sit at 0.96–0.98 at every time point. The counts give 0.39–0.67 for perpetration and 0.54–0.77 for experience. The direction agrees, but on the relative-reduction scale the size of the effect differs by a factor of 9 to 28 for perpetration (8 to 27 for experience).
Figure 2. For each of the 24 reported ratios that have counts beside them, the difference between a candidate quantity computed from those counts and the reported value. The orange line at 0 is the reported value. Filled blue circles: ratio of mean 1/2 codes, (1 + p₁)/(1 + p₀) (for Table 4, girls over boys). Crosses: the same ratio after approximate sex standardization (Table 3 cumulative rows only, because the model was adjusted for sex). Hollow triangles: inverted "no event" ratio. Hollow squares: 1 + risk difference. The risk ratio and the odds ratio are off this scale (in the Table 3 cumulative rows the risk ratio is 0.20–0.58 below the reported values and the odds ratio 0.23–0.59 below). The two alternatives agree with the reported values only when events are rare, and they drift away as prevalence rises. The ratio of 1/2-code means stays within 0.005 everywhere, and within 0.0004 once sex is accounted for.
Why it matters
- The headline effect size. The Abstract and Results report perpetration "risk ratios" of 0.96–0.98, and the Limitations section turns these into "about a 3% reduction in risk". Read as risk ratios, these values say the intervention barely changed risk. The counts printed in the same sentence (9 of 561 vs 20 of 491 at 3 months) show the risk was cut by more than half at 3–12 months and by about a third at 42–54 months. The paper therefore understates its own finding. A reader who takes the numbers at face value, such as a meta-analyst pooling RRs or a program planner weighing cost against effect, would be misled by a factor of 9 to 28 on the relative-reduction scale (relative reduction from the counts divided by 1 − reported RR, at the five perpetration time points; 8 to 27 at the five experience time points).
- Precision. The reported intervals (for example 0.959–0.997) are intervals for the code-mean ratio. On the risk-ratio scale, the unadjusted intervals from the same counts are 0.18–0.86 at 3 months and 0.48–0.94 at 54 months. A meta-analysis would weight these estimates by the reported precision and would greatly overweight them.
- Table 4 and the sex comparisons. The column is labeled "Boys vs Girls, RR", but every value is below 1 although after baseline boys report forced sex 3 to 18 times as often as girls. The values match the girls/boys code-mean ratio. The Results text's qualitative statement that boys report more is correct, but the tabulated ratios point the other way. The intervention × sex interaction ratios (1.05–1.08 in the cumulative rows, 1.02 in the pooled 3–54 month rows) and the boys-only effects quoted in the Results (0.92–0.96) come from the same models. Their magnitudes cannot be read as multiplicative effects on risk, and the boys-only effects understate the effect for the same reason. The arm-by-sex counts needed to recompute them are not published.
- What still holds. For a 1/2-coded variable, a test of equal means is a test of equal proportions, so the coding does not invalidate the hypothesis tests. The unadjusted two-proportion P values from the counts are of the same order as the reported ones and slightly smaller, as expected before allowing for clustering. The Key Points, which give raw percentages (9% vs 14% at 54 months), are correct. The qualitative conclusions (the intervention reduced reported perpetration, and boys report more forced sex) are supported by the counts. What needs correcting is every effect estimate and CI, and the statement about the size of the effect.
Explanations considered
- The values are sex- and cluster-adjusted GEE estimates, so they need not equal crude ratios. This explains the 0.002–0.005 difference between the crude code-mean ratio and the reported values, and approximate sex standardization removes it. It cannot explain the gap to the risk ratio. Even in the extreme case where all events fell within one sex, the within-sex perpetration risk ratio at 3 months would be 0.37–0.42 (and 0.63–0.72 at 54 months), far from 0.978. Clustering mainly changes the standard errors. A working correlation structure could shift a point estimate somewhat, but the sex-standardized code-mean ratio already reproduces the reported values to three decimals, so there is no remaining gap for clustering to explain.
- The ratio is of "no event" probabilities, or inverted. This does not fit. (1 − p₀)/(1 − p₁) matches when events are rare but differs by up to 0.016 in Table 3 and 0.042 in Table 4 (Figure 2). It also would not be a risk ratio for the event the paper describes.
- The values are 1 + risk difference, and "3%" is an absolute reduction. This does not fit the tables. It deviates by up to 0.010 in Table 3 and 0.030 in Table 4 (Figure 2), and the paper labels the values as risk ratios from Poisson models. The perpetration absolute differences are 2.5–4.8 percentage points, so "3%" as an absolute difference would also be inaccurate at most time points, and the sentence says "reduction in risk".
- The values are odds ratios. No. The odds ratios from the counts are 0.38–0.74.
- A transcription or typesetting slip. No. The same pattern holds for all 24 printed ratios that have counts beside them, in the Abstract, Results, Table 3 and Table 4, and the CI widths match the code-mean scale. The PMC and Europe PMC copies of the publisher's JATS carry the same numbers.
- The ratio of 1/2-code means is the target quantity and only the label "risk ratio" is loose. Even read this way, the "about a 3% reduction in risk" statement and the "Boys vs Girls, RR" column describe risk, and the ratio of 1/2-code means is not a measure of risk. The Methods state that the 1/2 coding was used and that risk ratios are reported. The numbers show that these two statements describe different quantities.
- Table 4 orientation. Apart from the coding, the Table 4 values match girls/boys, not boys/girls as labeled. The direction of comparison in that column is therefore also mislabeled.
Status
- Version of record. The jamanetwork.com page could not be read: on 2026-10-23 the doi.org redirect target and the full-text page returned a bot-verification page (HTTP 403), as they had in an earlier check. The quoted passages and the reported values in Tables 2–4 were confirmed in the PMC-archived JATS, the live PMC page and the Europe PMC full-text XML (both retrieved 2026-10-23), which are identical at the quoted places. In the Europe PMC XML each Table 3 value sits in its own cell, so the columns are not misaligned. This is not a conversion artifact.
- Supplements. The only supplement is the trial protocol (jamanetwopen-1-e181213-s001.pdf). It contains no data or analysis output and does not bear on the computation.
- Corrections and comments. As of 2026-10-23, the only correction is "Error in Byline Order" (doi:10.1001/jamanetworkopen.2018.4304, PMC6324432; dated 2018-10-05 in Europe PMC, 2018-10-12 in Crossref). It changes only the author order. Crossref lists no update relations for the article DOI. Europe PMC (PMID 30646113) lists only that erratum and the invited commentary (doi:10.1001/jamanetworkopen.2018.1222). PubPeer could not be checked directly: its site and API returned Cloudflare blocks (HTTP 403) on 2026-10-23 and in an earlier check. Reader comments on the publisher page could not be checked, because that page could not be read. A general web search for the article title with "correction", "comment" and "letter" found the article, its university press releases, repository copies and the byline-order correction, and no letter, comment or blog post on the effect estimates.
- Sources. Paper as archived in PMC (PMC6324287; Tables 1–4, Methods, Results, Discussion); the live PMC page and the Europe PMC full-text XML (2026-10-23); Crossref and Europe PMC records; a general web search (2026-10-23). The publisher's page and PubPeer were attempted but could not be read. Calculations:
recompute.py.txtin this folder.
Suggested correction
- Refit the GEE models with the outcomes coded 0/1, for example as a log-link Poisson or log-binomial model with robust standard errors, adjusted for sex and clustering as described. Replace every risk ratio and 95% CI in the Abstract, the Results text (including the boys-only estimates) and Table 3, including the pooled 3–54 month rows.
- Recompute Table 4 on the 0/1 scale, state the direction of the comparison (boys vs girls) so that it matches the values, and re-estimate the intervention × sex interaction ratios.
- Revise "The magnitude of the intervention effects in this trial was modest, about a 3% reduction in risk" to reflect the corrected estimates. The crude risk ratios from the published counts suggest relative reductions of roughly a third to more than a half for perpetration.
Recomputation script (python recompute.py.txt)
"""Recompute the intervention-effect "risk ratios" of PMC6324287
(JAMA Netw Open 2018, doi:10.1001/jamanetworkopen.2018.1213).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the paper as archived in PMC:
Table 1: arm sizes and number of girls per arm
Table 2: cumulative and time-specific incidence counts by arm
Table 3: reported RRs (95% CI) and P values for the intervention effect
Table 4: incidence counts by sex and reported "Boys vs Girls" RRs
Methods: outcomes "coded 1 for respondents not reporting ever experiencing the
event and 2 for those reporting ever experiencing the event", analysed with
GEE Poisson regression adjusted for gender and clustering.
If a log-link Poisson model is fitted to a variable Y coded 1/2 instead of 0/1, the
exponentiated coefficient estimates E[Y | arm 1] / E[Y | arm 0] = (1 + p1) / (1 + p0),
not the risk ratio p1 / p0.
All confidence intervals computed here are unadjusted (individuals treated as
independent, no clustering); they are for comparing scales, not substitutes for the
trial's own clustered analysis.
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (from the paper)
MONTHS = [3, 6, 12, 42, 54]
N_INT, N_CTL = 561, 491 # Table 1
GIRLS = {"int": 306, "ctl": 251} # Table 1, "Female, No. (%)"
BOYS = {"int": N_INT - GIRLS["int"], "ctl": N_CTL - GIRLS["ctl"]}
# Table 2 cumulative incidence (a/n1 intervention, b/n0 control) and Table 3 reported RR, CI, P
PERP = {3: (9, 561, 20, 491, 0.978, 0.959, 0.997, .02),
6: (17, 561, 35, 491, 0.964, 0.941, 0.988, .004),
12: (21, 561, 42, 491, 0.959, 0.934, 0.985, .002),
42: (41, 561, 56, 491, 0.967, 0.937, 0.998, .04),
54: (52, 561, 68, 491, 0.964, 0.932, 0.997, .03)}
EXPE = {3: (13, 559, 21, 491, 0.983, 0.963, 1.004, .11),
6: (22, 559, 33, 491, 0.977, 0.953, 1.002, .07),
12: (30, 559, 47, 491, 0.966, 0.939, 0.994, .02),
42: (50, 559, 67, 491, 0.963, 0.931, 0.996, .03),
54: (72, 559, 82, 491, 0.972, 0.937, 1.008, .12)}
# Table 2 time-specific incidence (intervention list, control list) and Table 3 row "3-54"
TS = {"Perpetration": ([(9, 561), (8, 552), (4, 544), (20, 540), (11, 520)],
[(20, 491), (15, 471), (7, 456), (14, 449), (12, 435)], 0.990),
"Experience": ([(13, 559), (9, 546), (8, 537), (20, 529), (22, 509)],
[(21, 491), (12, 470), (14, 458), (20, 444), (15, 424)], 0.992)}
# Table 4: boys a/n1, girls b/n0, reported "Boys vs Girls" RR
T4 = {"Perpetration": {0: (4, 499, 1, 558, 0.994), 3: (26, 495, 3, 557, 0.956),
6: (47, 495, 5, 557, 0.923), 12: (58, 495, 5, 557, 0.904),
42: (78, 495, 19, 557, 0.894), 54: (95, 495, 25, 557, 0.878)},
"Experience": {0: (5, 499, 2, 558, 0.994), 3: (32, 494, 2, 556, 0.943),
6: (51, 494, 4, 556, 0.914), 12: (72, 494, 5, 556, 0.882),
42: (91, 494, 26, 556, 0.885), 54: (113, 494, 41, 556, 0.875)}}
# ------------------------------------------------------------------ helpers
def rr(a, n1, b, n0):
return (a / n1) / (b / n0)
def rr_ci(a, n1, b, n0):
r = rr(a, n1, b, n0)
se = math.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n0)
return r, r * math.exp(-Z * se), r * math.exp(Z * se), se
def code_ratio(a, n1, b, n0):
"""Ratio of mean 1/2 codes: (1 + p1) / (1 + p0)."""
return (1 + a / n1) / (1 + b / n0)
def code_ratio_ci(a, n1, b, n0):
p1, p0 = a / n1, b / n0
m1, m0 = 1 + p1, 1 + p0
se = math.sqrt(p1 * (1 - p1) / (n1 * m1 ** 2) + p0 * (1 - p0) / (n0 * m0 ** 2))
r = m1 / m0
return r, r * math.exp(-Z * se), r * math.exp(Z * se), se
def noevent_inverted(a, n1, b, n0):
"""Control 'no event' proportion / intervention 'no event' proportion."""
return ((n0 - b) / n0) / ((n1 - a) / n1)
def one_plus_rd(a, n1, b, n0):
return 1 + a / n1 - b / n0
def odds_ratio(a, n1, b, n0):
return (a * (n0 - b)) / (b * (n1 - a))
def two_prop_p(a, n1, b, n0):
p = (a + b) / (n1 + n0)
se = math.sqrt(p * (1 - p) * (1 / n1 + 1 / n0))
zz = (a / n1 - b / n0) / se
return 2 * stats.norm.sf(abs(zz))
def sex_confounding(t, outcome, kind):
"""Approximate sex standardization. Expected arm-level quantity if each arm had the
pooled sex-specific risks of Table 4 and no intervention effect; the crude ratio
divided by this factor approximates a sex-adjusted ratio. kind='code' or 'binary'."""
a, nb, b, ng, _ = T4[outcome][t]
pb, pg = a / nb, b / ng
e_int = (BOYS["int"] * pb + GIRLS["int"] * pg) / N_INT
e_ctl = (BOYS["ctl"] * pb + GIRLS["ctl"] * pg) / N_CTL
if kind == "code":
return (1 + e_int) / (1 + e_ctl)
return e_int / e_ctl
def hr(title):
print("\n" + "=" * 100 + "\n" + title + "\n" + "=" * 100)
# ------------------------------------------------------------------ 1. Table 3 cumulative incidence
hr("1. Table 3 (and Abstract/Results): reported RR vs quantities computed from Table 2 counts")
rows_fig1 = []
rows_fig2 = []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
print(f"\n{outcome}")
print(" mo | counts | reported RR (95% CI) | ratio of 1/2-code means (unadj. CI) "
"| sex-std code ratio | risk ratio p1/p0 (unadj. CI) | sex-std RR | rel. reduction | abs. diff (pp)")
for t, (a, n1, b, n0, R, L, U, P) in d.items():
cr, cl, cu, cse = code_ratio_ci(a, n1, b, n0)
r, rl, ru, rse = rr_ci(a, n1, b, n0)
cr_std = cr / sex_confounding(t, outcome, "code")
r_std = r / sex_confounding(t, outcome, "binary")
print(f" {t:>2} | {a:>2}/{n1} v {b:>2}/{n0} | {R:.3f} ({L:.3f}-{U:.3f}) "
f"| {cr:.3f} ({cl:.3f}-{cu:.3f}) | {cr_std:.3f} "
f"| {r:.3f} ({rl:.2f}-{ru:.2f}) | {r_std:.3f} "
f"| {100 * (1 - r):.0f}% | {100 * (a / n1 - b / n0):+.1f}")
rows_fig1.append((outcome, t, R, L, U, r, rl, ru))
rows_fig2.append((f"{outcome[:4]}. {t} mo", R, cr, noevent_inverted(a, n1, b, n0),
one_plus_rd(a, n1, b, n0), "T3"))
hr("2. Precision: width of reported CI vs unadjusted CI on each scale (log scale half-width / 1.96)")
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, R, L, U, P) in d.items():
rep_se = math.log(U / L) / (2 * Z)
_, _, _, cse = code_ratio_ci(a, n1, b, n0)
_, _, _, rse = rr_ci(a, n1, b, n0)
print(f" {outcome[:4]} {t:>2} mo: reported SE(log) {rep_se:.4f}; unadjusted SE(log) code-mean "
f"ratio {cse:.4f} (ratio {rep_se / cse:.2f}); unadjusted SE(log RR) {rse:.3f} "
f"(ratio of RR SE to reported SE: {rse / rep_se:.0f}x)")
hr("3. Significance: reported P vs unadjusted two-proportion test from the same counts")
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, R, L, U, P) in d.items():
print(f" {outcome[:4]} {t:>2} mo: reported P {P}; unadjusted two-proportion z-test P "
f"{two_prop_p(a, n1, b, n0):.4f}")
# ------------------------------------------------------------------ 2. Table 3 time-specific row
hr("4. Table 3 row '3-54' (time-specific incidence pooled over the 5 visits)")
for outcome, (iv, cv, R) in TS.items():
a = sum(x for x, _ in iv); n1 = sum(y for _, y in iv)
b = sum(x for x, _ in cv); n0 = sum(y for _, y in cv)
print(f" {outcome}: {a}/{n1} vs {b}/{n0}; reported {R:.3f}; ratio of code means "
f"{code_ratio(a, n1, b, n0):.3f}; risk ratio {rr(a, n1, b, n0):.3f}")
rows_fig2.append((f"{outcome[:4]}. 3-54 pooled", R, code_ratio(a, n1, b, n0),
noevent_inverted(a, n1, b, n0), one_plus_rd(a, n1, b, n0), "T3"))
# ------------------------------------------------------------------ 3. Table 4 sex comparison
hr("5. Table 4 'Boys vs Girls' RR vs quantities computed from the Table 4 counts")
for outcome, d in T4.items():
print(f"\n{outcome}")
for t, (a, n1, b, n0, R) in d.items():
print(f" {t:>2} mo: boys {a}/{n1} ({100 * a / n1:.1f}%) girls {b}/{n0} ({100 * b / n0:.1f}%); "
f"reported {R:.3f}; girls/boys code-mean ratio {code_ratio(b, n0, a, n1):.3f}; "
f"boys/girls code-mean ratio {code_ratio(a, n1, b, n0):.3f}; "
f"boys/girls risk ratio {rr(a, n1, b, n0):.2f}")
rows_fig2.append((f"{outcome[:4]}. " + ("baseline" if t == 0 else f"{t} mo"), R, code_ratio(b, n0, a, n1),
noevent_inverted(b, n0, a, n1), one_plus_rd(b, n0, a, n1), "T4"))
# Table 4 also prints pooled 3-54 month "Boys vs Girls" values, with no counts beside them
T4_POOLED = {"Perpetration": 0.968, "Experience": 0.965}
print("\n Table 4 rows '3-54' (no counts printed, so not recomputable): "
+ ", ".join(f"{k} {v:.3f}" for k, v in T4_POOLED.items()))
allt4 = [v[4] for d in T4.values() for v in d.values()] + list(T4_POOLED.values())
print(f" Table 4 'Boys vs Girls' column: {len(allt4)} values in all, "
f"{sum(len(d) for d in T4.values())} with counts; range {min(allt4):.3f}-{max(allt4):.3f}; "
f"all below 1: {all(v < 1 for v in allt4)}")
t4dev = [code_ratio(b, n0, a, n1) - R for d in T4.values() for (a, n1, b, n0, R) in d.values()]
t4rr = [rr(a, n1, b, n0) for d in T4.values() for (a, n1, b, n0, R) in d.values()]
print(f" the 12 with counts: max |girls/boys code-mean ratio - reported| {max(abs(v) for v in t4dev):.4f}; "
f"boys/girls risk ratios {min(t4rr):.2f}-{max(t4rr):.2f}")
# ------------------------------------------------------------------ 4. candidate definitions
hr("6. Which quantity do the 24 reported values track? (candidate minus reported)")
cands = {"ratio of mean 1/2 codes": 2, "inverted 'no event' ratio": 3, "1 + risk difference": 4}
for name, k in cands.items():
dev = np.array([row[k] - row[1] for row in rows_fig2])
print(f" {name:<26}: mean dev {dev.mean():+.4f}, max |dev| {np.abs(dev).max():.4f}, "
f"RMS {math.sqrt((dev ** 2).mean()):.4f}")
dev_rr = []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, *_r) in d.items():
dev_rr.append(rr(a, n1, b, n0) - _r[0])
for tab in ("T3", "T4"):
sub = [r for r in rows_fig2 if r[5] == tab]
print(f" {tab} ({len(sub)} values): max |dev| code-mean {max(abs(r[2] - r[1]) for r in sub):.4f}, "
f"no-event {max(abs(r[3] - r[1]) for r in sub):.4f}, 1+RD {max(abs(r[4] - r[1]) for r in sub):.4f}")
dev_std = []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, R, *_r) in d.items():
dev_std.append(code_ratio(a, n1, b, n0) / sex_confounding(t, outcome, "code") - R)
print(f" sex-standardized ratio of 1/2-code means (Table 3 cumulative, 10 values): max |dev| "
f"{max(abs(v) for v in dev_std):.4f}; equal to reported at 3 decimals in "
f"{sum(abs(v) < 0.0005 for v in dev_std)} of 10")
print(f" risk ratio p1/p0 (Table 3 cumulative only): deviations {min(dev_rr):+.3f} to {max(dev_rr):+.3f}")
print(" odds ratios, Table 3 cumulative: " + ", ".join(
f"{outcome[:4]} {t}: {odds_ratio(*v[:4]):.3f}" for outcome, d in (("Perpetration", PERP),
("Experience", EXPE))
for t, v in d.items()))
dev_or = [odds_ratio(*v[:4]) - v[4] for d in (PERP, EXPE) for v in d.values()]
print(f" odds ratio (Table 3 cumulative only): deviations {min(dev_or):+.4f} to {max(dev_or):+.4f}")
dev_code_cum = [r[2] - r[1] for r in rows_fig2[:10]]
print(f" crude ratio of 1/2-code means (Table 3 cumulative, 10 values): deviations "
f"{min(dev_code_cum):+.4f} to {max(dev_code_cum):+.4f}")
noev_t4 = [r[3] - r[1] for r in rows_fig2 if r[5] == "T4"]
print(f" inverted 'no event' ratio, Table 4: lowest deviation {min(noev_t4):+.4f} "
f"(figure 2 y-axis lower limit -0.046)")
# ------------------------------------------------------------------ 5. magnitude statement
hr("7. Discussion: 'about a 3% reduction in risk'")
rep = np.array([v[4] for v in PERP.values()])
print(f" mean(1 - reported perpetration RR) = {100 * np.mean(1 - rep):.1f}%")
rrs = np.array([rr(*v[:4]) for v in PERP.values()])
print(" perpetration relative reductions from counts: "
+ ", ".join(f"{100 * (1 - x):.0f}%" for x in rrs))
rre = np.array([rr(*v[:4]) for v in EXPE.values()])
print(" experience relative reductions from counts: "
+ ", ".join(f"{100 * (1 - x):.0f}%" for x in rre))
print(" ratio of relative reduction from counts to (1 - reported RR), perpetration: "
+ ", ".join(f"{(1 - x) / (1 - R):.1f}" for x, R in zip(rrs, rep)))
repe = np.array([v[4] for v in EXPE.values()])
print(" ratio of relative reduction from counts to (1 - reported RR), experience: "
+ ", ".join(f"{(1 - x) / (1 - R):.1f}" for x, R in zip(rre, repe)))
fold = [(1 - x) / (1 - R) for x, R in zip(np.r_[rrs, rre], np.r_[rep, repe])]
print(f" range of that ratio over all 10 points in Figure 1: {min(fold):.1f} to {max(fold):.1f}")
print(" perpetration absolute differences (pp): "
+ ", ".join(f"{100 * (v[0] / v[1] - v[2] / v[3]):.1f}" for v in PERP.values()))
hr("8. Can adjustment for sex move the risk ratio to ~0.97? Worst case: all events in one sex")
for t, (a, n1, b, n0, R, *_r) in PERP.items():
lo = (a / BOYS["int"]) / (b / BOYS["ctl"])
hi = (a / GIRLS["int"]) / (b / GIRLS["ctl"])
print(f" perpetration {t:>2} mo: within-boys RR if all events were boys {lo:.3f}; "
f"within-girls RR if all events were girls {hi:.3f}; reported {R:.3f}")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
# Figure 1: reported RR vs risk ratio from the counts, log scale, one axis
fig, ax = plt.subplots(figsize=(7.2, 5.4))
labels, ys = [], []
y = 0
for i, (outcome, t, R, L, U, r, rl, ru) in enumerate(rows_fig1):
if i == 5:
y += 0.8
yy = -y
ax.plot([L, U], [yy + 0.17] * 2, color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(R, yy + 0.17, "o", color=ORANGE, ms=8, mec=BG, mew=1.2, zorder=3)
ax.plot([rl, ru], [yy - 0.17] * 2, color=BLUE, lw=2, solid_capstyle="round")
ax.plot(r, yy - 0.17, "D", color=BLUE, ms=7.5, mec=BG, mew=1.2, zorder=3)
ax.text(U * 1.025, yy + 0.17, f"{R:.3f}", va="center", ha="left", fontsize=10, color=INK2)
ax.text(rl / 1.04, yy - 0.17, f"{r:.2f}", va="center", ha="right", fontsize=10, color=INK2)
labels.append(f"{t} mo")
ys.append(yy)
y += 1
ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.1, 1.6)
ticks = [0.1, 0.2, 0.3, 0.5, 0.7, 1.0, 1.5]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{x:g}" for x in ticks])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_yticks(ys)
ax.set_yticklabels(labels)
ax.set_ylim(ys[-1] - 0.7, 0.9)
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.103, ys[0] + 0.55, "Perpetrating forced sex", fontsize=10.5, fontweight="bold", va="bottom")
ax.text(0.103, ys[5] + 0.55, "Experiencing forced sex", fontsize=10.5, fontweight="bold", va="bottom")
ax.text(1.02, 0.75, "no effect", fontsize=10, color=INK2, va="center")
ax.set_xlabel("Intervention vs control, ratio (log scale)")
h1 = ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2,
label="Reported \"risk ratio\" and 95% CI (Table 3)")[0]
h2 = ax.plot([], [], "D-", color=BLUE, ms=7.5, lw=2,
label="Risk ratio from the paper's counts (Table 2), unadjusted 95% CI")[0]
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.11), ncol=1,
frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: which quantity the reported values track
fig, ax = plt.subplots(figsize=(7.2, 4.6))
x = np.arange(len(rows_fig2))
dev_code = np.array([r[2] - r[1] for r in rows_fig2])
dev_noev = np.array([r[3] - r[1] for r in rows_fig2])
dev_rd = np.array([r[4] - r[1] for r in rows_fig2])
ax.axhline(0, color=ORANGE, lw=2, zorder=1)
ax.plot(x, dev_rd, "s", ms=8, mfc=BG, mec=BLUE, mew=1.6, label="1 + risk difference")
ax.plot(x, dev_noev, "^", ms=8.5, mfc=BG, mec=BLUE, mew=1.6,
label="Inverted \"no event\" ratio, (1 − p₀)/(1 − p₁)")
ax.plot(x, dev_code, "o", ms=8, color=BLUE, mec=BG, mew=1.0,
label="Ratio of mean 1/2 codes, (1 + p₁)/(1 + p₀)")
std_x, std_dev = [], []
for outcome, d in (("Perpetration", PERP), ("Experience", EXPE)):
for t, (a, n1, b, n0, R, *_r) in d.items():
std_x.append(len(std_x)) # same order as the first 10 rows of rows_fig2
std_dev.append(code_ratio(a, n1, b, n0) / sex_confounding(t, outcome, "code") - R)
ax.plot(std_x, std_dev, "X", ms=9, color=BLUE, mec=BG, mew=0.8, zorder=4,
label="Ratio of mean 1/2 codes, approx. sex-standardized (Table 3 cumulative rows)")
ax.text(len(x) - 0.5, 0.0015, "reported value (difference = 0)", color=INK, fontsize=10,
ha="right", va="bottom")
n3 = sum(1 for r in rows_fig2 if r[5] == "T3")
ax.axvline(n3 - 0.5, color=INK2, lw=0.8, ls=(0, (2, 2)))
ax.text((n3 - 1) / 2, 0.012, "Table 3: intervention vs control", ha="center", fontsize=10, color=INK2)
ax.text(n3 + (len(x) - n3 - 1) / 2, 0.012, "Table 4: sex comparison", ha="center",
fontsize=10, color=INK2)
ax.set_xticks(x)
ax.set_xticklabels([r[0] for r in rows_fig2], rotation=60, ha="right", fontsize=10)
ax.set_ylabel("Candidate minus reported value")
ax.set_ylim(-0.046, 0.016)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="upper left", bbox_to_anchor=(-0.02, -0.33), frameon=False, fontsize=10)
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract's $1.3 billion TB cost total exceeds the paper's own yearly costs about tenfold
Lancet Reg Health Am (2024) · doi:10.1016/j.lana.2024.100905 · PMC11747192
Summary
The Abstract (Findings) and the Discussion give the total direct cost of pulmonary TB to Brazil's public health system over 2015–2022 as more than, or about, $1.3 billion. The Results give the same total as 108 million dollars, with yearly costs between 9.98 million (2022) and 21.38 million (2017) dollars; eight such years can add up to at most $171 million, so $1.3 billion is impossible on the paper's own numbers. The authors' public data file reproduces the yearly values exactly and sums to $131.3 million (all records) or $124.6 million (the 426,040 analysed cases); $1.3 billion is about ten times either. The figure is the abstract's headline estimate of the economic burden, and no correction has been published.
What the paper reports
Abstract, Findings:
The total direct cost for pulmonary TB in Brazil during the seven years exceeded $1.3 billion, with retreatment cases accounting for $23.5 million.
Discussion, second paragraph:
The total TB cost to the public health system over the last seven years amounted to approximately $1.3 billion. While costs varied according to each population, attention is drawn to the expenditure on retreatment cases, totalling $23.5 million.
Results, "Evaluating the direct cost of pulmonary TB from 2015 to 2022":
The overall cost of pulmonary TB treatment in Brazil exhibited variations over the years, reaching its highest value in 2017 (21.38 million dollars) and the lowest in 2022 (9.98 million dollars), accumulating to a total cost of 108 million dollars spent over the 7-year period (Fig. 2a). Further stratifying the cost based on the history of anti-TB treatment, surprisingly, despite fluctuations in the total cost and the cost of new cases, the cost of retreatment cases remained stable over the years with an accumulated total of 23,5.million dollars (Fig. 2b).
Methods, "Tuberculosis direct cost analysis", on how the total is built:
After calculating the cost per case, we summed the value of each case for each year of the study.
Results, "Characteristics of the study participants":
Following these exclusions, our study comprised a total of 426,040 pulmonary TB cases.
Results, cost needed to reach a 90% cure rate:
Initially, an investment of $ 275 was identified as necessary for the overall population to achieve a 90% cure rate. Further delineating the groups based on treatment history, the new case subgroup required a similar investment of $276 to attain the 90% cure rate, mirroring the overall population. However, the retreatment group necessitated increased efforts, with an investment of $318 for the same target (Fig. 6).
The per-group investments elsewhere in the Results run from $217 (health workers) to $366 (homeless). Table 1 has columns headed N = 345,908 (new cases) and N = 80,132 (retreatment).
What the paper's own numbers show
Method. The Methods define the total as the sum of per-case costs, added up by year. The Results give the largest and smallest yearly totals, which bound any total over the study period: 2015–2022 spans eight calendar years (the paper says seven), so the total must lie between 8 × 9.98 and 8 × 21.38 million dollars. I then checked the total against the paper's case counts and its per-case costs. Finally, I summed the per-case cost column (ind_cost) of the data file the paper links in its Data sharing statement (data/SINAN_TB.parquet in the authors' public repository, commit 6e74aaf of 2024-09-09, SHA-256 checked), following the authors' own code (scripts/Main_code.Rmd), which builds Fig. 2a as the sum of ind_cost by year over all records. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Total direct cost, 2015–2022 | Abstract (Findings); Discussion | "exceeded $1.3 billion"; "approximately $1.3 billion" | Not possible: at most $171.0M from the stated yearly maximum |
| Total direct cost, 2015–2022 | Results (Fig. 2a) | 108 million dollars | Within the bounds: $69.9M–$149.7M (7 years), $79.8M–$171.0M (8 years) |
| Ratio of the two totals | Abstract vs Results | $1.3B vs $108M | 12.04× |
| Years at the 2017 maximum needed to reach $1.3B | Results | 7-year period | 60.8 years |
| Cost per analysed case (426,040) | Results, Fig. 1 | not given | $253 if $108M; $3,051 if $1.3B |
| Paper's modelled cost per case for 90% cure | Results, Figs 6–7 | $217–$366 (overall $275) | $1.3B per case is 8.3× the highest |
| $1.3B spread over all 743,823 notified TB cases | Results | not given | $1,748 per case |
| Retreatment ($23.5M) as share of total | Abstract; Results; Table 1 | $23.5M | 21.8% of $108M (retreatment is 18.8% of cases); 1.8% of $1.3B |
| Implied cost per new case vs per retreatment case | Table 1 counts | retreatment needs more ($318 vs $276) | $244 vs $293 if $108M; $3,690 vs $293 if $1.3B |
| Yearly totals, authors' data (million USD) | Fig. 2a; Results | 21.38 (2017), 9.98 (2022) | 13.98, 16.48, 21.38, 20.81, 20.06, 14.02, 14.60, 9.99 (2015–2022) |
| Total, authors' data, all 458,297 records | Fig. 2a | 108 million dollars | $131.3M |
| Total, authors' data, 426,040 analysed cases | Results; Table 1 | — | $124.6M (row counts 345,908 new and 80,132 retreatment match Table 1) |
| New cases only, authors' data, all records | Fig. 2b | — | $107.8M |
| Retreatment only, authors' data, all records | Results; Fig. 2b | 23,5 million dollars | $23.6M |
| Highest cost of any single case | authors' data | — | $490.24; 426,040 cases at that cost = $208.9M |
| $1.3B / data total | — | — | 9.90× (all records); 10.44× (analysed cases) |
| Data total in BRL, year by year at Table S3 rates | Supplementary Table S3 | — | R$530M (0.53B BRL) |
Every check in the paper's own text and data points to a total of roughly $100–$130 million. The $1.3 billion in the Abstract and Discussion is about 12 times the Results figure and about 10 times what the authors' data add up to.
A second, smaller point appears in the data. The Results call 108 million dollars the overall total, but the authors' data and the yearly values plotted in Fig. 2a sum to $131.3 million. $108 million matches the new-case subtotal ($107.8 million), and $107.8M + $23.6M (retreatment) = $131.3M. The Results total therefore appears to leave out retreatment cases. This does not change the main finding: every candidate total lies between about $108 and $131 million.
Figure
Figure 1. The two statements of the same quantity, the total direct cost of pulmonary TB over 2015–2022. Left: the Abstract (Findings) and Discussion, $1,300 million. Right: the Results, which give the yearly maximum and minimum and a total of $108 million; the two differ by a factor of 12. Orange marks values printed in the paper. The blue lines below are recomputed by recompute.py.txt: the range an eight-year total can take given the stated yearly extremes, and the totals of the authors' public data.
Figure 2. The same totals on one linear axis that starts at zero, in million US$. Orange dots: values reported in the paper (Abstract and Discussion, 1,300; Results, 108). Blue: recomputed values (the range 80–171 allowed by the stated yearly maximum and minimum over eight years; the authors' data summed over all records, 131, over the 426,040 analysed cases, 125, and over new cases only, 108). Only the Abstract value lies outside the range the paper's own yearly figures allow.
Why it matters
- It is the paper's headline economic result. The total direct cost opens the Abstract's Findings and the Discussion's report of results. It is the number most likely to be quoted in citing papers, policy documents and budget discussions, and the paper presents itself as the first national estimate of this cost for Brazil.
- The overstatement is about tenfold. A reader of the Abstract would take the seven-year cost to the public health system as more than $1.3 billion (about $160–185 million a year). The paper's own Results and data give about $108–131 million in total, roughly $14–19 million a year, or about $250–$290 per case.
- It distorts the retreatment share. Read against $1.3 billion, the $23.5 million spent on retreatment cases looks like about 2% of spending. Against the data total it is about 18–22%, close to the 18.8% share of cases that are retreatments. The paper singles out retreatment spending in both the Abstract and the Discussion.
- Other results are not affected. The number-needed-to-treat estimates for directly observed treatment, the per-case investments needed to reach a 90% cure rate ($217–$366), and the yearly pattern in Fig. 2 come from per-case costs and models, not from the overall total, and they agree with the data.
Explanations considered
- A different currency. Not supported. The Methods convert costs from Brazilian reais to US dollars using yearly rates (Supplementary Table S3: 3.19–5.40 BRL per USD), and all reported values carry a dollar sign. $108 million is R$0.34–0.58 billion at those rates, and the authors' data total converted year by year is R$0.53 billion. Neither gives 1.3 billion. The authors' Portuguese abstract (Supplementary mmc2) also reads "excedeu US$ 1,3 bilhão", in dollars.
- A wider population. Not supported. Spread over all 743,823 TB notifications in 2015–2022, including those the study excluded, $1.3 billion is still $1,748 per case. That is about 5–8 times every per-case cost in the paper. In the authors' data the most expensive single case costs $490.24; even if all 426,040 analysed cases cost that much, the total would be $208.9 million.
- A broader definition of cost. Not supported. The Limitations say the analysis is "omitting costs related to hospitalization, maintenance of health unit infrastructure, indirect costs such as productivity loss, undiagnosed cases, death, patient expenses, and the broader societal impact". The paper offers no wider cost measure that could produce $1.3 billion.
- The length of the period. Not supported. 2015–2022 spans eight calendar years, although the paper says seven, but eight years at the 2017 maximum still come to only $171 million. Reaching $1.3 billion would take 60.8 years at that rate.
- A per-case or cumulative figure misread. Not supported. The Methods describe the total as the sum of per-case costs by year, and Fig. 2a is described as "the cumulative economic burden over the seven years". The yearly values in the Results and the data are already cumulative within each year.
- A typographical slip in the Results rather than the Abstract. Not supported. The yearly maximum and minimum in the Results, the yearly sums in the authors' data, the retreatment share and the per-case costs all agree with a total of about $108–131 million. Only the Abstract and Discussion figure disagrees with them. An earlier conference abstract of this study (MEDTROP 2024) states that the total "exceeded $108 million". The leading digits of $1.3 billion match the data total of $131 million, which would fit a misplaced decimal point, but the paper gives no basis for deciding how the figure arose.
Status
- Version of record. The publisher pages (thelancet.com and sciencedirect.com, pii S2667193X24002321) returned HTTP 403. The version of record was read on the live PMC page and in the publisher full-text XML served by Europe PMC; both contain "$1.3 billion" in the Abstract Findings and the Discussion and "108 million dollars" in the Results, identical to the archived text quoted above. This is not a conversion artefact.
- Supplements. Europe PMC supplementary files checked: mmc1.docx (Supplementary Methods, Figs S1–S6, Tables S1–S5) and mmc2.docx (Portuguese abstract). The Supplementary Methods state that the per-case costs were summed by year "and then summed the years to obtain the total economic cost or burden"; no supplementary table gives an overall total. The Portuguese abstract repeats "US$ 1,3 bilhão".
- Data and code. The authors' public repository (linked in the paper) holds
data/SINAN_TB.parquet(458,297 records, committed 2024-09-09) andscripts/Main_code.Rmd. Summing itsind_costcolumn by year reproduces the yearly values in the Results. - Corrections and comments. No erratum, correction or retraction notice was found on Crossref, PMC or Europe PMC. PubPeer's search page and API returned a human-verification block and could not be queried directly; a web search found no PubPeer thread or published correction for this article.
- Sources. PMC11747192; Europe PMC full-text XML; authors' data and code at the GitHub address given in the paper's Data sharing statement (commit 6e74aaf); MEDTROP 2024 conference abstract.
- As of 2026-10-24.
Suggested correction
In the Abstract (Findings) and the Discussion (second paragraph), the total direct cost should be replaced by the total from the study's data. The authors should confirm which population the headline total covers. Summed over all records, as in Fig. 2a, it is about $131 million; over the 426,040 analysed cases it is about $125 million. For example: "The total direct cost for pulmonary TB in Brazil during 2015–2022 was approximately $131 million, with retreatment cases accounting for $23.5 million." The same figure should replace "approximately $1.3 billion" in the Discussion. The Results sentence giving "a total cost of 108 million dollars" should be checked as well, since $108 million matches the new-case subtotal rather than the overall total shown in Fig. 2a. "23,5.million" should read "23.5 million". The authors may also wish to say "eight years" (2015–2022) instead of "seven years" in the title, Abstract, Results and Discussion.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC11747192 (Lancet Reg Health Am 2024; doi:10.1016/j.lana.2024.100905)
# Total direct cost of pulmonary TB, 2015-2022: "$1.3 billion" (Abstract, Discussion)
# versus "108 million dollars" (Results, Fig. 2a).
#
# Run: python recompute.py.txt
# Needs numpy and matplotlib. Part B also needs pandas + pyarrow and network access
# to download the authors' public data file (pinned commit, checked by SHA-256);
# without them Part B is skipped and the figures use the values recorded below,
# labelled as such in the output.
#
# Prints every recomputed number used in writeup.md and writes fig1.svg/png and
# fig2.svg/png into the folder this script is in.
import hashlib, os, sys, tempfile, urllib.request
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- Part A
# Numbers printed in the paper (as archived in PMC)
TOT_ABSTRACT = 1.3e9 # Abstract (Findings) "exceeded $1.3 billion"; Discussion "approximately $1.3 billion"
TOT_RESULTS = 108e6 # Results: "total cost of 108 million dollars spent over the 7-year period"
RET_COST = 23.5e6 # Abstract/Discussion "$23.5 million"; Results "23,5.million dollars"
YMAX, YMIN = 21.38e6, 9.98e6 # Results: highest year 2017, lowest year 2022
N_ANALYSED = 426_040 # Results / Fig. 1
N_NEW, N_RET = 345_908, 80_132 # Table 1 column headers
N_NOTIFIED = 743_823 # Results: all TB cases reported 2015-2022
CURE90 = {"overall": 275, "new cases": 276, "retreatment": 318,
"lowest group (health workers)": 217, "highest group (homeless)": 366} # Results, Figs 6-7
# Supplementary Table S3: USD to BRL ratio by year
FX = {2015: 3.33, 2016: 3.49, 2017: 3.19, 2018: 3.65, 2019: 3.95, 2020: 5.16, 2021: 5.40, 2022: 5.16}
def m(x):
return f"${x/1e6:,.1f}M"
print("=" * 72)
print("PART A - from the numbers printed in the paper")
print("=" * 72)
print(f"N_new + N_ret = {N_NEW + N_RET:,} (paper: {N_ANALYSED:,})")
print(f"Retreatment share of analysed cases: {N_RET/N_ANALYSED:.1%}")
print()
print("Bounds on the total implied by the stated yearly maximum and minimum:")
bounds = {}
for n in (7, 8):
lo, hi = n * YMIN, n * YMAX
bounds[n] = (lo, hi)
print(f" {n} years: {m(lo)} to {m(hi)}")
print(" (2015-2022 inclusive is 8 calendar years; the paper says 'seven years')")
print(f" 108M inside the 8-year bounds: {bounds[8][0] <= TOT_RESULTS <= bounds[8][1]}")
print(f" 1.3B inside the 8-year bounds: {bounds[8][0] <= TOT_ABSTRACT <= bounds[8][1]}")
print()
print(f"Ratio 1.3B / 108M = {TOT_ABSTRACT/TOT_RESULTS:.2f}")
print(f"Ratio 1.3B / 8-year upper bound = {TOT_ABSTRACT/bounds[8][1]:.2f}")
print(f"Years at the 2017 maximum needed to reach 1.3B: {TOT_ABSTRACT/YMAX:.1f}")
print(f"Average per year needed for 1.3B over 8 years: {m(TOT_ABSTRACT/8)} "
f"({TOT_ABSTRACT/8/YMAX:.1f} x the stated maximum year)")
print()
print("Cost per analysed case:")
print(f" 108M / {N_ANALYSED:,} = ${TOT_RESULTS/N_ANALYSED:,.0f}")
print(f" 1.3B / {N_ANALYSED:,} = ${TOT_ABSTRACT/N_ANALYSED:,.0f}")
print(f" 1.3B / {N_NOTIFIED:,} (all notified TB) = ${TOT_ABSTRACT/N_NOTIFIED:,.0f}")
print(f" Paper's modelled cost to reach 90% cure: ${min(CURE90.values())}-${max(CURE90.values())} per case "
f"(overall ${CURE90['overall']})")
print(f" 1.3B per notified case / modelled cost: {TOT_ABSTRACT/N_NOTIFIED/max(CURE90.values()):.1f}x to "
f"{TOT_ABSTRACT/N_NOTIFIED/min(CURE90.values()):.1f}x")
print(f" 1.3B per analysed case / highest modelled cost = {TOT_ABSTRACT/N_ANALYSED/max(CURE90.values()):.1f}x")
print()
print("Retreatment ($23.5M) as a share of the total:")
print(f" of 108M: {RET_COST/TOT_RESULTS:.1%} of 1.3B: {RET_COST/TOT_ABSTRACT:.1%}")
print(f" per retreatment case: ${RET_COST/N_RET:,.0f}")
print(f" implied per new case if total = 108M: ${(TOT_RESULTS-RET_COST)/N_NEW:,.0f}")
print(f" implied per new case if total = 1.3B: ${(TOT_ABSTRACT-RET_COST)/N_NEW:,.0f}")
print()
print("Currency check (Table S3 USD-to-BRL ratios 3.19-5.40):")
print(f" 108M USD x {min(FX.values())} = {TOT_RESULTS*min(FX.values())/1e9:.2f}B BRL; "
f"x {max(FX.values())} = {TOT_RESULTS*max(FX.values())/1e9:.2f}B BRL")
# ---------------------------------------------------------------- Part B
# The authors' public data file (linked in the paper's Data sharing statement)
COMMIT = "6e74aafaafd3c1ee379a417b70e303084d1946a0"
URL = f"https://raw.githubusercontent.com/rodriguesmsb/neapTB/{COMMIT}/data/SINAN_TB.parquet"
SHA = "6ba042b6419035362bec4a3440dbdbe4b23dcf12ec59e8734c861e40a34a5ed0"
# Values from a run of Part B on 2026-10-24, used only if the data cannot be loaded
RECORDED = {"all": 131.32396e6, "analysed": 124.55771e6, "new_all": 107.75637e6,
"ret_all": 23.56759e6, "new_an": 102.99102e6, "ret_an": 21.56669e6,
"years": {2015: 13.98, 2016: 16.48, 2017: 21.38, 2018: 20.81,
2019: 20.06, 2020: 14.02, 2021: 14.60, 2022: 9.99}}
print()
print("=" * 72)
print("PART B - from the authors' public data (SINAN_TB.parquet, commit " + COMMIT[:7] + ")")
print("=" * 72)
D = None
try:
import pandas as pd
tmp = tempfile.mkdtemp()
path = os.path.join(tmp, "SINAN_TB.parquet")
urllib.request.urlretrieve(URL, path)
h = hashlib.sha256(open(path, "rb").read()).hexdigest()
assert h == SHA, f"SHA-256 mismatch: {h}"
df = pd.read_parquet(path, columns=["nu_ano", "tratamento", "situa_ence", "ind_cost"])
os.remove(path)
print(f"rows: {len(df):,}; SHA-256 verified")
# Fig. 2a in the authors' code (scripts/Main_code.Rmd): sum(ind_cost) by year over all rows
yrs = (df.groupby("nu_ano")["ind_cost"].sum() / 1e6).round(2)
print("Yearly sum of ind_cost (million USD):",
", ".join(f"{int(k)}: {v:.2f}" for k, v in yrs.items()))
print(f" max {yrs.max():.2f} in {int(yrs.idxmax())} (paper: 21.38 in 2017); "
f"min {yrs.min():.2f} in {int(yrs.idxmin())} (paper: 9.98 in 2022)")
# the paper's analysed cohort: the four outcome categories used in the authors' models
an = df[df["situa_ence"].isin(["Cure", "Death", "Treatment incomplete", "Failure"])]
g_all = df.groupby("tratamento")["ind_cost"].agg(["sum", "count"])
g_an = an.groupby("tratamento")["ind_cost"].agg(["sum", "count"])
print(f"analysed-cohort rows: {len(an):,} (paper: {N_ANALYSED:,}); "
f"new {int(g_an.loc[0,'count']):,}, retreatment {int(g_an.loc[1,'count']):,} "
f"(Table 1: {N_NEW:,}, {N_RET:,})")
D = {"all": df["ind_cost"].sum(), "analysed": an["ind_cost"].sum(),
"new_all": g_all.loc[0, "sum"], "ret_all": g_all.loc[1, "sum"],
"new_an": g_an.loc[0, "sum"], "ret_an": g_an.loc[1, "sum"],
"years": {int(k): float(v) for k, v in yrs.items()},
"n_all": len(df), "mean_all": df["ind_cost"].mean(),
"mean_an": an["ind_cost"].mean(), "max": df["ind_cost"].max()}
except Exception as e: # no network, no pandas/pyarrow, or changed file
print(f"Part B skipped ({type(e).__name__}: {e}); using values recorded on 2026-10-24")
D = dict(RECORDED)
print(f"Total, all {D.get('n_all', 458297):,} rows (basis of Fig. 2a): {m(D['all'])}")
print(f" new cases (tratamento 0): {m(D['new_all'])}; retreatment (tratamento 1): {m(D['ret_all'])}")
print(f"Total, analysed cohort of 426,040: {m(D['analysed'])}")
print(f" new cases: {m(D['new_an'])}; retreatment: {m(D['ret_an'])}")
print(f"Sum of the rounded yearly values: {m(sum(D['years'].values())*1e6)}")
if "mean_all" in D:
print(f"Mean cost per row: ${D['mean_all']:,.0f} (all rows), ${D['mean_an']:,.0f} (analysed); "
f"max per case ${D['max']:,.2f}")
print(f" {N_ANALYSED:,} cases all at the maximum per-case cost: {m(N_ANALYSED*D['max'])}")
print(f"1.3B / data total (all rows) = {TOT_ABSTRACT/D['all']:.2f}")
print(f"1.3B / data total (analysed) = {TOT_ABSTRACT/D['analysed']:.2f}")
print(f"Results 108M vs data new-case subtotal: {m(D['new_all'])}")
print(f"Retreatment $23.5M as share of data total: {RET_COST/D['all']:.1%} (all records), "
f"{RET_COST/D['analysed']:.1%} (analysed)")
print(f"Per year: data total {m(D['all']/8)} (8 yrs) to {m(D['all']/7)} (7 yrs); "
f"108M {m(TOT_RESULTS/8)} (8 yrs); 1.3B {m(TOT_ABSTRACT/8)} (8 yrs) to {m(TOT_ABSTRACT/7)} (7 yrs)")
brl = sum(D["years"][y] * 1e6 * FX[y] for y in FX)
print(f"Data total converted to BRL year by year (Table S3): R${brl/1e6:,.0f}M = {brl/1e9:.2f}B BRL")
# ---------------------------------------------------------------- Figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG})
def esc(t): # literal dollar signs, not mathtext
return t.replace("$", r"\$")
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
plt.close(fig)
# ---- Figure 1: the two conflicting statements
fig = plt.figure(figsize=(7.2, 4.1))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 60); ax.axis("off")
def panel(x, w, title, lines, value, edge):
ax.add_patch(FancyBboxPatch((x, 13), w, 40, boxstyle="round,pad=0,rounding_size=1.6",
fc="white", ec=edge, lw=2))
ax.text(x + 2.2, 49.5, esc(title), fontsize=11, fontweight="bold", color=INK, va="top")
ax.text(x + 2.2, 43.5, esc("\n".join(lines)), fontsize=10, color=INK2, va="top", linespacing=1.45)
ax.text(x + w / 2, 17.5, esc(value), fontsize=17, fontweight="bold", color=edge, ha="center")
panel(2, 43, "Abstract and Discussion",
["“The total direct cost for pulmonary",
"TB in Brazil during the seven years",
"exceeded $1.3 billion”",
"“...amounted to approximately",
"$1.3 billion”"],
"$1,300 million", ORANGE)
panel(55, 43, "Results (Fig. 2a)",
["highest year 2017: 21.38 million dollars",
"lowest year 2022: 9.98 million dollars",
"“accumulating to a total cost of",
"108 million dollars spent over the",
"7-year period”"],
"$108 million", ORANGE)
ax.annotate("", xy=(55, 33), xytext=(45, 33),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(50, 36, "same total", ha="center", fontsize=10, color=INK)
ax.text(50, 28.2, f"{TOT_ABSTRACT/TOT_RESULTS:.0f}×\napart", ha="center", va="top",
fontsize=11, fontweight="bold", color=INK)
ax.text(2, 7.5, esc(f"Recomputed: the yearly maximum and minimum bound an 8-year total to "
f"{m(bounds[8][0])}–{m(bounds[8][1])}."), fontsize=10, color=BLUE)
ax.text(2, 2.5, esc(f"The authors' public data sum to {m(D['all'])} (all records) and "
f"{m(D['analysed'])} (426,040 analysed cases)."), fontsize=10, color=BLUE)
ax.text(2, 57, "Orange: values reported in the paper. Blue: recomputed from the paper's own numbers and data.",
fontsize=10, color=INK2)
save(fig, "fig1")
# ---- Figure 2: magnitudes on one linear axis
rows = [ # label, value, lo, hi, kind
("Abstract and Discussion", TOT_ABSTRACT, None, None, "rep"),
("Results text (Fig. 2a)", TOT_RESULTS, None, None, "rep"),
("Range allowed by stated yearly\nmax and min (8 years)", None, bounds[8][0], bounds[8][1], "rec"),
("Authors' data: all records\n(sum behind Fig. 2a)", D["all"], None, None, "rec"),
("Authors' data: 426,040\nanalysed cases", D["analysed"], None, None, "rec"),
("Authors' data: new cases only\n(all records)", D["new_all"], None, None, "rec"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = list(range(len(rows)))[::-1]
for y, (lab, v, lo, hi, kind) in zip(ys, rows):
c = ORANGE if kind == "rep" else BLUE
if v is None:
ax.plot([lo / 1e6, hi / 1e6], [y, y], color=c, lw=2, solid_capstyle="butt")
for e in (lo, hi):
ax.plot([e / 1e6] * 2, [y - 0.18, y + 0.18], color=c, lw=2)
ax.text(hi / 1e6 + 22, y, f"{lo/1e6:.0f}–{hi/1e6:.0f}", va="center", fontsize=10, color=INK)
else:
ax.plot([0, v / 1e6], [y, y], color=c, lw=2, alpha=0.35)
ax.plot(v / 1e6, y, "o", ms=9, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(v / 1e6 + (22 if v < 1e9 else -22), y, f"{v/1e6:,.0f}", va="center",
ha="left" if v < 1e9 else "right", fontsize=10, color=INK,
bbox=None if v < 1e9 else dict(fc=BG, ec="none", pad=1))
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlim(0, 1400); ax.set_ylim(-0.7, len(rows) - 0.3)
ax.set_xlabel("Total direct cost of pulmonary TB, 2015–2022 (million US$)", color=INK)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2); ax.tick_params(colors=INK2, length=0)
from matplotlib.lines import Line2D
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported in the paper"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE,
label="Recomputed from the paper's numbers / authors' data")],
loc="lower right", frameon=False, fontsize=10)
save(fig, "fig2")
print()
print("Figures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Abstract says most muddy coasts erode over 1 m/yr; Table 1 shows 30.5%
Nat Commun (2023) · doi:10.1038/s41467-023-43819-6 · PMC10716179
Summary
The abstract states, and the article's web summary repeats, that "the majority of the world's muddy coasts are eroding at rates exceeding 1 m/yr over the last three decades". The paper's own Table 1 puts 30.5% of muddy coastline in the four erosion classes beyond −1 m/yr, 40.3% in the stable class and 29.2% in the accretion classes. The Results text gives the same split (31%, 40%, 29%) and reports net global accretion of +0.18 m/yr, a gain of 579 km². Eroding coasts are therefore neither a majority nor the largest class. This is one of the two findings the abstract reports, so readers of the abstract alone are told that most muddy coasts are retreating, whereas the paper finds erosion and accretion nearly balanced by length (30.5% vs 29.2%) and a small net gain in area.
What the paper reports
Abstract:
Furthermore, the majority of the world’s muddy coasts are eroding at rates exceeding 1 m/yr over the last three decades.
Web summary (a separate abstract element of type "web-summary" in the archived XML; on nature.com it is the article's description metadata used in search and link previews, not printed on the article page):
14% of the world’s coastlines are muddy and the majority of them are eroding at rates exceeding 1 m per year over the last three decades, according to an automated classification method that identifies global coastlines.
Results, "Dynamics of muddy coasts vs. sandy coasts", second paragraph:
The analysis of coastline dynamics reveals that 40% of the world’s muddy coasts are stable (absolute long-term change rates of <1 m/year over the period 1984–2016); 31% are persistently eroding, while 29% are accreting (Table 1).
On a global scale, the world’s muddy coasts have accreted 0.18 m/year on average over the three decades of the study period, i.e. a total gain of 579 km2 over this period.
Results, same section, first paragraph:
After filtering the muddy coastline transects for a minimum number of historic data points, temporal coverage, potential ice coverage and outliers, 97.7% of the mud coasts remain suitable for further analysis.
Methods, "Dynamic coastline detection":
In addition, the thresholds for accretion and erosion classes have been increased to 1 m/year to focus on the more dynamic muddy shorelines.
Table 1, "Chronic coastline dynamics classification scheme", muddy-coast column (% of muddy coastline): very extreme accretion (>10 m/yr) 8.8; extreme accretion (5 to 10) 5.4; severe accretion (3 to 5) 4.3; intense accretion (1 to 3) 10.7; stable (−1 to 1) 40.3; intense erosion (−1 to −3) 12.2; severe erosion (−3 to −5) 4.8; extreme erosion (−5 to −10) 5.7; very extreme erosion (<−10) 7.8.
What the paper's own numbers show
Method. The abstract's threshold (1 m/yr) is the same as the boundary between the stable class and the erosion and accretion classes in Table 1 and the Methods. So the share of muddy coastline "eroding at rates exceeding 1 m/yr" is the sum of Table 1's four erosion classes. I summed the muddy column by direction, checked that the column totals 100%, bounded the effect of rounding and of the 2.3% of coastline removed by filtering, and computed the one alternative denominator under which erosion is a majority. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Muddy coast eroding > 1 m/yr | Abstract; web summary | "the majority" (> 50%) | 30.5% (12.2 + 4.8 + 5.7 + 7.8) |
| Muddy coast eroding > 1 m/yr | Results, "Dynamics of muddy coasts vs. sandy coasts", para. 2 | 31% | 30.5% (agrees) |
| Muddy coast stable, |rate| < 1 m/yr | Results, same section, para. 2; Table 1 | 40%; 40.3 | 40.3% (the largest class) |
| Muddy coast accreting > 1 m/yr | Results, same section, para. 2; Table 1 | 29% | 29.2% (8.8 + 5.4 + 4.3 + 10.7) |
| Muddy column total | Table 1 | — | 100.0% |
| Eroding share allowing for rounding of the four entries | Table 1 | — | 30.3% to 30.7% |
| Eroding share if all filtered-out coast (2.3%) were eroding | Results, same section, para. 1 | 97.7% retained | at most 32.1% |
| Changing by > 1 m/yr in either direction | Table 1 | — | 59.7% |
| Eroding share among coasts changing by > 1 m/yr only | Table 1 | — | 51.1% (30.5 / 59.7) |
| Accreting minus eroding | Table 1 | — | −1.3 percentage points |
| Global mean change rate of muddy coasts | Results, same section, para. 2 | +0.18 m/yr; +579 km² | not recomputable from the paper; sign is net accretion |
Erosion falls about 20 percentage points short of a majority of all muddy coastline. The only reading that gives a majority is a different denominator, coasts that change by more than 1 m/yr, and even then the majority is 51.1%.
Figure
Figure 1. The two statements about the same quantity: the share of all muddy coastline eroding faster than 1 m/yr over 1984–2016. Left (orange outline): the abstract's wording, repeated in the web summary used for search and link previews, which implies more than 50%. Right (grey outline): the Results text and the sum of the four erosion classes of Table 1 (blue, recomputed by recompute.py.txt), with the stable and accretion shares from the same table. The quoted text is abridged with ellipses; the full sentences are given above.
Figure 2. Shares of muddy coastline on a single 0–100% axis. Blue dots are recomputed from Table 1; open orange circles are the rounded values in the Results text; the orange bar from 50% to 100% is the range the abstract's "the majority" implies for the top row. The vertical line marks 50%. The fourth row assigns all of the 2.3% of coastline removed by filtering to erosion, the most favourable case for the abstract. The fifth row changes the denominator to coasts that move by more than 1 m/yr in either direction; it is the only quantity that crosses 50%, and the abstract does not state that restriction.
Why it matters
- It is a headline finding. The abstract reports two results: where muddy coasts are, and how they are changing. The second is stated in the abstract and repeated in the web summary, which nature.com uses as the article's description in search and link previews.
- It misstates the balance of change, not only the size. By length, the body finds erosion and accretion nearly balanced (30.5% vs 29.2%, erosion ahead by 1.3 points), and by net area and mean rate it finds a gain of 579 km² (+0.18 m/yr). The abstract reads as a finding that most muddy coasts are in retreat. Readers who cite the abstract for a "majority eroding" figure would be citing a number the paper does not contain.
- The body and the analysis are consistent. Table 1, the Results text and the net-accretion figure agree with one another. The error is confined to how the abstract summarizes them. The body's point that muddy coasts are more dynamic than sandy ones (59.7% vs 30.2% changing by more than 1 m/yr) is not affected.
Explanations considered
- A different denominator. Partly matches the numbers, but not the wording. Among coasts that change by more than 1 m/yr, eroding coasts are 51.1%, a bare majority. The abstract says "the majority of the world's muddy coasts", with no restriction to changing coasts.
- "Majority" used loosely for "largest group". Does not hold. Stable coast (40.3%) is the largest class; eroding coast (30.5%) is second, ahead of accreting by 1.3 percentage points.
- Rounding. Does not account for it. Each Table 1 entry is printed to one decimal, so the sum of the four erosion entries is uncertain by at most ±0.2 percentage points (30.3% to 30.7%).
- Coastline removed by filtering. Does not account for it. 97.7% of muddy coastline was kept. If every removed stretch were eroding faster than 1 m/yr, the eroding share would be at most 32.1%.
- Net loss rather than share of coastline. Does not hold. The paper reports net accretion of +0.18 m/yr and a 579 km² gain, so a net-change reading also contradicts "eroding".
- A different threshold or unit. Does not hold. The Methods set the erosion and accretion thresholds at 1 m/year, the figure the abstract uses, and Table 1's erosion classes start at −1 m/yr.
- A different period or dataset. Does not hold. The abstract's "last three decades" and the Results' 1984–2016 refer to the same analysis, and the Results sentence cites Table 1 directly.
- "Most muddy coasts are dynamic" written as "eroding". Consistent with the numbers: 59.7% of muddy coastline changes by more than 1 m/yr in one direction or the other. This would explain the wording but would still need a correction, since the published sentence says eroding.
Status
- Version of record. Checked on the publisher's site (nature.com/articles/s41467-023-43819-6, including the separate Table 1 page). The abstract sentence, the web-summary sentence, the Results sentence and the Table 1 values are worded identically to the PMC archive and to the Europe PMC full-text XML. This is not a conversion artifact. On nature.com the visible abstract ends at "...over the last three decades."; the web-summary sentence is not printed on the article page but appears in its description metadata (og:description, twitter:description, and appended to the abstract in the description, dc.description and JSON-LD fields).
- Supplements and data. The Supplementary Information PDF (34 pages) gives no global eroding, stable or accreting percentages and does not restate the majority claim. The source data on Zenodo (doi:10.5281/zenodo.7582197) hold the coast-type classification of 1,826,995 transects but no shoreline change rates, so Table 1 cannot be recomputed from deposited data. The check here is of internal consistency between the abstract and the paper's own Table 1 and text.
- Corrections and comments. None found. Crossref (no update-to or updated-by records), PMC and Europe PMC notices, the article page (no change history or author correction), PubPeer, and a web search for corrections, errata and commentary returned nothing. The Europe PMC search endpoint was unavailable (HTTP 503) at the time of checking; the PMC and Europe PMC notice lookups themselves returned no entries.
- As of 2026-10-24.
Suggested correction
Replace the abstract sentence with one that matches Table 1, for example:
Furthermore, over the last three decades about 31% of the world’s muddy coasts eroded at rates exceeding 1 m/yr, 29% accreted at such rates and 40% were stable, with a small net global gain.
and the web summary with, for example:
14% of the world’s coastlines are muddy; about 60% of them changed by more than 1 m per year over the last three decades, with erosion and accretion roughly balanced, according to an automated classification method that identifies global coastlines.
Recomputation script (python recompute.py.txt)
"""Recompute the muddy-coast erosion share in PMC10716179
(Nat Commun 2023, "Global distribution and dynamics of muddy coasts",
doi:10.1038/s41467-023-43819-6).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PubMed Central (identical on the
publisher's version of record): Table 1 (muddy and sandy columns, % of coastline per
dynamic class), and the percentages given in the Results section
"Dynamics of muddy coasts vs. sandy coasts".
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs: Table 1
# (class, rate band, sandy %, muddy %), in the order printed in Table 1
TABLE1 = [
("Very extreme accretion", ">10 m/yr", 1.6, 8.8),
("Extreme accretion", "5 to 10 m/yr", 1.8, 5.4),
("Severe accretion", "3 to 5 m/yr", 2.3, 4.3),
("Intense accretion", "1 to 3 m/yr", 10.1, 10.7),
("Stable", "-1 to 1 m/yr", 69.7, 40.3),
("Intense erosion", "-1 to -3 m/yr", 9.8, 12.2),
("Severe erosion", "-3 to -5 m/yr", 1.9, 4.8),
("Extreme erosion", "-5 to -10 m/yr", 1.5, 5.7),
("Very extreme erosion", "<-10 m/yr", 1.2, 7.8),
]
# ------------------------------------------------------------------ inputs: Results text
TEXT_STABLE, TEXT_ERODING, TEXT_ACCRETING = 40, 31, 29 # Results, "Dynamics of muddy coasts vs. sandy coasts", para. 2 (%)
RETAINED = 97.7 # Results, same section, para. 1: % of mud coasts kept after filtering
MEAN_RATE = 0.18 # Results, same section, para. 2: global mean, m/yr (accretion)
NET_GAIN_KM2 = 579 # Results, same section, para. 2
ROUND_HALF = 0.05 # each Table 1 entry is printed to one decimal
# ------------------------------------------------------------------ recomputation
mud = [r[3] for r in TABLE1]
sand = [r[2] for r in TABLE1]
acc, stable, ero = sum(mud[:4]), mud[4], sum(mud[5:])
total = sum(mud)
dynamic = acc + ero
ero_of_dynamic = 100 * ero / dynamic
upper_bound = ero * RETAINED / 100 + (100 - RETAINED)
round_err = 4 * ROUND_HALF
s_acc, s_stable, s_ero = sum(sand[:4]), sand[4], sum(sand[5:])
print("Table 1, muddy coasts (% of muddy coastline)")
print(f" eroding >1 m/yr = {' + '.join(f'{v:.1f}' for v in mud[5:])} = {ero:.1f}% (Results text: {TEXT_ERODING}%)")
print(f" stable |r|<1 m/yr = {stable:.1f}% (Results text: {TEXT_STABLE}%)")
print(f" accreting >1 m/yr = {' + '.join(f'{v:.1f}' for v in mud[:4])} = {acc:.1f}% (Results text: {TEXT_ACCRETING}%)")
print(f" column total = {total:.1f}%")
print(f" rounding: four erosion entries at +/-{ROUND_HALF} each -> eroding share {ero - round_err:.1f}% to {ero + round_err:.1f}%")
print(f" eroding is a majority of all muddy coasts (>50%)? {ero > 50}")
print(f" eroding is the largest of the three classes? {ero > max(stable, acc)} (largest: stable, {stable:.1f}%)")
print(f" changing by >1 m/yr in either direction = {dynamic:.1f}%")
print(f" eroding among those changing by >1 m/yr only = {ero:.1f}/{dynamic:.1f} = {ero_of_dynamic:.1f}%")
print(f" shortfall of eroding share below 50% = {50 - ero:.1f} percentage points")
print(f" eroding or stable (not accreting) = {ero + stable:.1f}%")
print(f" accreting minus eroding = {acc - ero:+.1f} percentage points")
print(f" upper bound if all {100 - RETAINED:.1f}% filtered-out coast were eroding = "
f"{ero:.1f} x {RETAINED / 100:.3f} + {100 - RETAINED:.1f} = {upper_bound:.1f}%")
print(f" reported global mean rate {MEAN_RATE:+.2f} m/yr, net gain {NET_GAIN_KM2} km2 (sign: accretion)")
print("Table 1, sandy coasts (for comparison)")
print(f" eroding {s_ero:.1f}%, stable {s_stable:.1f}%, accreting {s_acc:.1f}%, total {sum(sand):.1f}%, changing > 1 m/yr {s_ero + s_acc:.1f}%")
# ------------------------------------------------------------------ figure style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def panel(ax, x, y, w, h, edge, title, lines):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=edge, lw=2, transform=ax.transAxes))
ax.text(x + 0.025, y + h - 0.05, title, transform=ax.transAxes, fontsize=11,
fontweight="bold", va="top", color=INK)
yy = y + h - 0.15
for txt, kw in lines:
style = dict(fontsize=10.5, color=INK, va="top")
style.update(kw)
ax.text(x + 0.025, yy, txt, transform=ax.transAxes, **style)
yy -= 0.072 * (txt.count("\n") + 1) + 0.035
# ------------------------------------------------------------------ fig1: abstract vs body
fig = plt.figure(figsize=(7.2, 3.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.axis("off")
L = (0.025, 0.25, 0.42, 0.72)
R = (0.555, 0.25, 0.42, 0.72)
panel(ax, *L, ORANGE, "Abstract and web summary", [
("“the majority of the world’s muddy\ncoasts are eroding at rates\nexceeding 1 m/yr over the last\nthree decades”", {}),
("Implied share eroding > 1 m/yr:", {"color": INK2, "fontsize": 10}),
("more than 50%", {"color": ORANGE, "fontweight": "bold", "fontsize": 12}),
])
panel(ax, *R, INK2, "Results and Table 1", [
("“40% … are stable …; 31% are\npersistently eroding, while 29%\nare accreting (Table 1)”", {}),
("Table 1 erosion classes summed:", {"color": INK2, "fontsize": 10}),
(f"{' + '.join(f'{v:.1f}' for v in mud[5:])} = {ero:.1f}%", {"color": BLUE, "fontweight": "bold", "fontsize": 12}),
(f"stable {stable:.1f}% accreting {acc:.1f}%", {"color": INK2, "fontsize": 10}),
])
ax.add_patch(FancyArrowPatch((L[0] + L[2], 0.585), (R[0], 0.585), transform=ax.transAxes,
arrowstyle="<->", mutation_scale=14, lw=2, color=INK))
ax.text(0.5, 0.63, "conflict", transform=ax.transAxes, ha="center", fontsize=10.5,
fontweight="bold", color=INK)
ax.text(0.5, 0.135,
f"Same quantity, share of all muddy coastline eroding > 1 m/yr (1984–2016): "
f"abstract > 50%, Table 1 {ero:.1f}%.",
transform=ax.transAxes, ha="center", fontsize=10, color=INK)
ax.text(0.5, 0.05,
"Orange: wording in the abstract and web summary. Blue: recomputed from the paper’s Table 1.",
transform=ax.transAxes, ha="center", fontsize=10, color=INK2)
save(fig, "fig1")
# ------------------------------------------------------------------ fig2: shares on one axis
rows = [
("Eroding > 1 m/yr\n(all muddy coast)", TEXT_ERODING, ero),
("Stable, |rate| < 1 m/yr", TEXT_STABLE, stable),
("Accreting > 1 m/yr", TEXT_ACCRETING, acc),
("Eroding, if every filtered-out\ntransect (2.3%) were eroding", None, upper_bound),
("Eroding, among coasts\nchanging > 1 m/yr only", None, ero_of_dynamic),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.15)
ys = list(range(len(rows)))[::-1]
ax.axvline(50, color=INK2, lw=1, zorder=1)
ax.text(49, -0.55, "50% (majority)", ha="right", va="bottom", fontsize=10, color=INK2)
# the abstract's claim: an open-ended range above 50% on the first row
y0 = ys[0]
ax.plot([50, 100], [y0, y0], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([50], [y0], marker="|", color=ORANGE, ms=14, mew=2, zorder=3)
ax.text(75, y0 + 0.22, "abstract: “the majority” (> 50%)", ha="center", va="bottom",
fontsize=10, color=INK)
for y, (lab, rep, rec) in zip(ys, rows):
if rep is not None:
ax.plot(rep, y, "o", ms=13, mfc="none", mec=ORANGE, mew=2, zorder=4)
ax.plot(rec, y, "o", ms=8, color=BLUE, zorder=5)
if abs(rec - 50) < 4:
ax.text(rec + 2.5, y, f"{rec:.1f}%", ha="left", va="center", fontsize=10, color=INK)
else:
ax.text(rec, y - 0.32, f"{rec:.1f}%", ha="center", va="top", fontsize=10, color=INK)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.set_xlim(0, 100)
ax.set_ylim(-0.7, len(rows) - 0.2)
ax.set_xticks(range(0, 101, 10))
ax.set_xlabel("Share of muddy coastline (%)")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "o", ms=13, mfc="none", mec=ORANGE, mew=2, ls="none")
h2, = ax.plot([], [], color=ORANGE, lw=2, marker="|", ms=12, mew=2)
h3, = ax.plot([], [], "o", ms=8, color=BLUE, ls="none")
fig.legend([h1, h2, h3], ["Reported in Results text (rounded)", "Reported in abstract",
"Recomputed from Table 1"],
loc="upper center", ncol=3, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.995),
handlelength=1.6, columnspacing=1.2)
save(fig, "fig2")
print("wrote fig1.svg, fig1.png, fig2.svg, fig2.png")
Reported nationwide energy, cost and CO2 savings are 1,000 times what the paper's inputs give
Nat Commun (2025) · doi:10.1038/s41467-025-64962-2 · PMC12575868
Summary
The paper simulates an electrochromic smart window in office buildings in all 50 U.S. states and reports an average HVAC saving of 26.72 MJ/m² against low-e glass. It then applies that saving to the 97 billion ft² of U.S. commercial floor space and reports, in the Abstract, Results and Discussion, a nationwide saving of 66.87 billion MWh, worth $2.67–7.35 trillion and avoiding 0.87–66.94 billion t of CO2. The paper's own two inputs give 26.72 MJ/m² × 9.01 × 10⁹ m² = 2.41 × 10¹¹ MJ, which is 66.9 million MWh (66.9 TWh). The reported figure is 999.8 times larger, and the cost and CO2 totals, which Supplementary Eqs. S10–S11 compute from it, are 1,000 times too large as well: they should be about $2.7–8.4 billion and 0.87–67 million t. These totals are the abstract's headline impact claim. The reported energy figure is about 16 times all U.S. utility-scale electricity generation, so no reading of the inputs makes it possible.
What the paper reports
Abstract:
Simulations indicate that a large-scale deployment across the U.S. can save 66.87 billion MWh, amounting to $7.35 trillion, and reduce CO2 emissions by 66.94 billion tons.
With its rapid switching, long-term durability, and scalability, this device presents significant economic and environmental advantages for real-world applications.
Results, section "HVAC energy-saving capabilities as building smart windows", paragraph 4 (the "54" after "2018" is the citation to reference 54, the EIA Commercial Buildings Energy Consumption Survey):
As shown in Table S7 (Supplementary Information), the average energy savings across U.S. states is calculated to be 26.72 MJ m-2. The U.S. Energy Information Administration (EIA) reported that the total commercial building area in the U.S. was about 97 billion square feet in 201854, allowing for a calculation of total energy savings across the country to be as high as 66.87 billion MWh.
The economic costs of producing this large amount of electricity using nuclear, coal, gas, photovoltaic, onshore, and offshore wind power generation systems are estimated to range from $2.67 trillion to $7.35 trillion (Fig. 7d and Table S8, Eq. S10, Supplementary Information).
Even clean energy sources, like nuclear, photovoltaic, and wind, produce 0.87 billion tons of CO2, while coal, a highly polluting energy source, has an astounding 66.94 billion tons of CO2 emissions.
It holds significant economic and environmental value, aiding in energy conservation, emission reduction, and achieving net-zero energy buildings globally.
Discussion, paragraph 1:
The smart window consistently displayed higher energy efficiency across all states in the U.S., with an average value of 26.72 MJ m⁻². If implemented nationwide in office buildings, it is estimated to save 66.87 billion MWh of electricity. Depending on the power generation methods, the savings would translate into electricity costs ranging from $2.67 trillion to $7.35 trillion and a reduction in carbon emissions of between 0.87 billion and 66.94 billion tons.
Methods, building energy simulation:
A year-long simulation of monthly building energy consumption was performed for Phoenix, Arizona, along with an annual analysis of building energy consumption across various climate zones in all 50 U.S. states to investigate regional impacts.
Supplementary Information (MOESM1), Table S8 and Eqs. S10–S11. The equations are C = LCOE × E / 10¹² (trillion USD) and T = f × E / 10¹² (billion t CO2), "where … LCOE = levelized cost of electricity (USD/MWh), f = CO2 emission factor (kg CO2/MWh), and E = electricity consumption (MWh)". Table S8 lists the following LCOE (USD/MWh), cost (trillion USD), emission factor (kg/MWh) and CO2 (billion t): Nuclear 110 / 7.35 / 13 / 0.87; Coal 105 / 7.02 / 1001 / 66.94; Gas CCGT 60 / 4.01 / 486 / 32.50; Solar PV 55 / 3.68 / 43 / 2.88; Wind onshore 40 / 2.67 / 13 / 0.87; Wind offshore 125 / 8.36 / 13 / 0.87. The source data file (MOESM3, sheet "Fig. 7d") gives the same costs and CO2 totals.
All quotes from the article were checked verbatim against the PMC-archived text. The supplementary passages were read in the publisher's MOESM1 file.
What the paper's own numbers show
Method. I took the 50 per-state savings from Supplementary Table S7 and checked their mean and that each equals the low-e value minus the device value. I converted the stated floor area with the exact factor 1 ft² = 0.09290304 m², multiplied, and converted with 1 MWh = 3,600 MJ. I then applied the paper's own Eqs. S10–S11 and the Table S8 LCOE and emission factors to the recomputed energy. For scale, the recomputed and reported totals are compared with two EIA figures: U.S. utility-scale generation in 2023 and total energy use in U.S. commercial buildings in 2018. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Mean annual HVAC saving, 50 states | Results; Discussion; Table S7 | 26.72 MJ/m² | 26.716 MJ/m² (mean of the 50 rows; each row = low-e − device) |
| U.S. commercial floor area | Results (ref. 54) | 97 billion ft² | 97 × 10⁹ × 0.09290304 = 9.0116 × 10⁹ m² |
| Nationwide annual saving, energy | — | — | 26.72 × 9.0116 × 10⁹ = 2.408 × 10¹¹ MJ |
| Nationwide annual saving, electricity | Abstract; Results; Discussion | 66.87 billion MWh (6.687 × 10¹⁰ MWh) | 6.689 × 10⁷ MWh = 66.89 million MWh (66.9 TWh) |
| Ratio reported / recomputed | — | — | 999.8 |
| Same total in kWh | — | — | 66.89 billion kWh |
| Per-area saving implied by 66.87 billion MWh | derived | — | 26,714 MJ/m², which is 489 times the largest single-state saving (54.60 MJ/m², Hawaii) and 12.8 times the largest total annual HVAC use in Table S7 (2,080.05 MJ/m², Alaska, low-e glass) |
| Cost of the saved electricity (Eq. S10) | Abstract; Results; Discussion; Table S8 | $2.67–7.35 trillion | $2.68–7.36 billion for the technologies in the stated range; $8.36 billion for offshore wind |
| CO2 associated with the saved electricity (Eq. S11) | Abstract; Results; Discussion; Table S8 | 0.87–66.94 billion t | 0.87–66.95 million t |
| Unit values implied by the text ranges | derived | $2.67–7.35 trillion; 0.87–66.94 billion t | $39.9–109.9 per MWh and 13.0–1,001 kg CO2/MWh, which match the Table S8 LCOE and emission factors. The cost and CO2 totals are therefore Eq. S10–S11 applied to 66.87 billion MWh, and carry the same factor of 1,000 |
| Offshore wind cost vs stated range (secondary) | Table S8 vs Results text | Text: "$2.67 trillion to $7.35 trillion", including offshore wind | Table S8 lists offshore wind at 8.36 (125 USD/MWh), above the stated upper end |
| Solar PV CO2 vs text (secondary) | Table S8 vs Results text | Text: "nuclear, photovoltaic, and wind, produce 0.87 billion tons" | Table S8 gives 2.88 for solar PV (43 kg/MWh) |
| Pre-revision values (peer review file, MOESM2, Table R1) | — | 24.439 MJ/m² → "61.1"; 26.72 MJ/m² → "66.87", in a row labelled "(MWh)" | 61.18 and 66.89 million MWh; the same ratio of about 1,000 (998.8 and 999.8) if the listed values are read as billion MWh, as in the paper |
| Reported total vs U.S. utility-scale generation, 2023 (EIA: 4,178 TWh) | external | 66,870 TWh | Reported = 16.0 times all U.S. generation; recomputed = 1.60% of it |
| Reported total vs energy used in U.S. commercial buildings, 2018 (EIA CBECS: 6.8 quadrillion Btu = 1,993 TWh) | external | 66,870 TWh | Reported = 33.6 times all commercial-building energy; recomputed = 3.36% of it |
The recomputed electricity figure (66.89 million MWh) and the reported one (66.87 billion MWh) agree in their leading digits and differ only in the multiplier. The small difference in the last digit is consistent with rounding of the mean saving or of the conversion factor (with the unrounded Table S7 mean, the recomputed value is 66.88 million MWh). Applying Eqs. S10–S11 to the reported E reproduces every Table S8 entry to within 0.01, so the supplement is internally consistent with the inflated E. The discrepancy enters at the conversion of the energy product to MWh.
Figure
Figure 1. The arithmetic chain behind the nationwide totals, using only numbers stated in the paper and its supplement. The mean per-area saving (Table S7) and the commercial floor area (ref. 54, converted to m²) give 2.408 × 10¹¹ MJ per year. The highlighted, dashed step is the conversion to MWh. At 3,600 MJ per MWh the total is 66.9 million MWh (blue, left). The paper reports 66.87 billion MWh (orange, right), which equals the same total expressed in kWh. The bottom row applies the paper's Eqs. S10–S11 and Table S8 factors. With the recomputed energy, the costs are $2.68–7.36 billion ($8.36 billion for offshore wind) and the CO2 is 0.87–66.95 million t. The paper reports the same digits in trillions of dollars and billions of tonnes.
Figure 2. Annual energy on a single logarithmic axis, in TWh (1 TWh = 1 million MWh). The orange dot is the paper's reported nationwide saving (66,870 TWh). The blue dot is the value recomputed from the paper's own inputs (66.9 TWh). The ratio between them is 999.8. The gray squares are external reference points from the U.S. EIA: all U.S. utility-scale electricity generation in 2023 (4,178 TWh) and all energy used in U.S. commercial buildings in 2018 (6.8 quadrillion Btu, 1,993 TWh). The recomputed saving is 3.4% of commercial-building energy use. The reported saving is 33.6 times that use and 16 times the country's entire electricity output.
Why it matters
The nationwide totals are the only quantitative statement of the device's real-world impact in the Abstract. The abstract's closing claim of "significant economic and environmental advantages" rests largely on them, and the Results ("substantial economic and environmental benefits") and Discussion repeat them. As printed, they overstate the energy, money and CO2 saved by three orders of magnitude. A reader of the abstract would take away savings of trillions of dollars and tens of billions of tonnes of CO2 per year. These figures are larger than the entire U.S. electricity system, and the CO2 figure is larger than recent estimates of total global annual emissions. The values the paper's inputs actually support are billions of dollars and tens of millions of tonnes. That is still a meaningful result, but it is a different claim. The correct values appear nowhere in the paper or its supplement, so a reader can recover them only by redoing the calculation. Table S8, Fig. 7d and the Fig. 7d source data carry the inflated values, so anyone reusing the supplement would inherit the error.
The following are not affected: the device chemistry and electrochromic measurements (switching times of 1.6 s and 0.8 s, 64.23% optical modulation, capacity, energy recovery of 44.4% and 31.0%, cycling stability), the outdoor temperature test, the SHGC values, and the per-area EnergyPlus results (Fig. 7a–c, Tables S6–S7, including the 26.72 MJ/m² average). The direction of the comparison with low-e glass is unchanged.
Explanations considered
- Multi-year or lifetime horizon. A factor of 1,000 would need 1,000 years of annual savings. The Methods describe annual simulations, and the paper states no lifetime, discounting or multi-year multiplier anywhere. This does not explain the numbers.
- Floor area read in m² instead of ft². 26.72 MJ/m² × 97 × 10⁹ m² = 720 million MWh, a factor of 10.8 rather than 1,000. This does not explain the numbers.
- Per window area rather than per floor area. In the simulated building (20 m × 10 m floor, twelve 3 m × 2 m windows), floor area is 200 m² and window area is 72 m². Whichever area the 26.72 MJ/m² refers to, re-basing changes the total by a factor of about 3 at most, and the national window area is smaller than the national floor area. This does not explain the numbers.
- Primary vs site energy, or thermal vs electrical energy. Source-to-site multipliers for electricity are about 3, not 1,000, and the paper treats the saving directly as electricity ("save 66.87 billion MWh of electricity"). This does not explain the numbers.
- A larger building stock. The paper uses commercial floor area (and the Discussion says "office buildings", a subset). Even adding all U.S. residential floor space would multiply the area by a small single-digit factor, not by 1,000. This does not explain the numbers.
- Per-area saving larger than stated. For 66.87 billion MWh the average saving would have to be 26,714 MJ/m². That is 12.8 times the largest total annual HVAC consumption of any state in Table S7, which would mean saving more energy than the buildings use. This does not explain the numbers.
- Text-conversion artifact in PMC. The figures are written out in words ("billion MWh", "trillion", "billion tons"). They are identical in the PMC text, the publisher's HTML and the publisher's PDF, and they are carried consistently into Table S8, Fig. 7d and the source data. This does not explain the numbers.
- A kWh value labelled as MWh. The reported 66.87 is numerically the recomputed total expressed in billions of kWh (66.89 billion kWh). The pre-revision values in the peer review file show nearly the same relationship (24.439 MJ/m² gives 61.18 billion kWh; the file lists "61.1", a ratio of 998.8). This matches the numbers exactly, but it is a description of where the factor of 1,000 enters, not a justification: the paper's Eqs. S10–S11 define E in MWh and use it as MWh.
Status
As of 2026-10-23:
- Version of record. I fetched the publisher's article page (https://www.nature.com/articles/s41467-025-64962-2, published 2025-10-30) and its 12-page PDF. Both carry "66.87 billion MWh", "$7.35 trillion", "66.94 billion tons", "26.72 MJ m−2", "97 billion square feet" and "$2.67 trillion to $7.35 trillion", identical to the PMC-archived text. No correction or amendment notice appears on the page.
- Supplements. All three electronic supplementary files from the publisher were read. MOESM1 (Supplementary Information): Table S7 (50 states; mean saving 26.716 MJ/m²), Table S8, and Eqs. S10–S11 with E defined in MWh. MOESM3 (source data): sheets "Fig. 7c" (the same 50 savings) and "Fig. 7d" (the same costs in trillions and CO2 in billions of tonnes as Table S8). MOESM2 (peer review file): the added formulas (R10–R11) and a table of pre- and post-revision values ("Total energy savings across U.S. states (MWh)": 61.1 and 66.87). None of these contains a multiplier or definition that reconciles the inputs with the reported total, and none contains the correctly scaled value.
- Corrections. No correction, erratum or retraction was found. Sources: Crossref metadata for the DOI (no update-to, updated-by or relation entries), the Crossref
filter=updates:query (0 results), PMC and Europe PMC notices (none;src/known.py), and the publisher's article page (none). - PubPeer. The PubPeer search API returned a human-verification challenge and could not be queried directly. A web search for the title together with "pubpeer", "correction" or "erratum" found no thread or notice about this article.
- External reference values (used only for scale): EIA, "What is U.S. electricity generation by energy source?" (https://www.eia.gov/tools/faqs/faq.php?id=427), "about 4,178 billion kilowatthours" of utility-scale generation in 2023 (preliminary); EIA CBECS (https://www.eia.gov/consumption/commercial/), U.S. commercial buildings "consumed 6.8 quadrillion British thermal units of energy" in 2018.
Suggested correction
Abstract. Replace "can save 66.87 billion MWh, amounting to $7.35 trillion, and reduce CO2 emissions by 66.94 billion tons" with:
can save 66.9 million MWh (66.9 TWh) per year, amounting to up to $7.36 billion, and reduce CO2 emissions by up to 66.95 million tons.
Results, paragraph 4. Replace "as high as 66.87 billion MWh" with "as high as 66.9 million MWh (66.9 TWh)". Replace "from $2.67 trillion to $7.35 trillion" with "from $2.68 billion to $8.36 billion". This upper end includes offshore wind, which the sentence lists; alternatively, offshore wind can be removed from the list and the range given as $2.68–7.36 billion. Replace "0.87 billion tons" and "66.94 billion tons" with "0.87 million tons" and "66.95 million tons". For solar PV, the Table S8 factor gives 2.88 million tons, so "photovoltaic" should be dropped from the 0.87 group or given its own value.
Discussion, paragraph 1. Make the same replacements: 66.9 million MWh; $2.68–8.36 billion (or $2.68–7.36 billion); 0.87–66.95 million tons.
Supplementary Table S8 and Fig. 7d (and source data sheet "Fig. 7d"). Relabel the cost column "billions, USD" and the CO2 column "million tons" (the digits are correct for E = 66.9 million MWh to within rounding). Alternatively, keep the trillion/billion headings and divide every entry by 1,000. In Eqs. S10–S11, the stated units then become C in billion USD = LCOE × E / 10⁹ and T in million t = f × E / 10⁹, with E = 6.69 × 10⁷ MWh.
Recomputation script (python recompute.py.txt)
"""Recompute the nationwide energy, cost and CO2 totals in PMC12575868
(Nat Commun 2025; doi:10.1038/s41467-025-64962-2) from the paper's own stated inputs,
and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy and matplotlib)
Inputs are copied from the published article (Abstract, Results "HVAC energy-saving
capabilities as building smart windows", Discussion) and from Supplementary Information
Tables S7 and S8 and Eqs. S10-S11 (MOESM1_ESM.docx). Two external reference values, used
only for the plausibility comparison, are from the U.S. EIA and are labelled as such.
"""
import os
import numpy as np
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs from the paper
# Supplementary Table S7: annual HVAC energy, MJ per m2 (EESD, low-e glass, saving), 50 states
S7 = {
"Alabama": (274.50, 308.85, 34.35), "Alaska": (2074.00, 2080.05, 6.05),
"Arizona": (346.10, 389.50, 43.40), "Arkansas": (361.15, 395.80, 34.65),
"California": (70.80, 92.85, 22.05), "Colorado": (244.25, 267.50, 23.25),
"Connecticut": (367.25, 384.80, 17.55), "Delaware": (281.10, 303.15, 22.05),
"Florida": (448.30, 501.50, 53.20), "Georgia": (250.80, 280.20, 29.40),
"Hawaii": (398.85, 453.45, 54.60), "Idaho": (228.95, 252.00, 23.05),
"Illinois": (411.15, 434.10, 22.95), "Indiana": (373.80, 392.10, 18.30),
"Iowa": (502.75, 529.10, 26.35), "Kansas": (387.00, 420.15, 33.15),
"Kentucky": (304.25, 329.65, 25.40), "Louisiana": (370.35, 412.60, 42.25),
"Maine": (394.90, 410.05, 15.15), "Maryland": (292.85, 318.60, 25.75),
"Massachusetts": (260.90, 278.60, 17.70), "Michigan": (387.80, 409.40, 21.60),
"Minnesota": (595.05, 615.90, 20.85), "Mississippi": (330.80, 368.10, 37.30),
"Missouri": (413.10, 444.00, 30.90), "Montana": (293.85, 308.80, 14.95),
"Nebraska": (304.70, 328.10, 23.40), "Nevada": (304.20, 351.50, 47.30),
"New Hampshire": (413.55, 431.10, 17.55), "New Jersey": (244.40, 268.90, 24.50),
"New Mexico": (183.40, 212.00, 28.60), "New York": (258.55, 284.50, 25.95),
"North Carolina": (252.50, 283.75, 31.25), "North Dakota": (812.25, 828.90, 16.65),
"Ohio": (335.35, 359.35, 24.00), "Oklahoma": (360.75, 398.90, 38.15),
"Oregon": (150.15, 167.40, 17.25), "Pennsylvania": (289.55, 309.30, 19.75),
"Rhode Island": (185.30, 203.15, 17.85), "South Carolina": (323.55, 363.95, 40.40),
"South Dakota": (623.05, 645.75, 22.70), "Tennessee": (308.10, 345.40, 37.30),
"Texas": (358.35, 400.40, 42.05), "Utah": (215.55, 240.00, 24.45),
"Vermont": (469.00, 486.40, 17.40), "Virginia": (283.25, 315.90, 32.65),
"Washington": (124.90, 140.75, 15.85), "West Virginia": (307.15, 328.20, 21.05),
"Wisconsin": (467.85, 487.15, 19.30), "Wyoming": (381.80, 396.05, 14.25),
}
SAVING_REPORTED = 26.72 # MJ/m2, Results and Discussion
AREA_FT2 = 97e9 # "about 97 billion square feet in 2018" (EIA CBECS, ref 54)
E_REPORTED_MWH = 66.87e9 # "66.87 billion MWh" (Abstract, Results, Discussion)
# Supplementary Table S8: LCOE (USD/MWh), cost (trillion USD), f (kg CO2/MWh), CO2 (billion t)
S8 = {
"Nuclear": (110, 7.35, 13, 0.87),
"Coal": (105, 7.02, 1001, 66.94),
"Gas CCGT": (60, 4.01, 486, 32.50),
"Solar PV": (55, 3.68, 43, 2.88),
"Wind onshore": (40, 2.67, 13, 0.87),
"Wind offshore": (125, 8.36, 13, 0.87),
}
TEXT_COST_RANGE = (2.67, 7.35) # trillion USD, Abstract/Results/Discussion
TEXT_CO2_RANGE = (0.87, 66.94) # billion t, Results/Discussion
# Peer review file (MOESM2), pre-revision vs revised values (Table R1)
PRF = {"before": (24.439, 61.1), "after": (26.72, 66.87)} # (MJ/m2, "Total energy savings ... (MWh)")
# ---------------------------------------------------------------- unit constants
FT2_TO_M2 = 0.09290304 # exact
MJ_PER_MWH = 3600.0 # exact
MJ_PER_KWH = 3.6 # exact
J_PER_BTU = 1055.05585 # International Table Btu
# ---------------------------------------------------------------- external references (plausibility only)
US_GEN_2023_TWH = 4178.0 # EIA FAQ id=427: "about 4,178 billion kilowatthours" utility-scale, 2023 (preliminary)
CBECS_2018_QUADS = 6.8 # EIA CBECS 2018: commercial buildings "consumed 6.8 quadrillion British thermal units"
print("=== 1. Mean saving across 50 states (Table S7) ===")
names = list(S7)
eesd = np.array([S7[k][0] for k in names])
lowe = np.array([S7[k][1] for k in names])
sav = np.array([S7[k][2] for k in names])
print(f"n = {len(sav)}; mean saving = {sav.mean():.4f} MJ/m2 (reported {SAVING_REPORTED})")
print(f"every saving = low-e minus EESD: {np.allclose(lowe - eesd, sav, atol=0.006)}")
print(f"savings range {sav.min():.2f}-{sav.max():.2f} MJ/m2; "
f"total HVAC range: EESD {eesd.min():.2f}-{eesd.max():.2f}, low-e {lowe.min():.2f}-{lowe.max():.2f} MJ/m2")
print("\n=== 2. Nationwide energy saving from the paper's inputs ===")
area_m2 = AREA_FT2 * FT2_TO_M2
E_MJ = SAVING_REPORTED * area_m2
E_MWH = E_MJ / MJ_PER_MWH
E_KWH = E_MJ / MJ_PER_KWH
print(f"area = 97e9 ft2 x {FT2_TO_M2} = {area_m2:.4e} m2 ({area_m2/1e9:.2f} billion m2)")
print(f"E = {SAVING_REPORTED} MJ/m2 x {area_m2:.4e} m2 = {E_MJ:.4e} MJ")
print(f" = {E_MWH:.4e} MWh = {E_MWH/1e6:.2f} million MWh = {E_MWH/1e6:.2f} TWh")
print(f" = {E_KWH:.4e} kWh = {E_KWH/1e9:.2f} billion kWh")
E_MWH_mean = sav.mean() * area_m2 / MJ_PER_MWH
print(f"with the unrounded Table S7 mean ({sav.mean():.4f}): {E_MWH_mean/1e6:.2f} million MWh")
ratio = E_REPORTED_MWH / E_MWH
print(f"reported 66.87e9 MWh / recomputed = {ratio:.1f}")
implied = E_REPORTED_MWH * MJ_PER_MWH / area_m2
print(f"per-area saving implied by 66.87e9 MWh = {implied:,.0f} MJ/m2 ({implied/1000:.2f} GJ/m2)")
print(f" = {implied/lowe.max():.1f} x the largest total annual HVAC use in Table S7 "
f"({lowe.max():.2f} MJ/m2, low-e glass, Alaska)")
print(f" = {implied/sav.max():.0f} x the largest single-state saving ({sav.max():.2f} MJ/m2)")
print(f"if 97e9 were read as m2 rather than ft2: {SAVING_REPORTED*97e9/MJ_PER_MWH/1e6:.0f} million MWh "
f"(factor {SAVING_REPORTED*97e9/MJ_PER_MWH/E_MWH:.2f}, not 1000)")
print(f"years of savings needed to reach 66.87e9 MWh: {ratio:.0f}")
print("\n=== 3. Peer review file (pre-revision values) ===")
for k, (s, rep) in PRF.items():
e = s * area_m2 / MJ_PER_MWH
print(f"{k:6s}: {s} MJ/m2 -> {e/1e6:.2f} million MWh; table lists {rep} -> ratio "
f"{rep*1e9/e:.1f} if read as billion MWh")
print("\n=== 4. Cost and CO2 (Table S8; Eq. S10 C = LCOE*E/1e12, Eq. S11 T = f*E/1e12) ===")
print(f"{'Technology':14s} {'LCOE':>5s} {'S8 cost':>8s} {'LCOE*Erep':>10s} {'cost w/ E':>10s} "
f"{'f':>5s} {'S8 CO2':>7s} {'f*Erep':>7s} {'CO2 w/ E':>9s}")
print(f"{'':14s} {'$/MWh':>5s} {'T$':>8s} {'T$':>10s} {'B$':>10s} {'kg/MWh':>5s} {'Bt':>7s} {'Bt':>7s} {'Mt':>9s}")
costs_true, co2_true = {}, {}
for k, (l, c, f, t) in S8.items():
costs_true[k] = l * E_MWH / 1e9 # billion USD
co2_true[k] = f * E_MWH / 1e9 # kg -> million t
print(f"{k:14s} {l:5d} {c:8.2f} {l*E_REPORTED_MWH/1e12:10.2f} {costs_true[k]:10.2f} "
f"{f:5d} {t:7.2f} {f*E_REPORTED_MWH/1e12:7.2f} {co2_true[k]:9.2f}")
ct = sorted(costs_true.values())
print(f"recomputed cost range: ${ct[0]:.2f}-{ct[-1]:.2f} billion "
f"(without offshore wind: ${ct[0]:.2f}-{sorted(ct)[-2]:.2f} billion); reported "
f"${TEXT_COST_RANGE[0]}-{TEXT_COST_RANGE[1]} trillion")
co = sorted(co2_true.values())
print(f"recomputed CO2 range: {co[0]:.2f}-{co[-1]:.2f} million t; reported "
f"{TEXT_CO2_RANGE[0]}-{TEXT_CO2_RANGE[1]} billion t")
print(f"implied $/MWh from the text range: {TEXT_COST_RANGE[0]*1e12/E_REPORTED_MWH:.1f} to "
f"{TEXT_COST_RANGE[1]*1e12/E_REPORTED_MWH:.1f}")
print(f"implied kg CO2/MWh from the text range: {TEXT_CO2_RANGE[0]*1e12/E_REPORTED_MWH:.1f} to "
f"{TEXT_CO2_RANGE[1]*1e12/E_REPORTED_MWH:.1f}")
print(f"Table S8 offshore wind cost {S8['Wind offshore'][1]} T$ lies outside the text range "
f"{TEXT_COST_RANGE}: {S8['Wind offshore'][1] > TEXT_COST_RANGE[1]}")
print(f"Table S8 solar PV CO2 {S8['Solar PV'][3]} Bt vs text 'nuclear, photovoltaic, and wind ... 0.87'")
print("\n=== 5. Plausibility against U.S. totals (EIA) ===")
cbecs_twh = CBECS_2018_QUADS * 1e15 * J_PER_BTU / 3.6e15
rep_twh, true_twh = E_REPORTED_MWH / 1e6, E_MWH / 1e6
print(f"reported {rep_twh:,.0f} TWh; recomputed {true_twh:.1f} TWh")
print(f"US utility-scale generation 2023 = {US_GEN_2023_TWH:,.0f} TWh -> reported is "
f"{rep_twh/US_GEN_2023_TWH:.1f}x; recomputed is {100*true_twh/US_GEN_2023_TWH:.2f}%")
print(f"US commercial-building energy 2018 = {CBECS_2018_QUADS} quads = {cbecs_twh:,.0f} TWh -> reported is "
f"{rep_twh/cbecs_twh:.1f}x; recomputed is {100*true_twh/cbecs_twh:.2f}%")
# ================================================================ figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
def box(ax, x, y, w, h, title, body, edge=SEC, fill="white", tcol=TXT, ls="-", lw=1.2):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc=fill, ec=edge, lw=lw, ls=ls))
ax.text(x + w / 2, y + h - 1.6, title, ha="center", va="top", fontsize=10, color=tcol, weight="bold")
ax.text(x + w / 2, y + h - 5.2, body, ha="center", va="top", fontsize=10, color=TXT, linespacing=1.35)
def arrow(ax, p, q, col=SEC):
ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=13, lw=1.5, color=col,
shrinkA=0, shrinkB=0))
def sci(x):
e = int(np.floor(np.log10(x)))
return f"{x/10**e:.3f} × 10$^{{{e}}}$"
# ---------------------------------------------------------------- fig1: calculation chain
fig = plt.figure(figsize=(7.2, 6.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 100); ax.axis("off")
ax.set_facecolor(BG)
box(ax, 2, 85, 43, 12, "Mean saving per m² (Table S7)",
f"mean of 50 states = {sav.mean():.3f} MJ/m²\n(paper: {SAVING_REPORTED} MJ/m²)")
box(ax, 55, 85, 43, 12, "U.S. commercial floor area (ref. 54)",
f"97 × 10$^{{9}}$ ft² × {FT2_TO_M2:.4f}\n= {area_m2/1e9:.2f} × 10$^{{9}}$ m²")
arrow(ax, (23.5, 84.2), (40, 77.6)); arrow(ax, (76.5, 84.2), (60, 77.6))
box(ax, 22, 65, 56, 12, "Annual saving in energy units",
f"{SAVING_REPORTED} MJ/m² × {area_m2/1e9:.2f} × 10$^{{9}}$ m² = {sci(E_MJ)} MJ")
arrow(ax, (50, 64.2), (50, 58.4))
# highlighted divergence step
ax.add_patch(FancyBboxPatch((6, 45), 88, 13, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="#fdf0ea", ec=ORANGE, lw=1.6, ls=(0, (4, 2))))
ax.text(50, 56.4, "Step where the figures diverge: MJ → MWh", ha="center", va="top",
fontsize=10.5, weight="bold", color=TXT)
ax.text(50, 52.4, f"1 MWh = 3,600 MJ, so {sci(E_MJ)} MJ = {sci(E_MWH)} MWh.\n"
f"The paper's figure equals the same total in kWh ({E_KWH/1e9:.2f} billion kWh), stated as MWh.",
ha="center", va="top", fontsize=10, color=TXT, linespacing=1.35)
arrow(ax, (28, 44.2), (26, 37.6), BLUE); arrow(ax, (72, 44.2), (74, 37.6), ORANGE)
box(ax, 2, 25, 46, 12.5, "Recomputed from the paper's inputs",
f"{E_MWH/1e6:.1f} million MWh\n({E_MWH/1e6:.1f} TWh per year)", edge=BLUE, tcol=BLUE, lw=2)
box(ax, 52, 25, 46, 12.5, "Reported (Abstract, Results, Discussion)",
f"66.87 billion MWh\n(ratio to recomputed: {ratio:.1f})", edge=ORANGE, tcol=ORANGE, lw=2)
arrow(ax, (25, 24.2), (25, 18.2), BLUE); arrow(ax, (75, 24.2), (75, 18.2), ORANGE)
box(ax, 2, 3, 46, 15, "Eqs. S10–S11 with this E",
f"cost ${ct[0]:.2f}–{sorted(ct)[-2]:.2f} billion\n(offshore wind ${ct[-1]:.2f} billion)\n"
f"CO$_2$ {co[0]:.2f}–{co[-1]:.2f} million t", edge=BLUE, tcol=BLUE, lw=2)
box(ax, 52, 3, 46, 15, "Reported (Table S8 and text)",
f"cost ${TEXT_COST_RANGE[0]}–{TEXT_COST_RANGE[1]} trillion\n(Table S8 offshore wind 8.36 trillion)\n"
f"CO$_2$ {TEXT_CO2_RANGE[0]}–{TEXT_CO2_RANGE[1]} billion t", edge=ORANGE, tcol=ORANGE, lw=2)
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: scale on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.3), facecolor=BG)
ax.set_facecolor(BG)
rows = [
("Paper: nationwide saving", rep_twh, ORANGE, "o", f"{rep_twh:,.0f} TWh (66.87 billion MWh)"),
("Recomputed from the\npaper's own inputs", true_twh, BLUE, "o", f"{true_twh:.1f} TWh (66.9 million MWh)"),
("All U.S. utility-scale\nelectricity, 2023 (EIA)", US_GEN_2023_TWH, SEC, "s", f"{US_GEN_2023_TWH:,.0f} TWh"),
("All energy used in U.S.\ncommercial buildings,\n2018 (EIA CBECS)", cbecs_twh, SEC, "s",
f"{cbecs_twh:,.0f} TWh (6.8 quadrillion Btu)"),
]
ys = np.arange(len(rows))[::-1]
for (lab, v, c, m, txt), y in zip(rows, ys):
ax.plot([v], [y], marker=m, ms=10, color=c, mec=c, ls="none", zorder=3)
if v > 2e4:
ax.text(v / 1.35, y, txt, ha="right", va="center", fontsize=10, color=c)
else:
ax.text(v * 1.35, y, txt, ha="left", va="center", fontsize=10, color=c)
ax.set_xscale("log")
ax.set_xlim(10, 3e5)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=TXT)
ax.set_xlabel("Annual energy, TWh (log scale; 1 TWh = 1 million MWh)", fontsize=10)
ax.grid(axis="x", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.annotate("", xy=(true_twh, ys[1] + 0.5), xytext=(rep_twh, ys[1] + 0.5),
arrowprops=dict(arrowstyle="<->", color=SEC, lw=1.2))
ax.text(np.sqrt(rep_twh * true_twh), ys[1] + 0.42, f"ratio {ratio:.1f}", ha="center", va="top",
fontsize=10, color=SEC)
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png")
Abstract calls a non-significant hemorrhage timing slope significant; the Results report a different, reproducible slope
Neurology (2025) · doi:10.1212/WNL.0000000000214184 · PMC12473174
Summary
This meta-analysis of nine observational studies (435 patients with mechanical heart valves who resumed anticoagulation after intracranial hemorrhage) uses a meta-regression on days off anticoagulation to time the restart, and its Abstract calls the inverse association with recurrent hemorrhage "significant" although the Abstract's own figures (coefficient −0.039, 95% CI −0.093 to 0.015, p = 0.13) are not significant. The Results report a different coefficient for the same analysis (−0.1094, SE 0.0517, p = 0.0345), and refitting the model from the study-level events in Figure 3A and the days off anticoagulation in Table 1 reproduces that value exactly, while no subset of studies or model variant reproduces the Abstract's coefficient with its interval. The Abstract's "50% reduction at 11 days" fits neither coefficient, and its ischemic stroke coefficient has the opposite sign to the one in the Results. The timing result is the paper's stated novelty and the basis of its 10–15 day recommendation, so the Abstract misstates the paper's central finding.
What the paper reports
The quotations are from the article as archived in PubMed Central. The archive prints minus signs as hyphens; the publisher's PDF prints the same numbers with true minus signs.
Abstract, Results:
Meta-regression demonstrated a significant inverse association between time to AC resumption and risk of recurrent ICH (regression coefficient -0.039; 95% CI -0.093 to 0.015; p = 0.13), corresponding to an approximate 50% relative reduction in risk at 11 days after ICH.
No significant time-dependent association was observed for ischemic stroke (coefficient -0.013; 95% CI -0.065 to 0.039; p = 0.61).
The pooled incidence of recurrent ICH after AC reinitiation was 11.4% (95% CI 8.2–15.6; I2 = 0%)
Abstract, Discussion:
Meta-regression suggested a lower risk of recurrent ICH with later AC resumption, with a potential risk reduction at approximately 11 days after ICH.
Methods, Statistical Analysis (the only meta-regression the paper describes):
To investigate whether the timing of VKA reinitiation influenced the risk of ischemic stroke and ICH, we performed meta-regression analyses using the number of days off AC as a continuous moderator.
Studies with zero events were handled by applying a continuity correction of 0.5 to all cells.
Results, "Recurrence of ICH After VKA Reinitiation Following ICH":
Between-study heterogeneity was low, with τ2 of 0.0365, τ of 0.1911, and I2 of 11.9% (95% CI 0.0%–53.4%).
Meta-regression analysis revealed a significant inverse association between the number of days off AC and the risk of recurrent ICH. The regression coefficient was -0.1094 (SE = 0.0517; p = 0.0345), indicating that each additional day without AC was associated with a statistically significant reduction in the log-odds of recurrence. The model explained all observed heterogeneity, and the test for residual heterogeneity was nonsignificant (p = 0.708) (Figure 3, panel B).
Using the regression model, we identified that a 25% relative reduction in risk occurred at approximately 3.4 days, a 50% reduction at 7.9 days, and a 75% reduction at 14.9 days after AC resumption.
Results, "Ischemic Stroke Before VKA Reinitiation After ICH":
Meta-regression revealed no significant association between the number of days off AC and the risk of AIS (coefficient: 0.0388, SE = 0.0765, p = 0.6123) (Figure 2, panel B).
The model predicted that a 25% increase in risk (to 5.0%) would occur after approximately 6.0 days, a 50% increase (to 6.0%) after 11.0 days, and a 75% increase (to 7.0%) after 15.2 days of AC interruption.
Results, "Time-Dependent Risk Modeling of AIS and ICH After AC Interruption":
The estimated baseline risk of ICH at day 0 was 20%, which declined progressively with longer AC interruption. A 25% risk reduction in ICH occurred at approximately 6 days, 50% at 7.9 days, and 75% at 3.4 days.
Figure 3 legend, panel C:
Key thresholds indicate when the risk drops to 75% (∼3.4 days), 50% (∼7.9 days), and 25% (∼14.9 days) of the initial value.
Discussion:
Our results suggest that the risk of ICH recurrence decreases significantly with longer AC interruption. Specifically, a 50% reduction in risk is expected at approximately 7.9 days, with a 25% and 75% reduction at 3.4 and 14.9 days, respectively.
These opposing temporal trajectories underscore a potential “window of opportunity” between 10 and 15 days after ICH where the balance of hemorrhagic and thrombotic risk may be most favorable.
In conclusion, our analysis suggests that a delay of approximately 10–15 days before resuming AC may optimize the balance between hemorrhagic and thromboembolic risk in patients with MHVs after ICH.
What the paper's own numbers show
Method. (1) For each printed coefficient with a 95% CI, I took the SE as (upper − lower) / (2 × 1.96) and the two-sided Wald p as 2·(1 − Φ(|b/SE|)). (2) I refitted the meta-regression myself. Events and totals per study come from the forest plots (Figure 3A for recurrent ICH, Figure 2A for ischemic stroke). The moderator is the "Days AC discontinued" column of Table 1. The model is an inverse-variance regression of logit proportions on days, with 0.5 added to the cells of zero-event studies, and τ² estimated by DerSimonian–Laird and by REML. As checks I also added 0.5 to every study, fitted a binomial logistic regression with no correction, dropped each study in turn, and searched every subset of three to nine studies. The inputs reproduce the paper's totals: 435 patients, 45 ICH events and 21 ischemic strokes. (3) The days to a given relative change in risk come from logit p(t) = logit p₀ + b·t, where p₀ is the day-0 risk from the model intercept. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| ICH slope: CI vs "significant" | Abstract, Results | −0.039 (95% CI −0.093 to 0.015), p = 0.13, "significant" | CI contains 0; implied SE 0.0276, z = −1.42, p = 0.157 (one-sided 0.078). Not significant at 0.05, nor at the paper's 0.10 threshold for the Q test |
| ICH slope, SE, p | Results | −0.1094, SE 0.0517, p = 0.0345 | z = −2.12, p = 0.0343, 95% CI −0.211 to −0.008 |
| ICH slope refitted from Fig 3A + Table 1 | derived | — | −0.1094, SE 0.0517, p = 0.0345, τ² = 0 (DL and REML), residual QE = 4.61, p = 0.708: matches the Results exactly |
| ICH slope, other variants | derived | — | 0.5 added to all studies −0.1141 (p = 0.022); binomial GLM −0.1055 (p = 0.038); leave-one-out range −0.1215 to −0.0610 |
| ICH slope SE that the data allow | derived | Abstract implies 0.0276 | smallest SE over every subset of 3–9 studies = 0.0517; dropping studies can only widen the interval, so no subset gives the Abstract's interval |
| Abstract vs Results ICH slope | Abstract vs Results | −0.039 vs −0.1094 | ratio 2.81; SE ratio 1.88. Not a weeks/days rescaling (that would give a factor of 7, −0.766) |
| Days to 25/50/75% ICH reduction, b = −0.1094 | Results; Discussion; Fig 3C legend | 3.4 / 7.9 / 14.9 | 3.43 / 7.87 / 14.88 (day-0 risk 0.267 from the intercept) |
| Days to 50% ICH reduction | Abstract (Results and Discussion) | ~11 | 7.9 with b = −0.1094; 22.1 with b = −0.039 (17.8 on a log-risk scale). An 11-day halving would need b ≈ −0.078 |
| Same thresholds, second statement | Results, Time-Dependent Risk Modeling | 6 / 7.9 / 3.4 (25 / 50 / 75%) | the 75% value precedes the 25% value; the ICH section gives 3.4 / 7.9 / 14.9 |
| ICH day-0 risk | Results, Time-Dependent Risk Modeling | 20% | model intercept gives 26.7%; with 20%, b = −0.1094 gives 3.18 / 7.41 / 14.24 days, not 3.4 / 7.9 / 14.9 |
| Ischemic stroke slope | Abstract vs Results | Abstract −0.013 (CI −0.065 to 0.039), p = 0.61; Results +0.0388, SE 0.0765, p = 0.6123 | opposite signs; Abstract p from CI 0.624, Results p 0.612; the Results SE is 2.9 times the Abstract's |
| Ischemic stroke slope refitted from Fig 2A + Table 1 | derived | — | +0.0198 (p = 0.79); 0.5 added to all studies +0.0251; GLM +0.1139. Not +0.0388 exactly, but every variant is positive |
| Days to 25/50/75% stroke increase | Results | 6.0 / 11.0 / 15.2 (baseline 4%) | 6.02 / 10.99 / 15.24 with b = +0.0388. With the Abstract's −0.013 the risk never rises |
| Recurrent ICH I² | Abstract vs Results, Fig 3A | 0% vs 11.9% | not recomputable from the printed data; the two statements conflict |
The Results coefficient is reproduced to four decimals from the paper's own study-level data, together with its SE, its p value and the residual-heterogeneity p value. Its threshold days also follow from it. The Abstract's coefficient sits at the midpoint of its own interval, so the Abstract figures agree with one another. But they cannot come from these nine studies under the model the Methods describe, because the interval is about half as wide as the full dataset allows. Taken on their own terms they also describe a non-significant association, which the Abstract calls significant.
Figure
Figure 1. Slope of the meta-regression of event risk (log-odds) on days off anticoagulation, with 95% CIs, for recurrent ICH (top) and ischemic stroke (bottom). Orange circles are values printed in the paper. For the Abstract the interval is the printed one; for the Results it is ±1.96 × the printed SE. Blue squares are refits from the events and totals in Figures 2A and 3A and the days in Table 1. They use a logit inverse-variance meta-regression with 0.5 added to zero-event studies (the version that reproduces the Results) or to every study, or a binomial logistic regression. The vertical line marks a slope of 0 (no association). For recurrent ICH, the Abstract's interval crosses 0 although the Abstract calls the association significant. Every refit lies near the Results value of −0.1094, and none lies near −0.039. For ischemic stroke the Abstract's slope is negative while the Results' slope and the refit are positive.
Figure 2. Days off anticoagulation at which the modeled recurrent-ICH risk has fallen by 25% (open circle), 50% (filled square) and 75% (open diamond). Orange rows are printed in the paper: the Abstract gives only a 50% value (about 11 days, dashed line). The Results' ICH section gives 3.4 / 7.9 / 14.9, and the Results' risk-modeling section gives 6 / 7.9 / 3.4. Blue rows are computed from the logit model with the day-0 risk of 26.7% given by the refitted intercept, once with the Results slope (−0.1094) and once with the Abstract slope (−0.039). The Results slope reproduces 3.4 / 7.9 / 14.9. The Abstract slope gives 9.6 / 22.1 / 41.8. Neither puts the 50% reduction at 11 days.
Why it matters
The paper presents the meta-regression of risk on time off anticoagulation as its "key element of novelty". The 25/50/75% thresholds derived from it support the Discussion's "window of opportunity" and the final recommendation of a delay of about 10–15 days. The Abstract is the part most readers and indexing services see. It reports this result with a coefficient, interval and p value that show no significant association, calls the association significant, and attaches a time point (11 days) that follows from neither coefficient. A reader of the Abstract cannot tell whether the timing effect is supported. Its reported numbers say it is not, while the body says it is, with a 50% reduction at 7.9 days rather than 11.
The body's estimate is reproducible from the published study-level data, so the Abstract, not the Results, appears to misreport the analysis. Two further points matter for interpretation. They are context, not errors. First, the significance of the reproduced slope (p = 0.034) rests on nine study-level points, and dropping either the 165-patient cohort or the 66-patient cohort removes it (p = 0.36 and p = 0.19). The leave-one-out analysis that the paper reports covers the pooled proportions, not the meta-regression. Second, the moderator mixes means and medians of days off anticoagulation, a limitation the Methods acknowledge.
The ischemic stroke mismatch (−0.013 in the Abstract, +0.0388 in the Results) does not change the paper's conclusion that there is no significant association. It does change direction, though, and only the positive slope produces the rising stroke-risk curve and the 6.0 / 11.0 / 15.2 day thresholds that the window argument uses.
Explanations considered
- A different model in the Abstract (adjusted, subgroup or sensitivity analysis). The Methods describe one meta-regression per outcome with days off anticoagulation as the only moderator. No other model, adjustment or subgroup is described, and the Abstract does not label its numbers as coming from a different analysis. This explanation is not supported by the text.
- A different continuity correction, τ² estimator or link. I refitted with 0.5 added to zero-event studies only (−0.1094, the Results value), with 0.5 added to every study (−0.1141), and as a binomial logistic regression with no correction (−0.1055). τ² was estimated as 0 by both DerSimonian–Laird and REML, so the random-effects and fixed-effect slopes coincide. None of these variants is near −0.039. It does not account for the numbers.
- A smaller or earlier set of studies. The Abstract gives the search end as "December 2023" and the Methods give "22 April 2025", so the Abstract may describe a different dataset. Leaving out any single study gives slopes from −0.1215 to −0.0610. Some subsets of four to six studies give a slope near −0.039, but with SEs of 0.073 to 0.107. Removing studies cannot narrow the interval, and the smallest SE in any subset is 0.0517, against the 0.0276 that the Abstract's interval implies. The Abstract's interval cannot be obtained from these studies with this model. Even if it came from other data, the word "significant" would still contradict it.
- A rescaled moderator (weeks or log days). Expressing time in weeks multiplies the per-day slope by 7 (−0.766), and log(days) gives −0.509. The Abstract and Results values differ by a factor of 2.81, so no rescaling of this kind links them.
- A different significance convention. The only relaxed threshold in the paper is p < 0.10 for the Cochran Q heterogeneity test. Even at 0.10, p = 0.13 (or 0.157 recomputed from the interval) is not significant, and a t reference distribution (7–8 df) gives a larger p of about 0.20. This does not account for "significant".
- One-sided testing. The one-sided p implied by the interval is 0.078, which is still above 0.05, and the paper does not describe one-sided tests. This does not account for "significant".
- A typesetting or conversion artifact. The publisher's typeset PDF (the version of record, as deposited in PMC) and the Europe PMC full-text XML carry the same numbers as the PMC archive. The mismatch is in the version of record.
- The 11 days as a rounding of 7.9 or as another model's output. Rounding cannot turn 7.9 into 11. In the body, 11 days appears only as the time to a 50% increase in ischemic stroke risk (11.0 days). This report does not infer how the number came to be in the Abstract.
Status
As of 2026-10-24:
- Version of record. I checked the publisher's typeset PDF (Wolters Kluwer on behalf of the AAN, 10 pages, deposited in the PMC open-access set as PMC12473174.1). The Abstract's −0.039 / −0.093 to 0.015 / p = 0.13 / "significant" / 11 days, the ischemic stroke −0.013, the Results' −0.1094 (SE 0.0517, p = 0.0345) and the Results' +0.0388 all appear there as quoted. Publication history in the PDF: received June 11, 2025; accepted July 22, 2025. The Figure 3B fitted line (publisher image) falls by about 0.1 log-odds per day, consistent with −0.1094, and Figure 3C marks 3.4 / 7.9 / 14.9 days.
- Supplement. The article lists online supplementary material (eTables 1–2, eFigures, eMethods). It is not in the PMC or Europe PMC deposit, and the publisher site returned HTTP 403, so I could not inspect it. It was not needed for the recomputation, because the study-level inputs are in the main text (Figures 2A and 3A, Table 1).
- Corrections and notices. Crossref update metadata, PMC notices and Europe PMC notices list no correction, erratum, expression of concern or retraction (checked with
src/known.py). A web search found no correction or published comment. - PubPeer. An earlier lookup through the PubPeer API in this review returned no comments for doi:10.1212/WNL.0000000000214184. A re-check on 2026-10-24 was blocked by the site (HTTP 403), and a web search for the DOI found no PubPeer thread.
Sources: doi:10.1212/WNL.0000000000214184; PMC12473174; PMC open-access bucket s3://pmc-oa-opendata/PMC12473174.1/ (PDF and figure images); Crossref and Europe PMC APIs; PubPeer.
Suggested correction
- Abstract, Results. Report the meta-regression that the Results describe and that the published study-level data reproduce. For example: "Meta-regression showed an inverse association between time to AC resumption and risk of recurrent ICH (regression coefficient −0.109 per day; 95% CI −0.211 to −0.008; p = 0.034), corresponding to an approximate 50% relative reduction in risk at 7.9 days." If the Abstract's −0.039 (95% CI −0.093 to 0.015; p = 0.13) is the intended result, describe the analysis it comes from and remove "significant", because the interval includes 0.
- Abstract, Discussion. Change "approximately 11 days" to the value that the chosen coefficient implies (7.9 days for −0.1094).
- Abstract, ischemic stroke. Replace "coefficient -0.013; 95% CI -0.065 to 0.039; p = 0.61" with the Results value (0.039 per day; SE 0.077; 95% CI −0.111 to 0.189; p = 0.61), or explain the difference.
- Abstract, recurrent ICH heterogeneity. Reconcile I² = 0% with the Results and Figure 3A value of 11.9%.
- Abstract, Methods. Reconcile the search end date ("December 2023") with the Methods ("22 April 2025").
- Results, Time-Dependent Risk Modeling. Change "25% ... at approximately 6 days, 50% at 7.9 days, and 75% at 3.4 days" to 3.4 / 7.9 / 14.9 days, and reconcile the stated 20% day-0 ICH risk with the model, whose intercept implies about 27% and whose thresholds use that value.
- Optional, for interpretation. Report a leave-one-out analysis for the meta-regression itself. Without the 165-patient cohort or the 66-patient cohort the slope is no longer significant (p = 0.36 and 0.19).
Recomputation script (python recompute.py.txt)
"""Recompute the time-to-resumption meta-regressions of PMC12473174
(Neurology 2025, doi:10.1212/WNL.0000000000214184, meta-analysis of anticoagulation
reinitiation after intracranial hemorrhage in patients with mechanical heart valves).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs copied from the paper (PMC archive and publisher PDF):
Abstract, Results: recurrent ICH meta-regression coefficient -0.039, 95% CI -0.093 to 0.015, p = 0.13,
"approximate 50% relative reduction in risk at 11 days"
ischemic stroke coefficient -0.013, 95% CI -0.065 to 0.039, p = 0.61
Results, recurrent ICH: coefficient -0.1094, SE 0.0517, p = 0.0345; residual heterogeneity p = 0.708;
25/50/75% risk reduction at 3.4 / 7.9 / 14.9 days
Results, time-dependent modeling: ICH baseline risk 20%; 25/50/75% reduction at 6 / 7.9 / 3.4 days
Results, ischemic stroke (AIS): coefficient 0.0388, SE 0.0765, p = 0.6123; baseline 4.0%;
25/50/75% increase at 6.0 / 11.0 / 15.2 days
Figure 3A (forest plot, recurrent ICH): events/total per study
Figure 2A (forest plot, AIS): events/total per study
Table 1, "Days AC discontinued": moderator per study (means, or medians where stated)
Methods: "Proportions were pooled using the logit transformation method with the inverse variance
approach"; "continuity correction of 0.5"; "meta-regression analyses using the number of days off AC
as a continuous moderator"; R "meta" package.
"""
import os
import numpy as np
from scipy import stats, optimize
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.isf(0.025) # 1.959964
def p2(z):
return 2 * stats.norm.sf(abs(z))
def odds(p):
return p / (1 - p)
def expit(x):
return 1 / (1 + np.exp(-x))
# ------------------------------------------------------------------ study-level inputs
STUDIES = [f"row{i + 1}" for i in range(9)] # the nine studies in Table 1 / Fig 2A / Fig 3A order
N = np.array([6, 13, 4, 26, 10, 66, 4, 165, 141], float) # Fig 3A / Fig 2A "Total"
ICH = np.array([0, 1, 0, 1, 2, 12, 0, 11, 18], float) # Fig 3A "Events"
AIS = np.array([0, 0, 0, 0, 0, 1, 0, 9, 11], float) # Fig 2A "Events"
DAYS = np.array([19, 7, 1, 8, 1.4, 6.7, 8.5, 12.7, 10], float) # Table 1 "Days AC discontinued"
print("=== 0. Study-level inputs ===")
for s, n, e, a, d in zip(STUDIES, N, ICH, AIS, DAYS):
print(f" {s:10s} n={n:4.0f} ICH events={e:3.0f} AIS events={a:3.0f} days off AC={d}")
print(f" totals: n={N.sum():.0f} (paper: 435), ICH events={ICH.sum():.0f} (paper: 45), "
f"AIS events={AIS.sum():.0f} (paper: 21)")
R = {}
# ------------------------------------------------------------------ 1. abstract ICH statistics
print("\n=== 1. Abstract, recurrent ICH: b = -0.039, 95% CI -0.093 to 0.015, p = 0.13, 'significant' ===")
lo, hi, b = -0.093, 0.015, -0.039
se = (hi - lo) / (2 * Z)
z = b / se
R["abs_ich"] = dict(b=b, lo=lo, hi=hi, se=se, p=p2(z))
print(f" CI midpoint = {(lo + hi) / 2:.4f} (reported coefficient -0.039)")
print(f" CI contains 0: {lo < 0 < hi}")
print(f" implied SE = {se:.4f}, z = {z:.3f}, two-sided p = {p2(z):.3f}, one-sided p = {p2(z) / 2:.3f}")
print(f" |z| implied by the reported p = 0.13: {stats.norm.isf(0.065):.3f}")
for df in (7, 8):
print(f" t reference, df = {df}: two-sided p = {2 * stats.t.sf(abs(z), df):.3f}")
print(" at either alpha 0.05 or the paper's Q-test alpha 0.10, p = 0.13 and p = 0.157 are not significant")
# ------------------------------------------------------------------ 2. Results ICH statistics
print("\n=== 2. Results, recurrent ICH: b = -0.1094, SE = 0.0517, p = 0.0345 ===")
b, se = -0.1094, 0.0517
z = b / se
R["res_ich"] = dict(b=b, se=se, lo=b - Z * se, hi=b + Z * se, p=p2(z))
print(f" z = {z:.3f}, two-sided normal p = {p2(z):.4f}, 95% CI {b - Z * se:.4f} to {b + Z * se:.4f}")
for df in (7, 8):
print(f" t reference, df = {df}: two-sided p = {2 * stats.t.sf(abs(z), df):.4f}")
print(f" ratio of coefficients, Results / Abstract = {0.1094 / 0.039:.2f}")
print(f" ratio of SEs, Results / Abstract = {0.0517 / R['abs_ich']['se']:.2f}")
# ------------------------------------------------------------------ 3. independent meta-regression
def metareg(e, n, x, cc="zero"):
"""Inverse-variance meta-regression of logit proportions on x.
cc='zero': add 0.5 to both cells of zero-event studies only; cc='all': to every study.
Returns slope, SE, p, intercept, tau2 (DerSimonian-Laird and REML), QE, p(QE)."""
e = e.copy(); n = n.copy()
m = (e == 0) if cc == "zero" else np.ones_like(e, bool)
e[m] += 0.5; n[m] += 1
y = np.log(e / (n - e)); v = 1 / e + 1 / (n - e)
X = np.column_stack([np.ones_like(x), x])
def wls(t2):
W = np.diag(1 / (v + t2)); C = np.linalg.inv(X.T @ W @ X); beta = C @ X.T @ W @ y
return beta, C, W
beta0, _, W0 = wls(0.0)
r = y - X @ beta0
QE = float(r @ W0 @ r); k = len(y)
P = W0 - W0 @ X @ np.linalg.inv(X.T @ W0 @ X) @ X.T @ W0
t2_dl = max(0.0, (QE - (k - 2)) / np.trace(P))
def nll(t2):
beta, C, W = wls(t2); rr = y - X @ beta
return 0.5 * (np.sum(np.log(v + t2)) + np.log(np.linalg.det(X.T @ W @ X)) + rr @ W @ rr)
t2_reml = optimize.minimize_scalar(nll, bounds=(0, 5), method="bounded").x
t2_reml = 0.0 if t2_reml < 1e-4 else t2_reml
beta, C, _ = wls(t2_reml)
s = float(np.sqrt(C[1, 1]))
return dict(b=float(beta[1]), se=s, p=p2(beta[1] / s), lo=beta[1] - Z * s, hi=beta[1] + Z * s,
b0=float(beta[0]), t2_dl=t2_dl, t2_reml=t2_reml, QE=QE, pQE=stats.chi2.sf(QE, k - 2))
def glm_binomial(e, n, x):
"""Binomial logistic regression of events on x by IRLS (no continuity correction)."""
X = np.column_stack([np.ones_like(x), x]); beta = np.zeros(2)
for _ in range(100):
mu = expit(X @ beta); w = n * mu * (1 - mu)
zz = X @ beta + (e - n * mu) / w
new = np.linalg.solve(X.T @ (w[:, None] * X), X.T @ (w * zz))
if np.max(np.abs(new - beta)) < 1e-12:
beta = new; break
beta = new
mu = expit(X @ beta); w = n * mu * (1 - mu)
s = float(np.sqrt(np.linalg.inv(X.T @ (w[:, None] * X))[1, 1]))
return dict(b=float(beta[1]), se=s, p=p2(beta[1] / s), lo=beta[1] - Z * s, hi=beta[1] + Z * s)
print("\n=== 3. Meta-regression recomputed from Fig 3A events/totals and Table 1 days off AC ===")
for name, e in (("ICH", ICH), ("AIS", AIS)):
for cc in ("zero", "all"):
f = metareg(e, N, DAYS, cc)
R[f"{name}_{cc}"] = f
print(f" {name}, logit, 0.5 added to {'zero-event studies' if cc == 'zero' else 'every study'}: "
f"slope = {f['b']:.4f}, SE = {f['se']:.4f}, p = {f['p']:.4f}, 95% CI {f['lo']:.3f} to {f['hi']:.3f}; "
f"tau2 DL = {f['t2_dl']:.4f}, REML = {f['t2_reml']:.4f}; QE = {f['QE']:.3f}, p(QE) = {f['pQE']:.3f}; "
f"intercept = {f['b0']:.3f} (risk at day 0 = {expit(f['b0']):.3f})")
g = glm_binomial(e, N, DAYS)
R[f"{name}_glm"] = g
print(f" {name}, binomial logistic regression (no correction): slope = {g['b']:.4f}, SE = {g['se']:.4f}, "
f"p = {g['p']:.4f}, 95% CI {g['lo']:.3f} to {g['hi']:.3f}")
print(" -> ICH, zero-event correction: reproduces the Results (-0.1094, 0.0517, 0.0345, residual p 0.708)")
print(" -> no variant gives the Abstract's -0.039 (CI -0.093 to 0.015, p = 0.13)")
print(" -> AIS: no variant reproduces 0.0388 exactly, but every variant gives a positive slope")
# ------------------------------------------------------------------ 3b. could a subset or rescaling give -0.039?
import itertools # noqa: E402
def wls_fixed(idx, xx):
e = ICH[idx].copy(); n = N[idx].copy(); m = e == 0; e[m] += 0.5; n[m] += 1
y = np.log(e / (n - e)); v = 1 / e + 1 / (n - e)
X = np.column_stack([np.ones(len(y)), xx[idx]]); W = np.diag(1 / v)
C = np.linalg.inv(X.T @ W @ X); beta = C @ X.T @ W @ y
return float(beta[1]), float(np.sqrt(C[1, 1]))
print("\n=== 3b. ICH: leave-one-out, all subsets, and log(days) ===")
loo = []
for i, sname in enumerate(STUDIES):
b_, s_ = wls_fixed([j for j in range(9) if j != i], DAYS)
loo.append(b_)
print(f" without {sname:10s}: slope = {b_:.4f}, SE = {s_:.4f}, p = {p2(b_ / s_):.3f}")
print(f" leave-one-out slope range: {min(loo):.4f} to {max(loo):.4f}")
min_se = min(wls_fixed(list(c), DAYS)[1] for k in range(3, 10) for c in itertools.combinations(range(9), k))
print(f" smallest slope SE over every subset of 3-9 studies: {min_se:.4f} "
f"(Abstract's CI implies {R['abs_ich']['se']:.4f}); removing studies can only increase the SE")
near = sorted((abs(wls_fixed(list(c), DAYS)[0] + 0.039), c) for k in range(3, 10)
for c in itertools.combinations(range(9), k))[:3]
for _, c in near:
b_, s_ = wls_fixed(list(c), DAYS)
print(f" subset with slope closest to -0.039: {len(c)} studies "
f"({', '.join(STUDIES[j] for j in c)}): slope = {b_:.4f}, SE = {s_:.4f}")
b_, s_ = wls_fixed(list(range(9)), np.log(DAYS))
print(f" moderator log(days): slope = {b_:.4f}, SE = {s_:.4f} (not on the per-day scale)")
print(f" moderator in weeks would multiply the per-day slope by 7: {-0.1094 * 7:.4f}")
# ------------------------------------------------------------------ 4. abstract AIS statistics
print("\n=== 4. Ischemic stroke: Abstract b = -0.013 (CI -0.065 to 0.039, p = 0.61) vs Results b = +0.0388 (SE 0.0765, p = 0.6123) ===")
lo, hi, b = -0.065, 0.039, -0.013
se = (hi - lo) / (2 * Z)
R["abs_ais"] = dict(b=b, lo=lo, hi=hi, se=se, p=p2(b / se))
print(f" Abstract: CI midpoint = {(lo + hi) / 2:.4f}, implied SE = {se:.4f}, p = {p2(b / se):.3f}")
b, se = 0.0388, 0.0765
R["res_ais"] = dict(b=b, se=se, lo=b - Z * se, hi=b + Z * se, p=p2(b / se))
print(f" Results: z = {b / se:.3f}, p = {p2(b / se):.4f}, 95% CI {b - Z * se:.4f} to {b + Z * se:.4f}")
print(" The two coefficients have opposite signs; the Results SE is 2.9 times the Abstract SE "
f"({0.0765 / R['abs_ais']['se']:.2f})")
# ------------------------------------------------------------------ 5. time thresholds
def t_logit(beta, p0, mult):
"""Days until risk reaches mult * p0 under logit(p(t)) = logit(p0) + beta * t."""
return float(np.log(odds(p0 * mult) / odds(p0)) / beta)
def t_lograte(beta, mult):
return float(np.log(mult) / beta)
print("\n=== 5. Days to 25/50/75% change in risk implied by each coefficient ===")
p0_model = float(expit(R["ICH_zero"]["b0"]))
print(f" ICH day-0 risk from the recomputed model intercept: {p0_model:.3f}")
REDUCE = (0.75, 0.5, 0.25)
rows = {}
rows["res_b_model_p0"] = [t_logit(-0.1094, p0_model, m) for m in REDUCE]
rows["res_b_p0_20"] = [t_logit(-0.1094, 0.20, m) for m in REDUCE]
rows["res_b_lograte"] = [t_lograte(-0.1094, m) for m in REDUCE]
rows["abs_b_model_p0"] = [t_logit(-0.039, p0_model, m) for m in REDUCE]
rows["abs_b_p0_20"] = [t_logit(-0.039, 0.20, m) for m in REDUCE]
rows["abs_b_lograte"] = [t_lograte(-0.039, m) for m in REDUCE]
labels = {
"res_b_model_p0": "b = -0.1094, logit, day-0 risk from model intercept",
"res_b_p0_20": "b = -0.1094, logit, day-0 risk 20% (Results text)",
"res_b_lograte": "b = -0.1094, log-risk approximation",
"abs_b_model_p0": "b = -0.039, logit, day-0 risk from model intercept",
"abs_b_p0_20": "b = -0.039, logit, day-0 risk 20%",
"abs_b_lograte": "b = -0.039, log-risk approximation",
}
for k, v in rows.items():
print(f" ICH {labels[k]:55s}: 25% / 50% / 75% reduction at {v[0]:5.2f} / {v[1]:5.2f} / {v[2]:5.2f} days")
print(" reported: Results 3.4 / 7.9 / 14.9; Time-dependent section 6 / 7.9 / 3.4; Abstract 50% at ~11")
# slope that would put the 50% reduction at 11 days
b11_logit = float(np.log(odds(p0_model * 0.5) / odds(p0_model)) / 11)
b11_lograte = float(np.log(0.5) / 11)
print(f" slope needed for a 50% ICH reduction at 11 days: {b11_logit:.4f} (logit, model day-0 risk), "
f"{b11_lograte:.4f} (log-risk)")
ais_res = [t_logit(0.0388, 0.04, m) for m in (1.25, 1.5, 1.75)]
ais_abs = [t_logit(-0.013, 0.04, m) for m in (1.25, 1.5, 1.75)]
print(f" AIS b = +0.0388, baseline 4%: 25/50/75% increase at {ais_res[0]:.2f} / {ais_res[1]:.2f} / {ais_res[2]:.2f} days "
"(reported 6.0 / 11.0 / 15.2)")
print(f" AIS b = -0.013 (Abstract), baseline 4%: {ais_abs[0]:.2f} / {ais_abs[1]:.2f} / {ais_abs[2]:.2f} days "
"(negative: with a negative slope the risk never rises)")
R["thr"] = rows
# ------------------------------------------------------------------ figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": FG, "axes.labelcolor": FG,
"xtick.color": FG2, "ytick.color": FG2, "axes.edgecolor": FG2, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
LW, MS = 2.0, 9 # 2 px lines (in points ~ px at 72 dpi), markers >= 8
def style(ax):
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f" wrote {stem}.svg and {stem}.png")
print("\n=== Figures ===")
# ---- Figure 1: coefficient dot-and-interval plot
M = "−"
def fmt(v, nd):
return (f"{v:+.{nd}f}").replace("-", M)
f1 = [
# (group, label, estimate dict, color, note)
("ICH", "Abstract", R["abs_ich"], ORANGE,
f"{M}0.039 ({M}0.093, 0.015), p = 0.13\ntext: “significant”"),
("ICH", "Results", R["res_ich"], ORANGE, f"{M}0.1094, SE 0.0517, p = 0.0345"),
("ICH", "Recomputed, zero cells +0.5", R["ICH_zero"], BLUE,
f"{fmt(R['ICH_zero']['b'], 4)}, SE {R['ICH_zero']['se']:.4f}, p = {R['ICH_zero']['p']:.4f}"),
("ICH", "Recomputed, all cells +0.5", R["ICH_all"], BLUE,
f"{fmt(R['ICH_all']['b'], 4)}, p = {R['ICH_all']['p']:.3f}"),
("ICH", "Recomputed, binomial GLM", R["ICH_glm"], BLUE,
f"{fmt(R['ICH_glm']['b'], 4)}, p = {R['ICH_glm']['p']:.3f}"),
("AIS", "Abstract", R["abs_ais"], ORANGE, f"{M}0.013 ({M}0.065, 0.039), p = 0.61"),
("AIS", "Results", R["res_ais"], ORANGE, "+0.0388, SE 0.0765, p = 0.6123"),
("AIS", "Recomputed, zero cells +0.5", R["AIS_zero"], BLUE,
f"{fmt(R['AIS_zero']['b'], 4)}, p = {R['AIS_zero']['p']:.2f}"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.9))
ypos, y, prev = [], 0.0, None
for g, *_ in f1:
if prev is not None and g != prev:
y += 1.0
ypos.append(y); y += 1; prev = g
ypos = -np.array(ypos)
for (g, lab, d, col, note), yy in zip(f1, ypos):
ax.plot([d["lo"], d["hi"]], [yy, yy], color=col, lw=LW, solid_capstyle="butt", zorder=3)
mk = "o" if col == ORANGE else "s"
ax.plot(d["b"], yy, mk, color=col, ms=MS, zorder=4, mec=BG, mew=1)
ax.annotate(note, xy=(1.0, yy), xycoords=("axes fraction", "data"), xytext=(10, 0),
textcoords="offset points", va="center", ha="left", fontsize=10, color=FG2,
linespacing=1.15)
ax.axvline(0, color=FG2, lw=0.9, zorder=2)
ax.set_yticks(ypos)
ax.set_yticklabels([r[1] for r in f1], fontsize=10)
for t, r in zip(ax.get_yticklabels(), f1):
t.set_color(r[3])
ax.set_xlim(-0.23, 0.2)
ax.set_xticks([-0.2, -0.1, 0, 0.1, 0.2])
ax.set_xticklabels([f"{M}0.2", f"{M}0.1", "0\nnull", "0.1", "0.2"])
ax.set_xlabel("Slope: change in log-odds of the event per day off anticoagulation, with 95% CI")
ax.xaxis.set_label_coords(0.62, -0.13)
style(ax)
for i, txt in ((0, "Recurrent ICH after resumption (Figure 3A data)"),
(5, "Ischemic stroke before resumption (Figure 2A data)")):
ax.annotate(txt, xy=(0, ypos[i] + 0.9), xycoords=("figure fraction", "data"), xytext=(8, 0),
textcoords="offset points", fontsize=10.5, fontweight="bold", va="center", ha="left",
bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"), zorder=6)
ax.set_ylim(ypos[-1] - 0.7, ypos[0] + 1.3)
handles = [Line2D([], [], color=ORANGE, marker="o", lw=LW, ms=MS, label="printed in the paper"),
Line2D([], [], color=BLUE, marker="s", lw=LW, ms=MS, label="recomputed from the paper's study-level data")]
fig.legend(handles=handles, loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 1.0))
fig.subplots_adjust(left=0.315, right=0.615, top=0.89, bottom=0.15)
save(fig, "fig1")
# ---- Figure 2: days to 25/50/75% ICH risk reduction
f2 = [
("Abstract", [None, 11.0, None], ORANGE, "~11 (50% only)"),
("Results, ICH section", [3.4, 7.9, 14.9], ORANGE, "3.4 / 7.9 / 14.9"),
("Results, risk modeling", [6.0, 7.9, 3.4], ORANGE, "6 / 7.9 / 3.4"),
("Recomputed, b = −0.1094", rows["res_b_model_p0"], BLUE,
" / ".join(f"{v:.1f}" for v in rows["res_b_model_p0"])),
("Recomputed, b = −0.039", rows["abs_b_model_p0"], BLUE,
" / ".join(f"{v:.1f}" for v in rows["abs_b_model_p0"])),
]
MARK = ["o", "s", "D"]
fig, ax = plt.subplots(figsize=(7.2, 4.0))
ys = -np.arange(len(f2), dtype=float)
ys[3:] -= 0.5
for (lab, vals, col, note), yy in zip(f2, ys):
xs = [v for v in vals if v is not None]
ax.plot([min(xs), max(xs)], [yy, yy], color=GRID, lw=LW, zorder=2)
for v, mk in zip(vals, MARK):
if v is None:
continue
ax.plot(v, yy, mk, color=col, ms=MS, zorder=4, mew=2,
mfc=col if mk == "s" else BG, mec=col)
ax.text(45, yy, note, va="center", ha="left", fontsize=10, color=FG2)
ax.axvline(11, color=ORANGE, lw=0.9, ls=(0, (3, 3)), zorder=1)
ax.text(11.4, ys[-1] - 0.45, "11 days (Abstract)", color=ORANGE, fontsize=10, va="center", ha="left")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in f2], fontsize=10)
for t, r in zip(ax.get_yticklabels(), f2):
t.set_color(r[2])
ax.set_xlim(0, 44)
ax.set_xticks(range(0, 45, 5))
ax.set_xlabel("Days off anticoagulation at which recurrent-ICH risk has fallen by 25%, 50% or 75%")
style(ax)
ax.set_ylim(ys[-1] - 0.7, ys[0] + 0.7)
handles = [Line2D([], [], color=FG2, marker="o", mfc=BG, mew=2, lw=0, ms=MS, label="25% reduction"),
Line2D([], [], color=FG2, marker="s", lw=0, ms=MS, label="50% reduction"),
Line2D([], [], color=FG2, marker="D", mfc=BG, mew=2, lw=0, ms=MS, label="75% reduction"),
Line2D([], [], color=ORANGE, lw=LW, label="printed in the paper"),
Line2D([], [], color=BLUE, lw=LW, label="recomputed")]
fig.legend(handles=handles, loc="upper center", ncol=3, frameon=False, fontsize=10,
bbox_to_anchor=(0.5, 1.0), handletextpad=0.3, columnspacing=1.0)
fig.subplots_adjust(left=0.285, right=0.80, top=0.83, bottom=0.15)
save(fig, "fig2")
Text and Fig 4 caption give opposite verdicts on which HFpEF subtype shows significant PSD mediation
PLoS Comput Biol (2026) · doi:10.1371/journal.pcbi.1013891 · PMC12829950
Summary
The paper's main subtype finding concerns which of two HFpEF clusters shows significant mediation through peak strain dispersion (PSD). On this point it gives two opposite accounts. In the first, Results 2.5 (with 0.313) and Discussion 3.4 (with no number) say the Cluster 0 eGFR→PSD→GWW effect "did not reach statistical significance". Discussion 3.2 says the same but gives 0.2671, and the abstract Results (0.313) and the Conclusion (Section 4, no number) describe it only as a "trend". The same passages say Cluster 1 shows significant BNP mediation through PSD. The values −0.4877 and 0.5389 with P < 0.05 appear in the abstract Results and Results 2.5; Discussion 3.2 gives these values without P, and Discussion 3.4 and Section 4 give no numbers. In the second account, the Fig 4 caption, the abstract's Conclusion and the Author summary say the reverse: Cluster 0 mediation is "significant" and Cluster 1 mediation is "negligible" or "weaker". The paper's own Fig 4 sides with the caption. Its Cluster 0 eGFR→GWW interval excludes zero, all of its Cluster 1 intervals reach or cross zero, and its Cluster 1 BNP estimates are −0.024 and 0.001, not −0.4877 and 0.5389. Because the abstract contradicts itself on the paper's headline result, a reader cannot tell from the article which subtype the PSD mediation finding applies to.
What the paper reports
Account A, the text:
"In Cluster 0, eGFR showed a trend toward mediating its effects on GWW through PSD (indirect effect = 0.313), though this effect did not reach statistical significance. [...] In contrast, Cluster 1 demonstrated significant mediation effects: BNP’s effect on GWW was significantly mediated through PSD (indirect effect = -0.4877, P < 0.05), and BNP’s effect on GWE was entirely mediated through PSD (indirect effect = 0.5389, P < 0.05)." — Results 2.5, first paragraph
"In Cluster 0, eGFR showed a trend toward mediating its effects on GWW through PSD (indirect effect = 0.313), reflecting complex cardiorenal interactions in younger patients with severe renal disease. In contrast, Cluster 1 demonstrated significant mediation effects: BNP’s effect on GWW was significantly mediated through PSD (indirect effect = -0.4877, P < 0.05), and BNP’s effect on GWE was entirely mediated through PSD (indirect effect = 0.5389, P < 0.05)." — Abstract, Results. The Cluster 1 numbers and P values are the same as in Results 2.5; the Cluster 0 effect is called a "trend", without the "did not reach statistical significance" verdict.
"In contrast, Cluster 1 demonstrated significant and pronounced mediation effects, with BNP showing the strongest indirect effects on both GWW (50.0% mediation ratio) and GWE (111.2% mediation ratio)" — Results 2.5, second paragraph (describing Fig 5)
"In the younger-renal dominant phenotype, eGFR showed a trend toward mediating its effects on myocardial work through PSD (indirect effect = 0.2671), though this effect did not reach statistical significance." — Discussion 3.2. The same paragraph continues: "the older-cardiac dominant phenotype exhibited significant mediation effects, with BNP’s effect on GWW significantly mediated through PSD (indirect effect = -0.4877, 50.0% mediation ratio), and BNP’s effect on GWE entirely mediated through PSD (indirect effect = 0.5389, 111.2% mediation ratio)", with no P values.
"Cluster 0 represents the younger-renal dominant phenotype [...], showing a trend toward eGFR-mediated effects on myocardial work through PSD, though this effect did not reach statistical significance. [...] Cluster 1represents the older-cardiac dominant phenotype [...], demonstrating significant BNP-mediated effects on myocardial work through PSD" — Discussion 3.4 (no numbers)
"a younger, renal-dominant subtype with pronounced dyssynchrony showing a trend toward PSD-mediated effects, and an older, cardiac-dominant subtype with significant BNP-mediated effects through PSD on myocardial work." — Section 4, Conclusion (no numbers; Cluster 0 is described as a "trend")
Account B, the caption and summaries:
"Cluster 0 shows significant eGFR to GWW mediation via PSD (indirect effect = 0.313, mediation ratio = 1.00), while Cluster 1 shows negligible mediation effects, highlighting PSD’s critical role specifically in the younger, renal-dominant phenotype." — Fig 4 caption
"Cluster 0 shows prominent PSD-mediated effects through cardiorenal interactions, while Cluster 1 demonstrates weaker PSD mediation, suggesting age-related mechanisms operate through pathways less dependent on myocardial mechanical dyssynchrony." — Abstract, Conclusion
"this measurement worked differently in each patient group, with younger patients showing strong coordination effects and older patients showing weaker effects." — Author summary
"Cluster 0 demonstrates age to GWE (0.4797) and complete eGFR to GWW mediation (1.00)" — Fig 5 caption
The significance rule:
"Statistical significance was determined using bias-corrected bootstrap confidence intervals, with effects considered significant if the 95% confidence interval did not include zero." — Methods 5.7
What the paper's own numbers show
Method. I compared each number and verdict in the text with the two figures that present the same analysis. Fig 4 is a forest plot of indirect effects with 95% bootstrap intervals for eight paths: eGFR or BNP, through PSD, to GWE or GWW, in each of the two clusters. Each point estimate is printed as a label on the plot. The interval ends are not printed, so I measured them from the publisher's full-resolution image. Each panel's axis frame spans −0.1 to 0.8, which gives 0.00088 units per pixel. As a check on that calibration, the dashed zero line reads as −0.0008 and −0.0012 in the two panels. Fig 5 is a heatmap: panel A gives indirect effects and panel B gives "normalized mediation ratios". I transcribed both panels and tested how panel B was derived. recompute.py.txt repeats every step and prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's own figures |
|---|---|---|---|
| Cluster 0 eGFR→PSD→GWW, significance | Results 2.5 (0.313), Discussion 3.2 (0.2671) and 3.4 (no number) vs Fig 4 caption | Text: "did not reach statistical significance" (abstract Results and Section 4: "trend"). Caption: "significant" | Fig 4: 0.313, 95% CI ≈ 0.03 to 0.70. The lower end is 36 pixels right of zero, so the interval excludes 0 and the effect is significant under the Methods 5.7 rule |
| Cluster 0 eGFR→PSD→GWW, point estimate | Results 2.5, abstract, Fig 4 caption vs Discussion 3.2 | 0.313 vs 0.2671 | Apparently the same effect with two values (Discussion 3.2 says "myocardial work"; the parallel Results sentence says GWW): difference 0.0459 (14.7% of 0.313). Fig 4 prints 0.313 |
| Cluster 1 BNP→PSD→GWW | Abstract Results, Results 2.5 (Discussion 3.2: same value, no P) | −0.4877, P < 0.05 | Fig 4: −0.024, CI ≈ −0.064 to 0.002 (upper end at 0 within pixel resolution). The text value is 20× larger and lies 0.423 outside the plotted interval |
| Cluster 1 BNP→PSD→GWE | Abstract Results, Results 2.5 (Discussion 3.2: same value, no P) | 0.5389, P < 0.05 | Fig 4: 0.001, CI ≈ −0.002 to 0.004, which includes 0. The text value lies 0.535 outside the plotted interval |
| Cluster 1, all four paths | Fig 4 caption vs Results 2.5 | Caption: "negligible". Text: significant | No Cluster 1 interval in Fig 4 excludes 0. All four lie within −0.064 to 0.211, and the largest point estimate is 0.060 |
| Cluster 1 BNP mediation ratios | Results 2.5, Discussion 3.2 vs Fig 5B | GWW 50.0%, GWE 111.2% | Fig 5B, BNP Cluster 1: GWW 0.07, GWE 0.00. Taken with the text's indirect effects, the text's ratios imply total effects of −0.9754 (GWW) and +0.4846 (GWE), and the paper reports neither |
| What Fig 5B contains | Fig 5B, "Mediation Ratios (Normalized)" | Read as mediation ratios | All 16 cells of Fig 5B equal the matching Fig 5A cell divided by the largest Fig 5A cell (14.8963, age, Cluster 0, GWW), rounded to 2 dp. Panel B is therefore a rescaling of panel A, not indirect/total effect, and no cell can exceed 1.00, so 111.2% cannot come from it |
| "complete eGFR to GWW mediation (1.00)", Cluster 0 | Fig 5 caption (and "mediation ratio = 1.00" in the Fig 4 caption) | 1.00 | The Fig 5B eGFR Cluster 0 GWW cell is 0.00, and its Fig 5A indirect effect is 0.0000. The only 1.00 cell is age, Cluster 0, GWW |
| Age, Cluster 1, GWW "60% mediation ratio" | Fig 5 caption | 60% | 8.9378 / 14.8963 = 0.6000. This is a ratio to the largest heatmap cell, not a proportion mediated |
The two figures therefore agree with Account B (caption, abstract Conclusion, Author summary). The Cluster 1 numbers in Account A (−0.4877, 0.5389, 50.0%, 111.2%) appear in neither figure.
Fig 5 labels its rows "Mediators by Cluster" and titles panel A "Indirect Effects of Mediators on Outcomes", and Methods 5.7 names PSD as the exposure and RWT, eGFR and BNP as the mediators, the reverse of the eGFR→PSD→GWW direction in Fig 4 and the Results. The paper itself, however, presents the Fig 5 cells as effects through PSD: Results 2.5 describes them as "indirect effects on myocardial work parameters through PSD", and the Fig 5 caption speaks of "complete eGFR to GWW mediation". Under that description, Fig 4 and Fig 5A disagree on the Cluster 0 eGFR→GWW indirect effect (0.313 vs 0.0000). The comparisons with Fig 5 in this write-up follow the paper's own description of the figure.
Figure
Figure 1. Indirect effects through PSD for the three paths the text discusses. Orange dots are the values the text states, labelled with the verdict and the place it is stated (the "did not reach statistical significance" verdict with 0.313 is in Results 2.5; the P < 0.05 values are in the abstract Results and Results 2.5). The open orange circle is the Discussion 3.2 value (0.2671) for apparently the same Cluster 0 effect. Blue diamonds and bars are the point labels printed on the paper's Fig 4 and the 95% interval ends measured from that figure. The thin vertical line marks zero (no mediation). The Cluster 0 effect the text calls non-significant has an interval that excludes zero. The two Cluster 1 effects the text calls significant at P < 0.05 are plotted in Fig 4 as −0.024 and 0.001, with intervals that reach or include zero, and the text's values lie far outside those intervals.
Figure 2. Where each account appears. In Account A, the Cluster 0 verdict "did not reach statistical significance" is in Results 2.5 (with 0.313), Discussion 3.2 (with 0.2671) and Discussion 3.4 (no number), while the abstract Results and Section 4 call the effect a "trend"; the Cluster 1 values with P < 0.05 are in the abstract Results and Results 2.5. Account A and Account B (Fig 4 caption, abstract Conclusion, Author summary) give opposite significance verdicts for the same effects. The paper's Fig 4 intervals agree with Account B.
Why it matters
The paper's main contribution, as its title and Discussion 3.2 frame it, is the claim that PSD acts as a mediator in a subtype-specific way: in which subtype, and on which pathway. On that claim the abstract contradicts itself. Its Results sentence assigns significant mediation to the older, cardiac-dominant Cluster 1 through BNP, and its Conclusion assigns "prominent" mediation to the younger, renal-dominant Cluster 0 and "weaker" mediation to Cluster 1. The Author summary follows the Conclusion. The Discussion builds a mechanistic interpretation on Account A, that "neurohormonal activation in older patients primarily manifests as myocardial mechanical dyssynchrony" (Discussion 3.2; Results 2.5 says the same in a different word order). The paper's own Fig 4 does not support that interpretation: its Cluster 1 BNP indirect effects are near zero, with intervals that include zero. A reader who relies on the abstract Results, the Discussion or the Conclusion section will draw the opposite subtype conclusion from one who relies on the figure, its caption or the abstract Conclusion. That bears on the paper's suggestion of subtype-specific risk stratification and therapeutic targeting.
Separately, the "mediation ratios" in Fig 5B and in the captions are each effect divided by the largest effect in the heatmap. They do not describe the share of a total effect that runs through PSD, so "complete ... mediation (1.00)" and "60% mediation ratio" do not mean what the wording suggests.
Explanations considered
- PMC conversion or typesetting artifact. Ruled out. The publisher's JATS XML, the publisher's HTML page and the PMC record carry the same passages word for word, and the figures were taken from the publisher.
- The caption uses "significant" loosely, meaning large rather than statistically significant. This does not fit. The caption describes "bootstrap-validated indirect effects", and the Fig 4 interval for this effect excludes zero, which is significant in the paper's own statistical sense. The text, for its part, says explicitly "did not reach statistical significance". Both statements are about statistical significance, and they disagree.
- The text and the figure report effects on different scales, for example standardized vs raw. This could explain different magnitudes for the BNP paths (−0.4877 vs −0.024). It cannot explain the significance verdicts, because rescaling an estimate and its interval does not change whether the interval contains zero. The Cluster 0 eGFR estimate is also identical (0.313) in the text and in Fig 4, so the two report on the same scale.
- The text's ratios are classical proportions mediated, and ratios above 100% are possible under inconsistent mediation. That is true in general, and it would make 111.2% admissible as a number. However, the paragraph cites these ratios as the content of Fig 5 ("Panel B shows the normalized mediation ratios"), and Fig 5B is a rescaling with a maximum of 1.00 in which BNP Cluster 1 is 0.00 and 0.07. The total effects that the text's ratios imply (−0.9754 and 0.4846) are not reported anywhere in the paper.
- Error in reading the figure. Pixel measurement has limited precision (0.00088 units per pixel), but the conclusions do not depend on it. The Cluster 0 lower bound lies 36 pixels to the right of the zero line. The Cluster 1 BNP point estimates are printed as labels (−0.024 and 0.001) and differ from the text's values by 20× and by more than 300×.
- Fig 5 shows a different path from Fig 4. Fig 5 labels its rows as mediators, and Methods 5.7 names PSD as the exposure and eGFR and BNP as mediators. If Fig 5A were the PSD→eGFR/BNP→outcome direction, its eGFR and BNP cells would not be comparable with Fig 4. This would remove the Fig 4 vs Fig 5A disagreement (0.313 vs 0.0000), but it would not change the main finding, which rests on the text vs Fig 4 and its caption. It would also conflict with the paper's own account of Fig 5 in Results 2.5 ("through PSD") and in the Fig 5 caption ("complete eGFR to GWW mediation"), which this write-up follows.
- The text and the figures come from different runs or versions of the analysis. This would be consistent with the pattern: the text's Cluster 1 values appear in no figure, the Discussion's 0.2671 differs from 0.313 elsewhere, Fig 5A gives 0.0000 for an effect that Fig 4 gives as 0.313, and Fig 5 is titled "Mediation Analysis Heatmap: Indirect Effects and Mediation Ratios (New Data)". If so, it explains how the discrepancy could arise, but it does not resolve which numbers are final. The paper does not say, and no interval table or bootstrap output is provided that would settle it.
Status
As of 2026-10-24:
- Version of record. The current publisher version (JATS XML and HTML at journals.plos.org; published 14 Jan 2026) and the PMC record were checked, and all contain the passages quoted above. Crossref lists a publisher "new version" update dated 23 Jan 2026. The text checked is the current version.
- Corrections. Crossref and PMC list no correction, expression of concern or retraction. The Europe PMC notice service was unavailable when checked. The article page shows no correction notice.
- Supplements. S1 Fig is a PSD reproducibility analysis. S2 Fig is a study schematic, and its caption repeats 0.313 for Cluster 0 eGFR→GWW. S1 Data holds the reproducibility measurements (40 rows). S2 Data holds the per-patient values (150 patients × 50 variables) with no cluster labels. The linked code repository (Zenodo doi:10.5281/zenodo.17988473, v1.0.0) describes its mediation code as "a simplified demonstration" without bootstrap or covariates, and includes no result outputs. The Zenodo record lists hpef_data.csv ("Anonymized patient dataset (84 patients)"), but the archive contains only README.md, analysis_code.py, requirements.txt and two .md files, and its data description calls itself "a template based on the manuscript". No attempt was made to re-cluster S2 Data. The cluster-specific bootstrap intervals cannot be reproduced reliably from these materials, because the cluster labels, the 84-patient subset, the covariate set and the bootstrap settings are not provided, so it is not possible to say independently which account is correct. What can be said is that the paper's figures and its text disagree.
- PubPeer. A web search found no PubPeer entry for the DOI or title. PubPeer's own search could not be queried directly because its bot protection blocked automated access, so an existing thread cannot be ruled out entirely.
- Sources. Article and figures: https://doi.org/10.1371/journal.pcbi.1013891 and https://pmc.ncbi.nlm.nih.gov/articles/PMC12829950/. Crossref record: https://api.crossref.org/works/10.1371/journal.pcbi.1013891.
Suggested correction
- Report, in a table, the indirect effect, bias-corrected 95% bootstrap interval and P value for every path in Fig 4 and Fig 5, for each cluster, and state which analysis run produced them.
- Using those results, make a single account consistent across the abstract (both Results and Conclusion), the Author summary, Results 2.5, Discussion 3.2 and 3.4, Section 4, and the Fig 4, Fig 5 and S2 Fig captions. If Fig 4 is correct, the text's Cluster 1 values (−0.4877, 0.5389, P < 0.05; 50.0%, 111.2%) and the statement that the Cluster 0 eGFR effect "did not reach statistical significance" need to be corrected. If the text is correct, Fig 4, its caption, the abstract Conclusion and the Author summary need to be corrected.
- Reconcile the Cluster 0 eGFR→GWW indirect effect across Discussion 3.2 (0.2671, stated for "myocardial work"), Results and Fig 4 (0.313), and Fig 5A (0.0000), and state in the Fig 5 labels and caption which variable is the exposure and which the mediator.
- Describe Fig 5B as effects scaled to the largest cell, not as mediation ratios. Remove "complete eGFR to GWW mediation (1.00)", because the 1.00 cell is age, Cluster 0, GWW, and remove "mediation ratio = 1.00" from the Fig 4 caption, or replace these with proportions mediated (indirect / total effect) computed directly.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12829950 (PLoS Comput Biol 2026, doi:10.1371/journal.pcbi.1013891)
# Cross-checks the mediation results stated in the text against the paper's Fig 4 and Fig 5,
# prints every number used in writeup.md, and regenerates fig1.* and fig2.* in this folder.
#
# Run: python recompute.py.txt
# Needs numpy and matplotlib. Fig 4's interval bars are measured from the publisher's image,
# which is downloaded to a temporary folder (not saved here). If the download fails, the
# measurements recorded below (made with this same code on 2026-10-24) are used instead.
import os, io, tempfile, urllib.request
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
FIG4_URL = ("https://journals.plos.org/ploscompbiol/article/figure/image?size=large"
"&id=10.1371/journal.pcbi.1013891.g004")
# ---------------------------------------------------------------------------
# 1. Values stated in the text. Where each appears:
# 0.313 with "did not reach statistical significance": Results 2.5 (first paragraph).
# The abstract Results gives 0.313 as a "trend"; Discussion 3.2 gives the verdict with 0.2671;
# Discussion 3.4 gives the verdict with no number; Section 4 says "trend", no number.
# -0.4877 and 0.5389 with P < 0.05: abstract Results and Results 2.5. Discussion 3.2 gives the
# values (with 50.0% / 111.2% ratios) without P; Discussion 3.4 and Section 4 give no numbers.
# ---------------------------------------------------------------------------
TEXT = {
"C0 eGFR->PSD->GWW": dict(ie=0.313, verdict="did not reach statistical significance"),
"C1 BNP->PSD->GWW": dict(ie=-0.4877, verdict="P < 0.05", ratio=0.500),
"C1 BNP->PSD->GWE": dict(ie=0.5389, verdict="P < 0.05", ratio=1.112),
}
DISCUSSION_C0_EGFR = 0.2671 # Discussion 3.2, apparently the same Cluster 0 eGFR effect
# (3.2 says "on myocardial work"; Results 2.5 says GWW)
CAPTION_C0_EGFR = 0.313 # Fig 4 caption ("significant", mediation ratio = 1.00)
# ---------------------------------------------------------------------------
# 2. Fig 4: point labels printed above each bar (read from the image)
# ---------------------------------------------------------------------------
PATHS = ["BNP->PSD->GWE", "BNP->PSD->GWW", "eGFR->PSD->GWE", "eGFR->PSD->GWW"]
FIG4_POINT = {
(0, "BNP->PSD->GWE"): 0.001, (0, "BNP->PSD->GWW"): -0.012,
(0, "eGFR->PSD->GWE"): -0.010, (0, "eGFR->PSD->GWW"): 0.313,
(1, "BNP->PSD->GWE"): 0.001, (1, "BNP->PSD->GWW"): -0.024,
(1, "eGFR->PSD->GWE"): -0.002, (1, "eGFR->PSD->GWW"): 0.060,
}
# Fallback: bar ends measured with measure_fig4() below on 2026-10-24
FIG4_CI_RECORDED = {
(0, "BNP->PSD->GWE"): (-0.0025, 0.0036), (0, "BNP->PSD->GWW"): (-0.0394, 0.0019),
(0, "eGFR->PSD->GWE"): (-0.0254, 0.0019), (0, "eGFR->PSD->GWW"): (0.0317, 0.7008),
(1, "BNP->PSD->GWE"): (-0.0020, 0.0041), (1, "BNP->PSD->GWW"): (-0.0644, 0.0015),
(1, "eGFR->PSD->GWE"): (-0.0082, 0.0023), (1, "eGFR->PSD->GWW"): (-0.0249, 0.2114),
}
def measure_fig4():
"""Read the 95% CI bar ends of both Fig 4 panels from the publisher's PNG.
Calibration: each panel's axes frame spans x = -0.1 .. 0.8 (the first and last tick labels
sit on the frame). The dashed x = 0 line is used as a check of the calibration."""
tmp = tempfile.mkdtemp(prefix="pmc12829950_")
path = os.path.join(tmp, "g004.png")
req = urllib.request.Request(FIG4_URL, headers={"User-Agent": "Mozilla/5.0"})
with urllib.request.urlopen(req, timeout=60) as r, open(path, "wb") as f:
f.write(r.read())
img = (mpimg.imread(path)[:, :, :3] * 255).astype(int)
H, W, _ = img.shape
dark = img.sum(2) < 150
cols = [x for x in range(W) if dark[:, x].sum() > 0.68 * H] # vertical frame edges
groups = []
for x in cols:
if groups and x - groups[-1][-1] <= 3:
groups[-1].append(x)
else:
groups.append([x])
edges = [float(np.mean(g)) for g in groups]
assert len(edges) == 4, edges
frames = [(edges[0], edges[1]), (edges[2], edges[3])]
s = W / 2000.0 # bar centres located on a 2000-px-wide copy
centres = {"BNP->PSD->GWE": 140, "BNP->PSD->GWW": 355, "eGFR->PSD->GWE": 570, "eGFR->PSD->GWW": 785}
out, zero_px, unit_px = {}, [], []
for c, (L, R) in enumerate(frames):
to_val = lambda px: -0.1 + 0.9 * (px - L) / (R - L)
unit_px.append(0.9 / (R - L))
# check: dashed zero line = column with most dark pixels inside the frame
inner = dark[:, int(L) + 5:int(R) - 5].sum(0)
zero_px.append(to_val(int(L) + 5 + int(np.argmax(inner))))
for name, yd in centres.items():
xs = []
# rows above and below the marker (radius ~11 px), inside the bar (half height ~53 px)
for off in list(range(-35, -17)) + list(range(18, 36)):
y = int(round((yd + off) * s))
row = img[y, int(L) + 4:int(R) - 4]
coloured = np.where((row.max(1) - row.min(1)) > 40)[0]
if coloured.size:
xs += [coloured.min() + int(L) + 4, coloured.max() + int(L) + 4]
out[(c, name)] = (round(to_val(min(xs)), 4), round(to_val(max(xs)), 4))
return out, zero_px, unit_px
try:
FIG4_CI, ZERO_CHECK, UNIT = measure_fig4()
SOURCE = "measured now from the publisher's image"
except Exception as e: # offline etc.
FIG4_CI, ZERO_CHECK, UNIT = FIG4_CI_RECORDED, None, [0.9 / 1025] * 2
SOURCE = f"recorded values (download failed: {e.__class__.__name__})"
# ---------------------------------------------------------------------------
# 3. Fig 5 heatmap cell values (read from the image). Panel A = indirect effects,
# Panel B = "Mediation Ratios (Normalized)". Columns (GWE, GWW).
# ---------------------------------------------------------------------------
FIG5_A = {"AGE C0": (0.4797, 14.8963), "AGE C1": (0.2878, 8.9378),
"EGFR C0": (0.5784, 0.0000), "EGFR C1": (0.3470, 0.0000),
"BNP C0": (0.0229, 1.8517), "BNP C1": (0.0138, 1.1110),
"LVEF C0": (0.2189, 1.3294), "LVEF C1": (0.1313, 0.7976)}
FIG5_B = {"AGE C0": (0.03, 1.00), "AGE C1": (0.02, 0.60),
"EGFR C0": (0.04, 0.00), "EGFR C1": (0.02, 0.00),
"BNP C0": (0.00, 0.12), "BNP C1": (0.00, 0.07),
"LVEF C0": (0.01, 0.09), "LVEF C1": (0.01, 0.05)}
# ---------------------------------------------------------------------------
# 4. Print the checks
# ---------------------------------------------------------------------------
def excl0(ci, tol):
lo, hi = ci
if lo > tol or hi < -tol:
return "excludes 0"
if abs(lo) <= tol or abs(hi) <= tol:
return "end at 0 within resolution"
return "includes 0"
print(f"Fig 4 interval ends: {SOURCE}")
if ZERO_CHECK:
print(" calibration check, dashed zero line read as x =",
", ".join(f"{z:+.4f}" for z in ZERO_CHECK), "(should be 0)")
print(" resolution: 1 pixel = " + ", ".join(f"{u:.5f}" for u in UNIT) + " units per panel")
TOL = 0.003 # ~3 pixels, the width of the dashed zero line
for c in (0, 1):
for p in PATHS:
lo, hi = FIG4_CI[(c, p)]
print(f" Cluster {c} {p:15s} point {FIG4_POINT[(c, p)]:+.3f} CI ~[{lo:+.3f}, {hi:+.3f}] "
f"{excl0((lo, hi), TOL)}")
print("\nText vs Fig 4")
lo, hi = FIG4_CI[(0, "eGFR->PSD->GWW")]
print(f" C0 eGFR->PSD->GWW: text {TEXT['C0 eGFR->PSD->GWW']['ie']} '{TEXT['C0 eGFR->PSD->GWW']['verdict']}'; "
f"Fig 4 {FIG4_POINT[(0, 'eGFR->PSD->GWW')]} with CI ~[{lo:.2f}, {hi:.2f}] -> lower end is "
f"{lo / UNIT[0]:.0f} pixels right of 0; significant by Methods 5.7 rule")
for key, p in [("C1 BNP->PSD->GWW", "BNP->PSD->GWW"), ("C1 BNP->PSD->GWE", "BNP->PSD->GWE")]:
t, f = TEXT[key]["ie"], FIG4_POINT[(1, p)]
lo, hi = FIG4_CI[(1, p)]
lo_r, hi_r = sorted([abs(t) / (abs(f) - 0.0005), abs(t) / (abs(f) + 0.0005)]) # label rounded to 3 dp
print(f" {key}: text {t:+.4f} ({TEXT[key]['verdict']}); Fig 4 {f:+.3f}, CI ~[{lo:+.3f}, {hi:+.3f}]; "
f"|text|/|Fig 4| = {abs(t / f):.0f}x (allowing for rounding of the label: {lo_r:.0f}x to {hi_r:.0f}x); "
f"text value outside Fig 4 CI: {not (lo <= t <= hi)}; distance from Fig 4 CI = "
f"{(lo - t) if t < lo else (t - hi):.3f}")
lo1 = min(FIG4_CI[(1, p)][0] for p in PATHS); hi1 = max(FIG4_CI[(1, p)][1] for p in PATHS)
print(f" All Cluster 1 intervals in Fig 4 lie within [{lo1:+.3f}, {hi1:+.3f}]; "
f"largest |point| = {max(abs(FIG4_POINT[(1, p)]) for p in PATHS):.3f}")
print(f" Cluster 1 intervals that exclude 0: "
f"{[p for p in PATHS if excl0(FIG4_CI[(1, p)], TOL) == 'excludes 0']}")
print("\nApparently the same Cluster 0 eGFR effect in two places (Discussion 3.2 says 'myocardial work')")
print(f" Results/abstract/caption/Fig 4 {CAPTION_C0_EGFR} vs Discussion 3.2 {DISCUSSION_C0_EGFR}: "
f"difference {CAPTION_C0_EGFR - DISCUSSION_C0_EGFR:.4f} ({(CAPTION_C0_EGFR - DISCUSSION_C0_EGFR) / CAPTION_C0_EGFR:.1%} of 0.313)")
print("\nText mediation ratios (Results 2.5, Discussion 3.2) vs Fig 5B")
for key, col in [("C1 BNP->PSD->GWW", 1), ("C1 BNP->PSD->GWE", 0)]:
r = TEXT[key]["ratio"]; ie = TEXT[key]["ie"]
print(f" {key}: text ratio {r:.1%}; Fig 5B BNP (Cluster 1) {['GWE', 'GWW'][col]} = {FIG5_B['BNP C1'][col]:.2f}; "
f"Fig 5A indirect effect {FIG5_A['BNP C1'][col]:.4f}; implied total effect from text = {ie / r:+.4f}")
print(f" Fig 5A eGFR (Cluster 0) GWW indirect effect = {FIG5_A['EGFR C0'][1]:.4f}, ratio {FIG5_B['EGFR C0'][1]:.2f} "
f"(Fig 4: 0.313; Fig 5 caption: 'complete eGFR to GWW mediation (1.00)')")
print(f" Fig 5B cell equal to 1.00: {[k for k, v in FIG5_B.items() if v[1] == 1.00]} / GWW")
print("\nFig 5B reproduced as Fig 5A divided by the largest Fig 5A cell")
mx = max(max(v) for v in FIG5_A.values())
print(f" largest Fig 5A cell = {mx} (AGE Cluster 0, GWW)")
ok = 0
for k in FIG5_A:
calc = tuple(round(v / mx, 2) for v in FIG5_A[k])
match = all(abs(a - b) < 0.006 for a, b in zip(calc, FIG5_B[k]))
ok += match
print(f" {k:8s} A/max = ({FIG5_A[k][0] / mx:.4f}, {FIG5_A[k][1] / mx:.4f}) -> ({calc[0]:.2f}, {calc[1]:.2f}); "
f"Fig 5B ({FIG5_B[k][0]:.2f}, {FIG5_B[k][1]:.2f}) match={match}")
print(f" {ok} of {len(FIG5_A)} rows (all 16 cells) match" if ok == len(FIG5_A) else f" {ok} rows match")
print(f" Fig 5 caption '60% mediation ratio' for age Cluster 1 GWW = 8.9378 / 14.8963 = {8.9378 / mx:.4f}")
# ---------------------------------------------------------------------------
# 5. Figures
# ---------------------------------------------------------------------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"axes.edgecolor": SEC, "svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, bbox_inches="tight", **kw)
plt.close(fig)
# fig1: dot-and-interval, text values vs Fig 4 values on one axis
rows = [("Cluster 0\neGFR → PSD → GWW", "C0 eGFR->PSD->GWW", (0, "eGFR->PSD->GWW")),
("Cluster 1\nBNP → PSD → GWW", "C1 BNP->PSD->GWW", (1, "BNP->PSD->GWW")),
("Cluster 1\nBNP → PSD → GWE", "C1 BNP->PSD->GWE", (1, "BNP->PSD->GWE"))]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
ax.axvline(0, color=SEC, lw=1, zorder=1)
ax.text(-0.01, -0.5, "no mediation (0)", color=SEC, fontsize=10, ha="right", va="center")
for i, (lab, tk, fk) in enumerate(rows):
y = 2 - i
t = TEXT[tk]
ax.plot(t["ie"], y + 0.18, "o", ms=9, color=ORANGE, zorder=3)
lbl = {0: f"{t['ie']:g}: “{t['verdict']}” (Results 2.5)",
1: f"{t['ie']:g}, {t['verdict']}\n(abstract, Results 2.5)",
2: f"{t['ie']:g}, {t['verdict']} (abstract, Results 2.5)"}[i]
pos = {0: (0.012, y + 0.38, "left", "bottom"), 1: (-0.45, y + 0.18, "left", "center"),
2: (t["ie"], y + 0.38, "center", "bottom")}[i]
ax.text(pos[0], pos[1], lbl.replace("-", "\u2212"), color=ORANGE, fontsize=10,
ha=pos[2], va=pos[3], linespacing=1.25)
lo, hi = FIG4_CI[fk]; pt = FIG4_POINT[fk]
ax.plot([lo, hi], [y - 0.18] * 2, color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
for e in (lo, hi):
ax.plot([e, e], [y - 0.26, y - 0.10], color=BLUE, lw=2, zorder=2)
ax.plot(pt, y - 0.18, "D", ms=8, color=BLUE, zorder=3)
note = f"{pt:g}, 95% CI {lo:.2f} to {hi:.2f}" if i == 0 else f"{pt:g}, 95% CI {lo:.3f} to {hi:.3f}"
note = note.replace("-", "\u2212")
ax.text(hi + 0.02 if i == 0 else 0.07, y - 0.18, note, color=BLUE, fontsize=10, va="center",
ha="left" if i else "left")
# Discussion value for the same Cluster 0 effect
ax.plot(DISCUSSION_C0_EGFR, 2 + 0.18, "o", ms=9, mfc=BG, mec=ORANGE, mew=2, zorder=4)
ax.set_yticks([2, 1, 0]); ax.set_yticklabels([r[0] for r in rows])
ax.set_ylim(-0.6, 3.25); ax.set_xlim(-0.6, 0.85)
ax.set_xlabel("Indirect effect via PSD (paper's units)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "o", ms=9, color=ORANGE, label="Stated in the text (abstract Results, Results 2.5, Discussion 3.2)")
h3, = ax.plot([], [], "o", ms=9, mfc=BG, mec=ORANGE, mew=2,
label=f"Discussion 3.2 value for the Cluster 0 eGFR effect ({DISCUSSION_C0_EGFR})")
h2, = ax.plot([], [], "D-", ms=8, lw=2, color=BLUE, label="Plotted in the paper's Fig 4 (point label; CI bar measured)")
ax.legend(handles=[h1, h3, h2], loc="lower left", bbox_to_anchor=(0, 1.0), frameon=False, fontsize=10,
ncol=1, borderaxespad=0.2)
save(fig, "fig1")
# fig2: the two accounts of which subtype shows significant PSD mediation
fig, ax = plt.subplots(figsize=(7.2, 6.6))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG); ax.axis("off")
ax.set_xlim(0, 100); ax.set_ylim(-8, 90)
TOP, BOT = 88, 22
def panel(x, w, title, where, lines, edge, y_lines):
ax.add_patch(FancyBboxPatch((x, BOT), w, TOP - BOT, boxstyle="round,pad=0.3,rounding_size=1.5",
fc="white", ec=edge, lw=1.5))
ax.text(x + 1.5, TOP - 2, title, fontsize=11, fontweight="bold", va="top")
ax.text(x + 1.5, TOP - 6.5, where, fontsize=10, color=SEC, va="top", linespacing=1.3)
ax.text(x + 1.5, y_lines, lines, fontsize=10, va="top", linespacing=1.35)
panel(0.5, 42, "Account A (text)",
"Verdict: Results 2.5 (0.313),\nDiscussion 3.2 (0.2671),\nDiscussion 3.4 (no number).\n"
"Abstract Results and Section 4\nsay only \u201ctrend\u201d.",
"Cluster 0, eGFR\u2192PSD\u2192GWW\n= 0.313: \u201cdid not reach\nstatistical significance\u201d\n\n"
"Cluster 1, BNP\u2192PSD\u2192GWW\n= \u22120.4877 and BNP\u2192PSD\u2192\nGWE = 0.5389: \u201cP < 0.05\u201d\n"
"(abstract Results, Results 2.5;\nDiscussion 3.2 gives the\nvalues without P)",
ORANGE, 61)
panel(57.5, 42, "Account B", "Fig 4 caption;\nabstract Conclusion;\nAuthor summary",
"Cluster 0, eGFR\u2192GWW\n= 0.313: \u201csignificant eGFR\nto GWW mediation via PSD\u201d\n(Fig 4 caption)\n\n"
"Cluster 1: \u201cnegligible\nmediation effects\u201d\n(Fig 4 caption); \u201cweaker\nPSD mediation\u201d\n"
"(abstract Conclusion)",
SEC, 61)
ax.add_patch(FancyArrowPatch((43.3, 50), (57, 50), arrowstyle="<|-|>", mutation_scale=14, color=TXT, lw=2))
ax.text(50.15, 52, "same\neffects,\nopposite\nverdicts", ha="center", va="bottom", fontsize=10, color=TXT)
lo, hi = FIG4_CI[(0, "eGFR->PSD->GWW")]
ax.add_patch(FancyBboxPatch((57.5, -7.5), 42, 23, boxstyle="round,pad=0.3,rounding_size=1.5",
fc=BG, ec=BLUE, lw=1.5))
ax.text(59, 14.8, "Fig 4's own bars", color=BLUE, fontsize=10, fontweight="bold", va="top")
ax.text(59, 11.2, f"Cluster 0 eGFR\u2192GWW: CI\n{lo:.2f} to {hi:.2f}, excludes 0\n"
f"Cluster 1 BNP: {FIG4_POINT[(1, 'BNP->PSD->GWW')]:g} and\n{FIG4_POINT[(1, 'BNP->PSD->GWE')]:g}; both CIs\nreach or cross 0".replace("-", "\u2212"),
color=BLUE, fontsize=10, va="top", linespacing=1.3)
ax.add_patch(FancyArrowPatch((79, 16.2), (79, 21.5), arrowstyle="-|>", mutation_scale=12, color=BLUE, lw=2))
ax.text(80.8, 18.8, "agrees with B", color=BLUE, fontsize=10, va="center")
ax.text(1.5, 13, "Account A's Cluster 1 numbers\n(\u22120.4877, 0.5389, 50.0%,\n111.2%) appear in neither\nFig 4 nor Fig 5.",
fontsize=10, color=SEC, va="top", linespacing=1.3)
save(fig, "fig2")
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Anhedonia–TSPO correlation reported as P < 0.001 at N = 15 corresponds to P ≈ 0.1
Proc Natl Acad Sci U S A (2024) · doi:10.1073/pnas.2406005121 · PMC11363315
Summary
The association named in the title, between anhedonia severity and the LPS-induced change in TSPO availability ([11C]PBR28 VT) in the PTSD group, is reported as r = −0.425, P < 0.001 in the Results and as r = −0.45, P < 0.001 in the Fig. 2 legend, which states N = 15. For a correlation (or the slope of a simple linear regression) on 15 participants, these r values correspond to two-sided P = 0.114 and P = 0.092, and the 95% confidence interval for r includes zero in both cases (about −0.77 to +0.11). P < 0.001 would need |r| > 0.760 at N = 15, or about 51 to 57 independent participants at the reported r. No standard reading of the test that I could find (one-sided, rank-based, R², partial correlation, the N = 12 CAPS-5 subset) brings the P value near 0.001, so on the paper's own numbers the title association is not statistically significant. The main group comparison (PTSD vs. control, P = 0.021) does not depend on this test.
What the paper reports
Quotations are verbatim from the PMC-archived text (the PMC text renders the minus sign as a hyphen; the typeset text uses U+2212).
Title:
Microglia-mediated neuroimmune suppression in PTSD is associated with anhedonia
Results, "PTSD Is Associated with Lower TSPO Response to LPS Administration", paragraph 1:
In the PTSD group, greater baseline severity of anhedonia symptoms on the clinical administered PTSD scale (CAPS) was associated with a more “blunted” neuroimmune TSPO response to LPS (r = -0.425, P < 0.001) (Fig. 2).
Fig. 2 legend:
Individual data for N = 15 participants in the PTSD group demonstrating that a smaller percent change in [11C]PBR28 VT values following LPS administration was significantly associated with worse anhedonia scores from the CAPS (r = -0.45; P < 0.001) using linear regression analysis. Gray shading indicates the 95% CI for the linear regression.
Results, "Demographic and Clinical Characteristics":
Fifteen individuals who met DSM-5 criteria for PTSD participated in the study (9M and 6F).
Table 1, "Symptom scores" row:
CAPS-5 total * N = 12 | 0.6 (1.5) | 32 (11) | 10.95 | <0.001
Materials and Methods, "Trauma History Assessment":
To examine baseline PTSD symptom clusters associated with LPS-induced changes in TSPO availability, we computed sum scores of CAPS measures that assessed the seven symptom dimensions comprising the hybrid model of PTSD symptom structure (47)—intrusions, avoidance, negative affect, anhedonia, externalizing behaviors, anxious arousal, and dysphoric arousal.
Materials and Methods, "Statistical Analysis" (step 4):
Fourth, to determine whether certain PTSD symptom clusters were associated with changes in TSPO availability pre- and post-LPS administration in the PTSD group, we conducted a multivariable linear regression that included intrusion, avoidance, negative alterations in cognitions and mood, and alterations in arousal and reactivity entered as independent variables and LPS-induced changes in TSPO availability as the dependent variable.
Abstract:
Greater anhedonic symptoms in the PTSD group were associated with a more suppressed neuroimmune response.
Discussion, paragraph 3:
Another notable finding in this study is that greater severity of anhedonic symptoms in the PTSD group was associated with a more suppressed neuroimmune response.
Discussion, conclusion paragraph:
They further demonstrate a relationship between anhedonic symptoms of PTSD and neuroimmune suppression, which suggests that restoring neuroimmune function may potentially help mitigate these symptoms and serve as a treatment for PTSD.
What the paper's own numbers show
Method. The Fig. 2 legend describes a linear regression of percent change in VT on anhedonia score across N = 15 PTSD participants, reported with r. For a simple linear regression, the slope t test equals the test of Pearson's r: t = r·√(n − 2)/√(1 − r²) with n − 2 degrees of freedom. I applied it to both reported r values at n = 15 (df = 13), computed the 95% CI for r by the Fisher z transform, found the |r| and the n that P < 0.001 would need, and then repeated the test under each alternative reading listed in "Explanations considered". Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Anhedonia vs. % change VT, text | Results, "PTSD Is Associated with Lower TSPO Response…", para 1 | r = −0.425, P < 0.001 | t = −1.693, df = 13, two-sided P = 0.114 (one-sided 0.057) |
| Same association, figure | Fig. 2 legend (N = 15) | r = −0.45, P < 0.001 | t = −1.817, df = 13, two-sided P = 0.092 (one-sided 0.046) |
| 95% CI for r (Fisher z, n = 15) | derived | not reported | −0.770 to +0.112 (r = −0.425); −0.782 to +0.081 (r = −0.45) |
| |r| needed for two-sided P < 0.001 | derived, n = 15 (df = 13) | — | |t| > 4.221, i.e. |r| > 0.760 |
| n needed for P < 0.001 at the reported r | derived | — | 57 (r = −0.425); 51 (r = −0.45) |
| Fisher z test instead of t | derived, n = 15 | — | P = 0.116 / 0.093 |
| Spearman ρ = −0.45, exact permutation null | derived, n = 15 | — | P = 0.094; P < 0.001 needs |ρ| ≥ 0.775 |
| Only the 12 participants with CAPS-5 totals | Table 1, "CAPS-5 total * N = 12" | — | P = 0.168 / 0.142 (|r| > 0.823 needed) |
| 0.425 / 0.45 read as R² | alternative reading | — | r = −0.652 / −0.671, P = 0.0085 / 0.0062 |
| Partial r, 3 or 6 other covariates | Methods step 4; hybrid model | — | P = 0.168 / 0.142 (3); 0.254 / 0.224 (6) |
| Standardized β, uncorrelated predictors (best case) | Methods step 4 | — | P < 0.001 needs model R² > 0.914 (4 predictors) or > 0.957 (7 predictors) for β = −0.425 |
| 60 points treated as independent (4 regions × 15) | not described anywhere in the paper | — | P = 0.00071 / 0.00031 |
The direction and size of the association are unchanged by the recalculation: r is about −0.43 to −0.45. What changes is its significance. At n = 15 the reported P < 0.001 is not attainable for |r| below 0.760, and both reported r values are well below that. The two r values also differ from each other (−0.425 rounds to −0.43, not −0.45), but this smaller discrepancy does not matter for the conclusion, since neither value supports P < 0.001.
Figure
Figure 1. The anhedonia–TSPO correlation in the PTSD group (N = 15) on the r scale. Orange diamonds are the r values printed in the Results text (−0.425) and in the Fig. 2 legend (−0.45), both reported with P < 0.001. Blue intervals are 95% confidence intervals recomputed from each r with the Fisher z transform at n = 15 (−0.77 to +0.11 and −0.78 to +0.08), with the corresponding two-sided P values (0.114 and 0.092). The shaded orange region (|r| > 0.760) is the set of r values that could produce two-sided P < 0.001 with 15 participants. The thin grey line marks r = 0. Both intervals cross zero, and neither reported r is in the region the reported P value requires.
Figure 2. Recomputed P values under each alternative reading of the reported statistic, on a log scale. Filled circles use r = −0.425 (text); open squares use r = −0.45 (legend). The shaded orange region left of the dashed line is the reported P < 0.001; the grey line marks P = 0.05. Every reading that uses the stated 15 participants, or the 12 with CAPS-5 totals, gives P between 0.006 and 0.25. Only treating about 60 data points as independent (for example four regions × 15 participants, which the paper does not describe and which contradicts "N = 15" in the legend) gives P < 0.001.
Why it matters
- It is the title claim. "Neuroimmune suppression in PTSD is associated with anhedonia" rests on this one test. The same association is restated in the abstract ("Greater anhedonic symptoms in the PTSD group were associated with a more suppressed neuroimmune response"), in the abstract's closing claim that the suppressed response "is associated with the severity of symptoms", in the introduction's summary of findings, in the Discussion ("Another notable finding…"), and in the conclusion, which links it to restoring neuroimmune function as a possible treatment.
- The significance changes, not just the precision. On the paper's own numbers the association is not significant at the conventional 0.05 level (two-sided P = 0.09 to 0.11), and the 95% CI for r runs from a strong negative correlation to a weak positive one. The data are consistent with a moderate negative association, but they do not establish one.
- It was one of several symptom tests. The Methods describe sum scores for seven hybrid-model symptom dimensions, and the Results report that PCL-5, PCL-IV and full CAPS scores were not significantly correlated. No correction for multiple comparisons is described. A correction would raise the P value further.
- What is not affected. The primary result, the smaller LPS-induced TSPO increase in PTSD vs. controls (P = 0.021; Cohen d = 0.89), is a separate test and is not touched by this recalculation. The CRP–TSPO correlations also reproduce from their r and n (see the side observation below).
Explanations considered
- A different N. Not supported. The Fig. 2 legend says "N = 15 participants in the PTSD group", and Table 1 lists 15 PTSD participants. If only the 12 participants with CAPS-5 totals (Table 1) were analysed, P would be larger (0.17 / 0.14), not smaller.
- A one-sided test. Does not explain it. One-sided P is 0.057 / 0.046, still about 50 times the reported bound.
- Spearman instead of Pearson. Does not explain it. The usual t approximation for Spearman's ρ is the same formula. Under the exact permutation null at n = 15, P(|ρ| ≥ 0.45) = 0.094, and P < 0.001 needs |ρ| ≥ 0.775. The legend calls the analysis "linear regression".
- The printed value is R², not r. Does not explain it. If 0.425 or 0.45 were R², |r| would be 0.65 to 0.67 and P = 0.006 to 0.009, still not below 0.001. The text and legend both label the value r.
- A partial correlation or standardized coefficient from the multivariable model in Methods step 4. Does not explain it. Adjusting for 3 or 6 other symptom scores removes degrees of freedom and gives P = 0.14 to 0.25. A standardized β of −0.425 could reach P < 0.001 only if the whole model explained more than about 91% (4 predictors) or 96% (7 predictors) of the variance in TSPO change, even in the best case of uncorrelated predictors. Correlated symptom scores would require more. The legend describes a bivariate regression plotted with a 95% CI band, which matches a simple correlation.
- Pooling region-level values. This is the only reading I found that gives P < 0.001: about 51 to 57 or more points analysed as independent, for example four regions × 15 participants. The paper does not describe this. It defines the circuit as one gray-matter-weighted average per participant, the legend states N = 15, and such pooling would treat repeated measures from the same person as independent.
- Rounding of r. Does not explain the P value. The difference between −0.425 and −0.45 is larger than rounding, but at n = 15 any r between them gives P ≈ 0.09 to 0.11.
- A conversion or typesetting artifact. Ruled out. The PMC HTML page and the Europe PMC JATS XML both carry "r = −0.425, P < 0.001" and "r = −0.45; P < 0.001", with an encoded less-than sign and a true minus sign. The publisher's Fig. 2 image itself prints "r = -0.425, p<0.001" on the panel. Its 95% CI band is wide enough to contain a horizontal line, which is what a slope that is not significant at 0.05 looks like, and approximate readings of the plotted points give a correlation of about the printed size. So the printed r appears to describe the plotted data, and the P value is the number that does not fit.
- A correction already issued. None found (see Status).
- Side observation, not part of the main finding. In the CRP analysis the Results give r = −0.72, P = 0.019 for the PTSD subgroup (n = 10), and the Fig. 3 legend gives the same r with "P = 0.19". The recomputed value is P = 0.019, so the legend's 0.19 is very likely a typographical slip. The pooled CRP correlation (r = −0.58, n = 14) recomputes to P = 0.030, matching the text's 0.029; the legend gives 0.02. Separately, the text describes higher baseline CRP as associated with "greater severity of negative affect symptoms" but reports a negative coefficient (r = −0.61); the sign or the wording may need checking. None of these affects the main finding.
Status
- Version of record. The publisher's pages (pnas.org, reached through doi.org) returned HTTP 403 to automated requests. I checked the published text through its deposited copies: the PMC HTML page (https://pmc.ncbi.nlm.nih.gov/articles/PMC11363315/), the Europe PMC full-text JATS XML, and the publisher's Fig. 2 image from the Europe PMC supplementary bundle. All agree with the passages quoted here. Every quotation was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The only SI file,
pnas.2406005121.sapp.pdf(9 pages, "Appendix 01"), holds a cytokine time-course figure, regional VT before and after LPS (Table S1) and cytokine data (Table S2). It has no anhedonia, CAPS, correlation or regression analysis and no participant-level data. No data repository is cited. The supplement does not resolve the discrepancy. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title with "PubPeer" found no thread. Two later items discuss the paper: a PNAS letter, "Microglia-mediated neuroimmune suppression in posttraumatic stress disorder" (doi:10.1073/pnas.2417472121, PMC11621761), which concerns the interpretation of the TSPO signal, and a research highlight in Psychoradiology (doi:10.1093/psyrad/kkaf004). Neither, as far as I could see, addresses the anhedonia statistic. - Sources. PMC article PMC11363315; Europe PMC full-text XML and supplementary files for PMC11363315; SI Appendix
pnas.2406005121.sapp.pdf; Crossref; Europe PMC; PubPeer; PMC11621761; https://journal.hep.com.cn/psyrad/EN/10.1093/psyrad/kkaf004; https://doi.org/10.1073/pnas.2406005121.
Suggested correction
The authors could report the exact test behind the anhedonia association: whether r is Pearson or Spearman, the n analysed, the degrees of freedom, any covariates, and the unit of analysis (one circuit value per participant or region-level values). If the analysis is the bivariate regression on 15 participants described in the Fig. 2 legend, the Results sentence would read
… was associated with a more “blunted” neuroimmune TSPO response to LPS (r = −0.425, P = 0.11) (Fig. 2).
with the Fig. 2 legend and panel changed to match (one r value, the same in both places, and P ≈ 0.1 rather than P < 0.001), and "significantly associated" removed from the legend. The 95% CI for r (about −0.77 to +0.11) could be added. The abstract, introduction, Discussion and conclusion statements that anhedonia, or symptom severity, is associated with a more suppressed neuroimmune response would then need to be described as a non-significant, exploratory trend, and the title revisited accordingly. Separately, the Fig. 3 legend's "P = 0.19" for the PTSD-only CRP correlation would read "P = 0.019", matching the Results.
Recomputation script (python recompute.py.txt)
"""Recompute the anhedonia-TSPO correlation test in PMC11363315
(Proc Natl Acad Sci U S A 2024, doi:10.1073/pnas.2406005121).
Run: python recompute.py.txt
Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png)
into the folder that holds this script. Inputs are only numbers printed in the paper.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
# ---- numbers printed in the paper ----------------------------------------
R_TEXT = -0.425 # Results, "PTSD Is Associated with Lower TSPO Response...": r = -0.425, P < 0.001
R_LEG = -0.45 # Fig. 2 legend: r = -0.45; P < 0.001, "N = 15 participants in the PTSD group"
N = 15 # Fig. 2 legend; Table 1 PTSD (N = 15)
N_CAPS5 = 12 # Table 1: "CAPS-5 total * N = 12"
P_REPORTED = 0.001 # both places: P < 0.001
def p_from_r(r, n, k=0):
"""t test of a (partial) correlation with k extra covariates; returns t, df, two-sided p."""
df = n - 2 - k
t = r * np.sqrt(df) / np.sqrt(1 - r * r)
return t, df, 2 * stats.t.sf(abs(t), df)
def r_crit(n, alpha=0.001, k=0):
df = n - 2 - k
t = stats.t.isf(alpha / 2, df)
return t / np.sqrt(df + t * t)
def fisher_ci(r, n, level=0.95):
z = np.arctanh(r)
h = stats.norm.isf((1 - level) / 2) / np.sqrt(n - 3)
return np.tanh(z - h), np.tanh(z + h)
res = {}
print("1. Bivariate correlation test, N = 15 (df = 13)")
for lab, r in (("text", R_TEXT), ("legend", R_LEG)):
t, df, p = p_from_r(r, N)
lo, hi = fisher_ci(r, N)
res[lab] = dict(r=r, t=t, p=p, lo=lo, hi=hi)
print(f" {lab:6s} r = {r:+.3f}: t = {t:.3f}, df = {df}, two-sided p = {p:.4f}, "
f"one-sided p = {p/2:.4f}; 95% CI for r (Fisher z) = {lo:+.3f} to {hi:+.3f}")
print("\n2. What P < 0.001 would require")
tc = stats.t.isf(P_REPORTED / 2, N - 2)
rc15 = r_crit(N)
print(f" two-sided P < 0.001 at df = 13 needs |t| > {tc:.3f}, i.e. |r| > {rc15:.3f}")
print(f" at N = 12 (df = 10) it needs |r| > {r_crit(N_CAPS5):.3f}")
for r in (R_TEXT, R_LEG):
nmin = next(n for n in range(5, 1000) if p_from_r(r, n)[2] < P_REPORTED)
print(f" smallest N at which r = {r:+.3f} gives two-sided p < 0.001: {nmin}")
print("\n3. Alternative readings (all two-sided unless stated)")
alt = [] # (label, p_text, p_legend)
alt.append(("Bivariate, N = 15", res["text"]["p"], res["legend"]["p"]))
alt.append(("One-sided, N = 15", res["text"]["p"] / 2, res["legend"]["p"] / 2))
pz = [2 * stats.norm.sf(abs(np.arctanh(r) * np.sqrt(N - 3))) for r in (R_TEXT, R_LEG)]
print(f" Fisher z test, N = 15: p = {pz[0]:.4f} / {pz[1]:.4f}")
alt.append(("Fisher z test, N = 15", pz[0], pz[1]))
p12 = [p_from_r(r, N_CAPS5)[2] for r in (R_TEXT, R_LEG)]
print(f" N = 12 (CAPS-5 subset of Table 1): p = {p12[0]:.4f} / {p12[1]:.4f}")
alt.append(("N = 12 (CAPS-5 subset)", p12[0], p12[1]))
pr2 = []
for R2 in (abs(R_TEXT), abs(R_LEG)):
r = -np.sqrt(R2)
t, df, p = p_from_r(r, N)
pr2.append(p)
print(f" if {R2} were R^2: r = {r:.3f}, t = {t:.3f}, p = {p:.4f}")
alt.append(("Value read as R², N = 15", pr2[0], pr2[1]))
for k in (3, 6):
pk = [p_from_r(r, N, k)[2] for r in (R_TEXT, R_LEG)]
print(f" partial r with {k} other covariates (df = {N-2-k}): p = {pk[0]:.4f} / {pk[1]:.4f}")
alt.append((f"Partial r, {k} covariates", pk[0], pk[1]))
p60 = [p_from_r(r, 60)[2] for r in (R_TEXT, R_LEG)]
t60 = [p_from_r(r, 60)[0] for r in (R_TEXT, R_LEG)]
print(f" hypothetical 60 independent points (4 regions x 15; not described in the paper): "
f"t = {t60[0]:.3f} / {t60[1]:.3f}, p = {p60[0]:.5f} / {p60[1]:.5f}")
alt.append(("60 pooled points\n(not described)", p60[0], p60[1]))
rng = np.random.default_rng(1)
perm = np.array([stats.spearmanr(np.arange(N), rng.permutation(N))[0] for _ in range(200000)])
print(f" Spearman exact null (200,000 permutations, n = 15): P(|rho| >= 0.45) = {np.mean(np.abs(perm) >= 0.45 - 1e-12):.4f}; "
f"|rho| needed for P < 0.001 = {np.quantile(np.abs(perm), 0.999):.3f}")
print("\n4. Standardized beta in a multivariable model, best case (predictors uncorrelated)")
print(" t = b*sqrt(df)/sqrt(1 - R^2_model); model R^2 needed for |t| > critical value")
for k, lab in ((4, "4 DSM-5 clusters (Methods, step 4)"), (7, "7 hybrid-model dimensions")):
df = N - k - 1
tcrit = stats.t.isf(P_REPORTED / 2, df)
for b in (abs(R_TEXT), abs(R_LEG)):
need = 1 - b * b * df / tcrit ** 2
print(f" {lab}: df = {df}, |t| > {tcrit:.3f}; b = {b}: model R^2 must exceed {need:.3f}")
print("\n5. Side check: CRP vs TSPO (Spearman, t approximation)")
for r, n, rep in ((-0.58, 14, "P = 0.029 (text), P = 0.02 (Fig. 3 legend)"),
(-0.72, 10, "P = 0.019 (text), P = 0.19 (Fig. 3 legend)")):
t, df, p = p_from_r(r, n)
print(f" r = {r}, n = {n}: t = {t:.3f}, p = {p:.4f}; reported {rep}")
# ---- figures ---------------------------------------------------------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": TXT, "ytick.color": TXT, "axes.edgecolor": SEC,
"font.family": "DejaVu Sans", "svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, bbox_inches="tight", **kw)
plt.close(fig)
# Figure 1: r with recomputed 95% CI vs the r range that P < 0.001 requires
fig, ax = plt.subplots(figsize=(7.2, 3.0))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
ax.axvspan(-1, -rc15, color=ORANGE, alpha=0.13, lw=0)
ax.axvline(-rc15, color=ORANGE, lw=1.2, ls="--")
ax.text(-rc15 + 0.03, 1.62, f"\u2190 Range of r that the reported\n P < 0.001 requires at N = 15\n (|r| > {rc15:.3f})",
color=ORANGE, va="top", ha="left", fontsize=10)
ax.axvline(0, color=SEC, lw=0.9)
ax.text(0.01, 1.62, "r = 0 (no association)", color=SEC, va="top", ha="left", fontsize=10)
rows = [("Results text\nr = −0.425, “P < 0.001”", res["text"]),
("Fig. 2 legend\nr = −0.45, “P < 0.001”", res["legend"])]
for i, (lab, d) in enumerate(rows):
y = 0.6 - i * 0.6
ax.plot([d["lo"], d["hi"]], [y, y], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot([d["lo"], d["hi"]], [y, y], "|", color=BLUE, ms=12, mew=2)
ax.plot(d["r"], y, "D", color=ORANGE, ms=9, zorder=3)
ax.text(d["hi"] + 0.03, y, f"95% CI {d['lo']:+.2f} to {d['hi']:+.2f}\nrecomputed p = {d['p']:.3f}".replace("-", "\u2212"),
color=BLUE, va="center", ha="left", fontsize=10)
ax.set_yticks([0.6, 0.0]); ax.set_yticklabels([r[0] for r in rows])
ax.set_ylim(-0.4, 1.7); ax.set_xlim(-1, 0.75)
ax.set_xlabel("Correlation r between anhedonia score and % change in [11C]PBR28 VT (PTSD group, N = 15)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "D", color=ORANGE, ms=9, label="r reported in the paper")
h2, = ax.plot([], [], "-", color=BLUE, lw=2, label="95% CI recomputed (Fisher z, N = 15)")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.28), ncol=2,
frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: recomputed p under each reading vs the reported P < 0.001
fig, ax = plt.subplots(figsize=(7.2, 4.3))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
ax.set_xscale("log"); ax.set_xlim(1e-4, 1)
ax.axvspan(1e-4, P_REPORTED, color=ORANGE, alpha=0.13, lw=0)
ax.axvline(P_REPORTED, color=ORANGE, lw=1.2, ls="--")
ax.axvline(0.05, color=SEC, lw=0.9)
ny = len(alt)
for i, (lab, pa, pb) in enumerate(alt):
y = ny - 1 - i
ax.plot([min(pa, pb), max(pa, pb)], [y, y], color=BLUE, lw=2, alpha=0.5)
ax.plot(pa, y, "o", color=BLUE, ms=8, zorder=3)
ax.plot(pb, y, "s", mfc=BG, mec=BLUE, mew=2, ms=8, zorder=3)
ax.set_yticks(range(ny)); ax.set_yticklabels([a[0] for a in alt][::-1])
ax.set_ylim(-0.7, ny + 0.6)
ax.text(1.1e-4, ny - 0.1, "Reported:\nP < 0.001", color=ORANGE, ha="left", va="center", fontsize=10)
ax.text(0.055, ny - 0.1, "P = 0.05", color=SEC, ha="left", va="center", fontsize=10)
ax.set_xlabel("Two-sided p-value (log scale); one-sided row as labelled")
ax.grid(axis="x", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "o", color=BLUE, ms=8, label="recomputed from r = −0.425 (text)")
h2, = ax.plot([], [], "s", mfc=BG, mec=BLUE, mew=2, ms=8, label="recomputed from r = −0.45 (legend)")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.4, -0.13), ncol=2,
frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written to", HERE)
Abstract states P < 0.05 for all three SGA sex differences; two exceed 0.05 in Table 3
Am J Epidemiol (2018) · doi:10.1093/aje/kwy141 · PMC6211242
Summary
The abstract says that the associations of being small for gestational age (SGA) with height-for-age (HAZ), weight-for-age (WAZ) and neuropsychological development (ASQ z score) were larger for male than for female infants "(P < 0.05 for all)". The paper's own tests of that sex difference (Table 3, row "H0: No sex difference for SGA"; Results, third paragraph) give P = 0.063 for HAZ, 0.059 for WAZ and 0.043 for ASQ. Only ASQ is below 0.05, and the Results text and the Table 3 footnotes themselves label HAZ and WAZ as significant only at the 10% level. A Wald test rebuilt from the sex-specific coefficients and 95% CIs in Table 3 reproduces the three values (0.064, 0.060, 0.044) as two-sided P values. The abstract therefore overstates the statistical support for the paper's central sex-specific finding for two of the three outcomes. The estimates themselves are not in question.
What the paper reports
Abstract:
For being small for gestational age, associations with height-for-age, weight-for-age, and neuropsychological development were also negative, but they were systematically larger for male than for female infants (P < 0.05 for all). These results suggest that male fetuses may be more vulnerable to intrauterine adversity than female fetuses.
Methods, "Statistical analysis and empirical strategy":
Finally, we tested whether the observed sex-specific differences were statistically significant in pooled linear models with interaction terms. We used robust standard errors to account for any heteroskedasticity in the data.
Results, third paragraph:
The estimated coefficients for HAZ and WAZ were more than twice as large for boys as for girls, and the interaction term was significant at the 10% level (P = 0.063 and 0.059, respectively). Estimated sex differences with respect to SGA were largest for ASQ scores, which showed no evidence of associations with SGA for girls but large negative associations for boys (z score difference, -0.55, 95% confidence intervals: 1.11, 0.01; statistically significant at the 10% level). We rejected the null of equal associations with a P value of 0.043.
Table 3, SGA rows and the test row (coefficient, 95% CI; columns HAZ, WAZ, ASQ z score). Cell contents are verbatim; the "Girls ·" and "Boys ·" prefixes stand for the table's group headings, and each P value, which spans two columns in the table, is shown once:
Girls · SGA | -0.485b | -0.839, -0.130 | -0.509b | -0.830, -0.188 | 0.094 | -0.180, 0.368
Boys · SGA | -1.108b | -1.662, -0.553 | -1.177b | -1.794, -0.559 | -0.549d | -1.110, 0.013
H0: No sex difference for SGA (P value) | 0.063d | 0.059d | 0.043c
Table 3 footer (excerpt):
The 95% CIs are based on heteroskedasticity-robust standard errors. P values are based on a pooled model with linear interaction terms.bP < 0.01.cP < 0.05.dP < 0.1.
What the paper's own numbers show
Method. I compared the abstract's "P < 0.05 for all" with the three interaction P values printed in Table 3. To check that those printed values are ordinary two-sided P values, and not one-sided values that a reader might halve, I rebuilt an approximate Wald test from the stratified estimates in Table 3. Each stratum's standard error is (upper − lower) / (2 × 1.96) from its robust 95% CI. The boys − girls difference has SE √(SEgirls2 + SEboys2), since the strata are independent samples, and P = 2(1 − Φ(|z|)). This approximation does not use the authors' pooled model, which constrains the covariate coefficients to be equal across sexes, so small differences from the printed P values are expected. I applied the same check to the prematurity rows as a control. I also tested the alternative reading that "P < 0.05" refers to the SGA coefficients themselves, and adjusted the three interaction P values for multiplicity. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Sex difference in SGA association, HAZ | Table 3 test row; Results ¶3; Abstract | 0.063 (footnote d, P < 0.1); abstract: P < 0.05 | 0.064 two-sided (difference −0.623, SE 0.336, z = −1.86); not below 0.05 |
| Sex difference in SGA association, WAZ | Table 3 test row; Results ¶3; Abstract | 0.059 (footnote d, P < 0.1); abstract: P < 0.05 | 0.060 two-sided (difference −0.668, SE 0.355, z = −1.88); not below 0.05 |
| Sex difference in SGA association, ASQ | Table 3 test row; Results ¶3; Abstract | 0.043 (footnote c, P < 0.05) | 0.044 two-sided (difference −0.643, SE 0.319, z = −2.02); below 0.05 |
| Outcomes with interaction P < 0.05 | Abstract vs Table 3 | "for all" (3 of 3) | 1 of 3 |
| 95% CI of boys − girls difference (approximate) | from Table 3 CIs | not reported | HAZ −1.281 to 0.035; WAZ −1.364 to 0.028; ASQ −1.268 to −0.018 |
| One-sided equivalents | — | not stated anywhere | 0.032, 0.030, 0.022 (Wald); reported ÷ 2 = 0.0315, 0.0295, 0.0215 |
| Control: sex difference for prematurity | Table 3 test row | 0.354, 0.730, 0.705 | 0.355, 0.732, 0.704 two-sided |
| Boys / girls SGA coefficient ratio | Table 3; Results "more than twice" | — | HAZ 2.28, WAZ 2.31 |
| SGA coefficient P, girls | Table 3 | HAZ and WAZ footnote b (P < 0.01); ASQ unmarked | HAZ 0.0073, WAZ 0.0019, ASQ 0.50 |
| SGA coefficient P, boys | Table 3 | HAZ and WAZ footnote b; ASQ footnote d (P < 0.1) | HAZ 0.0001, WAZ 0.0002, ASQ 0.055 |
| Pooled adjusted SGA coefficient, ASQ | Table 2 | −0.152 (−0.415, 0.112), unmarked | P = 0.26 |
| Interaction P adjusted for 3 tests | from Table 3 | — | Bonferroni 0.189, 0.177, 0.129; Holm 0.129 for all three; Benjamini–Hochberg 0.063 for all three |
The Wald approximation matches each printed interaction P value to within 0.002, for SGA and for the prematurity control alike. The printed values are therefore two-sided P values, and the paper's own labels agree: footnote d (P < 0.1) on 0.063 and 0.059, footnote c (P < 0.05) on 0.043, and "significant at the 10% level" in the Results. On the alternative reading, "P < 0.05 for all" also fails: the SGA–ASQ coefficient is not significant at 0.05 in girls (P = 0.50), in boys (P = 0.055, CI −1.110 to 0.013) or in the pooled sample (P = 0.26). Under no reading do all three outcomes meet P < 0.05.
Figure
Figure 1. Left: the abstract sentence, whose final clause "(P < 0.05 for all)" covers all three outcomes. Right: the row "H0: No sex difference for SGA" from Table 3, with the reported P values (orange, bold) and the two-sided P values recomputed from the Table 3 sex-specific coefficients and 95% CIs (blue). The connector marks the conflict: two of the three reported P values, for HAZ and WAZ, are above 0.05. The Results text and the Table 3 footnotes give the same 10%-level classification. All numbers are from the paper or from recompute.py.txt.
Figure 2. P value for the null of no sex difference in the SGA association, for each outcome, on a linear axis from 0. Orange circles: values reported in Table 3. Blue squares: two-sided values recomputed from the Table 3 CIs. The solid line at 0.05 is the threshold the abstract states for all three; the shaded region left of it is P < 0.05; the dashed line marks 0.10. The reported and recomputed values agree to within 0.002, and HAZ and WAZ fall between 0.05 and 0.10 on both.
Why it matters
- The abstract overstates the evidence for the paper's main hypothesis. The paper is framed as a test of whether in-utero adaptation differs by sex. The interaction tests are its direct evidence, and the abstract reports all three as P < 0.05 when the paper's own table gives 0.063, 0.059 and 0.043. Readers who rely on the abstract, or on indexing services that reproduce it, take away three significant sex differences instead of one.
- The conclusion rests mostly on these tests. The sex differences for prematurity were not significant (P = 0.354, 0.730, 0.705), so the closing statement that "adverse birth outcomes may be more harmful to the development of male infants than to the development of female infants" rests mainly on the SGA interactions.
- Under multiplicity adjustment none of the three is below 0.05. With three outcomes tested, Bonferroni gives 0.189, 0.177 and 0.129, and Holm and Benjamini–Hochberg give 0.129 and 0.063 respectively for all three. The paper does not claim a multiplicity correction; the point is only that the statement "P < 0.05 for all" is further from the paper's evidence than the 1-of-3 count alone suggests.
- The estimates are not affected. The boys − girls differences (about −0.62, −0.67 and −0.64 SD) and the ratio of more than two for HAZ and WAZ are as printed. The abstract's hedged wording ("may be more vulnerable") remains a fair summary of a suggestive pattern. What changes is the significance level attached to it.
- A related wording point. The abstract calls the SGA associations with neuropsychological development "also negative", but the pooled adjusted coefficient is −0.152 (P = 0.26) and the girls' coefficient is +0.094 (P = 0.50). The Discussion describes the same result as "no association with neuropsychological development" overall.
Explanations considered
- The printed P values are one-sided. Does not account for it. Halving them would put all three below 0.05 (0.0315, 0.0295, 0.0215), but the Wald test rebuilt from the Table 3 CIs gives two-sided P values of 0.064, 0.060 and 0.044, which match the printed 0.063, 0.059 and 0.043. The same check reproduces the prematurity P values as two-sided. The Methods and table footers never mention one-sided tests, and the Results call HAZ and WAZ "significant at the 10% level".
- "P < 0.05" refers to the SGA associations themselves, not to the sex difference. Does not account for it. The SGA–ASQ association is not significant at 0.05 for boys (P = 0.055; CI −1.110 to 0.013 includes 0), for girls (P = 0.50) or pooled (P = 0.26).
- The abstract summarizes a different model, population or SGA standard. Not supported. The abstract summarizes the final adjusted sex-specific models, which are the Table 3 models. The Discussion says the sex-specific associations "do not change qualitatively" under alternative SGA standards, and the Web Material (Web Tables 1 and 2, INTERGROWTH and Oken standards) gives sex-specific coefficients but no interaction P values. No P value below 0.05 for HAZ or WAZ appears anywhere in the paper or supplement.
- Footnote conventions or a text-conversion artifact. Ruled out. Table 3 marks 0.063 and 0.059 with footnote d (P < 0.1) and 0.043 with c (P < 0.05). The publisher's HTML page, the PMC page and the Europe PMC JATS XML all carry "(P < 0.05 for all)" in the abstract and the same Table 3 values and footnotes. (Table 2 uses the opposite letter assignment, c = P < 0.1 and d = P < 0.05, but Table 3 defines its own footnotes and uses them consistently.)
- A correction already issued. None found (see Status).
- Side observation, not part of the main finding. The Results give the boys' SGA–ASQ interval as "95% confidence intervals: 1.11, 0.01"; Table 3 gives −1.110, 0.013. The minus sign on the lower limit is missing in the text.
Status
- Version of record. doi.org/10.1093/aje/kwy141 resolves to the publisher's page (academic.oup.com/aje/article/187/11/2324/5047920). Its abstract ends "...systematically larger for male than for female infants (P < 0.05 for all)", and its Results text and Table 3 carry the same P values and footnotes as quoted above. The PMC HTML page and the Europe PMC full-text XML agree. Every quotation here was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The single Web Material PDF (from the Europe PMC supplementary bundle) holds Web Tables 1 and 2, the sex-specific SGA and preterm coefficients under INTERGROWTH and Oken standards. It reports no interaction P values and nothing that supports "P < 0.05 for all". The other two files in the bundle are images for Figure 1.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.py(re-run 2026-10-24) returns no Crossref update relations, PMC notices or Europe PMC notices. The Crossref record has empty update-to, updated-by and relation fields, a Crossref filter for updates to this DOI returns nothing, and neither the OUP article page nor the PMC page shows a correction link. The Europe PMC REST search returned HTTP 503 on repeated attempts, so its comment/correction list was checked only throughsrc/known.py. PubPeer could not be queried automatically (its API requires human verification); web searches found no PubPeer thread and no Retraction Watch entry for this article. - Sources. PMC article PMC6211242; https://doi.org/10.1093/aje/kwy141 (publisher page); https://pmc.ncbi.nlm.nih.gov/articles/PMC6211242/; Europe PMC full-text XML and supplementary files for PMC6211242; Crossref; PubPeer and Retraction Watch (web search).
Suggested correction
In the abstract, replace
but they were systematically larger for male than for female infants (P < 0.05 for all).
with, for example,
but they were systematically larger for male than for female infants (P for sex difference = 0.063, 0.059, and 0.043, respectively).
or, keeping a threshold form, "(P < 0.1 for all; P < 0.05 for neuropsychological development)". The authors may also wish to revise "were also negative" for neuropsychological development, since that association was not significant overall (−0.152; 95% CI −0.415, 0.112) and was positive though not significant in girls (0.094). In the third paragraph of the Results, "95% confidence intervals: 1.11, 0.01" should read "95% confidence intervals: -1.11, 0.01", as in Table 3.
Recomputation script (python recompute.py.txt)
"""Recompute the sex-difference tests for small-for-gestational-age (SGA) in PMC6211242
(Am J Epidemiol 2018, doi:10.1093/aje/kwy141, "Overall and Sex-Specific Associations
Between Fetal Adversity and Child Development at Age 1 Year: Evidence From Brazil").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 3: sex-specific SGA and preterm coefficients with heteroskedasticity-robust 95% CIs,
and the rows "H0: No sex difference for SGA (P value)" and
"H0: No sex difference for prematurity (P value)".
Table 2: pooled adjusted SGA coefficients with 95% CIs.
Abstract: "... systematically larger for male than for female infants (P < 0.05 for all)."
Method for the cross-check. The paper's P values come from a pooled model with linear
interaction terms; the data and covariance matrix are not published. An approximation
uses only the printed numbers: SE = (upper - lower) / (2 * 1.959964) for each stratum,
difference = boys - girls, SE_diff = sqrt(SE_girls^2 + SE_boys^2) (the strata are
independent samples), z = difference / SE_diff, two-sided P = 2 * (1 - Phi(|z|)).
The pooled interaction model constrains covariate coefficients to be equal across sexes,
so its P value need not match this exactly; close agreement shows which test (one- or
two-sided) the printed values correspond to.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs
# Table 3: (coefficient, lower, upper) for girls and boys; printed interaction P; footnote
T3_SGA = {
"HAZ": ((-0.485, -0.839, -0.130), (-1.108, -1.662, -0.553), 0.063, "d (P < 0.1)"),
"WAZ": ((-0.509, -0.830, -0.188), (-1.177, -1.794, -0.559), 0.059, "d (P < 0.1)"),
"ASQ": ((0.094, -0.180, 0.368), (-0.549, -1.110, 0.013), 0.043, "c (P < 0.05)"),
}
T3_PRETERM = {
"HAZ": ((-0.043, -0.444, 0.358), (0.275, -0.266, 0.816), 0.354),
"WAZ": ((0.112, -0.214, 0.438), (0.212, -0.257, 0.682), 0.730),
"ASQ": ((-0.507, -1.001, -0.013), (-0.392, -0.722, -0.062), 0.705),
}
# Table 2, adjusted estimates, SGA row
T2_SGA_ADJ = {"HAZ": (-0.728, -1.038, -0.418), "WAZ": (-0.783, -1.114, -0.453),
"ASQ": (-0.152, -0.415, 0.112)}
ABSTRACT_THRESHOLD = 0.05
def se_from_ci(lo, hi):
return (hi - lo) / (2 * Z975)
def p_two(est, se):
return 2 * norm.sf(abs(est / se))
def wald_diff(g, b):
sg, sb = se_from_ci(g[1], g[2]), se_from_ci(b[1], b[2])
d = b[0] - g[0]
sd = np.hypot(sg, sb)
z = d / sd
return dict(se_g=sg, se_b=sb, diff=d, se=sd, z=z, p2=p_two(d, sd), p1=norm.sf(abs(z)),
lo=d - Z975 * sd, hi=d + Z975 * sd)
# ---------------------------------------------------------------- 1. reported P vs 0.05
print("=== 1. Table 3 'H0: No sex difference for SGA' P values vs the abstract's 'P < 0.05 for all' ===")
for k, (g, b, prep, foot) in T3_SGA.items():
print(f"{k}: reported P = {prep:.3f} footnote {foot:13s} below 0.05? {prep < ABSTRACT_THRESHOLD}")
n_below = sum(v[2] < ABSTRACT_THRESHOLD for v in T3_SGA.values())
print(f"Outcomes with reported interaction P < 0.05: {n_below} of {len(T3_SGA)}")
# ---------------------------------------------------------------- 2. Wald cross-check
print("\n=== 2. Approximate Wald test of boys - girls, from the Table 3 CIs ===")
print(f"{'':4s} {'SE girls':>8s} {'SE boys':>8s} {'diff':>7s} {'SE diff':>8s} {'z':>6s} "
f"{'95% CI of diff':>18s} {'P 2-sided':>9s} {'P 1-sided':>9s} {'reported':>8s} {'reported/2':>10s}")
W = {}
for k, (g, b, prep, _) in T3_SGA.items():
w = wald_diff(g, b)
W[k] = w
print(f"{k:4s} {w['se_g']:8.3f} {w['se_b']:8.3f} {w['diff']:7.3f} {w['se']:8.3f} {w['z']:6.2f} "
f"({w['lo']:7.3f}, {w['hi']:6.3f}) {w['p2']:9.3f} {w['p1']:9.3f} {prep:8.3f} {prep / 2:10.4f}")
print("\nSame check for prematurity (reported P values 0.354, 0.730, 0.705):")
for k, (g, b, prep) in T3_PRETERM.items():
w = wald_diff(g, b)
print(f"{k}: diff = {w['diff']:.3f}, SE = {w['se']:.3f}, P two-sided = {w['p2']:.3f}, "
f"one-sided = {w['p1']:.3f}; reported {prep:.3f}")
# ---------------------------------------------------------------- 3. ratios
print("\n=== 3. Boys / girls ratio of SGA coefficients (Results: 'more than twice as large') ===")
for k in ("HAZ", "WAZ"):
g, b = T3_SGA[k][0][0], T3_SGA[k][1][0]
print(f"{k}: {b} / {g} = {b / g:.2f}")
# ---------------------------------------------------------------- 4. alternative reading
print("\n=== 4. Alternative reading: are the SGA coefficients themselves P < 0.05? ===")
for k, (g, b, _, _) in T3_SGA.items():
print(f"{k}: girls P = {p_two(g[0], se_from_ci(g[1], g[2])):.4f} "
f"boys P = {p_two(b[0], se_from_ci(b[1], b[2])):.4f}")
for k, (c, lo, hi) in T2_SGA_ADJ.items():
print(f"Table 2 pooled adjusted SGA, {k}: coefficient {c}, P = {p_two(c, se_from_ci(lo, hi)):.2g}")
# ---------------------------------------------------------------- 5. multiplicity
print("\n=== 5. The three SGA interaction P values adjusted for three tests ===")
ks = list(T3_SGA)
p = np.array([T3_SGA[k][2] for k in ks])
m = len(p)
bonf = np.minimum(1, p * m)
order = np.argsort(p)
holm = np.empty(m)
running = 0
for rank, i in enumerate(order):
running = max(running, min(1, (m - rank) * p[i]))
holm[i] = running
bh = np.empty(m)
running = 1
for rank in range(m - 1, -1, -1):
i = order[rank]
running = min(running, p[i] * m / (rank + 1))
bh[i] = running
for k, pi, a, h, q in zip(ks, p, bonf, holm, bh):
print(f"{k}: P = {pi:.3f} Bonferroni = {a:.3f} Holm = {h:.3f} Benjamini-Hochberg = {q:.3f}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: abstract vs Results/Table 3
fig = plt.figure(figsize=(7.2, 3.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 3.9)
ax.axis("off")
def panel(x, y, w, h, title, sub):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=SEC, lw=1.2))
ax.text(x + 0.14, y + h - 0.2, title, fontsize=12, fontweight="bold", va="top")
ax.text(x + 0.14, y + h - 0.47, sub, fontsize=10, color=SEC, va="top")
# left panel: abstract
panel(0.1, 0.95, 2.9, 2.8, "Abstract", "sentence on SGA")
ax.text(0.24, 2.95, "“…associations with height-for-age,\n"
"weight-for-age, and neuropsycho-\n"
"logical development were also\n"
"negative, but they were system-\n"
"atically larger for male than for\n"
"female infants", fontsize=10, va="top", linespacing=1.35)
ax.text(0.24, 1.6, "(P < 0.05 for all).”", fontsize=11.5, va="top", color=ORANGE,
fontweight="bold")
ax.text(0.24, 1.25, "states: 3 of 3 below 0.05", fontsize=10, color=SEC, va="top")
# right panel: Results / Table 3
x0 = 4.0
panel(x0, 0.95, 3.1, 2.8, "Results ¶3 / Table 3", "row “H0: No sex difference for SGA”")
ax.text(x0 + 1.32, 2.98, "reported", fontsize=10, color=ORANGE, ha="center", fontweight="bold")
ax.text(x0 + 2.38, 2.98, "recomputed¹", fontsize=10, color=BLUE, ha="center", fontweight="bold")
rows = [("HAZ", "HAZ"), ("WAZ", "WAZ"), ("ASQ", "ASQ")]
for i, (k, lab) in enumerate(rows):
yy = 2.62 - i * 0.36
ax.text(x0 + 0.14, yy, lab, fontsize=10.5, fontweight="bold", va="center")
ax.text(x0 + 1.32, yy, f"P = {T3_SGA[k][2]:.3f}", fontsize=10.5, color=ORANGE, ha="center",
va="center", fontweight="bold")
ax.text(x0 + 2.38, yy, f"P = {W[k]['p2']:.3f}", fontsize=10.5, color=BLUE, ha="center",
va="center")
ax.text(x0 + 0.14, 1.6, "Results: HAZ and WAZ “significant at\nthe 10% level”. Footnotes: d (P < 0.1)\n"
"on HAZ and WAZ; c (P < 0.05) on ASQ", fontsize=10,
va="top", color=SEC, linespacing=1.3)
# connector
arr = FancyArrowPatch((3.03, 2.35), (3.97, 2.35), arrowstyle="<->", mutation_scale=16,
color=TXT, lw=2)
ax.add_patch(arr)
ax.text(3.475, 2.55, "conflict", fontsize=10.5, ha="center", fontweight="bold")
ax.text(3.475, 2.17, "2 of 3\nP > 0.05", fontsize=10, ha="center", va="top", color=TXT,
linespacing=1.25)
ax.text(0.15, 0.62, "¹ Two-sided Wald test of boys − girls SGA coefficients, SEs taken from the "
"Table 3 robust 95% CIs\n (recompute.py.txt). It matches the reported values, so they "
"are two-sided.", fontsize=10, color=SEC, va="top", linespacing=1.3)
save(fig, "fig1")
# Figure 2: P values on one axis
fig, ax = plt.subplots(figsize=(7.2, 2.9), facecolor=BG)
ax.set_facecolor(BG)
labels = ["HAZ (height-for-age)", "WAZ (weight-for-age)", "ASQ (neuropsych.)"]
ys = np.array([2, 1, 0])
for spine in ("top", "right", "left"):
ax.spines[spine].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvspan(0, 0.05, color=GRID, alpha=0.35, lw=0)
ax.axvline(0.05, color=TXT, lw=1)
ax.axvline(0.10, color=SEC, lw=0.8, ls="--")
ax.text(0.0495, 2.62, "0.05: abstract says\n“P < 0.05 for all”", ha="right", va="center",
fontsize=10, color=TXT)
ax.text(0.1005, 2.62, "0.10", ha="left", va="center", fontsize=10, color=SEC)
for y, k in zip(ys, ks):
pr, pc = T3_SGA[k][2], W[k]["p2"]
ax.plot(pr, y + 0.13, "o", ms=10, color=ORANGE, zorder=3)
ax.plot(pc, y - 0.13, "s", ms=9, color=BLUE, zorder=3)
# labels sit right of the markers, or left of them when the marker is just below 0.05,
# so no label crosses the 0.05 line
for v, yy, c in ((pr, y + 0.13, ORANGE), (pc, y - 0.13, BLUE)):
left = v < 0.05
ax.text(v - 0.0025 if left else v + 0.0025, yy, f"{v:.3f}", va="center",
ha="right" if left else "left", fontsize=10, color=c,
bbox=dict(facecolor=BG, edgecolor="none", pad=0.6))
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10.5, color=TXT)
ax.tick_params(axis="y", length=0)
ax.set_xlim(0, 0.11)
ax.set_ylim(-0.5, 2.95)
ax.set_xticks([0, 0.02, 0.04, 0.05, 0.06, 0.08, 0.10])
ax.set_xticklabels(["0", "0.02", "0.04", "", "0.06", "0.08", "0.10"])
ax.set_xlabel("P value for no sex difference in the SGA association")
handles = [Line2D([], [], marker="o", ls="", ms=10, color=ORANGE, label="reported (Table 3)"),
Line2D([], [], marker="s", ls="", ms=9, color=BLUE,
label="recomputed from Table 3 CIs (two-sided)")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(-0.3, -0.46), ncol=2, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.25, right=0.97, top=0.97, bottom=0.34)
save(fig, "fig2")
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Table 3's COVID-19 death risks of 0.10% and 0.30% do not match the paper's event counts
Ann Rheum Dis (2021) · doi:10.1136/annrheumdis-2021-219845 · PMC8206171
Summary
The abstract, Results, Table 3 and Discussion give the risk of death from COVID-19 among all 110,567 patients with inflammatory joint disease (IJD) as 0.10%, against 0.07% in matched referents, a crude excess of 0.03 per 100 persons. The paper defines risk as events divided by individuals at risk on 1 March 2020, and Table 3 gives 161 COVID-19 deaths, so the risk is 161/110,567 = 0.146%. The crude excess is then 0.076 per 100 (about 7.6 per 10,000), not 0.03 (3 per 10,000), and the crude risk ratio from the counts (2.09) equals the paper's unadjusted hazard ratio. The rheumatoid arthritis (RA) row has a similar mismatch, 134/53,455 = 0.251% against a printed 0.30%. All 28 other risk cells in Table 3 match events divided by cohort size. The understated figures feed the abstract and the Discussion's patient-counselling statements. The hazard ratios and the qualitative conclusion that absolute risks are low are not affected.
What the paper reports
All quotations were checked verbatim against the PMC-archived text.
Abstract, Results:
Among patients with IJD, the risks of hospitalisation (0.5% vs 0.3% in their population referents), admission to intensive care (0.04% vs 0.03%) and death (0.10% vs 0.07%) due to COVID-19 were low.
Methods, Statistics (definition of risk):
In each cohort, we next calculated absolute risks for hospitalisation, admission to intensive care and death due to COVID-19, defined as the ratio of the number of incident events 1 March through 1 September 2020 and the number of individuals at risk in each cohort 1 March 2020. We calculated excess risks (IJD vs the general population) as the difference in risk between the IJD and its population comparator cohort, and relative risks via unadjusted and adjusted Cox models as described above.
Table 1, cohort sizes (columns RA, other IJD, all IJDs combined, matched general population referents):
Individuals* | 53 455 | 57 112 | 110 567 | 484 277
Table 3, columns: outcome | N events (risk, %) in the IJD cohort | N events (risk, %) in the general population | crude excess risk per 100 patients | HR model 1 | HR model 2. The "All" block:
Death, COVID-19 | 161 (0.10%) | 338 (0.07%) | 0.03 | 2.09 (1.73 to 2.52) | 1.18 (0.97 to 1.44)
Table 3, "RA" block:
Death, COVID-19 | 134 (0.30%) | 245 (0.11%) | 0.19 | 2.28 (1.85 to 2.81) | 1.27 (1.02 to 1.59)
Table 3, "Other IJD" block:
Death, COVID-19 | 27 (0.05%) | 93 (0.04%) | 0.01 | 1.34 (0.87 to 2.05) | 0.83 (0.54 to 1.28)
Table 3 footnote for the excess-risk column:
*Defined as the difference between the risk in the inflammatory joint disease cohort and that in its matched population comparator cohort.
Results, subsection on risks, excess risks and relative risks for COVID-19-related and other outcomes:
Among all individuals with IJD, the risk for hospitalisation because of COVID-19 during March through September 2020 was 0.5% (vs 0.3% in their general population referents), 0.04% (vs 0.03%) for admission to ICD due to COVID-19 and 0.10% (vs 0.07%) for death from COVID-19.
Discussion, first paragraph, principal finding (3):
(3) In absolute terms, the risks for admission to hospital due to COVID-19 (0.5%, an additional 0.2 per 100 persons compared with the general population), to intensive care due to COVID-19 (0.04%, an additional 0.01 per 100 persons) and for death due to COVID-19 (0.10%, an additional 0.03 per 100 persons) among patients with IJDs were low.
Discussion, third paragraph:
Our results have important implications for patient counselling in that they suggest that (1) The absolute risk of death from COVID-19 among individuals with IJD between March and September was in the order of 1 in a 1000, (2) The additional risk in individuals with IJD compared with the general population was in the order of 3 per 10000
What the paper's own numbers show
Method. I applied the paper's stated definition (events ÷ individuals at risk on 1 March 2020) to every cell of Table 3, using the cohort sizes in Table 1, and checked whether each computed risk falls inside the interval that rounds to the printed value. I then recomputed the crude excess (the difference in risks) and the crude risk ratio, and compared the ratio with the unadjusted Cox HR (model 1). Over six months with risks below 10%, the crude risk ratio and the unadjusted HR should be close. The paper does not print the sizes of the RA and other-IJD referent cohorts. I bounded them from the referent cells of Table 3, which together give 220,724 to 223,453 for RA, and used the midpoint, 222,089. No RA conclusion below depends on where in that range the value lies. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Subgroup COVID-19 deaths, IJD | Table 3 | 134 (RA) + 27 (other IJD); 161 (all) | 134 + 27 = 161; cohort sizes 53,455 + 57,112 = 110,567 |
| Risk of COVID-19 death, all IJDs | Table 3; Abstract; Results; Discussion | 0.10% | 0.146% (161/110,567); 0.10% would require 106 to 116 deaths |
| Risk of COVID-19 death, referents | Table 3; Abstract; Results | 0.07% | 0.070% (338/484,277), consistent |
| Crude excess risk, all IJDs | Table 3; Discussion | 0.03 per 100 ("3 per 10000") | 0.076 per 100 (7.58 per 10,000) |
| Absolute risk, all IJDs, per 1000 | Discussion ("1 in a 1000") | about 1 | 1.46 |
| Crude risk ratio, all IJDs | implied by Table 3 | 0.10/0.07 = 1.43 from the printed percentages | 2.09 from the counts; HR model 1 is 2.09 |
| Risk of COVID-19 death, RA | Table 3 | 0.30% | 0.251% (134/53,455); 0.30% would require 158 to 163 deaths |
| Risk of COVID-19 death, RA referents | Table 3 | 0.11% | 0.110% (245/222,089; 0.1096 to 0.1110 across the bounded range), consistent |
| Crude excess risk, RA | Table 3 | 0.19 per 100 | 0.140 per 100 (0.1397 to 0.1410) |
| Crude risk ratio, RA | implied by Table 3 | 0.30/0.11 = 2.73 from the printed percentages | 2.27 from the counts; HR model 1 is 2.28 |
| Risk of COVID-19 death, other IJD | Table 3 | 0.05% | 0.047% (27/57,112), consistent |
| The other 13 IJD-cohort risks | Table 3 | e.g. 8.1%, 0.5%, 0.04%, 1.2%, 9.9%, 0.7% | e.g. 8.114%, 0.525%, 0.041%, 1.185%, 9.868%, 0.709%: all inside their rounding intervals |
| Crude risk ratio vs HR model 1, other 13 rows | Table 3 | HR 0.78 to 2.02 | crude ratios within 0.03 of the HR in every row |
Every other risk cell in Table 3, 13 in the IJD columns and 15 in the referent columns, matches events ÷ cohort size at its printed precision. Every crude risk ratio computed from the counts is within 0.03 of the tabulated HR model 1, the two COVID-19 death rows included. The two COVID-19 death percentages, 0.10% for all IJDs and 0.30% for RA, are the only cells that disagree with their own counts. In both cases the printed excess equals the difference of the printed percentages (0.10 − 0.07 = 0.03 and 0.30 − 0.11 = 0.19), so the excess column carries the discrepancy forward.
Figure
Figure 1. COVID-19 death, 1 March to 1 September 2020, per 100 persons, on a linear axis starting at zero. Orange dots are the values printed in Table 3 (all IJDs: risk 0.10%, crude excess 0.03 per 100; RA: risk 0.30%, crude excess 0.19 per 100). Blue dots are the values recomputed from the same table's event counts and Table 1's cohort sizes, using the paper's definition of risk (all IJDs: 0.146% and 0.076; RA: 0.251% and 0.140, with the RA referent denominator bounded from Table 3 as described above). For all IJDs, the printed excess is about 2.5 times smaller than the recomputed one. For RA, the printed values are higher than the recomputed ones.
Figure 2. Each IJD-cohort risk in Table 3, shown as the recomputed risk (events ÷ cohort size) divided by the printed risk, on a log scale where 1 means exact agreement. The orange band in each row is the range of risks that round to the printed value, on the same ratio scale; it is wide for values printed to one significant figure (e.g. 0.02%) and narrow for values such as 8.1%. Blue dots inside their band are consistent with rounding. Thirteen of the 15 rows are consistent. The all-IJD COVID-19 death cell (0.146% vs printed 0.10%, ratio 1.46) and the RA COVID-19 death cell (0.251% vs printed 0.30%, ratio 0.84) are not.
Why it matters
- A headline absolute risk is understated. The abstract's only figure for COVID-19 mortality in IJD is "0.10% vs 0.07%". By the paper's own definition and counts the IJD figure is 0.15%, about 1.5 times the printed value.
- The excess risk used for patient counselling is understated about 2.5-fold. The Discussion states that the additional risk "was in the order of 3 per 10000" and the absolute risk "in the order of 1 in a 1000". The counts give about 7.6 per 10,000 and about 1.5 per 1000. The paper presents these two numbers as implications for patient counselling, so the understatement reaches the advice that the paper suggests clinicians give.
- The RA row has the reverse problem. For RA the printed risk (0.30%) and crude excess (0.19 per 100) are higher than the counts give (0.25% and 0.14 per 100).
- What is not affected. The direction of every comparison stays the same. The Cox hazard ratios (2.09 and 1.18 for all IJDs; 2.28 and 1.27 for RA) and their confidence intervals agree with the counts, as the crude ratios show, and are not changed by this finding. So are the statements about statistical precision, the all-cause mortality and hospitalisation results, and the drug-specific analyses. The conclusion that absolute and excess risks of COVID-19 death are low holds qualitatively at 0.15% and 0.08 per 100. Only the stated sizes are affected.
Explanations considered
- Rounding. Does not account for it. 0.146% rounds to 0.15% at the precision shown. Only 106 to 116 deaths among 110,567 people would round to 0.10%. For RA, only 158 to 163 deaths among 53,455 would round to 0.30%.
- A different denominator, such as person-time. Does not account for it. The paper defines risk per individual at risk on 1 March 2020, and all 28 other risk cells follow that definition. Dividing by the 55,336 person-years reported for the IJD cohort gives 0.291 per 100 person-years, not 0.10. A risk of 0.10% from 161 deaths would need a denominator of about 153,000 to 169,000, which matches none of the cohort sizes in Table 1 (53,455; 57,112; 110,567; 484,277).
- The count, not the percentage, is the wrong figure. Not supported. The all-IJD count of 161 equals the RA and other-IJD counts added together (134 + 27), and the referent count of 338 equals 245 + 93. The crude risk ratio from the counts (2.086) reproduces HR model 1 (2.09), whereas the printed percentages imply 1.43. The same holds for RA: 2.27 from the counts against an HR of 2.28, and 2.73 from the printed percentages. A count near 111 would be inconsistent with the subgroup counts and with the hazard ratio.
- A cumulative-incidence estimator with censoring (e.g. Kaplan–Meier). Does not account for it. Censoring for emigration or other reasons can only make such an estimate equal to or larger than events ÷ baseline N, never 30% smaller. The paper also states that it used events ÷ individuals at risk.
- A text-conversion artifact. Ruled out. The cells are plain decimals in the publisher-deposited JATS XML, with no superscripts, signs or special characters. The same 0.10% appears in four separate places (abstract, Results text, Table 3 and Discussion), and the PubMed abstract reads the same.
- Supplementary material with a different basis. The single supplementary file has no alternative denominator or outcome table for COVID-19 death in the whole IJD cohort (see Status).
- A possible source of the RA figure (observation only). The printed RA risk, 0.30%, equals the all-IJD death count over the RA denominator (161/53,455 = 0.301%). This could reflect a cell-level mix-up, but the published material cannot confirm it. It does not explain the all-IJD figure of 0.10%.
Status
- Version of record. The publisher's site (ard.bmj.com, reached through doi.org) returned a Cloudflare challenge (HTTP 403) to automated requests for the HTML and for the PDFs, both the early-access PDFs and 80/8/1086.full.pdf. I checked the publisher-deposited JATS XML in Europe PMC, the live PMC page and the PubMed abstract (PMID 33622688). All three agree with every passage quoted here. Table 3 is encoded as a real table, not an image. Every quotation was checked verbatim against the PMC-archived text.
- Supplements. The single supplementary file,
annrheumdis-2021-219845supp001.pdf(27 pages, from the Europe PMC supplementary bundle), was read in full. It covers a sulfasalazine post hoc analysis, data sources and ICD-10/ATC codes, cohort characteristics for 2015–2019, covariates, DMARD cohort characteristics, standardised mean differences and daily COVID-19 deaths in Sweden. It has no outcome table, denominator or person-time basis that would account for the 0.10% or 0.30% figures. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by entries and an empty relation field, and the PMC page has no correction banner. PubMed lists one CommentsCorrections entry, "CommentIn: Ann Rheum Dis 2023;82(11):e223" (doi:10.1136/annrheumdis-2021-220938). Its title refers to a different article (the TICOSPA trial in axial spondyloarthritis), and it is not a correction notice. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title with "PubPeer", "erratum" or "correction" found no thread or notice for this article. The only correction it found belongs to a related paper in RMD Open, concerns author-name spelling and has no bearing on these figures. - Sources. PMC article PMC8206171; Europe PMC full-text XML and supplementary files for PMC8206171; PubMed records 33622688 and 34226191; Crossref records for 10.1136/annrheumdis-2021-219845 and 10.1136/annrheumdis-2021-220938; PubPeer; https://doi.org/10.1136/annrheumdis-2021-219845.
Suggested correction
If the counts are correct, as the subgroup sums and hazard ratios indicate, the following changes would make the text agree with the paper's own definition and data.
- Table 3, "All" block, COVID-19 death row: change "161 (0.10%) | 338 (0.07%) | 0.03" to "161 (0.15%) | 338 (0.07%) | 0.08".
- Table 3, "RA" block, COVID-19 death row: change "134 (0.30%) | 245 (0.11%) | 0.19" to "134 (0.25%) | 245 (0.11%) | 0.14". The authors can confirm the excess from the exact size of the RA referent cohort; the bounded range gives 0.14 throughout.
- Abstract: "death (0.10% vs 0.07%)" becomes "death (0.15% vs 0.07%)".
- Results: "0.10% (vs 0.07%) for death from COVID-19" becomes "0.15% (vs 0.07%) for death from COVID-19".
- Discussion, finding (3): "(0.10%, an additional 0.03 per 100 persons)" becomes "(0.15%, an additional 0.08 per 100 persons)".
- Discussion, patient-counselling paragraph: "in the order of 1 in a 1000" becomes "in the order of 1.5 in 1000", and "in the order of 3 per 10000" becomes "in the order of 8 per 10000".
If the event counts are instead the erroneous figures, the counts, subgroup counts and hazard ratios would all need to be corrected together, since they are mutually consistent as printed.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC8206171 (Ann Rheum Dis 2021, doi:10.1136/annrheumdis-2021-219845)
# Run: python recompute.py.txt -> prints every recomputed number and writes fig1/fig2 (.svg, .png) here.
# All inputs are copied from the article as archived in PubMed Central (Table 1, Table 3, Methods, Results).
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# Cohort sizes alive on 1 March 2020 (Table 1; Methods)
N = {"All": 110567, "RA": 53455, "Other": 57112}
NC_ALL = 484277 # matched general population referents, all IJDs
PY_ALL = 55336 # person-years, all IJDs (Results)
# Table 3: (cohort, outcome, IJD events, IJD % reported, GP events, GP % reported, excess reported, HR model 1)
# Percentages kept as printed strings so that the number of printed decimals is known (0.3 vs 0.30).
ROWS = [
("All", "Hospitalisation, all causes", 8971, "8.1", 24273, "5.0", 3.1, 1.65),
("All", "Hospitalisation, COVID-19", 581, "0.5", 1443, "0.3", 0.2, 1.77),
("All", "Admission to ICU, COVID-19", 45, "0.04", 162, "0.03", 0.01, 1.22),
("All", "Death, all causes", 1310, "1.2", 3036, "0.6", 0.6, 1.90),
("All", "Death, COVID-19", 161, "0.10", 338, "0.07", 0.03, 2.09),
("RA", "Hospitalisation, all causes", 5275, "9.9", 13072, "5.9", 4.0, 1.71),
("RA", "Hospitalisation, COVID-19", 379, "0.7", 784, "0.4", 0.3, 2.02),
("RA", "Admission to ICU, COVID-19", 31, "0.06", 79, "0.04", 0.02, 1.63),
("RA", "Death, all causes", 968, "1.8", 2026, "0.9", 0.9, 1.99),
("RA", "Death, COVID-19", 134, "0.30", 245, "0.11", 0.19, 2.28),
("Other", "Hospitalisation, all causes", 3696, "6.5", 11201, "4.3", 2.2, 1.54),
("Other", "Hospitalisation, COVID-19", 202, "0.4", 659, "0.3", 0.1, 1.41),
("Other", "Admission to ICU, COVID-19", 14, "0.02", 83, "0.03", -0.01, 0.78),
("Other", "Death, all causes", 342, "0.6", 1010, "0.4", 0.2, 1.56),
("Other", "Death, COVID-19", 27, "0.05", 93, "0.04", 0.01, 1.34),
]
def half(p):
"""Half-width of the rounding interval of a printed percentage string."""
d = len(p.split(".")[1]) if "." in p else 0
return 0.5 * 10 ** (-d)
print("== 1. Subgroup counts add up")
print(f"IJD N: 53,455 + 57,112 = {N['RA'] + N['Other']:,} (Table 1 gives 110,567)")
print(f"COVID-19 deaths, IJD: 134 + 27 = {134 + 27} (Table 3 'All' row gives 161)")
print(f"COVID-19 deaths, referents: 245 + 93 = {245 + 93} (Table 3 'All' row gives 338)")
print("\n== 2. All IJDs, death from COVID-19 (Table 3; Abstract; Results; Discussion)")
r1 = 100 * 161 / N["All"]
r0 = 100 * 338 / NC_ALL
print(f"IJD risk 161/110,567 = {r1:.4f}% (reported 0.10%)")
print(f"Referent risk 338/484,277 = {r0:.4f}% (reported 0.07%)")
print(f"Crude excess = {r1 - r0:.4f} per 100 = {100*(r1 - r0):.2f} per 10,000 (reported 0.03 per 100; Discussion '3 per 10000')")
print(f"Absolute risk = {10*r1:.2f} per 1000 (Discussion 'in the order of 1 in a 1000')")
print(f"Crude risk ratio from counts = {r1 / r0:.3f} (HR model 1 = 2.09)")
print(f"Ratio of the reported percentages 0.10/0.07 = {0.10 / 0.07:.3f}")
print(f"Deaths implied by 0.10% of 110,567 = {0.001 * N['All']:.1f}; any count from "
f"{0.00095 * N['All']:.1f} to {0.00105 * N['All']:.1f} rounds to 0.10%, i.e. 106 to 116 deaths")
print(f"Denominator implied by 161 deaths at 0.10% = {161 / 0.001:,.0f} "
f"(range {161 / 0.00105:,.0f} to {161 / 0.00095:,.0f})")
print(f"Rate per person-time 161/55,336 PY = {100 * 161 / PY_ALL:.3f} per 100 person-years")
print(f"Excess using the paper's displayed referent value 0.07: {round(r1, 2):.2f} - 0.07 = {round(r1, 2) - 0.07:.2f} per 100")
print(f"Understatement of excess: {(r1 - r0) / 0.03:.2f}-fold; of absolute risk: {r1 / 0.10:.2f}-fold")
print("\n== 3. Comparator cohort size for RA (not printed in the paper; bounded from Table 3)")
def nbounds(n, p):
h = half(p); v = float(p)
return n / ((v + h) / 100), n / ((v - h) / 100)
lo_ra, hi_ra = nbounds(13072, "5.9")
lo_ot, hi_ot = nbounds(11201, "4.3")
a = max(lo_ra, NC_ALL - hi_ot)
b = min(hi_ra, NC_ALL - lo_ot)
for (n, p) in [(784, "0.4"), (2026, "0.9"), (245, "0.11"), (79, "0.04")]:
lo, hi = nbounds(n, p)
a, b = max(a, lo), min(b, hi)
print(f"RA referent N from 13,072 (5.9%): {lo_ra:,.0f} to {hi_ra:,.0f}")
print(f"Other-IJD referent N from 11,201 (4.3%): {lo_ot:,.0f} to {hi_ot:,.0f}")
print(f"RA referent N consistent with all RA and Other-IJD referent cells: {a:,.0f} to {b:,.0f}")
NC_RA = (a + b) / 2
NC = {"All": NC_ALL, "RA": NC_RA, "Other": NC_ALL - NC_RA}
print(f"Midpoint used below: RA {NC_RA:,.0f}; Other IJD {NC_ALL - NC_RA:,.0f}")
print("\n== 4. RA, death from COVID-19 (Table 3 RA row)")
q1 = 100 * 134 / N["RA"]
q0 = 100 * 245 / NC_RA
print(f"RA risk 134/53,455 = {q1:.4f}% (reported 0.30%)")
print(f"RA referent risk 245/{NC_RA:,.0f} = {q0:.4f}% (reported 0.11%; range {100*245/b:.4f} to {100*245/a:.4f})")
print(f"Crude excess = {q1 - q0:.4f} per 100 (range {q1 - 100*245/a:.4f} to {q1 - 100*245/b:.4f}); reported 0.19")
print(f"Crude risk ratio = {q1 / q0:.3f} (HR model 1 = 2.28); ratio of reported % 0.30/0.11 = {0.30 / 0.11:.3f}")
print(f"Deaths implied by 0.30% of 53,455 = {0.003 * N['RA']:.0f} (range {0.00295 * N['RA']:.0f} to {0.00305 * N['RA']:.0f})")
print(f"Numerical coincidence: 161/53,455 = {100 * 161 / N['RA']:.4f}% (the all-IJD count over the RA denominator)")
print("\n== 5. Every Table 3 cell: reported % vs events/N, and crude risk ratio vs HR model 1")
print(f"{'cohort':6} {'outcome':28} {'IJD rep':>7} {'IJD calc':>8} {'ok':>3} {'GP rep':>6} {'GP calc':>7} {'ok':>3} "
f"{'exc rep':>7} {'exc calc':>8} {'RR':>5} {'HR1':>5}")
records = []
for c, o, n1, p1, n0, p0, ex, hr in ROWS:
c1 = 100 * n1 / N[c]
c0 = 100 * n0 / NC[c]
ok1 = abs(c1 - float(p1)) <= half(p1) + 1e-12
ok0 = abs(c0 - float(p0)) <= half(p0) + 1e-12
records.append((c, o, p1, c1, ok1))
print(f"{c:6} {o:28} {p1:7} {c1:8.3f} {'yes' if ok1 else 'NO':>3} {p0:6} {c0:7.3f} {'yes' if ok0 else 'NO':>3} "
f"{ex:7} {c1 - c0:8.3f} {c1 / c0:5.2f} {hr:5}")
bad = [r for r in records if not r[4]]
print(f"IJD-cohort cells outside their rounding interval: {len(bad)} of {len(records)}: "
+ "; ".join(f"{r[0]} {r[1]}" for r in bad))
# ---------------- Figures ----------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: the two headline cells, reported vs recomputed, linear axis from zero
items = [
("All IJDs: risk of COVID-19 death", 0.10, r1),
("All IJDs: crude excess vs referents", 0.03, r1 - r0),
("RA: risk of COVID-19 death", 0.30, q1),
("RA: crude excess vs referents", 0.19, q1 - q0),
]
ref_items = [("All IJDs: referent risk", 0.07, r0), ("RA: referent risk", 0.11, q0)]
fig, ax = plt.subplots(figsize=(7.2, 3.8))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
labels = [t[0] for t in items]
ys = list(range(len(items)))[::-1]
for y, (lab, rep, calc) in zip(ys, items):
ax.plot([rep, calc], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(rep, y, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(calc, y, "o", ms=9, color=BLUE, zorder=3)
lo, hi = sorted([rep, calc])
rep_txt = f"{rep:.2f}"; calc_txt = f"{calc:.3f}"
ax.annotate(rep_txt, (rep, y), xytext=(0, 9), textcoords="offset points", ha="center",
color=ORANGE, fontsize=10, fontweight="bold")
ax.annotate(calc_txt, (calc, y), xytext=(0, 9), textcoords="offset points", ha="center",
color=BLUE, fontsize=10, fontweight="bold")
ax.set_yticks(ys); ax.set_yticklabels(labels)
ax.set_ylim(-0.7, len(items) - 0.3)
ax.set_xlim(0, 0.34)
ax.set_xlabel("Per 100 persons: risk (%) or crude excess risk per 100", color=INK2)
ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported in Table 3"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed: events ÷ cohort size")]
ax.legend(handles=handles, loc="lower center", bbox_to_anchor=(0.4, 1.0), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: every IJD-cohort risk in Table 3 as recomputed / reported, with the rounding interval of the
# reported value expressed on the same ratio scale (log x); 1 = exact agreement.
fig, ax = plt.subplots(figsize=(7.2, 5.6))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
cohort_name = {"All": "All IJDs", "RA": "RA", "Other": "Other IJD"}
ys = list(range(len(records)))[::-1]
ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
for y, (c, o, ps, calc, ok) in zip(ys, records):
p = float(ps); h = half(ps)
lo, hi = (p - h) / p, (p + h) / p
ax.plot([lo, hi], [y, y], color=ORANGE, lw=7, alpha=0.45, solid_capstyle="butt", zorder=2)
ax.plot(calc / p, y, "o", ms=8, color=BLUE, zorder=4)
if not ok:
ax.annotate(f"{calc:.3f}% vs {ps}%", (max(calc / p, hi), y),
xytext=(10, -3.5), textcoords="offset points", color=INK, fontsize=10, fontweight="bold")
ax.set_yticks(ys)
ax.set_yticklabels([f"{cohort_name[c]}: {o}" for c, o, *_ in records], fontsize=10)
ax.set_xscale("log")
ax.set_xlim(0.7, 2.7)
ticks = [0.75, 0.9, 1.0, 1.1, 1.25, 1.5, 2.0, 2.5]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks]); ax.minorticks_off()
ax.set_xlabel("Recomputed risk ÷ printed risk (log scale; 1 = agreement)", color=INK2)
ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
for yy in (4.5, 9.5):
ax.axhline(yy, color=GRID, lw=0.8)
handles = [Line2D([], [], color=ORANGE, lw=7, alpha=0.45,
label="Range that rounds to the printed value"),
Line2D([], [], marker="o", ls="", ms=8, color=BLUE, label="Recomputed from the paper's counts")]
ax.legend(handles=handles, loc="lower center", bbox_to_anchor=(0.3, 1.0), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written to", HERE)
Abstract reports adverse-event rates of 17% and 10%; Results and Table 3 give 62% and 55%
Arthritis Rheumatol (2019) · doi:10.1002/art.41012 · PMC6900077
Summary
The abstract of this phase IIb/III trial of fasinumab in osteoarthritis pain states that treatment-emergent adverse event (TEAE) rates "were 17% with fasinumab and 10% with placebo". The Results text and Table 3 give the TEAE incidence for the 16-week treatment period as 62% and 55%, and Table 3's counts reproduce those figures exactly (209/337 = 62.0%; 45/82 = 54.9%). No period, organ class, population or definition in the paper yields 17% and 10% together, while the discontinuation figures in the same abstract sentence do match the body. The abstract's only overall tolerability figure therefore understates the absolute adverse-event burden by about 45 percentage points in each arm and overstates the relative excess (1.7-fold implied vs 1.13-fold in the data).
What the paper reports
Abstract, Results:
Treatment-emergent adverse event rates were 17% with fasinumab and 10% with placebo, and 4% and 1% of patients, respectively, discontinued treatment.
Patients and Methods, Safety end points:
Safety was evaluated based on the frequency of treatment-emergent adverse events (TEAEs), AEs of special interest (adjudicated arthropathy and sympathetic nervous system dysfunction), and laboratory tests.
Results, Safety, paragraph 2:
During the 16-week treatment period, the incidence of TEAEs was 62% with fasinumab and 55% with placebo (Table 3). Nervous system and musculoskeletal symptoms were more frequent following treatment with fasinumab (17% and 19%, respectively) than with placebo (9% and 17%, respectively).
Results, Safety, paragraph 3:
The incidence of serious TEAEs during treatment was low (2% with placebo versus 1% with fasinumab).
A low proportion of patients discontinued therapy due to TEAEs (4% [n = 14] in the fasinumab group and 1% [n = 1] in the placebo group).
Results, Safety, paragraph 4:
During the 20-week follow-up, the incidence of TEAEs was higher in the combined fasinumab group than in the placebo group (48% versus 38%) (Table 3)
Table 3 ("TEAEs reported in >3% of patients during the treatment and follow-up periods, by system organ class (safety analysis set)"; columns: placebo n = 82 | 1 mg n = 85 | 3 mg n = 84 | 6 mg n = 85 | 9 mg n = 83 | combined n = 337), treatment period and follow-up period summary rows:
≥1 TEAE | 45 (54.9) | 54 (63.5) | 52 (61.9) | 55 (64.7) | 48 (57.8) | 209 (62.0)
≥1 posttreatment AE | 31 (37.8) | 36 (42.4) | 38 (45.2) | 42 (49.4) | 44 (53.0) | 160 (47.5)
Results, Disposition of the patients:
During the study, 419 patients received ≥1 dose of study medication (1 patient randomized to receive placebo and 1 randomized to receive 9 mg fasinumab discontinued before drug administration).
Discussion, paragraph 5:
Fasinumab was generally well tolerated in the present study. Although the rate of TEAEs was higher with fasinumab than with placebo, there were few discontinuations attributable to TEAEs.
What the paper's own numbers show
Method. I took the counts and group sizes from Table 3 (safety analysis set), checked that the four dose columns add up to the combined column in every row of Table 3 (summary, organ-class and preferred-term rows, 42 in all), and recomputed every printed percentage in all six columns. I compared the abstract's 17%/10% with the treatment-period and follow-up-period rows and with every system-organ-class row in Table 3. I also listed which integer counts would print as 17% of 337 and 10% of 82, and checked the full-analysis-set denominators (338 and 83). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Any TEAE, fasinumab, treatment period | Abstract, Results | 17% | 209/337 = 62.0% |
| Any TEAE, placebo, treatment period | Abstract, Results | 10% | 45/82 = 54.9% |
| Any TEAE, fasinumab vs placebo | Results, Safety ¶2 | 62% vs 55% | 62.0% vs 54.9% (agrees) |
| ≥1 TEAE, combined column | Table 3 | 209 (62.0) | 54 + 52 + 55 + 48 = 209 of 85 + 84 + 85 + 83 = 337; 62.0% (agrees) |
| Any AE, follow-up period | Results, Safety ¶4; Table 3 | 48% vs 38%; 160 (47.5) vs 31 (37.8) | 160/337 = 47.5%; 31/82 = 37.8% (agrees; not 17%/10%) |
| Discontinuation due to TEAE | Abstract; Results, Safety ¶3 | 4% vs 1% (n = 14, n = 1) | 14/337 = 4.2%; 1/82 = 1.2% (agrees) |
| Nervous system disorders, treatment period | Results, Safety ¶2; Table 3 | 17% vs 9% | 57/337 = 16.9%; 7/82 = 8.5% |
| Vascular disorders, treatment period | Results, Safety ¶3; Table 3 | 6% vs 10% | 19/337 = 5.6%; 8/82 = 9.8% |
| Serious TEAEs, treatment period | Results, Safety ¶3 | 1% vs 2% | no counts given; not 17%/10% |
| Counts that would print as 17% and 10% | derived | — | 56–58 of 337 fasinumab patients and exactly 8 of 82 placebo patients |
| Table 3 rows giving 17% and 10% together | Table 3 (all 42 rows, both periods) | — | none |
| Any TEAE with full-analysis-set denominators | Disposition (338, 83) | — | 209/338 = 61.8%; 45/83 = 54.2% |
| Abstract minus Table 3, treatment period | derived | — | −45.0 pp (fasinumab), −44.9 pp (placebo) |
| Table 3 rate ÷ abstract rate | derived | — | 3.65 (fasinumab), 5.49 (placebo) |
| Risk ratio, fasinumab vs placebo | derived | 17/10 = 1.70 implied | (209/337)/(45/82) = 1.130 |
| Absolute difference | derived | 7 pp implied | 7.1 pp |
| Fisher exact test, 209/337 vs 45/82 | derived | — | OR 1.34, two-sided p = 0.26 |
Table 3 is internally consistent: in all 42 rows (summary, organ-class and preferred-term) the dose columns sum to the combined column, and all 252 printed percentages, in every dose column, match their counts. The Results text agrees with Table 3. Only the abstract's TEAE rates differ.
Figure
Figure 1. The conflicting statements, side by side. Left: the abstract's sentence, with the TEAE rates it prints in orange. Right: the Results (Safety, paragraph 2) sentence on the same quantity and the Table 3 "≥1 TEAE" counts, with the percentages recomputed from those counts in blue (209/337 = 62.0%, 45/82 = 54.9%). Both describe any TEAE in the 16-week treatment period. The discontinuation figures in the same abstract sentence (4% and 1%) agree with the body (14/337 = 4.2%, 1/82 = 1.2%).
Figure 2. Every summary row and system-organ-class row of Table 3, plotted as the placebo rate (x) against the combined fasinumab rate (y), both recomputed from the counts (blue; filled for the any-adverse-event rows, hollow for organ classes; circles for the treatment period, squares for follow-up; preferred-term rows are omitted, as none exceeds 11% in either arm). The orange cross is the abstract's pair, 17% vs 10%. No row lies at that point: the nervous-system row matches the fasinumab figure but not the placebo one (16.9% vs 8.5%), and the vascular row matches the placebo figure but not the fasinumab one (5.6% vs 9.8%). The gray diagonal marks equal rates in the two arms.
Why it matters
The abstract sentence is the abstract's only statement of overall tolerability, and readers who use the abstract alone (indexing services, evidence summaries, secondary coverage) receive the wrong figure.
- Absolute burden. The abstract implies that about 1 in 6 fasinumab-treated patients and 1 in 10 placebo patients had a TEAE. Table 3 shows about 3 in 5 and 1 in 2 (62.0% and 54.9%). The abstract understates both arms by about 45 percentage points.
- Relative excess. The abstract's figures imply a 1.7-fold risk with fasinumab (a 70% relative increase). The data give a 1.13-fold risk (OR 1.34; Fisher exact p = 0.26). The absolute difference is about 7 points either way (7 pp as stated, 7.1 pp recomputed). So the abstract makes the overall burden look smaller and the difference between arms look proportionally larger than the data show.
- What is unaffected. The direction is the same in both versions: TEAEs were more frequent with fasinumab. The discontinuation figures (4% vs 1%), the arthropathy figures, the serious-TEAE statement in the body and all efficacy results are not affected. The body's statements that fasinumab was "generally well tolerated" and that "the rate of TEAEs was higher with fasinumab than with placebo" are worded consistently with Table 3. The abstract's Conclusion ("The observed benefit-to-risk relationship favors further clinical development to explore the lowest doses") rests mainly on the efficacy and arthropathy results, which are correct, but a reader weighing that conclusion from the abstract alone would be weighing it against an adverse-event rate of 17% rather than 62%.
Explanations considered
- A different study period. This does not account for the numbers. The treatment period gives 62.0% vs 54.9% and the follow-up period gives 47.5% vs 37.8%.
- A single system organ class. This does not account for the numbers. Nervous system disorders give 16.9% (17%) for fasinumab but 8.5% (9%) for placebo, and the Results text itself prints them as 17% vs 9%. Vascular disorders give 9.8% (10%) for placebo but 5.6% for fasinumab. No Table 3 row in either period gives 17% and 10% together; that would need 56–58 of 337 fasinumab patients and 8 of 82 placebo patients in the same row (Figure 2).
- Serious TEAEs. This does not account for the numbers. The Results text gives 1% with fasinumab and 2% with placebo.
- Treatment-related (drug-related) TEAEs. This cannot be checked, and the wording argues against it. The paper reports no treatment-related TEAE rate anywhere, and the abstract uses the same term, "treatment-emergent adverse event", that the Methods define as the safety end point.
- A different analysis population. This does not account for the numbers. Table 3 uses the safety analysis set (82 and 337), and the abstract itself refers to "1 of 337 treated patients". Using the randomized numbers (83 and 338) gives 54.2% and 61.8%.
- A conversion or typesetting artifact. This is excluded as far as the available copies allow. The same abstract sentence appears in the PMC article, the Europe PMC full-text XML and the PubMed record (PMID 31207169).
- A correction elsewhere. None was found (see Status).
Status
- Version of record. The publisher's landing page and PDF (acrjournals.onlinelibrary.wiley.com, reached from the DOI) returned HTTP 403 to automated access and could not be read. The abstract sentence, the Results sentences and the Table 3 rows quoted above are identical in the PMC article (live page and archived text), the Europe PMC full-text XML, and the PubMed and Europe PMC abstract records. Every quote above was checked verbatim against the PMC-archived text.
- Supplements. The single supporting file (ART-71-1824-s001.docx, from the Europe PMC supplementary-files endpoint) holds Supplementary Methods, Supplementary Table 1 (WOMAC physical function) and Supplementary Figures 1–4. It contains no adverse-event tables and does not resolve the discrepancy. A ClinicalTrials.gov record that appears to correspond to this study (NCT02447276) has no posted results; the article gives no registration number, so the match is unconfirmed.
- Corrections and comments. As of 2026-10-23 no correction, erratum, retraction or expression of concern was found. Crossref lists no update relations for the DOI; PMC, Europe PMC and the PubMed record list no correction or comment links; the Crossref Labs record shows no Retraction Watch entry. Web searches for an erratum, correction or PubPeer discussion found none. PubPeer itself could not be queried directly (API 404 / access challenge).
- Sources. PMC article PMC6900077; Europe PMC full text and supplementary files; PubMed PMID 31207169; Crossref and Crossref Labs records for doi:10.1002/art.41012; ClinicalTrials.gov NCT02447276.
Suggested correction
- In the Abstract (Results), replace "Treatment-emergent adverse event rates were 17% with fasinumab and 10% with placebo" with "Treatment-emergent adverse event rates during the 16-week treatment period were 62% with fasinumab and 55% with placebo", matching the Results text and Table 3 (209/337 and 45/82). The discontinuation figures in the same sentence (4% and 1%) are correct and can stay.
- If the 17% and 10% were intended to describe a different quantity, name it in the abstract and report it, with counts, in the Results or Table 3, since no such quantity currently appears in the paper.
- Ask indexing services (PubMed, Europe PMC) to carry the corrected abstract.
Recomputation script (python recompute.py.txt)
"""Recompute the adverse-event rates of PMC6900077
(Arthritis Rheumatol 2019, doi:10.1002/art.41012).
Run: python recompute.py.txt (numbers + fig1/fig2)
python recompute.py.txt --candidates (also the five review candidates)
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes, next to this file:
fig1.svg/png, fig2.svg/png (the figures used in writeup.md)
candidates/cand1..cand5.svg/png (five candidate figure designs for internal review; written
only with --candidates; not part of the journal package)
Inputs are copied from the paper as archived in PMC:
Table 3 (safety analysis set): every row (summary, system organ class and preferred term),
all six columns, counts and printed percentages, as TABLE3_TEXT below
Results, Safety paragraph 3: discontinuations due to TEAEs (n = 14 fasinumab, n = 1 placebo)
Results, Disposition: randomized n = 338 fasinumab, 83 placebo (full analysis set)
Abstract, Results: "Treatment-emergent adverse event rates were 17% with fasinumab and
10% with placebo, and 4% and 1% of patients, respectively, discontinued treatment."
"""
import os
import re
import sys
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
from scipy.stats import fisher_exact # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
WRITE_CANDIDATES = "--candidates" in sys.argv[1:]
# ------------------------------------------------------------------ inputs (from the paper)
N_PBO, N_FAS = 82, 337 # Table 3 header, safety analysis set
DOSES = {"1 mg": 85, "3 mg": 84, "6 mg": 85, "9 mg": 83}
N_RAND_PBO, N_RAND_FAS = 83, 338 # Results, Disposition (randomized)
ABSTRACT = {"teae_fas": 17, "teae_pbo": 10, "disc_fas": 4, "disc_pbo": 1} # Abstract, %
TEXT = {"teae_fas": 62, "teae_pbo": 55, "fu_fas": 48, "fu_pbo": 38} # Results text, %
DISC = {"fas": 14, "pbo": 1} # Results, Safety paragraph 3
SERIOUS_TEXT = {"fas": 1, "pbo": 2} # Results, Safety paragraph 3, % (no counts given)
# Table 3, every row as printed: period (T = treatment, F = follow-up) | row | placebo |
# 1 mg | 3 mg | 6 mg | 9 mg | combined; each cell "n (%)", or "0".
TABLE3_TEXT = """\
T | ≥1 TEAE | 45 (54.9) | 54 (63.5) | 52 (61.9) | 55 (64.7) | 48 (57.8) | 209 (62.0)
T | Infections and infestations | 13 (15.9) | 18 (21.2) | 17 (20.2) | 16 (18.8) | 21 (25.3) | 72 (21.4)
T | Upper respiratory tract infection | 1 (1.2) | 5 (5.9) | 3 (3.6) | 3 (3.5) | 7 (8.4) | 18 (5.3)
T | Urinary tract infection | 3 (3.7) | 5 (5.9) | 1 (1.2) | 4 (4.7) | 3 (3.6) | 13 (3.9)
T | Sinusitis | 3 (3.7) | 1 (1.2) | 2 (2.4) | 2 (2.4) | 4 (4.8) | 9 (2.7)
T | Musculoskeletal and connective tissue disorders | 14 (17.1) | 16 (18.8) | 15 (17.9) | 18 (21.2) | 14 (16.9) | 63 (18.7)
T | Arthralgia | 2 (2.4) | 9 (10.6) | 5 (6.0) | 8 (9.4) | 5 (6.0) | 27 (8.0)
T | Back pain | 2 (2.4) | 4 (4.7) | 1 (1.2) | 2 (2.4) | 3 (3.6) | 10 (3.0)
T | Joint swelling | 0 | 1 (1.2) | 6 (7.1) | 3 (3.5) | 0 | 10 (3.0)
T | Pain in extremity | 3 (3.7) | 3 (3.5) | 1 (1.2) | 2 (2.4) | 4 (4.8) | 10 (3.0)
T | Musculoskeletal pain | 4 (4.9) | 0 | 1 (1.2) | 2 (2.4) | 1 (1.2) | 4 (1.2)
T | Myalgia | 0 | 1 (1.2) | 0 | 2 (2.4) | 0 | 3 (0.9)
T | Osteoarthritis | 0 | 0 | 0 | 0 | 3 (3.6) | 3 (0.9)
T | Nervous system disorders | 7 (8.5) | 15 (17.6) | 14 (16.7) | 15 (17.6) | 13 (15.7) | 57 (16.9)
T | Headache | 5 (6.1) | 7 (8.2) | 2 (2.4) | 4 (4.7) | 4 (4.8) | 17 (5.0)
T | Paresthesia | 0 | 2 (2.4) | 4 (4.8) | 0 | 4 (4.8) | 10 (3.0)
T | Dizziness | 2 (2.4) | 3 (3.5) | 2 (2.4) | 3 (3.5) | 1 (1.2) | 9 (2.7)
T | Hypoesthesia | 1 (1.2) | 2 (2.4) | 3 (3.6) | 2 (2.4) | 1 (1.2) | 8 (2.4)
T | Gastrointestinal disorders | 7 (8.5) | 9 (10.6) | 12 (14.3) | 9 (10.6) | 5 (6.0) | 35 (10.4)
T | Nausea | 3 (3.7) | 6 (7.1) | 1 (1.2) | 2 (2.4) | 1 (1.2) | 10 (3.0)
T | Diarrhea | 3 (3.7) | 1 (1.2) | 1 (1.2) | 4 (4.7) | 2 (2.4) | 8 (2.4)
T | Dry mouth | 1 (1.2) | 3 (3.5) | 4 (4.8) | 1 (1.2) | 0 | 8 (2.4)
T | Vomiting | 0 | 1 (1.2) | 3 (3.6) | 1 (1.2) | 2 (2.4) | 7 (2.1)
T | Vascular disorders | 8 (9.8) | 6 (7.1) | 4 (4.8) | 5 (5.9) | 4 (4.8) | 19 (5.6)
T | Orthostatic hypotension | 3 (3.7) | 3 (3.5) | 3 (3.6) | 3 (3.5) | 2 (2.4) | 11 (3.3)
T | Hypertension | 4 (4.9) | 1 (1.2) | 1 (1.2) | 2 (2.4) | 1 (1.2) | 5 (1.5)
T | Skin and subcutaneous tissue disorders | 1 (1.2) | 6 (7.1) | 4 (4.8) | 2 (2.4) | 5 (6.0) | 17 (5.0)
T | Rash | 0 | 1 (1.2) | 2 (2.4) | 1 (1.2) | 4 (4.8) | 8 (2.4)
F | ≥1 posttreatment AE | 31 (37.8) | 36 (42.4) | 38 (45.2) | 42 (49.4) | 44 (53.0) | 160 (47.5)
F | Musculoskeletal and connective tissue disorders | 10 (12.2) | 15 (17.6) | 20 (23.8) | 25 (29.4) | 23 (27.7) | 83 (24.6)
F | Arthralgia | 5 (6.1) | 3 (3.5) | 11 (13.1) | 12 (14.1) | 9 (10.8) | 35 (10.4)
F | Rapidly progressive OA | 0 | 2 (2.4) | 2 (2.4) | 5 (5.9) | 7 (8.4) | 16 (4.7)
F | OA | 0 | 3 (3.5) | 4 (4.8) | 3 (3.5) | 4 (4.8) | 14 (4.2)
F | Musculoskeletal pain | 2 (2.4) | 2 (2.4) | 5 (6.0) | 4 (4.7) | 1 (1.2) | 12 (3.6)
F | Joint swelling | 1 (1.2) | 1 (1.2) | 2 (2.4) | 4 (4.7) | 1 (1.2) | 8 (2.4)
F | Pain in extremity | 1 (1.2) | 1 (1.2) | 1 (1.2) | 3 (3.5) | 2 (2.4) | 7 (2.1)
F | Infections and infestations | 7 (8.5) | 11 (12.9) | 15 (17.9) | 8 (9.4) | 12 (14.5) | 46 (13.6)
F | Upper respiratory tract infection | 0 | 2 (2.4) | 7 (8.3) | 3 (3.5) | 0 | 12 (3.6)
F | Urinary tract infection | 2 (2.4) | 3 (3.5) | 2 (2.4) | 0 | 1 (1.2) | 6 (1.8)
F | Bronchitis | 1 (1.2) | 0 | 1 (1.2) | 0 | 3 (3.6) | 4 (1.2)
F | Vascular disorders | 4 (4.9) | 2 (2.4) | 5 (6.0) | 3 (3.5) | 5 (6.0) | 15 (4.5)
F | Orthostatic hypotension | 4 (4.9) | 2 (2.4) | 4 (4.8) | 0 | 4 (4.8) | 10 (3.0)
"""
def parse_table3(text):
out = []
for line in text.strip().splitlines():
c = [x.strip() for x in line.split("|")]
cells = []
for x in c[2:]:
m = re.fullmatch(r"(\d+)(?: \(([\d.]+)\))?", x)
n = int(m.group(1))
cells.append((n, float(m.group(2)) if m.group(2) else (0.0 if n == 0 else None)))
out.append((c[0], c[1], cells))
return out
TABLE3 = parse_table3(TABLE3_TEXT)
COL_N = [N_PBO] + list(DOSES.values()) + [N_FAS]
COL_NAMES = ["placebo", "1 mg", "3 mg", "6 mg", "9 mg", "combined"]
# Summary and system-organ-class rows used in the figures (subset of TABLE3, cross-checked below)
# (placebo n, [1, 3, 6, 9 mg n], combined n, printed placebo %, printed combined %)
TREAT = {
">=1 TEAE": (45, [54, 52, 55, 48], 209, 54.9, 62.0),
"Infections and infestations": (13, [18, 17, 16, 21], 72, 15.9, 21.4),
"Musculoskeletal and connective tissue": (14, [16, 15, 18, 14], 63, 17.1, 18.7),
"Nervous system": (7, [15, 14, 15, 13], 57, 8.5, 16.9),
"Gastrointestinal": (7, [9, 12, 9, 5], 35, 8.5, 10.4),
"Vascular": (8, [6, 4, 5, 4], 19, 9.8, 5.6),
"Skin and subcutaneous tissue": (1, [6, 4, 2, 5], 17, 1.2, 5.0),
}
FOLLOW = {
">=1 posttreatment AE": (31, [36, 38, 42, 44], 160, 37.8, 47.5),
"Musculoskeletal and connective tissue": (10, [15, 20, 25, 23], 83, 12.2, 24.6),
"Infections and infestations": (7, [11, 15, 8, 12], 46, 8.5, 13.6),
"Vascular": (4, [2, 5, 3, 5], 15, 4.9, 4.5),
}
def pct(a, b):
return 100.0 * a / b
def r1(x):
return round(x + 1e-9, 1)
# ------------------------------------------------------------------ 1. table arithmetic
print("=== 1. Table 3 arithmetic (safety analysis set) ===")
assert sum(DOSES.values()) == N_FAS
print(f"dose-group sizes {list(DOSES.values())} sum to {sum(DOSES.values())} = combined n {N_FAS}")
# 1a. every Table 3 row, every column
n_rows = len(TABLE3)
n_cells = sum(len(c) for _, _, c in TABLE3)
sum_bad = [(per, k) for per, k, c in TABLE3 if sum(n for n, _ in c[1:5]) != c[5][0]]
pct_bad = [(per, k, COL_NAMES[j], n, p, r1(pct(n, COL_N[j])))
for per, k, c in TABLE3 for j, (n, p) in enumerate(c) if p is None or r1(pct(n, COL_N[j])) != p]
print(f"all Table 3 rows: {n_rows} rows ({sum(1 for r in TABLE3 if r[0] == 'T')} treatment, "
f"{sum(1 for r in TABLE3 if r[0] == 'F')} follow-up), {n_cells} cells checked")
print(f" rows where 1 + 3 + 6 + 9 mg counts != combined count: {sum_bad if sum_bad else 'none'}")
print(f" cells (any column) where printed % != count/n rounded to 0.1: {pct_bad if pct_bad else 'none'}")
for label, rows in (("treatment period", TREAT), ("follow-up period", FOLLOW)):
print(f"\n[{label}]")
print(f" {'row':40s} {'pbo n':>5s} {'pbo %':>6s} {'fas n':>5s} {'fas %':>6s} doses sum printed % match")
for k, (p, ds, c, pp, pc) in rows.items():
ok_sum = sum(ds) == c
ok_pct = r1(pct(p, N_PBO)) == pp and r1(pct(c, N_FAS)) == pc
print(f" {k:40s} {p:5d} {pct(p, N_PBO):6.1f} {c:5d} {pct(c, N_FAS):6.1f} {str(ok_sum):9s} {ok_pct}")
# cross-check the figure subset against the full table
lookup = {(per, k): c for per, k, c in TABLE3}
for per, rows in (("T", TREAT), ("F", FOLLOW)):
for k, (p, ds, c, pp, pc) in rows.items():
full = [key for key in lookup if key[0] == per and key[1].replace("≥", ">=").startswith(k)]
assert len(full) == 1, (per, k, full)
cells = lookup[full[0]]
assert [n for n, _ in cells] == [p] + ds + [c] and cells[0][1] == pp and cells[5][1] == pc, k
print("\nsummary/organ-class subset used for the figures matches the full table")
teae_f, teae_p = pct(209, N_FAS), pct(45, N_PBO)
fu_f, fu_p = pct(160, N_FAS), pct(31, N_PBO)
print("\n=== 2. Any TEAE: abstract vs body ===")
print(f"treatment period, fasinumab: 209/337 = {teae_f:.1f}% (Results text {TEXT['teae_fas']}%, abstract {ABSTRACT['teae_fas']}%)")
print(f"treatment period, placebo: 45/82 = {teae_p:.1f}% (Results text {TEXT['teae_pbo']}%, abstract {ABSTRACT['teae_pbo']}%)")
print(f"follow-up period, fasinumab: 160/337 = {fu_f:.1f}% (Results text {TEXT['fu_fas']}%)")
print(f"follow-up period, placebo: 31/82 = {fu_p:.1f}% (Results text {TEXT['fu_pbo']}%)")
print(f"gap, abstract vs Table 3 (treatment): fasinumab {teae_f - 17:.1f} pp, placebo {teae_p - 10:.1f} pp")
print(f"Table 3 / abstract: fasinumab {teae_f / 17:.2f}x, placebo {teae_p / 10:.2f}x")
print(f"abstract implies risk ratio 17/10 = {17 / 10:.2f}; Table 3 gives ({209}/{N_FAS})/({45}/{N_PBO}) = {teae_f / teae_p:.3f}")
print(f"absolute difference: abstract {17 - 10} pp; Table 3 {teae_f - teae_p:.1f} pp")
odds, p = fisher_exact([[209, N_FAS - 209], [45, N_PBO - 45]])
print(f"Fisher exact test, 209/337 vs 45/82: OR = {odds:.2f}, two-sided p = {p:.2f}")
print(f"full analysis set denominators: 209/338 = {pct(209, N_RAND_FAS):.1f}%, 45/83 = {pct(45, N_RAND_PBO):.1f}%")
print("\n=== 3. Discontinuations due to TEAEs (same abstract sentence) ===")
print(f"fasinumab 14/337 = {pct(14, N_FAS):.1f}% (abstract 4%); placebo 1/82 = {pct(1, N_PBO):.1f}% (abstract 1%)")
print("\n=== 4. Which counts would print as 17% and 10%? ===")
f17 = [n for n in range(N_FAS + 1) if round(pct(n, N_FAS)) == 17]
p10 = [n for n in range(N_PBO + 1) if round(pct(n, N_PBO)) == 10]
print(f"fasinumab counts rounding to 17% of 337: {f17[0]}-{f17[-1]} "
f"({pct(f17[0], N_FAS):.1f}%-{pct(f17[-1], N_FAS):.1f}%)")
print(f"placebo counts rounding to 10% of 82: {p10} ({pct(p10[0], N_PBO):.1f}%)")
allrows = [("treatment", k, v) for k, v in TREAT.items()] + [("follow-up", k, v) for k, v in FOLLOW.items()]
hits = [(per, k) for per, k, (p_, _, c, _, _) in allrows
if round(pct(c, N_FAS)) == 17 and round(pct(p_, N_PBO)) == 10]
print(f"Table 3 summary/organ-class rows with fasinumab 17% AND placebo 10%: {hits if hits else 'none'}")
hits_all = [(per, k) for per, k, c in TABLE3
if round(pct(c[5][0], N_FAS)) == 17 and round(pct(c[0][0], N_PBO)) == 10]
print(f"any Table 3 row (incl. preferred terms) with fasinumab 17% AND placebo 10%: "
f"{hits_all if hits_all else 'none'}")
for per, k, (p_, _, c, _, _) in allrows:
if round(pct(c, N_FAS)) == 17 or round(pct(p_, N_PBO)) == 10:
print(f" partial match ({per}) {k}: fasinumab {pct(c, N_FAS):.1f}%, placebo {pct(p_, N_PBO):.1f}%")
print(f"serious TEAEs during treatment (Results text): fasinumab {SERIOUS_TEXT['fas']}%, placebo {SERIOUS_TEXT['pbo']}%")
# ------------------------------------------------------------------ figure style
BG, TXT, SEC, ORANGE, BLUE = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6"
GRID, PANEL = "#e4e3df", "#f3f2ef"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": "#bdbcb8", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def clean(ax, grid_axis="x"):
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
def panel(ax, x, y, w, h, title, lines):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=PANEL, ec="#cfcdc8", lw=1))
ax.text(x + 0.02, y + h - 0.035, title, ha="left", va="top", fontsize=11, weight="bold")
yy = y + h - 0.13
for ln, col, size in lines:
ax.text(x + 0.02, yy, ln, ha="left", va="top", fontsize=size, color=col)
yy -= 0.068 * (ln.count("\n") + 1) + 0.02
# ---- candidate 1: two text panels joined by an annotated connector (abstract vs body)
def cand_text_panels():
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
panel(ax, 0.02, 0.17, 0.40, 0.78, "Abstract, Results", [
("“Treatment-emergent adverse\nevent rates were", TXT, 10.5),
("17% with fasinumab and\n10% with placebo”", ORANGE, 11.5),
("Same sentence: discontinuations\n4% and 1%", SEC, 10),
])
panel(ax, 0.58, 0.17, 0.40, 0.78, "Results, Safety ¶2 and Table 3", [
("“the incidence of TEAEs was\n62% with fasinumab and\n55% with placebo (Table 3)”", TXT, 10.5),
("Table 3, ≥1 TEAE:\n209/337 = 62.0% (fasinumab)\n45/82 = 54.9% (placebo)", BLUE, 11),
("Discontinuations 14/337 = 4.2%\nand 1/82 = 1.2%", SEC, 10),
])
ax.annotate("", xy=(0.575, 0.62), xytext=(0.425, 0.62),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=2))
ax.text(0.5, 0.66, "same quantity,\nsame period", ha="center", va="bottom", fontsize=10, color=SEC)
ax.text(0.5, 0.58, "17 vs 62\n10 vs 55", ha="center", va="top", fontsize=11, weight="bold")
ax.annotate("", xy=(0.575, 0.31), xytext=(0.425, 0.31),
arrowprops=dict(arrowstyle="<->", color=SEC, lw=1.2, ls="--"))
ax.text(0.5, 0.33, "agree", ha="center", va="bottom", fontsize=10, color=SEC)
ax.text(0.02, 0.07, "Orange: as printed in the abstract. Blue: recomputed from Table 3 counts\n"
"(any TEAE, 16-week treatment period, safety analysis set).", ha="left", va="center",
fontsize=10, color=SEC)
return fig
# ---- candidate 2: dot plot, abstract vs recomputed, on one 0-100% axis
def cand_dot_plot():
fig, ax = plt.subplots(figsize=(W, 3.3))
rows = [("Any TEAE, fasinumab", 17, teae_f), ("Any TEAE, placebo", 10, teae_p),
("Discontinued for TEAE, fasinumab", 4, pct(14, N_FAS)),
("Discontinued for TEAE, placebo", 1, pct(1, N_PBO))]
for i, (lab, rep, rec) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([rep, rec], [y, y], color="#bdbcb8", lw=2, zorder=1)
ax.scatter(rep, y, s=210, fc="none", ec=ORANGE, lw=2.5, zorder=4,
label="Abstract (as printed)" if i == 0 else None)
ax.scatter(rec, y, s=90, color=BLUE, zorder=3, ec=BG, lw=1.5,
label="Table 3 / Results counts, recomputed" if i == 0 else None)
if abs(rep - rec) > 5:
ax.text(rep, y + 0.22, f"{rep}%", ha="center", va="bottom", fontsize=10, color=TXT)
ax.text(rec, y + 0.22, f"{rec:.1f}%", ha="center", va="bottom", fontsize=10, color=TXT)
else:
ax.text(max(rep, rec) + 4, y, f"{rep}% vs {rec:.1f}% (agree)", ha="left", va="center",
fontsize=10, color=SEC)
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows][::-1], color=TXT)
ax.set_xlim(0, 100); ax.set_ylim(-0.6, len(rows) - 0.25)
ax.set_xlabel("Patients (%), 16-week treatment period, safety analysis set")
clean(ax)
fig.legend(*ax.get_legend_handles_labels(), loc="upper center", ncol=2, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0, 1, 0.9))
return fig
# ---- candidate 3: every Table 3 row as a point; the abstract's pair falls on none
def cand_search_scatter():
fig, ax = plt.subplots(figsize=(W, 5.4))
for per, rows, mk in (("treatment", TREAT, "o"), ("follow-up", FOLLOW, "s")):
for k, (p_, _, c, _, _) in rows.items():
overall = k.startswith(">=1")
ax.scatter(pct(p_, N_PBO), pct(c, N_FAS), s=70 if overall else 55, marker=mk,
fc=BLUE if overall else BG, ec=BLUE, lw=2, zorder=3)
ax.scatter(10, 17, s=90, marker="X", color=ORANGE, zorder=4, ec=BG, lw=1)
callouts = [("Nervous system", "nervous system, treatment:\n", (1.0, 34), (-0.2, 0.9)),
("Vascular", "vascular, treatment: ", (24, 5), (0.6, 0.0))]
for k, name, (tx, ty), (dx, dy) in callouts:
p_, _, c, _, _ = TREAT[k]
x0, y0 = pct(p_, N_PBO), pct(c, N_FAS)
ax.annotate(f"{name}{y0:.1f}% vs {x0:.1f}%", xy=(x0 + dx, y0 + dy), xytext=(tx, ty),
fontsize=10, color=SEC, va="center",
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8, shrinkA=2, shrinkB=4))
ax.text(teae_p - 1.5, teae_f, f"any TEAE, treatment\n{teae_f:.1f}% vs {teae_p:.1f}%",
fontsize=10, color=TXT, va="center", ha="right")
ax.text(fu_p - 1.5, fu_f, f"any AE, follow-up\n{fu_f:.1f}% vs {fu_p:.1f}%",
fontsize=10, color=TXT, va="center", ha="right")
ax.annotate("Abstract: 17% vs 10%\n(no Table 3 row is at this point)",
xy=(10.6, 17.6), xytext=(22, 31), fontsize=10, color=TXT, va="center",
bbox=dict(fc=BG, ec="none", pad=1.5),
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8, shrinkA=2, shrinkB=2))
ax.plot([0, 70], [0, 70], color="#bdbcb8", lw=1, zorder=0)
ax.text(60, 55, "equal rates", fontsize=10, color=SEC)
ax.set_xlim(0, 70); ax.set_ylim(0, 70)
ax.set_xlabel("Placebo, % of 82 patients"); ax.set_ylabel("Fasinumab combined, % of 337 patients")
clean(ax, "both")
from matplotlib.lines import Line2D
h = [Line2D([], [], marker="X", ls="", color=ORANGE, ms=10, label="Abstract (as printed)"),
Line2D([], [], marker="o", ls="", mfc=BLUE, mec=BLUE, ms=8, label="Any TEAE, treatment (recomputed)"),
Line2D([], [], marker="s", ls="", mfc=BLUE, mec=BLUE, ms=8, label="Any AE, follow-up (recomputed)"),
Line2D([], [], marker="o", ls="", mfc=BG, mec=BLUE, mew=2, ms=8, label="Organ-class rows, treatment"),
Line2D([], [], marker="s", ls="", mfc=BG, mec=BLUE, mew=2, ms=8, label="Organ-class rows, follow-up")]
fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, 0.005), ncol=2, frameon=False,
fontsize=10, columnspacing=2.0, handletextpad=0.4)
fig.subplots_adjust(left=0.10, right=0.97, top=0.97, bottom=0.28)
return fig
# ---- candidate 4: bars from zero, abstract vs treatment period vs follow-up
def cand_bars():
fig, ax = plt.subplots(figsize=(W, 3.4))
groups = [("Fasinumab (n = 337)", 17, teae_f, fu_f), ("Placebo (n = 82)", 10, teae_p, fu_p)]
h = 0.24
for i, (g, rep, tr, fu) in enumerate(groups):
y = 1 - i
ax.barh(y + h, rep, height=h - 0.03, color=ORANGE, label="Abstract: any TEAE (as printed)" if i == 0 else None)
ax.barh(y, tr, height=h - 0.03, color=BLUE, label="Table 3: ≥1 TEAE, treatment period" if i == 0 else None)
ax.barh(y - h, fu, height=h - 0.03, color=BG, ec=BLUE, lw=1.5, hatch="///",
label="Table 3: ≥1 AE, follow-up period" if i == 0 else None)
for yy, v, fmt in ((y + h, rep, f"{rep}%"), (y, tr, f"{tr:.1f}%"), (y - h, fu, f"{fu:.1f}%")):
ax.text(v + 1, yy, fmt, va="center", fontsize=10, color=TXT)
ax.set_yticks([1, 0]); ax.set_yticklabels([g[0] for g in groups], color=TXT)
ax.set_xlim(0, 100); ax.set_ylim(-0.55, 1.75)
ax.set_xlabel("Patients with at least one adverse event (%)")
clean(ax)
ax.legend(loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
return fig
# ---- candidate 5: calculation flow from Table 3 counts to the printed figures
def cand_flow():
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
xs = [0.01, 0.29, 0.54, 0.79]; ws = [0.21, 0.17, 0.17, 0.20]
heads = ["Table 3 counts", "Recomputed", "Results text", "Abstract"]
for x, w, t in zip(xs, ws, heads):
ax.text(x + w / 2, 0.93, t, ha="center", va="center", fontsize=11, weight="bold")
rows = [(0.62, ["54+52+55+48\n= 209 of 337", f"209/337\n= {teae_f:.1f}%",
"“62% with\nfasinumab”", "“17% with\nfasinumab”"]),
(0.27, ["45 of 82\n(placebo)", f"45/82\n= {teae_p:.1f}%", "“55% with\nplacebo”",
"“10% with\nplacebo”"])]
for yc, cells in rows:
for j, (x, w, s) in enumerate(zip(xs, ws, cells)):
ec = ORANGE if j == 3 else (BLUE if j == 1 else "#cfcdc8")
ax.add_patch(FancyBboxPatch((x, yc - 0.13), w, 0.26, boxstyle="round,pad=0,rounding_size=0.012",
fc=PANEL, ec=ec, lw=2 if j in (1, 3) else 1))
ax.text(x + w / 2, yc, s, ha="center", va="center", fontsize=10.5,
color=TXT)
for j in range(3):
x0 = xs[j] + ws[j] + 0.005; x1 = xs[j + 1] - 0.005
ok = j < 2
ax.annotate("", xy=(x1, yc), xytext=(x0, yc),
arrowprops=dict(arrowstyle="->", color=SEC if ok else ORANGE, lw=2,
ls="-" if ok else "--"))
ax.text((x0 + x1) / 2, yc + 0.04, "=" if ok else "\u2260", ha="center", va="bottom",
fontsize=14, color=SEC if ok else TXT, weight="bold")
ax.text(0.01, 0.05, "Blue outline: recomputed from the paper's counts. Orange outline: abstract as printed.\n"
"= agrees after rounding; \u2260 disagrees. Any TEAE, 16-week treatment period, safety analysis set.",
fontsize=10, color=SEC, va="center")
return fig
CANDIDATES = [("cand1_text_panels", cand_text_panels), ("cand2_dot_plot", cand_dot_plot),
("cand3_search_scatter", cand_search_scatter), ("cand4_bars", cand_bars),
("cand5_calc_flow", cand_flow)]
if WRITE_CANDIDATES:
os.makedirs(CAND, exist_ok=True)
for name, fn in CANDIDATES:
save(fn(), os.path.join(CAND, name))
# Provisional choice for the write-up (to be swapped once a candidate is picked):
save(cand_text_panels(), os.path.join(HERE, "fig1"))
save(cand_search_scatter(), os.path.join(HERE, "fig2"))
print("\nwrote fig1, fig2" + (" and candidates/cand1..cand5" if WRITE_CANDIDATES else "") + " (.svg and .png)")
PIK3CA progression-free survival P = 0.016 cannot be derived from its reported HR and CI
BMC Med (2023) · doi:10.1186/s12916-023-03017-z · PMC10426213
Summary
This longitudinal ctDNA analysis of the phase Ib MONALEESASIA trial reports that patients with baseline PIK3CA alterations had shorter median progression-free survival (PFS) than patients without them, 12.7 vs 19.2 months, with P = 0.016. The Abstract gives this P value alongside the TP53 result and concludes that PIK3CA alterations "were associated with inferior outcomes". In the Results, the same P value is printed next to a hazard ratio of 0.93 (95% CI 0.52–1.69) in a sentence that calls the difference "not statistically significant". A Wald test back-calculated from that HR and CI gives z = −0.24 and P = 0.81, while the same check reproduces the paper's P values for TP53, bTMB and ctDNA detectability. The PIK3CA association in the Abstract therefore has no statistical support in the paper's own estimate. The paper's main findings on ctDNA monitoring do not depend on it.
What the paper reports
Abstract, Results:
Patients with alterations in PIK3CA and TP53 at baseline had shorter median progression-free survival compared with patients with wild-type PIK3CA and TP53, 12.7 and 7.3 months vs 19.2 and 19.4 months, respectively (P = .016 and P = .0001, respectively).
Abstract, Conclusions:
Higher ctDNA levels and PIK3CA and TP53 alterations detected at baseline were associated with inferior outcomes.
Results, "Gene alterations at baseline" (the PIK3CA and TP53 sentences, with the bTMB sentence that follows):
Median PFS of patients with TP53 mutations was significantly shorter than those without TP53 mutations (7.3 months vs 19.4 months; HR, 0.26; 95% CI, 0.12–0.53; P < 0.001; Fig. 7A). Median PFS of patients with PIK3CA mutations was 12.7 months vs 19.2 months for those without PIK3CA mutations; however, the difference was not statistically significant (HR, 0.93; 95% CI, 0.52–1.69; P = 0.016; Fig. 7B). Patients with undetectable ctDNA experienced a median PFS of 37.8 months (Fig. 7B).
In patients with detectable ctDNA, the median PFS of patients with high bTMB (> 10 mutations per megabase [mut/Mb]) was 12.7 months vs 19.4 months for those with low bTMB (< 10 mut/Mb); this numerically shorter PFS difference was not statistically significant (HR, 0.83; 95% CI, 0.46–1.5; P = 0.53; Fig. 7C).
Same paragraph, last sentence:
Therefore, in this particular cohort, neither bTMB nor PIK3CA status seems to provide an additional prognostic effect beyond ctDNA fraction.
Results, "ctDNA fraction by BOR and PFS" (used below as a control comparison):
More generally, patients with no detectable ctDNA at baseline had a PFS almost twice that observed in patients with detectable ctDNA at baseline: 38.0 months vs 19.2 months, respectively (hazard ratio [HR], 0.45; 95% CI, 0.24–0.84; P = 0.01; Fig. 1C).
Methods, "Statistical analysis":
A Cox proportional-hazards model was used to evaluate differences in PFS in the MONALEESASIA cohorts by ctDNA detectable vs undetectable status and mutation status in PIK3CA, TP53.
Methods, "Sample collection and ctDNA analysis":
All comparisons between mutated vs nonmutated (e.g., TP53 mutated vs nonmutated) were performed on patients with detectable ctDNA.
Figure 7 caption:
Progression-free survival for ribociclib plus ET in patients with or without mutations. A TP53. B PIK3CA. C High versus low TMB. Analysis by TMB (high versus low) only included patients with detectable ctDNA.
Figure 7 itself is an image. In the publisher's full-resolution file (12916_2023_3017_Fig7_HTML.png), panel B prints "P=. 016", panel A "P=.0001" and panel C "P=.53". Panels A and B each draw three Kaplan–Meier curves (mutated, nonmutated and "not detected/low power"); panel C draws two. The numbers at risk at time 0 are 12, 42 and 28 in panel A, 24, 30 and 28 in panel B, and 27 and 27 in panel C.
What the paper's own numbers show
Method. For a Cox model, a 95% Wald confidence interval is exp(b ± 1.96·SE), where b is the log hazard ratio. The standard error can therefore be recovered from the printed limits as SE = (ln upper − ln lower) / (2 × 1.96). The Wald statistic is z = ln(HR) / SE, and the two-sided P value is 2·(1 − Φ(|z|)). I applied this to all four PFS hazard ratios that the paper prints with a CI and a P value, using the three non-PIK3CA comparisons as controls for the method. I checked that each printed HR sits at the log-scale midpoint of its CI, that rounding of the printed values cannot move the result, and what HR or CI a P value of 0.016 would require. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| PIK3CA mutated vs nonmutated, P | Results; Abstract; Fig. 7B | HR 0.93 (95% CI 0.52–1.69); P = 0.016 | SE(ln HR) = 0.301, z = −0.24, Wald P = 0.81 |
| PIK3CA: rounding range of HR and CI limits | derived | HR 0.93, CI 0.52–1.69 | over HR 0.925–0.935 and limits ±0.005: P = 0.79–0.83 |
| PIK3CA: CI log-midpoint | derived | HR 0.93 | 0.937, so the HR and CI fit each other |
| PIK3CA: |z| that P = 0.016 implies | derived | P = 0.016 | |z| = 2.41, ten times the 0.24 that the CI gives |
| PIK3CA: CI that P = 0.016 would need at HR 0.93 | derived | CI 0.52–1.69 (SE 0.301) | SE 0.030, CI 0.877–0.987 (one tenth of the printed width) |
| PIK3CA: HR that P = 0.016 would need with the printed CI width | derived | HR 0.93 | 0.485 (CI 0.27–0.87) or 2.063 (CI 1.14–3.72) |
| TP53 mutated vs nonmutated, P (control) | Results; Abstract; Fig. 7A | HR 0.26 (0.12–0.53); P < 0.001 (Results), P = .0001 (Abstract, Fig. 7A) | z = −3.55, Wald P = 0.00038: consistent with < 0.001 |
| bTMB high vs low, P (control) | Results; Fig. 7C | HR 0.83 (0.46–1.5); P = 0.53 | z = −0.62, Wald P = 0.54: consistent |
| ctDNA undetectable vs detectable, P (control) | Results; Fig. 1C | HR 0.45 (0.24–0.84); P = 0.01 | z = −2.50, Wald P = 0.012: consistent |
| Numbers at risk at time 0 | Fig. 7A / 7B / 7C | 12 + 42 + 28; 24 + 30 + 28; 27 + 27 | 82, 82 and 54; the mutated vs nonmutated comparisons use the 54 patients with detectable ctDNA |
| Median PFS, undetectable ctDNA | Results (Fig. 1C text vs Fig. 7B text) | 38.0 months; 37.8 months | differ by 0.2 months (minor, separate from the P value issue) |
The method reproduces three of the four printed P values to within rounding. The PIK3CA value is the exception, by a factor of about 50 in P and 10 in z. The PIK3CA confidence interval contains 1, and an HR of 0.93 is close to no difference, so the Results sentence's own description ("not statistically significant") matches the HR and CI. It does not match the P value printed in the same parentheses.
Figure
Figure 1. For each of the four PFS comparisons that the paper reports with an HR, a 95% CI and a P value, the orange dot is the P value printed in the paper and the blue dot is the two-sided Wald P value recomputed from that HR and CI. The axis is logarithmic; the dashed line marks P = 0.05. For TP53 the orange dot is the Abstract and Figure 7A value (.0001); the Results give P < 0.001, which the recomputed 0.00038 satisfies. For bTMB and ctDNA detectability the two values coincide within rounding. For PIK3CA the printed P = 0.016 lies on the other side of 0.05 from the P = 0.81 that its own HR and CI imply.
Figure 2. The PIK3CA hazard ratio on a log scale, with HR = 1 as the reference line. Orange: the reported HR 0.93 with its 95% CI 0.52–1.69, which gives Wald P = 0.81. Blue: what P = 0.016 would require, either a CI one tenth as wide around the same HR (0.88–0.99), or, with the reported CI width, an HR of 0.48 or 2.06 whose interval excludes 1. Neither resembles the reported estimate, so a misprint in the HR alone or in one CI limit alone cannot produce the printed P value.
Why it matters
The PIK3CA result appears in the Abstract's Results with P = .016, paired with the TP53 result, and the Abstract's Conclusions list PIK3CA alterations among the baseline features "associated with inferior outcomes". A reader of the Abstract would take PIK3CA as a statistically supported adverse prognostic marker in this cohort. If the hazard ratio and confidence interval are correct, the comparison gives P ≈ 0.81 and an HR close to 1 with a wide interval (0.52–1.69). On that reading the data show no evidence of a PFS difference by PIK3CA status among patients with detectable ctDNA, which is also what the Results sentence and the paragraph's closing statement ("neither bTMB nor PIK3CA status seems to provide an additional prognostic effect beyond ctDNA fraction") say.
The effect is limited to this result. The full-text Conclusions do not rely on PIK3CA, and the Discussion describes the PIK3CA effect as "numerically shorter". The paper's main findings are unaffected: the decline of ctDNA on treatment, the relation between baseline ctDNA and PFS (HR 0.45, P = 0.01, reproduced as 0.012), the lead time of ctDNA increase before radiological progression, and the TP53 association (reproduced as P = 0.00038). The Discussion sentence "Patients with alterations in these genes experienced a shorter median PFS compared with patients with wild-type versions" remains true as a statement about medians but has no statistical support for PIK3CA.
A smaller point of interpretation: the printed HRs for TP53 (0.26), PIK3CA (0.93) and bTMB (0.83) are all below 1 although the mutated or high-bTMB groups have the shorter medians. They therefore appear to express the nonmutated or low-bTMB group relative to the other (for PIK3CA, about 1.08, 95% CI 0.59–1.92, mutated vs nonmutated). The direction does not change any P value in this write-up.
Explanations considered
- Typographical error in the HR or one CI limit. This does not account for the numbers. The printed HR lies at the log-scale midpoint of its CI (0.937 vs 0.93), so the three numbers fit each other. With the printed CI width, P = 0.016 would need an HR of 0.485 or 2.06; at HR 0.93 it would need a CI of 0.88–0.99. Changing any one printed number by a plausible typo does not get close.
- Rounding. This does not account for the numbers. Over the full rounding range of the HR and both limits, the Wald P stays between 0.79 and 0.83.
- A different test (for example a log-rank or likelihood-ratio test) for the same two groups. This is unlikely to account for the numbers. For a single two-group Cox comparison, Wald, score (log-rank) and likelihood-ratio tests usually agree closely; a gap from P = 0.81 to P = 0.016 would be extraordinary. The same Wald check reproduces the paper's P values for bTMB and for ctDNA detectability, which are two-group comparisons of similar size.
- The P value in Figure 7B is a three-group test that includes the ctDNA-undetectable patients. This is the most likely source of the number, but it is not confirmed. Panels 7A and 7B each draw three curves (mutated, nonmutated, and "not detected/low power", median PFS 37.8 months in the undetectable group), and the Methods name no test for the figures. The P values printed in those two panels fit a global test better than the pairwise one. A rough approximation that adds the 1-df detectable-vs-undetectable Wald component (χ² = 6.24, from Fig. 1C) to the pairwise component gives a 2-df P of about 0.04 for the PIK3CA panel (printed .016; 8.27 is needed for exactly 0.016) and about 0.00008 for the TP53 panel (printed .0001), whereas the pairwise TP53 Wald P is 0.00038. Panel 7C, which has only two curves, prints P = .53, matching its pairwise HR. If this is the explanation, the printed P = 0.016 tests whether PFS differs among the three groups, a difference driven largely by the long PFS of patients with undetectable ctDNA. It does not test PIK3CA mutated vs nonmutated, so it cannot support the Abstract's comparison "with wild-type PIK3CA", nor sit alongside an HR that the Methods say is computed only in patients with detectable ctDNA. The approximation ignores the correlation between components, so it shows only that the order of magnitude fits.
- A conversion or typesetting error in the archived text. This is excluded. The PMC text, the publisher-supplied JATS XML in Europe PMC, and the publisher's Figure 7 image all carry P = .016 for PIK3CA.
- The value is explained in a supplement. This is excluded. Neither supplementary file reports PIK3CA survival statistics (see Status).
Status
- Version of record. I compared the PMC-archived text with the publisher-supplied JATS XML in Europe PMC (PMC10426213/fullTextXML). Both carry the Abstract, Results and Methods sentences quoted above with identical values. I also read the publisher's full-resolution Figure 7 image (media.springernature.com, 12916_2023_3017_Fig7_HTML.png) for the printed P values, curves and numbers at risk. The Springer HTML and PDF pages could not be fetched automatically: they redirect to a cookie-authorization page. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py, except the Figure 7 panel contents, which were read from the image. - Supplements. Additional file 1 (12916_2023_3017_MOESM1_ESM.xlsx) is the 572-gene panel list (eTable 2). Additional file 2 (12916_2023_3017_MOESM2_ESM.docx) holds Supplementary Table 1 (cfDNA samples per cohort), Supplementary Table 2 (ctDNA change vs tumor change by response group) and Supplementary Fig. 1. Neither reports PIK3CA PFS, HRs or P values, and there is no patient-level survival data, so the correct P value cannot be computed from published material.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. A web search for corrections or comments on the article found none. The PubPeer API and search page refused automated access (Cloudflare challenge and HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC10426213; Europe PMC full-text XML for PMC10426213; publisher Figure 7 image and supplementary files from static-content.springer.com; Crossref; Europe PMC; trial registration NCT02333370 (cited in the paper).
Suggested correction
- Report the two-group Cox P value for PIK3CA mutated vs nonmutated among patients with detectable ctDNA, which the printed HR 0.93 (95% CI 0.52–1.69) implies is about 0.8, in the Results sentence and in the Abstract.
- Revise the Abstract's Results and Conclusions so that PIK3CA alterations are not presented as associated with inferior PFS on the basis of P = .016, for example by limiting the statistical association to TP53 and describing the PIK3CA difference in medians as not statistically significant.
- State in the Methods or the Figure 7 legend which test produced the P values printed on Figure 7A and 7B. If they are global tests across three groups including the ctDNA-undetectable patients, say so, and do not cite them as mutated vs wild-type comparisons.
- State the reference group of the hazard ratios (the HRs below 1 appear to compare nonmutated or low-bTMB patients with the others).
- Minor: reconcile the median PFS for patients with undetectable ctDNA (38.0 months in the Fig. 1C text, 37.8 months in the Fig. 7B text), or explain the difference.
Recomputation script (python recompute.py.txt)
"""Recompute the PFS P values of PMC10426213
(BMC Med 2023, doi:10.1186/s12916-023-03017-z, longitudinal ctDNA analysis of MONALEESASIA).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md,
writes fig1.svg/png and fig2.svg/png next to this file, and writes five candidate
figure designs to candidates/ (c1..c5, .svg and .png).
Inputs are copied from the paper as archived in PMC:
Results, "ctDNA fraction by BOR and PFS": ctDNA undetectable vs detectable, HR 0.45 (0.24-0.84), P = 0.01 (Fig. 1C)
Results, "Gene alterations at baseline": TP53 HR 0.26 (0.12-0.53), P < 0.001 (Fig. 7A)
PIK3CA HR 0.93 (0.52-1.69), P = 0.016 (Fig. 7B)
bTMB HR 0.83 (0.46-1.5), P = 0.53 (Fig. 7C)
Abstract: PIK3CA P = .016, TP53 P = .0001
Figure 7 (publisher image, numbers at risk at t = 0): A 12 mut / 42 nonmut / 28 not detected;
B 24 mut / 30 nonmut / 28 not detected; C 27 / 27
Method: for a Cox model, a 95% Wald CI is exp(b +/- 1.96 SE), so SE = (ln hi - ln lo) / (2 * 1.96),
z = ln(HR) / SE and the two-sided Wald P = 2 * (1 - Phi(|z|)).
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z975 = stats.norm.isf(0.025)
# ---------------------------------------------------------------- inputs (from the paper)
# key: (label, HR, lo, hi, reported P as a number for plotting, reported P as printed, location)
ROWS = [
("pik3ca", "PIK3CA mut vs nonmut", 0.93, 0.52, 1.69, 0.016, "P = 0.016 (Results); P = .016 (Abstract, Fig. 7B)", "Results; Abstract; Fig. 7B"),
("tp53", "TP53 mut vs nonmut", 0.26, 0.12, 0.53, 0.0001, "P < 0.001 (Results); P = .0001 (Abstract, Fig. 7A)", "Results; Abstract; Fig. 7A"),
("tmb", "bTMB high vs low", 0.83, 0.46, 1.50, 0.53, "P = 0.53 (Results); P = .53 (Fig. 7C)", "Results; Fig. 7C"),
("ctdna", "ctDNA undetectable vs detectable", 0.45, 0.24, 0.84, 0.01, "P = 0.01 (Results, Fig. 1C)", "Results; Fig. 1C"),
]
AT_RISK = {"Fig. 7A (TP53)": (12, 42, 28), "Fig. 7B (PIK3CA)": (24, 30, 28), "Fig. 7C (bTMB)": (27, 27)}
MEDIANS = {"PIK3CA mut": 12.7, "PIK3CA nonmut": 19.2, "TP53 mut": 7.3, "TP53 nonmut": 19.4,
"undetectable (Fig. 7B text)": 37.8, "undetectable (Fig. 1C text)": 38.0, "detectable (Fig. 1C text)": 19.2}
def wald(hr, lo, hi):
se = (math.log(hi) - math.log(lo)) / (2 * Z975)
z = math.log(hr) / se
return se, z, 2 * stats.norm.sf(abs(z))
R = {}
print("=== 1. Wald P back-calculated from each reported HR and 95% CI (two-sided) ===")
for key, lab, hr, lo, hi, prep, prep_txt, loc in ROWS:
se, z, p = wald(hr, lo, hi)
mid = math.exp((math.log(lo) + math.log(hi)) / 2)
R[key] = dict(lab=lab, hr=hr, lo=lo, hi=hi, se=se, z=z, p=p, prep=prep, prep_txt=prep_txt,
zrep=stats.norm.isf(prep / 2))
print(f" {lab:34s} HR {hr} ({lo}-{hi}): SE(lnHR) = {se:.3f}, z = {z:+.2f}, P = {p:.2g} ({p:.4f})"
f" reported: {prep_txt}")
print(f" {'':34s} CI log-midpoint = {mid:.3f} (HR printed {hr}); |z| implied by reported P = {R[key]['zrep']:.2f}")
pk = R["pik3ca"]
print("\n=== 2. What P = 0.016 would require for PIK3CA ===")
z016 = stats.norm.isf(0.016 / 2)
se_req = abs(math.log(pk["hr"])) / z016
lo_req, hi_req = math.exp(math.log(pk["hr"]) - Z975 * se_req), math.exp(math.log(pk["hr"]) + Z975 * se_req)
hr_lo_req, hr_hi_req = math.exp(-z016 * pk["se"]), math.exp(z016 * pk["se"])
R["req"] = dict(z016=z016, se_req=se_req, lo_req=lo_req, hi_req=hi_req, hr_lo_req=hr_lo_req, hr_hi_req=hr_hi_req)
print(f" |z| for two-sided P = 0.016: {z016:.2f}; reported CI gives |z| = {abs(pk['z']):.2f} "
f"({z016 / abs(pk['z']):.1f} times smaller)")
print(f" at HR 0.93, P = 0.016 needs SE(lnHR) = {se_req:.4f} (reported CI implies {pk['se']:.3f}, "
f"{pk['se'] / se_req:.1f} times larger), i.e. a 95% CI of {lo_req:.3f}-{hi_req:.3f}")
for h in (hr_lo_req, hr_hi_req):
print(f" with the reported CI width, P = 0.016 needs HR = {h:.3f}, CI "
f"{math.exp(math.log(h) - Z975 * pk['se']):.2f}-{math.exp(math.log(h) + Z975 * pk['se']):.2f}")
print(f" reported CI 0.52-1.69 contains HR = 1: {pk['lo'] < 1 < pk['hi']}")
print("\n=== 3. Sensitivity to rounding of the printed HR and CI limits (PIK3CA) ===")
ps = []
for hr in np.arange(0.925, 0.9351, 0.005):
for lo in np.arange(0.515, 0.5251, 0.005):
for hi in np.arange(1.685, 1.6951, 0.005):
ps.append(wald(hr, lo, hi)[2])
print(f" HR 0.925-0.935, lo 0.515-0.525, hi 1.685-1.695: Wald P ranges {min(ps):.3f}-{max(ps):.3f}")
print("\n=== 4. A possible source of the figure P values: a 3-group test (rough approximation) ===")
chi_det = R["ctdna"]["z"] ** 2
chi_pk = R["pik3ca"]["z"] ** 2
chi_tp = R["tp53"]["z"] ** 2
need016, need0001 = stats.chi2.isf(0.016, 2), stats.chi2.isf(0.0001, 2)
p3_pk = stats.chi2.sf(chi_det + chi_pk, 2)
p3_tp = stats.chi2.sf(chi_det + chi_tp, 2)
R["three"] = dict(chi_det=chi_det, chi_pk=chi_pk, chi_tp=chi_tp, p3_pk=p3_pk, p3_tp=p3_tp,
need016=need016, need0001=need0001)
print(f" Wald chi2 components: detectable vs undetectable {chi_det:.2f}; PIK3CA pairwise {chi_pk:.3f}; TP53 pairwise {chi_tp:.2f}")
print(f" 2-df chi2 needed for P = 0.016: {need016:.2f}; for P = 0.0001: {need0001:.2f}")
print(f" PIK3CA panel (mut / nonmut / not detected): approx chi2 {chi_det + chi_pk:.2f}, 2-df P ~ {p3_pk:.3f}"
f" (printed .016; pairwise Wald {R['pik3ca']['p']:.2f})")
print(f" TP53 panel (mut / nonmut / not detected): approx chi2 {chi_det + chi_tp:.2f}, 2-df P ~ {p3_tp:.1g}"
f" (printed .0001; pairwise Wald {R['tp53']['p']:.5f})")
print(" (Approximation only: adds a 1-df detectable-vs-undetectable component to the 1-df pairwise one.)")
print("\n=== 5. Numbers at risk at t = 0 in Figure 7 ===")
for k, v in AT_RISK.items():
print(f" {k}: {' + '.join(map(str, v))} = {sum(v)}")
print(f" detectable ctDNA in panels A/B: {24 + 30} = {12 + 42}; PIK3CA mut share of detectable {24 / 54:.1%}, of all 82 {24 / 82:.1%}")
print(f" TP53 mut share of detectable {12 / 54:.1%}, of all 82 {12 / 82:.1%}")
print("\n=== 6. Other small observations ===")
print(f" median PFS, undetectable ctDNA: {MEDIANS['undetectable (Fig. 1C text)']} months (Fig. 1C text) vs "
f"{MEDIANS['undetectable (Fig. 7B text)']} months (Fig. 7B text)")
print(" The TP53, PIK3CA and bTMB HRs are < 1 although the mutated / high-bTMB group has the shorter median,")
print(" so they appear to express the nonmutated / low group relative to the other; the Wald P is unaffected.")
print(f" e.g. 1/0.93 = {1 / 0.93:.2f} (1/1.69 = {1 / 1.69:.2f} to 1/0.52 = {1 / 0.52:.2f}); "
f"1/0.26 = {1 / 0.26:.2f} (1/0.53 = {1 / 0.53:.2f} to 1/0.12 = {1 / 0.12:.2f})")
# ---------------------------------------------------------------- figure style
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, NEUTRAL = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"lines.linewidth": 1.5, "xtick.labelsize": 10, "ytick.labelsize": 10.5,
})
LW = 1.5 # 1.5 pt = 2 px at 96 dpi
MS = 9 # marker size in points (>= 8 px)
def save(fig, folder, stem):
fig.savefig(os.path.join(folder, stem + ".svg"))
fig.savefig(os.path.join(folder, stem + ".png"), dpi=200)
plt.close(fig)
def clean(ax, grid_axis="x"):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
ORDER = ["pik3ca", "tp53", "tmb", "ctdna"]
YLAB = {"pik3ca": "PIK3CA mutated vs not\n(Fig. 7B)", "tp53": "TP53 mutated vs not\n(Fig. 7A)",
"tmb": "bTMB high vs low\n(Fig. 7C)", "ctdna": "ctDNA undetectable vs\ndetectable (Fig. 1C)"}
PREP_SHORT = {"pik3ca": "0.016", "tp53": ".0001", "tmb": "0.53", "ctdna": "0.01"}
def fmtp(p):
if p < 0.001:
return f"{p:.5f}".rstrip("0")
return f"{p:.2g}" if p < 0.1 else f"{p:.2f}"
# ---- C1: P-value ladder (reported P vs P recomputed from HR/CI, log axis) ----
REP_LABEL = {"pik3ca": "reported 0.016", "tp53": "reported .0001 (Abstract); < 0.001 (Results)",
"tmb": "reported 0.53", "ctdna": "reported 0.01"}
def draw_c1(folder, stem):
fig, ax = plt.subplots(figsize=(7.2, 4.2))
for i, k in enumerate(ORDER):
r = R[k]
y = len(ORDER) - 1 - i
ax.plot([r["prep"], r["p"]], [y, y], color=NEUTRAL, lw=LW, zorder=2)
ax.plot(r["prep"], y, "o", ms=MS, color=ORANGE, zorder=3)
ax.plot(r["p"], y, "o", ms=MS, color=BLUE, zorder=4, mfc=BLUE if k != "tmb" else "none", mew=2)
ax.annotate(REP_LABEL[k], (r["prep"], y), xytext=(-8 if k == "tp53" else 0, 8), textcoords="offset points",
ha={"tmb": "right", "tp53": "left"}.get(k, "center"), va="bottom", fontsize=10, color=TXT)
ax.annotate("recomputed " + fmtp(r["p"]), (r["p"], y), xytext=(0, -9), textcoords="offset points",
ha="center" if k != "tmb" else "right", va="top", fontsize=10, color=TXT)
ax.axvline(0.05, color=SEC, lw=0.8, ls=(0, (3, 2)), zorder=1)
ax.text(0.052, -0.85, "P = 0.05", ha="left", va="bottom", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(2e-5, 2)
ax.set_ylim(-0.9, len(ORDER) - 0.45)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([YLAB[k] for k in reversed(ORDER)])
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.1, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.1", "1"])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlabel("Two-sided P value (log scale)")
clean(ax)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="P as reported in the paper"),
Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, label="Wald P from the paper's own HR and 95% CI")],
loc="upper center", bbox_to_anchor=(0.4, -0.17), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, folder, stem)
# ---- C2: the interval P = 0.016 would require (HR log axis) ----
def draw_c2(folder, stem):
q = R["req"]
se = pk["se"]
rows = [
("Reported\nHR 0.93 (0.52–1.69), P = 0.016", pk["hr"], pk["lo"], pk["hi"], ORANGE, "o"),
("95% CI that P = 0.016 would need\nat HR 0.93", pk["hr"], q["lo_req"], q["hi_req"], BLUE, "o"),
("HR that P = 0.016 would need\nwith the reported CI width", q["hr_lo_req"],
math.exp(math.log(q["hr_lo_req"]) - Z975 * se), math.exp(math.log(q["hr_lo_req"]) + Z975 * se), BLUE, "D"),
("", q["hr_hi_req"], math.exp(math.log(q["hr_hi_req"]) - Z975 * se),
math.exp(math.log(q["hr_hi_req"]) + Z975 * se), BLUE, "D"),
]
ys = [3, 2, 1, 0.55]
fig, ax = plt.subplots(figsize=(7.2, 3.8))
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
for (lab, hr, lo, hi, col, mk), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=LW * 1.4, solid_capstyle="butt", zorder=2)
ax.plot(hr, y, mk, ms=MS, color=col, zorder=3)
ax.annotate(f"{hr:.2f} ({lo:.2f}–{hi:.2f})", (hi, y), xytext=(6, 0), textcoords="offset points",
va="center", fontsize=10, color=TXT)
ax.text(pk["hr"], 2.32, f"Wald P from this interval: {pk['p']:.2f}", ha="center", va="bottom", fontsize=10, color=SEC)
ax.text(1.03, 0.2, "HR = 1 (no difference)", ha="left", va="bottom", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(0.2, 10)
ax.set_ylim(0.15, 3.45)
ax.set_yticks([3, 2, 0.775])
ax.set_yticklabels([rows[0][0], rows[1][0], rows[2][0]], fontsize=10)
ax.set_xticks([0.25, 0.5, 1, 2, 4])
ax.set_xticklabels(["0.25", "0.5", "1", "2", "4"])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlabel("Hazard ratio for PFS, PIK3CA comparison, with 95% CI (log scale)")
clean(ax)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="reported in the paper"),
Line2D([], [], marker="o", color=BLUE, lw=LW, ms=MS, label="what P = 0.016 would require")],
loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, folder, stem)
# ---- C3: z-score comparison (linear |z| axis, 1.96 reference) ----
def draw_c3(folder, stem):
fig, ax = plt.subplots(figsize=(7.2, 3.4))
for i, k in enumerate(ORDER):
r = R[k]
y = len(ORDER) - 1 - i
a, b = r["zrep"], abs(r["z"])
ax.plot([a, b], [y, y], color=NEUTRAL, lw=LW, zorder=2)
ax.plot(a, y, "o", ms=MS, color=ORANGE, zorder=3)
ax.plot(b, y, "o", ms=MS, color=BLUE, zorder=3)
ax.annotate(f"{a:.2f}", (a, y), xytext=(0, 9), textcoords="offset points", ha="center", fontsize=10, color=TXT)
ax.annotate(f"{b:.2f}", (b, y), xytext=(0, -17), textcoords="offset points", ha="center", fontsize=10, color=TXT)
ax.axvline(Z975, color=SEC, lw=0.8, ls=(0, (3, 2)))
ax.text(Z975, len(ORDER) - 0.45, "|z| = 1.96 (P = 0.05)", ha="center", va="bottom", fontsize=10, color=SEC)
ax.set_xlim(0, 4.4)
ax.set_ylim(-0.6, len(ORDER) - 0.2)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([YLAB[k] for k in reversed(ORDER)])
ax.set_xlabel("|z| (standard errors from no difference)")
clean(ax)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="|z| implied by the reported P"),
Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, label="|z| from the reported HR and 95% CI")],
loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, folder, stem)
# ---- C4: forest plot with reported / recomputed P columns ----
def draw_c4(folder, stem):
fig = plt.figure(figsize=(7.2, 3.3))
ax = fig.add_axes([0.27, 0.19, 0.4, 0.62])
ax.set_xscale("log")
ax.set_xlim(0.1, 2.2)
ax.set_ylim(-0.6, len(ORDER) - 0.4)
for i, k in enumerate(ORDER):
r = R[k]
y = len(ORDER) - 1 - i
ax.plot([r["lo"], r["hi"]], [y, y], color=TXT, lw=LW, zorder=2)
ax.plot(r["hr"], y, "s", ms=MS - 1, color=TXT, zorder=3)
fig.text(0.70, ax.transData.transform((0, y))[1] / fig.dpi / 3.3, PREP_SHORT[k], color=ORANGE,
fontsize=10.5, va="center", ha="left", fontweight="bold" if k == "pik3ca" else "normal")
fig.text(0.83, ax.transData.transform((0, y))[1] / fig.dpi / 3.3, fmtp(r["p"]), color=BLUE,
fontsize=10.5, va="center", ha="left", fontweight="bold" if k == "pik3ca" else "normal")
ax.axvline(1, color=SEC, lw=0.8)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([YLAB[k] for k in reversed(ORDER)], fontsize=10)
ax.set_xticks([0.125, 0.25, 0.5, 1, 2])
ax.set_xticklabels(["0.125", "0.25", "0.5", "1", "2"])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlabel("HR and 95% CI as printed (log scale)")
clean(ax)
fig.text(0.70, 0.84, "reported P", color=ORANGE, fontsize=10.5, ha="left", va="bottom")
fig.text(0.83, 0.84, "recomputed P\n(Wald, HR/CI)", color=BLUE, fontsize=10.5, ha="left", va="bottom")
save(fig, folder, stem)
# ---- C5: two text panels with an annotated connector ----
def draw_c5(folder, stem):
fig = plt.figure(figsize=(7.2, 4.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 10)
ax.set_ylim(0, 6.1)
ax.axis("off")
def panel(x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.12",
fc="#ffffff", ec=edge, lw=LW))
ax.text(x + 0.2, y + h - 0.22, title, fontsize=10.5, fontweight="bold", va="top", color=TXT)
ax.text(x + 0.2, y + h - 0.68, body, fontsize=10, va="top", color=TXT, linespacing=1.4)
panel(0.2, 2.85, 4.55, 2.75, "Abstract (Results; Conclusions)",
'"Patients with alterations in PIK3CA\nand TP53 at baseline had shorter\nmedian progression-free survival\n'
'... (P = .016 and P = .0001,\nrespectively)."\n'
'"... PIK3CA and TP53 alterations ...\nwere associated with inferior outcomes."', ORANGE)
panel(5.25, 2.85, 4.55, 2.75, "Results, Gene alterations at baseline",
'"Median PFS of patients with PIK3CA\nmutations was 12.7 months vs 19.2\nmonths ...; however, the difference\n'
'was not statistically significant\n(HR, 0.93; 95% CI, 0.52–1.69;\nP = 0.016; Fig. 7B)."', ORANGE)
panel(2.55, 0.2, 4.9, 1.75, "Recomputed from the HR and 95% CI",
f"SE(ln HR) = {pk['se']:.3f}, z = {pk['z']:.2f}, Wald P = {pk['p']:.2f}\n"
f"P = 0.016 would need |z| = {R['req']['z016']:.2f}, i.e. a\n"
f"95% CI of {R['req']['lo_req']:.2f}–{R['req']['hi_req']:.2f} at HR 0.93", BLUE)
ax.add_patch(FancyArrowPatch((4.78, 4.25), (5.22, 4.25), arrowstyle="<->", mutation_scale=14, color=SEC, lw=LW))
ax.text(5.0, 5.85, "Same comparison and the same P value: cited as an association in the Abstract,"
"\ncalled not statistically significant in the Results",
ha="center", va="center", fontsize=10, color=SEC)
ax.add_patch(FancyArrowPatch((7.5, 2.85), (6.5, 1.98), arrowstyle="->", mutation_scale=14, color=SEC, lw=LW))
ax.text(7.25, 2.4, "CI 0.52–1.69 includes 1", fontsize=10, color=SEC, va="center", ha="left")
ax.add_patch(FancyArrowPatch((2.5, 2.85), (3.5, 1.98), arrowstyle="->", mutation_scale=14, color=SEC, lw=LW))
ax.text(2.75, 2.4, "P = .016 is not\nreproduced", fontsize=10, color=SEC, va="center", ha="right")
save(fig, folder, stem)
DRAW = {"c1_p_ladder": draw_c1, "c2_required_interval": draw_c2, "c3_z_scores": draw_c3,
"c4_forest_with_p_columns": draw_c4, "c5_text_panels": draw_c5}
for stem, fn in DRAW.items():
fn(CAND, stem)
# defaults used in writeup.md
draw_c1(HERE, "fig1")
draw_c2(HERE, "fig2")
print("\nFigures written: fig1 (= candidate c1), fig2 (= candidate c2), candidates/c1..c5")
Abstract reports all pocket-depth measures significant for CVD mortality; Table 2 shows none are
BMC Med (2023) · doi:10.1186/s12916-023-03139-4 · PMC10642059
Summary
This cohort study (SHIP-START, n = 3047, 125 CVD deaths) relates periodontal and inflammatory markers to mortality. Its abstract states that "For CVD mortality, all PPD related variables showed significant associations". The paper's own Table 2 shows the opposite: for CVD mortality, none of the three pocket probing depth (PPD) measures is significant. All six confidence intervals (per unit and per SD) include 1, the printed P values are 0.275, 0.302 and 0.337, and the paper's Results text says the same thing. The numbers in the table are internally consistent: Wald P values recomputed from the printed intervals are 0.274, 0.303 and 0.336. The abstract sentence is the only single-exposure CVD result it reports. Read alone, it reverses the finding for the periodontal marker that the paper describes as best reflecting current periodontal inflammation. The only published correction (January 2025) concerns funding and leaves the sentence unchanged.
What the paper reports
Abstract, Results:
For CVD mortality, all PPD related variables showed significant associations.
Results, "Inflammation and periodontitis as risk factors for mortality":
However, compared with the results for CAL and missing teeth, the PPD measures showed only weak associations with mortality, as indicated by smaller effect sizes. For CVD mortality, neither mean PPD nor extent PPD ≥ 3 mm/4 mm showed a relevant association (Table 2). Classical Cox proportional hazards models with fixed baseline exposures and covariates (Additional file 1: Table S4) confirmed these results.
Table 2, PPD rows, CVD-mortality columns. The columns are HR (95% CI) per 1-unit increase | HR (95% CI) per SD increase | P-value:
Mean PPD, mm | 1.111 (0.920; 1.341) | 1.078 (0.942; 1.233) | 0.275
Sites with PPD ≥ 3 mm, % | 1.004 (0.997; 1.011) | 1.096 (0.921; 1.305) | 0.302
Sites with PPD ≥ 4 mm, % | 1.004 (0.996; 1.012) | 1.070 (0.932; 1.228) | 0.337
Methods, "Statistical analyses", on the significance threshold:
Regarding the interaction terms, two-sided P values < 0.10 were considered statistically significant. Otherwise, 0.05 was set as the cut-off for significance.
A related sentence in the Abstract, Results, which comes just before the CVD sentence:
All markers of periodontitis and inflammation showed apparent associations with all-cause mortality (HRs per SD-increase: mean PPD: 1.068 (95% confidence interval (CI): 0.988–1.155) […]
The Abstract, Conclusions, and the main-text Conclusions:
Both periodontitis and inflammation were significantly associated with all-cause mortality and CVD mortality.
Periodontitis and systemic inflammation were both significantly associated with all considered types of mortality.
Discussion, on how PPD is interpreted:
measures of PPD, which best reflects current periodontal inflammation
What the paper's own numbers show
Method. Table 2 prints, for each exposure and outcome, an HR and 95% CI per unit, an HR and 95% CI per SD, and a single P value. I assumed each interval is a Wald interval, symmetric on the log-HR scale, as Cox model output is. On that basis I recomputed P from each printed interval: SE = (ln U − ln L)/(2 × 1.96), z = ln HR / SE, P = 2[1 − Φ(|z|)]. I also worked the other way and rebuilt each per-SD interval from the HR and the printed P. To confirm that the table's rows and columns are aligned, I ran the same check on the other CVD rows and on the PPD rows for the other two outcomes. I repeated it for Table S4 in Additional file 1, the time-invariant sensitivity models. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Mean PPD, CVD mortality, per SD | Table 2 | 1.078 (0.942; 1.233), P = 0.275 | CI includes 1; P = 0.274; CI rebuilt from P: 0.942–1.234 |
| Mean PPD, CVD mortality, per 1 mm | Table 2 | 1.111 (0.920; 1.341) | CI includes 1; P = 0.273 |
| Sites PPD ≥ 3 mm, CVD mortality, per SD | Table 2 | 1.096 (0.921; 1.305), P = 0.302 | CI includes 1; P = 0.303; CI rebuilt from P: 0.921–1.304 |
| Sites PPD ≥ 3 mm, CVD mortality, per 1 % | Table 2 | 1.004 (0.997; 1.011) | CI includes 1; P = 0.262 (anywhere from 0.174 to 0.359 within 3-decimal rounding of the HR and limits) |
| Sites PPD ≥ 4 mm, CVD mortality, per SD | Table 2 | 1.070 (0.932; 1.228), P = 0.337 | CI includes 1; P = 0.336; CI rebuilt from P: 0.932–1.228 |
| Sites PPD ≥ 4 mm, CVD mortality, per 1 % | Table 2 | 1.004 (0.996; 1.012) | CI includes 1; P = 0.326 (0.239–0.419 within rounding) |
| PPD measures significant for CVD mortality (P < 0.05) | Abstract vs Table 2 | Abstract: "all" (3 of 3) | 0 of 3 printed P; 0 of 6 recomputed P (both scalings); 0 of 6 CIs exclude 1 |
| One-sided P (printed P halved), the three PPD measures | derived | — | 0.138, 0.151, 0.169, all still > 0.05 |
| Control: mean CAL, CVD, per SD | Table 2 | 1.225 (1.042; 1.439), P = 0.014 | P = 0.014 |
| Control: missing teeth, CVD, per SD | Table 2 | 1.226 (1.025; 1.466), P = 0.026 | P = 0.026 |
| Control: periodontitis score, CVD, per SD | Table 2 | 1.399 (1.076; 1.820), P = 0.012 | P = 0.012 |
| Control: fibrinogen, CVD, per SD | Table 2 | 1.128 (0.981; 1.297), P = 0.090 | P = 0.091 |
| Control: sites CAL ≥ 3 mm, CVD, per SD | Table 2 | 1.288 (0.976; 1.702), P = 0.074 | P = 0.074 |
| Control: sites CAL ≥ 4 mm, CVD, per SD | Table 2 | 1.217 (1.002; 1.479), P = 0.047 | P = 0.048 |
| PPD measures, time-invariant models, CVD, per SD | Table S4 | 1.080 (0.927; 1.259) P = 0.321; 1.057 (0.883; 1.264) P = 0.547; 1.057 (0.917; 1.219) P = 0.444 | P = 0.324, 0.545, 0.445; 0 of 3 significant |
| PPD measures significant, other outcomes | Table 2 | all-cause P = 0.098, 0.025, 0.030; non-CVD P = 0.166, 0.030, 0.037 | 2 of 3 for each outcome; mean PPD not significant for either |
| PPD measures significant, other outcomes | Table S4 | all-cause P = 0.020, 0.031, 0.042; non-CVD P = 0.023, 0.018, 0.035 | 3 of 3 for each outcome |
| Mean PPD, all-cause mortality, per SD (abstract: "apparent association") | Abstract; Table 2 | 1.068 (0.988–1.155), P = 0.098 | CI includes 1; P = 0.099 |
Every recomputed P agrees with the printed P to within rounding, both for the PPD rows and for the control rows. The only exceptions are the per-unit rows for extent PPD, and those have little precision because the HR and its limits are printed to three decimals close to 1. The table is therefore internally consistent. What disagrees with it is the abstract sentence.
Figure
Figure 1. The left panel shows the abstract sentence on CVD mortality. The right panel shows the Results sentence and the three Table 2 rows it summarises, with the per-SD hazard ratio (95% CI), the P value printed in the paper (orange circle) and the P value recomputed from the printed interval (blue square). The connector names the conflict. The abstract implies 3 of 3 PPD measures are significant; the table has 0 of 3. All numbers are printed in the paper or by recompute.py.txt. The paper's own figures are not reproduced.
Figure 2. Hazard ratios per SD increase for CVD mortality from Table 2, on a log axis with a reference line at HR = 1. For each row, the orange interval is the 95% CI printed in the paper and the blue interval is the 95% CI rebuilt from the HR and the printed P. The printed P values are listed on the right. The two intervals coincide for every row, so the HR, CI and P columns are mutually consistent. All three PPD intervals cross 1. The three control rows from the same table (mean CAL, missing teeth, periodontitis score) do not, which shows that significant CVD associations in this table are reported as such.
Why it matters
Most readers see only the abstract, and in it this sentence is the only result given for a single exposure and CVD mortality. It tells them that every pocket-depth measure is significantly associated with cardiovascular death, but the paper's analysis finds no PPD measure significantly associated with CVD mortality as a main effect, in either the main models (Table 2) or the sensitivity models (Table S4). The difference matters for interpretation. The paper describes PPD as the measure that "best reflects current periodontal inflammation". In this paper, the CVD association is carried by markers of cumulative periodontal damage instead: mean CAL, extent CAL ≥ 4 mm, missing teeth and the periodontitis score. Someone relying on the abstract might cite this study as evidence that current periodontal inflammation, measured as pocket depth, predicts cardiovascular death, but the paper's main-effect analyses do not support that.
The error does not overturn the paper's other main results. The periodontitis score, mean CAL, missing teeth, leukocytes, CRP and the inflammation score are all significant for CVD mortality in Table 2. The additive-interaction (RERI) results are separate analyses and are not affected. Two related statements are broader than the table supports, though. First, the abstract lists mean PPD (P = 0.098) among markers with "apparent associations" with all-cause mortality. Second, both Conclusions say periodontitis and inflammation were significantly associated with CVD mortality, and that is not true for any PPD measure, for extent CAL ≥ 3 mm (P = 0.074) or for fibrinogen (P = 0.090).
Explanations considered
- Table 2 is wrong and the abstract is right. This does not fit. For every PPD row and every control row, the P values recomputed from the printed intervals match the printed P values. The Results text describes the same null result in words, and Table S4 shows it again in separately fitted time-invariant models on the same cohort. Three parts of the paper agree with one another and disagree with the abstract.
- Per-unit versus per-SD scaling. This is not the explanation. All six CVD intervals include 1, under both scalings.
- A different model (time-invariant exposures, competing risks). This is not the explanation. The Results say Table S4 "confirmed these results", and in Table S4 the CVD P values are 0.321, 0.547 and 0.444. The Methods mention competing-risk models (subdistribution hazards) for CVD mortality, but neither the paper nor Additional file 1 reports their results. No reported model makes PPD significant for CVD mortality as a main effect.
- One-sided testing or the 0.10 threshold. This is not the explanation. The Methods set 0.05 as the cut-off for main effects (and describe P values as two-sided for interaction terms). If the printed main-effect P values are two-sided, halving them for a one-sided test gives 0.138–0.169, still above 0.05. The 0.10 threshold applies only to interaction terms, and the main-effect P values (0.275–0.337) are above it anyway.
- The sentence refers to interaction terms. Under this reading the sentence could be defended. Table S5 in Additional file 1 flags, by symbol, multiplicative interactions significant at P < 0.10 for CVD mortality (the paper's threshold for interaction terms). At least one is flagged for every PPD measure: mean PPD with fibrinogen, CRP and the inflammation score; sites PPD ≥ 3 mm with fibrinogen; and sites PPD ≥ 4 mm with fibrinogen, CRP and the inflammation score. The placement and wording point to a main-effect reading, though. The sentence follows directly after the list of main-effect HRs per SD increase for all-cause mortality. It speaks of "PPD related variables" having "associations" and names no interaction or inflammation marker. The next sentence, "Interaction modelling revealed some variation with respect to mortality type and exposure combinations", is where the abstract turns to interactions. As printed, the sentence reads as a main-effect claim, which Table 2 and Table S4 contradict. At best it is ambiguous. The suggested correction below removes the ambiguity under either reading.
- The sentence describes another outcome column. I checked this. In Table 2, two of the three PPD measures are significant for all-cause mortality and two of three for non-CVD mortality (mean PPD is not, P = 0.098 and 0.166). In Table S4, all three are significant for all-cause and for non-CVD mortality. So "all PPD related variables showed significant associations" matches the non-CVD and all-cause columns of Table S4, not any column of the main Table 2. The paper does not say which outcome or model the sentence was meant to describe. As printed, it names CVD mortality, and for CVD mortality it is contradicted by both tables.
- Conversion or typesetting artefact. This is ruled out. The publisher's HTML abstract and Table 2 page, the PMC text and the Europe PMC full-text XML all carry the same sentence and the same cell values.
Status
As of 2026-10-23:
- Version of record. The publisher's article page (link.springer.com, which doi.org and bmcmedicine.biomedcentral.com both redirect to) sits behind a JavaScript challenge, so it was loaded in headless Chromium. It shows the same abstract sentence and the same Results sentence. Its Table 2 page has CVD-mortality cells identical to the PMC copy. The Europe PMC full-text XML and the PMC web page agree. The publisher PDF could not be retrieved (both the Springer and the Europe PMC PDF links returned a challenge page). Every quotation above was checked verbatim against the PMC-archived text.
- Supplement. The single supplementary file, Additional file 1 (12916_2023_3139_MOESM1_ESM.docx), was downloaded from the publisher's static-content server and its Table S4 read. Table S4 agrees with Table 2 that no PPD measure is significant for CVD mortality as a main effect (values in the table above), so it does not resolve the discrepancy. Its Table S5 was also read. It prints RERI values for CVD mortality and uses symbols to flag PPD × inflammation multiplicative interactions for CVD mortality as significant at P < 0.10, for each of the three PPD measures with at least one inflammation marker (see "Explanations considered"). The main text, Results, "Interaction between periodontitis and systemic inflammation", gives two such interactions for CVD mortality, "extent PPD and fibrinogen: HR = 1.147, P = 0.006; extent PPD and inflammation score: HR = 1.134, P = 0.042", and says that these HRs "ranged from 1.12 to 1.18". It does not say whether "extent PPD" there means ≥ 3 mm or ≥ 4 mm. These interaction results are the only PPD-related CVD results that the paper reports as significant.
- Corrections. One correction exists: doi:10.1186/s12916-025-03862-0 (BMC Med 2025, PMC11734466, dated 15 January 2025), found through Crossref update metadata and Europe PMC notices. Its full text adds a funding source that was "mistakenly omitted" and says nothing about the abstract or Table 2. The publisher's page still carries the abstract sentence, and the correction notice is shown on it. No retraction or expression of concern was found.
- PubPeer. The PubPeer search page and API refused automated access (a Cloudflare challenge, HTTP 403), including from headless Chromium. A web search for the title together with "PubPeer" turned up no PubPeer thread, only the article, the correction and repository copies. That there is no PubPeer thread is likely but not confirmed.
- Sources. PMC10642059 (article), PMC11734466 (correction), https://link.springer.com/article/10.1186/s12916-023-03139-4 (version of record), Additional file 1 (12916_2023_3139_MOESM1_ESM.docx).
Suggested correction
In the Abstract, Results, replace "For CVD mortality, all PPD related variables showed significant associations." with a sentence consistent with Table 2 and the Results text. For example: "For CVD mortality, none of the PPD-related variables showed a significant association (HRs per SD increase: mean PPD 1.078 (95% CI 0.942–1.233); extent PPD ≥ 3 mm 1.096 (0.921–1.305); extent PPD ≥ 4 mm 1.070 (0.932–1.228)), whereas mean CAL, extent CAL ≥ 4 mm, missing teeth, the periodontitis score and the inflammation markers other than fibrinogen did." The authors may also wish to review three related statements. First, the description of mean PPD (P = 0.098) as showing an "apparent association" with all-cause mortality. Second and third, the two Conclusions that periodontitis was significantly associated with CVD mortality, which hold for mean CAL, extent CAL ≥ 4 mm, missing teeth and the periodontitis score but not for the PPD measures (or for extent CAL ≥ 3 mm, P = 0.074).
Recomputation script (python recompute.py.txt)
"""Recompute the CVD-mortality results for the pocket probing depth (PPD) measures
in PMC10642059 (BMC Med 2023;21:430, doi:10.1186/s12916-023-03139-4).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
fig1.svg/png (= candidates/cand_A) abstract vs Results/Table 2 text panels
fig2.svg/png (= candidates/cand_B) reported vs recomputed 95% CIs, log HR axis
candidates/cand_A..cand_E.svg/png five candidate figure designs
Inputs are copied from the paper as archived in PMC (Table 2) and from
Additional file 1 (12916_2023_3139_MOESM1_ESM.docx, Table S4).
Method. Each 95% CI is assumed symmetric on the log-HR scale (Wald), so
SE = (ln U - ln L) / (2 * 1.959964), z = ln(HR) / SE, P = 2 * (1 - Phi(|z|)).
In reverse, a CI is rebuilt from the HR and the reported two-sided P:
SE = |ln HR| / Phi^-1(1 - P/2), CI = HR * exp(+/- 1.959964 * SE).
"""
import os
import shutil
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z975 = norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (paper)
# Table 2: (per-unit HR, lo, hi), (per-SD HR, lo, hi), reported P -- one P per row
T2 = {
"all-cause": {
"Mean PPD": ((1.097, 0.983, 1.224), (1.068, 0.988, 1.155), 0.098),
"PPD >= 3 mm": ((1.005, 1.000, 1.009), (1.125, 1.015, 1.247), 0.025),
"PPD >= 4 mm": ((1.005, 1.000, 1.010), (1.091, 1.008, 1.181), 0.030),
},
"CVD": {
"Mean PPD": ((1.111, 0.920, 1.341), (1.078, 0.942, 1.233), 0.275),
"PPD >= 3 mm": ((1.004, 0.997, 1.011), (1.096, 0.921, 1.305), 0.302),
"PPD >= 4 mm": ((1.004, 0.996, 1.012), (1.070, 0.932, 1.228), 0.337),
},
"non-CVD": {
"Mean PPD": ((1.100, 0.961, 1.258), (1.070, 0.972, 1.178), 0.166),
"PPD >= 3 mm": ((1.006, 1.001, 1.011), (1.151, 1.014, 1.308), 0.030),
"PPD >= 4 mm": ((1.006, 1.000, 1.012), (1.108, 1.006, 1.220), 0.037),
},
}
# Table 2, CVD mortality, other exposures (per-SD HR, lo, hi, reported P) -- controls
T2_CVD_OTHER = {
"Mean CAL": (1.225, 1.042, 1.439, 0.014),
"CAL >= 3 mm": (1.288, 0.976, 1.702, 0.074),
"CAL >= 4 mm": (1.217, 1.002, 1.479, 0.047),
"Missing teeth": (1.226, 1.025, 1.466, 0.026),
"Periodontitis score": (1.399, 1.076, 1.820, 0.012),
"Leukocytes": (1.316, 1.143, 1.514, None), # printed "< 0.001"
"Fibrinogen": (1.128, 0.981, 1.297, 0.090),
"CRP (log)": (1.236, 1.028, 1.488, 0.024),
"Inflammation score": (1.385, 1.141, 1.681, 0.001),
}
# Additional file 1, Table S4 (time-invariant models), per-SD HR, lo, hi, reported P
S4 = {
"all-cause": {
"Mean PPD": (1.109, 1.017, 1.209, 0.020),
"PPD >= 3 mm": (1.127, 1.011, 1.255, 0.031),
"PPD >= 4 mm": (1.091, 1.003, 1.186, 0.042),
},
"CVD": {
"Mean PPD": (1.080, 0.927, 1.259, 0.321),
"PPD >= 3 mm": (1.057, 0.883, 1.264, 0.547),
"PPD >= 4 mm": (1.057, 0.917, 1.219, 0.444),
},
"non-CVD": {
"Mean PPD": (1.128, 1.017, 1.253, 0.023),
"PPD >= 3 mm": (1.176, 1.028, 1.345, 0.018),
"PPD >= 4 mm": (1.116, 1.008, 1.237, 0.035),
},
}
PPD = ["Mean PPD", "PPD >= 3 mm", "PPD >= 4 mm"]
OUTCOMES = ["all-cause", "CVD", "non-CVD"]
def p_from_ci(hr, lo, hi):
se = (np.log(hi) - np.log(lo)) / (2 * Z975)
z = np.log(hr) / se
return 2 * norm.sf(abs(z)), se, z
def ci_from_p(hr, p):
se = abs(np.log(hr)) / norm.isf(p / 2)
return hr * np.exp(-Z975 * se), hr * np.exp(Z975 * se)
# ------------------------------------------------------------------ 1. CVD PPD rows
print("1. Table 2, CVD mortality, PPD measures (Wald P recomputed from the printed CI)")
print(f" {'measure':12s} {'scale':8s} {'HR (95% CI)':22s} {'SE(lnHR)':>8s} {'z':>6s} "
f"{'P recomp':>8s} {'P printed':>9s} CI includes 1")
REC = {}
for m in PPD:
unit, sd, prep = T2["CVD"][m]
for scale, (hr, lo, hi) in (("per unit", unit), ("per SD", sd)):
p, se, z = p_from_ci(hr, lo, hi)
REC[(m, scale)] = p
print(f" {m:12s} {scale:8s} {hr:.3f} ({lo:.3f}; {hi:.3f}) {se:8.4f} {z:6.2f} "
f"{p:8.3f} {prep:9.3f} {lo <= 1 <= hi}")
n_sig_sd = sum(T2["CVD"][m][2] < 0.05 for m in PPD)
n_sig_rec = sum(REC[(m, s)] < 0.05 for m in PPD for s in ("per unit", "per SD"))
n_ci_excl = sum(not (c[1] <= 1 <= c[2]) for m in PPD for c in T2["CVD"][m][:2])
print(f" PPD measures with printed P < 0.05 for CVD mortality: {n_sig_sd} of 3")
print(f" recomputed P < 0.05: {n_sig_rec} of 6 (both scalings); CIs excluding 1: {n_ci_excl} of 6")
print(" per-SD P range printed: %.3f-%.3f; recomputed: %.3f-%.3f" % (
min(T2["CVD"][m][2] for m in PPD), max(T2["CVD"][m][2] for m in PPD),
min(REC[(m, "per SD")] for m in PPD), max(REC[(m, "per SD")] for m in PPD)))
print(" one-sided P (printed P halved): " +
", ".join(f"{m} {T2['CVD'][m][2] / 2:.3f}" for m in PPD))
# per-unit rows with CI printed to 3 decimals near 1: how much can rounding move P?
for m in ("PPD >= 3 mm", "PPD >= 4 mm"):
hr, lo, hi = T2["CVD"][m][0]
ps = [p_from_ci(h, l, u)[0] for h in (hr - 5e-4, hr + 5e-4)
for l in (lo - 5e-4, lo + 5e-4) for u in (hi - 5e-4, hi + 5e-4)]
print(f" {m} per unit: P compatible with 3-decimal rounding of HR/CI: "
f"{min(ps):.3f}-{max(ps):.3f} (printed {T2['CVD'][m][2]:.3f})")
# ------------------------------------------------------------------ 2. CI rebuilt from P
print("\n2. 95% CI rebuilt from the per-SD HR and the printed P (CVD mortality)")
REBUILT = {}
for m in PPD:
hr, lo, hi = T2["CVD"][m][1]
rl, rh = ci_from_p(hr, T2["CVD"][m][2])
REBUILT[m] = (rl, rh)
print(f" {m:12s} printed ({lo:.3f}; {hi:.3f}) rebuilt ({rl:.3f}; {rh:.3f})")
# ------------------------------------------------------------------ 3. control rows
print("\n3. Control rows, Table 2 CVD mortality, per SD (table internal consistency)")
for k, (hr, lo, hi, p) in T2_CVD_OTHER.items():
pc = p_from_ci(hr, lo, hi)[0]
ptxt = "< 0.001" if p is None else f"{p:.3f}"
pctxt = "< 0.001" if pc < 0.001 else f"{pc:.3f}"
print(f" {k:20s} {hr:.3f} ({lo:.3f}; {hi:.3f}) P recomp {pctxt} printed {ptxt} "
f"{'sig' if (p is None or p < 0.05) else 'n.s.'}")
# ------------------------------------------------------------------ 4. other outcomes
print("\n4. Same PPD rows for the other outcomes (Table 2 per SD and Table S4 per SD)")
SIGCOUNT = {}
for src, data in (("Table 2", {o: {m: (*T2[o][m][1], T2[o][m][2]) for m in PPD} for o in OUTCOMES}),
("Table S4", S4)):
for o in OUTCOMES:
cells = []
n = 0
for m in PPD:
hr, lo, hi, p = data[o][m]
pc = p_from_ci(hr, lo, hi)[0]
n += p < 0.05
cells.append(f"{m} P {p:.3f} (recomp {pc:.3f})")
SIGCOUNT[(src, o)] = n
print(f" {src:8s} {o:9s} {n} of 3 significant: " + "; ".join(cells))
print(" all-cause mean PPD, Table 2 per SD: 1.068 (0.988; 1.155), P recomp "
f"{p_from_ci(1.068, 0.988, 1.155)[0]:.3f} (printed 0.098)")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
LW = 2 * 72 / 100 # 2 px at 100 px/in in points
MS = 8 # marker size in points (>= 8 px)
def save(fig, name, also=None):
for ext in ("svg", "png"):
p = os.path.join(CAND, f"{name}.{ext}")
fig.savefig(p, dpi=200, bbox_inches="tight", pad_inches=0.12)
if also:
shutil.copyfile(p, os.path.join(HERE, f"{also}.{ext}"))
plt.close(fig)
def box(ax, x, y, w, h, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc="white", ec=edge, lw=1.2, transform=ax.transAxes))
LBL = {"Mean PPD": "Mean PPD", "PPD >= 3 mm": "Sites PPD ≥ 3 mm",
"PPD >= 4 mm": "Sites PPD ≥ 4 mm"}
# ---- A: two text panels joined by an annotated connector -----------------------
fig = plt.figure(figsize=(7.2, 4.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
box(ax, 0.01, 0.36, 0.33, 0.60, INK2)
box(ax, 0.47, 0.02, 0.52, 0.94, INK2)
ax.text(0.03, 0.92, "Abstract, Results", fontsize=11, weight="bold", va="top")
ax.text(0.03, 0.83, "“For CVD mortality, all\nPPD related variables\nshowed significant\nassociations.”",
fontsize=10.5, va="top", style="italic", linespacing=1.4)
ax.text(0.03, 0.52, "Implies: 3 of 3 PPD\nmeasures significant\nfor CVD mortality",
fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(0.49, 0.92, "Results text and Table 2", fontsize=11, weight="bold", va="top")
ax.text(0.49, 0.85, "“For CVD mortality, neither mean PPD nor\nextent PPD ≥ 3 mm/4 mm showed a relevant\n"
"association (Table 2).”", fontsize=10, va="top", style="italic", linespacing=1.4)
yh = 0.585
ax.text(0.49, yh, "CVD mortality,\nHR per SD (95% CI)", fontsize=10, color=INK2, va="center", linespacing=1.2)
ax.plot([0.775], [yh], "o", ms=MS, color=ORANGE)
ax.text(0.79, yh, "P\nprinted", fontsize=10, color=INK2, va="center", linespacing=1.1)
ax.plot([0.885], [yh], "s", ms=MS, color=BLUE)
ax.text(0.90, yh, "P\nrecomp.", fontsize=10, color=INK2, va="center", linespacing=1.1)
ax.plot([0.49, 0.975], [yh - 0.055, yh - 0.055], color=GRID, lw=1)
for i, m in enumerate(PPD):
y = yh - 0.13 - i * 0.115
hr, lo, hi = T2["CVD"][m][1]
ax.text(0.49, y + 0.025, LBL[m], fontsize=10, va="center")
ax.text(0.49, y - 0.028, f"{hr:.3f} ({lo:.3f}–{hi:.3f})", fontsize=10, va="center", color=INK2)
ax.text(0.79, y, f"{T2['CVD'][m][2]:.3f}", fontsize=10, va="center")
ax.text(0.90, y, f"{REC[(m, 'per SD')]:.3f}", fontsize=10, va="center")
ax.text(0.49, 0.055, "Every CI includes 1; every P > 0.05\n(the paper's cut-off for main effects).",
fontsize=10, color=INK2, va="bottom", linespacing=1.3)
ax.add_patch(FancyArrowPatch((0.345, 0.72), (0.465, 0.72), arrowstyle="<->", mutation_scale=14,
lw=LW, color=INK))
ax.text(0.405, 0.69, "Conflict", ha="center", va="top", fontsize=10.5, weight="bold")
ax.text(0.405, 0.63, "Abstract:\n3 of 3\n\nTable 2:\n0 of 3", ha="center", va="top", fontsize=10,
linespacing=1.3)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="printed in the paper"),
Line2D([], [], marker="s", ls="", ms=MS, color=BLUE,
label="recomputed from the\nprinted 95% CI (Wald,\nlog scale)")],
loc="lower left", bbox_to_anchor=(0.01, 0.03), frameon=False, fontsize=10, handletextpad=0.4)
save(fig, "cand_A", also="fig1")
# ---- B: reported vs rebuilt CI on one log axis, with control rows ---------------
rows = [(LBL[m], *T2["CVD"][m][1], REBUILT[m]) for m in PPD]
ctrl = []
for k in ("Mean CAL", "Missing teeth", "Periodontitis score"):
hr, lo, hi, p = T2_CVD_OTHER[k]
ctrl.append((k + " (control)", hr, lo, hi, ci_from_p(hr, p)))
allrows = rows + ctrl
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.array([6.5, 5.5, 4.5, 2.5, 1.5, 0.5])
for y, (lab, hr, lo, hi, (rl, rh)) in zip(ys, allrows):
ax.plot([lo, hi], [y + 0.15] * 2, color=ORANGE, lw=LW, solid_capstyle="round")
ax.plot(hr, y + 0.15, "o", ms=MS, color=ORANGE, mec=BG, mew=1)
ax.plot([rl, rh], [y - 0.15] * 2, color=BLUE, lw=LW, solid_capstyle="round")
ax.plot(hr, y - 0.15, "s", ms=MS - 1, color=BLUE, mec=BG, mew=1)
ax.axvline(1, color=INK, lw=1, zorder=1.5)
ax.set_xscale("log")
ticks = [0.9, 1.0, 1.2, 1.4, 1.6, 1.8]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:.1f}" for t in ticks]); ax.minorticks_off()
ax.set_xlim(0.86, 2.2)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in allrows], fontsize=10)
for t in ax.get_yticklabels()[3:]:
t.set_color(INK2)
ax.set_ylim(-0.1, 7.6)
ax.axhline(3.5, color=GRID, lw=1)
ax.text(0.87, 7.25, "PPD measures (abstract: “significant”)", fontsize=10, weight="bold", va="center",
bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
ax.text(2.18, 7.25, "printed P", ha="right", fontsize=10, color=INK2, va="center")
ax.text(0.87, 3.15, "Other periodontal measures, same table", fontsize=10, color=INK2, va="center",
bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
for y, m in zip(ys[:3], PPD):
ax.text(2.18, y, f"{T2['CVD'][m][2]:.3f}", ha="right", va="center", fontsize=10, color=INK2)
for y, k in zip(ys[3:], ("Mean CAL", "Missing teeth", "Periodontitis score")):
ax.text(2.18, y, f"{T2_CVD_OTHER[k][3]:.3f}", ha="right", va="center", fontsize=10, color=INK2)
ax.set_xlabel("Hazard ratio per SD increase, CVD mortality (log scale; line at HR = 1)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="95% CI printed in Table 2"),
Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS - 1,
label="95% CI rebuilt from HR and printed P")],
loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_B", also="fig2")
# ---- C: significance grid, PPD x outcome x table --------------------------------
fig, ax = plt.subplots(figsize=(7.2, 3.4))
xs = {("Table 2", o): i for i, o in enumerate(OUTCOMES)}
xs.update({("Table S4", o): i + 3.6 for i, o in enumerate(OUTCOMES)})
for (src, o), x in xs.items():
if o == "CVD":
ax.add_patch(plt.Rectangle((x - 0.45, -0.5), 0.9, 3.0, fc="#f1f0ec", ec="none", zorder=0))
for j, m in enumerate(PPD):
y = 2 - j
if src == "Table 2":
hr, lo, hi = T2[o][m][1]; p = T2[o][m][2]
else:
hr, lo, hi, p = S4[o][m]
pc = p_from_ci(hr, lo, hi)[0]
filled = p < 0.05
ax.plot(x - 0.17, y + 0.12, "o", ms=11, mfc=ORANGE if filled else BG, mec=ORANGE, mew=2)
ax.plot(x + 0.17, y + 0.12, "s", ms=10, mfc=BLUE if pc < 0.05 else BG, mec=BLUE, mew=2)
ax.text(x, y - 0.25, f"{p:.3f}", ha="center", va="center", fontsize=10,
color=INK if filled else INK2)
ax.set_yticks([2, 1, 0]); ax.set_yticklabels([LBL[m] for m in PPD])
ax.set_xticks(list(xs.values())); ax.set_xticklabels([o for (_, o) in xs], fontsize=10)
for (src, x0) in (("Table 2 (time-varying, main)", 1), ("Table S4 (time-invariant)", 4.6)):
ax.text(x0, 3.55, src, ha="center", fontsize=10.5, weight="bold")
ax.text(1, -0.8, "CVD column: 0 of 3 significant", ha="center", fontsize=10, weight="bold")
ax.text(4.6, -0.8, "CVD column: 0 of 3 significant", ha="center", fontsize=10, weight="bold")
ax.set_xlim(-0.6, 6.2); ax.set_ylim(-1.0, 3.1)
ax.xaxis.tick_top()
for s in ax.spines.values():
s.set_visible(False)
ax.tick_params(length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=10, mfc=ORANGE, mec=ORANGE, label="printed P < 0.05"),
Line2D([], [], marker="o", ls="", ms=10, mfc=BG, mec=ORANGE, mew=2, label="printed P ≥ 0.05"),
Line2D([], [], marker="s", ls="", ms=9, mfc=BLUE, mec=BLUE, label="recomputed P < 0.05"),
Line2D([], [], marker="s", ls="", ms=9, mfc=BG, mec=BLUE, mew=2, label="recomputed P ≥ 0.05")],
loc="upper center", bbox_to_anchor=(0.5, -0.12), ncol=4, frameon=False, fontsize=9.5,
handletextpad=0.3, columnspacing=1.0,
title="Number under each pair: printed P (HR per SD)", title_fontsize=10)
save(fig, "cand_C")
# ---- D: P values on one log axis with the 0.05 line -----------------------------
fig, ax = plt.subplots(figsize=(7.2, 3.6))
items = [(f"{LBL[m]} (Table 2)", T2["CVD"][m][2], REC[(m, 'per SD')]) for m in PPD]
items += [(f"{LBL[m]} (Table S4)", S4["CVD"][m][3], p_from_ci(*S4["CVD"][m][:3])[0]) for m in PPD]
items += [(f"{k} (Table 2, control)", T2_CVD_OTHER[k][3], p_from_ci(*T2_CVD_OTHER[k][:3])[0])
for k in ("Mean CAL", "Missing teeth")]
yy = np.arange(len(items))[::-1].astype(float)
yy[:6] += 0.5
for y, (lab, p, pc) in zip(yy, items):
ax.plot([min(p, pc), max(p, pc)], [y, y], color=GRID, lw=LW, zorder=1)
ax.plot(p, y + 0.13, "o", ms=MS, color=ORANGE, mec=BG, mew=1, zorder=3)
ax.plot(pc, y - 0.13, "s", ms=MS - 1, color=BLUE, mec=BG, mew=1, zorder=3)
ax.axvline(0.05, color=INK, lw=1)
ax.text(0.052, yy[-1] - 0.75, "P = 0.05", fontsize=10, va="bottom")
ax.axvspan(0.005, 0.05, color="#f1f0ec", zorder=0)
ax.text(0.0065, yy[0] + 0.15, "significant\nat α = 0.05", fontsize=10, color=INK2, va="top")
ax.set_xscale("log"); ax.set_xlim(0.005, 1)
ax.set_xticks([0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1]); ax.set_xticklabels(["0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.minorticks_off()
ax.set_yticks(yy); ax.set_yticklabels([i[0] for i in items], fontsize=10)
ax.set_ylim(yy[-1] - 0.8, yy[0] + 0.6)
ax.set_xlabel("P value, CVD mortality, HR per SD (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="P printed in the paper"),
Line2D([], [], marker="s", ls="", ms=MS - 1, color=BLUE, label="P recomputed from printed CI")],
loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_D")
# ---- E: count comparison --------------------------------------------------------
fig, ax = plt.subplots(figsize=(7.2, 2.6))
bars = [("Abstract claim", 3, ORANGE, "o"),
("Table 2, printed P", SIGCOUNT[("Table 2", "CVD")], ORANGE, "o"),
("Table 2, P recomputed from CI", sum(REC[(m, 'per SD')] < 0.05 for m in PPD), BLUE, "s"),
("Table S4, printed P", SIGCOUNT[("Table S4", "CVD")], ORANGE, "o"),
("Table S4, P recomputed from CI", sum(p_from_ci(*S4["CVD"][m][:3])[0] < 0.05 for m in PPD), BLUE, "s")]
yb = np.arange(len(bars))[::-1]
for y, (lab, n, c, mk) in zip(yb, bars):
ax.plot([0, n], [y, y], color=c, lw=LW, solid_capstyle="round")
ax.plot(n, y, mk, ms=MS + 1, color=c, mec=BG, mew=1)
ax.text(n + 0.08, y, f"{n} of 3", va="center", fontsize=10)
ax.set_yticks(yb); ax.set_yticklabels([b[0] for b in bars], fontsize=10)
ax.set_xlim(-0.1, 3.5); ax.set_xticks([0, 1, 2, 3])
ax.set_xlabel("PPD measures significant for CVD mortality (P < 0.05), out of 3")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="stated or printed in the paper"),
Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS, label="recomputed from the paper's CIs")],
loc="upper center", bbox_to_anchor=(0.35, -0.3), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_E")
print("\nwrote fig1/fig2 and candidates/cand_A..cand_E (svg, png) in", HERE)
All reported HPV prevalence confidence intervals are about 20 to 140 times narrower than the counts allow
BMC Med (2025) · doi:10.1186/s12916-025-04246-0 · PMC12211446
Summary
The Abstract, Results and Table 1 report a cervicovaginal HPV prevalence of 6.2% (51 of 822 women) with a 95% confidence interval of 6.18%–6.22%, and give intervals of ±0.01 to ±0.02 percentage points for every genotype-specific prevalence. From the paper's own counts, the exact binomial 95% interval for 51/822 is 4.65%–8.08%, 86 times as wide as the reported one, and the genotype intervals are 22 to 143 times as wide as reported; the point estimates themselves are correct. Four of the six reported intervals are reproduced to the printed decimal by one arithmetic step, a 95% half-width computed on the proportion scale (0.0165 for 51/822) added to a prevalence expressed in percent. Prevalence is the study's primary outcome, so its stated precision is overstated about 80-fold, which bears on the Discussion's comparison with 2009 estimates (the 2009 HPV16/18 value of 1.2% lies inside the corrected interval) and on any meta-analysis that uses the published interval.
What the paper reports
The prevalence of HPV infection was 6.2% (95% CI: 6.18%-6.22%), while the prevalence of high-risk genotypes 16 and 18 was 1.94% (95% CI: 1.93%-1.95%). The prevalence of 12 pooled high-risk HPV infection was 4.14% (95% CI:4.13%-4.15%).
Abstract, Results
The prevalence of cervicovaginal HPV infection among ever-married women of the 35-year-old age cohort was 6.2% (95% CI: 6.18%-6.22%), as 51, had tested positive for HPV infection out of 822 participants. The prevalence of high-risk carcinogenic HPV genotypes 16(n = 14) and 18(n = 2) was 1.94% (95% CI: 1.93%-1.95%), while the other 12 pooled high-risk HPV genotypes (n = 34) was 4.14% (95% CI: 4.13%-4.15%) (Table 1).
Results, paragraph 2
Table 1 ("Prevalence of cervico-vaginal HPV infection in the District") has a column headed "95% CI for the prevalence %". Its rows are: 12 pooled positive, 34, 4.14, 4.13-4.15; 16 positive, 14, 1.7, 1.69-1.71; 18 positive, 2, 0.24, 0.23-0.25; 16 & 12 pooled positive, 01, 0.12, 0.1-0.13; Negative, 771, 93.8; Total, 822.
On the design and analysis, the paper says:
In the district, there are 413 PHM areas.
Methods, paragraph 3 (each Public Health Midwife area is a cluster, and three women were selected from each)
the degree of accuracy desired specified was 0.016 (3.3/100 × 2). Therefore we needed 479 women.
Methods, paragraph 2 (the required sample size was then "multiplied by the design effect, which was taken as 1.1")
Data were entered by using SPSS version 20.
Methods, paragraph 8
Moreover, high-risk genotypes 16 and 18 prevalence were elevated from 1.2% to 1.94% within the same duration.
Discussion, paragraph 2
What the paper's own numbers show
Method. Each prevalence is a count out of n = 822. For each count I computed the 95% interval three ways: the Clopper–Pearson exact interval, the Wilson interval, and the Wald interval p ± 1.96·√(p(1−p)/n). I also computed the Wald interval with its variance inflated by the paper's stated design effect of 1.1. I then tested one candidate arithmetic route: take 1.96·SE on the proportion scale and add it, unconverted, to the prevalence in percent. All numbers come from recompute.py.txt.
The counts are internally consistent: 34 + 14 + 2 + 1 = 51 positives, 51 + 771 = 822, and 14 + 2 = 16 for HPV16 or 18. Every reported point estimate equals its count divided by 822 (for 16/822 = 1.946% the paper prints 1.94%, truncated rather than rounded, which does not matter here).
| Quantity | Location | Reported | Recomputed from the paper's counts |
|---|---|---|---|
| Any HPV, 51/822 | Abstract; Results p2 | 6.2% (95% CI 6.18–6.22) | 6.204%; exact 4.65–8.08; Wilson 4.75–8.07; Wald 4.56–7.85; Wald with design effect 1.1 4.47–7.93 |
| HPV16 or 18, 16/822 | Abstract; Results p2 | 1.94% (1.93–1.95) | 1.946%; exact 1.12–3.14; Wilson 1.20–3.14 |
| 12 pooled high-risk types, 34/822 | Abstract; Results p2; Table 1 | 4.14% (4.13–4.15) | 4.136%; exact 2.88–5.73; Wilson 2.97–5.72 |
| HPV16, 14/822 | Table 1 | 1.7% (1.69–1.71) | 1.703%; exact 0.93–2.84; Wilson 1.02–2.84 |
| HPV18, 2/822 | Table 1 | 0.24% (0.23–0.25) | 0.243%; exact 0.03–0.88; Wilson 0.07–0.88 |
| HPV16 & 12 pooled, 1/822 | Table 1 | 0.12% (0.1–0.13) | 0.122%; exact 0.00–0.68; Wilson 0.02–0.69 |
| Half-width of the overall interval | derived | 0.02 points | 1.65 points (Wald), 82 times the reported half-width; the exact interval is 86 times as wide as the reported one |
| How much too narrow, genotype rows (exact width ÷ reported width) | derived | — | HPV16 or 18: 101×; 12 pooled: 143×; HPV16: 95×; HPV18: 42×; HPV16 & 12 pooled: 22× |
| Sample size a ±0.02-point interval would need at 6.2% | derived | — | about 5.59 million |
| Population size at which a finite-population correction would give ±0.02 points | derived | — | 822.1, i.e. the sample itself |
| Percent prevalence ± 1.96·SE in proportion units, centred on the reported rounded prevalence | derived | 6.18–6.22; 1.93–1.95; 4.13–4.15; 1.69–1.71; 0.23–0.25; 0.1–0.13 | 6.18–6.22; 1.93–1.95; 4.13–4.15; 1.69–1.71 (all four exact matches); 0.24–0.24; 0.12–0.12 |
| Sample-size calculation | Methods p2 | 479; 915 | 1.96²·0.033·0.967/0.016² = 478.9; 479·1.1/(1−0.424) = 914.8 (consistent) |
| Planned precision vs achieved | Methods p2 (derived) | ±1.6 points (d = 0.016) at an assumed 3.3% | The 95% half-width at the observed 6.2% with n = 822 is 1.65 points, close to the planned ±1.6 points and about 80 times the reported 0.02 |
The paper's own sample-size calculation targeted a precision of ±1.6 percentage points (d = 0.016). The correct half-width at the observed prevalence, 1.65 points, is close to the planned ±1.6 points (the plan assumed 3.3% and n = 479, so the two need not coincide). The reported interval is roughly 80 times narrower than the precision the study was designed to achieve.
The arithmetic route above reproduces four of the six reported intervals exactly: 6.18–6.22, 1.93–1.95, 4.13–4.15 and 1.69–1.71. For the two smallest counts (2 and 1 positives), that route gives half-widths of 0.0034 and 0.0024 points. The paper reports 0.23–0.25 and 0.1–0.13, half-widths of 0.010 and 0.015 points, about 3 and 6 times what that route gives, so the route does not reproduce these two rows. They are still 42 and 22 times narrower than the exact intervals.
(An aside that does not affect the finding: the parenthetical "3.3/100 × 2" evaluates to 0.066, not 0.016. The stated 479 follows from d = 0.016, which is close to 3.3/100 ÷ 2 = 0.0165.)
Figure
Figure 1. Each row is one prevalence reported in the Abstract, the Results or Table 1, with its count out of 822. Orange diamonds show the paper's reported estimate and 95% CI; every reported interval is narrower than its marker, so its bar is hidden. Blue circles with capped bars show the exact (Clopper–Pearson) 95% CI computed from the same count. The labels to the right give each interval, marked "exact" or "reported". The dashed gray lines mark the 2009 values that the Discussion compares against (3.3%, 2.1% and 1.2%). The 1.2% value for HPV16 or 18 lies inside the exact interval for this study. The axis starts at zero and is linear.
Figure 2. Arithmetic for the overall interval. Start from 51/822, compute the standard error and the 95% half-width (0.0165 on the proportion scale), then express the interval in percent. Multiplying the half-width by 100 gives 6.20% ± 1.65 points (blue, 4.56%–7.85% by the Wald method). Adding 0.0165 unconverted to the printed 6.2% gives 6.1835%–6.2165%, which rounds to 6.18%–6.22% (orange), the reported interval to the second decimal place. The same step reproduces the reported intervals for HPV16 or 18, the 12 pooled types and HPV16 (Table above).
Why it matters
- Primary outcome. The study's stated objective is to determine prevalence. Its headline interval tells readers the prevalence is known to within ±0.02 percentage points. The data support only about ±1.7 points (exact 4.65%–8.08%).
- Comparisons in the Discussion. The Discussion says HPV16/18 prevalence "were elevated from 1.2% to 1.94%" since 2009. With the reported interval (1.93–1.95%), 1.2% looks far outside the estimate. With the exact interval (1.12–3.14%), 1.2% lies inside it, and a two-sided exact binomial test of 16/822 against a fixed 1.2% gives p = 0.054. That test also ignores the 2009 study's own sampling error and the differences in district and age range. The comparisons for overall prevalence (3.3% vs 6.2%, p = 3.3 × 10⁻⁵ against a fixed 3.3%) and for the other high-risk types (2.1% vs 4.14%, p = 0.0003) still point the same way once corrected. Those tests too treat the 2009 values as known exactly.
- The same intervals in an earlier article. An article from the same survey (BMC Public Health 2021, doi:10.1186/s12889-021-10190-4, PMC7805031) prints the same overall interval, "6.2% (95%CI: 6.18–6.22%)", in its Abstract and Results, and the same genotype intervals in its Table 4 (4.13–4.15, 1.69–1.71, 0.23–0.25, 0.1–0.13). Crossref, PMC and Europe PMC list no correction for it either. A correction to one article would leave the other unchanged.
- Use by others. In an inverse-variance meta-analysis, the reported interval implies a variance about 82.5² ≈ 6,800 times too small (the Wald half-width is 82.5 times the reported one). A pooled estimate that used it would be dominated by this one study.
- What is not affected. The point estimates, the counts, and the odds ratios from the logistic regression do not depend on the prevalence intervals. The abstract's conclusions are about risk factors and stand as they are on this point.
Explanations considered
- A different, valid interval method (Wilson, exact, Wald). No. All three give half-widths of 1.6–1.7 points for 51/822, and none comes within an order of magnitude of ±0.02.
- Design effect or cluster adjustment. No. The Methods state a design effect of 1.1 for the cluster sample of 413 PHM areas. A design effect of 1 or more widens the interval: 4.47–7.93% with 1.1.
- Finite-population correction. No. To shrink the half-width to 0.02 points, the target population would have to be about 822.1 women, which is the sample itself. The Methods describe selecting three women per cluster from eligible-couple registers across the district, so the sample is not a census.
- Units: proportion instead of percent. Not as a labelling issue. The text, the Abstract and the Table 1 header all state the intervals in percent. If the limits were proportions, 6.18–6.22 would mean 618%–622%. The units issue that does fit is internal to the calculation: a half-width in proportion units (0.0165) added to a prevalence in percent (6.2). This reproduces four of the six reported intervals exactly (Figure 2).
- An interval for a different quantity (such as a mean or a weighted estimate). Nothing in the Methods describes one. The Statistical analysis paragraph names only prevalence estimation, bivariate analysis and logistic regression, and the table column is headed "95% CI for the prevalence %".
- Rounding or transcription. No. A rounding slip could not shrink six independently computed intervals by a factor of 20 to 140.
- Conversion artefact in the archived text. No. The same intervals appear in three places in both the PMC text and the publisher-supplied JATS XML on Europe PMC (Abstract, Results paragraph 2, Table 1), and the abstract intervals appear identically in the PubMed record. The dash character in the ranges differs between sources (the Europe PMC JATS writes the Results range with a minus sign, U+2212, where the Abstract and the PMC text use a hyphen), but the digits and decimal points are identical everywhere.
- Misaligned table cells. No. Each Table 1 row's prevalence equals its own count divided by 822, and each row's interval sits on its own prevalence.
Status
As of 2026-10-23:
- Version of record. The publisher's page (doi.org → bmcmedicine.biomedcentral.com → link.springer.com) redirected to a login, and both publisher PDF links returned HTML rather than a PDF, so the publisher's page could not be read directly. Three other sources built from the publisher's files were checked and agree on every interval they contain: the publisher-supplied JATS full text on Europe PMC, the PMC article, and the PubMed abstract (PMID 40598371).
- Supplements. Both supplementary files were opened: Annexure 1, the questionnaire, and Annexure 2, the specimen-collection guideline. Neither contains analysis output or a description of the interval method. The data are available only from the corresponding author on request.
- Corrections and comments. None found.
src/known.py(Crossrefupdate-to, PMC and Europe PMC notices) returns nothing for this article or for the 2021 BMC Public Health article that carries the same intervals, and the Europe PMC record for this article has no comment or correction list. The PubMed record has no CommentsCorrections entries (erratum, comment or retraction). Web searches for the DOI and title with "correction", "erratum" and "PubPeer" found no notice and no PubPeer thread (rechecked on 2026-10-23); the PubPeer API returned HTTP 403 when queried directly. - Sources. PMC12211446 via
src/show.py(Abstract, Methods paragraphs 2, 3 and 8, Results paragraph 2, Table 1, Discussion paragraph 2);src/known.py 10.1186/s12916-025-04246-0; NCBI E-utilities efetch for PMID 40598371 (abstract text and CommentsCorrections); Europe PMC fullTextXML for PMC7805031 (2021 article);recompute.py.txtin this folder.
Suggested correction
Replace the prevalence intervals with intervals computed from the counts. Exact (Clopper–Pearson) 95% CIs, which a design effect of 1.1 would widen slightly, are:
- Any HPV: 6.2% (95% CI 4.65%–8.08%)
- HPV16 or 18: 1.94% (1.12%–3.14%)
- 12 pooled high-risk types: 4.14% (2.88%–5.73%)
- HPV16: 1.70% (0.93%–2.84%)
- HPV18: 0.24% (0.03%–0.88%)
- HPV16 and 12 pooled: 0.12% (0.00%–0.68%)
Apply these in the Abstract, Results paragraph 2 and Table 1. If the authors prefer a design-based interval, the Wald interval with the stated design effect for the overall prevalence is 4.47%–7.93%. The Discussion's statement that HPV16/18 prevalence rose from 1.2% to 1.94% should note that 1.2% lies within the corrected interval for this study. The same intervals in the Abstract, Results and Table 4 of the 2021 BMC Public Health article from this survey (doi:10.1186/s12889-021-10190-4) would need the same correction.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC12211446 (BMC Med 2025, doi:10.1186/s12916-025-04246-0)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2 (SVG + PNG)
# in the folder that holds this script. Inputs are the counts printed in the paper
# (Results paragraph 2, Table 1); nothing else is assumed.
# Paragraph numbers count from 1 (show.py's [sec:X#pK] tags are 0-based, so #p1 = paragraph 2).
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
N = 822 # participants tested (Results p2, Table 1 total)
Z = stats.norm.ppf(0.975) # 1.95996
DEFF = 1.1 # design effect stated in Methods p2
# label, count, reported prevalence (%), reported 95% CI (%), where reported
ROWS = [
("Any HPV", 51, 6.2, (6.18, 6.22), "Abstract; Results p2"),
("HPV16 or 18", 16, 1.94, (1.93, 1.95), "Abstract; Results p2"),
("12 pooled hrHPV", 34, 4.14, (4.13, 4.15), "Abstract; Results p2; Table 1"),
("HPV16", 14, 1.7, (1.69, 1.71), "Table 1"),
("HPV18", 2, 0.24, (0.23, 0.25), "Table 1"),
("HPV16 & 12 pooled", 1, 0.12, (0.10, 0.13), "Table 1"),
]
def wald(x, n, deff=1.0):
p = x / n
se = np.sqrt(deff * p * (1 - p) / n)
return 100 * (p - Z * se), 100 * (p + Z * se)
def wilson(x, n):
p = x / n
den = 1 + Z**2 / n
c = (p + Z**2 / (2 * n)) / den
h = Z * np.sqrt(p * (1 - p) / n + Z**2 / (4 * n * n)) / den
return 100 * (c - h), 100 * (c + h)
def clopper_pearson(x, n):
lo = stats.beta.ppf(0.025, x, n - x + 1) if x > 0 else 0.0
hi = stats.beta.ppf(0.975, x + 1, n - x) if x < n else 1.0
return 100 * lo, 100 * hi
def unit_slip(x, n):
"""Percent prevalence +/- 1.96*SE where the SE is left on the proportion scale."""
p = x / n
hw = Z * np.sqrt(p * (1 - p) / n) # proportion units
return 100 * p - hw, 100 * p + hw, hw
print("=== 1. Counts")
print(f"34 + 14 + 2 + 1 = {34 + 14 + 2 + 1} positives; 51 + 771 = {51 + 771} total")
print(f"HPV16 or 18 = 14 + 2 = {14 + 2}")
print("\n=== 2. Point estimates and 95% CIs (percent), n = 822")
res = {}
for lab, x, prep, (lo, hi), loc in ROWS:
p = 100 * x / N
w, wd, wi, cp = wald(x, N), wald(x, N, DEFF), wilson(x, N), clopper_pearson(x, N)
s_lo, s_hi, s_hw = unit_slip(x, N)
rep_hw = (hi - lo) / 2
wald_hw = (w[1] - w[0]) / 2
res[lab] = dict(x=x, p=p, rep=(lo, hi), cp=cp, wil=wi, slip=(s_lo, s_hi))
print(f"{lab:18s} {x:2d}/822 = {p:.3f}% (reported {prep}) | reported CI {lo:.2f}-{hi:.2f} (half-width {rep_hw:.3f} pp)")
print(f"{'':18s} Wald {w[0]:.2f}-{w[1]:.2f} | Wald x DEFF 1.1 {wd[0]:.2f}-{wd[1]:.2f} | "
f"Wilson {wi[0]:.2f}-{wi[1]:.2f} | Clopper-Pearson {cp[0]:.2f}-{cp[1]:.2f}")
print(f"{'':18s} Wald half-width {wald_hw:.2f} pp = {wald_hw / rep_hw:.0f} x reported half-width")
print(f"{'':18s} percent p +/- 1.96*SE(proportion units, {s_hw:.4f}): {s_lo:.3f}-{s_hi:.3f}"
f" -> rounded {s_lo:.2f}-{s_hi:.2f}")
r_lo, r_hi = prep - s_hw, prep + s_hw
match = (round(r_lo, 2), round(r_hi, 2)) == (lo, hi)
res[lab]["slip_rep"] = (r_lo, r_hi)
print(f"{'':18s} same, centred on the reported (rounded) {prep}: {r_lo:.4f}-{r_hi:.4f}"
f" -> rounded {r_lo:.2f}-{r_hi:.2f}; equals reported CI: {match}")
print(f"{'':18s} reported half-width / route half-width (pp): {rep_hw:.3f} / {s_hw:.4f} = {rep_hw / s_hw:.1f}")
print(f"{'':18s} exact CI width {cp[1] - cp[0]:.2f} pp = {(cp[1] - cp[0]) / (hi - lo):.0f} x reported width")
print("\n=== 3. How the overall interval compares")
p = 51 / N
se = np.sqrt(p * (1 - p) / N)
print(f"SE of 51/822 = {se:.5f} as a proportion = {100 * se:.3f} percentage points")
print(f"95% half-width = {100 * Z * se:.3f} pp; reported half-width 0.02 pp; ratio {100 * Z * se / 0.02:.1f}")
print(f"SE alone / 0.02 = {100 * se / 0.02:.1f}")
cp51 = clopper_pearson(51, N)
print(f"Exact (Clopper-Pearson) half-width = {(cp51[1] - cp51[0]) / 2:.2f} pp")
n_req = p * (1 - p) * (Z / 0.0002) ** 2
print(f"Sample size a +/-0.02 pp Wald interval would need: {n_req:,.0f}")
f = (0.02 / (100 * Z * se)) ** 2 # required finite-population factor (N-n)/(N-1)
Npop = (N - f) / (1 - f)
print(f"Finite-population factor needed: (Npop-n)/(Npop-1) = {f:.5f} -> Npop = {Npop:.1f}")
print(f"Inverse-variance weight implied by reported CI vs Wald CI: {(100 * Z * se / 0.02) ** 2:,.0f} x")
print("\n=== 4. Sample-size calculation in Methods p2")
pp, d = 0.033, 0.016
n0 = 1.96**2 * pp * (1 - pp) / d**2
print(f"1.96^2*0.033*0.967/0.016^2 = {n0:.1f} (reported 479)")
print(f"479*1.1/(1-0.424) = {479 * 1.1 / (1 - 0.424):.1f} (reported 915)")
print(f"Planned 95% half-width at 3.3%: {100 * d:.1f} pp; at observed 6.2%, n=822: {100 * Z * se:.2f} pp")
print(f"(The parenthetical '3.3/100 x 2' evaluates to {3.3 / 100 * 2:.3f}; 0.016 is close to 3.3/100/2 = {3.3 / 100 / 2:.4f})")
print("\n=== 5. Discussion comparisons with 2009 values (one-sample exact binomial, two-sided;")
print(" treats the 2009 figure as fixed, so it ignores that study's own sampling error)")
for lab, x, ref in [("Any HPV vs 3.3%", 51, 0.033), ("HPV16 or 18 vs 1.2%", 16, 0.012),
("12 pooled vs 2.1%", 34, 0.021)]:
t = stats.binomtest(x, N, ref)
cp = clopper_pearson(x, N)
inside = cp[0] <= 100 * ref <= cp[1]
print(f"{lab:22s} {x}/822 = {100 * x / N:.2f}%; exact CI {cp[0]:.2f}-{cp[1]:.2f}; "
f"2009 value inside exact CI: {inside}; p = {t.pvalue:.4f}"
f"{' (< 1e-4: ' + format(t.pvalue, '.1e') + ')' if t.pvalue < 1e-4 else ''}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Fig 1: dot-and-interval, reported vs exact CI, one linear axis in percent
refs2009 = {"Any HPV": 3.3, "HPV16 or 18": 1.2, "12 pooled hrHPV": 2.1}
order = ["Any HPV", "12 pooled hrHPV", "HPV16 or 18", "HPV16", "HPV18", "HPV16 & 12 pooled"]
fig, ax = plt.subplots(figsize=(7.2, 5.2))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
for i, lab in enumerate(order):
r = res[lab]
yb, yo = i - 0.17, i + 0.17
ax.plot(r["cp"], [yb, yb], color=BLUE, lw=2, solid_capstyle="butt")
for xv in r["cp"]:
ax.plot([xv, xv], [yb - 0.09, yb + 0.09], color=BLUE, lw=2)
ax.plot(r["p"], yb, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.plot(r["rep"], [yo, yo], color=ORANGE, lw=2)
ax.plot(r["p"], yo, "D", ms=8, color=ORANGE, mec=BG, mew=1.2, zorder=3)
ax.text(r["cp"][1] + 0.15, yb, f"exact {r['cp'][0]:.2f}–{r['cp'][1]:.2f}", va="center",
fontsize=10, color=INK)
ax.text(r["cp"][1] + 0.15, yo, f"reported {r['rep'][0]:.2f}–{r['rep'][1]:.2f}", va="center",
fontsize=10, color=INK2)
if lab in refs2009:
v = refs2009[lab]
ax.plot([v, v], [i - 0.38, i + 0.38], color=INK2, lw=1, ls=(0, (2, 2)))
ax.text(v, i - 0.42, f"2009: {v}%", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_yticks(range(len(order)))
ax.set_yticklabels([f"{lab}\n({res[lab]['x']}/822)" for lab in order])
ax.invert_yaxis()
ax.set_xlim(-0.15, 11.2)
ax.set_xticks(range(0, 11, 2))
ax.spines["bottom"].set_bounds(0, 10)
ax.set_xlabel("Prevalence, % (95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1 = ax.plot([], [], "-D", color=ORANGE, lw=2, ms=8, label="Reported in the paper (95% CI; bars hidden behind the markers)")[0]
h2 = ax.plot([], [], "-o", color=BLUE, lw=2, ms=8, label="Exact (Clopper–Pearson) 95% CI recomputed from the paper's counts")[0]
h3 = ax.plot([], [], color=INK2, lw=1, ls=(0, (2, 2)), label="2009 value cited in the Discussion")[0]
ax.legend(handles=[h1, h2, h3], loc="upper left", bbox_to_anchor=(-0.05, -0.15), ncol=1,
frameon=False, fontsize=10)
ax.set_title("Reported vs exact 95% intervals for the same counts (n = 822)",
loc="left", fontsize=11, color=INK)
fig.tight_layout()
save(fig, "fig1")
# Fig 2: arithmetic chain for the overall interval; the divergent step highlighted
hw_prop = Z * se
fig = plt.figure(figsize=(7.2, 3.6))
ax = fig.add_axes([0.01, 0.01, 0.98, 0.98])
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 100)
ax.set_ylim(0, 65)
ax.axis("off")
def box(x, y, w, h, title, body, edge=INK2, lw=1.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc=BG, ec=edge, lw=lw))
ax.text(x + w / 2, y + h - 2.6, title, ha="center", va="top", fontsize=10, color=INK2)
ax.text(x + w / 2, y + h / 2 - 2.2, body, ha="center", va="center", fontsize=10.5, color=INK)
def arrow(p0, p1, color=INK2, lw=1.2):
ax.add_patch(FancyArrowPatch(p0, p1, arrowstyle="-|>", mutation_scale=12, color=color, lw=lw))
box(1, 44, 29, 15, "Counts (Results p2)", f"51 positive / 822\np = {p:.5f}")
box(35.5, 44, 29, 15, "SE (proportion)", f"√(p(1−p)/n) = {se:.5f}")
box(70, 44, 29, 15, "Half-width (proportion)", f"1.96 × SE = {hw_prop:.4f}")
arrow((30.6, 51.5), (35, 51.5))
arrow((65.1, 51.5), (69.5, 51.5))
# branch
box(3, 6, 42, 24, "Same units: convert half-width to %",
f"{hw_prop:.4f} × 100 = {100 * hw_prop:.2f} pp\n6.20% ± {100 * hw_prop:.2f} pp\n→ {wald(51, N)[0]:.2f}% to {wald(51, N)[1]:.2f}%",
edge=BLUE, lw=2)
box(55, 6, 42, 24, "Mixed units: proportion added to %",
f"6.2% ± {hw_prop:.4f}\n→ {res['Any HPV']['slip_rep'][0]:.2f}% to {res['Any HPV']['slip_rep'][1]:.2f}%\n(paper: 6.18% to 6.22%)",
edge=ORANGE, lw=2)
arrow((84.5, 43.3), (76, 31), color=ORANGE, lw=2)
arrow((84.5, 43.3), (24, 31), color=BLUE, lw=2)
ax.text(83, 36, "factor of 100\nlost here", ha="left", va="center",
fontsize=10, color=INK)
ax.text(24, 2.0, "Recomputed (blue)", ha="center", fontsize=10, color=INK2)
ax.text(76, 2.0, "Reproduces the reported interval (orange)", ha="center", fontsize=10, color=INK2)
ax.text(1, 62.5, "How the overall 95% CI of 6.18%–6.22% is reproduced", ha="left", va="center",
fontsize=11, color=INK)
save(fig, "fig2")
print("\nFigures written to", HERE)
Significant C. difficile reduction in abstract contradicts Table 4: 4 vs 0 events, P = 1
BMC Med (2025) · doi:10.1186/s12916-025-04130-x · PMC12219783
Summary
This retrospective claims-database study compares flomoxef with cefmetazole for hospitalised urinary tract infection. The Abstract, the Results and the Discussion all state that C. difficile infection was significantly less common with flomoxef, and the Results give P = 0.03 and an incidence of 1.1% in the cefmetazole group. The paper's own Table 4 reports 4 cases among 4796 cefmetazole patients (0.1%) and 0 among 673 flomoxef patients, with P = 1. Recomputation from those counts gives a two-sided Fisher P of 1.00, and no standard test comes close to 0.03. The safety claim in the Abstract is therefore not supported by the paper's data. The text also gives the two kidney-failure P values the other way round from the tables, though that result is significant on every test. The preprint of the same study reported 63 vs 2 cases (P = 0.04) under a broader case definition. This fits the text not having been fully updated when Table 4 was revised.
What the paper reports
Abstract, Results:
Moreover, the flomoxef group demonstrated a significantly lower frequency of adverse events such as C. difficile infection and renal failure.
Results, "Safety outcome" (the whole paragraph):
The incidences of AEs related to the gastrointestinal, renal, and hepatic systems and hypersensitivity reactions were analysed (Table 4). Renal failure was the most reported AE in the cefmetazole group, affecting 7.5% of patients. This was followed by hepatic dysfunction, occurring in 2.3% of the cases. Furthermore, gastrointestinal AEs included C. difficile infection in 1.1% of the patients in the cefmetazole group. In the flomoxef group, renal failure (3.6%) was also the most common adverse event, followed by hepatic dysfunction (2.5%) and other forms of kidney dysfunction (0.9%). C. difficile infection (P = 0.03) and kidney failure (P = 0.04) were significantly less common in the flomoxef group than in the cefmetazole group. In the safety analysis of Dataset 3, similar to the findings in Dataset 1, the cefmetazole group showed a significantly higher incidence of kidney failure compared to the flomoxef group (P < 0.01) (Additional File 6).
Discussion, opening paragraph:
The incidence rates of C. difficile infection and renal failure after therapy were significantly lower in the flomoxef group than in the cefmetazole group.
Table 4, "Safety endpoints of Dataset 1" (selected rows; columns are cefmetazole n = 4796, flomoxef n = 673, P value):
Non-C. difficile diarrhoea [% (N)] | 23 (0.5) | 0 (0.0) | 0.14
C. difficile infection [% (N)] | 4 (0.1) | 0 (0.0) | 1
Kidney failure [% (N)] | 360 (7.5) | 24 (3.6) | < 0.01
Other kidney dysfunction [% (N)] | 54 (1.1) | 6 (0.9) | 0.73
Additional File 6, "Safety Endpoints of Dataset 3" (propensity-matched, cefmetazole n = 580, flomoxef n = 580), as printed in the publisher's file:
Clostridium difficile infection [% (N)] | 3 (0·5) | 0 (0·0) | 1
Kidney failure [% (N)] | 36 (6·2) | 20 (3·4) | 0·04
Methods, "Patient outcomes" and "Statistical analyses":
Safety outcomes were analysed using Dataset 1.
For safety outcomes, the frequencies were calculated and descriptively analysed for gastrointestinal, renal, hepatic, and hypersensitive AEs.
Statistical tests were two-tailed, with a p < 0.05 considered statistically significant, and performed using R version 4.2.1 (R Foundation for Statistical Computing, Vienna, Austria).
Additional File 4 defines C. difficile infection as "A04.7 Enterocolitis due to Clostridium difficile and include the prescription of oral vancomycin or metronidazole", recorded from the index date to 1 month.
What the paper's own numbers show
Method. The Methods do not name the test used for the safety outcomes, so for each 2×2 table I computed the three standard two-sided tests: Fisher's exact test, Pearson's chi-square with Yates' continuity correction (the default of R's chisq.test for 2×2 tables), and Pearson's chi-square without correction. The inputs are the event counts and group sizes printed in Table 4 and Additional File 6. As a check on the method, I first recomputed every P value in both tables. Yates' chi-square reproduces each printed value in Table 4 to the printed precision: 0.138 (printed 0.14), 1.00 (1), 2.5 × 10⁻⁴ (< 0.01), 0.727 (0.73), 0.758 (0.76) and 1.00 (1). So the tables appear to have been produced with R's default test, and they are internally consistent. The one exception is a minor point in Additional File 6, noted below. Every number here is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| C. difficile P, Dataset 1 (4/4796 vs 0/673) | Results text | P = 0.03, "significantly less common" | Fisher 1.00; Yates 1.00; uncorrected chi-square 0.454. One-sided Fisher 0.591 |
| C. difficile P, Dataset 1 | Table 4 | 1 | Agrees (Fisher 1.00, Yates 1.00) |
| C. difficile incidence, cefmetazole | Results text | 1.1% | 4/4796 = 0.083% (Table 4 prints 0.1). 1.1% of 4796 would be 53 cases |
| Row printed as 1.1% in Table 4 | Table 4 | Other kidney dysfunction 54 (1.1) | 54/4796 = 1.13% |
| C. difficile P, Dataset 3 (3/580 vs 0/580) | Additional File 6 | 1 | Fisher 0.249; Yates 0.248; uncorrected 0.083. Not significant on any test, though not 1 |
| Smallest split that could give P < 0.05 | (derived) | — | With 0 flomoxef cases, at least 28 cefmetazole cases (0.58%) are needed for a two-sided Fisher P < 0.05 (P = 0.042) |
| Kidney failure P, Dataset 1 (360/4796 vs 24/673) | Results text | P = 0.04 | Fisher 7.2 × 10⁻⁵; Yates 2.5 × 10⁻⁴; uncorrected 1.8 × 10⁻⁴ |
| Kidney failure P, Dataset 1 | Table 4 | < 0.01 | Agrees |
| Kidney failure P, Dataset 3 (36/580 vs 20/580) | Results text | P < 0.01 | Fisher 0.039; Yates 0.040; uncorrected 0.028 |
| Kidney failure P, Dataset 3 | Additional File 6 | 0.04 | Agrees |
| C. difficile, preprint Table 4 (63/4796 vs 2/673) | medRxiv v1 | 63 (1.3) vs 2 (0.3), P = 0.04 | 1.31% vs 0.30%; Yates 0.037; Fisher 0.021; uncorrected 0.023 |
There are four C. difficile cases in 5469 patients. If risk were equal in the two groups, the expected number in the flomoxef group would be 4 × 673/5469 = 0.49. So the observed split, all four in the larger cefmetazole group, is the single most likely outcome under the null, with probability 0.591. That is why the two-sided exact P is 1. The P = 0.03 in the text cannot be reached from these counts with any of the usual tests, one-sided or two-sided. It also cannot be reached from the Dataset 3 counts, where the best case is the uncorrected chi-square at 0.083.
The two kidney-failure P values in the text are the reverse of the tables. The text's "P = 0.04" for Dataset 1 is the Dataset 3 value (Additional File 6; recomputed 0.040). Its "P < 0.01" for Dataset 3 matches Dataset 1 (Table 4; recomputed 2.5 × 10⁻⁴). Both are below 0.05, so the conclusion on kidney failure is unchanged.
Figure
Figure 1. Reported and recomputed two-sided P values for C. difficile infection and kidney failure. Each outcome and dataset has two rows. The upper row shows the values the paper reports: filled orange circles for the Results text and open orange diamonds for Table 4 or Additional File 6. An arrow marks a value reported only as "< 0.01". The lower row shows the values recomputed from the printed counts: the blue dot is Fisher's exact test, and the blue line spans Fisher's test, Yates-corrected chi-square and uncorrected chi-square. The dashed line marks P = 0.05, the paper's threshold. For C. difficile in Dataset 1, the text's P = 0.03 lies on the significant side of the threshold, while Table 4 and every recomputed test sit at or near P = 1. In Dataset 3 every recomputed value is also above the threshold. For kidney failure, the text's values for the two datasets are the reverse of the tables. The direction of the result is unaffected.
Figure 2. The C. difficile result in the preprint and in the published article. In the medRxiv preprint (v1, 2024-07-17), C. difficile infection was defined by ICD-10 code A04.7 alone. Table 4 reported 63 (1.3%) vs 2 (0.3%) cases, P = 0.04, which Yates' chi-square reproduces (0.037). In the published article the definition also requires oral vancomycin or metronidazole (Additional File 4), and Table 4 reports 4 vs 0 cases, P = 1. The Abstract, Results and Discussion of the published article still describe a significant reduction. The text's 1.1% and P = 0.03 match the C. difficile row of neither version.
Why it matters
The Abstract names a lower frequency of C. difficile infection as one of two safety advantages of flomoxef, and the Discussion repeats it in its summary of findings. C. difficile infection is a clinically important outcome in antibiotic choice. A reader of the Abstract would conclude that the study found evidence of lower risk with flomoxef. The paper's own table shows four events in total, none of them in the flomoxef group. That count gives no evidence either way: the study has essentially no power for this outcome, and the observed split is what equal risk would most often produce. The Discussion also notes that median hospital stay with flomoxef was about half that with cefmetazole, so adverse events had a shorter in-hospital window in which to be recorded in the flomoxef group. That is a further reason for caution with any between-group difference in these counts.
The error does not affect the paper's formal conclusions in the Abstract and main text, which rest on length of stay, 28-day mortality, carbapenem use and the in vitro data. The kidney-failure difference is significant on every test in both datasets, so the second safety claim stands, apart from the swapped P values. The 1.1% figure overstates the cefmetazole incidence about thirteen-fold (1.1% vs 0.083%).
Explanations considered
- A conversion or typesetting error in the archived text. Excluded. The PMC text, the publisher-supplied JATS XML in Europe PMC, and the publisher's article page (see Status) carry the same sentences and the same Table 4 cells. None of the affected values involve minus signs, superscripts or decimal mid-dots, and the table rows line up.
- The text reports a different dataset or a different test from Table 4. This does not account for the numbers. The Methods state that safety was analysed in Dataset 1, which is Table 4. For Dataset 1, Fisher's exact test, Yates' chi-square and uncorrected chi-square give 1.00, 1.00 and 0.454, and one-sided Fisher gives 0.591. For Dataset 3 (Additional File 6) they give 0.249, 0.248 and 0.083. None of these is 0.03 or below 0.05. A one-sided test is also ruled out by the Methods' statement that tests were two-tailed.
- The text reflects an earlier analysis. This fits the evidence best, but it cannot be confirmed from published material. In the medRxiv preprint, C. difficile was defined by A04.7 alone, and Table 4 reported 63 vs 2 cases with P = 0.04. Those counts would support a significant difference (Yates 0.037, Fisher 0.021). The preprint's Results said "C. difficile infection in 1.3% of patients in the cefmetazole group" and "Kidney failure (P=0.04) and C. difficile infection (P<0.01) were significantly less common in the flomoxef group than in the cefmetazole group." The preprint's Abstract made the same C. difficile claim. In the published version the definition was narrowed (A04.7 plus oral vancomycin or metronidazole) and Table 4 was updated to 4 vs 0, P = 1. The significance statements in the Abstract, Results and Discussion stayed. The published text's specific values (1.1%, P = 0.03) match neither the preprint's nor the published table's C. difficile row, so their exact origin is unknown. The 1.1% matches the "other kidney dysfunction" row in both versions of Table 4. In the preprint the text and table also disagreed: the text gave kidney failure P = 0.04 and C. difficile P < 0.01, the reverse of its Table 4 (< 0.01 and 0.04).
- The kidney-failure values are a transposition between datasets. This accounts for those two numbers exactly. The text's Dataset 1 value equals the Additional File 6 value (0.04) and its Dataset 3 value equals the Table 4 value (< 0.01). Recomputation supports the tables in both cases.
- The claim could rest on a model-based estimate not shown in the paper. Not supported. The Methods describe the safety analysis as descriptive frequencies, and no adjusted safety estimate appears anywhere in the article or its supplements. With four events and none in the flomoxef group, a covariate-adjusted regression would face complete separation and could not give a stable estimate.
A minor, separate point: Additional File 6 prints P = 1 for C. difficile in Dataset 3 (3/580 vs 0/580), where Fisher's test and Yates' chi-square both give 0.25. This does not change any conclusion, since neither value is significant.
Status
As of 2026-10-24:
- Version of record. The publisher's article page (link.springer.com/article/10.1186/s12916-025-04130-x, rendered in headless Chromium because plain requests met a bot check) and its Table 4 page carry the same Abstract sentence, Results paragraph and Table 4 cells as the PMC-archived text. The Europe PMC JATS XML (PMC12219783/fullTextXML) also matches. Every quotation from the published article above was checked verbatim against the PMC text with
src/show.py. The exception is Additional File 6, which was read from the publisher's file. - Supplements. There are six additional files (12916_2025_4130_MOESM1–6_ESM.docx), retrieved through the Europe PMC supplementary-files service. Additional File 4 holds the adverse-event definitions and no counts. Additional File 6 holds the Dataset 3 safety table quoted above. The other files cover definitions of UTI, complications and medical history, and the MIC distribution.
- Preprint. medRxiv doi:10.1101/2024.07.16.24310469, version 1 (posted 2024-07-17), is the only version, and medRxiv links it to the published DOI. Its text was read from the medRxiv JATS source XML, its Table 4 from the table image (T4.large.jpg) and its definitions from its supplementary PDF (Supplementary Table 4).
- Corrections and comments. I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, only a "has-preprint" relation, and PMC and Europe PMC list no correction notices. A web search for comments on the article found none. PubPeer refused automated access (HTTP 403, Cloudflare challenge), so it could not be checked directly.
- Sources. PMC article PMC12219783; Europe PMC full-text XML and supplementary files; publisher article and Table 4 pages at link.springer.com; Crossref metadata for 10.1186/s12916-025-04130-x; medRxiv preprint 10.1101/2024.07.16.24310469 (API record, JATS XML, Table 4 image, supplementary PDF).
Suggested correction
- Remove the statement that C. difficile infection was significantly less frequent with flomoxef from the Abstract (Results), the Results "Safety outcome" paragraph and the opening paragraph of the Discussion. Report instead that C. difficile infection occurred in 4 of 4796 cefmetazole patients (0.1%) and none of 673 flomoxef patients (P = 1), too few events to compare the groups.
- Correct the cefmetazole C. difficile incidence in the Results from 1.1% to 0.1% (4/4796).
- Swap the kidney-failure P values in the Results to match the tables: P < 0.01 for Dataset 1 (Table 4) and P = 0.04 for Dataset 3 (Additional File 6).
- Name the test used for the safety comparisons in the Methods (the printed values match Pearson's chi-square with Yates' continuity correction). Check the Dataset 3 C. difficile P value in Additional File 6 (3 vs 0 of 580 gives P ≈ 0.25, not 1).
- Optionally, state in the Methods that the C. difficile case definition requires oral vancomycin or metronidazole in addition to ICD-10 A04.7, so readers comparing with the preprint can see why the counts differ.
Recomputation script (python recompute.py.txt)
"""Recompute the safety-outcome P values of PMC12219783 (BMC Med 2025, doi:10.1186/s12916-025-04130-x)
from the counts the paper itself prints, and regenerate fig1/fig2 in this folder.
Run: python recompute.py.txt
Needs numpy, scipy, matplotlib.
Inputs (all transcribed from the published article or its preprint):
Table 4 (Dataset 1, all patients): cefmetazole n = 4796, flomoxef n = 673
Additional File 6 (Dataset 3, propensity-matched): 580 vs 580
medRxiv preprint v1 (doi:10.1101/2024.07.16.24310469, posted 2024-07-17), Table 4:
C. difficile infection 63 (1.3) vs 2 (0.3), P = 0.04 (same n = 4796 / 673 as the published Dataset 1)
The Methods do not name the test used for the safety outcomes, so three standard
tests for a 2x2 table are computed: Fisher's exact test (two-sided), Pearson chi-square
with Yates continuity correction (R's chisq.test default for 2x2 tables) and Pearson
chi-square without correction.
"""
import os
import numpy as np
from scipy.stats import fisher_exact, chi2_contingency
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
def tests(a, n1, b, n2):
t = np.array([[a, n1 - a], [b, n2 - b]])
pf = fisher_exact(t)[1]
py = chi2_contingency(t, correction=True)[1]
pr = chi2_contingency(t, correction=False)[1]
return pf, py, pr
# ---------------------------------------------------------------- Table 4, Dataset 1
N1, N2 = 4796, 673
table4 = { # outcome: (cefmetazole events, flomoxef events, reported P in Table 4)
"Non-C. difficile diarrhoea": (23, 0, "0.14"),
"C. difficile infection": (4, 0, "1"),
"Kidney failure": (360, 24, "< 0.01"),
"Other kidney dysfunction": (54, 6, "0.73"),
"Liver dysfunction": (108, 17, "0.76"),
"Rash": (2, 0, "1"),
}
print("=== Table 4, Dataset 1 (cefmetazole n = 4796, flomoxef n = 673) ===")
print(f"{'outcome':28s} {'%cmz':>6s} {'%fmx':>6s} {'Fisher':>9s} {'Yates':>9s} {'chi2':>9s} {'Table 4':>8s}")
R1 = {}
for k, (a, b, rep) in table4.items():
pf, py, pr = tests(a, N1, b, N2)
R1[k] = (pf, py, pr)
print(f"{k:28s} {100*a/N1:6.2f} {100*b/N2:6.2f} {pf:9.3g} {py:9.3g} {pr:9.3g} {rep:>8s}")
# ---------------------------------------------------------------- Additional File 6, Dataset 3
M = 580
af6 = {
"Non-C. difficile diarrhoea": (1, 0, "1"),
"C. difficile infection": (3, 0, "1"),
"Kidney failure": (36, 20, "0.04"),
"Other kidney dysfunction": (7, 6, "1"),
"Liver dysfunction": (13, 16, "0.71"),
"Rash": (1, 0, "1"),
}
print("\n=== Additional File 6, Dataset 3 (580 vs 580, propensity-matched) ===")
print(f"{'outcome':28s} {'%cmz':>6s} {'%fmx':>6s} {'Fisher':>9s} {'Yates':>9s} {'chi2':>9s} {'AF6':>8s}")
R3 = {}
for k, (a, b, rep) in af6.items():
pf, py, pr = tests(a, M, b, M)
R3[k] = (pf, py, pr)
print(f"{k:28s} {100*a/M:6.2f} {100*b/M:6.2f} {pf:9.3g} {py:9.3g} {pr:9.3g} {rep:>8s}")
# ---------------------------------------------------------------- C. difficile details
print("\n=== C. difficile, Dataset 1: 4/4796 vs 0/673 ===")
p_one = fisher_exact([[4, N1 - 4], [0, N2]], alternative="greater")[1]
print("one-sided Fisher P (fewer with flomoxef) = %.3f" % p_one)
print("probability all 4 events fall in the cefmetazole group under equal risk = C(4796,4)/C(5469,4) = %.3f" % p_one)
exp_fmx = 4 * N2 / (N1 + N2)
print("expected flomoxef events under equal risk = 4 x 673/5469 = %.2f" % exp_fmx)
print("cefmetazole incidence 4/4796 = %.3f%%" % (100 * 4 / N1))
print("events implied by the text's 1.1%% of 4796 = %.0f" % (0.011 * N1))
print("other kidney dysfunction 54/4796 = %.2f%% (Table 4 prints 1.1)" % (100 * 54 / N1))
print("text 1.1%% vs table 4/4796: ratio %.1f" % (1.1 / (100 * 4 / N1)))
for a in range(1, 200):
if fisher_exact([[a, N1 - a], [0, N2]])[1] < 0.05:
print("smallest cefmetazole count (flomoxef 0) giving two-sided Fisher P < 0.05: %d (%.2f%%), P = %.3f"
% (a, 100 * a / N1, fisher_exact([[a, N1 - a], [0, N2]])[1]))
break
print("Kidney failure D1 Fisher P = %.2g, Yates P = %.2g, uncorrected P = %.2g" % R1["Kidney failure"])
print("Kidney failure D3 Fisher P = %.3f, Yates P = %.3f, uncorrected P = %.3f" % R3["Kidney failure"])
print("C. difficile D3 Fisher P = %.3f, Yates P = %.3f, uncorrected P = %.3f" % R3["C. difficile infection"])
# ---------------------------------------------------------------- preprint
print("\n=== medRxiv preprint v1, Table 4: C. difficile 63/4796 vs 2/673, reported P = 0.04 ===")
pre = tests(63, N1, 2, N2)
print("incidence %.2f%% vs %.2f%%" % (100 * 63 / N1, 100 * 2 / N2))
print("Fisher P = %.3f, Yates P = %.3f, uncorrected P = %.3f" % pre)
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK, "axes.edgecolor": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# fig1: reported vs recomputed P values, one log axis
rows = [
# label, sublabel, (text P, is '<' bound), (table/supplement P, is '<' bound), recomputed triple
("C. difficile, Dataset 1", "4/4796 vs 0/673 (Table 4)", (0.03, False), (1.0, False), R1["C. difficile infection"]),
("C. difficile, Dataset 3", "3/580 vs 0/580 (Additional File 6)", None, (1.0, False), R3["C. difficile infection"]),
("Kidney failure, Dataset 1", "360/4796 vs 24/673 (Table 4)", (0.04, False), (0.01, True), R1["Kidney failure"]),
("Kidney failure, Dataset 3", "36/580 vs 20/580 (Additional File 6)", (0.01, True), (0.04, False), R3["Kidney failure"]),
]
fig, ax = plt.subplots(figsize=(7.2, 4.8))
ax.set_xscale("log")
ax.set_xlim(2e-5, 2.2)
ax.set_ylim(len(rows) - 0.4, -0.75)
ax.axvline(0.05, color=INK2, lw=0.9, ls=(0, (4, 3)), zorder=1)
ax.text(0.05, -0.62, "P = 0.05", ha="center", va="bottom", color=INK2, fontsize=10)
ax.grid(axis="x", which="major", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([f"{r[0]}\n{r[1]}" for r in rows], fontsize=10)
for lbl in ax.get_yticklabels():
lbl.set_linespacing(1.4)
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 0.1, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "", "0.1", "1"])
ax.set_xlabel("Two-sided P value (log scale)")
off = 0.2
def bound_arrow(x, y):
ax.annotate("", xy=(0.0035, y), xytext=(x, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=12), zorder=3)
ax.text(0.0032, y, "< 0.01", ha="right", va="center", color=INK2, fontsize=10)
for i, (lab, sub, ptext, ptab, (pf, py, pr)) in enumerate(rows):
lo, hi = min(pf, py, pr), max(pf, py, pr)
y = i + off
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([lo, hi], [y, y], "|", color=BLUE, ms=9, mew=2, zorder=2)
ax.plot(pf, y, "o", color=BLUE, ms=9, zorder=3, mec=BG, mew=1.2)
y2 = i - off
for val, kind in ((ptab, "tab"), (ptext, "text")):
if val is None:
continue
x, is_bound = val
if is_bound:
bound_arrow(x, y2)
if kind == "tab":
ax.plot(x, y2, "D", color=BG, mec=ORANGE, mew=2, ms=8, zorder=4)
else:
ax.plot(x, y2, "o", color=ORANGE, ms=9, zorder=5, mec=BG, mew=1.2)
legend = [
Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, mec=BG, label="Reported in Results text"),
Line2D([], [], marker="D", ls="", color=BG, mec=ORANGE, mew=2, ms=8, label="Reported in Table 4 / Additional File 6"),
Line2D([], [], marker="o", ls="-", color=BLUE, lw=2, ms=9, mec=BG,
label="Recomputed from the counts: Fisher exact (dot); line spans Fisher, Yates and uncorrected chi-square"),
]
fig.legend(handles=legend, loc="lower left", bbox_to_anchor=(0.01, 0.0), ncol=1, frameon=False,
fontsize=10, handletextpad=0.5)
fig.subplots_adjust(left=0.36, right=0.98, top=0.94, bottom=0.32)
save(fig, "fig1")
# fig2: the C. difficile row in the preprint and the published article
fig = plt.figure(figsize=(7.2, 4.75))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.75)
ax.axis("off")
def panel(x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.06",
fc="#ffffff", ec=edge, lw=1.4))
ax.text(x + 0.14, y + h - 0.16, title, ha="left", va="top", fontsize=10.5, fontweight="bold", color=INK)
yy = y + h - 0.5
for txt, col, sz in lines:
ax.text(x + 0.14, yy, txt, ha="left", va="top", fontsize=sz, color=col, linespacing=1.35)
yy -= 0.24 * (txt.count("\n") + 1) + 0.08
pre_y, pre_f, pre_r = pre[1], pre[0], pre[2]
panel(0.15, 2.35, 3.3, 2.05, "medRxiv preprint v1 (posted 2024-07-17)", [
("Table 4: C. difficile infection", INK2, 10),
("63 (1.3%) vs 2 (0.3%), P = 0.04", ORANGE, 10.5),
("Definition: ICD-10 A04.7 only", INK2, 10),
(f"Recomputed: Yates P = {pre_y:.3f}, Fisher P = {pre_f:.3f}", BLUE, 10),
], GRID)
panel(3.75, 2.35, 3.3, 2.05, "Published article, Table 4 (2025)", [
("Table 4: C. difficile infection", INK2, 10),
("4 (0.1%) vs 0 (0.0%), P = 1", ORANGE, 10.5),
("Definition: A04.7 plus oral vancomycin\nor metronidazole (Additional File 4)", INK2, 10),
(f"Recomputed: Fisher P = {R1['C. difficile infection'][0]:.2f}", BLUE, 10),
], GRID)
panel(0.15, 0.36, 6.9, 1.48, "Published article, Abstract, Results text and Discussion (2025)", [
("“significantly lower frequency of adverse events such as C. difficile infection” (Abstract)", INK, 10),
("“C. difficile infection in 1.1% of the patients in the cefmetazole group”; “C. difficile infection (P = 0.03)”", ORANGE, 10),
("Neither 1.1% nor P = 0.03 matches the C. difficile row of either version of Table 4", INK2, 10),
], ORANGE)
ax.annotate("", xy=(3.73, 3.4), xytext=(3.47, 3.4),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=2, mutation_scale=12))
ax.text(3.6, 4.47, "definition and counts revised", ha="center", va="bottom", fontsize=10, color=INK2)
ax.annotate("", xy=(5.4, 1.86), xytext=(5.4, 2.33),
arrowprops=dict(arrowstyle="<|-|>", color=ORANGE, lw=2, mutation_scale=12))
ax.text(5.28, 2.075, "Conflict: the text still reports a significant difference", ha="right", va="center",
fontsize=10, color=INK)
lx = 0.15
for col, txt in ((ORANGE, "Reported in the paper or preprint"), (BLUE, "Recomputed from the printed counts")):
ax.add_patch(FancyBboxPatch((lx, 0.12), 0.14, 0.14, boxstyle="square,pad=0", fc=col, ec="none"))
ax.text(lx + 0.22, 0.19, txt, ha="left", va="center", fontsize=10, color=INK2)
lx += 3.0
save(fig, "fig2")
print("\nfigures written to", HERE)
Abstract and conclusions call the 57.4% concordant trials discordant; discordance was 42.6%
BMC Med (2025) · doi:10.1186/s12916-025-04328-z · PMC12403310
Summary
The study compares the data sharing plans that 376 clinical trials stated in their registrations with those stated in their publications, and its Abstract (Results and Conclusions) and main Conclusions section say that more than half of the trials were discordant, citing "216/376, 57.4%". By the paper's own Methods definitions and Table 2 (155 Yes/Yes, 13 Yes/No, 147 No/Yes, 61 No/No), those 216 trials (57.4%) are the concordant ones, and 160/376 (42.6%, 95% CI 37.7–47.6%) are discordant, as the Results text, Table 3, Supplementary Table S6 and the Discussion ("over 40% were discordant") also state. Concordance is one of the study's two stated outcomes, so a reader of the abstract alone gets its headline figure the wrong way round: a minority of trials, not a majority, had discordant plans. No test result or odds ratio changes; the correction is to three sentences.
What the paper reports
Abstract, Results:
Over half (216/376, 57.4%) had discordance between registration and publications.
Abstract, Conclusions:
More than half of published trials have data sharing discordance between registration and publication.
Main text, Conclusions section:
More than half of published trials have discordance between data sharing plans that they submit in trial registration versus publications.
The rest of the paper uses the same 216 trials as the concordant group.
Methods, "Study outcomes":
Subsequently, data sharing concordance was assessed: (1) plan to share data in registration and publication (both Yes in registration and publication, i.e., “Yes/Yes”) and (2) no plan to share data in registration and publication (both No in registration and publication, i.e., “No/No”). Discordance between registered and published plans to share data included (1) plan to share data in registration but no plan to share data in the publication (Yes in registration but No in publication, i.e., “Yes/No”) and (2) no plan to share in registration but a plan to share in publication (No in registration but Yes in publication, i.e., “No/Yes”).
Results, "Concordance between registered and published plans to share data":
A total of 376 trials were included for assessing the concordance, among which 216 (57.4%) had concordance between registered and published plans to share data (n = 155 and 61 for Yes/Yes and No/No, respectively; Table 2). For the discordance, there were 13 (3.5%) and 147 (39.1%) trials as Yes/No and No/Yes, respectively.
Table 2 (registration rows × publication columns): Yes/Yes 155 (41.2%), Yes/No 13 (3.5%), No/Yes 147 (39.1%), No/No 61 (16.2%); total 376 (100%).
Table 3 column headers: "Concordance between registered and published plans to share data", with sub-columns "(n = 160)" and "Yes(n = 216)". The archived text has no label in front of the first sub-column; the counts identify it as the discordant group.
Discussion, paragraph 1:
It was found that more than half of trials published in high-profile journals did not plan to share data in registration, and over 40% were discordant between registration and publication plans.
Discussion, paragraph 4:
Despite the ICMJE requirement, we found discordant data sharing plans in over 40% of trials, a finding that requires attention to increase consistency and reliability.
All quotes above were checked verbatim against the PMC-archived article text.
What the paper's own numbers show
Method. I took the four cell counts from Table 2 and applied the Methods definitions: concordant = Yes/Yes + No/No, discordant = Yes/No + No/Yes. I gave each share a Wilson 95% confidence interval. I re-ran the McNemar test on the two discordant cells as a consistency check. I then applied the same definitions to the two sensitivity tables in Additional file 1 (Table S7a, 78 undecided/missing registrations removed; Table S7b, those registrations counted as "Yes"). All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Trials assessed for concordance | Results; Table 2 | 376 | 155 + 13 + 147 + 61 = 376 |
| Concordant (Yes/Yes + No/No) | Results; Table 2; Table 3 "Yes" | 216 (57.4%) "had concordance" | 155 + 61 = 216/376 = 57.4% (95% CI 52.4–62.3%) |
| Discordant (Yes/No + No/Yes) | Results; Table 2; Table 3 | 13 (3.5%) + 147 (39.1%); Table 3 n = 160 | 13 + 147 = 160/376 = 42.6% (95% CI 37.7–47.6%) |
| Share discordant | Abstract, Results | "Over half (216/376, 57.4%) had discordance" | 160/376 = 42.6%; 216/376 is the concordant share |
| Is discordance more than half? | Abstract Conclusions; main Conclusions | "More than half" | No: 160 < 188 (half of 376); the CI upper bound, 47.6%, is below 50% |
| Discordance | Discussion (twice) | "over 40%" | 42.6%, consistent |
| Overstatement in abstract | Abstract vs Table 2 | 57.4% | 57.4 − 42.6 = 14.8 percentage points |
| Concordance, undecided removed | Results; Table S7a | 64.7% | 193/298 = 64.8%; discordant 105/298 = 35.2% (30.0–40.8%) |
| Concordance, undecided as Yes | Results; Table S7b | 65.9% | 248/376 = 66.0%; discordant 128/376 = 34.0% (29.4–39.0%) |
| McNemar test, 13 vs 147 | Results | P < 0.001 | χ² = 112.2 (110.6 with continuity correction), P ≈ 3e-26; exact P ≈ 7e-30 |
| Registered plan to share | Abstract | 44.6% (171/383) | 171/383 = 44.6% |
Every Table 2 cell and margin percentage reproduces exactly. The sensitivity figures differ from the recomputed ones by 0.1 point because the paper sums its printed component percentages (52.0% + 12.7% and 55.8% + 10.1%), two of which are rounded down (38/298 = 12.75%, printed 12.7%; 210/376 = 55.85%, printed 55.8%). This difference does not affect the finding. Only the abstract and conclusion sentences disagree with the data.
Figure
Figure 1. On the left (orange) are the Abstract Results and Conclusions sentences, which call 216/376 (57.4%) of trials discordant; the main Conclusions section repeats this. On the right is the paper's Table 2, redrawn as a 2 × 2 grid (rows: plan to share in the registration; columns: plan to share in the publication). Cells with solid blue borders are concordant by the Methods definition: Yes/Yes 155 + No/No 61 = 216 (57.4%). Cells with dashed borders are discordant: Yes/No 13 + No/Yes 147 = 160 (42.6%). The Results text describes the same 216 trials as having "concordance". The arrow links the abstract's 216 to the Table 2 total it equals, the concordant sum.
Figure 2. Share of trials whose registered and published data sharing plans disagree, on a full 0–100% axis. Orange: the value given in the abstract and conclusions (57.4%, "over half"). Blue: values recomputed from the paper's own counts, with Wilson 95% confidence intervals. The rows are the main analysis (Table 2, 160/376) and the two sensitivity analyses in Additional file 1 (Table S7a, 105/298; Table S7b, 128/376). The thin vertical line marks half of the trials. Every recomputed estimate and confidence interval lies below that line. The only value above it is the reported one, and that figure is the concordant share.
Why it matters
Concordance between registered and published data sharing plans is one of the study's two stated outcomes, and the abstract and conclusions give its central figure the wrong way round. A reader of the abstract alone learns that a majority (57.4%) of trials in six high-profile journals changed their data sharing plans between registration and publication. By the paper's own counts, a majority (57.4%) were consistent and 42.6% were discordant. The 95% interval for discordance, 37.7–47.6%, excludes 50%, so the error is not a matter of sampling uncertainty. The size of the problem is overstated by about 15 percentage points and pushed across the majority threshold. Abstract and conclusion sentences are the parts most likely to be quoted in later literature and in policy discussion of ICMJE data sharing requirements.
The following results are not affected: the 44.6% (171/383) of registrations with a plan to share; the drug-intervention odds ratio (2.71); the COVID-19 odds ratio for concordance (0.59), which was modelled with the correct groups (Table 3, n = 216 vs n = 160); the McNemar test; the content-level discordance among the 155 Yes/Yes trials; and the sensitivity analyses. The broader message, that a substantial share of trials report inconsistent data sharing plans, still holds at 42.6%. The Discussion already states it correctly ("over 40%").
Explanations considered
- A different definition of discordance. The Methods give a single definition (Yes/No + No/Yes), and no other trial-level definition appears anywhere in the paper. This does not explain the numbers.
- A different population or denominator. The abstract's 216/376 uses the same 376 trial publications as Table 2 and the Results. These are the trials analysed after the later-starting registration ID was chosen for each of the 7 pooled publications. The only other denominator, 383 registration IDs, is used just for the 171/383 figure, which is consistent. This does not explain the numbers.
- Content-level discordance. The abstract also reports "significant discordance" in SAPs/protocols, code and IPD. That analysis covers only the 155 Yes/Yes trials. A count of 216 cannot be built from it, and 216 (57.4%) matches the concordant total (155 + 61) exactly. This does not explain the numbers.
- Sensitivity analyses. These report concordance of 64.7% and 65.9%, so discordance is about 35% and 34%. Discordance is below half in every analysis the paper reports. This does not explain the numbers.
- Rounding or presentation. The error is not a rounding or threshold effect: 160 versus 216 out of 376 is a swap of the two complementary groups.
- Text-conversion artifact in PMC. The same wording appears in the PMC page and in the publisher-supplied JATS full text served by Europe PMC. The error also appears in three separately written sentences in two sections. An archiving artifact is therefore unlikely.
- Correct elsewhere in the paper. The Methods, Results text, Table 2, Table 3, Supplementary Table S6 ("Concordance (n =216)" and "Discordant groups (n =160)") and the Discussion ("over 40% were discordant") all treat 216 as concordant and 160 as discordant. The conflict is confined to the abstract and the two conclusion statements, which is consistent with a wording error in those sentences.
Status
As of 2026-10-23:
- Version of record. The publisher's HTML and PDF could not be opened from this machine: doi.org redirects through bmcmedicine.biomedcentral.com to link.springer.com, which requires a login. The text checked was the PMC-archived article (pmc.ncbi.nlm.nih.gov/articles/PMC12403310/) and the publisher-supplied JATS full text served by Europe PMC. Both carry the three sentences quoted above, and Table 2 is identical in both.
- Supplements. Additional file 1 (12916_2025_4328_MOESM1_ESM.docx, from static-content.springer.com) was read. Table S6 labels the groups "Concordance (n =216)" and "Discordant groups (n =160)". Tables S7a and S7b give 155/13/92/38 (N = 298) and 210/36/92/38 (N = 376). Discordance is below half in both. The supplement supports the reading given here and does not resolve the conflict.
- Corrections. No correction, erratum or expression of concern was found. Crossref update metadata: none. Crossref
filter=updates:query: 0 results. PMC notices: none. Europe PMC (PMID 40890807) lists no comment or correction. Crossref record:update-to,updated-byandrelationempty (rechecked 2026-10-23). Web searches for the DOI or title together with "correction" or "erratum", run during verification, returned nothing relevant. - PubPeer. PubPeer could not be queried directly: its search page returned HTTP 403 and its API returned a Cloudflare human-verification page (rechecked 2026-10-23). Web searches run while this finding was verified, for the title together with "PubPeer", found no PubPeer thread; a further search on 2026-10-23 could not be run because the session's search quota was exhausted.
Suggested correction
Abstract, Results. Replace "Over half (216/376, 57.4%) had discordance between registration and publications." with:
Over half (216/376, 57.4%) had concordance between registration and publications, and 160/376 (42.6%) had discordance.
Abstract, Conclusions. Replace "More than half of published trials have data sharing discordance between registration and publication." with:
More than 40% of published trials have data sharing discordance between registration and publication.
Main Conclusions section. Replace "More than half of published trials have discordance between data sharing plans that they submit in trial registration versus publications." with:
More than 40% of published trials (160/376, 42.6%) have discordance between data sharing plans that they submit in trial registration versus publications.
The corrected wording matches the Discussion's own statement ("over 40% were discordant"). Optionally, the unlabeled first sub-column of Table 3 could be headed "No" to match "Yes(n = 216)".
Recomputation script (python recompute.py.txt)
"""Recompute the registration-vs-publication data sharing concordance in PMC12403310
(BMC Med 2025; doi:10.1186/s12916-025-04328-z) from the paper's own printed counts,
and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Inputs are copied from the published article (Abstract, Methods, Results, Table 2,
Table 3, Discussion) and from Additional file 1 (Tables S6 and S7).
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Table 2: rows = plan to share in registration, columns = plan to share in publication
YY, YN, NY, NN = 155, 13, 147, 61
N_REP = 376
ABSTRACT_K, ABSTRACT_PCT = 216, 57.4 # "Over half (216/376, 57.4%) had discordance"
# Additional file 1, Table S7 (same cell order: Yes/Yes, Yes/No, No/Yes, No/No)
S7A = (155, 13, 92, 38) # after removing 78 trials with undecided/missing registered plans
S7B = (210, 36, 92, 38) # undecided/missing registered plans treated as "Yes"
S7_REPORTED_CONC = {"S7a": 64.7, "S7b": 65.9} # Results, sensitivity paragraph
REG_YES, REG_N = 171, 383 # Abstract: 44.6% (171/383)
def pct(k, n):
return 100.0 * k / n
def wilson(k, n, z=1.959964):
p = k / n
d = 1 + z**2 / n
c = (p + z**2 / (2 * n)) / d
h = z * np.sqrt(p * (1 - p) / n + z**2 / (4 * n**2)) / d
return 100 * (c - h), 100 * (c + h)
print("=== 1. Table 2 cells and margins ===")
N = YY + YN + NY + NN
print(f"N = {N} (reported {N_REP})")
for lab, k in [("Yes/Yes", YY), ("Yes/No", YN), ("No/Yes", NY), ("No/No", NN)]:
print(f" {lab:8s} {k:4d} {pct(k, N):5.1f}%")
print(f" registration Yes {YY+YN} ({pct(YY+YN, N):.1f}%), No {NY+NN} ({pct(NY+NN, N):.1f}%)")
print(f" publication Yes {YY+NY} ({pct(YY+NY, N):.1f}%), No {YN+NN} ({pct(YN+NN, N):.1f}%)")
print("\n=== 2. Concordance and discordance (Methods definitions) ===")
conc, disc = YY + NN, YN + NY
lo_c, hi_c = wilson(conc, N)
lo_d, hi_d = wilson(disc, N)
print(f"concordant = Yes/Yes + No/No = {YY} + {NN} = {conc}/{N} = {pct(conc, N):.1f}% "
f"(Wilson 95% CI {lo_c:.1f} to {hi_c:.1f})")
print(f"discordant = Yes/No + No/Yes = {YN} + {NY} = {disc}/{N} = {pct(disc, N):.1f}% "
f"(Wilson 95% CI {lo_d:.1f} to {hi_d:.1f})")
print(f"Abstract figure {ABSTRACT_K}/{N_REP} = {pct(ABSTRACT_K, N_REP):.1f}% "
f"-> equals the concordant count: {ABSTRACT_K == conc}")
print(f"Is discordance more than half? {disc > N/2} (half of {N} = {N/2:.0f})")
print(f"Discordance overstated by {ABSTRACT_PCT - round(pct(disc, N), 1):.1f} percentage points "
f"(unrounded {pct(conc, N) - pct(disc, N):.2f})")
print(f"Discussion 'over 40% were discordant': {pct(disc, N):.1f}% > 40% -> {pct(disc, N) > 40}")
print("Table 3 column sizes printed: n = 160 and 'Yes' n = 216 "
f"-> discordant {disc}, concordant {conc}")
print("\n=== 3. McNemar test on the discordant cells (reported P < 0.001) ===")
x2 = (YN - NY) ** 2 / (YN + NY)
x2cc = (abs(YN - NY) - 1) ** 2 / (YN + NY)
p_exact = stats.binomtest(YN, YN + NY, 0.5).pvalue
print(f"chi2 = {x2:.1f}, P = {stats.chi2.sf(x2, 1):.1e}; continuity-corrected chi2 = {x2cc:.1f}, "
f"P = {stats.chi2.sf(x2cc, 1):.1e}; exact P = {p_exact:.1e}")
print("\n=== 4. Sensitivity analyses (Additional file 1, Table S7) ===")
sens = {}
for lab, cells in [("S7a", S7A), ("S7b", S7B)]:
a, b, c, d = cells
n = a + b + c + d
cc, dd = a + d, b + c
lo, hi = wilson(dd, n)
sens[lab] = (dd, n, pct(dd, n), lo, hi)
print(f"{lab}: N = {n}; concordant {cc} ({pct(cc, n):.1f}%; paper {S7_REPORTED_CONC[lab]}%"
f"); discordant {dd} ({pct(dd, n):.1f}%, "
f"Wilson 95% CI {lo:.1f} to {hi:.1f})")
print("\n=== 5. Registration-level figure (different denominator, consistent) ===")
print(f"{REG_YES}/{REG_N} = {pct(REG_YES, REG_N):.1f}% (reported 44.6%)")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle, FancyArrowPatch
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
# ---- fig1: abstract/conclusions vs Results/Table 2, with the 2x2 table drawn
fig = plt.figure(figsize=(7.2, 4.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100); ax.set_ylim(0, 60); ax.axis("off"); ax.set_facecolor(BG)
# left panel: what the abstract and conclusions say
ax.add_patch(FancyBboxPatch((1.5, 6), 38.5, 48, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="white", ec=ORANGE, lw=1.5))
ax.text(3.5, 51.2, "Abstract (Results, Conclusions)\n+ main Conclusions section",
fontsize=10.5, weight="bold", va="top")
ax.text(3.5, 41.5, "As reported", color=ORANGE, fontsize=10, weight="bold", va="top")
ax.text(3.5, 37, "“Over half (216/376, 57.4%)\nhad discordance between\nregistration and publications.”",
fontsize=10, va="top", style="italic")
ax.text(3.5, 22.5, "“More than half of published\ntrials have data sharing\ndiscordance …”",
fontsize=10, va="top", style="italic")
ax.text(3.5, 9.5, "Reads 216 (57.4%) as discordant", fontsize=10, color=SEC, va="center")
# right panel: Table 2 as a 2x2 grid (rows = registration, columns = publication)
x0, y0, cw, ch = 60, 14, 16, 12
ax.text(59, 53.2, "Results text + Table 2 (same 376 trials)", fontsize=10.5, weight="bold", va="top")
ax.text(x0 + cw, 47.6, "Publication: plan to share?", ha="center", fontsize=10, color=SEC)
ax.text(x0 + cw / 2, 40.5, "Yes", ha="center", fontsize=10, color=SEC)
ax.text(x0 + 1.5 * cw, 40.5, "No", ha="center", fontsize=10, color=SEC)
ax.text(x0 + 2 * cw + 1.2, y0 + 1.5 * ch, "Yes", ha="left", va="center", fontsize=10, color=SEC)
ax.text(x0 + 2 * cw + 1.2, y0 + 0.5 * ch, "No", ha="left", va="center", fontsize=10, color=SEC)
ax.text(x0 + 2 * cw + 6.3, y0 + ch, "Registration: plan to share?", ha="center", va="center",
rotation=90, fontsize=10, color=SEC)
cells = [(0, 1, "Yes/Yes", YY, True), (1, 1, "Yes/No", YN, False),
(0, 0, "No/Yes", NY, False), (1, 0, "No/No", NN, True)]
for cx, cy, lab, k, is_conc in cells:
ax.add_patch(Rectangle((x0 + cx * cw, y0 + cy * ch), cw, ch, fc="#e6eef9" if is_conc else "white",
ec=BLUE if is_conc else SEC, lw=2 if is_conc else 1,
ls="-" if is_conc else (0, (3, 2))))
ax.text(x0 + cx * cw + cw / 2, y0 + cy * ch + ch * 0.62, f"{k}", ha="center", va="center",
fontsize=12, weight="bold", color=BLUE if is_conc else TXT)
ax.text(x0 + cx * cw + cw / 2, y0 + cy * ch + ch * 0.27, f"{lab} ({pct(k, N):.1f}%)",
ha="center", va="center", fontsize=10, color=SEC)
ax.text(x0 + cw, 10, f"Concordant (solid): {YY} + {NN} = {conc} ({pct(conc, N):.1f}%)",
ha="center", va="center", fontsize=10, color=BLUE, weight="bold")
ax.text(x0 + cw, 6.3, f"Discordant (dashed): {YN} + {NY} = {disc} ({pct(disc, N):.1f}%)",
ha="center", va="center", fontsize=10, color=TXT)
ax.text(x0 + cw, 2.4, "Results: \u201c216 (57.4%) had concordance\u201d",
ha="center", va="center", fontsize=10, color=SEC, style="italic")
# connector naming the conflict
arr = FancyArrowPatch((41.2, 33), (52.0, 10.6), connectionstyle="arc3,rad=-0.3",
arrowstyle="-|>", mutation_scale=14, lw=2, color=TXT)
ax.add_patch(arr)
ax.text(49.6, 41.5, "same 216 trials:\n\u201cdiscordance\u201d in\nthe abstract,\n\u201cconcordance\u201d in\nResults & Table 2",
ha="center", va="center", fontsize=10, color=TXT)
save(fig, "fig1")
# ---- fig2: share of trials discordant, reported vs recomputed, 0-100% axis
rows = [
("Abstract / Conclusions\n(as reported)", ABSTRACT_PCT, None, ORANGE, "o"),
(f"Table 2, main analysis\n({disc}/{N}, recomputed)", pct(disc, N), (lo_d, hi_d), BLUE, "o"),
(f"Table S7a, undecided removed\n({sens['S7a'][0]}/{sens['S7a'][1]}, recomputed)",
sens["S7a"][2], sens["S7a"][3:], BLUE, "s"),
(f"Table S7b, undecided as Yes\n({sens['S7b'][0]}/{sens['S7b'][1]}, recomputed)",
sens["S7b"][2], sens["S7b"][3:], BLUE, "s"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.4), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1]
for y, (lab, v, ci, col, mk) in zip(ys, rows):
if ci is not None:
ax.plot(ci, [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot(v, y, mk, color=col, ms=9, zorder=3)
txt = f"{v:.1f}%" if ci is None else f"{v:.1f}% ({ci[0]:.1f}–{ci[1]:.1f})"
ax.text((ci[1] if ci else v) + 2, y, txt, va="center", fontsize=10, color=col)
ax.axvline(50, color=SEC, lw=0.8)
ax.text(50.8, ys[0] + 0.5, "half of trials", fontsize=10, color=SEC, va="center")
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 100); ax.set_ylim(-0.6, len(rows) - 0.2)
ax.set_xlabel("Trials with discordant registered vs published data sharing plans (%)")
ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o", color=ORANGE, ms=8, label="reported in paper")
ax.plot([], [], "o", color=BLUE, ms=8, label="recomputed from the paper's own counts (Wilson 95% CI)")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, fontsize=10, frameon=False)
save(fig, "fig2")
print("\nfigures written to", HERE)
Abstract states no severe treatment-related adverse events; the paper's Table 3 and Results report two
BMC Med (2026) · doi:10.1186/s12916-026-04695-1 · PMC13032705
Summary
The Abstract's only safety result reads "No dose-limiting toxicities (DLTs) or severe treatment-related adverse events (TRAEs) were reported". The Discussion repeats this as "No DLTs or serious TRAEs were observed". The paper's own Results text and Table 3 report grade ≥ 3 TRAEs in 2 of the 87 Part III patients (2.3%; exact 95% CI 0.3% to 8.1%): a grade 5 (fatal) dyspnea in the head and neck squamous cell carcinoma (HNSCC) cohort and a grade 3 anemia in the sarcoma cohort. Table 3 also counts 2 patients under "TRSAE", which its footnote defines as "treatment-related severe adverse events". Under the CTCAE v5.0 grading the paper uses, grade 3 means severe, so the zero in the summary statements contradicts the paper's own count. The paper is also inconsistent about the fatal event: it calls the grade 5 dyspnea a TRAE, yet says that none of the 6 deaths "was attributed to T3011". In a first-in-human trial, a possible treatment-related death is a safety signal readers need to see in the Abstract.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py. The passages from the Abstract, Results, Table 3 and Discussion were also checked against the publisher's version of record.
Abstract, Results (the last sentence of the paragraph, which reports the Part III results):
No dose-limiting toxicities (DLTs) or severe treatment-related adverse events (TRAEs) were reported.
Abstract, Conclusions:
T3011 shows a favorable safety profile, immune-modulating effects, and preliminary efficacy in patients with advanced solid tumors.
Results > Safety (Part III, N = 87):
Grade ≥ 3 TRAEs occurred in 2 (2.3%) patients, one case occurred in an HNSCC patient (grade 5, dyspnea), and another in a sarcoma patient (grade 3, anemia). Adverse events led to T3011 discontinuation in 4 patients (4.6%) and death in 6 patients (6.9%). The causes of death included pulmonary infection (n = 1), dyspnea (n = 1), respiratory failure (n = 1), cerebral infarction (n = 1), and death of unknown cause (n = 2). However, none of these was attributed to T3011.
Overall, the NCI CTCAE grades for these common TRAEs with an incidence exceeding 5% were predominantly grade 1 or 2, with only one grade 3 event (anemia) occurring in the sarcoma cohort.
Table 3, "Safety summary" (columns HNSCC N = 28 / BC N = 12 / Sarcoma N = 44 / Melanoma N = 1 / Others N = 2 / Total N = 87):
TRSAE | 1 (3.6) | 0 | 1 (2.3) | 0 | 0 | 2 (2.3)
≥ Grade 3 TRAE | 1 (3.6) | 0 | 1 (2.3) | 0 | 0 | 2 (2.3)
Table 3 footnote:
TRSAE treatment-related severe adverse events
Discussion, first paragraph:
No DLTs or serious TRAEs were observed, consistent with previously reported general tolerable safety profiles of intratumoral oncolytic viral therapies
Conclusions:
demonstrating a favorable safety profile, immune-modulating effects, and preliminary efficacy, particularly in patients with recurrent or metastatic HNSCC
Methods (grading and the DLT definition):
Adverse events were graded according to the NCI-CTCAE version 5.0.
AEs were coded using the MedDRA version 26.0 or higher and categorized by system organ class and preferred terms, severity (CTCAE v5.0 grades 1–5), and relationship to the study drug.
The DLT evaluation period was defined as 28 days from the initial administration of T3011.
Grade 5 toxicities (death) and other toxicity-driven premature treatment discontinuations adjudicated by investigators/sponsors were also considered DLTs.
Results > Patients and treatments > Part I and II:
Based on the safety data from both Part I and Part II, no DLT events were reported. No SAE, TRAE leading to discontinuation of T3011 or death was observed.
What the paper's own numbers show
Method. I transcribed all ten rows of Table 3 and all seven rows of Table 4. For each row I summed the five cohort counts, compared the sum with the printed total, and recomputed every printed percentage from the cohort sizes (28, 12, 44, 1, 2; total 87). I then checked the percentages in the Results > Safety text against the same denominators. Exact (Clopper-Pearson) 95% intervals were computed for the grade ≥ 3 TRAE proportions. Every number below is printed by recompute.py.txt.
The tables are arithmetically consistent: every row sum and every percentage in Tables 3 and 4 matches the printed value. (Two cosmetic points in the PMC-archived Table 3: the row label reads "≥ Grade TEAE", without the 3, where the Results give the matching 17.2% for "≥ Grade 3 AEs"; and the sarcoma cell of "TEAE leading to death" is printed as "1" with no percentage, which would be 2.3%.) The problem is not arithmetic. The summary statements give a count of zero where the paper's own table and text give two.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Severe TRAEs, Part III | Abstract, Results | none ("No … severe treatment-related adverse events (TRAEs) were reported") | 2 of 87 (Table 3 and Results text) |
| Serious TRAEs | Discussion | none ("No DLTs or serious TRAEs were observed") | 2 of 87 under TRSAE in Table 3; one of the two is graded fatal |
| ≥ Grade 3 TRAE, total | Table 3; Results > Safety | 2 (2.3%) | 1+0+1+0+0 = 2; 2/87 = 2.30% (exact 95% CI 0.3% to 8.1%) |
| TRSAE ("treatment-related severe adverse events"), total | Table 3 | 2 (2.3%) | 1+0+1+0+0 = 2; 2/87 = 2.3% |
| ≥ Grade 3 TRAE, HNSCC | Table 3; Results (grade 5 dyspnea) | 1 (3.6%) | 1/28 = 3.57% (exact 95% CI 0.1% to 18.3%) |
| ≥ Grade 3 TRAE, sarcoma | Table 3; Results (grade 3 anemia) | 1 (2.3%) | 1/44 = 2.27% (exact 95% CI 0.1% to 12.0%) |
| Grade ≥ 3 TRAEs if the dyspnea were judged unrelated | — | — | 1/87 = 1.15% (exact 95% CI 0.03% to 6.2%): still not zero |
| TEAE leading to death | Table 3; Results > Safety | 6 (6.9%), "none of these was attributed to T3011" | 3+1+1+0+1 = 6; 6/87 = 6.9%; listed causes 1+1+1+1+2 = 6, one of them dyspnea |
| Grade 5 TRAE | Results > Safety | 1 (dyspnea, HNSCC) | conflicts with 0 deaths attributed to T3011 in the same paragraph |
| Any-grade TEAE | Table 3; Results | 77 (88.5%) | 24+8+42+1+2 = 77; 77/87 = 88.5% |
| ≥ Grade 3 TEAE | Table 3; Results | 15 (17.2%) | 4+2+8+0+1 = 15; 15/87 = 17.2% |
| Any-grade TRAE | Table 3; Results | 55 (63.2%) | 12+7+33+1+2 = 55; 55/87 = 63.2% |
| SAE | Table 3 | 13 (14.9%) | 3+1+8+0+1 = 13; 13/87 = 14.9% |
| TRAEs ≥ 5% (7 terms) | Table 4; Results | 9, 9, 6, 6, 5, 5, 5 of 87 | all row sums and percentages match |
For reference, an observed count of 0 of 87 would carry an exact 95% upper bound of 4.2%; the reported count of 2 gives an interval reaching 8.1%.
Figure
Figure 1. The two summary statements (orange panel: Abstract, Results and Discussion) and the paper's own detailed safety reporting for Part III (blue panel: Results > Safety and Table 3), with the cohort counts of the two relevant Table 3 rows summed by recompute.py.txt. Conflict 1: the summary statements say zero severe or serious TRAEs, while the detail gives 2 of 87 patients (2.3%; exact 95% CI 0.3% to 8.1%), under both "≥ Grade 3 TRAE" and "TRSAE", which the Table 3 footer defines as treatment-related severe adverse events. Conflict 2 (gray panel): the same Results paragraph calls one of the two a grade 5 dyspnea TRAE, which under CTCAE v5.0 means death related to the adverse event, but then says none of the 6 deaths, one of whose listed causes is dyspnea, "was attributed to T3011". All text in the figure is quoted or summarized from the paper; no figure or image from the paper is reproduced.
Why it matters
- The Abstract's only safety finding is contradicted by the paper's own table. Safety is the primary endpoint of Parts I and II and a key secondary endpoint of Part III. A reader of the Abstract is told that no severe TRAEs occurred. The body reports two patients with grade ≥ 3 TRAEs, one of them graded fatal.
- The omitted event is a possible treatment-related death in a first-in-human trial. The paper itself grades the HNSCC dyspnea as a grade 5 TRAE. Whether or not that attribution is right, the reader needs to know that the investigators recorded it. The Abstract and Discussion do not mention it.
- The cohort singled out in the Conclusions is the one with the fatal event. The Conclusions describe "a favorable safety profile … particularly in patients with recurrent or metastatic HNSCC". The only grade 5 TRAE was in the HNSCC cohort (1/28, 3.6%; exact 95% CI 0.1% to 18.3%).
- The attribution of the death is unresolved within the paper. One sentence makes the dyspnea a grade 5 TRAE; three sentences later, none of the deaths is attributed to T3011. Both cannot be right, and the paper does not say which is. Under the Methods' DLT definition, a related grade 5 toxicity counts as a DLT. The paper does not say whether DLT assessment applied in Part III or when the death occurred relative to the 28-day DLT window, so whether the event bears on the "no DLTs" statement cannot be determined from the paper.
- What the error does not change. Table 3 is arithmetically consistent, and a rate of 2/87 grade ≥ 3 TRAEs is low. A broad description of the safety profile as tolerable may well survive the correction. The DLT result for the dose-escalation Parts I and II, where the Results report no DLT and no SAE, is not contradicted. The error is in the summary statements, which give zero where the paper's data give two.
Explanations considered
- The Abstract sentence refers only to Parts I and II. The Results do say that no DLT and no SAE occurred in Parts I and II, so "No DLTs" fits those parts. But the sentence closes the Abstract's Results paragraph, which reports only Part III ("Among the 87 patients treated with T3011 monotherapy in Part III …"), and it does not name a part. The Discussion repeats the claim for the study at the RP2D. The Conclusions then rest a "favorable safety profile" on it, for patients with advanced solid tumors in general. Even on this reading, the Abstract would leave out the Part III grade ≥ 3 TRAEs entirely. Does not account for the discrepancy.
- "Severe" means something other than CTCAE grade ≥ 3. The paper grades AEs by CTCAE v5.0, where grade 3 is "severe or medically significant" and grade 5 is death related to the AE. The paper's own Table 3 counts 2 patients as treatment-related severe AEs (TRSAE). If "severe" were instead read as "serious" (the Discussion's word), the fatal event is serious by the standard regulatory definition (ICH E2A: an event that results in death), and the TRSAE row, which sits directly below the SAE row, still counts 2. Under every definition the paper uses, the count is 2, not 0. Does not account for it.
- The grade 5 dyspnea was later judged unrelated, and the Abstract reflects that. This would reconcile the Abstract with the "none of these was attributed to T3011" sentence. It would still leave the grade 3 anemia, which the paper consistently reports as treatment-related ("only one grade 3 event (anemia) occurring in the sarcoma cohort" and "anemia (6.9%, Grade ≤ 3)" in Results > Safety; anemia is listed among the TRAEs in Table 4). One severe TRAE (1/87) still contradicts "no severe TRAEs". It would also make Table 3 and the Results text wrong. Does not account for it.
- Conversion or typesetting artifact. The same wording and numbers appear in the publisher's version of record, the PMC page, the Europe PMC full-text XML and the local PMC archive. Table 3 is internally consistent, and its counts agree with the Results text. Ruled out.
- The supplement resolves it. Additional file 3 ("AE details") lists AE names and dates by participant identifier, but it has no CTCAE grade, causality, seriousness or outcome columns, so it cannot show which events were grade ≥ 3 or treatment-related. It neither confirms nor resolves either conflict.
Status
- Version of record. Checked on the publisher's page (https://link.springer.com/article/10.1186/s12916-026-04695-1, rendered in headless Chromium because plain fetches met a JavaScript challenge; BMC Med 24:177, published 20 February 2026). The Abstract, Results > Safety, Table 3 (including the TRSAE footnote) and Discussion passages quoted above read the same there. The PMC page and the Europe PMC full-text XML match as well.
- Supplements. The publisher lists three additional files. File 1 is the CONSORT checklist. File 2 (Figs. S1 to S6 and Table S1, PD-L1 CPS) contains no safety or grade data. File 3 (.xlsx; AE details and RNA-seq data) has an "AE details" sheet with 614 rows for 91 participant identifiers in four columns (identifier, AE occurrence, start date, AE name). It records no grade, causality, seriousness or outcome.
- Corrections and comments. As of 2026-10-24, Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction, expression of concern or retraction notice (
src/known.py). PubPeer could not be queried directly: its API and search page returned a Cloudflare human-verification challenge (HTTP 403). Web searches for the DOI and the title with "PubPeer" found no thread. - Sources. PMC13032705 (via
src/show.py); https://doi.org/10.1186/s12916-026-04695-1; the publisher's version of record and Additional files 1 to 3; Europe PMC; Crossref; NCI CTCAE v5.0 (grade definitions); ICH E2A (definition of a serious adverse event).
Suggested correction
- Replace the Abstract sentence "No dose-limiting toxicities (DLTs) or severe treatment-related adverse events (TRAEs) were reported." with a statement that matches the Results, for example: "No DLTs occurred in Parts I and II. In Part III, grade ≥ 3 TRAEs occurred in 2 of 87 patients (2.3%): one grade 5 dyspnea (HNSCC) and one grade 3 anemia (sarcoma)."
- Revise the Discussion sentence "No DLTs or serious TRAEs were observed" in the same way, and reconsider the "favorable safety profile … particularly in patients with recurrent or metastatic HNSCC" wording in the Conclusions in light of the grade 5 TRAE in that cohort.
- Resolve the attribution of the fatal dyspnea. Either it was treatment-related, in which case "none of these was attributed to T3011" should be corrected, or it was not, in which case Table 3 (≥ Grade 3 TRAE and TRSAE) and the Results sentence should be corrected to 1 (1.1%). In either case the remaining grade 3 anemia means the Abstract cannot say that no severe TRAEs occurred.
- State whether DLT assessment applied to Part III and, if it did, whether the grade 5 event occurred within the 28-day DLT window.
- Minor: correct the Table 3 row label "≥ Grade TEAE" to "≥ Grade 3 TEAE", and add the missing percentage (2.3) to the sarcoma cell of "TEAE leading to death".
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13032705 write-up and regenerate fig1.
Run: python recompute.py.txt
All inputs are transcribed from the article (BMC Med 2026;24:177,
doi:10.1186/s12916-026-04695-1), Part III safety set, Tables 3 and 4.
"""
import os
from scipy.stats import beta
HERE = os.path.dirname(os.path.abspath(__file__))
COHORTS = ["HNSCC", "BC", "Sarcoma", "Melanoma", "Others"]
N = [28, 12, 44, 1, 2]
NT = 87
assert sum(N) == NT, "cohort sizes must add to the Part III safety set"
# Table 3: (cohort counts, printed total, printed cohort %, printed total %)
# None = printed as a bare count with no percentage.
T3 = {
"TEAE": ([24, 8, 42, 1, 2], 77, [85.7, 66.7, 95.5, 100.0, 100.0], 88.5),
"TRAE": ([12, 7, 33, 1, 2], 55, [42.9, 58.3, 75.0, 100.0, 100.0], 63.2),
"SAE": ([3, 1, 8, 0, 1], 13, [10.7, 8.3, 18.2, None, 50.0], 14.9),
"TRSAE": ([1, 0, 1, 0, 0], 2, [3.6, None, 2.3, None, None], 2.3),
">= Grade 3 TEAE": ([4, 2, 8, 0, 1], 15, [14.3, 16.7, 18.2, None, 50.0], 17.2),
">= Grade 3 TRAE": ([1, 0, 1, 0, 0], 2, [3.6, None, 2.3, None, None], 2.3),
"TEAE -> discontinuation of T3011": ([4, 0, 0, 0, 0], 4, [14.3, None, None, None, None], 4.6),
"TEAE -> discontinuation of study": ([4, 0, 0, 0, 1], 5, [14.3, None, None, None, 50.0], 5.7),
"TEAE -> death": ([3, 1, 1, 0, 1], 6, [10.7, 8.3, None, None, 50.0], 6.9),
"TEAE -> interruption of T3011": ([1, 1, 6, 0, 0], 8, [3.6, 8.3, 13.6, None, None], 9.2),
}
T4 = {
"Pyrexia": ([1, 2, 5, 0, 1], 9, 10.3),
"Influenza-like illness": ([0, 0, 9, 0, 0], 9, 10.3),
"Facial edema": ([5, 0, 0, 0, 1], 6, 6.9),
"Anemia": ([0, 1, 5, 0, 0], 6, 6.9),
"Pain": ([0, 1, 4, 0, 0], 5, 5.7),
"AST elevation": ([0, 1, 3, 0, 1], 5, 5.7),
"Proteinuria": ([0, 1, 4, 0, 0], 5, 5.7),
}
def pct(x, n):
return round(100 * x / n, 1)
def exact_ci(x, n, level=0.95):
"""Clopper-Pearson exact binomial interval, in percent."""
a = 1 - level
lo = 0.0 if x == 0 else beta.ppf(a / 2, x, n - x + 1)
hi = 1.0 if x == n else beta.ppf(1 - a / 2, x + 1, n - x)
return 100 * lo, 100 * hi
print("=== Table 3: row sums and percentages (Part III, N = 87)")
all_ok = True
for k, (c, tot, p, pt) in T3.items():
rc = [pct(x, n) for x, n in zip(c, N)]
bad = [(COHORTS[i], p[i], rc[i]) for i in range(5) if p[i] is not None and abs(p[i] - rc[i]) > 0.05]
ok = sum(c) == tot and abs(pct(tot, NT) - pt) < 0.05 and not bad
all_ok &= ok
print(f" {k:34s} {'+'.join(map(str, c))} = {sum(c):2d} (printed {tot:2d}); "
f"{sum(c)}/87 = {pct(sum(c), NT):5.1f}% (printed {pt}); cohort % mismatches: {bad or 'none'}")
print(" Table 3 internally consistent:", all_ok)
print("\n=== Table 4: TRAEs >= 5%")
for k, (c, tot, pt) in T4.items():
print(f" {k:24s} sum = {sum(c)} (printed {tot}); {sum(c)}/87 = {pct(sum(c), NT)}% (printed {pt})")
print("\n=== Results > Safety text against Table 3")
for label, x in [("AEs any grade", 77), (">= Grade 3 AEs", 15), ("Grade >= 3 TRAEs", 2),
("AE -> T3011 discontinuation", 4), ("AE -> death", 6), ("TRAEs any grade", 55)]:
print(f" {label:28s} {x}/87 = {pct(x, NT)}%")
deaths = {"pulmonary infection": 1, "dyspnea": 1, "respiratory failure": 1,
"cerebral infarction": 1, "unknown cause": 2}
print(" Listed causes of death sum to", sum(deaths.values()), "(text: 6 patients)")
print("\n=== The contradiction")
sev_abstract = 0 # Abstract: "No ... severe treatment-related adverse events (TRAEs) were reported."
g3_trae = sum(T3[">= Grade 3 TRAE"][0])
trsae = sum(T3["TRSAE"][0])
print(f" Severe TRAEs per Abstract: {sev_abstract}")
print(f" >= Grade 3 TRAE per Table 3: {g3_trae}; TRSAE ('treatment-related severe adverse events') per Table 3: {trsae}")
print(f" Results text: grade 5 dyspnea (HNSCC) + grade 3 anemia (sarcoma) = {1 + 1}")
lo, hi = exact_ci(2, NT)
print(f" 2/87 = {100 * 2 / NT:.2f}% (exact 95% CI {lo:.1f}% to {hi:.1f}%)")
lo, hi = exact_ci(1, 28)
print(f" HNSCC 1/28 = {100 / 28:.2f}% (exact 95% CI {lo:.1f}% to {hi:.1f}%)")
lo, hi = exact_ci(1, 44)
print(f" Sarcoma 1/44 = {100 / 44:.2f}% (exact 95% CI {lo:.1f}% to {hi:.1f}%)")
lo, hi = exact_ci(1, NT)
print(f" If only the grade 3 anemia were treatment-related: 1/87 = {100 / NT:.2f}% "
f"(exact 95% CI {lo:.2f}% to {hi:.1f}%), still not 0")
lo, hi = exact_ci(0, NT)
print(f" For reference, 0/87 would give an exact 95% upper bound of {hi:.1f}%")
print(" Grade 5 TRAE reported (1) vs deaths attributed to T3011 per text (0): conflict =", 1 != 0)
# ---------------------------------------------------------------- figure
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"svg.fonttype": "none"})
fig = plt.figure(figsize=(7.2, 5.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2); ax.set_ylim(0, 5.9); ax.axis("off"); ax.set_facecolor(BG)
def panel(x, y, w, h, color, title, body, fs=10):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=color, lw=2))
ax.add_patch(FancyBboxPatch((x, y + h - 0.34), w, 0.34, boxstyle="round,pad=0,rounding_size=0.08",
fc=color, ec=color, lw=2))
ax.text(x + 0.12, y + h - 0.17, title, color="white", fontsize=10.5, weight="bold", va="center")
ax.text(x + 0.12, y + h - 0.48, body, color=INK, fontsize=fs, va="top", linespacing=1.35)
ax.text(0.15, 5.7, "What the paper says about severe treatment-related AEs (Part III, N = 87)",
fontsize=11.5, weight="bold", color=INK, va="center")
# Left: summary statements (orange = the paper's reported headline)
panel(0.15, 2.7, 3.0, 2.8, ORANGE, "Summary statements",
"Abstract, Results\n"
"\u201cNo dose-limiting toxicities\n(DLTs) or severe treatment-\nrelated adverse events\n(TRAEs) were reported.\u201d\n\n"
"Discussion\n"
"\u201cNo DLTs or serious TRAEs\nwere observed\u201d\n\n"
"Count stated: 0")
# Right: the paper's own detail (blue = counts summed from the paper's own table)
panel(4.05, 2.7, 3.0, 2.8, BLUE, "Results > Safety and Table 3",
"Results > Safety\n"
"\u201cGrade \u2265 3 TRAEs occurred in 2\n(2.3%) patients\u201d: grade 5\ndyspnea (HNSCC), grade 3\nanemia (sarcoma)\n\n"
"Table 3, cohorts summed\n"
"\u2265 Grade 3 TRAE: 1+0+1+0+0 = 2\n"
"TRSAE*: 1+0+1+0+0 = 2\n"
"*footer: \u201ctreatment-related\n severe adverse events\u201d\n"
"2/87 = 2.3% (exact 95% CI 0.3\u20138.1%)", fs=9.6)
# Connector 1
ax.add_patch(FancyArrowPatch((3.17, 4.1), (4.03, 4.1), arrowstyle="<|-|>", mutation_scale=14,
color=INK2, lw=2))
ax.text(3.6, 4.36, "Conflict 1", ha="center", fontsize=10, weight="bold", color=INK)
ax.text(3.6, 3.9, "0 vs 2\nof 87", ha="center", va="top", fontsize=10, color=INK2, linespacing=1.25)
# Bottom: same paragraph on deaths
panel(0.15, 0.62, 6.9, 1.3, INK2, "Results > Safety, same paragraph",
"\u201cAdverse events led to T3011 discontinuation in 4 patients (4.6%) and death in 6 patients\n"
"(6.9%). The causes of death included pulmonary infection (n = 1), dyspnea (n = 1), [\u2026]\n"
"However, none of these was attributed to T3011.\u201d Treatment-related deaths stated: 0",
fs=9.6)
ax.add_patch(FancyArrowPatch((5.55, 2.68), (5.55, 1.94), arrowstyle="<|-|>", mutation_scale=14,
color=INK2, lw=2))
ax.text(5.4, 2.31, "Conflict 2: a grade 5 (fatal) TRAE\nvs no death attributed to T3011",
ha="right", va="center", fontsize=10, color=INK, linespacing=1.2)
ax.text(0.15, 0.32, "Orange: the paper's summary statements. Blue: the paper's own detailed counts, "
"summed across cohorts\nby recompute.py.txt. CTCAE v5.0 (the paper's grading): grade 3 = severe; "
"grade 5 = death related to AE.", fontsize=9, color=INK2, va="center", linespacing=1.3)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nWrote fig1.svg and fig1.png to", HERE)
Abstract reports no severe cardiac toxicity or hand-foot syndrome; the Results report five Grade 3 cases
BMC Med (2026) · doi:10.1186/s12916-026-04895-9 · PMC13267627
Summary
This phase I dose-escalation study gave liposomal mitoxantrone (Lipo-MIT) with capecitabine to 26 patients with HER2-negative advanced breast cancer. Its abstract says the combination showed "no reports of severe cardiac toxicity or hand-foot syndrome". The paper's "Bullet Points" list (third bullet) and the Discussion say much the same. The paper's own Table 2, Table 3 and Results text tell a different story. Hand-foot syndrome (HFS) occurred in 13 of 26 patients (50.0%), and 5 of 26 (19.2%) had Grade 3 HFS, which is the highest grade CTCAE v5.0 defines for this event. In the Q4W cohort, whose schedule was chosen as the recommended phase 2 dose (RP2D), the rates were 7 of 11 (63.6%) and 3 of 11 (27.3%). The abstract statement is therefore false whichever way "severe" is read. It misdescribes one of the most frequent clinically relevant toxicities in the summary that most readers will rely on.
What the paper reports
Abstract, Results:
The combination demonstrated a manageable safety profile, with no reports of severe cardiac toxicity or hand-foot syndrome.
Bullet Points (third bullet):
Manageable safety profile with no severe short-term cardiac toxicity or hand-foot syndrome observed.
Discussion, paragraph 3:
Our initial data support this design rationale: the combination was well-tolerated, with a notable absence of severe short-term cardiac events and Hand-Foot Syndrome.
Discussion, paragraph 3 (same paragraph, continuing):
offering a potential “re-challenge” strategy with a distinct toxicity profile (e.g., lower rates of severe palmar-plantar erythrodysesthesia typically associated with PLD) [29, 30].
Results, Safety and tolerability, paragraph 3:
Other common non-hematological TRAEs included fatigue (61.5%), anorexia (53.8%), and hand-foot syndrome (50.0%). While most non-hematological events were mild to moderate, Grade 3 hand-foot syndrome was observed in 5 patients (19.2%) and was managed through dose interruptions or reductions.
The same paragraph ends:
No Grade 4 non-hematological TRAEs or treatment-related deaths were recorded.
Table 2, "Treatment-related adverse events (TRAEs) in the total population (N = 26)", row "Hand-foot syndrome": any grade 13 (50.0), Grade 3–4 5 (19.2). Footer: "AEs are coded according to NCI-CTCAE v5.0."
Table 3, "Treatment-related adverse events (TRAEs) and treatment exposure by dosing schedule", row "Hand-foot syndrome": Q3W (N = 15) any grade 6 (40.0), Grade 3–4 2 (13.3); Q4W (N = 11) any grade 7 (63.6), Grade 3–4 3 (27.3).
Abstract, Conclusions (for context, not itself in error):
The combination of Lipo-MIT and capecitabine showed preliminary antitumor activity and manageable tolerability in heavily pretreated HER2-negative ABC.
What the paper's own numbers show
Method. I recomputed every printed HFS percentage from its count and denominator. I checked that the two cohorts in Table 3 add up to the totals in Table 2. I then compared those counts with the count of zero that the abstract sentence implies, under both readings of "severe". The confidence intervals are exact (Clopper–Pearson) binomial intervals computed from the printed counts; the paper itself gives no interval for adverse events. Everything below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| HFS, any grade, all patients | Table 2; Results, Safety p3 | 13 (50.0%) | 13/26 = 50.0% (exact 95% CI 29.9–70.1%) |
| HFS, Grade 3–4, all patients | Table 2; Results, Safety p3 ("Grade 3 ... 5 patients") | 5 (19.2%) | 5/26 = 19.2% (6.6–39.4%) |
| HFS, Q3W cohort, any / Grade 3–4 | Table 3 | 6 (40.0) / 2 (13.3) | 6/15 = 40.0% / 2/15 = 13.3% |
| HFS, Q4W cohort (RP2D schedule), any / Grade 3–4 | Table 3 | 7 (63.6) / 3 (27.3) | 7/11 = 63.6% (30.8–89.1%) / 3/11 = 27.3% (6.0–61.0%) |
| Cohort sums vs total | Table 3 vs Table 2 | 13 any grade; 5 Grade 3–4; N = 26 | 6 + 7 = 13; 2 + 3 = 5; 15 + 11 = 26 (all consistent) |
| Grade of the 5 Grade 3–4 events | Results, Safety p3 | "Grade 3"; "No Grade 4 non-hematological TRAEs" | all 5 are Grade 3, the top CTCAE v5.0 grade for palmar-plantar erythrodysesthesia |
| HFS count implied by the abstract | Abstract, Results | "no reports of ... hand-foot syndrome" = 0 | 13 any grade; 5 Grade 3 |
| Severe HFS implied by Bullet Points and Discussion | Bullet Points (third bullet); Discussion p3 | "no severe ... hand-foot syndrome" = 0 | 5 Grade 3 (19.2%) |
| Share of HFS cases that were Grade 3 | Table 2 | — | 5/13 = 38.5% |
| HFS among non-hematological Grade 3–4 TRAEs | Table 2 | HFS 5, stomatitis 1, all others 0 | HFS accounts for 5 of the 6 non-hematological Grade 3–4 entries |
Every percentage the paper prints for HFS is arithmetically correct, and the tables agree with each other and with the Results text. The error is not in the data. It is in three summary sentences that contradict the data.
Figure
Figure 1. On the left are the three summary sentences as printed (abstract Results, Bullet Points third bullet, Discussion paragraph 3). On the right are the paper's own HFS data as printed: Table 2 (N = 26), the Results sentence and Table 3 by cohort. The blue text at the bottom right is recomputed from the Table 3 counts. The arrow marks the conflict. Grade 3 is the highest grade CTCAE v5.0 defines for palmar-plantar erythrodysesthesia, so the five Grade 3 cases are severe events under the grading system the paper uses.
Figure 2. HFS rates recomputed from the printed counts in Tables 2 and 3, with exact (Clopper–Pearson) 95% confidence intervals. Filled circles show any grade; open circles show Grade 3–4, all of which were Grade 3. The orange diamond is the count of zero implied by the abstract. Every interval excludes zero; the lowest lower bound is 1.7% (Q3W Grade 3, 2/15), and the full-population intervals start at 29.9% (any grade) and 6.6% (Grade 3). The highest rates are in the Q4W cohort, whose schedule was chosen as the RP2D.
Why it matters
- The abstract inverts a safety finding. For many readers, and for indexing and evidence-synthesis services, the abstract is the paper. In this study HFS was the most frequent non-hematological Grade 3 toxicity: it accounts for 5 of the 6 non-hematological Grade 3–4 entries in Table 2. The abstract says it did not occur.
- The RP2D cohort is the most affected. Clinicians planning phase II use of 22 mg/m² Q4W need the Q4W rates: about two in three patients had HFS of any grade (7/11) and about one in four had Grade 3 HFS (3/11). The summary text gives no hint of this.
- The Discussion's mechanistic argument relies on the claim. The Discussion says the 60-nm liposome shields "healthy tissues such as the myocardium and skin", and that the "initial data support this design rationale", citing the absence of HFS. With HFS in half of patients, the data do not support the skin-sparing part of that argument. Because capecitabine also causes HFS, this design cannot attribute the HFS to either drug. That uncertainty leaves the argument unsupported rather than rescuing it.
- The comparison with pegylated liposomal doxorubicin (PLD). The Discussion suggests that Lipo-MIT may offer "lower rates of severe palmar-plantar erythrodysesthesia typically associated with PLD". This study's own Grade 3 rate (19.2% overall, 27.3% at the RP2D schedule) gives no internal support for that framing. The cited references may support it for Lipo-MIT alone. That is not established for this combination.
- What does not change. The primary endpoints stand: no dose-limiting toxicities (DLTs) in cycle 1 and an RP2D of 22 mg/m² Q4W. The abstract Conclusions ("manageable tolerability") are compatible with the data, since the Results say the HFS was managed by dose interruption or reduction. The cardiac part of the sentence is not contradicted by anything in the paper. The title is not affected.
Explanations considered
- "Severe" applies to both nouns ("no severe cardiac toxicity or [severe] hand-foot syndrome"). Does not account for the numbers. The paper grades adverse events with NCI-CTCAE v5.0 (Methods; Table 2 footer). For palmar-plantar erythrodysesthesia that scale defines only Grades 1–3, and Grade 3 is the severe grade. The Results report "Grade 3 hand-foot syndrome ... in 5 patients (19.2%)", and Tables 2 and 3 agree (5 = 2 + 3). So the sentence is false on the narrow reading as well as the broad one.
- A different population or denominator. Does not account for the numbers. The abstract describes all 26 enrolled patients. Tables 2 and 3 use the full safety population: "Safety was assessed in all patients who received at least one dose", and the Results say "All 26 enrolled patients completed the Cycle 1 DLT observation window". The cohort denominators add up (15 + 11 = 26), and every percentage recomputes exactly.
- HFS excluded as a capecitabine toxicity rather than a toxicity of the combination. Does not account for the numbers. Table 2 defines TRAEs as events "considered by the investigator to be related to the study treatment", which is the combination. The abstract sentence is also about "the combination". The paper contains no statement that leaves HFS out of the combination's safety profile.
- "Short-term" limits the claim to the cycle-1 DLT window. Partly accounts for the Bullet Points and Discussion wording, but not the abstract. The Bullet Points and the Discussion say "short-term". Under the Methods, any Grade ≥3 non-hematological toxicity in cycle 1 would have been a DLT, with no exemption for HFS, and the Results report no DLTs. If "short-term" means cycle 1, those two sentences could be technically true. The abstract sentence carries no "short-term" qualifier, though. Neither the paper nor its supplement says when the Grade 3 HFS events occurred, and the Bullet Points and the Discussion sentence do not mention that five Grade 3 cases occurred.
- A conversion or archiving artifact. Ruled out. The publisher's version of record carries the same sentences and table rows as the PMC archive and the Europe PMC XML (see Status).
- The supplement or registry resolves it. Ruled out. The only supplementary file (Additional file 1) holds a single Cox model table for PFS, with no adverse-event data. The registry record (NCT06156761) has no posted results.
Status
- Version of record. The publisher's page (link.springer.com/article/10.1186/s12916-026-04895-9; history line "Published: 28 April 2026, Version of record: 15 June 2026"; BMC Med vol 24, article 356) was fetched with a headless browser. It carries the same abstract sentence, Bullet Points item, Results paragraph, Discussion sentence and Table 2 and Table 3 HFS rows as the PMC archive. Its text also matches the Europe PMC full-text XML. Every quotation above was checked verbatim against the PMC-archived text.
- Supplements. There is one supplementary file, Additional file 1 (12916_2026_4895_MOESM1_ESM.docx). It is identical in the Europe PMC bundle and on the publisher's static-content server. It contains only Table S1, an exploratory multivariable Cox model for PFS, and has no adverse-event data.
- Registry. The ClinicalTrials.gov record NCT06156761 has no posted results; its status is listed as unknown.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no notices. The PubPeer API and the PubPeer search page both refused automated access (HTTP 403, Cloudflare challenge). A general web search for PubPeer discussion of the paper returned no PubPeer page, so whether PubPeer has any comments remains unconfirmed.
- Sources. PMC article PMC13267627; the publisher's page and its Tables 2 and 3; Europe PMC full-text XML and supplementary-files bundle; Additional file 1; ClinicalTrials.gov NCT06156761; Crossref; Europe PMC; PubPeer (attempted).
Suggested correction
In the abstract Results, replace "The combination demonstrated a manageable safety profile, with no reports of severe cardiac toxicity or hand-foot syndrome." with a statement that matches Table 2. For example: "The combination demonstrated a manageable safety profile with no severe cardiac toxicity; hand-foot syndrome occurred in 50.0% of patients (Grade 3 in 19.2%) and was managed with dose interruption or reduction." Revise the third Bullet Points item the same way. In the Discussion, either remove "and Hand-Foot Syndrome" from "a notable absence of severe short-term cardiac events and Hand-Foot Syndrome", or, if, as the absence of DLTs implies, no Grade ≥3 HFS occurred in cycle 1, state that and that 5 patients (19.2%) had Grade 3 HFS in later cycles. Also revisit the skin-sparing "design rationale" and the comparison with PLD's palmar-plantar erythrodysesthesia in light of those rates. It would help readers to report the Q4W (RP2D) HFS rates (7/11 any grade, 3/11 Grade 3) next to the RP2D in the abstract.
Recomputation script (python recompute.py.txt)
"""Recompute the hand-foot syndrome (HFS) counts of PMC13267627
(BMC Med 2026, doi:10.1186/s12916-026-04895-9) and compare them with the
abstract's safety sentence.
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (identical in the publisher's
version of record):
Table 2: TRAEs in the total population (N = 26), NCI-CTCAE v5.0
Table 3: TRAEs by dosing schedule (Q3W N = 15, Q4W N = 11)
Results > Safety and tolerability, paragraph 3
"""
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Patch # noqa: E402
from scipy.stats import beta # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
N = {"total": 26, "Q3W": 15, "Q4W": 11}
# (any grade n, printed %), (Grade 3-4 n, printed %)
HFS = {
"total": ((13, 50.0), (5, 19.2)), # Table 2
"Q3W": ((6, 40.0), (2, 13.3)), # Table 3
"Q4W": ((7, 63.6), (3, 27.3)), # Table 3
}
RESULTS_TEXT = {"any_pct": 50.0, "g3_n": 5, "g3_pct": 19.2} # Results > Safety and tolerability, paragraph 3
# CTCAE v5.0, "Palmar-plantar erythrodysesthesia syndrome": defined grades are 1-3 only,
# Grade 3 = "Severe skin changes ... with pain; limiting self care ADL". Grade 3 is the top grade.
CTCAE_PPE_MAX_GRADE = 3
# Abstract Results: "no reports of severe cardiac toxicity or hand-foot syndrome"
ABSTRACT_IMPLIED = {"any": 0, "severe": 0}
def pct(a, b):
return 100.0 * a / b
def cp(k, n, a=0.05):
lo = 0.0 if k == 0 else beta.ppf(a / 2, k, n - k + 1)
hi = 1.0 if k == n else beta.ppf(1 - a / 2, k + 1, n - k)
return 100 * lo, 100 * hi
# ---------------------------------------------------------------- recomputation
print("Percentages recomputed from the printed counts")
rows = []
for grp in ("total", "Q3W", "Q4W"):
(na, pa), (ng, pg) = HFS[grp]
for lab, k, p in (("any grade", na, pa), ("Grade 3-4", ng, pg)):
r = pct(k, N[grp])
lo, hi = cp(k, N[grp])
rows.append((grp, lab, k, N[grp], r, lo, hi))
print(f" {grp:5s} {lab:9s}: {k}/{N[grp]} = {r:.2f}% (printed {p}) "
f"match={round(r, 1) == p}; exact 95% CI {lo:.1f}-{hi:.1f}%")
print("\nCohort sums (Table 3) vs total (Table 2)")
s_any = HFS["Q3W"][0][0] + HFS["Q4W"][0][0]
s_g3 = HFS["Q3W"][1][0] + HFS["Q4W"][1][0]
N_sum = N["Q3W"] + N["Q4W"]
print(f" N: 15 + 11 = {N_sum} (Table 2 N = {N['total']}) match={N_sum == N['total']}")
print(f" any grade: 6 + 7 = {s_any} (Table 2: 13) match={s_any == HFS['total'][0][0]}")
print(f" Grade 3-4: 2 + 3 = {s_g3} (Table 2: 5) match={s_g3 == HFS['total'][1][0]}")
print(f" Results text: any {RESULTS_TEXT['any_pct']}%, Grade 3 {RESULTS_TEXT['g3_n']} "
f"({RESULTS_TEXT['g3_pct']}%) match="
f"{RESULTS_TEXT['g3_n'] == s_g3 and round(pct(s_g3, 26), 1) == RESULTS_TEXT['g3_pct']}")
print("\nAbstract sentence vs tables")
print(f" reading A (no HFS of any grade): implied {ABSTRACT_IMPLIED['any']}, tables {s_any}")
print(f" reading B (no severe HFS; CTCAE max grade {CTCAE_PPE_MAX_GRADE} = severe): "
f"implied {ABSTRACT_IMPLIED['severe']}, tables {s_g3} Grade 3 "
f"(no Grade 4 non-hematological TRAEs per Results text)")
print(f" Grade 3 share of HFS cases: {s_g3}/{s_any} = {pct(s_g3, s_any):.1f}%")
g3_other = 1 # Table 2: stomatitis is the only other non-hematological Grade 3-4 TRAE (1)
print(f" Non-hematological Grade 3-4 TRAE events in Table 2: HFS {s_g3}, stomatitis {g3_other}"
f" -> HFS = {s_g3}/{s_g3 + g3_other} of them")
# ---------------------------------------------------------------- style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# ---------------------------------------------------------------- fig1: text panels
fig = plt.figure(figsize=(7.2, 5.0))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def panel(x, y, w, h, title, items, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.6",
fc="white", ec=edge, lw=1.5))
ax.text(x + 2, y + h - 3.2, title, fontsize=11.5, weight="bold", va="top")
yy = y + h - 10.5
for loc, txt, col in items:
ax.text(x + 2, yy, loc, fontsize=10, color=INK2, va="top")
yy -= 4.6
wrapped = textwrap.fill(txt, 37)
ax.text(x + 2, yy, wrapped, fontsize=10.5, color=col, va="top", linespacing=1.35)
yy -= 5.2 * (wrapped.count("\n") + 1) + 3.0
left = [
("Abstract, Results", "“no reports of severe cardiac toxicity or hand-foot syndrome”",
INK),
("Bullet Points (third bullet)", "“no severe short-term cardiac toxicity or hand-foot syndrome observed”",
INK),
("Discussion, paragraph 3", "“a notable absence of severe short-term cardiac events "
"and Hand-Foot Syndrome”", INK),
]
right = [
("Table 2 (N = 26, CTCAE v5.0)", "Hand-foot syndrome: any grade 13 (50.0%); "
"Grade 3–4 5 (19.2%)", INK),
("Results, Safety and tolerability", "“Grade 3 hand-foot syndrome was observed in "
"5 patients (19.2%)”", INK),
("Table 3 (Q3W N = 15 | Q4W N = 11)", "any grade 6 | 7; Grade 3–4 2 | 3", INK),
]
panel(2, 14, 40, 78, "Summary statements", left, ORANGE)
panel(58, 14, 40, 78, "Results and Tables 2–3", right, ORANGE)
# recomputed line in the right panel
ax.text(60, 31, f"Recomputed from Table 3:\n{HFS['Q3W'][0][0]} + {HFS['Q4W'][0][0]} = {s_any}/26 "
f"= {pct(s_any, 26):.1f}% any grade\n{HFS['Q3W'][1][0]} + {HFS['Q4W'][1][0]} = {s_g3}/26 "
f"= {pct(s_g3, 26):.1f}% Grade 3", fontsize=10.5, color=BLUE, va="top",
linespacing=1.35, weight="bold")
ax.add_patch(FancyArrowPatch((42.6, 53), (57.4, 53), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=INK))
ax.text(50, 57, "conflict", ha="center", fontsize=10.5, weight="bold")
ax.text(50, 49, "Grade 3 is\nCTCAE’s top\n(“severe”)\ngrade for HFS",
ha="center", va="top", fontsize=10, color=INK2, linespacing=1.25)
ax.text(2, 96.5, "Hand-foot syndrome: summary text vs the paper's own counts",
fontsize=11.5, weight="bold", va="top")
handles = [Patch(fc="white", ec=ORANGE, lw=1.5, label="Text and numbers as printed in the paper"),
Line2D([], [], color=BLUE, lw=0, marker="s", ms=8,
label="Recomputed from the paper's Table 3 counts")]
ax.legend(handles=handles, loc="lower center", bbox_to_anchor=(0.5, 0.005), ncol=2,
frameon=False, fontsize=10, handlelength=1.4)
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: rates with exact CIs
fig, ax = plt.subplots(figsize=(7.2, 3.9))
fig.subplots_adjust(left=0.30, right=0.97, top=0.86, bottom=0.25)
labels = {"total": "All patients (N = 26)", "Q3W": "Q3W cohort (N = 15)",
"Q4W": "Q4W cohort, RP2D\nschedule (N = 11)"}
ypos, yt, ytl = {}, [], []
y = 0
for grp in ("Q4W", "Q3W", "total"):
ypos[(grp, "Grade 3-4")] = y
ypos[(grp, "any grade")] = y + 0.38
yt.append(y + 0.19)
ytl.append(labels[grp])
y += 1.25
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for grp, lab, k, n, r, lo, hi in rows:
yy = ypos[(grp, lab)]
ax.plot([lo, hi], [yy, yy], color=BLUE, lw=2, solid_capstyle="butt")
filled = lab == "any grade"
ax.plot([r], [yy], marker="o", ms=8, color=BLUE, mfc=BLUE if filled else BG, mew=2,
lw=0)
if hi < 80:
ax.text(hi + 1.5, yy, f"{k}/{n} = {r:.1f}%", va="center", fontsize=10, color=INK)
else:
ax.text(lo - 1.5, yy, f"{k}/{n} = {r:.1f}%", va="center", ha="right", fontsize=10,
color=INK)
yt0 = ypos[("total", "any grade")]
ax.axvline(0, color=INK2, lw=0.8)
ax.plot([0], [yt0 + 0.0], marker="D", ms=9, color=ORANGE, lw=0, clip_on=False, zorder=5)
ax.annotate("Abstract: “no reports of ...\nhand-foot syndrome” (0)",
xy=(0.8, yt0 + 0.05), xytext=(4, yt0 + 0.42), fontsize=10, color=INK, va="bottom",
arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.2))
ax.set_yticks(yt)
ax.set_yticklabels(ytl, fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlim(0, 100)
ax.set_ylim(-0.5, 3.75)
ax.set_xlabel("Patients with treatment-related hand-foot syndrome, % (exact 95% CI)")
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
fig.text(0.02, 0.955, "Hand-foot syndrome rates implied by Tables 2 and 3", fontsize=11.5,
weight="bold", va="top")
handles = [Line2D([], [], color=BLUE, marker="o", ms=8, mfc=BLUE, lw=2, label="Any grade"),
Line2D([], [], color=BLUE, marker="o", ms=8, mfc=BG, mew=2, lw=2,
label="Grade 3–4 (all Grade 3)"),
Line2D([], [], color=ORANGE, marker="D", ms=9, lw=0, label="Abstract claim")]
fig.legend(handles=handles, loc="lower center", ncol=3, frameon=False, fontsize=10,
bbox_to_anchor=(0.55, 0.0))
fig.text(0.97, 0.10, "Blue: recomputed from the paper's printed counts", ha="right",
fontsize=10, color=INK2)
save(fig, "fig2")
print("\nwrote fig1.svg/png, fig2.svg/png")
Abstract labels the age-and-sex adjusted all-cancer hazard ratio as the multivariable adjusted estimate
BMJ (2017) · doi:10.1136/bmj.j108 · PMC5266623
Summary
The abstract and the Discussion give the hazard ratio for death from all cancers combined, for the most distressed group (GHQ-12 score 7-12) against the least distressed (0-6), as "multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48". In the paper's own Table 5, 1.32 (1.18 to 1.48) is the age and sex adjusted estimate (4353 deaths, n=163 363), and the Results text and Figures 2 and 3 describe it the same way. The multivariable estimate in Table 5 is 1.26 (1.11 to 1.42) (3800 deaths, n=146 008). The other six hazard ratios in the same abstract sentence all equal Table 5 multivariable rows exactly, so only the all-cancer figure comes from the other row. The association stays positive and statistically significant, but the fully adjusted excess risk is 26% rather than 32%. The abstract does not show that adjustment lowered the all-cancer estimate, and the Discussion uses the mislabelled figure to call an earlier meta-analysis estimate "of markedly lower magnitude".
What the paper reports
Abstract, Results. The all-cancer estimate is the first of seven, all under one "After adjustment for ..." clause:
After adjustment for age, sex, education, socioeconomic status, body mass index (BMI), and smoking and alcohol intake, and with reverse causality (by left censoring) and missing data (by imputation) taken into account, relative to people in the least distressed group (GHQ-12 score 0-6), death rates in the most distressed group (score 7-12) were consistently raised for cancer of all sites combined (multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48) and cancers not related to smoking (1.45, 1.23 to 1.71), as well as carcinoma of the colorectum (1.84, 1.21 to 2.78), prostate (2.42, 1.29 to 4.54), pancreas (2.76, 1.47 to 5.19), oesophagus (2.59, 1.34 to 5.00), and for leukaemia (3.86, 1.42 to 10.5).
Discussion, "Comparison with other studies":
the aggregated result for depression and cancer for all malignancies across the 13 studies included unity (relative risk 1.12, 95% confidence interval 0.99 to 1.26). This was of markedly lower magnitude than our estimate between psychological distress and overall cancer (multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48).
Results, paragraph beginning "During a mean (SD) follow-up":
In the 16 studies in aggregate, after adjustment for age and sex, higher levels of distress were associated with a 32% greater risk of total cancer mortality (hazard ratio 1.32, 95% confidence interval 1.18 to 1.48).
Table 5, "All cancers". The columns are deaths, number of people, distress score 0, 1-3, 4-6 and 7-12, P value for trend, and distress score 7-12 v 0-6:
Age and sex | 4353 | 163 363 | 1 (ref) | 0.99 (0.92 to 1.08) | 1.05 (0.91 to 1.20) | 1.31 (1.16 to 1.49) | <0.001 | 1.32 (1.18 to 1.48)
Multivariable† | 3800 | 146 008 | 1 | 0.95 (0.88 to 1.03) | 0.93 (0.78 to 1.10) | 1.22 (1.07 to 1.38) | 0.04 | 1.26 (1.11 to 1.42)
Table 5 footnote:
†Hazard ratio adjusted for age, sex (except in single sex analyses), BMI, educational attainment, smoking status, and frequency of alcohol consumption.
Methods, "Statistical analyses", which defines the two models:
Hazard ratios were minimally adjusted (age and sex only) and maximally adjusted (age, sex, BMI, educational attainment, smoking status, and frequency of alcohol consumption).
Figure 3 legend. In the typeset figure, the "All cancers combined" row reads 1.32 (1.18 to 1.48) under "adjusted for age and sex" and 1.26 (1.11 to 1.42) under "multivariable adjusted":
Hazard ratios are for psychological distress score of 7-12 (most distressed) relative to 0-6, and are age and—except in single sex analyses—sex adjusted (I2=15%), or multivariable adjusted (age, sex, BMI, educational attainment, smoking status, and alcohol consumption; I2=37%)
Figure 2 legend. The pooled "Overall" row of this forest plot is 4353 deaths, 1.32 (1.18 to 1.48):
Hazard ratios (adjusted for age and sex) are for psychological distress score of 7-12 (most distressed) relative to 0-6.
What the paper's own numbers show
Method. I copied the seven hazard ratios and 95% CIs from the abstract's Results sentence. For the same seven outcomes I copied both Table 5 rows, age (and sex) and multivariable, from the column "Distress score (7-12 v 0-6)". I then checked which row each abstract value equals, digit for digit. To show that the two all-cancer rows are separate estimates and not rounding variants of one number, I computed the geometric (log-scale) midpoint of each CI and the standard error of the log hazard ratio, (ln upper − ln lower)/(2 × 1.96). Finally, I repeated the Discussion's comparison with the earlier meta-analysis for each all-cancer estimate. That comparison is a z test on the difference of log ratios, treating the two estimates as independent. It is illustrative only: one estimate is a hazard ratio for distress and death, the other a relative risk for major depression and cancer occurrence. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| All cancers, labelled "multivariable adjusted" | Abstract, Results | 1.32 (1.18 to 1.48) | Equals the Table 5 age and sex row in all three numbers. The Table 5 multivariable row is 1.26 (1.11 to 1.42) |
| All cancers, labelled "multivariable adjusted" | Discussion, "Comparison with other studies" | 1.32 (1.18 to 1.48) | Same as above |
| All cancers, labelled "after adjustment for age and sex" | Results text | 1.32 (1.18 to 1.48) | Equals the Table 5 age and sex row |
| All cancers, age and sex | Table 5 | 1.32 (1.18 to 1.48); 4353 deaths; n=163 363; P for trend <0.001 | CI log-midpoint 1.322; SE(log HR) 0.0578; z = 4.80 |
| All cancers, multivariable | Table 5 | 1.26 (1.11 to 1.42); 3800 deaths; n=146 008; P for trend 0.04 | CI log-midpoint 1.255; SE(log HR) 0.0628; z = 3.68 |
| Other six abstract hazard ratios (not related to smoking, colorectum, prostate, pancreas, oesophagus, leukaemia) | Abstract vs Table 5 | 1.45, 1.84, 2.42, 2.76, 2.59, 3.86, each with its CI | 6 of 6 equal the Table 5 multivariable rows exactly. None equals the age and sex rows (1.38, 1.75, 2.29, 2.52, 2.42, 2.89) |
| Effect of adjustment | Table 5 | — | All cancers: 1.32 → 1.26 (down). The other six outcomes: all go up (for example leukaemia 2.89 → 3.86) |
| Excess all-cancer mortality, most v least distressed | Abstract (implied) vs Table 5 multivariable | 32% | 26%. The reported excess is 1.23 times the multivariable excess, and the multivariable excess is 19% smaller |
| Comparison with the earlier meta-analysis, RR 1.12 (0.99 to 1.26) | Discussion | 1.32 / 1.12 = 1.18; excess 32% v 12% | With 1.32: z = 1.95, P = 0.052. With 1.26: ratio 1.13, excess 26% v 12%, z = 1.34, P = 0.18 |
The two all-cancer estimates are each internally consistent: each printed hazard ratio sits at the log-midpoint of its own CI. They also come from different samples, with 553 deaths and 17 355 people fewer in the multivariable model. The paper states the reason. Results, paragraph beginning "Table 5 shows the four categories":
With different models being based on different analytical samples because of missing covariates, we recomputed these effects estimates in a non-missing dataset—that is, the same sample size in both models—and our results were unchanged.
So the paper attributes the different sample sizes to missing covariates, and it reports that re-estimating both models in the same (non-missing) sample left the results "unchanged". It gives no figures for that analysis, so the share of the 1.32 v 1.26 gap due to the extra covariates rather than the change of sample cannot be read from the paper.
Figure
Figure 1. Where the all-cancer estimate appears in the paper. Left: the abstract's wording, shortened at the ellipsis; the Discussion uses the same label and numbers. The estimate is in orange. Right: the two Table 5 rows for all cancers with their deaths and sample sizes. The solid orange arrow joins the abstract value to the Table 5 row that prints the identical numbers, which is the age and sex row. The dashed blue arrow joins the abstract's label, "multivariable adjusted", to the row that label names, which prints a different estimate, 1.26 (1.11 to 1.42). The quoted text and numbers are from the paper; the arrows and annotations are mine.
Figure 2. The seven hazard ratios (95% CI, log scale) in the abstract's Results sentence (orange filled circles), shown with the Table 5 multivariable estimate (blue diamonds) and the Table 5 age and sex estimate (open gray circles) for the same outcome. The thin vertical line marks a hazard ratio of 1. The label on the right of each row says which Table 5 row the abstract value equals. For the six site-specific or subgroup outcomes it is the multivariable row. For all cancers combined it is the age and sex row. All plotted values are from the paper's abstract and Table 5 as printed.
Why it matters
- The abstract gives the wrong model for its first and broadest estimate. The sentence says the all-cancer hazard ratio is adjusted for education, socioeconomic status, BMI, smoking and alcohol. Table 5 shows it is adjusted for age and sex only. A reader of the abstract alone gets a fully adjusted excess risk of 32% where the paper's multivariable model gives 26%, and a CI of 1.18 to 1.48 in place of 1.11 to 1.42. The abstract is also the text reproduced in PubMed and in secondary coverage.
- It hides the direction in which adjustment moved this estimate. Because every abstract value is presented as multivariable, a reader cannot tell that, for all cancers combined, adding covariates lowered the estimate from 1.32 to 1.26 and the P for trend from <0.001 to 0.04. For the other six outcomes in the sentence, the multivariable estimates are higher than the age and sex ones. So all cancers is the one outcome in the sentence where adjustment lowered the estimate, and it is the one where the age and sex value is shown. This matters for a reader weighing confounding by health behaviours, which the paper discusses.
- The Discussion uses the figure in an argument. The Discussion calls the earlier meta-analysis estimate, RR 1.12 (0.99 to 1.26), "of markedly lower magnitude" than "our estimate ... (multivariable adjusted hazard ratio 1.32 ...)". Against 1.32 the ratio is 1.18 and a simple test of the difference gives z = 1.95 (P = 0.052). Against the multivariable 1.26 the ratio is 1.13, with z = 1.34 (P = 0.18), and the two CIs overlap from 1.11 to 1.26. The contrast the Discussion draws is weaker than stated.
- Scope. The all-cancer association keeps its direction and its statistical significance: 1.26 (1.11 to 1.42) still excludes 1. The title, the abstract's Conclusion and the Discussion's "Principal findings" concern site-specific cancers, and the abstract reports those correctly. The error is a mislabelled headline estimate. It does not reach the paper's central site-specific findings.
Explanations considered
- The 1.32 comes from another multivariable analysis, such as the imputed-data or left-censored sensitivity analyses the abstract mentions. This does not account for the numbers. Those analyses are in the web appendices (appendix 1 table C; appendix 2 fig A), which are not deposited in PMC or Europe PMC and which I could not inspect. Four things in the paper argue against it. (a) The other six estimates in the same sentence, under the same "left censoring ... imputation" wording, equal the non-imputed Table 5 multivariable rows exactly. (b) The Results say the imputed meta-analysis "gave similar results to those seen with the non-missing dataset as reported in the present paper", which identifies the reported estimates as the non-imputed ones. (c) The Results say that in the left-censored analysis "most of the associations between distress and cancer were largely unaffected (fig A in appendix 2)", which presents it as a check rather than the source of the headline figure. (d) 1.32 (1.18 to 1.48) matches the Table 5 age and sex row in all three numbers, and the Results text and Figures 2 and 3 label it age and sex adjusted. Another model reproducing all three numbers to two decimals would be a coincidence. A related point: the abstract says all seven estimates take left censoring and imputation into account, but the Table 5 footnote, which defines the rows six of them come from, lists covariates only.
- A model with additional covariates (appendix 1 table B). I could not inspect that table. The Results say only that, in this analysis, "The strength of the relation between distress and total cancer was little changed", and give no figure for it. The abstract's covariate list is close to the main model (Table 5 footnote, Methods, Figure 3 legend) but not identical: the abstract names "socioeconomic status" as well as education, while those three places list educational attainment only. The Results describe the main model's covariates as including "socioeconomic position (education)", so the abstract's "socioeconomic status" corresponds to education in the main model. Table B's extra covariates do include one socioeconomic measure, area based deprivation, but the abstract does not mention the other three (physical activity, C reactive protein, quantity of alcohol consumed), and the six other estimates in the same sentence equal the main-model rows of Table 5.
- The same-sample re-analysis reported in the Results. The paper says it "recomputed these effects estimates in a non-missing dataset—that is, the same sample size in both models—and our results were unchanged". This does not account for 1.32 being labelled multivariable. In that analysis the multivariable model is fitted to the complete-case sample, and the multivariable estimate in the complete-case sample is the 1.26 (1.11 to 1.42) already shown in Table 5 (3800 deaths, n=146 008). Refitting in the complete-case sample changes the age and sex estimate, not the multivariable one, so the re-analysis gives no route by which the multivariable estimate becomes 1.32 (1.18 to 1.48), the numbers of the full-sample age and sex row.
- Rounding, or two presentations of one estimate. Ruled out. The two CIs have log-scale midpoints of 1.322 and 1.255, each matching its own printed hazard ratio, and they come from different samples (4353 deaths in 163 363 people v 3800 deaths in 146 008).
- The error is in Table 5, not in the abstract. Not supported. The Table 5 all-cancer rows agree with the Results text, with Figure 2 (pooled age and sex estimate 1.32 (1.18 to 1.48), 4353 deaths) and with Figure 3 (1.32 under age and sex, 1.26 under multivariable).
- An artefact of the PMC text conversion. Ruled out. The same abstract and Discussion wording and the same Table 5 cells appear in the publisher's typeset PDF (see Status). No minus signs, superscripts or special decimal marks are involved.
Status
- Version of record. I checked the PMC archived text (PMC5266623, via
src/show.py) and the publisher's typeset PDF ("the bmj | BMJ 2017;356:j108", 11 pages, created 19 January 2017), held in the UCL Discovery repository (eprint 1537638). Their abstract and Discussion wording and their Table 5 cells are identical. Text extracted from the PDF's Figure 2 and Figure 3 gives the values described above. Every quotation in this write-up was confirmed verbatim against the PMC text. The live bmj.com page returned HTTP 403 to automated access in an earlier check in this pipeline, which also found the same abstract sentence in Europe PMC full text and in PubMed (PMID 28122812). - Supplements. The web appendices (appendix 1 tables A-D; appendix 2 fig A) are not deposited in PMC or Europe PMC, and bmj.com was not reachable, so they were not inspected. The Europe PMC supplementary bundle holds only the four main figures.
- Corrections. As of 2026-10-23, I found no correction, erratum, retraction or expression of concern. Crossref shows no
update-to/updated-byrelation for 10.1136/bmj.j108, and a Crossref query for works updating this DOI returns 0 results.src/known.pylists no Crossref, PMC or Europe PMC notices. Europe PMC's comment/correction list for PMID 28122812 has two "Comment in" entries (Cancer 2017;123(13):2389 and MMW Fortschr Med 2017;159(15):38) and no correction. An earlier web search in this pipeline found no erratum. I could not repeat the web search in this session because the search budget was used up. - PubPeer. The PubPeer publications API, queried for this DOI on 2026-10-23, returned no feedback (
"feedbacks":[]). - Separate minor observation (not part of this finding). For liver cancer, the multivariable estimate is printed as 4.24 (0.66 to 27.0) in Table 5 but as 4.24 (0.66 to 8.52) in the Figure 3 text of the typeset PDF. The log-midpoint of 0.66 to 27.0 is 4.22, consistent with 4.24. The log-midpoint of 0.66 to 8.52 is 2.37, which is not. 8.52 is the upper limit of the liver age and sex CI.
Suggested correction
- In the abstract's Results, replace "cancer of all sites combined (multivariable adjusted hazard ratio 1.32, 95% confidence interval 1.18 to 1.48)" with "cancer of all sites combined (multivariable adjusted hazard ratio 1.26, 95% confidence interval 1.11 to 1.42)". Alternatively, keep 1.32 (1.18 to 1.48) and label it "age and sex adjusted", which would also need the sentence's opening clause to be reworded.
- Make the same change in the Discussion, "Comparison with other studies", and reconsider describing the earlier meta-analysis estimate, 1.12 (0.99 to 1.26), as "of markedly lower magnitude".
- Clarify in the abstract whether the reported estimates are from the main (complete-case, not left-censored) analysis, as Table 5 and the Results indicate, rather than "with reverse causality (by left censoring) and missing data (by imputation) taken into account".
- Separately, reconcile the liver multivariable upper confidence limit in Figure 3 (8.52) with Table 5 (27.0).
Recomputation script (python recompute.py.txt)
"""Recompute the all-cancer hazard ratios of PMC5266623
(BMJ 2017;356:j108, doi:10.1136/bmj.j108).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PubMed Central:
Abstract, Results: seven hazard ratios (GHQ-12 score 7-12 v 0-6), all introduced as
adjusted for age, sex, education, SES, BMI, smoking and alcohol
Table 5, last column "Distress score (7-12 v 0-6)": the "Age and sex" (or "Age") row and the
"Multivariable" row for each outcome, with deaths and number of people
Discussion, "Comparison with other studies": prior meta-analysis RR 1.12 (0.99 to 1.26)
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from scipy.stats import norm # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
# Table 5: outcome -> (minimal row (HR, lo, hi), multivariable row (HR, lo, hi))
# For prostate the minimal row is labelled "Age" (single-sex outcome).
T5 = {
"All cancers": ((1.32, 1.18, 1.48), (1.26, 1.11, 1.42)),
"Not related to smoking": ((1.38, 1.19, 1.60), (1.45, 1.23, 1.71)),
"Colorectal": ((1.75, 1.21, 2.53), (1.84, 1.21, 2.78)),
"Prostate": ((2.29, 1.36, 3.86), (2.42, 1.29, 4.54)),
"Pancreas": ((2.52, 1.47, 4.32), (2.76, 1.47, 5.19)),
"Oesophagus": ((2.42, 1.36, 4.32), (2.59, 1.34, 5.00)),
"Leukaemia": ((2.89, 1.32, 6.31), (3.86, 1.42, 10.5)),
}
# Table 5, All cancers: deaths and number of people in each model
T5_ALL_N = {"age_sex": (4353, 163363), "multivariable": (3800, 146008)}
# Abstract, Results (the first is explicitly "multivariable adjusted hazard ratio")
ABSTRACT = {
"All cancers": (1.32, 1.18, 1.48),
"Not related to smoking": (1.45, 1.23, 1.71),
"Colorectal": (1.84, 1.21, 2.78),
"Prostate": (2.42, 1.29, 4.54),
"Pancreas": (2.76, 1.47, 5.19),
"Oesophagus": (2.59, 1.34, 5.00),
"Leukaemia": (3.86, 1.42, 10.5),
}
# Discussion, "Comparison with other studies": the same all-cancer figure, "multivariable adjusted"
DISCUSSION_ALL = (1.32, 1.18, 1.48)
# Results p4: "after adjustment for age and sex ... (hazard ratio 1.32, 95% CI 1.18 to 1.48)"
RESULTS_P4_AGESEX = (1.32, 1.18, 1.48)
# Discussion: prior meta-analysis (depression and cancer incidence), "relative risk 1.12, 0.99 to 1.26"
PRIOR_META = (1.12, 0.99, 1.26)
def log_se(t):
_, lo, hi = t
return (math.log(hi) - math.log(lo)) / (2 * Z)
def midpoint(t):
_, lo, hi = t
return math.exp((math.log(lo) + math.log(hi)) / 2)
def num(x):
return f"{x:.1f}" if x >= 10 else f"{x:.2f}"
def fmt(t):
return f"{num(t[0])} ({num(t[1])} to {num(t[2])})"
print("=== 1. Which Table 5 row does each abstract hazard ratio come from? ===")
print(f"{'outcome':24s} {'abstract':20s} {'T5 age(/sex)':20s} {'T5 multivariable':20s} matches")
MATCH = {}
for k, v in ABSTRACT.items():
a, m = T5[k]
tag = "multivariable row" if v == m else ("AGE AND SEX ROW" if v == a else "neither")
MATCH[k] = tag
print(f"{k:24s} {fmt(v):20s} {fmt(a):20s} {fmt(m):20s} {tag}")
n_mv = sum(t == "multivariable row" for t in MATCH.values())
print(f"-> {n_mv} of {len(ABSTRACT)} abstract estimates equal the multivariable row; "
f"the all-cancer estimate equals the age and sex row.")
print("Change from age(/sex) to multivariable estimate, by outcome:")
for k in ABSTRACT:
a, m = T5[k]
print(f" {k:24s} {a[0]:.2f} -> {m[0]:.2f} ({'up' if m[0] > a[0] else 'down'})")
print("Discussion figure equals Table 5 age and sex row:", DISCUSSION_ALL == T5["All cancers"][0])
print("Results p4 (age and sex) figure equals Table 5 age and sex row:",
RESULTS_P4_AGESEX == T5["All cancers"][0])
print("\n=== 2. Are the two all-cancer rows separate, internally consistent estimates? ===")
AS, MV = T5["All cancers"]
for lab, t in (("age and sex", AS), ("multivariable", MV)):
se = log_se(t)
zval = math.log(t[0]) / se
p = 2 * norm.sf(abs(zval))
print(f"{lab:14s} HR {t[0]:.2f}: log SE {se:.4f}; geometric midpoint of CI {midpoint(t):.3f}; "
f"z = {zval:.2f}; two-sided P ~ {p:.1e}")
print(f"deaths/people: age and sex {T5_ALL_N['age_sex']}, multivariable {T5_ALL_N['multivariable']}; "
f"difference {T5_ALL_N['age_sex'][0] - T5_ALL_N['multivariable'][0]} deaths, "
f"{T5_ALL_N['age_sex'][1] - T5_ALL_N['multivariable'][1]} people")
print("\n=== 3. Size of the overstatement ===")
ex_rep, ex_mv = AS[0] - 1, MV[0] - 1
print(f"excess risk: reported {ex_rep*100:.0f}% v multivariable {ex_mv*100:.0f}%; "
f"reported excess is {ex_rep/ex_mv:.2f} x the multivariable excess; "
f"multivariable excess is {(1 - ex_mv/ex_rep)*100:.0f}% smaller")
print(f"ratio of hazard ratios 1.32/1.26 = {AS[0]/MV[0]:.3f}")
print(f"lower CI bound: {AS[1]:.2f} v {MV[1]:.2f}; upper: {AS[2]:.2f} v {MV[2]:.2f}")
print("\n=== 4. Discussion contrast with the prior meta-analysis (illustrative only) ===")
# Naive two-sample z test on the log scale, treating the estimates as independent. The two
# studies differ in exposure (distress v major depression) and outcome (mortality v incidence);
# this only shows how much the contrast depends on which all-cancer figure is used.
se_prior = log_se(PRIOR_META)
print(f"prior meta-analysis RR 1.12: log SE {se_prior:.4f}; excess {(PRIOR_META[0]-1)*100:.0f}%")
for lab, t in (("as reported (1.32)", AS), ("multivariable (1.26)", MV)):
d = math.log(t[0]) - math.log(PRIOR_META[0])
se = math.sqrt(log_se(t) ** 2 + se_prior ** 2)
zval = d / se
print(f"{lab:22s}: ratio to 1.12 = {t[0]/PRIOR_META[0]:.3f}; excess {(t[0]-1)*100:.0f}% v 12% "
f"({(t[0]-1)/(PRIOR_META[0]-1):.1f} x); z for difference = {zval:.2f}, "
f"P = {2*norm.sf(abs(zval)):.3f}; CIs overlap from {max(t[1], PRIOR_META[1]):.2f} "
f"to {min(t[2], PRIOR_META[2]):.2f}")
print("\n=== 5. Separate minor observation: liver, multivariable (Table 5 v typeset Figure 3) ===")
LIVER_T5_MV = (4.24, 0.66, 27.0) # Table 5 (PMC and typeset PDF)
LIVER_F3_MV = (4.24, 0.66, 8.52) # Figure 3 text in the typeset PDF
LIVER_T5_AS = (2.88, 0.97, 8.52) # Table 5 age and sex row
for lab, t in (("Table 5 multivariable", LIVER_T5_MV), ("Figure 3 multivariable", LIVER_F3_MV),
("Table 5 age and sex", LIVER_T5_AS)):
print(f"{lab:24s} {fmt(t):22s} geometric midpoint of CI {midpoint(t):.2f} (printed HR {t[0]})")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8d8c88"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": "#c9c8c4",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# ---- Figure 1: abstract/discussion label v Table 5 rows
fig = plt.figure(figsize=(7.2, 4.7))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.7)
ax.axis("off")
def panel(x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.14, y + h - 0.18, title, fontsize=11, fontweight="bold", va="top", color=TXT)
# Left panel: abstract and discussion
panel(0.15, 1.2, 2.95, 3.35, "Abstract, Results\n(repeated in the Discussion)", ORANGE)
ax.text(0.29, 3.62,
"\u201cAfter adjustment for age, sex,\neducation, socioeconomic status,\nbody mass index (BMI), and\n"
"smoking and alcohol intake \u2026\ncancer of all sites combined\n(multivariable adjusted hazard\nratio",
fontsize=10, va="top", color=SEC, linespacing=1.3)
ax.text(0.29, 1.88, f"{AS[0]:.2f}, 95% CI {AS[1]:.2f} to {AS[2]:.2f})\u201d",
fontsize=11.5, fontweight="bold", color=ORANGE, va="center")
ax.text(0.29, 1.5, "label: multivariable adjusted", fontsize=10, color=SEC, va="center", style="italic")
# Right panel: Table 5 rows
panel(4.0, 1.2, 3.05, 3.35, "Table 5, All cancers\n(score 7-12 v 0-6)", "#9b9a96")
rows = [
(3.6, "Age and sex", f"{T5_ALL_N['age_sex'][0]} deaths, n = {T5_ALL_N['age_sex'][1]:,}".replace(",", " "),
fmt(AS), ORANGE),
(2.6, "Multivariable\u2020", f"{T5_ALL_N['multivariable'][0]} deaths, n = {T5_ALL_N['multivariable'][1]:,}"
.replace(",", " "), fmt(MV), BLUE),
]
for y, lab, nn, val, col in rows:
ax.text(4.14, y, lab, fontsize=10.5, fontweight="bold", va="center", color=TXT)
ax.text(4.14, y - 0.27, nn, fontsize=10, va="center", color=SEC)
ax.text(4.14, y - 0.57, val, fontsize=11.5, fontweight="bold", va="center", color=col)
ax.text(4.14, 1.5, "\u2020 age, sex, BMI, education,\n smoking, alcohol", fontsize=10, color=SEC, va="center",
linespacing=1.25)
# Connectors
ax.add_patch(FancyArrowPatch((3.14, 1.95), (3.98, 3.03), arrowstyle="-|>", mutation_scale=14,
color=ORANGE, lw=2, connectionstyle="arc3,rad=-0.15"))
ax.text(3.52, 3.1, "identical\nvalue", fontsize=10, color=ORANGE, ha="center", va="center")
ax.add_patch(FancyArrowPatch((3.14, 1.5), (3.98, 2.03), arrowstyle="-|>", mutation_scale=14,
color=BLUE, lw=2, linestyle=(0, (4, 3))))
ax.text(3.55, 1.38, "row the\nlabel names", fontsize=10, color=BLUE, ha="center", va="top")
ax.text(0.15, 0.55,
"The Results text gives the same 1.32 (1.18 to 1.48) \u201cafter adjustment for age and sex\u201d.\n"
f"The other {n_mv} hazard ratios in the abstract all equal Table 5 multivariable rows (fig 2).",
fontsize=10, color=SEC, va="center", linespacing=1.35)
save(fig, "fig1")
# ---- Figure 2: all seven abstract estimates against both Table 5 rows (log scale)
names = list(ABSTRACT)
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.subplots_adjust(left=0.25, right=0.76, top=0.86, bottom=0.14)
off = {"abstract": 0.0, "mv": -0.24, "as": 0.24}
for i, k in enumerate(names):
y = -i
a, m = T5[k]
v = ABSTRACT[k]
for t, dy, col, mk, fc in ((a, off["as"], GRAY, "o", BG), (m, off["mv"], BLUE, "D", BLUE),
(v, off["abstract"], ORANGE, "o", ORANGE)):
ax.plot([t[1], t[2]], [y + dy] * 2, color=col, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(t[0], y + dy, marker=mk, ms=8, mfc=fc, mec=col, mew=2, zorder=3, ls="none")
tag = "= age and sex row" if MATCH[k] != "multivariable row" else "= multivariable row"
ax.text(1.02, y, tag, fontsize=10, va="center", ha="left", transform=ax.get_yaxis_transform(),
color=ORANGE if MATCH[k] != "multivariable row" else SEC,
fontweight="bold" if MATCH[k] != "multivariable row" else "normal")
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.9, 12)
ticks = [1, 1.5, 2, 3, 5, 10]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([-i for i in range(len(names))])
ax.set_yticklabels(names)
ax.set_ylim(-len(names) + 0.5, 0.5)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Hazard ratio, distress score 7-12 v 0-6 (log scale; 95% CI)")
h = [plt.Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Abstract (“multivariable adjusted”)"),
plt.Line2D([], [], color=BLUE, marker="D", ms=8, lw=2, label="Table 5, multivariable row"),
plt.Line2D([], [], color=GRAY, marker="o", ms=8, mfc=BG, mew=2, lw=2, label="Table 5, age and sex row")]
fig.legend(handles=h, loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 1.0))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Primary-outcome 95% confidence interval is about 80 times too narrow for the reported failure counts
BMJ Glob Health (2024) · doi:10.1136/bmjgh-2023-014170 · PMC11331853
Summary
This non-inferiority trial compared spinal anaesthesia failure between trained medical officers (MOs) and consultant anaesthesiologists (CAs). It reports 7/231 (3.0%) versus 5/191 (2.6%) failures, a difference of 0.4 percentage points, with a 95% CI of 0.36–0.43% in the Abstract and Results and 0.36 to 0.44 in Table 3. From those counts, the 95% CI for the difference is −2.75 to 3.58 percentage points (Wald) or −3.29 to 3.85 (Newcombe). The printed interval is therefore about 79–90 times too narrow, and it excludes zero although the printed p = 0.7997, which the same counts reproduce exactly. The non-inferiority conclusion survives, because every recomputed upper bound stays below the 5-point margin. But the margin is cleared by about 1.2–1.4 points, not the 4.6 points the printed interval implies, and the emergent-case subgroup interval has the same defect.
What the paper reports
Abstract, Findings:
Spinal anaesthesia failure rate for MOs (7, 3.0%) was non-inferior to those of CA (5, 2.6%); difference in success rate of 0.4% (95% CI=0.36–0.43%; p=0.80).
Results, third paragraph (primary outcome):
There was no statistically significant difference in the primary outcome–delivery of spinal anaesthesia into the intrathecal space in three or fewer attempts (table 3). MOs were unsuccessful in 7 (3.0%) patients, while the CAs were unsuccessful in 5 (2.6%) patients, with a difference in success rate of 0.4% (CI=0.36–0.43%; p=0.7997).
Table 3, "Primary outcome measure: failure to deliver spinal anaesthesia". The columns are: outcome | N | Medical officers (n=231) | Consultant anaesthesiologists (n=191) | % between-group difference (95% CI) | P value. The footer reads "*Fisher’s exact test, otherwise χ2 test."
Failure to deliver spinal anaesthesia in all cases | 422 | 7 (3.0%) | 5 (2.6%) | 0.4 (0.36 to 0.44) | 0.7997
Failure to deliver spinal anaesthesia in emergent cases | 61 | 1 (3.6%) | 2 (6.1%) | -2.5 (-2.92 to -2.08) | 1.0000*
Failure due to inaccessibility of intrathecal space | 422 | 7 (3.0%) | 5 (2.6%) | 0.4 (0.36 to 0.44) | 0.7997
Table 2 gives the emergent-case denominators (columns: MO | CA | P value):
Emergency | 28 (12.2%) | 33 (17.3%) | 0.1435
Abstract, Methods, which defines the margin:
The primary outcome measure was spinal anaesthesia failure with a non-inferiority margin of 5%.
Methods, Statistical analysis, which names the analysis methods:
Patient characteristics, spinal anaesthesia failure rates and incidence of postoperative complications were analysed with T-test and Wilcoxon rank sum for continuous variables, and χ squared and Fisher’s exact tests for discrete variables.
The conclusions that rest on this result. Abstract, Interpretation:
This study demonstrates that failure rates of spinal anaesthesia care provided by trained MOs are non-inferior to care provided by CAs in low-risk surgical patients. This may support policy measures that use task-sharing as a means of expanding anaesthesia care capacity in rural Indian hospitals.
Discussion, fourth paragraph:
We believe this study may facilitate regional and national policy measures on the practical viability of task-sharing.
What the paper's own numbers show
Method. I took the failure counts and arm sizes from Table 3 (7/231 and 5/191) and the emergent-case denominators from Table 2 (28 and 33). Those emergent denominators are the only pair summing to 61 that reproduces the printed 3.6% and 6.1%. For each comparison I computed the difference in failure proportions (MO − CA) and its unpooled standard error, then the 95% CI by three standard methods for a difference of two independent proportions: Wald, Newcombe (hybrid Wilson score) and Agresti–Caffo. I also computed the uncorrected and Yates-corrected chi-square and the Fisher exact p-values, to check that the counts match the printed p-values. From the half-width of each printed interval I derived the standard error it implies under a normal approximation, and the p-value that standard error would imply. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| MO failure rate, all cases | Abstract; Results; Table 3 | 7 (3.0%), n = 231 | 3.03% |
| CA failure rate, all cases | Abstract; Results; Table 3 | 5 (2.6%), n = 191 | 2.62% |
| Difference, MO − CA | Abstract; Results; Table 3 | 0.4 | 0.413 percentage points (pp) |
| p-value, all cases | Table 3; Results (0.80 in Abstract) | 0.7997 (χ²) | 0.7997, uncorrected chi-square (Yates 1.0000; Fisher 1.0000) |
| 95% CI of difference, all cases | Abstract; Results | 0.36–0.43% | Wald −2.75 to 3.58 pp; Newcombe −3.29 to 3.85; Agresti–Caffo −3.06 to 3.71 |
| 95% CI of difference, all cases | Table 3, rows 1 and 3 | 0.36 to 0.44 | as above |
| Width of the 95% CI, all cases | Table 3; Abstract | 0.08 (Table 3); 0.07 (Abstract) | 6.33 pp (Wald): 79 times the Table 3 width and 90 times the Abstract width |
| Standard error, all cases | not reported | 0.0204 pp, implied by the Table 3 interval | 1.615 pp (Wald), 79 times larger |
| p-value implied by the printed interval, all cases | derived | z = 0.40 / 0.0204 = 19.6 | p ≈ 1.6 × 10⁻⁸⁵, against the printed 0.7997 |
| Emergent cases, rates and difference | Table 3, row 2; Table 2 | 1 (3.6%) vs 2 (6.1%); −2.5 | 1/28 = 3.57%, 2/33 = 6.06%; −2.49 pp |
| 95% CI, emergent cases | Table 3, row 2 | −2.92 to −2.08 | Wald −13.14 to 8.17 pp; Newcombe −16.35 to 12.32; width 21.31 pp, 25 times the printed 0.84 |
| p-value, emergent cases | Table 3, row 2 | 1.0000 (Fisher) | Fisher 1.0000; the printed interval would imply p ≈ 1.9 × 10⁻³¹ |
| Upper 95% bound vs the 5 pp margin, all cases | Abstract; Methods | 0.43 or 0.44, leaving about 4.6 pp | 3.58 (Wald), 3.71 (Agresti–Caffo), 3.85 (Newcombe), leaving 1.42, 1.29 and 1.15 pp; non-inferiority still holds |
| Upper bound of a two-sided 90% interval (one-sided 5% test) | derived | not reported | 3.07 pp, leaving 1.93 pp |
The counts and the p-value agree exactly, so the counts are not in doubt. Only the intervals are inconsistent with them. Both printed all-cases intervals lie entirely above zero, which would mean a significant difference in favour of CAs, while the stated p = 0.80 and the text's "no statistically significant difference" say the opposite. The emergent-case interval lies entirely below zero, which would mean MOs had significantly fewer failures, while its printed p is 1.0000.
Figure
Each row shows a point estimate and its 95% confidence interval for the difference in failure rate between medical officers and consultant anaesthesiologists (MO − CA, percentage points), on a single linear axis. Orange rows are the paper's printed values: 0.4 (0.36 to 0.43) in the Abstract and Results text, 0.4 (0.36 to 0.44) in Table 3, and −2.5 (−2.92 to −2.08) for emergent cases. All three printed intervals are narrower than the plotted dot. Blue rows are recomputed from the paper's own counts (7/231 vs 5/191; 1/28 vs 2/33) with the Wald and Newcombe methods. The thin solid line marks no difference (0), and the dashed line marks the 5-point non-inferiority margin from the Methods. The recomputed all-cases intervals span zero, as the printed p = 0.7997 requires, and still end below the margin. The recomputed emergent-case interval is roughly 21 points wide and crosses both zero and the margin.
Why it matters
The confidence interval's upper bound is the statistic on which a non-inferiority trial's conclusion rests, and this one appears in the Abstract, the Results text and Table 3. As printed, it suggests that the difference in failure rates is known to within about ±0.04 percentage points. The counts support only about ±3.2 points, so the data are compatible with MOs having a failure rate up to about 3.6–3.9 points higher than CAs. Non-inferiority at the 5-point margin still holds under every standard method I checked. But the evidence is much less decisive than the printed interval suggests, and a reader using the printed interval would overstate it. That matters because the Interpretation and Discussion present the trial as support for regional and national task-sharing policy.
The printed intervals also contradict the paper's own tests. An interval that excludes zero cannot accompany p = 0.80, so a reader cannot tell from the paper which of the two to believe without recomputing. In the emergent-case subgroup, the printed interval implies that MOs were reliably better in emergencies. Recomputed, the subgroup (61 patients, 3 failures) is uninformative in either direction. Any later meta-analysis or sample-size calculation that took the printed interval or its implied standard error at face value would understate the uncertainty about 80-fold.
Explanations considered
- Different population or denominator. This does not account for the numbers. The Abstract, the Results text and Table 3 rows 1 and 3 all use the same 422 patients and the same 7/231 versus 5/191. The emergent denominators (28 and 33) come from Table 2 and reproduce the printed percentages.
- Scale: proportion versus percentage points. This does not account for the numbers. The estimate 0.4 is in percentage points (0.004 as a proportion). On the proportion scale the Wald interval is −0.028 to 0.036, and on the percentage-point scale it is −2.75 to 3.58. Neither resembles 0.36 to 0.44.
- Ratio rather than difference. This does not account for the numbers. A risk-ratio interval would be centred near 1.16 and contain 1 (0.37 to 3.59), not sit around 0.4. The column heading reads "% between-group difference (95% CI)".
- One-sided or 90% interval. This does not account for the numbers. The upper bound of a two-sided 90% interval (equivalent to a one-sided 5% test) is 3.07 pp, still about seven times the printed 0.44.
- Choice of interval method. This does not account for the numbers. Wald, Newcombe and Agresti–Caffo intervals all span zero and run from about −3 to +4 pp. The paper does not name its interval method.
- Adjusted or stratified analysis. This does not account for the numbers. The Statistical analysis section lists only t-tests, Wilcoxon rank-sum, chi-square and Fisher's exact tests, with no regression, stratification or clustering, and none of these would shrink a standard error about 79-fold. The printed p = 0.7997 matches an unadjusted, uncorrected chi-square on these counts to four decimals, which confirms that the unadjusted 2 × 2 table was analysed.
- Rounding or typesetting. This does not account for the numbers. The interval appears three times in two forms (0.36–0.43% and 0.36 to 0.44), with ordinary decimal points, and the emergent row keeps its minus signs. A width error of this size cannot come from rounding.
- A possible arithmetic route (diagnostic only). If the estimate is kept in percentage points but the standard error is used as a proportion, 0.413 ± 1.96 × 0.0161 gives 0.38 to 0.44, and 0.4 ± 1.96 × 0.0161 gives 0.37 to 0.43. These are close to the printed all-cases intervals. The same route applied to the emergent row gives −2.60 to −2.38, not the printed −2.92 to −2.08. So this route does not explain every printed interval, and the actual computation cannot be identified without the analysis code. Either way, the printed intervals cannot be reconciled with the printed counts.
- Wording "difference in success rate". This is a separate, minor point. The Abstract and Results call 0.4 a "difference in success rate", but 3.0% − 2.6% is a difference in failure rates. A difference in success rates (MO − CA) would be −0.4. This affects the sign convention, not the width problem.
Status
- Version of record. The publisher's pages (gh.bmj.com, reached from the DOI, including the full-text, PDF and info pages) returned HTTP 403 to automated access, and the PMC HTML page showed only a CAPTCHA. I therefore checked the publisher-deposited JATS XML from Europe PMC against the PMC archive used here. The two agree word for word at every quoted place: the Abstract's "0.36–0.43%", the Results' "(CI=0.36–0.43%; p=0.7997)", and Table 3's three rows, each value in its own table cell. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements and data. The JATS XML lists no supplementary material, and the Europe PMC supplementary-files bundle holds only Figure 1. The data availability statement offers de-identified data on request, so no public dataset exists. The ClinicalTrials.gov record NCT04438811 is marked completed with no posted results. No source was available to reconstruct the authors' interval method, and none is needed, because the counts in Table 3 are enough to show that the printed intervals are impossible.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment links (PMID 39153752). A web search for the title with "correction" returned only the article and index listings. The PubPeer API request for the DOI was blocked by a Cloudflare challenge, so I could not determine whether PubPeer has comments on this article.
- Sources. PMC article PMC11331853; Europe PMC full-text XML for PMC11331853; Europe PMC supplementary-files endpoint; ClinicalTrials.gov NCT04438811; Crossref; DOI landing page https://doi.org/10.1136/bmjgh-2023-014170 (publisher page not reachable).
Suggested correction
- Replace the 95% CI for the all-cases difference in the Abstract, the Results text and Table 3 (rows 1 and 3) with an interval computed from 7/231 versus 5/191. For example, the Wald interval is −2.75 to 3.58 percentage points and the Newcombe interval is −3.29 to 3.85. State which method is used.
- Replace the emergent-case interval in Table 3 (row 2), for example with the Wald interval −13.1 to 8.2 or the Newcombe interval −16.4 to 12.3 percentage points.
- Report the non-inferiority comparison explicitly, as the corrected upper bound against the 5-point margin. State whether the test is one-sided at 2.5% (95% CI) or at 5% (90% CI, upper bound 3.07).
- Make the Abstract, the Results text and Table 3 agree. They currently give 0.43 and 0.44 as the upper bound.
- Describe 0.4 as a difference in failure rates (MO − CA), or report −0.4 if it is meant as a difference in success rates.
- Review the wording of the Interpretation and Discussion in light of the wider interval: non-inferiority holds, but with about 1.2–1.4 points of headroom rather than about 4.6.
Recomputation script (python recompute.py.txt)
"""Recompute the primary-outcome confidence intervals of PMC11331853
(BMJ Glob Health 2024, doi:10.1136/bmjgh-2023-014170) from the paper's own counts,
and regenerate the figures in this script's folder.
Run: python recompute.py.txt
Needs numpy, scipy, matplotlib.
Inputs, all from the paper (Table 2, Table 3, abstract, Results):
all cases: MO 7/231 failures, CA 5/191 failures
emergent cases: MO 1/28, CA 2/33 (denominators: Table 2 row "Emergency", 28 | 33;
they also reproduce the printed 3.6% and 6.1%)
printed CIs: 0.36-0.43 (abstract, Results), 0.36-0.44 (Table 3), -2.92 to -2.08 (Table 3 emergent)
NI margin: 5 percentage points (Methods)
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = stats.norm.ppf(0.975) # 1.959964
def wilson(x, n, z=Z):
p = x / n; den = 1 + z * z / n
c = (p + z * z / (2 * n)) / den
h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
return c - h, c + h
def analyse(a, n1, b, n2, label):
p1, p2 = a / n1, b / n2; d = p1 - p2
se = np.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
r = dict(p1=p1, p2=p2, d=d, se=se)
r["wald"] = (d - Z * se, d + Z * se)
r["wald90"] = (d - 1.645 * se, d + 1.645 * se)
l1, u1 = wilson(a, n1); l2, u2 = wilson(b, n2)
r["newcombe"] = (d - np.sqrt((p1 - l1) ** 2 + (u2 - p2) ** 2), d + np.sqrt((u1 - p1) ** 2 + (p2 - l2) ** 2))
q1, q2 = (a + 1) / (n1 + 2), (b + 1) / (n2 + 2)
seac = np.sqrt(q1 * (1 - q1) / (n1 + 2) + q2 * (1 - q2) / (n2 + 2))
r["agresti_caffo"] = (q1 - q2 - Z * seac, q1 - q2 + Z * seac)
t = np.array([[a, n1 - a], [b, n2 - b]])
r["p_chi2"] = stats.chi2_contingency(t, correction=False)[1]
r["p_yates"] = stats.chi2_contingency(t, correction=True)[1]
r["p_fisher"] = stats.fisher_exact(t)[1]
print(f"--- {label}")
print(f" MO {a}/{n1} = {100*p1:.2f}% CA {b}/{n2} = {100*p2:.2f}% difference MO-CA = {100*d:.3f} pp")
print(f" Wald SE = {100*se:.3f} pp")
for k in ("wald", "newcombe", "agresti_caffo", "wald90"):
lo, hi = r[k]; print(f" {k:14s} CI: {100*lo:6.2f} to {100*hi:5.2f} pp (width {100*(hi-lo):.2f} pp)")
print(f" p: uncorrected chi-square {r['p_chi2']:.4f}; Yates {r['p_yates']:.4f}; Fisher {r['p_fisher']:.4f}")
return r
print("Emergent denominators consistent with 3.6% and 6.1% and N=61:",
[(n1, 61 - n1) for n1 in range(10, 52) if round(100 / n1, 1) == 3.6 and round(200 / (61 - n1), 1) == 6.1])
A = analyse(7, 231, 5, 191, "All cases (Table 3 rows 1 and 3)")
E = analyse(1, 28, 2, 33, "Emergent cases (Table 3 row 2)")
print("\n--- Printed intervals")
printed = {"abstract": (0.40, 0.36, 0.43), "table": (0.40, 0.36, 0.44), "emergent": (-2.5, -2.92, -2.08)}
for k, (est, lo, hi) in printed.items():
se_imp = (hi - lo) / (2 * Z)
print(f" {k:9s}: {est} ({lo} to {hi}); width {hi-lo:.2f} pp; implied SE {se_imp:.4f} pp")
wa = 100 * (A["wald"][1] - A["wald"][0]); we = 100 * (E["wald"][1] - E["wald"][0])
print(f" all cases: Wald width {wa:.2f} pp = {wa/0.08:.0f} x Table 3 width, {wa/0.07:.0f} x abstract width")
print(f" all cases: Wald SE {100*A['se']:.3f} pp = {100*A['se']/((0.44-0.36)/(2*Z)):.0f} x SE implied by Table 3 interval")
print(f" emergent: Wald width {we:.2f} pp = {we/0.84:.0f} x printed width")
# p-value that the printed Table 3 interval would imply under a normal approximation
se_t = (0.44 - 0.36) / (2 * Z); z_t = 0.40 / se_t
p_implied = 2 * stats.norm.sf(z_t)
print(f" Table 3 interval implies z = {z_t:.1f}, two-sided p = {p_implied:.1e} (printed p = 0.7997)")
se_e = (-2.08 + 2.92) / (2 * Z); z_e = 2.5 / se_e
p_implied_e = 2 * stats.norm.sf(z_e)
print(f" emergent interval implies z = {z_e:.1f}, two-sided p = {p_implied_e:.1e} (printed p = 1.0000)")
print("\n--- Arithmetic route that reproduces the all-cases interval (diagnostic only)")
d_pp = 100 * A["d"]
print(f" {d_pp:.3f} pp +/- 1.96 x {A['se']:.4f} (SE left in proportion units) = {d_pp - Z*A['se']:.3f} to {d_pp + Z*A['se']:.3f}")
print(f" 0.4 pp +/- 1.96 x {A['se']:.4f} = {0.4 - Z*A['se']:.3f} to {0.4 + Z*A['se']:.3f}")
de_pp = 100 * E["d"]
print(f" emergent: {de_pp:.3f} +/- 1.96 x {E['se']:.4f} = {de_pp - Z*E['se']:.3f} to {de_pp + Z*E['se']:.3f} (printed -2.92 to -2.08: not reproduced)")
print("\n--- Other scales and summaries (Explanations considered)")
print(f" Wald CI on the proportion scale: {A['wald'][0]:.3f} to {A['wald'][1]:.3f}")
rr = A["p1"] / A["p2"]; se_lrr = np.sqrt(1/7 - 1/231 + 1/5 - 1/191)
print(f" risk ratio {rr:.2f}, 95% CI {rr*np.exp(-Z*se_lrr):.2f} to {rr*np.exp(Z*se_lrr):.2f}")
print(f" half-widths: printed Table 3 {(0.44-0.36)/2:.2f} pp; Wald {Z*100*A['se']:.2f} pp")
print(f" 90% upper bound {100*A['wald90'][1]:.2f} pp = {100*A['wald90'][1]/0.44:.1f} x the printed 0.44")
print("\n--- Non-inferiority margin (5 pp)")
for k in ("wald", "newcombe", "agresti_caffo", "wald90"):
hi = 100 * A[k][1]; print(f" {k:14s} upper bound {hi:.2f} pp -> headroom {5 - hi:.2f} pp; below margin: {hi < 5}")
for k in ("abstract", "table"):
print(f" printed {k:8s} upper bound {printed[k][2]} pp -> headroom {5 - printed[k][2]:.2f} pp")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG})
def save(fig, path):
fig.savefig(path + ".svg"); fig.savefig(path + ".png", dpi=200); plt.close(fig)
def style(ax):
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def interval_rows(ax, rows):
for i, (lab, est, lo, hi, col, txt) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot([lo, lo], [y - .12, y + .12], color=col, lw=2); ax.plot([hi, hi], [y - .12, y + .12], color=col, lw=2)
ax.plot(est, y, "o", ms=8, color=col, mec=BG, mew=1.5, zorder=3)
if txt:
ax.text(1.02, y, txt, va="center", fontsize=10, color=INK2, transform=ax.get_yaxis_transform())
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows][::-1])
ax.set_ylim(-0.7, len(rows) - 0.3)
def pp(x): return 100 * x
# Candidate A (default fig1): every printed vs recomputed interval on one axis
def cand_a(path):
fig, ax = plt.subplots(figsize=(7.2, 4.6))
rows = [
("All cases, printed (Abstract, text)", 0.40, 0.36, 0.43, ORANGE, "0.4 (0.36 to 0.43)"),
("All cases, printed (Table 3)", 0.40, 0.36, 0.44, ORANGE, "0.4 (0.36 to 0.44)"),
("All cases, recomputed Wald", pp(A["d"]), pp(A["wald"][0]), pp(A["wald"][1]), BLUE, f"{pp(A['d']):.2f} ({pp(A['wald'][0]):.2f} to {pp(A['wald'][1]):.2f})"),
("All cases, recomputed Newcombe", pp(A["d"]), pp(A["newcombe"][0]), pp(A["newcombe"][1]), BLUE, f"{pp(A['d']):.2f} ({pp(A['newcombe'][0]):.2f} to {pp(A['newcombe'][1]):.2f})"),
("Emergent, printed (Table 3)", -2.5, -2.92, -2.08, ORANGE, "−2.5 (−2.92 to −2.08)"),
("Emergent, recomputed Wald", pp(E["d"]), pp(E["wald"][0]), pp(E["wald"][1]), BLUE, f"{pp(E['d']):.2f} ({pp(E['wald'][0]):.1f} to {pp(E['wald'][1]):.1f})"),
]
rows = [(a, b, c, d, e, f.replace("-", "−")) for a, b, c, d, e, f in rows]
style(ax); ax.axvline(0, color=INK2, lw=0.8)
ax.axvline(5, color=INK2, lw=1.2, ls=(0, (4, 3)))
ax.text(5.3, -0.55, "NI margin, 5 pp", fontsize=10, color=INK2, va="center")
interval_rows(ax, rows)
ax.axhline(1.5, color=GRID, lw=0.8)
ax.set_xlim(-15, 10); ax.set_xlabel("Failure-rate difference, MO − CA (pp), with 95% CI")
ax.text(1.02, len(rows) - 0.3, "estimate (95% CI)", fontsize=10, color=INK, transform=ax.get_yaxis_transform(), va="bottom")
ax.plot([], [], "o-", color=ORANGE, lw=2, ms=8, label="As printed")
ax.plot([], [], "o-", color=BLUE, lw=2, ms=8, label="Recomputed from the paper's counts (7/231 vs 5/191; 1/28 vs 2/33)")
fig.legend(*ax.get_legend_handles_labels(), loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=2, frameon=False)
fig.subplots_adjust(left=0.37, right=0.75, top=0.84, bottom=0.12); save(fig, path)
# Candidate B: primary outcome only, emphasising headroom to the margin
def cand_b(path):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
rows = [
("Printed, Abstract and Results", 0.40, 0.36, 0.43, ORANGE, ""),
("Printed, Table 3", 0.40, 0.36, 0.44, ORANGE, ""),
("Recomputed, Wald 95%", pp(A["d"]), pp(A["wald"][0]), pp(A["wald"][1]), BLUE, ""),
("Recomputed, Newcombe 95%", pp(A["d"]), pp(A["newcombe"][0]), pp(A["newcombe"][1]), BLUE, ""),
("Recomputed, Agresti–Caffo 95%", pp(A["d"]), pp(A["agresti_caffo"][0]), pp(A["agresti_caffo"][1]), BLUE, ""),
]
style(ax); ax.axvline(0, color=INK2, lw=0.8); ax.axvline(5, color=INK2, lw=1.2, ls=(0, (4, 3)))
interval_rows(ax, rows)
for y, hi, col in ((4, 0.43, ORANGE), (2, pp(A["wald"][1]), BLUE)):
ax.annotate("", xy=(5, y + 0.3), xytext=(hi, y + 0.3), arrowprops=dict(arrowstyle="<->", color=INK2, lw=1))
ax.text(hi - 0.1, y + 0.3, f"{5 - hi:.2f} pp to margin", ha="right", va="center", fontsize=10, color=INK2)
ax.text(5.1, 0, "NI margin\n5 pp", fontsize=10, color=INK2, va="center")
ax.set_xlim(-4, 6.6); ax.set_xlabel("Failure-rate difference, MO − CA (pp), all cases, 95% CI")
ax.plot([], [], "o-", color=ORANGE, lw=2, ms=8, label="As printed")
ax.plot([], [], "o-", color=BLUE, lw=2, ms=8, label="Recomputed from 7/231 vs 5/191")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False)
fig.tight_layout(); save(fig, path)
# Candidate C: arithmetic flow, showing the step where units diverge
def cand_c(path):
fig, ax = plt.subplots(figsize=(7.2, 4.0)); ax.axis("off"); ax.set_xlim(0, 10); ax.set_ylim(0.4, 6.0)
def box(x, y, w, h, txt, ec, fc=BG, bold=False):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.04,rounding_size=0.12", fc=fc, ec=ec, lw=2))
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=10.5, fontweight="bold" if bold else "normal")
box(0.2, 4.7, 2.6, 1.1, "Counts\n7/231 vs 5/191", INK2)
box(3.4, 4.7, 3.0, 1.1, f"Difference\n{A['d']:.5f} = {pp(A['d']):.2f} pp", INK2)
box(7.0, 4.7, 2.8, 1.1, f"Wald SE\n{A['se']:.4f} = {pp(A['se']):.2f} pp", INK2)
for x0, x1 in ((2.8, 3.4), (6.4, 7.0)):
ax.annotate("", xy=(x1, 5.25), xytext=(x0, 5.25), arrowprops=dict(arrowstyle="->", color=INK2, lw=1.5))
box(0.3, 1.9, 4.4, 1.5, f"Same units (pp):\n{pp(A['d']):.2f} ± 1.96 × {pp(A['se']):.2f}\n= {pp(A['wald'][0]):.2f} to {pp(A['wald'][1]):.2f} pp".replace("-", "−"), BLUE)
box(5.3, 1.9, 4.4, 1.5, f"Mixed units:\n{pp(A['d']):.2f} pp ± 1.96 × {A['se']:.4f}\n= {pp(A['d'])-Z*A['se']:.2f} to {pp(A['d'])+Z*A['se']:.2f}", ORANGE, fc="#fdf0ea")
ax.annotate("", xy=(2.5, 3.4), xytext=(4.9, 4.7), arrowprops=dict(arrowstyle="->", color=BLUE, lw=1.5))
ax.annotate("", xy=(3.2, 3.4), xytext=(7.6, 4.7), arrowprops=dict(arrowstyle="->", color=BLUE, lw=1.5))
ax.annotate("", xy=(6.8, 3.4), xytext=(5.1, 4.7), arrowprops=dict(arrowstyle="->", color=ORANGE, lw=1.5))
ax.annotate("", xy=(7.5, 3.4), xytext=(8.4, 4.7), arrowprops=dict(arrowstyle="->", color=ORANGE, lw=1.5))
ax.text(2.5, 1.2, "Recomputed interval: spans 0,\nconsistent with the printed p = 0.7997", ha="center", va="center", fontsize=10, color=INK2)
ax.text(7.5, 1.2, "Printed: 0.36 to 0.43 (Abstract), 0.36 to 0.44 (Table 3).\nA reconstruction only: this route does not\nreproduce the emergent row (−2.92 to −2.08)", ha="center", va="center", fontsize=10, color=INK2)
ax.text(7.6, 4.05, "SE taken as a proportion\n(×100 omitted)", ha="left", fontsize=10, color=INK, style="italic", va="center")
fig.tight_layout(); save(fig, path)
# Candidate D: interval widths, printed vs recomputed
def cand_d(path):
fig, ax = plt.subplots(figsize=(7.2, 3.2))
labs = ["All cases", "Emergent cases"]
rep = [0.44 - 0.36, -2.08 + 2.92]; rec = [wa, we]
y = np.array([1, 0]); h = 0.34
ax.barh(y + h / 2 + 0.01, rep, height=h, color=ORANGE, label="Printed 95% CI width")
ax.barh(y - h / 2 - 0.01, rec, height=h, color=BLUE, label="Recomputed Wald 95% CI width")
for yy, r, c in zip(y, rep, rec):
ax.text(r + 0.3, yy + h / 2, f"{r:.2f} pp", va="center", fontsize=10, color=INK2)
ax.text(c + 0.3, yy - h / 2, f"{c:.2f} pp ({c/r:.0f}× wider)", va="center", fontsize=10, color=INK2)
style(ax); ax.set_yticks(y); ax.set_yticklabels(labs); ax.set_xlim(0, 28)
ax.set_xlabel("Width of the 95% CI for the difference in failure rate (percentage points)")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False)
fig.tight_layout(); save(fig, path)
# Candidate E: p-values, printed vs implied by the printed CI (log scale)
def cand_e(path):
fig, ax = plt.subplots(figsize=(7.2, 3.4))
rows = [("All cases", 0.7997, A["p_chi2"], p_implied), ("Emergent cases", 1.0, E["p_fisher"], p_implied_e)]
for i, (lab, prep, prec, pimp) in enumerate(rows):
y = 1.5 * (1 - i)
ax.plot([pimp, prep], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(prep, y + 0.08, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(prec, y - 0.08, "D", ms=8, color=BLUE, zorder=3)
ax.plot(pimp, y, "o", ms=9, mfc=BG, mec=ORANGE, mew=2, zorder=3)
ax.text(pimp, y + 0.22, f"implied by printed CI: p ≈ {pimp:.0e}", ha="center", fontsize=10, color=INK2, va="bottom")
ax.text(prep * 0.3, y - 0.22, f"printed p = {prep:.4f}; recomputed {prec:.4f}", ha="right", fontsize=10, color=INK2, va="top")
ax.axvline(0.05, color=INK2, lw=0.8); ax.text(0.05, 2.15, "p = 0.05 ", fontsize=10, color=INK2, ha="right")
ax.set_xscale("log"); ax.set_xlim(1e-95, 30); style(ax)
ax.set_yticks([1.5, 0]); ax.set_yticklabels([r[0] for r in rows]); ax.set_ylim(-0.7, 2.3)
ax.set_xlabel("Two-sided p-value (log scale)")
ax.plot([], [], "o", color=ORANGE, ms=9, label="Printed p")
ax.plot([], [], "o", mfc=BG, mec=ORANGE, mew=2, ms=9, label="p implied by printed CI")
ax.plot([], [], "D", color=BLUE, ms=8, label="Recomputed from counts")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=3, frameon=False)
fig.tight_layout(); save(fig, path)
cand_a(os.path.join(HERE, "fig1"))
for f, n in ((cand_a, "a_all_intervals"), (cand_b, "b_margin_headroom"), (cand_c, "c_unit_flow"),
(cand_d, "d_ci_widths"), (cand_e, "e_pvalues")):
f(os.path.join(CAND, "cand_" + n))
print("\nFigures written to", HERE, "and", CAND)
Reported complete-abortion risk ratio of 2.55 should be about 1.02 by the paper's own conversion
BMJ Glob Health (2025) · doi:10.1136/bmjgh-2024-016024 · PMC11749663
Summary
For complete abortion in the review's only task-shifting trial (midlevel providers vs doctors, 1032 participants), the Abstract, the Results text and Table 2 all report a risk ratio (RR) of 2.55 (95% CI 0.82 to 4.27), described as "equivalent" or "little or no difference". The same Table 2 row gives risks of 96.1% and 97.3% and an adjusted risk difference (RD) of 1.49% (−0.17% to 3.14%), and its footnote § says the RR was converted from that RD with a 96% control risk; done in consistent units, that conversion gives RR 1.016 (0.998 to 1.033), whereas leaving the RD in percentage points, (0.96 + 1.49)/0.96, reproduces the published 2.55 (0.82 to 4.27) exactly. With a 96.1% control risk no RR can exceed 1/0.961 = 1.04, so the published value is impossible, and its interval is 100 times too wide on the linear scale. The verbal conclusion of no meaningful difference stands, but the first quantitative result in the Abstract misstates the size and precision of the effect, the correct value appears nowhere in the paper, and no correction has been found.
What the paper reports
Low certainty evidence indicates task shifting of medical abortion by doctors to midlevel providers may result in equivalent complete abortion (RR: 2.55, 95% CI: 0.82 to 4.27).
— Abstract, Results
Low certainty evidence indicates that compared with doctor-led medical abortion, nurse-led medical abortion may lead to little or no difference in abortion success (RR: 2.55, 95% CI: 0.82 to 4.27, one trial, 1032 participants) or the occurrence of an incomplete abortion (RR: 0.92, 95% CI: 0.45 to 1.89, one trial, 1032 participants, table 2). The certainty of evidence was downgraded by two levels due to serious imprecision (data from a single trial and CI including both benefit and harm) and serious risk of bias (no blinding of participants, personnel and outcome assessors).
— Results › Effects of interventions › Comparison 1. Who provides care (task shifting)
Patient health status: complete abortion | 96.1% | 97.3% | RR 2.55 (0.82 to 4.27)§ | 1032 (1 RCT) | Low†‡⊕⊕⊝⊝ | RD 1.49%¶ (–0.17% to 3.14%)
— Table 2 (Summary of findings 1), "complete abortion" row. Columns: outcome | assumed risk, control | corresponding risk, intervention | relative effect (95% CI) | participants | certainty | comments
§The effect in RR was converted from the adjusted risk difference provided in the report using control group risk of 96%.
— Table 2, footnote §
¶The risk difference calculated with intention-to-treat analysis estimated from a generalised estimating equation model with treatment as the fixed effect and service provider as a random effect and adjusted with woman’s age and length of gestation.
— Table 2, footnote ¶
the assumed risk in the comparison group and the relative effect of the intervention
— Table 2, footnote * (which says that the intervention-group risk is based on these two quantities)
What the paper's own numbers show
Method. Footnote § describes the conversion. If the control risk is p₀ and the risk difference is RD (both as proportions), the intervention risk is p₀ + RD, so RR = (p₀ + RD) / p₀. Applied to the point estimate and to each CI limit, this maps the RD interval linearly onto an RR interval. I applied it with p₀ = 0.96 (the footnote) and p₀ = 0.961 (Table 2), first with the RD as a proportion and then with the RD left as a percentage-point number. I also checked the result against the two risks in the same row of Table 2, and against the shape of the published interval. As an outside cross-check I used the counts in the source trial's abstract (Lancet 2011;377:1155-61, PMID 21458058). All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| RR, complete abortion | Abstract; Results Comparison 1; Table 2 | 2.55 (0.82 to 4.27) | 1.016 (0.998 to 1.033), from footnote § with RD as a proportion (same to 3 dp with p₀ 0.96 or 0.961) |
| Same formula, RD left in percentage points | Table 2 footnote § | — | (0.96 + 1.49)/0.96 = 2.552; (0.96 − 0.17)/0.96 = 0.823; (0.96 + 3.14)/0.96 = 4.271 → 2.55 (0.82 to 4.27), an exact match to the published values |
| Intervention risk implied by the conversion | Table 2 | 97.3% | 0.96 + 0.0149 = 0.975 (correct units); 0.96 + 1.49 = 2.45, i.e. 245% (published units) |
| Ratio of the two risks in Table 2 | Table 2 | 96.1% vs 97.3% | 97.3 / 96.1 = 1.012 |
| Largest RR possible with control risk 96.1% | Table 2 | — | 1 / 0.961 = 1.041 |
| Intervention risk implied by RR 2.55 (footnote *) | Table 2 | 97.3% | 0.961 × 2.55 = 2.451 (245%) |
| Shape of the reported RR interval | Table 2 | 0.82 to 4.27 around 2.55 | Linear gaps 1.73 below, 1.72 above (symmetric on the linear scale); log gaps 1.135 vs 0.516. RR width / RD width = 1.042 ≈ 1/0.96 = 1.042, i.e. a linear rescaling of the RD interval by 1/0.96 |
| Width of the published interval relative to the recomputed one | Table 2 | 0.82 to 4.27 | 100 times wider on the linear scale; 48.6 times wider on the log scale |
| Adjusted RD, complete abortion | Table 2 comments | 1.49% (−0.17% to 3.14%) | Not reproducible from counts (GEE-adjusted, footnote ¶); taken as given. Unadjusted from trial counts: 1.19% (−0.99% to 3.37%); trial abstract: 1.24% (−0.53 to 3.02) |
| Unadjusted RR, complete abortion (outside check) | Source trial abstract: 504/518 vs 494/514 | — | 1.012 (0.990 to 1.035) |
| RR, incomplete abortion (control check) | Results; Table 2 row 2 | 0.92 (0.45 to 1.89) | 14/518 vs 15/514: 0.926 (0.45 to 1.90), consistent |
The adjacent incomplete-abortion RR is reproduced from the trial's counts. This shows that the rest of the row's arithmetic holds, and that the discrepancy is confined to the RD-to-RR conversion.
Figure
Figure 1. The conversion described in Table 2 footnote §, from the inputs stated in Table 2 to the risk ratio. The two lanes differ only at the first step, which is shaded: whether the adjusted risk difference enters the formula as a proportion (0.0149, blue, recomputed) or as a percentage-point number (1.49, orange). The orange lane reproduces the published RR 2.55 (0.82 to 4.27) to two decimals. It does so by way of an intervention "risk" of 245%, and its result lies above 1/0.961 = 1.04, the largest RR possible with the 96.1% control risk in the same table row. The blue lane gives RR 1.016 (0.998 to 1.033), which agrees with the ratio of the tabulated risks (1.012). All values are computed in recompute.py.txt.
Figure 2. Risk ratio for complete abortion (midlevel providers vs doctors) on a log scale, with a thin reference line at RR = 1. Orange: the value published in the Abstract, Results and Table 2. Blue: the footnote § conversion recomputed with the RD as a proportion (its interval, 0.998 to 1.033, is narrower than the marker at this scale), and the plain ratio of the two Table 2 risks. The shaded region marks RRs above 1/0.961 = 1.04, which cannot occur when the control risk is 96.1%. The published point estimate and most of its interval lie in that region.
Why it matters
- It is the headline number for the outcome. The value appears three times: as the first quantitative result in the Abstract, in the Results text and in the summary-of-findings table. A reader who takes the RR at face value would read it as about 2.5 times the rate of complete abortion with a very wide interval. The trial data in the same row show nearly identical rates (97.3% vs 96.1%) and a tight interval (about 1.00 to 1.03).
- The number is internally contradictory. The text calls it "equivalent" / "little or no difference", which agrees with the RD but not with an RR of 2.55. Footnote * says the intervention risk is based on the control risk and the relative effect, and 96.1% × 2.55 is 245%, not 97.3%.
- It can propagate. Anyone who extracts the RR from the Abstract or Table 2 for an overview, a meta-analysis or a guideline would carry forward a ratio wrong by a factor of about 2.5, with an interval 100 times too wide on the linear scale (about 49 times on the log scale). The correct value is not stated anywhere in the paper, so a reader cannot pick it up from elsewhere in the text.
- The GRADE rationale cites the interval. The imprecision downgrade is justified as "CI including both benefit and harm" and, in footnote ‡, "wider CI". The correct interval (0.998 to 1.033) still just includes 1, so this wording is not strictly wrong, and a downgrade for a single trial may well stand. But the published interval of 0.82 to 4.27 makes the imprecision look far larger than it is.
- What does not change. The direction of the effect, the lack of statistical significance, the qualitative reading (equivalence), the low-certainty rating and the review's overall conclusions on task shifting are not affected.
Explanations considered
- A different effect measure (odds ratio, or the RR of a non-complete outcome). From the trial counts, the odds ratio for complete abortion is 1.457 (0.73 to 2.92). The RR of a non-complete abortion (doctor vs midlevel) is 1.440. Neither is 2.55 (0.82 to 4.27), and Table 2 labels the value as an RR.
- Covariate adjustment. Footnote ¶ says the RD came from a GEE model adjusted for age and gestation, which explains why the RD (1.49%) differs from the crude 1.19% to 1.24%. But no adjustment can produce a risk ratio above 1/0.961 = 1.04 when the control risk is 96.1%, and converting the adjusted RD gives 1.016.
- A different control risk. The footnote uses 96%, and Table 2 shows 96.1%. Both give RR 1.016 (0.998 to 1.033) with consistent units, and both give 2.55 (0.82 to 4.27) when the RD is not divided by 100. The choice of control risk is immaterial.
- The log-scale asymmetry is just a feature of the conversion. That is correct: an interval converted linearly from an RD interval is symmetric on the linear scale, as observed here. So the asymmetry on the log scale is not itself evidence of an error. The error is the factor of 100 in the RD, which the linear structure of the published interval confirms (its width is exactly 1/0.96 times the width of the RD interval in percentage points).
- A typesetting or XML conversion artefact. The same values appear in three separate places (Abstract, Results text, Table 2 cell) in the archived publisher XML. They all agree with each other and with an exact arithmetic reproduction, so a rendering artefact does not explain them.
- A rounding or transcription slip. A transposed digit or a rounding error would not produce three numbers (point estimate and both limits) that all match one systematic formula to two decimals.
- The supplementary material contains another calculation. Online supplemental file 1 (search strategy, study characteristics, extraction and risk-of-bias tables) and file 2 (reflexivity statement) give no further effect estimate or RR calculation for this outcome.
None of these accounts for the published numbers. The stated conversion, carried out with the RD in percentage points rather than as a proportion, reproduces them exactly.
Status
As of 2026-10-23:
- Version of record. The publisher's typeset PDF (PMC11749663.1.pdf from the PMC open-data bucket; 13 pages, "BMJ Glob Health 2025;10:e016024", created 2025-01-03) carries the same values in all three places: the Abstract, Results › Comparison 1, and the Table 2 complete-abortion row with footnote §. The minus signs in the RD interval are intact, and the Table 2 columns line up, so no cell is misplaced. The PMC and Europe PMC XML agree with the PDF. Every quote above was checked verbatim against the PMC text with
src/show.py. The publisher's HTML page (gh.bmj.com, reached from the DOI redirect) returned HTTP 403. - Supplements. I checked online supplemental file 1 (PDF, 24 pages, Tables S1–S7 and Figure S1) and file 2 (DOCX, reflexivity statement). Table S6 describes the trial (535 and 542 randomised) but gives no complete-abortion RR or RD. Neither file restates or corrects the figure.
- Corrections and notices.
src/known.pyreturned no Crossref updates, no PMC notices and no Europe PMC notices. The Crossref works record has noupdate-to,updated-byorrelationentries, and Crossrefrelation.objectandupdatesfilters on the DOI return 0 works. A Europe PMC search for the DOI returns only the original article. Earlier web searches for a correction, erratum or comment found none. The web-search quota was used up during this final check, so those searches were not repeated today. - PubPeer. The PubPeer search page and API return HTTP 403 (a Cloudflare challenge) from this machine, so PubPeer could not be checked directly. Earlier web searches found no PubPeer thread for this article.
- Sources. PMC11749663 (https://pmc.ncbi.nlm.nih.gov/articles/PMC11749663/); Crossref API (https://api.crossref.org/works/10.1136/bmjgh-2024-016024); Europe PMC REST search; source trial abstract, PubMed 21458058 (Lancet 2011;377:1155-61, doi:10.1016/S0140-6736(10)62229-5), retrieved through NCBI E-utilities.
Suggested correction
In the Abstract, in Results › Comparison 1 and in Table 2 (complete abortion row), replace "RR: 2.55, 95% CI: 0.82 to 4.27" with "RR: 1.02, 95% CI: 1.00 to 1.03". Unrounded, this is 1.016 (0.998 to 1.033), from (0.96 + RD)/0.96 with RD = 0.0149 (−0.0017 to 0.0314). If two decimals would hide the fact that the lower limit lies just below 1, report three decimals: 1.016 (0.998 to 1.033). The authors may also wish to:
- re-check the GRADE imprecision rationale for this outcome against the corrected interval, and
- check whether the same conversion step was applied elsewhere. For example, Table 3 footnote † describes an RR derived from an OR with an assumed control risk.
Recomputation script (python recompute.py.txt)
"""Recompute the complete-abortion risk ratio in PMC11749663 (BMJ Glob Health 2025,
doi:10.1136/bmjgh-2024-016024), Table 2, and regenerate fig1/fig2 in this folder.
Run: python recompute.py.txt
Needs numpy, scipy, matplotlib.
"""
from pathlib import Path
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle
HERE = Path(__file__).resolve().parent
z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (paper)
RR_REP = (2.55, 0.82, 4.27) # Abstract; Results Comparison 1; Table 2 row "complete abortion"
RD_PP = (1.49, -0.17, 3.14) # Table 2 comments: adjusted RD, percentage points (footnote ¶)
P0_FOOT = 0.96 # footnote §: "control group risk of 96%"
CTRL, INTV = 0.961, 0.973 # Table 2 illustrative risks, control / intervention
print("=" * 72)
print("1. Reported values (paper)")
print(f" RR {RR_REP[0]} (95% CI {RR_REP[1]} to {RR_REP[2]})")
print(f" adjusted RD {RD_PP[0]}% ({RD_PP[1]}% to {RD_PP[2]}%); control 96.1%, intervention 97.3%")
def rr_from_rd(rd, p0):
"""Footnote §: RR = (p0 + RD) / p0, with RD and p0 on the same scale."""
return (p0 + rd) / p0
print("\n2. Footnote § conversion done in consistent units (RD / 100)")
correct = [rr_from_rd(x / 100, P0_FOOT) for x in RD_PP]
print(" p0 = 0.96 : RR = %.4f (%.4f to %.4f) -> %.3f (%.3f to %.3f)" % (*correct, *correct))
correct961 = [rr_from_rd(x / 100, CTRL) for x in RD_PP]
print(" p0 = 0.961: RR = %.4f (%.4f to %.4f)" % tuple(correct961))
print(" implied intervention risk 0.96 + 0.0149 = %.4f" % (P0_FOOT + RD_PP[0] / 100))
print("\n3. Same formula with RD left in percentage points (not divided by 100)")
slip = [rr_from_rd(x, P0_FOOT) for x in RD_PP]
print(" p0 = 0.96 : RR = %.3f (%.3f to %.3f) -> rounds to %.2f (%.2f to %.2f)" % (*slip, *slip))
slip961 = [rr_from_rd(x, CTRL) for x in RD_PP]
print(" p0 = 0.961: RR = %.3f (%.3f to %.3f)" % tuple(slip961))
print(" implied intervention 'risk' 0.96 + 1.49 = %.2f (i.e. %.0f%%)" % (P0_FOOT + RD_PP[0], 100 * (P0_FOOT + RD_PP[0])))
print("\n4. Consistency checks within Table 2")
ratio_tab = INTV / CTRL
print(f" ratio of Table 2 risks 97.3/96.1 = {ratio_tab:.4f}")
print(f" largest possible RR with control risk 96.1%: 1/0.961 = {1 / CTRL:.4f}")
print(f" intervention risk implied by RR 2.55 (footnote *): 0.961 x 2.55 = {CTRL * 2.55:.3f} ({100 * CTRL * 2.55:.0f}%)")
print(f" published / correct RR = {RR_REP[0] / correct[0]:.2f}; published / correct CI width = "
f"{(RR_REP[2] - RR_REP[1]) / (correct[2] - correct[1]):.1f} (exactly 100 before rounding: "
f"{(slip[2] - slip[1]) / (correct[2] - correct[1]):.1f})")
print(f" published / correct CI width on the log scale = "
f"{np.log(RR_REP[2] / RR_REP[1]) / np.log(correct[2] / correct[1]):.1f}")
print(f" intervention risk implied by RR {correct[0]:.4f}: 0.961 x {correct[0]:.4f} = {CTRL * correct[0]:.4f}")
print("\n5. Shape of the reported interval")
lo, hi = RR_REP[0] - RR_REP[1], RR_REP[2] - RR_REP[0]
print(f" linear gaps: {lo:.2f} below, {hi:.2f} above (midpoint {(RR_REP[1] + RR_REP[2]) / 2:.3f})")
llo, lhi = np.log(RR_REP[0]) - np.log(RR_REP[1]), np.log(RR_REP[2]) - np.log(RR_REP[0])
print(f" log gaps: {llo:.3f} below, {lhi:.3f} above")
print(f" RR width / RD width = {(RR_REP[2] - RR_REP[1]) / (RD_PP[2] - RD_PP[1]):.4f}; 1/0.96 = {1 / 0.96:.4f}")
print("\n6. Source trial (Lancet 2011;377:1155-61, PMID 21458058, abstract counts)")
a, n1 = 504, 518 # midlevel providers: complete abortions / analysed
b, n0 = 494, 514 # doctors
p1, p0 = a / n1, b / n0
rr = p1 / p0
se = np.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n0)
print(f" complete: {a}/{n1} = {p1:.4f} vs {b}/{n0} = {p0:.4f}; n = {n1 + n0}")
print(f" unadjusted RR = {rr:.4f} ({np.exp(np.log(rr) - z * se):.4f} to {np.exp(np.log(rr) + z * se):.4f})"
f" -> {rr:.3f} ({np.exp(np.log(rr) - z * se):.3f} to {np.exp(np.log(rr) + z * se):.3f})")
rd = p1 - p0
serd = np.sqrt(p1 * (1 - p1) / n1 + p0 * (1 - p0) / n0)
print(f" unadjusted RD = {100 * rd:.2f}% ({100 * (rd - z * serd):.2f}% to {100 * (rd + z * serd):.2f}%);"
" trial abstract reports 1.24% (-0.53 to 3.02)")
print(f" footnote § conversion of the trial-abstract RD 1.24%: RR = {rr_from_rd(0.0124, P0_FOOT):.4f}")
inc1, inc0 = n1 - a, n0 - b - 5 # 5 failed abortions in doctor arm, none in midlevel arm
r = (inc1 / n1) / (inc0 / n0)
s = np.sqrt(1 / inc1 - 1 / n1 + 1 / inc0 - 1 / n0)
print(f" incomplete: {inc1}/{n1} vs {inc0}/{n0}: RR {r:.3f} ({np.exp(np.log(r) - z * s):.2f} to "
f"{np.exp(np.log(r) + z * s):.2f}); paper reports 0.92 (0.45 to 1.89)")
print("\n7. Other effect measures, to rule them out as the source of 2.55 (0.82 to 4.27)")
orr = (a / (n1 - a)) / (b / (n0 - b))
seo = np.sqrt(1 / a + 1 / (n1 - a) + 1 / b + 1 / (n0 - b))
print(f" odds ratio, complete abortion = {orr:.3f} ({np.exp(np.log(orr) - z * seo):.2f} to {np.exp(np.log(orr) + z * seo):.2f})")
print(f" RR of non-complete abortion, doctor vs midlevel = {((n0 - b) / n0) / ((n1 - a) / n1):.3f}")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(HERE / f"{name}.svg", facecolor=BG)
fig.savefig(HERE / f"{name}.png", dpi=200, facecolor=BG)
plt.close(fig)
# fig1: arithmetic chain, two lanes
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0.5, 64)
ax.axis("off")
def box(x0, y0, w, h, text, edge, lw=2, face="white", bold_first=True, fs=10):
ax.add_patch(FancyBboxPatch((x0, y0), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc=face, ec=edge, lw=lw))
lines = text.split("\n")
n = len(lines)
for i, ln in enumerate(lines):
yy = y0 + h / 2 + (n - 1) / 2 * 3.3 - i * 3.3
ax.text(x0 + w / 2, yy, ln, ha="center", va="center", fontsize=fs, color=INK,
fontweight="bold" if (bold_first and i == 0) else "normal")
def arrow(x0, x1, y, color):
ax.annotate("", xy=(x1, y), xytext=(x0, y),
arrowprops=dict(arrowstyle="-|>", color=color, lw=2, mutation_scale=12))
box(2, 50.5, 96, 12,
"Inputs stated in Table 2\n"
"adjusted RD = 1.49% (−0.17% to 3.14%) · control risk 96% (footnote §)\n"
"conversion: RR = (0.96 + RD) / 0.96",
INK2, lw=1.2, face="#f3f2ef")
cols = [(2, 22), (27, 22), (52, 22), (77, 21)]
# highlight the diverging step (behind the first box of each lane only)
for yy in (29, 7):
ax.add_patch(FancyBboxPatch((0.9, yy - 1.1), 24.2, 15.7, boxstyle="round,pad=0,rounding_size=1.2",
fc="#e6e5e0", ec="none", zorder=0))
ax.text(13, 2.6, "units diverge here", ha="center", va="center", fontsize=10,
color=INK, fontweight="bold")
lanes = [
(29, BLUE, "Recomputed: RD divided by 100 before use (consistent units)",
["RD entered\n0.0149\n(−0.0017 to 0.0314)",
"Intervention risk\n0.96 + 0.0149\n= 0.975 (97.5%)",
f"RR = 0.975 / 0.96\n{correct[0]:.3f}\n({correct[1]:.3f} to {correct[2]:.3f})",
f"Check vs Table 2\n97.3 / 96.1 = {ratio_tab:.3f}\nconsistent"]),
(7, ORANGE, "Published: RD left in percentage points (not divided by 100)",
["RD entered\n1.49\n(−0.17 to 3.14)",
f"Intervention risk\n0.96 + 1.49\n= {P0_FOOT + RD_PP[0]:.2f} (245%)",
f"RR = 2.45 / 0.96\n{slip[0]:.2f}\n({slip[1]:.2f} to {slip[2]:.2f})",
f"Check vs Table 2\nabove max {1 / CTRL:.2f}\ninconsistent"]),
]
for y0, color, label, texts in lanes:
ax.add_patch(Rectangle((2.2, y0 + 15.2), 2.4, 2.4, fc=color, ec="none"))
ax.text(5.6, y0 + 16.4, label, ha="left", va="center", fontsize=10, color=INK)
for (x0, w), t in zip(cols, texts):
box(x0, y0, w, 13.5, t, color)
for (xa, wa), (xb, _) in zip(cols[:-1], cols[1:]):
arrow(xa + wa, xb, y0 + 6.75, color)
ax.text(62.0, 2.6, "= published 2.55 (0.82 to 4.27)", ha="center", va="center",
fontsize=10, color=INK, fontstyle="italic")
save(fig, "fig1")
# fig2: dot-and-interval on log scale
fig, ax = plt.subplots(figsize=(7.2, 2.9), facecolor=BG)
fig.subplots_adjust(left=0.035, right=0.97, top=0.97, bottom=0.18)
ax.set_facecolor(BG)
xmax_possible = 1 / CTRL
ax.axvspan(xmax_possible, 8, color="#efeeea", zorder=0, lw=0)
ax.text(1.08, -0.42, f"RR > 1/0.961 = {xmax_possible:.2f} is impossible with a 96.1% control risk",
fontsize=10, color=INK2, va="center", ha="left")
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
rows = [
(2.0, ORANGE, "o", RR_REP, "Published (Abstract, Results, Table 2): RR 2.55 (0.82 to 4.27)"),
(1.0, BLUE, "o", correct, f"Footnote § conversion, recomputed: RR {correct[0]:.3f} ({correct[1]:.3f} to {correct[2]:.3f})"),
(0.0, BLUE, "D", (ratio_tab, None, None), f"Ratio of Table 2 risks 97.3% / 96.1% = {ratio_tab:.3f}"),
]
for y, c, m, (pt, l, h), lab in rows:
if l is not None:
ax.plot([l, h], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=3)
ax.plot(pt, y, marker=m, ms=9 if m == "o" else 8, color=c, mec=BG, mew=1.5, zorder=4, ls="none")
ax.text(0.52, y + 0.36, lab, fontsize=10, color=INK, va="center", ha="left")
ax.set_xscale("log")
ax.set_xlim(0.5, 8)
ax.set_ylim(-0.7, 2.6)
ax.set_xticks([0.5, 0.82, 1, 1.5, 2, 2.55, 4.27, 8])
ax.set_xticklabels(["0.5", "0.82", "1", "1.5", "2", "2.55", "4.27", "8"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([])
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s in ("left", "right", "top"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.set_xlabel("Risk ratio, complete abortion, midlevel providers vs doctors (log scale; line at RR = 1)", fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png in", HERE)
Abstract's per-case costs for Xpert Ultra and TB-LAMP do not match Table 2, and the paper's model output rests on two different Xpert test costs
BMJ Glob Health (2025) · doi:10.1136/bmjgh-2025-019270 · PMC12336617
Summary
The abstract gives the cost per TB case detected as US$17 for TB-LAM, US$20 for Xpert MTB/RIF Ultra and US$22 for TB-LAMP. The Results text and Table 2 give US$16.78, US$14.59 and US$20.11. These round to 17, 15 and 20, so the TB-LAM figure agrees and the other two do not.
The paper's decision tree (its Figure 1) shows where Table 2 comes from and why the paper also prints a second Xpert figure. Rebuilt from its own node values, the tree reproduces Table 2's costs, effects and all three NMBs exactly, and the Figure 1 ICER label of 186.55 (Table 2's 187.18 follows from its rounded entries). But its Xpert branch uses a test cost of US$7.97, which is Table 1B's Xpert reagents line. Table 1B's Xpert total is US$19.09 and the Results text gives US$21.01. Run with US$21.01, the same tree gives US$21.80 for Xpert. This matches the Discussion's US$21.79 and, to the cent, the Table 3 Xpert value (2729.29). The abstract's US$20 matches neither run. It would need an Xpert test cost of US$17.75, which the paper does not print.
The error matters because the two runs order the strategies differently. In Table 2, Xpert is the cheapest strategy and TB-LAM costs US$2.19 more per case. With the full Xpert test cost, TB-LAM is cheaper and also more effective, so it dominates, as the Discussion says. The abstract's figures also put TB-LAM below Xpert. TB-LAM has the highest net monetary benefit in every version, so the paper's main recommendation does not depend on which figures are right.
What the paper reports
Abstract, Results:
TB/LAM was found to be the most cost-effective option at a cost of US$17 per TB case detected when compared with US$20 and US$22 per TB case detected for the baseline strategy of Xpert MTB/RIF Ultra and TB-LAMP, respectively. These ICERs are consistent with willingness-to-pay thresholds set at three times Nigeria’s gross domestic product (GDP) and remained robust over a wide range of costs and epidemiological parameter inputs.
Results, "Testing of TB in PLWH", first paragraph:
TB-LAM was found to be the most cost-effective diagnostic strategy, at a cost of US$16.78 per TB case detected, compared with US$20.11 for TB-LAMP and US$14.59 for Xpert MTB/RIF Ultra (table 2).
While the per test costs for TB-LAM, TB-LAMP and Xpert MTB/RIF Ultra were US$11.92, US$17.62 and US$21.01, respectively (as shown in table 1B); multiple individuals with presumptive TB must be tested to identify a single confirmed TB case, especially given the estimated TB prevalence among PLWH of 22%.
Results, "Decision model":
Figure 1 presents a decision tree model outlining the diagnostic pathways and associated accuracy probabilities, utility and cost calculations for the four modalities. It incorporates data from table 1A, B.
Figure 1 (decision tree, PMC image bmjgh-10-8-g001.jpg). The node labels have the form "($) cost; (c) effect". I transcribed them from the image.
Xpertgene_Ultra_algo ("ICER: Baseline"): root ($) 14.5994; (c) 0.521. Active TB ($) 12.4316; (c) 0.986. No active TB ($) 15.1414; (c) 0.4048. Xpert node after a negative CXR ($) 18.17; (c) 0.95 under Active TB, and ($) 18.17; (c) 0.04 under No active TB.
TB LAM_algo ("ICER: 186.5503"): root ($) 16.7798; (c) 0.5327. Active TB ($) 13.5376; (c) 0.918. No active TB ($) 17.5904; (c) 0.4364. LAM node ($) 22.12; (c) 0.707 and ($) 22.12; (c) 0.091.
TB LAMP_algo ("ICER: Strongly Dominated"): root ($) 20.1139; (c) 0.5085. Active TB ($) 15.2288; (c) 0.8986. No active TB ($) 21.3352; (c) 0.411. LAMP node ($) 28.16; (c) 0.638 and ($) 28.16; (c) 0.05.
Terminal payoffs: "cost_cxr; (c) post_case" after a positive CXR, and "cost_cxr+cost_ultra" (or +cost_LAM, +cost_LAMP) after a negative CXR. The counts printed at the right of the Xpert branches are 144.0, 53.2 and 2.8 under Active TB, and 304.0, 19.84 and 476.16 under No active TB.
Table 2, "Cost-effectiveness of results for XpertMTB/RIF, TB LAMP and TB LAM per additional TB case detected" (footer: "Willingness to pay ($5280).*Difference."):
Testing options | Cost | Effect | *Cost | *Effect | ICER | Dominance | NMB
Xpert MTB/RIF (ULTRA) | $14.59 | 0.521 | – | – | | | 2736.49
TB LAM | $16.78 | 0.5327 | 2.19 | 0.0117 | 187.1794 | | 2796.02
TB LAMP | $20.11 | 0.5085 | $3.33 | -0.0242 | -137.60 | Strongly Dominated | 2664.91
Table 1A (mean values):
TB prevalence among PLWH | Prevalence | 0.22†
CXR | Sensitivity | 0.72
CXR | Specificity | 0.62
Table 1B (columns Xpert MTB/RIF Ultra, TB LAM, Chest X-ray, TB LAMP, Source):
Reagents | 7.97 | 3.5 | 0 | 6 | 4 32 33
Total cost | 19.09 | 11.92 | 10.2‡ | 17.62 | 16
Figure 2 legend:
A vertical dashed red line indicates the baseline ICER ($187).
Discussion, second paragraph:
As detailed in table 3, TB LAM outperformed Xpert MTB/RIF Ultra and TB LAMP in terms of both cost and diagnostic yield. TB LAM had a cost of US$16.78 and an effectiveness of 0.532, surpassing Xpert MTB/RIF Ultra (US$21.79, effectiveness 0.521) and TB LAMP (US$20.11, effectiveness 0.506). The ICER for TB LAM was calculated at US$124.72 per additional case detected, further confirming its financial and clinical advantage.
Discussion, fifth paragraph:
Xpert MTB/RIF Ultra, though offering rapid diagnostics, was found to be less favourable due to its higher cost and lower cost-effectiveness.
Table 3, "Value of information analysis for TB screening strategies", strategy rows (column headed "Expected value of perfect information (EVPI)"):
TB LAMP_algo | $2678.22
TB LAM_algo | $2796.02
Xpertgene_Ultra_algo | $2729.29
Methods, "Willingness to pay":
We adopted Nigeria’s gross domestic product (GDP) per capita value of US$1760, such that willingness to pay would be presumed to be three times this amount (US$5280).
What the paper's own numbers show
Method. I compared the abstract's three costs with the Results text and Table 2, and with their rounding to whole dollars. I checked that Table 2 holds together: each increment should be the difference between rows, each ICER the ratio of the increments, and each NMB should equal 5280 × effect − cost. I then rebuilt the Figure 1 tree. Each strategy starts with a CXR (US$10.20, sensitivity 0.72, specificity 0.62, from Table 1A/B). A positive CXR ends the branch, and a negative CXR is followed by the confirmatory test. The strategy cost is therefore 10.20 + P(CXR negative) × test cost. I read each test cost off the tree as its test node minus 10.20, rolled the tree back at the prevalence the figure uses, and compared every node with the printed value. I then reran the Xpert branch with each Xpert test cost printed in the paper (7.97, 19.09, 21.01 and the component sum 21.89). For the Discussion's ICER of 124.72, I tried every ratio (cost difference ÷ effect difference) that can be formed from the paper's strategy-level costs (per case and per test: the Table 2, Discussion and abstract values, the per-test costs, the CXR cost and the Xpert reagents line, 13 values in all) and its five strategy-level effects (0.521, 0.5327, 0.5085, 0.532, 0.506). I then repeated the search with the node-level and component values added, to see how specific a match is. I also compared each test's sensitivity and specificity at the tree nodes with Table 1A. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| TB-LAM cost per case detected | Abstract vs Results, Table 2 | US$17 vs US$16.78 | 16.78 rounds to 17: consistent |
| Xpert Ultra cost per case detected | Abstract vs Results, Table 2 | US$20 vs US$14.59 | 14.59 rounds to 15; the abstract is 5.41 higher |
| TB-LAMP cost per case detected | Abstract vs Results, Table 2, Discussion | US$22 vs US$20.11 | 20.11 rounds to 20; the abstract is 1.89 higher |
| Xpert Ultra cost per case detected | Discussion | US$21.79 | rounds to 22; a third value, different from both the abstract (20) and Table 2 (14.59) |
| Cost order of the three strategies | Abstract / Table 2 / Discussion | — | LAM < Xpert < LAMP / Xpert < LAM < LAMP / LAM < LAMP < Xpert |
| ΔCost, ΔEffect, ICER of LAM vs Xpert | Table 2; Figure 2 legend | 2.19, 0.0117, 187.1794 ($187) | 16.78 − 14.59 = 2.19; 0.5327 − 0.521 = 0.0117; 2.19 / 0.0117 = 187.18 |
| ΔCost, ΔEffect, ICER of LAMP vs LAM | Table 2 | 3.33, −0.0242, −137.60 | 3.33; −0.0242; −137.60 |
| Test cost used in each tree branch | Figure 1 test nodes | Xpert 18.17, LAM 22.12, LAMP 28.16 | minus CXR 10.20: Xpert 7.97 (Table 1B reagents line, not the 19.09 total), LAM 11.92 (= Table 1B total), LAMP 17.96 (Table 1B total 17.62) |
| Xpert nodes, rebuilt | Figure 1 | Active TB 12.4316 / 0.986; No active TB 15.1414 / 0.4048; root 14.5994 / 0.521 | 0.72 × 10.20 + 0.28 × 18.17 = 12.4316; 0.38 × 10.20 + 0.62 × 18.17 = 15.1414; 0.2 × 12.4316 + 0.8 × 15.1414 = 14.5994; effects 0.986, 0.4048, 0.52104 |
| LAM and LAMP nodes, rebuilt | Figure 1 | LAM root 16.7798 / 0.5327; LAMP root 20.1139 / 0.5085 | 16.7798 / 0.53273; 20.1139 / 0.50853; all six branch nodes also match |
| ICER of LAM vs Xpert from the tree | Figure 1 label; Table 2 | 186.5503; 187.1794 | 186.5503 from unrounded tree values; 187.18 comes from the rounded Table 2 entries |
| NMB from the unrounded tree values | Table 2 | 2736.49 / 2796.02 / 2664.91 | 2736.49 / 2796.02 / 2664.91 (exact). From Table 2's rounded effects they come out 0.14 to 0.20 lower |
| Xpert cost per case, tree rerun with each printed Xpert test cost | Table 1B reagents / Table 1B total / Results text / component sum | 7.97 / 19.09 / 21.01 / 21.89 | 14.60 / 20.74 / 21.80 / 22.28 |
| Xpert value in Table 3 | Table 3 | 2729.29 | tree with Xpert test cost 21.01: NMB 2729.29 (exact); cost implied by 2729.29 is 21.80, i.e. a test cost of 21.02 |
| Xpert test cost needed for the abstract's US$20 | Abstract | US$20 | (20 − 10.20) / 0.552 = 17.75, printed nowhere in the paper |
| LAM − Xpert cost per case | Table 2; tree with Xpert at 21.01 | +2.19 | +2.18 with Xpert at 7.97; −5.02 with Xpert at 21.01 (LAM then cheaper and more effective) |
| Prevalence used by the tree | Figure 1 vs Table 1A, Results text | 0.22 (22%) | the Figure 1 node values and cohort counts reproduce only at 0.20 (200 with TB per 1000); at 0.22 the Xpert root would be 14.5452 |
| ICER of LAM vs Xpert using the Discussion's costs and effects | Discussion | US$124.72 | (16.78 − 21.79) / (0.532 − 0.521) = −455.45 (LAM cheaper and more effective, i.e. dominant) |
| Ratios of strategy-level costs and effects that give 124.72 | — | — | over the 13 strategy-level costs and 5 strategy-level effects, only one ratio is within 0.05 (the script finds it in both sign orientations): (20.11 − 16.78) / (0.5327 − 0.506) = 124.72, the LAMP-minus-LAM cost difference over a LAM-minus-LAMP effect difference that uses the Discussion's LAMP effect |
| Same search with node-level and component values added | — | — | with the Figure 1 node values, Table 1B components, increments, NMB and Table 3 values, sensitivities, specificities and prevalences added (41 costs, 28 probabilities), 34 distinct ratios fall within 0.05 of 124.72, e.g. (21.01 − 14.5994) / (0.95 − 0.8986). With a pool this large, near-matches arise by chance, so these carry no weight. The strategy-level match above is the one built from the quantities an ICER compares |
| LAMP value in Table 3 | Table 3 | 2678.22 | implies a LAMP cost of 6.81 at the tree effect, below the CXR cost alone; it matches no run |
| NMB with the abstract's costs (Table 2 effects) | — | — | Xpert 2730.9, LAM 2795.7, LAMP 2662.9 |
| NMB with Table 2 costs; with Xpert at 21.80 | — | — | LAM highest in both (2796.0 vs Xpert 2736.5 or 2729.3, LAMP 2664.9) |
| Xpert per-test cost | Table 1B total vs components vs Results text | 19.09; 21.01 | components 7.97 + 5.11 + 5.92 + 2.89 = 21.89. The LAM (11.92) and LAMP (17.62) totals equal their component sums |
| TB-LAM accuracy used by the tree | Figure 1 LAM nodes vs Table 1A | sensitivity (c) 0.707; false-positive (c) 0.091 | Table 1A gives sensitivity 0.701 and specificity 0.91 (false-positive 0.09). The LAM nodes reproduce only with 0.707 and 0.091 (0.72 + 0.28 × 0.707 = 0.91796 → 0.918; with 0.701, 0.91628; 0.38 + 0.62 × 0.091 = 0.43642 → 0.4364; with 0.09, 0.43580). The Results cohort (2200 × 0.701 = 1542.2 → 1542) and the supplement use 0.701 |
| TB-LAMP cases detected in a 10 000 cohort | Results text; Table 1A sensitivity 0.64 | 1404 (63.8%), missed 796 | 2200 × 0.64 = 1408 (missed 792); 1404 corresponds to 0.638, the value in Figure 1 and the online supplement |
Table 2 is the output of the Figure 1 tree. Every node in the figure can be rebuilt from Table 1A's CXR accuracy, a CXR cost of 10.20, a prevalence of 0.20, each test's sensitivity and specificity as shown at its node, and one test cost per strategy. The rebuilt tree gives the root costs and effects in Table 2, the Figure 1 ICER label of 186.5503, and all three Table 2 NMBs to the cent. For TB-LAM the test cost in the tree is the Table 1B total (11.92). For Xpert it is 7.97, which is only the reagents line of Table 1B. It is not the table's total (19.09), its component sum (21.89) or the per-test cost the Results text states (21.01). When the same tree is run with the Results text's 21.01, Xpert comes out at US$21.80, which is the Discussion's US$21.79 and reproduces Table 3's Xpert value of 2729.29 exactly. So the paper contains the output of two runs of one model. They differ only in the Xpert test cost. Table 2, the Results text, the Figure 1 labels and the tornado plot's dashed baseline line ($187) come from the 7.97 run. The Discussion's cost figures and Table 3's Xpert value come from the 21.01 run. The tornado plot's Xpert bars, read approximately, sit near the 21.01 run (see Status).
The abstract's figures match neither run. Its US$17 for TB-LAM is common to both. Its US$20 for Xpert is not 14.60, 20.74 (with 19.09), 21.80 or 22.28, and would need a test cost of 17.75. Its US$22 for TB-LAMP is not 20.11 in either run. In the 21.01 run, however, Xpert rounds to 22 and TB-LAMP to 20, which are the abstract's two numbers with the strategy labels exchanged. That would fit a labelling slip in the abstract, but the paper does not say which run the abstract reports. Whichever Xpert test cost is intended, the abstract, the Results/Table 2 and the Discussion cannot all be correct together.
Figure
Figure 1. The cost per TB case detected (US$) as printed in the abstract's Results sentence (left) and in the Results text and Table 2 (right). All six numbers are copied from the paper. Arrows join the same strategy. The TB-LAM figures agree (16.78 rounds to 17). The Xpert Ultra figures (20 vs 14.59, which rounds to 15) and the TB-LAMP figures (22 vs 20.11, which rounds to 20) do not. As a result, the abstract puts TB-LAM below Xpert in cost, while Table 2 puts Xpert below TB-LAM.
Figure 2. Xpert Ultra cost per TB case detected (US$, linear axis from zero). Blue: the paper's Figure 1 tree rebuilt and rerun with each Xpert test cost printed in the paper (cost per case = 10.20 + 0.552 × test cost at the tree's prevalence of 0.20). Orange: the values the paper reports. Table 2's 14.59 matches the run with the reagents line alone (7.97). The Discussion's 21.79 matches the run with the Results text's per-test cost (21.01). The abstract's 20 matches no printed input.
Why it matters
- The abstract misstates the paper's headline result. The abstract's Results sentence is the only place in the abstract that quantifies the comparison. Its Xpert and TB-LAMP costs appear in neither model run.
- The cost ordering depends on which Xpert test cost is used, and the paper uses two. In Table 2 (Xpert test cost 7.97), Xpert is the cheapest strategy (US$14.59). TB-LAM costs US$2.19 more per case detected and is preferred because it is slightly more effective (0.5327 vs 0.521), at an ICER of about US$187 per additional case. With the per-test cost the Results text gives (21.01), Xpert costs US$21.80 and TB-LAM is both cheaper and more effective, so it dominates. That is the Discussion's statement that TB-LAM "outperformed Xpert MTB/RIF Ultra and TB LAMP in terms of both cost and diagnostic yield". These are different economic findings, and the paper presents the first as its result (Table 2, Figure 1, the $187 tornado baseline line) and the second in its Discussion. The 7.97 used in Table 2 is the Xpert reagents price alone. The LAM branch uses LAM's full cost (11.92), and Table 1B lists supply, labour and equipment costs for Xpert as well, so Table 2 probably understates Xpert's cost relative to TB-LAM. The abstract's figures also put TB-LAM below Xpert, but they match neither run.
- The reported ICERs disagree. Table 2 and the tornado plot's baseline line give US$187 for TB-LAM vs Xpert (Figure 1: 186.55). The Discussion gives US$124.72 "for TB LAM". With the Discussion's own costs, TB-LAM would be dominant, with no positive ICER against Xpert. The only combination of the paper's strategy-level costs and effects that reproduces 124.72 pairs the LAMP–LAM cost difference with a LAMP effect (0.506) that appears only in the Discussion.
- The recommendation does not change. At the paper's willingness to pay of US$5280, TB-LAM has the highest NMB under the abstract's costs (2795.7 vs 2730.9 and 2662.9), under Table 2's costs (2796.0 vs 2736.5 and 2664.9) and with Xpert at 21.80 (Xpert 2729.3). The NMB differences come almost entirely from the effect column, so a few dollars of cost cannot reorder them. This holds on the paper's own effect measure. See the side observations for what that measure counts. The issue is misreported key results and an Xpert cost input that is inconsistent with the paper's own cost table, not an overturned conclusion.
Explanations considered
- Rounding. This accounts for TB-LAM only (16.78 rounds to 17). Xpert's 14.59 rounds to 15 and TB-LAMP's 20.11 rounds to 20. No rounding convention turns them into 20 and 22.
- A different metric in the abstract, such as cost per test or incremental cost. Ruled out. The abstract says "per TB case detected", the same metric as Table 2 and the Results text. The per-test costs (LAM 11.92, LAMP 17.62, Xpert 7.97, 19.09 or 21.01) do not give 17, 20 and 22. The incremental costs in Table 2 (2.19 and 3.33) do not either.
- The abstract reports the run with the full Xpert test cost. Partly fits. In that run, Xpert is US$21.80, TB-LAM US$16.78 and TB-LAMP US$20.11, which round to 22, 17 and 20. The abstract has 17, 20 and 22, so the TB-LAM figure fits and the Xpert and TB-LAMP figures fit only with their labels exchanged. The tree with the Table 1B total of 19.09 gives Xpert US$20.74, which rounds to 21, not 20.
- Two model runs with different Xpert test costs. Confirmed from the paper's own figure. Table 2 and Figure 1 use an Xpert test cost of 7.97, the Table 1B reagents line. The Discussion's US$21.79 and Table 3's Xpert value of 2729.29 match the same tree run with 21.01, the per-test cost in the Results text. The Discussion's effects for LAM and LAMP (0.532, 0.506) also differ slightly from Table 2's (0.5327, 0.5085). The paper does not say which run is the base case, and it does not explain why 7.97 was used. Table 1B and the Results text both give a higher Xpert per-test cost.
- Probabilistic (mean) rather than deterministic results. Ruled out for the main numbers. Table 2 and Table 3's Xpert and LAM values are deterministic tree outputs, reproduced to the cent. Only Table 3's LAMP value (2678.22) matches no run, since it implies a LAMP cost of 6.81, below the CXR cost alone. That does not produce the abstract's figures either.
- A text-conversion artifact. Unlikely. The same numbers appear in the PMC HTML, the Europe PMC full-text XML and the PubMed abstract. The figures are plain digits with no superscripts or special characters to lose, and the Figure 1 image carries the same Table 2 values.
- Already corrected. No correction found (see Status).
- Side observations, not part of the main finding.
- Prevalence. The published tree uses a TB prevalence of 0.20. Its root values (0.2 × 12.4316 + 0.8 × 15.1414 = 14.5994; 0.2 × 0.986 + 0.8 × 0.4048 = 0.521) and its counts (200 with TB per 1000) reproduce only at that value. Table 1A, the Results text and the Discussion give 0.22 (22%). At 0.22 the Xpert root would be 14.5452.
- TB-LAMP inputs. The LAMP node (28.16 = 10.20 + 17.96) implies a LAMP test cost of 17.96, against Table 1B's 17.62. The tree uses a LAMP sensitivity of 0.638. Table 1A gives 0.64, and the cohort figure (1404, 63.8%) and the online supplement use 0.638.
- TB-LAM inputs. The tree's LAM node shows a sensitivity of 0.707 ("(c) 0.707") and a false-positive rate of 0.091, and the LAM branch values reproduce only with these. Table 1A gives a sensitivity of 0.701 and a specificity of 0.91 (false-positive rate 0.09), and the Results cohort (1542 = 2200 × 0.701) and the supplement use 0.701.
- Xpert per-test cost. Table 1B's Xpert total (19.09) is not the sum of its own components (21.89). The Results text gives a third value (21.01). The LAM and LAMP totals equal their components.
- What the effect column counts. In Figure 1 every positive result scores one effect unit, including a positive CXR and positive tests among people without TB. Of Xpert's effect of 0.521, 0.1972 comes from people with TB and 0.3238 from people without TB. TB-LAM's higher effect than Xpert (+0.0117) is the sum of −0.0136 among people with TB and +0.0253 among people without TB. Among people with TB, Xpert detects more, as the paper's cohort numbers (2090 vs 1542) show. The Figure 2 legend defines post_case as the "proportion of true-positive cases initiated on treatment".
- What the cost column is. The root value of the tree is the expected cost per person entering the pathway. It is not divided by the cases detected, although the Results text describes it as an average cost per TB case detected that "accounts for the cumulative testing required to detect each true TB case". Divided by the effect, the tree gives US$28.02 per positive result for Xpert, US$31.50 for TB-LAM and US$39.55 for TB-LAMP.
- Labels. The abstract calls the per-case costs "ICERs", which they are not. Table 3 shows strategy values under a column headed "Expected value of perfect information (EVPI)".
Status
- Version of record. The publisher's site (gh.bmj.com, through the doi.org redirect and at the article URL) returned HTTP 403 to automated requests, so I could not read BMJ's own HTML or PDF. I checked three copies of the publisher-deposited text instead: the PMC HTML page for PMC12336617, the Europe PMC full-text XML, and the PubMed abstract (PMID 40763982). All three carry the numbers quoted here. Every text quotation was checked verbatim against the PMC-archived text with
src/show.py. - Figures. I read the Figure 1 (decision tree) and Figure 2 (tornado plot) images as deposited in PMC (
bmjgh-10-8-g001.jpg,bmjgh-10-8-g002.jpg, from the Europe PMC supplementary bundle). The Figure 1 node values quoted above are transcribed from the image and were checked by rebuilding the tree inrecompute.py.txt. Every branch node reproduces. The tornado plot's horizontal axis is labelled "Cost per EV ($)". Its bars can only be read approximately from the image, so I do not rely on it for any number. Read approximately, the Xpert Ultra bars for the low-influence parameters (prev, cxr_sens, post_case, ultra_sens, ultra_spec) sit at about 21 to 22 and none lies near 14.59, which fits the 21.01 run (Xpert 21.80), not the 7.97 run. The TB-LAM bars sit at about 16 to 21. Only the labelled dashed baseline line ($187) matches the 7.97 run. - Supplements. The only supplement, online supplemental file 1 (
bmjgh-10-8-s001.docx, taken from the Europe PMC supplementary bundle; its checksum matches the one in the article XML), holds one table of cases detected and missed in a hypothetical cohort of 10 000 (TB-LAM 0.701 / 1542 / 658, Xpert 0.95 / 2090 / 110, TB-LAMP 0.638 / 1404 / 796). It has no cost figures. No model file (the Amua tree) or source data are published, although the Data Availability statement says data are in a public repository. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations, PMC notices or Europe PMC notices. The Crossref record has empty update-to, updated-by and relation fields, and a Crossref search for updates to this DOI returns none. BMJ-style correction DOIs (…019270corr1,…corr2) do not resolve. The PubMed record has no CommentsCorrections entries. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title with "PubPeer" found no thread. - Sources. PMC article PMC12336617, including the Figure 1 (decision tree) and Figure 2 (tornado plot) images; Europe PMC full-text XML and supplementary files; PubMed 40763982; Crossref; PubPeer; https://doi.org/10.1136/bmjgh-2025-019270.
Suggested correction
The authors should confirm which Xpert MTB/RIF Ultra test cost the model is meant to use. The Figure 1 tree, and therefore Table 2, the Results text, the ICER of $187 and the tornado plot's baseline line, uses 7.97, which is Table 1B's reagents line alone. Table 1B gives a total of 19.09 (its components sum to 21.89), and the Results text gives 21.01. The TB-LAM branch uses TB-LAM's full per-test cost. Once the Xpert input is settled, the abstract, Table 2, the Results text, Figure 1 and the tornado plot (its bars and its baseline line) should all report the same run. On the paper's own tree, an Xpert test cost of 21.01 gives US$21.80 per case for Xpert, which matches the Discussion's US$21.79 and Table 3. With that cost TB-LAM (US$16.78) would be cheaper and more effective than Xpert, so it would dominate rather than have an ICER of $187. An Xpert test cost of 19.09 gives US$20.74. In either case the abstract's Xpert and TB-LAMP figures (US$20 and US$22) should be replaced by the values of the chosen run; neither run gives them as labelled. The Discussion's ICER of US$124.72 matches no comparison of TB-LAM with Xpert and should be recomputed. The prevalence used in the tree (0.20) should be reconciled with the 0.22 stated in Table 1A and the text, along with the LAMP test cost (17.96 in the tree, 17.62 in Table 1B), the LAMP sensitivity (0.638 vs 0.64), the TB-LAM sensitivity (0.707 in the tree; 0.701 in Table 1A, the Results cohort and the supplement), the TB-LAM false-positive rate (0.091 in the tree; 1 − 0.91 = 0.09 from Table 1A) and the Table 1B Xpert total (19.09 vs a component sum of 21.89).
Recomputation script (python recompute.py.txt)
"""Recompute the cost-per-case figures of PMC12336617
(BMJ Glob Health 2025, doi:10.1136/bmjgh-2025-019270,
"Cost-effectiveness of different tuberculosis diagnostic approaches in Nigeria
based on decision analytical modelling").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract (Results): LAM US$17, Xpert MTB/RIF Ultra US$20, TB-LAMP US$22 per TB case detected.
Results text and Table 2: cost per case Xpert 14.59, LAM 16.78, LAMP 20.11; effects
0.521, 0.5327, 0.5085; increments 2.19 / 0.0117 and 3.33 / -0.0242; ICERs 187.1794 and
-137.60; NMB 2736.49, 2796.02, 2664.91; willingness to pay US$5280 (3 x GDP per capita 1760).
Discussion: LAM US$16.78, effectiveness 0.532; Xpert US$21.79, effectiveness 0.521;
LAMP US$20.11, effectiveness 0.506; "The ICER for TB LAM was calculated at US$124.72".
Table 3 (column headed EVPI): LAMP_algo 2678.22, LAM_algo 2796.02, Xpertgene_Ultra_algo 2729.29.
Table 1B per-test costs: Xpert components 7.97 + 5.11 + 5.92 + 2.89, total 19.09;
LAM 3.5 + 2.5 + 5.92 + 0 = 11.92; LAMP 6 + 5.4 + 5.92 + 0.3 = 17.62; CXR 10.2.
Results text per-test costs: LAM 11.92, LAMP 17.62, Xpert 21.01.
Table 1A sensitivities: LAM 0.701, Xpert 0.95, LAMP 0.64; prevalence 0.22.
(Figure 1 uses LAM sensitivity 0.707 and false-positive rate 0.091; LAMP sensitivity 0.638.)
Results cohort: 10 000 PLWH, ~2200 TB cases; LAM 1542/658, Xpert 2090/110, LAMP 1404 (63.8%)/796.
Table 1A: CXR sensitivity 0.72, specificity 0.62; Xpert specificity 0.96; LAM 0.91; LAMP 0.95.
Figure 1 (decision tree, PMC image bmjgh-10-8-g001.jpg), node labels "($) cost; (c) effect":
Xpert root 14.5994 / 0.521; Active TB 12.4316 / 0.986; No active TB 15.1414 / 0.4048;
CXR-negative test nodes 18.17 / 0.95 (TB) and 18.17 / 0.04 (no TB); "ICER: Baseline".
LAM root 16.7798 / 0.5327; Active TB 13.5376 / 0.918; No active TB 17.5904 / 0.4364;
CXR-negative test nodes 22.12 / 0.707 and 22.12 / 0.091; "ICER: 186.5503".
LAMP root 20.1139 / 0.5085; Active TB 15.2288 / 0.8986; No active TB 21.3352 / 0.411;
CXR-negative test nodes 28.16 / 0.638 and 28.16 / 0.05; "ICER: Strongly Dominated".
Terminal payoffs: CXR+ -> cost_cxr, post_case; CXR- then test+ -> cost_cxr+cost_test, post_case;
CXR- then test- -> cost_cxr+cost_test, neg_case. Cohort counts at the right of each branch,
e.g. Xpert 144.0, 53.2, 2.8 (active TB) and 304.0, 19.84, 476.16 (no active TB).
"""
import itertools
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
S = ["Xpert", "LAM", "LAMP"]
# ------------------------------------------------------------------ inputs
GDP = 1760
WTP = 3 * GDP
ABSTRACT = {"Xpert": 20, "LAM": 17, "LAMP": 22}
T2_COST = {"Xpert": 14.59, "LAM": 16.78, "LAMP": 20.11}
T2_EFF = {"Xpert": 0.521, "LAM": 0.5327, "LAMP": 0.5085}
T2_NMB = {"Xpert": 2736.49, "LAM": 2796.02, "LAMP": 2664.91}
T2_DCOST = {"LAM": 2.19, "LAMP": 3.33}
T2_DEFF = {"LAM": 0.0117, "LAMP": -0.0242}
T2_ICER = {"LAM": 187.1794, "LAMP": -137.60}
DISC_COST = {"Xpert": 21.79, "LAM": 16.78, "LAMP": 20.11}
DISC_EFF = {"Xpert": 0.521, "LAM": 0.532, "LAMP": 0.506}
DISC_ICER = 124.72
T3_VAL = {"Xpert": 2729.29, "LAM": 2796.02, "LAMP": 2678.22}
T1B_COMP = {"Xpert": [7.97, 5.11, 5.92, 2.89], "LAM": [3.5, 2.5, 5.92, 0], "LAMP": [6, 5.4, 5.92, 0.3]}
T1B_TOTAL = {"Xpert": 19.09, "LAM": 11.92, "LAMP": 17.62}
TEXT_PER_TEST = {"Xpert": 21.01, "LAM": 11.92, "LAMP": 17.62}
CXR = 10.2
SENS = {"LAM": 0.701, "Xpert": 0.95, "LAMP": 0.64}
COHORT_TB = 2200
COHORT_REP = {"LAM": (1542, 658), "Xpert": (2090, 110), "LAMP": (1404, 796)}
def nmb(c, e):
return WTP * e - c
print(f"Willingness to pay = 3 x {GDP} = {WTP}")
print("\n=== 1. Abstract vs Table 2: cost per TB case detected ===")
for k in S:
print(f"{k:6s} abstract US${ABSTRACT[k]:>3} Table 2 US${T2_COST[k]:6.2f} -> rounds to {round(T2_COST[k]):>3}"
f" difference abstract - Table 2 = {ABSTRACT[k] - T2_COST[k]:+.2f}"
f" | Discussion US${DISC_COST[k]:.2f} -> rounds to {round(DISC_COST[k])}")
print("Cost ordering, abstract: ", " < ".join(sorted(S, key=ABSTRACT.get)))
print("Cost ordering, Table 2: ", " < ".join(sorted(S, key=T2_COST.get)))
print("Cost ordering, Discussion: ", " < ".join(sorted(S, key=DISC_COST.get)))
print("\n=== 2. Table 2 internal consistency ===")
for k in ["LAM", "LAMP"]:
ref = "Xpert" if k == "LAM" else "LAM"
dc = T2_COST[k] - T2_COST[ref]
de = T2_EFF[k] - T2_EFF[ref]
print(f"{k} vs {ref}: dCost {dc:.2f} (printed {T2_DCOST[k]}), dEffect {de:.4f} (printed {T2_DEFF[k]}),"
f" ICER {dc/de:.4f} (printed {T2_ICER[k]})")
for k in S:
print(f"NMB {k:6s} = {WTP} x {T2_EFF[k]} - {T2_COST[k]} = {nmb(T2_COST[k], T2_EFF[k]):.2f}"
f" (printed {T2_NMB[k]}; difference {T2_NMB[k] - nmb(T2_COST[k], T2_EFF[k]):+.2f})")
print(f"(effects printed to 4 decimals: +/-0.00005 x {WTP} = +/-{0.00005*WTP:.3f} in NMB)")
print("\n=== 3. Costs implied by printed NMB-type values: cost = WTP x effect - value ===")
IMPL_T2 = {k: WTP * T2_EFF[k] - T2_NMB[k] for k in S}
IMPL_T3 = {k: WTP * T2_EFF[k] - T3_VAL[k] for k in S}
for k in S:
print(f"{k:6s} from Table 2 NMB {T2_NMB[k]}: {IMPL_T2[k]:.2f} from Table 3 value {T3_VAL[k]}: {IMPL_T3[k]:.2f}")
print("(implied costs move by up to 0.26 because Table 2 rounds effects to 3-4 decimals; see section 7)")
print(f"LAMP Table 3 value with Discussion effect 0.506: {WTP*0.506 - T3_VAL['LAMP']:.2f}")
print(f"Xpert NMB with Discussion cost 21.79 and effect 0.521: {nmb(21.79, 0.521):.2f} (Table 3: 2729.29)")
print("\n=== 4. The Discussion's ICER of US$124.72 ===")
dc = DISC_COST["LAM"] - DISC_COST["Xpert"]
de = DISC_EFF["LAM"] - DISC_EFF["Xpert"]
print(f"LAM vs Xpert with Discussion values: dCost {dc:.2f}, dEffect {de:.4f}, ratio {dc/de:.2f}"
" (LAM cheaper and more effective: dominant)")
cost_pool = {"Xpert T2": 14.59, "LAM": 16.78, "LAMP": 20.11, "Xpert Disc": 21.79, "abs 17": 17, "abs 20": 20,
"abs 22": 22, "LAM/test": 11.92, "LAMP/test": 17.62, "Xpert/test text": 21.01,
"Xpert/test table": 19.09, "CXR": 10.2,
"Xpert/reagents": 7.97}
eff_pool = {"Xpert": 0.521, "LAM T2": 0.5327, "LAMP T2": 0.5085, "LAM Disc": 0.532, "LAMP Disc": 0.506}
hits = []
for (a, ca), (b, cb) in itertools.permutations(cost_pool.items(), 2):
for (e1, v1), (e2, v2) in itertools.permutations(eff_pool.items(), 2):
if v1 != v2:
r = (ca - cb) / (v1 - v2)
if abs(r - DISC_ICER) < 0.05:
hits.append((a, b, e1, e2, r))
uniq = {}
for h in hits: # (a-b)/(e1-e2) and (b-a)/(e2-e1) are the same ratio
uniq.setdefault(frozenset([(h[0], h[2]), (h[1], h[3])]), h)
print(f"All (cost_a - cost_b)/(eff_1 - eff_2) over printed values within 0.05 of 124.72: {len(hits)} found,"
f" {len(uniq)} distinct after merging sign-flipped duplicates")
for h in uniq.values():
print(f" ({h[0]} - {h[1]}) / ({h[2]} - {h[3]}) = {h[4]:.4f}")
print(f"i.e. (20.11 - 16.78) / (0.5327 - 0.506) = {3.33/0.0267:.2f}")
print("(this search covers only the strategy-level costs, per case and per test, and the five strategy-level"
" effects listed above)")
# Widened pool: node-level and component values. Coincidental near-matches appear once these are included,
# so the strategy-level search above is the only one that supports a specific reading of 124.72.
wide_cost = dict(cost_pool)
wide_cost.update({"T2 dCost LAM": 2.19, "T2 dCost LAMP": 3.33, "Xpert root": 14.5994, "Xpert tb": 12.4316,
"Xpert notb": 15.1414, "Xpert node": 18.17, "LAM root": 16.7798, "LAM tb": 13.5376,
"LAM notb": 17.5904, "LAM node": 22.12, "LAMP root": 20.1139, "LAMP tb": 15.2288,
"LAMP notb": 21.3352, "LAMP node": 28.16, "Xpert supply": 5.11, "labour": 5.92,
"Xpert equip": 2.89, "LAM reag": 3.5, "LAM supply": 2.5, "LAMP reag": 6, "LAMP supply": 5.4,
"LAMP equip": 0.3, "T2 NMB Xpert": 2736.49, "T2 NMB LAM": 2796.02, "T2 NMB LAMP": 2664.91,
"T3 Xpert": 2729.29, "T3 LAMP": 2678.22, "zero": 0.0})
wide_eff = dict(eff_pool)
wide_eff.update({"Xpert tb e": 0.986, "Xpert notb e": 0.4048, "Xpert sens": 0.95, "Xpert fpos": 0.04,
"LAM tb e": 0.918, "LAM notb e": 0.4364, "LAM node sens": 0.707, "LAM fpos": 0.091,
"LAMP tb e": 0.8986, "LAMP notb e": 0.411, "LAMP node sens": 0.638, "LAMP fpos": 0.05,
"T1A LAM sens": 0.701, "T1A LAMP sens": 0.64, "Xpert spec": 0.96, "LAM spec": 0.91,
"LAMP spec": 0.95, "CXR sens": 0.72, "CXR spec": 0.62, "prev 0.22": 0.22, "prev 0.20": 0.20,
"T2 dEff LAM": 0.0117, "T2 dEff LAMP": -0.0242})
wide = {}
for (a, ca), (b, cb) in itertools.combinations(wide_cost.items(), 2):
for (e1, v1), (e2, v2) in itertools.permutations(wide_eff.items(), 2):
if v1 != v2:
for num, nm in ((ca - cb, f"{a} - {b}"), (cb - ca, f"{b} - {a}")):
r = num / (v1 - v2)
if abs(r - DISC_ICER) < 0.05:
wide.setdefault(frozenset([(a, e1), (b, e2)]) if num == ca - cb else
frozenset([(b, e1), (a, e2)]), (nm, e1, e2, r))
print(f"Widened pool ({len(wide_cost)} costs incl. Figure 1 nodes, Table 1B components, increments, NMB,"
f" Table 3 and zero; {len(wide_eff)} probabilities incl. node effects, sensitivities, specificities,"
f" prevalences): {len(wide)} distinct ratios within 0.05 of 124.72, e.g.")
for nm, e1, e2, r in sorted(wide.values(), key=lambda h: abs(h[3] - DISC_ICER))[:6]:
print(f" ({nm}) / ({e1} - {e2}) = {r:.4f}")
print("\n=== 5. NMB ranking under each set of costs (Table 2 effects) ===")
for lab, cs in [("Abstract", ABSTRACT), ("Table 2", T2_COST),
("Discussion Xpert 21.79", {"Xpert": 21.79, "LAM": 16.78, "LAMP": 20.11})]:
v = {k: nmb(cs[k], T2_EFF[k]) for k in S}
print(f"{lab:24s} " + " ".join(f"{k} {v[k]:.1f}" for k in S) + f" -> highest: {max(v, key=v.get)}")
print("NMB with Discussion costs and effects: " +
" ".join(f"{k} {nmb(DISC_COST[k], DISC_EFF[k]):.1f}" for k in S))
print(f"LAM - Xpert: dCost Table 2 {T2_COST['LAM']-T2_COST['Xpert']:+.2f}, abstract {ABSTRACT['LAM']-ABSTRACT['Xpert']:+d},"
f" Discussion {DISC_COST['LAM']-DISC_COST['Xpert']:+.2f}")
print("\n=== 6. Table 1B per-test costs ===")
for k in S:
print(f"{k:6s} components {T1B_COMP[k]} sum {sum(T1B_COMP[k]):.2f}; table total {T1B_TOTAL[k]}; Results text {TEXT_PER_TEST[k]}")
print("\n=== 7. Rebuild the Figure 1 decision tree ===")
CXR_SENS, CXR_SPEC = 0.72, 0.62
FIG1 = {"Xpert": dict(root=(14.5994, 0.521), tb=(12.4316, 0.986), notb=(15.1414, 0.4048), test_node=18.17,
sens=0.95, fpos=0.04),
"LAM": dict(root=(16.7798, 0.5327), tb=(13.5376, 0.918), notb=(17.5904, 0.4364), test_node=22.12,
sens=0.707, fpos=0.091),
"LAMP": dict(root=(20.1139, 0.5085), tb=(15.2288, 0.8986), notb=(21.3352, 0.411), test_node=28.16,
sens=0.638, fpos=0.05)}
def tree(cost_test, sens, fpos, prev=0.20, cxr=CXR):
"""Expected cost and effect, rolled back as in Figure 1 (every positive result scores 1)."""
c_tb = CXR_SENS * cxr + (1 - CXR_SENS) * (cxr + cost_test)
c_no = (1 - CXR_SPEC) * cxr + CXR_SPEC * (cxr + cost_test)
e_tb = CXR_SENS + (1 - CXR_SENS) * sens
e_no = (1 - CXR_SPEC) + CXR_SPEC * fpos
return dict(c_tb=c_tb, c_no=c_no, e_tb=e_tb, e_no=e_no,
cost=prev * c_tb + (1 - prev) * c_no, eff=prev * e_tb + (1 - prev) * e_no)
TEST_COST = {}
for k in S:
f = FIG1[k]
TEST_COST[k] = round(f["test_node"] - CXR, 2)
print(f"{k}: CXR-negative test node {f['test_node']} = {CXR} + {TEST_COST[k]:.2f} -> cost_test = {TEST_COST[k]:.2f}"
f" (Table 1B total {T1B_TOTAL[k]})")
TREE = {}
for k in S:
f = FIG1[k]
t = tree(TEST_COST[k], f["sens"], f["fpos"])
TREE[k] = t
print(f"{k:6s} active TB {t['c_tb']:.4f}/{t['e_tb']:.4f} (fig {f['tb'][0]}/{f['tb'][1]});"
f" no TB {t['c_no']:.4f}/{t['e_no']:.4f} (fig {f['notb'][0]}/{f['notb'][1]});"
f" root at prev 0.20 {t['cost']:.4f}/{t['eff']:.5f} (fig {f['root'][0]}/{f['root'][1]};"
f" Table 2 {T2_COST[k]}/{T2_EFF[k]})")
x = TREE["Xpert"]
print(f"Xpert root: 0.2 x {x['c_tb']:.4f} + 0.8 x {x['c_no']:.4f} = {x['cost']:.4f};"
f" 0.2 x {x['e_tb']:.3f} + 0.8 x {x['e_no']:.4f} = {x['eff']:.5f}")
icer = (TREE["LAM"]["cost"] - TREE["Xpert"]["cost"]) / (TREE["LAM"]["eff"] - TREE["Xpert"]["eff"])
print(f"ICER LAM vs Xpert from the tree: {icer:.4f} (Figure 1 label 186.5503; Table 2 187.1794 from rounded inputs)")
for k in S:
print(f"NMB {k:6s} from unrounded tree values: {nmb(TREE[k]['cost'], TREE[k]['eff']):.4f} (Table 2 {T2_NMB[k]})")
p_neg = 0.20 * (1 - CXR_SENS) + 0.80 * CXR_SPEC
print(f"P(CXR negative) at prevalence 0.20 = {p_neg:.3f}; strategy cost = {CXR} + {p_neg:.3f} x cost_test")
print("\n--- Xpert cost per case under each printed Xpert test cost (same tree, prevalence 0.20) ---")
XPERT_INPUTS = [("Table 1B reagents line", 7.97), ("Table 1B total", 19.09), ("Results text", 21.01),
("Table 1B component sum", 21.89)]
XPERT_RUNS = {}
for lab, cu in XPERT_INPUTS:
t = tree(cu, 0.95, 0.04)
XPERT_RUNS[cu] = t["cost"]
print(f"cost_ultra {cu:5.2f} ({lab:22s}): Xpert cost {t['cost']:.4f}, NMB {nmb(t['cost'], t['eff']):.2f},"
f" LAM - Xpert cost {TREE['LAM']['cost'] - t['cost']:+.2f}")
print(f"Discussion Xpert US$21.79; Table 3 Xpert 2729.29 = WTP x {x['eff']:.5f} - cost -> cost "
f"{WTP * x['eff'] - 2729.29:.4f} -> cost_ultra {(WTP * x['eff'] - 2729.29 - CXR) / p_neg:.2f}")
print(f"(with Table 2's rounded effect 0.521 the same value gives cost {WTP*0.521 - 2729.29:.2f},"
f" cost_ultra {(WTP*0.521 - 2729.29 - CXR)/p_neg:.2f})")
print(f"cost_ultra needed for the abstract's US$20: {(20 - CXR) / p_neg:.2f}; for US$22: {(22 - CXR) / p_neg:.2f}")
print(f"Table 3 LAMP 2678.22 -> implied cost with the tree effect {TREE['LAMP']['eff']:.5f}:"
f" {WTP * TREE['LAMP']['eff'] - 2678.22:.2f}")
print("\n--- Prevalence: Figure 1 versus Table 1A ---")
for pv in (0.20, 0.22):
t = tree(7.97, 0.95, 0.04, prev=pv)
print(f"prev {pv:.2f}: Xpert root {t['cost']:.4f} / {t['eff']:.4f}")
for k in S:
f = FIG1[k]
print(f"{k:6s} cohort per 1000 implied by Figure 1: TB {1000*0.2:.0f} -> CXR+ {200*CXR_SENS:.1f},"
f" test+ {200*(1-CXR_SENS)*f['sens']:.2f}; no TB 800 -> CXR+ {800*(1-CXR_SPEC):.1f},"
f" test+ {800*CXR_SPEC*f['fpos']:.2f}")
print("\n--- What the effect column counts (Figure 1 scores every positive result) ---")
for k in S:
t = TREE[k]
tp, fp = 0.2 * t["e_tb"], 0.8 * t["e_no"]
print(f"{k:6s} effect {t['eff']:.4f} = with TB {tp:.4f} + without TB {fp:.4f};"
f" cost per positive {t['cost']/t['eff']:.2f}; cost per positive with TB {t['cost']/tp:.2f}")
print(f"LAM - Xpert: with TB {0.2*(TREE['LAM']['e_tb']-TREE['Xpert']['e_tb']):+.4f},"
f" without TB {0.8*(TREE['LAM']['e_no']-TREE['Xpert']['e_no']):+.4f}")
print("\n=== 8. Cohort of 10 000 (2200 TB cases) ===")
for k in ["LAM", "Xpert", "LAMP"]:
d = COHORT_TB * SENS[k]
print(f"{k:6s} sensitivity {SENS[k]}: detected {d:.1f}, missed {COHORT_TB-d:.1f}; printed {COHORT_REP[k]}")
print(f"LAMP printed 1404 / 2200 = {1404/2200:.4f}")
print("Test accuracy in the tree (Figure 1 nodes) vs Table 1A:")
SPEC = {"LAM": 0.91, "Xpert": 0.96, "LAMP": 0.95}
for k in S:
f = FIG1[k]
print(f"{k:6s} tree sens {f['sens']} vs Table 1A {SENS[k]}; tree false-positive {f['fpos']} vs 1 - Table 1A"
f" specificity {1 - SPEC[k]:.3f}")
for sv in (0.707, 0.701):
print(f"LAM active-TB node effect with sensitivity {sv}: 0.72 + 0.28 x {sv} = {0.72 + 0.28*sv:.5f}"
f" (Figure 1: 0.918); cohort 2200 x {sv} = {2200*sv:.1f} (Results: 1542)")
for fp in (0.091, 0.09):
print(f"LAM no-TB node effect with false-positive rate {fp}: 0.38 + 0.62 x {fp} = {0.38 + 0.62*fp:.5f}"
f" (Figure 1: 0.4364)")
# ------------------------------------------------------------------ figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- Figure 1: abstract vs Results/Table 2 text panels
fig = plt.figure(figsize=(7.2, 4.4), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.4)
ax.axis("off")
PW, PY, PH = 2.45, 0.85, 2.95
def panel(x, title, sub, rows, lab):
ax.add_patch(FancyBboxPatch((x, PY), PW, PH, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=SEC, lw=1.2))
ax.text(x + 0.15, PY + PH - 0.22, title, fontsize=12, fontweight="bold", va="top")
ax.text(x + 0.15, PY + PH - 0.55, sub, fontsize=10, color=SEC, va="top")
ys = {}
for i, (name, val) in enumerate(rows):
yy = PY + PH - 1.15 - i * 0.6
ax.text(x + 0.15, yy, name, fontsize=10.5, va="center")
ax.text(x + PW - 0.12, yy, val, fontsize=12.5, va="center", ha="right", color=ORANGE,
fontweight="bold")
ys[name] = yy
ax.text(x + 0.15, PY + 0.2, lab, fontsize=10, color=SEC, va="center")
return ys
ax.text(0.2, 4.15, "Cost per TB case detected (US$), as printed in two places of the same paper",
fontsize=11.5, fontweight="bold", va="center")
rows_a = [("TB-LAM", "17"), ("Xpert Ultra", "20"), ("TB-LAMP", "22")]
rows_b = [("TB-LAM", "16.78"), ("Xpert Ultra", "14.59"), ("TB-LAMP", "20.11")]
XA, XB = 0.2, 4.55
ya = panel(XA, "Abstract", "Results sentence", rows_a, "Order: LAM < Xpert < LAMP")
yb = panel(XB, "Results text / Table 2", "Model output, as in Figure 1", rows_b, "Order: Xpert < LAM < LAMP")
notes = {"TB-LAM": ("agrees", "16.78 \u2248 17"),
"Xpert Ultra": ("conflict", "14.59 \u2248 15, not 20"),
"TB-LAMP": ("conflict", "20.11 \u2248 20, not 22")}
xm = (XA + PW + XB) / 2
for name in ya:
agree = name == "TB-LAM"
ax.add_patch(FancyArrowPatch((XA + PW + 0.05, ya[name]), (XB - 0.05, yb[name]), arrowstyle="<->",
mutation_scale=11, lw=1.2 if agree else 2,
color=SEC if agree else TXT, linestyle="--" if agree else "-"))
top, bot = notes[name]
ax.text(xm, ya[name] + 0.08, top, fontsize=10, ha="center", va="bottom",
color=SEC if agree else TXT, fontweight="normal" if agree else "bold")
ax.text(xm, ya[name] - 0.08, bot, fontsize=10, ha="center", va="top", color=SEC)
ax.text(0.2, 0.42, "All numbers are printed in the paper (orange). Arrows join the same strategy.", fontsize=10,
color=SEC, va="center")
ax.text(0.2, 0.17, "The abstract's Xpert and TB-LAMP values fit no rounding of Table 2; "
"LAM and Xpert swap order.", fontsize=10, color=SEC, va="center")
save(fig, "fig1")
# ---- Figure 2: Xpert cost per case, reported vs the Figure 1 tree rerun with each printed Xpert test cost
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
ax.set_facecolor(BG)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(SEC)
rows = [("Tree, Xpert test 7.97\n(Table 1B reagents line)", XPERT_RUNS[7.97], ("Table 2 / Figure 1", 14.59, "s")),
("Tree, Xpert test 19.09\n(Table 1B total)", XPERT_RUNS[19.09], None),
("Tree, Xpert test 21.01\n(Results text)", XPERT_RUNS[21.01], ("Discussion", 21.79, "D")),
("Tree, Xpert test 21.89\n(Table 1B component sum)", XPERT_RUNS[21.89], None),
("Abstract", None, ("Abstract", 20, "o"))]
n = len(rows)
for idx, (lab, val, rep) in enumerate(rows):
y = n - 1 - idx
if val is not None:
ax.plot(val, y + 0.13, "o", ms=9, color=BLUE, zorder=4)
ax.text(val - 0.6, y + 0.13, f"{val:.2f}", color=BLUE, fontsize=10, ha="right", va="center")
if rep is not None:
rl, rv, mk = rep
filled = mk == "o"
ax.plot(rv, y - 0.13, mk, ms=10, mfc=ORANGE if filled else "white", mec=ORANGE, mew=2, zorder=5)
ax.text(rv - 0.7, y - 0.13, f"{rl}: {rv:g}", color=ORANGE, fontsize=10, ha="right", va="center")
ax.annotate(f"needs a test cost of {(20 - CXR) / p_neg:.2f},\nprinted nowhere", (20, -0.13), xytext=(18.6, 0.5),
fontsize=10, color=SEC, va="center", ha="right", arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
ax.set_yticks(range(n))
ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10, color=TXT)
ax.tick_params(axis="y", length=0)
ax.set_xlim(0, 25)
ax.set_ylim(-0.6, n - 0.4)
ax.set_xlabel("Xpert Ultra cost per TB case detected (US$)", fontsize=10.5)
handles = [Line2D([], [], ls="", marker="o", ms=8, color=BLUE,
label=f"Recomputed: Figure 1 tree ({CXR:.2f} + {p_neg:.3f} \u00d7 Xpert test cost)"),
Line2D([], [], ls="", marker="s", ms=8, mfc="white", mec=ORANGE, mew=2, label="Reported: Table 2"),
Line2D([], [], ls="", marker="D", ms=8, mfc="white", mec=ORANGE, mew=2, label="Reported: Discussion"),
Line2D([], [], ls="", marker="o", ms=8, color=ORANGE, label="Reported: abstract")]
fig.legend(handles=handles[:1], loc="upper left", fontsize=10, frameon=False, bbox_to_anchor=(0.01, 1.0))
fig.legend(handles=handles[1:], loc="upper left", fontsize=10, frameon=False, ncol=3, bbox_to_anchor=(0.01, 0.94))
fig.tight_layout(rect=(0, 0, 1, 0.86))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract's caregiver-burden results match 14 of the 17 studies analysed in Results and Figure 3
BMJ Glob Health (2026) · doi:10.1136/bmjgh-2024-016028 · PMC13059920
Summary
This systematic review reports a DerSimonian–Laird meta-analysis of caregiver burden, measured with the Zarit Burden Interview (ZBI), which is its primary outcome. The Abstract gives a pooled mean difference (MD) of 8.7 (95% CI 4.1 to 13.3) and delivery-mode subgroups of 10 + 2 + 2 = 14 studies. The Results text and Figure 3 give MD 8.1 (4.0 to 12.2) from 17 studies, split 10 + 4 + 3, with different remote and hybrid estimates. Re-pooling the 17 study rows printed in Figure 3 reproduces the Results figures. Leaving out the three studies dated 2024–2025 reproduces every Abstract figure, and no other way of dropping three studies comes close. The Abstract therefore appears to describe an analysis made before the 2025 search update. The pooled effect stays positive and statistically significant either way, but the Abstract overstates it by 0.6 points. It also presents the hybrid subgroup as significant (5.8 to 36.0) when the paper's own analysis gives an interval that includes zero (−2.7 to 31.7).
What the paper reports
Abstract, Findings:
NPIs in LMICs have a significant improvement post intervention on caregiver burden (MD 8·7, 95% CI 4·1 to 13·3; I2=89·9%)
Interventions were delivered in person (10 studies; MD 6.4, 95% CI 1.4 to 11.5; I² = 88.7%), remotely (two studies; MD 8.0, 95% CI 1.9 to 14.1; I² = 37.5%) or as a hybrid of both (two studies; MD 20.9, 95% CI 5.8 to 36.0; I² = 90.1%), with all showing improvement in caregiver burden except for two studies.
Key-messages box ("What this study adds"). This box appears in the publisher-supplied JATS XML (Europe PMC) but not in the PMC text extract:
We identified 32 studies (35 reports), including pooled results from 14 studies (2664 participants) using the Zarit Burden Interview.
Results, "Meta-analysis of the effects of NPIs":
Of the original 21 studies assessing the effects of NPIs on caregiver burden, 17 (2011 participants) contributed data to the meta-analysis using the ZBI.
Results indicated that NPIs in LMICs have a significant improvement on caregiver burden immediately or to first follow-up post intervention (MD 8·1, 95% CI 4·0 to 12·2); although there was substantial heterogeneity between studies (I2=89·0%), there was consistency in direction of effects with 15 suggesting an improvement in the ZBI (figure 2).
Overall, 11 studies were determined to have low risk of bias or some concern of bias (MD 6.7, 95% CI 1·9 to 11·5; I2=86·6%) and high risk of bias for the remaining six
10 studies were delivered in person (MD 6·4, 95% CI 1·4 to 11·5; I2=88·7%), four remotely (MD 7.6, 95% CI 4.2 to 11.1; I2=0.0%) and three a combination of inperson and remote (MD 14.5, 95% CI -2.7 to 31.7; I2=95·8%).
There were significant differences between delivery method subgroups (p<0·001)
Methods, "Search strategy and selection criteria", and "Data synthesis and statistical methods":
Searches were conducted in April–July 2021 and updated in April–May 2023 and re-updated in June 2025.
Meta-analyses were conducted using DerSimonian-Laird random-effects models
Discussion:
There were no clear patterns of intervention effect based on delivery mode
Figure 3 ("Forest plot of Zarit Burden Interview and intervention delivery modes") is an image. It lists 17 studies: 10 in person (dated 2009–2023), 4 remote (two dated 2021 and 2022, two dated 2024) and 3 hybrid (two dated 2019, one dated 2025). It prints subgroup results of 6.4 (1.4, 11.5), I² 88.7%; 7.6 (4.2, 11.1), I² 0.0%; and 14.5 (−2.7, 31.7), I² 95.8%, with an overall result of 8.1 (4.0, 12.2), I² 89.0%, and "Heterogeneity between groups: p < 0.001". These are the Results values.
What the paper's own numbers show
Method. I transcribed the MD and 95% CI of each of the 17 study rows in Figure 3. Each study's standard error was recovered as (upper − lower) / (2 × 1.96). I then pooled the rows with the DerSimonian–Laird random-effects model that the Methods name, overall and by delivery mode. This was done twice: once with all 17 rows, and once with only the 14 rows dated 2023 or earlier, which leaves out the two remote studies dated 2024 and the hybrid study dated 2025. To test whether that subset is the only one that fits, I pooled all 18 ways of keeping the 10 in-person studies, 2 of the 4 remote studies and 2 of the 3 hybrid studies, as the Abstract's counts require. Rounding was checked by perturbing every printed input by up to ±0.05. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Studies in the delivery-mode subgroups | Abstract | 10 + 2 + 2 | sum = 14 |
| Studies in the delivery-mode subgroups | Results; Figure 3 | 10 + 4 + 3 | sum = 17, which matches the 17 studies in the ZBI meta-analysis and the 11 + 6 risk-of-bias split |
| Studies in the pooled ZBI analysis | Key-messages box | 14 studies (2664 participants) | 2664 is the participant total for all 32 included studies (Results, Discussion); the ZBI analysis is stated as 17 studies, 2011 participants |
| Overall ZBI MD, all 17 rows | Results; Figure 3 | 8.1 (4.0 to 12.2), I² 89.0% | 8.11 (4.00 to 12.22), I² 89.0% |
| Overall ZBI MD | Abstract | 8.7 (4.1 to 13.3), I² 89.9% | not reproducible from 17 rows; 14 rows dated ≤ 2023 give 8.69 (4.06 to 13.32), I² 90.0% |
| In person | Abstract; Results; Figure 3 | 10 studies, 6.4 (1.4 to 11.5), I² 88.7% (identical in both) | 6.44 (1.41 to 11.46), I² 88.7% (all 10 are dated ≤ 2023) |
| Remote | Results; Figure 3 | 4 studies, 7.6 (4.2 to 11.1), I² 0.0% | 7.62 (4.14 to 11.11), I² 0.0% |
| Remote | Abstract | 2 studies, 8.0 (1.9 to 14.1), I² 37.5% | the 2 remote studies dated 2021–2022 give 8.00 (1.94 to 14.06), I² 36.3% (34.5–37.9% under input rounding) |
| Hybrid | Results; Figure 3 | 3 studies, 14.5 (−2.7 to 31.7), I² 95.8% | 14.52 (−2.74 to 31.77), I² 95.9% |
| Hybrid | Abstract | 2 studies, 20.9 (5.8 to 36.0), I² 90.1% | the 2 hybrid studies dated 2019 give 20.95 (5.78 to 36.12), I² 90.2% |
| Other ways to keep 2 remote + 2 hybrid studies | derived | — | of 18 subsets, only the "dated ≤ 2023" one matches all nine Abstract MD/CI values (largest gap 0.12); the next best is off by 1.97 |
| Rounding range, overall MD | derived | 8.1 (Results), 8.7 (Abstract) | 8.09–8.12 for 17 rows; 8.67–8.70 for 14 rows |
| Implied two-sided P, overall | derived | CI excludes 0 in both | Abstract P = 0.00021; Results P = 0.00011 |
| Implied two-sided P, hybrid | derived | Abstract CI excludes 0; Results CI includes 0 | Abstract P = 0.0067; Results P = 0.098 |
| Between-subgroup heterogeneity | Results; Figure 3 | p < 0.001 | fixed-effect Q decomposition: Q = 15.8, df = 2, P = 0.00036, which is consistent; a test on the three random-effects subgroup means gives P = 0.67 |
| Studies with no improvement | Abstract; Results | "except for two"; 15 suggest improvement | 2 rows with MD ≤ 0 and 15 with MD > 0 among the 17; the same 2 rows are among the 14 dated ≤ 2023 |
The 17-row pooling reproduces the Results values to within the 0.1 rounding of the printed inputs. The 14-row pooling does the same for the Abstract values. The one remaining gap of about 0.1, in the upper limit of the hybrid subgroup, appears in both sets (36.12 vs 36.0 and 31.77 vs 31.7), so it comes from input rounding, not from a difference between the sets. The Abstract's numbers for distress (NPIQ, MD 4.3) and depression (BDI, MD 5.9) are the same as those in the Results. The supplementary NPIQ and BDI forest plots contain no study dated after 2019. That fits an update that added studies only to the ZBI analysis.
Side observations, separate from the main finding. (a) The archived Figure 2, captioned "Forest plot for Zarit Burden Interview main results", is a funnel plot that prints p = 0.024. It shows 18 markers, not 17. One of them lies at about MD 8.1, SE 2.1, which is where the pooled 17-study estimate falls (8.11, SE 2.10). (b) The Abstract says "Of 2369 records screened". Figure 1 gives 2365, which equals 2942 + 335 + 222 + 196 − 1330. The Results sentence "474 full-text publications were screened (350 from databases and registers and 109 identified via other methods)" has parts that sum to 459.
Figure
Figure 1. The caregiver-burden (ZBI) results as printed in the Abstract (left) and in the Results text and Figure 3 (right), in orange. Values are mean differences in ZBI points with 95% CIs. The key-messages box gives only the study count (14). The in-person subgroup is identical in both places. The total study count, the overall estimate and both the remote and hybrid subgroups differ, and so does the number of studies in those two subgroups. The gray notes below the panels give the two-sided P value implied by each hybrid estimate and CI, computed in recompute.py.txt. Under the Abstract's numbers the hybrid subgroup is significant; under the Results numbers it is not.
Figure 2. Pooled ZBI mean differences with 95% CIs on one axis. Orange marks the values reported in the paper and blue marks DerSimonian–Laird estimates recomputed from the 17 study rows of Figure 3. Squares pair each Abstract value with the pooled estimate from the 14 studies dated 2023 or earlier; circles pair each Results / Figure 3 value with the pooled estimate from all 17 studies. The thin vertical line marks MD = 0 (no effect), and k gives the number of studies. Each reported value sits on its recomputed counterpart, so the Abstract and the Results are each internally consistent but describe different study sets. The largest consequence is in the hybrid row, where dropping the 2025 study moves the estimate from 14.5 to 20.9 and the interval off zero.
Why it matters
Caregiver burden is the review's primary outcome, and the Abstract is the part most readers, indexers and guideline developers use. Its numbers come from a 14-study analysis, while the paper's stated analysis, its Figure 3 and its risk-of-bias subgroups use 17 studies. The headline survives: in both versions the pooled MD is positive and its CI excludes zero (implied P about 0.0002 and 0.0001). The Abstract overstates the magnitude by 0.6 ZBI points, about 7% of 8.1, and reports a slightly wider interval.
The delivery-mode results are affected more. The Abstract reports that interventions delivered as a hybrid of in-person and remote contact give the largest effect, MD 20.9, with a CI (5.8 to 36.0) that excludes zero. In the 17-study analysis the hybrid estimate is 14.5 and its CI (−2.7 to 31.7) includes zero, so this subgroup gives no statistically significant evidence of benefit on its own. A reader of the Abstract would infer that hybrid delivery is clearly effective and probably the most effective mode. That inference conflicts with the paper's own Results and with its Discussion ("no clear patterns of intervention effect based on delivery mode"). The remote subgroup also changes, from 2 studies with I² 37.5% to 4 studies with I² 0.0%. Remote delivery stays significant in both versions.
The key-messages box adds a second inconsistency. It describes the ZBI analysis as "14 studies (2664 participants)", but 2664 is the participant total for all 32 included studies, and the Results give 2011 participants for the 17-study ZBI analysis.
The between-subgroup test (p < 0.001) matches a fixed-effect (inverse-variance) decomposition of heterogeneity. A test on the random-effects subgroup means gives P = 0.67. That is a methodological point about interpretation, not an error, but readers should know that the "significant differences between delivery method subgroups" rest on the fixed-effect statistic, which the very large within-subgroup heterogeneity (I² up to 95.8%) makes hard to interpret.
Explanations considered
- Typographical error in the Abstract. This does not account for the numbers. Every value in the overall, remote and hybrid results differs (MD, both CI limits, I² and the study counts), while the in-person values agree exactly. Every differing value is reproduced, to rounding, by one coherent analysis of a specific subset of studies.
- The Abstract reports a different but deliberate analysis, such as a sensitivity analysis. Nothing in the paper describes a 14-study ZBI analysis. The only sensitivity analysis named in the Results removes studies at high risk of bias, and it splits the studies 11 + 6, not 14 + 3. The Abstract presents its numbers as the main burden result.
- The Abstract reflects the analysis before the June 2025 search update. This is consistent with all the numbers. The three studies whose removal reproduces the Abstract are exactly the three dated 2024–2025 in Figure 3. All 10 in-person studies are dated 2023 or earlier, which explains why the in-person subgroup is identical in both places. The Methods state that searches were "re-updated in June 2025". The paper does not say this happened, so it remains an inference, though it is the only one of the 18 possible subsets that fits.
- Transcription error in my reading of Figure 3. Pooling all 17 transcribed rows reproduces the published 17-study results to within 0.1 on every MD, CI limit and I², so the transcription is accurate at the precision that matters here.
- Conversion or typesetting error in the archived text. This is excluded. The PMC text, the publisher-supplied JATS XML in Europe PMC and the PubMed abstract carry the same Abstract values. Figure 3, which is the publisher's own image, carries the Results values.
- A correction already issued. This is excluded as of the date below (see Status).
Status
- Version of record. The publisher's site (doi.org redirects to gh.bmj.com/content/11/1/e016028) returned HTTP 403 to automated requests, so its HTML could not be read directly. I compared the PMC-archived text with the publisher-deposited JATS XML in Europe PMC (PMC13059920/fullTextXML) and the PubMed record (PMID 41506756; first published 2026-01-08). All three carry the Abstract and Results values quoted above. Every quotation was confirmed verbatim against the PMC text with
src/show.py, except the key-messages box, which was confirmed in the Europe PMC XML, and the contents of Figures 1–3, which were read from the published images (bmjgh-11-1-g001.jpg to g003.jpg). - Supplements. The single supplement, online supplemental file 1 (bmjgh-11-1-s001.docx, from the Europe PMC supplementary-files archive), holds search strategies, excluded full texts, study characteristics, risk-of-bias figures and the secondary-outcome forest plots (NPIQ, BDI and WHOQOL-BREF, overall and by delivery mode). It contains no ZBI forest plot, no 14-study analysis and no study-level ZBI data beyond Figure 3.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. Web searches for corrections or comments on the article found none. PubPeer refused automated access (Cloudflare challenge, HTTP 403), so it could not be checked directly.
- Sources. PMC article PMC13059920; Europe PMC full-text XML and supplementary-files archive for PMC13059920; Europe PMC/PubMed record 41506756; Crossref metadata for 10.1136/bmjgh-2024-016028; PROSPERO CRD42021283611 (cited in the paper).
Suggested correction
- Replace the Abstract's caregiver-burden results with those of the 17-study analysis reported in the Results and Figure 3: MD 8.1 (95% CI 4.0 to 12.2; I² = 89.0%). Replace the delivery-mode results with in person (10 studies; MD 6.4, 1.4 to 11.5; I² = 88.7%), remote (four studies; MD 7.6, 4.2 to 11.1; I² = 0.0%) and hybrid (three studies; MD 14.5, −2.7 to 31.7; I² = 95.8%).
- Revise any Abstract wording that implies every delivery mode produced a statistically significant improvement, since the hybrid subgroup's interval includes zero.
- In the key-messages box, change "pooled results from 14 studies (2664 participants)" to the ZBI analysis as reported in the Results, "17 studies (2011 participants)".
- Minor, in the same correction: replace the Figure 2 image or caption so that the forest plot of the main ZBI result is shown (the archived image is a funnel plot), and confirm whether the funnel plot should show 17 or 18 points. Reconcile the records-screened count (2369 in the Abstract, 2365 in Figure 1) and the full-text breakdown in the Results (350 + 109 against 474).
- Optionally, state in the Methods or the Figure 3 legend that the between-subgroup p value is the fixed-effect (inverse-variance) heterogeneity statistic.
Recomputation script (python recompute.py.txt)
"""Recompute the caregiver-burden (Zarit Burden Interview, ZBI) meta-analysis of PMC13059920
(BMJ Glob Health 2026, doi:10.1136/bmjgh-2024-016028).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper:
Figure 3 (forest plot of ZBI by delivery mode): 17 study rows, each a mean difference (MD)
with its 95% CI, printed to one decimal. Study labels are replaced here by delivery mode and
publication year (IP = in person, R = remote, H = in person & remote "hybrid").
Abstract: overall 8.7 (4.1 to 13.3), I2 89.9%; in person 10 studies 6.4 (1.4 to 11.5), I2 88.7%;
remote two studies 8.0 (1.9 to 14.1), I2 37.5%; hybrid two studies 20.9 (5.8 to 36.0), I2 90.1%.
Results: 17 studies; overall 8.1 (4.0 to 12.2), I2 89.0%; in person 10, 6.4 (1.4 to 11.5), I2 88.7%;
remote four, 7.6 (4.2 to 11.1), I2 0.0%; hybrid three, 14.5 (-2.7 to 31.7), I2 95.8%;
between-subgroup p<0.001.
Method (as stated in the paper's Methods): DerSimonian-Laird random effects, inverse-variance
weights; each study's SE is recovered from its printed 95% CI as (upper - lower) / (2 * 1.96).
"""
import itertools
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.isf(0.025)
# ------------------------------------------------------------------ inputs (Figure 3)
# (label, mode, year, MD, lower, upper)
S = [
("IP1", "in", 2009, 2.4, -4.3, 9.1), ("IP2", "in", 2010, 2.5, -9.0, 13.9),
("IP3", "in", 2011, 3.6, -2.8, 10.0), ("IP4", "in", 2015, 13.1, 7.3, 18.9),
("IP5", "in", 2017, 12.2, 7.6, 16.8), ("IP6", "in", 2017, 17.5, 15.4, 19.7),
("IP7", "in", 2018, 4.9, -2.5, 12.4), ("IP8", "in", 2018, -0.8, -7.7, 6.0),
("IP9", "in", 2020, -2.6, -9.6, 4.5), ("IP10", "in", 2023, 7.3, 2.7, 11.9),
("R1", "rem", 2021, 4.7, -2.5, 11.9), ("R2", "rem", 2022, 10.9, 4.4, 17.4),
("R3", "rem", 2024, 6.3, -2.2, 14.9), ("R4", "rem", 2024, 7.5, 1.3, 13.8),
("H1", "hyb", 2019, 28.3, 23.6, 32.9), ("H2", "hyb", 2019, 12.8, 4.5, 21.1),
("H3", "hyb", 2025, 2.2, -3.6, 8.0),
]
MODES = [("in", "In person"), ("rem", "Remote"), ("hyb", "Hybrid")]
# reported values: (k, MD, lo, hi, I2)
REP_ABS = {"all": (14, 8.7, 4.1, 13.3, 89.9), "in": (10, 6.4, 1.4, 11.5, 88.7),
"rem": (2, 8.0, 1.9, 14.1, 37.5), "hyb": (2, 20.9, 5.8, 36.0, 90.1)}
REP_RES = {"all": (17, 8.1, 4.0, 12.2, 89.0), "in": (10, 6.4, 1.4, 11.5, 88.7),
"rem": (4, 7.6, 4.2, 11.1, 0.0), "hyb": (3, 14.5, -2.7, 31.7, 95.8)}
# overall k in the abstract is not printed; 14 is the sum of the abstract's subgroup counts
# and the number printed in the key-messages box ("pooled results from 14 studies").
def dl(y, se):
"""DerSimonian-Laird random-effects pooling. Returns k, MD, lo, hi, I2 (%), SE, Q, tau2."""
y, se = np.asarray(y, float), np.asarray(se, float)
w = 1 / se**2
fe = (w * y).sum() / w.sum()
Q = (w * (y - fe) ** 2).sum()
k = len(y)
tau2 = max(0.0, (Q - (k - 1)) / (w.sum() - (w**2).sum() / w.sum()))
ws = 1 / (se**2 + tau2)
m = (ws * y).sum() / ws.sum()
sm = np.sqrt(1 / ws.sum())
I2 = max(0.0, (Q - (k - 1)) / Q) * 100 if Q > 0 else 0.0
return dict(k=k, md=m, lo=m - Z * sm, hi=m + Z * sm, I2=I2, se=sm, Q=Q, tau2=tau2)
def pool(rows):
return dl([r[3] for r in rows], [(r[5] - r[4]) / (2 * Z) for r in rows])
def fmt(r):
return f"k={r['k']:2d} MD {r['md']:5.2f} ({r['lo']:5.2f} to {r['hi']:5.2f}) I2 {r['I2']:4.1f}%"
def fmtrep(t):
return f"k={t[0]:2d} MD {t[1]:5.1f} ({t[2]:5.1f} to {t[3]:5.1f}) I2 {t[4]:4.1f}%"
def sub(rows, mode):
return [r for r in rows if r[1] == mode]
EARLY = [r for r in S if r[2] <= 2023]
REC17 = {"all": pool(S), **{m: pool(sub(S, m)) for m, _ in MODES}}
REC14 = {"all": pool(EARLY), **{m: pool(sub(EARLY, m)) for m, _ in MODES}}
print("=== 1. Counts ===")
for m, name in MODES:
print(f" {name:10s} Figure 3: {len(sub(S, m)):2d} studies dated 2023 or earlier: {len(sub(EARLY, m)):2d}"
f" abstract: {REP_ABS[m][0]:2d} Results: {REP_RES[m][0]:2d}")
print(f" sum: Figure 3 = {len(S)}, dated <=2023 = {len(EARLY)}, abstract 10+2+2 = {10 + 2 + 2},"
f" Results 10+4+3 = {10 + 4 + 3}; Results text: 17 studies in the ZBI meta-analysis;"
f" risk-of-bias split 11+6 = {11 + 6}")
print(" studies dated 2024-2025 in Figure 3:", [f"{r[0]} ({r[2]})" for r in S if r[2] > 2023])
print(f" studies with MD <= 0: {sum(r[3] <= 0 for r in S)}; with MD > 0: {sum(r[3] > 0 for r in S)}"
" (Results: '15 suggesting an improvement'; abstract and Results: improvement 'except for two')")
print("\n=== 2. DerSimonian-Laird on all 17 rows of Figure 3 vs the Results text ===")
for key, name in [("all", "Overall")] + MODES:
print(f" {name:10s} recomputed {fmt(REC17[key])} Results: {fmtrep(REP_RES[key])}")
print(f" overall pooled SE = {REC17['all']['se']:.2f}; tau2 = {REC17['all']['tau2']:.1f}")
print("\n=== 3. DerSimonian-Laird on the 14 rows dated 2023 or earlier vs the abstract ===")
for key, name in [("all", "Overall")] + MODES:
print(f" {name:10s} recomputed {fmt(REC14[key])} abstract: {fmtrep(REP_ABS[key])}")
print("\n=== 4. Is the 'drop the three 2024-2025 studies' subset unique? ===")
# The abstract keeps 10 in-person, 2 of 4 remote and 2 of 3 hybrid studies: 6 x 3 = 18 candidate subsets.
rem, hyb, inp = sub(S, "rem"), sub(S, "hyb"), sub(S, "in")
cands = []
for rk in itertools.combinations(rem, 2):
for hk in itertools.combinations(hyb, 2):
rows = inp + list(rk) + list(hk)
o, rr, hh = pool(rows), pool(list(rk)), pool(list(hk))
dist = max(abs(o["md"] - 8.7), abs(o["lo"] - 4.1), abs(o["hi"] - 13.3),
abs(rr["md"] - 8.0), abs(rr["lo"] - 1.9), abs(rr["hi"] - 14.1),
abs(hh["md"] - 20.9), abs(hh["lo"] - 5.8), abs(hh["hi"] - 36.0))
cands.append((dist, [r[0] for r in rk] + [r[0] for r in hk], o, rr, hh))
cands.sort(key=lambda c: c[0])
print(f" {len(cands)} candidate subsets; largest absolute gap to the abstract's 9 MD/CI numbers:")
for dist, ids, o, rr, hh in cands[:4]:
print(f" keep {ids}: max gap {dist:.2f}; overall {o['md']:.2f} ({o['lo']:.2f}-{o['hi']:.2f}),"
f" remote {rr['md']:.2f}, hybrid {hh['md']:.2f}")
n_within = sum(c[0] <= 0.15 for c in cands)
print(f" subsets within 0.15 of all nine abstract numbers: {n_within}")
print("\n=== 5. Rounding sensitivity (each printed MD and CI limit perturbed uniformly by +/-0.05) ===")
rng = np.random.default_rng(20261024)
sims = {"17": [], "14": []}
arr = np.array([[r[3], r[4], r[5]] for r in S])
early_idx = np.array([r[2] <= 2023 for r in S])
for _ in range(4000):
a = arr + rng.uniform(-0.05, 0.05, arr.shape)
se = (a[:, 2] - a[:, 1]) / (2 * Z)
sims["17"].append([dl(a[:, 0], se)["md"], dl(a[:, 0], se)["I2"]])
sims["14"].append([dl(a[early_idx, 0], se[early_idx])["md"], dl(a[early_idx, 0], se[early_idx])["I2"]])
SIM = {}
for key, lab in [("17", "17 studies"), ("14", "14 studies")]:
v = np.array(sims[key])
lo_md, hi_md = np.percentile(v[:, 0], [2.5, 97.5])
lo_i2, hi_i2 = np.percentile(v[:, 1], [2.5, 97.5])
SIM[key] = (lo_md, hi_md, lo_i2, hi_i2)
print(f" {lab}: overall MD {lo_md:.2f} to {hi_md:.2f}; I2 {lo_i2:.1f}% to {hi_i2:.1f}% (central 95%)")
rem14 = [r for r in EARLY if r[1] == "rem"]
i2s = []
for _ in range(4000):
a = np.array([[r[3], r[4], r[5]] for r in rem14]) + rng.uniform(-0.05, 0.05, (2, 3))
i2s.append(dl(a[:, 0], (a[:, 2] - a[:, 1]) / (2 * Z))["I2"])
hyb14 = [r for r in EARLY if r[1] == "hyb"]
hh = []
for _ in range(4000):
a = np.array([[r[3], r[4], r[5]] for r in hyb14]) + rng.uniform(-0.05, 0.05, (2, 3))
d = dl(a[:, 0], (a[:, 2] - a[:, 1]) / (2 * Z)); hh.append([d["md"], d["lo"], d["hi"]])
hh = np.array(hh)
print(f" hybrid, 2 studies: MD {np.percentile(hh[:, 0], 2.5):.2f} to {np.percentile(hh[:, 0], 97.5):.2f},"
f" upper limit {np.percentile(hh[:, 2], 2.5):.2f} to {np.percentile(hh[:, 2], 97.5):.2f} (abstract 20.9, 36.0)")
print(f" remote, 2 studies: I2 {np.percentile(i2s, 2.5):.1f}% to {np.percentile(i2s, 97.5):.1f}%"
f" (abstract 37.5%)")
print("\n=== 6. Implied two-sided P from the printed estimate and 95% CI (Wald, normal) ===")
IMPL = {}
for lab, (md, lo, hi) in [("abstract overall", (8.7, 4.1, 13.3)), ("Results overall", (8.1, 4.0, 12.2)),
("abstract hybrid", (20.9, 5.8, 36.0)), ("Results hybrid", (14.5, -2.7, 31.7)),
("abstract remote", (8.0, 1.9, 14.1)), ("Results remote", (7.6, 4.2, 11.1))]:
se = (hi - lo) / (2 * Z)
p = 2 * stats.norm.sf(abs(md / se))
IMPL[lab] = p
print(f" {lab:17s} MD {md} ({lo} to {hi}): SE {se:.2f}, z {md / se:.2f}, P = {p:.2g}")
print(f" overall MD difference abstract - Results: {8.7 - 8.1:.1f} points ({(8.7 - 8.1) / 8.1 * 100:.0f}% of 8.1)")
print(f" hybrid MD difference abstract - Results: {20.9 - 14.5:.1f} points")
print("\n=== 7. Between-subgroup heterogeneity (17 studies; Results: p<0.001) ===")
y = np.array([r[3] for r in S]); se = np.array([(r[5] - r[4]) / (2 * Z) for r in S]); w = 1 / se**2
Qtot = (w * (y - (w * y).sum() / w.sum()) ** 2).sum()
Qwith = 0.0
for m, _ in MODES:
idx = np.array([r[1] == m for r in S])
yi, wi = y[idx], w[idx]
Qwith += (wi * (yi - (wi * yi).sum() / wi.sum()) ** 2).sum()
Qb = Qtot - Qwith
print(f" fixed-effect (inverse-variance) decomposition: Q_between = {Qb:.1f}, df = 2, p = {stats.chi2.sf(Qb, 2):.2g}")
mm = np.array([REC17[m]["md"] for m, _ in MODES]); ss = np.array([REC17[m]["se"] for m, _ in MODES])
wm = 1 / ss**2
Qre = (wm * (mm - (wm * mm).sum() / wm.sum()) ** 2).sum()
print(f" test on the three random-effects subgroup means: Q = {Qre:.2f}, df = 2, p = {stats.chi2.sf(Qre, 2):.2f}")
print("\n=== 8. Side check: screening counts (not part of the main finding) ===")
db = [2942, 335, 222, 196]
print(f" Figure 1 database records {'+'.join(map(str, db))} = {sum(db)}; minus 1330 duplicates = {sum(db) - 1330}"
f" (Figure 1 'Records screened' 2365; abstract 'Of 2369 records screened')")
print(f" Results: 3845 identified - 1330 duplicates = {3845 - 1330}; other methods 142+4+4 = {142 + 4 + 4};"
f" {sum(db)} + {142 + 4 + 4} = {sum(db) + 150}")
print(f" Results: '474 full-text publications ... (350 from databases and registers and 109 identified via other"
f" methods)': 350 + 109 = {350 + 109}; Figure 1 reports sought 365 + 109 = {365 + 109}")
print(f" Figure 2 (archived image is a funnel plot): the pooled 17-study estimate would sit at MD"
f" {REC17['all']['md']:.2f}, SE {REC17['all']['se']:.2f}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
def ci(a, b, c):
return f"{a:.1f} ({b:.1f} to {c:.1f})".replace("-", "−")
# ---- Figure 1: abstract vs Results, two text panels joined by an annotated connector
fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 5.0); ax.axis("off")
ax.text(0.15, 4.78, "Abstract and Results report different ZBI analyses", fontsize=12, weight="bold", va="center")
pw, top, bot = 2.8, 4.5, 0.75
panels = [(0.15, "Abstract", "and key-messages box", REP_ABS), (7.05 - pw, "Results text", "and Figure 3", REP_RES)]
rows = [("Studies by delivery mode", None), ("Overall", "all"), ("In person", "in"), ("Remote", "rem"), ("Hybrid", "hyb")]
ys = [3.6, 3.0, 2.4, 1.8, 1.2]
for x0, title, subtitle, rep in panels:
ax.add_patch(FancyBboxPatch((x0, bot), pw, top - bot, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=INK2, lw=1.0))
ax.text(x0 + 0.15, top - 0.2, title, fontsize=11.5, weight="bold", va="center")
ax.text(x0 + 0.15, top - 0.43, subtitle, fontsize=10, color=INK2, va="center")
for (lab, key), yy in zip(rows, ys):
if key is None:
ks = [rep[m][0] for m, _ in MODES]
txt = f"{ks[0]} + {ks[1]} + {ks[2]} = {sum(ks)}"
else:
k, md, lo, hi, i2 = rep[key]
txt = ci(md, lo, hi) + f", I\u00b2 {i2:.1f}%"
if key != "all":
lab = f"{lab}, {k} studies"
ax.text(x0 + 0.15, yy + 0.13, lab, fontsize=10, color=INK2, va="center")
ax.text(x0 + 0.15, yy - 0.13, txt, fontsize=10.5, color=ORANGE, va="center",
weight="normal" if key == "in" else "bold")
ax.text(0.15 + 0.15, bot - 0.2, "hybrid CI excludes 0 (implied P = %.3f)" % IMPL["abstract hybrid"],
fontsize=10, color=INK2, va="center")
ax.text(7.05 - pw + 0.15, bot - 0.2, "hybrid CI includes 0 (implied P = %.2f)" % IMPL["Results hybrid"],
fontsize=10, color=INK2, va="center")
xl, xr = 0.15 + pw + 0.05, 7.05 - pw - 0.05
xm = (xl + xr) / 2
for yy, note in [(ys[0], "14 vs 17"), (ys[1], "8.7 vs 8.1"), (ys[3], "2 vs 4 studies"), (ys[4], "2 vs 3 studies")]:
ax.add_patch(FancyArrowPatch((xl, yy), (xr, yy), arrowstyle="<->", mutation_scale=11, color=INK, lw=1.4))
ax.text(xm, yy + 0.08, note, fontsize=10, ha="center", va="bottom", color=INK)
ax.plot([xl, xr], [ys[2]] * 2, color=INK2, lw=1, ls=(0, (2, 2)))
ax.text(xm, ys[2] + 0.08, "identical", fontsize=10, ha="center", va="bottom", color=INK2)
handles = [Line2D([], [], ls="none", marker="s", ms=8, color=ORANGE)]
ax.legend(handles, ["values as printed in the paper (MD in ZBI points, 95% CI)"], loc="lower left",
bbox_to_anchor=(0.01, -0.01), frameon=False, fontsize=10, handletextpad=0.4)
save(fig, "fig1")
# ---- Figure 2: dot-and-interval, reported vs recomputed, on one MD axis
fig, ax = plt.subplots(figsize=(7.2, 5.6), facecolor=BG)
ax.set_facecolor(BG)
groups = [("all", "Overall"), ("in", "In person"), ("rem", "Remote"), ("hyb", "Hybrid")]
yticks, ylabels = [], []
y0 = 0
for key, name in groups:
for lab, rep, rec, mk, setlab in [("Abstract", REP_ABS[key], REC14[key], "s", "14 studies to 2023"),
("Results / Fig. 3", REP_RES[key], REC17[key], "o", "all 17 studies")]:
yr, yc = y0 + 0.14, y0 - 0.14
ax.plot([rep[2], rep[3]], [yr, yr], color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(rep[1], yr, marker=mk, ms=8, color=ORANGE, mec=BG, mew=1.2, ls="none")
ax.plot([rec["lo"], rec["hi"]], [yc, yc], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(rec["md"], yc, marker=mk, ms=8, color=BLUE, mec=BG, mew=1.2, ls="none")
yticks.append(y0); ylabels.append(f"{name}: {lab}" if lab == "Abstract" else lab)
ax.text(41.5, y0, f"k={rep[0]}", va="center", ha="right", fontsize=10, color=INK2)
y0 -= 0.75
y0 -= 0.45
ax.axvline(0, color=INK2, lw=0.8)
ax.set_yticks(yticks); ax.set_yticklabels(ylabels)
for t, lab in zip(ax.get_yticklabels(), ylabels):
if ":" in lab:
t.set_fontweight("bold"); t.set_color(INK)
ax.set_xlim(-6, 42)
ax.set_ylim(y0 + 0.6, 0.55)
ax.set_xlabel("Mean difference in ZBI score (positive = less burden with intervention)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, mec=BG),
Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, mec=BG),
Line2D([], [], color=INK2, lw=0, marker="s", ms=8),
Line2D([], [], color=INK2, lw=0, marker="o", ms=8)]
ax.legend(handles, ["reported in the paper", "recomputed (DL) from Figure 3 rows",
"square: abstract; 14 studies to 2023",
"circle: Results; all 17 studies"],
loc="upper center", bbox_to_anchor=(0.4, -0.13), frameon=False, fontsize=10, ncol=2,
columnspacing=1.2, handletextpad=0.5)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract's patient-handwashing confidence interval is impossible and conflicts with the paper's own non-significant clustered estimate
BMJ Glob Health (2016) · doi:10.1136/bmjgh-2016-000103 · PMC5321376
Summary
The abstract reports two behaviour changes after an infection prevention and control (IPC) workshop: maintenance of distance, RR 1.09 (95% CI 1.02 to 1.16), and patient handwashing, RR 1.19 (95% CI 1.3 to 1.25). The handwashing interval cannot be a confidence interval, because its lower limit (1.3) is above its upper limit (1.25) and the point estimate (1.19) lies outside it even if the limits are swapped. The paper's own clustered GEE analysis in Table 4, the analysis the Methods specify, gives handwashing RR 1.27 (0.95 to 1.71) and distance RR 1.11 (0.83 to 1.48). Neither is significant, and the Results text says handwashing "increased though not significantly". The abstract's distance figure equals, to two decimals, a crude risk ratio computed from the Table 4 counts that ignores clustering. As printed, the abstract presents as statistically significant two improvements that the body of the paper reports as non-significant.
What the paper reports
Abstract, Results:
The maintenance of distance (RR 1.09, 95% CI 1.02 to 1.16) and patient handwashing (RR 1.19, 95% CI 1.3 to 1.25) improved to >90%.
Results, "Adherence to IPC behaviours", first paragraph:
Patient handwashing, with or without HCW prompting, increased though not significantly from 82% to 99% (RR 1.21, 95% CI 0.95 to 1.71).
Results, "Adherence to IPC behaviours", second paragraph:
Virtually all HCWs stood 1.5 m from patients, increasing from 91% to 99% at follow-up (RR 1.11, 95% CI 0.83 to 1.48).
Table 4, screening block (columns: baseline correct, per cent, n = 90 | follow-up correct, per cent, n = 131 | RR | 95% CI):
Patient washed hands directly or washed on direction from HCW | 74 | 82 | 130 | 99 | 1.27 | 0.95 to 1.71†
Stood 1.5 m from patient | 82 | 91 | 130 | 99 | 1.11 | 0.83 to 1.48†
Table 4 footnote:
*Risk ratio using binomial regression (family: binomial, link: log) accounting for clustering at the health facility level (GEE).
†Indicates that a Poisson regression (family: Poisson, link: log) was used due to the failure of the binomial model to converge.
Methods, "Data analysis":
Generalised estimating equations (GEE) with robust SEs accounted for repeated measures among HCWs and clustering within PHUs.
For all statistical tests, a significance level of p<0.05 was chosen.
Discussion, first paragraph:
During workshops, HCWs focused on improving screening, maintaining physical distance and encouraging patient handwashing; changes in these domains were reflected in the improvements seen in these behaviours at follow-up. Significant improvements were not consistent across behaviours, partly due to several high baseline values (>80%).
What the paper's own numbers show
Method. I checked the abstract's handwashing interval for internal validity (lower limit no greater than upper limit, point estimate inside). From the Table 4 counts I recomputed each percentage and a crude follow-up versus baseline risk ratio with a log-scale Wald 95% CI, ignoring clustering. For each printed Table 4 interval I took the geometric centre, sqrt(lower × upper), which equals the point estimate of a symmetric log-scale interval, and the standard error it implies, ln(upper/lower)/3.92. I then compared the abstract's and the Results text's figures with both analyses. The GEE estimates themselves cannot be re-derived, because they need facility- and worker-level data. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Handwashing 95% CI, ordering | Abstract | 1.3 to 1.25 | lower limit 1.3 > upper limit 1.25: not a valid interval |
| Handwashing RR inside its CI? | Abstract | RR 1.19 | 1.19 lies outside 1.25–1.3, so it is outside the interval even with the limits swapped |
| Patient handwashing, % correct | Table 4; Results text | 82% → 99% | 74/90 = 82.2%, 130/131 = 99.2% |
| Handwashing RR, crude (ignoring clustering) | from Table 4 counts | — | 1.21 (1.10 to 1.33); log SE 0.050 |
| Handwashing RR, GEE Poisson | Table 4 | 1.27 (0.95 to 1.71) | centre of printed CI 1.275: internally consistent; implied robust log SE 0.150; includes 1 |
| Handwashing RR | Results text | 1.21 (0.95 to 1.71) | point estimate = crude 1.207; interval = the Table 4 GEE interval (centred on 1.275) |
| Stood 1.5 m at screening, % correct | Table 4; Results text | 91% → 99% | 82/90 = 91.1%, 130/131 = 99.2% |
| Distance RR, crude (ignoring clustering) | from Table 4 counts | — | 1.089 (1.019 to 1.164), i.e. 1.09 (1.02 to 1.16) |
| Distance RR | Abstract | 1.09 (1.02 to 1.16) | identical to the crude, unclustered value above |
| Distance RR, GEE Poisson | Table 4; Results text | 1.11 (0.83 to 1.48) | centre of printed CI 1.108: consistent; includes 1 |
| Stood 1.5 m at consultation, for contrast | Table 4 | 35/54 (65%) → 29/32 (91%); 1.18 (0.92 to 1.51) | crude 1.40 (1.12 to 1.75); neither matches the abstract |
| Hypothetical "1.13 to 1.25" for handwashing | — | — | geometric centre 1.188 (≈ 1.19), but it implies log SE 0.026: 0.52 × the crude SE and 0.17 × the GEE robust SE |
| Handwashing counts by district | Supplement, Annex 1 | Bo 34/45 → 58/58; Kenema 40/45 → 72/73 | sum 74/90 → 130/131, matching Table 4 |
Three things follow. First, the abstract's handwashing interval is not a confidence interval for any estimate. Second, the abstract's distance result reproduces exactly a crude, unclustered analysis of the Table 4 counts, while the Methods and Table 4 describe a clustered GEE analysis whose interval for the same comparison includes 1. Third, the abstract's handwashing estimate (1.19) matches neither the crude ratio (1.21) nor the GEE estimate (1.27). Its pattern resembles the distance figure: both of the abstract's intervals exclude 1, and both of the corresponding GEE intervals include 1.
Figure
Risk ratios for patient handwashing (top) and for standing 1.5 m from the patient at screening (bottom), on a log scale with a reference line at 1 (no change). Orange circles are values printed in the paper; blue squares are crude, unclustered Wald estimates recomputed from the Table 4 counts. The abstract's handwashing "interval" is drawn at its printed limits, with the lower limit (1.3) to the right of the upper limit (1.25) and the point estimate (1.19) outside both. The abstract's distance interval coincides with the crude recomputation. The paper's GEE intervals for both behaviours, in Table 4 and in the Results text, cross 1.
The abstract's two effect estimates (left) set against the Results text and Table 4 (right) and against the crude recomputation from the Table 4 counts (bottom). The abstract's intervals lie entirely above 1; the body's intervals for the same two behaviours include 1. The abstract's distance figure reproduces the unclustered calculation; its handwashing figure reproduces nothing in the paper.
Why it matters
These are the only two quantitative effect estimates in the abstract, and its sentence reads as a statistically significant improvement in both behaviours. The analysis described in the Methods (GEE with robust standard errors, significance at p<0.05) does not support that reading. Patient handwashing, RR 1.27 (0.95 to 1.71), and distance at screening, RR 1.11 (0.83 to 1.48), are both non-significant in Table 4, and the Results text says so for handwashing. A reader of the abstract alone, or a reviewer extracting effect sizes from it, would record two significant improvements and, for handwashing, an interval that cannot be used. Clustering matters here: the GEE robust standard errors implied by Table 4 are about three times (handwashing) and four times (distance) the unclustered ones, so the unclustered intervals are much too narrow for observations clustered within eight facilities.
The overstatement of significance is confined to the abstract. The Discussion describes the percentage improvements, which did occur (82% to 99% and 91% to 99%), and its next sentence states that "Significant improvements were not consistent across behaviours, partly due to several high baseline values (>80%)". That agrees with Table 4, where, of the handwashing items, only the prompted ones (screener asked patient to wash hands, RR 1.45, 1.16 to 1.80; patient washed hands on direction, RR 1.49, 1.19 to 1.86) are significant. The abstract carries no such qualification. Its conclusions are qualitative (favourable attitudes towards IPC, gaps in practice, the need for socio-behavioural improvement and a secure supply chain), and they do not depend on these two estimates.
Explanations considered
- Archiving or conversion error. Ruled out. The PMC XML, the Europe PMC record and the publisher-typeset PDF all print "RR 1.19, 95% CI 1.3 to 1.25". No minus signs, superscripts or decimal symbols are involved, and the Table 4 cells line up in both the XML and the PDF.
- A different row or population. The abstract says the behaviours "improved to >90%". For handwashing, only the combined row (82% → 99%) reaches 90%. The prompted rows reach 80%. For distance, both the screening row (91% → 99%) and the consultation row (65% → 91%) end above 90%, but only the screening row reproduces 1.09 (1.02 to 1.16). The consultation row gives 1.40 (1.12 to 1.75) crude and 1.18 (0.92 to 1.51) by GEE.
- A different model. Table 4's footnote explains that this row used a Poisson GEE because the binomial model did not converge. That model gives 1.27 (0.95 to 1.71), and the printed interval is centred on 1.275, so the table is internally consistent. The crude estimate is 1.21 (1.10 to 1.33). The abstract's 1.19 matches neither.
- Unadjusted figures in the abstract. This accounts exactly for the distance figure. It does not account for the handwashing figure: on the same crude basis handwashing would be 1.21 (1.10 to 1.33). Even if the abstract was meant to report crude estimates, those estimates ignore the clustering that the Methods say the analysis accounts for, and they disagree with Table 4 on significance.
- A typographical slip ("1.3" for "1.13"). An interval of 1.13 to 1.25 would be centred on 1.188 and would match 1.19, so a dropped digit would explain the impossible ordering. It would not explain where the estimate comes from. It is not the crude ratio or the GEE estimate, and such an interval would need a standard error about half the crude unclustered one and a sixth of the GEE one. Nor would it explain why the abstract implies significance when the Results text says the change "increased though not significantly".
- A one-sided interval or another stated convention. None is stated. The Methods give a significance level of p<0.05 for all tests, with GEE and an exchangeable working correlation.
- The Results text's "RR 1.21, 95% CI 0.95 to 1.71". Its point estimate is the crude ratio (1.207). Its interval is the Table 4 GEE interval, whose centre is 1.275. So the text pairs a crude point estimate with a model-based interval. This is a smaller inconsistency, but it does not affect the text's conclusion that the change was not significant, which agrees with Table 4.
Status
- Version of record. The DOI resolves to the publisher's page at gh.bmj.com (https://gh.bmj.com/lookup/doi/10.1136/bmjgh-2016-000103), which returned HTTP 403 to automated access. The version of record was read from a copy of the publisher-typeset PDF (12 pages, CC BY 4.0) on the RKI edoc repository and cross-checked against PMC and Europe PMC. All three carry the abstract sentence, the Results text and Table 4 exactly as quoted above.
- Supplement. The only supplement (bmjgh-2016-000103supp.pdf, "Annex 1. Proportions of correct IPC events before and after the workshop, by district") gives counts and percentages by district but no risk ratios or intervals. Its handwashing counts sum to Table 4's 74/90 and 130/131. It neither explains nor corrects the abstract's figures.
- Corrections. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no notices.
- PubPeer. The PubPeer API and search page refused automated access (HTTP 403; human verification required). A web search for the title together with "pubpeer" found no PubPeer thread. Search engines do not reliably index PubPeer, so a manual check at pubpeer.com is still advisable.
- Sources. PMC5321376 (read with
src/show.py);src/known.py 10.1136/bmjgh-2016-000103(Crossref, PMC and Europe PMC notices); https://doi.org/10.1136/bmjgh-2016-000103; https://edoc.rki.de/bitstream/handle/176904/2475/27svLccxFCudc.pdf (typeset PDF); Europe PMC fullTextXML and supplementaryFiles for PMC5321376.
Suggested correction
The abstract's Results sentence should report the paper's primary, clustered analysis. Using Table 4 it would read, for example: "Maintenance of distance during screening (91% to 99%; RR 1.11, 95% CI 0.83 to 1.48) and patient handwashing (82% to 99%; RR 1.27, 95% CI 0.95 to 1.71) reached >90%, although neither change was statistically significant; HCW prompting of patient handwashing (RR 1.45, 95% CI 1.16 to 1.80) and handwashing on prompting (RR 1.49, 95% CI 1.19 to 1.86) increased significantly." If the authors intended to report crude estimates in the abstract, that should be stated, the handwashing figure should read 1.21 (1.10 to 1.33), and the abstract should note that the clustered analysis is not significant. Separately, the Results text's handwashing estimate should read RR 1.27 to match its interval and Table 4. The Discussion already notes that significant improvements were not consistent across behaviours and needs no change.
Recomputation script (python recompute.py.txt)
"""Recompute the patient-handwashing and distance risk ratios of PMC5321376
(BMJ Glob Health 2016;1:e000103, doi:10.1136/bmjgh-2016-000103).
Run: python recompute.py.txt # numbers + fig1, fig2
python recompute.py.txt --candidates DIR # also write design1..5 to DIR
python recompute.py.txt --fig1 3 --fig2 0 # use design 3 as fig1, no fig2
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png (and fig2.svg/png unless --fig2 0) next to this file. Five candidate
figure designs exist; fig1/fig2 are design1/design2 unless chosen otherwise.
Inputs are copied from the paper as archived in PMC (identical in the typeset PDF):
Abstract, Results: distance RR 1.09 (1.02 to 1.16); handwashing RR 1.19 (1.3 to 1.25)
Results text: handwashing RR 1.21 (0.95 to 1.71); stood 1.5 m RR 1.11 (0.83 to 1.48)
Table 4: counts, percentages, GEE risk ratios and 95% CIs
Supplement (Annex 1): handwashing counts by district (Bo, Kenema)
"""
import argparse
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ap = argparse.ArgumentParser()
ap.add_argument("--candidates", help="directory to write candidate designs 1-5 into")
ap.add_argument("--fig1", type=int, default=1, help="design number used as fig1")
ap.add_argument("--fig2", type=int, default=2, help="design number used as fig2 (0 = none)")
ARGS = ap.parse_args()
Z = 1.959964
# ---------------------------------------------------------------- inputs (from the paper)
# Table 4 rows: (baseline correct, baseline n, follow-up correct, follow-up n, RR, lo, hi)
# Screening block: baseline n = 90, follow-up n = 131. Consultation block: 54 and 32.
TABLE4 = {
"Screener asked patient to wash hands": (56, 90, 105, 131, 1.45, 1.16, 1.80),
"Patient washed hands on direction from HCW": (54, 90, 105, 131, 1.49, 1.19, 1.86),
"Patient washed hands directly or on direction (dagger)": (74, 90, 130, 131, 1.27, 0.95, 1.71),
"Stood 1.5 m from patient, screening (dagger)": (82, 90, 130, 131, 1.11, 0.83, 1.48),
"Stood 1.5 m from patient, consultation": (35, 54, 29, 32, 1.18, 0.92, 1.51),
}
TABLE4_PCT = { # printed per cent columns (baseline, follow-up)
"Screener asked patient to wash hands": (62, 80),
"Patient washed hands on direction from HCW": (60, 80),
"Patient washed hands directly or on direction (dagger)": (82, 99),
"Stood 1.5 m from patient, screening (dagger)": (91, 99),
"Stood 1.5 m from patient, consultation": (65, 91),
}
ABSTRACT = {"handwashing": (1.19, 1.3, 1.25), "distance": (1.09, 1.02, 1.16)}
RESULTS_TEXT = {"handwashing": (1.21, 0.95, 1.71), "distance": (1.11, 0.83, 1.48)}
ANNEX_HANDWASH = {"Bo": ((34, 45), (58, 58)), "Kenema": ((40, 45), (72, 73))}
HW = "Patient washed hands directly or on direction (dagger)"
DS = "Stood 1.5 m from patient, screening (dagger)"
DC = "Stood 1.5 m from patient, consultation"
def crude_rr(a1, n1, a0, n0):
"""Unclustered risk ratio, follow-up vs baseline, with a log-scale Wald 95% CI."""
rr = (a1 / n1) / (a0 / n0)
se = math.sqrt(1 / a1 - 1 / n1 + 1 / a0 - 1 / n0)
return rr, rr * math.exp(-Z * se), rr * math.exp(Z * se), se
def geo_centre(lo, hi):
return math.sqrt(lo * hi)
def log_se(lo, hi):
return math.log(hi / lo) / (2 * Z)
R = {} # recomputed numbers used by the figures
print("== 1. The abstract's handwashing interval ==")
rr, lo, hi = ABSTRACT["handwashing"]
print(f"abstract RR {rr}, 95% CI {lo} to {hi}")
print(f" lower limit <= upper limit? {lo <= hi} ({lo} > {hi})")
print(f" interval contains the RR? {min(lo, hi) <= rr <= max(lo, hi)}")
print(f" with the limits swapped (1.25 to 1.3), contains 1.19? {1.25 <= rr <= 1.3}")
print(f" hypothetical '1.13 to 1.25': geometric centre {geo_centre(1.13, 1.25):.3f}, "
f"implied log-scale SE {log_se(1.13, 1.25):.4f}")
print("\n== 2. Table 4 counts: percentages, crude RR, and the centre of each printed CI ==")
for k, (b, n0, f, n1, RR, L, H) in TABLE4.items():
r, l, h, se = crude_rr(f, n1, b, n0)
R[k] = (r, l, h, se)
pb, pf = TABLE4_PCT[k]
print(f"{k}:")
print(f" baseline {b}/{n0} = {100*b/n0:.1f}% (printed {pb}); follow-up {f}/{n1} = "
f"{100*f/n1:.1f}% (printed {pf})")
print(f" crude unclustered RR {r:.3f} (Wald 95% CI {l:.3f} to {h:.3f}); log SE {se:.4f}")
print(f" Table 4 GEE RR {RR} ({L} to {H}); geometric centre of the printed CI "
f"{geo_centre(L, H):.3f}; implied robust log SE {log_se(L, H):.4f}")
print("\n== 3. Where the abstract's numbers come from ==")
r, l, h, _ = R[DS]
print(f"distance: abstract 1.09 (1.02 to 1.16) vs crude screening distance "
f"{r:.2f} ({l:.2f} to {h:.2f}) -> exact match after rounding: "
f"{(round(r, 2), round(l, 2), round(h, 2)) == (1.09, 1.02, 1.16)}")
r2, l2, h2, _ = R[DC]
print(f" consultation distance, for contrast: crude {r2:.2f} ({l2:.2f} to {h2:.2f}); "
f"GEE 1.18 (0.92 to 1.51)")
r, l, h, se = R[HW]
print(f"handwashing: abstract 1.19 (1.3 to 1.25); crude {r:.2f} ({l:.2f} to {h:.2f}); "
f"Table 4 GEE 1.27 (0.95 to 1.71)")
print(f" a CI of 1.13 to 1.25 needs log SE {log_se(1.13, 1.25):.4f}, "
f"{log_se(1.13, 1.25)/se:.2f} x the crude unclustered SE ({se:.4f}) and "
f"{log_se(1.13, 1.25)/log_se(0.95, 1.71):.2f} x the GEE robust SE ({log_se(0.95, 1.71):.4f})")
for k, nm in ((HW, "handwashing"), (DS, "distance, screening")):
L, H = TABLE4[k][5], TABLE4[k][6]
print(f" {nm}: GEE robust log SE / crude log SE = {log_se(L, H)/R[k][3]:.2f}")
print("\n== 4. Results text: handwashing RR 1.21 (0.95 to 1.71) ==")
print(f" crude point estimate {(130/131)/(74/90):.3f}; geometric centre of 0.95-1.71 = "
f"{geo_centre(0.95, 1.71):.3f} (Table 4 prints 1.27)")
print(f" does 0.95-1.71 exclude 1? {0.95 > 1}; does the abstract's distance CI exclude 1? "
f"{1.02 > 1}; Table 4 distance CI 0.83-1.48 excludes 1? {0.83 > 1}")
print("\n== 5. Supplement Annex 1, handwashing by district ==")
bt = sum(v[0][0] for v in ANNEX_HANDWASH.values()); bn = sum(v[0][1] for v in ANNEX_HANDWASH.values())
ft = sum(v[1][0] for v in ANNEX_HANDWASH.values()); fn = sum(v[1][1] for v in ANNEX_HANDWASH.values())
print(f" baseline {bt}/{bn}, follow-up {ft}/{fn} (Table 4: 74/90, 130/131)")
# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
LW, MS = 2.0, 8
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight")
fig.savefig(p + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def log_axis(ax, lo=0.8, hi=1.9, ticks=(0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.5, 1.7)):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvline(1.0, color=INK2, lw=0.9, zorder=1)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def interval(ax, y, est, lo, hi, color, marker, label=None, dashed=False):
ax.plot([lo, hi], [y, y], color=color, lw=LW, ls="--" if dashed else "-",
solid_capstyle="butt", zorder=2)
ax.plot([lo, lo], [y - 0.12, y + 0.12], color=color, lw=LW, zorder=2)
ax.plot([hi, hi], [y - 0.12, y + 0.12], color=color, lw=LW, zorder=2)
ax.plot(est, y, marker, color=color, ms=MS, mec=BG, mew=1.5, zorder=3, label=label)
hw_c, ds_c = R[HW], R[DS]
# ---------------------------------------------------------------- design 1: forest plot
def design1():
fig, ax = plt.subplots(figsize=(4.72, 4.4))
rows = [
# y, label, (est, lo, hi), kind
(7, "Abstract", (1.19, 1.3, 1.25), "abs"),
(6, "Results text", RESULTS_TEXT["handwashing"], "rep"),
(5, "Table 4 (GEE)", (1.27, 0.95, 1.71), "rep"),
(4, "Crude, ignoring clustering", hw_c[:3], "rec"),
(2.5, "Abstract", ABSTRACT["distance"], "rep"),
(1.5, "Table 4 / text (GEE)", (1.11, 0.83, 1.48), "rep"),
(0.5, "Crude, ignoring clustering", ds_c[:3], "rec"),
]
for y, lab, (e, l, h), kind in rows:
if kind == "abs":
# printed limits drawn where they sit; the 'interval' runs backwards
ax.plot([1.25, 1.3], [y, y], color=ORANGE, lw=LW, ls=(0, (3, 2)), zorder=2)
ax.plot(1.3, y, marker="|", color=ORANGE, ms=14, mew=LW)
ax.plot(1.25, y, marker="|", color=ORANGE, ms=14, mew=LW)
ax.plot(e, y, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=3)
ax.annotate("lower 1.3", (1.3, y), (1.3, y + 0.42), ha="center", fontsize=10, color=INK2)
ax.annotate("upper 1.25", (1.25, y), (1.235, y - 0.55), ha="center", fontsize=10, color=INK2)
ax.text(1.02, y, "1.19 (1.3 to 1.25):\nlower limit > upper;\n1.19 lies outside", va="center",
ha="left", fontsize=10, color=INK,
transform=matplotlib.transforms.blended_transform_factory(ax.transAxes, ax.transData))
else:
c, m = (ORANGE, "o") if kind == "rep" else (BLUE, "s")
interval(ax, y, e, l, h, c, m)
txt = f"{e:.2f} ({l:.2f} to {h:.2f})"
ax.text(1.02, y, txt, va="center", ha="left", fontsize=10, color=INK,
transform=matplotlib.transforms.blended_transform_factory(ax.transAxes, ax.transData))
ax.set_yticks([r[0] for r in rows])
ax.set_yticklabels([r[1] for r in rows], fontsize=10, color=INK)
ax.text(0.0, 1.0, "Patient handwashing, 74/90 (82%) \u2192 130/131 (99%)", transform=ax.transAxes,
fontsize=10.5, weight="bold", ha="left", va="bottom")
ax.text(0.8, 3.15, "Stood 1.5 m at screening, 82/90 (91%) \u2192 130/131 (99%)", fontsize=10.5,
weight="bold", ha="left", va="bottom", transform=ax.transData,
bbox=dict(fc=BG, ec="none", pad=1.5), zorder=4)
log_axis(ax, 0.8, 1.8)
ax.set_ylim(-0.1, 7.7)
ax.set_xlabel("Risk ratio, follow-up vs baseline (log scale); line at 1 = no change")
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Reported in the paper")
h2, = ax.plot([], [], "s", color=BLUE, ms=MS,
label="Recomputed from Table 4 counts (unclustered Wald)")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.16), ncol=1,
frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- design 2: two text panels
def box(ax, x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.015",
fc="#ffffff", ec=edge, lw=1.5))
ax.text(x + 0.02, y + h - 0.035, title, weight="bold", fontsize=10.5, va="top")
ax.text(x + 0.02, y + h - 0.10, "\n".join(lines), fontsize=10, va="top", color=INK,
linespacing=1.45)
def design2():
fig = plt.figure(figsize=(7.2, 3.0))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(-0.015, 1.015); ax.set_ylim(-0.025, 1.0); ax.axis("off")
box(ax, 0.0, 0.47, 0.41, 0.51, "Abstract, Results",
["\u201cmaintenance of distance", "(RR 1.09, 95% CI 1.02 to 1.16)",
"and patient handwashing", "(RR 1.19, 95% CI 1.3 to 1.25)",
"improved to >90%\u201d"], ORANGE)
box(ax, 0.57, 0.47, 0.43, 0.51, "Results text / Table 4",
["Handwashing \u201cincreased", "though not significantly\u201d:",
"RR 1.21 (text), 1.27 (Table 4),", "both 95% CI 0.95 to 1.71",
"Stood 1.5 m: RR 1.11 (0.83 to 1.48)"], ORANGE)
box(ax, 0.0, 0.0, 1.0, 0.29, "Recomputed from Table 4 counts, ignoring clustering",
[f"Stood 1.5 m, screening: {ds_c[0]:.2f} ({ds_c[1]:.2f} to {ds_c[2]:.2f}), same as abstract",
f"Patient handwashing: {hw_c[0]:.2f} ({hw_c[1]:.2f} to {hw_c[2]:.2f}), not the abstract's "
"1.19 (1.3 to 1.25)"],
BLUE)
ax.annotate("", xy=(0.565, 0.74), xytext=(0.415, 0.74),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(0.49, 0.78, "conflict", ha="center", fontsize=10, weight="bold")
ax.text(0.49, 0.70, "CIs > 1\nvs\nCIs span 1", ha="center", va="top", fontsize=10,
color=INK2)
ax.annotate("", xy=(0.22, 0.30), xytext=(0.22, 0.46),
arrowprops=dict(arrowstyle="<-", color=BLUE, lw=2))
ax.text(0.235, 0.38, "distance matches crude", fontsize=10, color=INK2, va="center")
return fig
# ---------------------------------------------------------------- design 3: zoomed number line
def design3():
fig, ax = plt.subplots(figsize=(6.52, 2.9))
y_abs, y_rec, y_gee = 2.0, 1.0, 0.0
ax.plot([1.25, 1.3], [y_abs, y_abs], color=ORANGE, lw=LW, ls=(0, (3, 2)))
for x, lab, dy in [(1.3, "printed lower limit 1.3", 0.32), (1.25, "printed upper limit 1.25", -0.45)]:
ax.plot(x, y_abs, marker="|", color=ORANGE, ms=16, mew=LW)
ax.text(x, y_abs + dy, lab, ha="center", fontsize=10, color=INK2)
ax.plot(1.19, y_abs, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5)
ax.text(1.175, y_abs, "RR 1.19", ha="right", va="center", fontsize=10, color=INK2)
interval(ax, y_rec, *hw_c[:3], BLUE, "s")
ax.text(hw_c[2] * 1.01, y_rec, f"{hw_c[0]:.2f} ({hw_c[1]:.2f} to {hw_c[2]:.2f})", va="center", fontsize=10)
interval(ax, y_gee, 1.27, 0.95, 1.71, ORANGE, "o")
ax.text(1.72, y_gee, "1.27 (0.95 to 1.71)", va="center", fontsize=10)
ax.set_yticks([y_abs, y_rec, y_gee])
ax.set_yticklabels(["Abstract (reported)", "Crude from Table 4 counts\n(recomputed)",
"Table 4 GEE (reported)"], color=INK)
log_axis(ax, 0.9, 2.1, (0.9, 1.0, 1.1, 1.2, 1.3, 1.5, 1.7, 2.0))
ax.set_ylim(-0.6, 2.7)
ax.set_xlabel("Patient handwashing risk ratio (log scale); line at 1 = no change")
return fig
# ---------------------------------------------------------------- design 4: provenance grid
def design4():
fig = plt.figure(figsize=(7.2, 3.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2); ax.set_ylim(0, 3.4); ax.axis("off")
cx = (1.9, 3.42, 4.94) # column centres, in inches
cols = ["Abstract\n(reported)", "Crude from counts\n(recomputed)", "Table 4 GEE\n(reported)"]
for j, c in enumerate(cols):
ax.text(cx[j], 3.1, c, ha="center", va="center", fontsize=10, weight="bold",
color=ORANGE if j != 1 else BLUE)
rows = [("Stood 1.5 m,\nscreening", "1.09 (1.02\u20131.16)",
f"{ds_c[0]:.2f} ({ds_c[1]:.2f}\u2013{ds_c[2]:.2f})", "1.11 (0.83\u20131.48)",
"same", "differs"),
("Patient\nhandwashing", "1.19 (1.3\u20131.25)",
f"{hw_c[0]:.2f} ({hw_c[1]:.2f}\u2013{hw_c[2]:.2f})", "1.27 (0.95\u20131.71)",
"differs", "differs")]
for i, (lab, a, c, g, m1, m2) in enumerate(rows):
y = 2.2 - i * 1.0
ax.text(0.05, y, lab, va="center", fontsize=10, weight="bold")
for j, v in enumerate((a, c, g)):
ax.add_patch(FancyBboxPatch((cx[j] - 0.725, y - 0.3), 1.45, 0.6,
boxstyle="round,pad=0.0,rounding_size=0.06",
fc="#ffffff", ec=BLUE if j == 1 else ORANGE, lw=1.5))
ax.text(cx[j], y, v, ha="center", va="center", fontsize=10)
ax.text(5.78, y, f"vs crude: {m1}\nvs GEE: {m2}", ha="left",
va="center", fontsize=10, color=INK2)
ax.text(0.05, 0.35, "Only the GEE intervals include 1 (no significant change).\n"
"The abstract's handwashing interval has its lower limit above its upper limit.",
fontsize=10, color=INK2, va="center")
return fig
# ---------------------------------------------------------------- design 5: crude vs GEE, all rows
def design5():
keys = [HW, DS, "Screener asked patient to wash hands",
"Patient washed hands on direction from HCW", DC]
labels = ["Patient washed hands\n(direct or prompted)", "Stood 1.5 m, screening",
"Screener asked patient\nto wash hands", "Patient washed hands\non direction",
"Stood 1.5 m, consultation"]
fig, ax = plt.subplots(figsize=(6.62, 4.4))
for i, k in enumerate(keys):
y = len(keys) - 1 - i
b, n0, f, n1, RR, L, H = TABLE4[k]
interval(ax, y + 0.17, RR, L, H, ORANGE, "o")
c = R[k]
interval(ax, y - 0.17, c[0], c[1], c[2], BLUE, "s")
# abstract values overlaid as open diamonds
ax.plot(1.19, 4.0, "D", mfc="none", mec=ORANGE, ms=MS, mew=LW)
ax.plot(1.09, 3.0, "D", mfc="none", mec=ORANGE, ms=MS, mew=LW)
ax.text(1.19, 4.48, "abstract 1.19\n(CI printed 1.3 to 1.25)", ha="center", fontsize=10, color=INK2)
ax.text(1.02, 2.62, "abstract 1.09 (1.02 to 1.16)", ha="left", fontsize=10, color=INK2, va="top")
ax.set_yticks(range(len(keys)))
ax.set_yticklabels(labels[::-1], color=INK)
log_axis(ax, 0.8, 2.0, (0.8, 0.9, 1.0, 1.2, 1.5, 2.0))
ax.set_ylim(-0.6, 5.15)
ax.set_xlabel("Risk ratio, follow-up vs baseline (log scale); line at 1 = no change")
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Table 4 GEE (reported)")
h2, = ax.plot([], [], "s", color=BLUE, ms=MS, label="Crude Wald from Table 4 counts (recomputed)")
h3, = ax.plot([], [], "D", mfc="none", mec=ORANGE, ms=MS, mew=LW, label="Abstract (reported)")
ax.legend(handles=[h1, h2, h3], loc="upper center", bbox_to_anchor=(0.4, -0.15), ncol=2,
frameon=False, fontsize=10)
return fig
designs = {1: design1, 2: design2, 3: design3, 4: design4, 5: design5}
if ARGS.candidates:
os.makedirs(ARGS.candidates, exist_ok=True)
for i, d in designs.items():
save(d(), os.path.join(ARGS.candidates, f"design{i}"))
print(f"\nwrote design1..5 (.svg, .png) to {ARGS.candidates}")
save(designs[ARGS.fig1](), os.path.join(HERE, "fig1"))
if ARGS.fig2:
save(designs[ARGS.fig2](), os.path.join(HERE, "fig2"))
print(f"wrote fig1 (design {ARGS.fig1})" + (f" and fig2 (design {ARGS.fig2})" if ARGS.fig2 else ""))
Ebola review calls case-fatality change non-significant while reporting OR 1.6, 95% CI excluding 1, p<0.001
BMJ Glob Health (2020) · doi:10.1136/bmjgh-2019-001955 · PMC7061886
Summary
This review of 34 Ebola virus disease outbreaks (1976–2019) states in its Abstract, Results and Abstract Conclusions that the case fatality rate (CFR) did not change significantly over four decades. It supports that statement with "OR in 2019 vs 1976=1.6 (95% CI 1.5 to 1.8), p<0.001". That interval excludes 1 and the P value is below 0.001, so the statistic contradicts the claim it is cited for. The Results text gives a different interval (1.5 to 1.6) for the same OR. The OR also points the wrong way for the paper's own CFRs (71% falling to 62% gives an OR of about 0.67, not 1.6). The Table 3 CFRs for three of the four periods and the overall CFR of "66% (62 to 71)" cannot be reproduced from the outbreak counts in Table 1. Those counts give a pooled CFR that falls from 71.3% to 41.5% (crude OR 0.29, 95% CI 0.24 to 0.34) and an overall CFR of 43.1%. The CFR trend is the review's only quantitative inference, and the Abstract Conclusions use it to support the claim that control challenges persist. As published, the reported statistics do not support the sentence they accompany. Whether the real trend is significant depends on an analysis the paper does not describe: a patient-level comparison finds a large decline, while analyses that allow for variation between outbreaks find no clear trend.
What the paper reports
Abstract, Methods:
Analyses of trends in case fatality were performed by calculating ORs between times.
Abstract, Results:
In the past four decades, a total of 34 EVD outbreaks affecting 34 356 cases and causing 14 823 deaths were reported in 11 countries in Sub-Saharan Africa. The overall case fatality rate (95% CI) was 66% (62 to 71) and did not change substantially over time (OR in 2019 vs 1976=1.6 (95% CI 1.5 to 1.8), p<0.001).
Abstract, Conclusions:
Sub-Saharan Africa continues to face considerable challenges in EVD control, whereby there are no significant changes in case fatality rates observed during the past four decades.
Methods, "Data extraction and analysis":
Additional analyses of trends in case fatality rates were performed by calculating ORs between periods using MedCalc statistical software (https://www.medcalc.org/calc/odds_ratio.php).
Results, "Context of EVD", fourth paragraph:
Case fatality rates varied from 71% during outbreaks in 1976–1986 to 62% during the later years. However, there was no statistically significant change over time (OR in 2019 vs 1976: 1.6 (95% CI 1.5 to 1.6)). The overall trend from 1976 to 2019 outbreaks shows a linear rate of change over time (table 3, figure 3).
Results, "Context of EVD", third paragraph:
During the past four decades, approximately 14 853 people have died from EVD outbreaks.
Table 3, "OR for case fatality rate of Ebola virus disease according to period" (columns: 1976–1986 | 1987–1997 | 1998–2008 | 2009–2019):
Death (n) | 454 | 348 | 733 | 13 288
Case fatality rate (%) | 71 | 67 | 64 | 62
OR (95% CI) | 1.6 (1.4 to 1.8) | 1.7 (1.5 to 1.9) | 1.4 (1.3 to 1.6) | 1.6 (1.5 to 1.6)
P value | <0.001 | <0.001 | <0.001 | <0.001
Table 1 lists each outbreak's year, country, virus, cases, deaths and CFR. Its total row:
Total | | | 34 356 | 14 823 |
Table 2, sex rows (columns: case | death | CFR (%)):
Gender | Male | 15 803 | 6373 | 46
Gender | Female | 18 553 | 8449 | 54
Table 3 does not state the comparison group or the direction of its ORs, and the paper does not say whether the analysis treated individual patients or outbreaks as the unit.
What the paper's own numbers show
Method. I entered the 34 outbreaks of Table 1 (year, cases, deaths) and assigned each multi-year outbreak (for example "2000–2001") to its first year. Grouping them into the four Table 3 periods reproduces Table 3's death counts exactly (454, 348, 733, 13 288), which confirms the grouping. For each period I computed the pooled CFR (deaths ÷ cases). I then computed crude odds ratios of death with Woolf 95% CIs, the CFR odds ratio implied by Table 3's own CFRs, and a chi-square test of equal CFR across periods. For each reported OR I also derived the z statistic its CI implies. These patient-level calculations treat every case as independent, as a crude OR from a 2×2 table does. The cases are concentrated in a few outbreaks: the 2014 outbreaks in Liberia and Sierra Leone alone account for 72.2% of all cases. So I also ran three outbreak-level analyses, which the paper does not report: a Spearman correlation of outbreak CFR with year, a quasi-binomial logistic model, and a DerSimonian–Laird random-effects meta-regression. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Does the cited OR show "no change"? | Abstract, Results | 1.6 (95% CI 1.5 to 1.8), p<0.001; "did not change substantially" | CI excludes 1; implied z = 10.1. By its own statistics the change is significant |
| Same OR in the body | Results text | 1.6 (95% CI 1.5 to 1.6); "no statistically significant change" | CI excludes 1 (implied z ≈ 28); also a different CI from the Abstract's for the same quantity |
| Table 3 ORs | Table 3 | 1.6, 1.7, 1.4, 1.6; all CIs above 1; all P < 0.001 | every CI excludes 1; none is a "no change" result |
| Direction of the OR, using Table 3's CFRs | Table 3; Results | 62% vs 71%, reported as "OR in 2019 vs 1976" = 1.6 | (0.62/0.38) ÷ (0.71/0.29) = 0.67 (0.56 to 0.79), p = 4.1 × 10⁻⁶; the inverse is 1.50 (1.26 to 1.78). A fall in CFR gives an OR below 1 |
| Deaths per period | Table 3 vs Table 1 | 454, 348, 733, 13 288 | 454, 348, 733, 13 288 (identical) |
| CFR, 1976–1986 | Table 3 | 71% | 454/637 = 71.3% |
| CFR, 1987–1997 | Table 3 | 67% | 348/467 = 74.5% |
| CFR, 1998–2008 | Table 3 | 64% | 733/1199 = 61.1% |
| CFR, 2009–2019 | Table 3; Results | 62% | 13 288/32 053 = 41.5% |
| OR of death, 2009–2019 vs 1976–1986 | from Table 1 | — | 0.285 (0.240 to 0.339), p ≈ 9 × 10⁻⁴⁶; the inverse is 3.50 |
| Equal CFR across the four periods | from Table 1 | "no statistically significant change" | χ² = 588, df = 3, p ≈ 4 × 10⁻¹²⁷ (patient level) |
| Overall CFR | Abstract | "66% (62 to 71)", labelled a 95% CI | pooled 14 823/34 356 = 43.1% (binomial 95% CI 42.6 to 43.7); unweighted mean of the 34 outbreak CFRs 61.8%. 66.0% is the mean of the four Table 3 CFRs and 62–71 is their range |
| Total deaths | Table 1; Abstract vs Results text | 14 823 vs "approximately 14 853" | Table 1 rows sum to 14 823; the text differs by 30 |
| CFR by sex | Table 2 | male 46, female 54 | male 6373/15 803 = 40.3%, female 8449/18 553 = 45.5%. 46.0% and 54.0% are each sex's share of all cases |
| Outbreak-level trend (not in the paper) | from Table 1 | — | Spearman ρ = −0.16 (p = 0.35); quasi-binomial OR 2009–2019 vs 1976–1986 0.29 (0.04 to 2.09), p = 0.21, dispersion 122; random-effects meta-regression OR per decade 0.81 (0.62 to 1.05), p = 0.11 |
The table supports three findings.
- The reported inference contradicts the reported wording. Every OR the paper prints (Abstract, Results, the four Table 3 columns) has a CI that excludes 1 and a P value below 0.001. The sentences that cite them say there was no significant change.
- The reported OR does not match the reported CFRs. For CFRs of 71% and then 62%, the OR of death for the later period is 0.67. Its inverse, 1.50 (1.26 to 1.78 with the Table 1 period sizes), is close to the reported 1.6 but describes the earlier period against the later. Neither the Abstract's CI (1.5 to 1.8) nor the Results' CI (1.5 to 1.6) is reproduced.
- Table 3's CFRs, apart from the first period, do not follow from Table 1. Table 3's death counts match Table 1 exactly. Its CFRs for 1987–1997, 1998–2008 and 2009–2019 differ from deaths ÷ cases by −7.5, +2.9 and +20.5 percentage points. The largest gap is in the period that holds 93% of all cases (32 053 of 34 356). They do not match the unweighted means of the outbreak CFRs either (76.5%, 53.5%, 64.9%, 58.9%). The Abstract's "66% (62 to 71)" is reproduced exactly by the mean and range of the four Table 3 CFRs, not by any pooled estimate or confidence interval.
Figure
Figure 1. Odds ratios of death with 95% CIs on a log scale. The thin vertical line is OR = 1 (no change). Orange: every OR the paper reports for the CFR trend (Abstract, Results text, and the four columns of Table 3). All lie to the right of 1 with intervals that exclude it, and all carry P < 0.001, yet the paper says there was no significant change. Blue diamonds are recomputed from the paper's own numbers. The upper one is the OR implied by Table 3's CFRs (62% vs 71%). The lower one is the crude OR for 2009–2019 vs 1976–1986 from the Table 1 counts. Both lie to the left of 1, the opposite direction from the reported 1.6. The hollow blue square is the same Table 1 comparison with a CI widened for variation between outbreaks (quasi-binomial logistic model, dispersion 122). It is not an analysis the paper reports. It shows that the data are compatible with no change once outbreaks rather than patients are treated as the unit.
Figure 2. Case fatality rate for each Table 3 period and overall, on an axis from 0 to 100%. Orange circles: Table 3's CFRs and the Abstract's overall 66%. Blue diamonds: deaths ÷ cases from the outbreak rows of Table 1, grouped so that the deaths equal Table 3's. The first period agrees (71% vs 71.3%). The others differ, most of all in 2009–2019 (62% vs 41.5%) and overall (66% vs 43.1%).
Why it matters
The CFR trend is the only quantitative inference in the review. It is stated in the Abstract Results, the Abstract Conclusions and the Results text, and the Abstract Conclusions present it as evidence for the review's framing ("continues to face considerable challenges in EVD control, whereby there are no significant changes in case fatality rates"). A reader who takes the Abstract at face value gets a claim (no significant change), a statistic that contradicts it (CI excluding 1, p<0.001), and an overall CFR (66%) that is about 23 percentage points above the pooled figure implied by the paper's own outbreak table (43.1%).
The paper's numbers do not settle what the correct conclusion is, and this write-up does not claim to. Two readings are defensible, and neither matches the paper.
- Patient-level (as a crude OR implies). The CFR fell substantially and significantly, from 71.3% to 41.5% (OR 0.29, 95% CI 0.24 to 0.34). Under this reading the conclusion "no significant changes" is reversed.
- Outbreak-level. The fall is driven largely by the very large 2014 West African outbreaks (Sierra Leone 26%, Liberia 45%), and CFR varies greatly between outbreaks (Pearson dispersion 122). Allowing for that variation, the decline is not statistically significant (quasi-binomial p = 0.21; random-effects meta-regression OR per decade 0.81, 95% CI 0.62 to 1.05; Spearman ρ = −0.16, p = 0.35). Under this reading "no clear trend" may be defensible. It would then rest on a different analysis, with intervals that include 1, not on the published OR of 1.6 with p<0.001.
Either way, the published statistics cannot support the published sentence, and the overall CFR and three of the four period CFRs are not what the paper's own table implies. The review's qualitative synthesis (epidemiological, health-system and sociocultural challenges) and its recommendations (one-health surveillance, community engagement, research on diagnostics, drugs and vaccines, health-system resilience) do not depend on these numbers.
Explanations considered
- "Did not change substantially" means not clinically important, not non-significant. This does not account for the text. The Results say "no statistically significant change" and the Abstract Conclusions say "no significant changes", and the pooled CFR difference from Table 1 (about 30 percentage points) is large on any clinical scale.
- The OR compares each period with a reference, and 1.6 is in the stated direction. This does not account for the numbers. Table 3 does not name a reference group. If "2019 vs 1976" means the later period against the earlier, any fall in CFR gives an OR below 1: 0.67 from Table 3's CFRs, 0.29 from Table 1's counts. An OR of 1.6 can describe only the earlier period against the later. Even then, a CI excluding 1 means a significant change.
- The OR compares single years, 2019 against 1976. This does not rescue it. The only 2019 outbreak in Table 1 (Uganda) had 4 deaths among 4 cases, so the OR is undefined. Using the 2018–2019 DRC outbreak (2150/3220 = 66.8%) against the 1976 outbreaks (431/602 = 71.6%) gives OR 0.80 (0.66 to 0.97), p = 0.021, which is still below 1 and still significant.
- The Table 3 ORs are some other ratio. I tried several standard quantities: pairwise ORs between periods; each period's OR, risk ratio and CFR ratio against all other periods; and their inverses. None reproduces all four ORs with their CIs. The risk ratios of the first three periods against all others (1.67, 1.74, 1.44) are close to the reported 1.6, 1.7 and 1.4. Their CIs differ (1.59–1.76, 1.65–1.84, 1.37–1.51), and the fourth period's risk ratio is 0.62, not 1.6. The paper's method cannot be identified from what is published.
- Table 3's CFRs are outbreak-averaged rather than pooled. This does not account for the numbers. The unweighted means of outbreak CFRs per period are 76.5%, 53.5%, 64.9% and 58.9%, not 71, 67, 64 and 62. The overall mean across 34 outbreaks is 61.8%, not 66%.
- "66% (62 to 71)" is a genuine 95% CI from a model. Possible, but the numbers coincide exactly with the arithmetic mean (66.0%) and range (62–71) of the four Table 3 CFRs. A pooled estimate from Table 1 would be 43.1%, with a patient-level 95% CI of 42.6 to 43.7, about 1 point wide rather than 9.
- The statement is correct under an outbreak-level analysis. This is partly possible, as shown above: the trend is not significant once between-outbreak variation is allowed for. But the paper reports no such analysis, and the statistics it does report (CIs excluding 1, P < 0.001) are the opposite of a non-significant result. The text and its supporting statistics cannot both be right as printed.
- A conversion or typesetting artefact in PMC. Excluded. The publisher's PDF carries the same Abstract, Results and Table 3 text (see Status).
- Different case counts in Table 3 than in Table 1. Table 3 prints no case counts, but its deaths equal Table 1's period sums exactly. A different set of cases would have to produce identical deaths in all four periods, which is implausible.
Status
- Version of record. The live publisher pages (doi.org → gh.bmj.com) refused automated access (HTTP 403), and the PMC HTML page returned a bot check. The publisher's PDF (gh.bmj.com/content/bmjgh/5/3/e001955.full.pdf, as archived by the Internet Archive on 2023-12-05) was text-extracted. It carries the same Abstract sentences, the same Results sentence with "1.6 (95% CI 1.5 to 1.6)", and the same Table 3 cells as the PMC and Europe PMC XML. Every quotation above was confirmed verbatim against the PMC-archived text.
- Supplements and data. The article has no supplementary data. The Europe PMC supplementary bundle contains only the three figure images, the JATS XML has no supplementary-material element, and the data availability statement reads "Data are available upon request." Figure 3 (cases by year with a linear CFR trendline) reports no OR or test and was not reproduced here.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations (
update-toabsent,relationempty). PMC and Europe PMC list no correction or comment links (PMID 32201623). The Crossref Labs record, which includes Retraction Watch data, lists no updates, and the archived publisher landing page shows no notice. The PubPeer API refused automated access in this session (Cloudflare challenge). A web search for the title with "correction", "erratum" or "PubPeer" found only unrelated errata. - Sources. PMC article PMC7061886; Europe PMC full-text XML; publisher PDF (Internet Archive copy); Crossref REST API; Europe PMC search; web search.
Suggested correction
- Replace "did not change substantially over time" (Abstract), "there was no statistically significant change over time" (Results) and "there are no significant changes in case fatality rates" (Abstract Conclusions) with wording that matches whatever analysis is reported. The current ORs, with CIs excluding 1 and P < 0.001, indicate a significant difference.
- State the comparison and direction of every OR in Table 3 (which period against which reference), and the unit of analysis. Report a single, consistent CI for the "2019 vs 1976" OR; the Abstract gives 1.5 to 1.8 and the Results 1.5 to 1.6.
- Recompute the Table 3 CFRs as deaths ÷ cases from Table 1 (71.3%, 74.5%, 61.1%, 41.5%) and add the case counts to the table, or explain how the printed 67%, 64% and 62% were derived.
- Replace the overall "66% (62 to 71)" with the pooled CFR from Table 1 (43.1%) and a valid interval, or label it as the mean and range of the period CFRs if that is what it is.
- Because CFR varies widely between outbreaks and one region contributes most of the cases, consider a trend analysis that accounts for clustering by outbreak (for example random-effects or quasi-binomial models). Base the conclusion on its estimate and interval.
- Minor: correct "approximately 14 853" deaths to 14 823 (Table 1 total), and change the Table 2 "CFR (%)" entries for sex (46, 54) to 40.3 and 45.5, or relabel the column as the share of cases. The Table 2 age groups sum to 33 456 cases, 900 fewer than the 34 356 total, and the table does not say how the remaining cases are classified.
Recomputation script (python recompute.py.txt)
"""Recompute the case-fatality trend statistics of PMC7061886
(BMJ Glob Health 2020, doi:10.1136/bmjgh-2019-001955).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 1: year, cases and deaths of each of the 34 outbreaks (printed CFR column also kept)
Table 2: cases and deaths by sex; cases by age group and source of infection
Table 3: deaths, CFR (%), OR (95% CI) and P value for four periods
Abstract and Results text: overall CFR, ORs "in 2019 vs 1976"
Multi-year outbreaks in Table 1 (e.g. "2000-2001") are assigned to their first year.
With that rule the period death counts equal Table 3's exactly.
"""
import itertools
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (from the paper)
# Table 1: (first year, country, cases, deaths, printed CFR %)
T1 = [
(1976, "DRC", 318, 280, 88), (1976, "South Sudan", 284, 151, 53), (1977, "DRC", 1, 1, 100),
(1979, "South Sudan", 34, 22, 65), (1994, "Gabon", 52, 31, 60), (1994, "Cote d'Ivoire", 1, 0, 0),
(1995, "DRC", 315, 250, 79), (1996, "Gabon", 37, 21, 57), (1996, "Gabon", 60, 45, 75),
(1996, "South Africa", 2, 1, 50), (2000, "Uganda", 425, 224, 53), (2001, "Gabon", 65, 53, 82),
(2001, "Rep. Congo", 57, 43, 75), (2002, "Rep. Congo", 143, 128, 89), (2003, "Rep. Congo", 35, 29, 83),
(2004, "South Sudan", 17, 7, 41), (2005, "Rep. Congo", 12, 10, 83), (2007, "DRC", 264, 187, 71),
(2007, "Uganda", 149, 37, 25), (2008, "DRC", 32, 15, 47), (2011, "Uganda", 1, 1, 100),
(2012, "Uganda", 11, 4, 36), (2012, "DRC", 36, 13, 36), (2012, "Uganda", 6, 3, 50),
(2014, "DRC", 69, 49, 71), (2014, "Guinea", 3814, 2544, 67), (2014, "Liberia", 10678, 4810, 45),
(2014, "Sierra Leone", 14124, 3659, 26), (2014, "Nigeria", 20, 8, 40), (2014, "Mali", 8, 6, 75),
(2017, "DRC", 8, 4, 50), (2018, "DRC", 54, 33, 61), (2018, "DRC", 3220, 2150, 67),
(2019, "Uganda", 4, 4, 100),
]
T1_TOTAL = (34356, 14823) # Table 1 "Total" row; also Abstract
TEXT_DEATHS = 14853 # Results, "Context of EVD": "approximately 14 853 people have died"
PERIODS = [(1976, 1986), (1987, 1997), (1998, 2008), (2009, 2019)]
LABELS = ["1976–1986", "1987–1997", "1998–2008", "2009–2019"]
T3_DEATHS = [454, 348, 733, 13288]
T3_CFR = [71, 67, 64, 62]
T3_OR = [(1.6, 1.4, 1.8), (1.7, 1.5, 1.9), (1.4, 1.3, 1.6), (1.6, 1.5, 1.6)] # all P < 0.001
ABS_OR = (1.6, 1.5, 1.8) # Abstract: "OR in 2019 vs 1976=1.6 (95% CI 1.5 to 1.8), p<0.001"
RES_OR = (1.6, 1.5, 1.6) # Results: "OR in 2019 vs 1976: 1.6 (95% CI 1.5 to 1.6)"
ABS_CFR = (66, 62, 71) # Abstract: "overall case fatality rate (95% CI) was 66% (62 to 71)"
T2_SEX = {"Male": (15803, 6373, 46), "Female": (18553, 8449, 54)} # cases, deaths, printed CFR
T2_AGE = {"0–15": 7215, "15–60": 24049, "60+": 2192}
T2_SOURCE = {"Health facility": 13742, "Household": 20614}
def orci(a, b, c, d):
"""OR of a/b vs c/d with Woolf (log) 95% CI and two-sided Wald P."""
o = (a * d) / (b * c)
se = math.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
lo, hi = math.exp(math.log(o) - Z * se), math.exp(math.log(o) + Z * se)
return o, lo, hi, 2 * stats.norm.sf(abs(math.log(o) / se))
yr = np.array([r[0] for r in T1], float)
n = np.array([r[2] for r in T1], float)
dth = np.array([r[3] for r in T1], float)
C, D = int(n.sum()), int(dth.sum())
print("=== 1. Totals (Table 1) ===")
print(f"sum of 34 outbreaks: cases {C} (Table 1 total {T1_TOTAL[0]}), deaths {D} (Table 1 total {T1_TOTAL[1]}; "
f"Results text {TEXT_DEATHS}, difference {TEXT_DEATHS - D})")
print(f"pooled overall CFR = {D}/{C} = {100 * D / C:.1f}%")
print("Table 1 printed CFR column vs deaths/cases, where the rounding differs:")
for r in T1:
if int(100 * r[3] / r[2] + 0.5) != r[4]:
print(f" {r[0]} {r[1]}: {r[3]}/{r[2]} = {100 * r[3] / r[2]:.2f}% printed {r[4]}")
print("\n=== 2. Periods of Table 3, rebuilt from Table 1 ===")
P = []
for (a, b), lab, rd, rc in zip(PERIODS, LABELS, T3_DEATHS, T3_CFR):
m = (yr >= a) & (yr <= b)
c, d = int(n[m].sum()), int(dth[m].sum())
P.append((c, d))
print(f"{lab}: {m.sum():2d} outbreaks, cases {c:5d}, deaths {d:5d} (Table 3: {rd}; equal: {d == rd}), "
f"pooled CFR {100 * d / c:.1f}% (Table 3: {rc}%), difference {rc - 100 * d / c:+.1f} points; "
f"unweighted mean of outbreak CFRs {np.mean(100 * dth[m] / n[m]):.1f}%")
pooled_cfr = [100 * d / c for c, d in P]
print("\n=== 3. Overall CFR, Abstract '66% (62 to 71)' ===")
se_p = math.sqrt((D / C) * (1 - D / C) / C)
print(f"pooled: {100 * D / C:.1f}% (binomial 95% CI {100 * (D / C - Z * se_p):.1f}–{100 * (D / C + Z * se_p):.1f})")
print(f"difference reported - pooled: {ABS_CFR[0] - 100 * D / C:.1f} points; "
f"share of all cases in 2009–2019: {100 * P[3][0] / C:.1f}%; "
f"pooled fall 1976–1986 to 2009–2019: {pooled_cfr[0] - pooled_cfr[3]:.1f} points")
print(f"unweighted mean of the 34 outbreak CFRs: {np.mean(100 * dth / n):.1f}%")
print(f"unweighted mean of the four Table 3 CFRs: {np.mean(T3_CFR):.1f}%; their range: {min(T3_CFR)}–{max(T3_CFR)}")
print("\n=== 4. What the reported ORs themselves imply ===")
for lab, (o, lo, hi) in [("Abstract", ABS_OR), ("Results", RES_OR)] + \
[(f"Table 3 {l}", t) for l, t in zip(LABELS, T3_OR)]:
se = (math.log(hi) - math.log(lo)) / (2 * Z)
zz = math.log(o) / se if se > 0 else float("inf")
print(f"{lab:22s} OR {o} ({lo} to {hi}): CI excludes 1: {lo > 1}; implied z = {zz:.1f}")
print("Abstract and Results print different CIs for the same 'OR in 2019 vs 1976':", ABS_OR[1:], RES_OR[1:])
print("\n=== 5. Direction: OR of death, later vs earlier ===")
c1, d1 = P[0]
c4, d4 = P[3]
o_rep = orci(0.62 * c4, c4 - 0.62 * c4, 0.71 * c1, c1 - 0.71 * c1)
print(f"from the reported CFRs 62% vs 71% (Table 1 period sizes): OR {o_rep[0]:.3f} "
f"({o_rep[1]:.3f}–{o_rep[2]:.3f}), p = {o_rep[3]:.1e}; inverse {1 / o_rep[0]:.2f} "
f"({1 / o_rep[2]:.2f}–{1 / o_rep[1]:.2f})")
print(f"odds at reported CFRs: {[round(p / (100 - p), 2) for p in T3_CFR]}; "
f"0.62/0.38 / (0.71/0.29) = {(0.62 / 0.38) / (0.71 / 0.29):.3f}")
o_crude = orci(d4, c4 - d4, d1, c1 - d1)
print(f"from Table 1 counts, 2009–2019 vs 1976–1986: OR {o_crude[0]:.3f} ({o_crude[1]:.3f}–{o_crude[2]:.3f}), "
f"p = {o_crude[3]:.1e}; inverse {1 / o_crude[0]:.2f}")
print("pairwise crude ORs (row period vs column period) from Table 1:")
for i, j in itertools.combinations(range(4), 2):
(ci, di), (cj, dj) = P[i], P[j]
o = orci(dj, cj - dj, di, ci - di)
print(f" {LABELS[j]} vs {LABELS[i]}: {o[0]:.2f} ({o[1]:.2f}–{o[2]:.2f}) p = {o[3]:.1e}")
tab = np.array([[d, c - d] for c, d in P])
chi2, pchi, dfchi, _ = stats.chi2_contingency(tab)
print(f"chi-square test of equal CFR across the 4 periods (patient-level): chi2 = {chi2:.0f}, df = {dfchi}, p = {pchi:.1e}")
print("\n=== 6. Allowing for between-outbreak variation (not in the paper) ===")
rho = stats.spearmanr(yr, dth / n)
print(f"Spearman correlation, outbreak CFR vs year (34 outbreaks): rho = {rho.statistic:.2f}, p = {rho.pvalue:.2f}")
def logit_fit(X):
b = np.zeros(X.shape[1])
for _ in range(100):
p = 1 / (1 + np.exp(-X @ b))
W = n * p * (1 - p)
b = np.linalg.solve(X.T @ (W[:, None] * X), X.T @ (W * (X @ b) + dth - n * p))
p = 1 / (1 + np.exp(-X @ b))
cov = np.linalg.inv(X.T @ ((n * p * (1 - p))[:, None] * X))
phi = np.sum((dth - n * p) ** 2 / (n * p * (1 - p))) / (len(n) - X.shape[1])
return b, cov, phi
Xp = np.column_stack([np.ones_like(yr)] + [((yr >= a) & (yr <= b)).astype(float) for a, b in PERIODS[1:]])
b, cov, phi = logit_fit(Xp)
se_b, se_q = math.sqrt(cov[3, 3]), math.sqrt(cov[3, 3] * phi)
dfq = len(n) - Xp.shape[1]
q_or = (math.exp(b[3]), math.exp(b[3] - stats.t.ppf(.975, dfq) * se_q), math.exp(b[3] + stats.t.ppf(.975, dfq) * se_q),
2 * stats.t.sf(abs(b[3] / se_q), dfq))
print(f"logistic model with period factors: OR 2009–2019 vs 1976–1986 = {math.exp(b[3]):.3f} "
f"(binomial 95% CI {math.exp(b[3] - Z * se_b):.3f}–{math.exp(b[3] + Z * se_b):.3f}); "
f"Pearson dispersion = {phi:.0f}")
print(f" quasi-binomial (dispersion-scaled, t on {dfq} df) 95% CI {q_or[1]:.3f}–{q_or[2]:.3f}, p = {q_or[3]:.3f}")
Xt = np.column_stack([np.ones_like(yr), (yr - 1976) / 10])
bt, covt, phit = logit_fit(Xt)
set_q = math.sqrt(covt[1, 1] * phit)
dft = len(n) - 2
print(f"linear trend: OR per decade {math.exp(bt[1]):.3f}; binomial p = {2 * stats.norm.sf(abs(bt[1] / math.sqrt(covt[1, 1]))):.1e}; "
f"quasi-binomial 95% CI {math.exp(bt[1] - stats.t.ppf(.975, dft) * set_q):.3f}–"
f"{math.exp(bt[1] + stats.t.ppf(.975, dft) * set_q):.3f}, p = {2 * stats.t.sf(abs(bt[1] / set_q), dft):.3f}")
# DerSimonian-Laird random-effects meta-regression of outbreak logit CFR on decade (0.5 continuity correction)
dd, nn = dth + 0.5, n + 1
lg, v = np.log(dd / (nn - dd)), 1 / dd + 1 / (nn - dd)
w = 1 / v
B = np.linalg.solve(Xt.T @ (w[:, None] * Xt), Xt.T @ (w * lg))
Q = np.sum(w * (lg - Xt @ B) ** 2)
Pm = np.diag(w) - (w[:, None] * Xt) @ np.linalg.inv(Xt.T @ (w[:, None] * Xt)) @ (Xt.T * w)
tau2 = max(0.0, (Q - dft) / np.trace(Pm))
w = 1 / (v + tau2)
cvr = np.linalg.inv(Xt.T @ (w[:, None] * Xt))
B = cvr @ (Xt.T @ (w * lg))
sr = math.sqrt(cvr[1, 1])
print(f"random-effects meta-regression: tau^2 = {tau2:.2f}; OR per decade {math.exp(B[1]):.3f} "
f"({math.exp(B[1] - Z * sr):.3f}–{math.exp(B[1] + Z * sr):.3f}), p = {2 * stats.norm.sf(abs(B[1] / sr)):.3f}")
share = {k: 100 * float(n[[r[1] == k and r[0] == 2014 for r in T1]].sum()) / C for k in ("Liberia", "Sierra Leone")}
print(f"share of all cases from the 2014 Liberia and Sierra Leone outbreaks: {share['Liberia'] + share['Sierra Leone']:.1f}%")
print("\n=== 7. Single-year reading of 'OR in 2019 vs 1976' ===")
m76, m19 = yr == 1976, yr == 2019
print(f"1976 outbreaks: {int(dth[m76].sum())}/{int(n[m76].sum())} = {100 * dth[m76].sum() / n[m76].sum():.1f}%; "
f"2019 outbreak (Uganda): {int(dth[m19].sum())}/{int(n[m19].sum())} = 100% (OR undefined: no survivors)")
o = orci(2150, 3220 - 2150, dth[m76].sum(), n[m76].sum() - dth[m76].sum())
print(f"2018–2019 DRC outbreak (2150/3220 = {100 * 2150 / 3220:.1f}%) vs 1976: OR {o[0]:.2f} ({o[1]:.2f}–{o[2]:.2f}), p = {o[3]:.1e}")
print("\n=== 8. Attempts to reproduce the Table 3 ORs ===")
for i, (c, d) in enumerate(P):
oc, od = C - c, D - d
o = orci(d, c - d, od, oc - od)
rr = (d / c) / (od / oc)
se = math.sqrt(1 / d - 1 / c + 1 / od - 1 / oc)
print(f"{LABELS[i]} vs all other periods: OR {o[0]:.2f} ({o[1]:.2f}–{o[2]:.2f}); risk ratio {rr:.2f} "
f"({rr * math.exp(-Z * se):.2f}–{rr * math.exp(Z * se):.2f}); inverse risk ratio {1 / rr:.2f}; "
f"period CFR / overall CFR {(d / c) / (D / C):.2f} [Table 3: {T3_OR[i]}]")
print("\n=== 9. Other quantities (Table 2, Results) ===")
for s, (c, d, pr) in T2_SEX.items():
print(f"{s}: CFR {d}/{c} = {100 * d / c:.1f}% (printed {pr}); share of cases {100 * c / C:.1f}%; share of deaths {100 * d / D:.1f}%")
print(f"male + female cases {sum(v[0] for v in T2_SEX.values())}, deaths {sum(v[1] for v in T2_SEX.values())}")
print(f"age 15–60 share of cases: {100 * T2_AGE['15–60'] / C:.1f}% (Results: 70%); age groups sum {sum(T2_AGE.values())}")
print(f"health facility + household cases: {sum(T2_SOURCE.values())}")
# ------------------------------------------------------------------ figures
BG, INK, MUTED, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK, "ytick.color": INK, "axes.edgecolor": MUTED,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: ORs on one log axis
rows = [
("Abstract: “OR in 2019 vs 1976”", ABS_OR, ORANGE, "o", "1.6 (1.5–1.8)"),
("Results text: “OR in 2019 vs 1976”", RES_OR, ORANGE, "o", "1.6 (1.5–1.6)"),
] + [(f"Table 3, column {l}", t, ORANGE, "o", f"{t[0]} ({t[1]}–{t[2]})") for l, t in zip(LABELS, T3_OR)] + [
("Table 3 CFRs 62% vs 71%", o_rep[:3], BLUE, "D", f"{o_rep[0]:.2f} ({o_rep[1]:.2f}–{o_rep[2]:.2f})"),
("Table 1 counts, 2009–19 vs 1976–86", o_crude[:3], BLUE, "D",
f"{o_crude[0]:.2f} ({o_crude[1]:.2f}–{o_crude[2]:.2f})"),
("Same, outbreak-level CI", q_or[:3], BLUE, "s",
f"{q_or[0]:.2f} ({q_or[1]:.2f}–{q_or[2]:.2f})"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.4))
ypos = np.arange(len(rows))[::-1].astype(float)
ypos[-3:] -= 0.5
for y, (lab, (o, lo, hi), col, mk, txt) in zip(ypos, rows):
hollow = mk == "s"
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot(o, y, mk, ms=8, color=col, mfc=BG if hollow else col, mew=2)
ax.text(4.3, y, txt, va="center", ha="left", fontsize=10, color=col if col == BLUE else "#b8481c")
ax.axvline(1, color=MUTED, lw=0.8, zorder=0)
ax.text(1.1, ypos[0] + 0.85, "OR = 1, no change", ha="left", va="bottom", fontsize=10, color=MUTED)
ax.set_xscale("log")
ax.set_xlim(0.03, 4.2)
ax.set_xticks([0.05, 0.1, 0.2, 0.5, 1, 2])
ax.set_xticklabels(["0.05", "0.1", "0.2", "0.5", "1", "2"])
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in rows])
ax.set_ylim(ypos[-1] - 0.7, ypos[0] + 1.5)
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio of death (95% CI), log scale")
ax.text(0.9, ypos[0] + 0.85, "← lower CFR later", ha="right", va="bottom", fontsize=10, color=MUTED)
fig.legend(handles=[
Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper (all P < 0.001)"),
Line2D([], [], color=BLUE, marker="D", ms=8, lw=2, label="Recomputed from the paper’s numbers, patient-level CI"),
Line2D([], [], color=BLUE, marker="s", ms=8, mfc=BG, mew=2, lw=2,
label="Recomputed, CI allowing for variation between outbreaks (quasi-binomial)"),
], loc="lower left", ncol=1, frameon=False, fontsize=10, bbox_to_anchor=(0.02, 0.0))
fig.subplots_adjust(left=0.40, right=0.80, top=0.97, bottom=0.30)
save(fig, "fig1")
# Figure 2: period CFRs, reported vs pooled from Table 1
fig, ax = plt.subplots(figsize=(7.2, 3.6))
cats = LABELS + ["Overall"]
rep = T3_CFR + [ABS_CFR[0]]
rec = pooled_cfr + [100 * D / C]
x = np.arange(len(cats), dtype=float) * 1.4
x[-1] += 0.35
for xi, r, c in zip(x, rep, rec):
ax.plot([xi - 0.07, xi + 0.07], [r, c], color=GRID, lw=2, zorder=1)
ax.plot(xi - 0.07, r, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(xi + 0.07, c, "D", ms=8, color=BLUE, zorder=3)
ax.text(xi - 0.19, r, f"{r}%", va="center", ha="right", fontsize=10, color="#b8481c")
ax.text(xi + 0.19, c, f"{c:.1f}%", va="center", ha="left", fontsize=10, color=BLUE)
ax.set_xticks(x)
ax.set_xticklabels([f"{l}\n{c:,} cases".replace(",", " ") for l, (c, _) in zip(LABELS, P)] + [f"Overall\n{C:,} cases".replace(",", " ")])
ax.set_ylim(0, 100)
ax.set_ylabel("Case fatality rate (%)")
ax.set_xlim(-0.6, x[-1] + 0.6)
ax.axvline((x[-2] + x[-1]) / 2, color=GRID, lw=0.8)
ax.grid(axis="y", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.legend(handles=[
Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0, label="Reported (Table 3; Abstract for overall)"),
Line2D([], [], color=BLUE, marker="D", ms=8, lw=0, label="Deaths ÷ cases from Table 1"),
], loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 1.0))
fig.subplots_adjust(left=0.1, right=0.97, top=0.86, bottom=0.2)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
The abstract's deaths-averted figures disagree with the Results and with the stated status quo
BMJ Glob Health (2020) · doi:10.1136/bmjgh-2019-001886 · PMC7170464
Summary
The abstract reports that screening options 1–3 would avert "7700 (27%)" cervical cancer (CC) deaths and option 4 "11 800 (40%)" deaths among women living with HIV in care in Kenya, 2020–2040. The Results report 10 702 (23.9%) and 15 612 (34.9%) for the same comparisons. The Results figures match the paper's stated status-quo total of 44 742 deaths: 10 702 / 44 742 = 23.9% and 15 612 / 44 742 = 34.9%. The abstract's do not: 7700 / 44 742 = 17.2% and 11 800 / 44 742 = 26.4%. Its count and percentage pairs imply a status-quo total of about 28 500–29 500 deaths, which appears nowhere in the paper or its supplement. The case figures agree between abstract and Results, so the conflict is limited to mortality. It affects the abstract's headline estimate of lives saved for every option. The direction and ranking of the options do not change.
What the paper reports
Abstract, Results paragraph:
All options will reduce CC cases and deaths compared with no scale-up. Options 1–3 will perform similarly, averting approximately 28 000 (33%) CC cases and 7700 (27%) deaths.
In contrast, option 4 would maximise health outcomes, averting 43 200 (50%) CC cases and 11 800 (40%) deaths, given greater care engagement.
Results, "Burden of HPV-related cervical disease under status quo", second paragraph:
These will reach a cumulative total of 86 115 CC cases and 44 742 CC-related deaths between 2020 and 2040 (figure 2A, B and supplementary material table S3).
Results, "Impact of CC screening and treatment options", first paragraph:
VIA-Cryo only, VIA-Cryo plus LEEP and HPV-DNA-Cryo plus LEEP are expected to perform similarly, averting approximately 32.6% (28 000) of CC cases and 23.9% (10 702) of CC deaths by 2040, compared with status quo (figure 2A, B - and supplementary table S3).
Results, "Impact of CC screening and treatment options", second paragraph:
In contrast to the above, scaling up an option with enhanced technologies (ie, same-day HPV-DNA test or digitally enhanced VIA) could reduce CC cases and deaths by 50.2% (43 200) and 34.9% (15 612), respectively (figure 2A, B - and supplementary table S3).
Methods, "Scale-up of CC screening and treatment care options":
Outcomes were compared with a scenario with no scale-up in CC screening and treatment, henceforth referred to as status quo.
All results were based on an average of 100 model runs.
The Discussion gives no mortality figures. The online supplement's Table S3 ("Predicted health outcomes among WHIV in care in Kenya from 2020 to 2040") lists CC cases and deaths in four five-year periods, 2020–2024 to 2035–2039, for the status quo and options 1–3. Table S5 gives the same for the sensitivity scenario "Enhanced technologies only", which matches the description of option 4.
What the paper's own numbers show
Method. I divided each reported count of deaths or cases averted by the status-quo total stated in the Results (44 742 deaths, 86 115 cases). I also divided each count by its own reported percentage, which gives the status-quo total that the count and percentage pair implies. As a second, independent source, I summed the five-year periods in supplementary Tables S3 and S5 to get each scenario's cumulative cases and deaths, and the numbers averted relative to the status quo. Finally I checked every contiguous window of those five-year periods, to test whether some shorter horizon reproduces the abstract's figures. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Status-quo CC deaths, 2020–2040 | Results, status-quo paragraph | 44 742 | Table S3 sum (2020–2039): 41 704 (3038 fewer) |
| Status-quo CC cases, 2020–2040 | Results, status-quo paragraph | 86 115 | Table S3 sum (2020–2039): 86 116 |
| Options 1–3, deaths averted | Abstract | 7700 (27%) | 7700 / 44 742 = 17.2%; implied status quo 7700 / 0.27 = 28 519 |
| Options 1–3, deaths averted | Results, "Impact", para 1 | 10 702 (23.9%) | 10 702 / 44 742 = 23.9%; implied status quo 44 778 |
| Option 4, deaths averted | Abstract | 11 800 (40%) | 11 800 / 44 742 = 26.4%; implied status quo 11 800 / 0.40 = 29 500 |
| Option 4, deaths averted | Results, "Impact", para 2 | 15 612 (34.9%) | 15 612 / 44 742 = 34.9%; implied status quo 44 734 |
| Abstract minus Results, deaths averted | Abstract vs Results | — | options 1–3: −3002 deaths, +3.1 points; option 4: −3812 deaths, +5.1 points |
| Abstract percentages applied to 44 742 | Abstract | 27%, 40% | 12 080 and 17 897 deaths, matching neither count |
| Options 1–3, cases averted | Abstract; Results | 28 000 (33%); 28 000 (32.6%) | 28 000 / 86 115 = 32.5%; Table S3 mean of the three options 28 096 (32.6%) |
| Option 4, cases averted | Abstract; Results | 43 200 (50%); 43 200 (50.2%) | 43 200 / 86 115 = 50.2%; Table S5 "Enhanced technologies only" 43 226 (50.2%) |
| Options 1–3, deaths averted, from supplement | Table S3 | — | 8978, 9633, 10 236 (mean 9616; 23.1% of 41 704) |
| Option 4, deaths averted, from supplement | Table S5 | — | 13 898 (33.3% of 41 704) |
| Option 4 advantage over options 1–3 | Abstract vs Results | 4100 deaths, 13 points (abstract) | 4910 deaths, 11.0 points (Results) |
The Results' two mortality pairs both imply a status quo within 36 deaths of the stated 44 742, the difference expected from rounding the percentages to one decimal. The abstract's two pairs imply a status quo of 28 519 and 29 500. No death total of that size appears in the main text or in Tables S3 and S5, and the paper describes only one status-quo scenario. The case figures in the abstract and Results agree with each other, with the stated 86 115 and with the supplement, so the inconsistency is confined to deaths.
The supplement does not settle which mortality figures are right. Its status-quo deaths sum to 41 704, not 44 742. The gap of 3038 is about one year of deaths at the 2035–2039 rate (14 042 / 5 = 2808 per year), so the text's total may include the year 2040, which the five-year periods leave out. But the text's case total (86 115) equals the 2020–2039 sum (86 116), so the case and death totals in the same sentence would then cover different spans. Against the supplement's own baseline, the deaths averted (mean 9616 for options 1–3, 13 898 for option 4) are closer to the Results' figures than to the abstract's in both count and percentage (23.1% and 33.3%).
Figure
Figure 1. The four outcome statements the abstract and the Results both make, as short excerpts (the "Options 1–3" and "Option 4" prefixes are added to identify each statement; the numbers are as printed). Black rows and grey connectors: cases averted, which agree. Orange rows and connectors: deaths averted, which differ by 3002 deaths for options 1–3 and 3812 for option 4. Blue, bottom: each count of deaths averted divided by the status-quo total that the Results state, 44 742 deaths. The Results' percentages are reproduced. The abstract's are not: its counts correspond to 17.2% and 26.4%, not the printed 27% and 40%.
Figure 2. The cumulative status-quo CC death total, as stated in the Results (orange) and as implied by the paper's other numbers (blue): each reported count of deaths averted divided by its reported percentage, and the sum of the four five-year periods of supplementary Table S3. The Results pairs imply 44 778 and 44 734, against a stated 44 742. The supplement sums to 41 704. The abstract pairs imply 28 519 and 29 500, a baseline that appears nowhere in the paper. The axis starts at zero.
Why it matters
- The headline mortality estimates in the abstract are internally inconsistent. For both groups of options, the abstract's count and percentage cannot both be right against the paper's own status quo. Read against 44 742 deaths, the abstract understates the lives saved (by 3002 for options 1–3 and 3812 for option 4) and overstates the relative reduction (by about 3 and 5 percentage points). Most readers, and the indexing services that reproduce the abstract, see only these figures. In a modelling study meant to inform programme choices, the number of deaths averted is a main output.
- The comparison between options shifts. The abstract puts option 4's advantage over options 1–3 at 4100 deaths and 13 percentage points. The Results put it at 4910 deaths and 11.0 points.
- The qualitative conclusions are not affected. Every version of the numbers (abstract, Results, supplement) agrees that all options reduce deaths, that options 1–3 perform similarly, and that option 4 averts the most deaths. The case estimates, incidence projections and programmatic findings about losses to follow-up and screening frequency do not depend on the disputed figures.
- A second, smaller inconsistency sits between the main text and the supplement. The status-quo death total in the text (44 742) is 3038 higher than the supplement's tabulated periods (41 704), while the case total in the same sentence matches the supplement. Neither the abstract's nor the Results' mortality figures can be reproduced exactly from Tables S3 and S5.
Explanations considered
- Rounding of approximate counts. Ruled out. The abstract calls the figures approximate, but 7700 against 44 742 is 17.2%, nowhere near 27%. Reaching 27% would take 12 080 deaths averted, and no rounding of 10 702 gives 7700.
- A different baseline or outcome definition for the abstract. Not supported. The Methods define a single status quo, and the abstract's case percentages (33%, 50%) are computed against the same 86 115 cases as the Results. A death baseline of 28 500–29 500 appears nowhere in the text or supplement.
- A shorter time horizon. Does not account for the numbers. Of all contiguous windows of the supplement's five-year periods, only 2025–2039 gives percentages near the abstract's (27.5% for options 1–3 and 39.8% for option 4). But that window gives 9640 and 13 968 deaths averted, not 7700 and 11 800. It also gives 35.6% and 55.6% of cases averted, not the abstract's 33% and 50%. No window reproduces the abstract's counts.
- Deaths averted per case averted. Coincidence only. 7700 / 28 000 = 27.5%, which matches the abstract's "27%" for options 1–3, but 11 800 / 43 200 = 27.3%, not 40%.
- Reliance on a single option rather than the mean of options 1–3. Does not account for the numbers. The three options avert 8978, 9633 and 10 236 deaths according to Table S3, none close to 7700.
- Abstract carried over from an earlier analysis. Cannot be tested from the published material. The figures could come from an earlier model run or version of the manuscript, but the paper gives no indication of this, and whatever the source, the published abstract and Results disagree.
- A text-conversion artifact. Ruled out. The numbers are plain integers and percentages with no superscripts, minus signs or table cells involved. The same values appear in the PMC HTML page, the Europe PMC JATS XML (which uses thin-space thousands separators) and the PMC archive used here.
- The Results, not the abstract, being in error. Less likely. The Results' counts and percentages are consistent with the stated 44 742, and the supplement's deaths averted (9616 and 13 898 on average) are closer to the Results than to the abstract. The supplement's lower baseline (41 704) does mean that the Results' exact figures are not reproduced either; only the authors' model output can resolve which totals are intended.
- Side observation, not part of the main finding. The Results give the status-quo age-standardised incidence in 2020–2024 as 241 per 100 000 women-years. Table S3 gives 237 for the same period. The 2035–2039 values agree (218).
Status
- Version of record. The publisher's site (gh.bmj.com, reached through doi.org) and the publisher PDF returned HTTP 403 to automated requests. I checked the publisher-supplied text in three archived copies: the PMC HTML page, the Europe PMC full-text JATS XML, and the PMC archive read with
src/show.py. All three carry the numbers quoted above. Every quotation was checked verbatim against the PMC text. - Supplements. The online supplement (
bmjgh-2019-001886supp001.pdf, 22 pages, from the Europe PMC supplementary bundle) was text-extracted with PyMuPDF. Tables S3 and S5 supply the five-year values used above. Figure 2B of the paper (cumulative CC-related deaths), read visually, shows the status quo at about 45 000 deaths by 2040, consistent with the Results text. No numbers were taken from the figure. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, PMC notices or Europe PMC notices. The Crossref record has no update-to or updated-by entries, the PubMed record (PMID 32337077) has no CommentsCorrections entries, and Europe PMC lists no comments or corrections. The PubPeer API returned HTTP 404 for the DOI, and a web search for the title with "PubPeer" or "correction" found no thread or notice. - Sources. PMC article PMC7170464; Europe PMC full-text XML and supplementary files for PMC7170464; supplement
bmjgh-2019-001886supp001.pdf; Crossref; PubMed; Europe PMC; PubPeer; https://doi.org/10.1136/bmjgh-2019-001886.
Suggested correction
The authors could check the mortality figures against their model output and make the abstract agree with the Results. If the Results are correct, replace in the abstract
averting approximately 28 000 (33%) CC cases and 7700 (27%) deaths
with
averting approximately 28 000 (33%) CC cases and 10 700 (24%) deaths
and
averting 43 200 (50%) CC cases and 11 800 (40%) deaths
with
averting 43 200 (50%) CC cases and 15 600 (35%) deaths
The authors may also wish to state the period covered by the status-quo totals of 86 115 cases and 44 742 deaths, since the death total does not match the sum of the 2020–2039 periods in supplementary Table S3 (41 704) while the case total does. They may also wish to check the 2020–2024 age-standardised incidence (241 in the Results, 237 in Table S3).
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC7170464 (BMJ Glob Health 2020; doi:10.1136/bmjgh-2019-001886)
# Run: python recompute.py.txt (needs only the standard library and matplotlib)
# Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png) next to this file.
import itertools, os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---- Numbers printed in the main text -------------------------------------------------
SQ_CASES_TXT, SQ_DEATHS_TXT = 86115, 44742 # Results, status quo, "between 2020 and 2040"
ABS = {"opt1-3": dict(cases=28000, cases_pct=33, deaths=7700, deaths_pct=27),
"opt4": dict(cases=43200, cases_pct=50, deaths=11800, deaths_pct=40)}
RES = {"opt1-3": dict(cases=28000, cases_pct=32.6, deaths=10702, deaths_pct=23.9),
"opt4": dict(cases=43200, cases_pct=50.2, deaths=15612, deaths_pct=34.9)}
# ---- Online supplement (bmjgh-2019-001886supp001.pdf) ---------------------------------
# Table S3 (status quo, options 1-3) and Table S5 ("Enhanced technologies only" = option 4)
# CC cases and CC deaths for 2020-24, 2025-29, 2030-34, 2035-39
SUPP = {
"Status quo": ([15120, 18643, 21875, 30478], [6608, 9194, 11860, 14042]),
"VIA-Cryo only": ([12135, 12209, 13916, 20066], [6644, 8216, 8532, 9334]),
"VIA-Cryo plus LEEP": ([12452, 11907, 13890, 20187], [6502, 8354, 8254, 8961]),
"HPV-DNA-Cryo plus LEEP": ([12379, 11804, 13542, 19574], [6750, 8350, 7820, 8548]),
"Enhanced technologies only": ([11392, 8099, 9365, 14034], [6678, 8074, 6796, 6258]),
}
OPT13 = ["VIA-Cryo only", "VIA-Cryo plus LEEP", "HPV-DNA-Cryo plus LEEP"]
OPT4 = "Enhanced technologies only"
def pct(a, b): return 100.0 * a / b
print("== 1. Abstract and Results checked against the Results' status-quo totals ==")
print(f"status quo (Results): {SQ_CASES_TXT} cases, {SQ_DEATHS_TXT} deaths")
for src, d in (("Abstract", ABS), ("Results", RES)):
for k, v in d.items():
print(f"{src:8s} {k:6s} deaths {v['deaths']:6d} reported {v['deaths_pct']}% -> "
f"{v['deaths']}/{SQ_DEATHS_TXT} = {pct(v['deaths'], SQ_DEATHS_TXT):.1f}% ; "
f"implied status-quo deaths = {v['deaths']/(v['deaths_pct']/100):.0f}")
print(f"{'':8s} {k:6s} cases {v['cases']:6d} reported {v['cases_pct']}% -> "
f"{v['cases']}/{SQ_CASES_TXT} = {pct(v['cases'], SQ_CASES_TXT):.1f}% ; "
f"implied status-quo cases = {v['cases']/(v['cases_pct']/100):.0f}")
for k in ("opt1-3", "opt4"):
print(f"{k}: Results minus Abstract deaths averted = {RES[k]['deaths']-ABS[k]['deaths']}; "
f"percentage points (Abstract - Results) = {ABS[k]['deaths_pct']-RES[k]['deaths_pct']:.1f}")
print(f"{k}: abstract % applied to 44 742 = {ABS[k]['deaths_pct']/100*SQ_DEATHS_TXT:.0f} deaths")
print(f"option 4 gain over options 1-3: abstract {ABS['opt4']['deaths']-ABS['opt1-3']['deaths']} deaths, "
f"{ABS['opt4']['deaths_pct']-ABS['opt1-3']['deaths_pct']} points; Results "
f"{RES['opt4']['deaths']-RES['opt1-3']['deaths']} deaths, "
f"{RES['opt4']['deaths_pct']-RES['opt1-3']['deaths_pct']:.1f} points")
print("\n== 2. Supplement Tables S3/S5, sums over the four 5-year periods (2020-2039) ==")
sqc, sqd = sum(SUPP["Status quo"][0]), sum(SUPP["Status quo"][1])
print(f"status quo: cases {sqc} (Results text {SQ_CASES_TXT}, diff {SQ_CASES_TXT-sqc}); "
f"deaths {sqd} (Results text {SQ_DEATHS_TXT}, diff {SQ_DEATHS_TXT-sqd})")
av = {}
for k, (c, d) in SUPP.items():
if k == "Status quo": continue
ac, ad = sqc - sum(c), sqd - sum(d); av[k] = (ac, ad)
print(f"{k:28s} total cases {sum(c):6d} deaths {sum(d):6d} | averted cases {ac:6d} ({pct(ac,sqc):.1f}%) "
f"deaths {ad:6d} ({pct(ad,sqd):.1f}% of {sqd}; {pct(ad,SQ_DEATHS_TXT):.1f}% of {SQ_DEATHS_TXT})")
mc = sum(av[k][0] for k in OPT13) / 3; md = sum(av[k][1] for k in OPT13) / 3
print(f"mean of options 1-3: cases averted {mc:.0f} ({pct(mc,sqc):.1f}%), deaths averted {md:.0f} ({pct(md,sqd):.1f}%)")
print(f"range of options 1-3 deaths averted: {min(av[k][1] for k in OPT13)}-{max(av[k][1] for k in OPT13)}")
print("\n== 3. Every contiguous window of 5-year periods (looking for 7700/27% and 11 800/40%) ==")
labels = ["2020-24", "2025-29", "2030-34", "2035-39"]
for r in range(1, 5):
for s in range(0, 5 - r):
idx = range(s, s + r)
sq_d = sum(SUPP["Status quo"][1][i] for i in idx); sq_c = sum(SUPP["Status quo"][0][i] for i in idx)
o13_d = sq_d - sum(sum(SUPP[k][1][i] for i in idx) for k in OPT13) / 3
o13_c = sq_c - sum(sum(SUPP[k][0][i] for i in idx) for k in OPT13) / 3
o4_d = sq_d - sum(SUPP[OPT4][1][i] for i in idx); o4_c = sq_c - sum(SUPP[OPT4][0][i] for i in idx)
print(f"{labels[s]}..{labels[s+r-1]}: opt1-3 deaths {o13_d:7.0f} ({pct(o13_d,sq_d):5.1f}%) cases {o13_c:7.0f} "
f"({pct(o13_c,sq_c):5.1f}%) | opt4 deaths {o4_d:6.0f} ({pct(o4_d,sq_d):5.1f}%) cases {o4_c:6.0f} ({pct(o4_c,sq_c):5.1f}%)")
print("\n== 4. Side observation: status-quo age-standardised incidence 2020-2024 ==")
print("Results text: 241 per 100 000 women-years; Table S3 status quo ASI 2020-2024: 237")
# ---- Figures --------------------------------------------------------------------------
BG, INK, INK2, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e3e2de"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# Figure 1: abstract vs Results text, with the shared denominator
fig = plt.figure(figsize=(7.2, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 5.0); ax.axis("off")
def panel(x, y, w, h, title, sub):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=INK2, lw=1.0))
ax.text(x + 0.12, y + h - 0.2, title, fontsize=11.5, weight="bold", color=INK, va="center")
ax.text(x + 0.12, y + h - 0.45, sub, fontsize=10, color=INK2, va="center")
LX, RX, W = 0.15, 4.35, 2.7
panel(LX, 0.95, W, 3.6, "Abstract", "Results paragraph")
panel(RX, 0.95, W, 3.6, "Main text", "Results section, paragraphs 1–2")
rows = [ # y, left excerpt, right excerpt, connector label, conflict?
(3.55, "Options 1–3: 28 000 (33%)\nCC cases", "Options 1–3: 32.6% (28 000)\nof CC cases", "agree", False),
(2.80, "Options 1–3: 7700 (27%)\ndeaths", "Options 1–3: 23.9% (10 702)\nof CC deaths", "differ by\n3002 deaths", True),
(2.05, "Option 4: 43 200 (50%)\nCC cases", "Option 4: 50.2% (43 200)\nCC cases", "agree", False),
(1.30, "Option 4: 11 800 (40%)\ndeaths", "Option 4: 34.9% (15 612)\ndeaths", "differ by\n3812 deaths", True),
]
for y, lt, rt, lab, bad in rows:
col = ORANGE if bad else INK2
ax.text(LX + 0.15, y, lt, fontsize=10, color=ORANGE if bad else INK, va="center", linespacing=1.25)
ax.text(RX + 0.15, y, rt, fontsize=10, color=ORANGE if bad else INK, va="center", linespacing=1.25)
ax.annotate("", xy=(RX - 0.02, y), xytext=(LX + W + 0.02, y),
arrowprops=dict(arrowstyle="<->", color=col, lw=2 if bad else 1.2))
ax.text((LX + W + RX) / 2, y + 0.08, lab, fontsize=10, color=col, ha="center", va="bottom",
weight="bold" if bad else "normal", linespacing=1.1)
# shared denominator and recomputed check
ax.text(3.6, 0.70, "Status quo stated in the Results: 44 742 CC deaths, 2020–2040",
fontsize=10, color=INK, ha="center", va="center")
ax.text(LX + 0.15, 0.22, "7700 / 44 742 = 17.2% (not 27%)\n11 800 / 44 742 = 26.4% (not 40%)",
fontsize=10, color=BLUE, va="center", linespacing=1.3)
ax.text(RX + 0.15, 0.22, "10 702 / 44 742 = 23.9% (matches)\n15 612 / 44 742 = 34.9% (matches)",
fontsize=10, color=BLUE, va="center", linespacing=1.3)
ax.text(3.6, 4.80, "Orange: conflicting excerpts as printed. Blue: recomputed against the stated status quo.",
fontsize=10, color=INK2, ha="center", va="center")
save(fig, "fig1")
# Figure 2: status-quo death total stated vs implied by each reported pair
items = [
("Results: stated, 2020–2040", SQ_DEATHS_TXT, "stated"),
("Results: 10 702 ÷ 23.9%", RES["opt1-3"]["deaths"] / 0.239, "implied"),
("Results: 15 612 ÷ 34.9%", RES["opt4"]["deaths"] / 0.349, "implied"),
("Table S3: sum, 2020–2039", sqd, "implied"),
("Abstract: 7700 ÷ 27%", ABS["opt1-3"]["deaths"] / 0.27, "implied"),
("Abstract: 11 800 ÷ 40%", ABS["opt4"]["deaths"] / 0.40, "implied"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.4))
fig.subplots_adjust(left=0.33, right=0.94, top=0.83, bottom=0.17)
ys = list(range(len(items)))[::-1]
for (lab, v, kind), y in zip(items, ys):
c = ORANGE if kind == "stated" else BLUE
ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(v, y, "o", ms=9, color=c, zorder=3)
ax.text(v - 900, y + 0.05, f"{v:,.0f}".replace(",", " "), ha="right", va="bottom", fontsize=10, color=c)
ax.set_yticks(ys); ax.set_yticklabels([i[0] for i in items], color=INK)
ax.set_xlim(0, 50000); ax.set_ylim(-0.6, len(items) - 0.4)
ax.set_xticks(range(0, 50001, 10000)); ax.set_xticklabels([f"{t:,}".replace(",", " ") for t in range(0, 50001, 10000)])
ax.set_xlabel("Status-quo CC deaths (cumulative)", color=INK2)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2); ax.tick_params(colors=INK2, length=0)
ax.plot([], [], "o", ms=9, color=ORANGE, label="stated in the paper")
ax.plot([], [], "o", ms=9, color=BLUE, label="implied by the paper's numbers (recomputed)")
ax.legend(loc="lower left", bbox_to_anchor=(-0.45, 1.02), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nfigures written:", ", ".join(f"{n}.{e}" for n in ("fig1", "fig2") for e in ("svg", "png")))
Kilkari birth-timeliness effect: abstract CI conflicts with Table 5, whose P = 0.03 contradicts its CI
BMJ Glob Health (2021) · doi:10.1136/bmjgh-2021-005489 · PMC8728358
Summary
The Abstract reports that exposure to Kilkari immunisation calls was significantly associated with timely vaccination at birth, with a probit coefficient of 0.08 and a 95% CI of 0.08 to 0.24. A lower limit equal to the point estimate is not possible for a probit coefficient. The Results text and Table 5 give the same estimate as 0.08 (−0.08 to 0.24), an interval that includes zero, yet Table 5 prints P = 0.03. Back-calculated from that interval, the Wald test gives z = 0.98 and P = 0.33. The same check reproduces the printed P value in 127 of the 130 cells of Table 5, including the other four Kilkari cells, and the Kilkari birth cell is the only one where the printed P and the CI disagree about significance. So the paper's own numbers do not establish that the birth-timeliness effect is statistically significant. That effect is one of the paper's two positive Kilkari findings, and the Abstract, Discussion and Conclusion all state it.
What the paper reports
Abstract, Results:
Exposure to Kilkari content on immunisation was significantly associated with an increase in men’s knowledge (but not women’s) about child immunisation (OR: 1.23, 95% CI 1.02 to1.48) and an increase in the timeliness of the child receiving vaccination at birth (Probit coefficient: 0.08, 95% CI 0.08 to 0.24).
Abstract, Conclusion:
Mobile messaging programmes, supported by mass media messages, may provide one important source for bolstering awareness, uptake and timeliness of immunisation services.
Results, "Immunisation knowledge", paragraph 6:
Timely immunisation at birth was significantly associated with listening to ≥50% of the cumulative content of Kilkari immunisation calls (0.08, –0.08 to 0.24) as well as higher women’s knowledge scores (0.05, 0.02 to 0.08).
Table 5 title:
Table 5 Results of Heckman two-stage analysis of determinants of timely immunization (by vaccination time-groups)
Table 5, Kilkari row. Each time-group has three columns, headed "Odds Ratio", "95% CI" and "P value" (N=3230). The Birth triplet comes first, followed by 6–10 weeks, 10–14 weeks, 14–18 weeks and 9–10 months:
Listening to >=50% immunisation calls | 0.08 | -0.08 to 0.24 | 0.03 | 0.01 | -0.14 to 0.16 | 0.87 | 0.15 | 0.00 to 0.29 | 0.05 | 0.13 | -0.03 to 0.30 | 0.12 | 0.08 | -0.07 to 0.22 | 0.32
Methods, "Data analysis > Immunisation practice":
The analysis of determinants of full immunisation and timely immunisation of basic and comprehensive package, vaccination at birth, at 6–10 weeks, 10–14 weeks, 14–18 weeks and 9–10 months were done using Heckman probit model.
Discussion, first paragraph:
It was also associated with higher likelihood of children 0–12 months receiving birth vaccines on time but had no association with overall full and timely immunisation for the basic or comprehensive immunisation package.
Discussion, "Determinants of immunisation coverage":
Kilkari exposure was not associated with overall improvements of full and timely immunisation coverage but did increase timely immunisation at birth. This finding may be linked to the timing of Kilkari calls and parental listening patterns.
Conclusion:
While this intervention increased father’s knowledge and improved the timeliness of immunisation at birth, it did not increase mother’s knowledge, nor did it improve overall child (0–12 months) immunisation completeness and timeliness.
What the paper's own numbers show
Method. In a probit model the coefficient is on the scale of the linear index, so a 95% Wald interval is b ± 1.96·SE and is symmetric around b. The standard error can therefore be recovered from the printed limits as SE = (upper − lower) / (2 × 1.96), with z = b / SE and two-sided P = 2·(1 − Φ(|z|)). I applied this to every cell of Table 5: 26 rows × 5 time-groups = 130 estimates. For each cell I also computed the full range of the Wald P over the rounding of the estimate and both limits (±0.005, or ±0.05 where a limit is printed to one decimal, for example "-0.0"). I counted a printed P as consistent if its own rounding interval overlaps that range. I then asked what interval P = 0.03 would require at b = 0.08. The Table 3 odds ratio quoted in the Abstract was checked the same way on the log scale. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Kilkari → timely birth vaccination: CI centring | Abstract | 0.08 (95% CI 0.08 to 0.24) | midpoint 0.16; distance from the estimate 0.00 below and 0.16 above, which cannot occur for a Wald CI |
| Same estimate: CI centring | Results para 6; Table 5, Birth | 0.08 (−0.08 to 0.24) | midpoint 0.080; 0.16 on each side, so the estimate and CI fit each other |
| Same estimate: P value | Table 5, Birth | P = 0.03 | SE = 0.0816, z = 0.98, Wald P = 0.327 (0.28 to 0.37 over rounding); one-sided P = 0.164 |
| CI that P = 0.03 would need at b = 0.08 | derived | −0.08 to 0.24 (SE 0.0816) | |z| = 2.17, SE = 0.0369, 95% CI 0.008 to 0.152; the printed CI implies an SE 2.2 times larger |
| Estimate that P = 0.03 would need with the printed CI width | derived | 0.08 | 0.177 (CI 0.017 to 0.337) |
| Kilkari, 6–10 weeks (control) | Table 5 | 0.01 (−0.14 to 0.16), P = 0.87 | Wald P = 0.896: consistent |
| Kilkari, 10–14 weeks (control) | Table 5 | 0.15 (0.00 to 0.29), P = 0.05 | Wald P = 0.043 (0.030 to 0.058): consistent |
| Kilkari, 14–18 weeks (control) | Table 5 | 0.13 (−0.03 to 0.30), P = 0.12 | Wald P = 0.123: consistent |
| Kilkari, 9–10 months (control) | Table 5 | 0.08 (−0.07 to 0.22), P = 0.32 | Wald P = 0.280 (0.234 to 0.327): consistent |
| All Table 5 cells | Table 5 | 130 estimates with CI and P | 127 consistent within rounding; 3 not: the Kilkari birth cell (0.03 vs 0.28 to 0.37), Poster/Newspaper/Handouts at 10–14 weeks (0.97 vs 0.79 to 0.94) and AWW at 9–10 months (<0.01 vs 0.011 to 0.020) |
| Cells where printed P < 0.05 but the Wald P is ≥ 0.05 across the whole rounding range | Table 5 | — | 1: the Kilkari birth cell |
| Kilkari → men's knowledge (OR) | Abstract; Table 3 | OR 1.23 (1.02 to 1.48), P = 0.03 | log-scale midpoint 1.229, Wald P = 0.029: consistent |
| Kilkari in Table 4 (full and timely, basic and all 19) | Table 4 | P = 0.74, 0.56, 0.12, 0.11 | Wald P = 0.685, 0.589, 0.142, 0.125: consistent |
The Table 5 CI and estimate for the Kilkari birth cell fit each other, and the same check works for the rest of the table. Only the printed P = 0.03 is out of line, by a factor of about 11 in P and 2.2 in |z|. The Abstract CI shares its upper limit (0.24) with Table 5, and its lower limit has the same magnitude as Table 5's with the opposite sign.
Figure
Figure 1. All three rows show the same estimate, the probit coefficient of 0.08 for listening to at least 50% of Kilkari immunisation content, with timely vaccination at birth as the outcome. The vertical line marks no effect (0). Orange rows are printed in the paper. In the Abstract the interval starts at the point estimate. In the Results text and Table 5 the interval runs from −0.08 to 0.24, crosses zero and implies a Wald P of 0.33, although Table 5 prints P = 0.03. The blue row is recomputed: it shows the 95% CI that P = 0.03 would require around the same estimate, 0.008 to 0.152, less than half the printed width. No two rows agree, so the paper's three statements about this estimate (Abstract CI, Table 5 CI, Table 5 P) cannot all be right.
Figure 2. Each point is one cell of Table 5 that prints a numeric P value (100 of 130 cells; the other 30 print "<0.01"). The horizontal axis is the printed P. The vertical axis is the two-sided Wald P recomputed from the printed 95% CI, and the vertical bar is the range of that recomputed P over the rounding of the estimate and both limits. Both axes are logarithmic. The diagonal marks agreement, and the dashed lines mark 0.05. The 99 other cells lie on or near the diagonal, and their bars almost always reach it. The Kilkari birth cell (highlighted, labelled) sits far above it: printed P = 0.03, recomputed P = 0.33 (rounding range 0.28 to 0.37).
Why it matters
The paper's two positive findings for Kilkari are a higher immunisation knowledge score among fathers (OR 1.23, 1.02 to 1.48, which checks out at P = 0.029) and more timely vaccination at birth. The second appears in the Abstract's Results with an interval that excludes zero. The Results text calls it "significantly associated". The Discussion says Kilkari "did increase timely immunisation at birth" and explains the effect by the timing and listenership of the week-18 call. The Conclusion says the intervention "improved the timeliness of immunisation at birth". The Abstract's concluding sentence about bolstering "timeliness of immunisation services" relies mainly on this result, because Table 4 shows no association between Kilkari and overall timeliness (Wald P 0.142 and 0.125, printed 0.12 and 0.11).
The interval given in the body, −0.08 to 0.24, implies P ≈ 0.33. If that interval is correct, the data are compatible with no effect on birth-dose timeliness, the claims of a significant increase are not supported, and the Discussion's explanation for an effect at birth has nothing to explain. If instead the estimate and P = 0.03 are correct, the interval would be about 0.01 to 0.15. The effect would then remain significant and the printed CI would be wrong. The published record cannot tell these apart, and the Abstract interval matches neither. The direction of the estimate (positive, 0.08) is the same under every reading.
The finding that Kilkari had no association with overall full or timely immunisation (Table 4) is unaffected, and so is the association with fathers' knowledge (Table 3).
Explanations considered
- The Abstract quotes a different model, outcome or time-group. This does not account for the numbers. The Abstract labels the value a "Probit coefficient" for "vaccination at birth", which is the Birth column of Table 5 (Heckman two-stage, N = 3230). The Results text quotes the same estimate as 0.08 (–0.08 to 0.24). No other estimate of 0.08 for birth timeliness appears in the paper, and Table 4 covers overall completeness and timeliness, not the birth dose.
- Asymmetric interval from a transformation, bootstrap or profile likelihood. This does not account for the Abstract interval. Probit coefficients are reported on the linear-index scale, the paper's own CI for this estimate is symmetric (0.16 on each side), and an interval with zero width below the estimate cannot come from any of these methods. Rounding cannot produce it either.
- A minus sign lost in the Abstract. This would explain the Abstract interval exactly. The Abstract's lower limit has the same magnitude as Table 5's with the opposite sign, and the upper limits match. The same kind of discrepancy occurs elsewhere in the paper's tables. In Table 4, 9 of the 13 cells whose estimate lies more than 0.05 from its CI midpoint become centred if a positive upper limit is read as negative. One example is "Employed", printed in Table 4 as −0.28 (−0.38 to 0.18) and given in the Results text as "(-0.28, –0.38 to -0.18)". A lost sign does not explain the Table 5 P value.
- A conversion artifact in the archived text. This is excluded. The Europe PMC full-text JATS XML encodes the Table 5 minus sign as U+2212 and the Results-text minus as an en dash, while the Abstract string has no minus character at all. The Table 5 cells line up in five estimate / CI / P triplets, with no merged or shifted cells.
- Rounding of the printed values. This does not account for the P value. Over every combination of rounding of the estimate and both limits, the Wald P stays between 0.28 and 0.37.
- A one-sided test. This does not account for it. The one-sided P from the printed CI is 0.164.
- Table 5 P values computed with a method other than the Wald test (for example, a likelihood-ratio test or a different variance estimator). This is unlikely. The Wald check reproduces 127 of 130 Table 5 P values within rounding, including the other four Kilkari cells, and the two other mismatches are small and do not change any conclusion. A different method that matched every other cell but moved this one from 0.33 to 0.03 would be unusual.
- A one-digit slip in the P value. This is consistent with the numbers but not confirmed. The recomputed P rounds to 0.33, and the printed value is 0.03. If this is the explanation, the CI is right, the effect is not significant, and the text, Abstract, Discussion and Conclusion overstate it.
- The probit coefficients are mislabelled as odds ratios. The Table 5 column headers read "Odds Ratio" over values that include negative numbers, so the cells must be probit coefficients, as the Methods ("Heckman probit model") and the Abstract ("Probit coefficient") indicate; Table 4 heads the same kind of estimates "Coefficient". The header is a labelling error. It does not change the check, because the CI for an odds ratio would be symmetric on the log scale and could not contain negative values.
- The values are explained in a supplement. This is excluded. See Status.
Status
- Version of record. The publisher pages (gh.bmj.com, HTML and full-text PDF) returned HTTP 403 to automated access. I checked two independent renderings of the text BMJ deposited: the live PMC article (PMC8728358) and the Europe PMC full-text JATS XML (BMJ Glob Health vol. 6, Suppl 5, e005489). Both carry the Abstract, Results and Table 5 values quoted above. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. The only supplementary file, bmjgh-2021-005489supp001.pdf, contains Annexures 1–5: the Kilkari message content, the vaccination schedule, distributions of knowledge scores, distributions of full and timely immunisation, and the probit selection equation for having the MCP card. None reports the coefficient, SE, CI or P for the birth-timeliness outcome equation, and no data or code are provided.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI. PMC and Europe PMC list no correction notices. The PubMed record (PMID 34312153) has no linked comments or corrections. A web search for corrections or comments found none. PubPeer could not be checked directly, because its search page returned HTTP 403 and its API returned 404 to automated requests.
- Sources. PMC article PMC8728358; Europe PMC full-text XML and supplementary files for PMC8728358; Crossref; PubMed (E-utilities) record 34312153; trial registration NCT03576157 (cited in the paper).
Suggested correction
- Report the correct coefficient, SE, 95% CI and P value for Kilkari exposure in the Birth column of Table 5, taken from the original model output. As printed, the CI (−0.08 to 0.24) implies P ≈ 0.33, while the printed P = 0.03 implies a CI of about 0.01 to 0.15.
- Correct the Abstract's interval ("95% CI 0.08 to 0.24"), whose lower limit equals the point estimate, so that it matches the corrected Table 5.
- If the corrected P is above 0.05, revise the Abstract's Results and Conclusion, the Results sentence ("significantly associated"), the Discussion ("did increase timely immunisation at birth" and the explanation that follows) and the Conclusion ("improved the timeliness of immunisation at birth").
- Relabel the Table 5 column headers from "Odds Ratio" to "Coefficient", as in Table 4.
- Check the sign of the CI limits throughout Table 4. Thirteen cells there have an estimate far from the midpoint of the printed interval, nine of them consistent with a lost minus sign on the upper limit. Also check the four wealth cells in the "All 19 vaccines" full-immunisation column, whose intervals (for example 0.00 to 0.07 around 0.23) do not contain the estimate.
Recomputation script (python recompute.py.txt)
"""Recompute the Kilkari birth-timeliness estimate of PMC8728358
(BMJ Glob Health 2021, doi:10.1136/bmjgh-2021-005489).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract, Results: "Probit coefficient: 0.08, 95% CI 0.08 to 0.24" (Kilkari exposure, timely vaccination at birth)
"OR: 1.23, 95% CI 1.02 to1.48" (Kilkari exposure, men's knowledge)
Results, para 6: "(0.08, -0.08 to 0.24)"
Table 3, row 3: men 1.23 | 1.02 to 1.48 | 0.03
Table 5: every row, copied below as TABLE5 (Heckman two-stage probit, N=3230;
five time-groups, each an estimate | 95% CI | P value triplet)
Method: a probit coefficient's 95% Wald CI is b +/- 1.96 SE, so SE = (hi - lo) / (2 * 1.96),
z = b / SE and the two-sided P = 2 * (1 - Phi(|z|)). For an odds ratio the same is done on the log scale.
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.isf(0.025)
# ---------------------------------------------------------------- Table 5 (verbatim cells)
GROUPS = ["Birth", "6-10 weeks", "10-14 weeks", "14-18 weeks", "9-10 months"]
TABLE5 = """
Listening to >=50% immunisation calls | 0.08 | -0.08 to 0.24 | 0.03 | 0.01 | -0.14 to 0.16 | 0.87 | 0.15 | 0.00 to 0.29 | 0.05 | 0.13 | -0.03 to 0.30 | 0.12 | 0.08 | -0.07 to 0.22 | 0.32
Father | 0.01 | -0.02 to 0.03 | 0.65 | 0.01 | -0.02 to 0.03 | 0.59 | 0.01 | -0.02 to 0.03 | 0.53 | 0.02 | -0.01 to 0.05 | 0.11 | -0.02 | -0.04 to 0.01 | 0.17
Mother | 0.05 | 0.02 to 0.08 | <0.01 | -0.03 | -0.05 to 0.00 | 0.06 | -0.01 | -0.04 to 0.01 | 0.33 | -0.02 | -0.05 to 0.01 | 0.26 | -0.01 | -0.04 to 0.02 | 0.41
Gender of child - Male (Base: Female) | 0.07 | -0.04 to 0.18 | 0.23 | 0.00 | -0.10 to 0.11 | 0.94 | -0.06 | -0.17 to 0.05 | 0.28 | -0.14 | -0.27 to -0.0 | 0.04 | -0.21 | -0.32 to -0.1 | <0.01
OBC | 0.04 | -0.09 to 0.16 | 0.58 | 0.04 | -0.08 to 0.16 | 0.51 | -0.06 | -0.18 to 0.06 | 0.32 | -0.06 | -0.20 to 0.07 | 0.36 | 0.03 | -0.09 to 0.15 | 0.62
SC/ST | 0.11 | -0.04 to 0.25 | 0.17 | -0.13 | -0.27 to 0.01 | 0.08 | -0.09 | -0.24 to 0.06 | 0.23 | -0.14 | -0.31 to 0.03 | 0.11 | -0.09 | -0.24 to 0.05 | 0.21
Poor | -0.19 | -0.36 to -0.0 | 0.02 | 0.33 | 0.17 to 0.49 | <0.01 | 0.28 | 0.11 to 0.45 | <0.01 | 0.34 | 0.11 to 0.56 | <0.01 | 0.15 | -0.01 to 0.32 | 0.07
Middle | -0.06 | -0.24 to 0.12 | 0.52 | 0.32 | 0.15 to 0.49 | <0.01 | 0.23 | 0.05 to 0.41 | 0.01 | 0.35 | 0.11 to 0.59 | <0.01 | 0.30 | 0.12 to 0.47 | <0.01
Richer | -0.10 | -0.29 to 0.09 | 0.32 | 0.37 | 0.19 to 0.55 | <0.01 | 0.24 | 0.05 to 0.44 | 0.01 | 0.39 | 0.15 to 0.63 | <0.01 | 0.31 | 0.13 to 0.50 | <0.01
Richest | -0.34 | -0.55 to -0.10 | <0.01 | 0.38 | 0.19 to 0.57 | <0.01 | 0.36 | 0.16 to 0.57 | <0.01 | 0.50 | 0.25 to 0.75 | <0.01 | 0.36 | 0.16 to 0.55 | <0.01
Years of education | 0.01 | -0.01 to 0.03 | 0.33 | 0.00 | -0.01 to 0.02 | 0.68 | 0.00 | -0.02 to 0.01 | 0.87 | 0.01 | -0.01 to 0.03 | 0.21 | 0.02 | 0.00 to 0.03 | 0.06
Employed | 0.09 | -0.02 to 0.19 | 0.12 | -0.11 | -0.22 to -0.01 | 0.03 | -0.12 | -0.22 to -0.01 | 0.03 | -0.03 | -0.15 to 0.10 | 0.68 | -0.07 | -0.18 to 0.03 | 0.16
Daily purchase | -0.03 | -0.22 to 0.17 | 0.79 | -0.05 | -0.25 to 0.14 | 0.59 | -0.06 | -0.26 to 0.14 | 0.56 | -0.04 | -0.28 to 0.20 | 0.76 | 0.05 | -0.14 to 0.25 | 0.58
Pregnancy | 0.09 | -0.11 to 0.29 | 0.37 | 0.05 | -0.14 to 0.24 | 0.59 | 0.10 | -0.10 to 0.30 | 0.31 | 0.08 | -0.15 to 0.32 | 0.5 | -0.10 | -0.29 to 0.09 | 0.32
Number of children | -0.03 | -0.08 to 0.02 | 0.28 | -0.07 | -0.12 to -0.01 | <0.01 | -0.08 | -0.13 to -0.01 | <0.01 | -0.07 | -0.13 to -0.01 | 0.03 | -0.02 | -0.07 to 0.03 | 0.53
Owns personal mobile | 0.05 | -0.08 to 0.19 | 0.42 | 0.07 | -0.06 to 0.19 | 0.29 | 0.08 | -0.05 to 0.21 | 0.21 | 0.08 | -0.06 to 0.22 | 0.28 | -0.12 | -0.24 to 0.01 | 0.07
Phone with >12 hours | 0.06 | -0.09 to 0.20 | 0.45 | -0.06 | -0.20 to 0.08 | 0.39 | -0.03 | -0.18 to 0.11 | 0.65 | 0.00 | -0.17 to 0.17 | 1.00 | 0.04 | -0.11 to 0.18 | 0.62
Often with 0 balance | 0.10 | -0.08 to 0.28 | 0.28 | -0.09 | -0.27 to 0.08 | 0.31 | -0.03 | -0.20 to 0.15 | 0.77 | -0.07 | -0.28 to 0.13 | 0.49 | 0.02 | -0.15 to 0.20 | 0.81
Place of most of the vaccinations - AWW | -0.03 | -0.20 to 0.15 | 0.77 | -0.03 | -0.20 to 0.14 | 0.70 | -0.09 | -0.26 to 0.08 | 0.30 | -0.14 | -0.33 to 0.05 | 0.15 | -0.26 | -0.43 to -0.01 | <0.01
Asked advice from HW | -0.01 | -0.11 to 0.10 | 0.88 | -0.16 | -0.26 to -0.01 | <0.01 | -0.13 | -0.23 to -0.01 | 0.02 | -0.11 | -0.23 to 0.02 | 0.10 | -0.05 | -0.15 to 0.05 | 0.33
Received information from HW | 0.01 | -0.15 to 0.16 | 0.95 | 0.02 | -0.13 to 0.18 | 0.76 | -0.01 | -0.17 to 0.14 | 0.89 | -0.07 | -0.25 to 0.11 | 0.47 | 0.11 | -0.05 to 0.26 | 0.18
Television | 0.13 | 0.02 to 0.25 | 0.02 | -0.16 | -0.27 to -0.0 | <0.01 | -0.10 | -0.21 to 0.01 | 0.08 | -0.07 | -0.20 to 0.06 | 0.27 | -0.07 | -0.18 to 0.04 | 0.19
Poster / Newspaper / Handouts | -0.07 | -0.19 to 0.05 | 0.25 | 0.07 | -0.05 to 0.18 | 0.26 | 0.01 | -0.12 to 0.11 | 0.97 | 0.01 | -0.12 to 0.14 | 0.91 | 0.02 | -0.10 to 0.13 | 0.76
Hoshangabad | -0.17 | -0.37 to 0.04 | 0.11 | -0.13 | -0.34 to 0.08 | 0.22 | -0.20 | -0.41 to 0.00 | 0.05 | -0.22 | -0.46 to 0.02 | 0.07 | -0.06 | -0.26 to 0.14 | 0.57
Rewa | -0.51 | -0.71 to -0.30 | <0.01 | -0.67 | -0.82 to -0.50 | <0.01 | -0.74 | -0.89 to -0.50 | <0.01 | -0.77 | -0.93 to -0.60 | <0.01 | -0.53 | -0.68 to -0.30 | <0.01
Rajgarh | -0.24 | -0.40 to -0.01 | 0.01 | -0.44 | -0.60 to -0.20 | <0.01 | -0.34 | -0.50 to -0.10 | <0.01 | -0.39 | -0.57 to -0.20 | <0.01 | -0.28 | -0.43 to -0.10 | <0.01
"""
# Table 4 (verbatim cells; columns: full basic, full all-19, timely basic, timely all-19)
T4GROUPS = ["Full, basic 8", "Full, all 19", "Timely, basic 8", "Timely, all 19"]
TABLE4 = """
Listening to ≥50% immunisation calls | 0.03 | -0.12 to 0.17 | 0.74 | 0.04 | -0.10 to 0.19 | 0.56 | 0.09 | -0.03 to 0.21 | 0.12 | 0.09 | -0.02 to 0.21 | 0.11
Father | 0.00 | -0.02 to 0.03 | 0.91 | 0.00 | -0.03 to 0.02 | 0.85 | 0.01 | -0.01 to 0.03 | 0.49 | 0.01 | -0.01 to 0.03 | 0.38
Mother | -0.02 | -0.04 to 0.01 | 0.27 | 0.00 | -0.03 to 0.02 | 0.84 | -0.01 | -0.03 to 0.01 | 0.04 | -0.01 | -0.03 to 0.01 | 0.45
Gender of child—male (base: female) | -0.03 | -0.17 to 0.11 | 0.66 | -0.14 | -0.24 to 0.03 | 0.01 | -0.07 | -0.15 to 0.02 | 0.13 | -0.05 | -0.14 to 0.03 | 0.21
OBC | 0.01 | -0.11 to 0.13 | 0.82 | -0.01 | -0.13 to 0.11 | 0.91 | 0.00 | -0.09 to 0.10 | 0.93 | -0.01 | -0.10 to 0.09 | 0.91
SC/ST | 0.13 | -0.02 to 0.18 | 0.02 | -0.04 | -0.18 to 0.10 | 0.61 | -0.10 | -0.22 to 0.01 | 0.04 | -0.10 | -0.21 to 0.01 | 0.08
Poor | 0.26 | 0.11 to 0.42 | <0.01 | 0.23 | 0.00 to 0.07 | <0.01 | 0.21 | 0.08 to 0.34 | <0.01 | 0.22 | 0.10 to 0.35 | <0.01
Middle | 0.38 | 0.21 to 0.56 | <0.01 | 0.36 | 0.00 to 0.19 | <0.01 | 0.30 | 0.17 to 0.44 | <0.01 | 0.28 | 0.15 to 0.42 | <0.01
Richer | 0.30 | 0.13 to 0.48 | <0.01 | 0.28 | 0.00 to 0.10 | <0.01 | 0.33 | 0.19 to 0.47 | <0.01 | 0.30 | 0.16 to 0.44 | <0.01
Richest | 0.35 | 0.16 to 0.54 | <0.01 | 0.28 | 0.00 to 0.09 | <0.01 | 0.33 | 0.18 to 0.48 | <0.01 | 0.33 | 0.18 to 0.48 | <0.01
Years of education | -0.01 | -0.03 to 0.01 | 0.18 | -0.01 | -0.03 to 0.01 | 0.26 | 0.01 | 0.00 to 0.02 | 0.05 | 0.01 | 0.00 to 0.02 | 0.07
Employed | -0.28 | -0.38 to 0.18 | <0.01 | -0.26 | -0.36 to 0.10 | <0.01 | -0.08 | -0.16 to 0.00 | 0.04 | -0.09 | -0.17 to 0.02 | 0.03
Daily purchase | 0.00 | -0.19 to 0.19 | 0.96 | -0.03 | -0.22 to 0.16 | 0.74 | 0.02 | -0.13 to 0.17 | 0.78 | -0.02 | -0.17 to 0.13 | 0.76
Pregnancy | -0.16 | -0.35 to 0.04 | 0.11 | -0.10 | -0.30 to 0.09 | 0.28 | -0.01 | -0.16 to 0.14 | 0.89 | 0.03 | -0.11 to 0.18 | 0.65
Number of children | 0.00 | -0.05 to 0.05 | 0.95 | 0.00 | -0.05 to 0.05 | 0.94 | -0.06 | -0.10 to 0.02 | 0.00 | -0.06 | -0.10 to 0.01 | <0.01
Owns personal mobile | 0.01 | -0.12 to 0.13 | 0.02 | 0.02 | -0.10 to 0.14 | 0.73 | 0.05 | -0.05 to 0.14 | 0.36 | 0.04 | -0.05 to 0.14 | 0.36
Phone with >12 hours | -0.11 | -0.25 to 0.03 | 0.12 | -0.09 | -0.23 to 0.04 | 0.18 | -0.03 | -0.14 to 0.09 | 0.65 | -0.03 | -0.14 to 0.08 | 0.59
Often with 0 balance | 0.05 | -0.12 to 0.22 | 0.56 | 0.05 | -0.12 to 0.22 | 0.55 | -0.02 | -0.16 to 0.11 | 0.73 | 0.01 | -0.12 to 0.14 | 0.90
Place of most of the vaccinations—AWW | -0.04 | -0.21 to 0.13 | 0.05 | -0.03 | -0.20 to 0.14 | 0.72 | -0.10 | -0.24 to 0.03 | 0.13 | -0.10 | -0.23 to 0.04 | 0.15
Asked advice from health worker | -0.08 | -0.18 to 0.03 | 0.14 | -0.05 | -0.16 to 0.05 | 0.30 | -0.14 | -0.22 to 0.06 | <0.01 | -0.12 | -0.20 to 0.04 | <0.01
Received information from health worker | 0.13 | -0.02 to 0.27 | 0.10 | 0.11 | -0.04 to 0.26 | 0.15 | 0.05 | -0.07 to 0.17 | 0.38 | 0.04 | -0.08 to 0.15 | 0.55
Television | -0.10 | -0.21 to 0.01 | 0.06 | -0.10 | -0.20 to 0.01 | 0.08 | -0.09 | -0.18 to 0.01 | 0.03 | -0.12 | -0.20 to 0.05 | 0.01
Poster/newspaper/handouts | 0.08 | -0.04 to 0.19 | 0.04 | 0.08 | -0.03 to 0.19 | 0.16 | 0.01 | -0.08 to 0.10 | 0.90 | 0.02 | -0.07 to 0.11 | 0.65
Hoshangabad | 0.17 | -0.02 to 0.37 | 0.08 | 0.12 | -0.07 to 0.32 | 0.22 | -0.18 | -0.34 to 0.03 | 0.02 | -0.06 | -0.21 to 0.09 | 0.41
Rewa | -0.43 | -0.57 to 0.29 | <0.01 | -0.52 | -0.67 to 0.3 | <0.01 | -0.70 | -0.81 to 0.58 | <0.01 | -0.68 | -0.79 to 0.5 | <0.01
Rajgarh | -0.17 | -0.32 to 0.01 | 0.03 | -0.21 | -0.36 to 0.02 | 0.01 | -0.47 | -0.60 to 0.35 | <0.01 | -0.39 | -0.51 to 0.2 | <0.01
"""
def half_unit(s):
"""Half of the last printed decimal place: '0.24' -> 0.005, '-0.0' -> 0.05."""
d = len(s.split(".")[1]) if "." in s else 0
return 0.5 * 10 ** (-d)
def wald_p(b, se):
return 2 * stats.norm.sf(abs(b) / se)
def parse_cells():
cells = []
for line in TABLE5.strip().splitlines():
c = [x.strip() for x in line.split("|")]
for j, g in enumerate(GROUPS):
b_s, ci_s, p_s = c[1 + 3 * j: 4 + 3 * j]
lo_s, hi_s = ci_s.split(" to ")
cells.append(dict(row=c[0], group=g, b_s=b_s, lo_s=lo_s, hi_s=hi_s, p_s=p_s,
b=float(b_s), lo=float(lo_s), hi=float(hi_s)))
return cells
def check_cell(c):
"""Wald P from the printed CI, and its full range over the rounding of b, lo and hi."""
hb, hl, hh = half_unit(c["b_s"]), half_unit(c["lo_s"]), half_unit(c["hi_s"])
se = (c["hi"] - c["lo"]) / (2 * Z975)
p = wald_p(c["b"], se)
se_min = max((c["hi"] - c["lo"] - hl - hh) / (2 * Z975), 1e-9)
se_max = (c["hi"] - c["lo"] + hl + hh) / (2 * Z975)
babs_max = abs(c["b"]) + hb
babs_min = max(abs(c["b"]) - hb, 0.0)
p_min, p_max = wald_p(babs_max, se_min), wald_p(babs_min, se_max)
if c["p_s"].startswith("<"):
r_lo, r_hi = 0.0, float(c["p_s"][1:])
else:
r = float(c["p_s"])
r_lo, r_hi = max(r - half_unit(c["p_s"]), 0.0), r + half_unit(c["p_s"])
consistent = (r_lo <= p_max) and (r_hi >= p_min)
mid = (c["lo"] + c["hi"]) / 2
return dict(se=se, p=p, p_min=p_min, p_max=p_max, r_lo=r_lo, r_hi=r_hi, ok=consistent, mid=mid,
z=c["b"] / se)
cells = parse_cells()
for c in cells:
c.update(check_cell(c))
K = {c["group"]: c for c in cells if c["row"].startswith("Listening")}
kb = K["Birth"]
print("=== 1. Kilkari >=50% calls -> timely vaccination at birth (Table 5, Birth column) ===")
print(f" Table 5 / Results text: b = 0.08, 95% CI -0.08 to 0.24, P = 0.03")
print(f" CI midpoint = {kb['mid']:.3f}; distance below = {kb['b'] - kb['lo']:.2f}, above = {kb['hi'] - kb['b']:.2f}")
print(f" SE = {kb['se']:.4f}, z = {kb['z']:.2f}, two-sided Wald P = {kb['p']:.3f}")
print(f" over the rounding of b, lo, hi: P = {kb['p_min']:.2f} to {kb['p_max']:.2f}")
print(f" one-sided P (positive direction) = {kb['p'] / 2:.3f}")
ab_lo, ab_hi, ab_b = 0.08, 0.24, 0.08
print(f" Abstract: b = 0.08, 95% CI 0.08 to 0.24")
print(f" CI midpoint = {(ab_lo + ab_hi) / 2:.2f}; distance below = {ab_b - ab_lo:.2f}, above = {ab_hi - ab_b:.2f}")
print(f" abstract upper limit = Table 5 upper limit ({ab_hi} vs {kb['hi']}); "
f"abstract lower limit = {ab_lo}, Table 5 lower limit = {kb['lo']} (same magnitude, opposite sign)")
z03 = stats.norm.isf(0.03 / 2)
se03 = 0.08 / z03
lo03, hi03 = 0.08 - Z975 * se03, 0.08 + Z975 * se03
print(f" If b = 0.08 and P = 0.03 were both right: |z| = {z03:.2f}, SE = {se03:.4f}, "
f"95% CI {lo03:.3f} to {hi03:.3f}")
print(f" SE implied by the printed CI is {kb['se'] / se03:.1f} times the SE implied by P = 0.03")
print(f" Wald P / printed P = {kb['p'] / 0.03:.1f}; |z| for P = 0.03 / |z| from the CI = {z03 / kb['z']:.1f}")
b03 = z03 * kb["se"]
print(f" If the CI width were right and P = 0.03: b would be {b03:.3f} "
f"(CI {b03 - Z975 * kb['se']:.3f} to {b03 + Z975 * kb['se']:.3f})")
print(f" Wald P to two decimals: {kb['p']:.2f} (printed P = 0.03)")
print("\n=== 2. The other Kilkari cells of Table 5 (controls for the method) ===")
for g in GROUPS:
c = K[g]
print(f" {g:12s} b = {c['b_s']:>5s} ({c['lo_s']} to {c['hi_s']}), P = {c['p_s']:>5s} -> "
f"SE = {c['se']:.4f}, Wald P = {c['p']:.3f} (rounding range {c['p_min']:.3f}-{c['p_max']:.3f}) "
f"{'consistent' if c['ok'] else 'INCONSISTENT'}")
print("\n=== 3. All Table 5 cells: reported P vs Wald P, allowing for rounding ===")
n = len(cells)
bad = [c for c in cells if not c["ok"]]
print(f" cells checked: {n}; inconsistent beyond rounding: {len(bad)}")
for c in sorted(bad, key=lambda c: -abs(math.log10(max(c['p_min'], 1e-12)) - math.log10(max(c['r_hi'], 1e-12)))):
print(f" {c['row'][:38]:38s} {c['group']:12s} b = {c['b_s']:>5s} ({c['lo_s']} to {c['hi_s']}), "
f"P = {c['p_s']:>5s}; Wald P = {c['p']:.3f}, rounding range {c['p_min']:.3f}-{c['p_max']:.3f}")
sig_flip = [c for c in bad if (c["r_hi"] <= 0.05) != (c["p_min"] < 0.05)]
print(f" of these, cells where the reported P is below 0.05 but the whole rounding range of the Wald P is not:")
for c in cells:
if c["r_hi"] <= 0.05 and c["p_min"] >= 0.05:
print(f" {c['row'][:38]:38s} {c['group']:12s} P = {c['p_s']}, Wald P >= {c['p_min']:.3f}")
asym = [c for c in cells if abs(c["mid"] - c["b"]) > half_unit(c["b_s"]) + 0.5 * (half_unit(c["lo_s"]) + half_unit(c["hi_s"]))]
print(f" cells whose estimate is off the CI midpoint by more than rounding allows: {len(asym)}")
for c in asym:
print(f" {c['row'][:38]:38s} {c['group']:12s} b = {c['b_s']}, CI {c['lo_s']} to {c['hi_s']}, midpoint {c['mid']:.3f}")
print(f" Kilkari birth cell: estimate {kb['b_s']} vs CI midpoint {kb['mid']:.3f} (symmetric)")
print("\n=== 4. Abstract OR for men's knowledge (Table 3: 1.23, 1.02 to 1.48, P = 0.03) ===")
l = np.log([1.23, 1.02, 1.48])
se_or = (l[2] - l[1]) / (2 * Z975)
print(f" log-scale CI midpoint = {math.exp((l[1] + l[2]) / 2):.3f}; SE(log OR) = {se_or:.4f}; "
f"Wald P = {wald_p(l[0], se_or):.3f}: consistent with P = 0.03")
print("\n=== 5. Context: Table 4, estimates that lie far from the midpoint of their printed CI ===")
t4 = []
for line in TABLE4.strip().splitlines():
c = [x.strip() for x in line.split("|")]
for j, g in enumerate(T4GROUPS):
b, ci, pv = c[1 + 3 * j: 4 + 3 * j]
lo, hi = [float(v) for v in ci.split(" to ")]
t4.append((c[0], g, float(b), lo, hi, pv))
off = [r for r in t4 if abs((r[3] + r[4]) / 2 - r[2]) > 0.05]
print(f" Table 4 cells: {len(t4)}; estimate more than 0.05 from its CI midpoint: {len(off)}")
for r in off:
flip = -r[4]
print(f" {r[0][:38]:38s} {r[1]:15s} b = {r[2]:+.2f}, CI {r[3]:+.2f} to {r[4]:+.2f}, P = {r[5]:>5s}; "
f"midpoint {(r[3] + r[4]) / 2:+.3f}; with upper limit sign-flipped: midpoint {(r[3] + flip) / 2:+.3f}")
fixed = [r for r in off if r[4] > 0 and abs((r[3] - r[4]) / 2 - r[2]) <= 0.05]
print(f" of these, cells with a positive upper limit that become centred (within 0.05) if that limit is negative: "
f"{len(fixed)}; the other {len(off) - len(fixed)} are the four wealth cells of the 'Full, all 19' column "
f"(lower limit 0.00 for every one)")
k4 = [r for r in t4 if r[0].startswith("Listening")]
for r in k4:
se = (r[4] - r[3]) / (2 * Z975)
print(f" Table 4 Kilkari {r[1]:15s} b = {r[2]:.2f} ({r[3]:.2f} to {r[4]:.2f}), P = {r[5]}: Wald P = {wald_p(r[2], se):.3f}")
# ---------------------------------------------------------------- figures
INK, INK2, GRID, BG = "#0b0b0b", "#52514e", "#e4e3df", "#fcfcfb"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: the three intervals for the same estimate, one axis
rows = [
("Abstract", 0.08, 0.08, 0.24, ORANGE, "reported", "CI 0.08 to 0.24\nlower limit = estimate"),
("Results text and Table 5", 0.08, -0.08, 0.24, ORANGE, "reported",
f"CI −0.08 to 0.24, P = 0.03\nWald P from this CI: {kb['p']:.2f}"),
("CI that P = 0.03 would need", 0.08, lo03, hi03, BLUE, "recomputed",
f"CI {lo03:.3f} to {hi03:.3f}\n(SE {se03:.3f} vs {kb['se']:.3f})"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.4))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ys = [2, 1, 0]
for y, (lab, b, lo, hi, col, kind, note) in zip(ys, rows):
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
for x in (lo, hi):
ax.plot([x, x], [y - 0.09, y + 0.09], color=col, lw=2, zorder=3)
ax.plot([b], [y], "o", ms=9, color=col, mec=BG, mew=1.2, zorder=4)
ax.text(0.27, y, note, va="center", ha="left", fontsize=10, color=INK2)
ax.axvline(0, color=INK2, lw=0.9, zorder=1)
ax.text(0.0, 2.62, "no effect (0)", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10.5, color=INK)
ax.set_ylim(-0.6, 2.85)
ax.set_xlim(-0.13, 0.52)
ax.set_xticks(np.arange(-0.1, 0.31, 0.1))
ax.set_xticklabels([f"{t:.1f}".replace("-", "−") for t in np.arange(-0.1, 0.31, 0.1)])
ax.set_xlabel("Probit coefficient: Kilkari exposure → timely vaccination at birth", x=0.36)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
leg = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="reported in the paper"),
Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, label="recomputed from the paper's numbers")]
ax.legend(handles=leg, loc="lower center", bbox_to_anchor=(0.42, 1.04), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: reported vs Wald P for every Table 5 cell with a numeric P value
num = [c for c in cells if not c["p_s"].startswith("<")]
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ax.plot([0.004, 1.2], [0.004, 1.2], color=INK2, lw=0.9, zorder=1)
ax.axhline(0.05, color=GRID, lw=1.2, ls="--", zorder=1)
ax.axvline(0.05, color=GRID, lw=1.2, ls="--", zorder=1)
for c in num:
r = float(c["p_s"])
is_k = c is kb
if is_k:
continue
ax.plot([r, r], [max(c["p_min"], 0.004), c["p_max"]], color=BLUE, lw=1, alpha=0.35, zorder=2)
ax.plot([r], [max(c["p"], 0.004)], "o", ms=5, color=BLUE, alpha=0.55, mec="none", zorder=3)
r = float(kb["p_s"])
ax.plot([r, r], [kb["p_min"], kb["p_max"]], color=ORANGE, lw=2, zorder=4)
ax.plot([r], [kb["p"]], "o", ms=10, color=ORANGE, mec=BG, mew=1.2, zorder=5)
ax.annotate(f"Kilkari → timely birth vaccination\nprinted P = 0.03, Wald P from its CI = {kb['p']:.2f}",
xy=(r, kb["p"]), xytext=(0.0055, 0.6), fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.9))
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_xlim(0.004, 1.2)
ax.set_ylim(0.004, 1.2)
ticks = [0.01, 0.03, 0.05, 0.1, 0.3, 1]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.set_yticks(ticks)
ax.set_yticklabels([str(t) for t in ticks])
ax.minorticks_off()
ax.set_xlabel("P value printed in Table 5")
ax.set_ylabel("Wald P recomputed from the printed 95% CI")
ax.grid(color=GRID, lw=0.6, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
leg = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=9, label="Kilkari, birth column"),
Line2D([], [], color=BLUE, lw=1, marker="o", ms=6, alpha=0.6,
label=f"other Table 5 cells (n = {len(num) - 1})"),
Line2D([], [], color=INK2, lw=0.9, label="reported = recomputed")]
ax.legend(handles=leg, loc="lower right", frameon=False, fontsize=10, handlelength=1.6)
fig.tight_layout()
save(fig, "fig2")
print(f"\nfig2: {len(num)} cells with a numeric P value plotted ({n - len(num)} cells print P < 0.01)")
print("figures written: fig1.svg, fig1.png, fig2.svg, fig2.png")
Abstract reports p=0.002 for an odds ratio whose own 95% CI (0.86 to 7.36) includes 1
BMJ Glob Health (2022) · doi:10.1136/bmjgh-2022-009400 · PMC9494011
Summary
The abstract reports that identifying as "gay or queer" was associated with discrimination from healthcare providers since COVID-19 ("OR=2.51, 95% CI 0.86 to 7.36, p=0.002"), and Table 3 marks the same estimate, which belongs to the queer/asexual/pansexual/questioning group (aOR 2.52), as ** (p≤0.01). The interval includes 1: reconstructing the p-value from it on the log-odds scale gives p ≈ 0.09, and p=0.002 would need an interval of about 1.40 to 4.50; the bisexual healthcare estimate (aOR 2.74, CI 0.96 to 7.81, also **) likewise gives p ≈ 0.06. These two estimates are the paper's only basis for saying that queer and bisexual respondents had higher odds of discrimination from healthcare providers, so those statements in the abstract, Results and Discussion are not supported by the paper's own intervals. The gay respondents' healthcare estimate (aOR 3.45, CI 1.23 to 9.72) does exclude 1, at p ≈ 0.02 rather than the p≤0.01 its marker indicates, so the broader finding of elevated odds for some subgroups survives.
What the paper reports
Abstract, Results:
Identifying as gay or queer was associated with increased odds of experiencing the same or more discrimination from government representatives (OR=1.89, 95% CI 1.04 to 3.45, p=0.045) and healthcare providers (OR=2.51, 95% CI 0.86 to 7.36, p=0.002) due to COVID-19.
Results, paragraph on the final (healthcare provider) adjusted model:
Sexual orientation was associated with increased odds of experiencing discrimination or violence from healthcare providers since COVID-19, with gay (OR=3.45, 95% CI 1.22 to 9.71), bisexual (OR=2.74, 95% CI 0.96 to 7.81) or queer/asexual/pansexual (OR=2.51, 95% CI 0.86 to 7.36) orientation being significantly associated.
Table 3, multivariable logistic regression. Each outcome has five columns: OR | aOR | significance marker | 95% CI lower | 95% CI upper. The outcomes are, in order, discrimination based on sexual orientation and/or gender identity, from police and/or military, from government representatives, and from health providers. Heterosexual respondents are the reference category.
Gay | 1.83 | 1.82 | * | 1.05 | 3.17 | 0.98 | 1.27 | * | 1.05 | 1.80 | 1.73 | 1.89 | * | 1.04 | 3.45 | 3.07 | 3.45 | ** | 1.23 | 9.72
Bisexual | 1.36 | 1.30 | | 0.73 | 2.28 | 0.77 | 0.72 | | 0.39 | 1.32 | 1.07 | 1.13 | | 0.61 | 2.09 | 2.58 | 2.74 | ** | 0.96 | 7.81
Queer/asexual/pansexual/questioning | 1.93 | 2.00 | * | 1.07 | 3.72 | 0.78 | 0.62 | | 0.32 | 1.19 | 1.53 | 1.30 | * | 1.21 | 2.47 | 2.91 | 2.52 | ** | 0.86 | 7.36
Table 3 footer:
*P≤0.05; **p≤0.01; ***p=0.0.
Results, paragraph on the government-representatives model:
though only a gay (OR=1.89, 95% CI 1.04 to 3.45) and queer/asexual/pansexual (OR=1.30, 95% CI 0.68 to 2.47) orientation were significant when compared with their heterosexual counterparts.
Results, paragraph on the police/military model:
Sexual orientation nor gender identity was not significantly associated with a change in odds of experiencing violence or discrimination from police and/or military.
Discussion, second paragraph:
Specifically, gay and queer/asexual/pansexual/questioning individuals had higher odds of experiencing violence and discrimination from police, government representatives and healthcare providers when compared with their heterosexual peers. Bisexual individuals also had higher odds of experiencing violence and discrimination from healthcare providers.
Methods, Statistical analyses. No other inference method, test direction or significance threshold is described:
multivariable logistic regression analyses were conducted to examine the factors associated with experiences of stigma and discrimination
Table 1. The reference category for all sexual-orientation contrasts is small:
Heterosexual | 74 | 0.95
What the paper's own numbers show
Method. For a logistic-regression odds ratio with a Wald 95% interval, the interval is symmetric on the log scale, so SE(log OR) = [ln(upper) − ln(lower)] / (2 × 1.96). Then z = ln(OR) / SE, and the two-sided p = 2·[1 − Φ(|z|)]. The geometric midpoint √(lower × upper) should reproduce the point estimate, and this checks that the OR and CI belong together. To allow for rounding, I also recomputed p over every combination of the OR and both bounds shifted by ±0.005, which is the "p range" below. The conclusion does not depend on the reconstruction method in any case: a two-sided 95% interval that contains 1 cannot go with p < 0.05.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Healthcare providers, "gay or queer": OR 2.51 (0.86–7.36) | Abstract | p=0.002 | CI midpoint 2.52; SE 0.548; z 1.68; p = 0.093 (rounding range 0.091–0.095) |
| Same estimate, queer/asexual/pansexual/questioning: aOR 2.52 (0.86–7.36) | Table 3 | ** (p≤0.01) |
p = 0.091 (0.090–0.093) |
| Same estimate, 2.51 (0.86–7.36) | Results text | "significantly associated" | p = 0.093; CI includes 1 |
| 95% CI that OR 2.51 with p=0.002 would require | — | — | z 3.09; SE 0.298; 1.40 to 4.50 (the reported interval is 1.84 times wider on the log scale) |
| Same interval, read as a 90% / 99% CI | — | stated as 95% CI | p = 0.158 / 0.027 (neither is 0.002) |
| One-sided p for the same estimate | — | — | 0.046 |
| Healthcare providers, bisexual: aOR 2.74 (0.96–7.81) | Table 3; Results text | ** (p≤0.01); "significantly associated" |
p = 0.059 (0.058–0.061); CI includes 1 |
| Healthcare providers, gay: aOR 3.45 (1.23–9.72) | Table 3 | ** (p≤0.01) |
p = 0.019 (0.018–0.019): significant at 0.05, but not at the p≤0.01 the marker indicates |
| Healthcare providers, gay: OR 3.45 (1.22–9.71) | Results text | "significantly associated" | p = 0.019 (0.019–0.020) |
| Healthcare providers, lesbian: aOR 2.52 (0.50–12.87) | Table 3 | no marker | p = 0.265 (shown for comparison; consistent) |
| Government representatives, gay: aOR 1.89 (1.04–3.45) | Abstract; Table 3 | p=0.045; * |
p = 0.037 (0.036–0.039); broadly consistent |
| Government representatives, queer: aOR 1.30 (1.21–2.47) | Table 3 | * |
CI midpoint 1.73, not 1.30: the row's OR and CI cannot both be right |
| Government representatives, queer: OR 1.30 (0.68–2.47) | Results text | "significant" | CI midpoint 1.30 (coherent); p = 0.425 (0.415–0.435); CI includes 1 |
| Police/military, queer: aOR 0.62 (0.32–1.19) | Table 3 | no marker | p = 0.154; OR below 1 and not significant |
| Healthcare providers, disability: aOR 1.35 (1.07–1.71) | Abstract; Table 3 | p=0.009; ** |
p = 0.012 (0.010–0.015); consistent within rounding |
| Heterosexual reference group | Table 1 | n = 74 | 7758 − 6125 − 22 − 1158 − 379 = 74 (consistent) |
The healthcare-provider interval for the queer/asexual/pansexual/questioning group is internally coherent: its geometric midpoint (2.516) matches the reported point estimate (2.51 in the text, 2.52 in Table 3). What disagrees with the other numbers is the abstract's p-value and the table's ** marker. The same holds for the bisexual row (midpoint 2.738 against 2.74).
Figure
Figure 1. Adjusted odds ratios for discrimination from healthcare providers, by sexual orientation (reference: heterosexual, n = 74), on a log scale. The thin vertical line marks OR = 1. Orange dots and lines show each odds ratio and 95% CI as the paper reports it, and the orange text gives the significance the paper attaches to it (abstract p-value or Table 3 marker). Blue text gives the two-sided p recomputed from the same OR and CI with a log-scale Wald reconstruction. The dashed blue interval under the abstract row is the 95% CI (1.40–4.50) that an OR of 2.51 would need for p = 0.002 to be correct. The reported interval (0.86–7.36) is much wider and crosses 1. For the queer/asexual/pansexual/questioning and bisexual groups, the reported intervals include 1 (recomputed p = 0.09 and 0.06), although the abstract, Results text and Table 3 present them as significant at p = 0.002 or p ≤ 0.01. The gay-group interval excludes 1 (p = 0.019). All numbers come from the paper or from recompute.py.txt.
Why it matters
The abstract is the part of the paper most readers see, and it is where this estimate appears with p=0.002. At the conventional 0.05 level, the paper's own interval does not support a significant association between queer/asexual/pansexual/questioning identity and discrimination from healthcare providers. The same applies to bisexual identity (CI 0.96 to 7.81). The Discussion's statements that queer/asexual/pansexual/questioning and bisexual individuals "had higher odds" of discrimination from healthcare providers rest on these two estimates.
The abstract's wording, "gay or queer", also joins two estimates. The government figure (1.89, 1.04 to 3.45) is the gay estimate. The healthcare figure (2.51, 0.86 to 7.36) is the queer/asexual/pansexual/questioning estimate. The gay healthcare estimate is a different number (aOR 3.45, 1.23 to 9.72). A reader of the abstract cannot tell that the p=0.002 healthcare figure belongs to the 379-person queer/asexual/pansexual/questioning group rather than to gay respondents.
The same Discussion sentence also extends to police and government representatives, but the paper's other numbers do not support it for the queer group either. Its police/military aOR is 0.62 (0.32 to 1.19), and the Results state that sexual orientation was not significantly associated with police/military discrimination. Its government-representatives estimate appears as 1.30 with CI 1.21 to 2.47 in Table 3 (an interval whose midpoint is 1.73, not 1.30) and as 1.30 with CI 0.68 to 2.47 in the Results text. The text version is internally coherent and gives p ≈ 0.43.
What survives: the gay healthcare-provider estimate excludes 1 (p ≈ 0.02). So do the disability, immigrant and ethnic-minority estimates for healthcare providers in Table 3. The paper's broad finding that some marginalised subgroups reported higher odds of discrimination from healthcare providers therefore does not depend on the erroneous values. What fails is the claim of significance for the queer (and bisexual) healthcare-provider contrasts, which the abstract reports with a p-value that is over 40 times smaller than its own interval implies.
Explanations considered
- A correction already published.
known.py(Crossref, PMC and Europe PMC notices) returned nothing. Crossref shows noupdate-to,updated-byor relations. Europe PMC lists only the original article. Does not apply. - Text-conversion or extraction error. The same values appear in the abstract, the Results text and Table 3. They also appear identically in two separate renderings of the publisher's JATS full text (the PMC copy and the Europe PMC XML), where the Table 3 cells are separate elements (2.91 | 2.52 | ** | 0.86 | 7.36). Does not account for the numbers.
- The interval is not a 95% CI. Table 3 and the text both label it a 95% CI. Even read as a 99% CI, it implies p = 0.027. Read as a 90% CI, it implies p = 0.158. Neither gives 0.002. Does not account for it.
- One-sided test. The Methods never mention one-sided tests. A one-sided p from this interval is 0.046, still more than 20 times 0.002. Does not account for it.
- Unadjusted vs adjusted estimate. The abstract's 2.51 (0.86 to 7.36) matches the adjusted Table 3 row (aOR 2.52, the same interval) and not the crude OR (2.91). The p-value could belong to the crude model, but the paper reports no crude CI or p-value, and the abstract attaches p=0.002 to this OR and CI. This cannot be verified from the paper. Even if it were so, the Table 3
**on the adjusted row would still conflict with its own interval. - A global or joint test for sexual orientation. p=0.002 might be a likelihood-ratio test for the whole sexual-orientation variable. The paper does not report such a test, and the abstract presents the p-value as belonging to the single OR and CI. This would also not explain the
**on the queer and bisexual rows of Table 3. It does not reconcile the reported figures, though it cannot be excluded as the source of the number. - Wald vs profile-likelihood or exact intervals. With a reference group of only 74 heterosexual respondents, the log-OR may be somewhat skewed, and a likelihood-based interval could differ from a Wald one. But any valid two-sided 95% interval that contains 1 corresponds to p > 0.05 under the matching test. Moving p from about 0.09 to 0.002 would need an interval roughly half as wide on the log scale (1.40 to 4.50). Does not account for it.
- Multiplicity adjustment. Adjusting for multiple comparisons would make p-values larger, not smaller. Does not apply.
- The abstract's estimate is the gay estimate. The gay healthcare-provider estimate is OR 3.07 / aOR 3.45 (1.23 to 9.72; 1.22 to 9.71 in the text), with p ≈ 0.019. That is not p=0.002 either. In any case, 2.51 (0.86 to 7.36) is unambiguously the queer/asexual/pansexual/questioning row. Does not account for it.
- Rounding. Over every combination of the OR and the bounds shifted by ±0.005, p stays between 0.091 and 0.095. Does not account for it.
Status
As of 2026-10-23:
- Version of record. The publisher site (gh.bmj.com, reached through the doi.org redirect) returned HTTP 403 to automated access, and the PMC web page served only a browser check. I therefore checked the publisher-supplied JATS full text archived at Europe PMC (fullTextXML for PMC9494011). It matches the local PMC copy in the abstract sentence, the Results sentence and the Table 3 cells quoted above, so the error is in the version of record as archived.
- Supplementary material. None. The JATS has no supplementary-material elements, and Europe PMC reports hasSuppl = N. The data availability statement says data are available on reasonable request, at the discretion of the partner social networking company. No public data or output can settle whether the p-value, the marker or the interval is the wrong value.
- Corrections and notices. None found. I checked Crossref (
update-to,updated-by,relation: none), Crossref's Retraction Watch-derived update data (none), PMC and Europe PMC notices (none; Europe PMC search on the DOI returns only PMID 36130772), and a web search for a correction or erratum (no notice for this article). - PubPeer. The PubPeer API required human verification and could not be queried directly. Web searches found no PubPeer thread for this DOI. PubPeer is therefore not confirmed.
Suggested correction
- Abstract, Results: report the healthcare-provider estimate with a p-value consistent with its interval (about p=0.09 if the interval 0.86 to 7.36 is correct), and do not describe the association as significant. Alternatively, if the interval is the wrong value, give the correct one. Attribute each estimate to the right group: the government estimate (1.89, 1.04 to 3.45) is for gay respondents, and the healthcare estimate (2.51, 0.86 to 7.36) is for queer/asexual/pansexual/questioning respondents. If the abstract is meant to describe gay respondents' healthcare-provider estimate, that is aOR 3.45 (95% CI 1.23 to 9.72), p ≈ 0.02.
- Table 3, health-provider columns: remove the
**markers from the queer/asexual/pansexual/questioning row (CI 0.86 to 7.36) and the bisexual row (CI 0.96 to 7.81), and change the gay row's**to*(CI 1.23 to 9.72, p ≈ 0.02), unless the authors' output shows different intervals. - Table 3, government-representatives columns, queer/asexual/pansexual/questioning row: reconcile the lower bound (1.21 in the table vs 0.68 in the text) and the
*marker with the OR of 1.30. - Results (healthcare-provider and government-representatives paragraphs) and Discussion: restate which sexual-orientation groups had intervals excluding 1, specifically that the queer/asexual/pansexual/questioning and bisexual healthcare-provider estimates, and the queer police and government estimates, were not statistically significant at the 0.05 level.
- Methods: state how p-values and confidence intervals were computed (Wald or likelihood-based, two-sided) and the significance threshold used.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the p-values implied by the odds ratios and 95% CIs reported in
PMC9494011 (BMJ Glob Health 2022, doi:10.1136/bmjgh-2022-009400), and redraw
the figure for the write-up. Run: python recompute.py.txt
Every input number below is copied from the article (abstract, Results text,
Table 1 or Table 3). Outputs fig1.svg / fig1.png next to this file."""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z95 = norm.ppf(0.975)
def wald(or_, lo, hi, level=0.95):
"""Log-scale Wald reconstruction: SE from the CI width, p from OR/SE."""
z = norm.ppf(1 - (1 - level) / 2)
se = (np.log(hi) - np.log(lo)) / (2 * z)
gm = np.sqrt(lo * hi) # point estimate the CI implies
p = 2 * norm.sf(abs(np.log(or_)) / se) # two-sided, from stated OR
p_gm = 2 * norm.sf(abs(np.log(gm)) / se) # two-sided, from CI midpoint
p_one = norm.sf(abs(np.log(or_)) / se) # one-sided
return gm, se, p, p_gm, p_one
def p_range(or_, lo, hi, d_or=0.005, d_ci=0.005):
"""Smallest and largest two-sided p compatible with 2-decimal rounding."""
ps = []
for o in (or_ - d_or, or_ + d_or):
for l in (lo - d_ci, lo + d_ci):
for h in (hi - d_ci, hi + d_ci):
ps.append(wald(o, l, h)[2])
return min(ps), max(ps)
# label, OR, lower, upper, what the paper reports about significance
ROWS = [
("Abstract: healthcare, 'gay or queer'", 2.51, 0.86, 7.36, "p=0.002"),
("Results text: healthcare, queer/asex./pans.", 2.51, 0.86, 7.36, "'significantly associated'"),
("Table 3: healthcare, queer/asex./pans./quest.", 2.52, 0.86, 7.36, "** (p<=0.01)"),
("Table 3: healthcare, bisexual", 2.74, 0.96, 7.81, "** (p<=0.01)"),
("Results text: healthcare, bisexual", 2.74, 0.96, 7.81, "'significantly associated'"),
("Table 3: healthcare, gay", 3.45, 1.23, 9.72, "** (p<=0.01)"),
("Results text: healthcare, gay", 3.45, 1.22, 9.71, "'significantly associated'"),
("Table 3: healthcare, lesbian", 2.52, 0.50, 12.87, "(no marker)"),
("Abstract/Table 3: government, gay", 1.89, 1.04, 3.45, "p=0.045 / *"),
("Table 3: government, queer/asex./pans./quest.", 1.30, 1.21, 2.47, "*"),
("Results text: government, queer/asex./pans.", 1.30, 0.68, 2.47, "'significant'"),
("Table 3: police, queer/asex./pans./quest.", 0.62, 0.32, 1.19, "(no marker)"),
("Abstract/Table 3: healthcare, disability", 1.35, 1.07, 1.71, "p=0.009 / **"),
]
print("Log-scale Wald reconstruction from each reported OR and 95% CI")
print(f"{'quantity':47s} {'OR':>5} {'95% CI':>12} {'CI mid':>6} {'SE':>6} "
f"{'p(2-s)':>7} {'p range (rounding)':>18} {'p(1-s)':>7} reported")
for lab, o, l, h, rep in ROWS:
gm, se, p, p_gm, p1 = wald(o, l, h)
lo_p, hi_p = p_range(o, l, h)
print(f"{lab:47s} {o:5.2f} {l:5.2f}-{h:<6.2f} {gm:6.2f} {se:6.3f} "
f"{p:7.3f} {lo_p:8.3f}-{hi_p:<9.3f} {p1:7.3f} {rep}")
# What would p=0.002 require for OR=2.51?
z_req = norm.isf(0.002 / 2)
se_req = np.log(2.51) / z_req
lo_req, hi_req = 2.51 * np.exp(-Z95 * se_req), 2.51 * np.exp(Z95 * se_req)
print(f"\nFor OR=2.51 with two-sided p=0.002: z={z_req:.2f}, SE={se_req:.3f}, "
f"required 95% CI {lo_req:.2f} to {hi_req:.2f}")
_, se_rep, _, _, _ = wald(2.51, 0.86, 7.36)
p_rep = wald(2.51, 0.86, 7.36)[2]
print(f"Reported CI implies SE={se_rep:.3f}, z={np.log(2.51)/se_rep:.2f}; "
f"ratio of SEs = {se_rep/se_req:.2f}")
print(f"Recomputed p / reported p = {p_rep/0.002:.0f}; "
f"one-sided p / 0.002 = {wald(2.51, 0.86, 7.36)[4]/0.002:.0f}")
for lab, l, h in (("queer healthcare", 0.86, 7.36), ("bisexual healthcare", 0.96, 7.81),
("queer government (Table 3)", 1.21, 2.47)):
print(f"CI geometric midpoint, {lab}: {np.sqrt(l*h):.3f}")
# Alternative readings of the interval level
print("\nIf the interval 0.86-7.36 were read at another confidence level:")
for lvl in (0.90, 0.95, 0.99):
print(f" {int(lvl*100)}% CI -> two-sided p = {wald(2.51, 0.86, 7.36, lvl)[2]:.3f}")
# Reference group size (Table 1) and the identity check
n, gay, les, bi, q, het = 7758, 6125, 22, 1158, 379, 74
print(f"\nTable 1: heterosexual reference n = {het}; "
f"7758 - (gay+lesbian+bisexual+queer) = {n - gay - les - bi - q}")
# ---------------------------------------------------------------- figure
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans",
"text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK})
# Healthcare-provider model, sexual-orientation rows (Table 3), plus the
# abstract's sentence; reported marker vs recomputed p.
fig_rows = [
("Abstract (healthcare,\n'gay or queer')", 2.51, 0.86, 7.36, "p = 0.002"),
("Table 3: queer/asexual/\npansexual/questioning", 2.52, 0.86, 7.36, "** (p ≤ 0.01)"),
("Table 3: bisexual", 2.74, 0.96, 7.81, "** (p ≤ 0.01)"),
("Table 3: gay", 3.45, 1.23, 9.72, "** (p ≤ 0.01)"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.3), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.64, top=0.89, bottom=0.13)
ax.set_facecolor(BG)
ax.set_xscale("log")
ax.set_xlim(0.4, 16)
ticks = [0.5, 1, 2, 4, 8, 16]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvline(1, color=INK2, lw=1)
ax.text(1.06, -0.75, "OR = 1 (no association)", color=INK2, fontsize=10,
ha="left", va="center")
ys = np.array([4.9, 3.1, 1.6, 0.1]) # extra room under the abstract row
tr = ax.get_yaxis_transform() # x in axes fraction, y in data units
for y, (lab, o, l, h, rep) in zip(ys, fig_rows):
p = wald(o, l, h)[2]
ax.plot([l, h], [y, y], color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(o, y, "o", ms=9, color=ORANGE, mec=BG, mew=1)
ax.text(1.03, y + 0.30, f"{o:.2f} ({l:.2f}–{h:.2f})", transform=tr,
color=INK2, fontsize=10, va="center", ha="left")
ax.text(1.03, y, f"reported: {rep}", transform=tr, color=ORANGE,
fontsize=10, va="center", ha="left")
ax.text(1.03, y - 0.30, f"recomputed: p = {p:.3f}", transform=tr,
color=BLUE, fontsize=10, va="center", ha="left", fontweight="bold")
# interval that p = 0.002 would require (abstract row)
y0 = ys[0] - 0.75
ax.plot([lo_req, hi_req], [y0, y0], color=BLUE, lw=2, ls=(0, (4, 2)))
ax.plot(2.51, y0, "D", ms=8, color=BLUE, mec=BG, mew=1)
ax.text(1.03, y0, f"CI that p = 0.002 needs:\n{lo_req:.2f}–{hi_req:.2f}",
transform=tr, color=BLUE, fontsize=10, va="center", ha="left")
ax.set_yticks(list(ys) + [y0])
ax.set_yticklabels([r[0] for r in fig_rows] + ["abstract OR, if\np = 0.002 were right"])
for t in ax.get_yticklabels()[-1:]:
t.set_color(BLUE)
ax.set_ylim(-1.05, ys[0] + 0.6)
ax.set_xlabel("Adjusted OR, discrimination from healthcare\nproviders (log scale), with 95% CI")
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
handles = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8,
label="Reported OR and 95% CI (paper)"),
Line2D([], [], color=BLUE, lw=2, ls=(0, (4, 2)), marker="D", ms=7,
label="Recomputed from the paper's numbers")]
fig.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, 0.99),
ncol=2, frameon=False, fontsize=10)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg and fig1.png")
Abstract reports whole-cohort estimates as sexual-minority results, making two non-significant subgroup findings appear significant
BMJ Glob Health (2022) · doi:10.1136/bmjgh-2022-010231 · PMC9791420
Summary
The paper's second aim is to compare the mental health of sexual minority adolescents with and without HIV, and the abstract reports that "Sexual minority adolescents living with HIV had significantly higher odds of high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013)", along with three significant estimates for sexual minority adolescents with unknown HIV status. All four of these estimates, with both CI limits and the p-value, match the HIV-status rows of Table 1 exactly. Table 1 is the whole-cohort model (N=1247, heterosexual and sexual minority adolescents together), and the main text itself labels 2.42 as the result "In the overall cohort". The sexual-minority-only model in Table 2 (N=497) gives different values, and two of the four claims are not significant there: HIV-positive status and high anxiety, AOR 2.09 (0.93 to 4.69), p=0.073; unknown status and suicidal attempt/ideation, AOR 1.47 (0.98 to 2.22), p=0.064. The paper's headline comparison of sexual minority and heterosexual adolescents is unaffected, but the abstract's Results sentence on the second aim misstates which findings reached significance and contradicts the abstract's own Conclusion.
What the paper reports
Abstract, Background (the second aim):
and to determine associations between the mental health status of sexual minority adolescents living with and without HIV.
Abstract, Methods:
A second multivariate regression model was developed to establish associations between HIV status among sexual minority individuals and the dependent variables.
Abstract, Results:
Sexual minority adolescents living with HIV had significantly higher odds of high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013), while those with unknown HIV status had significantly higher odds of depressive symptoms (AOR: 3.82, 95% CI: 2.78 to 5.27; p<0.001), high general anxiety (AOR: 3.09; 95% CI: 2.29 to 4.17; p<0.001) and suicidal attempt/ideation (AOR: 1.65; 95% CI: 1.22 to 2.24; p=0.001).
Abstract, Conclusion:
Although there was no significant difference in the mental health status of sexual minority adolescents living and not living with HIV, sexual minority adolescents with unknown HIV status reported worse mental health than their HIV-negative peers.
Methods, Data analysis:
The second set of multivariate regression analyses included data from only sexual minority individuals.
Significance was set at 5%.
Results, third paragraph (describing Table 1):
In the overall cohort, self-reported HIV-positive status among adolescents was associated with increased odds of high general anxiety (AOR: 2.42; 95% CI:1.21 to 4.84; p=0.013), but not depression or suicidal attempt/ideation.
Results, fourth paragraph (describing Table 2):
Having an unknown HIV status was associated with significantly higher odds of experiencing depressive symptoms (AOR: 3.11, 95% CI: 1.76 to 5.49, p<0.001) and high general anxiety (AOR: 2.32, 95% CI: 1.55 to 3.48, p<0.001).
Table 1, "Multivariate logistic regression analysis to determine association between sexual orientation, HIV status, mental health status of adolescents 13–19 years old in Nigeria (N=1247)", HIV-positive row (columns: total | depressive symptoms yes | no | AOR (95% CI) p | high general anxiety yes | no | AOR (95% CI) p):
Positive | 41(3.3) | 29(70.7) | 12(29.3) | 1.81(0.85 to 3.88)0.125 | 17(41.5) | 24(58.5) | 2.42(1.21 to 4.84)0.013
Table 2, "Multivariate logistic regression analysis to determine association between HIV status and mental health status among sexual minority adolescents 13–19 years old in Nigeria (N=497)", HIV-positive row (columns as above, then suicidal ideation yes | no | AOR (95% CI) p):
HIV positive | 29 (5.8) | 24(82.8) | 5(17.2) | 2.25(0.78 to 6.48)0.132 | 14(48.3) | 15(51.7) | 2.09(0.93 to 4.69)0.073 | 12(41.4) | 17(58.6) | 1.47(0.66 to 3.28)0.352
Table 2, unknown-status row:
Unknown HIV status | 222 (44.7) | 201(90.5) | 21(9.5) | 3.11(1.76 to 5.49)<0.001 | 130(58.6) | 92(41.4) | 2.32(1.55 to 3.48)<0.001 | 109(49.1) | 113(50.9) | 1.47(0.98 to 2.22)0.064
Results, fifth paragraph (exploratory analyses, describing supplementary Table S1, heterosexual adolescents only):
Exploratory analyses among heterosexual participants showed that, like sexual minority individuals, unknown HIV status was associated with an increased likelihood of reporting depressive and anxiety symptoms and self-harm/suicidal ideation in this study. The adjusted ORs were comparable in these latter multivariate models among heterosexual and sexual minority participants, though these were relatively larger in sexual minority individuals (see online supplemental table 1).
Supplementary Table S1 (N=750), unknown-status estimates for depressive symptoms, high general anxiety and suicidal ideation:
3.92 (2.66-5.78) <0.001 | 4.16 (2.63-6.56) <0.001 | 2.06 (1.28-3.30) 0.003
Discussion, first paragraph:
Among sexual minority adolescents, those with unknown HIV status had significantly higher odds of reporting depressive symptoms and generalised anxiety disorder. These findings partially support the hypotheses.
What the paper's own numbers show
Method. I matched each of the four estimates in the abstract's Results sentence on HIV status against the HIV-status rows of Table 1 (whole cohort) and Table 2 (sexual minorities only), comparing the AOR, both CI limits and the p-value. For every estimate I recomputed the Wald z and two-sided p implied by the printed 95% CI (SE = [ln(upper) − ln(lower)] / 3.92; z = ln(AOR)/SE), to confirm that each table is internally consistent and that the printed p-values are two-sided. As a model-free check on the key subgroup estimate, I computed crude odds ratios from the Table 2 counts. Finally, I checked that Table 2 and supplementary Table S1 (heterosexual adolescents only, N=750) partition the Table 1 cohort, which confirms which table covers which population. I also compared the unknown-status estimates in Table S1 with those in Table 2, as the Results' exploratory-analysis paragraph does. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| HIV-positive vs negative, high general anxiety, given for sexual minority adolescents | Abstract, Results | 2.42 (1.21 to 4.84), p=0.013 | Identical to Table 1 (whole cohort); Wald p from CI 0.0125 |
| Same estimate, whole cohort | Table 1; Results ¶3 ("In the overall cohort") | 2.42 (1.21 to 4.84), p=0.013 | SE 0.354, z 2.50, p 0.0125; CI excludes 1 |
| Same estimate, sexual minorities only | Table 2 | 2.09 (0.93 to 4.69), p=0.073 | SE 0.413, z 1.79, p 0.074; CI includes 1, not significant |
| Crude OR, same comparison, sexual minorities | Table 2 counts, 14/29 vs 79/246 | — | 1.97 (0.91 to 4.29), p=0.086 |
| Unknown vs negative, depressive symptoms, given for sexual minority adolescents | Abstract, Results | 3.82 (2.78 to 5.27), p<0.001 | Identical to Table 1; Table 2 value is 3.11 (1.76 to 5.49), p<0.001 |
| Unknown vs negative, high general anxiety, given for sexual minority adolescents | Abstract, Results | 3.09 (2.29 to 4.17), p<0.001 | Identical to Table 1; Table 2 value is 2.32 (1.55 to 3.48), p<0.001 |
| Unknown vs negative, suicidal attempt/ideation, given for sexual minority adolescents | Abstract, Results | 1.65 (1.22 to 2.24), p=0.001 | Identical to Table 1 (Wald p 0.0012); Table 2 value is 1.47 (0.98 to 2.22), p=0.064 (Wald p 0.065); not significant |
| Abstract estimates that match Table 1 / Table 2 exactly | Abstract vs Tables 1 and 2 | — | 4 of 4 / 0 of 4 |
| Abstract estimates with CI excluding 1 | Abstract vs Table 2 | 4 of 4 | 2 of 4 for the same estimates in Table 2 |
| Ratio of abstract AOR to Table 2 AOR | Abstract vs Table 2 | — | 1.16, 1.23, 1.33, 1.12 (the abstract's values are larger than the subgroup estimates for all four) |
| HIV-positive vs negative, depressive symptoms / suicidal attempt/ideation, sexual minorities | Table 2 | 2.25 (0.78 to 6.48), p=0.132 / 1.47 (0.66 to 3.28), p=0.352 | Wald p 0.133 / 0.346; neither significant (abstract does not report these) |
| Unknown vs negative, depressive symptoms / high general anxiety / suicidal attempt/ideation, heterosexuals vs sexual minorities | Table S1 vs Table 2; Results ¶5 ("like sexual minority individuals ... self-harm/suicidal ideation"; AORs "relatively larger in sexual minority individuals") | S1: 3.92 / 4.16 / 2.06 (p=0.003); Table 2: 3.11 / 2.32 / 1.47 (p=0.064) | The Table S1 AOR is larger than the Table 2 AOR for 3 of 3 outcomes; the Table 2 suicidal-ideation CI includes 1 |
| Sample sizes of the HIV-status rows | Tables 1, 2 and S1 | N=1247, 497, 750 | Rows sum to 1247, 497 and 750; Table 2 + Table S1 = Table 1 for each status (246+549=795; 29+12=41; 222+189=411) |
The Wald p-values recomputed from the printed CIs agree with the printed p-values for every estimate in both tables (largest difference 0.009, for a non-significant Table 1 estimate, 0.703 vs 0.712), so both tables are internally consistent, the printed p-values are two-sided, and the discrepancy lies only in the abstract's attribution of Table 1 values to the sexual-minority subgroup.
A second, smaller inconsistency in Table 1. In the depressive-symptom and high-anxiety columns of Table 1, the yes/no counts in the "Negative" and "Unknown" HIV rows do not add up to those rows' totals: "Negative" (n=795) shows 332/79 and 195/216, which sum to 411, and "Unknown" (n=411) shows 330/465 and 126/669, which sum to 795. Adding the Table 2 and Table S1 counts gives 330/465 and 126/669 for HIV-negative adolescents and 332/79 and 195/216 for those with unknown status, so these count cells (and their percentages, e.g. 332/411 = 80.8%) appear to have been placed in each other's rows. The suicidal-ideation column and the HIV-positive row add up correctly, and the column totals (691, 338, 309) match the Results text. This does not change any AOR, but it reverses the descriptive picture: as printed, Table 1 shows 80.8% of HIV-negative and 41.5% of unknown-status adolescents with depressive symptoms, when the subgroup tables imply the reverse.
Figure
Figure 1. Left: the abstract's Results sentence on sexual minority adolescents by HIV status, with two of its four estimates (orange). Top right: the same two rows of Table 1, the whole-cohort model (N=1247), which carry identical numbers; the other two abstract estimates (3.82 and 3.09) also match Table 1 exactly. Bottom right: the corresponding rows of Table 2, the model restricted to sexual minority adolescents (N=497), which is the population the abstract names (blue). Both Table 2 confidence intervals include 1, so neither association is significant at the paper's stated 5% level. All numbers are printed in the paper; recompute.py.txt checks each p-value against its CI.
Figure 2. Each of the four estimates the abstract reports for sexual minority adolescents (orange circles, identical to Table 1, whole cohort) next to the same estimate from the sexual-minority-only model in Table 2 (blue squares), on a log odds-ratio axis with the null value (OR = 1) as a thin vertical reference line. All four Table 2 estimates are smaller, and the intervals for HIV-positive status with high general anxiety and for unknown status with suicidal attempt/ideation cross 1.
Why it matters
The comparison of sexual minority adolescents living with and without HIV is one of the paper's two stated aims and has its own model (Table 2). The abstract's Results sentence on that aim reports four significant associations, but all four come from the whole-cohort model, where heterosexual adolescents make up 60% of the sample. In the subgroup the abstract names, HIV-positive status is not significantly associated with any of the three outcomes, and unknown status is not significantly associated with suicidal attempt/ideation; the two effects that do remain significant are 19% to 25% smaller than the abstract states (3.11 vs 3.82 and 2.32 vs 3.09).
The abstract is the part of the paper most often read, indexed and quoted. A reader who relies on it would conclude that HIV-positive sexual minority adolescents in this sample had significantly higher odds of high anxiety, and that unknown HIV status was associated with suicidal attempt/ideation in sexual minority adolescents. Neither is what the paper's subgroup analysis found. The abstract also contradicts itself: its Conclusion ("no significant difference in the mental health status of sexual minority adolescents living and not living with HIV") matches Table 2, while its Results sentence does not. Results paragraphs 3 and 4, the Discussion's summary of findings (its first paragraph) and the tables report the subgroup findings correctly; the Discussion's later paragraph on HIV status agrees with Table 2 that HIV-positive status had no significant effect, although it describes the comparison as one with "heterosexual peers" rather than with HIV-negative sexual minority peers, which is the Table 2 comparison. One sentence in the exploratory-analysis paragraph (Results, fifth paragraph) repeats the suicidal-ideation claim ("like sexual minority individuals, unknown HIV status was associated with ... self-harm/suicidal ideation"), and the same paragraph says the adjusted ORs were "relatively larger in sexual minority individuals", although the heterosexual unknown-status AORs in Table S1 (3.92, 4.16, 2.06) are larger than the sexual-minority ones in Table 2 (3.11, 2.32, 1.47) for all three outcomes. The headline comparison of sexual minority with heterosexual adolescents (AORs 5.54, 3.56 and 2.95) is not affected.
Explanations considered
- The abstract quotes a different sexual-minority model. This does not hold. All four abstract estimates match Table 1 to every printed digit (AOR, both CI limits and p), and none match Table 2. Table 1 reports sexual-orientation and HIV-status estimates side by side for N=1247; its HIV-status rows sum to 1247. The Results text describes 2.42 as the estimate "In the overall cohort". Table 2 is the only model the paper reports for sexual minorities alone.
- Tables 1 and 2 are mislabelled, and Table 1 is really the subgroup model. This does not hold. Table 1's HIV rows sum to 1247 and Table 2's to 497, and for each HIV status the Table 2 count plus the Table S1 (heterosexual) count equals the Table 1 count. Table 1 also contains the sexual-orientation rows that only a whole-cohort model can estimate.
- The abstract sentence was meant to describe the whole cohort. If so, the sentence names the wrong population: it begins "Sexual minority adolescents living with HIV" and refers to "those with unknown HIV status" in the same breath. The abstract's Methods do list HIV status among the variables of the first, whole-cohort model, but they then introduce the second model specifically "to establish associations between HIV status among sexual minority individuals and the dependent variables", and the Results sentence follows the sexual-orientation results as if reporting that model. Either way, the abstract presents whole-cohort estimates as sexual-minority results.
- A one-sided test. This does not account for the numbers. Halving the Table 2 p-values (0.073 and 0.064) would give values below 0.05, but the paper prints two-sided p-values throughout (the recomputed two-sided Wald p-values match the printed ones), its stated criterion is "Significance was set at 5%", and its Conclusion and Discussion treat these Table 2 results as non-significant. In any case, the abstract's numbers are not Table 2's numbers at all.
- Rounding or a conversion artefact. This does not hold. The differences (2.42 vs 2.09; 1.65 vs 1.47; lower CI limits 1.21 vs 0.93 and 1.22 vs 0.98) are far beyond rounding. The PMC text and the publisher-deposited JATS XML in Europe PMC carry identical values, with each AOR, CI and p in its own table cell.
- The supplement contains the abstract's figures. This does not hold. The only supplementary file (Table S1) is a model for heterosexual adolescents alone (N=750), in which HIV-positive status and high anxiety give AOR 3.12 (0.80 to 12.28), p=0.103. It contains no sexual-minority or whole-cohort model.
Status
- Version of record. The publisher's article page and PDF (gh.bmj.com/content/7/12/e010231) returned HTTP 403 to automated requests on 2026-10-23. The text was compared in the PMC article (PMC9791420) and in the publisher-deposited JATS XML via Europe PMC; both carry the same numbers at every quoted place.
- Supplement. The single supplementary file, bmjgh-2022-010231supp001.pdf (Table S1, heterosexual adolescents only, N=750), was obtained through the Europe PMC supplementary-files service and read in full. It does not resolve the discrepancy; its counts were used above to check Table 1.
- Corrections and comments. As of 2026-10-23, Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction, erratum, expression of concern or retraction (PMID 36564086). The publisher page could not be checked for notices or rapid responses (HTTP 403).
- PubPeer. The PubPeer API and search page required human verification and could not be queried directly on 2026-10-23. Web searches on 2026-10-23 for the title and DOI together with PubPeer and correction found no thread or notice.
- Sources. PMC article PMC9791420; Europe PMC full-text XML and supplementary files for PMC9791420; Crossref record for 10.1136/bmjgh-2022-010231; PubMed 36564086.
Suggested correction
Replace the abstract's Results sentence on HIV status with the estimates from the sexual-minority model (Table 2), for example:
Among sexual minority adolescents, HIV-positive status was not significantly associated with depressive symptoms, high general anxiety (AOR: 2.09; 95% CI: 0.93 to 4.69; p=0.073) or suicidal attempt/ideation, while those with unknown HIV status had significantly higher odds of depressive symptoms (AOR: 3.11; 95% CI: 1.76 to 5.49; p<0.001) and high general anxiety (AOR: 2.32; 95% CI: 1.55 to 3.48; p<0.001), but not of suicidal attempt/ideation (AOR: 1.47; 95% CI: 0.98 to 2.22; p=0.064).
If the authors prefer to keep the whole-cohort estimates in the abstract, the sentence should say that they come from the whole cohort (sexual minority and heterosexual adolescents together, the model reported in Table 1), and the Table 2 results should be added for the sexual-minority comparison that the second aim and the Conclusion refer to.
In the exploratory-analysis paragraph of the Results, the phrase "like sexual minority individuals" should not extend to self-harm/suicidal ideation, which was not significantly associated with unknown status among sexual minority adolescents (Table 2: AOR 1.47; 95% CI: 0.98 to 2.22; p=0.064), and the statement that the adjusted ORs were "relatively larger in sexual minority individuals" should be checked against Table S1, whose unknown-status AORs are the larger ones.
In Table 1, the depressive-symptom and high-general-anxiety counts and percentages in the "Negative" and "Unknown" HIV-status rows should be checked against the data; the subgroup tables imply that they belong in each other's rows. Two CIs in the education rows also cannot contain their point estimates and look like typographical errors: Table 1, primary education, suicidal ideation, "0.28(1.13 to 0.64)", and Table 2, secondary education, depressive symptoms, "0.58(1.19 to 1.76)".
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC9791420 write-up and regenerate the figures.
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Figures are written next to this file:
fig1.svg/png, fig2.svg/png figures used in writeup.md (provisional: fig1 = cand1, fig2 = cand2
until a design is chosen from the candidates)
candidates/cand1..cand5_*.svg/png five candidate figure designs, kept out of the deliverable files
All inputs are typed in from the article as archived in PubMed Central (PMC9791420):
Abstract, Results text, Table 1 (whole cohort, N=1247), Table 2 (sexual-minority
adolescents only, N=497), and the online supplement bmjgh-2022-010231supp001.pdf,
Table S1 (heterosexual adolescents only, N=750).
"""
import os
import shutil
import textwrap
import numpy as np
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
Z975 = norm.ppf(0.975)
# ---------------------------------------------------------------------------
# 1. Inputs, as printed in the paper
# ---------------------------------------------------------------------------
# (AOR, lower, upper, printed p)
ABSTRACT_SM = { # Abstract, Results: attributed to "Sexual minority adolescents"
"HIV-positive: high anxiety": (2.42, 1.21, 4.84, "0.013"),
"Unknown status: depression": (3.82, 2.78, 5.27, "<0.001"),
"Unknown status: high anxiety": (3.09, 2.29, 4.17, "<0.001"),
"Unknown status: suicidal att./id.": (1.65, 1.22, 2.24, "0.001"),
}
TABLE1 = { # Table 1, whole cohort (N=1247), HIV status rows vs HIV-negative
"HIV-positive: depression": (1.81, 0.85, 3.88, "0.125"),
"HIV-positive: high anxiety": (2.42, 1.21, 4.84, "0.013"),
"HIV-positive: suicidal att./id.": (1.15, 0.56, 2.36, "0.712"),
"Unknown status: depression": (3.82, 2.78, 5.27, "<0.001"),
"Unknown status: high anxiety": (3.09, 2.29, 4.17, "<0.001"),
"Unknown status: suicidal att./id.": (1.65, 1.22, 2.24, "0.001"),
}
TABLE2 = { # Table 2, sexual-minority adolescents only (N=497)
"HIV-positive: depression": (2.25, 0.78, 6.48, "0.132"),
"HIV-positive: high anxiety": (2.09, 0.93, 4.69, "0.073"),
"HIV-positive: suicidal att./id.": (1.47, 0.66, 3.28, "0.352"),
"Unknown status: depression": (3.11, 1.76, 5.49, "<0.001"),
"Unknown status: high anxiety": (2.32, 1.55, 3.48, "<0.001"),
"Unknown status: suicidal att./id.": (1.47, 0.98, 2.22, "0.064"),
}
TABLES1_HIVPOS_ANX = (3.12, 0.80, 12.28, "0.103") # Table S1, heterosexuals only (N=750)
TABLES1_UNKNOWN = { # Table S1, heterosexuals only (N=750), unknown status vs HIV-negative
"Unknown status: depression": (3.92, 2.66, 5.78, "<0.001"),
"Unknown status: high anxiety": (4.16, 2.63, 6.56, "<0.001"),
"Unknown status: suicidal att./id.": (2.06, 1.28, 3.30, "0.003"),
}
# Outcome counts (yes, no) by HIV status
OUTCOMES = ["depression", "high anxiety", "suicidal att./id."]
T2_COUNTS = { # Table 2, sexual minorities (N=497)
"negative": {"n": 246, "depression": (174, 72), "high anxiety": (79, 167), "suicidal att./id.": (77, 169)},
"positive": {"n": 29, "depression": (24, 5), "high anxiety": (14, 15), "suicidal att./id.": (12, 17)},
"unknown": {"n": 222, "depression": (201, 21), "high anxiety": (130, 92), "suicidal att./id.": (109, 113)},
}
S1_COUNTS = { # Table S1, heterosexuals (N=750)
"negative": {"n": 549, "depression": (156, 393), "high anxiety": (47, 502), "suicidal att./id.": (72, 477)},
"positive": {"n": 12, "depression": (5, 7), "high anxiety": (3, 9), "suicidal att./id.": (0, 12)},
"unknown": {"n": 189, "depression": (131, 58), "high anxiety": (65, 124), "suicidal att./id.": (39, 150)},
}
T1_COUNTS = { # Table 1, whole cohort (N=1247), rows exactly as printed
"negative": {"n": 795, "depression": (332, 79), "high anxiety": (195, 216), "suicidal att./id.": (149, 646)},
"positive": {"n": 41, "depression": (29, 12), "high anxiety": (17, 24), "suicidal att./id.": (12, 29)},
"unknown": {"n": 411, "depression": (330, 465), "high anxiety": (126, 669), "suicidal att./id.": (148, 263)},
}
T1_TOTAL_YES = {"depression": 691, "high anxiety": 338, "suicidal att./id.": 309} # Results, paragraph 1
# ---------------------------------------------------------------------------
# 2. Calculations
# ---------------------------------------------------------------------------
def wald_from_ci(aor, lo, hi):
"""SE of log(AOR), Wald z and two-sided p implied by a symmetric 95% CI on the log scale."""
se = (np.log(hi) - np.log(lo)) / (2 * Z975)
z = np.log(aor) / se
return se, z, 2 * norm.sf(abs(z)), float(np.exp((np.log(lo) + np.log(hi)) / 2))
def crude_or(a, b, c, d):
"""Crude OR (exposed yes/no = a/b, reference yes/no = c/d), Woolf 95% CI and Wald p."""
orr = (a * d) / (b * c)
se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
return orr, np.exp(np.log(orr) - Z975 * se), np.exp(np.log(orr) + Z975 * se), 2 * norm.sf(abs(np.log(orr) / se))
def fmt_p(p):
return "<0.0001" if p < 1e-4 else f"{p:.4f}"
WALD = {}
print("=" * 100)
print("A. Wald check of every estimate in the HIV-status rows: p implied by the printed 95% CI")
print("=" * 100)
print(f"{'source':10s} {'estimate':36s} {'AOR (95% CI)':>20s} {'CI mid':>7s} {'SE':>6s} {'z':>6s} "
f"{'p from CI':>10s} {'printed p':>9s} {'CI excl. 1':>10s}")
for src, table in [("Abstract", ABSTRACT_SM), ("Table 1", TABLE1), ("Table 2", TABLE2),
("Table S1", {"HIV-positive: high anxiety": TABLES1_HIVPOS_ANX, **TABLES1_UNKNOWN})]:
for k, (a, lo, hi, rep) in table.items():
se, z, p, mid = wald_from_ci(a, lo, hi)
WALD[(src, k)] = (se, z, p)
print(f"{src:10s} {k:36s} {a:5.2f} ({lo:4.2f} to {hi:5.2f}) {mid:7.2f} {se:6.3f} {z:6.2f} "
f"{fmt_p(p):>10s} {rep:>9s} {str(lo > 1 or hi < 1):>10s}")
print()
print("=" * 100)
print("B. Which model do the abstract's four 'sexual minority' figures come from?")
print("=" * 100)
for k, v in ABSTRACT_SM.items():
in_t1 = TABLE1[k] == v
in_t2 = TABLE2[k] == v
t2 = TABLE2[k]
print(f"{k:36s} abstract {v[0]} ({v[1]} to {v[2]}) p={v[3]:7s} | identical to Table 1: {in_t1} | "
f"identical to Table 2: {in_t2} | Table 2 value {t2[0]} ({t2[1]} to {t2[2]}) p={t2[3]}")
n_match_t1 = sum(TABLE1[k] == v for k, v in ABSTRACT_SM.items())
print(f"-> {n_match_t1} of {len(ABSTRACT_SM)} abstract estimates match Table 1 exactly (AOR, both CI limits, p); "
f"{sum(TABLE2[k] == v for k, v in ABSTRACT_SM.items())} match Table 2.")
for k in ["Unknown status: depression", "Unknown status: high anxiety"]:
print(f" Table 2 AOR is {100*(1-TABLE2[k][0]/ABSTRACT_SM[k][0]):.1f}% smaller than the abstract's, {k}")
print(f" heterosexual share of the whole cohort: 750/1247 = {100*750/1247:.1f}%")
diffs = [abs(WALD[(src, k)][2] - float(v[3])) for src, t in [("Table 1", TABLE1), ("Table 2", TABLE2)]
for k, v in t.items() if not v[3].startswith("<")]
print(f" largest |Wald p - printed p| over Tables 1 and 2: {max(diffs):.4f}")
sig_abs = sum(v[1] > 1 for v in ABSTRACT_SM.values())
sig_t2 = sum(TABLE2[k][1] > 1 for k in ABSTRACT_SM)
print(f"-> CI excludes 1: {sig_abs} of 4 in the abstract; {sig_t2} of 4 for the same estimates in Table 2.")
for k in ABSTRACT_SM:
print(f" ratio abstract/Table 2 AOR, {k:36s}: {ABSTRACT_SM[k][0] / TABLE2[k][0]:.2f}")
print()
print("=" * 100)
print("B2. Results paragraph 5 (exploratory analyses): unknown status, heterosexuals (Table S1) vs sexual minorities (Table 2)")
print("=" * 100)
for k, v in TABLES1_UNKNOWN.items():
t2 = TABLE2[k]
print(f"{k:36s} Table S1 {v[0]} ({v[1]} to {v[2]}) p={v[3]:7s} | Table 2 {t2[0]} ({t2[1]} to {t2[2]}) p={t2[3]:7s} "
f"| Table 2 CI excludes 1: {t2[1] > 1} | larger AOR: {'Table 2 (sexual minorities)' if t2[0] > v[0] else 'Table S1 (heterosexuals)'}")
n_s1_larger = sum(TABLES1_UNKNOWN[k][0] > TABLE2[k][0] for k in TABLES1_UNKNOWN)
print(f"-> Table S1 AOR larger than Table 2 AOR for {n_s1_larger} of 3 unknown-status outcomes.")
print()
print("=" * 100)
print("C. Crude ORs from the Table 2 counts (sexual minorities), vs HIV-negative")
print("=" * 100)
CRUDE_T2 = {}
for grp in ["positive", "unknown"]:
for o in OUTCOMES:
a, b = T2_COUNTS[grp][o]
c, d = T2_COUNTS["negative"][o]
orr, lo, hi, p = crude_or(a, b, c, d)
key = ("HIV-positive: " if grp == "positive" else "Unknown status: ") + o
CRUDE_T2[key] = (orr, lo, hi, p)
print(f"{key:36s} {a}/{a+b} vs {c}/{c+d}: crude OR {orr:.2f} (95% CI {lo:.2f} to {hi:.2f}), p={'<0.001' if p < 0.001 else format(p, '.3f')} "
f"| Table 2 AOR {TABLE2[key][0]}")
print()
print("=" * 100)
print("D. Arithmetic of the HIV-status rows")
print("=" * 100)
for name, cnt, N in [("Table 2", T2_COUNTS, 497), ("Table S1", S1_COUNTS, 750), ("Table 1", T1_COUNTS, 1247)]:
print(f"{name}: HIV rows sum to {sum(v['n'] for v in cnt.values())} (stated N={N})")
for g in ["negative", "positive", "unknown"]:
s = T2_COUNTS[g]["n"] + S1_COUNTS[g]["n"]
print(f" {g:8s}: Table 2 n + Table S1 n = {T2_COUNTS[g]['n']} + {S1_COUNTS[g]['n']} = {s}; "
f"Table 1 n = {T1_COUNTS[g]['n']} -> equal: {s == T1_COUNTS[g]['n']}")
print("Row check, Table 1 (yes + no should equal the row total):")
for g in ["negative", "positive", "unknown"]:
for o in OUTCOMES:
y, n = T1_COUNTS[g][o]
exp = (T2_COUNTS[g][o][0] + S1_COUNTS[g][o][0], T2_COUNTS[g][o][1] + S1_COUNTS[g][o][1])
print(f" {g:8s} {o:18s}: printed {y}/{n} (sum {y+n}, row total {T1_COUNTS[g]['n']}, "
f"{'ok' if y + n == T1_COUNTS[g]['n'] else 'MISMATCH'}); Table 2 + Table S1 give {exp[0]}/{exp[1]}")
for o in OUTCOMES:
s = sum(T1_COUNTS[g][o][0] for g in T1_COUNTS)
print(f" Table 1 'yes' column sum, {o:18s}: {s} (Results text: {T1_TOTAL_YES[o]})")
print(" Table 1 printed % for 'Negative' depression: 332/411 = "
f"{100*332/411:.1f}% (printed 80.8); 332/795 = {100*332/795:.1f}%")
# ---------------------------------------------------------------------------
# 3. Figures
# ---------------------------------------------------------------------------
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
from matplotlib.transforms import Bbox
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, PANEL = "#eb6834", "#2a78d6", "#e4e3df", "#f3f2ef"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path", # text as outlines: no dependence on installed fonts
"axes.linewidth": 0.8,
})
LW = 2.0 # 2 px lines
MS = 9 # marker size (pt), >= 8 px
W = 7.2 # figure width (in)
SHORT = { # row labels for the four abstract claims
"HIV-positive: high anxiety": "HIV-positive vs negative\nhigh general anxiety",
"Unknown status: depression": "Unknown vs negative\ndepressive symptoms",
"Unknown status: high anxiety": "Unknown vs negative\nhigh general anxiety",
"Unknown status: suicidal att./id.": "Unknown vs negative\nsuicidal attempt/ideation",
}
KEYS = list(ABSTRACT_SM)
def save(fig, stem):
"""Save at exactly W inches wide: crop height to the content, never widen past the figure."""
os.makedirs(CAND, exist_ok=True)
fig.canvas.draw()
bb = fig.get_tightbbox(fig.canvas.get_renderer())
if bb.x0 < -0.01 or bb.x1 > W + 0.01:
print(f"WARNING {stem}: content spans x={bb.x0:.2f}..{bb.x1:.2f} in, beyond the {W} in figure")
box = Bbox.from_extents(0, bb.y0 - 0.12, W, bb.y1 + 0.12)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(CAND, f"{stem}.{ext}"), bbox_inches=box, **kw)
plt.close(fig)
print(f"wrote {stem}.svg / {stem}.png")
def ci_str(v):
return f"{v[0]:.2f} ({v[1]:.2f} to {v[2]:.2f}), p={v[3]}"
def p_print(v): # printed p in "p=0.013" / "p<0.001" form
return f"p{v[3]}" if v[3].startswith("<") else f"p={v[3]}"
# --- Candidate 1: text-panel diagram (abstract vs Table 1 vs Table 2) --------------------------
def cand1():
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 74); ax.axis("off")
def panel(x, y, w, h, title, sub):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc=PANEL, ec="#c9c7c1", lw=1))
ax.text(x + 1.8, y + h - 2.6, title, fontsize=11.5, weight="bold", va="top")
ax.text(x + 1.8, y + h - 6.6, sub, fontsize=10, color=INK2, va="top")
# Abstract panel
panel(0.5, 6, 41, 60, "Abstract, Results", "sentence on sexual minority adolescents")
quote = ("“Sexual minority adolescents living with HIV had significantly higher odds of "
"high general anxiety (AOR: 2.42; 95% CI: 1.21 to 4.84; p=0.013), while those with unknown "
"HIV status had significantly higher odds of … suicidal attempt/ideation "
"(AOR: 1.65; 95% CI: 1.22 to 2.24; p=0.001).”")
ax.text(2.8, 52, textwrap.fill(quote, 41), fontsize=10, va="top", style="italic", linespacing=1.35)
a1, a4 = ABSTRACT_SM[KEYS[0]], ABSTRACT_SM[KEYS[3]]
ax.text(2.8, 21, "HIV-positive, high anxiety", fontsize=10, color=INK2)
ax.text(2.8, 17.2, ci_str(a1), fontsize=10.5, color=ORANGE, weight="bold")
ax.text(2.8, 12, "Unknown status, suicidal att./ideation", fontsize=10, color=INK2)
ax.text(2.8, 8.2, ci_str(a4), fontsize=10.5, color=ORANGE, weight="bold")
# Table 1 panel
t1a, t1d = TABLE1[KEYS[0]], TABLE1[KEYS[3]]
panel(58.5, 38, 41, 28, "Table 1: whole cohort", "N=1247, sexual minority and heterosexual")
ax.text(60.3, 52, "HIV-positive, high anxiety", fontsize=10, color=INK2)
ax.text(60.3, 48.2, ci_str(t1a), fontsize=10.5, color=INK, weight="bold")
ax.text(60.3, 44, "Unknown status, suicidal att./ideation", fontsize=10, color=INK2)
ax.text(60.3, 40.2, ci_str(t1d), fontsize=10.5, color=INK, weight="bold")
# Table 2 panel
t2a, t2d = TABLE2[KEYS[0]], TABLE2[KEYS[3]]
panel(58.5, 6, 41, 28, "Table 2: sexual minorities only", "N=497, the group the abstract names")
ax.text(60.3, 20, "HIV-positive, high anxiety", fontsize=10, color=INK2)
ax.text(60.3, 16.2, ci_str(t2a), fontsize=10.5, color=BLUE, weight="bold")
ax.text(60.3, 12, "Unknown status, suicidal att./ideation", fontsize=10, color=INK2)
ax.text(60.3, 8.2, ci_str(t2d), fontsize=10.5, color=BLUE, weight="bold")
# connectors
ax.add_patch(FancyArrowPatch((42, 44), (58.2, 50), arrowstyle="-|>", mutation_scale=14,
lw=LW, color=INK2, connectionstyle="arc3,rad=-0.15"))
ax.text(50.2, 53.5, "identical\nnumbers\n(all 4\nestimates)", ha="center", fontsize=10, color=INK, linespacing=1.15)
ax.add_patch(FancyArrowPatch((42, 28), (58.2, 22), arrowstyle="-|>", mutation_scale=14,
lw=LW, color=INK2, ls=(0, (4, 3)), connectionstyle="arc3,rad=0.15"))
ax.text(50.2, 10, "population\nnamed in\nthe abstract", ha="center", fontsize=10, color=INK, linespacing=1.15)
ax.text(99.5, 2.2, "Table 2: both CIs include 1, not significant at the paper's 5% level",
ha="right", fontsize=10, color=INK2)
# legend
h = [Line2D([], [], marker="s", ls="", ms=MS, color=ORANGE, label="value printed in the abstract"),
Line2D([], [], marker="s", ls="", ms=MS, color=BLUE, label="paper's own sexual-minority model (Table 2)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
save(fig, "cand1_text_panels")
# --- Candidate 2: dot-and-interval plot, log OR axis -------------------------------------------
def cand2():
fig, ax = plt.subplots(figsize=(W, 4.3))
fig.subplots_adjust(left=0.235, right=0.725, bottom=0.14, top=0.80)
tr = ax.get_yaxis_transform() # x in axes fraction, y in data
off = 0.17
for i, k in enumerate(KEYS):
y = len(KEYS) - 1 - i
for dy, v, col, mk in [(off, ABSTRACT_SM[k], ORANGE, "o"), (-off, TABLE2[k], BLUE, "s")]:
ax.plot([v[1], v[2]], [y + dy] * 2, color=col, lw=LW, solid_capstyle="butt")
ax.plot(v[0], y + dy, marker=mk, ms=MS, color=col, mec=BG, mew=1.5, ls="")
ax.text(1.03, y + dy, f"{v[0]:.2f} ({v[1]:.2f}–{v[2]:.2f}) {p_print(v)}",
va="center", fontsize=10, color=INK, transform=tr)
ax.axvline(1, color=INK, lw=1.0, zorder=2)
ax.text(1.03, len(KEYS) - 0.45, "no association (OR = 1)", fontsize=10, color=INK, va="bottom")
ax.set_xscale("log"); ax.set_xlim(0.5, 8)
ticks = [0.5, 1, 2, 4, 8]
ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
ax.set_yticks(range(len(KEYS))); ax.set_yticklabels([SHORT[k] for k in KEYS][::-1], fontsize=10)
ax.set_ylim(-0.6, len(KEYS) - 0.2)
ax.set_xlabel("Adjusted odds ratio (log scale) with 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(1.03, len(KEYS) - 0.45, "AOR (95% CI), printed p", fontsize=10, color=INK2, va="bottom", transform=tr)
h = [Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS,
label="Abstract, given for sexual minorities (= Table 1, whole cohort, N=1247)"),
Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS,
label="Table 2, sexual-minority model (N=497)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), bbox_transform=fig.transFigure,
ncol=1, frameon=False, fontsize=10)
save(fig, "cand2_forest")
# --- Candidate 3: Wald z recomputed from each printed CI, with the 5% threshold ---------------
def cand3():
fig, ax = plt.subplots(figsize=(W, 3.9))
fig.subplots_adjust(left=0.235, right=0.97, bottom=0.17, top=0.86)
for i, k in enumerate(KEYS):
y = len(KEYS) - 1 - i
za = WALD[("Abstract", k)][1]; zt = WALD[("Table 2", k)][1]
ax.plot([zt, za], [y, y], color="#b9b7b0", lw=LW, zorder=1)
ax.plot(za, y, "o", ms=MS + 1, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(zt, y, "s", ms=MS + 1, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(za + 0.25, y + 0.16, f"z={za:.2f}", fontsize=10, color=INK, va="bottom")
ax.text(zt - 0.25, y + 0.16, f"z={zt:.2f}", fontsize=10, color=INK, va="bottom", ha="right")
ax.axvline(Z975, color=INK, lw=1.0, zorder=2)
ax.text(Z975 + 0.08, -0.62, "z = 1.96 (two-sided p = 0.05)", fontsize=10, color=INK, va="bottom")
ax.set_xlim(0, 9.5); ax.set_ylim(-0.7, len(KEYS) - 0.3)
ax.set_yticks(range(len(KEYS))); ax.set_yticklabels([SHORT[k] for k in KEYS][::-1], fontsize=10)
ax.set_xlabel("Wald z = ln(AOR) / SE, SE recomputed from the printed 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="from the abstract's CI (= Table 1)"),
Line2D([], [], marker="s", ls="", ms=MS, color=BLUE, label="from Table 2's CI (sexual minorities)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), bbox_transform=fig.transFigure,
ncol=2, frameon=False, fontsize=10)
save(fig, "cand3_wald_z")
# --- Candidate 4: comparison grid (abstract | Table 1 | Table 2) ------------------------------
def cand4():
fig = plt.figure(figsize=(W, 4.3))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(-2, 58); ax.axis("off")
cols = [(0.5, "Estimate"), (26, "Abstract\n(for sexual minorities)"),
(51.5, "Table 1\n(whole cohort,\nN=1247)"), (76, "Table 2\n(sexual minorities,\nN=497)")]
for x, t in cols:
ax.text(x, 53, t, fontsize=10, weight="bold", va="top")
ax.plot([0.5, 99.5], [45, 45], color=INK2, lw=0.8)
for i, k in enumerate(KEYS):
y = 40 - i * 10
if i % 2 == 0:
ax.add_patch(FancyBboxPatch((0.5, y - 4.6), 99, 9.2, boxstyle="square,pad=0", fc=PANEL, ec="none"))
ax.text(0.5, y, SHORT[k], fontsize=10, va="center", linespacing=1.2)
a, t1, t2 = ABSTRACT_SM[k], TABLE1[k], TABLE2[k]
for x, v, col in [(26, a, ORANGE), (51.5, t1, INK), (76, t2, BLUE)]:
ax.text(x, y + 1.6, f"{v[0]:.2f} ({v[1]:.2f}–{v[2]:.2f})", fontsize=10.5, color=col,
weight="bold", va="center")
sig = v[1] > 1 or v[2] < 1
ax.text(x, y - 2.0, f"{p_print(v)} · {'CI excludes 1' if sig else 'CI includes 1'}",
fontsize=10, color=INK2, va="center")
ax.text(48.9, y + 1.6, "=", fontsize=12, color=INK, ha="center", va="center", weight="bold")
ax.text(0.5, 2.0, "'=' marks an exact match of AOR, both CI limits and p.\nOrange: value printed in the abstract; "
"blue: paper's own sexual-minority model.", fontsize=10, color=INK2, va="center", linespacing=1.3)
save(fig, "cand4_grid")
# --- Candidate 5: one estimate, all the paper's versions of it --------------------------------
def cand5():
k = KEYS[0]
rows = [
("Abstract: “sexual minority\nadolescents”", ABSTRACT_SM[k], ORANGE, "o", True),
("Table 1: whole cohort\n(N=1247)", TABLE1[k], INK2, "D", False),
("Table 2: sexual minorities\n(N=497)", TABLE2[k], BLUE, "s", True),
("Crude OR, Table 2 counts\n(14/29 vs 79/246)", CRUDE_T2[k], BLUE, "s", False),
("Table S1: heterosexuals\nonly (N=750)", TABLES1_HIVPOS_ANX, INK2, "D", False),
]
fig, ax = plt.subplots(figsize=(W, 4.4))
fig.subplots_adjust(left=0.29, right=0.725, bottom=0.14, top=0.74)
tr = ax.get_yaxis_transform()
for i, (lab, v, col, mk, filled) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([v[1], v[2]], [y, y], color=col, lw=LW, solid_capstyle="butt")
ax.plot(v[0], y, marker=mk, ms=MS, color=col if filled else BG, mec=col, mew=2, ls="")
p = v[3] if isinstance(v[3], str) else f"{v[3]:.3f}"
ptxt = f"p{p}" if p.startswith("<") else f"p={p}"
ax.text(1.03, y, f"{v[0]:.2f} ({v[1]:.2f}–{v[2]:.2f}) {ptxt}", va="center", fontsize=10, transform=tr)
ax.axvline(1, color=INK, lw=1.0, zorder=2)
ax.text(1.03, len(rows) - 0.55, "OR = 1", fontsize=10, color=INK, va="bottom")
ax.set_xscale("log"); ax.set_xlim(0.5, 13)
ticks = [0.5, 1, 2, 4, 8]
ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("OR, high general anxiety, HIV-positive vs negative\n(log scale, 95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="printed in the abstract"),
Line2D([], [], marker="s", color=BLUE, lw=LW, ms=MS, label="sexual-minority data (filled: Table 2 AOR; open: crude OR recomputed)"),
Line2D([], [], marker="D", color=INK2, lw=LW, ms=MS, mfc=BG, label="other populations in the paper")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, 1.0), bbox_transform=fig.transFigure,
ncol=1, frameon=False, fontsize=10)
save(fig, "cand5_one_estimate")
cand1(); cand2(); cand3(); cand4(); cand5()
for src, dst in [("cand1_text_panels", "fig1"), ("cand2_forest", "fig2")]:
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"{src}.{ext}"), os.path.join(HERE, f"{dst}.{ext}"))
print("fig1 = cand1_text_panels, fig2 = cand2_forest (provisional, pending the choice of design)")
CGRP infusion rate is 5 µg/kg/min in Abstract and Results, 50 ng/kg/min in Methods
Brain (2026) · doi:10.1093/brain/awag008 · PMC13431663
Summary
The paper's main intervention is a 20-minute intra-carotid infusion of CGRP in anaesthetized rats. The Abstract and the Results give its rate as 5 µg/kg/min. The Methods and the Figure 1 and Figure 4 legends give 50 ng/kg/min for the same infusion. 5 µg/kg/min is 5,000 ng/kg/min, so the two values differ by exactly 100-fold. Both are described as the dose from human studies. The human provocation studies the paper cites infused 1.5–2 µg/min, about 21–29 ng/kg/min for a 70 kg adult. That supports 50 ng/kg/min and puts the Abstract's value at 175–233 times the human rate. The neuron counts and statistics do not depend on which rate is right. The paper's claim that the effect occurs at a physiologically relevant "human dose" does depend on it.
What the paper reports
The Abstract states the rate of the main intervention:
"Intra-carotid CGRP infusion (5 µg/kg/min, 20 min) activated 62% of Aδ-fibres and 56% of C-fibres" (Abstract)
The Methods give a rate 100 times lower for the same intra-carotid infusion, with the same duration and a stated human basis:
"CGRP (α calcitonin gene-related peptide rat; Sigma Aldrich) was infused using the intra-carotid artery cannula at a rate of 50 ng/kg/min for 20 min (dose from human studies)" (Materials and methods, "CGRP infusion protocol"; followed by citation 8)
The Results use the Abstract's value for the intra-carotid infusion, and the same value for the intravenous infusion:
"However, when delivered by intra-carotid infusion at the human dose (5 µg/kg/min for 20 min), CGRP, but not saline vehicle, produced sustained activation of first- and second-order neurons" (Results, "Activation and sensitization", paragraph 2)
"When administered either by intraperitoneal injection (100 µg/kg), or by intravenous infusion at the low dose used in human headache provocation studies (5 µg/kg/min for 20 min), CGRP failed to produce activation of first- or second-order neurons" (Results, "Activation and sensitization", paragraph 1)
Two figure legends use the Methods value:
"tested for activation by intra-carotid infusion of CGRP (50 ng/kg/min for 20 min) (A–D) or saline (E–H)." (Figure 1 legend)
"tested for activation by intra-carotid infusion of CGRP (50 ng/kg/min for 20 min) (A–C)." (Figure 4 legend)
The Discussion gives no number but describes the dose as human and physiological:
"Similar to previous findings,20 we also found that intravenous infusion of CGRP at the human dose failed to produce nociceptive neuronal activation."
"This change in route allowed us to achieve nociceptive neuronal activation without resorting to raising the dose to non-physiological levels and potentially inducing effects unrelated to CGRP’s role in headache." (Discussion, paragraph 2)
The paper also refers to "the human CGRP provocation studies9,10" (Discussion, paragraph 4). Its animals were "Male and female Sprague Dawley rats (250–320 g, 10 to 12 weeks old)" (Methods, "Surgical preparation").
What the paper's own numbers show
Method. I collected every per-minute CGRP rate in the PMC archive copy. "5 µg/kg/min" occurs three times (Abstract, Results paragraphs 1 and 2) and "50 ng/kg/min" three times (Methods, Fig 1 and Fig 4 legends). Every statement gives a duration of 20 min. I converted both rates to one unit, computed the total given over 20 minutes and per animal at the stated body masses, and compared the total with the paper's intraperitoneal dose. For an outside anchor I used the two human provocation studies the paper cites as refs 9 and 10. Their PubMed abstracts give fixed rates of 2 µg/min and 1.5 µg/min, each for 20 min. I converted these to ng/kg/min for a 60–80 kg adult, and also scaled them to rat by body surface area (FDA conversion factors, ×37/6). As a control I checked that the Abstract's activation percentages match the Results counts. All numbers are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Intra-carotid CGRP rate | Abstract; Results para 2 | 5 µg/kg/min, 20 min | 5 × 1000 = 5,000 ng/kg/min |
| Intra-carotid CGRP rate | Methods; Fig 1 and Fig 4 legends | 50 ng/kg/min, 20 min | 0.05 µg/kg/min |
| Ratio of the two rates | Abstract/Results vs Methods/legends | — | 5,000 / 50 = 100-fold |
| Total over 20 min | from the rates above | — | 100 µg/kg at 5 µg/kg/min; 1 µg/kg at 50 ng/kg/min |
| Intraperitoneal dose | Results para 1 | 100 µg/kg | equals the 20-min total at 5 µg/kg/min; 100 times the total at 50 ng/kg/min |
| Total per rat (250–320 g) | Methods (body mass) | — | 25–32 µg at 5 µg/kg/min; 250–320 ng at 50 ng/kg/min |
| Intravenous CGRP rate | Results para 1 (no i.v. rate in Methods) | 5 µg/kg/min, 20 min | the Methods give no separate value to check it against |
| Human provocation rate, ref 9 (2 µg/min) | cited studies | — | 25.0–33.3 ng/kg/min at 80–60 kg; 28.6 at 70 kg |
| Human provocation rate, ref 10 (1.5 µg/min) | cited studies | — | 18.8–25.0 ng/kg/min at 80–60 kg; 21.4 at 70 kg |
| 50 ng/kg/min relative to human rate | Methods vs refs 9, 10 | "dose from human studies" | 1.75–2.33× at 70 kg (1.5–2.7× over 60–80 kg) |
| 5 µg/kg/min relative to human rate | Abstract/Results vs refs 9, 10 | "the human dose" | 175–233× at 70 kg (150–267× over 60–80 kg); 350 µg/min for a 70 kg adult |
| Human range scaled to rat by body surface area | refs 9, 10, ×37/6 | — | 116–206 ng/kg/min; 50 is 2.3–4.1× below, 5,000 is 24–43× above |
| Activation, Aδ / C-fibres | Abstract (62%, 56%) vs Results (10/16, 9/16) | 62%, 56% | 62.5%, 56.25% (consistent) |
| Activation, first- / second-order | Results (19/32, 18/24) | 59%, 75% | 59.4%, 75.0% (consistent) |
The two rates cannot describe one infusion. The 50 ng/kg/min value lies within about a factor of 2 of the cited human rates. The 5 µg/kg/min value is more than 100 times above them, and more than 20 times above them even after scaling to rat by body surface area. At 5 µg/kg/min the 20-minute total would equal the intraperitoneal dose (100 µg/kg) that the Results list beside the "low dose" infusion.
Figure
Figure 1. The two orange-bordered panels quote the paper. On the left are the three statements of 5 µg/kg/min: the Abstract, Results paragraph 2 (intra-carotid) and Results paragraph 1 (intravenous). On the right are the three statements of 50 ng/kg/min: the Methods and the Fig 1 and Fig 4 legends. All six give a 20-minute infusion. Five describe the intra-carotid infusion, and Results paragraph 1 describes the intravenous one. The blue strip shows the arithmetic from recompute.py.txt. 5 µg/kg/min is 5,000 ng/kg/min, 100 times 50 ng/kg/min. Over 20 minutes that is 100 µg/kg against 1 µg/kg, and the 100 µg/kg total equals the paper's intraperitoneal dose.
Figure 2. CGRP infusion rate on a log axis. Orange dots are the two rates reported in the paper. Blue intervals are computed in recompute.py.txt from the human studies the paper cites (refs 9 and 10: 2 and 1.5 µg/min for 20 min). Each interval spans a 60–80 kg adult, and its dot marks 70 kg. The light blue band covers both studies (18.8–33.3 ng/kg/min). The third row scales that human range to rat by body surface area (×37/6), giving 116–206 ng/kg/min. The Methods and legend value, 50 ng/kg/min, is 1.75–2.33 times the 70 kg human rate. The Abstract and Results value, 5,000 ng/kg/min, is 175–233 times it. That is two orders of magnitude, and on a log axis the difference is shown at true scale.
Why it matters
The infusion rate defines the paper's main experiment, and the paper's interpretation rests on it being a human-relevant dose. The Abstract is the part most often read and indexed, and it gives 5 µg/kg/min. A reader who uses that value to replicate the experiment or to compare it with other studies would give 100 times the dose in the Methods. The Results use the same value twice, once for the positive intra-carotid result ("at the human dose") and once for the negative intravenous result ("the low dose used in human headache provocation studies").
The counts, percentages and tests (19/32 and 18/24 neurons activated, Table 1, the lidocaine experiments) do not depend on which rate is printed, and the Abstract's percentages match the Results counts. The finding that intra-carotid CGRP activated these neurons is therefore not affected by the discrepancy. What is affected is the stated dose. The paper itself frames this as the key point: the intra-carotid route gave activation "without resorting to raising the dose to non-physiological levels". If the Abstract's value were the one given, the infusion would be about 175–233 times the cited human rate. Its 20-minute total would also equal the intraperitoneal dose, and that claim could not stand. If, as the evidence suggests, 50 ng/kg/min was given, the claim stands but the Abstract and Results misstate it. The intravenous comparison has a further uncertainty. The Results give the intravenous rate as 5 µg/kg/min, and the Methods give no separate intravenous rate. So it cannot be confirmed from the paper that the intravenous and intra-carotid infusions used the same rate. That comparison is the paper's main evidence that route, not dose, explains the different results.
Explanations considered
Unit typesetting or conversion artifact. Not supported. The two values differ in both the number (5 vs 50) and the prefix (µ vs n). A µ/n substitution alone would give 5 ng/kg/min or 50 µg/kg/min, and neither matches the other statement. The version of record on the publisher's site has the same six statements with the same values (fetched 2026-10-24). The PMC and Europe PMC copies match it.
Two different doses (for example, intravenous and intra-carotid). Ruled out for the intra-carotid infusion. The Abstract and Results paragraph 2 give 5 µg/kg/min for the intra-carotid route, and the Methods and both legends give 50 ng/kg/min for the same route, all for 20 min. The Methods describe a single CGRP infusion protocol, through the intra-carotid cannula. For the intravenous infusion the paper gives only 5 µg/kg/min (Results paragraph 1), and the Discussion calls both the "human dose". So the paper presents one intended rate.
Confusion with the paper's other, genuinely different doses. Not applicable. The intraperitoneal dose (100 µg/kg) and the trigeminal ganglion injection (Fig 7 legend, "300 ng CGRP") are single amounts, not per-minute rates, and are not in conflict. The fact that 5 µg/kg/min for 20 minutes equals exactly 100 µg/kg is noted above. The paper gives no reason for that equality.
A rat dose scaled up from the human dose. Does not account for 5 µg/kg/min. Scaling the cited human rates to rat by body surface area gives 116–206 ng/kg/min. That is 2.3–4.1 times above 50 ng/kg/min but 24–43 times below 5,000 ng/kg/min. The paper describes no scaling and calls the dose the dose "from human studies".
The Methods and legends hold the error, not the Abstract. Possible in principle but not supported by the paper's wording. The Abstract and Results call the 5 µg/kg/min infusion "the human dose" and "the low dose used in human headache provocation studies". The cited human studies infused 1.5–2 µg/min in total, which for a 70 kg adult would correspond to 5 µg/kg/min only if it were given 175–233 times over. For an adult the Abstract's rate would mean 350 µg/min. The Methods value fits the cited studies within a factor of about 2. The Discussion's "without resorting to raising the dose to non-physiological levels" also fits the lower value. The Methods cite reference 8 for the dose, an earlier study from the same group (Brain 2024, PMC11292906). An earlier audit stage found no CGRP infusion rate in its full text, so it cannot settle the question. The text alone cannot prove which rate was given; only the laboratory records can.
Status
As of 2026-10-24:
- Version of record. The publisher's article page (academic.oup.com, reached via doi.org) was read on 2026-10-24. It has 5 µg/kg/min in the Abstract and twice in the Results, and 50 ng/kg/min in the Methods and in the Fig 1 and Fig 4 legends, matching the PMC archive copy. The article history lists received 4 August 2025, published 8 January 2026, and "Corrected and typeset: 13 July 2026". The page shows no correction notice linked to that production date. The Europe PMC full-text XML, fetched on 2026-10-24, has the same three occurrences of each rate. An earlier audit stage found the same wording on the PMC article page.
- Supplements. The PMC/Europe PMC XML has no supplementary-material elements, and the PMC page lists no supplementary files. The publisher's page has a "Supplementary Data" tab, but an earlier audit stage found no supplementary files in it. The data availability statement says the data are "available from the corresponding author, upon reasonable request". Reference 8, the cited source of the dose, gives no CGRP infusion rate (earlier audit stage).
- Corrections.
src/known.pyreports no Crossref update records and no PMC or Europe PMC correction, retraction or expression-of-concern notice. An earlier audit stage found noupdate-to,updated-byor relation entries in the Crossref record, and no correction DOI in a Crossref title search. A web search on 2026-10-24 for the title with "correction", "erratum" or "PubPeer" found no notice for this paper. - PubPeer. Not checked: pubpeer.com returned HTTP 403 to both the API and the search page on 2026-10-24, as it had at the earlier stage.
- Human reference doses. Refs 9 and 10 were identified from the article's reference list in the Europe PMC XML. Their rates (2 µg/min and 1.5 µg/min, each for 20 min, i.v.) are from their PubMed abstracts (PMID 11993614; PMID 20855363).
Sources: PMC13431663 (PubMed Central archive); https://academic.oup.com/brain/article/149/8/2760/8417004; Europe PMC full-text XML for PMC13431663; Crossref and PMC notice records for 10.1093/brain/awag008; PubMed records 11993614 (Cephalalgia 2002;22:54–61) and 20855363 (Cephalalgia 2010;30:1179–86).
Suggested correction
The authors should confirm the administered rate against their infusion records. If it was 50 ng/kg/min, as the Methods, both figure legends and the cited human studies indicate, the Abstract should read "Intra-carotid CGRP infusion (50 ng/kg/min, 20 min)". Results, "Activation and sensitization", paragraph 2 should read "at the human dose (50 ng/kg/min for 20 min)". The authors should also state the rate actually used for the intravenous infusion in Results paragraph 1, and add it to the Methods ("CGRP infusion protocol"), which describes only the intra-carotid infusion. If 5 µg/kg/min was given instead, the Methods and the Fig 1 and Fig 4 legends would need correcting. The descriptions of this dose as "the human dose", the "low dose used in human headache provocation studies" and not "non-physiological" would then also need revisiting.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13431663 (Brain 2026, doi:10.1093/brain/awag008)
# Intra-carotid CGRP infusion rate: 5 ug/kg/min (Abstract, Results) vs 50 ng/kg/min (Methods,
# Fig 1 and Fig 4 legends).
#
# Usage: python recompute.py.txt
# Needs: numpy, matplotlib. If /home/ubuntu/papers/src/show.py is available, every quote is
# checked verbatim against the PubMed Central archive copy of the article.
# Output: prints every recomputed number used in writeup.md; writes fig1.svg/png and
# fig2.svg/png next to this script.
import os, subprocess
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- 1. statements in the paper
QUOTES = [ # (location, verbatim text)
("Abstract", "CGRP was administered via intra-carotid infusion."),
("Abstract", "Intra-carotid CGRP infusion (5 µg/kg/min, 20 min) activated 62% of Aδ-fibres and 56% of C-fibres"),
("Methods, CGRP infusion protocol",
"CGRP (α calcitonin gene-related peptide rat; Sigma Aldrich) was infused using the intra-carotid artery "
"cannula at a rate of 50 ng/kg/min for 20 min (dose from human studies)"),
("Methods, Surgical preparation",
"Male and female Sprague Dawley rats (250–320 g, 10 to 12 weeks old)"),
("Methods, Surgical preparation",
"Two cannulas were surgically implanted in the animals for later infusion of drugs: an intrafemoral vein "
"cannula and an intra-carotid artery cannula (common carotid artery)."),
("Results, Activation and sensitization, para 1",
"When administered either by intraperitoneal injection (100 µg/kg), or by intravenous infusion at the low "
"dose used in human headache provocation studies (5 µg/kg/min for 20 min), CGRP failed to produce "
"activation of first- or second-order neurons"),
("Results, Activation and sensitization, para 2",
"However, when delivered by intra-carotid infusion at the human dose (5 µg/kg/min for 20 min), CGRP, but "
"not saline vehicle, produced sustained activation of first- and second-order neurons"),
("Results, Activation and sensitization, para 2",
"Intra-carotid CGRP infusion produced increased firing in 19/32 first-order neurons (59%; 10/16 Aδ, 9/16 "
"C-fibres) and 18/24 second-order dorsal horn neurons (75%; 9/12 HT, 9/12 WDR)"),
("Figure 1 legend",
"tested for activation by intra-carotid infusion of CGRP (50 ng/kg/min for 20 min) (A–D) or saline (E–H)."),
("Figure 4 legend",
"tested for activation by intra-carotid infusion of CGRP (50 ng/kg/min for 20 min) (A–C)."),
("Figure 7 legend", "2 h after injection (300 ng CGRP, 10 ml)"),
("Discussion, para 2",
"Similar to previous findings,20 we also found that intravenous infusion of CGRP at the human dose failed "
"to produce nociceptive neuronal activation."),
("Discussion, para 2",
"This change in route allowed us to achieve nociceptive neuronal activation without resorting to raising "
"the dose to non-physiological levels and potentially inducing effects unrelated to CGRP’s role in headache."),
("Discussion, para 4",
"the immediate and brief headache seen in the human CGRP provocation studies9,10"),
]
def check_quotes():
show, py = "/home/ubuntu/papers/src/show.py", "/home/ubuntu/papers-data/venv/bin/python"
print("1) Quotes (verbatim check against the PMC archive copy)")
full = None
if os.path.exists(show) and os.path.exists(py):
r = subprocess.run([py, show, "PMC13431663", "--full"], cwd="/home/ubuntu/papers",
capture_output=True, text=True)
full = r.stdout if r.returncode == 0 else None
if full is not None:
# the archive sets number-unit gaps as U+2005 (four-per-em space); compare with plain spaces
full = full.replace("\u2005", " ").replace("\u2009", " ").replace("\u00a0", " ")
for loc, q in QUOTES:
tag = "not checked (archive tool unavailable)" if full is None else \
("found verbatim" if q in full else "NOT FOUND")
print(" %-48s %s" % (loc, tag))
if full is not None:
assert all(q in full for _, q in QUOTES), "a quote was not found verbatim"
# how many times each rate occurs in the archive text
print(" occurrences of '5 µg/kg/min': %d; of '50 ng/kg/min': %d"
% (full.count("5 µg/kg/min"), full.count("50 ng/kg/min")))
# ---------------------------------------------------------------- 2. the paper's own numbers
REP_HIGH_UG = 5.0 # ug/kg/min, Abstract and Results (x2)
REP_LOW_NG = 50.0 # ng/kg/min, Methods and Fig 1, Fig 4 legends
DUR_MIN = 20.0 # min, stated with both rates
IP_UG_PER_KG = 100.0 # ug/kg, intraperitoneal dose (Results)
RAT_KG = (0.250, 0.320) # body mass range, Methods
def arithmetic():
print("\n2) Arithmetic from the paper's own numbers")
high_ng = REP_HIGH_UG * 1000.0
fold = high_ng / REP_LOW_NG
print(" 5 µg/kg/min x 1000 ng/µg = %.0f ng/kg/min" % high_ng)
print(" %.0f / %.0f = %.0f-fold" % (high_ng, REP_LOW_NG, fold))
tot_high = REP_HIGH_UG * DUR_MIN # ug/kg
tot_low = REP_LOW_NG * DUR_MIN / 1000.0 # ug/kg
print(" total over 20 min: 5 µg/kg/min -> %.0f µg/kg; 50 ng/kg/min -> %.0f µg/kg" % (tot_high, tot_low))
print(" intraperitoneal dose (Results): %.0f µg/kg -> equal to the 5 µg/kg/min total? %s"
% (IP_UG_PER_KG, tot_high == IP_UG_PER_KG))
print(" intraperitoneal dose / 20-min total at 50 ng/kg/min = %.0f-fold" % (IP_UG_PER_KG / tot_low))
for m in RAT_KG:
print(" rat %.0f g: total %.0f µg at 5 µg/kg/min vs %.0f ng at 50 ng/kg/min"
% (m * 1000, tot_high * m, tot_low * m * 1000))
# abstract percentages vs Results counts
for lab, k, n, rep in [("Aδ", 10, 16, 62), ("C", 9, 16, 56), ("first-order", 19, 32, 59),
("second-order", 18, 24, 75)]:
print(" %-13s %d/%d = %.2f%% (reported %d%%)" % (lab, k, n, 100 * k / n, rep))
return dict(high_ng=high_ng, fold=fold, tot_high=tot_high, tot_low=tot_low)
# ---------------------------------------------------------------- 3. the cited human provocation dose
# Refs 9 and 10 of the paper (cited in the Discussion as "the human CGRP provocation studies9,10"):
# ref 9 (Cephalalgia 2002;22:54-61): human alphaCGRP 2 ug/min i.v. for 20 min (PubMed abstract)
# ref 10 (Cephalalgia 2010;30:1179-86): 1.5 ug/min i.v. for 20 min (PubMed abstract)
# These are fixed rates per person; convert to ng/kg/min for an assumed 60-80 kg adult (70 kg central).
HUMAN = {"ref 10 (1.5 µg/min)": 1.5, "ref 9 (2 µg/min)": 2.0}
ADULT_KG = (60.0, 70.0, 80.0)
KM_RAT, KM_HUMAN = 6.0, 37.0 # body-surface-area conversion factors (FDA 2005 guidance)
def human():
print("\n3) Human provocation rates from the studies the paper cites, per kg")
out = {}
for lab, ug in HUMAN.items():
r = [ug * 1000.0 / kg for kg in ADULT_KG]
out[lab] = r
print(" %-20s 60 kg: %.1f 70 kg: %.1f 80 kg: %.1f ng/kg/min" % (lab, r[0], r[1], r[2]))
lo = min(v[2] for v in out.values()); hi = max(v[0] for v in out.values())
lo70 = min(v[1] for v in out.values()); hi70 = max(v[1] for v in out.values())
print(" range 60-80 kg: %.1f-%.1f ng/kg/min; at 70 kg: %.1f-%.1f" % (lo, hi, lo70, hi70))
hi_ng = REP_HIGH_UG * 1000
print(" 50 ng/kg/min / human: %.1f-%.1fx (70 kg: %.2f-%.2fx)"
% (REP_LOW_NG / hi, REP_LOW_NG / lo, REP_LOW_NG / hi70, REP_LOW_NG / lo70))
print(" 5000 ng/kg/min / human: %.0f-%.0fx (70 kg: %.0f-%.0fx)"
% (hi_ng / hi, hi_ng / lo, hi_ng / hi70, hi_ng / lo70))
print(" 5 µg/kg/min for a 70 kg adult would be %.0f µg/min (vs 1.5-2 µg/min)" % (REP_HIGH_UG * 70))
f = KM_HUMAN / KM_RAT
blo, bhi = lo * f, hi * f
print(" body-surface-area scaling to rat (x%d/%d = %.2f): %.0f-%.0f ng/kg/min" % (KM_HUMAN, KM_RAT, f, blo, bhi))
print(" 50 ng/kg/min is %.1f-%.1fx below that; 5000 ng/kg/min is %.0f-%.0fx above it"
% (blo / REP_LOW_NG, bhi / REP_LOW_NG, hi_ng / bhi, hi_ng / blo))
return dict(lo=lo, hi=hi, lo70=lo70, hi70=hi70, rows=out, bsa=(blo, bhi))
# ---------------------------------------------------------------- 4. figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": FG,
"axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
"axes.edgecolor": FG2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def fig1(a):
W, H = 7.2, 5.3
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
def panel(x, y, w, h, title, items):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=ORANGE, lw=1.5))
ax.text(x + 0.14, y + h - 0.14, title, ha="left", va="top", fontsize=11, weight="bold")
yy = y + h - 0.52
for loc, body, nlines in items:
ax.text(x + 0.14, yy, loc, ha="left", va="top", fontsize=10, color=FG2)
ax.text(x + 0.14, yy - 0.24, body, ha="left", va="top", fontsize=10, linespacing=1.3)
yy -= 0.24 + 0.205 * nlines + 0.2
panel(0.12, 2.0, 3.05, 3.18, "Abstract, Results: 5 µg/kg/min",
[("Abstract", "“Intra-carotid CGRP infusion\n(5 µg/kg/min, 20 min)”", 2),
("Results, para 2 (intra-carotid)", "“at the human dose\n(5 µg/kg/min for 20 min)”", 2),
("Results, para 1 (intravenous)", "“at the low dose used in human\nheadache provocation studies\n"
"(5 µg/kg/min for 20 min)”", 3)])
panel(4.03, 2.0, 3.05, 3.18, "Methods, legends: 50 ng/kg/min",
[("Methods, CGRP infusion protocol", "“at a rate of 50 ng/kg/min\nfor 20 min (dose from\nhuman studies)”", 3),
("Fig 1 and Fig 4 legends", "“intra-carotid infusion of CGRP\n(50 ng/kg/min for 20 min)”", 2)])
ax.add_patch(FancyArrowPatch((3.19, 3.75), (4.01, 3.75), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=FG))
ax.text(3.6, 3.98, "conflict", ha="center", va="bottom", fontsize=10, weight="bold")
ax.text(3.6, 3.52, "same\ninfusion,\n100-fold\napart", ha="center", va="top", fontsize=10, color=FG2)
ax.add_patch(FancyBboxPatch((0.12, 0.12), 6.96, 1.6, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="#eef4fb", ec=BLUE, lw=1.5))
ax.text(0.26, 1.6, "Recomputed from the paper's own numbers (recompute.py.txt)", ha="left", va="top",
fontsize=10, weight="bold", color=BLUE)
ax.text(0.26, 1.27, "5 µg/kg/min × 1000 = %.0f ng/kg/min = %.0f × 50 ng/kg/min"
% (a["high_ng"], a["fold"]), ha="left", va="top", fontsize=10)
ax.text(0.26, 0.97, "Over 20 min: %.0f µg/kg versus %.0f µg/kg" % (a["tot_high"], a["tot_low"]),
ha="left", va="top", fontsize=10)
ax.text(0.26, 0.67, "At 5 µg/kg/min the 20-min total (%.0f µg/kg) equals the intraperitoneal dose\n"
"that the Results list beside the “low dose used in human headache provocation studies”" % a["tot_high"],
ha="left", va="top", fontsize=10, color=FG2, linespacing=1.3)
save(fig, "fig1")
def fig2(h):
fig, ax = plt.subplots(figsize=(7.2, 3.5))
fig.subplots_adjust(left=0.36, right=0.97, top=0.84, bottom=0.17)
rows = [] # (label, lo, mid, hi, color, kind)
r10, r9 = h["rows"]["ref 10 (1.5 µg/min)"], h["rows"]["ref 9 (2 µg/min)"]
rows.append(("Human study, ref 10\n1.5 µg/min, 60–80 kg", r10[2], r10[1], r10[0], BLUE))
rows.append(("Human study, ref 9\n2 µg/min, 60–80 kg", r9[2], r9[1], r9[0], BLUE))
rows.append(("Human range scaled to rat\nby body surface area (×37/6)", h["bsa"][0],
np.sqrt(h["bsa"][0] * h["bsa"][1]), h["bsa"][1], BLUE))
rows.append(("Methods; Fig 1, Fig 4 legends\n50 ng/kg/min", None, REP_LOW_NG, None, ORANGE))
rows.append(("Abstract; Results (×2)\n5 µg/kg/min = 5,000 ng/kg/min", None, REP_HIGH_UG * 1000, None, ORANGE))
ys = np.arange(len(rows))[::-1]
ax.axvspan(h["lo"], h["hi"], color=BLUE, alpha=0.08, lw=0)
for y, (lab, lo, mid, hi, c) in zip(ys, rows):
if lo is not None:
ax.plot([lo, hi], [y, y], color=c, lw=2, solid_capstyle="butt")
ax.plot([lo, hi], [y, y], "|", color=c, ms=10, mew=2)
if "scaled" not in lab:
ax.plot([mid], [y], "o", color=c, ms=8)
else:
ax.plot([mid], [y], "o", color=c, ms=9)
ax.text(REP_LOW_NG * 1.25, ys[3], "%.2f–%.2f× human rate at 70 kg"
% (REP_LOW_NG / h["hi70"], REP_LOW_NG / h["lo70"]), va="center", fontsize=10, color=FG)
ax.text(REP_HIGH_UG * 1000 / 1.25, ys[4], "%.0f–%.0f× human rate at 70 kg" %
(REP_HIGH_UG * 1000 / h["hi70"], REP_HIGH_UG * 1000 / h["lo70"]),
va="center", ha="right", fontsize=10, color=FG)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=FG)
ax.set_xscale("log"); ax.set_xlim(8, 12000)
ax.set_xticks([10, 100, 1000, 10000]); ax.set_xticklabels(["10", "100", "1,000", "10,000"]); ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("CGRP infusion rate (ng/kg/min, log scale)")
ax.grid(axis="x", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
fig.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=0, ms=8, label="rate reported in the paper"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=8,
label="computed from the human studies the paper cites (dot: 70 kg)")],
loc="upper left", bbox_to_anchor=(0.02, 0.995), ncol=1, frameon=False, fontsize=10)
save(fig, "fig2")
if __name__ == "__main__":
check_quotes()
a = arithmetic()
h = human()
fig1(a); fig2(h)
print("\nfigures written to", HERE)
Abstract names G3244A where the body names G3242A as the second termination-abolishing pathogenic mutation
Cell (2010) · doi:10.1016/j.cell.2010.05.018 · PMC2887341
Summary
The Summary (abstract) of this crystal-structure paper on the human mitochondrial transcription terminator MTERF1 ends its results with a disease link: "the mitochondrial pathogenic G3249A and G3244A mutations interfere with key interactions for sequence recognition, eliminating termination." The body of the paper names a different second mutation, G3242A, in each of the three places where it states this result: the Introduction, the Results section on pathogenic mutations, and the Conclusions. The body also describes G3244A in terms that cannot fit the abstract sentence: its base pair makes "no specific interaction" with MTERF1 in the crystal structure, the mutation does not affect DNA binding, and its effect on termination is "moderate". A reader of the abstract alone is therefore told that the wrong variant loses a recognition contact and abolishes termination, and is never told about G3242A, the variant the paper's own data support.
What the paper reports
Abstract (Summary), final results sentence:
Finally, we have demonstrated that the mitochondrial pathogenic G3249A and G3244A mutations interfere with key interactions for sequence recognition, eliminating termination.
Introduction, last paragraph:
We have shown that two mutations, G3249A that causes a variant of Kearns-Sayre syndrome (Seneca et al., 2001) and G3242A that is associated with an uncharacterized mitochondrial disorder (Mimaki et al., 2009), interfere with protein-DNA interactions that are key for sequence recognition by MTERF1 and as a consequence eliminate its transcriptional termination activity.
Results and discussion, "Pathogenic mutations in the leu-tRNA", first paragraph (structure and binding):
All remaining mutations would not be expected to severely conflict with MTERF1 sequence recognition, except for G3242A and G3249A. G3242 interacts in the structure with Arg251, while G3249 forms a double hydrogen bond with Arg387 (Figure 6B). Both of these mutations would eliminate a guanine-arginine interaction: the hydrogen bond between O6 of the guanine and the amino group of the arginine could not be formed in the mutant DNA. Consistent with the mild decrease in binding observed in the R251A mutant, G3242A resulted in weak or no effect on binding. However, the G3249A mutation appears to completely eliminate specific DNA binding (Figure 6A and 3SE), consistent with the effect of the R387A mutation.
Same section, second paragraph (termination assays, Figure 6C/D):
As expected, only residual termination is observed in both orientations with the G3249A mutation. In addition, consistent with what was observed with the R251A mutant, the G3242A mutation, while not substantially altering binding, results nevertheless in a strong decrease in termination.
Unexpectedly, one additional mutation appears to moderately affect termination: G3244A, while not affecting DNA binding, results in a moderate but reproducible effect on termination. Interestingly, no specific interaction is observed in the crystal structure with any of the nucleotides in this base pair. However, it is adjacent to the extrahelical A:T base pair and displays a high base pair buckle (see Experimental Procedures).
Results and discussion, "Conclusions":
An important conclusion derived from the crystal structure is that two pathogenic mutations that interfere with arginine-guanine interactions involved in sequence recognition by MTERF1 result in a strong impairment in the ability of MTERF1 to terminate transcription. G3242A does not substantially affect binding, but very strongly reduces the ability of MTERF1 to terminate transcription, while G3249A appears to completely eliminate specific binding by MTERF1 and consequently severely impairs termination. Their strong effect on termination suggests that the pathogenic effects of the G3249A and G3242A MtDNA mutations might be at least partially mediated by interfering with the activity of MTERF1 at the leu-tRNA site.
Further experiments will be needed to address whether the G3249A and G3242A mutations result in transcriptional alterations in vivo and if this effect is sufficient to explain their clinical phenotype.
What the paper's own numbers show
Method. No arithmetic is involved. The error concerns which mutation is named, so I checked the paper's internal consistency. I parsed the PMC-archived full text and counted every mention of the five leu-tRNA variants the paper discusses (G3242A, A3243G, A3243T, G3244A, G3249A) in each section. I then extracted the pair of mutations that each section names as disrupting sequence recognition and eliminating or strongly impairing termination. Finally, I tabulated what the body says about each candidate's structural contact, its effect on binding and its effect on termination. recompute.py.txt holds the excerpts verbatim, checks all 33 excerpt sentences against the archived text (all found), and prints every value below.
| Quantity | Location | Reported | Recomputed from the paper's own text |
|---|---|---|---|
| Second mutation said to disrupt recognition and eliminate termination | Abstract (Summary) | G3244A (paired with G3249A) | Contradicted by all three body sections below |
| The same pair | Introduction, last paragraph | G3249A and G3242A | G3242A |
| Mutations expected to conflict with sequence recognition | Results, Pathogenic mutations, first paragraph | "except for G3242A and G3249A" | G3242A |
| The same pair | Conclusions (pair named twice; G3242A mentioned 4 times) | G3249A and G3242A | G3242A |
| Agreement of the three body sections with each other | Introduction, Results, Conclusions | — | All three name G3249A + G3242A |
| Agreement of the abstract with the body | Abstract vs body | — | No. The pairs differ in one mutation, G3244A vs G3242A, a single-character difference |
| Mentions of G3242A / G3244A / G3249A | Abstract | 0 / 1 / 1 | G3242A, the body's second mutation, never appears in the abstract |
| Mentions of G3242A / G3244A / G3249A | Introduction | 1 / 0 / 1 | — |
| Mentions of G3242A / G3244A / G3249A | Results, Pathogenic mutations | 3 / 2 / 3 | G3244A appears only here, as "one additional mutation" |
| Mentions of G3242A / G3244A / G3249A | Conclusions | 4 / 0 / 4 | — |
| Mentions in the whole article | All sections | G3242A 8, G3244A 3, G3249A 9 | — |
| Structural contact of the mutated base pair | Results, Pathogenic mutations | G3242 contacts Arg251; G3249 is double hydrogen-bonded to Arg387; G3244 base pair: "no specific interaction is observed" | Only G3242A and G3249A can remove a recognition contact |
| Effect on DNA binding | Results, Pathogenic mutations | G3242A "weak or no effect"; G3244A "not affecting DNA binding"; G3249A "completely eliminate specific DNA binding" | — |
| Effect on termination | Results, Pathogenic mutations; Conclusions | G3249A "only residual termination"; G3242A "strong decrease" / "very strongly reduces"; G3244A "moderate but reproducible effect" | G3244A's effect is moderate, not elimination |
By the paper's own account, G3244A meets neither condition in the abstract sentence. It removes no recognition contact, and it does not eliminate termination. G3242A meets the first condition and, in the body's words, comes close to meeting the second ("very strongly reduces").
Figure
The top panel holds the abstract's only sentence about the pathogenic mutations, with the second mutation it names (G3244A) in orange. The two lower panels hold verbatim excerpts from the body of the same article. The left panel collects what the Introduction, Results and Conclusions say about G3242A, the mutation they name alongside G3249A: it contacts Arg251, and it strongly decreases termination while leaving binding largely intact. G3242A does not appear in the abstract (dashed connector). The right panel collects everything the body says about G3244A. It appears only in the Results, as "one additional mutation" whose base pair makes no specific contact with MTERF1, which does not affect binding and which has a moderate effect on termination. The solid connector marks the conflict between the abstract's claim about G3244A and the body's description of it. All text is quoted from the PMC-archived article, and the section counts come from recompute.py.txt.
Why it matters
- It is the abstract's only disease-related result. The abstract's closing claim ("suggest a link between mitochondrial disease and the regulation of mitochondrial transcription") rests on this sentence. The version-of-record Highlights mention "Two mitochondrial pathogenic mutations" without naming them, so the abstract is the only summary that names the variants. In the abstract, the second disease variant tied to loss of MTERF1 function is the wrong one.
- Both the mechanism and the size of the effect are misattributed. For G3244A the abstract claims loss of a sequence-recognition contact and elimination of termination. The body reports no contact, normal binding and a moderate effect, which it tentatively explains by destabilization of a buckled G:C pair next to the everted A:T pair. A reader of the abstract would place G3244A in the wrong mechanistic category.
- The variant the data support is missing. G3242A, which the Conclusions call one of "two pathogenic mutations that interfere with arginine-guanine interactions", is not named in the abstract. Abstract-only readers would not learn that this variant was studied or what effect it had.
- The abstract is the version that circulates. The same sentence appears in PubMed/Europe PMC (PMID 20550934), in indexing services and in the OSTI record. It is the text most likely to be quoted, for example when variant pathogenicity evidence is assessed from the literature.
- What is not affected. The structure, the DNA unwinding and three-nucleotide eversion, the role of base flipping in stable binding and termination, and the body's account of the pathogenic mutations are all internally consistent. The data for G3242A and G3244A (Figure 6A, C, D) are described consistently in the body.
Explanations considered
- A single-character slip (G3242A typed as G3244A). The two names differ in one digit, and both variants are discussed in the same section, so this is plausible. It explains how the sentence could arise. It does not make the printed abstract correct, because the abstract still attributes a recognition contact and abolished termination to a variant that the paper shows has neither.
- A text-conversion or archiving artifact. Ruled out. The version of record (the Cell full-text page, read from a Wayback Machine snapshot because the live publisher pages refused automated access) has the same Summary sentence, and the same body sentences that name G3242A. The PMC author manuscript, PubMed, Europe PMC, Semantic Scholar and OSTI 1020213 all carry the same abstract wording.
- A different numbering convention, for example heavy-strand versus light-strand positions. Ruled out. mtDNA positions number base pairs, not strands, and the paper uses one numbering throughout. It discusses G3242, A3243 and G3244 as three distinct, adjacent positions with different structural roles (Arg251 contact, everted nucleotide, buckled base pair without contact).
- The abstract summarizing a different experiment, such as the forward-orientation assay or the supplement. Not supported. The Results describe the termination assays in both orientations together ("The results (Figure 6C/D) are mostly consistent with the DNA binding data") and give no orientation in which G3244A eliminates termination. The supplement (Figures S1–S3: alignment, conservation, ITC titrations) concerns binding, and the body states that G3244A does not affect binding. I could not download the supplement files or view the Figure 6 images (see Status), so this rests on the text's description of those data.
- "Eliminating termination" used loosely for "strongly impairing". Even on this reading the sentence fits G3242A ("very strongly reduces") but not G3244A ("moderate"), and the structural half of the sentence ("interfere with key interactions for sequence recognition") still excludes G3244A. The body itself uses "eliminate" for G3242A in the Introduction but "strong decrease" in the Results. That is a lesser wording difference within the body, separate from the abstract error.
- A deliberate claim that G3244A acts through sequence recognition. Not supported anywhere in the paper. The only mechanism the body proposes for G3244A is a slight destabilization of the bound protein, offered with "It is possible that".
Status
- Version of record. Checked. The live publisher pages (doi.org, cell.com full text and PDF, ScienceDirect) returned HTTP 403 or a redirect wall to automated access. The Elsevier API without a key gives metadata only (open archive, cover date 2010-06-11). I read the version of record from a Wayback Machine snapshot of the Cell full-text page (web.archive.org/web/20230111224332/https://www.cell.com/cell/fulltext/S0092-8674(10)00554-4). Its Summary sentence is identical to the PMC text. Its Introduction, Results and Conclusions name G3242A exactly as quoted above. Its Highlights say only "Two mitochondrial pathogenic mutations interfere with transcriptional termination" and do not name the mutations. All quotes in this write-up were confirmed verbatim against the PMC-archived text (author manuscript NIHMS208049).
- Supplements. PMC lists NIHMS208049-supplement-01.doc and -02.pdf. The version of record lists Figures S1–S3. Neither set could be downloaded (PMC download wall, no Europe PMC supplementary bundle, cell.com 403), and the Figure 6 images could not be fetched either. The conflict is between two passages of the main text, so the supplement cannot remove it.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI; its only relation is a Faculty Opinions review link. PMC and Europe PMC list no correction notices. Europe PMC links one "Comment in" item, a 2010 Cell Metabolism commentary (PMID 20620988), which is not a correction. The PMC record carries a related-article link to a retraction notice (PMC4698954); that notice concerns a different, cited paper on a related protein, not this article. PubPeer's API refused automated access (Cloudflare challenge), and a web search for the title on PubPeer returned no PubPeer entry. A web search for the abstract wording with "erratum" or "correction" found nothing relevant.
- Sources. PMC article PMC2887341 (author manuscript NIHMS208049); Wayback Machine snapshot of the Cell full-text page; PubMed/Europe PMC record PMID 20550934; Crossref metadata for 10.1016/j.cell.2010.05.018; OSTI 1020213; PubPeer (via API and web search).
Suggested correction
- In the Summary (abstract), replace "G3249A and G3244A" with "G3249A and G3242A", so that the sentence reads: "Finally, we have demonstrated that the mitochondrial pathogenic G3249A and G3242A mutations interfere with key interactions for sequence recognition, eliminating termination."
- Consider aligning the strength of the wording with the Results and Conclusions. G3249A abolishes specific binding and leaves only residual termination, while G3242A barely affects binding but strongly reduces termination. "Severely impairing termination" would match both. The same applies to "eliminate" for G3242A in the Introduction.
- Optionally, mention in the abstract that G3244A, which makes no contact with MTERF1, has a moderate effect on termination, because the body presents this as an unexpected finding.
- Because the abstract propagates through PubMed and indexing services, a published correction notice linked to the article record would be more effective than a silent change to the online text.
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC2887341
(Cell 2010; doi:10.1016/j.cell.2010.05.018), and redraw fig1 in this file's folder.
Run: python recompute.py.txt (needs matplotlib)
There is no arithmetic to redo: the error is a mismatch between the Summary
(abstract) and the body text in which pathogenic mtDNA mutation is named as the
second one that disrupts sequence recognition and eliminates termination. This
script therefore
1. counts every mention of the leu-tRNA mutations (G3242A, A3243G, A3243T,
G3244A, G3249A) in each section,
2. extracts the pair of mutations each section names as eliminating or
strongly impairing termination,
3. tabulates what the body says about G3242A and G3244A (structural contact,
effect on binding, effect on termination), and
4. draws fig1.
The paragraphs used are copied verbatim below from the article as archived in
PubMed Central (author manuscript). If the repository tool src/show.py is
reachable, the script also re-reads the archived full text, checks that every
excerpt occurs in it verbatim, and repeats the counts on the whole article.
"""
import os
import re
import subprocess
import collections
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
SECTIONS = collections.OrderedDict()
SECTIONS["Abstract (Summary)"] = (
"Defects in mitochondrial gene expression are associated with aging and disease. Mterf proteins "
"have been implicated in modulating transcription, replication and protein synthesis. We have solved "
"the structure of a member of this family, the human mitochondrial transcriptional terminator MTERF1, "
"bound to dsDNA containing the termination sequence. The structure indicates that upon sequence "
"recognition MTERF1 unwinds the DNA molecule, promoting eversion of three nucleotides. Base flipping "
"is critical for stable binding and transcriptional termination. Additional structural and biochemical "
"results provide insight into the DNA binding mechanism and explain how MTERF1 recognizes its target "
"sequence. Finally, we have demonstrated that the mitochondrial pathogenic G3249A and G3244A mutations "
"interfere with key interactions for sequence recognition, eliminating termination. Our results provide "
"insight into the role of mterf proteins and suggest a link between mitochondrial disease and the "
"regulation of mitochondrial transcription.")
SECTIONS["Introduction"] = (
"Interestingly, the pathogenic A3243G MELAS mutation was shown to decrease the ability of MTERF1 to "
"terminate transcription in vitro (Hess et al., 1991) and yet no obvious transcriptional alterations "
"are observed in cells carrying this MELAS mutation (Chomyn et al. 1992). "
"We have shown that two mutations, G3249A that causes a variant of Kearns-Sayre syndrome (Seneca et al., "
"2001) and G3242A that is associated with an uncharacterized mitochondrial disorder (Mimaki et al., 2009), "
"interfere with protein-DNA interactions that are key for sequence recognition by MTERF1 and as a "
"consequence eliminate its transcriptional termination activity.")
SECTIONS["Results: Pathogenic mutations (binding)"] = (
"It is important to note that the pathogenic effects of these mutations might be simply due to an "
"alteration on the leu-tRNA structure, and this appears indeed to be the case for the A3243G mutation "
"(Sasarman et al., 2008). A3243G is frequently identified in MELAS patients and has been previously "
"shown to slightly reduce MTERF1 binding (Hess et al., 1991) and proposed to interfere in vitro with "
"transcriptional termination (Hess et al., 1991; Chomyn et al., 1992). "
"Our measurements confirm that A3243G results in slightly decreased binding (Figure 6A and 3SE). "
"A similar binding affinity was observed for the A3243T mutation (Figure 6A). "
"All remaining mutations would not be expected to severely conflict with MTERF1 sequence recognition, "
"except for G3242A and G3249A. G3242 interacts in the structure with Arg251, while G3249 forms a double "
"hydrogen bond with Arg387 (Figure 6B). Both of these mutations would eliminate a guanine-arginine "
"interaction: the hydrogen bond between O6 of the guanine and the amino group of the arginine could "
"not be formed in the mutant DNA. Consistent with the mild decrease in binding observed in the R251A "
"mutant, G3242A resulted in weak or no effect on binding. However, the G3249A mutation appears to "
"completely eliminate specific DNA binding (Figure 6A and 3SE), consistent with the effect of the "
"R387A mutation.")
SECTIONS["Results: Pathogenic mutations (termination)"] = (
"As expected, only residual termination is observed in both orientations with the G3249A mutation. "
"In addition, consistent with what was observed with the R251A mutant, the G3242A mutation, while not "
"substantially altering binding, results nevertheless in a strong decrease in termination. The A3243G "
"mutation moderately reduces termination, while termination on A3243T is only marginally weaker than "
"for the wt sequence. The reduction in termination observed for A3243G is consistent with what was "
"previously reported (Hess et al., 1991). Unexpectedly, one additional mutation appears to moderately "
"affect termination: G3244A, while not affecting DNA binding, results in a moderate but reproducible "
"effect on termination. Interestingly, no specific interaction is observed in the crystal structure "
"with any of the nucleotides in this base pair. However, it is adjacent to the extrahelical A:T base "
"pair and displays a high base pair buckle (see Experimental Procedures). It is possible that the "
"strength of the G:C base pair in the wt sequence is important to maintain this conformation and that "
"the G3244A mutation thus leads to a slight destabilization of the bound MTERF1 that is sufficient to "
"make it more readily displaced by the RNA polymerase.")
SECTIONS["Conclusions"] = (
"An important conclusion derived from the crystal structure is that two pathogenic mutations that "
"interfere with arginine-guanine interactions involved in sequence recognition by MTERF1 result in a "
"strong impairment in the ability of MTERF1 to terminate transcription. G3242A does not substantially "
"affect binding, but very strongly reduces the ability of MTERF1 to terminate transcription, while "
"G3249A appears to completely eliminate specific binding by MTERF1 and consequently severely impairs "
"termination. Their strong effect on termination suggests that the pathogenic effects of the G3249A "
"and G3242A MtDNA mutations might be at least partially mediated by interfering with the activity of "
"MTERF1 at the leu-tRNA site. The G3249A mutation leads to the development of a variant of "
"Kearns-Sayre syndrome (Seneca et al., 2001), a mitochondrial myopathy, while the G3242A mutation has "
"been associated with an uncharacterized mitochondrial disorder (Mimaki et al., 2009). "
"Further experiments will be needed to address whether the G3249A and G3242A mutations result in "
"transcriptional alterations in vivo and if this effect is sufficient to explain their clinical phenotype.")
FIG6_LEGEND = ("The DNA residues involved in arginine guanine interactions with MTERF1 are indicated "
"by an asterisk.")
MUT = re.compile(r"\b[GA]32(?:42|43|44|49)[AGT]\b")
ORDER = ["G3242A", "A3243G", "A3243T", "G3244A", "G3249A"]
def counts(text):
return collections.Counter(MUT.findall(text))
def sentences(text):
return [s.strip() for s in re.split(r"(?<=\.)\s+(?=[A-Z])", text) if s.strip()]
# ------------------------------------------------- 1. mentions per section
print("=== 1. Mentions of each leu-tRNA mutation, by section (verbatim excerpts) ===")
print(f" {'section':45s} " + " ".join(f"{m:>7s}" for m in ORDER))
EXC = {}
for sec, txt in SECTIONS.items():
c = counts(txt)
EXC[sec] = c
print(f" {sec:45s} " + " ".join(f"{c.get(m, 0):7d}" for m in ORDER))
# ------------------------------------- 2. which pair each section names
print("\n=== 2. The pair each section names as disrupting recognition / eliminating termination ===")
PAIR_PATTERNS = [
("Abstract (Summary)", r"pathogenic (G\d{4}A) and (G\d{4}A) mutations interfere"),
("Introduction", r"two mutations, (G\d{4}A) that causes.*?and (G\d{4}A) that is associated"),
("Results: Pathogenic mutations (binding)", r"except for (G\d{4}A) and (G\d{4}A)\."),
("Conclusions", r"pathogenic effects of the (G\d{4}A) and (G\d{4}A) MtDNA mutations"),
]
PAIRS = {}
for sec, pat in PAIR_PATTERNS:
m = re.search(pat, SECTIONS[sec])
PAIRS[sec] = frozenset(m.groups())
print(f" {sec:45s} {' + '.join(sorted(PAIRS[sec], reverse=True))}")
body = {p for s, p in PAIRS.items() if s != "Abstract (Summary)"}
print(f" body sections agree with each other: {len(body) == 1}")
print(f" abstract pair equals body pair: {PAIRS['Abstract (Summary)'] in body}")
diff = PAIRS["Abstract (Summary)"] ^ next(iter(body))
print(f" mutations that differ: {sorted(diff)} (one digit apart: "
f"{sum(a != b for a, b in zip(*sorted(diff)))} character differs)")
print(f" G3242A mentions in the abstract: {EXC['Abstract (Summary)'].get('G3242A', 0)}; "
f"G3244A mentions in Introduction + Conclusions: "
f"{EXC['Introduction'].get('G3244A', 0) + EXC['Conclusions'].get('G3244A', 0)}")
# ------------------------------------- 3. what the body says about each
print("\n=== 3. What the body says about G3242A, G3244A and G3249A ===")
ALL = " ".join(v for k, v in SECTIONS.items() if k != "Abstract (Summary)")
KEYS = {
"G3242A": ["G3242 interacts", "G3242A resulted", "the G3242A mutation, while", "G3242A does not"],
"G3244A": ["G3244A, while", "no specific interaction", "the G3244A mutation thus"],
"G3249A": ["G3249 forms", "the G3249A mutation appears", "residual termination", "G3249A appears"],
}
for mut, keys in KEYS.items():
print(f" {mut}:")
for s in sentences(ALL):
if any(k in s for k in keys):
print(" -", textwrap.shorten(s, 230, placeholder=" ..."))
SUMMARY = { # attribute -> (G3242A, G3244A, G3249A), each a verbatim phrase from the body
"contact in the crystal structure": ("G3242 interacts in the structure with Arg251",
"no specific interaction is observed",
"G3249 forms a double hydrogen bond with Arg387"),
"effect on DNA binding": ("weak or no effect on binding", "not affecting DNA binding",
"completely eliminate specific DNA binding"),
"effect on termination": ("a strong decrease in termination",
"a moderate but reproducible effect on termination",
"only residual termination"),
}
print("\n attribute | G3242A | G3244A | G3249A")
for a, vals in SUMMARY.items():
for v in vals:
assert v in ALL, v
print(f" {a:33s} | " + " | ".join(vals))
# ------------------------- 4. optional cross-check against the archived full text
print("\n=== 4. Cross-check against the archived full text (src/show.py) ===")
FULLCOUNTS = None
show = os.path.join(HERE, "..", "..", "..", "src", "show.py")
py = "/home/ubuntu/papers-data/venv/bin/python"
if os.path.exists(show) and os.path.exists(py):
out = subprocess.run([py, show, "PMC2887341", "--full"], capture_output=True, text=True,
cwd=os.path.abspath(os.path.join(HERE, "..", "..", ".."))).stdout
flat = re.sub(r"\s+", " ", out)
excerpts = [s for t in SECTIONS.values() for s in sentences(t)] + [FIG6_LEGEND]
missing = [s for s in excerpts if s not in flat]
print(f" excerpt sentences found verbatim: {len(excerpts) - len(missing)} of {len(excerpts)}")
for s in missing:
print(" NOT FOUND:", s[:120])
sec = "header"
FULLCOUNTS = collections.defaultdict(collections.Counter)
for line in out.splitlines():
if line.startswith("ABSTRACT:"):
sec = "Abstract"
elif line.startswith("## "):
sec = line[3:].strip()
elif line.startswith("BACK MATTER"):
sec = "Back matter (figure legends)"
FULLCOUNTS[sec].update(MUT.findall(line))
print(" mentions by section in the whole article:")
for s, c in FULLCOUNTS.items():
if c:
print(f" {s[:60]:60s} " + ", ".join(f"{m} x{c[m]}" for m in ORDER if c[m]))
tot = sum(FULLCOUNTS.values(), collections.Counter())
print(" whole article: " + ", ".join(f"{m} x{tot[m]}" for m in ORDER))
else:
print(" src/show.py not reachable; skipped (the verbatim excerpts above were used).")
# ================================================================ figure
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, EDGE = "#eb6834", "#2a78d6", "#c9c7c1"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f" wrote {stem}.svg and {stem}.png")
print("\n=== Figure ===")
fig = plt.figure(figsize=(7.2, 7.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
R = fig.canvas.get_renderer()
def panel(x, y, w, h, title, accent):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc="white", ec=EDGE, lw=1.2))
ax.add_patch(FancyBboxPatch((x, y + h - 0.9), w, 0.9, boxstyle="square,pad=0",
fc=accent, ec="none"))
ax.text(x + 1.8, y + h - 2.2, title, fontsize=11, fontweight="bold", va="top")
def runs(x, y, parts, size=10.5):
"""Draw text segments on one line; return the extent of each segment in data units.
A part is (text, kwargs) or (text, kwargs, gap), gap being an x shift before it."""
ext = []
for part in parts:
txt, kw = part[0], part[1]
x += part[2] if len(part) > 2 else 0.0
t = ax.text(x, y, txt, fontsize=size, va="top", **kw)
bb = t.get_window_extent(R).transformed(ax.transData.inverted())
ext.append(bb)
x = bb.x1
return ext
def lines(x, y, rows, step=3.3, size=10.5):
out = []
for i, r in enumerate(rows):
out.append(runs(x, y - i * step, r if isinstance(r, list) else [(r, {})], size))
return out
HO = dict(color=ORANGE, fontweight="bold")
HB = dict(color=BLUE, fontweight="bold")
SEC = dict(color=INK2)
# --- Abstract panel (top)
panel(2, 76, 96, 22, "Abstract (Summary), final results sentence", ORANGE)
lines(3.8, 92.2, [
"\u201cFinally, we have demonstrated that the mitochondrial pathogenic",
[("G3249A and ", {}), ("G3244A", HO, -0.55), (" mutations interfere with key interactions for", {})],
"sequence recognition, eliminating termination.\u201d",
])
miss = runs(3.8, 81.6, [("G3242A: not mentioned in the abstract", SEC)], size=10)[0]
pair = runs(55.0, 81.6, [("Pair named: G3249A + ", SEC), ("G3244A", HO, -0.2)], size=10)
g3244_abs = pair[1]
# --- Body panel, G3242A (bottom left)
panel(2, 8, 47, 48, "Body on G3242A", BLUE)
lines(3.8, 50.5, [
[("Introduction", dict(fontweight="bold"))],
"“two mutations, G3249A … and",
[("G3242A", HB), (" … eliminate its", {})],
"transcriptional termination activity”",
[("Results", dict(fontweight="bold"))],
"“G3242 interacts in the structure",
"with Arg251”",
"“the G3242A mutation, while not",
"substantially altering binding, results",
"nevertheless in a strong decrease in",
"termination”",
[("Conclusions", dict(fontweight="bold"))],
"“the G3249A and G3242A MtDNA",
"mutations”; G3242A named 4 times",
], step=2.95, size=10)
# --- Body panel, G3244A (bottom right)
panel(51, 8, 47, 48, "Body on G3244A", BLUE)
lines(52.8, 50.5, [
[("Results (its only section)", dict(fontweight="bold"))],
"“Unexpectedly, one additional",
"mutation appears to moderately affect",
[("termination: ", {}), ("G3244A", HB), (", while not", {})],
"affecting DNA binding, results in a",
"moderate but reproducible effect on",
"termination.”",
"“no specific interaction is observed",
"in the crystal structure with any of",
"the nucleotides in this base pair”",
[("Introduction, Conclusions", dict(fontweight="bold"))],
"not mentioned",
], step=2.95, size=10)
# --- connectors
cx = (g3244_abs.x0 + g3244_abs.x1) / 2
ax.add_patch(FancyArrowPatch((cx, 75.6), (cx, 56.4), arrowstyle="-|>",
mutation_scale=14, lw=2, color=INK))
ax.text(cx - 2.0, 66.0, "Conflict: the abstract says a recognition\n"
"contact is lost and termination is\neliminated; the body reports no contact\n"
"and a moderate effect", fontsize=10, ha="right", va="center", color=INK)
mx = 12.0
ax.add_patch(FancyArrowPatch((mx, 75.6), (mx, 56.4), arrowstyle="-|>", mutation_scale=14, lw=2,
color=INK2, linestyle=(0, (3, 2))))
ax.text(mx + 1.5, 66.0, "the body's second\nmutation, G3242A", fontsize=10,
ha="left", va="center", color=INK2)
ax.text(2.0, 4.6, "Orange: as printed in the abstract. Blue: the body of the same article "
"(Introduction,", fontsize=10, ha="left", va="center", color=INK2)
ax.text(2.0, 1.8, "Results, Conclusions). All text quoted verbatim from the PMC-archived article.",
fontsize=10, ha="left", va="center", color=INK2)
save(fig, "fig1")
Abstract says metformin upregulates intestinal G6PC; the paper's own Results report a decrease
Cell Rep (2026) · doi:10.1016/j.celrep.2026.117791 · PMC13560472
Summary
The Abstract, the "In brief" summary and one Results subsection heading state that high metformin concentrations "upregulate" G6PC expression in intestinal epithelial cells (IECs), in the Abstract's words "to prevent glucose release". Every data statement in the paper reports the opposite: G6PC protein is reduced in IECs of metformin-treated HFD-fed mice (Figures 5E/5G) and in metformin-treated or AMPK-overexpressing primary IECs (Figures 5M, 6A–6C), G6pc mRNA is suppressed (Figure 6D), and the Figure 6 title, the Results conclusion and the Discussion all say G6PC goes down; of 13 statements giving a direction for this effect, the 3 summary-level ones say "up" and the 10 in the body say "down". The paper's own metabolite data (glucose 6-phosphate up 5-fold, glucose down) and its G6PC overexpression experiment fit a decrease, not an increase. The data and conclusions stand, but the most-read parts of the article give the first step of its intestinal mechanism the wrong sign.
What the paper reports
Statements that give an increase.
Abstract (Summary):
Here, we showed that high metformin concentrations upregulate G6PC expression through AMP-activated protein kinase (AMPK) activation to prevent glucose release in intestinal epithelial cells (IECs).
In brief (the opening clause, which names the authors, is omitted here):
in intestinal epithelial cells (IECs), high metformin concentrations upregulate G6PC expression to prevent glucose release and inhibit mitochondrial activity, leading to increased glucose utilization and lactate overproduction
Results, subsection heading:
Upregulation of G6PC expression by metformin increases glycolysis in IECs
Statements that give a decrease.
Results, subsection "Inhibition of mitochondrial respiration leads to increased glucose uptake and glycolysis in IECs":
However, it is important to note that we found that metformin treatment significantly reduced G6PC protein levels in IECs of HFD-fed mice (Figures 5E and 5G).
Results, the paragraph directly under the "Upregulation of G6PC expression..." heading:
Metformin treatment decreased G6PC protein levels in IECs of HFD-fed mice (Figures 5E and 5G), and treatment with high metformin concentration (1 mM) activated AMPK or overexpression of functional AMPKα1β1γ1 complex significantly decreased G6PC protein levels in primary IECs (Figures 5M and 6A–6C). Metformin treatment suppressed the mRNA levels of G6pc and Pck1 (Figure 6D).
The same paragraph, concluding:
Collectively, decreased G6PC expression by metformin leads to the accumulation of glucose 6-phosphate and augmentation of glycolysis in IECs.
Figure 6 legend, title:
Reduction of G6PC expression by metformin increases lactate production in primary epithelial cells
Discussion, paragraph 3:
we found that metformin treatment significantly decreased G6PC protein levels in the IECs of HFD-fed mice and in primary IECs; this will decrease glucose release from IECs because G6PC removes the phosphate group from glucose 6-phosphate and glucose will be then released from the cells with the bidirectional function of glucose transporters.
These data indicate that a decrease in G6PC protein by metformin leads to increased glycolysis in IECs.
Discussion, summary paragraph:
High metformin concentrations in the intestine downregulate G6PC expression to stimulate glycolysis inside IECs, and with inhibited mitochondrial complex 1 activity, leading to increased glucose utilization through glycolysis and lactate release.
Supporting experiments described in the Results.
Compared to IECs with expression of adenoviral GFP controls, overexpression of G6PC led to more than 6-fold reduction of metformin-stimulated accumulation of fluorescent glucose analog 2-NDBG in IECs (Figures 6F and 6G). Overexpression of G6PC also led to a 75% reduction of lactate levels in metformin-treated IECs (Figure 6H).
found that metformin treatment led to a 5-fold increase in glucose 6-phosphate levels; however, with decreased glucose levels in IECs (Figures 6I and 6J), resulting in drastically increased ratio of glucose 6-phosphate/glucose (Figure 6K).
All quotes above were checked verbatim against the PMC-archived article text.
What the paper's own numbers show
Method. This is a direction-of-effect contradiction, so there are no test statistics to recompute. I made a complete inventory instead. I searched the full archived text (abstract, In brief, headings, body and figure legends) for every mention of G6PC or G6pc. I kept each statement that gives a direction for metformin's effect on G6PC in IECs and recorded where it sits and which way it points. I first did this with a regex scan, then read each hit by hand; the hand count adds the two mRNA statements ("suppressed", "inhibition of the expression") that the regex missed and drops one regex false positive. I then checked whether the paper's own reported metabolite changes (Figures 6I–6K) and its G6PC overexpression results (Figures 6E–6H) fit an increase or a decrease. G6PC catalyses glucose 6-phosphate + H₂O → glucose + Pᵢ, the last step before glucose can leave the cell. All of this is in recompute.py.txt, which also re-checks every quote against the archived text.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Direction, metformin → G6PC in IECs | Abstract | "upregulate G6PC expression ... to prevent glucose release" | Up; contradicts every data statement |
| Direction, metformin → G6PC in IECs | In brief | "upregulate G6PC expression to prevent glucose release" | Up; contradicts every data statement |
| Direction, metformin → G6PC in IECs | Results subsection heading | "Upregulation of G6PC expression by metformin increases glycolysis" | Up; the paragraph beneath it reports a decrease |
| G6PC protein, IECs of HFD-fed mice | Results; Figs 5E, 5G | "significantly reduced"; "decreased" | Down |
| G6PC protein, primary IECs, 1 mM metformin or AMPKα1β1γ1 overexpression | Results; Figs 5M, 6A–6C | "significantly decreased" | Down |
| G6pc mRNA, primary IECs | Results; Fig 6D | "suppressed" | Down |
| Mechanistic conclusion | Results; Fig 6 title; Discussion (×3) | "decreased G6PC expression", "Reduction of G6PC expression", "a decrease in G6PC protein", "downregulate G6PC expression" | Down |
| Tally of directional statements | Whole archived text | n/a | 13 statements: 3 up (all in Abstract, In brief, one heading), 10 down (all in Results text, Fig 6 legend, Discussion) |
| Statements citing a data figure | Results | n/a | 4 of 4 point down; none points up |
| G6P/glucose ratio | Results; Figs 6I–6K | G6P 5-fold up, glucose decreased, ratio "drastically increased" | Ratio fold = 5 / (glucose fold) > 5 for any fall in glucose: substrate up, product down, which fits lower G6PC activity |
| G6PC overexpression with metformin | Results; Figs 6F–6H | 2-NDBG accumulation down > 6-fold; lactate down 75% | Down to < 16.7% and 25% of control; restoring G6PC blunts metformin's effect, as a metformin-induced decrease predicts |
No experiment in the paper reports metformin raising G6PC in IECs. The one increase in G6pc the paper reports is in the liver of intestinal AMPKα1/2 knockout mice (Figure 4F), a different tissue and a genetic manipulation, not a metformin effect in IECs.
Figure
Figure 1. Every statement in the archived article that gives a direction for metformin's effect on G6PC in IECs, grouped by where it appears. Orange, left: the three statements that say "upregulate"/"Upregulation" (Abstract, In brief, one Results subsection heading). Blue, right: the statements that say G6PC is reduced, decreased, suppressed or downregulated (five are shown; five more in the Results and Discussion are listed in the table above and in recompute.py.txt). All of the data-backed statements (Figures 5E/5G, 5M, 6A–6D) are on the right. The connector marks the conflict: the two groups give opposite directions for the same effect, in the same cells, at the same high metformin concentrations.
Figure 2. Why the data fit a decrease. The top row shows the reaction G6PC catalyses (glucose 6-phosphate to free glucose, which can then leave the cell through bidirectional glucose transporters) and the branch to glycolysis and lactate. The orange box lists what an increase in G6PC, as the Abstract says, would predict. The blue box lists what a decrease would predict. The grey box lists what the Results text reports: glucose 6-phosphate up 5-fold with glucose down (Figures 6I–6K), G6PC overexpression cutting metformin-stimulated 2-NDBG accumulation more than 6-fold and lactate by 75% (Figures 6F–6H), and G6PC protein and mRNA down. Each reported finding matches the decrease. The effect sizes are as stated in the Results text; nothing is read from the paper's images.
Why it matters
The change in G6PC is the first step of the intestinal half of the paper's model: high metformin concentrations in IECs lower G6PC through AMPK, glucose 6-phosphate builds up and is routed into glycolysis, and the extra lactate goes through the portal vein to the liver. The Abstract and the In brief are the parts of the article that are most widely read, indexed and quoted, and they give this step the wrong sign. A reader who sees only them learns that metformin raises intestinal G6PC, the opposite of what the paper measured. Such a reader might cite the paper as evidence for G6PC induction, or design experiments on that premise. The Abstract sentence is also internally inconsistent: G6PC frees glucose for export, so raising it would promote glucose release, not "prevent" it. A careful reader may notice that, but a reader skimming the abstract or an automated summary may not. The same wrong word in a Results subsection heading means even a reader who opens the Results may first meet "Upregulation" before the text beneath it reports a decrease.
The underlying data, figures and conclusions are unaffected. Every data statement, the Figure 6 title and the Discussion consistently report a decrease, and the overall claim that metformin's actions in the intestine and liver together improve hyperglycemia does not depend on this wording. The problem is confined to the three summary-level statements.
Explanations considered
A different model, tissue, dose or time point. The Abstract specifies "high metformin concentrations" in IECs. That matches the conditions under which the body reports a decrease: 1–2 mM metformin in primary IECs (Figures 5M, 6A–6D) and oral metformin in IECs of HFD-fed mice (Figures 5E/5G). The body reports no IEC condition in which G6PC rises. This does not account for the contradiction.
The liver G6pc result. The Results report "significantly higher mRNA levels of rate-limited gluconeogenic genes G6pc and Pck1 in the liver" of intestinal AMPKα1/2 knockout mice (Figure 4F). That is a knockout effect in the liver, compared with floxed controls, not metformin acting on IECs. It cannot be the source of a statement about high metformin concentrations in IECs. Does not account for it.
G6PC overexpression. Figures 6E–6H use adenoviral overexpression of G6PC, an increase made by the investigators, not by metformin. In those cells metformin's effects on 2-NDBG accumulation and lactate were reduced, which supports the decrease model. Does not account for it.
"Upregulate" in some other sense (e.g. upregulated activity, or regulation in general). The Abstract says "upregulate G6PC expression", and the body reports expression (protein and mRNA) going down. No passage reports increased G6PC activity; the G6P/glucose data (Figures 6I–6K) point to lower activity. Does not account for it.
Text-conversion or manuscript-stage artefact. The PMC record is the NIH author manuscript, not the typeset version. However, the publisher-supplied PubMed record (PMID 42555357) carries the same "upregulate G6PC expression" Abstract sentence verbatim, so the wording is in the publisher's own abstract metadata as well as the manuscript. A single-word conversion error would not change the same verb in three separate places while leaving ten others intact. Does not account for it.
A wording slip rather than a data error. This is the most plausible reading of the evidence: the data and the detailed text agree with each other, and only three summary statements disagree. It does not reconcile the two directions, but it does mean the correction needed is to wording, not to results.
Status
As of 2026-10-23:
- Version of record. The PMC record (PMC13560472) is the NIH author manuscript ("published before final editing"; Europe PMC metadata: NIHPA Author Manuscript, not the version of record). The publisher's full text could not be retrieved: doi.org and linkinghub.elsevier.com returned only a redirect stub, and the cell.com and ScienceDirect pages returned HTTP 403. The publisher-deposited PubMed record (PMID 42555357; Cell Rep 45(8):117791; e-published 2026-08-05; status "aheadofprint") was re-fetched on 2026-10-23, and its Abstract contains the "upregulate G6PC expression" sentence verbatim. Whether the typeset Results heading and Discussion match the manuscript could not be confirmed directly.
- Supplement. No supplement is archived with the PMC manuscript (has-supplement = no). The publisher's supplementary file could not be reached (HTTP 403). A supplement could not settle a wording contradiction of this kind; the main-text figures already show the decrease.
- Corrections and retractions. None found.
known.py: no Crossref updates, no PMC or Europe PMC notices. Crossref record re-checked on 2026-10-23: noupdate-to,updated-byor relations. The PubMed record carries no CommentsCorrections entries. - PubPeer and other discussion. PubPeer could not be queried directly (its search and API returned HTTP 403). Earlier web searches by DOI and by title combined with "correction", "erratum" and "PubPeer" found nothing relevant. Web searching was not available for a final re-check on 2026-10-23.
Sources: PMC-archived article text (via src/show.py PMC13560472); PubMed efetch for PMID 42555357; Crossref REST API for doi:10.1016/j.celrep.2026.117791; Europe PMC metadata; src/known.py.
Suggested correction
Change "upregulate" to "downregulate" in the Abstract and the In brief, and "Upregulation" to "Downregulation" in the Results subsection heading, so that they match the Results, Figure 6 and the Discussion:
- Abstract: "Here, we showed that high metformin concentrations downregulate G6PC expression through AMP-activated protein kinase (AMPK) activation to prevent glucose release in intestinal epithelial cells (IECs)."
- In brief: "...high metformin concentrations downregulate G6PC expression to prevent glucose release and inhibit mitochondrial activity..."
- Results heading: "Downregulation of G6PC expression by metformin increases glycolysis in IECs"
The same changes would need to be made to the abstract held by PubMed and other indexers. No data, figure or statistical result needs to change.
Recomputation script (python recompute.py.txt)
"""Inventory of every statement in PMC13560472 (Cell Rep 2026; doi:10.1016/j.celrep.2026.117791)
that gives the direction of metformin's effect on G6PC in intestinal epithelial cells (IECs),
plus the arithmetic implied by the paper's own Figure 6I-6K description.
Redraws fig1/fig2 (SVG + PNG at 200 dpi) in this file's folder.
Run: python recompute.py.txt (needs matplotlib)
If run from the repository root (/home/ubuntu/papers) with src/show.py available, it also
re-checks every quoted passage verbatim against the PMC-archived article text.
"""
import os
import subprocess
import sys
import textwrap
from collections import Counter
HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC13560472"
# ---------------------------------------------------------------- inputs (verbatim quotes)
# (id, location, part of paper, direction stated for metformin -> G6PC, verbatim excerpt)
STATEMENTS = [
("A1", "Abstract (Summary)", "summary", "UP",
"high metformin concentrations upregulate G6PC expression through AMP-activated protein "
"kinase (AMPK) activation to prevent glucose release in intestinal epithelial cells (IECs)"),
("A2", "In brief", "summary", "UP",
"high metformin concentrations upregulate G6PC expression to prevent glucose release"),
("A3", "Results, subsection heading", "summary", "UP",
"Upregulation of G6PC expression by metformin increases glycolysis in IECs"),
("B1", "Results, 'Inhibition of mitochondrial respiration...' (Figs 5E, 5G)", "body", "DOWN",
"metformin treatment significantly reduced G6PC protein levels in IECs of HFD-fed mice "
"(Figures 5E and 5G)"),
("B2", "Results, under the 'Upregulation of G6PC...' heading (Figs 5E, 5G)", "body", "DOWN",
"Metformin treatment decreased G6PC protein levels in IECs of HFD-fed mice (Figures 5E and 5G)"),
("B3", "Results, same paragraph (Figs 5M, 6A-6C)", "body", "DOWN",
"significantly decreased G6PC protein levels in primary IECs (Figures 5M and 6A–6C)"),
("B4", "Results, same paragraph (Fig 6D)", "body", "DOWN",
"Metformin treatment suppressed the mRNA levels of G6pc and Pck1 (Figure 6D)"),
("B5", "Results, same paragraph (conclusion)", "body", "DOWN",
"Collectively, decreased G6PC expression by metformin leads to the accumulation of glucose "
"6-phosphate and augmentation of glycolysis in IECs."),
("B6", "Results, same paragraph", "body", "DOWN",
"the inhibition of the expression of G6pc and Pck1 genes by metformin would block the use of "
"lactate/pyruvate as substrates for gluconeogenesis"),
("B7", "Figure 6 legend, title", "body", "DOWN",
"Reduction of G6PC expression by metformin increases lactate production in primary epithelial cells"),
("B8", "Discussion, paragraph 3", "body", "DOWN",
"we found that metformin treatment significantly decreased G6PC protein levels in the IECs of "
"HFD-fed mice and in primary IECs; this will decrease glucose release from IECs"),
("B9", "Discussion, paragraph 3", "body", "DOWN",
"These data indicate that a decrease in G6PC protein by metformin leads to increased glycolysis in IECs."),
("B10", "Discussion, summary paragraph", "body", "DOWN",
"High metformin concentrations in the intestine downregulate G6PC expression to stimulate "
"glycolysis inside IECs"),
]
# Other passages used in the write-up (verbatim), checked the same way
OTHER_QUOTES = [
"overexpression of G6PC led to more than 6-fold reduction of metformin-stimulated accumulation "
"of fluorescent glucose analog 2-NDBG in IECs (Figures 6F and 6G)",
"Overexpression of G6PC also led to a 75% reduction of lactate levels in metformin-treated IECs (Figure 6H)",
"metformin treatment led to a 5-fold increase in glucose 6-phosphate levels; however, with decreased "
"glucose levels in IECs (Figures 6I and 6J), resulting in drastically increased ratio of glucose "
"6-phosphate/glucose (Figure 6K)",
"G6PC removes the phosphate group from glucose 6-phosphate and glucose will be then released from "
"the cells with the bidirectional function of glucose transporters",
"The above data demonstrate that high metformin concentrations can inhibit glucose production and "
"release in the IECs.",
"Our results show that high concentrations of metformin in the intestine can prevent glucose release "
"and stimulate glycolysis",
"The mRNA levels of G6pc, Pck1, and Gapdh in the liver of HFD-fed male mice",
]
# Figure 6I-6K, as described in the Results text
G6P_FOLD = 5.0 # "a 5-fold increase in glucose 6-phosphate levels"
GLUCOSE_DIRECTION = "decreased" # "with decreased glucose levels in IECs"
G6PC_OE_2NDBG_FOLD = 6.0 # "more than 6-fold reduction" of 2-NDBG accumulation
G6PC_OE_LACTATE_RED = 0.75 # "a 75% reduction of lactate levels"
# ---------------------------------------------------------------- 0. verbatim check (optional)
def norm(s):
return " ".join(s.replace("’", "'").split())
def archived_text():
repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
show = os.path.join(repo, "src", "show.py")
if not os.path.exists(show):
return None
r = subprocess.run([sys.executable, show, PMCID, "--full"], capture_output=True, text=True, cwd=repo)
return r.stdout if r.returncode == 0 else None
txt = archived_text()
print("=== 0. Verbatim check against the PMC-archived text ===")
if txt is None:
print("src/show.py not available; skipping (quotes were checked when the write-up was made)")
else:
t = norm(txt)
allq = [s[4] for s in STATEMENTS] + OTHER_QUOTES
ok = [norm(q) in t for q in allq]
for q, k in zip(allq, ok):
if not k:
print(" NOT FOUND:", q[:90])
print(f"{sum(ok)}/{len(ok)} quoted passages found verbatim")
# ---------------------------------------------------------------- 1. tally
print("\n=== 1. Direction stated for metformin's effect on G6PC in IECs ===")
for sid, loc, part, d, q in STATEMENTS:
print(f" {sid:4s} {d:4s} [{part:7s}] {loc}")
tally = Counter((s[2], s[3]) for s in STATEMENTS)
n_up = sum(1 for s in STATEMENTS if s[3] == "UP")
n_down = sum(1 for s in STATEMENTS if s[3] == "DOWN")
print(f"Total directional statements: {len(STATEMENTS)} (UP {n_up}, DOWN {n_down})")
print(f"UP statements located in Abstract / In brief / heading: {tally[('summary', 'UP')]} of {n_up}")
print(f"DOWN statements located in Results text / Fig 6 legend / Discussion: {tally[('body', 'DOWN')]} of {n_down}")
data_backed = [s for s in STATEMENTS if "Figure" in s[4]]
print(f"Quoted statements that cite a data figure panel: {len(data_backed)}; directions: "
f"{sorted(set(s[3] for s in data_backed))}")
# ---------------------------------------------------------------- 2. biochemistry consistency
print("\n=== 2. Consistency of Figs 6I-6K with the direction of G6PC ===")
print("G6PC reaction: glucose 6-phosphate + H2O -> glucose + Pi (last step before glucose exit)")
# ratio fold = (G6P fold) / (glucose fold); any glucose fold < 1 gives a ratio fold > G6P fold
print(f" ratio fold = {G6P_FOLD:.0f} / (glucose fold); glucose fold < 1 -> ratio fold > {G6P_FOLD:.0f}: "
f"{all(G6P_FOLD / g > G6P_FOLD for g in (0.999, 0.5, 0.1))}")
print(f"With glucose {GLUCOSE_DIRECTION}, the G6P/glucose ratio rises by more than {G6P_FOLD:.0f}-fold: "
"substrate builds up while product falls, the pattern expected when G6PC activity is LOWER.")
print(f"G6PC overexpression: 2-NDBG accumulation reduced >{G6PC_OE_2NDBG_FOLD:.0f}-fold "
f"(to <{100 / G6PC_OE_2NDBG_FOLD:.1f}% of GFP control); lactate reduced {G6PC_OE_LACTATE_RED:.0%} "
f"(to {100 * (1 - G6PC_OE_LACTATE_RED):.0f}% of control).")
print("Restoring G6PC blunting metformin's effect is what a G6PC decrease predicts; an increase predicts the opposite.")
# ---------------------------------------------------------------- 3. figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "svg.fonttype": "none",
"figure.facecolor": BG, "savefig.facecolor": BG})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), **kw)
plt.close(fig)
print(f"wrote {name}.svg and {name}.png")
def box(ax, x, y, w, h, color, lw=2, ls="-", fc=BG):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
linewidth=lw, edgecolor=color, facecolor=fc, linestyle=ls))
def entries(ax, x, y_top, width_chars, items, color, gap=0.016, line_h=0.0265):
y = y_top
for loc, quote in items:
ax.text(x, y, loc, color=color, fontsize=10, fontweight="bold", va="top")
y -= line_h
for ln in textwrap.wrap(quote, width_chars):
ax.text(x, y, ln, color=INK, fontsize=10, va="top")
y -= line_h
y -= gap
return y
# ----- Figure 1: summary statements vs body statements
fig = plt.figure(figsize=(7.2, 7.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.975, "Direction of metformin's effect on G6PC in intestinal epithelial cells",
fontsize=12, fontweight="bold", color=INK, va="top")
ax.text(0.03, 0.942, "Every statement in the archived article that gives a direction, grouped by where it appears",
fontsize=10, color=INK2, va="top")
L = (0.03, 0.14, 0.405, 0.765) # x, y, w, h
R = (0.555, 0.14, 0.415, 0.765)
box(ax, *L, ORANGE); box(ax, *R, BLUE)
ax.text(L[0] + 0.02, L[1] + L[3] - 0.02, f"INCREASE: {n_up} statements", color=ORANGE,
fontsize=11, fontweight="bold", va="top")
ax.text(L[0] + 0.02, L[1] + L[3] - 0.05, "Abstract, In brief, one Results heading", color=INK2,
fontsize=10, va="top")
ax.text(R[0] + 0.02, R[1] + R[3] - 0.02, f"DECREASE: {n_down} statements", color=BLUE,
fontsize=11, fontweight="bold", va="top")
ax.text(R[0] + 0.02, R[1] + R[3] - 0.05, "Results text, Figure 6 title, Discussion", color=INK2,
fontsize=10, va="top")
left_items = [
("Abstract", "“…high metformin concentrations upregulate G6PC expression through "
"… AMPK activation to prevent glucose release in … IECs”"),
("In brief", "“…high metformin concentrations upregulate G6PC expression to "
"prevent glucose release…”"),
("Results, subsection heading", "“Upregulation of G6PC expression by metformin "
"increases glycolysis in IECs”"),
]
right_items = [
("Results (Figs 5E, 5G)", "“metformin treatment significantly reduced G6PC protein "
"levels in IECs of HFD-fed mice”"),
("Results (Figs 5M, 6A–6C)", "“significantly decreased G6PC protein levels in "
"primary IECs”"),
("Results (Fig 6D)", "“Metformin treatment suppressed the mRNA levels of G6pc”"),
("Figure 6 title", "“Reduction of G6PC expression by metformin increases lactate "
"production”"),
("Discussion, summary", "“High metformin concentrations in the intestine "
"downregulate G6PC expression”"),
]
yl = entries(ax, L[0] + 0.02, L[1] + L[3] - 0.095, 37, left_items, ORANGE, gap=0.03)
for i, ln in enumerate(textwrap.wrap("The heading sits directly above a paragraph that reports a decrease "
"(Figs 5E/5G, 5M, 6A\u20136D).", 40)):
ax.text(L[0] + 0.02, yl - 0.01 - 0.0265 * i, ln, color=INK2, fontsize=10, va="top")
yend = entries(ax, R[0] + 0.02, R[1] + R[3] - 0.095, 36, right_items, BLUE, gap=0.014)
ax.text(R[0] + 0.02, yend - 0.002, f"+ {n_down - len(right_items)} more in Results and Discussion",
color=INK2, fontsize=10, va="top")
arr = FancyArrowPatch((L[0] + L[2] + 0.005, 0.60), (R[0] - 0.005, 0.60), arrowstyle="<->",
mutation_scale=14, linewidth=2, color=INK)
ax.add_patch(arr)
ax.text(0.495, 0.615, "opposite", ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
ax.text(0.495, 0.585, "direction", ha="center", va="top", fontsize=10, fontweight="bold", color=INK)
ax.text(0.03, 0.115, "Every data statement (Figs 5E/5G, 5M, 6A–6D) reports a decrease. No experiment in the "
"paper", fontsize=10, color=INK, va="top")
ax.text(0.03, 0.087, "reports metformin increasing G6PC.", fontsize=10, color=INK, va="top")
ax.text(0.03, 0.045, "Orange: summary wording (increase). Blue: wording that matches the paper's data (decrease).",
fontsize=10, color=INK2, va="top")
save(fig, "fig1")
# ----- Figure 2: what each direction predicts vs what Figs 6I-6K and 6E-6H report
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.975, "What each direction predicts, and what the paper measured",
fontsize=12, fontweight="bold", color=INK, va="top")
# reaction strip
y0 = 0.80
for x, lab in ((0.05, "Glucose\n6-phosphate"), (0.40, "Glucose\n(in IEC)"), (0.73, "Glucose\nreleased")):
box(ax, x, y0 - 0.045, 0.21, 0.10, INK2, lw=1.5)
ax.text(x + 0.105, y0 + 0.005, lab, ha="center", va="center", fontsize=10, color=INK)
for (xa, xb, lab) in ((0.265, 0.395, "G6PC"), (0.615, 0.725, "GLUT")):
ax.add_patch(FancyArrowPatch((xa, y0), (xb, y0), arrowstyle="-|>", mutation_scale=14,
linewidth=2, color=INK))
ax.text((xa + xb) / 2, y0 + 0.015, lab, ha="center", va="bottom", fontsize=10,
fontweight="bold", color=INK)
ax.add_patch(FancyArrowPatch((0.155, y0 - 0.05), (0.155, y0 - 0.115), arrowstyle="-|>",
mutation_scale=14, linewidth=2, color=INK))
ax.text(0.17, y0 - 0.085, "glycolysis → lactate", fontsize=10, color=INK, va="center")
rows = [
("If metformin INCREASED G6PC (Abstract)", ORANGE, [
"more G6P made into glucose; G6P/glucose falls",
"more glucose free to leave the cell, not less",
"extra G6PC should mimic metformin, not oppose it",
]),
("If metformin DECREASED G6PC (Results)", BLUE, [
"G6P builds up, feeds glycolysis; G6P/glucose rises",
"less glucose released (\u201cprevent glucose release\u201d)",
"extra G6PC should blunt metformin\u2019s effect",
]),
]
measured = [
f"G6P \u00d7{G6P_FOLD:.0f}, glucose down (Figs 6I, 6J)",
f"G6P/glucose up > {G6P_FOLD:.0f}-fold (Fig 6K)",
f"Extra G6PC: 2-NDBG down > {G6PC_OE_2NDBG_FOLD:.0f}-fold, lactate down "
f"{G6PC_OE_LACTATE_RED:.0%} (Figs 6F\u20136H)",
"G6PC protein down (Figs 5E/5G, 5M, 6A\u20136C)",
"G6pc mRNA down (Fig 6D)",
]
ax.text(0.03, 0.60, "Prediction", fontsize=10, color=INK2, fontweight="bold")
ax.text(0.625, 0.60, "Reported in the paper", fontsize=10, color=INK2, fontweight="bold")
yy = 0.555
for title, col, lines in rows:
box(ax, 0.03, yy - 0.215, 0.565, 0.235, col)
ax.text(0.045, yy, title, color=col, fontsize=10, fontweight="bold", va="top")
for i, ln in enumerate(lines):
ax.text(0.045, yy - 0.045 - 0.05 * i, "\u2022 " + ln, color=INK, fontsize=10, va="top")
yy -= 0.27
box(ax, 0.625, 0.065, 0.345, 0.51, INK2, lw=1.5)
ym = 0.545
for item in measured:
for ln in textwrap.wrap(item, 26):
ax.text(0.64, ym, ln, color=INK, fontsize=10, va="top")
ym -= 0.033
ym -= 0.014
ax.text(0.64, ym - 0.01, "Matches: DECREASE", color=BLUE, fontsize=10, fontweight="bold", va="top")
ax.add_patch(FancyArrowPatch((0.623, 0.17), (0.598, 0.17), arrowstyle="-|>", mutation_scale=16,
linewidth=2, color=BLUE))
ax.text(0.03, 0.035, "Effect sizes are as stated in the Results text; nothing is read from the paper's images.",
fontsize=10, color=INK2, va="top")
save(fig, "fig2")
IL-6 × leptin feeding interaction: reported F(1,11) = 0.563 gives p = 0.47, not 0.05
Cell Rep (2019) · doi:10.1016/j.celrep.2019.02.044 · PMC6418345
Summary
The Results report a two-way repeated-measures ANOVA on 6-h chow intake after co-injection of subthreshold leptin and IL-6 into the lateral parabrachial nucleus (lPBN; Figure 3J, n = 12 rats). They give the IL-6 × leptin interaction as "F(1, 11) = 0.563, p = 0.05". An F of 0.563 on (1, 11) degrees of freedom has an upper-tail p of 0.469. A p of 0.05 would take F = 4.84, about 8.6 times larger. The reported F and p therefore cannot both be correct. Taken at face value, the F shows no statistically significant interaction. This interaction is the only test of interaction reported for the paper's claim, repeated in the abstract, highlights, Discussion and conclusion, that IL-6 and leptin cooperate in the lPBN to reduce feeding. Every other F/p pair in the Results (15 of 16) is consistent with its recomputed p.
What the paper reports
Results, "Cooperative Interaction of Leptin and IL-6 on Food Intake Control":
Results indicated that IL-6 and leptin indeed interact to suppress food intake (two-way ANOVA: IL-6 alone: F(1, 11) = 0.21, p > 0.05; leptin alone F(1, 11) = 0.88, p > 0.05; interaction IL-6 × leptin F(1, 11) = 0.563, p = 0.05; Figure 3J).
Same paragraph, two sentences later:
BAT region temperature was unaffected (IL-6 alone: F(1, 11) = 4.524, p > 0.05; leptin alone F(1, 11) = 7.652, p < 0.05; interaction IL-6 × leptin F(1, 11) = 1.528, p > 0.05; Figure 3K).
Same paragraph, interpretation:
These results suggest that leptin and IL-6 can acutely act in concert, possibly by sharing a common downstream target, ultimately amplifying each other’s reduction of food intake.
Abstract (Summary):
Parabrachial IL-6 interacted with leptin to reduce feeding.
Highlights:
IL-6 interacts with leptin at the level of the lPBN to reduce food intake
Discussion, third paragraph:
In line with our hypothesis, injection of subthreshold doses of both ligands produced a clear reduction in feeding when co-applied to the lPBN indicating that IL-6 can amplify leptin’s anorexic effects at the level of the lPBN.
Discussion, concluding paragraph:
The latter point, along with the identified here cooperative interaction of exogenously applied IL-6 with leptin in the lPBN, highlights the emerging potential importance of the local circuit identified here for development of obesity therapeutics.
Figure 3 legend (panels J and K):
In a new cohort of 11-week-old male rats (n = 12) chow intake and BAT temperature were measured after intra-lPBN delivery of sub-threshold doses of IL-6 (0.1 μg) or leptin (0.1 μg) alone, or delivery of the two substances together into the lPBN.
two-way ANOVA (J and K) and one-way ANOVA (B–E, G, and I) were used for comparisons with Sidak’s post hoc test.
STAR Methods, microinjections, and Quantification and Statistical Analysis:
All injections were performed in a counterbalanced Latin square design, where each animal received each treatment.
p < 0.05 was considered statistically significant.
What the paper's own numbers show
Method. For every F statistic in the Results, which reports them in three sections (the lPBN leptin dose-response, Figure 3B–3I; the leptin/IL-6 interaction experiment, Figure 3J/3K; and the lPBN IL-6 knockdown, Figure 4G/4H), I computed the ANOVA p-value as the upper-tail probability P(Fdf1, df2 ≥ F) with scipy.stats.f.sf, using the F and degrees of freedom exactly as printed. With 12 rats each receiving all four treatments, F(1, 11) is the expected test for each effect in a 2 × 2 within-subject ANOVA. For df1 = 1, F equals t², so I also report the equivalent t and its one-sided p. All numbers below are printed by recompute.py.txt.
| quantity | location | reported | recomputed |
|---|---|---|---|
| IL-6 × leptin interaction, 6-h chow intake, F(1, 11) = 0.563 | Results, Fig 3J | p = 0.05 | p = 0.469 |
| F(1, 11) needed for p = 0.05 | derived | (implied by p = 0.05) | 4.844 (8.6 × the reported F) |
| F(1, 11) range whose p rounds to 0.05 (0.045 ≤ p < 0.055) | derived | n/a | 4.61 to 5.11 |
| Equivalent t(11) = √0.563; one-sided p | derived | n/a | t = 0.750; p = 0.234 |
| p for F = 0.563 with df1 = 1 and any df2 from 1 to 106 | derived | n/a | 0.45 to 0.59 |
| Smallest F (df1 = 1) that can reach p < 0.05 for any df2 | derived | n/a | 3.841 |
| Leptin dose, 6-h chow intake, F(2, 10) = 5.496 (printed "F (2, 10)") | Results, Fig 3B | p < 0.05 | p = 0.0245 (consistent) |
| Leptin dose, 12-h chow intake, F(2, 10) = 6.685 | Results, Fig 3C | p < 0.05 | p = 0.0143 (consistent) |
| Leptin dose, 24-h chow intake, F(2, 10) = 7.464 | Results, Fig 3D | p < 0.05 | p = 0.0104 (consistent) |
| Leptin dose, 24-h body weight, F(2, 12) = 7.55 | Results, Fig 3E | p < 0.01 | p = 0.00754 (consistent) |
| Leptin dose, core body temperature, F(2, 34) = 1.765 | Results, Fig 3G | p > 0.05 | p = 0.187 (consistent) |
| Leptin dose, locomotor activity, F(2, 34) = 3.493 | Results, Fig 3I | p < 0.05 | p = 0.0417 (consistent) |
| IL-6 main effect, chow intake, F(1, 11) = 0.21 | Results, Fig 3J | p > 0.05 | p = 0.656 (consistent) |
| Leptin main effect, chow intake, F(1, 11) = 0.88 | Results, Fig 3J | p > 0.05 | p = 0.368 (consistent) |
| IL-6 main effect, BAT temperature, F(1, 11) = 4.524 | Results, Fig 3K | p > 0.05 | p = 0.0569 (consistent) |
| Leptin main effect, BAT temperature, F(1, 11) = 7.652 | Results, Fig 3K | p < 0.05 | p = 0.0184 (consistent) |
| IL-6 × leptin interaction, BAT temperature, F(1, 11) = 1.528 | Results, Fig 3K | p > 0.05 | p = 0.242 (consistent) |
| Knockdown, interaction, F(17, 544) = 8.345 | Results, Fig 4G | p < 0.01 | p = 5.05 × 10-19 (consistent) |
| Knockdown, treatment, F(1, 32) = 7.548 | Results, Fig 4G | p < 0.05 | p = 0.00978 (consistent) |
| Knockdown, interaction, F(17, 544) = 4.944 | Results, Fig 4H | p < 0.01 | p = 5.28 × 10-10 (consistent) |
| Knockdown, treatment, F(1, 32) = 3.462 | Results, Fig 4H | p > 0.05 | p = 0.072 (consistent) |
Fifteen of the sixteen F/p pairs in the Results agree with the recomputation. The one that does not is the interaction on which the cooperative-effect claim rests. Its recomputed p is 9.4 times the printed value.
Figure
Figure 1. Each row is one F test from the Results paragraph on the leptin/IL-6 interaction experiment (Figure 3J, 6-h chow intake; Figure 3K, BAT temperature). Orange shows what the paper reports: a bound ("p > 0.05" or "p < 0.05") is drawn as the range it allows, and the one exact value (p = 0.05) as a dot. Blue dots are the upper-tail p-values recomputed from the printed F and degrees of freedom. The thin vertical line marks p = 0.05; the paper's own criterion is p < 0.05. Five recomputed values fall inside the range the paper states. For the IL-6 × leptin interaction on chow intake, F(1, 11) = 0.563 gives p = 0.469, not 0.05.
Figure 2. The density of the F distribution with (1, 11) degrees of freedom. The p-value of an ANOVA F test is the area to the right of the observed F. For the reported F = 0.563 (blue line) that area, shaded light blue, is 0.469. The reported p = 0.05 corresponds to the area to the right of F = 4.84 (orange line); that tail is too thin to see at this scale, and the arrow marks where it lies. The dashed line marks F = 5.63, a possible decimal-shift reading of the printed value, which would give p = 0.037. Nothing in the published material confirms that reading.
Why it matters
- The cooperative-effect claim rests on this one statistic. The leptin and IL-6 doses were chosen to be subthreshold, and neither main effect on 6-h chow intake was significant (p = 0.656 and 0.368). The claim that the two ligands "interact to suppress food intake" therefore depends entirely on the interaction term. With the printed F, the interaction has p = 0.469, so the data as reported do not show an interaction.
- The claim reaches the paper's summary statements. It appears in the abstract ("Parabrachial IL-6 interacted with leptin to reduce feeding"), in the highlights, in the Figure 3 title ("Cooperative Interaction of Leptin and IL-6 at the Level of the lPBN"), in the Discussion ("IL-6 can amplify leptin’s anorexic effects"), and in the concluding paragraph, where it is cited as relevant to obesity therapeutics. The Results also propose a possible mechanism (a shared downstream target, likely involving STAT3 phosphorylation).
- Even the printed p does not meet the paper's criterion. The Methods define significance as p < 0.05. As printed, p = 0.05 does not satisfy the paper's p < 0.05 criterion, so the text calls an interaction established at a printed p that the paper's own threshold would not count as significant.
- The paper's other main conclusions are not affected. The effects of lPBN IL-6 on food intake, body weight and BAT thermogenesis, and the effects of IL-6 knockdown, come from separate experiments and separate tests. The leptin dose-response F tests (Figure 3B–3I) and the Figure 4G/4H F tests all recompute consistently.
Explanations considered
- Different degrees of freedom. Does not account for it. With n = 12 rats in a within-subject (Latin square) design, F(1, 11) is the expected test. For F = 0.563 with df1 = 1, every denominator df from 1 to 106 gives p between 0.45 and 0.59. With df1 = 1, no F below 3.84 can reach p < 0.05 for any df2.
- A one-sided test. Does not account for it. The one-sided p for the equivalent t = 0.750 is 0.234, and the Methods describe no one-sided tests.
- The lower tail of F. Does not account for it. P(F ≤ 0.563) = 0.531, and the lower tail is not the ANOVA p-value.
- Rounding. Does not account for it. A p that rounds to 0.05 needs F between 4.61 and 5.11 on (1, 11) df. 0.563 is far outside that range.
- A decimal slip in F. Possible, but not verifiable. F = 5.63 would give p = 0.037, which is close to 0.05 though it would more naturally be reported as "p < 0.05". The supplements contain no per-animal data or ANOVA output to check it. If F was mistyped, the interaction would meet the threshold, and the correction would concern the reported statistics rather than the conclusion.
- A post hoc p placed next to the interaction F. Possible, but not what the text says. The Figure 3 legend states that Sidak's post hoc tests were used, and Figure 3J marks the combination group with a single asterisk. The p = 0.05 could be such a pairwise comparison (for example, combination versus vehicle). The sentence, however, attaches it to "interaction IL-6 × leptin F(1, 11) = 0.563". A post hoc difference between one treatment cell and vehicle would also not by itself establish an interaction (synergy), which requires the combined effect to exceed the sum of the two single-ligand effects.
- A text-conversion artifact. Ruled out. The publisher's typeset version shows the same values (see Status).
- A correction already issued. None found (see Status).
Status
- Version of record. The typeset Cell Reports article, Document S2 "Article plus Supplemental Information" (mmc2.pdf, 32 pages, in the PMC supplementary bundle retrieved from Europe PMC), was checked. On page 8 it reads "interaction IL-6 × leptin F(1, 11) = 0.563, p = 0.05; Figure 3J". The PDF text layer encodes "×" as "3", but a rendered image of the page shows ×, and the numbers are identical. The Europe PMC full-text XML and the live PMC page carry the same passage. The publisher's HTML pages (cell.com, sciencedirect.com) returned HTTP 403 to automated requests, and the doi.org route ended at a redirect stub. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. Document S1 (mmc1.pdf, Figures S1–S7) and Document S2 were searched. The Figure S4 legend (24-h chow intake for the same experiment) gives no interaction statistic. Neither file contains per-animal data or ANOVA output, and the PMC record lists no source-data file. The interaction F therefore cannot be reconstructed from published material.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations, PMC notices or Europe PMC notices. The Crossref record has empty update-to and updated-by fields, and the Crossref Labs record, which includes Retraction Watch data, lists no updates. The Europe PMC core record (PMID 30865890) has no comment or correction entries. The PubPeer API returned HTTP 404 for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC6418345; Europe PMC full-text XML and supplementary files (mmc1.pdf, mmc2.pdf); Crossref and Crossref Labs; Europe PMC; PubPeer; https://doi.org/10.1016/j.celrep.2019.02.044.
Suggested correction
The authors should consult the original ANOVA output for the Figure 3J experiment and correct the interaction statistic in the Results sentence
interaction IL-6 × leptin F(1, 11) = 0.563, p = 0.05
so that F and p agree. If F = 0.563 is correct, the interaction p is 0.47. The statements that IL-6 and leptin interact or cooperate to reduce feeding (abstract, highlights, Results, Figure 3 title, Discussion and conclusion) would then need to be revised or withdrawn, since neither main effect nor the interaction is significant. If the F value is the error (for example 5.63, for which p = 0.037), the corrected F and its exact p should be printed. In either case, as printed, p = 0.05 does not satisfy the paper's p < 0.05 criterion. If p = 0.05 is a Sidak post hoc comparison rather than the interaction test, it should be labelled as such and reported separately from the interaction F.
Recomputation script (python recompute.py.txt)
"""Recompute every F-test p-value reported in the Results of PMC6418345 (Figures 3B-3E,
3G, 3I, 3J/3K and 4G/4H)
(Cell Rep 2019, doi:10.1016/j.celrep.2019.02.044, "Parabrachial Interleukin-6 Reduces
Body Weight and Food Intake and Increases Thermogenesis to Regulate Energy Metabolism").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Results, "Leptin Acts in the lPBN to Alter Food Intake, but Not Body Temperature"
(one-way ANOVA): "F (2, 10) = 5.496, p < 0.05; Figure 3B", "F(2, 10) = 6.685, p < 0.05;
Figure 3C", "F(2, 10) = 7.464, p < 0.05; Figure 3D", "F(2, 12) = 7.55, p < 0.01;
Figure 3E", "F(2, 34) = 1.765, p > 0.05; Figures 3F and 3G", "Figures 3H and 3I;
F(2, 34) = 3.493, p < 0.05".
Results, "Cooperative Interaction of Leptin and IL-6 on Food Intake Control":
6-h chow intake (Figure 3J): "IL-6 alone: F(1, 11) = 0.21, p > 0.05; leptin alone
F(1, 11) = 0.88, p > 0.05; interaction IL-6 × leptin F(1, 11) = 0.563, p = 0.05"
BAT temperature (Figure 3K): "IL-6 alone: F(1, 11) = 4.524, p > 0.05; leptin alone
F(1, 11) = 7.652, p < 0.05; interaction IL-6 × leptin F(1, 11) = 1.528, p > 0.05"
Results, lPBN IL-6 knockdown (Figure 4G/4H): F(17, 544) = 8.345 and 4.944 (p < 0.01),
F(1, 32) = 7.548 (p < 0.05) and 3.462 (p > 0.05).
Methods: "p < 0.05 was considered statistically significant."
Figure 3 legend: n = 12 rats, each receiving every treatment (Latin square), two-way ANOVA.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ALPHA = 0.05
# label, F, df1, df2, reported relation, reported p
ROWS = [
("Fig 3B leptin dose, 6-h chow", 5.496, 2, 10, "<", 0.05),
("Fig 3C leptin dose, 12-h chow", 6.685, 2, 10, "<", 0.05),
("Fig 3D leptin dose, 24-h chow", 7.464, 2, 10, "<", 0.05),
("Fig 3E leptin dose, body weight", 7.55, 2, 12, "<", 0.01),
("Fig 3G leptin dose, core temp", 1.765, 2, 34, ">", 0.05),
("Fig 3I leptin dose, activity", 3.493, 2, 34, "<", 0.05),
("Fig 3J IL-6 main effect", 0.21, 1, 11, ">", 0.05),
("Fig 3J leptin main effect", 0.88, 1, 11, ">", 0.05),
("Fig 3J IL-6 × leptin interaction", 0.563, 1, 11, "=", 0.05),
("Fig 3K IL-6 main effect", 4.524, 1, 11, ">", 0.05),
("Fig 3K leptin main effect", 7.652, 1, 11, "<", 0.05),
("Fig 3K IL-6 × leptin interaction", 1.528, 1, 11, ">", 0.05),
("Fig 4G interaction", 8.345, 17, 544, "<", 0.01),
("Fig 4G treatment", 7.548, 1, 32, "<", 0.05),
("Fig 4H interaction", 4.944, 17, 544, "<", 0.01),
("Fig 4H treatment", 3.462, 1, 32, ">", 0.05),
]
def consistent(rel, rep, p):
if rel == ">":
return p > rep
if rel == "<":
return p < rep
return round(p, 2) == rep
print("1. Upper-tail p = P(F(df1,df2) >= F) for every F test in the Results (three sections)")
recomputed = {}
for lab, F, d1, d2, rel, rep in ROWS:
p = stats.f.sf(F, d1, d2)
recomputed[lab] = p
ok = "consistent" if consistent(rel, rep, p) else "INCONSISTENT"
print(f" {lab:36s} F({d1},{d2}) = {F:<6} reported p {rel} {rep:<5} recomputed p = {p:.3g} -> {ok}")
n_ok = sum(consistent(r[4], r[5], recomputed[r[0]]) for r in ROWS)
print(f" {n_ok} of {len(ROWS)} F/p pairs consistent")
F_INT = 0.563
p_int = stats.f.sf(F_INT, 1, 11)
print("\n2. The Fig 3J interaction, F(1,11) = 0.563, reported p = 0.05")
print(f" upper-tail p = {p_int:.3f}")
t = np.sqrt(F_INT)
print(f" equivalent t(11) = sqrt(F) = {t:.3f}")
print(f" two-sided p for t = {2 * stats.t.sf(t, 11):.3f}")
print(f" one-sided p for t = {stats.t.sf(t, 11):.3f}")
print(f" left tail P(F <= 0.563) = {stats.f.cdf(F_INT, 1, 11):.3f}")
print(f" ratio recomputed / reported p = {p_int / 0.05:.1f}")
F_CRIT = stats.f.isf(ALPHA, 1, 11)
print("\n3. What F(1,11) the reported p would need")
print(f" F(1,11) critical value at p = 0.05 = {F_CRIT:.3f}")
lo, hi = stats.f.isf(0.055, 1, 11), stats.f.isf(0.045, 1, 11)
print(f" F giving a p that rounds to 0.05 (0.045 <= p < 0.055): {lo:.2f} to {hi:.2f}")
print(f" factor between needed F and reported F = {F_CRIT / F_INT:.1f}")
print("\n4. Other degrees of freedom for F = 0.563 (df1 = 1)")
df2s = [1, 2, 5, 10, 11, 20, 50, 100, 1000, 10**6]
ps = [stats.f.sf(F_INT, 1, d) for d in df2s]
for d, p in zip(df2s, ps):
print(f" df2 = {d:>7}: p = {p:.3f}")
print(f" range over df2 = 1 ... 1e6: {min(ps):.2f} to {max(ps):.2f}")
print(f" smallest F that reaches p < 0.05 with df1 = 1 for any df2 (df2 -> inf): {stats.chi2.isf(ALPHA, 1):.3f}")
F_SHIFT = 5.63
p_shift = stats.f.sf(F_SHIFT, 1, 11)
print("\n5. Decimal-shift reading F = 5.63 (not verifiable without data)")
print(f" p for F(1,11) = 5.63 = {p_shift:.3f}")
# ------------------------------------------------------------------ style
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# ------------------------------------------------------------------ Figure 1
fig_rows = [r for r in ROWS if r[0].startswith(("Fig 3J", "Fig 3K"))][::-1] # Figure 3J and 3K statistics, top to bottom as in the text
XMIN, XMAX = 0.005, 1.0
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
ax.axvline(ALPHA, color=SEC, lw=0.9, zorder=1)
ax.text(ALPHA, len(fig_rows) - 0.35, "p = 0.05 (paper's threshold: p < 0.05)",
color=SEC, fontsize=10, ha="center", va="bottom")
for i, (lab, F, d1, d2, rel, rep) in enumerate(fig_rows):
p = recomputed[lab]
hl = rel == "="
if rel == ">":
ax.plot([rep, XMAX], [i + 0.13] * 2, color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([rep], [i + 0.13], marker="|", ms=9, mew=2, color=ORANGE, zorder=3)
elif rel == "<":
ax.plot([XMIN, rep], [i + 0.13] * 2, color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([rep], [i + 0.13], marker="|", ms=9, mew=2, color=ORANGE, zorder=3)
else:
ax.plot([rep], [i + 0.13], "o", ms=9, color=ORANGE, zorder=3)
ax.annotate("", xy=(p, i - 0.13), xytext=(rep, i - 0.13),
arrowprops=dict(arrowstyle="-|>", color=SEC, lw=1.2, shrinkA=6, shrinkB=7))
ax.text(rep * 0.9, i + 0.13, "reported p = 0.05", color=ORANGE, fontsize=10,
ha="right", va="center")
ax.text(p * 0.97, i + 0.02, f"recomputed p = {p:.3f}", color=BLUE, fontsize=10,
ha="right", va="bottom")
ax.plot([p], [i - 0.13], "o", ms=9, color=BLUE, mfc=BLUE, zorder=4)
labels = [f"{lab}\nF({d1},{d2}) = {F}" for lab, F, d1, d2, rel, rep in fig_rows]
ax.set_yticks(range(len(fig_rows)))
ax.set_yticklabels(labels, fontsize=10)
for tl, r in zip(ax.get_yticklabels(), fig_rows):
if r[4] == "=":
tl.set_fontweight("bold")
ax.set_ylim(-0.6, len(fig_rows) - 0.1)
ax.set_xticks([0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1])
ax.set_xticklabels(["0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_xlabel("p-value (log scale)")
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [
Line2D([], [], color=ORANGE, lw=2, label="reported in paper (bound drawn as a range, exact value as a dot)"),
Line2D([], [], color=BLUE, marker="o", ms=8, ls="none", label="recomputed from the reported F and df"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.38, -0.2), ncol=1,
frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig1")
# ------------------------------------------------------------------ Figure 2
x = np.linspace(0.001, 8, 2000)
y = stats.f.pdf(x, 1, 11)
YMAX = 1.2
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.plot(x, np.minimum(y, YMAX * 2), color=SEC, lw=1.6, zorder=3)
m1 = x >= F_INT
ax.fill_between(x[m1], 0, y[m1], color=BLUE, alpha=0.18, lw=0, zorder=1)
m2 = x >= F_CRIT
ax.fill_between(x[m2], 0, y[m2], color=ORANGE, alpha=0.85, lw=0, zorder=2)
ax.axvline(F_INT, color=BLUE, lw=2, zorder=4)
ax.axvline(F_CRIT, color=ORANGE, lw=2, zorder=4)
ax.axvline(F_SHIFT, color=SEC, lw=1.2, ls=(0, (3, 3)), zorder=4)
ax.text(F_INT + 0.12, 0.98, f"reported F = {F_INT}\narea to the right = {p_int:.3f}\n(the p this F implies)",
color=BLUE, fontsize=10, va="top")
ax.text(F_CRIT - 0.12, 0.60, f"F needed for p = 0.05:\n{F_CRIT:.2f}",
color="#b8481c", fontsize=10, va="top", ha="right")
ax.text(F_SHIFT + 0.12, 0.98, f"F = {F_SHIFT}\n(decimal-shift reading)\np = {p_shift:.3f}",
color=SEC, fontsize=10, va="top")
p_crit_area = stats.f.sf(F_CRIT, 1, 11)
ax.annotate(f"orange tail right of F = {F_CRIT:.2f}:\narea = {p_crit_area:.3f} = reported p\n(too thin to see at this scale)",
xy=(5.3, 0.006), xytext=(5.75, 0.36), color="#b8481c", fontsize=10, va="top",
arrowprops=dict(arrowstyle="-|>", color="#b8481c", lw=1.4, shrinkA=2, shrinkB=1))
ax.set_xlim(0, 8)
ax.set_ylim(0, YMAX)
ax.set_xlabel("F statistic, df = (1, 11)")
ax.set_ylabel("probability density")
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Oral cancer test's accuracy table counts 101 subjects in a cohort the paper gives as 92
Cell Rep Med (2024) · doi:10.1016/j.xcrm.2024.101447 · PMC10983043
Summary
The paper reports that the beta-defensin index (BDI) detects oral squamous cell carcinoma (OSCC) with 98.2% sensitivity (95% CI 90.3–99.9) and 82.6% specificity (95% CI 68.6–92.2) in "a multi-center validation study". These figures come from Table 1, whose four cells (TP 54, TN 38, FP 8, FN 1) add up to 101, while the text twice says the analysis covers 92 subjects: 54 cancers and 38 benign lesions in the discovery and validation cohorts. The printed TP and TN equal the cancer and benign totals, so the 8 false positives and 1 false negative are counted twice, and the denominators become 55 and 46 instead of 54 and 38. With the paper's own counts, which the subject-level labels in supplementary Table S1 confirm, specificity is 30/38 = 78.9% (exact 95% CI 62.7–90.4), not 82.6%. Sensitivity barely changes (53/54 = 98.1%). Because the abstract presents these as validation results, it also matters that they pool the discovery cohort in which the 1.25 cutoff was chosen. The validation cohorts alone give 96.6% sensitivity and 78.3% specificity.
What the paper reports
Abstract (Summary):
A multi-center validation study shows sensitivity and specificity values of 98.2% (95% confidence interval [CI] 90.3–99.9) and 82.6% (95% CI 68.6–92.2), respectively.
Highlights:
A multi-center validation study yields sensitivity and specificity of 98% and 83%
Results, "Beta-defensin index: Proof-of-concept, discovery, and validation studies" (study design paragraph):
We recruited a total of 92 subjects for the discovery and validation study
Results, discovery-phase study:
Of the 40 oral lesions, 25 were diagnosed by pathology review as being cancerous while 15 were benign (see Table S1B diagnoses).
We arrived at a BDI cutoff value of 1.25 and found 100% sensitivity with an acceptable specificity value of 80% (Figures 4B and 4C).
Figure 4 legend, panel D:
(D) BDI values for 23 non-cancerous and 29 OSCC lesions from the three validation cohorts (blue: internal validation cohort [n = 21]; green: external validation cohort 1 [n = 19]; orange: external validation cohort 2 [n = 12]).
Results, "Sample size and power calculation":
In the validation cohort (total 52 subjects; 29 OSCC: mean = 6.1, SD = 6.9; 23 benign: mean = 0.8, SD = 0.64)
Results, validation study:
We then calculated the sensitivity and specificity of the BDI, based on the threshold value of 1.25 from the discovery cohort, and found overall sensitivity of 98.2% and specficity of 82.6%. For all the subjects (n = 92), sensitivity/specificity and positive/negative predictive values, with a confidence interval of 95%, are shown in Table 1. Among 92 subjects, we recorded eight false positives (FPs) and one FN (Tables 1 and S1).
Table 1, "Sensitivity, specificity, and positive and negative predictive values of BDI in OSCC detection" (columns: values (BDI vs. biopsy) | 95% confidence interval):
True positive, TP | 54 | –
True negative, TN | 38 | –
False positive, FP | 8 | –
False negative, FN | 1 | –
Sensitivity, TP/(TP + FN) | 98.2 | 90.28%–99.95%
Specificity, TN/(TN + FP) | 82.6 | 68.58%–92.18%
Positive predictive value | 87.10 | 78.22%–92.69%
Negative predictive value | 97.44 | 84.44%–99.63%
Results, "Benchmarking of BDI against other OSCC biomarkers/platforms":
Figure S5 shows that the BDI is the most sensitive (98%) in detecting OSCC
Discussion:
In the discovery and validation studies, we reported eight FP cases.
Additionally, one of the benefits of using the BDI would be the reduction of unnecessary biopsies conducted by dentists in the United States from >90% to 17.4% when considering the FP rate of 82.6%.
what we believe happened in our one and only FN result obtained through the validation study at CMC (where the BDI score was reported as 0.23 [Figure 4D and Table S1])
What the paper's own numbers show
Method. I added up the cancer and benign counts that the paper gives for the discovery cohort (Results) and the three validation cohorts (Figure 4D legend; power-calculation paragraph), and checked them against the site sizes in the STAR Methods (40, 21, 19 and 12 subjects). I then summed the four Table 1 cells and derived the disease-positive (TP + FN) and disease-negative (TN + FP) totals they imply. I recomputed every Table 1 estimate from its printed cells to identify the computation behind it: exact Clopper–Pearson intervals for sensitivity and specificity, and the logit interval of Mercaldo et al. (2007), as used by MedCalc, for the predictive values. Finally I rebuilt the 2×2 table from the stated cohort (54 cancers, 38 benign) and the stated 8 FP and 1 FN, and checked it against the TP/TN/FP/FN label for each subject in supplementary Table S1 (mmc2.xlsx), cohort by cohort. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Subjects, discovery + validation | Results; STAR Methods (40 + 21 + 19 + 12) | 92 | 92 |
| Cancers | Results (discovery 25); Fig 4D legend (validation 29) | 25 + 29 | 54 |
| Benign lesions | Results (discovery 15); Fig 4D legend (validation 23) | 15 + 23 | 38 |
| Sum of Table 1 cells | Table 1 | TP 54, TN 38, FP 8, FN 1 (text: n = 92) | 54 + 38 + 8 + 1 = 101; the excess of 9 equals FP + FN |
| Cancers implied by Table 1 (TP + FN) | Table 1 | — | 55 (cohort has 54) |
| Benign lesions implied by Table 1 (TN + FP) | Table 1 | — | 46 (cohort has 38) |
| Sensitivity | Table 1; Abstract; Results | 98.2 (90.28–99.95) | 54/55 = 98.18% (90.28–99.95): reproduces the printed value from the printed cells. Stated cohort: 53/54 = 98.15% (90.11–99.95) |
| Specificity | Table 1; Abstract; Results | 82.6 (68.58–92.18) | 38/46 = 82.61% (68.58–92.18): reproduces the printed value. Stated cohort: 30/38 = 78.95% (62.68–90.45) |
| Positive predictive value | Table 1 | 87.10 (78.22–92.69) | 54/62 = 87.10% (78.22–92.69): reproduces the printed value. Stated cohort: 53/61 = 86.89% (78.14–92.47) |
| Negative predictive value | Table 1 | 97.44 (84.44–99.63) | 38/39 = 97.44% (84.44–99.63): reproduces the printed value. Stated cohort: 30/31 = 96.77% (81.04–99.53) |
| Discovery cohort counts | Results (100% sensitivity, 80% specificity in 25 + 15); Table S1B | 100%, 80% | TP 25, FN 0, TN 12, FP 3 (text and Table S1B agree) |
| Validation cohorts, pooled | Table S1C–S1E (21 + 12 + 19 subjects) | — | TP 28, FN 1, TN 18, FP 5. Sensitivity 28/29 = 96.55% (82.24–99.91); specificity 18/23 = 78.26% (56.30–92.54) |
| Discovery + validation, from Table S1 labels | Table S1B–S1E | — | TP 53, TN 30, FP 8, FN 1, N = 92: identical to the table rebuilt from the stated counts |
| Specificity by cohort | Table S1B–S1E | — | discovery 12/15; internal 4/6; external (12 subjects) 10/12; external (19 subjects) 4/5 |
| Location of the single FN | Discussion; Table S1E | "validation study at CMC", BDI 0.23 | the only FN label in Table S1 is in the 19-subject external cohort, BDI 0.23 |
| Share of benign lesions classed positive | Discussion ("17.4%") | 17.4% | 17.4% = 100 − 82.6 = 8/46; stated cohort: 8/38 = 21.1% |
| Highlights | Highlights | 98% and 83% | stated cohort: 98% and 79% |
All Table 1 estimates and confidence intervals follow exactly from the printed cells, so the table is arithmetically consistent with itself. What is wrong is the cells: they describe 55 cancers and 46 benign lesions, whereas the study had 54 and 38. The 8 FP and 1 FN given in the text (and listed one by one in Table S1) match the stated cohort; only TP and TN are off, by exactly FN (1) and FP (8). All 92 Table S1 labels agree with the BDI value and the 1.25 cutoff.
Figure
Figure 1. Left (orange): the 2×2 table as printed in Table 1, with its row and column sums. Right (blue): the same table rebuilt from the paper's stated cohort of 54 cancers and 38 benign lesions and its stated 8 false positives and 1 false negative; the subject-level labels in Table S1 give the same four counts. The shaded TP and TN cells are the only ones that differ. The printed table's columns sum to 55 and 46 (N = 101), one and eight more than the cohort contains, and its specificity is therefore 38/46 = 82.6% instead of 30/38 = 78.9%. Intervals are exact Clopper–Pearson 95% CIs.
Figure 2. The four accuracy estimates in Table 1 with their 95% CIs: as reported (orange circles, from the printed cells) and as recomputed from the stated 92-subject cohort (blue squares). Sensitivity, PPV and NPV move by at most 0.7 points. Specificity falls by 3.7 points to 78.9%, and the lower limit of its interval falls from 68.6% to 62.7%. Sensitivity and specificity use Clopper–Pearson intervals, PPV and NPV the logit interval, the same methods that reproduce the printed CIs exactly. The horizontal axis starts at 50%.
Why it matters
Sensitivity and specificity are the paper's main quantitative result. They appear in the abstract, the highlights, Table 1, the Results and the Discussion, and the specificity feeds the Discussion's estimate of how far the test would cut unnecessary biopsies.
- Specificity is overstated. With the cohort the paper describes, specificity is 78.9% (62.7–90.4), not 82.6% (68.6–92.2). The highlights' "83%" becomes 79%. PPV (86.9% vs 87.1%) and NPV (96.8% vs 97.4%) are slightly overstated too. Sensitivity is essentially unchanged (98.1% vs 98.2%), so the benchmarking statement that the BDI is "the most sensitive (98%)" still holds as rounded.
- The unnecessary-biopsy estimate changes. The Discussion's 17.4% is 100 − 82.6, the proportion of benign lesions the test would call positive. (The text calls 82.6% the "FP rate"; it is the specificity, and 17.4% is the false-positive rate.) With the stated cohort that proportion is 8/38 = 21.1%.
- The headline figures are not validation-only figures. The abstract and highlights call them results of "a multi-center validation study", but the Results state that Table 1 covers "all the subjects (n = 92)", which includes the 40-subject discovery cohort in which the 1.25 cutoff was selected to maximise sensitivity. In the 52 validation subjects alone, sensitivity is 28/29 = 96.6% (82.2–99.9) and specificity 18/23 = 78.3% (56.3–92.5).
The overall conclusion, that the BDI separates OSCC from benign lesions with high sensitivity and moderate specificity, survives. What changes is the size of the headline specificity and its interval, and the description of which subjects those numbers come from.
Explanations considered
- The proof-of-concept cohort is included. This does not account for the numbers. That cohort has 8 subjects, 4 with OSCC and 4 benign (Results; STAR Methods), so adding it gives 100 subjects, 58 cancers and 42 benign, not 101, 55 and 46. Its Table S1A rows carry no TP/TN/FP/FN labels, and the Results say Table 1 covers the 92.
- The point-of-care (microfluidic) cohort is included. This does not account for the numbers either. The STAR Methods give it 5 subjects, so adding it gives 97 (or 105 with the proof-of-concept cohort), and no combination of the stated cohorts gives 55 cancers and 46 benign lesions.
- The tables count lesions rather than subjects. The Figure 4 legends count lesions (15 + 25 and 23 + 29), and these also total 92 and match the site sizes (40 + 21 + 19 + 12), so lesions and subjects coincide here.
- Oropharyngeal lesions were added back. The Results say these were excluded from the discovery and validation studies, and no count of them is given that would produce +1 cancer and +8 benign.
- Only the validation cohorts are tabulated. These contain 29 cancers and 23 benign lesions (52 subjects), which does not match the table.
- Rounding, or a different interval method. Neither works. 53/54 = 98.15% would print as 98.1, not 98.2, and its exact lower limit is 90.11, not 90.28. With FP = 8, a specificity of 82.6% requires TN = 38, i.e. 46 benign lesions. Every printed interval is reproduced to two decimals by standard methods applied to the printed cells (54/55, 38/46, 54/62, 38/39), so the cells, not the interval method, determine the printed values.
- A conversion or typesetting artifact. Excluded. The publisher's own typeset PDF (bundled as "Document S2. Article plus supplemental information") shows the same Table 1 cells, text and abstract as the PMC archive. In the archived XML each count sits in its own table cell, with no stray signs or characters.
- The text, not the table, is wrong (the cohort really had 55 cancers and 46 benign lesions). Contradicted by the paper's own records. The four cohort counts are given in several independent places (Results text, Figure 4 legends, power calculation, STAR Methods), and the subject-level Table S1 lists exactly 54 cancers and 38 benign lesions in these cohorts, with TP 53, TN 30, FP 8 and FN 1.
Status
- Version of record. The publisher's HTML could not be loaded (the DOI redirect returned only a redirect stub, and cell.com returned HTTP 403). I checked instead the Elsevier-typeset PDF of the article (mmc4.pdf, "Cell Reports Medicine, 5 (2024) 101447", created 2024-03-15) from the Europe PMC supplementary-files bundle, and the Europe PMC full-text XML. Both match the PMC archive: abstract (p. 2), "92 subjects" (p. 6), "n = 92" and "Among 92 subjects" (p. 7), Table 1 (p. 8) and the 17.4% sentence (p. 11). Every quote above was checked verbatim against the PMC-archived text.
- Supplements. I downloaded the Europe PMC supplementary-files bundle and opened Table S1 (mmc2.xlsx), which gives each subject's diagnosis, BDI value and TP/TN/FP/FN label. The per-cohort counts above were tallied from it (only aggregate counts are reported here). The bundle also holds Document S1 (Figures S1–S7, Tables S2–S4), Data S1 (whole-exome data) and the article PDF.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC (PMID 38442713) list no correction or comment links. The PMC record carries one "retraction-forward" link, to PMC10830194; it belongs to reference 26, a cited 2022 Stem Cells International paper that was later retracted, and is not a notice about this article. PubPeer's API and search page refused automated access (HTTP 403, a Cloudflare challenge), so PubPeer could not be checked directly. A web search for the title together with "correction" and "PubPeer" returned only the article and press coverage, so the absence of a PubPeer thread is not fully confirmed.
- Sources. PMC article PMC10983043; Europe PMC full text and supplementary files (mmc1–mmc4); Crossref; PubPeer (not reachable).
Suggested correction
- Replace the Table 1 cells with the counts from Table S1 (TP 53, TN 30, FP 8, FN 1; N = 92) and recompute the four estimates and their CIs: sensitivity 98.1% (90.1–99.95), specificity 78.9% (62.7–90.4), PPV 86.9% (78.1–92.5), NPV 96.8% (81.0–99.5).
- Update the Abstract ("82.6% (95% CI 68.6–92.2)"), the Highlights ("83%"), the Results text ("specficity of 82.6%") and the Discussion's biopsy estimate (17.4% becomes about 21.1%). In the Discussion, describe 82.6% (corrected, 78.9%) as specificity rather than "FP rate".
- Either describe the pooled figures as discovery-plus-validation results, or report the validation-only estimates in the Abstract and Highlights (sensitivity 28/29 = 96.6%, specificity 18/23 = 78.3%), since the cutoff was selected in the discovery cohort.
Recomputation script (python recompute.py.txt)
"""Recompute the diagnostic-accuracy table of PMC10983043
(Cell Rep Med 2024;5:101447, doi:10.1016/j.xcrm.2024.101447).
Run: python recompute.py.txt [path/to/mmc2.xlsx]
Needs numpy, scipy and matplotlib (openpyxl only if mmc2.xlsx is given).
Prints every recomputed number used in writeup.md and writes, next to this file,
fig1.svg/png, fig2.svg/png (the figures used in writeup.md)
candidates/c1..c5_*.svg/png (five alternative figure designs)
Inputs are copied from the paper as archived in PMC:
Results (discovery-phase study): 40 subjects, 25 cancerous, 15 benign; cutoff 1.25
gives 100% sensitivity and 80% specificity
Results (validation study) and Figure 4D legend: 29 OSCC, 23 non-cancerous lesions
in three cohorts of 21, 19 and 12 subjects
Results: "Among 92 subjects, we recorded eight false positives (FPs) and one FN"
Table 1: TP 54, TN 38, FP 8, FN 1 and the four accuracy estimates with 95% CIs
STAR Methods: proof-of-concept cohort of 8 subjects (4 OSCC + 4 benign, Results)
Per-cohort classification counts are aggregated from the TP/TN/FP/FN column of
Table S1 (supplement mmc2.xlsx). Pass the file path to re-derive them from the
subject-level rows; only aggregate counts are used, no subject-level data are printed.
"""
import os
import sys
import numpy as np
from scipy.stats import beta, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (paper)
TABLE1 = dict(TP=54, TN=38, FP=8, FN=1)
TABLE1_PRINTED = { # estimate, CI low, CI high (percent)
"Sensitivity": (98.2, 90.28, 99.95),
"Specificity": (82.6, 68.58, 92.18),
"PPV": (87.10, 78.22, 92.69),
"NPV": (97.44, 84.44, 99.63),
}
ABSTRACT_PRINTED = {"Sensitivity": (98.2, 90.3, 99.9), "Specificity": (82.6, 68.6, 92.2)}
DISC = dict(cancer=25, benign=15, sens=1.00, spec=0.80) # Results, discovery
VAL = dict(cancer=29, benign=23) # Results, Fig 4D legend
SITES = dict(discovery=40, internal=21, external_19=19, external_12=12) # Methods
STATED_N, STATED_FP, STATED_FN = 92, 8, 1 # Results, validation
POC = dict(n=8, cancer=4, benign=4) # proof-of-concept
CUTOFF = 1.25
# Table S1 (mmc2.xlsx), TP/TN/FP/FN column, aggregated per cohort
S1 = {
"Discovery (S1B, n=40)": dict(TP=25, TN=12, FP=3, FN=0),
"Internal validation (S1C, n=21)": dict(TP=15, TN=4, FP=2, FN=0),
"External, 12 subjects (S1D)": dict(TP=0, TN=10, FP=2, FN=0),
"External, 19 subjects (S1E)": dict(TP=13, TN=4, FP=1, FN=1),
}
def s1_from_xlsx(path):
"""Re-derive the per-cohort counts from Table S1; check labels against BDI > 1.25."""
import openpyxl
ws = openpyxl.load_workbook(path, data_only=True).worksheets[0]
out, sec, mism = {}, None, 0
for r in list(ws.iter_rows(values_only=True))[1:]:
if r[0] and str(r[0]).upper().startswith("TABLE"):
sec = str(r[0]).split("]")[0] + "]"
out[sec] = dict(TP=0, TN=0, FP=0, FN=0, cancer=0, benign=0)
continue
if sec is None or r[6] is None:
continue
canc = str(r[5]).strip().upper().startswith("Y")
out[sec]["cancer" if canc else "benign"] += 1
if r[11]:
lab = str(r[11]).strip()
out[sec][lab] += 1
pos = float(r[6]) > CUTOFF
exp = {(True, True): "TP", (False, False): "TN",
(True, False): "FP", (False, True): "FN"}[(pos, canc)]
mism += exp != lab
return out, mism
# ------------------------------------------------------------------ statistics
def cp(x, n, a=0.05):
"""Exact Clopper-Pearson interval."""
lo = 0.0 if x == 0 else beta.ppf(a / 2, x, n - x + 1)
hi = 1.0 if x == n else beta.ppf(1 - a / 2, x + 1, n - x)
return lo, hi
def logit_pv(TP, TN, FP, FN):
"""PPV/NPV with the Mercaldo et al. (2007) logit CI (as in MedCalc)."""
z = norm.ppf(0.975)
npos, nneg = TP + FN, TN + FP
se, sp = TP / npos, TN / nneg
ppv, npv = TP / (TP + FP), TN / (TN + FN)
v_ppv = ((1 - se) / se) / npos + (sp / (1 - sp)) / nneg
v_npv = (se / (1 - se)) / npos + ((1 - sp) / sp) / nneg if se < 1 else np.nan
f = lambda p, v: (1 / (1 + np.exp(-(np.log(p / (1 - p)) - z * np.sqrt(v)))),
1 / (1 + np.exp(-(np.log(p / (1 - p)) + z * np.sqrt(v)))))
return (ppv, *f(ppv, v_ppv)), (npv, *f(npv, v_npv))
def metrics(c):
TP, TN, FP, FN = c["TP"], c["TN"], c["FP"], c["FN"]
se, sp = TP / (TP + FN), TN / (TN + FP)
(ppv, pl, ph), (npv, nl, nh) = logit_pv(TP, TN, FP, FN)
return {
"Sensitivity": (100 * se, *(100 * v for v in cp(TP, TP + FN)), f"{TP}/{TP + FN}"),
"Specificity": (100 * sp, *(100 * v for v in cp(TN, TN + FP)), f"{TN}/{TN + FP}"),
"PPV": (100 * ppv, 100 * pl, 100 * ph, f"{TP}/{TP + FP}"),
"NPV": (100 * npv, 100 * nl, 100 * nh, f"{TN}/{TN + FN}"),
}
def show(label, c):
n = sum(c[k] for k in ("TP", "TN", "FP", "FN"))
print(f"\n--- {label}: TP={c['TP']} TN={c['TN']} FP={c['FP']} FN={c['FN']} "
f"N={n} cancers(TP+FN)={c['TP'] + c['FN']} benign(TN+FP)={c['TN'] + c['FP']}")
m = metrics(c)
for k, (est, lo, hi, frac) in m.items():
meth = "Clopper-Pearson" if k in ("Sensitivity", "Specificity") else "logit"
print(f" {k:<11} {frac:>6} = {est:6.2f}% 95% CI {lo:.2f}-{hi:.2f} ({meth})")
return m
def add(*cs):
return {k: sum(c[k] for c in cs) for k in ("TP", "TN", "FP", "FN")}
# ------------------------------------------------------------------ 1. head counts
print("=" * 78)
print("1. Head counts stated in the paper")
print(f" Subjects by site 40+21+19+12 = {sum(SITES.values())}")
print(f" Validation subjects 21+19+12 = {SITES['internal'] + SITES['external_19'] + SITES['external_12']}"
f" (= {VAL['cancer']} OSCC + {VAL['benign']} benign = {VAL['cancer'] + VAL['benign']})")
canc = DISC["cancer"] + VAL["cancer"]
ben = DISC["benign"] + VAL["benign"]
print(f" Cancers 25+29 = {canc}; benign 15+23 = {ben}; total = {canc + ben}")
t1n = sum(TABLE1.values())
print(f" Table 1 cells 54+38+8+1 = {t1n} (stated n = {STATED_N}; excess = {t1n - STATED_N}"
f" = FP+FN = {TABLE1['FP'] + TABLE1['FN']})")
print(f" Table 1 implies cancers TP+FN = {TABLE1['TP'] + TABLE1['FN']} (stated {canc}),"
f" benign TN+FP = {TABLE1['TN'] + TABLE1['FP']} (stated {ben})")
print(f" Table 1 TP = {TABLE1['TP']} equals the stated cancer total ({canc});"
f" TN = {TABLE1['TN']} equals the stated benign total ({ben})")
# ------------------------------------------------------------------ 2. accuracy
print("\n" + "=" * 78)
print("2. Accuracy estimates")
m_print = show("Table 1 cells as printed", TABLE1)
print(" Printed in Table 1:", {k: v for k, v in TABLE1_PRINTED.items()})
for k, (e, lo, hi) in TABLE1_PRINTED.items():
r = m_print[k]
print(f" {k:<11} printed {e} ({lo}-{hi}) vs recomputed {r[0]:.2f} ({r[1]:.2f}-{r[2]:.2f})"
f" -> {'match' if abs(round(r[1], 2) - lo) < 0.006 and abs(round(r[2], 2) - hi) < 0.006 else 'DIFFER'}")
implied = dict(TP=canc - STATED_FN, TN=ben - STATED_FP, FP=STATED_FP, FN=STATED_FN)
m_impl = show("Implied by stated cohort (54 cancers, 38 benign) with 8 FP and 1 FN", implied)
pooled_s1 = add(*S1.values())
m_s1 = show("Table S1 labels, discovery + validation pooled", pooled_s1)
print(f" identical to the cohort-implied table: {pooled_s1 == implied}")
disc = S1["Discovery (S1B, n=40)"]
val = add(*[v for k, v in S1.items() if not k.startswith("Discovery")])
print(f"\n Discovery check: sensitivity {disc['TP']}/{disc['TP'] + disc['FN']} ="
f" {disc['TP'] / (disc['TP'] + disc['FN']):.0%}; specificity {disc['TN']}/{disc['TN'] + disc['FP']}"
f" = {disc['TN'] / (disc['TN'] + disc['FP']):.0%} (paper: 100%, 80%)")
print(f" Discovery counts implied by the text alone: TP = {round(DISC['sens'] * DISC['cancer'])},"
f" TN = {round(DISC['spec'] * DISC['benign'])}, FP = {DISC['benign'] - round(DISC['spec'] * DISC['benign'])}")
m_val = show("Validation cohorts only (S1C+S1D+S1E)", val)
for k, c in S1.items():
nb = c["TN"] + c["FP"]
print(f" {k:<34} specificity {c['TN']}/{nb} = {100 * c['TN'] / nb:.1f}%"
+ (f"; sensitivity {c['TP']}/{c['TP'] + c['FN']}" if c['TP'] + c['FN'] else ""))
print("\n Abstract (rounded to 1 dp):", ABSTRACT_PRINTED)
for k in ("Sensitivity", "Specificity"):
print(f" {k}: Table 1 cells -> {m_print[k][0]:.1f} ({m_print[k][1]:.2f}-{m_print[k][2]:.2f});"
f" stated cohort -> {m_impl[k][0]:.1f} ({m_impl[k][1]:.2f}-{m_impl[k][2]:.2f})")
print(f" Highlights '98% and 83%' -> stated cohort {m_impl['Sensitivity'][0]:.0f}% and"
f" {m_impl['Specificity'][0]:.0f}%")
# ------------------------------------------------------------------ 3. derived claims
print("\n" + "=" * 78)
print("3. Derived claims")
print(f" Discussion '17.4%' = 100 - 82.6 = {100 - 82.6:.1f}% (i.e. 1 - specificity, 8/46 = {100 * 8 / 46:.1f}%)")
print(f" With the stated cohort: 1 - specificity = 8/38 = {100 * 8 / 38:.1f}%")
print(f" Point-estimate change in specificity: {m_print['Specificity'][0]:.2f} -> {m_impl['Specificity'][0]:.2f}"
f" ({m_impl['Specificity'][0] - m_print['Specificity'][0]:+.2f} points);"
f" lower CI bound {m_print['Specificity'][1]:.2f} -> {m_impl['Specificity'][1]:.2f}")
# ------------------------------------------------------------------ 4. alternatives
print("\n" + "=" * 78)
print("4. Innocent explanations: can any stated grouping give 55 cancers / 46 benign / 101?")
print(f" + proof-of-concept cohort: {STATED_N} + {POC['n']} = {STATED_N + POC['n']};"
f" cancers {canc + POC['cancer']}, benign {ben + POC['benign']}")
print(f" validation only: {VAL['cancer']} cancers, {VAL['benign']} benign, n = {VAL['cancer'] + VAL['benign']}")
print(f" discovery only: {DISC['cancer']} cancers, {DISC['benign']} benign, n = {DISC['cancer'] + DISC['benign']}")
print(f" rounding: 53/54 = {100 * 53 / 54:.3f}% -> {100 * 53 / 54:.1f}; CP lower bound {100 * cp(53, 54)[0]:.2f}"
f" vs printed 90.28; 54/55 CP lower bound {100 * cp(54, 55)[0]:.2f}")
print(f" 30/38 = {100 * 30 / 38:.2f}%; 38/46 = {100 * 38 / 46:.2f}%; with FP = 8, TN/(TN+8) = 0.826 needs TN = 38, i.e. 46 benign")
if len(sys.argv) > 1:
print("\n" + "=" * 78)
print("5. Re-derived from", os.path.basename(sys.argv[1]))
out, mism = s1_from_xlsx(sys.argv[1])
for k, v in out.items():
print(f" {k}: {v}")
print(f" labels disagreeing with BDI > {CUTOFF}: {mism}")
# ------------------------------------------------------------------ figures
ORANGE, BLUE = "#eb6834", "#2a78d6"
INK, INK2, BG, GRID = "#0b0b0b", "#52514e", "#fcfcfb", "#e4e3df"
TINT_O, TINT_B = "#fbe3d9", "#dde9f8"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
LW = 1.5 # 2 px at 96 px/in
def save(fig, path):
fig.savefig(path + ".svg")
fig.savefig(path + ".png", dpi=200)
plt.close(fig)
def box(ax, x, y, w, h, fc=BG, ec=INK2, lw=1.0, r=0.012):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle=f"round,pad=0,rounding_size={r}",
fc=fc, ec=ec, lw=lw, transform=ax.transAxes))
def f1(v):
"""One decimal, except keep two when one decimal would print 100.0."""
t = f"{v:.1f}"
return f"{v:.2f}" if t == "100.0" else t
# --- design 3: two 2x2 tables (default fig1)
def fig_two_tables(path):
fig = plt.figure(figsize=(7.2, 3.5))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
panels = [
(0.025, "Table 1 as printed", ORANGE, TINT_O, TABLE1, m_print,
"the paper's reported cells"),
(0.525, "Rebuilt from the stated cohort", BLUE, TINT_B, implied, m_impl,
"54 cancers, 38 benign; 8 FP, 1 FN"),
]
for x0, title, col, tint, c, m, sub in panels:
ax.add_patch(Rectangle((x0, 0.865), 0.011, 0.075, fc=col, ec="none", transform=ax.transAxes))
ax.text(x0 + 0.022, 0.915, title, fontsize=12, weight="bold", va="center")
ax.text(x0 + 0.022, 0.865, sub, fontsize=10, color=INK2, va="center")
cw, ch = 0.10, 0.14
gx, gy = x0 + 0.10, 0.44
for i, lab in enumerate(("Cancer", "Benign")):
ax.text(gx + (i + 0.5) * cw, gy + 2 * ch + 0.03, lab, ha="center", fontsize=10.5, color=INK2)
ax.text(gx + 2 * cw + 0.06, gy + 2 * ch + 0.03, "Total", ha="center", fontsize=10.5, color=INK2)
ax.text(gx - 0.012, gy + 1.5 * ch, "BDI +", ha="right", va="center", fontsize=10.5, color=INK2)
ax.text(gx - 0.012, gy + 0.5 * ch, "BDI −", ha="right", va="center", fontsize=10.5, color=INK2)
cells = [(0, 1, "TP", c["TP"]), (1, 1, "FP", c["FP"]), (0, 0, "FN", c["FN"]), (1, 0, "TN", c["TN"])]
for i, j, lab, v in cells:
hl = lab in ("TP", "TN")
ax.add_patch(Rectangle((gx + i * cw, gy + j * ch), cw, ch, fc=tint if hl else BG,
ec=INK2, lw=0.8, transform=ax.transAxes))
ax.text(gx + i * cw + 0.008, gy + j * ch + ch - 0.028, lab, fontsize=10, color=INK2, va="center")
ax.text(gx + (i + 0.5) * cw, gy + (j + 0.40) * ch, str(v), fontsize=14,
weight="bold" if hl else "normal", ha="center", va="center")
ncan, nben = c["TP"] + c["FN"], c["TN"] + c["FP"]
n = ncan + nben
for i, v in ((0, ncan), (1, nben)):
ax.text(gx + (i + 0.5) * cw, gy - 0.065, str(v), ha="center", fontsize=12, weight="bold")
ax.text(gx + 2 * cw + 0.06, gy - 0.065, f"N = {n}", ha="center", fontsize=12, weight="bold")
ax.plot([gx, gx + 2 * cw + 0.115], [gy - 0.018] * 2, color=INK2, lw=0.8, transform=ax.transAxes)
ok = (ncan, nben, n) == (canc, ben, STATED_N)
ax.text(gx + 2 * cw + 0.115, gy - 0.145,
"= stated 54 / 38 / 92" if ok else
f"stated 54 / 38 / 92: +{ncan - canc}, +{nben - ben}, +{n - STATED_N}",
ha="right", fontsize=10, color=INK2)
se, sp = m["Sensitivity"], m["Specificity"]
ax.text(x0 + 0.022, 0.155, f"Sensitivity {se[3]} = {f1(se[0])}% ({f1(se[1])}–{f1(se[2])})",
fontsize=10.5)
ax.text(x0 + 0.022, 0.085, f"Specificity {sp[3]} = {f1(sp[0])}% ({f1(sp[1])}–{f1(sp[2])})",
fontsize=10.5, weight="bold")
ax.plot([0.5, 0.5], [0.06, 0.95], color=GRID, lw=1.0, transform=ax.transAxes)
ax.text(0.5, 0.018, "Shaded cells differ between the two tables. 95% CIs: Clopper–Pearson.",
ha="center", fontsize=10, color=INK2)
save(fig, path)
# --- design 4: dot-and-interval (default fig2)
def fig_dots(path):
rows = ["Sensitivity", "Specificity", "PPV", "NPV"]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.subplots_adjust(left=0.16, right=0.67, top=0.86, bottom=0.17)
for i, k in enumerate(rows):
y = len(rows) - 1 - i
e, lo, hi = TABLE1_PRINTED[k]
ax.plot([lo, hi], [y + 0.14] * 2, color=ORANGE, lw=LW, solid_capstyle="butt")
ax.plot(e, y + 0.14, "o", color=ORANGE, ms=8, mec=BG, mew=1.2)
r = m_impl[k]
ax.plot([r[1], r[2]], [y - 0.14] * 2, color=BLUE, lw=LW, solid_capstyle="butt")
ax.plot(r[0], y - 0.14, "s", color=BLUE, ms=8, mec=BG, mew=1.2)
ax.text(101.5, y + 0.14, f"{f1(e)} ({f1(lo)}–{f1(hi)}) {m_print[k][3]}", va="center", fontsize=10, color=INK)
ax.text(101.5, y - 0.14, f"{f1(r[0])} ({f1(r[1])}–{f1(r[2])}) {r[3]}", va="center", fontsize=10, color=INK)
ax.set_yticks(range(len(rows))); ax.set_yticklabels(rows[::-1], fontsize=11, color=INK)
ax.set_xlim(50, 100); ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("Estimate, % (95% CI); axis starts at 50%")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o-", color=ORANGE, ms=8, lw=LW, label="Reported (Table 1 cells, N = 101)")
ax.plot([], [], "s-", color=BLUE, ms=8, lw=LW, label="Recomputed (stated cohort, N = 92)")
ax.legend(loc="lower left", bbox_to_anchor=(-0.02, 1.0), ncol=2, frameon=False, fontsize=10,
handlelength=1.8, columnspacing=1.2)
save(fig, path)
# --- design 1: text vs table
def fig_text_vs_table(path):
fig = plt.figure(figsize=(7.2, 3.7))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
box(ax, 0.02, 0.30, 0.59, 0.62)
ax.text(0.04, 0.86, "Results text and Figure 4 legends", fontsize=11.5, weight="bold", va="center")
lines = [
("Discovery:", "“25 were diagnosed ... as being cancerous"),
("", "while 15 were benign”"),
("Validation:", "“23 non-cancerous and 29 OSCC lesions”"),
("Pooled:", "“For all the subjects (n = 92)”"),
("", "“eight false positives (FPs) and one FN”"),
]
for i, (a, b) in enumerate(lines):
y = 0.77 - i * 0.075
ax.text(0.04, y, a, fontsize=10, color=INK2, va="center")
ax.text(0.155, y, b, fontsize=10, va="center")
ax.add_patch(Rectangle((0.02, 0.30), 0.008, 0.62, fc=BLUE, ec="none", transform=ax.transAxes))
ax.text(0.04, 0.36, f"→ {canc} cancers + {ben} benign = {STATED_N}", fontsize=11, weight="bold", va="center")
box(ax, 0.69, 0.30, 0.29, 0.62)
ax.add_patch(Rectangle((0.69, 0.30), 0.008, 0.62, fc=ORANGE, ec="none", transform=ax.transAxes))
ax.text(0.71, 0.86, "Table 1", fontsize=11.5, weight="bold", va="center")
for i, (k, v) in enumerate(TABLE1.items()):
y = 0.77 - i * 0.075
ax.text(0.72, y, {"TP": "True positive", "TN": "True negative", "FP": "False positive",
"FN": "False negative"}[k], fontsize=10, va="center")
ax.text(0.955, y, str(v), fontsize=10.5, weight="bold", ha="right", va="center")
ax.text(0.72, 0.43, "Sensitivity 98.2", fontsize=10, va="center")
ax.text(0.72, 0.36, "Specificity 82.6", fontsize=10, va="center")
ax.annotate("", xy=(0.685, 0.61), xytext=(0.615, 0.61), xycoords="axes fraction",
arrowprops=dict(arrowstyle="<->", color=INK2, lw=LW))
ax.text(0.65, 0.655, "conflict", ha="center", fontsize=10, color=INK2)
ax.text(0.5, 0.17, f"Table 1 cells sum to 54 + 38 + 8 + 1 = {t1n}, not {STATED_N}: they imply "
f"{TABLE1['TP'] + TABLE1['FN']} cancers and {TABLE1['TN'] + TABLE1['FP']} benign lesions.",
ha="center", fontsize=10.5)
ax.text(0.5, 0.09, f"TP and TN equal the cohort totals ({canc}, {ben}), so the 8 FP and 1 FN are counted twice.",
ha="center", fontsize=10.5, color=INK2)
save(fig, path)
# --- design 2: calculation flow
def fig_flow(path):
fig = plt.figure(figsize=(7.2, 3.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
box(ax, 0.02, 0.33, 0.19, 0.32)
ax.text(0.115, 0.59, "Stated cohort", ha="center", fontsize=10.5, weight="bold")
ax.text(0.115, 0.51, f"{canc} cancers", ha="center", fontsize=10.5)
ax.text(0.115, 0.44, f"{ben} benign", ha="center", fontsize=10.5)
ax.text(0.115, 0.37, "8 FP, 1 FN", ha="center", fontsize=10, color=INK2)
paths = [
(0.60, ORANGE, "As reported in Table 1",
[f"TP = {canc}\nTN = {ben}", "FP 8, FN 1 added\non top: N = 101", f"Se 54/55 = {m_print['Sensitivity'][0]:.1f}%\nSp 38/46 = {m_print['Specificity'][0]:.1f}%"],
"cohort totals used\nas TP and TN"),
(0.08, BLUE, "Recomputed from the paper's counts",
[f"TP = {canc} − 1 = {implied['TP']}\nTN = {ben} − 8 = {implied['TN']}", "FP 8, FN 1 inside\nthe totals: N = 92",
f"Se 53/54 = {m_impl['Sensitivity'][0]:.1f}%\nSp 30/38 = {m_impl['Specificity'][0]:.1f}%"],
"errors taken out\nof the totals"),
]
xs = [0.28, 0.49, 0.70]
ws = [0.18, 0.18, 0.28]
h = 0.24
for y0, col, lab, steps, note in paths:
ax.add_patch(Rectangle((0.28, y0 + h + 0.035), 0.011, 0.055, fc=col, ec="none", transform=ax.transAxes))
ax.text(0.30, y0 + h + 0.062, lab, fontsize=10.5, weight="bold", va="center")
for j, (x, w, s) in enumerate(zip(xs, ws, steps)):
last = j == len(steps) - 1
box(ax, x, y0, w, h, ec=col if last else INK2, lw=LW if last else 0.8)
ax.text(x + w / 2, y0 + h / 2, s, ha="center", va="center", fontsize=10,
weight="bold" if last else "normal", linespacing=1.4)
if j:
ax.annotate("", xy=(x, y0 + h / 2), xytext=(xs[j - 1] + ws[j - 1], y0 + h / 2),
xycoords="axes fraction", arrowprops=dict(arrowstyle="->", color=INK2, lw=1.0))
ax.annotate("", xy=(xs[0], y0 + h / 2), xytext=(0.21, 0.49), xycoords="axes fraction",
arrowprops=dict(arrowstyle="->", color=INK2, lw=1.0, connectionstyle="arc3,rad=0"))
save(fig, path)
# --- design 5: icon array by cell
def fig_icons(path):
fig = plt.figure(figsize=(7.2, 3.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 72); ax.set_ylim(0, 30)
s, gap, per = 1.0, 0.25, 30
rows = [("TP", implied["TP"], TABLE1["TP"]), ("TN", implied["TN"], TABLE1["TN"]),
("FP", implied["FP"], TABLE1["FP"]), ("FN", implied["FN"], TABLE1["FN"])]
y = 24.0
for lab, n_ok, n_tab in rows:
ax.text(1.0, y - 0.6, lab, fontsize=11, weight="bold", va="center")
for k in range(n_tab):
cx, cy = 5 + (k % per) * (s + gap), y - (k // per) * (s + gap) - 1
extra = k >= n_ok
ax.add_patch(Rectangle((cx, cy), s, s, fc=BG if extra else BLUE,
ec=ORANGE if extra else BLUE, lw=1.2,
hatch="////" if extra else None))
ax.text(5 + per * (s + gap) + 1.0, y - 0.6,
f"Table 1: {n_tab} stated cohort: {n_ok}" + (f" (+{n_tab - n_ok})" if n_tab > n_ok else ""),
fontsize=10.5, va="center")
y -= 2.5 + 2.5 * ((n_tab - 1) // per)
ax.add_patch(Rectangle((5, 3.6), s, s, fc=BLUE, ec=BLUE))
ax.text(6.6, 4.1, "subject counted in both (Table S1 label; recomputed)", fontsize=10, va="center")
ax.add_patch(Rectangle((5, 1.4), s, s, fc=BG, ec=ORANGE, lw=1.2, hatch="////"))
ax.text(6.6, 1.9, "extra count present only in Table 1 as reported (9 in all: N = 101 vs 92)",
fontsize=10, va="center")
ax.text(1.0, 28.0, "Each square is one count in the 2×2 table", fontsize=11.5, weight="bold", va="center")
save(fig, path)
fig_two_tables(os.path.join(HERE, "fig1"))
fig_dots(os.path.join(HERE, "fig2"))
fig_text_vs_table(os.path.join(CAND, "c1_text_vs_table"))
fig_flow(os.path.join(CAND, "c2_calculation_flow"))
fig_two_tables(os.path.join(CAND, "c3_two_tables"))
fig_dots(os.path.join(CAND, "c4_dot_interval"))
fig_icons(os.path.join(CAND, "c5_icon_array"))
print("\nFigures written to", HERE, "and", CAND)
# ------------------------------------------------------------------ contact sheet of the five designs
def contact_sheet(path):
import matplotlib.image as mpimg
names = [("c1_text_vs_table", "1. Text vs table: conflicting excerpts joined by a connector"),
("c2_calculation_flow", "2. Calculation flow: how 54 / 38 became TP / TN"),
("c3_two_tables", "3. Two 2×2 tables side by side (used as fig1)"),
("c4_dot_interval", "4. Dot-and-interval plot of the four estimates (used as fig2)"),
("c5_icon_array", "5. Icon array: one square per count, extra counts hatched")]
imgs = [mpimg.imread(os.path.join(CAND, n + ".png")) for n, _ in names]
hs = [im.shape[0] / im.shape[1] * 7.2 for im in imgs]
fig = plt.figure(figsize=(7.2, sum(hs) + 0.45 * len(imgs)))
y = 1.0
H = sum(hs) + 0.45 * len(imgs)
for im, h, (_, lab) in zip(imgs, hs, names):
y -= 0.45 / H
fig.text(0.01, y + 0.12 / H, lab, fontsize=11, weight="bold", color=INK)
ax = fig.add_axes([0, y - h / H, 1, h / H]); ax.imshow(im); ax.axis("off")
y -= h / H
fig.savefig(path + ".png", dpi=110)
plt.close(fig)
contact_sheet(os.path.join(CAND, "all_five_designs"))
print("Contact sheet written to", os.path.join(CAND, "all_five_designs.png"))
Adult–child FP–salience difference called significant after FDR, but Table 3 gives P = 0.130
Cereb Cortex (2026) · doi:10.1093/cercor/bhag167 · PMC13641069
Summary
This fMRI study compares functional connectivity in 15 children and 16 adults during a conversation with ChatGPT. The Abstract names FP–salience as one of the two main between-network couplings that are stronger in adults, and the Results say that the FP–salience and FP–DAN findings "remained significant" after Benjamini–Hochberg FDR correction at q = .05. The paper's own Table 3 reports the FP–salience group effect as F(1,27) = 2.434, P = 0.130, which is not significant even before correction, and the table does not mark that row as significant. Recomputing the test from the printed F reproduces P = 0.130. A Benjamini–Hochberg correction over the 21 between-network tests leaves only FP–DAN significant (adjusted p = 0.024), with FP–salience at 0.228. The adult-greater FP–salience coupling in the Abstract therefore has no statistical support in the paper's reported analysis.
What the paper reports
Abstract:
Adults showed stronger within-network connectivity in control/attention systems, most robustly in FP, with additional effects in salience and DAN. Adults also exhibited greater cross-network integration, particularly FP-salience and FP-DAN couplings, alongside broader adult-greater effects among memory- and attention-related pairs.
Abstract, final sentence:
This highlights FP-centric pathways and their salience-gated interactions as candidate substrates by which creativity supports goal-directed co-creative dialog with AI.
Results, "Neuroimaging measures", between-networks analysis, third paragraph:
Overall, these results suggest a broader and more integrated pattern of between-networks connectivity in adults relative to children, particularly involving the FP, DAN,VAN, memory, and salience systems. After correcting for multiple comparisons (FDR correction (Benjamini and Hochberg 1995), Benjamini–Hochberg, q = .05), the results for the FP-salience and FP-DAN findings remained significant (Table 3). See also Fig. 2.
Results, between-networks analysis, second paragraph (the last of the pairs the text lists as significant; FP–salience is not among them):
Finally, significant group effects were found for connectivity between the FP-VAN, F(1,27) = 5.820, P = 0.023, η2P = .177, and FP-DAN, F(1,27) = 13.252, P = 0.001, η2P = .329, with adults again showing higher connectivity.
Table 3 ("Between networks, functional connectivity of EF, attention, and sensory after controlling for FD"), FP–Salience row, and the table footnote:
FP–Salience | 0.138 ± 0.040 | 0.180 ± 0.060 | F = 2.434, P = 0.130, η2P = .083 | A > C
Bold represents significant results.
Methods, "Statistical analyses":
a multivariate analysis of variance (MANOVA) was conducted with group (children, adults) as a between-subjects factor and framewise displacement (FD) as a covariate to control for head motion.
Results, "Moderation results" (a different test on the same pair):
Specifically, significant moderation effects emerged for the FP–Salience, F(1,26) = 5.869, P = 0.023, partial η2P = .184, and Salience–VAN, F(1,26) = 4.575, P = 0.042, partial η2P = .150, pairs, indicating that the relationship between creativity and connectivity strength differed between children and adults for these networks.
Discussion:
Children also demonstrated reduced functional connectivity between the selected cognitive networks (FP-salience, FP-DAN, DMN-memory).
Results, "Power analysis":
Our achieved sample (children, n = 15; adults, n = 16) is therefore well-powered for large effects, consistent with prior literature, but underpowered for moderate effects, guiding the interpretation of null or small effects accordingly.
What the paper's own numbers show
Method. Each row of Table 3 is a group effect with one numerator degree of freedom, so P = Pr(F(1, 27) ≥ F) and partial η² = F / (F + 27). I recomputed both from the printed F for all 21 between-network pairs and compared them with the printed values. I then applied the Benjamini–Hochberg step-up procedure at q = .05, the correction the paper names, to the 21 between-network P values (the family of Table 3). The adjusted p for the test ranked i of m is the minimum over j ≥ i of m·p(j)/j. I repeated the correction with the printed P values, with the 7 within-network tests of Table 2 as their own family, and with all 28 tests pooled. Individual-level data are not available, so the ANCOVA itself cannot be refitted. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| FP–salience group effect, P | Table 3 | F(1,27) = 2.434, P = 0.130 | P = 0.130 (0.1304); not significant before correction |
| FP–salience partial η² | Table 3 | .083 | 2.434 / 29.434 = 0.083 |
| FP–salience, one-sided P | derived | – | 0.065, still above .05 |
| FP–salience after BH, 21 between-network tests | Results ("remained significant"); Abstract | significant at q = .05 | rank 12 of 21, critical value 0.0286, adjusted p = 0.228 |
| FP–salience after BH, printed P values | derived | – | 21 × 0.130 / 12 = 0.228 |
| FP–salience after BH, 28 tests pooled | derived | – | adjusted p = 0.243 |
| FP–salience after BH, any family size | derived | – | a BH-adjusted p is never below the raw p, so P = 0.130 fails for every family |
| FP–DAN group effect, P | Table 3; Results | P < 0.001 (Table 3); P = 0.001 (Results) | 0.0011 |
| FP–DAN after BH, 21 tests | Results ("remained significant") | significant | adjusted p = 0.024: significant, the only pair that is |
| Next-smallest BH-adjusted p (CO–salience and seven others) | derived | – | 0.062; no other between-network pair survives |
| DMN–memory | Table 3; Discussion | P = 0.017, listed among reduced couplings | adjusted p = 0.062 |
| Agreement of all printed P and η² with their F | Table 3 | 21 rows | all 21 within rounding of F (CO–DMN: F = 0.006 gives 0.936–0.941, printed 0.937) |
| Rows printed in bold | Table 3 (publisher XML) | 9 bold rows | exactly the 9 rows with P < .05; FP–salience not bold |
| Within-FP group effect after BH, 7 within-network tests | Table 2; Results ("the FP findings remained significant") | significant | P = 0.006, adjusted p = 0.044: significant |
| Within-salience and within-DAN after BH, 7 tests | Table 2; Abstract ("additional effects") | P = 0.025, 0.027 | adjusted p = 0.062 each |
| FP–salience group × creativity interaction | Results, Moderation | F(1,26) = 5.869, P = 0.023, η² = .184 | P = 0.023, η² = 0.184; a test of slopes, not of group means |
The table is internally consistent. Every printed P and η² follows from its F, and the bold marking matches uncorrected P < .05. The only statement that the numbers do not support is the sentence saying FP–salience "remained significant" after FDR, and the Abstract and Discussion statements that rest on it.
Figure
Figure 1. For each of the 21 between-network pairs in Table 3, the orange dot is the uncorrected P printed in the table and the blue diamond is the Benjamini–Hochberg adjusted p recomputed over the 21 tests from the printed F(1,27) values. Pairs are ordered by P. The axis is logarithmic and the dashed line marks q = .05. FP–DAN is printed as P < 0.001 and drawn at 0.001 (left-pointing marker); its recomputed P is 0.0011. Only FP–DAN has an adjusted p below .05 (0.024). FP–salience, which the Results describe as remaining significant after FDR, has a printed P of 0.130 and an adjusted p of 0.228, the twelfth of 21. Eight pairs with uncorrected P between 0.008 and 0.026 share an adjusted p of 0.062.
Figure 2. The Abstract and the Results text (left) present FP–salience, together with FP–DAN, as a significant adult-greater coupling after Benjamini–Hochberg correction. The Table 3 row the text cites (right) gives F = 2.434, P = 0.130 before any correction, and the row is not bold, although the footnote says bold marks significant results. Recomputing from F(1,27) = 2.434 gives the same P and η², and a BH-adjusted p of 0.228.
Why it matters
FP–salience is one of the two between-network couplings that the Abstract names as greater in adults, and the Abstract's closing sentence builds on it ("FP-centric pathways and their salience-gated interactions"). The Discussion lists it first among the couplings that are reduced in children. A reader of the Abstract would take it as a corrected, significant group difference. The paper's own test gives P = 0.130 for that difference, and after the correction the paper describes, only FP–DAN remains among the 21 between-network pairs.
The paper's main conclusion does not depend on this result alone. The within-FP group difference (P = 0.006, BH-adjusted 0.044 across the 7 within-network tests) and the FP–DAN coupling (adjusted 0.024 across 21 tests) both survive the correction the paper names, so lower FP engagement in children still has support. The "salience-gated" framing loses its group-difference evidence. What remains for FP–salience is the group × creativity interaction (P = 0.023), which is uncorrected and tests whether the creativity slope differs between groups, not whether adults have higher connectivity. Two other statements also rest on uncorrected results: the Discussion's DMN–memory coupling (P = 0.017, adjusted 0.062) and the Abstract's "additional effects in salience and DAN" within networks (P = 0.025 and 0.027, adjusted 0.062 each). The Abstract does not say these were corrected, but a reader is not told that they do not survive the correction applied elsewhere.
Explanations considered
- A conversion or typesetting error in the table. This is excluded. The printed F, P and η² agree with one another (F = 2.434 gives P = 0.1304 and η² = 0.083), the row is not bold in the publisher-supplied XML, and the Results paragraphs that list the uncorrected significant pairs do not include FP–salience. The publisher page, PMC and Europe PMC carry the same values.
- The FDR sentence uses a different family or ordering. This does not account for the numbers. A Benjamini–Hochberg adjusted p can never be smaller than the raw p, so a raw P of 0.130 cannot pass q = .05 in any family. With the 21 Table 3 tests the adjusted p is 0.228; pooled with the 7 within-network tests it is 0.243.
- A one-sided test. This does not account for the numbers. Halving P gives 0.065, which is above .05 before any correction.
- The FDR sentence refers to an analysis without the motion covariate. This does not account for the numbers. A Student t test from the Table 3 means and SDs (n = 15 and 16) gives P = 0.030 for FP–salience. Across the 18 pairs whose descriptives can be used, BH places it fifth (critical value 0.0139) with an adjusted p of 0.109, so it would still not be significant. The Salience–VAN, Salience–DAN and VAN–DAN rows were left out of this check because their printed SDs (0.000–0.007) with mean differences of 0.135–0.184 cannot be reconciled with their F values below 2. That is a separate inconsistency in those three rows and does not bear on FP–salience.
- The sentence refers to the moderation result. This does not account for the claim. The FP–salience group × creativity interaction is significant before correction (F(1,26) = 5.869, P = 0.023, reproduced), but it tests whether the association with creativity differs between groups. The sentence in question sits in the group-difference section, cites Table 3, and pairs FP–salience with FP–DAN, which has no reported interaction. The Abstract states the result as "greater cross-network integration" in adults.
- "FP-salience" was written where another FP pair was meant. This does not rescue the sentence. Every other between-network pair, including CO–salience, FP–VAN and Memory–FP, has a BH-adjusted p of 0.062 or more, so no substitution produces a second surviving pair. A substitution would also leave the Abstract and Discussion statements about FP–salience unsupported.
- Degrees of freedom. This does not change anything. With 31 participants, a model with group and FD as predictors has 28 error degrees of freedom, while the tables print 27; the η² values match 27. With 28, FP–salience gives P = 0.130 as well.
Status
- Version of record. The publisher page (academic.oup.com, reached through doi.org/10.1093/cercor/bhag167), the PMC article and the Europe PMC full-text XML were compared. All three carry the same Abstract, Results sentences and Table 3 values quoted above. In the Europe PMC XML the bold rows of Table 3 are CO–FP, CO–Salience, DMN–Memory, DMN–FP, Memory–FP, Memory–VAN, Memory–DAN, FP–VAN and FP–DAN; the FP–Salience row is not bold. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. The article has one supplementary file (supp_bhag167.docx). It holds a title page and the two ChatGPT prompts used for children and adults. It has no connectivity data, no FDR-adjusted P values and no source data. The article mentions no data deposit, so the ANCOVA cannot be refitted.
- Corrections and comments. As of 2026-10-24 no correction, erratum, expression of concern or retraction was found. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. A web search for corrections or comments on the article found none. The PubPeer API and search pages refused automated access (Cloudflare challenge, HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC13641069; Europe PMC full-text XML and supplementary files for PMC13641069; the publisher's article page at academic.oup.com; Crossref; Europe PMC.
Suggested correction
- In the Results, state that after Benjamini–Hochberg correction at q = .05 only the FP–DAN between-network difference remained significant, and remove FP–salience from that sentence. Its group effect was F(1,27) = 2.434, P = 0.130 (adjusted p ≈ 0.23 over the 21 between-network tests).
- In the Abstract, remove FP–salience from the couplings described as greater in adults, or describe it as not significant. Revise the closing sentence so that the "salience-gated" interpretation rests only on the exploratory group × creativity interaction, and say that this interaction is uncorrected.
- In the Discussion, revise "reduced functional connectivity between the selected cognitive networks (FP-salience, FP-DAN, DMN-memory)" so that only FP–DAN is described as surviving correction.
- Report the FDR-adjusted P values for Tables 2 and 3, and state which within-network and between-network effects in the Abstract are uncorrected (within-network salience and DAN; the "broader adult-greater effects" among memory- and attention-related pairs).
- Minor: report FP–DAN consistently (P = 0.001 in the Results, P < 0.001 in Table 3; the F gives 0.0011), state the error degrees of freedom for 31 participants, and check the means and SDs printed for Salience–VAN, Salience–DAN and VAN–DAN, which cannot be reconciled with their F values.
Recomputation script (python recompute.py.txt)
"""Recompute the between-network tests of PMC13641069
(Cereb Cortex 2026, doi:10.1093/cercor/bhag167, "Lower engagement of cognitive control in children on ChatGPT").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 2 (within-network, 7 tests) and Table 3 (between-network, 21 tests): group means +/- SD,
ANCOVA F(1,27), P and partial eta-squared, group effect controlling for framewise displacement.
Methods, Participants: 15 children and 16 adults.
Results, Moderation: FP-Salience group x creativity interaction F(1,26) = 5.869, P = 0.023, eta2p = .184.
Method: P = Pr(F(1,df2) >= F). Partial eta-squared = F / (F + df2) for a 1-df effect.
Benjamini-Hochberg (BH) step-up adjusted p: p_adj(i) = min over j >= i of (m * p_(j) / j), capped at 1.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
N_CHILD, N_ADULT = 15, 16
DF2 = 27 # as printed in Tables 2 and 3
Q = 0.05
# ---------------------------------------------------------------- inputs (from the paper)
# Table 3: pair, child mean, child SD, adult mean, adult SD, F, printed P (None = "< 0.001"), printed eta2p, bold in table
T3 = [
("CO-DMN", .133, .070, .119, .123, 0.006, 0.937, .000, False),
("CO-Memory", .015, .096, .085, .110, 3.453, 0.074, .113, False),
("CO-FP", .043, .074, .105, .052, 5.628, 0.025, .173, True),
("CO-Salience", .109, .076, .207, .079, 8.182, 0.008, .233, True),
("CO-VAN", .165, .123, .182, .103, 0.311, 0.582, .011, False),
("CO-DAN", .121, .077, .116, .086, 0.494, 0.488, .018, False),
("DMN-Memory", .111, .045, .150, .091, 6.491, 0.017, .194, True),
("DMN-FP", .090, .039, .115, .065, 5.570, 0.026, .171, True),
("DMN-Salience", .111, .060, .115, .081, 0.480, 0.494, .017, False),
("DMN-VAN", .196, .072, .188, .091, 0.015, 0.902, .001, False),
("DMN-DAN", .116, .073, .118, .086, 0.896, 0.352, .032, False),
("Memory-FP", .079, .064, .104, .096, 6.449, 0.017, .193, True),
("Memory-Salience", .049, .059, .079, .103, 3.990, 0.056, .129, False),
("Memory-VAN", .091, .073, .133, .152, 5.514, 0.026, .170, True),
("Memory-DAN", .086, .076, .144, .100, 5.944, 0.022, .180, True),
("FP-Salience", .138, .040, .180, .060, 2.434, 0.130, .083, False),
("FP-VAN", .095, .059, .167, .088, 5.820, 0.023, .177, True),
("FP-DAN", .039, .059, .126, .043, 13.252, None, .329, True),
("Salience-VAN", .012, .000, .189, .007, 1.687, 0.205, .059, False),
("Salience-DAN", .010, .005, .145, .005, 1.824, 0.188, .063, False),
("VAN-DAN", .002, .007, .186, .007, 0.338, 0.566, .012, False),
]
# Table 2: network, F, printed P, printed partial eta2
T2 = [
("Cingulo-Opercular", 0.006, 0.940, .000),
("Frontoparietal (FP)", 8.773, 0.006, .245),
("Memory", 1.938, 0.175, .067),
("Default Mode (DMN)", 0.497, 0.487, .018),
("Salience", 5.653, 0.025, .173),
("Dorsal Attention (DAN)", 5.487, 0.027, .169),
("Ventral Attention (VAN)", 1.562, 0.222, .055),
]
MOD_F, MOD_DF2, MOD_P, MOD_ETA = 5.869, 26, 0.023, .184
def bh_adjust(p):
p = np.asarray(p, float)
m = len(p)
order = np.argsort(p)
adj = np.empty(m)
prev = 1.0
for rank in range(m, 0, -1):
i = order[rank - 1]
prev = min(prev, p[i] * m / rank)
adj[i] = prev
ranks = np.empty(m, int)
ranks[order] = np.arange(1, m + 1)
return adj, ranks
names = [r[0] for r in T3]
iFS = names.index("FP-Salience")
iFD = names.index("FP-DAN")
print("=== 1. Table 3: P and partial eta-squared recomputed from each printed F(1,27) ===")
pF = np.array([stats.f.sf(r[5], 1, DF2) for r in T3])
print(f"{'pair':16s} {'F':>7s} {'P_printed':>9s} {'P_from_F':>9s} {'eta_prn':>7s} {'eta_F':>6s} {'bold':>5s}")
for r, p in zip(T3, pF):
prn = "<0.001" if r[6] is None else f"{r[6]:.3f}"
print(f"{r[0]:16s} {r[5]:7.3f} {prn:>9s} {p:9.4f} {r[7]:7.3f} {r[5]/(r[5]+DF2):6.3f} {str(r[8]):>5s}")
def p_range(F): # P over the rounding interval of a 3-decimal F
return stats.f.sf(F + 0.0005, 1, DF2), stats.f.sf(max(F - 0.0005, 0), 1, DF2)
mism = [r[0] for r in T3 if r[6] is not None and not (p_range(r[5])[0] - 0.0005 <= r[6] <= p_range(r[5])[1] + 0.0005)]
print("rows whose printed P lies outside the P range of the rounded F:", mism or "none")
lo, hi = p_range(0.006)
print(f"CO-DMN: F = 0.006 (0.0055-0.0065) gives P = {lo:.4f}-{hi:.4f}; printed 0.937")
bold_vs_p = all((p < 0.05) == r[8] for r, p in zip(T3, pF))
print("bold rows are exactly the rows with P(F) < .05:", bold_vs_p)
print(f"FP-Salience: P(F(1,27)=2.434) = {pF[iFS]:.4f}; with df2=28: {stats.f.sf(2.434, 1, 28):.4f}; "
f"eta2p = 2.434/29.434 = {2.434/29.434:.4f} (df2=28 would give {2.434/30.434:.4f})")
print(f"FP-Salience one-sided P (half of the two-sided F test) = {pF[iFS]/2:.4f}")
print(f"FP-DAN: P(F(1,27)=13.252) = {pF[iFD]:.5f} (printed 'P < 0.001' in Table 3, 'P = 0.001' in Results)")
print(f"Error df implied by 31 participants, group + FD covariate: {N_CHILD + N_ADULT - 3} (paper prints 27)")
print("\n=== 2. Benjamini-Hochberg over the 21 between-network tests (q = .05), P from F ===")
adj21, rk21 = bh_adjust(pF)
for rank in range(1, 22):
i = int(np.where(rk21 == rank)[0][0])
print(f"rank {rank:2d} {names[i]:16s} p={pF[i]:.5f} crit={Q*rank/21:.5f} p_BH={adj21[i]:.4f} {'SIG' if adj21[i] <= Q else ''}")
print(f"FP-Salience: rank {rk21[iFS]}, critical value {Q*rk21[iFS]/21:.4f}, BH-adjusted p = {adj21[iFS]:.4f}")
print(f"FP-DAN: rank {rk21[iFD]}, BH-adjusted p = {adj21[iFD]:.4f}")
print("pairs significant after BH:", [names[i] for i in range(21) if adj21[i] <= Q])
others = sorted(adj21[i] for i in range(21) if i != iFD)
print(f"smallest BH-adjusted p among the other 20 pairs: {others[0]:.4f}")
iDM = names.index("DMN-Memory")
print(f"DMN-Memory: raw p = {pF[iDM]:.4f}, BH-adjusted p = {adj21[iDM]:.4f}")
# Same with the printed P values (FP-DAN printed as < 0.001 -> use 0.001)
p_prn = np.array([0.001 if r[6] is None else r[6] for r in T3])
adjp, rkp = bh_adjust(p_prn)
print(f"Using printed P values instead: FP-Salience BH-adjusted p = {adjp[iFS]:.4f} "
f"(= 21 x 0.130 / 12 = {21*0.130/12:.4f}); FP-DAN = {adjp[iFD]:.4f}")
print("\n=== 3. Family-size sensitivity for FP-Salience ===")
print("BH-adjusted p is never smaller than the raw p, so raw p = 0.130 > q = .05 fails for any family.")
for m in (1, 2, 7, 21, 28):
print(f" best case (FP-Salience ranked last of m={m:2d}, all others smaller): adjusted p = {pF[iFS]:.4f}")
# within-network family (Table 2) and the pooled 28-test family
p2 = np.array([stats.f.sf(r[1], 1, DF2) for r in T2])
adj7, rk7 = bh_adjust(p2)
print("\nWithin-network (Table 2), BH over 7 tests:")
for r, p, a, k in zip(T2, p2, adj7, rk7):
print(f" {r[0]:24s} F={r[1]:6.3f} P_printed={r[2]:.3f} P_from_F={p:.4f} eta_F={r[1]/(r[1]+DF2):.3f} rank={k} p_BH={a:.4f} {'SIG' if a <= Q else ''}")
pool = np.concatenate([p2, pF])
adj28, _ = bh_adjust(pool)
pool_names = [r[0] for r in T2] + names
print("Pooled 28 tests, significant after BH:", [n for n, a in zip(pool_names, adj28) if a <= Q])
print(f" FP-Salience pooled BH-adjusted p = {adj28[7 + iFS]:.4f}; FP-DAN = {adj28[7 + iFD]:.4f}; "
f"within-FP = {adj28[1]:.4f}")
print("\n=== 4. Innocent-explanation checks ===")
print("(a) Unadjusted Student t test from Table 3 means/SDs (n = 15 / 16), i.e. ignoring the FD covariate:")
pt = np.full(21, np.nan)
BAD = ("Salience-VAN", "Salience-DAN", "VAN-DAN")
for k, r in enumerate(T3):
if r[0] not in BAD:
t, p = stats.ttest_ind_from_stats(r[3], r[4], N_ADULT, r[1], r[2], N_CHILD, equal_var=True)
pt[k] = p
print(f" {r[0]:16s} t={t:8.3f} p={p:.4f} (ANCOVA P from F = {pF[k]:.4f})")
else:
print(f" {r[0]:16s} excluded: printed SDs of 0.000-0.007 are not consistent with its F")
ok = ~np.isnan(pt)
adjt, rkt = bh_adjust(pt[ok])
idx_ok = np.where(ok)[0]
j = int(np.where(idx_ok == iFS)[0][0])
m_t = int(ok.sum())
print(f" FP-Salience t-test p = {pt[iFS]:.4f}; BH over {m_t} t tests (three rows excluded): rank {rkt[j]}, "
f"critical {Q*rkt[j]/m_t:.4f}, adjusted {adjt[j]:.4f} -> {'SIG' if adjt[j] <= Q else 'not significant'}")
print(" Salience-VAN, Salience-DAN, VAN-DAN: mean differences of 0.177, 0.135 and 0.184 with SDs of 0.000-0.007")
print(" cannot give F < 2; these three rows' descriptives are not consistent with their F tests (not used further).")
print("(b) FP-Salience moderation (group x creativity), Results 'Moderation results':")
print(f" P(F(1,26)=5.869) = {stats.f.sf(MOD_F, 1, MOD_DF2):.4f} (printed {MOD_P}); eta2p = {MOD_F/(MOD_F+MOD_DF2):.3f} (printed {MOD_ETA})")
print(" This tests whether the creativity slope differs between groups, not whether the group means differ.")
print(f"(c) Descriptive FP-Salience difference, adults - children = {0.180-0.138:.3f} (direction A > C matches the table)")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK, "text.color": INK, "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: printed P (orange) and recomputed BH-adjusted p (blue) for all 21 pairs, one log axis
order = np.argsort(pF) # smallest p at the top
fig, ax = plt.subplots(figsize=(7.2, 6.6))
y = np.arange(21)[::-1]
for yy, i in zip(y, order):
prn = 0.001 if T3[i][6] is None else T3[i][6]
hl = i in (iFS, iFD)
if hl:
ax.axhspan(yy - 0.45, yy + 0.45, color="#f1efe9", zorder=0, lw=0)
ax.plot([prn, adj21[i]], [yy, yy], color="#b9b7b1", lw=2, zorder=1, solid_capstyle="butt")
ax.scatter(prn, yy, s=64, color=ORANGE, zorder=3, edgecolor=BG, linewidth=1.2,
marker="<" if T3[i][6] is None else "o")
ax.scatter(adj21[i], yy, s=64, color=BLUE, zorder=3, edgecolor=BG, linewidth=1.2, marker="D")
ax.axvline(Q, color=INK2, lw=1, ls="--", zorder=2)
ax.text(Q, 21.05, " q = .05", color=INK2, fontsize=10, va="bottom", ha="left")
ax.set_xscale("log")
ax.set_xlim(5e-4, 1.6)
ax.set_ylim(-0.7, 21.6)
ax.set_yticks(y)
ax.set_yticklabels([names[i].replace("-", "–") for i in order])
for lab, i in zip(ax.get_yticklabels(), order):
if i in (iFS, iFD):
lab.set_fontweight("bold")
ax.set_xticks([0.001, 0.01, 0.05, 0.1, 0.5, 1])
ax.set_xticklabels(["0.001", "0.01", "0.05", "0.1", "0.5", "1"])
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("p value (log scale)")
yFS = y[list(order).index(iFS)]
yFD = y[list(order).index(iFD)]
ax.annotate("Results text: “remained significant”\nafter FDR. Table 3: P = 0.130;\nrecomputed BH-adjusted p = %.3f" % adj21[iFS],
xy=(pF[iFS] / 1.2, yFS), xytext=(0.00062, yFS - 3.3), fontsize=10, color=INK, ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.annotate("only pair below q after BH\n(p_BH = %.3f)" % adj21[iFD],
xy=(adj21[iFD] * 1.15, yFD), xytext=(0.075, yFD - 0.15), fontsize=10, color=INK, ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
leg = [Line2D([], [], marker="o", ls="", color=ORANGE, ms=8, label="P printed in Table 3"),
Line2D([], [], marker="D", ls="", color=BLUE, ms=8, label="BH-adjusted p, recomputed (21 tests)")]
ax.legend(handles=leg, loc="lower left", bbox_to_anchor=(-0.32, 1.035), ncol=2, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
fig.text(0.012, 0.012, "Pairs ordered by P recomputed from F(1,27). FP–DAN is printed as P < 0.001 (drawn at 0.001).",
fontsize=9.5, color=INK2)
fig.tight_layout(rect=(0, 0.03, 1, 1))
save(fig, "fig1")
# Figure 2: text-versus-table contradiction
fig = plt.figure(figsize=(7.2, 4.3))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.3)
ax.axis("off")
def panel(x, w, title, lines, y0=0.45, h=3.15):
ax.add_patch(FancyBboxPatch((x, y0), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="#ffffff", ec="#c9c7c1", lw=1))
ax.text(x + 0.15, y0 + h - 0.22, title, fontsize=11, fontweight="bold", va="top", color=INK)
yy = y0 + h - 0.62
for txt, col, mk in lines:
tx = x + 0.15
if mk:
ax.scatter([x + 0.22], [yy - 0.09], s=55, marker=mk[0], color=mk[1], zorder=5, edgecolor="none")
tx = x + 0.38
ax.text(tx, yy, txt, fontsize=10, va="top", color=col)
yy -= 0.205 * (txt.count("\n") + 1) + 0.13
panel(0.15, 3.25, "Abstract and Results text", [
("Abstract:", INK2, None),
("“Adults also exhibited greater cross-\nnetwork integration, particularly\nFP-salience and FP-DAN couplings”", INK, None),
("Results, between-networks analysis:", INK2, None),
("“...the results for the FP-salience\nand FP-DAN findings remained\nsignificant (Table 3).”", INK, None),
])
panel(3.8, 3.25, "Table 3, FP–Salience row", [
("Printed in Table 3:", INK2, None),
("children 0.138 ± 0.040\nadults 0.180 ± 0.060\nF = 2.434, P = 0.130, η²p = .083\nrow not bold (“Bold represents\nsignificant results”)", INK, ("o", ORANGE)),
("Recomputed from F(1,27) = 2.434:", INK2, None),
("P = %.3f, η²p = %.3f\nBH-adjusted p = %.3f\n(rank %d of 21; only FP–DAN passes)" % (pF[iFS], 2.434 / 29.434, adj21[iFS], rk21[iFS]), INK, ("D", BLUE)),
])
ax.add_patch(FancyArrowPatch((3.42, 2.05), (3.78, 2.05), arrowstyle="<->", mutation_scale=12, color=INK2, lw=1.5))
ax.text(3.6, 3.95, "Conflict: the text calls FP–salience significant after BH-FDR (q = .05);\n"
"the table it cites gives P = 0.130 before any correction.",
ha="center", va="center", fontsize=10, color=INK)
ax.scatter([0.25], [0.2], s=55, marker="o", color=ORANGE, edgecolor="none")
ax.text(0.38, 0.2, "printed in the paper", fontsize=10, color=INK2, va="center")
ax.scatter([2.15], [0.2], s=55, marker="D", color=BLUE, edgecolor="none")
ax.text(2.28, 0.2, "recomputed from the paper’s own F", fontsize=10, color=INK2, va="center")
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png")
Population odds ratio of 45.2 for the RE5 variant does not follow from the stated counts
Circulation (2024) · doi:10.1161/CIRCULATIONAHA.124.069041 · PMC11670919
Summary
The paper reports that a rare SCN5A enhancer variant (RE5) was carried by 9 of 231 Thai patients with Brugada syndrome (BrS) and by 6 of 10 043 individuals in the Genomics Thailand database, and gives an odds ratio of 45.2 (95% CI 14.5–141.3, P = 5.6E-12) for that comparison; the Abstract cites 45.2 as the variant's odds ratio. From the stated counts the odds ratio is 67.8 (95% CI 23.9–192.2), by the same Wald method that reproduces the paper's other odds ratio (20.2 against study controls) to every digit. The printed P value matches the stated counts exactly, whereas the printed OR and CI match exactly a table with 6 case carriers instead of 9. The error understates the enrichment by about one third in the Abstract, Results and Discussion, but it does not change the direction, the significance or the paper's conclusions.
What the paper reports
Abstract, Results:
A rare noncoding variant in an SCN5A intronic enhancer region was highly enriched in patients with BrS (detected in 3.9% of cases with a case-control odds ratio of 45.2).
Results, "Identification of Rare Noncoding Variants at SCN5A–SCN10A Locus", first paragraph (the denominators):
After sample-level quality control, a total of 231 unrelated probands and 500 ancestry-matched controls were included, with principal component analysis demonstrating the cohorts were of comparable ancestral background (Figure 1B).
Same section, fourth paragraph:
The RE5 variant was detected in 3.9% (n=9) of cases versus 0.2% (n=1) of controls (P=2.0E-04; OR, 20.2 [2.5–160.6]). We searched the Genomics Thailand database, comprising 10 043 individuals with a range of clinical and pharmacogenetics indications (https://genomicsthailand.com/Genomic/home), to attain a more precise estimate of the variant frequency in Thailand. It was detected in 6 individuals (0.06%), yielding an OR of 45.2 (14.5–141.3) compared with the BrS cohort (P=5.6E-12).
Methods, "Genetic Variant Analysis" (the test used):
Noncoding variants at the SCN5A–SCN10A locus were assessed for enrichment in cases versus controls using Fisher exact test (for samples with or without variants).
Discussion, first paragraph:
The variant is highly enriched in patients with BrS, with an OR between 20 and 45 based on our study controls and the Genomics Thailand database, respectively, and is located within a characterized enhancer region for SCN5A (RE5).
What the paper's own numbers show
Method. Following the Methods, each comparison is a 2×2 table of carriers and non-carriers. I computed the crude odds ratio a·d/(b·c) with a 95% Wald (Woolf) interval, exp(ln OR ± 1.96·√(1/a + 1/b + 1/c + 1/d)), and the two-sided Fisher exact P value; for completeness, also the conditional maximum-likelihood OR with its exact CI. I first checked the method on the paper's study-control comparison, then applied it to the Genomics Thailand comparison. I then tried the other plausible ways to compute the number (allele counts, a ratio of proportions, pooling the two control sources) and searched all integer tables near the stated counts for ones that reproduce the printed OR and both CI limits. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| RE5 carriers, BrS cases | Results ¶4; Abstract | 3.9% (n=9) of 231 | 9/231 = 3.90% |
| RE5 carriers, study controls | Results ¶4 | 0.2% (n=1) of 500 | 1/500 = 0.20% |
| RE5 carriers, Genomics Thailand | Results ¶4 | 6 of 10 043 (0.06%) | 0.060% |
| OR, BrS vs study controls (method check) | Results ¶4; Discussion ¶1 | 20.2 (2.5–160.6), P = 2.0E-04 | Wald 20.2 (2.5–160.6); Fisher P = 2.04E-04 (matches) |
| OR, BrS vs Genomics Thailand | Results ¶4; Abstract; Discussion ¶1 | 45.2 (14.5–141.3) | Wald 67.8 (23.9–192.2); conditional MLE 67.6, exact CI 21.3–233.0 |
| Fisher P, BrS vs Genomics Thailand | Results ¶4 | 5.6E-12 | 5.6E-12 (matches) |
| Allele-based OR (all carriers heterozygous) | check | 45.2 | 66.5 (23.6–187.6) |
| Ratio of proportions (9/231)/(6/10 043) | check | 45.2 | 65.2 |
| Genomics Thailand and study controls pooled (7 of 10 543) | check | 45.2 | 61.0 (22.5–165.3) |
| Table with 6 case carriers, 6/222 vs 6/10 037 | check | 45.2 (14.5–141.3), P = 5.6E-12 | Wald 45.2 (14.5–141.3); Fisher P = 9.3E-08 |
| SE(ln OR) implied by the printed CI | derived | 14.5–141.3 | 0.581 (stated counts give 0.531; 6/222 vs 6/10 037 gives 0.581) |
| Genomics Thailand carriers that would give 45.2 with 9/222 cases | derived | 6 | 9; but 9/222 vs 9/10 034 gives CI 17.8–115.0, not 14.5–141.3 |
| Genomics Thailand size that would give 45.2 with 9 vs 6 carriers | derived | 10 043 | about 6 696 |
| Ultrarare SCN5A coding variants, 13/231 vs 5/500 (counts implied by the percentages) | Results ¶1 | 5.6% vs 1.0%, P = 4.0E-04 | Fisher P = 4.06E-04 (consistent) |
| p.Arg965Cys, 15/231 vs 2/500 | Results ¶1 | 6.5% vs 0.4%, P = 1.6E-06 | Fisher P = 1.59E-06 (consistent) |
| Other low-frequency coding variants, 9/231 vs 15/500 | Results ¶1 | 3.9% vs 3.0%, P = 0.51 | Fisher P = 0.511 (consistent) |
| RE5 carriers with cardiac arrest | Abstract; Results | 89% (8/9) | 88.9% (consistent) |
The search over integer tables (1–20 case carriers, 200–240 case non-carriers, 1–20 population carriers, population totals of 10 043 or 10 543) found two tables whose Wald OR and both limits round to 45.2, 14.5 and 141.3. One is 6/222 vs 6/10 037: the stated table with the case-carrier cell set to 6, the Genomics Thailand carrier count. The other, 6/233 vs 6/10 537, needs 239 cases and does not match the 7 carriers in the pooled controls, so it does not correspond to any group in the paper. Setting the population carrier count to 9 instead (9/222 vs 9/10 034) gives OR 45.2, but its CI is 17.8–115.0, so it does not reproduce the printed interval. Neither do the allele-based, proportion-ratio or pooled calculations.
The printed P value, by contrast, matches only the stated table. The 6-carrier table would give P = 9.3E-08, about 16 500 times larger than the printed 5.6E-12.
Figure
Figure 1. Odds ratios for RE5 carriage, with 95% confidence intervals, on a logarithmic axis; the thin vertical line marks OR = 1. Orange circles are the values printed in the paper; blue squares are Wald odds ratios recomputed from the counts the paper states. For the study-control comparison (9/231 vs 1/500), the two coincide, which confirms the method. For the Genomics Thailand comparison (9/231 vs 6/10 043), the printed 45.2 (14.5–141.3) lies below the recomputed 67.8 (23.9–192.2). The gray diamond is a table that is not stated anywhere in the paper, 6 case carriers instead of 9 (6/222 vs 6/10 037). It reproduces the printed OR and both CI limits exactly.
Figure 2. The three numbers printed for the Genomics Thailand comparison (orange row), set against the values from the stated table (blue row) and from the 6-carrier table (gray row). "Matches" and "differs" compare each value with the printed value in the same column. The stated counts reproduce the printed P value but not the OR or CI; the 6-carrier table reproduces the OR and CI but not the P value. The printed P value and the printed OR therefore come from different tables.
Why it matters
The odds ratio of 45.2 is the only effect size in the Abstract's Results, where it is presented as the variant's "case-control odds ratio". The Discussion repeats it as the upper end of "an OR between 20 and 45". It is the figure a reader would quote for the size of the RE5 enrichment. From the counts the paper states, the odds ratio against the Thai population database is about 68 (95% CI 24–192), and the ratio of carrier proportions is about 65. The printed value understates the enrichment by about one third. Its CI is also shifted down: the printed lower limit of 14.5 is well below the recomputed 23.9.
The error is conservative, and it does not change any conclusion. With the stated counts the enrichment is larger, the printed P value already reflects them, and the comparison stays highly significant. The other headline numbers do not depend on this odds ratio and are unaffected: the 3.9% case frequency, the study-control OR of 20.2, the estimate of about 43 000 carriers in Thailand (based on the 0.06% database frequency), the hiPSC-CM functional results and the pathogenicity argument. A smaller point: the Abstract calls 45.2 a "case-control" odds ratio, but the Results compute it against a population database "with a range of clinical and pharmacogenetics indications", not against the study's ancestry-matched controls. The case-control figure in the narrower sense is 20.2.
Explanations considered
- A different OR method (exact or conditional MLE instead of Wald). This does not account for the numbers. The conditional MLE for the stated table is 67.6 (exact CI 21.3–233.0), essentially the same as the Wald 67.8. The Wald method reproduces the paper's study-control OR, 20.2 (2.5–160.6), to every printed digit.
- Allele counts instead of carrier counts. This does not account for the numbers. All RE5 carriers are heterozygous (the supplementary Table S8 lists Hom = 0 for Genomics Thailand), and an allele-based OR is 66.5 (23.6–187.6). The Methods also say the test compared "samples with or without variants".
- A ratio of proportions rather than an odds ratio. This does not account for the numbers. (9/231)/(6/10 043) = 65.2.
- Pooling Genomics Thailand with the study controls. This does not account for the numbers. With 7 carriers among 10 543 people, the OR is 61.0 (22.5–165.3). Table S8 lists the pooled "Thai all controls" as AC = 7.
- A subset of cases or a different database size. No subset is described. No supplementary table gives a case subset or denominator that would produce 45.2. A database of about 6 696 people would be needed with 9 vs 6 carriers, but the Methods, Results and Table S8 (AN = 20 086 alleles) all give 10 043.
- A typographical error in the OR. A single mistyped digit cannot explain it, because the OR and both CI limits agree with each other: the log-midpoint of 14.5–141.3 is 45.3, and the interval's width implies SE(ln OR) = 0.581, which matches the 6/222 vs 6/10 037 table and not the stated one (0.531). The three numbers fit a single table with 6 case carriers. This is consistent with the case-carrier cell having been entered as 6, the database carrier count. The paper does not say how the value was computed, so this cannot be confirmed from published material.
- A conversion or typesetting error in the archived text. This is excluded. The PMC text, the publisher-supplied JATS XML in Europe PMC and the medRxiv preprint (doi:10.1101/2023.12.19.23299785) all carry 45.2 (14.5–141.3) with P = 5.6E-12.
Status
- Version of record. The publisher's site (ahajournals.org, reached from doi.org) returned HTTP 403 to automated requests, and so did the Europe PMC PDF route. The wording above was confirmed in the PMC article page, the Europe PMC full-text XML (publisher JATS) and the local PMC archive, which carry identical numbers. Every quotation in this write-up was checked verbatim against the PMC-archived text with
src/show.py. The medRxiv preprint carries the same 45.2 (14.5–141.3) and P = 5.6E-12, so the value predates production. - Supplements. The PMC supplementary bundle has three files: cir-151-31-s002.pdf (Supplemental Methods and the captions of Tables S1–S8), cir-151-31-s003.xlsx (Tables S1–S8) and cir-151-31-s004.docx (genotyping primers). Table S8 lists Genomics Thailand RE5 as AC = 6, AN = 20 086, Hom = 0, the Thai study controls as AC = 1/AN = 1 000 and "Thai all controls" as AC = 7/AN = 21 086. Table S5 lists the 9 RE5-carrying probands. No supplementary table resolves the discrepancy.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. A web search for corrections or comments on the article found none. PubPeer could not be checked directly: its search page returned HTTP 403 and its API returned 404 for this DOI.
- Sources. PMC article PMC11670919; Europe PMC full-text XML and supplementary files for PMC11670919; medRxiv preprint 10.1101/2023.12.19.23299785; Crossref; Europe PMC.
Suggested correction
- In the Results, replace "yielding an OR of 45.2 (14.5–141.3) compared with the BrS cohort (P=5.6E-12)" with the value from the stated counts (9/231 cases vs 6/10 043 Genomics Thailand individuals): OR 67.8 (95% CI 23.9–192.2), P = 5.6E-12. If the authors prefer the conditional MLE with an exact interval, that is 67.6 (21.3–233.0).
- Correct the Abstract ("a case-control odds ratio of 45.2") to match. Since 45.2 compares cases with a population database rather than with the study's matched controls, the Abstract could also give the study-control OR of 20.2, or describe the 68 as an OR against the Thai population database.
- In the Discussion, change "an OR between 20 and 45" to "an OR between 20 and 68" (or the equivalent wording).
- If the corrected value appears in any figure or graphical abstract not checked here (for example Figure 1A, which shows ORs for each haplotype, or Figure 4), update it there too.
Recomputation script (python recompute.py.txt)
"""Recompute the RE5 odds ratios of PMC11670919
(Circulation 2024, doi:10.1161/CIRCULATIONAHA.124.069041).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs, copied from the paper as archived in PMC:
Results, p0: 231 unrelated probands (BrS cases) and 500 ancestry-matched controls.
Results, p3: RE5 in 3.9% (n=9) of cases vs 0.2% (n=1) of controls,
P=2.0E-04, OR 20.2 (2.5-160.6).
Genomics Thailand, 10 043 individuals: detected in 6 (0.06%),
OR 45.2 (14.5-141.3) vs the BrS cohort, P=5.6E-12.
Methods: enrichment tested with Fisher exact test on samples with or without variants.
Table S8: Genomics Thailand RE5 AC=6, AN=20086, Hom=0 (6 heterozygous carriers).
Discussion: "OR between 20 and 45"; "~43 000 carriers of the RE5 variant alone in Thailand".
Method: crude odds ratio a*d/(b*c) on carriers vs non-carriers with a 95% Wald (Woolf) CI
exp(ln OR +/- 1.96*sqrt(1/a+1/b+1/c+1/d)); conditional MLE and exact CI from scipy;
two-sided Fisher exact P from scipy.
"""
import math
import os
import numpy as np
from scipy.stats import fisher_exact
from scipy.stats.contingency import odds_ratio
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = 1.959964
N_CASE, N_CTRL, N_GT = 231, 500, 10043
A_CASE, A_CTRL, A_GT = 9, 1, 6
def wald(a, b, c, d):
o = a * d / (b * c)
se = math.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
return o, o * math.exp(-Z * se), o * math.exp(Z * se), se
def show(label, a, b, c, d, reported=""):
o, lo, hi, se = wald(a, b, c, d)
t = [[a, b], [c, d]]
r = odds_ratio(t)
ci = r.confidence_interval()
p = fisher_exact(t)[1]
print(f"{label}\n table {t} reported: {reported}\n"
f" Wald OR {o:.2f} ({lo:.2f}-{hi:.2f}), SE(lnOR) {se:.4f} | conditional MLE {r.statistic:.2f}, "
f"exact CI {ci.low:.2f}-{ci.high:.2f} | Fisher P (two-sided) {p:.3g}")
return dict(o=o, lo=lo, hi=hi, p=p, se=se)
print("=== 1. Proportions ===")
print(f"cases {A_CASE}/{N_CASE} = {100*A_CASE/N_CASE:.2f}% (reported 3.9%)")
print(f"study controls {A_CTRL}/{N_CTRL} = {100*A_CTRL/N_CTRL:.2f}% (reported 0.2%)")
print(f"Genomics Thailand {A_GT}/{N_GT} = {100*A_GT/N_GT:.3f}% (reported 0.06%)")
print("\n=== 2. Odds ratios on carriers vs non-carriers ===")
R = {}
R["ctrl"] = show("BrS cases vs study controls", A_CASE, N_CASE - A_CASE, A_CTRL, N_CTRL - A_CTRL,
"OR 20.2 (2.5-160.6), P=2.0E-04")
R["gt"] = show("BrS cases vs Genomics Thailand (stated counts)", A_CASE, N_CASE - A_CASE, A_GT, N_GT - A_GT,
"OR 45.2 (14.5-141.3), P=5.6E-12")
print("\n=== 3. Alternative calculations for the Genomics Thailand comparison ===")
show("allele-based (2N alleles; all carriers heterozygous, Table S8 Hom=0)",
A_CASE, 2 * N_CASE - A_CASE, A_GT, 2 * N_GT - A_GT, "45.2")
R["pooled"] = show("Genomics Thailand + study controls pooled (7 of 10 543)",
A_CASE, N_CASE - A_CASE, A_GT + A_CTRL, N_GT + N_CTRL - A_GT - A_CTRL, "45.2")
print(f"ratio of proportions (9/231)/(6/10043) = {(A_CASE/N_CASE)/(A_GT/N_GT):.2f} (reported 45.2)")
R["six"] = show("case-carrier cell entered as 6 (not a stated count)", 6, N_CASE - A_CASE, A_GT, N_GT - A_GT,
"OR 45.2 (14.5-141.3), P=5.6E-12")
p_six_one = fisher_exact([[6, 222], [6, 10037]], alternative="greater")[1]
p_gt_one = fisher_exact([[9, 222], [6, 10037]], alternative="greater")[1]
print(f"one-sided Fisher P: stated table {p_gt_one:.3g}; 6-carrier table {p_six_one:.3g}")
print("\n=== 4. Reverse-engineering the printed OR and CI ===")
lo_p, hi_p = 14.5, 141.3
print(f"log-midpoint of printed CI: sqrt(14.5*141.3) = {math.sqrt(lo_p*hi_p):.2f}")
se_imp = (math.log(hi_p) - math.log(lo_p)) / (2 * Z)
print(f"SE(lnOR) implied by printed CI: {se_imp:.4f} (stated table gives {R['gt']['se']:.4f})")
print(f"Genomics Thailand N that would give 45.2 with 9 vs 6 carriers: "
f"{45.2*222*6/9 + 6:.0f} (stated 10 043)")
print(f"Genomics Thailand carriers that would give 45.2 with 9/222 cases: "
f"{9*10037/(222*45.2):.2f} (stated 6)")
show("population carrier cell set to 9 (not a stated count)", 9, 222, 9, 10034, "OR 45.2 (14.5-141.3)")
print(f"P of 6-carrier table / printed P 5.6E-12 = {R['six']['p']/5.6e-12:.0f}")
print("Integer tables searched: case carriers a=1..20, case non-carriers b=200..240,")
print(" population carriers c=1..20, population total 10 043 or 10 543 (pooled); keep tables whose")
print(" Wald OR and both CI limits round to 45.2, 14.5 and 141.3:")
hits = []
for a in range(1, 21):
for b in range(200, 241):
for c in range(1, 21):
for tot in (N_GT, N_GT + N_CTRL):
d = tot - c
o, lo, hi, _ = wald(a, b, c, d)
if round(o, 1) == 45.2 and round(lo, 1) == 14.5 and round(hi, 1) == 141.3:
hits.append((a, b, c, d))
for h in hits:
print(" ", h)
print(f" {len(hits)} table(s) found")
print("\n=== 5. Other numbers checked (consistent) ===")
for lab, a, n1, c, n2, rep in [
("ultrarare SCN5A coding, 13/231 vs 5/500", 13, 231, 5, 500, "5.6% vs 1.0%, P=4.0E-04"),
("p.Arg965Cys, 15/231 vs 2/500", 15, 231, 2, 500, "6.5% vs 0.4%, P=1.6E-06"),
("other low-frequency coding, 9/231 vs 15/500", 9, 231, 15, 500, "3.9% vs 3.0%, P=0.51"),
]:
p = fisher_exact([[a, n1 - a], [c, n2 - c]])[1]
print(f"{lab}: {100*a/n1:.2f}% vs {100*c/n2:.2f}%, Fisher P {p:.3g} (reported {rep})")
print(f"RE5 carriers with cardiac arrest 8/9 = {800/9:.1f}% (reported 89%)")
print(f"Thai population implied by ~43 000 carriers at 6/10 043: {43000/(6/10043)/1e6:.1f} million")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GRAY = "#eb6834", "#2a78d6", "#8a8984"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "text.color": INK})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: dot-and-interval, log OR axis
fig, ax = plt.subplots(figsize=(7.2, 4.1), facecolor=BG)
ax.set_facecolor(BG)
rows = [
# y, label, (o, lo, hi), colour, marker, text
(3.15, "Study controls, reported", (20.2, 2.5, 160.6), ORANGE, "o", "20.2 (2.5–160.6)"),
(2.85, "Study controls, recomputed", (R["ctrl"]["o"], R["ctrl"]["lo"], R["ctrl"]["hi"]), BLUE, "s",
f"{R['ctrl']['o']:.1f} ({R['ctrl']['lo']:.1f}–{R['ctrl']['hi']:.1f})"),
(1.55, "Genomics Thailand, reported", (45.2, 14.5, 141.3), ORANGE, "o", "45.2 (14.5–141.3)"),
(1.25, "Genomics Thailand, recomputed", (R["gt"]["o"], R["gt"]["lo"], R["gt"]["hi"]), BLUE, "s",
f"{R['gt']['o']:.1f} ({R['gt']['lo']:.1f}–{R['gt']['hi']:.1f})"),
(0.35, "6 case carriers instead of 9", (R["six"]["o"], R["six"]["lo"], R["six"]["hi"]), GRAY, "D",
f"{R['six']['o']:.1f} ({R['six']['lo']:.1f}–{R['six']['hi']:.1f})"),
]
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(1.06, 3.62, "OR = 1", color=INK2, fontsize=10, va="top")
for y, lab, (o, lo, hi), col, mk, txt in rows:
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="round", zorder=2)
ax.plot(o, y, marker=mk, ms=9, color=col, mec=BG, mew=1.5, zorder=3)
ax.text(hi * 1.12, y, txt, va="center", fontsize=10, color=INK)
ax.set_xscale("log")
ax.set_xlim(1, 2000)
ax.set_xticks([1, 3, 10, 30, 100, 300, 1000])
ax.set_xticklabels(["1", "3", "10", "30", "100", "300", "1000"])
ax.set_ylim(-0.1, 3.7)
ax.set_yticks([3.0, 1.4, 0.35])
ax.set_yticklabels(["BrS vs study controls\n9/231 vs 1/500",
"BrS vs Genomics Thailand\n9/231 vs 6/10 043",
"Table matching the\nprinted 45.2 (not stated)\n6/222 vs 6/10 037"], fontsize=10)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Odds ratio for RE5 carriage (log scale), point estimate and 95% CI", color=INK)
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper"),
Line2D([], [], color=BLUE, marker="s", ms=8, lw=2, label="Recomputed from the stated counts"),
Line2D([], [], color=GRAY, marker="D", ms=8, lw=2, label="Recomputed from a 6-carrier table")]
ax.legend(handles=handles, loc="center right", frameon=False, fontsize=10, bbox_to_anchor=(1.0, 0.575))
fig.tight_layout()
save(fig, "fig1")
# Figure 2: which table reproduces which printed number
fig, ax = plt.subplots(figsize=(7.2, 2.9), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 10)
ax.set_ylim(0, 4)
ax.axis("off")
cols = [(3.8, "Odds ratio"), (6.0, "95% CI"), (8.5, "Fisher P (two-sided)")]
for x, h in cols:
ax.text(x, 3.6, h, ha="center", fontsize=10.5, color=INK2, weight="bold")
rowsd = [
(2.85, "Printed in the paper", ORANGE, ["45.2", "14.5–141.3", "5.6E-12"], [None] * 3),
(1.75, "Stated counts\n9/222 vs 6/10 037", BLUE,
[f"{R['gt']['o']:.1f}", f"{R['gt']['lo']:.1f}–{R['gt']['hi']:.1f}", f"{R['gt']['p']:.1e}".replace("e-", "E-")],
["differs", "differs", "matches"]),
(0.65, "6 case carriers\n6/222 vs 6/10 037", GRAY,
[f"{R['six']['o']:.1f}", f"{R['six']['lo']:.1f}–{R['six']['hi']:.1f}", f"{R['six']['p']:.1e}".replace("e-", "E-")],
["matches", "matches", "differs"]),
]
for y, lab, col, vals, verdict in rowsd:
ax.plot(0.15, y, marker="o" if col == ORANGE else ("s" if col == BLUE else "D"), ms=9, color=col)
ax.text(0.4, y, lab, va="center", fontsize=10.5, color=INK)
for (x, _), v, vd in zip(cols, vals, verdict):
ax.text(x, y + (0.13 if vd else 0), v, ha="center", va="center", fontsize=11.5, color=INK)
if vd:
ax.text(x, y - 0.25, ("✓ " if vd == "matches" else "✗ ") + vd, ha="center",
va="center", fontsize=10, color=INK if vd == "matches" else INK2,
weight="bold" if vd == "matches" else "normal")
for yl in (3.35, 2.3, 1.2):
ax.plot([0, 10], [yl, yl], color=GRID, lw=1)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract reports type 2 diabetes odds ratios as evidence of higher case-fatality in type 1 diabetes
Circulation (2018) · doi:10.1161/CIRCULATIONAHA.118.034986 · PMC6287897
Summary
This national Scottish cohort study of 3.25 million people reports, in its Abstract, that 30-day case-fatality after an incident heart failure hospitalization "was higher in type 1 diabetes mellitus for men (odds ratio, 0.96; 95% CI, 0.95–0.96) and women (odds ratio, 0.98; 95% CI, 0.97–0.98)". Both odds ratios are below 1 and both intervals lie entirely below 1, so as printed they mean 4% and 2% lower odds of death, not higher. The same figures, digit for digit, are the Results' estimates for type 2 diabetes versus no diabetes. The Results give the type 1 estimates as OR 1.91 (95% CI 1.68–2.18) in men and OR 1.31 (1.05–1.65) in women, and the model coefficients in the online supplement reproduce both sets. The paper presents raised case-fatality in type 1 diabetes as a first-reported finding, so the Abstract's numbers misstate one of its main results: they point the wrong way and understate the type 1 odds ratio about 2.0-fold in men and 1.34-fold in women. The direction of the finding stated in the body of the paper, and the analysis itself, are not affected.
What the paper reports
Abstract, Results:
Thirty-day case-fatality was similar among people with type 2 diabetes mellitus and without diabetes mellitus, but was higher in type 1 diabetes mellitus for men (odds ratio, 0.96; 95% CI, 0.95–0.96) and women (odds ratio, 0.98; 95% CI, 0.97–0.98).
Abstract, Conclusions:
Despite falling incidence, particularly in type 1 diabetes mellitus, heart failure remains ≈2-fold higher than in people without diabetes mellitus, with higher case-fatality in those with type 1 diabetes mellitus.
Results, "Case-Fatality of Incident Heart Failure Hospitalization", paragraph 2 (the paragraph beginning "Thirty-day case-fatality results"; last sentence). Paragraph 1 of this section is a single sentence giving the overall 30-day case-fatality (14.2%).
However, it was similar in people with type 2 diabetes mellitus and in people without diabetes mellitus, for men the OR was 0.96 (95% CI, 0.95–0.96), and for women the OR was 0.98 (95% CI, 0.97–0.98).
Same section, paragraph 3 (beginning "However, case-fatality was higher"):
However, case-fatality was higher among people with type 1 diabetes mellitus compared with people without diabetes mellitus; the difference was larger for men (OR, 1.91; 95% CI, 1.68–2.18) than for women (OR, 1.31; 95% CI, 1.05–1.65).
Same section, paragraph 2 (first sentence), on where the model is given:
Thirty-day case-fatality results obtained from logistic regression models adjusting for age, sex, and deprivation are shown in Figure 3 (see the Appendix in the online-only Data Supplement for model coefficients, including interactions).
Discussion, paragraph 6 (beginning "Unlike incidence, for case-fatality"):
However, among people with type 1 diabetes mellitus, case-fatality was 1.3-fold higher in women and 1.9-fold higher in men.
Discussion, paragraph 7 (beginning "Women therefore have"):
In combination, this means that compared with women without diabetes mellitus, women with type 1 diabetes mellitus have around a 2.5-fold higher risk of having an incident heart failure admission which results in death within 30 days. For the equivalent comparison in men, there is almost a 4-fold difference in risk.
Discussion, paragraph 11 (beginning "Fewer studies have examined"):
Indeed, ours is the first population-based study of which we are aware to compare case-fatality after heart failure admissions in people with and without type 1 diabetes mellitus, and we are not aware that the high case-fatality in patients with type 1 diabetes mellitus has been reported previously.
Online-only Data Supplement (cir-138-2774-s001.pdf), Table S11, "Heart failure case-fatality model, coefficients and standard errors" (logistic regression; selected rows, estimate and standard error):
sex 0.041 (0.02); typet1dm 0.649 (0.104); typet2dm -0.046 (0.031); sex:typet1dm -0.375 (0.158); sex:typet2dm 0.021 (0.044)
What the paper's own numbers show
Method. I compared the Abstract's odds ratios with the two sets of odds ratios in the Results, string for string. For each odds ratio I computed the change in odds, (OR − 1) × 100, and noted which side of 1 its confidence interval lies on. I checked the internal consistency of the type 1 estimates (log-scale midpoint of the CI, implied standard error and z) and their agreement with the rounded figures in the Discussion. I then recomputed all four diabetes-versus-no-diabetes odds ratios from the supplement's Table S11 by exponentiating the coefficients: the main-effect diabetes terms give the estimates for men, and adding the sex-by-diabetes interaction gives the estimates for women (the sex term is positive and the paper says case-fatality "was higher in women than in men", so sex = 1 codes women). I also tested two other readings of the Abstract's numbers: the reciprocal of the printed ORs, and type 1 versus type 2. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| OR, case-fatality, "type 1" vs none, men | Abstract, Results | 0.96 (95% CI 0.95–0.96), called "higher" | 4% lower odds (CI 5% to 4% lower); CI entirely below 1 |
| OR, "type 1" vs none, women | Abstract, Results | 0.98 (0.97–0.98), called "higher" | 2% lower odds (CI 3% to 2% lower); CI entirely below 1 |
| OR, type 2 vs none, men / women | Results, Case-Fatality ¶2 | 0.96 (0.95–0.96) / 0.98 (0.97–0.98) | identical to the Abstract figures (string comparison: True) |
| OR, type 1 vs none, men | Results, Case-Fatality ¶3 | 1.91 (1.68–2.18) | 91% higher odds (CI 68% to 118% higher); CI log-midpoint 1.914; z = 9.74 |
| OR, type 1 vs none, women | Results, Case-Fatality ¶3 | 1.31 (1.05–1.65) | 31% higher odds (CI 5% to 65% higher); CI log-midpoint 1.316; z = 2.34 |
| Rounded type 1 ORs | Discussion ¶6 | "1.3-fold" (women), "1.9-fold" (men) | 1.3 and 1.9: match the Results, Case-Fatality ¶3 values, not the Abstract |
| Type 1 vs none from model, men | Table S11 | typet1dm 0.649 | exp(0.649) = 1.914 |
| Type 1 vs none from model, women | Table S11 | 0.649 + (−0.375) | exp(0.274) = 1.315 (1.314–1.317 over coefficient rounding; Results 1.31) |
| Type 2 vs none from model, men | Table S11 | typet2dm −0.046 | exp(−0.046) = 0.955, rounds to 0.96 |
| Type 2 vs none from model, women | Table S11 | −0.046 + 0.021 | exp(−0.025) = 0.975, rounds to 0.98 |
| Size of the misstatement | derived | Abstract 0.96 / 0.98 | correct type 1 OR is 1.91/0.96 = 1.99× (men) and 1.31/0.98 = 1.34× (women) the printed value |
| Alternative reading: reciprocal | derived | 0.96 / 0.98 | 1/0.96 = 1.042, 1/0.98 = 1.020: not 1.91 or 1.31 |
| Alternative reading: type 1 vs type 2 | derived | 0.96 / 0.98 | 1.91/0.96 = 1.99, 1.31/0.98 = 1.34: not 0.96 or 0.98 |
| Women, incidence × case-fatality | Discussion ¶7 | "around a 2.5-fold higher risk" | 2 × 1.31 = 2.62 (consistent); with the Abstract value, 2 × 0.98 = 1.96 |
The Abstract's two odds ratios and four confidence limits are the type 2 estimates from Case-Fatality paragraph 2 of the Results. The type 1 estimates in Case-Fatality paragraph 3 are internally consistent, agree with the Discussion, and are reproduced from the supplement's model coefficients. Neither alternative reading turns the Abstract's numbers into a type 1 estimate.
Minor points found in the same checks (separate from the main issue).
- The OR per calendar year for case-fatality is 0.967 (95% CI 0.961–0.973) in the Abstract and 0.967 (0.961–0.974) in Results, Case-Fatality paragraph 4 (beginning "Trends in case-fatality"). Table S12 (year −0.033, SE 0.003) gives 0.9675 (0.9619–0.9732), so the Abstract limit is the one that matches. The 3.3% annual decline matches 1 − 0.967.
- The same paragraph labels both interaction terms "calendar year/type 2 interaction". The first, 1.011 (0.959–1.065), sits in the type 1 clause and matches Table S12's year:typet1dm term (exp(0.01) = 1.010, CI 0.958–1.065), so its label should read type 1.
- Wald intervals computed from the Table S11 standard errors are 1.56–2.35 for type 1 vs none in men (paper 1.68–2.18) and 0.90–1.01 for type 2 vs none in men (paper 0.95–0.96), and Table S11 gives z = −1.48, P = 0.138 for the type 2 term. The type 2 interval printed in the paper lies entirely below 1, although the text calls the groups "similar"; the supplement's P value fits the word "similar". The paper's intervals may have been derived another way, so this is noted, not counted as an error.
- The sensitivity-analysis crude rates reproduce: 34 893 / 38 080 361.9 py = 0.92, 11 551 / 1 885 715.8 py = 6.13 and 635 / 237 683.5 py = 2.67 per 1000 py (printed 0.9, 6.1, 2.7).
Figure
Figure 1. Left: the Abstract's sentence on case-fatality, with the two odds ratios it gives for type 1 diabetes. Right: the two paragraphs of the Results (Case-Fatality section) that give the diabetes-versus-no-diabetes odds ratios, paragraph 2 for type 2 diabetes and paragraph 3 for type 1 diabetes. The values printed in the Abstract (orange) are identical, to every digit, to the type 2 values in paragraph 2, which the paper calls "similar" to no diabetes. The type 1 values printed in paragraph 3 (black: 1.91 and 1.31) do not appear in the Abstract. Bottom (blue): the same four odds ratios recomputed by exponentiating the case-fatality model coefficients in Supplement Table S11, which assign 0.955 and 0.975 to type 2 and 1.914 and 1.315 to type 1.
Figure 2. Odds ratios for 30-day case-fatality versus no diabetes on a log scale, with OR = 1 as the reference line. Orange dots and lines are the odds ratios and 95% confidence intervals printed in the paper: the Abstract's values for "higher in type 1" and the Results values for type 2 (Case-Fatality paragraph 2) and type 1 (paragraph 3). Blue diamonds are the odds ratio for the comparison each row names, recomputed from Table S11 coefficients (point estimates only; Table S11 gives no covariance for the women's sums). For the Results rows the printed and recomputed values coincide. For the Abstract rows the printed value lies left of 1 while the type 1 value the sentence describes lies at 1.914 in men and 1.315 in women; the dashed arrows mark the gap.
Why it matters
Raised case-fatality in type 1 diabetes is the paper's one new case-fatality result: the Discussion describes the study as the first population-based comparison of case-fatality after heart failure admission in people with and without type 1 diabetes, and the Abstract's Conclusions rest on it ("with higher case-fatality in those with type 1 diabetes mellitus"). Readers who see only the Abstract, for example in PubMed, have no other source for the effect size. A reader who takes the Abstract's numbers at face value would conclude that the odds of death within 30 days are 4% (men) and 2% (women) lower in type 1 diabetes, with intervals that exclude 1, which contradicts the word "higher" in the same sentence. A reader who notices the contradiction cannot recover the correct effect size from the Abstract. The correct estimates (91% and 31% higher odds) are about 2.0 and 1.34 times the printed odds ratios.
The error does not reach the analysis or the direction of the paper's conclusions. The Abstract's wording, the Results, the Discussion (1.9-fold and 1.3-fold), and the combined-risk statements (about 2.5-fold in women, using 1.31; almost 4-fold in men) all state or use the correct type 1 values, and the full-text Conclusion states the correct direction ("case-fatality also being higher in people with type 1 diabetes mellitus") without giving values. Figure 3 and Table 2, which show predicted and stratified case-fatality, do not carry the wrong odds ratios. The incidence results are not involved.
Explanations considered
- The Abstract's numbers are type 1 estimates of a different contrast. This does not account for the numbers. The reciprocals of the printed ORs are 1.042 and 1.020, and type 1 versus type 2 gives 1.99 and 1.34. Neither reading produces 0.96 and 0.98, and the printed intervals, 0.95–0.96 and 0.97–0.98, are identical to the ones the Results print for type 2. (They are narrower than even the type 2 term in Table S11 supports; see the note on Wald intervals above.)
- Rounding or a typographical slip in one digit. This does not account for the numbers. All six figures (two odds ratios, four confidence limits) match the type 2 estimates exactly, and they differ from the type 1 estimates in every digit and in direction.
- The Abstract describes a different model from the Results. This does not account for the numbers. The Abstract says all estimates are adjusted for age, sex, socioeconomic status and calendar year, but the type 2 figures it carries are the ones the Results give from the cross-sectional case-fatality model, and Table S11 reproduces them. The calendar-year model (Table S12) has no sex-by-diabetes interaction, so it cannot yield a pair of sex-specific estimates; its diabetes terms give exp(0.47) = 1.60 for type 1 and exp(−0.017) = 0.983 for type 2, so in that model too the type 1 odds ratio is above 1.
- A conversion or typesetting error in the archived text. This is excluded. The publisher-supplied JATS XML in Europe PMC, the PubMed abstract (PMID 29950404) and the university repository copy labelled "Published Version" all carry the same Abstract sentence with 0.96 (0.95–0.96) and 0.98 (0.97–0.98). The numbers involve no minus signs, superscripts or table cells that conversion could alter.
- The supplement explains the Abstract's figures. This is excluded. The supplement's case-fatality model (Table S11) assigns these values to type 2 diabetes and gives type 1 odds ratios of 1.914 (men) and 1.315 (women; 1.31 within coefficient rounding), as in the Results.
- Most likely account. The Abstract appears to have taken the odds ratios from the type 2 sentence of the Results and attached them to the type 1 clause. The source of the transposition cannot be determined from the published material.
Status
- Version of record. The publisher pages (ahajournals.org, reached from doi.org) returned HTTP 403 to automated requests for both HTML and PDF, and the PMC page returned a reCAPTCHA. The publisher text was therefore confirmed through three independent copies: the publisher-supplied JATS XML in Europe PMC (PMC6287897/fullTextXML), the PubMed abstract (PMID 29950404, via E-utilities efetch) and the University of Glasgow repository record (eprints.gla.ac.uk/163955), whose file is labelled "Published Version". All three carry the Abstract sentence quoted above with the same values. Every quotation from the article above was confirmed verbatim against the PMC-archived text with
src/show.py; paragraph numbers count every paragraph of the section from 1, including the one-sentence opening paragraph of the Case-Fatality section. The Table S11 and S12 values (a condensed list of selected rows, not a verbatim excerpt) were checked against the supplement PDF, cir-138-2774-s001.pdf, pages 13–14. - Supplements. The online-only Data Supplement (cir-138-2774-s001.pdf, 22 pages), obtained from the Europe PMC supplementary-files archive for PMC6287897, was read. Table S11 (case-fatality model) and Table S12 (case-fatality over time) are quoted and used above. They confirm that 0.96 and 0.98 are type 2 estimates and 1.91 and 1.31 are type 1 estimates.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI; PMC and Europe PMC list no correction notices or comment links (Europe PMC
commentCorrectionListis empty). A web search for corrections, errata or comments on the article found none. The PubPeer API refused automated access (human-verification challenge), so PubPeer could not be checked directly. - Sources. PMC article PMC6287897; Europe PMC full-text XML and supplementary files for PMC6287897; PubMed record 29950404; University of Glasgow repository record 163955; Crossref metadata for doi:10.1161/CIRCULATIONAHA.118.034986.
Suggested correction
- In the Abstract, Results, replace the odds ratios in the case-fatality sentence with the type 1 versus no-diabetes estimates from the Results: "…but was higher in type 1 diabetes mellitus for men (odds ratio, 1.91; 95% CI, 1.68–2.18) and women (odds ratio, 1.31; 95% CI, 1.05–1.65)." If the type 2 estimates are wanted in the Abstract, attach them to the type 2 clause.
- In Results, Case-Fatality paragraph 4 (beginning "Trends in case-fatality"), relabel the first interaction term (1.011; 95% CI, 0.959–1.065) as the calendar year/type 1 interaction.
- Reconcile the upper confidence limit of the OR per calendar year (0.973 in the Abstract, 0.974 in the Results); Table S12 supports 0.973.
- Optional: state how the confidence intervals for the case-fatality odds ratios were obtained, since Wald intervals from the Table S11 standard errors are wider (for example 1.56–2.35 for type 1 in men, and 0.90–1.01 for type 2 in men).
Recomputation script (python recompute.py.txt)
"""Recompute the 30-day case-fatality odds ratios of PMC6287897
(Circulation 2018, doi:10.1161/CIRCULATIONAHA.118.034986,
"Incidence of Hospitalization for Heart Failure and Case-Fatality Among 3.25 Million People
With and Without Diabetes Mellitus").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC and from its online-only Data Supplement
(cir-138-2774-s001.pdf):
Abstract, Results: "higher in type 1 diabetes mellitus for men (odds ratio, 0.96; 95% CI, 0.95-0.96)
and women (odds ratio, 0.98; 95% CI, 0.97-0.98)"
Abstract, Results: OR per calendar year 0.967 (95% CI 0.961-0.973)
(Paragraphs are numbered from 1 within each section; Case-Fatality paragraph 1 is the one-sentence
overall figure "14.2% (16406/115701) ... died within 30 days".)
Results, Case-Fatality para 2: type 2 vs none, men OR 0.96 (0.95-0.96), women OR 0.98 (0.97-0.98)
Results, Case-Fatality para 3: type 1 vs none, men OR 1.91 (1.68-2.18), women OR 1.31 (1.05-1.65)
Results, Case-Fatality para 4: OR per calendar year 0.967 (0.961-0.974);
"calendar year/type 2 interaction" 1.011 (0.959-1.065) [type 1 sentence]
"calendar year/type 2 interaction" 0.994 (0.979-1.009) [type 2 sentence]
Discussion para 6: type 1 case-fatality "1.3-fold higher in women and 1.9-fold higher in men";
Discussion para 7: women "2-fold higher heart failure incidence" and "around a 2.5-fold higher risk"
Results, Sensitivity Analyses: 34 893 / 38 080 361.9 py; 11 551 / 1 885 715.8 py; 635 / 237 683.5 py
Supplement Table S11 (case-fatality logistic model, estimate (SE)):
sex 0.041 (0.02); typet1dm 0.649 (0.104); typet2dm -0.046 (0.031);
sex:typet1dm -0.375 (0.158); sex:typet2dm 0.021 (0.044)
Supplement Table S12 (case-fatality over time): year -0.033 (0.003); typet1dm 0.47; typet2dm -0.017;
year:typet1dm 0.01 (0.027); year:typet2dm -0.006 (0.008)
The sex coefficient is positive and the paper says case-fatality "was higher in women than in men",
so sex = 1 codes women and the main-effect diabetes terms are the estimates for men.
"""
import math
import os
from scipy import stats
import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.isf(0.025)
# ------------------------------------------------------------------ inputs (from the paper)
ABSTRACT = {"men": (0.96, 0.95, 0.96), "women": (0.98, 0.97, 0.98)} # labelled type 1 vs none
RES_T2 = {"men": (0.96, 0.95, 0.96), "women": (0.98, 0.97, 0.98)} # Results, Case-Fatality para 2, type 2 vs none
RES_T1 = {"men": (1.91, 1.68, 2.18), "women": (1.31, 1.05, 1.65)} # Results, Case-Fatality para 3, type 1 vs none
S11 = {"sex": (0.041, 0.02), "typet1dm": (0.649, 0.104), "typet2dm": (-0.046, 0.031),
"sex:typet1dm": (-0.375, 0.158), "sex:typet2dm": (0.021, 0.044)}
S12 = {"year": (-0.033, 0.003), "year:typet1dm": (0.01, 0.027), "year:typet2dm": (-0.006, 0.008)}
def se_from_ci(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z)
def wald_ci(b, se):
return math.exp(b - Z * se), math.exp(b + Z * se)
print("=== 1. Abstract vs Results: are the Abstract's 'type 1' ORs the Results' type 2 ORs? ===")
print(f" Abstract ORs (labelled type 1 vs none): {ABSTRACT}")
print(f" Results para 2 ORs (type 2 vs none): {RES_T2}")
print(f" Results para 3 ORs (type 1 vs none): {RES_T1}")
print(f" Abstract == Results type 2: {ABSTRACT == RES_T2}; Abstract == Results type 1: {ABSTRACT == RES_T1}")
print("\n=== 2. Direction of each OR (percent change in odds, CI side) ===")
pct = {}
for name, d in (("Abstract 'type 1'", ABSTRACT), ("Results type 1", RES_T1), ("Results type 2", RES_T2)):
for sex, (o, lo, hi) in d.items():
side = "entirely below 1" if hi < 1 else ("entirely above 1" if lo > 1 else "includes 1")
pct[(name, sex)] = (100 * (o - 1), 100 * (lo - 1), 100 * (hi - 1))
print(f" {name:18s} {sex:5s}: OR {o} ({lo}-{hi}) -> odds {100 * (o - 1):+.0f}% "
f"(CI {100 * (lo - 1):+.0f}% to {100 * (hi - 1):+.0f}%); CI {side}")
print("\n=== 3. Internal consistency of the type 1 ORs in Results, Case-Fatality para 3 ===")
t1z = {}
for sex, (o, lo, hi) in RES_T1.items():
se = se_from_ci(lo, hi)
mid = math.exp((math.log(lo) + math.log(hi)) / 2)
z = math.log(o) / se
t1z[sex] = z
print(f" type 1 {sex:5s}: OR {o}, CI log-midpoint {mid:.3f}, SE(lnOR) {se:.3f}, z = {z:.2f}, "
f"P = {2 * stats.norm.sf(abs(z)):.2g}")
print(f" Discussion '1.9-fold' (men) and '1.3-fold' (women): rounded ORs {round(RES_T1['men'][0], 1)}, "
f"{round(RES_T1['women'][0], 1)}")
print(f" Women, incidence x case-fatality: 2 x {RES_T1['women'][0]} = {2 * RES_T1['women'][0]:.2f} "
f"(Discussion: 'around a 2.5-fold'); with the Abstract value 2 x {ABSTRACT['women'][0]} = "
f"{2 * ABSTRACT['women'][0]:.2f}")
print("\n=== 4. Size of the misstatement ===")
ratio = {}
for sex in ("men", "women"):
ratio[sex] = RES_T1[sex][0] / ABSTRACT[sex][0]
print(f" {sex:5s}: correct type 1 OR / Abstract OR = {RES_T1[sex][0]} / {ABSTRACT[sex][0]} = {ratio[sex]:.2f}")
print(" Alternative readings checked:")
for sex in ("men", "women"):
print(f" {sex:5s}: reciprocal of Abstract OR 1/{ABSTRACT[sex][0]} = {1 / ABSTRACT[sex][0]:.3f}; "
f"type 1 vs type 2 = {RES_T1[sex][0]}/{RES_T2[sex][0]} = {RES_T1[sex][0] / RES_T2[sex][0]:.2f}")
print("\n=== 5. ORs recomputed from Supplement Table S11 coefficients ===")
b1, se1 = S11["typet1dm"]
b2, se2 = S11["typet2dm"]
bi1, bi2 = S11["sex:typet1dm"][0], S11["sex:typet2dm"][0]
S11OR = {
("t1", "men"): math.exp(b1), ("t1", "women"): math.exp(b1 + bi1),
("t2", "men"): math.exp(b2), ("t2", "women"): math.exp(b2 + bi2),
}
print(f" type 1 vs none, men = exp({b1}) = {S11OR[('t1', 'men')]:.3f} (Results {RES_T1['men'][0]}; Abstract {ABSTRACT['men'][0]})")
print(f" type 1 vs none, women = exp({b1} {bi1:+}) = exp({b1 + bi1:.3f}) = {S11OR[('t1', 'women')]:.3f}"
f" (Results {RES_T1['women'][0]}; Abstract {ABSTRACT['women'][0]})")
print(f" type 2 vs none, men = exp({b2}) = {S11OR[('t2', 'men')]:.3f} (Results {RES_T2['men'][0]})")
print(f" type 2 vs none, women = exp({b2} {bi2:+}) = exp({b2 + bi2:.3f}) = {S11OR[('t2', 'women')]:.3f}"
f" (Results {RES_T2['women'][0]})")
# rounding range of the women type 1 coefficient sum (each coefficient printed to 3 dp)
lo_sum, hi_sum = (b1 - 0.0005) + (bi1 - 0.0005), (b1 + 0.0005) + (bi1 + 0.0005)
print(f" rounding range for women type 1 (coefficients +/-0.0005): exp({lo_sum:.4f}..{hi_sum:.4f}) = "
f"{math.exp(lo_sum):.4f}-{math.exp(hi_sum):.4f}, so 1.31 is within rounding")
print("\n=== 6. Side observation: Wald CIs from Table S11 standard errors (men, main effects) ===")
w1, w2 = wald_ci(b1, se1), wald_ci(b2, se2)
z2 = b2 / se2
print(f" type 1 vs none, men: exp({b1} +/- 1.96*{se1}) = {w1[0]:.2f}-{w1[1]:.2f} (paper: 1.68-2.18; "
f"paper CI implies SE {se_from_ci(1.68, 2.18):.3f})")
print(f" type 2 vs none, men: exp({b2} +/- 1.96*{se2}) = {w2[0]:.2f}-{w2[1]:.2f} (paper: 0.95-0.96; "
f"S11 z = {z2:.2f}, P = {2 * stats.norm.sf(abs(z2)):.3f})")
print(" (Not counted as an error here: the paper's CIs may have been derived another way.)")
print("\n=== 7. Calendar-year terms (Abstract vs Results, Case-Fatality para 4 vs Supplement Table S12) ===")
by, sey = S12["year"]
wy = wald_ci(by, sey)
print(f" OR per calendar year: exp({by}) = {math.exp(by):.4f}, Wald CI {wy[0]:.4f}-{wy[1]:.4f}; "
f"Abstract 0.967 (0.961-0.973), Results 0.967 (0.961-0.974); upper limits differ by "
f"{0.974 - 0.973:.3f}")
print(f" decline per year from OR 0.967: {100 * (1 - 0.967):.1f}%")
for term, rep in (("year:typet1dm", "1.011 (0.959-1.065), labelled 'calendar year/type 2 interaction' in the type 1 clause"),
("year:typet2dm", "0.994 (0.979-1.009), labelled 'calendar year/type 2 interaction'")):
b, se = S12[term]
w = wald_ci(b, se)
print(f" {term:14s}: exp({b}) = {math.exp(b):.3f}, Wald CI {w[0]:.3f}-{w[1]:.3f}; Results: {rep}")
print(f" Table S12 diabetes terms (no sex interaction): type 1 exp(0.47) = {math.exp(0.47):.2f}, "
f"type 2 exp(-0.017) = {math.exp(-0.017):.3f}")
print("\n=== 8. Sensitivity-analysis crude rates (unaffected; check only) ===")
for ev, py, rep in ((34893, 38080361.9, 0.9), (11551, 1885715.8, 6.1), (635, 237683.5, 2.7)):
print(f" {ev} / {py} py = {1000 * ev / py:.2f} per 1000 py (reported {rep})")
# ------------------------------------------------------------------ figures
BG, INK, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, PANEL, EDGE = "#eb6834", "#2a78d6", "#e4e3df", "#f3f2ee", "#cfcdc6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": SEC, "ytick.color": INK,
"axes.edgecolor": EDGE, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
print(f" wrote {name}.svg and {name}.png")
def fmt(o, lo, hi):
return f"OR {o:.2f} (95% CI {lo:.2f}–{hi:.2f})"
print("\n=== Figures ===")
# ---------------- Figure 1: Abstract vs Results text panels
fig = plt.figure(figsize=(7.2, 5.5))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def panel(x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc=PANEL, ec=EDGE, lw=1))
ax.text(x + 2, y + h - 3, title, fontsize=11, weight="bold", va="top")
# left: Abstract
panel(2, 33, 43, 59, "Abstract, Results")
ax.text(4, 83, "“Thirty-day case-fatality was similar\namong people with type 2 diabetes\n"
"mellitus and without diabetes mellitus,\nbut was higher in type 1 diabetes\nmellitus for…”",
fontsize=10, va="top", color=INK, linespacing=1.35)
ax.text(4, 62, "men", fontsize=10, color=SEC, va="center")
ax.text(13, 62, fmt(*ABSTRACT["men"]), fontsize=10, color=ORANGE, weight="bold", va="center")
ax.text(4, 55.5, "women", fontsize=10, color=SEC, va="center")
ax.text(13, 55.5, fmt(*ABSTRACT["women"]), fontsize=10, color=ORANGE, weight="bold", va="center")
ax.text(4, 45, "Claim: higher in type 1.\nBoth ORs < 1, both CIs below 1:\n4% and 2% lower odds.",
fontsize=10, color=SEC, va="center", linespacing=1.3)
# right: Results
panel(55, 33, 43, 59, "Results, Case-Fatality")
ax.text(57, 83, "¶2 type 2 vs no diabetes: “similar”", fontsize=10, va="top", weight="bold")
ax.text(57, 76, "men", fontsize=10, color=SEC, va="center")
ax.text(66, 76, fmt(*RES_T2["men"]), fontsize=10, color=ORANGE, weight="bold", va="center")
ax.text(57, 69.5, "women", fontsize=10, color=SEC, va="center")
ax.text(66, 69.5, fmt(*RES_T2["women"]), fontsize=10, color=ORANGE, weight="bold", va="center")
ax.text(57, 60, "¶3 type 1 vs no diabetes: “higher”", fontsize=10, va="top", weight="bold")
ax.text(57, 53, "men", fontsize=10, color=SEC, va="center")
ax.text(66, 53, fmt(*RES_T1["men"]), fontsize=10, color=INK, weight="bold", va="center")
ax.text(57, 46.5, "women", fontsize=10, color=SEC, va="center")
ax.text(66, 46.5, fmt(*RES_T1["women"]), fontsize=10, color=INK, weight="bold", va="center")
ax.text(57, 39, "Discussion: “1.3-fold higher in women\nand 1.9-fold higher in men”",
fontsize=10, color=SEC, va="center", linespacing=1.3)
# connector: Abstract numbers -> identical type 2 numbers
ax.add_patch(FancyArrowPatch((44.5, 59), (56, 73), connectionstyle="arc3,rad=-0.25",
arrowstyle="-|>", mutation_scale=12, lw=2, color=ORANGE))
ax.text(50, 95.5, "Identical digits: the Abstract’s “type 1” ORs are the Results’ type 2 ORs",
fontsize=10.5, ha="center", va="center", weight="bold", color=INK)
ax.text(50, 52, "same\nnumbers", fontsize=10, ha="center", va="center", color=SEC, linespacing=1.2,
bbox=dict(boxstyle="round,pad=0.25", fc=BG, ec="none"))
# bottom strip: recomputed from Table S11
ax.add_patch(FancyBboxPatch((2, 7.5), 96, 21.5, boxstyle="round,pad=0,rounding_size=1.2",
fc=BG, ec=BLUE, lw=1.2))
ax.text(4, 26, "Recomputed from the paper’s own model: exp(coefficient), Supplement Table S11",
fontsize=10, weight="bold", va="top")
ax.text(4, 19, "type 2 vs none", fontsize=10, color=SEC, va="center")
ax.text(24, 19, f"men exp({b2}) = {S11OR[('t2', 'men')]:.3f}", fontsize=10, color=BLUE, weight="bold",
va="center")
ax.text(58, 19, f"women exp({b2 + bi2:.3f}) = {S11OR[('t2', 'women')]:.3f}", fontsize=10, color=BLUE,
weight="bold", va="center")
ax.text(4, 12.5, "type 1 vs none", fontsize=10, color=SEC, va="center")
ax.text(24, 12.5, f"men exp({b1}) = {S11OR[('t1', 'men')]:.3f}", fontsize=10, color=BLUE, weight="bold",
va="center")
ax.text(58, 12.5, f"women exp({b1 + bi1:.3f}) = {S11OR[('t1', 'women')]:.3f}", fontsize=10, color=BLUE,
weight="bold", va="center")
leg = [Line2D([], [], marker="s", ls="", ms=8, color=ORANGE, label="printed, type 2 (Abstract, ¶2)"),
Line2D([], [], marker="s", ls="", ms=8, color=INK, label="printed, type 1 (¶3)"),
Line2D([], [], marker="s", ls="", ms=8, color=BLUE, label="recomputed (Table S11)")]
fig.legend(handles=leg, loc="lower center", bbox_to_anchor=(0.5, -0.005), ncol=3, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.2)
fig.subplots_adjust(bottom=0.06)
save(fig, "fig1")
# ---------------- Figure 2: one log axis, the three printed estimates per sex
rows = [] # (y, label, printed (o,lo,hi), recomputed OR for the comparison the row names)
y = 0
for sex in ("women", "men"):
rows.append((y, "Results ¶2: type 2 vs none", RES_T2[sex], S11OR[("t2", sex)], sex)); y += 1
rows.append((y, "Results ¶3: type 1 vs none", RES_T1[sex], S11OR[("t1", sex)], sex)); y += 1
rows.append((y, "Abstract: “higher in type 1”", ABSTRACT[sex], S11OR[("t1", sex)], sex)); y += 1.9
fig, ax = plt.subplots(figsize=(7.2, 4.6))
ax.set_xscale("log")
ax.set_xlim(0.8, 2.6)
ax.axvline(1, color=SEC, lw=1, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for yy, lab, (o, lo, hi), rec, sex in rows:
ax.plot([lo, hi], [yy, yy], color=ORANGE, lw=2, solid_capstyle="round", zorder=3)
ax.plot(o, yy, "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.plot(rec, yy, "D", ms=9, mfc="none", mec=BLUE, mew=2, zorder=5)
if lab.startswith("Abstract"):
ax.annotate("", xy=(rec * 0.965, yy), xytext=(o * 1.04, yy),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=1.5, ls=(0, (3, 2)), mutation_scale=11),
zorder=2)
ax.text(rec * 1.045, yy, f"{rec / o:.2f}× the\nprinted value", fontsize=10, color=SEC,
ha="left", va="center", linespacing=1.15)
ax.set_yticks([r[0] for r in rows])
ax.set_yticklabels([r[1] for r in rows], fontsize=10)
for sex, yy in (("Women", rows[2][0] + 0.75), ("Men", rows[5][0] + 0.75)):
ax.text(0.0, yy, sex, transform=ax.get_yaxis_transform(), fontsize=11, weight="bold", ha="right",
va="center")
ax.set_ylim(-0.6, rows[-1][0] + 1.05)
ticks = [0.8, 0.9, 1.0, 1.25, 1.5, 2.0, 2.5]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_xlabel("Odds ratio for 30-day case-fatality vs no diabetes (log scale)")
ax.text(0.985, 1.0, "← lower odds", fontsize=10, color=SEC, ha="right", va="bottom",
transform=ax.get_xaxis_transform())
ax.text(1.015, 1.0, "higher odds →", fontsize=10, color=SEC, ha="left", va="bottom",
transform=ax.get_xaxis_transform())
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
leg = [Line2D([], [], marker="o", color=ORANGE, lw=2, ms=9, label="printed in the paper (OR, 95% CI)"),
Line2D([], [], marker="D", ls="", ms=9, mfc="none", mec=BLUE, mew=2,
label="recomputed from Table S11 (comparison in the row)")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=2, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.2)
fig.subplots_adjust(left=0.35, right=0.97, top=0.86, bottom=0.12)
save(fig, "fig2")
Survival difference called significant in abstract has log-rank P = 0.055, above the stated 0.05 threshold
Clin Cancer Res (2024) · doi:10.1158/1078-0432.CCR-23-1242 · PMC11474168
Summary
The abstract, the Discussion and the closing summary describe overall survival (OS) in PAM50 HER2-enriched triple-negative breast cancer as "significantly" better than, longer than or different from OS in basal tumors. The only test the paper reports for this comparison is a log-rank test in the Figure 5B legend, with P = 0.055. The Methods set a two-tailed significance level of 0.05, so the result is not significant by the paper's own criterion, and re-running the log-rank test on the published Supplementary Table S2 gives the same answer (χ² = 3.680, P = 0.0551). This is the only survival finding in the abstract and the authors present it as a new observation. The direction of the difference is unchanged; only the claim of statistical significance is unsupported.
What the paper reports
Abstract, Results:
The majority of TNBC were classified as basal or HER2-enriched by PAM50, the latter showing significantly improved overall survival.
Results, section "Overall survival and molecular subtypes", second paragraph (neutral wording):
One notable finding from this analysis was a prolongation of OS in patients whose tumors, although TNBC by histology, classified as HER2-enriched on PAM50 subtyping (Fig. 5B).
Figure 5 legend, panel B (the reported test):
B, Kaplan–Meier plot showing overall survival for patients with TNBC based on PAM50 gene subtype. Basal is marked in red and HER2 is marked in pink. Numbers of patients at risk over time are depicted under the graph (log-rank test, P = 0.055).
Discussion, fifth paragraph:
In addition, we found a significantly longer OS among the small number of patients whose tumors were classified as “HER2-enriched” by PAM50. This is, to our knowledge, a new observation and might reflect an inherent favorable biology in these tumors.
Discussion, closing summary paragraph:
As for patients’ survival, we observed that the two major PAM50 subtypes had significantly different OS as well as a trend for higher immune-related gene expression in patients with better OS.
Materials and Methods, "Survival analysis":
For all Kaplan-Meier plots, a log-rank test was used for generating P values.
Materials and Methods, "Statistical analyses":
All tests were two-tailed, the statistically significant level was set at 0.05, and P values were not adjusted for multiple comparisons.
Results, section on PDL1 positivity (the same threshold applied to a P value of similar size):
There was a nonsignificant (P = 0.056) positive correlation between the percentage of PDL1-positive TC and IC
What the paper's own numbers show
Method. I compared the reported log-rank P with the stated α, checked whether rounding or a different number of groups could bring it below 0.05, and converted it to the implied test statistic. I then re-ran the log-rank test on the per-patient data published as Supplementary Table S2 (sheet "metadata_with_normalized_genes": PAM50 call, vital status, days to event). The test was written by hand in the Mantel–Haenszel form used by R's survdiff and checked against scipy.stats.logrank. As in the main analysis, deaths are events and patients recorded "Alive" or "unknown" are censored. The supplement is read into memory only, and no per-patient value is reproduced here. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Log-rank P, OS, basal vs HER2-enriched | Fig. 5B legend | 0.055 | 0.055 > 0.05; P − α = 0.005 |
| Significance level | Methods, "Statistical analyses" | 0.05, two-tailed | — |
| Values that round to 0.055 | derived | — | 0.0545 to 0.0555, all above 0.05 |
| χ² implied by P = 0.055, df = 1 | derived | — | 3.682 (3.841 needed for P = 0.05) |
| χ² implied by P = 0.055, df = 2 (if LumB were a third group) | derived | — | 5.801 (5.991 needed) |
| Implied two-sided |Z| | derived | — | 1.919 (1.960 needed) |
| PAM50 counts basal / HER2-E / LumB / not available | Table 1 | 64 / 12 / 3 / 16 (67 / 13 / 3 / 17%) | sum 79 + 16 = 95; percentages match |
| Log-rank re-run, basal vs HER2-enriched | Suppl. Table S2 | (paper: P = 0.055) | χ² = 3.680, df = 1, P = 0.0551; basal n = 63, 53 deaths (46.59 expected); HER2-E n = 12, 9 deaths (15.41 expected) |
| Kaplan–Meier median OS | Suppl. Table S2 | not stated | basal 510 days; HER2-enriched 1258 days |
| Variant: LumB added as third group | Suppl. Table S2 | not reported | χ² = 4.661, df = 2, P = 0.097 |
| Variant: 5 basal patients with "unknown" vital status dropped | Suppl. Table S2 | not reported | χ² = 3.848, df = 1, P = 0.0498 |
| One-sided P | derived | not used (Methods: two-tailed) | 0.0275 |
The re-run on the supplementary data reproduces the published P to three decimals, so the P value itself is correct. Under the paper's stated convention the comparison is not statistically significant. HER2-enriched patients did have numerically longer median OS (1258 vs 510 days), and fewer deaths than expected under the null (9 observed vs 15.41 expected). The evidence amounts to a trend from 12 patients.
Figure
Figure 1. P values for the PAM50 overall-survival comparison on a linear axis from 0. The thin vertical line is the paper's stated two-tailed α = 0.05. Orange: the P printed in the Figure 5B legend, 0.055; the short bar marks every value that rounds to it (0.0545 to 0.0555). Filled blue: the same basal vs HER2-enriched log-rank test re-run on Supplementary Table S2, P = 0.0551. Open blue: variants the paper does not report. Adding the 3 LumB patients as a third group gives P = 0.097. Dropping the 5 basal patients with "unknown" vital status instead of censoring them gives P = 0.0498. Halving the reported P, as a one-sided test would, gives 0.0275, but the Methods state that all tests were two-tailed. The analysis that reproduces the paper's own P lies above α, as does every value consistent with the printed figure.
Why it matters
- It is the abstract's only survival result. The abstract's Results end with "the latter showing significantly improved overall survival", and the Discussion calls the finding new and suggests it "might reflect an inherent favorable biology in these tumors". A reader of the abstract would take this as a statistically supported prognostic difference between PAM50 subtypes within TNBC.
- The paper's own threshold says otherwise. The paper applies 0.05 strictly elsewhere: it calls a PDL1 correlation with P = 0.056 "nonsignificant", and describes OS differences between TNBC subtypes, AR status, PDL1 status and Rb status as not significant. The PAM50 result, at P = 0.055, is the one exception.
- The evidence is thin. The comparison rests on 12 HER2-enriched patients (9 deaths). An unreported variant, excluding rather than censoring 5 patients with unknown vital status, moves P to just below 0.05 (0.0498), and adding the 3 LumB patients moves it to 0.097. A result this close to the threshold is fragile however it is classified.
- The main conclusions are unaffected. The paper is mainly a multiomic resource and biomarker-correlation study. Its findings on Rb mRNA and protein versus RB1 mutation, the two LAR clusters, PDL1 TC/IC concordance and HER2-low versus HER2-zero do not depend on this survival comparison.
Explanations considered
- The 0.05 level applies only to the PRISM analyses. The sentence setting α sits in the "Statistical analyses" paragraph, which describes Fisher, χ² and Mann–Whitney tests run in PRISM. Survival was analysed separately in R, and that paragraph names no α. But the paper states no other threshold anywhere, and 0.055 is above 0.05 under any usual convention. This does not explain the wording.
- Rounding. Every value that rounds to 0.055 lies between 0.0545 and 0.0555, above 0.05, and the re-run gives 0.0551. Does not explain it.
- A one-sided test. A one-sided P would be 0.0275, but nothing in the paper says the survival test was one-sided, and the only stated convention is two-tailed. Does not explain it.
- A different number of groups. If the legend's P came from a three-group test including LumB, the implied χ² (5.801, df = 2) is still below the 5.991 cutoff. The supplement gives P = 0.097 for three groups and 0.0551 for two, so the printed 0.055 is the two-group test. Does not explain it.
- A different test or model. No Cox model, adjusted analysis or other P value for the PAM50 OS comparison appears in the text, tables or figure legends. The Results cite Supplementary Fig. S7 only for features of HER2-enriched tumors (S7A–S7G) and for OS by TNBC subtype (S7H). Does not explain it.
- A different handling of patients with unknown vital status. Excluding the 5 basal patients whose vital status is "unknown" gives P = 0.0498. The paper does not report this analysis, and the published P (0.055) is the one obtained by censoring them, which is the usual practice. This variant shows the result is borderline; it does not show the reported analysis was significant.
- A text-conversion artifact. Ruled out. The PMC page, the Europe PMC full-text JATS XML and the PubMed abstract carry the same wording and the same "P = 0.055" as plain text, and the supplementary data reproduce that value.
- A correction already issued. None found (see Status).
Status
- Version of record. The publisher's page (aacrjournals.org, reached through doi.org) returned HTTP 403 to automated requests. The PMC page for PMC11474168, the Europe PMC full-text XML (publisher JATS) and the PubMed record (PMID 39136550) all contain the quoted sentences and the Figure 5B P value as plain text. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The supplements are deposited on Figshare under the Crossref "is-supplemented-by" DOIs 10.1158/1078-0432.27231288 to .27231324. Supplementary Table S2 (Figshare file 49797945) holds per-patient PAM50 calls, vital status and days to event, and was used for the re-run above. It confirms the published P and does not support a claim of significance at 0.05.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref updates and no PMC or Europe PMC notices. The Crossref record has no update-to or updated-by entries; its relations list only has-preprint and is-supplemented-by. The Europe PMC record has no comment or correction list, and the PubMed record has no CommentsCorrections entries. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC11474168; Europe PMC full-text XML and record; PubMed 39136550; Crossref; Figshare supplementary files (Table S2, file 49797945); PubPeer; https://doi.org/10.1158/1078-0432.CCR-23-1242.
Suggested correction
In the abstract, replace
the latter showing significantly improved overall survival.
with wording such as
the latter showing a trend toward longer overall survival that did not reach statistical significance (log-rank P = 0.055).
In the Discussion, replace "we found a significantly longer OS among the small number of patients" with "we observed a numerically longer OS (log-rank P = 0.055) among the small number of patients", and in the closing summary replace "the two major PAM50 subtypes had significantly different OS" with "the two major PAM50 subtypes showed a trend toward different OS". The authors may also wish to state the threshold used for the survival analyses in the "Survival analysis" paragraph, and to give the group sizes and the handling of patients with unknown vital status in the Figure 5B legend.
Recomputation script (python recompute.py.txt)
"""Recheck the PAM50 overall-survival significance claim in PMC11474168
(Clin Cancer Res 2024, doi:10.1158/1078-0432.CCR-23-1242,
"Integrative Multiomic Profiling of Triple-Negative Breast Cancer for Identifying Suitable Therapies").
Run: python recompute.py.txt [path/to/ccr-23-1242_supplementary_table_s2_suppst2.xlsx]
Needs numpy, scipy and matplotlib; openpyxl for part 3. Prints every recomputed number used
in writeup.md and writes fig1.svg / fig1.png next to this file.
Part 1-2 use only numbers printed in the paper as archived in PMC:
Fig. 5B legend: "(log-rank test, P = 0.055)", basal vs HER2-enriched (PAM50).
Methods, Statistical analyses: "All tests were two-tailed, the statistically significant
level was set at 0.05". Methods, Survival analysis: "For all Kaplan-Meier plots, a
log-rank test was used for generating P values."
Table 1: PAM50 Basal 64, HER2-E 12, LumB 3, Not available 16, N = 95.
Results: PDL1 TC vs IC correlation called "nonsignificant (P = 0.056)".
Part 3 re-runs the log-rank test on the published Supplementary Table S2 (Figshare file
49797945). It is read into memory only; no per-patient value is written to disk or printed.
If the file cannot be read or downloaded, part 3 is skipped and the figure uses the values
recorded below (from a run of this same code on 2026-10-24).
"""
import io
import os
import sys
import urllib.request
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
S2_URL = "https://api.figshare.com/v2/file/download/49797945"
# ---------------------------------------------------------------- inputs printed in the paper
ALPHA = 0.05 # Methods, two-tailed
P_REP = 0.055 # Fig. 5B legend
LO, HI = 0.0545, 0.0555 # every value that rounds to 0.055 (3 decimals)
TABLE1 = dict(Basal=64, HER2E=12, LumB=3, NA=16)
N_COHORT = 95
N_OS = 94 # Results: OS evaluated "in 94 patients"
P_PDL1_TEXT = 0.056 # Results: "nonsignificant (P = 0.056)"
P_PDL1_LEGEND = 0.0567 # Fig. 3A legend
# ---------------------------------------------------------------- 1. the reported P vs alpha
print("=== 1. Reported log-rank P (Fig. 5B) against the stated alpha ===")
print(f"reported P = {P_REP}; alpha = {ALPHA}; P > alpha: {P_REP > ALPHA}; P - alpha = {P_REP - ALPHA:.3f}")
print(f"rounding interval [{LO}, {HI}); lower end above alpha: {LO > ALPHA}")
chi_rep = {}
for df in (1, 2):
chi_rep[df] = stats.chi2.isf(P_REP, df)
print(f"df={df}: chi2 implied by P = 0.055 is {chi_rep[df]:.3f}; chi2 needed for P = 0.05 is {stats.chi2.isf(ALPHA, df):.3f}")
print(f"implied |Z| (two-sided) = {stats.norm.isf(P_REP / 2):.3f}; needed {stats.norm.isf(ALPHA / 2):.3f}")
print(f"one-sided P, if a direction had been prespecified (Methods say two-tailed) = {P_REP / 2:.4f}")
print(f"elsewhere: PDL1 TC vs IC P = {P_PDL1_TEXT} (legend {P_PDL1_LEGEND}) is called 'nonsignificant'")
# ---------------------------------------------------------------- 2. Table 1 counts
print()
print("=== 2. Table 1 PAM50 counts ===")
classified = TABLE1["Basal"] + TABLE1["HER2E"] + TABLE1["LumB"]
print(f"classified = {classified} (Table 1 total 79); + not available {TABLE1['NA']} = {classified + TABLE1['NA']} (N = {N_COHORT})")
print("percent of 95:", {k: round(100 * v / N_COHORT) for k, v in TABLE1.items()}, "classified", round(100 * classified / N_COHORT))
print(f"basal + HER2-E = {TABLE1['Basal'] + TABLE1['HER2E']} patients at most in Fig. 5B (OS cohort {N_OS})")
# ---------------------------------------------------------------- 3. re-run on Supplementary Table S2
def logrank(time, event, group):
"""k-group log-rank test (Mantel-Haenszel form, as in R survdiff). Returns chi2, df, P."""
time, event, group = map(np.asarray, (time, event, group))
levels = sorted(set(group))
k = len(levels)
O = np.zeros(k); E = np.zeros(k); V = np.zeros((k, k))
for t in np.unique(time[event == 1]):
at_risk = time >= t
n = at_risk.sum()
d = ((time == t) & (event == 1)).sum()
nj = np.array([(at_risk & (group == g)).sum() for g in levels])
dj = np.array([((time == t) & (event == 1) & (group == g)).sum() for g in levels])
O += dj
E += d * nj / n
if n > 1:
f = d * (n - d) / (n - 1)
V += f * (np.diag(nj / n) - np.outer(nj, nj) / n ** 2)
u = (O - E)[:-1]
chi = float(u @ np.linalg.solve(V[:-1, :-1], u))
return chi, k - 1, float(stats.chi2.sf(chi, k - 1)), dict(zip(levels, zip(O, E)))
def km_median(time, event):
order = np.argsort(time)
t, e = np.asarray(time)[order], np.asarray(event)[order]
s = 1.0
for ti in np.unique(t[e == 1]):
n = (t >= ti).sum(); d = ((t == ti) & (e == 1)).sum()
s *= 1 - d / n
if s <= 0.5:
return ti
return np.nan
def load_s2():
src = sys.argv[1] if len(sys.argv) > 1 else None
try:
if src:
data = open(src, "rb").read()
else:
req = urllib.request.Request(S2_URL, headers={"User-Agent": "Mozilla/5.0"})
data = urllib.request.urlopen(req, timeout=60).read()
import openpyxl
ws = openpyxl.load_workbook(io.BytesIO(data), read_only=True)["metadata_with_normalized_genes"]
rows = [r for r in ws.iter_rows(min_row=4, values_only=True) if r[3] is not None]
# columns: E "Vitial Status at time of data curation", F "Days to event", N "PAM50"
return [(r[13], r[4], r[5]) for r in rows]
except Exception as exc: # noqa: BLE001
print(f"(Supplementary Table S2 not available: {exc}; part 3 skipped)")
return None
# values from a run of part 3 on 2026-10-24, used only if the supplement cannot be read
SUPP = {"main": 0.0551, "lumb": 0.0974, "drop_unknown": 0.0498}
print()
print("=== 3. Log-rank test re-run on Supplementary Table S2 ===")
rows = load_s2()
if rows is not None:
print(f"rows: {len(rows)}; PAM50 counts:",
{g: sum(1 for r in rows if r[0] == g) for g in ("Basal", "HER2-E", "LumB", "NA")})
usable = [r for r in rows if isinstance(r[2], (int, float))]
print(f"rows with numeric days to event: {len(usable)} (one Basal row has 'na')")
def run(groups, drop_unknown=False, label=""):
sel = [r for r in usable if r[0] in groups and not (drop_unknown and r[1] == "unknown")]
t = np.array([r[2] for r in sel], float)
e = np.array([1 if r[1] == "Dead" else 0 for r in sel]) # Alive / unknown -> censored
g = np.array([r[0] for r in sel])
chi, df, p, oe = logrank(t, e, g)
desc = ", ".join(f"{k}: n={int((g == k).sum())}, deaths={int(o)}, expected={ex:.2f}" for k, (o, ex) in oe.items())
print(f"{label}: chi2 = {chi:.3f}, df = {df}, P = {p:.4f} [{desc}]")
return t, e, g, p
t, e, g, p_main = run(("Basal", "HER2-E"), label="Basal vs HER2-E, 'unknown' censored (as Fig. 5B)")
for k in ("Basal", "HER2-E"):
print(f" KM median OS {k}: {km_median(t[g == k], e[g == k]):.0f} days")
try:
x = stats.logrank(stats.CensoredData(uncensored=t[(g == 'Basal') & (e == 1)], right=t[(g == 'Basal') & (e == 0)]),
stats.CensoredData(uncensored=t[(g == 'HER2-E') & (e == 1)], right=t[(g == 'HER2-E') & (e == 0)]))
print(f" cross-check scipy.stats.logrank: chi2 = {x.statistic ** 2:.3f}, P = {x.pvalue:.4f}")
except AttributeError:
print(" (scipy.stats.logrank needs scipy >= 1.11; cross-check skipped)")
_, _, _, p_lumb = run(("Basal", "HER2-E", "LumB"), label="Sensitivity: three groups incl. LumB")
_, _, _, p_drop = run(("Basal", "HER2-E"), drop_unknown=True, label="Sensitivity: 'unknown' vital status dropped")
n_unk = sum(1 for r in usable if r[0] in ("Basal", "HER2-E") and r[1] == "unknown")
print(f" patients with 'unknown' vital status among Basal/HER2-E: {n_unk}")
SUPP = {"main": p_main, "lumb": p_lumb, "drop_unknown": p_drop}
print(f"P_main rounds to {p_main:.3f}; matches reported 0.055: {round(p_main, 3) == P_REP}")
# ---------------------------------------------------------------- figure 1
ORANGE, BLUE, FG, FG2, BG = "#eb6834", "#2a78d6", "#0b0b0b", "#52514e", "#fcfcfb"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": FG,
"axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2})
fig, ax = plt.subplots(figsize=(7.2, 4.1), facecolor=BG)
ax.set_facecolor(BG)
rows_fig = [
("Reported\n(Fig. 5B legend)", P_REP, (LO, HI), ORANGE, True,
"P = 0.055 (rounding range 0.0545–0.0555)"),
("Recomputed\n(Suppl. Table S2)", SUPP["main"], None, BLUE, True,
f"P = {SUPP['main']:.4f}"),
("Variant: LumB\nadded (3 groups)", SUPP["lumb"], None, BLUE, False,
f"P = {SUPP['lumb']:.3f}"),
("Variant: 5 'unknown'\nstatus dropped", SUPP["drop_unknown"], None, BLUE, False,
f"P = {SUPP['drop_unknown']:.4f} (not reported)"),
("One-sided version\nof reported P", P_REP / 2, None, BLUE, False,
f"P = {P_REP / 2:.4f} (not used by paper)"),
]
ys = np.arange(len(rows_fig))[::-1]
for y, (lab, p, ci, col, filled, txt) in zip(ys, rows_fig):
if ci:
ax.plot(ci, [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot(ci, [y, y], "|", color=col, ms=12, mew=2)
ax.plot(p, y, "o", ms=9, mew=2, color=col, mfc=col if filled else BG, zorder=3)
ax.text(p + 0.0035, y, txt, color=col, fontsize=10, va="center",
bbox=dict(boxstyle="square,pad=0.1", fc=BG, ec="none"))
ax.axvline(ALPHA, color=FG2, lw=1, zorder=1)
ax.text(ALPHA + 0.001, -0.62, "α = 0.05 (paper's Methods, two-tailed)", color=FG2, fontsize=10, ha="left", va="bottom")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows_fig], fontsize=10)
ax.set_xlim(0, 0.13)
ax.set_ylim(-0.75, len(rows_fig) - 0.5)
ax.set_xlabel("P value (linear scale from 0)")
ax.grid(axis="x", color="#e4e3df", lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(FG2)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D # noqa: E402
fig.legend(handles=[Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="reported in the paper"),
Line2D([], [], marker="o", ls="", color=BLUE, ms=9, label="recomputed (main analysis)"),
Line2D([], [], marker="o", ls="", color=BLUE, mfc=BG, mew=2, ms=9, label="recomputed (variant)")],
loc="upper center", bbox_to_anchor=(0.5, 1.0), ncol=3, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
fig.tight_layout(rect=(0, 0, 1, 0.92))
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
print()
print("wrote fig1.svg and fig1.png")
Abstract reports all-grade adverse event rates as grade ≥3 treatment-related rates in a phase I trial
Clin Cancer Res (2025) · doi:10.1158/1078-0432.CCR-24-1983 · PMC12163594
Summary
The Abstract of this first-in-human phase I trial of intratumoral SAR441000, given alone or with cemiplimab, states that the most common grade ≥3 treatment-related adverse events (TRAEs) were fatigue in escalation (28.6% with monotherapy, 66.7% with the combination) and injection-site pain in expansion (31.7%). All three percentages equal the all-grade, all-causality treatment-emergent AE (TEAE) rates in Table 2B (6/21, 10/15 and 13/41). In the same table, grade ≥3 fatigue is 0/21 and 0/15, and grade ≥3 injection-site pain is 1/41 (2.4%). The paper's own Table 2A and Results text add that no monotherapy patient had a grade ≥3 TRAE of any kind and that only 1 of 15 escalation-combination patients did. Safety was a primary objective, and this sentence is the Abstract's only quantitative safety result. As printed, it overstates severe treatment-related toxicity by about 10-fold or more.
What the paper reports
The most common grade ≥3 treatment–related adverse events was fatigue in the escalation phase (monotherapy: 28.6% and combination therapy: 66.7%) and injection-site pain (31.7%) in the expansion phase.
— Abstract, Results
Of the 21 patients, 19 (90.5%) experienced any-grade AEs, of which the most common were vomiting and fatigue in 6 (28.6%) patients each, followed by hypertension, chills, and pain in extremities in 4 (19.0%) patients each. Nine (42.9%) patients experienced grade ≥3 AEs (Table 2). Furthermore, 13 (61.9%) patients reported TRAEs, but none were grade ≥3 or considered serious; grade ≥5 AEs were not reported.
— Results, Safety, "Escalation monotherapy"
All 15 patients experienced any-grade AEs, with the most common being fatigue (10/15; 66.7%), followed by hypertension and nausea in 5 (33.3%) patients each. Grade ≥3 AEs—hypertension, pyrexia, and anemia—were experienced by one (6.7%) patient each (Table 2). Ten (66.7%) patients experienced any-grade TRAEs, with the most common being fatigue (n = 7; 46.7%). One (6.7%) patient experienced grade ≥3 syncope (Supplementary Table S5).
— Results, Safety, "Escalation combination therapy"
All 41 patients experienced any-grade AEs, of which injection-site pain (n = 13; 31.7%) was the most common all-grade AE, followed by diarrhea, decreased appetite, and asthenia (n = 11 each; 26.8%; Table 2). [...] Thirty-three (80.5%) patients had any-grade TRAEs, of which six (14.6%) patients had TRAEs of grade ≥3 that were diarrhea, injection-site pain, and fatigue (Supplementary Table S5).
— Results, Safety, "Expansion combination"
Patients with any TRAE of grade ≥3 | 0 | 1 (6.7) | 6 (14.6)
— Table 2A ("AEs: (A) summary and (B) TEAEs occurring in ≥15% of patients"). The columns are escalation monotherapy (N = 21), escalation combination (N = 15) and expansion combination (N = 41).
Fatigue | 6 (28.6) | 0 | 10 (66.7) | 0 | 9 (22.0) | 1 (2.4)
Injection-site pain | 0 | 0 | 0 | 0 | 13 (31.7) | 1 (2.4)
— Table 2B, same three cohorts. Each cohort has an "All grades" column followed by a "Grade ≥3" column. The table footer defines the rows as TEAEs.
Intratumoral administration of SAR441000 in combination with cemiplimab was generally well tolerated with antitumor activity in the locoregional disease setting.
— Abstract, Conclusions
What the paper's own numbers show
Method. Each incidence is 100 × (patients with the event) / (patients in the cohort), with the cohort sizes from the Abstract and Table 2 (21, 15, 41). For each of the Abstract's three figures, the script tries every incidence the paper reports for the same AE and cohort: all-grade and grade ≥3 rates, all-causality TEAEs (Table 2B) and treatment-related AEs (Supplementary Table S5). It also checks the figure against the cohort's ceiling for grade ≥3 TRAEs of any kind (Table 2A). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Fatigue, escalation monotherapy, labelled "grade ≥3 treatment-related" | Abstract | 28.6% | equals all-grade TEAE 6/21 = 28.6% (Table 2B) |
| Grade ≥3 fatigue, escalation monotherapy | Table 2B; Suppl. Table S5 | 0; 0 | 0/21 = 0.0% (all causes and treatment-related) |
| Any grade ≥3 TRAE, escalation monotherapy | Table 2A; Results | 0; "none were grade ≥3" | 0/21 = 0.0%. Grade ≥3 TR fatigue of 28.6% would need 6.0 patients |
| Fatigue, escalation combination, labelled "grade ≥3 treatment-related" | Abstract | 66.7% | equals all-grade TEAE 10/15 = 66.7% (Table 2B) |
| Grade ≥3 fatigue, escalation combination | Table 2B; Suppl. Table S5 | 0; 0 | 0/15 = 0.0% |
| Any grade ≥3 TRAE, escalation combination | Table 2A; Results | 1 (6.7%) | 1/15 = 6.7%. Grade ≥3 TR fatigue of 66.7% would need 10.0 patients |
| Treatment-related fatigue, any grade, escalation combination | Results; Suppl. Table S5 | n = 7; 46.7% | 7/15 = 46.7%, so 66.7% is not a TRAE rate at any grade |
| Injection-site pain, expansion, labelled "grade ≥3 treatment-related" | Abstract | 31.7% | equals all-grade TEAE 13/41 = 31.7% (Table 2B) |
| Grade ≥3 injection-site pain, expansion | Table 2B; Suppl. Table S5 | 1 (2.4); 1 (2.4) | 1/41 = 2.4%; the Abstract's figure is 13.0 times this |
| Treatment-related injection-site pain, any grade, expansion | Suppl. Table S5 | 11 (26.8) | 11/41 = 26.8% |
| Any grade ≥3 TRAE, expansion | Table 2A; Results | 6 (14.6%) | 6/41 = 14.6% |
| Largest number of patients with any single grade ≥3 TRAE term, expansion | Suppl. Table S5 | 7 terms with 1 (2.4) each | 1/41 = 2.4%; no term is more common than any other |
| Ratio of Abstract figure to the cohort's any-grade ≥3 TRAE rate | derived | — | monotherapy: 28.6% vs 0% (no finite ratio); combination: 66.7 / 6.7 = 10.0; expansion: 31.7 / 14.6 = 2.2 |
All three Abstract figures match the all-grade, all-causality TEAE rate exactly. None matches any grade ≥3 rate or any treatment-related rate the paper reports. In the two escalation cohorts the stated figures cannot be grade ≥3 TRAE rates at all. Even if every patient with a grade ≥3 TRAE of any kind had had grade ≥3 fatigue, the maximum would be 0/21 and 1/15.
Figure
Figure 1. The left panel shows the Abstract sentence and its three percentages (orange). Each arrow joins a percentage to the Table 2B row it reproduces exactly. In all three rows the match is the "All grades" cell (orange), the count of patients with the event at any grade and from any cause. The blue cells show the grade ≥3 rate recomputed from the adjacent "Grade ≥3" count in the same row: 0/21, 0/15 and 1/41. Underneath, the right panel gives the cohort-level ceiling from Table 2A (any grade ≥3 TRAE: 0/21, 1/15, 6/41) and the Results text on monotherapy and on treatment-related fatigue. Only patient counts and percentages printed in the paper, or computed from them by recompute.py.txt, are shown.
Figure 2. The same comparison on one axis (percent of patients in each cohort, 0–100%). Orange dots are the Abstract's figures. Each sits inside an open ring marking the all-grade, all-causality rate from Table 2B. Blue diamonds are the grade ≥3 rates of the same AE (Table 2B), and open blue squares are the cohort's rate of grade ≥3 TRAEs of any kind (Table 2A). That second rate is an upper bound for any single grade ≥3 TRAE. In escalation monotherapy the diamond and the square coincide at 0%. In each cohort the Abstract's figure lies well above that upper bound.
Why it matters
Overall safety and tolerability are part of the trial's primary objective. The Abstract sentence quoted above is its only quantitative statement on safety. A reader of the Abstract alone would conclude that two-thirds of patients in escalation combination had severe treatment-related fatigue, and that nearly a third of expansion patients had severe treatment-related injection-site pain. The paper's tables say neither happened: no escalation patient had grade ≥3 fatigue, and 1 of 41 expansion patients had grade ≥3 injection-site pain. The sentence also misidentifies the "most common" grade ≥3 TRAE. Monotherapy had none. In escalation combination, the one patient with a grade ≥3 TRAE is described as having syncope. In expansion, Supplementary Table S5 lists seven grade ≥3 TRAE terms with one patient each, so no single term stands out.
The error runs against the paper's own conclusion. As printed, the Abstract's figures sit uneasily with "generally well tolerated"; the correct figures (grade ≥3 TRAEs of any kind in 0%, 6.7% and 14.6% of patients) support it. Abstract-level safety figures from first-in-human trials are often carried into reviews, meta-analyses of intratumoral immunotherapies and dose-selection discussions. A 10-fold or larger overstatement of severe toxicity could be passed on in those places.
The error does not affect the maximum administered dose (4,000 µg), the response counts, the pharmacokinetic and pharmacodynamic results, or the Discussion. The Discussion describes AE frequencies without the mislabelled percentages. It ends: "SAR441000 demonstrated a generally acceptable safety profile in both monotherapy and combination therapy cohorts, although a higher number of high-grade AEs were observed in the combination therapy cohort."
Explanations considered
- A different population or denominator. This does not account for the numbers. The Abstract gives the same cohort sizes (N = 21, 15, 41) as Table 2. The ratios 6/21, 10/15 and 13/41 reproduce 28.6%, 66.7% and 31.7% exactly.
- A treatment-related table rather than Table 2B. This does not account for the numbers. Supplementary Table S5 ("Treatment-related adverse events") gives fatigue as 2 (9.5) and 7 (46.7) at all grades and 0 at grade ≥3 in the two escalation cohorts. It gives injection-site pain in expansion as 11 (26.8) at all grades and 1 (2.4) at grade ≥3. None of these is 28.6%, 66.7% or 31.7%.
- A grade ≥3 rate that includes events not attributed to treatment. This does not account for the numbers. Grade ≥3 fatigue from any cause is also 0/21 and 0/15 (Table 2B), and grade ≥3 injection-site pain from any cause is 1/41.
- Patients counted once per event, or events rather than patients. This does not account for the numbers. Ten grade ≥3 fatigue events in escalation combination would still need grade ≥3 fatigue to appear in Table 2B or Supplementary Table S5, and both show 0. The cohort had only one patient with any grade ≥3 TRAE.
- A different time window or study phase. This does not account for the numbers. The Abstract sentence, Table 2 and the Results subsections all refer to the escalation and expansion phases over the treatment-emergent period. The table footnotes (timing of grade 5 events, definition of discontinuation) do not apply to these rows.
- A conversion or typesetting artefact in the archived copy. This was ruled out. The PMC HTML, the Europe PMC full-text XML and the PubMed abstract (PMID 40152791) give the same Abstract sentence. Table 2B lines up with its "All grades | Grade ≥3" header in each source checked. The numbers contain no signs or superscripts that could have been misread.
- The Abstract meant "all-grade" or "TEAE" and the qualifier is a wording slip. This is the most likely description of what went wrong, but it is not an innocent reading of the sentence as printed. "Grade ≥3 treatment-related" changes both the severity and the attribution of the event, and the figures are wrong under that label.
A smaller inconsistency sits in the paper's supporting material. The Results text names the single grade ≥3 TRAE in escalation combination as syncope, citing Supplementary Table S5. That table lists syncope as 1 (6.7) at all grades and 0 at grade ≥3, and lists dizziness and peripheral neuropathy as 1 (6.7) each at grade ≥3. This does not change the main finding: Table 2A gives one patient with any grade ≥3 TRAE in that cohort either way.
Status
As of 2026-10-23:
- Version of record: checked indirectly. The publisher's article page and the PDF link listed by Crossref (aacrjournals.org, /clincancerres/article/31/12/2358/762781/) returned HTTP 403 to automated requests. The PMC article page, the Europe PMC full-text XML and the PubMed record (PMID 40152791, built from the publisher's metadata) all give the Abstract sentence and the Table 2 rows word for word as quoted above.
- Supplement: checked. The Europe PMC supplementary package (Supplementary Figures S1–S4, Supplementary Tables S1–S9, Annexure 1) was downloaded. Supplementary Table S5 (treatment-related AEs, all grades and grade ≥3 by cohort) is cited above. It confirms grade ≥3 treatment-related fatigue of 0/21 and 0/15, and grade ≥3 treatment-related injection-site pain of 1/41. It does not resolve the discrepancy.
- Corrections: none found.
src/known.pyreturned no Crossref updates, PMC notices or Europe PMC notices. The Crossref record has no update-to or updated-by entries. The Europe PMC record has no comment or correction links. A web search for errata or comments on the article found none. - PubPeer: not reachable by automated query. The API returned "Human verification required" (HTTP 403). A web search found no PubPeer thread. A manual search at pubpeer.com is still advisable.
Sources: doi.org/10.1158/1078-0432.CCR-24-1983 · PMC12163594 · Europe PMC record · Europe PMC supplementary files · Crossref works record · ClinicalTrials.gov NCT03871348
Suggested correction
The Abstract sentence could be replaced with figures that match Table 2 and Supplementary Table S5. One option:
Grade ≥3 treatment-related adverse events occurred in 0 of 21 (0%), 1 of 15 (6.7%) and 6 of 41 (14.6%) patients in escalation monotherapy, escalation combination and expansion combination, respectively. The most common any-grade adverse events were fatigue in escalation (monotherapy: 28.6%; combination: 66.7%) and injection-site pain (31.7%) in expansion.
The authors could also check which preferred term was recorded for the single grade ≥3 TRAE in escalation combination. The Results text names syncope, while Supplementary Table S5 lists dizziness and peripheral neuropathy.
Recomputation script (python recompute.py.txt)
"""Recompute the abstract's grade >=3 treatment-related AE figures of PMC12163594
(Clin Cancer Res 2025, doi:10.1158/1078-0432.CCR-24-1983) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, matplotlib.
Prints every recomputed number used in writeup.md. Writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file, plus five candidate figure
designs (cand1..cand5, .svg/.png) into ./candidates/.
All inputs are patient counts copied from the paper:
Table 2A (AE summary), Table 2B (TEAEs occurring in >=15% of patients, all causality),
the Results subsections "Escalation monotherapy", "Escalation combination therapy",
"Expansion combination", and Supplementary Table S5 (treatment-related AEs).
Incidence (%) = 100 * patients with the event / patients in the cohort.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ----------------------------------------------------------------------------- inputs
N = {"mono": 21, "comb": 15, "exp": 41}
LABEL = {"mono": "Escalation monotherapy", "comb": "Escalation combination",
"exp": "Expansion combination"}
# Abstract, Results: "The most common grade >=3 treatment-related adverse events was
# fatigue in the escalation phase (monotherapy: 28.6% and combination therapy: 66.7%)
# and injection-site pain (31.7%) in the expansion phase."
ABSTRACT = {"mono": ("Fatigue", 28.6), "comb": ("Fatigue", 66.7),
"exp": ("Injection-site pain", 31.7)}
# Table 2B (TEAEs, all causality): (all grades n, grade >=3 n)
TEAE = {
("mono", "Fatigue"): (6, 0),
("comb", "Fatigue"): (10, 0),
("exp", "Fatigue"): (9, 1),
("mono", "Injection-site pain"): (0, 0),
("comb", "Injection-site pain"): (0, 0),
("exp", "Injection-site pain"): (13, 1),
}
# Supplementary Table S5 (treatment-related AEs): (all grades n, grade >=3 n)
TRAE = {
("mono", "Fatigue"): (2, 0),
("comb", "Fatigue"): (7, 0),
("exp", "Fatigue"): (7, 1),
("mono", "Injection-site pain"): (0, 0),
("comb", "Injection-site pain"): (1, 0),
("exp", "Injection-site pain"): (11, 1),
}
# Table 2A: patients with any TRAE / any TRAE of grade >=3 (any preferred term)
ANY_TRAE = {"mono": 13, "comb": 10, "exp": 33}
ANY_TRAE_G3 = {"mono": 0, "comb": 1, "exp": 6}
# Table 2A: patients with grade >=3 TEAEs (any term, all causality)
ANY_TEAE_G3 = {"mono": 9, "comb": 5, "exp": 21}
# Supplementary Table S5: preferred terms with a grade >=3 TRAE, patients per term
S5_G3_TERMS = {
"mono": {},
"comb": {"Dizziness": 1, "Neuropathy peripheral": 1},
"exp": {"Diarrhoea": 1, "Muscular weakness": 1, "Fatigue": 1,
"Injection site pain": 1, "(term ending '...deterioration')": 1,
"Troponin increased": 1, "Hyperlipasaemia": 1},
}
def pct(n, d):
return round(100 * n / d, 1)
# ----------------------------------------------------------------------- recompute
print("=" * 78)
print("1. The abstract's three figures against the paper's own counts")
print("=" * 78)
rows = []
for k in ("mono", "comb", "exp"):
term, rep = ABSTRACT[k]
a, g3 = TEAE[(k, term)]
ta, tg3 = TRAE[(k, term)]
d = N[k]
r = dict(k=k, term=term, rep=rep, a=a, g3=g3, ta=ta, tg3=tg3, d=d,
p_all=pct(a, d), p_g3=pct(g3, d), p_tall=pct(ta, d), p_tg3=pct(tg3, d),
p_ceiling=pct(ANY_TRAE_G3[k], d), need=rep * d / 100)
rows.append(r)
print(f"\n{LABEL[k]} (N = {d}), {term}")
print(f" Abstract, labelled 'grade >=3 treatment-related': {rep}%")
print(f" All-grade TEAE, all causality (Table 2B): {a}/{d} = {r['p_all']}%"
f" -> matches abstract: {r['p_all'] == rep}")
print(f" Grade >=3 TEAE, all causality (Table 2B): {g3}/{d} = {r['p_g3']}%")
print(f" Any-grade treatment-related (Suppl. Table S5): {ta}/{d} = {r['p_tall']}%")
print(f" Grade >=3 treatment-related (Suppl. Table S5): {tg3}/{d} = {r['p_tg3']}%")
print(f" Any grade >=3 TRAE of any type (Table 2A): {ANY_TRAE_G3[k]}/{d} = {r['p_ceiling']}%")
print(f" Patients needed for {rep}%: {rep}*{d}/100 = {r['need']:.2f}"
f" (ceiling from Table 2A: {ANY_TRAE_G3[k]})")
print("\n" + "=" * 78)
print("2. Overstatement factors")
print("=" * 78)
for r in rows:
k = r["k"]
c = ANY_TRAE_G3[k]
if c == 0:
print(f"{LABEL[k]}: abstract {r['rep']}% vs any grade >=3 TRAE 0/{r['d']} = 0% (no finite ratio)")
else:
print(f"{LABEL[k]}: abstract {r['rep']}% / any grade >=3 TRAE {r['p_ceiling']}% = "
f"{(r['rep'] / 100) / (c / r['d']):.1f}x (in patients: {r['a']} vs {c})")
for r in rows:
if r["tg3"]:
print(f"{LABEL[r['k']]}: abstract {r['rep']}% vs grade >=3 TR {r['term'].lower()} "
f"{r['p_tg3']}% = {r['a'] / r['tg3']:.1f}x ({r['a']} vs {r['tg3']} patients)")
print("\n" + "=" * 78)
print("3. Other figures quoted in the write-up")
print("=" * 78)
for k in ("mono", "comb", "exp"):
print(f"{LABEL[k]}: any TRAE {ANY_TRAE[k]}/{N[k]} = {pct(ANY_TRAE[k], N[k])}%; "
f"grade >=3 TEAE (any) {ANY_TEAE_G3[k]}/{N[k]} = {pct(ANY_TEAE_G3[k], N[k])}%; "
f"grade >=3 TRAE (any) {ANY_TRAE_G3[k]}/{N[k]} = {pct(ANY_TRAE_G3[k], N[k])}%")
print(f"Expansion fatigue: all-grade TEAE 9/41 = {pct(9, 41)}%, grade >=3 1/41 = {pct(1, 41)}%")
print("Grade >=3 TRAE terms in Suppl. Table S5 (patients per term):")
for k in ("mono", "comb", "exp"):
terms = S5_G3_TERMS[k]
mx = max(terms.values()) if terms else 0
print(f" {LABEL[k]}: {len(terms)} terms, max patients per term = {mx}"
f"{' (' + str(round(100 * mx / N[k], 1)) + '%)' if mx else ''}: "
f"{', '.join(terms) if terms else 'none'}")
print(" Escalation combination: Results text names the one grade >=3 TRAE as 'syncope';"
" S5 lists syncope as 1 (6.7) all grades, 0 grade >=3.")
# --------------------------------------------------------------------------- style
BG, FG, FG2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
ORANGE_T, BLUE_T = "#fbe3d9", "#dde9f8"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": FG,
"axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG,
"axes.edgecolor": GRID, "figure.facecolor": BG, "axes.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", facecolor=BG)
fig.savefig(p + ".png", dpi=200, facecolor=BG)
plt.close(fig)
def box(ax, x, y, w, h, fc, ec, lw=1.0, r=0.012):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle=f"round,pad=0,rounding_size={r}",
fc=fc, ec=ec, lw=lw, transform=ax.transAxes))
SHORT = {"mono": "Escalation mono (N = 21)", "comb": "Escalation combo (N = 15)",
"exp": "Expansion combo (N = 41)"}
# ------------------------------------------------- candidate 1 / fig1: text panels
def draw_text_panels():
fig = plt.figure(figsize=(W, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
# left panel: abstract
L0, L1 = 0.025, 0.36
box(ax, L0, 0.20, L1 - L0, 0.71, "#ffffff", GRID)
ax.text(L0 + 0.015, 0.875, "Abstract, Results", fontsize=11, fontweight="bold", va="top")
ax.text(L0 + 0.015, 0.815,
"“The most common grade ≥3\ntreatment–related adverse\n"
"events was fatigue in the\nescalation phase\n(monotherapy: 28.6% and\n"
"combination therapy:\n66.7%) and injection-site\npain (31.7%) in the\n"
"expansion phase.”",
fontsize=10, va="top", color=FG, linespacing=1.3, style="italic")
ys = [0.395, 0.315, 0.235]
for y, r in zip(ys, rows):
box(ax, L0 + 0.012, y - 0.03, L1 - L0 - 0.024, 0.06, ORANGE_T, ORANGE, 1.2)
ax.text(L0 + 0.025, y, f"{r['rep']}%", color=FG, fontsize=10.5, fontweight="bold",
va="center")
ax.text(L0 + 0.105, y, f"{r['term'].replace('Injection-site pain', 'Inj.-site pain')},"
f" {('esc. mono', 'esc. combo', 'exp.')[ys.index(y)]}",
color=FG, fontsize=10, va="center")
# right panel: Table 2B + Table 2A + text
R0, R1 = 0.53, 0.985
box(ax, R0, 0.20, R1 - R0, 0.71, "#ffffff", GRID)
ax.text(R0 + 0.015, 0.875, "Table 2B (TEAEs, all causality)", fontsize=11,
fontweight="bold", va="top")
cA, cG = R0 + 0.27, R0 + 0.385
ax.text(cA, 0.765, "All\ngrades", ha="center", va="bottom", fontsize=10, color=FG2)
ax.text(cG, 0.765, "Grade\n≥3", ha="center", va="bottom", fontsize=10, color=FG2)
ax.plot([R0 + 0.015, R1 - 0.015], [0.755, 0.755], color=GRID, lw=1, transform=ax.transAxes)
names = ["Fatigue\nesc. mono, N=21", "Fatigue\nesc. combo, N=15",
"Inj.-site pain\nexpansion, N=41"]
yr = [0.68, 0.565, 0.45]
for y, nm, r in zip(yr, names, rows):
ax.text(R0 + 0.015, y, nm, fontsize=10, va="center", linespacing=1.15)
box(ax, cA - 0.06, y - 0.035, 0.12, 0.07, ORANGE_T, ORANGE, 1.2)
ax.text(cA, y, f"{r['a']} ({r['p_all']})", ha="center", va="center", fontsize=10.5,
fontweight="bold")
box(ax, cG - 0.045, y - 0.035, 0.09, 0.07, BLUE_T, BLUE, 1.2)
ax.text(cG, y, f"{r['p_g3']}%", ha="center", va="center", fontsize=10.5,
fontweight="bold")
ax.plot([R0 + 0.015, R1 - 0.015], [0.39, 0.39], color=GRID, lw=1, transform=ax.transAxes)
ax.text(R0 + 0.015, 0.37,
"Table 2A, any grade ≥3 TRAE:\n"
f" 0/21 · 1/15 ({rows[1]['p_ceiling']}%) · 6/41 ({rows[2]['p_ceiling']}%)\n"
"Results: monotherapy TRAEs “none\n were grade ≥3”; combo "
f"treatment-\n related fatigue 7/15 = {rows[1]['p_tall']}%",
fontsize=10, va="top", color=FG2, linespacing=1.3)
# connectors: abstract value -> all-grade cell
for y0, y1 in zip(ys, yr):
ax.annotate("", xy=(R0 - 0.004, y1), xytext=(L1 - 0.01, y0),
xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.4,
connectionstyle="arc3,rad=-0.15", shrinkA=0, shrinkB=2))
ax.text(0.445, 0.86, "same value,\ndifferent label:\n\u2018grade \u22653 TR\u2019\nvs \u2018all grades\u2019",
ha="center", va="top", fontsize=10, color=FG2, linespacing=1.3)
ax.text(0.5, 0.952, "Each abstract figure equals the all-grade, all-cause cell of Table 2B;\n"
"the grade \u22653 cells are 0, 0 and 1 patient.", ha="center", va="center",
fontsize=10, color=FG, fontweight="bold", linespacing=1.25)
# legend / key at bottom
box(ax, 0.025, 0.115, 0.03, 0.045, ORANGE_T, ORANGE, 1.2, r=0.006)
ax.text(0.065, 0.1375, "value printed in the paper (abstract figure and the Table 2B cell it equals)",
va="center", fontsize=10, color=FG2)
box(ax, 0.025, 0.045, 0.03, 0.045, BLUE_T, BLUE, 1.2, r=0.006)
ax.text(0.065, 0.0675, "grade ≥3 rate recomputed from Table 2B counts (n / N)",
va="center", fontsize=10, color=FG2)
return fig
save(draw_text_panels(), os.path.join(HERE, "fig1"), os.path.join(CAND, "cand1_text_panels"))
# ------------------------------------------------------ candidate 2 / fig2: dot plot
def draw_dots():
fig, ax = plt.subplots(figsize=(W, 3.5))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.17)
ylab = [f"{SHORT[r['k']]}\n{r['term']}" for r in rows]
yy = [2, 1, 0]
ax.set_xlim(-2, 100); ax.set_ylim(-0.6, 2.6)
ax.set_yticks(yy); ax.set_yticklabels(ylab, fontsize=10)
ax.set_xticks(range(0, 101, 20))
ax.set_xticklabels([f"{t}%" for t in range(0, 101, 20)])
ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Patients in cohort (%)", color=FG2)
for y, r in zip(yy, rows):
ax.plot([r["p_g3"], r["rep"]], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(r["p_all"], y, "o", ms=15, mfc="none", mec=FG2, mew=1.5, zorder=2)
ax.plot(r["rep"], y, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(r["p_ceiling"], y, "s", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=3)
ax.plot(r["p_g3"], y, "D", ms=8, color=BLUE, zorder=4)
ax.text(r["rep"] + 2.2, y + 0.08, f"{r['rep']}%", color=FG, fontsize=10, va="bottom",
fontweight="bold")
g3txt = f"{r['p_g3']}%" if r["p_g3"] == r["p_ceiling"] else \
f"{r['p_g3']}% / {r['p_ceiling']}%"
ax.text(max(r["p_g3"], r["p_ceiling"]) + 2.2, y - 0.1, g3txt, color=FG, fontsize=10,
va="top")
h = [plt.Line2D([], [], ls="none", marker="o", ms=9, color=ORANGE),
plt.Line2D([], [], ls="none", marker="o", ms=13, mfc="none", mec=FG2, mew=1.5),
plt.Line2D([], [], ls="none", marker="D", ms=8, color=BLUE),
plt.Line2D([], [], ls="none", marker="s", ms=9, mfc=BG, mec=BLUE, mew=2)]
lab = ["Abstract (‘grade ≥3 TRAE’)",
"All grades, all causes (Table 2B)",
"Grade ≥3, this AE (Table 2B)",
"Any grade ≥3 TRAE (Table 2A)"]
fig.legend(h, lab, loc="upper center", ncol=2, frameon=False, fontsize=10,
bbox_to_anchor=(0.56, 1.0), handletextpad=0.4, columnspacing=1.2)
return fig
save(draw_dots(), os.path.join(HERE, "fig2"), os.path.join(CAND, "cand2_dot_plot"))
# ------------------------------------------------------ candidate 3: grouped bars
def draw_bars():
fig, ax = plt.subplots(figsize=(W, 4.6))
fig.subplots_adjust(left=0.36, right=0.95, top=0.95, bottom=0.11)
metrics = [("Abstract (‘grade ≥3 TR’)", "rep", ORANGE),
("All grades, all cause", "p_all", BLUE),
("All grades, treatment-related", "p_tall", BLUE),
("Grade ≥3, all cause", "p_g3", BLUE),
("Grade ≥3, treatment-related", "p_tg3", BLUE)]
ticks, labs = [], []
y = 0
for gi, r in enumerate(rows):
for name, key, col in metrics:
v = r[key]
ax.barh(y, v, height=0.72, color=col, zorder=2)
ax.text(v + 1.2, y, f"{v}%", va="center", fontsize=10, color=FG)
ticks.append(y); labs.append(name)
y -= 1
ax.text(-0.53, y + len(metrics) + 0.95, f"{SHORT[r['k']]}: {r['term'].lower()}",
transform=ax.get_yaxis_transform(), fontsize=10, fontweight="bold", va="center")
y -= 1.6
ax.set_yticks(ticks); ax.set_yticklabels(labs, fontsize=10)
ax.set_xlim(0, 100)
ax.set_xticks(range(0, 101, 20)); ax.set_xticklabels([f"{t}%" for t in range(0, 101, 20)])
ax.xaxis.grid(True, color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Patients in cohort (%). Orange: abstract figure; blue: Table 2B / Suppl. Table S5 counts ÷ N",
color=FG2, fontsize=10, loc="right")
return fig
save(draw_bars(), os.path.join(CAND, "cand3_grouped_bars"))
# ------------------------------------------------------- candidate 4: calc flow
def draw_flow():
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
hdr = [(0.03, "Table 2B count"), (0.30, "÷ cohort N"), (0.53, "Result"),
(0.775, "Abstract label")]
for x, t in hdr:
ax.text(x, 0.94, t, fontsize=10, color=FG2, va="center", fontweight="bold")
ys = [0.74, 0.47, 0.20]
for y, r in zip(ys, rows):
term = r["term"].lower()
# all-grade path
box(ax, 0.03, y + 0.015, 0.24, 0.085, "#ffffff", GRID)
ax.text(0.04, y + 0.0575, f"{r['a']} pts, {term},\nany grade", fontsize=10, va="center")
ax.text(0.30, y + 0.0575, f"÷ {r['d']}", fontsize=10.5, va="center")
box(ax, 0.53, y + 0.015, 0.22, 0.085, ORANGE_T, ORANGE, 1.2)
ax.text(0.64, y + 0.0575, f"{r['p_all']}% (any grade)", fontsize=10, ha="center",
va="center")
ax.text(0.775, y + 0.0575, f"“grade ≥3 TR” {r['rep']}%\n= the any-grade rate",
fontsize=10, va="center", color=FG)
# grade >=3 path
box(ax, 0.03, y - 0.09, 0.24, 0.085, "#ffffff", GRID)
ax.text(0.04, y - 0.0475, f"{r['g3']} pt{'s' if r['g3'] != 1 else ''}, {term},\ngrade ≥3",
fontsize=10, va="center")
ax.text(0.30, y - 0.0475, f"÷ {r['d']}", fontsize=10.5, va="center")
box(ax, 0.53, y - 0.09, 0.22, 0.085, BLUE_T, BLUE, 1.2)
ax.text(0.64, y - 0.0475, f"{r['p_g3']}% (grade ≥3)", fontsize=10, ha="center",
va="center")
ax.text(0.775, y - 0.0475, "not reported\nin the abstract", fontsize=10, va="center",
color=FG2)
for yy in (y + 0.0575, y - 0.0475):
ax.annotate("", xy=(0.525, yy), xytext=(0.36, yy), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.4))
ax.text(0.03, y + 0.125, f"{LABEL[r['k']]} (N = {r['d']})", fontsize=10,
fontweight="bold", va="center")
ax.text(0.03, 0.035, "Orange: rate printed in the paper. Blue: grade ≥3 rate recomputed "
"from the same table.", fontsize=10, color=FG2, va="center")
return fig
save(draw_flow(), os.path.join(CAND, "cand4_calc_flow"))
# ------------------------------------------------------- candidate 5: patient grid
def draw_grid():
fig = plt.figure(figsize=(W, 4.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 72); ax.set_ylim(0, 37)
s, gap = 1.25, 0.35
x0 = 32
y = 33.5
for r in rows:
ax.text(1, y + 0.6, f"{LABEL[r['k']]}, {r['term'].lower()} (N = {r['d']})",
fontsize=10, fontweight="bold", va="center")
y -= 3.2
for lab, cnt, col, colt in (
(f"Abstract (‘grade ≥3 TR’): {r['rep']}% → {round(r['need'])} pts", round(r["need"]), ORANGE,
ORANGE_T),
(f"Table 2B grade ≥3: {r['g3']} pt{'s' if r['g3'] != 1 else ''}", r["g3"], BLUE,
BLUE_T)):
ax.text(1, y + s / 2, lab, fontsize=10, va="center")
for i in range(r["d"]):
xi = x0 + i * (s + gap) if r["d"] <= 27 else x0 + (i % 21) * (s + gap)
yi = y if r["d"] <= 27 or i < 21 else y - (s + gap)
filled = i < cnt
ax.add_patch(Rectangle((xi, yi), s, s, fc=col if filled else BG,
ec=col if filled else "#bdbbb5", lw=0.8))
y -= 3.2 if r["d"] <= 27 else 4.8
y -= 0.6
ax.text(1, 1.2, "Each square is one patient. Filled: patients with the event at the rate "
"named on the left.", fontsize=10, color=FG2, va="center")
return fig
save(draw_grid(), os.path.join(CAND, "cand5_patient_grid"))
print("\nFigures written to", HERE, "and", CAND)
Abstract reports 11 of 34 combination-arm responders; Results and Table 2 support 8 confirmed, 9 overall
Clin Cancer Res (2021) · doi:10.1158/1078-0432.CCR-21-2160 · PMC9401547
Summary
The abstract's only efficacy result for this first-in-human trial of MK-4830 is that "Eleven of 34 patients in the dose-escalation phase who received combination therapy achieved objective responses", which implies a response rate of 32.4%. The Results and Table 2 place those 11 responders across all 84 treated patients: 1 on monotherapy, 8 confirmed plus 1 unconfirmed among the 34 patients in the combination arm (part C), and 1 among the 18 patients who crossed over from monotherapy. The combination arm therefore has 8 of 34 confirmed responders (23.5%, the 24% ORR in Table 2), or 9 of 34 (26.5%) counting the unconfirmed one, and no grouping in the paper gives 11 responders among combination-treated patients. ORR was a tertiary endpoint and the safety conclusions are unaffected, but the abstract overstates the arm's response count by 3 patients and its response rate by 8.8 percentage points.
What the paper reports
All quotations were checked verbatim against the PMC-archived text.
Abstract, Results:
Of 84 patients, 50 received monotherapy and 34 received combination therapy.
Eleven of 34 patients in the dose-escalation phase who received combination therapy achieved objective responses; 5 previously had progressive disease on anti–PD-1/PD-L1 therapies.
Results, "Tumor response" (one paragraph, quoted in order):
Eleven of 84 patients (10 confirmed and 1 unconfirmed) with heavily pretreated advanced solid tumors in the dose-escalation phase of the trial achieved an objective response. One of 50 patients in the monotherapy group achieved a confirmed partial response (PR) for an ORR of 2% (Table 2)
Eight of 34 patients receiving combination therapy achieved a confirmed response (CR, n = 1; PR, n = 7) and 6 additional patients had SD ≥6 months (per RECIST v1.1 by investigator review) for an ORR of 24% and a DCR of 41% (Table 2; Supplementary Table S2). One additional patient in the combination treatment group achieved an unconfirmed PR. In the crossover group, 1 patient with MSI-high colorectal cancer achieved a confirmed PR.
In addition, 5 of 11 patients (45%) who achieved a response (4 confirmed and 1 unconfirmed) had previous disease progression on an anti–PD-1/PD-L1 alone or in combination with other agents.
Reductions in target lesion size of ≥30% were observed in the patient receiving monotherapy and the 10 patients receiving combination therapy (including those who crossed over; Fig. 2C and D).
Results, "Safety":
Of the 52 patients treated with combination therapy (n = 34 in part C; n = 18 who crossed over from parts A and B), 48 (92%) experienced AEs (Table 1)
Results, "Biomarker analysis > PD-L1":
Of the 34 patients in the dose-escalation phase who received combination therapy, 25 [6 responders (CR or PR), 19 nonresponders (SD, PD, or missing, including 5 patients with SD and PFS of ≥6 months)] had evaluable PD-L1 status (Fig. 3A).
Table 2, "Summary of confirmed tumor response per RECIST v1.1 by investigator assessment" (excerpt; columns are MK-4830 monotherapy, MK-4830 + pembrolizumab, crossover to MK-4830 + pembrolizumab):
Confirmed response, n (%) | n = 50 | n = 34 | n = 18
ORR | 1 (2)a | 8 (24)b | 1 (6)c
CR | 0 | 1 (3) | 0
PR | 1 (2) | 7 (21) | 1 (6)
Discussion, second paragraph:
45% of patients achieved a PR with MK-4830 plus pembrolizumab after previous disease progression on an anti–PD-1/PD-L1 alone or in combination with other agents
What the paper's own numbers show
Method. I tallied responders by treatment group from the Results text and Table 2. I then tried every numerator and denominator the paper defines for combination therapy: part C confirmed, part C including the unconfirmed PR, and all 52 patients who ever received the combination (part C plus crossover). I also checked Table 2's row sums and percentages, and computed exact (Clopper–Pearson) 95% CIs to show how large the discrepancy is. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Responders, all 84 dose-escalation patients | Results, Tumor response | 11 of 84 (10 confirmed, 1 unconfirmed) | 1 mono + 8 part C confirmed + 1 part C unconfirmed + 1 crossover = 11 of 84 (13.1%); 10 confirmed, 1 unconfirmed. Consistent |
| Responders, 34 combination (part C) patients | Abstract, Results | 11 of 34 (implies 32.4%) | 8 of 34 confirmed (23.5%); 9 of 34 with the unconfirmed PR (26.5%) |
| Responders on any combination therapy | Safety gives 52 combination-treated; Tumor response gives "the 10 patients receiving combination therapy (including those who crossed over)" | — | 8 + 1 + 1 = 10 of 52 (19.2%); 11 is not reachable |
| Combination ORR | Table 2; Results | 8 (24%) | 8/34 = 23.5%. Consistent |
| Combination responders by tumor type | Table 2, footnote b | 2 + 2 + 1 + 1 + 1 + 1 | 8. Consistent with Table 2 |
| Combination DCR | Results | 41% | (8 + 6)/34 = 41.2%. Consistent |
| Combination row of Table 2 | Table 2 | CR 1, PR 7, SD 9, PD 16, no assessment 1 | sums to 34. Consistent |
| Excess in the abstract | — | — | 11 − 8 = 3 patients; 32.4 − 23.5 = 8.8 percentage points (37.5% relative); 5.9 points above 9/34 |
| Exact 95% CI | — | not reported | 11/34: 17.4–50.5%; 8/34: 10.7–41.2%; 9/34: 12.9–44.4%; 10/52: 9.6–32.5% |
| Responders with prior anti-PD-1/PD-L1 progression | Abstract; Results | "5" (abstract, of combination responders); "5 of 11 (45%)" (Results, of all responders) | 5/11 = 45.5% |
The Results paragraph is self-consistent: its group counts add to exactly the 11 of 84 it states, and the separate statement that 10 combination-treated patients (counting crossovers) had at least 30% tumor shrinkage matches the 8 + 1 + 1 combination-treated responders. Table 2 and its footnote both give 8 confirmed combination responders. The abstract keeps the numerator of the whole-trial statement (11) and swaps the denominator for the part C count (34). The combination arm's own count is 8 confirmed, or 9 with the unconfirmed PR, so 11 of 34 overstates it.
Figure
Figure 1. Left: the abstract's statement and the response rate it implies (orange, 11/34 = 32.4%). Right: the Results statement for all 84 patients and the per-group counts given in the Results text and Table 2. They add to 11 responders across monotherapy, part C combination and crossover. The part C combination arm has 8 confirmed responders (23.5%), or 9 (26.5%) counting one unconfirmed PR (blue, recomputed). The connector marks the conflict: the abstract's numerator is the total for all 84 patients, and its denominator is the 34 patients in part C.
Figure 2. Objective response rate with exact (Clopper–Pearson) 95% confidence intervals on one linear axis starting at 0. Orange: the rate implied by the abstract (11/34). Blue: the rates the paper's own counts give for the combination arm (8/34 confirmed as in Table 2; 9/34 including the unconfirmed PR) and for every patient who received combination therapy, including crossovers (10/52). The intervals overlap because the arm is small. The figure shows the size of the overstatement. It is not a test.
Why it matters
- It is the abstract's only efficacy number. The abstract reports no ORR. A reader of the abstract alone, or of an index that shows only the abstract, would infer a combination-arm response rate of about 32% (11/34), where the paper's Table 2 gives 24% confirmed. Early-phase response rates in small arms are often cited when sizing later trials or comparing agents, so the overstatement can spread through citations.
- The "5 previously had progressive disease" clause inherits the error. In the Results, 5 of 11 is a share of all responders in every group. The abstract presents the 5 as a subset of combination-arm responders. The paper does not say whether all 5 were in part C, so the abstract's framing cannot be checked. The Discussion's "45% of patients achieved a PR with MK-4830 plus pembrolizumab after previous disease progression" has the same denominator problem: 45% is 5 of 11 responders across all groups, not a share of combination patients.
- What is not affected. The primary objectives (safety and tolerability), the absence of dose-limiting toxicities, the pharmacokinetic and target-engagement results, Table 2 and the Results text all stand. The abstract's conclusion that the combination showed "antitumor activity" also holds at 8 or 9 of 34. ORR was a tertiary objective. The error is confined to how the response count is presented in the abstract, and the matching phrasing in the Discussion.
Explanations considered
- The abstract counts every patient who ever received the combination. Does not account for the numbers. The paper's own wider denominator is 52 (34 in part C plus 18 crossovers), and the responders on combination therapy number 8 + 1 + 1 = 10, which the Results confirms ("the 10 patients receiving combination therapy (including those who crossed over)"). That gives 10 of 52, not 11 of 34. Adding crossover responders to the 34-patient denominator would still give only 10.
- The abstract includes unconfirmed or immune-related responses. Does not account for the numbers. The Results reports exactly one unconfirmed combination-arm PR, giving 9 of 34. Table 2 footnote e ("Eleven patients have been evaluated by immune Response Evaluation Criteria in Solid Tumors") is attached to the crossover column's 13 patients without a RECIST assessment, not to part C, and the paper reports no immune-related RECIST responses for any group.
- A different data cutoff or analysis set. Not supported. The paper states a single clinical cutoff (July 10, 2020), and the abstract's denominator of 34 is the same part C population as Table 2. Supplementary Table S2 lists 8 confirmed combination responders. The biomarker section reports 6 responders among 25 PD-L1-evaluable part C patients, consistent with at most 9.
- A text-conversion artifact. Ruled out. The numbers are spelled out ("Eleven of 34") in running text. The same wording is in the publisher-deposited PubMed abstract (PMID 34598945), the OpenAlex abstract and the live PMC page.
- Already corrected. No. The article has two published corrections, and neither changes the abstract (see Status). The April 2022 correction explicitly states that the combination-arm responder count "was correct in Table 2 (n = 8)".
- Side observations, not part of the main finding. (1) Table 2 prints the monotherapy "SD with PFS ≥6 months" cell as "5 (1)". 5/50 is 10%, so the percentage is a typographical slip. (2) "A total of 51% of patients with responses previously received ≥3 lines of therapy" cannot be a share of 11 responders (5/11 = 45.5%, 6/11 = 54.5%) or of 10 confirmed responders (50%, 60%). (3) All other percentages in Table 2 match their counts, and the Safety percentages for 52 patients (92%, 54%, 8%) recompute correctly.
Status
- Version of record. The publisher's page (aacrjournals.org, reached through doi.org) returned HTTP 403 behind a bot challenge, for the article and for its correction notices. I checked the PMC article page (which shows the article as corrected), the PMC-archived XML used for every quotation here, the publisher-deposited PubMed abstract (PMID 34598945) and the OpenAlex record. All carry the "Eleven of 34" sentence and the Results and Table 2 figures quoted above. The article appeared online in 2021 and in print as Clin Cancer Res 2022;28(1):57–70.
- Supplements. The Europe PMC supplementary bundle has five PDFs: Fig. S1 (disposition), Fig. S2 (pharmacokinetics and receptor occupancy), Fig. S3 (best overall response and duration by arm), Table S1 (baseline characteristics, n = 50/34/84) and Table S2 (characteristics of confirmed responders: monotherapy n = 1; MK-4830 + pembrolizumab n = 8). Table S2 supports 8 confirmed combination responders. Nothing in the supplement supports 11 in that arm.
- Corrections. Crossref lists two publisher corrections. (1) Clin Cancer Res 2022;28(8):1734, doi:10.1158/1078-0432.CCR-22-0564, 14 April 2022: "the number of patients who had ORR to MK-4830 + pembrolizumab was correct in Table 2 (n = 8), but footnote 'b' contained incorrect information regarding cancer types and the number of patients with each cancer type." (2) Clin Cancer Res 2022;28(18):4158, doi:10.1158/1078-0432.CCR-22-2320, 15 September 2022 (PMC9475239): "the Spearman correlations were correct in the text and incorrect in Fig. 4." Neither mentions the abstract. I read the text of both notices from their OpenAlex abstract records. PMC and Europe PMC list no other notices, and there is no retraction or expression of concern.
- PubPeer. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread, as of 2026-10-24.
- Sources. PMC9401547 (article and supplementary files); PMC9475239; PubMed 34598945; Crossref records for 10.1158/1078-0432.CCR-21-2160, 10.1158/1078-0432.CCR-22-0564 and 10.1158/1078-0432.CCR-22-2320; OpenAlex; PubPeer; https://doi.org/10.1158/1078-0432.CCR-21-2160. Status as of 2026-10-24.
Suggested correction
In the abstract, replace
Eleven of 34 patients in the dose-escalation phase who received combination therapy achieved objective responses; 5 previously had progressive disease on anti–PD-1/PD-L1 therapies.
with wording that matches the Results and Table 2, for example
Eight of 34 patients who received combination therapy in the dose-escalation phase achieved a confirmed objective response (ORR 24%) and 1 had an unconfirmed response; overall, 11 of 84 patients responded, of whom 5 previously had progressive disease on anti–PD-1/PD-L1 therapies.
If the 5 patients with prior anti-PD-1/PD-L1 progression were all in the combination arm, the authors may prefer to say so explicitly. In the Discussion, "45% of patients achieved a PR with MK-4830 plus pembrolizumab after previous disease progression" could read "5 of 11 responders (45%) had previous disease progression on an anti–PD-1/PD-L1". The authors may also wish to correct the monotherapy "SD with PFS ≥6 months" cell in Table 2 to "5 (10)" and check the denominator behind "51% of patients with responses".
Recomputation script (python recompute.py.txt)
"""Recompute the response counts of PMC9401547
(Clin Cancer Res 2022;28(1):57-70, online 2021, doi:10.1158/1078-0432.CCR-21-2160,
"First-in-Class Anti-immunoglobulin-like Transcript 4 Myeloid-Specific Antibody MK-4830
Abrogates a PD-1 Resistance Mechanism in Patients with Advanced Solid Tumors").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract (Results): "Of 84 patients, 50 received monotherapy and 34 received combination
therapy." ... "Eleven of 34 patients in the dose-escalation phase who received combination
therapy achieved objective responses; 5 previously had progressive disease on
anti-PD-1/PD-L1 therapies."
Results > Tumor response: "Eleven of 84 patients (10 confirmed and 1 unconfirmed)";
1 of 50 confirmed PR in monotherapy; 8 of 34 confirmed (CR 1, PR 7) in combination,
ORR 24%, DCR 41%, 6 additional SD >= 6 months; 1 additional unconfirmed PR in combination;
1 confirmed PR in the crossover group; "5 of 11 patients (45%)"; "51% of patients with
responses"; ">=30% ... in the patient receiving monotherapy and the 10 patients receiving
combination therapy (including those who crossed over)".
Results > Safety: "Of the 52 patients treated with combination therapy (n = 34 in part C;
n = 18 who crossed over from parts A and B)".
Table 2 (confirmed response, n (%)), columns monotherapy n=50, combination n=34, crossover n=18.
"""
import os
import numpy as np
from scipy.stats import beta
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
N_TOTAL, N_MONO, N_PARTC, N_CROSS = 84, 50, 34, 18 # crossover patients are a subset of the 50
N_ANY_COMBO = 52 # Safety section: 34 + 18
MONO_CONF, PARTC_CONF, PARTC_UNCONF, CROSS_CONF = 1, 8, 1, 1
ABSTRACT_K, ABSTRACT_N = 11, 34 # Abstract
RESULTS_K, RESULTS_N = 11, 84 # Results > Tumor response
RESULTS_CONF, RESULTS_UNCONF = 10, 1
# Table 2 as printed: label -> (mono (k, printed %), combination, crossover); None = not printed
TABLE2 = {
"ORR": ((1, 2), (8, 24), (1, 6)),
"CR": ((0, None), (1, 3), (0, None)),
"PR": ((1, 2), (7, 21), (1, 6)),
"SD": ((11, 22), (9, 26), (1, 6)),
"SD with PFS >=6 months": ((5, 1), (6, 18), (0, None)),
"PD": ((34, 68), (16, 47), (3, 17)),
"No RECIST assessment": ((4, 8), (1, 3), (13, 72)),
}
NS = (N_MONO, N_PARTC, N_CROSS)
def pct(k, n):
return 100.0 * k / n
def cp(k, n, a=0.05):
lo = 0.0 if k == 0 else beta.ppf(a / 2, k, n - k + 1)
hi = 1.0 if k == n else beta.ppf(1 - a / 2, k + 1, n - k)
return 100 * lo, 100 * hi
# ---------------------------------------------------------------- 1. where the 11 responders are
print("=== 1. Responders by group (Results text and Table 2) ===")
assert N_MONO + N_PARTC == N_TOTAL
total = MONO_CONF + PARTC_CONF + PARTC_UNCONF + CROSS_CONF
print(f"monotherapy confirmed {MONO_CONF} + part C confirmed {PARTC_CONF} + part C unconfirmed {PARTC_UNCONF}"
f" + crossover confirmed {CROSS_CONF} = {total}")
print(f" confirmed = {MONO_CONF + PARTC_CONF + CROSS_CONF} (Results: {RESULTS_CONF}); unconfirmed = {PARTC_UNCONF} (Results: {RESULTS_UNCONF})")
print(f" total {total} of {N_TOTAL} = {pct(total, N_TOTAL):.1f}% (Results: 'Eleven of 84')")
print(f" matches Results statement: {total == RESULTS_K}")
print()
print("=== 2. Candidate numerators and denominators for the combination arm ===")
partc_any = PARTC_CONF + PARTC_UNCONF
any_combo = PARTC_CONF + PARTC_UNCONF + CROSS_CONF
assert N_PARTC + N_CROSS == N_ANY_COMBO
cands = [
("part C, confirmed (Table 2)", PARTC_CONF, N_PARTC),
("part C, confirmed + unconfirmed", partc_any, N_PARTC),
("any combination exposure (part C + crossover), incl. unconfirmed", any_combo, N_ANY_COMBO),
("any combination exposure, confirmed only", PARTC_CONF + CROSS_CONF, N_ANY_COMBO),
("all 84 patients, all responders", total, N_TOTAL),
("as printed in Abstract", ABSTRACT_K, ABSTRACT_N),
]
CI = {}
for lab, k, n in cands:
lo, hi = cp(k, n)
CI[lab] = (k, n, pct(k, n), lo, hi)
print(f" {lab:66s} {k:2d}/{n:2d} = {pct(k, n):5.1f}% exact 95% CI {lo:4.1f}-{hi:4.1f}%")
print(f" responders while on combination (incl. crossover): {any_combo}"
f" [Results: '>=30% reductions ... the 10 patients receiving combination therapy (including those who crossed over)']")
print(f" responders not on monotherapy alone: {total - MONO_CONF}")
print(f" can any combination-arm count reach 11? {11 in (PARTC_CONF, partc_any, any_combo, PARTC_CONF + CROSS_CONF)}")
d_abs = pct(ABSTRACT_K, ABSTRACT_N) - pct(PARTC_CONF, N_PARTC)
print(f" Abstract 11/34 minus Table 2 8/34: {d_abs:.1f} percentage points; ratio {ABSTRACT_K / PARTC_CONF:.3f}"
f" (+{100 * (ABSTRACT_K / PARTC_CONF - 1):.1f}% relative)")
d_abs9 = pct(ABSTRACT_K, ABSTRACT_N) - pct(partc_any, N_PARTC)
print(f" Abstract 11/34 minus 9/34: {d_abs9:.1f} percentage points; ratio {ABSTRACT_K / partc_any:.3f}")
print()
print("=== 3. Table 2 internal checks ===")
for j, (name, n) in enumerate(zip(("monotherapy", "combination", "crossover"), NS)):
print(f" {name} (n = {n}):")
for lab, cells in TABLE2.items():
k, p = cells[j]
flag = "" if p is None or round(pct(k, n)) == p else " <-- printed % does not match"
print(f" {lab:24s} {k:3d}/{n} = {pct(k, n):5.1f}% printed {p}{flag}")
s = sum(TABLE2[x][j][0] for x in ("CR", "PR", "SD", "PD", "No RECIST assessment"))
print(f" CR+PR+SD+PD+no assessment = {s} (n = {n})")
orr, sd6 = TABLE2["ORR"][1][0], TABLE2["SD with PFS >=6 months"][1][0]
print(f" combination DCR = ORR {orr} + SD>=6mo {sd6} = {orr + sd6}/34 = {pct(orr + sd6, 34):.1f}% (Results: 41%)")
print(f" combination ORR {orr}/34 = {pct(orr, 34):.1f}% (Results: 24%)")
foot_b = dict(MSI_high_colorectal=2, gastric=2, HNSCC=1, Merkel=1, NSCLC=1, papillary_thyroid=1)
print(f" Table 2 footnote b (combination responders by tumor type, as archived in PMC): sum = {sum(foot_b.values())}")
print()
print("=== 4. Other percentages of responders (Results) ===")
print(f" 5 of 11 with prior anti-PD-1/PD-L1 progression: {pct(5, 11):.1f}% (printed 45%)")
print(" '51% of patients with responses' had >=3 prior lines; nearest k/11:",
", ".join(f"{k}/11 = {pct(k, 11):.1f}%" for k in (5, 6)),
"; nearest k/10:", ", ".join(f"{k}/10 = {pct(k, 10):.1f}%" for k in (5, 6)))
types = dict(HNSCC=2, gastric=2, colorectal=2, NSCLC=1, dendritic_sarcoma=1, papillary_thyroid=1, Merkel=1)
doses = {100: 1, 300: 1, 800: 4, 1600: 4}
print(f" 'Ten patients ... had continued PRs': tumor types sum to {sum(types.values())}, doses sum to {sum(doses.values())}")
print(f" Safety denominators: 48/52 = {pct(48, 52):.1f}%, 28/52 = {pct(28, 52):.1f}%, 4/52 = {pct(4, 52):.1f}%")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
PANEL = "#f3f2ee"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# ---- fig1: abstract vs Results/Table 2
fig = plt.figure(figsize=(7.2, 4.3))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 60)
ax.axis("off")
def panel(x0, w, title):
ax.add_patch(FancyBboxPatch((x0, 6), w, 52, boxstyle="round,pad=0,rounding_size=1.5",
fc=PANEL, ec=GRID, lw=1))
ax.text(x0 + 2, 55, title, fontsize=11, fontweight="bold", va="top", color=INK)
# left: abstract
L0, LW, R0, RW = 1, 31, 45, 54
panel(L0, LW, "Abstract, Results")
ax.text(L0 + 2, 48, "\u201cEleven of 34 patients in\nthe dose-escalation phase\nwho received combination\ntherapy achieved objective\nresponses\u201d",
fontsize=10, va="top", color=INK, linespacing=1.35)
pa = CI["as printed in Abstract"]
ax.text(L0 + 2, 23, "Implied combination ORR", fontsize=10, color=INK2, va="center")
ax.text(L0 + 2, 17.5, f"{ABSTRACT_K} / {ABSTRACT_N} = {pa[2]:.1f}%", fontsize=13, fontweight="bold", color=ORANGE, va="center")
ax.text(L0 + 2, 12.5, "as printed", fontsize=10, color=ORANGE, va="center")
# right: Results + Table 2
panel(R0, RW, "Results, Tumor response and Table 2")
ax.text(R0 + 2, 48, "\u201cEleven of 84 patients (10 confirmed and\n1 unconfirmed) ... in the dose-escalation\nphase of the trial achieved an objective\nresponse\u201d",
fontsize=10, va="top", color=INK, linespacing=1.35)
XR = R0 + RW - 2
rows = [
("Monotherapy (n = 50)", f"{MONO_CONF} confirmed", INK, "normal"),
("Combination, part C (n = 34)", f"{PARTC_CONF} + {PARTC_UNCONF} unconfirmed", BLUE, "bold"),
("Crossover (n = 18, of the 50)", f"{CROSS_CONF} confirmed", INK, "normal"),
]
y = 29.5
for lab, val, c, fw in rows:
ax.text(R0 + 2, y, lab, fontsize=10, color=INK2, va="center")
ax.text(XR, y, val, fontsize=10, color=c, va="center", ha="right", fontweight=fw)
y -= 4.2
ax.plot([R0 + 2, XR], [y + 2.1, y + 2.1], color=INK2, lw=0.8)
ax.text(R0 + 2, y - 0.2, "All groups (n = 84)", fontsize=10, color=INK2, va="center")
ax.text(XR, y - 0.2, f"{total} responders", fontsize=10, color=INK, va="center", ha="right", fontweight="bold")
p8, p9 = CI["part C, confirmed (Table 2)"], CI["part C, confirmed + unconfirmed"]
ax.text(R0 + 2, 12.5, f"Combination ORR {PARTC_CONF} / 34 = {p8[2]:.1f}%", fontsize=11, color=BLUE,
va="center", fontweight="bold")
ax.text(R0 + 2, 8.6, f"{partc_any} / 34 = {p9[2]:.1f}% counting the unconfirmed PR", fontsize=10, color=BLUE, va="center")
# connector
ax.annotate("", xy=(R0 - 0.8, 17.5), xytext=(L0 + LW + 0.8, 17.5),
arrowprops=dict(arrowstyle="<|-|>", color=INK2, lw=1.5, mutation_scale=12))
ax.text((L0 + LW + R0) / 2, 20, "the total for\nall 84 patients,\nput over the\n34 in part C", fontsize=10,
color=INK2, ha="center", va="bottom", linespacing=1.2)
handles = [Line2D([], [], ls="", marker="s", ms=9, color=ORANGE, label="value printed in the abstract"),
Line2D([], [], ls="", marker="s", ms=9, color=BLUE, label="recomputed from the Results text and Table 2")]
ax.legend(handles=handles, loc="lower center", bbox_to_anchor=(0.5, -0.01), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# ---- fig2: implied ORR vs recomputed ORR with exact CIs, one axis
fig, ax = plt.subplots(figsize=(7.2, 2.9))
items = [
("Abstract: 11 / 34", CI["as printed in Abstract"], ORANGE),
("Part C, confirmed + unconfirmed: 9 / 34", CI["part C, confirmed + unconfirmed"], BLUE),
("Part C, confirmed (Table 2): 8 / 34", CI["part C, confirmed (Table 2)"], BLUE),
("Any combination exposure: 10 / 52", CI["any combination exposure (part C + crossover), incl. unconfirmed"], BLUE),
]
ys = np.arange(len(items))[::-1]
for (lab, (k, n, p, lo, hi), c), yv in zip(items, ys):
ax.plot([lo, hi], [yv, yv], color=c, lw=2, solid_capstyle="butt")
ax.plot(p, yv, "o", ms=9, color=c)
ax.text(p, yv + 0.22, f"{p:.1f}%", ha="center", va="bottom", fontsize=10, color=c)
ax.set_yticks(ys)
ax.set_yticklabels([it[0] for it in items])
ax.set_xlim(0, 60)
ax.set_ylim(-0.6, len(items) - 0.25)
ax.set_xlabel("Objective response rate, % (exact 95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="o", color=ORANGE, lw=2, ms=8, label="as printed in the abstract"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=8, label="recomputed from the paper's own counts")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, -0.28), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print()
print("figures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Table 1 crystallographic statistics do not match the AcrIIA19 structure described in the text and deposited
Commun Biol (2025) · doi:10.1038/s42003-025-09417-6 · PMC12855784
Summary
The paper's central result is a crystal structure of the anti-CRISPR protein AcrIIA19. The Results text describes it as a C222₁ crystal with four molecules in the asymmetric unit (ASU), solved at 1.98 Å, and refers the reader to Table 1 for the crystallographic statistics. Table 1 instead reports a tetragonal P4₁2₁2 cell (a = b = 64.43, c = 62.82 Å), data to 1.45 Å and one molecule in the ASU. These cannot describe the same crystal: four molecules per ASU in Table 1's cell would give a Matthews coefficient of 0.56 ų/Da, far below the physical minimum. The deposited entry the paper cites (PDB 9WA8) agrees with the text, not with Table 1, so the published data-collection and refinement statistics (resolution, R factors, completeness, B factors, atom counts, geometry) do not appear to belong to the structure the paper analyses.
What the paper reports
Results, "High-resolution structural analysis of AcrIIA19 reveals a previously uncharacterized structural fold", paragraph 1:
Following successful crystallization, the structure of AcrIIA19 was solved at a 1.98 Å resolution using MR, with the AlphaFold2 model used as a search template. Details on crystallographic and refinement statistics are listed in Table 1.
Same section, paragraph 2:
The crystal belonged to space group C2221 and contained four molecules (MolA to MolD) in the asymmetric unit (ASU; Fig. 1 C). The final structural model covered the complete AcrIIA19 (residues M1 to G115).
B-factor analysis showed relatively higher values at the N-terminus; however, the overall B-factor was low (average 48.8 Å), indicating that the structure was highly rigid (Fig. 1G).
The observation that all four molecules in the same ASU exhibited nearly identical structures served as additional indirect evidence supporting the high structural stability of AcrIIA19 (Fig. 1H).
Table 1, "Data collection and refinement statistics" (selected rows, as archived):
Space group | P 41 21 2
a, b, c (Å) | a = 64.43, b = 64.43, c = 62.82
Resolution range (Å) | 28.81–1.45
Total reflections | 616,092
Unique reflections | 24,054
Wilson B-factor (Å2) | 20.90
Rwork (%)a | 20.91 (27.03)
Rfree (%)a | 21.89 (25.83)
No. of molecules in the asymmetric unit | 1
No. of non-hydrogen atoms | 821
Macromolecules | 733
Average B-factor values (Åb) | 23.55
(Both resolution-range rows, under "Data collection" and under "Refinement", read 28.81–1.45. On the publisher's page the space group is typeset P 4₁ 2₁ 2.)
Figure 1 legend, panels C and H:
C Four molecules of AcrIIA19 in the same crystallographic asymmetry unit (ASU).
H Four superimposed AcrIIA19 structures in ASU.
Results, "AcrIIA19 forms a stable dimer in solution", paragraph 2:
The PISA complex formation score, which ranges from 0 to 1 with higher values indicating greater biological relevance, was 1.0 for both the AB and CD dimers.
Since the AB dimer and CD dimer represent essentially the same dimeric form (Fig. 2C, D), we selected the AB dimer as a representative structure to analyze in detail.
Data availability (back matter):
The coordinate and structural factors have been deposited in the Research Collaboratory for Structural Bioinformatics (RCSB) Protein Data Bank (PDB) under the PDB code 9WA8.
What the paper's own numbers show
Method. I compared the three crystallographic descriptors the text gives (space group, resolution, ASU content) with Table 1. I then tested whether each description is physically self-consistent. For each cell I computed the volume, and from it the Matthews coefficient Vm = V / (n_sym × n_mol × M), with n_sym = 8 general positions (true of both P4₁2₁2 and C222₁) and M = 14,545 Da, the chain mass of the deposited construct (residues 1–115 plus the LEHHHHHH tag). I counted the non-hydrogen atoms of a complete residue 1–115 chain from its sequence, and I enumerated the symmetry-unique reflections that Table 1's cell, space group and resolution range allow. Finally I compared both descriptions with the deposited entry PDB 9WA8 (RCSB metadata, coordinates and wwPDB validation report). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Space group | Results ¶2 vs Table 1 | C222₁ (text); P4₁2₁2 (Table 1) | Orthorhombic vs tetragonal: different crystal systems, not alternative settings. PDB 9WA8: C 2 2 2₁ |
| High-resolution limit | Results ¶1 vs Table 1 | 1.98 Å (text); 28.81–1.45 Å (Table 1, both rows) | Differ by 0.53 Å (26.8% of 1.98). PDB 9WA8: 28.95–1.98 Å |
| Molecules in the ASU | Results ¶2, Fig. 1C/H vs Table 1 | four (MolA to MolD); 1 | PDB 9WA8: 4 chains (A–D) |
| Cell | Table 1 | a = b = 64.43, c = 62.82 Å | Tetragonal metric (a = b), consistent with P4₁2₁2. V = 260,780 ų. PDB 9WA8 cell 63.41 × 98.98 × 196.66 Å, V = 1,234,301 ų (4.73× larger) |
| Matthews coefficient, Table 1 cell, 1 molecule per ASU | derived from Table 1 | — | 2.24 ų/Da (45% solvent): normal |
| Matthews coefficient, Table 1 cell, 4 molecules per ASU | derived from Table 1 + text | — | 0.56 ų/Da: impossible (protein crystals typically lie at about 1.7–3.5) |
| Matthews coefficient, deposited cell, 4 molecules per ASU | PDB 9WA8 | — | 2.65 ų/Da (54% solvent); the entry itself records 2.65 and 53.62% |
| Protein non-H atoms | Table 1 vs sequence | 733 | One complete chain M1–G115 has 946 (deposited chains C and D: 945 each), so 733 is 0.77 of one complete chain; four chains need about 3784. PDB 9WA8: 3845 |
| Unique reflections | Table 1 | 24,054 at 99.98% complete | Table 1's cell and space group allow 24,053 to 28.81–1.45 Å: consistent. The deposited cell would allow 109,549 over that range |
| Multiplicity | Table 1 | 25.6 | 616,092 / 24,054 = 25.6: consistent. PDB 9WA8 redundancy 13.2 |
| R-work / R-free | Table 1 vs PDB 9WA8 | 20.91 / 21.89% | PDB 9WA8: 23.88 / 27.5%; 42,609 reflections in refinement vs Table 1's 24,052 |
| Average B-factor | Results ¶2 vs Table 1 | 48.8 (text); 23.55, Wilson 20.90 (Table 1) | PDB 9WA8: mean atomic B 64.4, Wilson B 50.9 (validation report). The text's value is not reproduced by either source; it is far from Table 1's |
Table 1 is internally consistent: its cell is tetragonal as its space group requires, the reflection count matches the cell and resolution almost exactly, the multiplicity matches its totals, and the atom count fits a single chain. It therefore reads as a complete statistics table for a single-chain, 1.45 Å, P4₁2₁2 crystal. The Results text, the Figure 1 and 2 legends, the dimer analysis and the deposited entry all describe a different crystal: C222₁, 1.98 Å, four chains. Table 1's model is also not a complete 1–115 chain, which the text says the final model is.
Figure
Figure 1. The four descriptors on which the Results text and Table 1 disagree, as printed (orange). Each connector names the conflict. The blue lines are recomputed from Table 1's own cell: with the eight symmetry copies of P4₁2₁2 and a 14,545 Da chain, one molecule per ASU leaves a normal 2.24 ų/Da, while the text's four molecules would leave 0.56 ų/Da, which is impossible. The gray strip gives the deposited entry PDB 9WA8 that the paper's Data availability statement names; it matches the text on space group, resolution and ASU content.
Figure 2. Matthews coefficient (unit-cell volume per dalton of protein) for each combination of cell and ASU content, on a scale starting at zero. The shaded band is the typical range for protein crystals (about 1.7–3.5 ų/Da). Table 1's cell is plausible only with the one molecule Table 1 lists. The four molecules described in the text and in Figure 1C/H fit only the deposited C222₁ cell (2.65 ų/Da, the value recorded in the PDB entry).
Why it matters
The crystal structure is the paper's headline deliverable ("Here, we present the crystal structure of AcrIIA19" in the Abstract). Table 1 is where a reader, referee or re-user checks its quality: resolution, completeness, R-work/R-free, geometry, Ramachandran statistics and B factors. As published, those numbers appear to describe a different crystal. The quality indicators a reader would rely on for the 1.98 Å, four-chain structure are therefore not in the paper. Its deposited R-free (27.5%) is noticeably higher than Table 1's 21.89%, and the deposited 1.98 Å resolution is lower than Table 1's 1.45 Å.
The four-molecule ASU is not incidental. The near-identity of the four copies is offered as evidence of rigidity (Fig. 1H), and the dimer analysis (PISA scores for the AB and CD dimers, the AB interface, Fig. 2B–H) depends on four chains in the ASU. Those analyses are consistent with the deposited entry, so they are not themselves refuted; what is missing is the correct statistical description of the data behind them.
The main biological conclusions (a new fold, a homodimer in solution by SEC-MALS, WED-domain binding and inhibition of several Cas9 orthologs) rest largely on the deposited structure and on independent biochemical data, and are not contradicted by this problem.
Two smaller inconsistencies sit in the same section and do not change any conclusion. The Figure 2C legend says "The model shows two dimers formed by MolA/MolD and MolC/MolD", which names MolD twice, while the text analyses AB and CD dimers. The B-factor is given in Å in the text and "Åb" in Table 1, where Ų is meant.
Explanations considered
- Two crystal forms, one described in the text and one in the table. This does not account for the paper as written. The text says that the statistics for the structure it describes "are listed in Table 1"; Table 1 has a single data column and no label for a second form; and only one entry (9WA8) is deposited. A second crystal form would have to be described and deposited separately.
- Resolution range vs a single high-resolution limit. This does not account for the numbers. Table 1's high-resolution limit is 1.45 Å in both its data-collection and refinement rows, not 1.98 Å. The deposited entry's limit is 1.98 Å.
- "Molecules in the ASU" counted per unique chain or per protomer. This does not account for the numbers. The row is the literal ASU content, and Table 1's cell is too small to hold four chains per ASU (Vm 0.56 ų/Da). Its 733 protein atoms are fewer than one complete chain.
- Alternative space-group setting or notation. Not possible. C222₁ is orthorhombic and P4₁2₁2 is tetragonal; no change of setting converts one into the other, and the two cells differ in volume by a factor of 4.73.
- A typing error in one or two Table 1 fields. This is unlikely. The cell, space group, reflection count, multiplicity, atom count and ASU content agree with one another for a single-chain tetragonal crystal, so many fields would have to be wrong together, consistently.
- Conversion or typesetting artifact in PMC. Excluded. The publisher's article page and its separate Table 1 page carry the same values as the PMC archive. Each value sits in its own labelled cell, with no minus signs or decimal shifts involved.
- The text, not the table, is wrong. This does not fit the deposited entry. The PDB entry cited in Data availability (primary citation: this DOI) records C 2 2 2₁, 1.98 Å, four chains and a Matthews coefficient of 2.65, which agrees with the text and the figure legends.
The paper does not say what crystal Table 1 describes. The average B-factor in the text (48.8) matches neither Table 1 (23.55) nor the deposited coordinates (mean 64.4; Wilson estimate 50.9). That leaves its source unclear.
Status
- Version of record. The publisher's article page (nature.com/articles/s42003-025-09417-6) and its Table 1 page give the same values as the PMC archive: "space group C 222 1", "four molecules (MolA to MolD)", "1.98 Å", "average 48.8 Å" in the text; "P 4 1 2 1 2", a = b = 64.43, c = 62.82, 28.81–1.45, 1 molecule in the ASU, 733 macromolecule atoms in Table 1. Every quote above was checked verbatim against the PMC-archived text. The Data availability sentence is in the article's back matter.
- Deposited data. PDB 9WA8 (deposited 2025-08-11, released 2026-07-08; primary citation this DOI): space group C 2 2 21, cell 63.41 × 98.98 × 196.66 Å, 4 polymer chains, 28.95–1.98 Å, R-work 0.2388, R-free 0.275, 42,893 reflections, redundancy 13.2, 3904 atoms (3845 protein, 59 water), Matthews coefficient 2.65. Checked through the RCSB REST API, the mmCIF header and coordinates, and the wwPDB validation report (Wilson B 50.871).
- Supplements. All publisher supplementary files were checked: the Transparent Peer Review File (the reviewers do not raise the crystallographic statistics), the Supplementary Information (uncropped gels), the description of additional files and Supplementary Data 1 (assay data only). The Reporting Summary was listed but not opened. None contains crystallographic statistics.
- Corrections and comments. As of 2026-10-23, no correction, erratum, retraction or expression of concern was found. Crossref lists no update relations, PMC and Europe PMC list no notices, and the publisher's page shows none. The PubPeer API and search page both refused automated access (HTTP 403, Cloudflare), so PubPeer could not be checked directly. A web search found no PubPeer thread or correction.
- Sources. PMC article PMC12855784; https://www.nature.com/articles/s42003-025-09417-6 and its /tables/1 page; Europe PMC full-text XML; https://www.rcsb.org/structure/9wa8; https://data.rcsb.org/rest/v1/core/entry/9WA8; https://files.rcsb.org/download/9WA8.cif; the wwPDB validation report for 9WA8; Crossref.
Suggested correction
- Replace Table 1 with the data-collection and refinement statistics for the deposited 9WA8 dataset: space group C222₁, cell 63.41 × 98.98 × 196.66 Å, 28.95–1.98 Å, four molecules per ASU, and the matching reflection counts, completeness, multiplicity, I/σ(I), R-merge, R-work/R-free, atom counts, B factors and geometry. The values should match those in the PDB entry and its validation report.
- If the present Table 1 describes another AcrIIA19 crystal (or another protein) from the same work, identify it, state its relation to the structure analysed, and deposit it, or remove it.
- Reconcile the "average 48.8" B-factor in the Results with the corrected table, and give its unit as Ų.
- Minor: correct the Figure 2C legend ("MolA/MolD and MolC/MolD") to the AB and CD dimers analysed in the text.
Recomputation script (python recompute.py.txt)
"""Recompute the crystallographic consistency checks for PMC12855784
(Commun Biol 2025, doi:10.1038/s42003-025-09417-6).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file, plus five candidate
figure designs in ./candidates/ (cand1..cand5 .svg/.png).
Inputs are copied from the paper as archived in PMC (Results text and Table 1)
and from the deposited entry the paper's Data availability statement names,
PDB 9WA8 (RCSB entry metadata, mmCIF header, coordinates and wwPDB validation
report, retrieved 2026-10-23). No network access is needed: the deposited values
are written out below.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
import matplotlib.ticker # noqa: E402,F401
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs: the paper
TEXT = { # Results, "High-resolution structural analysis of AcrIIA19 ...", paragraphs 1-2
"space_group": "C2221", "d_high": 1.98, "mol_asu": 4, "mean_B": 48.8,
"residues": (1, 115), # "The final structural model covered the complete AcrIIA19 (residues M1 to G115)"
}
TABLE1 = { # Table 1, "Data collection and refinement statistics"
"space_group": "P 41 21 2", "cell": (64.43, 64.43, 62.82), "angles": (90, 90, 90),
"d_low": 28.81, "d_high": 1.45, "total_refl": 616092, "unique_refl": 24054,
"multiplicity": 25.6, "completeness": 99.98, "wilson_B": 20.90,
"refl_refine": 24052, "Rwork": 20.91, "Rfree": 21.89, "mol_asu": 1,
"atoms_nonH": 821, "atoms_macro": 733, "atoms_ligand": 1, "atoms_solvent": 87,
"mean_B": 23.55, "mean_B_macro": 22.41,
}
# ---------------------------------------------------------------- inputs: deposited PDB 9WA8
PDB = {
"space_group": "C 2 2 21", "cell": (63.41, 98.98, 196.66), "Z_PDB": 32,
"d_low": 28.95, "d_high": 1.98, "unique_refl": 42893, "completeness": 97.92,
"redundancy": 13.2, "refl_refine": 42609, "Rwork": 23.88, "Rfree": 27.5,
"chains": 4, "atoms_total": 3904, "atoms_protein": 3845, "atoms_solvent": 59,
"matthews_deposited": 2.65, "solvent_pct_deposited": 53.62,
"chain_fw_Da": 14545.0, # entity 1 formula weight (residues 1-115 + LEHHHHHH tag)
"wilson_B_validation": 50.871,
"mean_B_coords": 64.36, # mean B_iso_or_equiv over all 3904 deposited atoms
"chain_atoms": {"A": 1002, "B": 953, "C": 945, "D": 945}, # protein atoms per chain
"chain_last_res": {"A": 121, "B": 116, "C": 115, "D": 115},
}
SEQ = ("MKLIINIEDKNYKYLTELAQQDNTNIGSIVNNLIQTHITDVNESYRSVDKKELDEFSRVMQHYFHEDLASMYDVIG"
"SDEELSTDKQMLKVYKKLYQDVALRNGIALELFNAYKKGLEHHHHHH") # PDB 9WA8 entity 1 sequence
# Number of general positions (symmetry operators incl. centring) per cell
NSYM = {"P41212": 8, "C2221": 8}
# Typical range of Matthews coefficients for protein crystals (about 1.7-3.5; Matthews 1968)
VM_RANGE = (1.7, 3.5)
def vol(c):
return c[0] * c[1] * c[2] # all angles are 90 degrees in both cells
def matthews(V, nsym, nmol, mw):
return V / (nsym * nmol * mw)
def solvent(vm):
return 100 * (1 - 1.230 / vm)
# ---------------------------------------------------------------- 1. direct comparison
print("=== 1. Results text vs Table 1 vs deposited PDB 9WA8 ===")
rows = [
("Space group", TEXT["space_group"], TABLE1["space_group"], PDB["space_group"]),
("High-resolution limit (A)", TEXT["d_high"], TABLE1["d_high"], PDB["d_high"]),
("Molecules in ASU", TEXT["mol_asu"], TABLE1["mol_asu"], PDB["chains"]),
("Average B (A^2)", TEXT["mean_B"], TABLE1["mean_B"], f"{PDB['mean_B_coords']} (coords), Wilson {PDB['wilson_B_validation']}"),
]
for r in rows:
print(f" {r[0]:28s} text={r[1]!s:8s} table1={r[2]!s:10s} PDB={r[3]}")
print(f" resolution difference text - table = {TEXT['d_high'] - TABLE1['d_high']:.2f} A "
f"({100 * (TEXT['d_high'] - TABLE1['d_high']) / TEXT['d_high']:.1f}% of 1.98)")
a, b, c = TABLE1["cell"]
print(f" Table 1 cell a=b ({a}={b}), c={c}: tetragonal metric, as P41212 requires; "
f"C2221 (orthorhombic, C-centred) needs no a=b, and PDB cell a/b = {PDB['cell'][0] / PDB['cell'][1]:.3f}")
# ---------------------------------------------------------------- 2. Matthews coefficients
print("\n=== 2. Cell volume and Matthews coefficient (chain MW = 14,545 Da, PDB entity 1) ===")
mw = PDB["chain_fw_Da"]
V1 = vol(TABLE1["cell"])
V2 = vol(PDB["cell"])
VM = {
"t1_1": matthews(V1, NSYM["P41212"], 1, mw),
"t1_4": matthews(V1, NSYM["P41212"], 4, mw),
"pdb_4": matthews(V2, NSYM["C2221"], 4, mw),
"pdb_1": matthews(V2, NSYM["C2221"], 1, mw),
}
print(f" Table 1 cell volume = {V1:,.0f} A^3")
print(f" PDB 9WA8 cell volume = {V2:,.0f} A^3 (ratio PDB/Table1 = {V2 / V1:.2f})")
print(f" Table 1 cell, P41212, 1 molecule/ASU (as Table 1) : Vm = {VM['t1_1']:.2f} A^3/Da, solvent {solvent(VM['t1_1']):.0f}%")
print(f" Table 1 cell, P41212, 4 molecules/ASU (as text) : Vm = {VM['t1_4']:.2f} A^3/Da, solvent {solvent(VM['t1_4']):.0f}%"
f" -> below the typical minimum {VM_RANGE[0]}")
print(f" PDB cell, C2221, 4 molecules/ASU (text, deposit): Vm = {VM['pdb_4']:.2f} A^3/Da, solvent {solvent(VM['pdb_4']):.0f}%"
f" (deposited value {PDB['matthews_deposited']}, {PDB['solvent_pct_deposited']}%)")
print(f" PDB cell, C2221, 1 molecule/ASU : Vm = {VM['pdb_1']:.2f} A^3/Da -> above the typical maximum {VM_RANGE[1]}")
print(f" molecules per cell: Table 1 = {NSYM['P41212'] * 1}, text = {NSYM['C2221'] * 4} (PDB Z = {PDB['Z_PDB']})")
# ---------------------------------------------------------------- 3. atom counts
print("\n=== 3. Non-hydrogen atom counts ===")
HEAVY = dict(G=4, A=5, S=6, C=6, V=7, T=7, P=7, I=8, L=8, D=8, N=8, M=8, E=9, Q=9, K=9,
H=10, F=11, R=11, Y=12, W=14)
seq115 = SEQ[:115]
assert seq115[0] == "M" and seq115[-1] == "G" # "residues M1 to G115"
full115 = sum(HEAVY[x] for x in seq115) + 1 # +1 for the C-terminal OXT
print(f" complete chain M1-G115 from sequence: {full115} non-H atoms (deposited chains C, D: "
f"{PDB['chain_atoms']['C']}, {PDB['chain_atoms']['D']})")
print(f" Table 1 macromolecule atoms = {TABLE1['atoms_macro']} = {TABLE1['atoms_macro'] / full115:.2f} x one complete chain; "
f"four complete chains = {4 * full115}")
print(f" PDB 9WA8 protein atoms = {PDB['atoms_protein']} = {PDB['atoms_protein'] / TABLE1['atoms_macro']:.2f} x Table 1")
print(f" Table 1 sum check: {TABLE1['atoms_macro']} + {TABLE1['atoms_ligand']} + {TABLE1['atoms_solvent']} = "
f"{TABLE1['atoms_macro'] + TABLE1['atoms_ligand'] + TABLE1['atoms_solvent']} (reported {TABLE1['atoms_nonH']})")
# ---------------------------------------------------------------- 4. reflection counts
print("\n=== 4. Unique reflections expected from each cell and resolution range ===")
def n_unique(cell, dlo, dhi, sg):
a, b, c = cell
H = int(a / dhi) + 1
K = int(b / dhi) + 1
L = int(c / dhi) + 1
h, k, l = np.meshgrid(np.arange(0, H + 1), np.arange(0, K + 1), np.arange(0, L + 1), indexing="ij")
s2 = (h / a) ** 2 + (k / b) ** 2 + (l / c) ** 2 # 1/d^2, orthogonal cell
ok = (s2 <= 1 / dhi ** 2) & (s2 >= 1 / dlo ** 2) & ~((h == 0) & (k == 0) & (l == 0))
if sg == "P41212": # Laue 4/mmm asymmetric unit h>=k>=0, l>=0; absences 00l l=4n, h00 h=2n
ok &= h >= k
ok &= ~((h == 0) & (k == 0) & (l % 4 != 0))
ok &= ~((k == 0) & (l == 0) & (h % 2 != 0))
elif sg == "C2221": # Laue mmm asymmetric unit h,k,l>=0; C-centring h+k=2n; 00l l=2n
ok &= (h + k) % 2 == 0
ok &= ~((h == 0) & (k == 0) & (l % 2 != 0))
return int(ok.sum())
N1 = n_unique(TABLE1["cell"], TABLE1["d_low"], TABLE1["d_high"], "P41212")
N2 = n_unique(PDB["cell"], PDB["d_low"], PDB["d_high"], "C2221")
N1x = n_unique(PDB["cell"], TABLE1["d_low"], TABLE1["d_high"], "C2221")
print(f" Table 1 cell, P41212, 28.81-1.45 A: {N1:,} possible; reported {TABLE1['unique_refl']:,} "
f"at {TABLE1['completeness']}% -> implies {TABLE1['unique_refl'] / (TABLE1['completeness'] / 100):,.0f}")
print(f" PDB cell, C2221, 28.95-1.98 A: {N2:,} possible; deposited {PDB['unique_refl']:,} "
f"at {PDB['completeness']}% -> implies {PDB['unique_refl'] / (PDB['completeness'] / 100):,.0f}")
print(f" PDB cell, C2221, if it had been 28.81-1.45 A: {N1x:,} possible (Table 1 reports {TABLE1['unique_refl']:,})")
print(f" multiplicity check Table 1: {TABLE1['total_refl']:,} / {TABLE1['unique_refl']:,} = "
f"{TABLE1['total_refl'] / TABLE1['unique_refl']:.1f} (reported {TABLE1['multiplicity']})")
# ---------------------------------------------------------------- 5. ratios to the deposit
print("\n=== 5. Table 1 and text values relative to the deposited entry ===")
RATIOS = [ # label, text value, Table 1 value, PDB value
("High-resolution limit", TEXT["d_high"], TABLE1["d_high"], PDB["d_high"]),
("Molecules in ASU", TEXT["mol_asu"], TABLE1["mol_asu"], PDB["chains"]),
("Unit-cell volume", None, V1, V2),
("Unique reflections", None, TABLE1["unique_refl"], PDB["unique_refl"]),
("Protein non-H atoms", None, TABLE1["atoms_macro"], PDB["atoms_protein"]),
("Multiplicity", None, TABLE1["multiplicity"], PDB["redundancy"]),
("R-free (%)", None, TABLE1["Rfree"], PDB["Rfree"]),
]
for lab, t, tb, p in RATIOS:
ts = f"text/PDB={t / p:.2f}" if t is not None else "text: not stated"
print(f" {lab:22s} Table1/PDB = {tb / p:.2f} {ts}")
print(f" Rwork Table 1 {TABLE1['Rwork']}% vs PDB {PDB['Rwork']}%; reflections in refinement {TABLE1['refl_refine']:,} vs {PDB['refl_refine']:,}")
print(f" Average B: text {TEXT['mean_B']}; Table 1 {TABLE1['mean_B']} (Wilson {TABLE1['wilson_B']}); "
f"PDB coordinates mean {PDB['mean_B_coords']}, validation Wilson {PDB['wilson_B_validation']}")
# ================================================================= figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 1.5,
})
def save(fig, path_stem):
fig.savefig(path_stem + ".svg")
fig.savefig(path_stem + ".png", dpi=200)
plt.close(fig)
SG_T = "C222₁" # C2221
SG_TB = "P4₁2₁2" # P41212
# ---------- design 1: two text panels joined by annotated connectors (+ deposit strip)
def design_contradiction(stem):
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 78)
ax.axis("off")
PT, PB = 67, 25 # panel top / bottom
for x0, title, sub in [(2, "Results text", "High-resolution structural\nanalysis, paragraphs 1\u20132"),
(62, "Table 1", "Data collection and\nrefinement statistics")]:
ax.add_patch(FancyBboxPatch((x0, PB), 36, PT - PB, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=SEC, lw=1.0))
ax.text(x0 + 1.5, PT - 3, title, fontsize=12, fontweight="bold", va="center")
ax.text(x0 + 1.5, PT - 8.3, sub, fontsize=10, color=SEC, va="center", linespacing=1.15)
left = [("space group", SG_T), ("solved at", "1.98 \u00c5 resolution"),
("asymmetric unit", "four molecules (MolA to MolD)"), ("overall B-factor", "average 48.8")]
right = [("Space group", SG_TB), ("Resolution range", "28.81\u20131.45 \u00c5"),
("Molecules in the ASU", "1"), ("Average B-factor", "23.55")]
conflict = ["orthorhombic vs\ntetragonal", "1.98 vs 1.45 \u00c5", "4 vs 1", "48.8 vs 23.55"]
ys = [51.5, 43.5, 35.5, 28.5]
for y, (l1, v1), (l2, v2), cf in zip(ys, left, right, conflict):
ax.text(3.5, y + 1.6, l1, fontsize=10, color=SEC, va="center")
ax.text(3.5, y - 1.3, v1, fontsize=10.5, color=ORANGE, fontweight="bold", va="center")
ax.text(63.5, y + 1.6, l2, fontsize=10, color=SEC, va="center")
ax.text(63.5, y - 1.3, v2, fontsize=10.5, color=ORANGE, fontweight="bold", va="center")
ax.annotate("", xy=(61.0, y), xytext=(39.0, y),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.5, shrinkA=0, shrinkB=0))
ax.text(50, y + 0.9, cf, ha="center", va="bottom", fontsize=10, color=TXT, linespacing=1.05)
ax.text(50, PT - 3, "conflict", ha="center", va="center", fontsize=11, fontweight="bold", color=TXT)
# recomputed strip
ax.text(2.5, 20.5, "Recomputed from Table 1's own cell (8 symmetry copies per cell, 14,545 Da chain):",
fontsize=10, color=BLUE, va="center")
ax.text(2.5, 17.2, f"1 molecule per ASU (as Table 1 states): Vm = {VM['t1_1']:.2f} \u00c5\u00b3/Da, a normal value",
fontsize=10, color=BLUE, va="center", fontweight="bold")
ax.text(2.5, 13.9, f"4 molecules per ASU (as the text states): Vm = {VM['t1_4']:.2f} \u00c5\u00b3/Da, physically impossible",
fontsize=10, color=BLUE, va="center", fontweight="bold")
# deposit strip
ax.add_patch(FancyBboxPatch((2, 1.6), 96, 7, boxstyle="round,pad=0.3,rounding_size=1.0",
fc="#f1f0ec", ec="none"))
ax.text(3.5, 6.6, "Deposited entry PDB 9WA8 (named in Data availability) matches the text:",
fontsize=10, color=SEC, va="center")
ax.text(3.5, 3.5, f"{SG_T} \u00b7 1.98 \u00c5 \u00b7 4 chains \u00b7 cell 63.41 \u00d7 98.98 \u00d7 196.66 \u00c5"
f" \u00b7 Vm {VM['pdb_4']:.2f} \u00c5\u00b3/Da", fontsize=10, color=SEC, va="center")
# legend
for x, col, lab in [(2.5, ORANGE, "printed in the paper"), (36, BLUE, "recomputed from Table 1"),
(71, SEC, "deposited entry")]:
ax.add_patch(Rectangle((x, 73.2), 2.2, 2.6, fc=col, ec="none"))
ax.text(x + 3.2, 74.5, lab, fontsize=10, color=TXT, va="center")
save(fig, stem)
# ---------- design 2: scorecard matrix
def design_matrix(stem):
rows = [
("Space group", SG_T, SG_TB, SG_T),
("Crystal system", "orthorhombic", "tetragonal", "orthorhombic"),
("Cell a, b, c (Å)", "not stated", "64.43, 64.43, 62.82", "63.41, 98.98, 196.66"),
("Resolution limit (\u00c5)", "1.98", "1.45", "1.98"),
("Molecules in ASU", "4 (MolA–MolD)", "1", "4 chains"),
("Average B-factor", "48.8", "23.55", "64.4 (Wilson 50.9)"),
("Protein non-H atoms", "not stated", "733", "3845"),
("Unique reflections", "not stated", "24,054", "42,893"),
("R-work / R-free (%)", "not stated", "20.91 / 21.89", "23.9 / 27.5"),
]
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(-0.4, len(rows) + 2.0)
ax.axis("off")
xs = [2, 27, 44, 66.5]
heads = ["", "Results text", "Table 1", "PDB 9WA8"]
top = len(rows) + 1.2
for x, h in zip(xs, heads):
ax.text(x, top, h, fontsize=11, fontweight="bold", va="center")
ax.plot([2, 98], [top - 0.5, top - 0.5], color=SEC, lw=1)
for i, (lab, t, tb, p) in enumerate(rows):
y = len(rows) - i
if i % 2 == 0:
ax.add_patch(Rectangle((1, y - 0.48), 98, 0.96, fc="#f1f0ec", ec="none"))
ax.text(xs[0], y, lab, fontsize=10, va="center", color=TXT)
for x, v, col in [(xs[1], t, ORANGE), (xs[2], tb, ORANGE), (xs[3], p, SEC)]:
ns = v == "not stated"
ax.text(x, y, v, fontsize=10, va="center", color=SEC if ns else col,
fontweight="normal" if (ns or col == SEC) else "bold", style="italic" if ns else "normal")
diff_tb = tb.split(" ")[0] != p.split(" ")[0]
mark = "✕ differs" if diff_tb else "= same"
ax.text(98.5, y, mark, fontsize=10, va="center", ha="right", color=TXT if diff_tb else SEC)
ax.text(98.5, top, "Table 1 vs\ndeposit", fontsize=10, ha="right", va="center", color=SEC, linespacing=1.05)
ax.text(2, 0.1, "Orange: printed in the paper. Gray: deposited entry PDB 9WA8 (named in Data availability).",
fontsize=10, color=SEC, va="center")
save(fig, stem)
# ---------- design 3: unit cells to scale
def design_cells(stem):
fig, axs = plt.subplots(1, 2, figsize=(7.2, 5.0), gridspec_kw=dict(width_ratios=[1, 1], wspace=0.3))
a1, b1, c1 = TABLE1["cell"]
a2, b2, c2 = PDB["cell"]
for ax, (w1, h1, w2, h2, xl, yl, ttl) in zip(axs, [
(a1, b1, a2, b2, "a (Å)", "b (Å)", "a × b face"),
(a1, c1, a2, c2, "a (Å)", "c (Å)", "a × c face")]):
ax.add_patch(Rectangle((0, 0), w2, h2, fc="none", ec=SEC, lw=2))
ax.add_patch(Rectangle((0, 0), w1, h1, fc="none", ec=ORANGE, lw=2, ls=(0, (4, 2))))
ax.set_xlim(0, 105)
ax.set_ylim(0, 205 if "c face" in ttl else 105)
ax.set_aspect("equal")
ax.set_xlabel(xl)
ax.set_ylabel(yl)
ax.set_title(ttl, fontsize=11, loc="left")
ax.grid(color=GRID, lw=0.6)
ax.set_axisbelow(True)
ax.tick_params(labelsize=10)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
axs[0].text(4, 32, "Table 1\n64.43 \u00d7 64.43", color=TXT, fontsize=10, va="center")
axs[0].text(4, 82, "PDB 9WA8\n63.41 \u00d7 98.98", color=SEC, fontsize=10, va="center")
axs[1].text(4, 31, "Table 1\n64.43 \u00d7\n62.82", color=TXT, fontsize=10, va="center")
axs[1].text(4, 130, "PDB 9WA8\n63.41 \u00d7\n196.66", color=SEC, fontsize=10, va="center")
fig.legend(handles=[Line2D([], [], color=ORANGE, lw=2, ls=(0, (4, 2)), label=f"Table 1 cell ({SG_TB}), V = {V1/1e3:,.0f} × 10³ ų"),
Line2D([], [], color=SEC, lw=2, label=f"Deposited 9WA8 cell ({SG_T}), V = {V2/1e3:,.0f} × 10³ ų")],
loc="upper left", bbox_to_anchor=(0.06, 0.995), frameon=False, fontsize=10, ncol=1)
fig.text(0.06, 0.065, f"Both space groups have 8 symmetry copies per cell; the deposited cell is {V2 / V1:.2f}\u00d7 larger.\n"
f"Recomputed from Table 1's cell: Vm = {VM['t1_1']:.2f} \u00c5\u00b3/Da for 1 molecule per ASU (normal),\n"
f"{VM['t1_4']:.2f} \u00c5\u00b3/Da for 4 molecules per ASU (physically impossible).",
fontsize=10, color=BLUE, va="center", linespacing=1.2)
fig.subplots_adjust(left=0.1, right=0.98, top=0.82, bottom=0.24)
save(fig, stem)
# ---------- design 4: Matthews coefficient dot plot
def design_matthews(stem):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
items = [
("Table 1 cell, P4\u20812\u20812, 1 molecule per ASU (as Table 1 states)", VM["t1_1"], BLUE, "o"),
("Table 1 cell, P4\u20812\u20812, 4 molecules per ASU (as the text states)", VM["t1_4"], BLUE, "o"),
("Deposited 9WA8 cell, C222\u2081, 4 molecules per ASU (text and deposit)", VM["pdb_4"], SEC, "D"),
]
ax.axvspan(*VM_RANGE, color="#eceae4", zorder=0)
ax.text(VM_RANGE[1] - 0.05, 3.05, f"typical range for protein\ncrystals, about {VM_RANGE[0]}\u2013{VM_RANGE[1]}",
ha="right", va="top", fontsize=10, color=SEC, linespacing=1.1)
for i, (lab, v, col, m) in enumerate(items):
y = 2 - i
ax.text(0.02, y + 0.3, lab, fontsize=10, color=TXT, va="center", ha="left")
ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(v, y, marker=m, ms=10, color=col, mec=BG, mew=1.5, ls="none", zorder=3)
ax.text(v + 0.09, y, f"{v:.2f}", va="center", fontsize=10.5, color=TXT, fontweight="bold")
ax.set_yticks([])
ax.set_xlim(0, 4.6)
ax.set_ylim(-0.4, 3.1)
ax.set_xlabel("Matthews coefficient Vm (\u00c5\u00b3/Da): cell volume \u00f7 (8 \u00d7 molecules per ASU \u00d7 14,545 Da)")
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
fig.legend(handles=[Line2D([], [], marker="o", color=BLUE, ls="none", ms=9, label="recomputed from Table 1's cell"),
Line2D([], [], marker="D", color=SEC, ls="none", ms=9, label="computed from the deposited cell")],
loc="lower center", bbox_to_anchor=(0.5, 0.0), frameon=False, fontsize=10, ncol=2)
fig.tight_layout(rect=(0, 0.07, 1, 1))
save(fig, stem)
# ---------- design 5: ratio-to-deposit dot plot (log scale)
def design_ratios(stem):
fig, ax = plt.subplots(figsize=(7.2, 4.2))
n = len(RATIOS)
for i, (lab, t, tb, p) in enumerate(RATIOS):
y = n - 1 - i
ax.plot(tb / p, y, "o", ms=10, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(tb / p / 1.07, y, f"{tb / p:.2f}", fontsize=10, ha="right", va="center", color=TXT)
if t is not None:
ax.plot(t / p, y, "o", ms=10, mfc=BG, mec=ORANGE, mew=2, zorder=4)
ax.axvline(1, color=SEC, lw=1)
ax.text(1.03, -0.45, "same as deposit", fontsize=10, color=SEC, va="center")
ax.set_xscale("log")
ax.set_xlim(0.1, 2.5)
ax.set_xticks([0.1, 0.2, 0.5, 1, 2])
ax.set_xticklabels(["0.1", "0.2", "0.5", "1", "2"])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_yticks(range(n))
ax.set_yticklabels([r[0] for r in RATIOS][::-1], fontsize=10)
ax.set_ylim(-0.6, n - 0.1)
ax.set_xlabel("Value in the paper ÷ value in deposited PDB 9WA8 (log scale)")
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, ls="none", ms=9, label="Table 1 (reported)"),
Line2D([], [], marker="o", mfc=BG, mec=ORANGE, mew=2, ls="none", ms=9,
label="Results text (reported)")],
loc="lower left", bbox_to_anchor=(0.0, 1.0), frameon=False, fontsize=10, ncol=2)
fig.tight_layout()
save(fig, stem)
design_contradiction(os.path.join(HERE, "fig1"))
design_matthews(os.path.join(HERE, "fig2"))
design_contradiction(os.path.join(CAND, "cand1_contradiction_panels"))
design_matrix(os.path.join(CAND, "cand2_scorecard"))
design_cells(os.path.join(CAND, "cand3_unit_cells_to_scale"))
design_matthews(os.path.join(CAND, "cand4_matthews"))
design_ratios(os.path.join(CAND, "cand5_ratio_to_deposit"))
print("\nwrote fig1, fig2 and candidates/cand1..cand5 (.svg, .png)")
Abstract reports lighter liver copper isotopes in infected mice; the paper's own data show heavier
Commun Biol (2026) · doi:10.1038/s42003-025-09464-z · PMC12881638
Summary
The abstract says that Plasmodium berghei (Pb) infection gave a "significantly lighter Cu isotopic composition in the liver". The Results, the Discussion and the supplementary data all report the opposite: infected livers have higher δ65Cu than uninfected livers, and by the paper's own Eq. 3 a higher δ65Cu means isotopically heavier Cu. The per-mouse values in the Supplementary Data file give mean wet-liver δ65Cu of +0.09‰ in uninfected mice, +1.03‰ at 7 days post infection (DPI) and +0.65‰ at 10 DPI. Every one of the 16 untreated infected mice (7 and 10 DPI) lies above every control (Mann–Whitney p = 0.0007 at each time point). The abstract is where most readers meet this result, and the wrong sign there also makes the abstract's own mechanism (export of isotopically light Cu from the liver through ceruloplasmin) read as self-contradictory. The measurements, the statistics and the conclusions in the body of the paper are not affected.
What the paper reports
Abstract, fourth sentence:
A discrete hepatic Cu deficiency, but significantly lighter Cu isotopic composition in the liver were observed, alongside a release of Cu into the bloodstream through ceruloplasmin, indicating a contribution of Cu in potentiating Fe release from hepatic stores.
Results, section "Mice respond to infection with changes in the metal isotopic composition…", fourth paragraph (Fig. 3F, G):
As for Fe, also the isotopic compositions of Zn and Cu differ significantly between Pb-infected and uninfected mice (Fig. 3F, G). In wet livers, Pb infection led to lower δ66Zn values in comparison to uninfected mice, while it led to higher δ65Cu values compared to controls. Changes in the isotopic composition of Zn and Cu were also observed in perfused livers, albeit without statistical significance for Zn.
Results, same section, fifth paragraph:
Different to the observation for Fe, the largest absolute shifts in the isotope ratios of Zn and Cu were found at 7 DPI (onset of infection) rather than at 10 DPI (later stage).
Results, section "Host metal isotopic compositions are partly restored by antimalarial treatment", second and fourth paragraphs:
Similarly to Fe, the Zn and Cu isotope ratios shifted towards the values of uninfected mice as a result of antimalarial treatment (Fig. 4C, Table S3, Fig. S6).
Systematically, we found a negative correlation between δ56Fe values in the liver and the plasmatic concentration of hepcidin and a positive correlation between δ65Cu values in the liver and the plasmatic concentration of CP (Fig. 4D and E).
Discussion, eighth paragraph:
In parallel, a hepatic Cu deficiency was observed. Consistent to this deficiency, an increase in the heavy Cu isotope was noticed in Pb infection, where the highest increase has coincided with the onset of infection and anemia. We interpreted that when hepatocytes are gradually delivering Cu that is enriched in the light Cu isotope via CP into the plasma, then an enrichment in the heavier Cu isotope in the liver can occur. Antimalarial treatment has decreased the plasmatic levels of CP and the hepatic δ65Cu values, which is consistent with a profile of restoration of the Cu homeostasis.
Discussion, tenth paragraph:
In support to this, we have seen a hepatic Cu deficiency and a depletion in the lighter isotope, which is encompassed by an increased plasmatic Cu level.
Materials and methods, "Instrumentation and measurements…" (Eq. 3 is printed as δ65Cu = [(65Cu/63Cu)sample / (65Cu/63Cu)NIST SRM 976 − 1] × 1000):
δ value expresses the relative difference in the ratio of a heavy to a light isotope between a sample and an isotopic reference material. Therefore, a positive δ value indicates that isotopic fractionation has led to an enrichment in the heavier isotope in the sample compared to the reference material, while a negative δ value indicates a depletion in the heavier isotope.
Supplementary Information, Table S4 ("Metal isotope ratios in wet liver samples…", δ65Cu column, mean ± SD, ‰):
Untreated, 0 DPI 0.08 ± 0.12 · Untreated, 7 DPI 1.02 ± 0.29 · AQ 24 h, 10 DPI 0.37 ± 0.11 · CQ 24 h, 10 DPI 0.37 ± 0.11 · CQ 48 h, 10 DPI 0.31 ± 0.07 · Untreated, 10 DPI 0.65 ± 0.19
All quotations from the article were checked verbatim against the PMC-archived text with src/show.py; the Table S4 excerpt is from the publisher's Supplementary Information PDF (MOESM2), re-laid on one line.
What the paper's own numbers show
Method. First, the sign: from Eq. 3, the difference between two δ65Cu values is 1000 × (Rinfected − Rcontrol) / RNIST 976, where R is the 65Cu/63Cu ratio. Because the reference ratio is positive, a higher δ65Cu always means more 65Cu relative to 63Cu, that is, isotopically heavier Cu, whether the δ values themselves are positive or negative. Second, the size: I took the per-mouse liver δ65Cu values from the paper's Supplementary Data file (MOESM4, sheet "Liver"), recomputed each group's mean and SD, checked them against SI Tables S4 and S5, and compared infected with uninfected livers with the test the paper uses (two-sided Mann–Whitney, exact). The sheet labels the untreated infected groups "LP" and "HP" without a time point; their means and SDs match Table S4's 7 DPI and 10 DPI rows respectively, and that is how they are assigned here. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Direction of the infection effect on liver Cu isotopes | Abstract | "significantly lighter Cu isotopic composition in the liver" | heavier: every body-text statement and every data source below gives higher δ65Cu in infected liver |
| Direction, same comparison | Results (Fig. 3G); Discussion paragraphs 8 and 10 | "higher δ65Cu values compared to controls"; "an increase in the heavy Cu isotope"; "a depletion in the lighter isotope" | consistent with each other and with Eq. 3: heavier |
| Wet liver δ65Cu, uninfected (0 DPI) | Table S4; MOESM4 | 0.08 ± 0.12‰ | +0.086 ± 0.124‰ (n = 6) |
| Wet liver δ65Cu, infected 7 DPI | Table S4; MOESM4 | 1.02 ± 0.29‰ | +1.025 ± 0.297‰ (n = 8); shift +0.94‰; 8 of 8 mice above every control; U = 48, p = 0.00067 |
| Wet liver δ65Cu, infected 10 DPI | Table S4; MOESM4 | 0.65 ± 0.19‰ | +0.650 ± 0.194‰ (n = 8); shift +0.56‰; 8 of 8 mice above every control; U = 48, p = 0.00067 |
| Perfused liver δ65Cu, 0 vs 10 DPI | Table S5; MOESM4 | 0.29 ± 0.25 vs 0.59 ± 0.17‰, p = 0.0306 | +0.309 ± 0.287 (n = 5; does not reproduce the printed 0.29 ± 0.25) vs +0.660 ± 0.263‰ (n = 9), shift +0.35‰, p = 0.042; without one infected row (see below) +0.591 ± 0.174‰ (n = 8), shift +0.28‰ |
| Treated (10 DPI) vs untreated 10 DPI | Results; Discussion p8; Table S4 | treatment "decreased … the hepatic δ65Cu values", toward uninfected | CQ 24 h +0.377, CQ 48 h +0.316, AQ 24 h +0.376‰ vs untreated +0.650‰ (shifts −0.27 to −0.33‰; p = 0.0070, 0.00062, 0.0059); all still above uninfected +0.086‰ |
| Direction of each element's shift in wet liver, 0 → 7 / 10 DPI | Table S4 means | Abstract groups liver Fe, Cu and Zn as lighter | δ56Fe −0.42 / −0.94‰ (lighter); δ66Zn −0.29 / −0.20‰ (lighter); δ65Cu +0.94 / +0.57‰ (heavier) |
| Size of the 7 DPI shift as a ratio | Eq. 3 | — | 65Cu/63Cu higher by a factor of 1.00094 in infected liver |
The per-mouse δ65Cu values reproduce every Table S4 δ65Cu mean and SD to within 0.01‰ (largest difference 0.008‰). The printed values look truncated rather than rounded: 11 of the 12 equal the recomputed value cut to two decimals (for example uninfected 0.086 → 0.08, CQ 48 h SD 0.078 → 0.07), and the twelfth, the 10 DPI mean of 0.6497‰, is printed as 0.65. So for δ65Cu, Table S4, the Supplementary Data and the Results sentence describe the same measurements. This check covers δ65Cu only; for the other elements the sheet and Table S4 do not all agree (for example, CQ 24 h δ56Fe is −2.04 ± 0.22‰ in the sheet against −1.96 ± 0.10‰ in Table S4), which does not bear on the Cu direction. On those measurements the infected livers are heavier than the controls without overlap at both time points. There is no subset, time point or tissue preparation in the paper's liver data where infection makes δ65Cu lower. The only significantly "lighter Cu" results in the paper are in purified parasitized red blood cells (pRBCs; Δ65Cu = −0.88‰ against whole blood) and, as the Discussion puts it, in "blood and purified RBCs", not in the liver.
Figure
Figure 1. The conflicting statements side by side. Left (orange): the abstract sentence, which states that infected livers have isotopically lighter Cu, that is, lower δ65Cu than controls. Right: the Results and Discussion sentences on the same comparison, which state higher δ65Cu and an increase in the heavy isotope, and (blue) the wet-liver group means recomputed from the per-mouse values in the paper's Supplementary Data file. Under Eq. 3 of the paper, a higher δ65Cu means a heavier composition, so the two panels give opposite signs for one result.
Figure 2. Liver δ65Cu (wet liver) for each mouse in the paper's Supplementary Data (open circles), with the group mean ± SD recomputed from those values (blue circles) and the mean ± SD printed in SI Table S4 (orange diamonds). The dashed line is the uninfected mean (+0.086‰). Up on this axis is isotopically heavier. Infected mice at 7 and 10 DPI all lie above all controls (exact two-sided Mann–Whitney p = 0.0007 for each). The three antimalarial-treated groups (10 DPI) lie between the untreated infected and the uninfected groups. The orange arrow marks the direction the abstract's "lighter" describes; no group of livers in the paper moves that way. The axis starts below zero only to show that direction; it does not change the visible size of any difference.
Why it matters
- A headline result has the wrong sign where most readers see it. The abstract names three hepatic isotope findings, for Fe, Cu and Zn. For Cu it states the direction opposite to the data. A reader who relies on the abstract, or on a database or review that quotes it, will record that malaria infection makes liver Cu isotopically lighter. The data show a heavier shift of about +0.9‰ at 7 DPI and +0.6‰ at 10 DPI, which is large compared with the uninfected SD of 0.12‰.
- The abstract's mechanism reads as self-contradictory. The same abstract sentence links the liver signal to "a release of Cu into the bloodstream through ceruloplasmin". The Discussion explains this release as hepatocytes exporting Cu "enriched in the light Cu isotope", which leaves the liver heavier. With "lighter" in the abstract, the stated observation and the stated mechanism point in opposite directions.
- The cross-element pattern is misdescribed. As printed, the abstract implies that liver Fe, Zn and Cu all become isotopically lighter. In the paper's data, liver Fe and Zn become lighter and liver Cu becomes heavier (Table S4), a contrast the body of the paper treats as informative about ceruloplasmin-mediated Cu export.
- What is not affected. The measurements, the group statistics, the treatment effects, the correlation of liver δ65Cu with ceruloplasmin and the conclusions in the Results, Discussion and Conclusions are all stated on the correct (heavier) basis and are internally consistent. The error is confined to one word of the abstract.
Explanations considered
- A reversed or unusual δ convention. Ruled out. Eq. 3 puts the sample's 65Cu/63Cu in the numerator, and the Methods text says explicitly that a positive δ means enrichment in the heavier isotope. The paper uses the same reading everywhere else (for example "a significant decrease in δ66Zn and δ65Cu values were observed, indicating that purified pRBCs are enriched in the lighter Zn and Cu isotopes").
- "Lighter" refers to a different compartment. Not supported. The paper does find lighter Cu in purified pRBCs (Δ65Cu = −0.88‰ against whole blood) and describes "a lighter Cu isotope signature in blood and purified RBCs". But the abstract sentence says "in the liver" and pairs the isotope finding with the "hepatic Cu deficiency", and pRBC findings are covered by the abstract's second sentence. The wet and perfused liver data contain no lighter shift.
- A different comparison, such as treated versus untreated. Not able to explain it. Treatment lowers liver δ65Cu (by 0.27 to 0.33‰ against untreated 10 DPI mice), but that is the reversal of the infection effect, and the abstract describes the infection effect ("in the liver were observed, alongside a release of Cu into the bloodstream through ceruloplasmin"). Even treated livers stay heavier than uninfected ones (+0.32 to +0.38‰ against +0.09‰).
- Wet versus perfused liver, or 7 versus 10 DPI. Not able to explain it. Both preparations and both time points shift the same way (heavier). In the perfused liver the shift is smaller (+0.35‰ with all nine infected rows, p = 0.042; +0.28‰ for the eight rows that reproduce Table S5's 0.59 ± 0.17‰). Table S5's infected mean and SD match the infected perfused group without the row the sheet labels "Liver untreated P.Berghei LR-4" (δ65Cu 1.21‰). Table S5's uninfected perfused summary (0.29 ± 0.25‰) is not reproduced by the five uninfected perfused rows either (0.31 ± 0.29‰); no subset that leaves out one row matches it, and using the population SD (0.257) would bring the SD close to the printed value but leaves the mean unexplained. The printed p = 0.0306 is not reproduced exactly by an exact two-sided Mann–Whitney test on either set of rows (0.042 with nine, 0.065 with eight). These are side points about Table S5 and does not affect the direction, which is heavier in every version.
- The aging literature cited in the Discussion. Not able to explain it. Discussion paragraph 10 mentions that "In the aging process in mice, a gradual depletion in the heavier Cu isotope in the liver is observed", citing earlier work. That sentence is about aging, not this experiment's infected-versus-control comparison.
- A text-conversion or archiving artifact. Ruled out. The conflict is in words ("lighter" against "higher δ65Cu", "increase in the heavy Cu isotope"), not in symbols or exponents, and the publisher's HTML carries the same wording as the PMC and Europe PMC copies.
- A correction already issued. None found (see Status).
Status
- Version of record. The publisher's article page (nature.com, reached through doi.org; published 3 January 2026) gives the abstract sentence "A discrete hepatic Cu deficiency, but significantly lighter Cu isotopic composition in the liver were observed, alongside a release of Cu into the bloodstream through ceruloplasmin…" and the Results sentence "…while it led to higher δ65Cu values compared to controls", word for word as in the PMC archive and the Europe PMC full-text XML. The Discussion sentences quoted above are also identical in the publisher's HTML. Every quotation was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The article lists five supplementary files. I read the Supplementary Information PDF (MOESM2: Tables S4 and S5 give liver δ65Cu means, all higher in infected mice), the Supplementary Data workbook (MOESM4: sheet "Liver" gives the per-mouse values used here, which reproduce Table S4's δ65Cu column to within 0.01‰) and the Transparent Peer Review file (MOESM1), which contains no discussion of the direction of the liver Cu shift in the abstract. The supplements resolve the question in favour of the body text: liver Cu is heavier in infection.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices; the Crossref record has empty update-to and updated-by fields, and the publisher's page shows no Author Correction notice. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC12881638; Europe PMC full-text XML; the publisher's article page at https://doi.org/10.1038/s42003-025-09464-z and its supplementary files MOESM1, MOESM2 and MOESM4; Crossref; Europe PMC; PubPeer.
Suggested correction
In the abstract, replace
A discrete hepatic Cu deficiency, but significantly lighter Cu isotopic composition in the liver were observed
with
A discrete hepatic Cu deficiency, but significantly heavier Cu isotopic composition in the liver were observed
This matches the Results (Fig. 3G: higher δ65Cu in infected wet livers), the Discussion ("an increase in the heavy Cu isotope"; "a depletion in the lighter isotope"), SI Table S4 and the per-mouse Supplementary Data. No other change to the text, figures or conclusions is needed for this point. The authors may separately wish to check the perfused-liver δ65Cu summary and p-value in SI Table S5 against the uninfected and infected perfused rows of the Supplementary Data, as noted under Explanations considered.
Recomputation script (python recompute.py.txt)
"""Check the direction of the liver Cu isotope shift in PMC12881638
(Commun Biol 2026, doi:10.1038/s42003-025-09464-z,
"Iron, copper and zinc isotope ratios in malaria infection reveal signatures of the
Plasmodium biology and host mice response to the infection").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper and its supplements:
* Methods, Eq. 3: d65Cu = [(65Cu/63Cu)_sample / (65Cu/63Cu)_NIST SRM 976 - 1] x 1000.
* Supplementary Information (MOESM2), Table S4 (wet liver, mean +- SD) and Table S5
(perfused liver, mean +- SD, Mann-Whitney p).
* Supplementary Data (MOESM4_ESM.xlsx), sheet "Liver", column d65Cu (permil), one value
per mouse, rounded here to 4 decimals. Group codes follow the sheet's own "Description"
column: UI = uninfected; LP and HP = untreated infected (matched to 7 and 10 DPI by
their means and SDs, see section 2); CQ-24, CQ-48 = chloroquine-treated, 10 DPI;
LR-UI / LR-I = perfused liver, uninfected / infected. The one relabelling: the seven
amodiaquine rows (Description AQ-24 ... AQ-30, sample names "S. Amodiaquine (1)" to
"(7)") are pooled here as one group, "AQ" (= Table S4 "AQ 24 h, 10 DPI").
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
# MOESM4, sheet "Liver", d65Cu (permil) per mouse
LIVER = {
"UI": [0.1203, 0.0838, -0.1183, 0.1761, 0.0205, 0.2331],
"LP": [1.4924, 0.8149, 1.1340, 1.3079, 1.0833, 0.5820, 1.0148, 0.7734],
"HP": [0.6356, 0.4404, 0.7949, 1.0177, 0.5032, 0.6561, 0.6988, 0.4510],
"CQ-24": [0.2914, 0.4986, 0.2416, 0.3679, 0.2826, 0.5409, 0.4637, 0.3323],
"CQ-48": [0.2897, 0.2812, 0.3696, 0.3035, 0.4609, 0.3376, 0.2916, 0.1915],
"AQ": [0.3720, 0.5865, 0.3357, 0.3023, 0.4695, 0.3152, 0.2505],
"LR-UI": [0.2233, 0.1430, -0.0229, 0.5208, 0.6827],
"LR-I": [0.7168, 0.4972, 0.8380, 1.2108, 0.2620, 0.5219, 0.6049, 0.5776, 0.7067],
}
# SI Table S4, wet liver, mean +- SD (permil): d56Fe, d66Zn, d65Cu
S4 = {
"Untreated, 0 DPI": ((-1.38, 0.21), (-0.42, 0.02), (0.08, 0.12)),
"Untreated, 7 DPI": ((-1.80, 0.21), (-0.71, 0.03), (1.02, 0.29)),
"Untreated, 10 DPI": ((-2.32, 0.11), (-0.62, 0.06), (0.65, 0.19)),
"AQ 24 h, 10 DPI": ((-1.94, 0.07), (-0.58, 0.06), (0.37, 0.11)),
"CQ 24 h, 10 DPI": ((-1.96, 0.10), (-0.53, 0.03), (0.37, 0.11)),
"CQ 48 h, 10 DPI": ((-1.94, 0.08), (-0.54, 0.04), (0.31, 0.07)),
}
# SI Table S5, perfused liver d65Cu mean +- SD, and Mann-Whitney p
S5 = {"0 DPI": (0.29, 0.25), "10 DPI": (0.59, 0.17), "p": 0.0306}
# 65Cu/63Cu of NIST SRM 976 (certified 63Cu/65Cu = 2.2440), used only to illustrate Eq. 3
R_NIST976 = 1 / 2.2440
def d65(R):
"""Eq. 3 of the paper."""
return (R / R_NIST976 - 1) * 1000
def mw(a, b):
r = stats.mannwhitneyu(a, b, alternative="two-sided", method="exact")
return r.statistic, r.pvalue
# ------------------------------------------------------------------ 1. Eq. 3 sign
print("=== 1. Sign convention of Eq. 3 ===")
print(f"65Cu/63Cu of NIST SRM 976 = 1/2.2440 = {R_NIST976:.5f}")
for f in (0.999, 1.001):
R = R_NIST976 * f
print(f" sample ratio = reference x {f}: d65Cu = {d65(R):+.2f} permil "
f"({'more' if f > 1 else 'less'} 65Cu -> {'heavier' if f > 1 else 'lighter'})")
print(" d(infected) - d(control) = 1000 x (R_inf - R_ctrl) / R_ref, so a higher d65Cu "
"always means a larger 65Cu/63Cu ratio (isotopically heavier), whatever the sign of d itself.")
# ------------------------------------------------------------------ 2. per-mouse data vs Table S4/S5
print("\n=== 2. Liver d65Cu per group: MOESM4 per-mouse values vs SI tables ===")
REPORTED = {"UI": S4["Untreated, 0 DPI"][2], "LP": S4["Untreated, 7 DPI"][2],
"HP": S4["Untreated, 10 DPI"][2], "CQ-24": S4["CQ 24 h, 10 DPI"][2],
"CQ-48": S4["CQ 48 h, 10 DPI"][2], "AQ": S4["AQ 24 h, 10 DPI"][2],
"LR-UI": S5["0 DPI"], "LR-I": S5["10 DPI"]}
SUM = {}
for g, v in LIVER.items():
x = np.array(v)
m, sd = x.mean(), x.std(ddof=1)
SUM[g] = (m, sd, len(x))
rm, rsd = REPORTED[g]
print(f" {g:6s} n={len(x)} recomputed {m:+.3f} +- {sd:.3f} table {rm:+.2f} +- {rsd:.2f}"
f" |diff| mean {abs(m-rm):.3f}, SD {abs(sd-rsd):.3f}"
f" truncated to 2 dp: {np.trunc(m*100)/100:+.2f} +- {np.trunc(sd*100)/100:.2f}")
S4G = ("UI", "LP", "HP", "CQ-24", "CQ-48", "AQ")
dmax = max(max(abs(SUM[g][0] - REPORTED[g][0]), abs(SUM[g][1] - REPORTED[g][1])) for g in S4G)
ntrunc = sum(1 for g in S4G for a, b in ((SUM[g][0], REPORTED[g][0]), (SUM[g][1], REPORTED[g][1]))
if abs(np.trunc(a * 100) / 100 - b) < 1e-9)
print(f" Table S4 d65Cu (6 groups x mean, SD): largest |recomputed - printed| = {dmax:.4f} permil;"
f" cells equal after truncation to 2 dp: {ntrunc}/12")
x = np.array(LIVER["LR-UI"])
print(f" LR-UI (Table S5 0 DPI {S5['0 DPI'][0]} +- {S5['0 DPI'][1]}): sample SD {x.std(ddof=1):.3f},"
f" population SD {x.std(ddof=0):.3f}")
for k in range(len(x)):
y = np.delete(x, k)
print(f" without row {k+1}: mean {y.mean():+.3f} +- {y.std(ddof=1):.3f}")
# ------------------------------------------------------------------ 3. infection vs control
print("\n=== 3. Infected vs uninfected liver d65Cu (Mann-Whitney, exact, two-sided) ===")
COMP = [("LP (7 DPI)", "LP", "UI"), ("HP (10 DPI)", "HP", "UI"), ("perfused, infected", "LR-I", "LR-UI")]
DIFF = {}
for lab, a, b in COMP:
U, p = mw(LIVER[a], LIVER[b])
dm = SUM[a][0] - SUM[b][0]
DIFF[a] = dm
n_above = sum(1 for x in LIVER[a] if x > max(LIVER[b]))
print(f" {lab:20s} mean shift {dm:+.3f} permil ratio factor 1 + {dm/1000:.5f} "
f"U = {U:.0f}, p = {p:.5f} infected mice above every control: {n_above}/{len(LIVER[a])}")
print(f" SI Table S5 reports p = {S5['p']} for perfused liver d65Cu")
# Table S5 (0.59 +- 0.17) matches the infected perfused group without its 4th row, the one the
# sheet labels "Liver untreated P.Berghei LR-4" (d65Cu 1.2108); the paper names Grubbs' test for outliers.
lr8 = [x for k, x in enumerate(LIVER["LR-I"]) if k != 3]
U8, p8 = mw(lr8, LIVER["LR-UI"])
print(f" perfused infected without row LR-4: n={len(lr8)} mean {np.mean(lr8):+.3f} +- {np.std(lr8, ddof=1):.3f}"
f" shift {np.mean(lr8)-SUM['LR-UI'][0]:+.3f} U = {U8:.0f}, p = {p8:.4f}")
x = np.array(LIVER["LR-I"]); gstat = np.max(np.abs(x - x.mean())) / x.std(ddof=1)
n = len(x); tcrit = stats.t.ppf(1 - 0.05 / (2 * n), n - 2)
gcrit = (n - 1) / np.sqrt(n) * np.sqrt(tcrit**2 / (n - 2 + tcrit**2))
print(f" Grubbs' G for LR-I = {gstat:.3f} (two-sided critical value at alpha 0.05, n={n}: {gcrit:.3f})")
# ------------------------------------------------------------------ 4. treatment
print("\n=== 4. Treated vs untreated 10 DPI (HP) ===")
for g in ("CQ-24", "CQ-48", "AQ"):
U, p = mw(LIVER[g], LIVER["HP"])
print(f" {g:6s} mean {SUM[g][0]:+.3f} vs HP {SUM['HP'][0]:+.3f} (UI {SUM['UI'][0]:+.3f}) "
f"shift {SUM[g][0]-SUM['HP'][0]:+.3f} U = {U:.0f}, p = {p:.5f}")
# ------------------------------------------------------------------ 5. direction of each element (Table S4)
print("\n=== 5. Direction of the infection shift in wet liver, by element (Table S4 means) ===")
for k, el in enumerate(("d56Fe", "d66Zn", "d65Cu")):
c = S4["Untreated, 0 DPI"][k][0]
for t in ("Untreated, 7 DPI", "Untreated, 10 DPI"):
v = S4[t][k][0]
print(f" {el}: 0 DPI {c:+.2f} -> {t[11:]} {v:+.2f} shift {v-c:+.2f} "
f"-> {'heavier' if v > c else 'lighter'}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# ---- Figure 1: text-vs-text contradiction diagram
fig = plt.figure(figsize=(7.2, 4.0))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100); ax.set_ylim(-7, 60); ax.axis("off")
def panel(x, y, w, h, edge, title, lines, title_color):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.6,rounding_size=1.5",
fc="white", ec=edge, lw=2))
ax.text(x + 1.5, y + h - 2.2, title, fontsize=11, fontweight="bold", color=title_color, va="top")
yy = y + h - 9.0
for txt, col, sz in lines:
ax.text(x + 1.5, yy, txt, fontsize=sz, color=col, va="top", linespacing=1.3)
yy -= 4.3 * (txt.count("\n") + 1) + 3.0
panel(1, 6, 37, 50, ORANGE, "Abstract", [
("\u201cA discrete hepatic Cu\ndeficiency, but significantly\nlighter Cu isotopic composition\nin the liver were observed \u2026\u201d", INK, 10),
("Reads as: infected liver has\nlower \u03b4\u2076\u2075Cu than controls", ORANGE, 10),
], ORANGE)
panel(54, 6, 45, 50, INK2, "Results, Discussion, SI data", [
("Results: \u201cPb infection led to \u2026\nhigher \u03b4\u2076\u2075Cu values compared\nto controls\u201d", INK, 10),
("Discussion: \u201can increase in the\nheavy Cu isotope was noticed\nin Pb infection\u201d", INK, 10),
(f"Per-mouse data (MOESM4), wet liver:\n0 DPI {SUM['UI'][0]:+.2f}\u2030 \u2192 7 DPI {SUM['LP'][0]:+.2f}\u2030,\n10 DPI {SUM['HP'][0]:+.2f}\u2030 (higher = heavier)", BLUE, 10),
], INK2)
arr = FancyArrowPatch((40.2, 31), (52.8, 31), arrowstyle="<|-|>", mutation_scale=16, lw=2, color=INK)
ax.add_patch(arr)
ax.text(46.5, 34, "opposite\nsign", ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
ax.text(46.5, 28, "lighter\nvs\nheavier", ha="center", va="top", fontsize=10, color=INK2)
ax.text(1, 3.2, "Orange: wording reported in the abstract. Blue: group means recomputed\nfrom the paper's per-mouse supplementary data (recompute.py.txt).", fontsize=10, color=INK2, va="top")
save(fig, "fig1")
# ---- Figure 2: per-mouse liver d65Cu by group, reported vs recomputed means
ORDER = [("UI", "Uninfected\n0 DPI"), ("LP", "Infected\n7 DPI"), ("HP", "Infected\n10 DPI"),
("CQ-24", "CQ 24 h\n10 DPI"), ("CQ-48", "CQ 48 h\n10 DPI"), ("AQ", "AQ 24 h\n10 DPI")]
fig, ax = plt.subplots(figsize=(7.2, 4.2))
rng = np.random.default_rng(1)
ctrl = SUM["UI"][0]
ax.axhline(ctrl, color=INK2, lw=0.8, ls=(0, (4, 3)), zorder=1)
ax.text(5.55, ctrl - 0.03, "uninfected\nmean", fontsize=10, color=INK2, va="top", ha="right")
for i, (g, _) in enumerate(ORDER):
v = np.array(LIVER[g])
jit = rng.uniform(-0.13, 0.13, len(v))
ax.scatter(i - 0.12 + jit, v, s=64, facecolors="none", edgecolors=INK2, lw=1.1, zorder=3)
m, sd, n = SUM[g]
ax.errorbar(i + 0.22, m, yerr=sd, fmt="o", ms=8, color=BLUE, lw=2, capsize=0, zorder=4)
rm, rsd = REPORTED[g]
ax.errorbar(i + 0.38, rm, yerr=rsd, fmt="D", ms=8, color=ORANGE, lw=2, capsize=0, zorder=4)
for g, i in (("LP", 1), ("HP", 2)):
p = mw(LIVER[g], LIVER["UI"])[1]
ax.text(i + 0.1, max(LIVER[g]) + 0.07, f"vs uninfected\np = {p:.4f}", ha="center", va="bottom",
fontsize=10, color=BLUE)
ax.annotate("", xy=(-0.42, -0.55), xytext=(-0.42, -0.15),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2))
ax.text(-0.33, -0.36, "abstract's “lighter”\nwould lie down here", fontsize=10, color=ORANGE, va="center")
ax.set_xticks(range(len(ORDER)), [lab for _, lab in ORDER], fontsize=10)
ax.set_ylabel("Liver δ⁶⁵Cu (‰, vs NIST SRM 976)\nhigher = isotopically heavier")
ax.set_ylim(-0.6, 1.85); ax.set_xlim(-0.55, 5.65)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
handles = [Line2D([], [], marker="o", ls="", mfc="none", mec=INK2, ms=8, label="each mouse (MOESM4)"),
Line2D([], [], marker="o", ls="-", color=BLUE, ms=8, label="mean ± SD recomputed"),
Line2D([], [], marker="D", ls="-", color=ORANGE, ms=8, label="mean ± SD reported (Table S4)")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=3, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Sahel-wide foliage increases of 6% and 20% do not match the relative increases implied by the paper's own totals
Commun Biol (2019) · doi:10.1038/s42003-019-0383-9 · PMC6478729
Summary
The Abstract and Results report that satellite (VOD) estimates show Sahel-wide increases of 6% in total herbaceous mass (AGH) and 20% in total woody foliage mass (WPF) between 1992 and 2012. The same Results sentence gives the totals behind these percentages: AGH 0.28 Pg in 1992 and 0.31 Pg in 2012, WPF 0.16 Pg and 0.24 Pg. Those totals give relative increases of 10.7% and 50.0%. Allowing for rounding of the two-decimal totals, the possible ranges are 7.0–14.5% and 42.4–58.1%, so neither reported figure can be reached as an increase over each component's own 1992 value. Two denominators the paper does not state (1992 total vegetation mass, or the sum of the two years) can reach both figures within rounding of the printed totals, but the unrounded Supplementary Data does not reproduce the 20% WPF figure on either (see "Explanations considered"). The totals agree with the rest of the paragraph and with the paper's Supplementary Data, which points to the two percentages as the inconsistent figures. They are the only Sahel-wide figures in the Abstract and the paper's only Sahel-wide relative (percentage) increases. On the printed totals, the woody increase is understated about 2.5-fold (2.1–2.9-fold over rounding; 2.3-fold from the Supplementary Data endpoints) and the herbaceous increase about 1.8-fold (1.2–2.4-fold over rounding; 1.46-fold from the Supplementary Data endpoints). The direction of the main finding, that woody foliage rose faster than herbaceous mass, is unchanged.
What the paper reports
Abstract:
Satellite observations confirm these findings for the majority of the Sahel, with total herbaceous/woody foliage mass increases by 6%/20%.
Results, "Scaling field-site observations to the Sahel", paragraph 1 (the three sentences are consecutive):
For the entire western and central Sahel area, the summed AGH increased by 0.0025 Pg yr-1 over the 21-year period, and WPF by 0.004 Pg yr-1. Total AGH (WPF) was 0.28 Pg (0.16 Pg) in 1992 and 0.31 Pg (0.24 Pg) in 2012, which is a relative increase of 6% for AGH and 20% for WPF. The share of VOD estimated WPF (as related to the total vegetation mass) increased from 35% in 1992 to 43% in 2012, which is comparable to the percentage numbers observed from the Ferlo field samples.
The same paragraph, earlier:
According to fitted trends derived from VOD metrics from 1992 to 2012, AGH increased by + 5 ± 7 kg ha-1 yr-1 and WPF increased by +9 ± 8 kg ha-1 yr-1.
and later, giving the size of the significant-trend areas:
Areas with a significant (P < 0.05) increase in AGH cover a smaller region (289,375 km²; ~7%) than areas with a significant increase in WPF (2,711,875 km²; ~64%)
Discussion, paragraph 5, which builds on the composition of the increase:
However, a considerable portion of the observed increase in vegetation production consists of woody foliage mass, the fodder value of which is generally lower than that of herbaceous plants and would benefit primarily browsers (goats) over grazers (sheep and cattle).
What the paper's own numbers show
Method. The Results sentence defines the percentages as a "relative increase" computed from the stated 1992 and 2012 totals, so I took (2012 − 1992) / 1992 for each component. The totals are printed to two decimals, so I also computed the smallest and largest increase possible when each total varies by ±0.005 Pg. I then checked the totals against the WPF share given in the next sentence. Next I tried every other basis that could plausibly give 6% and 20%: the stated fitted trends, the per-hectare trends, other denominators, and the yearly Sahel series in the paper's Supplementary Data (file 42003_2019_383_MOESM4_ESM.csv, columns "Agh sahel" and "Wpf sahel", kg/ha, 1992–2012). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| AGH relative increase, 1992→2012 | Results p1; Abstract | 6% | (0.31 − 0.28) / 0.28 = 10.7%; 7.0–14.5% over rounding |
| WPF relative increase, 1992→2012 | Results p1; Abstract | 20% | (0.24 − 0.16) / 0.16 = 50.0%; 42.4–58.1% over rounding |
| Recomputed / reported | derived | — | Printed totals: AGH 1.79×, WPF 2.50×; over rounding AGH 1.17–2.42×, WPF 2.12–2.90×; Supplementary Data endpoints AGH 1.46×, WPF 2.33× |
| What the reported % would mean in Pg | derived | 6% of 0.28; 20% of 0.16 | 0.017 Pg and 0.032 Pg, against stated differences of 0.03 and 0.08 Pg |
| WPF share of total mass, 1992 | Results p1 | 35% | 0.16 / 0.44 = 36.4% (consistent) |
| WPF share of total mass, 2012 | Results p1 | 43% | 0.24 / 0.55 = 43.6% (consistent) |
| Summed trend × 20 years | Results p1 | AGH 0.0025, WPF 0.004 Pg/yr | AGH 0.050 Pg = 17.9% of 0.28; WPF 0.080 Pg = 50.0% of 0.16 (18.7% / 52.5% over 21 years) |
| Per-hectare trend × 20 years / 1992 per-hectare mass | Results p1 | 5 and 9 kg/ha/yr | AGH 14.9%; WPF 47.1% (study area of about 4.19 million km² implied by 289,375 km² ≈ 7% and 2,711,875 km² ≈ 64%) |
| Pg/yr trend converted to kg/ha/yr with that area | Results p1 | 5 and 9 kg/ha/yr | 5.97 and 9.56 kg/ha/yr (roughly consistent) |
| Supplementary Data, endpoints | MOESM4 csv | — | AGH 595.34 → 647.31 kg/ha (+8.7%); WPF 337.62 → 494.97 kg/ha (+46.6%) |
| Supplementary Data, fitted OLS line 1992→2012 | MOESM4 csv | slopes 5 and 9 kg/ha/yr | slopes 5.41 and 8.68 kg/ha/yr; AGH +17.5%, WPF +51.8% |
| Supplementary Data scaled to Pg (area implied by 0.44 Pg in 1992, 4.72 × 10⁸ ha) | MOESM4 csv | 0.28→0.31 and 0.16→0.24 Pg; 0.0025 and 0.004 Pg/yr | 0.281→0.305 and 0.159→0.233 Pg; 0.0026 and 0.0041 Pg/yr |
| Supplementary Data, WPF share | MOESM4 csv | 35% → 43% | 36.2% → 43.3% |
The Pg totals, the trends, the shares and the supplementary series agree with one another. Only the two percentages do not. On the paper's totals, total vegetation (AGH + WPF) rose from 0.44 to 0.55 Pg, or 25.0%. The woody-to-herbaceous contrast in relative increase is 50.0 / 10.7 = 4.7× on the printed totals (5.3× from the Supplementary Data endpoints), compared with 20 / 6 = 3.3× as reported.
Figure
Figure 1. Relative increase from 1992 to 2012 for each component, on one axis starting at zero. Orange dots are the percentages reported in the Abstract and Results (6% for AGH, 20% for WPF). Blue dots are the increases recomputed from the totals stated in the same sentence (10.7% and 50.0%). The blue bars show every value the increase could take if each two-decimal total is anywhere within ±0.005 Pg of its printed value. Both reported values lie outside these ranges, by 1.0 percentage point for AGH and 22.4 points for WPF.
Figure 2. Every basis tried for the two percentages. Each panel has its own scale, starting at zero. The orange line marks the reported value. Filled blue dots show each component's change measured against its own 1992 value: from the stated totals, from the Supplementary Data endpoints and fitted line, and from the stated Pg/yr and kg/ha/yr trends over 20 years. For the kg/ha/yr row the denominator is the 1992 per-hectare mass, 0.28 or 0.16 Pg spread over the study area of about 4.19 × 10⁸ ha implied by the significant-trend areas (669 and 382 kg/ha). The closest are 8.7% for AGH and 46.6% for WPF. Hollow dots use other denominators, none of which the paper states. Only two land near both reported values. Dividing by the sum of the two years gives 5.1% and 20.0%. Dividing by the 1992 total vegetation mass gives 6.8% and 18.2%. These are discussed under "Explanations considered".
Why it matters
The 6%/20% figures are the only Sahel-wide figures in the Abstract and the paper's only Sahel-wide relative (percentage) increases for the satellite analysis; the Results also give the Sahel-wide trends in kg/ha/yr and Pg/yr, which are not affected. The Abstract uses them as the quantitative evidence that "Satellite observations confirm these findings for the majority of the Sahel". A reader who cites the Abstract would take the woody foliage increase to be 20% when the paper's own totals give 50%, and the herbaceous increase to be 6% when they give about 11%.
The errors do not reverse anything. Both components still increase, and woody foliage still increases faster than herbaceous mass. With the recomputed values the contrast is larger (4.7× on the printed totals and 5.3× from the Supplementary Data endpoints, rather than 3.3×), so the paper's qualitative conclusion is, if anything, stronger. The per-hectare trends (field data: +17 vs +3 kg/ha/yr; VOD: +9 vs +5 kg/ha/yr), the 35%→43% WPF share, the rainfall decomposition and the significance tests do not depend on the two percentages and are not affected. The Discussion's statement that "a considerable portion of the observed increase in vegetation production consists of woody foliage mass" is not wrong. On the stated totals the woody share of the total increase is 0.08 of 0.11 Pg, about 73%, which the 6%/20% figures do not convey.
So this is a misstatement of magnitude in a headline number, not a change in direction. The correct values cannot be read off anywhere in the paper and have to be recomputed.
Explanations considered
- Rounding of the printed totals. This does not account for the numbers as stated, that is, as each component's increase over its own 1992 value. With every total free to vary by ±0.005 Pg, AGH can range only from 7.0% to 14.5% and WPF from 42.4% to 58.1% (Figure 1).
- Percentages from the fitted trend rather than the endpoints. This does not account for the numbers. The stated summed trends over 20 years give 17.9% and 50.0% (18.7% and 52.5% over 21 years). The per-hectare trends give 14.9% and 47.1%. The supplementary series' fitted line gives 17.5% and 51.8%. All are further from 6% and 20% than the endpoint values.
- The Pg totals, not the percentages, are misprinted. This is unlikely. The totals reproduce the 35% and 43% WPF shares stated in the next sentence (36.4% and 43.6%). Scaled from the Supplementary Data, they come to 0.281/0.305 Pg and 0.159/0.233 Pg, and the trend of that series matches the stated 0.0025 and 0.004 Pg/yr. The supplementary series itself gives increases of 8.7% and 46.6%.
- Increase relative to the 2012 value or to the mean of the two years. This does not account for the numbers: 9.7%/33.3% and 10.2%/40.0%.
- Each component's increase relative to total vegetation mass in 1992 (AGH + WPF = 0.44 Pg). This is the strongest alternative. From the printed totals it gives 6.8% and 18.2%, and over rounding of the totals the ranges are 4.4–9.3% for AGH and 15.6–20.9% for WPF, so on the printed totals both reported values can be reached on this basis. The case against it rests on three points. The paper calls the figures "a relative increase of 6% for AGH and 20% for WPF" and, in the Abstract, increases in "total herbaceous/woody foliage mass", not changes measured against total vegetation. The basis is not stated anywhere in the paper. And the unrounded Supplementary Data gives 5.6% for AGH, which rounds to 6%, but 16.9% for WPF, which rounds to 17%. The Supplementary Data therefore reproduces the AGH figure but not the WPF figure.
- Difference divided by the sum of the two years. From the printed totals this gives 5.1% and 20.0%, and over rounding the ranges are 3.4–6.8% for AGH and 17.5–22.5% for WPF, so both reported values can also be reached on this basis. This is not a relative increase in the usual sense: it equals half the change relative to the two-year mean. It is not stated in the paper. From the unrounded Supplementary Data it gives 4.2% and 18.9%, which would round to 4% and 19%, so the Supplementary Data reproduces neither figure on this basis.
- Change in each component's share of total vegetation. The WPF share rises from 36.4% to 43.6%, a relative change of 20.0%. The AGH share falls by 11.4%, though, so this basis cannot give a 6% increase for AGH.
- Conversion or typesetting artifact. Excluded. The publisher PDF, the publisher HTML, the PMC-archived text and the Europe PMC XML all print the same plain digits ("0.28", "0.31", "0.16", "0.24", "6%", "20%"). The passages contain no minus signs, superscripts or table cells that conversion could have altered.
No basis that the paper states, or that measures each component against its own baseline, reproduces 6% and 20%. Two unstated denominators (the 1992 total vegetation mass and the sum of the two years) can reach both values within rounding of the printed totals. Neither is described in the paper, and on the unrounded Supplementary Data the first reproduces the AGH figure but not the WPF figure, while the second reproduces neither. The published material does not show how the percentages were obtained.
Status
- Version of record. I checked the publisher PDF (nature.com/articles/s42003-019-0383-9.pdf, text extracted with pdftotext) for this write-up. The Abstract and the Results sentence appear there exactly as quoted. An earlier audit in this review found the same wording and numbers on the publisher HTML page and in the Europe PMC full-text XML. Every quote above was confirmed verbatim against the PMC-archived text (
src/show.py PMC6478729 --grep). PMC renders "kg ha−1 yr−1" as "kg ha-1 yr-1", and the quotes follow PMC. - Supplements checked. In the earlier audit: MOESM1 (Reporting Summary), MOESM2 (Supplementary Information, Supplementary Figs 1–8 and Table 1), MOESM3 (description of supplementary items) and MOESM4 (Supplementary Data .csv), all from static-content.springer.com. I downloaded and re-read MOESM4 for this write-up. None defines the percentages differently. MOESM4 was used for the recomputation above and supports the stated totals, not the percentages.
- Corrections and comments. As of 2026-10-23, I found no correction, erratum, author correction, retraction or expression of concern. The Crossref record for the DOI has no update-to, updated-by or relation entries, and a Crossref query for works that update this DOI returned 0 results. PMC and Europe PMC list no related notices (
src/known.py). The publisher's article page shows no correction or change history. For PubPeer, the extension API returned a Cloudflare challenge page to automated access, and an earlier DOI lookup returned "Not Found". A web search for the title together with PubPeer, correction or erratum found only copies of the article and repository records. - Sources. Publisher article page · PMC6478729 · Crossref record · Supplementary Data (MOESM4)
Suggested correction
- In the Results, "Scaling field-site observations to the Sahel", paragraph 1, replace "which is a relative increase of 6% for AGH and 20% for WPF" with the increases implied by the unrounded totals. The printed totals give about 11% for AGH and 50% for WPF, and the Supplementary Data series gives 8.7% and 46.6% from endpoints.
- Make the same change in the Abstract ("total herbaceous/woody foliage mass increases by 6%/20%").
- If a different basis was intended, for example the change relative to total vegetation mass, state that basis in both places and recompute both figures on it from the unrounded data.
- Optionally, report the increase in total vegetation (0.44 → 0.55 Pg, about 25%) and the woody share of that increase (about 73%), which make the fodder implications in the Discussion concrete.
Recomputation script (python recompute.py.txt)
"""Recompute the Sahel-wide relative increases reported in PMC6478729
(Commun Biol 2019, doi:10.1038/s42003-019-0383-9).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (Results, "Scaling field-site
observations to the Sahel", paragraph 1) and from the Supplementary Data file
(42003_2019_383_MOESM4_ESM.csv, columns "Agh sahel" and "Wpf sahel", kg/ha).
AGH = above-ground herbaceous mass; WPF = woody plant foliage mass.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
TOT = {"AGH": (0.28, 0.31), "WPF": (0.16, 0.24)} # Pg, 1992 and 2012 (Results p1)
REPORTED = {"AGH": 6.0, "WPF": 20.0} # % "relative increase" (Results p1, Abstract)
TREND_PG = {"AGH": 0.0025, "WPF": 0.004} # Pg/yr, summed trend (Results p1)
TREND_HA = {"AGH": 5.0, "WPF": 9.0} # kg/ha/yr, fitted VOD trend (Results p1)
SHARE_REPORTED = (35.0, 43.0) # % WPF share of total, 1992 and 2012
SIG_AREA = {"AGH": (289375, 0.07), "WPF": (2711875, 0.64)} # km2 and "~%" of area (Results p1)
HALF = 0.005 # totals are printed to 2 decimals
# Supplementary Data (MOESM4), Sahel VOD estimates in kg/ha, 1992..2012
YEARS = np.arange(1992, 2013)
CSV = {
"AGH": [595.3383, 594.5558, 564.6489, 618.3494, 566.197, 449.3458, 814.1302, 983.0184,
661.9179, 710.5017, 506.1419, 829.6328, 539.4238, 802.3855, 687.8374, 726.8941,
722.1816, 575.7401, 921.7831, 587.7719, 647.3133],
"WPF": [337.616, 338.2679, 338.3046, 434.1801, 373.5995, 313.1169, 301.23, 391.4529,
495.0082, 439.4241, 429.4322, 403.0291, 458.2273, 429.9664, 464.0112, 453.4704,
493.9232, 486.6456, 452.9526, 531.8839, 494.9722],
}
CSV = {k: np.array(v) for k, v in CSV.items()}
def pct(x):
return f"{100 * x:.1f}%"
def grid(x):
return np.linspace(x - HALF, x + HALF, 101)
def rng(f, *vals):
"""Min and max of f over every combination of the rounded inputs."""
mesh = np.meshgrid(*[grid(v) for v in vals], indexing="ij")
out = f(*mesh)
return out.min(), out.max()
R = {} # results reused by the figures
print("=== 1. Relative increase from the stated totals: (2012 - 1992) / 1992 ===")
for k, (a, b) in TOT.items():
rel = (b - a) / a
lo, hi = rng(lambda x, y: (y - x) / x, a, b)
R[k] = dict(rel=100 * rel, lo=100 * lo, hi=100 * hi)
print(f" {k}: ({b} - {a}) / {a} = {pct(rel)} reported {REPORTED[k]:.0f}%")
print(f" range over rounding of both totals (+/-{HALF} Pg): {pct(lo)} to {pct(hi)}")
print(f" ratio recomputed / reported = {100 * rel / REPORTED[k]:.2f}")
print(f" reported % of the 1992 total = {REPORTED[k] / 100 * a:.3f} Pg "
f"(stated difference {b - a:.2f} Pg)")
print(f" WPF/AGH contrast: reported {REPORTED['WPF'] / REPORTED['AGH']:.1f}x, "
f"recomputed {R['WPF']['rel'] / R['AGH']['rel']:.1f}x")
print("\n=== 2. Consistency of the totals with the stated WPF share (35% -> 43%) ===")
s92 = TOT["WPF"][0] / (TOT["WPF"][0] + TOT["AGH"][0])
s12 = TOT["WPF"][1] / (TOT["WPF"][1] + TOT["AGH"][1])
print(f" 1992: 0.16 / 0.44 = {pct(s92)} (reported {SHARE_REPORTED[0]:.0f}%)")
print(f" 2012: 0.24 / 0.55 = {pct(s12)} (reported {SHARE_REPORTED[1]:.0f}%)")
print(f" total vegetation 0.44 -> 0.55 Pg: {pct(0.55 / 0.44 - 1)}")
dw, da = TOT["WPF"][1] - TOT["WPF"][0], TOT["AGH"][1] - TOT["AGH"][0]
print(f" woody share of the total increase: {dw:.2f} / {dw + da:.2f} Pg = {pct(dw / (dw + da))}")
print("\n=== 3. Alternative bases (none is stated in the paper) ===")
ALT = {}
for k, (a, b) in TOT.items():
o = "WPF" if k == "AGH" else "AGH"
oa, ob = TOT[o]
ALT.setdefault(k, {})
ALT[k]["Fitted trend x 20 yr / 1992 total"] = 100 * TREND_PG[k] * 20 / a
ALT[k]["Fitted trend x 21 yr / 1992 total"] = 100 * TREND_PG[k] * 21 / a
ALT[k]["Difference / 2012 total"] = 100 * (b - a) / b
ALT[k]["Difference / mean of both years"] = 100 * (b - a) / ((a + b) / 2)
ALT[k]["Difference / sum of both years"] = 100 * (b - a) / (a + b)
ALT[k]["Difference / 1992 AGH+WPF"] = 100 * (b - a) / (a + oa)
ALT[k]["Difference / 2012 AGH+WPF"] = 100 * (b - a) / (b + ob)
print(f" {k}: trend {TREND_PG[k]} Pg/yr x 20 = {TREND_PG[k] * 20:.3f} Pg "
f"(endpoint difference {b - a:.2f} Pg)")
for name, v in ALT[k].items():
print(f" {name:<36s} {v:5.1f}%")
lo, hi = rng(lambda x, y: (y - x) / (x + y), *TOT["AGH"])
lo2, hi2 = rng(lambda x, y: (y - x) / (x + y), *TOT["WPF"])
print(f" rounding range, difference / sum: AGH {pct(lo)} to {pct(hi)}; WPF {pct(lo2)} to {pct(hi2)}")
lo, hi = rng(lambda x, y, w: (y - x) / (x + w), TOT["AGH"][0], TOT["AGH"][1], TOT["WPF"][0])
lo2, hi2 = rng(lambda w, y, x: (y - w) / (x + w), TOT["WPF"][0], TOT["WPF"][1], TOT["AGH"][0])
print(f" rounding range, difference / 1992 AGH+WPF: AGH {pct(lo)} to {pct(hi)}; "
f"WPF {pct(lo2)} to {pct(hi2)}")
s_agh92, s_agh12 = 1 - s92, 1 - s12
print(f" relative change of each component's share of total: WPF {pct(s12 / s92 - 1)}, "
f"AGH {pct(s_agh12 / s_agh92 - 1)}")
print("\n=== 4. Per-hectare trends and the implied study area ===")
areas = {k: a / p for k, (a, p) in SIG_AREA.items()}
for k, v in areas.items():
print(f" area implied by {k} significant area: {SIG_AREA[k][0]} / {SIG_AREA[k][1]} = {v:,.0f} km2")
area_ha = np.mean(list(areas.values())) * 100
print(f" mean implied area = {area_ha:.3e} ha")
for k, (a, b) in TOT.items():
per_ha = TREND_PG[k] * 1e12 / area_ha
base = a * 1e12 / area_ha
v = 100 * TREND_HA[k] * 20 / base
ALT[k]["Per-ha trend x 20 yr / 1992 per-ha mass"] = v
print(f" {k}: {TREND_PG[k]} Pg/yr -> {per_ha:.2f} kg/ha/yr (reported {TREND_HA[k]:.0f}); "
f"1992 mass {base:.0f} kg/ha; {TREND_HA[k]:.0f} x 20 / {base:.0f} = {v:.1f}%")
print("\n=== 5. Supplementary Data (MOESM4), Sahel kg/ha series 1992-2012 ===")
tot_csv = CSV["AGH"] + CSV["WPF"]
area_csv = 0.44e12 / tot_csv[0] # ha implied by the paper's 0.44 Pg total for 1992
print(f" 1992 AGH+WPF = {tot_csv[0]:.2f} kg/ha; area implied by 0.44 Pg = {area_csv:.3e} ha")
CSVR = {}
for k, y in CSV.items():
o = "WPF" if k == "AGH" else "AGH"
slope, icpt = np.polyfit(YEARS, y, 1)
end = (y[-1] - y[0]) / y[0]
fit0, fit1 = icpt + slope * YEARS[0], icpt + slope * YEARS[-1]
CSVR[k] = dict(end=100 * end, trend=100 * (fit1 - fit0) / fit0)
print(f" {k}: {y[0]:.2f} -> {y[-1]:.2f} kg/ha; scaled {y[0] * area_csv / 1e12:.3f} -> "
f"{y[-1] * area_csv / 1e12:.3f} Pg")
print(f" OLS slope {slope:.2f} kg/ha/yr = {slope * area_csv / 1e12:.4f} Pg/yr")
print(f" endpoint increase {pct(end)}; fitted-line increase {CSVR[k]['trend']:.1f}%")
print(f" difference / 1992 AGH+WPF {pct((y[-1] - y[0]) / tot_csv[0])}; "
f"difference / sum of both years {pct((y[-1] - y[0]) / (y[-1] + y[0]))}")
print(f" WPF share: {pct(CSV['WPF'][0] / tot_csv[0])} (1992), {pct(CSV['WPF'][-1] / tot_csv[-1])} (2012)")
for k in CSVR:
print(f" {k}: closest filled-dot value to reported {REPORTED[k]:.0f}% is "
f"{min([R[k]['rel'], CSVR[k]['end'], CSVR[k]['trend'], ALT[k]['Fitted trend x 20 yr / 1992 total'], ALT[k]['Per-ha trend x 20 yr / 1992 per-ha mass']], key=lambda v: abs(v - REPORTED[k])):.1f}%")
for k in R:
print(f" {k}: gap between reported value and nearest end of rounding range: {R[k]['lo'] - REPORTED[k]:.1f} points")
print("\n=== 6. Understatement factor (recomputed / reported) ===")
for k in R:
print(f" {k}: printed totals {R[k]['rel'] / REPORTED[k]:.2f}x; over rounding "
f"{R[k]['lo'] / REPORTED[k]:.2f}x to {R[k]['hi'] / REPORTED[k]:.2f}x; "
f"Supplementary Data endpoints {CSVR[k]['end'] / REPORTED[k]:.2f}x")
print(f" WPF/AGH contrast from Supplementary Data endpoints: {CSVR['WPF']['end'] / CSVR['AGH']['end']:.1f}x")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"axes.spines.top": False, "axes.spines.right": False,
})
MS = 9 # marker size in points (>= 8 px)
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: reported vs recomputed relative increase, with rounding range
fig, ax = plt.subplots(figsize=(7.2, 3.2))
rows = [("WPF", "Woody foliage (WPF)\n0.16 → 0.24 Pg"), ("AGH", "Herbaceous (AGH)\n0.28 → 0.31 Pg")]
for i, (k, lab) in enumerate(rows):
y = i
r = R[k]
ax.plot([r["lo"], r["hi"]], [y, y], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
for xe in (r["lo"], r["hi"]):
ax.plot([xe, xe], [y - 0.09, y + 0.09], color=BLUE, lw=2, zorder=2)
ax.plot(r["rel"], y, "o", ms=MS + 1, color=BLUE, zorder=3)
ax.plot(REPORTED[k], y, "o", ms=MS + 1, color=ORANGE, zorder=3)
if r["lo"] - REPORTED[k] > 3: # connector only where the gap is wide enough to draw
ax.annotate("", xy=(r["lo"] - 0.6, y), xytext=(REPORTED[k] + 0.9, y),
arrowprops=dict(arrowstyle="-", color=SEC, lw=1, ls=(0, (2, 2))))
ax.text(REPORTED[k], y - 0.2, f"reported {REPORTED[k]:.0f}%", ha="center", va="top",
color=ORANGE, fontsize=10, fontweight="bold")
ax.text(r["rel"], y + 0.2, f"recomputed {r['rel']:.1f}%", ha="center", va="bottom",
color=BLUE, fontsize=10, fontweight="bold")
ax.text(r["hi"] + 1.0, y, f"{r['lo']:.1f}–{r['hi']:.1f}%\nwith rounding", ha="left",
va="center", color=SEC, fontsize=10, linespacing=1.1)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([lab for _, lab in rows])
ax.set_ylim(-0.6, 1.55)
ax.set_xlim(0, 70)
ax.set_xlabel("Relative increase 1992 → 2012 (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[
Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="Reported in Abstract and Results"),
Line2D([], [], marker="o", ls="-", lw=2, ms=MS, color=BLUE,
label="Recomputed from the stated Pg totals (bar: range over rounding)"),
], loc="upper center", bbox_to_anchor=(0.5, -0.27), frameon=False, ncol=1, fontsize=10)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: every basis tried, for both components, against the reported values
order = [
("Stated totals: (2012 − 1992) / 1992", lambda k: R[k]["rel"], "main"),
("Supplementary Data: endpoints", lambda k: CSVR[k]["end"], "main"),
("Supplementary Data: fitted line", lambda k: CSVR[k]["trend"], "main"),
("Stated Pg/yr trend × 20 yr / 1992 total", lambda k: ALT[k]["Fitted trend x 20 yr / 1992 total"], "main"),
("Stated kg/ha/yr trend × 20 yr / 1992 kg/ha", lambda k: ALT[k]["Per-ha trend x 20 yr / 1992 per-ha mass"], "main"),
("Difference / 2012 total", lambda k: ALT[k]["Difference / 2012 total"], "alt"),
("Difference / mean of both years", lambda k: ALT[k]["Difference / mean of both years"], "alt"),
("Difference / 1992 AGH+WPF", lambda k: ALT[k]["Difference / 1992 AGH+WPF"], "alt"),
("Difference / sum of both years", lambda k: ALT[k]["Difference / sum of both years"], "alt"),
]
fig, axes = plt.subplots(1, 2, figsize=(7.2, 4.4), sharey=True)
for ax, k in zip(axes, ("AGH", "WPF")):
n = len(order)
for i, (lab, f, kind) in enumerate(order):
y = n - 1 - i
v = f(k)
face = BLUE if kind == "main" else BG
ax.plot(v, y, "o", ms=MS, color=BLUE, mfc=face, mew=2, zorder=3)
ax.text(v + (1.4 if k == "AGH" else 3.4), y, f"{v:.1f}", va="center", fontsize=10, color=TXT)
ax.axvline(REPORTED[k], color=ORANGE, lw=2, zorder=2)
ax.text(REPORTED[k] + (0.5 if k == "AGH" else 1.2), n - 0.35, f"reported {REPORTED[k]:.0f}%", color=ORANGE, ha="left",
va="bottom", fontsize=10, fontweight="bold")
ax.axhline(n - 5.5, color=SEC, lw=0.8, ls=(0, (3, 3)))
ax.set_xlim(0, 24 if k == "AGH" else 60)
ax.set_ylim(-0.6, n + 0.3)
ax.set_title("Herbaceous (AGH)" if k == "AGH" else "Woody foliage (WPF)", fontsize=11,
loc="left", pad=16)
ax.set_xlabel("Increase 1992 → 2012 (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
axes[0].set_yticks(range(len(order)))
axes[0].set_yticklabels([lab for lab, _, _ in order][::-1], fontsize=10)
fig.legend(handles=[
Line2D([], [], color=ORANGE, lw=2, label="Reported (6% / 20%)"),
Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, mew=2,
label="Component's change over its own 1992 value (endpoints or fitted trend)"),
Line2D([], [], marker="o", ls="", ms=MS, color=BLUE, mfc=BG, mew=2,
label="Other denominators (none is stated in the paper)"),
], loc="lower center", ncol=1, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.17, 1, 1))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract swaps the retina and vitreous ranibizumab half-lives given in the Results and Discussion
Commun Biol (2020) · doi:10.1038/s42003-019-0731-9 · PMC6949284
Summary
The abstract reports half-lives for labeled ranibizumab of 3.34 days at the retina and 4.52 days in the vitreous. The Results and the Discussion both give the same two numbers the other way round: 4.52 days in the retina and 3.34 days in the vitreous. The fitted lines in Figure 6a give about 4.55 days for the retina and 3.29 days for the vitreous, so the paper's own figure agrees with the body text and the abstract has the compartment labels exchanged. The half-lives are the abstract's only quantitative result. With the abstract's labels, the vitreous value (4.52 days) falls outside the 2.6–4 day range of earlier non-human primate (NHP) studies that the Discussion uses to validate the method. With the body's labels (3.34 days) it falls inside.
What the paper reports
All quotations are verbatim from the article as archived in PMC. The superscript reference numbers that PMC flattens into the text ("days5,8,28", "ranibizumab29") are written here as superscripts.
Abstract:
Here, we show a proof-of-concept for labeled ranibizumab with observed half-lives of 3.34 and 4.52 days at the retina and in the vitreous, respectively.
Results, section "In vivo distribution of a long-acting anti-VEGF antibody in the eye", second paragraph:
Maximal ranibizumab concentrations were estimated to reach 0.1 and 0.028 mg/mL in the vitreous and retina, respectively (Fig. 6a). Further pharmacokinetic analysis showed that labeled ranibizumab exhibited a half-life and an area under the curve (AUC) of 4.52 days and 0.22 day × mg/mL in the retina. In the vitreous, the calculated half-life, AUC, clearance, and volume distribution were 3.34 days, 0.57 day × mg/mL, 0.44 mL/day, and 4.78 mL, respectively.
Discussion, second paragraph:
The pharmacokinetic analysis for ranibizumab showed a half-life of 3.34 days in the vitreous, which is consistent with half-life measured in NHPs (2.6–4 days5,8,28) and in humans for ranibizumab29 with invasive measures. By contrast, the half-life measured in the retina with this technology was higher than reported elsewhere (4.52 days as compared to 2.6–2.3 days from ref. 5).
Clearance and volume of distribution were also similar between both approaches (0.594 mL/day for clearance and 2.75 mL for volume of distribution from enzyme-linked immunosorbent assay (ELISA) as compared to 0.44 mL/day and 4.78 mL from ocular tracking).
Methods, "Data analyses":
The half-lives of labeled antibodies in the vitreous and at the retina were then measured using a non-compartmental analysis on mean concentrations for all data points tested in each compartment using at least five time points with time range between Day 2 and last Day tested as the curve fitting approach and an uniform weighting (WinNonlin, Pharsight Corp., Mountain View, CA).
From the vitreous results, clearance (CL) was calculated from the formula: CL = dose/AUC, where dose is the amount of protein injected IVT (0.25 mg)
The volume of distribution (V) was calculated from the equation: V = CL/kel, where kel is the elimination rate constant obtained from the slope of the terminal data from the concentration–time plot.
Figure 6 legend, panel a:
Pharmacokinetics in both vitreous and retina were then determined by non-compartmental analysis based on mean concentrations at 40% intensity.
What the paper's own numbers show
Method. I tabulated the half-life given to each compartment in the abstract, the Results and the Discussion. I then made three independent checks of which assignment is correct.
- Figure 6a plots log concentration against time for each compartment, with a fitted line. I measured the slope of each fitted line in the publisher's full-size image, calibrated against the panel's own axis ticks (decades on the y-axis, days 0 and 30 on the x-axis), and converted it to a half-life (t1/2 = ln 2 / kel).
- I compared each assignment with the literature ranges the Discussion cites.
- I checked the vitreous clearance and volume against the Methods formulas.
Every number below is printed by recompute.py.txt, which also re-measures the Figure 6a lines when given the image file.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Ranibizumab t1/2, retina | Abstract | 3.34 d | Results and Discussion give 4.52 d (mismatch) |
| Ranibizumab t1/2, vitreous | Abstract | 4.52 d | Results and Discussion give 3.34 d (mismatch) |
| Ranibizumab t1/2, retina | Results; Discussion | 4.52 d | the two sections agree |
| Ranibizumab t1/2, vitreous | Results; Discussion | 3.34 d | the two sections agree |
| Slope of fitted line, vitreous panel | Figure 6a | (not stated) | 0.0914 decades/day, kel 0.211/day, t1/2 3.29 d |
| Slope of fitted line, retina panel | Figure 6a | (not stated) | 0.0662 decades/day, kel 0.152/day, t1/2 4.55 d |
| Vitreous t1/2 vs NHP range 2.6–4 d | Discussion | "consistent with" | 3.34 d is inside the range; 4.52 d is outside it |
| Retina t1/2 vs ref. 5 (2.3–2.6 d) | Discussion | "higher than reported elsewhere" | 4.52 d is 1.74 times 2.6 d; 3.34 d would be 1.28 times |
| Vitreous clearance CL = dose/AUC | Results; Methods | 0.44 mL/day | 0.25 / 0.57 = 0.4386 mL/day (matches) |
| Ratio of the two half-lives | — | — | 4.52 / 3.34 = 1.353 |
Figure 6a resolves the conflict. The vitreous panel's fitted line falls by a factor of 10 about every 10.9 days, which gives a half-life of 3.29 days. The retina panel's line falls more slowly, by a factor of 10 about every 15.1 days, which gives 4.55 days. Both values are within 1.5% of the body-text numbers for the same compartment (1.42% and 0.64%). They differ from the abstract's numbers by 27% and 36%. The body text and the figure therefore agree, and only the abstract differs. Its two numbers are exactly the body's two numbers with the compartment labels exchanged.
Figure
Figure 1. The two conflicting passages. Left: the abstract sentence and the assignment it states, retina 3.34 days and vitreous 4.52 days (orange). Right: the Results and Discussion, which both state retina 4.52 days and vitreous 3.34 days (blue). The arrows follow each number from one passage to the other: each one moves to the other compartment. The right panel also gives the half-lives implied by the slopes of the fitted lines in the paper's Figure 6a (4.55 days retina, 3.29 days vitreous), measured as described above. These agree with the body text.
Figure 2. Ranibizumab half-life by compartment on a linear axis starting at zero. Orange dots: abstract. Blue dots: Results and Discussion. Open diamonds: half-life from the slope of the fitted line in Figure 6a. Grey bands: the literature ranges the Discussion compares each compartment with, 2.6–4 days in NHPs for the vitreous and 2.3–2.6 days from ref. 5 for the retina. The Figure 6a slopes sit on the body-text values. Under the body-text assignment, the vitreous value lies inside its literature band, as the Discussion says ("consistent with"). Under the abstract's assignment it lies outside.
Why it matters
- The abstract's only quantitative result is misattributed. The two half-lives are the only numbers in the abstract, and they are presented as the proof of concept for the method. A reader of the abstract, or of an index or database that copies it, takes away the opposite pairing to the one the paper's data support.
- It reverses which compartment clears more slowly. According to the body text and Figure 6a, the retina has the longer apparent half-life, 35% longer than the vitreous (4.52 / 3.34 = 1.35). The abstract implies that the vitreous does.
- It undercuts the validation argument as summarised in the abstract. The Discussion supports the imaging method by noting that the vitreous half-life is "consistent with" invasive NHP measurements of 2.6–4 days. This holds for 3.34 days and not for the abstract's 4.52 days. The Discussion's second point, that the retinal value exceeds tissue-extraction values, also weakens with the abstract's labels: 1.28 times the upper end of ref. 5 instead of 1.74 times.
- What is not affected. The values themselves, the body text, Figure 6a, the vitreous clearance (0.44 mL/day, which reproduces as 0.25 / 0.57), the free-dye half-lives (2.68 and 2.66 days), the Day 21 comparison of ranibizumab with the long-acting antibody, and the feasibility conclusion all stand. The error is confined to the labels in one sentence of the abstract.
Explanations considered
- A different analyte. Does not account for it. All three passages concern labeled ranibizumab. The free dye has its own half-lives (2.68 days vitreous, 2.66 days retina), and neither is 3.34 or 4.52.
- A different analysis, time window or definition of half-life. Does not account for it. The Methods and the Figure 6 legend describe a single non-compartmental analysis on mean concentrations (Day 2 to the last day, uniform weighting), applied to both compartments. The paper describes no second analysis that could produce the same two numbers with the compartments reversed.
- An ambiguous "respectively". Does not account for it. The abstract lists the values and the compartments in explicit order: "3.34 and 4.52 days at the retina and in the vitreous, respectively". It has only one reading.
- The body, not the abstract, is in error. Not supported. The Results and the Discussion agree with each other. The Discussion's reasoning only works with the body's assignment: it calls the vitreous value "consistent with" 2.6–4 days and the retinal value "higher than" 2.3–2.6 days. Most directly, the slopes of the fitted lines in Figure 6a give 3.29 days for the vitreous and 4.55 days for the retina.
- A text-conversion artifact. Ruled out. The publisher's HTML on nature.com has the same wording and numbers in the abstract, Results and Discussion, and so do the PMC text and the PubMed abstract.
- A correction already issued. None found (see Status).
- Side observation, not part of the finding. The reported vitreous volume of distribution, 4.78 mL, does not follow from the Methods formula V = CL/kel with either half-life. With CL = 0.44 mL/day it gives 2.12 mL for 3.34 days and 2.87 mL for 4.52 days. Reaching 4.78 mL would need a half-life of about 7.5 days (7.53 days). By comparison, the ELISA values from ref. 5 that the Discussion quotes (CL 0.594 mL/day, V 2.75 mL) imply 3.21 days, within the NHP range. The software may have used a different AUC internally (for example AUC extrapolated to infinity), so I do not treat this as an error, and it does not help decide which half-life belongs to which compartment. Separately, the "threefold vitreous:retina ratio" in the Discussion corresponds to an AUC ratio of 2.59 (0.57 / 0.22) and a peak-concentration ratio of 3.57 (0.1 / 0.028). This is within the looseness of the word "threefold".
Status
- Version of record. I downloaded the publisher's article page (https://www.nature.com/articles/s42003-019-0731-9). Its abstract contains "half-lives of 3.34 and 4.52 days at the retina and in the vitreous, respectively", and its Results and Discussion contain the passages quoted above, unchanged. The PMC HTML, the Europe PMC full text and the PubMed abstract (PMID 31925329) carry the same text. I checked every quotation verbatim against the PMC-archived text with
src/show.py. Figure 6a was measured in the publisher's full-size image of Figure 6. - Supplements. The publisher lists two supplementary files. The Supplementary Information (
42003_2019_731_MOESM1_ESM.pdf, 5 pages; Supplementary Figs. 1–4: dye comparison, free-dye localization, injection sites, calibration curves) contains no half-life, AUC or other pharmacokinetic values. The Reporting Summary (MOESM2) is a checklist. There are no source data. The Data availability statement says the datasets are available from the corresponding author on reasonable request. The supplements cannot settle the discrepancy, but the main-text Figure 6a does. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, PMC notices or Europe PMC notices. The Crossref record has no update-to or updated-by entries, the PubMed record has no CommentsCorrections entries, the Europe PMC record lists no comments or corrections, and the publisher's page shows no correction notice. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC6949284; https://www.nature.com/articles/s42003-019-0731-9 and its Figure 6 image and supplementary files; Europe PMC; PubMed (PMID 31925329); Crossref; PubPeer; https://doi.org/10.1038/s42003-019-0731-9.
Suggested correction
In the abstract, replace
observed half-lives of 3.34 and 4.52 days at the retina and in the vitreous, respectively
with
observed half-lives of 4.52 and 3.34 days at the retina and in the vitreous, respectively
which matches the Results, the Discussion and the fitted lines in Figure 6a. The authors may also wish to state how the vitreous volume of distribution of 4.78 mL was obtained. The Methods formula V = CL/kel gives about 2.1 mL with the reported clearance (0.44 mL/day) and vitreous half-life (3.34 days).
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC6949284
(Commun Biol 2020, doi:10.1038/s42003-019-0731-9,
"Non-invasive molecular tracking method that measures ocular drug distribution
in non-human primates").
Run: python recompute.py.txt (uses the stored Figure 6a pixel measurements)
python recompute.py.txt Fig6_HTML.png (re-measures them from the publisher's image)
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract: "observed half-lives of 3.34 and 4.52 days at the retina and in the vitreous,
respectively"
Results: retina half-life 4.52 days, AUC 0.22 day x mg/mL; vitreous half-life 3.34 days,
AUC 0.57 day x mg/mL, clearance 0.44 mL/day, volume of distribution 4.78 mL;
maximal concentrations 0.1 (vitreous) and 0.028 (retina) mg/mL;
free dye half-lives 2.68 (vitreous) and 2.66 (retina) days
Discussion: vitreous 3.34 days "consistent with" NHP literature 2.6-4 days; retina
"higher than reported elsewhere (4.52 days as compared to 2.6-2.3 days from ref. 5)";
ELISA (ref. 5) clearance 0.594 mL/day and volume 2.75 mL; "threefold vitreous:retina ratio"
Methods: dose 0.25 mg; CL = dose/AUC; V = CL/kel; kel from the terminal slope.
Figure 6a pixel measurements: the green regression lines and the axis ticks of the
publisher's full-size Figure 6 image (42003_2019_731_Fig6_HTML.png, 1416 x 1335 px).
"""
import math
import os
import sys
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
LN2 = math.log(2)
# ---------------------------------------------------------------- inputs
ABSTRACT = {"retina": 3.34, "vitreous": 4.52}
RESULTS = {"retina": 4.52, "vitreous": 3.34}
DISCUSSION = {"retina": 4.52, "vitreous": 3.34}
DOSE = 0.25 # mg
AUC = {"retina": 0.22, "vitreous": 0.57} # day x mg/mL
CMAX = {"retina": 0.028, "vitreous": 0.1} # mg/mL
CL_REP, V_REP = 0.44, 4.78 # vitreous, mL/day and mL
FREE_DYE = {"vitreous": 2.68, "retina": 2.66}
NHP_RANGE = (2.6, 4.0) # Discussion, vitreous comparison
REF5_RETINA = (2.3, 2.6) # Discussion, "2.6-2.3 days from ref. 5"
ELISA_CL, ELISA_V = 0.594, 2.75 # Discussion, ref. 5
# Figure 6a, stored pixel measurements (see measure_fig6 below for how they were taken)
FIG6 = {
# decade ticks 0.1, 0.01, 0.001 (y px); x ticks day 0 and day 30 (x px); line slope (dy/dx px)
"vitreous": dict(dec=(215.0, 348.5, 479.5), x0=136.5, x30=662.5, slope=0.68964),
"retina": dict(dec=(224.5, 357.5, 488.5), x0=877.5, x30=1403.5, slope=0.49823),
}
def measure_fig6(path):
"""Fit the green regression line in each Figure 6a panel and read the axis ticks."""
import matplotlib.image as mpimg
im = (mpimg.imread(path)[:, :, :3] * 255).astype(int)
r, g, b = im[:, :, 0], im[:, :, 1], im[:, :, 2]
green = (g > 100) & (g - r > 40) & (g - b > 40)
dark = (r < 60) & (g < 60) & (b < 60)
out = {}
for name, (x0, x1) in {"vitreous": (130, 680), "retina": (870, 1416)}.items():
ys, xs = np.nonzero(green[:700, x0:x1]); xs = xs + x0
ux = np.unique(xs); uy = np.array([ys[xs == u].mean() for u in ux])
slope = np.polyfit(ux, uy, 1)[0]
ax = x0 + int(np.argmax(dark[:700, x0:x0 + 60].sum(0)))
rows = np.nonzero(dark[:700, ax - 12:ax - 2].sum(1) >= 8)[0]
groups = []
for y in rows:
(groups[-1].append(y) if groups and y - groups[-1][-1] <= 2 else groups.append([y]))
ticks = [float(np.mean(gp)) for gp in groups]
# the long (major) ticks at 0.1, 0.01, 0.001 are the ones that sit just above each long gap
gaps = np.diff(ticks); major = [ticks[i + 1] for i in np.argsort(gaps)[-4:]]
major = sorted(major)[:3]
xa = 550 + int(np.argmax(dark[550:700, x0:x1].sum(1)))
xc = np.nonzero(dark[xa + 3:xa + 12, x0:x1].sum(0) >= 6)[0] + x0
xg = []
for x in xc:
(xg[-1].append(x) if xg and x - xg[-1][-1] <= 2 else xg.append([x]))
xt = [float(np.mean(gp)) for gp in xg]
out[name] = dict(dec=tuple(major), x0=xt[0], x30=xt[-1], slope=float(slope))
return out
if len(sys.argv) > 1:
FIG6 = measure_fig6(sys.argv[1])
print("Figure 6a re-measured from", sys.argv[1], FIG6)
# ---------------------------------------------------------------- 1. which value goes where
print("=== 1. Ranibizumab half-life (days) by section ===")
for c in ("retina", "vitreous"):
ok = ABSTRACT[c] == RESULTS[c] == DISCUSSION[c]
print(f"{c:9s} Abstract {ABSTRACT[c]:.2f} Results {RESULTS[c]:.2f} Discussion {DISCUSSION[c]:.2f} "
f"{'consistent' if ok else 'MISMATCH'}")
print(f"Abstract values = Results values with the labels exchanged: "
f"{ABSTRACT['retina'] == RESULTS['vitreous'] and ABSTRACT['vitreous'] == RESULTS['retina']}")
print(f"ratio of the two half-lives 4.52/3.34 = {4.52/3.34:.3f}")
# ---------------------------------------------------------------- 2. Figure 6a slopes
print()
print("=== 2. Half-life implied by the fitted lines in Figure 6a ===")
FIG_T = {}
for c, d in FIG6.items():
px_per_decade = (d["dec"][-1] - d["dec"][0]) / (len(d["dec"]) - 1)
px_per_day = (d["x30"] - d["x0"]) / 30
dec_per_day = d["slope"] * px_per_day / px_per_decade
kel = dec_per_day * math.log(10)
FIG_T[c] = LN2 / kel
print(f"{c:9s} {px_per_decade:.2f} px/decade, {px_per_day:.3f} px/day, line slope {dec_per_day:.4f} "
f"decades/day -> kel {kel:.3f}/day, t1/2 {FIG_T[c]:.2f} d "
f"(Results {RESULTS[c]:.2f}, Abstract {ABSTRACT[c]:.2f})")
print(f"{'':9s} {1/dec_per_day:.2f} days per tenfold fall; differs from Results value by "
f"{100*abs(FIG_T[c]-RESULTS[c])/RESULTS[c]:.2f}% and from Abstract value by "
f"{100*abs(FIG_T[c]-ABSTRACT[c])/ABSTRACT[c]:.1f}%")
# ---------------------------------------------------------------- 3. literature comparisons
print()
print("=== 3. The Discussion's literature comparisons under each assignment ===")
for lab, a in (("Results/Discussion", RESULTS), ("Abstract", ABSTRACT)):
v, r = a["vitreous"], a["retina"]
print(f"{lab:19s} vitreous {v:.2f} inside NHP range {NHP_RANGE}: {NHP_RANGE[0] <= v <= NHP_RANGE[1]}; "
f"retina {r:.2f} / ref.5 upper {REF5_RETINA[1]} = {r/REF5_RETINA[1]:.2f}x")
# ---------------------------------------------------------------- 4. vitreous PK identities
print()
print("=== 4. Vitreous clearance and volume (Methods: CL = dose/AUC, V = CL/kel) ===")
cl = DOSE / AUC["vitreous"]
print(f"CL = {DOSE}/{AUC['vitreous']} = {cl:.4f} mL/day (reported {CL_REP})")
for t in (3.34, 4.52):
kel = LN2 / t
print(f"t1/2 {t}: kel {kel:.4f}/day; V = {CL_REP}/kel = {CL_REP/kel:.2f} mL; with CL {cl:.4f}: "
f"{cl/kel:.2f} mL (reported {V_REP})")
print(f"t1/2 implied by V {V_REP} and CL {CL_REP}: {V_REP*LN2/CL_REP:.2f} d; with CL {cl:.4f}: {V_REP*LN2/cl:.2f} d")
print(f"ELISA (ref. 5): t1/2 implied by V {ELISA_V} and CL {ELISA_CL}: {ELISA_V*LN2/ELISA_CL:.2f} d")
cl_r = DOSE / AUC["retina"]
print(f"(for comparison, dose/AUC with the retina AUC would be {cl_r:.3f} mL/day)")
# ---------------------------------------------------------------- 5. side checks
print()
print("=== 5. Side checks ===")
print(f"vitreous:retina AUC ratio {AUC['vitreous']}/{AUC['retina']} = {AUC['vitreous']/AUC['retina']:.2f}; "
f"Cmax ratio {CMAX['vitreous']}/{CMAX['retina']} = {CMAX['vitreous']/CMAX['retina']:.2f}")
print(f"free dye half-lives (not involved): vitreous {FREE_DYE['vitreous']}, retina {FREE_DYE['retina']}")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# ---- Figure 1: the two passages side by side, with the swap drawn between them
fig = plt.figure(figsize=(7.2, 4.1))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.1); ax.axis("off")
def panel(x, w, title, sub, excerpt, rows, color):
ax.add_patch(FancyBboxPatch((x, 0.35), w, 3.35, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=GRID, lw=1.2))
ax.text(x + 0.15, 3.48, title, fontsize=12, fontweight="bold", va="center")
ax.text(x + 0.15, 3.18, sub, fontsize=10, color=INK2, va="center")
ax.text(x + 0.15, 2.95, excerpt, fontsize=10, color=INK2, va="top", style="italic", linespacing=1.35)
ys = {}
for i, (comp, val) in enumerate(rows):
y = 1.35 - i * 0.55
ax.text(x + 0.15, y, comp, fontsize=11, va="center")
ax.text(x + w - 0.15, y, f"{val:.2f} d", fontsize=12, fontweight="bold", color=color,
va="center", ha="right")
ys[comp] = y
return ys
L = panel(0.2, 2.75, "Abstract", "as printed",
"“…observed half-lives of 3.34\nand 4.52 days at the retina\nand in the vitreous,\nrespectively.”",
[("Retina", ABSTRACT["retina"]), ("Vitreous", ABSTRACT["vitreous"])], ORANGE)
R = panel(4.25, 2.75, "Results and Discussion", "both sections agree",
"“…4.52 days … in the retina.”\n“In the vitreous, … 3.34 days”\n"
"“…3.34 days in the vitreous”\n(Figure 6a slopes: retina {:.2f} d,\nvitreous {:.2f} d)".format(
FIG_T["retina"], FIG_T["vitreous"]),
[("Retina", RESULTS["retina"]), ("Vitreous", RESULTS["vitreous"])], BLUE)
for val in (3.34, 4.52):
yl = [L[c] for c in L if ABSTRACT[c.lower()] == val][0]
yr = [R[c] for c in R if RESULTS[c.lower()] == val][0]
ax.add_patch(FancyArrowPatch((2.98, yl), (4.22, yr), arrowstyle="-|>", mutation_scale=12,
lw=1.5, color=INK2, shrinkA=2, shrinkB=2))
ax.text(3.6, 1.95, "same two numbers,\nlabels exchanged", ha="center", va="bottom", fontsize=10,
color=INK, linespacing=1.3)
ax.text(3.6, 0.12, "Orange: abstract assignment. Blue: assignment in the body text, which Figure 6a supports.",
ha="center", va="center", fontsize=10, color=INK2)
save(fig, "fig1")
# ---- Figure 2: the two assignments against the literature ranges the Discussion uses
fig, ax = plt.subplots(figsize=(7.2, 3.7))
fig.subplots_adjust(left=0.13, right=0.97, top=0.95, bottom=0.36)
rows = {"Vitreous": 1.0, "Retina": 0.0}
bands = {"Vitreous": (NHP_RANGE, "NHP literature\n(Discussion), 2.6–4 d"),
"Retina": (REF5_RETINA, "ref. 5,\n2.3–2.6 d")}
for comp, y in rows.items():
(lo, hi), lab = bands[comp]
ax.add_patch(plt.Rectangle((lo, y - 0.22), hi - lo, 0.44, fc=GRID, ec="none", zorder=1))
ax.text(lo - 0.08, y + 0.0, lab, ha="right", va="center", fontsize=10, color=INK2)
c = comp.lower()
ax.plot([ABSTRACT[c]], [y + 0.1], "o", ms=10, mfc=ORANGE, mec=ORANGE, zorder=3)
ax.plot([RESULTS[c]], [y - 0.1], "o", ms=10, mfc=BLUE, mec=BLUE, zorder=3)
ax.plot([FIG_T[c]], [y - 0.1], "D", ms=8, mfc="none", mec=INK, mew=1.5, zorder=4)
ax.text(ABSTRACT[c] + 0.12, y + 0.14, f"{ABSTRACT[c]:.2f}", fontsize=10, color=ORANGE, va="center")
ax.text(max(RESULTS[c], FIG_T[c]) + 0.12, y - 0.14, f"{RESULTS[c]:.2f}", fontsize=10, color=BLUE,
va="center")
ax.set_yticks(list(rows.values())); ax.set_yticklabels(list(rows.keys()), fontsize=11, color=INK)
ax.set_ylim(-0.5, 1.5); ax.set_xlim(0, 6)
ax.set_xlabel("Ranibizumab half-life (days)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="o", ls="", ms=10, mfc=ORANGE, mec=ORANGE, label="Abstract (as printed)"),
Line2D([], [], marker="o", ls="", ms=10, mfc=BLUE, mec=BLUE, label="Results and Discussion"),
Line2D([], [], marker="D", ls="", ms=8, mfc="none", mec=INK, mew=1.5,
label="slope of Figure 6a fitted line"),
plt.Rectangle((0, 0), 1, 1, fc=GRID, ec="none", label="range the Discussion compares with")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False,
fontsize=10)
save(fig, "fig2")
print()
print("figures written to", HERE)
Headline TCF7L2 transethnic estimate is printed with β, SE and P values that cannot coexist
Diabetes Care (2023) · doi:10.2337/dc23-0243 · PMC10369131
Summary
The paper's headline result is a transethnic meta-analysis association between TCF7L2 rs7903146 and age at diagnosis of type 2 diabetes, with P = 2.4 × 10⁻¹². Two different estimates are printed with that P value. Table 2 gives β −0.05 and SE 0.08, which corresponds to P = 0.53. The Abstract and Results give β −0.436 and SE 0.02, which corresponds to P ≈ 10⁻¹⁰⁵. Neither pair fits the printed P value at any rounding of the printed digits. A fixed-effect pool of the six cohort rows that Table 2 itself prints gives β −0.430 (SE 0.064), P = 1.6 × 10⁻¹¹. This agrees with the Abstract's β and with the order of the printed P value, so the association itself is probably sound and the misreported values are the Table 2 row and the Abstract's SE. The same table also prints a European meta-analysis estimate (−0.02, SE 0.02) that disagrees with its own cohort rows (pooled −0.334, SE 0.070). That row makes the South Asian versus European contrast, which the paper emphasizes, look about 46-fold rather than about 2.8-fold. The transethnic CDKAL1 row in Table 2 has the same β/SE/P incompatibility.
What the paper reports
In the archived text quoted below, the superscript exponents are flattened ("10-12" is 10⁻¹²) and minus signs appear as hyphens. The publisher's XML has proper superscripts and U+2212 minus signs.
Abstract, Results:
The strongest genome-wide significant variants at chromosome 10q25.3 in TCF7L2 (rs7903146; P = 2.4 × 10-12, β = -0.436; SE 0.02) and chromosome 6p22.3 in CDKAL1 (rs9368219; P = 2.29 × 10-8; β = -0.053; SE 0.01) were directionally consistent across ethnic groups and present at similar frequencies
Results, "Transethnic Meta-analysis of GWAS for Age at T2D Diagnosis" (the SNP is printed as rs79603146 here):
Our meta-analysis revealed two previously known T2D loci at chromosome 10q25.2 near transcription factor 7-like 2 (TCF7L2) (rs79603146, P < 2.4 × 10-12, β = -0.436; SE 0.02; P-heterogeneity = 0.01)
Table 2 ("Summary statistics of the most significant SNPs from the meta-analysis"), rs7903146, last two rows. The columns are study cohort | β | SE | EAF | P:
Meta-analysis (Europeans) | -0.02 | 0.02 | 0.34 | 0.004
Transethnic meta-analysis | -0.05 | 0.08 | 0.35 | 2.4 × 10-12
Table 2, rs9368219 (CDKAL1), last row:
Transethnic meta-analysis | -0.05 | 0.23 | 0.21 | 2.3 × 10-08
Table 2 prints these cohort rows for rs7903146 (β, SE, P): DMDSC data freeze 1, −1.26, 0.20, 1.0 × 10⁻¹⁰. DMDSC data freeze 2, −0.40, 0.36, 2.7 × 10⁻³. UKBB (SAS), −0.33, 0.37, 0.3. Meta-analysis (SAS), −0.92, 0.16, 1.1 × 10⁻⁸. GoDARTS, −0.02, 0.17, 0.11. GoSHARE, −0.90, 0.26, 0.0008. UKBB (Europeans), −0.35, 0.08, 1.3 × 10⁻⁵.
Methods, "Ethnic-Specific Meta-analysis of GWAS":
We performed the meta-analyses using a fixed-effects method in METAL software (26), which assumes that the effect allele is the same for each study within an ancestry.
Results, same section as above, and Conclusions:
however, a large difference was observed in the size of the estimate of the effects between the South Indian and European cohorts
Interestingly, the effect size of the variants was much lower in the cohorts of European descent.
Interestingly, the well-established T2D signal at TCF7L2 was much more strongly associated with age at onset in the South Indian population compared with the European population, despite the allele frequency not differing between these ancestral groups.
What the paper's own numbers show
Method. In a fixed-effect meta-analysis of the kind the Methods describe (METAL), the reported P value is the two-sided normal P of z = β/SE. For every row of Table 2 and for the Abstract, I computed z and P from the printed β and SE and compared them with the printed P value. To allow for rounding, I took the full range of |β/SE| over the rounding interval of each printed β and SE. I then pooled the printed cohort rows by inverse-variance weighting (β = Σwᵢβᵢ / Σwᵢ, SE = (Σwᵢ)^−½, wᵢ = 1/SEᵢ²). This is the fixed-effect estimate METAL computes under its standard-error scheme. As a check of the method, the pool of the three South Asian WDR11 rows gives P = 2.46 × 10⁻⁸, which is the value printed in Supplementary Table 7. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Transethnic rs7903146, P from β/SE | Table 2 | β −0.05, SE 0.08, P 2.4 × 10⁻¹² | z = −0.625, P = 0.53 (95% CI for β −0.21 to 0.11) |
| Same, over the rounding of β and SE | Table 2 | β −0.05, SE 0.08 | |z| 0.53–0.73, P 0.46–0.60 |
| |z| that P = 2.4 × 10⁻¹² requires | Table 2; Abstract | P 2.4 × 10⁻¹² | |z| = 7.009. At β −0.05 this needs SE 0.0071; at SE 0.08 it needs |β| 0.561 |
| Transethnic rs7903146, P from β/SE | Abstract; Results | β −0.436, SE 0.02, P 2.4 × 10⁻¹² | z = −21.8, P = 2.3 × 10⁻¹⁰⁵. Over SE 0.015–0.025, the largest P is 5.8 × 10⁻⁶⁸ |
| SE that β −0.436 and P 2.4 × 10⁻¹² imply | Abstract | SE 0.02 | SE 0.062 (95% CI −0.558 to −0.314) |
| SE implied by Supplementary Table 6 | Supp. Table 6 (TransmetaAAO) | β −0.4338, P 2.4e-12, no SE | SE 0.062 |
| Fixed-effect pool of the six rs7903146 cohort rows | Table 2 | (transethnic row −0.05, SE 0.08) | β −0.430, SE 0.064, z −6.74, P 1.6 × 10⁻¹¹ |
| Pool of the printed SAS and European meta rows | Table 2 | (transethnic row −0.05, SE 0.08) | β −0.034, SE 0.020, P 0.088 |
| Largest possible SE for a fixed-effect pool | Table 2 | transethnic SE 0.08 | The pooled SE must be smaller than the smallest cohort SE, which is 0.08 (UKBB Europeans); the six-row pool gives 0.064 |
| South Asian meta, rs7903146 | Table 2 | −0.92, SE 0.16, P 1.1 × 10⁻⁸ | P from β/SE 8.9 × 10⁻⁹. Pool of the three SAS rows −0.924, SE 0.158, P 5.0 × 10⁻⁹ (consistent) |
| European meta, rs7903146 | Table 2 | −0.02, SE 0.02, P 0.004 | P from β/SE 0.32. Pool of the three European rows −0.334, SE 0.070, P 1.7 × 10⁻⁶ |
| South Asian/European effect ratio, rs7903146 | Table 2 | −0.92 vs −0.02 (46-fold) | −0.924 vs −0.334 (2.8-fold). Difference −0.590, z −3.42, P 6.3 × 10⁻⁴ |
| Transethnic rs9368219 (CDKAL1), P from β/SE | Table 2 | β −0.05, SE 0.23, P 2.3 × 10⁻⁸ | z = −0.22, P = 0.83. The printed P needs |z| = 5.59 |
| Transethnic rs9368219, P from β/SE | Abstract; Results | β −0.053, SE 0.01, P 2.29 × 10⁻⁸ | z = −5.30, P 1.2 × 10⁻⁷. Over SE 0.005–0.015, |z| is 3.5–10.7, so this is consistent within rounding |
| Table 2 rows whose P fits their β and SE | Table 2, all 27 rows | — | 15 rows fit within rounding. 8 rows miss by more than 0.5 in |z|: TCF7L2 DMDSC 2, GoDARTS, European meta and transethnic; CDKAL1 DMDSC 2, GoDARTS and transethnic; WDR11 GoDARTS |
| Sample sizes | Table 1; Abstract | SAS 7,835 and 1,268; European 6,302 and 18,596; total 34,001 | Strata sums 7,835, 1,268, 6,302 and 18,596; total 34,001 (all consistent) |
| Younger-onset UKBB Europeans, rs7903146 | Table 1 | −0.26, SE 0.01, P 0.015 | z = −26, P ≈ 5 × 10⁻¹⁴⁹. The same kind of mismatch, in a secondary table |
The two transethnic TCF7L2 versions are not near misses. The Table 2 row misses the printed P by 6.3 units of |z| beyond rounding, and the Abstract's version misses it by 10.4. All three printed numbers would fit each other with β ≈ −0.43 and SE ≈ 0.062. That combination is what the Abstract's β, the supplement's β and P, and the pool of Table 2's own cohort rows all point to. Only two printed numbers disagree with it: Table 2's β and SE (−0.05, 0.08) and the Abstract's SE (0.02).
Figure
Figure 1. Estimates for rs7903146 (TCF7L2), with 95% intervals (β ± 1.96 SE) on one β axis; the thin vertical line is β = 0. Orange shows the values as printed in Table 2 or the Abstract. Blue shows the fixed-effect inverse-variance pool of the cohort rows that Table 2 prints. The two right-hand columns give the printed P and the P that each β and SE imply. Bold marks where the two disagree. The Table 2 transethnic interval crosses zero although the printed P is 2.4 × 10⁻¹². The Abstract's interval, drawn from SE 0.02, is narrower than its marker. The pooled estimate (−0.430, SE 0.064) sits on the Abstract's β with a realistic interval. The South Asian row matches its pool. The European row (−0.02, SE 0.02) lies far from the pool of its own three cohorts (−0.334, SE 0.070).
Figure 2. For each of the 27 rows of Table 2, and for the Abstract's value for each SNP, the orange diamond is the |z| implied by the printed P value. The blue dot is |β/SE| from the printed β and SE. The pale blue band is the range of |β/SE| that rounding of the printed β and SE allows. The dashed line marks genome-wide significance (P = 5 × 10⁻⁸). Where the diamond and the dot coincide, the row is internally consistent, as it is for 15 of the 27 rows, including every South Asian meta-analysis row. Bold labels mark rows where the printed P lies more than 0.5 in |z| outside the rounding band. The two transethnic TCF7L2 versions and the transethnic CDKAL1 row show the largest gaps. The CDKAL1 Abstract value fits only because its SE is printed to one significant digit.
Why it matters
The TCF7L2 transethnic association is the first result in the Abstract. A reader who checks the numbers finds two incompatible estimates for it. One of them, the Table 2 row, has a 95% interval of −0.21 to 0.11 that includes no effect. The pool of the paper's own cohort rows suggests that the association is real and genome-wide significant (P ≈ 10⁻¹¹), with β about −0.43 per T allele. The Abstract's SE of 0.02 overstates its precision about threefold, and the Table 2 row misstates both the estimate and its precision. Readers who take the summary statistics forward into look-ups, replication power calculations, Mendelian randomization or later meta-analyses would carry those errors with them. Figure 1A of the paper is the overall meta-analysis forest plot for this SNP, so it may be affected as well. The figure is an image and I did not read values from it.
The paper's interpretive claims rest on the contrast between ancestries: "the effect size of the variants was much lower in the cohorts of European descent" and the TCF7L2 signal "was much more strongly associated with age at onset in the South Indian population". In Table 2, the printed European meta-analysis β (−0.02) is one forty-sixth of the South Asian β (−0.92). The paper's own European cohort rows pool to −0.334 (SE 0.070, P 1.7 × 10⁻⁶), and the South Asian pool is 2.8 times that. The direction of the claim survives, because the difference between the two pools is still clear (z −3.42, P 6 × 10⁻⁴). Its size, as Table 2 presents it, does not. Read at face value, the printed European row would also suggest that TCF7L2 has almost no effect on age at diagnosis in Europeans (P 0.32 from its own β and SE). The European cohort rows do not support that reading.
For CDKAL1, the Abstract's β/SE/P triple is consistent within the coarse rounding of SE 0.01, but the Table 2 transethnic row (−0.05, SE 0.23, P 2.3 × 10⁻⁸) is not, since it corresponds to P 0.83. Pooling the six CDKAL1 cohort rows does not resolve this. The GoDARTS row (−0.03, SE 0.02) carries 94% of the weight, and its own printed P (0.004) does not match its β and SE. So the transethnic CDKAL1 significance cannot be verified from the printed numbers.
Explanations considered
- Rounding of the printed β and SE. This does not explain the numbers. Over the full rounding interval, the Table 2 transethnic row gives P 0.46–0.60. The Abstract's SE of 0.02 (0.015–0.025) gives P no larger than 5.8 × 10⁻⁶⁸. The printed P needs |z| = 7.01.
- A different trait scale, for example β per SD of age at diagnosis. This does not explain the numbers. Rescaling the trait multiplies β and SE by the same factor, so z = −0.625 and P = 0.53 are unchanged.
- The P value comes from a different scheme, such as METAL's sample-size-weighted Z, or from a mixed-model test whose P is not exactly β/SE. This could explain small row-level differences, but not these. A sample-size-weighted Z built from the printed cohort P values and Table 1 sample sizes gives P 9.4 × 10⁻¹⁷, not 2.4 × 10⁻¹². Whatever produced the P value, the printed β and SE are a summary estimate in their own right, and a fixed-effect pool of six cohorts cannot have an SE of 0.08 when one cohort alone already has SE 0.08. The Methods specify fixed effects. A random-effects SE could be larger than 0.08, but a random-effects analysis would then give a P value far larger than 2.4 × 10⁻¹², not smaller.
- The transethnic analysis used different data from the cohort rows (for example a different SNP set or imputation). This is unlikely. The Methods describe transancestry meta-analyses "across all the study cohorts", and the pool of the printed cohort rows reproduces the Abstract's β (−0.430 vs −0.436) and the supplement's β (−0.4338) closely.
- Typesetting or conversion error in the archived text. This is excluded. The publisher's JATS XML (Europe PMC) carries the same values, with true minus signs and superscript exponents. The bioRxiv preprint's abstract (10.1101/2022.09.14.508063, v1) carries the same β −0.436, SE 0.02 and P 2.4e-12, according to the version check that preceded this write-up.
- The supplement gives the correct values. This is only partly so. Supplementary Table 6 gives β −0.4338 and P 2.4e-12 for the transethnic TCF7L2 result, which together imply SE 0.062, but it has no SE column. It therefore supports the Abstract's β and P, and it is consistent with the pooled reconstruction. It does not correct Table 2. It also gives a different South Asian TCF7L2 estimate (Supplementary Table 7: β −0.72, P 1.8e-08) from Table 2 (−0.92, P 1.1 × 10⁻⁸), which is a further unexplained inconsistency.
The pooled values above are a reconstruction from rounded published numbers. Several cohort rows are themselves internally inconsistent (8 of 27 Table 2 rows), so the reconstruction shows what the printed inputs imply. It does not replace the authors' analysis output.
Status
- Version of record. The publisher page (diabetesjournals.org/care/article/46/8/1515/151527) returned HTTP 403 to automated access, and the PMC web page showed a browser-verification screen. The publisher-supplied JATS XML in Europe PMC (PMC10369131/fullTextXML) opened and has the same values as the PMC archive copy for the Abstract, the Results sentence, Table 1 and Table 2. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. Figshare 10.2337/figshare.23063396 holds "Suppplement_Informations (1).pdf" and "Supplementary Tables 6-8.xlsx". I downloaded and read the spreadsheet for this write-up. Sheet TransmetaAAO gives TCF7L2 rs7903146 RAF 0.3428, P 2.4e-12, Het_P 0.007 and Beta −0.4338, and CDKAL1 rs9368219 P 2.291e-08, Het_P 0.0004 and Beta −0.057. Neither entry has an SE. Sheet MetaAnalysis_SAS gives TCF7L2 β −0.72 (P 1.8e-08), CDKAL1 β −0.92 (P 6.59e-08) and WDR11 β 1.44 (P 2.46e-08). The European sheet does not list TCF7L2 or CDKAL1. The earlier version check read the PDF, which has no SE for the transethnic estimate.
- Corrections and comments. As of 2026-10-24, I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref updates, PMC notices or Europe PMC notices. Crossref lists only the preprint and the figshare supplement as relations. Web searches for the title with "erratum", "correction" and "PubPeer", and a Retraction Watch site search, found nothing relevant. The PubPeer API required a Turnstile challenge and the Europe PMC search endpoint returned 503 errors, so neither could be checked directly. - Sources. PMC article PMC10369131; Europe PMC full-text XML for PMC10369131; figshare 10.2337/figshare.23063396; bioRxiv 10.1101/2022.09.14.508063; Crossref metadata for 10.2337/dc23-0243.
Suggested correction
- Replace the Table 2 transethnic rs7903146 row (β −0.05, SE 0.08) with the β and SE from the METAL output that produced P = 2.4 × 10⁻¹². The published inputs imply β ≈ −0.43 and SE ≈ 0.062–0.064.
- Correct the SE in the Abstract and in the Results (0.02), so that β, SE and P agree. Also correct the SNP identifier in the Results (rs79603146 should be rs7903146).
- Check the Table 2 European meta-analysis row for rs7903146 (−0.02, SE 0.02, P 0.004) against the analysis output. The three European cohort rows pool to about −0.33 (SE 0.07). Then revise the Results and Conclusions statements about the size of the South Asian versus European difference if needed.
- Check the Table 2 transethnic rs9368219 row (−0.05, SE 0.23, P 2.3 × 10⁻⁸) and the other rows whose P does not match their β and SE (TCF7L2 DMDSC 2 and GoDARTS; CDKAL1 DMDSC 2 and GoDARTS; WDR11 GoDARTS), as well as the Table 1 UKBB European younger-onset row (SE 0.01).
- Reconcile the South Asian TCF7L2 estimate between Table 2 (−0.92, P 1.1 × 10⁻⁸) and Supplementary Table 7 (−0.72, P 1.8e-08). Add an SE column to Supplementary Tables 6–8.
- Confirm that the overall estimate in Figure 1A matches the corrected values.
Recomputation script (python recompute.py.txt)
"""Recompute the summary statistics of PMC10369131
(Diabetes Care 2023, doi:10.2337/dc23-0243, GWAS meta-analysis of age at diagnosis of T2D
in South Indian and European cohorts).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied as printed from the paper as archived in PMC (and identical in the
publisher's JATS XML in Europe PMC):
Table 2 (all 27 rows: three SNPs x nine rows: beta, SE, P as printed)
Table 1 (younger-onset and older-onset rows, sample sizes)
Abstract / Results, "Transethnic Meta-analysis": TCF7L2 rs7903146 P = 2.4e-12, beta -0.436, SE 0.02;
CDKAL1 rs9368219 P = 2.29e-8, beta -0.053, SE 0.01
Abstract: WDR11 rs3011366 P = 3.255e-8, beta 1.44, SE 0.25 (Results text: SE 0.26)
Supplementary Table 6, sheet TransmetaAAO (figshare 10.2337/figshare.23063396):
TCF7L2 beta -0.4338, P 2.4e-12 (no SE column)
Methods: the meta-analyses are fixed-effects (METAL); with the STDERR scheme this is the
inverse-variance-weighted (IVW) estimate below, and the P value is the two-sided Wald P of beta/SE.
"""
import os
from decimal import Decimal
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (as printed)
# (row label, beta, SE, P) - kept as strings so that rounding ranges can be derived
T2 = {
"rs7903146 (TCF7L2)": [
("DMDSC data freeze 1", "-1.26", "0.20", "1.0e-10"),
("DMDSC data freeze 2", "-0.40", "0.36", "2.7e-03"),
("UKBB (SAS)", "-0.33", "0.37", "0.3"),
("Meta-analysis (SAS)", "-0.92", "0.16", "1.1e-08"),
("GoDARTS", "-0.02", "0.17", "0.11"),
("GoSHARE", "-0.90", "0.26", "0.0008"),
("UKBB (Europeans)", "-0.35", "0.08", "1.3e-05"),
("Meta-analysis (Europeans)", "-0.02", "0.02", "0.004"),
("Transethnic meta-analysis", "-0.05", "0.08", "2.4e-12"),
],
"rs9368219 (CDKAL1)": [
("DMDSC data freeze 1", "-1.20", "0.21", "4.3e-08"),
("DMDSC data freeze 2", "-0.38", "0.38", "6.1e-02"),
("UKBB (SAS)", "-0.59", "0.40", "0.14"),
("Meta-analysis (SAS)", "-0.92", "0.17", "6.6e-08"),
("GoDARTS", "-0.03", "0.02", "0.004"),
("GoSHARE", "-0.80", "0.30", "0.007"),
("UKBB (Europeans)", "-0.17", "0.09", "0.07"),
("Meta-analysis (Europeans)", "-0.05", "0.02", "0.03"),
("Transethnic meta-analysis", "-0.05", "0.23", "2.3e-08"),
],
"rs3011366 (WDR11)": [
("DMDSC data freeze 1", "1.35", "0.32", "3.1e-05"),
("DMDSC data freeze 2", "1.02", "0.57", "0.07"),
("UKBB (SAS)", "2.44", "0.68", "3.4e-04"),
("Meta-analysis (SAS)", "1.44", "0.26", "3.3e-08"),
("GoDARTS", "0.21", "0.11", "0.36"),
("GoSHARE", "-0.31", "1.09", "0.77"),
("UKBB (Europeans)", "-0.21", "0.41", "0.60"),
("Meta-analysis (Europeans)", "-0.09", "0.08", "0.20"),
("Transethnic meta-analysis", "0.21", "0.07", "0.01"),
],
}
ABSTRACT = { # (label, beta, SE, P) as printed in the Abstract (and Results text for the first two)
"rs7903146 (TCF7L2)": ("Abstract / Results text", "-0.436", "0.02", "2.4e-12"),
"rs9368219 (CDKAL1)": ("Abstract / Results text", "-0.053", "0.01", "2.29e-8"),
"rs3011366 (WDR11)": ("Abstract", "1.44", "0.25", "3.255e-8"),
}
SUPP_T6_TCF7L2 = (-0.4338, 2.4e-12) # Supplementary Table 6, TransmetaAAO: beta, P (no SE)
SAS = ["DMDSC data freeze 1", "DMDSC data freeze 2", "UKBB (SAS)"]
EUR = ["GoDARTS", "GoSHARE", "UKBB (Europeans)"]
# Table 1 sample sizes (younger onset, older onset) and Results text totals
N_T1 = {"DMDSC data freeze 1": (5191, 610), "DMDSC data freeze 2": (2102, 392), "UKBB (SAS)": (542, 266),
"GoDARTS": (768, 6231), "GoSHARE": (543, 3612), "UKBB (Europeans)": (4991, 8753)}
N_META_T1 = {"SAS": (7835, 1268), "EUR": (6302, 18596)}
# Table 1 rs7903146 rows (beta, SE, P as printed); "<" P values are kept as bounds
T1_TCF = {
"younger": [("DMDSC data freeze 1", "-1.10", "0.20", "8.6e-8"), ("DMDSC data freeze 2", "-0.62", "0.29", "0.03"),
("UKBB (SAS)", "-0.38", "0.35", "0.4"), ("Meta-analysis (SAS)", "-0.84", "0.15", "<0.0001"),
("GoDARTS", "-0.43", "0.26", "0.11"), ("GoSHARE", "-0.37", "0.35", "0.0008"),
("UKBB (Europeans)", "-0.26", "0.01", "0.015"), ("Meta-analysis (Europeans)", "-0.29", "0.08", "0.001")],
}
Z975 = stats.norm.isf(0.025)
def half_unit(s):
"""Half of the last printed digit, e.g. '-0.05' -> 0.005, '2.4e-12' -> 0.05e-12."""
d = Decimal(s.lstrip("<"))
return float(Decimal(1).scaleb(d.as_tuple().exponent)) / 2
def p_wald(b, se):
z = b / se
return z, 2 * stats.norm.sf(abs(z))
def z_of_p(p):
return stats.norm.isf(p / 2)
def z_range(bs, ses):
"""Range of |beta/SE| over the rounding intervals of the printed beta and SE."""
b, s = abs(float(bs)), float(ses)
hb, hs = half_unit(bs), half_unit(ses)
return max(b - hb, 0) / (s + hs), (b + hb) / (s - hs)
def zp_range(ps):
p, h = float(ps), half_unit(ps)
return z_of_p(p + h), z_of_p(p - h)
def ivw(rows):
b = np.array([float(r[1]) for r in rows])
s = np.array([float(r[2]) for r in rows])
w = 1 / s ** 2
bb = (w * b).sum() / w.sum()
ss = np.sqrt(1 / w.sum())
q = (w * (b - bb) ** 2).sum()
return dict(b=bb, se=ss, z=bb / ss, p=2 * stats.norm.sf(abs(bb / ss)), q=q, phet=stats.chi2.sf(q, len(b) - 1))
def fmtp(p):
if p >= 0.01:
return f"{p:.2f}"
if p >= 0.001:
return f"{p:.3f}"
m, e = f"{p:.1e}".split("e")
return f"{m}e{int(e)}"
# ------------------------------------------------------------------ 1. every Table 2 row
print("=== 1. Table 2 and Abstract: two-sided Wald P from the printed beta and SE vs the printed P ===")
print(" |z| range: over the rounding interval of the printed beta and SE; gap = how far the |z| implied by")
print(" the printed P (over its own rounding interval) lies outside that range")
ROWS = [] # for fig 2
for snp, rows in T2.items():
print(f" {snp}")
for lab, bs, ses, ps in rows + [ABSTRACT[snp]]:
b, se, p = float(bs), float(ses), float(ps)
z, pc = p_wald(b, se)
lo, hi = z_range(bs, ses)
zlo, zhi = zp_range(ps)
gap = max(zlo - hi, lo - zhi, 0)
ROWS.append(dict(snp=snp, lab=lab, b=b, se=se, p=p, z=abs(z), pc=pc, zlo=lo, zhi=hi, zp=z_of_p(p), gap=gap))
flag = " <-- inconsistent" if gap > 0.5 else (" (slightly outside)" if gap > 0 else "")
print(f" {lab:27s} beta {bs:>6s} SE {ses:>4s}: |z| = {abs(z):5.2f} [{lo:5.2f}-{hi:5.2f}], P = {pc:.2g}"
f" | printed P {ps:>8s} -> |z| = {z_of_p(p):5.2f}; gap {gap:4.2f}{flag}")
n_bad = sum(r["gap"] > 0.5 for r in ROWS if r["lab"] != "Abstract / Results text" and r["lab"] != "Abstract")
n_ok = sum(r["gap"] == 0 for r in ROWS if r["lab"] not in ("Abstract / Results text", "Abstract"))
print(f" Table 2: {n_ok} of 27 rows consistent within rounding; {n_bad} rows off by more than 0.5 in |z| beyond rounding")
# ------------------------------------------------------------------ 2. transethnic TCF7L2
print("\n=== 2. Transethnic TCF7L2 rs7903146 ===")
zt = z_of_p(2.4e-12)
print(f" |z| needed for two-sided P = 2.4e-12: {zt:.3f}")
z, p = p_wald(-0.05, 0.08)
print(f" Table 2 row beta -0.05, SE 0.08: z = {z:.3f}, P = {p:.2f}; 95% CI {-0.05 - Z975 * 0.08:.3f} to {-0.05 + Z975 * 0.08:.3f}")
lo, hi = z_range("-0.05", "0.08")
print(f" over rounding (beta -0.055..-0.045, SE 0.075..0.085): |z| {lo:.3f}-{hi:.3f}, "
f"P {2 * stats.norm.sf(hi):.2f}-{2 * stats.norm.sf(lo):.2f}")
print(f" SE that P = 2.4e-12 needs with beta -0.05: {0.05 / zt:.4f}; beta it needs with SE 0.08: {0.08 * zt:.3f}")
z, p = p_wald(-0.436, 0.02)
print(f" Abstract beta -0.436, SE 0.02: z = {z:.1f}, P = {p:.1e}; 95% CI {-0.436 - Z975 * 0.02:.3f} to {-0.436 + Z975 * 0.02:.3f}")
lo, hi = z_range("-0.436", "0.02")
print(f" over rounding (SE 0.015..0.025): |z| {lo:.1f}-{hi:.1f}, largest P {2 * stats.norm.sf(lo):.1e}")
se_req = 0.436 / zt
print(f" SE that P = 2.4e-12 needs with beta -0.436: {se_req:.4f}; 95% CI {-0.436 - Z975 * se_req:.3f} to {-0.436 + Z975 * se_req:.3f}")
print(f" Supplementary Table 6: beta {SUPP_T6_TCF7L2[0]}, P {SUPP_T6_TCF7L2[1]} -> implied SE {abs(SUPP_T6_TCF7L2[0]) / zt:.4f}")
d = {r[0]: r for r in T2["rs7903146 (TCF7L2)"]}
R = {}
R["all6"] = ivw([d[k] for k in SAS + EUR])
R["sas"] = ivw([d[k] for k in SAS])
R["eur"] = ivw([d[k] for k in EUR])
R["2meta"] = ivw([d["Meta-analysis (SAS)"], d["Meta-analysis (Europeans)"]])
for k, lab in [("all6", "six cohort rows"), ("sas", "three SAS cohort rows"), ("eur", "three European cohort rows"),
("2meta", "printed SAS and European meta rows")]:
r = R[k]
print(f" IVW of the {lab:34s}: beta {r['b']:.3f}, SE {r['se']:.3f}, z {r['z']:.2f}, P {r['p']:.2g}; "
f"95% CI {r['b'] - Z975 * r['se']:.3f} to {r['b'] + Z975 * r['se']:.3f}; Q {r['q']:.1f}, P-het {r['phet']:.2g}")
print(f" SAS meta printed -0.92 (0.16), P 1.1e-8: Wald P from printed beta/SE = {p_wald(-0.92, 0.16)[1]:.1e}")
print(f" EUR meta printed -0.02 (0.02), P 0.004: Wald P from printed beta/SE = {p_wald(-0.02, 0.02)[1]:.2f}")
print(f" ratio SAS/EUR effect: printed {0.92 / 0.02:.0f}-fold; from cohort rows {R['sas']['b'] / R['eur']['b']:.1f}-fold")
dz = (R["sas"]["b"] - R["eur"]["b"]) / np.hypot(R["sas"]["se"], R["eur"]["se"])
print(f" SAS minus EUR pooled from cohort rows: {R['sas']['b'] - R['eur']['b']:.3f}, z = {dz:.2f}, P = {2 * stats.norm.sf(abs(dz)):.1e}")
print(f" fixed-effect pooled SE cannot exceed the smallest cohort SE in the pool (UKBB Europeans 0.08); "
f"six-row pool SE = {R['all6']['se']:.3f}")
for f in (5, 10):
print(f" rescaling beta and SE by 1/{f}: z = {(-0.05 / f) / (0.08 / f):.3f} (unchanged)")
# sample-size-weighted Z (METAL SAMPLESIZE scheme) from the printed cohort P values and signs
NTOT = {"DMDSC data freeze 1": 5191 + 610, "DMDSC data freeze 2": 2102 + 392, "UKBB (SAS)": 542 + 266,
"GoDARTS": 768 + 6231, "GoSHARE": 543 + 3612, "UKBB (Europeans)": 4991 + 8753}
ks = SAS + EUR
zs = np.array([np.sign(float(d[k][1])) * z_of_p(float(d[k][3])) for k in ks])
w = np.sqrt([NTOT[k] for k in ks])
Zss = (w * zs).sum() / np.sqrt((w ** 2).sum())
print(f" sample-size-weighted Z from the printed cohort P values (N = Table 1 strata sums): "
f"Z = {Zss:.2f}, P = {2 * stats.norm.sf(abs(Zss)):.1e}")
# ------------------------------------------------------------------ 3. CDKAL1
print("\n=== 3. Transethnic CDKAL1 rs9368219 ===")
z, p = p_wald(-0.05, 0.23)
print(f" Table 2 row beta -0.05, SE 0.23: z = {z:.2f}, P = {p:.2f}; printed P 2.3e-8 needs |z| = {z_of_p(2.3e-8):.2f}")
z, p = p_wald(-0.053, 0.01)
lo, hi = z_range("-0.053", "0.01")
print(f" Abstract beta -0.053, SE 0.01: z = {z:.2f}, P = {p:.1e}; over SE rounding 0.005..0.015 |z| {lo:.1f}-{hi:.1f} "
f"(printed P 2.29e-8 needs |z| = {z_of_p(2.29e-8):.2f}: within rounding)")
dc = {r[0]: r for r in T2["rs9368219 (CDKAL1)"]}
r = ivw([dc[k] for k in SAS + EUR])
print(f" IVW of the six CDKAL1 cohort rows: beta {r['b']:.3f}, SE {r['se']:.3f}, P {r['p']:.2g} "
f"(GoDARTS row -0.03/0.02 carries {(1 / 0.02 ** 2) / sum(1 / float(dc[k][2]) ** 2 for k in SAS + EUR):.0%} of the weight)")
# ------------------------------------------------------------------ 4. WDR11 control
print("\n=== 4. WDR11 rs3011366 (control) ===")
dw = {r[0]: r for r in T2["rs3011366 (WDR11)"]}
r = ivw([dw[k] for k in SAS])
print(f" IVW of the three SAS cohort rows: beta {r['b']:.3f}, SE {r['se']:.3f}, P {r['p']:.3g} "
f"(Table 2 SAS meta: 1.44, 0.26, 3.3e-8; Supplementary Table 7: 1.44, P 2.46e-08)")
# ------------------------------------------------------------------ 5. Table 1 and sample sizes
print("\n=== 5. Table 1 ===")
print(f" younger-onset SAS N: {sum(N_T1[k][0] for k in SAS)} (printed {N_META_T1['SAS'][0]}); "
f"EUR N: {sum(N_T1[k][0] for k in EUR)} (printed {N_META_T1['EUR'][0]})")
print(f" older-onset SAS N: {sum(N_T1[k][1] for k in SAS)} (printed {N_META_T1['SAS'][1]}); "
f"EUR N: {sum(N_T1[k][1] for k in EUR)} (printed {N_META_T1['EUR'][1]})")
print(f" total N over all strata: {sum(a + b for a, b in N_T1.values())} (Abstract 34,001)")
for lab, bs, ses, ps in T1_TCF["younger"]:
z, pc = p_wald(float(bs), float(ses))
print(f" younger onset {lab:27s} beta {bs} SE {ses}: |z| = {abs(z):5.2f}, P = {pc:.2g} (printed {ps})")
dy = {r[0]: r for r in T1_TCF["younger"]}
r = ivw([dy[k] for k in EUR])
print(f" IVW of the three younger-onset European rows: beta {r['b']:.3f}, SE {r['se']:.4f} "
f"(printed meta -0.29, SE 0.08; the UKBB row's SE 0.01 dominates)")
r = ivw([dy[k] for k in SAS])
print(f" IVW of the three younger-onset SAS rows: beta {r['b']:.3f}, SE {r['se']:.3f}, P {r['p']:.2g} (printed -0.84, 0.15, <0.0001)")
# ================================================================== figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"axes.edgecolor": SEC, "xtick.color": SEC, "ytick.color": TXT, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "axes.linewidth": 0.8,
})
LW = 2 * 72 / 100 * 1.0 # 2 px at 100 px/in ~ 1.44 pt
MS = 8
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
print(f" wrote {name}.svg and {name}.png")
print("\n=== figures ===")
# ---------------- Figure 1: estimates and 95% CIs for rs7903146 (one beta axis)
from matplotlib.transforms import blended_transform_factory # noqa: E402
s2 = R["sas"]
e2 = R["eur"]
a6 = R["all6"]
F1 = [ # (group, label, beta, se, kind, printed P text, P from beta and SE)
("Transethnic", "Table 2 row", -0.05, 0.08, "rep", "2.4e-12", p_wald(-0.05, 0.08)[1]),
("Transethnic", "Abstract and Results", -0.436, 0.02, "rep", "2.4e-12", p_wald(-0.436, 0.02)[1]),
("Transethnic", "Pool of 6 cohort rows", a6["b"], a6["se"], "rec", "–", a6["p"]),
("South Asian", "Table 2 row", -0.92, 0.16, "rep", "1.1e-8", p_wald(-0.92, 0.16)[1]),
("South Asian", "Pool of 3 SAS rows", s2["b"], s2["se"], "rec", "–", s2["p"]),
("European", "Table 2 row", -0.02, 0.02, "rep", "0.004", p_wald(-0.02, 0.02)[1]),
("European", "Pool of 3 EUR rows", e2["b"], e2["se"], "rec", "–", e2["p"]),
]
GNAME = {"Transethnic": "Transethnic meta-analysis", "South Asian": "South Asian meta-analysis",
"European": "European meta-analysis"}
ypos, y, prev, gtop = [], 0.0, None, {}
for g, *_ in F1:
if g != prev:
y += 0.5 if prev is None else 1.3
gtop[g] = y - 0.85
ypos.append(y)
y += 1
prev = g
ymax = y
fig = plt.figure(figsize=(7.2, 5.4))
ax = fig.add_axes([0.27, 0.20, 0.29, 0.66])
ax.set_ylim(ymax + 0.35, -0.6)
tr = blended_transform_factory(fig.transFigure, ax.transData)
XL, XB, XP1, XP2 = 0.26, 0.585, 0.80, 0.925
for (g, lab, b, se, kind, prt, pc), yy in zip(F1, ypos):
c = ORANGE if kind == "rep" else BLUE
ax.plot([b - Z975 * se, b + Z975 * se], [yy, yy], color=c, lw=LW, solid_capstyle="butt", zorder=2)
ax.plot(b, yy, "o", ms=MS, color=c, mec=BG, mew=1.2, zorder=3)
ax.text(XL, yy, lab, transform=tr, ha="right", va="center", fontsize=10, color=TXT)
bt = f"{b:.3f} ({se:.3f})" if kind == "rec" else f"{b:g} ({se:g})"
ax.text(XB, yy, bt.replace("-", "−"), transform=tr, ha="left", va="center", fontsize=10, color=TXT)
ax.text(XP1, yy, prt, transform=tr, ha="center", va="center", fontsize=10, color=TXT)
ax.text(XP2, yy, fmtp(pc), transform=tr, ha="center", va="center", fontsize=10,
color=TXT, fontweight="bold" if (kind == "rep" and g != "South Asian") else "normal")
for g, yt in gtop.items():
ax.text(0.015, yt, GNAME[g], transform=tr, ha="left", va="center", fontsize=10.5, fontweight="bold", color=TXT)
hy = -0.35
ax.text(XB, hy, "β (SE)", transform=tr, ha="left", va="bottom", fontsize=10, color=SEC)
ax.text(XP1, hy, "printed\nP", transform=tr, ha="center", va="bottom", fontsize=10, color=SEC)
ax.text(XP2, hy, "P from\nβ and SE", transform=tr, ha="center", va="bottom", fontsize=10, color=SEC)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.set_xlim(-1.4, 0.2)
ax.set_yticks([])
ax.set_xticks([-1.0, -0.5, 0])
ax.set_xticklabels(["−1.0", "−0.5", "0"])
ax.grid(axis="x", color=GRID, lw=0.7, zorder=0)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.set_xlabel("β per T allele, with 95% CI (β ± 1.96 SE)", fontsize=10)
ax.text(0.02, ymax + 0.25, "β = 0", ha="left", va="bottom", fontsize=10, color=SEC)
fig.text(0.015, 0.975, "rs7903146 (TCF7L2): printed estimates vs pools of the paper's own cohort rows",
fontsize=11, fontweight="bold", ha="left", va="top")
h = [Line2D([], [], color=ORANGE, lw=LW, marker="o", ms=MS, mec=BG, label="As printed in the paper (Table 2, Abstract)"),
Line2D([], [], color=BLUE, lw=LW, marker="o", ms=MS, mec=BG,
label="Recomputed: fixed-effect inverse-variance pool of the cohort rows printed in Table 2")]
fig.legend(handles=h, loc="lower left", bbox_to_anchor=(0.005, 0.0), ncol=1, frameon=False, fontsize=10)
save(fig, "fig1")
# ---------------- Figure 2: |z| from printed P vs |beta/SE| for every Table 2 row and the Abstract
SHORT = {"DMDSC data freeze 1": "DMDSC 1", "DMDSC data freeze 2": "DMDSC 2", "UKBB (SAS)": "UKBB SAS",
"Meta-analysis (SAS)": "Meta SAS", "GoDARTS": "GoDARTS", "GoSHARE": "GoSHARE",
"UKBB (Europeans)": "UKBB EUR", "Meta-analysis (Europeans)": "Meta EUR",
"Transethnic meta-analysis": "Transethnic", "Abstract / Results text": "Abstract / text",
"Abstract": "Abstract"}
fig = plt.figure(figsize=(7.2, 9.0))
ax = fig.add_axes([0.25, 0.165, 0.71, 0.775])
yy, yt, ylab, gy = 0, [], [], {}
order = []
for snp in T2:
rs = [r for r in ROWS if r["snp"] == snp]
gy[snp] = yy
for r in rs:
order.append((yy, r))
yy += 1
yy += 1.2
tot = yy
for y0, r in order:
y = tot - y0
bad = r["gap"] > 0.5
ax.plot([r["zlo"], r["zhi"]], [y, y], color=BLUE, lw=5, alpha=0.28, solid_capstyle="butt", zorder=1)
ax.plot([r["z"], r["zp"]], [y, y], color=SEC, lw=0.8, zorder=1)
ax.plot(r["z"], y, "o", ms=MS, color=BLUE, mec=BG, mew=1.0, zorder=3)
ax.plot(r["zp"], y, "D", ms=MS - 1, color=ORANGE, mec=BG, mew=1.0, zorder=4)
ax.text(-0.4, y, SHORT[r["lab"]], ha="right", va="center", fontsize=10,
fontweight="bold" if bad else "normal", color=TXT, clip_on=False)
for snp, y0 in gy.items():
ax.text(-0.4, tot - y0 + 0.95, snp, ha="right", va="center", fontsize=10.5, fontweight="bold",
color=TXT, clip_on=False)
zg = z_of_p(5e-8)
ax.axvline(zg, color=SEC, lw=0.8, ls=(0, (4, 3)), zorder=0)
ax.text(zg + 0.2, 1.0, "P = 5e-8", ha="left", va="center", fontsize=10, color=SEC)
ax.set_xlim(0, 30)
ax.set_ylim(0.2, tot + 0.9)
ax.set_yticks([])
ax.grid(axis="x", color=GRID, lw=0.7, zorder=0)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.set_xlabel("|z| (two-sided normal test)", fontsize=10)
# direct labels for the transethnic rows that fail
for y0, r in order:
y = tot - y0
if r["lab"] == "Transethnic meta-analysis" and r["gap"] > 0.5:
ax.annotate(f"printed P {fmtp(r['p'])} · β/SE gives P = {fmtp(r['pc'])}", (r["zp"], y), xytext=(9, 0),
textcoords="offset points", fontsize=10, color=TXT, ha="left", va="center")
if r["snp"].startswith("rs7903146") and r["lab"] == "Abstract / Results text":
ax.annotate(f"printed P {fmtp(r['p'])}", (r["zp"], y), xytext=(0, -15), textcoords="offset points",
fontsize=10, color=TXT, ha="center", va="center")
ax.annotate(f"β/SE gives P = {fmtp(r['pc'])}", (r["z"], y), xytext=(0, -15), textcoords="offset points",
fontsize=10, color=TXT, ha="center", va="center")
fig.text(0.02, 0.985, "Printed P vs printed β and SE: all 27 rows of Table 2, and the Abstract",
fontsize=11, fontweight="bold", ha="left", va="top")
h = [Line2D([], [], color=ORANGE, lw=0, marker="D", ms=MS - 1, mec=BG, label="|z| implied by the printed P"),
Line2D([], [], color=BLUE, lw=0, marker="o", ms=MS, mec=BG, label="|z| = |β / SE| from the printed β and SE"),
Line2D([], [], color=BLUE, lw=5, alpha=0.28, label="range of |β / SE| allowed by rounding of β and SE")]
fig.legend(handles=h, loc="lower left", bbox_to_anchor=(0.01, 0.025), ncol=1, frameon=False, fontsize=10)
fig.text(0.025, 0.008, "bold row labels: printed P outside the rounding range by > 0.5 in |z|",
ha="left", va="bottom", fontsize=10, color=SEC)
save(fig, "fig2")
Correlations printed as significant in this insulin-clearance study are too small to reach P < 0.05
Diabetes Care (2023) · doi:10.2337/dc23-1267 · PMC10698223
Summary
The paper reports that insulin clearance correlates negatively with liver lipid content (HCL) in type 2 diabetes (T2D) and with HbA1c, each at "P < 0.05", but prints coefficients such as r = −0.004, −0.005 and −0.037 in a T2D group of at most 489 people and r = −0.058 and −0.127 in a control group of at most 167. At those sizes |r| must be at least 0.089 (T2D) or 0.152 (controls) to reach two-sided P < 0.05; the recomputed P values for nine correlations described as significant run from 0.10 to 0.99, and seven of them underlie statements in the abstract. These correlations are the only correlational evidence for the abstract's conclusion that clearance "correlates negatively with liver lipid content rather in T2D". The regression lines drawn in Fig. 3 are far steeper than the printed coefficients allow; the paper does not show whether the coefficients, their labels or the significance statements are wrong.
What the paper reports
Abstract, Results:
HCL negatively associated with ICIVGTT and ICMMT in T2D (r = -0.005 and r = -0.037) and CON (r = -0.127 and r = -0.058, all P < 0.05).
In T1D, ICCLAMP, ICIVGTT, and ICMMT correlated negatively with HbA1c (all P < 0.05).
Abstract, Conclusions:
Insulin clearance is reduced in both T1D and T2D within the first year after diagnosis but correlates negatively with liver lipid content rather in T2D.
Results, Correlation Analyses (T2D):
In T2D, ICCLAMP correlated negatively with HbA1c (r = -0.004; P < 0.05) (Fig. 3B) but not with HCL or M value (Supplementary Fig. 3A and C).
On the other hand, ICIVGTT correlated positively with M value (r = -0.178; P < 0.05) (Supplementary Fig. 3F) and negatively with HCL (r = -0.005; P < 0.05) (Fig. 3D). ICMMT correlated negatively with HCL (r = -0.037; P < 0.05) (Fig. 3E).
Results, Correlation Analyses (controls, CON):
Similarly, ICMMT correlated positively with whole-body insulin sensitivity (r = 0.047; P < 0.05) (Supplementary Fig. 3I) and negatively with HCL (r = -0.058; P < 0.05) (Supplementary Fig. 3G) in CON.
Research Design and Methods, Statistics:
Pearson or Spearman correlations were used for numerical variables.
A two-tailed P value <0.05 was considered to indicate a statistically significant difference or correlation.
Figure 3 legend (Supplementary Fig. 3 uses similar wording):
Statistically significant correlations are depicted with linear regression lines and the respective coefficients (r).
Conclusions section (the article's discussion), final paragraph:
3) correlates with liver lipid content rather in T2D and with impaired glycemic control rather in T1D
The group sizes are CON 167, T1D 306 and T2D 489 (Abstract; Table 1). The Supplementary Fig. 3 legends (figshare 10.2337/figshare.24156732) print the same coefficients next to each regression line. They also give the three T1D coefficients behind the abstract's "In T1D ... all P < 0.05": r = −0.001 (ICCLAMP), −0.002 (ICIVGTT) and −0.012 (ICMMT), each against HbA1c.
What the paper's own numbers show
Method. The usual test of a correlation coefficient is t = r·√((n − 2)/(1 − r²)) with n − 2 degrees of freedom. I computed the two-sided P for each printed r at the full group size, which is the largest n any analysis could have had and so gives the smallest possible P. A smaller analysis sample only makes P larger. For the HCL correlations I also used the smaller number of participants with an HCL measurement, taken from the per-bar n printed under Fig. 2 (participants with plus without steatosis). Fisher's z, the usual large-sample test for Spearman's ρ, gives P values that differ from these by at most 0.001. I also inverted the test to find the critical |r| at each n.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Critical |r| for two-sided P < 0.05 | Statistics ("two-tailed") | — | T2D (n = 489): 0.089; T1D (n = 306): 0.112; CON (n = 167): 0.152; T2D with HCL (n = 129–143): 0.164–0.173 |
| T2D ICIVGTT vs HCL | Abstract; Results; Fig. 3C; Supp. Fig. 3D | r = −0.005, P < 0.05 | P = 0.91 (n = 489); 0.95 (n = 129) |
| T2D ICMMT vs HCL | Abstract; Results; Fig. 3E; Supp. Fig. 3G | r = −0.037, P < 0.05 | P = 0.41 (n = 489); 0.66 (n = 143) |
| CON ICIVGTT vs HCL | Abstract; Results; Supp. Fig. 3D | r = −0.127, P < 0.05 | P = 0.10 (n = 167); 0.20 (n = 105) |
| CON ICMMT vs HCL | Abstract; Results; Supp. Fig. 3G | r = −0.058, P < 0.05 | P = 0.46 (n = 167); 0.53 (n = 120) |
| T2D ICCLAMP vs HbA1c | Results; Fig. 3B; Supp. Fig. 3B | r = −0.004, P < 0.05 | P = 0.93 (n = 489) |
| T1D ICCLAMP vs HbA1c | Abstract and Results ("all P < 0.05"); Fig. 3B; Supp. Fig. 3B | r = −0.001 | P = 0.99 (n = 306) |
| T1D ICIVGTT vs HbA1c | Abstract and Results ("all P < 0.05"); Fig. 3D; Supp. Fig. 3E | r = −0.002 | P = 0.97 (n = 306) |
| T1D ICMMT vs HbA1c | Abstract and Results ("all P < 0.05"); Fig. 3F (line drawn, no label); Supp. Fig. 3H | r = −0.012 | P = 0.83 (n = 306) |
| CON ICMMT vs M value | Results; Supp. Fig. 3I | r = 0.047, P < 0.05 | P = 0.55 (n = 167) |
| n needed for P < 0.05 | — | — | |r| = 0.004: 240,092; 0.005: 153,659; 0.037: 2,807; 0.058: 1,143; 0.127: 239 |
| For comparison: CON ICIVGTT vs M value | Abstract; Results; Supp. Fig. 3F | r = 0.199, P < 0.05 | P = 0.0099 (consistent) |
| For comparison: T2D ICIVGTT vs M value | Abstract (r = 0.178); Results (r = −0.178, called "positively") | P < 0.05 | P = 7.6 × 10⁻⁵ (consistent; the sign differs between Abstract and Results) |
| For comparison: CON ICMMT vs M value | Abstract (r = 0.176); Results and Supp. Fig. 3I (r = 0.047) | P < 0.05 | P = 0.023 for 0.176; 0.55 for 0.047 |
| For comparison: all groups ICIVGTT vs HbA1c | Results ("negatively associated") | r = 0.092, P < 0.05 | P = 0.0043 (significance consistent; the positive sign contradicts "negatively") |
The coefficients that do reach significance (r = 0.199, 0.178, 0.176, 0.092) show that the test applied here does not reject every correlation in the paper. Only the near-zero coefficients fail.
The plotted lines. Figure 3 draws the "significant" correlations as fitted lines. I measured those lines on the published image and converted the slope b to the coefficient it implies, r = b·SD(x)/SD(y), using the group SDs printed in Table 1 and the Abstract (details in recompute.py.txt). The lines are much steeper than the printed coefficients allow. For T2D ICCLAMP vs HbA1c the line implies r ≈ −0.27, against a printed −0.004. For T1D ICCLAMP vs HbA1c it implies ≈ −0.32 (printed −0.001), for T2D ICMMT vs HCL ≈ −0.18 (printed −0.037), and for T1D ICMMT vs HbA1c ≈ −0.26 (printed −0.012). These estimates are approximate: besides the pixel measurement, the conversion assumes an untransformed least-squares (Pearson) line, whereas the Methods also mention log transforms and Spearman correlations, and it uses whole-group SDs as stand-ins for the SDs of the unreported analysis samples. They suggest that the plotted data carry associations of moderate size, and that the printed numbers are not the correlation coefficients of those data.
Figure
Each row is one correlation that the paper describes as significant. The orange line and arrowhead show the paper's claim, P < 0.05. The filled blue dot is the two-sided P recomputed from the printed r at the full group size, which is the smallest P that coefficient could have. The open blue dot, for the liver-fat rows, is the same calculation at the number of participants with an HCL measurement (Fig. 2). The thin vertical line marks P = 0.05. The nine upper rows all fall to the right of it, between P = 0.10 and 0.99. The four lower rows are correlations from the same paper whose printed r does reach P < 0.05, shown for comparison.
For the four lines that could be converted (an SD is reported for both variables), the orange dot is the printed r and the open blue diamond is the r implied by the slope of the line drawn in Fig. 3 (slope × SDx/SDy, measured from the image, approximate). The gray band marks |r| values too small for P < 0.05: at the full group size for the HbA1c rows and at the HCL subset size for the HCL row. Every printed value lies inside the band, and every line lies outside it. The small blue squares test one possible explanation, discussed below: they show the |r| that each printed value would imply if it were a P value rather than r, at the Fig. 2 subset size.
Why it matters
- Abstract and title. The abstract's only correlational evidence for a liver-lipid link in T2D is "r = −0.005 and r = −0.037 ... P < 0.05". Its contrast with CON rests on r = −0.127 and −0.058, and these also fail (P ≥ 0.10 and P ≥ 0.46 even at the full group size). The abstract's statement that all three clearance measures correlate with HbA1c in T1D rests on r = −0.001, −0.002 and −0.012 (P = 0.83–0.99). The abstract conclusion ("correlates negatively with liver lipid content rather in T2D"), the final conclusion point 3, and the title's pairing of liver lipids with T2D and glycemic control with T1D all draw on these correlations.
- Results and Conclusions section. The Conclusions section (the article's discussion) repeats that "ICCLAMP negatively correlated with HbA1c also in T2D", which rests on r = −0.004.
- What is not affected. The main finding, lower clearance in recent-onset T1D and T2D than in CON, comes from group comparisons (Abstract; Fig. 1) and does not depend on these correlations. The steatosis comparison, lower ICMMT with steatosis in T2D and CON (Fig. 2D), also stands and independently points toward a liver-fat link.
- Assessment. The plotted lines suggest that real negative associations of moderate size may exist. The liver-lipid and HbA1c correlation findings are therefore unsupported as printed, not shown to be false. A reader cannot tell from the paper which associations are significant, how strong they are, or whether the T2D-versus-T1D contrast in the title holds.
Explanations considered
- Text-conversion artifact (lost digits, minus signs or superscripts). Ruled out. The same values appear in the Europe PMC JATS XML, on the live PMC page, in the PubMed abstract, and as text printed inside the published Fig. 3 and Supplementary Fig. 3 images.
- Smaller analysis samples (missing MRS, MMT or clamp data). Ruled out. A smaller n only raises P. The calculation already uses the largest possible n.
- Spearman instead of Pearson. Ruled out. The Methods allow either, but Fisher's z (the large-sample test for ρ) gives P values that differ by at most 0.001.
- One-tailed tests. Ruled out, and the Methods specify two-tailed tests. The one-tailed critical |r| is still 0.074 (T2D), 0.094 (T1D) and 0.128 (CON), above every failing coefficient.
- Adjusted (partial) correlations from regression models. Ruled out. A partial correlation is tested with fewer degrees of freedom, so its critical |r| is higher still. A coefficient of 0.005 cannot reach P < 0.05 at any n in this study, whatever the adjustment.
- The printed value is R² rather than r. Does not account for the numbers. √0.005 = 0.071 and √0.004 = 0.063 still give P = 0.12 and 0.16 at n = 489, and √0.001 to √0.012 give P = 0.06–0.58 in T1D. This reading also does not match the line slopes (≈ −0.27 against √0.004 = 0.063).
- The printed value is a regression slope rather than r. Accounts for one panel at most. The slope of the Fig. 3C line, about −0.0052 a.u. per % HCL, matches its label 0.005. The other lines do not match: −0.018 and −0.020 for labels 0.004 and 0.001 in Fig. 3B, −0.029 for 0.002 in Fig. 3D, and −0.0072 for 0.037 in Fig. 3E.
- The printed value is a P value rather than r. Consistent with the four measurable lines, but not confirmed. Read as P values at the Fig. 2 subset sizes, 0.004, 0.001, 0.037 and 0.012 imply |r| = 0.263, 0.315, 0.175 and 0.240, close to the line-implied 0.27, 0.32, 0.18 and 0.26. This fits only if each analysis used the participants with HCL data, and the paper does not report the n of any correlation. The reading also leaves the CON HCL values unexplained: read as P values, 0.127 and 0.058 would be non-significant, which contradicts "all P < 0.05". The minus signs printed in the text also do not fit P values.
None of these readings accounts for every printed number. Only the authors' analysis output can settle what each number is.
Status
As of 2026-10-24.
- Version of record. The publisher page (diabetesjournals.org, reached by DOI redirect) returned HTTP 403 to automated access, so its HTML and PDF were not read directly. Three other copies of the published article carry identical numbers in the Abstract and Results, with true minus signs (U+2212): the Europe PMC full-text XML, the PMC article page, and the PubMed abstract (PMID 37874983). The published Fig. 3 image (PMC dc231267f3.jpg) prints the labels "r=0.001", "r=0.004", "r=0.005", "r=0.002" and "r=0.037" next to downward-sloping fitted lines.
- Supplement. The figshare item 10.2337/figshare.24156732 (Supplementary Table 1 and Supplementary Figs. 1–3) was downloaded and checked. The Supplementary Fig. 3 legends print the same coefficients: −0.001 and −0.004 (B), −0.127 and −0.005 (D), −0.002 (E), 0.199 and 0.178 (F), −0.058 and −0.037 (G), −0.012 (H) and 0.047 (I). The supplement does not report an n or a P value for any correlation.
- Corrections and retractions. None found. I checked Crossref update records, PMC and Europe PMC correction/retraction notices, and the Europe PMC comment-correction list.
- PubPeer. No thread found. Direct queries to the PubPeer API and search page were refused (human verification required). A web search restricted to pubpeer.com found no entry for this DOI or title.
- Sources. doi:10.2337/dc23-1267; PMC10698223; PubMed 37874983; supplement on figshare.
Other inconsistencies in the same section, not needed for the main finding:
- The Abstract gives r = 0.178 for T2D ICIVGTT vs M value. The Results give r = −0.178 and call the correlation positive.
- The Abstract gives r = 0.176 for CON ICMMT vs M value. The Results and Supplementary Fig. 3I give r = 0.047.
- The Results say ICIVGTT was "negatively associated" with HbA1c across all groups, but print r = 0.092.
- The Results cite "Fig. 3D" for T2D ICIVGTT vs HCL. Per the Fig. 3 legend, that correlation is in panel C.
Suggested correction
- Report each correlation in full. For every correlation in the Abstract, Results, Fig. 3 and Supplementary Fig. 3, the authors could give the coefficient type (Pearson or Spearman, unadjusted or adjusted, and the covariates), the number of participants analyzed, the coefficient and the exact two-sided P value. A supplementary table would hold this best.
- Replace the figure labels. The labels in Fig. 3 and Supplementary Fig. 3 should be replaced with the actual coefficients.
- Remove failing claims. Any correlation that is not significant once recomputed should no longer be described as significant. In that case, the abstract's liver-lipid and T1D-HbA1c statements, the final conclusion point 3 and the title should be revised to rest on the group and steatosis comparisons.
- Fix the smaller inconsistencies. These are the sign of the T2D ICIVGTT–M value coefficient, the CON ICMMT–M value coefficient (0.176 vs 0.047), the word "negatively" with r = 0.092, and the Fig. 3 panel citation.
Recomputation script (python recompute.py.txt)
"""Recompute the significance of the correlations reported in PMC10698223
(Diabetes Care 2023, doi:10.2337/dc23-1267, "Reduced Insulin Clearance Differently
Relates to Increased Liver Lipid Content and Worse Glycemic Control in Recent-Onset
Type 2 and Type 1 Diabetes").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs, all from the paper as published:
Group sizes (Abstract, Table 1): CON 167, T1D 306, T2D 489.
Analysis sizes for the liver-fat (HCL) subsets: the per-bar n printed under Fig. 2
(participants without + with steatosis, which requires an HCL measurement).
Correlation coefficients: Abstract; Results, "Correlation Analyses"; the legend labels
of Supplementary Fig. 3 (figshare 10.2337/figshare.24156732), which also give the
T1D coefficients that the main text does not print.
Group SDs: Table 1 (HbA1c, HCL, M value) and Abstract (IC_CLAMP, IC_MMT).
Fitted-line end points: measured in pixels on the published Fig. 3 (PMC image
dc231267f3.jpg, 700 x 642 px). Axis ticks were located in the image and each
coloured regression line was fitted to its own pixels (RANSAC, then least squares);
the end points below are that fit converted to data units. They are approximate.
Statistics section: "Pearson or Spearman correlations ... A two-tailed P value <0.05".
"""
import os
import re
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
import matplotlib.transforms # noqa: E402,F401
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import Patch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
def p_from_r(r, n):
"""Two-sided P for a correlation r with n pairs (t test, df = n - 2)."""
t = r * np.sqrt((n - 2) / (1 - r * r))
return 2 * stats.t.sf(abs(t), n - 2)
def p_fisher(r, n):
"""Two-sided P from Fisher's z (large-sample, also usual for Spearman)."""
z = np.arctanh(r) * np.sqrt(n - 3)
return 2 * stats.norm.sf(abs(z))
def r_crit(n, alpha=0.05):
t = stats.t.isf(alpha / 2, n - 2)
return t / np.sqrt(n - 2 + t * t)
def r_from_p(p, n):
t = stats.t.isf(p / 2, n - 2)
return t / np.sqrt(n - 2 + t * t)
def n_needed(r, alpha=0.05):
n = 4
while p_from_r(r, n) >= alpha:
n = int(n * 1.01) + 1
lo, hi = max(4, int(n / 1.01) - 2), n
while lo < hi: # exact smallest n
mid = (lo + hi) // 2
if p_from_r(r, mid) < alpha:
hi = mid
else:
lo = mid + 1
return lo
# ------------------------------------------------------------------ inputs
N = {"CON": 167, "T1D": 306, "T2D": 489, "ALL": 167 + 306 + 489}
# Fig. 2 per-bar n (without steatosis + with steatosis)
FIG2_N = {
("CON", "M"): 112 + 22, ("T1D", "M"): 122 + 9, ("T2D", "M"): 66 + 87,
("CON", "CLAMP"): 82 + 16, ("T1D", "CLAMP"): 99 + 7, ("T2D", "CLAMP"): 51 + 67,
("CON", "IVGTT"): 89 + 16, ("T1D", "IVGTT"): 100 + 7, ("T2D", "IVGTT"): 54 + 75,
("CON", "MMT"): 101 + 19, ("T1D", "MMT"): 100 + 9, ("T2D", "MMT"): 62 + 81,
}
# (label, group, IC measure, partner, printed r, where printed, n for HCL subset or None)
CLAIMS = [
# stated P < 0.05 but printed r too small
("T2D IC_IVGTT vs HCL", "T2D", "IVGTT", "HCL", -0.005, "Abstract; Results; Supp. Fig. 3D; Fig. 3C", True),
("T2D IC_MMT vs HCL", "T2D", "MMT", "HCL", -0.037, "Abstract; Results; Supp. Fig. 3G; Fig. 3E", True),
("CON IC_IVGTT vs HCL", "CON", "IVGTT", "HCL", -0.127, "Abstract; Results; Supp. Fig. 3D", True),
("CON IC_MMT vs HCL", "CON", "MMT", "HCL", -0.058, "Abstract; Results; Supp. Fig. 3G", True),
("T2D IC_CLAMP vs HbA1c", "T2D", "CLAMP", "HbA1c", -0.004, "Results; Supp. Fig. 3B; Fig. 3B", False),
("T1D IC_CLAMP vs HbA1c", "T1D", "CLAMP", "HbA1c", -0.001, "Abstract and Results ('all P < 0.05'); Fig. 3B; Supp. Fig. 3B", False),
("T1D IC_IVGTT vs HbA1c", "T1D", "IVGTT", "HbA1c", -0.002, "Abstract and Results ('all P < 0.05'); Fig. 3D; Supp. Fig. 3E", False),
("T1D IC_MMT vs HbA1c", "T1D", "MMT", "HbA1c", -0.012, "Abstract and Results ('all P < 0.05'); Fig. 3F (line, no label); Supp. Fig. 3H", False),
("CON IC_MMT vs M value", "CON", "MMT", "M", 0.047, "Results; Supp. Fig. 3I", False),
]
CONSISTENT = [
("CON IC_IVGTT vs M value", "CON", "IVGTT", "M", 0.199, "Abstract; Results; Supp. Fig. 3F", False),
("T2D IC_IVGTT vs M value", "T2D", "IVGTT", "M", 0.178, "Abstract; Supp. Fig. 3F (Results: -0.178)", False),
("CON IC_MMT vs M (Abstract r)", "CON", "MMT", "M", 0.176, "Abstract", False),
("All IC_IVGTT vs HbA1c", "ALL", "IVGTT", "HbA1c", 0.092, "Results ('negatively')", False),
]
# ------------------------------------------------------------------ 1. critical r
print("=== 1. Two-sided critical |r| for P < 0.05 ===")
for g in ("T2D", "T1D", "CON", "ALL"):
print(f" {g}: n = {N[g]:4d} critical |r| = {r_crit(N[g]):.4f}")
for g in ("T2D", "T1D", "CON"):
print(f" {g}: one-tailed critical |r| (alpha 0.05) = {r_crit(N[g], 0.10):.4f}")
for k in (("T2D", "IVGTT"), ("T2D", "MMT"), ("CON", "IVGTT"), ("CON", "MMT")):
n = FIG2_N[k]
print(f" HCL subset {k[0]} IC_{k[1]} (Fig. 2 n = {n}): critical |r| = {r_crit(n):.4f}")
# ------------------------------------------------------------------ 2. P values
print("\n=== 2. Recomputed two-sided P for each printed r ===")
hdr = f" {'correlation':36s} {'r':>7s} {'n_max':>5s} {'P(t)':>8s} {'P(Fisher)':>9s} {'n_HCL':>6s} {'P(t,n_HCL)':>10s}"
rows = []
for block, items in (("stated P < 0.05", CLAIMS), ("consistent, for comparison", CONSISTENT)):
print(f" -- {block}")
print(hdr)
for lab, g, ic, partner, r, where, hcl in items:
n = N[g]
p, pf = p_from_r(r, n), p_fisher(r, n)
nh = FIG2_N[(g, ic)] if hcl else None
ph = p_from_r(r, nh) if hcl else None
rows.append(dict(lab=lab, r=r, n=n, p=p, nh=nh, ph=ph, ok=block.startswith("consistent")))
extra = f"{nh:6d} {ph:10.3f}" if hcl else f"{'':6s} {'':10s}"
print(f" {lab:36s} {r:7.3f} {n:5d} {p:8.4f} {pf:9.4f} {extra} [{where}]")
print("\n smallest n at which |r| would reach two-sided P < 0.05:")
for r in (0.001, 0.002, 0.004, 0.005, 0.012, 0.037, 0.047, 0.058, 0.127):
print(f" |r| = {r:.3f}: n >= {n_needed(r):,}")
# ------------------------------------------------------------------ 3. alternative readings
print("\n=== 3. Alternative readings of the printed numbers ===")
print(" (a) printed value is R^2: r = sqrt(value), P at the largest possible n")
for lab, g, ic, partner, r, where, hcl in CLAIMS:
rr = np.sqrt(abs(r))
print(f" {lab:24s} sqrt({abs(r):.3f}) = {rr:.3f} P(n={N[g]}) = {p_from_r(rr, N[g]):.4f}")
print(" (b) printed value is a P value: implied |r| (HCL rows at the Fig. 2 subset n, others at group size)")
for lab, g, ic, partner, r, where, hcl in CLAIMS:
n = FIG2_N[(g, ic)] if hcl else N[g]
print(f" {lab:24s} P = {abs(r):.3f} -> |r| = {r_from_p(abs(r), n):.3f} (n = {n})")
# ------------------------------------------------------------------ 4. fitted lines in Fig. 3
# (panel, group, IC SD [Abstract], partner SD [Table 1], x0, y0, x1, y1, printed label, label in Supp. Fig. 3)
SD = {("T1D", "CLAMP"): 0.07, ("T2D", "CLAMP"): 0.06, ("T1D", "MMT"): 0.35, ("T2D", "MMT"): 0.33,
("T1D", "HbA1c"): 1.1, ("T2D", "HbA1c"): 0.9, ("T2D", "HCL"): 8.4}
LINES = [
# panel, label, group, IC, partner, x0, y0, x1, y1, Fig. 3 label, Supp. Fig. 3 value, n for critical r
("3B", "T2D IC_CLAMP vs HbA1c", "T2D", "CLAMP", "HbA1c", 4.57, 0.245, 9.97, 0.147, "0.001 or 0.004", -0.004, 489),
("3B", "T1D IC_CLAMP vs HbA1c", "T1D", "CLAMP", "HbA1c", 4.49, 0.165, 10.00, 0.054, "0.001 or 0.004", -0.001, 306),
("3E", "T2D IC_MMT vs HCL", "T2D", "MMT", "HCL", 0.98, 1.067, 37.84, 0.803, "0.037", -0.037, FIG2_N[("T2D", "MMT")]),
("3F", "T1D IC_MMT vs HbA1c", "T1D", "MMT", "HbA1c", 4.49, 0.870, 9.89, 0.421, "(none)", -0.012, 306),
]
OTHER_SLOPES = [ # IC_IVGTT lines: no SD of IC_IVGTT is reported, so only the slope is given
("3C", "T2D IC_IVGTT vs HCL", 0.20, 0.474, 38.04, 0.277, "0.005"),
("3D", "T1D IC_IVGTT vs HbA1c", 4.54, 0.368, 9.97, 0.212, "0.002"),
]
print("\n=== 4. Fitted regression lines drawn in Fig. 3 (pixel-measured end points) ===")
print(" implied r = slope x SD(x) / SD(y), the identity for a least-squares line of y on x;")
print(" it assumes an untransformed Pearson least-squares line and uses whole-group SDs (Table 1,")
print(" Abstract) as stand-ins for the unreported analysis-sample SDs, so the result is approximate")
implied = []
for pan, lab, g, ic, partner, x0, y0, x1, y1, flab, sval, ncrit in LINES:
b = (y1 - y0) / (x1 - x0)
ri = b * SD[(g, partner)] / SD[(g, ic)]
implied.append(dict(lab=lab, pan=pan, printed=sval, implied=ri, rc=r_crit(ncrit), n=ncrit,
asp=-r_from_p(abs(sval), FIG2_N[(g, ic)])))
print(f" Fig. {pan} {lab:24s} slope = {b:+.4f} a.u. per unit; SDx = {SD[(g, partner)]}, SDy = {SD[(g, ic)]}"
f" -> implied r = {ri:+.3f}; P(n={ncrit}) = {p_from_r(ri, ncrit):.1e};"
f" printed r = {sval:+.3f} (Fig. 3 label {flab}); critical |r| = {r_crit(ncrit):.3f}")
print(" If the Supp. Fig. 3 value were a P value: implied |r| at the full group n and at the Fig. 2 IC subset n")
for pan, lab, g, ic, partner, x0, y0, x1, y1, flab, sval, ncrit in LINES:
nsub = FIG2_N[(g, ic)]
print(f" Fig. {pan} {lab:24s} P = {abs(sval):.3f}: |r| = {r_from_p(abs(sval), N[g]):.3f} (n = {N[g]}),"
f" {r_from_p(abs(sval), nsub):.3f} (n = {nsub})")
for pan, lab, x0, y0, x1, y1, flab in OTHER_SLOPES:
b = (y1 - y0) / (x1 - x0)
print(f" Fig. {pan} {lab:24s} slope = {b:+.4f} a.u. per unit (Fig. 3 label {flab})")
# ------------------------------------------------------------------ figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10, "axes.edgecolor": FG2, "axes.labelcolor": FG, "xtick.color": FG2,
"ytick.color": FG, "text.color": FG, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "font.family": "DejaVu Sans", "svg.fonttype": "path",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# ---- fig1: reported P < 0.05 vs recomputed P, log axis
def pretty(lab):
g, rest = lab.split(None, 1)
rest = re.sub(r"IC_(\w+)", lambda m: r"IC$_{\rm " + m.group(1) + "}$", rest)
return f"{g} {rest}"
fig, ax = plt.subplots(figsize=(7.2, 6.6))
fig.subplots_adjust(left=0.30, right=0.73, top=0.91, bottom=0.19)
XMAX = 1.6 # a little past P = 1 so markers near 1 are not clipped
XMIN = 1e-5
ys, y = [], 0.0
for i, row in enumerate(rows):
if i == len(CLAIMS):
y += 1.2 # gap before comparison rows
ys.append(y)
y += 1
ys = np.array(ys)
tr = matplotlib.transforms.blended_transform_factory(ax.transAxes, ax.transData)
for yy, row in zip(ys, rows):
ax.plot([XMIN, 0.05], [yy - 0.24] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot([0.05], [yy - 0.24], marker="<", color=ORANGE, ms=8, ls="none")
if row["ph"] is not None: # open dot on its own sub-row so it never hides behind the filled one
ax.plot([row["p"], row["ph"]], [yy + 0.04, yy + 0.34], color=BLUE, lw=1.2, alpha=0.6, clip_on=False)
ax.plot([row["ph"]], [yy + 0.34], marker="o", mfc=BG, mec=BLUE, mew=1.5, ms=8, ls="none",
clip_on=False, zorder=4)
ax.plot([row["p"]], [yy + 0.04], marker="o", color=BLUE, ms=8, ls="none", clip_on=False, zorder=5)
ax.text(1.04, yy, f"{row['r']:+.3f}", transform=tr, ha="left", va="center", fontsize=10, color=ORANGE)
if row["p"] >= 0.095:
ptxt = f"{row['p']:.2f}" # 0.10, not 0.1
elif row["p"] >= 0.001:
ptxt = f"{row['p']:.2g}"
else:
ptxt = f"{row['p']:.1e}"
ax.text(1.24, yy, ptxt, transform=tr, ha="left", va="center", fontsize=10, color=BLUE)
ax.text(1.04, -1.05, "printed\nr", transform=tr, ha="left", va="bottom", fontsize=10, color=FG2)
ax.text(1.24, -1.05, "recomputed\nP", transform=tr, ha="left", va="bottom", fontsize=10, color=FG2)
ax.axvline(0.05, color=FG2, lw=0.8)
ax.text(0.05 * 0.85, -1.05, "P = 0.05", ha="right", va="bottom", fontsize=10, color=FG2)
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.set_xticks([1e-5, 1e-4, 1e-3, 1e-2, 1e-1, 1])
ax.set_ylim(ys[-1] + 0.7, -1.1)
ax.set_yticks(ys)
ax.set_yticklabels([pretty(r["lab"]) for r in rows], fontsize=10)
ax.set_xlabel("Two-sided P value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
sep = (ys[len(CLAIMS) - 1] + ys[len(CLAIMS)]) / 2
# (no rule: the gap and the italic label separate the comparison rows)
ax.text(XMIN * 1.3, sep - 0.05, "for comparison: pairs that do reach P < 0.05", ha="left", va="center",
fontsize=10, color=FG2, style="italic", bbox=dict(fc=BG, ec="none", pad=1))
handles = [
Line2D([0], [0], color=ORANGE, lw=2, marker="<", ms=8, label="Reported in the paper: P < 0.05"),
Line2D([0], [0], color=BLUE, marker="o", ms=8, ls="none",
label="Recomputed from printed r at the full group size (smallest possible P)"),
Line2D([0], [0], marker="o", mfc=BG, mec=BLUE, mew=1.5, ms=8, ls="none",
label="Recomputed at the liver-fat (HCL) subset n given in Fig. 2"),
]
fig.legend(handles=handles, loc="lower left", bbox_to_anchor=(0.02, 0.0), ncol=1, frameon=False, fontsize=10)
save(fig, "fig1")
# ---- fig2: printed r vs r implied by the plotted regression line
fig, ax = plt.subplots(figsize=(7.2, 4.3))
yl = np.arange(len(implied))
for yy, d in zip(yl, implied):
ax.fill_betweenx([yy - 0.32, yy + 0.32], -d["rc"], d["rc"], color="#eeedea", lw=0)
ax.plot([d["printed"]], [yy], marker="o", color=ORANGE, ms=8, ls="none")
ax.plot([d["implied"]], [yy], marker="D", mfc=BG, mec=BLUE, mew=1.5, ms=8, ls="none")
ax.plot([d["asp"]], [yy + 0.2], marker="s", color=BLUE, ms=7, ls="none", alpha=0.85)
ax.plot([d["implied"], d["printed"]], [yy, yy], color=FG2, lw=0.8, ls=":")
ax.text(max(d["printed"], 0) + 0.016, yy, f"{d['printed']:+.3f}", color=ORANGE, fontsize=10, va="center")
ax.text(d["implied"] - 0.012, yy - 0.36, f"≈ {d['implied']:+.2f}", color=BLUE, fontsize=10, va="center", ha="right")
ax.axvline(0, color=FG2, lw=0.8)
ax.set_yticks(yl)
ax.set_yticklabels([f"Fig. {d['pan']} {pretty(d['lab'])}" for d in implied], fontsize=10)
ax.set_ylim(len(implied) - 0.4, -0.7)
ax.set_xlim(-0.40, 0.20)
ax.set_xlabel("Correlation coefficient r")
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [
Line2D([0], [0], marker="o", color=ORANGE, ms=8, ls="none", label="Printed r (Supp. Fig. 3 legend)"),
Line2D([0], [0], marker="D", mfc=BG, mec=BLUE, mew=1.5, ms=8, ls="none",
label="r implied by the line drawn in Fig. 3 (slope × SDx / SDy, approx.)"),
Line2D([0], [0], marker="s", color=BLUE, ms=7, ls="none", alpha=0.85,
label="−|r| if the printed value were a P value (Fig. 2 subset n)"),
Patch(color="#eeedea", label="|r| too small for P < 0.05 at that n"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.35, -0.22), frameon=False, fontsize=10, ncol=1)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Table 3 extrapolates VA costs with the 75% population share instead of the stated 5%
Diabetes Care (2009) · doi:10.2337/dc09-0670 · PMC2732170
Summary
The paper projects per-patient lifetime costs of Hurricane Katrina for three health care systems with the CORE Diabetes Model. It then scales them up to the diabetic adults of greater New Orleans (964,677 adults × 9.2%), with population shares stated as MCLNO 20%, VA 5% and TUHC 75%. In Table 3, the MCLNO and TUHC extrapolations reproduce exactly at their stated shares. All three VA extrapolations (direct, indirect, total), however, equal the per-patient effect times 0.75 × 964,677 × 9.2%, which is the TUHC share, not the stated 5%. The VA total is therefore 15 times too large ($260.1 million instead of $17.3 million). The population lifetime cost, the only dollar figure in the Abstract ("USD $504 million"), would be about $261.5 million (±10% range about $235–288 million) at the paper's own stated shares, not $504 million ($454–555 million).
What the paper reports
Methods, "Statistical analysis" (the extrapolation and its assumptions):
We further extrapolated the prevalence-based mean impact of an individual patient to the entire population in the greater New Orleans area using 2000 U.S. Census data.
MCLNO represented 20% of the population, VA represented 5%, and TUHC/privately insured represented 75%; the adult population affected in the greater New Orleans area was 964,677 adults; and an estimated diabetes prevalence of 9.2%
Table 3, footnotes to the "Extrapolated cost (US$)" column (they apply to every row):
Adult population (age ≥18 years): 964,677
Prevalence of diabetes in 2006 (Louisiana): 9.2%
Table 3, block headings: "MCLNO (20%)", "TUHC (75%)", "VA (5%)". Rows under the VA (5%) heading and the final row (columns: Effect | SD | P | Extrapolated cost | −10% variation | +10% variation):
Direct cost | 2,683 | 5,423 | <0.0001 | 178,587,759 | 160,728,983 | 196,446,535
Indirect cost | 1,224 | 2,850 | <0.0001 | 81,472,761 | 73,325,485 | 89,620,037
Total cost | 3,907 | 7,201 | <0.0001 | 260,060,520 | 234,054,468 | 286,066,572
Population cost | | | | 504,221,426 | 453,799,283 | 554,643,569
For comparison, the total-cost rows of the other two systems in Table 3:
Total cost | 5,243 | 8,664 | <0.0001 | 93,063,548 (MCLNO (20%) block)
Total cost | 2,270 | 6,927 | <0.0001 | 151,097,359 (TUHC (75%) block)
Abstract, "Results":
We estimate a lifetime cost of USD $504 million for the adult population affected, with the largest economic impact seen among MCLNO patients.
with the economic impact being quite substantial because of the large population size affected
Results, paragraph 3:
The largest economic impact was seen in MCLNO patients with $5,243 over their lifetime, followed by VA patients ($3,907), and privately insured TUHC patients ($2,270).
we estimated a lifetime cost of USD $504 million for the population affected by Hurricane Katrina. A 10% variation around the mean cost of each sample produced an estimate of lifetime costs of USD $454–$555 million due to the impact of Hurricane Katrina.
What the paper's own numbers show
Method. The diabetic adult population is N = 964,677 × 0.092 = 88,750.28. For each system and each cost row of Table 3, I divided the reported extrapolated cost by the per-patient effect. That gives the number of patients the effect was multiplied by, and dividing by N gives the population share actually used. I then recomputed each extrapolation at the share stated in the Methods and in the Table 3 heading, and summed the systems. I also checked that direct + indirect = total for every row, that the ±10% columns are 0.9× and 1.1× the extrapolated cost, and that the Population cost row is the sum of the three system totals. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Diabetic adults in the area | Methods; Table 3 footnotes | 964,677 adults; prevalence 9.2% | 88,750.28 |
| MCLNO patients implied (direct, indirect, total rows) | Table 3 | 62,160,699 / 3,502; 30,902,849 / 1,741; 93,063,548 / 5,243 | 17,750.06 each = 0.2000 × N (stated 20%) |
| TUHC patients implied (direct, indirect, total rows) | Table 3 | 122,475,392 / 1,840; 28,621,967 / 430; 151,097,359 / 2,270 | 66,562.71 each = 0.7500 × N (stated 75%) |
| VA patients implied (direct, indirect, total rows) | Table 3 | 178,587,759 / 2,683; 81,472,761 / 1,224; 260,060,520 / 3,907 | 66,562.71 each = 0.7500 × N (stated 5%: 4,437.51) |
| VA direct extrapolated cost | Table 3 | 178,587,759 | 11,905,851 at 5% (reported is 15.00×) |
| VA indirect extrapolated cost | Table 3 | 81,472,761 | 5,431,517 at 5% (15.00×) |
| VA total extrapolated cost | Table 3 | 260,060,520 | 17,337,368 at 5% (15.00×) |
| MCLNO and TUHC total extrapolated costs | Table 3 | 93,063,548; 151,097,359 | 93,063,548; 151,097,359 (reproduce to the dollar) |
| Population cost | Table 3; Abstract; Results | 504,221,426 ("USD $504 million") | Sum of the reported rows 504,221,427; unrounded with VA at 75% exactly 504,221,426.00. With VA at the stated 5%: 261,498,275 |
| ±10% range of the population cost | Table 3; Results | 453,799,283 – 554,643,569 ("USD $454–$555 million") | 0.9× and 1.1× of the reported total, as printed. With VA at 5%: 235,348,447 – 287,648,102 |
| Sum of population shares used | Methods vs Table 3 (implied) | 20% + 75% + 5% = 100% | 20% + 75% + 75% = 170%: 150,875 patients extrapolated vs 88,750 diabetic adults |
| Ranking of population-level costs by system | Table 3 | VA 260.1M > TUHC 151.1M > MCLNO 93.1M | With VA at 5%: TUHC 151.1M > MCLNO 93.1M > VA 17.3M |
| Overstatement of the population cost | derived | — | 504,221,426 − 261,498,275 = 242,723,151; reported / recomputed = 1.928 |
The internal arithmetic of Table 3 is otherwise consistent: per-patient direct + indirect equals total in every system (3,502 + 1,741 = 5,243; 1,840 + 430 = 2,270; 2,683 + 1,224 = 3,907), the extrapolated direct + indirect equals the extrapolated total, and every ±10% cell is 0.9× or 1.1× its extrapolated cost (to within $1). The only departure from the stated method is the multiplier in the three VA rows, and it carries through unchanged into the Population cost row, its ±10% range, the Abstract and the Results.
Figure
Figure 1. Extrapolated lifetime cost in US$ millions, drawn from zero on one axis. Orange bars are the values printed in Table 3. Blue bars are the per-patient effect from Table 3 multiplied by 964,677 × 9.2% × the share stated in the Methods and in the Table 3 heading (20%, 75%, 5%); the blue population bar is the sum of the three blue system bars. MCLNO and TUHC reproduce exactly. The VA row is 15 times the value at its stated 5% share, and the population cost falls from $504.2 million to $261.5 million.
Figure 2. The number of diabetic adults each system's per-patient cost is multiplied by. The orange bar is implied by the reported Table 3 costs (reported extrapolated cost ÷ per-patient effect, identical for the direct, indirect and total rows). The blue bar uses the stated shares. The vertical line is the whole diabetic adult population (964,677 × 9.2% = 88,750). The reported extrapolation counts 150,875 patients, 170% of the population, because the VA block uses the same 66,563 patients as the TUHC block rather than 4,438.
Why it matters
The population lifetime cost is the only dollar figure in the Abstract ("We estimate a lifetime cost of USD $504 million for the adult population affected") and is repeated with its sensitivity range in the Results. At the paper's own stated assumptions it is overstated about 1.93-fold: about $261.5 million (±10%: about $235–288 million) instead of $504 million ($454–555 million). The whole overstatement comes from the VA block, whose printed contribution ($260.1 million) would be $17.3 million at 5%. A reader citing the paper's economic burden estimate would be using a figure roughly twice what the described method produces.
The error also changes the relative picture at population level. As printed, Table 3 makes the VA the largest contributor to population cost (VA $260.1M > TUHC $151.1M > MCLNO $93.1M). At the stated shares the VA is the smallest (TUHC $151.1M > MCLNO $93.1M > VA $17.3M). The Abstract's phrase "with the largest economic impact seen among MCLNO patients" holds per patient (the Results give $5,243 vs $3,907 vs $2,270) and is unaffected.
What is not affected: the clinical results (changes in A1C, blood pressure and lipids, Table 2, and their P values), the per-patient CORE Diabetes Model estimates of life expectancy, QALE and costs in Table 3, the disparity conclusions, and the direction of the economic finding. A modelled population cost of about $261 million is still large, so the qualitative statement that the economic impact was substantial stands; its magnitude is roughly halved.
Explanations considered
- The VA costs were extrapolated to a different base population (for example, all veterans). This does not account for the numbers. The Methods describe one base for all three systems (964,677 adults × 9.2%) and one set of shares. The Table 3 footnotes apply to the whole column and name no veteran-specific population. And the VA multiplier is not some other population: it equals 0.75 × 964,677 × 0.092 to the cent (66,562.71), the TUHC multiplier.
- The VA and TUHC labels or shares are swapped. This does not account for the numbers. The TUHC rows reproduce exactly at 75%; TUHC at 5% would give $10,073,157, not $151,097,359. Only the VA block departs from its stated share.
- Rounding, units or a misread digit. This does not account for the numbers. The MCLNO and TUHC rows reproduce to the dollar, the gap in the VA rows is an exact factor of 15.00 (0.75 / 0.05) in all three rows, and the same factor runs through the ±10% columns and the Population cost row. A single transcription slip would not reproduce consistently across nine VA cells.
- The 5% is the error, and 75% was intended for the VA. This does not account for the numbers either. The 5% appears in the Methods, in the Table 3 heading "VA (5%)", and is the value that makes the shares sum to 100%. With 75% for the VA, the shares sum to 170% and the extrapolation counts 150,875 patients out of 88,750 diabetic adults, which cannot be a share of one population.
- A text-conversion artifact in the archived copy. This is ruled out. The same figures appear in the live PMC page and in the Europe PMC full-text XML; all are plain positive integers with thousands separators, so no minus signs, decimals, superscripts or shifted cells are involved.
- The population total was computed separately and only the VA rows are misprinted. This does not account for the numbers. The printed Population cost (504,221,426) equals the unrounded sum of the three system totals with the VA at 75% (504,221,426.00), so the total was built from the same 75% multiplier.
Status
- Version of record. The publisher's page (doi.org/10.2337/dc09-0670, which redirects to diabetesjournals.org, Diabetes Care 32(9):1632) and the publisher PDF returned HTTP 403 or a challenge page to automated access. I checked the publisher-deposited full text instead: the live PMC article page and the Europe PMC full-text XML. Both carry the Table 3 values, the "VA (5%)" heading, the Methods share statement and the Abstract figure exactly as quoted above. Every quote in this write-up was confirmed verbatim against the PMC-archived text.
- Supplements. The Europe PMC supplementary-files archive for the article holds only the images of Figures 1–3. The JATS XML has no supplementary-material element. The CORE Diabetes Model inputs and outputs and the population calculation are not published, so they could not be checked.
- Corrections and comments. As of 2026-10-23, no correction, erratum, retraction or expression of concern was found. Crossref lists no update relations for the DOI (including the updated-by field that carries Retraction Watch data). PMC and Europe PMC list no correction notices (PMID 19542210; no comment/correction links), and the PubMed record has no CommentsCorrections or Erratum entries. Europe PMC lists one 2011 letter to the editor (PMID 22190890) that cites the paper only in passing. Earlier web searches for an erratum or a PubPeer thread found none; the PubPeer search page and API returned HTTP 403 to automated access, so PubPeer could not be checked directly. A further web search on 2026-10-23 could not be run because the session's search quota was exhausted.
- Sources. PMC: https://pmc.ncbi.nlm.nih.gov/articles/PMC2732170/ ; Europe PMC full-text XML and REST records for PMID 19542210; Crossref REST API for 10.2337/dc09-0670; PubMed E-utilities record for PMID 19542210.
Suggested correction
- Recompute the VA extrapolated costs in Table 3 at the stated 5% share: direct about $11,905,851 (−10%/+10%: $10,715,266 / $13,096,436), indirect about $5,431,517 ($4,888,366 / $5,974,669), total about $17,337,368 ($15,603,631 / $19,071,105).
- Correct the Population cost row to about $261,498,275 (−10%/+10%: $235,348,447 / $287,648,102), and the Abstract and Results figures from "USD $504 million" and "USD $454–$555 million" to about $261 million and $235–$288 million.
- If the 75% multiplier for the VA was in fact intended, state the VA base population and its source, and explain how the shares relate to a single diabetic population of 88,750 adults; otherwise, the shares as stated (20/75/5) should govern all three blocks.
- While revising Table 3, the VA QALE entry (−1.208, larger in magnitude than the VA life-expectancy change of −0.264 and unlike the other two systems, where the QALE change is smaller than the life-expectancy change) may be worth confirming against the model output. This write-up does not establish it as an error.
Recomputation script (python recompute.py.txt)
"""Recompute the population-level cost extrapolation in Table 3 of PMC2732170
(Diabetes Care 2009;32(9):1632, doi:10.2337/dc09-0670).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Methods (Statistical analysis): population shares MCLNO 20%, VA 5%, TUHC 75%;
964,677 adults; diabetes prevalence 9.2%
Table 3: per-patient effects (US$) and extrapolated costs with -10%/+10% columns
Table 3 footnotes: 964,677 adults (age >=18 y); prevalence 9.2%
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (paper)
ADULTS = 964_677
PREV = 0.092
STATED_SHARE = {"MCLNO": 0.20, "TUHC": 0.75, "VA": 0.05} # Methods; Table 3 headings
# per-patient effect (US$): direct, indirect, total (Table 3, "Effect")
EFFECT = {"MCLNO": (3502, 1741, 5243), "TUHC": (1840, 430, 2270), "VA": (2683, 1224, 3907)}
# extrapolated cost (US$): direct, indirect, total (Table 3, "Extrapolated cost")
REPORTED = {
"MCLNO": (62_160_699, 30_902_849, 93_063_548),
"TUHC": (122_475_392, 28_621_967, 151_097_359),
"VA": (178_587_759, 81_472_761, 260_060_520),
}
# -10% and +10% columns (Table 3)
REPORTED_PM10 = {
"MCLNO": ((55_944_629, 68_376_769), (27_812_564, 33_993_134), (83_757_193, 102_369_903)),
"TUHC": ((110_227_853, 134_722_931), (25_759_770, 31_484_163), (135_987_623, 166_207_094)),
"VA": ((160_728_983, 196_446_535), (73_325_485, 89_620_037), (234_054_468, 286_066_572)),
}
REPORTED_POP = (504_221_426, 453_799_283, 554_643_569) # Table 3 "Population cost"
SYSTEMS = ("MCLNO", "TUHC", "VA")
PARTS = ("direct", "indirect", "total")
# ------------------------------------------------------------------ recomputation
N = ADULTS * PREV
print("=== 1. Base population ===")
print(f"diabetic adults = {ADULTS:,} x {PREV} = {N:,.2f}")
print(f"stated shares sum to {sum(STATED_SHARE.values()):.2f}")
print("\n=== 2. Patients implied by each reported extrapolated cost ===")
implied_share = {}
recomputed = {}
for s in SYSTEMS:
sh = STATED_SHARE[s]
print(f"\n[{s}] stated share {sh:.0%} -> {N * sh:,.1f} diabetic adults")
recomputed[s] = []
shares = []
for part, eff, rep in zip(PARTS, EFFECT[s], REPORTED[s]):
n_impl = rep / eff
shares.append(n_impl / N)
rec = eff * N * sh
recomputed[s].append(rec)
print(f" {part:8s}: reported {rep:>13,}; {rep:,} / {eff:,} = {n_impl:,.2f} patients"
f" = {n_impl / N:.4f} x N; at stated share: {eff:,} x {N * sh:,.2f} = {rec:,.0f}"
f" (reported / recomputed = {rep / rec:.2f})")
implied_share[s] = sum(shares) / 3
d, i, t = EFFECT[s]
rd, ri, rt = REPORTED[s]
print(f" checks: {d:,} + {i:,} = {d + i:,} (table {t:,}); "
f"{rd:,} + {ri:,} = {rd + ri:,} (table {rt:,})")
print("\n=== 3. The +/-10% columns are 0.9x and 1.1x the extrapolated cost ===")
for s in SYSTEMS:
for part, rep, (lo, hi) in zip(PARTS, REPORTED[s], REPORTED_PM10[s]):
print(f" {s:5s} {part:8s}: 0.9x = {0.9 * rep:,.0f} (table {lo:,}); "
f"1.1x = {1.1 * rep:,.0f} (table {hi:,})")
print("\n=== 4. Population totals ===")
sum_rep = sum(REPORTED[s][2] for s in SYSTEMS)
exact_rep = sum(EFFECT[s][2] * N * (0.75 if s == "VA" else STATED_SHARE[s]) for s in SYSTEMS)
print(f"sum of reported system totals = {sum_rep:,} (table {REPORTED_POP[0]:,}); "
f"unrounded with VA at 75% = {exact_rep:,.2f}")
print(f"reported +/-10%: 0.9x = {0.9 * REPORTED_POP[0]:,.0f}, 1.1x = {1.1 * REPORTED_POP[0]:,.0f} "
f"(table {REPORTED_POP[1]:,} - {REPORTED_POP[2]:,})")
va5 = recomputed["VA"][2]
corr = REPORTED["MCLNO"][2] + REPORTED["TUHC"][2] + va5
print(f"VA total at stated 5% = {va5:,.0f} (VA direct {recomputed['VA'][0]:,.0f}, "
f"indirect {recomputed['VA'][1]:,.0f})")
for part, rec in zip(PARTS, recomputed["VA"]):
print(f" VA {part:8s} at 5%: {rec:,.0f}; -10% {0.9 * rec:,.0f}; +10% {1.1 * rec:,.0f}")
print(f"population total with VA at 5% = 93,063,548 + 151,097,359 + {va5:,.0f} = {corr:,.0f}")
print(f"+/-10% with VA at 5%: {0.9 * corr:,.0f} - {1.1 * corr:,.0f}")
print(f"reported / corrected population total = {REPORTED_POP[0] / corr:.3f}")
print(f"overstatement = {REPORTED_POP[0] - corr:,.0f} "
f"(= VA reported total - VA at 5% = {REPORTED['VA'][2] - va5:,.0f})")
print("\n=== 5. Shares and patients actually used ===")
for s in SYSTEMS:
print(f" {s:5s}: implied share {implied_share[s]:.4f} (stated {STATED_SHARE[s]:.2f})")
used = sum(implied_share.values())
print(f"implied shares sum to {used:.4f}; patients extrapolated = {N * used:,.0f} "
f"vs {N:,.0f} diabetic adults ({used:.2f}x)")
print("\n=== 6. Alternative readings ===")
print(f"if the VA and TUHC labels were swapped, TUHC at 5% = 2,270 x {N * 0.05:,.2f} = "
f"{2270 * N * 0.05:,.0f} (table 151,097,359)")
print(f"VA multiplier / stated VA multiplier = 0.75 / 0.05 = {0.75 / 0.05:.0f}")
print("\n=== 7. Ranking of system totals ===")
rep_rank = sorted(SYSTEMS, key=lambda s: -REPORTED[s][2])
cor_tot = {"MCLNO": REPORTED["MCLNO"][2], "TUHC": REPORTED["TUHC"][2], "VA": va5}
cor_rank = sorted(SYSTEMS, key=lambda s: -cor_tot[s])
print("reported: " + " > ".join(f"{s} {REPORTED[s][2] / 1e6:.1f}M" for s in rep_rank))
print("VA at 5%: " + " > ".join(f"{s} {cor_tot[s] / 1e6:.1f}M" for s in cor_rank))
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK, "text.color": INK, "svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# fig1: reported vs recomputed totals, US$ millions, one axis from zero
rows = [("MCLNO (20%)", REPORTED["MCLNO"][2], recomputed["MCLNO"][2]),
("TUHC (75%)", REPORTED["TUHC"][2], recomputed["TUHC"][2]),
("VA (5%)", REPORTED["VA"][2], recomputed["VA"][2]),
("Population cost", REPORTED_POP[0], corr)]
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
ax.set_facecolor(BG)
h = 0.34
for k, (lab, rep, rec) in enumerate(rows):
y = len(rows) - 1 - k
ax.barh(y + h / 2 + 0.01, rep / 1e6, height=h, color=ORANGE, edgecolor=BG, linewidth=2)
ax.barh(y - h / 2 - 0.01, rec / 1e6, height=h, color=BLUE, edgecolor=BG, linewidth=2)
ax.text(rep / 1e6 + 6, y + h / 2, f"{rep / 1e6:,.1f}", va="center", color=INK, fontsize=10)
ax.text(rec / 1e6 + 6, y - h / 2, f"{rec / 1e6:,.1f}", va="center", color=INK, fontsize=10)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows][::-1])
ax.axhline(0.5, color=INK2, linewidth=0.8)
ax.set_xlim(0, 590)
ax.set_xlabel("Extrapolated lifetime cost, US$ millions (Table 3)")
ax.grid(axis="x", color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
from matplotlib.patches import Patch # noqa: E402
ax.legend(handles=[Patch(color=ORANGE, label="Reported in Table 3"),
Patch(color=BLUE, label="Recomputed at the stated share\n(effect x 964,677 x 9.2% x share)")],
loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
ax.text(REPORTED["VA"][2] / 1e6 + 55, 1 + h / 2 + 0.01,
"<- computed with the 75% share,\n not the stated 5% (15x)", va="center", color=INK2, fontsize=10)
fig.tight_layout()
save(fig, "fig1")
# fig2: diabetic adults counted in the extrapolation vs the diabetic adults available
fig, ax = plt.subplots(figsize=(7.2, 3.2), facecolor=BG)
ax.set_facecolor(BG)
bars = [("Implied by the\nreported costs", ORANGE,
[("MCLNO", N * implied_share["MCLNO"]), ("TUHC", N * implied_share["TUHC"]),
("VA", N * implied_share["VA"])]),
("Stated shares\n20% / 75% / 5%", BLUE,
[("MCLNO", N * 0.20), ("TUHC", N * 0.75), ("VA", N * 0.05)])]
for k, (lab, col, segs) in enumerate(bars):
y = 1 - k
left = 0
for name, v in segs:
ax.barh(y, v / 1e3, left=left / 1e3, height=0.55, color=col, edgecolor=BG, linewidth=2)
txt = f"{name}\n{v:,.0f}"
if v > 10_000:
ax.text((left + v / 2) / 1e3, y, txt, ha="center", va="center", color="white", fontsize=10)
else:
ax.text((left + v) / 1e3 + 1.5, y + 0.43, f"{name} {v:,.0f}", ha="left", va="center",
color=INK, fontsize=10)
left += v
ax.text(left / 1e3 + 1.5, y, f"{left:,.0f}\n({left / N:.0%})", va="center", color=INK, fontsize=10)
ax.axvline(N / 1e3, color=INK2, linewidth=1)
ax.text(N / 1e3 + 1.5, 1.68, f"all diabetic adults in the area\n964,677 x 9.2% = {N:,.0f}",
color=INK2, fontsize=10, va="center")
ax.set_yticks([1, 0])
ax.set_yticklabels([b[0] for b in bars])
ax.set_ylim(-0.5, 1.95)
ax.set_xlim(0, 175)
ax.set_xlabel("Diabetic adults the costs are multiplied by (thousands)")
ax.grid(axis="x", color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Abstract and Table 3 give a myocardial infarction confidence interval that excludes its own estimate
Diabetes Care (2009) · doi:10.2337/dc09-1297 · PMC2782995
Summary
The abstract and Table 3 report the ratio of relative risks (RRs) of aspirin for myocardial infarction (MI), in patients with diabetes versus without, as 1.19 with a 95% CI of 0.82–1.17. The upper limit, 1.17, is below the point estimate, 1.19, so the interval cannot be right as printed. The paper's stated Altman–Bland method, applied to the subgroup RRs printed in the same table, gives 1.19 (95% CI 0.82–1.73), P = 0.35. The same calculation reproduces the mortality and ischemic stroke rows exactly. The corrected interval still includes 1, so the paper's conclusion of no significant interaction stands. But the printed interval is about half as wide as it should be on the log scale, and it makes a headline result look much more precise than the data allow. The 2018 erratum to this article does not address the interval.
What the paper reports
Abstract, Results:
The ratios of RRs comparing the benefit of aspirin among patients with diabetes compared with patients without diabetes for mortality, myocardial infarction, and ischemic stroke were 1.12 (95% CI 0.92–1.35), 1.19 (0.82–1.17), and 0.70 (0.25–1.97), respectively.
Table 3, "Aspirin-diabetes subgroup interaction analyses", MI row. The columns are RR with diabetes | RR without diabetes | P interaction | Ratio of RRs pooled across trials† | Ratio of RRs pooled within trials‡ | Ratio of RRs Bayesian analyses§:
Myocardial infarction | 0.86 (0.67–1.11) | 0.72 (0.55–0.95) | 0.36 | 1.19 (0.82–1.17) | 1.02 (0.71–1.47) | 1.06 (0.16–6.71)
Table 3, footnote † (the column that holds the error):
Ratios of RRs in this method are pooled across randomized controlled trials and subgroups of randomized controlled trials of patients with diabetes versus without diabetes; then a ratio of pooled estimates is estimated. Numbers in parentheses represent 95% CIs, which indicate a significant interaction if it excludes 1.
Research Design and Methods, "Interaction between the effect of aspirin and diabetes status":
First, we estimated the ratio of RRs and its 95% CI using the method of Altman and Bland (13). This estimate compares the pooled RRs across subgroups of patients with diabetes and without across trials.
Results, "Prevention of cardiovascular events with aspirin" (the subgroup RRs that feed the ratio):
Estimates among patients with diabetes were 0.97 (0.87–1.08; I2 0%), 0.86 (0.67–1.11; I2 53%), and 0.62 (0.31–1.24; I2 67%), respectively. The corresponding estimates among patients without diabetes were 0.87 (0.75–1.02; I2 0%), 0.72 (0.55–0.95; I2 70%), and 0.89 (0.41–1.94; I2 0%), respectively.
Abstract, Conclusions:
Whereas estimates of benefit among patients with diabetes remain imprecise, our analysis suggests that the relative benefit of aspirin is similar in patients with and without diabetes.
What the paper's own numbers show
Method. Altman and Bland (BMJ 2003;326:219) compare two independent estimates on the log scale. The ratio is RR₁/RR₂, and SE(ln ratio) = √(SE₁² + SE₂²), where each SE comes from its published 95% CI as (ln upper − ln lower)/(2 × 1.96). I applied this to the "RR with diabetes" and "RR without diabetes" columns of Table 3 for all three outcomes. As cross-checks, I also computed (a) the upper limit that is log-symmetric with the printed lower limit about the printed point, (b) the interval implied by the printed P interaction, and (c) the ratio from the MI subgroup subtotals in the corrected Fig. 2B published in the 2018 erratum. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| MI ratio of RRs, point | Abstract; Table 3, across-trials column | 1.19 | 1.194 (0.86/0.72) |
| MI ratio of RRs, 95% CI | Abstract; Table 3, across-trials column | 0.82–1.17 | 0.82–1.73 (0.823–1.733) |
| MI two-sided P for the ratio | Table 3, P interaction | 0.36 | 0.35 |
| MI SE(ln ratio) | derived | 0.091 (implied by 0.82–1.17) | 0.190 |
| Log-scale midpoint of the printed MI CI | derived | should equal the point, 1.19 | 0.979 |
| Upper limit log-symmetric to printed 0.82 about 1.19 | derived | — | 1.727 |
| 95% CI implied by printed P = 0.36 and point 1.19 | derived | — | 0.82–1.73 |
| MI ratio from 2018 erratum Fig. 2B subtotals, 0.85 (0.66–1.10) and 0.72 (0.55–0.94) | Erratum, Fig. 2B | "Subgroup interaction: p=0.38" | 1.181 (0.815–1.709), p = 0.38 |
| Mortality ratio of RRs (control row) | Abstract; Table 3 | 1.12 (0.92–1.35), P = 0.28 | 1.115 (0.924–1.345), p = 0.26 |
| Ischemic stroke ratio of RRs (control row) | Abstract; Table 3 | 0.70 (0.25–1.97), P = 0.50 | 0.697 (0.246–1.974), p = 0.50 |
Applied to the paper's own inputs, the method reproduces both control rows to the printed precision. (The mortality point, 0.97/0.87 = 1.1149, sits on the rounding boundary between 1.11 and 1.12. The small P differences fit with recomputing from rounded inputs.) For MI it reproduces the point estimate (1.19), the lower limit (0.82) and the P value (0.35 vs 0.36). Only the upper limit fails, at 1.73 against the printed 1.17. Three independent routes all put the upper limit between 1.71 and 1.73: log symmetry with the printed lower limit, the printed P value, and the erratum's corrected subtotals.
Figure
Figure 1. Ratio of RRs of aspirin (diabetes vs no diabetes), with 95% CIs, on a log axis. For each outcome, the upper line (orange circle) is the value printed in the abstract and in Table 3's "pooled across trials" column. The lower line (blue square) is recomputed by recompute.py.txt from the Table 3 subgroup RRs using the paper's stated Altman–Bland method. The dashed vertical line marks a ratio of 1 (no difference). For mortality and ischemic stroke the two intervals coincide. For MI the printed interval (0.82–1.17) stops short of its own point estimate (1.19). The recomputed interval runs to 1.73. Both MI intervals include 1. The recomputed mortality point shows as 1.11 rather than 1.12 only because 1.1149 rounds down.
Why it matters
- It is a headline result. The MI ratio is one of three numbers in the abstract's Results, and it is the only support the abstract gives for its MI conclusion. Abstract-only readers, indexers and later meta-analysts will take the printed interval as it stands.
- The printed interval overstates the precision a great deal. On the log scale, the printed interval implies a standard error of 0.091. The paper's inputs give 0.190, about twice as large. As printed, the upper limit of 1.17 suggests that the relative MI benefit of aspirin in diabetes is at most modestly smaller than without diabetes. Taking the no-diabetes RR of 0.72, a ratio of 1.17 corresponds to an RR in diabetes of at most 0.84, which is still a benefit. The recomputed upper limit of 1.73 corresponds to an RR in diabetes of up to 1.25, which includes no benefit at all. This bears on the abstract's statement that the relative benefit is "similar" and on the Implications statement that between-group differences were "inconsistent and of small magnitude". The data are compatible with a small difference but do not rule out a large one.
- The paper's conclusion of non-significance is unchanged. The corrected interval (0.82–1.73) still includes 1 and agrees with the printed P interaction of 0.36. The Results statement that "All three methods yielded nonsignificant ratios of RR" remains true. The point estimate and its direction are unchanged.
- A reader can see that something is wrong but cannot repair it. Because the upper limit sits below the point estimate, the error is detectable. The correct value is not printed anywhere in the paper or in the erratum. To recover it, a reader has to recompute it.
Explanations considered
- A different method or a different interval type. This does not account for the numbers. The Methods name the Altman–Bland method, and footnote † defines the column as an ordinary 95% CI. The same method with the same inputs reproduces the mortality and stroke rows, and the MI point, lower limit and P value. No method yields a 95% interval that excludes its own point estimate.
- A value copied from another column. Ruled out. The other MI columns are 1.02 (0.71–1.47) within trials and 1.06 (0.16–6.71) Bayesian. Neither contains 1.17.
- A text-conversion artifact (lost minus sign, superscript read as a digit, shifted table cell). Ruled out. The identical string "1.19 (0.82–1.17)" appears in both the abstract and Table 3. Table 3 is live text in separate cells in the PMC/Europe PMC XML, not an image. The same string appears in the PubMed abstract (PMID 19741185) and in the PMC HTML page. The PMC copy is the publisher-deposited version (not an author manuscript).
- Different inputs to the ratio (for example, unrounded subgroup estimates or a corrected trial denominator). This does not account for the numbers. Rounding in the inputs moves the limits by about 0.01. The 2018 erratum's corrected MI subtotals give 0.82–1.71, still nowhere near 1.17.
- Already corrected. No. The only correction notice (see Status) concerns the Fig. 2 panel labels and the ETDRS denominators. It does not mention Table 3 or the abstract.
- The intended value. Three independent routes give an upper limit near 1.71–1.73, so a reporting slip in the upper limit is the simplest explanation consistent with the numbers. The exact intended value cannot be fixed from the rounded published inputs. Note that 1.71, the value from the erratum's subtotals, has the same digits as 1.17. That is an observation, not evidence of how the error arose.
Status
As of 2026-10-24.
- Version of record. The publisher's site (diabetesjournals.org) and its PDF links did not serve the article to automated requests (HTTP 403, or an HTML page instead of the PDF). The PMC copy is publisher-deposited (JATS metadata: is-manuscript no). The PMC HTML, the Europe PMC full-text XML and the PubMed abstract all carry "1.19 (0.82–1.17)" in the abstract, and the PMC and Europe PMC copies carry it in Table 3 as well. A web-search snippet of the abstract shows the same string.
- Supplements. The article lists no supplementary material. The Europe PMC file bundle holds only the two figure images.
- Corrections. Crossref and Europe PMC (via
src/known.py) list one notice, doi:10.2337/dc18-er06a (PMC5961400, PMID 29686158; Diabetes Care 2018;41(6):1321–1322, online 2018-04-23). It states that the Fig. 2 plots were mislabeled and that the ETDRS denominators were wrong, and it republishes Fig. 2. It does not mention Table 3 or the MI ratio of RRs in the abstract. Its corrected Fig. 2B prints MI subtotals of 0.85 (0.66, 1.10) with diabetes and 0.72 (0.55, 0.94) without, and "Subgroup interaction: p=0.38". These support an upper limit near 1.7 (see table above). No retraction or expression of concern was found. - PubPeer. The PubPeer API and search page refused automated access (HTTP 404 on the legacy API, Cloudflare challenge on the search page), so PubPeer could not be queried directly. A web search for the title together with "PubPeer" found no thread for this article.
- Sources. PMC article PMC2782995 (via
src/show.py); Europe PMC full-text XML and supplementary-file bundles for PMC2782995 and PMC5961400; erratum figuredc18er06af2.jpg; Crossref; PubMed PMID 19741185 and 29686158; https://doi.org/10.2337/dc09-1297; Altman DG, Bland JM, BMJ 2003;326:219.
Suggested correction
In the abstract (Results) and in Table 3 (myocardial infarction row, column "Ratio of RRs pooled across trials"), the 95% CI for the ratio of RRs should read approximately 0.82–1.73 in place of 0.82–1.17. That is the value the paper's stated Altman–Bland method gives from the Table 3 subgroup RRs. If the authors recompute from the unrounded estimates, or from the subtotals corrected in the 2018 erratum (which give 0.82–1.71), they should print their own figure. The point estimate (1.19) and the P interaction (0.36) need no change. The authors may also wish to state that the corrected interval, while still including 1, is compatible with a substantially smaller relative MI benefit of aspirin in patients with diabetes than the printed interval suggested.
Recomputation script (python recompute.py.txt)
"""Recompute the across-trial ratio of relative risks (Table 3) in PMC2782995
(Diabetes Care 2009;32:2300-2306, doi:10.2337/dc09-1297).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg and fig1.png next to this file.
Method (as stated in the paper's Methods): the ratio of RRs is estimated "using the
method of Altman and Bland", i.e. ratio = RR_diabetes / RR_no_diabetes, with
SE(log ratio) = sqrt(SE_1^2 + SE_2^2), where each SE is recovered from the published
95% CI as (ln upper - ln lower) / (2 * 1.96).
Inputs: the subgroup RRs and 95% CIs printed in Table 3 (columns "RR with diabetes" and
"RR without diabetes"; the same values appear in the Results text), and, as a cross-check,
the MI subgroup subtotals printed in the corrected Fig. 2B of the 2018 erratum
(doi:10.2337/dc18-er06a, PMC5961400).
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# --------------------------------------------------------------- inputs (Table 3, verbatim)
# outcome: (RR with diabetes, lo, hi), (RR without diabetes, lo, hi),
# reported ratio of RRs pooled across trials (point, lo, hi), reported P interaction
TABLE3 = {
"Mortality": ((0.97, 0.87, 1.08), (0.87, 0.75, 1.02), (1.12, 0.92, 1.35), 0.28),
"Myocardial infarction": ((0.86, 0.67, 1.11), (0.72, 0.55, 0.95), (1.19, 0.82, 1.17), 0.36),
"Ischemic stroke": ((0.62, 0.31, 1.24), (0.89, 0.41, 1.94), (0.70, 0.25, 1.97), 0.50),
}
# Other Table 3 columns for the MI row (to check whether 1.17 came from another column)
MI_WITHIN = (1.02, 0.71, 1.47) # ratio of RRs pooled within trials
MI_BAYES = (1.06, 0.16, 6.71) # Bayesian analysis, 95% credible interval
# 2018 erratum, corrected Fig. 2B (myocardial infarction): subgroup subtotals and
# "Subgroup interaction: p=0.38"
ERRATUM_MI = ((0.85, 0.66, 1.10), (0.72, 0.55, 0.94), 0.38)
def se_from_ci(lo, hi):
return (np.log(hi) - np.log(lo)) / (2 * Z)
def altman_bland(a, b):
"""Ratio of two RRs with 95% CI and two-sided p (Altman & Bland 2003)."""
s1, s2 = se_from_ci(a[1], a[2]), se_from_ci(b[1], b[2])
lr = np.log(a[0] / b[0])
s = np.hypot(s1, s2)
return dict(ratio=np.exp(lr), log_ratio=lr, se1=s1, se2=s2, se=s,
lo=np.exp(lr - Z * s), hi=np.exp(lr + Z * s),
p=2 * norm.sf(abs(lr) / s))
def main():
res = {}
print("Across-trial ratio of RRs (Altman-Bland) from the Table 3 subgroup RRs")
print("-" * 78)
for k, (a, b, rep, p_rep) in TABLE3.items():
r = altman_bland(a, b)
res[k] = r
mid = np.exp((np.log(rep[1]) + np.log(rep[2])) / 2)
print(f"{k}")
print(f" SE(ln RR) diabetes = {r['se1']:.3f}, no diabetes = {r['se2']:.3f}, "
f"SE(ln ratio) = {r['se']:.3f}")
print(f" recomputed ratio {r['ratio']:.3f} 95% CI {r['lo']:.3f}-{r['hi']:.3f} "
f"(rounded {r['ratio']:.2f}, {r['lo']:.2f}-{r['hi']:.2f}) p = {r['p']:.2f}")
print(f" reported ratio {rep[0]:.2f} 95% CI {rep[1]:.2f}-{rep[2]:.2f} "
f"P interaction = {p_rep:.2f}")
print(f" reported CI contains reported point? {rep[1] <= rep[0] <= rep[2]}; "
f"log-scale midpoint of reported CI = {mid:.3f}")
mi = res["Myocardial infarction"]
rep = TABLE3["Myocardial infarction"][2]
print()
print("Myocardial infarction: further checks")
print("-" * 78)
up_sym = rep[0] ** 2 / rep[1]
lo_sym = rep[0] ** 2 / rep[2]
print(f" upper limit log-symmetric to printed lower 0.82 about 1.19: {up_sym:.3f}")
print(f" lower limit log-symmetric to printed upper 1.17 about 1.19: {lo_sym:.3f}")
# interval implied by the reported P interaction (0.36), taking the reported point
p_rep = TABLE3["Myocardial infarction"][3]
se_p = np.log(rep[0]) / norm.isf(p_rep / 2)
print(f" SE(ln ratio) implied by reported P = {p_rep}: {se_p:.3f} -> 95% CI "
f"{np.exp(np.log(rep[0]) - Z*se_p):.2f}-{np.exp(np.log(rep[0]) + Z*se_p):.2f}")
# SE implied by the printed interval itself
se_printed = se_from_ci(rep[1], rep[2])
print(f" SE(ln ratio) implied by printed CI 0.82-1.17: {se_printed:.3f} "
f"(recomputed {mi['se']:.3f}; ratio {se_printed/mi['se']:.2f})")
print(f" printed upper 1.17 < printed point 1.19 by {rep[0]-rep[2]:.2f}")
# erratum cross-check
ea, eb, ep = ERRATUM_MI
er = altman_bland(ea, eb)
print(f" from 2018 erratum Fig. 2B subtotals 0.85 (0.66-1.10) / 0.72 (0.55-0.94): "
f"ratio {er['ratio']:.3f}, 95% CI {er['lo']:.3f}-{er['hi']:.3f}, p = {er['p']:.2f} "
f"(erratum prints p = {ep})")
# other columns
print(f" other MI columns: within-trial {MI_WITHIN[0]} ({MI_WITHIN[1]}-{MI_WITHIN[2]}), "
f"Bayesian {MI_BAYES[0]} ({MI_BAYES[1]}-{MI_BAYES[2]}); neither has 1.17 as a limit")
# what the two intervals mean on the RR scale (illustration, taking RR_no_dm = 0.72)
rr_nodm = TABLE3["Myocardial infarction"][1][0]
print(f" with RR(no diabetes) = {rr_nodm}: printed upper 1.17 -> RR(diabetes) up to "
f"{rr_nodm*rep[2]:.2f}; recomputed upper {mi['hi']:.2f} -> up to {rr_nodm*mi['hi']:.2f}")
make_figure(res)
def make_figure(res):
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC})
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
outcomes = list(TABLE3)
ypos = {k: 2 * (len(outcomes) - 1 - i) for i, k in enumerate(outcomes)}
off = 0.32
for k in outcomes:
y = ypos[k]
rep = TABLE3[k][2]
r = res[k]
# reported (orange), above
ax.plot([rep[1], rep[2]], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot([rep[0]], [y + off], "o", color=ORANGE, ms=8, zorder=3)
# recomputed (blue), below
ax.plot([r["lo"], r["hi"]], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="butt")
ax.plot([r["ratio"]], [y - off], "s", color=BLUE, ms=8, zorder=3)
ax.text(2.55, y + off, f"{rep[0]:.2f} ({rep[1]:.2f}–{rep[2]:.2f})",
va="center", ha="left", color=ORANGE, fontsize=10.5)
ax.text(2.55, y - off, f"{r['ratio']:.2f} ({r['lo']:.2f}–{r['hi']:.2f})",
va="center", ha="left", color=BLUE, fontsize=10.5)
# annotate the impossible interval
y_mi = ypos["Myocardial infarction"] + off
ax.annotate("printed upper limit 1.17\nis below the point 1.19",
xy=(1.17, y_mi), xytext=(0.265, y_mi + 0.6),
fontsize=10, color=TXT, ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=SEC, lw=1))
ax.axvline(1.0, color=SEC, lw=1.0, ls=(0, (4, 3)), zorder=1)
ax.set_xscale("log")
ticks = [0.25, 0.5, 1, 2]
ax.set_xticks(ticks)
ax.set_xticklabels(["0.25", "0.5", "1", "2"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_xlim(0.2, 2.5)
ax.set_ylim(-0.9, 2 * (len(outcomes) - 1) + 1.35)
ax.set_yticks([ypos[k] for k in outcomes])
ax.set_yticklabels(outcomes, color=TXT)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color="#e4e3df", lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.set_xlabel("Ratio of RRs, diabetes vs no diabetes (log scale; 1 = no difference)",
color=SEC)
handles = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8,
label="Reported (abstract and Table 3)"),
Line2D([], [], color=BLUE, lw=2, marker="s", ms=8,
label="Recomputed from the Table 3 subgroup RRs (Altman–Bland)")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=1,
frameon=False, fontsize=10)
fig.text(0.985, 0.955, "ratio (95% CI)", ha="right", va="top", color=SEC, fontsize=10)
fig.subplots_adjust(left=0.235, right=0.78, top=0.94, bottom=0.33)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nwrote fig1.svg and fig1.png")
if __name__ == "__main__":
main()
Mortality-rate 95% CI is far narrower than two deaths in 610.1 person-years allow
Diabetes Care (2021) · doi:10.2337/dc20-2458 · PMC7985423
Summary
This research letter reports 2 deaths among 49 islet-transplant recipients over 610.1 person-years, and a mortality rate of 3.28 (95% CI 2.12–5.05) per 1,000 person-years. The point estimate is correct (2 / 610.1 × 1,000 = 3.278), but the interval cannot come from 2 deaths: the exact Poisson 95% CI is 0.40–11.84, and no standard method gives a lower limit above 0.90 or an upper limit below 7.82. The printed interval is about as wide as one would expect from 20 to 23 deaths. This matters because the printed upper limit (5.05) lies just below all three comparator-cohort rates quoted in the letter (5.31, 7.99 and 10.79), whereas the correct interval contains all three.
What the paper reports
Results paragraph (cohort):
we retrospectively evaluated a cohort of 49 T1D subjects who underwent ITx and were followed during 2000–2020
Results paragraph (follow-up):
Person-years of follow-up was 610.1, and time after ITx was median (25th–75th) 13.8 (8.2–17.7) years.
Results paragraph (deaths):
Two subjects (4.08%) with T1D diagnosis in 1963 and 1977, respectively, died during the 20-year follow-up.
Results paragraph (the rate and its interval):
incidence rate of mortality (95% CI) was 3.28 (2.12–5.05) per 1,000 person-years (Fig. 1)
Discussion paragraph (comparator rates):
The Pittsburgh Epidemiology of Diabetes Complications Study cohort observed a mortality incidence rate of 10.79 per 1,000 person-years (T1D diagnosis between 1950 and 1980) and 5.31 per 1,000 person-years in the subcohort (66.9%) diagnosed between 1965 and 1980.
The population-based Allegheny County Type 1 Diabetes Registry cohort, with T1D diagnosed between 1965 and 1979, showed an incidence rate of mortality of 7.99 per 1,000 person-years.
Discussion paragraph (interpretation):
In our study, we observed a lower mortality incidence rate, although these comparisons are limited by the eligibility criteria, including absence of albuminuria.
Our results suggest that ITx is not associated with increased mortality regardless of the use of long-term immunosuppressive therapy.
It is reasonable to consider the hypothesis that the lowest cumulative hyperglycemic exposure and prevention of SHEs provided by ITx represent protection against cardiovascular disease and mortality in patients with T1D.
Figure 1 legend (excerpt):
Mortality incidence rate in 1,000 person-years (bar graph inset).
The letter names no CI method, model or adjustment. The only analyses it describes are a Kaplan–Meier curve and this crude incidence rate.
What the paper's own numbers show
Method. A crude mortality rate is d / PY, with the number of deaths d treated as Poisson. The exact (Garwood) 95% CI for the count is [χ²(0.025; 2d) / 2, χ²(0.975; 2d + 2) / 2], divided by the person-years. For comparison I also computed the mid-p exact, Byar, score, Jeffreys, log-Wald (SE of the log rate = 1/√d) and plain Wald intervals. Because SE(log rate) = 1/√d, the width of a CI on the log scale shows how many events it was computed from. I applied this to the printed interval, and I checked the rounding range of the printed limits. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Deaths, % of cohort | Results | 2 (4.08%) of 49 | 2/49 = 4.08% |
| Mortality rate per 1,000 PY | Results | 3.28 | 2 / 610.1 × 1,000 = 3.278 |
| 95% CI, exact Poisson (Garwood) | Results | 2.12–5.05 | 0.40–11.84 |
| 95% CI, mid-p exact | Results | 2.12–5.05 | 0.55–10.83 |
| 95% CI, log-Wald (SE = 1/√2) | Results | 2.12–5.05 | 0.82–13.11 |
| 95% CI, score / Jeffreys / Byar | Results | 2.12–5.05 | 0.90–11.95 / 0.68–10.52 / 0.37–11.84 |
| 95% CI, Wald (normal) | Results | 2.12–5.05 | −1.27 to 7.82 |
| Upper / lower limit ratio (log-scale width) | derived | 5.05 / 2.12 = 2.38 (0.87) | exact 29.8 (3.39, 3.9 times wider); log-Wald 16.0 (2.77) |
| Centering of the printed limits | derived | 3.28 | geometric mean 3.27; est/lower 1.547, upper/est 1.540 |
| SE(log rate) implied by the printed CI | derived | 0.221 | 2 deaths give 1/√2 = 0.707 (3.2 times larger) |
| Events implied by the printed width | derived | 2 deaths | 1/0.221² = 20.4 (20.2–20.6 over the rounding range of the limits); exact-CI width matches about 23 |
| Exact 95% CI at rate 3.28 with 20 / 21 / 25 events | derived | 2.12–5.05 | 2.00–5.07 / 2.03–5.01 / 2.12–4.84 |
| Confidence level the printed CI would have (log-Wald, d = 2) | derived | 95% | 46.1% |
| Exact CI at lower confidence levels, d = 2 | derived | 2.12–5.05 | 90%: 0.58–10.32; 80%: 0.87–8.72; 50%: 1.58–6.43 |
All standard 95% methods give a lower limit between 0.37 and 0.90 (the plain Wald limit is negative) and an upper limit between 7.82 and 13.11. The printed interval 2.12–5.05 lies inside every one of them and on the log scale the exact interval is 3.9 times as wide (3.39 against 0.87). Its limits are almost symmetric around 3.28 on the log scale, which fits a log-Wald construction with about 20 events. It does not fit the 2 deaths that the letter reports.
Figure
Figure 1. The mortality rate per 1,000 person-years on a log scale. Orange: the 95% CI printed in the letter, 2.12–5.05. Blue: 95% CIs recomputed from the letter's own 2 deaths in 610.1 person-years by three standard methods (exact Poisson 0.40–11.84, mid-p 0.55–10.83, log-Wald 0.82–13.11). Markers sit at the point estimate, 3.28, which all rows share. The dashed lines mark the three comparator rates quoted in the letter's Discussion (EDC subcohort 5.31, Allegheny County Registry 7.99, EDC 10.79). The printed interval ends just below all three. Every recomputed interval contains all three.
Figure 2. The width of an exact Poisson 95% CI for a rate, expressed as the ratio of its upper to its lower limit, depends only on the number of events (blue curve). With the cohort's 2 deaths the ratio is 11.84 / 0.40 = 29.8. The printed interval has a ratio of 5.05 / 2.12 = 2.38 (orange line), which the exact interval reaches only at about 23 deaths. A log-Wald construction gives about 20 (table above).
Why it matters
The mortality rate is the letter's main quantitative result. It is the only estimate given with a confidence interval, and Figure 1's inset compares it with the rates of other cohorts. The point estimate is unaffected. The interval, however, overstates the precision of a rate based on 2 deaths: its implied log-scale standard error is 0.221, against 0.707 for 2 events, and the exact interval is 3.9 times as wide on the log scale.
The comparison with other cohorts depends on that precision. With the printed interval, all three comparator rates (5.31, 7.99, 10.79) lie above the upper limit (5.05), which reads as a clearly lower mortality in transplant recipients. With the correct interval (0.40–11.84) all three lie inside it. Put as a one-sided Poisson calculation, a cohort with 610.1 person-years would see 2 or fewer deaths with probability 0.37 if its true rate were 5.31, 0.14 at 7.99, and 0.04 at 10.79. Only the oldest comparator (diagnosed 1950–1980) is borderline, and the letter itself notes that the transplant cohort was selected for absence of albuminuria.
The letter's cautious statements hold: "we observed a lower mortality incidence rate" is true of the point estimates, and the letter already says that "these comparisons are limited by the eligibility criteria". The conclusion that transplantation "is not associated with increased mortality" remains consistent with the data, but the data cannot exclude a mortality rate as high as that of the comparison populations. Any reading of the result as showing a statistically lower mortality, and the hypothesis of a protective effect that the Discussion raises, have less support than the printed interval suggests.
Explanations considered
- A different CI method. This does not account for the numbers. Exact, mid-p, Byar, score, Jeffreys, log-Wald and Wald intervals for 2 events all have lower limits of 0.90 or less and upper limits of 7.82 or more. With 2 events, the width of any interval is set by the count, not by the choice of method.
- A different confidence level. This does not account for the numbers. The letter labels the interval "95% CI". To give 2.12–5.05 from 2 deaths, a log-Wald interval would need a confidence level of about 46%. The exact 50% interval is 1.58–6.43, still wider than the printed one.
- Different inputs (another count or another denominator). This does not account for the numbers as published. The point estimate 3.28 reproduces exactly from 2 deaths and 610.1 person-years, and 4.08% reproduces from 2 of 49, so the stated numerator and denominator are the ones the rate uses. The printed interval is centered on 3.28 (geometric mean of the limits 3.27). Its width matches about 20–23 events at that rate. The letter reports no analysis with that many events. The 2020 meeting abstract of the same study also reports 2 deaths among 49 recipients.
- Rounding. This does not account for the numbers. Over the rounding range of the printed limits (2.115–2.125, 5.045–5.055), the implied number of events stays between 20.2 and 20.6.
- A typographical error in one limit. This does not account for the numbers. Both limits are wrong in the same direction (the lower too high, the upper too low), and they are symmetric about 3.28 on the log scale. A single mistyped digit would not produce that pattern.
- A conversion or typesetting error in the archived text. This is excluded. The PMC article page and the Europe PMC full-text XML give the same sentence word for word (the string "3.28 (2.12–5.05)" occurs once in the XML), and the local archived copy agrees.
- An explanation in a supplement or in the figure. This is not available. The letter has no supplementary material. The Figure 1 legend describes the inset as a bar graph of incidence rates and does not mention intervals. The figure image could not be retrieved, so it was not checked.
Status
- Version of record. The publisher's pages could not be fetched: doi.org/10.2337/dc20-2458 redirects to diabetesjournals.org (care/article/44/4/e67/138613), which returned HTTP 403, and the publisher PDF listed in Crossref also returned 403. The final published text was read on the PMC article page (publication status ppublish) and in the Europe PMC full-text XML. Both carry the quoted sentences with identical values. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. The Figure 1 legend excerpt was confirmed against the same archived full text. - Supplements. None. The Europe PMC XML has no supplementary-material elements (its only graphics are Figure 1 and its thumbnail), the PMC page lists no supplementary material, and the Europe PMC supplementary-files endpoint returned 404. The letter (pages e67–e68) has no data supplement. The related ADA 2020 meeting abstract (doi:10.2337/db20-206-lb) confirms 2 deaths and n = 49 but reports no person-years, rate or CI.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern, retraction or published comment. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. A web search for corrections or comments on the article found none. The PubPeer search API refused automated access (HTTP 403, human verification required), so PubPeer could not be checked directly.
- Sources. PMC article PMC7985423; Europe PMC full-text XML for PMC7985423; Crossref records for 10.2337/dc20-2458 and 10.2337/db20-206-lb; Europe PMC notices; web search.
Suggested correction
- Replace the interval in the Results with one computed from 2 deaths in 610.1 person-years, for example "3.28 (95% CI 0.40–11.84) per 1,000 person-years (exact Poisson)", and name the method.
- If an interval or error bars for the Miami cohort appear in the Figure 1 inset, correct them in the same way.
- Revise the comparison with the other cohorts to say that, with 2 deaths, the confidence interval for the transplant cohort includes all three comparator rates (5.31, 7.99 and 10.79 per 1,000 person-years), so the data are compatible both with lower mortality and with mortality similar to those populations.
- Qualify the hypothesis of a protective effect on mortality accordingly, in line with the letter's closing call for larger data sets.
Recomputation script (python recompute.py.txt)
"""Recompute the mortality-rate confidence interval of PMC7985423
(Diabetes Care 2021, doi:10.2337/dc20-2458, survival after islet transplantation, 20-year follow-up).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the letter as archived in PMC (Results paragraph):
49 subjects; 2 deaths ("Two subjects (4.08%) ... died"); 610.1 person-years of follow-up;
median follow-up 13.8 (25th-75th 8.2-17.7) years;
reported incidence rate of mortality (95% CI) 3.28 (2.12-5.05) per 1,000 person-years.
Comparator rates quoted in the Discussion paragraph, per 1,000 person-years:
EDC (diagnosed 1950-1980) 10.79; EDC subcohort (diagnosed 1965-1980) 5.31; ACR (1965-1979) 7.99.
Method: a crude rate d / PY with d Poisson-distributed. The exact (Garwood) 95% CI for the
count is [chi2(0.025; 2d) / 2, chi2(0.975; 2d + 2) / 2]; other standard methods are shown
for comparison. The SE of the log rate is 1 / sqrt(d), so a CI's width on the log scale
reveals the number of events it was computed from.
"""
import math
import os
import numpy as np
from scipy import optimize, stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.isf(0.025)
# ---------------------------------------------------------------- inputs (from the paper)
N, D, PY = 49, 2, 610.1
REP = (3.28, 2.12, 5.05)
COMP = [("EDC subcohort, diagnosed 1965-80", 5.31), ("ACR, diagnosed 1965-79", 7.99),
("EDC, diagnosed 1950-80", 10.79)]
K = 1000.0 / PY
def exact(d, py, level=0.95):
a = 1 - level
lo = stats.chi2.ppf(a / 2, 2 * d) / 2 if d > 0 else 0.0
hi = stats.chi2.ppf(1 - a / 2, 2 * d + 2) / 2
return lo * 1000 / py, hi * 1000 / py
rate = D * K
print("=== 1. Point estimate and counts ===")
print(f" rate = {D} / {PY} x 1000 = {rate:.3f} per 1,000 PY (reported {REP[0]})")
print(f" deaths {D}/{N} = {100 * D / N:.2f}% (reported 4.08%)")
print(f" mean follow-up = {PY}/{N} = {PY / N:.2f} y (median reported 13.8)")
print("\n=== 2. 95% CI for 2 deaths in 610.1 PY, by method (per 1,000 PY) ===")
CI = {}
CI["Exact Poisson (Garwood)"] = exact(D, PY)
f_lo = lambda m: stats.poisson.sf(D, m) + 0.5 * stats.poisson.pmf(D, m) - 0.025 # noqa: E731
f_hi = lambda m: stats.poisson.cdf(D - 1, m) + 0.5 * stats.poisson.pmf(D, m) - 0.025 # noqa: E731
CI["Mid-p exact"] = (optimize.brentq(f_lo, 1e-6, 50) * K, optimize.brentq(f_hi, 1e-6, 50) * K)
CI["Byar"] = (D * (1 - 1 / (9 * D) - Z / (3 * math.sqrt(D))) ** 3 * K,
(D + 1) * (1 - 1 / (9 * (D + 1)) + Z / (3 * math.sqrt(D + 1))) ** 3 * K)
CI["Score (Wilson-type)"] = ((D + Z ** 2 / 2 - Z * math.sqrt(D + Z ** 2 / 4)) * K,
(D + Z ** 2 / 2 + Z * math.sqrt(D + Z ** 2 / 4)) * K)
CI["Jeffreys"] = (stats.gamma.ppf(0.025, D + 0.5) * K, stats.gamma.ppf(0.975, D + 0.5) * K)
CI["Log-Wald, SE = 1/sqrt(d)"] = (rate * math.exp(-Z / math.sqrt(D)), rate * math.exp(Z / math.sqrt(D)))
CI["Wald (normal)"] = (rate - Z * math.sqrt(D) * K, rate + Z * math.sqrt(D) * K)
for name, (lo, hi) in CI.items():
print(f" {name:28s} {lo:6.2f} - {hi:6.2f} " + (f"(upper/lower = {hi / lo:.1f})" if lo > 0 else "(lower limit negative)"))
print(f" {'Reported':28s} {REP[1]:6.2f} - {REP[2]:6.2f} (upper/lower = {REP[2] / REP[1]:.2f})")
lo_all = [v[0] for k, v in CI.items() if k != "Wald (normal)"]
hi_all = [v[1] for v in CI.values()]
print(f" lower limits (all methods except Wald, whose lower limit is negative): {min(lo_all):.2f}-{max(lo_all):.2f}; "
f"upper limits (all methods): {min(hi_all):.2f}-{max(hi_all):.2f}")
print("\n=== 3. What the reported interval implies ===")
gm = math.sqrt(REP[1] * REP[2])
se_log = math.log(REP[2] / REP[1]) / (2 * Z)
print(f" est/lower = {REP[0] / REP[1]:.3f}, upper/est = {REP[2] / REP[0]:.3f}; geometric mean of limits = {gm:.2f}")
print(f" implied SE(log rate) = {se_log:.3f} (2 deaths give 1/sqrt(2) = {1 / math.sqrt(2):.3f}, "
f"{(1 / math.sqrt(2)) / se_log:.1f} times larger)")
ex = CI["Exact Poisson (Garwood)"]
print(f" log-scale width: printed {math.log(REP[2] / REP[1]):.3f}; exact {math.log(ex[1] / ex[0]):.3f} "
f"({math.log(ex[1] / ex[0]) / math.log(REP[2] / REP[1]):.1f} times wider); log-Wald {2 * Z / math.sqrt(D):.3f}")
print(f" implied number of events 1/SE^2 = {1 / se_log ** 2:.1f}")
for dd in (15, 18, 20, 21, 22, 25):
lo, hi = exact(dd, dd / REP[0] * 1000)
print(f" exact 95% CI with {dd:2d} events at rate 3.28: {lo:.2f}-{hi:.2f}")
zimp = math.log(REP[2] / REP[1]) / 2 * math.sqrt(D)
print(f" if log-Wald with d = 2: implied z = {zimp:.3f}, two-sided confidence level "
f"{100 * (2 * stats.norm.cdf(zimp) - 1):.1f}%")
for cl in (0.90, 0.80, 0.68, 0.50):
lo, hi = exact(D, PY, cl)
zz = stats.norm.isf((1 - cl) / 2)
print(f" {int(cl * 100)}% exact: {lo:.2f}-{hi:.2f}; {int(cl * 100)}% log-Wald: "
f"{rate * math.exp(-zz / math.sqrt(D)):.2f}-{rate * math.exp(zz / math.sqrt(D)):.2f}")
p = D / N
print(f" binomial 2/49 = {p:.4f}; Wald 95% half-width {Z * math.sqrt(p * (1 - p) / N):.4f}; "
f"exact (Clopper-Pearson) {stats.beta.ppf(0.025, D, N - D + 1):.4f}-{stats.beta.ppf(0.975, D + 1, N - D):.4f}")
# Rounding range of the reported limits cannot change the implied event count materially
ev = [1 / (math.log(h / l) / (2 * Z)) ** 2 for l in (2.115, 2.125) for h in (5.045, 5.055)]
print(f" implied events over rounding range of the limits (2.115-2.125, 5.045-5.055): {min(ev):.1f}-{max(ev):.1f}")
print("\n=== 4. Comparator rates against the cohort's 2 deaths ===")
ex_lo, ex_hi = CI["Exact Poisson (Garwood)"]
for lab, r in COMP:
mu = r * PY / 1000
print(f" {lab:34s} {r:5.2f}: inside reported CI? {REP[1] <= r <= REP[2]}; inside exact CI? "
f"{ex_lo <= r <= ex_hi}; expected deaths in 610.1 PY = {mu:.2f}; P(X <= 2) = {stats.poisson.cdf(D, mu):.3f}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": SEC, "svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: reported vs recomputed 95% CI on one log-scale rate axis, comparator rates as reference lines
rows = [("Reported in the letter", REP[1], REP[2], ORA, "o"),
("Exact Poisson (Garwood), 2 deaths", *CI["Exact Poisson (Garwood)"], BLU, "o"),
("Mid-p exact, 2 deaths", *CI["Mid-p exact"], BLU, "s"),
("Log-Wald, 2 deaths", *CI["Log-Wald, SE = 1/sqrt(d)"], BLU, "D")]
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
style(ax)
ax.set_xscale("log")
ax.set_xlim(0.3, 20)
ys = np.arange(len(rows))[::-1]
for (lab, lo, hi, c, m), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=c, lw=2, solid_capstyle="butt")
ax.plot([lo, hi], [y, y], ls="none", marker="|", ms=10, mew=2, color=c)
ax.plot([rate], [y], marker=m, ms=9, color=c, mec=BG, mew=1)
ax.text(lo / 1.06, y + 0.28, f"{lo:.2f}", ha="center", va="bottom", fontsize=10, color=c)
ax.text(hi * 1.06, y + 0.28, f"{hi:.2f}", ha="center", va="bottom", fontsize=10, color=c)
top = len(rows) - 0.2
for i, (lab, r) in enumerate(COMP):
ytop = top + 0.42 * i
ax.plot([r, r], [-0.6, ytop + 0.12], color=SEC, lw=0.8, ls=(0, (3, 3)), zorder=0)
name, era = lab.split(", diagnosed ")
ax.text(r / 1.03, ytop, f"{name} ({era}): {r}", ha="right", va="center", fontsize=10, color=SEC)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10.5)
for t, r in zip(ax.get_yticklabels(), rows):
t.set_color(r[3])
ax.tick_params(axis="y", length=0)
ax.set_ylim(-0.6, len(rows) + 0.85)
ticks = [0.5, 1, 2, 5, 10, 20]
ax.set_xticks(ticks)
ax.set_xticklabels(["0.5", "1", "2", "5", "10", "20"])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlabel("Mortality rate per 1,000 person-years (log scale)")
leg = [Line2D([], [], color=ORA, lw=2, marker="o", ms=8, label="Reported 95% CI"),
Line2D([], [], color=BLU, lw=2, marker="o", ms=8, label="Recomputed (2 deaths, 610.1 PY)"),
Line2D([], [], color=SEC, lw=0.8, ls=(0, (3, 3)), label="Comparator rates")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10,
handlelength=2.2, columnspacing=1.2)
fig.tight_layout(rect=(0, 0, 1, 0.92))
save(fig, "fig1")
# Figure 2: width of the exact 95% CI (upper/lower ratio) as a function of the number of deaths
d_grid = np.arange(1, 61)
ratio = np.array([exact(int(d), 1000)[1] / exact(int(d), 1000)[0] for d in d_grid])
rep_ratio = REP[2] / REP[1]
d_match = d_grid[np.argmin(np.abs(np.log(ratio) - math.log(rep_ratio)))]
print(f"\n=== 5. Exact-CI width vs events (fig2) ===")
print(f" exact upper/lower at d = 2: {ratio[1]:.1f}; reported upper/lower: {rep_ratio:.2f}; "
f"closest d on the grid: {d_match} (ratio {ratio[d_match - 1]:.2f})")
for dd in (20, 21):
print(f" d = {dd}: exact upper/lower = {ratio[dd - 1]:.2f}")
fig, ax = plt.subplots(figsize=(7.2, 3.8), facecolor=BG)
style(ax)
ax.grid(axis="y", which="major", color=GRID, lw=0.8)
ax.set_xscale("log")
ax.set_yscale("log")
ax.plot(d_grid, ratio, color=BLU, lw=2, label="Exact Poisson 95% CI, upper / lower limit")
ax.axhline(rep_ratio, color=ORA, lw=2, label=f"Reported CI 2.12-5.05: upper / lower = {rep_ratio:.2f}")
ax.plot([2], [ratio[1]], marker="o", ms=9, color=BLU, mec=BG)
ax.annotate(f"2 deaths (this cohort):\n11.84 / 0.40 = {ratio[1]:.1f}", (2, ratio[1]), xytext=(3.2, ratio[1] * 0.95),
fontsize=10, color=BLU, va="center")
ax.plot([d_match], [ratio[d_match - 1]], marker="o", ms=9, color=ORA, mec=BG)
ax.annotate(f"reported width matches about {d_match} deaths", (d_match, rep_ratio), xytext=(d_match, rep_ratio * 2.1),
fontsize=10, color=ORA, ha="center", arrowprops=dict(arrowstyle="-", color=ORA, lw=1))
ax.set_xlim(0.9, 65)
ax.set_ylim(1.5, 400)
ax.set_xticks([1, 2, 5, 10, 20, 50])
ax.set_xticklabels(["1", "2", "5", "10", "20", "50"])
ax.set_yticks([2, 5, 10, 30, 100, 300])
ax.set_yticklabels(["2", "5", "10", "30", "100", "300"])
for a in (ax.xaxis, ax.yaxis):
a.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.spines["left"].set_visible(True)
ax.set_xlabel("Number of deaths (log scale)")
ax.set_ylabel("Upper / lower 95% limit (log scale)")
ax.legend(loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Abstract reports a 13.2-point TIR gain for baseline GMI >8.0; Table 1 implies 22.3 points
Diabetes Care (2022) · doi:10.2337/dc22-1217 · PMC9862393
Summary
The abstract states that time in target range (TIR, 70–180 mg/dL) improved by 13.2 percentage points after the switch from predictive low glucose suspend (PLGS) to automated insulin delivery (AID) in people with baseline GMI >8.0 (N = 4,064). Table 1 gives this group a TIR of 32.5% on PLGS and 55.7%, 54.5% and 54.1% in AID months 1 to 3. That is a gain of 23.2, 22.0 and 21.6 points, or 22.3 points when the three AID months are averaged, which is the convention the abstract's other figures follow. The Results ("by >20 percentage points") and the Conclusions ("22–23 percentage point change") agree with the table, so the abstract is the only place with 13.2. The abstract figure understates the largest stratified effect by about 9 points (41%), and it makes that effect look smaller than the under-18 effect (14.0) when the table shows it is about 1.6 times as large.
What the paper reports
Abstract, Results:
Compared with PLGS, AID increased TIR on average from 58.4 to 70.5%.
Glycemic improvements were most pronounced in those <18 years old (TIR improvement 14.0 percentage points) and those with baseline GMI >8.0 (TIR improvement 13.2 percentage points).
Results, second paragraph:
Glycemic outcomes improved most in those with baseline GMI >8.0. These improvements were highly selective: individuals in an optimal state of glycemic control (baseline GMI ≤6.9) had TIR improved by ∼4.0 percentage points and had rate of hypoglycemia on AID compared with PLGS reduced by 0.9 percentage points, while those with baseline GMI >8.0 had TIR improved by >20 percentage points, but time below range also increased slightly (Table 1).
Conclusions (the paper's discussion section), fourth paragraph:
The group with the highest GMI, >8.0, showed the greatest percent improvement in TIR with a 22–23 percentage point change, and they achieved this change with no clinically significant increase in time spent in a state of hypoglycemia. Percent time >250 mg/dL was reduced by nearly one-half along with a reduction in time >180 mg/dL of 22–23 percentage points.
Table 1, "By baseline GMI", % TIR (columns PLGS | AID month 1 | AID month 2 | AID month 3; data are means ± SD):
≤6.9 (N = 4,450) | 80.6 ± 7.3 | 84.2 ± 6.8 | 83.8 ± 7.3 | 83.6 ± 7.6
6.9–7.4 (N = 5,359) | 65.9 ± 5.4 | 75.4 ± 7.0 | 74.8 ± 7.7 | 74.4 ± 8.1
7.4–8.0 (N = 5,018) | 51.8 ± 5.0 | 67.5 ± 7.8 | 66.6 ± 8.5 | 66.2 ± 8.9
>8.0 (N = 4,064) | 32.5 ± 9.5 | 55.7 ± 11.0 | 54.5 ± 11.5 | 54.1 ± 12.0
What the paper's own numbers show
Method. First I worked out the convention the abstract uses for "improvement", using the overall rows of Table 1. The abstract's overall figures (TIR 58.4 to 70.5%, GMI 7.5 to 7.1, time <70 mg/dL 1.66 to 1.46%) equal the PLGS value and the mean of the three AID months. Then I applied the same convention to every TIR row stratified by baseline GMI and by age. For the under-18 group, which the abstract reports as one figure, I pooled the 0–12 and 13–18 rows weighted by their N. I also computed each month separately, and the same quantities for time >180 mg/dL. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Overall TIR, PLGS → AID | Abstract; Table 1 | 58.4 → 70.5%; "average difference" 12.06 | mean of AID months 70.50%; change 12.10 (consistent) |
| Overall GMI and time <70 mg/dL, PLGS → AID | Abstract; Table 1 | 7.5 → 7.1; 1.66 → 1.46% | 7.47 → 7.08; 1.66 → 1.46% (consistent) |
| TIR change, age <18 | Abstract | 14.0 | 13.93 (N-weighted; 0–12: 14.13, 13–18: 13.80) (consistent) |
| TIR change, age <18 | Results | "≥13" | 13.0 to 15.2 in every month for both groups (consistent) |
| TIR change, baseline GMI >8.0 | Abstract | 13.2 | 22.27 (54.77 − 32.5); months 1–3: 23.2 / 22.0 / 21.6 |
| TIR change, baseline GMI >8.0 | Results | ">20" | 21.6 to 23.2 (consistent) |
| TIR change, baseline GMI >8.0 | Conclusions, para. 4 | "22–23" | 22.27 (consistent) |
| Time >180 mg/dL reduction, GMI >8.0 | Conclusions, para. 4 | "22–23" | 22.53; months 1–3: 23.4 / 22.3 / 21.9 (consistent) |
| TIR change, other GMI strata | Table 1 (text gives only "∼4.0" in Results and "3–4" in Conclusions for ≤6.9) | — | ≤6.9: 3.27; 6.9–7.4: 8.97; 7.4–8.0: 14.97 |
| Ratio of GMI >8.0 gain to <18 gain | Abstract vs Table 1 | 13.2 / 14.0 = 0.94 | 22.27 / 13.93 = 1.60 |
| Any baseline-GMI TIR change (month 1, 2, 3 or mean) within 0.15 of 13.2 | Table 1 | — | none |
The abstract's figure for the GMI >8.0 group cannot be obtained from Table 1 by any time point or summary used elsewhere in the paper. The abstract's other numbers can. Under the abstract's own convention, the value is 22.3. That agrees with the Results and with the Conclusions, which both describe the same group and the same outcome.
A small side observation, not part of the main finding: the abstract gives overall time >180 mg/dL on AID as 28.1%, while the mean of the three AID months is 28.0% (Table 1's average difference of −11.86 gives 28.04%). The difference is 0.1 points and has no bearing on any conclusion.
Figure
Figure 1. The conflicting passages side by side. Left (orange): the abstract's figure for the GMI >8.0 group. Right (blue): the Table 1 values for the same group, with the month-by-month changes and their mean recomputed from the table. Their mean (22.3) is 9.1 points larger than the abstract figure. Bottom: the Results and Conclusions describe the same group and outcome, and both agree with the table, not with the abstract. All quoted text is verbatim from the article as archived in PMC.
Figure 2. Change in % TIR from PLGS to AID (mean of AID months 1–3 minus PLGS), recomputed from Table 1 for the four baseline-GMI strata and for the pooled under-18 group (blue circles). The two values the abstract reports are shown as orange diamonds. For under-18s, the abstract's 14.0 agrees with the recomputed 13.9. For GMI >8.0, the abstract's 13.2 is 9.1 points below the recomputed 22.3. It is also below the recomputed value for the next stratum down (GMI 7.4–8.0, 15.0). The axis starts at zero.
Why it matters
- The wrong number is in the abstract, and only there. The abstract singles out two subgroups as having the most pronounced glycemic improvements, and the GMI >8.0 group is one of them. Readers who use only the abstract, including indexing services and secondary summaries, get an improvement of 13.2 points for this group instead of about 22.
- It reverses the comparison the abstract draws. At face value, the abstract says that people with poor baseline control (13.2) gained slightly less than people under 18 (14.0). Table 1 shows they gained about 1.6 times as much (22.3 vs 13.9). The abstract's 13.2 is also smaller than the recomputed gain of the GMI 7.4–8.0 stratum (15.0). That conflicts with the paper's own statement, in the Results, that outcomes "improved most in those with baseline GMI >8.0".
- It understates the spread across strata. The abstract's conclusion that changes with AID "may differ greatly across different demographic and clinical groups" still holds, but the table's range across GMI strata (about 3 to 22 points) is wider than the abstract implies.
- The main conclusions are not affected. The Table 1 data, the Results and the Conclusions section all carry the correct magnitude. The overall effect (58.4 to 70.5%), the comparison with randomized trials and the Conclusions section's emphasis on people with poorly controlled diabetes rest on the correct values. The error is confined to one figure in the abstract.
Explanations considered
- A different time point. Does not explain it. The change for GMI >8.0 is 23.2 in month 1, 22.0 in month 2 and 21.6 in month 3. No single month comes close to 13.2.
- A different summary convention. Does not explain it. Every other figure in the abstract (overall TIR, GMI, time below range, and the under-18 TIR gain of 14.0) matches the mean of the AID months minus PLGS. That convention gives 22.27 for GMI >8.0.
- A different outcome for the same group. Does not explain it. The reduction in time >180 mg/dL for GMI >8.0 is 21.9 to 23.4. Table 1 gives only TIR, time <70 mg/dL and time >180 mg/dL by baseline GMI, and the time <70 mg/dL changes are below 1 point.
- A different stratum. Does not explain it. The other GMI strata gain 3.3, 9.0 and 15.0 points. No baseline-GMI change at any time point or as a mean lies within 0.15 of 13.2. Five values elsewhere in Table 1 do (all the hits
recompute.py.txtfinds among the TIR and time >180 mg/dL rows): the TIR gain for age 0–12 in month 3 (13.3) and for females in month 1 (13.3), and the reduction in time >180 mg/dL for age 0–12 in month 3 (13.3), for age 19–25 in month 1 (13.3) and for females in month 1 (13.2). None of these is labelled as, or is a plausible stand-in for, the GMI >8.0 group. - An adjusted model estimate. Not supported by the paper. The Methods describe a general linear model with four repeated measures, but the paper reports no adjusted per-stratum estimate. The Results and Conclusions describe this group with the unadjusted table values (">20" and "22–23"). An adjustment that shrinks a raw gain of about 22 points to 13.2 would also contradict those two statements.
- A relative rather than absolute change. Does not explain it. The abstract says "percentage points". The relative change for GMI >8.0 (22.27 / 32.5) is 69%.
- A text-conversion artifact. Ruled out. The same "13.2" appears in the PMC text, in the Europe PMC full-text XML, in the PubMed abstract deposited by the publisher, and in the abstract reproduced on the journal's figshare supplement record. Table 1 reads the same in PMC and in Europe PMC.
- A correction already issued. None found (see Status).
Status
- Version of record. The publisher's landing page (reached from doi.org) and the publisher's PDF returned HTTP 403 to automated requests, so the typeset page could not be read directly. Four other copies of the published text agree on the passages quoted here: (1) the PMC article page; (2) the Europe PMC full-text XML, word for word; (3) the PubMed abstract (PMID 36126177), deposited by the publisher, which reads "TIR improvement 13.2 percentage points"; (4) the abstract reproduced on the journal's figshare supplement record (doi:10.2337/figshare.20645448, posted 2022-09-20). Every quotation was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The online supplement (
DC22-1217-supplement.pdf, 4 pages, version 1 of the figshare record) contains only Supplementary Figures 1–4: patient selection flow, meal boluses, total boluses and boluses by sex. It has no TIR tables or subgroup estimates and does not bear on the discrepancy. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by entries. The PubMed record has no CommentsCorrections entries. A Crossref search found no correction item linked to this DOI. The PubPeer API returned no record (HTTP 404) for the DOI. Web searches for the title or DOI together with "erratum", "correction" or "PubPeer" found no notice or thread for this article. - Sources. PMC article PMC9862393; Europe PMC full-text XML; PubMed 36126177; figshare record doi:10.2337/figshare.20645448 and its supplement PDF; Crossref; PubPeer; https://doi.org/10.2337/dc22-1217.
Suggested correction
In the abstract, replace
those with baseline GMI >8.0 (TIR improvement 13.2 percentage points)
with
those with baseline GMI >8.0 (TIR improvement 22.3 percentage points)
The value 22.3 is the mean of AID months 1–3 minus PLGS from Table 1 ((55.7 + 54.5 + 54.1) / 3 − 32.5 = 22.27), the same convention the abstract uses for its other figures, and it agrees with the Results (">20") and the Conclusions ("22–23"). If the authors computed this value from the underlying patient-level data, their exact figure may differ slightly in the last decimal. The authors may also wish to reconsider the wording "most pronounced in those <18 years old ... and those with baseline GMI >8.0". The table shows the GMI >8.0 gain is about 1.6 times the under-18 gain, so the two gains are not comparable in size.
Recomputation script (python recompute.py.txt)
"""Recompute the stratified TIR changes in Table 1 of PMC9862393
(Diabetes Care 2022, doi:10.2337/dc22-1217,
"Biobehavioral Changes Following Transition to Automated Insulin Delivery:
A Large Real-life Database Analysis").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 1, overall rows, and the rows for % TIR and % time >180 mg/dL by age-group,
by sex and by baseline GMI (columns: PLGS, AID month 1, AID month 2, AID month 3).
Abstract: "Compared with PLGS, AID increased TIR on average from 58.4 to 70.5%",
"<18 years old (TIR improvement 14.0 percentage points)",
"baseline GMI >8.0 (TIR improvement 13.2 percentage points)".
Results: "baseline GMI >8.0 had TIR improved by >20 percentage points".
Conclusions (Discussion), paragraph 4: "22–23 percentage point change" in TIR and a
"reduction in time >180 mg/dL of 22–23 percentage points".
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Table 1)
# (N, PLGS, AID m1, AID m2, AID m3)
TIR_GMI = {"≤6.9": (4450, 80.6, 84.2, 83.8, 83.6),
"6.9–7.4": (5359, 65.9, 75.4, 74.8, 74.4),
"7.4–8.0": (5018, 51.8, 67.5, 66.6, 66.2),
">8.0": (4064, 32.5, 55.7, 54.5, 54.1)}
TIR_AGE = {"0–12": (1877, 50.2, 65.4, 64.1, 63.5),
"13–18": (2759, 50.3, 65.0, 64.0, 63.3),
"19–25": (1670, 54.7, 68.3, 67.4, 67.4),
"26–64": (9335, 60.8, 72.9, 72.3, 71.9),
"≥65": (3250, 65.2, 76.0, 75.5, 75.3)}
TIR_SEX = {"Female": (10495, 58.5, 71.8, 71.0, 70.6),
"Male": (8392, 58.3, 70.3, 69.7, 69.3)}
TAR_GMI = {"≤6.9": (4450, 16.2, 13.5, 13.8, 14.1),
"6.9–7.4": (5359, 32.4, 23.2, 23.8, 24.2),
"7.4–8.0": (5018, 47.1, 31.3, 32.2, 32.6),
">8.0": (4064, 66.9, 43.5, 44.6, 45.0)}
TAR_AGE = {"0–12": (1877, 48.2, 32.9, 34.3, 34.9),
"13–18": (2759, 48.4, 33.8, 34.8, 35.5),
"19–25": (1670, 43.6, 30.3, 31.1, 31.2),
"26–64": (9335, 37.4, 25.5, 26.1, 26.5),
"≥65": (3250, 33.4, 22.7, 23.2, 23.5)}
TAR_SEX = {"Female": (10495, 39.9, 26.7, 27.6, 28.0),
"Male": (8392, 40.0, 28.2, 28.8, 29.2)}
OVERALL = { # PLGS, m1, m2, m3, printed "AID-PLGS average difference", abstract PLGS->AID
"% TIR": (58.4, 71.1, 70.4, 70.0, 12.06, (58.4, 70.5)),
"GMI": (7.47, 7.05, 7.08, 7.10, -0.39, (7.5, 7.1)),
"% time >180 mg/dL": (39.9, 27.4, 28.1, 28.5, -11.86, (39.9, 28.1)),
"% time <70 mg/dL": (1.66, 1.46, 1.47, 1.46, -0.20, (1.66, 1.46)),
}
ABS_GMI8 = 13.2 # Abstract, TIR improvement, baseline GMI >8.0
ABS_U18 = 14.0 # Abstract, TIR improvement, <18 years
def changes(row):
n, b, *m = row
return n, [x - b for x in m], float(np.mean(m) - b)
# ---------------------------------------------------------------- 1. overall convention
print("=== 1. Overall rows: abstract convention = mean of AID months 1-3 minus PLGS ===")
for k, (b, m1, m2, m3, pdiff, (ab, aa)) in OVERALL.items():
avg = np.mean([m1, m2, m3])
print(f"{k:20s} PLGS {b:6.2f} mean AID {avg:6.3f} diff {avg - b:+7.3f}"
f" | Table 'average difference' {pdiff:+.2f} | abstract {ab} -> {aa}")
# ---------------------------------------------------------------- 2. TIR by baseline GMI
print("\n=== 2. % TIR change (AID - PLGS) by baseline GMI ===")
G = {}
for k, row in TIR_GMI.items():
n, d, a = changes(row)
G[k] = a
print(f"GMI {k:8s} N={n:5d} m1 {d[0]:5.1f} m2 {d[1]:5.1f} m3 {d[2]:5.1f} mean-of-months {a:6.2f}")
print(f"GMI >8.0: abstract {ABS_GMI8} vs recomputed {G['>8.0']:.2f};"
f" shortfall {G['>8.0'] - ABS_GMI8:.2f} points"
f" ({100 * (G['>8.0'] - ABS_GMI8) / G['>8.0']:.0f}% of the recomputed change)")
print(f"GMI >8.0 mean AID TIR = {np.mean(TIR_GMI['>8.0'][2:]):.2f}%")
# ---------------------------------------------------------------- 3. TIR by age; pooled <18
print("\n=== 3. % TIR change by age-group ===")
A = {}
for k, row in TIR_AGE.items():
n, d, a = changes(row)
A[k] = a
print(f"age {k:6s} N={n:5d} m1 {d[0]:5.1f} m2 {d[1]:5.1f} m3 {d[2]:5.1f} mean-of-months {a:6.2f}")
n1, n2 = TIR_AGE["0–12"][0], TIR_AGE["13–18"][0]
U18 = (n1 * A["0–12"] + n2 * A["13–18"]) / (n1 + n2)
U18_unw = (A["0–12"] + A["13–18"]) / 2
print(f"<18 pooled (N-weighted) {U18:.2f}; unweighted {U18_unw:.2f}; abstract {ABS_U18}")
print(f"ratio GMI>8.0 / <18 (recomputed) = {G['>8.0'] / U18:.2f}; abstract ratio 13.2/14.0 = {ABS_GMI8 / ABS_U18:.2f}")
# ---------------------------------------------------------------- 4. TAR >180 for GMI >8.0
print("\n=== 4. % time >180 mg/dL reduction (PLGS - AID) ===")
for k, (n, b, *m) in TAR_GMI.items():
print(f"GMI {k:8s} m1 {b - m[0]:5.1f} m2 {b - m[1]:5.1f} m3 {b - m[2]:5.1f} mean {b - np.mean(m):6.2f}")
# ---------------------------------------------------------------- 5. search for 13.2
print("\n=== 5. Every TIR/TAR change in Table 1 within 0.15 of 13.2 (any stratum, month, or mean) ===")
hits = []
for metric, tab, sign in [("TIR", TIR_GMI, 1), ("TIR", TIR_AGE, 1), ("TIR", TIR_SEX, 1),
("TAR>180", TAR_GMI, -1), ("TAR>180", TAR_AGE, -1),
("TAR>180", TAR_SEX, -1)]:
for k, (n, b, *m) in tab.items():
vals = [sign * (x - b) for x in m] + [sign * (np.mean(m) - b)]
for lab, v in zip(["m1", "m2", "m3", "mean"], vals):
if abs(round(v, 2) - 13.2) <= 0.15:
hits.append((metric, k, lab, round(v, 2)))
print(hits if hits else "none")
gmi_hits = [h for h in hits if h[1] in TIR_GMI]
print("of which in a baseline-GMI stratum:", gmi_hits if gmi_hits else "none")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: abstract vs body text-panel diagram
fig = plt.figure(figsize=(7.2, 4.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
def panel(x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.02, y + h - 0.035, title, fontsize=11, weight="bold", va="top", color=TXT)
yy = y + h - 0.115
for txt, col, wt in lines:
ax.text(x + 0.02, yy, txt, fontsize=10, va="top", color=col, weight=wt)
yy -= 0.068
gm = TIR_GMI[">8.0"]
panel(0.03, 0.40, 0.40, 0.50, "Abstract (Results)",
[("“...those with baseline GMI >8.0", TXT, "normal"),
("(TIR improvement", TXT, "normal"),
("13.2 percentage points).”", ORANGE, "bold")], ORANGE)
panel(0.55, 0.40, 0.42, 0.50, "Table 1, GMI >8.0 (N = 4,064)",
[(f"% TIR, PLGS: {gm[1]}", TXT, "normal"),
(f"AID months 1–3: {gm[2]} / {gm[3]} / {gm[4]}", TXT, "normal"),
(f"Change: {gm[2]-gm[1]:.1f} / {gm[3]-gm[1]:.1f} / {gm[4]-gm[1]:.1f}", BLUE, "bold"),
(f"Mean of months: {G['>8.0']:.1f}", BLUE, "bold"),
("(changes recomputed from the table)", SEC, "normal")], BLUE)
arr = FancyArrowPatch((0.45, 0.65), (0.53, 0.65), arrowstyle="<->", mutation_scale=14, lw=2, color=SEC)
ax.add_patch(arr)
ax.text(0.49, 0.71, "differ by", ha="center", fontsize=10, color=SEC)
ax.text(0.49, 0.585, f"{G['>8.0'] - ABS_GMI8:.1f} pts", ha="center", fontsize=10, color=SEC, weight="bold")
ax.add_patch(FancyBboxPatch((0.03, 0.03), 0.94, 0.28, boxstyle="round,pad=0.008,rounding_size=0.015",
fc="white", ec=GRID, lw=1.2))
ax.text(0.05, 0.27, "Elsewhere in the same paper (agree with Table 1)", fontsize=11, weight="bold", va="top")
ax.text(0.05, 0.19, "Results: “...baseline GMI >8.0 had TIR improved by >20 percentage points”",
fontsize=10, va="top")
ax.text(0.05, 0.125, "Conclusions: “...the greatest percent improvement in TIR with a 22–23",
fontsize=10, va="top")
ax.text(0.05, 0.075, " percentage point change”", fontsize=10, va="top")
save(fig, "fig1")
# Figure 2: TIR change by stratum, abstract values vs recomputed
labels = ["GMI ≤6.9", "GMI 6.9–7.4", "GMI 7.4–8.0", "GMI >8.0", "Age <18 (pooled)"]
rec = [G["≤6.9"], G["6.9–7.4"], G["7.4–8.0"], G[">8.0"], U18]
rep = {3: ABS_GMI8, 4: ABS_U18}
fig, ax = plt.subplots(figsize=(7.2, 3.6), facecolor=BG)
ax.set_facecolor(BG)
y = np.arange(len(labels))[::-1]
for yi, v in zip(y, rec):
ax.plot([0, v], [yi, yi], color=GRID, lw=2, zorder=1)
ax.scatter(rec, y, s=80, color=BLUE, zorder=3, label="Recomputed from Table 1 (mean of AID months − PLGS)")
for i, v in rep.items():
ax.scatter([v], [y[i]], s=80, marker="D", facecolor=ORANGE, edgecolor=ORANGE, zorder=4,
label="Reported in abstract" if i == 3 else None)
for i, (yi, v) in enumerate(zip(y, rec)):
if i == 4:
ax.text(v - 0.5, yi - 0.12, f"{v:.1f}", color=BLUE, fontsize=10, va="top", ha="right")
else:
ax.text(v + 0.5, yi + 0.12, f"{v:.1f}", color=BLUE, fontsize=10, va="bottom")
for i, v in rep.items():
if i == 4:
ax.text(v + 0.5, y[i] + 0.12, f"{v:.1f}", color=ORANGE, fontsize=10, va="bottom")
else:
ax.text(v - 0.5, y[i] + 0.12, f"{v:.1f}", color=ORANGE, fontsize=10, va="bottom", ha="right")
ax.set_yticks(y); ax.set_yticklabels(labels, color=TXT)
ax.set_xlim(0, 25); ax.set_ylim(-0.6, len(labels) - 0.3)
ax.set_xlabel("Change in % time in range, AID vs PLGS (percentage points)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
fig.legend(loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=2, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
ax.set_ylim(-0.8, len(labels) - 0.4)
fig.tight_layout(rect=(0, 0, 1, 0.9))
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png")
Reported p values for HbA1c and other secondary outcomes contradict their own confidence intervals
Diabetologia (2025) · doi:10.1007/s00125-025-06585-2 · PMC12779728
Summary
This 76-participant randomised trial compared video consultations with in-person care in adults with type 1 diabetes who use insulin pumps. Table 2 gives the between-group difference in HbA1c change as −2.01 mmol/mol with a 95% CI of −5.32 to 1.29 and p < 0.001, and the Abstract and the Discussion present a favourable, statistically significant HbA1c effect. That interval includes zero and implies a two-sided p of about 0.23. A p below 0.001 would need a standard error of at most 0.61, about a third (36%) of the 1.69 implied by the printed interval. The same mismatch between interval and p value appears in every other Table 2 row that has a p value, including the treatment-satisfaction and quality-of-life outcomes the Abstract calls superior and inferior. As printed, none of the secondary-outcome claims of statistical significance is supported by the paper's own intervals. The primary result for time in range, no clinically meaningful difference, is not affected.
What the paper reports
Table 2, "Primary and secondary endpoints" (columns: endpoint | video group | control group | difference (95% CI) | p value). The footer reads "Estimates are reported as least squares means ± SEs".
Change in HbA1c, mmol/mol | -4.13 ± 1.43 | -2.12 ± 1.38 | -2.01 (-5.32, 1.29) | <0.001
Change in HbA1c, % | -0.38 ± 0.13 | -0.19 ± 0.13 | -0.18 (-0.49, 0.12) |
The other Table 2 rows that have a p value:
TiR (3.9–10.0 mmol/l), % | 64.28 ± 2.91 | 63.49 ± 2.50 | 0.8 (-5.30, 6.89) | 0.251
DTQSc (range -18 to 18) | 10.23 ± 1.61 | 9.01 ± 1.42 | 1.22 (-2.51, 4.96) | 0.004
DTQSs (range 0–36) | 30.06 ± 1.25 | 28.09 ± 1.13 | 1.98 (-0.98, 4.93) | <0.001
ADDQoL19 (range -9 to +3) | -1.6 ± 0.33 | -1.33 ± 0.29 | -0.27 (-1.07, 0.53) | 0.004
Methods, "Statistical analysis":
All 95% CIs and p values are two-sided.
the key secondary outcomes were analysed in a prioritised order (i.e. ‘gatekeeping procedure’) [17].
The statistical tests were performed and interpreted in sequence until one of the analyses failed to show a statistically significant difference at a statistical significance level of 0.05
Missing data were imputed using retrieved participants from the same randomised treatment group, with results combined according to Rubin’s rule [18]. The main analyses are reported as least squares means and SEs, with differences between groups and corresponding 95% CIs adjusted for stratification and baseline outcome measurements.
Results, second paragraph. The text repeats the intervals but gives no p values for the secondary outcomes:
the observed difference of 0.8 percentage points (95% CI -5.3, 6.9; p=0.25)
the intervention group showed better treatment satisfaction compared with the controls, with a potential difference of 1.22 points for DTSQc (95% CI -2.51, 4.96) and 1.98 points for DTSQs (95% CI -0.98, 4.93)
Regarding glycaemic control, only HbA1c demonstrated a favourable difference, being lower in the intervention group than in the control group, with a mean difference of -2.01 mmol/mol (95% CI -5.32, 1.29) (-0.18%; -0.49%, 0.12%).
Results, "Sensitivity analyses and adherence to protocol":
No discrepancies were observed between the main findings and those from sensitivity analyses for primary or secondary endpoints
Abstract, Results:
For secondary outcomes, the video group was superior in terms of treatment satisfaction and reduction in HbA1c. However, the video group experienced an inferior impact on quality of life.
Abstract, Conclusions/interpretation:
However, compared with control, the intervention was associated with superior treatment satisfaction and a favourable effect on HbA1c, albeit with an inferior impact on quality of life.
Discussion, first paragraph:
However, although a statistically significant difference was found for several of the secondary outcomes
Discussion, second paragraph:
We did find a statistically significant improvement in HbA1c, albeit not necessarily clinically relevant, in the video consultation group.
Discussion, on quality of life:
Hence, one might speculate that although statistically different, the finding might not be clinically meaningful
Discussion, concluding paragraph:
However, better treatment satisfaction and a greater reduction in HbA1c were seen, although with a greater negative impact on quality of life.
What the paper's own numbers show
Method. The Methods describe a two-sided Wald-type comparison: an ANCOVA difference in least squares means with a 95% CI. For such a comparison the CI and the p value come from the same estimate and standard error, so each one fixes the other. For each Table 2 row I took the SE as the CI width divided by 2 × 1.96, computed z = difference / SE and the two-sided normal p value, and compared it with the printed p. As a second, independent check I computed the SE of the difference from the two arm SEs, assuming the arm estimates are uncorrelated, as they are in a randomised parallel-group trial. I also asked what SE and CI the printed p would need, and repeated the CI-to-p calculation for the sensitivity analyses in ESM Tables 1 and 2. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| HbA1c difference, video − control (mmol/mol) | Table 2; Results | −2.01 (95% CI −5.32, 1.29) | LS means −4.13 − (−2.12) = −2.01; CI midpoint −2.015, so the interval is centred on the estimate |
| SE of the HbA1c difference, from the CI | Table 2 | — | half-width 3.305; SE = 3.305 / 1.96 = 1.686 |
| HbA1c two-sided p, from the CI | Table 2 | <0.001 | z = −1.19; p = 0.233 (t with 60 or 70 df: 0.24) |
| HbA1c p, from the arm SEs | Table 2 | <0.001 | SE = √(1.43² + 1.38²) = 1.987; z = −1.01; p = 0.31; 95% CI (−5.91, 1.89) |
| HbA1c one-sided p | derived | — (Methods: all p values two-sided) | 0.117 |
| SE and CI that p = 0.001 would require | derived | — | SE ≤ 0.611, CI half-width ≤ 1.20, i.e. about (−3.21, −0.81), against a printed half-width of 3.305 |
| HbA1c difference, % scale | Table 2 | −0.18 (−0.49, 0.12), p cell blank | SE 0.156, p = 0.25; −2.01 / 10.929 = −0.184 (−0.487, 0.118), so the two scales agree with each other |
| TiR (primary) | Table 2; Abstract; Results | 0.8 (−5.30, 6.89), p = 0.251 | SE 3.110, z = 0.26, p = 0.80; p = 0.251 would need a CI half-width of 1.37, not 6.09 |
| DTSQc | Table 2 | 1.22 (−2.51, 4.96), p = 0.004 | SE 1.906, z = 0.64, p = 0.52 |
| DTSQs | Table 2 | 1.98 (−0.98, 4.93), p < 0.001 | SE 1.508, z = 1.31, p = 0.19 |
| ADDQoL19 | Table 2 | −0.27 (−1.07, 0.53), p = 0.004 | SE 0.408, z = −0.66, p = 0.51 |
| TbR < 3.0 mmol/l | Table 2 | 0.05 (−0.34, 0.44), p = 0.200 | p = 0.80 |
| Sensitivity analyses with a printed p | ESM Table 1 (per protocol); ESM Table 2 (baseline carried forward) | ESM T1 TiR 3.69 (−3.36, 10.75), p 0.294; ESM T2 TiR −0.56 (−5.49, 4.36), p 0.819; ESM T1 DTSQc 1.30 (−4.79, 7.39), p 0.658; ESM T2 DTSQc 0.94 (−3.57, 5.45), p 0.674 | in the same order: p = 0.305, 0.824, 0.676 and 0.683; these agree with their CIs to within 0.02 |
| HbA1c in the sensitivity analyses | ESM Table 1 (per protocol); ESM Table 2 (baseline carried forward) | −3.42 (−7.54, 0.69); −1.31 (−4.02, 1.4); p shown as "-" | p = 0.10 and 0.34; both intervals include zero |
All six Table 2 rows with both a printed p and a 95% CI are inconsistent with themselves, and in each case the printed p is smaller than the interval allows. Four of the six fall below 0.05 although every one of the six intervals includes zero. By contrast, the four p values printed in the ESM sensitivity tables match their intervals.
Side observation. In four Table 2 rows without a p value, the printed difference does not equal video minus control. TbR 3.0–3.8 mmol/l: 1.70 − 1.46 = 0.24, printed −0.20. TaR > 13.9 mmol/l: −1.54, printed 0.35. TaR 10.1–13.9 mmol/l: −0.88, printed −2.83. Glycaemic variability: 36.86 − 36.86 = 0.00, printed 2.24. In each case the printed CI is centred on the printed difference, not on the difference of the LS means. These rows do not feed the claims discussed here, but they suggest the Table 2 entries as a whole need checking.
Figure
Figure 1. For the primary outcome and the four secondary outcomes that the paper's conclusions rest on, the orange circle is the printed p (a left-pointing triangle marks "<0.001", which is an upper bound, drawn at 0.001), and the blue diamond is the two-sided p recomputed from the printed CI as described above. The axis is logarithmic, and the vertical line marks p = 0.05. Every recomputed value lies above 0.05, between 0.19 and 0.80, while four of the printed values lie below it. The gap is largest for HbA1c and DTSQs, where the printed p is at least 190 to 230 times smaller than the interval implies. The sixth row with a p value, TbR < 3.0 mmol/l (printed 0.200, recomputed 0.80), is not drawn. All values are from Table 2 or from recompute.py.txt.
Figure 2. The HbA1c difference on its own scale (mmol/mol, video − control). The orange interval is the printed 95% CI (−5.32, 1.29). The solid blue interval is the CI implied by the two arm SEs, 1.43 and 1.38 (−5.91, 1.89; p = 0.31). The dashed blue interval is hypothetical: the 95% CI that a p of 0.001 would require around the same estimate (−3.21, −0.81). Both intervals built from the paper's numbers cross zero (vertical line). Only an interval less than half as wide as the printed one, and entirely below zero, would be consistent with p < 0.001.
Why it matters
The trial's primary outcome, time in range, showed no meaningful difference, and that conclusion is unaffected: the CI (−5.30, 6.89) and the stated decision rule both point the same way, whichever p value is right. The paper's positive findings all come from the secondary outcomes. The Abstract reports that the video group "was superior in terms of treatment satisfaction and reduction in HbA1c" and had "an inferior impact on quality of life", and the Abstract conclusion and several Discussion paragraphs, including the concluding one, repeat these findings. The Discussion explicitly calls the HbA1c improvement "statistically significant" and the quality-of-life difference "statistically different". Those characterisations match the printed p values (<0.001, <0.001, 0.004, 0.004) but not the printed intervals, all of which include zero and imply p between 0.19 and 0.52. On the intervals, none of the secondary differences is statistically significant at 0.05. That includes the % HbA1c row, which has no p value but whose CI (−0.49, 0.12) gives p ≈ 0.25.
The paper's own gatekeeping rule points the same way. The Methods say the key secondary outcomes were analysed in a prioritised order, "listed in order under 'Outcomes and data collection'", and that the tests were interpreted in sequence until one failed at 0.05. The first outcome listed there is DTSQc, whose interval (−2.51, 4.96) includes zero and implies p = 0.52. On the intervals, the sequence therefore stops at its first step, and none of the later secondary outcomes (DTSQs, ADDQoL19, HbA1c) would be interpreted as confirmatory. If, as is usual, the sequence starts with the primary outcome, it would stop even earlier, since time in range was not significant either (printed p = 0.251; p from the CI 0.80).
Readers of the Abstract are therefore told of a favourable HbA1c effect and of superior treatment satisfaction that the reported data do not show to be different from no effect. A meta-analysis that took the HbA1c result from the text or CI would treat it as null. A reader or guideline that relied on the Abstract would treat it as positive.
Explanations considered
- Conversion or typesetting error in PMC. Ruled out. The publisher's version of record shows the same Table 2 values, including "<0.001" for HbA1c and a blank p cell for the % row, as do the PMC page and the Europe PMC XML.
- Rounding. Cannot account for the gap. The recomputed p values (0.19 to 0.80) differ from the printed ones (<0.001 to 0.251) by factors of 3 to more than 200, which rounding of the CI limits to two decimals cannot produce. The HbA1c interval is centred on its estimate (midpoint −2.015), and its % version converts exactly from the mmol/mol version, so the interval itself is not garbled.
- t rather than normal reference distribution. With 60 to 70 degrees of freedom the HbA1c p is 0.24, slightly larger, not smaller.
- One-sided test. The Methods say all p values are two-sided. A one-sided HbA1c p would still be 0.117.
- Multiplicity adjustment applied to CIs but not p values, or the reverse. The Methods say no multiplicity adjustment was applied. An adjustment would in any case raise p values, not lower them.
- Correlated arm estimates. Taking the arm SEs at face value makes the inconsistency worse (p = 0.31). For the SE of the difference to shrink to 0.61, the two arms' LS means would have to be correlated at about 0.91, which is not plausible for independently randomised parallel groups.
- Multiple imputation (Rubin's rules) affecting the CI but not the p value. For p < 0.001 to be right while the CI is also right, about 87% of the CI's variance would have to come from between-imputation variability. Only 13 of 76 participants (17%) did not complete the trial. The per-protocol analysis, which uses no imputation, also gives an HbA1c interval that includes zero (−7.54, 0.69; p ≈ 0.10).
- The p column reports a different test. For example, the p value could be for a within-group change from baseline, or for another term in the ANCOVA model. The video arm's within-group HbA1c change gives p = 0.004 (−4.13 / 1.43), which is still not <0.001. The other possibilities cannot be checked without the data. In any case the table pairs each p value with a between-group difference and CI, and the text interprets the p values as tests of that difference, so the inconsistency in the reported results would remain.
- The CI is wrong and the p value is right. This cannot be excluded without the data, but the paper's other numbers argue against it. The arm SEs independently imply an even wider interval. The interval is repeated in the Results text on two scales that agree with each other. Both sensitivity analyses give HbA1c intervals that include zero, and the Results state that they agree with the main analysis. The p values printed in the ESM tables also match their own CIs, so the mismatch is confined to the main Table 2.
None of these explanations reconciles the printed p values with the printed intervals.
Status
As of 2026-10-23:
- Version of record. On 2026-10-23 the publisher's Table 2 page (link.springer.com/article/10.1007/s00125-025-06585-2/tables/2) shows "Change in HbA1c, mmol/mol | −4.13 ± 1.43 | −2.12 ± 1.38 | −2.01 (−5.32, 1.29) | <0.001", the % row with a blank p cell, and the same values in the other rows, including the four differences that do not match their LS means. The publisher's article page carries the same Abstract and Discussion sentences. The article was published online on 7 November 2025 in Diabetologia 69(2):321–329 (February 2026). The PMC page and the Europe PMC full-text XML give identical values. Every quote above was checked verbatim against the PMC-archived text with
src/show.py, which renders minus signs as hyphens. - Supplements. The single ESM file (125_2025_6585_MOESM1_ESM.pdf, "ESM Tables", 5 pages) was downloaded from static-content.springer.com and read in full. ESM Tables 1 and 2 (sensitivity analyses) give HbA1c intervals that include zero, with no p value. ESM Table 4 (subgroups) gives HbA1c intervals that all include zero. No other HbA1c p value appears anywhere. Patient-level data are not public; the Data availability statement says they are available from the corresponding author on reasonable request.
- Corrections.
src/known.pylists no Crossref update relations and no PMC or Europe PMC correction, retraction or expression-of-concern notice. A Crossreffilter=updates:query for the DOI returns 0 results. - PubPeer. The PubPeer API and site returned HTTP 403 to automated requests on 2026-10-23, so PubPeer could not be checked directly. A web search for the title together with PubPeer found no thread.
Sources: PMC12779728 (PubMed Central archive, read with src/show.py); https://doi.org/10.1007/s00125-025-06585-2 and the publisher's Table 2 page; ESM file https://static-content.springer.com/esm/art%3A10.1007%2Fs00125-025-06585-2/MediaObjects/125_2025_6585_MOESM1_ESM.pdf; Europe PMC full-text XML; https://api.crossref.org/works?filter=updates:10.1007/s00125-025-06585-2; ClinicalTrials.gov NCT04612933 (registry entry, not used for any number here).
Suggested correction
- Re-derive the between-group p values for every Table 2 row from the same ANCOVA and multiple-imputation model that produced the differences and CIs, and correct the p column. Add the p value for the % HbA1c row, or state that it is the same test as the mmol/mol row.
- Revise the statements that depend on statistical significance of the secondary outcomes so that they match the corrected p values and the intervals. These are the Abstract Results and Conclusions/interpretation, and the first, second, quality-of-life and concluding paragraphs of the Discussion. If the intervals are correct, these differences should be described as not statistically significant, with the CI given.
- State how the gatekeeping procedure was applied: whether the sequence starts with the primary outcome, and how the secondary outcomes were interpreted given that the first listed key secondary outcome, DTSQc, has an interval that includes zero (−2.51, 4.96).
- Check the difference column for TbR 3.0–3.8 mmol/l, both TaR rows and glycaemic variability, where the printed difference does not equal video minus control.
Recomputation script (python recompute.py.txt)
"""Recompute the Table 2 p values of PMC12779728
(Diabetologia 2025, doi:10.1007/s00125-025-06585-2).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file, plus five candidate
figure designs (cand_A ... cand_E) and a contact sheet in ./candidates/.
Inputs are copied from the paper as archived in PubMed Central (identical in the
publisher's version of record) and from the single ESM file (ESM Tables 1 and 2).
Table 2 reports least squares means +- SE for each arm, the between-group difference
with its 95% CI, and a p value. Methods: "All 95% CIs and p values are two-sided."
"""
import os
import re
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z975 = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (Table 2)
# name, LS mean video, SE, LS mean control, SE, difference, CI low, CI high, printed p
TABLE2 = [
("TiR (primary), %", 64.28, 2.91, 63.49, 2.50, 0.80, -5.30, 6.89, "0.251"),
("DTSQc", 10.23, 1.61, 9.01, 1.42, 1.22, -2.51, 4.96, "0.004"),
("DTSQs", 30.06, 1.25, 28.09, 1.13, 1.98, -0.98, 4.93, "<0.001"),
("ADDQoL19", -1.60, 0.33, -1.33, 0.29, -0.27, -1.07, 0.53, "0.004"),
("Change in HbA1c, mmol/mol", -4.13, 1.43, -2.12, 1.38, -2.01, -5.32, 1.29, "<0.001"),
("Change in HbA1c, %", -0.38, 0.13, -0.19, 0.13, -0.18, -0.49, 0.12, ""),
("TbR <3.0, %", 0.44, 0.18, 0.39, 0.16, 0.05, -0.34, 0.44, "0.200"),
("TbR 3.0-3.8, %", 1.70, 0.24, 1.46, 0.21, -0.20, -0.71, 0.31, "-"),
("TaR >13.9, %", 8.92, 2.29, 10.46, 1.87, 0.35, -3.04, 3.75, "-"),
("TaR 10.1-13.9, %", 24.30, 2.31, 25.18, 1.86, -2.83, -6.40, 0.74, "-"),
("Glycaemic variability, %", 36.86, 1.70, 36.86, 1.44, 2.24, -0.43, 4.90, "-"),
]
# ESM Tables 1 (per protocol) and 2 (ITT, baseline observation carried forward):
# rows that print a p value, plus the HbA1c rows (p printed as "-").
ESM = [
("ESM T1 per protocol, TiR", 3.69, -3.36, 10.75, "0.294"),
("ESM T1 per protocol, DTSQc", 1.30, -4.79, 7.39, "0.658"),
("ESM T2 BOCF, TiR", -0.56, -5.49, 4.36, "0.819"),
("ESM T2 BOCF, DTSQc", 0.94, -3.57, 5.45, "0.674"),
("ESM T1 per protocol, HbA1c mmol/mol", -3.42, -7.54, 0.69, "-"),
("ESM T2 BOCF, HbA1c mmol/mol", -1.31, -4.02, 1.40, "-"),
]
N_RAND, N_COMPLETE = 76, 32 + 31 # Abstract: 76 randomised, 32 + 31 completed
IFCC_PER_NGSP = 10.929 # mmol/mol per 1 percentage point of HbA1c
def se_from_ci(lo, hi):
return (hi - lo) / (2 * Z975)
def p_two(z):
return 2 * stats.norm.sf(abs(z))
def z_from_p(p):
return stats.norm.isf(p / 2)
def p_value_of(printed):
return float(printed.lstrip("<"))
# ---------------------------------------------------------------- 1. every Table 2 row
print("1. Table 2: p value implied by each row's own 95% CI and by its arm SEs (normal)")
print(f"{'endpoint':27s} {'V-C':>6s} {'diff':>6s} {'CImid':>6s} {'SE_CI':>6s} {'z_CI':>6s}"
f" {'p_CI':>6s} {'SE_arm':>6s} {'p_arm':>6s} {'p_printed':>9s}")
REC = {}
for n, mv, sv, mc, sc, d, lo, hi, p in TABLE2:
se_ci = se_from_ci(lo, hi)
z_ci = d / se_ci
se_arm = float(np.hypot(sv, sc))
p_arm = p_two((mv - mc) / se_arm)
REC[n] = dict(d=d, lo=lo, hi=hi, se=se_ci, z=z_ci, p=p_two(z_ci), se_arm=se_arm,
p_arm=p_arm, lsdiff=mv - mc, printed=p)
print(f"{n:27s} {mv - mc:6.2f} {d:6.2f} {(lo + hi) / 2:6.2f} {se_ci:6.3f} {z_ci:6.2f}"
f" {p_two(z_ci):6.3f} {se_arm:6.3f} {p_arm:6.3f} {p:>9s}")
TESTED = ["TiR (primary), %", "DTSQc", "DTSQs", "ADDQoL19", "Change in HbA1c, mmol/mol"]
print("\n |z| implied by the printed p (largest p allowed by '<'):")
for n in TESTED:
r = REC[n]
zp = z_from_p(p_value_of(r["printed"]))
print(f" {n:27s} printed p {r['printed']:>6s} -> |z| {'>=' if r['printed'].startswith('<') else '='}"
f" {zp:.2f}; CI gives |z| = {abs(r['z']):.2f}; SE the printed p needs = {abs(r['d']) / zp:.3f}"
f" (CI half-width {abs(r['d']) / zp * Z975:.2f}, printed {(r['hi'] - r['lo']) / 2:.2f})")
# ---------------------------------------------------------------- 2. HbA1c in detail
h = REC["Change in HbA1c, mmol/mol"]
d, lo, hi = h["d"], h["lo"], h["hi"]
print("\n2. HbA1c, mmol/mol")
print(f" LS means: -4.13 - (-2.12) = {-4.13 - -2.12:.2f}; CI midpoint {(lo + hi) / 2:.3f}")
print(f" CI half-width {(hi - lo) / 2:.3f}; SE {h['se']:.3f}; z {h['z']:.3f}; "
f"two-sided p {h['p']:.3f}; one-sided p {h['p'] / 2:.3f}")
for df in (60, 70):
print(f" t with df {df}: two-sided p {2 * stats.t.sf(abs(h['z']), df):.3f}")
print(f" SE from arm SEs sqrt(1.43^2+1.38^2) = {h['se_arm']:.3f}; z {d / h['se_arm']:.3f}; p {h['p_arm']:.3f};"
f" 95% CI ({d - Z975 * h['se_arm']:.2f}, {d + Z975 * h['se_arm']:.2f})")
z001 = z_from_p(0.001)
se_need = abs(d) / z001
HB_NEED = (d - Z975 * se_need, d + Z975 * se_need)
print(f" p = 0.001 needs |z| {z001:.3f}, SE {se_need:.3f}, CI half-width {se_need * Z975:.2f},"
f" i.e. 95% CI ({HB_NEED[0]:.2f}, {HB_NEED[1]:.2f})")
print(f" SE ratio printed CI / needed: {h['se'] / se_need:.2f}; needed / printed {se_need / h['se']:.3f}"
f" ({100 * se_need / h['se']:.0f}%); variance ratio {(h['se'] / se_need) ** 2:.1f}")
rho = (1.43 ** 2 + 1.38 ** 2 - se_need ** 2) / (2 * 1.43 * 1.38)
print(f" correlation between the two LS means needed to shrink the arm-SE difference SE to {se_need:.3f}: {rho:.2f}")
miss = 1 - N_COMPLETE / N_RAND
print(f" missing outcome share {N_RAND - N_COMPLETE}/{N_RAND} = {miss:.3f}; share of the CI's variance that "
f"would have to come from imputation alone for p<0.001: {1 - (se_need / h['se']) ** 2:.3f}")
print(f" within-group z (LS mean / SE): video {-4.13 / 1.43:.2f}, p {p_two(-4.13 / 1.43):.4f};"
f" control {-2.12 / 1.38:.2f}, p {p_two(-2.12 / 1.38):.3f}")
hp = REC["Change in HbA1c, %"]
print(f" % row: SE {hp['se']:.3f}, z {hp['z']:.2f}, p {hp['p']:.3f}")
print(f" mmol/mol -> % (/ {IFCC_PER_NGSP}): diff {d / IFCC_PER_NGSP:.3f}, CI ({lo / IFCC_PER_NGSP:.3f},"
f" {hi / IFCC_PER_NGSP:.3f}); arms {-4.13 / IFCC_PER_NGSP:.3f}, {-2.12 / IFCC_PER_NGSP:.3f}")
# ---------------------------------------------------------------- 3. ESM rows
print("\n3. ESM sensitivity tables: p implied by the printed CI")
for n, dd, l, u, p in ESM:
se = se_from_ci(l, u)
print(f" {n:38s} {dd:6.2f} ({l}, {u}) SE {se:.3f} p_CI {p_two(dd / se):.3f} printed {p}")
# ---------------------------------------------------------------- 4. side observation
print("\n4. Printed differences that do not equal video minus control LS means")
for n in ["TbR 3.0-3.8, %", "TaR >13.9, %", "TaR 10.1-13.9, %", "Glycaemic variability, %"]:
r = REC[n]
print(f" {n:27s} video - control = {r['lsdiff']:6.2f}; printed difference {r['d']:6.2f}"
f" (CI midpoint {(r['lo'] + r['hi']) / 2:.2f})")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"axes.spines.top": False, "axes.spines.right": False, "svg.fonttype": "none",
})
LABELS = {"TiR (primary), %": "TiR, % (primary)", "DTSQc": "DTSQc", "DTSQs": "DTSQs",
"ADDQoL19": "ADDQoL19", "Change in HbA1c, mmol/mol": "HbA1c change, mmol/mol"}
MS = 9 # marker size in points (>= 8 px)
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight")
fig.savefig(p + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def fmt_p(p):
return f"{p:.2f}" if p >= 0.01 else f"{p:.3f}"
def mn(text):
"""Typographic minus signs for numbers drawn in figures."""
return re.sub(r"(?<![A-Za-z])-(?=\d)", "\u2212", text)
MASK = dict(boxstyle="square,pad=0.1", fc=BG, ec="none")
def design_A(paths):
"""Dot-and-interval (dumbbell) on one log p axis, five tested rows."""
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.arange(len(TESTED))[::-1]
for y, n in zip(ys, TESTED):
r = REC[n]
pr = p_value_of(r["printed"])
ax.plot([pr, r["p"]], [y, y], color=INK2, lw=1, alpha=0.5, zorder=1)
mk = "<" if r["printed"].startswith("<") else "o"
ax.plot(pr, y, marker=mk, ms=MS + 1, color=ORANGE, mec=BG, mew=1.5, ls="none", zorder=3)
ax.plot(r["p"], y, marker="D", ms=MS - 1, color=BLUE, mec=BG, mew=1.5, ls="none", zorder=3)
ax.annotate(f"printed {r['printed']}", (pr, y), xytext=(-8, 9), textcoords="offset points",
ha="right", fontsize=10, color=INK, bbox=MASK, zorder=4)
ax.annotate(f"from CI {fmt_p(r['p'])}", (r["p"], y), xytext=(8, 9), textcoords="offset points",
ha="left", fontsize=10, color=INK, bbox=MASK, zorder=4)
ax.axvline(0.05, color=INK2, lw=1, zorder=0)
ax.text(0.05, len(TESTED) - 0.35, "p = 0.05", ha="center", va="bottom", fontsize=10, color=INK2, bbox=MASK)
ax.set_xscale("log")
ax.set_xlim(3e-4, 3)
ax.set_xticks([0.001, 0.01, 0.05, 0.1, 0.5, 1])
ax.set_xticklabels(["0.001", "0.01", "0.05", "0.1", "0.5", "1"])
ax.set_yticks(ys)
ax.set_yticklabels([LABELS[n] for n in TESTED], color=INK)
ax.set_ylim(-0.6, len(TESTED) - 0.1)
ax.set_xlabel("Two-sided p value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
h1 = plt.Line2D([], [], marker="o", color=ORANGE, ls="none", ms=MS, label="p printed in Table 2 ('<' = upper bound)")
h2 = plt.Line2D([], [], marker="D", color=BLUE, ls="none", ms=MS - 1, label="p recomputed from the row's own 95% CI")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
ax.set_title("Table 2: printed p values against the p values their 95% CIs imply",
loc="left", fontsize=11.5, color=INK)
save(fig, *paths)
def design_B(paths):
"""HbA1c only: printed CI vs CI from arm SEs vs the CI that p<0.001 would need."""
fig, ax = plt.subplots(figsize=(7.2, 3.2))
rows = [
("Printed in Table 2", d, lo, hi, ORANGE, "-", "o", f"printed p <0.001; this CI implies p = {fmt_p(h['p'])}"),
("From the arm SEs\n(1.43 and 1.38)", d, d - Z975 * h["se_arm"], d + Z975 * h["se_arm"], BLUE, "-", "D",
f"p = {fmt_p(h['p_arm'])}"),
("Needed for p = 0.001", d, HB_NEED[0], HB_NEED[1], BLUE, (0, (4, 2)), "D", "hypothetical: half-width 1.20"),
]
for i, (lab, c, l, u, col, ls, mk, note) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([l, u], [y, y], color=col, lw=2, ls=ls, solid_capstyle="round")
mfc = col if i < 2 else BG
ax.plot(c, y, marker=mk, ms=MS, color=col, mfc=mfc, mec=col, mew=2, ls="none")
ax.text(u + 0.15, y, mn(f"({l:.2f}, {u:.2f})"), va="center", fontsize=10, color=INK, bbox=MASK, zorder=4)
ax.text(l, y + 0.2, note, ha="left", va="bottom", fontsize=10, color=INK2, bbox=MASK, zorder=4)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0.05, -0.55, "no difference", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(range(len(rows))[::-1])
ax.set_yticklabels([r[0] for r in rows], color=INK)
ax.set_ylim(-0.6, len(rows) - 0.2)
ax.set_xlim(-7, 4.2)
ax.set_xlabel("Difference in HbA1c change, video − control (mmol/mol), 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
h1 = plt.Line2D([], [], marker="o", color=ORANGE, lw=2, ms=MS, label="printed in the paper")
h2 = plt.Line2D([], [], marker="D", color=BLUE, lw=2, ms=MS - 1, label="recomputed from the paper's numbers")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2, frameon=False, fontsize=10)
ax.set_title("HbA1c difference −2.01 mmol/mol: the printed CI crosses zero", loc="left", fontsize=11.5)
save(fig, *paths)
def design_C(paths):
"""Same five rows on a linear |z| axis: z the printed p requires vs z the CI gives."""
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.arange(len(TESTED))[::-1]
for y, n in zip(ys, TESTED):
r = REC[n]
zp = z_from_p(p_value_of(r["printed"]))
zc = abs(r["z"])
ax.plot([zc, zp], [y, y], color=INK2, lw=1, alpha=0.5)
mk = ">" if r["printed"].startswith("<") else "o"
ax.plot(zp, y, marker=mk, ms=MS + 1, color=ORANGE, mec=BG, mew=1.5, ls="none", zorder=3)
ax.plot(zc, y, marker="D", ms=MS - 1, color=BLUE, mec=BG, mew=1.5, ls="none", zorder=3)
ax.annotate(f"{'≥' if mk == '>' else ''}{zp:.2f}", (zp, y), xytext=(0, 9), textcoords="offset points",
ha="center", fontsize=10)
ax.annotate(f"{zc:.2f}", (zc, y), xytext=(0, 9), textcoords="offset points", ha="center", fontsize=10)
ax.axvline(Z975, color=INK2, lw=1)
ax.text(Z975, len(TESTED) - 0.35, "|z| = 1.96 (p = 0.05)", ha="center", va="bottom", fontsize=10, color=INK2,
bbox=MASK)
ax.set_xlim(0, 4)
ax.set_yticks(ys)
ax.set_yticklabels([LABELS[n] for n in TESTED], color=INK)
ax.set_ylim(-0.6, len(TESTED) - 0.1)
ax.set_xlabel("|z| = |difference| / SE")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
h1 = plt.Line2D([], [], marker="o", color=ORANGE, ls="none", ms=MS, label="|z| the printed p requires ('>' = at least)")
h2 = plt.Line2D([], [], marker="D", color=BLUE, ls="none", ms=MS - 1, label="|z| the printed 95% CI gives")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
ax.set_title("Test statistic implied by each printed p value and by its CI", loc="left", fontsize=11.5)
save(fig, *paths)
def design_D(paths):
"""Graphic table: rows with the printed CI drawn in SE units, printed p and CI-implied p."""
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ys = np.arange(len(TESTED))[::-1]
for y, n in zip(ys, TESTED):
r = REC[n]
zc = r["z"]
ax.plot([zc - Z975, zc + Z975], [y, y], color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(zc, y, "o", ms=MS, color=ORANGE, mec=BG, mew=1.5)
ax.plot(5.2, y, "o", ms=MS, color=ORANGE, mec=BG)
ax.text(5.6, y, r["printed"], ha="left", va="center", fontsize=10.5, color=INK)
ax.plot(8.0, y, "D", ms=MS - 1, color=BLUE, mec=BG)
ax.text(8.4, y, fmt_p(r["p"]), ha="left", va="center", fontsize=10.5, color=INK, weight="bold")
ax.axvline(0, color=INK2, lw=1)
top = len(TESTED) - 0.45
for x, t in [(0, "95% CI in SE units"), (5.9, "Printed p"), (8.6, "p from CI")]:
ax.text(x, top, t, ha="center", va="bottom", fontsize=10, color=INK2, weight="bold", bbox=MASK)
ax.set_xlim(-3.6, 10.0)
ax.set_ylim(-0.6, len(TESTED) - 0.05)
ax.set_yticks(ys)
ax.set_yticklabels([mn(f"{LABELS[n]}\n{r['d']:.2f} ({r['lo']:.2f}, {r['hi']:.2f})")
for n, r in ((n, REC[n]) for n in TESTED)], color=INK, fontsize=10)
ax.set_xticks([-2, 0, 2])
ax.set_xticklabels(["−2", "0", "+2"])
ax.tick_params(axis="y", length=0)
for s in ("left", "bottom"):
ax.spines[s].set_visible(False)
ax.set_xlabel("CI drawn as (difference ÷ SE) ± 1.96; every interval crosses 0", fontsize=10, color=INK2)
h1 = plt.Line2D([], [], marker="o", color=ORANGE, lw=2, ms=MS, label="printed in Table 2")
h2 = plt.Line2D([], [], marker="D", color=BLUE, ls="none", ms=MS - 1, label="recomputed from the printed CI")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, *paths)
def design_E(paths):
"""Calculation flow from the printed HbA1c CI to its p value, set against the printed p and the claims."""
fig, ax = plt.subplots(figsize=(7.2, 4.4))
ax.set_xlim(0, 10)
ax.set_ylim(-0.1, 6.2)
ax.axis("off")
def box(x, y, w, hgt, text, edge, bold=False, fs=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, hgt, boxstyle="round,pad=0.02,rounding_size=0.12",
fc=BG, ec=edge, lw=2))
ax.text(x + w / 2, y + hgt / 2, text, ha="center", va="center", fontsize=fs, color=INK,
weight="bold" if bold else "normal", wrap=True)
def arrow(x0, y0, x1, y1, col=INK2):
ax.annotate("", (x1, y1), (x0, y0), arrowprops=dict(arrowstyle="-|>", color=col, lw=1.5))
box(0.1, 4.7, 3.1, 1.1, "Table 2, printed\n−2.01 (−5.32, 1.29)", ORANGE, bold=True)
box(3.6, 4.7, 2.9, 1.1, f"CI half-width\n(1.29 + 5.32) / 2\n= {(hi - lo) / 2:.3f}", BLUE, fs=10)
box(6.9, 4.7, 3.0, 1.1, f"SE = {(hi - lo) / 2:.3f} / 1.96\n= {h['se']:.3f}", BLUE)
box(6.9, 3.0, 3.0, 1.1, mn(f"z = −2.01 / {h['se']:.3f}\n= {h['z']:.2f}"), BLUE)
box(3.6, 3.0, 2.9, 1.1, f"two-sided p\n= {h['p']:.2f}", BLUE, bold=True)
box(0.1, 3.0, 3.1, 1.1, "Table 2, printed\np < 0.001", ORANGE, bold=True)
arrow(3.2, 5.25, 3.6, 5.25)
arrow(6.5, 5.25, 6.9, 5.25)
arrow(8.4, 4.7, 8.4, 4.1)
arrow(6.9, 3.55, 6.5, 3.55)
ax.text(3.4, 3.55, "≠", ha="center", va="center", fontsize=20, color=INK, weight="bold")
ax.text(1.6, 2.8, f"conflict: p < 0.001 needs SE ≤ {se_need:.2f},\n"
f"i.e. CI half-width ≤ {se_need * Z975:.2f} (printed {(hi - lo) / 2:.3f})",
ha="left", va="top", fontsize=10, color=INK)
box(0.1, 0.0, 9.8, 1.75,
"Claims that rest on the printed p\n"
"Abstract: “the video group was superior in terms of treatment satisfaction\nand reduction in HbA1c”\n"
"Discussion: “We did find a statistically significant improvement in HbA1c”", INK2, fs=10)
arrow(1.0, 3.0, 1.0, 1.7, ORANGE)
h1 = plt.Line2D([], [], color=ORANGE, lw=2, label="printed in the paper")
h2 = plt.Line2D([], [], color=BLUE, lw=2, label="recomputed from the paper's numbers")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, 1.06), ncol=2, frameon=False, fontsize=10)
save(fig, *paths)
design_A([os.path.join(HERE, "fig1"), os.path.join(CAND, "cand_A_pvalue_dumbbell")])
design_B([os.path.join(HERE, "fig2"), os.path.join(CAND, "cand_B_hba1c_intervals")])
design_C([os.path.join(CAND, "cand_C_z_dumbbell")])
design_D([os.path.join(CAND, "cand_D_graphic_table")])
design_E([os.path.join(CAND, "cand_E_calculation_flow")])
def contact_sheet(path):
"""Five candidate designs side by side, labelled A-E, for choosing one."""
names = ["cand_A_pvalue_dumbbell", "cand_B_hba1c_intervals", "cand_C_z_dumbbell",
"cand_D_graphic_table", "cand_E_calculation_flow"]
titles = ["A p values, log axis (fig1)", "B HbA1c intervals (fig2)", "C |z| statistic, linear axis",
"D graphic table, CI in SE units", "E calculation flow, HbA1c"]
fig, axs = plt.subplots(3, 2, figsize=(14, 15))
for ax in axs.flat:
ax.axis("off")
for ax, n, t in zip(axs.flat, names, titles):
ax.imshow(plt.imread(os.path.join(CAND, n + ".png")))
ax.set_title(t, loc="left", fontsize=13, weight="bold", color=INK)
fig.tight_layout()
fig.savefig(path, dpi=110, facecolor=BG)
plt.close(fig)
contact_sheet(os.path.join(CAND, "contact_sheet.png"))
print("\nFigures written: fig1, fig2, candidates/cand_A ... cand_E (.svg and .png) and candidates/contact_sheet.png")
Abstract reports a 2-point fall in IHD prevalence; the paper's Table 2 shows no change
Epidemiology (2026) · doi:10.1097/EDE.0000000000001932 · PMC12851546
Summary
The abstract and the Results say that a two-thirds cut in red and processed meat would lower the prevalence of ischemic heart disease (IHD) in 2050 by 2 percentage points (%pt), 95% prediction interval (PI) 1 to 4. They say a 100% increase in whole grain would lower it by 2%pt (PI 0 to 3). The paper's Table 2 gives an IHD difference of 0 (−1 to 1) for both scenarios, and its scenario and current-diet IHD levels are both 4%. The two figures in the text are exactly the Table 2 differences for the Healthy state (+2, 1 to 4 for meat; +2, 0 to 3 for whole grain), and the paper's own Discussion says these diets keep IHD prevalence constant. As a result, the abstract's IHD estimates, the meat estimate's PI that excludes zero, and the abstract conclusion that more plant-based diets reduce IHD prevalence are not supported by the paper's own projections. The mortality findings and the pooling method are not affected.
What the paper reports
Abstract, Results:
We estimated that a two-thirds reduction in red and processed meat consumption would decrease the prevalence of IHD by 2 (95% prediction intervals [PI] 1, 4) percentage points (%pt) in the 2017 cohort and deaths by 2%pt (95% PI 1, 4) by 2050. We estimated that a 100% increase in whole grain consumption would reduce IHD by 2%pt (95% PI 0, 3) and deaths by 2%pt (95% PI 0, 3).
Abstract, Conclusions:
Our estimates support the hypothesis that more plant-based diets reduce mortality and IHD prevalence.
Results, "Projections":
The large total scenario was associated with nearly 3 percentage points (%pt) increased prevalence of healthy individuals and over 4%pt lower deaths, but also increased the prevalence of IHD by 1 %pt. We estimate that a large reduction in meat consumption could decrease the prevalence of individuals with IHD by 2%pt (95% PI 1, 4) and reduce deaths by 2%pt (95% PI 1, 4) by 2050. For a large increase in whole grain consumption, the estimated decrease in prevalence of individuals with IHD was as high as 2%pt (95% PI 0, 3) and in deaths 2%pt (95% PI 0, 3).
Methods, "Projections" (how the IHD state and the differences are defined):
...and for IHD morbidity, those whose simulated time of IHD occurred before and death after the time point...
We compared the scenarios with the current diet scenario by calculating the differences in the state probabilities along their 95% PI.
Table 2, 2050 rows (state probability, %, with 95% PI; difference from current diets, %pt, with 95% PI):
Current diets | 2050 | Healthy 33 (32–34) | IHD 4 (3–5) | Dead w/o IHD 46 (45–47) | Dead w IHD 18 (17–19)
Large; Meat | 2050 | Healthy 35 (34–36), difference 2 (1–4) | IHD 4 (4 – 5), difference 0 (-1 to 1) | Dead w/o IHD 44 (43–45), difference -2 (-4 to 0) | Dead w IHD 17 (16–18), difference -1 (-2 to 1)
Large; Whole grain | 2050 | Healthy 34 (33–35), difference 2 (0–3) | IHD 4 (4 – 5), difference 0 (-1 to 1) | Dead w/o IHD 44 (43–46), difference -2 (-3 to 0) | Dead w IHD 17 (17–18), difference 0 (-1 to 1)
(Rows are shown here as one line per scenario; the cell values are as printed.)
Table 2 footer:
The differences of the large dietary change scenarios were calculated with respect to the current diets scenario in 2050.
Discussion, fifth paragraph:
Both reductions in red and processed meat, and increases in whole grain consumption would raise the prevalence of healthy individuals and reduce deaths by 2050, while maintaining the prevalence of IHD constant.
Conclusion:
We estimate that dietary changes, such as reducing red and processed meat consumption, and increasing whole grain intake, can improve cardiovascular health and reduce mortality. These findings align well with previous research and may have implications for policy actions.
What the paper's own numbers show
Method. I took the 2050 rows of Table 2 for current diets and for the large meat, whole grain and total scenarios. For each state I recomputed the difference from the printed levels (scenario minus current) and compared it with the printed difference. I then compared both with the figures in the text. A stated "decrease of 2 (PI 1, 4)" corresponds to a difference of −2 with PI −4 to −1 in the table's sign convention. Because the table rounds to whole percentage points, I also checked whether any unrounded values could make the text and table figures agree. I checked two other IHD definitions, the total-death figures, and the age and sex subgroups in Table 3. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Meat: change in IHD prevalence, 2050 | Abstract; Results | decrease of 2 (95% PI 1, 4), i.e. −2 (−4 to −1) | Table 2 IHD difference 0 (−1 to 1); levels 4 − 4 = 0 |
| Meat: could rounding reconcile them? | Abstract vs Table 2 | −2 vs 0 | unrounded values that print as −2 lie in [−2.5, −1.5) and those that print as 0 lie in [−0.5, 0.5); the ranges do not overlap. The text's point estimate −2 is outside the table's PI (−1 to 1) |
| Meat: does the PI exclude zero? | Abstract; Results vs Table 2 | yes (1 to 4) | no: Table 2 IHD PI is −1 to 1 |
| Meat: change in Healthy prevalence | Table 2 | +2 (1–4) | levels 35 − 33 = +2. Same magnitude and PI as the text's IHD figure; it is the only Table 2 row that matches |
| Whole grain: change in IHD prevalence | Abstract; Results | decrease of 2 (95% PI 0, 3), i.e. −2 (−3 to 0) | Table 2 IHD difference 0 (−1 to 1); levels 4 − 4 = 0; no rounding overlap |
| Whole grain: change in Healthy prevalence | Table 2 | +2 (0–3) | same magnitude and PI as the text's IHD figure (levels 34 − 33 = +1, within rounding of +2). The Dead w/o IHD row, −2 (−3 to 0), also matches when read as a reduction |
| "Ever IHD" (IHD + dead with IHD), meat | Table 2, derived | — | 22 → 21, change −1 (summed differences 0 + (−1) = −1), not −2 |
| "Ever IHD", whole grain | Table 2, derived | — | levels 22 → 21 (−1); summed differences 0 + 0 = 0 |
| Meat: change in total deaths | Abstract; Results | decrease of 2 (95% PI 1, 4) | levels (44 + 17) − (46 + 18) = −3; summed death differences −2 + (−1) = −3; minus the change in the alive states −(2 + 0) = −2. The unrounded total lies between −3 and −2, so −2 is consistent. A PI for total deaths is not reported and cannot be derived from the two per-state PIs |
| Whole grain: change in total deaths | Abstract; Results | decrease of 2 (95% PI 0, 3) | summed death differences −2 + 0 = −2; minus alive change −(2 + 0) = −2; consistent |
| Total scenario: Healthy, IHD, deaths | Results, "Projections" | "nearly 3", IHD "by 1", deaths "over 4" | Table 2: +3 (2–4), +1 (0–2), and −6 + 2 = −4. The text matches the table here |
| Meat, IHD difference in age/sex subgroups | Table 3 | — | 0, +1, 0, +1, +1, +1; none is −2 |
| Sum of the four rounded state probabilities | Table 2, 2050 | — | 101 (current), 100 (meat), 99 (whole grain), 100 (total): the cells are rounded independently |
In both scenarios the IHD figure given in the text equals the Healthy-state difference in Table 2, in point estimate and in both PI limits. The IHD row of the same table shows no change, and so do the printed IHD levels (4% under every diet). The Discussion's statement that IHD prevalence stays constant agrees with the table. The same Results paragraph reads the total scenario's IHD row correctly (+1%pt), so the paragraph is not using a different IHD definition throughout.
Figure
Figure 1. Left: verbatim excerpts of the IHD figures in the abstract (the Results repeat them), with the stated values in orange. Right: the 2050 rows of Table 2 for the large meat and whole grain scenarios, giving the printed difference from current diets with its 95% PI. Beside it, in blue, is the same difference recomputed from the printed state levels. The solid arrow joins each text figure to the IHD row it describes, which shows 0 (−1 to 1). The dashed arrow joins it to the Healthy row, whose difference is identical to the text figure: +2 (1 to 4) for meat and +2 (0 to 3) for whole grain. The whole-grain Healthy levels give 34 − 33 = +1, which is within rounding of the printed +2.
Figure 2. The differences from current diets in 2050, in percentage points (scenario − current), on one axis, with the null value 0 as a thin vertical line. Orange: the IHD change stated in the abstract and Results, converted to the table's sign convention (−2, −4 to −1 for meat; −2, −3 to 0 for whole grain). Black open circles: the IHD difference printed in Table 2, 0 (−1 to 1), with the difference recomputed from the printed levels (4 − 4 = 0) as a blue cross. Gray diamonds: the Healthy difference printed in Table 2, the mirror image of the stated IHD change. For meat, the stated interval excludes zero but the table's IHD interval does not. The stated point estimate lies outside the table's IHD interval in both scenarios.
Why it matters
- The abstract's main applied estimates do not match the paper's results. A reader of the abstract would conclude that cutting meat by two-thirds lowers IHD prevalence by 2%pt with a PI that excludes zero, and that more whole grain does the same. The paper's own projections show no change in IHD prevalence in either scenario.
- The abstract conclusion depends on it. "More plant-based diets reduce mortality and IHD prevalence" is supported for mortality by Table 2 (death states fall by about 2 to 3%pt), but not for IHD prevalence. The paper's Discussion says IHD prevalence stays constant. The Conclusion's "can improve cardiovascular health" and its reference to policy implications now rest on more people staying healthy and fewer deaths, not on lower IHD prevalence.
- The stated precision is overstated. For meat, the text's PI (1 to 4) excludes zero. The table's IHD PI (−1 to 1) does not.
- What is not affected. The pooling method, the simulation study, the hazard-ratio results, Table 2 itself, and the direction and size of the projected fall in deaths are all unaffected. The stated 2%pt fall in total deaths is consistent with the table within rounding. The stated total-death PIs cannot be checked from the published per-state intervals, so I do not count them as errors.
Explanations considered
- A different time point, population or subgroup. Not supported. The text says "by 2050" and, in the abstract, "in the 2017 cohort", which is the full-sample 2050 block of Table 2. The subgroup projections in Table 3 give meat-scenario IHD differences of 0 or +1, never −2. The supplement's eTable 8 covers subgroups only.
- A different IHD definition (for example, ever having had IHD). Not supported. Methods define the IHD state as people whose simulated IHD came before the time point and whose death came after it, which is the Table 2 "IHD" row. Counting everyone who ever had IHD (IHD plus dead with IHD) changes by −1 for meat (22 to 21), not −2 with a PI of 1 to 4. In the same paragraph, the total scenario's IHD change (+1%pt) is read from the IHD row.
- Sign convention or wording of a reduction as a positive number. Not able to explain it. The IHD difference is 0 (−1 to 1), so no change of sign gives 2 (1, 4) or 2 (0, 3).
- Rounding. Not able to explain it. A printed 0 corresponds to an unrounded value in [−0.5, 0.5) and a printed 2 to one in [1.5, 2.5). The two cannot be the same quantity, and the text's point estimate lies outside the table's own PI.
- The text figures are correct and the table's IHD row is wrong. Not supported by the paper. The printed IHD levels (4% under current diets and under both scenarios) agree with the 0 difference, and the Discussion independently says IHD prevalence stays constant. The text figures do equal, to the last digit, a different row of the same table (Healthy).
- A text-conversion artifact (for example, lost minus signs or shifted table cells). Ruled out. In the Europe PMC full-text XML the minus signs are true U+2212 characters and every value sits in its own correctly aligned table cell. The live PMC page shows the same values.
- A total-death PI that contradicts the table. An earlier screening pass suggested this, but it is not established. A PI for the sum of the two death states needs the joint bootstrap draws. It cannot be built from the two marginal PIs, and both marginal intervals including zero does not imply that the interval for their sum includes zero. Because IHD hardly changes, the total-death change is about minus the Healthy change, so a death interval similar to the Healthy interval is plausible.
- A correction already issued. None found (see Status).
Status
- Version of record. The DOI resolves to the publisher's Ovid/LWW page, which returned HTTP 402 (paywall) to automated requests, so I could not read the typeset page there. I checked the publisher-deposited full text in two other places: the live PMC article page and the Europe PMC full-text XML. Both carry the abstract and Results sentences quoted above, the same Table 2 cells, and the Discussion sentence. Crossref lists the article as Epidemiology volume 37, issue 2, pages 187–197. Every quotation here was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The supplementary digital content (https://links.lww.com/EDE/C292; a 46-page PDF with eAppendix 1–4 and eTables 1–8) was opened and its text extracted. eAppendix 3 describes the projection and bootstrap steps. eTable 8 gives 2050 projections by sex and age subgroup only. The supplement has no full-sample total-death interval and no other IHD definition that would give a 2%pt (1, 4) reduction. It confirms that the reported contrasts are differences in state probabilities, as in Table 2.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by field and an empty relation field, and a Crossref search for works related to this DOI returned none. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC12851546; Europe PMC full-text XML for PMC12851546; supplementary digital content at links.lww.com/EDE/C292; Crossref REST API; Europe PMC; PubPeer; https://doi.org/10.1097/EDE.0000000000001932.
Suggested correction
In the abstract, Results, replace
would decrease the prevalence of IHD by 2 (95% prediction intervals [PI] 1, 4) percentage points (%pt) in the 2017 cohort and deaths by 2%pt (95% PI 1, 4) by 2050. We estimated that a 100% increase in whole grain consumption would reduce IHD by 2%pt (95% PI 0, 3) and deaths by 2%pt (95% PI 0, 3).
with wording that follows Table 2, for example:
would increase the prevalence of healthy individuals by 2 (95% prediction intervals [PI] 1, 4) percentage points (%pt) in the 2017 cohort and reduce deaths by 2%pt by 2050, with no change in the prevalence of IHD (0%pt, 95% PI −1, 1). We estimated that a 100% increase in whole grain consumption would increase the prevalence of healthy individuals by 2%pt (95% PI 0, 3) and reduce deaths by 2%pt, with no change in IHD prevalence (0%pt, 95% PI −1, 1).
The authors would need to give the total-death PIs from their bootstrap draws. The same changes apply to the two sentences in the Results "Projections" paragraph. The abstract conclusion could read "Our estimates support the hypothesis that more plant-based diets reduce mortality", consistent with the Discussion's statement that IHD prevalence stays constant. If the authors intended a different IHD quantity for these two scenarios, the correction should define it and report its values, since none of the paper's tables contains a −2%pt IHD change.
Recomputation script (python recompute.py.txt)
"""Recompute the IHD and death projections stated in the text of PMC12851546
(Epidemiology 2026;37(2):187-197, doi:10.1097/EDE.0000000000001932,
"Novel Pooling Method for Nonlinear Cohort Analysis and Meta-analysis Estimates:
Predicting Health Outcomes from Dietary Changes") from the paper's own Table 2.
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 2, 2050 block: "Current diets", "Large; Meat", "Large; Whole grain", "Large; Total"
(state probability estimate in %, and difference vs current diets with its 95% PI).
Table 3, "Large; Meat" IHD differences for the six age/sex subgroups.
Abstract and Results ("Projections"): the stated IHD and death reductions.
All values are percentages or percentage points (%pt). Differences are scenario minus
current diets, as in Table 2. A stated "decrease of 2 (95% PI 1, 4)" is therefore a
difference of -2 with PI (-4, -1).
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
STATES = ["Healthy", "IHD", "Dead w/o IHD", "Dead w IHD"]
# ---------------------------------------------------------------- inputs: Table 2, 2050
CURRENT = {"Healthy": 33, "IHD": 4, "Dead w/o IHD": 46, "Dead w IHD": 18}
SCEN = {
"Meat": {
"level": {"Healthy": 35, "IHD": 4, "Dead w/o IHD": 44, "Dead w IHD": 17},
"diff": {"Healthy": (2, 1, 4), "IHD": (0, -1, 1),
"Dead w/o IHD": (-2, -4, 0), "Dead w IHD": (-1, -2, 1)},
},
"Whole grain": {
"level": {"Healthy": 34, "IHD": 4, "Dead w/o IHD": 44, "Dead w IHD": 17},
"diff": {"Healthy": (2, 0, 3), "IHD": (0, -1, 1),
"Dead w/o IHD": (-2, -3, 0), "Dead w IHD": (0, -1, 1)},
},
"Total": {
"level": {"Healthy": 36, "IHD": 5, "Dead w/o IHD": 40, "Dead w IHD": 19},
"diff": {"Healthy": (3, 2, 4), "IHD": (1, 0, 2),
"Dead w/o IHD": (-6, -8, -4), "Dead w IHD": (2, 0, 3)},
},
}
# Abstract and Results text: stated reductions (magnitude, PI low, PI high), as printed
TEXT = {
"Meat": {"IHD": (2, 1, 4), "deaths": (2, 1, 4)},
"Whole grain": {"IHD": (2, 0, 3), "deaths": (2, 0, 3)},
}
# Results text for the total scenario: "nearly 3" healthy up, "over 4" deaths down, IHD up 1
TEXT_TOTAL = {"Healthy": "+3 ('nearly 3')", "deaths": "-4 ('over 4')", "IHD": "+1"}
# Table 3, Large; Meat, IHD difference by subgroup (2050)
TABLE3_MEAT_IHD = {"All, 50": (0, -4, 4), "All, 60": (1, -2, 4), "Female, 50": (0, -5, 5),
"Female, 60": (1, -4, 4), "Male, 50": (1, -5, 6), "Male, 60": (1, -2, 4)}
def as_change(red):
"""A stated reduction r (lo, hi) as a scenario-minus-current difference."""
r, lo, hi = red
return (-r, -hi, -lo)
def rounding_bounds(x):
"""Half-open interval of unrounded values that print as integer x."""
return (x - 0.5, x + 0.5)
def fmt(t):
e, lo, hi = t
return f"{e:+d} ({lo:+d} to {hi:+d})"
print("=== 0. Table 2, 2050: sum of the four rounded state probabilities ===")
print(f" Current diets: {sum(CURRENT.values())}")
for s, d in SCEN.items():
print(f" {s}: {sum(d['level'].values())}")
print(" (sums of 99-101 show the cells are independently rounded to whole percent)")
for s in ["Meat", "Whole grain"]:
L, D, T = SCEN[s]["level"], SCEN[s]["diff"], TEXT[s]
print(f"\n=== {s} scenario (large change), 2050 ===")
print(f" {'state':13s} {'current':>7s} {'scenario':>8s} {'level diff':>10s} Table 2 difference (PI)")
for k in STATES:
print(f" {k:13s} {CURRENT[k]:7d} {L[k]:8d} {L[k]-CURRENT[k]:+10d} {fmt(D[k])}")
t_ihd = as_change(T["IHD"])
print(f" Text: IHD decrease {T['IHD'][0]} (PI {T['IHD'][1]}, {T['IHD'][2]})"
f" -> as a difference {fmt(t_ihd)}")
print(f" Table 2 IHD difference: {fmt(D['IHD'])}; recomputed from levels: "
f"{L['IHD']} - {CURRENT['IHD']} = {L['IHD']-CURRENT['IHD']:+d}")
lo_t, hi_t = rounding_bounds(t_ihd[0])
lo_d, hi_d = rounding_bounds(D["IHD"][0])
print(f" Unrounded values consistent with text point {t_ihd[0]:+d}: [{lo_t}, {hi_t});"
f" with table point {D['IHD'][0]:+d}: [{lo_d}, {hi_d}) -> overlap: {max(lo_t, lo_d) < min(hi_t, hi_d)}")
print(f" Text point {t_ihd[0]:+d} inside Table 2 IHD PI [{D['IHD'][1]}, {D['IHD'][2]}]? "
f"{D['IHD'][1] <= t_ihd[0] <= D['IHD'][2]}")
print(f" Text IHD PI excludes 0? {not (t_ihd[1] <= 0 <= t_ihd[2])}; "
f"Table 2 IHD PI excludes 0? {not (D['IHD'][1] <= 0 <= D['IHD'][2])}")
print(f" Table 2 Healthy difference {fmt(D['Healthy'])}; Healthy levels "
f"{L['Healthy']} - {CURRENT['Healthy']} = {L['Healthy']-CURRENT['Healthy']:+d}")
print(f" Text IHD figure (magnitude, PI) == Table 2 Healthy row (est, PI)? "
f"{T['IHD'] == D['Healthy']}")
match = [k for k in STATES if D[k] == T["IHD"] or D[k] == t_ihd]
print(f" Table 2 rows equal to the text IHD figure (either sign): {match}")
ever_c = CURRENT["IHD"] + CURRENT["Dead w IHD"]
ever_s = L["IHD"] + L["Dead w IHD"]
print(f" 'Ever IHD' (IHD + dead w IHD): {ever_c} -> {ever_s}, change {ever_s-ever_c:+d}; "
f"summed Table 2 differences {D['IHD'][0] + D['Dead w IHD'][0]:+d}")
# deaths
dc = CURRENT["Dead w/o IHD"] + CURRENT["Dead w IHD"]
ds = L["Dead w/o IHD"] + L["Dead w IHD"]
sum_d = D["Dead w/o IHD"][0] + D["Dead w IHD"][0]
comp = -(D["Healthy"][0] + D["IHD"][0])
print(f" Deaths from levels: {dc} -> {ds}, change {ds-dc:+d}")
print(f" Deaths as sum of the two death-state differences: {sum_d:+d}")
print(f" Deaths as minus the change in the alive states (Healthy + IHD): {comp:+d}")
# bounds on the unrounded total-death difference implied by each route
b1 = (rounding_bounds(D["Dead w/o IHD"][0])[0] + rounding_bounds(D["Dead w IHD"][0])[0],
rounding_bounds(D["Dead w/o IHD"][0])[1] + rounding_bounds(D["Dead w IHD"][0])[1])
b2 = (-(rounding_bounds(D["Healthy"][0])[1] + rounding_bounds(D["IHD"][0])[1]),
-(rounding_bounds(D["Healthy"][0])[0] + rounding_bounds(D["IHD"][0])[0]))
both = (max(b1[0], b2[0]), min(b1[1], b2[1]))
print(f" Unrounded total-death difference: from death rows ({b1[0]}, {b1[1]}),"
f" from alive rows ({b2[0]}, {b2[1]}), jointly ({both[0]}, {both[1]})")
t_d = as_change(T["deaths"])
print(f" Text death figure as a difference: {fmt(t_d)}; point {t_d[0]} within joint bounds? "
f"{both[0] <= t_d[0] <= both[1]}")
print(f" Text death figure (magnitude, PI) == Table 2 Healthy row? {T['deaths'] == D['Healthy']}"
" (expected if IHD barely changes, since deaths = -(Healthy + IHD))")
print(" Total-death PI: not derivable from the two marginal death-state PIs (needs joint draws)")
print("\n=== Total scenario (all changes), 2050: Results text vs Table 2 ===")
L, D = SCEN["Total"]["level"], SCEN["Total"]["diff"]
print(f" Healthy: text {TEXT_TOTAL['Healthy']}; Table 2 {fmt(D['Healthy'])}; levels "
f"{L['Healthy']}-{CURRENT['Healthy']} = {L['Healthy']-CURRENT['Healthy']:+d}")
print(f" IHD: text {TEXT_TOTAL['IHD']}; Table 2 {fmt(D['IHD'])}; levels "
f"{L['IHD']}-{CURRENT['IHD']} = {L['IHD']-CURRENT['IHD']:+d}")
print(f" Deaths: text {TEXT_TOTAL['deaths']}; summed Table 2 death rows "
f"{D['Dead w/o IHD'][0] + D['Dead w IHD'][0]:+d}; minus alive rows "
f"{-(D['Healthy'][0] + D['IHD'][0]):+d}")
print(" -> for the total scenario the text reads the IHD row, and agrees with Table 2")
print("\n=== Table 3, Large; Meat, IHD difference by subgroup ===")
for g, t in TABLE3_MEAT_IHD.items():
print(f" {g:11s} {fmt(t)}")
print(f" any subgroup estimate equal to -2? {any(t[0] == -2 for t in TABLE3_MEAT_IHD.values())}")
# ================================================================ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# ---------------------------------------------------------------- fig 1: text vs Table 2
W, H = 7.2, 7.25
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.axis("off")
LX, LW = 0.12, 2.6 # left panel (text excerpts)
RX, RW = 3.5, 3.6 # right panel (Table 2)
ROW_H = 0.33
C_STATE, C_TAB, C_LEV = RX + 0.12, RX + 1.88, RX + 2.98
def box(x, y, w, h, ec=INK2, lw=1.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.06",
fc="white", ec=ec, lw=lw))
ax.text(LX, H - 0.25, "Abstract and Results text", fontsize=11.5, weight="bold", va="center")
ax.text(LX, H - 0.52, "verbatim excerpts", fontsize=10, color=INK2, va="center")
ax.text(RX, H - 0.25, "Table 2, full sample, 2050", fontsize=11.5, weight="bold", va="center")
ax.text(RX, H - 0.52, "difference vs current diets, %pt (95% PI)", fontsize=10, color=INK2,
va="center")
EXCERPT = {
"Meat": ("Meat: two-thirds reduction",
["“...would decrease the", "prevalence of IHD by"],
["2 (95% prediction intervals", "[PI] 1, 4)"],
["percentage points (%pt)...”"]),
"Whole grain": ("Whole grain: 100% increase",
["“...would reduce IHD by"],
["2%pt (95% PI 0, 3)"],
["...”"]),
}
NOTE = {"Meat": ["Conflict: the IHD row is 0 (−1 to 1);", "the text figure is the Healthy row"],
"Whole grain": ["Conflict: the IHD row is 0 (−1 to 1);",
"the text figure is the Healthy row"]}
TOPS = {"Meat": H - 1.05, "Whole grain": H - 4.2}
LINE = 0.27
for s, top in TOPS.items():
D, L = SCEN[s]["diff"], SCEN[s]["level"]
# ---- right: table block
bh = ROW_H * 5 + 0.1
box(RX, top - bh, RW, bh)
yh = top - 0.5 * ROW_H - 0.02
ax.text(RX, top + 0.2, f"Large; {s}", fontsize=10.5, weight="bold", va="center")
ax.text(C_STATE, yh, "State", fontsize=10, color=INK2, va="center")
ax.text(C_TAB, yh, "Table 2", fontsize=10, color=INK2, va="center", ha="center")
ax.text(C_LEV, yh, "levels", fontsize=10, color=BLUE, va="center", ha="center")
ry = {}
for i, k in enumerate(STATES):
y = top - (i + 1.5) * ROW_H - 0.05
ry[k] = y
hl = k in ("Healthy", "IHD")
if hl:
ax.add_patch(plt.Rectangle((RX + 0.05, y - ROW_H / 2 + 0.02), RW - 0.1, ROW_H - 0.04,
fc="#f1f0ec", ec="none"))
e, lo, hi = D[k]
w = "bold" if hl else "normal"
ax.text(C_STATE, y, k, fontsize=10, va="center", weight=w)
ax.text(C_TAB, y, f"{e:+d} ({lo} to {hi})".replace("+0 ", "0 ").replace("-", "−"),
fontsize=10, va="center", ha="center", weight=w)
ax.text(C_LEV, y, f"{L[k]}−{CURRENT[k]} = {L[k]-CURRENT[k]:+d}".replace("+0", "0")
.replace("= -", "= −"), fontsize=10, va="center", ha="center", color=BLUE)
# ---- left: excerpt box
head, pre, num, post = EXCERPT[s]
lb = 0.5 + LINE * (len(pre) + len(num) + len(post)) + 0.42
box(LX, top - lb, LW, lb, ec=ORANGE, lw=1.6)
y = top - 0.25
ax.text(LX + 0.13, y, head, fontsize=10.5, weight="bold", va="center")
y -= 0.36
for t in pre:
ax.text(LX + 0.13, y, t, fontsize=10, va="center")
y -= LINE
y_num = y - LINE * (len(num) - 1) / 2
for t in num:
ax.text(LX + 0.13, y, t, fontsize=10.5, va="center", color=ORANGE, weight="bold")
y -= LINE
for t in post:
ax.text(LX + 0.13, y, t, fontsize=10, va="center")
y -= LINE
ax.text(LX + 0.13, top - lb + 0.2, "Abstract; same claim in Results", fontsize=10,
color=INK2, style="italic", va="center")
# ---- note under the excerpt box
for j, t in enumerate(NOTE[s]):
ax.text(LX, top - lb - 0.25 - j * 0.26, t, fontsize=10, weight="bold", va="center")
# ---- connectors
x0, x1 = LX + LW, RX
ax.add_patch(FancyArrowPatch((x0, y_num - 0.02), (x1 + 0.05, ry["IHD"]), arrowstyle="-|>",
mutation_scale=12, lw=2, color=INK,
connectionstyle="arc3,rad=0.0"))
ax.add_patch(FancyArrowPatch((x0, y_num + 0.04), (x1 + 0.05, ry["Healthy"]), arrowstyle="-|>",
mutation_scale=12, lw=1.6, color=INK2, ls=(0, (3, 2)),
connectionstyle="arc3,rad=0.0"))
xm = (x0 + x1) / 2
ax.text(xm, max(ry["Healthy"], y_num) + 0.3, "identical", fontsize=10, color=INK2, ha="center",
va="center")
ax.text(xm, min(ry["IHD"], y_num) - 0.3, "conflict", fontsize=10, color=INK, weight="bold",
ha="center", va="center")
handles = [Line2D([], [], color=ORANGE, lw=0, marker="s", ms=8,
label="figure the text gives for IHD"),
Line2D([], [], color=BLUE, lw=0, marker="s", ms=8,
label="recomputed: scenario − current level, Table 2")]
fig.legend(handles=handles, loc="lower left", bbox_to_anchor=(0.01, 0.0), ncol=2,
frameon=False, fontsize=10, handlelength=1.2, columnspacing=1.5)
save(fig, "fig1")
# ---------------------------------------------------------------- fig 2: one axis, intervals
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
ax.set_facecolor(BG)
labels, ypos = [], []
groups = {"Meat": 8.0, "Whole grain": 3.8}
GROUP_TITLE = {"Meat": "Meat, two-thirds reduction", "Whole grain": "Whole grain, 100% increase"}
for s, g in groups.items():
D, L = SCEN[s]["diff"], SCEN[s]["level"]
ax.text(-5.0, g + 0.05, GROUP_TITLE[s], fontsize=10.5, weight="bold", va="center",
bbox=dict(fc=BG, ec="none", pad=1.5))
e, lo, hi = as_change(TEXT[s]["IHD"])
y = g - 1
ax.plot([lo, hi], [y, y], color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(e, y, "o", color=ORANGE, ms=9)
ax.text(max(hi, 0) + 0.3, y, f"{e} ({lo} to {hi})".replace("-", "−"), va="center",
color=ORANGE, fontsize=10)
labels.append("IHD, as stated in text"); ypos.append(y)
e, lo, hi = D["IHD"]
y = g - 2
ax.plot([lo, hi], [y, y], color=INK, lw=2, solid_capstyle="butt")
ax.plot(e, y, "o", mfc="white", mec=INK, mew=2, ms=10)
ax.plot(L["IHD"] - CURRENT["IHD"], y, "x", color=BLUE, ms=7, mew=2)
ax.text(hi + 0.3, y, f"0 (−1 to 1); levels {L['IHD']}−{CURRENT['IHD']} = 0",
va="center", color=INK, fontsize=10)
labels.append("IHD row, Table 2"); ypos.append(y)
e, lo, hi = D["Healthy"]
y = g - 3
ax.plot([lo, hi], [y, y], color=INK2, lw=2, solid_capstyle="butt")
ax.plot(e, y, "D", mfc="white", mec=INK2, mew=1.5, ms=8)
ax.text(hi + 0.3, y, f"+{e} ({lo} to {hi})", va="center", color=INK2, fontsize=10)
labels.append("Healthy row, Table 2"); ypos.append(y)
ax.axvline(0, color=INK2, lw=0.8)
ax.set_yticks(ypos)
ax.set_yticklabels(labels, fontsize=10)
ax.set_xlim(-5, 8.6)
ax.set_xticks(range(-5, 6))
ax.set_ylim(0.4, 8.5)
ax.set_xlabel("Difference from current diets in 2050, scenario − current (%pt)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.spines["bottom"].set_bounds(-5, 5)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="stated in Abstract/Results"),
Line2D([], [], color=INK, lw=2, marker="o", mfc="white", mec=INK, ms=8,
label="printed in Table 2"),
Line2D([], [], color=BLUE, lw=0, marker="x", mew=2, ms=8,
label="recomputed from levels")]
ax.legend(handles=handles, loc="upper left", bbox_to_anchor=(-0.27, -0.15), ncol=3,
frameon=False, fontsize=10, handlelength=1.6, columnspacing=1.0)
fig.subplots_adjust(left=0.235, right=0.99, top=0.98, bottom=0.24)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract reports 1-year MACE-or-death rates as MACE rates for each perioperative myocardial injury aetiology
Eur Heart J (2023) · doi:10.1093/eurheartj/ehac798 · PMC10263270
Summary
The abstract says that 51%, 41%, 57%, 64% and 25% of patients with extra-cardiac PMI, type 1 MI (T1MI), tachyarrhythmia, acute heart failure (AHF) and likely type 2 MI (lT2MI) "had MACE" within 1 year, compared with 7% of patients without PMI. In the paper's own Table 2 these five numbers are the rates of the secondary composite endpoint, MACE or all-cause death (51%, 42%, 57%, 64%, 25%). The MACE rates in the same table, the Results text and the Discussion are 30%, 37%, 49%, 56% and 17%. The 7% comparator, however, is the no-PMI MACE rate; the no-PMI composite rate is 12%. So the abstract's headline sentence overstates the 1-year MACE risk for every aetiology (by 21 percentage points for extra-cardiac PMI and by 8.5 points for lT2MI, the most common aetiology), widens the contrast with patients without PMI, and reverses the order of extra-cardiac PMI and T1MI. The paper's qualitative conclusions and its adjusted hazard ratios are not affected.
What the paper reports
Abstract, Methods and results:
Major adverse cardiac events (MACEs) including acute myocardial infarction, AHF (both only from day 3 to avoid inclusion bias), life-threatening arrhythmia, and cardiovascular death as well as all-cause death were assessed during 1-year follow-up.
Outcomes differed starkly according to aetiology: in patients with extra-cardiac PMI, T1MI, tachyarrhythmia, AHF, and lT2MI 51%, 41%, 57%, 64%, and 25% had MACE, and 38%, 27%, 40%, 49%, and 17% patients died within 1 year, respectively, compared to 7% and 9% in patients without PMI.
Methods, Endpoints:
The primary endpoints were the occurrence of MACE and all-cause death following different PMI aetiologies within 1 year. [...] MACE was defined as a composite endpoint of acute myocardial infarction, AHF, life-threatening arrhythmia, and cardiovascular death. [...] A composite of MACE and all-cause death was a secondary endpoint.
Results, Follow-up:
At least one MACE occurred in 684/7754 patients at 1 year (8.8%). A total of 818/7754 patients died within 1 year (10.5%), with death occurring during the index hospitalization in 154/817 patients. The composite of MACE or all-cause death occurred in 1160/7754 patients (15.0%).
Results, MACE after PMI according to aetiology:
All PMI aetiologies showed substantially increased 1-year MACE rates vs. patients without PMI (7%, 95% CI 6%–8%), and there was a stark difference within PMI aetiologies (Table 2, Figure 3A): extra-cardiac PMI, T1MI, tachyarrhythmia, and AHF were associated with very high rates of MACE [30% (95% CI 20–47), 37% (95% CI 24–47), 49% (95% CI 34–75), and 56% (95% CI 38–70), respectively], and lT2MI was associated with high rates of MACE (17%, 95% CI 22%–28%).
Table 2, "Outcome following perioperative myocardial infarction/injury" (columns: No PMI, Type 1 MI, Tachy-arrhythmia, Acute heart failure, Likely type 2 MI, Extra-cardiac PMI sepsis, Extra-cardiac PMI other; n = 6738, 71, 47, 39, 750, 60, 49):
MACE at 1 year | 456 (7%)[6%–8%] | 26 (37%)[24%–47%] | 23 (49%)[34%–75%] | 22 (56%)[38%–70%] | 124 (17%)[14%–20%] | 18 (30%)[20%–47%] | 15 (31%)[17%–45%]
All-cause death within 1 year | 591 (9%)[8%–9%] | 20 (28%)[17%–38%] | 19 (40%)[25%–53%] | 19 (49%)[30%–62%] | 127 (17%)[14%–20%] | 21 (35%)[22%–46%] | 21 (43%)[27%–55%]
MACE or all-cause death at 1 year | 834 (12%)[12%–13%] | 30 (42%)[29%–53%] | 27 (57%)[41%–69%] | 25 (64%)[45%–76%] | 188 (25%)[22%–28%] | 32 (53%)[39%–64%] | 24 (49%)[23%–63%]
Discussion, second paragraph:
Second, the MACE and death rate differed substantially among PMI aetiologies: 5 of 10 patients with PMI due to AHF or tachyarrhythmia developed at least one MACE within 1 year, vs. 4 of 10 with T1MI, and 1.5 in 10 in lT2MI.
What the paper's own numbers show
Method. I took the event counts and group sizes printed in Table 2, summed the two extra-cardiac columns (sepsis, n = 60; other, n = 49) because the abstract reports extra-cardiac PMI as one group, and computed crude proportions (events / n) for MACE, all-cause death and MACE or all-cause death. Table 2's printed percentages equal these crude proportions after rounding in all 21 cells. I then checked that the Table 2 rows carry the right labels: their column sums against the totals in the Results text, and each composite count against the bounds it must satisfy (at least the larger of MACE and death, at most MACE plus non-cardiovascular death). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Extra-cardiac PMI, "had MACE" | Abstract | 51% | MACE (18+15)/109 = 30.3%; MACE or death (32+24)/109 = 51.4% |
| T1MI, "had MACE" | Abstract | 41% | MACE 26/71 = 36.6%; MACE or death 30/71 = 42.3% |
| Tachyarrhythmia, "had MACE" | Abstract | 57% | MACE 23/47 = 48.9%; MACE or death 27/47 = 57.4% |
| AHF, "had MACE" | Abstract | 64% | MACE 22/39 = 56.4%; MACE or death 25/39 = 64.1% |
| lT2MI, "had MACE" | Abstract | 25% | MACE 124/750 = 16.5%; MACE or death 188/750 = 25.1% |
| No PMI comparator | Abstract | 7% | MACE 456/6738 = 6.8%; MACE or death 834/6738 = 12.4% |
| Extra-cardiac PMI from printed column percentages | Table 2 | 30%, 31% (MACE); 53%, 49% (composite) | MACE (60×30 + 49×31)/109 = 30.4%; composite (60×53 + 49×49)/109 = 51.2% |
| MACE rates by aetiology | Results text | 30, 37, 49, 56, 17% | match the MACE row above, not the abstract |
| Deaths at 1 year (same order) | Abstract | 38, 27, 40, 49, 17; 9% | 38.5, 28.2, 40.4, 48.7, 16.9; 8.8% (consistent; T1MI 1 point low) |
| Column sums of the MACE, death and composite rows | Table 2 vs Results text | 684, 818, 1160 | 684, 818, 1160 (exact) |
| Bound max(MACE, death) ≤ composite ≤ MACE + non-CV death | Table 2, all 7 columns | — | holds in every column (e.g. lT2MI 127 ≤ 188 ≤ 199) |
| lT2MI MACE 95% CI | Results text vs Table 2 | "17%, 95% CI 22%–28%" vs [14%–20%] | 22%–28% is the composite's interval in Table 2; a Wilson interval on 124/750 is 14.0%–19.4% |
Four of the five PMI values in the abstract equal the rounded composite (MACE or all-cause death) rate; the fifth, T1MI, is 41% against a composite of 42.3% and a MACE rate of 36.6%, so it too sits with the composite. The abstract's death figures all match the all-cause death row. The no-PMI comparator in the same sentence, 7%, matches the MACE row and not the composite (12.4%). Because the Table 2 rows sum exactly to the separately stated totals and respect the logical bounds between them, the table's labels are internally consistent, and the mismatch lies in the abstract's label.
Figure
Figure 1. The abstract sentence (top; values printed in the abstract in orange) set against the two relevant Table 2 rows, with each rate recomputed in blue from the table's own counts (bottom). The left connector shows that the five PMI values printed under the label "MACE" (51%, 41%, 57%, 64%, 25%) correspond to the secondary composite row, MACE or all-cause death (51%, 42%, 57%, 64%, 25%), not to the MACE row (30%, 37%, 49%, 56%, 17%). The right connector shows that the comparator in the same sentence, 7%, corresponds to the no-PMI MACE row; the no-PMI composite rate is 12%. Extra-cardiac PMI is the sum of the table's sepsis and other columns (n = 60 + 49).
Figure 2. For each group, the recomputed 1-year MACE rate (filled blue circle), the recomputed MACE-or-death rate (open blue circle) and the value the abstract prints as "had MACE" (orange diamond), on a linear axis starting at zero. For all five PMI aetiologies the abstract value sits on or next to the composite, and the gap to the true MACE rate is 4 to 21 percentage points. For the no-PMI group the abstract value sits on the MACE rate. All values are crude proportions from Table 2 counts, which reproduce the table's printed percentages.
Why it matters
- The abstract's absolute risk estimates for a primary endpoint are overstated. MACE is a primary endpoint. The abstract's "had MACE" figures exceed the true MACE rates by 20.7 points for extra-cardiac PMI (68% relative), 4.4 for T1MI (12%), 8.1 for tachyarrhythmia (16%), 7.6 for AHF (13%) and 8.5 for lT2MI (51%). lT2MI is 750/1016 (73.8%) of all PMI, so the most frequently applicable figure is half again too high: about 1 in 4 rather than the Discussion's "1.5 in 10".
- The contrast with patients without PMI is inflated. Composite rates in the PMI groups are set against the MACE-only rate in the no-PMI group. Read at face value, the abstract implies that lT2MI carries 25/7 = 3.6 times the no-PMI MACE risk; the MACE-to-MACE ratio is 2.4 and the composite-to-composite ratio is 2.0. For extra-cardiac PMI the implied 7.3 compares with 4.5 for MACE and 4.2 for the composite.
- The ranking of aetiologies changes. In the abstract, extra-cardiac PMI (51%) appears to carry a higher MACE risk than T1MI (41%). On the MACE endpoint the order is reversed (30.3% vs 36.6%), consistent with the Results text and with the adjusted hazard ratios (3.0 vs 3.2).
- What is not affected. Every PMI aetiology still has higher 1-year MACE and death rates than patients without PMI, the multivariable adjusted hazard ratios (Table 3) are reported separately and are unchanged, and the Results text, Table 2 and Discussion give the correct MACE rates. The error is in how the abstract labels and juxtaposes the figures, which are the absolute risks a reader of the abstract alone is most likely to quote.
- A related slip in the Results text. The lT2MI MACE rate is given as "17%, 95% CI 22%–28%". An interval that excludes its own point estimate cannot be right; 22%–28% is the composite's interval in Table 2, and the MACE interval there is 14%–20%.
Explanations considered
- The Table 2 rows, not the abstract, are mislabelled. Ruled out. The MACE, all-cause death and composite rows sum to 684, 818 and 1160, exactly the totals the Results text gives for MACE, death and MACE or death. In every column the composite count lies between the larger of MACE and death and the sum of MACE and non-cardiovascular death, which is what a correctly labelled composite must do.
- "MACE" in the abstract means MACE including all-cause death. Ruled out. The abstract's own Methods sentence defines MACE as acute MI, AHF, life-threatening arrhythmia and cardiovascular death, "as well as all-cause death", listing death as a separate outcome; the Methods define MACE without non-cardiovascular death and make the composite with all-cause death a separate, secondary endpoint. The same abstract sentence reports deaths separately, and its no-PMI comparator (7%) is the MACE-only rate.
- Kaplan–Meier estimates rather than crude proportions. Does not account for it. The Methods say the univariable analysis used Kaplan–Meier estimates, and Table 2's footer says its intervals come from survival analysis, but the printed Table 2 percentages equal the crude proportions, and the Results text gives MACE rates of 30, 37, 49, 56 and 17%. No reading of the MACE endpoint in the paper yields 51, 41, 57, 64 and 25%. Censoring could explain the 1-point T1MI difference (41% vs 42.3%), not the 4- to 21-point gaps to MACE.
- A different time point. Ruled out. The abstract states 1 year, as does Table 2. The only other horizon in the paper, 120 days, is used for the lT2MI prognostic model, and rates over a shorter window would be lower, not higher.
- Adjusted rather than unadjusted figures. Ruled out. The abstract gives percentages of patients, not hazard ratios, and the next sentence ("These associations persisted in multivariable analysis") refers to the adjusted analysis separately.
- Extra-cardiac PMI defined differently in the abstract. Does not account for it. Summing the two extra-cardiac columns gives 30.3% for MACE and 51.4% for the composite; weighting the printed column percentages gives 30.4% and 51.2%. Only the composite matches the abstract's 51%.
- Text-conversion artefact in the PMC copy. Ruled out. The version of record on the publisher's site, the Europe PMC full-text XML and the PubMed record carry the same abstract sentence and the same Table 2 cells.
No explanation was found under which the five abstract values are 1-year MACE rates. The figures are consistent with the composite row having been used in place of the MACE row for the PMI groups only.
Status
As of 2026-10-24:
- Version of record. The publisher's article page (reached through doi.org) has the same abstract sentence ("51%, 41%, 57%, 64%, and 25% had MACE ... compared to 7% and 9% in patients without PMI") and the same Table 2 cells as PMC; the Results text there also gives "17%, 95% CI 22%–28%" for lT2MI. The Europe PMC full-text XML and the PubMed record (PMID 36705050) agree. The publisher's structured graphical abstract gives per-aetiology MACE rates that match the Table 2 MACE row (37, 49, 56, 17, 30 and 7%), not the abstract text.
- Supplement. The supplementary data file (ehac798_Supplementary_Data.docx) was checked. Its Table S1 gives MACE within 1 year as 456 (7%) for no PMI and 228 (23%) for PMI overall; the count equals the sum of the Table 2 MACE counts over the PMI groups (26+23+22+124+18+15 = 228). It does not give per-aetiology 1-year MACE rates that would support the abstract values.
- Corrections. No correction, erratum, expression of concern or retraction is listed in Crossref, PubMed Central or Europe PMC.
- PubPeer. A web search found no PubPeer entry for the article; PubPeer's own search could not be queried directly from this machine (it requires interactive verification), so the absence of comments there is not confirmed.
- Sources. PMC10263270 as archived in PubMed Central; https://doi.org/10.1093/eurheartj/ehac798; Europe PMC full text and supplementary files for PMC10263270; PubMed 36705050; Crossref metadata for the DOI.
Suggested correction
In the abstract, replace
in patients with extra-cardiac PMI, T1MI, tachyarrhythmia, AHF, and lT2MI 51%, 41%, 57%, 64%, and 25% had MACE, and 38%, 27%, 40%, 49%, and 17% patients died within 1 year, respectively, compared to 7% and 9% in patients without PMI.
with
in patients with extra-cardiac PMI, T1MI, tachyarrhythmia, AHF, and lT2MI 30%, 37%, 49%, 56%, and 17% had MACE, and 38%, 28%, 40%, 49%, and 17% patients died within 1 year, respectively, compared to 7% and 9% in patients without PMI.
(the MACE and death figures from Table 2 and the Results text). If the composite is meant to be reported, the sentence should instead name it ("51%, 42%, 57%, 64%, and 25% had MACE or died") and give the matching no-PMI composite rate, 12%. Separately, in the Results, the lT2MI MACE interval "95% CI 22%–28%" should read "95% CI 14%–20%", as in Table 2.
Recomputation script (python recompute.py.txt)
"""Recompute the per-aetiology 1-year outcome rates in PMC10263270
(Eur Heart J 2023, doi:10.1093/eurheartj/ehac798).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are the event counts and group sizes printed in Table 2 of the paper
("Outcome following perioperative myocardial infarction/injury"), the percentages
printed in Table 2, the Results text and the abstract. Rates here are crude
proportions (events / n); the paper's Table 2 percentages agree with them to the
printed precision. Extra-cardiac PMI is split in Table 2 into "sepsis" (n = 60) and
"other" (n = 49); the abstract reports it as one group, so the two columns are summed.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ Table 2 (counts)
COLS = ["No PMI", "T1MI", "Tachyarrhythmia", "AHF", "lT2MI", "Extra-cardiac, sepsis",
"Extra-cardiac, other"]
N = np.array([6738, 71, 47, 39, 750, 60, 49])
MACE = np.array([456, 26, 23, 22, 124, 18, 15]) # "MACE at 1 year"
DEATH = np.array([591, 20, 19, 19, 127, 21, 21]) # "All-cause death within 1 year"
NONCV = np.array([422, 4, 7, 8, 75, 17, 10]) # "Non-cardiovascular death"
COMP = np.array([834, 30, 27, 25, 188, 32, 24]) # "MACE or all-cause death at 1 year"
# percentages printed in Table 2
T2_MACE_PCT = [7, 37, 49, 56, 17, 30, 31]
T2_DEATH_PCT = [9, 28, 40, 49, 17, 35, 43]
T2_COMP_PCT = [12, 42, 57, 64, 25, 53, 49]
# totals in the Results text ("Follow-up")
TEXT_TOTALS = {"n": 7754, "MACE": 684, "death": 818, "composite": 1160}
# ------------------------------------------------------------------ abstract (verbatim order)
# "in patients with extra-cardiac PMI, T1MI, tachyarrhythmia, AHF, and lT2MI 51%, 41%, 57%,
# 64%, and 25% had MACE, and 38%, 27%, 40%, 49%, and 17% patients died within 1 year,
# respectively, compared to 7% and 9% in patients without PMI."
GROUPS = ["Extra-cardiac PMI", "T1MI", "Tachyarrhythmia", "AHF", "lT2MI", "No PMI"]
ABS_MACE = np.array([51, 41, 57, 64, 25, 7])
ABS_DEATH = np.array([38, 27, 40, 49, 17, 9])
# Results text, "MACE after PMI according to aetiology" (same order; no-PMI first sentence)
RES_MACE = np.array([30, 37, 49, 56, 17, 7])
def collapse(x):
"""Return counts in abstract order, with the two extra-cardiac columns summed."""
x = np.asarray(x)
return np.array([x[5] + x[6], x[1], x[2], x[3], x[4], x[0]])
n6, mace6, death6, comp6 = collapse(N), collapse(MACE), collapse(DEATH), collapse(COMP)
p_mace = 100 * mace6 / n6
p_death = 100 * death6 / n6
p_comp = 100 * comp6 / n6
def report():
print("1. Internal consistency of Table 2")
print(f" column sums: n={N.sum()} (text {TEXT_TOTALS['n']}), MACE={MACE.sum()} "
f"(text {TEXT_TOTALS['MACE']}), death={DEATH.sum()} (text {TEXT_TOTALS['death']}), "
f"MACE-or-death={COMP.sum()} (text {TEXT_TOTALS['composite']})")
print(" bound check max(MACE, death) <= composite <= MACE + non-CV death:")
for c, a, b, k, d in zip(COLS, MACE, DEATH, COMP, NONCV):
print(f" {c:22s} {max(a, b):4d} <= {k:4d} <= {a + d:4d} {max(a, b) <= k <= a + d}")
print(" Table 2 printed % vs crude count/n (rounded):")
for c, i in zip(COLS, range(7)):
r = [round(100 * MACE[i] / N[i]), round(100 * DEATH[i] / N[i]), round(100 * COMP[i] / N[i])]
print(f" {c:22s} printed {T2_MACE_PCT[i]:2d}/{T2_DEATH_PCT[i]:2d}/{T2_COMP_PCT[i]:2d}"
f" crude {r[0]:2d}/{r[1]:2d}/{r[2]:2d}")
print("\n2. Abstract 'had MACE' vs recomputed MACE and MACE-or-death (abstract order)")
print(f" {'group':18s} {'n':>5s} {'abstract':>8s} {'MACE':>14s} {'MACE or death':>16s} closer to")
for g, n, a, m, mc, k, kc in zip(GROUPS, n6, ABS_MACE, p_mace, mace6, p_comp, comp6):
closer = "composite" if abs(a - k) < abs(a - m) else "MACE"
print(f" {g:18s} {n:5d} {a:7d}% {mc:4d}/{n:<4d}={m:5.1f}% {kc:4d}/{n:<4d}={k:5.1f}% {closer}")
print(" Results-text MACE rates:", ", ".join(f"{v}%" for v in RES_MACE))
print(f" extra-cardiac from printed column %: MACE (60*30+49*31)/109 = "
f"{(60 * 30 + 49 * 31) / 109:.1f}%, composite (60*53+49*49)/109 = "
f"{(60 * 53 + 49 * 49) / 109:.1f}%")
print("\n3. Abstract deaths vs recomputed all-cause death")
for g, a, d, dc, n in zip(GROUPS, ABS_DEATH, p_death, death6, n6):
print(f" {g:18s} abstract {a:2d}% recomputed {dc}/{n} = {d:5.1f}%")
print("\n4. Size of the overstatement (abstract value vs recomputed MACE)")
for g, a, m in zip(GROUPS[:5], ABS_MACE[:5], p_mace[:5]):
print(f" {g:18s} +{a - m:4.1f} percentage points, {100 * (a / m - 1):4.0f}% relative")
print(f" lT2MI share of PMI: {N[4]}/{N[1:].sum()} = {100 * N[4] / N[1:].sum():.1f}%")
print(f" all-PMI MACE count (cf. Supplementary Table S1, 228): {MACE[1:].sum()}/{N[1:].sum()}"
f" = {100 * MACE[1:].sum() / N[1:].sum():.1f}%")
print("\n5. Ratio to the no-PMI comparator")
m0, k0 = p_mace[5], p_comp[5]
print(f" no-PMI MACE {m0:.1f}%, no-PMI MACE-or-death {k0:.1f}% (abstract comparator 7%)")
print(f" {'group':18s} {'abstract/7%':>12s} {'MACE/MACE':>10s} {'comp/comp':>10s}")
for g, a, m, k in zip(GROUPS[:5], ABS_MACE[:5], p_mace[:5], p_comp[:5]):
print(f" {g:18s} {a / 7:12.2f} {m / m0:10.2f} {k / k0:10.2f}")
print("\n6. Confidence interval for lT2MI MACE in the Results text")
print(" Results: '17%, 95% CI 22%–28%'; Table 2 MACE [14%–20%]; Table 2 MACE-or-death [22%–28%]")
# Wilson interval on the crude proportion, as a rough check of where 17% can sit
for lab, x in [("MACE", MACE[4]), ("MACE or death", COMP[4])]:
p, nn, z = x / N[4], N[4], 1.959964
c = (p + z * z / (2 * nn)) / (1 + z * z / nn)
h = z * np.sqrt(p * (1 - p) / nn + z * z / (4 * nn * nn)) / (1 + z * z / nn)
print(f" Wilson 95% CI, lT2MI {lab}: {100 * p:.1f}% [{100 * (c - h):.1f}%, {100 * (c + h):.1f}%]")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK})
def save(fig, stem):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{stem}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
def fig1():
"""Text-vs-table diagram: the abstract's numbers against the two Table 2 rows."""
from matplotlib.path import Path
W, H = 7.2, 5.7
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.axis("off")
renderer = fig.canvas.get_renderer()
XL, XR = 0.40, 6.80 # panel edges; connectors run outside them
def words(x, y, parts):
"""Draw a run of differently styled strings; return (x_start, x_end) of each."""
spans = []
for s, col, w in parts:
t = ax.text(x, y, s, fontsize=10, va="center", color=col, weight=w, style="italic")
wd = t.get_window_extent(renderer=renderer).width / fig.dpi
spans.append((x, x + wd))
x += wd
return spans
# --- abstract panel
ya0, ya1 = 4.30, 5.50
ax.add_patch(FancyBboxPatch((XL, ya0), XR - XL, ya1 - ya0,
boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=ORANGE, lw=1.5))
ax.text(XL + 0.15, 5.30, "Abstract (Methods and results)", fontsize=10, weight="bold",
va="center")
words(XL + 0.15, 5.00, [("\u201cin patients with extra-cardiac PMI, T1MI, tachyarrhythmia, "
"AHF, and lT2MI", INK, "normal")])
l2 = words(XL + 0.15, 4.72, [("51%, 41%, 57%, 64%, and 25%", ORANGE, "bold"),
(" had MACE", INK, "bold"), (", \u2026", INK, "normal")])
l3 = words(XL + 0.15, 4.46, [("\u2026 compared to ", INK, "normal"), ("7%", ORANGE, "bold"),
(" and 9% in patients without PMI.\u201d", INK, "normal")])
# --- table panel
yt0, yt1 = 1.05, 3.45
ax.add_patch(FancyBboxPatch((XL, yt0), XR - XL, yt1 - yt0,
boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=INK2, lw=1.0))
ax.text(XL + 0.15, 3.25, "Results, Table 2: counts recomputed as events / n at 1 year",
fontsize=10, weight="bold", va="center")
heads = ["Extra-\ncardiac*", "T1MI", "Tachy-\narrhythmia", "AHF", "lT2MI", "No PMI"]
colx = [2.55, 3.30, 4.07, 4.82, 5.47, 6.25]
for hx, h in zip(colx, heads):
ax.text(hx, 2.88, h, ha="center", va="center", fontsize=10, color=INK2)
ym, yc = 2.28, 1.58
rows = [("MACE\n(primary endpoint)", p_mace, mace6, ym),
("MACE or all-cause\ndeath (secondary)", p_comp, comp6, yc)]
for lab, pv, cnt, y in rows:
ax.text(XL + 0.15, y, lab, va="center", fontsize=10)
for j, (hx, v, c) in enumerate(zip(colx, pv, cnt)):
ax.text(hx, y + 0.10, f"{v:.0f}%", ha="center", va="center", fontsize=11,
color=BLUE, weight="bold")
ax.text(hx, y - 0.14, f"{c}/{n6[j]}", ha="center", va="center", fontsize=10,
color=INK2)
# cells the abstract actually reproduces
bx0, bx1 = XL + 0.07, 5.84
ax.add_patch(Rectangle((bx0, yc - 0.31), bx1 - bx0, 0.62, fill=False, ec=ORANGE, lw=2,
ls="--"))
nx0, nx1 = 5.85, 6.66
ax.add_patch(Rectangle((nx0, ym - 0.31), nx1 - nx0, 0.62, fill=False, ec=ORANGE, lw=2,
ls="--"))
def connector(pts):
codes = [Path.MOVETO] + [Path.LINETO] * (len(pts) - 1)
ax.add_patch(FancyArrowPatch(path=Path(pts, codes), arrowstyle="-|>",
mutation_scale=14, color=ORANGE, lw=2,
joinstyle="round", capstyle="round"))
# left: the five PMI values -> composite row (runs down the left margin)
xs = l2[0][0] - 0.04
connector([(xs, 4.72), (0.20, 4.72), (0.20, yc), (bx0 - 0.01, yc)])
# right: the 7% comparator -> no-PMI MACE cell (runs down the right margin)
x7 = 0.5 * (l3[1][0] + l3[1][1])
connector([(x7, 4.36), (x7, 4.08), (7.02, 4.08), (7.02, ym), (nx1 + 0.01, ym)])
ax.text(XL + 0.05, 3.92, "Conflict 1: the five PMI values labelled \u201cMACE\u201d are the "
"MACE-or-death row.", fontsize=10, ha="left", va="center")
ax.text(XL + 0.05, 3.64, "Conflict 2: the 7% comparator is the no-PMI MACE row, not "
"MACE-or-death (12%).", fontsize=10, ha="left", va="center")
ax.text(XL, 0.80, "* Table 2 sepsis and other columns summed (n = 60 + 49).", fontsize=10,
va="center", color=INK2)
ax.text(XL, 0.45, "Orange: as printed in the abstract. Blue: recomputed from Table 2 counts.",
fontsize=10, va="center", color=INK2)
save(fig, "fig1")
def fig2():
"""Dot comparison: abstract value vs recomputed MACE and recomputed MACE-or-death."""
fig, ax = plt.subplots(figsize=(7.2, 4.1), facecolor=BG)
ax.set_facecolor(BG)
y = np.arange(len(GROUPS))[::-1]
for yi, m, k in zip(y, p_mace, p_comp):
ax.plot([m, k], [yi, yi], color=GRID, lw=2, zorder=1)
ax.scatter(p_mace, y, s=90, color=BLUE, zorder=3, label="MACE, recomputed")
ax.scatter(p_comp, y, s=90, facecolor=BG, edgecolor=BLUE, lw=2, zorder=3,
label="MACE or death, recomputed")
ax.scatter(ABS_MACE, y, s=110, marker="D", facecolor="none", edgecolor=ORANGE, lw=2, zorder=4,
label="Abstract, \u201chad MACE\u201d")
for yi, a in zip(y, ABS_MACE):
ax.text(a, yi + 0.3, f"{a}%", ha="center", va="bottom", fontsize=10, color=ORANGE)
for yi, m, k in zip(y, p_mace, p_comp):
ax.text(m, yi - 0.32, f"{m:.1f}", ha="center", va="top", fontsize=10, color=BLUE)
ax.text(k, yi - 0.32, f"{k:.1f}", ha="center", va="top", fontsize=10, color=BLUE)
ax.set_yticks(y)
ax.set_yticklabels([f"{g} (n={n})" for g, n in zip(GROUPS, n6)])
ax.set_xlim(0, 72)
ax.set_ylim(-0.8, len(GROUPS) - 0.2)
ax.set_xlabel("Patients with the event within 1 year (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
fig.tight_layout(rect=(0, 0, 1, 0.92))
fig.legend(loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False,
fontsize=10, handletextpad=0.3, columnspacing=1.4)
save(fig, "fig2")
if __name__ == "__main__":
report()
fig1()
fig2()
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract gives P < .001 for MS and primary MR; Table 3 gives .049 and .034
Eur Heart J (2024) · doi:10.1093/eurheartj/ehae523 · PMC11452747
Summary
The abstract states that, on multivariable regression, concordance with recommended treatment was significantly poorer in women with mitral stenosis (MS) and primary mitral regurgitation (MR), "both P < .001". The Results text and Table 3 report this same analysis (intervention scheduled or performed among patients with a class I indication, adjusted for EuroSCORE II) as OR 0.38 (95% CI 0.15–1.00), P = .049 for MS and OR 0.60 (95% CI 0.38–0.96), P = .034 for primary MR. Wald P-values back-calculated from the printed ORs and CIs are about .046 and .031, which agree with Table 3 and not with the abstract. No other analysis in the paper gives P < .001 for either subgroup. The finding supports one of the abstract's three conclusions, and the abstract presents two borderline results, one with a CI that reaches 1.00, as highly significant.
What the paper reports
Abstract, Results:
On multivariable regression analysis, concordance with recommended treatment was significantly poorer in women with MS and primary MR (both P < .001).
Abstract, Conclusions (repeated as point (ii) of the main Conclusions section):
(ii) concordance with recommended intervention for MS and primary MR was significantly lower for women;
Results, "Clinical management strategies", third paragraph:
Regarding treatment in concordance with class I indications of the 2012 ESC/EACTS guidelines and the 2014 AHA/ACC guidelines,7,8 the overall rate of interventions scheduled or performed was documented to be significantly lower in female patients (OR, 0.72; 95% CI 0.58; 0.90; P = .004). Particularly in women with MS (OR, 0.38; 95% CI 0.15; 1.00; P = .049) and primary MR (OR, 0.60; 95% CI 0.38; 0.96, P = .034), the analysis showed a significantly lower rate of interventional treatment in concordance with respective guidelines after adjustment for the EuroSCORE II (Figure 2B).
Table 3, block "Intervention scheduled or performed among concordant patients" (columns: crude women OR [95% CI] | adjusted women OR [95% CI] | P-value adjusted OR | number of obs used):
All patients, n = 2000 | 0.74 [0.60;0.91] | 0.72 [0.58;0.90] | .004 | 1831 (91.6%)
Mitral stenosis, n = 168 | 0.46 [0.20;1.08] | 0.38 [0.15;1.00] | .049 | 157 (93.5%)
Primary mitral regurgitation, n = 414 | 0.64 [0.42;0.98] | 0.60 [0.38;0.96] | .034 | 380 (91.8%)
Methods, "Statistical analysis":
A two-sided P-value of <.05 was considered statistically significant.
Results, "Clinical management strategies", first paragraph:
The interaction between sex and type of valve disease was not statistically significant across all analyses.
What the paper's own numbers show
Method. For each odds ratio in Table 3 I reconstructed the standard error of log OR from the printed 95% CI, SE = [ln(upper) − ln(lower)] / (2 × 1.96), and computed a two-sided Wald P from z = ln(OR) / SE. Because the OR and CI limits are rounded to two decimals, I computed z two ways: from the printed point OR, and from the CI midpoint on the log scale. The spread between the two shows the effect of rounding. I also inverted the calculation to find how narrow the CI would have to be, at the printed point estimate, for P to fall below .001. Every number below is printed by recompute.py.txt. Exact regression results cannot be reproduced without individual-level data, which are not published.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| P, concordant MS, adjusted | Abstract | < .001 | — |
| P, concordant MS, adjusted | Results p3; Table 3 | .049 (OR 0.38 [0.15;1.00]) | .046 (point OR) to .050 (CI midpoint); SE 0.484, z = −2.00 |
| P, concordant primary MR, adjusted | Abstract | < .001 | — |
| P, concordant primary MR, adjusted | Results p3; Table 3 | .034 (OR 0.60 [0.38;0.96]) | .031 to .033; SE 0.236, z = −2.16 |
| |z| implied by the printed P | derived | .049; .034; .001 | 1.97; 2.12; 3.29 |
| CI needed for P < .001 at the same OR, MS | derived | 0.15–1.00 | no wider than 0.21–0.68 (SE ≤ 0.294; 1.65 times narrower) |
| CI needed for P < .001 at the same OR, primary MR | derived | 0.38–0.96 | no wider than 0.44–0.81 (SE ≤ 0.155; 1.52 times narrower) |
| P, concordant MS, crude | Table 3 | OR 0.46 [0.20;1.08], no P printed | .071 to .075 |
| P, concordant primary MR, crude | Table 3 | OR 0.64 [0.42;0.98], no P printed | .039 to .040 |
| P, concordant all patients, adjusted | Results p3; Table 3 | .004 (OR 0.72 [0.58;0.90]) | .0034 to .0037 |
| P, all interventions scheduled/performed, MS, adjusted | Table 3 | NS (OR 0.62 [0.30;1.27]) | .19 |
| P, all interventions scheduled/performed, primary MR, adjusted | Table 3 | NS (OR 0.88 [0.64;1.20]) | .41 to .43 |
| P, transcatheter approach, MS, adjusted | Table 3 | .027 (OR 3.53 [1.15;10.82]) | .027 |
| P, intervention withheld, MS, adjusted | Table 3 | NS (OR 2.42 [0.91;6.42]) | .076 to .077 |
| Ratio of printed P to the abstract's bound | Table 3 vs Abstract | .049 and .034 vs .001 | 49 and 34 |
The Table 3 P-values agree with the printed confidence intervals to within rounding, as do the other rows checked (all patients, transcatheter MS). The table and the Results text are therefore consistent with each other. The abstract is the only place that gives P < .001 for MS or primary MR. To reach that level at the printed point estimates, the standard errors would have to be about 1.5 to 1.65 times smaller than the published CIs imply, which corresponds to roughly 2.3 to 2.7 times as much statistical information.
Figure
Figure 1. The conflicting statements side by side. Left: the abstract's Results sentence, which gives "both P < .001" for MS and primary MR. Right: the same analysis as reported in the Results text and the concordant-patient block of Table 3 (adjusted for EuroSCORE II), with the printed ORs, 95% CIs and P-values in orange. The blue values are Wald P-values recomputed from those printed ORs and CIs by recompute.py.txt; they agree with Table 3.
Figure 2. Adjusted odds ratio (women vs men) for intervention scheduled or performed among patients with a concordant class I indication, on a log scale. The vertical line marks OR = 1. Orange: the OR and 95% CI printed in Table 3. Blue: at the same point estimate, the widest symmetric (on the log scale) 95% CI that would give a two-sided Wald P below .001, as computed by recompute.py.txt. The printed intervals are much wider than an abstract value of P < .001 would require, and the MS interval reaches 1.00.
Why it matters
- The abstract misstates the strength of evidence for a headline finding. Conclusion (ii), "concordance with recommended intervention for MS and primary MR was significantly lower for women", is one of three conclusions in the abstract and rests on these two results. Readers who rely on the abstract, or on indexing services that reproduce it (the PubMed abstract carries the same "both P < .001"), are told the evidence is strong. The paper's own analysis puts it close to the .05 threshold.
- The direction and nominal significance do not change. Both adjusted ORs are below 1 and both printed P-values are below the paper's stated alpha of .05. The conclusion survives as worded. What changes is how strong the evidence is: P ≈ .05 and P ≈ .03, not P < .001.
- The results are fragile in ways the abstract does not show. The MS upper confidence limit is 1.00. The crude MS estimate (OR 0.46 [0.20;1.08]) is not significant (P ≈ .07). The concordant-patient block of Table 3 reports four valve-type subgroups (AS, AR, MS, primary MR). A Bonferroni adjustment for those four would give about .20 for MS and .14 for primary MR. The paper did not prespecify such an adjustment, so this is context, not a re-analysis. The paper also reports that the sex-by-valve-type interaction "was not statistically significant across all analyses", which limits how far the MS and primary MR subgroups can be singled out from the others.
- The Discussion builds on the subgroup findings. It offers explanations for why "Class I indications to intervene on women with primary MR were significantly less often followed" and notes that "Similarly to MR, women with concordant indications for the treatment of MS were also less likely to undergo intervention." These passages give no P-values and are consistent with the Table 3 values. A reader who weighs them against the abstract's P < .001 would overrate how secure the subgroup differences are.
Explanations considered
- The abstract refers to the crude rather than the adjusted model. Not supported. The crude ORs are weaker: MS 0.46 [0.20;1.08] gives P ≈ .07 and primary MR 0.64 [0.42;0.98] gives P ≈ .04.
- The abstract refers to a different outcome for MS and primary MR. Not supported. In Table 3, "Intervention scheduled or performed" across all patients is NS for both MS (0.62 [0.30;1.27]) and primary MR (0.88 [0.64;1.20]). Transcatheter approach for MS is P = .027, transcatheter approach for MR is NS, and intervention withheld is NS for both MS and MR. The only P < .001 entries in Table 3 are two all-patient rows (transcatheter approach, OR 1.39; intervention withheld, OR 1.45), and neither is specific to MS or primary MR or concerns concordance.
- The abstract refers to the pooled concordance result or to an interaction test. Not supported. The pooled concordant-patient OR is 0.72 [0.58;0.90], P = .004, which is also not below .001. The Results state that the sex-by-valve-type interaction was not significant in any analysis.
- A different significance convention or one-sided test. Not supported. The Methods state a two-sided P < .05 threshold and no other convention. A one-sided test would halve the P-values to about .025 and .017, still far from .001.
- Rounding of the printed OR and CI. Can explain only small differences. The P-values recomputed from the rounded values range from .046 to .050 (MS) and .031 to .033 (primary MR), which bracket or sit next to the printed .049 and .034. Rounding cannot move them by a factor of 34 to 49.
- A conversion or typesetting artifact in the archived text. Ruled out. The publisher's version of record, the Europe PMC full-text XML and the PubMed abstract all read "both P < .001" in the abstract, and all give .049 and .034 in the Results and Table 3 (see Status).
- The paper's figures support P < .001. Not supported. According to the version-of-record check, Figure 2B and panel B of the Structured Graphical Abstract mark MS 0.38 and primary MR 0.60 with a single asterisk, which their legend defines as p < 0.05, with three asterisks reserved for p < 0.001. The figures agree with Table 3.
- A correction already issued. None found (see Status).
Status
- Version of record. The publisher's page (doi.org resolves to academic.oup.com, Eur Heart J 45(37):3818) has the same abstract sentence, "(both P < .001)", the same Results paragraph (P = .049 and P = .034) and the same Table 3 rows. The Europe PMC full-text XML and the PubMed abstract (PMID 39212374) agree. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The Europe PMC supplementary bundle for PMC11452747 holds only figure images (the Structured Graphical Abstract and Figures 1 to 4). It has no data supplement and no further analysis of MS or primary MR. No individual-level data are published.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, corrigendum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by entries. The only CommentsCorrections link on the PubMed record is a companion editorial (doi:10.1093/eurheartj/ehae524). The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC11452747; publisher page via https://doi.org/10.1093/eurheartj/ehae523; Europe PMC full-text XML and supplementary files; PubMed record 39212374; Crossref; PubPeer.
Suggested correction
In the abstract, replace
concordance with recommended treatment was significantly poorer in women with MS and primary MR (both P < .001).
with
concordance with recommended treatment was significantly poorer in women with MS (OR 0.38, 95% CI 0.15–1.00, P = .049) and primary MR (OR 0.60, 95% CI 0.38–0.96, P = .034).
The Results text, Table 3, Figure 2B and the Structured Graphical Abstract already give these values and would not need to change.
Recomputation script (python recompute.py.txt)
"""Recompute the numbers in the PMC11452747 write-up and regenerate its figures.
Paper: Sex-related differences in severe native valvular heart disease: the
ESC-EORP Valvular Heart Disease II survey. Eur Heart J 2024. doi:10.1093/eurheartj/ehae523
Every input below is copied from Table 3 ("Intervention scheduled or performed
among concordant patients" and other blocks) and the Results text.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Outputs fig1.svg/png and fig2.svg/png in this script's folder.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
Z95 = norm.isf(0.025) # 1.959964
# (OR, lower, upper, P as printed in Table 3)
rows = {
"Concordant, MS, adjusted": (0.38, 0.15, 1.00, ".049"),
"Concordant, MS, crude": (0.46, 0.20, 1.08, "(none)"),
"Concordant, primary MR, adjusted": (0.60, 0.38, 0.96, ".034"),
"Concordant, primary MR, crude": (0.64, 0.42, 0.98, "(none)"),
"Concordant, all patients, adjusted": (0.72, 0.58, 0.90, ".004"),
"Sched/performed, MS, adjusted": (0.62, 0.30, 1.27, "NS"),
"Sched/performed, primary MR, adj.": (0.88, 0.64, 1.20, "NS"),
"Transcatheter, MS, adjusted": (3.53, 1.15, 10.82, ".027"),
"Withheld, MS, adjusted": (2.42, 0.91, 6.42, "NS"),
"Withheld, MR, adjusted": (0.98, 0.70, 1.38, "NS"),
}
def wald(o, l, u):
"""SE of log OR from a 95% CI, and two-sided Wald P two ways:
from the printed point OR, and from the CI midpoint on the log scale."""
se = (np.log(u) - np.log(l)) / (2 * Z95)
z_pt = np.log(o) / se
z_mid = ((np.log(u) + np.log(l)) / 2) / se
return se, z_pt, 2 * norm.sf(abs(z_pt)), 2 * norm.sf(abs(z_mid))
print("1. Wald P-values back-calculated from Table 3 ORs and 95% CIs")
print(f"{'row':36s} {'OR':>5s} {'95% CI':>14s} {'printed P':>9s} {'SE':>6s} {'z':>6s} {'P(point)':>9s} {'P(mid)':>7s}")
res = {}
for k, (o, l, u, rp) in rows.items():
se, z, p_pt, p_mid = wald(o, l, u)
res[k] = (se, z, p_pt, p_mid)
print(f"{k:36s} {o:5.2f} [{l:5.2f};{u:5.2f}] {rp:>9s} {se:6.3f} {z:6.2f} {p_pt:9.4f} {p_mid:7.4f}")
print("\n2. |z| implied by the printed P-values (two-sided)")
for name, p in [("MS, P = .049", 0.049), ("primary MR, P = .034", 0.034), ("P = .001 boundary", 0.001)]:
print(f" {name:22s} |z| = {norm.isf(p / 2):.3f}")
print("\n3. What P < .001 would require at the printed point estimates")
zc = norm.isf(0.0005)
need = {}
for name, key in [("MS", "Concordant, MS, adjusted"), ("primary MR", "Concordant, primary MR, adjusted")]:
o, l, u, _ = rows[key]
se_obs = res[key][0]
se_need = abs(np.log(o)) / zc
lo_need = np.exp(np.log(o) - Z95 * se_need)
up_need = np.exp(np.log(o) + Z95 * se_need)
need[name] = (o, l, u, lo_need, up_need)
print(f" {name:11s}: printed CI [{l:.2f}; {u:.2f}], SE(log OR) = {se_obs:.3f}; "
f"P < .001 needs SE <= {se_need:.3f}, i.e. CI no wider than [{lo_need:.3f}; {up_need:.3f}]; "
f"SE would have to shrink by a factor of {se_obs / se_need:.2f} "
f"(roughly {(se_obs / se_need) ** 2:.1f} times the information)")
print("\n4. Size of the discrepancy and multiplicity context")
for name, p in [("MS", 0.049), ("primary MR", 0.034)]:
print(f" {name:11s}: printed P / .001 = {p / 0.001:.0f}; Bonferroni over the 4 valve-type subgroups "
f"(AS, AR, MS, primary MR) of the concordant-patient block of Table 3: {min(1, 4 * p):.3f}")
print("\n5. Rows of Table 3 with P < .001 (for reference; none is an MS or primary MR row)")
print(" Transcatheter approach, all patients: OR 1.39 [1.18;1.63], P <.001")
print(" Intervention withheld, all patients: OR 1.45 [1.23;1.71], P <.001")
for k in ["Transcatheter, MS, adjusted", "Withheld, MS, adjusted", "Withheld, MR, adjusted"]:
print(f" {k}: P(point) = {res[k][2]:.4f} (printed {rows[k][3]})")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT, "ytick.color": TXT,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: two text panels joined by a connector naming the conflict
p_ms, p_mr = res["Concordant, MS, adjusted"][2], res["Concordant, primary MR, adjusted"][2]
fig = plt.figure(figsize=(7.2, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 3.9); ax.axis("off")
for x0, w in [(0.15, 2.75), (3.85, 3.2)]:
ax.add_patch(FancyBboxPatch((x0, 0.55), w, 2.95, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(0.3, 3.3, "Abstract, Results", fontsize=11.5, weight="bold", va="top")
ax.text(0.3, 2.95, "“concordance with recommended\ntreatment was significantly poorer\n"
"in women with MS and primary MR”", fontsize=10.5, va="top", color=SEC, linespacing=1.35)
ax.text(0.3, 1.75, "both P < .001", fontsize=15, weight="bold", color=ORANGE, va="top")
ax.text(0.3, 1.2, "reported in the abstract", fontsize=10, color=ORANGE, va="top")
ax.text(4.0, 3.3, "Results text and Table 3", fontsize=11.5, weight="bold", va="top")
ax.text(4.0, 3.0, "concordant patients, adj. for EuroSCORE II", fontsize=10, color=SEC, va="top")
ax.text(4.0, 2.6, "MS: OR 0.38 [0.15; 1.00]", fontsize=10.5, va="top")
ax.text(4.0, 2.3, "P = .049", fontsize=13, weight="bold", color=ORANGE, va="top")
ax.text(5.1, 2.27, f"from the CI: P ≈ {p_ms:.3f}", fontsize=10.5, color=BLUE, va="top")
ax.text(4.0, 1.75, "Primary MR: OR 0.60 [0.38; 0.96]", fontsize=10.5, va="top")
ax.text(4.0, 1.45, "P = .034", fontsize=13, weight="bold", color=ORANGE, va="top")
ax.text(5.1, 1.42, f"from the CI: P ≈ {p_mr:.3f}", fontsize=10.5, color=BLUE, va="top")
ax.text(4.0, 0.95, "orange = printed in the paper\nblue = recomputed from its OR and CI",
fontsize=10, color=SEC, va="top", linespacing=1.3)
ax.add_patch(FancyArrowPatch((2.98, 1.95), (3.77, 1.95), arrowstyle="<->", mutation_scale=14,
lw=2, color=TXT))
ax.text(3.375, 2.15, "same\nanalysis", ha="center", va="bottom", fontsize=10, color=TXT)
ax.text(3.375, 1.75, "conflict:\n< .001 vs\n.049, .034", ha="center", va="top", fontsize=10,
color=TXT, weight="bold")
ax.text(0.15, 0.28, "Right panel: Table 3 block “Intervention scheduled or performed among concordant patients”",
fontsize=10, color=SEC, va="center")
save(fig, "fig1")
# Figure 2: printed OR and CI vs the CI that P < .001 would require, log scale
fig, ax = plt.subplots(figsize=(7.2, 3.3))
fig.subplots_adjust(left=0.2, right=0.97, top=0.8, bottom=0.18)
labels = []
ypos = {"MS": 1.0, "primary MR": 0.0}
for name, (o, l, u, lo_n, up_n) in need.items():
y = ypos[name]
ax.plot([l, u], [y + 0.14] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot([o], [y + 0.14], "o", color=ORANGE, ms=9)
ax.plot([lo_n, up_n], [y - 0.14] * 2, color=BLUE, lw=2, solid_capstyle="butt")
ax.plot([o], [y - 0.14], "D", color=BLUE, ms=8)
pr = rows["Concordant, MS, adjusted" if name == "MS" else "Concordant, primary MR, adjusted"][3]
ax.text(u * 1.04, y + 0.14, f"printed: {l:.2f}–{u:.2f}, P = {pr}", va="center", fontsize=10, color=ORANGE)
ax.text(up_n * 1.04, y - 0.14, f"needed for P < .001: upper ≤ {up_n:.2f}", va="center",
fontsize=10, color=BLUE)
ax.axvline(1.0, color=SEC, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.1, 4.0)
ticks = [0.1, 0.2, 0.5, 1.0, 2.0]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.set_yticks([1.0, 0.0]); ax.set_yticklabels(["Mitral stenosis\n(n used = 157)", "Primary MR\n(n used = 380)"])
ax.set_ylim(-0.55, 1.5)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Adjusted odds ratio, women vs men (log scale); 1 = no difference")
ax.plot([], [], "o-", color=ORANGE, ms=9, lw=2, label="Printed in Table 3: OR and 95% CI")
ax.plot([], [], "D-", color=BLUE, ms=8, lw=2, label="Widest 95% CI compatible with P < .001 (recomputed)")
ax.legend(loc="lower left", bbox_to_anchor=(-0.2, 1.02), ncol=1, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg, fig1.png, fig2.svg, fig2.png")
Abstract reports p<0.001 for the matched cystic fibrosis versus rheumatoid arthritis comparison; Table 4 gives p=0.055
Eur Respir J (2023) · doi:10.1183/13993003.00174-2023 · PMC10600351
Summary
The abstract reports the propensity-matched comparison of major adverse cardiac events (MACE) between people with cystic fibrosis (CF) and people with rheumatoid arthritis (RA) as HR 1.21 (95% CI 1.00–1.48) with p<0.001. Table 4 gives the same HR and CI for this comparison, with p=0.055. Back-calculating from the printed HR and CI gives p = 0.057, or 0.047–0.068 if the rounded values vary within their rounding ranges. That agrees with Table 4 and rules out p<0.001, which would need a CI of about 1.08–1.36. The only p<0.001 in the RA row of Table 4 belongs to the unadjusted analysis (HR 0.75), so the abstract presents a borderline, non-significant comparison as highly significant. The Results and Discussion also describe CF risk as "greater than" RA risk. The paper's main finding, a higher MACE risk in CF than in matched general-population comparators (HR 1.65, p<0.001), is not affected.
What the paper reports
Abstract, Results:
In propensity score-matched analyses, PwCF had higher risk of MACE than matched general population comparators (hazard ratio (HR) 1.65, 95% CI 1.40–1.95; p<0.001) and an equivalent or higher relative risk compared with other inflammatory conditions considered “high risk” for cardiovascular disease, including systemic lupus erythematosus (HR 0.95, 95% CI 0.82–1.09; p=0.44), rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001) and HIV (HR 0.93, 95% CI 0.82–1.06; p=0.29).
Table 4, "Rheumatoid arthritis" row. The columns are: unadjusted analysis (CF | events | non-CF | events | HR (95% CI) | p-value), then propensity score-matched analysis (the same six columns):
Rheumatoid arthritis | 6265 | 227 | 97 198 | 4547 | 0.75 (0.66–0.86) | <0.001 | 5901 | 223 | 5901 | 177 | 1.21 (1.00–1.48) | 0.055
Table 4 footer:
Propensity score matching based on known cardiac risk factors (age, sex, diabetes, hypertension, hypercholesterolaemia, family history of heart disease, smoking and chronic kidney disease).
Methods, "Statistical analysis":
After propensity score matching, a Cox's proportional hazards model was used to compare the matched cohorts.
Results, "Outcomes compared with other inflammatory diseases":
In CF, risk of MACE was equivalent to SLE and HIV (HR 0.95, 95% CI 0.82–1.09; p=0.44; and HR 0.93, 95% CI 0.82–1.06; p=0.29, respectively) and was greater than rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48).
Discussion, first paragraph:
Furthermore, cardiac risk in PwCF was equivalent to or higher than other chronic inflammatory diseases considered “high risk” for cardiac disease.
Discussion, fifth paragraph (the paragraph beginning "Chronic inflammation is a hallmark of CF"):
Here, we found that despite adjusting for the presence of classic cardiac risk factors, risk of cardiac disease in CF was equivalent to or greater than that of other inflammatory conditions.
What the paper's own numbers show
Method. Hazard ratios from a Cox model are normally reported with a Wald 95% CI on the log scale. For such a CI, SE(log HR) = (ln upper − ln lower) / (2 × 1.96), z = ln HR / SE, and the two-sided p = 2(1 − Φ(|z|)). This cannot reproduce the exact Cox p-value without the individual event times, but it is the standard check that a printed HR, CI and p-value are consistent. I applied it to the four matched comparisons the abstract reports. To allow for rounding, I also varied the RA HR and both bounds over every value that rounds to the printed two decimals. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CF vs RA, matched: p-value | Abstract, Results | HR 1.21 (1.00–1.48); p<0.001 | SE(log HR) 0.100, z = 1.91, p = 0.057 |
| CF vs RA, matched: p-value | Table 4 | HR 1.21 (1.00–1.48); p = 0.055 | p = 0.057 (agrees) |
| CF vs RA, matched: p over rounding of HR and CI | Table 4 / Abstract | — | 0.047–0.068 |
| CF vs RA, matched: HR against CI midpoint | Table 4 / Abstract | 1.21 | geometric midpoint √(1.00 × 1.48) = 1.217 (consistent) |
| CF vs RA, matched: what p<0.001 would need | — | — | |z| > 3.29, SE ≤ 0.058, so a 95% CI no wider than 1.08–1.36 |
| CF vs RA, matched: one-sided p | — | — | 0.028 |
| CF vs RA, unadjusted | Table 4 | HR 0.75 (0.66–0.86); p<0.001 | z = −4.26, p = 2.0 × 10⁻⁵ (agrees); the only p<0.001 in the RA row |
| CF vs RA, matched: crude event counts | Table 4 | CF 223/5901 vs RA 177/5901 | 3.78% vs 3.00%; crude risk ratio 1.26 (1.04–1.53), p = 0.020 (ignores time to event and censoring, so it does not replace the Cox p) |
| CF vs SLE, matched (cross-check) | Abstract / Table 4 | HR 0.95 (0.82–1.09); p = 0.44 / 0.439 | p = 0.48 (rough agreement, within rounding of the CI) |
| CF vs HIV, matched (cross-check) | Abstract / Table 4 | HR 0.93 (0.82–1.06); p = 0.29 / 0.293 | p = 0.27 (rough agreement) |
| CF vs general population, matched (cross-check) | Abstract / Table 3 | HR 1.65 (1.40–1.95); p<0.001 | z = 5.92, p = 3 × 10⁻⁹ (agrees) |
The same back-calculation agrees with the printed p-values for the other three matched comparisons. For RA it agrees with Table 4 but not with the abstract. A CI whose lower bound is 1.00 is by construction close to p = 0.05, and a two-sided p<0.001 would require the lower bound to be at least about 1.08.
Figure
Figure 1. P-values for the four propensity-matched MACE comparisons named in the abstract, on one log-scale axis. Orange diamonds are the values printed in the abstract and orange squares the values printed in Table 3 (general population) or Table 4 (SLE, RA, HIV). A left-pointing arrow marks a value printed as "p<". Blue circles are the p-values recomputed from the printed HR and 95% CI. For RA, the blue bar spans the p-values obtained when the HR and both CI bounds vary within their rounding ranges (0.047–0.068). The dashed line marks p = 0.05 and the dotted line p = 0.001. For the general population, SLE and HIV the three values agree. For RA, Table 4 and the recomputed value sit on the 0.05 line, while the abstract places the result beyond 0.001. The recomputed general-population p (3 × 10⁻⁹) lies beyond the left edge of the axis.
Figure 2. Where the abstract's RA values occur in Table 4. The abstract's HR and CI (1.21, 1.00–1.48) are those of the propensity-matched columns, whose p-value is 0.055. The only p<0.001 in the RA row is in the unadjusted columns, for an HR of 0.75 (0.66–0.86) that points the other way. The figure shows where the printed values coincide. It does not show how the abstract's value arose.
Why it matters
- The abstract misstates the significance of a highlighted comparison. The abstract lists RA among the inflammatory conditions against which CF has "an equivalent or higher relative risk", and gives p<0.001 for it. The paper's own table, and the HR and CI themselves, put this comparison at p ≈ 0.055–0.057, just above the conventional 0.05 threshold. Readers who see only the abstract, or who index it, will take the RA comparison as strong evidence of excess risk in CF.
- The "greater than" claims rest on this comparison alone. The Results say CF risk "was greater than rheumatoid arthritis". The first and fifth Discussion paragraphs say CF risk was "equivalent to or higher than" or "equivalent to or greater than" that of other inflammatory conditions. SLE and HIV are both described as equivalent (p = 0.44 and 0.29), so the "higher" or "greater" part depends only on RA. With a CI whose lower bound is 1.00 and p = 0.055, the data are compatible with no difference, so "greater than" is not established at the 0.05 level. The point estimate still favours higher risk in CF.
- What is not affected. The comparison with matched general-population comparators (HR 1.65, 1.40–1.95, p<0.001), the SLE and HIV comparisons, the UK CF Registry risk-factor analyses, and the abstract's conclusion that "PwCF are at increased risk of adverse cardiac disease events" do not depend on this p-value. The argument in the Discussion that risk tools such as QRISK may need CF-specific adjustment, as they have for SLE, RA and HIV, holds equally well if CF risk is similar to RA risk.
Explanations considered
- A different analysis. Table 4 does contain a p<0.001 for RA, but in the unadjusted columns, with HR 0.75 (0.66–0.86). That HR points the other way (lower MACE risk in CF than in RA). The abstract pairs a p<0.001 with the matched HR and CI. The coincidence is shown in Figure 2. It would explain how the two values came to differ but would not make the abstract's p-value correct for the matched comparison it is attached to.
- Rounding. Letting the HR and both CI bounds take any value that rounds to 1.21, 1.00 and 1.48 gives p between 0.047 and 0.068. That is nowhere near 0.001, and the table's three-decimal p of 0.055 sits inside this range. Rounding does not account for the discrepancy.
- A one-sided test. The Methods mention no one-sided tests. A one-sided p from the same HR and CI would be 0.028, still not <0.001.
- Another model or population. The Methods describe a single approach: 1:1 propensity matching followed by a Cox proportional hazards model on the matched cohorts. The Table 4 footer lists the matching covariates and mentions no other model. Supplementary Table S4 (CF vs RA) gives the same matched MACE result, HR 1.21 (1.00–1.48), p = 0.055. No other analysis in the paper or supplement reports the matched RA comparison with p<0.001.
- Crude counts. Comparing 223/5901 with 177/5901 events without time to event gives p = 0.020. That is not <0.001 either, and it is not the analysis the paper reports.
- Conversion artifact. The publisher-typeset one-page summary PDF in the supplementary bundle and the Europe PMC full-text XML print the same abstract p<0.001 and the same Table 4 p = 0.055. The discrepancy is in the published article, not an artifact of the PMC conversion.
- A later correction. None was found (see Status).
None of these accounts for p<0.001. The Table 4 value of 0.055 agrees with the recomputation, so the abstract's value is the one in error.
A side observation, not part of this finding: the Table 4 SLE matched row lists the CF group as n = 223 with 385 events and the comparator as n = 5223. The Results text states that 5223 PwCF were matched with SLE comparators, so the "223" looks like a dropped digit. This was not examined further.
Status
As of 2026-10-23:
- Version of record. The publisher's pages (publications.ersnet.org, erj.ersjournals.com) returned HTTP 403 to automated access. The publisher-typeset one-page summary PDF (ERJ-00174-2023.Shareable.pdf, from the Europe PMC supplementary bundle) prints the same abstract text, "rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001)". The Europe PMC JATS XML and the live PMC page carry the same abstract and the same Table 4 matched RA p-value of 0.055.
- Supplement. ERJ-00174-2023.Supplement.pdf (17 pages). Supplementary Table S4 (CF vs RA) gives the matched MACE row as CF 5901 / 223 events vs RA 5901 / 177 events, HR 1.21 (1.00–1.48), p 0.055, and the unadjusted row as HR 0.75 (0.66–0.86), p<0.001. The supplement agrees with Table 4 and does not resolve the discrepancy.
- Corrections. Crossref lists no update-to or updated-by relations for the DOI. PMC and Europe PMC list no correction, erratum, retraction or expression of concern, and a Europe PMC search for errata returned none. The only linked item is an editorial, "Cystic fibrosis and the cardiovascular system: the unexpected heartache" (doi:10.1183/13993003.01253-2023; PMID 37884293), which is commentary rather than a correction.
- PubPeer and other commentary. PubPeer blocked automated access (403 and a Cloudflare challenge). A web search found no PubPeer thread or Retraction Watch entry for this article.
- Sources. The article as archived in PMC (PMC10600351: abstract, Methods, Results, Tables 3 and 4, Discussion); the Europe PMC full-text XML and supplementary files; Crossref, PMC and Europe PMC notice records for doi:10.1183/13993003.00174-2023.
Suggested correction
In the Abstract, Results, replace "rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001)" with "rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p=0.055)", in line with Table 4 and Supplementary Table S4.
Because this comparison does not reach p<0.05, the authors may also wish to revise the wording that reads it as an established difference. In the Results, "and was greater than rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48)" could read "and was numerically higher than, but not significantly different from, rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p=0.055)". The phrases "equivalent or higher" (Abstract), "equivalent to or higher than" (Discussion, first paragraph) and "equivalent to or greater than" (Discussion, fifth paragraph) could be qualified in the same way.
Recomputation script (python recompute.py.txt)
"""Recompute the p-values of the propensity-matched MACE comparisons in PMC10600351
(Eur Respir J 2023, doi:10.1183/13993003.00174-2023).
Run: python recompute.py.txt # prints all numbers, writes fig1/fig2 (.svg, .png)
python recompute.py.txt --candidates # also writes five candidate figure designs
# to candidates/ (A-E) for review
Needs numpy, scipy and matplotlib.
Inputs are copied from the paper as archived in PMC:
Abstract (Results), Results "Outcomes compared with other inflammatory diseases",
Table 3 (CF vs general population) and Table 4 (CF vs SLE, RA, HIV).
Method: for a ratio reported with a 95% Wald CI on the log scale,
SE(log HR) = (ln upper - ln lower) / (2 * 1.959964), z = ln HR / SE,
two-sided p = 2 * (1 - Phi(|z|)).
The exact Cox p-value cannot be rebuilt without individual event times; this
back-calculation is the standard check of internal consistency.
"""
import itertools
import os
import sys
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
# ----------------------------------------------------------------- inputs (from the paper)
# name: (HR, lower, upper, p in Table 3/4, p in abstract)
MATCHED = {
"General population": (1.65, 1.40, 1.95, "<0.001", "<0.001"), # Table 3, abstract
"SLE": (0.95, 0.82, 1.09, "0.439", "0.44"), # Table 4, abstract
"Rheumatoid arthritis": (1.21, 1.00, 1.48, "0.055", "<0.001"), # Table 4, abstract
"HIV": (0.93, 0.82, 1.06, "0.293", "0.29"), # Table 4, abstract
}
RA_UNADJ = (0.75, 0.66, 0.86, "<0.001") # Table 4, RA row, unadjusted columns
RA_MATCHED_EVENTS = (223, 5901, 177, 5901) # Table 4: CF events/n, RA events/n
SLE_ROW_N = (223, 385, 5223, 389) # Table 4 SLE matched: CF n, events, non-CF n, events
def p_from_ci(hr, lo, hi):
se = (np.log(hi) - np.log(lo)) / (2 * Z975)
z = np.log(hr) / se
return se, z, 2 * norm.sf(abs(z))
def fmt_p(p):
return f"{p:.2g}" if p < 0.001 else f"{p:.3f}"
# ----------------------------------------------------------------- 1. back-calculated p
print("1. p-values back-calculated from the reported HR and 95% CI (matched analyses)")
RES = {}
for k, (h, lo, hi, p_tab, p_abs) in MATCHED.items():
se, z, p = p_from_ci(h, lo, hi)
RES[k] = (se, z, p)
print(f" {k:21s} HR {h:.2f} ({lo:.2f}-{hi:.2f}): SE(logHR)={se:.4f} z={z:+.3f} "
f"p={fmt_p(p)} table {p_tab}, abstract {p_abs}")
print(f" General population p = {RES['General population'][2]:.1e}")
# ----------------------------------------------------------------- 2. rounding range for RA
h0, l0, u0 = MATCHED["Rheumatoid arthritis"][:3]
grid = []
for h, l, u in itertools.product(np.linspace(h0 - .005, h0 + .005, 21),
np.linspace(l0 - .005, l0 + .005, 21),
np.linspace(u0 - .005, u0 + .005, 21)):
grid.append(p_from_ci(h, l, u)[2])
P_LO, P_HI = min(grid), max(grid)
print("\n2. RA matched: p over every HR, lower and upper bound consistent with rounding to 2 dp")
print(f" p range {P_LO:.4f} - {P_HI:.4f}")
print(f" geometric midpoint of CI sqrt(1.00*1.48) = {np.sqrt(l0 * u0):.3f} (reported HR 1.21)")
print(f" one-sided p = {RES['Rheumatoid arthritis'][2] / 2:.4f}")
# ----------------------------------------------------------------- 3. what p<0.001 would need
z001 = norm.isf(0.0005)
se_need = np.log(h0) / z001
CI_NEED = (h0 * np.exp(-Z975 * se_need), h0 * np.exp(Z975 * se_need))
print("\n3. RA matched: what a two-sided p<0.001 would require at HR 1.21")
print(f" |z| > {z001:.3f}; SE(logHR) <= {se_need:.4f} (reported CI implies "
f"{RES['Rheumatoid arthritis'][0]:.4f}); 95% CI would be at most "
f"{CI_NEED[0]:.2f}-{CI_NEED[1]:.2f}")
# ----------------------------------------------------------------- 4. the unadjusted RA row
se_u, z_u, p_u = p_from_ci(*RA_UNADJ[:3])
print("\n4. RA unadjusted (Table 4): HR 0.75 (0.66-0.86), reported p<0.001")
print(f" SE={se_u:.4f} z={z_u:+.3f} p={p_u:.1e} -> the only p<0.001 in the RA row")
# ----------------------------------------------------------------- 5. crude counts
a, n1, b, n2 = RA_MATCHED_EVENTS
rr = (a / n1) / (b / n2)
se_rr = np.sqrt(1 / a - 1 / n1 + 1 / b - 1 / n2)
z_rr = np.log(rr) / se_rr
CRUDE = (rr, rr * np.exp(-Z975 * se_rr), rr * np.exp(Z975 * se_rr), 2 * norm.sf(abs(z_rr)))
print("\n5. RA matched event counts 223/5901 vs 177/5901 (ignores time to event)")
print(f" risk {a / n1 * 100:.2f}% vs {b / n2 * 100:.2f}%; crude risk ratio {CRUDE[0]:.3f} "
f"(95% CI {CRUDE[1]:.3f}-{CRUDE[2]:.3f}), p={CRUDE[3]:.4f}")
# ----------------------------------------------------------------- 6. side note (not part of the claim)
print("\n6. Side note: Table 4 SLE matched row lists CF n=%d with %d events, against %d "
"non-CF; the Results text says 5223 PwCF were matched" % SLE_ROW_N[:3])
# ================================================================= figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, path):
fig.savefig(path + ".svg", bbox_inches="tight")
fig.savefig(path + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def style_ax(ax, grid_axis="x"):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
def fig_pvalue_axis(path):
"""Design A: reported vs recomputed p-values on one log axis, with alpha 0.05."""
order = ["General population", "SLE", "Rheumatoid arthritis", "HIV"]
labels = {"General population": "CF vs general population\nHR 1.65 (1.40–1.95)",
"SLE": "CF vs SLE\nHR 0.95 (0.82–1.09)",
"Rheumatoid arthritis": "CF vs rheumatoid arthritis\nHR 1.21 (1.00–1.48)",
"HIV": "CF vs HIV\nHR 0.93 (0.82–1.06)"}
XMIN, XMAX = 1e-5, 1.0
fig, ax = plt.subplots(figsize=(7.2, 4.8))
ys = {k: len(order) - 1 - i for i, k in enumerate(order)}
ax.axvline(0.05, color=INK2, lw=1, ls=(0, (4, 3)))
ax.text(0.05 * 1.12, 3.5, "p = 0.05", color=INK2, fontsize=10, va="center", ha="left")
ax.axvline(0.001, color=INK2, lw=0.8, ls=":")
ax.text(0.001 * 1.12, 3.5, "p = 0.001", color=INK2, fontsize=10, va="center", ha="left")
D = 0.24
def lt_arrow(x, y, mk):
ax.annotate("", xy=(x / 6, y), xytext=(x, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2))
ax.plot(x, y, mk, ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=4)
for k in order:
y = ys[k]
p_tab, p_abs = MATCHED[k][3], MATCHED[k][4]
for val, mk, yy in ((p_abs, "D", y + D), (p_tab, "s", y)):
if val.startswith("<"):
lt_arrow(float(val[1:]), yy, mk)
else:
ax.plot(float(val), yy, mk, ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=4)
yb = y - D
p_rec = RES[k][2]
if p_rec < XMIN:
ax.annotate("", xy=(XMIN, yb), xytext=(XMIN * 8, yb),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2))
ax.text(XMIN * 11, yb, f"recomputed: {p_rec:.0e} (beyond axis)", color=INK,
fontsize=10, va="center")
continue
if k == "Rheumatoid arthritis":
ax.plot([P_LO, P_HI], [yb, yb], color=BLUE, lw=2, solid_capstyle="butt")
for xv in (P_LO, P_HI):
ax.plot([xv, xv], [yb - .08, yb + .08], color=BLUE, lw=2)
ax.plot(p_rec, yb, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=4)
y = ys["Rheumatoid arthritis"]
ax.text(0.001 / 9, y + D, "abstract: p<0.001 ", color=INK, fontsize=10,
ha="right", va="center", fontweight="bold")
ax.text(0.055 / 1.4, y, "Table 4: 0.055", color=INK, fontsize=10, va="center", ha="right")
ax.text(P_LO / 1.25, y - D, f"recomputed: {RES['Rheumatoid arthritis'][2]:.3f} "
f"(rounding range {P_LO:.3f}–{P_HI:.3f})", color=INK, fontsize=10, va="center",
ha="right")
ax.set_yticks([ys[k] for k in order], [labels[k] for k in order])
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.set_xticks([1e-5, 1e-4, 1e-3, 1e-2, 0.05, 1e-1, 1],
["0.00001", "0.0001", "0.001", "0.01", "0.05", "0.1", "1"])
ax.minorticks_off()
ax.set_ylim(-0.5, 3.75)
ax.set_xlabel("Two-sided p-value (log scale)")
style_ax(ax)
h = [plt.Line2D([], [], marker="D", ls="", ms=8, color=ORANGE, label="Reported in abstract\n(arrow: “p<” value)"),
plt.Line2D([], [], marker="s", ls="", ms=8, color=ORANGE, label="Reported in Table 3 / 4"),
plt.Line2D([], [], marker="o", ls="-", lw=2, ms=8, color=BLUE,
label="Recomputed from HR and 95% CI\n(bar: range over rounding)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.4, -0.15), ncol=3,
frameon=False, fontsize=9.5, handletextpad=0.4, columnspacing=1.2)
ax.set_title("Propensity-matched MACE comparisons: reported and recomputed p-values",
loc="left", fontsize=11.5, color=INK, x=-0.38, pad=18)
save(fig, path)
def box(ax, x, y, w, h, title, body, edge, fs=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.015",
fc="#ffffff", ec=edge, lw=2, transform=ax.transAxes,
clip_on=False))
ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, fontsize=11,
fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.13, body, transform=ax.transAxes, fontsize=fs,
va="top", color=INK, linespacing=1.45)
def fig_text_panels(path):
"""Design B: abstract vs Table 4 excerpts with a connector naming the conflict."""
fig = plt.figure(figsize=(7.2, 3.3))
ax = fig.add_axes([0.02, 0.12, 0.96, 0.84])
ax.set_axis_off()
box(ax, 0.0, 0.36, 0.44, 0.60, "Abstract, Results",
"“rheumatoid arthritis\n(HR 1.21, 95% CI 1.00–1.48;\n p<0.001)”", ORANGE, fs=11)
box(ax, 0.56, 0.36, 0.44, 0.60, "Table 4, propensity-matched",
"Rheumatoid arthritis\nCF 223/5901 vs RA 177/5901\nHR 1.21 (1.00–1.48)\np-value 0.055",
ORANGE, fs=11)
ax.annotate("", xy=(0.555, 0.66), xytext=(0.445, 0.66), xycoords="axes fraction",
arrowprops=dict(arrowstyle="<|-|>", color=INK2, lw=2))
ax.text(0.5, 0.71, "same HR\nand CI", ha="center", fontsize=10, color=INK2,
transform=ax.transAxes)
ax.text(0.5, 0.61, "different\np-value", ha="center", fontsize=10, color=INK,
fontweight="bold", transform=ax.transAxes, va="top")
ax.text(0.0, 0.18, f"Recomputed from HR 1.21 and CI 1.00–1.48: p = "
f"{RES['Rheumatoid arthritis'][2]:.3f} (range {P_LO:.3f}–{P_HI:.3f} over rounding).",
transform=ax.transAxes, fontsize=10.5, color=BLUE, fontweight="bold")
ax.text(0.0, 0.06, "Orange boxes: text as printed in the paper. Blue: recomputed here.",
transform=ax.transAxes, fontsize=9.5, color=INK2)
save(fig, path)
def fig_hr_needed(path):
"""Design C: reported CI vs the widest CI compatible with p<0.001, on a log HR axis."""
fig, ax = plt.subplots(figsize=(7.2, 2.6))
rows = [("Reported (abstract and Table 4)\nHR 1.21, 95% CI 1.00–1.48", (l0, u0), ORANGE),
(f"Widest 95% CI that would give\np<0.001 at HR 1.21: "
f"{CI_NEED[0]:.2f}–{CI_NEED[1]:.2f}", CI_NEED, BLUE)]
ax.axvline(1, color=INK2, lw=1)
ax.text(1.005, 1.55, "HR = 1 (no difference)", color=INK2, fontsize=10, ha="left")
for i, (lab, (lo, hi), c) in enumerate(rows):
y = 1 - i
ax.plot([lo, hi], [y, y], color=c, lw=2)
ax.plot(h0, y, "o", ms=9, color=c, mec=BG, mew=1.5)
ax.text(hi * 1.02, y, f"{lo:.2f}–{hi:.2f}", color=INK, fontsize=10, va="center")
ax.set_yticks([1, 0], [r[0] for r in rows])
ax.set_xscale("log")
ax.set_xlim(0.9, 1.7)
ax.set_xticks([0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6], ["0.9", "1.0", "1.1", "1.2", "1.3",
"1.4", "1.5", "1.6"])
ax.minorticks_off()
ax.set_ylim(-0.5, 1.8)
ax.set_xlabel("Hazard ratio, CF vs rheumatoid arthritis (propensity-matched, log scale)")
style_ax(ax)
save(fig, path)
def fig_z(path):
"""Design D: |z| implied by each reported CI against the 0.05 and 0.001 thresholds."""
order = ["General population", "SLE", "Rheumatoid arthritis", "HIV"]
fig, ax = plt.subplots(figsize=(7.2, 3.0))
for i, k in enumerate(order):
y = len(order) - 1 - i
z = abs(RES[k][1])
ax.plot([0, z], [y, y], color=BLUE, lw=2)
ax.plot(z, y, "o", ms=8, color=BLUE, mec=BG, mew=1.5)
ax.text(z + 0.12, y, f"|z| = {z:.2f}", va="center", fontsize=10)
yra = 1
ax.plot(z001, yra + 0.28, "D", ms=8, color=ORANGE)
ax.text(z001 + 0.12, yra + 0.28, "abstract's p<0.001 needs |z| > 3.29", color=INK,
fontsize=10, va="center")
for zz, lab in ((Z975, "p = 0.05"), (z001, "p = 0.001")):
ax.axvline(zz, color=INK2, lw=1, ls=(0, (4, 3)))
ax.text(zz + 0.06, 3.55, lab, ha="left", fontsize=10, color=INK2)
ax.set_yticks(range(4)[::-1], ["vs general population", "vs SLE", "vs rheumatoid arthritis",
"vs HIV"])
ax.set_xlim(0, 6.8)
ax.set_ylim(-0.5, 3.8)
ax.set_xlabel("|z| implied by the reported HR and 95% CI (blue, recomputed)")
style_ax(ax)
save(fig, path)
def fig_row_origin(path):
"""Design E: the Table 4 RA row with both columns; the abstract's p matches the wrong one."""
fig = plt.figure(figsize=(7.2, 3.7))
ax = fig.add_axes([0.02, 0.1, 0.96, 0.8])
ax.set_axis_off()
box(ax, 0.01, 0.58, 0.46, 0.38, "Table 4, RA: unadjusted",
"HR 0.75 (0.66–0.86)\np-value <0.001", INK2, fs=11)
box(ax, 0.53, 0.58, 0.46, 0.38, "Table 4, RA: propensity-matched",
"HR 1.21 (1.00–1.48)\np-value 0.055", INK2, fs=11)
box(ax, 0.12, 0.04, 0.76, 0.30, "Abstract, Results",
"“rheumatoid arthritis (HR 1.21, 95% CI 1.00–1.48; p<0.001)”", ORANGE, fs=10.5)
ax.annotate("", xy=(0.66, 0.36), xytext=(0.76, 0.565), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=2))
ax.text(0.73, 0.46, "HR and CI as printed here", fontsize=10, color=INK2,
transform=ax.transAxes, va="center")
ax.annotate("", xy=(0.34, 0.36), xytext=(0.24, 0.565), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, ls="--"))
ax.text(0.27, 0.46, "the row's only p<0.001\n(a different analysis, HR < 1)",
fontsize=10, color=INK, transform=ax.transAxes, va="center", ha="right")
ax.text(0.01, -0.07, f"Recomputed from the matched HR and CI: p = "
f"{RES['Rheumatoid arthritis'][2]:.3f}, consistent with Table 4's 0.055 (blue = "
"recomputed here).", transform=ax.transAxes, fontsize=10, color=BLUE)
ax.text(0.01, 1.04, "Where the abstract's RA values match Table 4", transform=ax.transAxes,
fontsize=11.5, color=INK)
save(fig, path)
# write the chosen figures
fig_pvalue_axis(os.path.join(HERE, "fig1"))
fig_row_origin(os.path.join(HERE, "fig2"))
print("\nwrote fig1.svg/png (p-value axis) and fig2.svg/png (Table 4 row vs abstract)")
if "--candidates" in sys.argv:
cdir = os.path.join(HERE, "candidates")
os.makedirs(cdir, exist_ok=True)
for tag, fn in (("A_pvalue_axis", fig_pvalue_axis), ("B_text_panels", fig_text_panels),
("C_ci_needed", fig_hr_needed), ("D_z_thresholds", fig_z),
("E_table_row_origin", fig_row_origin)):
fn(os.path.join(cdir, tag))
print("wrote candidates/A-E (.svg, .png)")
Abstract transposes two of four unilateral and bilateral effusion mortality figures given correctly in Results
Eur Respir J (2015) · doi:10.1183/09031936.00217114 · PMC4857137
Summary
This cohort of 308 patients undergoing thoracentesis (247 unilateral, 61 bilateral) reports in its abstract that bilateral effusion, relative to unilateral, carried higher mortality, giving "17% versus 47%" with the 30-day hazard ratio and "36% versus 69%" with the 1-year hazard ratio. The Results instead give unilateral mortality as 17% (30 days) and 47% (1 year) and bilateral mortality as 36% and 69%, and only these figures reproduce the paper's reported totals of 65 and 158 deaths (43/247 + 22/61 and 116/247 + 42/61); no count out of 61 even rounds to 47%. The abstract, including the version of record indexed in PubMed, has therefore transposed 47% and 36%: it overstates bilateral 30-day mortality (47% instead of 36%), understates unilateral 1-year mortality (36% instead of 47%), and makes the absolute gap look 11 percentage points larger at both time points. The hazard ratios and the direction of the finding are unaffected, but this is the comparison the paper presents as its new contribution, and the incorrect 47% figure has since been cited.
What the paper reports
Abstract, Results (PMC author manuscript):
Patients with bilateral pleural effusion, relative to unilateral, were associated with higher risk of death at 30 days and 1 year (17% versus 47%; HR 2.58 CI [1.44–4.63] and 36% versus 69%; HR 2.32 CI [1.55–3.48])
Abstract, Results, as published in Eur Respir J 46(2):495–502 and indexed in PubMed (PMID 25837039):
Patients with bilateral, relative to unilateral, pleural effusion were associated with higher risk of death at 30 days and 1 year (17% versus 47% (hazard ratio (HR) 2.58, 95% CI 1.44-4.63) and 36% versus 69% (HR 2.32, 95% CI 1.55-3.48), respectively).
Results, first paragraph:
A total of 247 unilateral and 61 bilateral thoracenteses were performed. Overall mortality across all groups was 21% (65 of 308 patients) at 30-days and 51% (158 of 308 patients) at one year
Results, second paragraph:
As calculated from Figure 1, thirty-day and one-year mortality of patients with unilateral effusion was 17% and 47%, compared with 36% and 69% in patients with bilateral effusion.
Relative to patients with unilateral effusion, unadjusted mortality of patients with bilateral effusion was higher at thirty-days (p=0.0003) and at one-year (p=0.0001).
Results, fifth paragraph (Table 3, row "Bilateral", reads the same: 2.58 (1.44, 4.63), 0.002 | 2.32 (1.55, 3.48), <0.001):
was significantly associated with higher 30-day and 1-year mortality (HR 2.58 [1.44–4.63] p=0.002 and HR 2.32 [1.55–3.48] p<0.001, respectively)
Methods, Statistical Analysis, on what the percentages are:
Unadjusted 30-day and 1-year mortalities were calculated and diagrammed for subgroups according to the following hierarchy: laterality (bilateral versus unilateral) and malignancy (yes versus no).
Discussion, on the weight the paper gives the comparison:
The study is novel in evaluating mortality associated with bilateral effusions.
It is also the first study to report the association between bilateral pleural effusion and mortality.
Abstract, Conclusions:
Bilateral pleural effusion is distinctly associated with high mortality.
What the paper's own numbers show
Method. The group sizes (247 and 61) and the overall death counts (65 and 158 of 308) come from the Results. For each printed percentage I listed every integer number of deaths k for which k/n rounds to that percentage. I then asked which pairing of unilateral and bilateral percentages can reproduce the reported overall totals: the pairing in the Results (unilateral 17%/47%, bilateral 36%/69%), or the pairing in the abstract, where each "x% versus y%" is tied to one time point by the hazard ratio that follows it (30 days: 17% vs 47%; 1 year: 36% vs 69%). For each correct split I recomputed the risk difference and crude risk ratio. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Group sizes | Abstract; Results ¶1 | 247 unilateral, 61 bilateral | 247 + 61 = 308 |
| Overall deaths, 30 days | Results ¶1 | 65 of 308 (21%) | 21.1% |
| Overall deaths, 1 year | Results ¶1 | 158 of 308 (51%) | 51.3% |
| Unilateral, 30 days | Results ¶2; abstract | 17% | 43/247 = 17.4% (41–43 deaths round to 17%) |
| Bilateral, 30 days | Results ¶2 | 36% | 22/61 = 36.1% (22 is the only count giving 36%) |
| Bilateral, 30 days | Abstract (30-day pair, with HR 2.58) | 47% | not attainable: 28/61 = 45.9%, 29/61 = 47.5%; the value is the unilateral 1-year figure |
| Unilateral, 1 year | Results ¶2 | 47% | 116/247 = 47.0% |
| Unilateral, 1 year | Abstract (1-year pair, with HR 2.32) | 36% | would mean 88–90 deaths; the value is the bilateral 30-day figure |
| Bilateral, 1 year | Results ¶2; abstract | 69% | 42/61 = 68.9% |
| Total 30-day deaths implied, Results pairing | derived | 65 | 0.17 × 247 + 0.36 × 61 = 64.0 (63–65 with rounding); the only exact split is 43 + 22 = 65 |
| Total 30-day deaths implied, abstract pairing | derived | 65 | 0.17 × 247 + 0.47 × 61 = 70.7 (about 69–72); no integer split gives 65 |
| Total 1-year deaths implied, Results pairing | derived | 158 | 0.47 × 247 + 0.69 × 61 = 158.2 (157–159); the only exact split is 116 + 42 = 158 |
| Total 1-year deaths implied, abstract pairing | derived | 158 | 0.36 × 247 + 0.69 × 61 = 131.0 (130–132); no integer split gives 158 |
| Bilateral minus unilateral, 30 days | Abstract vs Results | 47 − 17 = 30 pp | 36.1 − 17.4 = 18.7 pp (19 pp from the rounded Results figures); crude RR 2.07 (2.76 as printed in the abstract) |
| Bilateral minus unilateral, 1 year | Abstract vs Results | 69 − 36 = 33 pp | 68.9 − 47.0 = 21.9 pp (22 pp from the rounded Results figures); crude RR 1.47 (1.92 as printed in the abstract) |
| Adjusted HR, bilateral vs unilateral | Abstract; Results ¶5; Table 3 | 2.58 (1.44–4.63) at 30 days; 2.32 (1.55–3.48) at 1 year | identical in all three places; not affected |
The four numbers in the abstract are the right four numbers, but 47% and 36% have swapped places. The Results figures, the death counts they imply and the reported overall totals all agree with one another. The abstract's pairing agrees with none of them.
One side observation does not form part of the finding. Pearson chi-square tests on the recomputed counts give unadjusted p = 0.0014 (30 days) and 0.0022 (1 year), and Fisher exact tests give 0.0026 at both time points. The Results report p = 0.0003 and 0.0001. The paper does not name the test, and a log-rank test on time to death could legitimately give different values, so I leave this unresolved.
Figure
Figure 1. The upper panel quotes the abstract's sentence and lays out its four percentages as the sentence assigns them: each "x% versus y%" pair is followed by its time-point hazard ratio (HR 2.58 is the 30-day HR and 2.32 the 1-year HR in Table 3), and in each pair bilateral is the higher value. The values as printed are shown in orange. The lower panel quotes the Results and shows, in blue, the death counts recomputed from the paper's own numbers: the only integer split that matches both the Results percentages and the reported totals of 65 and 158 deaths. The unilateral 30-day (17%) and bilateral 1-year (69%) cells agree. The bilateral 30-day and unilateral 1-year cells hold each other's values: the abstract's 47% is the unilateral 1-year mortality (116/247), and its 36% is the bilateral 30-day mortality (22/61).
Figure 2. Total deaths implied by applying each pairing's percentages to the 247 unilateral and 61 bilateral patients, on one axis of deaths that starts at zero. Filled blue circles use the Results pairing and open blue circles the abstract's pairing. The ranges in the labels allow for rounding of the printed percentages. Orange lines mark the reported totals (65/308 at 30 days, 158/308 at 1 year). The Results pairing reproduces both totals. The abstract's pairing implies about 71 deaths at 30 days and 131 at 1 year, and neither range contains the reported total.
Why it matters
- The comparison is the paper's stated novel contribution. The Discussion calls the bilateral analysis "novel" and "the first study to report" this association, and the abstract's last Results sentence is where most readers will take the absolute risks from.
- The absolute risks in the abstract are wrong in a way a reader cannot detect from the abstract alone. A bilateral 30-day mortality of 47% (the true value is 36%) and a unilateral 1-year mortality of 36% (the true value is 47%) both look plausible. Read as printed, the excess mortality with bilateral effusion is 30 pp instead of about 19 pp at 30 days, and 33 pp instead of about 22 pp at 1 year. The implied crude risk ratios are 2.76 instead of 2.07, and 1.92 instead of 1.47. The abstract also understates the 1-year mortality of the larger, unilateral group by 11 pp.
- The error has propagated. A 2022 article in Frontiers in Medicine (doi:10.3389/fmed.2022.828783) cites this study in its Background as reporting bilateral-effusion mortality of "47% at 30 days, 69% at 1 year". Its Discussion later gives the correct "17% for unilateral vs. 36% for bilateral" at 1 month.
- What is not affected. The direction of the association, the adjusted hazard ratios (2.58 and 2.32, consistent across the abstract, Results and Table 3), the reported significance, the etiology-specific mortality figures, the main-text Results, and the qualitative conclusion that bilateral effusion is associated with higher mortality all stand. The error lies in two absolute mortality figures in the abstract.
Explanations considered
- The pairs mean unilateral (30 days vs 1 year) and bilateral (30 days vs 1 year). Under this reading "17% versus 47%" would be the unilateral group over time and "36% versus 69%" the bilateral group, which would match the Results. This does not account for the sentence as printed. The sentence compares bilateral "relative to unilateral" and attaches HR 2.58 (the 30-day HR in Table 3) to the first pair and HR 2.32 (the 1-year HR) to the second. The version of record adds "respectively", which ties the first pair to 30 days and the second to 1 year. A reader therefore takes 17% vs 47% as the 30-day comparison and 36% vs 69% as the 1-year comparison. This reading may explain how the slip arose, but it does not make the printed text correct.
- The abstract quotes adjusted rather than crude mortality. This does not account for the numbers. The Methods describe the subgroup percentages as unadjusted, the Results say they were "calculated from Figure 1", the abstract presents no separate adjusted percentages, and the abstract uses exactly the same four values as the Results.
- A different subgroup, population or time point. This does not account for the numbers. The abstract and the Results use the same 247/61 split and the same two time points, and name no subgroup.
- Rounding. This does not account for the numbers. No number of deaths out of 61 rounds to 47% (28/61 = 45.9%, 29/61 = 47.5%), and neither of the abstract's pairings can reproduce the reported totals of 65 and 158 deaths under any rounding (Figure 2).
- A conversion artifact in the PMC author manuscript. This is ruled out. The publisher's copy-edited abstract in PubMed has the same pairing.
- The error was corrected later. No correction was found (see Status).
Status
- Version of record. The publisher's pages (doi.org redirects to publications.ersnet.org; erj.ersjournals.com/content/46/2/495 and its PDF) returned HTTP 403 to automated access, so I could not read the publisher's full-text Results directly. The final published abstract was checked through the PubMed record (PMID 25837039, Eur Respir J 2015;46(2):495–502, retrieved with NCBI E-utilities on 2026-10-23). It is the publisher's copy-edited text, and it contains the same pairing with "respectively" added. The PMC record (PMC4857137) is the NIH author manuscript. Every quote from it above was confirmed verbatim against the PMC-archived text.
- Supplements. PMC lists no supplementary material, and Europe PMC reports no open-access supplementary files. Supplementary material would not change the finding: the Results text and the overall totals already fix the correct values.
- Corrections and comments. As of 2026-10-23, I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI. PMC and Europe PMC list no correction notices (the Europe PMC commentCorrectionList is empty). The PubMed record (revised 2022-03-17) has no CommentsCorrections entry. Web searches for an erratum or a PubPeer thread found none. The PubPeer API returned HTTP 403 to automated access, so PubPeer could not be queried directly.
- Sources. PMC4857137 (author manuscript, as archived); PubMed 25837039 (E-utilities efetch); Crossref and Europe PMC REST records for doi:10.1183/09031936.00217114; Frontiers in Medicine 2022, doi:10.3389/fmed.2022.828783 (the citing article).
Suggested correction
In the Results sentence of the abstract, change
(17% versus 47%; HR 2.58 CI [1.44–4.63] and 36% versus 69%; HR 2.32 CI [1.55–3.48])
to
(36% versus 17%; HR 2.58 CI [1.44–4.63] and 69% versus 47%; HR 2.32 CI [1.55–3.48])
or, keeping the original order, "(17% versus 36% … and 47% versus 69% …)", stated explicitly as unilateral versus bilateral. The version of record needs the same change, with "respectively" kept. No change is needed to the main text, Table 3 or the conclusions. Because the incorrect figure has already been cited, a published correction notice linked to the article would be more useful than a silent update.
Recomputation script (python recompute.py.txt)
"""Recompute the unilateral vs bilateral mortality figures of PMC4857137
(Eur Respir J 2015;46(2):495-502, doi:10.1183/09031936.00217114).
Run: python recompute.py.txt
Needs scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (author manuscript):
Results p0: 247 unilateral and 61 bilateral thoracenteses; 65 of 308 deaths at
30 days, 158 of 308 at 1 year
Results p1: unilateral 17% (30 d) and 47% (1 y); bilateral 36% (30 d) and 69% (1 y);
unadjusted p = 0.0003 (30 d) and 0.0001 (1 y)
Abstract: "(17% versus 47%; HR 2.58 ...) and (36% versus 69%; HR 2.32 ...)"
Table 3: adjusted HR bilateral 2.58 (1.44, 4.63) at 30 d, 2.32 (1.55, 3.48) at 1 y
"""
import os
import textwrap
from scipy.stats import chi2_contingency, fisher_exact
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
N_U, N_B = 247, 61 # Results p0, Abstract
N_ALL = 308
TOTAL = {"30-day": 65, "1-year": 158} # Results p0
RESULTS_PCT = {"30-day": (17, 36), "1-year": (47, 69)} # (unilateral, bilateral), Results p1
ABSTRACT_PCT = {"30-day": (17, 47), "1-year": (36, 69)} # (unilateral, bilateral) as paired in the abstract
HR = {"30-day": (2.58, 1.44, 4.63), "1-year": (2.32, 1.55, 3.48)} # Abstract, Results p4, Table 3
P_REPORTED = {"30-day": "0.0003", "1-year": "0.0001"} # Results p1, test not named
def rnd(x):
"""Round half up to an integer percentage, as printed."""
return int(x + 0.5)
def counts_for(pct, n):
"""All integer counts k (0..n) whose k/n prints as pct when rounded to a whole percent."""
return [k for k in range(n + 1) if rnd(100 * k / n) == pct]
print("=== 1. Group sizes ===")
print(f"unilateral + bilateral = {N_U} + {N_B} = {N_U + N_B} (reported {N_ALL})")
for t in TOTAL:
print(f"overall {t} deaths: {TOTAL[t]}/{N_ALL} = {100 * TOTAL[t] / N_ALL:.1f}%")
print("\n=== 2. Which death counts are compatible with each printed percentage ===")
for t in ("30-day", "1-year"):
for label, (pu, pb) in (("Results", RESULTS_PCT[t]), ("Abstract", ABSTRACT_PCT[t])):
cu, cb = counts_for(pu, N_U), counts_for(pb, N_B)
print(f"{t} {label:8s}: unilateral {pu}% of {N_U} -> deaths {cu}; "
f"bilateral {pb}% of {N_B} -> deaths {cb if cb else 'NONE (no k/61 rounds to %d%%)' % pb}")
print("k/61 near 47%:", [(k, round(100 * k / N_B, 1)) for k in range(26, 31)])
print("\n=== 3. Implied total deaths under each pairing vs the reported total ===")
RANGES = {}
for t in ("30-day", "1-year"):
for label, (pu, pb) in (("Results", RESULTS_PCT[t]), ("Abstract", ABSTRACT_PCT[t])):
point = pu / 100 * N_U + pb / 100 * N_B
cu, cb = counts_for(pu, N_U), counts_for(pb, N_B)
if cb:
lo, hi = min(cu) + min(cb), max(cu) + max(cb)
else: # bilateral percentage not attainable: use the real-valued rounding interval
lo = (pu - 0.5) / 100 * N_U + (pb - 0.5) / 100 * N_B
hi = (pu + 0.5) / 100 * N_U + (pb + 0.5) / 100 * N_B
RANGES[(t, label)] = (point, lo, hi)
inside = lo <= TOTAL[t] <= hi
print(f"{t} {label:8s}: {pu/100:.2f}*{N_U} + {pb/100:.2f}*{N_B} = {point:.1f}; "
f"totals compatible with rounding {lo:.0f}-{hi:.0f}; reported {TOTAL[t]} -> "
f"{'consistent' if inside else 'INCONSISTENT'}")
print("\n=== 4. Integer splits that reproduce both the percentages and the reported totals ===")
SPLITS = {}
for t in ("30-day", "1-year"):
for label, (pu, pb) in (("Results", RESULTS_PCT[t]), ("Abstract", ABSTRACT_PCT[t])):
sols = [(u, TOTAL[t] - u) for u in counts_for(pu, N_U)
if (TOTAL[t] - u) in counts_for(pb, N_B)]
print(f"{t} {label:8s} pairing {pu}% vs {pb}%: solutions (unilateral, bilateral) = {sols}")
if label == "Results":
SPLITS[t] = sols[0]
for t, (u, b) in SPLITS.items():
print(f"{t}: unilateral {u}/{N_U} = {100*u/N_U:.1f}%, bilateral {b}/{N_B} = {100*b/N_B:.1f}%, "
f"sum {u + b} (reported {TOTAL[t]})")
print("\n=== 5. Size of the comparison: Results vs the abstract's pairing ===")
for t in ("30-day", "1-year"):
u, b = SPLITS[t]
ru, rb = 100 * u / N_U, 100 * b / N_B
au, ab = ABSTRACT_PCT[t]
print(f"{t}: from counts RD = {rb:.1f} - {ru:.1f} = {rb - ru:.1f} pp, crude RR = {rb / ru:.2f}")
print(f"{t}: Results text RD = {RESULTS_PCT[t][1]} - {RESULTS_PCT[t][0]} = "
f"{RESULTS_PCT[t][1] - RESULTS_PCT[t][0]} pp, RR = {RESULTS_PCT[t][1] / RESULTS_PCT[t][0]:.2f}")
print(f"{t}: abstract RD = {ab} - {au} = {ab - au} pp, RR = {ab / au:.2f}")
print(f"{t}: abstract overstates the gap by {(ab - au) - (RESULTS_PCT[t][1] - RESULTS_PCT[t][0])} pp")
print("\n=== 6. Hazard ratios (not affected) ===")
for t, (h, lo, hi) in HR.items():
print(f"{t}: HR {h} ({lo}-{hi}) - identical in Abstract, Results p4 and Table 3")
print("\n=== 7. Side note: unadjusted tests on the recomputed counts (test in paper not named) ===")
for t in ("30-day", "1-year"):
u, b = SPLITS[t]
tab = [[b, N_B - b], [u, N_U - u]]
_, p, _, _ = chi2_contingency(tab, correction=False)
_, py, _, _ = chi2_contingency(tab, correction=True)
pf = fisher_exact(tab)[1]
print(f"{t}: reported p = {P_REPORTED[t]}; Pearson chi-square p = {p:.4f}; "
f"Yates p = {py:.4f}; Fisher exact p = {pf:.4f}")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5,
"svg.fonttype": "none", "figure.facecolor": BG, "axes.facecolor": BG})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ---- fig1: abstract vs Results, two text panels; the swapped cells are joined by crossing connectors
FH = 5.5
fig = plt.figure(figsize=(7.2, FH))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, FH)
ax.axis("off")
COLS = [2.2, 3.45, 4.7, 5.95] # unilateral 30 d, bilateral 30 d, unilateral 1 y, bilateral 1 y
HEAD = ["Unilateral\n30 days", "Bilateral\n30 days", "Unilateral\n1 year", "Bilateral\n1 year"]
X0, W = 0.12, 6.96
def panel(y_bottom, h):
ax.add_patch(FancyBboxPatch((X0, y_bottom), W, h, boxstyle="round,pad=0.0,rounding_size=0.06",
fc="white", ec="#cfcdc7", lw=1))
# Panel A: abstract
panel(3.2, 2.2)
ax.text(X0 + 0.15, 5.18, "Abstract, Results sentence", fontsize=11.5, weight="bold", color=INK, va="center")
qa = ("\u201cPatients with bilateral pleural effusion, relative to unilateral, were associated with higher "
"risk of death at 30 days and 1 year (17% versus 47%; HR 2.58 CI [1.44\u20134.63] and 36% versus 69%; "
"HR 2.32 CI [1.55\u20133.48])\u201d")
ax.text(X0 + 0.15, 4.98, "\n".join(textwrap.wrap(qa, 86)), fontsize=10, color=INK2, style="italic",
va="top", linespacing=1.3)
for (c0, c1), lab in (((COLS[0], COLS[1]), "30-day pair, HR 2.58"), ((COLS[2], COLS[3]), "1-year pair, HR 2.32")):
ax.plot([c0 - 0.35, c1 + 0.35], [4.12, 4.12], color=INK2, lw=1)
ax.text((c0 + c1) / 2, 4.2, lab, ha="center", va="bottom", fontsize=10, color=INK2)
for x, hd in zip(COLS, HEAD):
ax.text(x, 3.88, hd, ha="center", va="center", fontsize=10, color=INK2, linespacing=1.1)
ax.text(X0 + 0.15, 3.45, "As printed", fontsize=10, color=ORANGE, weight="bold",
va="center", linespacing=1.1)
A = ["17%", "47%", "36%", "69%"]
for j, (x, v) in enumerate(zip(COLS, A)):
sw = j in (1, 2)
ax.text(x, 3.45, v, ha="center", va="center", fontsize=12, color=ORANGE if sw else INK,
weight="bold" if sw else "normal",
bbox=dict(boxstyle="round,pad=0.3", fc="white", ec=ORANGE if sw else "none", lw=1.6))
# Panel B: Results
panel(0.12, 1.98)
u30, b30 = SPLITS["30-day"]
u1, b1 = SPLITS["1-year"]
B = [(u30, N_U), (b30, N_B), (u1, N_U), (b1, N_B)]
ax.text(X0 + 0.15, 1.83, "Results,\nparagraph 2", fontsize=11.5, weight="bold", color=INK, va="center",
linespacing=1.1)
ax.text(X0 + 0.15, 1.33, "Recomputed\nfrom counts", fontsize=10, color=BLUE, weight="bold", va="center",
linespacing=1.1)
for j, (x, (k, n)) in enumerate(zip(COLS, B)):
sw = j in (1, 2)
ax.text(x, 1.62, f"{k}/{n}\n= {100 * k / n:.1f}%", ha="center", va="center", fontsize=11,
color=BLUE if sw else INK, weight="bold" if sw else "normal", linespacing=1.15,
bbox=dict(boxstyle="round,pad=0.3", fc="white", ec=BLUE if sw else "none", lw=1.6))
qb = ("\u201cthirty-day and one-year mortality of patients with unilateral effusion was 17% and 47%, "
"compared with 36% and 69% in patients with bilateral effusion\u201d. Counts: the only integer "
"split matching these percentages and the reported 65 and 158 total deaths.")
ax.text(X0 + 0.15, 0.98, "\n".join(textwrap.wrap(qb, 86)), fontsize=10, color=INK2, style="italic",
va="top", linespacing=1.3)
# connectors between the panels
ytop, ybot = 3.18, 2.02
for ja, jb in ((1, 2), (2, 1)): # abstract cell ja holds the value of Results cell jb
ax.add_patch(FancyArrowPatch((COLS[ja], ytop), (COLS[jb], ybot), arrowstyle="-|>",
mutation_scale=14, lw=2, color=INK, shrinkA=2, shrinkB=2))
ym = (ytop + ybot) / 2
XL = COLS[1] - 0.35
ax.text(XL, ym + 0.27, "47% and 36% transposed", fontsize=11, weight="bold", color=INK, va="center", ha="right")
ax.text(XL, ym - 0.02, "each value sits in the other's cell", fontsize=10, color=INK2, va="center", ha="right")
ax.text(XL, ym - 0.28, "17% and 69% agree", fontsize=10, color=INK2, va="center", ha="right")
save(fig, "fig1")
# ---- fig2: implied total deaths under each pairing vs the reported totals
fig, ax = plt.subplots(figsize=(7.2, 3.3))
fig.subplots_adjust(left=0.25, right=0.97, top=0.84, bottom=0.17)
rows = [("30-day", "Results"), ("30-day", "Abstract"), ("1-year", "Results"), ("1-year", "Abstract")]
ylab = {("30-day", "Results"): "30 days: Results\n(17% vs 36%)",
("30-day", "Abstract"): "30 days: Abstract\n(17% vs 47%)",
("1-year", "Results"): "1 year: Results\n(47% vs 69%)",
("1-year", "Abstract"): "1 year: Abstract\n(36% vs 69%)"}
ypos = {rows[0]: 4.0, rows[1]: 3.0, rows[2]: 1.4, rows[3]: 0.4}
for r in rows:
point, lo, hi = RANGES[r]
y = ypos[r]
filled = r[1] == "Results"
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(point, y, "o", ms=9, mfc=BLUE if filled else BG, mec=BLUE, mew=2)
lab = f"{point:.0f} (range {lo:.0f}\u2013{hi:.0f})"
if hi < TOTAL[r[0]] - 5: # marker left of the reported line: label on the left
ax.text(lo - 4, y, lab, va="center", ha="right", fontsize=10, color=BLUE)
else:
ax.text(hi + 4, y, lab, va="center", fontsize=10, color=BLUE)
for t, (ya, yb) in (("30-day", (3.0, 4.0)), ("1-year", (0.4, 1.4))):
ax.plot([TOTAL[t]] * 2, [ya - 0.35, yb + 0.35], color=ORANGE, lw=2)
ax.text(TOTAL[t], yb + 0.5, f"reported {TOTAL[t]}/308", ha="center", fontsize=10, color=ORANGE)
ax.set_yticks([ypos[r] for r in rows], [ylab[r] for r in rows], fontsize=10)
ax.set_ylim(-0.3, 4.9)
ax.set_xlim(0, 230)
ax.set_xticks(range(0, 226, 25))
ax.set_xlabel("Total deaths implied by the group percentages (247 unilateral, 61 bilateral)",
fontsize=10, color=INK)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0, colors=INK)
ax.tick_params(axis="x", colors=INK2)
ax.plot([], [], "o", ms=9, mfc=BLUE, mec=BLUE, mew=2, label="Recomputed, Results pairing")
ax.plot([], [], "o", ms=9, mfc=BG, mec=BLUE, mew=2, label="Recomputed, abstract pairing")
ax.plot([], [], color=ORANGE, lw=2, label="Reported total deaths")
fig.legend(loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10,
handlelength=1.6, columnspacing=1.0)
save(fig, "fig2")
print("\nfigures written to", HERE)
Dorsal attention network mediation reports a negative indirect effect with an entirely positive confidence interval
Hum Brain Mapp (2023) · doi:10.1002/hbm.26474 · PMC10619388
Summary
The paper reports that the temporal variability of the dorsal attention network (DAN) mediates the link between anhedonia and attention in adolescents with major depressive disorder, with "indirect effect = -0.438, 95% confidence interval: [0.926, 1.079]". The estimate is negative but the interval is entirely positive, and the estimate lies 1.364 below the interval's lower bound, which is 17.8 interval half-widths. The paper's own Figure 7c(C) labels −0.438 as the direct path c′, not as the indirect effect. Its path coefficients give an indirect effect of a × b = −0.509 × 0.263 = −0.134, which also lies outside the printed interval. This result is the only inferential support for the fMRI half of the title claim and for an "Importantly" sentence in the Abstract, so as printed that claim has no valid confidence interval. The paper's other mediation result (EEG Microstate C-to-D transition) is internally consistent in the Results text.
What the paper reports
Results, "Network temporal properties associated with MDD", paragraph 4 (the DAN mediation):
Temporal variability in DAN mediated the relationship between anhedonia and attention (direct effect = -0.271; indirect effect = -0.438, 95% confidence interval: [0.926, 1.079]; Figure 7d).
Results, "Network temporal properties associated with MDD", paragraph 3 (the inputs to that mediation):
Correlation analyses revealed that temporal variability was significantly negatively correlated with anhedonia (r = -0.266, p = 0.049; Figure 7c(A)) and positively correlated with attention scores in adolescents with MDD (r = 0.419, p = 0.019; Figure 7c(B)).
Results, "Characteristics of EEG microstate in adolescents with MDD", paragraph 3 (the second mediation, given for comparison):
The transition rate from Microstates C to D mediated the relationship between anhedonia and attention (direct effect = -0.497; indirect effect = -0.389, 95% confidence interval: [-0.571, -0.032], Figure 3c(H)).
Figure 7, caption:
Correlation analyses indicated that the temporal variability was significantly positively correlated with anhedonia and negatively related with attention scores in MDD patients (c(A,B)). Mediation analysis revealed that the temporal variability of DAN mediated the relationship from anhedonia to attention (c(C)).
Figure 7, panel c(C). The panel is an image; these labels were read from the publisher's graphic (Europe PMC file HBM-44-5749-g009.jpg). The paths are a = −0.509 (anhedonia to DAN variability) and b = 0.263 (DAN variability to attention), and the anhedonia-to-attention arrows carry:
−0.438(c′)
−0.572(c)
Bootstrap estimate [95% CI] = −0.271* [0.926, 1.079]
Figure 3, panel c(H) (file HBM-44-5749-g001.jpg). Here a = 0.416 and b = −0.260, and:
-0.389 (c′)
-0.497 (c)
Bootstrap Estimate [95% CI] = 2.241* [-0.571,-0.032]
The finding is stated as a headline result in the Abstract:
Importantly, the temporal variability of DAN mediated the relationship between anhedonia and attention deficits in the patient group.
and in the Discussion, opening paragraph:
Importantly, the dynamic features of the attentional network (transition probability of C-D and temporal variability of the DAN) mediated the relationship between anhedonia and attentional deficits in the patient group.
Discussion, "Adolescent with MDD and network dynamic attributes":
The temporal variability of DAN mediates the relationship between anhedonia and attention.
The title reads "Attention network mediates the association between anhedonia and attentional deficit". The Methods ("Statistical analyses") say that anhedonia was the independent variable, attention the dependent variable and the C-to-D conversion rate the mediator. They do not describe how the mediation confidence intervals were obtained, and they do not mention the DAN mediation model.
What the paper's own numbers show
Method. For each mediation I checked whether the printed estimate lies inside its printed 95% CI. I computed the implied total effect (c = c′ + ab) and proportion mediated from the text, and the indirect effect a × b and c − c′ from the figure's path labels. I checked the sign that a × b must take given the reported correlations and paths. I tested whether any other DAN quantity the paper prints could be the one the interval describes, and whether the coefficients could be standardized. Finally I worked out the sample sizes that the reported r and p values imply. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| DAN indirect effect vs its CI | Results 3.6 p4 | −0.438, CI [0.926, 1.079] | Outside the CI. 1.364 below the lower bound (17.8 half-widths of 0.0765); CI midpoint 1.0025 |
| Sign-flipped DAN CI | Results 3.6 p4 | [0.926, 1.079] | [−1.079, −0.926] does not contain −0.438 either |
| DAN implied total effect, proportion mediated | Results 3.6 p4 | direct −0.271, indirect −0.438 | c = −0.709, proportion 0.618. Neither lies in the CI |
| DAN indirect effect from figure paths | Fig 7c(C) | a = −0.509, b = 0.263 | a × b = −0.134; c − c′ = −0.572 − (−0.438) = −0.134 (the paths agree with each other) |
| What −0.438 is | Results 3.6 p4 vs Fig 7c(C) | "indirect effect" in text | Labelled c′ (direct path) in the figure |
| What −0.271 is | Results 3.6 p4 vs Fig 7c(C) | "direct effect" in text | Labelled "Bootstrap estimate" in the figure; outside [0.926, 1.079] |
| Any printed DAN quantity inside the CI? | Text and Fig 7c(C) | c, a, b, c′, −0.271, a × b, c′/c (0.766), c/c′ (1.306) | None |
| Sign of the DAN indirect effect | Results p3; Fig 7 caption; Fig 7c(C) | r = −0.266 and +0.419; caption gives the opposite pair; a < 0, b > 0 | a × b < 0 under every source. The negative estimate is the plausible value and the positive CI is not |
| Standardized-coefficient check | derived | indirect −0.438 | If standardized, a × b would be −0.066; Fig 7 a = −0.509 ≠ r = −0.266. The coefficients are unstandardized, so the CI cannot be rebuilt from the text |
| C-D indirect effect vs its CI | Results 3.2 p3 | −0.389, CI [−0.571, −0.032] | Inside; CI negative and excludes 0. Total −0.886, proportion 0.439 |
| C-D indirect effect from figure paths | Fig 3c(H) | a = 0.416, b = −0.260 | a × b = −0.108 = c − c′; inside the CI. The figure's "Bootstrap Estimate" 2.241 lies outside its own CI |
| Sample size behind the DAN correlations | Results 3.6 p3 | r = −0.266, p = 0.049; r = 0.419, p = 0.019 | Consistent only with n = 55 and n = 31 respectively (with n = 188, p would be 2.2 × 10⁻⁴ and 2.2 × 10⁻⁹) |
Figure
Figure 1. The two mediation results on one axis of unstandardized indirect effect, with the null value 0 as a thin vertical line. Orange circles and bars are the indirect effects and 95% CIs printed in the Results text. Blue diamonds are a × b recomputed from the path coefficients in the paper's Figures 3c(H) and 7c(C); the paper reports no interval for these. For the EEG C-to-D mediation, the printed estimate (−0.389) and the recomputed a × b (−0.108) both lie inside the printed interval. For the DAN mediation, the printed estimate (−0.438) lies 1.364 below an interval that is entirely positive, and the recomputed a × b (−0.134) lies outside it too.
Figure 2. The DAN mediation numbers as printed in the Results text (left) and in the path-diagram labels of the paper's Figure 7c(C) (right), transcribed rather than reproduced. Connector 1: the text calls −0.438 the indirect effect, while the figure labels it c′, the direct path. Connector 2: the text calls −0.271 the direct effect, while the figure labels it the bootstrap estimate. The bottom panel recomputes the indirect effect from the figure's own paths (−0.134). None of the three candidate values lies inside the shared interval [0.926, 1.079].
Why it matters
The title's claim that the "attention network mediates the association between anhedonia and attentional deficit" rests on two analyses: an EEG microstate transition (C to D) and fMRI DAN temporal variability. The DAN result is the only statistical support for the fMRI half of that claim. It appears in the Abstract as an "Importantly" finding and is restated three times in the Discussion. As printed, its 95% CI contradicts its own estimate in sign and location. The paper's figure and text also disagree about which printed number is the indirect effect. A reader cannot tell what the indirect effect is (−0.438, −0.271 or −0.134), what its interval is, or whether that interval excludes zero. The interval is narrow and centred near 1. If it came from a ratio-type quantity, for which 1 is the null, it would include the null. If it belongs to the negative a × b, its true limits are not reported.
The inputs to the mediation are unclear too. The Figure 7 caption reverses the signs of both correlations given in the Results text. One Discussion sentence calls the DAN–attention correlation negative ("the temporal variability of the DAN was significantly negatively correlated with attention scores"), while the Results report r = 0.419. The two r/p pairs fit only n = 55 and n = 31, not the 188 patients enrolled, and the paper does not say which subsample each analysis used. A separate inconsistency: Results paragraph 3 says DAN variability was "significantly higher in adolescents with MDD than in HC (t = 2.582, p = .010)", while the Abstract, the Figure 7 caption and the Discussion say it was weaker.
The EEG C-to-D mediation is unaffected in the text: its estimate lies inside its interval, both negative. It still supports the broader statement that an attention-related dynamic measure mediates the anhedonia–attention link. Its figure, however, prints a "Bootstrap Estimate" of 2.241 outside its own interval, and the figure labels −0.389 as c′ rather than as the indirect effect.
Explanations considered
- The interval belongs to another DAN quantity. No. The direct effect (−0.271), the implied total effect (−0.709), the proportion mediated (0.618), the figure's a (−0.509), b (0.263), c (−0.572), c′ (−0.438) and a × b (−0.134) all lie outside [0.926, 1.079]. So do the ratios c′/c (0.766) and c/c′ (1.306).
- A lost minus sign. No. The negated interval [−1.079, −0.926] does not contain −0.438, −0.271 or −0.134. Also, the publisher's XML carries Unicode minus signs elsewhere in the same sentence but none in this interval, and the figure graphic prints the same positive interval. So this is not a text-conversion artefact.
- Standardized versus unstandardized coefficients. A change of scale cannot move a negative estimate into a positive interval. Standardized coefficients would give a × b ≈ −0.066 in any case, so the printed values are unstandardized.
- Bootstrap asymmetry. Percentile and bias-corrected bootstrap intervals can be asymmetric about the estimate. They still contain the estimate they are built around, and they cannot have the opposite sign to it.
- The interval is for a ratio or exponentiated effect. The interval's midpoint near 1 fits that reading. But the paper reports no such quantity, and exp(−0.134) = 0.875 and exp(−0.438) = 0.645 are both outside the interval. Under that reading the interval would include the null value 1, and the mediation would not be significant.
- A different sign convention in the figure caption. The caption's sign pair (positive with anhedonia, negative with attention) also gives a × b < 0. So no sign convention in the paper makes a positive indirect effect plausible.
- The interval is correct and the estimate is wrong. This cannot be ruled out from the paper alone. Every source in the paper, however, implies a negative indirect effect, so the positive interval is the less plausible number.
Status
- Version of record. The Wiley article page and PDF returned HTTP 403 to automated access. I checked the publisher-deposited full text as archived in PMC and the Europe PMC full-text XML (Wiley JATS). Both carry the quoted values. I read the mediation panels from the publisher's figure graphics in the Europe PMC package (HBM-44-5749-g001.jpg and -g009.jpg).
- Supplements. Neither PMC nor the Europe PMC XML lists supplementary material. The Europe PMC supplementary-files package contains only the article's figure graphics, with no data or supplementary tables.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment links (PMID 37683097). The PubPeer API and search page refused automated access (HTTP 403), so PubPeer could not be checked. A general web search found no correction or discussion.
- Sources. PMC article PMC10619388; Europe PMC full-text XML and supplementary-files package for PMC10619388; Crossref record for 10.1002/hbm.26474; Europe PMC record PMID 37683097.
Suggested correction
- Re-run or re-extract the DAN mediation and report the indirect effect (a × b), its 95% CI, the direct effect c′ and the total effect c. Give the method used for the interval (for example the bootstrap type and number of resamples), the software, and whether the coefficients are unstandardized. Update the Results text, the Figure 7c(C) panel, and the Abstract and Discussion sentences that rest on this result.
- Make the labels in the Results text consistent with the path diagrams of Figures 3c(H) and 7c(C). The text currently calls the figures' c′ the "indirect effect", and calls the "bootstrap estimate" (DAN) or the total effect c (C-D) the "direct effect". Also correct the C-D panel's bootstrap estimate (2.241), which lies outside its own interval.
- Correct the Figure 7 caption's correlation signs, and the Discussion sentence calling the DAN–attention correlation negative, to match the Results (r = −0.266 with anhedonia, r = 0.419 with attention), or correct the Results if the caption is right.
- State the number of patients in each correlation and mediation analysis (the r/p pairs imply about 55 and 31). Reconcile Results paragraph 3 ("significantly higher in adolescents with MDD") with the Abstract and Discussion ("significantly weaker").
- If a valid interval for the DAN indirect effect includes zero, revise the title, the Abstract and the Discussion so that the mediation claim rests on the EEG result alone.
Recomputation script (python recompute.py.txt)
"""Recompute the mediation results of PMC10619388
(Hum Brain Mapp 2023, doi:10.1002/hbm.26474).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png, fig2.svg/png and five candidate figure designs
(candidates/cand_A ... cand_E) next to this file.
Inputs are copied from the paper as archived in PMC (Wiley JATS) and from the
publisher's figure graphics in the Europe PMC package:
Results 3.2, p3: C-D mediation, "direct effect = -0.497; indirect effect = -0.389,
95% confidence interval: [-0.571, -0.032]"
Results 3.6, p3: DAN correlations r = -0.266 (p = 0.049), r = 0.419 (p = 0.019)
Results 3.6, p4: DAN mediation, "direct effect = -0.271; indirect effect = -0.438,
95% confidence interval: [0.926, 1.079]"
Figure 3 c(H) path labels: a = 0.416, b = -0.260, -0.389 (c'), -0.497 (c),
"Bootstrap Estimate [95% CI] = 2.241* [-0.571,-0.032]"
Figure 7 c(C) path labels: a = -0.509, b = 0.263, -0.438 (c'), -0.572 (c),
"Bootstrap estimate [95% CI] = -0.271* [0.926, 1.079]"
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (from the paper)
TEXT = { # Results text
"C-D": dict(direct=-0.497, indirect=-0.389, lo=-0.571, hi=-0.032),
"DAN": dict(direct=-0.271, indirect=-0.438, lo=0.926, hi=1.079),
}
FIG = { # path-diagram labels in the publisher's figure graphics
"C-D": dict(a=0.416, b=-0.260, cprime=-0.389, c=-0.497, boot=2.241, lo=-0.571, hi=-0.032),
"DAN": dict(a=-0.509, b=0.263, cprime=-0.438, c=-0.572, boot=-0.271, lo=0.926, hi=1.079),
}
DAN_R = [(-0.266, 0.049, "anhedonia"), (0.419, 0.019, "attention")] # Results 3.6 p3
# ------------------------------------------------------------------ style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e3e2de"
LW, MS = 1.5, 8 # 1.5 pt = 2 px at 96 dpi; markers 8 pt
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
W = 7.2
MINUS = "−"
def f(x, d=3):
s = f"{x:.{d}f}"
return s.replace("-", MINUS)
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight")
fig.savefig(p + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
# ------------------------------------------------------------------ 1. containment checks
print("1. Estimate-versus-interval checks (Results text)")
for k, t in TEXT.items():
mid, hw = (t["lo"] + t["hi"]) / 2, (t["hi"] - t["lo"]) / 2
inside = t["lo"] <= t["indirect"] <= t["hi"]
total = t["direct"] + t["indirect"]
print(f" {k}: indirect {t['indirect']}, CI [{t['lo']}, {t['hi']}]; midpoint {mid:.4f}, "
f"half-width {hw:.4f}; inside={inside}; CI excludes 0={not (t['lo'] <= 0 <= t['hi'])}")
print(f" implied total c' + ab = {total:.3f}; proportion mediated = {t['indirect']/total:.3f}")
if not inside:
d = t["lo"] - t["indirect"]
print(f" distance to nearest bound = {d:.3f} = {d/hw:.1f} half-widths; "
f"to far bound = {t['hi'] - t['indirect']:.3f}")
print(f" sign-flipped CI [{-t['hi']}, {-t['lo']}] contains estimate: "
f"{-t['hi'] <= t['indirect'] <= -t['lo']}")
# ------------------------------------------------------------------ 2. figure path arithmetic
print("\n2. Path arithmetic from the figure labels")
AB = {}
for k, g in FIG.items():
ab, diff = g["a"] * g["b"], g["c"] - g["cprime"]
AB[k] = ab
print(f" {k}: a*b = {g['a']} x {g['b']} = {ab:.4f}; c - c' = {g['c']} - ({g['cprime']}) = {diff:.4f}; "
f"proportion mediated ab/c = {ab/g['c']:.3f}")
print(f" a*b inside printed CI [{g['lo']}, {g['hi']}]: {g['lo'] <= ab <= g['hi']}; "
f"bootstrap estimate {g['boot']} inside: {g['lo'] <= g['boot'] <= g['hi']}")
dan_vals = {"c (total, Fig 7)": FIG["DAN"]["c"], "a (Fig 7)": FIG["DAN"]["a"],
"'indirect' text = c' Fig 7": -0.438, "'direct' text = bootstrap est. Fig 7": -0.271,
"a*b": AB["DAN"], "b (Fig 7)": FIG["DAN"]["b"],
"c'/c": FIG["DAN"]["cprime"] / FIG["DAN"]["c"], "c/c'": FIG["DAN"]["c"] / FIG["DAN"]["cprime"],
"text total -0.709": -0.709, "proportion 0.618": 0.618}
print(" DAN candidate quantities and whether each lies in [0.926, 1.079]:")
for name, v in dan_vals.items():
print(f" {name:40s} {v:8.3f} {0.926 <= v <= 1.079}")
# ------------------------------------------------------------------ 3. sign logic
print(f" exp(a*b) = {np.exp(AB['DAN']):.3f}; exp(-0.438) = {np.exp(-0.438):.3f}; "
f"inside [0.926, 1.079]: {0.926 <= np.exp(AB['DAN']) <= 1.079}, {0.926 <= np.exp(-0.438) <= 1.079}")
print("\n3. Sign logic")
print(" DAN, Results text: r(anhedonia, var) = -0.266, r(var, attention) = +0.419 -> a*b < 0")
print(" DAN, Fig 7 caption: var positively with anhedonia, negatively with attention -> a*b < 0")
print(" DAN, Fig 7 paths: a = -0.509, b = +0.263 -> a*b < 0")
print(" C-D: a = +0.416, b = -0.260 -> a*b < 0; printed CI is negative (consistent)")
# ------------------------------------------------------------------ 4. standardized-coefficient check
rmx, rmy = -0.266, 0.419
c_tot = TEXT["DAN"]["direct"] + TEXT["DAN"]["indirect"]
b_std = (rmy - rmx * c_tot) / (1 - rmx ** 2)
print(f"\n4. If text coefficients were standardized: r_xy = {c_tot:.3f}, b = {b_std:.3f}, "
f"a*b = {rmx*b_std:.3f} (printed indirect -0.438) -> not standardized")
print(f" Fig 7 a = {FIG['DAN']['a']} differs from r = {rmx}: unstandardized path, so no CI can be rebuilt")
# ------------------------------------------------------------------ 5. implied n
def pval(r, n):
t = r * np.sqrt((n - 2) / (1 - r * r))
return 2 * stats.t.sf(abs(t), n - 2)
print("\n5. Sample sizes consistent with the reported r and p (r rounded to 3 dp, p to 3 dp)")
for r0, p0, lab in DAN_R:
ns = set()
for r in np.linspace(abs(r0) - 0.0005, abs(r0) + 0.0005, 21):
ns |= {n for n in range(10, 300) if round(pval(r, n), 3) == p0}
print(f" DAN variability vs {lab}: r = {r0}, p = {p0} -> n in {sorted(ns)} "
f"(p at n=188 would be {pval(r0, 188):.3e})")
# ================================================================== figures
NOTE = "Values from the paper's Results text (orange) and recomputed from its own figure path labels (blue)."
def cand_A(paths):
"""Dot-and-interval plot, one axis, both mediations."""
fig, ax = plt.subplots(figsize=(W, 3.3))
rows = [("C–D transition (EEG)", "C-D", 1.0), ("DAN temporal variability (fMRI)", "DAN", 0.0)]
for lab, k, y in rows:
t = TEXT[k]
ax.plot([t["lo"], t["hi"]], [y + 0.14] * 2, color=ORANGE, lw=LW * 2, solid_capstyle="butt")
for xb in (t["lo"], t["hi"]):
ax.plot([xb, xb], [y + 0.07, y + 0.21], color=ORANGE, lw=LW)
ax.plot(t["indirect"], y + 0.14, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=4)
ax.plot(AB[k], y - 0.14, "D", color=BLUE, ms=MS - 1, mec=BG, mew=1.5, zorder=4)
ax.text(t["indirect"], y + 0.30, f"{f(t['indirect'])}", ha="center", va="bottom", fontsize=10, color=INK)
ax.text(AB[k], y - 0.30, f"a×b = {f(AB[k])}", ha="center", va="top", fontsize=10, color=INK)
ax.text(-1.55, y, lab, ha="left", va="center", fontsize=10.5, color=INK, fontweight="bold")
t = TEXT["DAN"]
ax.text((t["lo"] + t["hi"]) / 2, 0.30, f"[{f(t['lo'])}, {f(t['hi'])}]", ha="center", va="bottom", fontsize=10)
ax.annotate("", xy=(t["lo"] - 0.02, 0.14), xytext=(t["indirect"] + 0.03, 0.14),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1, shrinkA=0, shrinkB=0))
ax.text((t["lo"] + t["indirect"]) / 2 + 0.15, 0.04, "estimate lies 1.364 below\nthe interval's lower bound",
ha="center", va="top", fontsize=10, color=INK2)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.text(0.01, 1.62, "null (0)", fontsize=10, color=INK2, ha="left", va="top")
ax.set_xlim(-1.6, 1.3)
ax.set_ylim(-0.55, 1.65)
ax.set_yticks([])
ax.set_xticks(np.arange(-1.0, 1.25, 0.25))
ax.set_xticklabels([f(v, 2) for v in np.arange(-1.0, 1.25, 0.25)])
ax.set_xlabel("Indirect effect (anhedonia → mediator → attention), unstandardized")
ax.grid(axis="x", color=GRID, lw=0.6)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
h1, = ax.plot([], [], "o-", color=ORANGE, ms=MS, lw=LW * 2, label="Reported in Results text: indirect effect and 95% CI")
h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed: a × b from the paper's figure path labels (no CI)")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.24), ncol=1, frameon=False, fontsize=10)
save(fig, *paths)
def cand_B(paths):
"""DAN-only number line: every printed DAN quantity versus the printed CI."""
fig, ax = plt.subplots(figsize=(W, 3.4))
t = TEXT["DAN"]
ax.axvspan(t["lo"], t["hi"], color=ORANGE, alpha=0.18, lw=0)
ax.plot([t["lo"], t["hi"]], [0, 0], color=ORANGE, lw=LW * 2)
ax.text((t["lo"] + t["hi"]) / 2, 1.05, "printed 95% CI\n[0.926, 1.079]", ha="center", va="bottom", fontsize=10)
pts = [ # (value, label, colour, marker, label x, label y)
(FIG["DAN"]["c"], "c (total), Fig 7", ORANGE, "o", -0.70, 0.55),
(FIG["DAN"]["a"], "a path, Fig 7", ORANGE, "o", -0.62, -0.55),
(-0.438, "'indirect' (text)\n= c\u2032 (figure)", ORANGE, "o", -0.40, 1.05),
(-0.271, "'direct' (text) =\n'bootstrap estimate' (figure)", ORANGE, "o", -0.20, -1.0),
(AB["DAN"], "a\u00d7b = c \u2212 c\u2032\n(recomputed)", BLUE, "D", 0.10, 0.55),
(FIG["DAN"]["b"], "b path, Fig 7", ORANGE, "o", 0.36, -0.55),
]
for v, lab, col, mk, tx, ty in pts:
ax.plot([v, tx], [0, ty * 0.85], color=INK2, lw=0.7, zorder=1)
ax.plot(v, 0, mk, color=col, ms=MS, mec=BG, mew=1.5, zorder=4)
ax.text(tx, ty, f"{f(v)} {lab}", ha="center", va="bottom" if ty > 0 else "top", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.axhline(0, color=GRID, lw=0.8, zorder=0)
ax.set_xlim(-0.95, 1.25)
ax.set_ylim(-1.75, 1.85)
ax.set_yticks([])
ax.set_xlabel("DAN mediation: every coefficient the paper prints (unstandardized)")
ax.grid(axis="x", color=GRID, lw=0.6)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Reported (text or figure labels)")
h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed from reported paths")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.3), ncol=2, frameon=False)
save(fig, *paths)
def box(ax, x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.02",
fc="white", ec=edge, lw=LW, clip_on=False))
ax.text(x + 0.02, y + h - 0.035, title, fontsize=10.5, fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.11, body, fontsize=10, va="top", color=INK, linespacing=1.5)
def cand_C(paths):
"""Two text panels (Results text vs Figure 7c(C) labels) joined by annotated connectors."""
fig, ax = plt.subplots(figsize=(W, 4.2))
ax.set_xlim(-0.02, 1.02)
ax.set_ylim(0, 1)
ax.axis("off")
box(ax, 0.0, 0.52, 0.47, 0.46, "Results, section 3.6 (text)",
"direct effect = −0.271\nindirect effect = −0.438\n95% CI: [0.926, 1.079]", ORANGE)
box(ax, 0.53, 0.52, 0.47, 0.46, "Figure 7c(C) (path-diagram labels)",
"−0.438 (c′) −0.572 (c)\na = −0.509 b = 0.263\nBootstrap estimate [95% CI]\n= −0.271* [0.926, 1.079]",
ORANGE)
for (y0, y1), num in [((0.80, 0.865), "1"), ((0.86, 0.66), "2")]:
ax.add_patch(FancyArrowPatch((0.40, y0), (0.55, y1), arrowstyle="<->", mutation_scale=12,
color=INK2, lw=1.2))
ax.text(0.475, (y0 + y1) / 2 + (0.035 if num == "1" else -0.06), num, ha="center", va="center",
fontsize=10, fontweight="bold", color=INK,
bbox=dict(boxstyle="circle,pad=0.2", fc=BG, ec=INK2, lw=0.8))
ax.text(0.5, 0.465, "Conflict 1: −0.438 is called 'indirect' in the text but 'c′' (direct) in the figure.\n"
"Conflict 2: −0.271 is called 'direct' in the text but the 'bootstrap estimate' in the figure.",
ha="center", va="top", fontsize=10, color=INK)
box(ax, 0.0, 0.0, 1.0, 0.30, "Recomputed from the figure's own paths",
f"a × b = −0.509 × 0.263 = {f(AB['DAN'])} c − c′ = −0.572 − (−0.438) = "
f"{f(FIG['DAN']['c'] - FIG['DAN']['cprime'])}\n"
"Neither −0.438, −0.271 nor −0.134 lies inside [0.926, 1.079].", BLUE)
save(fig, *paths)
def cand_D(paths):
"""Calculation-flow diagram for the DAN indirect effect."""
fig, ax = plt.subplots(figsize=(W, 3.0))
ax.set_xlim(-0.02, 1.02)
ax.set_ylim(0, 1)
ax.axis("off")
def node(x, y, w, txt, col):
ax.add_patch(FancyBboxPatch((x, y), w, 0.2, boxstyle="round,pad=0.01,rounding_size=0.02",
fc="white", ec=col, lw=LW, clip_on=False))
ax.text(x + w / 2, y + 0.1, txt, ha="center", va="center", fontsize=10, color=INK)
def arrow(x0, y0, x1, y1):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=12, color=INK2, lw=1.2))
node(0.00, 0.72, 0.22, "a = −0.509\n(Fig 7c(C))", ORANGE)
node(0.00, 0.40, 0.22, "b = 0.263\n(Fig 7c(C))", ORANGE)
node(0.00, 0.08, 0.22, "c = −0.572\nc′ = −0.438", ORANGE)
node(0.32, 0.56, 0.22, f"a × b = {f(AB['DAN'])}", BLUE)
node(0.32, 0.08, 0.22, f"c − c′ = {f(FIG['DAN']['c'] - FIG['DAN']['cprime'])}", BLUE)
arrow(0.22, 0.82, 0.32, 0.68)
arrow(0.22, 0.50, 0.32, 0.64)
arrow(0.22, 0.18, 0.32, 0.18)
node(0.64, 0.56, 0.36, "Text: indirect = −0.438\n95% CI [0.926, 1.079]", ORANGE)
arrow(0.54, 0.66, 0.64, 0.66)
arrow(0.43, 0.28, 0.43, 0.56)
ax.text(0.82, 0.48, "Estimate ≠ a×b (−0.438 vs −0.134)\n"
"CI contains neither, and is\nentirely positive", ha="center", va="top", fontsize=10, color=INK)
ax.text(0.45, 0.40, "agree", fontsize=10, color=INK2, ha="left")
save(fig, *paths)
def cand_E(paths):
"""Sign-check grid: each mediation component's sign, by source."""
cols = ["X → M\n(a path)", "M → Y\n(b path)", "a×b", "printed\nindirect", "printed\n95% CI"]
rows = [
("DAN\nResults text", ["−\nr = −0.266", "+\nr = 0.419", "−", "−\n−0.438", "+\n[0.926, 1.079]"], [0, 0, 0, 0, 1]),
("DAN\nFig 7 caption", ["+\n'positively'", "−\n'negatively'", "−", "", ""], [1, 1, 0, 0, 0]),
("DAN\nFig 7 paths", ["−\na = −0.509", "+\nb = 0.263", f"−\n{f(AB['DAN'])}", "", ""], [0, 0, 0, 0, 0]),
("C–D\nResults, Fig 3", ["+\na = 0.416", "−\nb = −0.260", f"−\n{f(AB['C-D'])}", "−\n−0.389", "−\n[−0.571, −0.032]"], [0, 0, 0, 0, 0]),
]
fig, ax = plt.subplots(figsize=(W, 4.0))
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
x0, rh = 0.17, 0.165
widths = [0.16, 0.16, 0.13, 0.13, 0.25]
lefts = [x0 + sum(widths[:j]) for j in range(5)]
for j, c in enumerate(cols):
ax.text(lefts[j] + widths[j] / 2, 0.94, c, ha="center", va="center", fontsize=10, fontweight="bold")
ax.plot([0, 1], [0.865, 0.865], color=INK2, lw=0.8)
for i, (lab, vals, bad) in enumerate(rows):
y = 0.78 - i * rh
ax.text(0.0, y, lab, ha="left", va="center", fontsize=10, linespacing=1.3)
ax.plot([0, 1], [y - rh / 2, y - rh / 2], color=GRID, lw=0.8)
for j, (v, b) in enumerate(zip(vals, bad)):
if b:
ax.add_patch(FancyBboxPatch((lefts[j] + 0.008, y - rh / 2 + 0.012), widths[j] - 0.016, rh - 0.024,
boxstyle="round,pad=0,rounding_size=0.01", fc="none", ec=ORANGE, lw=LW))
ax.text(lefts[j] + widths[j] / 2, y, v, ha="center", va="center", fontsize=10, linespacing=1.3,
color=BLUE if j == 2 else INK)
ax.text(0.0, -0.02, "X = anhedonia, M = mediator, Y = attention.\nOrange outline: a reported sign that conflicts with the other sources.\n"
"Blue: a×b recomputed from the reported signs or path labels.", fontsize=10, color=INK2, va="bottom")
save(fig, *paths)
cand_A([os.path.join(HERE, "fig1"), os.path.join(CAND, "cand_A_dot_interval")])
cand_C([os.path.join(HERE, "fig2"), os.path.join(CAND, "cand_C_text_panels")])
cand_B([os.path.join(CAND, "cand_B_number_line")])
cand_D([os.path.join(CAND, "cand_D_calc_flow")])
cand_E([os.path.join(CAND, "cand_E_sign_grid")])
print("\nFigures written to", HERE, "and", CAND)
Abstract places the statistical-regularity bias reduction at the low-probability location; Results show the opposite
Hum Brain Mapp (2023) · doi:10.1002/hbm.26520 · PMC10681649
Summary
The abstract states that statistical regularities "reduced this attention bias when the target was presented at a low probability location". The Results report the opposite direction. In BA17 and BA19 the spatial bias vector lies further from the target for the high-probability target location (HTPL > LTPL in distance, t(68) = 4.93 and 2.85). The Methods define a larger distance as less bias towards the target, and target-quadrant (ROI_IN) activity is lower for HTPL (LTPL > HTPL, t(206) = 3.04). Every reported effect size recomputes from its t statistic (d = t/√n), and the Discussion and Conclusions describe the direction as the Results do, so this is an error in the abstract's wording, not in the analysis. It still matters because the direction is what separates the paper's two competing hypotheses: the abstract, read on its own, points to the account the paper's data argue against.
What the paper reports
Abstract:
Salience acted primarily by reducing the attention bias towards the target location when associated with irrelevant distractors, while statistical regularities reduced this attention bias when the target was presented at a low probability location, particularly at the lower levels of the visual hierarchy.
Introduction, last paragraph (the two hypotheses):
In the priority maps framework, we expected increased activation for targets presented in high versus low probability location (HTPL > LTPL), with sensory salience that should further boost these spatially specific responses when the target is also the salient item (Awh et al., 2012; Zhang et al., 2021). As an alternative hypothesis, if mechanisms of “expectation suppression” apply also to recurring targets (see Zhang et al., 2021, for the case of recurring distractors), one may predict a reduction of the spatial bias towards the representation of the target location, for targets presented at the high versus low probability locations (HTPL < LTPL).
Materials and Methods, "Group-level analyses of spatially specific occipital responses" (definition of the outcome):
Any increase of distance between this point and the subject-specific vector expresses a modification of the spatial bias away from the target location (i.e., a decrease of the vector's magnitude and/or a deviation of the vector's sense away from the target location, see Figure 5).
Results, "Attention priority signals modulate activity in the occipital cortex" (ROI analysis):
Specifically, the ROI_IN activity was larger in LTPL compared with HTPL (“LTPL > HTPL”, t(206) = 3.04, p < 0.005, Cohen's d = 0.21), while target-probability did not affect activity in ROI_DIAG (“LTPL > HTPL”, p > 0.9).
This revealed a significant effect of target-probability location in BA17 (“LTPL > HTPL”, t(68) = 3.42, p < 0.003, Cohen's d = 0.41), while this effect did not reach significance in BA18 (p > 0.9) and BA19 (p > 0.2).
Same section (bias-vector analysis):
The comparisons between HTPL and LTPL, separately for the three BAs, revealed a significant effect of statistical regularities in BA17 (HTPL > LTPL, t(68) = 4.93, p < 0.001, Cohen's d = 0.59) and BA19 (HTPL > LTPL, t(68) = 2.85, p = 0.017, Cohen's d = 0.34), but not in BA18 (p > 0.9). In BA17 and BA19, the spatial bias vector was more distant from the target location in HTPL than in LTPL, which explains the target-probability location × BA interaction.
Same section, summary paragraph:
In sum, we found evidence that statistical regularities modulate the spatial bias in area BA17 and BA19, with a loss of the spatial specificity towards the target location in the HTPL condition.
Discussion, "Occipital spatial representations are modulated according to statistical regularities and salience":
In both regions, these analyses showed a loss of the attentional bias for targets presented at the high- compared to the low target-probability location (Figure 5).
The finding of a reduced bias towards the target location in the high-target probability condition is consistent with the “expectation suppression” account.
Conclusions:
Statistical regularities decreased the spatial bias towards the target representation primarily in BA17, and this was independent of parietal-occipital interactions.
What the paper's own numbers show
Method. I took each reported test of target-probability location (HTPL vs LTPL) and fixed its direction from the printed contrast and the Methods definition. For the bias-vector distance, "HTPL > LTPL" means the HTPL vector is further from the target point (−11, 11), that is, less bias towards the target at HTPL. For ROI_IN activity, "LTPL > HTPL" means a weaker response in the target quadrant at HTPL. To check that the printed statistics are internally consistent (so that the direction is not a typo in the test itself), I recomputed Cohen's d for a paired test as d = t/√n. Here n = df + 1. The degrees of freedom correspond to 23 participants × 3 salience conditions (n = 69, df 68) and 23 × 3 salience conditions × 3 Brodmann areas (n = 207, df 206). I also recomputed two-sided p values from t and df. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's t and df |
|---|---|---|---|
| Bias-vector distance, BA17 | Results, bias-vector analysis | HTPL > LTPL, t(68) = 4.93, p < 0.001, d = 0.59 | d = 4.93/√69 = 0.594; two-sided p = 5.6 × 10⁻⁶. Less bias at HTPL |
| Bias-vector distance, BA19 | Results, bias-vector analysis | HTPL > LTPL, t(68) = 2.85, p = 0.017, d = 0.34 | d = 0.343; two-sided p = 0.0058 (× 3 = 0.0173, matching the reported 0.017 if adjusted for three BAs; the paper does not name the correction). Less bias at HTPL |
| Bias-vector distance, BA18 | Results, bias-vector analysis | p > 0.9 | No effect in either direction |
| ROI_IN activity, BA17 | Results, ROI analysis | LTPL > HTPL, t(68) = 3.42, p < 0.003, d = 0.41 | d = 0.412; two-sided p = 0.0011. Weaker target response at HTPL |
| ROI_IN activity, all BAs | Results, ROI analysis | LTPL > HTPL, t(206) = 3.04, p < 0.005, d = 0.21 | d = 3.04/√207 = 0.211; two-sided p = 0.0027. Weaker target response at HTPL |
| Consistency check: ROI_IN > ROI_DIAG, BA17 / BA18 / BA19 | Results, ROI analysis | t(137) = 5.46 / 13.48 / 15.79; d = 0.46 / 1.15 / 1.34 | d = 0.465 / 1.147 / 1.344 (n = 138) |
| Consistency check: SDD > STD in ROI_DIAG | Results, ROI analysis | t(137) = 6.69, d = 0.57 | d = 0.569 |
| Direction of the statistical-regularity effect | Abstract | Bias reduced at the low-probability location | All four significant tests: reduced at the high-probability location; none at the low-probability location |
All reported effect sizes agree with their t statistics to the printed precision, so the signs and contrasts are internally consistent. Every significant test of target-probability location points one way: towards a smaller target bias at HTPL. The abstract assigns the reduction to the other location.
Figure
Figure 1. The left panel quotes the abstract sentence on statistical regularities. The right panel gives the bias-vector and ROI_IN contrasts and a Results sentence, quoted from section 3.2 of the paper. The abstract puts the reduced bias towards the target at the low-probability location (LTPL). The Results put it at the high-probability location (HTPL). The footer notes the Methods definition that links a larger bias-vector distance to less bias, and that the Discussion and Conclusions follow the Results.
Figure 2. Each significant test of target-probability location in the Results, shown as a signed Cohen's d (positive = smaller bias towards the target at HTPL). Orange circles show the d printed in the paper; blue diamonds show d = t/√n recomputed from the paper's t and df. The two agree for every test, and every test lies on the HTPL side of zero. The abstract's wording corresponds to the left side, where no reported test lies. The BA18 bias-vector and ROI_IN tests (p > 0.9) and the BA19 ROI_IN test (p > 0.2) are not drawn, because no t or d is given for them.
Why it matters
- It is one of the abstract's two headline occipital findings. The sentence contrasts the effect of salience with the effect of statistical regularities. With the location reversed, a reader of the abstract alone (or of the PubMed record) takes away the opposite of the paper's result.
- The direction decides between the paper's hypotheses. The Introduction sets out two predictions. The priority-map account predicts a stronger target response at the high-probability location (HTPL > LTPL); the expectation-suppression account predicts a reduced target bias there (HTPL < LTPL). The Results and Discussion support expectation suppression ("The finding of a reduced bias towards the target location in the high-target probability condition is consistent with the 'expectation suppression' account"). The abstract's wording, a reduction at the low-probability location, implies a relatively stronger bias at the high-probability location, which is the pattern the priority-map account predicts.
- The data and statistics are not affected. The size and significance of the effects are as reported (BA17 d = 0.59; BA19 d = 0.34, unadjusted two-sided p ≈ 0.006; no effect in BA18). Only the abstract's statement of the direction needs correcting.
Explanations considered
- Is the distance measure inverted, so that "more distant" means more bias? No. The Methods say an increase in distance "expresses a modification of the spatial bias away from the target location", and the Results add that "the spatial bias vector was more distant from the target location in HTPL than in LTPL". The bias is therefore smaller at HTPL.
- Could the abstract be summarising a different analysis? No. The ROI_IN/ROI_DIAG analysis points the same way as the bias vectors: ROI_IN activity is larger for LTPL than HTPL overall (t(206) = 3.04) and in BA17 (t(68) = 3.42). The retinotopic-atlas replication in Supplementary Figure S2 (Panel C legend) also states "HTPL reduced the bias towards the target position". No analysis in the paper or its supplement reports a reduced target bias at LTPL.
- Could "this attention bias" refer to some other bias, for example towards distractors? No. In the abstract, "this attention bias" points back to "the attention bias towards the target location" in the same sentence. The Results and Discussion describe the statistical-regularity effect as acting on the target representation, at HTPL.
- Could "low probability location" be read as the location the target was not expected at, describing the same contrast from the other side? Only if the sentence said the bias was stronger at the low-probability location. As written, it says the bias was reduced when the target was at the low-probability location, which is the reverse of HTPL < LTPL.
- Could the reversed contrast be a typo in the statistics rather than in the abstract? No. The printed d values match t/√n for every test, the Results prose and Figure 5 description ("the dark-colored vectors (HTPL) that are further away from the 45° dotted-line") agree with the contrasts, and the Discussion and Conclusions describe the same direction.
- Conversion or archiving artifact? No. The PMC text, the Europe PMC XML, the PubMed abstract and the publisher's typeset PDF all carry the same abstract wording. The bioRxiv preprint (10.1101/2023.09.25.557901 v1) has it too.
A separate observation, which does not bear on the direction: the degrees of freedom show that the paired tests pool the three salience conditions (df 68 = 23 × 3 − 1) or the salience conditions and BAs (df 206 = 23 × 9 − 1) as if they were independent observations, so the p values may be smaller than a test on 23 participant-level means would give. This does not change which location shows the reduced bias.
Status
As of 2026-10-24:
- Version of record. The Wiley page returned HTTP 403, so I checked the publisher's typeset Early View PDF (20 pages, with the Wiley footer and "Downloaded from onlinelibrary.wiley.com/doi/10.1002/hbm.26520" on each page) in a copy deposited in an institutional repository. Its abstract has the same "low probability location" sentence, and its Results section 3.2.2 has the same contrasts and numbers. The PMC web text, the Europe PMC full-text XML and the PubMed abstract (PMID 37877138) match. Every quote from the article above was confirmed verbatim against the PMC-archived text; the Figure S2 legend wording comes from the supplement (Data S1).
- Supplements. The only supplement is Data S1 (HBM-44-6439-s001.docx), retrieved from the Europe PMC supplementary-files bundle. Figure S1 explains the bias-vector computation. Figure S2 repeats the bias-vector analysis with a probabilistic retinotopic atlas, and its legend states that HTPL reduced the bias towards the target position (significant in ROI_V2, F(2,110) = 5.6, and ROI_V3, F(2,110) = 4.9). The supplement therefore agrees with the Results, not the abstract. The data are available from the corresponding author on request; no source data are posted.
- Corrections. I found no correction, erratum, expression of concern or retraction. Crossref lists no update-to or updated-by relations (only has-preprint 10.1101/2023.09.25.557901), and a Crossref search for updates to this DOI returned none. PMC shows no notice, and the PubMed record has no CommentsCorrections entries. Web searches found no erratum, no Retraction Watch entry and no PubPeer thread.
- Searches that could not be completed. The Europe PMC search API returned 503 on every attempt, and PubPeer returned 403 to both the page and the API, so neither could be checked directly.
Suggested correction
In the Abstract, replace
statistical regularities reduced this attention bias when the target was presented at a low probability location
with
statistical regularities reduced this attention bias when the target was presented at a high probability location
This leaves the rest of the sentence ("particularly at the lower levels of the visual hierarchy") unchanged. It agrees with the bias-vector results in BA17 and BA19, the ROI_IN results, Supplementary Figure S2, the Discussion and the Conclusions. If the PubMed and indexing records are generated from the article, they should be updated as well.
Recomputation script (python recompute.py.txt)
"""Check the direction of the statistical-regularity effect in PMC10681649
(Hum Brain Mapp 2023, doi:10.1002/hbm.26520).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (Methods "Participants",
Results "Attention priority signals modulate activity in the occipital cortex").
Paired t statistics: Cohen's d for a paired (one-sample) test is d = t / sqrt(n),
with n = df + 1 observations.
"""
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
N_SUBJ = 23 # "Twenty-three participants were included in the final analyses"
N_SAL = 3 # salience conditions HD, STD, SDD
N_BA = 3 # BA17, BA18, BA19
# (label, contrast as printed, t, df, reported d, reported p, sign)
# sign = +1 if the contrast means LESS bias towards the target at HTPL than at LTPL.
# bias-vector distance: larger distance from the target point (-11, 11) = less bias,
# so "HTPL > LTPL" in distance -> less bias at HTPL (+1)
# ROI_IN activity: "LTPL > HTPL" -> weaker target representation at HTPL (+1)
TESTS = [
("Bias-vector distance, BA17", "HTPL > LTPL", 4.93, 68, 0.59, "p < 0.001", +1),
("Bias-vector distance, BA19", "HTPL > LTPL", 2.85, 68, 0.34, "p = 0.017", +1),
("ROI_IN activity, BA17", "LTPL > HTPL", 3.42, 68, 0.41, "p < 0.003", +1),
("ROI_IN activity, all BAs", "LTPL > HTPL", 3.04, 206, 0.21, "p < 0.005", +1),
]
# Further paired tests in the same Results section (consistency check of d = t/sqrt(n))
OTHER = [
("ROI_IN > ROI_DIAG, BA17", 5.46, 137, 0.46),
("ROI_IN > ROI_DIAG, BA18", 13.48, 137, 1.15),
("ROI_IN > ROI_DIAG, BA19", 15.79, 137, 1.34),
("SDD > STD, ROI_DIAG", 6.69, 137, 0.57),
]
R = {}
print("=== 1. Observation counts implied by the degrees of freedom ===")
for k, lab in ((N_SAL, "23 participants x 3 salience conditions"),
(2 * N_SAL, "23 x 6 conditions"),
(N_SAL * N_BA, "23 x 3 salience x 3 BAs")):
n = N_SUBJ * k
print(f" {lab:40s} = {n:3d} observations -> df = {n - 1}")
print("\n=== 2. Statistical-regularity tests: effect size, p value, direction ===")
for lab, con, t, df, d_rep, p_rep, sign in TESTS:
n = df + 1
d = t / np.sqrt(n)
p2 = 2 * stats.t.sf(abs(t), df)
R[lab] = dict(d=d, d_rep=d_rep, p2=p2, sign=sign)
where = "HTPL" if sign > 0 else "LTPL"
print(f" {lab:28s} {con}: t({df}) = {t} n = {n}")
adj = f"; x3 (one per BA) = {min(1, 3 * p2):.3g}" if "BA1" in lab else " (pooled over BAs, no x3)"
print(f" d = {t}/sqrt({n}) = {d:.3f} (reported {d_rep});"
f" two-sided p = {p2:.2g}{adj} (reported {p_rep})")
print(f" -> bias towards the target is smaller at {where}")
print("\n=== 3. Other paired tests in the same section (d = t/sqrt(n)) ===")
for lab, t, df, d_rep in OTHER:
print(f" {lab:26s} t({df}) = {t:5.2f}: d = {t / np.sqrt(df + 1):.3f} (reported {d_rep})")
print("\n=== 4. Direction summary ===")
n_htpl = sum(1 for v in R.values() if v["sign"] > 0)
print(f" significant statistical-regularity tests in the Results: {len(R)}")
print(f" pointing to a smaller target bias at HTPL: {n_htpl}; at LTPL: {len(R) - n_htpl}")
print(" Abstract: bias reduced 'when the target was presented at a low probability location' (LTPL)")
# ---------------------------------------------------------------- figure style
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# ---------------------------------------------------------------- fig1: abstract vs body
# Both panels quote the paper; neither holds a recomputed value, so both use neutral edges.
fig = plt.figure(figsize=(7.2, 4.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.4)
ax.axis("off")
def panel(x, y, w, h, title, body, hl):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=SEC, lw=1.5))
ax.text(x + 0.14, y + h - 0.16, title, fontsize=11, fontweight="bold", va="top", color=TXT)
ax.text(x + 0.14, y + h - 0.52, body, fontsize=10, va="top", color=TXT, linespacing=1.35)
ax.plot([x + 0.14, x + w - 0.14], [y + 0.78, y + 0.78], color=GRID, lw=1)
ax.text(x + 0.14, y + 0.14, hl, fontsize=10.5, fontweight="bold", va="bottom", color=TXT)
abs_txt = "\n".join(textwrap.wrap(
"\u201c\u2026statistical regularities reduced this attention bias when "
"the target was presented at a low probability location, particularly "
"at the lower levels of the visual hierarchy.\u201d", 33))
panel(0.12, 0.80, 3.18, 3.0, "Abstract", abs_txt,
"Reduced at the LOW-probability\nlocation (LTPL)")
res_txt = "\n".join(
["Distance of bias vector from target:",
" BA17: HTPL > LTPL, t(68) = 4.93",
" BA19: HTPL > LTPL, t(68) = 2.85",
"\u201cthe spatial bias vector was more",
"distant from the target location in",
"HTPL than in LTPL\u201d",
"ROI_IN activity:",
" LTPL > HTPL, t(206) = 3.04"])
panel(3.90, 0.80, 3.18, 3.0, "Results, section 3.2", res_txt,
"Reduced at the HIGH-probability\nlocation (HTPL)")
ax.annotate("", xy=(3.86, 1.18), xytext=(3.34, 1.18),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=2))
ax.text(3.60, 1.42, "opposite\nlocation", ha="center", va="bottom", fontsize=10,
fontweight="bold", color=TXT)
ax.text(3.60, 4.10, "Where is the occipital bias towards the target reduced?",
ha="center", va="center", fontsize=11.5, fontweight="bold", color=TXT)
ax.text(3.60, 0.38,
"Methods: a larger distance from the target point (\u221211, 11) means less bias towards the target.\n"
"The Discussion and Conclusions agree with the Results (reduction at HTPL).",
ha="center", va="center", fontsize=10, color=SEC)
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: signed effect sizes
labels = [t[0] for t in TESTS]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.20)
y = np.arange(len(labels))[::-1]
for yi, lab in zip(y, labels):
v = R[lab]
ax.plot([v["sign"] * v["d_rep"]], [yi + 0.12], "o", ms=9, color=ORANGE, zorder=3,
label="Reported d (paper)" if yi == y[0] else None)
ax.plot([v["sign"] * v["d"]], [yi - 0.12], "D", ms=8, color=BLUE, zorder=3,
label="Recomputed d = t/√n" if yi == y[0] else None)
ax.text(v["sign"] * v["d"] + 0.04, yi - 0.12, f"{v['d']:.2f}", va="center", fontsize=10, color=BLUE)
ax.text(v["sign"] * v["d_rep"] + 0.04, yi + 0.14, f"{v['d_rep']:.2f}", va="center",
fontsize=10, color=ORANGE)
ax.axvline(0, color=SEC, lw=1)
ax.set_yticks(y)
ax.set_yticklabels(labels)
ax.set_xlim(-0.8, 0.8)
ax.set_ylim(-0.6, len(labels) - 0.4)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Cohen's d, signed: positive = smaller target bias at HTPL")
ax.text(0.40, len(labels) - 0.30, "Smaller bias at HTPL\n(Results, Discussion)", ha="center",
va="bottom", fontsize=10, color=SEC)
ax.text(-0.40, len(labels) - 0.30, "Smaller bias at LTPL\n(Abstract's wording)", ha="center",
va="bottom", fontsize=10, color=SEC)
ax.text(-0.40, 1.5, "no reported test\non this side", ha="center", va="center",
fontsize=10, color=SEC, style="italic")
ax.legend(loc="lower left", bbox_to_anchor=(0.0, -0.02), frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Testosterone–volume regressions report F(1,88) values that give p ≈ 0.08, not the stated 0.005 and 0.007
Hum Brain Mapp (2025) · doi:10.1002/hbm.70356 · PMC12442050
Summary
The Results report that testosterone significantly predicts larger Right Lobule I–IV and Right Crus II volumes: "adjusted R2 = 1, F(1,88) = 3.15, … p = 0.005" and "adjusted R2 = 0.1, F(1,88) = 3.09, … p = 0.007", with testosterone said to explain "approximately 14% of the variance". These numbers cannot all be true. Each model has a single predictor, and for such a model F(1,88) = 3.15 and 3.09 give p = 0.079 and 0.082, above the paper's own threshold of 0.05, and imply R2 ≈ 0.035 (adjusted ≈ 0.023). An adjusted R2 of 1 is impossible with any finite F, and Figure 9 labels the same models with yet other values (R2 = 0.185 and 0.128, both p = 0.005). This is the paper's only significant hormone–structure result, and the Discussion builds on it, so as published the result cannot be checked.
What the paper reports
Results, section "Sex Steroid Hormones Are Associated With Lobular Cerebellar Volume", only paragraph (the PMC text flattens the superscript in "R 2"):
The regression results indicated that only T levels were significantly associated with volume in the Right I–IV and Right Crus II (Figure 9). These results indicate that higher T levels are significantly associated with larger volumes in the Right I–IV (adjusted R 2 = 1, F(1,88) = 3.15, beta = 1.49E-06, p = 0.005) and Right Crus II (adjusted R 2 = 0.1, F(1,88) = 3.09, beta = 4.18E-06, p = 0.007) regions. The T hormone can explain approximately 14% of the variance in the volumes of these regions. No significant results were found for other hormones.
Methods, "Structural Data Statistical Analysis", third paragraph:
For these analyses, ROI volumes were normalized to each subject's TIV, generating a percentage to account for individual differences in brain size. Each model included hormone level as the independent variable and the TIV-normalized volume of the ROI as the dependent variable. Statistical significance was assessed using p < 0.05.
Methods, "Participants":
A total of 138 participants (mean age = 57 ± 13.3 years, range = 35–86; 54% female) were included.
Figure 9 legend:
Linear regression analysis of testosterone levels predicting cerebellar subregion volumes in Right I–IV and Right Crus II.
Figure 9 panel labels (text inside the figure image, HBM-46-e70356-g001.jpg, read from the image): "R² = 0.185, p = 0.005" (Right I–IV) and "R² = 0.128, p = 0.005" (Right Crus II). The y-axes are labelled "Right I-IV (mm³)" and "Right Crus II (mm³)".
Discussion, fifth paragraph:
Interestingly, testosterone also showed a significant relationship with cerebellar structure, with higher levels predicting larger volumes in Lobules I–IV and Right Crus II in our sample. This suggests that testosterone may contribute to cerebellar preservation during aging, potentially supporting the maintenance of motor and cognitive functions (Volman et al. 2011).
Discussion, sixth paragraph:
Moreover, the ability of testosterone to predict larger cerebellar volumes in Right I–IV and Right Crus II underscores its potential as a biomarker for brain health and aging (Moffat 2005; Andersen et al. 2011; Yeap and Flicker 2022).
What the paper's own numbers show
Method. Methods state that each hormone–volume model has hormone level as its only predictor. For an ordinary least-squares regression with an intercept and one predictor, the overall F(1, n − 2) equals the squared t statistic of the slope, so the model p value and the slope p value are the same number. The model also satisfies R2 = F / (F + df2) and adjusted R2 = 1 − (1 − R2)(n − 1)/(n − 2), which for df2 = 88 reduces to (F − 1)/(F + 88). With df2 = 88, n = 90. I computed p from the upper tail of the F(1,88) distribution (scipy), then derived R2 and adjusted R2 from F, and worked backwards from each reported p and R2 to the F it would need. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| p, Right I–IV | Results | 0.005 | 0.079 from F(1,88) = 3.15 |
| p, Right Crus II | Results | 0.007 | 0.082 from F(1,88) = 3.09 |
| Critical F at the paper's α = 0.05 | Methods (p < 0.05) | — | F(1,88) = 3.95; both reported F values fall below it |
| F needed for the reported p | Results | 3.15 / 3.09 | 8.29 (for p = 0.005) / 7.63 (for p = 0.007) |
| Adjusted R2, Right I–IV | Results | 1 | 0.024 (R2 = 0.035); adjusted R2 = (F − 1)/(F + 88) < 1 for every finite F |
| Adjusted R2, Right Crus II | Results | 0.1 | 0.023 (R2 = 0.034); an adjusted R2 of 0.1 would need F(1,88) = 10.89 (p = 0.0014) |
| Variance explained by testosterone | Results | "approximately 14%" | 3.5% and 3.4%; R2 = 0.14 would need F(1,88) = 14.33 (p = 0.00028) |
| R2 and p, Right I–IV | Figure 9 | R² = 0.185, p = 0.005 | at df (1,88), R² = 0.185 gives F = 19.98, p = 2.3 × 10−5 |
| R2 and p, Right Crus II | Figure 9 | R² = 0.128, p = 0.005 | at df (1,88), R² = 0.128 gives F = 12.92, p = 0.00054; the same p for two different R² at the same n is not possible |
| Sample size of the models | Methods vs Results | 138 participants; df2 = 88 | df2 = 88 means n = 90; n = 138 would give df2 = 136 |
Three sets of values describe the same two regressions, and no two of them agree: the F values (p ≈ 0.08, R2 ≈ 0.03), the text's p values and adjusted R2 (p = 0.005 and 0.007; adjusted R2 1 and 0.1; "14%"), and Figure 9's labels (R2 0.185 and 0.128, both p = 0.005). Only the F values fall short of significance. The others place the association anywhere from p = 0.007 down to p ≈ 2 × 10−5. The 90 versus 138 gap is not counted as an error here (see Explanations considered).
Figure
Figure 1. P values for the two testosterone–volume regressions, on a log scale. Filled orange circles are the p values given in the Results text (0.005 and 0.007); open orange circles are the p values printed on the Figure 9 panels (0.005 for both). Blue diamonds are the p values that the reported F(1,88) = 3.15 and 3.09 imply for a one-predictor regression (0.079 and 0.082). The dashed line marks the paper's significance threshold, p = 0.05. The reported and recomputed values sit on opposite sides of it, 16 and 12 times apart. The log scale is used because p values span orders of magnitude; the plot does not otherwise rescale the difference.
Figure 2. Proportion of variance explained for the same two models, on a linear axis from 0 to 1. Filled orange circles are values in the Results text (adjusted R2 = 1 and 0.1; "approximately 14%", shown on both R2 rows because the text gives one figure for both regions). Open orange circles are the R2 labels on Figure 9 (0.185 and 0.128). Blue diamonds are the R2 and adjusted R2 implied by the reported F(1,88) values (0.035 and 0.024 for Right I–IV; 0.034 and 0.023 for Right Crus II). The dotted line at 1 is the largest value R2 can take, which an adjusted R2 reaches only with a perfect fit. Small vertical offsets within a row only keep markers from overlapping.
Why it matters
- The paper's only significant hormone–structure result rests on these numbers. The Introduction gives as the "second primary aim" whether hormone levels were "associated with cerebellar structure and connectivity". For structure, the Results report that "only T levels were significantly associated with volume", and that "No significant results were found for other hormones". If the F values are correct, neither association reaches p < 0.05. If the p values are correct, the F values and the variance-explained figures are wrong. From what is published, a reader cannot tell which.
- Conclusions built on the result. The Discussion states that testosterone "showed a significant relationship with cerebellar structure", that it "may contribute to cerebellar preservation during aging", and that its prediction of larger volumes "underscores its potential as a biomarker for brain health and aging" and could inform "potential hormone-based interventions". It also says testosterone "was also notable in its associations especially with structure". The final paragraph states "we demonstrated novel associations between sex steroid hormones and both connectivity patterns and structure", and the abstract's last sentence refers to "the significant role of sex steroid hormones in modulating cerebellar connectivity and structure". For structure, each of these rests only on these two regressions.
- The effect size is overstated whichever way it is resolved. The text's "approximately 14%" exceeds what any of the reported test statistics imply: 3.4–3.5% from the F values, about 8–9% if the p values are taken as correct (F of 7.6 to 8.3).
- What is not affected. The functional-connectivity results, which have their own tables with T(127) and FDR-corrected p values, and the sex and age-group differences in volume do not depend on these regressions.
Explanations considered
- A one-sided test. Does not account for the numbers. A one-sided p from t = √F is 0.040 (Right I–IV) and 0.041 (Right Crus II), not 0.005 or 0.007, and the Methods describe no directional test.
- The values printed as F are really t statistics. This partly reconciles the numbers but does not fit them. Read as t(88), 3.15 and 3.09 give F = 9.92 and 9.55, R2 = 0.101 and 0.098, and adjusted R2 = 0.091 and 0.088, which round to the 0.1 given for Right Crus II. But the two-sided p would be 0.0022 and 0.0027, not 0.005 and 0.007; the adjusted R2 for Right I–IV would still not be 1; and neither the "14%" nor Figure 9's R2 of 0.185 and 0.128 would match. Nothing in the paper indicates that these are t values.
- Different degrees of freedom. Does not account for the numbers. With the full sample (df2 = 136) or the df of the functional tables (T(127)), F = 3.15 still gives p = 0.078 and F = 3.09 gives p = 0.081.
- The df of 88 reflects missing hormone data. Possible, and so the 90 versus 138 gap is not counted as an error. The paper does not say how many participants had usable hormone values for these models. A low-resolution copy of Figure 9 appears to hold more than 90 points (a rough automated marker count gives about 120 to 130; overlapping markers make an exact count impossible), so the analysed n is uncertain. Whatever n is, it does not reconcile F, p and R2: the F-to-p conversion barely changes with df2 (above), and with n = 129 or 138 Figure 9's R2 values would give p below 10−4.
- Covariates in the models. Ruled out by the Methods, which name hormone level as the only independent variable. Extra covariates would also not make an adjusted R2 of 1 compatible with F = 3.15.
- "Adjusted R2 = 1" is a typo for 0.1. Likely for that single value. The bioRxiv preprint (PMC11230255, Table 10) gives "Adjusted R2=.1" for both regions, so the published "1" most likely differs from ".1" only by a lost leading decimal point. This does not resolve the conflict: F = 3.15 implies an adjusted R2 of 0.024, and an adjusted R2 of 0.1 would need F = 10.89. The preprint has the same F values (3.15 and 3.09) alongside the same slope p values (0.005 and 0.007), and says "The models explain approximately 14% of the variance".
- The p values belong to the slope and the F to the model. Does not account for the numbers. In a one-predictor regression the two tests are the same test (t2 = F), so they cannot differ.
- Figure 9 shows a different model. Possible. Its y-axes are in mm³, whereas the Methods say the regressions used TIV-normalised volumes, so its R2 values may come from a regression on raw volumes. But this does not account for the text. Within the figure, R2 = 0.185 and 0.128 cannot both have p = 0.005 at the same n: at n = 90 they give p = 2.3 × 10−5 and 0.00054, and p = 0.005 would need n of about 42 and 61 respectively.
- A PMC conversion or typesetting artifact. Ruled out. The Europe PMC JATS XML and the live PMC page carry the same numbers with the superscript of R² and the minus signs intact. The numbers are in running text, not table cells. The Figure 9 values are in the publisher's own image.
- A correction already issued. None found (see Status).
- Side note, not part of the finding. The abstract gives the age groups as 38 EMA, 48 LMA and 42 OA, which sum to 128. The Methods give 48 EMA, which with the same LMA and OA counts sums to the stated 138.
Status
- Version of record. The publisher's site (Wiley Online Library, reached through doi.org) returned HTTP 403 to automated requests, so the typeset PDF was not read. The publisher-supplied JATS text was checked in the Europe PMC full-text XML and on the live PMC page: both show the passages quoted above, in section 3.3.2. Figure 9 was read from the publisher's image file (
HBM-46-e70356-g001.jpg) in the Europe PMC supplementary bundle. Every text quotation was checked verbatim against the PMC-archived text withsrc/show.py. - Supplements. Data S1 (
HBM-46-e70356-s001.docx), the only supplementary file, holds Supplementary Tables 1–3: coordinates, T statistics and pFDR for the functional-connectivity correlations with testosterone and progesterone. It has no output for the structural regressions, no n and no standard errors. The earlier bioRxiv preprint (doi:10.1101/2024.06.24.600454; PMC11230255, Table 10) was also checked, as described above. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has empty update-to, updated-by and relation fields. The Europe PMC record (PMID 40960201) lists only the preprint ("Update of"). The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC12442050; Europe PMC full-text XML, metadata and supplementary files for PMC12442050; bioRxiv preprint PMC11230255 (Europe PMC full-text XML); Crossref; PubPeer; https://doi.org/10.1002/hbm.70356.
Suggested correction
The authors could re-run the two testosterone–volume regressions and report, for each, the analysed n, R2, adjusted R2, F with its degrees of freedom, the slope with its standard error and units, and the p value, all from the same model output. If the corrected statistics confirm p < 0.05, the passage in section 3.3.2 should be replaced with those values, the Figure 9 labels made consistent with them (or labelled as a separate raw-volume model, if that is what they show), and the "approximately 14%" changed to the R2 actually obtained. If the F values of 3.15 and 3.09 are correct, the associations are not significant at the paper's α = 0.05 (p = 0.079 and 0.082). In that case the Results sentence "only T levels were significantly associated with volume" should be withdrawn, along with the Discussion statements that testosterone "showed a significant relationship with cerebellar structure" and "underscores its potential as a biomarker", and the structural part of the closing claims. It would also help to state why the models have df2 = 88 when 138 participants were included, and to correct the EMA group size in the abstract (38) to match the Methods (48).
Recomputation script (python recompute.py.txt)
"""Recompute the testosterone-volume regression statistics of PMC12442050
(Hum Brain Mapp 2025, doi:10.1002/hbm.70356,
"The Influence of Sex and Sex-Steroid Hormones on Cerebellar Structure and
Functional Connectivity Across Adulthood").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs copied from the paper:
Results, "Sex Steroid Hormones Are Associated With Lobular Cerebellar Volume":
Right I-IV: adjusted R2 = 1, F(1,88) = 3.15, beta = 1.49E-06, p = 0.005
Right Crus II: adjusted R2 = 0.1, F(1,88) = 3.09, beta = 4.18E-06, p = 0.007
"The T hormone can explain approximately 14% of the variance"
Figure 9 panel labels (image HBM-46-e70356-g001.jpg):
Right I-IV "R2 = 0.185, p = 0.005"; Right Crus II "R2 = 0.128, p = 0.005"
Methods: one predictor (hormone level) per model; p < 0.05; 138 participants.
Tables of the functional analyses: T(127).
Abstract: 38 EMA, 48 LMA, 42 OA; Methods: 48 EMA, 48 LMA, 42 OA.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
D1, D2 = 1, 88 # reported degrees of freedom
N = D2 + D1 + 1 # simple regression: n = df2 + 2 = 90
ALPHA = 0.05 # Methods
ROWS = [
# region, F, reported p (text), reported adjusted R2 (text), Figure 9 R2, Figure 9 p
("Right I–IV", 3.15, 0.005, 1.0, 0.185, 0.005),
("Right Crus II", 3.09, 0.007, 0.1, 0.128, 0.005),
]
R2_TEXT = 0.14 # "approximately 14% of the variance"
def r2_from_F(F, d1=D1, d2=D2):
return F * d1 / (F * d1 + d2)
def adj_from_r2(r2, n=N, k=D1):
return 1 - (1 - r2) * (n - 1) / (n - k - 1)
def F_from_r2(r2, d1=D1, d2=D2):
return r2 / (1 - r2) * d2 / d1
def r2_from_adj(adj, n=N, k=D1):
return 1 - (1 - adj) * (n - k - 1) / (n - 1)
R = {}
print("=== 1. Recomputed from each reported F(1,88) ===")
print(f"n implied by df2 = 88 in a one-predictor model: {N}")
Fcrit = stats.f.isf(ALPHA, D1, D2)
print(f"critical F(1,88) at alpha = 0.05: {Fcrit:.3f}")
for reg, F, p_rep, adj_rep, r2_fig, p_fig in ROWS:
p = stats.f.sf(F, D1, D2)
r2 = r2_from_F(F)
adj = adj_from_r2(r2)
t = np.sqrt(F)
Fneed = stats.f.isf(p_rep, D1, D2)
R[reg] = dict(F=F, p=p, r2=r2, adj=adj, p_rep=p_rep, adj_rep=adj_rep,
r2_fig=r2_fig, p_fig=p_fig, Fneed=Fneed)
print(f"{reg}: F(1,88) = {F}")
print(f" p = {p:.4f} (reported {p_rep}); ratio p/p_reported = {p/p_rep:.1f}")
print(f" R2 = F/(F+88) = {r2:.4f}; adjusted R2 = {adj:.4f} (reported {adj_rep})")
print(f" F needed for p = {p_rep}: {Fneed:.2f} (R2 {r2_from_F(Fneed):.3f}, adjusted {adj_from_r2(r2_from_F(Fneed)):.3f})")
print()
print("=== 2. Reported adjusted R2 values ===")
print("adjusted R2 for one predictor and df2 = 88 equals (F - 1)/(F + 88), which is < 1 for every finite F")
for F in (3.15, 1e3, 1e6):
print(f" F = {F:g}: adjusted R2 = {(F-1)/(F+88):.6f}")
for adj in (0.1,):
r2 = r2_from_adj(adj)
F = F_from_r2(r2)
print(f"adjusted R2 = {adj} -> R2 = {r2:.4f}, F(1,88) = {F:.2f}, p = {stats.f.sf(F, D1, D2):.5f}")
print()
print("=== 3. 'approximately 14% of the variance' ===")
F14 = F_from_r2(R2_TEXT)
print(f"R2 = 0.14 at df (1,88) -> F = {F14:.2f}, p = {stats.f.sf(F14, D1, D2):.6f}, adjusted R2 = {adj_from_r2(R2_TEXT):.4f}")
print(f"variance explained implied by the F values: {100*R['Right I–IV']['r2']:.1f}% and {100*R['Right Crus II']['r2']:.1f}%")
print()
print("=== 4. Figure 9 panel labels ===")
for reg, d in R.items():
Ff = F_from_r2(d["r2_fig"])
print(f"{reg}: R2 = {d['r2_fig']} at df (1,88) -> F = {Ff:.2f}, p = {stats.f.sf(Ff, D1, D2):.2g} (label p = {d['p_fig']})")
# sample size at which this R2 would give exactly p = 0.005
for n in range(5, 400):
if stats.f.sf(F_from_r2(d["r2_fig"], 1, n - 2), 1, n - 2) <= d["p_fig"]:
print(f" smallest n giving p <= {d['p_fig']} with R2 = {d['r2_fig']}: n = {n}")
break
for n in (129, 138):
Fn = F_from_r2(d["r2_fig"], 1, n - 2)
print(f" with n = {n}: F(1,{n-2}) = {Fn:.2f}, p = {stats.f.sf(Fn, 1, n-2):.2g}")
print()
print("=== 5. Alternative readings of F = 3.15 / 3.09 ===")
for reg, d in R.items():
F = d["F"]
t = np.sqrt(F)
print(f"{reg}:")
print(f" one-sided p from t = sqrt(F) = {t:.3f}: {stats.t.sf(t, D2):.4f}")
print(f" value read as a t(88), two-sided: p = {2*stats.t.sf(F, D2):.4f}")
r2t = F**2 / (F**2 + D2)
print(f" value read as a t(88): F = t^2 = {F**2:.2f}, implied R2 = {r2t:.4f}, adjusted R2 = {adj_from_r2(r2t):.4f}")
for d2 in (127, 136):
print(f" F(1,{d2}) = {F}: p = {stats.f.sf(F, 1, d2):.4f}")
print(f"n = 138 in a simple regression would give df2 = {138-2}")
print()
print("=== 6. Side note: age-group counts ===")
print(f"Abstract 38 + 48 + 42 = {38+48+42}; Methods 48 + 48 + 42 = {48+48+42}; stated total 138")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "axes.linewidth": 0.8,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def style(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
# Figure 1: p values on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.1))
style(ax)
ys = {"Right I–IV": 1.0, "Right Crus II": 0.0}
for reg, d in R.items():
y = ys[reg]
ax.plot([d["p_rep"], d["p"]], [y, y], color=GRID, lw=2, zorder=1)
ax.scatter([d["p_rep"]], [y + 0.12], s=80, color=ORANGE, zorder=3)
ax.text(d["p_rep"], y + 0.32, f"reported p = {d['p_rep']}", ha="center", va="bottom", fontsize=10, color=ORANGE)
ax.scatter([d["p"]], [y], s=80, color=BLUE, marker="D", zorder=3)
ax.text(d["p"], y + 0.2, f"p from F(1,88) = {d['F']}:\n{d['p']:.3f}", ha="center", va="bottom", fontsize=10, color=BLUE)
ax.scatter([d["p_fig"]], [y - 0.12], s=80, facecolor=BG, edgecolor=ORANGE, lw=2, zorder=3)
ax.text(d["p_fig"] * 1.08, y - 0.2, f"Figure 9 label p = {d['p_fig']}", ha="left", va="top", fontsize=10, color=ORANGE)
ax.axvline(ALPHA, color=INK2, lw=1, ls="--", zorder=2)
ax.text(ALPHA * 1.04, -0.55, "paper's threshold p = 0.05", ha="left", va="bottom", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.0012, 0.25)
ax.set_xticks([0.002, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2])
ax.set_xticklabels(["0.002", "0.005", "0.01", "0.02", "0.05", "0.1", "0.2"])
ax.set_ylim(-0.62, 1.75)
ax.set_yticks([1.0, 0.0])
ax.set_yticklabels(["Right I–IV", "Right Crus II"], fontsize=10.5, color=INK)
ax.set_xlabel("p value (log scale)")
handles = [
Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="reported in Results text"),
Line2D([], [], marker="o", ls="", markerfacecolor=BG, markeredgecolor=ORANGE, mew=2, ms=9, label="reported in Figure 9"),
Line2D([], [], marker="D", ls="", color=BLUE, ms=8, label="recomputed from reported F(1,88)"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, -0.24), ncol=3, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
save(fig, "fig1")
# Figure 2: variance explained on one linear axis (0 to 1)
fig, ax = plt.subplots(figsize=(7.2, 3.9))
style(ax)
rI, rC = R["Right I–IV"], R["Right Crus II"]
# each point: (y, value, label, colour, filled, marker, label dy, label ha, label dx)
pts = [
(4, rI["adj_rep"], "1 (text)", ORANGE, True, "o", 0.3, "right", 0.0),
(4, rI["adj"], f"{rI['adj']:.3f}", BLUE, True, "D", -0.32, "left", -0.01),
(3.1, R2_TEXT, "≈ 0.14 (text: “approximately 14%”)", ORANGE, True, "o", 0.25, "left", -0.01),
(2.9, rI["r2_fig"], f"{rI['r2_fig']} (Figure 9)", ORANGE, False, "o", -0.25, "left", 0.0),
(3, rI["r2"], f"{rI['r2']:.3f}", BLUE, True, "D", -0.32, "left", -0.01),
(2, rC["adj_rep"], f"{rC['adj_rep']} (text)", ORANGE, True, "o", 0.3, "center", 0.0),
(2, rC["adj"], f"{rC['adj']:.3f}", BLUE, True, "D", -0.32, "left", -0.01),
(1.1, R2_TEXT, "≈ 0.14 (text)", ORANGE, True, "o", 0.25, "left", -0.01),
(0.9, rC["r2_fig"], f"{rC['r2_fig']} (Figure 9)", ORANGE, False, "o", -0.25, "left", 0.012),
(1, rC["r2"], f"{rC['r2']:.3f}", BLUE, True, "D", -0.32, "left", -0.01),
]
for y, v, lab, c, filled, mk, dy, ha, dx in pts:
if filled:
ax.scatter([v], [y], s=80, color=c, marker=mk, zorder=3)
else:
ax.scatter([v], [y], s=80, facecolor=BG, edgecolor=c, lw=2, marker=mk, zorder=4)
ax.text(v + dx, y + dy, lab, ha=ha, va="center", fontsize=10, color=c)
ax.axvline(1.0, color=INK2, lw=1, ls=":", zorder=2)
ax.text(0.985, 2.5, "maximum\npossible\nvalue", ha="right", va="center", fontsize=10, color=INK2)
ax.set_xlim(-0.02, 1.04)
ax.set_xticks(np.arange(0, 1.01, 0.1))
ax.set_ylim(0.45, 4.55)
ax.set_yticks([4, 3, 2, 1])
ax.set_yticklabels(["Right I–IV\nadjusted R²", "Right I–IV\nR²", "Right Crus II\nadjusted R²", "Right Crus II\nR²"],
fontsize=10.5, color=INK)
ax.set_xlabel("proportion of variance explained")
handles = [
Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="reported in Results text"),
Line2D([], [], marker="o", ls="", markerfacecolor=BG, markeredgecolor=ORANGE, mew=2, ms=9, label="reported in Figure 9"),
Line2D([], [], marker="D", ls="", color=BLUE, ms=8, label="recomputed from reported F(1,88)"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.45, -0.17), ncol=3, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
save(fig, "fig2")
print()
print("figures written:", ", ".join(os.path.join(HERE, f) for f in ("fig1.svg", "fig1.png", "fig2.svg", "fig2.png")))
Paired t-test p-values for one-year cortical thickness change do not match the reported t statistics
Hum Brain Mapp (2009) · doi:10.1002/hbm.20740 · PMC2903209
Summary
The paper's only real-MRI validation (Section IV.B) tests one-year change in mean cortical thickness in 9 patients with Alzheimer's disease (AD) and 13 controls. It reports paired t statistics of −1.200 and −1.495 in AD (df = 8) and −0.274 and +0.175 in controls (df = 12), with p = 0.001, 0.001, 0.009 and 0.001. For those t statistics the two-sided p-values are 0.264, 0.173, 0.789 and 0.864. Table I independently gives t values of about the same size and p-values of 0.19 to 0.84. With the correct p-values none of the four within-group changes is statistically significant. That undoes the paper's statement that both methods "detect a systematic change" in both groups. It also undoes footnote 9's statement that FreeSurfer's increase in controls "is statistically significant", which the paper uses in its comparison with FreeSurfer.
What the paper reports
Section IV.B (Real MRI Data), paragraph 3, sample:
Our dataset includes pairs of scans over a one-year interval from 22 subjects (total of 44 scans), of whom 9 had been diagnosed with Alzheimer’s disease at their first scan, and 13 were age-matched normal subjects.
Section IV.B (Real MRI Data), paragraph 5, the tests in question:
Both techniques are able to detect a systematic change in cortical thickness over time, both in the AD (Alzheimer’s disease) group and in controls. Our method found thickness declined in AD (tpaired = -1.200; p=0.001), and in controls (tpaired = -0.274; p=0.009); FreeSurfer found thickness declined in AD (tpaired = -1.495; p=0.001), and increased in controls (tpaired = +0.175; p=0.001).9
Footnote 9, attached to that sentence (in the PMC text it sits in the back matter):
This increase, which is anatomically not expected, is statistically significant. No increase is found with our proposed method.
Section IV.B, paragraph 6, the baseline/follow-up correlations (the next paragraph):
Our technique gives measures that correlated highly over the 1 year time-interval in AD (p=0.001) and in controls (p=0.009), and so did FreeSurfer (p=0.001 both for AD and for normal subjects).
Section IV.B, paragraph 9, the interpretation:
Both our method and FreeSurfer report a few subjects that show increases (less pronounced with our approach), while on average our proposed method shows decline for both populations (much more significant for the AD subjects, as expected). Even so, FreeSurfer shows a net increase for the control subjects, not very large but still statistically significant. A number of reasons might explain this, including registration issues and segmentation errors, which are minimized in our proposed technique.
Table I ("Statistical Results on Real MRI Data"; relative change in mean thickness over one year), rows for the two groups (columns: proposed algorithm | FreeSurfer; spacing normalised):
Mean change for AD subjects | -1.42 % | -1.15 % Mean change for normal subjects | -0.19 % | +0.20 % SD of change for AD subjects | 3.52 | 2.40 SD of change for normal subjects | 3.39 | 2.82
The Abstract mentions this dataset without giving numbers ("demonstrate its performance on both artificial volumes and real 3D brain MRI data from subjects with Alzheimer’s disease and healthy individuals"). The Conclusion says the authors "presented reasonable results for longitudinal MRI scans of Alzheimer’s disease and normal subjects".
What the paper's own numbers show
Method. The tests compare each subject's baseline and follow-up thickness, so df = n − 1: 8 for AD and 12 for controls. I computed the two-sided p-value for each reported t from the t distribution, along with the one-sided p-value and the |t| that the reported p would require. As an independent check I rebuilt each t from Table I as mean % change / (SD / √n), with a rounding envelope of ±0.005 on the mean and SD. I also used Table I's "all subjects" rows to confirm the group sizes. All values come from recompute.py.txt (scipy.stats).
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Proposed method, AD: paired t, df = 8 | Sec IV.B para 5 | t = −1.200; p = 0.001 | two-sided p = 0.264 (one-sided 0.132); p = 0.001 needs |t| ≥ 5.04 |
| FreeSurfer, AD: paired t, df = 8 | Sec IV.B para 5 | t = −1.495; p = 0.001 | two-sided p = 0.173 (one-sided 0.087); p = 0.001 needs |t| ≥ 5.04 |
| Proposed method, controls: paired t, df = 12 | Sec IV.B para 5 | t = −0.274; p = 0.009 | two-sided p = 0.789 (one-sided 0.394); p = 0.009 needs |t| ≥ 3.11 |
| FreeSurfer, controls: paired t, df = 12 | Sec IV.B para 5; footnote 9 | t = +0.175; p = 0.001; "statistically significant" | two-sided p = 0.864 (one-sided 0.432); p = 0.001 needs |t| ≥ 4.32 |
| Same four t, with unlimited df (normal approximation) | — | — | p = 0.230, 0.135, 0.784, 0.861; no choice of df gives the reported p |
| t from Table I, proposed method, AD (−1.42 %, SD 3.52, n = 9) | Table I | (text: −1.200) | t = −1.210 (rounding range −1.216 to −1.204); p = 0.261 |
| t from Table I, FreeSurfer, AD (−1.15 %, SD 2.40, n = 9) | Table I | (text: −1.495) | t = −1.438 (−1.447 to −1.428); p = 0.189 |
| t from Table I, proposed method, controls (−0.19 %, SD 3.39, n = 13) | Table I | (text: −0.274) | t = −0.202 (−0.208 to −0.196); p = 0.843 |
| t from Table I, FreeSurfer, controls (+0.20 %, SD 2.82, n = 13) | Table I | (text: +0.175) | t = +0.256 (+0.249 to +0.263); p = 0.803 |
| Mean change, all subjects (weighted from group rows) | Table I | −0.69 % / −0.35 % | −0.693 % / −0.352 % |
| SD of change, all subjects (pooled from group rows) | Table I | 3.42 / 2.68 | 3.416 / 2.683 |
| Group separation, (mean normal − mean AD) / SD all | Table I | 0.36 / 0.50 | 0.360 / 0.504 |
| p-values of the t-tests vs p-values of the baseline/follow-up correlations | Sec IV.B paras 5 and 6 | 0.001, 0.009, 0.001, 0.001 (para 5); 0.001, 0.009, 0.001, 0.001 (para 6) | identical, in the same order; at n = 9 a correlation of r = 0.898 gives two-sided p = 0.001 |
The Table I checks confirm n = 9 and n = 13. All four printed t values are close to, but slightly outside, the range implied by Table I's rounded means and SDs. They differ from the Table I values by 0.010 and 0.058 (AD) and by 0.072 and 0.081 (controls). The paper does not say whether the tests used percent change or absolute thickness, which could account for those differences. They do not affect the conclusion: every t value the paper's numbers support gives a two-sided p between 0.17 and 0.86.
A separate check on the next paragraph found a smaller inconsistency of the same kind. It does not affect the main finding:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Agreement of change between methods, all subjects (r = 0.572, n = 22) | Sec IV.B para 7 | p = 0.0028 | two-sided p = 0.0054 (one-sided 0.0027) |
| Agreement of change between methods, controls (r = 0.583, n = 13) | Sec IV.B para 7 | p = 0.0011 | two-sided p = 0.0365 (one-sided 0.0183) |
| Agreement of change between methods, AD (r = 0.558, n = 9) | Sec IV.B para 7 | p = 0.0047 | two-sided p = 0.1185 (one-sided 0.0592) |
Figure
Figure 1. The four paired t-tests of one-year change in mean cortical thickness in Section IV.B, on one log-scaled p-value axis. Orange dots are the p-values printed in the paper. Filled blue dots are the two-sided p-values for the printed t statistic and df (8 for 9 AD patients, 12 for 13 controls). Hollow blue rings are the two-sided p-values for t rebuilt from Table I's mean and SD of percent change. The thin vertical line marks p = 0.05. In every row the two recomputed values agree with each other (0.17 to 0.86) and lie well above the printed value, on the other side of the p = 0.05 line. All four changes are non-significant at the conventional level.
Figure 2. The left panel shows the four paired t-tests from Section IV.B paragraph 5, with the printed t, the printed p (orange) and the two-sided p for that t (blue). The right panel shows the p-values that paragraph 6 gives for the baseline/follow-up thickness correlations in the same four method-by-group cells. The printed p-values in the two paragraphs are identical, cell for cell. That identity is a fact about the text. It is consistent with, but does not prove, the explanation that the correlation p-values were entered in place of the t-test p-values (see Explanations considered).
Why it matters
The paper is mainly a methods paper. Its central contribution is defining thickness as a minimum line integral over a gray-matter probability map, together with the algorithm, the synthetic-data validation and the run time, and none of that depends on these p-values. The method-agreement correlation for all 22 subjects (r = 0.572) is also unaffected in substance.
The longitudinal AD/control dataset is the paper's only test on real MRI, and these four tests are its main quantitative result. With the corrected values:
- "Both techniques are able to detect a systematic change in cortical thickness over time, both in the AD … group and in controls" is not supported. Neither method detects a statistically significant one-year change in either group (p = 0.17 to 0.86 with n = 9 and n = 13). The mean changes are in the expected direction for AD. With these sample sizes and SDs of change, they do not differ significantly from zero.
- The comparative argument against FreeSurfer loses its statistical basis. Footnote 9 and paragraph 9 describe FreeSurfer's mean increase in controls as "statistically significant" and attribute it to "registration issues and segmentation errors, which are minimized in our proposed technique." The increase has p = 0.864 (t = +0.175), or 0.803 from Table I, and is consistent with no change. Neither method's control-group mean differs significantly from zero, so on these numbers the paper cannot conclude that one method is biased and the other is not.
- "Much more significant for the AD subjects" compares p-values (0.001 vs 0.009) that are not the p-values of these tests.
The descriptive content of Table I (means, SDs, group separation 0.36 vs 0.50) is internally consistent and not affected.
Explanations considered
- Different sample size or df. This does not account for the numbers. Section IV.B gives 9 AD patients and 13 controls, and Table I's pooled "all subjects" mean and SD reproduce exactly from those group sizes. No df rescues the values: even with unlimited df, t = −1.200 gives p = 0.230 and t = +0.175 gives p = 0.861.
- One-sided tests. This does not account for the numbers. The one-sided p-values are 0.132, 0.087, 0.394 and 0.432, all far above the reported values.
- The "t" values are something other than t statistics (for example mean differences or effect sizes). This does not account for the numbers. They match mean / (SD/√n) from Table I closely (within 0.06 for both AD tests), so they behave as one-sample t statistics on the paired change.
- A non-parametric test (Wilcoxon signed-rank) produced the p-values. This is not plausible for AD. With n = 9 the smallest attainable exact two-sided p is 2/2⁹ = 0.0039, so p = 0.001 cannot arise. It would also be inconsistent with the label "tpaired".
- Tests run across many vertices or voxels rather than subjects. Nothing in the paper indicates this. Table I summarises one percent change per subject, and as noted above, the reported t values cannot give these p-values at any df.
- Typographical error in the t values rather than the p-values. This does not account for the numbers. The t values agree with Table I, which is reported independently. Reaching p = 0.001 would need |t| ≥ 5.04 (AD) or ≥ 4.32 (controls), which is inconsistent with Table I's means and SDs.
- The p-values belong to a different analysis. This is consistent with the numbers. The four printed p-values are identical, in the same order, to those the next paragraph gives for the baseline/follow-up thickness correlations. Correlations of about r = 0.90 (n = 9) and r = 0.69 to 0.80 (n = 13) would produce them, which is plausible for test-retest thickness. Standard paired-samples output reports a correlation significance next to the paired-t significance. This would explain how the values could have been transposed, but it does not make them correct for the t-tests.
- PMC conversion or transcription artefact. This is ruled out. The publisher's typeset PDF (page 3193) carries the same numbers.
Status
- Version of record. The Wiley article pages (doi.org redirect to onlinelibrary.wiley.com, including /pdf, /pdfdirect and /full) returned HTTP 403. I checked the publisher's typeset PDF that PMC hosts instead (HBM-30-3188.pdf, 12 pages, "Human Brain Mapping 30:3188–3199 (2009)", © 2009 Wiley-Liss, created 2009-09-11). Page 3193 carries the paragraph 5 sentence, footnote 9 and the paragraph 6 correlation p-values exactly as quoted. Page 3194 carries Table I with the same values. Every quotation above was also confirmed verbatim against the PMC-archived text.
- Supplements. The article has no supporting information (Europe PMC hasSuppl = N), and no individual-subject data are deposited. Figure 7 shows per-subject bars only. Nothing available could resolve the discrepancy.
- Corrections and comments. As of 2026-10-23, no correction, erratum, expression of concern, retraction or published comment was found. Crossref lists no update relations (update-to / updated-by empty; filter=updates returns 0 items). PMC and Europe PMC list no notices (src/known.py). The PubMed record (PMID 19219850) has no CommentsCorrectionsList or ErratumIn entries. PubPeer's search page returned HTTP 403 to automated access and its publication API returned no record for the DOI. Web searches for the DOI with "correction", "erratum" or "PubPeer" found nothing.
Suggested correction
- Replace the four p-values in Section IV.B paragraph 5 with the p-values of the paired t-tests. For the printed t statistics these are, two-sided, 0.264 (proposed method, AD), 0.789 (proposed method, controls), 0.173 (FreeSurfer, AD) and 0.864 (FreeSurfer, controls). Also state whether the tests used percent change or absolute thickness, because the printed t values differ slightly from those implied by Table I.
- Revise the opening sentence of paragraph 5 ("Both techniques are able to detect a systematic change … both in the AD group and in controls"), footnote 9 ("is statistically significant"), and the paragraph 9 statements "much more significant for the AD subjects" and "not very large but still statistically significant", together with the inference about FreeSurfer's registration and segmentation errors that rests on them.
- Check the p-values for the method-agreement correlations in paragraph 7. For r = 0.583 (n = 13) and r = 0.558 (n = 9) the two-sided values are 0.037 and 0.119, not 0.0011 and 0.0047. Also state whether the tests are one- or two-sided (the all-subjects value of 0.0028 matches a one-sided test).
Recomputation script (python recompute.py.txt)
"""Recompute the paired t-test p-values in PMC2903209
(Hum Brain Mapp 2009;30:3188-3199, doi:10.1002/hbm.20740), Section IV.B "Real MRI Data".
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg / fig1.png and fig2.svg / fig2.png next to this file.
Inputs copied from the paper (as archived in PMC and in the publisher's PDF):
Sec IV.B para 3: 22 subjects scanned twice one year apart; 9 AD, 13 age-matched controls.
Sec IV.B para 5: four paired t statistics and their p-values.
Sec IV.B para 6: p-values of the baseline/follow-up thickness correlations.
Sec IV.B para 7: Pearson r (and p) for agreement of change between the two methods.
Table I: mean and SD of the one-year percent change in mean thickness, by group.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (from the paper)
N_AD, N_NC = 9, 13 # Sec IV.B para 3
# label: (reported t, n, reported p) # Sec IV.B para 5
REPORTED = {
"Proposed method, AD": (-1.200, N_AD, 0.001),
"Proposed method, controls": (-0.274, N_NC, 0.009),
"FreeSurfer, AD": (-1.495, N_AD, 0.001),
"FreeSurfer, controls": (+0.175, N_NC, 0.001),
}
# label: (mean % change, SD of % change, n) # Table I
TABLE_I = {
"Proposed method, AD": (-1.42, 3.52, N_AD),
"Proposed method, controls": (-0.19, 3.39, N_NC),
"FreeSurfer, AD": (-1.15, 2.40, N_AD),
"FreeSurfer, controls": (+0.20, 2.82, N_NC),
}
TABLE_I_ALL = { # method: (mean all, SD all, group separation)
"Proposed method": (-0.69, 3.42, 0.36),
"FreeSurfer": (-0.35, 2.68, 0.50),
}
# Baseline/follow-up correlation p-values, Sec IV.B para 6 (same order as REPORTED)
CORR_P_PARA5 = {
"Proposed method, AD": 0.001, "Proposed method, controls": 0.009,
"FreeSurfer, AD": 0.001, "FreeSurfer, controls": 0.001,
}
# Agreement of change between methods, Sec IV.B para 7: (r, n, reported p)
AGREEMENT = {
"All subjects": (0.572, N_AD + N_NC, 0.0028),
"Controls": (0.583, N_NC, 0.0011),
"AD": (0.558, N_AD, 0.0047),
}
def p_two(t, df):
return 2 * stats.t.sf(abs(t), df)
def p_one(t, df):
return stats.t.sf(abs(t), df)
def t_needed(p, df, sided=2):
return stats.t.isf(p / sided, df)
def r_pvalue(r, n, sided=2):
t = r * np.sqrt((n - 2) / (1 - r * r))
return sided * stats.t.sf(abs(t), n - 2)
def main():
rows = []
print("== 1. Reported paired t statistics vs reported p-values (Sec IV.B para 5) ==")
for lab, (t, n, prep) in REPORTED.items():
df = n - 1
p2, p1 = p_two(t, df), p_one(t, df)
print(f"{lab:27s} t={t:+.3f} df={df:2d} two-sided p={p2:.3f} one-sided p={p1:.3f} "
f"reported p={prep} |t| needed for reported p: two-sided {t_needed(prep, df):.2f}, "
f"one-sided {t_needed(prep, df, 1):.2f}")
rows.append([lab, t, df, prep, p2])
print("\n== 2. Even with unlimited df (normal approximation), the reported t cannot give the reported p ==")
for lab, (t, n, prep) in REPORTED.items():
print(f"{lab:27s} t={t:+.3f} two-sided p (df=inf) = {2 * stats.norm.sf(abs(t)):.3f}")
print("\n== 3. t rebuilt from Table I: mean % change / (SD / sqrt n) ==")
for i, (lab, (m, sd, n)) in enumerate(TABLE_I.items()):
t = m / (sd / np.sqrt(n))
p2 = p_two(t, n - 1)
# rounding envelope: mean +/- 0.005, SD +/- 0.005
ts = [(m + dm) / ((sd + ds) / np.sqrt(n)) for dm in (-0.005, 0.005) for ds in (-0.005, 0.005)]
print(f"{lab:27s} t={t:+.3f} (rounding range {min(ts):+.3f} to {max(ts):+.3f}) df={n - 1:2d} "
f"two-sided p={p2:.3f} reported t={REPORTED[lab][0]:+.3f} "
f"|reported - Table I| = {abs(REPORTED[lab][0] - t):.3f} "
f"inside rounding range: {min(ts) <= REPORTED[lab][0] <= max(ts)}")
rows[i].append(t)
rows[i].append(p2)
for lab, (t, n, _) in REPORTED.items():
m_impl = t * TABLE_I[lab][1] / np.sqrt(n)
print(f" mean % change implied by reported t and Table I SD, {lab}: {m_impl:+.3f} "
f"(Table I: {TABLE_I[lab][0]:+.2f})")
print("\n== 4. Table I internal consistency (confirms n = 9 and 13) ==")
for meth in TABLE_I_ALL:
mA, sA, _ = TABLE_I[f"{meth}, AD"]
mN, sN, _ = TABLE_I[f"{meth}, controls"]
mall_rep, sdall_rep, gs_rep = TABLE_I_ALL[meth]
mall = (N_AD * mA + N_NC * mN) / (N_AD + N_NC)
ss = (N_AD - 1) * sA**2 + (N_NC - 1) * sN**2 + N_AD * (mA - mall)**2 + N_NC * (mN - mall)**2
sdall = np.sqrt(ss / (N_AD + N_NC - 1))
gs = (mN - mA) / sdall_rep
print(f"{meth:16s} mean all={mall:+.3f} (rep {mall_rep:+.2f}) SD all={sdall:.3f} (rep {sdall_rep}) "
f"group separation={gs:.3f} (rep {gs_rep})")
print("\n== 5. Exact Wilcoxon signed-rank: smallest attainable two-sided p ==")
for n in (N_AD, N_NC):
print(f"n={n}: minimum two-sided p = 2/2^{n} = {2 / 2**n:.4f}")
print("\n== 6. Para 5 p-values vs para 6 baseline/follow-up correlation p-values ==")
for lab, (_, _, prep) in REPORTED.items():
print(f"{lab:27s} para 5 (t-test) p={prep} para 6 (correlation) p={CORR_P_PARA5[lab]} "
f"identical={prep == CORR_P_PARA5[lab]}")
for n, p in [(N_AD, 0.001), (N_NC, 0.009), (N_NC, 0.001)]:
tc = t_needed(p, n - 2)
print(f" correlation r giving two-sided p={p} at n={n}: r = {tc / np.sqrt(tc**2 + n - 2):.3f}")
print("\n== 7. Side check: Pearson r p-values for agreement of change (Sec IV.B para 7) ==")
for lab, (r, n, prep) in AGREEMENT.items():
print(f"{lab:13s} r={r} n={n} two-sided p={r_pvalue(r, n):.4f} one-sided p={r_pvalue(r, n, 1):.4f} "
f"reported p={prep}")
make_figure(rows)
make_figure2(rows)
def make_figure(rows):
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC})
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
y = np.arange(len(rows))[::-1]
for yi, (lab, t, df, prep, p2, t_tab, p_tab) in zip(y, rows):
lo, hi = min(prep, p2, p_tab), max(prep, p2, p_tab)
ax.plot([lo, hi], [yi, yi], color="#b9b7b1", lw=2, zorder=1)
ax.plot([min(p2, p_tab), max(p2, p_tab)], [yi, yi], color=BLUE, lw=2, zorder=2)
ax.scatter(prep, yi, s=80, color=ORANGE, zorder=3, edgecolor=ORANGE)
ax.scatter(p_tab, yi, s=190, facecolor=BG, edgecolor=BLUE, lw=2, zorder=4)
ax.scatter(p2, yi, s=70, color=BLUE, zorder=5, edgecolor=BLUE)
ax.annotate(f"reported\np = {prep:g}", (prep, yi), xytext=(0, 9), textcoords="offset points",
ha="center", va="bottom", fontsize=10, color=ORANGE, linespacing=0.95)
ax.annotate(f"{p2:.3f} from t\n{p_tab:.3f} from Table I", (max(p2, p_tab), yi), xytext=(11, 0),
textcoords="offset points", ha="left", va="center", fontsize=10, color=BLUE,
linespacing=1.0)
ax.set_yticks(y)
ax.set_yticklabels([f"{r[0]}\nt = {r[1]:+.3f}, df = {r[2]}" for r in rows], fontsize=10, color=TXT)
ax.set_xscale("log")
ax.set_xlim(3e-4, 25)
ax.set_ylim(-0.6, len(rows) - 0.25)
ax.axvline(0.05, color=SEC, lw=1.0, zorder=0)
ax.text(0.05, len(rows) - 0.35, "p = 0.05 ", ha="right", va="top", fontsize=10, color=SEC)
ax.set_xticks([0.001, 0.01, 0.05, 0.1, 1])
ax.set_xticklabels(["0.001", "0.01", "0.05", "0.1", "1"])
ax.spines["bottom"].set_bounds(3e-4, 1)
minor = [m * 10.0**e for e in (-4, -3, -2, -1) for m in range(2, 10) if 3e-4 <= m * 10.0**e < 1]
ax.xaxis.set_minor_locator(matplotlib.ticker.FixedLocator(minor))
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlabel("Two-sided p-value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
handles = [
Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported p (Sec IV.B)"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed from reported t"),
Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=BLUE, mew=2,
label="Recomputed from Table I mean, SD, n"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.42, -0.2), ncol=2,
frameon=False, fontsize=10, handletextpad=0.3, columnspacing=1.2)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg, fig1.png")
def make_figure2(rows):
"""Two text panels: para 5 t-tests and para 6 correlations share the same four p-values."""
from matplotlib.patches import FancyBboxPatch
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, EDGE = "#eb6834", "#2a78d6", "#cfcdc8"
fig = plt.figure(figsize=(7.2, 4.1), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.1)
ax.axis("off")
ax.text(3.6, 3.85, "The four p-values of the t-tests (paragraph 5) repeat, in the same order,\n"
"the four p-values of the baseline/follow-up correlations (paragraph 6)",
ha="center", va="center", fontsize=10.5, color=TXT, style="italic", linespacing=1.15)
for x0, w in ((0.1, 3.9), (4.4, 2.7)):
ax.add_patch(FancyBboxPatch((x0, 0.55), w, 2.85, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=EDGE, lw=1.2))
ax.text(0.25, 3.2, "Sec IV.B, paragraph 5", fontsize=12, weight="bold", color=TXT, va="center")
ax.text(0.25, 2.93, "paired t-tests of one-year change", fontsize=10.5, color=SEC, va="center")
ax.text(4.55, 3.2, "Sec IV.B, paragraph 6", fontsize=12, weight="bold", color=TXT, va="center")
ax.text(4.55, 2.93, "baseline vs follow-up correlation", fontsize=10.5, color=SEC, va="center")
for x, lab, ha in ((0.25, "group", "left"), (2.35, "t", "right"), (2.55, "reported p \u2192\nrecomputed p", "left"),
(4.55, "group", "left"), (6.15, "reported p", "left")):
ax.text(x, 2.6, lab, fontsize=10, color=SEC, ha=ha, va="center")
for i, (lab, t, df, prep, p2, *_) in enumerate(rows):
yy = 2.2 - i * 0.43
short = lab.replace("Proposed method", "Proposed")
ax.text(0.25, yy, short, fontsize=10.5, color=TXT, va="center")
ax.text(2.35, yy, f"{t:+.3f}", fontsize=10.5, color=TXT, ha="right", va="center")
ax.text(2.55, yy, f"{prep:g}", fontsize=10.5, color=ORANGE, weight="bold", va="center")
ax.text(3.08, yy, f"\u2192 {p2:.3f}", fontsize=10.5, color=BLUE, weight="bold", va="center")
ax.text(4.55, yy, short, fontsize=10.5, color=TXT, va="center")
ax.text(6.15, yy, f"{CORR_P_PARA5[lab]:g}", fontsize=10.5, color=ORANGE, weight="bold", va="center")
ax.plot([4.0, 4.4], [yy, yy], color=SEC, lw=1.2, ls=(0, (2, 2)))
ax.text(0.1, 0.3, "orange = p as printed in the paper", fontsize=10, color=ORANGE, va="center")
ax.text(0.1, 0.08, "blue = two-sided p for the printed t (df = 8 for AD, 12 for controls), recomputed (scipy)",
fontsize=10, color=BLUE, va="center")
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
print("wrote fig2.svg, fig2.png")
if __name__ == "__main__":
main()
Abstract reports reliable cannabis-related subcortical reductions that the paper's own FDR-corrected Table 3 does not support
Hum Brain Mapp (2020) · doi:10.1002/hbm.25114 · PMC8675406
Summary
This ENIGMA-Addiction meta-analysis compares subcortical volumes and cortical thickness in alcohol use disorder (AUD) and cannabis use disorder (CUD) with published results for other psychiatric disorders. The Abstract states that the CUD meta-analysis "indicated reliable reductions in amygdala, accumbens and hippocampus volumes". The paper's own Table 3 gives FDR-adjusted p-values of .1608, .3623 and .1608 for these three regions. The Methods set significance at pFDR < .05, and the accumbens is not significant even before correction (p = .207; 95% CI −0.3787 to 0.0821). The Results, the Figure 2 caption and the Conclusion all say that no CUD subcortical difference survived correction. Recomputing the p-values from the table's effect sizes and standard errors, and applying Benjamini–Hochberg across the seven regions, reproduces the table. So the table is internally consistent, and the Abstract's CUD sentence is the part that disagrees with it. The paper's AUD findings are not affected.
What the paper reports
Abstract:
Meta-analysis of CUD case–control studies indicated reliable reductions in amygdala, accumbens and hippocampus volumes, with the former effect size comparable to, and the latter effect size around half of that reported for alcohol and SCZ.
Abstract, the next sentence, on CUD cortical thickness:
CUD was associated with lower cortical thickness in the frontal regions, particularly the medial orbitofrontal region, but this effect was not significant after correcting for multiple testing.
Abstract, last sentence:
Although modest, CUD results overlapped with findings reported for AUD and other psychiatric conditions, but appear to be most robustly related to reduce thickness of the medial orbitofrontal cortex.
Methods, "Statistical framework of meta-analysis":
Significance level was determined with a p FDR < .05 for all regions of interest.
Results, second paragraph (in full):
Figure 2 presents the case control comparisons for CUD, showing non-significant case–control differences after correcting for multiple comparisons.
Results, Figure 3 paragraph:
The effects reported for CUD in the amygdala and accumbens appear comparable to those reported for AUD and SCZ, but due to large CIs, these effects were not shown to significantly differ from non-effect line and the other disorders.
Results, Figure 4 paragraph (how the paper itself uses "reliable"):
However, the temporal pole for CUD did not survive FDR correction, so it is not a reliable finding.
Figure 2 caption (CUD subcortical forest plot):
All results are non-significant following false discovery rate correction.
Conclusion and Context:
No significant subcortical volume or cortical thickness variation was observed in cannabis-dependent subjects.
The absence of significant variations that survived FDR correction in CUD participants is interesting given the debate on the potential neurotoxic impact of cannabis dependence on the hippocampus and the medial orbitofrontal cortex
Table 3 (CUD vs controls), subcortical rows named in the Abstract, as printed (ROI | ES | SE | 95% CI.LB | 95% CI.UB | I² | p | p FDR):
Hippocampus | −0.2416 | 0.1071 | −0.4515 | −0.0318 | 0.00 | .024 | .1608
Amygdala | −0.2638 | 0.1322 | −0.5229 | −0.0048 | 29.6 | .0459 | .1608
Accumbens | −0.1483 | 0.1175 | −0.3787 | 0.0821 | 14.28 | .207 | .3623
Table 3 footer (excerpt):
p FDR, adjusted p-value for the seven sites following a BH correction
The footer also says "PFDR less than 0.05 are bolded". According to the version-of-record check (see Status), no cell in the CUD table is bold.
What the paper's own numbers show
Method. The paper reports random-effects meta-analytic Cohen's d (ES) with its standard error (SE), a 95% CI, an uncorrected p and a Benjamini–Hochberg (BH) adjusted pFDR for each region. For each of the seven CUD subcortical regions in Table 3, I computed z = ES/SE, the two-sided normal p = 2·(1 − Φ(|z|)) and the Wald interval ES ± 1.96·SE. I then applied BH across the seven reported uncorrected p-values, as the table footer describes. As controls for the method, I applied the same BH step to the seven AUD subcortical p-values in Table 2 and to the 34 CUD cortical p-values in Table 3. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CUD hippocampus, uncorrected p | Table 3 | .024 | z = −2.26, p = 0.0241 |
| CUD hippocampus, pFDR | Table 3 | .1608 | BH = 0.1607 (not < .05) |
| CUD amygdala, uncorrected p | Table 3 | .0459 | z = −2.00, p = 0.0460 |
| CUD amygdala, pFDR | Table 3 | .1608 | BH = 0.1607 (not < .05) |
| CUD accumbens, uncorrected p | Table 3 | .207 | z = −1.26, p = 0.2069 (not < .05) |
| CUD accumbens, 95% CI | Table 3 | [−0.3787, 0.0821] | [−0.3786, 0.0820]; includes 0 |
| CUD accumbens, pFDR | Table 3 | .3623 | BH = 0.3622 (not < .05) |
| Smallest CUD subcortical pFDR | Table 3 | .1608 | 0.1607; no region has pFDR < .05 |
| Uncorrected p the top-ranked region would need for pFDR < .05 | derived | .024 (hippocampus) | p < 0.0071 |
| CUD subcortical findings | Abstract | "reliable reductions in amygdala, accumbens and hippocampus volumes" | none survives FDR; accumbens is not significant uncorrected either |
| CUD/AUD effect-size ratio: amygdala, hippocampus | Abstract ("former … comparable to, latter … around half") | — | 0.67 and 0.48 (point estimates only) |
| CUD medial orbitofrontal thickness, pFDR | Table 3; Abstract last sentence | .0672 | BH over 34 cortical ROIs = 0.0680 (p printed to 3 decimals); not < .05 |
| Control: AUD subcortical BH | Table 2 | .0279, .6844, .0105, .1961, .0010, .0031, .0031 | 0.0279, 0.6844, 0.0105, 0.1961, 0.0007, 0.0030, 0.0030 (hippocampus p is printed as .0001; p = .00014 gives 0.0010) |
| Control: CUD cortical BH, 34 ROIs | Table 3 | — | largest difference from the reported pFDR 0.0065, explained by 3-decimal p-values |
| Side finding: CUD thalamus 95% CI upper bound | Table 3 | 0.0821 | ES + 1.96·SE = 0.2993; 0.0821 is the accumbens upper bound |
The recomputed p-values and intervals match Table 3 to the fourth decimal, and BH across the seven regions reproduces the reported pFDR. The same BH step reproduces the AUD pFDR values in Table 2, where five regions are significant. The CUD table is therefore consistent with the paper's stated method, and under that method no CUD subcortical region is significant. Two regions, hippocampus and amygdala, are nominally significant before correction. The accumbens is not significant at any level.
Figure
Figure 1. Left: the Abstract's CUD subcortical claim, the next Abstract sentence (which applies the multiple-testing standard to CUD cortex) and the Methods' significance criterion. Right: the Table 3 uncorrected p and pFDR for the three regions the Abstract names, with the reported pFDR in orange and the BH value recomputed from the seven reported p-values in blue, and the Figure 2 caption. All excerpts are verbatim. All three pFDR values are above the paper's .05 criterion, and the accumbens p = .207 is above .05 before any correction.
Figure 2. For each CUD subcortical region in Table 3, the open orange circle is the reported uncorrected p and the filled orange circle the reported pFDR, joined by a gray segment. Blue crosses are p-values recomputed from ES/SE, and blue diamonds are BH values recomputed across the seven regions. They sit on top of the reported values. The axis is logarithmic, and the dashed line marks 0.05, the paper's pFDR criterion. The three regions named in the Abstract are marked with an asterisk. Every filled marker lies to the right of the line, and the accumbens lies to the right of it even before correction.
Why it matters
The Abstract is the part of the paper most often read, indexed and cited. Its CUD sentence presents three subcortical volume reductions as "reliable" and goes on to scale them against alcohol and schizophrenia ("comparable to", "around half of"), which reads as a description of established effects. Under the paper's own criterion, which is stated in the Methods and applied throughout the body, none of these effects is established. The amygdala and hippocampus are nominally significant before correction. The accumbens is not significant even then, and its interval includes zero. A reader who relied on the Abstract would conclude that this ENIGMA meta-analysis found cannabis-related subcortical volume loss. The full text concludes the opposite: "No significant subcortical volume or cortical thickness variation was observed in cannabis-dependent subjects."
The Abstract's closing sentence has a related, milder problem. It says that CUD results "appear to be most robustly related to reduce thickness of the medial orbitofrontal cortex". The medial orbitofrontal pFDR is .0672, and the preceding Abstract sentence itself states that this effect was not significant after correction. "Most robustly" is therefore true only as a ranking of the CUD cortical effects, none of which is significant.
The direction of the point estimates (smaller volumes in CUD for hippocampus, amygdala and accumbens) is unaffected, and so are the effect-size ratios against AUD as point estimates. What changes is the strength of evidence: these are imprecise estimates consistent with no difference after correction, not reliable reductions. All AUD results are unaffected, as are the comparisons of AUD with schizophrenia, major depression and bipolar disorder, which form the paper's main conclusion.
Explanations considered
- Table 3 is wrong and the Abstract is right. This does not hold. The uncorrected p-values recomputed from ES/SE match the table to four decimals. BH across the seven regions reproduces the reported pFDR (0.1607, 0.1607, 0.3622 vs .1608, .1608, .3623). The same procedure reproduces the AUD table, so the method is the one the paper used. The intervals match ES ± 1.96·SE, except for one thalamus bound (see the last item). The Results, the Figure 2 caption and the Conclusion agree with the table.
- "Reliable" means nominally significant (uncorrected p < .05, or a CI that excludes zero). This would cover the hippocampus (p = .024) and the amygdala (p = .046). It does not cover the accumbens: p = .207, and the CI [−0.3787, 0.0821] includes zero. The paper also uses "reliable" to mean surviving FDR correction ("the temporal pole for CUD did not survive FDR correction, so it is not a reliable finding"). The Methods define significance as pFDR < .05, and the next Abstract sentence applies the correction standard to CUD cortex. So this reading does not account for the Abstract's three-region claim.
- The Abstract describes a different analysis, such as the sensitivity analysis without the adolescent sites. This is not supported. The Methods say that the Results report the analysis with the adolescent sites included. The supplement (Data S1) contains only Supplementary Figures 1–5, and none of them reports a CUD subcortical analysis in which amygdala, accumbens or hippocampus survives FDR correction (see Status). Supplementary Figure 4 is a leave-site-out analysis of the CUD caudate, not of the three regions in the Abstract.
- A different multiple-comparison family (for example, no correction across regions). The footer and Methods specify BH across the seven subcortical regions, and the AUD results in the same Abstract are described as significant "After correcting for multiple comparisons". Even with no correction at all, the accumbens is not significant.
- A conversion or typesetting error in the archived text. This is excluded. The PMC text, the publisher-supplied JATS XML in Europe PMC and the PubMed abstract carry the same Abstract sentence and the same Table 3 values (see Status).
- A separate copy error in Table 3 (side finding, not the claim under review). The CUD thalamus row prints CI upper bound 0.0821, but ES −0.0425 + 1.96 × 0.1744 = 0.2993. The printed value equals the accumbens upper bound in the same table. The thalamus p (.8074) and lower bound (−0.3844) fit ES and SE, so this looks like a copied cell. It does not affect any significance call.
Status
- Version of record. The Wiley article pages (doi.org redirect, /doi/pdf and /doi/full-xml) returned HTTP 403 to automated access, and the PMC HTML page returned a reCAPTCHA. I used the publisher-supplied JATS XML in Europe PMC (fullTextXML for PMC8675406; Hum Brain Mapp 43(1):399–413; first published online 2020-07-09) and the PubMed record (PMID 32643841). Both carry the Abstract sentence quoted above, and the XML carries the same Table 3 values, the same Figure 2 caption and the same Conclusion. The Table 3 footer says that values with pFDR < 0.05 are bolded, and no cell in the CUD table is bold. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. Data S1 (HBM-43-399-s001.docx), from the Europe PMC supplementary-files endpoint, holds Supplementary Figures 1–5 and supplementary references. Figures 1–2 show cortical thickness with and without the adolescent sites; Figure 3 an AUD accumbens leave-site-out analysis; Figure 4 a CUD caudate leave-site-out analysis; Figure 5 CUD site variability in two cortical regions. None reports a CUD subcortical result that survives FDR correction. The Data Availability statement says no new data are shared, so there is no site-level data to reanalyse.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no notices. A web search for the DOI with "PubPeer", "erratum" or "correction" found nothing about this article. The PubPeer API refused automated access (HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC8675406; Europe PMC full-text XML and supplementary files for PMC8675406; PubMed 32643841; Crossref metadata for 10.1002/hbm.25114.
Suggested correction
- Revise the Abstract's CUD subcortical sentence to match Table 3 and the Conclusion. For example: "Meta-analysis of CUD case–control studies showed no subcortical volume differences that survived FDR correction; hippocampus and amygdala volumes were nominally smaller in CUD (uncorrected p = .024 and .046), with point estimates about half (hippocampus) and two-thirds (amygdala) of those for AUD."
- Remove the accumbens from any statement of CUD volume reduction, or describe it as a non-significant point estimate (d = −0.15, 95% CI −0.38 to 0.08, p = .207).
- Revise the Abstract's last sentence so that the medial orbitofrontal result is described as the largest but non-significant CUD cortical effect (pFDR = .0672), not as the most robust one.
- Correct the CUD thalamus 95% CI upper bound in Table 3 from 0.0821 to 0.2993 (ES + 1.96·SE), or to the value in the original analysis output.
Recomputation script (python recompute.py.txt)
"""Recompute the CUD subcortical statistics in Table 3 of PMC8675406
(Hum Brain Mapp 2020, doi:10.1002/hbm.25114) and draw the write-up figures.
Run: python recompute.py.txt
Every input number below is copied from Table 2 or Table 3 of the paper.
Figures are written next to this file.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Table 3 (CUD vs controls), subcortical rows: ES, SE, CI.LB, CI.UB, p, pFDR
T3_SUB = {
"Thalamus": (-0.0425, 0.1744, -0.3844, 0.0821, 0.8074, 0.8074),
"Caudate": (-0.1526, 0.1062, -0.3608, 0.0556, 0.1508, 0.3519),
"Putamen": ( 0.0349, 0.1062, -0.1732, 0.2431, 0.7421, 0.8074),
"Pallidum": ( 0.1049, 0.1333, -0.1563, 0.3662, 0.4312, 0.6037),
"Hippocampus": (-0.2416, 0.1071, -0.4515, -0.0318, 0.0240, 0.1608),
"Amygdala": (-0.2638, 0.1322, -0.5229, -0.0048, 0.0459, 0.1608),
"Accumbens": (-0.1483, 0.1175, -0.3787, 0.0821, 0.2070, 0.3623),
}
ABSTRACT_ROIS = ["Amygdala", "Accumbens", "Hippocampus"]
# Table 3 (CUD), cortical thickness rows: p, pFDR (34 ROIs)
T3_CORT = {
"Bankssts": (.259, .4797), "Caudalanteriorcingulate": (.952, .9783),
"Caudalmiddlefrontal": (.015, .1031), "Cuneus": (.325, .5017),
"Entorhinal": (.512, .6217), "Fusiform": (.277, .4797),
"Inferior parietal": (.173, .3667), "Inferior temporal": (.609, .6681),
"Isthmus cingulate": (.978, .9783), "Lateral occipital": (.43, .5727),
"Lateral orbitofrontal": (.12, .3667), "Lingual": (.165, .3667),
"Medial orbitofrontal": (.002, .0672), "Middle temporal": (.918, .9749),
"Parahippocampal": (.18, .3667), "Paracentral": (.037, .2075),
"Pars opercularis": (.549, .6436), "Pars orbitalis": (.382, .5414),
"Pars triangularis": (.093, .3221), "Pericalcarine": (.043, .2076),
"Postcentral": (.15, .3667), "Posterior cingulate": (.382, .5414),
"Precentral": (.013, .1031), "Precuneus": (.183, .3667),
"Rostralanteriorcingulate": (.456, .5738), "Rostral middle frontal": (.594, .6681),
"Superior frontal": (.129, .3667), "Superior parietal": (.095, .3221),
"Superior temporal": (.312, .5017), "Supramarginal": (.438, .5727),
"Frontal pole": (.282, .4797), "Temporal pole": (.006, .0955),
"Transverse temporal": (.061, .2580), "Insula": (.014, .1031),
}
# Table 2 (AUD vs controls), subcortical rows: p, pFDR (control for the BH method)
T2_SUB = {
"Thalamus": (.0199, .0279), "Caudate": (.6844, .6844), "Putamen": (.006, .0105),
"Pallidum": (.1681, .1961), "Hippocampus": (.0001, .0010),
"Amygdala": (.0012, .0031), "Accumbens": (.0013, .0031),
}
def bh(p):
"""Benjamini-Hochberg adjusted p-values."""
p = np.asarray(p, float)
m = len(p)
o = np.argsort(p)
q = p[o] * m / np.arange(1, m + 1)
q = np.minimum.accumulate(q[::-1])[::-1]
out = np.empty(m)
out[o] = np.minimum(q, 1)
return out
# ------------------------------------------------- 1. CUD subcortical rows
names = list(T3_SUB)
p_rep = np.array([T3_SUB[n][4] for n in names])
q_rep = np.array([T3_SUB[n][5] for n in names])
q_from_rep_p = bh(p_rep)
z = np.array([T3_SUB[n][0] / T3_SUB[n][1] for n in names])
p_z = 2 * norm.sf(np.abs(z))
q_from_p_z = bh(p_z)
print("1. Table 3, CUD subcortical volumes (7 ROIs)")
print(f"{'ROI':12s} {'ES':>8s} {'SE':>7s} {'z':>6s} {'p(z)':>7s} {'p rep':>7s} "
f"{'CI recomputed':>20s} {'CI reported':>20s} {'BH(p rep)':>9s} {'BH(p(z))':>9s} {'pFDR rep':>8s}")
rec = {}
for i, n in enumerate(names):
es, se, lb, ub, p, pf = T3_SUB[n]
clo, chi = es - 1.96 * se, es + 1.96 * se
rec[n] = dict(z=z[i], p=p_z[i], q=q_from_rep_p[i], q_z=q_from_p_z[i], lo=clo, hi=chi)
print(f"{n:12s} {es:8.4f} {se:7.4f} {z[i]:6.2f} {p_z[i]:7.4f} {p:7.4f} "
f"[{clo:8.4f},{chi:8.4f}] [{lb:8.4f},{ub:8.4f}] {q_from_rep_p[i]:9.4f} {q_from_p_z[i]:9.4f} {pf:8.4f}")
print(" Smallest reported pFDR:", q_rep.min(), "| smallest recomputed:", round(q_from_rep_p.min(), 4))
print(" Any CUD subcortical pFDR < .05 (reported):", bool((q_rep < .05).any()),
"| recomputed:", bool((q_from_rep_p < .05).any()))
print(" Uncorrected p < .05:", [n for n in names if T3_SUB[n][4] < .05])
for n in ABSTRACT_ROIS:
es, se, lb, ub, p, pf = T3_SUB[n]
print(f" {n}: CI [{lb}, {ub}] excludes 0: {ub < 0 or lb > 0}; p = {p}; pFDR = {pf}")
# How small would the p-value need to be for pFDR < .05?
# rank-1 ROI: p*7 < .05 -> p < .0071
print(f" For the smallest p to reach pFDR < .05 alone: p < {0.05/7:.4f}")
# Side finding: thalamus CI
es, se, lb, ub, *_ = T3_SUB["Thalamus"]
print(f" Side finding, thalamus: reported CI [{lb}, {ub}]; ES +/- 1.96 SE = "
f"[{es-1.96*se:.4f}, {es+1.96*se:.4f}]; reported upper bound equals accumbens upper bound: "
f"{ub == T3_SUB['Accumbens'][3]}")
print(f" Thalamus CI implied by reported lower bound and ES (symmetric): upper = {2*es-lb:.4f}")
# Effect-size ratios quoted in the abstract (CUD vs AUD)
aud_es = {"Hippocampus": -0.5037, "Amygdala": -0.3942, "Accumbens": -0.3044}
for n in ABSTRACT_ROIS:
print(f" {n}: CUD/AUD effect-size ratio = {T3_SUB[n][0]/aud_es[n]:.2f}")
# ------------------------------------------------- 2. CUD cortical rows
cn = list(T3_CORT)
cp = np.array([T3_CORT[k][0] for k in cn])
cq = np.array([T3_CORT[k][1] for k in cn])
cq_rec = bh(cp)
print("\n2. Table 3, CUD cortical thickness (34 ROIs): BH(p) vs reported pFDR")
print(f" max |BH - reported| = {np.max(np.abs(cq_rec - cq)):.4f}")
i = cn.index("Medial orbitofrontal")
print(f" Medial orbitofrontal: p = {cp[i]}, BH = {cq_rec[i]:.4f}, reported pFDR = {cq[i]}")
print(" Any CUD cortical pFDR < .05:", bool((cq < .05).any()))
# ------------------------------------------------- 3. AUD subcortical (control)
ap = np.array([v[0] for v in T2_SUB.values()])
aq = np.array([v[1] for v in T2_SUB.values()])
aq_rec = bh(ap)
print("\n3. Table 2, AUD subcortical volumes (control): BH(p) vs reported pFDR")
for n, a, b, c in zip(T2_SUB, ap, aq_rec, aq):
print(f" {n:12s} p = {a:<7} BH = {b:.4f} reported = {c:.4f} sig = {c < .05}")
# Hippocampus p printed as .0001 (rounded); BH depends on the unrounded value
print(" AUD hippocampus BH if p = .00014:", round(bh(np.where(ap == .0001, .00014, ap))[4], 4))
# ------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, stem):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{stem}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: abstract vs. the paper's own Table 3 / Figure 2 caption / Conclusion
fig = plt.figure(figsize=(7.2, 4.4), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.4); ax.axis("off")
def panel(x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec="#c9c7c1", lw=1.2))
ax.text(x + 0.15, y + h - 0.2, title, fontsize=11.5, weight="bold", va="top")
# left: Abstract
panel(0.15, 1.05, 2.75, 3.2, "Abstract")
ax.text(0.3, 3.65,
"\u201cMeta-analysis of CUD case\u2013\ncontrol studies indicated\n"
"reliable reductions in\namygdala, accumbens and\nhippocampus volumes\u201d",
fontsize=10.5, va="top", linespacing=1.35)
ax.text(0.3, 1.17, "Methods: \u201cSignificance level was\ndetermined with a p FDR < .05\n"
"for all regions of interest.\u201d", fontsize=10, color=INK2, va="bottom", linespacing=1.25)
# right: Table 3 and body text
panel(4.1, 1.05, 2.95, 3.2, "Table 3 (CUD) and text")
ax.text(4.25, 3.68, "Region", fontsize=10, color=INK2, va="top")
ax.text(5.6, 3.68, "p", fontsize=10, color=INK2, va="top", ha="right")
ax.text(6.25, 3.68, "pFDR", fontsize=10, color=INK2, va="top", ha="right")
ax.text(6.9, 3.68, "BH", fontsize=10, color=INK2, va="top", ha="right")
for k, n in enumerate(ABSTRACT_ROIS):
yy = 3.38 - 0.27 * k
ax.text(4.25, yy, n, fontsize=10.5, va="top")
ax.text(5.6, yy, f"{T3_SUB[n][4]:.4f}".rstrip("0").lstrip("0"), fontsize=10.5,
va="top", ha="right", color=INK)
ax.text(6.25, yy, f"{T3_SUB[n][5]:.4f}".lstrip("0"), fontsize=10.5, va="top", ha="right",
color=ORANGE, weight="bold")
ax.text(6.9, yy, f"{rec[n]['q']:.4f}".lstrip("0"), fontsize=10.5, va="top", ha="right",
color=BLUE, weight="bold")
ax.text(4.25, 2.5, "orange = reported pFDR; blue = BH\nrecomputed from the 7 reported p",
fontsize=10, color=INK2, va="top", linespacing=1.25)
ax.text(4.25, 1.95, "Fig. 2 caption: \u201cAll results are non-\nsignificant following false\n"
"discovery rate correction.\u201d", fontsize=10, va="top", linespacing=1.25)
ax.text(0.3, 2.56, "Next sentence, on CUD cortex:\n\u201cthis effect was not significant\n"
"after correcting for multiple\ntesting\u201d", fontsize=10, color=INK2,
va="top", linespacing=1.25)
# connector
arr = FancyArrowPatch((2.95, 2.95), (4.05, 2.95), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=INK)
ax.add_patch(arr)
ax.text(3.5, 3.12, "conflict", ha="center", fontsize=10.5, weight="bold")
ax.text(3.5, 2.78, "all three\npFDR > .05", ha="center", va="top", fontsize=10, color=INK2)
ax.text(0.15, 0.72,
"None of the three regions meets the paper's pFDR < .05 criterion; accumbens is not\n"
"significant even before correction (p = .207; 95% CI \u22120.379 to 0.082 includes 0).",
fontsize=10.5, va="top", linespacing=1.35)
save(fig, "fig1")
# Figure 2: uncorrected p and FDR-adjusted p for the 7 CUD subcortical ROIs
order = sorted(names, key=lambda n: T3_SUB[n][4], reverse=True)
fig, ax = plt.subplots(figsize=(7.2, 4.0), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xscale("log")
ax.set_xlim(0.01, 1.2)
ys = np.arange(len(order))
for y, n in zip(ys, order):
p, q = T3_SUB[n][4], T3_SUB[n][5]
ax.plot([p, q], [y, y], color="#b9b7b1", lw=2, zorder=1, solid_capstyle="round")
ax.plot(p, y, "o", ms=10, mfc=BG, mec=ORANGE, mew=2, zorder=2)
ax.plot(q, y, "o", ms=10, mfc=ORANGE, mec=ORANGE, zorder=2)
ax.plot(rec[n]["p"], y, "x", ms=8, mec=BLUE, mew=2, zorder=3)
ax.plot(rec[n]["q"], y, "D", ms=5, mfc=BLUE, mec=BLUE, zorder=3)
ax.axvline(0.05, color=INK2, lw=1, ls="--", zorder=0)
ax.text(0.05 * 0.93, 0.0, "0.05\n(pFDR\ncriterion)", ha="right",
va="center", fontsize=10, color=INK2)
ax.set_yticks(ys)
ax.set_yticklabels([n + (" *" if n in ABSTRACT_ROIS else "") for n in order], fontsize=10.5)
for lab in ax.get_yticklabels():
if lab.get_text().endswith("*"):
lab.set_color(INK); lab.set_fontweight("bold")
ax.set_xticks([0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1])
ax.set_xticklabels(["0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_xlabel("p-value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#b9b7b1")
ax.tick_params(axis="y", length=0)
ax.set_ylim(-0.6, len(order) - 0.3)
handles = [
Line2D([], [], ls="", marker="o", ms=9, mfc=BG, mec=ORANGE, mew=2, label="reported p"),
Line2D([], [], ls="", marker="o", ms=9, mfc=ORANGE, mec=ORANGE, label="reported pFDR"),
Line2D([], [], ls="", marker="x", ms=8, mec=BLUE, mew=2, label="recomputed p (z = ES/SE)"),
Line2D([], [], ls="", marker="D", ms=5, mfc=BLUE, mec=BLUE, label="recomputed BH pFDR"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, -0.17), ncol=2,
frameon=False, fontsize=10, handletextpad=0.4, columnspacing=1.6)
ax.set_title("CUD subcortical volumes, Table 3 (* = named in the Abstract as \u201creliable\u201d)",
fontsize=10.5, loc="left", color=INK)
fig.subplots_adjust(left=0.19, right=0.97, top=0.92, bottom=0.30)
save(fig, "fig2")
print("\nFigures written to", HERE)
Table 4 brain-volume T values contradict their p values and the table's own means and SDs
Hum Brain Mapp (2022) · doi:10.1002/hbm.25798 · PMC9057088
Summary
Table 4 compares regional brain volumes between 49 healthy children and 53 children with ADHD by a two-independent-samples t test, and in 9 of its 10 rows the printed T value implies a p at least 50 times smaller than the printed p (for whole brain, T = 9.872 sits with p = .995, although no t of that size can have p above .064 at any degrees of freedom). Recomputed from the table's own means and SDs, only the hippocampus row matches its T (7.50 vs 7.401); every other row gives |t| between 0.41 and 1.42, none significant. As a result, lower frontal-lobe volume in ADHD, stated in the Abstract (table p = .008), recomputes to t = 1.42, p = .16, and lower total brain volume, stated in the Results (p < .05), has p = .995 in the table itself and p = .68 on recomputation. The paper's main finding, lower brain iron measured by QSM, rests on other tables and is not affected, but its volume findings and the Discussion built on them are.
What the paper reports
Methods, "Patient information" (group sizes):
53 children with ADHD were included in the study
49 children were included in the study
Methods, "Statistical analysis":
Two independent samples t test was used to compare the brain volume of children between the study group and the control group (p < .05 means the difference is statistically significant).
Table 4, caption and selected rows. The columns are Control group | Study group | T value | p value:
Volume values of brain regions in children [ x ¯ ± s, volume (cm3)]
Frontal | 189.77 ± 27.65 | 182.56 ± 23.53 | 12.871 | .008
Temporal | 138.32 ± 15.25 | 140.47 ± 18.34 | 10.132 | .213
Hippocampus | 12.37 ± 1.28 | 10.24 ± 1.56 | 7.401 | .002
TH | 14.97 ± 2.56 | 14.59 ± 2.39 | 8.601 | .804
Whole brain | 1452.89 ± 179.43 | 1437.96 ± 187.31 | 9.872 | .995
Abstract:
The volume of frontal lobe and hippocampus of children with ADHD was lower than that of healthy children (p < .05).
Results, "Brain volume comparison results":
The brain areas of the frontal lobe and hippocampus of children with ADHD are lower than healthy children (p < .05); the total brain volume of children with ADHD is lower than that of healthy children (p < .05) (Table 4).
Discussion, where the volume results are interpreted:
children with ADHD only have abnormal brain areas such as the frontal lobe and hippocampus (lower than healthy children)
The number of brain areas with abnormal iron content is more than the number of brain areas with abnormal volume
suggesting that the reduced volume of frontal and hippocampal brain regions may be due to decreased iron content in brain regions
the decrease in the volume of the frontal and temporal lobes and other brain regions may be due to
What the paper's own numbers show
Method. For every Table 4 row, I recomputed the two-sample t test from the printed means and SDs with n = 49 (control) and n = 53 (ADHD), using scipy.stats.ttest_ind_from_stats, both pooled-variance (Student) and Welch. I then made three checks. (i) The two-sided p each reported T implies on 100 df, the df of the stated test. (ii) The largest p any reported T could have at all, which occurs at df = 1. (iii) The |t| each reported p implies. As a control, the same procedure applied to Table 1 (age, weight, BMI and serum iron, which the Methods say were tested with the same test on the same groups) reproduces the published T and p values: t = 0.259, −0.904, −0.486 and 0.676 against 0.255, −0.904, −0.486 and 0.676. This confirms the group sizes and that "±" is the SD. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Frontal volume, T / p | Table 4 | 12.871 / .008 | t = 1.422, p = .158 (Welch 1.413, p = .161). T = 12.871 implies p = 6.1 × 10⁻²³ |
| Temporal volume, T / p | Table 4 | 10.132 / .213 | t = −0.641, p = .523 (ADHD mean is larger). T = 10.132 implies p = 5.1 × 10⁻¹⁷ |
| Hippocampus volume, T / p | Table 4 | 7.401 / .002 | t = 7.503, p = 2.6 × 10⁻¹¹. T matches; T = 7.401 implies p = 4.3 × 10⁻¹¹, not .002 |
| Thalamus (TH) volume, T / p | Table 4 | 8.601 / .804 | t = 0.775, p = .440. T = 8.601 implies p = 1.1 × 10⁻¹³ |
| Globus pallidus (GP) volume, T / p | Table 4 | 1.354 / .193 | t = −1.194, p = .235. The only row where T and p agree with each other (T implies p = .179) |
| Substantia nigra (SN) volume, T / p | Table 4 | 4.457 / .643 | t = −1.158, p = .249. T = 4.457 implies p = 2.2 × 10⁻⁵ |
| Putamen (PU) volume, T / p | Table 4 | 3.871 / .109 | t = 1.248, p = .215. T = 3.871 implies p = 1.9 × 10⁻⁴ |
| Red nucleus (RN) volume, T / p | Table 4 | 7.422 / .127 | t = 1.128, p = .262. T = 7.422 implies p = 3.9 × 10⁻¹¹ |
| Caudate nucleus (CN) volume, T / p | Table 4 | 3.214 / .566 | t = 0.977, p = .331. T = 3.214 implies p = .0018 |
| Whole-brain volume, T / p | Table 4 | 9.872 / .995 | t = 0.410, p = .682 (difference 14.93 cm³, SE 36.3). T = 9.872 implies p = 1.9 × 10⁻¹⁶ |
| Largest p possible for |t| = 9.872 | Table 4, whole brain | .995 | .064, even at df = 1 |
| |t| implied by p = .995 on 100 df | Table 4, whole brain | 9.872 | 0.006 |
| Whole-brain effect size | Table 4 | — | Cohen's d = 0.081. Reaching t = 9.872 at this d would need about 29 466 children per group |
| Frontal volume significance | Abstract; Results; Table 4 | lower in ADHD, p < .05 (table p = .008) | not significant, p = .158 |
| Total brain volume significance | Results | lower in ADHD, p < .05 | table p = .995; recomputed p = .682 |
| Hippocampus volume significance | Abstract; Results; Table 4 | lower in ADHD, p < .05 | confirmed, p = 2.6 × 10⁻¹¹ |
| Control check, Table 1 t values | Table 1 | 0.255 / −0.904 / −0.486 / 0.676 | 0.259 / −0.904 / −0.486 / 0.676 |
Table 4 therefore has two separate problems. First, the T column and the p column cannot both be correct in 9 of 10 rows. Second, the T column does not follow from the printed means and SDs. In 8 of 10 rows the printed T is 3.1 to 24 times the recomputed |t|. GP is close in size (1.354 vs 1.194) but, like temporal and SN, is printed as positive although the control mean is the smaller one. Only hippocampus agrees (ratio 0.99). The p column is closer to the summary data (all rows except frontal and hippocampus are non-significant in both), but it still does not reproduce them. Frontal is the decisive row: p = .008 is printed, and p = .158 follows from the table's own data.
A related observation. Table 3 (left vs right iron content within the ADHD group, n = 53) shows the same mismatch between its T and p columns. For example, SN has T = 4.872 with p = .993, and no t statistic of 4.872 can have a p above .129. For six of its nine rows, the reported T is also larger than a paired t test on those means and SDs could produce at any left–right correlation. Its conclusion of no left–right difference agrees with the printed p values and with an unpaired recomputation (all |t| ≤ 0.72), so this does not change a finding. It does show that the T columns in more than one table do not correspond to the stated tests. Table 2 (ANCOVA for iron content) is internally consistent: its F values reproduce its p values on F(1, 99), for example F = 1.013 gives p = .317 and F = 0.056 gives p = .813.
Figure
Figure 1. For each Table 4 row, the orange circle is the printed T value. Table 4 prints all T values without a sign, so they are drawn as positive. The blue diamond is the t statistic recomputed from the same row's means and SDs, with n = 49 controls and 53 children with ADHD, using a pooled-variance two-sample t test; positive means the control group is larger. The shaded band is |t| < 1.98, the region where p > .05 on 100 df. The two markers are offset vertically so that they stay visible where they coincide. Only the hippocampus row agrees. In every other row except GP, the printed T lies far outside the band while the recomputed t lies inside it. That includes frontal, which the Abstract reports as significantly smaller in ADHD.
Figure 2. Three two-sided p values per row on one log axis. The orange circle is the p printed in Table 4. The filled blue diamond is p recomputed from the row's means and SDs. The hollow blue square is the p that the printed T value implies on 100 df. The thin vertical line marks p = .05. If the table were internally consistent, the three markers would coincide in every row. They do so only for GP. For whole brain, the printed T implies p = 1.9 × 10⁻¹⁶, while the printed p is .995 and the recomputed p is .68. For frontal, the printed p (.008) lies on the significant side of .05 and the recomputed p (.158) on the other.
Why it matters
The paper's primary result is lower regional brain iron on quantitative susceptibility mapping in children with ADHD (Tables 2 and 3, and the ROC analysis in Table 5). Table 4 does not bear on it, and this write-up does not question it.
Table 4 is, however, the only source of the paper's volume findings, and two of them do not survive recomputation.
- Frontal-lobe volume. The Abstract, the Results and the Discussion all state that frontal volume is lower in ADHD (p < .05). The printed means differ by 7.21 cm³ against a pooled SD of 25.59 cm³ (d = 0.28). This gives t = 1.42, p = .16, which is not significant at the paper's own threshold.
- Total brain volume. The Results state that total brain volume is lower in ADHD (p < .05) and cite Table 4. The table itself prints p = .995 for whole brain, and recomputation gives p = .68 for a 1% difference.
- The Discussion's interpretation. The Discussion proposes that reduced iron delays development and so reduces the volume of the frontal (and temporal) lobes and hippocampus. It also contrasts the number of regions with abnormal iron against the number with abnormal volume. On the table's own data, only the hippocampal volume difference holds (t = 7.50). The frontal and whole-brain volume differences, and the temporal difference mentioned in the Discussion, have no support in Table 4.
Because the T column does not follow from the summary data in 9 of 10 rows, a reader also cannot use Table 4's T values for meta-analysis or to judge effect sizes. The means and SDs, which reproduce the Table 1 pattern, are the more usable part of the table, assuming they are correct.
Explanations considered
- The "±" values are standard errors, not SDs. This does not explain the table. The caption gives "x̄ ± s", and the Methods say data are shown as mean ± s. Treating them as SEMs would make every recomputed t smaller (whole brain t = 0.058), not larger. The Table 1 check reproduces only with "±" read as the SD.
- Different group sizes. This does not explain it. Table 1 reproduces exactly with n = 49 and 53. At the whole-brain effect size (d = 0.081), a t of 9.872 would need about 29 466 children per group.
- Welch (unequal-variance) t test. This does not explain it. Welch t values differ from the pooled values only in the second or third decimal (whole brain 0.411, frontal 1.413).
- An adjusted analysis, such as one for age, sex or intracranial volume. The Methods name covariate adjustment (ANCOVA for age) only for iron content, and they describe the volume comparison as a two-independent-samples t test. An adjusted model could in principle give a frontal p different from .16. It cannot make a test statistic of 9.872 sit with p = .995, or 10.132 with p = .213, under any t distribution: the largest p possible for |t| = 9.872 is .064, at df = 1. So an adjustment could explain at most one of the two columns, and the paper does not describe one.
- The T column holds a different statistic, for example Levene's F for equal variances, which SPSS prints next to the t test. Read as F(1, 100), the printed values give p = .0022 for whole brain, .0019 for temporal and .0042 for TH. These do not match the printed p values either, and a variance test on SDs as similar as 179.43 and 187.31 (variance ratio 1.09) would be unlikely to give F = 9.872. This reading does not account for the numbers.
- Rows or columns shifted during typesetting. No reordering of the printed values can match the recomputed ones. Nine recomputed |t| values lie between 0.41 and 1.42, but only one printed T (1.354) is below 3.2. Likewise, no printed p is near the recomputed hippocampus p of 2.6 × 10⁻¹¹ (the smallest printed p is .002). The PMC rendering and the Europe PMC XML have the same cells in the same order. The cells are plain text, with no lost minus signs, superscripts or decimal marks. So a conversion artifact is unlikely. The publisher's own HTML and PDF could not be opened (see Status).
- A typographical error in a single cell. This does not fit the pattern: 9 of 10 rows are affected, and the same T/p mismatch also appears in Table 3.
None of these explanations accounts for the table. The published material does not show how the T and p columns were produced.
Status
- Version of record. Wiley's article pages (onlinelibrary.wiley.com, /doi/full and /doi/pdfdirect) and the PMC PDF returned HTTP 403 or a bot challenge to automated access, so they were not read. I checked the publisher-deposited full text in two places, the PMC article (https://pmc.ncbi.nlm.nih.gov/articles/PMC9057088/) and the Europe PMC full-text XML (PMID 35107194). Table 4 is identical in both. Every quotation above was confirmed verbatim against the PMC-archived text.
- Supplements and data. The article has no supplementary-material files. The Data Availability Statement links to a private figshare item (https://figshare.com/s/1c22e32887bff32467eb). When it was opened in a browser, it was titled "ADHD DATA" and showed "No file added yet". No individual-level data were available to check Table 4 against.
- Corrections and comments. As of 2026-10-23, I found no correction, erratum, expression of concern or retraction. The Crossref work record has no update-to or updated-by entries. The Crossref filter
updates:10.1002/hbm.25798returns 0 items (re-run on 2026-10-23), and Crossref includes the Retraction Watch database. PMC lists no related notice. The Europe PMC record has no comment/correction list. Earlier web searches found no PubPeer thread or published comment on the article. On 2026-10-23, PubPeer's search page returned HTTP 403 to automated access and the web-search quota was exhausted, so a PubPeer thread could not be ruled out directly. - Sources. PMC full text (PMC9057088); Europe PMC REST API; Crossref REST API (api.crossref.org/works/10.1002/hbm.25798); the figshare link given in the article.
Suggested correction
- Re-run the volume comparisons from the source data with the test stated in the Methods, and replace the T and p columns of Table 4. If the analysis was adjusted (for example for age, sex or intracranial volume), say so in the Methods and the table footnote, and report the matching statistic and degrees of freedom.
- If the printed means and SDs are correct, revise the Abstract, the Results ("Brain volume comparison results") and the Discussion: lower frontal-lobe volume (recomputed p = .16) and lower total brain volume (table p = .995, recomputed p = .68) are not supported, and lower hippocampal volume is. Remove or qualify the Discussion passages that attribute reduced frontal and temporal volume to reduced iron.
- Check Table 3 the same way. Its T values are inconsistent with its p values, and in six rows with any paired test on the printed means and SDs. Report signed t statistics in Tables 3 and 4, as Table 1 does.
- Deposit the data in the figshare record named in the Data Availability Statement, which at the time of checking contained no files.
Recomputation script (python recompute.py.txt)
"""Recompute Table 4 (regional brain volumes) of PMC9057088
(Hum Brain Mapp 2022, doi:10.1002/hbm.25798).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the article as archived in PubMed Central:
Methods: 53 children with ADHD (study group), 49 healthy controls; brain volume
compared with a "Two independent samples t test"; data shown as mean ± s.
Table 1: patient information (used as a control check of n and the SD convention)
Table 2: iron content, ANCOVA F and p (checked for internal consistency only)
Table 3: left vs right iron content, ADHD group (T and p columns)
Table 4: volume of brain regions, mean ± s (cm3), T value, p value
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
N_C, N_S = 49, 53 # controls, ADHD
DF = N_C + N_S - 2 # 100
# ------------------------------------------------------------------ Table 4 (as printed)
# region, control mean, control s, study mean, study s, reported T, reported p
T4 = [
("Frontal", 189.77, 27.65, 182.56, 23.53, 12.871, .008),
("Temporal", 138.32, 15.25, 140.47, 18.34, 10.132, .213),
("Hippocampus", 12.37, 1.28, 10.24, 1.56, 7.401, .002),
("TH", 14.97, 2.56, 14.59, 2.39, 8.601, .804),
("GP", 3.62, 0.97, 3.87, 1.13, 1.354, .193),
("SN", 0.94, 0.16, 0.99, 0.26, 4.457, .643),
("PU", 16.97, 2.89, 16.18, 3.45, 3.871, .109),
("RN", 0.72, 0.22, 0.68, 0.13, 7.422, .127),
("CN", 9.26, 1.82, 8.96, 1.25, 3.214, .566),
("Whole brain", 1452.89, 179.43, 1437.96, 187.31, 9.872, .995),
]
# ------------------------------------------------------------------ Table 1 (control check)
T1 = [ # label, ctrl mean, ctrl s, study mean, study s, reported T, reported p
("Age", 8.56, 2.54, 8.44, 2.13, 0.255, .800),
("Weight", 28.15, 5.93, 29.23, 6.12, -0.904, .368),
("BMI", 17.19, 1.87, 17.38, 2.06, -0.486, .628),
("Trace iron", 7.38, 1.13, 7.24, 0.96, 0.676, .501),
]
# ------------------------------------------------------------------ Table 2 (F, p)
T2 = [("Frontal", 24.928, "<.001"), ("Hippocampus", 84.683, "<.001"), ("CN", 87.182, "<.001"),
("PU", 39.943, "<.001"), ("GP", 27.913, "<.001"), ("TH", 0.056, ".813"),
("Temporal", 1.013, ".317"), ("SN", 28.357, "<.001"), ("RN", 3.504, ".064")]
# ------------------------------------------------------------------ Table 3 (T, p)
T3 = [ # region, left mean, left s, right mean, right s, reported T, reported p
("Frontal", 60.18, 13.24, 61.93, 11.67, 1.823, .984),
("Temporal", 93.35, 16.41, 94.58, 14.27, 5.257, .092),
("Hippocampus", 55.27, 18.32, 56.89, 14.64, 2.874, .143),
("TH", 179.23, 23.43, 178.26, 32.54, 3.547, .752),
("GP", 393.56, 78.89, 392.47, 89.56, 1.168, .247),
("SN", 175.13, 38.03, 176.98, 45.21, 4.872, .993),
("PU", 57.65, 13.26, 56.18, 12.22, 2.146, .109),
("RN", 114.78, 26.67, 113.89, 33.78, 3.428, .136),
("CN", 321.89, 79.27, 322.96, 87.31, 1.224, .143),
]
def p2(t, df):
"""two-sided p for a t statistic"""
return 2 * stats.t.sf(abs(t), df)
print("=== 0. Control check: Table 1 reproduces with n = 49 / 53 and '±' = SD ===")
for lab, mc, sc, ms, ss, T, p in T1:
t, pp = stats.ttest_ind_from_stats(mc, sc, N_C, ms, ss, N_S, equal_var=True)
print(f" {lab:10s} recomputed t = {t:6.3f}, p = {pp:.3f} | reported T = {T:6.3f}, p = {p:.3f}")
print("\n=== 1. Table 4 recomputed from its own means and SDs (two independent samples t test) ===")
hdr = (f" {'region':12s} {'diff':>8s} {'SE':>7s} {'t_pool':>7s} {'p_pool':>8s} {'t_welch':>7s} "
f"{'p_welch':>8s} | {'T_rep':>7s} {'p_rep':>6s} {'p(T_rep|df=100)':>15s} {'p(T_rep|df=1)':>13s} "
f"{'|t| implied by p_rep':>20s}")
print(hdr)
R = []
for r, mc, sc, ms, ss, T, p in T4:
tp, pp = stats.ttest_ind_from_stats(mc, sc, N_C, ms, ss, N_S, equal_var=True)
tw, pw = stats.ttest_ind_from_stats(mc, sc, N_C, ms, ss, N_S, equal_var=False)
se = np.sqrt(sc**2 / N_C + ss**2 / N_S)
pT = p2(T, DF)
pT1 = p2(T, 1)
t_imp = stats.t.isf(p / 2, DF)
R.append(dict(region=r, diff=mc - ms, se=se, t=tp, p=pp, tw=tw, pw=pw, T=T, prep=p, pT=pT,
t_imp=t_imp))
print(f" {r:12s} {mc - ms:8.2f} {se:7.3f} {tp:7.3f} {pp:8.4f} {tw:7.3f} {pw:8.4f} | "
f"{T:7.3f} {p:6.3f} {pT:15.2e} {pT1:13.3f} {t_imp:20.3f}")
print(" (diff = control minus ADHD; SE = unpooled standard error of the difference)")
print(f" Hippocampus recomputed p = {R[2]['p']:.1e}")
print("\n=== 2. Rows where reported T and p contradict each other (df = 100) ===")
for x in R:
flag = "consistent" if abs(np.log10(x["pT"]) - np.log10(x["prep"])) < 0.3 else "INCONSISTENT"
print(f" {x['region']:12s} T={x['T']:6.3f} -> p={x['pT']:.2e}; reported p={x['prep']:.3f} {flag}")
print(" Largest two-sided p any |t| >= 9.872 can have, at the smallest possible df (1): "
f"{p2(9.872, 1):.3f}")
print(" Largest two-sided p for |t| = 4.457 (SN) at df = 1: "
f"{p2(4.457, 1):.3f}")
print("\n=== 3. Significance at alpha = .05: reported vs recomputed ===")
for x in R:
print(f" {x['region']:12s} reported p={x['prep']:.3f} ({'sig' if x['prep'] < .05 else 'ns'}) "
f"recomputed p={x['p']:.3g} ({'sig' if x['p'] < .05 else 'ns'})")
print("\n=== 4. Alternative readings ===")
wb = R[-1]
mc, sc, ms, ss = 1452.89, 179.43, 1437.96, 187.31
print(f" If '±' were SEM (whole brain): t = {(mc - ms) / np.hypot(sc, ss):.4f}")
sp = np.sqrt(((N_C - 1) * sc**2 + (N_S - 1) * ss**2) / DF)
d = (mc - ms) / sp
print(f" Whole brain Cohen's d = {d:.4f}; equal n per group needed for t = 9.872 at this d: "
f"~{2 * (9.872 / d) ** 2:.0f}")
fr = T4[0]
sp_f = np.sqrt(((N_C - 1) * fr[2]**2 + (N_S - 1) * fr[4]**2) / DF)
print(f" Frontal Cohen's d = {(fr[1] - fr[3]) / sp_f:.3f} (pooled SD {sp_f:.2f} cm3)")
print(f" Whole-brain difference as % of control mean: {100 * (mc - ms) / mc:.2f}%")
print(f" Variance-ratio F for whole-brain SDs: {(ss / sc) ** 2:.3f}")
print(" Ratio reported T / recomputed |t|:")
for x in R:
print(f" {x['region']:12s} {x['T'] / abs(x['t']):6.2f} (sign of recomputed t: {'+' if x['t'] > 0 else '-'})")
print(" Could any reordering of the printed columns match? For each recomputed value, "
"printed values within 15%:")
for x in R:
tm = [y["region"] for y in R if abs(y["T"] - abs(x["t"])) <= 0.15 * abs(x["t"])]
pm = [y["region"] for y in R if abs(y["prep"] - x["p"]) <= 0.15 * x["p"]]
print(f" {x['region']:12s} T from rows {tm or '-'}; p from rows {pm or '-'}")
print(" Reported T read as F(1,100) (e.g. a variance test), p:")
for x in R:
print(f" {x['region']:12s} F={x['T']:6.3f} -> p={stats.f.sf(x['T'], 1, DF):.4f} reported p={x['prep']:.3f}")
rec_p = [x["p"] for x in R]
rep_p = [x["prep"] for x in R]
rho, _ = stats.spearmanr(rec_p, rep_p)
print(f" Spearman correlation between recomputed and reported p across the 10 rows: {rho:.2f}")
print("\n=== 5. Table 2 (ANCOVA, iron): F vs p on F(1, 99) ===")
for r, F, p in T2:
print(f" {r:12s} F={F:7.3f} -> p={stats.f.sf(F, 1, 99):.3g} reported p={p}")
print("\n=== 6. Table 3 (left vs right iron, ADHD group n = 53): T vs p ===")
print(" r_needed = left-right correlation a paired t test would need to reach the reported T")
for r, ml, sl, mr, sr, T, p in T3:
t_ind, _ = stats.ttest_ind_from_stats(ml, sl, 53, mr, sr, 53)
sd_d = abs(ml - mr) * np.sqrt(53) / T
r_need = (sl**2 + sr**2 - sd_d**2) / (2 * sl * sr)
print(f" {r:12s} T={T:6.3f} reported p={p:.3f} p(T|df=52)={p2(T, 52):.2e} "
f"p(T|df=1)={p2(T, 1):.3f} independent-samples t={t_ind:6.3f} r_needed={r_need:.3f}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
labels = [x["region"] for x in R]
y = np.arange(len(R))[::-1]
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
ax.set_yticks(y)
ax.set_yticklabels(labels)
# Figure 1: reported T vs recomputed t on one t axis
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
style(ax)
crit = stats.t.isf(0.025, DF)
ax.axvspan(-crit, crit, color="#efeeea", zorder=0, lw=0)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
for yi, x in zip(y, R):
ax.plot([x["t"], x["T"]], [yi - 0.14, yi + 0.14], color="#c9c8c3", lw=2, zorder=2)
ax.plot(x["T"], yi + 0.14, "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.plot(x["t"], yi - 0.14, "D", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.text(14.4, yi, f"{x['T']:.2f} vs {x['t']:.2f}", va="center", ha="left", fontsize=10,
color=INK2)
ax.text(14.4, len(R) - 0.35, "reported vs\nrecomputed", va="bottom", ha="left", fontsize=10,
color=INK, fontweight="bold")
ax.text(0, -1.05, f"|t| < {crit:.2f}: p > .05 (df = 100)", ha="center", va="center",
fontsize=10, color=INK2)
ax.set_xlim(-3, 14)
ax.set_ylim(-1.5, len(R) - 0.4)
ax.set_xlabel("t statistic, control minus ADHD volume")
leg = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported T value (Table 4)"),
Line2D([], [], marker="D", ls="", ms=8, color=BLUE,
label="t recomputed from Table 4 means and SDs (n = 49 / 53)")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 1.13), ncol=1, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.16, right=0.80, top=0.86, bottom=0.11)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 2: three p values per row on one log axis
fig, ax = plt.subplots(figsize=(7.2, 4.8), facecolor=BG)
style(ax)
ax.set_xscale("log")
ax.axvline(0.05, color=INK2, lw=0.8, zorder=1)
ax.text(0.03, -0.55, "p = .05", ha="right", va="bottom", fontsize=10, color=INK2)
for yi, x in zip(y, R):
lo = min(x["pT"], x["p"], x["prep"])
hi = max(x["pT"], x["p"], x["prep"])
ax.plot([lo, hi], [yi, yi], color="#c9c8c3", lw=2, zorder=2)
ax.plot(x["pT"], yi + 0.2, "s", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
ax.plot(x["prep"], yi, "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=5)
ax.plot(x["p"], yi - 0.2, "D", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.set_xlim(1e-24, 3)
ax.set_ylim(-0.7, len(R) - 0.5)
ax.set_xticks([1e-24, 1e-20, 1e-16, 1e-12, 1e-8, 1e-4, 1])
ax.set_xlabel("two-sided p value (log scale)")
leg = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported p value (Table 4)"),
Line2D([], [], marker="D", ls="", ms=8, color=BLUE,
label="p recomputed from Table 4 means and SDs"),
Line2D([], [], marker="s", ls="", ms=8, mfc=BG, mec=BLUE, mew=2,
label="p implied by the reported T value (t, df = 100)")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.45, 1.2), ncol=1, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.16, right=0.97, top=0.82, bottom=0.11)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Stated best schizophrenia classifier (RaEn, accuracy 0.8409) is contradicted by the paper's Table 5
Hum Brain Mapp (2022) · doi:10.1002/hbm.26129 · PMC9783493
Summary
The paper compares seven entropy measures, computed on three brain atlases, for linear-SVM classification of schizophrenia (SCHZ), bipolar disorder (BP) and adult ADHD against healthy controls. The Abstract names range entropy (RaEn) as the best measure for SCHZ. The Results (twice), Discussion and Conclusions say that RaEn "performs the best, where RaEn achieves an accuracy of 0.8409", and refer the reader to Table 5. Table 5 has no SCHZ accuracy of 0.8409. RaEn's SCHZ accuracies there are 0.6932, 0.8068 and 0.7841, and the highest SCHZ accuracy of any method is 0.8295, for sample entropy (SaEn) on the Brainnetome atlas. The value 0.8409 appears in Table 5 only as a specificity (Brainnetome RaEn and SaEn) and as one sensitivity (Power FuEn). As a result, the entropy ranking for SCHZ in the Abstract and Conclusions does not follow from the paper's own table, and the accuracy given for it is too high. The equivalent claims for ADHD and BP match Table 5.
What the paper reports
Abstract, final sentence:
The core brain regions that affect the classification are given, and DiffEn performed best on ADHD, SaEn for BP, and RaEn for SCHZ.
Results, "Classification comparison of different BENs", paragraph 1:
When a two-sample t-test was done, we did a classification task based on entropy values calculated using different ROI levels and the whole brain based on three different atlases. The result is presented in Tables 5 and 6. Table 5 shows the classification results at the whole-brain level from the three atlases studies. From the ADHD classification results, it can be seen that DiffEn performs the best, where DiffEn achieves an accuracy of 0.8548. Among the classification results of BP, SaEn performs the best, where SaEn achieves an accuracy of 0.8824. For the classification results of SCHZ, RaEn performs the best, where RaEn achieves an accuracy of 0.8409. Refer to Table 5 for details. Table 6 is based on the best results of a whole-brain classification in Table 5.
Results, same section, paragraph 2 (the SCHZ sentence is repeated):
For the classification results of SCHZ, RaEn performs the best, where RaEn achieves an accuracy of 0.8409. Please refer to Table 5 for details.
Results, same section, paragraph 3:
For SCHZ, this experiment uses RaEn (accuracy = 0.8409) as analysis weight. The top 10 key features (brain areas) of the classifiers corresponding to RaEn are: L_Frontal Pole, R_Superior Temporal Gyrus, posterior division, R_Angular Gyrus, R_Frontal Pole, L_Frontal Pole, L_Frontal Pole, R_Supramarginal Gyrus, posterior division, R_Supramarginal Gyrus, posterior division, R_Frontal Orbital Cortex, and L_Frontal Pole.
Discussion, paragraph 1, and Conclusions (identical sentence in both):
For the classification results of SCHZ, RaEn performs the best, where RaEn achieves an accuracy of 0.8409.
The Abstract sentence is repeated verbatim at the end of the Discussion subsection "The entropy method comparison".
A similar-looking ranking, "DiffEn, RaEn, and SaEn performed better in ADHD, SCHZ, and BP, respectively", is not part of this issue. It first appears in Results, "Multiscale entropy analysis", paragraph 1, directly after the multiscale-entropy group comparison (Table 4, Figure 7), and in the Discussion and Conclusions it again follows the multiscale-entropy result and precedes the classification sentences. It is a statement about multiscale-entropy group differences, which Table 5 neither supports nor contradicts.
Table 5 ("Whole-brain level: Classification result"), SCHZ rows. Columns are ApEn, DiffEn, DispEn, FuEn, PeEn, RaEn, SaEn. The auc rows repeat the acc rows exactly and are omitted here.
| Atlas | Row | ApEn | DiffEn | DispEn | FuEn | PeEn | RaEn | SaEn |
|---|---|---|---|---|---|---|---|---|
| Dosenbach | acc | 0.5682 | 0.6932 | 0.7159 | 0.7727 | 0.5682 | 0.6932 | 0.7159 |
| Dosenbach | Sensitivity | 0.5909 | 0.7955 | 0.5682 | 0.8636 | 0.5682 | 0.7273 | 0.6818 |
| Dosenbach | Specificity | 0.5455 | 0.5909 | 0.8636 | 0.6818 | 0.5682 | 0.6591 | 0.75 |
| Brainnetome | acc | 0.5568 | 0.7841 | 0.5909 | 0.7955 | 0.0909 | 0.8068 | 0.8295 |
| Brainnetome | Sensitivity | 0.4318 | 0.8182 | 0.5909 | 0.8636 | 0.0227 | 0.7727 | 0.8182 |
| Brainnetome | Specificity | 0.6818 | 0.75 | 0.5909 | 0.7273 | 0.1591 | 0.8409 | 0.8409 |
| Power | acc | 0.6023 | 0.75 | 0.5909 | 0.75 | 0.0568 | 0.7841 | 0.7386 |
| Power | Sensitivity | 0.5682 | 0.7727 | 0.5227 | 0.8409 | 0.0455 | 0.7955 | 0.7273 |
| Power | Specificity | 0.6364 | 0.7273 | 0.6591 | 0.6591 | 0.0682 | 0.7727 | 0.75 |
Bold marks the cells set in bold in the article's JATS XML. The table footnote reads:
Note: Bold values represent the relative best classification result.
Table 6 ("The top 10 with the most prominent weight of SVM in classification") labels the SCHZ block "SCHZ(FuEn)". Its ten regions are the same, in the same order, as the list that the Results text attributes to RaEn.
Materials and Methods, "Resting-state fMRI data" and "Classification analysis":
31 HC and 31 ADHD; 34 HC and 34 BP; 42 HC and 42 SCHZ remained for the current study.
Subjects were separated into stratified training (90%) and test (10%) subsets for each HC and disease dataset. ... This was repeated using the same 10 permutations of randomized training and testing splits to distribute overall performance metrics for BEN classification.
What the paper's own numbers show
Method. I transcribed the acc, sensitivity and specificity rows of Table 5 for all three disorders and three atlases (63 cells per row type) from the PMC-archived JATS XML, which matches the publisher's PDF. For each disorder I found the highest accuracy cell and compared it with the method and accuracy that the text names. As an internal check I confirmed that every SCHZ accuracy equals (sensitivity + specificity)/2. That is what balanced classes would give, and it shows the table's acc rows really are accuracies and not mislabelled specificities. I also searched the SCHZ block for every cell equal to 0.8409, and found the per-class denominators that the four-decimal sensitivities and specificities imply. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Best method for SCHZ and its accuracy | Abstract; Results ×2; Discussion; Conclusions | RaEn, 0.8409 | Highest SCHZ accuracy in Table 5 is SaEn, 0.8295 (Brainnetome); RaEn is second at 0.8068 |
| RaEn SCHZ accuracy, highest over atlases | Table 5 | text: 0.8409 | 0.8068 (Brainnetome); 0.6932 Dosenbach, 0.7841 Power |
| SCHZ accuracy cells equal to 0.8409 | Table 5 | — | none of 21 |
| Cells equal to 0.8409 in the SCHZ block | Table 5 | — | Brainnetome Specificity (RaEn and SaEn); Power Sensitivity (FuEn) |
| Brainnetome RaEn accuracy from its own sensitivity and specificity | Table 5 | 0.8068 | (0.7727 + 0.8409)/2 = 0.8068 |
| Brainnetome SaEn accuracy from its own sensitivity and specificity | Table 5 | 0.8295 | (0.8182 + 0.8409)/2 = 0.8295 |
| Largest |acc − (sens + spec)/2| over the 21 SCHZ cells | Table 5 | — | 0.00005 (rounding only) |
| Best SCHZ method within each atlas | Table 5 | RaEn bold in all three atlases (and FuEn in Dosenbach) | Dosenbach FuEn 0.7727; Brainnetome SaEn 0.8295; Power RaEn 0.7841 |
| Gap between claimed and tabulated accuracy | derived | 0.8409 | 0.0341 above RaEn's best; 0.0114 above the best of any method |
| ADHD best method (control) | Results; Table 5 | DiffEn, 0.8548 | DiffEn 0.8548 (Power): matches |
| BP best method (control) | Results; Table 5 | SaEn, 0.8824 | SaEn 0.8824 (Power): matches |
| Classifier used for the SCHZ weights | Results p3 vs Table 6 | text: RaEn; Table 6: "SCHZ(FuEn)" | the two labels disagree, and neither is the highest-accuracy SCHZ model (SaEn) |
| Per-class denominator implied by sensitivity/specificity cells | Table 5 vs Methods | n per group: ADHD 31, BP 34, SCHZ 42 | only n = 31 (ADHD), 34 (BP) and 44 (SCHZ) fit all cells for n from 20 to 60 |
| 0.8409 as a fraction | derived | — | 37/44 (as a specificity), or 74/88 (as an accuracy on 88 predictions); not k/42 (35/42 = 0.8333, 37/42 = 0.8810) |
The text's claims for ADHD and BP match the maximum of the corresponding accuracy rows exactly, so the same reading of Table 5 is correct for two of the three disorders. For SCHZ the stated accuracy equals the Brainnetome RaEn (and SaEn) specificity in Table 5. The stated winner, RaEn, is the best SCHZ method only on the Power atlas (0.7841). On the Brainnetome atlas SaEn is higher (0.8295 vs 0.8068), and on the Dosenbach atlas FuEn is higher (0.7727 vs 0.6932).
A secondary observation: every SCHZ sensitivity and specificity in Table 5 is a multiple of 1/44, while the Methods report 42 subjects per SCHZ group. The ADHD and BP cells are multiples of 1/31 and 1/34, which match those groups' sizes. This is consistent with pooling the test predictions of the 10 repeated splits described in the Methods: 10 stratified 10% test sets of 84 subjects hold about 42 to 45 predictions per class, depending on how the test-set size is rounded. But the Methods do not say how the splits were combined, and the ADHD and BP denominators equal their group sizes (31 and 34) while the SCHZ denominator does not. This is a reporting gap rather than a clear error.
Figure
Figure 1. On the left are the Abstract and the Results/Discussion/Conclusions sentences that name RaEn as the best SCHZ classifier, with accuracy 0.8409 (orange). On the right is the Brainnetome-atlas SCHZ block of Table 5 for RaEn and SaEn. The accuracy cells (blue) are 0.8068 for RaEn and 0.8295 for SaEn. The value 0.8409 sits in the Specificity row for both methods. Each accuracy equals the mean of its sensitivity and specificity, so the 0.8409 in the text cannot be RaEn's Brainnetome accuracy as tabulated.
Figure 2. Every SCHZ accuracy in Table 5: seven entropy methods (rows) on three atlases (marker shape). The RaEn and SaEn cells are filled blue. The orange line is the 0.8409 that the text gives as RaEn's SCHZ accuracy, and the dotted line marks chance (0.5). No cell reaches the orange line. The highest SCHZ accuracy is SaEn on the Brainnetome atlas (0.8295), followed by RaEn on the same atlas (0.8068). The full 0–1 axis is shown. The two PeEn cells far below chance are as printed in Table 5.
Why it matters
The paper sets out to find which entropy measure is most suitable for each disorder. Its take-home message, in the final sentence of the Abstract and repeated in the Discussion, is a ranking: DiffEn for ADHD, SaEn for BP, RaEn for SCHZ. The Results, Discussion and Conclusions back the SCHZ part with the accuracy 0.8409. For SCHZ that ranking and the accuracy given for it are not what Table 5 shows. By the table, SaEn would be the best SCHZ measure overall (0.8295, Brainnetome), and RaEn's best accuracy is 0.8068. A reader choosing an entropy measure for SCHZ from the Abstract would pick a measure that the paper's own table places second. A reader citing the SCHZ accuracy would cite a number about 0.03 higher than any RaEn result in the table.
The Table 6 brain regions for SCHZ are affected too. The text says they come from the RaEn classifier with accuracy 0.8409, Table 6 says FuEn, and the text says Table 6 "is based on the best results of a whole-brain classification in Table 5", which for SCHZ would be SaEn. It is therefore unclear which classifier the ten listed SCHZ regions come from, and the paper's interpretation of those regions rests on that list.
The paper's other main findings do not depend on this number. These are the test–retest reliability analysis, lower multiscale entropy in all three disorders, the network-level group differences, and the ADHD and BP classification results, which match Table 5. Whole-brain SCHZ classification at about 0.80–0.83 accuracy remains supported by Table 5.
Explanations considered
- The acc and specificity rows of Table 5 are swapped or mislabelled. This does not account for the numbers. In every SCHZ column, acc equals (sensitivity + specificity)/2 to within rounding (largest difference 0.00005), which is what an accuracy with balanced classes must be. The rows are labelled correctly, and in Table 5 0.8409 is a specificity.
- A typographical error in the text (0.8068 meant). This could explain the number but not the method ranking. If the text meant 0.8068, RaEn would still not be the best SCHZ method, because SaEn reaches 0.8295 on the same atlas. A correction of the number alone would leave the Abstract's ranking unsupported.
- "Best" means best on one atlas, or best on average over atlases. This does not account for the claims. RaEn is the best SCHZ method on the Power atlas only, where its accuracy is 0.7841, not 0.8409. Averaged over the three atlases, FuEn is highest (0.7727), RaEn second (0.7614) and SaEn third (0.7613), and no average is 0.8409. The text applies the same wording ("performs the best, where ... achieves an accuracy of") to ADHD and BP, where it does mean the overall maximum of Table 5.
- 0.8409 is the accuracy of a different run, such as an ROI-level classification or another split, that is not shown in Table 5. This cannot be excluded from the published material. 0.8409 = 74/88 is an attainable accuracy for 88 pooled predictions. But each SCHZ sentence directs the reader to Table 5 ("Refer to Table 5 for details"), the paper reports no other classification results, and the Results say Table 6 is based on the best whole-brain results in Table 5. If such a run exists, the paper does not report it, so the claim cannot be checked.
- The bold marking in Table 5 defines "best". This does not resolve the conflict. The bold is on RaEn in all three SCHZ blocks, and also on FuEn in the Dosenbach block. On Brainnetome and Dosenbach it marks a value that is not the highest in its row (0.8068 < 0.8295; 0.6932 < 0.7727). The bold is consistent with the text's choice of RaEn, but not with the numbers in the table.
- A conversion or typesetting error in the archived copy. This is excluded. The PMC text, the Europe PMC JATS XML and the publisher's typeset PDF held in the PMC open-access bucket carry the same sentences and the same Table 5 values.
- The value is explained in a supplement. This is excluded: the article has no supplementary material (see Status).
Status
- Version of record. The Wiley landing page (via doi.org) returned HTTP 403 to automated access. I compared the PMC-archived text with the publisher's typeset PDF in the PMC open-access bucket (PMC9783493.1.pdf; 25 pages, cited as Hum Brain Mapp 2023;44(1):94–118, Table 5 on journal page 112) and with the Europe PMC full-text JATS XML. All three carry identical text and Table 5 values. The bold markings were read from the JATS XML. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. There is no supplementary material. The JATS XML has no supplementary-material element, the PMC open-access package holds only the figure images, and the Europe PMC supplementary-files endpoint returns only the article's figure images. The Data Availability Statement points to the public raw datasets (MSC, OpenNeuro ds000224; UCLA CNP, ds000030) and says the entropy code is available from the authors on request. No classifier outputs are deposited, so the published material cannot settle which SCHZ accuracy is correct.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI. PMC and Europe PMC list no correction notices. PubMed (PMID 36358029) lists no comments or corrections. A web search for corrections or comments on the article found none. The PubPeer API refused automated access (HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC9783493; Europe PMC full-text XML for PMC9783493; the publisher PDF in the PMC open-access dataset (s3://pmc-oa-opendata/PMC9783493.1/); Crossref; PubMed; Europe PMC.
Suggested correction
- In the Results (both paragraphs), the Discussion and the Conclusions, replace "RaEn performs the best, where RaEn achieves an accuracy of 0.8409" with the accuracy that Table 5 supports. Per Table 5, the best SCHZ accuracy is SaEn, 0.8295 (Brainnetome atlas), and RaEn's best is 0.8068 (Brainnetome atlas). If 0.8409 comes from an analysis not shown in Table 5, report that analysis and say how it differs.
- Revise the Abstract's final sentence, and its repeat at the end of the Discussion subsection "The entropy method comparison", so that the method named for SCHZ matches Table 5.
- Make the bold marking in Table 5 match its footnote, or redefine "relative best" in the footnote.
- State which SCHZ classifier produced the weights in Table 6 and the regions listed in the Results (the text says RaEn, Table 6 says FuEn), and make the label, the text and the accuracy quoted for that classifier agree.
- State how the test predictions of the 10 repeated splits were combined into the Table 5 sensitivities and specificities. The SCHZ cells imply 44 predictions per class, while the ADHD and BP denominators equal their group sizes (31 and 34).
Recomputation script (python recompute.py.txt)
# Recomputation for PMC9783493 (Hum Brain Mapp 2022, doi:10.1002/hbm.26129).
# Run: python recompute.py.txt
# Prints every recomputed number in writeup.md and writes fig1/fig2 (.svg, .png)
# next to this file. All inputs are transcribed from Table 5 of the article
# (PMC archive / Europe PMC JATS XML, identical to the publisher PDF).
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
METHODS = ["ApEn", "DiffEn", "DispEn", "FuEn", "PeEn", "RaEn", "SaEn"]
ATLASES = ["Dosenbach", "Brainnetome", "Power"]
# Table 5, whole-brain classification. acc, Sensitivity, Specificity rows.
# (The auc rows are identical to the acc rows in every block.)
T5 = {
"ADHD": {
"Dosenbach": {"acc": [0.5968,0.8065,0.2903,0.7097,0.5323,0.6774,0.7419],
"sens":[0.5161,0.8065,0.2258,0.7742,0.4194,0.7419,0.7419],
"spec":[0.6774,0.8065,0.3548,0.6452,0.6452,0.6129,0.7419]},
"Brainnetome": {"acc": [0.6613,0.8387,0.2258,0.8065,0.6613,0.7581,0.8065],
"sens":[0.6452,0.8387,0.3226,0.9032,0.6129,0.7419,0.7097],
"spec":[0.6774,0.8387,0.129,0.7097,0.7097,0.7742,0.9032]},
"Power": {"acc": [0.5968,0.8548,0.5323,0.7097,0.6613,0.7258,0.8065],
"sens":[0.8065,0.871,0.4516,0.9032,0.7742,0.7742,0.7742],
"spec":[0.3871,0.8387,0.6129,0.5161,0.5484,0.6774,0.8387]}},
"BP": {
"Dosenbach": {"acc": [0.3971,0.7647,0.3235,0.7941,0.4559,0.7353,0.8235],
"sens":[0.3235,0.7941,0.3235,0.8824,0.3824,0.7353,0.7647],
"spec":[0.4706,0.7353,0.3235,0.7059,0.5294,0.7353,0.8824]},
"Brainnetome": {"acc": [0.25,0.8235,0.5882,0.7941,0.5588,0.8529,0.8529],
"sens":[0.0294,0.7941,0.5294,0.8529,0.5,0.8529,0.8235],
"spec":[0.4706,0.8529,0.6471,0.7353,0.6176,0.8529,0.8824]},
"Power": {"acc": [0.1176,0.75,0.4265,0.7206,0.1176,0.8088,0.8824],
"sens":[0.1176,0.7647,0.3824,0.8235,0.2353,0.7941,0.8824],
"spec":[0.1176,0.7353,0.4706,0.6176,0,0.8235,0.8824]}},
"SCHZ": {
"Dosenbach": {"acc": [0.5682,0.6932,0.7159,0.7727,0.5682,0.6932,0.7159],
"sens":[0.5909,0.7955,0.5682,0.8636,0.5682,0.7273,0.6818],
"spec":[0.5455,0.5909,0.8636,0.6818,0.5682,0.6591,0.75]},
"Brainnetome": {"acc": [0.5568,0.7841,0.5909,0.7955,0.0909,0.8068,0.8295],
"sens":[0.4318,0.8182,0.5909,0.8636,0.0227,0.7727,0.8182],
"spec":[0.6818,0.75,0.5909,0.7273,0.1591,0.8409,0.8409]},
"Power": {"acc": [0.6023,0.75,0.5909,0.75,0.0568,0.7841,0.7386],
"sens":[0.5682,0.7727,0.5227,0.8409,0.0455,0.7955,0.7273],
"spec":[0.6364,0.7273,0.6591,0.6591,0.0682,0.7727,0.75]}},
}
# Cells set in bold in the SCHZ accuracy rows of Table 5 (JATS XML <bold>).
BOLD_SCHZ_ACC = {"Dosenbach": ["FuEn", "RaEn"], "Brainnetome": ["RaEn"], "Power": ["RaEn"]}
# Claims in the text
TEXT_BEST = {"ADHD": ("DiffEn", 0.8548), "BP": ("SaEn", 0.8824), "SCHZ": ("RaEn", 0.8409)}
N_PER_GROUP = {"ADHD": 31, "BP": 34, "SCHZ": 42} # Methods, Abstract, Conclusions
CLAIM = 0.8409
TOL = 5e-5
def cells(dis, metric):
return [(T5[dis][a][metric][i], a, m) for a in ATLASES for i, m in enumerate(METHODS)]
print("== 1. SCHZ accuracy cells in Table 5 (21 cells) ==")
for a in ATLASES:
row = T5["SCHZ"][a]["acc"]
print(f" {a:12s} " + " ".join(f"{m}={v:.4f}" for m, v in zip(METHODS, row)))
acc = sorted(cells("SCHZ", "acc"), reverse=True)
print(" Top three SCHZ accuracies:", [(f"{v:.4f}", a, m) for v, a, m in acc[:3]])
ra = {a: T5["SCHZ"][a]["acc"][5] for a in ATLASES}
print(" RaEn SCHZ accuracy by atlas:", ra, " max =", max(ra.values()))
print(" Any SCHZ accuracy equal to 0.8409?", any(abs(v - CLAIM) < TOL for v, _, _ in acc))
print(f" Gap: claimed 0.8409 - RaEn max {max(ra.values()):.4f} = {CLAIM - max(ra.values()):.4f};"
f" claimed - overall max {acc[0][0]:.4f} = {CLAIM - acc[0][0]:.4f}")
print(" Best SCHZ accuracy within each atlas:")
for a in ATLASES:
row = T5["SCHZ"][a]["acc"]; b = max(row)
print(f" {a:12s} {b:.4f} ({', '.join(m for m, v in zip(METHODS, row) if v == b)})"
f"; bolded in Table 5: {BOLD_SCHZ_ACC[a]}")
print(" Mean SCHZ accuracy over the three atlases:")
for i, m in enumerate(METHODS):
print(f" {m:6s} {sum(T5['SCHZ'][a]['acc'][i] for a in ATLASES)/3:.4f}")
print("\n== 2. Where 0.8409 appears in the SCHZ block ==")
for metric in ("acc", "sens", "spec"):
for v, a, m in cells("SCHZ", metric):
if abs(v - CLAIM) < TOL:
print(f" {metric:4s} {a:12s} {m}")
print("\n== 3. Accuracy = (sensitivity + specificity)/2 in every SCHZ cell ==")
maxdiff = 0
for a in ATLASES:
r = T5["SCHZ"][a]
for i in range(7):
maxdiff = max(maxdiff, abs(r["acc"][i] - (r["sens"][i] + r["spec"][i]) / 2))
print(f" max |acc - (sens+spec)/2| over 21 SCHZ cells = {maxdiff:.5f} (rounding only)")
b = T5["SCHZ"]["Brainnetome"]
print(f" Brainnetome RaEn: ({b['sens'][5]} + {b['spec'][5]})/2 = {(b['sens'][5]+b['spec'][5])/2:.4f}")
print(f" Brainnetome SaEn: ({b['sens'][6]} + {b['spec'][6]})/2 = {(b['sens'][6]+b['spec'][6])/2:.5f}")
print("\n== 4. Control check: text's best method and accuracy vs Table 5 maximum ==")
for dis in ("ADHD", "BP", "SCHZ"):
v, a, m = max(cells(dis, "acc"))
tm, tv = TEXT_BEST[dis]
ok = (m == tm and abs(v - tv) < TOL)
print(f" {dis:4s} text: {tm} {tv:.4f} | table max: {m} {v:.4f} ({a}) -> {'match' if ok else 'MISMATCH'}")
print("\n== 5. Implied number of subjects per class behind the sensitivity/specificity cells ==")
for dis in ("ADHD", "BP", "SCHZ"):
vals = [v for met in ("sens", "spec") for v, _, _ in cells(dis, met)]
fits = [n for n in range(20, 61) if all(abs(round(x * n) / n - x) < TOL for x in vals)]
print(f" {dis:4s} stated n per group = {N_PER_GROUP[dis]}; denominators n in 20..60 fitting all 42 cells: {fits}")
print(f" 0.8409 = 37/44 = {37/44:.4f}; as accuracy over 88: 74/88 = {74/88:.4f}")
print(f" with 42 per group: 35/42 = {35/42:.4f}, 36/42 = {36/42:.4f}, 37/42 = {37/42:.4f}")
import math
print(" Predictions per class if the test sets of the 10 repeated splits (stratified 10%) are pooled:")
for dis in ("ADHD", "BP", "SCHZ"):
n = N_PER_GROUP[dis]
print(f" {dis:4s} 10 x 10% x {n} = {10*0.1*n:.1f}; with a test set of ceil(0.1 x {2*n}) = {math.ceil(0.1*2*n)} "
f"subjects per split: {10*math.ceil(0.1*2*n)/2:.1f} per class")
print(f" RaEn Brainnetome acc 0.8068 = 71/88 = {71/88:.4f}; SaEn Brainnetome acc 0.8295 = 73/88 = {73/88:.4f}")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#9a9893"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5,
"text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: text-vs-table contradiction diagram
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")
def panel(x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec="#cfcdc8", lw=1))
ax.text(x + 1.5, y + h - 2.2, title, fontsize=10.5, weight="bold", va="top")
ax.text(50, 61.5, "The SCHZ accuracy in the text matches a specificity cell in Table 5",
ha="center", fontsize=11.5, weight="bold")
# left: text
panel(2, 11, 40, 46, "Text (Abstract, Results \u00d72,\nDiscussion, Conclusions)")
ax.text(3.5, 44.5, "Abstract:", fontsize=10, color=INK2, va="top")
ax.text(3.5, 41, "\u201c\u2026DiffEn performed best on\nADHD, SaEn for BP, and\nRaEn for SCHZ.\u201d",
fontsize=10, va="top", style="italic")
ax.text(3.5, 29.5, "Results, Discussion, Conclusions:", fontsize=10, color=INK2, va="top")
ax.text(3.5, 26, "\u201cFor the classification results\nof SCHZ, RaEn performs the\nbest, where RaEn achieves\nan accuracy of",
fontsize=10, va="top", style="italic")
ax.text(19.4, 15.6, "0.8409", fontsize=11, weight="bold", color=ORANGE, va="center")
ax.text(27.9, 15.6, ".\u201d", fontsize=10, va="center", style="italic")
# right: Table 5 Brainnetome SCHZ block
panel(56, 11, 42, 46, "Table 5, Brainnetome atlas,\nSCHZ rows")
cx = {"label": 58.5, "RaEn": 78, "SaEn": 90}
ty = {"head": 44, "acc": 37.5, "Sensitivity": 31, "Specificity": 24.5}
ax.text(cx["RaEn"], ty["head"], "RaEn", ha="center", weight="bold", fontsize=10.5)
ax.text(cx["SaEn"], ty["head"], "SaEn", ha="center", weight="bold", fontsize=10.5)
ax.plot([57.5, 96.5], [ty["head"] - 2.6] * 2, color="#cfcdc8", lw=1)
for row, key in (("acc", "acc"), ("Sensitivity", "sens"), ("Specificity", "spec")):
ax.text(cx["label"], ty[row], row, va="center", fontsize=10.5, color=INK2)
for m, i in (("RaEn", 5), ("SaEn", 6)):
v = b[key][i]
col = ORANGE if abs(v - CLAIM) < TOL else BLUE if key == "acc" else INK
ax.text(cx[m], ty[row], f"{v:.4f}", ha="center", va="center", fontsize=11,
color=col, weight="bold" if col != INK else "normal")
ax.add_patch(FancyBboxPatch((57.6, ty["Specificity"] - 2.4), 38.6, 4.8,
boxstyle="round,pad=0.2,rounding_size=1", fc="none", ec=ORANGE, lw=1.5))
ax.text(60, 18.2, "orange: equals the 0.8409 in the text", fontsize=9.5, color=INK2, va="center")
ax.text(60, 14.6, "blue: accuracy cells of Table 5", fontsize=9.5, color=INK2, va="center")
ax.plot([58.6], [18.2], "s", ms=6, color=ORANGE); ax.plot([58.6], [14.6], "s", ms=6, color=BLUE)
# connector
ax.add_patch(FancyArrowPatch((31.2, 15.6), (57.2, ty["Specificity"]), connectionstyle="arc3,rad=0.25",
arrowstyle="-|>", mutation_scale=14, lw=2, color=INK2))
ax.text(50, 6.0, "In Table 5, 0.8409 is RaEn's (and SaEn's) Brainnetome specificity, not an accuracy.",
ha="center", fontsize=10, weight="bold")
ax.text(50, 2.4, f"RaEn's highest SCHZ accuracy in Table 5 is {max(ra.values()):.4f}; "
f"the highest of any method is SaEn {acc[0][0]:.4f}.", ha="center", fontsize=10, color=INK2)
save(fig, "fig1")
# Figure 2: all 21 SCHZ accuracy cells vs the claimed value
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
markers = {"Dosenbach": "o", "Brainnetome": "s", "Power": "^"}
order = METHODS[::-1]
for a in ATLASES:
for i, m in enumerate(METHODS):
v = T5["SCHZ"][a]["acc"][i]
y = order.index(m) + {"Dosenbach": 0.18, "Brainnetome": 0, "Power": -0.18}[a]
hi = m in ("RaEn", "SaEn")
ax.plot(v, y, markers[a], ms=8.5 if hi else 8, mfc=BLUE if hi else "white",
mec=BLUE if hi else GRAY, mew=1.6, zorder=3, linestyle="none")
ax.axvline(CLAIM, color=ORANGE, lw=2, zorder=2)
ax.text(CLAIM + 0.01, 6.55, "0.8409: RaEn SCHZ accuracy\nstated in the text", color=ORANGE,
fontsize=10, va="top", ha="left")
ax.axvline(0.5, color=INK2, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(0.505, -0.75, "chance (0.5)", fontsize=9.5, color=INK2, va="bottom")
yb = order.index("SaEn"); yr = order.index("RaEn")
ax.text(0.05, yb - 0.15, f"SaEn, Brainnetome (square):\n{T5['SCHZ']['Brainnetome']['acc'][6]:.4f}, highest SCHZ\naccuracy in Table 5",
fontsize=9.5, color=INK, va="center")
ax.text(0.05, yr + 0.12, f"RaEn, Brainnetome (square):\n{T5['SCHZ']['Brainnetome']['acc'][5]:.4f}, RaEn's highest",
fontsize=9.5, color=INK, va="center")
ax.set_yticks(range(7)); ax.set_yticklabels(order, fontsize=10.5)
ax.set_xlim(0, 1); ax.set_ylim(-0.8, 6.7)
ax.set_xlabel("SCHZ vs HC whole-brain classification accuracy (Table 5)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#bdbbb6")
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], marker=markers[a], ls="none", mfc="white", mec=INK2, mew=1.4, ms=8, label=f"{a} atlas")
for a in ATLASES]
h.append(Line2D([], [], marker="o", ls="none", mfc=BLUE, mec=BLUE, ms=8, label="RaEn / SaEn cells (blue)"))
ax.legend(handles=h, loc="lower left", bbox_to_anchor=(0.0, 1.0), ncol=4, frameon=False,
fontsize=9.5, handletextpad=0.3, columnspacing=1.0)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png in", HERE)
Abstract reports a fetal HLA-B*39:06 effect at P < 0.001; its own P-value is 0.00384
Int J Epidemiol (2023) · doi:10.1093/ije/dyad142 · PMC10859143
Summary
The abstract and the Results say that HLA-B39:06 has a fetal allelic effect on birthweight at P < 0.001, the paper's Bonferroni threshold (α = 0.05/50 = 0.001). The same Results paragraph, and Supplementary Tables S7 and S8, give that fetal P-value as Pf = 3.84 × 10-3. The printed estimate and 95% CI, 0.090 (0.029, 0.151), imply P = 3.83 × 10-3, and the supplement's estimate and SE imply 3.88 × 10-3. So the reported Pf is internally consistent and the "P < 0.001" label is not: the effect is nominally significant but fails the threshold the paper set. This was the only fetal effect of a non-HLA-C allele that the abstract presents as passing the threshold, and no allele in Table S7 has an unconditional fetal P below 0.001. A second, smaller slip: the maternal 95% CI printed for B39:06 is not centred on its estimate.
What the paper reports
Quotations are verbatim from the PMC-archived text. Exponents are written here as superscripts, as they appear in the publisher's XML; the PMC plain text renders them as "10-3".
Abstract, Results:
Applying our SEM to imputed classical HLA alleles and own and offspring BW from the UKB replicated the previously reported association at the HLA-C locus and revealed strong evidence for maternal (HLA-A03:01, B35:01, B39:06, P <0.001) and fetal allelic effects (HLA-B39:06, P <0.001) of non-HLA-C alleles on BW.
Methods, "Application of SEM to BW data in the UK Biobank", on the tests used:
In addition to one degree of freedom tests where we evaluated the significance of the maternal and fetal components individually, we also compared the full model with a constrained model in which we fixed the maternal and fetal effects of the allele of interest to zero (i.e. a two degree of freedom test; Supplementary Note 6, Supplementary Table S6, available as Supplementary data at IJE online).
Results, "Empirical analysis in UK biobank", the significance threshold:
We further set a more stringent threshold for significance using Bonferroni correction for 50 tests (α = 0.05/50 = 0.001), on the rationale that the locus with the most coded alleles (HLA-B, 25 alleles) involved 50 statistical tests (i.e. one maternal and one fetal effect for each coded allele).
Same paragraph, next sentence:
Three of the alleles exhibited significant evidence for maternal effects, with P-values less than 0.001 (A03:01, B35:01, B39:06, labelled in Figure 4), and the HLA- B39:06 allele exhibited a significant fetal effect at P <0.001 (labelled in Figure 4).
Same paragraph, two sentences later:
The allele with the largest estimated maternal and fetal effects was B*39:06, which had a relatively low allele frequency of 0.69% and consequently wide confidence intervals for the effect sizes [maternal effect = -0.111 (-0.151, -0.029), Pm = 7.10 × 10-4; fetal effect = 0.090 (0.029, 0.151), Pf = 3.84 × 10-3].
Figure 4 legend:
Triangles indicate alleles where tests for both maternal and fetal effects reached the threshold; solid dots represent alleles which did not (P < 0.05). Alleles with P < 0.001 are explicitly labelled.
Discussion, fifth paragraph (first and last sentences):
The HLA-B*39:06 allele displayed the largest estimated maternal and fetal effects among all associated alleles.
These associations could be partly attributable to HLA-B*39:06, as our results indicate a positive fetal and a negative maternal effect of this allele on BW.
Supplementary Table S7, row B*3906 (columns: frequency; maternal effect, SE, P; fetal effect, SE, P):
B*3906 | 0.69% | -0.1108 | 0.0327 | 7.10E-04 | 0.0901 | 0.0312 | 3.84E-03
What the paper's own numbers show
Method. I treated the paper's 1-df maternal and fetal tests as Wald tests: two-sided P = 2Φ(−|β/SE|), with the SE taken either from Table S7 or from the printed 95% CI as (upper − lower)/(2 × 1.96). Each P was compared with the paper's own threshold, 0.001. To check that this approximates the paper's SEM tests, I applied it to the other effects quoted in the Results; it reproduces their P-values to within about 3% (for example A03:01 maternal 8.98 × 10-8 against a reported 9.21 × 10-8, and DRB111:04 fetal 1.36 × 10-3 against 1.34 × 10-3). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Bonferroni threshold | Results | α = 0.05/50 = 0.001 | 0.0010 |
| B*39:06 fetal P, as labelled | Abstract; Results | "P <0.001" | not supported (next rows) |
| B*39:06 fetal P, as printed | Results; Tables S7, S8 | Pf = 3.84 × 10-3 | 3.84 times the threshold; Bonferroni-adjusted P = 50 × Pf = 0.192 |
| B*39:06 fetal P from the printed CI | Results: 0.090 (0.029, 0.151) | — | SE 0.0311, z = 2.89, P = 3.83 × 10-3 |
| B*39:06 fetal P from β and SE | Table S7: 0.0901, SE 0.0312 | 3.84E-03 | z = 2.89, P = 3.88 × 10-3; 95% CI (0.029, 0.151), matching the text |
| B*39:06 fetal P, one-sided | derived from Table S7 | — | 1.94 × 10-3, still above 0.001 |
| Fetal estimate needed for P < 0.001 at SE 0.0312 | derived | — | |z| ≥ 3.29, so β ≥ 0.103 (the estimate is 0.090) |
| B*39:06 fetal 99.9% CI (the interval matching α = 0.001) | derived from Table S7 | — | (−0.013, 0.193), which includes zero |
| B*39:06 maternal P | Results; Table S7: −0.1108, SE 0.0327 | Pm = 7.10 × 10-4 | 7.03 × 10-4, below 0.001; the maternal claim holds |
| B*39:06 maternal 95% CI | Results | −0.111 (−0.151, −0.029) | (−0.175, −0.047) from Table S7. The printed interval's midpoint is −0.090, not −0.111; it is the fetal interval (0.029, 0.151) with the signs reversed |
| Alleles with unconditional fetal P < 0.001 | Table S7 (all 19 alleles with a maternal or fetal P < 0.05) | the text names B*39:06 | none; the smallest fetal P is DRB1*11:04, 1.34 × 10-3 |
| Alleles with maternal P < 0.001 | Table S7 | A03:01, B35:01, B*39:06 | A03:01, B35:01, B*39:06 (agrees) |
| B*39:06 fetal P, conditional on rs9366778 and rs6911024 | Table S8 | 6.04E-04 | 6.13 × 10-4, below 0.001 (a different model; see Explanations) |
| B*39:06 maternal P, conditional | Table S8 | 1.67E-02 | 1.66 × 10-2, above 0.001 |
| B*39:06 2-df joint test (maternal and fetal both zero) | Table S6; Supplementary Note | 7.65E-04 (Table S6); 7.56 × 10-4 (Note) | below 0.001, but a test of any effect, not of the fetal effect |
Two independent routes, from the printed CI and from the supplement's estimate and SE, both give a fetal P of about 0.0038, matching the printed Pf = 3.84 × 10-3. Only the label "P <0.001" in the abstract and the Results disagrees with it. A one-sided test would not close the gap. The maternal P for B*39:06 (7.10 × 10-4) is below 0.001, so the abstract's list of maternal effects is correct. Because Table S7 contains every allele with a nominal maternal or fetal effect, no allele omitted from it could have a fetal P below 0.001.
Figure
Figure 1. P-values for HLA-B*39:06 on a log scale; the solid vertical line is the paper's Bonferroni threshold, α = 0.05/50 = 0.001, and the dashed line is the nominal 0.05. Values left of the solid line pass the threshold. The top row (orange arrow) is the abstract and Results label "P < 0.001" for the fetal effect. The next row (orange) is the fetal P printed in the Results and in Table S7, 3.84 × 10-3. Blue rows are recomputed from the paper's own numbers: from the printed 95% CI (3.83 × 10-3), from the Table S7 estimate and SE (3.88 × 10-3), and as a one-sided test (1.94 × 10-3). All fetal values lie right of the threshold. The two shaded rows show, for comparison, the maternal P (reported 7.10 × 10-4, recomputed 7.03 × 10-4), which does pass it. All values are from the unconditional analysis. The log scale is used because P-values span orders of magnitude; the gap between 0.001 and 0.0038 is a factor of 3.8.
Figure 2. Estimates for HLA-B*39:06 with confidence intervals, on the effect scale used in the paper; zero is the reference line. Orange: as printed in the Results. Blue: computed from the Table S7 estimates and SEs. The printed fetal 95% CI matches the recomputed one exactly. The dashed blue interval is the fetal 99.9% CI, the interval that corresponds to a two-sided test at α = 0.001; it runs from −0.013 to 0.193 and includes zero, which shows the same failure to reach the threshold on the effect scale. The printed maternal interval (−0.151, −0.029) is centred on −0.090 (black cross), not on its estimate of −0.111; the interval implied by Table S7 is (−0.175, −0.047).
Why it matters
- An abstract-level finding is overstated. The abstract reports "strong evidence" for exactly one fetal allelic effect of a non-HLA-C allele, B*39:06 at P < 0.001. Under the paper's own multiple-testing criterion that effect does not qualify (Bonferroni-adjusted P = 0.19), and no other allele in Table S7 qualifies either. On the paper's own numbers, no non-HLA-C fetal effect reaches the stated threshold.
- The Results count changes in the same way. The Results report three Bonferroni-significant maternal effects and one fetal effect. On the tabulated values the count is three and none.
- The Discussion builds on it. The type 1 diabetes interpretation rests on "a positive fetal and a negative maternal effect of this allele on BW". The fetal estimate is still positive and nominally significant (its 95% CI excludes zero), so the reasoning is not voided, but its fetal half rests on weaker evidence than the abstract states.
- The rest of the paper is not affected. The SEM, the simulation and power results, the maternal associations at A03:01, B35:01 and B*39:06, and the replication at HLA-C do not depend on this label. The fetal point estimate and its direction are unchanged.
- The interval slip could mislead readers who reuse the CI. The maternal 95% CI printed for B*39:06 is shifted toward zero: its bounds are −0.151 and −0.029 instead of −0.175 and −0.047.
Explanations considered
- The printed Pf is a typo for a smaller value. Not supported. The supplement gives the same Pf (3.84E-03) in Tables S7 and S8, and both the printed CI and the Table S7 estimate and SE independently give P ≈ 0.0038.
- A one-sided test. Does not account for it. The one-sided fetal P is 1.94 × 10-3, still above 0.001, and the paper's other P-values reproduce as two-sided.
- The label refers to the conditional analysis. Table S8 also reports a model adjusted for rs9366778 and rs6911024, in which the B39:06 fetal P is 6.04 × 10-4, below 0.001. But the Results sentence that makes the claim, and the numbers quoted two sentences later, are the unconditional ones, and the text does not say that the fetal label comes from the conditional model. In that model the B39:06 maternal P is 1.67 × 10-2, so it would not support the paired maternal claim in the same sentence. Neither model supports both labels at once.
- The label refers to the 2-df joint test. B*39:06 has P2df = 7.65 × 10-4 (Table S6; 7.56 × 10-4 in the Supplementary Note), below 0.001. That test asks whether the maternal and fetal effects are both zero, so it cannot attribute an effect to the fetal component. The abstract and the Results attach P < 0.001 to the fetal component specifically, and the Methods describe the 1-df tests as the ones that "evaluated the significance of the maternal and fetal components individually".
- The Figure 4 label shows that the fetal effect passes. No. The legend says "Alleles with P < 0.001 are explicitly labelled". Only three alleles are labelled in Figure 4 (A03:01, B35:01, B39:06), exactly the three maternal hits; A03:01 is a labelled triangle although its fetal P is 6.92 × 10-3, and DRB111:04, the allele with the smallest fetal P (1.34 × 10-3), is not labelled. A label therefore marks an allele with at least one test below 0.001, which for B39:06 is the maternal test; the triangle marks only that both tests are below 0.05.
- A different threshold applies. No. The only threshold the paper gives for these claims is α = 0.05/50 = 0.001, and the abstract's "P <0.001" matches it.
- A text-conversion artifact. Ruled out. The publisher's page, the Europe PMC full-text XML (with true superscript exponents) and the PubMed abstract carry the same wording and values.
- The maternal CI comes from another column. No reading of the tables reproduces the printed (−0.151, −0.029). It is exactly the fetal interval with its signs reversed, is centred on −0.090 rather than −0.111, and implies an SE (0.0311) that is the fetal SE rather than the maternal SE (0.0327). The reported Pm agrees with Table S7, so this slip is confined to the interval.
Status
- Version of record. The publisher's page (academic.oup.com, reached through doi.org) has the same abstract sentence and the same Results values, as do the Europe PMC full-text XML and the PubMed abstract (PMID 37831898). Crossref gives online publication on 13 October 2023, volume 53, issue 1. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. I read the single supplementary file,
dyad142_supplementary_data.docx(Supplementary Notes, Figures S1–S8, Tables S1–S10), downloaded from Europe PMC. Tables S7 and S8 (unconditional) give B*3906 fetal 0.0901, SE 0.0312, P 3.84E-03 and maternal −0.1108, SE 0.0327, P 7.10E-04. Table S8 (conditional), Table S6 (2-df) and the Supplementary Note give the values cited under Explanations. The supplement does not resolve the discrepancy. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices; the Crossref record has empty update-to and updated-by fields, and a Crossref bibliographic search for a correction with this title returned only the article and its bioRxiv preprint (doi:10.1101/2022.08.23.505053). PubPeer's search API refused automated access (human verification required), and a web search for the title together with "PubPeer" found no PubPeer thread, so the absence of PubPeer comments is not fully confirmed. - Sources. PMC article PMC10859143; Europe PMC full-text XML and supplementary files; https://doi.org/10.1093/ije/dyad142 (publisher page); PubMed 37831898; Crossref; PubPeer (search attempted).
Suggested correction
In the abstract, replace
revealed strong evidence for maternal (HLA-A03:01, B35:01, B39:06, P <0.001) and fetal allelic effects (HLA-B39:06, P <0.001) of non-HLA-C alleles on BW.
with wording such as
revealed strong evidence for maternal allelic effects (HLA-A03:01, B35:01, B39:06, P <0.001) and suggestive evidence for a fetal allelic effect (HLA-B39:06, P = 3.8 × 10-3) of non-HLA-C alleles on BW.
In the Results, replace "and the HLA- B39:06 allele exhibited a significant fetal effect at P <0.001 (labelled in Figure 4)" with a statement that no fetal effect reached the Bonferroni threshold and that the B39:06 fetal effect was nominally significant (Pf = 3.84 × 10-3). If the authors wish, they can add that it falls below 0.001 in the analysis conditioned on rs9366778 and rs6911024 (Pf = 6.04 × 10-4, Table S8), identified as such. In the same paragraph, change the maternal interval for B*39:06 from "(-0.151, -0.029)" to "(-0.175, -0.047)", the interval implied by Table S7 (−0.1108, SE 0.0327), or to whatever interval the authors' fitted model gives.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
# Recomputation for PMC10859143 (Int J Epidemiol 2023, doi:10.1093/ije/dyad142).
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2 (.svg, .png)
# in the folder that holds this script. Inputs are typed in from the article (PMC) and from
# Supplementary Tables S6-S8 of dyad142_supplementary_data.docx.
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
def p2(b, se):
return 2 * norm.sf(abs(b / se))
def ci(b, se):
return (b - Z975 * se, b + Z975 * se)
def se_from_ci(lo, hi):
return (hi - lo) / (2 * Z975)
# ---------------------------------------------------------------- inputs
ALPHA = 0.05 / 50 # Results: "α = 0.05/50 = 0.001"
# Results text, B*39:06
TXT_M = (-0.111, -0.151, -0.029, 7.10e-4) # estimate, CI low, CI high, Pm
TXT_F = (0.090, 0.029, 0.151, 3.84e-3) # estimate, CI low, CI high, Pf
# Table S7 / Table S8 (unconditional), B*3906
S7_M = (-0.1108, 0.0327, 7.10e-4) # beta, SE, P
S7_F = (0.0901, 0.0312, 3.84e-3)
# Table S8, conditioned on rs9366778 and rs6911024, B*3906
S8C_M = (-0.1236, 0.0516, 1.67e-2)
S8C_F = (0.0990, 0.0289, 6.04e-4)
# 2-df joint tests
P2DF_NOTE = 7.56e-4 # Supplementary Note text
P2DF_S6 = 7.65e-4 # Table S6, unconditional column
# Other alleles quoted in the Results text (Table S7 values)
OTHERS = [("A*03:01 maternal", -0.0417, 0.0078, 9.21e-8),
("A*03:01 fetal", 0.0202, 0.0075, 6.92e-3),
("B*35:01 maternal", -0.0526, 0.0127, 3.58e-5),
("DRB1*11:04 fetal", -0.0711, 0.0222, 1.34e-3)]
# Table S7: fetal P-values of all 19 nominal alleles
S7_FETAL_P = {"A*03:01": 6.92e-3, "A*23:01": 9.55e-1, "A*24:02": 4.71e-1,
"B*35:01": 8.56e-1, "B*38:01": 1.72e-1, "B*39:06": 3.84e-3,
"B*Other": 4.50e-1, "C*03:03": 5.81e-2, "C*04:01": 2.51e-1,
"DRB1*01:01": 6.39e-1, "DRB1*08:01": 3.95e-2, "DRB1*11:04": 1.34e-3,
"DQB1*02:01": 3.32e-2, "DQB1*05:01": 7.41e-2, "DQB1*06:09": 7.58e-2,
"DQA1*01:01": 3.08e-1, "DPB1*01:01": 9.39e-1, "DPB1*03:01": 2.11e-1,
"DPB1*15:01": 2.87e-2}
# ---------------------------------------------------------------- calculations
print(f"Bonferroni threshold 0.05/50 = {ALPHA:.4f}")
b, lo, hi, prep = TXT_F
se_f_txt = se_from_ci(lo, hi)
p_f_txt = p2(b, se_f_txt)
print(f"\nB*39:06 fetal, Results text: SE from CI = {se_f_txt:.4f}, z = {b/se_f_txt:.3f}, "
f"two-sided P = {p_f_txt:.2e} (reported Pf = {prep:.2e}); below 0.001? {p_f_txt < ALPHA}")
b, s, prep = S7_F
p_f_s7 = p2(b, s)
p_f_1s = norm.sf(b / s)
print(f"B*39:06 fetal, Table S7: z = {b/s:.3f}, two-sided P = {p_f_s7:.2e} (reported {prep:.2e}); "
f"95% CI = ({ci(b,s)[0]:.3f}, {ci(b,s)[1]:.3f}); below 0.001? {p_f_s7 < ALPHA}")
print(f"B*39:06 fetal, one-sided P = {p_f_1s:.2e}; below 0.001? {p_f_1s < ALPHA}")
z_need = norm.isf(ALPHA / 2)
print(f"B*39:06 fetal, |z| needed for two-sided P < 0.001 = {z_need:.3f}; at SE {s} the estimate "
f"would need to be >= {z_need*s:.3f} (it is {b})")
z999 = norm.ppf(1 - ALPHA / 2)
f999 = (b - z999 * s, b + z999 * s)
print(f"B*39:06 fetal, 99.9% CI from Table S7 = ({f999[0]:.3f}, {f999[1]:.3f}); includes 0? {f999[0] < 0 < f999[1]}")
print(f"B*39:06 fetal, reported Pf / threshold = {prep/ALPHA:.2f}; Bonferroni-adjusted P = "
f"min(1, 50 x {prep:.2e}) = {min(1, 50*prep):.3f}")
b, s, prep = S7_M
p_m_s7 = p2(b, s)
m_lo, m_hi = ci(b, s)
print(f"\nB*39:06 maternal, Table S7: z = {b/s:.3f}, two-sided P = {p_m_s7:.2e} (reported {prep:.2e}); "
f"below 0.001? {p_m_s7 < ALPHA}")
print(f"B*39:06 maternal, 95% CI from Table S7 beta and SE = ({m_lo:.3f}, {m_hi:.3f})")
b, lo, hi, prep = TXT_M
print(f"B*39:06 maternal, Results-text CI ({lo}, {hi}): midpoint = {(lo+hi)/2:.3f} "
f"(estimate {b}), half-width = {(hi-lo)/2:.3f}, implied SE = {se_from_ci(lo,hi):.4f} "
f"(Table S7 SE {S7_M[1]})")
print(f" mirror of the fetal CI: -(fetal high, fetal low) = ({-TXT_F[2]}, {-TXT_F[1]})")
print(f" P implied by estimate -0.111 with the printed CI's SE = {p2(b, se_from_ci(lo,hi)):.2e}")
b, s, prep = S8C_F
print(f"\nB*39:06 fetal, conditional (Table S8): z = {b/s:.3f}, P = {p2(b,s):.2e} "
f"(reported {prep:.2e}); below 0.001? {p2(b,s) < ALPHA}")
b, s, prep = S8C_M
print(f"B*39:06 maternal, conditional (Table S8): z = {b/s:.3f}, P = {p2(b,s):.2e} "
f"(reported {prep:.2e}); below 0.001? {p2(b,s) < ALPHA}")
print(f"B*39:06 2-df joint test: P2df = {P2DF_NOTE:.2e} (Supp. Note), {P2DF_S6:.2e} (Table S6); "
f"below 0.001? {P2DF_NOTE < ALPHA and P2DF_S6 < ALPHA}")
print("\nOther alleles in the Results text (Wald P from Table S7 beta/SE vs reported):")
for name, b, s, rep in OTHERS:
print(f" {name}: P = {p2(b,s):.2e}, reported {rep:.2e}")
below = [k for k, v in S7_FETAL_P.items() if v < ALPHA]
print(f"Table S7 maternal P-values below 0.001 (per the three alleles named in the text): "
f"A*03:01 9.21e-08, B*35:01 3.58e-05, B*39:06 7.10e-04 -> all below? "
f"{all(v < ALPHA for v in (9.21e-8, 3.58e-5, 7.10e-4))}")
smallest = min(S7_FETAL_P, key=S7_FETAL_P.get)
print(f"Table S7 fetal P-values below 0.001: {below if below else 'none'}; smallest = "
f"{smallest} at {S7_FETAL_P[smallest]:.2e}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": SEC, "svg.fonttype": "none"})
LW = 1.5 # 2 px at 96 dpi = 1.5 pt
MS = 9
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: B*39:06 P-values against the paper's Bonferroni threshold (log scale)
rows = [
("Abstract and Results label:\n\"fetal ... P <0.001\"", "claim", None, ORANGE),
("Fetal Pf, Results text\nand Table S7", "dot", 3.84e-3, ORANGE),
("Fetal P from Results-text CI\n0.090 (0.029, 0.151)", "dot", p_f_txt, BLUE),
("Fetal P from Table S7\nβ 0.0901, SE 0.0312", "dot", p_f_s7, BLUE),
("Fetal P, one-sided,\nfrom Table S7", "dot", p_f_1s, BLUE),
("Maternal Pm, Results text\nand Table S7", "dot", 7.10e-4, ORANGE),
("Maternal P from Table S7\nβ −0.1108, SE 0.0327", "dot", p_m_s7, BLUE),
]
fig, ax = plt.subplots(figsize=(7.2, 5.0))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
ax.set_xscale("log")
ax.set_xlim(1e-4, 1e-1)
n = len(rows)
ys = np.arange(n)[::-1]
ax.axhspan(ys[-1] - 0.5, ys[-2] + 0.5, color="#f1f0ec", zorder=0)
ax.axvline(ALPHA, color=TXT, lw=0.8, zorder=1)
ax.text(ALPHA * 0.93, n - 0.45, "Bonferroni threshold\nα = 0.05/50 = 0.001",
ha="right", va="center", fontsize=10, color=TXT)
ax.axvline(0.05, color=SEC, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(0.05 * 0.93, n - 0.45, "nominal\n0.05", ha="right", va="center",
fontsize=10, color=SEC)
for y, (lab, kind, val, col) in zip(ys, rows):
if kind == "claim":
ax.annotate("", xy=(1.25e-4, y), xytext=(ALPHA, y),
arrowprops=dict(arrowstyle="-|>", color=col, lw=LW, mutation_scale=14))
ax.plot([ALPHA], [y], marker="o", ms=MS, mfc=BG, mec=col, mew=LW, zorder=3)
ax.text(ALPHA * 1.15, y, "claimed region: P < 0.001", va="center", fontsize=10, color=col)
else:
ax.plot([val], [y], marker="o", ms=MS, color=col, zorder=3)
left = val < ALPHA
ax.text(val / 1.15 if left else val * 1.15, y, ha="right" if left else "left", s= f"{val:.2e}".replace("e-0", " × 10⁻"),
va="center", fontsize=10, color=col)
# fix exponent rendering in direct labels
for t in ax.texts:
s = t.get_text()
if "× 10⁻" in s:
mant, ex = s.split(" × 10⁻")
sup = str.maketrans("0123456789", "⁰¹²³⁴⁵⁶⁷⁸⁹")
t.set_text(f"{mant} × 10⁻{ex.translate(sup)}")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
for tl, r in zip(ax.get_yticklabels(), rows):
tl.set_color(TXT)
ax.set_ylim(-0.6, n - 0.1)
ax.set_xlabel("Two-sided P-value for HLA-B*39:06 (log scale; one-sided where stated)")
ax.grid(axis="x", which="major", color=GRID, lw=0.6, zorder=0)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
h1 = ax.plot([], [], "o", color=ORANGE, ms=MS, label="reported in the paper")[0]
h2 = ax.plot([], [], "o", color=BLUE, ms=MS, label="recomputed from the paper's numbers")[0]
fig.legend(handles=[h1, h2], loc="lower center", ncol=2, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.06, 1, 1))
save(fig, "fig1")
# Figure 2: B*39:06 estimates and 95% CIs, printed vs recomputed (one axis, null at 0)
f_lo, f_hi = ci(S7_F[0], S7_F[1])
rows2 = [
("Fetal, Results text (95%)", TXT_F[0], TXT_F[1], TXT_F[2], ORANGE, "-"),
("Fetal, Table S7 \u03b2 and SE (95%)", S7_F[0], f_lo, f_hi, BLUE, "-"),
("Fetal, Table S7 (99.9%, \u03b1 = 0.001)", S7_F[0], f999[0], f999[1], BLUE, "--"),
("Maternal, Results text (95%)", TXT_M[0], TXT_M[1], TXT_M[2], ORANGE, "-"),
("Maternal, Table S7 \u03b2 and SE (95%)", S7_M[0], m_lo, m_hi, BLUE, "-"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
ys2 = np.arange(len(rows2))[::-1]
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
for y, (lab, est, lo, hi, col, ls) in zip(ys2, rows2):
ax.plot([lo, hi], [y, y], color=col, lw=LW, ls=ls, zorder=2)
ax.plot([lo, hi], [y, y], "|", color=col, ms=10, mew=LW, zorder=2)
ax.plot([est], [y], "o", color=col, ms=MS, zorder=3)
ax.text(hi + 0.008, y, f"{est:.3f} ({lo:.3f}, {hi:.3f})".replace("-", "\u2212"),
va="center", fontsize=10, color=col)
# midpoint of the printed maternal CI
mid = (TXT_M[1] + TXT_M[2]) / 2
ym = ys2[3]
ax.plot([mid], [ym], marker="x", color=TXT, ms=8, mew=1.5, zorder=4)
ax.annotate("midpoint of printed CI = \u22120.090", xy=(mid, ym), xytext=(-0.075, ym - 0.5),
fontsize=10, color=TXT, va="center",
arrowprops=dict(arrowstyle="-", color=TXT, lw=0.8))
ax.set_yticks(ys2)
ax.set_yticklabels([r[0] for r in rows2], fontsize=10)
for tl in ax.get_yticklabels():
tl.set_color(TXT)
ax.set_ylim(-0.7, len(rows2) - 0.4)
ax.set_xlim(-0.2, 0.42)
ax.set_xticks([-0.2, -0.1, 0, 0.1, 0.2])
ax.set_xlabel("HLA-B*39:06 effect on birthweight (paper's units)")
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
h1 = ax.plot([], [], "o-", color=ORANGE, ms=MS, lw=LW, label="printed in the Results text")[0]
h2 = ax.plot([], [], "o-", color=BLUE, ms=MS, lw=LW, label="recomputed from Table S7")[0]
fig.legend(handles=[h1, h2], loc="lower center", ncol=2, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.07, 1, 1))
save(fig, "fig2")
print("\nFigures written to", HERE)
Abstract reports the genetic risk score main effect as the gene-by-smoking interaction estimate
Int J Epidemiol (2017) · doi:10.1093/ije/dyw318 · PMC5837518
Summary
The abstract and the main Results paragraph give the paper's headline result, the interaction between the unweighted 26-SNP genetic risk score (uGRS) and ever/never smoking on FEV1/FVC, as βint = –0.036 (95% CI –0.040 to –0.032, P = 0.00057). Tables 1 and 2 give this same interaction, with the same P-value, as –0.0099 (CI –0.016 to –0.0043; SD 0.0029). The table values agree with each other and with P = 0.00057. The text's estimate and interval are instead the GRS main effect from Table 2 (–0.0363 ± 1.96 × 0.0021 = –0.0404 to –0.0322), and that interval on its own would imply P ≈ 1e-69. So the abstract states the interaction as about 3.6 times its tabulated size, with an interval that does not fit its own P-value. The direction of the interaction and its statistical significance are not affected.
What the paper reports
Abstract, Results:
We identified an interaction (βint = –0.036, 95% confidence interval, –0.040 to –0.032, P = 0.00057) between an unweighted 26 SNP genetic risk score and smoking status (ever/never) on the FEV1/FVC ratio.
Results, second paragraph (multivariate interaction tests):
The strongest signal was observed for the unweighted genetic risk score-by-smoking status interaction effect on FEV1/FVC (βint = –0.036, 95% CI –0.040 to –0.032, P = 0.00057).
Table 1, "Multivariate interaction tests of the 26 loci associated with pulmonary function", row 7 (Outcome | Exposure | Test | β̂int | (CI) | P-value):
FEV1/FVC | Smoking status | uGRS | –0.0099 | (–0.016, –0.0043) | 0.00057 b
Table 1 footer, in part:
^βint is the estimated interaction effect between the GRS and the outcome; and CI is the confidence interval of that estimate.
Table 2, "Summary of effect estimates for genetic risk score-by-smoking status interaction on FEV1/FVC", interaction-model rows (Predictors | Beta | SD | P-value):
GRS | –0.0363 | 0.0021 | 3.9 × 10–64
GRS × Smoking statusa | –0.0099 | 0.0029 | 5.7 × 10–4
Table 2 footer, in part:
GRS is the unweighted genetic risk score; beta is the effect estimates of each predictor; and SD the standard deviation of the each beta.
Results, fourth paragraph (replication):
overall interaction effect in the combined replication datasets β ^ i n t = –0.0027, 95% CI –0.0136, 0.0082 P = 0.63
Discussion, second paragraph:
Replication studies showed interaction effect estimates in the same direction as the discovery study but were not significant, and the magnitude of interaction effects were substantially smaller.
This sample size provides less than 50% power, at nominal significance of 5%, to detect interaction effects with the GRS.
What the paper's own numbers show
Method. I treated each estimate as a Wald estimate with a normal 95% interval, β ± 1.96 SE, and read the Table 2 "SD of each beta" as its standard error, as the table footer describes. From each (β, SE) pair I computed the interval and the two-sided P-value. From each printed interval I took the implied SE, (upper − lower) / 3.92, and the P-value that interval implies. I also asked what SE an estimate of –0.036 would need to give P = 0.00057. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Interaction, 95% CI from β and SE | Table 2 (–0.0099, SD 0.0029) vs Table 1 | (–0.016, –0.0043) | (–0.0156, –0.0042): matches Table 1 |
| Interaction, z and P | Table 2; Table 1; abstract | P = 5.7 × 10-4 = 0.00057 | z = –3.41, P = 6.4 × 10-4; SE 0.00287 (within rounding of 0.0029) gives exactly 0.00057 |
| GRS main effect, 95% CI | Table 2 (–0.0363, SD 0.0021) | not printed | (–0.0404, –0.0322), which rounds to (–0.040, –0.032): the interval printed in the text |
| GRS main effect, P | Table 2 | 3.9 × 10-64 | z = –17.3, P ≈ 6 × 10-67; SE between 0.00205 and 0.00215 gives 4 × 10-70 to 6 × 10-64, so the printed P is within rounding |
| Text interval: implied SE, z, P | Abstract; Results paragraph 2 | –0.036 (–0.040, –0.032), P = 0.00057 | SE 0.00204, z = –17.6, P ≈ 1 × 10-69 |
| SE and CI that P = 0.00057 needs for β = –0.036 | — | — | SE 0.0104, CI (–0.0565, –0.0155): a CI five times wider than the one printed |
| Ratio of text to table estimate | Abstract vs Tables 1 and 2 | –0.036 vs –0.0099 | 3.64 |
| Width of the 95% CI | Abstract vs Table 1 | 0.008 vs 0.0117 | the text interval is 0.68 times as wide |
| Discovery / replication ratio | Results paragraph 4 (–0.0027) | "substantially smaller" (Discussion) | 13.3-fold against the text value; 3.7-fold against the table value |
| Power of the combined replication (SE 0.00556 from its printed CI, two-sided α = 0.05) | Discussion: "less than 50% power" | — | 0.43 for a true βint of –0.0099; > 0.999 for a true βint of –0.036 |
The interaction estimate, its SE and its P-value in Tables 1 and 2 form one consistent set. The text pairs the interaction P-value with an estimate and interval that reproduce the main-effect row directly above it in Table 2, to within 0.0003. A P of 0.00057 is not compatible with the printed interval –0.040 to –0.032: that interval excludes zero by about 17.6 standard errors. The power statement in the Discussion fits the table value. With the replication SE implied by its own CI, a true interaction of –0.0099 gives about 43% power, which fits "less than 50%". A true interaction of –0.036 would give essentially 100% power. The power calculation in the paper is not fully described, so this is only a consistency check.
Figure
Figure 1. Estimates and 95% intervals on one axis (β for FEV1/FVC); the thin vertical line is the null value 0. Orange circles are values printed in the paper: the abstract and Results text (–0.036, –0.040 to –0.032), the Table 1 uGRS × smoking status row (–0.0099, –0.016 to –0.0043), and the combined replication estimate (–0.0027, –0.0136 to 0.0082). Blue diamonds are recomputed from the paper's own numbers: the Table 2 interaction row as –0.0099 ± 1.96 × 0.0029, the Table 2 GRS main-effect row as –0.0363 ± 1.96 × 0.0021, and the interval an estimate of –0.036 would need to have P = 0.00057 (SE 0.0104). In the shaded group, the text interval coincides with the main-effect interval and is far narrower than its P-value requires. The Table 1 row and the recomputed Table 2 interaction coincide. The axis is linear and includes zero.
Figure 2. Where each part of the text figure comes from. Left: the abstract sentence (the Results sentence is the same apart from "CI"). Right: the two interaction-model rows of Table 2, with intervals recomputed as β ± 1.96 SD (blue) and the Table 1 interval for comparison (orange). The estimate and CI in the text match the GRS main-effect row. The P-value matches the GRS × smoking status row. The footnote gives the SE, z and P implied by each.
Why it matters
- The headline quantity is misstated where most readers see it. The interaction is the paper's main finding and the only effect size in the abstract. A reader of the abstract or of PubMed sees an interaction of –0.036 per unit of GRS. That is 3.6 times the –0.0099 the tables support, and its interval is 0.68 times the width of the tabulated one. The size of the main genetic effect is reported in its place.
- The abstract's numbers do not fit together. An interval of –0.040 to –0.032 implies P ≈ 1e-69, not 0.00057. A reader who checks the numbers cannot tell from the abstract which part is right.
- It affects the comparison with replication. Against the text value, the combined replication estimate (–0.0027) is about 13 times smaller. Against the table value it is about 3.7 times smaller. The Discussion's statement that replication effects "were substantially smaller", and its argument that replication was underpowered, are better read against –0.0099. The stated power ("less than 50%") fits that value and not –0.036.
- What is not affected. The direction of the interaction, its P-value (0.00057) and its significance after the paper's Bonferroni correction (P < 4 × 10-3 for 12 tests) are all unchanged. Tables 1 and 2 use the correct value. Nothing in the paper indicates that the relative-risk analysis (Figure 2B) used the text figure. The qualitative conclusion, that genetic risk for low FEV1/FVC is associated with larger effects of smoking, does not depend on this number.
Explanations considered
- A different analysis, model or population. Does not explain it. The text names the same test as Table 1 row 7 (uGRS × smoking status, FEV1/FVC), in a paragraph that reports the Table 1 tests ("As shown in Table 1, ..."), and gives the same P-value, 0.00057. Table 2 is titled as a summary of the same interaction, and its interaction term (–0.0099, SD 0.0029, 5.7 × 10-4) agrees with Table 1. No other estimate of this interaction appears in the paper.
- Weighted rather than unweighted GRS. Does not explain it. The weighted-GRS estimates in Table 1 are –0.21, not –0.036, and the text says "unweighted".
- A rescaled or standardised coefficient. Does not explain it. The paper describes no rescaling. Rescaling β would scale its interval by the same factor and leave P unchanged. The text interval (SE ≈ 0.002) cannot go with P = 0.00057 at any scale for an estimate of –0.036.
- One-sided testing or a different reference distribution. Does not explain it. A one-sided test halves P, which cannot reconcile z ≈ 17.6 with z ≈ 3.4. With about 50 000 participants, a t-distribution is the same as the normal for this purpose.
- Rounding. Explains only the small gaps: 6.4 × 10-4 vs 5.7 × 10-4 for the interaction and 6 × 10-67 vs 3.9 × 10-64 for the main effect both fall within rounding of the printed SDs. Rounding cannot turn –0.0099 into –0.036.
- A conversion or typesetting artifact. Ruled out. The same –0.036 / –0.040 to –0.032 / P = 0.00057 appears in the publisher's version of record, in the PMC and Europe PMC full text, and in the PubMed abstract (PMID 28082375).
- The tables, not the text, are wrong. Not supported. The table estimate, SE, CI and P agree with each other. The Discussion's power statement fits the table value. The supplementary forest plot of study-specific estimates (Supplementary Figure 1) shows study estimates between about –0.025 and +0.01, with the largest-weight studies near –0.015 to 0. That fits a pooled value near –0.01 and could not pool to –0.036 with an interval of –0.040 to –0.032.
- Side observation, not part of this finding. The Results' first paragraph gives the single-SNP interaction for rs993925 on FEV1 as "βint = –0.036, 95% confidence interval (CI), –0.009 to –0.032, P = 0.007". That interval does not contain its own estimate. Supplementary Table 2 lists βint –0.036 and P 0.0070 for this SNP, so the interval bounds appear to be the part in error. This slip is separate from the main finding and does not bear on it.
Status
- Version of record. Checked on the publisher's page reached through https://doi.org/10.1093/ije/dyw318 (academic.oup.com). The abstract, Results paragraph 2, Table 1 and Table 2 carry the numbers quoted here. The PMC HTML page, the Europe PMC full-text XML and the PubMed abstract (PMID 28082375) agree with it. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. One supplementary file, "Supplementary Online Data Supplement" (18 pages, from the Europe PMC supplementary bundle). It holds the Supplementary Note (models and relative-risk derivation), Supplementary Tables 1–3 and Supplementary Figures 1–6. Supplementary Figure 1, the study-level forest plot for this interaction, is consistent with the table value (–0.0099) and not with the text value.
- Corrections and comments. As of 2026-10-24, I found no correction, erratum, corrigendum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC5837518; Europe PMC full text and supplementary files for PMC5837518; publisher's version of record at https://doi.org/10.1093/ije/dyw318; PubMed 28082375; Crossref; PubPeer.
Suggested correction
In the abstract, replace
(βint = –0.036, 95% confidence interval, –0.040 to –0.032, P = 0.00057)
with
(βint = –0.0099, 95% confidence interval, –0.016 to –0.0043, P = 0.00057)
and make the same change in the second paragraph of the Results, replacing "(βint = –0.036, 95% CI –0.040 to –0.032, P = 0.00057)" with "(βint = –0.0099, 95% CI –0.016 to –0.0043, P = 0.00057)". These are the values in Table 1 and Table 2. The authors may also wish to check the interval given for rs993925 in the first Results paragraph ("–0.036, 95% confidence interval (CI), –0.009 to –0.032"), which does not contain its estimate.
Recomputation script (python recompute.py.txt)
"""Recompute the headline interaction estimate of PMC5837518
(Int J Epidemiol 2017, doi:10.1093/ije/dyw318,
"Evidence for large-scale gene-by-smoking interaction effects on pulmonary function").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract and Results paragraph 2: beta_int = -0.036, 95% CI -0.040 to -0.032, P = 0.00057
(unweighted GRS x smoking status on FEV1/FVC).
Table 1, row FEV1/FVC | Smoking status | uGRS: -0.0099, (-0.016, -0.0043), P 0.00057.
Table 2, interaction model: GRS -0.0363, SD 0.0021, P 3.9e-64;
GRS x Smoking status -0.0099, SD 0.0029, P 5.7e-4.
Results paragraph 3, combined replication: -0.0027, 95% CI -0.0136, 0.0082, P = 0.63.
Discussion: combined replication (N = 18 316) "provides less than 50% power, at nominal
significance of 5%, to detect interaction effects with the GRS".
Results paragraph 1 (separate slip): rs993925 x smoking status on FEV1,
beta_int = -0.036, 95% CI -0.009 to -0.032, P = 0.007.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
def ci(b, se):
return b - Z * se, b + Z * se
def p2(b, se):
return 2 * norm.sf(abs(b / se))
def se_from_ci(lo, hi):
return (hi - lo) / (2 * Z)
# ---------------------------------------------------------------- inputs
TEXT = dict(b=-0.036, lo=-0.040, hi=-0.032, p=0.00057) # Abstract; Results para 2
T1 = dict(b=-0.0099, lo=-0.016, hi=-0.0043, p=0.00057) # Table 1 row 7
T2_INT = dict(b=-0.0099, se=0.0029, p=5.7e-4) # Table 2 GRS x smoking
T2_MAIN = dict(b=-0.0363, se=0.0021, p=3.9e-64) # Table 2 GRS main effect
REP = dict(b=-0.0027, lo=-0.0136, hi=0.0082, p=0.63) # combined replication
RS = dict(b=-0.036, lo=-0.009, hi=-0.032, p=0.007) # rs993925 sentence
BONF = 4e-3 # Results: 12 tests
print("=== 1. Table 2 interaction term (GRS x smoking status) ===")
lo, hi = ci(T2_INT["b"], T2_INT["se"])
z_int = T2_INT["b"] / T2_INT["se"]
print(f"95% CI = {T2_INT['b']} +/- 1.96 x {T2_INT['se']} = ({lo:.4f}, {hi:.4f}) [Table 1: (-0.016, -0.0043)]")
print(f"z = {z_int:.2f}, two-sided P = {p2(T2_INT['b'], T2_INT['se']):.2e} [reported 5.7e-4]")
zp = norm.isf(T1["p"] / 2)
se_for_p = abs(T2_INT["b"]) / zp
print(f"P = 0.00057 implies |z| = {zp:.3f}; with beta -0.0099 that needs SE = {se_for_p:.5f} "
f"(printed 0.0029; SE in [0.00285, 0.00295) gives P in "
f"[{p2(-0.0099, 0.00285):.2e}, {p2(-0.0099, 0.00295):.2e}])")
print(f"Interaction P vs Bonferroni threshold 4e-3: {T1['p']} < {BONF} -> {T1['p'] < BONF}")
print("\n=== 2. Table 1 uGRS row ===")
se1 = se_from_ci(T1["lo"], T1["hi"])
print(f"CI midpoint = {(T1['lo'] + T1['hi']) / 2:.5f}; implied SE = {se1:.5f}; "
f"P from beta and that SE = {p2(T1['b'], se1):.2e}; CI width = {T1['hi'] - T1['lo']:.4f}")
print("\n=== 3. Table 2 GRS main effect ===")
lo_m, hi_m = ci(T2_MAIN["b"], T2_MAIN["se"])
print(f"95% CI = {T2_MAIN['b']} +/- 1.96 x {T2_MAIN['se']} = ({lo_m:.4f}, {hi_m:.4f}) "
f"-> rounds to ({lo_m:.3f}, {hi_m:.3f})")
print(f"z = {T2_MAIN['b'] / T2_MAIN['se']:.1f}, P = {p2(T2_MAIN['b'], T2_MAIN['se']):.1e} [reported 3.9e-64]")
print(f"P range for SE in [0.00205, 0.00215): "
f"[{p2(-0.0363, 0.00215):.1e}, {p2(-0.0363, 0.00205):.1e}]")
print("\n=== 4. Abstract / Results text value ===")
se_t = se_from_ci(TEXT["lo"], TEXT["hi"])
z_t = TEXT["b"] / se_t
print(f"CI midpoint = {(TEXT['lo'] + TEXT['hi']) / 2:.4f}; implied SE = {se_t:.5f}; z = {z_t:.1f}; "
f"implied P = {p2(TEXT['b'], se_t):.1e} [printed 0.00057]; CI width = {TEXT['hi'] - TEXT['lo']:.4f}")
se_need = abs(TEXT["b"]) / zp
lo_n, hi_n = ci(TEXT["b"], se_need)
print(f"For beta -0.036 to have P = 0.00057, SE would be {se_need:.4f} and CI ({lo_n:.4f}, {hi_n:.4f})")
print(f"Ratio text / table estimate = {TEXT['b'] / T1['b']:.2f}; "
f"text CI width / Table 1 CI width = {(TEXT['hi'] - TEXT['lo']) / (T1['hi'] - T1['lo']):.2f}")
print(f"Text vs Table 2 main effect: |{TEXT['b']} - {T2_MAIN['b']}| = {abs(TEXT['b'] - T2_MAIN['b']):.4f}")
print("\n=== 5. Replication comparison and power statement ===")
se_r = se_from_ci(REP["lo"], REP["hi"])
print(f"Combined replication: implied SE = {se_r:.5f}, P = {p2(REP['b'], se_r):.2f} [reported 0.63]")
print(f"Discovery / replication ratio: text {TEXT['b'] / REP['b']:.1f}x, tables {T1['b'] / REP['b']:.1f}x")
def power(beta, se, alpha=0.05):
zc = norm.isf(alpha / 2)
d = abs(beta) / se
return norm.sf(zc - d) + norm.cdf(-zc - d)
pw_t, pw_c = power(T1["b"], se_r), power(TEXT["b"], se_r)
print(f"Approximate power of a Wald test with SE {se_r:.5f} (alpha 0.05): "
f"true beta -0.0099 -> {pw_t:.2f}; true beta -0.036 -> {pw_c:.6f}")
print(" (the Discussion states 'less than 50% power'; only the table value is consistent with that)")
print("\n=== 6. Separate slip: rs993925 sentence ===")
print(f"beta {RS['b']} inside its CI ({min(RS['lo'], RS['hi'])}, {max(RS['lo'], RS['hi'])})? "
f"{min(RS['lo'], RS['hi']) <= RS['b'] <= max(RS['lo'], RS['hi'])}")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ---- Figure 1: dot-and-interval, one axis
rows = [
# label, beta, lo, hi, kind, note
("Abstract and Results text\n(given as the interaction)", TEXT["b"], TEXT["lo"], TEXT["hi"], "rep",
f"printed P = 0.00057; CI implies P ≈ {p2(TEXT['b'], se_t):.0e}"),
("CI that P = 0.00057 would\nneed around −0.036", TEXT["b"], lo_n, hi_n, "rec",
f"SE {se_need:.4f}"),
("Table 2 GRS main effect\n−0.0363 ± 1.96 × 0.0021", T2_MAIN["b"], lo_m, hi_m, "rec",
f"({lo_m:.4f}, {hi_m:.4f}) ≈ text CI"),
("Table 1 uGRS × smoking\nstatus (FEV1/FVC)", T1["b"], T1["lo"], T1["hi"], "rep", "P = 0.00057"),
("Table 2 GRS × smoking\n−0.0099 ± 1.96 × 0.0029", T2_INT["b"], lo, hi, "rec",
f"P = {p2(T2_INT['b'], T2_INT['se']):.1e}"),
("Combined replication\n(Results text)", REP["b"], REP["lo"], REP["hi"], "rep", "P = 0.63"),
]
ys = [5.6, 4.6, 3.6, 2.2, 1.2, 0.0]
fig, ax = plt.subplots(figsize=(7.2, 4.8))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ax.axhspan(2.95, 6.2, color="#f1f0ec", zorder=0, lw=0)
for (lab, b, l, h, kind, note), y in zip(rows, ys):
col = ORANGE if kind == "rep" else BLUE
mk = "o" if kind == "rep" else "D"
ax.plot([l, h], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([l, l], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot([h, h], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot(b, y, marker=mk, ms=9 if kind == "rep" else 8, color=col, mec=col, zorder=4)
ax.text(h + 0.0012, y + 0.02, note, va="center", ha="left", fontsize=10, color=INK2)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.text(0.0006, -0.55, "null (0)", fontsize=10, color=INK2, va="center")
ax.text(-0.0595, 6.0, "text value and the rows that explain it", fontsize=10, color=INK2, va="center")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.8, 6.3)
ax.set_xlim(-0.06, 0.036)
ax.set_xlabel("Interaction estimate βint on FEV1/FVC, with 95% CI")
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
leg = [Line2D([0], [0], color=ORANGE, marker="o", lw=2, ms=8, label="reported in the paper"),
Line2D([0], [0], color=BLUE, marker="D", lw=2, ms=7, label="recomputed from the paper's numbers")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0, 1, 0.93))
save(fig, "fig1")
# ---- Figure 2: text panel vs Table 2 panel, with connectors
fig = plt.figure(figsize=(7.2, 4.3))
fig.patch.set_facecolor(BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.3)
ax.axis("off")
def panel(x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=INK2, lw=1))
ax.text(x + 0.12, y + h - 0.22, title, fontsize=10.5, fontweight="bold", va="center")
T = dict(fontsize=10, va="center")
panel(0.12, 1.25, 2.5, 2.55, "Abstract and Results text")
ax.text(0.24, 3.18, "“We identified an interaction", **T)
ax.text(0.24, 2.86, "(βint = –0.036, 95% confidence", color=ORANGE, **T)
ax.text(0.24, 2.56, "interval, –0.040 to –0.032,", color=ORANGE, **T)
ax.text(0.24, 2.08, "P = 0.00057)", color=ORANGE, **T)
ax.text(0.24, 1.76, "between an unweighted 26 SNP", **T)
ax.text(0.24, 1.46, "genetic risk score and …”", **T)
X0 = 3.9
panel(X0, 1.25, 3.2, 2.55, "Table 2, interaction model")
cols = [X0 + 0.12, X0 + 1.30, X0 + 1.92, X0 + 2.47]
for c, t in zip(cols, ["Predictor", "Beta", "SD", "P"]):
ax.text(c, 3.22, t, color=INK2, **T)
for c, t in zip(cols, ["GRS", "–0.0363", "0.0021", "3.9e–64"]):
ax.text(c, 2.82, t, **T)
ax.text(cols[0] + 0.15, 2.52, f"→ 95% CI ({lo_m:.3f}, {hi_m:.3f})", color=BLUE, **T)
for c, t in zip(cols, ["GRS × Smoking", "–0.0099", "0.0029", "5.7e–4"]):
ax.text(c, 2.06, t, **T)
ax.text(cols[0] + 0.15, 1.76, f"→ 95% CI ({lo:.4f}, {hi:.4f})", color=BLUE, **T)
ax.text(cols[0] + 0.15, 1.46, "Table 1: –0.0099 (–0.016, –0.0043)", color=ORANGE, **T)
ax.annotate("", xy=(X0 + 0.03, 2.76), xytext=(2.45, 2.71),
arrowprops=dict(arrowstyle="-|>", color=INK, lw=1.5, mutation_scale=12))
ax.text(3.26, 2.95, "estimate and CI\nmatch the\nmain effect", fontsize=10, ha="center", va="bottom")
ax.annotate("", xy=(X0 + 0.03, 2.03), xytext=(1.32, 2.08),
arrowprops=dict(arrowstyle="-|>", color=INK, lw=1.5, mutation_scale=12))
ax.text(3.26, 2.17, "P matches the\ninteraction", fontsize=10, ha="center", va="bottom")
for i, line in enumerate([
"Conflict: the text pairs the main-effect estimate and CI with the interaction P-value.",
f"The text CI implies SE {se_t:.5f}, z ≈ {abs(z_t):.1f} and P ≈ {p2(TEXT['b'], se_t):.0e};",
f"the interaction row gives z ≈ {abs(z_int):.2f} and P ≈ {p2(T2_INT['b'], T2_INT['se']):.1e}."]):
ax.text(0.12, 0.88 - 0.3 * i, line, fontsize=10, color=INK2, va="center")
leg = [Line2D([0], [0], color=ORANGE, lw=2, label="reported values"),
Line2D([0], [0], color=BLUE, lw=2, label="recomputed (beta ± 1.96 SD)")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Abstract reports low birthweight rising from 9.0% to 10.1%; Table 1 shows 8.2% to 8.3%
Int J Epidemiol (2018) · doi:10.1093/ije/dyy106 · PMC6422062
Summary
The abstract of this four-cohort study from Pelotas, Brazil, states that the prevalence of low birthweight (LBW, < 2500 g) "increased from 9.0% to 10.1%" between 1982 and 2015, and that LBW was "higher for boys". The paper's own Table 1 counts give 478/5811 = 8.2% in 1982 and 345/4151 = 8.3% in 2015. No cohort reaches 10.1%, and the abstract's 9.0% matches the 2004 value. The Results, Discussion and Key Messages all describe LBW as stable (P for trend = 0.80, which reproduces exactly from the counts), and Table 2 shows LBW higher for girls in all four cohorts. LBW is one of the two outcomes named in the title, so the abstract reports a change in direction and size, and a sex difference, that the paper's tables and text do not support. The preterm figures in the same abstract sentence check out.
What the paper reports
Abstract, Results (the full sentence):
Preterm births increased from 5.8% (1982) to 13.8% (2015), and LBW prevalence increased from 9.0% to 10.1%, being higher for boys and for children born to mothers with low income and brown or black skin colour.
Abstract, Methods:
Four population-based birth cohort studies were carried out in the city of Pelotas in 1982, 1993, 2004 and 2015, with samples ranging from 4231 to 5914 liveborn children. Low birthweight (LBW) was defined as <2500 g
Methods, statistical analysis paragraph:
All analyses were restricted to live, singleton newborns.
Results, first paragraph:
The numbers of liveborn singletons in the four cohorts were 5816, 5168, 4147 and 4164. Birthweight information was missing for 5 (0.09%), 17 (0.33%), 1 (0.02%) and 13 (0.31%) children, respectively.
Table 1, "<2500" row (1982 | 1993 | 2004 | 2015):
<2500 | 8.2 | 9.1 | 9.0 | 8.3
Results, paragraph on Table 1:
The proportion of newborns with low birthweight remained stable at around 8% to 9% throughout the period (P for trend = 0.80).
Results, paragraph on Table 2:
As expected due to the use of the same absolute cutoff for both sexes, prevalence was higher among girls than boys in the four cohorts. There was no evidence of a time trend in low birthweight for either sex.
Table 2, sex rows (% LBW, 95% CI; 1982 | 1993 | 2004 | 2015 | P for linear trend):
Male | 7.2 (6.3; 8.1) | 8.0 (6.9; 9.0) | 8.1 (7.0; 9.3) | 7.6 (6.5; 8.7) | 0.476
Female | 9.3 (8.2; 10.4) | 10.2 (9.1; 11.4) | 9.9 (8.6; 11.2) | 9.1 (7.8; 10.3) | 0.784
Discussion:
One important finding of our study is that—for the whole population—mean birthweight and LBW prevalence remained stable.
Key Messages box (in the PMC archive copy this box follows the Funding statement):
At the population level, mean birthweight remained stable from 1982 to 2015, as did low birthweight.
What the paper's own numbers show
Method. Table 1 gives the number of live singletons in each 500 g birthweight band, plus the number not weighed. For each cohort I added the four bands below 2500 g and divided by the number weighed (band total, which equals the cohort total minus those not weighed). I recomputed the chi-square test for linear trend over the four cohorts (Cochran-Armitage; the Methods name "the chi-square statistic for linear trend"), the 1982–2015 difference with a Wald 95% CI, and Wilson 95% CIs per cohort. As a control I recomputed the preterm percentages from Table 1's gestational-age counts. I compared the abstract's sex statement with Table 2, and searched every LBW value in Tables 1 and 2 for a 9.0 (1982) to 10.1 (2015) series. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| LBW, 1982 | Abstract; Table 1 | 9.0% (abstract); 8.2 (Table 1) | (17+40+92+329)/5811 = 478/5811 = 8.23% (95% CI 7.55–8.96) |
| LBW, 1993 | Table 1 | 9.1 | 468/5151 = 9.09% (8.33–9.90) |
| LBW, 2004 | Table 1 | 9.0 | 372/4146 = 8.97% (8.14–9.88), the value the abstract gives for 1982 |
| LBW, 2015 | Abstract; Table 1 | 10.1% (abstract); 8.3 (Table 1) | (15+24+56+250)/4151 = 345/4151 = 8.31% (95% CI 7.51–9.19) |
| Weighed + not weighed = cohort total | Table 1 | 5816, 5168, 4147, 4164 | 5811+5, 5151+17, 4146+1, 4151+13: all match |
| LBW over all singletons, including the unweighed | Table 1 | — | 8.22%, 9.06%, 8.97%, 8.29% (no material change) |
| Highest LBW in any cohort | Table 1 | — | 9.09% (1993); no cohort rounds to 10.1% |
| Change 1982 → 2015 | Abstract vs Table 1 | +1.1 points (9.0 → 10.1), a 12.2% relative rise | +0.09 points (95% CI −1.01 to +1.18), a 1.0% relative rise |
| Linear trend, four cohorts | Results | P = 0.80 | χ² = 0.062, Z = 0.25, P = 0.803 (same with ordinal scores) |
| LBW by sex | Abstract vs Table 2 | "higher for boys" (abstract) | Table 2: girls higher in all four cohorts, by 2.1, 2.2, 1.8 and 1.5 points (Table 2 P = 0.003, 0.005, 0.044, 0.089) |
| Any Table 2 row running 9.0 → 10.1 | Table 2 | — | none (girls 9.3 → 9.1; poorest quintile 13.5 → 11.7) |
| Preterm (< 37 weeks), control | Abstract; Table 1 | 5.8% (1982), 13.8% (2015) | 265/4595 = 5.77%; 576/4164 = 13.83%: match |
Table 1, the trend P value in the Results, the Discussion and the Key Messages agree with one another. Only the abstract's LBW figures and its sex statement are out of line. The preterm half of the same abstract sentence reproduces exactly.
Figure
Figure 1. The abstract's Results sentence on LBW (left, orange: text as printed in the abstract) beside the paper's own tables and Results text (right, blue). The upper connector marks the conflict over direction: the abstract says LBW "increased" from 9.0% to 10.1% (+1.1 points), while Table 1 prints 8.2, 9.1, 9.0 and 8.3 and the Results call LBW stable (P for trend = 0.80). Recomputed from the Table 1 counts, LBW is 478/5811 = 8.2% in 1982 and 345/4151 = 8.3% in 2015 (+0.1 points), with trend P = 0.803. The lower connector marks the conflict over sex: the abstract says LBW was higher for boys, while Table 2 gives higher values for girls in every cohort (9.3 vs 7.2 in 1982, 10.2 vs 8.0, 9.9 vs 8.1, 9.1 vs 7.6 in 2015). All values come from the paper or from recompute.py.txt.
Figure 2. LBW prevalence by cohort, recomputed from the Table 1 counts (blue circles joined by a line, with Wilson 95% CIs), and the two values the abstract gives for 1982 and 2015 (orange diamonds). The abstract gives endpoints only, so they are not joined. The y-axis starts at zero. The recomputed series is flat at 8.2–9.1% (trend P = 0.80). The abstract's 1982 value (9.0%) lies just above the 1982 CI (7.55% to 8.96%), and its 2015 value (10.1%) lies above the whole 2015 CI (7.51% to 9.19%) and above every cohort.
Why it matters
LBW is one of the two outcomes in the title, and the abstract is the part of the paper most readers, indexers and evidence summaries use. The abstract tells them LBW rose by 1.1 percentage points (12% in relative terms) over 33 years. The paper's own data and its own conclusions say LBW did not change (+0.09 points, P = 0.80). A reader who takes the abstract at face value would cite a rising LBW trend in southern Brazil, which this paper does not show. The sex statement is reversed: the paper explains the female excess as expected "due to the use of the same absolute cutoff for both sexes", and the abstract says the opposite.
The paper's main conclusions are not affected. The preterm increase (5.8% to 13.8%), the stable mean birthweight, the income-group trends in mean birthweight, the adjusted 160 g decline and the caesarean-section interpretation are all stated correctly in the abstract and body. A flat LBW series is also what the abstract's own Background and Conclusions imply ("did not show reductions", "offset the expected improvements"), so the correct figures fit the paper's argument better than the printed ones. The LBW gradients by income and skin colour in the same abstract sentence are broadly consistent with Table 2 and were not examined further here.
Explanations considered
- A different denominator (excluding vs including unweighed babies). LBW over weighed newborns gives 8.23% and 8.31%; over all live singletons, 8.22% and 8.29%. Neither gives 9.0% or 10.1%. Does not explain it.
- A different population: all live births, including multiples. The abstract's Methods describe "samples ranging from 4231 to 5914 liveborn children", larger than the singleton counts, and multiples are mostly LBW. The paper does not report non-singleton counts or their birthweights, so this cannot be fully excluded from the paper's numbers alone. To reach 10.1% in 2015 would need at least 83 extra live births, every one of them LBW. If the 5914 maximum is the 1982 cohort (98 non-singletons), reaching 9.0% would need about 55% of them to be LBW. Against this, the Methods say "All analyses were restricted to live, singleton newborns", the paper reports no all-births LBW rates or trend test, and it states that LBW was stable. Even if the abstract figures came from an all-births tabulation, the abstract would still report a population and a trend that the paper's methods exclude and that its results contradict. This would also not explain the sex statement.
- The earlier three-cohort report. The paper cites its own 2008 report on the 1982, 1993 and 2004 cohorts (Cad Saude Publica 2008;24 Suppl 3:S390–S398), whose abstract says "We found a slight increase in the period in the low birth weight prevalence from 9% to 10%". The abstract's 9.0% → 10.1% resembles that earlier description, but this paper's tables and text use different figures. This may account for where the wording came from; it does not reconcile it with this paper's data.
- Gestational-age method (LMP vs best obstetric estimate). This changes only the preterm figures (the LMP-only reanalysis gives 5.8%, 11.2%, 16.2%, 17.7%). Birthweight does not depend on it. Does not explain it.
- A subgroup. No Table 2 row runs from 9.0 in 1982 to 10.1 in 2015 (girls 9.3 → 9.1, poorest quintile 13.5 → 11.7, black mothers 10.3 → 8.2). Does not explain it.
- A rounding or transcription issue in Table 1. The printed percentages reproduce from the counts to one decimal place, the band totals plus the unweighed equal the stated cohort sizes, and the trend P reproduces to 0.803. The table is internally consistent; the discrepancy is between the abstract and the table.
- A text-conversion artefact. The same abstract sentence appears in the publisher's version of record, Europe PMC full text, PubMed and the PMC archive copy. Not a conversion artefact.
Status
As of 2026-10-23:
- Version of record. The DOI resolves to the publisher's page (Int J Epidemiol 48 Suppl 1:i46–i53; online June 2018, print issue April 2019). Its abstract contains the same sentence ("LBW prevalence increased from 9.0% to 10.1%, being higher for boys ..."), and its Table 1 gives 8.2, 9.1, 9.0 and 8.3. The same text appears in Europe PMC full-text XML, the PubMed record (PMID 29939270) and the PMC archive copy.
- Supplements. The PMC/Europe PMC record has no supplementary material (Europe PMC hasSuppl = N). The publisher page lists a "Supplementary Data" entry with no files linked. No source data are available. The full PDF is behind a subscription and was not opened.
- Corrections. None found. Crossref shows no update-to, updated-by or relation entries; PMC and Europe PMC list no correction, erratum or retraction notices; the only linked item is a commentary on the supplement (doi:10.1093/ije/dyy214, "Commentary: A tale of many cities in one"), which is not an erratum. Web searches for a correction, erratum or corrigendum, and Retraction Watch, returned nothing relevant.
- PubPeer. Could not be checked: both the API and the search page returned HTTP 403.
- Sources. https://doi.org/10.1093/ije/dyy106; https://academic.oup.com/ije/article/48/Supplement_1/i46/5042986; Europe PMC REST full text for PMC6422062; PubMed efetch for PMID 29939270; Crossref API; Europe PMC record for doi:10.1590/s0102-311x2008001500004 (the 2008 report's abstract).
Suggested correction
In the abstract's Results, replace "and LBW prevalence increased from 9.0% to 10.1%, being higher for boys and for children born to mothers with low income and brown or black skin colour" with wording that matches Table 1, Table 2 and the Results, for example: "while LBW prevalence remained stable (8.2% in 1982 and 8.3% in 2015; P for trend = 0.80), being higher for girls and for children born to mothers with low income and brown or black skin colour." If the 9.0% and 10.1% figures come from a different population, such as all live births including multiples, the authors could say so, give that population's counts, and explain why it differs from the singleton population that the Methods specify for all analyses.
Recomputation script (python recompute.py.txt)
"""Recompute the low-birthweight (LBW) trend reported in PMC6422062
(Int J Epidemiol 2018, doi:10.1093/ije/dyy106).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png, fig2.svg/png and five candidate figure designs
(candidates/candidate1..5.svg/png) next to this file.
Inputs are copied from the paper as archived in PubMed Central:
Table 1: birthweight distribution (n per 500 g band, not weighed), <2500 % row,
gestational-age distribution, number of children
Table 2: LBW prevalence (95% CI) by sex of the newborn
Abstract (Results): "LBW prevalence increased from 9.0% to 10.1%, being higher for boys"
Results text: "remained stable at around 8% to 9% throughout the period (P for trend = 0.80)"
"""
import math
import os
import numpy as np
from scipy.stats import chi2
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
YEARS = [1982, 1993, 2004, 2015]
BANDS = ["<1000", "1000-1499", "1500-1999", "2000-2499",
"2500-2999", "3000-3499", "3500-3999", ">=4000"]
BW = { # Table 1, n per birthweight band
1982: [17, 40, 92, 329, 1361, 2211, 1416, 345],
1993: [11, 24, 82, 351, 1283, 2040, 1080, 280],
2004: [24, 29, 70, 249, 1016, 1648, 912, 198],
2015: [15, 24, 56, 250, 960, 1713, 919, 214],
}
NOT_WEIGHED = {1982: 5, 1993: 17, 2004: 1, 2015: 13} # Table 1 "Not weighed"
N_CHILDREN = {1982: 5816, 1993: 5168, 2004: 4147, 2015: 4164} # Table 1 / Results text
T1_LBW = {1982: 8.2, 1993: 9.1, 2004: 9.0, 2015: 8.3} # Table 1 "<2500" row
GA = { # Table 1 gestational age: <37, 37-38, 39-41, 42+
1982: [265, 1007, 2854, 469], 1993: [517, 906, 2661, 518],
2004: [567, 1244, 2064, 267], 2015: [576, 1562, 1996, 30],
}
T1_PRETERM = {1982: 5.8, 1993: 11.2, 2004: 13.7, 2015: 13.8}
ABS_LBW = (9.0, 10.1) # Abstract: "increased from 9.0% to 10.1%"
ABS_PRETERM = (5.8, 13.8) # Abstract: "from 5.8% (1982) to 13.8% (2015)"
RESULTS_P_TREND = 0.80 # Results: "(P for trend = 0.80)"
T2_SEX = { # Table 2: % LBW (95% CI)
"Male": [(7.2, 6.3, 8.1), (8.0, 6.9, 9.0), (8.1, 7.0, 9.3), (7.6, 6.5, 8.7)],
"Female": [(9.3, 8.2, 10.4), (10.2, 9.1, 11.4), (9.9, 8.6, 11.2), (9.1, 7.8, 10.3)],
}
T2_SEX_P = [0.003, 0.005, 0.044, 0.089] # Table 2, P for sex difference per cohort
# Every other Table 2 row (1982 and 2015 values), to search for a 9.0 -> 10.1 pair
T2_OTHER = {"Q1": (13.5, 11.7), "Q2": (8.6, 8.0), "Q3": (6.9, 6.8), "Q4": (6.6, 8.4),
"Q5": (5.7, 6.7), "White": (7.8, 8.5), "Brown": (10.3, 7.7), "Black": (10.3, 8.2),
"Male": (7.2, 7.6), "Female": (9.3, 9.1)}
def wilson(k, n, z=1.959964):
p = k / n
d = 1 + z * z / n
c = (p + z * z / (2 * n)) / d
h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
return 100 * (c - h), 100 * (c + h)
# ---------------------------------------------------------------- 1. LBW per cohort
print("== 1. LBW (<2500 g) per cohort, from Table 1 counts ==")
lbw, wt, lbw_pct, lbw_ci = {}, {}, {}, {}
for y in YEARS:
b = BW[y]
wt[y] = sum(b)
lbw[y] = sum(b[:4])
lbw_pct[y] = 100 * lbw[y] / wt[y]
lbw_ci[y] = wilson(lbw[y], wt[y])
print(f"{y}: weighed {wt[y]} (+{NOT_WEIGHED[y]} not weighed = {wt[y] + NOT_WEIGHED[y]}, "
f"Table 1 N {N_CHILDREN[y]}); LBW n = {'+'.join(map(str, b[:4]))} = {lbw[y]}; "
f"{lbw[y]}/{wt[y]} = {lbw_pct[y]:.2f}% (Wilson 95% CI {lbw_ci[y][0]:.2f}-{lbw_ci[y][1]:.2f}); "
f"Table 1 prints {T1_LBW[y]}; over all {N_CHILDREN[y]} children = {100 * lbw[y] / N_CHILDREN[y]:.2f}%")
assert wt[y] + NOT_WEIGHED[y] == N_CHILDREN[y]
assert round(lbw_pct[y], 1) == T1_LBW[y]
print(f"Highest cohort value: {max(lbw_pct.values()):.2f}% ({max(lbw_pct, key=lbw_pct.get)})")
print(f"Abstract 9.0% equals the 2004 value ({lbw_pct[2004]:.2f}% -> {round(lbw_pct[2004], 1)}); "
f"1982 is {lbw_pct[1982]:.2f}%")
print("Cohorts whose LBW rounds to 10.1%:", [y for y in YEARS if round(lbw_pct[y], 1) == 10.1] or "none")
# ---------------------------------------------------------------- 2. change 1982 -> 2015
print("\n== 2. Change 1982 -> 2015 ==")
p1, n1 = lbw[1982] / wt[1982], wt[1982]
p2, n2 = lbw[2015] / wt[2015], wt[2015]
rd = 100 * (p2 - p1)
se = 100 * math.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
rd_ci = (rd - 1.959964 * se, rd + 1.959964 * se)
print(f"Recomputed change: {rd:+.2f} percentage points (95% CI {rd_ci[0]:+.2f} to {rd_ci[1]:+.2f})")
print(f"Abstract change: {ABS_LBW[1] - ABS_LBW[0]:+.1f} percentage points (9.0 -> 10.1)")
print(f"Relative change recomputed {100 * (p2 / p1 - 1):+.1f}%; abstract {100 * (ABS_LBW[1] / ABS_LBW[0] - 1):+.1f}%")
print(f"The abstract's +1.1 lies {'inside' if rd_ci[0] <= ABS_LBW[1] - ABS_LBW[0] <= rd_ci[1] else 'outside'} "
"this interval, so the issue is the printed endpoint values, not sampling error in the difference")
# ---------------------------------------------------------------- 3. trend test
print("\n== 3. Chi-square for linear trend (Cochran-Armitage) over the four cohorts ==")
r = np.array([lbw[y] for y in YEARS], float)
n = np.array([wt[y] for y in YEARS], float)
pbar = r.sum() / n.sum()
for label, x in (("calendar-year scores", np.array(YEARS, float)), ("ordinal scores 0-3", np.arange(4.0))):
xb = (n * x).sum() / n.sum()
T = (r * (x - xb)).sum()
V = pbar * (1 - pbar) * (n * (x - xb) ** 2).sum()
print(f"{label}: chi2 = {T * T / V:.3f}, Z = {T / math.sqrt(V):.2f}, P = {chi2.sf(T * T / V, 1):.3f} "
f"(Results prints P = {RESULTS_P_TREND})")
print(f"Pooled LBW over all cohorts: {int(r.sum())}/{int(n.sum())} = {100 * pbar:.2f}%")
# ---------------------------------------------------------------- 4. preterm (control: abstract matches here)
print("\n== 4. Preterm (<37 weeks), from Table 1 gestational-age counts ==")
for y in YEARS:
g = GA[y]
print(f"{y}: {g[0]}/{sum(g)} = {100 * g[0] / sum(g):.2f}% (Table 1 prints {T1_PRETERM[y]}); "
f"gestational age unknown for {N_CHILDREN[y] - sum(g)} of {N_CHILDREN[y]}")
print(f"Abstract preterm endpoints {ABS_PRETERM}: match Table 1 -> "
f"{(round(100 * GA[1982][0] / sum(GA[1982]), 1), round(100 * GA[2015][0] / sum(GA[2015]), 1)) == ABS_PRETERM}")
# ---------------------------------------------------------------- 5. sex
print("\n== 5. LBW by sex (Table 2) vs abstract 'higher for boys' ==")
for i, y in enumerate(YEARS):
m, f = T2_SEX["Male"][i][0], T2_SEX["Female"][i][0]
print(f"{y}: male {m}%, female {f}% -> higher for {'girls' if f > m else 'boys'} "
f"(difference {f - m:+.1f} points; Table 2 P = {T2_SEX_P[i]})")
print("Girls higher in all four cohorts:",
all(T2_SEX["Female"][i][0] > T2_SEX["Male"][i][0] for i in range(4)))
# ---------------------------------------------------------------- 6. search for 9.0 -> 10.1
print("\n== 6. Does any Table 1 or Table 2 series run from 9.0 (1982) to 10.1 (2015)? ==")
hits = [k for k, (a, b) in T2_OTHER.items() if abs(a - 9.0) < 0.05 and abs(b - 10.1) < 0.05]
print("Table 2 rows with 1982 = 9.0 and 2015 = 10.1:", hits or "none")
allvals = [v for row in T2_SEX.values() for v, *_ in row] + list(T1_LBW.values()) + \
[v for pair in T2_OTHER.values() for v in pair]
print("Any Table 1/2 LBW value equal to 10.1:", [v for v in allvals if abs(v - 10.1) < 0.05] or "none")
# ---------------------------------------------------------------- 7. could non-singletons explain 9.0 / 10.1?
print("\n== 7. Abstract Methods: 'samples ranging from 4231 to 5914 liveborn children' (all live births) ==")
print("The paper does not give non-singleton counts per cohort. Minimum number e of extra (non-singleton)")
print("live births, ALL of them LBW, needed to lift each cohort to the abstract value:")
for y, target in ((1982, ABS_LBW[0]), (2015, ABS_LBW[1])):
t = target / 100
e_min = (t * wt[y] - lbw[y]) / (1 - t)
print(f"{y}: to reach {target}% needs e >= {math.ceil(e_min)} extra live births, all LBW")
e82 = 5914 - N_CHILDREN[1982]
need = 0.090 * (wt[1982] + e82) - lbw[1982]
print(f"If the 5914 maximum is the 1982 cohort: {e82} non-singletons; reaching 9.0% needs about "
f"{need:.0f} of them ({100 * need / e82:.0f}%) to be LBW; all {e82} LBW would give "
f"{100 * (lbw[1982] + e82) / (wt[1982] + e82):.2f}%")
print("So a different (all-births) denominator cannot be excluded from the paper's numbers alone,")
print("but the Methods restrict all analyses to singletons and no such rates are reported.")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
ORANGE_BG, BLUE_BG = "#fdf0ea", "#eaf2fb"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"xtick.labelsize": 10, "ytick.labelsize": 10, "legend.fontsize": 10,
})
def save(fig, path_stem):
fig.savefig(path_stem + ".svg")
fig.savefig(path_stem + ".png", dpi=200)
plt.close(fig)
def clean(ax, grid_axis="y"):
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.spines["left"].set_color(GRID)
ax.spines["bottom"].set_color(SEC)
ax.grid(axis=grid_axis, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.tick_params(length=0)
def box(ax, x, y, w, h, edge, face, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
lw=2, ec=edge, fc=face, transform=ax.transAxes))
ax.text(x + 0.022, y + h - 0.035, title, transform=ax.transAxes, fontsize=11.5,
fontweight="bold", va="top", color=TXT)
def txt(ax, x, y, s, **kw):
kw.setdefault("fontsize", 10.5)
ax.text(x, y, s, transform=ax.transAxes, va="top", color=kw.pop("color", TXT), linespacing=1.4, **kw)
# ---- candidate 1: two text panels joined by annotated connectors (abstract vs body)
def cand_text_panels(stem):
fig = plt.figure(figsize=(7.2, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
txt(ax, 0.03, 0.97, "One abstract sentence, two conflicts with the paper's own table and text",
fontsize=12, fontweight="bold")
txt(ax, 0.03, 0.915, "LBW = birthweight < 2500 g; live singletons; Pelotas birth cohorts 1982-2015",
fontsize=10, color=SEC)
L, R, W, B, H = 0.03, 0.59, 0.38, 0.12, 0.74
box(ax, L, B, W, H, ORANGE, ORANGE_BG, "Abstract, Results")
box(ax, R, B, W, H, BLUE, BLUE_BG, "Table 1, Table 2, Results")
y1, y2 = 0.74, 0.40
txt(ax, L + 0.022, y1, "\"... and LBW prevalence\nincreased from 9.0%\nto 10.1%,")
txt(ax, L + 0.022, y2, "being higher for boys and\nfor children born to\nmothers with low income\n"
"and brown or black skin\ncolour.\"")
txt(ax, R + 0.022, y1, f"Table 1, '<2500' row (%):\n1982 {T1_LBW[1982]} 1993 {T1_LBW[1993]}\n"
f"2004 {T1_LBW[2004]} 2015 {T1_LBW[2015]}\n"
"Results: \"remained stable\nat around 8% to 9% ...\n(P for trend = 0.80)\"")
txt(ax, R + 0.022, y2, "Table 2, LBW % girls vs boys:\n"
+ "\n".join(f"{y} {T2_SEX['Female'][i][0]} vs {T2_SEX['Male'][i][0]}"
for i, y in enumerate(YEARS)))
for yy, lab in ((y1 - 0.06, "'increased'\n+1.1 pts\nvs stable\n+0.1 pts"),
(y2 - 0.06, "'boys'\nvs girls\nhigher in\nall four")):
ax.add_patch(FancyArrowPatch((L + W, yy), (R, yy), transform=ax.transAxes,
arrowstyle="<->", mutation_scale=14, lw=2, color=SEC))
ax.text((L + W + R) / 2, yy - 0.025, lab, transform=ax.transAxes, ha="center", va="top",
fontsize=10, color=TXT, linespacing=1.25)
txt(ax, 0.03, 0.085, "Orange: text as printed in the abstract. Blue: the paper's tables and Results text;\n"
f"Table 1 recomputes from its counts ({lbw[1982]}/{wt[1982]} = {lbw_pct[1982]:.1f}%, "
f"{lbw[2015]}/{wt[2015]} = {lbw_pct[2015]:.1f}%; trend P = 0.803).",
fontsize=10, color=SEC)
save(fig, stem)
# ---- candidate 2: LBW by cohort, recomputed with 95% CI, vs the abstract's two endpoints
def cand_trend(stem):
fig, ax = plt.subplots(figsize=(7.2, 4.2))
fig.subplots_adjust(left=0.10, right=0.76, top=0.84, bottom=0.12)
clean(ax)
xs = np.array(YEARS)
ys = np.array([lbw_pct[y] for y in YEARS])
lo = ys - np.array([lbw_ci[y][0] for y in YEARS]); hi = np.array([lbw_ci[y][1] for y in YEARS]) - ys
ax.errorbar(xs, ys, yerr=[lo, hi], fmt="o-", color=BLUE, lw=2, ms=8, capsize=0, elinewidth=2,
mec=BG, mew=1.5, zorder=3)
for x_, y_, l0 in zip(xs, ys, lo):
ax.text(x_, y_ - l0 - 0.2, f"{y_:.1f}", color=TXT, fontsize=10, va="top", ha="center")
ax.plot([1982, 2015], ABS_LBW, "D", color=ORANGE, ms=9, mec=BG, mew=1.5, zorder=4, ls="none")
for x_, v in zip((1982, 2015), ABS_LBW):
ax.text(x_ - 1.0, v, f"{v}", fontsize=10, va="center", ha="right")
ax.text(2016.8, ABS_LBW[1] + 0.3, "Abstract: \"increased\nfrom 9.0% to 10.1%\"\n(endpoints only)",
fontsize=10, va="center")
ax.text(2016.8, lbw_pct[2015] - 0.5, "Table 1 counts,\nrecomputed, 95% CI;\ntrend P = 0.80",
fontsize=10, va="top")
ax.set_ylim(0, 12); ax.set_xlim(1978, 2017)
ax.set_xticks(YEARS)
ax.set_ylabel("Low birthweight (%)")
fig.text(0.10, 0.955, "Low birthweight by cohort: the abstract vs the paper's own counts",
fontsize=12, fontweight="bold", va="top")
fig.text(0.10, 0.9, "Live singletons weighed at birth; the y-axis starts at zero", fontsize=10,
color=SEC, va="top")
ax.legend(handles=[Line2D([], [], color=ORANGE, marker="D", ls="none", ms=9, label="Reported in abstract"),
Line2D([], [], color=BLUE, marker="o", lw=2, ms=8, label="Recomputed from Table 1")],
loc="lower left", frameon=False, ncol=2)
save(fig, stem)
# ---- candidate 3: dumbbell of the 1982 -> 2015 change
def cand_dumbbell(stem):
fig, ax = plt.subplots(figsize=(7.2, 3.2))
fig.subplots_adjust(left=0.25, right=0.96, top=0.78, bottom=0.2)
clean(ax, "x")
rows = [("Abstract", ABS_LBW, ORANGE, "D", "change +1.1 points"),
("Table 1, recomputed", (lbw_pct[1982], lbw_pct[2015]), BLUE, "o",
f"change {rd:+.1f} points\n(95% CI {rd_ci[0]:+.1f} to {rd_ci[1]:+.1f})")]
for i, (lab, (a, b), c, m, ch) in enumerate(rows):
yy = 1 - i
ax.plot([a, b], [yy, yy], color=c, lw=2, zorder=2)
ax.plot([a], [yy], m, color=c, ms=9, mfc=BG, mew=2, zorder=3)
ax.plot([b], [yy], m, color=c, ms=9, mec=BG, mew=1.5, zorder=4)
ax.text(min(a, b) - 0.25, yy, f"1982 {a:.1f}", ha="right", va="center", fontsize=10)
ax.text(max(a, b) + 0.25, yy, f"2015 {b:.1f}", ha="left", va="center", fontsize=10)
ax.text(0.3, yy, ch, ha="left", va="center", fontsize=10, color=SEC)
ax.set_yticks([1, 0]); ax.set_yticklabels([r[0] for r in rows], fontsize=10.5, color=TXT)
ax.set_xlim(0, 12.5); ax.set_ylim(-0.6, 1.6)
ax.set_xlabel("Low birthweight (%); the axis starts at zero")
fig.text(0.03, 0.95, "Change in low birthweight, 1982 to 2015", fontsize=12, fontweight="bold", va="top")
fig.text(0.03, 0.88, "Open marker 1982, filled marker 2015", fontsize=10, color=SEC, va="top")
save(fig, stem)
# ---- candidate 4: sex panel (Table 2) vs the abstract's 'higher for boys'
def cand_sex(stem):
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.subplots_adjust(left=0.10, right=0.78, top=0.78, bottom=0.12)
clean(ax)
off = {"Female": 0.8, "Male": -0.8}
style = {"Female": dict(color=TXT, marker="o"), "Male": dict(color=SEC, marker="s")}
for sx, rows in T2_SEX.items():
xs = np.array(YEARS) + off[sx]
v = np.array([r_[0] for r_ in rows]); l_ = np.array([r_[1] for r_ in rows]); h_ = np.array([r_[2] for r_ in rows])
ax.errorbar(xs, v, yerr=[v - l_, h_ - v], fmt=style[sx]["marker"], color=style[sx]["color"],
ms=8, elinewidth=2, capsize=0, mfc=style[sx]["color"] if sx == "Female" else BG, mew=2)
ax.text(2017.5, v[-1], f"{'Girls' if sx == 'Female' else 'Boys'} (Table 2)", va="center", fontsize=10)
ax.set_xticks(YEARS); ax.set_xlim(1978, 2017); ax.set_ylim(0, 12)
ax.set_ylabel("Low birthweight (%), 95% CI")
fig.text(0.10, 0.96, "Table 2: LBW is higher for girls in all four cohorts", fontsize=12,
fontweight="bold", va="top")
fig.text(0.105, 0.885, "Abstract: \"LBW prevalence ... being higher for boys\"", fontsize=10.5,
va="top", bbox=dict(boxstyle="round,pad=0.3", fc=ORANGE_BG, ec=ORANGE, lw=1.5))
ax.text(1978.3, 1.6, "P, girls vs boys:", fontsize=10, color=SEC, va="center")
for i, y in enumerate(YEARS):
ax.text(y, 0.8, f"{T2_SEX_P[i]}", ha="center", fontsize=10, color=SEC, va="center")
save(fig, stem)
# ---- candidate 5: calculation flow from Table 1 counts to the percentages
def cand_flow(stem):
fig = plt.figure(figsize=(7.2, 4.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.03, 0.955, "From Table 1 counts to the LBW percentage", fontsize=12, fontweight="bold", va="top")
cols = [(0.06, "Table 1 counts\n< 2500 g"), (0.32, "Weighed\nnewborns"),
(0.53, "LBW %\nrecomputed"), (0.75, "Abstract")]
for x, h in cols:
ax.text(x, 0.85, h, fontsize=10.5, fontweight="bold", va="top", color=TXT)
for j, y in enumerate((1982, 2015)):
yy = 0.58 - j * 0.27
ax.text(0.03, yy + 0.1, str(y), fontsize=10.5, fontweight="bold", color=SEC)
ax.text(0.06, yy, "+".join(map(str, BW[y][:4])) + f"\n= {lbw[y]}", fontsize=10.5, va="center")
ax.text(0.32, yy, f"{N_CHILDREN[y]} - {NOT_WEIGHED[y]}\n= {wt[y]}", fontsize=10.5, va="center")
ax.text(0.53, yy, f"{lbw[y]}/{wt[y]}\n= {lbw_pct[y]:.2f}%", fontsize=10.5, va="center",
bbox=dict(boxstyle="round,pad=0.35", fc=BLUE_BG, ec=BLUE, lw=2))
ax.text(0.75, yy, f"{ABS_LBW[j]}%", fontsize=12, va="center",
bbox=dict(boxstyle="round,pad=0.35", fc=ORANGE_BG, ec=ORANGE, lw=2))
ax.text(0.71, yy, "vs", fontsize=10.5, va="center", ha="center", color=SEC)
for x0, x1 in ((0.255, 0.305), (0.465, 0.515)):
ax.annotate("", xy=(x1, yy), xytext=(x0, yy), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=SEC, lw=1.5))
ax.text(0.86, yy, f"{ABS_LBW[j] - lbw_pct[y]:+.1f} pts\nvs Table 1", fontsize=10, va="center", color=SEC)
ax.text(0.03, 0.12, f"Table 1 itself prints {T1_LBW[1982]} and {T1_LBW[2015]}. The abstract's 9.0% "
f"equals the 2004 value\n({lbw[2004]}/{wt[2004]} = {lbw_pct[2004]:.2f}%); no cohort reaches 10.1% "
f"(highest: {max(lbw_pct.values()):.1f}% in 1993).", fontsize=10, color=SEC, va="top", linespacing=1.4)
save(fig, stem)
builders = [cand_text_panels, cand_trend, cand_dumbbell, cand_sex, cand_flow]
for i, fn in enumerate(builders, 1):
fn(os.path.join(CAND, f"candidate{i}"))
# Default selection for writeup.md until a design is chosen: fig1 = candidate 1, fig2 = candidate 2
cand_text_panels(os.path.join(HERE, "fig1"))
cand_trend(os.path.join(HERE, "fig2"))
print("\nWrote fig1, fig2 and candidates/candidate1-5 (.svg, .png) in", HERE)
Abstract attributes the all-EGFR c-Met-high response rate (10/19, 52.6%) to the EGFR-mutant subgroup (n = 15)
J Clin Oncol (2022) · doi:10.1200/JCO.22.00739 · PMC9928626
Summary
The abstract reports that EGFR-mutation-positive (EGFR-M+) patients with c-Met-high tumors (H-score ≥ 225; n = 15) had an objective response rate (ORR) of 52.6%. No whole number of responders out of 15 gives 52.6% (7/15 = 46.7%, 8/15 = 53.3%). The figure is exactly 10/19, which the Results give as the ORR for all 19 c-Met-high patients, a group that includes three EGFR wild-type responders. The Results' own breakdown puts 7 responders among the 15 EGFR-M+ c-Met-high patients, an ORR of 46.7% (exact 95% CI 21.3 to 73.4). This is the abstract's only efficacy estimate for the subgroup the paper presents as of greatest clinical interest, and it overstates that estimate by 5.9 percentage points.
What the paper reports
Abstract, Results:
ORR for EGFR-M+ patients (n = 28) was 32.1%. Of EGFR-M+ patients, those who were c-Met high (n = 15) had an ORR of 52.6%.
Results, Efficacy, fourth paragraph (the paragraph describing Figure 2):
Among patients with c-Met–high scores (n = 19), one achieved a CR (1/19; 5.3%; EGFR-M+), nine achieved a PR (9/19; 47.4%; six EGFR-M+ and three EGFR-WT), five had SD (5/19; 26.3%; EGFR-M+), and four experienced progressive disease (4/19; 21.1%; three EGFR-M+ and one rare/unknown EGFR), indicating an ORR of 52.6% and DCR of 78.9%. No patients with lower c-Met scores achieved a CR; two of 16 had a PR (12.5%), 12 had SD (75%), and two had progressive disease (12.5%), indicating an ORR of 12.5% and DCR of 87.5%.
Results, Efficacy, second and third paragraphs:
For EGFR-M+ patients (n = 28), ORR was 32.1% (9/28; 95% CI, 15.9 to 52.4), with one CR (3.6%) and eight PR (28.6%).
ORR in EGFR-WT patients (n = 5) was 40% (2/5; two PR) and DCR was 80% (4/5; two PR and two SD). All three patients with rare/unknown EGFR mutations achieved SD.
Patients and Methods, Assessments, Efficacy:
ORR was defined as the proportion of patients with confirmed complete response (CR) or partial response (PR).
Results, Patient Characteristics:
Importantly, 53% were c-Met high (H-score ≥ 225).
Figure 2 legend:
HI indicates c-Met high (≥ 225; n = 19).
Context box, Relevance:
The subsets of greatest clinical interest are the patients with EGFR-M+ NSCLC who progressed on a prior EGFR TKI, and with high c-Met H-scores by immunohistochemistry.
Discussion, first paragraph:
This combination also showed promising efficacy, especially in c-Met–high EGFR-M+ patients, in agreement with each drug's mechanism of action.
What the paper's own numbers show
Method. I checked which response counts out of 15 could produce the abstract's 52.6%, and which denominators up to 40 could produce it at all. I then rebuilt the c-Met-high group from the Results breakdown by EGFR group and computed the ORR of its EGFR-M+ column, with the exact (Clopper-Pearson) interval the paper says it uses. As a check, I recomputed every ORR and disease control rate (DCR) in Table 3 and Table 4 from their printed counts, and tallied the per-patient labels in Figure 2 (EGFR group colour, c-Met ≥ 225 hatching, MET-amplification asterisk and best response, transcribed by eye). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| ORR, EGFR-M+ and c-Met high (n = 15) | Abstract, Results | 52.6% | 7/15 = 46.7% (exact 95% CI 21.3 to 73.4); 1 CR + 6 PR from the Results breakdown |
| Possible ORRs with n = 15 | (derived) | — | 7/15 = 46.7%, 8/15 = 53.3%; none rounds to 52.6% (52.6% × 15 = 7.89) |
| Fractions with denominator ≤ 40 that give 52.6% | (derived) | — | 10/19 and 20/38 only |
| ORR, all c-Met high (n = 19) | Results, Efficacy p4 | 52.6% | (1 + 9)/19 = 52.6% (CI 28.9 to 75.6); matches |
| DCR, all c-Met high | Results, Efficacy p4 | 78.9% | 15/19 = 78.9%; matches |
| Size of the EGFR-M+ c-Met-high group | Results, Efficacy p4 | n = 15 (Abstract) | 1 CR + 6 PR + 5 SD + 3 PD = 15; matches the abstract's n |
| DCR, EGFR-M+ and c-Met high | (derived) | not reported | 12/15 = 80.0% |
| Responders in the c-Met-high group who are not EGFR-M+ | Results, Efficacy p4 | three EGFR-WT PR | 10 − 7 = 3 |
| Upper limit for the EGFR-M+ c-Met-high ORR | Table 3 (9 EGFR-M+ responders in all) | — | 9/15 = 60.0%, if every EGFR-M+ responder were c-Met high |
| EGFR-M+ c-Met-high responders in Figure 2 | Figure 2 labels | — | 1 CR + 6 PR = 7 among 14 plotted (7/14 = 50.0%) |
| ORR, c-Met lower (n = 16) | Results, Efficacy p4 | 12.5% | 2/16 = 12.5% (CI 1.6 to 38.3); matches |
| c-Met-high share of efficacy set | Patient Characteristics | 53% | 19/36 = 52.8%; matches |
| ORR / DCR, EGFR-M+ (n = 28) | Table 3; Results p2 | 32.1% [15.9 to 52.4] / 85.7% [67.3 to 96.0] | 9/28 and 24/28; same values and CIs |
| ORR / DCR, EGFR-WT (n = 5) | Table 3 | 40.0% [5.3 to 85.3] / 80.0% [28.4 to 99.5] | 2/5 and 4/5; same values and CIs |
| ORR / DCR, rare/unknown (n = 3) | Table 3 | 0 [0.0 to 70.8] / 100 [29.2 to 100] | 0/3 and 3/3; same values and CIs |
| ORR / DCR, total (n = 36) | Table 3; Results p1 | 30.6% [16.3 to 48.1] / 86.1% [70.5 to 95.3] | 11/36 and 31/36; same values and CIs |
| Table 4 subgroup ORR / DCR | Table 4; Results p2-p3 | 31/77, 33/93, 27/73, 39/100 | 4/13, 10/13; 5/15, 14/15; 4/15, 11/15; 5/13, 13/13; all match |
All of the paper's other response rates reproduce from their printed counts, including every Clopper-Pearson interval in Table 3. The abstract's 52.6% is the only one that cannot be formed from its stated denominator. Its digits equal the all-EGFR c-Met-high rate (10/19) given in the Results, and the 15 is the size of the EGFR-M+ column of that same breakdown. The abstract thus pairs the n = 19 rate with the n = 15 label.
The figure for the EGFR-M+ c-Met-high subgroup does not depend on whether responses were confirmed. Table 3 (confirmed responses) and Figure 2 (best response) both show 9 EGFR-M+ responders in the whole cohort (1 CR, 8 PR). Seven of them carry the c-Met ≥ 225 hatching in Figure 2, the same 7 the Results paragraph lists.
Figure
Figure 1. Where the abstract's number comes from. Top left: the abstract sentence, with the value and the subgroup size it states in orange. Top right: the Results breakdown of the 19 c-Met-high patients by best response and EGFR group (counts as printed in the text). Bottom: the two response rates that breakdown supports, in blue. Pooling all columns gives (1 + 9)/19 = 52.6%, the value the Results report for all c-Met-high patients and the value the abstract prints. Using only the EGFR-M+ column, the group of 15 the abstract names, gives (1 + 6)/15 = 46.7%. No count out of 15 can give 52.6%.
Figure 2. Objective response rates on one axis, with exact 95% CIs. Blue dots and bars are recomputed from the paper's counts; the orange diamond is the value the abstract reports for EGFR-M+, c-Met-high patients. The grey ticks above the first row mark every rate that 15 patients can produce (k/15); 52.6% falls between 7/15 and 8/15. The thin orange line at 52.6% passes through the all-c-Met-high estimate (10/19), not the EGFR-M+ one. The correction moves the subgroup estimate down by 5.9 points. It remains well above the rate in patients with lower c-Met scores (2/16) and above the rate in all EGFR-M+ patients (9/28), with wide, overlapping intervals.
Why it matters
- It is the abstract's headline estimate for the paper's focal subgroup. The Context box names EGFR-M+ patients with high c-Met H-scores as "the subsets of greatest clinical interest", and the Discussion says the combination "showed promising efficacy, especially in c-Met–high EGFR-M+ patients". The abstract gives one number for that subgroup, 52.6%, which reads as "more than half respond". The paper's own counts give 7 of 15, 46.7%, which is "slightly under half".
- The number has been repeated as stated. A trade-press report of the paper (OncLive) gives "For c-MET high, EGFR-mutated patients (n = 15), the ORR was 52.6%." Readers who rely on the abstract or on secondary summaries get a subgroup estimate that is 5.9 points too high.
- The direction of the finding is unchanged. At 46.7%, EGFR-M+ c-Met-high patients still respond more often than all EGFR-M+ patients (32.1%) and much more often than patients with lower c-Met scores (12.5%). No formal test in the paper uses the 52.6%, and the abstract's conclusion of "encouraging antitumor activity and acceptable toxicity" does not depend on it. With n = 15 the interval is wide either way (21.3 to 73.4 for 7/15).
- The c-Met-stratified counts do not fully reconcile with the confirmed-response tables, so the confirmed subgroup rate may differ again. The c-Met paragraph totals 19 + 16 = 35 patients and 10 + 2 = 12 responders, against 36 patients and 11 confirmed responders in Table 3. It lists three EGFR-WT PRs among c-Met-high patients, while Table 3 has two EGFR-WT PRs in total, and one rare/unknown-EGFR patient with progressive disease, while Table 3 and the text say all three such patients had SD. Figure 2 shows the same pattern (35 patients plotted; 12 best-response CR or PR; three EGFR-WT PRs; one rare/unknown PD), so the paragraph appears to describe best responses in Figure 2 rather than confirmed responses. These differences involve EGFR-WT and rare/unknown patients and leave the EGFR-M+ c-Met-high responder count at 7. The size of that group is 15 in the text and 14 among the plotted patients (7/14 = 50.0%). Neither gives 52.6%.
Explanations considered
- The abstract meant all c-Met-high patients (n = 19). This accounts for the 52.6% exactly: (1 + 9)/19 = 52.63%. It does not fit the wording or the n. The sentence begins "Of EGFR-M+ patients" and gives n = 15, which is the EGFR-M+ count of the c-Met-high group in the Results (1 + 6 + 5 + 3). It cannot be n = 19 EGFR-M+ patients, because 4 of the 19 c-Met-high patients are EGFR-WT or rare/unknown.
- A different response definition (confirmed versus best response). Not able to explain it. Under either definition the EGFR-M+ responders number 9 in the whole cohort and 7 among the c-Met-high patients (Table 3; Figure 2; Results paragraph). And whatever the count, no integer out of 15 gives 52.6%.
- A different denominator, such as patients with a measurable target lesion. Not able to explain it. A rate of 52.6% to one decimal place needs a denominator of 19 or 38 (for n ≤ 40). The EGFR-M+ c-Met-high group has 15 patients in the text and 14 in Figure 2; neither number, nor any subset of it, can give 52.6%.
- Rounding. Ruled out. The nearest attainable values are 46.7% and 53.3%. Even 8/15 would be printed as 53.3%, and it would need a responder the Results paragraph and Figure 2 do not show.
- A text-conversion artifact. Ruled out. The same sentence, with n = 15 and 52.6%, appears in the PMC-archived text, the Europe PMC full-text XML and the PubMed abstract (PMID 36288547). The Results numbers come from running text, not from a converted table.
- A correction already issued. None found (see Status).
- Side observations, not part of the main finding. (1) The Discussion gives the ORR of the MET-amplified subset (n = 6) as 62.5%, which no count out of 6 can produce (3/6 = 50.0%, 4/6 = 66.7%; 62.5% is 5/8 or 10/16). Figure 2 marks six MET-amplified patients, of whom three had a PR, which is 3/6 = 50.0%. (2) The Results say that of the five MET-amplified c-Met-high patients "two had SD (50%)"; 2/5 = 40%. (3) The c-Met-stratified paragraph and Table 3 differ as described under "Why it matters". None of these affects the paper's conclusions.
Status
- Version of record. The publisher's page (ascopubs.org, reached through doi.org) returned HTTP 403 to automated requests. I checked three other copies of the published text: the PMC-archived article (as published in J Clin Oncol 2023;41(5):1105-1115, first published online in 2022), the Europe PMC full-text XML and the PubMed record (PMID 36288547). All three have the abstract sentence quoted above, with "(n = 15)" and "52.6%", and the same Results paragraph. Every quotation was checked verbatim against the PMC-archived text with
src/show.py. Tables 3 and 4 and Figure 2 are images in the archived article. I read them from the publisher's graphics in the Europe PMC supplementary package (jco-41-1105-g003, g005, g006). - Supplements. The Europe PMC supplementary package contains only the article's figure and table images. The online-only JCO Data Supplement (methods, pharmacokinetics, adverse events and time to response, per the text) was not reachable (HTTP 403) and is not hosted by PMC or Europe PMC. The question is settled by the article itself: the Results breakdown, Table 3 and Figure 2 agree that 7 EGFR-M+ c-Met-high patients responded.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has empty update-to and updated-by fields. The PubMed record's only CommentsCorrections entry is a "CommentIn" link to an editorial commentary. The PubPeer API returned no record (HTTP 404) for the DOI, and web searches for the title with "PubPeer", "erratum" or "correction" found no thread or notice. - Sources. PMC article PMC9928626; Europe PMC full-text XML and supplementary files for PMC9928626; PubMed (PMID 36288547); Crossref; PubPeer; OncLive, "Telisotuzumab Vedotin Plus Erlotinib Shows Encouraging Activity in Advanced c-Met+ NSCLC"; https://doi.org/10.1200/JCO.22.00739.
Suggested correction
In the abstract, replace
Of EGFR-M+ patients, those who were c-Met high (n = 15) had an ORR of 52.6%.
with
Of EGFR-M+ patients, those who were c-Met high (n = 15) had an ORR of 46.7% (7/15).
if the Results breakdown counts confirmed responses, or with the confirmed count out of 15 if it does not. Alternatively, keep 52.6% and describe it correctly: "Patients who were c-Met high (n = 19), regardless of EGFR status, had an ORR of 52.6%." The authors may also wish to say whether the c-Met-stratified paragraph of the Results reports best or confirmed responses, and to reconcile it with Table 3 (35 v 36 patients; 12 v 11 responders; EGFR-WT and rare/unknown counts). In the Discussion, they may wish to check the MET-amplified ORR of 62.5% against the six MET-amplified patients in Figure 2 (3/6 = 50.0%), and in the Results the "two had SD (50%)" of five patients (2/5 = 40%).
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the numbers in the write-up for PMC9928626 and regenerate its figures.
Paper: "Phase Ib Study of Telisotuzumab Vedotin in Combination With Erlotinib in
Patients With c-Met Protein-Expressing Non-Small-Cell Lung Cancer",
J Clin Oncol 2022 (online; print 2023;41(5):1105-1115), doi:10.1200/JCO.22.00739.
Every input below is copied from the paper: the abstract, the Results text,
Table 3 (efficacy by EGFR status), Table 4 (T790M / osimertinib subgroups), and the
per-patient annotations of Figure 2 (EGFR colour, c-Met >= 225 hatching, MET-amplification
asterisk, best response), transcribed by eye. Figure 2 itself is not redrawn.
Run: python recompute.py.txt
Figures are written next to this file as fig1.svg/png and fig2.svg/png.
"""
import os
from collections import Counter
from scipy.stats import beta
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyArrowPatch, FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
def pct(k, n):
return 100.0 * k / n
def cp(k, n, a=0.05):
"""Two-sided exact (Clopper-Pearson) CI in percent, the method the paper states."""
lo = 0.0 if k == 0 else beta.ppf(a / 2, k, n - k + 1)
hi = 1.0 if k == n else beta.ppf(1 - a / 2, k + 1, n - k)
return 100 * lo, 100 * hi
def r1(x):
return round(x + 1e-9, 1)
# ------------------------------------------------------------------ inputs (as printed)
ABSTRACT_N, ABSTRACT_ORR = 15, 52.6 # "c-Met high (n = 15) had an ORR of 52.6%"
# Results > Efficacy, paragraph on Figure 2: c-Met high (n = 19), by EGFR group
HI = {"CR": {"M": 1, "WT": 0, "R": 0},
"PR": {"M": 6, "WT": 3, "R": 0},
"SD": {"M": 5, "WT": 0, "R": 0},
"PD": {"M": 3, "WT": 0, "R": 1}}
HI_STATED = {"n": 19, "ORR": 52.6, "DCR": 78.9}
# same paragraph: lower c-Met scores (n = 16)
LO = {"CR": 0, "PR": 2, "SD": 12, "PD": 2}
LO_STATED = {"n": 16, "ORR": 12.5, "DCR": 87.5}
# Table 3 (best overall response; ORR/DCR use confirmed responses)
T3 = {"EGFR-M+": {"CR": 1, "PR": 8, "SD": 15, "PD": 4},
"EGFR-WT": {"CR": 0, "PR": 2, "SD": 2, "PD": 1},
"EGFR-rare/unknown": {"CR": 0, "PR": 0, "SD": 3, "PD": 0}}
T3_STATED = {"EGFR-M+": ("32.1 [15.9 to 52.4]", "85.7 [67.3 to 96.0]"),
"EGFR-WT": ("40.0 [5.3 to 85.3]", "80.0 [28.4 to 99.5]"),
"EGFR-rare/unknown": ("0 [0.0 to 70.8]", "100 [29.2 to 100]"),
"Total": ("30.6 [16.3 to 48.1]", "86.1 [70.5 to 95.3]")}
# Table 4 / Results text: EGFR-M+ subgroups, (responders, disease control, n), stated ORR, DCR
T4 = {"T790M+": (4, 10, 13, "31", "77"),
"non-T790M+": (5, 14, 15, "33", "93"),
"prior osimertinib": (4, 11, 15, "27", "73 (Table 4); 73.3 (text)"),
"no prior osimertinib": (5, 13, 13, "39", "100")}
# Figure 2 per-patient annotations (patients 1-35, left to right), transcribed:
# EGFR group (M = EGFR-M+, WT, R = rare/unknown), c-Met >= 225 hatching or "HI" label,
# MET-amplified asterisk, best response from the table under the plot.
# Patients 8 and 9 have no bar (0.0%, "HI"); the text places all c-Met-high SD in EGFR-M+,
# and the MET-amplification breakdown (4 M+, 1 WT, 1 rare) requires patient 9 to be EGFR-M+.
FIG2 = [
(1, "M", 1, 0, "PD"), (2, "M", 0, 0, "SD"), (3, "R", 0, 1, "SD"), (4, "M", 0, 0, "SD"),
(5, "M", 0, 0, "SD"), (6, "M", 0, 0, "SD"), (7, "M", 0, 0, "PD"), (8, "M", 1, 0, "SD"),
(9, "M", 1, 1, "SD"), (10, "M", 0, 0, "SD"), (11, "M", 0, 0, "SD"), (12, "M", 1, 0, "SD"),
(13, "M", 0, 0, "SD"), (14, "WT", 0, 0, "SD"), (15, "M", 1, 1, "SD"), (16, "M", 0, 0, "SD"),
(17, "R", 0, 0, "SD"), (18, "WT", 0, 0, "PD"), (19, "M", 1, 0, "SD"), (20, "M", 0, 0, "SD"),
(21, "M", 0, 0, "PD"), (22, "M", 1, 0, "PR"), (23, "M", 1, 0, "PD"), (24, "R", 1, 0, "PD"),
(25, "WT", 1, 1, "PR"), (26, "M", 1, 1, "PR"), (27, "M", 1, 1, "PR"), (28, "M", 1, 0, "PR"),
(29, "M", 1, 0, "PR"), (30, "M", 0, 0, "PR"), (31, "M", 0, 0, "PR"), (32, "WT", 1, 0, "PR"),
(33, "M", 1, 0, "PR"), (34, "M", 1, 0, "CR"), (35, "WT", 1, 0, "PR"),
]
# ------------------------------------------------------------------ 1. the abstract figure
print("1. Abstract: EGFR-M+ and c-Met high, n = 15, ORR 52.6%")
att = {k: r1(pct(k, 15)) for k in range(16)}
print(" attainable k/15 values:", ", ".join(f"{k}/15={v}" for k, v in att.items()))
print(" any k/15 that rounds to 52.6%?", any(v == ABSTRACT_ORR for v in att.values()))
print(f" 52.6% x 15 = {0.526 * 15:.2f} responders (not a whole number)")
pairs = [(k, n) for n in range(1, 41) for k in range(n + 1) if r1(pct(k, n)) == ABSTRACT_ORR]
print(" k/n pairs with n <= 40 that give 52.6%:", pairs)
# ------------------------------------------------------------------ 2. body breakdown
print("\n2. Results, c-Met-high paragraph (n = 19)")
n_hi = sum(sum(v.values()) for v in HI.values())
by_grp = {g: sum(HI[r][g] for r in HI) for g in ("M", "WT", "R")}
resp_hi = HI["CR"]["M"] + HI["CR"]["WT"] + HI["CR"]["R"] + sum(HI["PR"].values())
dc_hi = resp_hi + sum(HI["SD"].values())
print(f" total n = {n_hi}; by EGFR group {by_grp}")
print(f" ORR all c-Met high = {resp_hi}/{n_hi} = {pct(resp_hi, n_hi):.2f}% (stated {HI_STATED['ORR']})")
print(f" DCR all c-Met high = {dc_hi}/{n_hi} = {pct(dc_hi, n_hi):.2f}% (stated {HI_STATED['DCR']})")
for r in HI:
k = sum(HI[r].values())
print(f" {r}: {k}/19 = {pct(k, 19):.1f}%")
k_m = HI["CR"]["M"] + HI["PR"]["M"]
n_m = by_grp["M"]
dc_m = k_m + HI["SD"]["M"]
lo_m, hi_m = cp(k_m, n_m)
print(f" EGFR-M+ c-Met high: n = {n_m} (1 CR + 6 PR + 5 SD + 3 PD), "
f"ORR = {k_m}/{n_m} = {pct(k_m, n_m):.2f}% (exact 95% CI {lo_m:.1f} to {hi_m:.1f})")
print(f" EGFR-M+ c-Met high DCR = {dc_m}/{n_m} = {pct(dc_m, n_m):.1f}%")
print(f" abstract minus recomputed: {ABSTRACT_ORR - r1(pct(k_m, n_m)):.1f} percentage points")
lo19, hi19 = cp(resp_hi, n_hi)
print(f" all c-Met high 10/19 exact 95% CI {lo19:.1f} to {hi19:.1f}")
print(f" non-M+ c-Met-high responders: {resp_hi - k_m} (all EGFR-WT PR)")
print("\n lower c-Met scores (n = 16):")
n_lo = sum(LO.values())
k_lo = LO["CR"] + LO["PR"]
lo16, hi16 = cp(k_lo, n_lo)
print(f" n = {n_lo}; ORR {k_lo}/{n_lo} = {pct(k_lo, n_lo):.1f}% (CI {lo16:.1f} to {hi16:.1f}); "
f"DCR {n_lo - LO['PD']}/{n_lo} = {pct(n_lo - LO['PD'], n_lo):.1f}%")
print(f" SD 12/16 = {pct(12, 16):.1f}%, PD 2/16 = {pct(2, 16):.1f}%")
print(f" high + lower = {n_hi + n_lo} patients (efficacy set N = 36); "
f"responders {resp_hi + k_lo} (Table 3 total: 11)")
# ------------------------------------------------------------------ 3. Table 3
print("\n3. Table 3 (confirmed ORR / DCR), recomputed with Clopper-Pearson")
tot = Counter()
for g, d in T3.items():
n = sum(d.values())
k = d["CR"] + d["PR"]
dc = n - d["PD"]
tot.update(d)
a, b = cp(k, n)
c, e = cp(dc, n)
print(f" {g:18s} n={n:2d} ORR {k}/{n} = {pct(k, n):.1f} [{a:.1f} to {b:.1f}] "
f"DCR {dc}/{n} = {pct(dc, n):.1f} [{c:.1f} to {e:.1f}] stated {T3_STATED[g]}")
N = sum(tot.values())
K = tot["CR"] + tot["PR"]
DC = N - tot["PD"]
a, b = cp(K, N)
c, e = cp(DC, N)
print(f" {'Total':18s} n={N} ORR {K}/{N} = {pct(K, N):.1f} [{a:.1f} to {b:.1f}] "
f"DCR {DC}/{N} = {pct(DC, N):.1f} [{c:.1f} to {e:.1f}] stated {T3_STATED['Total']}")
print(f" upper bound on EGFR-M+ c-Met-high ORR if all 9 EGFR-M+ responders were c-Met high: "
f"9/15 = {pct(9, 15):.1f}%")
print(f" EGFR-WT PRs: c-Met-high paragraph 3 vs Table 3 total 2; "
f"rare/unknown PD: paragraph 1 vs Table 3 0")
# ------------------------------------------------------------------ 4. Figure 2 tallies
print("\n4. Figure 2 annotations (35 plotted patients)")
print(" plotted:", len(FIG2), " by EGFR group:", dict(Counter(p[1] for p in FIG2)))
print(" best responses:", dict(Counter(p[4] for p in FIG2)))
hi_pts = [p for p in FIG2 if p[2]]
lo_pts = [p for p in FIG2 if not p[2]]
print(f" hatched/HI (c-Met >= 225): {len(hi_pts)}; not hatched: {len(lo_pts)}")
for g in ("M", "WT", "R"):
print(f" c-Met high, {g}: {dict(Counter(p[4] for p in hi_pts if p[1] == g))}")
print(f" not c-Met high: {dict(Counter(p[4] for p in lo_pts))}")
m_hi = [p for p in hi_pts if p[1] == "M"]
m_hi_resp = sum(p[4] in ("CR", "PR") for p in m_hi)
print(f" EGFR-M+ c-Met-high responders in Fig 2: {m_hi_resp} of {len(m_hi)} plotted "
f"({pct(m_hi_resp, len(m_hi)):.1f}%); with the body's 15: {m_hi_resp}/15 = "
f"{pct(m_hi_resp, 15):.1f}%")
m_all_resp = sum(p[4] in ("CR", "PR") for p in FIG2 if p[1] == "M")
print(f" EGFR-M+ responders across Fig 2: {m_all_resp} (Table 3 confirmed: 9)")
print(f" all responders in Fig 2: {sum(p[4] in ('CR', 'PR') for p in FIG2)} (Table 3: 11)")
wt = {p[0]: p[4] for p in FIG2 if p[1] == "WT"}
rr = {p[0]: p[4] for p in FIG2 if p[1] == "R"}
print(f" EGFR-WT best responses {wt} (Table 3: 2 PR, 2 SD, 1 PD)")
print(f" rare/unknown best responses {rr} (Table 3: 3 SD)")
# ------------------------------------------------------------------ 5. other quantities
print("\n5. Other quantities in the text")
amp = [p for p in FIG2 if p[3]]
amp_resp = sum(p[4] in ("CR", "PR") for p in amp)
print(f" MET-amplified (asterisks): {[(p[0], p[1], p[4]) for p in amp]}")
print(f" MET-amplified ORR from Fig 2: {amp_resp}/{len(amp)} = {pct(amp_resp, len(amp)):.1f}% "
f"(Discussion: 62.5%)")
print(" attainable k/6:", [r1(pct(k, 6)) for k in range(7)],
"; 62.5% = 5/8 or 10/16")
print(f" MET-amplified c-Met high: PR 3/5 = {pct(3, 5):.0f}%, SD 2/5 = {pct(2, 5):.0f}% "
f"(text: 60% and 50%)")
print(f" c-Met high share 19/36 = {pct(19, 36):.1f}% (stated 53%); "
f"MET-amplified 6/36 = {pct(6, 36):.1f}% (stated 17%)")
for g, (k, dc, n, so, sd) in T4.items():
print(f" {g:22s} ORR {k}/{n} = {pct(k, n):.1f}% (stated {so}); "
f"DCR {dc}/{n} = {pct(dc, n):.1f}% (stated {sd})")
# ------------------------------------------------------------------ figures
BG, INK, MUTED = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, EDGE = "#eb6834", "#2a78d6", "#e4e3df", "#c9c8c3"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK, "ytick.color": INK,
"axes.edgecolor": MUTED, "figure.facecolor": BG, "axes.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- Figure 1: abstract sentence vs Results breakdown, with the two calculations
fig = plt.figure(figsize=(7.2, 6.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
fig.canvas.draw()
renderer = fig.canvas.get_renderer()
def runs(x, y, parts, size=10.5):
"""Draw text segments of different colour/weight on one line, left to right."""
inv = ax.transData.inverted()
for txt, color, weight in parts:
t = ax.text(x, y, txt, fontsize=size, color=color, weight=weight, va="center")
bb = t.get_window_extent(renderer=renderer)
x = inv.transform((bb.x1, bb.y0))[0]
return x
def panel(x, y, w, h, title, sub=None):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc="white", ec=EDGE, lw=1.2))
ax.text(x + 2, y + h - 3.0, title, fontsize=11, weight="bold", va="top")
if sub:
ax.text(x + 2, y + h - 7.4, sub, fontsize=10, color=MUTED, va="top")
# Abstract panel (top left)
panel(2, 54, 45, 43, "Abstract, Results")
yq = 86
for parts in ([("“ORR for EGFR-M+ patients", INK, "normal")],
[("(n = 28) was 32.1%. Of EGFR-M+", INK, "normal")],
[("patients, those who were c-Met", INK, "normal")],
[("high ", INK, "normal"), ("(n = 15)", ORANGE, "bold"), (" had an ORR", INK, "normal")],
[("of ", INK, "normal"), ("52.6%", ORANGE, "bold"), (".”", INK, "normal")]):
runs(4, yq, parts)
yq -= 4.8
ax.text(4, 58, "Reported: EGFR-M+ and c-Met high", fontsize=10, color=ORANGE, va="center")
# Results panel (top right)
panel(51, 54, 47, 43, "Results, Efficacy (Figure 2 text)", "c-Met high, n = 19, by EGFR group")
cols = [("", 53), ("M+", 70), ("WT", 79), ("rare", 87.5), ("all", 95)]
ty = 83
for lab, cx in cols[1:]:
ax.text(cx, ty, lab, fontsize=10, color=MUTED, ha="center", va="center", weight="bold")
rows = [("CR", 1, 0, 0), ("PR", 6, 3, 0), ("SD", 5, 0, 0), ("PD", 3, 0, 1)]
for i, (lab, m, w, r) in enumerate(rows):
yy = ty - 4.2 * (i + 1)
ax.text(53, yy, lab, fontsize=10.5, va="center")
for v, (_, cx) in zip((m, w, r, m + w + r), cols[1:]):
ax.text(cx, yy, str(v), fontsize=10.5, ha="center", va="center")
yy = ty - 4.2 * 5
ax.plot([53, 97], [yy + 2.2, yy + 2.2], color=EDGE, lw=1)
ax.text(53, yy, "n", fontsize=10.5, va="center", weight="bold")
for v, (_, cx) in zip((15, 3, 1, 19), cols[1:]):
ax.text(cx, yy, str(v), fontsize=10.5, ha="center", va="center", weight="bold")
ax.text(53, 57, "“…indicating an ORR of 52.6%” (all, n = 19)", fontsize=10,
color=MUTED, va="center")
# Bottom boxes: the two calculations
def calc_box(x, title, formula, note, color):
ax.add_patch(FancyBboxPatch((x, 5), 45, 27, boxstyle="round,pad=0,rounding_size=1.2",
fc="white", ec=color, lw=2))
ax.text(x + 2.5, 29.5, title, fontsize=10.5, weight="bold", va="top")
ax.text(x + 22.5, 19.5, formula, fontsize=13, ha="center", va="center", color=color,
weight="bold")
ax.text(x + 2.5, 11.0, note, fontsize=10, color=MUTED, va="center", linespacing=1.35)
calc_box(2, "All c-Met high (every EGFR group)", "(1 + 9) / 19 = 52.6%",
"includes 3 EGFR-WT partial\nresponses", BLUE)
calc_box(51, "EGFR-M+ and c-Met high", "(1 + 6) / 15 = 46.7%",
"no whole count out of 15 gives 52.6%:\n7/15 = 46.7%, 8/15 = 53.3%", BLUE)
# connectors
ax.add_patch(FancyArrowPatch((24.5, 53.5), (24.5, 32.5), arrowstyle="-|>", mutation_scale=14,
color=ORANGE, lw=2, linestyle=(0, (4, 2))))
ax.text(26, 43, "abstract's 52.6%\nis this number", fontsize=10, color=ORANGE, va="center")
ax.add_patch(FancyArrowPatch((63, 53.5), (35, 32.5), arrowstyle="-|>", mutation_scale=14,
color=BLUE, lw=2, connectionstyle="arc3,rad=0.0"))
ax.add_patch(FancyArrowPatch((74, 53.5), (74, 32.5), arrowstyle="-|>", mutation_scale=14,
color=BLUE, lw=2))
ax.text(48.5, 36.5, "all columns", fontsize=10, color=BLUE, va="center")
ax.text(75.5, 43, "M+ column only\n(n = 15, as in abstract)", fontsize=10, color=BLUE,
va="center")
ax.text(50, 2.2, "Orange: value and label printed in the abstract. "
"Blue: recomputed from the Results counts.", fontsize=10, color=MUTED, ha="center",
va="center")
save(fig, "fig1")
# ---- Figure 2: ORR estimates on one axis
rows2 = [
("EGFR-M+ and c-Met high\n(n = 15)", k_m, n_m, ABSTRACT_ORR),
("All c-Met high\n(n = 19)", resp_hi, n_hi, None),
("All EGFR-M+\n(n = 28, Table 3)", 9, 28, None),
("Lower c-Met scores\n(n = 16)", k_lo, n_lo, None),
]
fig, ax = plt.subplots(figsize=(7.2, 4.1))
fig.subplots_adjust(left=0.27, right=0.97, top=0.84, bottom=0.15)
ys = [3, 2, 1, 0]
for y in ys:
ax.axhline(y, color=GRID, lw=0.8, zorder=0)
for y, (lab, k, n, rep) in zip(ys, rows2):
est = pct(k, n)
lo, hi = cp(k, n)
ax.plot([lo, hi], [y - 0.12, y - 0.12], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(est, y - 0.12, "o", color=BLUE, ms=8, zorder=3)
ax.text(est, y - 0.32, f"{k}/{n} = {est:.1f}%", fontsize=10, color=BLUE, ha="center",
va="top")
if rep is not None:
ax.plot(rep, y + 0.16, "D", color=ORANGE, ms=8, zorder=3)
ax.text(rep + 1.6, y + 0.16, "abstract: 52.6%", fontsize=10, color=ORANGE, va="center")
for kk in range(16):
ax.plot([pct(kk, 15)] * 2, [y + 0.36, y + 0.46], color=MUTED, lw=1, zorder=1)
ax.text(0, y + 0.55, "possible values k/15", fontsize=10, color=MUTED, va="bottom")
ax.axvline(ABSTRACT_ORR, color=ORANGE, lw=0.8, zorder=0, alpha=0.7)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows2], fontsize=10.5)
ax.set_xlim(0, 100)
ax.set_ylim(-0.7, 3.95)
ax.set_xlabel("Objective response rate, % (bars: exact 95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="D", color=ORANGE, ls="", ms=8, label="Reported in the abstract"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=8,
label="Recomputed from the paper's counts")]
fig.legend(handles=handles, loc="upper center", ncol=2, frameon=False, fontsize=10,
bbox_to_anchor=(0.6, 0.99))
save(fig, "fig2")
print("\nfigures written to", HERE)
Figure 6D calls a TRPV4 knockout effect significant at p = 0.06706, above the 0.05 threshold
J Neurosci (2018) · doi:10.1523/JNEUROSCI.0897-18.2018 · PMC6181316
Summary
The Figure 6D legend says that conventional TRPV4 knockout (TRPV4KO) mice "showed significantly less photoreceptor cell death than WT mice" 24 h after retinal detachment. It marks the comparison with an asterisk and prints p = 0.06706. The Methods state one decision rule for these TUNEL comparisons: "p < 0.05 was considered significant". Since 0.06706 > 0.05, the comparison is not significant under the paper's own rule. Nothing in the paper (a one-sided test, a different threshold, a correction notice) reconciles the two. Of the 16 exact p-values printed in the paper, this is the only one above 0.05. The same comparison is the first result stated in the Abstract (a ~50% reduction in apoptotic photoreceptors). The paper's main conclusion still has other support, including the Müller-glia conditional knockout in the same legend (p = 0.020863, which is significant and consistent).
What the paper reports
Methods, "Statistical analysis":
The Mann–Whitney U test was used to compare unpaired values of TUNEL-positive cell density and proinflammatory cytokine concentration. The data analyses were performed using Excel 2016 (Microsoft) with add-in software Statce14@. p < 0.05 was considered significant.
Figure 6 legend, panel D:
Conventional TRPV4KO mice showed significantly less photoreceptor cell death than WT mice 24 h after retinal detachments (*p = 0.06706, n = 7 or 8 eyes). The TRPV4CKO mice showed significantly less photoreceptor cell death than WT mice (**p = 0.020863, n = 6 eyes). The TRPV4CKO results were perfectly matched with those in TRPV4KO, indicating that Müller glial TRPV4 activation accelerates photoreceptor cell death in RD.
Abstract:
After RD, the number of apoptotic photoreceptors was reduced by ∼50% in TRPV4KO mice relative to wild-type mice, indicating the possible involvement of TRPV4 activation in RD-induced photoreceptor cell death.
Results, TUNEL paragraph. This passage gives the size of the effect but does not call the KO result significant:
The number of dead photoreceptors in TRPV4KO mice was 50% less than that for WT mice (Fig. 6D).
Consistent with those previous reports and our data (Figs. 1–5), the number of dead photoreceptors in TRPV4CKO mice was 50% less than that for WT mice, and the results were perfectly matched with the result in the conventional TRPV4KO (Fig. 6D).
Discussion, second paragraph:
Indeed, our Müller glia-specific conditional KO mice can significantly reduce the RD-induced photoreceptor cell death (Fig. 6), and the reduction ratio of dead photoreceptors was perfectly the same as the conventional TRPV4KO mice (Fig. 6).
What the paper's own numbers show
Method. The paper gives no per-eye TUNEL counts, group summaries or U statistics, so the test cannot be rerun. Two checks are possible. First, compare each printed p with the threshold stated in the Methods. Second, ask whether 0.06706 is a plausible two-sided Mann–Whitney p for the stated sample sizes, and so whether it could be a misprint of a smaller value. For the second check I enumerated the exact null distribution of U for n = 7 vs 8 (6435 rank arrangements, assuming no ties), and for 7 vs 7 and 8 vs 8 because the legend says "7 or 8 eyes". I also computed the normal approximation with and without continuity correction. For context I collected every exact p-value printed in the paper. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Significance threshold | Methods, Statistical analysis | "p < 0.05 was considered significant" | α = 0.05 |
| WT vs conventional TRPV4KO, TUNEL+ photoreceptors at 24 h | Fig. 6D legend | "significantly less", *p = 0.06706, n = 7 or 8 eyes | 0.06706 > 0.05 by 0.01706 (1.34 × the threshold); not significant under the stated rule. Equivalent |z| = 1.831, against 1.960 needed |
| WT vs TRPV4CKO | Fig. 6D legend | "significantly less", **p = 0.020863, n = 6 eyes | 0.020863 < 0.05; significant, consistent |
| Other exact p-values in the paper | legends of Figs. 4, 5, 7, 8, 9, 10 | 14 values, from 0.000116 to 0.012851 | all < 0.05. Of the 16 printed exact p-values, 0.06706 is the only one ≥ 0.05; the next largest is 0.020863 |
| Hypothetical one-sided reading | not stated in the paper | — | 0.06706 / 2 = 0.03353 (would be < 0.05). If 0.06706 were already one-sided, the two-sided p would be 0.13412 |
| Exact two-sided Mann–Whitney p near the printed value, n = 7 vs 8, no ties | recomputation | 0.06706 | U = 11: 0.05408; U = 12: 0.07211. The printed value lies between them, and both are above 0.05 |
| Normal approximations closest to 0.06706, n = 7 vs 8 | recomputation | 0.06706 | U = 12: 0.06408 (no continuity correction), 0.07285 (with); U = 11 with correction: 0.05620 |
| Same check for n = 7 vs 7 and 8 vs 8 | recomputation | 0.06706 | 7 vs 7: exact 0.05303 / 0.07284 (U = 9 / 10); 8 vs 8: exact 0.06496 / 0.08298 (U = 14 / 15) |
| Exact p near a misprint value of 0.006706, n = 7 vs 8 | recomputation | — | U = 5: 0.00591; U = 6: 0.00932. A value near 0.0067 is also within the attainable range, so a misprint cannot be excluded |
The printed 0.06706 is not itself an exact no-ties value for any of these sample sizes. Ties in the TUNEL densities, or a normal approximation as statistics add-ins commonly use, would give values between the exact steps. As printed, 0.06706 falls where an ordinary two-sided Mann–Whitney result for about 7–8 eyes per group would fall when the rank separation is just short of significance (U = 11–12 of a possible 56 for 7 vs 8).
Figure
On a linear p-value axis starting at 0, the two orange dots are the p-values printed in the Figure 6D legend. Both comparisons are described there as "significantly less". The vertical line is the threshold stated in the Methods (p < 0.05), and the shaded band is the region the paper counts as significant. WT vs TRPV4CKO (p = 0.020863) lies inside it. WT vs conventional TRPV4KO (p = 0.06706) lies outside it. The blue open squares are recomputed: the two exact two-sided Mann–Whitney p-values on either side of the printed value for n = 7 vs 8 eyes with no ties (U = 11: 0.0541; U = 12: 0.0721). Both are above 0.05. Taken as printed, the knockout result is a two-sided value just short of significance.
All 16 exact p-values printed in the paper's figure legends, sorted and shown on a log scale. Every open dot is a comparison the legends call significant, and each lies below 0.05. The filled dot (Fig. 6D, WT vs TRPV4KO, p = 0.06706) is the only printed value above the threshold, and the legend calls it "significantly less" all the same. Values are copied from the legends; panel letters follow the legends' own labels.
Why it matters
The WT vs conventional TRPV4KO comparison is the first result in the Abstract ("reduced by ∼50% in TRPV4KO mice relative to wild-type mice"). It is also one half of an argument made twice in the paper: the CKO effect is "perfectly matched" with, or "perfectly the same as", the conventional KO effect, and this match is used to assign the effect to Müller glial TRPV4. Under the paper's criterion, the KO side of that comparison is a non-significant trend (p = 0.067, about 7–8 eyes per group), not an established reduction. The direction of the effect does not change. The 50% figure cannot be checked because no group values are given.
The paper's overall conclusion, that Müller glial TRPV4 contributes to photoreceptor death after retinal detachment, is weakened but not overturned. In the same legend, the Müller-glia/astrocyte conditional knockout reaches significance (p = 0.020863, n = 6). The agonist GSK increases photoreceptor death (Fig. 7A, p = 0.002622). The antagonist HC reduces it (Fig. 7B, p = 0.009823). The MCP-1 and macrophage experiments (Figs. 8–10) point the same way. Readers who rely on the global-knockout result as an independent replication, or quote the Abstract's ~50% reduction as statistically established, are relying on a comparison that the paper's own numbers do not support at α = 0.05.
Explanations considered
- A different or looser threshold for this figure. The Methods give a single rule, "p < 0.05 was considered significant", and name the Mann–Whitney test for TUNEL densities. The Figure 6 legend sets no other threshold. This does not explain the discrepancy.
- A one-sided test. Halving 0.06706 gives 0.0335, which would pass. The paper never says that any test was one-sided; searches for "one-tailed", "two-tailed" and "one-sided" return nothing. If 0.06706 were already one-sided, the two-sided value would be 0.134, which is further from significance. This does not explain it unless an unstated convention was used.
- The asterisk convention. The paper uses asterisks inconsistently. In Fig. 6D a single * marks p = 0.067 and ** marks p = 0.021. In Figs. 8A and 8C asterisks mean p < 0.01, and in Fig. 8B the asterisk means p < 0.05. No convention in the paper makes 0.067 count as significant, so this does not explain it. The published figure image, as described in the version-of-record check, marks WT vs KO with a single asterisk, which matches the legend.
- A misprint for 0.006706 (or similar). This cannot be excluded without the raw data. A value near 0.0067 is attainable for n = 7 vs 8 (exact values 0.0059 and 0.0093 at U = 5 and 6). But 0.06706 is also an ordinary two-sided value for these sample sizes, between the exact values 0.054 and 0.072 and close to the normal approximations at U = 11–12. Nothing else in the paper points to a smaller value. If it is a misprint, then the p-value is wrong rather than the significance label. Either way the published text is internally inconsistent.
- A different comparison, population or test. The legend gives the comparison (WT vs conventional TRPV4KO, 24 h after detachment, n = 7 or 8 eyes) and the p-value in the same sentence. The legend does not name the test. Some other legends in the paper use Student's t test, but the threshold the paper states is the same whichever test was used.
- Multiplicity adjustment. An adjustment could only raise a p-value, so it cannot turn 0.067 into a significant result.
- Rounding. 0.06706 is printed to five decimals. No rounding brings it below 0.05.
- Scope of the claim. The word "significantly" for the KO comparison appears only in the Figure 6D legend, and there it carries an asterisk. The Results text reports "50% less" without a significance claim. The Discussion's "significantly" refers to the CKO, which is valid. The Abstract states the ~50% reduction as a finding without a p-value. So the explicit inconsistency is confined to the Fig. 6D legend, and the Abstract states the same result as established.
- A later correction. No correction, erratum or retraction is listed (see Status).
Status
As of 2026-10-23:
- Version of record. The wording and numbers above are in the publisher's typeset PDF as hosted by PMC (14 pages; page 7 for the Figure 6 legend, page 6 for Statistical analysis). They are also in the live PMC page and the Europe PMC full-text XML. Relation symbols in the PDF were read from rendered page images, because its text layer drops them. The publisher's own site (jneurosci.org) refused automated access, so its HTML was not compared. The inconsistency is in the publisher's typeset text, not a conversion artifact.
- Supplements. The article has no supplementary data files. The Europe PMC supplementary bundle contains only the figure images. The Fig. 6D dot plot shows individual eyes but no values, so nothing in the paper can settle whether the p-value or the significance label is wrong.
- Corrections. Crossref lists no update-to or updated-by relations, and a Crossref filter for works that update this DOI returned none. PMC and Europe PMC list no correction, erratum, expression of concern or retraction (checked with
src/known.pyand Europe PMC search, PMID 30143574). - PubPeer. PubPeer's search refused automated access (Cloudflare verification), so it could not be checked directly. A web search for the title found no PubPeer entry.
- Sources. PMC6181316; doi:10.1523/JNEUROSCI.0897-18.2018; Crossref and Europe PMC REST APIs.
Suggested correction
The Figure 6D legend should be made consistent with the Methods. There are two possibilities:
- If p = 0.06706 is correct: change the legend to describe the KO vs WT difference as a non-significant trend, for example "TRPV4KO mice showed a trend toward less photoreceptor cell death than WT mice (p = 0.067, n = 7 or 8 eyes)". Remove the asterisk from the figure. Qualify the Abstract's "reduced by ∼50%" sentence and the "perfectly matched" / "perfectly the same" statements to match.
- If the printed value is a misprint: give the correct p-value.
In either case, it would help readers if the authors stated the group sizes for each genotype (which group had 7 eyes and which had 8), the test used for Fig. 6D, whether it was one- or two-sided and exact or approximate, and the group summaries (median and range, or the per-eye values) behind the "50% less" figure.
Recomputation script (python recompute.py.txt)
"""Recompute the numbers behind the Figure 6D significance claim in PMC6181316
(J Neurosci 2018, doi:10.1523/JNEUROSCI.0897-18.2018).
Run: python recompute.py.txt # prints every number, writes fig1/fig2 (.svg, .png)
python recompute.py.txt --candidates # also writes five candidate figure designs
# to candidates/ (A-E) for review
Needs numpy, scipy and matplotlib.
Inputs are copied from the paper as archived in PMC (PMC6181316):
Methods, "Statistical analysis": Mann-Whitney U test for TUNEL-positive cell density;
"p < 0.05 was considered significant."
Figure 6D legend: WT vs conventional TRPV4KO, "*p = 0.06706, n = 7 or 8 eyes";
WT vs TRPV4CKO, "**p = 0.020863, n = 6 eyes". Both called "significantly less".
Every other exact p-value printed in the figure legends (Figs. 4, 5, 7, 8, 9, 10).
The paper gives no per-eye values, so the test itself cannot be rerun. What can be
checked is (1) the printed p against the stated threshold and (2) whether the printed
p is a plausible Mann-Whitney value for the stated sample sizes. For (2) the exact
null distribution of U is enumerated (no ties assumed) and the normal approximation
is computed with and without continuity correction.
"""
import os
import sys
from itertools import combinations
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ----------------------------------------------------------------- inputs (from the paper)
ALPHA = 0.05 # Methods, Statistical analysis
P_KO = 0.06706 # Fig. 6D legend, WT vs conventional TRPV4KO, "n = 7 or 8 eyes"
P_CKO = 0.020863 # Fig. 6D legend, WT vs TRPV4CKO, "n = 6 eyes"
# every exact p-value printed in the paper (location, label, p, significance wording)
ALL_P = [
("Fig. 4C", "pressure-evoked currents", 0.00056),
("Fig. 5C", "TRPV4-evoked currents", 0.0058),
("Fig. 6D", "WT vs TRPV4KO, TUNEL", P_KO),
("Fig. 6D", "WT vs TRPV4CKO, TUNEL", P_CKO),
("Fig. 7A", "GSK vs control, TUNEL", 0.002622),
("Fig. 7B", "HC vs control, TUNEL", 0.009823),
("Fig. 8A", "MCP-1 mRNA, RD vs control", 0.000116),
("Fig. 8A", "MCP-1 mRNA, vs WT RD", 0.000232),
("Fig. 8B", "MCP-1 protein", 0.0010509),
("Fig. 8C", "GSK vs control", 0.000247),
("Fig. 8C", "Hypo vs control", 0.00944),
("Fig. 8C", "GSK+HC vs GSK", 0.001683),
("Fig. 8C", "Hypo+HC vs Hypo", 0.00092),
("Fig. 9A", "anti-MCP-1 vs PBS, TUNEL", 0.012611),
("Fig. 9A", "anti-MCP-1 vs IgG, TUNEL", 0.012851),
("Fig. 10", "CD11b+ cells", 0.00632),
]
# ----------------------------------------------------------------- helpers
def exact_U(n1, n2):
"""All C(n1+n2, n1) values of U for group 1 under H0, no ties."""
N = n1 + n2
return np.array([sum(i + 1 for i in c) - n1 * (n1 + 1) / 2
for c in combinations(range(N), n1)])
def exact_p2(Us, u):
return min(1.0, 2 * np.mean(Us <= u))
def normal_p2(u, n1, n2, cc=True):
mu = n1 * n2 / 2
sd = np.sqrt(n1 * n2 * (n1 + n2 + 1) / 12)
z = (abs(u - mu) - (0.5 if cc else 0.0)) / sd
return 2 * stats.norm.sf(z), z
# ----------------------------------------------------------------- 1. threshold check
print("1. Printed p-values against the stated threshold (p < 0.05)")
print(f" WT vs TRPV4KO : p = {P_KO} -> below 0.05? {P_KO < ALPHA}; "
f"exceeds 0.05 by {P_KO - ALPHA:.5f} ({P_KO / ALPHA:.3f} x the threshold)")
print(f" WT vs TRPV4CKO: p = {P_CKO} -> below 0.05? {P_CKO < ALPHA}")
print(f" z implied by a two-sided p of {P_KO}: {stats.norm.isf(P_KO / 2):.4f} "
f"(1.9600 needed for p = 0.05)")
# ----------------------------------------------------------------- 2. other p-values in the paper
print("\n2. All exact p-values printed in the paper")
above = [r for r in ALL_P if r[2] >= ALPHA]
for loc, lab, p in sorted(ALL_P, key=lambda r: r[2]):
print(f" {loc:8s} {lab:28s} p = {p:<10g} {'>= 0.05' if p >= ALPHA else '< 0.05'}")
print(f" {len(ALL_P)} printed p-values; {len(above)} at or above 0.05: "
f"{[(r[0], r[2]) for r in above]}")
print(f" largest p other than the KO comparison: "
f"{max(r[2] for r in ALL_P if r[2] != P_KO)}")
# ----------------------------------------------------------------- 3. hypothetical one-sided readings
print("\n3. One-sided readings (the paper states none)")
print(f" if 0.06706 were two-sided and halved: {P_KO / 2:.5f} (< 0.05: {P_KO / 2 < ALPHA})")
print(f" if 0.06706 were already one-sided, two-sided p = {min(1, 2 * P_KO):.5f}")
# ----------------------------------------------------------------- 4. attainable Mann-Whitney p-values
print("\n4. Mann-Whitney p-values attainable for the stated sample sizes (no ties)")
EXACT = {}
for n1, n2 in [(7, 8), (7, 7), (8, 8)]:
Us = exact_U(n1, n2)
EXACT[(n1, n2)] = Us
print(f" n = {n1} vs {n2}: {len(Us)} rank arrangements")
for u in range(8, 16):
pe = exact_p2(Us, u)
pc, _ = normal_p2(u, n1, n2, True)
pn, _ = normal_p2(u, n1, n2, False)
flag = " <- brackets 0.06706" if (exact_p2(Us, u - 1) < P_KO <= pe or
exact_p2(Us, u) < P_KO <= exact_p2(Us, u + 1)) else ""
print(f" U = {u:2d}: exact 2-sided {pe:.5f}; normal cc {pc:.5f}; "
f"normal no-cc {pn:.5f}{flag}")
U78 = EXACT[(7, 8)]
EX11, EX12 = exact_p2(U78, 11), exact_p2(U78, 12)
NORM = sorted([normal_p2(11, 7, 8, True)[0], normal_p2(12, 7, 8, True)[0],
normal_p2(12, 7, 8, False)[0]])
print(f" n = 7 vs 8: 0.06706 lies between exact values {EX11:.5f} (U = 11) and "
f"{EX12:.5f} (U = 12)")
print(f" n = 7 vs 8: normal approximations closest to 0.06706 (U = 11 with cc, U = 12 with and "
f"without cc) span {NORM[0]:.5f} to {NORM[-1]:.5f}")
print(f" gap between those two exact values: {EX12 - EX11:.5f}; 0.06706 is not itself an "
f"exact no-ties value")
P_TYPO = 0.006706
print(f" for reference, a value of {P_TYPO} would lie between exact values "
f"{exact_p2(U78, 5):.5f} (U = 5) and {exact_p2(U78, 6):.5f} (U = 6)")
U66 = exact_U(6, 6)
print(f" n = 6 vs 6 (CKO): exact U = 3 -> {exact_p2(U66, 3):.5f}, U = 4 -> "
f"{exact_p2(U66, 4):.5f}; normal cc U = 3 -> {normal_p2(3, 6, 6, True)[0]:.5f}, "
f"no-cc U = 4 -> {normal_p2(4, 6, 6, False)[0]:.5f}")
# ----------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GRAY = "#eb6834", "#2a78d6", "#8a8984"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "axes.facecolor": BG, "figure.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
LW, MS = 1.5, 8 # 1.5 pt = 2 px at 96 dpi; markers 8 pt
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
print(" wrote", os.path.relpath(path_noext, HERE) + ".svg/.png")
def clean(ax, grid_axis="x"):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
# ----------------------------------------------------------------- design A: p-value axis
def design_A(path):
from matplotlib.lines import Line2D
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ax.axvspan(0, ALPHA, color="#eef3fa", zorder=0)
ax.axvline(ALPHA, color=INK2, lw=1.0)
ax.text(ALPHA, 3.05, "stated threshold p = 0.05 (Methods)", ha="center", va="bottom",
fontsize=10, color=INK2)
ax.text(0.025, 2.85, "significant (p < 0.05)", ha="center", va="center",
fontsize=10, color=INK2)
ax.text(0.0775, 2.85, "not significant", ha="center", va="center",
fontsize=10, color=INK2)
y = {"cko": 0.4, "mw": 1.2, "ko": 2.1}
ax.plot(P_CKO, y["cko"], "o", ms=MS, color=ORANGE, zorder=3)
ax.annotate(f"p = {P_CKO} (**), \"significantly less\"", (P_CKO, y["cko"]),
xytext=(9, 0), textcoords="offset points", va="center", fontsize=10)
ax.plot(P_KO, y["ko"], "o", ms=MS, color=ORANGE, zorder=3)
ax.annotate(f"p = {P_KO} (*)\n\"significantly less\"", (P_KO, y["ko"]),
xytext=(9, 0), textcoords="offset points", va="center", fontsize=10)
ax.hlines(y["mw"], EX11, EX12, color=BLUE, lw=LW, zorder=2)
ax.plot([EX11, EX12], [y["mw"]] * 2, "s", ms=MS, color=BLUE, zorder=3, mfc=BG, mew=LW)
for u, pe, ha in ((11, EX11, "right"), (12, EX12, "left")):
ax.text(pe + (-0.0012 if ha == "right" else 0.0012), y["mw"] + 0.24,
f"U = {u}: {pe:.4f}", ha=ha, va="bottom", fontsize=10, color=INK2)
ax.plot([P_KO, P_KO], [y["mw"] + 0.12, y["ko"] - 0.12], color=GRAY, lw=1.0, ls=(0, (2, 2)))
ax.set_yticks([y["cko"], y["mw"], y["ko"]])
ax.set_yticklabels(["WT vs TRPV4CKO\n(n = 6 eyes)",
"neighbouring exact\nMann-Whitney p,\nn = 7 vs 8",
"WT vs TRPV4KO\n(n = 7 or 8 eyes)"], fontsize=10, color=INK)
ax.set_xlim(0, 0.10)
ax.set_ylim(0, 3.35)
ax.set_xlabel("two-sided p value (linear scale from 0)")
clean(ax)
h = [Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE,
label="reported in the Fig. 6D legend"),
Line2D([], [], marker="s", ls="-", lw=LW, ms=MS, color=BLUE, mfc=BG, mew=LW,
label="recomputed: exact two-sided p (no ties)")]
fig.legend(handles=h, loc="lower center", ncol=2, frameon=False, fontsize=10,
bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.07, 1, 1))
save(fig, path)
# ----------------------------------------------------------------- design B: all p-values in the paper
def design_B(path):
rows = sorted(ALL_P, key=lambda r: r[2])
fig, ax = plt.subplots(figsize=(7.2, 5.4))
ax.axvline(ALPHA, color=INK2, lw=1.0)
ax.text(ALPHA * 1.08, 7.5, "stated threshold p = 0.05", ha="left", rotation=90,
va="center", fontsize=10, color=INK2)
for i, (loc, lab, p) in enumerate(rows):
is_ko = p == P_KO
ax.hlines(i, 1e-4, p, color=GRID, lw=0.8, zorder=1)
ax.plot(p, i, "o", ms=MS if is_ko else MS - 1, color=ORANGE,
mfc=ORANGE if is_ko else BG, mew=LW, zorder=3)
if is_ko:
ax.text(0.0009, 1.3, "filled dot: p = 0.06706, the\nonly printed p above 0.05, yet\n"
"called \"significantly less\"", ha="left", va="center", fontsize=10,
color=INK)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([f"{r[0]} {r[1]}" for r in rows], fontsize=10)
for t, r in zip(ax.get_yticklabels(), rows):
t.set_color(INK if r[2] == P_KO else INK2)
if r[2] == P_KO:
t.set_fontweight("bold")
ax.set_xscale("log")
ax.set_xlim(8e-5, 0.3)
ax.set_ylim(-0.7, len(rows) - 0.3)
ax.set_xlabel("p value as printed in the figure legends (log scale)")
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 0.1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "0.1"])
clean(ax)
from matplotlib.lines import Line2D
h = [Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, mfc=BG, mew=LW,
label="reported p, called significant"),
Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE,
label="reported p = 0.06706 (Fig. 6D, KO)")]
fig.legend(handles=h, loc="lower center", ncol=2, frameon=False, fontsize=10,
bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.05, 1, 1))
save(fig, path)
# ----------------------------------------------------------------- design C: two text panels
def panel(ax, x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.01,rounding_size=0.02",
fc="#ffffff", ec=edge, lw=LW))
ax.text(x + 0.02, y + h - 0.04, title, fontsize=10, fontweight="bold", va="top")
ax.text(x + 0.02, y + h - 0.13, body, fontsize=10, va="top", linespacing=1.4)
def design_C(path):
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
panel(ax, 0.01, 0.31, 0.36, 0.66, "Methods, \"Statistical analysis\"",
"\"The Mann-Whitney U test\nwas used to compare\nunpaired values of\nTUNEL-positive cell\n"
"density ... p < 0.05 was\nconsidered significant.\"", INK2)
panel(ax, 0.63, 0.31, 0.36, 0.66, "Figure 6D legend",
"\"Conventional TRPV4KO\nmice showed significantly\nless photoreceptor cell\n"
"death than WT mice 24 h\nafter retinal detachments\n(*p = 0.06706, n = 7 or 8\neyes).\"",
ORANGE)
ax.annotate("", xy=(0.62, 0.66), xytext=(0.38, 0.66),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=LW))
ax.text(0.50, 0.70, "conflict", ha="center", va="bottom", fontsize=10, color=INK,
fontweight="bold")
ax.text(0.50, 0.62, "0.06706 is not\n< 0.05, but is\ncalled significant", ha="center",
va="top", fontsize=10, color=INK2)
ax.add_patch(FancyBboxPatch((0.02, 0.03), 0.96, 0.22,
boxstyle="round,pad=0.01,rounding_size=0.02",
fc="#eef3fa", ec=BLUE, lw=LW))
ax.text(0.5, 0.14, f"Recomputed: {P_KO} > {ALPHA} (by {P_KO - ALPHA:.5f}), so not significant "
"under the paper's rule.\nThe CKO result in the same legend "
f"(p = {P_CKO}) is below 0.05 and is consistent.", ha="center",
va="center", fontsize=10)
ax.text(0.01, 1.0, "gray frame: stated rule; orange frame: reported claim; blue frame: recomputed check",
fontsize=10, color=INK2, va="bottom")
ax.set_ylim(0, 1.07)
save(fig, path)
# ----------------------------------------------------------------- design D: exact ladder of attainable p
def design_D(path):
fig, ax = plt.subplots(figsize=(7.2, 3.4))
us = np.arange(5, 15)
pe = [exact_p2(U78, u) for u in us]
pc = [normal_p2(u, 7, 8, True)[0] for u in us]
ax.axhline(ALPHA, color=INK2, lw=1.0)
ax.text(14.6, ALPHA + 0.002, "stated threshold p = 0.05", ha="right", va="bottom",
fontsize=10, color=INK2)
ax.plot(us, pe, "s-", color=BLUE, lw=LW, ms=MS, mfc=BG, mew=LW,
label="recomputed: exact two-sided p, n = 7 vs 8")
ax.plot(us, pc, "o--", color=BLUE, lw=LW, ms=MS - 2, alpha=0.75,
label="recomputed: normal approx. with continuity correction")
ax.axhline(P_KO, color=ORANGE, lw=LW, ls=(0, (1, 2)),
label=f"reported p = {P_KO} (Fig. 6D, called significant)")
ax.text(4.6, P_KO + 0.002, f"reported {P_KO}", ha="left", va="bottom", fontsize=10)
ax.set_xticks(us)
ax.set_xlabel("Mann-Whitney U statistic (smaller U = larger group difference)")
ax.set_ylabel("two-sided p value")
ax.set_ylim(0, 0.13)
ax.set_xlim(4.5, 14.8)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.legend(loc="upper left", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ----------------------------------------------------------------- design E: ledger of the two comparisons
def design_E(path):
fig, ax = plt.subplots(figsize=(7.2, 2.8))
ax.set_xlim(0, 1)
ax.set_ylim(-0.12, 1)
ax.axis("off")
cols = [0.01, 0.30, 0.47, 0.71, 0.89]
heads = ["comparison", "printed p\n(reported)", "p < 0.05?\n(recomputed)",
"legend\nwording", "consistent?"]
for x, h in zip(cols, heads):
ax.text(x, 0.90, h, fontsize=10, color=INK2, va="top", fontweight="bold")
ax.plot([0.01, 0.99], [0.66, 0.66], color=GRID, lw=1.2)
rows = [("WT vs TRPV4KO\n(n = 7 or 8 eyes)", P_KO, 0.47),
("WT vs TRPV4CKO\n(n = 6 eyes)", P_CKO, 0.17)]
for lab, p, y in rows:
ok = p < ALPHA
ax.text(cols[0], y, lab, fontsize=10.5, va="center")
ax.text(cols[1], y, f"{p}", fontsize=11, va="center", color=INK)
ax.plot(cols[1] - 0.018, y, "o", ms=MS, color=ORANGE)
ax.text(cols[2], y, "yes" if ok else "no (> 0.05)", fontsize=10.5, va="center")
ax.plot(cols[2] - 0.018, y, "s", ms=MS, color=BLUE, mfc=BG, mew=LW)
ax.text(cols[3], y, "\"significantly\nless\"", fontsize=10.5, va="center")
ax.text(cols[4], y, "yes" if ok else "NO", fontsize=11, va="center",
fontweight="normal" if ok else "bold")
ax.plot([0.01, 0.99], [0.32, 0.32], color=GRID, lw=0.8)
ax.text(0.01, -0.08, "orange dot: printed in the paper; blue square: recomputed check", fontsize=10, color=INK2, va="bottom")
save(fig, path)
print("\n5. Figures")
design_A(os.path.join(HERE, "fig1"))
design_B(os.path.join(HERE, "fig2"))
if "--candidates" in sys.argv:
cd = os.path.join(HERE, "candidates")
os.makedirs(cd, exist_ok=True)
design_A(os.path.join(cd, "A_pvalue_axis"))
design_B(os.path.join(cd, "B_all_pvalues"))
design_C(os.path.join(cd, "C_text_panels"))
design_D(os.path.join(cd, "D_exact_ladder"))
design_E(os.path.join(cd, "E_ledger"))
Reported renal-cancer P=0.004 is P=0.24 under the paper's own stated Fisher test
JAMA (2010) · doi:10.1001/jama.2010.1877 · PMC3655326
Summary
The paper compares the prevalence of cancers in Cowden or Cowden-like (CS/CSL) patients with germline KILLIN promoter methylation against patients with germline PTEN mutations. The Abstract and the Results both report renal cell carcinoma in 4/45 methylation-positive patients and 6/155 PTEN-mutation-positive patients, with P=0.004. The Methods say these prevalences were compared with a two-tailed Fisher exact test. On those counts that test gives P=0.237 (odds ratio 2.42; exact 95% CI 0.48–10.72), and none of the other standard tests for a 2×2 table gives a value below P=0.09. The renal association is one of the two cancer risks named in the abstract Conclusions, and the Comment bases advice on kidney surveillance and test prioritization on it. The paper's primary finding (methylation in 45/123 patients) and its breast-cancer comparison are not affected.
What the paper reports
Abstract, Results:
Individuals with KILLIN-promoter methylation had 3-fold increased prevalence of breast cancer (35/42 vs 24/64, P<0.0001) and a >2-fold increase of kidney cancer (4/45 vs 6/155, P=0.004) over those with germline PTEN mutations.
Abstract, Conclusions:
Germline KILLIN- methylation is common among CS/CSL patients, and associated with increased risks of breast and renal cancer over PTEN mutation-positive patients. These observations need to be replicated.
Results, paragraph "Prevalent Cancers in Pathogenic PTEN Mutation Positive versus KILLIN Methylation":
Renal cell carcinoma was over-represented in the methylation positive subjects over PTEN mutation positive individuals (4/45 vs 6/155, P=0.004).
Methods, "Statistical Analysis":
The prevalence of component malignancies between KILLIN promoter methylation positive patients and germline pathogenic PTEN mutation positive patients was compared using the Fisher’s 2-tailed exact test with P<0.05 considered to be significant.
Comment, second paragraph:
Patients with CS-like features, especially where breast cancer and/or renal carcinomas are present in the individual and/or family, should be offered KILLIN methylation analysis first
Comment, third paragraph:
In contrast to germline PTEN mutations, germline methylation of the KILLIN promoter confers a significantly higher prevalence of female invasive breast cancer and renal cell carcinomas. The current national practice guidelines for those with PTEN germline mutations includes heightened surveillance of the female breasts and thyroid, but do not have awareness of renal cancer risk. If our observations of two- to three-fold increased risks of renal and/or breast cancer with KILLIN germline methylation over those of PTEN mutation holds, then extra vigilance for the organs-at-risk, breast and kidneys, is warranted.
Comment, "Conclusions":
Among CS/CSL patients, presence of germline KILLIN gene promoter hypermethylation was common and was associated with increased risk of breast and renal cancer compared with PTEN mutation-positive patients.
Every quote was checked verbatim against the article as archived in PubMed Central (the author manuscript). The publisher's version of record gives the same counts and P value in JAMA house style ("4/45 vs 6/155; P = .004").
What the paper's own numbers show
Method. The reported counts give the 2×2 table [[4, 41], [6, 149]]: renal cell carcinoma present or absent among 45 methylation-positive and 155 PTEN-mutation-positive patients. I applied the test the Methods name (two-tailed Fisher exact), then the other common tests for this table, to see whether any of them gives 0.004. I also worked out which counts would be needed to reach P=0.004. As a control, I recomputed the breast-cancer comparison from the same sentence and the methylation prevalence the same way. Every value below is printed by recompute.py.txt (scipy 1.18.1). The paper has no tables holding these counts; they appear only in the text.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Renal prevalence, methylated vs PTEN | Abstract Results; Results | 4/45 vs 6/155, ">2-fold" | 8.9% vs 3.9%; prevalence ratio 2.30 (consistent) |
| Expected renal cases among the 45 if prevalence were equal | derived | — | 2.25 (observed 4; 10 cases among 200 patients) |
| Fisher exact, two-tailed (the test the Methods name) | Abstract Results; Results | P=0.004 | P=0.237 |
| Fisher exact, one-tailed | same counts | — | P=0.164 |
| Fisher one-tailed mid-p | same counts | — | P=0.106 |
| Pearson chi-square, uncorrected / Yates-corrected | same counts | — | P=0.174 / 0.331 |
| Barnard / Boschloo exact, two-sided | same counts | — | P=0.212 / 0.208 |
| Exact binomial, 4/45 against a fixed rate of 6/155 (one-sided) | same counts | — | P=0.096 |
| Odds ratio (sample), Wald 95% CI, Wald P | same counts | not reported | 2.42 (0.65–8.99), P=0.186 |
| Odds ratio (conditional MLE), exact 95% CI | same counts | not reported | 2.41 (0.48–10.72) |
| Renal cases among the 45 needed for two-tailed Fisher P≤0.004 (PTEN group held at 6/155) | sensitivity | 4 | 8 (P=0.0038) |
| Renal cases among the 45 needed for P<0.05 (PTEN group held at 6/155) | sensitivity | 4 | 6 (P=0.029) |
| PTEN-group renal cases needed for P≤0.004 (methylated group held at 4/45) | sensitivity | 6/155 | 0/155 (P=0.0023); 1/155 gives P=0.0096 |
| Breast cancer, 35/42 vs 24/64 (control check) | Abstract Results; Results | P<0.0001 | Fisher two-tailed P=3.1×10⁻⁶ (consistent) |
| Methylation prevalence (control check) | Abstract Results | 45/123, 37% (95% CI 29–45%) | 36.6%; Wald 28.1–45.1%, exact 28.1–45.7% (consistent) |
The reported P value is about 1/59 of the value the stated test gives. The ">2-fold" prevalence ratio is correct. Only the P value, and the claim of statistical significance that rests on it, do not follow from the counts.
A smaller point, which does not change any conclusion: the Abstract calls the breast-cancer difference a "3-fold increased prevalence", but 35/42 vs 24/64 is 83.3% vs 37.5%, a prevalence ratio of 2.22 (odds ratio 8.33).
Figure
Figure 1. P values for renal cell carcinoma in 4/45 KILLIN-methylation-positive vs 6/155 PTEN-mutation-positive patients, on one logarithmic axis. The orange point is the value reported in the Abstract and Results (P=0.004). Blue points are recomputed from the same counts; the larger blue point is the two-tailed Fisher exact test named in the Methods (P=0.237), joined to the reported value by a dotted line. The dashed line marks the paper's significance threshold, P=0.05. Every recomputed value lies to the right of the threshold, between 0.096 and 0.331, so the conclusion does not depend on the choice of test.
Figure 2. Two-tailed Fisher exact P (blue, log scale) for x/45 methylation-positive patients with renal cancer against the reported 6/155 in the PTEN-mutation group, for x from 0 to 12. At the reported count of 4/45 the test gives P=0.237. The reported P=0.004 (orange dashed line) is first reached at 8/45 (P=0.0038), twice the reported count. A count of 6/45 would be needed to cross P=0.05.
Why it matters
- It changes significance. With the stated test, the renal comparison is far from the paper's own threshold of P<0.05 (P=0.24). The exact 95% CI for the odds ratio, 0.48–10.72, includes 1. With only 10 renal cancers among 200 patients, the data are compatible both with no difference and with a large one.
- It is a headline result. "Prevalence of component cancers in methylation-positive and PTEN mutation-positive patients" is a listed Main Outcome Measure. The renal association appears in the Abstract Results, the Abstract Conclusions ("associated with increased risks of breast and renal cancer") and the closing Conclusions.
- Clinical advice depends on it. The Comment says that current PTEN guidelines "do not have awareness of renal cancer risk", that "extra vigilance for the organs-at-risk, breast and kidneys, is warranted", and that patients with renal carcinomas in the individual or family should be offered KILLIN methylation analysis first. With the P value corrected, the paper's data no longer give statistically significant support for the kidney part of this advice.
- What is not affected. The primary finding (germline methylation in 45/123 PTEN-mutation-negative CS/CSL patients), the expression experiments, and the breast-cancer association (recomputed P=3.1×10⁻⁶) do not depend on this number. The paper itself calls the observations in need of replication and notes that the small sample "may result in type II error".
Explanations considered
- A different test was used. This does not account for the value. The Methods name the test exactly ("Fisher’s 2-tailed exact test"). I also tried one-tailed Fisher, mid-p, chi-square with and without continuity correction, Barnard, Boschloo, a binomial test against a fixed reference rate, and a Wald test on the odds ratio. The smallest result, 0.096, is 24 times larger than 0.004 (Figure 1).
- A slip between sections, or in conversion to PMC. This does not account for the value. The counts and the P value are identical in the Abstract and the Results, and the publisher's version of record matches the PMC archive. The values are plain text in prose, so no table cell, minus sign or decimal-point conversion is involved. A typographical error in the P value itself remains possible, but no standard analysis of the stated counts gives a value that rounds to 0.004.
- The counts, rather than the P value, are in error. Possible, but the paper gives no basis for it. The denominator 45 matches the number of methylation-positive patients reported elsewhere (45 of 123). With the PTEN group fixed at 6/155, reaching P≤0.004 requires 8/45 renal cancers. With the methylated group fixed at 4/45, it requires 0/155 in the PTEN group (P=0.0023). Keeping 6 PTEN-group cases, the group would have to be several times larger (6/500 gives P=0.0059; 6/1000 gives P=0.0005). The paper reports none of these figures. The 155-patient PTEN-mutation group is not described further in the text, whereas the breast comparison uses 64 PTEN-mutation-positive women.
- Other denominators in the paper. These do not account for the value. Substituting the breast-comparison denominators (4/42 vs 6/64), or 4/45 vs 6/64, gives P=1.0; 4/42 vs 6/155 gives P=0.225.
- Adjustment, stratification or person-time analysis. None is described. The paper reports crude prevalences, and the Methods name only the Fisher test for this comparison.
- The supplement holds a different analysis. Not verified. PMC lists one supplementary file. According to the text it holds the CS diagnostic criteria, primer sequences and a per-patient list of the methylation-positive individuals (eTable 1). It could not be retrieved (see Status). Nothing in the text says it holds a different renal analysis. eTable 1 might confirm or change the count of 4 renal cancers among the 45, but no table can make 4/45 vs 6/155 give P=0.004 under the stated test.
- Rounding. P=0.237 cannot round to 0.004.
None of these explanations accounts for the reported value.
Status
As of 2026-10-23:
- Version of record. The publisher's page (jamanetwork.com/journals/jama/fullarticle/10.1001/jama.2010.1877; JAMA 2010;304(24):2724) shows the same counts and "P = .004" in both the Abstract Results and the Results, and the same Methods wording on the two-tailed Fisher test. The PubMed abstract (PMID 21177507) and the PMC author manuscript (NIHMS467022) agree.
- Supplement. One file is listed in PMC ("KLB Supp_data_JAMA_revised.doc"). It could not be retrieved: the PMC download returned a proof-of-work/captcha page, Europe PMC reports the article as outside the open-access subset, and the JAMA supplemental-content tab showed no file link to an automated fetch. Its contents as described in the text are listed above.
- Corrections and notices. None found. The Crossref
updatesfilter for this DOI returns 0 records (re-queried 2026-10-23), and the work record has noupdate-toorupdated-by. PMC and Europe PMC list no correction or retraction notice (src/known.py). PubMed links only an accompanying editorial (JAMA 2010;304(24):2744-5) as "Comment in", and no erratum. The JAMA article page shows no correction notice. - PubPeer and other discussion. Earlier web searches found no PubPeer thread and no published comment on this point. The PubPeer API could not be queried directly (Cloudflare block, again on 2026-10-23), and a fresh web search on 2026-10-23 could not be run because the session's search budget was exhausted. Press coverage at the time repeated the >2-fold kidney-cancer finding.
- Sources. PMC3655326 (local archive via
src/show.py); the publisher's version of record; Crossref API; PubMed E-utilities; Europe PMC; web searches for correction, erratum and PubPeer. Recomputation:recompute.py.txtin this folder.
Suggested correction
In the Abstract Results and in the Results paragraph on prevalent cancers, replace "P=0.004" for renal cell carcinoma (4/45 vs 6/155) with the two-tailed Fisher exact value, P=0.24, and preferably give the odds ratio with its exact 95% CI (2.41, 0.48–10.72). Then revise the statements that rest on this result: the abstract Conclusions, the Comment ("confers a significantly higher prevalence of ... renal cell carcinomas", and the kidney-surveillance and testing-priority statements) and the closing Conclusions. They could say that renal cancer was more common in the methylation-positive group (8.9% vs 3.9%) but that the difference was not statistically significant in this sample. If the counts, rather than the P value, are in error, the correction should give the correct counts and describe the 155-patient PTEN-mutation comparison group. Separately, the breast-cancer difference described as "3-fold" corresponds to a prevalence ratio of 2.2 (83.3% vs 37.5%).
Recomputation script (python recompute.py.txt)
"""Recompute the renal-cancer comparison of PMC3655326
(JAMA 2010;304(24):2724-2731, doi:10.1001/jama.2010.1877).
Run: python recompute.py.txt
Needs numpy, scipy (>= 1.10 for odds_ratio / barnard / boschloo) and matplotlib.
Prints every recomputed number used in writeup.md and writes fig1.svg/png and
fig2.svg/png next to this file.
Inputs are copied from the paper's Abstract and Results ("Prevalent Cancers in
Pathogenic PTEN Mutation Positive versus KILLIN Methylation"). The paper has no
tables holding these counts; they appear only in the text.
Renal cell carcinoma: 4/45 KILLIN-methylated vs 6/155 PTEN-mutation-positive, reported P=0.004
Female invasive breast cancer: 35/42 vs 24/64, reported P<0.0001
Methylation prevalence: 45/123, reported 37% (95% CI 29-45%)
Methods: "compared using the Fisher's 2-tailed exact test with P<0.05 considered to be significant"
"""
import os
import numpy as np
from scipy import stats
from scipy.stats.contingency import odds_ratio
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (from the paper)
RENAL = dict(a=4, n1=45, c=6, n2=155, reported_p=0.004)
BREAST = dict(a=35, n1=42, c=24, n2=64)
METH = dict(k=45, n=123)
ALPHA = 0.05
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, TXT, TXT2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
def table(a, n1, c, n2):
return [[a, n1 - a], [c, n2 - c]]
def fisher2(a, n1, c, n2):
return stats.fisher_exact(table(a, n1, c, n2), alternative="two-sided")[1]
def main():
r = RENAL
t = table(r["a"], r["n1"], r["c"], r["n2"])
p1, p2 = r["a"] / r["n1"], r["c"] / r["n2"]
print("=== Renal cell carcinoma: 4/45 (KILLIN-methylated) vs 6/155 (PTEN mutation+) ===")
print(f"prevalence {100*p1:.1f}% vs {100*p2:.1f}%; prevalence ratio {p1/p2:.2f}")
sample_or = stats.fisher_exact(t)[0]
cond = odds_ratio(t, kind="conditional")
ci = cond.confidence_interval(0.95)
print(f"sample odds ratio {sample_or:.2f}; conditional MLE OR {cond.statistic:.2f}, "
f"exact 95% CI {ci.low:.2f}-{ci.high:.2f}")
(a_, b_), (c_, d_) = t
se_log = np.sqrt(1 / a_ + 1 / b_ + 1 / c_ + 1 / d_)
lo, hi = np.exp(np.log(sample_or) + np.array([-1, 1]) * 1.959964 * se_log)
p_wald = 2 * stats.norm.sf(abs(np.log(sample_or)) / se_log)
print(f"sample OR Wald 95% CI {lo:.2f}-{hi:.2f}, Wald P {p_wald:.3f}")
chi = stats.chi2_contingency(t, correction=False)
chiy = stats.chi2_contingency(t, correction=True)
print(f"total events {r['a'] + r['c']} of {r['n1'] + r['n2']}; "
f"expected in methylated group under no difference {chi.expected_freq[0][0]:.2f} (observed {r['a']})")
results = {}
results["Fisher exact, two-sided\n(the test named in Methods)"] = fisher2(r["a"], r["n1"], r["c"], r["n2"])
results["Fisher exact, one-sided"] = stats.fisher_exact(t, alternative="greater")[1]
pmf = [stats.hypergeom.pmf(k, 200, 10, 45) for k in range(11)]
results["Fisher one-sided mid-p"] = sum(pmf[5:]) + 0.5 * pmf[4]
results["Pearson chi-square"] = chi.pvalue
results["Yates-corrected chi-square"] = chiy.pvalue
results["Barnard exact, two-sided"] = stats.barnard_exact(t).pvalue
results["Boschloo exact, two-sided"] = stats.boschloo_exact(t).pvalue
results["Binomial 4/45 vs fixed rate 6/155,\none-sided"] = stats.binomtest(4, 45, 6 / 155, alternative="greater").pvalue
for k, v in results.items():
print(f" {k.replace(chr(10), ' '):55s} P = {v:.4f}")
p_fisher = results["Fisher exact, two-sided\n(the test named in Methods)"]
print(f"reported P=0.004; recomputed two-sided Fisher P={p_fisher:.3f}, "
f"ratio {p_fisher / r['reported_p']:.0f}x")
print(f"smallest recomputed P across all tests: {min(results.values()):.4f}")
print("\n=== Counts that would give two-sided Fisher P near 0.004 ===")
sweep_a = {a: fisher2(a, 45, 6, 155) for a in range(0, 13)}
for a, p in sweep_a.items():
print(f" {a:2d}/45 vs 6/155: P = {p:.5f}")
need = min(a for a, p in sweep_a.items() if p <= r["reported_p"] and a > 4)
print(f"smallest methylated count (of 45) with P <= 0.004, holding 6/155: {need} (P = {sweep_a[need]:.4f})")
need05 = min(a for a, p in sweep_a.items() if p < ALPHA and a > 4)
print(f"smallest methylated count (of 45) with P < 0.05, holding 6/155: {need05} (P = {sweep_a[need05]:.4f})")
sweep_c = {c: fisher2(4, 45, c, 155) for c in range(0, 7)}
for c, p in sweep_c.items():
print(f" 4/45 vs {c}/155: P = {p:.5f}")
print(" other denominators (from the paper, then hypothetical larger PTEN groups):")
for n1, n2 in [(42, 64), (45, 64), (42, 155), (45, 500), (45, 1000)]:
print(f" 4/{n1} vs 6/{n2}: P = {fisher2(4, n1, 6, n2):.4f}")
print(f"scipy {__import__('scipy').__version__}")
print("\n=== Breast cancer 35/42 vs 24/64 (reported P<0.0001, '3-fold') ===")
b = BREAST
bt = table(b["a"], b["n1"], b["c"], b["n2"])
bp1, bp2 = b["a"] / b["n1"], b["c"] / b["n2"]
print(f"prevalence {100*bp1:.1f}% vs {100*bp2:.1f}%; prevalence ratio {bp1/bp2:.2f}; "
f"sample odds ratio {stats.fisher_exact(bt)[0]:.2f}")
print(f"two-sided Fisher P = {fisher2(**{k: b[k] for k in ('a','n1','c','n2')}):.1e}")
print("\n=== Methylation prevalence 45/123 (reported 37%, 95% CI 29-45%) ===")
ph = METH["k"] / METH["n"]
se = np.sqrt(ph * (1 - ph) / METH["n"])
cp = stats.binomtest(METH["k"], METH["n"]).proportion_ci(method="exact")
print(f"{100*ph:.1f}%; Wald 95% CI {100*(ph-1.96*se):.1f}-{100*(ph+1.96*se):.1f}%; "
f"Clopper-Pearson {100*cp.low:.1f}-{100*cp.high:.1f}%")
fig1(results, r["reported_p"])
fig2(sweep_a, r["reported_p"])
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(TXT2)
ax.tick_params(colors=TXT2, labelsize=10)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
def fig1(results, reported):
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans"})
labels = ["Reported in the paper\n(Abstract and Results)"] + list(results.keys())
vals = [reported] + list(results.values())
y = np.arange(len(labels))[::-1]
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
style(ax)
ax.set_xscale("log")
ax.set_xlim(0.001, 1.0)
ax.axvline(ALPHA, color=TXT2, lw=1, ls=(0, (4, 3)))
ax.text(ALPHA * 1.08, y[0] + 0.55, "P = 0.05\n(paper's threshold)", color=TXT2, fontsize=10, va="center")
# gap between reported and recomputed Fisher value
ax.plot([vals[0], vals[1]], [y[0], y[1]], color=TXT2, lw=1, ls=":")
for i, (lab, v) in enumerate(zip(labels, vals)):
col = ORANGE if i == 0 else BLUE
ax.plot(v, y[i], "o", ms=10 if i < 2 else 8, color=col, zorder=3)
ax.text(v * (0.82 if i == 0 else 1.15), y[i], f"{v:.3f}", color=col, fontsize=10,
va="center", ha="right" if i == 0 else "left", fontweight="bold" if i < 2 else "normal")
ax.set_yticks(y)
ax.set_yticklabels(labels, fontsize=10, color=TXT)
ax.tick_params(axis="y", length=0)
ax.set_xticks([0.001, 0.004, 0.01, 0.05, 0.1, 0.25, 1.0])
ax.set_xticklabels(["0.001", "0.004", "0.01", "0.05", "0.1", "0.25", "1"])
ax.set_xlabel("P value for renal cancer, 4/45 vs 6/155 (log scale)", color=TXT, fontsize=10)
ax.set_ylim(y[-1] - 0.7, y[0] + 1.1)
h1, = ax.plot([], [], "o", ms=9, color=ORANGE, label="Reported by the paper")
h2, = ax.plot([], [], "o", ms=9, color=BLUE, label="Recomputed from the paper's counts")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.3, -0.16), ncol=2,
frameon=False, fontsize=10, labelcolor=TXT)
fig.tight_layout()
save(fig, "fig1")
def fig2(sweep, reported):
a = np.array(list(sweep.keys()))
p = np.array(list(sweep.values()))
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
style(ax)
ax.grid(axis="y", which="major", color=GRID, lw=0.8)
ax.set_yscale("log")
ax.plot(a, p, "-", color=BLUE, lw=2, zorder=2)
ax.plot(a, p, "o", ms=8, color=BLUE, zorder=3, label="Two-sided Fisher P, recomputed (6/155 held fixed)")
ax.axhline(reported, color=ORANGE, lw=2, ls="--", zorder=1)
ax.text(12.3, reported * 1.25, "Reported P = 0.004", color=ORANGE, ha="right", va="bottom", fontsize=10)
ax.axhline(ALPHA, color=TXT2, lw=1, ls=(0, (4, 3)))
ax.text(12.3, ALPHA * 1.15, "P = 0.05", color=TXT2, ha="right", va="bottom", fontsize=10)
ax.plot(4, sweep[4], "o", ms=13, mfc="none", mec=TXT, mew=1.5, zorder=4)
ax.annotate(f"Reported count 4/45\nP = {sweep[4]:.3f}", (4, sweep[4]), xytext=(5.6, 0.55),
fontsize=10, color=TXT, arrowprops=dict(arrowstyle="-", color=TXT2, lw=1))
ax.plot(8, sweep[8], "o", ms=13, mfc="none", mec=TXT, mew=1.5, zorder=4)
ax.annotate(f"8/45 would be needed\nP = {sweep[8]:.4f}", (8, sweep[8]), xytext=(8.6, 0.012),
fontsize=10, color=TXT, arrowprops=dict(arrowstyle="-", color=TXT2, lw=1))
ax.set_xticks(a)
ax.set_xlim(-0.5, 12.5)
ax.set_ylim(1e-5, 1.6)
ax.set_xlabel("Renal cancers among the 45 KILLIN-methylated patients (comparison group fixed at 6/155)",
color=TXT, fontsize=10)
ax.set_ylabel("Two-sided Fisher P (log scale)", color=TXT, fontsize=10)
ax.legend(loc="lower left", frameon=False, fontsize=10, labelcolor=TXT)
fig.tight_layout()
save(fig, "fig2")
if __name__ == "__main__":
main()
Two pooled pericarditis hazard ratios in Table 2 fall outside their own cohort estimates
JAMA Cardiol (2026) · doi:10.1001/jamacardio.2026.0258 · PMC13000738
Summary
The letter pools two biobank cohorts by inverse-variance meta-analysis and reports pooled pericarditis hazard ratios (HRs) of 1.66 (95% CI, 1.17-2.35) for large CHIP and 1.88 (1.45-2.43) for DNMT3A CHIP. Neither value can come from the cohort rows printed above it in Table 2: any weighted average of the large-CHIP cohort HRs (2.15 and 1.71) must lie between 1.71 and 2.15, and any weighted average of the DNMT3A cohort HRs (1.70 and 1.63) cannot exceed 1.70. Pooling the table's own cohort rows gives 1.86 (1.43-2.42) for large CHIP and 1.66 (1.17-2.35) for DNMT3A, so the printed pooled rows match the recomputed values with the two exposures exchanged, and the myocarditis block shows the same pattern. Two interpretive sentences in the Results rest on the printed values. The main finding is not affected: the any-CHIP point estimate recomputes (HR 1.65), with a printed CI (1.29-2.10) slightly narrower than the recomputed one (1.28-2.12).
What the paper reports
Methods, second paragraph:
Inverse variance-weighted meta-analysis (using fixed-effects models when I2 < 25% and random-effects models otherwise) was performed.
Results, first paragraph:
Any CHIP was associated with 1.65-fold increased pericarditis risk (95% CI, 1.29-2.10; I2 = 0). Large CHIP (variant allele fraction ≥10%) had a similar point estimate (hazard ratio [HR], 1.66 [95% CI, 1.17-2.35]; I2 = 0). DNMT3A CHIP accounted for the association (HR, 1.88 [95% CI, 1.45-2.43]; I2 = 0), while TET2 CHIP showed a directionally concordant association (1.35 [0.76-2.40]).
Results, second paragraph (myocarditis):
Point estimates for any CHIP (HR, 1.29 [95% CI, 0.84-1.97]), large CHIP (HR, 1.53 [0.92-2.54]), and DNMT3A CHIP (HR, 1.30 [0.81-2.09]) were positive but not significant.
There was moderate heterogeneity between cohorts (I2 = 37.1%-68.2%).
Table 2, "Multivariable-Adjusted Associations of CHIP and Gene-Specific CHIP Subtypes With Myocarditis and Pericarditis After Meta-Analysis of the VUMC BioVU and NIH All of Us Cohorts". These are the rows at issue (participants | events | adjusted HR (95% CI) | P):
| Outcome, exposure | Row | Participants | Events | HR (95% CI) | P |
|---|---|---|---|---|---|
| Pericarditis, Large CHIP | All of Us cohort | 6989 | 23 | 2.15 (1.40-3.30) | <.001 |
| BioVU cohort | 7562 | 39 | 1.71 (1.23-2.38) | .001 | |
| Meta-analysis | 14 551 | 62 | 1.66 (1.17-2.35) | .004 | |
| Pericarditis, DNMT3A CHIP | All of Us cohort | 4835 | 13 | 1.70 (0.98-2.97) | .06 |
| BioVU cohort | 3754 | 20 | 1.63 (1.04-2.56) | .03 | |
| Meta | 8589 | 33 | 1.88 (1.45-2.43) | <.001 | |
| Myocarditis, Large CHIP | All of Us cohort | 7013 | 3 | 0.54 (0.17-1.68) | .28 |
| BioVU cohort | 7573 | 16 | 1.56 (0.93-2.61) | .09 | |
| Meta-analysis | 14 586 | 19 | 1.53 (0.92-2.54) | .10 | |
| Myocarditis, DNMT3A CHIP | All of Us cohort | 4839 | 4 | 1.02 (0.38-2.75) | .97 |
| BioVU cohort | 3756 | 10 | 1.87 (0.99-3.53) | .05 | |
| Meta-analysis | 8595 | 14 | 1.30 (0.81-2.09) | .28 |
Discussion, first paragraph:
We and [ref 2] found that DNMT3A—but not TET2—CHIP was associated with pericarditis, although this may simply reflect statistical power.
(Bracketed text replaces the cited study's author name.)
What the paper's own numbers show
Method. I applied the method stated in Methods to every block of Table 2. Each cohort's log HR was taken with a standard error recovered from its 95% CI, SE = [ln(upper) − ln(lower)] / (2 × 1.96). The two log HRs were pooled with inverse-variance weights, and I computed Cochran's Q and I². I used the fixed-effect result when I² < 25% and the DerSimonian-Laird random-effects result otherwise, as the paper specifies. There is also a check that needs no standard errors at all: a pooled estimate built from positive weights is a weighted average of the cohort log HRs, so it must lie between the two cohort HRs. This holds for fixed and random effects and for pooling on either the log or the HR scale. To test whether rounding to two decimals could matter, I varied every printed cohort HR and CI bound by ±0.005 (all 729 combinations of −0.005, 0 and +0.005, plus 20 000 random draws) in all eight blocks of Table 2, under both fixed and random effects. A printed two-decimal value is counted as fitting when its own rounding interval (±0.005) overlaps the recomputed range. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own rows |
|---|---|---|---|
| Range any weighted average can take, pericarditis, large CHIP | Table 2 cohort rows | 2.15 and 1.71 | must lie in [1.71, 2.15] |
| Pooled HR, pericarditis, large CHIP | Table 2; Results ¶1 | 1.66 (1.17-2.35), P = .004 | 1.86 (1.43-2.42), P = 3.2 × 10⁻⁶; Q = 0.69, I² = 0% (fixed and random effects identical) |
| Same, allowing for rounding of inputs | Table 2 | 1.66 | 1.854 to 1.870 |
| SE of pooled log HR, pericarditis, large CHIP | Table 2 | 0.178 (implied by printed CI) | 0.133; a pooled SE cannot exceed the smaller cohort SE (BioVU 0.168) when τ² = 0 |
| Range any weighted average can take, pericarditis, DNMT3A | Table 2 cohort rows | 1.70 and 1.63 | must lie in [1.63, 1.70] |
| Pooled HR, pericarditis, DNMT3A | Table 2; Results ¶1 | 1.88 (1.45-2.43), P < .001 | 1.66 (1.17-2.35), P = .0046; Q = 0.01, I² = 0% |
| Same, allowing for rounding of inputs | Table 2 | 1.88 | 1.652 to 1.663 |
| SE of pooled log HR, pericarditis, DNMT3A | Table 2 | 0.132 (implied by printed CI) | 0.178 |
| Participant and event totals, large CHIP pericarditis | Table 2 | 14 551; 62 | 6989 + 7562 = 14 551; 23 + 39 = 62 (agree) |
| Participant and event totals, DNMT3A pericarditis | Table 2 | 8589; 33 | 4835 + 3754 = 8589; 13 + 20 = 33 (agree) |
| Pooled HR, myocarditis, large CHIP | Table 2; Results ¶2 | 1.53 (0.92-2.54), P = .10 | fixed effect 1.30 (0.81-2.09), P = .27; I² = 63.5% (61.6% to 65.2% under rounding), so the stated rule gives random effects 1.04 (0.38-2.86), P = .93 |
| Pooled HR, myocarditis, DNMT3A | Table 2; Results ¶2 | 1.30 (0.81-2.09), P = .28 | 1.57 (0.92-2.67), P = .10; I² = 2.0% |
The other four pooled rows. Re-pooled the same way, the two any-CHIP rows and the pericarditis TET2 row come close to the printed values, and the myocarditis TET2 row does not. Under the rounding test, none of the four is reproduced exactly:
| Pooled row (Table 2) | Reported | Recomputed (rule-chosen model) | Range allowing rounding of inputs | Assessment |
|---|---|---|---|---|
| Pericarditis, any CHIP | 1.65 (1.29-2.10) | 1.65 (1.28-2.12), fixed effect, I² = 0% | HR 1.639-1.651; lower 1.269-1.282; upper 2.110-2.134 | HR fits. Printed CI lies inside every recomputed CI: implied SE 0.124 (0.123-0.126) vs 0.129-0.131 recomputed |
| Pericarditis, TET2 | 1.35 (0.76-2.40), P = .31 | 1.38 (0.78-2.44), P = .27, fixed effect, I² = 0% | HR 1.371-1.387; lower 0.773-0.784; upper 2.426-2.463 | Close (1.38 vs 1.35), just outside the rounding range 1.37-1.39; CI width fits (SE 0.293 vs 0.290-0.294) |
| Myocarditis, any CHIP | 1.29 (0.84-1.97), P = .24 | 1.32 (0.86-2.01), P = .20, fixed effect, I² = 11.7% (7.0%-16.2%) | HR 1.307-1.326; lower 0.857-0.871; upper 1.990-2.021 | Close (1.32 vs 1.29), just outside the rounding range 1.31-1.33; the lower bound 0.84 is also outside 0.86-0.87 |
| Myocarditis, TET2 | 0.69 (0.21-2.26), P = .54 | 0.90 (0.29-2.82), P = .86, fixed effect, I² = 13.8% (8.0%-19.0%) | Fixed effect HR 0.880-0.925; random effects 0.848-0.904 | Not reproducible (0.88-0.93 under rounding), but 0.69 lies between the cohort HRs 0.37 and 1.40, so it is not an impossible value |
So the printed pooled rows are not, in general, an exact inverse-variance pool of the printed cohort rows, and differences of about 0.01 to 0.03 on the log scale appear even in rows with no other problem (any CHIP and TET2 for pericarditis, any CHIP for myocarditis). The large-CHIP and DNMT3A pericarditis rows differ by 0.115 and 0.126 in the HR alone and, unlike those rows, lie outside the range of their own cohort HRs. Myocarditis TET2 is a separate case: its printed HR is 0.268 on the log scale from the recomputed value, which is more than the large-CHIP and DNMT3A myocarditis rows (0.160 and 0.187). Both cohort CIs in that block are close to symmetric on the log scale (1.386 vs 1.397, and 2.001 vs 1.961), as Wald intervals are, so recovering SEs from the CIs does not explain the gap. The block has only 3 events (2 and 1), and I treat it as a side observation, not as part of the main finding.
The exchange pattern. The recomputed DNMT3A pericarditis pool, 1.66 (1.17-2.35) with P = .0046, reproduces the printed large-CHIP row, 1.66 (1.17-2.35) with P = .004. The largest log difference across HR and both bounds is 0.002, and all three printed values fall within the recomputed rounding ranges (HR 1.652-1.663, lower 1.162-1.175, upper 2.338-2.364). The recomputed large-CHIP pool, 1.86 (1.43-2.42), is within 0.011 on the log scale of the printed DNMT3A row, 1.88 (1.45-2.43), a gap similar in size to those in the any-CHIP and pericarditis TET2 rows. By contrast, each printed row differs from its own recomputed pool by 0.115 (large CHIP) and 0.126 (DNMT3A) on the log scale. Myocarditis shows the same pattern. The fixed-effect pool of the large-CHIP myocarditis cohorts, 1.30 (0.81-2.09), matches the printed DNMT3A row to two decimals (HR and both CI bounds; largest log difference 0.006; P = .27 recomputed vs .28 printed), and the DNMT3A pool, 1.57 (0.92-2.67) with P = .10, is close to the printed large-CHIP row, 1.53 (0.92-2.54) with P = .10 (largest log difference 0.052). In both outcome blocks the participant and event totals in the pooled rows agree with their own cohort rows. Only the HR, CI and P cells of the large-CHIP and DNMT3A pooled rows look exchanged. This is an inference from the numbers and has not been confirmed by the authors.
On its own, each printed myocarditis value is possible: 1.53 lies between 0.54 and 1.56, and 1.30 between 1.02 and 1.87. So the myocarditis evidence rests on the match with the exchanged values, not on an impossible value. Because the myocarditis TET2 row shows that a printed myocarditis pool can differ from re-pooling by more than these rows do, the myocarditis part of the finding is weaker than the pericarditis part.
Figure
Figure 1. Pericarditis rows of Table 2 for large CHIP (top) and DNMT3A CHIP (bottom), on one log-scale HR axis. Grey diamonds with lines are the two cohort estimates and 95% CIs as printed. Orange squares are the pooled meta-analysis rows as printed. Blue circles are the fixed-effect inverse-variance pools recomputed by recompute.py.txt from the two cohort rows directly above them (I² = 0% in both blocks, so random effects give the same values). The shaded band marks the interval between the two cohort point estimates, where any weighted average has to fall. Both printed pooled values lie outside their band, one below it and one above. Each printed value sits at, or close to, the recomputed value of the other exposure. The thin vertical line marks HR = 1.
Figure 2. The same layout for the myocarditis rows of Table 2. In both blocks the printed values lie inside the band, but the printed DNMT3A pool matches the recomputed fixed-effect large-CHIP pool to two decimals, and the printed large-CHIP pool is close to the recomputed DNMT3A pool. For large CHIP the recomputed I² is 63.5%, so under the paper's stated rule the random-effects estimate, 1.04 (0.38-2.86), would apply. The figure shows the fixed-effect value because that is the one matching a printed row.
Why it matters
- Two interpretive claims rest on the printed values. The Results say large CHIP "had a similar point estimate" to any CHIP (1.66 vs 1.65). Recomputed from the cohort rows, large CHIP is 1.86 (1.43-2.42) against 1.65 for any CHIP. The Results also say "DNMT3A CHIP accounted for the association" with HR 1.88. Recomputed, DNMT3A is 1.66 (1.17-2.35), essentially equal to any CHIP (1.65) rather than above it.
- The stated significance of each estimate changes in degree, not in direction. Recomputed, large CHIP is more significant than printed (P ≈ 3 × 10⁻⁶ rather than .004) and DNMT3A less so (P ≈ .005 rather than < .001). Both stay below .05, so the Discussion's statement that DNMT3A, but not TET2, CHIP was associated with pericarditis still holds.
- The main conclusion is unaffected. The any-CHIP pericarditis point estimate (1.65) recomputes from its cohort rows; the printed CI (1.29-2.10) is slightly narrower than the recomputed 1.28-2.12, which does not change the conclusion. The letter's conclusion that CHIP is a reproducible risk factor for pericarditis does not depend on the exchanged rows, and neither does its title. The letter has no structured abstract or key points that repeat the values.
- These rows are the ones later syntheses would extract. The letter is framed as a replication of a UK Biobank analysis, and its large-CHIP and gene-specific pooled estimates are the figures a comparison or a later meta-analysis would take from it.
Explanations considered
- Rounding of the printed inputs. Does not account for it. Varying every HR and CI bound by ±0.005 moves the large-CHIP pericarditis pool only between 1.854 and 1.870, and the DNMT3A pool only between 1.652 and 1.663. Rounding cannot bring the smaller large-CHIP cohort HR below 1.705 or the larger DNMT3A cohort HR above 1.705. Neither 1.66 nor 1.88 can be reached.
- Random effects instead of fixed effects. Does not account for it. Q is 0.69 and 0.01, so I² = 0 and τ² = 0, and the random-effects pool equals the fixed-effect pool. The paper itself prints I² = 0 for both. In any case a random-effects estimate is still a positively weighted average and must lie between the cohort HRs.
- Different weights or scale (for example, weighting by sample size or events, or averaging HRs rather than log HRs). Does not account for it. Any positively weighted average of 2.15 and 1.71 lies between them, and the same holds for 1.70 and 1.63.
- A one-stage pooled model on individual-level data rather than inverse-variance pooling. Not supported by the paper. Methods specify inverse variance-weighted meta-analysis, and the Results report I² for the pooled estimates, which belongs to a two-stage analysis. It would also not explain why each printed value matches the pool of the other exposure so closely.
- The authors' pooling differs slightly from re-pooling the rounded table values. This is likely, and it accounts for the small gaps in the any-CHIP and TET2 rows (see above): the authors would have pooled unrounded log HRs and SEs, possibly from different software defaults. It cannot account for the two pericarditis rows. Whatever the inputs, SEs or between-study variance, an inverse-variance pool (fixed or random effects) gives each cohort a positive weight, so the pooled HR is a weighted average of the cohort HRs and must lie between them. The printed cohort HRs, even allowing for rounding, bound the large-CHIP pool below by 1.705 and the DNMT3A pool above by 1.705, so 1.66 and 1.88 cannot come from them.
- The cohort rows, not the pooled rows, are wrong. Cannot be excluded from the published text alone. Two points favour an exchange in the pooled rows: the printed pooled values reproduce the recomputed pools of the other exposure to within 0.002 to 0.011 on the log scale, and the cohort rows and pooled totals agree with each other in every block. Only the authors' unrounded cohort estimates can settle it.
- A conversion or typesetting artifact in the archive. Ruled out. The publisher's full text (jamanetwork.com, reached through the DOI) and the PMC archive carry the same numbers cell for cell, and the Results text repeats the same values as the table. There are no minus signs, superscripts or merged cells involved.
- A correction already issued. None found (see Status).
- Side observation, not part of the main finding. The Results report myocarditis heterogeneity of "I2 = 37.1%-68.2%". From the printed rows, the four myocarditis I² values are 11.7%, 63.5%, 2.0% and 13.8%, and only large CHIP exceeds 25%. With two cohorts and rounded inputs I² is imprecise, so I do not treat this as an error, but the authors may wish to check it alongside the pooled rows.
- Side observation: myocarditis TET2. The printed pool, 0.69 (0.21-2.26), cannot be reproduced from its cohort rows, 1.40 (0.35-5.66) and 0.37 (0.05-2.63): re-pooling gives 0.90 (0.29-2.82), and 0.88 to 0.93 allowing for rounding (0.85 to 0.90 under random effects). The value lies within the cohort range, so it is not impossible, and with only 3 events a different estimation approach for this block cannot be excluded. I do not count it as an error.
Status
- Version of record. On 2026-10-24 I checked the publisher's full text at https://jamanetwork.com/journals/jamacardiology/fullarticle/2846692, reached through the DOI. Its Results sentences and the Table 2 rows for large CHIP and DNMT3A CHIP, under both outcomes, are identical to the PMC archive. Europe PMC's full-text XML for this article agrees as well. I checked every quotation above verbatim against the PMC-archived text with
src/show.py. - Supplements. The article has one supplement,
jamacardiol-e260258-s001.pdf(file name as distributed by PMC/Europe PMC), a one-page Data Sharing Statement. It contains no extra tables, forest plots or cohort-level estimates. Individual-level data are available only under BioVU and All of Us data access agreements. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Europe PMC record (PMID 41848713) lists no comments or corrections, and the publisher page links no correction notice. PubPeer's search page returned HTTP 403 and its API asked for human verification, so I could not query it directly. A web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC13000738; https://doi.org/10.1001/jamacardio.2026.0258 (publisher full text); Europe PMC full-text XML and supplementary files; Crossref; Europe PMC; PubPeer; web search.
Suggested correction
In Table 2, replace the pooled (meta-analysis) rows for large CHIP and DNMT3A CHIP, under both pericarditis and myocarditis, with values recomputed from the cohort-level estimates using the stated inverse-variance method. From the rounded table values these are approximately:
| Outcome, exposure | Printed | Recomputed from Table 2 cohort rows |
|---|---|---|
| Pericarditis, large CHIP | 1.66 (1.17-2.35), P = .004 | 1.86 (1.43-2.42), P < .001 |
| Pericarditis, DNMT3A CHIP | 1.88 (1.45-2.43), P < .001 | 1.66 (1.17-2.35), P = .005 |
| Myocarditis, large CHIP | 1.53 (0.92-2.54), P = .10 | fixed effect 1.30 (0.81-2.09), P = .27; or, since I² ≈ 64%, random effects 1.04 (0.38-2.86), P = .93 |
| Myocarditis, DNMT3A CHIP | 1.30 (0.81-2.09), P = .28 | 1.57 (0.92-2.67), P = .10 |
The authors should confirm these values from the unrounded cohort estimates. In the Results, the first paragraph would then read approximately:
Large CHIP (variant allele fraction ≥10%) had a higher point estimate (HR, 1.86 [95% CI, 1.43-2.42]; I2 = 0). DNMT3A CHIP was also associated with pericarditis (HR, 1.66 [95% CI, 1.17-2.35]; I2 = 0)
The myocarditis sentence would be updated in the same way. The authors may also wish to say which model (fixed or random effects) was applied to the large-CHIP myocarditis rows, given an I² of about 64%, and to check the reported myocarditis I² range.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC13000738 (JAMA Cardiol 2026, doi:10.1001/jamacardio.2026.0258)
# "Clonal Hematopoiesis and Risk of Incident Pericarditis and Myocarditis in 2 US Biobank Cohorts"
#
# Re-pools the cohort rows of Table 2 with the method stated in Methods:
# inverse variance-weighted meta-analysis of log hazard ratios, fixed effects when I2 < 25%,
# random effects (DerSimonian-Laird) otherwise. SE of each log HR is taken from its 95% CI.
# Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png) next to this file.
#
# Run: python recompute.py.txt
import itertools
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# Table 2 (as printed): cohort rows [(HR, lo, hi) All of Us, (HR, lo, hi) BioVU], printed pooled row, printed P
T = {
("Pericarditis", "Any CHIP"): ([(1.72, 1.14, 2.59), (1.60, 1.16, 2.22)], (1.65, 1.29, 2.10), "<.001"),
("Pericarditis", "Large CHIP"): ([(2.15, 1.40, 3.30), (1.71, 1.23, 2.38)], (1.66, 1.17, 2.35), ".004"),
("Pericarditis", "DNMT3A CHIP"): ([(1.70, 0.98, 2.97), (1.63, 1.04, 2.56)], (1.88, 1.45, 2.43), "<.001"),
("Pericarditis", "TET2 CHIP"): ([(1.01, 0.32, 3.16), (1.53, 0.79, 2.96)], (1.35, 0.76, 2.40), ".31"),
("Myocarditis", "Any CHIP"): ([(0.93, 0.43, 1.99), (1.53, 0.92, 2.52)], (1.29, 0.84, 1.97), ".24"),
("Myocarditis", "Large CHIP"): ([(0.54, 0.17, 1.68), (1.56, 0.93, 2.61)], (1.53, 0.92, 2.54), ".10"),
("Myocarditis", "DNMT3A CHIP"): ([(1.02, 0.38, 2.75), (1.87, 0.99, 3.53)], (1.30, 0.81, 2.09), ".28"),
("Myocarditis", "TET2 CHIP"): ([(1.40, 0.35, 5.66), (0.37, 0.05, 2.63)], (0.69, 0.21, 2.26), ".54"),
}
# Table 2 participant and event counts for the two pericarditis rows at issue (AoU, BioVU, pooled)
COUNTS = {
"Large CHIP": ((6989, 23), (7562, 39), (14551, 62)),
"DNMT3A CHIP": ((4835, 13), (3754, 20), (8589, 33)),
}
def se_from_ci(lo, hi):
return (np.log(hi) - np.log(lo)) / (2 * Z)
def pool(rows):
b = np.array([np.log(h) for h, _, _ in rows])
s = np.array([se_from_ci(l, u) for _, l, u in rows])
w = 1 / s**2
bf = (w * b).sum() / w.sum()
sf = 1 / np.sqrt(w.sum())
Q = (w * (b - bf) ** 2).sum()
df = len(b) - 1
I2 = max(0.0, (Q - df) / Q) * 100 if Q > 0 else 0.0
tau2 = max(0.0, (Q - df) / (w.sum() - (w**2).sum() / w.sum()))
wr = 1 / (s**2 + tau2)
br = (wr * b).sum() / wr.sum()
sr = 1 / np.sqrt(wr.sum())
model = "RE" if I2 >= 25 else "FE"
bb, ss = (br, sr) if model == "RE" else (bf, sf)
ci = lambda m, e: (np.exp(m), np.exp(m - Z * e), np.exp(m + Z * e))
return dict(FE=ci(bf, sf), RE=ci(br, sr), pFE=2 * stats.norm.sf(abs(bf / sf)),
pRE=2 * stats.norm.sf(abs(br / sr)), I2=I2, Q=Q, model=model,
chosen=ci(bb, ss), p=2 * stats.norm.sf(abs(bb / ss)), se=s, w=w, sef=sf)
f3 = lambda t: "%.2f (%.2f-%.2f)" % t
R = {k: pool(v[0]) for k, v in T.items()}
print("=" * 100)
print("1. Re-pooling every Table 2 block from its own cohort rows")
print("=" * 100)
for k, (rows, rep, prep) in T.items():
r = R[k]
lo = min(h for h, _, _ in rows); hi = max(h for h, _, _ in rows)
print(f"{k[0]:12s} {k[1]:12s} printed {f3(rep)} P{prep}")
print(f"{'':25s} cohort SEs {np.round(r['se'], 3)} weights {np.round(r['w'], 1)} Q={r['Q']:.2f} I2={r['I2']:.1f}%")
print(f"{'':25s} FE {f3(r['FE'])} P={r['pFE']:.2g} | RE {f3(r['RE'])} P={r['pRE']:.2g} | rule -> {r['model']}")
print(f"{'':25s} printed HR inside [{lo:.2f}, {hi:.2f}] (range of cohort HRs)? {lo <= rep[0] <= hi}")
sr = se_from_ci(rep[1], rep[2])
print(f"{'':25s} SE implied by printed CI {sr:.3f}; FE pooled SE from cohorts {r['sef']:.3f}; "
f"smallest cohort SE {r['se'].min():.3f}; P implied by printed HR and CI {2*stats.norm.sf(abs(np.log(rep[0])/sr)):.4f}")
print()
print("=" * 100)
print("2. Pericarditis, Large CHIP and DNMT3A CHIP: the key numbers")
print("=" * 100)
for g in ("Large CHIP", "DNMT3A CHIP"):
rows, rep, prep = T[("Pericarditis", g)]
r = R[("Pericarditis", g)]
lnb = [np.log(h) for h, _, _ in rows]
print(f"{g}: ln HR = {lnb[0]:.4f}, {lnb[1]:.4f}; weights {r['w'][0]:.1f}, {r['w'][1]:.1f}; "
f"pooled ln HR {np.log(r['FE'][0]):.4f}")
print(f" recomputed FE {f3(r['FE'])}, P = {r['pFE']:.2g}, Q = {r['Q']:.2f}, I2 = {r['I2']:.0f}% | printed {f3(rep)}, P {prep}")
(n1, e1), (n2, e2), (n, e) = COUNTS[g]
print(f" counts: {n1}+{n2}={n1+n2} (printed {n}); events {e1}+{e2}={e1+e2} (printed {e})")
print("Any CHIP pericarditis (control): recomputed", f3(R[("Pericarditis", "Any CHIP")]["FE"]),
"printed", f3(T[("Pericarditis", "Any CHIP")][1]))
print()
print("=" * 100)
print("3. Robustness to rounding of the printed inputs (each HR and CI bound varied by +/-0.005)")
print("=" * 100)
rng = np.random.default_rng(1)
def rounding_range(rows, n_rand=20000):
"""Min/max of FE and RE pooled HR, CI bounds, pooled SE and I2 over all inputs within +/-0.005."""
base = np.array(rows, float).ravel()
pts = [base + np.array(c) * 0.005 for c in itertools.product((-1, 0, 1), repeat=6)]
pts += list(base + rng.uniform(-0.005, 0.005, size=(n_rand, 6)))
out = {m: {"hr": [], "lo": [], "hi": [], "se": []} for m in ("FE", "RE")}
I2s = []
for p in pts:
r = pool([tuple(p[0:3]), tuple(p[3:6])])
for m in ("FE", "RE"):
h, l, u = r[m]
out[m]["hr"].append(h); out[m]["lo"].append(l); out[m]["hi"].append(u)
out[m]["se"].append((np.log(u) - np.log(l)) / (2 * Z))
I2s.append(r["I2"])
rng_ = {m: {q: (min(v), max(v)) for q, v in d.items()} for m, d in out.items()}
rng_["I2"] = (min(I2s), max(I2s))
return rng_
def printed_se_range(rep):
"""SE implied by the printed pooled CI, with each printed bound varied by +/-0.005."""
_, l, u = rep
return se_from_ci(l + 0.005, u - 0.005), se_from_ci(l - 0.005, u + 0.005)
# A printed 2-decimal value v stands for [v - 0.005, v + 0.005]; it "fits" when that overlaps the recomputed range
inr = lambda x, ab: x + 0.005 >= ab[0] and x - 0.005 <= ab[1]
RR = {}
for k in T:
rows, rep, _ = T[k]
RR[k] = rr = rounding_range(rows)
print(f"{k[0]:12s} {k[1]:12s} printed {f3(rep)}; I2 under rounding {rr['I2'][0]:.1f}-{rr['I2'][1]:.1f}% "
f"(rule -> {'RE' if rr['I2'][0] >= 25 else ('FE' if rr['I2'][1] < 25 else 'FE or RE')})")
for m in ("FE", "RE"):
d = rr[m]
flags = [("HR", rep[0], d["hr"]), ("lower", rep[1], d["lo"]), ("upper", rep[2], d["hi"])]
fit = ", ".join(f"{nm} {'in' if inr(v, ab) else 'OUT'}" for nm, v, ab in flags)
print(f"{'':25s} {m}: HR {d['hr'][0]:.3f}-{d['hr'][1]:.3f}; lower {d['lo'][0]:.3f}-{d['lo'][1]:.3f}; "
f"upper {d['hi'][0]:.3f}-{d['hi'][1]:.3f}; pooled SE {d['se'][0]:.3f}-{d['se'][1]:.3f} "
f"-> printed {fit}")
a, b = printed_se_range(rep)
lo_c = min(h for h, _, _ in rows) - 0.005; hi_c = max(h for h, _, _ in rows) + 0.005
print(f"{'':25s} SE implied by printed CI {se_from_ci(rep[1], rep[2]):.3f} ({a:.3f}-{b:.3f} under rounding); "
f"cohort-HR range allowing rounding [{lo_c:.3f}, {hi_c:.3f}] -> printed HR {'inside' if lo_c <= rep[0] <= hi_c else 'OUTSIDE'}")
print("Printed pooled row vs its own recomputed pool (rule-chosen model and FE): max |log difference| over HR, lo, hi")
for k in T:
rep = T[k][1]
dc = max(abs(np.log(x) - np.log(y)) for x, y in zip(R[k]["chosen"], rep))
dfe = max(abs(np.log(x) - np.log(y)) for x, y in zip(R[k]["FE"], rep))
print(f" {k[0]:12s} {k[1]:12s} chosen ({R[k]['model']}) {f3(R[k]['chosen'])}: {dc:.3f}; FE {f3(R[k]['FE'])}: {dfe:.3f}; "
f"HR only {abs(np.log(R[k]['chosen'][0] / rep[0])):.3f}")
# Log symmetry of the cohort CIs for Myocarditis TET2 (a Wald CI is symmetric on the log scale)
for name, (h, l, u) in zip(("All of Us", "BioVU"), T[("Myocarditis", "TET2 CHIP")][0]):
print(f"Myocarditis TET2 {name}: ln(HR)-ln(lo) = {np.log(h)-np.log(l):.3f}, ln(hi)-ln(HR) = {np.log(u)-np.log(h):.3f}")
# Smallest pooled HR any positive weights can give: the smaller cohort HR minus rounding
print("Smallest cohort HR for Large CHIP pericarditis, lowest value allowed by rounding: 1.705 > printed 1.66")
print("Largest cohort HR for DNMT3A pericarditis, highest value allowed by rounding: 1.705 < printed 1.88")
s_bio_min = se_from_ci(1.235, 2.375)
print(f"Smallest possible BioVU Large-CHIP SE after rounding {s_bio_min:.3f}; SE implied by printed pooled CI "
f"(rounded, smallest) {se_from_ci(1.175, 2.345):.3f} -> a pooled SE above the smallest cohort SE is impossible with tau2 = 0")
print()
print("=" * 100)
print("4. Exchange check: recomputed pool of one exposure vs printed row of the other")
print("=" * 100)
for outc in ("Pericarditis", "Myocarditis"):
L = R[(outc, "Large CHIP")]; D = R[(outc, "DNMT3A CHIP")]
pL = T[(outc, "Large CHIP")][1]; pD = T[(outc, "DNMT3A CHIP")][1]
print(f"{outc}: recomputed Large CHIP FE {f3(L['FE'])} P={L['pFE']:.2g} vs printed DNMT3A row {f3(pD)} P{T[(outc,'DNMT3A CHIP')][2]}")
print(f"{'':13s} recomputed DNMT3A FE {f3(D['FE'])} P={D['pFE']:.2g} vs printed Large CHIP row {f3(pL)} P{T[(outc,'Large CHIP')][2]}")
for name, rec, prn in (("LargeFE->printed DNMT3A", L["FE"], pD), ("DNMT3A FE->printed Large", D["FE"], pL)):
d = [abs(np.log(x) - np.log(y)) for x, y in zip(rec, prn)]
print(f"{'':13s} {name}: max |log difference| over HR, lo, hi = {max(d):.3f}")
print(f"{'':13s} unswapped: |log(rec Large / printed Large)| = {abs(np.log(L['FE'][0]/pL[0])):.3f}; "
f"|log(rec DNMT3A / printed DNMT3A)| = {abs(np.log(D['FE'][0]/pD[0])):.3f}")
for name, rr, prn in (("rec Large FE vs printed DNMT3A", RR[(outc, "Large CHIP")]["FE"], pD),
("rec DNMT3A FE vs printed Large", RR[(outc, "DNMT3A CHIP")]["FE"], pL)):
fit = ", ".join(f"{nm} {'in' if inr(v, rr[q]) else 'OUT'}"
for nm, q, v in (("HR", "hr", prn[0]), ("lower", "lo", prn[1]), ("upper", "hi", prn[2])))
print(f"{'':13s} {name}, allowing rounding: {fit}")
L = R[("Myocarditis", "Large CHIP")]
print(f"Myocarditis Large CHIP: I2 = {L['I2']:.1f}% -> the stated rule calls for random effects: RE {f3(L['RE'])} P={L['pRE']:.2f}")
print("Myocarditis recomputed I2 by exposure (text says I2 = 37.1%-68.2%):",
", ".join(f"{k[1]} {R[k]['I2']:.1f}%" for k in T if k[0] == "Myocarditis"))
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def forest(outcome, fname, xlim, ticks, title, notes):
from matplotlib.lines import Line2D
from matplotlib.patches import Patch
groups = ["Large CHIP", "DNMT3A CHIP"]
fig, ax = plt.subplots(figsize=(7.2, 5.6), facecolor=BG)
ax.set_facecolor(BG)
vt = ax.get_yaxis_transform() # x in axes coords, y in data coords
y = 0.0
ylab, ypos = [], []
ax.text(1.03, 0.75, "HR (95% CI)", transform=vt, ha="left", va="center", fontsize=10, color=INK2)
for gi, g in enumerate(groups):
rows, rep, _ = T[(outcome, g)]
rec = R[(outcome, g)]["FE"]
ax.text(-0.62, y, g + ", " + outcome.lower(), transform=vt, ha="left", va="center",
fontsize=11, fontweight="bold", color=INK)
y -= 0.9
top = y + 0.4
for name, (h, l, u) in zip(("All of Us cohort", "BioVU cohort"), rows):
ax.plot([l, u], [y, y], color=INK2, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(h, y, "D", color=INK2, ms=8, zorder=3, mec=BG, mew=1.5)
ylab.append(name); ypos.append(y)
ax.text(1.03, y, f3((h, l, u)), transform=vt, ha="left", va="center", fontsize=10, color=INK2)
y -= 0.8
hs = sorted(h for h, _, _ in rows)
y_print, y_rec = y, y - 1.25
ax.fill_between([hs[0], hs[1]], y_rec - 0.4, top, color="#e6e5e1", zorder=0, lw=0)
ax.plot([rep[1], rep[2]], [y_print, y_print], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(rep[0], y_print, "s", color=ORANGE, ms=9, zorder=3, mec=BG, mew=1.5)
ylab.append("Pooled, as printed"); ypos.append(y_print)
ax.text(1.03, y_print, f3(rep), transform=vt, ha="left", va="center", fontsize=10, color=INK)
if notes[gi]:
ax.text(xlim[0] * 1.03, y_print - 0.6, notes[gi], ha="left", va="center", fontsize=9.5,
color=INK2, style="italic")
ax.plot([rec[1], rec[2]], [y_rec, y_rec], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(rec[0], y_rec, "o", color=BLUE, ms=9, zorder=3, mec=BG, mew=1.5)
ylab.append("Pooled, recomputed"); ypos.append(y_rec)
ax.text(1.03, y_rec, f3(rec), transform=vt, ha="left", va="center", fontsize=10, color=INK)
y = y_rec - 1.3
ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(*xlim)
ax.set_xticks(ticks)
ax.set_xticklabels([("%g" % t) for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ypos)
ax.set_yticklabels(ylab)
ax.set_ylim(y + 0.75, 0.45)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for sname in ("top", "right", "left"):
ax.spines[sname].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Hazard ratio (log scale); vertical line at HR 1 = no association")
h = [Line2D([], [], color=INK2, marker="D", lw=2, ms=8, label="Cohort estimate (Table 2)"),
Line2D([], [], color=ORANGE, marker="s", lw=2, ms=9, label="Pooled, as printed (Table 2)"),
Line2D([], [], color=BLUE, marker="o", lw=2, ms=9, label="Pooled, recomputed (fixed effect)"),
Patch(color="#e6e5e1", label="Between the 2 cohort HRs")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.35, -0.12), ncol=2, frameon=False, fontsize=10)
fig.suptitle(title, x=0.02, y=0.985, ha="left", fontsize=11.5, color=INK)
fig.subplots_adjust(left=0.30, right=0.76, top=0.89, bottom=0.2)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{fname}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
forest("Pericarditis", "fig1", (0.8, 3.6), [0.8, 1, 1.5, 2, 3],
"Pericarditis: each printed pooled HR lies outside its own cohort range",
["below both cohort HRs; matches the recomputed DNMT3A pool",
"above both cohort HRs; near the recomputed Large CHIP pool"])
forest("Myocarditis", "fig2", (0.15, 6.0), [0.2, 0.5, 1, 2, 5],
"Myocarditis: each printed pooled row is close to the other exposure's recomputed pool",
["near the recomputed DNMT3A pool, " + f3(R[("Myocarditis", "DNMT3A CHIP")]["FE"]),
"matches the recomputed Large CHIP pool, " + f3(R[("Myocarditis", "Large CHIP")]["FE"])])
print("\nFigures written: fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract says gender-stereotype thumbnails increased with engagement; Results, Table 2 and Discussion report a decrease
JAMA Netw Open (2024) · doi:10.1001/jamanetworkopen.2024.13855 · PMC11137630
Summary
The abstract reports that, over 20 sequential engagements with recommended videos, "the gender stereotypes feature increased with more engagement in the recommendations feed (P for trend < .001)". The body reports the opposite direction for the same feature and the same test. The Results say gender stereotypes "decreased (P for trend < .0001)", the Discussion says prevalence "decreased for gender stereotypes", and Table 2 gives a negative Spearman correlation with screenshot number (ρ = −0.08), which with the paper's 2880 thumbnails corresponds to t = −4.31 and P = 1.7 × 10⁻⁵. The feature that did increase was visual loudness, which the Results report with the same P for trend (< .001) that the abstract gives. The trend over screenshots is how the study tests its stated hypothesis (that recommendations drift toward more problematic content), so a reader of the abstract alone gets the direction of that result reversed.
What the paper reports
Abstract, Results paragraph (final sentence):
Contrary to the hypotheses, problematic feature prevalence did not increase over time, but the gender stereotypes feature increased with more engagement in the recommendations feed (P for trend < .001).
Results, section "Trends Over 20 Screenshots":
Both Spearman correlations and tests of trend demonstrated that visual loudness increased over the 20 screenshots (P for trend < .001), while gender stereotypes decreased (P for trend < .0001) (Figure).
Discussion, first paragraph:
Contrary to our hypothesis, the prevalence of most codes did not increase over time when following recommendations over 20 sequential engagements; prevalence only increased slightly for visual loudness and decreased for gender stereotypes.
Table 2 ("Spearman Correlations Between Content Codes Among 2880 Thumbnails"), column "Screenshot No. (1-20)":
Visual loudness | .09 (<.001)
Gender stereotypes | -0.08 (<.001)
Methods, "Statistical Analysis", which defines both trend analyses:
To examine whether the frequency of content features changed over the course of 20 sequential screenshots, we conducted Spearman correlations between screenshot number (1 through 20) and each feature. We also dichotomized each feature as present (codes of levels 1 or 2) vs not present and tested significance of trend over 20 screenshots with the Cochran-Armitage trend test.
Introduction, the hypothesis the trend analysis tests:
We hypothesized that the prevalence of problematic thumbnail features would increase over time with sequential engagement with recommended videos (ie, approximating going down the rabbit hole).
The coding levels are defined in the Methods as "0 = absent, 1 = mild or latent, and 2 = clear or dominant", so a higher code means the feature is more present, and a positive correlation with screenshot number means more of the feature at later screenshots.
What the paper's own numbers show
Method. The paper does not publish screenshot-level counts (the Data Sharing Statement says "Data available: No"), so the Cochran-Armitage statistic itself cannot be recomputed. The direction can be checked in three ways from what is published. (1) I compared the direction stated in each place the paper reports the trend. (2) I converted the Table 2 correlation into a two-sided P value with the usual t approximation, t = ρ√((n − 2)/(1 − ρ²)), at n = 2880 (the table's stated denominator) and at n = 2574 (the number of unique thumbnails), and computed a Fisher-z 95% interval. Because ρ is printed to two decimals and Spearman ties are ignored, these values are approximate, but the sign is not affected. (3) As a supporting check only, I read by eye the level-0 (absent) bar heights in the paper's Figure, panels E (gender stereotypes) and A (visual loudness), converted them to counts of "present" thumbnails out of 144 per screenshot (12 search terms × 12 thumbnails), and ran a Cochran-Armitage test. Those readings are approximate (about ±1–2 percentage points per bar). I also checked the prevalence counts against Supplement 1, eTable 1. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Direction of gender stereotypes trend | Abstract, Results | "increased" (P for trend < .001) | decrease (opposite sign) |
| Direction of gender stereotypes trend | Results, Trends Over 20 Screenshots | "decreased" (P for trend < .0001) | decrease (agrees) |
| Direction of gender stereotypes trend | Discussion, first paragraph | "decreased" | decrease (agrees) |
| ρ, screenshot no. vs gender stereotypes | Table 2 | −0.08 (<.001) | t = −4.31; P = 1.7 × 10⁻⁵ (n = 2880), 4.8 × 10⁻⁵ (n = 2574); 95% CI −0.116 to −0.044 |
| ρ, screenshot no. vs visual loudness | Table 2 | .09 (<.001) | t = +4.85; P = 1.3 × 10⁻⁶; 95% CI +0.054 to +0.126 (an increase) |
| P for trend in the abstract's sentence | Abstract vs Results | < .001 | matches the Results' P for visual loudness (< .001), not gender stereotypes (< .0001) |
| Other four features, ρ with screenshot no. | Table 2 | −0.01 to 0.02, P = .29 to .73 | all 95% CIs include 0 (no trend) |
| Gender stereotypes, prevalence | Abstract, Table 1 | 525 (18.2%); 387 at level 1, 138 at level 2 | 387 + 138 = 525; 525/2880 = 18.2%; eTable 1 per-term counts also sum to 387 and 138 |
| Gender stereotypes present, screenshots 1–5 vs 16–20 | Figure, panel E (read by eye) | plotted only | about 25.0% vs 17.0%; Cochran-Armitage z = −4.27, P ≈ 2 × 10⁻⁵ (a decrease) |
| Visual loudness present, screenshots 1–5 vs 16–20 | Figure, panel A (read by eye) | plotted only | about 70.4% vs 81.4%; z = +5.94 (an increase; checks the reading method) |
All six prevalence totals in the abstract equal the Table 1 level 1 + level 2 sums, so the abstract's prevalence figures are sound; the problem is confined to the direction of the gender stereotypes trend.
A separate, minor inconsistency: the Results give drama and intrigue for the search term Roblox as "218 of 240 thumbnails [96.7%]". 218/240 is 90.8%. Supplement 1, eTable 1 lists Roblox drama and intrigue as 8 at level 0, 115 at level 1 and 117 at level 2, so 232 of 240 thumbnails (96.7%) had the feature. The percentage is right and the count in the text is wrong. It does not affect any conclusion.
Figure
Figure 1. The conflicting statements about the same feature and the same trend test. Left (orange outline): the abstract's sentence, which says gender stereotypes increased (P for trend < .001). Right (grey outline): the Results, Table 2 and Discussion, which all say or show a decrease. The double arrow marks the conflict; the note under it records that the abstract's P value is the one the Results give for visual loudness. Bottom (blue outline): the two-sided P implied by the Table 2 correlation, recomputed by recompute.py.txt; the negative sign means fewer gender-stereotype thumbnails at later screenshots. All quoted text is verbatim from the article as archived in PMC.
Figure 2. The six correlations with screenshot number (1–20) printed in Table 2 (orange dots), each with a 95% interval recomputed by recompute.py.txt (blue lines; Fisher z, n = 2880). The vertical line marks no trend (ρ = 0). Only two features have intervals that exclude 0: visual loudness, above 0 (more at later screenshots), and gender stereotypes, below 0 (fewer at later screenshots). The abstract describes the gender stereotypes trend as an increase. Intervals use the two-decimal ρ printed in the table, so they are approximate.
Why it matters
- It is the abstract's only trend finding, and the trend is how the study tests its hypothesis. The Objective is "to test whether VSP algorithmic recommendations offer more problematic thumbnail features over time", and the Introduction frames this as "going down the rabbit hole". The abstract tells readers that the one problematic feature the recommendations amplified was gender stereotypes (sexual objectification). The body shows that this feature became less common across the 20 engagements, and that no problematic feature increased.
- Abstracts are read and cited on their own. Readers, indexers and secondary sources that rely on the abstract will take away that recommendation feeds increase sexualized or stereotyped thumbnails over sequential viewing, a policy-relevant claim the paper's data do not support. The increase that did occur (visual loudness, an attention-capture feature rather than a problematic one) is not mentioned in the abstract.
- The main conclusions are not affected. The title, Key Points and Conclusions rest on how common problematic and attention-capturing thumbnails were, and those prevalence figures check out against Table 1 and eTable 1. With the direction corrected, the abstract's own "Contrary to the hypotheses" framing becomes stronger: no problematic feature increased, and one decreased.
Explanations considered
- A conversion or typesetting artifact. Not supported. The publisher's page, the PMC version, the Europe PMC full-text XML and the PubMed abstract all carry the same wording ("increased") in the abstract and the same "decreased" in the Results. Table 2 shows a true minus sign on the publisher's page.
- A reversed coding scale for this feature. Not supported. The levels are defined once for all features (0 absent, 2 clear or dominant), and Table 1 counts level 1 and level 2 as prevalence. The same sign convention gives visual loudness a positive ρ that the Results call an increase. The level-0 (absent) share in Figure panel E rises at later screenshots, which is a fall in presence.
- "More engagement" meaning something other than the screenshot sequence, such as co-occurrence with attention-capture features. Not supported. Gender stereotypes do co-occur with visual loudness (ρ = 0.08) and drama and intrigue (ρ = 0.12) in Table 2, but the abstract's sentence gives a "P for trend", which the Main Outcomes define as the Cochran-Armitage trend over 20 engagements, and it speaks of engagement "in the recommendations feed". The sentence also opens by contrasting with prevalence "over time".
- A small, unstable effect whose sign is uncertain. Not supported. The effect is small (ρ = −0.08; roughly 25% to 17% present from the first to the last five screenshots, by the approximate reading of panel E), but its interval excludes 0 (−0.116 to −0.044), and the Results report P < .0001. The sign does not depend on rounding or on whether n is 2880 or 2574.
- The abstract describing visual loudness under the wrong feature name. The numbers are consistent with this: visual loudness is the feature that increased, and its P for trend in the Results (< .001) is the P the abstract gives, whereas the Results give gender stereotypes P < .0001. This would account for both the direction and the P value, but the paper itself cannot confirm how the sentence came about.
Status
As of 2026-10-24:
- Version of record. The wording is the same in the publisher's version (doi.org resolves to jamanetwork.com, article 2819134), the PMC version, the Europe PMC full-text XML and the PubMed record (PMID 38809550). Quotes above were checked verbatim against the PMC text.
- Supplements. Supplement 1 (eTable 1, codes by search term; eTable 2, descriptions and examples) was checked. Its gender stereotypes counts sum to Table 1 (387 at level 1, 138 at level 2), and it gives the Roblox drama and intrigue count (232 of 240). It has no counts by screenshot, so it neither confirms nor resolves the trend direction. Supplement 2 is the Data Sharing Statement ("Data available: No").
- Corrections. None found. Crossref has no update-to or updated-by records and no relations to this DOI. PMC and Europe PMC list no correction or retraction notices, the PubMed record has no CommentsCorrections entries, and the publisher's page shows no correction notice. A web search found no erratum. The Europe PMC search API was unavailable when queried, so its comment/correction list could not be checked through that route.
- PubPeer. Could not be checked: the site was behind a human-verification page.
Sources: the article in PMC (PMC11137630) and on the publisher's site; Europe PMC full text and supplementary files; PubMed; Crossref REST API.
Suggested correction
In the Abstract, Results paragraph, replace
Contrary to the hypotheses, problematic feature prevalence did not increase over time, but the gender stereotypes feature increased with more engagement in the recommendations feed (P for trend < .001).
with wording consistent with the Results, Table 2 and the Discussion, for example:
Contrary to the hypotheses, problematic feature prevalence did not increase over time; the gender stereotypes feature decreased with more engagement in the recommendations feed (P for trend < .0001), and the visual loudness feature increased (P for trend < .001).
In the Results, "Attention-Capture Designs", the Roblox count for drama and intrigue should read 232 of 240 thumbnails (96.7%), as in eTable 1, rather than 218 of 240.
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up of PMC11137630
(JAMA Netw Open 2024, doi:10.1001/jamanetworkopen.2024.13855).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are the paper's own numbers: Table 1 (counts by level, N = 2880), Table 2
(Spearman correlations with screenshot number), the per-search-term counts in the
Results text, and the Methods design (12 search terms x 20 screenshots x 12
thumbnails). One supporting check uses the level-0 bar heights of the paper's Figure,
panels A and E, read by eye from the archived image (jamanetwopen-e2413855-g001.jpg);
those readings are approximate (about +/-1-2 percentage points per bar) and are not
drawn in any figure here.
"""
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
# ------------------------------------------------------------------ design (Methods)
TERMS, SHOTS, THUMBS = 12, 20, 12
N = TERMS * SHOTS * THUMBS # 2880 thumbnails
N_UNIQUE = 2574 # "2574 unique video thumbnails were coded"
PER_SHOT = TERMS * THUMBS # 144 thumbnails at each screenshot position
PER_TERM = SHOTS * THUMBS # 240 thumbnails per search term
# ------------------------------------------------------------------ Table 1 (level 1, level 2) and abstract totals
TABLE1 = { # feature: (level 1, level 2, abstract total, abstract %)
"Visual loudness": (1352, 926, 2278, 79.1),
"Drama and intrigue": (1606, 1030, 2636, 91.5),
"Lavish excess and wish fulfillment": (860, 426, 1286, 44.7),
"Creepy, bizarre, and disturbing": (842, 441, 1283, 44.6),
"Violence, peril, and pranks": (891, 279, 1170, 40.6),
"Gender stereotypes": (387, 138, 525, 18.2),
}
# ------------------------------------------------------------------ Table 2, "Screenshot No. (1-20)" column
TABLE2 = { # feature: (rho, reported P)
"Visual loudness": (0.09, "<.001"),
"Drama and intrigue": (-0.01, ".54"),
"Creepy, bizarre, and disturbing": (0.01, ".73"),
"Violence, peril, and pranks": (0.02, ".38"),
"Gender stereotypes": (-0.08, "<.001"),
"Lavish excess and wish fulfillment": (0.02, ".29"),
}
# Results, "Trends Over 20 Screenshots": Cochran-Armitage P for trend
RESULTS_TREND = {"Visual loudness": ("increased", "< .001"),
"Gender stereotypes": ("decreased", "< .0001")}
# ------------------------------------------------------------------ per-search-term counts quoted in Results (n of 240, %)
TERM_COUNTS = [
("Visual loudness, SML", 230, 95.8), ("Visual loudness, FGTeeV", 214, 89.2),
("Drama and intrigue, Roblox", 218, 96.7), ("Drama and intrigue, Minecraft", 228, 95.0),
("Drama and intrigue, FGTeeV", 228, 95.0), ("Lavish excess, MrBeast", 146, 60.8),
("Lavish excess, SML", 139, 57.9), ("Lavish excess, Roblox", 138, 57.5),
("Creepy/bizarre, Flamingo", 190, 79.2), ("Creepy/bizarre, memes", 155, 64.6),
("Violence/peril, Flamingo", 145, 60.4), ("Violence/peril, DanTDM", 115, 47.9),
("Gender stereotypes, Roblox", 101, 42.1), ("Gender stereotypes, Flamingo", 64, 26.7),
]
# ------------------------------------------------------------------ Supplement 1, eTable 1 (counts of levels 0, 1, 2 per term, n = 240)
ET1_TERMS = ["DanTDM", "FGTeev", "Flamingo", "Fortnite", "Memes", "Minecraft", "MrBeast",
"Pewdiepie", "Roblox", "Sml", "Try not to laugh", "Unspeakable"]
ET1_GENDER = [(202, 27, 11), (195, 33, 12), (176, 55, 9), (186, 39, 15), (205, 23, 12), (216, 17, 7),
(221, 15, 4), (206, 28, 6), (139, 63, 38), (187, 40, 13), (196, 33, 11), (226, 14, 0)]
ET1_DRAMA_ROBLOX = (8, 115, 117)
# ------------------------------------------------------------------ Figure level-0 (absent) % per screenshot, read by eye
LVL0_E = np.array([73, 71, 78, 74, 79, 75, 84, 84, 87, 87, 87, 87, 88, 81, 88, 87, 90, 78, 75, 85], float)
LVL0_A = np.array([28, 32, 33, 23, 32, 26, 25, 16, 23, 18, 7, 12, 15, 19, 13, 17, 26, 15, 15, 20], float)
def p_from_rho(r, n):
"""Two-sided P for a correlation r with n pairs (t approximation, df = n - 2)."""
t = r * np.sqrt((n - 2) / (1 - r * r))
return t, 2 * stats.t.sf(abs(t), n - 2)
def fisher_ci(r, n, level=0.95):
z, se = np.arctanh(r), 1 / np.sqrt(n - 3)
q = stats.norm.ppf(0.5 + level / 2)
return np.tanh(z - q * se), np.tanh(z + q * se)
def cochran_armitage(present, n, x):
tot = n * len(x)
p = present.sum() / tot
t = np.sum(x * (present - n * p))
var = p * (1 - p) * (np.sum(n * x ** 2) - np.sum(n * x) ** 2 / tot)
z = t / np.sqrt(var)
return z, 2 * stats.norm.sf(abs(z))
def main():
print(f"Design: {TERMS} terms x {SHOTS} screenshots x {THUMBS} thumbnails = {N}; "
f"{PER_SHOT} per screenshot position; {PER_TERM} per term")
print("\n== Table 1 level 1 + level 2 vs abstract totals ==")
for k, (l1, l2, tot, pct) in TABLE1.items():
s = l1 + l2
print(f" {k:36s} {l1}+{l2} = {s} (abstract {tot}, {'ok' if s == tot else 'MISMATCH'}); "
f"{100 * s / N:.2f}% (abstract {pct})")
print("\n== Table 2, screenshot number column: P implied by rho, 95% CI (Fisher z) ==")
rows = {}
for k, (r, prep) in TABLE2.items():
t, p = p_from_rho(r, N)
tu, pu = p_from_rho(r, N_UNIQUE)
lo, hi = fisher_ci(r, N)
rows[k] = (r, lo, hi)
print(f" {k:36s} rho={r:+.2f} t={t:+.2f} P(n={N})={p:.2g} P(n={N_UNIQUE})={pu:.2g} "
f"95% CI {lo:+.3f} to {hi:+.3f} reported P {prep}")
gs_t, gs_p = p_from_rho(-0.08, N)
print(f" -> gender stereotypes: sign negative (decrease); t = {gs_t:.2f}, P = {gs_p:.1e}; "
f"P < .0001: {gs_p < 1e-4}")
print(f" -> smallest |rho| giving P < .001 at n = {N}: "
f"{stats.t.ppf(1 - 0.0005, N - 2) / np.sqrt(N - 2 + stats.t.ppf(1 - 0.0005, N - 2) ** 2):.3f}")
print("\n== Direction statements for gender stereotypes ==")
print(" Abstract: increased (P for trend < .001)")
print(f" Results: {RESULTS_TREND['Gender stereotypes'][0]} (P for trend {RESULTS_TREND['Gender stereotypes'][1]})")
print(" Table 2: rho = -0.08 (decrease)")
print(" Discussion: decreased")
print(f" Results, visual loudness: {RESULTS_TREND['Visual loudness'][0]} "
f"(P for trend {RESULTS_TREND['Visual loudness'][1]}) -- same P as the abstract's sentence")
print("\n== Per-search-term percentages quoted in Results (n / 240) ==")
for lab, n, rep in TERM_COUNTS:
pct = 100 * n / PER_TERM
flag = " <-- mismatch" if abs(pct - rep) > 0.06 else ""
print(f" {lab:32s} {n}/240 = {pct:5.1f}% (reported {rep}){flag}")
print(f" count that would give 96.7%: {0.967 * PER_TERM:.1f} -> 232/240 = {100 * 232 / 240:.1f}%")
print("\n== Supplement 1, eTable 1 ==")
g = np.array(ET1_GENDER)
print(f" gender stereotypes: rows sum to 240 for every term: {bool((g.sum(1) == 240).all())}; "
f"level 1 total {g[:, 1].sum()} (Table 1: 387), level 2 total {g[:, 2].sum()} (Table 1: 138), "
f"present {g[:, 1:].sum()} = {100 * g[:, 1:].sum() / N:.1f}%")
for t in ("Roblox", "Flamingo"):
i = ET1_TERMS.index(t)
print(f" gender stereotypes, {t}: {g[i, 1]} + {g[i, 2]} = {g[i, 1:].sum()} of 240")
d0, d1, d2 = ET1_DRAMA_ROBLOX
print(f" drama and intrigue, Roblox: {d0}+{d1}+{d2} = {d0 + d1 + d2}; present {d1 + d2} of 240 = "
f"{100 * (d1 + d2) / 240:.1f}% (Results text: '218 of 240 thumbnails [96.7%]')")
print("\n== Supporting check: Figure panels E and A (level-0 bars read by eye, approximate) ==")
shots = np.arange(1, SHOTS + 1)
for lab, l0 in (("Gender stereotypes (panel E)", LVL0_E), ("Visual loudness (panel A)", LVL0_A)):
present_pct = 100 - l0
present = np.round(present_pct / 100 * PER_SHOT)
z, p = cochran_armitage(present, PER_SHOT, shots)
slope = np.polyfit(shots, present_pct, 1)[0]
x = np.repeat(shots, PER_SHOT)
y = np.concatenate([np.r_[np.ones(int(k)), np.zeros(PER_SHOT - int(k))] for k in present])
rs, ps = stats.spearmanr(x, y)
print(f" {lab}: present {present_pct[:5].mean():.1f}% at screenshots 1-5 vs "
f"{present_pct[15:].mean():.1f}% at 16-20; overall {present_pct.mean():.1f}% "
f"(implied {present.sum():.0f} of {N}); OLS slope {slope:+.2f} points/screenshot; "
f"Cochran-Armitage z = {z:+.2f}, P = {p:.1g}; Spearman(present) = {rs:+.3f}, P = {ps:.1g}")
fig1(gs_t, gs_p)
fig2(rows)
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
# ------------------------------------------------------------------ figures
def _box(ax, x, y, w, h, edge, lw=2.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=edge, lw=lw))
def _wrap(s, width):
return "\n".join(textwrap.wrap(s, width))
def _sci(p):
m, e = f"{p:.1e}".split("e")
sup = str.maketrans("-0123456789", "\u207b\u2070\u00b9\u00b2\u00b3\u2074\u2075\u2076\u2077\u2078\u2079")
return f"{m} \u00d7 10" + str(int(e)).translate(sup)
def fig1(gs_t, gs_p):
W, H = 7.2, 5.4 # axes in inches, so text heights are predictable
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.set_axis_off()
line = 10.5 * 1.4 / 72 # one wrapped line at 10.5 pt, spacing 1.4
top, bot = 5.25, 1.85
bw = 2.85
lx, rx = 0.15, W - 0.15 - bw
# left: abstract
_box(ax, lx, bot, bw, top - bot, ORANGE)
ax.text(lx + 0.15, top - 0.15, "Abstract, Results paragraph", fontsize=11, weight="bold", va="top")
ax.text(lx + 0.15, top - 0.45, "as reported in the paper", fontsize=10, va="top", color=ORANGE)
ax.text(lx + 0.15, top - 0.85,
_wrap("\u201c\u2026but the gender stereotypes feature increased with more engagement "
"in the recommendations feed (P for trend < .001).\u201d", 33),
fontsize=10.5, va="top", linespacing=1.4)
ax.text(lx + 0.15, bot + 0.18, "Direction: INCREASE", fontsize=11, weight="bold", color=ORANGE)
# right: body
_box(ax, rx, bot, bw, top - bot, INK2)
ax.text(rx + 0.15, top - 0.15, "Results, Table 2, Discussion", fontsize=11, weight="bold", va="top")
ax.text(rx + 0.15, top - 0.45, "as reported in the paper", fontsize=10, va="top", color=INK2)
body = [
"Results: \u201c\u2026visual loudness increased \u2026 (P for trend < .001), "
"while gender stereotypes decreased (P for trend < .0001)\u201d",
"Table 2: screenshot no. vs gender stereotypes, \u22120.08 (<.001)",
"Discussion: \u201c\u2026decreased for gender stereotypes\u201d",
]
yy = top - 0.85
for txt in body:
t = _wrap(txt, 35)
ax.text(rx + 0.15, yy, t, fontsize=10, va="top", linespacing=1.4)
yy -= line * 1.08 * (t.count("\n") + 1) + 0.16
ax.text(rx + 0.15, bot + 0.18, "Direction: DECREASE", fontsize=11, weight="bold")
# connector
cy = 3.75
ax.add_patch(FancyArrowPatch((lx + bw + 0.04, cy), (rx - 0.04, cy), arrowstyle="<|-|>",
mutation_scale=14, lw=2, color=INK))
ax.text(W / 2, cy + 0.12, "opposite\ndirection,\nsame feature", ha="center", va="bottom",
fontsize=10.5, weight="bold", linespacing=1.3)
ax.text(W / 2, cy - 0.14, "abstract\u2019s\nP < .001 is\nthe Results\u2019 P\nfor visual\nloudness",
ha="center", va="top", fontsize=10, color=INK2, linespacing=1.3)
# bottom: recomputation
_box(ax, lx, 0.15, W - 0.3, 1.5, BLUE)
ax.text(lx + 0.15, 1.5, "Recomputed from the paper\u2019s Table 2", fontsize=11, weight="bold",
va="top", color=BLUE)
ax.text(lx + 0.15, 1.15,
f"\u03c1 = \u22120.08 with n = {N} thumbnails gives t = {gs_t:.2f}".replace("-", "\u2212")
+ f", two-sided P = {_sci(gs_p)}.\n"
"The negative sign means fewer gender-stereotype thumbnails at later\n"
"screenshots: a decrease, as the body says, and the opposite of the abstract.",
fontsize=10.5, va="top", linespacing=1.4)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200)
plt.close(fig)
def fig2(rows):
order = sorted(rows, key=lambda k: rows[k][0])
fig, ax = plt.subplots(figsize=(7.2, 3.9))
fig.subplots_adjust(left=0.36, right=0.97, top=0.83, bottom=0.17)
ys = np.arange(len(order))
for y, k in zip(ys, order):
r, lo, hi = rows[k]
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(r, y, "o", ms=9, color=ORANGE, mec=ORANGE, zorder=3)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.set_yticks(ys)
ax.set_yticklabels([k.replace(" and wish fulfillment", "") for k in order], fontsize=10.5, color=INK)
ax.set_xlim(-0.15, 0.15)
ax.set_ylim(-0.6, len(order) - 0.4)
ax.set_xlabel("Spearman ρ with screenshot number (1–20), Table 2")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
gy = order.index("Gender stereotypes")
r_gs, _, hi_gs = rows["Gender stereotypes"]
ax.annotate("abstract: “increased”", xy=(hi_gs + 0.004, gy), xytext=(0.03, gy),
va="center", fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-|>", color=INK, lw=1.2))
ax.text(-0.148, len(order) - 0.15, "← fewer at later screenshots", fontsize=10, color=INK2,
va="bottom", ha="left", clip_on=False)
ax.text(0.148, len(order) - 0.15, "more at later screenshots →", fontsize=10, color=INK2,
va="bottom", ha="right", clip_on=False)
h1, = ax.plot([], [], "o", ms=9, color=ORANGE, label="ρ reported in Table 2")
h2, = ax.plot([], [], color=BLUE, lw=2, label=f"95% CI recomputed (Fisher z, n = {N})")
fig.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(0.02, 0.99), ncol=2,
frameon=False, fontsize=10)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200)
plt.close(fig)
if __name__ == "__main__":
main()
Key Points and abstract call the JAKi cancer risk significant though the paper's 95% CI includes 1
JAMA Netw Open (2024) · doi:10.1001/jamanetworkopen.2024.46336 · PMC11579790
Summary
The Key Points and the abstract's Conclusions and Relevance say that cancer risk was "statistically significantly" higher in people starting Janus kinase inhibitors (JAKis) than in people starting tumor necrosis factor inhibitors (TNFis). The paper's own estimate for that comparison is an adjusted hazard ratio (HR) of 1.36 with a 95% CI of 0.94-1.96. The interval includes 1. Working back from it gives a two-sided P of about 0.10, above the paper's stated threshold of 2-sided P < .05. The propensity-matched estimate (HR 1.79, 0.91-3.56; P about 0.09; shown as 1.8, 0.9-3.6 in eTable 10 of Supplement 1) and the crude rate ratio from Table 2 (1.21, 0.77-1.88) are not significant either, and the paper's Results, Discussion and main-text Conclusions all describe the JAKi result as not significant. The HR, its CI and its direction are reported correctly throughout. What is wrong is the significance label in the two most-read summaries, on the comparison of most safety interest (the JAKi cancer signal from the ORAL Surveillance trial).
What the paper reports
All passages are verbatim from the article as archived in PubMed Central.
The two summary statements:
"In this cohort study consisting of 25 305 individuals who initiated treatment with biologic or targeted synthetic disease-modifying antirheumatic drugs, a statistically significantly higher risk of incident cancer was seen with rituximab, abatacept, and JAKis compared with TNFis." (Key Points, Findings)
"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, individuals initiating rituximab, abatacept, and JAKis demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with those initiating TNFis in the first 2 years after initiation of biologic or targeted synthetic DMARDs." (Abstract, Conclusions and Relevance)
The estimate those statements rest on, which appears in the abstract Results, the main Results and Table 2:
"Multivariable Cox proportional hazards regression analysis showed that rituximab was associated with a higher risk of incident cancer compared with TNFis (hazard ratio [HR], 1.91; 95% CI, 1.17-3.14), followed by abatacept (HR, 1.47; 95% CI, 1.03-2.11), and JAKis (HR, 1.36; 95% CI, 0.94-1.96)." (Abstract, Results)
"JAKi | 2570 (9) | 276 (169-351) | 22 (0.9) | 94 (59-143) | 1.36 (0.94-1.96)" (Table 2, JAKi row: exposures, median days to diagnosis, crude cancer diagnoses, diagnoses per 10 000 person-years, adjusted HR)
The significance convention:
"A 2-sided P < .05 was considered statistically significant." (Methods, Statistical Analysis)
The body of the paper describes the same result as not significant:
"Also like the primary analysis, we found that those treated with JAKis did not have a statistically significant increase in risk of cancer (HR, 1.79; 95% CI, 0.91-3.56) (eTable 10 and eFigure 4 in Supplement 1)." (Results, propensity-matched analysis paragraph)
"We observed that, among patients with RA, rituximab, abatacept, and JAKi initiations were associated with increased risks of incident cancer after 90 days and within 2 years of initiation, compared with TNFi initiation, although only rituximab and abatacept initiations had statistically significant associations." (Discussion, first paragraph)
"Our findings of increased incidence rate and risk of cancer with JAKi exposure compared with TNFi exposure are consistent with the results of ORAL Surveillance,6 although the results were not statistically significant." (Discussion, paragraph on prior JAKi studies)
"In this cohort study of individuals with RA and new biologic or targeted synthetic DMARD exposures, we found that new rituximab and abatacept initiators demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with TNFi initiators in the first 2 years after biologic or targeted synthetic DMARD initiation." (Conclusions, main text)
What the paper's own numbers show
Method. Cox-model HRs come with Wald intervals on the log scale, so a reported HR and 95% CI fix the standard error: SE = [ln(upper) − ln(lower)] / (2 × 1.960). Then z = ln(HR) / SE and the two-sided P = 2 × [1 − Φ(|z|)]. This uses only the published numbers; the result is approximate because the bounds are rounded to two decimals. Before relying on it, I checked that the JAKi interval is symmetric about ln(1.36) on the log scale (0.369 below, 0.365 above), as a Wald interval should be. I applied the same calculation to every HR the paper reports, so the comparisons the paper correctly calls significant serve as a check. Separately, I recomputed the crude rates in Table 2 from their event counts. The person-years each rate implies give exact Poisson intervals that match the reported ones to within 1-2 units, so the inputs are internally consistent. From those I formed a crude JAKi/TNFi rate ratio and an exact conditional test. I did the same for the propensity-matched rates in eTable 10 of Supplement 1. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| JAKi vs TNFi adjusted HR (95% CI) | Abstract Results; Results; Table 2 | 1.36 (0.94-1.96) | CI includes 1; geometric midpoint of the bounds 1.36 (consistent) |
| JAKi SE(ln HR), z | derived from Table 2 | not reported | SE 0.187; z 1.64 (z ≥ 1.96 needed for P < .05) |
| JAKi two-sided P | derived from Table 2 | not reported; Key Points Findings and abstract Conclusions and Relevance say "statistically significantly" | 0.10 (not < .05) |
| Smallest HR that would be significant at this SE | derived | n/a | 1.44 |
| JAKi one-sided P (not the stated convention) | derived | Methods: "2-sided P < .05" | 0.050 |
| JAKi propensity-matched HR (95% CI) | Results | 1.79 (0.91-3.56), "did not have a statistically significant increase" | P 0.094 (not significant; agrees with the text) |
| JAKi propensity-matched HR (95% CI) | Supplement 1, eTable 10 | 1.8 (0.9-3.6); 22 vs 14 cancers | same as the Results value rounded to one decimal; P about 0.10 from the rounded bounds |
| Propensity-matched crude rates per 10 000 PY, JAKi vs TNFi | Supplement 1, eTable 10 | 100 (63-151), 22 events; 60 (33-101), 14 events | imply about 2200 and 2333 PY; exact Poisson CIs 63-151 and 33-101 (exact match); rate ratio 1.67 (0.85-3.26), exact conditional P 0.14 |
| JAKi crude rate per 10 000 PY | Table 2 | 94 (59-143), 22 events | implies about 2340 PY; exact Poisson CI 59-142 |
| TNFi crude rate per 10 000 PY | Table 2 | 78 (66-91), 162 events | implies about 20 769 PY; exact Poisson CI 66-91 |
| Crude rate ratio JAKi/TNFi | derived from Table 2 | n/a | 1.21 (0.77-1.88); exact conditional P 0.39 |
| Rituximab vs TNFi HR | Table 2 | 1.91 (1.17-3.14) | P 0.010 (significant; agrees with the text) |
| Abatacept vs TNFi HR | Table 2 | 1.47 (1.03-2.11) | P 0.035 (significant; agrees with the text) |
| IL-6i vs TNFi HR | Table 2 | 1.04 (0.57-1.92) | P 0.90 |
| Propensity-matched rituximab, abatacept, IL-6i HRs | Results | 4.37 (1.48-12.93); 3.12 (1.52-6.44); 5.65 (1.11-28.79), all called significant | P 0.008; 0.002; 0.037 (agree with the text) |
Every HR the paper calls significant has a back-calculated P below .05 and a CI that excludes 1. The JAKi primary estimate is the only one the Key Points and abstract call significant that does not meet either criterion. No JAKi analysis reported anywhere in the paper or in Supplement 1 does.
Figure
Figure 1. The left panel (Key Points and Abstract) quotes the Key Points Findings and the abstract's Conclusions and Relevance, which group JAKis with rituximab and abatacept as having a "statistically significantly" increased cancer risk. The right panel gives the estimate those statements rest on (Table 2 and Results) and the body's own description of it: the Discussion says only rituximab and abatacept had significant associations, and the main-text Conclusions name only those two drugs. The strip below gives the reported HR and CI (orange) and the z and two-sided P recomputed from that CI (blue). All quotes are verbatim from the PMC version; the recomputed values come from recompute.py.txt.
Figure 2. Every adjusted HR versus TNFi that the paper reports, primary (Table 2) and propensity-matched (Results), drawn as the reported point estimate and 95% CI (orange) on one log-scale axis. The thin vertical line marks HR = 1. The right-hand columns give the reported values and the two-sided P recomputed from each CI (blue). Both JAKi rows (shaded) cross 1, with P = 0.101 and 0.094; the label on the primary JAKi row marks the estimate that the Key Points and abstract call "statistically significantly" higher. Every row the paper calls significant has P < .05.
Why it matters
The JAKi versus TNFi comparison is the question that motivates the study: the Importance section opens with the increased cancer risk with tofacitinib versus TNFis in ORAL Surveillance. The Key Points Findings sentence is the paper's headline result and is what many readers, abstracting services and news summaries carry forward. As written, it tells a reader that this US claims cohort found a statistically significant JAKi cancer signal. It did not. The point estimate (HR 1.36) is compatible with anything from a 6% lower hazard to a 96% higher hazard, and the paper's other JAKi analyses agree. A reader who takes only the Key Points or the abstract would count this study as independent statistical support for the ORAL Surveillance finding, when the paper's own Discussion calls the JAKi result "not statistically significant".
The rest of the paper is not affected. The HR, its CI and the direction of the JAKi estimate are reported correctly, the rituximab and abatacept results are significant as stated, and the authors' interpretation (channeling bias as a likely explanation) does not depend on the JAKi label. The error is confined to two sentences (one in the Key Points, one in the abstract), which makes it easy to correct.
Explanations considered
- Another JAKi analysis is being described. Not supported. The propensity-matched HR, 1.79 (0.91-3.56), has P about 0.09, and the Results call it not significant. Supplement 1, eTable 10, gives the same matched result as 1.8 (0.9-3.6), from 22 vs 14 cancers; the matched crude rate ratio it implies, 1.67 (0.85-3.26), has exact P 0.14. The crude rate ratio implied by Table 2, 1.21 (0.77-1.88), has P 0.39. No JAKi estimate anywhere in the paper or Supplement 1 has a CI that excludes 1.
- A one-sided test or a looser alpha. Not supported. The Methods state "A 2-sided P < .05 was considered statistically significant." The back-calculated two-sided P is 0.10. Only a one-sided test brings it to 0.050, which is still not below .05.
- "Significant" refers to the higher incidence rates, not the HR. Not supported. The wording is "statistically significantly higher risk" and "statistically significantly increased risks". The JAKi crude rate, 94 (59-143), overlaps the TNFi rate, 78 (66-91), and the crude rate ratio is not significant either.
- Transcription or rounding error in the HR or CI. Not supported. HR 1.36 is the geometric midpoint of 0.94-1.96. The interval is symmetric on the log scale, and the same numbers appear in the abstract, the Results and Table 2. The crude rates match exact Poisson intervals from their event counts to within 1-2 units.
- A conversion artifact in the archived text. Ruled out. Both bounds are positive decimals, so no minus sign can have been lost, and nothing depends on superscripts or table alignment. The conflicting statements are in prose, and the same wording appears on the live PMC page, in the Europe PMC full-text XML and in the PubMed abstract.
- Rounding at the threshold. Not applicable. A lower bound of 0.94 is not a rounded 1.00, and z = 1.64 is far from 1.96. With this SE, an HR of at least 1.44 would have been needed for significance.
None of these accounts for the wording of the Key Points and abstract. Since the body of the paper states the result correctly, those two sentences are inconsistent with the paper itself.
Status
As of 2026-10-23:
- Version of record. The publisher's full-text page (jamanetwork.com, article 2826510) returned an automated-traffic check and could not be read directly. The wording quoted above was confirmed in three publisher-supplied copies: the live PMC page, the Europe PMC full-text JATS XML, and the PubMed abstract (PMID 39565623). The 0.94-1.96 interval appears three times in the XML (abstract, Results, Table 2).
- Supplement. Supplement 1 (eTables 1-11, eFigures 1-4) was downloaded through Europe PMC and opened. eTable 9 gives the matched TNFi vs JAKi cohort (2421 per arm; 14 vs 22 cancer outcomes). eTable 10 (page 14, extractable text) gives the propensity-matched JAKi vs TNFi result: 22 vs 14 cancers, 100 (63-151) vs 60 (33-101) per 10 000 person-years, adjusted HR 1.8 (0.9-3.6), consistent with the HR 1.79 (0.91-3.56) in the Results. The supplement contains no alternative JAKi estimate or P value that would support the wording of the Key Points and abstract.
- Corrections. Crossref, PMC and Europe PMC list no correction, erratum, expression of concern or retraction notice. A web search found no erratum.
- PubPeer. A web search found no PubPeer thread for this article. PubPeer's own search returned a human-verification challenge and could not be queried directly.
- Sources: PMC11579790; PubMed 39565623; doi:10.1001/jamanetworkopen.2024.46336; Europe PMC full-text and supplementary-files endpoints for PMC11579790; Crossref metadata for the DOI.
Suggested correction
Bring the Key Points Findings sentence and the abstract's Conclusions and Relevance sentence into line with the Results, Discussion and main-text Conclusions. For example:
- Key Points, Findings: "…a statistically significantly higher risk of incident cancer was seen with rituximab and abatacept compared with TNFis; the higher risk seen with JAKis was not statistically significant (HR, 1.36; 95% CI, 0.94-1.96)."
- Abstract, Conclusions and Relevance: "…individuals initiating rituximab and abatacept demonstrated higher incidence rates and statistically significantly increased risks of incident cancers compared with those initiating TNFis…; individuals initiating JAKis had a higher incidence rate, but the increase in risk was not statistically significant."
The numerical results need no change.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC11579790 write-up and regenerate its figures.
Paper: "Use of Biologic or Targeted Synthetic Disease-Modifying Antirheumatic Drugs
and Cancer Risk", JAMA Netw Open 2024, doi:10.1001/jamanetworkopen.2024.46336.
All inputs are copied from the article (Table 2 and Results) and from Supplement 1
(eTable 10, page 14, text layer extracted with PyMuPDF). Run:
python recompute.py.txt
Figures are written next to this file: five candidate designs (cand_A ... cand_E,
SVG + PNG at 200 dpi), and fig1/fig2, which are copies of the currently chosen
candidates (see CHOSEN below). The candidates are for choosing a design only;
once a design is chosen, set CHOSEN, re-run, and send only writeup.md,
fig1/fig2 and this script to the journal (not the cand_* files).
"""
import os
import shutil
import numpy as np
from scipy import stats
from scipy.stats import binomtest
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.transforms import blended_transform_factory
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.ppf(0.975)
# ---------------------------------------------------------------------------
# Inputs reported in the paper
# ---------------------------------------------------------------------------
# Adjusted HR vs TNFi (95% CI). Primary: Table 2, Abstract Results, Results.
# Propensity-matched: Results paragraph beginning "In the propensity analysis".
HR = {
("Primary", "Rituximab"): (1.91, 1.17, 3.14),
("Primary", "Abatacept"): (1.47, 1.03, 2.11),
("Primary", "JAKi"): (1.36, 0.94, 1.96),
("Primary", "IL-6i"): (1.04, 0.57, 1.92),
("PS-matched", "Rituximab"): (4.37, 1.48, 12.93),
("PS-matched", "Abatacept"): (3.12, 1.52, 6.44),
("PS-matched", "JAKi"): (1.79, 0.91, 3.56),
("PS-matched", "IL-6i"): (5.65, 1.11, 28.79),
}
# Table 2: crude cancer diagnoses, rate per 10 000 person-years (95% CI)
EVENTS = {
"TNFi": (162, 78, 66, 91),
"Rituximab": (14, 171, 94, 285),
"IL-6i": (8, 88, 38, 173),
"Abatacept": (30, 142, 96, 201),
"JAKi": (22, 94, 59, 143),
}
# Supplement 1, eTable 10 (propensity-matched TNFi vs JAKi, 2421 per arm per
# eTable 9): crude cancer diagnoses, rate per 10 000 PY (95% CI), adjusted HR
# as printed there (rounded to one decimal).
ETAB10 = {
"TNFi": (14, 60, 33, 101),
"JAKi": (22, 100, 63, 151),
}
ETAB10_HR = (1.8, 0.9, 3.6)
def wald_from_ci(hr, lo, hi):
"""Back-calculate SE(log HR), z and P from a log-scale Wald 95% CI."""
se = (np.log(hi) - np.log(lo)) / (2 * Z975)
z = np.log(hr) / se
p2 = 2 * stats.norm.sf(abs(z))
p1 = stats.norm.sf(z)
gmid = np.exp((np.log(lo) + np.log(hi)) / 2)
return se, z, p2, p1, gmid
print("=" * 78)
print("1. Wald back-calculation from each reported HR and 95% CI (log scale)")
print("=" * 78)
RES = {}
for key, (hr, lo, hi) in HR.items():
se, z, p2, p1, gmid = wald_from_ci(hr, lo, hi)
RES[key] = dict(hr=hr, lo=lo, hi=hi, se=se, z=z, p2=p2, p1=p1, gmid=gmid)
print(f"{key[0]:10s} {key[1]:9s} HR {hr:5.2f} ({lo:.2f}-{hi:.2f}) "
f"geo-mid {gmid:5.2f} SE {se:.3f} z {z:5.2f} "
f"P two-sided {p2:.3f} P one-sided {p1:.3f} CI excludes 1: {lo > 1}")
j = RES[("Primary", "JAKi")]
print("\nJAKi primary, symmetry check on the log scale:")
print(f" ln(HR) - ln(lower) = {np.log(1.36) - np.log(0.94):.3f}")
print(f" ln(upper) - ln(HR) = {np.log(1.96) - np.log(1.36):.3f}")
print(f" SE(log HR) = {j['se']:.4f}; z = {j['z']:.3f}; two-sided P = {j['p2']:.3f}")
print(f" |z| needed for two-sided P < .05: {Z975:.3f}")
print(f" Smallest HR significant at this SE: {np.exp(Z975 * j['se']):.2f}")
print("\n" + "=" * 78)
print("2. Table 2 crude rates: implied person-years and exact Poisson 95% CI")
print("=" * 78)
PY = {}
for k, (n, r, lo, hi) in EVENTS.items():
py = n / r * 1e4
PY[k] = py
l = stats.chi2.ppf(0.025, 2 * n) / 2
u = stats.chi2.ppf(0.975, 2 * n + 2) / 2
print(f"{k:10s} events {n:3d} rate {r:3d} implied PY ~{py:6.0f} "
f"exact CI {l / py * 1e4:4.0f}-{u / py * 1e4:4.0f} (reported {lo}-{hi})")
n1, n0 = 22, 162
irr = 94 / 78
se_irr = np.sqrt(1 / n1 + 1 / n0)
irr_lo, irr_hi = irr * np.exp(-Z975 * se_irr), irr * np.exp(Z975 * se_irr)
pfrac = PY["JAKi"] / (PY["JAKi"] + PY["TNFi"])
p_exact = binomtest(n1, n1 + n0, pfrac).pvalue
print(f"\nCrude IRR JAKi/TNFi = 94/78 = {irr:.2f} "
f"(Wald 95% CI {irr_lo:.2f}-{irr_hi:.2f}); "
f"exact conditional (binomial) two-sided P = {p_exact:.2f}")
print("\n" + "=" * 78)
print("3. Supplement 1, eTable 10 (propensity-matched TNFi vs JAKi)")
print("=" * 78)
PY10 = {}
for k, (n, r, lo, hi) in ETAB10.items():
py = n / r * 1e4
PY10[k] = py
l = stats.chi2.ppf(0.025, 2 * n) / 2
u = stats.chi2.ppf(0.975, 2 * n + 2) / 2
print(f"{k:10s} events {n:3d} rate {r:3d} implied PY ~{py:6.0f} "
f"exact CI {l / py * 1e4:4.0f}-{u / py * 1e4:4.0f} (reported {lo}-{hi})")
e10 = wald_from_ci(*ETAB10_HR)
print(f"eTable 10 adjusted HR {ETAB10_HR[0]} ({ETAB10_HR[1]}-{ETAB10_HR[2]}): "
f"SE {e10[0]:.3f}, z {e10[1]:.2f}, two-sided P {e10[2]:.3f} "
f"(bounds rounded to 1 decimal, so approximate)")
pm_ = HR[("PS-matched", "JAKi")]
print(f" Results text gives {pm_[0]} ({pm_[1]}-{pm_[2]}); rounded to 1 decimal: "
f"{pm_[0]:.1f} ({pm_[1]:.1f}-{pm_[2]:.1f}) -> matches eTable 10: "
f"{(round(pm_[0], 1), round(pm_[1], 1), round(pm_[2], 1)) == ETAB10_HR}")
n1m, n0m = ETAB10["JAKi"][0], ETAB10["TNFi"][0]
irr10 = ETAB10["JAKi"][1] / ETAB10["TNFi"][1]
se10 = np.sqrt(1 / n1m + 1 / n0m)
pf10 = PY10["JAKi"] / (PY10["JAKi"] + PY10["TNFi"])
p10 = binomtest(n1m, n1m + n0m, pf10).pvalue
print(f"Crude matched rate ratio 100/60 = {irr10:.2f} (Wald 95% CI "
f"{irr10 * np.exp(-Z975 * se10):.2f}-{irr10 * np.exp(Z975 * se10):.2f}); "
f"exact conditional two-sided P = {p10:.2f}")
# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
LW = 2.0 # 2 px-ish lines
MS = 8.5 # marker size in points (>= 8)
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def fmt_p(p):
return "<0.001" if p < 0.001 else f"{p:.3f}"
ORDER = ["Rituximab", "Abatacept", "JAKi", "IL-6i"]
# --- Candidate A: forest plot of reported HRs with recomputed P column -------
def cand_A():
rows = [("Primary", d) for d in ORDER] + [("PS-matched", d) for d in ORDER]
fig, ax = plt.subplots(figsize=(7.2, 5.2))
fig.subplots_adjust(left=0.14, right=0.645, top=0.91, bottom=0.11)
tr = blended_transform_factory(ax.transAxes, ax.transData)
ys = []
y = 0
for i, key in enumerate(rows):
if i == 4:
y += 0.8
ys.append(-y)
y += 1
for yy, key in zip(ys, rows):
r = RES[key]
is_j = key[1] == "JAKi"
if is_j:
ax.axhspan(yy - 0.45, yy + 0.45, color="#f3f1ec", zorder=0)
ax.plot([r["lo"], r["hi"]], [yy, yy], color=ORANGE, lw=LW, zorder=2,
solid_capstyle="butt")
ax.plot(r["hr"], yy, "o", color=ORANGE, ms=MS, zorder=3,
markeredgecolor=BG, markeredgewidth=1.2)
ax.text(1.02, yy, f"{r['hr']:.2f} ({r['lo']:.2f}-{r['hi']:.2f})",
va="center", ha="left", fontsize=10, color=INK, transform=tr)
ax.text(1.43, yy, fmt_p(r["p2"]), va="center", ha="left", fontsize=10,
color=BLUE, fontweight="bold" if is_j else "normal",
transform=tr)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.4, 40)
ticks = [0.5, 1, 2, 5, 10, 20, 40]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels([k[1] for k in rows])
ax.set_ylim(ys[-1] - 0.8, 1.6)
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Hazard ratio vs TNFi (log scale); vertical line = 1")
ax.text(0.405, ys[0] + 0.75, "Primary adjusted (Table 2)", fontsize=10,
color=INK2, fontweight="bold")
ax.text(0.405, ys[4] + 0.75, "Propensity-matched (Results)", fontsize=10,
color=INK2, fontweight="bold")
ax.text(1.02, 1.1, "Reported\nHR (95% CI)", fontsize=10, color=ORANGE,
fontweight="bold", ha="left", va="center", transform=tr)
ax.text(1.43, 1.1, "Recomputed\ntwo-sided P", fontsize=10, color=BLUE,
fontweight="bold", ha="left", va="center", transform=tr)
note = ax.text(2.15, ys[2], "Key Points/abstract call this\n"
"\"statistically significantly\"", fontsize=10,
color=INK, va="center", ha="left", linespacing=1.15)
# check that the label stays inside the plot area and the JAKi band
fig.canvas.draw()
bb = note.get_window_extent().transformed(ax.transData.inverted())
assert bb.x1 < ax.get_xlim()[1] and bb.y0 > ys[2] - 0.45 \
and bb.y1 < ys[2] + 0.45, bb
save(fig, "cand_A")
# --- Candidate B: back-calculated z statistics against the +/-1.96 line ------
def cand_B():
rows = [("Primary", d) for d in ORDER] + [("PS-matched", d) for d in ORDER]
labels = [f"{d}, {'primary' if a == 'Primary' else 'PS-matched'}"
for a, d in rows]
zs = [RES[k]["z"] for k in rows]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = -np.arange(len(rows), dtype=float)
ys[4:] -= 0.6
ax.axvspan(-Z975, Z975, color="#efeee9", zorder=0)
for xv in (-Z975, Z975):
ax.axvline(xv, color=INK2, lw=0.9, ls=(0, (4, 3)), zorder=1)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
for yy, zz, k in zip(ys, zs, rows):
ax.plot([0, zz], [yy, yy], color=BLUE, lw=LW, zorder=2,
solid_capstyle="butt")
ax.plot(zz, yy, "o", color=BLUE, ms=MS, zorder=3,
markeredgecolor=BG, markeredgewidth=1.2)
tx = zz + 0.08 if (zz + 0.08 > Z975 + 0.05 or zz < 1.0) else Z975 + 0.06
ax.text(tx, yy, f"z = {zz:.2f}, P = {fmt_p(RES[k]['p2'])}",
va="center", fontsize=10, color=INK)
ax.set_yticks(ys)
ax.set_yticklabels(labels)
ax.set_xlim(-0.6, 4.6)
ax.set_ylim(ys[-1] - 0.8, 2.0)
ax.set_xlabel("z = ln(HR) / SE, back-calculated from the reported 95% CI")
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.68, 1.15, "Not significant at the paper's\nstated 2-sided P < .05\n"
"(|z| < 1.96)", fontsize=10, color=INK2, ha="center", va="center")
ax.annotate("Key Points/abstract:\n\"statistically\nsignificantly\" higher",
xy=(j["z"], ys[2] - 0.12), xytext=(2.35, ys[3] - 0.05),
fontsize=10,
color=ORANGE, fontweight="bold", va="center",
arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.0))
fig.subplots_adjust(left=0.28, right=0.98, top=0.93, bottom=0.14)
save(fig, "cand_B")
# --- Candidate C: two text panels joined by an annotated connector ----------
def wrap_panel(ax, x, y, w, h, title, body):
ax.add_patch(FancyBboxPatch((x, y), w, h,
boxstyle="round,pad=0.008,rounding_size=0.015",
fc="#ffffff", ec=INK2, lw=1.0))
ax.text(x + 0.02, y + h - 0.04, title, fontsize=11, fontweight="bold",
va="top", color=INK)
ax.text(x + 0.02, y + h - 0.13, body, fontsize=10, va="top", color=INK,
linespacing=1.4)
def cand_C():
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
left = ("Key Points, Findings:\n"
"\"a statistically significantly higher\n"
"risk of incident cancer was seen with\n"
"rituximab, abatacept, and JAKis\n"
"compared with TNFis\"\n\n"
"Conclusions and Relevance:\n"
"\"...rituximab, abatacept, and JAKis\n"
"demonstrated ... statistically\n"
"significantly increased risks\"")
right = ("Table 2 and Results:\n"
"JAKis (HR, 1.36; 95% CI, 0.94-1.96)\n\n"
"Discussion:\n"
"\"only rituximab and abatacept\n"
"initiations had statistically\n"
"significant associations\"\n\n"
"Main-text Conclusions name only\n"
"rituximab and abatacept")
wrap_panel(ax, 0.03, 0.22, 0.40, 0.70, "Key Points and Abstract", left)
wrap_panel(ax, 0.57, 0.22, 0.40, 0.70, "Body of the paper", right)
arr = FancyArrowPatch((0.435, 0.57), (0.565, 0.57), arrowstyle="<->",
mutation_scale=14, color=INK2, lw=1.5)
ax.add_patch(arr)
ax.text(0.5, 0.64, "conflict", ha="center", fontsize=10, color=INK2,
fontweight="bold")
# bottom strip: the numbers that decide it
ax.text(0.03, 0.13, "Reported JAKi vs TNFi HR:", fontsize=10.5, color=INK2)
ax.text(0.30, 0.13, "1.36 (95% CI 0.94-1.96), CI includes 1", fontsize=10.5,
color=ORANGE, fontweight="bold")
ax.text(0.03, 0.06, "Recomputed from that CI:", fontsize=10.5, color=INK2)
ax.text(0.30, 0.06, f"z = {j['z']:.2f}, two-sided P = {j['p2']:.2f} "
"(paper's threshold: P < .05)", fontsize=10.5, color=BLUE,
fontweight="bold")
save(fig, "cand_C")
# --- Candidate D: every JAKi estimate the paper's numbers give, one axis -----
def cand_D():
pm = RES[("PS-matched", "JAKi")]
p_irr = 2 * stats.norm.sf(abs(np.log(irr) / se_irr))
items = [
("Adjusted HR, primary\n(Table 2)", j["hr"], j["lo"], j["hi"], ORANGE,
"reported", j["p2"]),
("Adjusted HR, propensity-\nmatched (Results)", pm["hr"], pm["lo"],
pm["hi"], ORANGE, "reported", pm["p2"]),
("Crude rate ratio\n94 / 78 per 10 000 PY", irr, irr_lo, irr_hi, BLUE,
"recomputed", p_exact),
]
fig, ax = plt.subplots(figsize=(7.2, 3.2))
tr = blended_transform_factory(ax.transAxes, ax.transData)
ys = [0, -1, -2]
for yy, (lab, est, lo, hi, col, kind, p) in zip(ys, items):
ax.plot([lo, hi], [yy, yy], color=col, lw=LW, solid_capstyle="butt",
zorder=2)
ax.plot(est, yy, "o" if kind == "reported" else "D", color=col, ms=MS,
markeredgecolor=BG, markeredgewidth=1.2, zorder=3)
ax.text(1.04, yy, f"{est:.2f} ({lo:.2f}-{hi:.2f})", va="center",
fontsize=10, color=INK, transform=tr)
ax.text(1.56, yy, f"{p:.2f}", va="center", fontsize=10, color=BLUE,
transform=tr)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.5, 4)
ticks = [0.5, 0.75, 1, 1.5, 2, 3, 4]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels([it[0] for it in items])
ax.set_ylim(-2.7, 1.0)
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("JAKi vs TNFi ratio (log scale); vertical line = 1")
ax.text(1.04, 0.62, "Estimate\n(95% CI)", fontsize=10, color=INK2,
fontweight="bold", transform=tr, va="center")
ax.text(1.56, 0.62, "Recomputed\nP", va="center", fontsize=10, color=BLUE,
fontweight="bold", transform=tr)
ax.plot([], [], "o", color=ORANGE, ms=MS, label="Reported in the paper")
ax.plot([], [], "D", color=BLUE, ms=MS,
label="Recomputed from Table 2 rates")
ax.legend(loc="lower center", bbox_to_anchor=(0.62, 1.02), ncol=2,
frameon=False, fontsize=10, handlelength=1.0)
fig.subplots_adjust(left=0.27, right=0.62, top=0.82, bottom=0.17)
save(fig, "cand_D")
# --- Candidate E: what each section says, drug by drug -----------------------
def cand_E():
cols = ["Key Points", "Abstract\nConclusions", "Discussion",
"Main-text\nConclusions", "Recomputed\nP (Table 2)"]
drugs = ["Rituximab", "Abatacept", "JAKi"]
S, N, O = "significant", "not significant", "not named"
says = {
"Rituximab": [S, S, S, S],
"Abatacept": [S, S, S, S],
"JAKi": [S, S, N, O],
}
fig, ax = plt.subplots(figsize=(7.2, 2.9))
ax.set_xlim(-0.5, len(cols) - 0.5)
ax.set_ylim(-len(drugs) + 0.5, 1.15)
ax.axis("off")
for ci, c in enumerate(cols):
ax.text(ci, 0.85, c, ha="center", va="center", fontsize=10,
fontweight="bold", color=BLUE if ci == 4 else INK)
ax.axhspan(-2.45, -1.55, xmin=0, xmax=1, color="#f3f1ec", zorder=0)
for ri, d in enumerate(drugs):
y = -ri
ax.text(-0.52, y, d, ha="right", va="center", fontsize=10.5,
fontweight="bold", clip_on=False)
for ci, v in enumerate(says[d]):
if v == S:
ax.plot(ci - 0.42, y, "o", color=ORANGE, ms=MS)
elif v == N:
ax.plot(ci - 0.42, y, "o", mfc=BG, mec=ORANGE, mew=2, ms=MS)
ax.text(ci - 0.33, y, v, va="center", fontsize=10,
color=INK if v != O else INK2)
p = RES[("Primary", d)]["p2"]
ax.text(4, y, f"{p:.3f}", ha="center", va="center", fontsize=10.5,
color=BLUE, fontweight="bold")
ax.plot([], [], "o", color=ORANGE, ms=MS,
label="paper says statistically significant")
ax.plot([], [], "o", mfc=BG, mec=ORANGE, mew=2, ms=MS,
label="paper says not significant")
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.02), ncol=2,
frameon=False, fontsize=10)
fig.subplots_adjust(left=0.15, right=0.99, top=0.97, bottom=0.18)
save(fig, "cand_E")
cand_A()
cand_B()
cand_C()
cand_D()
cand_E()
# fig1 / fig2 are copies of the chosen candidates
CHOSEN = {"fig1": "cand_C", "fig2": "cand_A"}
for fig_name, cand in CHOSEN.items():
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(HERE, f"{cand}.{ext}"),
os.path.join(HERE, f"{fig_name}.{ext}"))
print("\nFigures written to", HERE, "| chosen:", CHOSEN)
Week-24 HDRS-17 difference reported as P < .001; its t129 = 0.71 gives P = .48
JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.29427 · PMC12406064
Summary
This randomized trial compared measurement-based care (MBC) with standard care for major depressive disorder. For the secondary outcome of change in HDRS-17 score at week 24, it reports a reduction of 18.1 points (95% CI, 16.4–19.6) with MBC and 17.0 points (95% CI, 15.6–18.5) with standard care, with "t129 = 0.71; P < .001". The same values appear in the Abstract, the Results and Table 3, and the Abstract calls the difference "significantly but modestly greater". The printed statistic gives a two-sided P of 0.48: on 129 degrees of freedom, P < .001 needs |t| ≥ 3.37. Rebuilding the test from the printed group CIs gives t = 1.02 and P = 0.31, and the 95% CI of the 1.1-point difference (−1.04 to 3.24) includes zero. As printed, the Abstract presents the trial's prespecified end-of-study outcome for change in symptom severity as a significant benefit of MBC, but the paper's own numbers show no significant difference. The other week-24 efficacy results were already nonsignificant. The primary time-to-event outcomes are not affected.
What the paper reports
Abstract, Results:
After week 24, reduction in mean HDRS-17 scores was significantly but modestly greater in the MBC group than in the standard care group (-18.1 [95% CI, 16.4-19.6] points vs -17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P < .001).
The sentence just before it in the Abstract reads:
At week 24, there were no significant differences in rates of response or remission between groups.
Results, fourth paragraph (beginning "After week 24, the response rates..."):
After week 24, the reduction in mean HDRS-17 scores was larger in the MBC group than in the standard care group (-18.1 [95% CI, 16.4-19.6] points vs -17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P < .001) (Table 3)
Table 3 (Efficacy Outcomes), last row (measure | MBC group | standard care group | P value):
Change in HDRS-17 score, mean (95% CI), points | 18.1 (16.4-19.6) | 17.0 (15.6-18.5) | <.001
Methods, Statistical Analysis:
Baseline HDRS-17 scores, antidepressant doses, number of clinic visits, changes in HDRS-17 at week 24, and number of adverse effects were compared with independent, 2-tailed t tests or Mann-Whitney tests, as appropriate. The significance threshold was 2-tailed P < .05 for all analyses.
Supplement 1 (trial protocol and statistical analysis plan), Statistical analysis, in both the protocol and the SAP section:
Analysis of covariance will be used to compare the changes in HDRS scores between the two groups.
(The protocol version reads "changes in HDRS and YMRS scores".) In the same documents, independent-sample t tests, Mann-Whitney U tests and χ² tests are planned for sociodemographic and clinical characteristics, discontinuation, response and remission rates, and side effects. The published Methods instead list changes in HDRS-17 at week 24 among the outcomes compared with independent t tests or Mann-Whitney tests.
Table 2, week 24 (MBC vs standard care): "No." 64 vs 67; "HDRS-17 score, mean (SD)" 5.6 (4.2) vs 6.7 (4.7), P value ".14".
In the archived text, "t129" is t with subscript 129 (the degrees of freedom). The publisher's HTML prints the two changes as −18.1 and −17.0 with a true minus sign. All quotes were checked verbatim against the PMC-archived article text.
What the paper's own numbers show
Method. I made three independent checks, all with scipy (see recompute.py.txt).
- I converted the printed statistic, t = 0.71 on 129 df, to a two-sided P value.
- I rebuilt the between-group test from the printed group means and 95% CIs. I took each group's standard error as (upper − lower) / (2 × t₀.₉₇₅, n−1), using the week-24 sample sizes of 64 and 67 from Table 2. These sizes also match Table 3's percentages (57/64 = 89.1%, 58/67 = 86.6%) and the printed df (64 + 67 − 2 = 129). From those SEs I ran a pooled-variance t test, and as a check a Welch test and a normal approximation. To allow for rounding, I shifted each of the six printed values by ±0.05 and recomputed t at every combination of shifts.
- I recomputed the Table 2 week-24 comparison of HDRS-17 levels from the printed means, SDs and sample sizes.
- I approximated the analysis the protocol planned, an analysis of covariance (ANCOVA) of the change with baseline HDRS-17 as covariate. From the identity SD(change)² = SD(baseline)² + SD(week 24)² − 2r·SD(baseline)·SD(week 24), the pooled SDs (baseline 4.66 from Table 2, week 24 4.46 from Table 2, change 6.17 from the Table 3 CIs) give the baseline-to-week-24 correlation r. The ANCOVA residual SD is then SD(week 24)·√(1 − r²). This is approximate: Table 2's baseline SDs (5.0 and 4.3) are for all 154 randomized participants, not only the 131 week-24 completers. The completers' baseline means implied by Table 2 and Table 3 (18.1 + 5.6 and 17.0 + 6.7) are both 23.7, so baseline adjustment leaves the 1.1-point difference unchanged.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Two-sided P for t = 0.71, df = 129 | Abstract; Results, 4th paragraph | P < .001 | 0.479 (one-sided 0.239); 0.476–0.482 for t from 0.705 to 0.715 |
| |t| needed for two-sided P < .001 at df = 129 | derived | — | 3.37 (for P < .05: 1.98) |
| Degrees of freedom | Abstract; Table 2 week-24 "No." | 129 | 64 + 67 − 2 = 129, consistent with a two-group comparison |
| Difference in mean change, MBC − standard care | Table 3 | 18.1 vs 17.0 | 1.1 points |
| Group SEs implied by the 95% CIs | Table 3 | CI widths 3.2 and 2.9 | 0.80 (SD 6.41) and 0.73 (SD 5.94) |
| Between-group t from the CIs (pooled) | Table 3 | t129 = 0.71; P < .001 | t = 1.02, df 129, P = 0.31 (Welch t = 1.02, df 127, P = 0.31; normal approximation z = 1.00, P = 0.32) |
| Same, with ±0.05 rounding on all printed values | Table 3 | — | t 0.90 to 1.15; P 0.25 to 0.37 |
| 95% CI of the 1.1-point difference | derived from Table 3 | not reported | −1.04 to 3.24 (includes 0) |
| SE of the difference that P < .001 would need | derived | — | ≤ 0.33; the CIs imply 1.08, about 3.3 times larger |
| SE and 95% CI implied by the printed t = 0.71 | derived | — | SE 1.55; CI −1.97 to 4.17 (includes 0) |
| ANCOVA adjusted for baseline (protocol's planned analysis) | Supplement 1; Tables 2–3 SDs | not reported | residual df 128, so F₁,₁₂₈, not t₁₂₉; implied r ≈ 0.08 (0.04 and 0.13 by group); F₁,₁₂₈ ≈ 2.0, P ≈ 0.16 |
| Correlation the ANCOVA would need for P < .001 | derived | — | residual SD ≤ 1.87, i.e. r ≥ 0.91 (for P < .05: r ≥ 0.70) |
| Week-24 HDRS-17 level, 5.6 (4.2) n = 64 vs 6.7 (4.7) n = 67 | Table 2 | P = .14 | t = −1.41, df 129, P = 0.16 (also nonsignificant) |
| Within-group change from baseline (one-sample t) | from Table 3 CIs | — | MBC t = 22.6, df 63; standard care t = 23.4, df 66; both P < 10⁻³⁰ |
| Week-4 HDRS-17 level, 8.3 (4.9) vs 12.6 (8.1) | Table 2 | P < .001 | t = −3.72, df 135, P = 0.0003 (consistent) |
| Weeks 2, 8, 12 HDRS-17 level | Table 2 | .05, .02, .04 | 0.056, 0.017, 0.034 (consistent) |
| Response at week 24, 57/64 vs 58/67 | Table 3; Results, 4th paragraph | χ² = 0.19; P = .79 | Pearson χ² = 0.19, P = 0.66; Fisher exact P = 0.79 |
| Remission at week 24, 47/64 vs 42/67 | Table 3; Results, 4th paragraph | χ² = 1.74; P = .20 | Pearson χ² = 1.74, P = 0.19; Fisher exact P = 0.20 |
| Response at week 12, 58/64 vs 46/68 | Table 3 | P < .001 | Pearson P = 0.0012; Fisher exact P = 0.0013 |
Every check of the week-24 change comparison gives a P value between about 0.16 and 0.48. None comes near .001. The other HDRS-17 comparisons in Table 2 and the response and remission comparisons in Table 3 reproduce closely; the dose and visit comparisons in Table 2 were not checked. Two of them have small discrepancies that do not change significance: the week-24 rate P values match Fisher's exact test rather than the Pearson χ² printed beside them, and the week-12 response P is 0.0012–0.0013, not < .001. Those two points are outside the scope of this report. The printed t of 0.71 is not exactly reproducible from the printed CIs either (they give t ≈ 1.02). Whichever of the two is used, the result is clearly nonsignificant.
Figure
Figure 1. Two-sided P values for MBC versus standard care at week 24, on one log-scaled axis. The thin vertical line marks the paper's significance threshold of P = .05. Orange marks values printed in the paper and blue marks values recomputed from the paper's own numbers. The top row is the printed P < .001 for the difference in HDRS-17 change; the arrow shows that the printed value is an upper bound. The next three rows recompute the same comparison: from the printed statistic t129 = 0.71 (P = 0.48); from the printed group 95% CIs (P = 0.31; the short bar is the range when each printed value is shifted by ±0.05 for rounding, 0.25–0.37); and as the ANCOVA planned in the protocol, approximated from the printed SDs (P ≈ 0.16 at the implied baseline-to-week-24 correlation r ≈ 0.08). Each row's value is printed to the left of the threshold line, joined to its marker by a dotted leader. The bottom two rows show the closely related week-24 comparison of HDRS-17 levels in Table 2, as printed (P = .14) and as recomputed (P = 0.16). Only the printed P for the change score lies below the threshold.
Figure 2. The between-group difference in week-24 HDRS-17 change (MBC minus standard care, 1.1 points), on a points scale with zero (no difference) marked. Blue: the 95% CI of the difference implied by the printed group CIs (−1.04 to 3.24) and by the printed t129 = 0.71 (−1.97 to 4.17). Both include zero. Orange, dashed: the widest 95% CI that a 1.1-point difference could have if the printed P < .001 were correct (0.45 to 1.75, SE ≤ 0.33). That interval is about one third as wide as the one the paper's own CIs imply.
Why it matters
Change in HDRS-17 score at week 24 is a prespecified secondary outcome: the trial's end-of-study outcome for change in symptom severity. The Abstract reports it as the one positive week-24 efficacy finding ("significantly but modestly greater ... P < .001"), immediately after stating that week-24 response and remission rates did not differ. With the P value corrected, no week-24 efficacy outcome favours MBC significantly: response P = .79, remission P = .20, HDRS-17 level P = .14 (Table 2), and HDRS-17 change P ≈ .48 (from the printed t), ≈ .31 (from the printed CIs) or about .16 (the protocol's ANCOVA, approximated from the printed SDs). The difference still favours MBC by 1.1 points, but its 95% CI runs from about −1 to +3 points. That interval is compatible with no difference.
As printed, a reader of the Abstract would conclude that MBC produced both faster improvement and a reliably greater reduction in severity by the end of the study. The paper's own numbers support only the first. The error is repeated in three places (Abstract, Results, Table 3), with no correct value given anywhere in the paper.
Not affected: the primary outcomes, time to response and time to remission (log-rank P = .001 for both; Cox HR 1.53 and 1.80); the week-12 rate differences; and the formal Conclusions, which rest on the time-to-event results. The Key Points do not cite this result.
Explanations considered
- A within-group (change-from-baseline) test. This is the only reading that yields P < .001: the one-sample t statistics are 22.6 and 23.4, with P < 10⁻³⁰. It does not fit the paper, though. Each within-group test has 63 or 66 df, not 129. The reported t is 0.71, not over 20. The P value sits in Table 3's between-group P value column. Both text passages describe a comparison ("greater in the MBC group than in the standard care group"). It does not account for the numbers.
- A one-sided test. One-sided P = 0.24. The Methods specify 2-tailed tests. It does not account for the numbers.
- A Mann-Whitney test instead of a t test. The Methods allow either "as appropriate". The printed statistic, however, is a t with 129 df, not a U or z statistic. The summary data show a 1.1-point difference against within-group SDs of about 6 (a standardized difference of about 0.18). A rank test with P < .001 would need a much larger separation between the groups than these summaries suggest. A rank-based result cannot be fully ruled out from summary data alone, but nothing in the paper reports one, and it would not match the t129 = 0.71 printed next to the P value.
- A covariate-adjusted analysis. The trial protocol and statistical analysis plan (Supplement 1) planned exactly this: "Analysis of covariance will be used to compare the changes in HDRS scores between the two groups." The published Methods instead name independent t tests or Mann-Whitney tests for the change at week 24. An ANCOVA still does not fit the printed result. With baseline as a covariate, the model has 64 + 67 − 3 = 128 residual df, and the group effect would be reported as F on 1 and 128 df, not as t129. On the numbers, P < .001 for a 1.1-point adjusted difference needs F ≥ 11.3, which means a residual SD of at most 1.87 and a baseline-to-week-24 correlation of at least 0.91. The printed SDs (baseline 5.0 and 4.3, week 24 4.2 and 4.7, change about 6.17 from the CIs) imply a correlation of about 0.08 (0.04 in the MBC group, 0.13 with standard care). That is approximate, because the baseline SDs are for all randomized participants, but it is far below 0.91. At that correlation the ANCOVA gives F₁,₁₂₈ ≈ 2.0 and P ≈ 0.16; even r = 0.7 would give only P ≈ .05. It does not account for the numbers.
- The t statistic is the misprint, not the P value. If P < .001 were correct, |t| would have to be at least 3.37 and the SE of the difference at most 0.33. The printed group CIs independently imply an SE of about 1.08 and t ≈ 1.02. Table 2's week-24 levels (P = .14) also show no significant difference. The P value is the number that conflicts with everything else, whichever of the two printed statistics is used.
- Rounding, or asymmetric CIs. Shifting all six printed values by ±0.05 keeps P between 0.25 and 0.37. The MBC interval is slightly asymmetric (1.7 below the mean, 1.5 above), which may mean it is not a plain t interval (for example, a bootstrap interval). Even if both group SEs were halved, t would be about 2 and P about .04, still far from .001. Rounding does not account for the numbers.
- Text-conversion artifact. The publisher's current HTML, the PMC page and the Europe PMC JATS XML all carry the same values. In the XML, the statistic is marked up as italic t with subscript 129, "0.71" uses a normal decimal point, and "<" encodes "<". The Table 3 row has four cells, with "<.001" in the P value column. A conversion artifact is ruled out.
Status
As of 2026-10-23:
- Version of record. The DOI resolves to the publisher's article page (jamanetwork.com/journals/jamanetworkopen/fullarticle/2838317). I fetched it on 2026-10-23. It is the current, corrected version and carries the notice "This article was corrected on July 14, 2026, to fix the values reported in the last 3 rows of Table 1." Its Abstract, Results and Table 3 still read "t129 = 0.71; P < .001" and "<.001", as quoted above. The PMC HTML page and the Europe PMC JATS XML match.
- Corrections. There is one correction: "Errors in Table 1" (doi:10.1001/jamanetworkopen.2026.26785; PMC13370302; July 2026). It concerns only the annual participant income, household income and time-since-diagnosis medians and IQRs in Table 1, which had been calculated with the SPSS "HAVERAGE" method. It does not address the week-24 HDRS-17 change result. Crossref update metadata and PMC notices list nothing else; Europe PMC lists only this correction.
- Supplements. I read all three supplementary PDFs from the Europe PMC supplementary-files archive. Supplement 1 is the trial protocol and statistical analysis plan. It plans independent-sample t tests, Mann-Whitney U tests and χ² tests for baseline characteristics, discontinuation, response and remission rates and side effects, and specifies analysis of covariance for the between-group comparison of HDRS changes; the published Methods specify independent t tests or Mann-Whitney tests for that comparison. Supplement 1 contains no outcome numbers. Supplement 2 contains eTables 1–3 (MBC schedule, adherence, adverse effects) and the eFigure (time to discontinuation); none of them analyses week-24 HDRS-17 change. Supplement 3 is the data sharing statement. Nothing in the supplements resolves the inconsistency.
- PubPeer and other discussion. An earlier audit step on the same day found no PubPeer thread for the DOI or title. The secondary news coverage that was read (2 Minute Medicine) does not mention the inconsistency. The full text of the linked invited commentary (doi:10.1001/jamanetworkopen.2025.29440) could not be accessed: the publisher page returned a bot-verification page, the PDF text was not extractable, and it is not in PMC or Europe PMC full text. Whether it mentions the inconsistency is therefore unknown. The PubPeer search page and API returned a human-verification challenge (HTTP 403). A repeat web search for this write-up was not possible because the session's search allowance was used up.
- Sources. https://doi.org/10.1001/jamanetworkopen.2025.29427; https://pmc.ncbi.nlm.nih.gov/articles/PMC12406064/; Europe PMC full text and supplementary files for PMC12406064; https://doi.org/10.1001/jamanetworkopen.2026.26785 (PMC13370302); Supplement 1 (jamanetwopen-e2529427-s001.pdf, Europe PMC supplementary files); https://www.2minutemedicine.com/measurement-based-care-to-enhance-antidepressant-treatment-outcomes-in-major-depressive-disorder-a-randomized-clinical-trial/.
Suggested correction
The authors should check the analysis output for the week-24 change comparison. The P value is 0.48 if the printed t129 = 0.71 is right, and about 0.31 if the printed group CIs are right; whichever value the analysis produces should be reported with the matching statistic. Adding the between-group difference and its 95% CI would let readers check the result directly. Using the printed t:
Abstract, Results. Replace "After week 24, reduction in mean HDRS-17 scores was significantly but modestly greater in the MBC group than in the standard care group (... t129 = 0.71; P < .001)." with:
After week 24, reduction in mean HDRS-17 scores did not differ significantly between the MBC group and the standard care group (−18.1 [95% CI, 16.4-19.6] points vs −17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P = .48).
Results, fourth paragraph. Replace "the reduction in mean HDRS-17 scores was larger in the MBC group than in the standard care group (... t129 = 0.71; P < .001)" with:
the reduction in mean HDRS-17 scores did not differ significantly between the MBC group and the standard care group (−18.1 [95% CI, 16.4-19.6] points vs −17.0 [95% CI, 15.6-18.5] points; t129 = 0.71; P = .48)
Table 3, last row. Replace the P value "<.001" with ".48", or with the value from the verified analysis.
With these changes, the Abstract sentence "No differences were observed in other secondary outcomes" would also cover HDRS-17 change. The Conclusions do not need to change.
Recomputation script (python recompute.py.txt)
"""Recompute the week-24 between-group test of HDRS-17 change in PMC12406064
(JAMA Netw Open 2025; doi:10.1001/jamanetworkopen.2025.29427) from the paper's own
printed numbers, run the cross-checks used in the write-up, and redraw fig1/fig2
in this file's folder.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Every input below is copied from the published article (Abstract, Results,
Methods, Table 2, Table 3), except the analysis-plan quote in section 2b, which is
from Supplement 1 (trial protocol and statistical analysis plan).
"""
import os
import itertools
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Abstract, Results (fourth paragraph) and Table 3 (last row): change in HDRS-17 score, mean (95% CI)
M1, LO1, HI1 = 18.1, 16.4, 19.6 # MBC
M2, LO2, HI2 = 17.0, 15.6, 18.5 # standard care
T_REP, DF_REP = 0.71, 129 # "t129 = 0.71"
P_REP = 0.001 # "P < .001"
# Table 2, week 24 "No." row (consistent with Table 3: 57/64 = 89.1%, 58/67 = 86.6%)
N1, N2 = 64, 67
# Table 2: HDRS-17 mean (SD), n, reported P (MBC first, then standard care)
TABLE2 = {
"Baseline": ((24.1, 5.0, 76), (23.4, 4.3, 78), None),
"Wk 2": ((12.2, 5.3, 68), (14.6, 8.9, 72), ".05"),
"Wk 4": ((8.3, 4.9, 66), (12.6, 8.1, 71), "<.001"),
"Wk 8": ((7.7, 5.2, 64), (10.4, 7.4, 68), ".02"),
"Wk 12": ((6.9, 5.3, 64), (9.2, 6.9, 68), ".04"),
"Wk 24": ((5.6, 4.2, 64), (6.7, 4.7, 67), ".14"),
}
# Table 3 / Results (fourth paragraph): response and remission counts, reported chi2 and P
TABLE3 = [
("Response wk 12", (58, 64), (46, 68), None, "<.001"),
("Response wk 24", (57, 64), (58, 67), 0.19, ".79"),
("Remission wk 12", (46, 64), (35, 68), None, ".02"),
("Remission wk 24", (47, 64), (42, 67), 1.74, ".20"),
]
R = {} # numbers reused by the figures
def p2(t, df):
return 2 * stats.t.sf(abs(t), df)
print("=== 1. The reported statistic: t129 = 0.71 ===")
R["p_from_t"] = p2(T_REP, DF_REP)
R["t_crit_001"] = stats.t.isf(0.0005, DF_REP)
R["t_crit_05"] = stats.t.isf(0.025, DF_REP)
print(f"df check: n1 + n2 - 2 = {N1} + {N2} - 2 = {N1 + N2 - 2} (reported {DF_REP})")
print(f"two-sided P for t = {T_REP}, df = {DF_REP}: {R['p_from_t']:.3f} (one-sided {R['p_from_t']/2:.3f})")
print(f"|t| needed for two-sided P < .001 at df = {DF_REP}: {R['t_crit_001']:.2f}")
print(f"|t| needed for two-sided P < .05 at df = {DF_REP}: {R['t_crit_05']:.2f}")
print(f"largest two-sided P for t in [0.705, 0.715): "
f"{p2(0.705, DF_REP):.3f}; smallest: {p2(0.715, DF_REP):.3f}")
print("\n=== 2. Between-group test rebuilt from the reported group 95% CIs ===")
q1, q2 = stats.t.ppf(0.975, N1 - 1), stats.t.ppf(0.975, N2 - 1)
se1, se2 = (HI1 - LO1) / (2 * q1), (HI2 - LO2) / (2 * q2)
sd1, sd2 = se1 * np.sqrt(N1), se2 * np.sqrt(N2)
diff = M1 - M2
print(f"MBC: CI width {HI1-LO1:.1f} -> SE {se1:.3f}, implied SD {sd1:.2f} "
f"(CI extends {M1-LO1:.1f} below, {HI1-M1:.1f} above the mean)")
print(f"SC : CI width {HI2-LO2:.1f} -> SE {se2:.3f}, implied SD {sd2:.2f} "
f"(CI extends {M2-LO2:.1f} below, {HI2-M2:.1f} above the mean)")
print(f"difference in mean change: {M1} - {M2} = {diff:.1f} points")
# pooled-variance (Student) t test, df = 129, as the reported subscript implies
sp = np.sqrt(((N1 - 1) * sd1**2 + (N2 - 1) * sd2**2) / (N1 + N2 - 2))
se_p = sp * np.sqrt(1 / N1 + 1 / N2)
t_p = diff / se_p
R["t_ci"], R["p_ci"], R["se_ci"] = t_p, p2(t_p, N1 + N2 - 2), se_p
qd = stats.t.ppf(0.975, N1 + N2 - 2)
R["ci_diff"] = (diff - qd * se_p, diff + qd * se_p)
print(f"Student: pooled SD {sp:.2f}, SE(diff) {se_p:.3f}, t = {t_p:.2f}, df = {N1+N2-2}, "
f"P = {R['p_ci']:.2f}")
print(f" 95% CI of the difference: {R['ci_diff'][0]:.2f} to {R['ci_diff'][1]:.2f}")
# Welch
se_w = np.sqrt(se1**2 + se2**2)
df_w = se_w**4 / (se1**4 / (N1 - 1) + se2**4 / (N2 - 1))
t_w = diff / se_w
print(f"Welch : SE(diff) {se_w:.3f}, t = {t_w:.2f}, df = {df_w:.0f}, P = {p2(t_w, df_w):.2f}")
# normal approximation (SE = width / 3.92)
se_z = np.sqrt(((HI1 - LO1) / 3.92) ** 2 + ((HI2 - LO2) / 3.92) ** 2)
z = diff / se_z
print(f"z-approx: SE(diff) {se_z:.3f}, z = {z:.2f}, P = {2*stats.norm.sf(z):.2f}")
# rounding: every printed value may be off by up to 0.05; take all corners
ts = []
for a, b, l1, h1, l2, h2 in itertools.product(*[(-0.05, 0.05)] * 6):
s1 = ((HI1 + h1) - (LO1 + l1)) / (2 * q1)
s2 = ((HI2 + h2) - (LO2 + l2)) / (2 * q2)
ts.append(((M1 + a) - (M2 + b)) / np.sqrt(s1**2 + s2**2))
R["t_rng"] = (min(ts), max(ts))
R["p_rng"] = (p2(max(ts), DF_REP), p2(min(ts), DF_REP))
print(f"rounding (+/-0.05 on all six printed values): t {R['t_rng'][0]:.2f} to {R['t_rng'][1]:.2f}, "
f"two-sided P {R['p_rng'][0]:.2f} to {R['p_rng'][1]:.2f}")
R["se_needed"] = diff / R["t_crit_001"]
print(f"SE(diff) a 1.1-point difference would need for P < .001: {R['se_needed']:.2f} "
f"(the CIs imply {se_p:.2f}, about {se_p / R['se_needed']:.1f} times larger)")
R["se_from_t"] = diff / T_REP
R["ci_from_t"] = (diff - qd * R["se_from_t"], diff + qd * R["se_from_t"])
print(f"SE(diff) implied by t = 0.71 and diff = 1.1: {R['se_from_t']:.2f} "
f"(pooled SD {R['se_from_t'] / np.sqrt(1/N1 + 1/N2):.2f}); "
f"95% CI {R['ci_from_t'][0]:.2f} to {R['ci_from_t'][1]:.2f}")
R["ci_needed"] = (diff - qd * R["se_needed"], diff + qd * R["se_needed"])
print(f"95% CI of the difference if P were exactly .001: "
f"{R['ci_needed'][0]:.2f} to {R['ci_needed'][1]:.2f}")
print(f"standardized difference: {diff:.1f} / pooled SD {sp:.2f} = {diff/sp:.2f}")
t_half = diff / (se_p / 2)
print(f"if both group SEs were halved: t = {t_half:.2f}, P = {p2(t_half, DF_REP):.3f}")
print("\n=== 2b. The protocol's planned analysis: ANCOVA adjusted for baseline ===")
# Supplement 1 (protocol and SAP): "Analysis of covariance will be used to compare the
# changes in HDRS scores between the two groups." The published Methods instead name
# independent t tests or Mann-Whitney tests. An ANCOVA with group + baseline HDRS-17 would
# report F on 1 and N - 3 df, not t on N - 2 df.
DF_ANC = N1 + N2 - 3
print(f"residual df of a two-group model adjusted for baseline: {N1} + {N2} - 3 = {DF_ANC} "
f"(printed subscript {DF_REP}); statistic would be F(1, {DF_ANC}), not t{DF_REP}")
(bx1, bsx1, bn1), (bx2, bsx2, bn2), _ = TABLE2["Baseline"]
(wy1, wsy1, _), (wy2, wsy2, _), _ = TABLE2["Wk 24"]
# baseline SDs are for all randomized participants (n 76/78), week-24 SDs for completers
sx = np.sqrt(((bn1 - 1) * bsx1**2 + (bn2 - 1) * bsx2**2) / (bn1 + bn2 - 2))
sy = np.sqrt(((N1 - 1) * wsy1**2 + (N2 - 1) * wsy2**2) / (N1 + N2 - 2))
sd_chg = sp # pooled change SD implied by Table 3 CIs
r_pool = (sx**2 + sy**2 - sd_chg**2) / (2 * sx * sy)
r_g1 = (bsx1**2 + wsy1**2 - sd1**2) / (2 * bsx1 * wsy1)
r_g2 = (bsx2**2 + wsy2**2 - sd2**2) / (2 * bsx2 * wsy2)
print(f"pooled SDs: baseline {sx:.2f} (Table 2, all randomized), week 24 {sy:.2f} (Table 2), "
f"change {sd_chg:.2f} (from Table 3 CIs)")
print(f"implied baseline-to-week-24 correlation r = (sx^2 + sy^2 - s_change^2) / (2 sx sy) "
f"= {r_pool:.2f} (per group: MBC {r_g1:.2f}, standard care {r_g2:.2f})")
# completers' baseline means implied by Table 3 change + Table 2 week-24 mean
cb1, cb2 = M1 + wy1, M2 + wy2
print(f"completers' baseline means implied: MBC {M1} + {wy1} = {cb1:.1f}, "
f"SC {M2} + {wy2} = {cb2:.1f} (so baseline adjustment shifts the 1.1-point difference "
f"by {r_pool * sy / sx * abs(cb1 - cb2):.2f}; adjusted difference = {wy2 - wy1:.1f})")
k = np.sqrt(1 / N1 + 1 / N2)
def ancova(r):
beta = r * sy / sx
# adjusted difference in reduction = -(Y1 - Y2) + beta * (X1 - X2), completers' baselines
adj = (wy2 - wy1) + beta * (cb1 - cb2)
s_res = sy * np.sqrt(1 - r**2) * np.sqrt((N1 + N2 - 2) / DF_ANC)
t = adj / (s_res * k)
return s_res, t, p2(t, DF_ANC)
s_res, t_anc, p_anc = ancova(r_pool)
R["p_ancova"], R["r_pool"] = p_anc, r_pool
print(f"ANCOVA at r = {r_pool:.2f}: residual SD {s_res:.2f}, SE(adjusted diff) {s_res*k:.3f}, "
f"t = {t_anc:.2f}, F(1, {DF_ANC}) = {t_anc**2:.2f}, P = {p_anc:.2f}")
t_need = stats.t.isf(0.0005, DF_ANC)
sres_need = diff / (t_need * k)
r_need = np.sqrt(1 - (sres_need / sy) ** 2 * DF_ANC / (N1 + N2 - 2))
t_need05 = stats.t.isf(0.025, DF_ANC)
r_need05 = np.sqrt(1 - (diff / (t_need05 * k) / sy) ** 2 * DF_ANC / (N1 + N2 - 2))
R["r_need"] = r_need
print(f"for ANCOVA P < .001 (|t| >= {t_need:.2f}, F >= {t_need**2:.1f}): residual SD <= "
f"{sres_need:.2f}, i.e. r >= {r_need:.2f}; for P < .05: r >= {r_need05:.2f}")
for rr in (0.0, 0.3, 0.5, 0.7, 0.9):
_, tt, pp = ancova(rr)
print(f" r = {rr:.1f}: t = {tt:.2f}, F = {tt**2:.2f}, P = {pp:.3f}")
print("\n=== 3. Alternative reading: within-group change from baseline ===")
for name, m, se, n in [("MBC", M1, se1, N1), ("SC", M2, se2, N2)]:
tt = m / se
print(f"{name}: one-sample t = {tt:.1f}, df = {n-1}, P = {p2(tt, n-1):.0e}")
print("\n=== 4. Table 2 cross-checks (Student t from mean, SD, n) ===")
for lbl, ((ma, sa, na), (mb, sb, nb), rep) in TABLE2.items():
s = np.sqrt(((na - 1) * sa**2 + (nb - 1) * sb**2) / (na + nb - 2))
tt = (ma - mb) / (s * np.sqrt(1 / na + 1 / nb))
pp = p2(tt, na + nb - 2)
tw = (ma - mb) / np.sqrt(sa**2 / na + sb**2 / nb)
dfw = (sa**2/na + sb**2/nb)**2 / ((sa**2/na)**2/(na-1) + (sb**2/nb)**2/(nb-1))
if lbl == "Wk 24":
R["p_wk24_level"] = pp
print(f"{lbl:8s}: diff {ma-mb:+.1f}, t = {tt:.2f}, df = {na+nb-2}, P = {pp:.4f}; "
f"Welch P = {p2(tw, dfw):.4f} (reported {rep})")
print(f"Table 2 means: MBC 24.1 - 5.6 = {24.1-5.6:.1f}, SC 23.4 - 6.7 = {23.4-6.7:.1f} "
f"(Table 3 change, completers: 18.1 and 17.0)")
print("\n=== 5. Table 3 rates (Pearson chi2, Yates chi2, Fisher exact) ===")
for lbl, (a, na), (b, nb), chi_rep, rep in TABLE3:
tab = np.array([[a, na - a], [b, nb - b]])
c0 = stats.chi2_contingency(tab, correction=False)
c1 = stats.chi2_contingency(tab, correction=True)
pf = stats.fisher_exact(tab)[1]
print(f"{lbl:16s}: {a}/{na} = {100*a/na:.1f}% vs {b}/{nb} = {100*b/nb:.1f}%; "
f"Pearson chi2 = {c0[0]:.2f}, P = {c0[1]:.4f}; Yates chi2 = {c1[0]:.2f}, P = {c1[1]:.4f}; "
f"Fisher P = {pf:.4f} [reported chi2 {chi_rep}, P {rep}]")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK, "axes.edgecolor": INK2, "svg.fonttype": "none"})
LW = 1.5 # 2 px at 96 px/in
MS = 9
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
def style(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
# Fig 1: two-sided P values on one log axis
rows = [
("Week-24 HDRS-17 change\nas printed (Abstract, Results, Table 3)", P_REP, None, "rep"),
("Same test: P from the printed\nstatistic $t_{129}$ = 0.71", R["p_from_t"], None, "rec"),
("Same test: rebuilt from the\nprinted group 95% CIs", R["p_ci"], R["p_rng"], "rec"),
("Protocol's ANCOVA, approximated\nfrom the printed SDs", R["p_ancova"], None, "rec"),
("Week-24 HDRS-17 level\n(Table 2), as printed", 0.14, None, "rep"),
("Week-24 HDRS-17 level\n(Table 2), recomputed", R["p_wk24_level"], None, "rec"),
]
labels = ["P < .001",
f"P = {R['p_from_t']:.2f}",
f"P = {R['p_ci']:.2f} (rounding range {R['p_rng'][0]:.2f}-{R['p_rng'][1]:.2f})",
f"P \u2248 {R['p_ancova']:.2f} ($F_{{1,128}}$ \u2248 {t_anc**2:.1f}, r \u2248 {R['r_pool']:.2f})",
"P = .14",
f"P = {R['p_wk24_level']:.2f}"]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
style(ax)
ax.set_xscale("log")
ax.set_xlim(1e-4, 1.0)
ys = np.arange(len(rows))[::-1].astype(float)
ys[4:] -= 0.35
ax.axvline(0.05, color=INK2, lw=0.9, zorder=1)
ax.text(0.05, ys[0] + 0.62, "P = .05 (paper's threshold) ", color=INK2, fontsize=10,
ha="right", va="center")
XLAB = 0.04 # value labels end left of the threshold line
for y, (lab, p, rng, kind), txt in zip(ys, rows, labels):
c = ORANGE if kind == "rep" else BLUE
if rng is not None:
ax.plot(rng, [y, y], color=c, lw=LW, solid_capstyle="round", zorder=2)
if p == P_REP and kind == "rep":
ax.annotate("", xy=(1.6e-4, y), xytext=(P_REP, y),
arrowprops=dict(arrowstyle="-|>", color=c, lw=LW, mutation_scale=12), zorder=2)
ax.text(p * 1.35, y, txt, color=INK, fontsize=10, va="center")
else:
ax.plot([XLAB * 1.1, (rng[0] if rng else p) / 1.12], [y, y], color=GRID, lw=0.9,
ls=":", zorder=1)
ax.text(XLAB, y, txt, color=INK, fontsize=10, ha="right", va="center")
ax.plot([p], [y], marker="o", ms=MS, color=c, mec=BG, mew=1.5, zorder=3)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(ys[-1] - 0.6, ys[0] + 0.9)
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 0.1, 0.5, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "0.1", "0.5", "1"])
ax.set_xlabel("Two-sided P value, MBC vs standard care (log scale)")
handles = [Line2D([], [], ls="", marker="o", ms=MS, color=ORANGE, label="Reported in the paper"),
Line2D([], [], ls="", marker="o", ms=MS, color=BLUE,
label="Recomputed from the paper's own numbers")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(0, 1.0), ncol=2, frameon=False,
fontsize=10, handletextpad=0.3)
save(fig, "fig1")
# Fig 2: the 1.1-point difference and its 95% CI on one axis (HDRS-17 points)
rows2 = [
("95% CI implied by the\nprinted group CIs", R["ci_diff"], BLUE, "-"),
("95% CI implied by the\nprinted $t_{129}$ = 0.71", R["ci_from_t"], BLUE, "-"),
("Widest 95% CI compatible\nwith the printed P < .001", R["ci_needed"], ORANGE, "--"),
]
fig, ax = plt.subplots(figsize=(7.2, 2.9))
style(ax)
ax.axvline(0, color=INK2, lw=0.9, zorder=1)
ax.text(0, 2.75, "no difference ", color=INK2, fontsize=10, ha="right", va="center")
for y, (lab, (lo, hi), c, ls) in zip([2, 1, 0], rows2):
ax.plot([lo, hi], [y, y], color=c, lw=LW, ls=ls, solid_capstyle="round", zorder=2)
ax.plot([diff], [y], marker="o", ms=MS, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(hi + 0.12, y, f"{diff:.1f} ({lo:.2f} to {hi:.2f})", color=INK, fontsize=10, va="center")
ax.set_yticks([2, 1, 0])
ax.set_yticklabels([r[0] for r in rows2], fontsize=10)
ax.set_ylim(-0.6, 3.0)
ax.set_xlim(-3, 7)
ax.set_xlabel("Difference in week-24 HDRS-17 change, MBC minus standard care (points)")
handles = [Line2D([], [], color=ORANGE, lw=LW, ls="--", marker="o", ms=MS,
label="What the reported P < .001 requires"),
Line2D([], [], color=BLUE, lw=LW, marker="o", ms=MS,
label="Recomputed from the paper's own numbers")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(0, 1.0), ncol=2, frameon=False,
fontsize=10, handletextpad=0.3)
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png in", HERE)
Abstract reports overall-cohort length-of-stay estimates as the results for hypotensive patients
JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.33266 · PMC12457976
Summary
The abstract states that, among patients with hypotension on admission, splenic angioembolization (SAE) and observation were each associated with a shorter hospital length of stay (HLOS) than open splenectomy (OS): SAE β = -1.44 (95% CI, -1.79 to -1.09; P < .001) and observation β = -1.41 (95% CI, -1.73 to -1.09; P < .001). These six numbers are identical to the paper's Table 2 rows for the total cohort (n = 7567). Table 2's rows for the hypotension subgroup (SBP <90 mm Hg, n = 651) are different: SAE β = -1.65 (-3.24 to -0.06; P = .04) and observation β = -0.13 (-1.58 to 1.32; P = .86). The Results text reports only the SAE figure for this subgroup. For observation, the abstract therefore reports a highly significant benefit of about 1.4 days where the paper's own subgroup analysis, and its inverse-probability-weighted (IPW) sensitivity analysis, show no significant difference. The claim is repeated in words in the main-text Conclusions, and it supports the headline statement that nonoperative management had "favorable outcomes ... even in patients with hypotension on admission".
What the paper reports
Abstract, Results:
Among patients with hypotension, there was no mortality difference, but shorter hospital length of stay was found in the SAE (β = -1.44; 95% CI, -1.79 to -1.09; P < .001) and observation (β = -1.41; 95% CI, -1.73 to -1.09; P < .001) groups.
Results, "Multivariable Analysis for Other Outcomes". This paragraph gives the same two estimates for the total cohort, then reports the hypotension subgroup separately and names only SAE:
Multivariable linear regression analysis of continuous variable outcomes is given in Table 2. SAE (β = -1.44; 95% CI, -1.79 to -1.09; P < .001) and observation (β = -1.41; 95% CI, -1.73 to -1.09; P < .001) showed a negative association with HLOS days.
The same results were observed in normal SBP subgroup. In the hypotension subgroup, SAE was also negatively associated with HLOS days (β = -1.65; 95% CI, -3.24 to -0.06; P = .04) and ICULOS days (β = -1.45; 95% CI, -2.62 to -0.29; P = .01).
Table 2, "SAE or Observation vs OS Outcomes Using Multivariable Linear Regression and IPW". Columns: outcome | No. | mean (SD) | regression β (95% CI) | P value | IPW β (95% CI) | P value. Total block, HLOS days:
SAE | 1547 | 6.35 (4.17) | -1.44 (-1.79 to -1.09) | <.001 | -1.77 (-2.24 to -1.30) | <.001
Observation | 4521 | 6.04 (5.00) | -1.41 (-1.73 to -1.09) | <.001 | -1.60 (-2.05 to -1.14) | <.001
Table 2, SBP <90 mm Hg block, HLOS days (block total 651; OS row 323 | 9.40 (8.31) | 0 [Reference]):
SAE | 124 | 7.48 (4.80) | -1.65 (-3.24 to -0.06) | .04 | -1.67 (-2.89 to -0.46) | .007
Observation | 204 | 8.74 (8.01) | -0.13 (-1.58 to 1.32) | .86 | -0.48 (-2.00 to 1.04) | .54
Main-text Conclusions:
Furthermore, in patients with hypotension on admission, nonoperative management (either with SAE or observation) was associated with no increased risk of complications or mortality and shorter LOS.
Key Points, "Findings" (the abstract's Conclusions and Relevance uses the same closing phrase):
In this cohort study of 7567 patients, nonoperative management (angioembolization or observation) was associated with favorable outcomes when compared with surgery in isolated severe blunt splenic injury, even in patients with hypotension on admission.
What the paper's own numbers show
Method. I compared the abstract's two "hypotension" estimate-and-CI triples with every HLOS row in Table 2 (total, SBP <90, SBP ≥90 and failure blocks, regression and IPW columns). I then checked each row's internal consistency with a Wald approximation from the printed interval: SE = (upper − lower)/(2 × 1.96), z = β/SE, two-sided P = 2Φ(−|z|). I checked that the block sizes add up, and I compared CI widths with the group sizes. As a further check I computed the unadjusted mean differences and Welch t tests from the Table 2 means and SDs. Patient-level data are not available, so the adjusted models cannot be refitted. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| HLOS β, SAE vs OS, "among patients with hypotension" | Abstract | -1.44 (-1.79 to -1.09), P < .001 | Identical to the Table 2 Total row; matches no SBP <90 row |
| HLOS β, observation vs OS, "among patients with hypotension" | Abstract | -1.41 (-1.73 to -1.09), P < .001 | Identical to the Table 2 Total row; matches no SBP <90 row |
| HLOS β, SAE, hypotension subgroup | Table 2, SBP <90; Results | -1.65 (-3.24 to -0.06), P = .04 | SE 0.811, z = -2.03, P = 0.042: consistent with itself |
| HLOS β, observation, hypotension subgroup | Table 2, SBP <90 | -0.13 (-1.58 to 1.32), P = .86 | SE 0.740, z = -0.18, P = 0.861; CI includes 0 |
| IPW HLOS β, SAE / observation, hypotension subgroup | Table 2, SBP <90, IPW | -1.67 (-2.89 to -0.46), P = .007 / -0.48 (-2.00 to 1.04), P = .54 | P = 0.007 / 0.536; observation CI includes 0 |
| HLOS β, total cohort, SAE / observation | Table 2, Total | -1.44 (-1.79 to -1.09) / -1.41 (-1.73 to -1.09), both P < .001 | z = -8.06 / -8.64, both P < 10⁻¹⁵ |
| HLOS β, normotension subgroup, SAE / observation | Table 2, SBP ≥90 | -1.44 (-1.80 to -1.09) / -1.53 (-1.86 to -1.20) | Close to, but not identical with, the abstract (observation -1.53 vs -1.41) |
| Abstract's failure-subgroup HLOS figures (control) | Abstract vs Table 2, Failure | 2.50 (1.27-3.73); 0.71 (0.07-1.35) | Identical to the Table 2 Failure rows: the abstract's other subgroup result is transcribed correctly |
| Group sizes, HLOS blocks | Table 2 | 7567; 651; 6916 | 1499 + 1547 + 4521 = 7567; 323 + 124 + 204 = 651; 1176 + 1423 + 4317 = 6916; 651 + 6916 = 7567 |
| CI width, abstract vs SBP <90 row | Abstract; Table 2 | 0.70 vs 3.18 (SAE); 0.64 vs 2.90 (observation) | The subgroup intervals are 4.5 times as wide as the abstract's |
| Residual SD implied by each CI, SE/√(1/n_arm + 1/n_OS) | Table 2 | HLOS SD 7.69 (SBP <90), 5.27 (total) | SBP <90 rows: 7.68 and 8.27; total rows: 4.93 and 5.48. The abstract's CIs placed at the SBP <90 group sizes would imply 1.69 and 1.83, so covariates would have to explain about 95% of the HLOS variance in that subgroup, while the table's own subgroup rows show almost no such reduction |
| ICULOS β, observation, hypotension subgroup | Table 2, SBP <90 | -0.49 (-1.59 to 0.61), P = .38 | P = 0.383; CI includes 0 |
| Unadjusted mean HLOS difference vs OS, hypotension subgroup | Table 2 means (SD) | OS 9.40 (8.31), SAE 7.48 (4.80), observation 8.74 (8.01) | SAE -1.92 days (Welch P = 0.003); observation -0.66 days (Welch P = 0.364) |
Each Table 2 row is internally consistent: the printed P values follow from the printed intervals, and the block sizes add to the cohort total. The table is not misaligned. The discrepancy lies between the abstract sentence and the subgroup rows it describes. The CI-width check points the same way. The abstract's intervals are as narrow as one would expect from about 3000-6000 patients per comparison, not from the 447 (SAE) and 527 (observation) patients in the hypotension comparisons.
Figure
Figure 1. The abstract's sentence on patients with hypotension (top, orange outline) quoted next to the two Table 2 blocks it could refer to. All six numbers, for both arms, match the Total-cohort block (n = 7567) exactly. The SBP <90 mm Hg block (n = 651), which is the subgroup the sentence names, gives a different SAE estimate with an interval 4.5 times as wide, and an observation estimate of -0.13 days whose 95% CI includes 0. The blue markers give the P values recomputed from each printed interval with a Wald approximation. They agree with the table's own P values (.04 and .86), so the table rows are internally consistent.
Figure 2. The same estimates on one axis, in days of hospital stay relative to open splenectomy. The orange squares are the abstract's "hypotension" values, the blue circles are the Table 2 SBP <90 rows, and the hollow gray circles are the Table 2 total-cohort rows. The orange and gray intervals coincide. For observation, the subgroup interval (blue) crosses the null line at 0, and the abstract's (orange) does not. The right-hand column gives each printed P value, with the Wald P recomputed from the printed CI for the subgroup rows.
Why it matters
Whether nonoperative management, and observation in particular, is reasonable for hemodynamically unstable patients with high-grade splenic injury is a clinically consequential question. The abstract is the part of the paper most readers rely on. In the hypotension subgroup, Table 2 and the Results show no difference in mortality (HR 0.95 for SAE, 1.08 for observation) or in any complication (OR 0.71, P = .30; OR 1.15, P = .62). That leaves length of stay as the only outcome on which the subgroup shows a benefit. For observation, that benefit is not present in the paper's own subgroup analysis: β = -0.13 days (95% CI, -1.58 to 1.32; P = .86), with IPW giving -0.48 days (P = .54). The unadjusted difference is also small and not significant (-0.66 days, P = .36). The main-text Conclusions ("nonoperative management (either with SAE or observation) was associated with ... shorter LOS") and the Key Points' "favorable outcomes ... even in patients with hypotension on admission" both rest partly on the abstract's version.
For SAE the direction is unchanged, but the abstract overstates the precision. The subgroup estimate is -1.65 days with a CI reaching almost to 0 (-0.06) and P = .04, not P < .001 with a 0.7-day-wide interval.
The paper's other main findings do not depend on this sentence. These are lower morbidity and shorter stay in the overall cohort, the same in the normotension subgroup, no mortality difference, and the failure-subgroup results. A null result does not show that observation has no effect on length of stay in hypotensive patients. It shows only that this analysis does not support the significant effect the abstract reports.
Explanations considered
- The abstract reports a different model or weighting from the table. The abstract's two triples match no other HLOS row in Table 2: not the SBP <90 regression or IPW rows, the SBP ≥90 rows, or the total-cohort IPW rows. They match only the total-cohort regression rows, digit for digit. This does not account for the numbers.
- "Patients with hypotension" means something other than SBP <90 mm Hg. The Methods define the subgroup as "patients presenting with hypotension (SBP <90 mm Hg)", and the total cohort is not a hypotension group. The total cohort is 91% normotensive (6916 of 7567). This does not account for the numbers.
- The table rows are misaligned or mislabeled. Each row's printed P value follows from its own interval (SBP <90 SAE P = 0.042 vs .04 printed; observation 0.861 vs .86; IPW 0.007 and 0.536). The block sizes sum correctly (323 + 124 + 204 = 651). The SBP <90 intervals are as wide as their group sizes and SDs imply. The abstract's intervals would require covariates to remove about 95% of the outcome variance in that subgroup, far more than the table's own subgroup rows show. The Results text also quotes -1.65 for the hypotension subgroup, which confirms the table's SBP <90 values. This does not account for the numbers.
- The supplement holds a different hypotension analysis. Supplement 1 (eAppendix and eTables 1-8) gives only unadjusted HLOS medians for the hypotension subgroup (eTable 4: OS 7, SAE 6, observation 6; three-group P = .003). It reports no pairwise regression coefficients and nothing matching -1.44 or -1.41 for that subgroup. This does not account for the numbers.
- A text-conversion artifact. The abstract sentence, the Results paragraph and Table 2 read the same in the publisher-supplied JATS XML. The values are plain text, not images. This does not account for the numbers.
- The abstract sentence is meant to combine two findings: no mortality difference in hypotension, and shorter stay overall. The sentence attaches the length-of-stay result to "Among patients with hypotension" grammatically. The main-text Conclusions repeat the subgroup claim explicitly for both SAE and observation. Under this reading the abstract would still say nothing correct about length of stay in the subgroup. It would at most reduce the error to ambiguous wording, and the Conclusions sentence would still be unsupported for observation.
Status
As of 2026-10-23:
- Version of record. The publisher page (jamanetwork.com, reached via doi.org) returned a bot check (HTTP 403) to automated access, so it was not read directly. The wording was confirmed in the publisher-supplied JATS full text in Europe PMC (PMC12457976) and in the PMC archive text (read with
src/show.py). Both have the abstract sentence, the Results paragraph and the Table 2 cells as quoted. - Supplements. Supplement 1 (eAppendix, eTables 1-8) was read. It does not report the hypotension-subgroup HLOS regression coefficients and does not change Table 2. Supplement 2 is the data sharing statement (ACS-TQIP registry data, no patient-level data).
- Corrections and notices. None found. Crossref update records, PMC notices and Europe PMC notices for doi:10.1001/jamanetworkopen.2025.33266 are all empty. A web search for a correction or erratum returned none.
- PubPeer. The PubPeer search and API returned a human-verification challenge to automated access, so they could not be queried directly. A web search for the title returned no PubPeer entry. This is not a definitive negative.
- Sources. Article (doi); PMC12457976; Europe PMC full-text XML and supplementary files for PMC12457976; Crossref and Europe PMC REST records for the DOI.
Suggested correction
In the Abstract, Results, replace the hypotension sentence with the Table 2 SBP <90 mm Hg values, for example: "Among patients with hypotension, there was no mortality difference; hospital length of stay was shorter in the SAE group (β = -1.65; 95% CI, -3.24 to -0.06; P = .04) but not significantly different in the observation group (β = -0.13; 95% CI, -1.58 to 1.32; P = .86)." The main-text Conclusions should then be revised: in patients with hypotension, shorter LOS was found for SAE only. The phrase "even in patients with hypotension on admission" in the Key Points and in the Abstract's Conclusions and Relevance should be checked against the subgroup results as corrected. In that subgroup, mortality and complications did not differ, and the LOS benefit applies to SAE only.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12457976 (JAMA Netw Open 2025; doi:10.1001/jamanetworkopen.2025.33266)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates the figures
# (fig1, fig2 and three alternative designs, candidate3-5) as SVG and 200-dpi PNG
# in this script's own folder. Every input below is copied from the article as
# archived in PubMed Central (abstract and Table 2).
import os
import math
import numpy as np
from scipy.stats import norm, ttest_ind_from_stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs
# Abstract, Results: "Among patients with hypotension ... SAE (...) and observation (...)"
ABSTRACT = {"SAE": (-1.44, -1.79, -1.09, "<.001"),
"Observation": (-1.41, -1.73, -1.09, "<.001")}
# Table 2, HLOS days rows: (n, mean, SD, beta, lo, hi, P, IPW beta, IPW lo, IPW hi, IPW P)
T2 = {
"Total": {"OS": (1499, 8.38, 6.53),
"SAE": (1547, 6.35, 4.17, -1.44, -1.79, -1.09, "<.001", -1.77, -2.24, -1.30, "<.001"),
"Observation": (4521, 6.04, 5.00, -1.41, -1.73, -1.09, "<.001", -1.60, -2.05, -1.14, "<.001")},
"SBP <90": {"OS": (323, 9.40, 8.31),
"SAE": (124, 7.48, 4.80, -1.65, -3.24, -0.06, ".04", -1.67, -2.89, -0.46, ".007"),
"Observation": (204, 8.74, 8.01, -0.13, -1.58, 1.32, ".86", -0.48, -2.00, 1.04, ".54")},
"SBP >=90": {"OS": (1176, 8.10, 5.93),
"SAE": (1423, 6.25, 4.10, -1.44, -1.80, -1.09, "<.001", -1.80, -2.31, -1.28, "<.001"),
"Observation": (4317, 5.91, 4.77, -1.53, -1.86, -1.20, "<.001", -1.83, -2.30, -1.35, "<.001")},
"Failure": {"OS": (1499, 8.38, 6.53),
"SAE": (105, 10.76, 6.07, 2.50, 1.27, 3.73, "<.001", 2.44, 1.14, 3.75, "<.001"),
"Observation": (608, 8.88, 6.96, 0.71, 0.07, 1.35, ".03", 1.20, 0.19, 2.21, ".02")},
}
# Table 2, SBP <90, ICULOS days (used for the Conclusions' "shorter LOS")
ICU_HYPO = {"SAE": (-1.45, -2.62, -0.29, ".01"), "Observation": (-0.49, -1.59, 0.61, ".38")}
SUBGROUP_HLOS_N = {"Total": 7567, "SBP <90": 651, "SBP >=90": 6916}
SUBGROUP_HLOS_SD = {"Total": 5.27, "SBP <90": 7.69, "SBP >=90": 4.93}
def wald(b, lo, hi):
se = (hi - lo) / (2 * Z975)
z = b / se
return se, z, 2 * norm.sf(abs(z))
def fmt_p(p):
return f"{p:.2g}" if p < 0.001 else f"{p:.3f}"
# ---------------------------------------------------------------- 1. Wald check of every row
print("1. Wald check, SE = (upper - lower) / (2 x 1.96), z = beta / SE, two-sided P")
rows = [("Abstract 'hypotension'", a, *ABSTRACT[a]) for a in ("SAE", "Observation")]
for g in T2:
for a in ("SAE", "Observation"):
r = T2[g][a]
rows.append((f"Table 2 {g} MLR", a, r[3], r[4], r[5], r[6]))
rows.append((f"Table 2 {g} IPW", a, r[7], r[8], r[9], r[10]))
for a in ("SAE", "Observation"):
rows.append(("Table 2 SBP <90 ICULOS MLR", a, *ICU_HYPO[a]))
WALD = {}
for src, arm, b, lo, hi, p in rows:
se, z, pc = wald(b, lo, hi)
WALD[(src, arm)] = (se, z, pc)
print(f" {src:28s} {arm:11s} beta={b:6.2f} CI=({lo:5.2f},{hi:5.2f}) printed P={p:6s} "
f"SE={se:.3f} z={z:6.2f} recomputed P={fmt_p(pc):8s} CI excludes 0: {lo * hi > 0}")
# ---------------------------------------------------------------- 2. which block do the abstract numbers match?
print("\n2. Does the abstract's 'hypotension' triple (beta, lower, upper) equal a Table 2 HLOS row?")
for g in T2:
for col, idx in (("MLR", (3, 4, 5)), ("IPW", (7, 8, 9))):
m = [tuple(T2[g][a][i] for i in idx) == ABSTRACT[a][:3] for a in ("SAE", "Observation")]
print(f" {g:9s} {col}: SAE match={m[0]!s:5s} Observation match={m[1]}")
print(" (control) Abstract failure-subgroup HLOS: SAE 2.50 (1.27-3.73), observation 0.71 (0.07-1.35)"
f" = Table 2 Failure rows: {(T2['Failure']['SAE'][3:6] == (2.50, 1.27, 3.73)) and (T2['Failure']['Observation'][3:6] == (0.71, 0.07, 1.35))}")
# ---------------------------------------------------------------- 3. sample sizes
print("\n3. Sample sizes of the HLOS blocks")
for g in ("Total", "SBP <90", "SBP >=90"):
s = sum(T2[g][a][0] for a in ("OS", "SAE", "Observation"))
print(f" {g:9s}: {T2[g]['OS'][0]} + {T2[g]['SAE'][0]} + {T2[g]['Observation'][0]} = {s} "
f"(printed {SUBGROUP_HLOS_N[g]})")
print(f" 651 + 6916 = {651 + 6916}")
for g in ("Total", "SBP <90"):
for a in ("SAE", "Observation"):
print(f" patients in the {g} {a} vs OS comparison: {T2[g][a][0]} + {T2[g]['OS'][0]} = {T2[g][a][0] + T2[g]['OS'][0]}")
print(f" normotensive share of the total cohort: 6916 / 7567 = {6916 / 7567:.1%}")
# ---------------------------------------------------------------- 4. CI widths
print("\n4. CI widths: abstract vs the hypotension rows")
CIW = {}
for a in ("SAE", "Observation"):
wa = ABSTRACT[a][2] - ABSTRACT[a][1]
r = T2["SBP <90"][a]
wh = r[5] - r[4]
CIW[a] = (wa, wh)
print(f" {a:11s}: abstract width {wa:.2f}; Table 2 SBP <90 width {wh:.2f}; ratio {wh / wa:.1f}")
# unadjusted SE of a difference in means and the residual SD each CI implies
print("\n5. Residual SD implied by each CI, SE / sqrt(1/n_arm + 1/n_OS) (rough: ignores covariates' effect on precision)")
IMPLIED = {}
for g in ("Total", "SBP <90", "SBP >=90"):
for a in ("SAE", "Observation"):
r, os_ = T2[g][a], T2[g]["OS"]
k = math.sqrt(1 / r[0] + 1 / os_[0])
se = wald(r[3], r[4], r[5])[0]
IMPLIED[(g, a)] = (k, se, se / k)
print(f" Table 2 {g:9s} {a:11s}: k={k:.4f} SE={se:.3f} implied residual SD={se / k:.2f} "
f"(block SD of HLOS {SUBGROUP_HLOS_SD[g]})")
for a in ("SAE", "Observation"):
r, os_ = T2["SBP <90"][a], T2["SBP <90"]["OS"]
k = math.sqrt(1 / r[0] + 1 / os_[0])
se = wald(*ABSTRACT[a][:3])[0]
sd_raw = math.sqrt(((r[0] - 1) * r[2] ** 2 + (os_[0] - 1) * os_[2] ** 2) / (r[0] + os_[0] - 2))
r2 = 1 - (se / k) ** 2 / sd_raw ** 2
IMPLIED[("Abstract", a)] = (k, se, se / k)
print(f" Abstract CI placed in the SBP <90 subgroup, {a:11s}: k={k:.4f} SE={se:.3f} implied residual "
f"SD={se / k:.2f}; pooled SD of the two arms {sd_raw:.2f}; covariates would need R^2={r2:.2f}")
# ---------------------------------------------------------------- 6. unadjusted differences
print("\n6. Unadjusted mean HLOS difference vs OS (Table 2 means and SDs), Welch t test")
for g in ("SBP <90", "Total"):
os_ = T2[g]["OS"]
for a in ("SAE", "Observation"):
r = T2[g][a]
t, p = ttest_ind_from_stats(r[1], r[2], r[0], os_[1], os_[2], os_[0], equal_var=False)
print(f" {g:7s} {a:11s}: {r[1]:.2f} - {os_[1]:.2f} = {r[1] - os_[1]:+.2f} days; Welch t={t:.2f}, P={fmt_p(p)}")
# ================================================================ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "lines.linewidth": 2, "svg.fonttype": "none"})
M = "−" # minus sign
def n(x, d=2):
return f"{x:.{d}f}".replace("-", M)
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
plt.close(fig)
print(f" wrote {name}.svg and {name}.png")
def box(ax, x, y, w, h, edge, lw=2):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc="#ffffff", ec=edge, lw=lw, transform=ax.transAxes))
print("\n7. Figures")
pS, pO = (T2["SBP <90"]["SAE"], T2["SBP <90"]["Observation"])
tS, tO = (T2["Total"]["SAE"], T2["Total"]["Observation"])
wS = WALD[("Table 2 SBP <90 MLR", "SAE")]
wO = WALD[("Table 2 SBP <90 MLR", "Observation")]
# ---------------- fig1: text panels (design 1)
fig = plt.figure(figsize=(7.2, 4.8))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.autoscale(False)
box(ax, 0.02, 0.72, 0.96, 0.26, ORANGE)
ax.text(0.04, 0.95, "Abstract, Results", fontsize=11, weight="bold", va="top")
ax.text(0.04, 0.89,
"\u201cAmong patients with hypotension, there was no mortality difference, but shorter\n"
"hospital length of stay was found in the SAE (\u03b2 = " + n(-1.44) + "; 95% CI, " + n(-1.79) + " to " + n(-1.09) +
"; P < .001)\nand observation (\u03b2 = " + n(-1.41) + "; 95% CI, " + n(-1.73) + " to " + n(-1.09) +
"; P < .001) groups.\u201d", fontsize=10.5, va="top", linespacing=1.45)
for x0, title, lines in (
(0.02, "Table 2, Total cohort, n = 7567",
[("SAE", f"{n(tS[3])} ({n(tS[4])} to {n(tS[5])}), P < .001", None),
("Observation", f"{n(tO[3])} ({n(tO[4])} to {n(tO[5])}), P < .001", None)]),
(0.52, "Table 2, SBP <90 mm Hg, n = 651",
[("SAE", f"{n(pS[3])} ({n(pS[4])} to {n(pS[5])}), P = .04", f"recomputed P = {wS[2]:.3f}"),
("Observation", f"{n(pO[3])} ({n(pO[4])} to {n(pO[5])}), P = .86",
f"recomputed P = {wO[2]:.2f}; CI includes 0")])):
box(ax, x0, 0.095, 0.46, 0.395, GRAY)
ax.text(x0 + 0.02, 0.465, title, fontsize=11, weight="bold", va="top")
ax.text(x0 + 0.02, 0.41, "HLOS \u03b2 vs open splenectomy, days (95% CI)", fontsize=10, color=INK2, va="top")
for i, (arm, est, rec) in enumerate(lines):
y = 0.355 - i * 0.13
ax.text(x0 + 0.02, y, arm, fontsize=10.5, va="top", weight="bold")
ax.text(x0 + 0.02, y - 0.042, est, fontsize=10.5, va="top")
if rec:
ax.plot([x0 + 0.03], [y - 0.1], "o", ms=8, color=BLUE, clip_on=False)
ax.text(x0 + 0.05, y - 0.1, rec, fontsize=10, color=INK2, va="center")
# connectors
ax.annotate("", xy=(0.25, 0.495), xytext=(0.25, 0.715), xycoords="data",
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=2))
ax.text(0.27, 0.605, "identical, digit for digit\n(all six numbers)", fontsize=10, va="center", color=INK)
ax.annotate("", xy=(0.75, 0.495), xytext=(0.75, 0.715), xycoords="data",
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, linestyle=(0, (4, 3))))
ax.text(0.77, 0.605, "the subgroup the sentence\nnames: numbers differ", fontsize=10, va="center", color=INK)
# legend
h = [Line2D([], [], color=ORANGE, lw=2, label="claim as reported in the abstract"),
Line2D([], [], marker="o", ms=8, color=BLUE, lw=0, label="P recomputed from the printed 95% CI")]
ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.005), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# ---------------- fig2: forest plot on one axis (design 2)
def forest(ax, entries, xlim):
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for y, (lab, b, lo, hi, style, note) in entries.items():
col = {"abs": ORANGE, "sub": BLUE, "tot": GRAY}[style]
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="round", zorder=2)
ax.plot([b], [y], "s" if style == "abs" else "o", ms=9, color=col,
mfc=col if style != "tot" else BG, mew=2, zorder=3)
ax.text(xlim[1] + 0.08, y, note, va="center", fontsize=10, color=INK, clip_on=False)
ax.set_yticks(list(entries.keys()), [e[0] for e in entries.values()], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlim(*xlim)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
fig, ax = plt.subplots(figsize=(7.2, 4.3))
ent = {}
y = 7.6
for arm, sub, tot in (("SAE", pS, tS), ("Observation", pO, tO)):
a = ABSTRACT[arm]
w = WALD[("Table 2 SBP <90 MLR", arm)]
ent[y] = (f"{arm}: abstract, ‘hypotension’", a[0], a[1], a[2], "abs", "P < .001")
ent[y - 1] = (f"{arm}: Table 2, SBP <90 (n = 651)", sub[3], sub[4], sub[5], "sub",
f"P = {sub[6]} (recomputed {w[2]:.3f})")
ent[y - 2] = (f"{arm}: Table 2, total (n = 7567)", tot[3], tot[4], tot[5], "tot", "P < .001")
y -= 4
forest(ax, ent, (-3.6, 1.6))
ax.set_ylim(0.9, 8.3)
ax.set_xlabel("Adjusted difference in hospital length of stay vs open splenectomy, days (β, 95% CI)")
ax.text(0.05, 8.25, "longer stay →", fontsize=10, color=INK2, va="bottom")
ax.text(-0.05, 8.25, "← shorter stay", fontsize=10, color=INK2, va="bottom", ha="right")
h = [Line2D([], [], marker="s", ms=9, color=ORANGE, lw=2, label="abstract (reported)"),
Line2D([], [], marker="o", ms=9, color=BLUE, lw=2, label="Table 2 hypotension rows, P recomputed from CI"),
Line2D([], [], marker="o", ms=9, color=GRAY, mfc=BG, mew=2, lw=2, label="Table 2 total cohort")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.32, -0.16), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
# ---------------- candidate3: z statistics on one axis (design 3)
fig, ax = plt.subplots(figsize=(7.2, 3.3))
ax.axvspan(-1.96, 1.96, color="#efeeea", zorder=0)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0, 4.75, "|z| < 1.96: P > .05", ha="center", fontsize=10, color=INK2,
bbox=dict(fc="#efeeea", ec="none", pad=1))
yl = []
for i, arm in enumerate(("SAE", "Observation")):
za = WALD[("Abstract 'hypotension'", arm)][1]
zs = WALD[("Table 2 SBP <90 MLR", arm)][1]
y0 = 3.6 - i * 2.2
ax.plot([za], [y0], "s", ms=10, color=ORANGE)
ax.text(za + 0.25, y0, f"z = {n(za)}", va="center", fontsize=10)
ax.plot([zs], [y0 - 0.8], "o", ms=10, color=BLUE)
ax.text(zs + 0.3, y0 - 0.8, f"z = {n(zs)}, P = {WALD[('Table 2 SBP <90 MLR', arm)][2]:.2f}", va="center", fontsize=10,
bbox=dict(fc="#efeeea", ec="none", pad=1), zorder=4)
yl += [(y0, f"{arm}: abstract"), (y0 - 0.8, f"{arm}: Table 2, SBP <90")]
ax.set_yticks([a for a, _ in yl], [b for _, b in yl], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlim(-9.5, 4.5); ax.set_ylim(0.3, 5.1)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Wald z = β / SE, with SE = (upper − lower) / 3.92 from each printed 95% CI")
h = [Line2D([], [], marker="s", ms=10, color=ORANGE, lw=0, label="from the abstract's CI (reported)"),
Line2D([], [], marker="o", ms=10, color=BLUE, lw=0, label="from the Table 2 hypotension CI (recomputed)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False, fontsize=10)
save(fig, "candidate3")
# ---------------- candidate4: lookup grid (design 4)
fig = plt.figure(figsize=(7.2, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.autoscale(False)
cols = [0.03, 0.36, 0.68]
ax.text(cols[0], 0.93, "Table 2 HLOS row", weight="bold", fontsize=10.5)
ax.text(cols[1], 0.93, "SAE β (95% CI)", weight="bold", fontsize=10.5)
ax.text(cols[2], 0.93, "Observation β (95% CI)", weight="bold", fontsize=10.5)
ax.text(cols[0], 0.82, "Abstract, ‘hypotension’", fontsize=10.5, color=INK)
for c, arm in ((cols[1], "SAE"), (cols[2], "Observation")):
a = ABSTRACT[arm]
ax.plot([c - 0.012], [0.835], "s", ms=8, color=ORANGE, transform=ax.transAxes)
ax.text(c + 0.005, 0.82, f"{n(a[0])} ({n(a[1])} to {n(a[2])})", fontsize=10.5)
ax.plot([0.02, 0.98], [0.78, 0.78], color=INK2, lw=0.8, transform=ax.transAxes)
y = 0.70
for g, lab in (("Total", "Total, n = 7567"), ("SBP <90", "SBP <90, n = 651"), ("SBP >=90", "SBP ≥90, n = 6916")):
for col, idx, cl in (("MLR", (3, 4, 5), "regression"), ("IPW", (7, 8, 9), "IPW")):
ax.text(cols[0], y, f"{lab}, {cl}", fontsize=10.5, color=INK if g != "SBP <90" else INK, weight="bold" if g == "SBP <90" else "normal")
for c, arm in ((cols[1], "SAE"), (cols[2], "Observation")):
r = T2[g][arm]
tri = tuple(r[i] for i in idx)
same = tri == ABSTRACT[arm][:3]
ax.text(c + 0.005, y, f"{n(tri[0])} ({n(tri[1])} to {n(tri[2])})", fontsize=10.5)
if same:
ax.add_patch(FancyBboxPatch((c - 0.01, y - 0.025), 0.28, 0.075, boxstyle="round,pad=0,rounding_size=0.01",
fc="none", ec=ORANGE, lw=2, transform=ax.transAxes))
y -= 0.1
ax.text(0.03, 0.06, "Orange outline: identical to the abstract\u2019s \u2018hypotension\u2019 numbers. Only the Total-cohort\n"
"regression rows match; the SBP <90 rows (bold), which the sentence describes, do not.", fontsize=10, color=INK2)
save(fig, "candidate4")
# ---------------- candidate5: CI width vs sample size (design 5)
fig, ax = plt.subplots(figsize=(7.2, 4.0))
ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
labels = {("Total", "SAE"): "Total SAE", ("Total", "Observation"): "Total Obs",
("SBP >=90", "SAE"): "SBP≥90 SAE", ("SBP >=90", "Observation"): "SBP≥90 Obs",
("SBP <90", "SAE"): "SBP<90 SAE", ("SBP <90", "Observation"): "SBP<90 Obs"}
off = {("Total", "SAE"): (6, 6), ("Total", "Observation"): (6, -12), ("SBP >=90", "SAE"): (-70, 6),
("SBP >=90", "Observation"): (-70, -12), ("SBP <90", "SAE"): (8, 0), ("SBP <90", "Observation"): (8, -4)}
for key, lab in labels.items():
k, se, _ = IMPLIED[key]
ax.plot([k], [2 * Z975 * se], "o", ms=9, color=BLUE, zorder=3)
if key[0] == "SBP <90":
ax.annotate(lab, (k, 2 * Z975 * se), xytext=off[key], textcoords="offset points", fontsize=10, va="center")
ax.annotate("Total and SBP \u226590 rows\n(SAE and observation, 4 points)", (0.035, 0.72), xytext=(0.035, 1.35),
ha="center", fontsize=10, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
for arm, dy in (("SAE", 8), ("Observation", -12)):
k, se, _ = IMPLIED[("Abstract", arm)]
ax.plot([k], [2 * Z975 * se], "s", ms=9, color=ORANGE, zorder=3)
ax.annotate(f"abstract ‘hypotension’ {arm}", (k, 2 * Z975 * se), xytext=(8, dy),
textcoords="offset points", fontsize=10, va="center")
ax.set_xlim(0, 0.13); ax.set_ylim(0, 3.5)
ax.set_xlabel(r"$\sqrt{1/n_{\rm arm} + 1/n_{\rm OS}}$, from the Table 2 group sizes")
ax.set_ylabel("Width of the printed 95% CI, days")
for s in ("top", "right"):
ax.spines[s].set_visible(False)
h = [Line2D([], [], marker="o", ms=9, color=BLUE, lw=0, label="Table 2 HLOS rows (width from the printed CI)"),
Line2D([], [], marker="s", ms=9, color=ORANGE, lw=0, label="abstract CI, placed at the hypotension group sizes")]
ax.legend(handles=h, loc="upper left", frameon=False, fontsize=10)
save(fig, "candidate5")
The adjusted odds ratio for type 2 diabetes lies outside its own reported 95% CI
JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.40818 · PMC12579343
Summary
The paper's one significant adjusted result is the association of polycystic ovarian morphology (PCOM) with non–insulin-dependent diabetes (NIDD). The abstract, the Results text and Table 3 all print it the same way: aOR 2.92, 95% CI 1.06–2.82, P = .02. The point estimate is larger than the upper confidence limit, so the estimate and the interval cannot both be right. The paper's own numbers allow two quite different repairs: an adjusted OR of about 1.73 if the interval is right, or an interval of about 1.06–8.04 if the estimate is right. Either way, readers cannot tell from the paper how large or how precise the key adjusted effect is. The direction and nominal significance hold under both readings.
What the paper reports
Abstract, Results:
Multivariate analysis confirmed the association of PCOM with non–insulin-dependent diabetes (aOR, 2.92; 95% CI, 1.06-2.82; P = .02).
Results, section "Logistic Regression Analysis":
The multivariate analysis indicated a significantly higher risk of NIDD in women with PCOM, with an aOR of 2.92 (95% CI, 1.06-2.82; P = .02).
Table 3 ("Logistic Regression Analysis of Chronic Morbidity and Mortality Among Women With PCOM"), NIDD row, under the column "aOR vs women without PCOM (95% CI)" and "P value":
NIDD | 2.92 (1.06-2.82) | .02
Table 3, footnote a:
All models were adjusted for age at enrollment; body mass index (calculated as weight in kilograms divided by height in meters squared); fasting insulin, total cholesterol, and triglyceride levels; and follow-up duration.
Table 2, NIDD row (PCOM, n = 189 | non-PCOM, n = 151 | P value):
NIDD | 45 (23.80) | 14 (9.27) | <.001
Methods, "Statistical Analysis":
P values presented in the logistic regression analysis are adjusted for multiple comparisons using the false discovery rate method. A post hoc Bonferroni correction was applied to the primary outcome (non–insulin-dependent diabetes [NIDD]) to confirm robustness of the finding.
Discussion, "Strengths and Limitations":
It could be claimed that the statistically significant finding in NIDD was due to multiple analyses being performed; however, even when adjusting for multiple comparisons using the Bonferroni correction, this outcome remained significant.
What the paper's own numbers show
Method. In a logistic regression the 95% Wald interval for an odds ratio is exp(b ± 1.96·SE). The estimate must therefore lie inside the interval and equal its geometric midpoint, √(lower × upper). Given any two of estimate, lower limit, upper limit and P, the other two follow. Profile-likelihood and other exact intervals are not exactly log-symmetric, but they still always contain the maximum-likelihood estimate. To check that the paper's intervals are of this kind, I applied the same test to all 27 estimated rows of Table 3. I also computed the crude odds ratio from the Table 2 counts. It is not an adjusted estimate and cannot replace one, but it shows which range is plausible. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Is the NIDD aOR inside its CI? | Abstract; Results; Table 3 | 2.92 (1.06–2.82) | No. 2.92 exceeds the upper limit by 0.10. Rounding to 2 decimals leaves a gap of at least 0.09 |
| Geometric midpoint of the NIDD CI | Table 3 | estimate 2.92 | 1.73 |
| Reading A: CI 1.06–2.82 correct | Table 3 | aOR 2.92, P = .02 | aOR 1.73, SE(log OR) 0.250, z = 2.19, P = .028 (.027–.030 over the rounding range of both limits). Excess odds 73% instead of 192% |
| Reading B: aOR 2.92 and lower limit 1.06 correct | Table 3 | upper limit 2.82 | upper limit 8.04, SE 0.517, z = 2.07, P = .038 |
| Reading C: aOR 2.92 and upper limit 2.82 correct | Table 3 | lower limit 1.06 | lower limit 3.02, which is above the upper limit, so this reading is impossible |
| CI implied by aOR 2.92 with P = .02 | Table 3 | 1.06–2.82 | 1.18–7.20 (1.14–7.45 for P = .025) |
| Estimate implied by upper limit 2.82 with P = .02 | Table 3 | 2.92 | 1.76, with lower limit 1.09 |
| Crude NIDD OR, 45/189 vs 14/151 | Table 2 | 23.80% vs 9.27%, P < .001 | 23.81% vs 9.27%; OR 3.06 (Wald 95% CI 1.61–5.82); χ² = 12.37, P = .0004 |
| Other Table 3 rows: estimate ÷ CI midpoint | Table 3 | — | 0.91 to 1.33; 21 of 25 within 5%. The largest departures are rows with a printed lower limit of 0.01–0.02 |
| Other Table 3 rows: printed P vs Wald P from estimate and CI | Table 3 | — | median absolute difference 0.007 (maximum 0.035) |
| Hypothyroidism aOR (another row) | Table 3 | 0.55 (0.30–3.00), P = .052 | CI midpoint 0.95. If the estimate and lower limit are right, the upper limit is 1.01 with P = .053, matching the printed P |
| Time in study, mean difference | Table 2, footnote a | −1.42 (95% CI −6.25 to −2.22) | Interval does not contain −1.42. From the means and SDs, Welch 95% CI −2.28 to −0.56 |
| Age at death, mean difference | Table 2, footnote a; P in Table 2 and abstract | −16.58 (−35.77 to 2.60), P = .08 | From 54.25 (11.52), n = 7 vs 70.83 (13.64), n = 9: Welch CI −30.09 to −3.07, P = .020. The printed CI implies an SE of 8.94, against 6.30 from these SDs and counts |
| Age at follow-up, mean difference (control) | Table 2, footnote a | −5.96 (−7.86 to −4.06) | −5.96 (Welch −7.87 to −4.05): consistent |
The NIDD interval is the only one in Table 3 that excludes its own estimate. The other 25 rows sit on or near the line where estimate equals interval midpoint (Figure 2), so the paper's intervals behave as expected. The NIDD row is not.
The available evidence does not pick one reading. The crude OR (3.06) is close to 2.92, which favours Reading B. The printed P value, however, fits the printed interval better than the printed estimate. If it is a raw P, the interval alone implies P = .028, while the estimate with the lower limit implies P = .038. If it is FDR-adjusted, as the Methods state, the raw P behind it must be below .025, since adjustment can only raise a P value. Neither reading meets that exactly.
Figure
Figure 1. Odds ratio for non–insulin-dependent diabetes in women with vs without PCOM, on a log scale. The orange row is the adjusted estimate and 95% CI exactly as printed in the abstract, the Results text and Table 3: the dot (2.92) lies outside the bar (1.06–2.82). The blue rows are recomputed from the paper's own numbers. A keeps the printed interval and places the estimate at its geometric midpoint (1.73, P = .028). B keeps the printed estimate and lower limit and gives the log-symmetric upper limit (8.04, P = .038). The open circle is the unadjusted OR from the Table 2 counts (3.06, 1.61–5.82), shown for scale only. The thin vertical line marks OR = 1.
Figure 2. Each point is one outcome in Table 3. The x-axis is the printed aOR and the y-axis is the geometric midpoint of its printed 95% CI, both on log scales. For a standard logistic-regression interval the point lies on the diagonal. Twenty-five rows (open blue circles) lie on or near it; the largest departures, below the line at the lower left, are rows whose lower limits are printed as 0.01 or 0.02 and are therefore heavily rounded. The two orange points are off the line: NIDD (estimate 2.92, midpoint 1.73) and hypothyroidism (estimate 0.55, midpoint 0.95). This shows that the NIDD interval is the exception in a table whose intervals are otherwise centred on their estimates.
Why it matters
- It is the paper's key adjusted result, and it is printed in the abstract. The Key Points and the Conclusions describe PCOM as associated with an increased risk of type 2 diabetes. After adjustment for age, BMI, fasting insulin, lipids and follow-up, the only support for this is the NIDD aOR. Readers citing "aOR 2.92 (1.06–2.82)" are citing a pair of numbers that cannot both be correct.
- The size and precision of the effect cannot be recovered. Under Reading A, the adjusted excess odds are about 73% (OR 1.73), well under half the printed 192%. Under Reading B, the estimate stands but the interval runs from about 1 to 8, so the data are far less precise than the printed interval suggests. These two readings would lead to different conclusions in a meta-analysis, in a risk-communication setting or in a sample-size calculation.
- The conclusion that the association exists is not overturned. In both readings the lower limit stays at about 1.06 and the unadjusted P stays below .05, and the crude counts (23.8% vs 9.3%) point the same way. The error concerns how large and how precise the adjusted association is, not whether there is one.
- The robustness claim cannot be checked. The paper says the NIDD result "remained significant" after Bonferroni correction but does not state the number of comparisons. Taking the printed .02 as a raw P, a Bonferroni-corrected P stays below .05 for at most 2 comparisons (at most 3 if the true value is .015). At the raw P values of Readings A and B (.028, .038) it does so only for a single comparison. Table 3 reports 27 estimated outcomes. Until the correct estimate, interval and P are known, the claim cannot be verified.
Explanations considered
- Rounding. Ruled out. On two decimals the estimate is at least 2.915 and the upper limit at most 2.825, so the estimate exceeds the upper limit by at least 0.09.
- Misaligned table or conversion artifact. Ruled out. The same string, "2.92 (1.06-2.82)", appears in the abstract, the Results text and Table 3. In the article XML it sits in a single table cell next to the NIDD label, and the publisher's HTML carries the same values. No minus signs, superscripts or decimal marks are involved.
- An asymmetric (profile-likelihood, exact or Firth) interval. Does not explain it. Such intervals can be asymmetric on the log scale, but any likelihood-based interval contains the maximum-likelihood estimate. Every other row of Table 3 is centred on its estimate as closely as rounding allows (Figure 2).
- A bootstrap or other resampling interval. Not described anywhere in the paper. The Methods describe multivariable logistic regression only. A percentile interval excluding its own estimate would require extreme bias, and the other 25 rows give no sign of it.
- Estimate and interval from different models (for example, a crude estimate with an adjusted interval). Not supported by the numbers. The crude OR from the paper's counts is 3.06, not 2.92, and its interval (1.61–5.82) matches neither printed limit. The inverse of the printed estimate (1/2.92 = 0.34) does not fit the interval either.
- A different confidence level. Does not explain it. An interval at any level must contain the estimate.
- The P value as a tiebreaker. It does not settle the question. The printed P = .02 fits the printed interval (P = .028) better than it fits the printed estimate with the lower limit (P = .038). In the other 25 rows the printed P values match the unadjusted Wald P values computed from their own estimates and intervals (median difference 0.007), although the Methods describe FDR-adjusted P values. Which number is wrong can be decided only from the regression output.
- A correction already issued. None found (see Status).
Status
- Version of record. The DOI resolves to the publisher's page (jamanetwork.com, article 2840789, published online October 31, 2025). Its Abstract, its Results section "Logistic Regression Analysis" and its Table 3 carry "2.92 (1.06-2.82)" with P = .02 (rechecked on 2026-10-24). The PMC archive, the Europe PMC full-text XML and the PubMed abstract (PMID 41171273) carry the same values. Every quotation above was checked verbatim against the PMC text with
src/show.py. - Supplements. The only supplement (Supplement 1,
jamanetwopen-e2540818-s001.pdf) is a one-page Data Sharing Statement ("Data available: No"). No regression output, source data or extra tables are provided, so the supplement does not resolve the question. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or comment.
src/known.pyreturns no Crossref update relations and no PMC or Europe PMC notices. The Crossref record has empty update-to and updated-by fields. The PubMed record (last revised 2026-07-21) lists no CommentsCorrections entries. The publisher page shows no correction notice. Web searches for an "Error in" notice on this article found only unrelated JAMA Network Open corrections. Not fully checked: the Europe PMC search API returned HTTP 503 on repeated attempts, and PubPeer could not be searched (HTTP 403 on the search page and a Cloudflare block on the API). - Sources. PMC article PMC12579343; Europe PMC full-text XML and supplementary files; PubMed PMID 41171273; Crossref; https://doi.org/10.1001/jamanetworkopen.2025.40818 (jamanetwork.com/journals/jamanetworkopen/fullarticle/2840789).
Suggested correction
The authors should check the regression output for the NIDD model and correct the estimate, the interval, or both, in all three places: the abstract, the Results paragraph "Logistic Regression Analysis" and Table 3. In each, replace
aOR, 2.92; 95% CI, 1.06-2.82; P = .02
with the values from that output, so that the estimate lies inside its interval. The paper's numbers fit, for example, either about 1.73 (1.06–2.82) or about 2.92 (1.06–8.04), but only the original output can say which. If the correction changes the P value, the Bonferroni statement in the Discussion should be rechecked, giving the number of comparisons used.
While correcting this, the authors may also wish to check the other entries above:
- the hypothyroidism row of Table 3, "0.55 (0.30-3.00)", where an upper limit of about 1.01 would match the printed P = .052;
- the time-in-study mean difference in Table 2 footnote a, "-1.42 years (95% CI, -6.25 to -2.22 years)", which the means and SDs put at about −2.28 to −0.56;
- the age-at-death difference, "-16.58 years (95% CI, -35.77 to 2.60 years)", P = .08. From the printed means, SDs and death counts (7 and 9), this is about −30.09 to −3.07, P = .02. The two agree only if the ages at death were known for fewer women than died, which the paper would then need to state.
Finally, the authors could state whether the Table 3 P values are FDR-adjusted, since they match unadjusted Wald P values.
Recomputation script (python recompute.py.txt)
"""Recompute the adjusted odds ratio for non-insulin-dependent diabetes (NIDD) in PMC12579343
(JAMA Netw Open 2025, doi:10.1001/jamanetworkopen.2025.40818).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/.png and fig2.svg/.png next to this file.
Inputs are copied from the article as archived in PubMed Central:
Table 2 (counts and means/SDs, n = 189 PCOM vs 151 non-PCOM, and footnote a),
Table 3 (every adjusted odds ratio with its 95% CI and P value).
Method: a logistic-regression Wald interval is exp(b +/- 1.96*SE), so the point estimate
must lie inside it and equal its geometric midpoint sqrt(lower*upper). Any two of
(estimate, lower, upper, P) fix the other two; each pair is worked out below.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ------------------------------------------------------------------ inputs: Table 3 (verbatim)
TABLE3 = [ # outcome, aOR, lower, upper, printed P
("IDD", 0.63, 0.17, 2.26, .48),
("NIDD", 2.92, 1.06, 2.82, .02),
("Dyslipidemia", 1.00, 0.57, 1.75, .99),
("Receiving anticoagulation therapy", 1.09, 0.25, 5.78, .91),
("Ischemic heart disease", 1.04, 0.28, 3.82, .94),
("Arrhythmia", 0.47, 0.07, 2.94, .42),
("CVD or risk requiring aspirin treatment", 0.58, 0.25, 1.34, .20),
("Sleep apnea", 0.23, 0.02, 2.03, .18),
("Hypertension", 1.46, 0.82, 2.56, .17),
("Psoriasis", 0.77, 0.06, 8.70, .83),
("Hypothyroidism", 0.55, 0.30, 3.00, .052),
("Neurologic disease", 1.29, 0.49, 3.38, .60),
("Neuropathy or neuralgia", 1.04, 0.31, 3.46, .94),
("Respiratory disease", 1.00, 0.48, 2.06, .98),
("Asthma or COPD", 0.91, 0.43, 1.92, .80),
("Psychiatric disease total", 0.99, 0.55, 1.78, .98),
("Depression", 0.75, 0.39, 1.45, .39),
("Anxiety", 2.34, 0.60, 9.08, .21),
("Schizophrenia or psychotic medications", 0.92, 0.32, 2.66, .88),
("Malignant neoplasm total", 0.56, 0.19, 1.69, .31),
("Breast carcinoma", 0.71, 0.19, 2.60, .61),
("Lung carcinoma", 0.39, 0.01, 8.60, .55),
("Gastrointestinal disease total", 1.11, 0.61, 2.04, .72),
("GERD", 1.09, 0.59, 2.01, .76),
("Rheumatologic disease total", 2.16, 0.64, 7.30, .21),
("Gout", 2.84, 0.32, 24.70, .34),
("Osteoporosis", 1.62, 0.52, 5.06, .40),
]
# Table 2: NIDD counts
N_PCOM, N_NON = 189, 151
NIDD_PCOM, NIDD_NON = 45, 14
# Table 2: mean (SD) by group, and the mean differences in its footnote a
MEANS = [ # label, m1, s1, n1, m2, s2, n2, reported difference (lo, hi), printed P
("Age at follow-up", 61.71, 6.42, 189, 67.67, 10.46, 151, (-5.96, -7.86, -4.06), "<.001"),
("Time in study", 33.68, 2.34, 189, 35.10, 4.94, 151, (-1.42, -6.25, -2.22), "<.001"),
("Age at death", 54.25, 11.52, 7, 70.83, 13.64, 9, (-16.58, -35.77, 2.60), ".08"),
]
def wald_p(b, se):
return 2 * stats.norm.sf(abs(b) / se)
def from_ci(lo, hi):
"""Centre, SE and P implied by a 95% Wald interval alone."""
se = (np.log(hi) - np.log(lo)) / (2 * Z)
c = np.sqrt(lo * hi)
return c, se, wald_p(np.log(c), se)
def from_est_lo(est, lo):
se = (np.log(est) - np.log(lo)) / Z
return est ** 2 / lo, se, wald_p(np.log(est), se)
def from_est_p(est, p):
se = abs(np.log(est)) / stats.norm.isf(p / 2)
return est * np.exp(-Z * se), est * np.exp(Z * se), se
def from_hi_p(hi, p):
zp = stats.norm.isf(p / 2)
se = np.log(hi) / (zp + Z)
est = np.exp(zp * se)
return est, est * np.exp(-Z * se), se
def from_lo_p(lo, p):
zp = stats.norm.isf(p / 2)
se = np.log(lo) / (zp - Z)
est = np.exp(zp * se)
return est, est * np.exp(Z * se), se
print("=" * 78)
print("1. NIDD adjusted OR as printed: 2.92 (95% CI 1.06-2.82), P = .02")
print("=" * 78)
est, lo, hi, p = 2.92, 1.06, 2.82, .02
print(f"estimate inside its interval? {lo <= est <= hi}; estimate - upper limit = {est - hi:.2f}")
print(f"rounding: estimate in [2.915, 2.925), upper in [2.815, 2.825): gap at least {2.915 - 2.825:.3f}")
cA, seA, pA = from_ci(lo, hi)
print(f"\nReading A (interval correct): centre = {cA:.3f}, SE(log OR) = {seA:.3f}, "
f"z = {np.log(cA) / seA:.2f}, P = {pA:.4f}; excess odds {100 * (cA - 1):.0f}% vs {100 * (est - 1):.0f}% printed")
pr = []
for l_ in (1.055, 1.065):
for h_ in (2.815, 2.825):
pr.append(from_ci(l_, h_)[2])
print(f" P over the rounding range of both limits: {min(pr):.4f} to {max(pr):.4f}")
hiB, seB, pB = from_est_lo(est, lo)
print(f"Reading B (estimate and lower limit correct): upper = {hiB:.2f}, SE = {seB:.3f}, "
f"z = {np.log(est) / seB:.2f}, P = {pB:.4f}")
loC = est ** 2 / hi
print(f"Reading C (estimate and upper limit correct): lower = {loC:.2f} (interval {loC:.2f}-{hi})")
print(f"Interval ratio upper/lower printed = {hi / lo:.2f}; reading B = {hiB / lo:.2f}")
print("\nWhat the printed P = .02 implies when paired with each other number (P read as 0.015 / 0.02 / 0.025):")
for pp in (0.015, 0.02, 0.025):
l1, h1, s1 = from_est_p(est, pp)
e2, l2, s2 = from_hi_p(hi, pp)
e3, h3, s3 = from_lo_p(lo, pp)
print(f" P={pp:.3f}: with estimate 2.92 -> CI {l1:.2f}-{h1:.2f}; with upper 2.82 -> estimate {e2:.2f}, "
f"lower {l2:.2f}; with lower 1.06 -> estimate {e3:.2f}, upper {h3:.2f}")
print("Note: an FDR (Benjamini-Hochberg) adjusted P is never smaller than the raw P, so the raw P\n"
"behind a printed '.02' is below .025 if the printed value is adjusted.")
print("\nBonferroni: printed P = .02 times k comparisons")
for k in (1, 2, 3, 27):
print(f" k = {k:2d}: {min(1, 0.02 * k):.3f}")
print(f" largest k with P*k < .05: P=.02 -> {int(np.ceil(0.05 / 0.02) - 1)}; "
f"P=.015 -> {int(np.ceil(0.05 / 0.015) - 1)}; reading A P={pA:.3f} -> {int(np.ceil(0.05 / pA) - 1)}; "
f"reading B P={pB:.3f} -> {int(np.ceil(0.05 / pB) - 1)}")
print("\n" + "=" * 78)
print("2. Crude NIDD odds ratio from Table 2 counts (45/189 vs 14/151)")
print("=" * 78)
a, b = NIDD_PCOM, N_PCOM - NIDD_PCOM
c, d = NIDD_NON, N_NON - NIDD_NON
or_c = a * d / (b * c)
se_c = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
lo_c, hi_c = or_c * np.exp(-Z * se_c), or_c * np.exp(Z * se_c)
chi2, pchi, _, _ = stats.chi2_contingency([[a, b], [c, d]], correction=False)
print(f"percentages {100 * a / N_PCOM:.2f}% vs {100 * c / N_NON:.2f}% (printed 23.80 vs 9.27)")
print(f"crude OR = {or_c:.2f} (Wald 95% CI {lo_c:.2f}-{hi_c:.2f}), SE = {se_c:.3f}, Wald P = {wald_p(np.log(or_c), se_c):.5f}")
print(f"chi-square (no continuity correction) = {chi2:.2f}, P = {pchi:.5f} (printed <.001)")
print("\n" + "=" * 78)
print("3. Every Table 3 row: estimate vs geometric midpoint of its own CI, and printed P vs Wald P")
print("=" * 78)
rows = []
print(f"{'outcome':40s} {'aOR':>5s} {'CI':>13s} {'midpt':>6s} {'ratio':>6s} {'P':>5s} {'P(Wald)':>7s}")
for name, e, l_, h_, pp in TABLE3:
se = (np.log(h_) - np.log(l_)) / (2 * Z)
mid = np.sqrt(l_ * h_)
pw = wald_p(np.log(e), se)
rows.append((name, e, l_, h_, pp, mid, pw))
print(f"{name:40s} {e:5.2f} {l_:5.2f}-{h_:<6.2f} {mid:6.2f} {e / mid:6.2f} {pp:5.3f} {pw:7.3f}")
others = [r for r in rows if r[0] not in ("NIDD", "Hypothyroidism")]
ratios = np.array([r[1] / r[5] for r in others])
dp = np.array([abs(r[4] - r[6]) for r in others])
print(f"\nOther {len(others)} rows: estimate/midpoint ratio from {ratios.min():.2f} to {ratios.max():.2f}; "
f"{np.sum(np.abs(np.log(ratios)) < np.log(1.05))} of {len(others)} within 5%")
print(f"Other {len(others)} rows: |printed P - Wald P from estimate and CI width| median {np.median(dp):.3f}, "
f"max {dp.max():.3f}; {np.sum(dp <= 0.02)} of {len(others)} within 0.02")
print(f"Number of Table 3 outcomes with an estimate: {len(TABLE3)}")
print("\nHypothyroidism, printed 0.55 (0.30-3.00), P = .052")
cH, seH, pH = from_ci(0.30, 3.00)
hH, sH, pH2 = from_est_lo(0.55, 0.30)
print(f" inside? {0.30 <= 0.55 <= 3.00}; midpoint of CI = {cH:.2f}, P from CI alone = {pH:.3f}")
print(f" estimate and lower limit correct -> upper = {hH:.2f}, P = {pH2:.3f} (printed .052)")
print(f" estimate and upper limit correct -> lower = {0.55 ** 2 / 3.00:.2f}")
print("\n" + "=" * 78)
print("4. Mean differences in Table 2 footnote a")
print("=" * 78)
for name, m1, s1, n1, m2, s2, n2, (rd, rl, rh), rp in MEANS:
diff = m1 - m2
v = s1 ** 2 / n1 + s2 ** 2 / n2
se = np.sqrt(v)
df = v ** 2 / ((s1 ** 2 / n1) ** 2 / (n1 - 1) + (s2 ** 2 / n2) ** 2 / (n2 - 1))
t = stats.t.ppf(0.975, df)
pw = 2 * stats.t.sf(abs(diff) / se, df)
sp = np.sqrt(((n1 - 1) * s1 ** 2 + (n2 - 1) * s2 ** 2) / (n1 + n2 - 2))
sep = sp * np.sqrt(1 / n1 + 1 / n2)
tp = stats.t.ppf(0.975, n1 + n2 - 2)
pp_ = 2 * stats.t.sf(abs(diff) / sep, n1 + n2 - 2)
print(f"{name}: printed {rd} ({rl} to {rh}), P {rp}; reported CI contains its estimate? {rl <= rd <= rh}; "
f"CI midpoint {(rl + rh) / 2:.2f}")
print(f" recomputed diff {diff:.2f}; Welch CI {diff - t * se:.2f} to {diff + t * se:.2f} (df {df:.1f}, P = {pw:.4f}); "
f"pooled CI {diff - tp * sep:.2f} to {diff + tp * sep:.2f} (P = {pp_:.4f})")
if name == "Age at death":
se_rep = (rh - rl) / 2 / t
print(f" SE implied by printed CI (Welch df) = {se_rep:.2f} vs {se:.2f} from the SDs and n; "
f"P with that SE = {2 * stats.t.sf(abs(rd) / se_rep, df):.3f}")
k = (s1 ** 2 + s2 ** 2) / se_rep ** 2
print(f" equal group size that would give that SE: about {k:.1f} per group")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# ---- fig1: printed interval vs the intervals the paper's own numbers allow
items = [ # label, est, lo, hi, colour, filled, note
("Printed in abstract,\nResults and Table 3", est, lo, hi, ORANGE, True,
"aOR 2.92 lies outside\nits own interval 1.06–2.82"),
("A: if the interval\n1.06–2.82 is right", cA, lo, hi, BLUE, True,
f"centre {cA:.2f}, P = {pA:.3f}"),
("B: if aOR 2.92 and the\nlower limit 1.06 are right", est, lo, hiB, BLUE, True,
f"upper limit {hiB:.2f}, P = {pB:.3f}"),
("Unadjusted, from Table 2\ncounts 45/189 vs 14/151", or_c, lo_c, hi_c, BLUE, False,
f"crude OR {or_c:.2f} ({lo_c:.2f}–{hi_c:.2f})"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.arange(len(items))[::-1]
for y, (lab, e, l_, h_, col, filled, note) in zip(ys, items):
ax.plot([l_, h_], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=2)
for xx in (l_, h_):
ax.plot([xx, xx], [y - 0.13, y + 0.13], color=col, lw=2, zorder=2)
ax.plot(e, y, "o", ms=9, color=col, mfc=col if filled else BG, mew=2, zorder=3)
ax.text(12.5, y, note, va="center", ha="left", fontsize=10, color=INK if col == ORANGE else INK2)
y0 = ys[0]
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(1.0, y0 + 0.62, "OR = 1 (no association)", ha="left", va="center", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.8, 11)
ax.set_ylim(ys[-1] - 0.55, ys[0] + 0.85)
ticks = [1, 1.5, 2, 3, 5, 8]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys)
ax.set_yticklabels([it[0] for it in items], fontsize=10, color=INK)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Odds ratio for NIDD, PCOM vs non-PCOM (log scale), with 95% CI")
ax.set_title("NIDD: the printed estimate lies outside its own 95% CI", loc="left", fontsize=12,
fontweight="bold", color=INK, pad=12)
from matplotlib.lines import Line2D # noqa: E402
leg = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="Printed in the paper"),
Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, label="Recomputed from the paper's own numbers")]
fig.legend(handles=leg, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.30, right=0.70, top=0.88, bottom=0.25)
save(fig, "fig1")
# ---- fig2: every Table 3 row, printed aOR vs the geometric midpoint of its printed CI
fig, ax = plt.subplots(figsize=(7.2, 4.6))
lim = (0.15, 4.5)
ax.plot(lim, lim, color=INK2, lw=0.8, zorder=1)
ax.text(0.205, 0.24, "estimate = midpoint", rotation=45, rotation_mode="anchor", ha="left", va="bottom",
fontsize=10, color=INK2)
ox = [r[1] for r in others]
oy = [r[5] for r in others]
ax.plot(ox, oy, "o", ms=8, color=BLUE, mfc=BG, mew=1.6, zorder=2, ls="none",
label=f"Other {len(others)} outcomes in Table 3")
for r in rows:
if r[0] in ("NIDD", "Hypothyroidism"):
ax.plot(r[1], r[5], "o", ms=9, color=ORANGE, zorder=3, ls="none")
if r[0] == "NIDD":
txt = f"NIDD: aOR {r[1]:.2f},\nCI 1.06\u20132.82\ncentred at {r[5]:.2f}"
xy_t, ha = (4.4, 0.98), "right"
else:
txt = f"Hypothyroidism: aOR {r[1]:.2f},\nCI 0.30\u20133.00\ncentred at {r[5]:.2f}"
xy_t, ha = (0.165, 1.75), "left"
ax.annotate(txt, xy=(r[1], r[5]), xytext=xy_t, ha=ha, va="center", fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1, shrinkA=3, shrinkB=6))
ax.plot([], [], "o", ms=9, color=ORANGE, ls="none", label="Rows whose estimate is off-centre")
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_xlim(*lim)
ax.set_ylim(*lim)
t2 = [0.2, 0.5, 1, 2, 4]
ax.set_xticks(t2)
ax.set_xticklabels([f"{t:g}" for t in t2])
ax.set_yticks(t2)
ax.set_yticklabels([f"{t:g}" for t in t2])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.yaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.grid(color=GRID, lw=0.8, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_aspect("equal")
ax.set_xlabel("Printed adjusted odds ratio (Table 3, log scale)")
ax.set_ylabel("Geometric midpoint of the printed 95% CI")
ax.set_title("Table 3: the other 25 intervals sit on or near their estimates", loc="left", fontsize=12,
fontweight="bold", color=INK, pad=10)
ax.legend(loc="lower right", frameon=False, fontsize=10)
fig.subplots_adjust(left=0.08, right=0.98, top=0.9, bottom=0.12)
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Risk-difference confidence intervals for the primary outcome are about 23 to 31 times too narrow
JAMA Netw Open (2026) · doi:10.1001/jamanetworkopen.2026.25965 · PMC13434869
Summary
The Abstract and the Results report the trial's primary-outcome effect sizes, the absolute differences in first-pass intubation success between direct laryngoscopy (DL), Macintosh-blade video laryngoscopy (VLM) and hyperangulated video laryngoscopy (VLH), with 95% confidence intervals 0.22 to 0.33 percentage points wide, for example 4.66 (95% CI, 4.45-4.78) for VLM vs DL (written "DL vs VLM" in the paper). The paper's own Table 2 counts (625/799, 673/812, 711/812) reproduce every point estimate and, to within 0.03, the paper's own z statistics (exactly with a pooled standard error), but with the method the paper names (an unpooled z approach), they give intervals 6.9 to 7.7 points wide: 0.80 to 8.52, 5.69 to 12.99 and 1.24 to 8.12. The numbers needed to treat in the Discussion carry the same narrow intervals (VLM 21.5, reported 20.9-22.2; from the counts, 11.7-125.2). The direction and statistical significance of every comparison are unchanged, but the published intervals overstate the precision of the trial's headline result 23- to 31-fold. That matters most for VLM, where the data are compatible with a gain of anywhere from 0.8 to 8.5 points.
What the paper reports
Differences between success rates were tested applying the z test for unpooled variance and reported as absolute differences with corresponding 95% CIs.
Abstract, Main Outcomes and Measure
Unadjusted absolute risk differences for first-pass intubation success were 4.66 (95% CI, 4.45-4.78) percentage points for DL vs VLM, 9.34 (95% CI, 9.22-9.46) percentage points for DL vs VLH, and 4.68 (95% CI, 4.57-4.79) percentage points for VLM vs VLH.
Abstract, Results
In 2-sided z tests, VLM and VLH were significantly more effective in facilitating first-pass success than the DL (VLM vs DL: z = 2.36, P = .02; VLH vs DL: z = 4.98, P < .001). VLH was in turn superior to VLM (z = 2.66; P = .008). Unadjusted absolute risk differences for a successful first attempt were 4.66 (95% CI, 4.45-4.78) percentage points for DL vs VLM, 9.34 (95% CI, 9.22-9.46) percentage points for DL vs VLH, and 4.68 (95% CI, 4.57-4.79) percentage points for VLM vs VLH.
Results, Primary End Point and Crossover Between Groups
Relative risks (RRs) for successful first-attempt intubation were significantly higher for both VLM and VLH over DL (VLM vs DL: RR, 1.06 [95% CI, 1.01-1.11]; VLH vs DL: RR, 1.12 [95% CI, 1.07-1.17]) and for VLM over VLH (RR, 1.06 [95% CI, 1.01-1.10]) (eTable 2 in Supplement 1).
Results, same paragraph
Numbers needed to treat were 21.5 (95% CI, 20.9-22.2) for VLM and 10.7 (95% CI, 10.6-10.8) for VLH.
Discussion
Table 2 ("First-Pass Success Rates of Tracheal Intubations Using Different Laryngoscopes"), mITT population, row "Yes": DL 625 (78.2), VLM 673 (82.9), VLH 711 (87.6); row "Total": 799, 812, 812.
What the paper's own numbers show
Method. For each pair of arms I took the Table 2 counts and computed the risk difference, its unpooled standard error √(p₁(1−p₁)/n₁ + p₂(1−p₂)/n₂), and the 95% Wald interval RD ± 1.96·SE, which is the interval that goes with the unpooled z test the paper says it used. As checks I also computed the Newcombe (hybrid Wilson score) and Agresti–Caffo intervals, the z statistics (unpooled and pooled), the log-scale Wald interval for the relative risk, and the NNT interval obtained by inverting the risk-difference limits. Finally, I worked backwards from the reported intervals: dividing a reported width by 2 × 1.96 gives the standard error that interval implies, and dividing the reported risk difference by the reported z gives a second implied standard error. Comparisons are labelled higher-success arm vs lower-success arm (VLM vs DL, VLH vs DL, VLH vs VLM); the paper writes them as DL vs VLM, DL vs VLH and VLM vs VLH, with the same positive values. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| First-pass success, DL / VLM / VLH | Table 2 | 625/799 (78.2), 673/812 (82.9), 711/812 (87.6) | 78.22%, 82.88%, 87.56% |
| RD, VLM vs DL (pp) | Abstract; Results | 4.66 | 4.66 |
| 95% CI, VLM vs DL | Abstract; Results | 4.45 to 4.78 (width 0.33) | Wald 0.80 to 8.52 (width 7.72); Newcombe 0.79 to 8.52; Agresti–Caffo 0.79 to 8.51 |
| RD, VLH vs DL (pp) | Abstract; Results | 9.34 | 9.34 |
| 95% CI, VLH vs DL | Abstract; Results | 9.22 to 9.46 (width 0.24) | Wald 5.69 to 12.99 (width 7.31); Newcombe 5.68 to 12.99; Agresti–Caffo 5.66 to 12.97 |
| RD, VLH vs VLM (pp) | Abstract; Results | 4.68 | 4.68 |
| 95% CI, VLH vs VLM | Abstract; Results | 4.57 to 4.79 (width 0.22) | Wald 1.24 to 8.12 (width 6.89); Newcombe 1.23 to 8.13; Agresti–Caffo 1.22 to 8.12 |
| Recomputed width ÷ reported width | derived | — | 23.4×, 30.4×, 31.3× |
| z, VLM vs DL | Results | 2.36 (P = .02) | unpooled 2.37 (P = .018); pooled 2.36 |
| z, VLH vs DL | Results | 4.98 (P < .001) | unpooled 5.01 (P = 5.4 × 10⁻⁷); pooled 4.98 |
| z, VLH vs VLM | Results | 2.66 (P = .008) | unpooled 2.66 (P = .0077); pooled 2.66 |
| SE implied by reported RD ÷ reported z (pp) | derived | 1.975, 1.876, 1.759 | unpooled SE from counts 1.970, 1.864, 1.757 |
| SE implied by reported CI width ÷ 3.92 (pp) | derived | 0.084, 0.061, 0.056 | — |
| z that the reported CIs would imply | derived | — | 55.4, 152.6, 83.4 |
| Per-arm sample size at which a Wald CI would be as narrow as reported | derived | — | about 441,000, 746,000 and 796,000 (trial: 799–812) |
| Reported CI, distance below / above point | Abstract; Results | VLM vs DL −0.21 / +0.12; others −0.12 / +0.12 and −0.11 / +0.11 | Wald is symmetric (±3.86, ±3.65, ±3.44) |
| RR, VLM vs DL | Results; eTable 2 | 1.06 (1.01–1.11) | 1.060 (1.010–1.112) |
| RR, VLH vs DL | Results; eTable 2 | 1.12 (1.07–1.17) | 1.119 (1.070–1.171) |
| RR, VLH vs VLM | Results; eTable 2 | 1.06 (1.01–1.10) | 1.056 (1.014–1.100) |
| NNT, VLM vs DL | Discussion | 21.5 (20.9–22.2) | 21.5 (11.7–125.2) |
| NNT, VLH vs DL | Discussion | 10.7 (10.6–10.8) | 10.7 (7.7–17.6) |
| NNT limits from inverting the reported RD CIs | derived | — | VLM 20.9–22.5; VLH 10.6–10.8 |
Two independent routes give the same standard errors. The counts give 1.97, 1.86 and 1.76 points, and the paper's own risk differences divided by its own z statistics give 1.98, 1.88 and 1.76 points. The reported intervals imply standard errors of 0.06 to 0.08 points, about 23 to 31 times smaller. The relative-risk intervals in the same paragraph and in eTable 2 match the counts to the printed decimal, so the precision the paper reports on the ratio scale agrees with the wide intervals, not the narrow ones. The NNT intervals match the reciprocals of the reported risk-difference limits (exactly for VLH; for VLM the upper limit is 22.2 against 22.5), so they inherit the same error.
Figure
Figure 1. Each row is one of the three primary-outcome comparisons. Orange diamonds show the risk difference and 95% CI reported in the Abstract and Results. The reported intervals are narrower than the marker, so the bars are hidden behind it; the right-hand labels give their limits. Blue circles with capped bars show the 95% CI recomputed from the Table 2 counts with an unpooled Wald interval, the method the paper states. The solid gray line marks no difference and the dashed line marks the −5 point noninferiority margin. Every recomputed interval lies to the right of both, so significance and noninferiority are unchanged, but each is 23 to 31 times wider than reported. The axis is linear and includes zero.
Figure 2. Numbers needed to treat with VLM or VLH instead of DL to gain one additional first-pass success, as reported in the Discussion (orange diamonds) and as obtained by inverting the recomputed Wald risk-difference limits (blue circles). The axis is logarithmic because the NNT is a reciprocal. For VLM, the interval the counts support runs from about 12 to about 125 intubations, against a reported 20.9 to 22.2.
Why it matters
- The headline result. First-pass success is the primary outcome, and these intervals are the only interval estimates for it on the absolute scale in the Abstract. As printed, they say the VLM gain is known to lie between 4.45 and 4.78 points. The data support 0.8 to 8.5 points. The lower end is a small effect.
- Clinical translation. The NNT is the quantity a clinician or guideline panel would use. The published VLM interval (20.9–22.2) pins it near 21; the counts allow anything from about 12 to about 125. For VLH the corrected range is 7.7 to 17.6 instead of 10.6 to 10.8.
- Comparison between video blades. VLH vs VLM (the paper's "VLM vs VLH") is reported as 4.68 (4.57–4.79). The corrected interval, 1.24 to 8.12, still excludes zero, but the gap between the two blades is far less precisely determined than printed.
- Reuse. Meta-analyses and decision models that take the published interval, or back-calculate a standard error from it, would give this trial roughly 550 to 980 times its correct inverse-variance weight on the risk-difference scale (the square of the width ratios).
- What does not change. The point estimates, the ordering VLH > VLM > DL, the P values, the relative risks, the noninferiority conclusions and the superiority conclusions all stand. The Conclusions do not quote the intervals.
Explanations considered
- Different analysis population. The point estimates, the z statistics and the relative-risk intervals all match the mITT counts in Table 2, so the narrow intervals refer to the same 2423 patients. The per-protocol set (eTable 1: 590/758, 663/799, 704/804) is smaller and gives similarly wide intervals: 5.14 (1.20 to 9.08), 9.73 (5.99 to 13.46) and 4.58 (1.12 to 8.05). It does not account for the numbers.
- A different interval method. Wald, Newcombe score and Agresti–Caffo intervals all come out 6.9 to 7.7 points wide. A pooled standard error changes the z values only in the second decimal. No standard interval for about 800 patients per arm is 0.2 to 0.3 points wide; that width would need roughly 440,000 to 800,000 patients per arm.
- An adjusted or model-based estimate. The intervals are labelled "unadjusted" and sit next to unadjusted point estimates that equal the raw differences. Adjustment for site or covariates would not reduce the standard error 23- to 31-fold in a trial of this size, and the site-random-effect model is reported separately (eTable 3).
- A different confidence level. The text says "95% CI" each time. At the standard errors the counts imply, intervals of the reported widths would have confidence levels of about 5% to 7%, which no report uses.
- Units (proportion vs percentage points). All three quantities are labelled percentage points, the point estimates are correct in percentage points, and the reported limits are not the Wald limits divided or multiplied by 100. Unit confusion does not reproduce them.
- Conversion or typesetting artifact. The same intervals appear in the Abstract and, word for word, in the Results, as running text rather than table cells, in the archived JATS XML and, as checked earlier in this review, in the publisher's HTML. No dropped digit, sign or decimal mark turns 0.80–8.52 into 4.45–4.78.
- An arithmetic route that would reproduce the printed limits. I did not find one that reproduces all three intervals, and the first interval is asymmetric about its point estimate (−0.21 / +0.12), which a Wald interval cannot be. The write-up therefore states only that the printed intervals are inconsistent with the paper's own counts, z statistics and stated method; it does not say how they were produced.
Two smaller observations, which do not affect the main finding:
- The Results give the same z statistics (2.36 and 4.98) for the 1-sided noninferiority tests as for the 2-sided superiority tests. A noninferiority z against the stated −5 point margin, using the unpooled standard error, would be 4.90 (VLM) and 7.69 (VLH). Both conclusions hold either way. The reported z values match the pooled-variance statistic against zero to the printed decimal (2.36, 4.98, 2.66), while the unpooled values are 2.37, 5.01 and 2.66.
- The Results describe the third relative risk as "for VLM over VLH (RR, 1.06 ...)"; 1.06 is the ratio of VLH to VLM (87.56 / 82.88 = 1.056), and eTable 2 labels it "VLH vs. VLM".
Status
As of 2026-10-23:
- Version of record. The JAMA Network Open full text (via doi.org) carries the same intervals in the Abstract and Results, the same z statistics, the same Table 2 counts and the same NNT intervals as the PubMed Central archive (checked in an earlier step of this review; the publisher PDF was not retrievable). The quotes above were checked verbatim against the PMC-archived text.
- Supplements. Supplement 1 (eMethods, eFigure, eTables 1–8) was read; it gives no confidence intervals for the risk differences. Its eTable 2 relative-risk intervals match the counts (see above), and eTable 1 gives the per-protocol counts used above. Supplement 2 (protocol and statistical analysis plan) specifies one-sided and two-sided z tests with unpooled variance and a 95% interval for the difference judged against the −5 point margin; it does not describe a method that would give narrower intervals. Supplement 4 (data sharing statement) is covered under Data below. Supplement 3 (collaborators) was not relevant.
- Corrections and retractions. Crossref, PubMed Central and Europe PMC list no correction, erratum, expression of concern or retraction for this DOI.
- PubPeer. Web searches for the title and trial name found no PubPeer entry; PubPeer's own search endpoint was not reachable from this machine (HTTP 403), so its absence is not confirmed directly.
- Related content. PubMed links a commentary on the trial (PMID 42545704); it was not reviewed here.
- Data. The registration (NCT05228288) lists IPD sharing as No, but the article's Data Sharing Statement (Supplement 4) says deidentified participant data and a data dictionary are available from the corresponding author for non-commercial use. They were not requested for this check, so the Table 2 counts are its basis; with those data the journal or the authors could recompute the intervals directly.
Sources: the article in PubMed Central (PMC13434869) and at doi:10.1001/jamanetworkopen.2026.25965; Supplements 1, 2 and 4 from the Europe PMC supplementary files; ClinicalTrials.gov NCT05228288; Crossref and Europe PMC metadata.
Suggested correction
Replace the risk-difference intervals in the Abstract and in the Results (Primary End Point and Crossover Between Groups) with intervals computed from the Table 2 counts by the stated method. With the unpooled Wald interval these are:
- VLM vs DL: 4.66 (95% CI, 0.80–8.52) percentage points
- VLH vs DL: 9.34 (95% CI, 5.69–12.99) percentage points
- VLH vs VLM: 4.68 (95% CI, 1.24–8.12) percentage points
and replace the Discussion's NNT intervals with 21.5 (95% CI, 11.7–125.2) for VLM and 10.7 (95% CI, 7.7–17.6) for VLH. If a score-based interval is preferred, the Newcombe limits (0.79–8.52, 5.68–12.99, 1.23–8.13) differ only in the second decimal. Optionally, the authors could also report the noninferiority z statistics against the −5 point margin, state whether the reported z statistics used pooled or unpooled variance, and change "for VLM over VLH" to "for VLH over VLM".
Recomputation script (python recompute.py.txt)
# Recomputation for PMC13434869 (JAMA Netw Open 2026; doi:10.1001/jamanetworkopen.2026.25965)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2 (SVG + PNG)
# in the folder that holds this script. Inputs are the paper's own numbers only.
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975) # 1.959964
# ---- Inputs from the paper -------------------------------------------------
# Table 2 (mITT): first-pass success, No./total
mitt = {"DL": (625, 799), "VLM": (673, 812), "VLH": (711, 812)}
# eTable 1 (Supplement 1, per-protocol): first-pass success
pp = {"DL": (590, 758), "VLM": (663, 799), "VLH": (704, 804)}
# Abstract and Results: risk differences (pp) with 95% CI; Results: z statistics
# keys are (lower-success arm, higher-success arm)
reported = {
("DL", "VLM"): dict(rd=4.66, lo=4.45, hi=4.78, z=2.36, label="VLM vs DL"),
("DL", "VLH"): dict(rd=9.34, lo=9.22, hi=9.46, z=4.98, label="VLH vs DL"),
("VLM", "VLH"): dict(rd=4.68, lo=4.57, hi=4.79, z=2.66, label="VLH vs VLM"),
}
# Results / eTable 2: relative risks (mITT and per-protocol)
rr_rep = {("DL", "VLM"): (1.06, 1.01, 1.11), ("DL", "VLH"): (1.12, 1.07, 1.17),
("VLM", "VLH"): (1.06, 1.01, 1.10)}
rr_rep_pp = {("DL", "VLM"): (1.07, 1.01, 1.12), ("DL", "VLH"): (1.12, 1.07, 1.18),
("VLM", "VLH"): (1.06, 1.01, 1.10)}
# Discussion: numbers needed to treat (vs DL)
nnt_rep = {"VLM": (21.5, 20.9, 22.2), "VLH": (10.7, 10.6, 10.8)}
NI_MARGIN = -0.05 # Methods: noninferiority margin, 5 percentage points
def wilson(x, n, z=Z):
p = x / n
c = (p + z * z / (2 * n)) / (1 + z * z / n)
h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
return c - h, c + h
def newcombe(xb, nb, xa, na):
"""Newcombe hybrid score interval for pb - pa."""
pb, pa = xb / nb, xa / na
l1, u1 = wilson(xb, nb)
l2, u2 = wilson(xa, na)
d = pb - pa
return d - np.sqrt((pb - l1) ** 2 + (u2 - pa) ** 2), d + np.sqrt((u1 - pb) ** 2 + (pa - l2) ** 2)
def agresti_caffo(xb, nb, xa, na):
pb, pa = (xb + 1) / (nb + 2), (xa + 1) / (na + 2)
d = pb - pa
se = np.sqrt(pb * (1 - pb) / (nb + 2) + pa * (1 - pa) / (na + 2))
return d - Z * se, d + Z * se
def compare(data, a, b):
xa, na = data[a]
xb, nb = data[b]
pa, pb = xa / na, xb / nb
rd = pb - pa
se = np.sqrt(pa * (1 - pa) / na + pb * (1 - pb) / nb) # unpooled (stated method)
ppool = (xa + xb) / (na + nb)
se_pool = np.sqrt(ppool * (1 - ppool) * (1 / na + 1 / nb)) # pooled, for comparison
rr = pb / pa
se_lrr = np.sqrt(1 / xb - 1 / nb + 1 / xa - 1 / na)
return dict(pa=pa, pb=pb, rd=rd, se=se, se_pool=se_pool,
wald=(rd - Z * se, rd + Z * se), newc=newcombe(xb, nb, xa, na),
ac=agresti_caffo(xb, nb, xa, na),
z=rd / se, z_pool=rd / se_pool, z_ni=(rd - NI_MARGIN) / se,
rr=rr, rr_ci=(rr * np.exp(-Z * se_lrr), rr * np.exp(Z * se_lrr)))
print("=== Proportions (Table 2, mITT) ===")
for k, (x, n) in mitt.items():
print(f" {k}: {x}/{n} = {100 * x / n:.2f}%")
res = {}
print("\n=== Risk differences, mITT (percentage points) ===")
for (a, b), r in reported.items():
c = compare(mitt, a, b)
res[(a, b)] = c
rep_w = r["hi"] - r["lo"]
w = 100 * (c["wald"][1] - c["wald"][0])
se_rep_ci = rep_w / (2 * Z) # SE implied by reported CI width (pp)
se_rep_z = r["rd"] / r["z"] # SE implied by reported RD and z (pp)
print(f"{r['label']}: p = {100*c['pa']:.2f}% vs {100*c['pb']:.2f}%")
print(f" RD reported {r['rd']:.2f} recomputed {100*c['rd']:.2f}")
print(f" 95% CI reported {r['lo']:.2f} to {r['hi']:.2f} (width {rep_w:.2f})")
print(f" Wald (unpooled) {100*c['wald'][0]:.2f} to {100*c['wald'][1]:.2f} (width {w:.2f})")
print(f" Newcombe {100*c['newc'][0]:.2f} to {100*c['newc'][1]:.2f}")
print(f" Agresti-Caffo {100*c['ac'][0]:.2f} to {100*c['ac'][1]:.2f}")
print(f" Recomputed width / reported width = {w / rep_w:.1f}x")
print(f" Reported CI distances from point: -{r['rd']-r['lo']:.2f} / +{r['hi']-r['rd']:.2f}")
print(f" SE: recomputed {100*c['se']:.3f} pp; implied by reported z {se_rep_z:.3f} pp; "
f"implied by reported CI {se_rep_ci:.4f} pp")
print(f" z implied by reported CI = {r['rd']/se_rep_ci:.1f}")
print(f" z superiority: reported {r['z']:.2f}; unpooled {c['z']:.2f} "
f"(2-sided P = {2*stats.norm.sf(c['z']):.2g}, 1-sided P = {stats.norm.sf(c['z']):.2g}); "
f"pooled {c['z_pool']:.2f}")
print(f" z noninferiority vs -5 pp margin (unpooled) = {c['z_ni']:.2f}")
rr = rr_rep[(a, b)]
print(f" RR reported {rr[0]:.2f} ({rr[1]:.2f}-{rr[2]:.2f}); recomputed "
f"{c['rr']:.3f} ({c['rr_ci'][0]:.3f}-{c['rr_ci'][1]:.3f})")
# sample size per arm at which the Wald CI would be as narrow as reported
n_needed = (np.mean([mitt[a][1], mitt[b][1]])) * (100 * c['se'] / se_rep_ci) ** 2
print(f" Per-arm n for a CI this narrow at these rates: about {n_needed:,.0f}")
print("\n=== Further derived quantities ===")
for (a, b), r in reported.items():
c = res[(a, b)]
w_ratio = 100 * (c["wald"][1] - c["wald"][0]) / (r["hi"] - r["lo"])
half = (r["hi"] - r["lo"]) / 2
conf = 2 * stats.norm.cdf(half / (100 * c["se"])) - 1
print(f" {r['label']}: Wald half-width +/-{100*Z*c['se']:.2f} pp; inverse-variance weight ratio "
f"(reported/correct) = {w_ratio**2:.0f}x; confidence level of an interval of the reported "
f"width at the true SE = {100*conf:.1f}%")
print("\n=== Per-protocol (eTable 1): risk differences and RRs ===")
for (a, b), r in reported.items():
c = compare(pp, a, b)
rr = rr_rep_pp[(a, b)]
print(f"{r['label']}: RD {100*c['rd']:.2f} pp, Wald {100*c['wald'][0]:.2f} to {100*c['wald'][1]:.2f}; "
f"RR {c['rr']:.3f} ({c['rr_ci'][0]:.3f}-{c['rr_ci'][1]:.3f}) vs eTable 2 {rr[0]:.2f} ({rr[1]:.2f}-{rr[2]:.2f})")
print("\n=== Numbers needed to treat (vs DL), CI by inverting the Wald RD limits ===")
nnt = {}
for arm, key in (("VLM", ("DL", "VLM")), ("VLH", ("DL", "VLH"))):
c = res[key]
est, lo, hi = 1 / c["rd"], 1 / c["wald"][1], 1 / c["wald"][0]
nnt[arm] = (est, lo, hi)
r = nnt_rep[arm]
print(f" {arm}: reported {r[0]} ({r[1]}-{r[2]}); recomputed {est:.1f} ({lo:.1f}-{hi:.1f}); "
f"inverting the reported RD CI gives {100/reported[key]['hi']:.1f}-{100/reported[key]['lo']:.1f}")
# ---- Figures -----------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: risk differences, reported vs recomputed 95% CI, one linear axis
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
style(ax)
keys = list(reported)
for i, k in enumerate(keys):
y = len(keys) - 1 - i
r, c = reported[k], res[k]
yr, yc = y + 0.17, y - 0.17
ax.errorbar(r["rd"], yr, xerr=[[r["rd"] - r["lo"]], [r["hi"] - r["rd"]]], fmt="D", ms=8,
color=ORANGE, mec=ORANGE, elinewidth=2, capsize=4, capthick=2, zorder=3)
ax.errorbar(100 * c["rd"], yc, xerr=[[100 * (c["rd"] - c["wald"][0])], [100 * (c["wald"][1] - c["rd"])]],
fmt="o", ms=8, color=BLUE, elinewidth=2, capsize=4, capthick=2, zorder=3)
ax.text(14.6, yr, f"reported {r['lo']:.2f} to {r['hi']:.2f}", va="center", ha="left",
fontsize=10, color=INK)
ax.text(14.6, yc, f"recomputed {100*c['wald'][0]:.2f} to {100*c['wald'][1]:.2f}", va="center",
ha="left", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=1, zorder=1)
ax.axvline(-5, color=INK2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(0.15, 2.62, "no difference", fontsize=10, color=INK2, va="bottom")
ax.text(-4.85, 2.62, "noninferiority\nmargin, −5", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(range(len(keys)))
ax.set_yticklabels([reported[k]["label"] for k in reversed(keys)], fontsize=10.5, color=INK)
ax.set_xlim(-6, 14.3)
ax.set_ylim(-0.6, 3.15)
ax.set_xlabel("Difference in first-pass success, percentage points (95% CI)")
h1 = ax.errorbar([], [], xerr=[], fmt="D", ms=8, color=ORANGE, elinewidth=2, capsize=4,
label="Reported (Abstract and Results)")
h2 = ax.errorbar([], [], xerr=[], fmt="o", ms=8, color=BLUE, elinewidth=2, capsize=4,
label="Recomputed from Table 2 counts (unpooled Wald)")
ax.set_xticks([-5, 0, 5, 10])
fig.legend(handles=[h1, h2], loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False,
fontsize=10, handletextpad=0.4, columnspacing=1.2)
fig.subplots_adjust(left=0.14, right=0.71, top=0.97, bottom=0.27)
save(fig, "fig1")
# Figure 2: NNT vs DL, reported vs recomputed 95% CI, log axis
fig, ax = plt.subplots(figsize=(7.2, 2.7), facecolor=BG)
style(ax)
ax.set_xscale("log")
arms = ["VLM", "VLH"]
for i, arm in enumerate(arms):
y = len(arms) - 1 - i
r, c = nnt_rep[arm], nnt[arm]
yr, yc = y + 0.17, y - 0.17
ax.errorbar(r[0], yr, xerr=[[r[0] - r[1]], [r[2] - r[0]]], fmt="D", ms=8, color=ORANGE,
elinewidth=2, capsize=4, capthick=2, zorder=3)
ax.errorbar(c[0], yc, xerr=[[c[0] - c[1]], [c[2] - c[0]]], fmt="o", ms=8, color=BLUE,
elinewidth=2, capsize=4, capthick=2, zorder=3)
ax.text(195, yr, f"reported {r[0]} ({r[1]} to {r[2]})", va="center", fontsize=10)
ax.text(195, yc, f"recomputed {c[0]:.1f} ({c[1]:.1f} to {c[2]:.1f})", va="center", fontsize=10)
ax.set_xlim(5, 180)
ax.set_xticks([5, 10, 20, 50, 100])
ax.set_xticklabels(["5", "10", "20", "50", "100"])
ax.set_yticks(range(len(arms)))
ax.set_yticklabels([f"{a} vs DL" for a in reversed(arms)], fontsize=10.5, color=INK)
ax.set_ylim(-0.6, 1.6)
ax.set_xlabel("Number needed to treat (95% CI), log scale")
h1 = ax.errorbar([], [], xerr=[], fmt="D", ms=8, color=ORANGE, elinewidth=2, capsize=4,
label="Reported (Discussion)")
h2 = ax.errorbar([], [], xerr=[], fmt="o", ms=8, color=BLUE, elinewidth=2, capsize=4,
label="Recomputed: 1 / Wald RD limits")
fig.legend(handles=[h1, h2], loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.14, right=0.63, top=0.96, bottom=0.38)
save(fig, "fig2")
print("\nFigures written to", HERE)
Paper states a 69% hospitalization reduction; its own hazard and risk ratios imply about 40%
JAMA Netw Open (2020) · doi:10.1001/jamanetworkopen.2020.6445 · PMC7265099
Summary
In three places (the Abstract, the Results and the first paragraph of the Discussion), this randomized trial of social-needs navigation says that children in the navigator group were "69% less likely to be hospitalized", or had a "69% reduced risk", than children in the active control group. The paper's own estimates do not support that figure. The hazard ratio quoted alongside it, 0.59 (95% CI 0.38-0.94), means a 41% lower hazard, and the relative risk of 0.60, which recomputes exactly from the reported counts of 29/637 vs 50/663, means a 40% lower risk. A 69% reduction would require a ratio of 0.31, which lies outside every confidence interval the paper reports for this outcome. The direction and statistical significance of the finding are unaffected, but the headline size of the effect is overstated by about 1.7 times, in a paper that ties its conclusions to return-on-investment and cost-saving arguments.
What the paper reports
Abstract, Results (as archived in PMC; "05% CI" is in the original):
Children enrolled in the in-person navigator group had a decreased risk of hospitalization within 12 months (hazard ratio, 0.59; 05% CI, 0.38-0.94; P = .03), making them 69% less likely to be hospitalized.
Results, "Acute Care Utilization by Navigation Intervention and Active Control Group", third paragraph:
Using Cox proportional hazard regression, we found that during the study follow-up period, children enrolled in the navigator intervention group had a decreased risk of being hospitalized (hazard ratio, 0.59; 95% CI, 0.38-0.94; P = .03), making them 69% less likely to be hospitalized than children in the active control group.
Discussion, first paragraph:
We found that during the 12 months after study enrollment, children enrolled in the trial’s patient navigator intervention had a 69% reduced risk of hospitalization than children in the active control group, but the intervention had no effect on ED visits. This translated to only 4.6% of patient navigator families having their child hospitalized vs 7.5% among the active control group.
Results, same section, first paragraph (the raw counts and relative risk):
However, significantly fewer children from the in-person navigation group (29 [4.6%]) were admitted to the hospital during the year following enrollment compared with children from the active control group (50 [7.5%]; risk difference, -3.0%; 95% CI, -5.6% to -0.4%; relative risk, 0.60; 95% CI, 0.39 to 0.94).
Table 2, row "Hospitalized within 1 y": Model 1 (unadjusted, N = 1300) HR 0.59 (0.38-0.94), P = .03; Model 2 (adjusted for child, caregiver and site covariates) HR 0.56 (0.35-0.90), P = .02; Model 3 (additionally adjusted for poverty, n = 1116) HR 0.59 (0.35-0.99), P = .046.
The title, Key Points, abstract Conclusions and Table 2 give only the direction of the effect or the correct ratios; none of them uses the 69% figure.
What the paper's own numbers show
Method. Every effect measure is a ratio of the navigator group to the active control group, so the percentage reduction it implies is 1 − ratio, and the 95% CI converts the same way (1 − upper bound to 1 − lower bound). I applied this to the three Table 2 hazard ratios and to the relative risk. I recomputed the relative risk, risk difference and an odds ratio from the 2×2 counts given in the Results (Wald intervals on the log scale for ratios), and recovered P values from the printed CIs as a consistency check. I then looked for any quantity the paper reports, or that its numbers allow, that equals 69%. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Stated reduction in hospitalization | Abstract; Results para 3; Discussion para 1 | "69% less likely" / "69% reduced risk" | — |
| Hospitalized, navigator group | Results para 1; Discussion | 29/637 (4.6%) | 4.55% |
| Hospitalized, active control | Results para 1; Discussion | 50/663 (7.5%) | 7.54% |
| Risk difference (95% CI) | Results para 1 | −3.0% (−5.6% to −0.4%) | −2.99 points (−5.6 to −0.4); agrees |
| Relative risk (95% CI) | Results para 1 | 0.60 (0.39-0.94) | 0.604 (0.39-0.94); agrees. 39.6% lower risk (5.9% to 61.3%) |
| Odds ratio (95% CI) | not reported | — | 0.585 (0.37-0.94); 41.5% lower odds (6.3% to 63.5%) |
| Cox HR, Model 1, unadjusted | Abstract; Results para 3; Table 2 | 0.59 (0.38-0.94), P = .03 | 41% lower hazard (6% to 62%); P from CI = 0.022 |
| Cox HR, Model 2, adjusted | Table 2 | 0.56 (0.35-0.90), P = .02 | 44% lower hazard (10% to 65%); P from CI = 0.016 |
| Cox HR, Model 3, + poverty | Table 2; Results para 3 | 0.59 (0.35-0.99), P = .046 | 41% lower hazard (1% to 65%); P from CI = 0.047 |
| Ratio needed for a 69% reduction | — | — | 0.31; outside all three HR CIs and the RR CI |
| Hospitalizations per child, ratio | Results para 1 | 35 vs 55 total; means 0.05 vs 0.08 | (35/637)/(55/663) = 0.662 (33.8% fewer); 0.05/0.08 = 0.625 (37.5% fewer) |
| Inverse of Model 1 HR | derived | — | 1/0.59 = 1.695: the control group's hazard is 69% higher |
All the paper's printed ratios recompute correctly from its counts and CIs; the only figure that does not follow from them is the 69%. Across every measure the reduction is 34% to 44%, and the upper ends of the converted 95% CIs reach 61% to 65%, short of 69%. The single piece of arithmetic on the paper's numbers that yields 69% is 1/0.59 − 1 = 0.69, which describes how much higher the control group's hazard is than the navigator group's. It is not the navigator group's reduction (that is 1 − 0.59 = 0.41), because a ratio's reciprocal changes which group is the reference.
Figure
Figure 1. Percentage reduction in 12-month hospitalization for the navigator group relative to the active control group. The orange dot and dashed line mark the 69% stated in the Abstract, Results and Discussion. Blue dots are the reductions implied by the paper's own estimates (1 − ratio): the three Cox hazard ratios from Table 2, the relative risk from the Results (recomputed from 29/637 vs 50/663), and an odds ratio from the same counts, which the paper does not report. Blue bars are the corresponding 95% CIs converted the same way; for the hazard ratios and relative risk these are the paper's printed intervals. The thin line at 0% marks no effect. The axis runs from 0% to 100% reduction.
Figure 2. The hazard ratio of 0.59 compares the navigator group with the control group. Subtracting it from 1 (blue) gives the navigator group's reduction, 41%. Taking its reciprocal and subtracting 1 (orange) gives 0.69, the figure printed in the paper, but that quantity is the control group's excess hazard relative to the navigator group, a statement with the reference group reversed.
Why it matters
- The headline effect size is overstated in the most-read parts of the paper. The 69% figure appears in the Abstract and is repeated in the Results and the opening of the Discussion. Readers who take the effect size from those sentences receive a reduction about 1.7 times the one the data support (69% vs about 41%). The figure has been repeated in press coverage of the trial.
- The conclusions lean on the size of the effect. The Discussion says the findings "should inform health systems’ calculations on the return on investment of different social care programs", citing an average cost of "$13 400" per nonbirth pediatric hospitalization, and the Conclusions say such programs "could potentially result in large reductions in inpatient stays and associated cost savings". Any savings estimate built on a 69% reduction would be inflated by a factor of about 1.7 relative to the paper's own hazard ratio.
- What is not affected. The direction and significance of the hospitalization effect stand (P = .03 unadjusted, P = .046 fully adjusted, Pearson χ² P = 0.024 from the counts). The absolute risk difference, −3.0 percentage points (4.6% vs 7.5%), is reported correctly, as are all hazard ratios, the relative risk and the null finding for ED visits (HR 0.96).
Explanations considered
- A different model or adjustment. Does not account for it. The unadjusted, adjusted and poverty-adjusted hazard ratios imply 41%, 44% and 41% reductions. The sentence giving 69% is attached directly to the unadjusted HR 0.59 (0.38-0.94).
- A different effect measure. Does not account for it. The relative risk (0.60, 40% lower risk), an odds ratio from the same counts (0.585, 41.5% lower odds), the ratio of hospitalization counts per child (0.662, 33.8% fewer) and the ratio of mean hospitalizations (0.625, 37.5% fewer) all fall between 34% and 44%.
- The 4-month or multiple-imputation analyses in Supplement 2. Do not account for it. The 12-month multiple-imputation hazard ratios in eTable 3 are 0.61 (0.39-0.97), 0.60 (0.38-0.96) and 0.61 (0.38-0.96), implying 39-40% reductions. The 4-month hazard ratios in eTable 2 are 0.44, 0.43 and 0.40 (56-60% reductions), and their wide CIs do include 0.31; but the sentences giving 69% refer to hospitalization "within 12 months" or "during the study follow-up period" and quote the 12-month HR, and the paper describes the 4-month effects only as "similar". The supplement never uses the 69% figure.
- A confidence limit rather than the point estimate. Does not account for it. The most favorable converted limit of any 12-month estimate is 65% (Models 2 and 3), not 69%, and the sentence presents 69% as the effect, not as a bound.
- The reciprocal of the hazard ratio. Matches the number but not the statement. 1/0.59 − 1 = 0.69 means the control group's hazard was 69% higher than the navigator group's. That is a different claim from "navigator children were 69% less likely to be hospitalized", which would require HR 0.31. Nothing in the Methods, table footnotes or legends defines "less likely" in a way that would make the two equivalent.
- A text conversion or archiving artifact. Ruled out. The same three sentences appear identically on the publisher's version of record, the PMC HTML page and the Europe PMC full-text XML.
Two minor, unrelated inconsistencies were noticed and are not part of the claim. The abstract prints "05% CI" for 95% CI. The Results give P = .81 next to the unadjusted ED-visit HR of 0.96 (0.80-1.14), whereas Table 2 gives P = .62 for that model and .81 for Model 3; the CI implies P ≈ 0.65, consistent with Table 2.
Status
- Version of record. The publisher's article page (reached through the DOI at jamanetwork.com, checked 2026-10-24) contains the same three sentences, the same "05% CI" typo in the abstract, and the same Table 2 values as the PMC archive. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The Europe PMC supplementary bundle holds Supplement 1 (trial protocol), Supplement 2 (eTables 1-3) and Supplement 3 (data sharing statement). Supplement 2 was read in full: its hospitalization hazard ratios are those quoted above and it does not use the 69% figure. No individual-level data are provided, so the Cox models could not be refitted.
- Corrections and comments. As of 2026-10-24 no correction, erratum, expression of concern or retraction was found.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices, and the Crossref record has no update-to or updated-by entries. The PubMed record (PMID 32478849) links only to an invited commentary (doi:10.1001/jamanetworkopen.2020.6456), which cites the raw rates (about 5% vs 8%) and does not discuss the 69% figure. A PubMed erratum search returned no items, and neither the publisher nor the PMC page carries a correction notice. Web searches for a correction and for PubPeer or Retraction Watch coverage found nothing. Caveats: the Europe PMC search API returned HTTP 503 on every attempt, and PubPeer's API required human verification, so a PubPeer thread cannot be fully excluded. - Sources. PMC article PMC7265099; Europe PMC full-text XML and supplementary files for PMC7265099 (
jamanetwopen-3-e206445-s002.pdf); https://doi.org/10.1001/jamanetworkopen.2020.6445; Crossref; PubMed; PubPeer; web search.
Suggested correction
In the Abstract, replace
making them 69% less likely to be hospitalized.
with
corresponding to a 41% lower hazard of hospitalization.
In the Results ("Acute Care Utilization..."), replace
making them 69% less likely to be hospitalized than children in the active control group.
with
corresponding to a 41% lower hazard of hospitalization than children in the active control group.
In the Discussion, first paragraph, replace
had a 69% reduced risk of hospitalization than children in the active control group
with
had a 41% lower hazard of hospitalization (relative risk, 0.60, a 40% lower risk) than children in the active control group
The authors may also wish to correct "05% CI" to "95% CI" in the Abstract, and to give the ED-visit P value in the Results as P = .62 to match Table 2 Model 1.
Recomputation script (python recompute.py.txt)
"""Recompute the hospitalization effect size in PMC7265099
(JAMA Netw Open 2020, doi:10.1001/jamanetworkopen.2020.6445).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PMC (Results, first and third
paragraphs of "Acute Care Utilization..."; Table 2) and from Supplement 2 (eTables 2-3).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (paper)
A, N1 = 29, 637 # children hospitalized within 12 mo, navigator group
C, N0 = 50, 663 # children hospitalized within 12 mo, active control group
HOSP1, HOSP0 = 35, 55 # total hospitalizations (navigator, control)
MEAN1, MEAN0 = 0.05, 0.08 # mean hospitalizations per child as printed
STATED = 69.0 # "69% less likely" / "69% reduced risk"
RR_REPORTED = (0.60, 0.39, 0.94)
RD_REPORTED = (-3.0, -5.6, -0.4)
TABLE2 = { # hospitalized within 1 y: HR, lower, upper, printed P
"Model 1 (unadjusted)": (0.59, 0.38, 0.94, ".03"),
"Model 2 (adjusted)": (0.56, 0.35, 0.90, ".02"),
"Model 3 (+ poverty)": (0.59, 0.35, 0.99, ".046"),
}
ED_TABLE2 = {"Model 1": (0.96, 0.80, 1.14, ".62"), "Model 3": (0.98, 0.80, 1.19, ".81")}
ETABLE2 = {"Model 1": (0.44, 0.22, 0.87), "Model 2": (0.43, 0.22, 0.86), "Model 3": (0.40, 0.18, 0.89)}
ETABLE3 = {"Model 1": (0.61, 0.39, 0.97), "Model 2": (0.60, 0.38, 0.96), "Model 3": (0.61, 0.38, 0.96)}
def p_from_ci(est, lo, hi):
se = (np.log(hi) - np.log(lo)) / (2 * Z)
return 2 * stats.norm.sf(abs(np.log(est) / se))
def red(x):
"""percent reduction implied by a ratio x"""
return 100 * (1 - x)
print("== 2x2 table, hospitalized within 12 months ==")
p1, p0 = A / N1, C / N0
print(f"navigator {A}/{N1} = {100*p1:.2f}% control {C}/{N0} = {100*p0:.2f}%")
rd = p1 - p0
se_rd = np.sqrt(p1 * (1 - p1) / N1 + p0 * (1 - p0) / N0)
print(f"risk difference = {100*rd:.2f} pp (95% CI {100*(rd-Z*se_rd):.1f} to {100*(rd+Z*se_rd):.1f}); "
f"reported {RD_REPORTED}")
rr = p1 / p0
se_lrr = np.sqrt(1 / A - 1 / N1 + 1 / C - 1 / N0)
rr_lo, rr_hi = np.exp(np.log(rr) - Z * se_lrr), np.exp(np.log(rr) + Z * se_lrr)
print(f"relative risk = {rr:.3f} (95% CI {rr_lo:.2f}-{rr_hi:.2f}); reported {RR_REPORTED}")
print(f" -> {red(rr):.1f}% lower risk (95% CI {red(rr_hi):.1f}% to {red(rr_lo):.1f}%)")
OR = (A / (N1 - A)) / (C / (N0 - C))
se_lor = np.sqrt(1 / A + 1 / (N1 - A) + 1 / C + 1 / (N0 - C))
or_lo, or_hi = np.exp(np.log(OR) - Z * se_lor), np.exp(np.log(OR) + Z * se_lor)
print(f"odds ratio = {OR:.3f} (95% CI {or_lo:.2f}-{or_hi:.2f}) -> {red(OR):.1f}% lower odds "
f"(95% CI {red(or_hi):.1f}% to {red(or_lo):.1f}%)")
chi = stats.chi2_contingency([[A, N1 - A], [C, N0 - C]], correction=False)
print(f"Pearson chi-square P = {chi[1]:.4f}")
print(f"ratio of hospitalization counts per child = ({HOSP1}/{N1})/({HOSP0}/{N0}) = "
f"{(HOSP1/N1)/(HOSP0/N0):.3f} -> {red((HOSP1/N1)/(HOSP0/N0)):.1f}% fewer")
print(f"ratio of printed means = {MEAN1}/{MEAN0} = {MEAN1/MEAN0:.3f} -> {red(MEAN1/MEAN0):.1f}% fewer")
print("\n== Table 2, hospitalized within 1 y (Cox HR) ==")
for k, (hr, lo, hi, p) in TABLE2.items():
print(f"{k}: HR {hr} ({lo}-{hi}) -> {red(hr):.0f}% lower hazard (95% CI {red(hi):.0f}% to {red(lo):.0f}%); "
f"P printed {p}, P from CI = {p_from_ci(hr, lo, hi):.3f}; 1/HR = {1/hr:.3f} "
f"(control hazard {100*(1/hr-1):.1f}% higher)")
print(f"ratio needed for a {STATED:.0f}% reduction = {1-STATED/100:.2f}")
for k, (hr, lo, hi, p) in TABLE2.items():
print(f" inside {k} CI ({lo}-{hi})? {lo <= 1-STATED/100 <= hi}")
print(f" inside RR CI ({RR_REPORTED[1]}-{RR_REPORTED[2]})? {RR_REPORTED[1] <= 1-STATED/100 <= RR_REPORTED[2]}")
print("\n== Supplement 2 ==")
for name, tab in (("eTable 2 (4 months)", ETABLE2), ("eTable 3 (multiple imputation, 1 y)", ETABLE3)):
for k, (hr, lo, hi) in tab.items():
print(f"{name} {k}: HR {hr} ({lo}-{hi}) -> {red(hr):.0f}% lower hazard; "
f"0.31 inside CI? {lo <= 0.31 <= hi}")
print("\n== Minor: ED-visit P values ==")
for k, (hr, lo, hi, p) in ED_TABLE2.items():
print(f"ED {k}: HR {hr} ({lo}-{hi}) printed P {p}; P from CI = {p_from_ci(hr, lo, hi):.2f}")
print("Results text gives P = .81 next to the Model 1 estimate 0.96 (0.80-1.14).")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"svg.fonttype": "none"})
rows = [("Hazard ratio, Model 1 (unadjusted)\nTable 2: HR 0.59 (0.38-0.94)", 0.59, 0.38, 0.94),
("Hazard ratio, Model 2 (adjusted)\nTable 2: HR 0.56 (0.35-0.90)", 0.56, 0.35, 0.90),
("Hazard ratio, Model 3 (+ poverty)\nTable 2: HR 0.59 (0.35-0.99)", 0.59, 0.35, 0.99),
(f"Relative risk, 29/637 vs 50/663\nResults: RR 0.60 (0.39-0.94)", rr, rr_lo, rr_hi),
(f"Odds ratio, same counts\nnot reported: OR {OR:.2f} ({or_lo:.2f}-{or_hi:.2f})", OR, or_lo, or_hi)]
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
ax.set_facecolor(BG)
n = len(rows)
ys = np.arange(n, 0, -1) # 5..1
top = n + 1.1
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.text(0.8, 0.45, "no effect", color=INK2, fontsize=10, va="center")
ax.axvline(STATED, color=ORANGE, lw=2, ls=(0, (4, 3)), zorder=2)
ax.plot([STATED], [top], "o", color=ORANGE, ms=10, zorder=4, clip_on=False)
ax.annotate("Stated in Abstract, Results, Discussion:\n\"69% less likely to be hospitalized\"",
xy=(STATED, top), xytext=(STATED - 2, top), ha="right", va="center",
color=ORANGE, fontsize=10.5, fontweight="bold")
for (lab, est, lo, hi), y in zip(rows, ys):
ax.plot([red(hi), red(lo)], [y, y], color=BLUE, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([red(est)], [y], "o", color=BLUE, ms=9, zorder=4)
ax.text(red(est), y + 0.2, f"{red(est):.0f}%", color=BLUE, ha="center", va="bottom",
fontsize=10.5, fontweight="bold")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(0.1, top + 0.6)
ax.set_xlim(-5, 100)
ax.set_xticks(range(0, 101, 20))
ax.set_xticklabels([f"{t}%" for t in range(0, 101, 20)])
ax.set_xlabel("Reduction in 12-month hospitalization, navigator vs active control\n"
"(1 − ratio; bars are the paper's 95% CIs converted the same way)", color=INK)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D # noqa: E402
h = [Line2D([], [], color=ORANGE, marker="o", ms=9, lw=2, ls=(0, (4, 3))),
Line2D([], [], color=BLUE, marker="o", ms=9, lw=2)]
ax.legend(h, ["Reduction stated in the paper's text",
"Reduction implied by the paper's own estimates (95% CI)"],
loc="upper center", bbox_to_anchor=(0.32, -0.2), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# fig2: the two ways to turn HR 0.59 into a percentage
fig, ax = plt.subplots(figsize=(7.2, 2.9), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 10)
ax.set_ylim(0, 4)
ax.axis("off")
def box(x, y, w, h, text, edge, color=INK, bold=False):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.12",
fc=BG, ec=edge, lw=2))
ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", color=color, fontsize=10.5,
fontweight="bold" if bold else "normal")
def arrow(x0, y0, x1, y1, color):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=14,
color=color, lw=2))
box(0.1, 1.4, 2.4, 1.2, "Table 2, Model 1\nHR = 0.59\n(navigator vs control)", INK2)
box(3.0, 2.4, 3.45, 1.3, "1 − 0.59 = 0.41\nnavigator group's hazard\nis 41% lower", BLUE, BLUE, True)
box(3.0, 0.3, 3.45, 1.3, "1 ÷ 0.59 − 1 ≈ 0.69\ncontrol group's hazard\nis 69% higher", ORANGE, ORANGE, True)
arrow(2.55, 2.25, 2.95, 2.75, BLUE)
arrow(2.55, 1.75, 2.95, 1.25, ORANGE)
ax.text(6.65, 3.05, "Recomputed: the reduction\nthe HR implies for\nnavigator children", va="center",
color=BLUE, fontsize=10)
ax.text(6.65, 0.95, "Equals the paper's \"69%\", but\nit is the control group's\n"
"excess hazard, not a\nreduction for navigator children", va="center", color=ORANGE, fontsize=10)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nwrote fig1.svg/png and fig2.svg/png")
Abstract and Results text attach the 30-day follow-up estimate or CI to 7-day follow-up
JAMA Netw Open (2022) · doi:10.1001/jamanetworkopen.2022.18730 · PMC9237790
Summary
The abstract reports that patients in high-telemental-health-use counties were 13.7 percentage points (95% CI, 5.1-22.3) (6.5% relative increase) more likely to have outpatient follow-up within 7 days of a mental health hospitalization. The Results text gives the same outcome as 2.2 percentage points with the same interval, 5.1-22.3. In the paper's Table 2, the 7-day estimate is 2.2 (−1.0 to 5.5), a 6.5% relative change. The 30-day estimate is 13.7 (5.1 to 22.3), a 20.4% relative change. So the abstract pairs the 30-day estimate and interval with the 7-day relative change, and the Results text pairs the 7-day estimate with the 30-day interval, which does not contain it. According to Table 2, the 7-day association is not statistically significant (approximate z ≈ 1.33, p ≈ 0.18), while the abstract presents it as a large, significant effect.
What the paper reports
Abstract, Results:
Compared with no telemental health care in the county, patients in high-use counties were 1.2 percentage points (95% CI, 0.81-1.60 percentage points) (8.0% relative increase) more likely to have a minimum number of specialty mental health service visits, 13.7 percentage points (95% CI, 5.1-22.3 percentage points) (6.5% relative increase) more likely to have outpatient follow-up within 7 days of a mental health hospitalization, and 0.47 percentage points (95% CI, 0.25-0.69 percentage points) (7.6% relative increase) more likely to be hospitalized in a year.
Results, section "Changes in Likelihood of Receiving Other Forms of Utilization":
In our main model, compared with patients in the no telemental health service use counties, patients in high telemental health service use counties were 1.2 percentage points (95% CI, 0.81-1.60 percentage points) (8.0% relative increase) more likely to have a minimum of 4 specialty mental health service visits, 2.2 percentage points (95% CI, 5.1-22.3 percentage points) (6.5% relative increase) more likely to have outpatient follow-up within 7 days of a mental health hospitalization, and 0.47 percentage points (95% CI, 0.25-0.69 percentage points) (7.6% relative increase) more likely to be hospitalized in a year (Table 2).
Table 2, "Association Between Telemental Health Use in Nonmetropolitan Counties and Outcomes for Patients With Serious Mental Illness", excerpt. Columns: reference rate in 2010 in no use, %; then for low, moderate and high use, % difference compared with reference group (95% CI) and % change relative to no use group:
Follow-up after hospitalization, %
7 d | 34.0 | -0.3 (-2.0 to 1.5) | -0.8 | 1.1 (-1.5 to 3.7) | 3.2 | 2.2 (-1.0 to 5.5) | 6.5
30 d | 67.0 | 3.4 (-1.3 to 8.1) | 5.1 | 11.0 (4.1 to 17.9) | 16.4 | 13.7 (5.1 to 22.3) | 20.4
Methods, section "Quality Measures":
Consistent with a National Committee for Quality Assurance measure, we captured what fraction of patients with mental health hospitalizations had an outpatient mental health service visit within 7 or 30 days of discharge.
Methods, section "Statistical Analysis":
Statistical analysis was done with 2-sided hypothesis tests, with α = .05 as the significance threshold.
Conclusions:
In this cohort study, at a county level, among nonmetropolitan beneficiaries with SMI, we found that greater use of telemental health service was associated with modest increases in contact with outpatient specialty mental health professionals and a greater likelihood of follow-up after hospitalization.
What the paper's own numbers show
Method. I took the high-use column of Table 2 (difference, 95% CI and relative change against the 2010 no-use reference rate) for every outcome. For each row I checked that the relative change equals difference ÷ reference rate and that the difference sits at the midpoint of its interval. I then matched each number in the two text statements about 7-day follow-up to the Table 2 row it reproduces. To judge significance I took the standard error from the printed interval, SE = (upper − lower) ÷ (2 × 1.96), and formed an approximate Wald z = estimate ÷ SE. I repeated this over every combination of ±0.05 rounding in the three printed values. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| 7-day follow-up, high vs no use | Table 2 | 2.2 (−1.0 to 5.5); 6.5% | CI midpoint 2.25; 2.2 ÷ 34.0 = 6.47%. Consistent. CI includes 0 |
| 30-day follow-up, high vs no use | Table 2 | 13.7 (5.1 to 22.3); 20.4% | CI midpoint 13.70; 13.7 ÷ 67.0 = 20.45%. Consistent |
| 7-day follow-up as stated in the abstract | Abstract, Results | 13.7 (5.1-22.3); 6.5% relative | 13.7 and 5.1-22.3 are the 30-day row. 13.7 ÷ 34.0 = 40.3%, not 6.5%. 6.5% × 34.0 = 2.21 pp, the 7-day row |
| 7-day follow-up as stated in the Results text | Results, "Changes in Likelihood…" | 2.2 (5.1-22.3); 6.5% relative | 2.2 and 6.5% are the 7-day row. 5.1-22.3 is the 30-day interval (midpoint 13.70); 2.2 lies 2.9 pp below its lower bound |
| Abstract 7-day estimate ÷ Table 2 7-day estimate | Abstract vs Table 2 | 13.7 vs 2.2 | ratio 6.2 |
| Approximate test, 7-day | Table 2 interval | not reported | SE 1.66 pp; z = 1.33, p = 0.18 (over rounding: z 1.28 to 1.38, p 0.17 to 0.20) |
| Approximate test, 30-day | Table 2 interval | not reported | SE 4.39 pp; z = 3.12, p = 0.0018 (over rounding: p 0.0016 to 0.0020) |
| Visits estimate in the same two sentences | Abstract; Results vs Table 2 | 1.2 (0.81-1.60); 8.0% | Table 2: 1.2 (0.8 to 1.6); 8.0. Matches after rounding |
| Hospitalization estimate in the same two sentences | Abstract; Results vs Table 2 | 0.47 (0.25-0.69); 7.6% | Table 2: 0.5 (0.3 to 0.7); 7.6. Matches after rounding |
The other two estimates in each sentence agree with the Table 2 high-use column, so both sentences report the Table 2 main model. Within the follow-up statement, no single row of Table 2 gives all three numbers. In the abstract, the estimate and interval come from the 30-day row and the relative change from the 7-day row. In the Results text, the estimate and relative change come from the 7-day row and the interval from the 30-day row. Table 2 itself is internally consistent: each relative change equals difference ÷ reference rate, and each difference sits at the midpoint of its interval.
Figure
Figure 1. Shortened excerpts of the two text statements about 7-day follow-up (top) and the two follow-up rows of Table 2, high-use vs no-use column (bottom). Numbers stated in the text are in orange, and each carries a tag naming the Table 2 row it matches. The relative changes recomputed as difference ÷ reference rate are in blue. In the abstract, the estimate and CI match the 30-day row and the 6.5% matches the 7-day row; 13.7 ÷ 34.0 would be 40.3%. In the Results text, the estimate and 6.5% match the 7-day row and the CI matches the 30-day row, which puts the estimate outside its own interval.
Figure 2. The 7-day follow-up estimates on one axis in percentage points, with 95% CIs; the thin vertical line is no difference (0). Orange: the two text statements, the abstract's 13.7 (5.1 to 22.3) and the Results text's 2.2 with the interval 5.1 to 22.3, which does not contain it. Blue diamond: the 7-day difference implied by the text's own 6.5% relative change and the 34.0% reference rate (2.21 pp). Black: Table 2's 7-day row, 2.2 (−1.0 to 5.5), which crosses 0 (approximate z ≈ 1.33, p ≈ 0.18). Open grey: Table 2's 30-day row, 13.7 (5.1 to 22.3), shown for reference. The abstract's 7-day interval is identical to the 30-day one.
Why it matters
- A quantified abstract result is misstated. The abstract reports one of its three numeric findings as a 13.7-point, statistically significant increase in 7-day post-hospitalization follow-up. The paper's own table gives 2.2 points, about one-sixth as large, with an interval (−1.0 to 5.5) that includes zero. At the paper's stated threshold (2-sided, α = .05), the 7-day association is not significant.
- The Results text gives the same wrong impression of significance. Its point estimate is correct, but the interval it gives (5.1-22.3) excludes zero, whereas the correct interval (−1.0 to 5.5) does not.
- The qualitative conclusion rests on narrower evidence than the abstract suggests. The Key Points and Conclusions say greater telemental health use was associated with "greater likelihood of follow-up after hospitalization", and the Discussion reports increases in patients receiving "follow-up care". That statement is still supported by the 30-day measure: 13.7 points (5.1 to 22.3), 20.4% relative, p ≈ 0.002. It is not supported by the 7-day measure. The Methods define both windows as parts of the same quality measure. A reader of the abstract alone would conclude that both were clearly improved.
- The 30-day result is not reported in the abstract or the Results text. The 13.7-point figure appears in the abstract under the 7-day label, so the 30-day result appears only in Table 2.
- The direction of the 7-day estimate is unchanged. It is positive in every version. The other estimates in both sentences, the remaining rows of Table 2, and the paper's other conclusions (visits, hospitalization, medication adherence) are not affected.
Explanations considered
- A different model, population or subgroup. Not supported. The Results sentence says "In our main model" and cites Table 2. Its other two estimates, and the abstract's, match the Table 2 high-use column after rounding: 1.2 (0.81-1.60), 8.0%, and 0.47 (0.25-0.69), 7.6%. Table 3, the subgroup and sensitivity table, has no follow-up rows. No table in the paper reports a 7-day estimate of 13.7 or a 7-day interval of 5.1 to 22.3.
- A different reference rate or denominator for the relative change. Not able to explain it. 6.5% equals only 2.2 ÷ 34.0, the 7-day row. 13.7 gives 20.4% against the 30-day reference (67.0) and 40.3% against the 7-day reference (34.0). No denominator in the table gives 13.7 together with 6.5%.
- A different interval type, such as a one-sided or non-symmetric interval. Ruled out for the Results text. An interval that does not contain its own point estimate (2.2 against 5.1 to 22.3) cannot be a confidence interval for that estimate of any kind. The Table 2 intervals are symmetric about their estimates to rounding (midpoints 2.25 and 13.70).
- The abstract meant 30-day follow-up. If so, the label "within 7 days" and the 6.5% relative increase are both wrong, since the 30-day relative change is 20.4%. If the abstract meant 7-day follow-up, then 13.7 and its interval are wrong. Either way, the three numbers in the abstract do not belong to one outcome.
- Table 2 is the version in error. Not supported. Each Table 2 row is internally consistent: each difference sits at the midpoint of its interval, and each relative change equals difference ÷ reference rate (6.47% and 20.45%). The text, by contrast, combines values from two rows, and in the Results text the estimate falls outside its interval.
- A text-conversion artifact. Ruled out. The publisher's full text, the PMC HTML page, the Europe PMC JATS XML and the PubMed abstract all carry the same numbers.
- A correction already issued. None found (see Status).
- Side observations, not part of the main finding. (1) The Results text says "a minimum of 4 specialty mental health service visits" and the abstract says "a minimum number", while the abstract's Main Outcomes says "a minimum of 2". The estimate (1.2, 8.0%) matches the Table 2 row "Per quarter in year" (at least 1 visit per quarter). (2) For hospitalization, 0.47 ÷ 6.0 = 7.83%. Even allowing for rounding of 0.47 and 6.0, the ratio is 7.69% to 7.98%, slightly above the reported 7.6%. The difference is small and does not affect any conclusion. (3) In the Subgroup Analyses text and Table 3, the dual-eligible medication estimate is given as "-0.01 months (95% CI, -0.02 to -0.09 months)", where the point estimate also lies outside the stated interval. That outcome is not part of this finding and was not examined further.
Status
- Version of record. I checked the publisher's full text (jamanetwork.com, article 2793628, reached through doi.org), the PMC HTML page and the Europe PMC full-text XML. All three carry the abstract sentence, the Results sentence and the Table 2 values quoted above. The PubMed record (PMID 35759264) carries the same abstract. Every quotation here was checked verbatim against the PMC-archived text with
src/show.py. The Table 2 excerpt is shown with the PMC text's hyphen-minus signs. - Supplements. The single supplement (eTable 1, antipsychotic and antimanic drugs; eTable 2, data dictionary for Table 1; eFigure, regression model; eTable 3, HCPCS codes) has no outcome estimates, so it cannot resolve the discrepancy. Its contents were judged from the caption in the article XML and the PMC page, because a direct download returned an HTML page rather than the PDF.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by entries. The PubMed record has no CommentsCorrections entries. Neither the publisher page nor the PMC page shows a correction notice. Web searches for a correction or erratum found only the article and news coverage, and a Retraction Watch search found nothing. The PubPeer API returned no record (HTTP 404) for the DOI; the PubPeer website itself could not be searched (HTTP 403 and a Cloudflare challenge). The Europe PMC search service was unavailable (HTTP 503) on every attempt. - Sources. PMC article PMC9237790; https://doi.org/10.1001/jamanetworkopen.2022.18730 (publisher full text); Europe PMC full-text XML for PMC9237790; PubMed 35759264 (E-utilities); Crossref; PubPeer; Retraction Watch; web search.
Suggested correction
In the abstract, Results, replace
13.7 percentage points (95% CI, 5.1-22.3 percentage points) (6.5% relative increase) more likely to have outpatient follow-up within 7 days of a mental health hospitalization
with
2.2 percentage points (95% CI, −1.0 to 5.5 percentage points) (6.5% relative increase; not statistically significant) more likely to have outpatient follow-up within 7 days, and 13.7 percentage points (95% CI, 5.1-22.3 percentage points) (20.4% relative increase) more likely to have outpatient follow-up within 30 days of a mental health hospitalization
In the Results section "Changes in Likelihood of Receiving Other Forms of Utilization", change "2.2 percentage points (95% CI, 5.1-22.3 percentage points)" to "2.2 percentage points (95% CI, −1.0 to 5.5 percentage points)". The authors may wish to add the 30-day estimate in the same way. They may also wish to qualify "greater likelihood of follow-up after hospitalization" in the Key Points and Conclusions to say that the association was significant for 30-day but not 7-day follow-up, and to reconcile "a minimum of 4" in the Results with the "≥1 visit per quarter" row it reports.
Recomputation script (python recompute.py.txt)
"""Recompute the follow-up-after-hospitalization estimates of PMC9237790
(JAMA Netw Open 2022, doi:10.1001/jamanetworkopen.2022.18730,
"Association Between Telemedicine Use in Nonmetropolitan Counties and Quality of Care
Received by Medicare Beneficiaries With Serious Mental Illness").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 2, high-use (>0.30) column vs no-use reference: reference rate in 2010, % difference
(95% CI) and % change relative to no use, for every outcome row.
Abstract, Results: "1.2 percentage points (95% CI, 0.81-1.60 ...) (8.0% relative increase)",
"13.7 percentage points (95% CI, 5.1-22.3 ...) (6.5% relative increase) ... within 7 days",
"0.47 percentage points (95% CI, 0.25-0.69 ...) (7.6% relative increase) ... hospitalized".
Results, "Changes in Likelihood of Receiving Other Forms of Utilization": the same sentence
with "2.2 percentage points (95% CI, 5.1-22.3 ...) (6.5% relative increase) ... within 7 days".
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs
# Table 2, high-use column: (reference rate 2010 %, difference pp, CI low, CI high, reported % change)
TABLE2_HIGH = {
"≥1 visit per quarter": (15.1, 1.2, 0.8, 1.6, 8.0),
"≥1 visit per semester": (27.7, 1.3, 0.9, 1.8, 4.7),
"Follow-up within 7 d": (34.0, 2.2, -1.0, 5.5, 6.5),
"Follow-up within 30 d": (67.0, 13.7, 5.1, 22.3, 20.4),
"ED visits (no admission)": (15.1, 0.5, -0.5, 1.5, 3.4),
"Hospitalization": (6.0, 0.5, 0.3, 0.7, 7.6),
"Months of medication": (5.9, -0.03, -0.10, 0.05, -0.47),
}
# Text statements about the 7-day outcome: (difference, CI low, CI high, relative %)
ABSTRACT_7D = (13.7, 5.1, 22.3, 6.5)
RESULTS_7D = (2.2, 5.1, 22.3, 6.5)
# The two other estimates in the same two sentences
TEXT_VISITS = (1.2, 0.81, 1.60, 8.0)
TEXT_HOSP = (0.47, 0.25, 0.69, 7.6)
ref7, d7, lo7, hi7, rel7 = TABLE2_HIGH["Follow-up within 7 d"]
ref30, d30, lo30, hi30, rel30 = TABLE2_HIGH["Follow-up within 30 d"]
# ---------------------------------------------------------------- 1. Table 2 internal consistency
print("=== 1. Table 2, high-use vs no-use: internal consistency of each row ===")
print(f"{'row':26s} {'ref':>5s} {'diff':>6s} {'95% CI':>15s} {'midpt':>6s} {'diff/ref':>8s} {'reported':>8s} {'CI excl 0':>9s}")
for k, (ref, d, lo, hi, rel) in TABLE2_HIGH.items():
print(f"{k:26s} {ref:5.1f} {d:6.2f} {f'({lo}, {hi})':>15s} {(lo + hi) / 2:6.2f} "
f"{100 * d / ref:7.2f}% {rel:7.2f}% {str(lo > 0 or hi < 0):>9s}")
# ---------------------------------------------------------------- 2. the text statements
print("\n=== 2. The 7-day statements in the Abstract and Results text, checked against Table 2 ===")
for name, (d, lo, hi, rel) in [("Abstract", ABSTRACT_7D), ("Results text", RESULTS_7D)]:
print(f"{name}: {d} pp (95% CI {lo}-{hi}), {rel}% relative")
print(f" CI midpoint = {(lo + hi) / 2:.2f}; point estimate inside its CI: {lo <= d <= hi}")
print(f" {d}/{ref7} (7-d reference) = {100 * d / ref7:.1f}% "
f"{d}/{ref30} (30-d reference) = {100 * d / ref30:.1f}%")
print(f"Relative change 6.5% x 7-d reference 34.0 = {0.065 * ref7:.2f} pp (Table 2 7-d: {d7})")
print(f"Relative change 6.5% x 30-d reference 67.0 = {0.065 * ref30:.2f} pp (Table 2 30-d: {d30})")
print(f"Ratio abstract 7-d estimate / Table 2 7-d estimate = 13.7/2.2 = {13.7 / 2.2:.1f}")
print(f"Interval 5.1-22.3: lower bound minus 2.2 = {5.1 - 2.2:.1f} pp (2.2 lies below the interval)")
from decimal import Decimal, ROUND_HALF_UP
r1 = lambda v: float(Decimal(str(v)).quantize(Decimal("0.1"), ROUND_HALF_UP))
print("\n=== 3. The other two estimates in the same sentences match Table 2 ===")
for name, (d, lo, hi, rel), key in [("visits", TEXT_VISITS, "≥1 visit per quarter"),
("hospitalization", TEXT_HOSP, "Hospitalization")]:
ref, td, tlo, thi, trel = TABLE2_HIGH[key]
print(f"{name}: text {d} ({lo}-{hi}) {rel}% vs Table 2 {td} ({tlo} to {thi}) {trel}% "
f"-> text rounded half-up to Table 2 precision: {r1(d)} ({r1(lo)} to {r1(hi)})")
print(f"hospitalization 0.47/6.0 = {100 * 0.47 / 6.0:.2f}%; 0.5/6.0 = {100 * 0.5 / 6.0:.2f}%; "
f"range allowed by rounding of 0.47 and 6.0: {100 * 0.465 / 6.05:.2f}% to {100 * 0.475 / 5.95:.2f}% "
f"(reported 7.6%)")
# ---------------------------------------------------------------- 4. approximate significance
print("\n=== 4. Approximate Wald tests from the Table 2 intervals (normal approximation) ===")
def wald(d, lo, hi):
se = (hi - lo) / (2 * Z975)
z = d / se
return se, z, 2 * stats.norm.sf(abs(z))
for name, (d, lo, hi) in [("7 d", (d7, lo7, hi7)), ("30 d", (d30, lo30, hi30))]:
se, z, p = wald(d, lo, hi)
# rounding sensitivity: each printed value is +/- half a unit in the last decimal
zs = [wald(d + a, lo + b, hi + c)[1] for a in (-.05, .05) for b in (-.05, .05) for c in (-.05, .05)]
ps = [2 * stats.norm.sf(abs(x)) for x in zs]
print(f"{name}: SE = {se:.2f} pp, z = {z:.2f}, two-sided p = {p:.4f}; "
f"over rounding of the printed values z in [{min(zs):.2f}, {max(zs):.2f}], "
f"p in [{min(ps):.4f}, {max(ps):.4f}]")
se_t, z_t, p_t = wald(13.7, 5.1, 22.3)
print(f"Abstract's 7-day statement read at face value: z = {z_t:.2f}, p = {p_t:.4f}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- Figure 1: two text panels and the Table 2 rows, joined by annotated connectors
fig = plt.figure(figsize=(7.2, 5.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 78)
ax.axis("off")
def panel(x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec="#c9c8c3", lw=1.0))
ax.text(x + 1.5, y + h - 2.0, title, fontsize=10.5, fontweight="bold", va="top")
def excerpt(x0, est, src):
"""Short excerpt; each number tagged with the Table 2 row it matches."""
ax.text(x0, 64.5, "high-use counties were", fontsize=10, color=SEC, va="top")
ax.text(x0, 60.0, f"{est} percentage points", fontsize=11, color=ORANGE, fontweight="bold", va="top")
ax.text(x0 + 1, 56.4, src[0], fontsize=9, color=SEC, va="top", style="italic")
ax.text(x0, 52.6, "(95% CI, 5.1-22.3)", fontsize=11, color=ORANGE, fontweight="bold", va="top")
ax.text(x0 + 1, 49.0, src[1], fontsize=9, color=SEC, va="top", style="italic")
ax.text(x0 + 25, 52.6, "(6.5% relative)", fontsize=11, color=ORANGE, fontweight="bold", va="top")
ax.text(x0 + 26, 49.0, src[2], fontsize=9, color=SEC, va="top", style="italic")
ax.text(x0, 45.4, "more likely to have outpatient follow-up", fontsize=10, color=SEC, va="top")
ax.text(x0, 41.8, "within 7 days", fontsize=10, color=TXT, fontweight="bold", va="top")
panel(1.5, 38.0, 46, 31.5, "Abstract, Results")
excerpt(3, "13.7", ("matches Table 2, 30 d", "matches Table 2, 30 d", "matches Table 2, 7 d"))
panel(52.5, 38.0, 46, 31.5, "Results text (cites Table 2)")
excerpt(54, "2.2", ("matches Table 2, 7 d", "matches Table 2, 30 d", "matches Table 2, 7 d"))
# Table 2 panel (bottom)
panel(1.5, 1.5, 97, 21, "Table 2, high use vs no use (main model)")
cols = [3, 22, 37, 63, 80]
heads = ["Follow-up", "Reference\nrate, %", "Difference, pp\n(95% CI)", "Relative\nchange, %",
"Recomputed\nchange, %"]
for x, h in zip(cols, heads):
ax.text(x, 17.6, h, fontsize=9.5, color=BLUE if x == 80 else SEC, va="top", linespacing=1.1)
rows = [("7 d", ref7, f"{d7} (−1.0 to {hi7})", rel7), ("30 d", ref30, f"{d30} ({lo30} to {hi30})", rel30)]
for i, (lab, ref, dci, rel) in enumerate(rows):
y = 9.6 - 5.0 * i
d = d7 if i == 0 else d30
ax.text(cols[0], y, lab, fontsize=11, fontweight="bold", va="center")
ax.text(cols[1], y, f"{ref:.1f}", fontsize=11, va="center")
ax.text(cols[2], y, dci, fontsize=11, va="center")
ax.text(cols[3], y, f"{rel}", fontsize=11, va="center")
ax.text(cols[4], y, f"{d}/{ref:.1f} = {100 * d / ref:.1f}", fontsize=10, color=BLUE, va="center")
def connector(x, label):
ax.add_patch(FancyArrowPatch((x, 37.0), (x, 23.8), arrowstyle="<|-|>", mutation_scale=11,
lw=1.6, color=SEC))
ax.text(x + 1.6, 30.4, label, fontsize=9.5, color=TXT, va="center", linespacing=1.3)
connector(6, "Conflict: 30-day estimate and CI\nlabelled \"7 days\", paired with the\n"
f"7-day 6.5% (13.7/34.0 = {100 * 13.7 / 34.0:.1f}%)")
connector(57, "Conflict: 7-day estimate with the\n30-day CI; 2.2 lies below\nits own interval (5.1 to 22.3)")
ax.add_line(Line2D([55, 58], [75.6, 75.6], color=ORANGE, lw=4))
ax.text(58.8, 75.6, "stated in text", fontsize=9.5, color=SEC, va="center")
ax.add_line(Line2D([77, 80], [75.6, 75.6], color=BLUE, lw=4))
ax.text(80.8, 75.6, "recomputed", fontsize=9.5, color=SEC, va="center")
save(fig, "fig1")
# ---- Figure 2: dot-and-interval plot of the 7-day estimates on one axis
items = [
# label, estimate, lo, hi, color, kind
("Abstract: \"within 7 days\"", 13.7, 5.1, 22.3, ORANGE, "stated"),
("Results text: \"within 7 days\"", 2.2, 5.1, 22.3, ORANGE, "stated"),
("6.5% × 34.0 reference\n(7-day value implied by the text)", 0.065 * ref7, None, None, BLUE, "recomp"),
("Table 2: 7 d", d7, lo7, hi7, TXT, "table"),
("Table 2: 30 d (for reference)", d30, lo30, hi30, SEC, "table"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.6), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(items))[::-1]
for y, (lab, d, lo, hi, col, kind) in zip(ys, items):
if lo is not None:
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([lo, lo], [y - .12, y + .12], color=col, lw=2, zorder=2)
ax.plot([hi, hi], [y - .12, y + .12], color=col, lw=2, zorder=2)
mk = "D" if kind == "recomp" else "o"
fc = "white" if kind == "table" and col == SEC else col
ax.plot(d, y, marker=mk, ms=8.5, mfc=fc, mec=col, mew=2, zorder=3, ls="none")
txt = f"{d:.1f}" + (f" ({lo} to {hi})" if lo is not None else "")
ax.text(23.4, y, txt, va="center", fontsize=10, color=col if col != TXT else TXT)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.set_yticks(ys)
ax.set_yticklabels([it[0] for it in items], fontsize=10)
ax.set_xlim(-3, 37)
ax.spines["bottom"].set_bounds(-3, 23)
ax.set_xticks([0, 5, 10, 15, 20])
ax.set_xlabel("Difference in follow-up rate, high vs no use (percentage points, 95% CI)")
ax.xaxis.set_label_coords((10 + 3) / 40, -0.13)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#b5b4af")
ax.tick_params(axis="y", length=0)
se7, z7, p7 = wald(d7, lo7, hi7)
ax.text(-2.7, ys[3] - 0.42, f"includes 0: z ≈ {z7:.2f}, p ≈ {p7:.2f}", fontsize=9.5, color=TXT, va="center")
ax.text(5.5, ys[1] + 0.33, "point estimate outside its own interval", fontsize=9.5, color=ORANGE, va="center")
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="stated in text"),
Line2D([], [], color=BLUE, marker="D", ms=8, lw=0, label="recomputed"),
Line2D([], [], color=TXT, marker="o", ms=8, lw=2, label="Table 2 as printed")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, 1.16), ncol=3, frameon=False,
fontsize=9.5, handletextpad=0.4, columnspacing=1.0)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Reported HbA1c intervention effect of 0.02% lies outside its own 95% confidence interval of 0.03%–0.14%
JAMA Netw Open (2023) · doi:10.1001/jamanetworkopen.2022.50375 · PMC9856961
Summary
This secondary analysis of a cluster randomized trial (Texas Sprouts, 16 elementary schools) reports that, compared with control schools, the intervention produced a "0.02% reduction in mean hemoglobin A1c (95% CI, 0.03%-0.14%; P = .005)". The 0.02 figure appears in the Key Points, and the full estimate with its CI and P value appears in the Abstract, the Results text and Table 3. In each of those three places the estimate 0.02 lies below the lower limit of its own 95% CI. The four reported quantities (estimate 0.02, SE 0.01, CI 0.03 to 0.14, P = .005) cannot all be correct. The CI and the P value agree with each other and point to an effect of about 0.085 percentage points. The estimate and SE agree with neither, and the unadjusted difference in group changes in Table 3 (−0.03 vs +0.06, so 0.09) agrees with the CI. HbA1c is the only outcome behind the paper's conclusion that the intervention "improved glucose control". The Limitations say the improvement "may not be clinically relevant", and then add that "even small reductions in HbA1c levels reflect clinical improvements in glucose control". The improvement the paper reports, 0.02, is about one quarter of the size the paper's other numbers imply.
What the paper reports
Key Points, "Findings":
In this secondary analysis of a cluster randomized clinical trial of 695 children, a gardening, nutrition, and cooking intervention resulted in reductions of 0.02% for hemoglobin A1c and 6.40 mg/dL for low-density lipoprotein cholesterol levels.
Abstract, "Results":
Compared with control schools, children from Texas Sprouts schools had a 0.02% reduction in mean hemoglobin A1c (95% CI, 0.03%-0.14%; P = .005)
Results, paragraph on intervention effects:
Compared with children in the control schools, children in the Texas Sprouts intervention had a 0.02% reduction in mean HbA1c (95% CI, 0.03%-0.14%; P = .005)
Table 3, HbA1c row. Columns: intervention children, preintervention mean (SE), change mean (SE) | control children, preintervention mean (SE), change mean (SE) | differences in changes, mean (SE) [95% CI] | P value, intervention effect. (The minus sign is a Unicode minus in the publisher's XML.)
Hemoglobin A1c, %c | 236 | 5.27 (0.01) | -0.03 (0.02) | 221 | 5.22 (0.01) | 0.06 (0.02) | 0.02 (0.01) [0.03 to 0.14] | .005
Table 3, footnotes b and c:
Generalized linear mixed models with the identity link for continuous variables were used to test differences in metabolic parameters between the intervention and the control estimates, with schools as clusters fixed for their intervention effect and children nested within schools.
Children at 5 intervention schools and 5 control schools were analyzed for hemoglobin A1c.
Methods, "Statistical Analysis":
Data were analyzed using SPSS statistical software version 28 (IBM) and a 2-sided type I error level of α = .05 was used as the threshold for statistical significance.
The conclusions that rest on this result. Abstract, "Conclusions and Relevance":
In this cluster RCT, Texas Sprouts improved glucose control and reduced low-density lipoprotein cholesterol in high-risk youth.
Discussion, first paragraph:
To our knowledge, Texas Sprouts was the first cluster RCT to show that a school-based gardening, nutrition, and cooking intervention can improve glucose control and lower LDL cholesterol in elementary schoolchildren.
Discussion, paragraph on body composition:
The current intervention resulted in reductions in LDL cholesterol and improvements in HbA1c, independent of changes in body composition.
Discussion, "Limitations":
Another limitation is that the improvement in HbA1c was rather small, albeit significant, and may not be clinically relevant. However, health care professionals believe that even small reductions in HbA1c levels reflect clinical improvements in glucose control.
Conclusions:
In conclusion, the present RCT showed that a school-based gardening, nutrition, and cooking intervention resulted in a small, albeit significant, improvement in glycemic control and a reduction in LDL cholesterol in predominately low-income and racially diverse elementary schoolchildren.
What the paper's own numbers show
Method. The paper states an identity-link linear mixed model, so each 95% CI should be symmetric about its estimate, with half-width equal to a fixed multiplier times the SE. To find that multiplier I divided the CI half-width by the SE in the other rows of Table 3. In the seven rows without other problems it is 2.138 to 2.206 (median 2.148, not 1.96), which matches a t reference with about 14 degrees of freedom. I therefore used t(14) for the HbA1c checks, and give normal-reference values as well. For each pair of reported quantities (estimate, SE, CI, P) I worked out what one implies for the other. As an independent check I took the unadjusted difference between the two group changes in Table 3, with an SE from the two group SEs. That SE ignores clustering, so it is only a rough check. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Estimate inside its own 95% CI? | Abstract, Results, Table 3 | 0.02 with CI 0.03 to 0.14 | No: 0.02 is 0.01 below the lower limit |
| Centre of the reported CI | Table 3 | (estimate) 0.02 | (0.03 + 0.14) / 2 = 0.085 |
| SE implied by the CI width | Table 3 | 0.01 | 0.055 / 2.145 = 0.026 (t14); 0.028 (normal) |
| CI implied by estimate 0.02 and SE 0.01 | Table 3 | 0.03 to 0.14 | −0.001 to 0.041 (t14); 0.000 to 0.040 (normal) |
| P implied by estimate/SE (0.02/0.01 = 2.0) | Abstract, Results, Table 3 | .005 | .065 (t14); .046 (normal) |
| P implied by the reported CI | Abstract, Results, Table 3 | .005 | .0051 (t14); .0025 (normal); .004 to .007 for df 8 to 20 |
| Estimate needed for P = .005 if SE were 0.01 | Table 3 | 0.02 | 0.033 (t14); 0.028 (normal) |
| Unadjusted difference in changes, intervention − control | Table 3 change columns: −0.03 (0.02) vs 0.06 (0.02) | 0.02 | −0.09, i.e. a reduction of 0.09; SE 0.028, CI −0.145 to −0.035 (ignores clustering) |
| Sign of the difference column | Table 3, rows with clear effects (e.g. LDL changes −8.43 vs −3.53, difference −6.40) | HbA1c CI 0.03 to 0.14 (positive) | In those rows the column reads intervention − control, so a reduction should be negative: −0.14 to −0.03, which matches −0.145 to −0.035 |
| Reported effect vs CI centre | Key Points, Abstract | 0.02 | The CI centre is 4.25 times larger; the unadjusted difference is 4.5 times larger |
Three things follow. First, the estimate cannot belong to the reported interval, so at least one of them is wrong. Second, the CI and the P value fit together: under the multiplier the rest of Table 3 implies, the CI gives P = .0051, against the reported .005. The estimate 0.02 and SE 0.01 fit neither of them, and they are also much smaller than the unadjusted difference. Third, the CI's sign does not follow the convention the table uses in its rows with clear effects. In those rows (fasting glucose, total, LDL and non-HDL cholesterol) the column reads intervention − control, so an HbA1c reduction would be negative. With the sign restored, the HbA1c CI matches the unadjusted difference closely (−0.14 to −0.03 against −0.145 to −0.035). The most coherent reading of the paper's own numbers is an HbA1c reduction of about 0.085 to 0.09 percentage points, with the estimate and SE cells in error. The exact model estimate cannot be rebuilt from these rounded summaries, because elsewhere in Table 3 the model estimates differ from the unadjusted differences by up to about 2 SEs (for example, non-HDL cholesterol −6.13 against −3.83, about 2.2 SE). In fasting insulin and HDL the model estimate and the unadjusted difference differ in sign (+0.61 against −0.42; +0.43 against −0.31), which fits adjustment gaps of that size. So the sign convention is inferred from the rows with clear effects, not shown for every row.
Related observations in Table 3. These do not change the HbA1c finding, but they bear on how the table can be read:
- The P column does not follow from the displayed estimates and CIs in 6 of the other 8 rows. Fasting glucose has CI 1.60 to 6.84, which excludes zero (P ≈ .004 from the CI, t14), but P = .13 is printed. LDL has CI −8.97 to −3.82 (P ≈ .0001), but P = .048 is printed. The total cholesterol, HOMA-IR, HDL and non-HDL rows differ in the same way. So the "P value, intervention effect" column may come from a different test than the displayed difference; the paper does not say which. Among the rows whose CI excludes zero (fasting glucose, total, LDL and non-HDL cholesterol, HbA1c), HbA1c is the only one whose printed P matches its CI; the P values for fasting insulin (.474 from the CI, .46 printed) and triglycerides (.395 from the CI, .39 printed) also match their CIs.
- The triglycerides interval (−15.42 to 6.47) is not centred on its estimate 1.52 (centre −4.475), and its half-width is 4.3 times the SE, against about 2.15 in the other rows. A symmetric interval with upper limit 6.47 would have lower limit −3.43.
Figure
Figure 1. HbA1c effect in Table 3, on one axis in percentage points, with positive values meaning lower HbA1c in intervention schools than in control schools, as the paper's text reads "0.02% reduction (95% CI, 0.03%-0.14%)". The top row (orange) is what the paper reports: the dot (0.02) sits outside the bar (0.03 to 0.14). The blue rows are recomputed from the paper's own numbers. The second row is the interval that the reported estimate and SE would give (−0.001 to 0.041, P = .065), which does not match the reported CI or P. The third row is the centre and SE implied by the reported CI (0.085, SE 0.026, P = .0051), which matches the reported P. The fourth row is the unadjusted difference in the two group changes in Table 3 (−0.03 vs +0.06), shown as a reduction of 0.09 with CI 0.035 to 0.145. Its SE ignores clustering. The vertical line marks no effect. Intervals use the t(14) multiplier that the other rows of Table 3 imply.
Figure 2. Where each reported estimate sits within its own reported 95% CI, for all nine outcomes in Table 3, scaled so that the lower limit is 0, the centre 0.5 and the upper limit 1. Seven rows sit at the centre, as an identity-link model requires. Triglycerides sits off-centre (its lower limit, −15.42, is not symmetric about 1.52). HbA1c is the only estimate outside its interval.
Why it matters
- It is a headline result. The HbA1c effect is one of two results in the Key Points and the Abstract. The 0.02 figure appears in all four places the result is stated (Key Points, Abstract, Results, Table 3), and the estimate-outside-CI combination appears in three (Abstract, Results, Table 3).
- It carries the glucose conclusion. The Abstract says the intervention "improved glucose control", and the Discussion calls this a first for a cluster RCT. Fasting glucose, insulin and HOMA-IR were reported as showing no effect, so this claim rests on HbA1c alone.
- It changes the size judgement. The Limitations call the HbA1c improvement "rather small, albeit significant, and may not be clinically relevant", and then add that "health care professionals believe that even small reductions in HbA1c levels reflect clinical improvements in glucose control". The Conclusions call it "small". The improvement the paper reports, and the only HbA1c effect size it states, is 0.02 percentage points. The paper's own CI, P value and group changes point to roughly 0.085 to 0.09 percentage points, about 4 times larger. Whether that larger effect is clinically meaningful is a separate question, but a reader weighing these size statements against the reported 0.02 is weighing them against a figure the other numbers do not support.
- It does not, on the evidence shown, reverse the direction or the significance. The CI, the P value and the group changes all indicate lower HbA1c in intervention schools relative to control. So the qualitative conclusion is not overturned by these numbers; the reported magnitude and the internal consistency of the result are.
- Readers using the published numbers would be misled about the size. A meta-analyst taking 0.02 (SE 0.01) would enter an effect about one quarter the size, with an SE less than half of what the CI implies.
Explanations considered
- Model-adjusted estimate vs raw difference. Footnote b says the difference comes from a mixed model, so it need not equal the unadjusted −0.09. That can explain why 0.02 differs from 0.09. It cannot explain an estimate lying outside its own interval, and in the other rows the model estimate always sits at the centre of its CI.
- Asymmetric or transformed interval. The model uses an identity link, so the interval should be estimate ± multiplier × SE, as it is in seven other rows. Any valid interval, symmetric or not, must contain its point estimate.
- Different subsample, time point or units. The Abstract, Results and Table 3 all describe the same 457 children (236 and 221) at 5 intervention and 5 control schools (footnote c; Results, "Subsample B"), over the same 9-month intervention period (Abstract, "Interventions": "1 school year long (9 months)"; Abstract, "Main Outcomes and Measures" and "Results": "postintervention (9 months)" and "(9-month)" blood draws; Limitations: "only 9 months long"). The "n = 8 schools" in the Table 3 header is explained by footnote c. HbA1c is in % throughout; the SI footnote converts % to a fraction and does not affect the comparison.
- Sign convention. The text calls the effect a reduction and prints a positive CI, which would fit a convention in which positive means reduction. But in the Table 3 rows with clear effects the difference column reads intervention − control (fasting glucose +4.22 from changes 6.53 vs 2.34; LDL −6.40 from −8.43 vs −3.53; also total and non-HDL cholesterol). In fasting insulin and HDL the model estimate and the unadjusted difference differ in sign, which fits the adjustment gaps of up to about 2 SE noted above, so the convention rests on the rows with clear effects. Under that convention the HbA1c interval should be negative. Its sign appears dropped, and with the sign restored it matches the unadjusted difference. A sign convention cannot put 0.02 inside [0.03, 0.14].
- One-sided test. The Methods state a 2-sided α = .05.
- Rounding. All values are printed to two decimals. An estimate just under 0.025 (say 0.0249) with a lower limit just over it (0.0251) would round to 0.02 and 0.03 under any rounding rule, but the interval would then reach only 0.0002 below the estimate and about 0.115 above it, which an identity-link interval cannot do. It would also leave the SE (0.01) and P (.005) unexplained. Rounding cannot reconcile the four numbers.
- Conversion artifact. The publisher's version of record shows the same HbA1c values as PMC: 0.02 in the Key Points, and 0.02 with CI 0.03 to 0.14 and P = .005 in the Abstract, Results and Table 3. In the XML the minus signs of other rows (LDL, total cholesterol, non-HDL) are kept, so the positive HbA1c interval is not a lost character.
- A correct value in the supplements. Supplement 1 is the trial protocol, which predates the HbA1c measurements, and Supplement 2 is the data sharing statement. Neither contains model output, so neither resolves the discrepancy.
Status
- Version of record. Checked on the publisher's full text (jamanetwork.com/journals/jamanetworkopen/fullarticle/2800162, reached from doi.org), PMC (PMC9856961) and the Europe PMC full-text XML. All show the same HbA1c values in the Key Points, Abstract, Results and Table 3.
- Supplements. Supplement 1 (jamanetwopen-e2250375-s001.pdf, trial protocol, 15 pages) and Supplement 2 (jamanetwopen-e2250375-s002.pdf, data sharing statement, 1 page) were read. Neither contains HbA1c results. Participant data are stated to be available on request after an approved proposal and a data sharing agreement.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and a Crossref
updatesfilter returns no items. PMC and Europe PMC (PMID 36626172) list no correction or comment links. A web search found no erratum for this article. PubPeer refused automated access from this machine (HTTP 403 from both its API and its search page), so PubPeer could not be checked. - Sources. PMC article PMC9856961; publisher page https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2800162; Supplements 1 and 2; Crossref; Europe PMC; trial registration NCT02668744 (as cited by the paper, not used for any number here).
Suggested correction
- Report the HbA1c intervention effect from the fitted model with an estimate, SE, 95% CI and P value that agree with one another. Correct the Key Points, Abstract, Results text and Table 3 together. Use the sign convention of the Table 3 rows with clear effects (intervention − control, so a reduction is negative) in the table, and state the magnitude in the text.
- If, as the CI and P value suggest, the effect is about 0.08 to 0.09 percentage points, revisit the Limitations and Conclusions statements that the improvement was small and "may not be clinically relevant", which accompany the reported 0.02.
- State which test the "P value, intervention effect" column reports. For fasting glucose, HOMA-IR, total, LDL, HDL and non-HDL cholesterol the printed P values do not follow from the displayed differences and CIs (for example, fasting glucose has a CI of 1.60 to 6.84 but P = .13). If the P values are wrong, the statements that there were "no intervention effects" on glucose and other lipids need to be revisited too.
- Check the triglycerides interval (−15.42 to 6.47), which is not symmetric about its estimate 1.52, and remove the minus signs printed on SEs, as in "−6.40 (−1.20)".
Recomputation script (python recompute.py.txt)
"""Recompute the HbA1c intervention effect reported in PMC9856961
(JAMA Netw Open 2023, doi:10.1001/jamanetworkopen.2022.50375), Table 3.
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file:
fig1.svg/png, fig2.svg/png (the figures used in writeup.md)
candidates/cand_A..E.svg/png (five alternative designs for the main figure)
All inputs are copied from Table 3 of the article as archived in PMC (identical in the
publisher's version of record). Columns: intervention change, control change,
"Differences in changes, mean (SE) [95% CI]", "P value, intervention effect".
In the rows with clear effects (fasting glucose, total, LDL and non-HDL cholesterol) the
table's difference column reads intervention minus control (e.g. LDL: changes -8.43 vs
-3.53, difference -6.40), so a reduction in HbA1c would be negative there. In fasting
insulin and HDL the model estimate and the unadjusted difference differ in sign (section
4b), so the convention is inferred from the rows with clear effects.
"""
import math
import os
import numpy as np
from scipy import stats
from scipy.optimize import brentq
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (Table 3)
# name: (intervention change, control change, estimate, SE, CI low, CI high, reported P)
# SEs printed with a minus sign in the table (e.g. "-6.40 (-1.20)") are entered as positive.
ROWS = {
"Fasting glucose": (6.53, 2.34, 4.22, 1.22, 1.60, 6.84, 0.13),
"Fasting insulin": (0.84, 1.26, 0.61, 0.80, -1.16, 2.37, 0.46),
"HOMA-IR": (0.68, 0.57, 0.39, 0.18, -0.001, 0.77, 0.84),
"Total cholesterol": (-7.35, -3.23, -5.72, 1.24, -8.39, -3.04, 0.12),
"LDL cholesterol": (-8.43, -3.53, -6.40, 1.20, -8.97, -3.82, 0.048),
"Triglycerides": (5.92, 1.42, 1.52, 2.54, -15.42, 6.47, 0.39),
"HDL cholesterol": (0.28, 0.59, 0.43, 0.34, -0.30, 1.17, 0.74),
"Non-HDL chol.": (-7.62, -3.79, -6.13, 1.05, -8.38, -3.89, 0.08),
"Hemoglobin A1c": (-0.03, 0.06, 0.02, 0.01, 0.03, 0.14, 0.005),
}
GROUP_SE_A1C = (0.02, 0.02) # SEs of the intervention and control HbA1c changes
N_A1C = (236, 221) # children with HbA1c, intervention / control
Z = stats.norm.ppf(0.975)
I_CH, C_CH, EST, SE, LO, HI, P_REP = ROWS["Hemoglobin A1c"]
# ------------------------------------------------------------------ colours / style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "axes.edgecolor": INK2,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "axes.linewidth": 0.8,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", bbox_inches="tight")
fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def fp(p):
"""P value in the paper's style: no leading zero (.065, .0051)."""
d = 4 if p < 0.01 else 3
return f"{p:.{d}f}".lstrip("0")
def p2(z, df=None):
z = abs(z)
return 2 * (stats.t.sf(z, df) if df else stats.norm.sf(z))
def hdr(t):
print("\n" + "=" * 78 + "\n" + t + "\n" + "=" * 78)
# ================================================================== 1. HbA1c row
hdr("1. HbA1c row as reported (Table 3; same numbers in Abstract and Results)")
print(f"estimate {EST} SE {SE} 95% CI [{LO}, {HI}] P {P_REP}")
print(f"Is the estimate inside its own CI? {LO <= EST <= HI} "
f"(estimate is {LO - EST:.2f} below the lower limit)")
mid = (LO + HI) / 2
half = (HI - LO) / 2
print(f"CI midpoint = {mid:.3f}; CI half-width = {half:.3f}")
# ================================================================== 2. multiplier used elsewhere in Table 3
hdr("2. CI half-width / SE in the other rows of Table 3 (what multiplier the table uses)")
mults = {}
for k, (i, c, e, s, lo, hi, p) in ROWS.items():
m = (hi - lo) / 2 / s
sym = abs((hi - e) - (e - lo)) / ((hi - lo) / 2)
mults[k] = m
print(f"{k:18s} half-width/SE = {m:5.3f} midpoint {(lo + hi) / 2:8.3f} vs estimate {e:7.3f}")
reg = [m for k, m in mults.items() if k not in ("Triglycerides", "Hemoglobin A1c")]
m_med = float(np.median(reg))
print(f"\nThe 7 regular rows: multiplier {min(reg):.3f} to {max(reg):.3f}, median {m_med:.3f} "
f"(normal 1.960)")
df_med = brentq(lambda d: stats.t.ppf(0.975, d) - m_med, 2, 1000)
df_lo = brentq(lambda d: stats.t.ppf(0.975, d) - max(reg), 2, 1000)
df_hi = brentq(lambda d: stats.t.ppf(0.975, d) - min(reg), 2, 1000)
print(f"t quantile equal to the median multiplier: df = {df_med:.1f} "
f"(range of rows: df {df_lo:.1f} to {df_hi:.1f})")
DF = 14
TQ = stats.t.ppf(0.975, DF)
print(f"Used below: t reference with df = {DF}, t(0.975) = {TQ:.3f}")
# ================================================================== 3. what the reported numbers imply
hdr("3. HbA1c: reconstructing each reported quantity from the others")
ci_from_est_t = (EST - TQ * SE, EST + TQ * SE)
ci_from_est_z = (EST - Z * SE, EST + Z * SE)
print(f"CI from estimate 0.02 and SE 0.01, t({DF}): [{ci_from_est_t[0]:.4f}, {ci_from_est_t[1]:.4f}]")
print(f"CI from estimate 0.02 and SE 0.01, normal: [{ci_from_est_z[0]:.4f}, {ci_from_est_z[1]:.4f}]")
p_est_t = p2(EST / SE, DF)
p_est_z = p2(EST / SE)
print(f"P from 0.02/0.01 (ratio {EST / SE:.2f}): t({DF}) P = {p_est_t:.3f}; normal P = {p_est_z:.4f}"
f" (reported {P_REP})")
se_ci_t = half / TQ
se_ci_z = half / Z
print(f"SE implied by the CI width: t({DF}) {se_ci_t:.4f}; normal {se_ci_z:.4f} (reported {SE})")
p_ci_t = p2(mid / se_ci_t, DF)
p_ci_z = p2(mid / se_ci_z)
print(f"P implied by the CI (midpoint {mid:.3f}): t({DF}) ratio {mid / se_ci_t:.2f}, P = {p_ci_t:.4f}; "
f"normal ratio {mid / se_ci_z:.2f}, P = {p_ci_z:.4f} (reported {P_REP})")
for d in (8, 10, 12, 20):
q = stats.t.ppf(0.975, d)
print(f" sensitivity, df={d:2d}: P implied by the CI = {p2(mid / (half / q), d):.4f}")
z_rep = stats.norm.isf(P_REP / 2)
t_rep = stats.t.isf(P_REP / 2, DF)
# just under / just over 0.025, so the result does not depend on the rounding rule
print(f"Rounding check: estimate 0.0249 and lower limit 0.0251 (round to 0.02 / 0.03) would put "
f"the interval {0.0251 - 0.0249:.4f} below and {HI - 0.0249:.4f} above the estimate")
print(f"Estimate needed for P = .005 with SE 0.01: {t_rep * SE:.3f} (t{DF}) / {z_rep * SE:.3f} (normal)")
# ================================================================== 4. raw difference in group changes
hdr("4. Unadjusted difference in mean changes (Table 3 change columns)")
raw = I_CH - C_CH
se_raw = math.hypot(*GROUP_SE_A1C)
ci_raw = (raw - Z * se_raw, raw + Z * se_raw)
print(f"intervention change {I_CH:+.2f}, control change {C_CH:+.2f}: "
f"difference (I - C) = {raw:+.2f}")
print(f"SE from the two group SEs (ignores clustering) = {se_raw:.4f}; "
f"95% CI [{ci_raw[0]:.3f}, {ci_raw[1]:.3f}]; normal P = {p2(raw / se_raw):.4f}")
print(f"Reported CI with the table's sign convention restored: [{-HI:.2f}, {-LO:.2f}]")
print(f"Ratio of CI midpoint to the reported 0.02: {mid / EST:.2f}; of |raw difference|: "
f"{abs(raw) / EST:.2f}")
hdr("4b. Model estimate vs unadjusted difference in the other rows (how far they can differ)")
for k, (i, c, e, s, lo, hi, p) in ROWS.items():
print(f"{k:18s} unadjusted I-C {i - c:+7.2f} model estimate {e:+7.2f} "
f"gap {e - (i - c):+6.2f} ({(e - (i - c)) / s:+5.2f} SE)")
# ================================================================== 5. sign convention
hdr("5. Sign convention of the difference column")
print("Rows with clear effects (CI excludes zero), then the two rows where the model estimate and"
" the unadjusted difference differ in sign, then HbA1c:")
for k in ("Fasting glucose", "Total cholesterol", "LDL cholesterol", "Non-HDL chol.",
"Fasting insulin", "HDL cholesterol", "Hemoglobin A1c"):
i, c, e, s, lo, hi, p = ROWS[k]
print(f"{k:18s} I {i:+.2f} C {c:+.2f} I-C {i - c:+.2f} table estimate {e:+.2f} CI [{lo}, {hi}]")
# ================================================================== 6. P column across Table 3
hdr("6. Reported P vs P implied by each row's own estimate/SE and CI")
print(f"{'row':18s} {'est in CI':>9s} {'P est/SE t14':>12s} {'P CI t14':>10s} {'P est/SE z':>10s} "
f"{'P CI z':>9s} {'reported':>8s}")
P_ROWS = {}
for k, (i, c, e, s, lo, hi, p) in ROWS.items():
m = (lo + hi) / 2
sct, scz = (hi - lo) / 2 / TQ, (hi - lo) / 2 / Z
P_ROWS[k] = (p2(e / s, DF), p2(m / sct, DF), p)
print(f"{k:18s} {str(lo <= e <= hi):>9s} {p2(e / s, DF):12.4g} {p2(m / sct, DF):10.4g} "
f"{p2(e / s):10.4g} {p2(m / scz):9.4g} {p:8.3g}")
i, c, e, s, lo, hi, p = ROWS["Triglycerides"]
print(f"\nTriglycerides: lower limit that would be symmetric about {e} given the upper limit "
f"{hi}: {2 * e - hi:.2f} (printed {lo}); estimate +/- {TQ:.3f} x SE: "
f"[{e - TQ * s:.2f}, {e + TQ * s:.2f}]")
# ================================================================== figures
POS = dict(lw=2)
def axis_style(ax, xlabel):
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel(xlabel, color=INK)
# Reduction scale used in the HbA1c figures: positive = lower HbA1c in intervention
# schools relative to control, which is how the paper's text reads "0.02% reduction
# (95% CI, 0.03%-0.14%)". The unadjusted I-C difference is sign-flipped onto it.
RED_LABEL = ("Reduction in HbA1c vs control, percentage points\n"
"(positive = lower in intervention schools, as the paper's text reads it)")
forest_rows = [
("Reported in the paper\n(Abstract, Results, Table 3)", EST, LO, HI, ORANGE, "o",
f"0.02, CI 0.03 to 0.14, P = .005\nestimate lies outside its own interval"),
("Recomputed: interval from the\nreported estimate 0.02 and SE 0.01", EST,
ci_from_est_t[0], ci_from_est_t[1], BLUE, "o",
f"CI {ci_from_est_t[0]:.3f} to {ci_from_est_t[1]:.3f}, P = {fp(p_est_t)}"),
("Recomputed: centre and SE implied\nby the reported CI", mid, LO, HI, BLUE, "D",
f"{mid:.3f} (SE {se_ci_t:.3f}), P = {fp(p_ci_t)}"),
("Recomputed: unadjusted difference\nin group changes, -0.03 vs +0.06", -raw,
-ci_raw[1], -ci_raw[0], BLUE, "s",
f"{-raw:.2f}, CI {-ci_raw[1]:.3f} to {-ci_raw[0]:.3f}\n(ignores clustering)"),
]
def draw_forest(path):
fig, ax = plt.subplots(figsize=(7.2, 3.9))
n = len(forest_rows)
for j, (lab, e, lo, hi, col, mk, note) in enumerate(forest_rows):
y = n - 1 - j
ax.plot([lo, hi], [y, y], color=col, solid_capstyle="butt", zorder=3, **POS)
for xx in (lo, hi):
ax.plot([xx, xx], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot(e, y, mk, color=col, ms=9, mec=BG, mew=1.5, zorder=4)
ax.text(0.205, y, note, va="center", ha="left", fontsize=9.5, color=INK2)
# highlight the gap on the reported row
y0 = n - 1
ax.text(EST, y0 + 0.30, "0.02 < lower limit 0.03", ha="left", va="bottom",
fontsize=9.5, color=INK)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.text(0.001, -0.75, "no effect", fontsize=9, color=INK2, ha="left")
ax.set_yticks(range(n))
ax.set_yticklabels([r[0] for r in forest_rows][::-1], fontsize=10)
for t, r in zip(ax.get_yticklabels(), forest_rows[::-1]):
t.set_color(INK)
ax.set_xlim(-0.02, 0.36)
ax.set_xticks([0, 0.05, 0.10, 0.15, 0.20])
ax.set_ylim(-0.9, n - 0.35)
axis_style(ax, RED_LABEL)
ax.xaxis.set_label_coords(0.29, -0.13)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", lw=2, ms=8, label="Reported by the paper"),
Line2D([], [], color=BLUE, marker="o", lw=2, ms=8,
label="Recomputed from the paper's own numbers")]
fig.legend(handles=h, loc="upper left", bbox_to_anchor=(0.0, 0.985), ncol=2,
frameon=False, fontsize=9.5)
fig.text(0.0, 1.04, f"HbA1c effect, Table 3 (n = {N_A1C[0]} vs {N_A1C[1]} children, "
f"10 schools); 95% intervals, t({DF})", fontsize=10.5, color=INK, ha="left")
save(fig, path)
draw_forest(os.path.join(HERE, "fig1"))
draw_forest(os.path.join(CAND, "cand_A_forest"))
# ---- Candidate B: one number line, reported row only, annotated
def draw_numberline(path):
fig, ax = plt.subplots(figsize=(7.2, 2.3))
ax.plot([LO, HI], [0, 0], color=ORANGE, lw=2, solid_capstyle="butt")
for xx in (LO, HI):
ax.plot([xx, xx], [-0.12, 0.12], color=ORANGE, lw=2)
ax.plot(EST, 0, "o", color=ORANGE, ms=10, mec=BG, mew=1.5)
ax.plot(mid, 0, "D", color=BLUE, ms=9, mec=BG, mew=1.5)
ax.text(EST, -0.32, "reported estimate\n0.02", ha="center", va="top", color=INK)
ax.text(LO, 0.25, "reported lower\nlimit 0.03", ha="center", va="bottom", color=INK)
ax.text(HI, 0.25, "reported upper\nlimit 0.14", ha="center", va="bottom", color=INK)
ax.text(mid, -0.32, f"recomputed centre\nof that CI, {mid:.3f}", ha="center", va="top",
color=INK)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0.002, -0.9, "no effect", color=INK2, fontsize=9)
ax.set_ylim(-1.0, 0.9)
ax.set_xlim(-0.01, 0.16)
ax.set_yticks([])
axis_style(ax, "Reduction in HbA1c vs control, percentage points")
ax.set_title("A 95% CI must contain its estimate: 0.02 lies outside [0.03, 0.14]\n"
"orange = reported by the paper, blue = recomputed", loc="left",
fontsize=10.5)
save(fig, path)
draw_numberline(os.path.join(CAND, "cand_B_numberline"))
# ---- Candidate C: calculation-flow diagram
def box(ax, x, y, w, h, text, edge, fs=9.8, bold=False):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.04",
fc=BG, ec=edge, lw=2))
ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", fontsize=fs, color=INK,
fontweight="bold" if bold else "normal")
def arrow(ax, x0, y0, x1, y1, label=None, off=(0, 0.06)):
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.4))
if label:
ax.text((x0 + x1) / 2 + off[0], (y0 + y1) / 2 + off[1], label, ha="center",
fontsize=9, color=INK2)
def draw_flow(path):
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ax.set_xlim(-0.1, 10.1)
ax.set_ylim(0.3, 5.3)
ax.axis("off")
W, H, fs = 3.0, 1.45, 9.6
box(ax, 0.0, 3.4, W, H, "Table 3, change\nintervention \u22120.03\ncontrol +0.06", ORANGE, fs=fs)
box(ax, 3.5, 3.4, W, H, "Unadjusted difference\n\u22120.03 \u2212 0.06 = \u22120.09\n"
"(reduction of 0.09)", BLUE, fs=fs)
box(ax, 7.0, 3.4, W, H, f"SE {se_raw:.3f}\n(from the 2 group SEs)\nCI {-ci_raw[1]:.3f} to "
f"{-ci_raw[0]:.3f}\n(as a reduction)", BLUE, fs=fs)
box(ax, 0.0, 0.5, W, H, "Table 3, difference\n0.02 (SE 0.01)\nCI 0.03 to 0.14", ORANGE, fs=fs)
box(ax, 3.5, 0.5, W, H, f"Centre of that CI\n(0.03 + 0.14) / 2\n= {mid:.3f}", BLUE, fs=fs)
box(ax, 7.0, 0.5, W, H, f"SE implied by CI {se_ci_t:.3f}\nP implied by CI {fp(p_ci_t)}\n"
f"(reported P .005)", BLUE, fs=fs)
ym_top, ym_bot = 3.4 + H / 2, 0.5 + H / 2
for yy in (ym_top, ym_bot):
arrow(ax, 3.05, yy, 3.45, yy)
arrow(ax, 6.55, yy, 6.95, yy)
for xx in (1.5, 5.0):
ax.annotate("", xy=(xx, 0.5 + H + 0.08), xytext=(xx, 3.4 - 0.08),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.2, ls="--"))
yc = (0.5 + H + 3.4) / 2
ax.text(5.1, yc, f"agree: about 0.09 / {mid:.3f}", fontsize=9.5, color=INK, va="center")
ax.text(1.6, yc, "estimate 0.02 is outside\nits own CI [0.03, 0.14]", fontsize=9.5,
color=INK, va="center")
ax.text(0.0, 5.12, "HbA1c, Table 3. Orange boxes: reported by the paper. Blue boxes: "
"recomputed from those numbers.", fontsize=9.5, color=INK2)
save(fig, path)
draw_flow(os.path.join(CAND, "cand_C_flow"))
# ---- Candidate D: consistency graph of the four reported quantities
def draw_consistency(path):
fig, ax = plt.subplots(figsize=(7.2, 4.2))
ax.set_xlim(0, 10.4)
ax.set_ylim(0.1, 5.85)
ax.axis("off")
W, H = 2.2, 1.1
nodes = {"est": (0.1, 4.6, "Estimate\n0.02"), "se": (0.1, 0.3, "SE\n0.01"),
"ci": (8.1, 4.6, "95% CI\n0.03 to 0.14"), "p": (8.1, 0.3, "P value\n.005")}
for k, (x, y, t) in nodes.items():
box(ax, x, y, W, H, t, ORANGE, fs=10.5, bold=True)
def c(k):
x, y, _ = nodes[k]
return x + W / 2, y + H / 2
def edge(a, b, txt, ok, t=0.5, above=False):
(x0, y0), (x1, y1) = c(a), c(b)
ax.plot([x0, x1], [y0, y1], color=BLUE if ok else INK2, lw=2,
ls="-" if ok else (0, (4, 3)), zorder=0)
# a label placed above a horizontal link leaves the dashes visible
dy, va = (0.22, "bottom") if above else (0, "center")
ax.text(x0 + t * (x1 - x0), y0 + t * (y1 - y0) + dy, txt, ha="center", va=va,
fontsize=9.3, color=INK, zorder=5,
bbox=dict(boxstyle="round,pad=0.3", fc=BG, ec=BLUE if ok else INK2, lw=1))
edge("est", "ci", f"INCONSISTENT: estimate lies outside the CI\n"
f"(CI centre is {mid:.3f})", False, above=True)
edge("est", "p", f"INCONSISTENT:\n0.02 / 0.01 gives\nP = {fp(p_est_t)}, not .005", False,
t=0.3)
edge("se", "ci", f"INCONSISTENT:\nCI width implies\nSE {se_ci_t:.3f}, not 0.01", False,
t=0.7)
edge("ci", "p", f"consistent:\nCI implies\nP = {fp(p_ci_t)}", True)
edge("est", "se", f"INCONSISTENT:\ntogether give\nCI {ci_from_est_t[0]:.3f}\n"
f"to {ci_from_est_t[1]:.3f},\nnot 0.03 to 0.14", False)
fig.text(0.0, 1.0, "HbA1c, Table 3. Orange boxes: values reported by the paper. Each link is "
"a check recomputed\nfrom them (t reference, df = 14, as the other rows of "
"Table 3 imply).\nSolid blue link = consistent; dashed gray link = inconsistent.",
fontsize=9.5, color=INK2, va="bottom")
save(fig, path)
draw_consistency(os.path.join(CAND, "cand_D_consistency"))
# ---- Candidate E (also fig2): position of each estimate within its own CI, all of Table 3
def draw_position(path):
fig, ax = plt.subplots(figsize=(7.2, 3.8))
names = list(ROWS)[::-1]
ax.axvspan(0, 1, color="#efeeea", zorder=0, lw=0)
for x in (0, 0.5, 1):
ax.axvline(x, color=INK2, lw=0.8 if x != 0.5 else 0.6, ls="-" if x != 0.5 else ":",
zorder=1)
for y, k in enumerate(names):
i, c, e, s, lo, hi, p = ROWS[k]
pos = (e - lo) / (hi - lo)
out = not (lo <= e <= hi)
col = ORANGE
ax.plot(pos, y, "o", color=col, ms=9, mec=BG, mew=1.5, zorder=3)
if out or k == "Triglycerides":
ax.text(pos + (-0.04 if out else 0.04), y, f"{e} in [{lo}, {hi}]",
ha="right" if out else "left", va="center", fontsize=9.3, color=INK)
ax.set_yticks(range(len(names)))
ax.set_yticklabels(names, color=INK)
ax.set_xlim(-0.75, 1.35)
ax.set_xticks([0, 0.5, 1])
ax.set_xticklabels(["lower\nlimit", "centre", "upper\nlimit"])
axis_style(ax, "Where the reported estimate sits within its own reported 95% CI")
ax.grid(False)
ax.set_title("Table 3, all rows (orange = positions computed from reported values)",
loc="left", fontsize=10.5)
save(fig, path)
draw_position(os.path.join(HERE, "fig2"))
draw_position(os.path.join(CAND, "cand_E_table_overview"))
hdr("Figures written")
for f in sorted(os.listdir(HERE)) + ["candidates/" + g for g in sorted(os.listdir(CAND))]:
if f.endswith((".png", ".svg")):
print(" ", f)
Diabetes-only hyperglycemia primary outcome has denominators larger than the trial's possible diabetic population
JAMA Network Open (2023) · doi:10.1001/jamanetworkopen.2023.32517 · PMC10517374
Summary
This pilot randomized trial (ACTFAST-3) defines one of its two primary outcomes, avoidance of postoperative hyperglycemia, for patients with diabetes only, and states that "patients without diabetes were excluded from analysis". Yet the reported denominators (8676 intervention, 8815 control) exceed the most cases with diabetes that the paper's own Table 1 allows (5713 and 5887, even counting every case of unknown status as diabetic), and they equal exactly the analyzed cases with any glucose value in the Figure 1 flowchart. The published result (85.8% vs 85.8%; RR 1.00, 95% CI 0.99–1.01) therefore describes a population in which at least a third of cases, and probably about two-thirds, did not have diabetes. Its narrow confidence interval, which the Discussion describes as "tight confidence bounds", depends on that larger denominator, and simple calculations suggest an interval for the stated population would be roughly 2 to 3.5 times as wide.
What the paper reports
Methods, "Study Population":
For glycemic outcomes, patients without diabetes were excluded from analysis.
Methods, "Outcomes":
The primary outcomes were avoidance of hypothermia, defined as the proportion of patients who had a final recorded intraoperative core temperature greater than 36 °C, and avoidance of hyperglycemia, defined as the proportion of patients with diabetes who had a blood glucose level of 180 mg/dL or lower on arrival to the postanesthesia recovery area.
Abstract, "Main Outcomes and Measures":
(defined as the proportion of patients with diabetes who had a blood glucose level ≤180 mg/dL on arrival to the postanesthesia recovery area)
Abstract, "Results". The Results text (primary outcomes paragraph) repeats the same counts, percentages and RR:
There was no treatment effect on avoidance of hyperglycemia (7445 of 8676 patients [85.8%] in the intervention group vs 7559 of 8815 [85.8%] in the control group; rate ratio [RR], 1.00; 95% CI, 0.99-1.01)
Table 2, primary outcome row (columns: intervention, n = 12 980 | control, n = 13 274 | coefficient (95% CI) | P value):
No postoperative hyperglycemia | 7445/8676 (85.8) | 7559/8815 (85.8) | 1.00 (0.99 to 1.01) | >.99
Table 1, comorbidity row. Table 1 lists the control column first (control, n = 13 274 | intervention, n = 12 980):
Diabetes | 2909/10 296 (28.3) | 2824/10 091 (28.0)
Figure 1, the study flowchart. The flowchart is an image, so this text was read from the publisher's image file (jamanetwopen-e2332517-g001.jpg) rather than from a text conversion:
12980 Analyzed (1581 without temperature data, 4304 without glucose data)
13274 Analyzed (1652 without temperature data, 4459 without glucose data)
Figure 1 caption:
The analysis included all patients randomized during the study with outcomes present.
The findings and conclusions that rest on this outcome. Key Points, "Findings":
an intraoperative telemedicine decision support intervention did not significantly reduce postoperative hypothermia or hyperglycemia
Abstract, "Conclusions and Relevance":
anesthesia care quality measures did not differ between groups, with high confidence in the findings
Discussion, second paragraph:
There are several potential reasons for the lack of an intention-to-treat effect with tight confidence bounds on these quality of care outcomes.
Discussion, third paragraph:
Second, the overall adherence to each measure at baseline was high, making statistically significant improvements difficult to achieve.
What the paper's own numbers show
Method. Table 1, Table 2 and Figure 1 count cases in the same unit and with the same arm totals (12 980 and 13 274). For each arm I took the cases with diabetes from Table 1. I counted cases with unknown diabetes status as the arm total minus the Table 1 denominator. The sum of the two is an upper bound on the number of cases that could have diabetes. I compared that bound with the Table 2 hyperglycemia denominator, and compared the denominator with Figure 1's analyzed cases minus cases "without glucose data". I repeated the check for the two sensitivity analyses in Supplement 2 (eTables 4–7). Those tables print diabetes only as n (%), so there the bound uses the smallest known-status total consistent with the rounded percentage. For contrast I looked at the secondary outcome that is explicitly limited to patients with diabetes. Finally I checked the Table 2 arithmetic with crude, unclustered rate ratios. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Cases with diabetes, intervention | Table 1 | 2824/10 091 (28.0) | 28.0%; 12 980 − 10 091 = 2889 with unknown status |
| Cases with diabetes, control | Table 1 | 2909/10 296 (28.3) | 28.3%; 13 274 − 10 296 = 2978 with unknown status |
| Most cases that could have diabetes, intervention | Table 1 | — | 2824 + 2889 = 5713 |
| Most cases that could have diabetes, control | Table 1 | — | 2909 + 2978 = 5887 |
| Hyperglycemia denominator, intervention | Table 2; Abstract; Results | 8676 | 2963 more than the 5713 maximum; 3.07 × the documented diabetic cases |
| Hyperglycemia denominator, control | Table 2; Abstract; Results | 8815 | 2928 more than the 5887 maximum; 3.03 × the documented diabetic cases |
| Analyzed cases with glucose data, intervention | Figure 1 | 12 980 analyzed, 4304 without glucose data | 12 980 − 4304 = 8676, identical to the denominator |
| Analyzed cases with glucose data, control | Figure 1 | 13 274 analyzed, 4459 without glucose data | 13 274 − 4459 = 8815, identical to the denominator |
| Denominator as a share of all analyzed cases | Table 2 vs Figure 1 | — | 66.8% and 66.4%, against a diabetes prevalence of 28.0% and 28.3% |
| Share of the denominator that cannot have diabetes | derived | — | at least 34.2% (intervention) and 33.2% (control); 67.5% and 67.0% if only documented diabetes is counted |
| Sensitivity analysis, excluding repeat surgeries within 30 days | eTables 4–5 | denominators 8069 / 8169; diabetes 2614 (27.6%) / 2667 (27.6%) of 12 179 / 12 457 | most possible diabetic cases ≤ 5339 / 5478; denominators exceed this by 2730 / 2691 |
| Sensitivity analysis, ignoring prior assignment | eTables 6–7 | denominators 8693 / 8798; diabetes 2844 (28.2%) / 2889 (28.0%) of 12 965 / 13 289 | most possible diabetic cases ≤ 5741 / 5878; denominators exceed this by 2952 / 2920 |
| Diabetes-restricted secondary outcome (intraoperative glucose measurement), for contrast | Table 2 | denominators 1962 / 1996 | 69.5% and 68.6% of the documented diabetic cases, so this outcome does fit inside the diabetic population |
| No postoperative hyperglycemia, % and RR | Table 2 | 85.8 vs 85.8; RR 1.00 (0.99 to 1.01) | 85.81% vs 85.75%; crude RR 1.001 (unclustered 95% CI 0.989–1.013), consistent with the printed values for the printed denominators |
The Table 2 arithmetic is internally consistent. The problem is the population. The denominators equal every analyzed case with a glucose value, not the cases with diabetes. The same pattern appears in all three published versions of the analysis (the main analysis and both sensitivity analyses; Figure 2 below), so it is not a single transcription slip.
The comparison also turned up some smaller, separate inconsistencies. None of them changes a conclusion:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Arm sizes | Abstract vs Methods, Table 1, Table 2, Figure 1 | Abstract: "control group; n = 12 980" and "intervention group; n = 13 274" | Every other location gives intervention 12 980 and control 13 274, so the abstract's labels are swapped |
| Hypothermia denominators | Table 2 vs Figure 1 | 11 447 / 11 672 | 12 980 − 1581 = 11 399 and 13 274 − 1652 = 11 622, which differ by 48 and 50 |
| Intraoperative glucose measurement, intervention % | Table 2 | 1346/1962 (68.7) | 68.60%, which rounds to 68.6 |
| Control hypothermia % | Abstract | "[66.7.%]" | typographical (66.7%) |
Figure
Figure 1. Counts for each arm on one axis of surgical cases, drawn from zero. The orange bar is the reported denominator of the hyperglycemia primary outcome (Table 2), which the paper defines for patients with diabetes only. The blue bars come from the paper's other numbers: the cases with documented diabetes (Table 1); the most cases that could have diabetes if every case with unknown status had it (Table 1); and the analyzed cases with any glucose value (Figure 1: analyzed minus "without glucose data"). The gray bar is all analyzed cases. In both arms the reported denominator exceeds the largest possible diabetic population, by 2963 and 2928 cases, and equals the number of cases with glucose data exactly.
Figure 2. The same comparison for the main analysis (Tables 1–2) and the two sensitivity analyses in Supplement 2 (eTables 4–5, excluding surgeries within 30 days of an index operation; eTables 6–7, ignoring prior treatment assignment). Hollow blue circles are cases with documented diabetes as printed. Blue diamonds are the most cases that could have diabetes; for the sensitivity analyses this is an upper bound derived from the rounded percentage. Orange circles are the reported hyperglycemia denominators. The number on each row is the reported denominator minus that maximum. In all six arm-analyses the denominator lies about 2700–3000 cases above the largest possible diabetic population.
Why it matters
Avoidance of hyperglycemia is one of the trial's two primary outcomes. Its null result is reported in the Key Points, the Abstract, the Results, Table 2 and eTables 5 and 7. The paper's stated population for it is patients with diabetes, and the published protocol that the paper cites for its sample size (reference 22, F1000Research 2018;7:623) planned it that way: its power calculation states that "we assumed that the prevalence of diabetes mellitus among Barnes-Jewish Hospital surgical patients is about 20%", and its sample-size table plans glucose clusters about a quarter the size of the temperature clusters (59 vs 239 cases, a ratio of 0.247). The reported denominators are instead about two-thirds of all analyzed cases (0.668 and 0.664).
Three consequences follow from the published numbers alone.
- The estimate does not describe the stated population. At least 34% of each denominator cannot have diabetes, and about 67% did not if only documented diabetes is counted. Glucose above 180 mg/dL on arrival in recovery is presumably much less common in patients without diabetes. Including them raises the avoidance rate (85.8%) and, if their rates are similar between arms, pulls any between-arm ratio toward 1. The diabetes-only rates and RR cannot be recovered from the published tables.
- The "tight confidence bounds" depend on the larger denominator. With the reported denominators, the crude unclustered 95% CI for the RR is 0.989–1.013, which reproduces the published 0.99–1.01. Two illustrations show the scale of the change; neither is an estimate of the true result. (a) With denominators the size of the documented diabetic population (2824 and 2909) and no difference between arms, the same calculation gives 0.979–1.021 at an 85.8% avoidance rate and 0.959–1.043 at 60%, about 1.7 to 3.5 times as wide. (b) If the cases without diabetes had the same avoidance rate q in both arms and every documented diabetic case were in the denominator, the implied diabetes-only avoidance would be 77.1% vs 77.1% at q = 0.90 and 58.5% vs 58.9% at q = 0.99. The RR would stay between 0.994 and 1.000, with unclustered 95% CIs from 0.972–1.029 to 0.951–1.038, 2.3 to 3.6 times as wide. All of these come before the clustering and Bonferroni-corrected α that the paper applies to its CIs, which widen them further.
- The explanation offered for the null result is affected. The Discussion attributes the null results partly to high baseline adherence. For this outcome, the 85.8% baseline describes the mixed population and has not been shown for patients with diabetes; in the scenario above it would be closer to 59–77%.
The trial's other primary outcome, hypothermia, does not depend on this issue. Most secondary outcomes are null and, apart from those limited to diabetes, are not affected either. The scenario in point 2 suggests the null result for hyperglycemia may well survive a correct analysis, but with a substantially wider interval. As published, the hyperglycemia result cannot be read as a finding about patients with diabetes, and the "high confidence" statement in the Abstract rests partly on it.
Explanations considered
- Diabetes is under-ascertained in Table 1 because of missing status. This does not account for the numbers. The bound already counts all 2889 and 2978 cases with unknown status as diabetic, and the denominators still exceed it by about 2950 per arm.
- The tables use different units (cases vs patients). This does not account for the numbers. Table 1, Table 2 and Figure 1 all use the same totals (12 980 and 13 274), and the sensitivity analysis that keeps only one operation per 30-day window shows the same excess.
- The outcome used a broader definition of diabetes than Table 1. This is not plausible. The denominators would require two-thirds of all surgical cases to have diabetes, against 28% in Table 1 and the roughly 20% the protocol assumed.
- Figure 1's "without glucose data" counts already exclude cases without diabetes. This does not account for the numbers. If they did, the 8676 and 8815 cases with glucose data would all have diabetes, which contradicts Table 1's maximum of 5713 and 5887.
- A transcription or typesetting error in Table 2. This is unlikely. The same plain integers appear in the Abstract, the Results text and Table 2, they match the flowchart arithmetic exactly, and the two separately computed sensitivity analyses show the same excess. The publisher's page carries the same values as the PMC archive, so a conversion artifact is excluded.
- The analysis followed the registered definition, and the text is what is wrong. This is possible, and it would account for the numbers. ClinicalTrials.gov (NCT02830126, last update posted 2020-01-22) registers the outcome as "Proportion of cases with blood glucose <=180 mg/dL upon arrival to the anesthesia recovery area", with no diabetes restriction. The protocol's outcome table (F1000Research 2018;7:623, Table 1) is worded similarly ("Proportion of cases with blood glucose ≥180 mg/dl upon arrival to the post-anesthesia recovery area"). Its power calculation, however, is framed around patients with diabetes, as quoted above. If every case with a glucose value was meant to be analyzed, then the Abstract and Methods misdescribe the outcome population, the sentence "patients without diabetes were excluded from analysis" is incorrect for this outcome, and the departure from the protocol's diabetic framing is not reported. Either way, the paper's stated population and its reported numbers cannot both be right, and the published material does not show which one is in error.
- Glucose is routinely measured in patients without diabetes. This is consistent with the numbers and would explain how two-thirds of all cases came to have a glucose value. It does not reconcile that set with the stated restriction.
Status
- Version of record. The publisher's full text (jamanetwork.com, article 2809789, reached from the DOI) shows the same Study Population sentence, the same Table 1 diabetes row (2909/10 296 and 2824/10 091) and the same Table 2 hyperglycemia row (7445/8676, 7559/8815; 1.00, 0.99 to 1.01) as the PMC archive. I also checked the Europe PMC full-text XML and the publisher's Figure 1 image. Every quote above was confirmed verbatim against the PMC-archived text, except the Figure 1 flowchart text, which was read from the image.
- Supplements. I opened and text-extracted all four supplements from the Europe PMC supplementary-files bundle. Supplement 1 is the trial protocol; it does not define the glycemic primary outcome numerically. Supplement 2 holds eTable 1 (outcome definitions, which cover the secondary and clinical outcomes but not the two primary outcomes) and the sensitivity analyses in eTables 4–7, used above. Supplements 3 (nonauthor collaborators) and 4 (data sharing statement) are not relevant.
- Protocol and registry. I checked the published protocol (reference 22 cites version 1, doi:10.12688/f1000research.14897.1; I read version 2 as archived in PMC, PMC6039946) and the ClinicalTrials.gov record NCT02830126 through its API.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment links (PMID 37738052). The publisher's page shows no correction notice and no reader comments. The PubPeer API returned no feedback for the DOI; the PubPeer search page itself refused automated access (HTTP 403). A general web search could not be run in this session.
- Sources. PMC article PMC10517374; publisher page https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2809789; Supplement 2 (jamanetwopen-e2332517-s002.pdf); Figure 1 image (jamanetwopen-e2332517-g001.jpg); F1000Research protocol (PMC6039946); ClinicalTrials.gov NCT02830126; Crossref; Europe PMC; PubPeer.
Suggested correction
- Re-run the hyperglycemia primary outcome restricted to cases with diabetes, as the Methods state, and report the numerators, denominators, RR, Bonferroni-adjusted CI and Holm-adjusted P value. Update the Abstract, Key Points, Results text, Table 2, eTables 5 and 7, and Figure 2 if it uses the same population. Also report how many cases with diabetes had no glucose value.
- Alternatively, if the intended population was all cases with a postoperative glucose value, as in the registry wording, revise the outcome definition in the Abstract and Methods. Remove or qualify the sentence "For glycemic outcomes, patients without diabetes were excluded from analysis", report the change from the protocol's diabetic framing, and give the diabetes-only result as a secondary analysis.
- In either case, revisit the "high confidence" and "tight confidence bounds" statements as they apply to hyperglycemia.
- Minor: correct the swapped arm sizes in the Abstract; explain why the hypothermia denominators (11 447 / 11 672) differ from Figure 1 (11 399 / 11 622); change 68.7 to 68.6 for 1346/1962 if it is a crude proportion; and fix "[66.7.%]".
Recomputation script (python recompute.py.txt)
"""Recompute the hyperglycemia primary-outcome denominators of PMC10517374
(JAMA Netw Open 2023, doi:10.1001/jamanetworkopen.2023.32517).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC and from Supplement 2:
Figure 1 (flowchart image): analyzed cases and cases without temperature / glucose data
Table 1: cases with diabetes (No./total No. with known status)
Table 2: primary and secondary outcome counts
eTables 4-7 (Supplement 2): the two sensitivity analyses
Unit: the paper counts surgical cases (Table 1 and Table 2 use the same n per arm).
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
ARMS = ("intervention", "control")
N = {"intervention": 12980, "control": 13274} # Figure 1, Table 1, Table 2
NO_GLUCOSE = {"intervention": 4304, "control": 4459} # Figure 1 "without glucose data"
NO_TEMP = {"intervention": 1581, "control": 1652} # Figure 1 "without temperature data"
DIABETES = {"intervention": (2824, 10091), "control": (2909, 10296)} # Table 1 "Diabetes"
HYPERGLY = {"intervention": (7445, 8676), "control": (7559, 8815)} # Table 2 primary
HYPOTHERM = {"intervention": (7602, 11447), "control": (7783, 11672)} # Table 2 primary
GLU_MEAS = {"intervention": (1346, 1962), "control": (1274, 1996)} # Table 2 secondary
GLU_MEAS_PRINTED_PCT = {"intervention": 68.7, "control": 63.8}
# Sensitivity analyses (Supplement 2). Diabetes is printed as n (%) without a denominator.
SENS = {
"Excluding repeat surgeries (eTables 4-5)": {
"N": {"intervention": 12179, "control": 12457},
"dm": {"intervention": (2614, 27.6), "control": (2667, 27.6)},
"hyper": {"intervention": (6923, 8069), "control": (6989, 8169)},
},
"Ignoring prior assignment (eTables 6-7)": {
"N": {"intervention": 12965, "control": 13289},
"dm": {"intervention": (2844, 28.2), "control": (2889, 28.0)},
"hyper": {"intervention": (7469, 8693), "control": (7535, 8798)},
},
}
# Published protocol (ref 22, F1000Res 2018;7:623), sample-size table: cluster sizes
PROTO_CLUSTER = {"temperature": 239, "glucose": 59}
PROTO_DM_PREV = 0.20
def pct(a, b):
return 100.0 * a / b
def rr_ci(a1, n1, a0, n0, z=1.959964):
rr = (a1 / n1) / (a0 / n0)
se = math.sqrt(1 / a1 - 1 / n1 + 1 / a0 - 1 / n0)
return rr, math.exp(math.log(rr) - z * se), math.exp(math.log(rr) + z * se)
R = {} # results reused by the figures
print("=== 1. Hyperglycemia denominator vs the population it is defined for ===")
for g in ARMS:
d, known = DIABETES[g]
unknown = N[g] - known
max_dm = d + unknown
den = HYPERGLY[g][1]
with_glu = N[g] - NO_GLUCOSE[g]
R[g] = dict(d=d, unknown=unknown, max_dm=max_dm, den=den, with_glu=with_glu)
print(f"\n[{g}] n = {N[g]}")
print(f" Table 1 diabetes: {d}/{known} = {pct(d, known):.1f}%; status unknown: {N[g]} - {known} = {unknown}")
print(f" most cases that could have diabetes (all unknown counted as diabetes): {d} + {unknown} = {max_dm}")
print(f" Table 2 hyperglycemia denominator: {den}")
print(f" denominator - maximum possible diabetic cases = {den - max_dm}")
print(f" denominator / documented diabetic cases = {den / d:.2f}")
print(f" Figure 1: {N[g]} - {NO_GLUCOSE[g]} without glucose data = {with_glu} -> equals denominator: {with_glu == den}")
print(f" denominator as share of all analyzed cases: {pct(den, N[g]):.1f}%")
print(f" share of denominator that cannot have diabetes (lower bound): ({den} - {max_dm}) / {den} = {pct(den - max_dm, den):.1f}%")
print(f" share of denominator without documented diabetes if all {d} diabetic cases had glucose data: "
f"({den} - {d}) / {den} = {pct(den - d, den):.1f}%")
print("\n=== 2. Same check in the two sensitivity analyses (Supplement 2) ===")
SENS_R = {}
for name, v in SENS.items():
for g in ARMS:
d, p = v["dm"][g]
n = v["N"][g]
den = v["hyper"][g][1]
# Diabetes % is rounded to 0.1; the smallest known-status total consistent with it
known_min = math.ceil(d / ((p + 0.05) / 100))
max_dm = d + (n - known_min)
SENS_R[(name, g)] = dict(d=d, max_dm=max_dm, den=den)
print(f"{name} [{g}]: n={n}; diabetes {d} ({p}%) -> known-status total >= {known_min}; "
f"max possible diabetic cases <= {max_dm}; hyperglycemia denominator {den}; "
f"denominator - documented diabetes = {den - d}; denominator - max possible = {den - max_dm}")
print("\n=== 3. A diabetes-restricted glucose outcome for comparison (Table 2, secondary) ===")
for g in ARMS:
a, n = GLU_MEAS[g]
d = DIABETES[g][0]
print(f"{g}: intraoperative glucose measurement denominator {n} = {pct(n, d):.1f}% of {d} diabetic cases "
f"(fits inside the diabetic population: {n <= d})")
print("\n=== 4. Published protocol sample size (ref 22) ===")
print(f"glucose cluster size / temperature cluster size = {PROTO_CLUSTER['glucose']}/{PROTO_CLUSTER['temperature']} "
f"= {PROTO_CLUSTER['glucose'] / PROTO_CLUSTER['temperature']:.3f} (protocol assumed diabetes prevalence {PROTO_DM_PREV:.0%})")
print(f"reported denominators / analyzed cases: {HYPERGLY['intervention'][1] / N['intervention']:.3f}, "
f"{HYPERGLY['control'][1] / N['control']:.3f}")
print("\n=== 5. Table 2 arithmetic (crude, unclustered; the paper's CIs are cluster-robust and Bonferroni-adjusted) ===")
for name, dct in (("No postoperative hyperglycemia", HYPERGLY), ("No postoperative hypothermia", HYPOTHERM),
("Intraoperative glucose measurement", GLU_MEAS)):
a1, n1 = dct["intervention"]
a0, n0 = dct["control"]
rr, lo, hi = rr_ci(a1, n1, a0, n0)
print(f"{name}: {pct(a1, n1):.2f}% vs {pct(a0, n0):.2f}%; crude RR {rr:.3f} (unclustered 95% CI {lo:.3f}-{hi:.3f})")
for g in ARMS:
a, n = GLU_MEAS[g]
print(f"glucose measurement {g}: {a}/{n} = {pct(a, n):.3f}% -> one decimal {pct(a, n):.1f} (printed {GLU_MEAS_PRINTED_PCT[g]})")
print("\n=== 6. Precision at diabetes-sized denominators (illustration only) ===")
a1, n1 = HYPERGLY["intervention"]
a0, n0 = HYPERGLY["control"]
rr, lo, hi = rr_ci(a1, n1, a0, n0)
print(f"reported denominators {n1}/{n0}: unclustered 95% CI half-width on log scale {(math.log(hi) - math.log(lo)) / 2:.4f}")
for p in (0.858, 0.70, 0.60):
m1, m0 = DIABETES["intervention"][0], DIABETES["control"][0]
se = math.sqrt((1 - p) / (p * m1) + (1 - p) / (p * m0))
print(f" {m1}/{m0} cases at avoidance rate {p:.3f} and RR 1: unclustered 95% CI {math.exp(-1.96 * se):.3f}-{math.exp(1.96 * se):.3f}")
print("\n=== 6b. Illustrative diabetes-only scenario (assumption, not an estimate) ===")
print("Assume every documented diabetic case (Table 1) is in the denominator and the cases without")
print("diabetes avoid hyperglycemia at the same rate q in both arms; back out the diabetic-only rates.")
_rr0, _lo0, _hi0 = rr_ci(*HYPERGLY["intervention"], *HYPERGLY["control"])
REF_W = _hi0 - _lo0
for q in (0.90, 0.95, 0.97, 0.99):
a = {}
for g in ARMS:
num, den = HYPERGLY[g]
d = DIABETES[g][0]
a[g] = num - q * (den - d)
m1, m0 = DIABETES["intervention"][0], DIABETES["control"][0]
rr, lo, hi = rr_ci(a["intervention"], m1, a["control"], m0)
print(f" q = {q:.2f}: diabetic avoidance {pct(a['intervention'], m1):.1f}% vs {pct(a['control'], m0):.1f}%; "
f"RR {rr:.3f} (unclustered 95% CI {lo:.3f}-{hi:.3f}); CI width {hi - lo:.3f}, "
f"{(hi - lo) / REF_W:.1f} x the unclustered width at the reported denominators")
print(f" reference: unclustered CI width at reported denominators = {REF_W:.3f} ({_lo0:.3f}-{_hi0:.3f})")
print("\n=== 7. Other, smaller inconsistencies ===")
for g in ARMS:
exp_t = N[g] - NO_TEMP[g]
print(f"hypothermia {g}: Figure 1 gives {N[g]} - {NO_TEMP[g]} = {exp_t}; Table 2 denominator {HYPOTHERM[g][1]}; "
f"difference {HYPOTHERM[g][1] - exp_t}")
print("Abstract: 'control group; n = 12 980' and 'intervention group; n = 13 274'; Methods, Table 1, Table 2 and "
"Figure 1 give intervention 12 980 and control 13 274 (labels swapped in the abstract).")
print("Abstract prints '[66.7.%]' for the control hypothermia percentage.")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, CONTEXT = "#eb6834", "#2a78d6", "#e4e3df", "#d6d5d0"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def fmt(n):
return f"{n:,}"
# Figure 1: counts per arm, horizontal bars on one count axis
rows = [] # (label, value, color, group)
for g in ARMS:
r = R[g]
rows += [
(f"All analyzed cases (Figure 1)", N[g], CONTEXT, g),
(f"Cases with glucose data: {fmt(N[g])} − {fmt(NO_GLUCOSE[g])} (Figure 1)", r["with_glu"], BLUE, g),
("Hyperglycemia denominator (Table 2)", r["den"], ORANGE, g),
(f"Most cases that could have diabetes:\n{fmt(r['d'])} + {fmt(r['unknown'])} status unknown (Table 1)", r["max_dm"], BLUE, g),
("Cases with documented diabetes (Table 1)", r["d"], BLUE, g),
]
fig, ax = plt.subplots(figsize=(7.2, 7.0))
BAR_H = 0.52
y, ys = 0.0, []
heads = {}
for i, (lab, val, col, g) in enumerate(rows):
if i % 5 == 0:
heads[g] = y
y += 0.8
ys.append(y)
y += 1.0
if i % 5 == 4:
y += 0.5
ys = [-v for v in ys]
for (lab, val, col, g), yy in zip(rows, ys):
ax.barh(yy, val, height=BAR_H, color=col, edgecolor="none", zorder=2)
ax.text(val + 150, yy, fmt(val), va="center", ha="left", fontsize=10, color=TXT, zorder=3)
for g, yy in heads.items():
ax.text(-200, -yy, f"{g.capitalize()} arm (n = {fmt(N[g])})", fontsize=11, fontweight="bold",
ha="right", va="center", color=TXT)
for i, g in enumerate(ARMS):
r = R[g]
y_den, y_max = ys[5 * i + 2], ys[5 * i + 3]
y_arrow = (y_den + y_max) / 2
ax.plot([r["max_dm"], r["max_dm"]], [y_max - BAR_H / 2, y_den + BAR_H / 2], color=TXT, lw=1.2,
ls=(0, (3, 2)), zorder=4)
ax.annotate("", xy=(r["den"], y_arrow), xytext=(r["max_dm"], y_arrow),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.4, shrinkA=0, shrinkB=0), zorder=5)
ax.text(r["den"] + 150, y_arrow, f"{fmt(r['den'] - r['max_dm'])} more than\nthe maximum",
ha="left", va="center", fontsize=10, color=TXT, zorder=5, linespacing=1.0)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 15800)
ax.set_xticks(range(0, 12001, 4000))
ax.set_xlabel("Surgical cases")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: fmt(int(v))))
from matplotlib.patches import Patch # noqa: E402
fig.legend(handles=[Patch(color=ORANGE, label="Reported denominator of the diabetes-only primary outcome (Table 2)"),
Patch(color=BLUE, label="Implied by the paper's other numbers (Table 1, Figure 1)"),
Patch(color=CONTEXT, label="All analyzed cases (context)")],
loc="lower left", bbox_to_anchor=(0.02, 0.005), frameon=False, fontsize=10, ncol=1)
fig.tight_layout(rect=(0, 0.11, 1, 1))
save(fig, "fig1")
# Figure 2: the same gap in all three published analyses
items = [("Main analysis (Tables 1-2)", g, R[g]["d"], R[g]["max_dm"], R[g]["den"]) for g in ARMS]
for name in SENS:
for g in ARMS:
s = SENS_R[(name, g)]
items.append((name, g, s["d"], s["max_dm"], s["den"]))
fig, ax = plt.subplots(figsize=(7.2, 5.0))
yy = []
y = 0
last = None
for name, g, d, mx, den in items:
if last is not None and name != last:
y += 0.6
yy.append(-y)
last = name
y += 1
for (name, g, d, mx, den), yv in zip(items, yy):
ax.plot([d, den], [yv, yv], color=CONTEXT, lw=2, zorder=1)
ax.plot([d], [yv], "o", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=3)
ax.plot([mx], [yv], "D", ms=8, color=BLUE, zorder=3)
ax.plot([den], [yv], "o", ms=9, color=ORANGE, zorder=3)
ax.text(den + 200, yv, f"+{fmt(den - mx)}", va="center", fontsize=10, color=TXT)
ax.set_yticks(yy)
ax.set_yticklabels([f"{n.split(' (')[0]}, {g}" for n, g, *_ in items], fontsize=10)
ax.set_xlim(0, 10600)
ax.set_xticks(range(0, 10001, 2000))
ax.set_xlabel("Surgical cases")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: fmt(int(v))))
from matplotlib.lines import Line2D # noqa: E402
fig.legend(handles=[
Line2D([], [], marker="o", ls="none", ms=9, mfc=BG, mec=BLUE, mew=2, label="Cases with documented diabetes (Table 1, eTables 4, 6)"),
Line2D([], [], marker="D", ls="none", ms=8, color=BLUE, label="Most cases that could have diabetes (recomputed)"),
Line2D([], [], marker="o", ls="none", ms=9, color=ORANGE, label="Reported hyperglycemia denominator (Table 2, eTables 5, 7)"),
Line2D([], [], ls="none", label="+n: reported denominator minus the maximum possible"),
], loc="lower left", bbox_to_anchor=(0.02, 0.005), frameon=False, fontsize=10, ncol=1)
fig.tight_layout(rect=(0, 0.2, 1, 1))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Subgroup effects called significantly different although their 95% CIs overlap, contrary to the paper's criterion
JAMA Network Open (2024) · doi:10.1001/jamanetworkopen.2023.56619 · PMC10891466
Summary
The paper asks whether food insecurity modifies the association between SNAP receipt and nonadherence to antihypertensive medication. Its Methods say two subgroup estimates count as significantly different only if their 95% CIs do not overlap, and its Results then call the food-insecure estimate (−13.6 percentage points [pp]; 95% CI −25.0 to −2.3) and the food-secure estimate (5.0 pp; 95% CI −15.8 to 25.7) "statistically significantly different". Those intervals overlap from −15.8 to −2.3, and a Wald test on the same intervals gives a difference of 18.6 pp (95% CI −5.1 to 42.3; p = 0.12). The paper answers yes to its own effect-modification question, but neither its stated criterion nor a conventional test supports that answer. The food-insecure estimate on its own is not affected.
What the paper reports
The research question (Abstract, Key Points, Question):
"Is receipt of Supplemental Nutrition Assistance Program (SNAP) benefits associated with nonadherence to antihypertensive medications and, if so, does this association vary by food insecurity status?"
The criterion for effect modification (Methods, Statistical Analysis, second paragraph):
"The potential effect modification of food insecurity was tested by evaluating for statistical differences in the magnitude and direction of the PATEs for the food-secure and food-insecure subgroups; PATEs were considered statistically significantly different if their respective 95% CIs did not overlap."
The two subgroup estimates (Results, "Association Between SNAP Receipt and Nonadherence to Antihypertensive Medications"):
"Similarly, the rate of nonadherence to antihypertensive medication was similar (385 [56.7% weighted] vs 2671 [55.2% weighted]; PATE, 5.0 [95% CI, -15.8 to 25.7]) between SNAP recipients and nonrecipients among the 5354 patients who did not report experiencing food insecurity."
"In contrast, among the subgroup of 1338 antihypertensive medication users who experienced food insecurity, the rate of nonadherence to antihypertensive medications was 8.17 percentage points lower among SNAP recipients compared with nonrecipients (302 [56.0% weighted] vs 491 [60.0% weighted]; PATE, -13.6 [95% CI, -25.0 to -2.3])."
The conclusion drawn from them (Results, same paragraph, the sentence that follows):
"The observed PATE estimates among the food-secure vs food-insecure subgroups were statistically significantly different, suggesting that food insecurity modified the association between SNAP and nonadherence to antihypertensive medication therapy."
Table 2 gives the same values: food insecure, 302 (56.0) vs 491 (60.0), PATE −13.6 (−25.0 to −2.3); food secure, 385 (56.7) vs 2671 (55.2), PATE 5.0 (−15.8 to 25.7); overall, 687 (56.4) vs 3162 (55.8), PATE 1.7 (−21.2 to 24.6).
The finding is carried into the Discussion (first paragraph):
"Among a US sample of patients with hypertension identified from a linked, nationally representative dataset, we observed that SNAP had a differential association with nonadherence to antihypertensive medications between food-secure and food-insecure subgroups."
and into the Strengths (Discussion, Strengths and Limitations):
"Second, this study presented data suggesting that the potential consequences of SNAP receipt on adherence to antihypertensive medications are greatest among the food-insecure population. Thus, we have identified the food-insecure hypertension population as a target for food assistance intervention to address adherence to antihypertensive medications, especially when food assistance resources are limited."
(Quotes are from the PubMed Central archive, which prints minus signs as hyphens; the publisher's page prints the same values with true minus signs.)
What the paper's own numbers show
Method. All inputs come from Table 2 and the Results text. (1) The paper's criterion is applied as written: do the two reported 95% CIs overlap? (2) Each standard error (SE) is recovered from its CI as (upper − lower)/(2 × 1.96). Both CIs are symmetric about their point estimates, so this is valid. (3) The two subgroups are disjoint sets of participants, and the Methods describe a stratified analysis with subgroup-specific propensity scores. The difference is therefore tested with the usual Wald test for two independent estimates, z = (5.0 − (−13.6)) / √(SE₁² + SE₂²). (4) To check whether a different confidence level would rescue the claim, the confidence level at which the two intervals just stop overlapping is also computed. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Food-insecure PATE (95% CI) | Table 2; Results; Abstract | −13.6 (−25.0 to −2.3) | Symmetric (half-widths 11.4 / 11.3); SE 5.79 pp; single-group p = 0.019 |
| Food-secure PATE (95% CI) | Table 2; Results | 5.0 (−15.8 to 25.7) | Symmetric (half-widths 20.8 / 20.7); SE 10.59 pp |
| Do the two 95% CIs overlap? (paper's criterion) | Methods vs Results | "statistically significantly different" | Yes, they overlap, from −15.8 to −2.3 (13.5 pp). The criterion is not met |
| Food-insecure point estimate inside the food-secure 95% CI? | Table 2 | — | Yes (−13.6 lies within −15.8 to 25.7) |
| Difference, food secure − food insecure | derived from Table 2 | "statistically significantly different" | 18.6 pp, SE 12.07, 95% CI −5.1 to 42.3 (includes 0) |
| Wald test of the difference | derived from Table 2 | no test statistic given | z = 1.54; two-sided p = 0.123 (one-sided p = 0.062) |
| Confidence level at which the two CIs stop overlapping | derived | — | Only below 74.4%. The 90%, 85.1%, 83.4%, 80% and 75% intervals all overlap |
| Confidence level at which non-overlap would match p < 0.05 for these SEs | derived | — | 85.1%. The intervals overlap at that level too |
| Correlation between the subgroup estimates needed for p = 0.05 | derived | — | 0.45 |
| Nonadherent counts | Table 2; Results | 687; 3162; 3849 | 302 + 385 = 687; 491 + 2671 = 3162; 687 + 3162 = 3849 (consistent) |
| Subgroup sizes | Results | 5354 food secure; 1338 food insecure | 6692 − 1338 = 5354; 540 + 798 = 1338 (consistent) |
| Food-insecure gap described in the text | Results | "8.17 percentage points lower" | Weighted crude gap 56.0 − 60.0 = −4.0 pp; unweighted 55.9% − 61.5% = −5.6 pp; adjusted PATE −13.6 pp. 8.17 matches none of them (separate, minor) |
| Model behind the PATEs | Methods and Abstract vs Table 2 footnote b | Methods and Abstract: probit; footnote b: "from a logit model" | Internal inconsistency in the description (minor; does not change the CIs) |
The counts and sample sizes are mutually consistent, so the problem is not in the reported data. It lies only in the inference drawn from the two intervals.
Figure
Figure 1. All values are on one percentage-point axis (SNAP recipients minus nonrecipients; negative means less nonadherence with SNAP). Orange: the two subgroup PATEs and 95% CIs as reported in Table 2. The shaded band marks where the two intervals overlap (−15.8 to −2.3, 13.5 pp); under the paper's stated rule, any overlap means the estimates are not significantly different. Blue: the difference between subgroups (food secure − food insecure), recomputed from the same table, 18.6 pp (95% CI −5.1 to 42.3; z = 1.54; p = 0.12). The thin vertical line labelled "null (0)" marks no effect for the subgroup rows and no difference for the bottom row; the interval for the difference includes it.
Figure 2. Would a different confidence level change the conclusion? Blue curves: the food-insecure upper bound (solid) and the food-secure lower bound (dashed), recomputed from the SEs implied by Table 2 for confidence levels from 50% to 99%. Orange points: the reported 95% bounds (−2.3 and −15.8). The intervals overlap (shaded) at every level above 74.4%. That includes 85.1%, the level at which non-overlap of these two particular intervals would correspond to a two-sided Wald p < 0.05 (dotted line).
Why it matters
Effect modification by food insecurity is the paper's stated question. It is the second half of the Key Points question, it is the Objective ("To evaluate whether receipt of SNAP benefits can modify the consequences of food insecurity on nonadherence to antihypertensive medications."), and it is the purpose of the prespecified stratified analysis. The claim that the association differs by food-insecurity status underpins:
- the Key Points Findings and the Abstract Results ("among the food-insecure subgroup but not among their food-secure counterparts");
- the Key Points Meaning ("especially among food-insecure individuals") and the Abstract and main Conclusions ("especially if they are experiencing food insecurity");
- the opening of the Discussion ("a differential association ... between food-secure and food-insecure subgroups");
- the Strengths paragraph, which says the consequences of SNAP receipt are "greatest among the food-insecure population" and names that population "as a target for food assistance intervention".
On the paper's own numbers, the data are compatible with no difference between the subgroups: the difference has a 95% CI of −5.1 to 42.3 pp. A result that is significant in one subgroup and not in the other is not evidence that the subgroups differ, and the paper's non-overlap rule, conservative as it is, was the guard it set against that inference. What survives is the food-insecure estimate on its own (−13.6 pp, CI excluding 0, p ≈ 0.02 before any adjustment for multiple subgroups). The literal wording "associated ... among the food-insecure subgroup but not among their food-secure counterparts" therefore remains accurate as a description of the two subgroup estimates. The statement that food insecurity modified the association, the "differential association", and the targeting recommendations built on it are not supported. The hedged overall conclusion ("may be less likely ... especially if") is weakened rather than reversed.
Explanations considered
- The table and the text give different numbers. They do not. Table 2, the Results text and the Abstract all give −13.6 (−25.0 to −2.3); Table 2 and the Results give 5.0 (−15.8 to 25.7). Neither is a typo of the other.
- A sign was lost in conversion (for example, −2.3 was meant to be +2.3). No. Both intervals are symmetric about their point estimates (half-widths 11.4/11.3 and 20.8/20.7), and the publisher's own page prints these values with true minus signs. A positive upper bound would also make the food-insecure estimate itself non-significant, which contradicts the Abstract.
- Different estimands, scales or models. Both subgroup PATEs are average marginal effects on the percentage-point scale from the same weighted model specification. The probit/logit wording differs between the Methods and the Table 2 footnote, but that affects both subgroups alike and does not change the printed CIs.
- "Magnitude and direction" means opposite-signed point estimates are enough. The Methods attach that phrase to a statistical test with a specific rule (non-overlap of 95% CIs). Opposite signs of the point estimates do not satisfy that rule; the food-secure interval (−15.8 to 25.7) contains the food-insecure point estimate.
- A formal interaction test was run but not reported. The Methods name no such test, and Supplement 1 (eAppendix 2, the weighting and regression specification) contains no interaction term, heterogeneity test or other criterion. An unreported test cannot be ruled out, but the stated criterion is not met and a Wald test on the published intervals gives p = 0.12.
- A one-sided test, or a lower confidence level. Nothing in the paper states either. One-sided p = 0.062. The intervals separate only at confidence levels below 74.4% (Figure 2).
- The subgroup estimates are correlated, so the Wald SE is too large. For p = 0.05 the two estimates would need a correlation of about 0.45. They come from disjoint sets of participants with separately fitted weights; shared survey strata and clusters can induce some covariance, but one of that size is not plausible. In any case, the paper's own non-overlap criterion does not depend on the correlation.
- The error was corrected after publication. No correction, erratum or retraction was found (see Status).
One further observation does not change the conclusion. The overall-sample interval (1.7, −21.2 to 24.6; half-width 22.9) is about twice as wide as the food-insecure interval (half-width 11.4), although it covers five times as many participants. Without data or code this cannot be examined further; the analysis above uses the subgroup intervals exactly as published.
Status
As of 2026-10-23:
- Version of record. The publisher's page (doi.org redirects to jamanetwork.com, article 2815447) was read on 2026-10-23. It contains the criterion sentence ("PATEs were considered statistically significantly different if their respective 95% CIs did not overlap."), the "statistically significantly different, suggesting that food insecurity modified the association" sentence, the "8.17 percentage points lower" sentence, and the Table 2 rows food insecure −13.6 (−25.0 to −2.3) and food secure 5.0 (−15.8 to 25.7). The PMC archive (
src/show.py) and the Europe PMC full-text XML carry the same sentences and numbers, and the "from a logit model" footnote. The problem is in the version of record, not introduced by conversion. - Supplements. Supplement 1 (eAppendices 1 and 2, eFigure, eTable, eReferences) contains no subgroup comparison, interaction term or additional subgroup estimates. Supplement 2 is the Data Sharing Statement ("Data available: No"). Neither resolves the discrepancy, and no data or code are available to re-estimate the subgroup models.
- Corrections and comments. Crossref update metadata, PMC notices and Europe PMC notices show no correction, erratum, expression of concern or retraction; Europe PMC searches for an erratum in JAMA Network Open returned no hits. The only linked item is an invited commentary (PMID 38393734, "Food Security to Medication Adherence-Connecting Needs"), which is not a correction. The publisher page shows no correction notice. PubPeer could not be checked (its API and search page returned HTTP 403/404 to automated requests), and a general web search was not available in this session.
- Sources. https://doi.org/10.1001/jamanetworkopen.2023.56619 ; https://pmc.ncbi.nlm.nih.gov/articles/PMC10891466/ ; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC10891466/fullTextXML ; Crossref REST API record for the DOI.
Suggested correction
- Replace "The observed PATE estimates among the food-secure vs food-insecure subgroups were statistically significantly different, suggesting that food insecurity modified the association …" with a statement that matches the stated criterion, for example: "The 95% CIs for the food-secure and food-insecure subgroups overlapped; by our prespecified criterion, the PATEs were therefore not statistically significantly different, and we did not find statistical evidence that food insecurity modified the association."
- Preferably also report a direct estimate of the difference between subgroups (an interaction model, or a test of the difference between the two PATEs), with its CI and p value. From the published intervals this is approximately 18.6 pp (95% CI −5.1 to 42.3; p ≈ 0.12).
- Revise the Key Points (Findings, Meaning), the Abstract Results and Conclusions, the opening of the Discussion, the Strengths paragraph and the Conclusions so that they report the food-insecure estimate without presenting a difference between subgroups as established.
- Correct "8.17 percentage points lower" to agree with Table 2 (adjusted PATE −13.6; weighted crude difference −4.0), and make footnote b of Table 2 ("from a logit model") agree with the probit model described in the Methods and the Abstract, or vice versa.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC10891466 write-up and regenerate its figures.
Paper: "Supplemental Nutrition Assistance Program and Adherence to
Antihypertensive Medications", JAMA Network Open 2024,
doi:10.1001/jamanetworkopen.2023.56619 (PMC10891466).
All inputs are copied from Table 2 and the Results text of the paper.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Figures are written next to this file as fig1.svg/png and fig2.svg/png.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Table 2: PATE, percentage-point difference (95% CI)
ROWS = {
"overall": (1.7, -21.2, 24.6),
"food_insecure": (-13.6, -25.0, -2.3),
"food_secure": (5.0, -15.8, 25.7),
}
# Table 2: nonadherent, No. (weighted %), SNAP vs no SNAP
COUNTS = {
"overall": ((687, 56.4), (3162, 55.8)),
"food_insecure": ((302, 56.0), (491, 60.0)),
"food_secure": ((385, 56.7), (2671, 55.2)),
}
N_TOTAL, N_FI, N_FS = 6692, 1338, 5354 # Results text
N_FI_SNAP, N_FI_NOSNAP = 540, 798 # Results text
NONADH_TOTAL = 3849 # Results text
TEXT_FI_GAP = 8.17 # Results text, "8.17 percentage points lower"
Z975 = norm.ppf(0.975)
def section(title):
print("\n" + "=" * 72 + "\n" + title + "\n" + "=" * 72)
# ------------------------------------------- 1. SEs implied by the CIs
section("1. Standard errors implied by the reported 95% CIs")
SE = {}
for k, (est, lo, hi) in ROWS.items():
SE[k] = (hi - lo) / (2 * Z975)
print(f"{k:14s} PATE {est:6.1f} CI [{lo:6.1f}, {hi:5.1f}] "
f"lower half-width {est - lo:5.1f} upper half-width {hi - est:5.1f} "
f"midpoint {(lo + hi) / 2:6.2f} SE {SE[k]:6.2f} "
f"single-group two-sided p {2 * norm.sf(abs(est) / SE[k]):.3f}")
print("-> Each CI is symmetric about its estimate (to rounding), so no minus sign is lost.")
fi, fs = ROWS["food_insecure"], ROWS["food_secure"]
# ------------------------------------------- 2. The paper's own criterion
section("2. The paper's own criterion: do the two 95% CIs overlap?")
ov_lo, ov_hi = max(fi[1], fs[1]), min(fi[2], fs[2])
print(f"Overlap interval: [{ov_lo}, {ov_hi}] overlap = {ov_lo < ov_hi} width = {ov_hi - ov_lo:.1f} pp")
print(f"Food-insecure estimate {fi[0]} inside food-secure CI [{fs[1]}, {fs[2]}]: {fs[1] <= fi[0] <= fs[2]}")
print(f"Food-secure estimate {fs[0]} inside food-insecure CI [{fi[1]}, {fi[2]}]: {fi[1] <= fs[0] <= fi[2]}")
print(f"Food-secure CI covers 0: {fs[1] <= 0 <= fs[2]}; food-insecure CI covers 0: {fi[1] <= 0 <= fi[2]}")
# ------------------------------------------- 3. Wald test of the difference
section("3. Wald test of the difference between the subgroup PATEs (independent strata)")
d = fs[0] - fi[0]
se_d = np.hypot(SE["food_insecure"], SE["food_secure"])
z = d / se_d
p2 = 2 * norm.sf(abs(z))
p1 = norm.sf(abs(z))
d_lo, d_hi = d - Z975 * se_d, d + Z975 * se_d
print(f"Difference (food secure - food insecure) = {d:.1f} pp")
print(f"SE = sqrt({SE['food_insecure']:.2f}^2 + {SE['food_secure']:.2f}^2) = {se_d:.2f} pp")
print(f"z = {z:.2f}; two-sided p = {p2:.3f}; one-sided p = {p1:.3f}")
print(f"95% CI for the difference: {d_lo:.1f} to {d_hi:.1f} pp (includes 0: {d_lo <= 0 <= d_hi})")
# correlation between the two estimates that would be needed for p = 0.05
se_needed = d / Z975
rho = (SE["food_insecure"] ** 2 + SE["food_secure"] ** 2 - se_needed ** 2) / (
2 * SE["food_insecure"] * SE["food_secure"])
print(f"For two-sided p = 0.05 the SE of the difference would have to be {se_needed:.2f}, "
f"i.e. a correlation of {rho:.2f} between the two subgroup estimates "
f"(they come from disjoint groups of participants, fitted separately).")
# ------------------------------------------- 4. Narrower CI levels
section("4. At which confidence level would the intervals stop overlapping?")
for lvl in (0.95, 0.90, 0.851, 0.834, 0.80, 0.75):
zz = norm.ppf(0.5 + lvl / 2)
fi_hi = fi[0] + zz * SE["food_insecure"]
fs_lo = fs[0] - zz * SE["food_secure"]
print(f"{lvl * 100:5.1f}% CIs: food-insecure upper {fi_hi:6.2f}, food-secure lower {fs_lo:6.2f} "
f"-> overlap = {fs_lo < fi_hi}")
z_touch = d / (SE["food_insecure"] + SE["food_secure"])
lvl_touch = 2 * norm.cdf(z_touch) - 1
print(f"The intervals just touch at the {100 * lvl_touch:.1f}% level (z = {z_touch:.2f}).")
# level at which 'non-overlap' would coincide with a two-sided p = 0.05 Wald test, for these SEs
z_eq = Z975 * se_d / (SE["food_insecure"] + SE["food_secure"])
lvl_eq = 2 * norm.cdf(z_eq) - 1
print(f"For these two SEs, non-overlap of {100 * lvl_eq:.1f}% CIs is equivalent to two-sided p < 0.05; "
f"the intervals still overlap at that level ({100 * lvl_touch:.1f}% < {100 * lvl_eq:.1f}%).")
# ------------------------------------------- 5. Other numbers in the paragraph
section("5. Other numbers in the same Results paragraph")
s_snap = COUNTS["food_insecure"][0][0] + COUNTS["food_secure"][0][0]
s_nos = COUNTS["food_insecure"][1][0] + COUNTS["food_secure"][1][0]
print(f"SNAP nonadherent 302 + 385 = {s_snap} (reported {COUNTS['overall'][0][0]})")
print(f"No-SNAP nonadherent 491 + 2671 = {s_nos} (reported {COUNTS['overall'][1][0]})")
print(f"Total nonadherent 687 + 3162 = {687 + 3162} (reported {NONADH_TOTAL})")
print(f"Food-secure n = 6692 - 1338 = {N_TOTAL - N_FI} (reported {N_FS})")
print(f"Food-insecure n = 540 + 798 = {N_FI_SNAP + N_FI_NOSNAP} (reported {N_FI})")
raw_fi = COUNTS["food_insecure"][0][1] - COUNTS["food_insecure"][1][1]
raw_fs = COUNTS["food_secure"][0][1] - COUNTS["food_secure"][1][1]
print(f"Food-insecure weighted raw difference 56.0 - 60.0 = {raw_fi:.1f} pp; "
f"adjusted PATE {fi[0]} pp; text says {TEXT_FI_GAP} pp lower "
f"(matches neither: {abs(raw_fi) != TEXT_FI_GAP and abs(fi[0]) != TEXT_FI_GAP})")
print(f"Food-secure weighted raw difference 56.7 - 55.2 = {raw_fs:.1f} pp; adjusted PATE {fs[0]} pp")
uw_snap = 100 * COUNTS["food_insecure"][0][0] / N_FI_SNAP
uw_nos = 100 * COUNTS["food_insecure"][1][0] / N_FI_NOSNAP
print(f"Food-insecure unweighted: 302/540 = {uw_snap:.1f}%, 491/798 = {uw_nos:.1f}%, "
f"difference {uw_snap - uw_nos:.1f} pp (also not 8.17)")
# ------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans",
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK, "text.color": INK, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG,
})
def fmt(v):
return f"{v:.1f}".replace("-", "\u2212")
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
# Figure 1: the two reported subgroup CIs and the recomputed difference, one axis
fig, ax = plt.subplots(figsize=(7.2, 3.9))
rows = [
("Food insecure\n(Table 2, reported)", fi[0], fi[1], fi[2], ORANGE, "o"),
("Food secure\n(Table 2, reported)", fs[0], fs[1], fs[2], ORANGE, "o"),
("Difference, secure − insecure\n(recomputed from Table 2)", d, d_lo, d_hi, BLUE, "D"),
]
ys = [2.0, 1.0, -0.3]
ax.axvspan(ov_lo, ov_hi, ymin=0.38, ymax=0.97, color="#f1d9cf", alpha=0.6, lw=0, zorder=0)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
for (lab, est, lo, hi, col, mk), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([lo, lo], [y - 0.09, y + 0.09], color=col, lw=2, zorder=3)
ax.plot([hi, hi], [y - 0.09, y + 0.09], color=col, lw=2, zorder=3)
ax.plot(est, y, marker=mk, ms=9, color=col, zorder=4)
ax.text(hi + 1.2, y, f"{fmt(est)} ({fmt(lo)} to {fmt(hi)})", va="center", fontsize=10, color=col,
bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"), zorder=5)
ax.text(4.0, 2.6,
f"95% CIs overlap from {fmt(ov_lo)} to {fmt(ov_hi)} ({ov_hi - ov_lo:.1f} pp):\nthe paper's criterion for 'different' is not met",
ha="center", va="center", fontsize=10, color=INK, bbox=dict(boxstyle="square,pad=0.2", fc=BG, ec="none"))
ax.text(d, -0.75, f"Wald z = {z:.2f}, two-sided p = {p2:.2f}; CI includes 0",
ha="center", va="center", fontsize=10, color=BLUE, bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"), zorder=5)
ax.axhline(0.35, color=GRID, lw=1)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.set_ylim(-1.0, 2.9)
ax.set_xlim(-30, 64)
ax.set_xlabel("Percentage-point difference in nonadherence, SNAP vs no SNAP (PATE)")
ax.xaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.6, 0.45, "null (0)", fontsize=10, color=INK2, va="bottom")
save(fig, "fig1")
# Figure 2: interval bounds as a function of confidence level
lv = np.linspace(0.50, 0.99, 400)
zz = norm.ppf(0.5 + lv / 2)
fi_up = fi[0] + zz * SE["food_insecure"]
fs_lo = fs[0] - zz * SE["food_secure"]
fig, ax = plt.subplots(figsize=(7.2, 3.8))
ax.plot(lv * 100, fi_up, color=BLUE, lw=2, label="Food insecure: upper CI bound (recomputed)")
ax.plot(lv * 100, fs_lo, color=BLUE, lw=2, ls="--", label="Food secure: lower CI bound (recomputed)")
ax.plot([95], [fi[2]], "o", ms=9, color=ORANGE, zorder=5)
ax.plot([95], [fs[1]], "o", ms=9, color=ORANGE, zorder=5, label="Reported 95% CI bounds (Table 2)")
ax.annotate(fmt(fi[2]), (95, fi[2]), xytext=(-34, 4), textcoords="offset points", color=ORANGE, fontsize=10)
ax.annotate(fmt(fs[1]), (95, fs[1]), xytext=(-40, -4), textcoords="offset points", color=ORANGE, fontsize=10)
ax.fill_between(lv * 100, fs_lo, fi_up, where=fs_lo < fi_up, color="#f1d9cf", alpha=0.6, lw=0)
ax.axvline(lvl_touch * 100, color=INK2, lw=0.8)
ax.text(lvl_touch * 100 - 0.8, 4.0, f"intervals separate only at\nlevels below {lvl_touch * 100:.1f}%",
ha="right", va="center", fontsize=10, color=INK)
ax.axvline(lvl_eq * 100, color=INK2, lw=0.8, ls=":")
ax.text(lvl_eq * 100 + 0.5, 4.0, f"{lvl_eq * 100:.1f}%: non-overlap here\n\u21d4 Wald p < 0.05", ha="left", va="center",
fontsize=10, color=INK2)
ax.text(91.0, -9.5, "overlap", ha="center", fontsize=10, color=INK)
ax.set_xlim(50, 99)
ax.set_ylim(-27, 8)
ax.set_xlabel("Confidence level of both intervals (%)")
ax.set_ylabel("Percentage points")
ax.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="lower left", frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written to", HERE)
Printed stroke confidence interval excludes its own estimate; Table 2 shows the effect is non-significant
JAMA Network Open (2025) · doi:10.1001/jamanetworkopen.2025.39862 · PMC12569718
Summary
This UK Biobank cohort study used longitudinal targeted maximum likelihood estimation to estimate how 5-year risks of hospitalization for cardiovascular disease would change under hypothetical PM2.5 interventions at thresholds of 12, 10 and 9 µg/m³. The Abstract and the Results text give the stroke risk difference at the 9 µg/m³ threshold (the US EPA annual standard) as −2.26 per mille with a 95% CI of −8.97 to −20.64. That interval is impossible: its lower bound is above its upper bound, and the point estimate lies outside it. Table 2, Supplement 1 eTable 2, and a recomputation from the Table 2 risks all give an upper bound of about +20.6, so the interval includes zero and the stroke effect at 9 µg/m³ is not statistically significant. The Key Points nonetheless say that the 12 and 9 µg/m³ thresholds "would significantly reduce" stroke, heart failure and myocardial infarction. By Table 2, that holds for myocardial infarction, but not for stroke at 9 µg/m³ or for heart failure at 12 µg/m³. Two smaller sign discrepancies between the text and Table 2 point in the same direction.
What the paper reports
The PMC-archived text uses hyphen-minus signs, and the quotes below reproduce it. The publisher's version of record prints the same characters as true minus signs (U+2212).
Key Points, Findings:
implementing a hypothetical intervention for reducing PM2.5 exposure by 5% if it is above the thresholds of 12 and 9 µg/m3 would significantly reduce the 5-year risk of hospitalization for stroke, heart failure, and myocardial infarction, but not for arrhythmia.
Abstract, Results (the same sentence fragment also appears in the Results text, in the paragraph that describes Table 2):
the hypothetical PM2.5 intervention of reducing PM2.5 exposure by 5% if it is above the threshold of 9 µg/m3, the estimated 5-year risk difference of hospitalization for stroke was -2.26 per mille (95% CI, -8.97 to -20.64 per mille)
Results text, Table 2 paragraph:
For the threshold intervention of 9 µg/m3 compared with no intervention, the estimated 5-year risk difference of hospitalization for stroke was -2.26 per mille (95% CI, -8.97 to -20.64 per mille)
Table 2, stroke rows (columns: hypothetical intervention | 5-y risk (95% CI), per mille | risk difference (95% CI), per mille):
No intervention | 12.32 (11.92 to 12.72) | [Reference] ≤12 | 10.78 (10.07 to 11.67) | -1.54 (-2.21 to -0.73) ≤10 | 6.98 (3.76 to 19.60) | -5.34 (-8.59 to 7.24) ≤9 | 10.06 (3.37 to 32.97) | -2.26 (-8.97 to 20.64)
Supplement 1, eTable 2, stroke, main analysis (columns: no-intervention risk | ≤9 µg/m³ risk | risk difference):
Main analysis 12.32 (11.92, 12.72) 10.06 (3.37, 32.97) -2.26 (-8.97 to 20.64)
The two further discrepancies. Abstract, Results, stroke at 12 µg/m³:
the estimated 5-year risk difference of hospitalization for stroke was -1.54 per mille (95% CI, -2.21 to 0.73 per mille). However, the reduction in risk for arrhythmia was not statistically significant
Results text, stroke at 12 µg/m³, which agrees with Table 2:
the estimated 5-year risk difference of hospitalization for stroke was -1.54 per mille (95% CI, -2.21 to -0.73 per mille)
Results text, heart failure at 10 µg/m³:
for heart failure, -2.15 per mille (95% CI, -3.55 to -0.35 per mille)
Table 2, heart failure row ≤10:
≤10 | 4.35 (2.95 to 6.89) | -2.15 (-3.55 to 0.35)
Heart failure at 12 µg/m³, which agrees between the Results text and Table 2:
for heart failure, -0.51 per mille (95% CI, -1.07 to 0.35 per mille)
The significance criterion, from Methods, Statistical Analysis:
Two-sided P < .05 was considered statistically significant.
The statements that build on these results. Discussion, first paragraph:
Our findings indicate a decreased 5-year risk of hospitalization with a primary diagnosis of stroke, myocardial infarction, or heart failure for the hypothetical intervention of reducing PM2.5 exposure by 5% if it is above the annual average standards of 12 µg/m3, 10 µg/m3, or 9 µg/m3 compared with no intervention on ambient PM2.5. However, the 5-year risk difference was not statistically significant for the hospitalization with a primary diagnosis of arrhythmia.
Discussion, comparison with earlier studies:
While our study found decreased 5-year risks of stroke, myocardial infarction, and heart failure for the hypothetical PM2.5 intervention, the risk difference was not statistically significant for the outcome of arrhythmia.
Conclusions:
Our results indicate that a more stringent ambient PM2.5 standard in the United Kingdom may lower the risk of hospitalization for stroke, myocardial infarction, and heart failure.
What the paper's own numbers show
Method. Table 2 gives, for each of four outcomes, the 5-year risk with no intervention (the reference) and under each of the three threshold interventions, each with a 95% CI, plus the risk difference (RD) with its own 95% CI. I made two checks from Table 2 alone. (a) The RD point estimate should equal the intervention risk minus the reference risk. (b) The RD CI should be close to the intervention-risk CI minus the reference point estimate. Check (b) is an approximation: it ignores the uncertainty in the reference risk and its correlation with the intervention risk. It works here because the reference risks are estimated far more precisely (for stroke, 11.92 to 12.72) than the intervention risks (for stroke at 9 µg/m³, 3.37 to 32.97). I measured the approximation's accuracy against all 24 Table 2 RD bounds and used that accuracy as the yardstick for the text-versus-table discrepancies. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Accuracy of check (a), all 12 RD point estimates | Table 2 | — | every RD equals risk(intervention) − risk(no intervention) within 0.010 per mille (rounding) |
| Accuracy of check (b), all 24 RD CI bounds | Table 2 | — | every bound within 0.22 per mille of risk CI − reference risk (largest gap: arrhythmia ≤10 upper, 1.28 vs 1.06) |
| Stroke RD, ≤9 µg/m³ | Abstract; Results text | −2.26 (−8.97 to −20.64) | lower bound > upper bound; −2.26 is outside the interval, and stays outside if the bounds are reordered (−20.64 to −8.97) |
| Stroke RD, ≤9 µg/m³ | Table 2; eTable 2 | −2.26 (−8.97 to 20.64) | 10.06 − 12.32 = −2.26; (3.37 − 12.32, 32.97 − 12.32) = (−8.95 to +20.65). Includes 0 |
| Stroke RD upper bound, ≤9 µg/m³ | — | — | text vs recomputed: 41.29 apart; Table 2 vs recomputed: 0.01 apart. Using either end of the reference CI instead of 12.32 gives 20.25 to 21.05, so the bound is positive in every case |
| Stroke RD, ≤9 µg/m³, three robustness checks | eTable 2 | −3.39 (−9.25 to 18.47); −2.26 (−9.30 to 20.49); −4.02 (−9.26 to 14.91) | (−9.17 to 18.39); (−9.27 to 20.54); (−9.20 to 15.05). All include 0 |
| Stroke RD, ≤12 µg/m³ | Abstract | −1.54 (−2.21 to 0.73) | Table 2 and Results text: −2.21 to −0.73; recomputed (−2.25 to −0.65). The abstract's upper bound is 1.38 from the recomputed value, against 0.08 for Table 2 |
| Heart failure RD, ≤10 µg/m³ | Results text | −2.15 (−3.55 to −0.35) | Table 2: −3.55 to 0.35; recomputed (−3.55 to +0.39). The text's upper bound is 0.74 from the recomputed value, against 0.04 for Table 2. Includes 0 |
| Heart failure RD, ≤12 µg/m³ | Results text; Table 2 | −0.51 (−1.07 to 0.35) | recomputed (−1.13 to +0.41). Consistent, and includes 0 |
| Stroke RD, ≤10 µg/m³ | Results text; Table 2 | −5.34 (−8.59 to 7.24) | recomputed (−8.56 to +7.28). Consistent, and includes 0 |
| Myocardial infarction RD, ≤12 and ≤9 µg/m³ | Results text; Table 2 | −1.41 (−1.98 to −0.67); −8.64 (−9.16 to −6.38) | recomputed (−2.03 to −0.62); (−9.12 to −6.44). Consistent, and both exclude 0 |
| Heart failure RD, ≤9 µg/m³ | Abstract; Results text; Table 2 | −3.20 (−4.16 to −1.25) | recomputed (−4.10 to −1.30). Consistent, and excludes 0 |
| All other text values (abstract and Results text) | — | — | identical to Table 2 |
The paper does not report P values for the main risk differences. When a test and its 95% CI are derived consistently, a CI that includes zero corresponds to a two-sided P ≥ .05, which is the paper's own significance threshold. By Table 2, the significance of each risk difference is:
| Outcome | ≤12 µg/m³ | ≤10 µg/m³ | ≤9 µg/m³ |
|---|---|---|---|
| Stroke | significant | not significant | not significant |
| Myocardial infarction | significant | significant | significant |
| Heart failure | not significant | not significant (the Results text's −0.35 would make it appear significant) | significant |
| Arrhythmia | not significant | not significant | not significant |
The Key Points claim six significant reductions (stroke, heart failure and myocardial infarction, each at 12 and 9 µg/m³). Table 2 supports four of them. The two it does not support are stroke at 9 µg/m³, where only the impossible text interval excludes zero, and heart failure at 12 µg/m³, where the text and the table agree that the interval includes zero.
One further, minor observation. In Figure 3 of the paper (a forest plot of the 9 µg/m³ main analysis and robustness checks), the stroke main-analysis interval is drawn crossing the zero line, ending near 0.02 on the axis. That agrees with Table 2. The axis is labeled "per mille", but its tick values (−0.02 to 0.03) appear to be on the proportion scale, where 0.02 corresponds to 20 per mille.
Figure
Figure 1. The 5-year stroke risk difference for the 9 µg/m³ intervention versus no intervention, in per mille, on one linear axis. Orange marks are values printed in the paper; the hollow orange marker is the text value that disagrees with Table 2. Blue is the interval recomputed as the Table 2 risk CI under the intervention (3.37 to 32.97) minus the no-intervention risk (12.32). The thin vertical line is zero difference. The interval printed in the Abstract and Results text (−8.97 to −20.64) lies entirely to the left of its own point estimate. Table 2, eTable 2 and the recomputation agree on an interval of about −9 to +20.6, which includes zero. Plus signs in the value column are added for clarity; the paper prints positive bounds without a sign.
Figure 2. Three risk differences on one linear axis (per mille), using the same encoding as Figure 1. For stroke at 12 µg/m³, the Abstract prints an upper bound of +0.73. The Results text and Table 2 print −0.73, and the recomputed bound is −0.65. For heart failure at 10 µg/m³, the Results text prints −0.35, while Table 2 prints +0.35 and the recomputed bound is +0.39. In both cases the recomputation sides with Table 2, within the approximation's measured accuracy of 0.22 per mille. Heart failure at 12 µg/m³ is printed consistently, and its interval includes zero. Even so, the Key Points describe it as a significant reduction.
Why it matters
The 9 µg/m³ threshold is the one the paper ties to the US EPA primary annual standard, and the Abstract reports the 9 µg/m³ results first. The impossible stroke interval appears twice, in the Abstract and in the Results text. In both places all of the printed interval lies below zero, which makes the stroke reduction look statistically significant. The paper's own table says otherwise. Under the correct interval (−8.97 to +20.64 per mille) the data are compatible with anything from about a 9 per mille decrease to a 21 per mille increase in 5-year stroke hospitalization risk. Stroke is also not significant at 10 µg/m³, and it is not significant in any of the three robustness checks at 9 µg/m³. So the paper's data support a significant stroke reduction only at 12 µg/m³.
The Key Points sentence ("thresholds of 12 and 9 µg/m3 would significantly reduce ... stroke, heart failure, and myocardial infarction") is what most readers take from the article. Read naturally, it is not supported by Table 2 for stroke at 9 µg/m³ or for heart failure at 12 µg/m³. The Discussion's first paragraph lists decreased risk of stroke, myocardial infarction and heart failure at all three thresholds and contrasts them with arrhythmia, which "was not statistically significant". Read that way, the paragraph implies significance for stroke at 10 and 9 µg/m³ and for heart failure at 12 and 10 µg/m³, none of which Table 2 supports. The point estimates are all negative, so "decreased" is accurate as a description of direction only.
The scope of the problem is limited. Myocardial infarction is significant at every threshold, and heart failure at 9 µg/m³. The general conclusion that stricter PM2.5 standards may lower cardiovascular hospitalization is still supported by those results. What changes is the outcome-specific claim, which for stroke matters at the threshold most relevant to current policy.
Explanations considered
- A sign slip when the table values were transcribed into the text. This is the most plausible explanation, and it accounts for all three discrepancies. In each case one bound differs from Table 2 only by its sign: −20.64 for +20.64, +0.73 for −0.73, and −0.35 for +0.35. Table 2 agrees with the recomputation each time. It does not, however, rescue the Key Points or the Discussion, which match the mistyped text value for stroke at 9 µg/m³ rather than the table.
- A conversion artifact in PMC. Ruled out. The publisher's version of record, the Europe PMC full-text XML and the PMC page all print the same values.
- Table 2 is wrong and the text is right. Ruled out for stroke at 9 µg/m³. The text interval does not contain its own estimate, so it cannot be a valid confidence interval on any scale. Table 2's upper bound matches the risk CI (32.97 − 12.32 = 20.65). eTable 2 gives the same value, and the paper's Figure 3 draws the interval across zero. For the other two discrepancies, the recomputation sides with the table (0.08 and 0.04 per mille away) and not with the text (1.38 and 0.74 away). The approximation's largest error across all 24 table bounds is 0.22 per mille.
- The text uses a different interval method (for example a transformed scale or a different bootstrap). Ruled out for stroke at 9 µg/m³. Any valid interval contains its point estimate, and reordering the printed bounds (−20.64 to −8.97) still excludes −2.26. For the other two, nothing in the Methods describes a second interval method, and the remaining text values match Table 2 exactly.
- The recomputation itself is too crude. It cannot account for the numbers. Check (b) is only an approximation, but across all 24 Table 2 bounds it is accurate to 0.22 per mille. For stroke at 9 µg/m³ the discrepancy is 41 per mille. Using either end of the reference CI in place of its point estimate still gives an upper bound of 20.25 to 21.05.
- The Key Points use a looser reading of "12 and 9". This is possible. If the sentence means only that each outcome is significantly reduced at one or other threshold, Table 2 supports it: stroke at 12, heart failure at 9, myocardial infarction at both. But the Abstract's printed stroke interval at 9 µg/m³ lies wholly below zero, and the Discussion claims decreased risk at all three thresholds in contrast to the non-significant arrhythmia result. Both point to the stricter reading, under which two of the six claimed reductions are not significant.
- Significance was judged by P values rather than CIs. The paper reports no P values for the main risk differences, only CIs. The stated criterion is two-sided P < .05. With a consistently derived 95% CI that includes zero, that criterion is not met.
Status
As of 2026-10-23:
- Version of record. The publisher's full text (jamanetwork.com, article 2840619, reached through the doi.org redirect), the Europe PMC full-text XML and the PMC HTML page all carry the same numbers, with true minus signs (U+2212). In each, the Abstract and the Results text print the stroke interval at 9 µg/m³ as "−8.97 to −20.64", and Table 2 prints "−8.97 to 20.64". The Abstract's stroke interval at 12 µg/m³ (−2.21 to 0.73) and the Results text's heart failure interval at 10 µg/m³ (−3.55 to −0.35) also appear in the version of record. Every quote above was checked verbatim against the PMC-archived text.
- Supplements checked. Supplement 1 (eAppendices 1–2, eMethods, eTable 1 on subgroups, eTable 2 on the main analysis and robustness checks, eFigures 1–4). Its eTable 2 agrees with Table 2 for stroke at 9 µg/m³ (−2.26, −8.97 to 20.64), and all three stroke robustness checks include zero. Supplement 2 is the data sharing statement. Both were taken from the Europe PMC supplementary-files archive. The paper's Figure 3 image was also checked, and it draws the stroke interval at 9 µg/m³ across zero.
- Corrections and comments. I found no correction, erratum, retraction or expression of concern. Crossref lists no update-to or updated-by relations for the DOI. PMC has no linked notice. Europe PMC (PMID 41148138) has no comment/correction list, and a Europe PMC search for an erratum under the title returned nothing. Neither the publisher's page nor the PMC page shows a notice. PubPeer could not be checked, because its search page returned HTTP 403 to automated access and the API was blocked. A general web search was not available for this check.
Suggested correction
- In the Abstract (Results) and the Results text, change the stroke risk difference at 9 µg/m³ to "−2.26 per mille (95% CI, −8.97 to 20.64 per mille)", to match Table 2 and eTable 2.
- In the Abstract, change the stroke interval at 12 µg/m³ to "−2.21 to −0.73 per mille". In the Results text, change the heart failure interval at 10 µg/m³ to "−3.55 to 0.35 per mille".
- Revise the Key Points to match Table 2. For example: the 12 and 9 µg/m³ thresholds significantly reduced the risk for myocardial infarction. The reduction for stroke was significant at 12 µg/m³ but not at 10 or 9 µg/m³. The reduction for heart failure was significant at 9 µg/m³ but not at 12 or 10 µg/m³. No threshold significantly reduced the risk for arrhythmia.
- Revise the first Discussion paragraph and the comparison paragraph so that they do not imply significant stroke reductions at 10 and 9 µg/m³ or significant heart failure reductions at 12 and 10 µg/m³. Qualify the Conclusions sentence on stroke accordingly.
- Minor: check the Figure 3 axis, whose tick values appear to be proportions although the label says per mille.
Recomputation script (python recompute.py.txt)
"""Recompute the risk-difference confidence intervals of PMC12569718
(JAMA Netw Open 2025, doi:10.1001/jamanetworkopen.2025.39862).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (Abstract, Results text, Table 2)
and from Supplement 1, eTable 2. Units: per mille (per 1000 participants), 5-year risk.
Method. Table 2 reports, for each outcome, the 5-year risk under no intervention
(the reference) and under each threshold intervention, each with a 95% CI, plus the
risk difference (RD) with its own 95% CI. Two checks follow from Table 2 alone:
(a) RD point estimate = risk(intervention) - risk(no intervention);
(b) RD 95% CI ~= risk(intervention) CI - risk(no intervention) point estimate.
(b) is an approximation: it ignores the uncertainty in the reference risk and its
correlation with the intervention risk. It works here because the reference risk is
estimated far more precisely than the intervention risks. Its accuracy is measured
below against the 24 Table 2 RD bounds, and that measured accuracy is the yardstick
for the discrepancies between the text and the table.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.transforms import blended_transform_factory # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
OUTCOMES = ("stroke", "MI", "HF", "arrhythmia")
NAMES = {"stroke": "Stroke", "MI": "Myocardial infarction", "HF": "Heart failure",
"arrhythmia": "Arrhythmia"}
THRESHOLDS = (12, 10, 9)
# Table 2, "No intervention" rows: 5-y risk (95% CI), per mille
REF = {"stroke": (12.32, 11.92, 12.72), "MI": (11.47, 11.08, 11.84),
"HF": (6.50, 6.23, 6.76), "arrhythmia": (18.86, 18.36, 19.35)}
# Table 2, threshold rows: (5-y risk (95% CI)), (risk difference (95% CI)), per mille
T2 = {
"stroke": {12: ((10.78, 10.07, 11.67), (-1.54, -2.21, -0.73)),
10: ((6.98, 3.76, 19.60), (-5.34, -8.59, 7.24)),
9: ((10.06, 3.37, 32.97), (-2.26, -8.97, 20.64))},
"MI": {12: ((10.07, 9.44, 10.85), (-1.41, -1.98, -0.67)),
10: ((4.15, 3.47, 5.54), (-7.32, -8.05, -5.86)),
9: ((2.83, 2.35, 5.03), (-8.64, -9.16, -6.38))},
"HF": {12: ((5.99, 5.37, 6.91), (-0.51, -1.07, 0.35)),
10: ((4.35, 2.95, 6.89), (-2.15, -3.55, 0.35)),
9: ((3.29, 2.40, 5.20), (-3.20, -4.16, -1.25))},
"arrhythmia": {12: ((16.80, 14.07, 21.79), (-2.06, -4.79, 3.12)),
10: ((9.81, 6.28, 19.92), (-9.05, -12.59, 1.28)),
9: ((14.70, 6.14, 31.74), (-4.16, -12.70, 12.93))},
}
# Abstract, Results section: RD (95% CI) as printed
ABSTRACT = {("stroke", 9): (-2.26, -8.97, -20.64), ("MI", 9): (-8.64, -9.16, -6.38),
("HF", 9): (-3.20, -4.16, -1.25), ("arrhythmia", 9): (-4.16, -12.70, 12.93),
("stroke", 12): (-1.54, -2.21, 0.73), ("arrhythmia", 12): (-2.06, -4.79, 3.12)}
# Results text, paragraph describing Table 2: RD (95% CI) as printed
RESULTS = {("stroke", 12): (-1.54, -2.21, -0.73), ("MI", 12): (-1.41, -1.98, -0.67),
("HF", 12): (-0.51, -1.07, 0.35), ("arrhythmia", 12): (-2.06, -4.79, 3.12),
("stroke", 10): (-5.34, -8.59, 7.24), ("MI", 10): (-7.32, -8.05, -5.86),
("HF", 10): (-2.15, -3.55, -0.35), ("arrhythmia", 10): (-9.05, -12.59, 1.28),
("stroke", 9): (-2.26, -8.97, -20.64), ("MI", 9): (-8.64, -9.16, -6.38),
("HF", 9): (-3.20, -4.16, -1.25), ("arrhythmia", 9): (-4.16, -12.70, 12.93)}
# Supplement 1, eTable 2, stroke at <=9 ug/m3: (5-y risk (95% CI)), (RD (95% CI))
# The no-intervention risk is 12.32 (11.92, 12.72) in every row.
ETABLE2_STROKE = {
"Main analysis": ((10.06, 3.37, 32.97), (-2.26, -8.97, 20.64)),
"Further adjustment for NO2 and NOx (2010)": ((8.93, 3.15, 30.71), (-3.39, -9.25, 18.47)),
"Further adjustment for family history of CVD": ((10.06, 3.05, 32.86), (-2.26, -9.30, 20.49)),
"Further adjustment for BMI": ((8.30, 3.12, 27.37), (-4.02, -9.26, 14.91)),
}
# Key Points, Findings: thresholds 12 and 9 "would significantly reduce" these outcomes
KEY_POINTS_CLAIM = [(o, th) for o in ("stroke", "HF", "MI") for th in (12, 9)]
def approx_ci(o, risk):
"""RD 95% CI approximated as the intervention-risk CI minus the reference point."""
r0 = REF[o][0]
return risk[1] - r0, risk[2] - r0
def sig(lo, hi):
return "significant" if (hi < 0 or lo > 0) else "NOT significant (CI includes 0)"
def f(x):
return f"{x:+.2f}".replace("+-", "-")
# ---------------------------------------------------------------- 1. Table 2 internal check
print("1) Table 2 internal check")
print(" RD = risk(int) - risk(no int); RD CI ~= risk(int) CI - risk(no int)")
APPROX = {}
max_pt, max_bd = 0.0, 0.0
for o in OUTCOMES:
for th in THRESHOLDS:
risk, (d, dl, dh) = T2[o][th]
pt = risk[0] - REF[o][0]
al, ah = approx_ci(o, risk)
APPROX[(o, th)] = (pt, al, ah)
max_pt = max(max_pt, abs(pt - d))
max_bd = max(max_bd, abs(al - dl), abs(ah - dh))
print(f" {o:10s} <={th:2d}: RD reported {d:6.2f}, recomputed {pt:6.2f} | "
f"CI reported ({dl:6.2f}, {dh:6.2f}), recomputed ({al:6.2f}, {ah:6.2f}) | "
f"Table 2: {sig(dl, dh)}")
print(f" Largest |reported - recomputed| RD point estimate over 12 cells: {max_pt:.3f}")
print(f" Largest |reported - recomputed| RD CI bound over 24 bounds: {max_bd:.3f}")
# ---------------------------------------------------------------- 2. text vs Table 2
def check_text(label, d):
print(f"\n2) {label} vs Table 2")
out = []
for (o, th), (e, lo, hi) in d.items():
te, tl, thi = T2[o][th][1]
issues = []
if lo > hi:
issues.append("lower bound > upper bound")
if not (min(lo, hi) <= e <= max(lo, hi)):
issues.append("estimate outside its own CI")
if (e, lo, hi) != (te, tl, thi):
diffs = [n for n, a, b in (("estimate", e, te), ("lower", lo, tl), ("upper", hi, thi))
if a != b]
issues.append(f"differs from Table 2 ({te}, {tl} to {thi}) in: {', '.join(diffs)}")
_, al, ah = APPROX[(o, th)]
print(f" {o:10s} <={th:2d}: printed {e} ({lo} to {hi}) | Table 2 {te} ({tl} to {thi}) | "
f"recomputed ({al:.2f} to {ah:.2f}) | {'; '.join(issues) or 'OK'}")
if issues:
out.append((o, th))
print(f" cells with a problem: {out}")
return out
check_text("Abstract (Results)", ABSTRACT)
check_text("Results text", RESULTS)
print("\n Size of each text-vs-table discrepancy, against the approximation's accuracy:")
for label, (o, th), which, printed in (
("Abstract & Results, stroke <=9 upper", ("stroke", 9), 2, -20.64),
("Abstract, stroke <=12 upper", ("stroke", 12), 2, 0.73),
("Results, HF <=10 upper", ("HF", 10), 2, -0.35)):
tab = T2[o][th][1][which]
rec = APPROX[(o, th)][which]
print(f" {label:38s}: printed {printed:7.2f}, Table 2 {tab:6.2f}, recomputed {rec:6.2f} | "
f"|printed - recomputed| = {abs(printed - rec):5.2f} vs |Table 2 - recomputed| = "
f"{abs(tab - rec):4.2f}")
# The printed stroke <=9 interval cannot be made valid by reordering its bounds
e, lo, hi = ABSTRACT[("stroke", 9)]
print(f"\n Stroke <=9 printed interval reordered: ({min(lo, hi)} to {max(lo, hi)}); "
f"contains estimate {e}? {min(lo, hi) <= e <= max(lo, hi)}")
r = T2["stroke"][9][0]
print(f" Stroke <=9 RD upper bound using either end of the reference CI instead of 12.32: "
f"{r[2] - REF['stroke'][2]:.2f} to {r[2] - REF['stroke'][1]:.2f} (all > 0)")
# ---------------------------------------------------------------- 3. Key Points vs Table 2
print("\n3) Key Points: thresholds 12 and 9 'would significantly reduce' stroke, HF and MI")
for o, th in KEY_POINTS_CLAIM:
d, lo, hi = T2[o][th][1]
_, al, ah = APPROX[(o, th)]
print(f" {o:6s} <={th:2d}: Table 2 {d} ({lo} to {hi}) -> {sig(lo, hi)}; "
f"recomputed CI ({al:.2f} to {ah:.2f}) -> {sig(al, ah)}")
print("\n Significance grid from Table 2 (S = CI excludes 0, ns = CI includes 0):")
print(" " + " " * 22 + " ".join(f"<={th:2d}" for th in THRESHOLDS))
for o in OUTCOMES:
cells = [" S " if (T2[o][th][1][2] < 0 or T2[o][th][1][1] > 0) else " ns " for th in THRESHOLDS]
print(f" {NAMES[o]:22s}" + " ".join(cells))
# ---------------------------------------------------------------- 4. Supplement eTable 2
print("\n4) Supplement 1, eTable 2, stroke at <=9 (main analysis and robustness checks)")
for k, (risk, (d, dl, dh)) in ETABLE2_STROKE.items():
pt = risk[0] - REF["stroke"][0]
al, ah = risk[1] - REF["stroke"][0], risk[2] - REF["stroke"][0]
print(f" {k:45s}: RD {d} ({dl} to {dh}), recomputed {pt:.2f} ({al:.2f} to {ah:.2f}) "
f"-> {sig(dl, dh)}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f" wrote {stem}.svg and {stem}.png")
def interval(ax, y, est, lo, hi, color, marker="o", filled=True):
a, b = min(lo, hi), max(lo, hi)
ax.plot([a, b], [y, y], color=color, lw=2, solid_capstyle="butt", zorder=3)
for x in (a, b):
ax.plot([x, x], [y - 0.12, y + 0.12], color=color, lw=2, zorder=3)
ax.plot([est], [y], marker=marker, ms=9, color=color, mfc=color if filled else BG,
mew=2, zorder=4, ls="none")
def style(ax, xlo, xhi, xlabel):
ax.set_xlim(xlo, xhi)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel(xlabel, fontsize=10.5, color=TXT)
print("\n5) Figures")
def value_col(ax, y, ee, ll, hh):
ax.text(1.03, y, f"{f(ee)} ({f(ll)} to {f(hh)})".replace("-", "\u2212"),
transform=ax.get_yaxis_transform(), va="center", ha="left", fontsize=10, color=TXT)
LEG_REPORTED_BAD = Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, mfc=BG, mew=2,
label="Reported in text, differs from Table 2")
LEG_REPORTED = Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="Reported in table")
LEG_RECOMP = Line2D([], [], color=BLUE, lw=2, marker="o", ms=8,
label="Recomputed from Table 2 risks")
# ---- fig1: stroke at <=9 ug/m3
fig, ax = plt.subplots(figsize=(7.2, 3.9))
e, lo, hi = ABSTRACT[("stroke", 9)]
t_e, t_lo, t_hi = T2["stroke"][9][1]
s_e, s_lo, s_hi = ETABLE2_STROKE["Main analysis"][1]
r_e, r_lo, r_hi = APPROX[("stroke", 9)]
rows = [
("Abstract and Results text", e, lo, hi, ORANGE, False),
("Table 2", t_e, t_lo, t_hi, ORANGE, True),
("Supplement 1, eTable 2", s_e, s_lo, s_hi, ORANGE, True),
("Recomputed: Table 2 risk CI\nminus no-intervention risk", r_e, r_lo, r_hi, BLUE, True),
]
ys = [3, 2, 1, 0]
for (lab, ee, ll, hh, c, filled), y in zip(rows, ys):
interval(ax, y, ee, ll, hh, c, filled=filled)
value_col(ax, y, ee, ll, hh)
ax.text(1.03, 3.6, "RD (95% CI)", transform=ax.get_yaxis_transform(),
va="center", ha="left", fontsize=10, color=SEC)
ax.annotate("estimate lies outside its printed interval", xy=(e - 0.3, 3.12),
xytext=(-23.5, 3.62), fontsize=10, color=SEC, ha="left", va="center", zorder=5,
bbox=dict(fc=BG, ec="none", pad=1.5),
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.6, 3.95)
style(ax, -24, 24, "5-year risk difference for stroke, \u22649 \u00b5g/m\u00b3\nvs no intervention (per mille)")
ax.text(0.5, -0.55, "no difference", fontsize=10, color=SEC, va="bottom")
fig.legend(handles=[LEG_REPORTED_BAD, LEG_REPORTED, LEG_RECOMP], loc="upper center",
bbox_to_anchor=(0.5, 0.995), frameon=False, fontsize=10, ncol=2,
handlelength=1.6, columnspacing=1.2)
fig.subplots_adjust(left=0.29, right=0.71, top=0.80, bottom=0.18)
save(fig, "fig1")
# ---- fig2: the two smaller sign discrepancies and the HF <=12 cell
fig, ax = plt.subplots(figsize=(7.2, 4.9))
groups = [
("Stroke, \u226412 \u00b5g/m\u00b3", [("Abstract", ABSTRACT[("stroke", 12)], ORANGE, False),
("Results text, Table 2", T2["stroke"][12][1], ORANGE, True),
("Recomputed", APPROX[("stroke", 12)], BLUE, True)]),
("Heart failure, \u226410 \u00b5g/m\u00b3", [("Results text", RESULTS[("HF", 10)], ORANGE, False),
("Table 2", T2["HF"][10][1], ORANGE, True),
("Recomputed", APPROX[("HF", 10)], BLUE, True)]),
("Heart failure, \u226412 \u00b5g/m\u00b3", [("Results text, Table 2", T2["HF"][12][1], ORANGE, True),
("Recomputed", APPROX[("HF", 12)], BLUE, True)]),
]
y = 0
yt, yl, heads = [], [], []
for gname, items in groups:
heads.append((gname, y))
y -= 0.75
for lab, (ee, ll, hh), c, filled in items:
interval(ax, y, ee, ll, hh, c, filled=filled)
value_col(ax, y, ee, ll, hh)
yt.append(y)
yl.append(lab)
y -= 0.75
y -= 0.35
for gname, yy in heads:
ax.text(0.015, yy, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
ha="left", va="center",
fontsize=10.5, fontweight="bold", color=TXT)
ax.text(1.03, 0, "RD (95% CI)", transform=ax.get_yaxis_transform(), va="center", ha="left",
fontsize=10, color=SEC)
ax.set_yticks(yt)
ax.set_yticklabels(yl, fontsize=10)
ax.set_ylim(y + 0.6, 0.45)
style(ax, -4, 1, "5-year risk difference vs no intervention (per mille)")
fig.legend(handles=[LEG_REPORTED_BAD, LEG_REPORTED, LEG_RECOMP], loc="upper center",
bbox_to_anchor=(0.5, 0.995), frameon=False, fontsize=10, ncol=2,
handlelength=1.6, columnspacing=1.2)
fig.subplots_adjust(left=0.29, right=0.71, top=0.86, bottom=0.11)
save(fig, "fig2")
Registry share of Asian or Pacific Islander patients is 2.98%, not 5.9%, reversing reported underrepresentation
JAMA Network Open (2022) · doi:10.1001/jamanetworkopen.2022.39884 · PMC9634497
Summary
The study compares the racial and ethnic makeup of 8309 participants in US phase 1 metastatic-cancer drug trials with incident cases in the Cancer in North America (CiNA) registry. For the whole 2000-2018 period, the Results report that Asian or Pacific Islander (API) participants were underrepresented: 4.5% of trial participants against 5.9% of registry cases, a difference of −1.5 percentage points (P < .001). The registry count and denominator printed in the same sentence, 145 568 of 4 891 486, give 2.98%, not 5.9%. That count is the exact sum of the paper's two period counts, whose shares (2.7% and 3.4%) are printed correctly. Measured against 2.98%, trial participation is higher than the registry share under either trial count the paper gives: +1.49 points (P ≈ 2 × 10⁻¹⁵) with the Table's 371 of 8309, or +0.30 points (P = .12, not significant) with the sentence's 272 of 8308. The reported overall API underrepresentation, which the Conclusions repeat, therefore does not follow from the paper's own data. Separately, the API trial counts in the Table do not add up (121 + 151 = 272, but the "All" row is 371), and this bears on the API time trend.
What the paper reports
Results, third paragraph, the overall (2000-2018) comparison. The PMC-archived text has a hyphen before 1.5; the publisher's HTML and the Europe PMC XML have a true minus sign (U+2212):
an underrepresentation of Asian or Pacific Islander participants (trials, 272 of 8308 [4.5%]; CiNA, 145 568 of 4 891 486 [5.9%]; difference, -1.5 percentage points; P < .001)
The same paragraph, period-specific comparison (repeated in the Abstract):
Asian or Pacific Islander patients (2000-2011: trials, 121 of 3245 [3.7%]; CiNA, 75 033 of 2 800 711 [2.7%]; difference, 1.1 percentage points; 2012-2018: trials, 151 of 5063 [3.0%]; CiNA 70 535 of 2 090 775 [3.4%]; difference, -0.75 percentage points)
Results, second paragraph, the overall trial count. The Abstract gives "371 Asian or Pacific Islander participants (4.5%)":
Of the 8309 total participants, 23 (0.3%) were American Indian or Alaska Native, 371 (4.5%) were Asian or Pacific Islander
The Table, "Phase 1 trials", Asian or Pacific Islander column:
All (N = 8309) | 371 (4.5) 2000-2011 (n = 3245) | 121 (3.7) 2012-2018 (n = 5063) | 151 (3.0)
The Table, "Industry trials", same column:
2012-2018 (n = 4628) | 226 (4.9)
Methods, "Statistical Analysis", the test, the pooling and the sign convention:
Proportions accounting for the sample size of each trial were calculated for each racial and ethnic group by combining the total number of participants for each racial and ethnic group and dividing by the total number of trial participants
differences with 95% CIs were estimated using a 2-sample test for equality of proportions with continuity correction
with positive percentage differences indicating overrepresentation and negative percentage differences indicating underrepresentation
Results, third paragraph, the ratios shown in Figure 3:
especially Asian and Pacific Islander patients (ratio of trial to CiNA participants, 1.44 in 2000-2011 to 0.87 in 2012-2018)
Key Points, "Findings":
found worsening underrepresentation of American Indian or Alaska Native, Asian or Pacific Islander, Black, and Hispanic or Latinx participants over time
Conclusions:
These include underenrollment of Asian or Pacific Islander patients, Black patients, and Hispanic or Latinx patients in clinical trials overall and underenrollment of American Indian or Alaska Native patients, Asian or Pacific Islander patients, Black patients, and Hispanic or Latinx patients in trials conducted at academic institutions
What the paper's own numbers show
Method. I took every count printed in the Abstract, Results, Table and Supplement eTable, recomputed each share, and checked whether overall counts equal the sums of the period counts. I then reran each comparison with the test the Methods name: a two-sample test for equality of proportions with continuity correction (R's prop.test, whose statistic is the Yates-corrected χ² on the 2 × 2 table), with its continuity-corrected 95% CI. As a check that this is the paper's test, it reproduces the reported White 2000-2011 result (+0.76 points, P = .235; reported 0.8, P = .23) and the overall White (+2.01; reported 2.0) and Black (−5.64; reported −5.6) differences. It also matches every significance marker in the paper's Figure 2: P < .001 for API 2000-2011, P = .048 for American Indian or Alaska Native 2012-2018 (marked P < .05), and no marker for API 2012-2018 (P = .13). Every number below is printed by recompute.py.txt.
Additivity. The overall CiNA counts equal the sums of the period counts, for API (75 033 + 70 535 = 145 568), for the denominator (2 800 711 + 2 090 775 = 4 891 486), and for White and Black. The overall API registry share is therefore the pooled share of the two periods, so it must lie between 2.68% and 3.37%. It cannot be 5.9%.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CiNA API share, 2000-2011 | Abstract; Results ¶3 | 75 033 / 2 800 711 [2.7%] | 2.68% (consistent) |
| CiNA API share, 2012-2018 | Abstract; Results ¶3 | 70 535 / 2 090 775 [3.4%] | 3.37% (consistent) |
| CiNA API count, 2000-2018 | Results ¶3 | 145 568 | 75 033 + 70 535 = 145 568 (exact) |
| CiNA API share, 2000-2018 | Results ¶3 | 145 568 / 4 891 486 [5.9%] | 2.98%. 5.9% of 4 891 486 would be 288 598 cases (1.98 × the printed count) |
| CiNA groups summed, with API at 5.9% | derived | — | White + Black + AIAN + API = 5 000 409, which exceeds the 4 891 486 total by 108 923 |
| Trial API share, 2000-2018 (sentence) | Results ¶3 | 272 / 8308 [4.5%] | 3.27%. The bracketed 4.5% belongs to 371/8309 |
| Trial API share, 2000-2018 (Table) | Table "All"; Abstract; Results ¶2 | 371 / 8309 (4.5) | 4.47% (consistent) |
| Overall API difference, trials − CiNA, using 371/8309 | Results ¶3 | −1.5 pp, P < .001 | +1.49 pp (95% CI +1.04 to +1.94), P = 1.9 × 10⁻¹⁵ |
| Overall API difference, using 272/8308 | Results ¶3 | −1.5 pp, P < .001 | +0.30 pp (95% CI −0.09 to +0.69), P = .12 |
| Overall API difference from the printed shares | Results ¶3 | −1.5 pp | 4.5 − 5.9 = −1.4; 4.47 − 5.9 = −1.43. Neither gives −1.5 |
| Overall API trial/CiNA ratio | not reported | (underrepresentation, ratio < 1) | 1.50 (371/8309) or 1.10 (272/8308) |
| API difference, 2000-2011 | Abstract; Results ¶3 | +1.1 pp | +1.05 pp (95% CI +0.38 to +1.72), P = .0003 (rounding-level) |
| API difference, 2012-2018 | Abstract; Results ¶3 | −0.75 pp, no P value | −0.39 pp (95% CI −0.87 to +0.09), P = .13; printed shares give 3.0 − 3.4 = −0.4 |
| API trial/CiNA ratio, 2000-2011 → 2012-2018 | Results ¶3; Figure 3 | 1.44 → 0.87 | 1.39 → 0.88 |
| White overall difference (method check) | Abstract; Results ¶3 | +2.0 pp, P < .001 | +2.01 pp, P = 5.8 × 10⁻⁷ (consistent) |
| Black overall difference (method check) | Results ¶3 | −5.6 pp, P < .001 | −5.64 pp, P = 7.9 × 10⁻⁵⁷ (consistent) |
Against the registry share its own counts imply, the overall API difference has the reported size (1.49 points with the Table's count) but the opposite sign.
A second, separate inconsistency: the API trial counts. In the Table every other column adds up across the period rows (White 2780 + 4374 = 7154; Black 244 + 270 = 514; and so on). The API column does not.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| API, 2000-2011 + 2012-2018 vs "All" | Table, phase 1 trials | 121 + 151; All = 371 | 121 + 151 = 272, which is the count used in the Results sentence; 99 short of 371 |
| API, industry-funded subset vs all trials, 2012-2018 | Table | industry 226 (4.9); all trials 151 (3.0) | the subset exceeds the total by 75. The industry rows themselves add up (106 + 226 = 332) |
| "All" row minus the 2000-2011 row | Table | — | 371 − 121 = 250 (4.94% of 5063), which is ≥ the industry count of 226 |
| Supplement eTable (EGFR-positive trials added back) minus Table, overall vs by period | eTable; Table | — | additive for White (406 = 406), Black (17 = 17), AIAN (1 = 1) and Hispanic (4 = 4). For API, 870 − 371 = 499 overall but 598 by period. A 2012-2018 Table count of 250 would make API additive too |
| API difference, 2012-2018, if the count were 250 | derived | — | +1.56 pp (95% CI +0.96 to +2.17), P = 9.7 × 10⁻¹⁰; ratio 1.46 |
So the printed 151, the printed 371, or both cannot be right. The paper's other numbers (the industry subset and the eTable) are consistent with 371 overall and 250 in 2012-2018, but the paper alone does not establish which figure is wrong.
Smaller points, none of which changes a conclusion. The CiNA White share for 2012-2018, 1 735 077 / 2 090 775 = 82.99%, is printed as 82.9% (difference recomputes to +3.40 against the printed 3.5). The Table footnote gives 5076 participants in trials reporting ethnicity, but its period denominators sum to 1792 + 3295 = 5087. The overall Black comparison uses 8308 as the denominator against 8309 elsewhere. The text ratios for Black (0.64 → 0.42) and Hispanic or Latinx (1.55 → 0.98) recompute to 0.65 → 0.44 and 1.49 → 0.98. In the paper's Figure 3 the plotted API point estimates sit near 1.39 and 0.88, and the text values 1.44 and 0.87 lie close to the ends of the plotted intervals (read from the image, so approximate).
Figure
Figure 1. Asian or Pacific Islander shares on one percentage axis starting at zero. Orange rings are the values printed in the paper. Blue dots are the values recomputed from the numerator and denominator printed with them (listed under each row label). For each period the two agree (2.7 vs 2.68; 3.4 vs 3.37). For 2000-2018 the paper prints 5.9%, but its own count, 145 568 of 4 891 486, gives 2.98%. Because that count and denominator are the sums of the period counts, the pooled share has to fall in the shaded band between the two period shares. Both trial figures the paper gives for 2000-2018, 3.27% (272/8308, Results sentence, printed there as 4.5%) and 4.47% (371/8309, Table and Abstract), lie at or above 2.98%, not below it.
Figure 2. Trial share minus CiNA share, in percentage points, with 95% CIs from the test the paper names (two-sample test for equality of proportions with continuity correction). The thin vertical line marks no difference; by the paper's convention, negative values mean underrepresentation. Orange rings are printed values (the paper gives no CIs for these comparisons). For 2000-2018 the paper prints −1.5 (P < .001). Recomputed against the registry share its counts imply, the difference is +1.49 with the Table's count and +0.30 (P = .12) with the sentence's count. For 2000-2011 the printed and recomputed values agree. For 2012-2018 the printed −0.75 recomputes to −0.39 (P = .13). The hollow blue marker shows the 2012-2018 difference if the all-trial count were 371 − 121 = 250, the value the Table's "All" row and industry subset imply; it is a derived scenario, not a value printed in the paper.
Why it matters
- A reported finding is reversed. The Results state a significant overall underrepresentation of API participants over 2000-2018 (−1.5 points, P < .001). The paper's own counts show overrepresentation (+1.49 points, ratio 1.50, with the Table's 371) or no detectable difference (+0.30 points, P = .12, ratio 1.10, with the sentence's 272). Neither supports underrepresentation.
- The Conclusions repeat it. They list "underenrollment of Asian or Pacific Islander patients ... in clinical trials overall". The only overall API comparison in the paper is the erroneous one. If "overall" is instead read as all trials in the more recent period, the 2012-2018 difference recomputes to −0.39 points (P = .13), and the paper's own Figure 2 marks no significant difference there.
- The API part of the "worsening" claim is weaker than presented. The Key Points report worsening underrepresentation of API participants over time. With the printed period counts, the API difference moves from significant overrepresentation (+1.05 points) to a non-significant −0.39 points: a decline, but to parity rather than to underrepresentation. If the Table's "All" row is right and the 2012-2018 count is 250, API participants were overrepresented in both periods (+1.05 and +1.56 points) and there is no worsening for this group.
- Related statements in the Conclusions are also not supported by the paper's counts. These are derived comparisons the paper does not print, offered for the editor's attention rather than as the main finding. The Conclusions list API underenrollment at academic centers, but the Table's academic counts are above the CiNA share in both periods: 68/1806 (3.77%) vs 2.68%, +1.09 points, P = .005; 153/4145 (3.69%) vs 3.37%, +0.32 points, P = .28. The Conclusions also list Hispanic or Latinx underenrollment overall and at academic centers. Pooled from the paper's counts, the overall comparison is 401/5076 (7.90%) vs 325 415 / 4 891 486 (6.65%), +1.25 points (P = .0004). At academic centers it is 276/3523 (7.83%), +1.18 points (P = .005), and it is positive in both periods.
- What is not affected. The CiNA period shares and the comparisons for White and Black patients reproduce. The paper's central finding of strong and increasing underrepresentation of Black patients (−4.00 to −6.89 points) does not depend on these errors.
Explanations considered
- Text-conversion artifact (lost minus sign, decimal or digit). This does not account for the numbers. The publisher's HTML, the Europe PMC XML and the PMC archive carry the same sentence, the first two with a true minus sign. No dropped sign or decimal turns 2.98 into 5.9, and the sentence is running prose, not a table cell.
- A different registry population or denominator for the overall comparison. This does not account for the numbers. The sentence prints its own numerator and denominator (145 568 of 4 891 486), and that ratio is 2.98%. Both numbers are the exact sums of the period counts.
- A weighted rather than pooled share. This does not account for the numbers. The Methods describe pooled proportions. Any weighted average of the two period shares (2.68% and 3.37%) also lies between them.
- 5.9% is a typographical slip, and the difference and P value are right. This does not account for the numbers. Against 2.98%, the difference is positive with either trial count, so −1.5 and the stated underrepresentation cannot stand with a corrected share.
- The trial count, not the registry share, is the problem. This is only part of the picture. The sentence's trial figures are themselves inconsistent (272/8308 is 3.27%, yet it is printed as 4.5%), but neither trial count gives a negative difference against 2.98%. Only the registry share produces the reversed sign.
- The overall comparison comes from the sensitivity cohort that adds back the EGFR-positive trials. This does not account for the numbers. That cohort changes only the trial side; the registry is the same. The supplement's eTable gives trial counts only (API 870/9273 = 9.4% overall), and its eFigure lists CiNA API shares of 2.68% and 3.39% for the two periods, with no overall value and no 5.9%. Its higher trial share would make the difference even more positive.
None of these explains the printed 5.9%. The material published with the paper does not show how it arose.
Status
- Version of record. Checked. The DOI resolves to the publisher's full-text page (jamanetwork.com, article 2798149), which carries the sentence as quoted, with a true minus sign, and shows no correction notice. Its reader-comment section displays no comments. The Europe PMC full-text XML (publisher-supplied JATS) has the identical sentence. All quotes above were confirmed verbatim against the PMC-archived text.
- Supplement. Checked: jamanetwopen-e2239884-s001.pdf (eTable and eFigure, the sensitivity analysis including EGFR-positive trials), from the Europe PMC supplementary-files archive. The eTable gives trial counts only; the eFigure lists CiNA API shares of 2.68% (2000-2011) and 3.39% (2012-2018), consistent with the main text and never 5.9%. It does not resolve the discrepancy. No source data or code are deposited. The same archive holds the article's figures. Figure 2 shows CiNA API bars near 2.7% and 3.4% for the two periods and has no overall bar; Figure 3 is described above.
- Corrections and notices. None found as of 2026-10-23. Crossref lists no update-to or updated-by relations. There are no PMC or Europe PMC correction, retraction or expression-of-concern notices, and the Europe PMC record (PMID 36326764) has no comment or correction links.
- PubPeer. The PubPeer search page returned HTTP 403 and its API returned a human-verification page, so PubPeer could not be checked directly in this session. A web search could not be run in this session either. An earlier screening step in this review found no correction as of 2026-10-22.
Sources: doi.org/10.1001/jamanetworkopen.2022.39884; publisher full text; PMC9634497; Europe PMC full-text XML; Europe PMC supplementary files; Crossref record.
Suggested correction
- Results, overall API comparison. Recompute the registry share as 145 568 / 4 891 486 = 3.0% and settle the trial count (item 2). The sentence would then report an overrepresentation, "trials, 371 of 8309 [4.5%]; CiNA, 145 568 of 4 891 486 [3.0%]; difference, 1.5 percentage points; P < .001", or, if 272 of 8308 is correct, "trials, 272 of 8308 [3.3%]; CiNA, 145 568 of 4 891 486 [3.0%]; difference, 0.3 percentage points; P = .12", with no significant difference.
- Table, API column. Reconcile the "All" row (371) with the period rows (121 + 151 = 272), and the all-trial 2012-2018 count (151) with the industry-funded 2012-2018 count (226). Then update the 2012-2018 API comparison in the Abstract and Results (currently −0.75 points, which recomputes to −0.39 with 151), Figure 2, Figure 3 and the ratios in the text.
- Conclusions and Key Points. Revise the statement of API underenrollment "in clinical trials overall", and the API part of "worsening underrepresentation ... over time", in line with items 1 and 2. Check the statements on API and Hispanic or Latinx underenrollment overall and at academic centers against the Table's counts, which are above the CiNA shares.
- Minor. Change 82.9% to 83.0% for 1 735 077 / 2 090 775, reconcile the Hispanic or Latinx denominator (5076 vs 1792 + 3295 = 5087), use 8309 consistently, and state whether the ratios quoted in the text are point estimates.
Recomputation script (python recompute.py.txt)
"""Recompute the Asian or Pacific Islander (API) registry comparison of PMC9634497
(JAMA Netw Open 2022;5(11):e2239884, doi:10.1001/jamanetworkopen.2022.39884).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Results, paragraphs 2-3 (trial and CiNA counts, by period and overall)
Table (trial counts: all trials, industry-funded, academic centers)
Supplement eTable (the same Table with the EGFR+ NSCLC trials added back)
Test: the paper states "a 2-sample test for equality of proportions with continuity
correction" (R prop.test). prop.test's statistic equals the Yates-corrected chi-square on
the 2x2 table, and its CI is the Wald interval widened by the continuity correction.
"""
import math
import os
import numpy as np
from scipy.stats import chi2_contingency, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
# CiNA registry counts, Results paragraph 3 (period) and paragraph 2 (overall)
CINA_N = {"2000-2011": 2800711, "2012-2018": 2090775, "overall": 4891486}
CINA = { # group: (2000-2011, 2012-2018, overall as printed or None if not printed)
"White": (2378019, 1735077, 4113096),
"Black": (322701, 255625, 578326),
"API": (75033, 70535, 145568),
"AIAN": (10905, 9484, None),
"Hispanic": (169297, 156118, None),
}
CINA_API_PCT_PRINTED_OVERALL = 5.9
# Trial counts, Table ("All" rows) and Results text
TRIAL_N = {"2000-2011": 3245, "2012-2018": 5063, "overall": 8309}
TRIAL = { # group: (2000-2011, 2012-2018, Table "All")
"White": (2780, 4374, 7154),
"Black": (244, 270, 514),
"API": (121, 151, 371),
"AIAN": (10, 13, 23),
"Hispanic": (161, 240, 401),
}
HISP_TRIAL_N = {"2000-2011": 1792, "2012-2018": 3295, "overall": 5076} # Table footnote
API_TEXT = (272, 8308) # Results paragraph 2, overall API comparison
BLACK_TEXT = (514, 8308) # Results paragraph 2, overall Black comparison
# Table subsets (API column): (2000-2011, 2012-2018, All), and subset sizes
INDUSTRY_API = (106, 226, 332)
INDUSTRY_N = (2737, 4628, 7365)
ACADEMIC_API = (68, 153, 221)
ACADEMIC_N = (1806, 4145, 5951)
ACADEMIC_HISP = (62, 214, 276)
ACADEMIC_HISP_N = (841, 2682, None)
ACADEMIC_AIAN_ALL = 16
# Supplement eTable, "All Phase I" rows: (2001-2011, 2012-2018, All)
ETABLE = {
"White": (2906, 4654, 7560),
"Black": (253, 278, 531),
"API": (126, 744, 870),
"AIAN": (10, 14, 24),
"Hispanic": (164, 241, 405),
}
# Reported values (text) for the comparisons
REPORTED = {
"API overall": dict(trial=4.5, cina=5.9, diff=-1.5, p="< .001"),
"API 2000-2011": dict(trial=3.7, cina=2.7, diff=1.1, p="not given"),
"API 2012-2018": dict(trial=3.0, cina=3.4, diff=-0.75, p="not given"),
"White overall": dict(trial=86.1, cina=84.1, diff=2.0, p="< .001"),
"Black overall": dict(trial=6.2, cina=11.8, diff=-5.6, p="< .001"),
"White 2000-2011": dict(trial=85.7, cina=84.9, diff=0.8, p=".23"),
"White 2012-2018": dict(trial=86.4, cina=82.9, diff=3.5, p="< .001"),
"Black 2000-2011": dict(trial=7.5, cina=11.5, diff=-4.0, p="not given"),
"Black 2012-2018": dict(trial=5.3, cina=12.2, diff=-6.9, p="not given"),
"AIAN 2000-2011": dict(trial=0.3, cina=0.4, diff=-0.08, p="not given"),
"AIAN 2012-2018": dict(trial=0.3, cina=0.5, diff=-0.20, p="not given"),
"Hispanic 2000-2011": dict(trial=9.0, cina=6.0, diff=3.0, p="not given"),
"Hispanic 2012-2018": dict(trial=7.3, cina=7.5, diff=-0.2, p="not given"),
}
REPORTED_RATIOS = { # Results paragraph 3: (2000-2011, 2012-2018)
"API": (1.44, 0.87), "Black": (0.64, 0.42), "Hispanic": (1.55, 0.98)}
# ---------------------------------------------------------------- helpers
def pct(a, b):
return 100.0 * a / b
def prop_test(x1, n1, x2, n2, conf=0.95):
"""R prop.test(c(x1,x2), c(n1,n2)) with continuity correction.
Returns (p1 %, p2 %, diff pp, ci_lo pp, ci_hi pp, P)."""
p1, p2 = x1 / n1, x2 / n2
delta = p1 - p2
yates = min(0.5, abs(delta) / (1 / n1 + 1 / n2))
z = norm.ppf(0.5 + conf / 2)
width = z * math.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2) + yates * (1 / n1 + 1 / n2)
table = np.array([[x1, n1 - x1], [x2, n2 - x2]])
_, p, _, _ = chi2_contingency(table, correction=True)
return 100 * p1, 100 * p2, 100 * delta, 100 * (delta - width), 100 * (delta + width), p
def fmt_p(p):
return f"{p:.3f}" if p >= 0.001 else f"{p:.1e}"
def line(label, res, reported=None):
p1, p2, d, lo, hi, p = res
s = (f" {label:46s} trial {p1:6.2f}% CiNA {p2:6.2f}% diff {d:+6.2f} pp "
f"(95% CI {lo:+.2f} to {hi:+.2f}) P = {fmt_p(p)}")
if reported:
s += (f"\n {'':46s} reported: {reported['trial']}% vs {reported['cina']}%, "
f"diff {reported['diff']:+} pp, P {reported['p']}")
print(s)
results = {}
# ---------------------------------------------------------------- 1. sums and the CiNA share
print("1. Do the overall counts equal the sum of the period counts?")
print(f" CiNA N: {CINA_N['2000-2011']} + {CINA_N['2012-2018']} = "
f"{CINA_N['2000-2011'] + CINA_N['2012-2018']} (printed overall {CINA_N['overall']})")
for g, (a, b, o) in CINA.items():
print(f" CiNA {g:8s}: {a} + {b} = {a + b}" + (f" (printed overall {o})" if o else " (overall not printed)"))
print(f" Trial N: {TRIAL_N['2000-2011']} + {TRIAL_N['2012-2018']} = "
f"{TRIAL_N['2000-2011'] + TRIAL_N['2012-2018']} (Table 'All' N = {TRIAL_N['overall']})")
for g, (a, b, o) in TRIAL.items():
flag = "" if a + b == o else " <-- does not add up"
print(f" Trial {g:8s}: {a} + {b} = {a + b} (Table 'All' = {o}){flag}")
print(f" Hispanic trial N (footnote): {HISP_TRIAL_N['2000-2011']} + {HISP_TRIAL_N['2012-2018']} = "
f"{HISP_TRIAL_N['2000-2011'] + HISP_TRIAL_N['2012-2018']} (footnote overall {HISP_TRIAL_N['overall']})")
print("\n2. CiNA share of Asian or Pacific Islander patients")
api_c = CINA["API"]
cina_api_p00 = pct(api_c[0], CINA_N["2000-2011"])
cina_api_p12 = pct(api_c[1], CINA_N["2012-2018"])
cina_api_all = pct(api_c[2], CINA_N["overall"])
print(f" 2000-2011: {api_c[0]}/{CINA_N['2000-2011']} = {cina_api_p00:.2f}% (printed 2.7%)")
print(f" 2012-2018: {api_c[1]}/{CINA_N['2012-2018']} = {cina_api_p12:.2f}% (printed 3.4%)")
print(f" 2000-2018: {api_c[2]}/{CINA_N['overall']} = {cina_api_all:.2f}% (printed {CINA_API_PCT_PRINTED_OVERALL}%)")
print(f" A pooled share must lie between the period shares: {cina_api_p00:.2f}% to {cina_api_p12:.2f}%")
implied = round(CINA_API_PCT_PRINTED_OVERALL / 100 * CINA_N["overall"])
print(f" Count implied by 5.9% of {CINA_N['overall']}: {implied} "
f"({implied / api_c[2]:.2f} x the printed 145 568)")
w, b = CINA["White"][2], CINA["Black"][2]
aian = CINA["AIAN"][0] + CINA["AIAN"][1]
print(f" White + Black + AIAN + API (printed counts): {w} + {b} + {aian} + {api_c[2]} = "
f"{w + b + aian + api_c[2]} of {CINA_N['overall']}")
print(f" ... with API at 5.9%: {w + b + aian + implied} of {CINA_N['overall']} "
f"(exceeds the total by {w + b + aian + implied - CINA_N['overall']})")
print(f" White + Black + API(5.9%) alone: {w + b + implied} "
f"({'exceeds' if w + b + implied > CINA_N['overall'] else 'within'} the total)")
print("\n3. Trial share of Asian or Pacific Islander participants")
print(f" Results text: {API_TEXT[0]}/{API_TEXT[1]} = {pct(*API_TEXT):.2f}% (printed 4.5%)")
print(f" Table/Abstract: {TRIAL['API'][2]}/{TRIAL_N['overall']} = {pct(TRIAL['API'][2], TRIAL_N['overall']):.2f}% (printed 4.5%)")
print(f" {API_TEXT[0]} = {TRIAL['API'][0]} + {TRIAL['API'][1]} (sum of the Table's period rows): "
f"{API_TEXT[0] == TRIAL['API'][0] + TRIAL['API'][1]}")
# ---------------------------------------------------------------- 4. the tests
print("\n4. Two-sample tests with continuity correction (method check, then API)")
for key, (x1, n1, x2, n2) in {
"White overall": (7154, 8309, 4113096, 4891486),
"Black overall": (BLACK_TEXT[0], BLACK_TEXT[1], 578326, 4891486),
"White 2000-2011": (2780, 3245, 2378019, 2800711),
"White 2012-2018": (4374, 5063, 1735077, 2090775),
"Black 2000-2011": (244, 3245, 322701, 2800711),
"Black 2012-2018": (270, 5063, 255625, 2090775),
"AIAN 2000-2011": (10, 3245, 10905, 2800711),
"AIAN 2012-2018": (13, 5063, 9484, 2090775),
"Hispanic 2000-2011": (161, 1792, 169297, 2800711),
"Hispanic 2012-2018": (240, 3295, 156118, 2090775),
}.items():
results[key] = prop_test(x1, n1, x2, n2)
line(key, results[key], REPORTED[key])
print()
results["API overall, 371/8309"] = prop_test(371, 8309, 145568, 4891486)
results["API overall, 272/8308"] = prop_test(272, 8308, 145568, 4891486)
results["API 2000-2011"] = prop_test(121, 3245, 75033, 2800711)
results["API 2012-2018"] = prop_test(151, 5063, 70535, 2090775)
line("API overall, Table count 371/8309", results["API overall, 371/8309"], REPORTED["API overall"])
line("API overall, text count 272/8308", results["API overall, 272/8308"], REPORTED["API overall"])
line("API 2000-2011", results["API 2000-2011"], REPORTED["API 2000-2011"])
line("API 2012-2018", results["API 2012-2018"], REPORTED["API 2012-2018"])
d371 = results["API overall, 371/8309"][2]
print(f" Reported -1.5 vs recomputed {d371:+.2f} with 371/8309: same size, opposite sign")
print(f" Printed percentages: 4.5 - 5.9 = {4.5 - 5.9:+.1f}; 4.47 - 5.9 = "
f"{pct(371, 8309) - 5.9:+.2f}; 4.47 - 2.98 = {pct(371, 8309) - cina_api_all:+.2f}")
print(f" Printed percentages for 2012-2018: 3.0 - 3.4 = {3.0 - 3.4:+.1f} (printed -0.75)")
# ---------------------------------------------------------------- 5. ratios
print("\n5. Ratio of trial share to CiNA share (Results paragraph 3, Figure 3)")
ratio = {}
for g, (n_t, n_c) in {"API": (TRIAL_N, CINA_N), "Black": (TRIAL_N, CINA_N),
"Hispanic": (HISP_TRIAL_N, CINA_N)}.items():
r00 = (TRIAL[g][0] / n_t["2000-2011"]) / (CINA[g][0] / n_c["2000-2011"])
r12 = (TRIAL[g][1] / n_t["2012-2018"]) / (CINA[g][1] / n_c["2012-2018"])
ratio[g] = (r00, r12)
print(f" {g:8s}: {r00:.3f} -> {r12:.3f} (text: {REPORTED_RATIOS[g][0]} -> {REPORTED_RATIOS[g][1]})")
r_all_371 = (371 / 8309) / (145568 / 4891486)
r_all_272 = (272 / 8308) / (145568 / 4891486)
print(f" API overall: {r_all_371:.3f} (371/8309), {r_all_272:.3f} (272/8308)")
# ---------------------------------------------------------------- 6. Table consistency
print("\n6. Table: subsets vs all trials (API column)")
print(f" 2012-2018: industry {INDUSTRY_API[1]}, academic {ACADEMIC_API[1]}, all trials {TRIAL['API'][1]}"
f" -> industry subset exceeds all trials: {INDUSTRY_API[1] > TRIAL['API'][1]}")
print(f" Industry rows add up: {INDUSTRY_API[0]} + {INDUSTRY_API[1]} = {INDUSTRY_API[0] + INDUSTRY_API[1]} "
f"(All {INDUSTRY_API[2]}); academic: {ACADEMIC_API[0]} + {ACADEMIC_API[1]} = "
f"{ACADEMIC_API[0] + ACADEMIC_API[1]} (All {ACADEMIC_API[2]})")
print(f" Industry 2012-2018 API share: {INDUSTRY_API[1]}/{INDUSTRY_N[1]} = {pct(INDUSTRY_API[1], INDUSTRY_N[1]):.2f}% (printed 4.9)")
print(f" All-trials 2012-2018 API share: {TRIAL['API'][1]}/{TRIAL_N['2012-2018']} = "
f"{pct(TRIAL['API'][1], TRIAL_N['2012-2018']):.2f}% (printed 3.0)")
implied_12 = TRIAL["API"][2] - TRIAL["API"][0]
print(f" All-row minus 2000-2011 row: {TRIAL['API'][2]} - {TRIAL['API'][0]} = {implied_12} "
f"({pct(implied_12, TRIAL_N['2012-2018']):.2f}% of {TRIAL_N['2012-2018']}); >= industry {INDUSTRY_API[1]}: "
f"{implied_12 >= INDUSTRY_API[1]}")
print(" Supplement eTable minus Table, overall vs sum of periods (EGFR+ trials added back):")
for g, (e0, e1, eall) in ETABLE.items():
t0, t1, tall = TRIAL[g]
d_all = eall - tall
d_per = (e0 - t0) + (e1 - t1)
print(f" {g:8s}: eTable periods {e0}+{e1}={e0 + e1} (All {eall}); added overall {d_all}, "
f"added by period {d_per}{'' if d_all == d_per else ' <-- differs'}")
x = ETABLE["API"][2] - TRIAL["API"][2] - (ETABLE["API"][0] - TRIAL["API"][0])
x = ETABLE["API"][1] - x
print(f" 2012-2018 API count that makes the API column additive like the others: {x}")
results["API 2012-2018, count 250"] = prop_test(implied_12, 5063, 70535, 2090775)
line("API 2012-2018 if count were 371-121=250", results["API 2012-2018, count 250"])
print(f" ratio with 250: {(implied_12 / 5063) / (70535 / 2090775):.3f}")
# ---------------------------------------------------------------- 7. Conclusions sentence
print("\n7. Groups named in the Conclusions as under-enrolled 'overall' and 'at academic institutions'")
hisp_c = CINA["Hispanic"][0] + CINA["Hispanic"][1]
aian_c = CINA["AIAN"][0] + CINA["AIAN"][1]
results["Hispanic overall"] = prop_test(401, 5076, hisp_c, CINA_N["overall"])
results["AIAN overall"] = prop_test(23, 8309, aian_c, CINA_N["overall"])
line("Hispanic overall (401/5076 vs CiNA period sum)", results["Hispanic overall"])
line("AIAN overall (23/8309 vs CiNA period sum)", results["AIAN overall"])
line("Black overall", results["Black overall"])
line("API overall, Table count 371/8309", results["API overall, 371/8309"])
results["Academic API overall"] = prop_test(ACADEMIC_API[2], ACADEMIC_N[2], 145568, CINA_N["overall"])
results["Academic API 2000-2011"] = prop_test(ACADEMIC_API[0], ACADEMIC_N[0], 75033, CINA_N["2000-2011"])
results["Academic API 2012-2018"] = prop_test(ACADEMIC_API[1], ACADEMIC_N[1], 70535, CINA_N["2012-2018"])
ah_all_n = ACADEMIC_HISP_N[0] + ACADEMIC_HISP_N[1]
results["Academic Hispanic overall"] = prop_test(ACADEMIC_HISP[2], ah_all_n, hisp_c, CINA_N["overall"])
results["Academic Hispanic 2000-2011"] = prop_test(ACADEMIC_HISP[0], ACADEMIC_HISP_N[0], 169297, CINA_N["2000-2011"])
results["Academic Hispanic 2012-2018"] = prop_test(ACADEMIC_HISP[1], ACADEMIC_HISP_N[1], 156118, CINA_N["2012-2018"])
results["Academic AIAN overall"] = prop_test(ACADEMIC_AIAN_ALL, ACADEMIC_N[2], aian_c, CINA_N["overall"])
for k in ["Academic API overall", "Academic API 2000-2011", "Academic API 2012-2018",
"Academic Hispanic overall", "Academic Hispanic 2000-2011", "Academic Hispanic 2012-2018",
"Academic AIAN overall"]:
line(k, results[k])
print(f" (academic Hispanic denominator {ACADEMIC_HISP_N[0]} + {ACADEMIC_HISP_N[1]} = {ah_all_n}; "
f"276/{ah_all_n} = {pct(276, ah_all_n):.2f}%, printed 7.8)")
# ---------------------------------------------------------------- figures
INK, INK2 = "#0b0b0b", "#52514e"
BG, ORANGE, BLUE, GRID = "#fcfcfb", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "axes.edgecolor": INK2, "xtick.color": INK2, "ytick.color": INK,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
def style(ax):
ax.grid(axis="x", color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# Figure 1: shares, reported vs recomputed
rows1 = [ # label, reported %, recomputed %, recomputed label
("CiNA registry, 2000–2011\n75 033 / 2 800 711", 2.7, cina_api_p00),
("CiNA registry, 2012–2018\n70 535 / 2 090 775", 3.4, cina_api_p12),
("CiNA registry, 2000–2018\n145 568 / 4 891 486", 5.9, cina_api_all),
("Trials, 2000–2018 (Results text)\n272 / 8308", 4.5, pct(*API_TEXT)),
("Trials, 2000–2018 (Table, Abstract)\n371 / 8309", 4.5, pct(371, 8309)),
]
fig, ax = plt.subplots(figsize=(7.2, 4.7))
y = np.arange(len(rows1))[::-1].astype(float)
y[3:] -= 0.35 # gap between registry and trial rows
ax.axvspan(cina_api_p00, cina_api_p12, ymin=0.47, ymax=1.0, color=BLUE, alpha=0.10, linewidth=0)
ax.text((cina_api_p00 + cina_api_p12) / 2, y[0] + 0.55,
"a pooled 2000–2018 share\nmust fall in this band", ha="center", va="bottom",
fontsize=10, color=INK2)
for (lab, rep, rec), yy in zip(rows1, y):
ax.plot([min(rep, rec), max(rep, rec)], [yy, yy], color=GRID, linewidth=2, zorder=1)
ax.scatter([rep], [yy], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2, zorder=3)
ax.scatter([rec], [yy], s=70, color=BLUE, zorder=4)
ax.text(5.9 + 0.15, y[2], "printed 5.9%", va="center", color=ORANGE, fontsize=10.5)
ax.text(cina_api_all - 0.15, y[2] - 0.32, f"{cina_api_all:.2f}%", ha="right", va="center",
color=BLUE, fontsize=10.5)
ax.text(4.5 + 0.15, y[3], "printed 4.5%", va="center", color=ORANGE, fontsize=10.5)
ax.text(pct(*API_TEXT) - 0.15, y[3] - 0.32, f"{pct(*API_TEXT):.2f}%", ha="right", va="center",
color=BLUE, fontsize=10.5)
ax.text(4.5 + 0.15, y[4], "printed 4.5%", va="center", color=ORANGE, fontsize=10.5)
ax.text(pct(371, 8309) - 0.15, y[4] - 0.32, f"{pct(371, 8309):.2f}%", ha="right", va="center",
color=BLUE, fontsize=10.5)
ax.set_yticks(y)
ax.set_yticklabels([r[0] for r in rows1], fontsize=10)
ax.set_xlim(0, 7.6)
ax.set_ylim(y[-1] - 0.7, y[0] + 1.35)
ax.set_xlabel("Asian or Pacific Islander share, %")
style(ax)
h1 = ax.scatter([], [], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2,
label="printed in the paper")
h2 = ax.scatter([], [], s=70, color=BLUE, label="recomputed from the paper's own counts")
fig.legend(handles=[h1, h2], loc="lower center", frameon=False, fontsize=10, ncol=2,
bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.08, 1, 1))
save(fig, "fig1")
# Figure 2: differences trial - CiNA with 95% CIs
rows2 = [ # label, reported diff or None, result key or None, hollow
("2000–2018, as printed", REPORTED["API overall"]["diff"], None),
("2000–2018, Table count 371 / 8309", None, "API overall, 371/8309"),
("2000–2018, text count 272 / 8308", None, "API overall, 272/8308"),
("2000–2011, 121 / 3245", REPORTED["API 2000-2011"]["diff"], "API 2000-2011"),
("2012–2018, 151 / 5063", REPORTED["API 2012-2018"]["diff"], "API 2012-2018"),
("2012–2018, if 371 − 121 = 250", None, "API 2012-2018, count 250"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.0))
y = np.array([5.6, 4.6, 3.6, 2.2, 1.0, 0.0])
ax.axvline(0, color=INK2, linewidth=0.9, zorder=1)
for (lab, rep, key), yy in zip(rows2, y):
off = 0.14 if (rep is not None and key is not None) else 0.0
if key is not None:
_, _, d, lo, hi, p = results[key]
hollow = key.endswith("250")
ax.plot([lo, hi], [yy - off, yy - off], color=BLUE, linewidth=2, zorder=2)
ax.scatter([d], [yy - off], s=70, zorder=3, linewidth=2,
facecolor=BG if hollow else BLUE, edgecolor=BLUE)
ptxt = "P < .001" if p < 0.001 else f"P = {p:.2f}".replace("0.", ".")
ax.text(hi + 0.08, yy - off, f"{d:+.2f} ({ptxt})".replace("-", "\u2212"), va="center",
color=BLUE, fontsize=10)
if rep is not None:
ax.scatter([rep], [yy + off], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2, zorder=4)
ptxt = " (P < .001)" if key is None else ""
txt = f"{rep:+g}{ptxt}".replace("-", "\u2212")
if key is None:
ax.text(rep, yy + 0.24, txt.replace(" ", " "), va="bottom", ha="center",
color=ORANGE, fontsize=10)
else:
ax.text(rep + (0.14 if rep > 0 else -0.14), yy + off, txt, va="center",
ha="left" if rep > 0 else "right", color=ORANGE, fontsize=10)
ax.set_yticks(y)
ax.set_yticklabels([r[0] for r in rows2], fontsize=10)
ax.set_xlim(-2.6, 3.9)
ax.set_ylim(-0.6, 6.4)
ax.set_xlabel("Trial share minus CiNA share, percentage points\n"
"← under-representation over-representation →", fontsize=10.5)
style(ax)
h1 = ax.scatter([], [], s=150, facecolor="none", edgecolor=ORANGE, linewidth=2.2,
label="printed in the paper")
h2 = ax.errorbar([], [], xerr=[], fmt="o", color=BLUE, markersize=8, linewidth=2,
label="recomputed, 95% CI (prop.test)")
h3 = ax.scatter([], [], s=70, facecolor=BG, edgecolor=BLUE, linewidth=2,
label="recomputed for a count implied by the Table")
fig.legend(handles=[h1, h2, h3], loc="lower center", frameon=False, fontsize=10, ncol=2,
bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.12, 1, 1))
save(fig, "fig2")
print("\nFigures written to", HERE)
Pooled suicide-ideation effect is called non-significant although its reported 95% CI, 0.01 to 0.21, excludes zero
JAMA Pediatr (2025) · doi:10.1001/jamapediatrics.2025.1012 · PMC12090068
Summary
The Abstract and Results report the pooled association between safety planning interventions (SPIs) and suicide ideation (SI) as "no significant association" (Hedges g = 0.11; 95% CI, 0.01-0.21). An interval that lies wholly above 0 is not consistent with a non-significant two-sided test at the 5% level. Pooling the three study rows printed in Figure 2A with the method the Methods state (random effects with Hartung-Knapp CIs) reproduces the printed interval to within 0.01 (0.014 to 0.217 against the printed 0.01 to 0.21; jittering the printed inputs by ±0.005 moves the recomputed upper bound between 0.208 and 0.228) and gives t = 4.91 on 2 df, p = 0.039. On the paper's own method, the SI result is therefore nominally significant, and it lies on the side Figure 2A labels "Favors control". The conflict is with the Abstract and Results wording "no significant association". The Key Points and Discussion say SPIs were not associated with "reduction(s)" in SI, which is literally compatible with an unfavorable effect, although neither mentions it. The 2026 correction to this article did not address the SI wording.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py. The Abstract, Results and Methods sentences were also checked on the publisher's current (corrected) page.
Abstract, Results:
There was no significant association between safety planning interventions and suicide ideation (Hedges g = 0.11; 95% CI, 0.01-0.21), behavior (Hedges g = -0.09; 95% CI, -0.20 to 0.02), […] at follow-up.
Results, Meta-Analysis:
No significant associations were observed between SPIs and SI (Hedges g = 0.11; 95% CI, 0.01-0.21) (Figure 2A), SRB (Hedges g = -0.09; 95% CI, -0.20 to 0.02) (Figure 2B), […]
Methods, Statistical Analyses:
Values of 0.2, 0.5, and 0.8 were considered small, moderate, and large in magnitude, respectively.30 Effect size estimates were based on a random-effects model; Hartung-Knapp adjustments were applied to calculate the CIs.31 Study weights were derived from the random-effects model with restricted maximum likelihood estimation […]
Effect sizes were calculated and analyzed using R statistical software (R Project for Statistical Computing) and the metafor package.24,25
Key Points, Findings:
safety planning was not associated with reductions in suicide ideation, suicide-related behavior, suicide attempts, or suicide-related re-presentation to health care settings (eg, emergency departments; inpatient units) at follow-up.
Discussion:
We found no association between SPI and reduction in SI, SRB, SA, or re-presentation to acute care settings at follow-up 1 to 6 months after SPI.
[…] SPIs do not decrease SITBs for adolescents in crisis.
Figure 2A (forest plot, values read from the published image; studies identified here by reference number and year). Ref 40 (2018), n = 66: 0.09 (−0.16 to 0.33), weight 33.2%. Ref 35 (2011), n = 128: 0.12 (−0.06 to 0.29), weight 63.0%. Ref 38 (2019), n = 36: 0.27 (−0.46 to 0.99), weight 3.7%. Overall effect: 0.11 (0.01 to 0.21); I2 = 0%, P = .89. The x-axis reads "Favors intervention | Favors control". The Methods state no significance level or test for the pooled estimates.
What the paper's own numbers show
Method. Each study's standard error was taken from its printed 95% CI as (upper − lower)/(2 × 1.96). The three studies were then pooled by inverse variance under a random-effects model. The REML estimate of τ2 is 0 (Q = 0.22 on 2 df). I computed three intervals for the pooled estimate: the stated Hartung-Knapp (HK) interval (t distribution, k − 1 = 2 df, no truncation of the scale factor), a plain Wald z interval, and the "ad hoc" HK variant whose scale factor is floored at 1. As controls, the same code was run on the suicide-related behavior (SRB) pool in Figure 2B and on the two sensitivity pools in the Supplement 1 eFigure. To test rounding, all nine printed SI study values were jittered by ±0.005 (20,000 draws). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Pooled SI Hedges g | Abstract; Results; Figure 2A | 0.11 | 0.116 |
| Pooled SI 95% CI, HK (the stated method) | Abstract; Results; Figure 2A | 0.01 to 0.21 | 0.014 to 0.217 (upper bound 0.208 to 0.228 under ±0.005 jitter) |
| Pooled SI test under HK | not reported; text says "no significant association" | — | t = 4.91, df = 2, p = 0.039 |
| p implied by the printed interval itself, t(2) | — | — | 0.042 (0.035 to 0.050 over all rounding corners of 0.11, 0.01 and 0.21; 0.050 only at the corner 0.105, 0.005, 0.215, which is a conservative bound: a symmetric interval with bounds 0.005 and 0.215 has estimate 0.110 and p = 0.046) |
| p implied by the printed interval, normal approximation | — | — | 0.031 |
| HK lower bound, study rows jittered ±0.005 | — | — | minimum 0.004; above 0 in 100% of draws; p at most 0.047 |
| Heterogeneity, SI | Figure 2A | I2 = 0%, P = .89 | Q = 0.219 (df 2), P = 0.896, I2 = 0%, τ2 = 0 |
| Study weights, SI | Figure 2A | 33.2%, 63.0%, 3.7% | 32.5%, 63.8%, 3.7% (SEs from rounded CIs) |
| Alternative: Wald z random-effects CI | not reported | — | −0.024 to 0.256, p = 0.104 |
| Alternative: HK with scale factor floored at 1 | not reported | — | −0.191 to 0.423, p = 0.246 |
| Control: SRB pooled g and HK CI | Abstract; Results; Figure 2B | −0.09 (−0.20 to 0.02), "no significant" | −0.087 (−0.190 to 0.017), p = 0.076; contains 0, consistent. Lower bound 0.01 off; it rounds to −0.20 in 7.9% of jittered draws. Wald: −0.203 to 0.029, p = 0.141 |
| Control: SI sensitivity pool, 4 studies | Supplement 1 eFigure | 0.10 (−0.03 to 0.23), I2 = 0.0% | 0.103 (−0.032 to 0.239), HK p = 0.094; contains 0. Upper bound 0.01 off; it rounds to 0.23 in 6.5% of jittered draws. Wald: −0.034 to 0.241, p = 0.140 |
| Control: SRB sensitivity pool, 6 studies | Supplement 1 eFigure | −0.12 (−0.28 to 0.04), I2 = 15.0% | −0.117 (−0.277 to 0.044), HK p = 0.121; both bounds round to the printed values. Wald: −0.229 to −0.004, p = 0.043 (excludes 0). Floored HK equals HK here (q = 1.18 > 1) |
| SI pooled g against the paper's "small" threshold | Methods | 0.2 | 0.116 < 0.2 |
Which interval method fits the printed CIs (largest difference between a recomputed and a printed bound; printed bounds have two decimals):
| Pool | Printed | HK (stated) | Wald z | HK floored at 1 |
|---|---|---|---|---|
| SI, Figure 2A | 0.01 to 0.21 | 0.007 | 0.046 | 0.213 |
| SRB, Figure 2B | −0.20 to 0.02 | 0.010 | 0.009 | 0.081 |
| SI, eFigure (4 studies) | −0.03 to 0.23 | 0.009 | 0.011 | 0.096 |
| SRB, eFigure (6 studies) | −0.28 to 0.04 | 0.004 | 0.051 (excludes 0) | 0.004 (same as HK) |
Untruncated HK matches all four printed Hedges g intervals to within about 0.01. Wald and floored HK miss the main SI interval by 0.05 and 0.21. Wald also misses the eFigure SRB interval by 0.05; under Wald that interval would exclude 0 (p = 0.043), but as printed it does not. For SRB in Figure 2B and the eFigure SI pool, Wald fits about as well as HK, so those two pools do not separate the methods. For eFigure SRB, floored HK coincides with HK because q > 1. Floored HK misses three of the four printed intervals by 0.08 or more. So two pools, the main SI pool and the eFigure SRB pool, separate HK from Wald, and both favor HK. The SA and re-presentation risk-ratio pools were not recomputed. Two of the four checked pools miss by 0.01 at one bound: SRB in Figure 2B (−0.190 against a printed −0.20) and the eFigure SI pool (upper bound 0.239 against a printed 0.23). Jittering the inputs by ±0.005 reaches the printed value in only 7.9% and 6.5% of draws, so rounding of the printed study rows explains these gaps only partly. They are consistent with the pooling having been run on unrounded study data, as the small differences in the recomputed study weights also suggest (for example 49.2% and 43.7% against a printed 47.7% and 45.1% for SRB). Together with the Methods statement, this supports HK as the method behind the printed CIs. For SI, that interval excludes 0, and it stays above 0 in every jittered draw. The scale factor is q = 0.109 because the three studies agree more closely than their sampling errors predict (Q = 0.22 < 2), so the HK standard error (0.024) is about one-third of the plain random-effects standard error (0.071). The t(2) critical value of 4.30 does not make up for this. For the printed interval itself, p reaches 0.050 only at the extreme rounding corner (estimate 0.105, bounds 0.005 and 0.215), and it is below 0.05 at every other corner. That corner is a conservative bound, not an attainable case: a symmetric HK interval with bounds 0.005 and 0.215 would have estimate 0.110 and p = 0.046. Recomputed from the jittered study rows, p is at most 0.047.
Figure
Figure 1. Pooled Hedges g for suicide ideation with 95% CIs on one axis; the thin vertical line is the null (g = 0). Positive g is on the side the paper's Figure 2A labels "favors control". Top row (orange): the value printed in the Abstract, Results and Figure 2A, 0.11 (0.01 to 0.21), which the text calls "no significant association". Second row (blue, filled): the stated method, random effects with Hartung-Knapp CIs, recomputed from the three study rows of Figure 2A. It gives 0.116 (0.014 to 0.217), p = 0.039, matching the printed interval to within 0.01. Lower two rows (blue, open markers): two methods the paper does not state, the plain Wald z interval (−0.024 to 0.256, p = 0.104) and HK with the scale factor floored at 1 (−0.191 to 0.423, p = 0.246). Either would support "non-significant", but neither gives the printed interval.
Why it matters
- A headline sentence contradicts the interval printed beside it. The SI result appears in the Abstract, the Results and Figure 2A, and the Abstract and Results both call it "no significant association". A reader who checks the interval against that wording finds them in conflict, with nothing in the paper to resolve it.
- The direction matters for interpretation. The interval lies on the "Favors control" side of Figure 2A. Under the stated method, the data therefore show a very small (below the paper's 0.2 threshold for "small"), nominally significant difference in SI in the unfavorable direction (worse in the intervention arms, going by the Figure 2A axis labels, the only place the paper states the sign of g), not a null. The Key Points wording "not associated with reductions in suicide ideation" and the Discussion sentence "We found no association between SPI and reduction in SI, SRB, SA, or re-presentation to acute care settings at follow-up 1 to 6 months after SPI" are both literally compatible with this, although neither mentions an unfavorable association. The Abstract and Results wording "There was no significant association" / "No significant associations were observed" between SPIs and SI does not describe the reported interval.
- The overall conclusion is weakened in wording but not overturned. The effect (g = 0.116) is below the paper's own 0.2 threshold for "small". It rests on three small studies, none of which is individually significant. Its significance depends on the untruncated HK adjustment, which with k = 3 and Q < k − 1 is known to give narrow intervals. The plain random-effects and floored-HK analyses are non-significant, and so is the four-study sensitivity analysis. The conclusion that current evidence does not support safety planning as a standalone intervention is not changed by these numbers. The specific statement that ideation showed no significant association, however, is not what the paper's stated analysis yields.
- No test is reported for the pooled estimates. The Methods give neither a significance level nor the test behind the "significant" labels, so a reader cannot tell which analysis supports the wording.
Explanations considered
- Rounding of the printed interval. Does not account for it. A printed lower bound of 0.01 means a true value between 0.005 and 0.015, still above 0. Over all rounding corners of the estimate and both bounds, the implied t(2) p-value ranges from 0.035 to 0.050. It reaches 0.050 only at the single corner with estimate 0.105 and bounds 0.005 and 0.215, and is below 0.05 at all others. That corner is a conservative bound rather than an attainable case: a symmetric interval with these bounds would have estimate 0.110, p = 0.046. Rounding the study rows does not account for it either: with all nine inputs jittered by ±0.005, the recomputed HK lower bound stays above 0 in every draw (minimum 0.004), and p stays below 0.05 (maximum 0.047).
- A typesetting or conversion artifact, such as a lost minus sign. Ruled out. Both bounds are positive in the PMC text, the Europe PMC XML, the publisher's page and the Figure 2A image. In the same sentence, the SRB interval keeps its minus sign ("-0.20 to 0.02"). The diamond in Figure 2A lies wholly to the right of the zero line. The study-level rows independently give a lower bound of 0.014.
- The printed interval itself is wrong. Not supported. Under the method the Methods name, the three study rows of Figure 2A give 0.014 to 0.217: the lower bound matches 0.01 and the upper bound is 0.007 above the printed 0.21, within the 0.208 to 0.228 range that jittering the printed inputs produces. The same method matches the SRB interval and both eFigure intervals to within about 0.01 (the Wald interval fits two of those three about as well, but misses the main SI and eFigure SRB intervals by about 0.05).
- The significance label comes from a different test than the interval. Possible, but not stated. A Wald z test (p = 0.104) or HK with the scale floored at 1 (p = 0.246) would make "non-significant" correct. Neither, however, produces the printed interval, and the paper names neither. In metafor, which the paper used, the Hartung-Knapp option reports a t-test from the same standard error as its interval, which would give p ≈ 0.04 here. If a different test was used for the label, the paper does not say so, and the reported interval would then be inconsistent with it.
- A one-sided test, an adjusted alpha or multiplicity control. None is described. The Methods state no alpha level and no correction for the four outcomes.
- "No significant association" meant "no significant benefit". Not what the sentences say. The Key Points phrase "not associated with reductions" fits a significant unfavorable effect, but the Abstract and Results phrase "no significant association" does not.
- Different populations, time points or models. Not applicable. The Abstract, the Results and Figure 2A quote the same estimate and interval for the same three-study pool.
- The published correction. It does not cover this. The correction (doi:10.1001/jamapediatrics.2026.2781; online July 6, 2026, print September 2026) fixes a single data-entry error: rehospitalization data were entered in place of suicide-attempt data for one study. It changes the suicide-attempt risk ratios in the Abstract, Results, Figure 2C and the eFigure. The SI estimate, interval and wording are unchanged on the current publisher page.
Status
- Version of record. Checked on the publisher's current, corrected page (jamanetwork.com, reached through doi.org). It carries the note "This article was corrected on July 6, 2026, to fix a single data-entry error within the meta-analysis", and the Abstract, Results and Methods sentences quoted above are unchanged. The PMC archive and the Europe PMC full-text XML have the same SI text. The Figure 2A values were read from the forest-plot image in the PMC/Europe PMC supplementary bundle.
- Supplements. Supplement 1 (eTables 1-3 and an eFigure) was opened. The eFigure's SI sensitivity analysis adds one study (4 studies; 0.10, −0.03 to 0.23). It includes 0 and is matched to within 0.01 by the same code. It does not report the main three-study analysis. The eFigure values (study rows, weights and pooled results) were taken from the original 2025 Supplement 1 PDF in the PMC/Europe PMC bundle; the 2026 correction revised only the eFigure's suicide-attempt panel, which is not used here. Supplement 2 is a data sharing statement.
- Corrections and comments. As of 2026-10-24, the PMC record links forward to the correction (PMC13338843) and to an invited commentary and a letter; Crossref lists no update relations for the article. Europe PMC lists the same correction, "Data-Entry Error in Meta-Analysis of Suicide Prevention in Children and Adolescents" (PMC13338843, doi:10.1001/jamapediatrics.2026.2781), which concerns suicide attempts only (see above). Europe PMC also lists an accompanying letter on the data-entry error (doi:10.1001/jamapediatrics.2026.2731) and an invited commentary, "The Role of Safety Planning in Suicidal Behavior Prevention" (PMID 40388184). The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title with "PubPeer" found no thread.
- Sources. PMC12090068 (via
src/show.py); Europe PMC full text and supplementary files for PMC12090068; https://doi.org/10.1001/jamapediatrics.2025.1012; https://doi.org/10.1001/jamapediatrics.2026.2781; Crossref; Europe PMC; PubPeer.
Suggested correction
The authors may wish to state which test underlies the significance statements and to make the SI wording agree with the reported analysis. If the stated Hartung-Knapp analysis is retained, the Abstract and Results could read, for example:
Safety planning interventions showed a very small association with suicide ideation at follow-up (Hedges g = 0.11; 95% CI, 0.01-0.21; 3 studies; positive g, labelled "Favors control" in Figure 2A), below the 0.2 threshold for a small effect.
The paper does not state the sign convention for g other than through the Figure 2A axis labels, so the direction should be confirmed against the extracted data. The Key Points ("not associated with reductions in suicide ideation") and the Discussion ("We found no association between SPI and reduction in SI, SRB, SA, or re-presentation...") are literally accurate under this reading and need no correction, although the authors may wish to mention the direction of the SI estimate there. Instead of rewording the Abstract and Results, the authors could report the truncated (ad hoc) Hartung-Knapp interval, which is often recommended when Q < k − 1 with few studies, or the plain random-effects interval. Either would give an SI interval that includes 0. The method would then need to be stated in the Methods and applied consistently to all four outcomes. In either case, adding the pooled p-values and the significance level to the Methods or Figure 2 would let readers check the labels.
Recomputation script (python recompute.py.txt)
"""Recompute the pooled suicide-ideation estimate of PMC12090068
(JAMA Pediatr 2025, doi:10.1001/jamapediatrics.2025.1012).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg and fig1.png next to this file.
Inputs are the study rows printed in the paper's Figure 2A (suicide ideation)
and Figure 2B (suicide-related behavior), and the suicide-ideation and
suicide-related-behavior panels of the eFigure in Supplement 1. Studies are
identified here by the paper's reference number and year only. Each study's
standard error is taken from its printed 95% CI as (upper - lower) / (2 x 1.96).
Methods: "Effect size estimates were based on a random-effects model;
Hartung-Knapp adjustments were applied to calculate the CIs." and study weights
from REML. Positive g lies on the "Favors control" side of Figure 2A.
"""
import os
import numpy as np
from scipy import stats, optimize
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ------------------------------------------------------------------ inputs
# Figure 2A, suicide ideation: (ref, year, n, g, lower, upper, printed weight %)
SI = [("ref 40", 2018, 66, 0.09, -0.16, 0.33, 33.2),
("ref 35", 2011, 128, 0.12, -0.06, 0.29, 63.0),
("ref 38", 2019, 36, 0.27, -0.46, 0.99, 3.7)]
SI_PRINTED = (0.11, 0.01, 0.21) # "Overall effect 0.11 (0.01 to 0.21)"; I2 = 0%, P = .89
# Figure 2B, suicide-related behavior
SRB = [("ref 44", 2019, 139, -0.12, -0.29, 0.04, 47.7),
("ref 35", 2011, 131, -0.08, -0.26, 0.09, 45.1),
("ref 40", 2018, 66, 0.08, -0.43, 0.59, 5.2),
("ref 38", 2019, 36, 0.16, -0.67, 1.00, 1.9)]
SRB_PRINTED = (-0.09, -0.20, 0.02) # I2 = 0%, P = .81
# eFigure (original 2025 Supplement 1 in the PMC/Europe PMC bundle), panels a and b: analyses that add
# studies with SPI components in the control arm. The weights are the eFigure's own printed weights.
SI_EFIG = [("ref 39", 2021, 80, -0.25, -0.99, 0.50, 3.4),
("ref 40", 2018, 66, 0.09, -0.16, 0.33, 32.1),
("ref 35", 2011, 128, 0.12, -0.06, 0.29, 60.9),
("ref 38", 2019, 36, 0.27, -0.46, 0.99, 3.6)]
SI_EFIG_PRINTED = (0.10, -0.03, 0.23) # I2 = 0.0%, p = 0.78
SRB_EFIG = [("ref 42", 2019, 142, -0.70, -1.35, -0.06, 3.1),
("ref 39", 2021, 80, -0.62, -1.40, 0.17, 2.1),
("ref 44", 2019, 139, -0.12, -0.29, 0.04, 45.3),
("ref 35", 2011, 131, -0.08, -0.26, 0.09, 42.8),
("ref 40", 2018, 66, 0.08, -0.43, 0.59, 4.9),
("ref 38", 2019, 36, 0.16, -0.67, 1.00, 1.8)]
SRB_EFIG_PRINTED = (-0.12, -0.28, 0.04) # I2 = 15.0%, p = 0.32
def pool(g, lo, hi):
"""Random-effects pooling (REML tau^2) with Wald, Hartung-Knapp and HK-with-floor intervals."""
g, lo, hi = (np.asarray(x, float) for x in (g, lo, hi))
v = ((hi - lo) / (2 * Z)) ** 2
k = len(g)
df = k - 1
w = 1 / v
mu_fe = (w * g).sum() / w.sum()
Q = (w * (g - mu_fe) ** 2).sum()
def nll(t2): # REML negative log-likelihood
W = 1 / (v + t2)
m = (W * g).sum() / W.sum()
return 0.5 * (np.log(v + t2).sum() + np.log(W.sum()) + (W * (g - m) ** 2).sum())
r = optimize.minimize_scalar(nll, bounds=(0, 5), method="bounded")
tau2 = r.x if nll(r.x) < nll(0.0) - 1e-12 else 0.0
W = 1 / (v + tau2)
mu = (W * g).sum() / W.sum()
se = 1 / np.sqrt(W.sum())
q = (W * (g - mu) ** 2).sum() / df # Hartung-Knapp scale factor
tc = stats.t.ppf(0.975, df)
se_hk = se * np.sqrt(q)
se_hk1 = se * max(1.0, np.sqrt(q)) # ad hoc variant: q floored at 1
out = dict(k=k, df=df, Q=Q, pQ=stats.chi2.sf(Q, df), I2=max(0.0, (Q - df) / Q) * 100,
tau2=tau2, mu=mu, weights=100 * W / W.sum(), se=se, q=q, tcrit=tc,
wald=(mu - Z * se, mu + Z * se, 2 * stats.norm.sf(abs(mu / se)), mu / se),
hk=(mu - tc * se_hk, mu + tc * se_hk, 2 * stats.t.sf(abs(mu / se_hk), df), mu / se_hk, se_hk),
hk1=(mu - tc * se_hk1, mu + tc * se_hk1, 2 * stats.t.sf(abs(mu / se_hk1), df), se_hk1))
return out
def cols(rows):
return [r[3] for r in rows], [r[4] for r in rows], [r[5] for r in rows]
def report(name, rows, printed):
r = pool(*cols(rows))
print(f"\n== {name}: k={r['k']}, printed overall {printed[0]:.2f} ({printed[1]:.2f} to {printed[2]:.2f})")
print(f" Q={r['Q']:.3f} (df={r['df']}), P_Q={r['pQ']:.3f}, I2={r['I2']:.0f}%, tau2 (REML)={r['tau2']:.4f}")
print(f" weights % recomputed: {np.round(r['weights'], 1)} printed: {[x[6] for x in rows]}")
print(f" pooled g = {r['mu']:.4f}")
a, b, p, zz = r["wald"]
print(f" Wald z : {a:.3f} to {b:.3f}, z={zz:.3f}, p={p:.4f}")
a, b, p, t, s = r["hk"]
print(f" Hartung-Knapp : {a:.3f} to {b:.3f}, SE={s:.4f}, t={t:.3f} (df={r['df']}, crit {r['tcrit']:.3f}), "
f"p={p:.4f} [scale factor q={r['q']:.3f}]")
a, b, p, s = r["hk1"]
print(f" HK, q floored at 1: {a:.3f} to {b:.3f}, SE={s:.4f}, p={p:.4f}")
return r
print("Reconstruction of the pooled random-effects estimates from the paper's printed study rows")
si = report("Suicide ideation, Figure 2A", SI, SI_PRINTED)
srb = report("Suicide-related behavior, Figure 2B (control check)", SRB, SRB_PRINTED)
sie = report("Suicide ideation, eFigure panel a (4 studies)", SI_EFIG, SI_EFIG_PRINTED)
srbe = report("Suicide-related behavior, eFigure panel b (6 studies; q > 1, so HK and floored HK coincide)",
SRB_EFIG, SRB_EFIG_PRINTED)
# ---------------------------------------------- which interval method fits the printed CIs?
print("\n== Largest bound deviation |recomputed - printed| for each method (printed bounds have 2 decimals)")
for name, r, pr in [("SI, Figure 2A", si, SI_PRINTED), ("SRB, Figure 2B", srb, SRB_PRINTED),
("SI, eFigure a", sie, SI_EFIG_PRINTED), ("SRB, eFigure b", srbe, SRB_EFIG_PRINTED)]:
dev = {m: max(abs(r[m][0] - pr[1]), abs(r[m][1] - pr[2])) for m in ("hk", "wald", "hk1")}
zero = {m: "excludes 0" if r[m][0] > 0 or r[m][1] < 0 else "contains 0" for m in ("hk", "wald", "hk1")}
print(f" {name}: HK {dev['hk']:.3f} ({zero['hk']}), Wald {dev['wald']:.3f} ({zero['wald']}), "
f"floored HK {dev['hk1']:.3f} ({zero['hk1']}); printed interval "
f"{'excludes' if pr[1] > 0 or pr[2] < 0 else 'contains'} 0")
# ---------------------------------------------- p-value implied by the printed SI interval itself
print("\n== p-value implied by the printed SI interval 0.11 (0.01 to 0.21)")
tc2 = stats.t.ppf(0.975, 2)
for label, crit, pf in [("normal (z)", Z, lambda x: 2 * stats.norm.sf(x)),
("t, df = 2 (Hartung-Knapp, 3 studies)", tc2, lambda x: 2 * stats.t.sf(x, 2))]:
s = (0.21 - 0.01) / (2 * crit)
print(f" {label}: SE={s:.4f}, statistic={0.11 / s:.3f}, p={pf(0.11 / s):.4f}")
# worst case within rounding: lowest lower bound 0.005, highest upper bound 0.215, estimate 0.105
worst = []
for est in (0.105, 0.11, 0.115):
for lb in (0.005, 0.015):
for ub in (0.205, 0.215):
s = (ub - lb) / (2 * tc2)
worst.append(2 * stats.t.sf(est / s, 2))
print(f" t, df = 2, over all rounding corners (estimate 0.105-0.115, bounds +/-0.005): "
f"p from {min(worst):.4f} to {max(worst):.4f}")
# the maximum (estimate 0.105, bounds 0.005 and 0.215) is a conservative bound, not an attainable case:
# a symmetric t interval with bounds 0.005 and 0.215 has its estimate at the midpoint, 0.110
s = (0.215 - 0.005) / (2 * tc2)
print(f" symmetric interval with bounds 0.005 and 0.215: estimate {(0.005 + 0.215) / 2:.3f}, SE={s:.4f}, "
f"t={0.110 / s:.3f}, p={2 * stats.t.sf(0.110 / s, 2):.4f}")
# ---------------------------------------------- rounding of the study rows
rng = np.random.default_rng(20261024)
N = 20000
def jitter(rows, n=N):
"""Jitter every printed study value uniformly by +/-0.005 and re-pool with Hartung-Knapp."""
g0, lo0, hi0 = (np.array(x) for x in cols(rows))
k = len(rows)
out = np.empty((n, 3))
for i in range(n):
e = rng.uniform(-0.005, 0.005, size=(3, k))
r = pool(g0 + e[0], lo0 + e[1], hi0 + e[2])
out[i] = r["hk"][0], r["hk"][1], r["hk"][2]
return out
J = jitter(SI)
lows, highs, ps = J[:, 0], J[:, 1], J[:, 2]
print(f"\n== Rounding of the Figure 2A study rows (each printed value jittered uniformly by +/-0.005, {N} draws)")
print(f" Hartung-Knapp lower bound: min {lows.min():.4f}, 2.5th pct {np.percentile(lows, 2.5):.4f}, "
f"max {lows.max():.4f}; share above 0: {100 * (lows > 0).mean():.1f}%")
print(f" Hartung-Knapp upper bound: min {highs.min():.4f}, max {highs.max():.4f} (printed 0.21); "
f"share rounding to 0.21: {100 * (np.round(highs, 2) == 0.21).mean():.1f}%")
print(f" both bounds round to 0.01 and 0.21: {100 * ((np.round(lows, 2) == 0.01) & (np.round(highs, 2) == 0.21)).mean():.1f}%")
print(f" Hartung-Knapp p: median {np.median(ps):.4f}, max {ps.max():.4f}; share below 0.05: {100 * (ps < 0.05).mean():.1f}%")
print("\n== Same jitter for the three control pools (does rounding of the study rows explain the residual gaps?)")
for name, rows_, pr in [("SRB, Figure 2B", SRB, SRB_PRINTED), ("SI, eFigure a", SI_EFIG, SI_EFIG_PRINTED),
("SRB, eFigure b", SRB_EFIG, SRB_EFIG_PRINTED)]:
Jc = jitter(rows_, 5000)
lo_ok = np.round(Jc[:, 0], 2) == pr[1]
hi_ok = np.round(Jc[:, 1], 2) == pr[2]
print(f" {name}: HK lower bound {Jc[:, 0].min():.4f} to {Jc[:, 0].max():.4f} (printed {pr[1]:.2f}; "
f"rounds to it in {100 * lo_ok.mean():.1f}% of draws); "
f"upper bound {Jc[:, 1].min():.4f} to {Jc[:, 1].max():.4f} (printed {pr[2]:.2f}; "
f"rounds to it in {100 * hi_ok.mean():.1f}%); both bounds: {100 * (lo_ok & hi_ok).mean():.1f}%")
print("\n== Other checks")
print(f" Printed SI interval 0.01 to 0.21 contains 0: {0.01 <= 0 <= 0.21}")
print(f" Printed SRB interval -0.20 to 0.02 contains 0: {-0.20 <= 0 <= 0.02}")
print(f" SI pooled g relative to the paper's 'small' threshold of 0.2: {si['mu']:.3f} < 0.2 -> {si['mu'] < 0.2}")
# ------------------------------------------------------------------ figure
ORANGE, BLUE, INK, INK2, BG, GRID = "#eb6834", "#2a78d6", "#0b0b0b", "#52514e", "#fcfcfb", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "svg.fonttype": "none"})
rows = [ # (title, est, lo, hi, color, filled, right-hand label)
("Printed in the paper (Abstract, Results, Figure 2A)", *SI_PRINTED, ORANGE, True,
"0.11 (0.01 to 0.21)\nlabelled \"no significant\nassociation\""),
("Recomputed: random effects + Hartung-Knapp (stated method)",
si["mu"], si["hk"][0], si["hk"][1], BLUE, True,
f"{si['mu']:.3f} ({si['hk'][0]:.3f} to {si['hk'][1]:.3f})\np = {si['hk'][2]:.3f}"),
("Recomputed: plain random-effects Wald z (not stated)",
si["mu"], si["wald"][0], si["wald"][1], BLUE, False,
f"{si['mu']:.3f} ({si['wald'][0]:.3f} to {si['wald'][1]:.3f})\np = {si['wald'][2]:.3f}"),
("Recomputed: Hartung-Knapp, scale floored at 1 (not stated)",
si["mu"], si["hk1"][0], si["hk1"][1], BLUE, False,
f"{si['mu']:.3f} ({si['hk1'][0]:.3f} to {si['hk1'][1]:.3f})\np = {si['hk1'][2]:.3f}"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.5))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
xmin, xmax = -0.30, 0.50
ys = np.arange(len(rows))[::-1] * 1.0
for (title, est, lo, hi, c, filled, lab), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([lo, lo], [y - 0.09, y + 0.09], color=c, lw=2, zorder=3)
ax.plot([hi, hi], [y - 0.09, y + 0.09], color=c, lw=2, zorder=3)
ax.plot(est, y, "o", ms=9, mfc=c if filled else BG, mec=c, mew=2, zorder=4)
ax.text(xmin + 0.005, y + 0.34, title, color=INK, fontsize=10, va="bottom", ha="left", zorder=5,
bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"))
ax.text(xmax + 0.02, y, lab, color=c if c == ORANGE else BLUE, fontsize=10, va="center", ha="left",
clip_on=False)
ax.axvline(0, color=INK2, lw=0.9, zorder=2)
ax.set_xlim(xmin, xmax)
ax.set_ylim(-0.55, ys[0] + 0.9)
ax.set_yticks([])
ax.set_xticks(np.arange(-0.3, 0.51, 0.1))
ax.set_xticklabels([f"{t:.1f}" if abs(t) > 1e-9 else "0" for t in np.arange(-0.3, 0.51, 0.1)])
ax.grid(axis="x", color=GRID, lw=0.7, zorder=0)
for s in ("left", "right", "top"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(colors=INK2, labelsize=10)
ax.set_xlabel("Pooled Hedges g for suicide ideation, with 95% CI\n"
"← favors intervention favors control →", color=INK, fontsize=10.5)
ax.text(0.004, -0.50, "null (g = 0)", color=INK2, fontsize=10, ha="left", va="bottom")
fig.suptitle("The printed CI excludes 0; the stated method reproduces it within 0.01",
x=0.02, ha="left", color=INK, fontsize=11.5, fontweight="bold", y=0.985)
fig.text(0.02, 0.905, "Orange: value printed in the paper. Blue: recomputed from the three study rows in Figure 2A.",
color=INK2, fontsize=10, ha="left")
fig.subplots_adjust(left=0.045, right=0.71, top=0.86, bottom=0.17)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg and fig1.png")
Right anterior insula odds ratio of 0.96 lies outside its own reported 95% CI (0.01–0.75)
JAMA Psychiatry (2015) · doi:10.1001/jamapsychiatry.2014.1604 · PMC5103694
Summary
The paper's main predictive result, given in the abstract and in the Results, is that right anterior insula (AI) volume predicted a diagnosis of major depressive disorder (MDD) after imaging, with "odds ratio, 0.96; 95% CI, 0.01–0.75; P = .03". The point estimate 0.96 lies above the upper bound of its own 95% CI, which is impossible for any standard interval. The CI and the P value agree with each other: on the log-odds scale the CI is centred on an OR of about 0.087 (0.061–0.106 when the printed bounds are allowed to vary within their rounding), and that implies a Wald P of 0.026, which matches the reported .03. An OR of 0.96 with the same standard error would give P ≈ 0.97. The direction and statistical significance of the finding therefore stand, but the printed effect size understates the association the paper's own CI and P value imply, roughly a 91% rather than a 4% lower odds of later MDD per unit of whole-brain-adjusted volume.
What the paper reports
The abstract:
"When covarying for children’s lifetime history of MDD episodes, their experience of pathological guilt during preschool, as well as their sex and age at the time of imaging, schoolchildren’s right-side AI volume was a significant predictor of being diagnosed as having an MDD episode after imaging (odds ratio, 0.96; 95% CI, 0.01–0.75; P = .03)." (Abstract, Results)
The same model in the Results, together with the other estimates printed in the same paragraph:
"Children with an MDD diagnosis up to the time of imaging were approximately 11 times as likely as same-age peers without a prior diagnosis of MDD to have an MDD diagnosis after their imaging (odds ratio [OR], 11.38; 95% CI, 2.88–44.94; P < .01). However, even when including the robust effect of MDD up to the time of imaging, as well as children’s age, sex, and PO guilt symptoms (OR, 0.86; 95% CI, 0.28–2.65; P = .79), children with larger right-side AI volumes were significantly less likely to be diagnosed as having full MDD after their imaging (diagnosed on average 14 months after the imaging date [OR, 0.96; 95% CI, 0.01–0.75; P = .03]) (Figure 3). Left-side AI volume was not significantly associated with children’s MDD diagnosis after the time of imaging (OR, 2.70; 95% CI, 0.24–29.88; P = .42)." (Results, "Do AI Volumes Predict a Subsequent Depression Diagnosis?", para 1)
"Results indicated that smaller right AI volume remained a significant predictor (OR, 0.12; 95% CI, 0.01–0.99; P = .04) of DSM-5 MDD diagnosis after imaging when covarying for MDD from baseline up until the time of imaging, age, sex, PO guilt, and children’s stressful or traumatic life events." (Results, same paragraph, follow-up model with life-event covariates)
The model and the predictor scale:
"Binary logistic regression analysis was used to test whether schoolchildren’s right or left AI volume predicted the likelihood of being diagnosed as having MDD any time after their imaging." (Methods, Statistical Analysis, "AI Volume as a Predictor of Full MDD After Imaging")
"AI volumes were adjusted by the total segmented whole-brain volume (structure divided by whole-brain volume, times 1000) before all analyses." (Methods, "Magnetic Resonance Imaging Acquisition and AI Volume Analysis", para 2)
"Fourth was MDD diagnosed after the time of imaging (yes or no) (MDD after imaging variable [n = 24]). This variable is used as the dependent variable in the final analysis." (Methods, Measures, "DSM Psychiatric Diagnoses")
What the paper's own numbers show
Method. A logistic-regression Wald interval is symmetric on the log-odds scale. From a printed OR, lower bound L and upper bound U, the estimate implied by the interval is the geometric midpoint exp((ln L + ln U)/2), the standard error is SE = (ln U − ln L)/(2 × 1.96), and the two-sided P value is 2Φ(−|ln OR|/SE). This was applied to all five estimates in the Results paragraph. For the right AI estimate the printed bounds were also varied over every value that rounds to them (0.005–0.015 and 0.745–0.755). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Right AI OR, main model: is the OR inside its CI? | Abstract; Results para 1 | OR 0.96; 95% CI 0.01–0.75 | No: 0.96 > 0.75 |
| Right AI, OR implied by the printed CI | derived from Abstract / Results | 0.96 | 0.087 (0.061–0.106 allowing for rounding of the bounds) |
| Right AI, SE(log OR) implied by the CI | derived | — | 1.10 (1.00–1.28 allowing for rounding) |
| Right AI, P implied by the CI | Abstract; Results para 1 | P = .03 | 0.026 (z = −2.22; 0.024–0.029 allowing for rounding) |
| Right AI, P that OR 0.96 would have at that SE | derived | P = .03 | 0.97 (0.967–0.975) |
| Right AI, 95% CI that OR 0.96 would need for P = .03 | derived | 0.01–0.75 | 0.925–0.996 (SE 0.019) |
| Prior MDD | Results para 1 | OR 11.38; 2.88–44.94; P < .01 | midpoint 11.377; P = 0.001 (consistent) |
| PO guilt | Results para 1 | OR 0.86; 0.28–2.65; P = .79 | midpoint 0.861; P = 0.795 (consistent) |
| Left AI volume | Results para 1 | OR 2.70; 0.24–29.88; P = .42 | midpoint 2.678; P = 0.423 (consistent) |
| Right AI, + life-event covariates | Results para 1 | OR 0.12; 0.01–0.99; P = .04 | midpoint 0.099 (0.070–0.122 allowing for rounding); P = 0.048–0.050 (OR inside its CI; P close) |
In the four other estimates the printed OR sits at the log-midpoint of its CI to within rounding: for prior MDD, for example, ln(OR) − ln(L) = 1.374 and ln(U) − ln(OR) = 1.373. Only the right AI main-model estimate departs from this pattern: ln(0.96) − ln(0.01) = 4.564, while ln(0.75) − ln(0.96) = −0.247. The CI implies an SE of 1.10, close to the left AI estimate's 1.23 and the follow-up model's 1.17, so the interval's width is in line with the other volume estimates from the same 24 events. It looks asymmetric only because the printed OR lies outside it.
Figure
Figure 1. Each row is one estimate from the Results paragraph on MDD after imaging, on a single log-scale odds-ratio axis, with the thin vertical line at OR = 1. Orange circles and lines are the ORs and 95% CIs as printed in the paper. Open blue diamonds mark the geometric midpoint of each printed CI; for the bottom four estimates they coincide with, or sit close to, the printed OR. The top row is the right AI main-model estimate as printed: the OR (0.96) is to the right of its own interval (0.01–0.75). The second row shows the estimate implied by that interval, 0.087 (blue diamond; the printed CI is shown faded); the short thick blue bar spans 0.061–0.106, the values obtained when the printed bounds vary within their rounding. The third row shows the 95% CI (0.925–0.996, an open blue square with a short line hidden mostly behind it) that an OR of 0.96 would need in order to have P = .03. All recomputed values come from recompute.py.txt.
Figure 2. The blue curve gives the two-sided Wald P value for any right AI odds ratio, at the standard error (SE of log OR = 1.10) implied by the printed CI of 0.01–0.75. The reported P = .03 falls on the curve at OR ≈ 0.087 (filled blue diamond, P ≈ 0.026). At the printed OR of 0.96 the curve gives P ≈ 0.97 (open blue diamond). The orange circle marks the reported pair (OR 0.96, P = .03), which lies far off the curve: the printed CI and P are consistent with each other, and the printed OR is not consistent with either.
Why it matters
The right AI result is the third aim of the study (the Objectives ask "whether AI volumes predict subsequent likelihood of having an episode of MDD") and the only predictive result reported in the abstract. The Discussion builds on it: it says that "right-side AI volume was a significant and robust predictor of DSM-5 MDD diagnosis after imaging", that "these study findings provide evidence for the first structural brain biomarker to date of risk for recurrent depression in childhood", and that reduced AI volume "served as a statistically significant biomarker of later depression recurrence".
The direction and significance of the association are not affected. The printed CI excludes 1, the printed P is .03, the two agree, and the follow-up model with life-event covariates (OR 0.12; 0.01–0.99; P = .04) points the same way. What is wrong is the effect size, and the difference is large. Read at face value, 0.96 describes a 4% reduction in the odds of later MDD per unit of whole-brain-adjusted AI volume (structure ÷ whole-brain volume × 1000), and an estimate that close to 1 could not reach P = .03 with an interval this wide. The interval and P value instead imply an OR near 0.09, a 91% reduction per unit, though one that is very imprecisely estimated: the printed CI spans a 75-fold range and the outcome has 24 events. A reader who uses the printed OR would conclude that the predictive effect is negligible, or would find the result self-contradictory. The correct point estimate cannot be read from the paper; only the authors' regression output can supply it. Studies that pool or cite this estimate would carry the error forward.
Explanations considered
Typesetting or archiving artifact. Ruled out. The same "0.96; 95% CI, 0.01–0.75; P = .03" appears in the abstract and in the Results of the PMC author manuscript and of the version of record on jamanetwork.com. The number sits in running prose, so a lost minus sign, a misread mid-dot or a shifted table cell cannot explain it.
Different scale or units for the predictor. Ruled out. Rescaling AI volume (per unit vs per SD, or a different multiplier on the whole-brain adjustment) changes the OR and both CI bounds together. It cannot place the estimate outside its own interval.
A different model behind the OR than behind the CI. Not supported. The Methods describe a single binary logistic regression with age, sex, prior MDD, PO symptoms and AI volume, and the abstract and the Results attach the same OR, CI and P to the same covariate set.
Rounding. Ruled out. Allowing each printed bound to take any value that rounds to it moves the implied OR only between 0.061 and 0.106.
Profile-likelihood or other asymmetric interval. Ruled out as an explanation for an estimate outside the interval. Such intervals can sit off-centre around the estimate, but they always contain it. The other four estimates in the paragraph are also almost exactly log-symmetric around their printed ORs, which indicates Wald-type intervals.
One-sided test. Not relevant. A one-sided P value would change the P value but not whether the estimate lies inside its own CI.
A dropped digit (e.g. 0.096 printed as 0.96). Possible but unverified. 0.096 lies within the rounding range of the CI's midpoint (0.061–0.106) and gives P = 0.033 at the implied SE. The paper offers no way to confirm the exact value.
The claim that the CI is "implausibly wide". Not an independent error. The implied SE of 1.10 on the log scale is similar to the left AI estimate's (1.23) and the follow-up model's (1.17). This width is plausible for a continuous predictor with 24 events.
Side observations, not part of this finding. In the follow-up model the printed OR (0.12) lies inside its CI and close to the midpoint (0.099); the CI implies P ≈ 0.049 against a printed .04, a small difference that could reflect a different test statistic. Separately, the Table's group-comparison column prints several "OR" values with negative CI bounds (for example "OR, -0.71; 95% CI, -1.10 to 0.37" for female sex), which looks like a labelling or scale issue in that column. Neither affects the result above.
Status
As of 2026-10-23:
- Version of record. The jamanetwork.com full text (https://jamanetwork.com/journals/jamapsychiatry/fullarticle/1935483, reached through the DOI) was fetched on 2026-10-23. It prints "(odds ratio, 0.96; 95% CI, 0.01-0.75; P = .03)" in the abstract and "[OR, 0.96; 95% CI, 0.01-0.75; P = .03]" in the Results. An earlier audit stage found the neighbouring estimates (11.38, 0.86, 2.70 and 0.12, with their CIs and P values) identical to the PMC copy. The PMC copy (PMC5103694) is the NIH author manuscript and carries the same numbers.
- Supplements. The article has one supplement (eFigures 1–2 and eTables 1–2). According to the publisher's listing, it covers the imaging schedule, subgroup AI volume comparisons, subgroup characteristics and covariate effects on AI volume. None of these is described as holding the logistic regression of MDD after imaging, and direct downloads of the PDF from the publisher and from PMC did not return the file. No source data or analysis code is available.
- Corrections.
src/known.py(Crossref updates, PMC notices, Europe PMC notices) returned none. A Crossreffilter=updates:query for the DOI returned 0 results on 2026-10-23, and the Europe PMC record has no comment/correction links. No correction notice appears on the publisher page. - PubPeer. The PubPeer API returned HTTP 403 on 2026-10-23, so PubPeer could not be checked directly. A web search for the title with "pubpeer", "erratum" and "correction" found no thread or notice.
Sources: PMC5103694 (PubMed Central archive, read with src/show.py); https://jamanetwork.com/journals/jamapsychiatry/fullarticle/1935483; https://api.crossref.org/works?filter=updates:10.1001/jamapsychiatry.2014.1604; Europe PMC REST search for DOI:10.1001/jamapsychiatry.2014.1604.
Suggested correction
Replace the right-side AI volume odds ratio in the abstract and in the Results paragraph "Do AI Volumes Predict a Subsequent Depression Diagnosis?" with the value from the fitted logistic regression, checking that it lies within the reported 95% CI (0.01–0.75) and fits P = .03. The paper's own numbers place it near 0.09 (0.061–0.106). If the CI or P value was the term misreported instead, correct that term and state the corrected triplet in both places. The authors could also state the unit of AI volume for which the OR is expressed (per unit of structure ÷ whole-brain volume × 1000) so readers can interpret the magnitude. In the same correction they might review the "OR" entries with negative confidence bounds in the Table.
Recomputation script (python recompute.py.txt)
"""Recompute the numbers in the PMC5103694 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt
Inputs are the odds ratios, 95% CIs and P values printed in the paper
(Abstract; Results, "Do AI Volumes Predict a Subsequent Depression Diagnosis?", para 1).
Method: for a Wald interval on the log-odds scale, log(OR) is the midpoint of
log(lower) and log(upper), SE = (log(upper) - log(lower)) / (2 * 1.959964),
and the two-sided P = 2 * (1 - Phi(|log OR| / SE)).
"""
import os
import itertools
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.isf(0.025) # 1.959964
# label, reported OR, lower, upper, reported P (as printed)
ROWS = [
("Prior MDD (main model)", 11.38, 2.88, 44.94, "<.01"),
("PO guilt (main model)", 0.86, 0.28, 2.65, ".79"),
("Right AI volume (main model)", 0.96, 0.01, 0.75, ".03"),
("Left AI volume", 2.70, 0.24, 29.88, ".42"),
("Right AI volume (+ life events)", 0.12, 0.01, 0.99, ".04"),
]
def from_ci(lo, hi):
a, b = np.log(lo), np.log(hi)
se = (b - a) / (2 * Z)
mid = (a + b) / 2
return np.exp(mid), se, 2 * norm.sf(abs(mid) / se)
def wald_p(or_, se):
return 2 * norm.sf(abs(np.log(or_)) / se)
def half_unit(x):
"""Interval of values that round to x at the printed number of decimals (2 dp)."""
return x - 0.005, x + 0.005
print("== 1. Each reported OR against the CI it is printed with ==")
print(f"{'quantity':34s} {'OR':>6s} {'95% CI':>13s} {'OR in CI':>8s} {'geo-mid':>8s} "
f"{'SE(logOR)':>9s} {'P(mid)':>7s} {'P(rep OR)':>9s} {'rep P':>5s}")
res = {}
for lab, o, l, u, p in ROWS:
mid, se, pm = from_ci(l, u)
pr = wald_p(o, se)
res[lab] = dict(o=o, l=l, u=u, mid=mid, se=se, pm=pm, pr=pr, p=p)
print(f"{lab:34s} {o:6.2f} {l:5.2f}-{u:<7.2f} {str(l <= o <= u):>8s} {mid:8.3f} "
f"{se:9.3f} {pm:7.3f} {pr:9.3f} {p:>5s}")
print(f"{'':34s} log(OR) - log(lower) = {np.log(o)-np.log(l):6.3f}; "
f"log(upper) - log(OR) = {np.log(u)-np.log(o):6.3f}")
print("\n== 2. Right AI (main model): allow for rounding of the printed CI bounds ==")
# 0.01 could be anything in [0.005, 0.015); 0.75 in [0.745, 0.755)
lo_r, hi_r = half_unit(0.01), half_unit(0.75)
mids, ps, ses, p096 = [], [], [], []
for a, b in itertools.product(np.linspace(*lo_r, 201), np.linspace(*hi_r, 21)):
m, s, pm = from_ci(a, b)
mids.append(m); ps.append(pm); ses.append(s); p096.append(wald_p(0.96, s))
print(f"implied OR (geometric midpoint): {min(mids):.3f} to {max(mids):.3f}")
print(f"implied SE(log OR): {min(ses):.3f} to {max(ses):.3f}")
print(f"implied Wald P: {min(ps):.3f} to {max(ps):.3f} (reported P = .03)")
print(f"P for OR = 0.96 at those SEs: {min(p096):.3f} to {max(p096):.3f}")
print("\n== 3. CI that OR = 0.96 would need for a two-sided P = .03 ==")
z03 = norm.isf(0.03 / 2)
se_req = abs(np.log(0.96)) / z03
ci_req = np.exp(np.log(0.96) + np.array([-Z, Z]) * se_req)
print(f"z = {z03:.3f}; SE(log OR) = {se_req:.4f}; 95% CI = {ci_req[0]:.3f} to {ci_req[1]:.3f}")
# P range 0.025-0.035 (values that round to .03)
for pp in (0.025, 0.035):
s = abs(np.log(0.96)) / norm.isf(pp / 2)
c = np.exp(np.log(0.96) + np.array([-Z, Z]) * s)
print(f" if P = {pp:.3f}: 95% CI = {c[0]:.3f} to {c[1]:.3f}")
r = res["Right AI volume (main model)"]
print("\n== 4. Summary values used in the write-up ==")
print(f"Right AI main: geometric midpoint {r['mid']:.3f}, SE {r['se']:.2f}, z {np.log(r['mid'])/r['se']:.2f}, "
f"P {r['pm']:.3f}; P if OR were 0.96: {r['pr']:.2f}")
print(f"Implied odds reduction per unit volume: {100*(1-r['mid']):.0f}% (vs {100*(1-0.96):.0f}% for OR 0.96)")
print(f"Width of reported CI, upper/lower ratio: {0.75/0.01:.0f}-fold")
lr = res["Left AI volume"]
print(f"SE comparison: right AI {r['se']:.2f}, left AI {lr['se']:.2f}, right AI + life events "
f"{res['Right AI volume (+ life events)']['se']:.2f}")
print(f"Candidate 0.096 within rounding range of implied OR: {min(mids) <= 0.096 <= max(mids)}; "
f"P at OR 0.096, SE {r['se']:.3f}: {wald_p(0.096, r['se']):.3f}")
# life-events model rounding
mids2, ps2 = [], []
for a, b in itertools.product(np.linspace(*half_unit(0.01), 201), np.linspace(*half_unit(0.99), 21)):
m, s, pm = from_ci(a, b); mids2.append(m); ps2.append(pm)
print(f"Right AI + life events: implied OR {min(mids2):.3f}-{max(mids2):.3f}, implied P "
f"{min(ps2):.3f}-{max(ps2):.3f} (reported OR 0.12, P = .04)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e3e2de"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: dot-and-interval plot, one log axis
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
ax.set_facecolor(BG)
rows = [ # (y label, kind)
("Right AI volume, main model\n(as printed)", "rep_main"),
("Right AI volume, main model\n(OR implied by its printed CI)", "mid_main"),
("Right AI volume, main model\n(CI needed if OR = 0.96, P = .03)", "req"),
("Right AI volume + life events", "other:Right AI volume (+ life events)"),
("Left AI volume", "other:Left AI volume"),
("PO guilt", "other:PO guilt (main model)"),
("Prior MDD", "other:Prior MDD (main model)"),
]
n = len(rows)
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.text(1.0, n - 0.35, " OR = 1 (no association)", color=SEC, fontsize=9.5, va="center", ha="left")
for i, (lab, kind) in enumerate(rows):
y = n - 1 - i
if kind == "rep_main":
ax.plot([0.01, 0.75], [y, y], color=ORA, lw=2, zorder=2)
ax.plot(0.96, y, "o", ms=9, color=ORA, zorder=3)
ax.annotate("OR 0.96 printed outside\nits own CI 0.01–0.75", (0.96, y), xytext=(2.2, y),
color=ORA, fontsize=9.5, va="center",
arrowprops=dict(arrowstyle="-", color=ORA, lw=1))
elif kind == "mid_main":
ax.plot([0.01, 0.75], [y, y], color=ORA, lw=2, alpha=0.35, zorder=2)
ax.plot([min(mids), max(mids)], [y, y], color=BLU, lw=5, solid_capstyle="butt", zorder=3)
ax.plot(r["mid"], y, "D", ms=9, color=BLU, mfc=BG, mew=2, zorder=4)
ax.annotate(f"≈{r['mid']:.3f} (range {min(mids):.3f}–{max(mids):.3f}\nwith bound rounding); P ≈ {r['pm']:.3f}",
(1.12, y), color=BLU, fontsize=9.5, va="center")
elif kind == "req":
ax.plot(ci_req, [y, y], color=BLU, lw=2, zorder=2)
ax.plot(0.96, y, "s", ms=9, color=BLU, mfc=BG, mew=2, zorder=3)
ax.annotate(f"{ci_req[0]:.3f}–{ci_req[1]:.3f}", (ci_req[1], y), xytext=(2.2, y),
color=BLU, fontsize=9.5, va="center",
arrowprops=dict(arrowstyle="-", color=BLU, lw=1))
else:
d = res[kind.split(":", 1)[1]]
ax.plot([d["l"], d["u"]], [y, y], color=ORA, lw=2, zorder=2)
ax.plot(d["o"], y, "o", ms=9, color=ORA, zorder=3)
ax.plot(d["mid"], y, "D", ms=9, color=BLU, mfc="none", mew=1.6, zorder=4)
ax.set_yticks(range(n))
ax.set_yticklabels([lab for lab, _ in rows][::-1], fontsize=10)
ax.set_xscale("log")
ax.set_xlim(0.004, 120)
ax.set_ylim(-0.6, n - 0.1)
ax.set_xticks([0.01, 0.1, 1, 10, 100])
ax.set_xticklabels(["0.01", "0.1", "1", "10", "100"])
ax.set_xlabel("Odds ratio for MDD diagnosis after imaging (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORA, marker="o", ms=8, lw=2, label="Reported in the paper (OR and 95% CI)"),
Line2D([], [], color=BLU, marker="D", ms=8, mfc=BG, mew=2, lw=0,
label="Recomputed: geometric midpoint of the printed CI (thick bar: rounding range)"),
Line2D([], [], color=BLU, marker="s", ms=8, mfc=BG, mew=2, lw=2,
label="Recomputed: 95% CI implied by OR 0.96 with P = .03")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.42, -0.13), ncol=1, frameon=False, fontsize=9.5)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: Wald P as a function of the OR, at the SE implied by the printed CI
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
ax.set_facecolor(BG)
grid = np.exp(np.linspace(np.log(0.003), np.log(10), 600))
ax.plot(grid, wald_p(grid, r["se"]), color=BLU, lw=2, zorder=2,
label=f"Recomputed: Wald P for any OR, SE(log OR) = {r['se']:.2f} from the printed CI")
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.plot(0.96, 0.03, "o", ms=10, color=ORA, zorder=4, label="Reported pair: OR 0.96, P = .03")
ax.plot(r["mid"], r["pm"], "D", ms=9, color=BLU, zorder=4)
ax.plot(0.96, r["pr"], "D", ms=9, color=BLU, mfc=BG, mew=2, zorder=4)
ax.annotate(f"OR ≈ {r['mid']:.3f} gives P ≈ {r['pm']:.3f}", (r["mid"], r["pm"]), xytext=(0.006, 0.22),
color=BLU, fontsize=9.5, arrowprops=dict(arrowstyle="-", color=BLU, lw=1))
ax.annotate(f"OR 0.96 gives P ≈ {r['pr']:.2f}", (0.96, r["pr"]), xytext=(0.5, 0.90),
color=BLU, fontsize=9.5, ha="right", va="center",
arrowprops=dict(arrowstyle="-", color=BLU, lw=1))
ax.annotate("printed:\nOR 0.96, P = .03", (0.96, 0.03), xytext=(1.3, 0.10), color=ORA, fontsize=9.5,
arrowprops=dict(arrowstyle="-", color=ORA, lw=1))
ax.set_xscale("log")
ax.set_xlim(0.003, 10)
ax.set_ylim(0, 1.05)
ax.set_xticks([0.01, 0.1, 1, 10])
ax.set_xticklabels(["0.01", "0.1", "1", "10"])
ax.set_xlabel("Odds ratio for right AI volume (log scale)")
ax.set_ylabel("Two-sided P value")
ax.grid(color=GRID, lw=0.8, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=1, frameon=False, fontsize=9.5)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written to", HERE)
Hazard ratio for symptomatic COVID-19 (0.011) lies outside its own 95% confidence interval (0.063–0.198)
Kidney Int (2022) · doi:10.1016/j.kint.2022.07.008 · PMC9297679
Summary
This research letter compares 333 kidney transplant recipients (KTRs) who received tixagevimab–cilgavimab preexposure prophylaxis with 97 who did not. For its first outcome, symptomatic COVID-19, it reports a hazard ratio of 0.011 with a 95% CI of 0.063–0.198. A point estimate cannot lie below its own confidence interval. The geometric centre of the interval is 0.112, about ten times the printed value. The paper's other three HRs each sit at the centre of their intervals (ratios 0.997 to 1.027), which points to a dropped digit, with 0.011 printed for about 0.11. Direction and significance do not change. The printed value does overstate the effect size for the key outcome: it implies about a 99% lower hazard, while the interval implies 80–94%, about 89% at its centre.
What the paper reports
Main text, third paragraph (the letter has no abstract and no separate Results heading):
Among the 333 KTRs who received tixagevimab–cilgavimab, 12.3% developed symptomatic COVID-19 compared with 43.3% among those who did not (hazard ratio [HR], 0.011; 95% confidence interval [CI], 0.063–0.198; P < 0.001; Figure 1a).
The same paragraph, for the other three outcomes:
(HR, 0.046; 95% CI, 0.013–0.158; P < 0.001; Figure 1b)
(HR, 0.045; 95% CI, 0.008–0.240; P < 0.001; Figure 1c)
(HR, 0.076; 95% CI, 0.005–1.161; P = 0.066; Figure 1d)
Table 1, row "Symptomatic COVID-19" (columns: treated, n = 333 | untreated, n = 97; n (%)):
Symptomatic COVID-19 | 41 (12.3) | 42 (43.3)
Figure 1 legend:
Log-rank test was used to compare the 2 groups and was considered as significant when P < 0.05.
Main text, fourth paragraph:
The first conclusion of this real-life study is that tixagevimab–cilgavimab use was associated with (i) a significantly lower risk of symptomatic COVID-19 and (ii) fewer COVID-19–related hospitalizations (including intensive care unit), compared with patients who did not receive this prophylactic treatment.
What the paper's own numbers show
Method. A 95% CI for a hazard ratio, whether Wald-type from a Cox model or from a log-rank-based estimate, is built on the log scale: log HR ± 1.96 × SE. The estimate therefore lies inside the interval, at its geometric centre √(lower × upper). I computed that centre for each of the four reported intervals and compared it with the printed HR. I also took the SE of log HR implied by each interval's width, used it to recompute a two-sided P value, and checked the event percentages against the counts in Table 1. The paper does not say how the HRs were estimated: the Figure 1 legend names only the log-rank test. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| HR, symptomatic COVID-19 | Text, para. 3 | 0.011 (95% CI 0.063–0.198) | outside its CI; CI centre = 0.1117 |
| Printed HR ÷ CI centre, symptomatic | — | — | 0.098 (the centre is 10.2 × the printed HR) |
| Rounding check, symptomatic | — | 0.011 vs lower bound 0.063 | largest value shown as 0.011 is 0.0115; smallest shown as 0.063 is 0.0625 (5.4× apart) |
| Distance of 0.011 below the lower bound | — | — | 1.75 log units; the CI half-width is 0.57 log units |
| 95% CI that HR 0.011 would have with the same SE | — | — | 0.0062–0.0195 |
| HR, hospitalization | Text, para. 3 | 0.046 (0.013–0.158) | inside; centre 0.0453; ratio 1.015 |
| HR, ICU hospitalization | Text, para. 3 | 0.045 (0.008–0.240) | inside; centre 0.0438; ratio 1.027 |
| HR, death | Text, para. 3 | 0.076 (0.005–1.161) | inside; centre 0.0762; ratio 0.997 |
| P, symptomatic | Text, para. 3 | < 0.001 | 6.2 × 10⁻¹⁴ from the CI (z = −7.50); < 0.001 either way |
| P, hospitalization / ICU | Text, para. 3 | < 0.001 / < 0.001 | 1.2 × 10⁻⁶ / 0.00031 |
| P, death | Text, para. 3; Figure 1d panel | 0.066 (text); "Log-rank: P = 0.064" (figure) | 0.064 |
| Symptomatic COVID-19, treated / untreated | Table 1; text | 41 (12.3%) / 42 (43.3%) | 41/333 = 12.31%; 42/97 = 43.30% |
| Hospitalization | Table 1; text | 4 (1.2) / 11 (11.3) | 1.20% / 11.34% |
| ICU hospitalization | Table 1; text | 2 (0.6) / 6 (6.2) | 0.60% / 6.19% |
| Death | Table 1; text | 1 (0.3) / 2 (2) | 0.30% / 2.06% |
| Crude risk ratio, symptomatic | from Table 1 | — | 0.284 |
| HR ÷ crude risk ratio | from text and Table 1 | — | hospitalization 0.43, ICU 0.46, death 0.52; symptomatic 0.39 at the CI centre, 0.039 for the printed HR |
| Hazard reduction, symptomatic | — | 99% implied by HR 0.011 | 89% at the CI centre; 80% to 94% across the CI |
The counts and percentages are all correct, and the hospitalization, ICU and death HRs agree with their intervals to within the three-decimal rounding of the printed bounds. Only the symptomatic HR disagrees with its interval. Its interval width (SE of log HR 0.292) is the narrowest of the four, as expected for the outcome with the most events (83). A scale check that does not rely on the CI points the same way. For the three consistent outcomes, the HR is 0.43 to 0.52 times the crude risk ratio. At 0.112 the symptomatic HR would be 0.39 times its crude risk ratio, in line with the others. At 0.011 it would be 0.039 times.
Figure
Figure 1. The four hazard ratios reported in the letter, on one log-scaled axis. Orange dots are the HRs printed in the text and orange bars are the 95% CIs printed with them. Open blue diamonds mark the geometric centre of each printed CI, recomputed by recompute.py.txt. That centre is where a log-scale interval places its own estimate. The right-hand column gives the printed HR divided by that centre. For hospitalization, ICU hospitalization and death, the dot and diamond coincide (ratio 1.00 to 1.03). For symptomatic COVID-19 (shaded row), the printed 0.011 lies outside its interval and is 0.10 times the centre of 0.112. The thin vertical line marks HR = 1 (no difference). All values come from the paper's text or from recompute.py.txt.
Why it matters
- It is the key result. Symptomatic COVID-19 is the first outcome reported and the basis of the letter's first conclusion ("a significantly lower risk of symptomatic COVID-19"). The paragraph quoted above is the only place where the effect size is given. The paper has no abstract or results table that states it differently.
- The effect size is overstated about tenfold. HR 0.011 implies about a 99% lower hazard, close to complete protection. The reported interval implies 80–94% (about 89% at its centre). The printed figure also makes symptomatic infection look like the outcome most strongly prevented, ahead of hospitalization and ICU admission (HR about 0.045). The interval says the reverse: symptomatic infection was the least reduced of the morbidity outcomes. That fits the letter's own remark that the dose "offers an incomplete protection", with 12.3% of treated patients still developing symptomatic disease.
- Direction and significance do not change. The whole interval lies below 1, and the P value implied by the interval is far below 0.001. The qualitative conclusions hold, and so do the hospitalization, ICU and death results.
- Readers who extract effect sizes. Meta-analysts and guideline authors who take the point estimate from the text will get a value about ten times too extreme. Those who rebuild the log HR and SE from the CI will get the intended value, so the two routes give different answers. Any corrected estimate still comes from a retrospective, non-randomized comparison with an untreated group that differed at baseline: for example, cardiovascular events in the last 3 months were 0% in the treated group and 10.3% in the untreated group (Table 1).
Explanations considered
- Rounding. Cannot explain it. Any value that displays as 0.011 is at most 0.0115. Any lower bound that displays as 0.063 is at least 0.0625.
- An asymmetric or profile-likelihood interval. Cannot explain it. Such intervals can be off-centre, but they still contain their estimate. Here 0.011 lies 1.75 log units below the lower bound, three times the interval's half-width.
- The CI belongs to a different estimate (for example an adjusted model) or a different outcome. Not supported. The letter describes only one analysis per outcome and no adjusted model. Even an adjusted HR would have to lie inside its own interval. Figure 1a shows only Kaplan–Meier curves and "Log-rank: P < 0.001", with no HR or CI to compare against.
- A reporting convention specific to this paper. Ruled out by the paper's own numbers. The other three HR–CI pairs in the same paragraph sit at the geometric centres of their intervals (ratios 0.997 to 1.027). The death P recomputed from its interval (0.064) matches the log-rank P printed in Figure 1d. So these are ordinary log-scale intervals, and the symptomatic interval centres on 0.112.
- Conversion or typesetting in the archive. Not likely. The same sentence appears in the PMC rendering, in the Europe PMC full-text XML and in the local archived copy. All derive from the publisher's deposit of the final article (Kidney Int 102(4):936–938). The error is a digit, not a symbol or superscript that conversion might lose.
- Which number is wrong. The paper's own numbers favour the point estimate as the misprint. An HR of about 0.11 fits the interval, the pattern of the other three outcomes and the HR-to-crude-risk-ratio scale check. The exact intended value (0.110, 0.111 or 0.112) cannot be recovered from the published data. A misprinted interval cannot be excluded either, but it would require both bounds to be wrong while still fitting the scale check.
- Side inconsistencies (minor, not part of the main finding). (1) For death, the text gives P = 0.066 and the Figure 1d panel gives "Log-rank: P = 0.064". The value recomputed from the HR and CI is 0.064. The two figures may come from different tests, and the difference changes nothing. (2) The reasons for non-treatment are given as percentages of all 430 KTRs. Two are rounded (2.33% → 2.3, 11.86% → 11.9) and two are rounded down (4.88% → 4.8 for refusal, 3.49% → 3.4 for other reasons). The counts sum to 97, as stated.
Status
- Version of record. The quoted sentence, with HR 0.011 and CI 0.063–0.198, is identical in the PMC article page, the Europe PMC full-text XML and the archived text read with
src/show.py. That PMC copy is the publisher's deposit of the final article: Kidney Int 102(4):936–938, PMID 35870641. Search-engine snippets of the article show the same numbers. The publisher's own pages could not be opened by automated requests: doi.org led to a linkinghub.elsevier.com redirect stub, kidney-international.org and ScienceDirect returned HTTP 403, and the PMC PDF returned 403. Every quotation above was checked verbatim against the PMC-archived text withsrc/show.py --grep. The "Log-rank: P = 0.064" text was read from the Figure 1 image (gr1_lrg.jpg) archived in PMC. - Supplements. None. The PMC page and Europe PMC XML list no supplementary material, and the data statement reads "All relevant data are within the article." Figure 1 gives no HR or CI, so it cannot tell whether the HR or the interval is the misprint.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref filterupdates:10.1016/j.kint.2022.07.008finds no items. The Europe PMC record lists only a "Comment on" link to PMID 35618094, the earlier study the letter responds to, and no erratum. PubPeer's API refused automated requests (human verification required, HTTP 403). A web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC9297679 and its Figure 1 image; Europe PMC full-text XML and record for PMC9297679; Crossref; PubMed/Europe PMC (PMID 35870641); PubPeer; https://doi.org/10.1016/j.kint.2022.07.008.
Suggested correction
In the third paragraph, the HR for symptomatic COVID-19 should be checked against the original analysis output and the sentence corrected to read
(hazard ratio [HR], 0.11x; 95% confidence interval [CI], 0.063–0.198; P < 0.001; Figure 1a)
where 0.11x is the estimate from the model that produced the interval. The interval implies a value close to 0.112. If the interval is the misprinted part instead, both bounds should be corrected. The authors may also wish to state the method used to estimate the HRs, since the Figure 1 legend names only the log-rank test. They could also make the P value for death the same in the text (0.066) and in Figure 1d (0.064).
Recomputation script (python recompute.py.txt)
# Recomputation for PMC9297679 (Kidney Int 2022; doi:10.1016/j.kint.2022.07.008)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and writes fig1.svg / fig1.png
# into the folder that holds this script. Inputs are copied from the paper's text and Table 1.
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import matplotlib.transforms
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ---- Inputs from the paper -------------------------------------------------
N_T, N_C = 333, 97 # treated / untreated (text, Table 1)
# outcome: (events treated, events untreated, % reported T, % reported C,
# HR, CI low, CI high, P reported) -- text paragraph 3, Table 1
OUT = {
"Symptomatic COVID-19": (41, 42, "12.3", "43.3", 0.011, 0.063, 0.198, "<0.001"),
"Hospitalization": (4, 11, "1.2", "11.3", 0.046, 0.013, 0.158, "<0.001"),
"ICU hospitalization": (2, 6, "0.6", "6.2", 0.045, 0.008, 0.240, "<0.001"),
"Death from COVID-19": (1, 2, "0.3", "2", 0.076, 0.005, 1.161, "0.066"),
}
P_DEATH_FIG1D = 0.064 # "Log-rank: P = 0.064" printed in Figure 1d panel
REASONS = [("refusal", 21, "4.8"), ("recent cardiovascular events", 10, "2.3"),
("inability to plan injection", 51, "11.9"), ("other reasons", 15, "3.4")]
print("== 1. Event percentages (Table 1, text)")
for k, (eT, eC, rT, rC, *_ ) in OUT.items():
print(f" {k:22s} treated {eT}/{N_T} = {100*eT/N_T:.2f}% (reported {rT}) "
f"untreated {eC}/{N_C} = {100*eC/N_C:.2f}% (reported {rC})")
print("\n== 2. Each HR against its own 95% CI (log-scale Wald interval)")
rows = {}
for k, (eT, eC, rT, rC, hr, lo, hi, prep) in OUT.items():
centre = math.sqrt(lo * hi) # geometric centre of the CI
se = (math.log(hi) - math.log(lo)) / (2 * Z) # SE of log HR implied by the CI
z_c = math.log(centre) / se
p_c = 2 * norm.sf(abs(z_c))
z_r = math.log(hr) / se
p_r = 2 * norm.sf(abs(z_r))
rr = (eT / N_T) / (eC / N_C)
rows[k] = dict(hr=hr, lo=lo, hi=hi, centre=centre, se=se, rr=rr)
print(f" {k:22s} HR {hr} CI {lo}-{hi} inside CI: {lo <= hr <= hi}")
print(f" CI geometric centre = {centre:.4f}; HR / centre = {hr/centre:.3f}; "
f"SE(log HR) = {se:.3f}")
print(f" log distance HR-lower = {math.log(hr)-math.log(lo):+.3f}, "
f"upper-HR = {math.log(hi)-math.log(hr):+.3f}")
print(f" P from CI centre: z = {z_c:.2f}, P = {p_c:.2g} (reported P {prep}); "
f"P if HR were {hr} with this SE: {p_r:.2g}")
print(f" crude risk ratio = {rr:.3f}; HR / crude RR = {hr/rr:.3f}; "
f"CI centre / crude RR = {centre/rr:.3f}")
s = rows["Symptomatic COVID-19"]
print("\n== 3. Symptomatic COVID-19: size of the gap")
print(f" CI centre / printed HR = {s['centre']/s['hr']:.1f}")
print(f" printed HR / CI centre = {s['hr']/s['centre']:.3f}")
print(f" largest value that displays as 0.011 = 0.0115; smallest that displays as 0.063 = 0.0625;"
f" ratio = {0.0625/0.0115:.2f}")
print(f" printed HR lies {math.log(s['lo'])-math.log(s['hr']):.2f} log units below the lower bound; "
f"the CI half-width is {(math.log(s['hi'])-math.log(s['lo']))/2:.2f} log units")
print(f" 95% CI that would accompany HR 0.011 with the same SE: "
f"{0.011*math.exp(-Z*s['se']):.4f}-{0.011*math.exp(Z*s['se']):.4f}")
for cand in (0.011, 0.11, 0.111, 0.112):
print(f" candidate HR {cand}: inside CI {s['lo'] <= cand <= s['hi']}, "
f"ratio to CI centre {cand/s['centre']:.3f}, hazard reduction {100*(1-cand):.0f}%")
print(f" hazard reduction across the CI: {100*(1-s['hi']):.0f}% to {100*(1-s['lo']):.0f}%")
others = [rows[k]["hr"] / rows[k]["rr"] for k in OUT if k != "Symptomatic COVID-19"]
print(f" HR / crude RR, other three outcomes: {min(others):.2f} to {max(others):.2f}; "
f"symptomatic: CI centre {s['centre']/s['rr']:.2f}, printed {s['hr']/s['rr']:.3f}")
print("\n== 4. Death: P in text vs Figure 1d vs P implied by the CI")
d = rows["Death from COVID-19"]
p_d = 2 * norm.sf(abs(math.log(d["hr"]) / d["se"]))
print(f" text P = 0.066; Figure 1d log-rank P = {P_DEATH_FIG1D}; P from HR 0.076 and CI = {p_d:.3f}")
print("\n== 5. Reasons for no treatment, as % of all 430 KTRs")
for name, n, rep in REASONS:
pct = 100 * n / (N_T + N_C)
print(f" {name:30s} {n}/430 = {pct:.2f}% -> rounds to {pct:.1f} (reported {rep}); "
f"{100*n/N_C:.1f}% of the 97 untreated")
print(f" sum of reasons = {sum(n for _, n, _ in REASONS)} (untreated n = {N_C}); total = {N_T+N_C}")
# ---- Figure 1 --------------------------------------------------------------
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, BAND = "#eb6834", "#2a78d6", "#e4e3df", "#fbe9e1"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK, "svg.fonttype": "none"})
labels = list(OUT)
fig, ax = plt.subplots(figsize=(7.2, 4.3))
ys = [3.2, 2, 1, 0]
ax.axhspan(2.55, 4.0, color=BAND, zorder=0, lw=0)
ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
ax.text(0.93, -0.55, "HR = 1 (no difference)", ha="right", va="center", fontsize=10, color=INK2)
col = matplotlib.transforms.blended_transform_factory(ax.transAxes, ax.transData)
for y, k in zip(ys, labels):
r = rows[k]
ax.plot([r["lo"], r["hi"]], [y, y], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
for b in (r["lo"], r["hi"]):
ax.plot([b, b], [y - 0.1, y + 0.1], color=ORANGE, lw=2, zorder=2)
ax.plot(r["hr"], y, "o", ms=9, color=ORANGE, zorder=4)
ax.plot(r["centre"], y, "D", ms=9, mfc="none", mec=BLUE, mew=2, zorder=5)
sym = k == "Symptomatic COVID-19"
ax.text(1.04, y, f"{r['hr']/r['centre']:.2f}", transform=col, ha="left", va="center",
fontsize=10, color=ORANGE if sym else INK, weight="bold" if sym else "normal")
ax.text(1.04, 4.15, "HR ÷\nCI centre", transform=col, ha="left", va="bottom", fontsize=10,
color=INK2)
ax.annotate("Printed HR 0.011 lies\nbelow its own 95% CI",
xy=(0.011, 3.2), xytext=(0.0034, 3.62), fontsize=10, color=INK, ha="left",
va="center", arrowprops=dict(arrowstyle="-", color=INK2, lw=1, shrinkB=6))
ax.annotate(f"CI centre {s['centre']:.3f}\n= {s['centre']/s['hr']:.1f} × printed HR",
xy=(s["centre"], 3.2), xytext=(0.2, 3.62), fontsize=10, color=BLUE, ha="left",
va="center", arrowprops=dict(arrowstyle="-", color=BLUE, lw=1, shrinkB=7))
ax.set_xscale("log")
ax.set_xlim(0.003, 2.0)
ax.set_ylim(-0.8, 4.1)
ticks = [0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels(labels)
ax.set_xlabel("Hazard ratio, tixagevimab–cilgavimab vs none (log scale)", color=INK)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
handles = [
Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0, label="Reported HR"),
Line2D([], [], color=ORANGE, lw=2, label="Reported 95% CI"),
Line2D([], [], color=BLUE, marker="D", ms=9, mfc="none", mew=2, lw=0,
label="CI centre (recomputed)"),
]
fig.legend(handles=handles, loc="upper center", ncol=3, frameon=False, fontsize=10,
bbox_to_anchor=(0.5, 1.0), handletextpad=0.4, columnspacing=1.2)
fig.subplots_adjust(left=0.25, right=0.87, top=0.86, bottom=0.14)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
print("\nwrote fig1.svg, fig1.png in", HERE)
Abstract reports group 2's 1-year vaccine efficacy, 77% (67–84), as the value for group 1
Lancet (2021) · doi:10.1016/S0140-6736(21)00943-0 · PMC8121760
Summary
The abstract of this phase 2b trial of the R21/Matrix-M malaria vaccine has one 1-year efficacy figure: "77% (67–84) in group 1", where group 1 is the lower adjuvant dose (25 μg Matrix-M). Results paragraph 3 (unadjusted) and Table 1 (unadjusted and adjusted) give group 1 a 12-month efficacy of 71% (59–79). The value 77% (67–84) is group 2's 12-month estimate (50 μg Matrix-M), matching to the point estimate and both confidence limits, and no group-1 estimate anywhere in the paper equals it. The paper reads efficacy against a WHO goal of at least 75%. Read alone, the abstract places the lower-dose arm above that goal at 1 year and has its efficacy rising from 74% at 6 months, when the paper's own data show it below the goal and falling.
What the paper reports
Abstract, Findings:
Vaccine efficacy was 74% (95% CI 63–82) in group 1 and 77% (67–84) in group 2 at 6 months. At 1 year, vaccine efficacy remained high, at 77% (67–84) in group 1.
Abstract, Methods (the group definitions):
Group 1 received 5 μg R21 plus 25 μg MM, group 2 received 5 μg R21 plus 50 μg MM, and group 3, the control group, received rabies vaccinations.
Results, second paragraph (6 months, primary endpoint):
These cases of clinical malaria occurred in 43 (29%) of 146 participants in group 1, 38 (26%) of 146 participants in group 2, and 105 (71%) of 147 participants in group 3. A Cox regression model comparing group 1 with group 3 resulted in vaccine efficacy of 74% (95% CI 63–82; p<0·0001). Comparing group 2 with group 3 resulted in 77% efficacy (67–84; p<0·0001; figure 2).
Results, third paragraph (12 months):
These cases occurred in 50 (34%) of 146 participants in group 1, 39 (27%) of 146 participants in group 2, and 106 (72%) of 147 participants in group 3. Cox regression showed vaccine efficacy of 71% (95% CI 59–79; p<0·0001) for group 1, and 77% (67–84; p<0·0001) for group 2 (figure 2).
Table 1, primary case definition (>5000 parasites per μL), columns: timepoint | cases | unadjusted efficacy | adjusted efficacy:
Group 1 … 12 | 50/146 (34%) | 71% (59–79) | 71% (59–79)
Group 2 … 12 | 39/146 (27%) | 77% (67–84) | 76% (65–84)
Discussion, first paragraph (reference number after "75% or more" omitted):
We report malaria vaccine efficacy that reaches the WHO-specified efficacy goal of 75% or more in the target population of African children, over 12 months of follow-up. Efficacy of 77% was observed in the children who received R21 plus the higher dose of MM adjuvant
Discussion, seventh paragraph:
Efficacy with R21 and the higher adjuvant MM dose, administered largely before the malaria season, was 77% (95% CI 67–84), compared with 44% (37–50) reported for RTS,S/AS01 at this site
Research in context, Added value of this study:
This new vaccine, R21 adjuvanted with 50 μg Matrix-M (R21/MM), administered before the malaria season, demonstrates high-level efficacy, reaching the WHO-specified efficacy goal of at least 75% in the target population of African children over 1 year.
What the paper's own numbers show
Method. Table 1 states that efficacy is 1 minus the Cox hazard ratio. I made three checks, all printed by recompute.py.txt. (1) I compared the abstract's triplet, 77% (67–84), with every group-1 estimate printed in the paper and with the group-2 estimates. (2) Cox estimates cannot be reproduced exactly without event times, so as an independent check I approximated each efficacy from the printed case counts as 1 minus the ratio of cumulative hazards, with H = −ln(1 − cases/N) and SE(log HR) ≈ √(1/d₁ + 1/d₂). This approximation runs 1.1 to 3.8 points below Table 1 in all four cells, so it is useful for direction and size, not for the exact value. (3) I counted the new first cases between 6 and 12 months among the children still case-free at 6 months.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Group 1, 12 months | Abstract | 77% (67–84) | does not match any group-1 estimate in the paper |
| Group 1, 12 months, unadjusted | Results para 3; Table 1 | 71% (59–79) | HR 0.29 (0.21–0.41); count-based approximation 67.2% (54–77) |
| Group 1, 12 months, adjusted | Table 1 | 71% (59–79) | not 77% (67–84) |
| Group 2, 12 months, unadjusted | Results para 3; Table 1; Discussion | 77% (67–84) | identical to the abstract's "group 1" triplet; HR 0.23 (0.16–0.33); approximation 75.7% (65–83) |
| Group 1, 6 months | Abstract; Results para 2; Table 1 | 74% (63–82) | approximation 72.2% (60–80) |
| Group 2, 6 months | Abstract; Results para 2; Table 1 | 77% (67–84) | approximation 75.9% (65–83) |
| New first cases, 6 to 12 months | Results; Table 1 (43→50, 38→39, 105→106) | — | group 1: 7 of 103 still at risk (6.8%); group 2: 1 of 108 (0.9%); control: 1 of 42 (2.4%) |
| Change in group 1 efficacy, 6 to 12 months | Abstract vs Table 1 | abstract implies 74% → 77% | Table 1 74% → 71%; approximation 72.2% → 67.2% (−5.0 points) |
| Other group-1 estimates (secondary definition; Results para 5 adjusted; groups 1+2 pooled) | Table 1; Results paras 4–5 | 70% to 75%, CIs 60–81 | none equals 77% (67–84) |
| Group 1 and group 2, 12 months, all episodes (negative binomial rate ratio; a different measure from the Cox first-episode estimate) | Appendix p 2, Supplementary Table 2b | 69.6% (58.7–77.6) and 75.7% (66.3–82.4) | group 1 below 75% and below group 2; neither equals 77% (67–84) |
| Overstatement of group 1 at 12 months | Abstract | — | 6 percentage points; crosses the 75% goal |
Group 1 gained seven new cases between 6 and 12 months against one in the control group. Over that interval the risk in group 1 was higher than in the controls who were still case-free, so the 12-month estimate for group 1 should be lower than its 6-month estimate. Table 1 shows that fall (74% to 71%). The abstract's statement that group 1 "remained high, at 77%" would need a rise.
Figure
Figure 1. Left: the abstract's only 1-year efficacy sentence. Right: the 12-month values for the two vaccine arms as printed in Results paragraph 3 and Table 1 (primary case definition, unadjusted Cox). The solid arrow links the abstract's triplet to the identical group-2 row; the dashed arrow links it to the group-1 row, which has the same label and a different value. Orange text is printed in the paper. Blue text at the bottom is recomputed by recompute.py.txt from the paper's case counts as 1 minus the ratio of cumulative hazards (an approximation to the Cox estimate).
Figure 2. Vaccine efficacy (%, 95% CI) on one linear axis from 0 (no effect, solid line) to 100, with the WHO goal of 75% as a dashed line. Orange circles: Table 1 unadjusted Cox estimates. Open orange diamond with dashed interval: the abstract's 1-year value for group 1. Blue squares: approximations recomputed from the printed case counts. The abstract's value sits with the group-2 rows, not with group 1, whose estimate falls between 6 and 12 months by both the Cox and the count-based measure.
Why it matters
- It is the abstract's only 1-year efficacy statement. The abstract gives no 1-year value for group 2, so a reader of the abstract alone sees one 1-year number, and it is attached to the wrong arm.
- It moves the lower dose across the threshold the paper uses. The paper frames its result against the WHO goal of at least 75%. Group 1's correct 12-month efficacy, 71% (59–79), is below that goal; the abstract's 77% is above it. With the error, both adjuvant doses appear to meet the goal at 1 year. In the paper's own data only the 50 μg dose does.
- The abstract does not show the waning in the lower-dose arm or the gap between doses. The abstract shows group 1 going from 74% to 77%. The data show it falling to 71% while group 2 holds at 77%, a 6-point 12-month difference that bears on the choice of adjuvant dose. The paper reports no significant difference between groups 1 and 2, and that statement is not affected.
- The main conclusions are not affected. The Research in context panel and the Discussion attribute the 75% claim to the 50 μg dose, and that dose's 12-month efficacy is 77% (67–84) as reported. The title ("Efficacy of a low-dose candidate malaria vaccine…") and the abstract's Interpretation ("shows promising high-level efficacy") make no dose-specific or 75% claim. None of these is affected. Group 1's efficacy remains large and significant (p<0·0001, lower confidence limit 59%). The error is in which arm the abstract names, not in the analysis.
Explanations considered
- Adjusted rather than unadjusted estimate. Does not account for it. Table 1's adjusted 12-month value for group 1 is also 71% (59–79). The separately reported adjusted value in Results paragraph 5 is 75% (67–81), which differs in the point estimate and both limits.
- Secondary case definition. Does not account for it. Group 1 at 12 months is 70% (60–78) unadjusted and 71% (60–79) adjusted.
- Groups 1 and 2 pooled. Does not account for it. The pooled 12-month value is 74% (65–80) unadjusted and 73% (65–80) adjusted.
- A different time point. Does not account for it. Group 1's 6-month value is 74% (63–82), and the same abstract sentence already reports it. The 6-month and 12-month values for group 2 are both 77% (67–84).
- The value belongs to group 2. Accounts for it fully. 77% (67–84) is group 2's 12-month unadjusted estimate in Results paragraph 3 and Table 1, and the Discussion gives the same figure to "the higher dose of MM adjuvant" over 12 months. The group label in the abstract is the only element that differs.
- An archive or conversion artifact. Ruled out. The same sentence appears in the PMC text, the Europe PMC full-text XML and the PubMed abstract (PMID 33964223). The conflict is a group number and a whole triplet of values, not a symbol or digit that could be lost in conversion.
- A different outcome measure. Does not account for it. The appendix (p 2, Supplementary Table 2b) gives 12-month protective efficacy against all malaria episodes, from a negative binomial rate ratio, of 69.6% (58.7–77.6) for group 1 and 75.7% (66.3–82.4) for group 2. This is a different measure from the Cox estimate of time to first episode, so it is not expected to match Table 1, but it independently puts group 1 below 75% and below group 2, and neither value equals 77% (67–84).
- A side inconsistency nearby, which does not explain the main one. Results paragraph 5 reports adjusted 12-month efficacies of 75% (67–81) for group 1 and 77% (65–80) for group 2 "(table 1)", while Table 1's adjusted column gives 71% (59–79) and 76% (65–84). The 77% (65–80) interval is also lopsided on the hazard-ratio scale (log-distance 0.14 below the estimate and 0.42 above, a ratio of 3.0), whereas in all 24 Table 1 intervals the two log-distances differ by a ratio of at most 1.2. The 6-month group-1 count is printed as 29% in the abstract and Results and as 30% in Table 1; 43/146 is 29.45%.
Status
- Version of record. The publisher's pages (thelancet.com, and linkinghub.elsevier.com via the DOI) did not return the article text to automated requests. The publisher-deposited text was checked in the Europe PMC full-text XML for PMC8121760 and in the PubMed record (PMID 33964223), and both carry the abstract sentence "At 1 year, vaccine efficacy remained high, at 77% (67–84) in group 1." Results paragraph 3 and Table 1 in the Europe PMC XML give group 1 at 71% (59–79). Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The supplementary appendix (mmc1.pdf, 111 pages: case-incidence tables, the trial protocol and the statistical analysis plan) was downloaded from the Europe PMC supplementaryFiles endpoint and examined. It contains no Cox efficacy estimates by group beyond those in the paper, and no value of 77% (67–84) for group 1. Its Supplementary Table 2b (appendix p 2) gives 12-month protective efficacy against all malaria episodes (negative binomial rate ratio) of 69.6% (58.7–77.6) for group 1 and 75.7% (66.3–82.4) for group 2, which corroborates the ordering in Table 1 by a different measure. No participant-level data are published; the paper states that anonymised participant data will be made available on request when the trial is complete.
- Corrections and comments. As of 2026-10-24 no correction, erratum or retraction notice was found.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Europe PMC record lists only a "Comment in" link to the accompanying commentary (doi:10.1016/S0140-6736(21)01065-5). PubPeer's API returned HTTP 403 to automated requests, and a web search for the title with "PubPeer" found no thread. - Sources. PMC article PMC8121760; Europe PMC REST search, full-text XML and supplementaryFiles (appendix mmc1.pdf) for PMC8121760; PubMed 33964223; Crossref; PubPeer; https://doi.org/10.1016/S0140-6736(21)00943-0.
Suggested correction
In the abstract, Findings, replace
At 1 year, vaccine efficacy remained high, at 77% (67–84) in group 1.
with
At 1 year, vaccine efficacy was 71% (59–79) in group 1 and 77% (67–84) in group 2.
This agrees with Results paragraph 3 and Table 1. The authors may also wish to reconcile the adjusted 12-month estimates in Results paragraph 5, 75% (67–81) and 77% (65–80), with Table 1's adjusted column, 71% (59–79) and 76% (65–84), and the 29% versus 30% rounding of 43/146.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC8121760 write-up and redraw the figures.
Run: python recompute.py.txt
Inputs are numbers printed in the paper (Abstract, Results, Table 1) and its appendix
(Supplementary Table 2b); no external data. Results paragraphs are counted from the
screening/enrolment paragraph (para 1).
Outputs: printed numbers, and fig1.svg/.png, fig2.svg/.png in this script's folder.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (paper)
N = {1: 146, 2: 146, 3: 147} # mITT, primary case definition
cases = {6: {1: 43, 2: 38, 3: 105}, 12: {1: 50, 2: 39, 3: 106}}
# Table 1, primary case definition: VE % (95% CI), unadjusted and adjusted
table1 = {
(6, 1): {"unadj": (74, 63, 82), "adj": (74, 63, 82)},
(12, 1): {"unadj": (71, 59, 79), "adj": (71, 59, 79)},
(6, 2): {"unadj": (77, 67, 84), "adj": (76, 65, 84)},
(12, 2): {"unadj": (77, 67, 84), "adj": (76, 65, 84)},
}
abstract_12m_g1 = (77, 67, 84) # Abstract: "77% (67-84) in group 1"
results_p5_adj_12m = {1: (75, 67, 81), 2: (77, 65, 80)} # Results para 5 (adjusted)
# every other group-1 estimate printed anywhere in the paper
g1_all = {
"Table 1 primary 6 mo unadjusted": (74, 63, 82),
"Table 1 primary 6 mo adjusted": (74, 63, 82),
"Table 1 primary 12 mo unadjusted": (71, 59, 79),
"Table 1 primary 12 mo adjusted": (71, 59, 79),
"Table 1 secondary 6 mo unadjusted": (73, 63, 81),
"Table 1 secondary 6 mo adjusted": (74, 64, 81),
"Table 1 secondary 12 mo unadjusted": (70, 60, 78),
"Table 1 secondary 12 mo adjusted": (71, 60, 79),
"Results para 5 adjusted 12 mo": (75, 67, 81),
"Groups 1+2 combined, primary 12 mo unadjusted": (74, 65, 80),
"Groups 1+2 combined, primary 12 mo adjusted": (73, 65, 80),
}
def approx_ve(d_v, n_v, d_c, n_c):
"""Count-based approximation to the Cox VE: 1 - ratio of cumulative hazards,
H = -ln(1 - d/n); SE(log HR) ~ sqrt(1/d_v + 1/d_c). Ignores event timing and
censoring, so it is a cross-check of direction and size, not a replacement."""
hv, hc = -np.log(1 - d_v / n_v), -np.log(1 - d_c / n_c)
hr = hv / hc
se = np.sqrt(1 / d_v + 1 / d_c)
lo, hi = hr * np.exp(-1.96 * se), hr * np.exp(1.96 * se)
return 100 * (1 - hr), 100 * (1 - hi), 100 * (1 - lo)
print("1. Is 77% (67-84) a group-1 estimate anywhere in the paper?")
for k, v in g1_all.items():
print(f" {k:<48} {v[0]}% ({v[1]}-{v[2]}) equal: {v == abstract_12m_g1}")
print(f" Table 1 group 2 primary 12 mo unadjusted {table1[(12, 2)]['unadj']}"
f" equal: {table1[(12, 2)]['unadj'] == abstract_12m_g1}")
print(f" Table 1 group 2 primary 6 mo unadjusted {table1[(6, 2)]['unadj']}"
f" equal: {table1[(6, 2)]['unadj'] == abstract_12m_g1}")
print("\n2. Hazard ratios implied by the printed VE (HR = 1 - VE)")
for lab, (ve, lo, hi) in [("Abstract 'group 1', 12 mo", abstract_12m_g1),
("Table 1 group 1, 12 mo", table1[(12, 1)]["unadj"]),
("Table 1 group 2, 12 mo", table1[(12, 2)]["unadj"])]:
print(f" {lab:<28} HR {1 - ve / 100:.2f} (95% CI {1 - hi / 100:.2f}-{1 - lo / 100:.2f})")
print(f" Overstatement of group 1 at 12 mo: {abstract_12m_g1[0] - table1[(12, 1)]['unadj'][0]}"
" percentage points; WHO goal is >= 75%")
print("\n3. Crude risks (n/N)")
for t in (6, 12):
for g in (1, 2, 3):
print(f" {t:>2} mo group {g}: {cases[t][g]}/{N[g]} = {100 * cases[t][g] / N[g]:.1f}%")
print(f" totals: 6 mo {sum(cases[6].values())}, 12 mo {sum(cases[12].values())},"
f" increase {sum(cases[12].values()) - sum(cases[6].values())} (Results: 186, 195, +9)")
print("\n4. Count-based approximation vs Table 1 Cox estimates (unadjusted)")
approx = {}
for t in (6, 12):
for g in (1, 2):
a = approx_ve(cases[t][g], N[g], cases[t][3], N[3])
approx[(t, g)] = a
r = table1[(t, g)]["unadj"]
print(f" {t:>2} mo group {g}: approx {a[0]:.1f}% ({a[1]:.0f}-{a[2]:.0f});"
f" Table 1 {r[0]}% ({r[1]}-{r[2]}); Table 1 minus approx {r[0] - a[0]:.1f}")
print("\n5. What happened between 6 and 12 months")
for g in (1, 2, 3):
at_risk = N[g] - cases[6][g]
new = cases[12][g] - cases[6][g]
print(f" group {g}: {at_risk} still case-free at 6 mo, {new} new first cases"
f" ({100 * new / at_risk:.1f}% of those at risk)")
print(f" group 1 approx VE {approx[(6, 1)][0]:.1f}% -> {approx[(12, 1)][0]:.1f}%"
f" (change {approx[(12, 1)][0] - approx[(6, 1)][0]:+.1f});"
f" group 2 {approx[(6, 2)][0]:.1f}% -> {approx[(12, 2)][0]:.1f}%"
f" (change {approx[(12, 2)][0] - approx[(6, 2)][0]:+.1f})")
print(f" Table 1 group 1: 74% -> 71%; Abstract implies 74% -> 77%")
print("\n6. Side check: Results para 5 adjusted 12-mo values vs Table 1 adjusted column")
for g in (1, 2):
ve, lo, hi = results_p5_adj_12m[g]
hr, hlo, hhi = 1 - ve / 100, 1 - hi / 100, 1 - lo / 100
print(f" group {g}: Results {ve}% ({lo}-{hi}); Table 1 adjusted {table1[(12, g)]['adj']};"
f" log-HR distance to lower/upper limit {np.log(hr) - np.log(hlo):.2f}/"
f"{np.log(hhi) - np.log(hr):.2f}")
for g in (1, 2):
ve, lo, hi = table1[(12, g)]["adj"]
hr, hlo, hhi = 1 - ve / 100, 1 - hi / 100, 1 - lo / 100
print(f" Table 1 adjusted group {g} {ve}% ({lo}-{hi}): log-HR distances"
f" {np.log(hr) - np.log(hlo):.2f}/{np.log(hhi) - np.log(hr):.2f}")
tab_all = [(74,63,82),(71,59,79),(73,63,81),(70,60,78),(74,63,82),(71,59,79),(74,64,81),(71,60,79),
(77,67,84),(77,67,84),(79,70,85),(80,72,86),(76,65,84),(76,65,84),(78,69,85),(80,71,86),
(76,67,82),(74,65,80),(76,69,82),(75,68,81),(75,67,82),(73,65,80),(76,69,82),(75,67,81)]
ratios = []
for ve, lo, hi in tab_all: # all 24 Table 1 cells (unadjusted and adjusted)
hr, hlo, hhi = 1 - ve / 100, 1 - hi / 100, 1 - lo / 100
d_lo, d_hi = np.log(hr) - np.log(hlo), np.log(hhi) - np.log(hr)
ratios.append(max(d_lo, d_hi) / min(d_lo, d_hi))
print(f" All 24 Table 1 intervals: larger/smaller log-distance ratio at most {max(ratios):.2f}")
d_lo = np.log(0.23) - np.log(0.20); d_hi = np.log(0.35) - np.log(0.23)
print(f" Results para 5 group 2, 77% (65-80): ratio {d_hi / d_lo:.2f}")
print(f" Side check: 43/146 = {100 * 43 / 146:.2f}% (Abstract/Results print 29%, Table 1 prints 30%)")
print("\n7. Appendix, Supplementary Table 2b (all episodes, negative binomial rate ratio; not the Cox"
" first-episode measure)")
supp_2b = {1: (69.6, 58.7, 77.6), 2: (75.7, 66.3, 82.4)} # protective efficacy % (95% CI), 12 mo
for g, (ve, lo, hi) in supp_2b.items():
print(f" group {g}: {ve}% ({lo}-{hi}); below 75%: {ve < 75};"
f" equals 77% (67-84): {(ve, lo, hi) == abstract_12m_g1}")
print(f" group 1 below group 2: {supp_2b[1][0] < supp_2b[2][0]}")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 11, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: two text panels joined by a connector
fig = plt.figure(figsize=(7.2, 5.2), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(-0.3, 4.9); ax.axis("off")
ax.text(0.25, 4.62, "The abstract gives group 1 the 12-month estimate of group 2",
fontsize=13.5, weight="bold", va="center")
ax.text(0.25, 4.32, "Vaccine efficacy against clinical malaria, primary case definition, 12 months",
fontsize=10.5, color=INK2, va="center")
L = (0.25, 1.15, 2.8, 2.85) # x, y, w, h
R = (4.0, 1.15, 2.95, 2.85)
for (x, y, w, h) in (L, R):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.08",
fc="#f4f3f0", ec=INK2, lw=1.4))
# left panel: Abstract
x0, y0 = L[0] + 0.18, L[1] + L[3] - 0.25
ax.text(x0, y0, "ABSTRACT (Findings)", fontsize=10.5, weight="bold", color=INK2, va="top")
ax.text(x0, y0 - 0.38, "“At 1 year, vaccine efficacy\nremained high, at", fontsize=11,
va="top", linespacing=1.4)
ax.text(x0, y0 - 1.12, "77% (67–84) in group 1.”", fontsize=12, weight="bold",
color=ORANGE, va="top")
ax.text(x0, y0 - 1.65, "No 1-year value is given\nfor group 2.", fontsize=10.5,
color=INK2, va="top", linespacing=1.35)
# right panel: Results / Table 1
x1, y1 = R[0] + 0.18, R[1] + R[3] - 0.25
ax.text(x1, y1, "RESULTS (para 3) + TABLE 1", fontsize=10.5, weight="bold", color=INK2,
va="top")
ax.text(x1, y1 - 0.38, "Group 1 (25 μg Matrix-M)", fontsize=10.5, va="top")
ax.text(x1, y1 - 0.66, "50/146 cases → 71% (59–79)", fontsize=11.5, weight="bold",
color=ORANGE, va="top")
ax.text(x1, y1 - 1.06, "Group 2 (50 μg Matrix-M)", fontsize=10.5, va="top")
ax.text(x1, y1 - 1.34, "39/146 cases → 77% (67–84)", fontsize=11.5, weight="bold",
color=ORANGE, va="top")
ax.text(x1, y1 - 1.82, "Adjusted (Table 1): 71% and 76%.\nWHO goal: ≥75%.",
fontsize=10.5, color=INK2, va="top", linespacing=1.35)
# connector from the abstract value to the group-2 row (same triplet)
ab_y = y0 - 1.25
g2_y = y1 - 1.47
g1_y = y1 - 0.79
ax.add_patch(FancyArrowPatch((L[0] + L[2] - 0.05, ab_y), (R[0] + 0.1, g2_y),
arrowstyle="-|>", mutation_scale=14, lw=2, color=INK,
connectionstyle="arc3,rad=0"))
ax.text(3.53, g2_y - 0.12, "same value\nand CI", ha="center", va="top", fontsize=10,
weight="bold", linespacing=1.2)
ax.add_patch(FancyArrowPatch((L[0] + L[2] - 0.05, ab_y + 0.35), (R[0] + 0.1, g1_y),
arrowstyle="-|>", mutation_scale=12, lw=2, color=INK2,
linestyle=(0, (3, 2)), connectionstyle="arc3,rad=-0.15"))
ax.text(3.53, g1_y + 0.2, "same label,\nother value", ha="center", va="bottom", fontsize=10,
color=INK2, linespacing=1.2)
ax.text(0.25, 0.82, "Recomputed from the case counts (1 − ratio of cumulative hazards):",
fontsize=10.5, color=INK2, va="center")
ax.text(0.25, 0.5, f"group 1: {approx[(6, 1)][0]:.0f}% at 6 mo → {approx[(12, 1)][0]:.0f}% at 12 mo"
f" (+{cases[12][1] - cases[6][1]} cases vs +{cases[12][3] - cases[6][3]} in controls)",
fontsize=11, weight="bold", color=BLUE, va="center")
ax.text(0.25, 0.2, "Group 1's efficacy fell after 6 months; it did not rise to 77%.",
fontsize=10.5, color=BLUE, va="center")
ax.text(6.95, -0.15, "orange = printed in the paper blue = recomputed from its counts",
fontsize=10, color=INK2, va="center", ha="right")
# keep the legend line on its own row so it does not collide with the blue sentence
save(fig, "fig1")
# Figure 2: dot-and-interval plot, VE on one linear axis from 0 (null) to 100
rows = [ # (label, kind, (ve, lo, hi))
("Group 1, 6 months", "t1", table1[(6, 1)]["unadj"]),
(None, "ap", approx[(6, 1)]),
("Group 1, 12 months", "t1", table1[(12, 1)]["unadj"]),
(None, "ap", approx[(12, 1)]),
(None, "ab", abstract_12m_g1),
("Group 2, 6 months", "t1", table1[(6, 2)]["unadj"]),
(None, "ap", approx[(6, 2)]),
("Group 2, 12 months", "t1", table1[(12, 2)]["unadj"]),
(None, "ap", approx[(12, 2)]),
]
ypos, y, labels = [], 0.0, []
for lab, kind, _ in rows:
if lab is not None and ypos:
y += 0.55
ypos.append(y); y += 0.42
ypos = -np.array(ypos)
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
fig.subplots_adjust(left=0.25, right=0.97, top=0.86, bottom=0.2)
ax.set_facecolor(BG)
ax.axvline(0, color=INK2, lw=0.8)
ax.axvline(75, color=INK2, lw=0.9, ls=(0, (4, 3)))
ax.text(75, ypos[0] + 0.42, "WHO goal 75%", ha="center", va="bottom", fontsize=10, color=INK2,
bbox=dict(fc=BG, ec="none", pad=1))
for yy, (lab, kind, (ve, lo, hi)) in zip(ypos, rows):
if kind == "t1":
ax.plot([lo, hi], [yy, yy], color=ORANGE, lw=2)
ax.plot(ve, yy, "o", color=ORANGE, ms=8)
txt = f"{ve}% ({lo}–{hi})"
col = ORANGE
elif kind == "ab":
ax.plot([lo, hi], [yy, yy], color=ORANGE, lw=2, ls=(0, (2, 1.5)))
ax.plot(ve, yy, "D", mfc=BG, mec=ORANGE, mew=2, ms=8)
txt = f"{ve}% ({lo}–{hi}) Abstract, labelled group 1"
col = ORANGE
else:
ax.plot([lo, hi], [yy, yy], color=BLUE, lw=2)
ax.plot(ve, yy, "s", color=BLUE, ms=8)
txt = f"{ve:.0f}% ({lo:.0f}–{hi:.0f})"
col = BLUE
ax.text(3, yy, txt, va="center", ha="left", fontsize=10, color=col)
ax.set_yticks([yy for yy, r in zip(ypos, rows) if r[0]])
ax.set_yticklabels([r[0] for r in rows if r[0]], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlim(0, 100)
ax.set_ylim(ypos[-1] - 0.4, ypos[0] + 0.75)
ax.set_xlabel("Vaccine efficacy, % (95% CI) [0 = no effect]", fontsize=10.5)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
fig.text(0.03, 0.95, "Group 1 falls from 74% to 71%; only group 2 stays at 77%",
fontsize=13, weight="bold", va="center")
fig.text(0.03, 0.905, "Primary case definition; Table 1 unadjusted Cox estimates and count-based check",
fontsize=10, color=INK2, va="center")
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Table 1 (paper)"),
Line2D([], [], color=ORANGE, marker="D", mfc=BG, mew=2, ms=8, lw=2,
ls=(0, (2, 1.5)), label="Abstract (paper)"),
Line2D([], [], color=BLUE, marker="s", ms=8, lw=2,
label="Recomputed from counts")]
fig.legend(handles=handles, loc="lower center", ncol=3, frameon=False, fontsize=10,
bbox_to_anchor=(0.55, 0.0))
save(fig, "fig2")
print("\nFigures written to", HERE)
Abstract cites non-significant interaction odds ratios as evidence that correct fluid significantly reduced mortality
Lancet Child Adolesc Health (2018) · doi:10.1016/S2352-4642(18)30130-5 · PMC6004535
Summary
The abstract states that correct fluid prescription "significantly reduced the risk of early mortality (within 2 days) in all subgroups". It supports this with five adjusted odds ratios (AORs): 1·23, 0·95, 1·70, 1·50 and 0·86. These five values, with their CIs, are the "Multiplicative interaction" column of Table 2, which the table footnote defines as OR11/(OR10 × OR01), and they can be recomputed from the table's other cells. They measure how much the fluid association differs between children with and without a clinical sign. They are not the effect of correct fluid in either group. All five 95% CIs include 1 (approximate p = 0.50, 0.87, 0.07, 0.22, 0.58), and three of the five point estimates are above 1. So the statistics the abstract cites do not show a significant reduction in any subgroup. The same table, read for the correct-versus-wrong contrast within each stratum, does point to lower odds of early death with correct fluid in every subgroup. The headline direction may therefore hold, but the estimates, their label and the significance claim in the abstract's key fluid result are wrong as printed. Table 2 also contains two cell errors in its additive-interaction column.
What the paper reports
Abstract, Findings:
Correct fluid prescription significantly reduced the risk of early mortality (within 2 days) in all subgroups: abnormal respiratory signs (AOR 1·23, 0·68–2·24), abnormal circulatory signs (0·95, 0·53–1·73), pallor (1·70, 0·95–3·02), dehydration signs only (1·50, 0·79–2·88), and abnormal neurological signs (0·86, 0·51–1·48).
Abstract, Interpretation:
the prescription of recommended rehydration guidelines reduces risk of death
Research in context, Added value of this study:
Our findings also show that comorbidities are common and that correct fluid prescription is associated with reduced risk of death in these patients.
Methods, Data analysis (third paragraph, on effect modification):
We calculated the relative excess odds due to interaction, the attributable proportion, and the multiplicative interaction odds ratio (OR).
Results (effect-modification paragraph), where the same numbers appear under their correct name:
In fact, positive multiplicative interactions (OR>1) for association of incorrect fluid management with death in children with dehydration signs (OR 1·50, 95% CI 0·79–2·88), respiratory signs (OR 1·23, 0·68–2·24), and pallor (OR 1·70, 0·95–3·02) are evidence of interaction (figure 3).
Table 2, "Interactions of fluid management with risk factors for in-hospital early deaths" (columns: Multiplicative interaction | Fluid wrong and symptom present (OR10) | Fluid right and symptom present (OR11) | Fluid right and symptom absent (OR01) | RERI | Attributable proportion):
Pallor | 1·70 (0·95 to 3·02) | 2·16 (1·66 to 2·82) | 1·07 (0·69 to 1·67) | 0·29 (0·21 to 0·41) | -0·38 (-1·07 to 0·31) | -0·35 (-1·09 to 0·38)
Circulatory | 0·95 (0·53 to 1·73) | 1·92 (1·47 to 2·51) | 0·64 (0·40 to 1·03) | 0·35 (0·25 to 0·50) | -0·63 (-1·19 to -0·06) | -0·98 (-2·06 to 0·11)
Dehydration signs only | 1·50 (0·79 to 2·88) | 0·96 (0·70 to 1·33) | 0·38 (0·26 to 0·55) | 0·26 (0·15 to 0·45) | 0·25 (-0·18 to 0·48) | 0·40 (-0·53 to 1·33)
Neurological | 0·86 (0·51 to 1·48) | 3·57 (2·75 to 4·64) | 1·15 (0·77 to 1·72) | 0·37 (0·25 to 0·54) | -1·80 (-2·71 to -0·88) | -1·56 (-2·61 to -0·52)
Respiratory | 1·23 (0·68 to 2·24) | 3·18 (2·35 to 4·31) | 1·17 (0·80 to 1·73) | 0·30 (0·18 to 0·49) | -1·31 (-2·16 to -9·45) | -1·11 (-1·90 to -0·33)
Table 2 footnote (excerpts):
Interaction exists if RERI is not zero (RERI=OR11 – OR10 – OR01 + 1).
Multiplicative interaction=OR11/(OR10 × OR01).
The reference group was fluid wrong and symptom absent (OR00)=1·00.
Discussion, final paragraph:
Importantly, prescription of recommended rehydration guidelines reduces risk of death in this group of children.
What the paper's own numbers show
Method. I compared the five abstract values with each column of Table 2. Using the footnote formulas, I recomputed the multiplicative interaction OR, the RERI and the attributable proportion (AP) from the three cell ORs in each row (OR10, OR11, OR01; OR00 = 1). For each printed 95% CI, I took the standard error of log(OR) as (ln upper − ln lower)/(2 × 1.96) and derived an approximate two-sided Wald p value. Within the "symptom present" stratum, the effect of correct versus wrong fluid is OR11/OR10, because both cells share the reference OR00. Within the "symptom absent" stratum it is OR01. The paper does not report a CI for OR11/OR10. I approximated one by adding the two variances on the log scale and ignoring their covariance. Both estimates are contrasts against the same reference group, so their covariance is usually positive, and in that case the approximate interval is wider than the true one. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Respiratory "AOR" for correct fluid | Abstract; Table 2, multiplicative interaction | 1·23 (0·68–2·24) | Identical to Table 2 interaction OR; OR11/(OR10·OR01) = 1.17/(3.18 × 0.30) = 1.23; CI includes 1; p ≈ 0.50 |
| Circulatory "AOR" | Abstract; Table 2 | 0·95 (0·53–1·73) | Identical to interaction OR; 0.64/(1.92 × 0.35) = 0.95; CI includes 1; p ≈ 0.87 |
| Pallor "AOR" | Abstract; Table 2 | 1·70 (0·95–3·02) | Identical to interaction OR; 1.07/(2.16 × 0.29) = 1.71; CI includes 1; p ≈ 0.07 |
| Dehydration signs only "AOR" | Abstract; Table 2 | 1·50 (0·79–2·88) | Identical to interaction OR; 0.38/(0.96 × 0.26) = 1.52 (rounding of the inputs allows 1.47–1.58); CI includes 1; p ≈ 0.22 |
| Neurological "AOR" | Abstract; Table 2 | 0·86 (0·51–1·48) | Identical to interaction OR; 1.15/(3.57 × 0.37) = 0.87 (rounding allows 0.85–0.89); CI includes 1; p ≈ 0.58 |
| Cited CIs that exclude 1 | Abstract | "significantly reduced … in all subgroups" | 0 of 5; 3 of 5 point estimates are above 1 |
| Correct vs wrong fluid, symptom present (OR11/OR10) | Derived from Table 2 | not reported | Respiratory 0.37 (≈0.23–0.60); circulatory 0.33 (≈0.19–0.57); pallor 0.50 (≈0.30–0.83); dehydration only 0.40 (≈0.24–0.65); neurological 0.32 (≈0.20–0.52). Approximate CIs, covariance ignored |
| Correct vs wrong fluid, symptom absent (OR01) | Table 2 | 0·30, 0·35, 0·29, 0·26, 0·37 | Same values; all five printed CIs exclude 1 |
| Identity linking the two | Table 2 footnote | interaction = OR11/(OR10·OR01) | (OR11/OR10) / OR01 equals the interaction OR in every row: the interaction OR is the ratio of the two within-stratum fluid effects |
| Dehydration signs only, RERI | Table 2 | 0·25 (−0·18 to 0·48) | OR11 − OR10 − OR01 + 1 = 0.38 − 0.96 − 0.26 + 1 = 0.16; the printed CI's midpoint is 0.15 |
| Dehydration signs only, AP | Table 2 | 0·40 | RERI/OR11 = 0.16/0.38 = 0.42 with the recomputed RERI; 0.25/0.38 = 0.66 with the printed RERI |
| RERI and AP, other four rows | Table 2 | e.g. −0·38, −0·63, −1·80, −1·31 | Recompute to −0.38, −0.63, −1.79, −1.31; the AP CI midpoints in all four rows, and the RERI CI midpoints for circulatory, pallor and neurological, match the point estimates. The respiratory RERI CI midpoint (−5.81) does not (see next row) |
| Respiratory RERI CI | Table 2 | −1·31 (−2·16 to −9·45) | Upper limit (−9.45) is below both the point estimate and the lower limit, which is impossible; a symmetric interval around −1.31 would end near −0.46 |
So the abstract's numbers are the Table 2 interaction ORs. The cell ORs in each row reproduce them, so the column labels are internally consistent. None of the five is significant. The contrast the abstract's sentence describes (correct versus wrong fluid within a subgroup) is a different quantity. Table 2 points to it being below 1 in all five subgroups, in both strata, but the paper does not report it.
Figure
Figure 1. One log-scale odds-ratio axis; the vertical line marks OR = 1. Orange circles and lines are the five values the abstract cites as AORs for correct fluid prescription, with their printed 95% CIs. They are identical to Table 2's multiplicative-interaction column. Each crosses 1, and the approximate p value is printed beside it. Blue squares are the odds ratio for correct versus wrong fluid among children with the sign present, recomputed from the same Table 2 row as OR11/OR10. Their intervals are approximate because the covariance of the two estimates is not reported and is ignored here. All five blue estimates lie below 1. The abstract's sentence describes the blue quantity but cites the orange numbers.
Figure 2. Table 2's RERI column on a linear axis; the vertical line marks 0. Orange: RERI and 95% CI as printed. Blue squares: OR11 − OR10 − OR01 + 1 from the same row. Gray ticks: midpoint of the printed CI. In three rows (circulatory, pallor, neurological) the three agree. For dehydration signs only, the printed point (0.25) differs from both the formula (0.16) and the CI midpoint (0.15). For respiratory signs, the printed point (−1.31) matches the formula, but the printed interval runs from −2.16 to −9.45, so the point lies outside it and the CI midpoint (−5.81) is far from it.
Why it matters
The effect of correct fluid prescription is one of the paper's two stated aims and is named in its title. The abstract's Findings sentence is the only place in the summary that quantifies it, and the Interpretation ("reduces risk of death") and the closing Discussion sentence are the abstract-level and concluding statements of it. The body gives one other quantitative statement: "The predicted probability of early in-hospital death was reduced by 6% when derived from model II with correct fluid prescription compared with when fluid prescription is wrong" (Results, effect-modification paragraph). That is an overall model prediction, not a subgroup estimate, and it is printed without a CI or test, so it does not support "significantly … in all subgroups". A reader of the abstract alone would take the five AORs as subgroup treatment effects. Three are above 1 and none excludes 1, so the reader would see a contradiction or would misread interaction estimates as effects of fluid prescription. Meta-analysts and guideline reviewers who extract the abstract figures would record five non-significant ORs, some pointing to harm, for a result the paper calls a significant reduction. The Results paragraph also calls three non-significant interaction ORs "evidence of interaction". The data do not support that claim on the multiplicative scale. On the additive scale, the RERI CIs for pallor and dehydration signs only include 0, and the respiratory RERI CI is unusable as printed. The paper's own cells point to a consistent protective association of correct fluid. The defect is therefore in which estimates are reported and how they are described, not necessarily in the direction of the conclusion. The magnitude and precision of the subgroup effects cannot be checked from what is published.
Explanations considered
- Text-conversion or extraction artifact. Not supported. The version-of-record check found the same sentence and numbers in the publisher-deposited JATS XML in Europe PMC and in the separately supplied MEDLINE abstract (PMID 29971245). The publisher's typeset Figure 3 image was also inspected. Its interaction panel ("Adjusted odds ratios (model II-interactions)") shows the five abstract subgroups (plus airway signs) with CIs crossing 1. In the XML, Table 2's cells are aligned with their headers.
- The abstract values are the within-subgroup fluid effects and the Table 2 column is mislabelled. Not supported. Each value equals OR11/(OR10 × OR01) recomputed from the other cells in its row, within input rounding. The Results paragraph calls the same numbers "multiplicative interactions". Neither within-stratum effect (OR11/OR10 or OR01) is close to any of the five values.
- "Significantly" refers to something other than the cited ORs, such as the additive interaction. It does not account for the numbers. The sentence attaches the five ORs directly to the claim. RERI is a measure of interaction, not of the effect of fluid. Its printed CIs include 0 for pallor (−1.07 to 0.31) and dehydration signs only (−0.18 to 0.48), so it is not significant in all subgroups either.
- "Significantly" refers to the within-stratum correct-versus-wrong contrasts, and the wrong column was copied into the abstract. This would explain the wording. The approximate intervals for OR11/OR10 and the printed intervals for OR01 all exclude 1. But the paper never reports those subgroup estimates with CIs, so the claim remains unverifiable as published, and the numbers it does cite contradict it. The approximate intervals here ignore a covariance that only the authors' model output provides.
- Rounding. Rounding explains the small gaps between printed and recomputed interaction ORs (for example 1.50 against 1.52, inside the rounding range 1.47–1.58). It cannot move any CI to exclude 1. It also cannot explain the dehydration RERI (0.25 against 0.16, with a CI midpoint of 0.15 and an AP that matches 0.16) or the respiratory RERI interval, whose upper limit lies below its lower limit.
Status
As of 2026-10-23:
- Version of record. The publisher's HTML and PDF could not be opened directly. doi.org redirected to an Elsevier placeholder, and thelancet.com returned HTTP 403. The abstract sentence, Table 2 and the interaction values were confirmed in three publisher-derived copies: the publisher-deposited JATS XML in Europe PMC (CC BY), the MEDLINE abstract (PMID 29971245), and the publisher's Figure 3 image (gr3.jpg).
- Supplements. The Europe PMC supplementary bundle for PMC6004535 contains only the article figures (gr1–gr3). The JATS XML has no supplementary-material element or appendix reference. No source of subgroup-specific correct-versus-wrong fluid ORs with CIs was found. A web appendix listed on the journal site could not be ruled out, because that site returned HTTP 403.
- Corrections and retractions. Crossref update metadata, PMC notices and Europe PMC notices list no correction, erratum, expression of concern or retraction. A web search for the title with "correction" or "erratum" found no correction notice.
- Linked commentary. The PMC record links an accompanying Comment in the same journal: "Treating dehydrating diarrhoea at district hospital level in sub-Saharan Africa: from policy to reality", Lancet Child Adolesc Health 2018, pp 471–472, doi:10.1016/S2352-4642(18)30166-4, PMID 30169312. Its text was not read. It is not open access, Europe PMC holds no abstract or full text for it, and the journal and ScienceDirect pages returned HTTP 403. Whether it repeats or relies on the abstract's "significantly reduced … in all subgroups" claim is therefore not known.
- PubPeer. PubPeer's API returned a human-verification challenge (HTTP 403) to scripted access, so it could not be queried directly. A web search for the title together with "PubPeer" found no thread. Absence of a PubPeer discussion is therefore not fully confirmed.
- Sources. Article as archived in PMC (PMC6004535); Europe PMC JATS XML and supplementary files; MEDLINE record PMID 29971245; Crossref record for doi:10.1016/S2352-4642(18)30130-5; Europe PMC and Crossref records for the linked Comment (PMID 30169312, doi:10.1016/S2352-4642(18)30166-4), metadata only.
Suggested correction
- In the abstract Findings, replace the five interaction ORs with estimates of the quantity the sentence describes. One option is the adjusted OR for correct versus wrong fluid within each subgroup (OR11/OR10, with a CI from the fitted model including the covariance term). Another is the overall adjusted OR for correct fluid. Alternatively, keep the interaction ORs but describe them accurately, for example: "There was no evidence that the association of correct fluid prescription with early mortality differed by clinical sign on the multiplicative scale (interaction ORs 0·86–1·70, all 95% CIs including 1)." Remove "significantly" and "in all subgroups" unless subgroup-specific estimates with CIs excluding 1 are reported.
- In the Results effect-modification paragraph, revise "are evidence of interaction" for the three interaction ORs whose CIs include 1.
- In Table 2, correct the dehydration-signs-only RERI. The footnote formula gives 0.16, consistent with the printed CI midpoint (0.15) and the printed AP (0.40, against 0.42). Correct the respiratory RERI CI, whose printed upper limit (−9·45) cannot be right; a symmetric interval would end near −0.46.
- Consider adding a column to Table 2 with the within-stratum correct-versus-wrong fluid OR and its CI for each sign, so readers can check the abstract's claim directly.
Recomputation script (python recompute.py.txt)
"""Recompute the fluid-prescription subgroup results of PMC6004535
(Lancet Child Adolesc Health 2018, doi:10.1016/S2352-4642(18)30130-5).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes, next to this file:
fig1.svg/png the figure used in writeup.md (same design as candidate 1)
fig2.svg/png Table 2 additive-interaction (RERI) cells, reported vs recomputed
candidate1..5.svg/png five alternative designs for figure 1, for review
Inputs are copied from the paper as archived in PubMed Central:
Abstract, Findings: the five "AOR" values cited for correct fluid prescription
Table 2: multiplicative interaction OR, OR10, OR11, OR01, RERI and attributable
proportion (AP), each with its 95% CI. Reference group OR00 = 1 (fluid wrong,
symptom absent). Footnote: RERI = OR11 - OR10 - OR01 + 1; AP = RERI/OR11;
multiplicative interaction = OR11/(OR10 x OR01).
"""
import itertools
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
ROWS = ["Respiratory", "Circulatory", "Pallor", "Dehydration signs only", "Neurological"] # abstract order
# Abstract, Findings: "(AOR 1.23, 0.68-2.24) ..." in this order
ABSTRACT = {
"Respiratory": (1.23, 0.68, 2.24),
"Circulatory": (0.95, 0.53, 1.73),
"Pallor": (1.70, 0.95, 3.02),
"Dehydration signs only": (1.50, 0.79, 2.88),
"Neurological": (0.86, 0.51, 1.48),
}
# Table 2 (row label: columns as printed)
T2 = {
"Pallor": dict(MI=(1.70, 0.95, 3.02), OR10=(2.16, 1.66, 2.82), OR11=(1.07, 0.69, 1.67),
OR01=(0.29, 0.21, 0.41), RERI=(-0.38, -1.07, 0.31), AP=(-0.35, -1.09, 0.38)),
"Circulatory": dict(MI=(0.95, 0.53, 1.73), OR10=(1.92, 1.47, 2.51), OR11=(0.64, 0.40, 1.03),
OR01=(0.35, 0.25, 0.50), RERI=(-0.63, -1.19, -0.06), AP=(-0.98, -2.06, 0.11)),
"Dehydration signs only": dict(MI=(1.50, 0.79, 2.88), OR10=(0.96, 0.70, 1.33), OR11=(0.38, 0.26, 0.55),
OR01=(0.26, 0.15, 0.45), RERI=(0.25, -0.18, 0.48), AP=(0.40, -0.53, 1.33)),
"Neurological": dict(MI=(0.86, 0.51, 1.48), OR10=(3.57, 2.75, 4.64), OR11=(1.15, 0.77, 1.72),
OR01=(0.37, 0.25, 0.54), RERI=(-1.80, -2.71, -0.88), AP=(-1.56, -2.61, -0.52)),
"Respiratory": dict(MI=(1.23, 0.68, 2.24), OR10=(3.18, 2.35, 4.31), OR11=(1.17, 0.80, 1.73),
OR01=(0.30, 0.18, 0.49), RERI=(-1.31, -2.16, -9.45), AP=(-1.11, -1.90, -0.33)),
}
SHORT = {"Respiratory": "Respiratory", "Circulatory": "Circulatory", "Pallor": "Pallor",
"Dehydration signs only": "Dehydration only", "Neurological": "Neurological"}
def se_log(c):
"""SE of log(OR) implied by a symmetric-on-log 95% CI."""
return (np.log(c[2]) - np.log(c[1])) / (2 * Z)
def p_from_ci(c):
return 2 * norm.sf(abs(np.log(c[0]) / se_log(c)))
def f2(x):
return f"{x:.2f}"
# ---------------------------------------------------------------- 1. abstract = Table 2 interaction column
print("1) Abstract 'AOR' values vs Table 2 'Multiplicative interaction' column")
P_ABS = {}
for k in ROWS:
a, t = ABSTRACT[k], T2[k]["MI"]
P_ABS[k] = p_from_ci(a)
print(f" {k:23s} abstract {a} Table2 MI {t} identical={a == t} "
f"CI includes 1={a[1] <= 1 <= a[2]} point>1={a[0] > 1} approx two-sided p={P_ABS[k]:.2f}")
n_excl = sum(not (ABSTRACT[k][1] <= 1 <= ABSTRACT[k][2]) for k in ROWS)
n_above = sum(ABSTRACT[k][0] > 1 for k in ROWS)
print(f" CIs excluding 1: {n_excl} of 5; point estimates above 1: {n_above} of 5")
print(" geometric midpoint of each CI (check that CIs are symmetric on the log scale):")
for k in ROWS:
a = ABSTRACT[k]
print(f" {k:23s} sqrt(lo*hi)={np.sqrt(a[1] * a[2]):.3f} vs point {a[0]}")
# ---------------------------------------------------------------- 2. recompute interaction measures
print("\n2) Interaction measures recomputed from Table 2 component ORs (footnote formulas)")
REC = {}
for k in ROWS:
d = T2[k]
o10, o11, o01 = d["OR10"][0], d["OR11"][0], d["OR01"][0]
mi = o11 / (o10 * o01)
# range allowed by 2-decimal rounding of the three inputs
grid = [(o11 + e1) / ((o10 + e2) * (o01 + e3)) for e1, e2, e3 in itertools.product((-0.005, 0.005), repeat=3)]
reri = o11 - o10 - o01 + 1
ap = reri / o11
REC[k] = dict(MI=mi, MI_lo=min(grid), MI_hi=max(grid), RERI=reri, AP=ap, AP_repRERI=d["RERI"][0] / o11)
print(f" {k:23s} MI reported {f2(d['MI'][0])} recomputed {mi:.3f} (rounding range {min(grid):.3f}-{max(grid):.3f})"
f" | RERI reported {f2(d['RERI'][0])} recomputed {reri:.2f}"
f" | AP reported {f2(d['AP'][0])}; RERI_recomputed/OR11={ap:.2f}; RERI_reported/OR11={REC[k]['AP_repRERI']:.2f}")
print("\n CI midpoints (RERI and AP CIs are symmetric on the linear scale):")
for k in ROWS:
d = T2[k]
mr = (d["RERI"][1] + d["RERI"][2]) / 2
ma = (d["AP"][1] + d["AP"][2]) / 2
print(f" {k:23s} RERI CI midpoint {mr:.3f} (reported point {f2(d['RERI'][0])}) | AP CI midpoint {ma:.3f} (reported {f2(d['AP'][0])})")
r = T2["Respiratory"]["RERI"]
sym_upper = 2 * r[0] - r[1]
print(f" Respiratory RERI CI printed ({r[1]} to {r[2]}): upper < lower = {r[2] < r[1]}; "
f"upper < point = {r[2] < r[0]}; a symmetric interval around {r[0]} would end near {sym_upper:.2f}")
# ---------------------------------------------------------------- 3. effect of correct vs wrong fluid within strata
print("\n3) Correct vs wrong fluid within each stratum (implied by Table 2; same model per row)")
print(" symptom present: OR11/OR10; approximate CI ignores the covariance of the two log-ORs")
print(" symptom absent : OR01 (contrast with OR00 = 1), CI as printed")
WITHIN = {}
for k in ROWS:
d = T2[k]
ratio = d["OR11"][0] / d["OR10"][0]
s = np.sqrt(se_log(d["OR11"]) ** 2 + se_log(d["OR10"]) ** 2)
lo, hi = ratio * np.exp(-Z * s), ratio * np.exp(Z * s)
p = 2 * norm.sf(abs(np.log(ratio) / s))
WITHIN[k] = (ratio, lo, hi, p)
o01 = d["OR01"]
print(f" {k:23s} present: {ratio:.2f} (approx {lo:.2f}-{hi:.2f}, p{"<0.001" if p < 0.001 else "~" + format(p, ".3f")})"
f" | absent: OR01 {f2(o01[0])} ({f2(o01[1])}-{f2(o01[2])}), CI excludes 1={o01[2] < 1}")
# identity: interaction OR = (present-stratum effect) / (absent-stratum effect)
print(f" {'':23s} check: (OR11/OR10)/OR01 = {ratio / o01[0]:.3f} = interaction OR {REC[k]['MI']:.3f}")
rng = [WITHIN[k][0] for k in ROWS]
print(f" range OR11/OR10: {min(rng):.2f}-{max(rng):.2f}; range OR01: "
f"{min(T2[k]['OR01'][0] for k in ROWS):.2f}-{max(T2[k]['OR01'][0] for k in ROWS):.2f}")
# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
W = 7.2
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
TICKS = [0.1, 0.2, 0.5, 1, 2, 4]
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
plt.close(fig)
def log_axis(ax, lo=0.1, hi=4.6):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
tk = [t for t in TICKS if lo <= t <= hi]
ax.set_xticks(tk)
ax.set_xticklabels([str(t) for t in tk])
ax.set_xlim(lo, hi)
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvline(1, color=INK2, lw=0.9, zorder=1)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# ---------------------------------------------------------------- candidate 1: two-series forest plot
def cand1(name):
fig, ax = plt.subplots(figsize=(W, 4.0))
y = np.arange(len(ROWS))[::-1].astype(float)
off = 0.17
for yi, k in zip(y, ROWS):
a = ABSTRACT[k]
ax.plot([a[1], a[2]], [yi + off] * 2, color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(a[0], yi + off, "o", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(4.5, yi + off, f"{f2(a[0])} ({f2(a[1])}–{f2(a[2])}), p≈{P_ABS[k]:.2f}",
va="center", ha="left", fontsize=10, color=INK)
w = WITHIN[k]
ax.plot([w[1], w[2]], [yi - off] * 2, color=BLUE, lw=2, solid_capstyle="round")
ax.plot(w[0], yi - off, "s", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(4.5, yi - off, f"{f2(w[0])} (≈{f2(w[1])}–{f2(w[2])})", va="center", ha="left",
fontsize=10, color=INK2)
log_axis(ax, 0.17, 4.4)
ax.set_yticks(y)
ax.set_yticklabels([SHORT[k] for k in ROWS], fontsize=10.5, color=INK)
ax.set_ylim(-0.7, len(ROWS) - 0.3)
ax.set_xlabel("Odds ratio for early death (log scale)", color=INK2)
leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, mec=BG, label="Reported: 'AOR' cited in the abstract for correct fluid\n(= Table 2 multiplicative-interaction OR)"),
Line2D([], [], color=BLUE, marker="s", ms=8, mec=BG, label="Recomputed: correct vs wrong fluid, symptom present\n(OR11/OR10 from Table 2; approximate CI)")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, -0.17), ncol=2, frameon=False, fontsize=9.6 if False else 10,
handlelength=1.6, columnspacing=1.2)
ax.set_title("Abstract: “significantly reduced the risk of early mortality … in all subgroups”",
loc="left", fontsize=11, color=INK, pad=10)
save(fig, name)
# ---------------------------------------------------------------- candidate 2: abstract vs Table 2 text panels
def cand2(name):
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(-6, 54)
ax.axis("off")
def panel(x0, w, title, color):
ax.add_patch(FancyBboxPatch((x0, 4), w, 48, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="#ffffff", ec=color, lw=2))
ax.text(x0 + 2, 49, title, fontsize=11, weight="bold", va="top", color=INK)
panel(1, 43, "Abstract, Findings", ORANGE)
abs_txt = ("“Correct fluid prescription significantly\n reduced the risk of early mortality\n"
" (within 2 days) in all subgroups:”")
ax.text(3, 43.5, abs_txt, fontsize=10, va="top", color=INK, style="italic")
yy = 31
for k in ROWS:
a = ABSTRACT[k]
ax.text(4, yy, SHORT[k], fontsize=10, va="center", color=INK2)
ax.text(42, yy, f"AOR {f2(a[0])} ({f2(a[1])}–{f2(a[2])})", fontsize=10, va="center", ha="right", color=INK)
yy -= 5
ax.text(3, 6.5, "values as printed (reported)", fontsize=10, color=INK2)
panel(56, 43, "Table 2, “Multiplicative interaction”", BLUE)
ax.text(58, 43.5, "Footnote: interaction = OR11/(OR10 × OR01)\nRecomputed from the table's own cells:",
fontsize=10, va="top", color=INK)
yy = 31
for k in ROWS:
a = T2[k]["MI"]
ax.text(58, yy, SHORT[k], fontsize=10, va="center", color=INK2)
ax.text(97, yy, f"{f2(a[0])} ({f2(a[1])}–{f2(a[2])}) → {REC[k]['MI']:.2f}",
fontsize=10, va="center", ha="right", color=INK)
yy -= 5
ax.text(58, 6.5, "printed (CI) → recomputed from cells", fontsize=10, color=INK2)
ax.add_patch(FancyArrowPatch((44.8, 21), (55.2, 21), arrowstyle="<|-|>", mutation_scale=14, color=INK2, lw=2))
ax.text(50, 23.5, "same\nfive\nnumbers", ha="center", va="bottom", fontsize=10, color=INK)
ax.text(50, -1.5, "The abstract cites the interaction ORs as “AOR” for correct fluid. "
"They are ratios of two effects, not effects of correct fluid; 0 of 5 CIs exclude 1.",
ha="center", va="center", fontsize=10, color=INK, wrap=True)
save(fig, name)
# ---------------------------------------------------------------- candidate 3: calculation flow grid
def cand3(name):
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 52)
ax.axis("off")
cols = [(20, "OR10\nwrong fluid,\nsign present"), (33, "OR11\ncorrect fluid,\nsign present"),
(46, "OR01\ncorrect fluid,\nsign absent"), (66, "OR11/(OR10·OR01)\nrecomputed ≈\nabstract 'AOR'"),
(86, "OR11/OR10\ncorrect vs wrong,\nsign present")]
for x, lab in cols:
ax.text(x, 49, lab, ha="center", va="top", fontsize=10, color=INK, linespacing=1.15)
ax.text(33, 51.5, "Table 2, as printed (reported)", ha="center", fontsize=10, color=ORANGE, weight="bold")
ax.text(75, 51.5, "computed from those cells (recomputed)", ha="center", fontsize=10, color=BLUE, weight="bold")
ax.plot([13, 52], [37.5, 37.5], color=ORANGE, lw=2)
ax.plot([55, 97], [37.5, 37.5], color=BLUE, lw=2)
yy = 32
for k in ROWS:
d = T2[k]
ax.text(1, yy, SHORT[k], va="center", fontsize=10, color=INK)
for x, key in ((20, "OR10"), (33, "OR11"), (46, "OR01")):
ax.text(x, yy, f2(d[key][0]), ha="center", va="center", fontsize=10.5, color=INK)
ax.annotate("", xy=(54.5, yy), xytext=(50.5, yy), arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.2))
ax.text(66, yy, f"{REC[k]['MI']:.2f} (CI {f2(d['MI'][1])}–{f2(d['MI'][2])})", ha="center", va="center", fontsize=10.5, color=INK)
ax.text(86, yy, f"{WITHIN[k][0]:.2f}", ha="center", va="center", fontsize=10.5, color=INK, weight="bold")
yy -= 5.6
ax.text(65, 2.5, "all 5 CIs include 1", ha="center", fontsize=10, color=INK2)
ax.text(88, 2.5, "all 5 below 1", ha="center", fontsize=10, color=INK2)
save(fig, name)
# ---------------------------------------------------------------- candidate 4: interaction plot, small multiples
def cand4(name):
fig, axes = plt.subplots(1, 5, figsize=(W, 3.5), sharey=True)
for ax, k in zip(axes, ROWS):
d = T2[k]
ax.plot([0, 1], [1, d["OR01"][0]], color=ORANGE, lw=2, ls=(0, (2.5, 1.5)), marker="o", ms=8, mfc=BG, mec=ORANGE, mew=2)
ax.plot([0, 1], [d["OR10"][0], d["OR11"][0]], color=ORANGE, lw=2, marker="o", ms=8, mec=ORANGE, mew=1.2)
ax.set_yscale("log")
ax.set_ylim(0.13, 5)
ax.set_yticks([0.25, 0.5, 1, 2, 4])
ax.set_yticklabels(["0.25", "0.5", "1", "2", "4"])
ax.minorticks_off()
ax.axhline(1, color=INK2, lw=0.9, zorder=0)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_xlim(-0.35, 1.35)
ax.set_xticks([0, 1])
ax.set_xticklabels(["wrong", "correct"], fontsize=10)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title(f"{SHORT[k].replace(' only', '')}\ninteraction {f2(d['MI'][0])}", fontsize=10, color=INK)
ax.text(0.5, 0.03, f"×{WITHIN[k][0]:.2f} | ×{d['OR01'][0]:.2f}", transform=ax.transAxes,
ha="center", fontsize=10, color=BLUE)
axes[0].set_ylabel("OR for early death vs\nwrong fluid, sign absent (log)", color=INK2)
fig.supxlabel("Fluid prescription at admission", fontsize=10, color=INK2, y=0.06)
leg = [Line2D([], [], color=ORANGE, lw=2.5, marker="o", ms=8, mec=ORANGE, label="Reported, sign present (OR10 → OR11): solid line, filled dots"),
Line2D([], [], color=ORANGE, lw=2.5, ls=(0, (2.5, 1.5)), marker="o", ms=8, mfc=BG, mec=ORANGE, mew=2, label="Reported, sign absent (1 → OR01): dashed line, open dots"),
Line2D([], [], color=BLUE, lw=0, marker="$×$", ms=9, label="Recomputed change with correct fluid: present | absent")]
fig.legend(handles=leg, loc="lower center", ncol=1, frameon=False, fontsize=10, bbox_to_anchor=(0.5, -0.2), handlelength=4)
fig.tight_layout(rect=(0, 0.04, 1, 1))
save(fig, name)
# ---------------------------------------------------------------- candidate 5: cited values only, with p
def cand5(name):
fig, ax = plt.subplots(figsize=(W, 3.2))
y = np.arange(len(ROWS))[::-1].astype(float)
ax.axvspan(0.3, 1, color="#f1f0ec", zorder=0)
ax.text(0.31, len(ROWS) - 0.3, "OR < 1 (shaded)",
ha="left", va="center", fontsize=10, color=INK2)
for yi, k in zip(y, ROWS):
a = ABSTRACT[k]
ax.plot([a[1], a[2]], [yi, yi], color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(a[0], yi, "o", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(3.6, yi, f"p ≈ {P_ABS[k]:.2f}", va="center", fontsize=10.5, color=BLUE)
log_axis(ax, 0.3, 3.4)
ax.set_xticks([0.3, 0.5, 1, 2, 3])
ax.set_xticklabels(["0.3", "0.5", "1", "2", "3"])
ax.set_yticks(y)
ax.set_yticklabels([SHORT[k] for k in ROWS], color=INK)
ax.set_ylim(-0.6, len(ROWS) + 0.05)
ax.set_xlabel("'AOR' (95% CI) cited in the abstract, log scale", color=INK2)
leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, mec=BG, label="Reported in the abstract"),
Line2D([], [], color=BLUE, lw=0, marker="$p$", ms=9, label="Recomputed two-sided p from the printed CI")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False, fontsize=10)
save(fig, name)
# ---------------------------------------------------------------- figure 2: RERI cells
def fig2(name):
fig, ax = plt.subplots(figsize=(W, 3.4))
y = np.arange(len(ROWS))[::-1].astype(float)
off = 0.16
for yi, k in zip(y, ROWS):
d = T2[k]["RERI"]
lo, hi = sorted((d[1], d[2]))
ax.plot([lo, hi], [yi + off] * 2, color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(d[0], yi + off, "o", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(REC[k]["RERI"], yi - off, "s", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
mid = (d[1] + d[2]) / 2
ax.plot(mid, yi + off, "|", ms=12, mew=2, color=INK2, zorder=4)
k = "Dehydration signs only"
yd = y[ROWS.index(k)]
ax.annotate(f"printed {f2(T2[k]['RERI'][0])}; cells give {REC[k]['RERI']:.2f};\nCI midpoint {(T2[k]['RERI'][1] + T2[k]['RERI'][2]) / 2:.2f}",
xy=(0.5, yd), xytext=(1.2, yd), va="center", fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
k = "Respiratory"
yr = y[ROWS.index(k)]
ax.annotate("printed CI “−2·16 to −9·45”:\nupper limit below point and lower limit",
xy=(-2.16, yr + off), xytext=(-7.5, yr - 0.62), va="center", fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.axvline(0, color=INK2, lw=0.9)
ax.set_xlim(-10, 3.2)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_yticks(y)
ax.set_yticklabels([SHORT[k] for k in ROWS], color=INK)
ax.set_xlabel("RERI (relative excess risk due to interaction), linear scale; null = 0", color=INK2)
leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, mec=BG, label="Reported RERI (95% CI as printed)"),
Line2D([], [], color=BLUE, lw=0, marker="s", ms=8, mec=BG, label="Recomputed OR11 − OR10 − OR01 + 1"),
Line2D([], [], color=INK2, lw=0, marker="|", ms=12, mew=2, label="Midpoint of printed CI")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=3, frameon=False, fontsize=10,
columnspacing=1.0, handletextpad=0.4)
save(fig, name)
for fn, nm in ((cand1, "candidate1"), (cand2, "candidate2"), (cand3, "candidate3"),
(cand4, "candidate4"), (cand5, "candidate5"), (cand1, "fig1"), (fig2, "fig2")):
fn(nm)
print("\nFigures written to", HERE)
Several adjusted odds ratios in the abstract have confidence intervals narrower than the data allow
Lancet Global Health (2025) · doi:10.1016/S2214-109X(25)00061-0 · PMC12208783
Summary
The Abstract, Results and Table 2 report adjusted odds ratios (aORs) for catastrophic health expenditure (CHE; 76 cases among 296 participants) of 2·0 (95% CI 1·9–2·1, p<0·0001) for the second vs first poverty quintile (60 participants each) and 9·7 (9·6–9·8) for subcounty hospitals vs health centres (69 vs 164 participants; 9·8 (9·7–9·9) in Table 2). For a 0/1 covariate held by this many participants, no model-based maximum-likelihood logistic regression can give a standard error below a floor set by the group sizes, and the standard errors implied by the two reported intervals are 11 and 52 times smaller than that floor; three further abstract aORs (female sex, education above secondary school, county hospital) are also below it, by smaller margins. With the standard error implied by the Table 1 counts, the second-quintile aOR becomes 2.0 (0.86–4.67, p = 0.11), close to the paper's own five-facility sensitivity estimate of 2.1 (0.70–6.1) in Supplementary Table 4. The second quintile is the only poverty quintile reported with significantly raised adjusted odds, and the abstract's conclusion that CHE is "driven by socioeconomic … inequalities" leans on it; the facility-level associations keep their direction and significance, but their stated precision is overstated about 50-fold.
What the paper reports
CHE was associated with being aged 30–44 years old (aOR 2·6, 95% CI 1·3–5·3, p=0·010), being female (1·8, 1·3–2·7, p=0·0021), having higher than secondary school education (1·6, 1·1–2·3, p=0·0083), being accompanied during health-care-seeking visits (3·2, 1·7–5·9, p<0·0001), belonging to the second poverty quintile (2·0, 1·9–2·1, p<0·0001), and seeking care from subcounty hospitals (9·7, 9·6–9·8, p<0·0001) and county hospitals (25·1, 15·7–40·2, p<0·0001).
Abstract, Findings
…belonging to the second poverty quintile (2·0, 1·9–2·1, p<0·0001), and seeking care from subcounty (9·7, 9·6–9·8, p<0·0001) and county hospitals (25·1, 15·7–40·2, p<0·0001; table 2).
Results, paragraph on CHE incidence
Table 2 ("Univariable and multivariable logistic regression of health and social factors associated with catastrophic health expenditure"). Its columns are the unadjusted OR (n=296), p, the adjusted OR (n=296) and p. Two of its rows read:
Second quintile | 1·4 (0·8–2·4) | 0·30 | 2·0 (1·9–2·1) | <0·0001
Subcounty hospital | 9·7 (8·4–11·1) | <0·0001 | 9·8 (9·7–9·9) | <0·0001
Table 2
Table 1 gives the counts behind these rows. The first quintile has 13 CHE and 47 no CHE, the second quintile 15 and 45, health centres 11 and 153, subcounty hospitals 27 and 42, and the county hospital 38 and 25.
How the paper uses these results:
…driven by socioeconomic and sex inequalities and impaired access to health care and social protection.
Abstract, Interpretation
Independent drivers of CHE included being female, poverty, being accompanied by a carer during health-care seeking, and seeking care at subcounty or county hospitals.
Research in context, Added value of this study
Our findings showed that poorer quintiles were more likely to incur CHE than the richest quintile, although the relationship across poverty quintiles did not always reach significance and appeared non-linear.
Discussion, paragraph 3
On the analysis, the paper says "Univariable and multivariable logistic regression models generated unadjusted and adjusted odds ratios", that "for the 21 participants without household asset data, multiple imputation with chained equations was also used to impute poverty scores", and "Data were analysed in STATA (version 15) and R (version 4)." (Methods, Statistical analysis). The paper does not describe a clustered, robust or otherwise non-standard variance estimator.
What the paper's own numbers show
Method. For a logistic regression coefficient on a 0/1 indicator held by n₁ of N participants, the model-based variance is the corresponding diagonal element of the inverse Fisher information. For any positive-definite matrix, a diagonal element of the inverse is at least the corresponding element of the inverse of any principal submatrix that contains it, so [I⁻¹]ⱼⱼ ≥ [(I_sub)⁻¹]ⱼⱼ, where I_sub is the intercept-and-indicator block of the same information matrix, evaluated at the full model's fitted values p. That block's inverse gives 1/S₁ + 1/S₀, where S₁ and S₀ sum p(1 − p) over participants with and without the indicator. Since p(1 − p) ≤ 1/4 always, SE ≥ √(4/n₁ + 4/(N − n₁)) for any covariate set and any fitted values. Pooling multiply imputed fits by Rubin's rules can only add variance. Separately, I derived each reported interval's implied SE, ln(upper/lower)/(2 × 1.96), including the full range allowed by rounding the printed limits to one decimal place. I computed crude ORs with Wald intervals from the Table 1 counts. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| aOR, second vs first poverty quintile | Abstract; Results; Table 2 | 2·0 (1·9–2·1), p<0·0001 | Implied SE 0.0255 (0.013–0.038 allowing for rounding), z = 27. Minimum possible SE with n₁ = 60: 0.289, which is 11× the implied SE and 7.5× the widest SE rounding allows. Narrowest possible 95% CI at aOR 2.0: 1.13–3.53 (p = 0.017) |
| Same, allowing for imputation of quintile in 21 participants | derived | — | If n₁ is anywhere from 39 to 81, the minimum SE is still 0.261, 10× the implied SE |
| Crude OR, second vs first quintile (15/60 vs 13/60) | Table 1 | Table 2 unadjusted: 1·4 (0·8–2·4), p=0·30 | 1.21 (0.52–2.81), SE 0.433, p = 0.67 (Fisher exact p = 0.83). Crude SE ÷ reported adjusted SE = 17 (11–34 allowing for rounding) |
| aOR 2.0 with the count-based SE | derived | — | 95% CI 0.86–4.67, p = 0.11 |
| Second quintile, same outcome and covariates with a five-level facility term | Supplementary Table 4 | 2.1 (0.70–6.1), p=0.099 | Implied SE 0.552, 22× the Table 2 SE. The interval includes 1 (the Wald p from this interval is 0.18) |
| aOR, subcounty hospital vs health centre | Abstract; Results | 9·7 (9·6–9·8), p<0·0001 | Implied SE 0.0053. Minimum possible SE with n₁ = 69: 0.275, 52× larger (35× the widest SE rounding allows). Narrowest possible CI at 9.7: 5.66–16.63 |
| aOR, subcounty hospital | Table 2 | 9·8 (9·7–9·9) | Differs from the Abstract and Results (9·7, 9·6–9·8). Same problem: minimum possible CI at 9.8 is 5.72–16.80 |
| Crude OR, subcounty vs health centre (27/69 vs 11/164) | Table 1 | Table 2 unadjusted: 9·7 (8·4–11·1) | 8.94 (4.10–19.50), SE 0.398, which is 76× the SE implied by the abstract's adjusted interval. aOR 9.7 with this SE: 4.4–21.2 |
| aOR, female | Abstract; Table 2 | 1·8 (1·3–2·7), p=0·0021 | Implied SE 0.186 (at most 0.201 allowing for rounding) vs minimum 0.236. Narrowest possible CI 1.13–2.86, p = 0.013 |
| aOR, above secondary education | Abstract; Table 2 | 1·6 (1·1–2·3), p=0·0083 | Implied SE 0.188 (at most 0.206) vs minimum 0.344. Narrowest possible CI 0.82–3.14, p = 0.17. Supplementary Table 4 gives 1.4 (0.50–4.2), p = 0.50 |
| aOR, county hospital | Abstract; Table 2 | 25·1 (15·7–40·2) | Implied SE 0.240 (at most 0.241) vs minimum 0.284. Narrowest possible CI 14.4–43.8. Still strongly significant |
| aOR, third quintile | Table 2 | 0·7 (0·6–0·9), p=0·0009 | Implied SE 0.103 (at most 0.139) vs minimum 0.293. Narrowest possible CI 0.39–1.24. Supplementary Table 4 gives 0.73 (0.22–2.4), p = 0.60 |
| aORs for age 30–44, accompanied, fourth and fifth quintile, NHIF | Abstract; Table 2 | e.g. 2·6 (1·3–5·3); 3·2 (1·7–5·9) | Consistent with the minimum (implied SE ≥ minimum) |
| Unadjusted secondary vs primary education (23/94 vs 42/163) | Table 1; Table 2 | 1·3 (1·2–1·5), p<0·0001 | Crude 0.93 (0.52–1.68), p = 0.82. Table 1 gives p = 0·90 for education |
| Unadjusted age 30–44 vs 18–29 (39/117 vs 14/103) | Table 1; Table 2 | 3·7 (2·9–4·6) | Crude 3.18 (1.61–6.29), SE 0.348 vs implied 0.118 |
| Unadjusted county vs health centre (38/63 vs 11/164) | Table 1; Table 2 | 24 (19·9–28·6) | Crude 21.1 (9.56–46.7), SE 0.405 vs implied 0.093 |
In the Abstract, 5 of the 7 adjusted associations have reported intervals narrower than any maximum-likelihood logistic fit to these data can produce. The violation is severe for the second quintile (11×) and subcounty hospital (52×). It is moderate for education above secondary school (1.8×). It is small for female sex (1.3×) and county hospital (1.2×). The bound is conservative: it assumes every fitted probability is 0.5, whereas the overall CHE rate is 26%. Realistic standard errors are larger, as the crude SEs and Supplementary Table 4 show.
The univariable column of Table 2 has the same pattern. Several of its intervals are much narrower than the crude counts give, and the secondary-education OR points the opposite way from the counts. That column appears unchanged in Supplementary Table 4, for example 3.7 (2.9–4.6), 1.3 (1.2–1.5) and 1.4 (0.76–2.4).
Figure
Figure 1. Odds ratios of catastrophic health expenditure with 95% CIs on a log scale. The thin vertical line marks OR = 1. Orange shows values the paper reports. For the second poverty quintile these are the main-model aOR 2.0 (1.9–2.1) from the Abstract, Results and Table 2 (diamond), and the same model with a five-level facility term from Supplementary Table 4 (square). For subcounty hospitals they are the Abstract and Results value 9.7 (9.6–9.8) and the Table 2 value 9.8 (9.7–9.9). Blue shows values recomputed from the paper's own numbers. The first blue row in each group is the narrowest interval any maximum-likelihood logistic fit could give at the reported point estimate. The second keeps the reported point estimate but uses the standard error of the crude log OR from the Table 1 counts. The third is the crude OR and Wald interval from those counts. The reported main-model intervals are hidden behind their markers. For the second quintile, the interval from the paper's own sensitivity model, the interval with the count-based SE (0.86–4.67) and the crude interval all include 1.
Figure 2. For each of the seven adjusted odds ratios in the Abstract (point estimate in brackets), the orange diamond is the standard error of the log OR implied by the reported 95% CI. Its bar is the range allowed by rounding the printed limits. The blue dot is the smallest standard error any maximum-likelihood logistic regression can give for that indicator, √(4/n₁ + 4/(296 − n₁)), with n₁ from Table 1. The shaded band to its right is the attainable range. Five of the seven reported intervals fall below this floor. The second-poverty-quintile and subcounty-hospital estimates are 11 and 52 times below it. The axis is logarithmic.
Why it matters
- The poverty finding. The second quintile is the only poverty quintile reported with significantly raised odds of CHE. The fourth and fifth quintile CIs include 1. The third quintile is reported as significantly lower, 0·7 (0·6–0·9), p=0·0009, but that interval is also narrower than the floor allows (narrowest possible CI 0.39–1.24), and Supplementary Table 4 gives 0.73 (0.22–2.4), p=0.60. With an attainable standard error, the second-quintile association is at most borderline: p = 0.017 at the theoretical minimum SE and p = 0.11 at the count-based SE. The paper's own five-facility sensitivity model gives 2.1 (0.70–6.1). The statements that CHE is "driven by socioeconomic … inequalities" (Abstract, Interpretation) and that poverty is an "independent driver" (Research in context) depend on this estimate. The Discussion already notes that the quintile relationship "did not always reach significance and appeared non-linear".
- Education and sex. At the minimum attainable SE, education above secondary school is no longer significant (narrowest CI 0.82–3.14). Supplementary Table 4 gives 1.4 (0.50–4.2), p = 0.50 for it, and 2.0 (0.95–4.0), p = 0.07 for female sex. The female aOR stays nominally significant at the bound (p = 0.013), but the bound is a floor, not an estimate.
- Facility level. The subcounty and county hospital associations keep their direction and remain highly significant with any plausible SE. Only their stated precision is wrong, by about 50-fold for subcounty hospitals.
- What stands. The CHE incidence (76/296, 26%), the cost breakdown, the NHIF coverage figures and the coping-strategy results do not depend on these intervals.
- Downstream use. A meta-analysis that takes 2.0 (1.9–2.1) or 9.7 (9.6–9.8) at face value would weight these estimates by inverse variance, that is, about 130 times (second quintile, 11.3²) and about 2,700 times (subcounty, 52.3²) more than even the theoretical floor on the standard error allows.
Explanations considered
- Rounding of the printed limits. Does not account for it. The widest interval consistent with "1·9–2·1" is 1.85–2.15, with SE 0.038, still 7.5 times below the floor. For "9·6–9·8" the factor is 35.
- A typo in one limit. Does not account for it. The second-quintile interval appears identically in the Abstract, Results and Table 2, and its p<0·0001 matches the interval (z ≈ 27). The subcounty value differs between Abstract (9·7, 9·6–9·8) and Table 2 (9·8, 9·7–9·9), but both are equally narrow. Four other adjusted intervals and several univariable intervals are also below the floor, so this is not one mistyped number.
- Covariate adjustment. Does not account for it. Adding covariates cannot push a model-based logistic-regression SE below √(4/n₁ + 4/(N − n₁)), as shown in the Method.
- Multiple imputation (21 participants without household asset data (Methods); 26 participants with missing income, expenditure or poverty data (Discussion)). Does not account for it. Rubin's total variance is at least the average within-imputation variance, and each of those obeys the bound. Even if imputation moved all 21 participants into or out of the second quintile, the bound would still be 0.261, 10 times the implied SE.
- Exact, penalised (Firth) or bootstrap intervals. None is mentioned in the paper, and for cells of this size (11 to 153 participants per cell in Table 1) such methods give intervals of similar width to the Wald interval, not 10–50 times narrower.
- A cluster-robust (sandwich) variance with facilities as clusters. This is the one standard estimator not bound by the inequality above. The paper does not mention it, and with five facilities, one of which forms the whole county-hospital level, such estimates are known to be unstable and too small. As a check, I computed the facility-clustered SE for the unadjusted three-level model from the facility counts in Supplementary Table 1b. These counts sum exactly to the Table 1 level totals. The result for county hospital (0.097, or 0.108 with the G/(G − 1) factor) is close to the 0.093 implied by Table 2. For subcounty hospital the result is 0.265–0.296, against 0.071 implied by Table 2. The Table 2 unadjusted ORs also differ from the crude ORs (9.7 vs 8.94; 24 vs 21.1). So this mechanism reproduces some of the reported intervals but not others. It cannot be confirmed without the data, and it would not make the reported intervals valid.
- A different sample or outcome in the model. Not supported. Table 2 states n=296 for both columns and the same 10% CHE outcome.
Status
- Version of record. The publisher's pages (thelancet.com, sciencedirect.com) and PDF were not reachable by automated access (HTTP 403/429). The open-access text deposited by the publisher was checked in the Europe PMC full-text XML for PMC12208783 (fetched 2026-10-23), the PMC article page and the local PMC archive copy. All carry 2·0 (1·9–2·1) and 9·7 (9·6–9·8) in the Abstract and Results, and 2·0 (1·9–2·1) and 9·8 (9·7–9·9) in Table 2. Every quotation above was checked verbatim against the archive copy. These are numbers in running text and table cells, not a rendering or conversion artefact.
- Supplements. Supplementary appendix 1 (mmc1.pdf, 2,538,411 bytes, md5 02ee6b399598cdef96ddbb2b8a1936e9, from the Europe PMC supplementary files) was read in full text. It contains Supplementary Table 4 (same 10% CHE outcome and covariates, with a five-level facility term), where the second quintile is 2.1 (0.70–6.1), p=0.099, and Supplementary Table 1b, which gives CHE by facility (5/85, 6/79, 12/38, 15/31, 38/63; these sum to the Table 1 level totals). It does not describe the variance estimator of the main model, and no individual-level data or code are provided. mmc2.pdf is a partnership declaration with no relevant data.
- Corrections. Crossref lists no update-to or updated-by relations, and PMC and Europe PMC list no correction, retraction or comment notice (checked 2026-10-23).
- PubPeer. pubpeer.com refused automated access (HTTP 403), and a web search could not be run in this session, so the absence of a PubPeer thread is not confirmed. Europe PMC's link services for PMID 40580995 list only a data citation and Altmetric, with no PubPeer link.
- As of 2026-10-23.
Suggested correction
- Re-estimate Table 2, both columns, with model-based maximum-likelihood standard errors, or state and justify the variance estimator used. With five facilities, a cluster-robust estimator is not appropriate without small-cluster corrections. Report the method in the Statistical analysis section.
- Replace the second-quintile, subcounty-hospital, female, above-secondary, county-hospital and third-quintile aOR intervals and p values in the Abstract, Results and Table 2. Also replace the univariable intervals that do not match the Table 1 counts.
- Reconcile the subcounty-hospital aOR between the Abstract and Results (9·7, 9·6–9·8) and Table 2 (9·8, 9·7–9·9).
- If the corrected second-quintile and above-secondary-education intervals include 1, revise the statements that poverty is an independent driver of CHE (Abstract Interpretation, Research in context, Discussion) and the abstract's list of associations accordingly.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC12208783 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
All inputs are copied from the article (Table 1, Table 2, Abstract) or from its
Supplementary appendix 1 (mmc1.pdf: Supplementary Tables 1b and 4).
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
N, CHE = 296, 76
def crude(a, b, c, d):
"""a = CHE exposed, b = no CHE exposed, c = CHE reference, d = no CHE reference."""
OR = (a / b) / (c / d)
se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
lo, hi = np.exp(np.log(OR) - Z * se), np.exp(np.log(OR) + Z * se)
p = 2 * stats.norm.sf(abs(np.log(OR)) / se)
return OR, lo, hi, se, p
def se_from_ci(lo, hi):
return np.log(hi / lo) / (2 * Z)
def se_rounding_range(lo, hi, h=0.05):
"""(narrowest, widest) SE consistent with limits printed to one decimal place."""
return se_from_ci(lo + h, hi - h), se_from_ci(lo - h, hi + h)
def se_min(n1, n=N):
"""Hard lower bound on the model-based (ML) SE of a logistic coefficient for a
0/1 indicator held by n1 of n participants, whatever the other covariates:
Var(b_j) = [I^-1]_jj >= [(I restricted to intercept and x_j)^-1]_jj = 1/S1 + 1/S0,
with S1 = sum over x_j = 1 of p(1-p) <= n1/4 and S0 <= (n - n1)/4."""
return np.sqrt(4 / n1 + 4 / (n - n1))
def ci(OR, se):
return OR * np.exp(-Z * se), OR * np.exp(Z * se)
def p_of(OR, se):
return 2 * stats.norm.sf(abs(np.log(OR)) / se)
print("Totals from Table 1: CHE", CHE, "of", N, f"({CHE / N:.1%})")
# ---------------------------------------------------------------------------
print("\n=== 1. Crude ORs from Table 1 counts vs Table 2 'unadjusted' column ===")
rows = [
("Age 30-44 vs 18-29", 39, 78, 14, 89, "3.7 (2.9-4.6), p=0.0010"),
("Female vs male", 42, 131, 34, 89, "0.7 (0.4-2.0), p=0.71"),
("Secondary vs primary", 23, 71, 42, 121, "1.3 (1.2-1.5), p<0.0001"),
("Above secondary vs primary", 11, 28, 42, 121, "1.7 (1.0-2.8), p=0.053"),
("Quintile 2 vs 1", 15, 45, 13, 47, "1.4 (0.8-2.4), p=0.30"),
("Quintile 3 vs 1", 10, 48, 13, 47, "1.0 (0.8-1.3), p=0.91"),
("Quintile 5 vs 1", 24, 35, 13, 47, "2.2 (1.1-4.7), p=0.035"),
("Subcounty vs health centre", 27, 42, 11, 153, "9.7 (8.4-11.1), p<0.0001"),
("County vs health centre", 38, 25, 11, 153, "24 (19.9-28.6), p<0.0001"),
]
CR = {}
for lab, a, b, c, d, rep in rows:
OR, lo, hi, se, p = crude(a, b, c, d)
CR[lab] = (OR, lo, hi, se, p)
print(f"{lab:28s} crude {OR:5.2f} ({lo:5.2f}-{hi:5.2f}) SE {se:.3f} p={p:.2g} | Table 2: {rep}")
print("Fisher exact p, quintile 2 vs 1:", f"{stats.fisher_exact([[15, 45], [13, 47]])[1]:.2f}")
print("Fisher exact p, secondary vs primary:", f"{stats.fisher_exact([[23, 71], [42, 121]])[1]:.2f}")
# ---------------------------------------------------------------------------
print("\n=== 2. Reported adjusted ORs vs the hard lower bound on any ML standard error ===")
# label, n1 (participants in the category, Table 1), aOR, lo, hi, where, in_abstract
adj = [
("Age 30-44", 117, 2.6, 1.3, 5.3, True),
("Female", 173, 1.8, 1.3, 2.7, True),
("Above secondary", 39, 1.6, 1.1, 2.3, True),
("Accompanied", 59, 3.2, 1.7, 5.9, True),
("Second quintile", 60, 2.0, 1.9, 2.1, True),
("Subcounty (abstract/text)", 69, 9.7, 9.6, 9.8, True),
("County", 63, 25.1, 15.7, 40.2, True),
("Subcounty (Table 2)", 69, 9.8, 9.7, 9.9, False),
("Third quintile", 58, 0.7, 0.6, 0.9, False),
("Fourth quintile", 59, 0.5, 0.2, 1.2, False),
("Fifth quintile", 59, 2.1, 0.6, 7.7, False),
("NHIF no", 212, 1.5, 0.7, 3.5, False),
]
print(f"{'term':26s} {'reported':>18s} {'SE_rep':>7s} {'SE range (rounding)':>20s} {'SE_min':>7s} {'SE_min/SE_rep':>13s} min CI at reported aOR verdict")
FIG2 = []
for lab, n1, OR, lo, hi, abst in adj:
se = se_from_ci(lo, hi)
s_lo, s_hi = se_rounding_range(lo, hi)
smin = se_min(n1)
verdict = "BELOW BOUND" if s_hi < smin else "consistent"
clo, chi = ci(OR, smin)
print(f"{lab:26s} {OR:5.1f} ({lo:4.1f}-{hi:4.1f}) {se:7.4f} {s_lo:9.4f}-{s_hi:.4f} {smin:7.3f} {smin / se:13.1f} {clo:6.2f}-{chi:6.2f} {verdict}")
if abst:
FIG2.append((lab, OR, lo, hi, se, s_lo, s_hi, smin))
n_abs_fail = sum(1 for r in FIG2 if r[6] < r[7])
print(f"Abstract aORs whose reported CI is narrower than any ML fit allows: {n_abs_fail} of {len(FIG2)}")
for _lab, _n1, _lo, _hi in [("Second quintile", 60, 1.9, 2.1), ("Subcounty (abstract/text)", 69, 9.6, 9.8)]:
_r = se_min(_n1) / se_from_ci(_lo, _hi)
print(f"Inverse-variance weight inflation vs the floor, {_lab}: ({_r:.1f})^2 = {_r ** 2:.0f}x")
print("p-value if each abstract aOR kept its point estimate but had the minimum possible SE:")
for lab, OR, lo, hi, se, s_lo, s_hi, smin in FIG2:
print(f" {lab:26s} reported-CI p {p_of(OR, se):.2g} p at minimum SE {p_of(OR, smin):.2g}")
# ---------------------------------------------------------------------------
print("\n=== 3. Second quintile: the reported aOR 2.0 (1.9-2.1) in context ===")
se_rep = se_from_ci(1.9, 2.1)
s_lo, s_hi = se_rounding_range(1.9, 2.1)
print(f"implied SE {se_rep:.4f} (rounding range {s_lo:.4f}-{s_hi:.4f}); implied z = {np.log(2.0) / se_rep:.1f}")
crude_q2 = CR["Quintile 2 vs 1"]
print(f"crude SE {crude_q2[3]:.3f}; ratio crude/reported {crude_q2[3] / se_rep:.0f} (rounding range {crude_q2[3] / s_hi:.0f}-{crude_q2[3] / s_lo:.0f})")
print(f"bound/reported {se_min(60) / se_rep:.1f}x; bound/widest rounding-consistent SE {se_min(60) / s_hi:.1f}x")
# Poverty quintile was imputed for 21 participants: allow n1 anywhere in 60 +/- 21
bmin = min(se_min(n1) for n1 in range(60 - 21, 60 + 22))
print(f"lower bound if n1 for Q2 is anywhere in 39..81 (imputation): {bmin:.3f} -> {bmin / se_rep:.0f}x the reported SE, {bmin / s_hi:.1f}x the widest rounding-consistent SE")
for name, s in [("hard minimum SE", se_min(60)), ("count-based crude SE", crude_q2[3])]:
lo2, hi2 = ci(2.0, s)
print(f"aOR 2.0 with {name} {s:.3f}: 95% CI {lo2:.2f}-{hi2:.2f}, p={p_of(2.0, s):.3f}")
se_s4 = se_from_ci(0.70, 6.1)
print(f"Supplementary Table 4 (same model, five-facility term): 2.1 (0.70-6.1), p=0.099; implied SE {se_s4:.3f} = {se_s4 / se_rep:.0f}x the Table 2 SE")
print(f" Wald p recomputed from that CI: {p_of(2.1, se_s4):.2f}")
se_q3 = se_from_ci(0.6, 0.9)
print(f"Third quintile: Table 2 aOR 0.7 (0.6-0.9), p=0.0009, implied SE {se_q3:.3f} vs bound {se_min(58):.3f}; "
f"narrowest CI at 0.7: {ci(0.7, se_min(58))[0]:.2f}-{ci(0.7, se_min(58))[1]:.2f}; "
f"Supplementary Table 4: 0.73 (0.22-2.4), p=0.60")
# ---------------------------------------------------------------------------
print("\n=== 4. Subcounty hospital: 9.7 (9.6-9.8) in the abstract and text, 9.8 (9.7-9.9) in Table 2 ===")
for lab, OR, lo, hi in [("abstract/text", 9.7, 9.6, 9.8), ("Table 2", 9.8, 9.7, 9.9)]:
se = se_from_ci(lo, hi)
s_lo, s_hi = se_rounding_range(lo, hi)
print(f"{lab:14s} implied SE {se:.4f} (rounding range {s_lo:.4f}-{s_hi:.4f}); bound {se_min(69):.3f} = {se_min(69) / se:.0f}x; bound/widest rounding-consistent SE {se_min(69) / s_hi:.0f}x")
cs = CR["Subcounty vs health centre"]
print(f"crude SE {cs[3]:.3f} = {cs[3] / se_from_ci(9.6, 9.8):.0f}x the abstract SE; aOR 9.7 with crude SE: CI {ci(9.7, cs[3])[0]:.1f}-{ci(9.7, cs[3])[1]:.1f}")
print(f"aOR 9.7 with hard minimum SE: CI {ci(9.7, se_min(69))[0]:.2f}-{ci(9.7, se_min(69))[1]:.2f}")
cc = CR["County vs health centre"]
print(f"County: crude SE {cc[3]:.3f}; reported adjusted SE {se_from_ci(15.7, 40.2):.3f}; bound {se_min(63):.3f}")
# ---------------------------------------------------------------------------
print("\n=== 5. Could a facility-clustered (sandwich) SE explain the narrow intervals? ===")
# Supplementary Table 1b ('Incurred catastrophic health expenditure' row): CHE / n by facility; level: 0 = health centre, 1 = subcounty, 2 = county
fac = [("Mitunguu HC", 0, 5, 85), ("Laare HC", 0, 6, 79), ("Mutuati SCH", 1, 12, 38),
("Kanyakine SCH", 1, 15, 31), ("Meru TRH", 2, 38, 63)]
lev_y = {L: sum(y for _, l, y, _ in fac if l == L) for L in range(3)}
lev_n = {L: sum(n for _, l, _, n in fac if l == L) for L in range(3)}
print("level CHE/n from facilities:", [f"{lev_y[L]}/{lev_n[L]}" for L in range(3)], "(Table 1: 11/164, 27/69, 38/63)")
# Univariable model with level dummies is saturated at the level, so fitted p = level proportion.
# Scores per facility for the level log-odds theta_L: u_g = y_g - n_g p_L. A_L = n_L p_L (1 - p_L).
G = len(fac)
var = {}
for L in range(3):
pL = lev_y[L] / lev_n[L]
A = lev_n[L] * pL * (1 - pL)
B = sum((y - n * pL) ** 2 for _, l, y, n in fac if l == L)
var[L] = B / A ** 2
for lab, k in [("without small-sample factor", 1.0), ("with G/(G-1)", G / (G - 1))]:
se_sub = np.sqrt(k * (var[0] + var[1]))
se_cty = np.sqrt(k * (var[0] + var[2]))
print(f"cluster-robust SE {lab}: subcounty {se_sub:.3f}, county {se_cty:.3f}")
print(f"Table 2 unadjusted implied SEs: subcounty {se_from_ci(8.4, 11.1):.3f}, county {se_from_ci(19.9, 28.6):.3f}")
# ---------------------------------------------------------------------------
# Figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
BG, INK, INK2, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.grid(axis="x", color=GRID, lw=0.8, which="both")
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
# ---- Figure 1: second quintile and subcounty hospital
q2min = ci(2.0, se_min(60))
scmin = ci(9.7, se_min(69))
items = [
# group, label, OR, lo, hi, kind (rep / rec), marker
("Second vs first poverty quintile", "Reported aOR (Abstract, Results, Table 2)", 2.0, 1.9, 2.1, "rep", "D"),
("Second vs first poverty quintile", "Reported aOR, Supplementary Table 4 (five-facility model)", 2.1, 0.70, 6.1, "rep", "s"),
("Second vs first poverty quintile", "Narrowest CI any logistic ML fit allows at aOR 2.0", 2.0, *q2min, "rec", "o"),
("Second vs first poverty quintile", "aOR 2.0 with the count-based SE from Table 1", 2.0, *ci(2.0, crude_q2[3]), "rec", "o"),
("Second vs first poverty quintile", "Crude OR from Table 1 counts (15/60 vs 13/60)", crude_q2[0], crude_q2[1], crude_q2[2], "rec", "o"),
("Subcounty hospital vs health centre", "Reported aOR (Abstract, Results)", 9.7, 9.6, 9.8, "rep", "D"),
("Subcounty hospital vs health centre", "Reported aOR (Table 2)", 9.8, 9.7, 9.9, "rep", "D"),
("Subcounty hospital vs health centre", "Narrowest CI any logistic ML fit allows at aOR 9.7", 9.7, *scmin, "rec", "o"),
("Subcounty hospital vs health centre", "aOR 9.7 with the count-based SE from Table 1", 9.7, *ci(9.7, cs[3]), "rec", "o"),
("Subcounty hospital vs health centre", "Crude OR from Table 1 counts (27/69 vs 11/164)", cs[0], cs[1], cs[2], "rec", "o"),
]
fig, ax = plt.subplots(figsize=(7.2, 7.0), facecolor=BG)
style(ax)
STEP, GAP = 1.25, 1.1
ypos, y, last = [], 0.0, None
for g, *_ in items:
if g != last and last is not None:
y += GAP
ypos.append(y)
y += STEP
last = g
ypos = -np.array(ypos)
reptxt = {0: "2.0 (1.9–2.1)", 1: "2.1 (0.70–6.1)", 5: "9.7 (9.6–9.8)", 6: "9.8 (9.7–9.9)"}
for i, ((g, lab, OR, lo, hi, kind, mk), yy) in enumerate(zip(items, ypos)):
col = ORANGE if kind == "rep" else BLUE
ax.plot([lo, hi], [yy, yy], color=col, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([lo, lo], [yy - 0.13, yy + 0.13], color=col, lw=2)
ax.plot([hi, hi], [yy - 0.13, yy + 0.13], color=col, lw=2)
ax.plot(OR, yy, marker=mk, ms=8, color=col, mec=BG, mew=1.2, zorder=3)
txt = reptxt[i] if kind == "rep" else f"{OR:.2f} ({lo:.2f}–{hi:.2f})"
ax.text(1.02, yy, txt, transform=ax.get_yaxis_transform(), va="center", ha="left", fontsize=10, color=INK)
ax.text(0.34, yy + 0.2, lab, va="bottom", ha="left", fontsize=10, color=INK2)
# OR = 1 reference: short segment through each data row only, so it never crosses a label
ax.plot([1, 1], [yy - 0.3, yy + 0.18], color=INK2, lw=0.8, zorder=1)
ax.text(0.34, ypos[0] + 0.85, "Second vs first poverty quintile", fontsize=11, weight="bold", va="bottom")
ax.text(0.34, ypos[5] + 0.85, "Subcounty hospital vs health centre", fontsize=11, weight="bold", va="bottom")
ax.plot([1, 1], [ypos[-1] - 0.3, ypos[-1] - 0.6], color=INK2, lw=0.8, zorder=1)
ax.text(1.04, ypos[-1] - 0.75, "OR = 1 (no association)", ha="left", va="center", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.33, 30)
ax.set_xticks([0.5, 1, 2, 5, 10, 20])
ax.set_xticklabels(["0.5", "1", "2", "5", "10", "20"])
ax.set_ylim(ypos[-1] - 1.2, ypos[0] + 1.5)
ax.set_yticks([])
ax.set_xlabel("Odds ratio of catastrophic health expenditure, 95% CI (log scale)")
from matplotlib.lines import Line2D
leg = [Line2D([0], [0], color=ORANGE, lw=2, marker="D", ms=8, mec=BG, label="Reported in the paper"),
Line2D([0], [0], color=BLUE, lw=2, marker="o", ms=8, mec=BG, label="Recomputed from the paper's own numbers")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0.0, -0.11), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.03, right=0.77, top=0.98, bottom=0.17)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- Figure 2: standard errors for the seven aORs in the abstract
order = ["Age 30-44", "Accompanied", "County", "Female", "Above secondary", "Second quintile", "Subcounty (abstract/text)"]
names = {"Age 30-44": "Aged 30–44 years (2.6)", "Accompanied": "Accompanied (3.2)", "County": "County hospital (25.1)",
"Female": "Female (1.8)", "Above secondary": "Above secondary education (1.6)",
"Second quintile": "Second poverty quintile (2.0)", "Subcounty (abstract/text)": "Subcounty hospital (9.7)"}
d = {r[0]: r for r in FIG2}
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
style(ax)
for i, k in enumerate(order):
lab, OR, lo, hi, se, s_lo, s_hi, smin = d[k]
yy = -i
ax.plot([s_lo, s_hi], [yy, yy], color=ORANGE, lw=2)
ax.plot(se, yy, marker="D", ms=8, color=ORANGE, mec=BG, mew=1.2, zorder=3)
ax.plot(smin, yy, marker="o", ms=8, color=BLUE, mec=BG, mew=1.2, zorder=3)
ax.plot([smin, 1.2], [yy, yy], color=BLUE, lw=6, alpha=0.18, solid_capstyle="butt", zorder=1)
status = "too small" if s_hi < smin else "possible"
ax.text(1.03, yy, f"{se:.3f} vs {smin:.3f} {status}", transform=ax.get_yaxis_transform(),
va="center", fontsize=10, color=INK)
ax.set_yticks(-np.arange(len(order)))
ax.set_yticklabels([names[k] for k in order], fontsize=10, color=INK)
ax.set_xscale("log")
ax.set_xlim(0.002, 1.2)
ax.set_xticks([0.003, 0.01, 0.03, 0.1, 0.3, 1])
ax.set_xticklabels(["0.003", "0.01", "0.03", "0.1", "0.3", "1"])
ax.text(1.03, 0.55, "reported vs minimum", transform=ax.get_yaxis_transform(), va="bottom", fontsize=10, color=INK2)
ax.set_ylim(-len(order) + 0.4, 0.9)
ax.set_xlabel("Standard error of log odds ratio (log scale)")
leg = [Line2D([0], [0], color=ORANGE, lw=2, marker="D", ms=8, mec=BG,
label="Implied by reported 95% CI (bar: range allowed by rounding)"),
Line2D([0], [0], color=BLUE, lw=0, marker="o", ms=8, mec=BG,
label="Smallest SE any logistic ML fit can give (shaded: attainable)")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(-0.55, -0.17), ncol=1, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.35, right=0.72, top=0.93, bottom=0.3)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nFigures written to", HERE)
Abstract swaps the DALYs-averted estimates of the two dynamic choice HIV prevention policies
Lancet HIV (2025) · doi:10.1016/S2352-3018(25)00169-9 · PMC12854106
Summary
The abstract's Findings give 18 400 DALYs averted per year (95% CI 16 700–20 100) for dynamic choice HIV prevention (DCP) including long-acting cabotegravir and 56 400 (52 300–60 500) for DCP without cabotegravir. Table 5 prints the same two estimates and intervals with the labels reversed: 18 400 for DCP without cabotegravir and 56 400 for DCP including it. Every other number in the paper agrees with Table 5. Each policy's cost difference divided by its Table 5 DALYs gives the reported ICERs ($13.2M / 56 400 = $234; $8.6M / 18 400 = $467), and the reported net DALYs and the extended-dominance result also fit Table 5. With the abstract's labels, the same costs give $717 and $152, and DCP including cabotegravir would be dominated. As printed, the abstract reverses which policy delivers more health benefit, although its ICER and conclusion are right.
What the paper reports
Abstract, Findings:
Across setting-scenarios, both DCP policies led to DALYs being averted: 18 400 DALYs per year (95% CI 16 700–20 100) for DCP including cabotegravir and 56 400 DALYs per year (52 300–60 500) for DCP without cabotegravir in 10 million adults. Compared with no DCP, there was a mean increase in annual discounted costs over 50 years: $8·6 million (7·7–9·4) for DCP without cabotegravir and $13·2 million (11·6–14·8) for DCP including cabotegravir. Addition of long-acting cabotegravir PrEP to DCP was cost-effective (vs DCP without cabotegravir); the incremental cost-effectiveness ratio for DCP including cabotegravir (vs no DCP) was $234 per DALY averted.
Abstract, Findings (the incidence result, in the same paragraph):
These increases in PrEP use led to HIV incidence reductions, with incidence rate ratios of 0·89 (0·67–1·17) for DCP without cabotegravir and 0·64 (0·44–0·97) for DCP including cabotegravir, relative to no DCP.
Table 5, "HIV-related deaths, DALYs, and costs over 50 years and cost-effectiveness analysis compared with no DCP". Columns: No DCP | DCP without cabotegravir | DCP including cabotegravir. "Data are mean (95% CI); 90% range":
Difference in number of HIV-related deaths per year | ·· | -1300 (-1400 to -1200); -4500 to 800 | -3700 (-4000 to -3400); -12 300 to 200
Difference in DALYs per year over 50 years | ·· | -18 400 (-20 100 to -16 700); -68 600 to 19 000 | -56 400 (-60 500 to -52 300); -178 800 to 8200
Difference in annual cost over 50 years | ·· | $8·6 million (7·7 to 9·4); -6·1 to 32·3 | $13·2 million (11·6 to 14·8); -16·9 to 61·0
Difference in net DALYs* per year over 50 years | ·· | -1200 (-3600 to 1200); -65 900 to 55 800 | -29 900 (-35 400 to -24 400); -179 700 to 81 700
Incremental cost-effectiveness ratio (cost per DALY averted) | ·· | Dominated | $234
Incremental cost-effectiveness ratio (cost per DALY averted) for DCP compared with no DCP (if long-acting cabotegravir PrEP not available) | ·· | $467 | NA
Table 5 footnote: "*Given a cost-effectiveness threshold of $500 per DALY averted."
Results, paragraph 3:
Overall, as a mean across setting-scenarios, both DCP policies lead to DALYs being averted (table 5). There was, however, variability over setting-scenarios (appendix p 3); the policy with the least DALYs incurred was the status quo (no DCP or long-acting cabotegravir) in 6% of setting-scenarios, DCP without cabotegravir in 11% of setting-scenarios, and DCP including cabotegravir in 83% of setting-scenarios.
Results, paragraph 5:
The policy of DCP without cabotegravir was not a candidate for the cost-effective choice of intervention, as a combination of the policies of status quo and DCP including cabotegravir (represented by a point half way along the line between the two policies) was more effective and less costly than this policy (ie, the policy was extendedly dominated). The incremental cost-effectiveness ratio (ICER) for the policy of DCP including cabotegravir was $234 per DALY averted (figure). This policy incurred the lowest overall mean net DALYs at a cost-effectiveness threshold of $500 per DALY averted (table 5). If long-acting cabotegravir was not available, the ICER for DCP was $467.
Methods, "Cost-effectiveness analysis":
Net DALYs averted was calculated by DALYs averted plus the difference in costs divided by the cost-effectiveness threshold.
What the paper's own numbers show
Method. The abstract and Table 5 report the same quantity: mean DALYs per year over 50 years, discounted at 3%, for 10 million adults, compared with no DCP. Table 5 signs it as a difference, so a negative value means DALYs averted. I first compared the two pairs of estimates and confidence intervals. I then tested each possible assignment of the two DALY estimates to the two policies against three relationships the paper defines or reports. (1) ICER vs no DCP = difference in annual cost / DALYs averted, compared with the reported $234 and $467. (2) Difference in net DALYs = −DALYs averted + difference in cost / $500, compared with the Table 5 net-DALY row. (3) Extended dominance, which the Results state for DCP without cabotegravir. Finally I compared the ordering with the paper's other effectiveness outcomes: HIV deaths averted and incidence rate ratios. All inputs are the rounded values printed in the paper. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| DALYs averted/yr, DCP without cabotegravir | Table 5 | 18 400 (16 700–20 100) | consistent with every check below |
| DALYs averted/yr, DCP including cabotegravir | Table 5 | 56 400 (52 300–60 500) | consistent with every check below |
| DALYs averted/yr, DCP including cabotegravir | Abstract, Findings | 18 400 (16 700–20 100) | equals the Table 5 value for DCP without cabotegravir, CI included |
| DALYs averted/yr, DCP without cabotegravir | Abstract, Findings | 56 400 (52 300–60 500) | equals the Table 5 value for DCP including cabotegravir, CI included |
| ICER vs no DCP, DCP including cabotegravir | Abstract; Results p5; Table 5 | $234 | $13.2M / 56 400 = $234.04 (Table 5 labels); $13.2M / 18 400 = $717.39 (abstract labels) |
| ICER vs no DCP, DCP without cabotegravir | Results p5; Table 5 | $467 | $8.6M / 18 400 = $467.39 (Table 5 labels); $8.6M / 56 400 = $152.48 (abstract labels) |
| Net DALY difference, DCP without cabotegravir | Table 5 | −1200 | −18 400 + 17 200 = −1200 (Table 5 labels); −56 400 + 17 200 = −39 200 (abstract labels) |
| Net DALY difference, DCP including cabotegravir | Table 5 | −29 900 | −56 400 + 26 400 = −30 000 (Table 5 labels; a cost of $13.15M–$13.25M, which rounds to $13.2M, gives −30 100 to −29 900); −18 400 + 26 400 = +8000 (abstract labels) |
| Extended dominance of DCP without cabotegravir | Results p5; Table 5 "Dominated" | stated | Table 5 labels: $467 > $234, so it is extendedly dominated; the midpoint of no DCP and DCP including cabotegravir averts 28 200 DALYs at $6.6M, against 18 400 at $8.6M. Abstract labels: DCP including cabotegravir would avert fewer DALYs (18 400 < 56 400) at a higher cost, so it would be strongly dominated, which contradicts the paper |
| Incremental ICER, including vs without cabotegravir | Abstract ("cost-effective (vs DCP without cabotegravir)") | not printed | Table 5 labels: $4.6M / 38 000 = $121.05 per DALY, below the $500 threshold, consistent with the statement |
| HIV-related deaths averted per year | Table 5 | 1300 (without); 3700 (including) | ratio including/without 2.85; DALY ratio 3.07 with the Table 5 labels, 0.33 with the abstract labels |
| Incidence reduction vs no DCP | Abstract | IRR 0·89 (without); 0·64 (including) | 11% vs 36% reduction, so the policy including cabotegravir is the more effective one |
Figure
Figure 1. Left: the DALYs-averted sentence in the abstract's Findings, with its policy labels (orange) as printed. Right: the same estimates as printed in Table 5, where a negative difference means DALYs averted. The arrows join each abstract value to the identical value and interval in Table 5, which belongs to the other policy. Below, in blue: each policy's cost difference from Table 5 divided by its DALYs averted under each labelling. Only the Table 5 labelling reproduces the reported ICERs of $234 and $467. All values are from the paper or are printed by recompute.py.txt.
Figure 2. The ICER of each DCP policy against no DCP. Orange rings: the reported ICERs. Filled blue dots: cost difference / DALYs averted with the Table 5 labels. Open blue dots: the same calculation with the abstract's labels. The filled dots sit on the reported values. The open dots fall at $717, above the $500 threshold, for DCP including cabotegravir and at $152 for DCP without cabotegravir, which would reverse the paper's cost-effectiveness ranking. Dashed line: the $500 per DALY threshold used in the paper.
Why it matters
The sentence is the abstract's only statement of health benefit in DALYs. As printed, it says the policy without cabotegravir averts about three times as many DALYs as the policy including cabotegravir (56 400 vs 18 400). Table 5, the Results and the paper's conclusions all say the reverse. A reader who has only the abstract, or who takes the per-policy health gains from it for a review, a meta-analysis or a budget-impact estimate, would get the two policies' benefits the wrong way round. The abstract is then also inconsistent with itself. Its own costs ($8.6M and $13.2M) and its reported ICER of $234 cannot both hold with its DALY figures: with those figures, DCP including cabotegravir costs more and averts fewer DALYs than DCP without it, so it would be dominated rather than cost-effective.
The error does not affect the model results or the paper's conclusions. Table 5, the ICERs of $234 and $467, the net-DALY results, the extended-dominance finding and the Interpretation all use the correct assignment. The error is confined to the policy labels in one sentence of the abstract.
Explanations considered
- Different quantity, time horizon, population or comparator. Ruled out. The abstract and Table 5 both describe mean DALYs per year over 50 years for 10 million adults, compared with no DCP, and both print the same 95% CIs (16 700–20 100 and 52 300–60 500). The abstract states that "both DCP policies led to DALYs being averted" and gives the costs against no DCP, so it is not reporting an incremental comparison between the two DCP policies. Only the policy labels differ.
- Net DALYs rather than DALYs. Ruled out. The net-DALY differences in Table 5 are −1200 and −29 900, not 18 400 or 56 400.
- Table 5 is the version with the error, and the abstract is right. Ruled out by the paper's other results, all of which fit Table 5. These are both ICERs, both net-DALY values, the extended dominance of DCP without cabotegravir, the larger number of HIV deaths averted with cabotegravir (3700 vs 1300), the lower incidence rate ratio with cabotegravir (0·64 vs 0·89), and the statement that DCP including cabotegravir incurred the fewest DALYs in 83% of setting-scenarios. With the abstract's labels, none of the reported ICERs or net DALYs can be reproduced.
- Text conversion or column misalignment in the archived copy. Ruled out. The publisher PDF (Elsevier, "Published online September 11, 2025", obtained from an institutional repository deposit marked as the published version) has the same abstract sentence and the same Table 5 column order. The PubMed abstract (PMID 40946710) and the PMC/Europe PMC author manuscript text match as well. The values are whole numbers, so no decimal-point or minus-sign misreading is possible.
- The supplement gives a different picture. No. The appendix (126 pages) prints no aggregate figure of 18 400 or 56 400. Its plots of discounted annual DALYs and of DALYs averted against incremental cost show, qualitatively, more DALYs averted with DCP including cabotegravir. That agrees with Table 5, not with the abstract.
- Rounding. This accounts only for the 100-DALY gap between the recomputed (−30 000) and reported (−29 900) net DALYs for DCP including cabotegravir. It cannot account for a factor of three between the two labellings.
Status
As of 2026-10-23:
- Version of record: checked. The publisher PDF has the swapped labels in the abstract, and Table 5 is as described above. The PubMed abstract (PMID 40946710), the PMC archived text (PMC12854106, author manuscript NIHMS2140119) and Europe PMC are the same. The journal website (thelancet.com, sciencedirect.com) returned 403 or a bot check, so I could not view the online HTML version there.
- Supplement: the appendix (NIHMS2140119-supplement-Appendix.pdf) was checked as described above. It agrees with Table 5.
- Corrections: Crossref and PubMed list a publisher erratum, "Correction to Lancet HIV 2025; 12: e701–11" (Lancet HIV 2026;13(5):e296, doi:10.1016/S2352-3018(26)00080-9, PMID 42067279, May 2026). I could not get its text: the publisher page returned 403, and PubMed and Europe PMC list no abstract. I therefore cannot say whether this erratum corrects the swap described here. The PMC archived version still shows the swap. No PMC or Europe PMC correction notice was linked other than this erratum, and no retraction or expression of concern was found.
- PubPeer: searched for the DOI. The site returned a bot check to automated requests, so I could not confirm whether any comment exists.
- Sources: PMC12854106 (archived full text and Table 5); the publisher PDF from the repository deposit (eprints.whiterose.ac.uk/id/eprint/231798); PubMed records 40946710 and 42067279; Crossref metadata for the DOI; Europe PMC supplementary files.
Suggested correction
If the May 2026 erratum does not already address it, the abstract's Findings sentence should read:
Across setting-scenarios, both DCP policies led to DALYs being averted: 18 400 DALYs per year (95% CI 16 700–20 100) for DCP without cabotegravir and 56 400 DALYs per year (52 300–60 500) for DCP including cabotegravir in 10 million adults.
The same change should be made wherever the abstract is reproduced (PubMed, PMC, Europe PMC). No change to Table 5, the Results, the ICERs or the conclusions is needed.
Recomputation script (python recompute.py.txt)
"""Recompute the consistency checks for PMC12854106
(Lancet HIV 2025, doi:10.1016/S2352-3018(25)00169-9).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the paper as archived in PMC (same values in the
publisher PDF):
Table 5 (columns: No DCP | DCP without cabotegravir | DCP including cabotegravir),
rows "Difference in DALYs per year over 50 years", "Difference in annual cost
over 50 years", "Difference in net DALYs* per year over 50 years",
"Difference in number of HIV-related deaths per year", and the two ICER rows.
Abstract, Findings: the DALYs-averted sentence, costs, ICER and incidence rate ratios.
Methods: threshold of US$500 per DALY averted.
All figures are means per year over 50 years, discounted at 3%, for 10 million adults,
compared with no DCP.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
THRESHOLD = 500.0 # Methods, US$ per DALY averted
POL = ("without", "including") # DCP without / including cabotegravir
NAME = {"without": "DCP without cabotegravir", "including": "DCP including cabotegravir"}
# Table 5: difference in DALYs per year vs no DCP (negative = averted), mean (95% CI)
T5_DALY = {"without": (-18400, -20100, -16700), "including": (-56400, -60500, -52300)}
# Abstract, Findings: DALYs averted per year, as labelled in the abstract
ABS_DALY = {"including": (18400, 16700, 20100), "without": (56400, 52300, 60500)}
# Table 5 and abstract: difference in annual discounted cost vs no DCP, US$
COST = {"without": 8.6e6, "including": 13.2e6}
# Table 5: difference in net DALYs per year (threshold $500)
T5_NET = {"without": -1200, "including": -29900}
# Table 5: difference in HIV-related deaths per year
T5_DEATHS = {"without": -1300, "including": -3700}
# Abstract: incidence rate ratios vs no DCP
IRR = {"without": 0.89, "including": 0.64}
# Reported ICERs vs no DCP: $234 (abstract, Results p5, Table 5) and $467
# (Results p5, Table 5 "if long-acting cabotegravir PrEP not available")
ICER_REPORTED = {"without": 467, "including": 234}
# ---------------------------------------------------------------- derived
averted_t5 = {p: -T5_DALY[p][0] for p in POL} # Table 5 attribution
averted_abs = {p: ABS_DALY[p][0] for p in POL} # abstract attribution
def line(s=""):
print(s)
line("1. Same estimates, same CIs, opposite labels")
for p in POL:
m, lo, hi = T5_DALY[p]
line(f" Table 5 {NAME[p]:28s}: averted {-m} (95% CI {-hi} to {-lo})")
for p in POL:
m, lo, hi = ABS_DALY[p]
line(f" Abstract {NAME[p]:28s}: averted {m} (95% CI {lo} to {hi})")
same_set = sorted(-T5_DALY[p][0] for p in POL) == sorted(ABS_DALY[p][0] for p in POL)
swapped = all(ABS_DALY[p][0] == -T5_DALY[q][0] for p, q in (("including", "without"), ("without", "including")))
line(f" same pair of values: {same_set}; abstract value for each policy = Table 5 value of the other: {swapped}")
line()
line("2. ICER vs no DCP = cost difference / DALYs averted")
icer_t5 = {p: COST[p] / averted_t5[p] for p in POL}
icer_abs = {p: COST[p] / averted_abs[p] for p in POL}
for p in POL:
line(f" {NAME[p]:28s}: reported ${ICER_REPORTED[p]}; "
f"Table 5 attribution ${COST[p]/1e6:.1f}M/{averted_t5[p]} = ${icer_t5[p]:.2f}; "
f"abstract attribution ${COST[p]/1e6:.1f}M/{averted_abs[p]} = ${icer_abs[p]:.2f}")
line()
line("3. Net DALY difference = -(DALYs averted) + cost difference / threshold")
net_t5 = {p: -averted_t5[p] + COST[p] / THRESHOLD for p in POL}
net_abs = {p: -averted_abs[p] + COST[p] / THRESHOLD for p in POL}
for p in POL:
line(f" {NAME[p]:28s}: reported {T5_NET[p]}; cost/threshold = {COST[p]/THRESHOLD:.0f}; "
f"Table 5 attribution {net_t5[p]:.0f}; abstract attribution {net_abs[p]:+.0f}")
line(" (the 100 gap for 'including' is consistent with the costs being printed to one decimal "
f"in $ millions: $13.15M-$13.25M gives {-56400+13.15e6/THRESHOLD:.0f} to {-56400+13.25e6/THRESHOLD:.0f})")
line()
line("4. Dominance")
mid_daly = averted_t5["including"] / 2
mid_cost = COST["including"] / 2
line(f" Table 5 attribution: ICER(without) {icer_t5['without']:.0f} > ICER(including) {icer_t5['including']:.0f} "
f"-> 'without' extendedly dominated: {icer_t5['without'] > icer_t5['including']}")
line(f" midpoint of no DCP and 'including': {mid_daly:.0f} DALYs averted at ${mid_cost/1e6:.1f}M, "
f"vs 'without' {averted_t5['without']} at ${COST['without']/1e6:.1f}M")
inc_d = averted_t5["including"] - averted_t5["without"]
inc_c = COST["including"] - COST["without"]
line(f" incremental, including vs without: {inc_d} DALYs, ${inc_c/1e6:.1f}M -> ${inc_c/inc_d:.2f} per DALY")
strong = averted_abs["including"] < averted_abs["without"] and COST["including"] > COST["without"]
line(f" abstract attribution: 'including' averts {averted_abs['including']} < {averted_abs['without']} "
f"at higher cost -> strongly dominated by 'without': {strong}")
line()
line("5. Other outcomes in the paper")
line(f" HIV deaths averted per year: without {-T5_DEATHS['without']}, "
f"including {-T5_DEATHS['including']}; ratio {T5_DEATHS['including']/T5_DEATHS['without']:.2f}")
line(f" DALYs averted ratio including/without, Table 5: {averted_t5['including']/averted_t5['without']:.2f}; "
f"abstract: {averted_abs['including']/averted_abs['without']:.2f}")
line(f" incidence reduction vs no DCP: without {100*(1-IRR['without']):.0f}%, including {100*(1-IRR['including']):.0f}%")
# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834"
BLUE = "#2a78d6"
GRID = "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "text.parse_math": False,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def fmt(n):
return f"{n:,}".replace(",", " ")
# ---------------------------------------------------------------- fig1: abstract vs Table 5
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
for x0, title in ((2, "Abstract, Findings"), (54, "Table 5 and Results")):
ax.add_patch(FancyBboxPatch((x0, 30), 44, 62, boxstyle="round,pad=0.6,rounding_size=1.5",
fc="white", ec=INK2, lw=1))
ax.text(x0 + 2, 88, title, fontsize=11.5, fontweight="bold", va="top")
# Abstract panel (as printed)
a_inc = ABS_DALY["including"]
a_wo = ABS_DALY["without"]
ax.text(4, 78, f"{fmt(a_inc[0])} DALYs per year\n(95% CI {fmt(a_inc[1])}–{fmt(a_inc[2])})",
fontsize=10.5, va="top", color=INK)
ax.text(4, 67, "for DCP including cabotegravir", fontsize=10.5, va="top", color=ORANGE, fontweight="bold")
ax.text(4, 56, f"{fmt(a_wo[0])} DALYs per year\n(95% CI {fmt(a_wo[1])}–{fmt(a_wo[2])})",
fontsize=10.5, va="top", color=INK)
ax.text(4, 45, "for DCP without cabotegravir", fontsize=10.5, va="top", color=ORANGE, fontweight="bold")
ax.text(4, 37, "Reported ICER, including: $234 per DALY", fontsize=10, va="top", color=INK2)
# Table 5 panel (as printed)
t_wo = T5_DALY["without"]
t_inc = T5_DALY["including"]
ax.text(56, 78, "DCP without cabotegravir", fontsize=10.5, va="top", color=INK, fontweight="bold")
ax.text(56, 73, f"−{fmt(-t_wo[0])} DALYs per year\n(−{fmt(-t_wo[1])} to −{fmt(-t_wo[2])})",
fontsize=10.5, va="top")
ax.text(56, 56, "DCP including cabotegravir", fontsize=10.5, va="top", color=INK, fontweight="bold")
ax.text(56, 51, f"−{fmt(-t_inc[0])} DALYs per year\n(−{fmt(-t_inc[1])} to −{fmt(-t_inc[2])})",
fontsize=10.5, va="top")
ax.text(56, 39, "Reported ICERs: $467 (without),\n$234 (including)", fontsize=10, va="top", color=INK2)
# crossing connectors: each abstract number to the same number in Table 5
ax.add_patch(FancyArrowPatch((31, 75), (55, 69.5), arrowstyle="-|>", mutation_scale=12,
color=INK2, lw=1.5, connectionstyle="arc3,rad=0"))
ax.add_patch(FancyArrowPatch((31, 53), (55, 49.5), arrowstyle="-|>", mutation_scale=12,
color=INK2, lw=1.5, connectionstyle="arc3,rad=0"))
ax.text(50, 95.5, "Same estimates and CIs; the two policy labels are swapped",
ha="center", fontsize=11, fontweight="bold", va="center")
# recomputed check underneath
ax.text(2, 24, "Check with the paper's own costs (Table 5: +$8.6M without, +$13.2M including, vs no DCP):",
fontsize=10, va="top", color=INK)
ax.text(2, 18,
f"Table 5 labels: $13.2M / {fmt(averted_t5['including'])} = ${icer_t5['including']:.0f} and "
f"$8.6M / {fmt(averted_t5['without'])} = ${icer_t5['without']:.0f} (both match)",
fontsize=10, va="top", color=BLUE)
ax.text(2, 12,
f"Abstract labels: $13.2M / {fmt(averted_abs['including'])} = ${icer_abs['including']:.0f} and "
f"$8.6M / {fmt(averted_abs['without'])} = ${icer_abs['without']:.0f} (match neither)",
fontsize=10, va="top", color=BLUE)
ax.text(2, 2, "Orange: abstract labels as printed. Blue: recomputed from the paper's numbers.",
fontsize=10, ha="left", va="bottom", color=INK2)
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: ICER dot plot
fig, ax = plt.subplots(figsize=(7.2, 3.0))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.20)
ypos = {"including": 1, "without": 0}
for p in POL:
y = ypos[p]
ax.plot([min(icer_t5[p], icer_abs[p]), max(icer_t5[p], icer_abs[p])], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(ICER_REPORTED[p], y, "o", ms=13, mfc="none", mec=ORANGE, mew=2, zorder=3)
ax.plot(icer_t5[p], y, "o", ms=8, color=BLUE, zorder=4)
ax.plot(icer_abs[p], y, "o", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
right = icer_abs[p] > icer_t5[p]
ax.annotate(f"${icer_abs[p]:.0f}", (icer_abs[p], y), xytext=(10 if right else -10, 0),
textcoords="offset points", ha="left" if right else "right", va="center",
fontsize=10, color=BLUE)
ax.annotate(f"${ICER_REPORTED[p]} reported\n${icer_t5[p]:.0f} recomputed", (icer_t5[p], y),
xytext=(0, -12), textcoords="offset points", ha="center", va="top", fontsize=10, color=INK)
ax.axvline(THRESHOLD, color=INK2, lw=1, ls=(0, (3, 3)), zorder=2)
ax.text(THRESHOLD + 8, 1.55, "threshold $500", ha="left", fontsize=10, color=INK2)
ax.set_yticks([0, 1])
ax.set_yticklabels([NAME["without"], NAME["including"]])
ax.set_ylim(-0.75, 1.75)
ax.set_xlim(0, 800)
ax.set_xticks(range(0, 801, 100))
ax.set_xticklabels([f"${v}" for v in range(0, 801, 100)])
ax.set_xlabel("ICER vs no DCP (US$ per DALY averted)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "o", ms=13, mfc="none", mec=ORANGE, mew=2, ls="")
h2, = ax.plot([], [], "o", ms=8, color=BLUE, ls="")
h3, = ax.plot([], [], "o", ms=8, mfc=BG, mec=BLUE, mew=2, ls="")
fig.legend([h1, h2, h3], ["Reported ICER", "Cost / DALYs with Table 5 labels", "Cost / DALYs with abstract labels"],
loc="upper center", ncol=3, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.99),
handletextpad=0.3, columnspacing=1.2)
save(fig, "fig2")
line()
line("Figures written: fig1.svg/png, fig2.svg/png")
95% upper bound for zero neural tube defects among 384 dolutegravir-exposed outcomes is tenfold too small
Lancet HIV (2021) · doi:10.1016/S2352-3018(20)30268-X · PMC7869891
Summary
The paper's primary safety outcome is neural tube defect (NTD) prevalence after periconceptional dolutegravir (DTG) exposure. It observes no NTDs among 384 DTG-exposed and 1068 efavirenz (EFV)-exposed birth outcomes (Table 3). In the Abstract and the Results it reports a 95% Wilson confidence interval of 0–0·0010 for the DTG group and 0–0·0036 for the EFV group. For 0 events in n, the Wilson 95% upper bound is z²/(n + z²). With n = 384 that is 0·0099, about ten times the printed value; a bound of 0·0010 would need about 3838 outcomes, and no DTG-exposed pregnancy count in the paper comes close. The same method reproduces every other Wilson (prevalence) interval in the paper, including the EFV bound. The paper cites five DTG-exposed NTD rates from other studies: 1·04% (the Botswana estimate used in the Methods power calculation), 0·94%, 0·66% and 0·30% (Botswana) and 0·4% (the Antiretroviral Pregnancy Registry). The printed DTG bound (0·10%) lies below all five, so it appears to exclude every one of them. The bound implied by the paper's own counts (0·99%) contains four of them, 0·94%, 0·66%, 0·4% and 0·30%. Only the 1·04% power-calculation rate lies above it, by 0·05 percentage points.
What the paper reports
Abstract, Methods:
NTD incidences were calculated with 95% Wilson confidence intervals (95%CI).
Abstract, Findings:
There were no NTD in either DTG-exposed (0 [95%CI 0, 0·0010]) or EFV-exposed groups (0 [95%CI 0, 0·0036]).
Methods, "Variable definitions and statistical analysis":
We computed Wilson 95% confidence intervals (95%CI) for prevalence estimates of the primary outcomes.
Results, third paragraph:
The estimated NTD prevalence for DTG-exposed infants was 0 (95%CI 0-0·0010) and 0 (95%CI 0-0·0036) in EFV-exposed infants. Our composite outcome was more frequent in DTG-exposed fetuses (n=25, 7% [95%CI 0·044-0·094]) compared to those EFV-exposed (n=43, 4% [95%CI 0·030-0·054])
Table 3 ("Birth outcomes of periconceptionally DTG- and EFV- exposed women, Brazil, 2015-2018"), excerpt (selected rows; the P-value column and the "Outcome" row that carries its P value are omitted):
| Only EFV regimen(N=1068) | Any DTG regimen(N=384) |
Neural tube defects, n (%) | 0 (0) | 0 (0)
Live birth | 1025 (96) | 359 (93)
Stillbirth | 15 (1) | 2 (1)
Abortion | 28 (3) | 23 (6)
Results, "Updated NTD incidence estimate":
2 / (0·78 * 900 + 382) = 0·0018 or 1·8 per 1,000 DTG-exposed pregnancies (95%CI 0·5-6·7 per 1,000 DTG-exposed pregnancies)
Research in context panel, "Evidence before this study" (the comparison rates):
out of 426 women exposed to DTG at conception, four gave birth to infants with NTD (0·94%) compared to 0·10% among those exposed to other antiretrovirals. Since that initial warning, updated data from the Botswana study showed a decreased prevalence compared to the initial evaluation though statistically increased compared to women on non-DTG regimens (0·30% vs 0·10%). Other recent smaller studies from Botswana demonstrated increased NTD prevalence among women with DTG exposure (1 among 152 women or 0·66%) but statistical analyses comparing risk to other groups included the null. Lastly, data from North America and Europe using the Antiretroviral Pregnancy registry observed 1 NTD event among 248 DTG-exposed women (0·4%).
Methods, "Study design and data sources" (the power calculation):
Assuming a NTD prevalence of 0·06% for births among DTG-unexposed women and a prevalence of 1·04% in the DTG-exposed women (the estimated prevalence from Botswana available at the time)
Abstract, Interpretation:
Neither DTG or EFV exposure was associated with NTD in our national cohort; incidence of NTD is likely well under 1% among DTG-exposed HIV-positive women
The version of record repeats the bound. The publisher's edited abstract, as indexed in PubMed (PMID 33387477), reads:
There were no NTDs in either dolutegravir-exposed (0, 95% CI 0-0·0010) or efavirenz-exposed groups (0, 95% CI 0-0·0036).
What the paper's own numbers show
Method. I computed the Wilson score interval at the stated 95% level (z = 1·959964) for 0 events, using every denominator the paper gives for each group: birth outcomes (Table 3 N), women, and live births. For x = 0 the upper bound has the closed form z²/(n + z²). I inverted that formula to find the n at which the bound equals 0·0010. To rule out a different method or level, I also computed Clopper–Pearson, the rule of three, and Wilson bounds at 90% (equal to a one-sided 95% bound) and 99%. To test whether the authors' method and denominators are as stated, I recomputed the paper's other Wilson intervals: the composite outcome in each group and the updated DTG estimate. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| DTG NTD, 0/384 birth outcomes, Wilson 95% upper bound | Abstract (Findings); Results ¶3; Table 3 N | 0·0010 | 0·0099 (0·99%), 9·9 times the reported value |
| DTG NTD, 0/382 women | Abstract; Results | (same 0·0010) | 0·0100 |
| DTG NTD, 0/359 live births | Table 3 | (same 0·0010) | 0·0106 |
| EFV NTD, 0/1068 birth outcomes, Wilson 95% upper bound | Abstract (Findings); Results ¶3; Table 3 N | 0·0036 | 0·0036 (0·00358), matches |
| EFV NTD, 0/1045 women; 0/1025 live births | Results; Table 3 | (same 0·0036) | 0·0037; 0·0037 |
| DTG 0/384, other methods and levels | derived | 0·0010 | Clopper–Pearson 95% 0·0096; rule of three 0·0078; Wilson 90% (one-sided 95%) 0·0070; Wilson 99% 0·0170 |
| n at which a 0/n Wilson 95% upper bound equals 0·0010 | derived | — | 3838 (the value rounds to 0·0010 only for n from 3655 to 4039) |
| Largest DTG-exposed pregnancy or outcome counts | Abstract; Results; Table 3 | 382 women; 384 birth outcomes; updated DTG denominator 1084 (whole analysed cohort: 1427 women, 1452 outcomes) | all below 3655 |
| Larger counts in the paper that are not DTG-exposed pregnancy denominators | Results ¶1; Introduction | 3390 possible pregnancies screened; 2507 selected for investigation; 22 624 Brazilian women on DTG aged 15–49 | 0/n Wilson 95% upper bounds 0·0011, 0·0015 and 0·0002; none falls in the 3655–4039 range that gives 0·0010 |
| Composite outcome DTG, 25/384 | Results ¶3; Table 3 (2 stillbirths + 23 abortions) | 7% (0·044–0·094) | 6·5% (0·0445–0·0943), matches |
| Composite outcome EFV, 43/1068 | Results ¶3; Table 3 (15 + 28) | 4% (0·030–0·054) | 4·0% (0·0300–0·0538), matches |
| Updated DTG estimate, 2/(0·78 × 900 + 382) = 2/1084 | Results, updated estimate; Abstract | 0·0018 (0·0005–0·0067) | 0·00185 (0·00051–0·00670), matches |
| Table 3 column totals | Table 3 | N = 384; N = 1068 | 359 + 2 + 23 = 384; 1025 + 15 + 28 = 1068 |
The DTG NTD bound is the only Wilson (prevalence) interval in the paper that does not reproduce. Every other Wilson interval reproduces with the stated method and the Table 3 denominators. The bootstrap 95% CIs for the propensity-weighted odds ratios (Methods; Results; Figure 2 of the paper) depend on individual-level data and were not recomputed.
Figure
Figure 1. NTD prevalence intervals on one axis, in percent. Each pair of rows shows the interval as reported in the paper (orange) and as recomputed from the paper's own counts with the Wilson 95% method the paper states (blue). Dots are point estimates and the end ticks are upper bounds, with values printed next to them. For EFV (0/1068) and for the updated DTG estimate (2/1084) the reported and recomputed intervals coincide. For DTG (0/384) the recomputed upper bound is 0·99%, 9·9 times the reported 0·10%. The lower rows show every NTD rate the paper cites for comparison, each with a thin dashed line running up through the intervals: five DTG-exposed rates (filled diamonds: Botswana 1·04% from the power calculation, 0·94%, 0·66% and 0·30%, and the Pregnancy Registry 0·4%) and two rates without DTG exposure (hollow diamonds: Botswana non-DTG 0·10%, Brazil general population 0·06%). As printed, the DTG interval ends at 0·10% and lies below all five DTG-exposed rates. As recomputed, it contains four of them; only the 1·04% power-calculation rate lies above it.
Figure 2. The Wilson 95% upper bound for 0 events in n, z²/(n + z²), on log–log axes (blue line). The recomputed DTG (n = 384, 0·99%) and EFV (n = 1068, 0·36%) bounds lie on the line, and the EFV value matches the paper. The reported DTG bound of 0·0010 at n = 384 (orange dot, with a dashed orange line at its level) lies far off the line. The line reaches 0·0010 only at n = 3838 (hollow diamond), about ten times the DTG sample and larger than the whole analysed cohort (1452 birth outcomes).
Why it matters
NTD prevalence among women exposed to DTG at conception is the outcome the study was set up to measure, after the 2018 safety signal from Botswana. Its confidence interval is the cohort's own statement of how high the DTG-associated NTD risk could plausibly be. The misprinted bound is in the Abstract Findings, which is also the version of record's abstract and is indexed in PubMed, and in the Results. The correct value is printed nowhere in the paper, so a reader cannot recover it without recomputing.
The error changes what the cohort appears to exclude.
- As printed (upper bound 0·10%), 0/384 would appear to rule out every DTG-exposed NTD rate the paper cites: 1·04%, 0·94%, 0·66%, 0·4% and 0·30%. It would also put the DTG upper bound at the Botswana non-DTG background of 0·10%, implying the cohort alone could exclude any material DTG excess.
- As recomputed (upper bound 0·99%), the interval contains the DTG-exposed rates 0·94%, 0·66%, 0·4% and 0·30%, and both rates without DTG exposure (0·10% and 0·06%). Only the 1·04% rate assumed in the power calculation lies above it, by 0·05 percentage points. Zero events in 384 has probability 0·32 if the true prevalence is 0·30%, 0·21 at 0·4% and 0·079 at 0·66%. At 0·94% it has probability 0·027, just above the 0·025 tail of a two-sided 95% interval, consistent with 0·94% lying just inside the interval (the exact Clopper–Pearson bound is 0·96%); at 1·04% it is 0·018. The cohort by itself therefore cannot distinguish the Botswana updated rate from background. The Discussion acknowledges this point in words ("it is likely that our study was underpowered to detect a difference in NTD risk"). The printed interval is far narrower than that statement implies.
Also unaffected: the point estimates (0 NTDs in each group), the Table 3 counts, the composite-outcome intervals and regressions, and the updated estimate (0·18%, 95% CI 0·05–0·67%). The Interpretation's "well under 1%" is supported by the updated estimate, which is computed correctly. On the cohort data alone, though, the upper bound is 0·99%, just under 1%. The overall conclusions survive. What is overstated, about tenfold, is the precision of the primary-outcome estimate in the Abstract and Results.
Explanations considered
- A different denominator. This does not account for the numbers. The paper gives 384 birth outcomes (Table 3), 382 women and 359 live births for the DTG group, and these give upper bounds of 0·0099, 0·0100 and 0·0106. A bound of 0·0010 needs at least about 3655 observations. The largest DTG-exposed pregnancy count in the paper is the updated estimate's 1084, and the whole analysed cohort has 1452 outcomes. The paper does print larger counts (3390 possible pregnancies screened and 2507 selected for investigation in the Results, and 22 624 Brazilian women on DTG aged 15–49 in the Introduction), but none is a DTG-exposed pregnancy denominator, and none would give 0·0010 under the stated method in any case: their 0/n Wilson 95% upper bounds are 0·0011, 0·0015 and 0·0002.
- A different interval method or level. This does not account for the numbers. For 0/384, Clopper–Pearson gives 0·0096, the rule of three 0·0078, a one-sided 95% (two-sided 90%) Wilson bound 0·0070 and a 99% Wilson bound 0·0170. None is near 0·0010. The paper states Wilson 95% intervals twice.
- The method was applied differently throughout. This does not account for the numbers. The same Wilson 95% calculation reproduces the EFV NTD bound (0·0036), both composite-outcome intervals (0·044–0·094 and 0·030–0·054) and the updated estimate (0·0005–0·0067) to the printed precision. Only the DTG NTD bound departs.
- Percent versus proportion. This does not account for the numbers. The NTD bounds are printed as proportions, like the EFV bound. Read as a percentage, 0·0010% would need about 384 000 outcomes.
- A conversion or typesetting artefact in the archived manuscript. This is ruled out for the abstract: the publisher's edited abstract in PubMed prints the same 0·0010. The value also appears identically in two separate places in the manuscript.
- A typographical slip. This is the simplest account consistent with the numbers. The correct value, 0·0099, rounds to 0·010 at three decimals, and for 382 women it is 0·0100 at four. An extra or misplaced zero in either gives 0·0010. Whatever the route, the printed value is wrong and about ten times too small.
Status
- Version of record. The publisher's full text on thelancet.com and sciencedirect.com could not be opened by automated access (HTTP 403 and Cloudflare challenge pages), and doi.org resolved only to a redirect stub. The publisher's edited abstract was checked through PubMed (PMID 33387477, record revised 2026-01-27) and Europe PMC. It repeats "0, 95% CI 0-0·0010" for DTG and "0, 95% CI 0-0·0036" for EFV, and confirms the denominators of 384 DTG-exposed and 1068 EFV-exposed fetuses. The Results paragraph of the version of record could not be seen. The PMC text (author manuscript, NIHMS1664153) was used for every other quote, and each quote above was checked verbatim against it. Separately, the edited abstract gives the stillbirth-or-miscarriage counts as 23 (6%) and 28 (3%) with p=0·0037, where the manuscript abstract gives 25 (6·5%) and 43 (4·0%) with p=0·05. This difference does not bear on the NTD bound and is not analysed here.
- Supplements. The appendix (Supplemental Tables 1–6, including results for all women in Supplemental Table 3) is listed on the PMC page but could not be downloaded: PMC returned a placeholder or HTTP 403, Europe PMC reported it as not open access, and the publisher copy was blocked. It is unlikely to change the finding, because the version-of-record abstract itself confirms the denominators and no denominator or standard method gives 0·0010.
- Corrections and comments. As of 2026-10-23, none was found. Crossref lists no update relations for the DOI, PMC and Europe PMC list no correction notices, and the PubMed record has no erratum or comment links. A web search for the title with "correction", "erratum" or "PubPeer" found no correction or comment thread. The PubPeer API was blocked by a Cloudflare challenge, so PubPeer itself could not be queried directly.
- Sources. PMC7869891 as archived (Abstract, Methods, Results, Table 3, Research in context panel, Discussion); PubMed efetch for PMID 33387477; Crossref and PMC/Europe PMC notice queries;
recompute.py.txtin this folder.
Suggested correction
- In the Abstract (Findings) and the Results (third paragraph), replace the DTG NTD interval "0–0·0010" with the Wilson 95% interval for 0/384, 0–0·0099 (0–0·0100 if the 382 women are the intended denominator). Leave the EFV interval (0–0·0036) as it is.
- State the denominator used for the NTD prevalence (birth outcomes, women or live births), since the paper uses all three elsewhere.
- Optionally, note that the corrected DTG interval contains the DTG-exposed prevalences from the Research in context panel (0·94%, 0·66%, 0·4% and 0·30%), though not the 1·04% assumed in the power calculation, which fits the Discussion's statement that the study was underpowered for NTDs. The updated estimate, 0·0018 (0·0005–0·0067), needs no change.
Recomputation script (python recompute.py.txt)
"""Recompute the 95% Wilson confidence intervals for neural tube defect (NTD)
prevalence in PMC7869891 (Lancet HIV 2021, doi:10.1016/S2352-3018(20)30268-X).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PubMed Central (author manuscript):
Abstract (Findings) and Results paragraph 3: reported NTD intervals
Table 3: N = 384 (any DTG regimen) and N = 1068 (only EFV regimen) birth outcomes,
0 NTDs in each; live births 359 / 1025; stillbirths 2 / 15; abortions 23 / 28
Results (Characteristics): 382 DTG-exposed and 1045 EFV-exposed women
Results (Updated NTD incidence estimate): 2 / (0.78 * 900 + 382)
Methods: "We computed Wilson 95% confidence intervals (95%CI) for prevalence
estimates of the primary outcomes."
Research in context panel / Discussion: comparison rates 0.94%, 0.30%, 0.66%
(1/152) and 0.10% (Botswana), 0.4% (1/248, Antiretroviral Pregnancy
Registry) and 0.06% (Brazil, general population)
Methods (power calculation): assumed DTG prevalence 1.04% ("the estimated
prevalence from Botswana available at the time")
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975) # 1.959964
# ------------------------------------------------------------------ inputs (paper)
N_DTG_OUT, N_EFV_OUT = 384, 1068 # Table 3 column Ns (birth outcomes)
N_DTG_WOMEN, N_EFV_WOMEN = 382, 1045 # Results / Abstract
N_DTG_LIVE, N_EFV_LIVE = 359, 1025 # Table 3 live births
STILL = {"DTG": 2, "EFV": 15} # Table 3
ABORT = {"DTG": 23, "EFV": 28} # Table 3
COMPOSITE = {"DTG": 25, "EFV": 43} # Abstract / Results (manuscript)
REPORTED = { # Abstract Findings and Results paragraph 3
"DTG NTD upper": 0.0010,
"EFV NTD upper": 0.0036,
"DTG composite": (0.044, 0.094),
"EFV composite": (0.030, 0.054),
"updated": (0.0018, 0.0005, 0.0067),
}
UPD_NEW, UPD_FRAC, UPD_EVENTS = 900, 0.78, 2 # Results, updated estimate
CHART_CONFIRMED, CHART_REVIEWED = 382, 490
POWER_ASSUMED_DTG = 0.0104 # Methods power calculation: assumed DTG prevalence 1.04%
COMPARATORS = [ # (rate, label, DTG-exposed?) rates cited in the paper
(POWER_ASSUMED_DTG, "Botswana DTG, power calc. 1.04%", True), # Methods
(0.0094, "Botswana DTG, initial 0.94%", True), # Research in context
(0.0066, "Botswana DTG, 1/152, 0.66%", True), # Research in context
(0.0040, "Pregnancy Registry DTG, 1/248, 0.4%", True), # Research in context
(0.0030, "Botswana DTG, updated 0.30%", True), # Research in context
(0.0010, "Botswana non-DTG 0.10%", False), # Research in context
(0.0006, "Brazil, general population 0.06%", False), # Methods / Discussion
]
def wilson(x, n, conf=0.95):
z = stats.norm.ppf(1 - (1 - conf) / 2)
p = x / n
den = 1 + z * z / n
c = (p + z * z / (2 * n)) / den
h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
return max(0.0, c - h), c + h
def cp_upper(x, n, conf=0.95):
return stats.beta.ppf(1 - (1 - conf) / 2, x + 1, n - x)
def n_for_zero_upper(u, z=Z):
"""n at which the Wilson upper bound for 0/n equals u: z^2/(n+z^2) = u."""
return z * z / u - z * z
print("=" * 78)
print("1. Wilson 95% upper bound for 0 NTDs, every denominator in the paper")
print(" closed form for x = 0: upper = z^2 / (n + z^2), z = %.6f" % Z)
rows = [("DTG birth outcomes (Table 3 N)", N_DTG_OUT), ("DTG women", N_DTG_WOMEN),
("DTG live births (Table 3)", N_DTG_LIVE), ("EFV birth outcomes (Table 3 N)", N_EFV_OUT),
("EFV women", N_EFV_WOMEN), ("EFV live births (Table 3)", N_EFV_LIVE)]
for lab, n in rows:
w = wilson(0, n)[1]
print(f" {lab:32s} n={n:5d} Wilson={w:.5f} ({100*w:.2f}%) "
f"Clopper-Pearson={cp_upper(0, n):.5f} rule of three={3/n:.5f}")
print(f" reported: DTG {REPORTED['DTG NTD upper']:.4f} EFV {REPORTED['EFV NTD upper']:.4f}")
dtg_w = wilson(0, N_DTG_OUT)[1]
efv_w = wilson(0, N_EFV_OUT)[1]
print(f" ratio recomputed/reported, DTG: {dtg_w / REPORTED['DTG NTD upper']:.2f}")
print(f" EFV recomputed {efv_w:.5f} rounds to {round(efv_w, 4):.4f} (reported 0.0036)")
print("\n2. Other levels / methods for 0/384")
for conf in (0.90, 0.95, 0.99):
print(f" Wilson {conf:.0%} upper = {wilson(0, N_DTG_OUT, conf)[1]:.5f}")
print(f" one-sided 95% Wilson (z=1.645) = {wilson(0, N_DTG_OUT, 0.90)[1]:.5f}")
print(f" Clopper-Pearson 95% = {cp_upper(0, N_DTG_OUT):.5f}; rule of three = {3/N_DTG_OUT:.5f}")
print(f" 0.0010 read as a percentage = {0.0010/100:.6f} as a proportion "
f"(would need n of about {n_for_zero_upper(0.0010/100):,.0f})")
print("\n3. Sample size at which a 0/n Wilson 95% upper bound is 0.0010")
print(f" exactly 0.0010: n = {n_for_zero_upper(0.0010):.1f}")
print(f" rounds to 0.0010 (0.00095 <= upper < 0.00105): "
f"{n_for_zero_upper(0.00105):.1f} < n <= {n_for_zero_upper(0.00095):.1f}")
print(f" largest DTG-exposed pregnancy/outcome counts: {N_DTG_WOMEN} women, {N_DTG_OUT} outcomes; "
f"updated DTG denominator {UPD_FRAC*UPD_NEW + N_DTG_WOMEN:.0f}; "
f"whole analysed cohort {N_DTG_WOMEN + N_EFV_WOMEN} women, {N_DTG_OUT + N_EFV_OUT} outcomes")
LARGER = [ # larger counts printed in the paper that are not DTG-exposed pregnancy denominators
("possible pregnancies screened (Results para 1)", 3390),
("selected for investigation (Results para 1)", 2507),
("Brazilian women on DTG aged 15-49, May 2018 (Introduction)", 22624),
]
print(" larger counts in the paper (not DTG-exposed pregnancy denominators), 0/n Wilson 95% upper:")
for lab, n in LARGER:
u = wilson(0, n)[1]
print(f" {lab:60s} n={n:6d} upper={u:.5f} rounds to {round(u, 4):.4f} "
f"in rounding range for 0.0010? {n_for_zero_upper(0.00105) < n <= n_for_zero_upper(0.00095)}")
print("\n4. Checks that the stated method and denominators reproduce the paper's other CIs")
for arm, n in (("DTG", N_DTG_OUT), ("EFV", N_EFV_OUT)):
x = COMPOSITE[arm]
lo, hi = wilson(x, n)
print(f" composite {arm} {x}/{n}: {100*x/n:.1f}% Wilson {lo:.4f}-{hi:.4f} "
f"(reported {REPORTED[arm + ' composite'][0]:.3f}-{REPORTED[arm + ' composite'][1]:.3f})")
print(f" Table 3 stillbirths + abortions {arm}: {STILL[arm]} + {ABORT[arm]} = {STILL[arm]+ABORT[arm]}")
print(f" Table 3 live + still + abortion {arm}: "
f"{(N_DTG_LIVE if arm == 'DTG' else N_EFV_LIVE) + STILL[arm] + ABORT[arm]} (N = {n})")
print(f" chart-review fraction 382/490 = {CHART_CONFIRMED/CHART_REVIEWED:.4f}")
n_upd = UPD_FRAC * UPD_NEW + N_DTG_WOMEN
lo, hi = wilson(UPD_EVENTS, n_upd)
print(f" updated estimate 2/(0.78*900+382) = 2/{n_upd:.0f} = {UPD_EVENTS/n_upd:.5f} "
f"Wilson {lo:.5f}-{hi:.5f} (reported 0.0018, 0.0005-0.0067)")
print("\n5. What 0/384 can exclude: comparator rates cited in the paper")
print(" (a rate on the bound itself counts as not excluded)")
for r, lab, _ in COMPARATORS:
p0 = (1 - r) ** N_DTG_OUT
print(f" {lab:36s} inside reported CI (<=0.0010)? {r <= 0.0010!s:5s} "
f"inside recomputed CI (<={dtg_w:.4f})? {r <= dtg_w!s:5s} P(0 events in 384) = {p0:.4f}")
print(f" gap between 1.04% and the recomputed bound: {100*(POWER_ASSUMED_DTG - dtg_w):.3f} "
f"percentage points")
print(" two-sided 95% interval: each tail has probability 0.025")
# ------------------------------------------------------------------ figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, REF = "#eb6834", "#2a78d6", "#e4e3df", "#9a9893"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
# Figure 1: reported vs recomputed intervals on one axis (percent), with every
# rate the paper cites drawn as its own row and a thin dashed line up through
# the interval rows
from matplotlib.lines import Line2D # noqa: E402
upd_lo, upd_hi = wilson(UPD_EVENTS, n_upd)
items = [ # (label, point, lo, hi, color)
("DTG, 0/384 — reported", 0, 0, REPORTED["DTG NTD upper"], ORANGE),
("DTG, 0/384 — recomputed (Wilson)", 0, 0, dtg_w, BLUE),
("EFV, 0/1068 — reported", 0, 0, REPORTED["EFV NTD upper"], ORANGE),
("EFV, 0/1068 — recomputed (Wilson)", 0, 0, efv_w, BLUE),
("Updated DTG, 2/1084 — reported", 0.0018, 0.0005, 0.0067, ORANGE),
("Updated DTG, 2/1084 — recomputed", UPD_EVENTS / n_upd, upd_lo, upd_hi, BLUE),
]
ys = [12.2, 11.6, 10.4, 9.8, 8.6, 8.0]
rate_ys = [6.6 - 0.6 * i for i in range(len(COMPARATORS))] # descending rates, top to bottom
TOP = 12.9
LBL_BOX = dict(boxstyle="square,pad=0.12", fc=BG, ec="none")
fig, ax = plt.subplots(figsize=(7.2, 6.6))
for (r, lab, is_dtg), y in zip(COMPARATORS, rate_ys):
ax.plot([100 * r, 100 * r], [y, TOP], color=REF, lw=0.9, ls=(0, (3, 3)), zorder=1)
ax.plot([100 * r], [y], marker="D", ms=8, color=SEC if is_dtg else BG, mec=SEC, mew=1.5,
zorder=4)
ax.text(100 * r + 0.02, y, f"{100*r:.2f}%", va="center", ha="left", fontsize=10,
color=SEC, zorder=5)
for (lab, p, lo, hi, col), y in zip(items, ys):
ax.plot([100 * lo, 100 * hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([100 * hi], [y], marker="|", ms=10, mew=2, color=col, zorder=3)
ax.plot([100 * p], [y], marker="o", ms=8, color=col, zorder=4, mfc=col, mec=BG, mew=1)
ax.text(100 * hi + 0.015, y, f"{100*hi:.2f}%", va="center", ha="left", fontsize=10,
color=col, bbox=LBL_BOX, zorder=5)
ax.set_yticks(ys + rate_ys)
ax.set_yticklabels([it[0] for it in items] + [c[1].rsplit(" ", 1)[0].rstrip(",").replace("power calc.", "power calculation")
for c in COMPARATORS],
fontsize=10)
for t, c in zip(ax.get_yticklabels(), [TXT] * len(items) + [SEC] * len(COMPARATORS)):
t.set_color(c)
ax.text(-0.03, 7.25, "Rates the paper cites (filled = DTG-exposed, hollow = not DTG-exposed)",
ha="left", va="center", fontsize=10, color=SEC, bbox=LBL_BOX, zorder=5)
ax.set_ylim(rate_ys[-1] - 0.5, TOP)
ax.set_xlim(-0.03, 1.2)
ax.set_xlabel("NTD prevalence, % (point estimate and 95% CI)")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.annotate("", xy=(100 * dtg_w, 11.9), xytext=(100 * REPORTED["DTG NTD upper"], 11.9),
arrowprops=dict(arrowstyle="->", color=SEC, lw=1.2), zorder=5)
ax.text(100 * (dtg_w + REPORTED["DTG NTD upper"]) / 2, 11.93,
f"{dtg_w / REPORTED['DTG NTD upper']:.1f}× wider", ha="center", va="bottom",
fontsize=10, color=SEC, bbox=LBL_BOX, zorder=5)
ax.legend(handles=[Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="Reported in the paper"),
Line2D([], [], color=BLUE, lw=2, marker="o", ms=8,
label="Recomputed from the paper's counts (Wilson 95%)"),
Line2D([], [], color=SEC, lw=0.9, ls=(0, (3, 3)), marker="D", ms=8,
label="Rate cited in the paper")],
loc="upper center", bbox_to_anchor=(0.3, -0.1), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: Wilson 0/n upper bound as a function of n
n_grid = np.logspace(2, 4.2, 400)
fig, ax = plt.subplots(figsize=(7.2, 4.0))
ax.plot(n_grid, 100 * Z * Z / (n_grid + Z * Z), color=BLUE, lw=2)
n_req = n_for_zero_upper(REPORTED["DTG NTD upper"])
ax.axhline(100 * REPORTED["DTG NTD upper"], color=ORANGE, lw=1.2, ls=(0, (4, 3)))
ax.plot([N_DTG_OUT], [100 * REPORTED["DTG NTD upper"]], "o", ms=9, color=ORANGE)
ax.plot([N_DTG_OUT], [100 * dtg_w], "o", ms=9, color=BLUE, mec=BG)
ax.plot([N_EFV_OUT], [100 * efv_w], "s", ms=9, color=BLUE, mec=BG)
ax.plot([n_req], [100 * REPORTED["DTG NTD upper"]], "D", ms=8, mfc=BG, mec=BLUE, mew=2,
)
ax.annotate(f"DTG, n = 384: {100*dtg_w:.2f}%", (N_DTG_OUT, 100 * dtg_w), xytext=(10, 6),
textcoords="offset points", color=BLUE, fontsize=10)
ax.set_xscale("log")
ax.set_yscale("log")
ax.set_xlim(100, 15000)
ax.set_ylim(0.02, 5)
ax.set_xticks([100, 384, 1068, 3838, 10000])
ax.set_xticklabels(["100", "384", "1068", "3838", "10 000"])
ax.set_yticks([0.05, 0.1, 0.36, 1, 2])
ax.set_yticklabels(["0.05%", "0.10%", "0.36%", "1%", "2%"])
ax.minorticks_off()
ax.set_xlabel("Number of birth outcomes, n (log scale)")
ax.set_ylabel("Upper 95% bound, % (log scale)")
ax.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.annotate("Reported: 0.0010 at n = 384", (N_DTG_OUT, 100 * REPORTED["DTG NTD upper"]),
xytext=(10, -16), textcoords="offset points", color=ORANGE, fontsize=10)
ax.annotate(f"EFV, n = 1068: {100*efv_w:.2f}%\n(matches reported 0.0036)", (N_EFV_OUT, 100 * efv_w),
xytext=(12, 4), textcoords="offset points", color=BLUE, fontsize=10)
ax.annotate(f"n = {n_req:,.0f} needed for 0.0010", (n_req, 100 * REPORTED["DTG NTD upper"]),
xytext=(8, 8), textcoords="offset points", color=BLUE, fontsize=10)
ax.text(700, 2.3, "Line: Wilson 95% upper bound for 0 events in n", color=BLUE, fontsize=10, ha="left")
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png")
Sensitivity and specificity are interchanged in Tables 1 and 3 and the headline accuracy claim
Lancet Microbe (2025) · doi:10.1016/j.lanmic.2024.100987 · PMC11959112
Summary
The Abstract, Results and Discussion state that the eight-antigen MpoxPlex positivity score separates positive from presumed-negative sera "with 98% sensitivity and 95% specificity". Table 3, the source of those figures, has 63 positive and 435 negative samples. The printed "sensitivity" for the eight-antigen row, 98·39%, is exactly 428/435, and no fraction over 63 equals it. The printed "specificity", 95·24%, is exactly 60/63, and no fraction over 435 equals it. Applying the paper's own cutoffs to the authors' deposited per-sample data gives 60 of 63 positives detected and 428 of 435 negatives correctly negative. So the headline result is 95·2% sensitivity and 98·4% specificity, the reverse of what is printed. The same swap affects every row of Table 1 (12 antigens) and every row of Table 3 (12 antigen combinations). The paper's discrimination (AUC 0·987) and its choice of antigens are unaffected. What changes is the error profile readers are told to expect: the assay misses about 1 positive in 21, not 1 in 62, and gives a false positive in about 1 in 62 negatives, not 1 in 21.
What the paper reports
Abstract, Findings:
Using the results from a combination of eight antigens, we were able to distinguish samples as either post-vaccination or post-infection from negative samples with a sensitivity of 98% and a specificity of 95%.
Results, positivity-score paragraph:
The AUC for this combination was 0·987 (p<0·0001) and using a cutoff sample positivity score of five out of eight antigens could distinguish positive from presumed negative samples with 98% sensitivity and 95% specificity.
Discussion, first paragraph:
The combined results from eight of these antigens (MPXV antigens E8, B6, B2, M1, A35, and H3 and VACV antigens B5 and A33) were able to distinguish positive samples, either after vaccination or infection, from presumed negative samples with 98% sensitivity and 95% specificity.
Table 3 caption, column headings, and the Top 8 and Top 12 rows:
Summary characteristics of ROC curves and positivity score cutoffs generated by comparing the positivity scores of all positive samples (n=63) and all negative samples (n=435) to 12 MPXV and VACV antigens by ROC
| AUCs (95% CI) | Cutoff (positivity score) | Sensitivity (95% CI) | Specificity (95% CI)
Top 12 | 0·980 (0·954–1·00) | ≥5 | 95·17% (92·73–96·82) | 98·41% (91·54–99·92)
Top 8 | 0·987 (0·965–1·00) | ≥5 | 98·39% (96·72–99·22) | 95·24% (86·91–98·70)
Table 1 caption and first row (same column headings as Table 3):
Summary of ROC curves and MFI cutoffs for presumed negative samples (n=63) vs positive samples (n=435) to 12 MPXV and VACV antigens by ROC analysis
VACV antigen B5 | 0·980 (0·959–1·00) | >87·41 | 94·94% (92·46–96·64) | 95·24% (86·91–98·7)
Figure 1 legend, giving the group sizes:
There were 435 presumed negative samples, 24 post-MPXV infection samples, and 39 post-Imvanex vaccination samples (14–157 days after the second dose).
Methods (Study design) give the negative groups as 224 paediatric sera, 52 + 34 + 44 + 63 confounder sera (cytomegalovirus, Epstein–Barr virus, varicella zoster virus, rheumatoid factor) and 18 pre-vaccination sera. Data sharing statement:
Data will be made available on request or on Dec 18, 2024, at https.doi.org/10.17632/j4xdjfxskk.2.
What the paper's own numbers show
Method. Sensitivity is the share of positive samples (here 24 infected + 39 vaccinated = 63) that test positive. Specificity is the share of presumed negatives (224 + 52 + 34 + 44 + 63 + 18 = 435) that test negative. For each of the 48 percentages in Tables 1 and 3, I found every whole-number fraction k/63 and k/435 that rounds to the printed two-decimal value. I also compared the Wilson 95% interval of that fraction with the printed interval. Each printed value is matched by exactly one of the two denominators, so the denominator shows which group a column was computed on. As an independent check, I downloaded the per-sample MFIs that the authors deposited at Mendeley Data (doi:10.17632/j4xdjfxskk.2). I applied the paper's Table 1 MFI cutoffs and its Table 3 positivity-score cutoffs and counted true positives among the 63 positives and true negatives among the 435 negatives. The 39 vaccinee samples that reproduce the paper's counts are those taken 14, 43, 63, 84 and 122 days after dose two (8 + 10 + 7 + 7 + 7). The script reads only the group label and the 12 MFI columns. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Positives / negatives | Methods; Figure 1 legend; Table 3 caption | 24 + 39; 435 | 63 positives; 224+52+34+44+63+18 = 435 negatives |
| Top 8 "Sensitivity" | Table 3; Abstract, Results, Discussion ("98%") | 98·39% (96·72–99·22) | 428/435 = 98·39%, Wilson 96·72–99·22 (exact match). Attainable values over 63 are 61/63 = 96·83% and 62/63 = 98·41%, so 98·39% cannot be a rate among the positives |
| Top 8 "Specificity" | Table 3; Abstract, Results, Discussion ("95%") | 95·24% (86·91–98·70) | 60/63 = 95·24%, Wilson lower bound 86·91 (match). Attainable values over 435 are 414/435 = 95·17% and 415/435 = 95·40%, so 95·24% cannot be a rate among the negatives |
| Top 8, from deposited MFIs (score ≥5 of 8) | data deposit | — | TP 60, FN 3, TN 428, FP 7: sensitivity 60/63 = 95·24%, specificity 428/435 = 98·39% |
| Top 12 "Sensitivity" / "Specificity" | Table 3 | 95·17% / 98·41% | 414/435 (Wilson 92·73–96·82, exact) / 62/63 (Wilson lower 91·54, exact). Deposit: TP 62/63, TN 414/435 |
| Top 4 "Sensitivity" / "Specificity" | Table 3 | 99·77% / 95·24% | 434/435 / 60/63. Deposit: TP 60/63, TN 434/435 |
| All 12 Table 3 rows | Table 3 | "Sensitivity" 94·94–99·77%; "Specificity" 95·24–98·41% | every "Sensitivity" equals TN/435 and every "Specificity" equals TP/63, from the denominators (12/12) and from the deposit (12/12) |
| All 12 Table 1 antigens | Table 1 | e.g. VACV B5: 94·94% / 95·24% | every "Sensitivity" equals TN/435 and every "Specificity" equals TP/63, from the denominators and from the deposit (12/12). VACV B5: TN 413/435, TP 60/63 |
| Table 1 group sizes | Table 1 caption | "presumed negative samples (n=63) vs positive samples (n=435)" | negatives n=435, positives n=63 (also swapped) |
| MPXV A27, infection vs all others | Table 2; Abstract ("88%", "97%") | 87·5% (69·00–95·66) / 96·84% (94·84–98·07) | 21/24 and 459/474, Wilson intervals exact; deposit gives 21/24 and 459/474. Correctly labelled |
| AUC, Top 8 score | Table 3 | 0·987 | 0·9865 from the deposit. The AUC does not depend on the labels |
The printed confidence intervals use the Wilson score method for every lower bound. Some upper bounds of proportions close to 100% are slightly wider than Wilson (for 60/63 the printed upper bound is 98·70, Wilson gives 98·37). This affects neither the point estimates nor the denominator they imply.
Figure
Figure 1. Where the two headline percentages come from. Of the 63 positive samples (24 after infection, 39 after vaccination), 60 have an eight-antigen positivity score of 5 or more: 60/63 = 95·24%, the sensitivity. Of the 435 presumed negatives, 428 score below 5: 428/435 = 98·39%, the specificity. Table 3, the Abstract, the Results and the Discussion print 95·24% as specificity and 98·39% as sensitivity. Blue boxes: values recomputed from the paper's group sizes and confirmed with the deposited per-sample data. Orange boxes: labels as printed in Table 3.
Figure 2. The swap is systematic. For each Table 3 row (antigen combinations Top 12 to Top 1), orange markers are the printed "Sensitivity" (dot) and "Specificity" (square). Blue open markers are values recomputed from the authors' deposited MFIs with the paper's cutoffs: specificity, TN/435 (ring), and sensitivity, TP/63 (open square). In all 12 rows the printed "Sensitivity" sits inside the recomputed specificity ring, and the printed "Specificity" sits inside the recomputed sensitivity square. The x-axis runs from 90% to 100% because every value lies in that range.
Why it matters
This is the paper's primary diagnostic-accuracy result. It is the first quantitative finding in the Abstract and is restated in the Results and Discussion, and accuracy studies of this kind are cited mainly for these two numbers. As printed, the figures tell readers that the eight-antigen score misses about 1.6% of positive sera (1 in 62) and wrongly flags about 4.8% of negative sera (1 in 21). The data show the opposite: 3 of 63 positives were missed (4.8%) and 7 of 435 negatives were flagged (1.6%). The two uses the paper proposes for the assay depend on these numbers in different ways. For serosurveillance in a low-prevalence population, specificity sets the false-positive burden and the positive predictive value, and the assay performs better there than reported. For confirming past infection or vaccination in an individual, sensitivity matters more, and the assay performs worse there than reported. The swap also changes the per-antigen trade-offs in Table 1. MPXV A27 and VACV A27, for example, are printed as about 86% sensitive and about 45% specific in the all-positives comparison, but are in fact about 45% sensitive and about 86% specific. Sensitivity is also the less precisely estimated of the two numbers (63 positives versus 435 negatives), so the wider of the two printed intervals belongs to it and not to specificity.
The main conclusions survive. Both corrected values are at least 95%, the AUC and the antigen ranking do not depend on the labels, and the cutoffs were chosen to maximise the sum of sensitivity and specificity, which is the same whichever way the two are labelled. The infection-versus-vaccination result for MPXV A27 (Table 2, Abstract: 88% sensitivity, 97% specificity) is labelled correctly. The medRxiv preprint (doi:10.1101/2024.06.03.24308186, v1, 4 June 2024) gives the same reversed values in its abstract ("a sensitivity of 98.39% ... and specificity of 95.24%"), so they may already have been repeated in work citing either version.
Explanations considered
- The paper uses a non-standard definition, with negative sera as the target condition. This does not fit. The paper defines positives as post-infection or post-vaccination sera throughout. The Abstract says the assay distinguishes "post-vaccination or post-infection from negative samples", and Table 2 uses the standard orientation (sensitivity on the 24 infected sera). Under a negatives-as-target reading, Table 2's labels would be reversed instead, and the Abstract would still be describing detection of positives.
- The labels are right and the denominators are what was swapped. That would mean 98·39% is a rate among the 63 positives, but no k/63 equals 98·39% (62/63 = 98·41%). It would also mean 95·24% is a rate among the 435 negatives, but no k/435 equals 95·24%. The Wilson intervals match only the swapped denominators. Read literally, the printed "sensitivity" counts 428 of 435 negatives as test-positive, a 98% false-positive rate, which an AUC of 0·987 rules out.
- Rounding, or a different CI method. Neither explains it. Each point estimate is matched to two decimals by exactly one denominator, and the Wilson lower bounds match exactly. The only CI difference (some upper bounds slightly wider than Wilson) is too small to bear on which denominator was used.
- A different analysis population. The deposited data rule this out. With the 24 infected sera, the 39 vaccinee sera taken 14–122 days after dose two and the 435 negatives, the paper's cutoffs reproduce every one of the 48 Table 1 and Table 3 percentages exactly, once the labels are swapped.
- A conversion artifact in PMC. Excluded. The publisher's typeset PDF shows the same column order, values, Table 1 caption and Abstract wording as the PMC and Europe PMC XML.
Status
- Version of record. Checked against the publisher's typeset PDF (Lancet Microbe vol 6, April 2025, article 100987; published online 17 January 2025), using the copy in the University of Aberdeen repository (AURA), because thelancet.com returned HTTP 403 in this session. Also checked against the Europe PMC full-text XML. The Abstract, Results and Discussion sentences, Table 3's column order and values, and the Table 1 caption are identical in all of them. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements and data. The supplementary appendix (mmc1.pdf) holds ROC curves (Figure S1), ELISA correlations (Figure S2), sample counts (Table S1, confirming 435 negatives and 24 infection sera) and geometric mean MFIs (Table S2). It has no contingency counts. The authors' data deposit (Mendeley Data, doi:10.17632/j4xdjfxskk.2, "Raw Data.ods") holds per-sample MFIs, and these settle the labelling, as shown above. The deposit also contains dates of birth and sex. Those columns were not read or used.
- A separate, minor inconsistency. The Figure 1 legend, the Table 2 caption and the Methods describe the 39 vaccinee samples as taken 14–157 days after the second dose. The Methods' own timepoint counts for 14–157 days add up to 8 + 10 + 7 + 7 + 7 + 7 = 46. The 39 samples that reproduce the tables are the 14–122 day samples, which is closer to the supplement's description of these samples as "14 to 144 days" after two doses.
- Corrections. No correction, erratum, retraction or expression of concern was found in Crossref or PMC (checked 2026-10-24).
- Comments. PubPeer's search page returned a Cloudflare challenge and its API returned HTTP 404 in this session, so PubPeer could not be searched directly. A web search for the assay name together with PubPeer, sensitivity and specificity found no comment raising this issue.
- Sources. PMC11959112; Europe PMC full text and supplementary files; AURA repository copy of the version of record; Mendeley Data doi:10.17632/j4xdjfxskk.2 (version 2); medRxiv doi:10.1101/2024.06.03.24308186 (via the bioRxiv/medRxiv API); Crossref. As of 2026-10-24.
Suggested correction
- Abstract (Findings), Results and Discussion: replace "a sensitivity of 98% and a specificity of 95%" and "98% sensitivity and 95% specificity" with "a sensitivity of 95% and a specificity of 98%" (60/63 = 95·24%; 428/435 = 98·39%).
- Table 3: swap the contents of the Sensitivity and Specificity columns in all 12 rows, carrying each confidence interval with its estimate. For Top 8, for example: sensitivity 95·24% (86·91–98·70), specificity 98·39% (96·72–99·22).
- Table 1: make the same column swap for all 12 antigens, and correct the caption to "presumed negative samples (n=435) vs positive samples (n=63)".
- Optionally, align the stated vaccinee window for the 39 samples (14–157 days in the Methods, Figure 1 legend and Table 2 caption) with the samples actually analysed.
Recomputation script (python recompute.py.txt)
"""Recompute the sensitivity / specificity labels of PMC11959112
(Lancet Microbe 2025, doi:10.1016/j.lanmic.2024.100987, "Assessment of MpoxPlex ...").
Run: python3 recompute.py.txt
Needs numpy, scipy, matplotlib; pandas + odfpy for the optional deposit check (part C).
Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png) next to
this file.
Part A and B use only numbers printed in the paper (Tables 1-3, Methods, Figure 1 legend).
Part C downloads the authors' deposited per-sample MFIs (Mendeley Data,
doi:10.17632/j4xdjfxskk.2, file "Raw Data.ods") into a temporary directory, reads ONLY the
'Group' column and the 12 antigen MFI columns, prints aggregate counts, and deletes the
file. The deposit also holds identifiers, dates of birth and sex; these are never read,
printed or saved. If the download fails, part C falls back to the aggregate counts that an
earlier run printed (embedded below) and says so.
"""
import os
import sys
import tempfile
import urllib.request
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
def wilson(k, n):
p = k / n
d = 1 + Z * Z / n
c = (p + Z * Z / (2 * n)) / d
h = Z * np.sqrt(p * (1 - p) / n + Z * Z / (4 * n * n)) / d
return 100 * (c - h), 100 * (c + h)
def fit(pct, ns):
"""All (k, n) with n in ns whose k/n rounds to the printed two-decimal percentage."""
out = []
for n in ns:
for k in (int(np.floor(pct / 100 * n)), int(np.ceil(pct / 100 * n))):
if abs(100 * k / n - pct) < 0.005 + 1e-9 and (k, n) not in out:
out.append((k, n))
return out
# ------------------------------------------------------------------ group sizes (Methods)
NEG_PARTS = {"paediatric": 224, "CMV": 52, "EBV": 34, "VZV": 44, "rheumatoid factor": 63,
"pre-MVA-BN": 18}
N_NEG = sum(NEG_PARTS.values())
N_INF, N_VAC = 24, 39 # Figure 1 legend; Table 2 caption (n=39 post-MVA-BN)
N_POS = N_INF + N_VAC
# ------------------------------------------------------------------ Table 3 as printed
# row: (cutoff, sens, sens CI, spec, spec CI)
T3 = {
"Top 12": (5, 95.17, (92.73, 96.82), 98.41, (91.54, 99.92)),
"Top 11": (5, 96.55, (94.39, 97.90), 96.83, (89.14, 99.44)),
"Top 10": (5, 96.78, (94.67, 98.07), 96.83, (89.14, 99.44)),
"Top 9": (5, 98.16, (96.41, 99.07), 95.24, (86.91, 98.70)),
"Top 8": (5, 98.39, (96.72, 99.22), 95.24, (86.91, 98.70)),
"Top 7": (5, 98.39, (96.72, 99.22), 95.24, (86.91, 98.70)),
"Top 6": (5, 98.85, (97.34, 99.51), 95.24, (86.91, 98.70)),
"Top 5": (4, 98.62, (97.02, 99.37), 95.24, (86.91, 98.70)),
"Top 4": (4, 99.77, (98.71, 99.99), 95.24, (86.91, 98.70)),
"Top 3": (3, 99.08, (97.66, 99.64), 95.24, (86.91, 98.70)),
"Top 2": (2, 98.16, (96.41, 99.07), 95.24, (86.91, 98.70)),
"Top 1": (1, 94.94, (92.46, 96.64), 95.24, (86.91, 98.70)),
}
# ------------------------------------------------------------------ Table 1 as printed
# antigen (deposit column): (label, MFI cutoff, sens, sens CI lower, spec, spec CI lower)
T1 = {
"VACV B5R": ("VACV B5", 87.41, 94.94, 92.46, 95.24, 86.91),
"MPXV E8L": ("MPXV E8", 101.3, 96.55, 94.39, 95.24, 86.91),
"MPXV B6R": ("MPXV B6", 64.35, 94.71, 92.19, 95.24, 86.91),
"MPXV B2R": ("MPXV B2", 41.24, 93.10, 90.33, 96.83, 89.14),
"VACV A33R": ("VACV A33", 294.9, 92.41, 89.54, 93.65, 84.78),
"MPXV M1R": ("MPXV M1", 31.16, 90.11, 86.95, 92.06, 82.73),
"MPXV A35R": ("MPXV A35", 136.5, 91.26, 88.24, 87.30, 76.89),
"MPXV H3L": ("MPXV H3", 296.4, 91.72, 88.76, 87.30, 76.89),
"MPXV A29L": ("MPXV A29", 30.17, 73.79, 69.47, 63.49, 51.15),
"MPXV A5L": ("MPXV A5", 156.4, 63.91, 59.29, 71.43, 59.30),
"MPXV A27L": ("MPXV A27", 259.5, 86.67, 83.15, 46.03, 34.31),
"VACV A27L": ("VACV A27", 281.4, 85.29, 81.65, 44.44, 32.85),
}
ORDER = list(T1) # Table 3 footnote order: B5, E8, B6, B2, A33, M1, A35, H3, A29, A5, A27, A27
# Table 2 (infection vs everything else), MPXV A27 row
T2_A27 = (718.0, 87.5, (69.00, 95.66), 96.84, (94.84, 98.07))
print("=" * 78)
print("A. Group sizes from the paper")
print(" presumed negatives:", " + ".join(f"{v}" for v in NEG_PARTS.values()), "=", N_NEG)
print(f" positives used for Tables 1 and 3: {N_INF} infected + {N_VAC} vaccinated = {N_POS}")
print("=" * 78)
print("B. Which denominator reproduces each printed percentage (paper's numbers only)")
print(" Table 3")
for r, (c, se, seci, sp, spci) in T3.items():
a, b = fit(se, (N_POS, N_NEG)), fit(sp, (N_POS, N_NEG))
fa = "; ".join(f"{k}/{n} -> Wilson {wilson(k, n)[0]:.2f}-{wilson(k, n)[1]:.2f}" for k, n in a)
fb = "; ".join(f"{k}/{n} -> Wilson {wilson(k, n)[0]:.2f}-{wilson(k, n)[1]:.2f}" for k, n in b)
print(f" {r:6s} 'Sensitivity' {se:.2f} ({seci[0]:.2f}-{seci[1]:.2f}) = {fa or 'none'}")
print(f" {'':6s} 'Specificity' {sp:.2f} ({spci[0]:.2f}-{spci[1]:.2f}) = {fb or 'none'}")
n_ok = sum(fit(v[1], (N_POS, N_NEG)) == [(fit(v[1], (N_NEG,))[0][0], N_NEG)] for v in T3.values())
print(f" Table 3 'Sensitivity' values reproduced only by a k/435 fraction: {n_ok}/12")
n_ok = sum(fit(v[3], (N_POS, N_NEG)) == [(fit(v[3], (N_POS,))[0][0], N_POS)] for v in T3.values())
print(f" Table 3 'Specificity' values reproduced only by a k/63 fraction: {n_ok}/12")
print(" Nearest attainable values around the Top 8 entries:")
print(f" with n=63: 61/63 = {100*61/63:.2f}%, 62/63 = {100*62/63:.2f}% (98.39% not attainable)")
print(f" with n=435: 414/435 = {100*414/435:.2f}%, 415/435 = {100*415/435:.2f}% (95.24% not attainable)")
print(" Table 1 (point estimate -> k/n; Wilson lower bound vs printed lower bound)")
t1_fit = {}
for col, (lab, cut, se, selo, sp, splo) in T1.items():
a, b = fit(se, (N_POS, N_NEG)), fit(sp, (N_POS, N_NEG))
t1_fit[col] = (a, b)
sa = ", ".join(f"{k}/{n} (W lo {wilson(k, n)[0]:.2f})" for k, n in a)
sb = ", ".join(f"{k}/{n} (W lo {wilson(k, n)[0]:.2f})" for k, n in b)
print(f" {lab:9s} 'Sens' {se:.2f} [lo {selo:.2f}] = {sa} | 'Spec' {sp:.2f} [lo {splo:.2f}] = {sb}")
print(" Table 2, MPXV A27 (correctly labelled): "
f"21/24 = {100*21/24:.2f}% Wilson {wilson(21, 24)[0]:.2f}-{wilson(21, 24)[1]:.2f}; "
f"459/474 = {100*459/474:.2f}% Wilson {wilson(459, 474)[0]:.2f}-{wilson(459, 474)[1]:.2f}")
print(" Upper bounds: printed 86.91-98.70 for 60/63; Wilson gives "
f"{wilson(60, 63)[0]:.2f}-{wilson(60, 63)[1]:.2f}. Lower bounds match Wilson in every row;")
print(" some upper bounds of proportions near 100% do not (a different upper-bound rule), "
"which does not bear on the denominator.")
print("=" * 78)
print("C. Deposited per-sample MFIs (doi:10.17632/j4xdjfxskk.2)")
URL = ("https://data.mendeley.com/public-files/datasets/j4xdjfxskk/files/"
"5f601456-b94a-4b82-9b5a-be54dd3c5a91/file_downloaded")
VAC_GROUPS = [f"Post Dose 2: {d} days" for d in (14, 43, 63, 84, 122)]
# Aggregate counts printed by an earlier run of this script (fallback only).
FALLBACK = {
"n": (435, 24, 39),
"t1": {"VACV B5R": (60, 413), "MPXV E8L": (60, 420), "MPXV B6R": (60, 412),
"MPXV B2R": (61, 405), "VACV A33R": (59, 402), "MPXV M1R": (58, 392),
"MPXV A35R": (55, 397), "MPXV H3L": (55, 399), "MPXV A29L": (40, 321),
"MPXV A5L": (45, 278), "MPXV A27L": (29, 377), "VACV A27L": (28, 371)},
"t3": {12: (62, 414), 11: (61, 420), 10: (61, 421), 9: (60, 427), 8: (60, 428),
7: (60, 428), 6: (60, 430), 5: (60, 429), 4: (60, 434), 3: (60, 431),
2: (60, 427), 1: (60, 413)},
"t2": (21, 459), "auc8": None,
}
dep = None
try:
import pandas as pd
with tempfile.TemporaryDirectory() as td:
fn = os.path.join(td, "raw.ods")
req = urllib.request.Request(URL, headers={"User-Agent": "Mozilla/5.0 (recompute script)"})
with urllib.request.urlopen(req, timeout=120) as r, open(fn, "wb") as f:
f.write(r.read())
use = ["Group"] + ORDER
sh = pd.read_excel(fn, engine="odf", sheet_name=None)
neg = sh["NEGATIVE_Lx"][use]
vac = sh["VACCINATED_Lx"][use]
inf = sh["MPXV_Lx"][use]
del sh
vac39 = vac[vac.Group.isin(VAC_GROUPS)]
pos = pd.concat([inf, vac39])
cuts = pd.Series({c: T1[c][1] for c in ORDER})
dep = {"n": (len(neg), len(inf), len(vac39)), "t1": {}, "t3": {}}
for c in ORDER:
dep["t1"][c] = (int((pos[c] > cuts[c]).sum()), int((neg[c] <= cuts[c]).sum()))
for k in range(12, 0, -1):
cut = T3[f"Top {k}"][0]
ps = pos[ORDER[:k]].gt(cuts[ORDER[:k]]).sum(axis=1)
ns = neg[ORDER[:k]].gt(cuts[ORDER[:k]]).sum(axis=1)
dep["t3"][k] = (int((ps >= cut).sum()), int((ns < cut).sum()))
if k == 8:
x, y = ps.to_numpy()[:, None], ns.to_numpy()[None, :]
dep["auc8"] = float(((x > y) + 0.5 * (x == y)).mean())
rest = pd.concat([neg, vac39])
dep["t2"] = (int((inf["MPXV A27L"] > T2_A27[0]).sum()),
int((rest["MPXV A27L"] <= T2_A27[0]).sum()))
src = "downloaded now"
except Exception as e: # network or odfpy missing
dep = FALLBACK
src = f"download/parse failed ({type(e).__name__}); using embedded counts from an earlier run"
print(" source:", src)
nn, ni, nv = dep["n"]
print(f" samples: {nn} negatives; {ni} infected; {nv} vaccinees at 14, 43, 63, 84, 122 days "
f"after dose 2 -> positives {ni + nv}")
print(" Table 1 cutoffs applied to the deposit: TP/63 (sensitivity), TN/435 (specificity);")
print(" printed 'Sensitivity' and 'Specificity' for comparison")
m1 = 0
for c in ORDER:
tp, tn = dep["t1"][c]
lab, cut, se, _, sp, _ = T1[c]
ok = abs(100 * tn / nn - se) < 0.006 and abs(100 * tp / (ni + nv) - sp) < 0.006
m1 += ok
print(f" {lab:9s} >{cut:<6}: TP {tp}/63 = {100*tp/63:6.2f}% TN {tn}/435 = {100*tn/435:6.2f}%"
f" | printed Sens {se:.2f}, Spec {sp:.2f} swapped-match={ok}")
print(f" Table 1 rows where printed Sens = TN/435 and printed Spec = TP/63: {m1}/12")
m3 = 0
for k in range(12, 0, -1):
tp, tn = dep["t3"][k]
c, se, _, sp, _ = T3[f"Top {k}"]
ok = abs(100 * tn / nn - se) < 0.006 and abs(100 * tp / (ni + nv) - sp) < 0.006
m3 += ok
print(f" Top {k:<2} >={c}: TP {tp}/63 = {100*tp/63:6.2f}% TN {tn}/435 = {100*tn/435:6.2f}%"
f" | printed Sens {se:.2f}, Spec {sp:.2f} swapped-match={ok}")
print(f" Table 3 rows where printed Sens = TN/435 and printed Spec = TP/63: {m3}/12")
tp8, tn8 = dep["t3"][8]
print(f" Top 8 (>=5 of 8): TP {tp8}, FN {63 - tp8}, TN {tn8}, FP {435 - tn8}")
print(f" sensitivity {tp8}/63 = {100*tp8/63:.2f}% (Wilson {wilson(tp8, 63)[0]:.2f}-{wilson(tp8, 63)[1]:.2f})")
print(f" specificity {tn8}/435 = {100*tn8/435:.2f}% (Wilson {wilson(tn8, 435)[0]:.2f}-{wilson(tn8, 435)[1]:.2f})")
if dep.get("auc8") is not None:
print(f" AUC of the Top 8 positivity score from the deposit: {dep["auc8"]:.4f} (printed 0.987)")
a, b = dep["t2"]
print(f" Table 2 MPXV A27 >718.0: infected above {a}/24 = {100*a/24:.2f}%, "
f"others at or below {b}/474 = {100*b/474:.2f}% (printed Sens 87.5, Spec 96.84: match)")
print(" Reading the printed labels literally for Top 8: 98.39% of 63 positives is "
f"{0.9839*63:.2f} samples (not a whole number), and 428 of 435 negatives at score >=5 would")
print(" mean a 98% false-positive rate, which an AUC of 0.987 rules out.")
# ================================================================== figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def box(ax, x, y, w, h, text, ec, fc=BG, lw=1.6, fs=10.5, weight="normal", color=INK):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.012,rounding_size=0.015",
ec=ec, fc=fc, lw=lw))
ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", fontsize=fs,
weight=weight, color=color, linespacing=1.35)
def arrow(ax, p, q, color=INK2, lw=1.6):
ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=13, lw=lw,
color=color, shrinkA=2, shrinkB=2))
# ---- fig1: where each Top 8 percentage comes from, and the label it is given
fig = plt.figure(figsize=(W, 4.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
ax.text(0.025, 0.955, "Eight-antigen positivity score, cutoff ≥5 (Table 3, row 'Top 8')",
fontsize=11.5, weight="bold", va="center")
cols = [0.025, 0.335, 0.59, 0.805]
wid = [0.275, 0.21, 0.175, 0.175]
ax.text(cols[0] + wid[0] / 2, 0.86, "Sample group (Methods)", ha="center", color=INK2, fontsize=10)
ax.text(cols[1] + wid[1] / 2, 0.86, "Correctly classified", ha="center", color=INK2, fontsize=10)
ax.text(cols[2] + wid[2] / 2, 0.86, "Recomputed\nmeaning", ha="center", color=BLUE, fontsize=10,
weight="bold", va="center")
ax.text(cols[3] + wid[3] / 2, 0.86, "Label printed\nin the paper", ha="center", color=ORANGE,
fontsize=10, weight="bold", va="center")
rows = [
(0.53, f"{N_POS} positives\n({N_INF} infected + {N_VAC} vaccinated)",
f"{tp8} score ≥5\n{tp8}/{N_POS} = {100*tp8/N_POS:.2f}%",
f"Sensitivity\n{100*tp8/N_POS:.2f}%", "'Specificity'\n95.24%"),
(0.20, f"{N_NEG} presumed negatives\n(224+52+34+44+63+18)",
f"{tn8} score <5\n{tn8}/{N_NEG} = {100*tn8/N_NEG:.2f}%",
f"Specificity\n{100*tn8/N_NEG:.2f}%", "'Sensitivity'\n98.39%"),
]
h = 0.22
for y, g, cnt, rec, lab in rows:
box(ax, cols[0], y, wid[0], h, g, INK2, fs=10)
box(ax, cols[1], y, wid[1], h, cnt, INK2, fs=10)
box(ax, cols[2], y, wid[2], h, rec, BLUE, lw=2, weight="bold", color=BLUE)
box(ax, cols[3], y, wid[3], h, lab, ORANGE, lw=2, weight="bold", color=ORANGE)
arrow(ax, (cols[0] + wid[0], y + h / 2), (cols[1], y + h / 2))
arrow(ax, (cols[1] + wid[1], y + h / 2), (cols[2], y + h / 2))
ax.text((cols[2] + wid[2] + cols[3]) / 2, y + h / 2, "≠", ha="center", va="center",
fontsize=18, color=INK)
ax.text(0.025, 0.085,
"Denominator check: 98.39% = 428/435 exactly (Wilson 96.72–99.22, as printed);\n"
"no k/63 gives 98.39%. 95.24% = 60/63 exactly; no k/435 gives 95.24%. The counts 60\n"
"and 428 are reproduced from the authors' deposited per-sample MFIs and the paper's cutoffs.",
fontsize=10, color=INK2, va="center", linespacing=1.4)
save(fig, "fig1")
# ---- fig2: every Table 3 row, printed 'Sensitivity' vs the two recomputed rates
fig, ax = plt.subplots(figsize=(W, 4.6))
fig.subplots_adjust(left=0.13, right=0.97, top=0.78, bottom=0.12)
ks = list(range(12, 0, -1))
y = np.arange(len(ks))[::-1]
sens_rec = [100 * dep["t3"][k][0] / 63 for k in ks]
spec_rec = [100 * dep["t3"][k][1] / 435 for k in ks]
sens_pr = [T3[f"Top {k}"][1] for k in ks]
spec_pr = [T3[f"Top {k}"][3] for k in ks]
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.scatter(spec_rec, y, s=150, facecolors="none", edgecolors=BLUE, lw=2, zorder=3,
label="Recomputed specificity, TN/435 (deposited MFIs)")
ax.scatter(sens_rec, y, s=150, marker="s", facecolors="none", edgecolors=BLUE, lw=2, zorder=3,
label="Recomputed sensitivity, TP/63 (deposited MFIs)")
ax.scatter(sens_pr, y, s=40, color=ORANGE, zorder=4, label="Printed 'Sensitivity'")
ax.scatter(spec_pr, y, s=40, marker="s", color=ORANGE, zorder=4, label="Printed 'Specificity'")
ax.set_yticks(y)
ax.set_yticklabels([f"Top {k}" for k in ks])
ax.set_xlim(90, 100)
ax.set_xlabel("Percent of samples correctly classified (Table 3 rows; x-axis 90–100%)")
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="lower center", bbox_to_anchor=(0.43, 1.0), ncol=2, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.2)
fig.text(0.02, 0.955, "Table 3: printed 'Sensitivity' = recomputed specificity, in all 12 rows",
fontsize=11.5, weight="bold", ha="left")
save(fig, "fig2")
print("=" * 78)
print("figures written: fig1.svg/png, fig2.svg/png in", HERE)
Reported p<0.001 for mouse leak frequencies cannot be reproduced; the paper's counts give p=0.11–0.36
Lancet Neurology (2026) · doi:10.1016/S1474-4422(26)00140-7 · PMC13275636
Summary
The Results report that CSF leaks were significantly more frequent in all three Fbn2 mutant mouse strains than in wild-type mice: relative risks (RR) of 1.38, 1.33 and 1.27, 95% confidence intervals that exclude 1, and "all comparisons p<0·001". The legend of Figure 4A goes further, marking p<0.0001 for two of the strains. The paper names the χ² test for this comparison. Applied to the paper's own counts (15/19, 22/29 and 8/11 against 20/35 in wild-type), that test gives p = 0.109, 0.117 and 0.356. Fisher's exact test gives 0.14, 0.19 and 0.49, and the 95% CIs for the RRs are 0.96–2.00, 0.93–1.89 and 0.80–2.02, so all three include 1. The reported CIs are reproduced to two decimals only if each percentage is treated as a count out of 100 mice per group. The comparison is the main in-vivo support for the abstract's statement that the mutant mice "showed a predisposition for dural rupture". The paper's genetic burden results do not depend on it.
What the paper reports
Results, in-vivo paragraph (Marfan comparison, then Fbn2 strains):
There was no significant increase in CSF leak frequency in Marfan syndrome mice compared with wild-type mice (seven [54%] of 13 vs 20 [57%] of 35, respectively, p>0·05, relative risk [RR]=0·94, 95% CI 0·74–1·22, figure 4A). Contrastingly, these frequencies were significantly higher for Fbn2A1052T/+ mice (15 [79%] of 19), RR 1·38, 95% CI 1·14–1·69), for Fbn2D1581V/+ mice (22 [76%] of 29], RR 1·33, 95% CI 1·09–1·63), and for Fbn2M2387T/+ mice (eight [73%] of 11], RR 1·27, 95% CI 1·04–1·58), with all comparisons p<0·001 (figure 4A).
(The unmatched brackets are in the published text.)
Methods, Statistical analysis:
Proportions of mice with leaks were compared between mutant and wild-type mice by use of χ2 tests. p<0·05 was used as significance threshold.
Figure 4 legend:
(A) Proportion of CSF leak occurrence in mouse strains following intrathecal infusion.
*p<0·001. †p<0·0001.
The earlier check of the figure image (gr4.jpg from the publisher's supplementary package) found † on the A1052T and D1581V bars and * on the M2387T bar. The bar heights (about 57%, 54%, 79%, 76% and 73%) match the counts in the text.
Abstract, Findings:
Mice carrying variants equivalent to the three FBN2 variants in TB domains found in individuals with type 1b spontaneous spinal CSF leak (Fbn2A1052T/+, Fbn2D1581V/+, and Fbn2M2387T/+) showed a predisposition for dural rupture on controlled leak induction.
Research in context, Added value of this study:
Functional in-vitro studies and animal model data are supportive of a causal effect of the identified variants in FBN2.
Discussion, first paragraph:
We also showed increased CSF leak susceptibility and severity in fibrillin-2 mutant mice, and increased dural compliance in Marfan syndrome mice.
Discussion, mouse paragraph:
Our findings revealed that type 1b spontaneous spinal CSF leak-associated variants in the TB3, TB4, and TB7 domains of fibrillin-2 led to increased CSF leak frequency and severity in mice, a characteristic not attributable to increased pressure or volume at the time of leak.
Marfan syndrome mice also showed increased meningeal compliance without increased leak frequency or severity.
All quotes were checked verbatim against the PMC-archived text. Superscripts on the allele names (A1052T/+ etc.) are lost in plain text.
What the paper's own numbers show
Method. Each strain was compared with the wild-type group (20 of 35 mice with leaks) in a 2 × 2 table built from the counts printed in the Results. For each comparison I calculated the RR with its usual log-scale (Katz) 95% CI and the Pearson χ² test without continuity correction, which is the test named in the Methods. As checks I also ran the Yates-corrected χ², Fisher's exact test (two-sided and one-sided), Barnard's exact test, a pooled comparison of all Fbn2 mice, and a comparison against pooled controls (wild-type plus Marfan). To find where the reported intervals come from, I repeated the calculation with each printed percentage used as a count out of 100. All of this is in recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's counts |
|---|---|---|---|
| Leak frequencies | Results; Fig 4A | 79%, 76%, 73%, 54% vs 57% | 15/19 = 78.9%, 22/29 = 75.9%, 8/11 = 72.7%, 7/13 = 53.8%, 20/35 = 57.1% (consistent) |
| Fbn2 A1052T/+ vs WT, p | Results; Fig 4A (†) | p<0.001 (text); p<0.0001 (figure) | χ² p = 0.109 (Yates 0.192; Fisher 0.142; Barnard 0.123) |
| Fbn2 A1052T/+ RR (95% CI) | Results | 1.38 (1.14–1.69) | 1.38 (0.96–2.00) |
| Fbn2 D1581V/+ vs WT, p | Results; Fig 4A (†) | p<0.001 (text); p<0.0001 (figure) | χ² p = 0.117 (Yates 0.192; Fisher 0.186; Barnard 0.133) |
| Fbn2 D1581V/+ RR (95% CI) | Results | 1.33 (1.09–1.63) | 1.33 (0.93–1.89) |
| Fbn2 M2387T/+ vs WT, p | Results; Fig 4A (*) | p<0.001 | χ² p = 0.356 (Yates 0.569; Fisher 0.486; Barnard 0.449) |
| Fbn2 M2387T/+ RR (95% CI) | Results | 1.27 (1.04–1.58) | 1.27 (0.80–2.02) |
| Fbn1 C1039G/+ (Marfan) RR (95% CI), p | Results | 0.94 (0.74–1.22), p>0.05 | 0.94 (0.53–1.68), χ² p = 0.838. The conclusion (not significant) holds; the CI is too narrow |
| One-sided Fisher (mutant > WT) | check | n/a | 0.095, 0.095, 0.289 |
| SE(log RR): from the reported CI vs from the counts | check | 0.100, 0.103, 0.107, 0.128 | 0.188, 0.180, 0.236, 0.296 (1.75–2.32 times wider) |
| RR CI with percentages used as counts out of 100 | diagnostic | 1.14–1.69 / 1.09–1.63 / 1.04–1.58 / 0.74–1.22 | 79/100, 76/100, 73/100, 54/100 vs 57/100: 1.14–1.69 / 1.09–1.63 / 1.04–1.58 / 0.74–1.21 |
| χ² p with percentages used as counts out of 100 | diagnostic | p<0.001 (all three) | 0.0009 / 0.0044 / 0.018. Even this does not give p<0.001 for all three |
| All Fbn2 mice pooled (45/59) vs WT 20/35 | sensitivity | n/a | RR 1.33 (0.97–1.84); χ² p = 0.052; Fisher p = 0.066 |
| Each Fbn2 strain vs pooled controls (WT + Marfan, 27/48) | sensitivity | n/a | χ² p = 0.083, 0.083, 0.316 |
| Global 2 × 5 χ² across all five strains | sensitivity | n/a | χ² = 4.99, df = 4, p = 0.288 |
| Leaks needed for χ² p<0.001 vs WT 20/35 | sensitivity | observed 15/19, 22/29, 8/11 | ≥19/19, ≥28/29; not reachable with 11 mice (11/11 gives p = 0.008) |
Every printed percentage matches its count, so the counts are internally consistent. No standard test of these counts gives p<0.05 for any single strain, let alone p<0.001. For the M2387T group, p<0.001 cannot be reached against 20/35 even if all 11 mice had leaked.
Figure
Figure 1. Relative risk of CSF leak in each mutant strain compared with wild-type mice (20/35), on a log scale. Orange circles and bars are the RRs and 95% CIs printed in the Results. Blue squares and bars are the RRs and log-scale 95% CIs recomputed from the counts printed alongside them (shown under each strain name). The thin vertical line marks RR = 1. The point estimates agree. The recomputed intervals are 1.75 to 2.3 times wider on the log scale, and for all three Fbn2 strains they cross 1, whereas the reported intervals do not. The Marfan interval is also too narrow as printed, although both versions include 1.
Figure 2. p-values for each Fbn2 strain against wild-type, on a log scale. Orange triangles show the reported thresholds: filled for "p<0·001" in the Results text, open for the "p<0·0001" marker (†) in the Figure 4 legend. Each triangle marks an upper bound. Blue markers show the values recomputed from the paper's counts: filled squares for the Pearson χ² test named in the Methods, open diamonds for Fisher's exact test. The thin vertical line marks the paper's significance threshold, p = 0.05. The recomputed values lie two to three orders of magnitude above the reported bounds, and all lie above 0.05.
Why it matters
The in-vivo leak experiment is the paper's test of whether the FBN2 variants found in patients impair dural integrity in a living animal. The leak-frequency comparison is the only in-vivo measure the paper reports as showing a difference in leak occurrence between Fbn2 mice and wild-type mice. The abstract ("showed a predisposition for dural rupture on controlled leak induction"), the Research in context panel ("animal model data are supportive of a causal effect") and the Discussion ("led to increased CSF leak frequency") all rely on it. The contrast the Discussion draws with Marfan mice, which showed increased compliance "without increased leak frequency", also relies on it. Recomputed from the paper's own counts, none of the three strain-level differences is statistically significant at the paper's own threshold of 0.05. The pooled Fbn2 group is borderline (p = 0.052). The direction of effect is unchanged: every Fbn2 strain leaked more often than wild-type (73–79% vs 57%). With 11 to 29 mice per group, the data are compatible both with no difference and with a doubling of risk.
Several results are unaffected. The exome burden enrichment (p = 0.041, 0.004 and 0.0003 in the three control cohorts) rests on human data, and so do the domain-distribution and fibroblast-binding results. The separately reported larger pressure drop at leak onset in Fbn2 mice (Figure 4E–F) and the Marfan compliance result (Figure 4D) are different measurements; they could not be checked here because no per-mouse data are published. The genetic conclusion therefore still stands. What changes is that the leak-frequency data do not, by themselves, provide statistically significant in-vivo evidence for it.
Explanations considered
- A different test from the one named. The Methods name χ² tests. Pearson χ² (0.109, 0.117, 0.356), Yates-corrected χ², Fisher's exact (two-sided or one-sided), and Barnard's exact test all give p>0.09 for every strain. This does not account for the reported values.
- A different comparator or a global test. Comparing against pooled controls (wild-type plus Marfan) gives p = 0.083–0.316. A global 2 × 5 χ² gives p = 0.288. Pooling all Fbn2 mice gives p = 0.052. This does not account for the reported values.
- Different units (infusion trials rather than mice). The Figure 4A axis refers to trial outcome, but the Results report the counts as mice ("15 [79%] of 19"), the Methods describe the analysis as "Proportions of mice with leaks", and the bar heights match the mouse counts. The paper describes no repeated trials per mouse. If there were repeated trials per mouse, the mouse would remain the experimental unit, and treating trials as independent would understate the uncertainty rather than justify it. This does not account for the reported values on the evidence available.
- Percentages used as counts out of 100. Treating 79%, 76%, 73% and 54% as counts out of 100 per group (against 57/100) reproduces all four printed CIs to within 0.01. This is the only calculation found that does so. Even this calculation gives χ² p = 0.0009, 0.0044 and 0.018, so it does not reproduce "all comparisons p<0·001" or the p<0.0001 markers. The source of the p-values remains unidentified. The point estimates are unaffected, since an RR does not depend on the denominator.
- Correction for multiple comparisons. Any correction would make the p-values larger, not smaller.
- Rounding or typesetting. The discrepancies are a factor of 100 or more in p and roughly a factor of two in CI width. They appear in the text and in the figure markers, and they affect all three strains. A typesetting or rounding cause is not plausible.
- Text-conversion artifact in PMC. The same text appears in the publisher-supplied JATS in Europe PMC and on the live PMC page, and the figure markers come from the publisher's image. This does not account for the discrepancy.
- Data not in the paper. No per-mouse data or source data file is deposited, and the supplementary appendix has no contingency tables. If the printed counts were wrong, the recomputed values would change. However, the counts agree with the printed percentages and with the bar heights in Figure 4A, so they are what the paper presents as its data.
Status
As of 2026-10-23:
- Version of record. The publisher pages could not be opened from this machine: doi.org redirects to linkinghub.elsevier.com, which returned only a redirect stub, and thelancet.com and sciencedirect.com both returned HTTP 403. The text was checked in two archived copies, the publisher-supplied JATS full text served by Europe PMC and the PMC page (pmc.ncbi.nlm.nih.gov/articles/PMC13275636/). Both carry the Results passage, the Methods sentence and the Figure 4 legend quoted above.
- Supplements. The Europe PMC supplementary package was read in the earlier check. It holds mmc1.pdf (appendix: structural modelling, binding methods, co-IP blots, CRISPR sequences, uncropped blots) and the figure images gr1–gr4. It has no per-mouse leak data, no contingency tables and no further description of the leak-proportion test. It does not resolve the discrepancy. A re-download attempted for this write-up timed out.
- Corrections. None was found. Crossref update metadata (
src/known.py): none. PMC notices: none. Europe PMC (PMID 42309087) lists no comment or correction. The Crossref record has no update-to or relation entries. - PubPeer. The PubPeer search API returned a human-verification challenge, and the PubPeer search page returned HTTP 403, so PubPeer could not be queried. A web search could not be run for this write-up because the session's search budget was exhausted. No PubPeer discussion is known, but its absence is not confirmed.
Suggested correction
In the Results, in-vivo paragraph, the RR confidence intervals and p-values should be recomputed from the mouse-level counts. If the counts printed are correct, the passage would read approximately:
There was no significant increase in CSF leak frequency in Marfan syndrome mice compared with wild-type mice (seven [54%] of 13 vs 20 [57%] of 35, respectively, p=0·84, relative risk [RR]=0·94, 95% CI 0·53–1·68, figure 4A). Leak frequencies were numerically higher for Fbn2A1052T/+ mice (15 [79%] of 19; RR 1·38, 95% CI 0·96–2·00; p=0·11), for Fbn2D1581V/+ mice (22 [76%] of 29; RR 1·33, 95% CI 0·93–1·89; p=0·12), and for Fbn2M2387T/+ mice (eight [73%] of 11; RR 1·27, 95% CI 0·80–2·02; p=0·36), but none of these differences was statistically significant (χ² tests).
The significance markers on Figure 4A for the three Fbn2 strains would need to be removed or replaced with "NS". The Abstract ("showed a predisposition for dural rupture"), the Research in context panel and the two Discussion passages describing increased leak frequency should be revised to match. For example, they could state that leak frequency was higher in all three Fbn2 strains but not significantly so, and rest any in-vivo claim on the pressure-drop results, if those are confirmed. Depositing per-mouse leak data would let readers verify the corrected analysis.
Recomputation script (python recompute.py.txt)
"""Recompute the in-vivo CSF-leak frequency comparisons in PMC13275636
(Lancet Neurol 2026; doi:10.1016/S1474-4422(26)00140-7) from the paper's own printed
counts, and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy, scipy >= 1.7, matplotlib)
Inputs are copied from the published article (Results, in-vivo paragraph; Methods,
Statistical analysis; Figure 4 legend).
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (Results)
WT = (20, 35) # "20 [57%] of 35" wild-type mice with leaks
GROUPS = [
# label, (leaks, mice), reported RR, CI low, CI high, reported p, printed %
("Fbn2 A1052T/+", (15, 19), 1.38, 1.14, 1.69, "p<0.001 (Fig 4A: p<0.0001)", 79),
("Fbn2 D1581V/+", (22, 29), 1.33, 1.09, 1.63, "p<0.001 (Fig 4A: p<0.0001)", 76),
("Fbn2 M2387T/+", (8, 11), 1.27, 1.04, 1.58, "p<0.001 (Fig 4A: p<0.001)", 73),
("Fbn1 C1039G/+ (Marfan)", (7, 13), 0.94, 0.74, 1.22, "p>0.05", 54),
]
WT_PCT = 57
REPORTED_P_BOUND = {"Fbn2 A1052T/+": 1e-4, "Fbn2 D1581V/+": 1e-4, "Fbn2 M2387T/+": 1e-3}
def katz(a, n1, c, n0):
"""Relative risk with the usual log-scale (Katz) 95% CI."""
rr = (a / n1) / (c / n0)
se = np.sqrt(1 / a - 1 / n1 + 1 / c - 1 / n0)
return rr, rr * np.exp(-Z * se), rr * np.exp(Z * se), se
def wald_linear(a, n1, c, n0):
"""Delta-method CI on the untransformed RR scale (an alternative some software uses)."""
rr, _, _, se = katz(a, n1, c, n0)
return rr - Z * rr * se, rr + Z * rr * se
def table(a, n1, c, n0):
return np.array([[a, n1 - a], [c, n0 - c]])
def pvals(a, n1, c, n0):
t = table(a, n1, c, n0)
chi2_stat, p_chi, _, _ = stats.chi2_contingency(t, correction=False)
p_yates = stats.chi2_contingency(t, correction=True)[1]
p_fisher = stats.fisher_exact(t)[1]
p_fisher_1s = stats.fisher_exact(t, alternative="greater")[1]
p_barnard = stats.barnard_exact(t).pvalue
return dict(chi2=chi2_stat, chi=p_chi, yates=p_yates, fisher=p_fisher,
fisher1=p_fisher_1s, barnard=p_barnard)
print("=== 1. Percentages printed in the Results ===")
for lab, (a, n), *_rest in GROUPS:
print(f"{lab:24s} {a}/{n} = {100*a/n:.1f}% (printed {_rest[-1]}%)")
print(f"{'Wild-type':24s} {WT[0]}/{WT[1]} = {100*WT[0]/WT[1]:.1f}% (printed {WT_PCT}%)")
print("\n=== 2. Each strain vs wild-type 20/35, with the paper's counts ===")
RES = {}
for lab, (a, n1), rr_r, lo_r, hi_r, p_r, _ in GROUPS:
rr, lo, hi, se = katz(a, n1, *WT)
wl, wh = wald_linear(a, n1, *WT)
p = pvals(a, n1, *WT)
RES[lab] = dict(rr=rr, lo=lo, hi=hi, se=se, **p)
print(f"{lab}: {a}/{n1} vs {WT[0]}/{WT[1]}")
print(f" RR {rr:.3f}; log-scale 95% CI {lo:.2f}-{hi:.2f}; linear-scale 95% CI {wl:.2f}-{wh:.2f}"
f" | reported RR {rr_r} (95% CI {lo_r}-{hi_r})")
print(f" Pearson chi2 = {p['chi2']:.2f}, p = {p['chi']:.3f}; Yates p = {p['yates']:.3f}; "
f"Fisher p = {p['fisher']:.3f}; one-sided Fisher p = {p['fisher1']:.3f}; "
f"Barnard p = {p['barnard']:.3f} | reported {p_r}")
print("\n=== 3. Width of the reported CIs vs the width the counts imply ===")
for lab, (a, n1), rr_r, lo_r, hi_r, _, _ in GROUPS:
se_rep = (np.log(hi_r) - np.log(lo_r)) / (2 * Z)
se_true = RES[lab]["se"]
print(f"{lab:24s} SE(log RR) implied by reported CI = {se_rep:.3f}; from counts = {se_true:.3f}; "
f"ratio {se_true/se_rep:.2f}; implied effective sample size inflation ~{(se_true/se_rep)**2:.1f}x")
print("\n=== 4. Diagnostic: percentages used as if they were counts out of 100 per arm ===")
for lab, (a, n1), rr_r, lo_r, hi_r, _, pct in GROUPS:
rr, lo, hi, _ = katz(pct, 100, WT_PCT, 100)
p = pvals(pct, 100, WT_PCT, 100)
print(f"{lab:24s} {pct}/100 vs {WT_PCT}/100: RR {rr:.3f}, 95% CI {lo:.2f}-{hi:.2f} "
f"(reported {lo_r}-{hi_r}); chi2 p = {p['chi']:.4f}, Fisher p = {p['fisher']:.4f}")
if lab in REPORTED_P_BOUND:
g = stats.chisquare([pct, 100 - pct], f_exp=[WT_PCT, 100 - WT_PCT])
print(f"{'':24s} goodness-of-fit chi2 of {pct}% vs expected {WT_PCT}%, n=100: p = {g.pvalue:.1e}")
print("\n=== 5. Sensitivity: other comparisons that do not rescue p<0.001 ===")
a = sum(g[1][0] for g in GROUPS[:3]); n1 = sum(g[1][1] for g in GROUPS[:3])
rr, lo, hi, _ = katz(a, n1, *WT); p = pvals(a, n1, *WT)
print(f"All Fbn2 strains pooled {a}/{n1} vs WT 20/35: RR {rr:.2f} (95% CI {lo:.2f}-{hi:.2f}); "
f"chi2 p = {p['chi']:.3f}; Fisher p = {p['fisher']:.3f}")
ctrl = (WT[0] + GROUPS[3][1][0], WT[1] + GROUPS[3][1][1])
print(f"Controls pooled (WT + Marfan) = {ctrl[0]}/{ctrl[1]}")
for lab, (a, n1), *_ in GROUPS[:3]:
p = pvals(a, n1, *ctrl)
print(f" {lab} vs pooled controls: chi2 p = {p['chi']:.3f}, Fisher p = {p['fisher']:.3f}")
t = np.array([[g[1][0] for g in GROUPS] + [WT[0]],
[g[1][1] - g[1][0] for g in GROUPS] + [WT[1] - WT[0]]])
c5 = stats.chi2_contingency(t)
print(f"Global 2x5 chi2 across all five strains: chi2 = {c5[0]:.2f}, df = {c5[2]}, p = {c5[1]:.3f}")
print("\n=== 6. Leak count needed for Pearson chi2 p<0.001 vs WT 20/35 ===")
for lab, (a, n1), *_ in GROUPS[:3]:
need = [k for k in range(n1 + 1)
if k / n1 > WT[0] / WT[1] and pvals(k, n1, *WT)["chi"] < 0.001]
req = f"needs >= {need[0]}/{n1} leaks" if need else "impossible at any leak count"
print(f"{lab:24s} n = {n1}: {req} (observed {a}); "
f"if all {n1} leaked: p = {pvals(n1, n1, *WT)['chi']:.4f}")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.ticker import FixedLocator, FixedFormatter, NullLocator
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
def style(ax):
ax.set_facecolor(BG)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.xaxis.grid(True, color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
# ---- fig1: relative risk, reported vs recomputed 95% CI, log scale
labels = [f"{g[0]}\n{g[1][0]}/{g[1][1]} vs WT {WT[0]}/{WT[1]}" for g in GROUPS]
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
style(ax)
ys = np.arange(len(GROUPS))[::-1].astype(float)
off = 0.17
for y, (lab, (a, n1), rr_r, lo_r, hi_r, _, _) in zip(ys, GROUPS):
r = RES[lab]
ax.plot([lo_r, hi_r], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(rr_r, y + off, "o", color=ORANGE, ms=8, zorder=3)
ax.text(hi_r * 1.04, y + off, f"{lo_r:.2f}–{hi_r:.2f}", va="center", fontsize=10, color=ORANGE)
ax.plot([r["lo"], r["hi"]], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(r["rr"], y - off, "s", color=BLUE, ms=8, zorder=3)
ax.text(r["hi"] * 1.04, y - off, f"{r['lo']:.2f}–{r['hi']:.2f}", va="center", fontsize=10,
color=BLUE)
ax.axvline(1, color=SEC, lw=0.8)
ax.text(0.98, ys[0] + 0.55, "RR = 1 (no difference)", ha="right", fontsize=10, color=SEC)
ax.set_xscale("log")
ticks = [0.5, 0.75, 1, 1.5, 2, 3]
ax.xaxis.set_major_locator(FixedLocator(ticks))
ax.xaxis.set_major_formatter(FixedFormatter([str(t) for t in ticks]))
ax.xaxis.set_minor_locator(NullLocator())
ax.set_xlim(0.45, 3.0)
ax.set_ylim(-0.6, len(GROUPS) - 0.2)
ax.set_yticks(ys); ax.set_yticklabels(labels, fontsize=10)
ax.set_xlabel("Relative risk of CSF leak vs wild-type (log scale), with 95% CI")
ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2, label="reported in Results")
ax.plot([], [], "s-", color=BLUE, ms=8, lw=2, label="recomputed from the paper's counts")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.03), ncol=2, fontsize=10, frameon=False)
save(fig, "fig1")
# ---- fig2: p-values, reported bound vs recomputed, log scale
fig, ax = plt.subplots(figsize=(7.2, 3.5), facecolor=BG)
style(ax)
fb = GROUPS[:3]
ys2 = np.arange(len(fb))[::-1].astype(float)
for y, (lab, (a, n1), *_r) in zip(ys2, fb):
r = RES[lab]
b = REPORTED_P_BOUND[lab]
ax.plot(1e-3, y + off, "<", color=ORANGE, ms=9, zorder=3)
if b < 1e-3:
ax.plot(b, y + off, "<", color=ORANGE, ms=9, mfc=BG, mew=2, zorder=3)
ax.plot(r["chi"], y - off, "s", color=BLUE, ms=8, zorder=3)
ax.plot(r["fisher"], y - off, "D", color=BLUE, ms=8, mfc=BG, mew=2, zorder=3)
ax.plot([r["chi"], r["fisher"]], [y - off] * 2, color=BLUE, lw=2, zorder=2)
ax.text(max(r["chi"], r["fisher"]) * 1.25, y - off, f"{r['chi']:.2f} / {r['fisher']:.2f}",
va="center", fontsize=10, color=BLUE)
ax.axvline(0.05, color=SEC, lw=0.8)
ax.text(0.045, ys2[0] + 0.5, "p = 0.05", ha="right", fontsize=10, color=SEC)
ax.set_xscale("log")
pt = [1e-5, 1e-4, 1e-3, 1e-2, 0.05, 1]
ax.xaxis.set_major_locator(FixedLocator(pt))
ax.xaxis.set_major_formatter(FixedFormatter(["0.00001", "0.0001", "0.001", "0.01", "0.05", "1"]))
ax.xaxis.set_minor_locator(NullLocator())
ax.set_xlim(5e-6, 1.6)
ax.set_ylim(-0.55, len(fb) - 0.25)
ax.set_yticks(ys2)
ax.set_yticklabels([f"{g[0]}\n{g[1][0]}/{g[1][1]} vs WT {WT[0]}/{WT[1]}" for g in fb], fontsize=10)
ax.set_xlabel("p-value, mutant vs wild-type leak frequency (log scale)")
ax.plot([], [], "<", color=ORANGE, ms=9, label="reported: p<0.001 (Results text)")
ax.plot([], [], "<", color=ORANGE, ms=9, mfc=BG, mew=2, label="reported: p<0.0001 (Fig 4A legend)")
ax.plot([], [], "s", color=BLUE, ms=8, label="recomputed: Pearson χ² (named in Methods)")
ax.plot([], [], "D", color=BLUE, ms=8, mfc=BG, mew=2, label="recomputed: Fisher's exact")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.02), ncol=2, fontsize=10, frameon=False)
save(fig, "fig2")
print("\nfigures written to", HERE)
Scenario 5 diabetes total printed as 1528, while its own Table 6 cells sum to 12 108
Lancet Public Health (2016) · doi:10.1016/S2468-2667(16)30037-8 · PMC5543265
Summary
This modelling study estimates the health effects of six possible industry responses to the UK soft drinks industry levy. For scenario 5, the better case for a shift in market share towards low-sugar drinks, the Abstract and the Total row of Table 6 report 1528 fewer incident cases of type 2 diabetes per year, with a 95% uncertainty interval (UI) of 4414 to 21 785 and a rate of 19·7 per 100 000 person-years. The point estimate lies below the lower bound of its own interval. The eight age-sex cells of the same column sum to 12 108, and the printed rate of 19·7 implies about 12 100 cases on the population denominator that the other scenarios share. 1528 is also exactly the value in the Women aged ≥65 years cell of that column. The headline figure for this scenario is therefore about 8 times too small, and read literally it reverses the ranking of scenarios for diabetes that the Results text describes. The paper's main conclusions are unaffected.
What the paper reports
Abstract, Findings:
Changes to market share to increase the proportion of low-sugar drinks sold in the better-case scenario would result in 91 042 (4289–204 903; 0·6%) fewer adults and children with diabetes, 1528 (4414–21 785; 19·7) fewer incident cases of diabetes per year, and 172 718 (47 919–294 499; 2·8) fewer decayed, missing, or filled teeth annually.
The same paragraph, for the price scenario (scenario 3):
An increase in the price of SSBs in the better-case scenario would result in 81 594 (3588–182 669; 0·5%) fewer adults and children with obesity, 10 861 (3899–18 964; 17·7) fewer incident cases of diabetes per year, and 149 378 (45 231–262 013; 2·4) fewer decayed, missing, or filled teeth annually.
Table 6 ("Reduction in the number of cases of diabetes per year with each scenario"), scenario 5 column, row "Women aged ≥65 years", and the Total row (all six scenarios):
Women aged ≥65 years | 2429 (38·8) | 448 (7·2) | 1369 (21·8) | 495 (7·9) | 1528 (24·4) | -1549 (-2·5)
Total; 95% UI | 19 094 (31·1); 6920 to 32 678 | 3584 (5·8); 1289 to 6466 | 10 861 (17·7); 3899 to 18 964 | 3955 (6·4); 1420 to 7085 | 1528 (19·7); 4414 to 21 785 | -1238 (-2·0); -2359 to -455
Table 6 footer:
Data in parentheses are reductions in incidence per 100 000 person-years. UI=uncertainty interval.
Results, diabetes paragraph:
Across the scenarios modelled, the pattern of results seen with obesity is repeated for type 2 diabetes.
Results, dental caries paragraph:
As with results for obesity and diabetes, the better case for change in market share (scenario 5) and price change (scenario 3) scenarios had the next largest effects respectively.
Methods, "Health impact modelling":
We applied all results to the 2014 UK population29 and made separate estimates for each outcome by sex and age group using age-specific and sex-specific estimates of baseline SSB consumption and disease burden.
What the paper's own numbers show
Method. Table 6 gives, for each scenario, eight age-sex cells and a Total row, each as a count with a rate per 100 000 person-years in parentheses. Because the model is applied to the whole UK population split by sex and age group, the Total should equal the sum of the eight cells (up to rounding), and the Total count divided by the Total rate should give the same population denominator in every scenario. I checked both identities for all six scenarios, using scenarios 1 to 4 as controls for the method, and compared scenario 5 with scenario 3 cell by cell. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Scenario 5 total, fewer diabetes cases per year | Abstract Findings; Table 6 Total row | 1528 (19·7); 95% UI 4414 to 21 785 | 12 108 = 45 + 145 + 5327 + 1598 + 33 + 143 + 3289 + 1528 (sum of the 8 cells) |
| Scenario 5 total, from the printed rate | Table 6 Total row | 19·7 per 100 000 | 19·7 × 61.40 M person-years = 12 095 (rounding range 12 045 to 12 145) |
| Scenario 5 rate implied by each count | derived | 19·7 | 12 108 gives 19·72; 1528 gives 2·49 |
| Denominator needed for 1528 and 19·7 to agree | derived | — | 7.76 M person-years, against 61.4 M in scenarios 1 and 3 |
| Point estimate inside its own UI? | Table 6 Total row | 1528 vs 4414 to 21 785 | No (1528 < 4414); 12 108 is inside |
| Identity with another cell | Table 6, scenario 5 | Total 1528; Women ≥65 cell 1528 (24·4) | Identical values; the cell's own rate is consistent (24·4 × 6.26 M = 1528) |
| Size of the discrepancy | derived | 1528 | 12 108 / 1528 = 7.92 |
| Scenario 1 total (control) | Table 6 | 19 094 (31·1) | cell sum 19 094; denominator 61.40 M |
| Scenario 2 total (control) | Table 6 | 3584 (5·8) | cell sum 3584 |
| Scenario 3 total (control) | Table 6 | 10 861 (17·7) | cell sum 10 861; denominator 61.36 M |
| Scenario 4 total (control) | Table 6 | 3955 (6·4) | cell sum 3956 (rounding) |
| Scenario 5 ÷ scenario 3, per cell | Table 6 | — | 1.107 to 1.138 in all eight cells |
| Scenario 5 ÷ scenario 3, totals | Table 6 | 1528 / 10 861 = 0.141 | 12 108 / 10 861 = 1.115; rates 19·7 / 17·7 = 1.113; UI bounds 1.132 and 1.149 |
| Diabetes ranking of scenarios | Table 6 vs Table 5 (obesity) | printed totals: S1 > S3 > S4 > S2 > S5 > S6 | with 12 108: S1 > S5 > S3 > S4 > S2 > S6, identical to the obesity ranking |
| Secondary: Women ≥65, scenario 6 | Table 6 row "Women aged ≥65 years" | −1549 (−2·5) | −2·5 × 6.26 M = −157 (range −153 to −160); total −1238 minus the other seven cells (−1083) = −155 |
| Secondary: scenario 6 column sum | Table 6 | Total −1238 (−2·0) | cells sum to −2632, outside the UI −2359 to −455; the printed total and rate agree with each other (−1238 ÷ 61.40 M = −2·02) |
Three independent routes (the cell sum, the printed rate times the shared denominator, and the near-constant scenario 5 to scenario 3 ratio) all place the scenario 5 total at about 12 100. Only the printed count disagrees, and it is the one value that duplicates another cell. The same two identities reproduce the Total row of scenarios 1 to 4 exactly or within rounding, so the method does not explain the discrepancy.
Figure
Figure 1. For each of the six scenarios, the orange dot and bar are the total reduction in incident diabetes cases per year and its 95% UI, as printed in the Total row of Table 6. The blue squares are the sum of the eight age-sex cells in the same column. The open blue diamonds are the printed total rate multiplied by the 61.4 million person-years implied by scenario 1. In scenarios 1 to 4 all three values coincide. In scenario 5 both recomputations give about 12 100, inside the UI, while the printed total of 1528 lies below the lower bound of 4414. In scenario 6 the printed total and its rate agree, but the cells sum to −2632, because the Women ≥65 cell (−1549) is about ten times what its own rate implies; this is a separate, smaller inconsistency.
Figure 2. In every age-sex cell of Table 6, scenario 5 averts between 1.107 and 1.138 times as many diabetes cases as scenario 3 (gray dots). The recomputed totals keep that ratio (12 108 / 10 861 = 1.115, blue). The printed totals give 0.141 (1528 / 10 861, orange), a ratio that no cell comes close to. The vertical line marks a ratio of 1, where scenario 5 would equal scenario 3.
Why it matters
The scenario 5 diabetes figure is one of nine headline estimates in the Abstract's Findings (three outcomes for each of three better-case scenarios), and outside Table 6 it is the only place in the paper where the diabetes benefit of a market-share response is stated. As printed it is about 8 times too small. A reader comparing the Abstract's three better-case scenarios would conclude that a shift in market share prevents far fewer diabetes cases than a price rise (1528 vs 10 861), and in Table 6 it would rank fifth of six, below both worse-case reformulation and worse-case price scenarios. The paper's own cells give the opposite order: with about 12 100, scenario 5 is second only to reformulation for diabetes, exactly as for obesity in Table 5. This is what the Results state ("the pattern of results seen with obesity is repeated for type 2 diabetes"; "As with results for obesity and diabetes, the better case for change in market share (scenario 5) and price change (scenario 3) scenarios had the next largest effects respectively"). The printed 1528 contradicts both sentences.
Anyone citing or re-using the Abstract figure in a policy comparison of industry responses, or in a later model that takes this study's scenario results as inputs, would understate the diabetes benefit of increasing the market share of low-sugar drinks by about 10 500 cases per year.
The paper's main conclusions do not depend on this number. Reformulation remains the scenario with the largest benefit, the health impact still depends on how industry responds, and the Interpretation's statement that raising the market share of low-sugar products brings additional benefits is strengthened, not weakened, by the corrected value. The uncertainty interval as printed (4414 to 21 785) appears to belong to the corrected total, so no re-estimation is needed; only the point estimate needs replacing.
Two smaller inconsistencies sit next to this one. In the same Abstract sentence, 91 042 is described as "fewer adults and children with diabetes"; Table 5 and the Results give 91 042 as the reduction in the number of people with obesity. In Table 6, the Women ≥65 cell for scenario 6 reads −1549 (−2·5), but −2·5 per 100 000 on that group's 6.26 million person-years is about −157, and the printed scenario 6 total (−1238) leaves −155 for this cell. Neither changes any conclusion.
Explanations considered
- A different unit or definition for the Total. This does not account for the numbers. The Table 6 footer defines one unit for the whole table (cases per year, with reductions per 100 000 person-years in parentheses), and the totals of scenarios 1 to 4 are plain sums of their cells.
- The Total is for a subpopulation rather than the whole population. This does not account for the numbers. A total of 1528 with a rate of 19·7 would need a denominator of 7.76 million person-years, while scenarios 1 and 3 imply 61.4 million and the Methods say all results were applied to the 2014 UK population. The UI (4414 to 21 785) is also on the whole-population scale.
- A skewed Monte Carlo distribution, so the point estimate can sit outside the interval. This does not account for the numbers. A point estimate can sit off-centre in a simulation interval, but it cannot be below the 2.5th percentile by a factor of almost 3 while also disagreeing with its own cells and its own rate. In every other scenario the printed total is inside its UI, and the scenario 5 UI bounds are 1.13 and 1.15 times those of scenario 3, close to the 1.11 to 1.14 of the cells.
- Rounding. This does not account for the numbers. Rounding of the cells moves the sum by a few cases at most (scenario 4 differs by 1); rounding of the rate gives a range of 12 045 to 12 145.
- A conversion or typesetting error in the PMC copy. This is excluded. The publisher's typeset PDF (deposited as the Published Version in the University of Reading repository) and the Europe PMC full-text XML carry the same values in the Abstract and in Table 6.
- The supplement gives a different total. This does not apply. The appendix has no diabetes results by scenario (see Status), so it neither confirms nor contradicts the printed value.
- 1528 is correct and the cells are wrong. This does not account for the numbers. Seven of the eight cells, the total rate, and the UI would all have to be wrong together, and the cells keep the same ratio to scenario 3 in every age-sex group. The one value that differs is the total, and it equals another cell in the same column.
Status
- Version of record. The PMC-archived text was compared with two other copies: the publisher's typeset PDF (10 pages, CC-BY, deposited as the "Published Version" at centaur.reading.ac.uk/68564) and the Europe PMC full-text XML for PMC5543265. All three carry 1528 (4414–21 785; 19·7) in the Abstract and 1528 (19·7); 4414 to 21 785 in the Table 6 Total row, and the same cell values. The doi.org link, thelancet.com and ScienceDirect refused automated access (redirect stub or HTTP 403). Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. Crossref dates the article to the January 2017 issue (volume 2, issue 1, e15–e22). - Supplements. The appendix (mmc1.pdf, 8 pages) describes the modelling and gives supplementary tables 1–3 (price elasticity matrix, baseline SSB consumption, an energy-balance sensitivity analysis for obesity). It has no diabetes results by scenario, so it does not resolve the intended scenario 5 total. Table 6 itself pins it at about 12 100.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update-to or updated-by relation for the DOI, and a Crossref search for updates to the DOI returned none. PMC and Europe PMC list no correction notices. Crossref links one associated item in the same issue ("The impact of sugared drink taxation and industry response", doi:10.1016/S2468-2667(16)30039-1), an accompanying article, not a correction. A web search for corrections or comments found none. PubPeer refused automated access (HTTP 403 on search, 404 on the API), so it could not be checked directly.
- Sources. PMC article PMC5543265; Europe PMC full-text XML and supplementary files for PMC5543265; publisher PDF at centaur.reading.ac.uk/68564; Crossref records for 10.1016/S2468-2667(16)30037-8 and 10.1016/S2468-2667(16)30039-1.
Suggested correction
- Replace the scenario 5 total in the Table 6 Total row with the model's total for that scenario (the cells give 12 108; the printed rate 19·7 implies 12 045 to 12 145), keeping the rate 19·7 and the UI 4414 to 21 785 if these were produced for that total.
- Make the same change in the Abstract Findings: "about 12 100 (4414–21 785; 19·7) fewer incident cases of diabetes per year" in place of "1528 (4414–21 785; 19·7)".
- In the same Abstract sentence, change "91 042 (4289–204 903; 0·6%) fewer adults and children with diabetes" to "with obesity", to match Table 5 and the Results.
- In Table 6, check the scenario 6 cell for women aged ≥65 years: −1549 (−2·5) is inconsistent with its own rate and with the scenario 6 total; the table's other values imply about −155.
Recomputation script (python recompute.py.txt)
"""Recompute the Table 6 diabetes totals of PMC5543265
(Lancet Public Health 2017;2:e15-e22, doi:10.1016/S2468-2667(16)30037-8,
health impact assessment of the UK soft drinks industry levy).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from Table 6 ("Reduction in the number of cases of diabetes per year
with each scenario") as archived in PMC. Each cell is "count (reduction in incidence per
100 000 person-years)"; the Total row also gives the 95% uncertainty interval (UI).
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Table 6)
ROWS = ["Boys 4-10", "Boys 11-18", "Men 19-64", "Men >=65",
"Girls 4-10", "Girls 11-18", "Women 19-64", "Women >=65"]
CNT = np.array([
[71, 13, 40, 15, 45, -5],
[224, 44, 131, 49, 145, -15],
[8364, 1585, 4783, 1749, 5327, -549],
[2539, 469, 1431, 517, 1598, -160],
[52, 10, 29, 11, 33, -3],
[223, 43, 128, 47, 143, -15],
[5192, 972, 2950, 1073, 3289, -336],
[2429, 448, 1369, 495, 1528, -1549]])
RATE = np.array([
[2.6, 0.5, 1.5, 0.5, 1.6, -0.2],
[7.5, 1.5, 4.3, 1.6, 4.8, -0.5],
[43.5, 8.2, 24.9, 9.1, 27.6, -2.8],
[49.4, 9.1, 27.8, 10.1, 31.1, -3.1],
[2.0, 0.4, 1.1, 0.4, 1.2, -0.1],
[7.8, 1.5, 4.5, 1.7, 4.9, -0.5],
[26.7, 5.0, 15.1, 5.5, 16.9, -1.7],
[38.8, 7.2, 21.8, 7.9, 24.4, -2.5]])
TOT = np.array([19094, 3584, 10861, 3955, 1528, -1238])
TOTRATE = np.array([31.1, 5.8, 17.7, 6.4, 19.7, -2.0])
LO = np.array([6920, 1289, 3899, 1420, 4414, -2359])
HI = np.array([32678, 6466, 18964, 7085, 21785, -455])
# Table 5 (obesity) totals, used only for the ranking comparison
OBES = np.array([144383, 26774, 81594, 29555, 91042, -9211])
print("=== 1. Column sums of the eight age-sex cells vs the printed Total row ===")
SUM = CNT.sum(axis=0)
for s in range(6):
print(f" Scenario {s+1}: subgroup sum = {SUM[s]:6d} printed total = {TOT[s]:6d} "
f"UI {LO[s]} to {HI[s]} printed total inside UI: {LO[s] <= TOT[s] <= HI[s]} "
f"subgroup sum inside UI: {LO[s] <= SUM[s] <= HI[s]}")
print(" Scenario 5 sum written out: " + " + ".join(str(v) for v in CNT[:, 4]) + f" = {SUM[4]}")
print("\n=== 2. Population denominator implied by the Total row (count / rate) ===")
N = TOT / TOTRATE * 1e5
for s in range(6):
print(f" Scenario {s+1}: {TOT[s]} / {TOTRATE[s]} per 100 000 -> {N[s]/1e6:6.2f} million person-years")
# rounding interval of the denominator from scenarios 1 and 3 (rates printed to 1 decimal)
n1 = (TOT[0] / (TOTRATE[0] + 0.05) * 1e5, TOT[0] / (TOTRATE[0] - 0.05) * 1e5)
n3 = (TOT[2] / (TOTRATE[2] + 0.05) * 1e5, TOT[2] / (TOTRATE[2] - 0.05) * 1e5)
NLO, NHI = max(n1[0], n3[0]), min(n1[1], n3[1])
N0 = N[0]
print(f" rounding range from scenario 1: {n1[0]/1e6:.3f}-{n1[1]/1e6:.3f} M; scenario 3: "
f"{n3[0]/1e6:.3f}-{n3[1]/1e6:.3f} M; both: {NLO/1e6:.3f}-{NHI/1e6:.3f} M")
PY = CNT[:, 0] / RATE[:, 0] * 1e5
print(" person-years implied by each scenario 1 cell (M):", np.round(PY / 1e6, 2),
f" sum = {PY.sum()/1e6:.2f} M")
print("\n=== 3. Counts implied by the printed total rates x 61.4 M person-years ===")
for s in range(6):
print(f" Scenario {s+1}: {TOTRATE[s]} x {N0/1e6:.2f} M / 1e5 = {TOTRATE[s]*N0/1e5:7.0f} printed {TOT[s]}")
R5 = TOTRATE[4] * N0 / 1e5
R5LO, R5HI = (TOTRATE[4] - 0.05) * NLO / 1e5, (TOTRATE[4] + 0.05) * NHI / 1e5
print(f" Scenario 5: rate 19.7 (19.65-19.75) on {NLO/1e6:.3f}-{NHI/1e6:.3f} M -> {R5LO:.0f} to {R5HI:.0f}")
print(f" Scenario 5 subgroup sum as a rate: {SUM[4]/N0*1e5:.2f} per 100 000; printed 1528 as a rate: "
f"{TOT[4]/N0*1e5:.2f}")
print(f" denominator needed for 1528 and 19.7 to agree: {TOT[4]/TOTRATE[4]*1e5/1e6:.2f} M person-years")
print(f" scaling scenario 1 by rates: 19094 x 19.7/31.1 = {TOT[0]*TOTRATE[4]/TOTRATE[0]:.0f}; "
f"scaling scenario 3: 10861 x 19.7/17.7 = {TOT[2]*TOTRATE[4]/TOTRATE[2]:.0f}")
print(f" understatement factor: {SUM[4]} / {TOT[4]} = {SUM[4]/TOT[4]:.2f}")
print("\n=== 4. Scenario 5 relative to scenario 3, cell by cell ===")
RAT = CNT[:, 4] / CNT[:, 2]
for r, v in zip(ROWS, RAT):
print(f" {r:12s} {v:.3f}")
print(f" range {RAT.min():.3f}-{RAT.max():.3f}; printed totals 1528/10861 = {TOT[4]/TOT[2]:.3f}; "
f"subgroup sum 12108/10861 = {SUM[4]/TOT[2]:.3f}")
print(f" rates 19.7/17.7 = {TOTRATE[4]/TOTRATE[2]:.3f}; UI lower 4414/3899 = {LO[4]/LO[2]:.3f}; "
f"UI upper 21785/18964 = {HI[4]/HI[2]:.3f}")
print(f" 1528 equals the Women >=65 cell of scenario 5: {TOT[4] == CNT[7, 4]}")
print("\n=== 5. Ranking of scenarios ===")
def rank(v):
return " > ".join(f"S{i+1}" for i in np.argsort(-v))
print(" obesity (Table 5 totals): ", rank(OBES))
print(" diabetes, printed totals: ", rank(TOT.astype(float)))
fix = TOT.astype(float).copy(); fix[4] = SUM[4]
print(" diabetes, scenario 5 = 12 108: ", rank(fix))
print("\n=== 6. Secondary: Women >=65, scenario 6 ===")
w6_rate = RATE[7, 5] * PY[7] / 1e5
w6_rlo, w6_rhi = (RATE[7, 5] + 0.05) * PY[7] / 1e5, (RATE[7, 5] - 0.05) * PY[7] / 1e5
w6_resid = TOT[5] - CNT[:7, 5].sum()
print(f" printed -1549 (-2.5); person-years of this group (from scenario 1 cell) = {PY[7]/1e6:.2f} M")
print(f" -2.5 x {PY[7]/1e6:.2f} M / 1e5 = {w6_rate:.0f} (rounding range {w6_rlo:.0f} to {w6_rhi:.0f})")
print(f" printed total -1238 minus the other seven cells ({CNT[:7,5].sum()}) = {w6_resid}")
print(f" -1549 as a rate on {PY[7]/1e6:.2f} M: {CNT[7,5]/PY[7]*1e5:.1f} per 100 000 (printed -2.5)")
print(" scenario 6 / scenario 5 cell ratio:", np.round(CNT[:, 5] / CNT[:, 4], 3))
print(f" printed total -1238 as rate on {N0/1e6:.2f} M: {TOT[5]/N0*1e5:.2f} (printed -2.0); "
f"column sum {SUM[5]} as rate: {SUM[5]/N0*1e5:.2f}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"axes.spines.top": False, "axes.spines.right": False, "lines.linewidth": 2,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
def fmt(v):
return f"{v:,.0f}".replace(",", " ").replace("-", "−")
# Figure 1: per scenario, printed total with UI (orange) vs subgroup sum and rate-implied count (blue)
LAB = ["S1 reformulation, better", "S2 reformulation, worse", "S3 price, better",
"S4 price, worse", "S5 market share, better", "S6 market share, worse"]
fig, ax = plt.subplots(figsize=(7.2, 5.4))
ys = np.arange(6)[::-1]
RIMP = TOTRATE * N0 / 1e5
for s, y in zip(range(6), ys):
ax.plot([LO[s], HI[s]], [y, y], color=ORANGE, lw=2, alpha=0.55, solid_capstyle="butt", zorder=1)
ax.plot([LO[s]] * 2, [y - 0.12, y + 0.12], color=ORANGE, lw=2, alpha=0.55, zorder=1)
ax.plot([HI[s]] * 2, [y - 0.12, y + 0.12], color=ORANGE, lw=2, alpha=0.55, zorder=1)
ax.plot(TOT[s], y, "o", ms=10, color=ORANGE, zorder=3)
ax.plot(SUM[s], y + 0.24, "s", ms=8, color=BLUE, zorder=4)
ax.plot(RIMP[s], y - 0.24, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
ax.axvline(0, color=SEC, lw=0.8, zorder=0)
ax.set_yticks(ys)
ax.set_yticklabels(LAB)
ax.set_ylim(-0.7, 5.7)
ax.set_xlim(-4000, 34000)
ax.set_xticks([0, 5000, 10000, 15000, 20000, 25000, 30000])
ax.set_xticklabels([fmt(v) for v in ax.get_xticks()])
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.set_xlabel("Fewer incident cases of type 2 diabetes per year (Table 6)")
y5 = ys[4]
ax.text(TOT[4] + 700, y5 - 0.36, f"printed {fmt(TOT[4])} < UI lower bound {fmt(LO[4])}",
fontsize=10, color=TXT, ha="left", va="top")
ax.annotate(f"cells sum to {fmt(SUM[4])}", (SUM[4], y5 + 0.24), xytext=(HI[4] + 1200, y5 + 0.3),
fontsize=10, color=TXT, ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
ax.annotate(f"19.7 × 61.4 M\n= {fmt(RIMP[4])}", (RIMP[4], y5 - 0.24), xytext=(HI[4] + 1200, y5 - 0.25),
fontsize=10, color=TXT, ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
y6 = ys[5]
ax.annotate(f"cells sum to {fmt(SUM[5])}:\nWomen ≥65 cell (−1549)\nis the outlier",
(SUM[5], y6 + 0.24), xytext=(4200, y6 - 0.12), fontsize=10, color=TXT, ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
handles = [
Line2D([], [], color=ORANGE, marker="o", ms=9, lw=2, alpha=0.9, label="Reported total and 95% UI"),
Line2D([], [], color=BLUE, marker="s", ms=8, lw=0, label="Recomputed: sum of the 8 age-sex cells"),
Line2D([], [], color=BLUE, marker="D", ms=8, mfc=BG, mew=2, lw=0,
label="Recomputed: printed total rate × 61.4 M person-years"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.36, -0.15), ncol=1, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: scenario 5 / scenario 3 count ratio, cell by cell and for the total
fig, ax = plt.subplots(figsize=(7.2, 4.2))
names = ["Boys 4–10", "Boys 11–18", "Men 19–64", "Men ≥65",
"Girls 4–10", "Girls 11–18", "Women 19–64", "Women ≥65"]
ylab = names + ["Total"]
yy = np.arange(len(ylab))[::-1]
for v, y in zip(RAT, yy[:8]):
ax.plot(v, y, "o", ms=8, color=SEC, zorder=3)
ax.text(v + 0.03, y, f"{v:.3f}", va="center", ha="left", fontsize=10, color=SEC)
yt = yy[8]
ax.plot(TOT[4] / TOT[2], yt, "o", ms=10, color=ORANGE, zorder=3)
ax.text(TOT[4] / TOT[2] + 0.03, yt, f"{TOT[4]/TOT[2]:.3f} (1528 / 10 861, printed totals)", va="center",
ha="left", fontsize=10, color=ORANGE)
ax.plot(SUM[4] / TOT[2], yt, "s", ms=8, color=BLUE, zorder=3)
ax.text(SUM[4] / TOT[2] + 0.03, yt, f"{SUM[4]/TOT[2]:.3f} (12 108 / 10 861)", va="center",
ha="left", fontsize=10, color=BLUE)
ax.axhline(yt + 0.5, color=GRID, lw=1)
ax.axvline(1, color=SEC, lw=0.8, zorder=0)
ax.text(1.0, yy[0] + 0.55, "equal to scenario 3", ha="center", va="bottom", fontsize=10, color=SEC)
ax.set_yticks(yy)
ax.set_yticklabels(ylab)
ax.set_xlim(0, 1.75)
ax.set_ylim(-0.7, len(ylab) + 0.2)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.set_xlabel("Scenario 5 count ÷ scenario 3 count (Table 6)")
handles = [
Line2D([], [], color=SEC, marker="o", ms=8, lw=0, label="Age-sex cell (ratio of reported cells)"),
Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0, label="Total, as reported"),
Line2D([], [], color=BLUE, marker="s", ms=8, lw=0, label="Total, recomputed from the cells"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.4, -0.15), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract gives HIV-and-drug-use synergy the RERI that Table 2 reports for another pair
Lancet Reg Health Am (2025) · doi:10.1016/j.lana.2025.101305 · PMC12702379
Summary
The abstract's only interaction estimate says the strongest additive synergy was between HIV infection and drug use, "with a RERI of 225% [174–304%]", which is 2.25 [1.74–3.04] on the scale the tables use. Table 2 (composite unfavourable outcome) and the Results text both give the HIV & drug use RERI as 1.72 [1.28–2.28], and the row's own stratum-specific odds ratios reproduce 1.72 to 1.74. The value 2.25 [1.74–3.04], estimate and both confidence limits, is printed in Table 2 for a different pair, "Not PPL & retreatment", and that row's odds ratios reproduce it as well. So the abstract overstates the HIV and drug use estimate by about 31%, and its claim that this pair is "the strongest" does not hold for the composite outcome, where Not PPL & retreatment is larger. The corrected HIV and drug use interaction is still positive and its CI excludes 0, so the direction of the finding is unchanged.
What the paper reports
Abstract, Findings:
Interaction analyses revealed both synergistic and antagonistic effects, with the strongest additive synergy observed between HIV infection and drug use, with a RERI of 225% [174–304%].
Results, "Interactions among vulnerabilities impacting ATT outcomes", first paragraph (composite unfavourable outcome):
Particularly noteworthy was the synergistic interaction between HIV infection & drug use, which substantially elevated the risk beyond their independent effects on the additive scale (RERI = 1.72 [95% CI: 1.28–2.28]), however the ME indicated an antagonist effect (0.85 [0.81–0.89]).
Table 2, "Interaction effects of overlapping vulnerabilities on composite unfavorable ATT outcomes" (columns: RERI | AP | SI | multiplicative scale; the empty columns of each row are omitted):
HIV & drug use | 1.72 [1.28–2.28] | 0.28 [0.23–0.32] | 1.5 [1.4–1.62] | 0.85 [0.81–0.89]
Effect of HIV among non-drug users (A+ | B-) | 2.99 [2.71–3.3]
Effect of drug use among non–HIV (B+ | A-) | 2.42 [2.22–2.63]
Effect of HIV among drug users (A+ | B+) | 2.54 [2.27–2.83]
Effect of drug use among HIV (B+ | A+) | 2.05 [1.86–2.25]Not PPL & retreatment | 2.25 [1.74–3.04] | 0.49 [0.43–0.51] | 2.64 [2.48–2.81] | 1.63 [1.55–1.71]
Effect of not PPL among new cases (A+ | B-) | 1.78 [1.73–1.83]
Effect of retreatment among PPL (B+ | A-) | 1.59 [1.28–1.98]
Effect of not PPL among retreatment (A+ | B+) | 2.9 [2.78–3.02]
Effect of retreatment among not PPL (B+ | A+) | 2.6 [2.1–3.21]
("PPL" is the tables' label for people deprived of their liberty, so "Not PPL" means not incarcerated.)
Results, third paragraph (death, Table 3):
The interaction between PWH & drug use markedly increased the risk of death (RERI = 2.76 [95% CI: 1.92–3.86], no significant multiplicative interaction), consistent with the pattern observed in overall unfavorable outcomes.
Results, fifth paragraph (loss to follow-up, Table 4):
Finally, when focusing on LTFU cases, we found significant positive interactions including retreatment & not being incarcerated (RERI = 3.12 [95% CI: 2.41–4.25]; OR_INT = 1.94 [1.85–2.05], synergistic on both scales), underscoring that previously treated patients outside institutional settings face substantially increased risk of LTFU. Additionally, the interaction between living with HIV & drug use remained strongly synergistic (RERI = 1.98 [95% CI: 1.39–2.76], no significant multiplicative interaction), consistent with findings on unfavorable outcomes and mortality.
Methods, "Interaction analysis":
RERI was computed based on adjusted ORs from the GLM using the following formula:RERI = OR11 - (OR10 - OR01) + 1
What the paper's own numbers show
Method. Under each interaction row, Tables 2–4 print four stratum-specific odds ratios. (A+ | B−) is OR10 and (B+ | A−) is OR01, both against the doubly unexposed group. (A+ | B+) is OR11/OR01 and (B+ | A+) is OR11/OR10. So OR11 can be rebuilt two ways, as (A+ | B+) × OR01 and as (B+ | A+) × OR10. From the mean of the two I computed RERI = OR11 − OR10 − OR01 + 1, AP = RERI/OR11, SI = (OR11 − 1)/[(OR10 − 1) + (OR01 − 1)] and the multiplicative measure ME = OR11/(OR10 × OR01). This checks only that each row is internally consistent. Confidence intervals cannot be recomputed without the model's covariance matrix. I then converted the abstract's percentages to the tables' scale (divide by 100), compared them with every HIV & drug use estimate and with every other Table 2 row, and ranked the positive RERIs whose 95% CI excludes 0 in each table. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| RERI, HIV & drug use | Abstract | 225% [174–304%] | = 2.25 [1.74–3.04] on the tables' scale |
| RERI, HIV & drug use, composite outcome | Table 2; Results text | 1.72 [1.28–2.28] | OR11 = 2.54 × 2.42 = 6.15 or 2.05 × 2.99 = 6.13; RERI 1.73 (range 1.72–1.74); as a percentage, 172% [128–228%] |
| AP / SI / ME, HIV & drug use, composite | Table 2 | 0.28 / 1.5 / 0.85 | 0.28 / 1.51 / 0.85 (as percentages 28%, 150%, 85%, none of them 225%) |
| RERI, Not PPL & retreatment, composite outcome | Table 2 | 2.25 [1.74–3.04] | OR11 = 2.9 × 1.59 = 4.61 or 2.6 × 1.78 = 4.63; RERI 2.25 (range 2.24–2.26); ×100 = 225% [174–304%], identical to the abstract |
| AP / SI / ME, Not PPL & retreatment, composite | Table 2 | 0.49 / 2.64 / 1.63 | 0.49 / 2.64 / 1.63 |
| RERI, HIV & drug use, death | Table 3; Results text | 2.76 [1.92–3.86] | 2.77; ≠ 2.25 [1.74–3.04] |
| RERI, HIV & drug use, LTFU | Table 4; Results text | 1.98 [1.39–2.76] | 1.98 (joint OR rebuilt 6.05, as printed in Table 4); ≠ 2.25 [1.74–3.04] |
| Abstract value vs Table 2 HIV & drug use | Abstract vs Table 2 | 2.25 vs 1.72 | abstract is 30.8% higher |
| Largest positive RERI with CI excluding 0, composite | Table 2 | abstract: HIV & drug use | Not PPL & retreatment 2.25, then HIV & drug use 1.72, Not PPL & immigrants 0.61, HIV & alcohol 0.45, tobacco & alcohol 0.10 |
| Same ranking, death | Table 3 | (not stated in abstract) | HIV & drug use 2.76, then HIV & not diabetes 1.12, Not PPL & retreatment 1.03 |
| Same ranking, LTFU | Table 4 | (not stated in abstract) | Not PPL & retreatment 3.12, then HIV & drug use 1.98, Not PPL & immigrants 0.79 |
All five rebuilt rows agree with the printed RERI, AP, SI and ME to within 0.01. So the tables are internally consistent, and the discrepancy lies between the abstract and the tables.
Figure
Figure 1. The abstract sentence (left) and the two Table 2 rows it could refer to (right), quoted from the article as archived in PMC. The abstract attributes 225% [174–304%], which is 2.25 [1.74–3.04] on the table's scale, to HIV infection and drug use. Table 2 and the Results text give 1.72 [1.28–2.28] for that pair. The abstract's estimate and both confidence limits are identical to the Table 2 row "Not PPL & retreatment" (not incarcerated and retreatment case). Because 2.25 is larger than 1.72, the abstract's "strongest additive synergy" also does not describe HIV and drug use on the composite outcome. Orange marks values printed in the paper.
Figure 2. Every RERI estimate the paper prints for HIV and drug use, together with the Table 2 Not PPL & retreatment row and the abstract value, on one linear RERI axis (0 = no additive interaction, thin vertical line). Orange diamonds and bars are the estimates and 95% CIs as printed. The hollow diamond is the abstract figure divided by 100. Blue circles are point estimates that recompute.py.txt rebuilds from each row's stratum-specific odds ratios; CIs cannot be rebuilt from the published numbers. The dashed line and shaded band mark the abstract's estimate and interval. Only the Not PPL & retreatment row matches them exactly. None of the three HIV and drug use estimates (composite 1.72, death 2.76, LTFU 1.98) does.
Why it matters
The abstract sentence is the only quantitative interaction result the abstract gives. It reports the analysis the title names ("multivariate modelling with excess risk"), so readers who see only the abstract or an index entry will take it as the headline finding. As printed, it says that HIV infection with drug use adds an excess of 225% of the baseline odds beyond the sum of the two separate effects, and that this is the largest such excess in the study. The paper's own tables say three things instead:
- For the composite outcome the HIV and drug use excess is 1.72 [1.28–2.28]. The abstract's figure is about 31% higher, and its interval is shifted upward (1.74–3.04 against 1.28–2.28).
- For the composite outcome, and for loss to follow-up, the largest additive interaction is between not being incarcerated and being a retreatment case (2.25 and 3.12), not HIV and drug use. HIV and drug use is the largest significant RERI only for death (2.76), and that is not the number the abstract quotes.
- For the composite outcome, the multiplicative measure for HIV and drug use is below 1 (0.85 [0.81–0.89]), which the Results describe as "an antagonist effect". The abstract's "strongest ... synergy" holds on the additive scale only.
What survives is the qualitative conclusion. HIV and drug use show a positive additive interaction with a CI that excludes 0 for all three outcomes. The Interpretation and Research in context statements that this combination raises the risk of death and loss to follow-up beyond the separate effects are consistent with Tables 3 and 4. The error concerns the size of the headline estimate, its interval and the ranking claim, not the direction of the finding.
Explanations considered
- The abstract refers to a different outcome. The abstract sentence does not name the outcome. The paper's other HIV and drug use estimates are 2.76 [1.92–3.86] for death (Table 3) and 1.98 [1.39–2.76] for loss to follow-up (Table 4). Neither equals 2.25 [1.74–3.04]. This does not explain the number.
- A different scale or measure. Multiplying 1.72 [1.28–2.28] by 100 gives 172% [128–228%]. The other measures in the same row, AP 0.28, SI 1.5 and ME 0.85, give 28%, 150% and 85%. None of them is 225%. This does not explain the number.
- The Table 2 value 1.72 is the typo, and 2.25 is correct for HIV and drug use. The row's four stratum-specific odds ratios reproduce RERI 1.72–1.74, AP 0.28, SI 1.51 and ME 0.85, and the Results text independently prints 1.72 [1.28–2.28]. Rebuilding the row with the abstract's 2.25 would need OR11 ≈ 6.66, which neither (A+ | B+) × OR01 nor (B+ | A+) × OR10 gives. The table value is the one supported by the paper's numbers.
- A coincidental match. The abstract's estimate and both confidence limits (three numbers to two decimals) equal the Table 2 Not PPL & retreatment row, whose own stratum odds ratios reproduce 2.25 together with AP 0.49, SI 2.64 and ME 1.63. A three-number coincidence is implausible. The most direct reading is that the value from one Table 2 row was attached to another row's label. The write-up makes no claim about how this happened.
- A different version of the article. PubMed (PMID 41399540), Europe PMC and the PMC copy, all derived from the publisher's XML, carry the same abstract sentence and the same Table 2. A conversion artefact in one archive does not account for it.
- The Methods formula. As printed, the Methods formula reads "OR11 - (OR10 - OR01) + 1", which for HIV and drug use would give 6.57, not 2.25. The tables follow the standard OR11 − OR10 − OR01 + 1. This is a separate typographical point and does not produce the abstract value.
A separate observation, outside the claim above: the Results text cites the "Not PPL & immigrants" interaction as "RERI = -0.61 [95% CI: -0.08 to (-1.02)]" and calls it antagonistic, whereas Table 2 prints 0.61 [0.08–1.02], AP 0.22, SI 1.53. The table therefore shows a positive interaction for that pair.
Status
As of 2026-10-24:
- Version of record. The abstract sentence and Table 2 were checked in the PMC article page and the Europe PMC full-text XML, both supplied by the publisher. The PubMed record (PMID 41399540) and the Europe PMC abstract carry the same sentence, "RERI of 225% [174-304%]". The publisher's own landing pages could not be read: doi.org redirects to Elsevier linkinghub, which returned only a redirect stub, and thelancet.com returned HTTP 403.
- Supplement. mmc1.pdf (15 pages: Supplementary Methods, Supplementary Tables 1–11, Supplementary Figure 1) contains no interaction or RERI analysis ("RERI" does not appear in it), so it cannot settle the discrepancy. No source data or code is deposited.
- Corrections. None found. Crossref shows no update-to or updated-by relation. Crossref Labs, which carries the Retraction Watch data, lists no updates. PMC and Europe PMC show no correction, erratum, retraction or expression-of-concern notice, and the PubMed record lists no comment or correction links. A web search found no correction notice.
- PubPeer. A web search found no PubPeer thread. A direct check was not possible: pubpeer.com returned HTTP 403 to a DOI search, and the API endpoint queried returned "Not Found".
Suggested correction
In the abstract (Findings), replace
with the strongest additive synergy observed between HIV infection and drug use, with a RERI of 225% [174–304%].
with wording that matches Table 2, for example:
with significant additive synergy between HIV infection and drug use (RERI 1.72 [1.28–2.28]) and the largest additive interaction between non-incarceration and retreatment (RERI 2.25 [1.74–3.04]).
Alternatively, if the authors intended to refer to mortality, where HIV and drug use is the largest significant interaction, the sentence could name that outcome and quote RERI 2.76 [1.92–3.86] (Table 3). Separately, the Methods formula should read RERI = OR11 − OR10 − OR01 + 1, and the Results text for Not PPL & immigrants should match the sign printed in Table 2.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the interaction measures behind the write-up for PMC12702379
(Lancet Reg Health Am 2025, doi:10.1016/j.lana.2025.101305) and redraw the figures.
Run: python recompute.py.txt
Every number below is typed from the article as archived in PubMed Central
(Tables 2, 3 and 4, abstract and Results text). Figures are written next to this file.
"""
import math
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
# Stratum-specific ORs, as printed under each interaction row:
# or10 = effect of A among B- (A+ | B-)
# or01 = effect of B among A- (B+ | A-)
# a_b = effect of A among B+ (A+ | B+) = OR11 / OR01
# b_a = effect of B among A+ (B+ | A+) = OR11 / OR10
# followed by the printed RERI, AP, SI and multiplicative measure (ME).
ROWS = {
"T2 HIV & drug use (composite)": dict(or10=2.99, or01=2.42, a_b=2.54, b_a=2.05,
reri=(1.72, 1.28, 2.28), ap=0.28, si=1.50, me=0.85),
"T2 Not PPL & retreatment (composite)": dict(or10=1.78, or01=1.59, a_b=2.90, b_a=2.60,
reri=(2.25, 1.74, 3.04), ap=0.49, si=2.64, me=1.63),
"T3 HIV & drug use (death)": dict(or10=4.64, or01=1.70, a_b=4.77, b_a=1.75,
reri=(2.76, 1.92, 3.86), ap=0.34, si=1.64, me=1.03),
"T4 HIV & drug use (LTFU)": dict(or10=1.86, or01=3.21, a_b=1.88, b_a=3.26,
reri=(1.98, 1.39, 2.76), ap=0.33, si=1.64, me=1.01),
"T4 Not PPL & retreatment (LTFU)": dict(or10=1.66, or01=1.70, a_b=3.23, b_a=3.30,
reri=(3.12, 2.41, 4.25), ap=0.57, si=3.30, me=1.94),
}
# Table 4 also prints the joint OR directly for two of these rows.
T4_JOINT = {"T4 HIV & drug use (LTFU)": 6.05, "T4 Not PPL & retreatment (LTFU)": 5.48}
ABSTRACT_PCT = (225, 174, 304) # "with a RERI of 225% [174–304%]"
RESULTS_TEXT = (1.72, 1.28, 2.28) # Results: "RERI = 1.72 [95% CI: 1.28–2.28]"
# Every RERI row of Tables 2-4: (estimate, lower, upper)
T2 = {
"Tobacco & alcohol": (0.10, 0.04, 0.17), "Tobacco & drug": (0.02, -0.09, 0.15),
"Drug & alcohol": (-0.25, -0.35, -0.13), "Not PPL & retreatment": (2.25, 1.74, 3.04),
"Not PPL & immigrants": (0.61, 0.08, 1.02), "Not PPL & pregnancy": (-1.00, -4.52, 0.98),
"Immigrants & retreatment": (-0.06, -0.39, 0.55), "Immigrants & pregnancy": (1.02, -1.48, 9.17),
"Homelessness & immigrants": (-0.45, -1.08, 0.45), "Homelessness & pregnancy": (-0.28, -2.03, 8.8),
"Homelessness & retreatment": (-0.40, -0.78, 0.23), "Not HCW & retreatment": (0.26, -0.10, 0.56),
"HIV & alcohol": (0.45, 0.23, 0.73), "HIV & tobacco": (-0.04, -0.21, 0.16),
"HIV & drug use": (1.72, 1.28, 2.28), "HIV & not diabetes": (-0.12, -0.43, 0.16),
"Not diabetes & drug": (-0.17, -0.38, 0.03), "Not diabetes & tobacco": (-0.07, -0.15, 0.0),
"Not diabetes & alcohol": (0.07, -0.02, 0.16), "Pregnancy & retreatment": (0.92, -1.09, 7.97),
}
T3 = {
"Tobacco & alcohol": (0.55, 0.43, 0.68), "Tobacco & drug": (0.03, -0.09, 0.17),
"Drug & alcohol": (-0.02, -0.15, 0.14), "Not PPL & retreatment": (1.03, 0.46, 2.26),
"Not PPL & immigrants": (0.26, -1.02, 1.07), "Not PPL & pregnancy": (-2.24, -24.16, 7.3),
"Immigrants & retreatment": (-0.16, -0.91, 1.09), "Immigrants & pregnancy": (5.15, -11.68, 88.07),
"Homelessness & immigrants": (0.12, -0.88, 1.83), "Homelessness & pregnancy": (0.80, -18.02, 45.08),
"Homelessness & retreatment": (-0.45, -0.72, 0.31), "Not HCW & retreatment": (-0.17, -0.72, 0.22),
"HIV & alcohol": (0.46, 0.04, 1.01), "HIV & tobacco": (-0.58, -0.91, -0.21),
"HIV & drug use": (2.76, 1.92, 3.86), "HIV & not diabetes": (1.12, 0.56, 1.64),
"Not diabetes & drug": (-0.28, -0.50, -0.09), "Not diabetes & tobacco": (-0.25, -0.34, -0.16),
"Not diabetes & alcohol": (-0.03, -0.15, 0.09), "Pregnancy & retreatment": (-0.48, -21.6, 18.46),
}
T4 = {
"Tobacco & alcohol": (-0.09, -0.14, -0.03), "Tobacco & drug": (-0.03, -0.22, 0.18),
"Drug & alcohol": (-0.33, -0.52, -0.12), "Not PPL & retreatment": (3.12, 2.41, 4.25),
"Not PPL & immigrants": (0.79, 0.22, 1.22), "Not PPL & pregnancy": (-0.54, -4.01, 2.12),
"Immigrants & retreatment": (-0.13, -0.48, 0.56), "Immigrants & pregnancy": (0.49, -2.39, 7.55),
"Homelessness & immigrants": (-0.87, -1.59, 0.23), "Homelessness & pregnancy": (-1.10, -2.97, 9.15),
"Homelessness & retreatment": (-0.73, -1.21, 0.09), "Not HCW & retreatment": (0.33, -0.11, 0.68),
"HIV & alcohol": (-0.07, -0.19, 0.10), "HIV & tobacco": (0.14, 0.01, 0.34),
"HIV & drug use": (1.98, 1.39, 2.76), "HIV & not diabetes": (-0.13, -0.40, 0.10),
"Not diabetes & drug": (-0.62, -0.96, -0.31), "Not diabetes & tobacco": (0.10, 0.0, 0.19),
"Not diabetes & alcohol": (0.29, 0.17, 0.40), "Pregnancy & retreatment": (0.47, -1.91, 7.63),
}
def measures(r):
or11_a = r["a_b"] * r["or01"] # OR11 = (A+|B+) x OR01
or11_b = r["b_a"] * r["or10"] # OR11 = (B+|A+) x OR10
or11 = (or11_a + or11_b) / 2
reri = or11 - r["or10"] - r["or01"] + 1
ap = reri / or11
si = (or11 - 1) / ((r["or10"] - 1) + (r["or01"] - 1))
me = or11 / (r["or10"] * r["or01"])
return dict(or11_a=or11_a, or11_b=or11_b, or11=or11, reri=reri, ap=ap, si=si, me=me,
reri_a=or11_a - r["or10"] - r["or01"] + 1,
reri_b=or11_b - r["or10"] - r["or01"] + 1)
# ------------------------------------------------------------------ 1. internal consistency
print("1. Interaction measures rebuilt from each row's stratum-specific ORs")
print(f"{'row':38s}{'OR11a':>7s}{'OR11b':>7s}{'RERI range':>13s}{'RERI':>6s}{'(rep)':>7s}"
f"{'AP':>6s}{'(rep)':>7s}{'SI':>6s}{'(rep)':>7s}{'ME':>6s}{'(rep)':>7s}")
REC = {}
for k, r in ROWS.items():
m = measures(r)
REC[k] = m
rng = f"{min(m['reri_a'], m['reri_b']):.2f}-{max(m['reri_a'], m['reri_b']):.2f}"
print(f"{k:38s}{m['or11_a']:7.2f}{m['or11_b']:7.2f}{rng:>13s}{m['reri']:6.2f}({r['reri'][0]:.2f})"
f"{m['ap']:6.2f}({r['ap']:.2f}){m['si']:6.2f}({r['si']:.2f}){m['me']:6.2f}({r['me']:.2f})")
for k, j in T4_JOINT.items():
print(f" {k}: joint OR printed in Table 4 = {j:.2f}; rebuilt OR11 = {REC[k]['or11']:.2f}")
# ------------------------------------------------------------------ 2. abstract vs tables
print("\n2. Abstract figure 225% [174-304%] on the tables' scale:",
tuple(x / 100 for x in ABSTRACT_PCT))
cands = {
"T2 HIV & drug use (composite)": ROWS["T2 HIV & drug use (composite)"]["reri"],
"T2 Not PPL & retreatment (composite)": ROWS["T2 Not PPL & retreatment (composite)"]["reri"],
"T3 HIV & drug use (death)": ROWS["T3 HIV & drug use (death)"]["reri"],
"T4 HIV & drug use (LTFU)": ROWS["T4 HIV & drug use (LTFU)"]["reri"],
}
for k, v in cands.items():
pct = tuple(round(x * 100) for x in v)
print(f" {k:38s} x100 = {pct[0]}% [{pct[1]}-{pct[2]}%] "
f"identical to abstract: {pct == ABSTRACT_PCT}")
print(f" Results text value for HIV & drug use (composite): {RESULTS_TEXT}, "
f"x100 = {tuple(round(x * 100) for x in RESULTS_TEXT)}")
hd = ROWS["T2 HIV & drug use (composite)"]
print(f" Other measures of the T2 HIV & drug use row x100: AP {hd['ap'] * 100:.0f}%, "
f"SI {hd['si'] * 100:.0f}%, ME {hd['me'] * 100:.0f}% (none is 225%)")
over = ABSTRACT_PCT[0] / 100 / hd["reri"][0] - 1
print(f" Abstract RERI vs Table 2 HIV & drug use: 2.25 / 1.72 - 1 = {100 * over:.1f}% higher")
need = ABSTRACT_PCT[0] / 100 + hd["or10"] + hd["or01"] - 1
print(f" OR11 needed for RERI 2.25 with OR10 {hd['or10']} and OR01 {hd['or01']}: {need:.2f} "
f"(the row's stratum ORs give {REC['T2 HIV & drug use (composite)']['or11_a']:.2f} and "
f"{REC['T2 HIV & drug use (composite)']['or11_b']:.2f})")
# ------------------------------------------------------------------ 3. ranking claim
print("\n3. Positive RERIs whose 95% CI excludes 0, ranked (the abstract calls HIV & drug use "
"'the strongest')")
for name, t in (("Table 2, composite", T2), ("Table 3, death", T3), ("Table 4, LTFU", T4)):
sig = sorted([(k, v) for k, v in t.items() if v[1] > 0], key=lambda kv: -kv[1][0])
print(f" {name}:")
for i, (k, v) in enumerate(sig, 1):
print(f" {i}. {k:26s} {v[0]:5.2f} [{v[1]:.2f}, {v[2]:.2f}]")
top_all = max(t.items(), key=lambda kv: kv[1][0])
print(f" largest point estimate regardless of CI: {top_all[0]} {top_all[1][0]:.2f} "
f"[{top_all[1][1]:.2f}, {top_all[1][2]:.2f}]")
# ------------------------------------------------------------------ 4. Methods formula (side point)
m = REC["T2 HIV & drug use (composite)"]
r = ROWS["T2 HIV & drug use (composite)"]
printed = m["or11"] - (r["or10"] - r["or01"]) + 1
print("\n4. Side point: Methods prints 'RERI = OR11 - (OR10 - OR01) + 1'.")
print(f" For T2 HIV & drug use that formula gives {printed:.2f}; the standard "
f"OR11 - OR10 - OR01 + 1 gives {m['reri']:.2f}; Table 2 prints 1.72.")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
GRID, PANEL = "#e4e3df", "#f3f2ee"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "svg.fonttype": "none", "lines.linewidth": 2,
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
def fig1():
fig = plt.figure(figsize=(W, 4.7), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
ax.text(0.025, 0.965, "The abstract's RERI for HIV & drug use is Table 2's value for another pair",
fontsize=11, weight="bold", va="top")
L, R = (0.025, 0.37), (0.585, 0.975)
top, bot = 0.88, 0.14
for x0, x1 in (L, R):
ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot,
boxstyle="round,pad=0,rounding_size=0.015",
fc=PANEL, ec=GRID, lw=1))
ax.text(L[0] + 0.015, top - 0.035, "Abstract (Findings)", fontsize=12, weight="bold", va="top")
ax.text(L[0] + 0.015, top - 0.11,
wrap("“…with the strongest additive synergy observed between HIV infection "
"and drug use, with a RERI of 225% [174–304%].”", 30),
fontsize=10, va="top", linespacing=1.4)
ax.text(L[0] + 0.015, bot + 0.17, "on Table 2's scale:", fontsize=10, color=INK2, va="top")
ax.text(L[0] + 0.015, bot + 0.11, "2.25 [1.74–3.04]", fontsize=11, weight="bold",
color=ORANGE, va="top")
ax.text(R[0] + 0.015, top - 0.035, "Table 2, composite outcome", fontsize=11.5,
weight="bold", va="top")
# two table rows as sub-boxes
rows = [("HIV & drug use", "RERI 1.72 [1.28–2.28]", 0.66),
("Not PPL & retreatment", "RERI 2.25 [1.74–3.04]", 0.42)]
for lab, val, y in rows:
ax.add_patch(FancyBboxPatch((R[0] + 0.015, y - 0.075), R[1] - R[0] - 0.03, 0.15,
boxstyle="round,pad=0,rounding_size=0.01",
fc=BG, ec=GRID, lw=1))
ax.text(R[0] + 0.03, y + 0.045, lab, fontsize=10.5, weight="bold", va="center")
ax.text(R[0] + 0.03, y - 0.03, val, fontsize=10.5, color=ORANGE, va="center")
ax.text(R[0] + 0.015, bot + 0.135,
wrap("Results text, same pair: “RERI = 1.72 [95% CI: 1.28–2.28]”", 36),
fontsize=10, color=INK2, va="top", linespacing=1.35)
xm = (L[1] + R[0]) / 2
# connector to HIV & drug use row
ax.annotate("", xy=(R[0] + 0.008, 0.66), xytext=(L[1] + 0.008, 0.66),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=2, mutation_scale=14,
shrinkA=0, shrinkB=0))
ax.text(xm, 0.68, "same pair,\ndifferent value", ha="center", va="bottom",
fontsize=10, weight="bold", linespacing=1.25)
ax.text(xm, 0.64, "1.72, not 2.25", ha="center", va="top", fontsize=10, color=INK2)
# connector to Not PPL & retreatment row
ax.annotate("", xy=(R[0] + 0.008, 0.42), xytext=(L[1] + 0.008, 0.30),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=2, mutation_scale=14,
shrinkA=0, shrinkB=0, connectionstyle="arc3,rad=0.15"))
ax.text(xm + 0.005, 0.445, "identical estimate\nand CI, other pair", ha="center",
va="bottom", fontsize=10, weight="bold", linespacing=1.25)
ax.text(0.025, 0.09, "2.25 > 1.72, so in Table 2 HIV & drug use is not the strongest "
"additive synergy either.", fontsize=10, color=INK, va="top")
ax.text(0.025, 0.045, "Quotes verbatim from the article as archived in PMC. "
"Orange = values printed in the paper.", fontsize=10, color=INK2, va="top")
save(fig, "fig1")
def fig2():
items = [ # label, reported (est, lo, hi), recomputed point or None, marker style
("Abstract: “HIV infection and\ndrug use” (outcome not stated)",
tuple(x / 100 for x in ABSTRACT_PCT), None),
("Table 2, composite outcome:\nHIV & drug use",
ROWS["T2 HIV & drug use (composite)"]["reri"], REC["T2 HIV & drug use (composite)"]["reri"]),
("Table 2, composite outcome:\nNot PPL & retreatment",
ROWS["T2 Not PPL & retreatment (composite)"]["reri"],
REC["T2 Not PPL & retreatment (composite)"]["reri"]),
("Table 3, death:\nHIV & drug use",
ROWS["T3 HIV & drug use (death)"]["reri"], REC["T3 HIV & drug use (death)"]["reri"]),
("Table 4, loss to follow-up:\nHIV & drug use",
ROWS["T4 HIV & drug use (LTFU)"]["reri"], REC["T4 HIV & drug use (LTFU)"]["reri"]),
]
fig, ax = plt.subplots(figsize=(W, 4.3), facecolor=BG)
fig.subplots_adjust(left=0.33, right=0.97, top=0.86, bottom=0.2)
ax.set_facecolor(BG)
n = len(items)
ab = items[0][1]
ax.axvspan(ab[1], ab[2], color=ORANGE, alpha=0.07, lw=0, zorder=0)
ax.axvline(ab[0], color=ORANGE, lw=1, ls=(0, (4, 3)), zorder=1)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
for i, (lab, rep, rec) in enumerate(items):
y = n - 1 - i
ax.plot([rep[1], rep[2]], [y + 0.08, y + 0.08], color=ORANGE, lw=2, zorder=3,
solid_capstyle="butt")
if i == 0:
ax.plot(rep[0], y + 0.08, marker="D", ms=9, mfc=BG, mec=ORANGE, mew=2, zorder=4)
else:
ax.plot(rep[0], y + 0.08, marker="D", ms=8, color=ORANGE, zorder=4)
ax.plot(rec, y - 0.17, marker="o", ms=8, color=BLUE, zorder=4)
ax.text(rep[2] + 0.07, y + 0.08, f"{rep[0]:.2f} [{rep[1]:.2f}–{rep[2]:.2f}]",
va="center", fontsize=10, color=INK2)
ax.set_yticks(range(n))
ax.set_yticklabels([it[0] for it in items][::-1], fontsize=10)
ax.set_ylim(-0.6, n - 0.4)
ax.set_xlim(-0.2, 5.3)
ax.set_xlabel("RERI (additive interaction; 0 = no interaction)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(ab[0], n - 0.42, "abstract value", ha="center", va="bottom", fontsize=10,
color=ORANGE)
fig.text(0.02, 0.965, "Which estimate is 2.25 [1.74–3.04]? Only Not PPL & retreatment",
fontsize=11, weight="bold", va="top")
h = [Line2D([], [], color=ORANGE, marker="D", ms=8, lw=2,
label="reported in the paper (estimate, 95% CI)"),
Line2D([], [], color=BLUE, marker="o", ms=8, lw=0,
label="recomputed from the row's stratum ORs")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.32, -0.17), ncol=2,
frameon=False, fontsize=10, handletextpad=0.4, columnspacing=1.2)
save(fig, "fig2")
fig1()
fig2()
print("\nFigures written: fig1.svg/png, fig2.svg/png in", HERE)
The firearm death-in-ED estimate is reported as 88.82, 83.82, 77.63 and 379.00
Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2025.101343 · PMC12811418
Summary
The paper's headline mortality result is the adjusted relative risk (RR) of dying in the emergency department (ED) for women injured by firearms compared with women injured by other assault mechanisms. The same estimate appears with four different values: 88.82 ("89 times") in the abstract, 83.82 in Table 3, 77.63 in the Results text, and 379.00 in the Discussion. The paper's own numbers show where each value comes from. The abstract prints Table 3's confidence interval (72.38–97.06), and that interval is centred on 83.82, not 88.82. The Results paragraph that cites Table 3 reproduces the age 18–29 column of Table 4. The Discussion's 379.00 (266.15–539.7) is the odds ratio from the supplementary logistic regression, which the Methods say was set aside because it gave inflated estimates. All versions are far above 1, so the direction and significance of the association hold. But the size of the paper's key finding is misstated in the abstract, and the Discussion overstates it by a factor of about 4.5.
What the paper reports
The abstract:
Female patients who were injured by firearms had 89 times higher risk of dying in the ED (RR = 88.82; 95% CI 6 = 72.38–97.06) compared to female patients injured by non-firearm injury mechanisms.
— Abstract, Findings (the stray "6" is in the source, and in the PubMed abstract too)
Table 3, model (b), "firearm-related assault compared to all other adult female assault patients". The columns are all-assault RR, its CI, firearm RR, its CI:
In ED | 1.04 | 0.89, 1.21 | 83.82 | 72.38, 97.06
— Table 3, Death: In ED row
The Results paragraph on the firearm model, which ends by citing Table 3:
Female patients with firearm injuries had far greater risk of dying in the ED (RR = 77.63; 95% CI 62.60–96.27) or hospital (RR = 3.29; 95% CI 2.57–4.22). They also had substantially higher risk of being admitted to the hospital (RR = 19.94; 95% CI 18.04–22.02). Compared to White female patients with firearm injuries, Black (RR = 4.15; 95% CI 3.61–4.78), Hispanic (RR = 1.77; 95% CI 1.48–2.13) and Other (RR = 2.01; 95% CI 1.51–2.68) patients had greater risk of firearm injury (Table 3).
— Results, paragraph 4 ("In the firearm injury model for women…")
Table 4 gives the age-stratified models. In the In ED row, the second estimate is the 18–29 y firearm model:
In ED | 3.14 (2.45, 4.02) | 77.63 (62.60, 96.27)
— Table 4, Death: In ED row (first two of eight columns)
The same 18–29 y value restated:
There appeared to be a bimodal distribution of increased mortality for firearm injuries for 18–29 years (death in ED: RR = 77.65; 95% CI 62.60–96.27, death in hospital: RR = 3.29; 95% CI 2.57–4.22)
— Results, paragraph 5
The Discussion:
Female patients who were injured by firearms specifically had 379.00 (95% CI 266.15–539.7) times higher risk of dying in the ED compared to all-cause female assault patients.
— Discussion, paragraph 1
On the logistic model:
We initially employed a survey-weighted binary logistic regression model. However, several results were affected by sparse-data bias and quasi-complete separation, leading to inflated odds ratio estimates.
The results from the initial survey-weighted logistic regression are provided in the Supplement for comparison.
— Methods, statistical analysis
The initial logistic regression was affected by sparse-data bias and quasi-complete separation, which inflated the odds ratios for several variables.
— Discussion, final paragraph
Supplementary Table 1 ("Adjusted Logistic Regression Model…", file mmc1.docx), firearm model, Death row: "ED | 1.04 | 0.89 – 1.21 | 379.00 | 266.15 – 539.71".
Every quotation from the article was checked character for character against the text archived in PubMed Central (src/show.py PMC12811418). Each occurs exactly once, and recompute.py.txt reruns that check. The supplementary row is transcribed from the publisher's mmc1.docx. Paragraphs are numbered from 1 under the Results and Discussion headings.
What the paper's own numbers show
Method. The model is a survey-weighted Poisson GLM with a log link (Methods; Table 3 caption), so its 95% intervals are Wald intervals on the log scale: L = exp(b − 1.96·SE) and U = exp(b + 1.96·SE). The point estimate that belongs with a printed interval is therefore its geometric midpoint, exp(b) = √(L·U). Comparing each printed value with √(L·U), and then searching the paper's tables for the matching cell, shows which numbers belong together. No outside data are used. The underlying NEDS microdata are licensed and not public, so the model itself cannot be refitted. The check is limited to the internal consistency of the reported values.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Firearm vs other assault, death in ED, all ages | Table 3 (b) | 83.82 (72.38, 97.06) | √(72.38 × 97.06) = 83.816 → 83.82. Consistent (log distance 0.147 to each limit) |
| Same estimate | Abstract | "89 times", RR = 88.82 (72.38–97.06) | Same CI as Table 3, whose centre is 83.82. 88.82 is off-centre (log distance 0.205 to L, 0.089 to U), 0.77 SE above the centre, and 6.0% above Table 3. "89" is round(88.82); round(83.82) = 84 |
| Same estimate, "firearm injury model … (Table 3)" | Results para 4 | 77.63 (62.60–96.27) | √(62.60 × 96.27) = 77.631. Identical to Table 4, 18–29 y firearm column; 7.4% below Table 3 |
| Same paragraph: death in hospital, admitted, Black, Hispanic, Other | Results para 4 | 3.29 (2.57–4.22); 19.94 (18.04–22.02); 4.15 (3.61–4.78); 1.77 (1.48–2.13); 2.01 (1.51–2.68) | Table 3 gives 3.34 (2.79, 4.00); 16.50 (15.29, 17.80); 4.12 (3.75, 4.52); 1.87 (1.64, 2.12); 1.90 (1.55, 2.33); none of these 15 numbers agree. Table 4 18–29 y gives 3.29 (2.57, 4.22); 19.94 (18.05, 22.02); 4.16 (3.61, 4.78); 1.78 (1.48, 2.14); 2.01 (1.51, 2.68). Across all six outcome and race rows, 14 of 18 numbers are identical to Table 4 18–29 y and the other 4 are within 0.01 |
| Same paragraph: age | Results para 4 | 0.98 (0.97–0.98) | Matches Table 3 (0.98, 0.97–0.98), not Table 4 18–29 y (0.96, 0.95–0.98). This is the only row of the paragraph that comes from Table 3 |
| 18–29 y firearm, death in ED | Results para 5 | 77.65 (62.60–96.27) | Table 4 gives 77.63; √(L·U) = 77.631 |
| Firearm death in ED, "times higher risk" | Discussion para 1 | 379.00 (266.15–539.7) | √(266.15 × 539.7) = 379.00. Equal to Suppl. Table 1 logistic adjusted odds ratio 379.00 (266.15–539.71). It appears in no RR table and is 4.52 × Table 3's RR |
| Firearm model, Black vs White | Discussion para 1, "nearly five times" | — | Table 3 RR = 4.12 (3.75, 4.52), as in the abstract. Suppl. Table 1 OR = 4.74 (4.29–5.24). "Nearly five" fits the odds ratio |
| All-assault model, Native American vs White | Abstract | RR = 2.81 (2.67–2.97), no age group stated | Equal to Table 4, 18–29 y column (stated as such in Results para 7). The all-ages Table 3 value is 2.68 (2.60, 2.77) |
| All-assault model, 18–29 y, death in ED | Results para 5 | 3.14 (2.45–4.20) | Table 4 gives 3.14 (2.45, 4.02). √(2.45 × 4.02) = 3.138 fits 3.14, while √(2.45 × 4.20) = 3.208 does not, so 4.20 is a transposition of 4.02 |
So the one value that is internally consistent is Table 3's 83.82 (72.38–97.06). The abstract pairs a different point estimate with Table 3's interval. The two differ in a single digit (88.82 vs 83.82), which looks like a typographical slip. The Results paragraph on the firearm model reports the 18–29 y stratum of Table 4 instead of the all-ages model it cites. The Discussion reports the logistic odds ratio as a risk. The last three rows of the table are minor slips of the same kind and are listed for completeness.
Figure
Figure 1. The printed versions of one estimate, and the centres of their printed intervals. Panel A shows each printed value (orange dot) with its printed 95% CI (orange line), on a logarithmic axis that runs from the null value (thin line at 1) to 1000. Next to it is the geometric centre of that CI, √(L×U), recomputed (blue open diamond). The Results para 4 value equals the age 18–29 y value in Table 4. The Discussion value equals the logistic odds ratio in Supplementary Table S1 and lies about 4.5 times above the Poisson estimates. Panel B zooms in (log axis, 70–100) on the two rows that print the same interval, 72.38–97.06. Table 3's 83.82 sits at the centre (log distance 0.147 to each limit). The abstract's 88.82 does not (0.205 and 0.089). All intervals lie far from 1.
Figure 2. Results paragraph 4 against the two table columns it could come from. The left column (orange) holds the values printed in the Results paragraph that cites Table 3. The middle and right columns hold the firearm-model values printed in Table 3 (all ages) and Table 4 (age 18–29 y). A cell is blue and bold when all three of its numbers (estimate and both limits) equal the Results-text value to within 0.01, and grey otherwise. For death in the ED, death in hospital, admission and the three race contrasts, the Results text matches Table 4's 18–29 y column (14 of 18 numbers identical, the rest within 0.01) and none of Table 3's 18 numbers. Only the age coefficient matches Table 3.
Why it matters
- It is the headline quantitative finding. The abstract's only mortality estimate is this RR, and the abstract, Results and Discussion each lead with it. A reader, or an indexing service that shows only the abstract, gets 88.82. A reader of the Discussion gets 379. The value the paper's own tables support is 83.82: about 84 times, not 89. The Discussion overstates it by a factor of 4.52.
- The Discussion figure uses the model the paper itself rejects. The Methods and the closing Discussion paragraph explain that the logistic model was replaced because it "inflated the odds ratios". The 379.00 in the Discussion's summary of findings is that model's odds ratio, presented as "times higher risk". For an outcome this rare among non-firearm assaults, an odds ratio and a relative risk are not interchangeable. Here they differ by a factor of 4.5. The same paragraph's "nearly five times more likely to identify as Black" also matches the odds ratio (4.74), not the RR the abstract reports (4.12).
- The Results text attributes subgroup numbers to the overall model. The firearm-model paragraph says "(Table 3)" but gives the 18–29 y estimates for death in the ED, death in hospital, admission and race. For admission, the stated 19.94 is 21% higher than the all-ages 16.50. A reader who compares text with table finds that they disagree on most of the rows.
- What is not affected. Every version of the estimate excludes 1 by a wide margin, so the direction and statistical significance of the firearm–mortality association do not change. The title and the qualitative Conclusion ("significantly higher mortality rates") also stand. Table 3 and Table 4 are internally consistent: each of the 15 cells used here sits at the centre of its CI to within 0.006, which is rounding. The problem is therefore in how the tables were carried into the text, not in the models as tabulated.
Explanations considered
- The abstract reports a different model or population than Table 3. This does not work. The abstract's comparison ("compared to female patients injured by non-firearm injury mechanisms") is Table 3 model (b), and it prints exactly Table 3's interval, 72.38–97.06. No estimate of 88.82 appears in any table or in the supplement. A point estimate of 88.82 cannot come from a log-scale Wald interval of 72.38–97.06, because it lies 0.77 standard errors from the centre. The abstract's own interval therefore implies 83.82.
- A rounding or significant-figure difference. This does not work. 88.82 and 83.82 differ by 5.00 (6%), and "89 times" rounds 88.82, not 83.82 (84).
- Results paragraph 4 deliberately describes the youngest age group. This does not work. The paragraph opens "In the firearm injury model for women", gives no age group, and ends "(Table 3)". Its age coefficient (0.98, 0.97–0.98) is the all-ages Table 3 value; within a single age stratum the coefficient differs (0.96 for 18–29 y). The age-stratified results are introduced only in the next paragraph ("We re-ran 8 survey-weighted Poisson regression models by age category … in Table 4"). That paragraph restates the same 18–29 y firearm values, which confirms where they came from.
- The Discussion intentionally quotes the logistic model. The value is indeed the logistic odds ratio (Supplementary Table 1: 379.00, 266.15–539.71). But the sentence calls it "times higher risk" without saying it is an odds ratio from the supplementary model, and it sits in the summary of the paper's own findings. The Methods say that model gave inflated odds ratios and was replaced. The comparator in the sentence ("all-cause female assault patients") also differs from the model's comparator (assault by all other mechanisms). So even read as intentional, the sentence misdescribes the measure and does not report the paper's primary estimate.
- A conversion or typesetting artifact in PMC. This is ruled out. The same values appear in the Europe PMC full-text JATS XML (each of "RR = 88.82", "RR = 77.63", "RR = 77.65", "379.00" and "83.82" exactly once) and in the PubMed abstract (PMID 41551920), which also carries the stray "95% CI 6 =". The table cells line up with their headers, and no minus signs, superscripts or decimal commas are involved.
- A table rather than the text is wrong. This is unlikely. Each of the 15 Table 3 and Table 4 cells examined sits at the geometric centre of its own interval to within 0.006 (rounding), against 5.00 for the abstract's 88.82. The textual values match specific cells of other tables (Table 4 18–29 y; Supplementary Table 1). The only value with no source in any table is the abstract's 88.82, and its interval is Table 3's.
None of these explanations reconciles the four values. The consistent reading is that Table 3 holds the primary estimate, 83.82 (72.38–97.06), and that the abstract, the Results paragraph and the Discussion each misreport it in a different way.
Status
As of 2026-10-23:
- Version of record: checked through publisher-deposited copies. The publisher's own pages could not be opened: doi.org leads to linkinghub.elsevier.com, which returned a redirect stub, and thelancet.com and sciencedirect.com returned HTTP 403. The Europe PMC full-text XML for PMC12811418 and the PubMed record (PMID 41551920) both carry every quoted passage unchanged. The Elsevier article metadata lists The Lancet Regional Health – Americas, February 2026, open access, CC BY 4.0.
- Supplements: the article's only supplementary file, mmc1.docx ("Supplementary Tables", 31,729 bytes), was downloaded from the publisher CDN and read in full. It holds Supplementary Tables 1 and 2, both logistic regression (adjusted odds ratios), all ages and by age group. Supplementary Table 1 is the source of the Discussion's 379.00 (266.15–539.71) and of OR 4.74 for Black women. No RR of 88.82 appears in it.
- Corrections and notices:
src/known.py 10.1016/j.lana.2025.101343returns no Crossref update and no PMC or Europe PMC notice. The Europe PMC record lists no comment or correction links. No erratum, correction, expression of concern or retraction was found. - PubPeer: PubPeer's API returned "Human verification required" (HTTP 403), so it could not be queried directly. A web search for the title together with "pubpeer" returned no PubPeer page. A thread therefore cannot be ruled out with certainty, but none was found.
- Sources: PMC12811418 (
src/show.py); https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12811418/fullTextXML; PubMed 41551920; https://ars.els-cdn.com/content/image/1-s2.0-S2667193X25003540-mmc1.docx; https://doi.org/10.1016/j.lana.2025.101343.
Suggested correction
- Abstract, Findings: replace "89 times higher risk of dying in the ED (RR = 88.82; 95% CI 6 = 72.38–97.06)" with "84 times higher risk of dying in the ED (RR = 83.82; 95% CI 72.38–97.06)", matching Table 3.
- Results, paragraph 4: replace the age 18–29 y values with the all-ages Table 3 values: death in ED RR = 83.82 (72.38–97.06), death in hospital RR = 3.34 (2.79–4.00), admitted RR = 16.50 (15.29–17.80), Black RR = 4.12 (3.75–4.52), Hispanic RR = 1.87 (1.64–2.12), Other RR = 1.90 (1.55–2.33). Alternatively, label them explicitly as the 18–29 y estimates and cite Table 4.
- Results, paragraph 5: change "RR = 77.65" to "RR = 77.63", and change the upper limit of the 18–29 y all-assault death-in-ED interval from "4.20" to "4.02", matching Table 4.
- Discussion, paragraph 1: replace "379.00 (95% CI 266.15–539.7) times higher risk of dying in the ED compared to all-cause female assault patients" with the primary Poisson estimate: "an 84-fold higher risk of dying in the ED (RR = 83.82; 95% CI 72.38–97.06) compared to female patients injured by other assault mechanisms". If the logistic estimate is kept, label it as an odds ratio from the supplementary model. Similarly, "nearly five times more likely to identify as Black" should read about four times (RR = 4.12; 95% CI 3.75–4.52), matching the abstract and Table 3.
- Abstract, Native American estimate (optional): either state that RR = 2.81 (2.67–2.97) refers to women aged 18–29 y, or give the all-ages Table 3 value, RR = 2.68 (2.60–2.77).
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12811418 (Lancet Reg Health Am 2026, doi:10.1016/j.lana.2025.101343)
# One adjusted estimate (firearm vs other assault mechanisms, death in the ED) is printed
# with four different values in the abstract, Table 3, the Results text and the Discussion.
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib (numpy not required).
#
# Inputs (all transcribed from the paper; nothing is estimated from outside data)
# * Text quotes and Tables 3 and 4: the article as archived in PubMed Central (PMC12811418).
# When run inside the /home/ubuntu/papers repo, the script re-checks that every quote
# occurs exactly once in the archived full text via src/show.py (skipped elsewhere).
# * Supplementary Tables S1/S2: the article's only supplementary file, mmc1.docx
# ("Supplementary Tables"), from
# https://ars.els-cdn.com/content/image/1-s2.0-S2667193X25003540-mmc1.docx
# These are the survey-weighted LOGISTIC regression results (adjusted odds ratios), which
# the Methods say were set aside as inflated by sparse-data bias / quasi-complete separation.
#
# Method. For a ratio estimate with a Wald interval on the log scale,
# L = exp(b - z*SE), U = exp(b + z*SE) => exp(b) = sqrt(L*U).
# So the point estimate that belongs with a printed interval is the interval's geometric
# midpoint. Comparing each printed point estimate with sqrt(L*U) identifies which printed
# values belong together, and the matching table cell identifies where each value came from.
import math, os, subprocess
import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC12811418"
# ------------------------------------------------------------------ quotes (verbatim)
QUOTES = [
("Abstract, Findings",
"Female patients who were injured by firearms had 89 times higher risk of dying in the ED "
"(RR = 88.82; 95% CI 6 = 72.38–97.06) compared to female patients injured by non-firearm "
"injury mechanisms."),
("Abstract, Findings",
"Racial disparities were prevalent, with Native American women experiencing the greatest risk "
"of being assaulted (RR = 2.81; 95% CI 2.67–2.97)."),
("Abstract, Findings",
"Victims of firearm related assaults had nearly 4.12 times the risk of identifying as Black "
"compared with those assaulted without firearms (95% CI 3.75–4.52)."),
("Methods, statistical analysis",
"We initially employed a survey-weighted binary logistic regression model. However, several "
"results were affected by sparse-data bias and quasi-complete separation, leading to inflated "
"odds ratio estimates."),
("Methods, statistical analysis",
"The results from the initial survey-weighted logistic regression are provided in the "
"Supplement for comparison."),
("Results, paragraph 4 (firearm injury model)",
"Female patients with firearm injuries had far greater risk of dying in the ED (RR = 77.63; "
"95% CI 62.60–96.27) or hospital (RR = 3.29; 95% CI 2.57–4.22). They also had substantially "
"higher risk of being admitted to the hospital (RR = 19.94; 95% CI 18.04–22.02). Compared to "
"White female patients with firearm injuries, Black (RR = 4.15; 95% CI 3.61–4.78), Hispanic "
"(RR = 1.77; 95% CI 1.48–2.13) and Other (RR = 2.01; 95% CI 1.51–2.68) patients had greater "
"risk of firearm injury (Table 3)."),
("Results, paragraph 5 (age strata, Table 4)",
"There appeared to be a bimodal distribution of increased mortality for firearm injuries for "
"18–29 years (death in ED: RR = 77.65; 95% CI 62.60–96.27, death in hospital: RR = 3.29; "
"95% CI 2.57–4.22)"),
("Results, paragraph 5 (age strata, Table 4)",
"Female patients 18–29 years old, had the greatest risk of death, whether in the ED "
"(RR = 3.14; 95% CI 2.45–4.20)"),
("Discussion, paragraph 1",
"Female patients who were injured by firearms specifically had 379.00 (95% CI 266.15–539.7) "
"times higher risk of dying in the ED compared to all-cause female assault patients."),
("Discussion, paragraph 1",
"victims of firearm-related assaults nearly five times more likely to identify as Black than "
"those assaulted without firearms"),
("Discussion, final paragraph",
"The initial logistic regression was affected by sparse-data bias and quasi-complete "
"separation, which inflated the odds ratios for several variables."),
("Table 3, Death: In ED row (all-assault RR, CI, firearm RR, CI)",
"In ED | 1.04 | 0.89, 1.21 | 83.82 | 72.38, 97.06"),
("Table 4, Death: In ED row (18-29 firearm = 2nd estimate)",
"In ED | 3.14 (2.45, 4.02) | 77.63 (62.60, 96.27)"),
]
def check_quotes():
repo = "/home/ubuntu/papers"
py = "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.\n")
return
full = subprocess.run([py, "src/show.py", PMCID, "--full"], cwd=repo,
capture_output=True, text=True).stdout
print("Quote check against the PMC-archived text (occurrences; must be 1):")
for loc, q in QUOTES:
n = full.count(q)
print(f" [{'ok ' if n == 1 else 'BAD'}] {n} {loc}: {q[:70]}...")
print()
# ------------------------------------------------------------------ the estimates
# (key, label, location, measure, point, lo, hi)
ED_DEATH = [
("abs", "Abstract", "Abstract, Findings", "RR", 88.82, 72.38, 97.06),
("t3", "Table 3 (b), all ages", "Table 3, Death: In ED", "RR", 83.82, 72.38, 97.06),
("rp3", "Results p4 (cites Table 3)", "Results para 4", "RR", 77.63, 62.60, 96.27),
("t4", "Table 4, firearm, 18-29 y", "Table 4, Death: In ED", "RR", 77.63, 62.60, 96.27),
("rp4", "Results p5, 18-29 y", "Results para 5", "RR", 77.65, 62.60, 96.27),
("disc", "Discussion", "Discussion para 1", "'risk'", 379.00, 266.15, 539.7),
("s1", "Suppl. Table S1 (logistic OR)", "Suppl. Table S1, Death: ED", "OR", 379.00, 266.15, 539.71),
]
# Other age strata, Table 4 firearm In ED (for context in the figure)
T4_OTHER = [("30-39 y", 78.40, 59.30, 103.65), ("40-49 y", 92.63, 66.98, 128.10),
("50+ y", 110.20, 70.04, 173.38)]
def gmid(lo, hi):
return math.sqrt(lo * hi)
def logasym(p, lo, hi):
# (log U - log p) - (log p - log L); 0 when p is the geometric midpoint
return (math.log(hi) - math.log(p)) - (math.log(p) - math.log(lo))
def part1():
print("1. Death in the ED, firearm vs other assault mechanisms: printed value vs geometric "
"midpoint of its printed 95% CI")
print(f" {'source':34s} {'meas':6s} {'printed':>8s} {'CI':>17s} {'sqrt(L*U)':>10s} "
f"{'log-asym':>9s}")
for k, lab, loc, m, p, lo, hi in ED_DEATH:
print(f" {lab:34s} {m:6s} {p:8.2f} {lo:8.2f}-{hi:<8.2f} {gmid(lo, hi):10.3f} "
f"{logasym(p, lo, hi):9.4f}")
p, lo, hi = 88.82, 72.38, 97.06
se = (math.log(hi) - math.log(lo)) / (2 * 1.959964)
print(f" Abstract CI: log-distance from 88.82 to L = {math.log(p/lo):.4f}, to U = "
f"{math.log(hi/p):.4f}; from 83.82: {math.log(83.82/lo):.4f} and {math.log(hi/83.82):.4f}")
print(f" Implied SE(log RR) = {se:.4f}; 88.82 lies {(math.log(p)-math.log(gmid(lo,hi)))/se:.2f} "
f"SE above the CI centre")
print(f" 88.82 / 83.82 = {88.82/83.82:.4f} (abstract {100*(88.82/83.82-1):.1f}% above Table 3)")
print(f" '89 times' = round(88.82) = {round(88.82)}; round(83.82) = {round(83.82)}")
print(f" 77.63 / 83.82 = {77.63/83.82:.4f} (Results p4 {100*(1-77.63/83.82):.1f}% below Table 3)")
print(f" 379.00 / 83.82 = {379.00/83.82:.2f} (Discussion vs Table 3)")
print(f" Abstract 88.82 and Table 3 83.82 differ in one digit (8 vs 3, second digit)")
print()
# Results p4 values vs Table 3 firearm column and Table 4 18-29 firearm column
P3_ROWS = [
# quantity, Results p4, Table 3 firearm (all ages), Table 4 firearm 18-29
("Death in ED", (77.63, 62.60, 96.27), (83.82, 72.38, 97.06), (77.63, 62.60, 96.27)),
("Death in hospital",(3.29, 2.57, 4.22), (3.34, 2.79, 4.00), (3.29, 2.57, 4.22)),
("Admitted", (19.94, 18.04, 22.02), (16.50, 15.29, 17.80), (19.94, 18.05, 22.02)),
("Black", (4.15, 3.61, 4.78), (4.12, 3.75, 4.52), (4.16, 3.61, 4.78)),
("Hispanic", (1.77, 1.48, 2.13), (1.87, 1.64, 2.12), (1.78, 1.48, 2.14)),
("Other race", (2.01, 1.51, 2.68), (1.90, 1.55, 2.33), (2.01, 1.51, 2.68)),
("Age (per year)", (0.98, 0.97, 0.98), (0.98, 0.97, 0.98), (0.96, 0.95, 0.98)),
]
def part2():
print("2. Results paragraph 4 ('firearm injury model ... (Table 3)'): which table column do its "
"numbers match?")
print(f" {'quantity':18s} {'Results p4':>20s} {'Table 3 firearm':>20s} {'Table 4 18-29':>20s} "
f"exact(T3) exact(T4) within0.01(T4)")
tot3 = tot4 = near4 = n = 0
for q, r, t3, t4 in P3_ROWS:
e3 = sum(a == b for a, b in zip(r, t3)); e4 = sum(a == b for a, b in zip(r, t4))
w4 = sum(abs(a - b) <= 0.0101 for a, b in zip(r, t4))
fmt = lambda t: f"{t[0]:.2f} ({t[1]:.2f}-{t[2]:.2f})"
print(f" {q:18s} {fmt(r):>20s} {fmt(t3):>20s} {fmt(t4):>20s} {e3:>8d}/3 {e4:>8d}/3 {w4:>8d}/3")
if q != "Age (per year)":
tot3 += e3; tot4 += e4; near4 += w4; n += 3
print(f" Six outcome/race rows (18 numbers): exact match Table 3 = {tot3}/{n}, "
f"Table 4 18-29 = {tot4}/{n}, within 0.01 of Table 4 18-29 = {near4}/{n}")
print(" Age row matches Table 3 (0.98, 0.97-0.98), not Table 4 18-29 (0.96, 0.95-0.98).")
print()
def part3():
print("3. Other values checked")
# Discussion 'nearly five times ... Black' vs RR and OR
print(f" Black, firearm model: Table 3 RR 4.12 (3.75-4.52), centre {gmid(3.75,4.52):.3f}; "
f"Suppl. S1 OR 4.74 (4.29-5.24), centre {gmid(4.29,5.24):.3f}; "
f"Discussion 'nearly five times' is closer to the OR (5 - 4.74 = {5-4.74:.2f}; 5 - 4.12 = {5-4.12:.2f})")
# Abstract Native American 2.81
print(f" Native American, all-assault model: abstract 2.81 (2.67-2.97), centre "
f"{gmid(2.67,2.97):.3f}; Table 4 18-29 = 2.81 (2.67, 2.97); Table 3 all ages = 2.68 (2.60, 2.77), "
f"centre {gmid(2.60,2.77):.3f}")
# Results p5: 18-29 all-assault death in ED, upper limit 4.20 vs 4.02
print(f" 18-29 all-assault death in ED: Results p5 3.14 (2.45-4.20), centre {gmid(2.45,4.20):.3f}; "
f"Table 4 3.14 (2.45, 4.02), centre {gmid(2.45,4.02):.3f}")
# Results p5 18-29 firearm in hospital matches Table 4
print(f" 18-29 firearm in hospital: Results p5 3.29 (2.57-4.22) = Table 4; centre {gmid(2.57,4.22):.3f}")
# admission: Results text vs Table 3
print(f" Admitted: Results 19.94 / Table 3 16.50 = {19.94/16.50:.4f} ({100*(19.94/16.50-1):.0f}% higher)")
# internal consistency of the table cells used here: printed estimate vs CI centre
cells = [("T3 In ED", 83.82, 72.38, 97.06), ("T3 In hospital", 3.34, 2.79, 4.00),
("T3 Admit", 16.50, 15.29, 17.80), ("T3 Black", 4.12, 3.75, 4.52),
("T3 Hispanic", 1.87, 1.64, 2.12), ("T3 Other", 1.90, 1.55, 2.33),
("T4 18-29 In ED", 77.63, 62.60, 96.27), ("T4 18-29 In hospital", 3.29, 2.57, 4.22),
("T4 18-29 Admit", 19.94, 18.05, 22.02), ("T4 18-29 Black", 4.16, 3.61, 4.78),
("T4 18-29 Hispanic", 1.78, 1.48, 2.14), ("T4 18-29 Other", 2.01, 1.51, 2.68),
("T4 30-39 In ED", 78.40, 59.30, 103.65), ("T4 40-49 In ED", 92.63, 66.98, 128.10),
("T4 50+ In ED", 110.20, 70.04, 173.38)]
worst = max(abs(p - gmid(lo, hi)) for _, p, lo, hi in cells)
print(" Table cells used here, printed vs sqrt(L*U): " +
"; ".join(f"{k} {p:.2f}/{gmid(lo, hi):.2f}" for k, p, lo, hi in cells))
print(f" Largest |printed - sqrt(L*U)| among these table cells = {worst:.3f} "
f"(abstract 88.82 vs 83.82: {88.82 - gmid(72.38, 97.06):.3f})")
# every version excludes 1
print(" Lower CI limit of every version of the firearm ED-death estimate > 1: "
+ str(all(lo > 1 for *_, lo, hi in ED_DEATH)))
print()
# ------------------------------------------------------------------ figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT, "ytick.color": TXT,
"svg.fonttype": "none"})
def fig1():
rows = [ # label, sublabel, point, lo, hi
("Abstract", "“89 times”, RR = 88.82", 88.82, 72.38, 97.06),
("Table 3 (b), all ages", "Poisson RR = 83.82", 83.82, 72.38, 97.06),
("Results para 4", "“firearm injury model … (Table 3)”", 77.63, 62.60, 96.27),
("Table 4, age 18–29 y only", "Poisson RR = 77.63", 77.63, 62.60, 96.27),
("Discussion para 1", "“379.00 … times higher risk”", 379.00, 266.15, 539.7),
("Suppl. Table S1", "logistic odds ratio = 379.00", 379.00, 266.15, 539.71),
]
from matplotlib.lines import Line2D
fig, (ax, zx) = plt.subplots(2, 1, figsize=(7.2, 6.6), facecolor=BG,
gridspec_kw=dict(height_ratios=[6, 1.9], hspace=0.55))
ax.set_facecolor(BG); zx.set_facecolor(BG)
ys = [5 - i for i in range(len(rows))]
off = 0.24 # recomputed centre drawn just below the printed value
for y, (lab, sub, p, lo, hi) in zip(ys, rows):
ax.plot([lo, hi], [y + 0.08, y + 0.08], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([p], [y + 0.08], "o", ms=8, color=ORANGE, zorder=4)
g = gmid(lo, hi)
ax.plot([g], [y - off], marker="D", ms=8, mfc="none", mec=BLUE, mew=2, zorder=5, ls="none")
his = "539.7" if hi == 539.7 else f"{hi:.2f}" # Discussion prints 539.7
ax.text(1.01, y + 0.08, f"{p:.2f} ({lo:.2f}–{his})", transform=ax.get_yaxis_transform(),
va="center", fontsize=10, color=ORANGE)
ax.text(1.01, y - off, f"{g:.2f}", transform=ax.get_yaxis_transform(),
va="center", fontsize=10, color=BLUE)
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.text(1.07, -0.65, "null (ratio = 1)", fontsize=10, color=SEC, va="center")
ax.set_xscale("log")
ax.set_xlim(0.9, 1000)
ax.set_ylim(-0.95, 5.55)
ax.set_xticks([1, 10, 100, 1000]); ax.set_xticklabels(["1", "10", "100", "1000"])
ax.set_yticks(ys)
ax.set_yticklabels([f"{lab}\n{sub}" for lab, sub, *_ in rows], fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Firearm vs other assault mechanisms, death in the ED (ratio, log scale)")
ax.set_title("A. All printed versions of the estimate (full range, null at 1)", loc="left",
fontsize=10.5, color=TXT, x=-0.6)
# panel B: zoom on the two rows that print the same CI
for yy, p, lab in ((1, 88.82, "Abstract\nRR = 88.82"), (0, 83.82, "Table 3 (b)\nRR = 83.82")):
zx.plot([72.38, 97.06], [yy + 0.0, yy], color=ORANGE, lw=2, solid_capstyle="butt")
for e in (72.38, 97.06):
zx.plot([e, e], [yy - 0.15, yy + 0.15], color=ORANGE, lw=2)
zx.plot([p], [yy], "o", ms=8, color=ORANGE, zorder=4)
zx.plot([gmid(72.38, 97.06)], [yy], marker="D", ms=9, mfc="none", mec=BLUE, mew=2,
ls="none", zorder=5)
zx.set_yticks([1, 0]); zx.set_yticklabels(["Abstract\nRR = 88.82", "Table 3 (b)\nRR = 83.82"],
fontsize=10)
zx.set_xscale("log"); zx.set_xlim(70, 100); zx.set_ylim(-0.5, 1.5)
zx.set_xticks([72.38, 83.82, 88.82, 97.06])
zx.set_xticklabels(["72.38", "83.82", "88.82", "97.06"], fontsize=10)
zx.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
zx.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for sp in ("top", "right", "left"):
zx.spines[sp].set_visible(False)
zx.spines["bottom"].set_color(SEC); zx.tick_params(axis="y", length=0)
zx.set_title("B. Zoom (log scale, 70–100): one printed CI, two printed point estimates",
loc="left", fontsize=10.5, color=TXT, x=-0.6)
zx.text(1.01, 1, f"log distance\nto L {math.log(88.82/72.38):.3f}, to U {math.log(97.06/88.82):.3f}",
transform=zx.get_yaxis_transform(), va="center", fontsize=10, color=SEC)
zx.text(1.01, 0, f"log distance\nto L {math.log(83.82/72.38):.3f}, to U {math.log(97.06/83.82):.3f}",
transform=zx.get_yaxis_transform(), va="center", fontsize=10, color=SEC)
h = [Line2D([0], [0], color=ORANGE, lw=2, marker="o", ms=8,
label="value printed in the paper, with its 95% CI"),
Line2D([0], [0], ls="none", marker="D", ms=8, mfc="none", mec=BLUE, mew=2,
label="recomputed: geometric centre of that CI, √(L×U)")]
fig.legend(handles=h, loc="lower left", bbox_to_anchor=(0.2, 0.0), ncol=1, frameon=False,
fontsize=10, columnspacing=1.2, handletextpad=0.5)
fig.subplots_adjust(left=0.36, right=0.75, top=0.95, bottom=0.12)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
def fig2():
fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(-0.3, 4.7); ax.axis("off")
ax.text(0.15, 4.5, "Results paragraph 4 says it reports the all-ages firearm model (Table 3).",
fontsize=11, fontweight="bold", va="center")
ax.text(0.15, 4.2, "Its numbers are those of the age 18–29 y column of Table 4.",
fontsize=11, color=SEC, va="center")
cols = [(2.55, "Results para 4\n(text, “Table 3”)", ORANGE),
(4.15, "Table 3 (b)\nfirearm, all ages", TXT),
(5.75, "Table 4 (b)\nfirearm, 18–29 y", TXT)]
top = 3.62
for x, h, c in cols:
ax.text(x, top, h, ha="center", va="center", fontsize=10, fontweight="bold", color=c,
linespacing=1.25)
rows = [r for r in P3_ROWS]
dy = 0.39
y0 = top - 0.55
fmt = lambda t: f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})"
for i, (q, r, t3, t4) in enumerate(rows):
y = y0 - i * dy
if i % 2 == 0:
ax.add_patch(FancyBboxPatch((0.12, y - dy / 2 + 0.02), 6.96, dy - 0.04,
boxstyle="round,pad=0,rounding_size=0.04",
fc="#f1f0ec", ec="none", zorder=0))
ax.text(0.2, y, q, va="center", fontsize=10)
ax.text(cols[0][0], y, fmt(r), ha="center", va="center", fontsize=10, color=ORANGE,
fontweight="bold")
for (x, _, _), t in ((cols[1], t3), (cols[2], t4)):
same = all(abs(a - b) <= 0.0101 for a, b in zip(r, t))
ax.text(x, y, fmt(t), ha="center", va="center", fontsize=10,
color=BLUE if same else SEC, fontweight="bold" if same else "normal")
yb = y0 - len(rows) * dy + 0.05
six = [r for r in P3_ROWS if r[0] != "Age (per year)"]
e4 = sum(a == b for _, r, t3, t4 in six for a, b in zip(r, t4))
e3 = sum(a == b for _, r, t3, t4 in six for a, b in zip(r, t3))
ax.text(0.2, yb - 0.08,
"Blue bold = equal to the Results-text value to within 0.01 on all three numbers; grey = differs.\n"
f"Six outcome and race rows: {e4} of 18 numbers equal Table 4 18–29 y, the other {18-e4} within 0.01;\n"
f"{e3} of 18 identical to Table 3. Only the age row (last) matches Table 3, not Table 4 18–29 y.",
fontsize=10, color=SEC, va="top", linespacing=1.35)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
if __name__ == "__main__":
check_quotes()
part1()
part2()
part3()
fig1()
fig2()
print("Figures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Abstract's adjusted odds ratios for education and prenatal care differ from the paper's own Results
Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2026.101434 · PMC12995907
Summary
The Abstract reports that higher maternal education (aOR 2·41, 95% CI 2·35–2·48) and receipt of prenatal care (aOR 2·64, 95% CI 2·56–2·73) were associated with documented adequate partner treatment. The paper's national multivariable model, given in the Results text and in Fig. 2, has different values for the same two associations: 1·48 (1·39–1·59) for higher vs primary education and 3·05 (2·92–3·20) for prenatal care. In both cases the two intervals do not overlap. The race estimates in the same Abstract sentence match the Results exactly. Neither Abstract value appears anywhere else in the paper, and the Abstract's education interval is too narrow to be possible for a higher-vs-primary contrast given the Table 1 counts. Both associations keep their direction and statistical significance, but the Abstract overstates the education association, understates the prenatal-care association, and presents the two as about equal, while the Results show prenatal care as much the stronger of the two.
What the paper reports
Abstract, Findings:
Higher maternal education (aOR 2·41, 95% CI 2·35–2·48) and receipt of prenatal care (aOR 2·64, 95% CI 2·56–2·73) were associated with partner treatment, whereas odds were lower among Black (aOR 0·85, 95% CI 0·82–0·89) and Mixed women (aOR 0·81, 95% CI 0·79–0·84) and in the North region.
Abstract, Methods:
Multivariable logistic regression estimated adjusted odds ratios (aORs) and 95% CIs.
Results, paragraph on the national multivariable model:
Multivariable analysis included 214,259 participants, with 4069 exclusions due to missing maternal age. Prenatal care was the factor most strongly associated with adequate partner treatment (adjusted odds ratio [aOR] 3·05, 95% CI 2·92–3·20; p < 0·001).
Higher maternal education remained positively associated with partner treatment (higher vs primary: aOR 1·48, 1·39–1·59; secondary vs primary: aOR 1·23, 1·20–1·27; both p < 0·001)
Compared with White women, lower odds persisted among Black (aOR 0·85, 0·82–0·89; p < 0·001) and Mixed women (aOR 0·81, 0·79–0·84; p < 0·001)
Fig. 2 caption:
Unadjusted ORs are from separate univariable logistic regression models; aORs are from the multivariable logistic regression model including maternal age, race, education, prenatal care, and macro-region.
Fig. 2 itself is an image. Read from the figure file (gr2.jpg): pre-natal care "Yes", unadjusted 3·33 (3·22 – 3·46) and adjusted 3·05 (2·92 – 3·20); "Higher education", unadjusted 1·85 (1·73 – 1·97) and adjusted 1·48 (1·39 – 1·59). Fig. 3 (gr3.jpg), the region-stratified models, gives prenatal care "Yes" aORs of 2·11, 2·58, 3·70, 4·50 and 4·87 and higher-education aORs of 1·39, 1·72, 1·48, 1·38 and 1·41 (North, Northeast, Central-West, Southeast, South).
Discussion, second paragraph:
Prenatal care emerged as the strongest predictor of partner treatment, with approximately threefold higher odds compared with women without prenatal follow-up.
Table 1 (columns: all cases, n = 218,328 | adequate partner treatment, n = 50,609 | no adequate partner treatment, n = 167,719 | p-value), education and prenatal-care rows:
Primary | 94,267 (43·2%) | 19,367 (38·3%) | 74,900 (44·7%) | <0·001
Higher | 4427 (2·03%) | 1431 (2·83%) | 2996 (1·79%) | <0·001
No | 32,162 (14·7%) | 2627 (5·2%) | 29,535 (17·6%) | <0·001
Yes | 180,009 (82·4%) | 46,943 (92·8%) | 133,066 (79·3%) | <0·001
What the paper's own numbers show
Method. I compared each Abstract estimate with every other estimate of the same association in the paper: the Results text, Fig. 2 (adjusted and unadjusted) and Fig. 3 (five regional models). From the Table 1 counts I computed crude odds ratios with Woolf 95% CIs. From each printed 95% CI I recovered the implied standard error of the log odds ratio, SE = ln(upper/lower)/(2 × 1·96), together with the largest SE the two-decimal rounding allows. I then compared those SEs with a lower bound that holds for any maximum-likelihood logistic model fitted to these cases with model-based standard errors. When the model contains the levels of a categorical variable, the fitted probabilities within each level sum to the observed number of events in that level. As a result, the variance of the log odds ratio between two levels can be no smaller than the crude Woolf variance 1/a + 1/b + 1/c + 1/d computed from the level counts, whatever other covariates are in the model. Any subset of the cases, such as the n = 214,259 model sample or a single region, has smaller counts, so the bound only increases. Finally I searched every merge of levels of the model variables in Table 1 for a crude OR near 2·41 or 2·64. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Higher education, adjusted | Abstract | 2·41 (2·35–2·48) | Appears nowhere else in the paper |
| Higher vs primary education, adjusted | Results text; Fig. 2 | 1·48 (1·39–1·59) | Does not overlap the Abstract interval; the Abstract OR is 1·63 times larger (2·24 times on the log scale) |
| Higher vs primary education, crude | Fig. 2 (unadjusted); Table 1 counts | 1·85 (1·73–1·97) | 1·85 (1·73–1·97) from 1431/2996 vs 19,367/74,900, matching Fig. 2, so the Abstract value is not the crude OR either |
| SE(log OR) implied by the education interval | Abstract | 2·35–2·48 | 0·0137 (at most 0·0148 allowing for rounding) |
| SE(log OR) implied by the education interval | Results; Fig. 2 | 1·39–1·59 | 0·0343 |
| Smallest SE(log OR) possible for higher vs primary | Table 1 counts | — | 0·0331. The Abstract interval is 0·41 times this. Even counting only the 1431 treated women with higher education, SE ≥ 0·0264; the Abstract interval would need at least 4570 such women |
| Prenatal care, adjusted | Abstract | 2·64 (2·56–2·73) | Appears nowhere else in the paper |
| Prenatal care yes vs no, adjusted | Results text; Fig. 2 | 3·05 (2·92–3·20) | Does not overlap the Abstract interval; the Abstract OR is 0·87 times the Results value |
| Prenatal care yes vs no, crude | Table 1 counts | — | 3·97 (3·81–4·13) |
| SE(log OR) implied by the prenatal interval | Abstract | 2·56–2·73 | 0·0164 (at most 0·0174 allowing for rounding) |
| SE(log OR) implied by the prenatal interval | Results; Fig. 2 | 2·92–3·20 | 0·0234 |
| Smallest SE(log OR) possible for prenatal yes vs no | Table 1 counts | — | 0·0211. The Abstract interval is 0·78 times this. Even pooling "Ignored/Unknown" with "No", the bound is 0·0182 |
| Regional aORs, prenatal care | Fig. 3 | 2·11, 2·58, 3·70, 4·50, 4·87 | None is 2·64. The closest, Northeast 2·58 (2·37–2·80), has SE 0·0425, about 2.6 times the Abstract's |
| Regional aORs, higher education | Fig. 3 | 1·39, 1·72, 1·48, 1·38, 1·41 | None is near 2·41 |
| Black and Mixed vs White, adjusted | Abstract vs Results | 0·85 (0·82–0·89) and 0·81 (0·79–0·84) in both | Identical, so the Abstract sentence uses the national model for race |
| Relative strength, log OR of prenatal care ÷ log OR of higher education | Abstract vs Results | 2·64 vs 2·41; 3·05 vs 1·48 | 1·10 in the Abstract, 2·84 in the Results |
| Crude OR near 2·41 or 2·64 from any merge of levels | Table 1 (race, region, education, prenatal care) | — | None within 0·06 |
Table 1 is internally consistent: every variable sums to 50,609 treated and 167,719 untreated cases. The Results and Fig. 2 intervals are compatible with the bound (1·04 and 1·11 times it), as they should be. The Abstract intervals are not.
One smaller observation is separate from the claim. The Fig. 2 unadjusted OR for prenatal care "Yes", 3·33 (3·22–3·46), is not the Table 1 crude OR for yes vs no (3·97). It is reproduced exactly by yes vs "No" and "Ignored/Unknown" pooled: 3·33 (3·22–3·46). The "Ignored/Unknown" row in Fig. 2, 2·28 (2·11–2·47), is reproduced by ignored vs no alone. This suggests that the unadjusted "Yes" estimate may use a different reference group from the rest of the block. It does not affect the adjusted estimates.
Figure
Figure 1. Short excerpts from the Abstract (Findings, left) and from the Results text, whose values Fig. 2 repeats (right), for the four associations the Abstract quantifies. Orange arrows join two reported values of the same association that disagree: higher education 2·41 (2·35–2·48) vs 1·48 (1·39–1·59), and prenatal care 2·64 (2·56–2·73) vs 3·05 (2·92–3·20). In each case the intervals do not overlap. Dashed gray lines join values that are identical: Black women 0·85 (0·82–0·89) and Mixed women 0·81 (0·79–0·84). The excerpts are shortened; the full sentences are quoted above.
Figure 2. The standard error of the log odds ratio implied by each printed 95% CI, for higher vs primary education (top) and prenatal care yes vs no (bottom), on one linear axis that starts at zero. Orange diamonds are the Abstract intervals and hollow orange circles the Results / Fig. 2 intervals. The short orange bars show the range of SEs consistent with two-decimal rounding of the printed limits. The blue line in each group is the smallest SE any logistic model fitted to these cases can have, computed from the Table 1 counts (0·0331 and 0·0211). The shaded area to its left is unattainable. Both Results intervals lie just above the bound, and both Abstract intervals lie inside the unattainable area, the education interval at less than half the bound.
Why it matters
The Abstract is the most-read part of the paper and the only place, apart from the Results text and figures, where the paper puts numbers on its main associations. The title, the Research in context panel and the Conclusions describe these associations without numbers. In the Abstract:
- The education association is overstated. 2·41 is 63% larger than the Results' 1·48, and 2·24 times larger on the log-odds scale. The Abstract's interval (2·35–2·48) also suggests a precision that the 4427 women with higher education cannot provide.
- The prenatal-care association is understated. 2·64 is 13% smaller than 3·05. The Discussion's statement of "approximately threefold higher odds" matches the Results, not the Abstract.
- The relative ranking is altered. The Abstract presents education and prenatal care as roughly equal (log-OR ratio 1·10). In the Results, prenatal care is the strongest predictor, with a log OR 2·84 times that of higher education. The Discussion states that "public health prioritization should be guided primarily by effect size", so the size and ranking of these two associations are what a reader would act on.
The direction and statistical significance of both associations are the same in the Abstract and the Results. So are the paper's qualitative conclusions: partner treatment is low, it is more likely with prenatal care and higher education, and it is less likely among Black and Mixed women. The race estimates in the Abstract are correct. The error is in the reported size of two of the four quantified associations in the Abstract.
Explanations considered
- Rounding or typesetting of the same estimate. This does not account for the numbers. The intervals do not overlap for either association, and the differences (2·41 vs 1·48; 2·64 vs 3·05) are far larger than rounding. The publisher-supplied full text archived in PMC and Europe PMC has the same values, so a conversion artifact is excluded.
- The Abstract gives unadjusted odds ratios. This does not account for the numbers. The unadjusted ORs are 1·85 for higher education (Fig. 2 and Table 1) and 3·33 (Fig. 2) or 3·97 (Table 1) for prenatal care. The Abstract also labels its values "aOR".
- The Abstract gives a regional estimate. This does not account for the numbers. None of the five regional aORs in Fig. 3 equals 2·41 or 2·64, and every regional interval is far wider than the Abstract's. The SE bound also applies to any region, since each is a subset of the cases.
- The Abstract uses a different contrast, such as secondary-or-higher vs primary education. The paper does not describe one. An interval as narrow as 2·35–2·48 is possible only for a contrast between large groups, such as secondary vs primary (bound 0·0119). The paper reports that contrast as 1·23 (1·20–1·27), and the crude OR for secondary-or-higher vs primary is 1·47. For prenatal care, even pooling "Ignored/Unknown" with "No" gives a bound (0·0182) above the Abstract's largest possible SE (0·0174). Among the model variables in Table 1, no merge of levels gives a crude OR within 0·06 of either value. Arbitrary merges of the timing-of-diagnosis variable, which is not in the model, come within 0·06. All of them have SE of at least 0·0189, so they cannot produce the Abstract intervals either.
- The Abstract comes from a different model or an earlier version of the analysis. The paper gives no information on this, so it cannot be checked. Any logistic model fitted to these 218,328 cases, or to a subset of them, with model-based standard errors is excluded by the SE bound for the education interval, and also for the prenatal interval. A model fitted to a larger set of cases is not excluded by the bound. Such a model would have to count more women with documented partner treatment than the 50,609 in Table 1 (for example, at least 4570 treated women with higher education rather than 1431). The paper's outcome excludes the 78,734 records with missing partner-treatment status. The paper does not state how its CIs were computed. Robust (sandwich) standard errors are not mentioned, and the Results intervals are consistent with model-based ones.
- The Abstract values are correct and the Results and Fig. 2 are wrong. This is unlikely. Fig. 2, the Results text and the Discussion agree with each other. The Results and Fig. 2 intervals are compatible with the Table 1 counts, and the Abstract's education interval is not.
Status
- Version of record. The publisher's own pages could not be read: doi.org redirected to linkinghub.elsevier.com, which returned only a redirect stub, and thelancet.com and sciencedirect.com returned HTTP 403. The same text was checked in three archived copies of the publisher's full text: the PMC article page, the Europe PMC full-text XML (publisher-supplied JATS), and the local PMC archive. All three carry the Abstract values 2·41 (2·35–2·48) and 2·64 (2·56–2·73) and the Results values 3·05 (2·92–3·20) and 1·48 (1·39–1·59). In the text, 2·41 and 2·64 occur only in the Abstract. Every quote above was checked verbatim against the PMC-archived text, except the figure values, which were read from the figure images.
- Figures and supplements. I read the figure images Fig. 2 (gr2.jpg) and Fig. 3 (gr3.jpg) from the Europe PMC supplementary-files bundle. Neither contains 2·41 (2·35–2·48) or 2·64 (2·56–2·73). The only supplement, mmc1.pdf, holds Supplementary Table 1 (variable definitions and SINAN field codes) and Supplementary Table 2 (included vs excluded cases), with no regression output; it does not resolve the discrepancy. No code or analysis dataset is deposited. The data are public SINAN/SINASC records, but refitting the model was out of scope.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI (update-to and updated-by absent, relation empty). PMC and Europe PMC list no correction or comment links (PMID 41858869), and the PMC page carries only the standard editorial note on maps and affiliations. The PubPeer API was blocked by an automated-access challenge, so PubPeer could not be checked directly; a web search for the title together with correction, erratum or PubPeer found nothing.
- Sources. PMC article PMC12995907; Europe PMC full-text XML and supplementary files (gr2.jpg, gr3.jpg, mmc1.pdf); Crossref API; Europe PMC search; PubPeer; general web search.
Suggested correction
- Replace the Abstract's education and prenatal-care estimates with the values from the national multivariable model, stating the education contrast explicitly. For example: "Higher maternal education (higher vs primary: aOR 1·48, 95% CI 1·39–1·59) and receipt of prenatal care (aOR 3·05, 95% CI 2·92–3·20) were associated with partner treatment".
- If 2·41 (2·35–2·48) and 2·64 (2·56–2·73) come from another analysis, identify that analysis (model, sample, contrast and CI method) in the Methods and Results, and explain why the Abstract reports it instead of the main model.
- Minor, separate: check the reference group of the Fig. 2 unadjusted OR for prenatal care "Yes" (3·33), which matches yes vs "No" and "Ignored/Unknown" pooled rather than yes vs no (3·97), and state the reference in the figure if it differs from the adjusted estimate's.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12995907 (Lancet Reg Health Am 2026; doi:10.1016/j.lana.2026.101434)
# Abstract aORs for education and prenatal care vs the paper's own Results, Fig. 2, Fig. 3 and Table 1.
# Run: python recompute.py.txt (needs numpy + matplotlib)
# Prints every recomputed number used in writeup.md and writes fig1/fig2 (.svg, .png) and
# five candidate figure designs (candidates/cand1..cand5 .svg/.png) into this file's folder.
import math, itertools, os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyBboxPatch
from matplotlib.transforms import blended_transform_factory
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = 1.959964
# ---------------------------------------------------------------- inputs from the paper
# Table 1 counts: (adequate partner treatment, no adequate partner treatment)
T = {
"race": {"White": (14578, 39887), "Black": (4429, 16105), "Asian": (181, 584),
"Mixed": (27822, 98612), "Indigenous": (213, 544), "Ignored": (3386, 11987)},
"region": {"North": (4196, 15681), "Northeast": (14302, 50668), "Southeast": (20359, 71759),
"South": (7788, 20561), "Center-west": (3964, 9050)},
"edu": {"Primary": (19367, 74900), "Secondary": (17804, 47544), "Higher": (1431, 2996),
"Ignored": (12007, 42279)},
"pnc": {"No": (2627, 29535), "Yes": (46943, 133066), "Ignored": (1039, 5118)},
"timing": {"During prenatal care": (38593, 83773), "At delivery/curettage": (8290, 64508),
"After delivery": (2243, 13514), "Not performed": (217, 1286), "Ignored": (1266, 4638)},
}
N_TREATED, N_UNTREATED = 50609, 167719
# Reported estimates: (point, lo, hi)
ABS_EDU, ABS_PNC = (2.41, 2.35, 2.48), (2.64, 2.56, 2.73) # Abstract, Findings
RES_EDU, RES_PNC = (1.48, 1.39, 1.59), (3.05, 2.92, 3.20) # Results p5 and Fig. 2 (aOR)
RES_SEC = (1.23, 1.20, 1.27) # Results p5, secondary vs primary
FIG2_UNADJ_EDU, FIG2_UNADJ_PNC = (1.85, 1.73, 1.97), (3.33, 3.22, 3.46) # Fig. 2 unadjusted OR
FIG2_UNADJ_PNC_IGN = (2.28, 2.11, 2.47) # Fig. 2 unadjusted, PNC ignored vs no
ABS_RACE = {"Black": (0.85, 0.82, 0.89), "Mixed": (0.81, 0.79, 0.84)}
RES_RACE = {"Black": (0.85, 0.82, 0.89), "Mixed": (0.81, 0.79, 0.84)}
REGIONS = ["North", "Northeast", "Central-West", "Southeast", "South"] # Fig. 3 order
FIG3_PNC = [(2.11, 1.86, 2.40), (2.58, 2.37, 2.80), (3.70, 3.05, 4.48), (4.50, 4.11, 4.94), (4.87, 4.21, 5.62)]
FIG3_EDU = [(1.39, 1.12, 1.72), (1.72, 1.49, 1.99), (1.48, 1.16, 1.87), (1.38, 1.23, 1.55), (1.41, 1.21, 1.64)]
def orci(a, b, c, d):
"""Crude OR of (a treated, b untreated) vs reference (c, d), Woolf 95% CI and SE(log OR)."""
o = (a / b) / (c / d)
se = math.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
return o, o * math.exp(-Z * se), o * math.exp(Z * se), se
def se_ci(lo, hi):
return math.log(hi / lo) / (2 * Z)
def se_ci_maxround(lo, hi, dp=2):
"""Largest SE consistent with CI limits printed to dp decimals."""
h = 0.5 * 10 ** (-dp)
return math.log((hi + h) / (lo - h)) / (2 * Z)
def f(x): return f"{x:.2f}"
print("=" * 78)
print("1. Table 1 column totals")
for v, d in T.items():
print(f" {v:7s}: treated {sum(x[0] for x in d.values()):6d} (paper {N_TREATED}), "
f"untreated {sum(x[1] for x in d.values()):6d} (paper {N_UNTREATED})")
print("\n2. Crude ORs from Table 1 counts (Woolf 95% CI)")
crude = {}
pairs = {
"Higher vs Primary": (T["edu"]["Higher"], T["edu"]["Primary"]),
"Secondary vs Primary": (T["edu"]["Secondary"], T["edu"]["Primary"]),
"Edu Ignored vs Primary": (T["edu"]["Ignored"], T["edu"]["Primary"]),
"Secondary+Higher vs Primary": ((T["edu"]["Secondary"][0] + T["edu"]["Higher"][0],
T["edu"]["Secondary"][1] + T["edu"]["Higher"][1]), T["edu"]["Primary"]),
"PNC Yes vs No": (T["pnc"]["Yes"], T["pnc"]["No"]),
"PNC Ignored vs No": (T["pnc"]["Ignored"], T["pnc"]["No"]),
"PNC Yes vs (No+Ignored)": (T["pnc"]["Yes"], (T["pnc"]["No"][0] + T["pnc"]["Ignored"][0],
T["pnc"]["No"][1] + T["pnc"]["Ignored"][1])),
"Black vs White": (T["race"]["Black"], T["race"]["White"]),
"Mixed vs White": (T["race"]["Mixed"], T["race"]["White"]),
}
for k, (x, r) in pairs.items():
crude[k] = orci(*x, *r)
o, lo, hi, se = crude[k]
print(f" {k:28s} OR {o:.3f} ({lo:.2f}-{hi:.2f}) SE(log OR) {se:.4f}")
print(" Fig. 2 unadjusted: higher edu 1.85 (1.73-1.97); PNC yes 3.33 (3.22-3.46); PNC ignored 2.28 (2.11-2.47)")
print(" -> Fig. 2's unadjusted 'Yes' OR equals Yes vs (No+Ignored), not Yes vs No (side observation)")
print("\n3. SE(log OR) implied by each reported 95% CI (and largest SE allowed by 2-dp rounding)")
rep = [("Abstract, higher education", ABS_EDU), ("Results/Fig. 2, higher vs primary", RES_EDU),
("Abstract, prenatal care", ABS_PNC), ("Results/Fig. 2, prenatal yes vs no", RES_PNC),
("Results, secondary vs primary", RES_SEC), ("Fig. 2 unadjusted, higher edu", FIG2_UNADJ_EDU),
("Fig. 2 unadjusted, prenatal yes", FIG2_UNADJ_PNC)]
for lab, (p, lo, hi) in rep:
print(f" {lab:36s} {p:.2f} ({lo:.2f}-{hi:.2f}): SE {se_ci(lo, hi):.4f} "
f"(max with rounding {se_ci_maxround(lo, hi):.4f}); geometric midpoint {math.sqrt(lo*hi):.3f}")
print("\n4. Lower bounds on SE(log OR) for any logistic model fitted to these cases")
# For a logistic model that contains level indicators for the variable, the score equations make the
# fitted probabilities in each level sum to the observed events there, so the information carried by
# level A is at most n_A*pbar*(1-pbar) = 1/(1/a + 1/b). By Cauchy-Schwarz on the inverse information,
# Var(beta_A - beta_B) >= 1/I_A + 1/I_B >= (1/a + 1/b + 1/c + 1/d): the crude Woolf variance.
# Any subset of the 218,328 cases (the n = 214,259 model sample, a single region) has smaller
# counts, so the bound only grows. Weaker but more general: Var >= 1/a, a = treated cases in level.
b_edu = crude["Higher vs Primary"][3]
b_pnc = crude["PNC Yes vs No"][3]
b_edu_a = math.sqrt(1 / T["edu"]["Higher"][0])
b_pnc_a = math.sqrt(1 / T["pnc"]["No"][0])
print(f" Higher vs primary: SE >= {b_edu:.4f} (crude Woolf); SE >= {b_edu_a:.4f} from 1431 treated alone")
print(f" Prenatal yes vs no: SE >= {b_pnc:.4f} (crude Woolf); SE >= {b_pnc_a:.4f} from 2627 treated (No) alone")
se_ae, se_ap = se_ci(*ABS_EDU[1:]), se_ci(*ABS_PNC[1:])
se_re, se_rp = se_ci(*RES_EDU[1:]), se_ci(*RES_PNC[1:])
print(f" Abstract education SE {se_ae:.4f} = {se_ae/b_edu:.2f} x the bound; Results {se_re:.4f} = {se_re/b_edu:.2f} x")
print(f" Abstract prenatal SE {se_ap:.4f} = {se_ap/b_pnc:.2f} x the bound; Results {se_rp:.4f} = {se_rp/b_pnc:.2f} x")
need_a = 1 / se_ci_maxround(*ABS_EDU[1:]) ** 2
print(f" To allow the Abstract education CI, the higher-education level would need >= {need_a:.0f} "
f"treated cases (Table 1: 1431)")
need_p = 1 / se_ci_maxround(*ABS_PNC[1:]) ** 2
print(f" To allow the Abstract prenatal CI, the prenatal 'No' level would need >= {need_p:.0f} "
f"treated cases (Table 1: 2627)")
print(" Fig. 2 unadjusted prenatal SE "
f"{se_ci(*FIG2_UNADJ_PNC[1:]):.4f} vs bound for Yes vs (No+Ignored) {crude['PNC Yes vs (No+Ignored)'][3]:.4f}")
print("\n5. Size of the discrepancies")
for lab, a, r in [("Education", ABS_EDU, RES_EDU), ("Prenatal", ABS_PNC, RES_PNC)]:
print(f" {lab}: Abstract/Results OR ratio {a[0]/r[0]:.3f}; log OR {math.log(a[0]):.3f} vs "
f"{math.log(r[0]):.3f} (ratio {math.log(a[0])/math.log(r[0]):.2f}); CIs overlap: {a[1] <= r[2] and r[1] <= a[2]}")
print(f" Abstract: prenatal/education log-OR ratio {math.log(ABS_PNC[0])/math.log(ABS_EDU[0]):.2f}; "
f"Results: {math.log(RES_PNC[0])/math.log(RES_EDU[0]):.2f}")
for k in ABS_RACE:
print(f" {k}: Abstract {ABS_RACE[k]} vs Results {RES_RACE[k]} -> identical: {ABS_RACE[k] == RES_RACE[k]}")
print("\n6. Does 2.41 or 2.64 match any other estimate in the paper?")
for lab, vals in [("Fig. 3 prenatal", FIG3_PNC), ("Fig. 3 higher education", FIG3_EDU)]:
for reg, (p, lo, hi) in zip(REGIONS, vals):
print(f" {lab:24s} {reg:12s} {p:.2f} ({lo:.2f}-{hi:.2f}) SE {se_ci(lo, hi):.4f}")
def search(vars_):
hits = []
for v in vars_:
d = T[v]; keys = list(d)
for r1 in range(1, len(keys)):
for A in itertools.combinations(keys, r1):
rest = [k for k in keys if k not in A]
for r2 in range(1, len(rest) + 1):
for B in itertools.combinations(rest, r2):
a = sum(d[k][0] for k in A); b = sum(d[k][1] for k in A)
c = sum(d[k][0] for k in B); dd = sum(d[k][1] for k in B)
for (o, lo, hi, se), lab in ((orci(a, b, c, dd), (A, B)), (orci(c, dd, a, b), (B, A))):
for tgt in (2.41, 2.64):
if abs(o - tgt) < 0.06:
hits.append((v, " + ".join(lab[0]), "vs", " + ".join(lab[1]), round(o, 3), round(se, 4)))
return list(dict.fromkeys(hits))
h_model = search(["race", "region", "edu", "pnc"])
print(" Exhaustive crude-OR search over every merge of levels of the model variables in Table 1 (race, region,"
f" education, prenatal care), both directions, within 0.06 of 2.41 or 2.64: {h_model if h_model else 'none'}")
h_tim = search(["timing"])
print(f" Same search on timing of diagnosis (not a model covariate): {len(h_tim)} hits, all arbitrary merges, all with SE(log OR) >= 0.0189:")
for h in h_tim:
print(" ", h)
o, lo, hi, se = orci(*T["timing"]["During prenatal care"],
sum(v[0] for k, v in T["timing"].items() if k != "During prenatal care"),
sum(v[1] for k, v in T["timing"].items() if k != "During prenatal care"))
print(f" Timing, diagnosis during prenatal care vs all others: OR {o:.2f} ({lo:.2f}-{hi:.2f})")
# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
LW, MS = 1.5, 8
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK, "text.color": INK, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg")
fig.savefig(p + ".png", dpi=200)
plt.close(fig)
def style_axis(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
def fmt(t): # 2.41 (2.35–2.48) with the paper's middle-dot decimals
return f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})".replace(".", "·")
# ---------------------------------------------------------------- candidate 1: dot-and-interval
def cand_dot_interval(paths):
ce, cp = crude["Higher vs Primary"], crude["PNC Yes vs No"]
groups = [
("Higher vs primary education", [
("Abstract aOR", ABS_EDU, "abs"), ("Results text and Fig. 2 aOR", RES_EDU, "body"),
("Fig. 2 unadjusted OR", FIG2_UNADJ_EDU, "body"),
("Crude OR from Table 1 counts", ce[:3], "rec")]),
("Prenatal care, yes vs no", [
("Abstract aOR", ABS_PNC, "abs"), ("Results text and Fig. 2 aOR", RES_PNC, "body"),
("Fig. 2 unadjusted OR", FIG2_UNADJ_PNC, "body"),
("Crude OR from Table 1 counts", cp[:3], "rec")]),
]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.subplots_adjust(left=0.31, right=0.98, top=0.87, bottom=0.12)
y, yt, yl = 0, [], []
for gname, rows in groups:
ax.text(0.01, y + 0.15, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
ha="left", va="center", fontsize=10.5, fontweight="bold")
y -= 1
for lab, (p, lo, hi), kind in rows:
col = BLUE if kind == "rec" else ORANGE
mfc = BG if kind == "body" else col
mk = "s" if kind == "rec" else ("D" if kind == "abs" else "o")
ax.plot([lo, hi], [y, y], color=col, lw=LW, solid_capstyle="butt")
ax.plot(p, y, marker=mk, ms=MS, mfc=mfc, mec=col, mew=1.5, color=col)
ax.text(hi * 1.03, y, fmt((p, lo, hi)), va="center", fontsize=10, color=INK)
yt.append(y); yl.append(lab)
y -= 1
y -= 0.6
ax.axvline(1, color=INK2, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.9, 6.5)
ticks = [1, 1.5, 2, 3, 4, 5]
ax.set_xticks(ticks); ax.set_xticklabels([str(t).replace(".", "·") for t in ticks])
ax.minorticks_off()
ax.set_yticks(yt); ax.set_yticklabels(yl, fontsize=10)
ax.set_ylim(y + 0.6, 0.8)
ax.set_xlabel("Odds ratio (log scale); vertical line at 1 = no association")
style_axis(ax)
leg = [Line2D([], [], marker="D", color=ORANGE, mfc=ORANGE, ms=MS, lw=LW, label="Reported in the Abstract"),
Line2D([], [], marker="o", color=ORANGE, mfc=BG, mec=ORANGE, ms=MS, lw=LW, label="Reported in Results / Fig. 2"),
Line2D([], [], marker="s", color=BLUE, mfc=BLUE, ms=MS, lw=LW, label="Recomputed from Table 1")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False,
fontsize=10, handlelength=2.0, columnspacing=1.0)
save(fig, *paths)
# ---------------------------------------------------------------- candidate 2: two text panels
def cand_text_panels(paths):
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")
L0, L1, R0, R1 = 0.05, 2.6, 4.6, 7.15
top, bot = 4.05, 0.35
for x0, x1, title, sub in [(L0, L1, "Abstract", "Findings"),
(R0, R1, "Results text and Fig. 2", "national model, n = 214,259")]:
ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot, boxstyle="round,pad=0,rounding_size=0.06",
fc="#ffffff", ec=INK2, lw=1.0))
ax.text(x0 + 0.14, top + 0.33, title, fontsize=11.5, fontweight="bold", va="center")
ax.text(x0 + 0.14, top + 0.1, sub, fontsize=10, color=INK2, va="center")
rows = [
("Higher maternal education", "aOR 2·41, 95% CI 2·35–2·48",
"Higher vs primary education", "aOR 1·48, 1·39–1·59", "conflict"),
("Receipt of prenatal care", "aOR 2·64, 95% CI 2·56–2·73",
"Prenatal care (yes vs no)", "aOR 3·05, 95% CI 2·92–3·20", "conflict"),
("Black women", "aOR 0·85, 95% CI 0·82–0·89",
"Black vs White women", "aOR 0·85, 0·82–0·89", "same"),
("Mixed women", "aOR 0·81, 95% CI 0·79–0·84",
"Mixed vs White women", "aOR 0·81, 0·79–0·84", "same"),
]
ys = [3.55, 2.65, 1.65, 0.85]
notes = {0: ["Intervals do not overlap", "Abstract CI is narrower than", "any model on these cases allows"],
1: ["Intervals do not overlap", "Abstract CI is narrower than", "any model on these cases allows"]}
for i, ((ll, lv, rl, rv, kind), yy) in enumerate(zip(rows, ys)):
for x0, lab, val in [(L0, ll, lv), (R0, rl, rv)]:
ax.text(x0 + 0.14, yy + 0.15, lab, fontsize=10, color=INK2, va="center")
ax.text(x0 + 0.14, yy - 0.13, val, fontsize=10.5, fontweight="bold" if kind == "conflict" else "normal",
va="center")
if kind == "conflict":
ax.annotate("", xy=(R0 - 0.04, yy), xytext=(L1 + 0.04, yy),
arrowprops=dict(arrowstyle="<->", color=ORANGE, lw=LW))
ax.text((L1 + R0) / 2, yy + 0.2, "≠", ha="center", va="center", fontsize=16, color=INK)
ax.text((L1 + R0) / 2, yy - 0.2, "\n".join(notes[i][:1]), ha="center", va="top", fontsize=10, color=INK)
else:
ax.plot([L1 + 0.04, R0 - 0.04], [yy, yy], color=INK2, lw=1.0, ls=(0, (3, 2)))
ax.text((L1 + R0) / 2, yy + 0.14, "identical", ha="center", va="bottom", fontsize=10, color=INK2)
ax.plot([L0 + 0.14, R1 - 0.14], [2.12, 2.12], color=GRID, lw=1.0)
ax.text(3.6, 0.12, "Orange arrows: the two reported values of one estimate disagree. "
"Dashed gray: the values agree.", ha="center", va="center", fontsize=10, color=INK2)
save(fig, *paths)
# ---------------------------------------------------------------- candidate 3: interval width vs bound
def cand_se_bound(paths):
rows = [("Higher vs primary education", ABS_EDU, RES_EDU, b_edu),
("Prenatal care, yes vs no", ABS_PNC, RES_PNC, b_pnc)]
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.subplots_adjust(left=0.33, right=0.97, top=0.86, bottom=0.13)
yt, yl = [], []
y = 0
for gname, a, r, bnd in rows:
ax.text(0.01, y + 0.75, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
ha="left", va="center", fontsize=10.5, fontweight="bold")
ax.fill_betweenx([y - 1.45, y + 0.35], 0, bnd, color="#eeedea", lw=0, zorder=0)
ax.plot([bnd, bnd], [y - 1.45, y + 0.35], color=BLUE, lw=LW, zorder=2)
ax.text(bnd, y + 0.42, f"smallest SE possible: {bnd:.4f}".replace(".", "·"), color=INK,
fontsize=10, va="bottom", ha="center")
for lab, t, filled, dy in [("Abstract", a, True, 0), ("Results / Fig. 2", r, False, -1)]:
s, smax = se_ci(*t[1:]), se_ci_maxround(*t[1:])
smin = math.log((t[2] - 0.005) / (t[1] + 0.005)) / (2 * Z)
ax.plot([smin, smax], [y + dy, y + dy], color=ORANGE, lw=LW)
ax.plot(s, y + dy, "D" if filled else "o", ms=MS, color=ORANGE, mfc=ORANGE if filled else BG, mew=1.5)
ax.text(smax + 0.0006, y + dy, f"{s:.4f}".replace(".", "·"), va="center", fontsize=10)
yt.append(y + dy); yl.append(f"{lab}, CI " + f"{t[1]:.2f}–{t[2]:.2f}".replace(".", "·"))
ax.text(0.0005, y - 1.42, "below the bound", fontsize=10, color=INK2, va="bottom")
y -= 2.6
ax.set_xlim(0, 0.05)
ax.set_ylim(y + 0.9, 1.25)
ax.set_yticks(yt); ax.set_yticklabels(yl)
xt = [0, 0.01, 0.02, 0.03, 0.04, 0.05]
ax.set_xticks(xt); ax.set_xticklabels([f"{v:.2f}".replace(".", "·") for v in xt])
ax.set_xlabel("Standard error of log(odds ratio) implied by the printed 95% CI")
style_axis(ax)
leg = [Line2D([], [], marker="D", color=ORANGE, mfc=ORANGE, ms=MS, lw=LW, label="Abstract (bar: rounding range)"),
Line2D([], [], marker="o", color=ORANGE, mfc=BG, ms=MS, lw=LW, label="Results / Fig. 2"),
Line2D([], [], color=BLUE, lw=LW, label="Bound from Table 1 counts")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10,
handlelength=1.8, columnspacing=1.0)
save(fig, *paths)
# ---------------------------------------------------------------- candidate 4: every estimate in the paper
def cand_all_estimates(paths):
ce, cp = crude["Higher vs Primary"], crude["PNC Yes vs No"]
groups = [
("Higher vs primary education", ABS_EDU,
[("Results / Fig. 2 aOR", RES_EDU), ("Fig. 2 unadjusted OR", FIG2_UNADJ_EDU)] +
[(f"Fig. 3 aOR, {r}", t) for r, t in zip(REGIONS, FIG3_EDU)], ce[:3]),
("Prenatal care, yes vs no", ABS_PNC,
[("Results / Fig. 2 aOR", RES_PNC), ("Fig. 2 unadjusted OR", FIG2_UNADJ_PNC)] +
[(f"Fig. 3 aOR, {r}", t) for r, t in zip(REGIONS, FIG3_PNC)], cp[:3]),
]
fig, ax = plt.subplots(figsize=(7.2, 6.6))
fig.subplots_adjust(left=0.32, right=0.78, top=0.93, bottom=0.08)
y, yt, yl = 0, [], []
for gname, a, body, rc in groups:
ax.text(0.01, y + 0.1, gname, transform=blended_transform_factory(fig.transFigure, ax.transData),
ha="left", va="center", fontsize=10.5, fontweight="bold")
y -= 1
y0 = y
rows = [("Abstract aOR", a, "abs")] + [(l, t, "body") for l, t in body] + [("Crude OR, Table 1 counts", rc, "rec")]
for lab, (p, lo, hi), kind in rows:
col = BLUE if kind == "rec" else ORANGE
mk = {"abs": "D", "body": "o", "rec": "s"}[kind]
ax.plot([lo, hi], [y, y], color=col, lw=LW)
ax.plot(p, y, mk, ms=MS, color=col, mfc=BG if kind == "body" else col, mew=1.5)
ax.text(1.02, y, fmt((p, lo, hi)), va="center", ha="left", fontsize=10, transform=ax.get_yaxis_transform())
yt.append(y); yl.append(lab); y -= 1
ax.plot([a[0], a[0]], [y + 0.6, y0 + 0.4], color=ORANGE, lw=0.8, ls=(0, (3, 2)), zorder=0)
y -= 0.6
ax.axvline(1, color=INK2, lw=0.8)
ax.set_xscale("log"); ax.set_xlim(0.9, 6.0)
ticks = [1, 1.5, 2, 3, 4, 5]
ax.set_xticks(ticks); ax.set_xticklabels([str(t).replace(".", "·") for t in ticks]); ax.minorticks_off()
ax.set_yticks(yt); ax.set_yticklabels(yl)
ax.set_ylim(y + 0.6, 0.6)
ax.set_xlabel("Odds ratio (log scale); dashed line = Abstract value")
style_axis(ax)
leg = [Line2D([], [], marker="D", color=ORANGE, mfc=ORANGE, ms=MS, lw=LW, label="Abstract"),
Line2D([], [], marker="o", color=ORANGE, mfc=BG, ms=MS, lw=LW, label="Results, Fig. 2, Fig. 3"),
Line2D([], [], marker="s", color=BLUE, mfc=BLUE, ms=MS, lw=LW, label="Recomputed from Table 1")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=3, frameon=False, fontsize=10)
save(fig, *paths)
# ---------------------------------------------------------------- candidate 5: slope chart Abstract -> Results
def cand_slope(paths):
items = [("Prenatal care", ABS_PNC, RES_PNC), ("Higher education", ABS_EDU, RES_EDU),
("Black women", ABS_RACE["Black"], RES_RACE["Black"]),
("Mixed women", ABS_RACE["Mixed"], RES_RACE["Mixed"])]
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.subplots_adjust(left=0.1, right=0.98, top=0.88, bottom=0.1)
xa, xr = 0, 1
dy_a = {"Prenatal care": 1.035, "Higher education": 0.965, "Black women": 1.02, "Mixed women": 0.98}
dy_r = {"Prenatal care": 1.0, "Higher education": 1.0, "Black women": 1.02, "Mixed women": 0.98}
for lab, a, r in items:
changed = a != r
col = ORANGE if changed else INK2
ax.plot([xa, xr], [a[0], r[0]], color=col, lw=LW, ls="-" if changed else (0, (3, 2)))
for x, t in [(xa, a), (xr, r)]:
ax.plot([x, x], [t[1], t[2]], color=col, lw=LW + 1.5, solid_capstyle="butt")
ax.plot(x, t[0], "o", ms=MS, color=col, mfc=col if changed else BG, mew=1.5)
ax.text(xa - 0.06, a[0] * dy_a[lab], f"{lab} {a[0]:.2f}".replace(".", "·"), ha="right", va="center", fontsize=10)
ax.text(xr + 0.06, r[0] * dy_r[lab], f"{r[0]:.2f} {lab}".replace(".", "·"), ha="left", va="center", fontsize=10)
ax.set_yscale("log"); ax.set_ylim(0.7, 3.6)
yt = [0.75, 1, 1.5, 2, 2.5, 3, 3.5]
ax.set_yticks(yt); ax.set_yticklabels([str(v).replace(".", "·") for v in yt]); ax.minorticks_off()
ax.axhline(1, color=INK2, lw=0.8)
ax.set_xlim(-0.95, 1.95)
ax.set_xticks([xa, xr]); ax.set_xticklabels(["Abstract", "Results text / Fig. 2"], fontsize=10.5, fontweight="bold")
ax.set_ylabel("Adjusted odds ratio (log scale)")
for s in ("top", "right", "bottom"):
ax.spines[s].set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.tick_params(axis="x", length=0)
leg = [Line2D([], [], color=ORANGE, lw=LW, marker="o", ms=MS, label="Reported values differ"),
Line2D([], [], color=INK2, lw=LW, ls=(0, (3, 2)), marker="o", mfc=BG, ms=MS, label="Reported values identical")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=2, frameon=False, fontsize=10)
save(fig, *paths)
cand_dot_interval([os.path.join(CAND, "cand1_dot_interval")])
cand_text_panels([os.path.join(CAND, "cand2_text_panels"), os.path.join(HERE, "fig1")])
cand_se_bound([os.path.join(CAND, "cand3_se_bound"), os.path.join(HERE, "fig2")])
cand_all_estimates([os.path.join(CAND, "cand4_all_estimates")])
cand_slope([os.path.join(CAND, "cand5_slope")])
print("\nFigures written: fig1 (= candidates/cand2), fig2 (= candidates/cand3), candidates/cand1..cand5")
Abstract effect sizes for crowding, ageing and primary care coverage disagree with the paper's Table 2
Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2026.101572 · PMC13382474
Summary
The abstract reports three effects from "the final spatial GAMLSS model", each per one percentage-point increase. It gives +3.25% expected tuberculosis (TB) mortality for household crowding, +5.92% for the population aged 60 or older, and −0.30% for primary health care (PHC) coverage. The paper's Table 2 reports that model's coefficients: 0.0211, 0.0432 and −0.0025. Converted with the formula the Results themselves use, 100·(exp(β)−1), these give +2.13%, +4.41% and −0.25%. The Results text prints exactly those three values. So the abstract states each effect larger than the paper's own estimate, by 52%, 34% and 20%. No coefficient in Table 2 or in the supplement's sensitivity models (Table S3) reproduces all three abstract figures. The direction and significance of each association are unchanged, but the abstract's headline magnitudes do not match the model they cite.
What the paper reports
The abstract:
In the final spatial GAMLSS model, proportions of household crowding (>2 residents/bedroom) and elderly population (>59 years) were associated with 3.25% and 5.92% increases in expected TB mortality for each one percentage-point increase, respectively. In contrast, each one percentage-point increase in primary health care coverage was associated with a 0.30% decrease in expected TB mortality.
— Abstract, Findings
The Results, describing the same model:
Table 2 presents the final spatial GAMLSS model associated with the outcome at p-value <0.01.
— Results, "Final association model", paragraph 1
Each additional month was associated with a 0.48% [ (exp (0.0048) -1) ∗ 100] increase in the expected number of municipal-monthly TB deaths.
— Results, "Final association model", paragraph 2 (this states how coefficients are converted to percentages)
Among the contextual variables, each one percentage-point increase in the proportion of households with more than two residents per bedroom was associated with a 2.13% increase in expected TB deaths. Similarly, each one percentage-point increase in the proportion of residents aged 60 years or older was associated with a 4.41% increase,
— Results, "Final association model", paragraph 3
Each one percentage-point increase in coverage was associated with a 0.25% decrease in expected TB deaths.
— Results, "Final association model", paragraph 4
Table 2 ("Parametric covariate estimates from the spatial GAMLSS model for monthly TB deaths in São Paulo, Brazil, 2020–2024"). The columns are variable, coded variable, estimate, standard error, exponential of the estimate and p-value:
Households with more than 2 residents per bedroom (%) | prop_mais2_dorm | 0.0211 | 0.0073 | 1.0213 | =0.0041
Population aged 60 years or older (%) | prop_mais59 | 0.0432 | 0.0077 | 1.0442 | <0.0001
Primary health care coverage (%) | cobertura_aps | -0.0025 | 0.0007 | 0.9975 | =0.0002
— Table 2
I checked each quotation against the article as archived in PubMed Central (src/show.py PMC13382474). Each occurs exactly once. recompute.py.txt repeats that check.
What the paper's own numbers show
Method. The model uses a negative binomial type I distribution with a log link and an offset of log municipal population. A one-unit increase in a covariate therefore multiplies expected deaths by exp(β), a change of 100·(exp(β)−1)%. The Results state this formula for the time trend. All three covariates are measured in percent, so a one-unit increase is one percentage point. I applied the formula to the printed Table 2 coefficients. To allow for rounding, I also used the full range each coefficient could have before it was rounded to four decimals (β ± 0.00005). I inverted each abstract figure, ln(1 + p/100), to find the coefficient it would require. I formed 95% intervals as β ± 1.96·SE from the printed standard errors; the paper reports SEs and p-values but no intervals. Finally, I read every coefficient in the supplement's Table S3, which has the final model and four models that each drop one covariate. I tested whether any of them gives an abstract figure, either exponentiated or as 100·β.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Crowding (>2 residents/bedroom), % change per 1 pp | Abstract | +3.25% | +2.13% = 100·(exp(0.0211)−1); rounding range +2.127 to +2.138% |
| Crowding, % change per 1 pp | Results text; Table 2 | +2.13%; β 0.0211, exp(β) 1.0213 | +2.13% (consistent) |
| Coefficient needed for the abstract's +3.25% | Abstract vs Table 2 | β = 0.0211 | ln(1.0325) = 0.0320 |
| Population aged 60+, % change per 1 pp | Abstract | +5.92% | +4.41% = 100·(exp(0.0432)−1); rounding range +4.409 to +4.420% |
| Population aged 60+, % change per 1 pp | Results text; Table 2 | +4.41%; β 0.0432, exp(β) 1.0442 | +4.41% (+4.42% from the rounded 1.0442; consistent) |
| Coefficient needed for the abstract's +5.92% | Abstract vs Table 2 | β = 0.0432 | ln(1.0592) = 0.0575 |
| PHC coverage, % change per 1 pp | Abstract | −0.30% | −0.25% = 100·(exp(−0.0025)−1); rounding range −0.255 to −0.245% |
| PHC coverage, % change per 1 pp | Results text; Table 2 | −0.25%; β −0.0025, exp(β) 0.9975 | −0.25% (consistent) |
| Coefficient needed for the abstract's −0.30% | Abstract vs Table 2 | β = −0.0025 | ln(0.9970) = −0.0030 |
| Abstract ÷ Table 2 effect (crowding / age 60+ / PHC) | Abstract vs Table 2 | — | 1.52 / 1.34 / 1.20 |
| 95% CI implied by Table 2, % per 1 pp (crowding / age 60+ / PHC) | Table 2 (β ± 1.96 SE) | not reported | +0.68 to +3.60 / +2.85 to +6.00 / −0.39 to −0.11 |
| Other Table 2 rows: time trend, TB incidence, education, Black or brown population | Results text | +0.48%, +2.13%, −3.18%, +1.43% | +0.48%, +2.13%, −3.19%, +1.43% (consistent within rounding) |
| Crowding coefficient in the Table S3 sensitivity models | Supplement, Table S3 | 0.0040 / 0.0344 / 0.0381 | +0.40% / +3.50% / +3.88%; none is +3.25% |
| Age 60+ coefficient in the Table S3 sensitivity models | Supplement, Table S3 | 0.0357 / 0.0313 / 0.0259 / 0.0592 | +3.63% / +3.18% / +2.62% / +6.10%; none is +5.92% after conversion |
| PHC coefficient in the Table S3 sensitivity models | Supplement, Table S3 | −0.0025 / −0.0037 / −0.0024 / −0.0033 | −0.25% / −0.37% / −0.24% / −0.33%; none is −0.30% |
Within the body of the paper the numbers agree with each other. Table 2's coefficients, its exponentiated column and the Results text all match, as do the four other covariates in the table. The abstract is the only place where the three values differ. Each abstract figure needs a coefficient roughly 20–50% larger in absolute value than the one printed, which is far outside any rounding range.
Figure
Figure 1. The left panel shows the three per-percentage-point effects in the abstract's Findings. The right panel shows, for the same covariates of the same model, the Table 2 coefficient and its printed exponential, the percentage given in the Results text, and 100·(exp(β)−1) recomputed from the coefficient. Orange marks values as printed in the paper; blue marks values recomputed in recompute.py.txt. Each connector gives the ratio of the abstract value to the recomputed value. The Results text and the recomputation agree to two decimals in all three rows. The abstract does not agree with either in any row.
Figure 2. This figure adds the uncertainty. For each covariate, the blue dot is 100·(exp(β)−1) from Table 2 and the blue line is the 95% interval from β ± 1.96·SE. The paper reports the SEs but not the intervals. The orange diamond is the value in the abstract, and the thin vertical line marks no change (0%). Each panel has its own horizontal scale because the effects differ by more than tenfold. All three abstract values lie inside the interval, so sampling uncertainty does not separate them from the estimate. Even so, none of them is the estimate the paper reports. The age 60+ value (+5.92%) sits near the upper limit (+6.00%), and the crowding value (+3.25%) sits in the upper quarter of its interval.
Why it matters
The abstract is where most readers, and most citing papers, get a study's effect sizes. Here every number in the abstract's Findings is overstated relative to the model the abstract names. Crowding is 1.52 times the Table 2 value, age 60+ is 1.34 times and PHC coverage is 1.20 times. The overstatement grows when the effects are applied to realistic differences between municipalities. Over a 10-percentage-point difference in crowding, Table 2 implies +23.5% expected TB deaths (100·(exp(10·0.0211)−1)), while the abstract's per-point figure compounds to +37.7%. For the population aged 60+ the corresponding values are +54.0% and +77.7%. The PHC effect is small either way: −2.5% versus −3.0% per 10 points. The abstract's Interpretation ends by calling to "strengthen primary health care", and that recommendation rests on this association.
The qualitative findings are not affected. Every association keeps its direction and its significance, and each abstract value lies inside the confidence interval implied by Table 2. The Interpretation, Discussion and conclusions describe the associations without these numbers, so they stand. The error is one of reported magnitude, confined to the abstract.
Explanations considered
- A different model. The abstract names "the final spatial GAMLSS model", and the Results identify Table 2 as that model. The supplement has four more models in Table S3, each dropping one covariate. None of them gives the abstract's three values. Crowding there converts to +0.40%, +3.50% or +3.88%, and PHC coverage to −0.25%, −0.37%, −0.24% or −0.33%. One figure matches by coincidence of digits: the age 60+ coefficient in the model without the Municipal Development Index is 0.0592, which equals the abstract's 5.92 only if the coefficient is read as a percentage without exponentiating. Exponentiated, it is +6.10%. That model's other two coefficients give +3.88% and −0.33% (or 3.81 and −0.33 as 100·β), not +3.25% and −0.30%. So no single reported model, under either conversion, accounts for the three abstract figures. The supplement's R script also contains a commented-out stepwise (stepGAIC) specification with a different covariate set. No coefficients from it are published, so it cannot be checked.
- A different conversion. Under the log link, the paper's stated conversion is 100·(exp(β)−1). Reading the coefficients directly as 100·β gives +2.11%, +4.32% and −0.25%, which are also not the abstract values. Converting the abstract figures back gives coefficients of 0.0320, 0.0575 and −0.0030, none of which appears in Table 2.
- Rounding. Before rounding to four decimals, the Table 2 coefficients could range only over +2.127 to +2.138%, +4.409 to +4.420% and −0.255 to −0.245%. The abstract values are far outside these ranges. The 4.41% versus 4.42% difference between the text and the rounded exp(β) in Table 2 is rounding, and it is unrelated to 5.92%.
- A different unit or scaling. All three covariates are percentages, and both the abstract and the Results express the effect "for each one percentage-point increase". No rescaling (per SD, per 10 points) is mentioned, and none gives the abstract values. Per 10 points, for example, the Table 2 crowding effect is +23.5%.
- Text conversion or archiving artifact. The abstract numbers are plain prose. The same strings appear in the live PMC page, the Europe PMC full-text XML and the local PMC archive. The Table 2 cells line up with their rows, the minus sign on −0.0025 is intact, and the cells agree with the Results text. An artifact would not account for three different numbers.
- A deliberate rounding or summary in the abstract. The abstract gives two-decimal values, just as the Results do. They are not rounded versions of the Results values: rounding 2.13 does not give 3.25.
No explanation considered accounts for the abstract's three figures. The values in the body of the paper are internally consistent and reproduce from the printed coefficients.
Status
- Version of record. The publisher pages could not be read: doi.org redirects to linkinghub.elsevier.com, which returned only a redirect stub; thelancet.com and ScienceDirect returned HTTP 403; the Elsevier text-mining API returned 429 and then 400. The publisher-supplied text was therefore checked in two archived copies. Both the live PMC page (https://pmc.ncbi.nlm.nih.gov/articles/PMC13382474/, re-fetched 2026-10-23) and the Europe PMC full-text XML contain the abstract string with "3.25% and 5.92% increases" and "a 0.30% decrease". The PMC page also has the Results values "2.13%" and "0.25%".
- Supplement. Supplementary file mmc1.pdf (14 pages, via Europe PMC) was read in full. It contains Tables S1–S3 and the R GAMLSS script. Table S3's final-model column repeats Table 2 (0.0211, 0.0432, −0.0025).
- Corrections and retractions. None were found in Crossref update metadata, PMC notices or Europe PMC (
src/known.py, 2026-10-23). - PubPeer. The PubPeer API requires browser verification and returned no results to the automated query. Web searches for the title together with "PubPeer", "erratum" or "correction" found no thread or notice. No PubPeer discussion was found as of 2026-10-23, but the search could not be done directly on PubPeer.
As of 2026-10-23.
Suggested correction
In the abstract's Findings, replace the three effect sizes with the values the paper reports in its Results and Table 2. Adding the confidence intervals would also help:
In the final spatial GAMLSS model, proportions of household crowding (>2 residents/bedroom) and elderly population (>59 years) were associated with 2.13% and 4.41% increases in expected TB mortality for each one percentage-point increase, respectively. In contrast, each one percentage-point increase in primary health care coverage was associated with a 0.25% decrease in expected TB mortality.
If the authors meant the abstract to report a different model, the abstract should name that model and its coefficients should be published. As shown above, though, none of the reported models gives all three current figures. The other numbers in the Results that we checked need no change.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13382474 (Lancet Reg Health Am 2026, doi:10.1016/j.lana.2026.101572)
# Abstract vs Results / Table 2: per-percentage-point effects of household crowding,
# population aged 60+, and primary health care (PHC) coverage on expected TB mortality.
#
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1.{svg,png} and
# fig2.{svg,png} in the folder that holds this file. Needs matplotlib.
#
# Inputs
# * Table 2 (final spatial GAMLSS, negative binomial type I, log link, offset log population):
# coefficients, standard errors and exp(estimate) as printed in the article archived in
# PubMed Central (PMC13382474).
# * Table S3 (supplement mmc1.pdf): coefficients of the final model and of the four
# leave-one-covariate-out sensitivity models, as printed.
# * Quotes: copied verbatim from the PMC-archived article. When run inside the
# /home/ubuntu/papers repo, the script re-checks each quote with src/show.py (skipped elsewhere).
# * Percent change per unit = 100 * (exp(beta) - 1), the formula the Results state
# explicitly for the time trend: "0.48% [ (exp (0.0048) -1) * 100]".
import math, os, subprocess
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
# ------------------------------------------------------------------ quotes
QUOTES = {
"abstract": ("Abstract, Findings",
"In the final spatial GAMLSS model, proportions of household crowding (>2 residents/bedroom) "
"and elderly population (>59 years) were associated with 3.25% and 5.92% increases in expected "
"TB mortality for each one percentage-point increase, respectively. In contrast, each one "
"percentage-point increase in primary health care coverage was associated with a 0.30% decrease "
"in expected TB mortality."),
"results_time": ("Results, Final association model, para 2",
"Each additional month was associated with a 0.48% [ (exp (0.0048) -1) ∗ 100] increase in the "
"expected number of municipal-monthly TB deaths."),
"results_pos": ("Results, Final association model, para 3",
"Among the contextual variables, each one percentage-point increase in the proportion of "
"households with more than two residents per bedroom was associated with a 2.13% increase in "
"expected TB deaths. Similarly, each one percentage-point increase in the proportion of "
"residents aged 60 years or older was associated with a 4.41% increase,"),
"results_phc": ("Results, Final association model, para 4",
"Each one percentage-point increase in coverage was associated with a 0.25% decrease in "
"expected TB deaths."),
"results_t2": ("Results, Final association model, para 1",
"Table 2 presents the final spatial GAMLSS model associated with the outcome at p-value <0.01."),
"t2_crowd": ("Table 2",
"Households with more than 2 residents per bedroom (%) | prop_mais2_dorm | 0.0211 | 0.0073 | 1.0213 | =0.0041"),
"t2_age": ("Table 2",
"Population aged 60 years or older (%) | prop_mais59 | 0.0432 | 0.0077 | 1.0442 | <0.0001"),
"t2_phc": ("Table 2",
"Primary health care coverage (%) | cobertura_aps | -0.0025 | 0.0007 | 0.9975 | =0.0002"),
"sens": ("Results, Diagnostic plots, para 3",
"In all reduced models, the direction of the associations for the remaining covariates was "
"preserved (Table S3)"),
}
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.\n")
return
try:
full = subprocess.run([py, "src/show.py", "PMC13382474", "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e:
print(f"Quote check: could not load archived text ({e}); skipped.\n")
return
norm = lambda s: " ".join(s.replace("−", "-").split())
full_n = norm(full)
print("Quote check against the PMC-archived full text (occurrences; 1 = found once):")
for key, (loc, q) in QUOTES.items():
print(f" {key:13s} {full_n.count(norm(q))} ({loc})")
print()
# ------------------------------------------------------------------ paper's numbers
# Table 2: name -> (estimate, SE, exp(estimate) as printed)
T2 = {
"time_id": (0.0048, 0.0009, 1.0048),
"crowding": (0.0211, 0.0073, 1.0213),
"incid_tb": (0.0211, 0.0014, 1.0213),
"age60": (0.0432, 0.0077, 1.0442),
"educ": (-0.0324, 0.0037, 0.9681),
"black_brown": (0.0142, 0.0032, 1.0143),
"phc": (-0.0025, 0.0007, 0.9975),
"ipdm": (-3.7159, 0.5513, 0.0243),
}
# Results text, % change per unit (printed; "1,43%" in the text is read as 1.43)
TEXT = {"time_id": 0.48, "crowding": 2.13, "incid_tb": 2.13, "age60": 4.41,
"educ": -3.18, "black_brown": 1.43, "phc": -0.25}
# Abstract, Findings (decrease written as a negative change)
ABSTRACT = {"crowding": 3.25, "age60": 5.92, "phc": -0.30}
LABEL = {"crowding": "Household crowding (>2 residents/bedroom)",
"age60": "Population aged 60+",
"phc": "Primary health care coverage"}
# Table S3: final model and four sensitivity models (None = covariate not included)
S3_MODELS = ["final", "without crowding", "without education", "without Black/Brown",
"without Municipal Development Index"]
S3 = {
"crowding": [0.0211, None, 0.0040, 0.0344, 0.0381],
"age60": [0.0432, 0.0357, 0.0313, 0.0259, 0.0592],
"phc": [-0.0025, -0.0025, -0.0037, -0.0024, -0.0033],
}
pct = lambda b: 100 * (math.exp(b) - 1)
Z = 1.959964
def main():
check_quotes()
print("1. Table 2: percent change per unit, 100*(exp(beta)-1), vs Results text and abstract")
print(f" {'variable':12s} {'beta':>8s} {'exp(b)':>8s} {'T2 exp':>7s} {'%(beta)':>8s} "
f"{'%(T2exp)':>8s} {'text':>6s} {'abstract':>8s}")
for k, (b, se, e) in T2.items():
print(f" {k:12s} {b:8.4f} {math.exp(b):8.4f} {e:7.4f} {pct(b):8.3f} "
f"{100*(e-1):8.2f} {str(TEXT.get(k, '')):>6s} {str(ABSTRACT.get(k, '')):>8s}")
print()
print("2. Rounding range: beta printed to 4 decimals, so true beta lies in [b-0.00005, b+0.00005)")
for k in ABSTRACT:
b = T2[k][0]
lo, hi = pct(b - 0.00005), pct(b + 0.00005)
print(f" {k:9s} beta {b:+.4f}: {lo:+.3f}% to {hi:+.3f}% (abstract {ABSTRACT[k]:+.2f}%)")
print()
print("3. Coefficient the abstract value would require, ln(1 + p/100), and abstract / recomputed")
for k, p in ABSTRACT.items():
b = T2[k][0]
print(f" {k:9s} abstract {p:+.2f}% needs beta {math.log(1 + p/100):+.4f} "
f"(Table 2 {b:+.4f}); abstract/recomputed = {p/pct(b):.3f} "
f"(overstated by {100*(p/pct(b)-1):.0f}%)")
print()
print("4. 95% CI implied by Table 2 (beta +/- 1.96 SE), as percent change per percentage point")
for k in ABSTRACT:
b, se, _ = T2[k]
lo, hi = pct(b - Z*se), pct(b + Z*se)
inside = lo <= ABSTRACT[k] <= hi
print(f" {k:9s} {pct(b):+.2f}% (95% CI {lo:+.2f} to {hi:+.2f}); abstract {ABSTRACT[k]:+.2f}% "
f"{'inside' if inside else 'OUTSIDE'} the interval")
print()
print("5. Table S3 sensitivity models: 100*(exp(beta)-1) and 100*beta")
for k, row in S3.items():
print(f" {k}:")
for m, b in zip(S3_MODELS, row):
if b is None:
print(f" {m:38s} not included")
else:
print(f" {m:38s} beta {b:+.4f} -> {pct(b):+.2f}% (100*beta = {100*b:+.2f})")
print()
print("6. Does any reported coefficient reproduce an abstract value (within 0.005 pp)?")
for k, p in ABSTRACT.items():
hits = []
for m, b in zip(S3_MODELS, S3[k]):
if b is None:
continue
if abs(round(pct(b), 2) - p) < 0.005:
hits.append(f"{m}: 100*(exp(b)-1)")
if abs(round(100*b, 2) - p) < 0.005:
hits.append(f"{m}: 100*beta (not exponentiated)")
print(f" {k:9s} {p:+.2f}%: {hits if hits else 'no match in any reported model'}")
nb = S3["crowding"][4], S3["age60"][4], S3["phc"][4]
print(f" Model without Municipal Development Index, read the same way for all three "
f"(100*(exp(b)-1)): {pct(nb[0]):+.2f}%, {pct(nb[1]):+.2f}%, {pct(nb[2]):+.2f}%; "
f"(100*beta): {100*nb[0]:+.2f}, {100*nb[1]:+.2f}, {100*nb[2]:+.2f}")
print(" Abstract trio: +3.25, +5.92, -0.30")
print()
print("7. Scaled to a 10-percentage-point difference, 100*(exp(10*beta)-1)")
for k, p in ABSTRACT.items():
b = T2[k][0]
print(f" {k:9s} Table 2: {pct(10*b):+.1f}% abstract value compounded: "
f"{100*((1+p/100)**10-1):+.1f}%")
print()
fig1()
fig2()
print("Figures written to", HERE)
# ------------------------------------------------------------------ figures
def _style():
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none",
})
def _save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
def fig1():
_style()
fig = plt.figure(figsize=(7.2, 4.8), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off()
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.02, 0.955, "Same model, two sets of effect sizes", fontsize=13, weight="bold", va="top")
ax.text(0.02, 0.900, "% change in expected TB mortality per 1 percentage-point increase, final spatial GAMLSS", fontsize=10, color=INK2, va="top")
L0, L1, R0, R1 = 0.02, 0.34, 0.49, 0.98
top, bot = 0.83, 0.17
for x0, x1, head in ((L0, L1, "Abstract (Findings)"), (R0, R1, "Results text / Table 2")):
ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot, boxstyle="round,pad=0,rounding_size=0.015",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(x0 + 0.015, top - 0.03, head, fontsize=11, weight="bold", va="top")
rows = [("crowding", 0.64), ("age60", 0.46), ("phc", 0.28)]
for k, y in rows:
b = T2[k][0]; e = T2[k][2]; p = ABSTRACT[k]; r = pct(b)
# left panel
ax.text(L0 + 0.015, y + 0.035, LABEL[k].replace(" (>2 residents/bedroom)", ""),
fontsize=10, color=INK2, va="center")
ax.text(L0 + 0.015, y - 0.025, f"{p:+.2f}%", fontsize=14, weight="bold", color=ORANGE, va="center")
# right panel
ax.text(R0 + 0.015, y + 0.035, f"Table 2: β = {b:+.4f}, exp(β) = {e:.4f}", fontsize=10,
color=INK2, va="center")
ax.text(R0 + 0.015, y - 0.025, f"text {TEXT[k]:+.2f}%", fontsize=12, weight="bold",
color=ORANGE, va="center")
ax.text(R0 + 0.205, y - 0.025, f"100·(e$^{{β}}$−1) = {r:+.2f}%", fontsize=12, weight="bold",
color=BLUE, va="center")
# connector
ax.annotate("", xy=(R0 - 0.005, y), xytext=(L1 + 0.005, y),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
ax.text((L1 + R0) / 2, y + 0.022, "conflict", fontsize=10, color=INK, ha="center",
va="bottom", weight="bold")
ax.text((L1 + R0) / 2, y - 0.022, f"abstract {p / r:.2f}×", fontsize=10, color=INK2,
ha="center", va="top")
ax.text(0.02, 0.125, "Orange: as printed in the paper. Blue: recomputed from the Table 2 coefficient with",
fontsize=10, color=INK2, va="top")
ax.text(0.02, 0.080, "100·(exp(β)−1), the formula the Results apply to the time trend. No coefficient",
fontsize=10, color=INK2, va="top")
ax.text(0.02, 0.035, "reported in Table 2 or Table S3 gives all three abstract values.",
fontsize=10, color=INK2, va="top")
_save(fig, "fig1")
def fig2():
_style()
keys = ["crowding", "age60", "phc"]
fig, axes = plt.subplots(3, 1, figsize=(7.2, 4.8), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.96, top=0.86, bottom=0.19, hspace=1.05)
fig.text(0.02, 0.965, "Abstract values fall inside the 95% CIs, away from the Table 2 estimates",
fontsize=12, weight="bold", va="top")
fig.text(0.02, 0.915, "Percent change per 1 percentage-point increase; one panel per covariate, "
"each with its own scale", fontsize=10, color=INK2, va="top")
lims = {"crowding": (-0.5, 4.5), "age60": (-0.5, 7.0), "phc": (-0.5, 0.1)}
for ax, k in zip(axes, keys):
b, se, _ = T2[k]
est, lo, hi = pct(b), pct(b - Z*se), pct(b + Z*se)
ax.set_facecolor(BG)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.plot([lo, hi], [0, 0], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([est], [0], "o", ms=9, color=BLUE, zorder=3,
label="Recomputed from Table 2: 100·(exp(β)−1), 95% CI β ± 1.96 SE")
ax.plot([ABSTRACT[k]], [0], "D", ms=9, color=ORANGE, mec="white", mew=0.8, zorder=4,
label="Reported in the abstract")
ax.annotate(f"{est:+.2f}", (est, 0), xytext=(0, 9), textcoords="offset points",
ha="center", fontsize=10, color=BLUE)
ax.annotate(f"{ABSTRACT[k]:+.2f}", (ABSTRACT[k], 0), xytext=(0, -17), textcoords="offset points",
ha="center", fontsize=10, color=ORANGE)
ax.set_xlim(*lims[k]); ax.set_ylim(-1, 1)
ax.set_yticks([0]); ax.set_yticklabels([LABEL[k].replace(" (>2 residents/bedroom)", "").replace("Primary health care", "Primary health\ncare")], fontsize=10.5,
color=INK)
ax.tick_params(axis="y", length=0)
ax.tick_params(axis="x", labelsize=10)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#bdbcb7")
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{v:g}%"))
h, l = axes[0].get_legend_handles_labels()
fig.legend(h, l, loc="lower left", bbox_to_anchor=(0.01, 0.0), ncol=1, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.2)
_save(fig, "fig2")
if __name__ == "__main__":
main()
Abstract's "significant" April 2020 drop in early treatment access has a sign-flipped confidence limit
Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2026.101406 · PMC13404213
Summary
The Abstract's lead interrupted time-series result is a "significant level decrease" in April 2020 in the rate of colorectal cancer treatment started within 30 days, all treatments combined: −0.035 (95% CI −0.091 to −0.022). The Results text and Table 2 print the same estimate with the interval −0.091 to +0.022. Only that version is centred on the estimate (midpoint −0.0345, against −0.0565 for the Abstract's interval), so the Abstract's upper limit carries a sign error. The correct interval contains zero (back-calculated z = −1.21, p = 0.22), and so does the interval for chemotherapy and surgery (−0.040, −0.096 to 0.018, p = 0.17). The paper's own numbers therefore do not support the "significant" immediate drops claimed in the Abstract and the Results, although its separate 2020 ITAR estimates still show a reduction in early access.
What the paper reports
Abstract, Findings:
ITS analysis revealed a significant level decrease by April 2020 for early access to all the treatments combined (-0.035; 95% CI: -0.091 to -0.022), chemotherapy (-0.040; 95% CI: (-0.096 to 0.018) and surgery (-0.040; 95% CI: (-0.096 to 0.018), while radiotherapy (0.002; 95% CI: 0.000–0.004) showed no meaningful level change.
Results, "Interrupted time series", first paragraph:
According to Table 2 and Fig. 2, for treatments initiated within 30 days, a significant immediate level decrease was observed in April 2020 for all treatments combined (estimate: -0.035; 95% CI: -0.091 to 0.022), corresponding to an absolute reduction of 3.5% in ASTAR. Similar declines were identified for chemotherapy and surgery (both -0.040; 95% CI: -0.096 to 0.018), indicating a 4.0% drop in early access for these modalities. In contrast, radiotherapy showed a small but significant positive level change (0.002; 95% CI: 0.000–0.004).
Table 2, "≤30 days", row "COVID-19" (columns All treatment, Chemotherapy, Radiotherapy, Chemotherapy + Radiotherapy, Surgery; estimate and 95% CI):
-0.035 | (-0.091 to 0.022) | -0.040 | (-0.096 to 0.018) | 0.002∗ | (0.000–0.004) | -0.000a | (-0.000 to 0.000) | -0.040 | (-0.096 to 0.018)
Table 2, footer:
p-value: ∗<0.05; ∗∗<0.01; ∗∗∗<0.001.
Methods, "Statistical analysis":
ITS analysis was conducted using two models: one using ordinary least squares (OLS) regression and the other using generalized least squares (GLS) regression.
Statistical significance was set at p < 0.05.
The primary outcome of this study was the age-standardized treatment access rate (ASTAR) per 100,000 inhabitants.
Results, "Interrupted time series", second paragraph (used below as a cross-check on the handling of minus signs):
In other hand, for treatments initiated after 60 days, the pandemic was associated with significant negative level changes for all treatments combined (-0.045; 95% CI: -0.061 to -0.029), chemotherapy (-0.033; 95% CI: -0.045 to -0.020), and combined chemoradiotherapy (-0.045; 95% CI: -0.061 to -0.029)
Table 2, ">60 days", row "COVID-19":
-0.045 | (-0.061 to 0.029) | -0.033 | (-0.045 to 0.020) | -0.009∗∗∗ | (-0.015 to 0.004) | -0.045 | (0.061–0.029) | -0.002a | (-0.005 to 0.000)
Results, "Impact of COVID-19 on treatment access rate (ITAR)":
The ITAR analysis corroborated the ITS findings and allowed direct comparison between observed and expected access levels during 2020 as seen in Table 3.
What the paper's own numbers show
Method. The level change β2 is a linear coefficient from an OLS or GLS segmented regression. Its 95% confidence interval is the symmetric Wald interval β2 ± c·SE, so the printed estimate should sit at the midpoint of its interval. From the printed limits, SE = (upper − lower) / (2c) and z = β2 / SE. I used c = 1.96, and also the t quantile with 54 residual degrees of freedom (60 monthly points from 2018 to 2022, minus β0–β3 and one sine and one cosine harmonic term). I applied this to both printed versions of the ≤30-day all-treatment interval and to the other ≤30-day level changes named in the Abstract. As a cross-check, I audited all 33 Table 2 cells that have a full interval for (a) a midpoint far from the estimate and (b) an interval that excludes 0 but has no significance star. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| ≤30 days, all treatments, level change: CI as printed in the Abstract | Abstract | −0.035 (−0.091 to −0.022), "significant" | midpoint −0.0565, 0.0215 away from the estimate; upper − est minus est − lower = −0.043; not a Wald interval around −0.035 |
| Same quantity: CI as printed in the body | Results text; Table 2 | −0.035 (−0.091 to 0.022) | midpoint −0.0345, matching the estimate within rounding; contains 0 |
| Same quantity: SE, z, p from the body CI | derived | "significant" (Abstract, Results); no star (Table 2) | SE 0.0288, z = −1.21, two-sided p = 0.22 (t with 54 df: 0.22); one-sided p = 0.11 |
| Same quantity: rounding range | derived | three-decimal estimate and limits | p = 0.21 to 0.24 |
| Same quantity: what p = 0.05 would need | derived | SE 0.0288 | SE ≤ 0.0179, i.e. a 95% CI ending at 0.000 |
| ≤30 days, chemotherapy and surgery (identical values), level change | Abstract; Results; Table 2 | −0.040 (−0.096 to 0.018), "significant"; no star | midpoint −0.039; SE 0.0291, z = −1.38, p = 0.17 (t: 0.17) |
| ≤30 days, radiotherapy, level change | Abstract; Results; Table 2 | 0.002 (0.000–0.004); "no meaningful level change" (Abstract), "small but significant" (Results), one star (Table 2) | midpoint 0.002; z = 1.96 at face value; over the rounding range p = 0.001 to 0.24, so significance cannot be read from the print |
| >60 days, all treatments, level change | Results text vs Table 2 | text −0.045 (−0.061 to −0.029); table −0.045 (−0.061 to 0.029) | text midpoint −0.045 (centred); table midpoint −0.016 (not centred) |
| >60 days, chemotherapy, level change | Results text vs Table 2 | text −0.033 (−0.045 to −0.020); table −0.033 (−0.045 to 0.020) | text midpoint −0.0325 (centred); table midpoint −0.0125 |
| >60 days, chemoradiotherapy, level change | Results text vs Table 2 | text −0.045 (−0.061 to −0.029); table −0.045 (0.061–0.029) | table interval excludes its own estimate (midpoint +0.045) |
| Table 2 audit: midpoint far from estimate | Table 2, 33 cells | — | 6 cells: the three >60-day COVID-19 cells above, radiotherapy in the same row (−0.009, −0.015 to 0.004, midpoint −0.0055), the ≤30-day all-treatment "Post-pandemic start" cell (−0.004, −0.072 to 0.018, midpoint −0.027), and one >60-day intercept by a small margin |
| Table 2 audit: stars | Table 2, 33 cells | footer: ∗ p < 0.05 | 18 intervals exclude 0; 9 of them carry no star (for example 31–60 days, COVID-19, all treatments, 0.008, 0.001–0.014) |
Two results follow. First, the body version of the ≤30-day interval is the internally consistent one: it is centred on the estimate that all three locations print, whereas the Abstract's interval would need an estimate of about −0.0565. Second, that consistent interval contains zero, as does the chemotherapy and surgery interval, so neither level change reaches the paper's stated threshold of p < 0.05. The Table 2 stars agree (neither cell has one), but the audit shows the stars are applied inconsistently elsewhere in the table, so the conclusion here rests on the intervals, not on the stars.
Figure
Figure 1. Estimates and 95% confidence intervals for the immediate level change at April 2020 in the ≤30-day treatment access rate, in Table 2 units, with the vertical line at 0 (no level change). Orange: values as printed in the paper, with filled dots at the printed estimates; the top row is the Abstract's interval for all treatments (upper limit −0.022), the second row the interval printed in the Results text and Table 2 (upper limit +0.022), and the fourth row the interval for chemotherapy and surgery, which the paper prints with identical values. Blue diamonds mark the midpoint of each printed all-treatment interval: −0.0565 for the Abstract, which is far from the estimate of −0.035, and −0.0345 for the body, which coincides with it. Blue open dots and lines are the symmetric Wald intervals rebuilt around each printed estimate from the printed interval width, with the two-sided p value this implies (0.22 and 0.17). Both rebuilt intervals cross 0, although the paper calls both decreases "significant".
Why it matters
The ≤30-day level change is the first interrupted time-series result in the Abstract and the first in the Results section on the time series. It is the paper's only estimate of the immediate effect of the pandemic on timely treatment for all modalities combined, and the Abstract presents it, with the chemotherapy and surgery estimates, as a statistically significant drop. With the interval the paper prints in its body, the estimate is compatible with no immediate change (p ≈ 0.22), and the same holds for chemotherapy and surgery (p ≈ 0.17). A reader of the Abstract would see an interval that excludes zero and would not know that the paper's own table gives one that does not.
The reach of the error is limited. The paper's broader conclusion that early access fell in 2020 does not rest on these coefficients alone: the ITAR estimates in Table 3 for treatment within 30 days are 0.88 (0.79–0.98) for all treatments and 0.82 (0.68–0.96) for chemotherapy, surgery and chemoradiotherapy, and both intervals exclude 1. These measure a different quantity (observed over expected access across 2020, not the immediate step in April 2020), so they do not make the level change significant, and the Results sentence that the ITAR analysis "corroborated the ITS findings" should be read with that in mind. The spatial analysis and the >60-day level changes, whose Results-text intervals exclude zero, are also unaffected. The title, the Research in context panel and the Interpretation do not quote the ≤30-day coefficient.
Two related reporting points affect how the ITS results can be read. Table 2 drops minus signs in several >60-day cells (in one case the printed interval, 0.061–0.029, excludes its own estimate of −0.045), so readers working from the table would reach wrong conclusions there too. And the Results read the coefficient −0.035 as "an absolute reduction of 3.5% in ASTAR", while the Methods define ASTAR as a rate "per 100,000 inhabitants". On that definition −0.035 is a change of 0.035 per 100,000, not 3.5 percentage points; a rough check from the Table 2 intercept and pre-pandemic trend (0.117 + 0.005 × 27 ≈ 0.25 just before April 2020, ignoring the seasonal terms) puts it nearer 14% of the pre-pandemic level. The unit of the coefficients is not stated, so this is a point for the authors to clarify, not a finding.
Explanations considered
- The Abstract and the body describe different quantities. This does not account for the numbers. All three locations give the same estimate (−0.035) for the same row of the same model: ≤30 days, all treatments, the "COVID-19" level term at April 2020. Only the sign of the upper limit differs.
- The Abstract is right and the body dropped a minus sign. Table 2 does drop minus signs in other rows, so this had to be checked. It is excluded by the symmetry of the interval. For the Abstract's interval to be a Wald interval, the estimate would have to be about −0.0565, yet −0.035 is printed in the Abstract, the Results and Table 2, and the Results restate it as "3.5%". In the >60-day rows, where the table and the Results text disagree, it is the text's version that is centred on the estimate; in the ≤30-day row, the Results text and the table agree with each other and are the centred version.
- An asymmetric or transformed interval (for example on a log scale, or a bootstrap interval). This does not account for the numbers. The Methods describe OLS and GLS regressions of the monthly rate on its natural scale, whose coefficient intervals are symmetric, and the body interval for this coefficient, like most intervals in Table 2, is centred on its estimate.
- A one-sided test, t rather than normal critical values, or rounding. None of these changes the conclusion. The Methods set significance at p < 0.05 and mention no one-sided tests; even one-sided, p = 0.11. With a t distribution on 54 df, p = 0.22. Over the full rounding range of the three printed numbers, p stays between 0.21 and 0.24.
- "Significant" means substantively important, not statistically significant. This cannot be ruled out from the wording, but the Abstract attaches the word to a 95% CI, and the same paragraph of the Results uses "small but significant" for a radiotherapy change of 0.002 that carries a significance star. Read either way, the Abstract's printed interval excludes zero and the paper's own interval does not.
- The significance star convention settles it. It points the same way but is not decisive on its own: neither the all-treatment nor the chemotherapy and surgery cell has a star, but 9 of the 18 Table 2 intervals that exclude zero also have none. The argument above rests on the intervals.
- A conversion or typesetting error in the archived text. This is excluded. The PMC page, the publisher-supplied full-text XML in Europe PMC (where every minus sign is the U+2212 character) and the PubMed abstract (PMID 42516946) all carry −0.022 in the Abstract and 0.022 in the Results and Table 2.
- A correction or supplement resolves it. No correction notice and no supplementary material were found (see Status).
Status
- Version of record. The publisher's own pages (doi.org, which redirects to linkinghub.elsevier.com; thelancet.com; sciencedirect.com; the Lancet PDF; the Elsevier text-mining API) returned redirects or HTTP 403 to automated requests and could not be read. I compared three copies derived from the publisher's XML: the PMC article page (PMC13404213), the Europe PMC full-text XML, and the PubMed record (PMID 42516946). All three carry the Abstract interval −0.091 to −0.022 and the Results and Table 2 interval −0.091 to 0.022, so the discrepancy is in the published text. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. The PMC and Europe PMC full text contain no supplementary-material element, and the Europe PMC supplementary-files endpoint returns 404. The data-sharing statement points to the public DATASUS and IBGE databases; no model output or analysis code is deposited, so the unrounded estimates cannot be checked.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI (this also covers the Retraction Watch data held in Crossref), and PMC, Europe PMC and PubMed list no correction notices or linked comments. A web search for corrections or comments on the article found none. The PubPeer search API refused automated access (HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC13404213; Europe PMC full-text XML for PMC13404213; PubMed record 42516946; Crossref metadata for doi:10.1016/j.lana.2026.101406.
Suggested correction
- In the Abstract, correct the interval for the ≤30-day all-treatment level change to −0.091 to 0.022, or, if the unrounded model output gives a different interval, report that interval consistently in the Abstract, Results and Table 2.
- Revise the Abstract and the first Results paragraph on the time series so that the immediate April 2020 level changes for all treatments, chemotherapy and surgery within 30 days are not described as statistically significant (their 95% CIs include zero; back-calculated p ≈ 0.22 and 0.17), for example by describing them as point estimates of a decrease with intervals compatible with no change.
- Qualify the statement that the ITAR analysis "corroborated the ITS findings" for the ≤30-day window, since the ITAR and the level change measure different quantities and only the ITAR excludes the null.
- Correct the Table 2 cells whose intervals are not centred on their estimates, in particular the >60-day COVID-19 row (all treatments, chemotherapy, radiotherapy and chemoradiotherapy) and the ≤30-day all-treatment "Post-pandemic start" cell, and apply the significance stars consistently.
- Reconcile the radiotherapy wording ("no meaningful level change" in the Abstract, "small but significant" in the Results).
- State the unit of the Table 2 coefficients and correct the reading of −0.035 as "an absolute reduction of 3.5% in ASTAR" if ASTAR is per 100,000 inhabitants. Minor: the Methods describe the outcome of the regression model as "the monthly hand soap usage at time t", which appears to be text left over from another study and should read monthly ASTAR.
Recomputation script (python recompute.py.txt)
"""Recompute the interrupted time-series confidence intervals of PMC13404213
(Lancet Reg Health Am 2026, doi:10.1016/j.lana.2026.101406,
COVID-19 and access to colorectal cancer treatment in Brazil).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg and fig1.png next to this file.
Inputs are copied from the paper as archived in PMC (all values are level changes, beta2,
at April 2020 from the segmented regression of monthly ASTAR):
Abstract (Findings): all treatments <=30 days -0.035 (95% CI -0.091 to -0.022), "significant"
chemotherapy, surgery -0.040 (-0.096 to 0.018), "significant"
radiotherapy 0.002 (0.000-0.004), "no meaningful level change"
Results (ITS) and Table 2: all treatments <=30 days -0.035 (-0.091 to 0.022), "significant"
Table 2 footer: p-value: * <0.05; ** <0.01; *** <0.001
Methods: Y_t = b0 + b1*T + b2*Xlevel + b3*Xtrend + harmonic(month; 1; 12) + e_t,
monthly data 2018-2022 (60 months), significance at p < 0.05.
Method: an OLS or GLS regression coefficient has a symmetric Wald interval est +/- c*SE,
so the interval's midpoint should equal the estimate, SE = (upper - lower) / (2c), and
z = est / SE. c = 1.96 (normal) or the t quantile with 60 - 6 = 54 residual df
(b0..b3 plus one sine and one cosine term).
"""
import os
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.isf(0.025)
DF = 60 - 6
T = stats.t.isf(0.025, DF)
def wald(est, lo, hi, crit=Z):
se = (hi - lo) / (2 * crit)
z = est / se
return se, z
def pnorm2(z):
return 2 * stats.norm.sf(abs(z))
def pt2(z):
return 2 * stats.t.sf(abs(z), DF)
R = {}
print("=== 1. The <=30-day, all-treatments level change at April 2020: two printed intervals ===")
EST = -0.035
VERSIONS = [("abstract", "Abstract", -0.091, -0.022), ("table", "Results text and Table 2", -0.091, 0.022)]
for key, name, lo, hi in VERSIONS:
mid = (lo + hi) / 2
asym = (hi - EST) - (EST - lo)
se, z = wald(EST, lo, hi)
se_t, z_t = wald(EST, lo, hi, T)
R[key] = dict(lo=lo, hi=hi, mid=mid, se=se, z=z, p=pnorm2(z), p1=stats.norm.sf(abs(z)), pt=pt2(z_t))
print(f" {name:26s} CI ({lo:+.3f}, {hi:+.3f}): midpoint {mid:+.4f} (estimate {EST}); "
f"upper-est minus est-lower = {asym:+.4f}; contains 0: {lo <= 0 <= hi}")
print(f" {'':26s} SE = {se:.4f}, z = {z:+.2f}, two-sided p = {R[key]['p']:.3f}, one-sided p = {R[key]['p1']:.3f}; "
f"t({DF}) two-sided p = {R[key]['pt']:.3f}")
tb = R["table"]
print(f" distance of abstract midpoint from estimate: {abs(R['abstract']['mid'] - EST):.4f}; "
f"table midpoint from estimate: {abs(tb['mid'] - EST):.4f}")
# Symmetric interval rebuilt around the estimate from the Table 2 width
rec_lo, rec_hi = EST - Z * tb["se"], EST + Z * tb["se"]
print(f" interval rebuilt around -0.035 from the Table 2 width: {rec_lo:+.4f} to {rec_hi:+.4f}")
# Rounding: estimate in [-0.0355, -0.0345], limits +/-0.0005
ps = []
for e in (-0.0355, -0.0345):
for lo in (-0.0915, -0.0905):
for hi in (0.0215, 0.0225):
ps.append(pnorm2(wald(e, lo, hi)[1]))
print(f" over the rounding range of estimate and limits: two-sided p = {min(ps):.3f} to {max(ps):.3f}")
# What would the estimate need to be for the abstract interval to be a Wald interval?
print(f" estimate implied by the abstract interval (its midpoint): {R['abstract']['mid']:+.4f}")
# What SE would p = 0.05 need at -0.035?
se05 = abs(EST) / Z
print(f" for p = 0.05 at -0.035 the SE would have to be <= {se05:.4f} (Table 2 width implies {tb['se']:.4f}); "
f"the 95% CI would then end at {EST + Z * se05:+.3f}")
print("\n=== 2. The other <=30-day level changes named in the Abstract ===")
OTHER = [("chemo", "Chemotherapy (and Surgery, identical)", -0.040, -0.096, 0.018, "no star", "'significant' (Abstract, Results)"),
("rt", "Radiotherapy", 0.002, 0.000, 0.004, "star (*)", "'no meaningful level change' (Abstract); 'small but significant' (Results)")]
for key, name, est, lo, hi, star, called in OTHER:
se, z = wald(est, lo, hi)
se_t, z_t = wald(est, lo, hi, T)
R[key] = dict(est=est, lo=lo, hi=hi, se=se, z=z, p=pnorm2(z), pt=pt2(z_t), mid=(lo + hi) / 2)
print(f" {name:38s} {est:+.3f} ({lo:+.3f}, {hi:+.3f}): midpoint {(lo + hi) / 2:+.4f}; contains 0: {lo <= 0 <= hi}; "
f"SE = {se:.4f}, z = {z:+.2f}, p = {R[key]['p']:.3f} (t({DF}): {R[key]['pt']:.3f}); Table 2: {star}; called {called}")
# Radiotherapy rounding: lower limit printed 0.000 could be anywhere in [-0.0005, 0.0005)
rp = [pnorm2(wald(e, lo, hi)[1]) for e in (0.0015, 0.0025) for lo in (-0.0005, 0.0005) for hi in (0.0035, 0.0045)]
print(f" Radiotherapy over the rounding range: p = {min(rp):.3f} to {max(rp):.3f} (cannot be placed on either side of 0.05 from print)")
print("\n=== 3. Audit of Table 2: midpoint vs estimate, and star vs whether the CI excludes 0 ===")
# (time window, variable, column, estimate, lower, upper, stars printed)
TABLE2 = [
("<=30", "Intercept", "All", 0.117, 0.075, 0.159, 3), ("<=30", "Intercept", "Chemo", 0.027, -0.028, 0.081, 0),
("<=30", "Intercept", "RT", 0.013, 0.011, 0.014, 3), ("<=30", "Intercept", "Surgery", 0.0269, -0.028, 0.081, 0),
("<=30", "Trend", "All", 0.005, 0.003, 0.005, 3), ("<=30", "Trend", "Chemo", 0.005, 0.003, 0.006, 1),
("<=30", "Trend", "Surgery", 0.005, 0.003, 0.006, 1),
("<=30", "COVID-19", "All", -0.035, -0.091, 0.022, 0), ("<=30", "COVID-19", "Chemo", -0.040, -0.096, 0.018, 0),
("<=30", "COVID-19", "RT", 0.002, 0.000, 0.004, 1), ("<=30", "COVID-19", "Surgery", -0.040, -0.096, 0.018, 0),
("<=30", "Post start", "All", -0.004, -0.072, 0.018, 0), ("<=30", "Post start", "Chemo", -0.004, -0.008, 0.001, 0),
("<=30", "Post start", "Surgery", -0.004, -0.008, 0.001, 0),
("31-60", "Intercept", "All", 0.135, 0.131, 0.139, 0), ("31-60", "Intercept", "Chemo", 0.105, 0.101, 0.108, 0),
("31-60", "Intercept", "RT", 0.019, 0.017, 0.020, 3), ("31-60", "Intercept", "ChemoRT", 0.005, 0.004, 0.006, 3),
("31-60", "Intercept", "Surgery", 0.006, 0.005, 0.006, 0),
("31-60", "COVID-19", "All", 0.008, 0.001, 0.014, 0), ("31-60", "COVID-19", "Chemo", 0.007, 0.002, 0.013, 0),
("31-60", "COVID-19", "RT", 0.003, 0.000, 0.006, 1), ("31-60", "COVID-19", "Surgery", -0.002, -0.003, 0.000, 0),
(">60", "Intercept", "All", 0.195, 0.185, 0.204, 0), (">60", "Intercept", "Chemo", 0.136, 0.123, 0.143, 0),
(">60", "Intercept", "RT", 0.040, 0.036, 0.043, 3), (">60", "Intercept", "ChemoRT", 0.194, 0.185, 0.204, 0),
(">60", "Intercept", "Surgery", 0.009, 0.007, 0.010, 3),
(">60", "COVID-19", "All", -0.045, -0.061, 0.029, 0), (">60", "COVID-19", "Chemo", -0.033, -0.045, 0.020, 0),
(">60", "COVID-19", "RT", -0.009, -0.015, 0.004, 3), (">60", "COVID-19", "ChemoRT", -0.045, 0.061, 0.029, 0),
(">60", "COVID-19", "Surgery", -0.002, -0.005, 0.000, 0),
]
n_excl = n_excl_nostar = n_off = 0
for tw, var, col, e, lo, hi, st in TABLE2:
mid = (lo + hi) / 2
half = (hi - lo) / 2
off = abs(mid - e) > max(0.0015, 0.25 * half) # beyond rounding of three-decimal values
excl = not (lo <= 0 <= hi)
n_excl += excl
n_excl_nostar += excl and st == 0
n_off += off
flag = []
if off:
flag.append(f"midpoint {mid:+.4f} far from estimate")
if not (lo <= e <= hi):
flag.append("estimate outside its own CI")
if excl and st == 0:
flag.append("CI excludes 0 but no star")
if flag:
print(f" {tw:5s} {var:10s} {col:7s} {e:+.4f} ({lo:+.3f}, {hi:+.3f}) {'*' * st:3s} -> " + "; ".join(flag))
print(f" cells checked: {len(TABLE2)}; CI excludes 0: {n_excl}, of which without a star: {n_excl_nostar}; "
f"midpoint far from estimate: {n_off}")
# >60-day rows: text vs table
print(" >60-day rows, Results text vs Table 2:")
for name, e, text_ci, tab_ci in [("All treatments", -0.045, (-0.061, -0.029), (-0.061, 0.029)),
("Chemotherapy", -0.033, (-0.045, -0.020), (-0.045, 0.020)),
("Chemoradiotherapy", -0.045, (-0.061, -0.029), (0.061, 0.029))]:
print(f" {name:18s} est {e:+.3f}: text CI midpoint {sum(text_ci) / 2:+.4f}, table CI midpoint {sum(tab_ci) / 2:+.4f}")
print("\n=== 4. Unit of the coefficient ===")
level_mar2020 = 0.117 + 0.005 * 27 # intercept + pre-trend x months to March 2020 (T = 1..27 for Jan 2018..Mar 2020)
print(f" Table 2 <=30-day all-treatment intercept 0.117 + trend 0.005 x 27 months = {level_mar2020:.3f} "
f"(ASTAR units) just before April 2020; -0.035 relative to that = {-0.035 / level_mar2020 * 100:.1f}% "
f"(rough, ignores the seasonal terms)")
R["lvl"] = level_mar2020
# ---------------------------------------------------------------- figure
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e3e2de"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
ab, ch = R["abstract"], R["chemo"]
rows = [
# y, label, est, lo, hi, color, kind, right-hand note
(4, "All treatments, ≤30 days\nAbstract (as printed)", EST, ab["lo"], ab["hi"], ORA, "rep",
"called “significant”"),
(3, "All treatments, ≤30 days\nResults text and Table 2", EST, tb["lo"], tb["hi"], ORA, "rep",
"called “significant”;\nno star in Table 2"),
(2, "All treatments, ≤30 days\nrecomputed (Wald, Table 2 width)", EST, rec_lo, rec_hi, BLU, "rec",
f"p = {tb['p']:.2f} (z = {tb['z']:.2f})".replace("-", "\u2212")),
(1, "Chemotherapy; Surgery, ≤30 days\nAbstract, Results, Table 2", ch["est"], ch["lo"], ch["hi"], ORA, "rep",
"called “significant”;\nno star in Table 2"),
(0, "Chemotherapy; Surgery, ≤30 days\nrecomputed (Wald)", ch["est"], ch["est"] - Z * ch["se"], ch["est"] + Z * ch["se"],
BLU, "rec", f"p = {ch['p']:.2f} (z = {ch['z']:.2f})".replace("-", "\u2212")),
]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
ax.axvline(0, color=SEC, lw=0.9, zorder=1)
ax.text(0.0012, 4.62, "0 = no level change", color=SEC, fontsize=10, ha="left", va="center")
for y, lab, e, lo, hi, c, kind, note in rows:
ax.plot([lo, hi], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=2)
for xx in (lo, hi):
ax.plot([xx, xx], [y - 0.1, y + 0.1], color=c, lw=2, zorder=2)
ax.plot(e, y, "o", color=c, ms=8, zorder=3, mfc=c if kind == "rep" else BG, mew=2)
ax.text(0.035, y, note, color=TXT, fontsize=10, ha="left", va="center")
# CI midpoints (recomputed) for the two printed versions
for y, key in ((4, "abstract"), (3, "table")):
m = R[key]["mid"]
ax.plot(m, y, marker="D", color=BLU, ms=8, mfc=BG, mew=2, zorder=4, ls="none")
ax.annotate(f"CI midpoint {ab['mid']:.4f}".replace("-", "\u2212"), (ab["mid"], 4), xytext=(0, 10), textcoords="offset points",
ha="center", va="bottom", fontsize=10, color=BLU)
ax.annotate(f"−0.022", (ab["hi"], 4), xytext=(0, -11), textcoords="offset points",
ha="center", va="top", fontsize=10, color=ORA)
ax.annotate(f"+0.022", (tb["hi"], 3), xytext=(0, -11), textcoords="offset points",
ha="center", va="top", fontsize=10, color=ORA)
ax.set_yticks([r[0] for r in rows])
ax.set_yticklabels([r[1] for r in rows], fontsize=10)
ax.set_xlim(-0.11, 0.075)
ax.set_xticks([-0.10, -0.08, -0.06, -0.04, -0.02, 0, 0.02])
ax.set_xticklabels([f"{v:.2f}".replace("-", "\u2212") for v in (-0.10, -0.08, -0.06, -0.04, -0.02, 0, 0.02)])
ax.set_ylim(-0.6, 4.85)
ax.set_xlabel("Level change in ASTAR at April 2020 (Table 2 units), estimate and 95% CI", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.7, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [
Line2D([], [], color=ORA, lw=2, marker="o", ms=8, label="reported in the paper"),
Line2D([], [], color=BLU, lw=2, marker="o", ms=8, mfc=BG, mew=2, label="recomputed: estimate ± 1.96 SE"),
Line2D([], [], color=BLU, lw=0, marker="D", ms=8, mfc=BG, mew=2, label="recomputed: midpoint of printed CI"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.3, -0.17), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
print("\nwrote fig1.svg and fig1.png")
Abstract reports 15.4% survival before 24 weeks; the paper's own counts give 24.1%
Lancet Reg Health Am (2022) · doi:10.1016/j.lana.2022.100330 · PMC9903864
Summary
The abstract gives survival to discharge or transfer for US infants born before 24 weeks' gestation in 2007–2018 (N = 77,050) as 15.4%. The Results and Discussion give the same quantity for the same group as 18,533 survivors (24.1%), and 18,533 / 77,050 = 24.05%. That count is supported three more ways inside the paper: the reported deaths, the Table 1 subgroup rows (by sex, race, payer and birth weight, each summing to about 18,533 survivors) and the yearly trend, whose values average about 24%. No count, denominator or subgroup in the paper produces 15.4%. The paper presents these rates as a national benchmark for prenatal counselling, so a reader of the abstract alone would understate survival in this group by 8.7 percentage points (36% in relative terms). None of the paper's trend or regression results depend on the abstract figure.
What the paper reports
Survival rates for infants <24 weeks and with completed 24 weeks were 15.4% and 71.6%, respectively
— Abstract, Findings
In the group of infants with GA <24 weeks (N = 77,050), only 18,533 (24.1%) survived
— Results, 2nd paragraph (the same sentence goes on: "in infants born with completed 24 weeks of gestation (n = 47,295), a total of 33,872 (71.6%) survived")
mortality among all periviable infants was 71,940 (57.86%)
— Results, 1st paragraph (periviable infants: 124,345, "of those, 77,050 infants were born <24 weeks, and 47,295 were born at 24 weeks of gestation")
The study presented the national survival of 24.1% and 71.6% for preterm infants with GA <24 weeks and completed 24 weeks, respectively
— Discussion, 1st paragraph
The majority (71.6%) of infants born with GA of 24 weeks survived, whereas only 24.1% of infants <24 weeks survived
— Discussion, 2nd paragraph
In infants with GA <24 weeks, survival started at 18.4% in 2007 and was 31.9% by the year 2018
— Results, trend paragraph (also in Research in context: "survival of infants born at GA <24 weeks increased from 18.4% to 31.9%")
Female | 34,939 | 9124 (26.1) Male | 41,768 | 9409 (22.5)
— Table 1, Gender rows, columns for GA <24 weeks (Total | Alive (%)). The footer says "All numbers represent weighted data."
This study provides national benchmarks that can be used during prenatal counselling
— Conclusions
What the paper's own numbers show
Method. I took the counts the paper reports for infants born before 24 weeks (total, survivors, deaths across all periviable infants, and every Table 1 subgroup row) and recomputed the survival percentage in each way the paper's numbers allow. I then looked for any count, denominator or subgroup that would produce 15.4%. As a check against the trend, I read the yearly values off the paper's Figure 1 (upper panel, dotted line) by pixel digitization. This was calibrated on the axis labels and matches the two stated endpoints within 0.2 points. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Survival, GA <24 wk, 2007–2018 | Abstract, Findings | 15.4% | 18,533 / 77,050 = 24.05% |
| Survival, GA <24 wk, 2007–2018 | Results 2nd para; Discussion 1st and 2nd paras | 18,533 (24.1%) | 24.05% (agrees) |
| Survival, completed 24 wk | Abstract; Results; Discussion | 33,872 / 47,295 (71.6%) | 71.62% (agrees in all places) |
| Group sizes | Abstract; Results 1st para | 77,050 + 47,295; total 124,345 | 77,050 + 47,295 = 124,345 (agrees) |
| Mortality, all periviable | Results 1st para | 71,940 (57.86%) | 71,940 / 124,345 = 57.86%. Survivors 124,345 − 71,940 = 52,405 = 18,533 + 33,872 exactly, so 52,405 − 33,872 = 18,533 survivors <24 wk (24.05%) |
| Table 1 <24 wk survivors, rows summed | Table 1 | subgroup counts | sex 18,533 / 76,707 = 24.16%; race 18,535 / 77,050 = 24.06%; payer 18,533 / 77,051 = 24.05%; birth weight 18,534 / 77,051 = 24.05%; income 18,202 / 75,830 = 24.00% |
| Survivors needed for 15.4% | derived | — | 0.154 × 77,050 = 11,866 (11,827–11,904 round to 15.4%); no such count appears in the paper |
| Alternative denominator | derived | — | 18,533 / 124,345 (all ≤24 wk) = 14.90%, not 15.4%; 18,533 would need a denominator of 120,344 |
| Ratio of 24-wk to <24-wk survival | Abstract vs data | 71.6 / 15.4 = 4.65-fold | 71.62 / 24.05 = 2.98-fold |
| Yearly survival, <24 wk | Research in context; Results trend para; Discussion | 18.4% (2007) to 31.9% (2018) | Figure 1 reading (approx.): 18.2, 14.5, 21.0, 20.4, 21.0, 24.6, 24.6, 26.2, 28.7, 29.7, 30.3, 32.0; unweighted mean 24.27% |
| Pooled 15.4% vs yearly values | derived | — | Only 2008 (≈14.5%) lies below 15.4%. A pooled 15.4% would need at least (18.2 − 15.4) / (18.2 − 14.5) = 76% of the 12-year cohort to fall in 2008 |
Other inconsistencies noted in passing (not part of the main finding, and not needed for it):
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Survival by sex, completed 24 wk | Table 1 vs Results 2nd para, Abstract and Figure 1 lower panel | Female 17,635 / 25,480 (69.2); Male 16,213 / 21,766 (74.5); the text says "Survival among males was lower than females in infants <24 weeks (p < 0.001) and completed 24 weeks (p < 0.001)" | 69.2% and 74.5%: the counts and percentages agree, but they show higher crude survival in males, the opposite of the text |
| Survival by birth weight, completed 24 wk | Table 1 vs Results (four-group percentages 45.1% for ≤500 g and 73.8% for >500 g) | ≤500 g: 43,772 / 32,283 (45.1); >500 g: 3,523 / 1,590 (73.8) | 32,283 / 43,772 = 73.8% and 1,590 / 3,523 = 45.1%: each row's counts give the other row's printed percentage |
| Median length of stay, survivors | Results vs Table 2 | text 113 and 114 days; Table 2 118 and 110 days | not reconcilable from the reported numbers |
| Median charges, survivors | Results vs Table 2 (and Discussion) | text $1,050,953 and $1,016,751; Table 2 (and Discussion) $926,105 and $895,137 | not reconcilable from the reported numbers |
Figure
Figure 1. The contradiction in the paper's own text. Left: the abstract sentence, with the value at issue (15.4%, orange, as reported). Right: the same quantity elsewhere in the paper: the Results count of 18,533 survivors out of 77,050 (24.1%), the two Discussion statements of 24.1%, the Table 1 survivors by sex (9,124 + 9,409 = 18,533) and the stated trend endpoints. The connector marks that both sides describe the same group (GA <24 weeks, N = 77,050), the same period (2007–2018) and the same outcome (survival to discharge or transfer). Bottom (blue, recomputed): 18,533 / 77,050 = 24.05%. For comparison, 15.4% would require 11,866 survivors, and the nearest alternative denominator (all 124,345 periviable infants) gives 14.90%, which does not round to 15.4%. Values come from the paper or from recompute.py.txt.
Figure 2. Survival before 24 weeks on one percentage axis starting at 0. Orange circles are values reported in the paper. Blue diamonds are values recomputed from the paper's own counts: the direct ratio, the route through the reported deaths, and the Table 1 subgroup rows summed by sex, race/ethnicity, payer and birth weight. Every route drawn lands between 24.05% and 24.16% (the income rows, which cover 75,830 of the 77,050 infants and are not drawn, give 24.00%), matching the 24.1% in the Results and Discussion. Only the abstract sits at 15.4%. The bottom row shows that the one plausible alternative denominator gives 14.90%, not 15.4%. The shaded band runs from the reported 2007 value (18.4%) to the reported 2018 value (31.9%). It marks the stated endpoints, not the full yearly range (the paper's Figure 1 shows a lower value, about 14.5%, in 2008). The yearly values themselves are not drawn.
Why it matters
- It is the abstract's headline outcome. The paper's primary outcome is survival, and its stated purpose is to supply "national benchmarks that can be used during prenatal counselling". The abstract figure for infants before 24 weeks is 8.7 percentage points (36% relative) below what the paper's data show: about 1 survivor in 6.5 births instead of 1 in 4.2.
- The abstract is the most widely read and indexed version. The PubMed record (PMID 36777383) carries the same sentence with 15.4%, and so do other abstract indexes. Readers who do not open the full text cannot see the 24.1% in the body.
- It changes the contrast the abstract draws. The abstract implies 24-week infants survive 4.65 times as often as infants under 24 weeks (71.6 / 15.4). The data give 2.98 times (71.62 / 24.05).
- What does not change. The 24-week figure (71.6%), the upward trends (Z = 9.438 and Z = 3.30), the adjusted odds ratios, the IVH and NEC trends and the cost analysis do not use the abstract figure. The paper's conclusions are worded qualitatively and stand as written. The issue is the size of the headline estimate in the abstract.
Explanations considered
- A different denominator. The abstract names the group explicitly ("infants <24 weeks", following "77,050 infants <24 weeks"). The only other natural denominator, all 124,345 periviable infants, gives 18,533 / 124,345 = 14.90%, which does not round to 15.4%. To get 15.4% from 18,533 survivors, the denominator would have to be 120,344, and that number does not appear in the paper. Does not account for the numbers.
- A different count of survivors (for example unweighted or from an earlier extract). The Methods say "This study used weighted data for all analyses", and the Table 1 footer says all numbers are weighted. 15.4% of 77,050 is 11,866 survivors. No count near that figure appears in the text or tables, and every Table 1 subgroup row, plus the deaths reported in the Results, adds up to about 18,533. Nothing in the paper supports this explanation.
- A subgroup, a single year or a sub-period. No Table 1 row for infants before 24 weeks prints a value from 15.0% to 15.9%. The nearest are the "other payer" group (14.7%) and the ≤500 g group (7.4%), and the abstract sentence refers to all infants under 24 weeks, not to a subgroup. In the paper's Figure 1, one year (2008, about 14.5%) lies below 15.4%. So a pooled 15.4% is not strictly impossible from the trend alone, as an initial screening of this paper assumed. It would, however, require at least 76% of the twelve-year cohort to be born in 2008, against an even spread of about 6,400 per year. The unweighted mean of the yearly values (24.27%) agrees with 24.1%.
- Adjusted or model-based estimate. The survival percentages in the abstract, Results and Table 1 are crude weighted proportions. The paper reports odds ratios, not adjusted survival rates. Its companion figure in the same abstract sentence (71.6%) is the crude proportion, matching 33,872 / 47,295.
- Digit transposition. "15.4" is not a rearrangement of "24.1", so a simple transposition does not explain it. The figure has no evident arithmetic origin in the paper.
- A rendering or conversion artefact. The value reads 15.4% in the archived PMC text, in the publisher-typeset PDF deposited in PMC, in the Europe PMC full-text XML and in the PubMed abstract. The body text reads 24.1% in all of them. This is not an artefact of one rendering.
None of these accounts for 15.4%. The paper's counts, its tables and its trend all support 24.1%.
Status
As of 2026-10-23:
- Version of record. The publisher pages (ScienceDirect, thelancet.com) returned HTTP 403 or a bot challenge, and the DOI resolves only to a redirect page. I checked the publisher-typeset PDF instead: the 8-page PDF served from the PMC article page (
pmc.ncbi.nlm.nih.gov/articles/PMC9903864/pdf/main.pdf, document metadata "Creator: Elsevier", "The Lancet Regional Health - Americas, 14 (2022) 100330"). It carries 15.4% in the abstract, 18,533 (24.1%) in the Results, 24.1% twice in the Discussion, and the same Table 1 and Table 2 values used here. The Europe PMC full-text XML and the PubMed record (PMID 36777383, E-utilities efetch) agree: the indexed abstract reads 15.4%. Every quote above was checked verbatim against the PMC-archived text withsrc/show.py, so this is not a conversion artefact. - Supplements. The Europe PMC supplementary bundle contains the images of Figures 1 and 2 and
mmc1.docx(Supplementary Table S1: ICD-9/ICD-10 codes only). There are no survival counts or source data, so nothing in the supplement resolves the discrepancy. I used the paper's Figure 1 only for the approximate yearly reading described above (the 2008 dip below 15% is visible on the image); it is not reproduced here. - Corrections and notices. On 2026-10-23,
src/known.pyreturned no Crossref updates, PMC notices or Europe PMC notices. The Crossref REST record has noupdate-to,updated-byorrelationentries, and the Crossref filterupdates:10.1016/j.lana.2022.100330returns 0 works. The Europe PMC record has no comment/correction list, and the PubMed record has no CommentsCorrections links. An earlier web search in this verification found no erratum; I could not repeat it in this step because the session's web search quota was used up. - PubPeer. The PubPeer search page and its extension API returned HTTP 403 (Cloudflare challenge), so PubPeer could not be checked directly. An earlier web search for the DOI on PubPeer found no thread.
- Sources. https://pmc.ncbi.nlm.nih.gov/articles/PMC9903864/ ; https://pubmed.ncbi.nlm.nih.gov/36777383/ ; https://api.crossref.org/works/10.1016/j.lana.2022.100330 ; Europe PMC REST (core search for PMID 36777383, full-text XML and supplementaryFiles for PMC9903864).
Suggested correction
In the Abstract, Findings, replace "Survival rates for infants <24 weeks and with completed 24 weeks were 15.4% and 71.6%, respectively" with "Survival rates for infants <24 weeks and with completed 24 weeks were 24.1% and 71.6%, respectively", consistent with 18,533 / 77,050 in the Results, Discussion and Table 1. The same correction should be passed to PubMed and other indexes. The authors may also wish to check:
- the Table 1 rows for completed 24 weeks by sex (69.2% female vs 74.5% male), which run opposite to the text, the abstract odds ratio and Figure 1;
- the Table 1 rows for completed 24 weeks by birth weight, where each row's counts match the other row's percentage (for example, 1,590 / 3,523 = 45.1%, the value the text gives for ≤500 g); and
- the medians for length of stay and charges in the Results text (113 and 114 days; $1,050,953 and $1,016,751), which differ from Table 2 (118 and 110 days; $926,105 and $895,137).
Recomputation script (python recompute.py.txt)
"""Recompute the survival figures in PMC9903864 (Lancet Reg Health Am 2022,
doi:10.1016/j.lana.2022.100330) and regenerate fig1/fig2 in this folder.
Run: python recompute.py.txt
Needs numpy and matplotlib.
Every input below is copied from the article as archived in PubMed Central,
except YEARLY_LT24_APPROX, which is an approximate pixel reading of the
paper's Figure 1 (upper panel, dotted line). That reading is used only for the
pooling check in section 5 and is not drawn in either figure.
"""
from pathlib import Path
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = Path(__file__).resolve().parent
# ------------------------------------------------------------------ inputs (paper)
N_ALL = 124_345 # Abstract; Results, 1st paragraph: infants born at GA <=24 wk
N_LT24 = 77_050 # Abstract; Results, 1st paragraph; Table 1 header
N_24 = 47_295 # Abstract; Results, 1st paragraph; Table 1 header
ALIVE_LT24 = 18_533 # Results, 2nd paragraph: "only 18,533 (24.1%) survived"
ALIVE_24 = 33_872 # Results, 2nd paragraph: "a total of 33,872 (71.6%) survived"
DEATHS_ALL = 71_940 # Results, 1st paragraph: "mortality among all periviable infants was 71,940 (57.86%)"
ABSTRACT_LT24 = 15.4 # Abstract, Findings
ABSTRACT_24 = 71.6 # Abstract, Findings
RESULTS_LT24 = 24.1 # Results, 2nd paragraph; Discussion, 1st and 2nd paragraphs
TREND_2007, TREND_2018 = 18.4, 31.9 # Research in context; Results trend para; Discussion
# Table 1, GA <24 weeks: (total, alive) per row
T1_LT24 = {
"sex": [(34_939, 9_124), (41_768, 9_409)],
"race": [(22_834, 5_129), (21_632, 5_809), (11_801, 3_173), (20_783, 4_424)],
"payer": [(37_434, 10_607), (28_728, 6_326), (10_889, 1_600)],
"income": [(27_011, 6_706), (18_603, 4_577), (17_113, 4_017), (13_103, 2_902)],
"birth weight": [(36_392, 2_698), (40_659, 15_836)],
}
# Table 1, completed 24 weeks: (total, alive, printed %)
T1_24_SEX = {"Female": (25_480, 17_635, 69.2), "Male": (21_766, 16_213, 74.5)}
T1_24_BW = {"<=500 g": (43_772, 32_283, 45.1), ">500 g": (3_523, 1_590, 73.8)}
# Results trend text: four GA x BW groups
FOUR_GROUPS_TEXT = {"<24 wk, <=500 g": 7.4, "<24 wk, >500 g": 38.9,
"24 wk, <=500 g": 45.1, "24 wk, >500 g": 73.8}
# Results text vs Table 2 (surviving infants): medians
LOS_TEXT, LOS_T2 = (113, 114), (118, 110)
CHARGES_TEXT, CHARGES_T2 = (1_050_953, 1_016_751), (926_105, 895_137)
# Approximate pixel reading of Figure 1, upper panel, dotted line (<24 wk), 2007..2018.
# Calibrated on the 0%..80% axis labels; it reproduces the stated endpoints within 0.2 pt.
YEARS = list(range(2007, 2019))
YEARLY_LT24_APPROX = [18.2, 14.5, 21.0, 20.4, 21.0, 24.6, 24.6, 26.2, 28.7, 29.7, 30.3, 32.0]
def pct(a, b):
return 100.0 * a / b
out = {}
print("=" * 74)
print("1. Headline survival, GA <24 weeks and completed 24 weeks")
out["lt24"] = pct(ALIVE_LT24, N_LT24)
out["w24"] = pct(ALIVE_24, N_24)
print(f" N_LT24 + N_24 = {N_LT24 + N_24:,} (reported {N_ALL:,})")
print(f" <24 wk : 18,533 / 77,050 = {out['lt24']:.2f}% (Abstract {ABSTRACT_LT24}%, Results/Discussion {RESULTS_LT24}%)")
print(f" 24 wk : 33,872 / 47,295 = {out['w24']:.2f}% (Abstract {ABSTRACT_24}%, Results 71.6%)")
print(f" difference Abstract vs recomputed: {out['lt24'] - ABSTRACT_LT24:.2f} percentage points; "
f"relative understatement {100 * (1 - ABSTRACT_LT24 / out['lt24']):.0f}%")
print(f" 1 survivor per {100 / ABSTRACT_LT24:.1f} births (15.4%) vs 1 per {100 / out['lt24']:.1f} births (24.05%)")
print(f" 24 wk / <24 wk survival ratio: abstract {ABSTRACT_24 / ABSTRACT_LT24:.2f}x, data {out['w24'] / out['lt24']:.2f}x")
print("\n2. Route via the reported deaths (Results, 1st paragraph)")
surv_all = N_ALL - DEATHS_ALL
out["mort"] = pct(DEATHS_ALL, N_ALL)
out["via_deaths"] = pct(surv_all - ALIVE_24, N_LT24)
print(f" mortality 71,940 / 124,345 = {out['mort']:.2f}% (reported 57.86%)")
print(f" survivors 124,345 - 71,940 = {surv_all:,}; 18,533 + 33,872 = {ALIVE_LT24 + ALIVE_24:,}")
print(f" <24 wk survivors implied = {surv_all - ALIVE_24:,} -> {out['via_deaths']:.2f}% of 77,050")
print("\n3. Table 1 subgroup rows, GA <24 weeks, summed")
out["t1"] = {}
for name, rows in T1_LT24.items():
T = sum(r[0] for r in rows)
A = sum(r[1] for r in rows)
out["t1"][name] = pct(A, T)
print(f" by {name:<13}: total {T:>6,} alive {A:>6,} -> {pct(A, T):.2f}%")
print("\n4. What would be needed for the abstract value")
need = ABSTRACT_LT24 / 100 * N_LT24
print(f" survivors needed for 15.4% of 77,050: {need:,.0f} (no such count in the paper)")
lo, hi = (ABSTRACT_LT24 - 0.05) / 100 * N_LT24, (ABSTRACT_LT24 + 0.05) / 100 * N_LT24
print(f" range that rounds to 15.4%: {lo:,.0f} to {hi:,.0f} survivors")
out["alt_den"] = pct(ALIVE_LT24, N_ALL)
print(f" alternative denominator 18,533 / 124,345 (all periviable) = {out['alt_den']:.2f}%")
print(f" denominator that would turn 18,533 into 15.4%: {ALIVE_LT24 / (ABSTRACT_LT24 / 100):,.0f}")
print(f" all survivors / all periviable: {pct(ALIVE_LT24 + ALIVE_24, N_ALL):.2f}%")
print(f" ungrouped Table 1 rows (<24 wk) that print 15.x%: "
f"{[round(pct(a, t), 1) for rows in T1_LT24.values() for t, a in rows if 15 <= pct(a, t) < 16] or 'none'}")
print(f" digit check: '15.4' is a permutation of '24.1'? {sorted('15.4') == sorted('24.1')}")
print("\n5. Pooling check against the yearly trend")
eq = (ABSTRACT_LT24 * 12 - TREND_2007 - TREND_2018) / 10
eq2 = (out["lt24"] * 12 - TREND_2007 - TREND_2018) / 10
print(f" equal yearly weights, stated endpoints only: other 10 years must average {eq:.2f}% for 15.4%, "
f"{eq2:.2f}% for 24.05%")
y = np.array(YEARLY_LT24_APPROX)
print(f" Figure 1 reading (approx.): {', '.join(f'{v:.1f}' for v in y)}")
print(f" reading vs stated endpoints: 2007 {y[0]:.1f} vs {TREND_2007}; 2018 {y[-1]:.1f} vs {TREND_2018}")
print(f" unweighted mean of yearly values: {y.mean():.2f}%; minimum {y.min():.1f}% ({YEARS[int(y.argmin())]})")
others_min = np.delete(y, y.argmin()).min()
share = (others_min - ABSTRACT_LT24) / (others_min - y.min())
print(f" for a pooled 15.4%, the {YEARS[int(y.argmin())]} share of the cohort must be at least "
f"({others_min:.1f} - 15.4) / ({others_min:.1f} - {y.min():.1f}) = {100 * share:.0f}%")
print(f" average annual cohort if evenly spread: 77,050 / 12 = {N_LT24 / 12:,.0f}")
print("\n6. Side issues in Table 1 / Table 2 (not part of the main finding)")
for k, (t, a, p) in T1_24_SEX.items():
print(f" 24 wk {k:<6}: {a:,}/{t:,} = {pct(a, t):.1f}% (printed {p})")
print(" -> Table 1 shows higher crude survival for males at 24 wk; the text says males lower")
for k, (t, a, p) in T1_24_BW.items():
print(f" 24 wk {k:<7}: {a:,}/{t:,} = {pct(a, t):.1f}% (printed {p}; text gives {k} = "
f"{FOUR_GROUPS_TEXT['24 wk, ' + k]}%)")
print(" -> each 24-wk birth-weight row's counts give the other row's printed percentage")
print(f" 24 wk sex rows sum: total {sum(v[0] for v in T1_24_SEX.values()):,}, "
f"alive {sum(v[1] for v in T1_24_SEX.values()):,}")
print(f" LOS median, text {LOS_TEXT} vs Table 2 {LOS_T2} days")
print(f" charges median, text {CHARGES_TEXT} vs Table 2 {CHARGES_T2} USD")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(HERE / f"{stem}.svg")
fig.savefig(HERE / f"{stem}.png", dpi=200)
plt.close(fig)
# Figure 1: abstract vs body contradiction diagram
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.6)
ax.axis("off")
def panel(x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.14, y + h - 0.2, title, fontsize=11, fontweight="bold", va="top")
# left: abstract
panel(0.2, 1.55, 2.85, 2.65, "Abstract, Findings", ORANGE)
ax.text(0.34, 3.72, "“Survival rates for infants\n<24 weeks and with completed\n24 weeks were",
fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(0.34, 2.82, "15.4%", fontsize=22, fontweight="bold", color=ORANGE, va="top")
ax.text(1.48, 2.72, "and 71.6%,\nrespectively”", fontsize=10, va="top", color=INK2,
linespacing=1.35)
ax.text(0.34, 1.95, "N = 77,050 (<24 wk)", fontsize=10, color=INK2, va="top")
# right: results / discussion / table 1
panel(4.15, 1.55, 2.85, 2.65, "Results, Discussion, Table 1", INK2)
ax.text(4.29, 3.72, "Results: “(N = 77,050), only\n18,533 (24.1%) survived”",
fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(4.29, 3.0, "Discussion (twice): 24.1%", fontsize=10, va="top", color=INK2)
ax.text(4.29, 2.6, "Table 1, <24 wk alive by sex:\n9,124 + 9,409 = 18,533",
fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(4.29, 1.95, "Trend: 18.4% (2007) to 31.9% (2018)", fontsize=10, color=INK2, va="top")
# connector
ax.add_patch(FancyArrowPatch((3.12, 2.9), (4.08, 2.9), arrowstyle="<->", mutation_scale=14,
lw=2, color=INK))
ax.text(3.6, 3.2, "conflict", ha="center", fontsize=10, fontweight="bold")
ax.text(3.6, 2.7, "same group,\nperiod and\noutcome", ha="center", va="top",
fontsize=10, color=INK2, linespacing=1.3)
# recomputation strip
ax.add_patch(FancyBboxPatch((0.2, 0.3), 6.8, 0.95, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=BLUE, lw=1.5))
ax.text(0.34, 1.07, "Recomputed from the paper's own counts", fontsize=11, fontweight="bold",
va="top")
ax.text(0.34, 0.68, f"18,533 / 77,050 = {out['lt24']:.2f}%", fontsize=13, fontweight="bold",
color=BLUE, va="top")
ax.text(3.35, 0.73, f"15.4% would need {need:,.0f} survivors;\n"
f"18,533 / 124,345 (all ≤24 wk) = {out['alt_den']:.2f}%",
fontsize=10, color=INK2, va="top", linespacing=1.35)
ax.legend(handles=[Line2D([], [], marker="s", ls="", ms=9, color=ORANGE, label="value as reported (abstract)"),
Line2D([], [], marker="s", ls="", ms=9, color=BLUE, label="recomputed from paper's counts")],
loc="upper center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: dot comparison on one percentage axis starting at 0
rows = [
("Abstract, Findings", ABSTRACT_LT24, "rep"),
("Results; Discussion (×2)", RESULTS_LT24, "rep"),
("18,533 / 77,050", out["lt24"], "rec"),
("(124,345 − 71,940 deaths − 33,872) / 77,050", out["via_deaths"], "rec"),
("Table 1 rows summed: sex", out["t1"]["sex"], "rec"),
("Table 1 rows summed: race/ethnicity", out["t1"]["race"], "rec"),
("Table 1 rows summed: payer", out["t1"]["payer"], "rec"),
("Table 1 rows summed: birth weight", out["t1"]["birth weight"], "rec"),
("Alternative denominator: 18,533 / 124,345", out["alt_den"], "rec"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.subplots_adjust(left=0.47, right=0.97, top=0.86, bottom=0.13)
n = len(rows)
ypos = np.arange(n)[::-1]
ax.axvspan(TREND_2007, TREND_2018, color="#efeeea", zorder=0)
ax.text((TREND_2007 + TREND_2018) / 2, n - 0.35, "shaded: from the reported 2007\n(18.4%) to 2018 (31.9%) value",
ha="center", va="bottom", fontsize=10, color=INK2)
for yv, (lab, v, kind) in zip(ypos, rows):
c = ORANGE if kind == "rep" else BLUE
mk = "o" if kind == "rep" else "D"
ax.plot([0, v], [yv, yv], color=GRID, lw=1, zorder=1)
ax.plot(v, yv, marker=mk, ms=9, color=c, mec=BG, mew=1.5, zorder=3, ls="")
ax.text(v + 0.9, yv, f"{v:.2f}%" if kind == "rec" else f"{v:.1f}%", va="center",
fontsize=10, color=INK)
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlim(0, 40)
ax.set_ylim(-0.7, n + 0.6)
ax.set_xticks(range(0, 41, 5))
ax.set_xticklabels([f"{t}%" for t in range(0, 41, 5)])
ax.set_xlabel("Survival, GA <24 weeks, 2007–2018 pooled")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="reported in the paper"),
Line2D([], [], marker="D", ls="", ms=9, color=BLUE, label="recomputed from the paper's counts")],
loc="lower left", bbox_to_anchor=(-0.9, 1.08), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nfigures written:", ", ".join(sorted(p.name for p in HERE.glob("fig*.*"))))
Abstract reports all-cause grade ≥3 adverse event figures as treatment-related adverse events
Lancet Reg Health Eur (2024) · doi:10.1016/j.lanepe.2024.100875 · PMC10924203
Summary
This prospective real-world cohort of 151 patients treated with cemiplimab names grade ≥3 treatment-related adverse events (TRAEs) as one of its primary endpoints. The Abstract reports "Sixty-eight TRAEs occurred in 29.8% of patients." In the Results, 68 events in 45/151 patients (29.8%) is the count for grade ≥3 adverse events of any attribution. Grade ≥3 TRAEs were 42 events in 28/151 patients (18.5%), and the entries of Table 4 sum to 42. The Abstract therefore reports a primary safety endpoint with the all-cause figures, which overstate it by 26 events and 11.3 percentage points. The direction is unfavourable to the drug, so the paper's conclusion that cemiplimab was safe is not weakened, but anyone who takes the safety rate from the Abstract gets the wrong number.
What the paper reports
Abstract, Methods (primary endpoints):
Primary endpoints were objective clinical benefit rate (CBR), defined as objective response (OR) or stable disease (SD) at 16 weeks, physician-assessed CBR, defined as clinician’s documentation of improved disease or SD based on evaluation of all available clinical parameters at 16 weeks, objective response rate (ORR), and safety, defined as grade ≥ 3 treatment related adverse events (TRAEs) occurring up to 30 days after last drug administration.
Abstract, Findings:
Between February 2021 and December 2022, 151 patients started treatment.
Sixty-eight TRAEs occurred in 29.8% of patients. Most commonly reported TRAE was a kidney transplant rejection (9.5%).
Abstract, Interpretation:
Cemiplimab proved highly effective and safe in this real-world cohort of patients with laCSCC or mCSCC, confirming its therapeutic value in the treatment of advanced CSCC in daily clinical practice.
Methods:
Safety was measured by the frequency of grade ≥ 3 treatment related adverse events (TRAEs) and serious adverse events (SAEs) occurring up to 30 days after the last administration of drug.
All patients that received at least one administration of cemiplimab were included in the efficacy and safety analyses.
Results, Safety (the spelling "TREAs" is the paper's):
A total of 68 AEs of grade ≥ 3 occurred in 45 of 151 patients (29.8%). Of all grade ≥ 3 AEs, 42 (61.8%) were TREAs (Table 4), which occurred in 28 of 151 patients (18.5%). Of note, the most commonly reported TRAE was rejection of a kidney transplant (n = 4, 9.5%).
Other commonly reported TRAEs were hypertension (n = 3, 4.8%) and increased gamma glutamyl transferase (n = 3, 4.8%).
Table 4 title:
Possibly, probably, or definitely related adverse events grade ≥ 3.
Table 4 lists 31 event types with counts by grade 3, 4 and 5. Its only footnote reads "These were originally reported as acute kidney injuries", referring to the kidney transplant rejection row (2 grade 3, 2 grade 4).
What the paper's own numbers show
Method. I summed the grade 3, 4 and 5 columns of Table 4. I then recomputed each safety proportion from the counts printed in the Results, using N = 151, the treated cohort that the Methods say forms the safety population. For the per-event percentages (kidney transplant rejection, hypertension, GGT increase), I listed every denominator that reproduces the printed percentage after rounding to one decimal. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Grade ≥3 TRAE events | Abstract, Findings | "Sixty-eight TRAEs" | 42 (Table 4: 34 grade 3 + 7 grade 4 + 1 grade 5; Results also say 42) |
| Patients with a grade ≥3 TRAE | Abstract, Findings | 29.8% | 28/151 = 18.5% (Results) |
| All grade ≥3 AEs, any attribution | Results, Safety | 68 AEs in 45/151 (29.8%) | 45/151 = 29.8%; 45 is the only count out of 151 that rounds to 29.8% |
| Share of grade ≥3 AEs that were TRAEs | Results, Safety | 42 (61.8%) | 42/68 = 61.8% |
| Grade ≥3 TRAEs, total in Table 4 | Table 4 | (no total printed) | 42 events across 31 rows |
| Abstract minus TRAE figures | derived | — | 68 − 42 = 26 events (1.62×); 45 − 28 = 17 patients; 29.8 − 18.5 = 11.3 percentage points (1.61×) |
| Kidney transplant rejection | Abstract; Results | 9.5% (n = 4) | 4/42 = 9.5% of TRAE events; 42 is the only denominator that gives 9.5%. As a share of patients, 4/151 = 2.6% |
| Hypertension; GGT increase (side finding) | Results, Safety | n = 3, 4.8% each | 3/42 = 7.1% on the kidney-rejection denominator; 4.8% needs a denominator of 62 or 63 (3/68 = 4.4%) |
| Fatal AEs | Results, Safety | 3 (2%) | 3/151 = 2.0% |
| Discontinued because of an AE | Results, Safety | 15 (9.9%) | 15/151 = 9.9% |
| Kidney rejection among transplant recipients | Discussion | 4 of 7 (57.1%) | 4/7 = 57.1% |
Both numbers in the Abstract sentence, 68 events and 29.8%, equal the Results figures for all grade ≥3 adverse events. Neither equals the TRAE figures (42 events, 18.5%). The next Abstract sentence, which gives kidney transplant rejection as 9.5%, uses the TRAE event count of 42 as its denominator. So the two consecutive Abstract sentences rest on different event totals. All the other safety percentages in the Results reproduce from their counts, apart from the 4.8% figures noted as a side finding.
Figure
Figure 1. Left: the Abstract's definition of the safety endpoint and its two safety sentences, with the values it prints in orange. Right: the Results, Safety paragraph and Table 4, with the TRAE values recomputed from them in blue. The grey arrow shows that the Abstract's "68 TRAEs ... 29.8% of patients" equals the Results figures for all grade ≥3 AEs regardless of attribution (68 events, 45/151 patients). The dashed orange arrow marks the mismatch with the Results figures for treatment-related events (42 events, 28/151 = 18.5%). The blue arrow shows that the Abstract's kidney rejection figure, 9.5%, is 4/42, so it uses the TRAE count that the preceding sentence does not. All excerpts are verbatim; "[sic]" marks the paper's spelling "TREAs".
Figure 2. Percentage of the 151 treated patients with at least one grade ≥3 event. Top row: the primary safety endpoint (grade ≥3 TRAE), as printed in the Abstract (orange, 29.8%) and as computed from the Results count of 28 patients (blue, 18.5%). Bottom row: any grade ≥3 AE, computed from the Results count of 45 patients (blue, 29.8%). The Abstract's value lies on the all-cause row's value, 11.3 percentage points above the treatment-related rate. The axis starts at zero.
Why it matters
The Abstract names grade ≥3 TRAEs as a primary endpoint and gives one number for it. Readers who see only the Abstract (the PubMed record, indexing services, systematic reviews and health technology assessments that extract abstract-level data) will take the real-world grade ≥3 TRAE rate of cemiplimab in this cohort to be 29.8% of patients, with 68 events. The paper's own Results give 18.5% of patients and 42 events. The relative overstatement is about 1.6-fold. Real-world cohorts like this one are cited mainly to compare toxicity in older, frailer patients with the registration trials, and the Discussion notes that the cohort was older (median 78 years vs 72) and included patients with ECOG ≥ 2, autoimmune disease and organ transplants. A rate inflated by attribution-independent events would make the drug look more toxic in this population than the paper's data show.
The error does not reverse any conclusion. The correct TRAE rate is lower, so the Interpretation, "Cemiplimab proved highly effective and safe", is if anything better supported. The efficacy endpoints, the all-cause safety figures in the Results, Table 4, the fatal-AE and discontinuation figures, and the kidney transplant findings are unaffected. The title, the Research-in-context panel and the closing paragraphs of the Discussion (the article has no separate Conclusions section) give no safety number.
The side finding is minor. Hypertension and GGT increase (3 events each) are given as 4.8%, which no denominator printed in the paper reproduces: 3/42 = 7.1%, 3/68 = 4.4%, 3/28 = 10.7%. It does not change the ranking of events.
Explanations considered
- "TRAEs" in the Abstract is loose wording for adverse events in general. This does not account for the numbers. The Abstract's own Methods sentence defines the safety endpoint as "grade ≥ 3 treatment related adverse events (TRAEs)", and the paper's Methods use the same definition. The next Abstract sentence also uses "TRAE" in the treatment-related sense, with a percentage computed on 42 TRAEs.
- 68 is a count of TRAEs of all grades, or of a different category. This does not account for the numbers. The paper reports no all-grade TRAE count anywhere. The pair 68 events / 29.8% equals, in both numbers, the Results figure for all grade ≥3 AEs, and 45 is the only patient count out of 151 that rounds to 29.8%. A second category that happens to match both would be a coincidence.
- A different population or denominator. This does not account for the numbers. The Abstract and the Results both use the 151 treated patients, and the Methods state that all patients with at least one dose form the safety population. 28/151 = 18.5% and 45/151 = 29.8% both reproduce exactly.
- A different observation window. This does not account for the numbers. The Abstract and the Methods use the same window (up to 30 days after the last dose), and the Results give no second window.
- Table 4 covers only a subset of TRAEs. This does not account for the numbers. Table 4 is titled as the possibly, probably or definitely related grade ≥3 events, and it sums to 42, the same count the Results give for TRAEs.
- A conversion artifact in the archived text. This is excluded. The publisher's typeset PDF deposited in PMC, the Europe PMC full-text XML and the PubMed abstract all carry the same sentence (see Status).
- The 9.5% kidney rejection figure shows that the Abstract uses another TRAE total. This does not account for the numbers. 9.5% reproduces only with 42 as the denominator (4/42), which is the Results TRAE count, not 68 (4/68 = 5.9%). The 9.5% is a share of TRAE events, not of patients (4/151 = 2.6%); the Abstract's wording "(9.5%)" does not say which, but the Results give the same figure as an event count.
- The hypertension and GGT percentages (4.8%) use a different but stated denominator. No such denominator is stated. 4.8% reproduces only with 62 or 63 as the denominator, and neither number appears in the safety section. This is a separate, minor inconsistency and does not bear on the main finding.
Status
- Version of record. As of 2026-10-24. The publisher HTML (thelancet.com, sciencedirect.com) could not be fetched automatically: doi.org redirects to linkinghub.elsevier.com, which returns only a redirect stub, and both publisher sites returned HTTP 403. I checked the publisher's typeset PDF as deposited in the PMC Open Access dataset (PMC10924203.1.pdf; PDF metadata: Elsevier, "The Lancet Regional Health - Europe, 39 (2024) 100875", 12 pages) and the Europe PMC full-text XML. Both carry the quoted Abstract and Results sentences word for word, and Table 4 on page 10 of the PDF has the same 31 rows and values as the XML. The PubMed abstract (PMID 38464480) carries the same "Sixty-eight TRAEs occurred in 29.8% of patients" sentence. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. The single supplementary file (mmc1.pdf, 5 pages) contains Supplementary Tables 1A–1C (baseline characteristics by clinical benefit and response) and Supplementary Fig. S1 (flow diagram). Its only adverse-event content is the count of 15 discontinuations because of an adverse event. It has no grade ≥3 AE or TRAE listing that could reconcile 68 with 42 events or 45 with 28 patients. The data sharing statement says the raw data are available on request.
- Corrections and comments. I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI (update-to and updated-by absent, relation empty), and PMC and Europe PMC list no notices. A web search for corrections or comments on the article found none. The PubPeer API and search page refused automated access (Cloudflare challenge and HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC10924203; PMC Open Access PDF PMC10924203.1; Europe PMC full-text XML and supplementary files for PMC10924203; PubMed 38464480; Crossref record for 10.1016/j.lanepe.2024.100875.
Suggested correction
- In the Abstract, Findings, replace "Sixty-eight TRAEs occurred in 29.8% of patients" with the treatment-related figures given in the Results, for example: "Forty-two grade ≥ 3 TRAEs occurred in 28 of 151 patients (18.5%)." If the authors also wish to report all-cause events, they can add: "Grade ≥ 3 AEs of any attribution (68 events) occurred in 45 patients (29.8%)."
- In the Abstract, state that the kidney transplant rejection figure is a share of TRAE events (4 of 42, 9.5%), or give it per patient (4 of 151, 2.6%; 4 of the 7 transplant recipients).
- In the Results, Safety, correct the percentages for hypertension and increased GGT (3 events each) to the denominator used for kidney transplant rejection (3/42 = 7.1%), or state the denominator used. Correct the spelling "TREAs" to "TRAEs".
Recomputation script (python recompute.py.txt)
"""Recompute the grade >=3 adverse-event figures of PMC10924203
(Lancet Reg Health Eur 2024, doi:10.1016/j.lanepe.2024.100875,
cemiplimab in laCSCC/mCSCC, DRUG Access Protocol).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract, Findings: "151 patients started treatment"; "Sixty-eight TRAEs occurred in 29.8% of patients.";
"Most commonly reported TRAE was a kidney transplant rejection (9.5%)."
Results, Safety: 68 AEs of grade >=3 in 45 of 151 patients (29.8%); 42 (61.8%) were TRAEs (Table 4),
in 28 of 151 patients (18.5%); kidney transplant rejection n = 4, 9.5%;
hypertension n = 3, 4.8%; increased GGT n = 3, 4.8%; fatal AEs in three patients (2%);
15 patients (9.9%) discontinued treatment due to an AE.
Table 4: "Possibly, probably, or definitely related adverse events grade >= 3" (grade 3, 4, 5 counts).
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
N = 151 # treated cohort, all included in the safety analysis
ALL_AE_EVENTS, ALL_AE_PTS = 68, 45 # Results: all grade >=3 AEs, any attribution
TRAE_EVENTS_TXT, TRAE_PTS = 42, 28 # Results: grade >=3 TRAEs
ABS_EVENTS, ABS_PCT = 68, 29.8 # Abstract: "Sixty-eight TRAEs occurred in 29.8% of patients"
KIDNEY, HTN, GGT = 4, 3, 3
FATAL, DISC = 3, 15
TABLE4 = { # name: (grade 3, grade 4, grade 5)
"Alanine aminotransferase increased": (1, 0, 0), "Alkaline phosphatase increased": (1, 0, 0),
"Anemia": (1, 0, 0), "Aspartate aminotransferase increased": (1, 0, 0), "Colitis": (2, 0, 0),
"Creatinine increased": (1, 0, 0), "Delusions": (0, 0, 1), "Diarrhea": (2, 0, 0),
"Dizziness": (1, 0, 0), "Dyspnea": (1, 0, 0), "Fatigue": (1, 0, 0), "Fever": (1, 0, 0),
"Gamma glutamyltransferase increased": (3, 0, 0), "Hepatitis": (1, 0, 0),
"Hypercalcemia": (0, 2, 0), "Hypertension": (3, 0, 0), "Hyperthyroidism": (1, 0, 0),
"Hypophysitis": (1, 0, 0), "Hypotension": (1, 0, 0), "Interstitial nephritis": (1, 0, 0),
"Kidney transplant rejection": (2, 2, 0), "Lipase increased": (0, 1, 0),
"Nephrotic syndrome": (1, 0, 0), "Neutrophil count decreased": (0, 1, 0),
"Pancreatitis": (1, 0, 0), "Pneumonitis": (1, 0, 0), "Pruritus": (2, 0, 0),
"Rash maculo-papular": (1, 0, 0), "Sepsis": (0, 1, 0),
"Skin toxicity not otherwise specified": (1, 0, 0), "Vomiting": (1, 0, 0),
}
def pct(a, b):
return 100.0 * a / b
# ---------------------------------------------------------------- 1. Table 4
g3 = sum(v[0] for v in TABLE4.values())
g4 = sum(v[1] for v in TABLE4.values())
g5 = sum(v[2] for v in TABLE4.values())
T4 = g3 + g4 + g5
print("=== 1. Table 4 (treatment-related grade >=3 AEs) ===")
print(f" rows = {len(TABLE4)}; grade 3 = {g3}, grade 4 = {g4}, grade 5 = {g5}; total events = {T4}")
print(f" Results text says {TRAE_EVENTS_TXT} TRAEs -> {'matches' if T4 == TRAE_EVENTS_TXT else 'DIFFERS'}")
print(f" kidney transplant rejection rows: {sum(TABLE4['Kidney transplant rejection'])} events")
# ---------------------------------------------------------------- 2. proportions
print("\n=== 2. Patient and event proportions (N = 151) ===")
p_all = pct(ALL_AE_PTS, N)
p_trae = pct(TRAE_PTS, N)
share = pct(TRAE_EVENTS_TXT, ALL_AE_EVENTS)
print(f" any grade >=3 AE, patients: {ALL_AE_PTS}/{N} = {p_all:.2f}% (Results: 29.8%)")
print(f" grade >=3 TRAE, patients: {TRAE_PTS}/{N} = {p_trae:.2f}% (Results: 18.5%)")
print(f" TRAE share of grade >=3 AEs: {TRAE_EVENTS_TXT}/{ALL_AE_EVENTS} = {share:.2f}% (Results: 61.8%)")
print(f" Abstract '68 TRAEs in 29.8%': events match all-cause count ({ABS_EVENTS} == {ALL_AE_EVENTS}): "
f"{ABS_EVENTS == ALL_AE_EVENTS}; patients match all-cause rate ({ABS_PCT} == {round(p_all, 1)}): "
f"{ABS_PCT == round(p_all, 1)}; match TRAE figures ({TRAE_EVENTS_TXT}, {round(p_trae, 1)}%): "
f"{ABS_EVENTS == TRAE_EVENTS_TXT or ABS_PCT == round(p_trae, 1)}")
print(f" overstatement: events {ABS_EVENTS} - {TRAE_EVENTS_TXT} = {ABS_EVENTS - TRAE_EVENTS_TXT} "
f"({ABS_EVENTS / TRAE_EVENTS_TXT:.2f}x); patients {ALL_AE_PTS} - {TRAE_PTS} = {ALL_AE_PTS - TRAE_PTS}; "
f"percentage {p_all:.2f} - {p_trae:.2f} = {p_all - p_trae:.2f} points ({p_all / p_trae:.2f}x)")
print(f" non-related grade >=3 AEs: {ALL_AE_EVENTS - TRAE_EVENTS_TXT} events")
lo, hi = [x for x in range(1, N + 1) if round(pct(x, N), 1) == ABS_PCT][0], \
[x for x in range(1, N + 1) if round(pct(x, N), 1) == ABS_PCT][-1]
print(f" patient counts out of 151 that round to 29.8%: {lo}..{hi}")
# ---------------------------------------------------------------- 3. per-event percentages
print("\n=== 3. Denominators of the per-event percentages ===")
for lab, k, rep in [("kidney transplant rejection", KIDNEY, 9.5), ("hypertension", HTN, 4.8), ("increased GGT", GGT, 4.8)]:
print(f" {lab}: reported {rep}%; {k}/42 = {pct(k, 42):.2f}%, {k}/68 = {pct(k, 68):.2f}%, "
f"{k}/28 = {pct(k, 28):.2f}%, {k}/151 = {pct(k, N):.2f}%")
dens = [d for d in range(1, 400) if round(pct(k, d), 1) == rep]
print(f" denominators d with round({k}/d, 1 dp) = {rep}%: {dens}")
print(f" kidney rejection among transplant recipients: 4/7 = {pct(4, 7):.1f}% (Discussion: 57.1%)")
# ---------------------------------------------------------------- 4. other safety figures
print("\n=== 4. Other safety figures ===")
print(f" fatal AEs: {FATAL}/{N} = {pct(FATAL, N):.2f}% (reported 2%)")
print(f" discontinued due to AE: {DISC}/{N} = {pct(DISC, N):.2f}% (reported 9.9%)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
plt.close(fig)
# ---- Figure 1: Abstract vs Results/Table 4, two text panels joined by connectors
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 64)
ax.axis("off")
def panel(x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=edge, lw=1.5))
ax.text(x + 1.5, y + h - 2.2, title, fontsize=11, fontweight="bold", va="top", color=TXT)
# left: abstract
panel(1.5, 8, 37, 50, "Abstract", ORANGE)
ax.text(3.0, 50.5, "Methods (endpoint definition)", fontsize=10, color=SEC, va="top")
ax.text(3.0, 47.2, "“safety, defined as grade ≥ 3\ntreatment related adverse\nevents (TRAEs)”",
fontsize=10, va="top", color=TXT)
ax.text(3.0, 34.5, "Findings", fontsize=10, color=SEC, va="top")
ax.text(3.0, 31.2, "“Sixty-eight TRAEs occurred\nin 29.8% of patients.”", fontsize=10.5, va="top",
color=TXT, fontweight="bold")
ax.text(3.0, 21.0, "“Most commonly reported TRAE\nwas a kidney transplant\nrejection (9.5%).”",
fontsize=10, va="top", color=TXT)
ax.text(3.0, 9.6, "reported: 68 events, 29.8%", fontsize=10, color=ORANGE, fontweight="bold")
# right: body
panel(60.5, 8, 38, 50, "Results, Safety / Table 4", BLUE)
ax.text(62.0, 50.5, "All grade ≥ 3 AEs (any cause)", fontsize=10, color=SEC, va="top")
ax.text(62.0, 47.2, "“68 AEs of grade ≥ 3 occurred\nin 45 of 151 patients (29.8%)”",
fontsize=10, va="top", color=TXT)
ax.text(62.0, 34.5, "Treatment-related grade ≥ 3 AEs", fontsize=10, color=SEC, va="top")
ax.text(62.0, 31.2, "“42 (61.8%) were TREAs [sic]\n(Table 4), which occurred in\n28 of 151 patients (18.5%)”",
fontsize=10, va="top", color=TXT)
ax.text(62.0, 18.0, f"Table 4 sum: {g3} + {g4} + {g5} = {T4}\n"
f"{TRAE_PTS}/{N} = {p_trae:.1f}%; 4/{T4} = {pct(4, T4):.1f}%",
fontsize=10, va="top", color=BLUE)
ax.text(62.0, 9.6, f"recomputed: {T4} TRAEs, {p_trae:.1f}%", fontsize=10, color=BLUE,
fontweight="bold")
# connectors
a1 = FancyArrowPatch((39.6, 30.5), (60.0, 45.5), arrowstyle="-|>", mutation_scale=14, lw=2,
color=SEC, connectionstyle="arc3,rad=-0.15")
ax.add_patch(a1)
ax.text(49.5, 45.5, "same as\nall-cause\nAE figures", ha="center", va="bottom", fontsize=10, color=TXT)
a2 = FancyArrowPatch((39.6, 27.0), (60.0, 28.0), arrowstyle="-|>", mutation_scale=14, lw=2,
color=ORANGE, linestyle=(0, (4, 2)))
ax.add_patch(a2)
ax.text(49.5, 25.6, "not the\nTRAE figures", ha="center", va="top", fontsize=10, color=TXT)
a3 = FancyArrowPatch((39.6, 17.0), (60.0, 15.0), arrowstyle="-|>", mutation_scale=14, lw=2,
color=BLUE, connectionstyle="arc3,rad=0.1")
ax.add_patch(a3)
ax.text(49.5, 13.0, "9.5% = 4/42,\nthe TRAE\ncount", ha="center", va="top", fontsize=10, color=TXT)
ax.text(50, 62.5, "The abstract's TRAE sentence carries the all-cause grade ≥ 3 AE figures",
ha="center", va="top", fontsize=11.5, fontweight="bold", color=TXT)
ax.text(50, 3.0, "Orange: values printed in the abstract. Blue: values recomputed from the Results and Table 4.",
ha="center", va="center", fontsize=10, color=SEC)
save(fig, "fig1")
# ---- Figure 2: percentage of patients, abstract vs what the body's counts give
fig, ax = plt.subplots(figsize=(7.2, 2.6))
rows = ["Grade ≥ 3 TRAE\n(the abstract's endpoint)", "Any grade ≥ 3 AE\n(all causes)"]
y = [1, 0]
ax.set_xlim(0, 35)
ax.set_ylim(-0.6, 1.6)
ax.set_yticks(y)
ax.set_yticklabels(rows, fontsize=10.5, color=TXT)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Patients with at least one event (% of 151)", fontsize=10.5)
# TRAE row
ax.plot([p_trae, ABS_PCT], [1, 1], color=SEC, lw=2, zorder=2)
ax.plot(ABS_PCT, 1, "o", ms=10, color=ORANGE, zorder=3)
ax.plot(p_trae, 1, "o", ms=10, color=BLUE, zorder=3)
ax.annotate(f"Abstract: {ABS_PCT}%", (ABS_PCT, 1), xytext=(0, 11), textcoords="offset points",
ha="center", fontsize=10, color=ORANGE, fontweight="bold")
ax.annotate(f"{TRAE_PTS}/{N} = {p_trae:.1f}%", (p_trae, 1), xytext=(0, 11), textcoords="offset points",
ha="center", fontsize=10, color=BLUE, fontweight="bold")
ax.annotate(f"+{p_all - p_trae:.1f} points", ((p_trae + ABS_PCT) / 2, 1), xytext=(0, -16),
textcoords="offset points", ha="center", fontsize=10, color=SEC)
# all-cause row
ax.plot(p_all, 0, "o", ms=10, color=BLUE, zorder=3)
ax.annotate(f"{ALL_AE_PTS}/{N} = {p_all:.1f}%", (p_all, 0), xytext=(0, 11), textcoords="offset points",
ha="center", fontsize=10, color=BLUE, fontweight="bold")
ax.plot([ABS_PCT, p_all], [1, 0], color=ORANGE, lw=1, ls=(0, (3, 2)), zorder=1)
ax.text(ABS_PCT + 0.5, 0.5, "same value", fontsize=10, color=SEC, va="center")
leg = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="reported in the abstract"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="recomputed from the Results counts")]
ax.legend(handles=leg, loc="lower left", frameon=False, fontsize=10, bbox_to_anchor=(0.0, -0.02))
save(fig, "fig2")
print("\nfigures written: fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract gives the unknown-IVS and incident-TB mortality estimates as poor IVS and prior TB
Lancet Reg Health Eur (2024) · doi:10.1016/j.lanepe.2024.100989 · PMC11259909
Summary
This EuroSIDA cohort study of 20,364 people with HIV reports adjusted incidence rate ratios (aIRRs) for all-cause mortality. The abstract gives three of them as headline risk factors: off ART 2.42 (2.14–2.74), poor immunologic-virologic status (IVS) 4.2 (3.39–5.20) and prior TB 3.33 (2.75–4.03). In the paper's own Table 2 and Results text, the triple 4.2 (3.39–5.20) is the estimate for unknown IVS, while poor IVS is 7.61 (6.68–8.68). The triple 3.33 (2.75–4.03) is the estimate for TB diagnosed during follow-up, while TB before baseline is 1.45 (1.26–1.67). The abstract therefore understates the poor-IVS association by 45% and overstates the prior-TB association 2.3-fold. Every risk factor stays above 1 and statistically significant, so the paper's qualitative conclusions hold. The abstract also gives region-interaction p-values (IVS p = 0.02, TB p = 0.5) that differ from the Results (p = 0.001 and p = 0.22).
What the paper reports
Abstract, Findings:
Being off ART (aIRR 2.42; 95% CI 2.14–2.74), poor IVS (aIRR 4.2; 95% CI 3.39–5.20) and prior TB (aIRR 3.33; 95% CI 2.75–4.03) were associated with higher all-cause mortality. For all-cause mortality the effect of ART (test for interaction: p < 0.001) and IVS (p = 0.02), but not TB (p = 0.5) varied across regions.
Abstract, Interpretation:
A poor IVS, being off ART and prior active TB were related to higher mortality.
Abstract, Methods (the IVS categories):
Immunologic-virologic status (IVS) was categorized as poor (CD4-cell count ≤350 cells/mm3 and viral load (VL) > 200 copies/ml), good (CD4 ≥ 500 and VL < 200), or intermediate (remaining combinations). Participants missing either CD4-cell count or VL were categorized as unknown.
Methods, "Variable selection" (the TB categories):
history of active TB was classified as no history of TB, TB pre-baseline (or at baseline), and incident TB during follow-up
Results, "Risk factors for all-cause mortality":
The most important risk factors for all-cause mortality in the adjusted multivariable model were poor IVS (vs good IVS; aIRR 7.61; 95% CI 6.68–8.68) or unknown IVS (aIRR 4.20; 95% CI 3.39–5.20), incident active TB during follow-up (aIRR 3.33; 95% CI 2.75–4.03), being off ART (aIRR 2.42; 95% CI 2.14–2.74), and receiving care in Eastern Europe (vs. Western Europe; aIRR 1.55; 1.40–1.71).
Results, "Effect of ART-status, IVS and active TB on mortality across regions":
We also observed significant interactions on the risk of all-cause mortality between region of Europe and ART-status (p < 0.001 for interaction) and IVS (p = 0.001) but not for active TB (p = 0.22).
Table 2 ("Risk factors for all-cause and AIDS-related mortality in univariable and multivariable adjusted Poisson regression models"), all-cause columns, IRR (95% CI) | aIRR (95% CI):
Poor | 8.54 (7.63, 9.57) | 7.61 (6.68, 8.68)
Unknown | 4.26 (3.51, 5.18) | 4.2 (3.39, 5.2)
Off ART | 3.91 (3.5, 4.37) | 2.42 (2.14, 2.74)
History of TB, pre-baseline | 1.76 (1.53, 2.02) | 1.45 (1.26, 1.67)
Incident TB during follow-up | 5.55 (4.63, 6.65) | 3.33 (2.75, 4.03)
Eastern | 1.55 (1.4, 1.71) | 1.5 (1.33, 1.69)
Discussion:
The most important risk factors for all-cause mortality in the adjusted multivariable model were poor or unknown IVS, being off ART, and pre-baseline as well as incident active TB during follow-up.
What the paper's own numbers show
Method. No model needs to be refitted: this is a matching check. For each of the abstract's three estimates I searched Table 2's all-cause columns (adjusted and unadjusted) for a row whose point estimate and both confidence limits agree within rounding (±0.005). I then compared each abstract value with the Table 2 row that its label names. To confirm that each Table 2 row is internally coherent, I checked that each estimate equals the geometric mean of its CI limits, as a log-scale Wald interval from a Poisson model requires. To show that the swapped rows are distinct estimates, not near-duplicates, I computed an approximate z for each pair from the standard errors implied by the CIs, SE = (ln hi − ln lo)/(2 × 1.96). This ignores the covariance between coefficients and is only illustrative. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| "poor IVS" aIRR | Abstract, Findings | 4.2 (3.39–5.20) | Matches only Table 2 Unknown IVS, 4.2 (3.39, 5.2). Table 2 Poor IVS is 7.61 (6.68–8.68) |
| "prior TB" aIRR | Abstract, Findings | 3.33 (2.75–4.03) | Matches only Table 2 Incident TB during follow-up. Table 2 pre-baseline TB is 1.45 (1.26–1.67) |
| "off ART" aIRR (control) | Abstract, Findings | 2.42 (2.14–2.74) | Matches Table 2 Off ART, the row its label names |
| Row coherence, estimate vs sqrt(lo × hi) | Table 2 | 7.61; 4.20; 1.45; 3.33; 2.42 | 7.61; 4.20; 1.45; 3.33; 2.42 (all coherent) |
| Poor IVS: Table 2 / abstract | derived | — | 7.61 / 4.20 = 1.81; abstract is 44.8% lower |
| Poor IVS excess risk (aIRR − 1) | derived | 3.20 (abstract) | 6.61 (Table 2); abstract gives 48% of it |
| Prior TB: abstract / Table 2 pre-baseline | derived | — | 3.33 / 1.45 = 2.30-fold |
| Prior TB excess risk (aIRR − 1) | derived | 2.33 (abstract) | 0.45 (Table 2); abstract is 5.2-fold larger |
| CI overlap, abstract vs correctly labelled row | derived | — | none for poor IVS (5.20 < 6.68) or prior TB (2.75 > 1.67) |
| Approximate z, poor vs unknown IVS | derived | — | 4.64 |
| Approximate z, incident vs pre-baseline TB | derived | — | 6.86 |
| Supplement all-cause aIRRs (Tables S1, S2, S4) | mmc1.docx | — | poor IVS 6.80–10.25; pre-baseline TB 1.21–1.99; none equals 4.20 or 3.33 for these categories |
| Region × IVS interaction p | Abstract vs Results | 0.02 (Abstract) | 0.001 (Results) |
| Region × TB interaction p | Abstract vs Results | 0.5 (Abstract) | 0.22 (Results) |
| Eastern vs Western Europe "aIRR" | Results text vs Table 2 | 1.55 (1.40–1.71) | This is the unadjusted IRR in Table 2. The adjusted value is 1.50 (1.33–1.69) |
Off ART, the control, maps to its own row, so the abstract was drawn from the same adjusted all-cause column. The other two triples each match exactly one row of that column, but not the row their label names. The Results text, Table 2 and the Discussion agree with each other and disagree with the abstract.
Figure
Figure 1. Left: the three all-cause aIRRs in the abstract's Findings, with their labels. Right: the corresponding rows of Table 2's adjusted all-cause column, in table order. The off-ART value (black) matches the Off ART row. The values printed as "poor IVS" and "prior TB" (orange) match, in estimate and both CI limits, the Unknown IVS and Incident TB rows (arrows). The Table 2 values for the rows those labels actually name (blue) are 7.61 (6.68–8.68) for poor IVS and 1.45 (1.26–1.67) for pre-baseline TB. They appear nowhere in the abstract.
Figure 2. For each of the abstract's three all-cause risk factors, the orange interval is the aIRR and 95% CI the abstract gives, and the blue interval is the Table 2 aIRR for the category the label names. The axis is logarithmic; the thin line marks aIRR = 1. For off ART the two coincide. For poor IVS and prior (pre-baseline) TB the intervals do not overlap: the abstract understates the poor-IVS association and overstates the prior-TB association. All six intervals lie above 1, so no association changes direction or significance.
Why it matters
The abstract is the part of the paper that most readers, indexers and guideline reviewers use. Its Findings sentence gives exactly three quantified all-cause risk factors, and two of them are paired with the wrong category.
- Poor IVS. A reader of the abstract would conclude that poor immunologic-virologic control carries about a fourfold mortality rate. The paper's own estimate is about 7.6-fold, and its excess risk is roughly twice what the abstract implies. The abstract number belongs to people whose CD4 count or viral load is missing. That group matters in its own right: the Interpretation highlights Eastern Europe's high share of "poor or unknown IVS". Presenting the unknown-IVS estimate as poor IVS removes the distinction the paper draws.
- Prior TB. "Prior TB" and the Interpretation's "prior active TB" suggest a history of TB before entry, which the Methods call "TB pre-baseline (or at baseline)". That category has an aIRR of 1.45. The value of 3.33 belongs to TB diagnosed during follow-up, a different exposure with different clinical implications: an active disease episode rather than a past history. The abstract thus overstates the long-term mortality burden of a TB history and does not report the larger incident-TB effect under its own name.
- Interaction p-values. The abstract's IVS interaction (p = 0.02) is weaker, and its TB interaction (p = 0.5) less suggestive, than the Results report (p = 0.001 and p = 0.22). The significant/non-significant classification is the same in both places.
The study's main conclusions are not affected. Mortality fell over time, it remained higher in Eastern Europe, and poor IVS, being off ART and TB are associated with higher mortality. The Research-in-context panel, the Results, Table 2 and the Discussion all state these correctly. The separate Results statement that Eastern Europe has "aIRR 1.55; 1.40–1.71" uses the unadjusted IRR. The adjusted value, 1.50 (1.33–1.69), differs only slightly and does not change the conclusion.
Explanations considered
- The abstract reports a different model, period or region. This does not account for the numbers. The abstract calls them all-cause aIRRs, and all three match Table 2's adjusted all-cause column exactly, one of them (off ART) on the correct row. The stratified and sensitivity estimates in the supplement are all different numbers. Poor IVS ranges from 6.80 to 10.25 and pre-baseline TB from 1.21 to 1.99 across Tables S1 (by period), S2 (by region) and S4 (February 2020 cut-off). No supplementary estimate for those categories equals 4.20 or 3.33.
- "Poor IVS" in the abstract includes unknown IVS. This does not account for the numbers. The abstract's own Methods define poor, good, intermediate and unknown as separate categories, and Table 2 and the Results give each its own estimate. The value 4.2 (3.39–5.20) is the unknown row alone, not a pooled estimate.
- "Prior TB" loosely means any TB diagnosed before death, including incident TB. This is possible as wording, but it does not fully account for the numbers. The paper defines three TB categories, and "prior" corresponds most naturally to "pre-baseline". Even on the loose reading, the abstract would be reporting only the incident-TB estimate under a label that also covers pre-baseline TB, whose effect (1.45) is very different (approximate z = 6.86). The poor-IVS mismatch stands independently of this point.
- Rounding or a typographical slip in a single number. This does not account for the numbers. In each case all three numbers (estimate and both limits) are identical to another row, and each Table 2 row is internally coherent (estimate = sqrt(lo × hi)). The two estimates in each pair differ clearly (approximate z = 4.64 and 6.86).
- A conversion or typesetting artefact in the archived copy. This is excluded. The PMC text, the Europe PMC full-text XML, the PubMed abstract and the Elsevier-typeset PDF all carry the same abstract sentence and the same Table 2 rows.
- A published correction. None was found (see Status).
Status
- Version of record. I checked three copies, and all give identical numbers. The first is the Elsevier-typeset publisher PDF (27 pages; metadata "The Lancet Regional Health - Europe, 44 (2024) 100989", created 2024-06-25) held in an institutional repository. The second is the Europe PMC full-text XML, with Table 2 parsed from its table cells. The third is the PubMed abstract (PMID 39036304). The thelancet.com and sciencedirect.com pages returned HTTP 403 to automated access. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. I parsed the supplementary appendix (mmc1.docx, from the Europe PMC supplementary-files archive). It holds Table S1 (aIRRs by time period), S2 (by region), S3 (person-years), S4 (sensitivity analysis with a February 2020 cut-off) and S5 (CD4 and viral-load measurement rates). No table gives poor IVS as 4.2 (3.39–5.20) or prior/pre-baseline TB as 3.33 (2.75–4.03). The supplement does not resolve the discrepancy.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. A web search for corrections or comments on the article found none. The PubPeer API refused automated access (HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC11259909; Europe PMC full-text XML and supplementary files for PMC11259909; PubMed 39036304; publisher PDF (repositoriosaludmadrid.es); Crossref; Europe PMC.
Suggested correction
- Abstract, Findings: replace "poor IVS (aIRR 4.2; 95% CI 3.39–5.20) and prior TB (aIRR 3.33; 95% CI 2.75–4.03)" with "poor IVS (aIRR 7.61; 95% CI 6.68–8.68), unknown IVS (aIRR 4.20; 95% CI 3.39–5.20), TB before baseline (aIRR 1.45; 95% CI 1.26–1.67) and incident TB during follow-up (aIRR 3.33; 95% CI 2.75–4.03)", or an equivalent wording that pairs each estimate with its Table 2 category.
- Abstract, Interpretation: replace "prior active TB" with wording that matches the corrected Findings, such as "active TB before or during follow-up".
- Abstract, Findings: reconcile the region-interaction p-values with the Results (IVS p = 0.001, TB p = 0.22), or explain which test each refers to.
- Results, "Risk factors for all-cause mortality": give the adjusted Eastern vs Western Europe estimate as aIRR 1.50 (95% CI 1.33–1.69), as in Table 2, or label 1.55 (1.40–1.71) as unadjusted.
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC11259909
(Lancet Reg Health Eur 2024, doi:10.1016/j.lanepe.2024.100989; EuroSIDA all-cause and AIDS mortality 2001-2020).
Run: python recompute.py.txt
Needs matplotlib (numpy and scipy optional; only the standard library is used for arithmetic).
Prints every recomputed number used in writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (and identical in the Europe PMC XML and publisher PDF):
Abstract, Findings: "Being off ART (aIRR 2.42; 95% CI 2.14-2.74), poor IVS (aIRR 4.2; 95% CI 3.39-5.20)
and prior TB (aIRR 3.33; 95% CI 2.75-4.03) were associated with higher all-cause mortality.
For all-cause mortality the effect of ART (test for interaction: p < 0.001) and IVS (p = 0.02),
but not TB (p = 0.5) varied across regions."
Results, "Risk factors for all-cause mortality": poor IVS 7.61 (6.68-8.68), unknown IVS 4.20 (3.39-5.20),
incident TB 3.33 (2.75-4.03), off ART 2.42 (2.14-2.74), Eastern Europe "aIRR 1.55; 1.40-1.71"
Results, "Effect of ART-status, IVS and active TB ...": interaction p < 0.001 (ART), 0.001 (IVS), 0.22 (TB)
Table 2, all-cause columns, IRR (95% CI) | aIRR (95% CI)
Supplement (mmc1.docx, all-cause aIRRs) values are listed for the range check only.
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.ticker # noqa: E402
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = 1.959964
# ---------------------------------------------------------------- inputs (from the paper)
# Table 2, all-cause mortality: label -> (unadjusted IRR, lo, hi), (aIRR, lo, hi)
TABLE2 = {
"Poor IVS": ((8.54, 7.63, 9.57), (7.61, 6.68, 8.68)),
"Intermediate IVS": ((3.08, 2.79, 3.39), (2.82, 2.55, 3.12)),
"Unknown IVS": ((4.26, 3.51, 5.18), (4.20, 3.39, 5.20)),
"Off ART": ((3.91, 3.50, 4.37), (2.42, 2.14, 2.74)),
"ART naive": ((1.50, 1.28, 1.76), (1.17, 0.98, 1.40)),
"History of TB, pre-baseline": ((1.76, 1.53, 2.02), (1.45, 1.26, 1.67)),
"Incident TB during follow-up": ((5.55, 4.63, 6.65), (3.33, 2.75, 4.03)),
"Eastern Europe": ((1.55, 1.40, 1.71), (1.50, 1.33, 1.69)),
"Central Eastern Europe": ((0.79, 0.69, 0.89), (1.00, 0.88, 1.14)),
}
# Abstract, Findings: label as printed -> (aIRR, lo, hi), Table 2 row the label refers to
ABSTRACT = [
("off ART", (2.42, 2.14, 2.74), "Off ART"),
("poor IVS", (4.2, 3.39, 5.20), "Poor IVS"),
("prior TB", (3.33, 2.75, 4.03), "History of TB, pre-baseline"),
]
RESULTS_EASTERN = (1.55, 1.40, 1.71) # Results text, called "aIRR"
INTERACTION_P = {"ART": ("< 0.001", "< 0.001"), "IVS": ("0.02", "0.001"), "TB": ("0.5", "0.22")} # (abstract, Results)
# Supplement mmc1.docx, all-cause aIRR point estimates (Table S1 by period 2001-10/2011-15/2016-20,
# Table S2 by region, Table S4 sensitivity analysis with Feb 2020 cut-off)
SUPP = {
"Poor IVS": {"S1": (6.8, 10.25, 8.48), "S2": (7.5, 8.5, 7.44), "S4": (7.59,)},
"Unknown IVS": {"S1": (4.34, 3.78, 1.93), "S2": (2.18, 3.21, 4.47), "S4": (4.32,)},
"History of TB, pre-baseline": {"S1": (1.61, 1.21, 1.41), "S2": (1.34, 1.99, 1.76), "S4": (1.48,)},
"Incident TB during follow-up": {"S1": (5.27, 2.5, 2.26), "S2": (2.72, 3.02, 4.14), "S4": (3.43,)},
}
def se_log(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z975)
def same(a, b, tol=0.006):
return all(abs(x - y) < tol for x, y in zip(a, b))
def fmt(t):
return f"{t[0]:.2f} ({t[1]:.2f}-{t[2]:.2f})"
print("=== 1. Which Table 2 all-cause row does each abstract triple match? (estimate and both limits, tol 0.005) ===")
for lab, v, intended in ABSTRACT:
adj = [k for k, (u, a) in TABLE2.items() if same(v, a)]
unadj = [k for k, (u, a) in TABLE2.items() if same(v, u)]
print(f" Abstract '{lab}' {fmt(v)}: matches aIRR row(s) {adj}; unadjusted row(s) {unadj}; "
f"row the label names: '{intended}' = {fmt(TABLE2[intended][1])}")
print("\n=== 2. Internal consistency of each Table 2 aIRR row (log-symmetric Wald CI: estimate = sqrt(lo*hi)) ===")
for k in ["Poor IVS", "Intermediate IVS", "Unknown IVS", "Off ART", "History of TB, pre-baseline",
"Incident TB during follow-up", "Eastern Europe"]:
e, lo, hi = TABLE2[k][1]
print(f" {k:30s} aIRR {e:5.2f} sqrt(lo*hi) = {math.sqrt(lo * hi):5.2f} SE(ln aIRR) = {se_log(lo, hi):.3f}")
print("\n=== 3. Size of the discrepancies ===")
poor, unk = TABLE2["Poor IVS"][1], TABLE2["Unknown IVS"][1]
pre, inc = TABLE2["History of TB, pre-baseline"][1], TABLE2["Incident TB during follow-up"][1]
print(f" Poor IVS: abstract 4.20 vs Table 2 7.61 -> ratio {poor[0] / 4.2:.2f}; abstract understates by "
f"{100 * (poor[0] - 4.2) / poor[0]:.1f}%")
print(f" Excess risk (aIRR - 1): abstract {4.2 - 1:.2f} vs Table 2 {poor[0] - 1:.2f} -> {(4.2 - 1) / (poor[0] - 1) * 100:.0f}% of Table 2")
print(f" Prior/pre-baseline TB: abstract 3.33 vs Table 2 1.45 -> ratio {3.33 / pre[0]:.2f}")
print(f" Excess risk: abstract {3.33 - 1:.2f} vs Table 2 {pre[0] - 1:.2f} -> {(3.33 - 1) / (pre[0] - 1):.1f}-fold")
print(f" CI overlap poor IVS: abstract upper {5.20} vs Table 2 lower {poor[1]} -> overlap = {5.20 >= poor[1]}")
print(f" CI overlap prior TB: abstract lower {2.75} vs Table 2 upper {pre[2]} -> overlap = {2.75 <= pre[2]}")
def ztest(a, b):
return (math.log(a[0]) - math.log(b[0])) / math.hypot(se_log(a[1], a[2]), se_log(b[1], b[2]))
print("\n=== 4. Approximate z for the difference between the swapped rows (ignores covariance; illustrative) ===")
print(f" poor vs unknown IVS: z = {ztest(poor, unk):.2f}")
print(f" incident vs pre-baseline TB: z = {ztest(inc, pre):.2f}")
print("\n=== 5. Supplement range check (all-cause aIRR point estimates) ===")
for k, d in SUPP.items():
vals = [x for t in d.values() for x in t]
print(f" {k:30s} min {min(vals):5.2f} max {max(vals):5.2f} any equal to 4.20? {any(abs(x - 4.2) < 0.006 for x in vals)}"
f" any equal to 3.33? {any(abs(x - 3.33) < 0.006 for x in vals)}")
print("\n=== 6. Side findings ===")
e_u, e_a = TABLE2["Eastern Europe"]
print(f" Results 'aIRR 1.55; 1.40-1.71' for Eastern Europe: equals unadjusted IRR {fmt(e_u)}? {same(RESULTS_EASTERN, e_u)}; "
f"adjusted aIRR in Table 2 is {fmt(e_a)}")
for k, (a, r) in INTERACTION_P.items():
print(f" Region interaction p, {k}: abstract {a}, Results {r} -> {'same' if a == r else 'differ'}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# ---- Figure 1: abstract vs Table 2 text-panel diagram
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.6)
ax.axis("off")
LX, LW = 0.15, 2.65 # abstract panel
RX, RW = 4.25, 2.8 # Table 2 panel
TOP = 4.05
for x, w, title in [(LX, LW, "Abstract, Findings"), (RX, RW, "Table 2, all-cause aIRR")]:
ax.add_patch(FancyBboxPatch((x, 0.95), w, TOP - 0.95, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=SEC, lw=1))
ax.text(x + 0.15, TOP - 0.22, title, fontsize=11, fontweight="bold", va="center")
# abstract lines
ab_y = {"off ART": 3.25, "poor IVS": 2.55, "prior TB": 1.85}
ab_col = {"off ART": TXT, "poor IVS": ORANGE, "prior TB": ORANGE}
for lab, v, _ in ABSTRACT:
y = ab_y[lab]
ax.text(LX + 0.15, y + 0.13, lab, fontsize=10.5, va="center", color=TXT)
ax.text(LX + 0.15, y - 0.17, f"{v[0]:g} ({v[1]:.2f}–{v[2]:.2f})", fontsize=11, va="center",
color=ab_col[lab], fontweight="bold")
ax.text(LX + 0.15, 1.22, "“were associated with higher\nall-cause mortality”", fontsize=10, color=SEC, va="center")
# Table 2 rows
rows = [("Poor IVS", "Poor IVS", BLUE), ("Intermediate", "Intermediate IVS", SEC), ("Unknown IVS", "Unknown IVS", TXT),
("Off ART", "Off ART", TXT), ("TB pre-baseline", "History of TB, pre-baseline", BLUE),
("Incident TB", "Incident TB during follow-up", TXT)]
ty = {}
for i, (short, key, col) in enumerate(rows):
y = 3.4 - i * 0.4
ty[key] = y
e, lo, hi = TABLE2[key][1]
ax.text(RX + 0.15, y, short, fontsize=10.5, va="center", color=SEC if col == SEC else TXT)
ax.text(RX + RW - 0.12, y, f"{e:.2f} ({lo:.2f}–{hi:.2f})", fontsize=10.5, va="center", ha="right",
color=col, fontweight="bold" if col == BLUE else "normal")
def connect(y0, y1, col, ls, text, ty_off):
x0, x1 = LX + LW + 0.05, RX - 0.05
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=col, lw=2, ls=ls, shrinkA=2, shrinkB=2, mutation_scale=12))
xm, ym = (x0 + x1) / 2, (y0 + y1) / 2 + ty_off
ax.text(xm, ym, text, fontsize=10, ha="center", va="center", color=col,
bbox=dict(fc=BG, ec="none", pad=1.2))
ax.text(LX + 0.8, ab_y["off ART"] + 0.13, "(= Table 2 Off ART row)", fontsize=10, va="center", color=SEC)
connect(ab_y["poor IVS"] - 0.17, ty["Unknown IVS"], ORANGE, "-", "numbers of the\nUnknown IVS row", 0.38)
connect(ab_y["prior TB"] - 0.17, ty["Incident TB during follow-up"], ORANGE, "-", "numbers of the\nIncident TB row", -0.36)
ax.text(0.15, 0.62, "Orange: values the abstract prints under “poor IVS” and “prior TB”; they match other rows exactly.",
fontsize=10, color=TXT, va="center")
ax.text(0.15, 0.3, "Blue: Table 2 values for the rows those labels name (poor IVS 7.61; TB pre-baseline 1.45).",
fontsize=10, color=TXT, va="center")
ax.text(LX + 0.15, 4.35, "The abstract prints two Table 2 estimates under the wrong categories",
fontsize=11.5, fontweight="bold", va="center")
save(fig, "fig1")
# ---- Figure 2: dot-and-interval, abstract value vs Table 2 value for the same label (log scale)
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.subplots_adjust(left=0.3, right=0.97, top=0.9, bottom=0.27)
labels = [("Off ART", "Off ART"), ("Poor IVS\n(vs good)", "Poor IVS"), ("Prior TB (abstract) /\npre-baseline TB (Table 2)", "History of TB, pre-baseline")]
ab = {"Off ART": ABSTRACT[0][1], "Poor IVS": ABSTRACT[1][1], "History of TB, pre-baseline": ABSTRACT[2][1]}
for i, (lab, key) in enumerate(labels):
y = 2 - i
a = ab[key]
t = TABLE2[key][1]
for (e, lo, hi), col, dy in [(a, ORANGE, 0.14), (t, BLUE, -0.14)]:
ax.plot([lo, hi], [y + dy, y + dy], color=col, lw=2, solid_capstyle="butt")
ax.plot(e, y + dy, "o", color=col, ms=8)
ax.annotate(f"{e:.2f}", (hi, y + dy), xytext=(6, 0), textcoords="offset points", va="center",
fontsize=10, color=TXT)
ax.axvline(1, color=SEC, lw=0.8)
ax.text(1.03, 2.42, "aIRR = 1", fontsize=10, color=SEC, va="center")
ax.set_xscale("log")
ax.set_xlim(0.9, 11)
ticks = [1, 1.5, 2, 3, 4, 5, 7, 10]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([2, 1, 0])
ax.set_yticklabels([l for l, _ in labels], fontsize=10.5)
ax.set_ylim(-0.6, 2.6)
ax.set_xlabel("All-cause mortality, adjusted IRR (log scale), 95% CI", fontsize=10.5)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Abstract (as labelled)"),
Line2D([], [], color=BLUE, marker="o", ms=8, lw=2, label="Table 2 row with that label")],
loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
ax.set_title("Same label, different estimate: abstract vs Table 2", fontsize=11.5, fontweight="bold", loc="left")
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract gives each arm's effect-size ranges to the other: incentives versus information
Lancet Reg Health Eur (2024) · doi:10.1016/j.lanepe.2024.101102 · PMC11513846
Summary
The Findings section of the abstract attributes annual trend improvements of 6.66 to 1.25 percentage points and step changes up to 8.87 points to the incentivized indicators. It attributes step changes of 19.67 to 1.07 points and trend improvements of 1.09 to 0.34 points per year to information provision. In the paper's own Table 1, and in the Results text, the assignment is the reverse. Every one of the first set of numbers comes from rows labelled "Introduced as Informed", and every one of the second set from rows labelled "Introduced as Incentivized". The abstract's shares of indicators with a positive impact (9/12 informed, 7/11 incentivized) reproduce from the tables. But the abstract's description of what each strategy achieved is reversed: incentives produced the largest immediate step changes, and information produced the largest sustained trend changes.
What the paper reports
Abstract, Findings:
Informed indicators showed positive impacts in 75% (9/12) of cases, and incentivized indicators in 64% (7/11) of cases. Incentivized indicators displayed improvements in annual trends ranging from 6.66 to 1.25 percentage points, with step changes up to 8.87 percentage points. Information led to step changes ranging from 19.67 to 1.07 percentage points, along with trend improvements between 1.09 and 0.34 percentage points annually.
Abstract, Interpretation:
Our findings reveal that information provision alone can match or even surpass the impact of financial incentives in improving care quality and reducing practice variability.
Results, section "The impact of incentivization on newly introduced indicators":
Incentivization positively impacted indicator results or variability in seven of 11 new indicators (64%; 5/6 quaternary prevention and 2/3 control indicators; Fig. 2, Supplementary Figures S4–S6). Four showed significant step increases in results post-implementation (19.67 to 1.07 percentage point changes), while four demonstrated positive trend changes (1.09 to 0.34 percentage point improvements annually).
Results, section "The effect of information without incentivization on new indicators":
Specifically, six of 12 indicators displayed significantly positive changes in the post-intervention trends (6.66 to 1.25 percent point improvements per year), four of which showed additional step improvements in results after their introduction as informed (from 8.87 to 1.25 %-point increases; Fig. 2, Table 1, Supplementary Figure S4).
Table 1, rows that carry the end points of the ranges (columns: Indicator | Derived | Scheme change | Year | Baseline | Trend before | Step change | Change in trend | End result):
Reduction of alcohol consumption in high-risk drinkers | New | Introduced as Incentivized | 2014 | 21.67 | 1.39 (0.29–2.48)c | 19.67 (11.34–27.99)c | -0.7 (-3.14 to 1.74) | 46.58
Low cardiovascular risk with poorly indicated lipid-lowering agentsa | New | Introduced as Incentivized | 2014 | 7.55 | -0.25 (-0.33 to -0.18)c | -1.07 (-1.55 to -0.6)c | -0.34 (-0.48 to -0.2)c | 2.72
Adequacy of treatment for acute bronchitis, acute rhinopharyngitis, and influenza | New | Introduced as Incentivized | 2014 | 77.73 | -0.15 (-0.6 to 0.3) | 2.36 (0.74–3.98)c | 1.09 (0.52–1.65)c | 84.52
Diagnostic adequacy of cardiovascular disease | New | Introduced as Informed | 2013 | 13.53 | 6.08 (4.98–7.18)c | 8.87 (6.32–11.41)c | 6.66 (5.56–7.77)c | 52.88
Patients with excessive control of T2DMa | New | Introduced as Informed | 2017 | 41.42 | 1.08 (0.6–1.57)c | -2.15 (-4.65 to 0.36) | -1.25 (-1.74 to -0.77)c | 46.2
Table 1, footnote:
Inverse indicators (negative step and trend changes represent improvement).
Methods, Statistical analysis:
Significant improvements in either metric were classified as “positive impact”.
What the paper's own numbers show
Method. I transcribed the step change and change in trend, with their 95% CIs, for all 30 new-indicator rows of Table 1. These are the 11 rows "Introduced as Incentivized", the 12 "Introduced as Informed" and the 7 "Informed to Incentivized". An effect was counted as significant when its 95% CI excludes zero. This reproduces the table's significance marks in all 60 cells. An effect was counted as favourable when it is positive, or negative for an inverse indicator (Table 1 footnote a). It was then expressed as a magnitude. For each group I listed the favourable significant step and trend changes and took their range. Then I checked which group each number in the abstract comes from. To check the abstract's shares (9/12, 7/11), I applied the paper's definition of "positive impact" to Table 1 and to Table 2 (coefficient of variation, where a reduction is favourable). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Trend improvements, 6.66 to 1.25 pp/yr | Abstract: "Incentivized indicators" | assigned to incentivized | Introduced as Informed rows: 6.66, 5.70, 5.53, 3.18, 1.95, 1.25 (n = 6). No incentivized row has a favourable trend change above 1.09 |
| Step changes up to 8.87 pp | Abstract: "Incentivized indicators" | assigned to incentivized | Introduced as Informed row (cardiovascular disease diagnosis, 8.87). Largest incentivized step is 19.67 |
| Step changes, 19.67 to 1.07 pp | Abstract: "Information" | assigned to information | Introduced as Incentivized rows: 19.67, 19.66, 2.36, 1.48, 1.07 (n = 5; 1.48 and 1.07 are inverse indicators). Largest informed step is 8.87 |
| Trend improvements, 1.09 to 0.34 pp/yr | Abstract: "Information" | assigned to information | Introduced as Incentivized rows: 1.09, 1.09, 0.89, 0.34 (n = 4). Smallest informed trend is 1.25 |
| Incentivized step changes | Results, incentivization section | "Four", 19.67 to 1.07 | range agrees (19.67 to 1.07); count is 5 (3 positive steps plus 2 favourable negative steps of inverse indicators) |
| Incentivized trend changes | Results, incentivization section | "four", 1.09 to 0.34 | agrees: n = 4, 1.09 to 0.34 |
| Informed trend changes | Results, information section | "six", 6.66 to 1.25 | agrees: n = 6, 6.66 to 1.25 |
| Informed step changes | Results, information section | "four", "from 8.87 to 1.25" | count agrees (n = 4); range is 8.87 to 3.41. 1.25 is the smallest favourable trend, not a step |
| "Informed to Incentivized" rows (2015) | Table 1 | — | no favourable significant step or trend changes; the 5 significant trend changes are all negative (−8.93, −3.24, −2.81, −5.36, −0.67) |
| Share with positive impact, informed | Abstract | 75% (9/12) | 9/12 (6 on results, 8 on CV, 9 on either) |
| Share with positive impact, incentivized | Abstract | 64% (7/11) | 7/11 (6 on results, 5 on CV, 7 on either) |
| Largest step, incentivized vs informed | Table 1 | — | 19.67 vs 8.87 pp (ratio 2.22) |
| Largest trend change, informed vs incentivized | Table 1 | — | 6.66 vs 1.09 pp/yr (ratio 6.11) |
All four end points of the abstract's "incentivized" figures (6.66, 1.25 and 8.87) occur only among informed rows. All four end points of its "information" figures (19.67, 1.07, 1.09, 0.34) occur only among incentivized rows. The Results text and Table 1 agree on the assignment. The abstract's first sentence (the shares) also agrees with the tables. Only the two sentences giving effect sizes carry the labels the other way round.
Figure
Figure 1. Left: the effect-size ranges as printed in the abstract's Findings section, under the label the abstract gives them (orange). Right: the ranges recomputed from the favourable, significant Table 1 step and trend changes of the new indicators in each scheme-change group (blue), with inverse indicators sign-flipped as the Table 1 footnote directs. The crossing connectors show that each printed set matches the opposite group. The one difference in the informed step range (8.87 to 3.41 recomputed vs "8.87 to 1.25" in the Results text) is a separate small slip in the Results, noted in the footnote line. pp = percentage points.
Figure 2. Each blue dot is one favourable, significant effect from Table 1 for a new indicator in that arm (left: step changes, pp; right: changes in trend, pp per year). Each orange bar is the range the abstract prints for that arm. For "Incentivized" that range is "up to 8.87" for steps, drawn from 0, and 6.66 to 1.25 for trends; for "Information" it is 19.67 to 1.07 and 1.09 to 0.34. The two incentivized steps near 19.7 (alcohol, 19.67; hyperuricemia, 19.66) lie outside the abstract's incentivized step range, and all four incentivized trend changes lie below its incentivized trend range. The informed trend changes lie above the abstract's informed trend range. Axes start at zero.
Why it matters
- The abstract inverts each strategy's effect profile. As printed, information looks as if it produced the large immediate jumps (up to 19.67 points) but only small sustained gains (at most 1.09 points per year), and incentives the reverse. Table 1 shows the opposite. Incentives produced the two largest step changes (19.67 and 19.66 points), and information produced trend improvements up to 6.66 points per year, about six times the largest incentivized trend change (1.09). For a paper whose purpose is to compare the two strategies, this comparison is a key result.
- It bears on the evidence offered for the Interpretation. The abstract concludes that information "can match or even surpass" incentives. As printed, its largest step-change figure appears under information, which reads as direct support for "surpass". On the corrected assignment the conclusion can still be argued, from the larger sustained trend gains under information and from the shares of indicators improved (75% vs 64%). But the step-change evidence points the other way.
- The main analysis is not affected. The Results text and Table 1 assign the numbers correctly, and the shares of indicators with positive impact (9/12, 7/11) reproduce from Tables 1 and 2. The variability, derived-indicator and robustness results, and the finding that later incentivization of informed indicators added no gains, do not depend on the two swapped sentences. The title, the Research in context panel and the Discussion give only the shares, not the effect-size ranges. No estimate, direction or significance changes. The error is confined to the abstract, but the abstract is the part most readers and indexing services see.
Explanations considered
- Inverse indicators. These do not account for it, but they explain why two of the printed end points are written without a sign. The Table 1 footnote says negative changes are improvements for inverse indicators. The incentivized step −1.07 and trend −0.34 (low-risk patients on poorly indicated lipid-lowering agents), and the informed trend −1.25 (excessive control of T2DM), are favourable effects of size 1.07, 0.34 and 1.25. Taking magnitudes makes the incentivized and informed ranges match the Results text exactly. It does not move any number between groups.
- "Incentivized" in the abstract meaning the seven indicators informed in 2013 and incentivized in 2015. Ruled out. In Table 1 none of these rows has a significant step change, and all five significant trend changes are negative. The Results text says the same: incentivizing previously informed indicators "did not result in significant step changes in results or variability, nor in favorable changes in post-intervention trends". The abstract's "incentivized" figures (6.66, 1.25, 8.87) can only come from the "Introduced as Informed" rows.
- Derived indicators or a different subset. Ruled out. The abstract's next sentence treats derived indicators separately ("minor step increases up to 2.22 percentage points"). None of 19.67, 1.07, 1.09, 0.34, 6.66 or 8.87 appears among the derived rows of Table 1. The supplement does not contain 19.67, 8.87, 6.66, 1.07 or 1.25 anywhere.
- A different metric (variability rather than results). Ruled out. Table 2 values are coefficient-of-variation changes on a scale of hundredths (for example −0.18 to −0.001). The abstract's ranges are percentage points and match Table 1 exactly.
- A text-conversion artifact. Ruled out. The abstract's wording is the same in the live PMC page and in the Europe PMC full-text XML deposited by the publisher. The swap lies in which words go with which numbers, which a conversion step would not change.
- A correction already issued. None found (see Status).
- Side observations, not part of the main finding. (1) The incentivization section says "Four showed significant step increases" with the range 19.67 to 1.07. Table 1 has five favourable significant steps in that group (19.67, 19.66, 2.36, and the inverse-indicator steps −1.48 and −1.07). The range includes one inverse-indicator step but the count leaves out the other. (2) The information section gives the informed step improvements as "from 8.87 to 1.25". The smallest of the four informed steps is 3.41 (respiratory disease diagnosis). 1.25 is the smallest favourable informed trend change, which ends the previous range in the same sentence. (3) The incentivized row "Weight reduction in obese and overweight patients" has a significant unfavourable step (−2.86) together with a favourable trend change (0.89), and is counted among the positive impacts by the paper's either-metric rule. That follows the stated definition and is not an error.
Status
- Version of record. The publisher pages (doi.org resolves to linkinghub.elsevier.com, pii S2666776224002692; thelancet.com; sciencedirect.com) returned only a redirect stub or HTTP 403 to automated requests. The text was therefore checked in two copies of the publisher-deposited full text: the live PMC page (pmc.ncbi.nlm.nih.gov/articles/PMC11513846/) and the Europe PMC full-text XML. Both carry the abstract and Results passages quoted above and the same Table 1 values. Every quotation was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The single supplementary file,
mmc1.pdf("Supplementary Figures and Tables", 27 pages, from the Europe PMC supplementary bundle), holds indicator descriptions, a contingency table, ITS analyses of the 24 robustness-check indicators (results and CV), logit sensitivity analyses and supplementary figures. None of 6.66, 8.87, 19.67, 1.07 or 1.25 occurs in its text, and it contains no summary statement assigning the ranges to either arm. It neither resolves nor alters the finding. The group of each row is fixed by the "Scheme change" column of main-text Table 1. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. A web search for the title with "correction", "erratum" or "PubPeer" found no notice or thread. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the DOI with "PubPeer" found no thread. - Sources. PMC article PMC11513846; Europe PMC full-text XML and supplementary files for PMC11513846 (
mmc1.pdf); Crossref; Europe PMC; PubPeer; https://doi.org/10.1016/j.lanepe.2024.101102.
Suggested correction
In the abstract, Findings, replace
Incentivized indicators displayed improvements in annual trends ranging from 6.66 to 1.25 percentage points, with step changes up to 8.87 percentage points. Information led to step changes ranging from 19.67 to 1.07 percentage points, along with trend improvements between 1.09 and 0.34 percentage points annually.
with
Information led to improvements in annual trends ranging from 6.66 to 1.25 percentage points, with step changes up to 8.87 percentage points. Incentivized indicators displayed step changes ranging from 19.67 to 1.07 percentage points, along with trend improvements between 1.09 and 0.34 percentage points annually.
The authors may also wish to check two smaller points in the Results against Table 1. In the information section, the informed step improvements run from 8.87 to 3.41, not "to 1.25". In the incentivization section, five, not four, favourable significant step changes lie in the stated 19.67 to 1.07 range, if the inverse-indicator steps (−1.48 and −1.07) are counted as the range implies.
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC11513846
(Lancet Reg Health Eur 2024, doi:10.1016/j.lanepe.2024.101102; information provision vs financial
incentives in Catalan primary care, interrupted time series 2010-2019).
Run: python recompute.py.txt
Needs matplotlib. Arithmetic uses only the standard library.
Prints every recomputed number used in writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the article as archived in PubMed Central (identical in the Europe PMC full-text XML):
Table 1 (ITS on indicator results, percentage points): step change and change in trend, each with 95% CI,
for the 11 new indicators "Introduced as Incentivized", the 12 new indicators "Introduced as Informed"
and the 7 new indicators "Informed to Incentivized" (2015).
Table 2 (ITS on the coefficient of variation, CV): step and trend changes with 95% CI for the same rows
(used only to check the abstract's shares 9/12 and 7/11).
Table 1 footnote a: "Inverse indicators (negative step and trend changes represent improvement)."
Methods: significance at p < 0.05; "Significant improvements in either metric were classified as 'positive impact'".
Significance is taken as a 95% CI that excludes 0; this is checked against the table's significance marks.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Table 1, new indicators)
# name, inverse?, step (est, lo, hi, marked significant), trend change (est, lo, hi, marked significant)
INCENT = [ # "Introduced as Incentivized"
("Alcohol reduction, high-risk drinkers", False, (19.67, 11.34, 27.99, True), (-0.70, -3.14, 1.74, False)),
("Asymptomatic hyperuricemia treatment", False, (19.66, 16.60, 22.72, True), (0.25, -0.66, 1.16, False)),
("Bronchitis/rhinopharyngitis/flu treatment", False, (2.36, 0.74, 3.98, True), (1.09, 0.52, 1.65, True)),
("Inadequate gastropathy prevention, PPIs (inv.)", True, (-1.48, -2.50, -0.47, True), (-1.09, -1.36, -0.83, True)),
("Low CV risk, lipid-lowering agents (inv.)", True, (-1.07, -1.55, -0.60, True), (-0.34, -0.48, -0.20, True)),
("Weight reduction, obese/overweight", False, (-2.86, -5.42, -0.31, True), (0.89, 0.09, 1.68, True)),
("Acute tonsillitis treatment", False, (-0.85, -1.78, 0.08, False), (-0.27, -0.64, 0.11, False)),
("Acute gastroenteritis treatment", False, (-0.07, -0.73, 0.59, False), (0.16, -0.02, 0.34, False)),
("Adult triple viral vaccination", False, (0.10, -0.48, 0.68, False), (-0.78, -0.94, -0.62, True)),
("Anticoagulant control", False, (-2.12, -3.68, -0.57, True), (-0.62, -1.18, -0.06, True)),
("Incorrect PSA use >70 (inv.)", True, (-0.56, -1.63, 0.51, False), (1.21, 0.97, 1.45, True)),
]
INFORM = [ # "Introduced as Informed"
("Dx adequacy, cardiovascular disease", False, (8.87, 6.32, 11.41, True), (6.66, 5.56, 7.77, True)),
("Dx adequacy, hypertension", False, (6.02, 5.46, 6.58, True), (5.70, 5.46, 5.95, True)),
("Dx adequacy, obesity", False, (6.59, 5.11, 8.07, True), (1.95, 1.31, 2.59, True)),
("Dx adequacy, respiratory disease", False, (3.41, 3.03, 3.79, True), (3.18, 3.01, 3.34, True)),
("Annual ECG, AF risk population", False, (8.05, -2.15, 18.25, False), (5.53, 3.60, 7.45, True)),
("Excessive control of T2DM (inv.)", True, (-2.15, -4.65, 0.36, False), (-1.25, -1.74, -0.77, True)),
("Dx adequacy, hypercholesterolemia", False, (1.21, -2.23, 4.65, False), (0.81, -0.68, 2.30, False)),
("Dx adequacy, other health problems", False, (0.28, -4.63, 5.20, False), (-1.69, -3.81, 0.44, False)),
("Dx quality, T2DM", False, (-0.24, -0.84, 0.36, False), (-0.12, -0.38, 0.14, False)),
("T2DM albuminuria screening", False, (-2.92, -7.02, 1.18, False), (-1.22, -2.54, 0.09, False)),
("Antiplatelet, T2DM without CVD (inv.)", True, (2.51, 1.74, 3.28, True), (0.24, 0.09, 0.38, True)),
("Biennial heart rate, age >=60", False, (-2.38, -5.05, 0.30, False), (-2.09, -2.70, -1.48, True)),
]
INF2INC = [ # "Informed to Incentivized" (2015); none is an inverse indicator
("Dx adequacy, hypercholesterolemia", False, (8.37, -3.22, 19.96, False), (-0.15, -2.59, 2.30, False)),
("Dx adequacy, hypertension", False, (5.19, -4.90, 15.29, False), (-1.40, -3.68, 0.89, False)),
("Dx adequacy, cardiovascular disease", False, (-2.69, -12.01, 6.62, False), (-8.93, -10.94, -6.92, True)),
("Dx adequacy, obesity", False, (2.45, -3.82, 8.73, False), (-3.24, -4.61, -1.88, True)),
("Dx adequacy, other health problems", False, (-2.01, -8.17, 4.15, False), (-2.81, -3.55, -2.07, True)),
("Dx adequacy, respiratory disease", False, (-2.94, -6.26, 0.38, False), (-5.36, -6.05, -4.66, True)),
("Dx quality, T2DM", False, (0.10, -0.71, 0.90, False), (-0.67, -0.77, -0.57, True)),
]
# Table 2 (CV), same indicator names: (step est, lo, hi), (trend est, lo, hi). A CV reduction is favourable.
CV = {
"incent": {
"Alcohol reduction, high-risk drinkers": ((-0.14, -0.24, -0.04), (0.06, 0.03, 0.09)),
"Asymptomatic hyperuricemia treatment": ((-0.11, -0.12, -0.09), (0.001, -0.003, 0.005)),
"Bronchitis/rhinopharyngitis/flu treatment": ((-0.01, -0.01, -0.001), (-1e-4, -0.002, 0.002)),
"Weight reduction, obese/overweight": ((-0.01, -0.03, -0.003), (0.002, -0.001, 0.005)),
"Incorrect PSA use >70 (inv.)": ((-0.02, -0.04, 0.004), (-0.03, -0.04, -0.03)),
"Acute gastroenteritis treatment": ((-0.003, -0.01, 0.003), (-4e-4, -0.002, 0.001)),
"Low CV risk, lipid-lowering agents (inv.)": ((0.01, -0.01, 0.02), (-0.003, -0.01, 0.004)),
"Adult triple viral vaccination": ((0.005, -0.01, 0.01), (0.01, 0.01, 0.02)),
"Anticoagulant control": ((0.03, -0.01, 0.06), (0.02, 0.005, 0.03)),
"Inadequate gastropathy prevention, PPIs (inv.)": ((-0.001, -0.01, 0.004), (0.01, 0.01, 0.01)),
"Acute tonsillitis treatment": ((0.002, -0.01, 0.01), (0.01, 0.005, 0.01)),
},
"inform": {
"Dx adequacy, cardiovascular disease": ((-0.18, -0.24, -0.12), (-0.05, -0.08, -0.03)),
"Dx adequacy, hypertension": ((-0.06, -0.06, -0.06), (-0.05, -0.05, -0.05)),
"Dx adequacy, obesity": ((-0.04, -0.04, -0.03), (-0.04, -0.05, -0.04)),
"Antiplatelet, T2DM without CVD (inv.)": ((-0.03, -0.03, -0.02), (-0.004, -0.01, -9e-4)),
"Dx adequacy, hypercholesterolemia": ((-0.02, -0.02, -0.02), (-0.02, -0.03, -0.02)),
"Dx adequacy, respiratory disease": ((-0.02, -0.02, -0.01), (-0.02, -0.02, -0.02)),
"Dx quality, T2DM": ((-0.001, -0.002, -7e-4), (0.0, -2e-4, 2e-4)),
"Annual ECG, AF risk population": ((-0.02, -0.05, 0.01), (-0.02, -0.03, -0.01)),
"Dx adequacy, other health problems": ((0.02, 0.01, 0.03), (0.003, -0.002, 0.01)),
"Biennial heart rate, age >=60": ((0.05, 0.02, 0.08), (0.03, 0.02, 0.03)),
"Excessive control of T2DM (inv.)": ((0.02, -3e-4, 0.03), (0.01, 0.01, 0.01)),
"T2DM albuminuria screening": ((0.06, -9e-4, 0.11), (0.03, 0.01, 0.04)),
},
}
# What the paper prints
ABSTRACT = { # label in the abstract -> printed ranges
"Incentivized": {"trend": (6.66, 1.25), "step_max": 8.87},
"Information": {"step": (19.67, 1.07), "trend": (1.09, 0.34)},
}
RESULTS = { # Results text, per section
"Incentivized": {"n_step": 4, "step": (19.67, 1.07), "n_trend": 4, "trend": (1.09, 0.34)},
"Informed": {"n_trend": 6, "trend": (6.66, 1.25), "n_step": 4, "step": (8.87, 1.25)},
}
def sig(t):
est, lo, hi = t[:3]
return lo > 0 or hi < 0
def favourable(inverse, est):
return est < 0 if inverse else est > 0
def fav_sig(rows, which):
"""Favourable, significant effects as improvement magnitudes (sign flipped for inverse indicators)."""
k = 2 if which == "step" else 3
out = []
for r in rows:
t = r[k]
if sig(t) and favourable(r[1], t[0]):
out.append((abs(t[0]), r[0]))
return sorted(out, reverse=True)
def rng(vals):
return (max(v for v, _ in vals), min(v for v, _ in vals)) if vals else None
print("=== 0. Significance flags: does 'CI excludes 0' reproduce the table's marks? ===")
mism = 0
for rows in (INCENT, INFORM, INF2INC):
for r in rows:
for t in (r[2], r[3]):
mism += sig(t) != t[3]
print(f" rows checked: {len(INCENT) + len(INFORM) + len(INF2INC)}, mismatches between CI and marks: {mism}")
GROUPS = [("Introduced as Incentivized (11 new)", INCENT), ("Introduced as Informed (12 new)", INFORM),
("Informed to Incentivized, 2015 (7)", INF2INC)]
REC = {}
print("\n=== 1. Favourable significant effects in Table 1, by scheme change ===")
for lab, rows in GROUPS:
s, t = fav_sig(rows, "step"), fav_sig(rows, "trend")
REC[lab] = (s, t)
print(f" {lab}")
print(f" step changes n={len(s)}: {[v for v, _ in s]} range {rng(s)}")
for v, n in s:
print(f" {v:6.2f} {n}")
print(f" trend changes n={len(t)}: {[v for v, _ in t]} range {rng(t)}")
for v, n in t:
print(f" {v:6.2f} {n}")
inc_s, inc_t = REC[GROUPS[0][0]]
inf_s, inf_t = REC[GROUPS[1][0]]
print("\n=== 2. Which Table 1 group does each printed range come from? ===")
def source(value, kind):
hits = []
for lab, rows in GROUPS:
vals = REC[lab][0 if kind == "step" else 1]
if any(abs(v - value) < 0.005 for v, _ in vals):
hits.append(lab)
return hits
checks = [
("Abstract 'Incentivized' trend, upper", 6.66, "trend"), ("Abstract 'Incentivized' trend, lower", 1.25, "trend"),
("Abstract 'Incentivized' step, max", 8.87, "step"),
("Abstract 'Information' step, upper", 19.67, "step"), ("Abstract 'Information' step, lower", 1.07, "step"),
("Abstract 'Information' trend, upper", 1.09, "trend"), ("Abstract 'Information' trend, lower", 0.34, "trend"),
]
for lab, v, kind in checks:
print(f" {lab:38s} {v:6.2f} ({kind}) -> found among favourable significant {kind}s of: {source(v, kind)}")
print("\n=== 3. Results text vs Table 1 ===")
print(f" Incentivization section: steps 'Four', 19.67 to 1.07 -> Table 1: n={len(inc_s)}, {rng(inc_s)}")
print(f" trends 'four', 1.09 to 0.34 -> Table 1: n={len(inc_t)}, {rng(inc_t)}")
print(f" Information section: trends 'six', 6.66 to 1.25 -> Table 1: n={len(inf_t)}, {rng(inf_t)}")
print(f" steps 'four', 8.87 to 1.25 -> Table 1: n={len(inf_s)}, {rng(inf_s)}"
f" (1.25 is the smallest favourable trend, not a step)")
inc_pos_steps = [r for r in INCENT if sig(r[2]) and r[2][0] > 0 and not r[1]]
print(f" Incentivized rows with a significant positive step (non-inverse only): n={len(inc_pos_steps)} "
f"{[r[2][0] for r in inc_pos_steps]}; with inverse indicators' favourable steps added: n={len(inc_s)}")
print("\n=== 4. 'Informed to Incentivized' rows as an alternative reading of the abstract's 'Incentivized' ===")
s2, t2 = REC[GROUPS[2][0]]
neg = [r[3][0] for r in INF2INC if sig(r[3])]
print(f" favourable significant steps: {len(s2)}; favourable significant trends: {len(t2)}; "
f"significant trend changes (all negative): {neg}")
print("\n=== 5. Ratios between the arms on the corrected assignment ===")
print(f" largest step: incentivized {inc_s[0][0]} vs informed {inf_s[0][0]} -> ratio {inc_s[0][0] / inf_s[0][0]:.2f}")
print(f" largest trend: informed {inf_t[0][0]} vs incentivized {inc_t[0][0]} -> ratio {inf_t[0][0] / inc_t[0][0]:.2f}")
print(f" as printed, the abstract's largest step is under 'Information' ({ABSTRACT['Information']['step'][0]}) "
f"and its largest trend under 'Incentivized' ({ABSTRACT['Incentivized']['trend'][0]})")
print("\n=== 6. Abstract shares of indicators with 'positive impact' (results Table 1 or CV Table 2) ===")
for key, rows, printed in [("incent", INCENT, "7/11"), ("inform", INFORM, "9/12")]:
pos_res = {r[0] for r in rows if any(sig(t) and favourable(r[1], t[0]) for t in (r[2], r[3]))}
pos_cv = {n for n, (a, b) in CV[key].items() if any(sig(t) and t[0] < 0 for t in (a, b))}
both = pos_res | pos_cv
print(f" {key}: results-positive {len(pos_res)}/{len(rows)}; CV-positive {len(pos_cv)}/{len(rows)}; "
f"either {len(both)}/{len(rows)} = {100 * len(both) / len(rows):.0f}% (abstract: {printed})")
cv_step = [n for n, (a, b) in CV[key].items() if sig(a) and a[0] < 0]
cv_tr = [n for n, (a, b) in CV[key].items() if sig(b) and b[0] < 0]
print(f" CV significant step reductions n={len(cv_step)}; CV significant trend reductions n={len(cv_tr)}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def f2(x):
return f"{x:.2f}"
# ---- Figure 1: two text panels joined by crossing connectors
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.9)
ax.axis("off")
ax.text(0.15, 4.62, "The abstract prints each arm's effect sizes under the other arm's label",
fontsize=11.5, fontweight="bold", va="center")
LX, LW, RX, RW, BOT, TOP = 0.15, 2.75, 4.2, 2.85, 1.0, 4.25
for x, w, title in [(LX, LW, "Abstract, Findings"), (RX, RW, "Results text and Table 1")]:
ax.add_patch(FancyBboxPatch((x, BOT), w, TOP - BOT, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=SEC, lw=1))
ax.text(x + 0.15, TOP - 0.25, title, fontsize=11, fontweight="bold", va="center")
ab = ABSTRACT
left = [
(3.35, "“Incentivized indicators …”",
f"trends {f2(ab['Incentivized']['trend'][0])} to {f2(ab['Incentivized']['trend'][1])} pp/yr\n"
f"steps up to {f2(ab['Incentivized']['step_max'])} pp"),
(1.95, "“Information led to …”",
f"steps {f2(ab['Information']['step'][0])} to {f2(ab['Information']['step'][1])} pp\n"
f"trends {f2(ab['Information']['trend'][0])} to {f2(ab['Information']['trend'][1])} pp/yr"),
]
for y, lab, vals in left:
ax.text(LX + 0.15, y + 0.3, lab, fontsize=10.5, va="center", color=TXT)
ax.text(LX + 0.15, y - 0.12, vals, fontsize=10.5, va="center", color=ORANGE, fontweight="bold",
linespacing=1.35)
right = [
(3.35, "Introduced as Informed (12 new)",
f"trends {f2(rng(inf_t)[0])} to {f2(rng(inf_t)[1])} pp/yr (n={len(inf_t)})\n"
f"steps {f2(rng(inf_s)[0])} to {f2(rng(inf_s)[1])} pp (n={len(inf_s)})"),
(1.95, "Introduced as Incentivized (11 new)",
f"steps {f2(rng(inc_s)[0])} to {f2(rng(inc_s)[1])} pp (n={len(inc_s)})\n"
f"trends {f2(rng(inc_t)[0])} to {f2(rng(inc_t)[1])} pp/yr (n={len(inc_t)})"),
]
for y, lab, vals in right:
ax.text(RX + 0.15, y + 0.3, lab, fontsize=10.5, va="center", color=TXT)
ax.text(RX + 0.15, y - 0.12, vals, fontsize=10.5, va="center", color=BLUE, fontweight="bold",
linespacing=1.35)
x0, x1 = LX + LW + 0.06, RX - 0.06
for ya, yb in [(3.25, 1.85), (1.85, 3.25)]:
ax.annotate("", xy=(x1, yb), xytext=(x0, ya),
arrowprops=dict(arrowstyle="-|>", color=SEC, lw=2, shrinkA=2, shrinkB=2, mutation_scale=12))
ax.text((x0 + x1) / 2, 2.55, "same\nnumbers,\nlabels\nswapped", fontsize=10, ha="center", va="center",
color=TXT, bbox=dict(fc=BG, ec="none", pad=1.5), linespacing=1.2)
ax.text(LX + 0.15, 1.25, "printed range; label as in abstract", fontsize=10, color=SEC, va="center")
ax.text(RX + 0.15, 1.25, "recomputed from Table 1 rows", fontsize=10, color=SEC, va="center")
ax.text(0.15, 0.68, "Orange: ranges as printed in the abstract. Blue: favourable, significant (95% CI excludes 0)",
fontsize=10, va="center")
ax.text(0.15, 0.4, "Table 1 effects for the new indicators of each scheme; inverse indicators sign-flipped.",
fontsize=10, va="center")
ax.text(0.15, 0.12, "pp = percentage points. The Results text gives the informed steps as “8.87 to 1.25”.",
fontsize=10, color=SEC, va="center")
save(fig, "fig1")
# ---- Figure 2: favourable significant effects per arm (Table 1) vs the abstract's range for that label
fig, axes = plt.subplots(1, 2, figsize=(7.2, 3.5), sharey=True)
fig.subplots_adjust(left=0.17, right=0.98, top=0.8, bottom=0.27, wspace=0.12)
ROWS = {"Incentivized": 1, "Informed": 0}
abs_rng = { # abstract range printed under each label (Information = informed arm)
"step": {"Incentivized": (0, ab["Incentivized"]["step_max"]), "Informed": ab["Information"]["step"]},
"trend": {"Incentivized": ab["Incentivized"]["trend"], "Informed": ab["Information"]["trend"]},
}
data = {"step": {"Incentivized": inc_s, "Informed": inf_s}, "trend": {"Incentivized": inc_t, "Informed": inf_t}}
titles = {"step": "Step change (pp)", "trend": "Change in trend (pp per year)"}
for ax, kind in zip(axes, ["step", "trend"]):
for arm, y in ROWS.items():
lo, hi = sorted(abs_rng[kind][arm])
ax.plot([lo, hi], [y + 0.17, y + 0.17], color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot([lo, hi], [y + 0.17, y + 0.17], "|", color=ORANGE, ms=10, mew=2)
if kind == "step" and arm == "Incentivized":
ax.text(hi, y + 0.32, "“up to 8.87”", fontsize=10, color=ORANGE, ha="right", va="bottom")
vals = [v for v, _ in data[kind][arm]]
ax.plot(vals, [y - 0.15] * len(vals), "o", color=BLUE, ms=8, mfc=BLUE, mec=BG, mew=0.8, alpha=0.9)
ax.set_title(titles[kind], fontsize=10.5, loc="left")
ax.set_xlim(0, 21 if kind == "step" else 7.2)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
axes[0].set_yticks([1, 0])
axes[0].set_yticklabels(["Incentivized\n(11 new)", "Informed\n(12 new)"], fontsize=10.5)
axes[0].set_ylim(-0.55, 1.6)
fig.suptitle("Each arm's Table 1 effects match the range the abstract gives the other arm", x=0.02, ha="left",
fontsize=11.5, fontweight="bold")
fig.legend(handles=[Line2D([], [], color=ORANGE, lw=2, marker="|", ms=10, mew=2,
label="range the abstract prints for this arm"),
Line2D([], [], color=BLUE, lw=0, marker="o", ms=8,
label="Table 1: favourable significant effects")],
loc="lower center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.55, 0.0))
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract reports low-birth-weight estimates, including a nominally significant ASD effect, as preterm-birth results
Lancet Reg Health Eur (2025) · doi:10.1016/j.lanepe.2025.101247 · PMC11910110
Summary
The abstract's Findings say that, compared with term birth, preterm birth was associated with higher ASD symptoms, 3·23 (95% CI 0·37, 6·08) percentiles, and give preterm effects of 4·15 for ADHD symptoms and −5·44 for non-verbal IQ. All three values, estimates and both CI limits, are the cells of the birth weight <2500 g row of Table 4, not the gestational age <37 wk row. Table 4's preterm row gives ASD 0·60 (−2·60, 3·79), which is null (p ≈ 0·71 recomputed from the CI), ADHD 2·51 and non-verbal IQ −3·41; the Results and Discussion agree with the table and state explicitly that preterm birth was not associated with ASD symptoms. A reader of the abstract alone would therefore take away a preterm–ASD association that the paper's own analysis did not find, and preterm effects on ADHD and IQ about 1·6 times larger than estimated.
What the paper reports
Abstract, Findings (second sentence):
As compared to term birth, preterm birth was associated with higher internalizing problems (difference 3·43 (95% CI: 2·52, 4·33)) and externalizing problems (difference 2·31 (95% CI: 1·16, 3·46)), ADHD symptoms (difference 4·15 (95% CI: 3·15, 5·16)), ASD symptoms (difference 3·23 (95% CI: 0·37, 6·08)), and lower non-verbal IQ (difference -5·44 (95% CI: -7·44, -3·44)).
Results, "Birth outcomes, emotional, behavior and cognitive outcomes", paragraph 2:
As compared to children born term, those born preterm had increased scores for ADHD symptoms in childhood (2·51 percentiles (95% CI: 1·61, 3·40)) and early adolescence (2·33 percentiles (95% CI: 1·16, 3·50)).
Not preterm birth, but low birth weight was associated with ASD symptoms (3·23 percentiles (95% CI: 0·37, 6·08)).
Preterm birth and low birth weight were also associated with a -3·41 percentiles (95% CI: -5·46, -1·36) and -5·44 percentiles (95% CI: -7·44, -3·44) lower Non-verbal IQ in childhood respectively.
Discussion, paragraph 4:
The lack of association for preterm birth might also be explained by lower numbers since ASD was only available in two cohorts, that have both a rather healthy population, without extremely premature children.
Table 4 (confounder model; childhood ADHD, ASD and non-verbal IQ columns), as printed:
<37 | 2·51 (1·61, 3·40)∗∗n = 4522 | … | 0·60 (-2·60, 3·79)n = 279 | -3·41 (-5·46, -1·36)∗∗n = 780
<2500 | 4·15 (3·15, 5·16)∗∗n = 3531 | … | 3·23 (0·37, 6·08)∗N = 353 | -5·44 (-7·44, -3·44)∗∗N = 821
Table 4 footer: "∗p value < 0·05, ∗∗p value < 0·01". Methods, Statistical analysis:
To correct for multiple testing, we used the Bonferroni correction method taking account for five outcomes and specified multiple-test corrected p-values as <0·01.
What the paper's own numbers show
Method. Each number in the abstract's "preterm birth" sentence was matched, estimate and both CI limits, against every relevant cell of Tables 3 and 4 (confounder model). For each cell a standard error was recovered from the printed 95% CI as SE = (upper − lower)/3·92 and a two-sided Wald p-value computed. The basic-model (Table S5) and complete-case (Table S9) results in the paper's Appendix A were checked the same way, to rule out that the abstract quotes a different model or analysis set. All arithmetic is in recompute.py.txt.
| Quantity | Location | Reported | Recomputed / located |
|---|---|---|---|
| ASD symptoms, "preterm birth" | Abstract, Findings | 3·23 (0·37, 6·08) | Identical to Table 4 <2500 g ASD cell (N = 353); not in the <37 wk row |
| ASD symptoms, preterm <37 wk | Table 4 | 0·60 (−2·60, 3·79), n = 279, no asterisk | SE 1·630, z 0·37, p = 0·71; CI includes 0 |
| ASD symptoms, birth weight <2500 g | Table 4; Results | 3·23 (0·37, 6·08)∗ | SE 1·457, z 2·22, p = 0·027 (matches ∗; does not meet the paper's Bonferroni p < 0·01) |
| ADHD symptoms (childhood), "preterm birth" | Abstract, Findings | 4·15 (3·15, 5·16) | Identical to Table 4 <2500 g cell; preterm cell is 2·51 (1·61, 3·40), p < 0·001; abstract/preterm ratio 1·65 |
| Non-verbal IQ, "preterm birth" | Abstract, Findings | −5·44 (−7·44, −3·44) | Identical to Table 4 <2500 g cell; preterm cell is −3·41 (−5·46, −1·36), p = 0·001; ratio 1·60 |
| Internalizing (childhood), "preterm birth" | Abstract, Findings | 3·43 (2·52, 4·33) | Identical to Table 3 <37 wk childhood cell (correct) |
| Externalizing, "preterm birth" | Abstract, Findings | 2·31 (1·16, 3·46) | Table 3 <37 wk early-adolescence cell; the childhood cell is 2·28 (1·43, 3·13). Correct exposure, different age window from the other four numbers (minor) |
| ASD, preterm, basic model | Appendix A, Table S5 | 1·35 (−1·89, 4·60) | z 0·82, p = 0·41; not the abstract value |
| ASD, <2500 g, basic model | Appendix A, Table S5 | 3·71 (0·82, 6·60)∗ | not the abstract value; no basic-model cell equals any abstract value |
| ASD, preterm, complete-case | Appendix A, Table S9 | −0·50 (−4·41, 3·41) | z −0·25, p = 0·80; no complete-case cell equals any abstract value |
| LGA vs AGA, ADHD (side finding) | Abstract vs Table 5 | Abstract −1·09 (−1·73, 0·45); Table 5 −1·09 (−1·73, −0·45)∗ | With +0·45 the CI midpoint is −0·64, not −1·09, and the interval would include 0; Table 5's interval is symmetric about −1·09 (p = 0·0008). The abstract upper limit has lost its minus sign |
Three of the five numbers in the "preterm birth" sentence (ADHD, ASD, non-verbal IQ) reproduce the confounder-model <2500 g row of Table 4 exactly, to two decimals in all three positions; none of them matches any preterm cell in the main, basic-model or complete-case tables, in any age window. In every one of these analyses the preterm ASD estimate is null. The first two numbers (internalizing, externalizing) do come from the preterm row of Table 3.
Figure
Figure 1. The abstract's ASD value (orange, as reported) is identical in estimate and both CI limits to the Table 4 row for birth weight <2500 g, not the row for gestational age <37 wk. The preterm row, 0·60 (−2·60, 3·79) with n = 279, has a recomputed p of 0·71; the low-birth-weight row has p = 0·027 (blue, recomputed from the printed 95% CIs). The Results and Discussion text (right) attribute the association to low birth weight and describe a lack of association for preterm birth.
Figure 2. For each outcome in the abstract's "preterm birth" sentence, the value stated in the abstract (orange circles) is plotted against the preterm (<37 wk vs 37–42 wk) childhood estimate from Tables 3 and 4 (blue squares), with 95% CIs on one axis of outcome-percentile differences and a reference line at no difference. The right-hand column names the table cell each abstract value reproduces and the recomputed p-value of the preterm cell. Internalizing and externalizing agree; ADHD and non-verbal IQ are overstated by about 60%; for ASD the abstract's interval excludes 0 while the paper's preterm interval is centred near 0.
Why it matters
- A null result is presented as a positive finding. The abstract states that preterm birth was associated with ASD symptoms, with a CI excluding zero. The paper's own preterm estimate is 0·60 (−2·60, 3·79), p ≈ 0·71, and the Results ("Not preterm birth, but low birth weight") and Discussion ("The lack of association for preterm birth") say the opposite of the abstract.
- Effect sizes are overstated. The preterm effects on ADHD symptoms (2·51) and non-verbal IQ (−3·41) appear in the abstract as 4·15 and −5·44, about 1·65× and 1·60× larger. Direction and statistical significance are unchanged for these two.
- The abstract is the part most often read, indexed and reused. The same sentence is in the PubMed record and in the STROBE checklist in the paper's appendix. Evidence summaries or meta-analyses that extract preterm effects from the abstract would take the low-birth-weight values.
- Even the relocated value is weak. The 3·23 ASD estimate, correctly attributed to low birth weight, has p ≈ 0·027 and does not meet the paper's own Bonferroni threshold (p < 0·01); it rests on two cohorts and 353 exposed children.
- What is not affected. The tables, Results text, Discussion and the Interpretation ("Both fetal and early childhood growth are associated with emotional, behavioral and cognitive outcomes…") are internally consistent and do not depend on these abstract numbers.
Explanations considered
- Preterm birth used loosely to mean "adverse birth characteristics". Does not account for it. The sentence opens "As compared to term birth, preterm birth", which names a specific contrast, and the paper itself distinguishes the two exposures for ASD ("Not preterm birth, but low birth weight").
- A different model (basic vs confounder-adjusted). Does not account for it. The abstract values equal the confounder-model cells of Table 4. The basic-model Table S5 gives preterm ASD 1·35 (−1·89, 4·60) and low-birth-weight ASD 3·71 (0·82, 6·60); neither equals 3·23 (0·37, 6·08).
- A different age window or population. Does not account for the ASD or non-verbal IQ values: both outcomes were measured only in childhood, in one set of cohorts (ASD: GENR and INMA). For ADHD the other preterm cells are 2·33 (early adolescence) and −1·74 (late adolescence), neither 4·15. Only the externalizing value is explained this way: 2·31 is the preterm early-adolescence cell, while the other numbers in the sentence are childhood values.
- A sensitivity analysis. Does not account for it. The complete-case Table S9 gives preterm ASD −0·50 (−4·41, 3·41), and no cell equals an abstract value. The two-stage results are shown only graphically (Figure S4) and described as "less precise"; the abstract values instead equal the one-stage confounder-model cells exactly. Apart from the abstract text repeated in the STROBE checklist, the triples 3·23 (0·37, 6·08), 4·15 (3·15, 5·16) and −5·44 (−7·44, −3·44) appear nowhere in the appendix.
- Coincidental agreement. Three triples (estimate, lower, upper) matching one specific table row exactly cannot plausibly be coincidence.
- A text-conversion artifact in PMC. Ruled out: the same sentence is in the Europe PMC JATS XML, in the PubMed abstract (from the publisher's deposit) and in the authors' own STROBE checklist in Appendix A.
- A typographical slip. Fits the LGA ADHD upper limit (0·45 for −0·45), a lost minus sign. It does not fit the main issue, where whole estimate–CI triples correspond to a different exposure row. The values coincide with the adjacent row of Table 4; the paper does not say how the abstract sentence was compiled, and nothing here depends on how it arose.
Status
As of 2026-10-23:
- Version of record. The publisher's pages (thelancet.com, sciencedirect.com, linkinghub) returned 403 or a redirect stub to automated access, so the publisher-hosted HTML/PDF could not be read directly. The same abstract wording and the same Table 4 cells were confirmed in four independent copies of the published text: the PMC article (show.py and the PMC web page), the Europe PMC JATS full-text XML (Table 4 parsed cell by cell, with U+2212 minus signs), the PubMed record PMID 40094119, and the STROBE checklist in the paper's own Appendix A (item 1(b) repeats the abstract sentence; item 16 repeats the Results sentence). A PDF-to-HTML conversion artifact is therefore excluded.
- Supplements. Appendix A (mmc1.docx, the only supplementary file) was read: Tables S4–S5 (basic models) and S8–S9 (complete-case analyses) do not contain the abstract values under preterm birth; preterm ASD is 1·35 (−1·89, 4·60) in Table S5 and −0·50 (−4·41, 3·41) in Table S9, both null.
- Corrections. No correction, erratum or retraction found in Crossref update metadata, PMC notices, or the Europe PMC record (no comment/correction links);
known.pywas re-run on 2026-10-23 and still returns no updates or notices. Web searches for the DOI and title with "correction"/"erratum", run earlier in this audit, found none; a further web search on 2026-10-23 could not be run (search quota exhausted). Because the publisher's own pages could not be read, a notice posted only on the publisher site cannot be fully excluded. - PubPeer. PubPeer's site and API refused automated access (403 / Cloudflare challenge); web searches for a PubPeer thread on the title or DOI found none. Absence of a thread is therefore not fully confirmed.
- Sources.
src/show.py PMC11910110(Abstract, Methods, Results, Discussion, Tables 3–5);src/known.py 10.1016/j.lanepe.2025.101247; Europe PMC REST full-text XML and supplementaryFiles for PMC11910110; PubMed efetch for PMID 40094119.
Suggested correction
In the Abstract, Findings, replace the second sentence so that each estimate is attributed to its own exposure and age window, for example:
As compared to term birth, preterm birth was associated with higher internalizing problems (difference 3·43 (95% CI: 2·52, 4·33)), externalizing problems (difference 2·28 (95% CI: 1·43, 3·13)) and ADHD symptoms (difference 2·51 (95% CI: 1·61, 3·40)) in childhood, and with lower non-verbal IQ (difference −3·41 (95% CI: −5·46, −1·36)), but not with ASD symptoms (difference 0·60 (95% CI: −2·60, 3·79)). Low birth weight (<2500 g) was associated with higher ADHD symptoms (difference 4·15 (95% CI: 3·15, 5·16)) and ASD symptoms (difference 3·23 (95% CI: 0·37, 6·08)) and lower non-verbal IQ (difference −5·44 (95% CI: −7·44, −3·44)).
If the early-adolescence externalizing value (2·31 (1·16, 3·46)) is preferred, label it as early adolescence. Consider noting that the low-birth-weight ASD association (p ≈ 0·03) does not meet the paper's Bonferroni threshold of p < 0·01. In the next sentence on large size for gestational age, correct the ADHD interval to (95% CI: −1·73, −0·45), as in Table 5. The same corrections apply to the abstract text repeated in the STROBE checklist (Appendix A, item 1(b)).
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC11910110
(Lancet Reg Health Eur 2025; doi:10.1016/j.lanepe.2025.101247), and redraw
fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Every estimate below is copied from the article as archived in PubMed Central
(Abstract Findings, Results text, Tables 3, 4 and 5; confounder model), plus
the basic-model Table S5 and complete-case Table S9 from the paper's Appendix A
(mmc1.docx) as cross-checks.
Values are differences in outcome percentiles with 95% CIs.
The only assumption is the usual normal approximation used to recover a
standard error from a printed 95% CI: SE = (upper - lower) / (2 * 1.959964).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.isf(0.025)
# ------------------------------------------------------------------ inputs
# Table cells: key -> (estimate, lower, upper, n exposed)
T = {
# Table 3 (internalizing / externalizing), gestational age <37 wk vs 37-42 wk
("T3", "<37 wk", "Internalizing", "childhood"): (3.43, 2.52, 4.33, 4780),
("T3", "<37 wk", "Internalizing", "early adolescence"): (1.77, 0.57, 2.96, 2873),
("T3", "<37 wk", "Externalizing", "childhood"): (2.28, 1.43, 3.13, 4777),
("T3", "<37 wk", "Externalizing", "early adolescence"): (2.31, 1.16, 3.46, 2873),
# Table 3, birth weight <2500 g vs 2500-4500 g
("T3", "<2500 g", "Internalizing", "childhood"): (3.20, 2.20, 4.20, 3887),
("T3", "<2500 g", "Externalizing", "childhood"): (3.77, 2.83, 4.71, 3885),
# Table 4 (ADHD / ASD / non-verbal IQ), gestational age <37 wk
("T4", "<37 wk", "ADHD", "childhood"): (2.51, 1.61, 3.40, 4522),
("T4", "<37 wk", "ADHD", "early adolescence"): (2.33, 1.16, 3.50, 2878),
("T4", "<37 wk", "ADHD", "late adolescence"): (-1.74, -2.89, -0.58, 2668),
("T4", "<37 wk", "ASD", "childhood"): (0.60, -2.60, 3.79, 279),
("T4", "<37 wk", "Non-verbal IQ", "childhood"): (-3.41, -5.46, -1.36, 780),
# Table 4, birth weight <2500 g
("T4", "<2500 g", "ADHD", "childhood"): (4.15, 3.15, 5.16, 3531),
("T4", "<2500 g", "ADHD", "early adolescence"): (3.51, 2.15, 4.87, 2078),
("T4", "<2500 g", "ASD", "childhood"): (3.23, 0.37, 6.08, 353),
("T4", "<2500 g", "Non-verbal IQ", "childhood"): (-5.44, -7.44, -3.44, 821),
# Table 5, large for gestational age, ADHD
("T5", "LGA", "ADHD", "childhood"): (-1.09, -1.73, -0.45, 9718),
}
# Abstract, Findings: "As compared to term birth, preterm birth was associated
# with higher internalizing problems (...), externalizing problems (...), ADHD
# symptoms (...), ASD symptoms (...), and lower non-verbal IQ (...)."
ABSTRACT = [
("Internalizing", 3.43, 2.52, 4.33),
("Externalizing", 2.31, 1.16, 3.46),
("ADHD", 4.15, 3.15, 5.16),
("ASD", 3.23, 0.37, 6.08),
("Non-verbal IQ", -5.44, -7.44, -3.44),
]
# Abstract, LGA sentence: "lower ADHD symptoms (difference -1.09 (95% CI: -1.73, 0.45))"
ABSTRACT_LGA = (-1.09, -1.73, 0.45)
BONFERRONI_ALPHA = 0.01 # Methods: "specified multiple-test corrected p-values as <0.01"
def wald(est, lo, hi):
se = (hi - lo) / (2 * Z975)
z = est / se
return se, z, 2 * stats.norm.sf(abs(z))
def fmt(x):
return f"{x:+.2f}"
# ------------------------------------------------------------ 1. cell checks
print("=== 1. Table cells: SE from 95% CI, Wald z, two-sided p ===")
REC = {}
for k, (e, l, h, n) in T.items():
se, z, p = wald(e, l, h)
REC[k] = (se, z, p)
print(f" {k[0]} {k[1]:8s} {k[2]:14s} {k[3]:18s} est {fmt(e)} "
f"({fmt(l)}, {fmt(h)}) n={n:<5d} CI midpoint {(l+h)/2:+.3f} "
f"SE {se:.3f} z {z:+.2f} p {p:.4f}")
# -------------------------------------- 2. where the abstract numbers come from
print("\n=== 2. Abstract 'preterm birth' sentence: which table cell each number matches ===")
MATCH = {}
for name, e, l, h in ABSTRACT:
hits = [k for k, v in T.items() if v[:3] == (e, l, h)]
MATCH[name] = hits
pre = T[(("T3" if name in ("Internalizing", "Externalizing") else "T4"),
"<37 wk", name, "childhood")]
print(f" {name:14s} abstract {fmt(e)} ({fmt(l)}, {fmt(h)})")
print(f" matches: {[' / '.join(k) for k in hits]}")
print(f" Table preterm (<37 wk) childhood cell: {fmt(pre[0])} "
f"({fmt(pre[1])}, {fmt(pre[2])}); abstract / preterm ratio {e/pre[0]:.2f}")
# --------------------------------------------------------------- 3. ASD focus
print("\n=== 3. ASD symptoms in childhood (Table 4) ===")
se_pt, z_pt, p_pt = REC[("T4", "<37 wk", "ASD", "childhood")]
se_lb, z_lb, p_lb = REC[("T4", "<2500 g", "ASD", "childhood")]
print(f" preterm <37 wk : 0.60 (-2.60, 3.79), SE {se_pt:.3f}, z {z_pt:.2f}, p {p_pt:.3f}; "
f"CI contains 0: {-2.60 < 0 < 3.79}")
print(f" LBW <2500 g : 3.23 (0.37, 6.08), SE {se_lb:.3f}, z {z_lb:.2f}, p {p_lb:.3f}; "
f"passes Bonferroni p<{BONFERRONI_ALPHA}: {p_lb < BONFERRONI_ALPHA}")
print(f" Table 4 flags the LBW cell with a single asterisk (p<0.05), consistent with p {p_lb:.3f}")
# -------------------------------------------------- 4. size of the overstatement
print("\n=== 4. Overstatement of preterm effects in the abstract (ADHD, non-verbal IQ) ===")
for name in ("ADHD", "Non-verbal IQ"):
a = dict((x[0], x[1:]) for x in ABSTRACT)[name][0]
pre = T[("T4", "<37 wk", name, "childhood")]
p_pre = REC[("T4", "<37 wk", name, "childhood")][2]
print(f" {name:14s} abstract {fmt(a)} vs preterm {fmt(pre[0])}: "
f"difference {a-pre[0]:+.2f}, ratio {a/pre[0]:.2f}, preterm p {p_pre:.4f}")
# ------------------------------------------------ 5. externalizing age window
print("\n=== 5. Externalizing: age window ===")
for w in ("childhood", "early adolescence"):
e, l, h, n = T[("T3", "<37 wk", "Externalizing", w)]
print(f" preterm externalizing, {w:18s}: {fmt(e)} ({fmt(l)}, {fmt(h)})")
print(" abstract uses +2.31 (early adolescence) next to internalizing +3.43 (childhood)")
# --------------------------------------------- 6. LGA ADHD sign (side finding)
print("\n=== 6. LGA ADHD CI in abstract vs Table 5 ===")
e, l, h = ABSTRACT_LGA
print(f" abstract: {fmt(e)} ({fmt(l)}, {fmt(h)}): midpoint {(l+h)/2:+.2f}, CI contains 0: {l < 0 < h}")
e5, l5, h5, _ = T[("T5", "LGA", "ADHD", "childhood")]
se5, z5, p5 = REC[("T5", "LGA", "ADHD", "childhood")]
print(f" Table 5 : {fmt(e5)} ({fmt(l5)}, {fmt(h5)}): midpoint {(l5+h5)/2:+.2f}, p {p5:.4f}")
# ------------------------- 7. basic model (Appendix A, Table S5) as a cross-check
# Supplement Table S5 (basic model: sex, age at outcome, cohort), childhood columns.
S5 = {
("<37 wk", "ADHD"): (3.26, 2.36, 4.16),
("<37 wk", "ASD"): (1.35, -1.89, 4.60),
("<37 wk", "Non-verbal IQ"): (-3.58, -5.67, -1.49),
("<2500 g", "ADHD"): (5.15, 4.14, 6.16),
("<2500 g", "ASD"): (3.71, 0.82, 6.60),
("<2500 g", "Non-verbal IQ"): (-5.68, -7.71, -3.65),
}
print("\n=== 7. Basic model (Table S5): could the abstract values come from it? ===")
for (row, out), (e, l, h) in S5.items():
se, z, p = wald(e, l, h)
a = dict((x[0], x[1:]) for x in ABSTRACT)[out]
print(f" {row:8s} {out:14s} {fmt(e)} ({fmt(l)}, {fmt(h)}) z {z:+.2f} p {p:.4f} "
f"equals abstract value: {(e, l, h) == a}")
# --------------- 8. complete-case analysis (Appendix A, Table S9) as a cross-check
S9 = {
("<37 wk", "ADHD"): (2.40, 1.24, 3.56),
("<37 wk", "ASD"): (-0.50, -4.41, 3.41),
("<37 wk", "Non-verbal IQ"): (-2.87, -5.28, -0.45),
("<2500 g", "ADHD"): (4.37, 3.02, 5.72),
("<2500 g", "ASD"): (2.11, -1.28, 5.50),
("<2500 g", "Non-verbal IQ"): (-4.53, -6.90, -2.17),
}
print("\n=== 8. Complete-case analysis (Table S9): could the abstract values come from it? ===")
for (row, out), (e, l, h) in S9.items():
se, z, p = wald(e, l, h)
a = dict((x[0], x[1:]) for x in ABSTRACT)[out]
print(f" {row:8s} {out:14s} {fmt(e)} ({fmt(l)}, {fmt(h)}) z {z:+.2f} p {p:.4f} "
f"equals abstract value: {(e, l, h) == a}")
# ================================================================ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f" wrote {stem}.svg and {stem}.png")
print("\n=== Figures ===")
# ---- fig1: abstract vs Table 4 / Results diagram (ASD)
fig = plt.figure(figsize=(7.2, 5.8))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 80)
ax.axis("off")
renderer = fig.canvas.get_renderer()
def panel(x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc="white", ec="#c9c7c1", lw=1.2))
ax.text(x + 1.8, y + h - 2.6, title, fontsize=11, fontweight="bold", va="top")
def runs(x, y, parts):
"""Draw text segments one after another on a line; return each segment's extent.
A part is (text, kwargs) or (text, kwargs, gap) where gap is extra space before it."""
ends = []
for part in parts:
txt, kw = part[0], part[1]
x += part[2] if len(part) > 2 else 0.0
t = ax.text(x, y, txt, fontsize=10.5, va="top", **kw)
bb = t.get_window_extent(renderer).transformed(ax.transData.inverted())
x = bb.x1
ends.append((bb.x0, bb.x1, bb.y0))
return ends
# Abstract panel (top)
panel(2, 55, 96, 23, "Abstract, Findings")
runs(3.8, 71.0, [("\u201cAs compared to term birth, preterm birth was associated with higher", {})])
runs(3.8, 67.2, [("internalizing problems (\u2026) and externalizing problems (\u2026), ADHD symptoms (\u2026),", {})])
seg = runs(3.8, 63.4, [("ASD symptoms (difference", {}),
("3\u00b723 (95% CI: 0\u00b737, 6\u00b708)", dict(color=ORANGE, fontweight="bold"), -0.65),
("), and lower", {}, -0.7)])
runs(3.8, 59.6, [("non-verbal IQ (\u2026).\u201d", {})])
val_x0, val_x1, val_y0 = seg[1]
# Table 4 panel (bottom left)
panel(2, 8, 56, 33, "Table 4, ASD symptoms (childhood)")
runs(3.8, 33.6, [("Gestational age <37 wk (preterm)", dict(fontweight="bold"))])
runs(3.8, 29.8, [("0\u00b760 (\u22122\u00b760, 3\u00b779), n = 279, no asterisk", {})])
runs(3.8, 26.0, [(f"recomputed: z = {z_pt:.2f}, p = {p_pt:.2f}", dict(color=BLUE, fontweight="bold"))])
runs(3.8, 20.0, [("Birth weight <2500 g (low birth weight)", dict(fontweight="bold"))])
lbw = runs(3.8, 16.2, [("3\u00b723 (0\u00b737, 6\u00b708)*, N = 353", {})])
runs(3.8, 12.4, [(f"recomputed: z = {z_lb:.2f}, p = {p_lb:.3f}", dict(color=BLUE, fontweight="bold"))])
# Results / Discussion panel (bottom right)
panel(61, 8, 37, 33, "Results and Discussion")
for i, line in enumerate(["\u201cNot preterm birth, but low birth", "weight was associated with ASD",
"symptoms (3\u00b723 percentiles", "(95% CI: 0\u00b737, 6\u00b708)).\u201d"]):
runs(62.8, 33.6 - 3.8 * i, [(line, {})])
runs(62.8, 16.2, [("\u201cThe lack of association for", {})])
runs(62.8, 12.4, [("preterm birth \u2026\u201d (Discussion)", {})])
# connector: abstract value -> low-birth-weight row, routed down the left panel's right margin
y_lbw = 16.2 - 1.6
vx = (val_x0 + val_x1) / 2
x_lane, y_jog = 59.5, 50.0
ax.plot([vx, vx, x_lane, x_lane], [val_y0 - 0.6, y_jog, y_jog, y_lbw], color=INK, lw=2,
solid_joinstyle="round", solid_capstyle="round")
ax.add_patch(FancyArrowPatch((x_lane, y_lbw), (lbw[0][1] + 1.0, y_lbw), arrowstyle="-|>",
mutation_scale=14, lw=2, color=INK))
ax.text(x_lane + 2.0, 46.5, "identical estimate and CI:\nTable 4 low-birth-weight row",
fontsize=10, ha="left", va="center", color=INK)
ax.text(3.0, 46.5, "Labelled \u2018preterm birth\u2019 in the abstract;\nthe preterm row is 0\u00b760, p = 0\u00b771",
fontsize=10, ha="left", va="center", color=INK2)
ax.text(2.0, 4.2, "Orange: as reported in the abstract. Blue: recomputed from the printed 95% CI "
"(SE = width/3.92).", fontsize=10, ha="left", va="center", color=INK2)
save(fig, "fig1")
# ---- fig2: dot-and-interval, abstract 'preterm' values vs Table preterm cells
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.subplots_adjust(left=0.20, right=0.68, top=0.86, bottom=0.14)
names = [a[0] for a in ABSTRACT]
src = {
"Internalizing": "= <37 wk, childhood",
"Externalizing": "= <37 wk, early adol.",
"ADHD": "= <2500 g row",
"ASD": "= <2500 g row",
"Non-verbal IQ": "= <2500 g row",
}
ypos = np.arange(len(names))[::-1] * 1.0
off = 0.17
for y, (name, e, l, h) in zip(ypos, ABSTRACT):
tab = "T3" if name in ("Internalizing", "Externalizing") else "T4"
pe, pl, ph, pn = T[(tab, "<37 wk", name, "childhood")]
p_pre = REC[(tab, "<37 wk", name, "childhood")][2]
ax.plot([l, h], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="round")
ax.plot(e, y + off, "o", color=ORANGE, ms=8, mec=BG, mew=1.5)
ax.plot([pl, ph], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="round")
ax.plot(pe, y - off, "s", color=BLUE, ms=8, mec=BG, mew=1.5)
ptxt = "p < 0.001" if p_pre < 0.001 else f"p = {p_pre:.3f}" if p_pre < 0.01 else f"p = {p_pre:.2f}"
ax.annotate(f"{e:+.2f} {src[name]}".replace("-", "\u2212", 1), xy=(1.02, y + off), xycoords=("axes fraction", "data"),
fontsize=10, va="center", color=INK, annotation_clip=False)
ax.annotate(f"{pe:+.2f} {ptxt}".replace("-", "\u2212", 1), xy=(1.02, y - off), xycoords=("axes fraction", "data"),
fontsize=10, va="center", color=INK2, annotation_clip=False)
ax.axvline(0, color=INK2, lw=0.8, zorder=0)
ax.set_yticks(ypos)
ax.set_yticklabels(names)
ax.set_ylim(ypos.min() - 0.6, ypos.max() + 0.6)
ax.set_xlim(-8.5, 7)
ax.set_xticks(np.arange(-8, 7, 2))
ax.set_xlabel("Difference vs term birth, outcome percentiles (95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#c9c7c1")
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2,
label="Abstract, stated as preterm birth (reported)")
h2, = ax.plot([], [], "s-", color=BLUE, ms=8, lw=2,
label="Table 3/4 preterm <37 wk, childhood; p recomputed from CI")
fig.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(0.01, 0.995), frameon=False,
fontsize=10, ncol=1)
ax.annotate("source of abstract value\n/ preterm cell p", xy=(1.02, ypos.max() + 0.55),
xycoords=("axes fraction", "data"), fontsize=10, color=INK2, va="bottom",
annotation_clip=False)
save(fig, "fig2")
Reported overall 90-day mortality of 63.7% conflicts with the paper's own counts of 408 deaths
Lancet Reg Health Eur (2025) · doi:10.1016/j.lanepe.2025.101317 · PMC12266187
Summary
The abstract and the Results give the overall 90-day mortality of the 1109 included patients as "63.7% (n = 707)". Every breakdown of those same 1109 patients in the paper adds up to 408 deaths (36.8%) and 701 survivors (63.2%). The breakdowns are the three PROMISE risk groups, the two GRIm risk groups, and the PROMISE and GRIm death counts in the four cancer-type subgroups. The reported median overall survival of 4.78 months is longer than 90 days, which on its own requires 90-day mortality below 50%. The printed figure therefore overstates the cohort's primary-outcome event rate by 299 deaths, about 27 percentage points. It is also used in the Discussion to argue that mortality was higher than in the cohort in which PROMISE was developed.
What the paper reports
The headline figure, in the abstract:
PROMISE and GRIm scores were calculated for all 1109 patients, while CTI score was assessed in 333 patients with complete laboratory data. The 90-day mortality rate was 63.7% (n = 707).
— Abstract, Findings
The same figure opens the outcome paragraph of the Results, followed directly by the stratified counts:
The overall 90-day mortality rate was 63.7% (n = 707). For the PROMISE score, 271 patients (97.5%) in the low-risk group survived, with a 90-day mortality rate of 2.5% (n = 7). In the intermediate-risk group, 176 patients (81.9%) survived, and the 90-day mortality rate was 18.1% (n = 39). In the high-risk group, 254 patients (41.2%) survived, with a 90-day mortality rate of 58.8% (n = 362) (p < 0.0001). According to the PROMISE score, the median overall survival (OS) for the entire cohort was 4.78 months (95% CI: 4.29–5.28). For the GRIm score, the 90-day survival rate was 87.8% (n = 449) for low-risk patients and 42.2% (n = 252) for high-risk patients (p < 0.0001). The estimated median OS for the entire cohort was 4.63 months (95% CI: 4.14–5.12).
— Results, overall outcome paragraph
The risk-group sizes:
According to the PROMISE score, 278 patients (25.1%) were categorized as low risk, 215 patients (19.4%) as intermediate risk, and 616 patients (55.5%) as high risk. The GRIm score classified 511 patients (46.0%) as low risk and 598 patients (54.0%) as high risk.
— Results, patient characteristics paragraph (Table 1 column headers give the GRIm groups as "Low risk, n = 510 (46.0%) | High risk, n = 599 (54.0%)")
The cancer-type subgroups (PROMISE and GRIm sentences only; the CTI sentences refer to the 333-patient subset):
For lung cancer patients (n = 283), mortality rates based on the PROMISE score were 2.4% (n = 1) in the low-risk group, 3.6% (n = 2) in the intermediate-risk group, and 51.9% (n = 96) in the high-risk group (p < 0.0001). [...] The GRIm score showed a 7.9% (n = 9) mortality rate in the low-risk group and 53.3% (n = 90) in the high-risk group (p < 0.0001).
For gastrointestinal cancer patients (n = 350), the PROMISE score demonstrated mortality rates of 6.0% (n = 5) in the low-risk group, 23.7% (n = 18) in the intermediate-risk group, and 63.2% (n = 120) in the high-risk group (p < 0.0001). [...] The GRIm scores indicated a 13.6% (n = 20) mortality rate in the low-risk group compared to 60.6% (n = 123) in the high-risk group (p < 0.0001).
For breast cancer patients (n = 92), the PROMISE score revealed 90-day mortality rates of 2.8% (n = 1) in the low-risk group, 21.4% (n = 3) in the intermediate-risk group, and 59.5% (n = 25) in the high-risk group (p < 0.0001). [...] The GRIm score demonstrated mortality rates of 6.8% (n = 3) in the low-risk group compared to 54.2% (n = 26) in the high-risk group (p < 0.0001).
For patients with other cancers (n = 384), PROMISE score analysis showed no 90-day mortality in the low-risk group (n = 0), a mortality rate of 23.2% (n = 16) in the intermediate-risk group, and 60.8% (n = 121) in the high-risk group (p < 0.0001). [...] The GRIm scores revealed a 14.6% (n = 30) mortality rate in the low-risk group and 59.8% (n = 107) in the high-risk group (p < 0.0001).
— Results, section "Subgroup analysis by cancer type" ([...] marks omitted median-OS sentences)
The separately reported subset of 333 patients with a CTI score:
In this study, the 90-day mortality rate among 333 patients assessed using the CTI score was 35.1% (n = 117).
— Results, CTI subset paragraph
The Discussion:
According to the results of our study, the 90-day mortality rate was higher compared to the study by Mirallas et al., in which the PROMISE score was developed.
In the study by Mirallas et al., survival rates for low, intermediate, and high-risk groups were reported as 83.6%, 68.0%, and 33.4%, respectively.
— Discussion
Each quotation was checked against the article as archived in PubMed Central (src/show.py PMC12266187). Each one occurs exactly once (passages joined by [...] were checked sentence by sentence), and recompute.py.txt reruns that check.
What the paper's own numbers show
Method. The paper states that PROMISE and GRIm were calculated for all 1109 patients, and the PROMISE group sizes (278 + 215 + 616) and the cancer-type subgroup sizes (283 + 350 + 92 + 384) each sum to 1109. Each of these breakdowns is therefore exhaustive, and summing its death counts gives the cohort's total number of 90-day deaths. For PROMISE the paper gives deaths and survivors per group. For GRIm it gives survivors per group, so deaths are group size minus survivors, using either the text's or Table 1's group sizes. For the cancer types it gives deaths per risk group under each score. As an independent check, the Kaplan–Meier median OS is compared with the 90-day time point. The 333-patient CTI subset is recomputed the same way. No number was estimated; every input is a count or percentage printed in the text or Table 1. recompute.py.txt prints every value below and redraws both figures.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Overall 90-day mortality, 1109 patients | Abstract, Findings; Results, outcome paragraph | 63.7% (n = 707) | 408 deaths, 36.8% (every row below). Arithmetic check: 707/1109 = 63.75% |
| PROMISE deaths, low + intermediate + high | Results, outcome paragraph | 7, 39, 362 (2.5%, 18.1%, 58.8% of 278, 215, 616) | 7 + 39 + 362 = 408. Group percentages reproduce exactly |
| PROMISE survivors | Results, outcome paragraph | 271, 176, 254 | 271 + 176 + 254 = 701; 408 + 701 = 1109 |
| GRIm deaths (group size − survivors) | Results, outcome paragraph; Table 1 | survivors 449 (87.8%), 252 (42.2%); n 510/599 (Table 1) or 511/598 (text) | (510 − 449) + (599 − 252) = 61 + 347 = 408; with text n, 62 + 346 = 408. Survivors 449 + 252 = 701 |
| Deaths by cancer type, PROMISE | Results, "Subgroup analysis by cancer type" | lung 1+2+96, GI 5+18+120, breast 1+3+25, other 0+16+121 | 99 + 143 + 29 + 137 = 408 (n = 1109) |
| Deaths by cancer type, GRIm | Results, "Subgroup analysis by cancer type" | lung 9+90, GI 20+123, breast 3+26, other 30+107 | 99 + 143 + 29 + 137 = 408; per cancer type identical to PROMISE |
| Median OS vs 90-day mortality | Results, outcome paragraph | median OS 4.78 months ("According to the PROMISE score"); 4.63 months (stated after the GRIm results) | Both exceed 90 days (2.96 months), so Kaplan–Meier survival at 90 days is above 50% and 90-day mortality is below 50%. 707 deaths within 90 days would force Kaplan–Meier mortality at 90 days to at least 63.8% and the median below 90 days |
| CTI subset, 333 patients | Results, CTI subset paragraph | 35.1% (n = 117) | 117/333 = 35.1%. Also (166 − 147) + (167 − 69) = 117; PROMISE within subset 3 + 12 + 102 = 117; Combined score 0 + 1 + 116 = 117; CTI by cancer type 21 + 40 + 20 + 36 = 117. Consistent |
| Gap between stated and recomputed | — | 707 (63.7%) | 707 − 408 = 299 deaths; 63.75 − 36.79 = 27.0 percentage points |
Four exhaustive breakdowns, drawn from two scores and two different groupings of the same patients, agree on 408 deaths. They also agree on 701 survivors where survivors are given. The subset of 333 patients, reported the same way, is fully consistent, so the reporting of the stratified counts can be reproduced throughout. Only the single overall figure departs from them. Its percentage, 63.7%, is close to the share who survived (701/1109 = 63.2%), not the share who died.
Figure
Figure 1. Number of the 1109 included patients who died within 90 days. The orange bar is the overall figure printed in the abstract and Results (707, "63.7%"). Each blue bar sums the death counts the paper itself reports for one exhaustive breakdown of the same 1109 patients: the three PROMISE risk groups, the two GRIm risk groups (group size from Table 1 minus reported survivors), and the PROMISE and GRIm deaths in the four cancer-type subgroups. Segments are drawn in the order the formula on each row lists them; unlabeled thin segments are 7, 39 and 29. The dashed line marks the cohort size. All four breakdowns give 408 deaths (36.8%). Values come from the article text and Table 1 and are recomputed by recompute.py.txt.
Figure 2. The same comparison as percentages on one 0–100% axis. The printed overall mortality (orange, 63.7%) lies in the shaded region that the paper's own median OS of 4.78 months rules out. A Kaplan–Meier median longer than 90 days means fewer than half the patients had died by day 90. The mortality recomputed from the strata (36.8%) lies outside it. The printed value sits next to the recomputed survival share (blue diamond, 63.2%). The bottom row shows the separately reported CTI subset, where the reported value (orange ring) and the recomputed value (blue dot) coincide at 35.1%.
Why it matters
- The abstract misstates the event rate of the primary outcome. 90-day mortality is the outcome the scores are validated against. A reader taking the abstract at face value would conclude that nearly two in three patients with an unplanned oncology admission died within three months. The paper's own data indicate a little over one in three.
- The Discussion's comparison with the development cohort rests on it. The paper says 90-day mortality "was higher compared to the study by Mirallas et al." and attributes the difference to access to palliative care and to palliative patients being admitted to oncology wards. That comparison and its explanation should be re-examined with 36.8%. The stratum-level survival figures the Discussion quotes for the development study (83.6%, 68.0%, 33.4%) are each lower than this cohort's (97.5%, 81.9%, 41.2%). The paper does not report the development cohort's overall rate, so whether any difference remains cannot be judged from this article.
- Baseline risk used by readers. Anyone using the abstract as a baseline risk for this setting, for example to turn sensitivity and specificity into predictive values for local use, would start from the wrong figure.
- What is not affected by this error. The stratum-level mortality percentages, the Kaplan–Meier comparisons and the 333-patient CTI-subset counts do not use the 707 figure and are internally consistent. The direction of the main finding, that PROMISE and CTI separate low-risk from high-risk patients, does not depend on it.
- A separate observation, not part of this finding. Table 2 (n = 333) reports sensitivity, specificity, PPV, NPV and accuracy for each score. With the subset's own prevalence of 35.1% (117/333), the reported sensitivity and specificity imply different predictive values. For PROMISE, for example, they imply PPV 57.4%, NPV 91.3% and accuracy 72.9%, against the reported 78.0%, 83.0% and 79.9%. Taken one at a time, the reported PPV, NPV and accuracy correspond to prevalences of roughly 50–65%, not 35.1%. Table 2 gives no 2×2 counts, so the source of this cannot be determined. It is recorded here only so that Table 2 is not assumed to be unaffected;
recompute.py.txtprints the calculation (section 9).
Explanations considered
- A different denominator or population. Does not account for it. The 707 is attached to the full cohort in the abstract ("all 1109 patients") and the Results ("overall"). The PROMISE, GRIm and cancer-type breakdowns each cover exactly 1109 patients. The CTI subset has its own figure (117 of 333), which is consistent.
- Survivors labelled as deaths. Possibly related, but not an exact match. The paper's survivors total 701 (63.2%), not 707 (63.7%), so swapping the labels does not reproduce the printed numbers exactly. The closeness of 63.7% to 63.2% is consistent with a survival-type figure, or a mistyped count, having been entered in place of the mortality figure. The article does not allow this to be settled.
- A different time point (for example, deaths over the whole follow-up). Does not account for it. Both the abstract and the Results say "90-day mortality". The stratified counts that sum to 408 are explicitly 90-day counts. A median OS of 4.78 months (or 4.63 months) is incompatible with more than half the cohort dying by day 90.
- Patients excluded from the stratified analyses, for example censored before 90 days. Does not account for it. The stratified breakdowns account for all 1109 patients as either deaths or survivors (408 + 701 = 1109), leaving no patients unclassified. No patients are left over (1109 − 408 − 701 = 0) to supply 299 additional deaths.
- GRIm group sizes differ between text and Table 1 (511/598 vs 510/599). A minor separate inconsistency that does not affect the result: either version gives 408 deaths.
- Rounding. Does not account for it. Rounding only concerns the last digit: 707/1109 = 63.75%, which rounds to 63.8% rather than the printed 63.7%. The 27-point gap is unaffected.
- Conversion or typesetting artifact. Ruled out. The figures are plain prose and identical in the publisher-deposited JATS XML (Europe PMC) and the live PMC article. They involve no symbols, superscripts or table cells.
- A published correction. None found (see Status).
Status
- Version of record. The Elsevier and Lancet pages (linkinghub, ScienceDirect, thelancet.com HTML and PDF) returned HTTP 403 or a redirect page to scripted access, so the typeset page could not be read directly. The publisher-deposited JATS XML in Europe PMC and the live PMC article contain the same sentences in the abstract and the Results. The publisher's own Figure 2 image (from the Europe PMC supplementary bundle) gives numbers at risk at 3 months of 271/176/254 for PROMISE and 449/252 for GRIm, which sum to 701 in each case. These are consistent with 408 patients no longer at risk by about 90 days, not 707. That figure is cited here, not reproduced.
- Supplement. The single supplementary file (mmc1.docx, "Supplementary Figures S1–S6 and Table S1") was read in full. It contains univariable odds ratios, calibration statistics, Kaplan–Meier curves by cancer type, and baseline characteristics by CTI availability (333 vs 776). It reports no overall 90-day death count and does not resolve the discrepancy. No dataset is deposited.
- Corrections and notices. No correction, erratum, retraction or expression of concern was found. Crossref has no update-to or updated-by relations, and a Crossref
updatesfilter search returned none. PMC and Europe PMC list no notices; Europe PMC lists only the article (PMID 40672056) and a linked commentary in the same journal (PMC12173626). - PubPeer. PubPeer's search and API return a human-verification challenge to scripted access, so it could not be queried directly. A web search for the title and DOI found no PubPeer thread.
- Sources. Article as archived in PMC (PMC12266187); Europe PMC full-text XML and supplementary files; Crossref record for doi:10.1016/j.lanepe.2025.101317.
- As of 2026-10-23.
Suggested correction
In the abstract (Findings) and in the first sentence of the Results outcome paragraph, replace "63.7% (n = 707)" with the figure supported by the stratified data, which on the paper's counts is 36.8% (n = 408), after confirming it against the source dataset. In the Discussion, revisit the statement that 90-day mortality was higher than in the PROMISE development study, and the explanation offered for it, in light of the corrected rate. Two further points could be checked at the same time. One is the GRIm group sizes (511/598 in the text vs 510/599 in Table 1). The other is the predictive values in Table 2, which do not correspond to the 35.1% event rate of the 333-patient subset; giving the underlying 2×2 counts would let readers verify them.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12266187 (Lancet Reg Health Eur 2025, doi:10.1016/j.lanepe.2025.101317)
# Overall 90-day mortality "63.7% (n = 707)" versus the paper's own stratified counts.
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib.
#
# Inputs: every count and percentage below is copied from the article text or Table 1
# as archived in PubMed Central (PMC12266187). When run inside the /home/ubuntu/papers
# repo, the script re-checks each quoted passage against the archived full text with
# src/show.py (skipped elsewhere). Nothing is estimated or read off an image.
import os, subprocess
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ quotes
QUOTES = [
("Abstract, Findings",
"PROMISE and GRIm scores were calculated for all 1109 patients, while CTI score was "
"assessed in 333 patients with complete laboratory data. The 90-day mortality rate was "
"63.7% (n = 707)."),
("Results, overall outcome paragraph",
"The overall 90-day mortality rate was 63.7% (n = 707). For the PROMISE score, 271 patients "
"(97.5%) in the low-risk group survived, with a 90-day mortality rate of 2.5% (n = 7). In the "
"intermediate-risk group, 176 patients (81.9%) survived, and the 90-day mortality rate was "
"18.1% (n = 39). In the high-risk group, 254 patients (41.2%) survived, with a 90-day "
"mortality rate of 58.8% (n = 362) (p < 0.0001). According to the PROMISE score, the median "
"overall survival (OS) for the entire cohort was 4.78 months (95% CI: 4.29–5.28). For the GRIm "
"score, the 90-day survival rate was 87.8% (n = 449) for low-risk patients and 42.2% (n = 252) "
"for high-risk patients (p < 0.0001). The estimated median OS for the entire cohort was 4.63 "
"months (95% CI: 4.14–5.12)."),
("Results, patient characteristics paragraph",
"According to the PROMISE score, 278 patients (25.1%) were categorized as low risk, 215 patients "
"(19.4%) as intermediate risk, and 616 patients (55.5%) as high risk. The GRIm score classified "
"511 patients (46.0%) as low risk and 598 patients (54.0%) as high risk."),
("Table 1, column headers",
"Low risk, n = 510 (46.0%) | High risk, n = 599 (54.0%)"),
("Results, Subgroup analysis by cancer type (lung)",
"For lung cancer patients (n = 283), mortality rates based on the PROMISE score were 2.4% "
"(n = 1) in the low-risk group, 3.6% (n = 2) in the intermediate-risk group, and 51.9% (n = 96) "
"in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (lung)",
"The GRIm score showed a 7.9% (n = 9) mortality rate in the low-risk group and 53.3% (n = 90) "
"in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (gastrointestinal)",
"For gastrointestinal cancer patients (n = 350), the PROMISE score demonstrated mortality rates "
"of 6.0% (n = 5) in the low-risk group, 23.7% (n = 18) in the intermediate-risk group, and 63.2% "
"(n = 120) in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (gastrointestinal)",
"The GRIm scores indicated a 13.6% (n = 20) mortality rate in the low-risk group compared to "
"60.6% (n = 123) in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (breast)",
"For breast cancer patients (n = 92), the PROMISE score revealed 90-day mortality rates of 2.8% "
"(n = 1) in the low-risk group, 21.4% (n = 3) in the intermediate-risk group, and 59.5% (n = 25) "
"in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (breast)",
"The GRIm score demonstrated mortality rates of 6.8% (n = 3) in the low-risk group compared to "
"54.2% (n = 26) in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (other cancers)",
"For patients with other cancers (n = 384), PROMISE score analysis showed no 90-day mortality "
"in the low-risk group (n = 0), a mortality rate of 23.2% (n = 16) in the intermediate-risk "
"group, and 60.8% (n = 121) in the high-risk group (p < 0.0001)."),
("Results, Subgroup analysis by cancer type (other cancers)",
"The GRIm scores revealed a 14.6% (n = 30) mortality rate in the low-risk group and 59.8% "
"(n = 107) in the high-risk group (p < 0.0001)."),
("Results, CTI subset paragraph",
"In this study, the 90-day mortality rate among 333 patients assessed using the CTI score was "
"35.1% (n = 117)."),
("Results, overall outcome paragraph (CTI)",
"Similarly, in CTI score assessments, low-risk patients demonstrated an 88.6% survival rate "
"(n = 147), while high-risk patients had a 41.3% survival rate (n = 69) (p < 0.0001)."),
("Results, patient characteristics paragraph (CTI)",
"Among the 333 patients evaluated using the CTI score, 166 patients (49.8%) were categorized as "
"low risk and 167 patients (50.2%) as high risk."),
("Discussion",
"According to the results of our study, the 90-day mortality rate was higher compared to the "
"study by Mirallas et al., in which the PROMISE score was developed."),
("Discussion",
"In the study by Mirallas et al., survival rates for low, intermediate, and high-risk groups "
"were reported as 83.6%, 68.0%, and 33.4%, respectively."),
]
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.\n")
return
try:
full = subprocess.run([py, "src/show.py", "PMC12266187", "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e:
print(f"Quote check: could not load archived text ({e}); skipped.\n")
return
norm = lambda s: " ".join(s.split())
full_n = norm(full)
print("Quote check against the PMC-archived full text (occurrences):")
for loc, q in QUOTES:
n = full_n.count(norm(q))
print(f" {n} {loc}: {q[:70]}...")
print()
# ------------------------------------------------------------------ data (from the paper)
N = 1109 # included patients (Abstract; Table 1)
STATED_DEATHS, STATED_PCT = 707, 63.7
# PROMISE strata: (group size, survivors, deaths, reported mortality %)
PROMISE = {"low": (278, 271, 7, 2.5), "intermediate": (215, 176, 39, 18.1),
"high": (616, 254, 362, 58.8)}
# GRIm strata: survivors and reported survival %; group sizes as given in text and in Table 1
GRIM_SURV = {"low": (449, 87.8), "high": (252, 42.2)}
GRIM_N_TEXT = {"low": 511, "high": 598}
GRIM_N_TABLE1 = {"low": 510, "high": 599}
# Cancer-type subgroups: n, PROMISE deaths (low, int, high), GRIm deaths (low, high),
# CTI subgroup n, CTI deaths (low, high)
CANCER = {
"Lung": (283, (1, 2, 96), (9, 90), 62, (3, 18)),
"Gastrointestinal": (350, (5, 18, 120), (20, 123), 95, (8, 32)),
"Breast": (92, (1, 3, 25), (3, 26), 54, (4, 16)),
"Other": (384, (0, 16, 121), (30, 107), 122, (4, 32)),
}
# CTI subset (333 patients)
CTI_N, CTI_DEATHS, CTI_PCT = 333, 117, 35.1
CTI_GROUPS = {"low": (166, 147), "high": (167, 69)} # (n, survivors)
CTI_PROMISE = {"low": (114, 111, 3), "intermediate": (64, 52, 12), "high": (155, 53, 102)}
COMBINED_DEATHS = (0, 1, 116)
# Median OS for the full cohort (months)
MEDIAN_OS = {"PROMISE analysis": 4.78, "GRIm analysis": 4.63}
# Table 2 (n = 333): sensitivity, specificity, PPV, NPV, accuracy (%)
TABLE2 = {"PROMISE Score": (88.7, 64.3, 78.0, 83.0, 79.9),
"CTI Score": (88.2, 67.5, 79.1, 81.4, 81.0),
"PROMISE-CTI Combined score": (92.4, 81.1, 85.3, 89.6, 86.7)}
# Development study survival by PROMISE stratum, as quoted in the Discussion
MIRALLAS_SURV = (83.6, 68.0, 33.4)
def pct(a, b):
return 100.0 * a / b
def main():
check_quotes()
print("1. Stated overall figure")
print(f" 707/1109 = {pct(707, N):.2f}% (reported 63.7%)")
print("\n2. PROMISE strata (cover all 1109 patients)")
nsum = sum(v[0] for v in PROMISE.values())
for k, (n, s, d, rp) in PROMISE.items():
print(f" {k:12s} n={n}: survivors+deaths={s + d}; mortality {pct(d, n):.1f}% "
f"(reported {rp}%); survival {pct(s, n):.1f}%")
P_D = sum(v[2] for v in PROMISE.values())
P_S = sum(v[1] for v in PROMISE.values())
print(f" group sizes sum to {nsum}")
print(f" deaths 7+39+362 = {P_D}; survivors 271+176+254 = {P_S}; total {P_D + P_S}")
print(f" mortality {P_D}/{N} = {pct(P_D, N):.2f}%; survival {P_S}/{N} = {pct(P_S, N):.2f}%")
print("\n3. GRIm strata (deaths = group size - survivors)")
G_S = sum(v[0] for v in GRIM_SURV.values())
for label, ns in (("text", GRIM_N_TEXT), ("Table 1", GRIM_N_TABLE1)):
dl, dh = ns["low"] - GRIM_SURV["low"][0], ns["high"] - GRIM_SURV["high"][0]
print(f" n from {label} {ns['low']}/{ns['high']}: survival "
f"{pct(449, ns['low']):.1f}%/{pct(252, ns['high']):.1f}% (reported 87.8%/42.2%); "
f"deaths {dl}+{dh} = {dl + dh}")
print(f" survivors 449+252 = {G_S}; deaths 1109-{G_S} = {N - G_S}")
GT_D = (GRIM_N_TABLE1["low"] - 449, GRIM_N_TABLE1["high"] - 252)
print("\n4. Cancer-type subgroups")
print(f" n: 283+350+92+384 = {sum(v[0] for v in CANCER.values())}")
for k, (n, pd, gd, cn, cd) in CANCER.items():
print(f" {k:16s} n={n}: PROMISE deaths {'+'.join(map(str, pd))} = {sum(pd)}; "
f"GRIm deaths {'+'.join(map(str, gd))} = {sum(gd)}; CTI deaths "
f"{'+'.join(map(str, cd))} = {sum(cd)} of {cn}")
CP = [sum(v[1]) for v in CANCER.values()]
CG = [sum(v[2]) for v in CANCER.values()]
print(f" PROMISE total {'+'.join(map(str, CP))} = {sum(CP)}; "
f"GRIm total {'+'.join(map(str, CG))} = {sum(CG)}")
print(f" CTI subgroup n sum = {sum(v[3] for v in CANCER.values())}; CTI deaths sum = "
f"{sum(sum(v[4]) for v in CANCER.values())}")
print("\n5. CTI subset (333 patients) - internally consistent")
cd = [n - s for n, s in CTI_GROUPS.values()]
print(f" 117/333 = {pct(117, 333):.2f}% (reported 35.1%)")
print(f" CTI groups: (166-147)+(167-69) = {cd[0]}+{cd[1]} = {sum(cd)}")
print(f" PROMISE within subset: 3+12+102 = {sum(v[2] for v in CTI_PROMISE.values())}; "
f"n 114+64+155 = {sum(v[0] for v in CTI_PROMISE.values())}")
print(f" Combined score: 0+1+116 = {sum(COMBINED_DEATHS)}")
print("\n6. Median OS versus 90-day mortality")
m90 = 90 / 30.4375
for k, v in MEDIAN_OS.items():
print(f" {k}: median OS {v} months > 90 days ({m90:.2f} months): {v > m90}")
print(" A Kaplan-Meier median above 90 days means S(90 d) > 0.5, i.e. 90-day mortality < 50%.")
print(" The KM estimate of mortality by day 90 is never below (deaths observed by day 90)/N,")
print(f" so 707 deaths would force KM mortality >= {pct(707, N):.1f}% and a median <= 90 days.")
print("\n7. Gap between stated and recomputed")
print(f" 707 - 408 = {707 - 408} deaths; {pct(707, N):.2f} - {pct(408, N):.2f} = "
f"{pct(707, N) - pct(408, N):.1f} percentage points")
print(f" survivors 701 ({pct(701, N):.1f}%) vs stated 707 ({pct(707, N):.1f}%): differ by 6")
print(f" 1109 - 707 = {N - 707}")
print("\n8. Discussion comparison with the development study (stratum survival, %)")
here = [round(pct(v[1], v[0]), 1) for v in PROMISE.values()]
for g, a, b in zip(("low", "intermediate", "high"), here, MIRALLAS_SURV):
print(f" {g:12s}: this cohort {a}% vs development study {b}%")
print("\n9. Table 2 (n = 333): prevalence implied by each reported value")
p = CTI_DEATHS / CTI_N
for k, (se, sp, ppv, npv, acc) in TABLE2.items():
se, sp, ppv, npv, acc = se / 100, sp / 100, ppv / 100, npv / 100, acc / 100
p_ppv = (1 - sp) * ppv / (se * (1 - ppv) + (1 - sp) * ppv)
p_npv = (1 - npv) * sp / ((1 - npv) * sp + npv * (1 - se))
p_acc = (acc - sp) / (se - sp)
e_ppv = se * p / (se * p + (1 - sp) * (1 - p))
e_npv = sp * (1 - p) / (sp * (1 - p) + (1 - se) * p)
e_acc = se * p + sp * (1 - p)
print(f" {k}: implied prevalence from PPV {100*p_ppv:.1f}%, NPV {100*p_npv:.1f}%, "
f"accuracy {100*p_acc:.1f}%")
print(f" expected at 35.1% prevalence: PPV {100*e_ppv:.1f}, NPV {100*e_npv:.1f}, "
f"accuracy {100*e_acc:.1f} (reported {ppv*100:.1f}, {npv*100:.1f}, {acc*100:.1f})")
make_figures(P_D, GT_D, CP, CG)
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
GRID = "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": "#b9b8b3", "svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, bbox_inches="tight", **kw)
plt.close(fig)
print(f" wrote {name}.svg, {name}.png")
def make_figures(P_D, GT_D, CP, CG):
print("\nFigures")
# ---- fig1: death counts, stated vs sums of the paper's breakdowns
rows = [
("Stated overall deaths\n(Abstract; Results)", [STATED_DEATHS], ORANGE, ["707"]),
("PROMISE risk groups\n7 + 39 + 362 (low, int., high)",
[v[2] for v in PROMISE.values()], BLUE, ["7", "39", "362"]),
("GRIm risk groups, n − survivors\n(510 − 449) + (599 − 252)", list(GT_D), BLUE,
[str(x) for x in GT_D]),
("Cancer types, PROMISE deaths\n99 + 143 + 29 + 137", CP, BLUE,
[str(x) for x in CP]),
("Cancer types, GRIm deaths\n99 + 143 + 29 + 137", CG, BLUE,
[str(x) for x in CG]),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
shades = ["#2a78d6", "#5d9ae3", "#8fbbee", "#bcd6f5"]
for i, (lab, segs, col, txt) in enumerate(rows):
y = len(rows) - 1 - i
left = 0
for j, (s, t) in enumerate(zip(segs, txt)):
c = col if col == ORANGE else shades[j % 4]
ax.barh(y, s, left=left, height=0.56, color=c, edgecolor=BG, linewidth=2)
if s >= 60:
ax.text(left + s / 2, y, t, ha="center", va="center", fontsize=10,
color="white" if (col == ORANGE or j == 0) else INK)
left += s
tot = sum(segs)
ax.text(tot + 12, y, f"= {tot} ({pct(tot, N):.1f}% of {N})" if len(segs) > 1
else f"{tot} ({STATED_PCT}%)", va="center", fontsize=10.5,
color=INK, fontweight="bold")
ax.axvline(N, color=INK2, lw=1, ls=(0, (3, 3)))
ax.text(N, len(rows) - 0.35, f"all {N} patients", ha="right", va="bottom",
fontsize=10, color=INK2)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10)
ax.set_xlim(0, 1180)
ax.set_ylim(-0.6, len(rows) - 0.1)
ax.set_xlabel("Patients who died within 90 days (count)")
ax.xaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.patches import Patch
ax.legend(handles=[Patch(color=ORANGE, label="Reported in the paper"),
Patch(color=BLUE, label="Recomputed: sum of the paper's own subgroup counts")],
loc="upper center", bbox_to_anchor=(0.38, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# ---- fig2: percentages on one 0-100 axis with the median-OS bound
fig, ax = plt.subplots(figsize=(7.2, 2.9), facecolor=BG)
ax.set_facecolor(BG)
ax.axvspan(50, 100, color="#efeeea", zorder=0)
ax.axvline(50, color=INK2, lw=1, zorder=1)
ax.text(51.5, 0.25, "Ruled out by median OS 4.78 months (> 90 days):\n"
"90-day mortality must be below 50%", fontsize=10, color=INK2, va="center")
pts = [
(3, "Overall 90-day mortality,\nas printed (n = 707)", [(STATED_PCT, ORANGE, "o", "63.7%")]),
(2, "Overall 90-day mortality from\nthe paper's strata (408/1109)",
[(pct(408, N), BLUE, "o", f"{pct(408, N):.1f}%")]),
(1, "Share surviving 90 days from\nthe paper's strata (701/1109)",
[(pct(701, N), BLUE, "D", f"{pct(701, N):.1f}%")]),
(0, "CTI subset 90-day mortality\n(117/333): reported and recomputed",
[(CTI_PCT, ORANGE, "o", None), (pct(117, 333), BLUE, "o", f"{CTI_PCT}% (both)")]),
]
for y, lab, marks in pts:
for k, (x, c, m, t) in enumerate(marks):
ms = 14 if k == 0 else 8
ax.plot([x], [y], marker=m, ms=ms, color=c, mec=BG, mew=1.5, ls="none", zorder=3)
if t:
ax.text(x - 2.5, y + 0.02, t, ha="right", va="center", fontsize=10.5,
color=INK, fontweight="bold")
ax.set_yticks([p[0] for p in pts])
ax.set_yticklabels([p[1] for p in pts], fontsize=10)
ax.set_xlim(0, 100)
ax.set_ylim(-0.6, 3.7)
ax.set_xlabel("Percent of patients")
ax.xaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
ax.legend(handles=[Line2D([], [], marker="o", ls="none", ms=9, color=ORANGE,
label="Reported in the paper"),
Line2D([], [], marker="o", ls="none", ms=9, color=BLUE,
label="Recomputed from the paper's counts")],
loc="upper center", bbox_to_anchor=(0.3, -0.25), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
if __name__ == "__main__":
main()
Abstract reports univariate hazard ratios as adjusted estimates and attaches them to the wrong treatments
Lancet Reg Health Eur (2025) · doi:10.1016/j.lanepe.2025.101402 · PMC12859599
Summary
The Abstract states that "in the adjusted multivariate model" immunotherapy and targeted therapy had hazard ratios (HRs) for second primary cancer (SPC), vs chemotherapy, of 0.470 (95% CI 0.338–0.654) and 0.701 (0.574–0.855). In the Results these same two HR/CI pairs are reported as the univariate Cox estimates, with the treatments the other way round: 0.470 for targeted therapy and 0.701 for immunotherapy. The adjusted multivariate estimates in the Results, repeated in the Discussion, are 0.768 (0.622–0.947) for immunotherapy and 0.556 (0.385–0.803) for targeted therapy. The paper's own event counts and incidence rates support the Results' assignment: the width of each interval matches the number of events in the group the Results name, and the crude rate ratios rank targeted therapy as the more protective treatment. As a result, the Abstract overstates the adjusted risk reduction for immunotherapy (53% vs 23%), understates it for targeted therapy (30% vs 44%), and reverses which treatment looks more protective after adjustment. Both treatments remain associated with lower SPC risk in every version.
What the paper reports
In the adjusted multivariate model, both immunotherapy and targeted therapy were associated with a reduced risk of SPCs: HR = 0.470 (95% CI 0.338–0.654) and HR = 0.701 (95% CI 0.574–0.855), respectively. Additional associated factors with SPCs included metastatic disease, HR = 0.456 (95% CI 0.377–0.552); history of tobacco use, HR = 1.599 (95% CI 1.127–2.269); and current smoking, HR = 1.319 (95% CI 0.923–1.884) p < 0.001.
Abstract, Findings
In the univariate Cox regression, the following factors were explored for their association with the development of SPCs: type of treatment received, age at diagnosis, sex, tumour stage, histological type, and smoking habits were explored. Both targeted therapy and immunotherapy, when compared to chemotherapy, were identified as protective factors with HR = 0.470 (95% CI 0.338–0.654) and HR = 0.701 (95% CI 0.574–0.855), respectively.
Results, "Factors associated with second primary cancer", paragraph 1
Other factors associated with the development of SPCs included metastatic disease HR = 0.456 (95% CI 0.377–0.552), history of tobacco use HR = 1.599 (95% CI 1.127–2.269), and active tobacco use HR = 1.319 (95% CI 0.923–1.884); p < 0.001.
Results, "Factors associated with second primary cancer", paragraph 3 (between the univariate paragraph and the adjusted-model paragraph)
In the adjusted multivariate model, the effect of treatment was confirmed with immunotherapy and targeted therapy, HR = 0.768 (95% CI 0.622–0.947) and HR = 0.556 (95% CI 0.385–0.803); p < 0.001, respectively. Other associated factors such as age HR = 1.028 (95% CI 1.018–1.039), small cell histology HR = 0.704 (95% CI 0.513–0.965), and metastatic disease HR = 0.518 (95% CI 0.428–0.626); p < 0.001, were also found.
Results, "Factors associated with second primary cancer", paragraph 4
Multivariate Cox regression analysis confirmed that both immunotherapy and targeted therapy were protective for SPC compared to chemotherapy with HR = 0.768 (95% CI 0.622–0.947) and HR = 0.556 (95% CI 0.385–0.803); p < 0.001, respectively.
Discussion, paragraph 1
The inputs used for the checks below:
A total of 480 patients (2.3%) developed a SPC. The incidence rate per 100 person-years were 1.059 for the immunotherapy group, 0.661 for the targeted therapy group, and 1.791 for the chemotherapy group. […] By treatment group, 298 patients (62.1%) had received chemotherapy, 142 (29.6%) immunotherapy, and 40 (8.3%) targeted therapy. The proportion of patients who developed an SPC was lowest in the targeted therapy group (1.5%) followed by the immunotherapy group (2.1%) and the chemotherapy group (2.9%) (Table 3).
Results, "Characteristics of the population with a second primary cancer by treatment type", paragraph 1
How the Abstract uses the result:
Patients treated with immunotherapy and targeted therapy exhibited a significantly lower incidence of SPCs. Tobacco use was a key associated factor.
Abstract, Interpretation
What the paper's own numbers show
Method. The Abstract, Results and Discussion values were compared as printed. Two checks show which treatment each HR/CI pair belongs to. (1) The standard error of a log HR implied by a printed 95% CI is ln(upper/lower)/(2 × 1.96). For a Cox HR comparing two groups, this SE is approximately √(1/d₁ + 1/d₀), where d₁ and d₀ are the event counts in the two groups (142 immunotherapy, 40 targeted therapy and 298 chemotherapy events; Results and Table 3). The approximation is close for an unadjusted comparison and somewhat larger after adjustment. (2) The crude incidence-rate ratio of each group vs chemotherapy was computed from the reported rates per 100 person-years. Each CI was also checked for internal consistency: its log-scale midpoint should equal the printed HR. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Immunotherapy vs chemotherapy, "adjusted multivariate" | Abstract, Findings | 0.470 (0.338–0.654) | Results and Discussion give the adjusted HR as 0.768 (0.622–0.947). 0.470 (0.338–0.654) is the Results' univariate value for targeted therapy |
| Targeted therapy vs chemotherapy, "adjusted multivariate" | Abstract, Findings | 0.701 (0.574–0.855) | Results and Discussion give the adjusted HR as 0.556 (0.385–0.803). 0.701 (0.574–0.855) is the Results' univariate value for immunotherapy |
| SE(log HR) implied by 0.470 (0.338–0.654) | Abstract; Results ¶1 | — | 0.1684. Expected from events: targeted therapy (40 vs 298) 0.1684, immunotherapy (142 vs 298) 0.1020. Ratio implied/expected 1.000 for targeted therapy, 1.651 for immunotherapy |
| SE(log HR) implied by 0.701 (0.574–0.855) | Abstract; Results ¶1 | — | 0.1017. Matches immunotherapy (ratio 0.997), not targeted therapy (0.604) |
| SE(log HR) implied by adjusted 0.768 (0.622–0.947) | Results ¶4; Discussion | immunotherapy | 0.1072, 1.052 × the immunotherapy event-based value. Consistent |
| SE(log HR) implied by adjusted 0.556 (0.385–0.803) | Results ¶4; Discussion | targeted therapy | 0.1875, 1.114 × the targeted-therapy event-based value. Consistent |
| Log-scale midpoint of each CI | all four pairs | 0.470, 0.701, 0.768, 0.556 | 0.470, 0.701, 0.767, 0.556. Each pair is internally consistent, so no digit has been garbled |
| Crude SPC rate ratio vs chemotherapy | from rates per 100 person-years (1.059, 0.661, 1.791) | — | Immunotherapy 0.591 (0.484–0.722), targeted therapy 0.369 (0.265–0.513). Targeted therapy is the more protective group, as in the Results' univariate assignment |
| 12-month cumulative incidence ratio vs chemotherapy | Abstract; Results (1.22%, 0.85%, 0.67%) | — | Immunotherapy 0.697, targeted therapy 0.549. Same ordering |
| Hazard reduction, immunotherapy (100 × (1 − HR)) | Abstract vs Results ¶4 | 53.0% (abstract) | 23.2% (adjusted model). The abstract's figure is 2.28 times the adjusted one |
| Hazard reduction, targeted therapy | Abstract vs Results ¶4 | 29.9% (abstract) | 44.4% (adjusted model) |
| Which treatment is more protective after adjustment | Abstract vs Results ¶4 | immunotherapy (0.470 < 0.701) | targeted therapy (0.556 < 0.768) |
| Metastatic disease, listed in the abstract after the "adjusted" treatment HRs | Abstract vs Results ¶3, ¶4 | 0.456 (0.377–0.552) | 0.456 is the Results ¶3 value (between the univariate and adjusted paragraphs). The adjusted-model value is 0.518 (0.428–0.626) |
| Current smoking, "p < 0.001" | Abstract; Results ¶3 | 1.319 (0.923–1.884) | The CI includes 1. The Wald p implied by the CI is 0.13 (z = 1.52), so the "p < 0.001" cannot refer to this term on its own |
| Wald p implied by the adjusted CIs | Results ¶4; Discussion ("p < 0.001") | — | Immunotherapy z = −2.46, p = 0.014; targeted therapy z = −3.13, p = 0.0017. These are significant at 0.05, but neither is below 0.001; the printed p may refer to a joint test of the treatment variable |
| Crude SPC proportions | Abstract; Results; Table 3 | 2.9%, 2.1%, 1.5% | 298/10,137 = 2.94%, 142/6,820 = 2.08%, 40/2,628 = 1.52% with the followed-up denominators of Supplementary Figure S1 (sum 19,585). These are consistent and not part of the error |
So the Abstract's two "adjusted" treatment HRs are the univariate pair, with the treatment labels swapped. The event-count check fits the Results' labelling exactly (ratios 0.997 and 1.000) and misfits the Abstract's labelling by factors of 1.65 and 0.60. The crude rate ratios point the same way. The adjusted estimates in the body are consistent with the event counts and with the expected modest SE inflation after covariate adjustment.
Figure
Figure 1. Where each treatment HR appears in the paper and how it is labelled. Orange text gives the values as printed. The left panel is the Abstract's "adjusted multivariate model". The upper right panel is the Results' univariate Cox paragraph, which carries the same two HR/CI pairs with the treatment labels reversed. The lower right panel is the Results' adjusted multivariate paragraph, repeated in the Discussion, which carries different values under the same model label as the Abstract. The blue text below summarises the checks recomputed from the paper's own event counts and incidence rates (see recompute.py.txt). Both checks agree with the treatment labels in the Results.
Figure 2. For each of the four printed treatment HRs, the orange diamond is the standard error of the log HR implied by its 95% CI. The blue vertical lines are the approximate standard errors expected from the event counts, √(1/d₁ + 1/d₀): 0.102 for immunotherapy vs chemotherapy (142 vs 298 events, solid) and 0.168 for targeted therapy vs chemotherapy (40 vs 298 events, dashed). The interval of HR 0.470, which the Abstract assigns to immunotherapy, sits exactly on the targeted-therapy line. The interval of HR 0.701, which the Abstract assigns to targeted therapy, sits on the immunotherapy line. The two adjusted intervals from the Results lie just to the right of their own group's line, as expected after covariate adjustment. The axis starts at zero.
Why it matters
- The headline adjusted estimates. The Abstract is where readers, guideline authors and evidence syntheses usually take effect sizes from. Readers who take the Abstract's adjusted immunotherapy HR (0.470) would record a 53% lower SPC hazard, whereas the paper's adjusted model gives 23% (HR 0.768). For targeted therapy they would record 30% (HR 0.701) instead of 44% (HR 0.556).
- The ranking of treatments. After adjustment, the Abstract makes immunotherapy look more protective than targeted therapy. The body's adjusted model, the univariate model and the crude data (1.5% vs 2.1%; 0.661 vs 1.059 per 100 person-years) all rank targeted therapy first.
- Unadjusted shown as adjusted. The Discussion itself notes that the treatment associations "are likely influenced by the underlying clinical profiles of patients receiving each treatment". The Abstract presents unadjusted estimates as adjusted ones, so readers cannot see how much adjustment changed them. For immunotherapy the hazard reduction shrinks from 30% (univariate 0.701) to 23% (adjusted 0.768).
- Other factors in the Abstract. The Abstract's "additional associated factors" are also the values from Results ¶3, not the adjusted model (adjusted metastatic disease HR 0.518 vs 0.456 in the Abstract). The adjusted-model paragraph reports no tobacco HRs. Current smoking, given with "p < 0.001", has a CI that includes 1 (implied p = 0.13). The Interpretation ("Tobacco use was a key associated factor") rests on these values.
- What stands. All four treatment CIs exclude 1, so the qualitative conclusion holds in every version: both immunotherapy and targeted therapy are associated with lower SPC risk than chemotherapy. The crude incidence figures, cumulative incidence curves and SPC site distribution are not affected.
Explanations considered
- The abstract reports a different model (Fine–Gray rather than Cox). Does not account for it. The Methods describe "Univariate and multivariate Fine and Gray sub distribution hazard models … with sub hazard ratios (sHR)" as well as Cox models, but the paper reports no sHR for any covariate. In the body, 0.470 and 0.701 are called univariate Cox HRs, and the only values called adjusted multivariate are 0.768 and 0.556, both in the Results and in the Discussion. No passage gives 0.470 or 0.701 as an adjusted estimate from any model. If the Abstract came from another adjusted model, it would also be unlikely to reproduce the univariate CIs to three decimals.
- A subgroup analysis. Does not account for it. The within-subgroup multivariate HRs in Results ¶2 are 2.117 (0.989–4.534), 2.171 (1.160–4.063) and 1.411 (0.717–2.777), which are different numbers.
- The Results, not the Abstract, has the treatments swapped. Not supported. The CI widths match the event counts of the groups the Results names (implied/expected SE ratio 1.000 for 0.470 as targeted therapy, 0.997 for 0.701 as immunotherapy), and not the Abstract's assignment (1.65 and 0.60). The crude rate ratios (targeted 0.369, immunotherapy 0.591) and the 12-month cumulative incidences (0.67% vs 0.85%) also rank targeted therapy as more protective.
- A typo or digit transposition. Does not account for it. Each HR sits at the log-scale midpoint of its CI, so every pair is internally consistent. Four values (both HRs and both CIs) move together.
- A conversion or rendering artefact. Excluded. The values are in running text, not table cells. The publisher's typeset PDF carries the same wording and numbers (see Status).
- A supplementary table that reports 0.470/0.701 as adjusted. Not found. Supplementary Table S1 is a Cox model for overall survival, not for SPCs. Its treatment HRs (targeted 0.696/0.598, immunotherapy 0.719/0.564) match none of the disputed values.
Status
- Version of record. Checked in the publisher's typeset PDF (creator Elsevier, "The Lancet Regional Health - Europe, 56 (2025) 101402", 20 pages), obtained through the PMC /pdf/ endpoint. The Abstract, univariate-paragraph and adjusted-paragraph passages appear word for word. In the PDF text, 0.470 occurs only in the Abstract and in the univariate paragraph. The Europe PMC full-text XML, the live PMC page and the local PMC archive copy carry the same text. Every quotation above was checked verbatim against the archive copy. The ScienceDirect and thelancet.com pages refused automated access (HTTP 403 or a challenge page).
- Supplements. Both supplementary files were checked (Europe PMC supplementary files): mmc1.docx (a list of group members) and mmc2.pdf (5 pages: Figure S1 CONSORT diagram, Figure S2 overall survival curves, Table S1 Cox model for overall survival). Figure S1 confirms the SPC counts (immunotherapy 142/6,820, targeted therapy 40/2,628, chemotherapy 298/10,137). No supplement contains a Cox or Fine–Gray table for SPCs, so the treatment HRs appear only in the running text.
- Corrections. Crossref lists no update-to or updated-by relation for the DOI. PMC and Europe PMC list no correction, erratum, retraction or expression of concern (PMID 41624086; checked 2026-10-23).
- PubPeer. pubpeer.com refused automated access (HTTP 403). A web search for the title with "PubPeer" returned no PubPeer page. Europe PMC's link services for PMID 41624086 list only BioStudies, with no PubPeer link. The absence of a PubPeer thread is therefore likely but not confirmed.
- As of 2026-10-23.
Suggested correction
- In the Abstract, Findings, replace the treatment sentence with the adjusted estimates the Results and Discussion report: immunotherapy HR 0.768 (95% CI 0.622–0.947) and targeted therapy HR 0.556 (95% CI 0.385–0.803), vs chemotherapy. If the univariate values are kept as well, label them as univariate and attach them to the right treatments: targeted therapy 0.470 (0.338–0.654), immunotherapy 0.701 (0.574–0.855).
- Make the Abstract's "additional associated factors" match the model it describes. For the adjusted model, give metastatic disease 0.518 (0.428–0.626), age 1.028 (1.018–1.039) and small cell histology 0.704 (0.513–0.965). Report the adjusted tobacco HRs if the model included smoking, as the Methods list it among the covariates.
- Give a p value for each term, or say which test "p < 0.001" refers to. As printed, it is attached to current smoking, whose CI includes 1, and to two adjusted treatment HRs whose CIs imply p = 0.014 and p = 0.0017.
- Consider adding a table of the univariate and multivariate SPC models (Cox and, as the Methods describe, Fine–Gray) so that each estimate's model and covariate set is explicit. If needed, revise the Interpretation sentence on tobacco use to match the adjusted results.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC12859599 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
All inputs are copied from the article as archived in PubMed Central (Abstract,
Results "Characteristics of the population with a second primary cancer" and
"Factors associated with second primary cancer", Discussion) and from
Supplementary Figure S1 (CONSORT diagram, mmc2.pdf).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs ----
EVENTS = {"chemo": 298, "IT": 142, "TT": 40} # Results; Table 3
N_TREATED = {"chemo": 10650, "IT": 7182, "TT": 2742} # Results, treatment groups
N_FOLLOWED = {"chemo": 10137, "IT": 6820, "TT": 2628} # Supplementary Figure S1
RATE = {"chemo": 1.791, "IT": 1.059, "TT": 0.661} # SPC per 100 person-years
CIF12 = {"chemo": 1.22, "IT": 0.85, "TT": 0.67} # 12-month cumulative incidence, %
# (HR, lower, upper) exactly as printed
ABSTRACT_ADJ = {"IT": (0.470, 0.338, 0.654), "TT": (0.701, 0.574, 0.855)}
RESULTS_UNIV = {"TT": (0.470, 0.338, 0.654), "IT": (0.701, 0.574, 0.855)}
RESULTS_ADJ = {"IT": (0.768, 0.622, 0.947), "TT": (0.556, 0.385, 0.803)} # also Discussion
OTHER = {
"metastatic, abstract / Results p2": (0.456, 0.377, 0.552),
"metastatic, Results adjusted": (0.518, 0.428, 0.626),
"history of tobacco use": (1.599, 1.127, 2.269),
"current (active) smoking": (1.319, 0.923, 1.884),
}
NAME = {"IT": "immunotherapy", "TT": "targeted therapy", "chemo": "chemotherapy"}
def ci_stats(h, lo, hi):
se = np.log(hi / lo) / (2 * Z)
mid = np.exp((np.log(lo) + np.log(hi)) / 2)
z = np.log(h) / se
p = 2 * stats.norm.sf(abs(z))
return se, mid, z, p
def line(s=""):
print(s)
# ------------------------------------------------------- 1. crude data ----
line("1. Crude SPC proportions")
line(f" total: {sum(EVENTS.values())}/{sum(N_TREATED.values())} = "
f"{100*sum(EVENTS.values())/sum(N_TREATED.values()):.2f}% "
f"(followed-up denominators sum to {sum(N_FOLLOWED.values())})")
for k in ["chemo", "IT", "TT"]:
line(f" {NAME[k]:17s} {EVENTS[k]}/{N_TREATED[k]} = {100*EVENTS[k]/N_TREATED[k]:.2f}% "
f"{EVENTS[k]}/{N_FOLLOWED[k]} (Fig S1) = {100*EVENTS[k]/N_FOLLOWED[k]:.2f}%")
line("\n2. Crude incidence-rate ratios vs chemotherapy (from the reported rates)")
PY = {k: 100 * EVENTS[k] / RATE[k] for k in EVENTS}
for k in ["chemo", "IT", "TT"]:
line(f" person-years implied, {NAME[k]}: {PY[k]:.0f}")
CRUDE_IRR = {}
for k in ["IT", "TT"]:
irr = RATE[k] / RATE["chemo"]
se = np.sqrt(1 / EVENTS[k] + 1 / EVENTS["chemo"])
CRUDE_IRR[k] = (irr, irr * np.exp(-Z * se), irr * np.exp(Z * se))
line(f" {NAME[k]:17s} IRR = {irr:.3f} (95% CI {CRUDE_IRR[k][1]:.3f}-{CRUDE_IRR[k][2]:.3f})")
line(f" 12-month cumulative incidence ratio vs chemo: IT {CIF12['IT']/CIF12['chemo']:.3f}, "
f"TT {CIF12['TT']/CIF12['chemo']:.3f}")
# ------------------------------------------- 3. SE expected from events ----
line("\n3. SE(log HR) expected from event counts, sqrt(1/d1 + 1/d0)")
SE_EXP = {k: np.sqrt(1 / EVENTS[k] + 1 / EVENTS["chemo"]) for k in ["IT", "TT"]}
for k in SE_EXP:
line(f" {NAME[k]:17s} ({EVENTS[k]} vs {EVENTS['chemo']} events): {SE_EXP[k]:.4f}")
# ------------------------------------------- 4. SE implied by each CI ----
line("\n4. Each reported HR: SE implied by its CI, log-midpoint, Wald z and p, nearest group")
ROWS = [
("0.470 (0.338-0.654)", (0.470, 0.338, 0.654), "Abstract: IT 'adjusted' | Results: TT univariate"),
("0.701 (0.574-0.855)", (0.701, 0.574, 0.855), "Abstract: TT 'adjusted' | Results: IT univariate"),
("0.768 (0.622-0.947)", (0.768, 0.622, 0.947), "Results/Discussion: IT adjusted"),
("0.556 (0.385-0.803)", (0.556, 0.385, 0.803), "Results/Discussion: TT adjusted"),
]
IMPLIED = {}
for lab, (h, lo, hi), where in ROWS:
se, mid, z, p = ci_stats(h, lo, hi)
near = min(SE_EXP, key=lambda k: abs(SE_EXP[k] - se))
IMPLIED[lab] = se
line(f" {lab}: SE = {se:.4f}, log-midpoint = {mid:.3f}, z = {z:.2f}, p = {p:.2g}, "
f"nearest event-based SE: {NAME[near]} [{where}]")
for k in SE_EXP:
line(f" ratio implied/expected: univariate {NAME[k]} "
f"{ci_stats(*RESULTS_UNIV[k])[0]/SE_EXP[k]:.3f}; adjusted {ci_stats(*RESULTS_ADJ[k])[0]/SE_EXP[k]:.3f}; "
f"abstract '{NAME[k]}' {ci_stats(*ABSTRACT_ADJ[k])[0]/SE_EXP[k]:.3f}")
line("\n5. Other factors (Wald p from the printed CI)")
for lab, v in OTHER.items():
se, mid, z, p = ci_stats(*v)
line(f" {lab:36s} HR {v[0]:.3f} ({v[1]:.3f}-{v[2]:.3f}): SE {se:.4f}, z {z:.2f}, p {p:.2g}")
# ------------------------------------------- 6. comparisons and sizes ----
line("\n6. Abstract vs Results")
line(f" abstract 'adjusted' == Results adjusted? {ABSTRACT_ADJ == RESULTS_ADJ}")
line(f" abstract values == Results univariate values (set)? "
f"{set(ABSTRACT_ADJ.values()) == set(RESULTS_UNIV.values())}")
line(f" abstract assignment == Results univariate assignment? {ABSTRACT_ADJ == RESULTS_UNIV}")
for k in ["IT", "TT"]:
a, r = ABSTRACT_ADJ[k][0], RESULTS_ADJ[k][0]
line(f" {NAME[k]:17s} hazard reduction: abstract {100*(1-a):.1f}% vs adjusted {100*(1-r):.1f}% "
f"(difference {100*abs(a-r):.1f} points; ratio of reductions {(1-a)/(1-r):.2f})")
line(f" ranking after adjustment: abstract says {'IT' if ABSTRACT_ADJ['IT'][0] < ABSTRACT_ADJ['TT'][0] else 'TT'} "
f"more protective; Results adjusted says {'IT' if RESULTS_ADJ['IT'][0] < RESULTS_ADJ['TT'][0] else 'TT'}")
line(" all four CIs exclude 1: "
f"{all(v[2] < 1 for d in (ABSTRACT_ADJ, RESULTS_ADJ) for v in d.values())}")
# --------------------------------------------------------------- figures ----
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: text-panel diagram
fig = plt.figure(figsize=(7.2, 6.4), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 92); ax.axis("off")
def panel(x, y, w, h, title, sub, rows):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec="#bdbcb7", lw=1))
ax.text(x + 1.5, y + h - 3, title, fontsize=11, weight="bold", va="center")
ax.text(x + 1.2, y + h - 6.6, sub, fontsize=10, color=INK2, va="center", style="italic")
for i, (lab, val) in enumerate(rows):
yy = y + h - 11.5 - i * 8
ax.text(x + 1.5, yy, lab, fontsize=10.5, va="center")
ax.text(x + 3.5, yy - 3.5, val, fontsize=10.5, va="center", color=ORANGE, weight="bold")
PW = 37
panel(2, 41, PW, 26, "Abstract, Findings", '"adjusted multivariate model"',
[("Immunotherapy", "HR 0.470 (0.338–0.654)"), ("Targeted therapy", "HR 0.701 (0.574–0.855)")])
panel(61, 60, PW, 26, "Results, univariate", '"univariate Cox regression"',
[("Targeted therapy", "HR 0.470 (0.338–0.654)"), ("Immunotherapy", "HR 0.701 (0.574–0.855)")])
panel(61, 26, PW, 26, "Results ¶4; Discussion", '"adjusted multivariate model"',
[("Immunotherapy", "HR 0.768 (0.622–0.947)"), ("Targeted therapy", "HR 0.556 (0.385–0.803)")])
for (y0, y1, txt, ty) in [(63.5, 63.5, "same numbers,\ncalled\nunivariate,\ntreatments\nswapped", 72.5),
(46.5, 46.5, "same model\nlabel,\ndifferent\nnumbers", 38.5)]:
ax.add_patch(FancyArrowPatch((39.9, y0), (60.1, y1), arrowstyle="<|-|>", mutation_scale=12,
lw=2, color=INK2, shrinkA=0, shrinkB=0))
ax.text(50, ty, txt, ha="center", va="center", fontsize=10, color=INK, weight="bold",
bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(2, 86, "Orange: values as printed in the paper.", fontsize=10, color=ORANGE, weight="bold")
ax.text(2, 82, "Blue: recomputed from the paper's own numbers.", fontsize=10, color=BLUE, weight="bold")
se470, se701 = IMPLIED["0.470 (0.338-0.654)"], IMPLIED["0.701 (0.574-0.855)"]
recomp = [
f"The CI of HR 0.470 implies SE(log HR) = {se470:.3f}. Event counts predict {SE_EXP['TT']:.3f} for targeted therapy",
f"(40 vs 298 events) and {SE_EXP['IT']:.3f} for immunotherapy (142 vs 298). The CI of HR 0.701 implies {se701:.3f}.",
f"Crude SPC rate ratios vs chemotherapy: targeted therapy {CRUDE_IRR['TT'][0]:.3f}, immunotherapy {CRUDE_IRR['IT'][0]:.3f}.",
"Both checks agree with the Results' treatment labels, not the Abstract's.",
]
for i, t in enumerate(recomp):
ax.text(2, 18 - i * 4.2, t, fontsize=10, color=BLUE, va="center")
save(fig, "fig1")
# Figure 2: SE implied by each reported CI vs SE expected from event counts
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
ax.set_facecolor(BG)
YLAB = ["HR 0.470 (0.338–0.654)\nAbstract: immunotherapy, 'adjusted'\nResults: targeted, univariate",
"HR 0.701 (0.574–0.855)\nAbstract: targeted, 'adjusted'\nResults: immunotherapy, univariate",
"HR 0.768 (0.622–0.947)\nResults, Discussion:\nimmunotherapy, adjusted",
"HR 0.556 (0.385–0.803)\nResults, Discussion:\ntargeted, adjusted"]
ys = np.arange(len(ROWS))[::-1]
top = len(ROWS) + 0.5
for k, ls, dx, ha in [("IT", "-", -0.004, "right"), ("TT", "--", 0.004, "left")]:
ax.axvline(SE_EXP[k], color=BLUE, lw=2, ls=ls, zorder=1)
ax.text(SE_EXP[k] + dx, top - 0.05,
f"expected from events,\n{NAME[k]}\n({EVENTS[k]} vs {EVENTS['chemo']}): {SE_EXP[k]:.3f}",
color=BLUE, fontsize=10, va="top", ha=ha)
ax.scatter([IMPLIED[r[0]] for r in ROWS], ys, marker="D", s=70, color=ORANGE, zorder=3,
label="SE(log HR) implied by the printed 95% CI (paper's values)")
for y, r in zip(ys, ROWS):
ax.text(IMPLIED[r[0]] + 0.006, y, f"{IMPLIED[r[0]]:.3f}", color=ORANGE, fontsize=10,
ha="left", va="center", weight="bold")
ax.plot([], [], color=BLUE, lw=2, label="SE expected from event counts, sqrt(1/d1 + 1/d0)")
ax.set_yticks(ys); ax.set_yticklabels(YLAB, fontsize=10)
ax.set_xlim(0, 0.26); ax.set_ylim(-0.5, top)
ax.set_xlabel("Standard error of log hazard ratio vs chemotherapy")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color("#bdbcb7"); ax.tick_params(axis="y", length=0)
fig.legend(loc="lower center", ncol=1, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.1, 1, 1))
save(fig, "fig2")
line("\nFigures written: fig1.svg/png, fig2.svg/png")
Abstract's headline mortality reductions match the paper's no-services scenario, not its maximum-services scenario
Nat Commun (2025) · doi:10.1038/s41467-025-59060-2 · PMC12032021
Summary
The abstract of this individual-based modelling study of maternity services in Malawi states that maximising the use and quality of all services together reduces the stillbirth, maternal and neonatal mortality rates by 10%, 52% and 57%. The paper's own Tables 2–4, Results and Discussion give 13.6%, 63.7% and 44.2% for that scenario (about 14%, 64% and 44%), and recomputing from the tabulated means gives 13.3%, 63.7% and 44.4%. The abstract's three numbers equal, after rounding, the percentage increases the tables report for the opposite scenario, in which all maternity services are removed (9.6%, 51.8% and 57.0%). As a result the abstract understates the maternal mortality reduction by about 12 points, overstates the neonatal reduction by about 13 points, and reverses which of the two outcomes falls most. The direction of the result and the paper's qualitative conclusions are unaffected.
What the paper reports
Abstract:
Whilst current service delivery is inferred to avert morbidity and mortality, the largest reductions in the stillbirth, maternal and neonatal mortality rates were observed when the use and quality of all services was maximised concurrently (a 10%, 52% and 57% reduction respectively).
Results, "Impact of maternity service delivery on mortality and DALYs", paragraph 1 (maternal mortality, All services max.):
In this scenario the average MMR was 64% lower than the SQ scenario at 120 (95% CI [114.8, 125.2]) maternal deaths per 100,000 live births.
Same section, paragraph 4 (stillbirth):
A 14% reduction in the average SBR during the intervention period was observed in the scenario All services max.
Same section, paragraph 5 (neonatal mortality):
Maximum availability of maternity services, as defined in Table 1 as All services max., was predicted to reduce NMR by nearly half (44%) potentially averting over 3 million DALYs during the intervention period.
Discussion, paragraph 2:
As expected, scenarios where coverage and quality of all services were maximised in tandem led to the greatest percentage reduction in the average MMR (64%), NMR (44%) and SBR (14%).
Results, paragraph 3 (the comparison of the Status Quo with the no-services scenario):
Current delivery of all included services within the SQ scenario are estimated to reduce the average MMR by 171.3 (95%) CI [161.2, 181.4] deaths per 100,000 live births (a 52% difference from the scenario All services min.) between 2023 and 2030.
Tables 2, 3 and 4, rate columns ("Average rate (95% CI) / Mean difference from SQ (95% CI) / Percentage difference from SQ"). The archived cells run the three values together; spaces are added here between them:
Table 2, Status Quo (SQ): 330.5 (322.4, 338.6) Ref.
Table 2, All services max.: 120.0 (114.8, 125.2) –210.5 (–220.0, –201.1) –63.7%
Table 2, All services min.: 501.8 (493.0, 510.7) 171.3 (161.2, 181.4) 51.8%
Table 3, Status Quo (SQ): 16.6 (16.4, 16.8) Ref.
Table 3, All services max.: 14.4 (14.2, 14.6) –2.3 (–2.6, –2.0) –13.6%
Table 3, All services min.: 18.2 (18.0, 18.4) 1.9 (1.3, 1.9) 9.6%
Table 4, Status Quo (SQ): 16.9 (16.7, 17.1) Ref.
Table 4, All services max.: 9.4 (9.3, 9.5) –7.5 (–7.6, –7.3) –44.2%
Table 4, All services min.: 26.5 (26.3, 26.7) 9.6 (9.3, 9.9) 57.0%
Footnote to Tables 2–4:
Percentage difference from the SQ is only presented for those scenarios in which the 95% confidence interval does not contain 0.
What the paper's own numbers show
Method. The tables express each scenario as a percentage difference from the Status Quo (SQ), i.e. 100 × (scenario − SQ) / SQ. I applied that formula to the printed means for the All services max. and All services min. rows of all three rate columns, and to the total-count and DALY columns. Because the means are printed to one decimal, I also computed the full range of the percentage over all unrounded values that would print as the displayed means. I then tested every plausible alternative source of the triplet 10 / 52 / 57 (other outcome columns, other denominators, other scenarios). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| SBR reduction, All services max. vs SQ | Abstract | 10% | 13.3% from means 16.6 → 14.4 (12.7%–13.8% over rounding); not 10% |
| MMR reduction, All services max. vs SQ | Abstract | 52% | 63.7% from means 330.5 → 120.0; not 52% |
| NMR reduction, All services max. vs SQ | Abstract | 57% | 44.4% from means 16.9 → 9.4 (43.9%–44.8% over rounding); not 57% |
| SBR, All services max. vs SQ | Table 3; Results; Discussion | –2.3, –13.6%; "14%" | –2.2 and –13.3% from the levels; –13.6% lies inside the rounding range, so the table and text are consistent |
| MMR, All services max. vs SQ | Table 2; Results; Discussion | –210.5, –63.7%; "64%" | –210.5, –63.7%: consistent |
| NMR, All services max. vs SQ | Table 4; Results; Discussion | –7.5, –44.2%; "44%" | –7.5, –44.4%: consistent within rounding |
| Total stillbirths / maternal deaths / neonatal deaths, All services max. | Tables 3, 2, 4 | –13.3% / –63.6% / –44.0% | –13.3% / –63.6% / –44.0%: consistent, and none is 10 / 52 / 57 |
| Maternal / neonatal DALYs, All services max. | Tables 2, 4 | –67.2% / –44.4% | –67.2% / –44.4%: not 52 / 57 |
| SBR, All services min. vs SQ | Table 3 | +1.9 (1.3, 1.9), +9.6% | +1.6 from the levels, +9.6%; rounds to 10% |
| MMR, All services min. vs SQ | Table 2; Results ("a 52% difference") | +171.3, +51.8% | +171.3, +51.8%; rounds to 52% |
| NMR, All services min. vs SQ | Table 4 | +9.6, +57.0% | +9.6, +56.8%; the printed 57.0% rounds to 57% |
| Total stillbirths / maternal deaths / neonatal deaths, All services min. | Tables 3, 2, 4 | +9.5% / +51.7% / +56.9% | +9.5% / +51.7% / +56.9%; also round to 10 / 52 / 57 |
| Abstract minus table value, All services max. | derived | 10 / 52 / 57 | –3.6 / –11.7 / +12.8 percentage points (SBR / MMR / NMR) |
| Ranking of reductions across outcomes | derived | Abstract: NMR > MMR > SBR | Tables: MMR > NMR > SBR |
Every figure in the body for the All services max. scenario agrees with the tables. The abstract is the only place where the triplet 10 / 52 / 57 appears for that scenario, and the only row of the tables that reproduces it is All services min., which is a comparison in the opposite direction (mortality rises when services are removed).
A separate, minor inconsistency appears in the same table row: the All services min. SBR mean difference is printed as 1.9 with 95% CI (1.3, 1.9), so the point estimate equals its upper bound, while the printed levels (18.2 − 16.6) give 1.6. The printed 9.6% matches 1.6/16.6, not 1.9/16.6 (11.4%). This does not affect the main finding.
Figure
Figure 1. Left (orange): the abstract sentence and the three reductions it attributes to maximising all services. Right (blue): the same scenario in Tables 2–4, with the SQ and scenario means, the percentage the tables print (gray) and the percentage recomputed from those means (blue), and the Results/Discussion wording. The double arrow marks the conflict: the same scenario carries different numbers in the abstract and in the body. Bottom: the All services min. row of the same tables, the scenario in which all maternity services are removed. Its percentage increases from SQ (printed and recomputed) round to the abstract's 10%, 52% and 57%.
Figure 2. Magnitude of the change from the Status Quo for each mortality rate, on one linear axis starting at zero. Orange diamonds: the reductions stated in the abstract for All services max. Blue dots: the reductions for All services max. recomputed from the tabulated means. Open gray rings: the increases for All services min. recomputed from the tabulated means. For every outcome the orange diamond sits on the gray ring and away from the blue dot, and the order of MMR and NMR is reversed between the abstract and the tables.
Why it matters
The abstract sentence is the only place in the abstract that quantifies the paper's headline scenario, and readers who see only the abstract (or databases and summaries that reuse it) receive three wrong magnitudes. The maternal mortality reduction is understated (52% instead of about 64%), the stillbirth reduction is understated (10% instead of about 14%), and the neonatal mortality reduction is overstated (57% instead of about 44%). The abstract therefore also implies that neonatal mortality is the outcome most reduced by maximising services, whereas the paper's results show maternal mortality falls most. In a paper intended to inform investment in maternal and newborn health, which outcome responds most strongly is itself a policy-relevant result.
The error is confined to the abstract. The direction of every effect, its statistical significance in the tables, and the qualitative conclusion that maximising coverage and quality of all services together gives the largest reductions all stand. The findings that coverage without quality improvement has little effect, and the statements about the SDG 3.1 and 3.2 targets, rest on the tabulated rates and are not affected.
Explanations considered
- Rounding. This does not account for the numbers. Over every unrounded value that prints as the displayed means, the All services max. reductions range from 12.7% to 13.8% (SBR), are 63.7% (MMR), and range from 43.9% to 44.8% (NMR). None can round to 10%, 52% or 57%.
- A different outcome measure (total deaths or DALYs instead of rates). This does not account for the numbers. For All services max. the total-count reductions are 13.3%, 63.6% and 44.0%, and the DALY reductions are 67.2% (maternal) and 44.4% (neonatal).
- A different comparator or denominator. This does not account for the numbers. All services max. relative to All services min. gives reductions of 20.9%, 76.1% and 64.5%. The SQ relative to All services min., with the no-services scenario as denominator, gives 8.8%, 34.1% and 36.2%.
- A single-service scenario. This does not account for the numbers. The largest single-service reductions are IP max. for SBR (9.7%) and NMR (28.8%) and PN max. for MMR (42.0%); no single scenario gives 10 / 52 / 57, and the 57% NMR figure is larger than any single-service NMR reduction in Table 4.
- The All services min. vs SQ comparison. This accounts for all three numbers. The printed percentages 9.6%, 51.8% and 57.0% (and the recomputed 9.6%, 51.8% and 56.8%) round to 10%, 52% and 57%, and the total-count column gives the same rounded triplet. The Results text itself describes the MMR figure for that comparison as "a 52% difference from the scenario All services min.". These are increases in mortality when services are removed, measured relative to SQ, not reductions achieved by maximising services.
- A conversion or typesetting error in the archived text. This is excluded. The publisher's HTML carries the same abstract sentence, and the publisher's table pages match the PMC tables cell for cell (see Status).
- A later version or correction that resolves it. This is excluded; none was found (see Status).
Status
- Version of record. The publisher's article page (nature.com/articles/s41467-025-59060-2, dateModified 2025-04-25, re-fetched on 2026-10-24) carries the abstract sentence with "a 10%, 52% and 57% reduction respectively". The page's JSON-LD and meta descriptions repeat it, and so does the Europe PMC full-text XML. In the version-of-record check, the publisher's Table 2, 3 and 4 pages and the Results and Discussion wording matched the PMC text cell for cell. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements and history. The four electronic supplementary files were searched: the supplementary information (model documentation, parameters and costs), the peer review file and a short reporting document. None restates the headline percentages in a way that explains the abstract. The peer review file shows that the all-services maximisation and removal scenarios were added during review, together with updates to the abstract. The earlier preprint (Research Square, doi:10.21203/rs.3.rs-3937439/v1) has a different abstract that predates those scenarios. The 40.4 GB model archive on Zenodo (doi:10.5281/zenodo.14976918) was not opened; it is not needed, because the published tables settle the comparison.
- Corrections and comments. As of 2026-10-24 there is no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, PMC and Europe PMC list no correction notices, and the publisher's page shows none. A web search found no correction or public comment. PubPeer's search page and API refused automated access (HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC12032021; publisher HTML and table pages at nature.com; Europe PMC full-text XML; Crossref; publisher electronic supplementary material; Research Square preprint; Zenodo record metadata.
Suggested correction
- Replace the abstract's "(a 10%, 52% and 57% reduction respectively)" with the values the paper's tables report for All services max. versus the Status Quo: approximately 14%, 64% and 44% (Table 3: −13.6%; Table 2: −63.7%; Table 4: −44.2%) for the stillbirth, maternal and neonatal mortality rates.
- If the authors wish to keep the no-services comparison in the abstract, present it as a separate statement of what current service delivery averts (mortality rates about 10%, 52% and 57% higher without services, relative to SQ), labelled as such.
- Minor: in Table 3, check the All services min. SBR mean difference and CI, "1.9 (1.3, 1.9)", against the printed levels (18.2 − 16.6 = 1.6) and the printed 9.6%.
Recomputation script (python recompute.py.txt)
# Recompute the headline percentages of PMC12032021 (Nat Commun 2025,
# doi:10.1038/s41467-025-59060-2) from the paper's own Tables 2-4, and draw
# fig1 (abstract vs tables) and fig2 (which table row the abstract matches).
# Run: python recompute.py.txt (writes fig1/fig2 .svg and .png next to this file)
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Tables 2-4, columns "Average rate / Mean difference from SQ / Percentage
# difference from SQ" and the total-count and DALY columns, as printed.
SQ = {"SBR": 16.6, "MMR": 330.5, "NMR": 16.9,
"stillbirths": 98358, "maternal deaths": 19541, "neonatal deaths": 99902,
"maternal DALYs": 795868, "neonatal DALYs": 6981837}
# (level, printed mean difference, printed % difference)
MAX = {"SBR": (14.4, -2.3, -13.6), "MMR": (120.0, -210.5, -63.7), "NMR": (9.4, -7.5, -44.2),
"stillbirths": (85281, -13077, -13.3), "maternal deaths": (7113, -12427, -63.6),
"neonatal deaths": (55908, -43994, -44.0),
"maternal DALYs": (261359, -534509, -67.2), "neonatal DALYs": (3882514, -3099323, -44.4)}
MIN = {"SBR": (18.2, 1.9, 9.6), "MMR": (501.8, 171.3, 51.8), "NMR": (26.5, 9.6, 57.0),
"stillbirths": (107654, 9297, 9.5), "maternal deaths": (29642, 10101, 51.7),
"neonatal deaths": (156699, 56798, 56.9),
"maternal DALYs": (1233184, 437316, 55.0), "neonatal DALYs": (10964980, 3983143, 57.1)}
# Single-service "max." rows (rate % difference from SQ), used only to test
# whether the abstract triplet could come from another scenario.
SINGLE_MAX = {"AN max.": {"SBR": -4.4, "MMR": -9.8, "NMR": -11.3},
"IP max.": {"SBR": -9.7, "MMR": -23.7, "NMR": -28.8},
"PN max.": {"SBR": None, "MMR": -42.0, "NMR": -9.7}} # Table 3 omits PN scenarios
ABSTRACT = {"SBR": 10, "MMR": 52, "NMR": 57} # "a 10%, 52% and 57% reduction respectively"
RESULTS_TEXT = {"SBR": 14, "MMR": 64, "NMR": 44} # Results paragraphs and Discussion
RATES = ("SBR", "MMR", "NMR")
NAMES = {"SBR": "Stillbirth rate (SBR)", "MMR": "Maternal mortality ratio (MMR)",
"NMR": "Neonatal mortality rate (NMR)"}
DECIMALS = {"SBR": 1, "MMR": 1, "NMR": 1} # rates printed to 0.1
def pct(a, b):
return 100.0 * a / b
def rounding_range(sq, x, d=1):
"""Range of (x - sq)/sq in % over all unrounded values that print as sq and x."""
h = 0.5 * 10 ** (-d)
vals = [pct(xx - ss, ss) for ss in (sq - h, sq + h) for xx in (x - h, x + h)]
return min(vals), max(vals)
R = {} # recomputed numbers used by the figures
print("== 1. All services max. vs SQ (Tables 2-4)")
print(f"{'quantity':16s} {'SQ':>10s} {'max':>10s} {'diff(levels)':>13s} {'diff(print)':>12s} "
f"{'% (levels)':>10s} {'% (print diff)':>14s} {'% printed':>10s}")
for k in SQ:
v, d, p = MAX[k]
print(f"{k:16s} {SQ[k]:>10} {v:>10} {v - SQ[k]:>13.1f} {d:>12} "
f"{pct(v - SQ[k], SQ[k]):>10.1f} {pct(d, SQ[k]):>14.1f} {p:>10}")
for k in RATES:
v = MAX[k][0]
R[("max", k)] = -pct(v - SQ[k], SQ[k])
lo, hi = rounding_range(SQ[k], v, DECIMALS[k])
R[("maxrange", k)] = (-hi, -lo)
print(f" {k}: reduction from printed means {R[('max', k)]:.1f}%; over the rounding range of "
f"both means {-hi:.1f}% to {-lo:.1f}%; table prints {-MAX[k][2]:.1f}%; "
f"Results/Discussion text {RESULTS_TEXT[k]}%; abstract {ABSTRACT[k]}%")
print("\n== 2. All services min. vs SQ (Tables 2-4)")
for k in SQ:
v, d, p = MIN[k]
print(f"{k:16s} {SQ[k]:>10} {v:>10} {v - SQ[k]:>13.1f} {d:>12} "
f"{pct(v - SQ[k], SQ[k]):>10.1f} {pct(d, SQ[k]):>14.1f} {p:>10}")
for k in RATES:
v = MIN[k][0]
R[("min", k)] = pct(v - SQ[k], SQ[k])
lo, hi = rounding_range(SQ[k], v, DECIMALS[k])
print(f" {k}: increase from printed means {R[('min', k)]:.1f}% (rounding range {lo:.1f}% to "
f"{hi:.1f}%); table prints {MIN[k][2]:.1f}% -> rounds to {round(MIN[k][2])}%; "
f"abstract {ABSTRACT[k]}%")
k = "SBR"
print(f" Note: SBR min. printed mean difference {MIN[k][1]} vs {MIN[k][0] - SQ[k]:.1f} from the "
f"printed levels; its printed CI is (1.3, 1.9), so the mean equals the upper bound. "
f"{MIN[k][1]}/{SQ[k]} = {pct(MIN[k][1], SQ[k]):.1f}%, {MIN[k][0] - SQ[k]:.1f}/{SQ[k]} = "
f"{pct(MIN[k][0] - SQ[k], SQ[k]):.1f}% (table prints {MIN[k][2]}%).")
print("\n== 3. Which printed triplet rounds to the abstract's 10 / 52 / 57?")
cands = {
"All services max., rates (Tables 2-4)": [abs(MAX[k][2]) for k in RATES],
"All services max., totals (deaths)": [abs(MAX[k][2]) for k in ("stillbirths", "maternal deaths", "neonatal deaths")],
"All services max., DALYs (SBR has none)": [None, abs(MAX["maternal DALYs"][2]), abs(MAX["neonatal DALYs"][2])],
"All services min., rates (Tables 2-4)": [MIN[k][2] for k in RATES],
"All services min., totals (deaths)": [MIN[k][2] for k in ("stillbirths", "maternal deaths", "neonatal deaths")],
"All services min., DALYs (SBR has none)": [None, MIN["maternal DALYs"][2], MIN["neonatal DALYs"][2]],
"max. vs min. (min. as denominator), rates": [pct(MIN[k][0] - MAX[k][0], MIN[k][0]) for k in RATES],
"SQ vs min. (min. as denominator), rates": [pct(MIN[k][0] - SQ[k], MIN[k][0]) for k in RATES],
"AN max., rates": [abs(SINGLE_MAX["AN max."][k]) for k in RATES],
"IP max., rates": [abs(SINGLE_MAX["IP max."][k]) for k in RATES],
"PN max., rates (no SBR row)": [None if SINGLE_MAX["PN max."][k] is None else abs(SINGLE_MAX["PN max."][k]) for k in RATES],
}
target = [ABSTRACT[k] for k in RATES]
for name, vals in cands.items():
shown = ", ".join(" - " if v is None else f"{v:5.1f}" for v in vals)
match = all(v is not None and round(v) == t for v, t in zip(vals, target))
print(f" {name:44s} SBR/MMR/NMR = {shown} matches 10/52/57: {match}")
print("\n== 4. Size of the discrepancy (abstract minus All services max. table value, points)")
for k in RATES:
print(f" {k}: abstract {ABSTRACT[k]}% vs table {-MAX[k][2]:.1f}% -> {ABSTRACT[k] + MAX[k][2]:+.1f} points")
print(" Ranking of reductions, abstract:", " > ".join(sorted(RATES, key=lambda k: -ABSTRACT[k])))
print(" Ranking of reductions, tables: ", " > ".join(sorted(RATES, key=lambda k: MAX[k][2])))
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
LW = 1.5 # 2 px at 96 px/in = 1.5 pt
W = 7.2
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
plt.close(fig)
def wrap(s, w):
return "\n".join(textwrap.wrap(s, w))
# fig1: Abstract panel vs Tables panel, with a connector naming the conflict,
# and a third panel showing the All services min. row the abstract matches.
fig = plt.figure(figsize=(W, 6.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 84); ax.axis("off")
neg = lambda s: s.replace("-", "\u2212")
def panel(x, y, w, h, title, sub, ec):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=ec, lw=LW))
ax.text(x + 1.6, y + h - 1.4, title, fontsize=11, fontweight="bold", va="top")
ax.text(x + 1.6, y + h - 5.4, sub, fontsize=10, color=INK2, va="top")
# Abstract panel (reported, orange)
panel(1, 43, 42, 39, "Abstract", "Scenario: All services max.", ORANGE)
ax.text(2.6, 70.5, wrap("\u201c\u2026 the largest reductions in the stillbirth, maternal and "
"neonatal mortality rates were observed when the use and quality of all "
"services was maximised concurrently (a 10%, 52% and 57% reduction "
"respectively)\u201d", 40), fontsize=10, va="top", style="italic")
ax.text(2.6, 50.6, "reported reductions", fontsize=10, color=ORANGE, va="center")
for i, k in enumerate(RATES):
ax.text(2.6 + 13.8 * i, 46.2, neg(f"{k} -{ABSTRACT[k]}%"), fontsize=10.5,
fontweight="bold", color=ORANGE, va="center")
# Tables panel (blue = recomputed from the paper's own means)
panel(57, 43, 42, 39, "Tables 2\u20134, Results, Discussion", "Scenario: All services max. vs SQ",
BLUE)
ax.text(65.0, 70.5, "SQ \u2192 max.", fontsize=10, color=INK2, va="center")
ax.text(88.6, 70.5, "table", fontsize=10, color=INK2, va="center", ha="right")
ax.text(98.0, 70.5, "recomp.", fontsize=10, color=BLUE, va="center", ha="right")
for i, k in enumerate(RATES):
yy = 65.5 - 4.8 * i
ax.text(58.6, yy, k, fontsize=10, fontweight="bold", va="center")
ax.text(65.0, yy, f"{SQ[k]} \u2192 {MAX[k][0]}", fontsize=10, va="center")
ax.text(88.6, yy, neg(f"{MAX[k][2]:.1f}%"), fontsize=10, va="center", ha="right", color=INK2)
ax.text(98.0, yy, neg(f"-{R[('max', k)]:.1f}%"), fontsize=10, va="center", ha="right",
color=BLUE, fontweight="bold")
ax.text(58.6, 50.0, "Results and Discussion text:\n\u201cMMR (64%), NMR (44%)\nand SBR (14%)\u201d",
fontsize=10, va="center", style="italic")
# connector abstract <-> tables
ax.add_patch(FancyArrowPatch((44.2, 62), (55.8, 62), arrowstyle="<->", mutation_scale=14,
lw=LW, color=INK))
ax.text(50, 64, "same\nscenario,\ndifferent\nnumbers", ha="center", va="bottom", fontsize=10,
fontweight="bold")
# All services min. panel
panel(24, 2, 66, 25, "Tables 2\u20134: All services min. vs SQ",
"No maternity services: an increase in mortality, not a reduction", INK2)
ax.text(25.6, 13.4, "table", fontsize=10, va="center", color=INK2)
ax.text(25.6, 7.0, "recomputed", fontsize=10, va="center", color=BLUE)
for i, k in enumerate(RATES):
ax.text(43 + 15.5 * i, 13.4, f"{k} +{MIN[k][2]:.1f}%", fontsize=10, va="center", color=INK2)
ax.text(43 + 15.5 * i, 7.0, f"{k} +{R[('min', k)]:.1f}%", fontsize=10, va="center",
color=BLUE, fontweight="bold")
ax.add_patch(FancyArrowPatch((16, 42.2), (30, 28.6), arrowstyle="-|>", mutation_scale=14,
lw=LW, color=ORANGE, connectionstyle="arc3,rad=0.2"))
ax.text(1.5, 33.5, "10 / 52 / 57 are\nthis row rounded\nto whole percent", fontsize=10,
color=INK, va="center", fontweight="bold")
save(fig, "fig1")
# fig2: dot plot, % change vs SQ (magnitude), one row per outcome
fig, axd = plt.subplots(figsize=(W, 3.3))
ys = {"SBR": 2, "MMR": 1, "NMR": 0}
for k in RATES:
y = ys[k]
axd.plot(R[("max", k)], y + 0.12, "o", ms=9, color=BLUE, zorder=3)
axd.plot(R[("min", k)], y - 0.12, "o", ms=11, mfc="none", mec=INK2, mew=LW, zorder=3)
axd.plot(ABSTRACT[k], y - 0.12, "D", ms=8, color=ORANGE, zorder=4)
axd.annotate(f"{R[('max', k)]:.1f}%", (R[("max", k)], y + 0.12), xytext=(0, 9),
textcoords="offset points", ha="center", fontsize=10, color=BLUE)
axd.annotate(f"abstract {ABSTRACT[k]}%; min. row {R[('min', k)]:.1f}%", (ABSTRACT[k], y - 0.12),
xytext=(0, -16), textcoords="offset points", ha="center", fontsize=10, color=INK2)
axd.set_yticks([ys[k] for k in RATES]); axd.set_yticklabels([NAMES[k] for k in RATES])
axd.set_ylim(-0.7, 2.6); axd.set_xlim(0, 70)
axd.set_xlabel("Change from Status Quo, % (magnitude)")
axd.grid(axis="x", color=GRID, lw=0.8); axd.set_axisbelow(True)
for s in ("top", "right", "left"):
axd.spines[s].set_visible(False)
axd.tick_params(axis="y", length=0)
h1, = axd.plot([], [], "D", ms=8, color=ORANGE, ls="none",
label="Abstract: stated reduction, All services max.")
h2, = axd.plot([], [], "o", ms=9, color=BLUE, ls="none",
label="Reduction, All services max., recomputed from table means")
h3, = axd.plot([], [], "o", ms=11, mfc="none", mec=INK2, mew=LW, ls="none",
label="Increase, All services min., recomputed from table means")
axd.legend(handles=[h1, h2, h3], loc="upper center", bbox_to_anchor=(0.42, -0.24), ncol=1,
frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract and Discussion give near-loop median PFS as 11.1 months; Results and source data give 5.5
Nat Commun (2025) · doi:10.1038/s41467-025-61817-8 · PMC12460621
Summary
The paper's exploratory ZENITH20 analysis compares progression-free survival (PFS) under poziotinib between 63 patients with near-loop and 19 with far-loop EGFR exon 20 insertions. The abstract and the Discussion give the near-loop median PFS as 11.1 months (vs 3.5, p = 0.016). The Results paragraph that reports the Kaplan-Meier analysis (Fig. 5c) gives 5.5 months (vs 3.5, log-rank p = 0.015, HR 0.42, 95% CI 0.21-0.86). The paper's own patient-level Source Data reproduce the Results values exactly (5.45 vs 3.55 months, p = 0.0153, Cox HR 0.42, 0.21-0.86). The abstract values are reproduced exactly (11.14 vs 3.48 months, p = 0.0160) only when the progression/censoring indicator is reversed. The direction and significance of the near-loop advantage are unchanged, but the abstract roughly doubles the headline near-loop estimate and puts it close to the 12.0 months it reports for mobocertinib.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py.
Abstract:
Although the study’s primary efficacy endpoint was not met in the overall cohort, the exploratory analysis indicates poziotinib has superior benefit in EGFRex20 near- versus far-insertions showing greater mean tumor size reduction (-25.9% vs. -9.8%, p = 0.0014) and progression-free survival (PFS, 11.1 vs. 3.5 months, p = 0.016). In comparison, in the previously published EXCLAIM trial (NCT02716116), mobocertinib demonstrates similar activities across both groups in tumor size reduction (-38.5% vs. -34.1%, p = 0.59) and PFS (12.0 vs. 13.0 months, p = 0.99).
Results, section "Clinical response with poziotinib in near- vs. far- EGFRex20ins NSCLC", second paragraph:
In this cohort, 98 patients had at least one tumor measurement after poziotinib treatment, among those, 82 patients had exon 20 insertions location information per local next-generation sequencing (NGS) testing, with 63 (77%) patients whose tumor mutation were in the near-loop amino acids A767-P772 and 19 (23%) in the far-loop H773-C775 amino acids.
The median progression-free survival (PFS) was also significantly different in patients with near- vs far-loop insertions at 5.5 months (near) vs. 3.5 months (far) (Log Rank p = 0.015), with a PFS hazard ratio of 0.42 (near- vs. far- 95% Confidence Interval [CI] 0.21-0.86, Fig. 5c).
Results, same section, first paragraph (all treated patients):
Among the 17 responders, the median duration of response (DoR) was 7.4 months (95% CI, 3.7 to 9.7, Fig. 4b), and median progression-free survival (PFS) for all treated patients was 4.2 months (95% CI, 3.7 to 6.6).
Discussion, second paragraph:
In the ZENITH20 trial cohort 1, the clinical efficacy was limited in the total population, not meeting the previously defined efficacy endpoint, however, the near-loop insertions responded better to poziotinib both at tumor size reduction and at PFS (11.1 months [near] vs. 3.5 months [far]).
Methods, "Retrospective analysis of patient outcomes by mutation location":
progression-free survival (PFS), which was measured as time from the beginning of TKI treatment until radiologic progression or death
Methods, "Statistics and reproducibility":
Median PFS was determined using the Kaplan-Meier method. Hazard ratios and p-values were determined using GraphPad Prism software and the Mantel-Cox Log Rank method.
Fig. 5 legend, panel c:
c Kaplan-Meier plot of progression-free survival of patients with near loop and far loop mutants. The long-Rank Mantel-Cox approach was used to determine the p-value.
Supplementary Data 1 (MOESM3), column header in sheets "Near 64" and "Far 20":
Censor, 0=PD, 1=censor
What the paper's own numbers show
Method. The Source Data file (MOESM7, Fig 5.xlsx, sheet "Fig 5a-c") has one row per patient for the Fig. 5a-c analysis, with insertion location, best response, PFS in months and a PFS_status column. It has 63 near-loop and 19 far-loop rows, the same as the text. The sheet does not say which value of PFS_status marks progression, so I worked that out from two other sources in the paper's own files. (1) All 14 patients whose best response was progressive disease have PFS_status = 1. (2) Of the 75 patients who appear in both the Source Data and Supplementary Data 1, every one with PFS_status = 1 has "Censor" = 0 (= PD) in Supplementary Data 1, and every one with PFS_status = 0 has "Censor" = 1 (= censor), 51 and 24 patients respectively. Their PFS times are identical in the two files. So PFS_status = 1 is progression or death. With my own Kaplan-Meier and Mantel-Cox log-rank code and a one-covariate Cox model (Efron ties), I computed the near- vs far-loop medians, p-value and hazard ratio twice: once with PFS_status = 1 as the event, and once with the indicator reversed. Every number below is printed by recompute.py.txt, which downloads both files from the publisher.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Near-loop median PFS | Abstract; Discussion | 11.1 months | 5.45 months with PFS_status = 1 as event (42 of 63 events); 11.14 months only with the indicator reversed (21 events) |
| Near-loop median PFS | Results (Fig. 5c) | 5.5 months | 5.45 months |
| Far-loop median PFS | Abstract; Results; Discussion | 3.5 months | 3.55 months (11 of 19 events); 3.48 months with the indicator reversed |
| Log-rank p, near vs far | Abstract | 0.016 | 0.0153; 0.0160 with the indicator reversed |
| Log-rank p, near vs far | Results; Fig. 5c | 0.015 | 0.0153 |
| HR near vs far (95% CI) | Results | 0.42 (0.21-0.86) | 0.42 (0.21-0.86), Cox with Efron ties; 0.35 (0.14-0.85) with the indicator reversed |
| Near-minus-far difference in medians | Abstract/Discussion vs Results | 7.6 vs 2.0 months | 1.90 months; 7.66 with the indicator reversed |
| Gap between the two near-loop medians printed | Abstract vs Results | 11.1 vs 5.5 | 5.6 months; ratio 2.02 |
| Pooled KM median, all 82 patients with known location | not reported (all 115 treated: 4.2, 95% CI 3.7-6.6) | n/a | 4.99 months; 11.14 with the indicator reversed |
| Near-loop patients with PFS ≥ 12 months | Fig. 5c number at risk | n/a | 5 of 63 |
Under the coding that matches Supplementary Data 1, the Source Data reproduce every Results value to the printed precision: both medians, the p-value, the hazard ratio and both CI limits. Reversing the indicator, so that censored patients are treated as progressing, reproduces both abstract and Discussion numbers to the printed precision (11.14 → 11.1, 3.48 → 3.5, 0.0160 → 0.016). It does not reproduce the reported hazard ratio. No other coding I tried reproduces 11.1. Under the reversed coding the pooled Kaplan-Meier median of all 82 patients is also 11.14 months, far outside the 95% CI of 3.7 to 6.6 months reported for all 115 treated patients. The pooled median under the documented coding is 4.99 months, inside that interval. The populations differ (82 patients with at least one tumor measurement and a known insertion location vs all 115 treated), so this comparison is a plausibility check only.
Figure
Figure 1. The two versions of the same ZENITH20 near- vs far-loop PFS comparison, as printed. Left: the abstract and Discussion (11.1 vs 3.5 months, p = 0.016; neither gives a hazard ratio). Right: the Results paragraph citing Fig. 5c (5.5 vs 3.5 months, log-rank p = 0.015, HR 0.42, 95% CI 0.21-0.86). The patient groups (63 near-loop, 19 far-loop) and the far-loop median are the same in all three places. The conflicting values are in orange.
Figure 2. Median PFS by insertion location, in months, on a linear axis starting at 0. Filled circles are near-loop medians, open squares far-loop medians. Orange rows are values printed in the paper; blue rows are recomputed by recompute.py.txt from the paper's Source Data (sheet "Fig 5a-c", 63 near-loop and 19 far-loop patients). The p-values and hazard ratios on the right are log-rank and Cox (Efron) estimates, near vs far. The bottom pair: with PFS_status = 1 as progression, the coding confirmed by Supplementary Data 1, the data reproduce the Results and Fig. 5c (5.45 vs 3.55 months, p = 0.0153, HR 0.42). The top pair: the abstract and Discussion values are reproduced only when censored and progressing patients are swapped (11.14 vs 3.48 months, p = 0.0160). That swap gives HR 0.35, not the reported 0.42.
Why it matters
- The headline estimate is about twice the supported value. The abstract is the only place most readers see the result, and it gives near-loop poziotinib patients a median PFS of 11.1 months. The paper's own analysis (Results, Fig. 5c, Source Data) supports 5.5 months. The near-minus-far difference becomes 7.6 months in the abstract instead of about 2 months.
- It changes the drug comparison the abstract invites. The next sentence of the abstract reports mobocertinib PFS of 12.0 (near) and 13.0 (far) months in EXCLAIM. With 11.1 months, poziotinib in near-loop disease looks almost as good as mobocertinib. With 5.5 months it is less than half. The two trials are separate single-arm studies, so this comparison is informal either way, but the abstract places the numbers side by side.
- The 11.1-month figure also conflicts with the cohort as a whole. The paper reports a median PFS of 4.2 months (95% CI 3.7-6.6) for all 115 treated patients, and the trial did not meet its primary endpoint. Near-loop patients are 77% of those with known location, so a near-loop median of 11.1 months is hard to reconcile with the overall figure. The value recomputed from the source data, 5.5 months, is consistent with it.
- What does not change. Under the documented coding the near-loop group still has significantly longer PFS than the far-loop group (log-rank p = 0.015, HR 0.42, 95% CI 0.21-0.86). The paper's qualitative conclusion, that insertion location affects poziotinib sensitivity, is supported by its own Results. The tumor-shrinkage comparison (-25.9% vs -9.8%) is not affected.
Explanations considered
- A different population or subgroup. Not supported. The abstract, Results and Discussion all refer to ZENITH20 cohort 1 and the near- vs far-loop comparison, and the far-loop median is 3.5 months in all three places. The Results define the groups as 63 near-loop and 19 far-loop patients, and the Source Data sheet behind Fig. 5a-c has exactly those 82 patients. The 11.1-month value is reproduced from this same set of 82 patients, so it is not a figure for another subset. It is also not the earlier single-center trial the Results cite (ref. 22), and not one of the EXCLAIM values (12.0 and 13.0 months).
- A different endpoint, time point or model. Not supported. The Methods define PFS once (from TKI start to radiologic progression or death) and describe one Kaplan-Meier and Mantel-Cox analysis. No second data cut-off, adjusted model or alternative endpoint is described.
- The Results value is the one in error. Not supported. Fig. 5c, which the Results cite, prints p = 0.015. The Source Data reproduce the Results medians, p-value, hazard ratio and CI limits exactly under the coding confirmed by Supplementary Data 1 and by the best-response column.
- Uncertainty about which
PFS_statusvalue means progression. Resolved from the paper's own files. All 14 patients with best response "PD" havePFS_status= 1. All 75 patients who appear in both files havePFS_status= 1 exactly when Supplementary Data 1 marks them "0=PD". With the reversed coding, none of the 14 PD patients would count as progressing, which cannot be right. - How the 11.1 arose. The abstract and Discussion values (11.1, 3.5 and p = 0.016) are all reproduced to the printed precision by one specific change to the data: counting censored patients as events and events as censored. This is consistent with a second run of the analysis using the inverted indicator. The paper does not say how the abstract numbers were produced, so this is a mechanism that fits the numbers, not an established account.
- A conversion or typesetting artifact. Ruled out. The numbers are plain decimals in running text, with no superscripts, signs or table cells. They are identical in the publisher's HTML, the Europe PMC full-text XML and the PMC text.
- Side observations, not part of the main finding. (1) Supplementary Data 1 lists 64 near-loop and 20 far-loop patients, 75 of whom are among the 82 in the Source Data (7 are only in the Source Data, 9 only in Supplementary Data 1). On its own 84 patients it gives 5.26 vs 3.55 months (log-rank p = 0.0036), which also does not support 11.1. (2) The Results define the far-loop group as "H773-C775" and the Methods as "between H773-R776". (3) Only the Cox model with Efron ties reproduces the reported HR and CI. The Mantel-Haenszel and O/E-ratio estimates from the same data are 0.33 (0.13-0.81) and 0.46 (0.20-1.10). The Methods attribute hazard ratios to the Mantel-Cox method, so the authors may wish to say which estimator was used.
Status
- Version of record. I downloaded the publisher's HTML page (nature.com/articles/s41467-025-61817-8) and checked the Europe PMC full-text XML. Both carry the same three passages as the PMC archive: "11.1 vs. 3.5 months, p = 0.016" in the abstract, "5.5 months (near) vs. 3.5 months (far) (Log Rank p = 0.015)" with HR 0.42 (0.21-0.86) in the Results, and "11.1 months [near] vs. 3.5 months [far]" in the Discussion. The publisher's Fig. 5c prints p = 0.015, with 63 near-loop and 19 far-loop patients at risk at time 0. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The article lists seven ESM files. I read MOESM1 (Supplementary Information), MOESM3 (Supplementary Data 1, ZENITH20 near/far patient-level data), MOESM4 (Supplementary Data 2, EXCLAIM) and MOESM7 (Source Data ZIP, including
Fig 5.xlsx). The analysis uses MOESM7 sheet "Fig 5a-c" (SHA-256 of the ZIP: 8dd564f4…c583) and MOESM3 (SHA-256 552de51f…ee95). None of the supplementary files gives an 11.1-month near-loop median under the documented coding, and none explains the abstract value. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has empty update-to and updated-by fields, and the Europe PMC record (PMID 40993146) has no comment or correction entries. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC12460621; publisher article page and ESM files at https://doi.org/10.1038/s41467-025-61817-8 (static-content.springer.com, MOESM1, MOESM3, MOESM4, MOESM7); Europe PMC full text and record; Crossref; PubPeer.
Suggested correction
In the abstract, replace
progression-free survival (PFS, 11.1 vs. 3.5 months, p = 0.016)
with
progression-free survival (PFS, 5.5 vs. 3.5 months, p = 0.015)
and in the second paragraph of the Discussion, replace
(11.1 months [near] vs. 3.5 months [far])
with
(5.5 months [near] vs. 3.5 months [far])
These are the values in the Results, in Fig. 5c and in the Source Data under the coding that matches Supplementary Data 1. The authors may also wish to state that the hazard ratio of 0.42 comes from a Cox model, if that is how it was estimated. They may also wish to reconcile the patient lists of Supplementary Data 1 (64/20) and the Source Data (63/19), and the far-loop range given in the Results (H773-C775) and the Methods (H773-R776).
Recomputation script (python recompute.py.txt)
"""Recompute the ZENITH20 near- vs far-loop PFS comparison of PMC12460621
(Nat Commun 2025, doi:10.1038/s41467-025-61817-8, "Poziotinib for EGFR exon 20-insertion
NSCLC: Clinical efficacy of the phase 2 ZENITH trial and differential impact of EGFR exon 20
insertion location on sensitivity").
Run: python recompute.py.txt (downloads the two supplementary files from the publisher)
python recompute.py.txt SRC.zip SD1.xlsx (use local copies instead)
Needs numpy, scipy, pandas (+ openpyxl) and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file. Nothing else is written.
Inputs (the paper's own supplementary files, as published by Springer Nature):
MOESM7 Source Data (ZIP) -> "Source Data 2025.6.13/Fig 5.xlsx", sheet "Fig 5a-c":
one row per patient: Location (Near loop / Far loop), PFS (months), PFS_status.
MOESM3 Supplementary Data 1 (XLSX), sheets "Near 64" and "Far 20", with a column headed
"Censor, 0=PD, 1=censor" and a PFS column.
Patient identifiers in these files are used only to match rows between the two files;
none is printed.
Reported values (copied from the paper as archived in PMC):
Abstract: "progression-free survival (PFS, 11.1 vs. 3.5 months, p = 0.016)"
Results: "5.5 months (near) vs. 3.5 months (far) (Log Rank p = 0.015), with a PFS hazard
ratio of 0.42 (near- vs. far- 95% Confidence Interval [CI] 0.21-0.86, Fig. 5c)"
Discussion: "(11.1 months [near] vs. 3.5 months [far])"
Results: all 115 treated patients, median PFS 4.2 months (95% CI, 3.7 to 6.6)
Results: 63 near-loop and 19 far-loop patients with known insertion location
"""
import hashlib
import io
import os
import sys
import urllib.request
import zipfile
import numpy as np
import pandas as pd
from scipy.stats import chi2, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ESM = ("https://static-content.springer.com/esm/art%3A10.1038%2Fs41467-025-61817-8/"
"MediaObjects/41467_2025_61817_{}_ESM.{}")
URL_SRC = ESM.format("MOESM7", "zip")
URL_SD1 = ESM.format("MOESM3", "xlsx")
REP = dict(abs_near=11.1, abs_far=3.5, abs_p=0.016,
res_near=5.5, res_far=3.5, res_p=0.015, res_hr=0.42, res_lo=0.21, res_hi=0.86,
all_med=4.2, all_lo=3.7, all_hi=6.6, n_near=63, n_far=19)
def get(path_or_url):
if os.path.exists(path_or_url):
with open(path_or_url, "rb") as f:
return f.read()
req = urllib.request.Request(path_or_url, headers={"User-Agent": "Mozilla/5.0"})
with urllib.request.urlopen(req, timeout=120) as r:
return r.read()
src_bytes = get(sys.argv[1] if len(sys.argv) > 1 else URL_SRC)
sd1_bytes = get(sys.argv[2] if len(sys.argv) > 2 else URL_SD1)
print("Source Data ZIP sha256:", hashlib.sha256(src_bytes).hexdigest())
print("Supplementary Data 1 sha256:", hashlib.sha256(sd1_bytes).hexdigest())
zf = zipfile.ZipFile(io.BytesIO(src_bytes))
name = [n for n in zf.namelist() if n.endswith("Fig 5.xlsx") and "__MACOSX" not in n][0]
fig5 = pd.read_excel(io.BytesIO(zf.read(name)), sheet_name="Fig 5a-c")
sd1_sheets = pd.read_excel(io.BytesIO(sd1_bytes), sheet_name=None)
# ---------------------------------------------------------------- survival tools
def km(t, e):
"""Kaplan-Meier step function: event times and S just after each."""
t = np.asarray(t, float); e = np.asarray(e, int)
S, ts, ss = 1.0, [0.0], [1.0]
for u in np.unique(t[e == 1]):
n = (t >= u).sum(); d = ((t == u) & (e == 1)).sum()
S *= 1 - d / n; ts.append(u); ss.append(S)
return np.array(ts), np.array(ss)
def km_median(t, e):
ts, ss = km(t, e)
idx = np.where(ss <= 0.5 + 1e-12)[0]
return ts[idx[0]] if len(idx) else np.nan
def logrank(t1, e1, t2, e2):
"""Mantel-Cox log-rank: returns p and the Mantel-Haenszel HR (group 1 vs group 2)."""
t = np.r_[t1, t2]; e = np.r_[e1, e2]; g = np.r_[np.ones(len(t1)), np.zeros(len(t2))]
O = E = V = 0.0
for u in np.unique(t[e == 1]):
r = t >= u; n = r.sum(); n1 = (r & (g == 1)).sum()
d = ((t == u) & (e == 1)).sum(); d1 = ((t == u) & (e == 1) & (g == 1)).sum()
O += d1; E += d * n1 / n
if n > 1:
V += d * (n1 / n) * (1 - n1 / n) * (n - d) / (n - 1)
p = chi2.sf((O - E) ** 2 / V, 1)
b = (O - E) / V
z = norm.ppf(0.975)
O2, E2 = e.sum() - O, e.sum() - E
lr = (O / E) / (O2 / E2); se = np.sqrt(1 / E + 1 / E2) # "logrank" HR (O/E ratio)
return (p, np.exp(b), np.exp(b - z / np.sqrt(V)), np.exp(b + z / np.sqrt(V)),
lr, lr * np.exp(-z * se), lr * np.exp(z * se))
def cox_efron(t, e, x):
"""One-covariate Cox model, Efron ties, Newton-Raphson. Returns HR and 95% CI."""
t = np.asarray(t, float); e = np.asarray(e, int); x = np.asarray(x, float)
b = 0.0
for _ in range(50):
U = I = 0.0
for u in np.unique(t[e == 1]):
r = t >= u; D = (t == u) & (e == 1); d = D.sum()
w = np.exp(b * x)
s0, s1, s2 = w[r].sum(), (w * x)[r].sum(), (w * x * x)[r].sum()
d0, d1, d2 = w[D].sum(), (w * x)[D].sum(), (w * x * x)[D].sum()
U += x[D].sum()
for k in range(d):
f = k / d
a0, a1, a2 = s0 - f * d0, s1 - f * d1, s2 - f * d2
U -= a1 / a0
I += a2 / a0 - (a1 / a0) ** 2
step = U / I; b += step
if abs(step) < 1e-10:
break
se = 1 / np.sqrt(I); z = norm.ppf(0.975)
return np.exp(b), np.exp(b - z * se), np.exp(b + z * se)
# ---------------------------------------------------------------- 1. data checks
a = fig5.copy()
a["PFS"] = a["PFS"].astype(float)
near = a[a.Location == "Near loop"]; far = a[a.Location == "Far loop"]
print("\n=== 1. Source Data, sheet 'Fig 5a-c' ===")
print(f"near-loop n = {len(near)}, far-loop n = {len(far)} (paper: {REP['n_near']} and {REP['n_far']})")
print("PFS_status counts, near:", near.PFS_status.value_counts().sort_index().to_dict(),
" far:", far.PFS_status.value_counts().sort_index().to_dict())
ct = pd.crosstab(a.BOR, a.PFS_status)
print("best overall response x PFS_status:\n" + ct.to_string())
n_pd = int((a.BOR == "PD").sum()); n_pd1 = int(((a.BOR == "PD") & (a.PFS_status == 1)).sum())
print(f"patients with best response PD: {n_pd}; of these with PFS_status = 1: {n_pd1}")
# match to Supplementary Data 1 ("Censor, 0=PD, 1=censor")
rows = []
for sh, v in sd1_sheets.items():
v = v.iloc[:, :7].copy()
v.columns = ["id", "mut", "bor", "chg", "cens", "pfs", "dor"]
v["sheet"] = sh
rows.append(v)
sd1 = pd.concat(rows, ignore_index=True)
hdr = [c for c in list(sd1_sheets.values())[0].columns if "Censor" in str(c)][0]
print(f"Supplementary Data 1 censor column header: '{hdr}'")
m = a.merge(sd1, left_on="Subject ID", right_on="id", how="inner")
x = pd.crosstab(m.PFS_status, m.cens)
print(f"patients in both files: {len(m)}; PFS identical in {int(np.isclose(m.PFS, m.pfs.astype(float)).sum())}")
print("Source Data PFS_status (rows) x Supp. Data 1 censor code (cols):\n" + x.to_string())
print(f"Supp. Data 1 sheet sizes: " + ", ".join(f"{k} = {len(v)}" for k, v in sd1_sheets.items()))
print(f"only in Source Data: {len(a) - len(m)}; only in Supp. Data 1: {len(sd1) - len(m)}")
n12 = int((near.PFS >= 12).sum())
print(f"near-loop patients with PFS >= 12 months: {n12} of {len(near)}")
# ---------------------------------------------------------------- 2. the two codings
print("\n=== 2. Kaplan-Meier medians, log-rank p, hazard ratios (near vs far) ===")
R = {}
for key, lab, code in [("as", "PFS_status = 1 counted as the event (as coded; matches Supp. Data 1 '0=PD')", 1),
("rev", "PFS_status = 0 counted as the event (indicator reversed)", 0)]:
en = (near.PFS_status == code).astype(int).values
ef = (far.PFS_status == code).astype(int).values
mn, mf = km_median(near.PFS, en), km_median(far.PFS, ef)
p, mh, mhlo, mhhi, lr, lrlo, lrhi = logrank(near.PFS.values, en, far.PFS.values, ef)
hr, lo, hi = cox_efron(np.r_[near.PFS, far.PFS], np.r_[en, ef],
np.r_[np.ones(len(near)), np.zeros(len(far))])
allm = km_median(a.PFS, (a.PFS_status == code).astype(int))
R[key] = dict(mn=mn, mf=mf, p=p, hr=hr, lo=lo, hi=hi, mh=mh, mhlo=mhlo, mhhi=mhhi,
ev_n=int(en.sum()), ev_f=int(ef.sum()), allm=allm)
print(f"{lab}:")
print(f" events near {en.sum()}/{len(en)}, far {ef.sum()}/{len(ef)}")
print(f" median PFS near {mn:.2f} mo, far {mf:.2f} mo, difference {mn - mf:.2f} mo")
print(f" log-rank p = {p:.4f}")
print(f" Cox (Efron) HR near vs far = {hr:.2f} (95% CI {lo:.2f}-{hi:.2f})")
print(f" Mantel-Haenszel HR near vs far = {mh:.2f} (95% CI {mhlo:.2f}-{mhhi:.2f})")
print(f" O/E-ratio (logrank) HR near vs far = {lr:.2f} (95% CI {lrlo:.2f}-{lrhi:.2f})")
print(f" pooled KM median of all {len(a)} patients = {allm:.2f} mo "
f"(paper, all 115 treated: {REP['all_med']} [{REP['all_lo']}-{REP['all_hi']}])")
print("reported, Results/Fig. 5c: 5.5 vs 3.5, p = 0.015, HR 0.42 (0.21-0.86)")
print("reported, Abstract/Discussion: 11.1 vs 3.5, p = 0.016")
# Supplementary Data 1 as a sensitivity check (its own coding: 0 = PD)
sn = sd1[sd1.sheet.str.startswith("Near")]; sf = sd1[sd1.sheet.str.startswith("Far")]
tn, en1 = sn.pfs.astype(float).values, (sn.cens == 0).astype(int).values
tf, ef1 = sf.pfs.astype(float).values, (sf.cens == 0).astype(int).values
p1 = logrank(tn, en1, tf, ef1)[0]
SD1 = dict(mn=km_median(tn, en1), mf=km_median(tf, ef1), p=p1)
print(f"Supplementary Data 1 (0 = PD as event): near {SD1['mn']:.2f} mo (n = {len(sn)}), "
f"far {SD1['mf']:.2f} mo (n = {len(sf)}), log-rank p = {p1:.4f}")
# ---------------------------------------------------------------- 3. simple arithmetic
print("\n=== 3. Arithmetic on the reported values ===")
print(f"11.1 - 5.5 = {REP['abs_near'] - REP['res_near']:.1f} months; ratio 11.1/5.5 = {REP['abs_near'] / REP['res_near']:.2f}")
print(f"near-minus-far gap: abstract {REP['abs_near'] - REP['abs_far']:.1f} mo, Results {REP['res_near'] - REP['res_far']:.1f} mo")
print(f"median ratio far/near (exponential HR proxy): 3.5/5.5 = {3.5 / 5.5:.2f}, 3.5/11.1 = {3.5 / 11.1:.2f}")
# ---------------------------------------------------------------- figures
ORANGE, BLUE, INK, SEC, BG, GRID = "#eb6834", "#2a78d6", "#0b0b0b", "#52514e", "#fcfcfb", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none"})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# fig1: abstract/discussion vs results, two text panels and a connector
fig = plt.figure(figsize=(7.2, 4.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.0); ax.axis("off")
def panel(x, y, w, h, title, lines):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=SEC, lw=1.2))
ax.text(x + 0.15, y + h - 0.2, title, fontsize=11.5, weight="bold", va="top")
yy = y + h - 0.62
for segs in lines:
xx = x + 0.15
for txt, col, wt in segs:
t = ax.text(xx, yy, txt, color=col, fontsize=10.5, weight=wt, va="top")
fig.canvas.draw()
bb = t.get_window_extent(renderer=fig.canvas.get_renderer())
xx += bb.width / fig.dpi
yy -= 0.33
panel(0.2, 1.55, 3.05, 2.25, "Abstract and Discussion",
[[("Near- vs far-loop insertions, ZENITH20", SEC, "normal")],
[("median PFS", SEC, "normal")],
[("11.1", ORANGE, "bold"), (" vs 3.5 months", INK, "normal")],
[("p = 0.016", ORANGE, "bold")],
[("(no hazard ratio given)", SEC, "normal")]])
panel(3.95, 1.55, 3.05, 2.25, "Results (cites Fig. 5c)",
[[("Near- vs far-loop insertions, ZENITH20", SEC, "normal")],
[("median PFS", SEC, "normal")],
[("5.5", ORANGE, "bold"), (" vs 3.5 months", INK, "normal")],
[("log-rank p = 0.015", ORANGE, "bold")],
[("HR 0.42 (95% CI 0.21-0.86)", INK, "normal")]])
ax.annotate("", xy=(3.93, 2.68), xytext=(3.27, 2.68),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(3.6, 2.85, "conflict", ha="center", fontsize=10, color=INK, weight="bold")
ax.text(3.6, 1.18, "Same comparison in all three places: 63 near-loop vs 19 far-loop patients;\n"
"far-loop median 3.5 months everywhere. The near-loop median differs by "
f"{REP['abs_near'] - REP['res_near']:.1f} months\n(11.1 vs 5.5), and the p-value by 0.001.",
ha="center", va="top", fontsize=10, color=INK, linespacing=1.45)
ax.text(0.2, 0.2, "Orange: values as printed in the paper (PMC12460621).", fontsize=10, color=SEC)
save(fig, "fig1")
# fig2: dot plot of medians, reported vs recomputed from the Source Data
rows = [("Abstract & Discussion\n(reported)", REP["abs_near"], REP["abs_far"], ORANGE,
f"p = {REP['abs_p']:.3f}"),
("Source Data, indicator\nreversed (0 = event)", R["rev"]["mn"], R["rev"]["mf"], BLUE,
f"p = {R['rev']['p']:.4f}\nHR {R['rev']['hr']:.2f}"),
("Results & Fig. 5c\n(reported)", REP["res_near"], REP["res_far"], ORANGE,
f"p = {REP['res_p']:.3f}\nHR {REP['res_hr']:.2f}"),
("Source Data, as coded\n(1 = progression)", R["as"]["mn"], R["as"]["mf"], BLUE,
f"p = {R['as']['p']:.4f}\nHR {R['as']['hr']:.2f}")]
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.84, top=0.86, bottom=0.15)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1]
for y, (lab, n_, f_, col, note) in zip(ys, rows):
ax.plot([f_, n_], [y, y], color=col, lw=2, zorder=2)
ax.plot(n_, y, "o", ms=10, color=col, zorder=3)
ax.plot(f_, y, "s", ms=9, mfc=BG, mec=col, mew=2, zorder=3)
ax.text(n_, y + 0.22, f"{n_:.2f}".rstrip("0").rstrip(".") if col == BLUE else f"{n_}",
ha="center", va="bottom", fontsize=10, color=col, weight="bold")
ax.text(f_, y + 0.22, f"{f_:.2f}" if col == BLUE else f"{f_}",
ha="center", va="bottom", fontsize=10, color=col)
ax.text(14.6, y, note, ha="left", va="center", fontsize=10, color=col, clip_on=False)
ax.axhline(1.5, color=SEC, lw=0.8, ls=(0, (3, 3)))
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlim(0, 14); ax.set_ylim(-0.6, len(rows) - 0.25)
ax.set_xticks(range(0, 15, 2))
ax.set_xlabel("Median progression-free survival (months)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=SEC, label="near-loop (n = 63)"),
Line2D([], [], marker="s", ls="", ms=8, mfc=BG, mec=SEC, mew=2, label="far-loop (n = 19)"),
Line2D([], [], color=ORANGE, lw=2, label="reported in the paper"),
Line2D([], [], color=BLUE, lw=2, label="recomputed from the paper's Source Data")]
fig.legend(handles=handles, loc="upper center", ncol=2, frameon=False, fontsize=10,
bbox_to_anchor=(0.55, 1.0))
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Reported Fisher's exact p < 0.01 for 18/20 versus 3/6 patent grafts recomputes to 0.062
Nat Commun (2025) · doi:10.1038/s41467-025-68075-8 · PMC12868887
Summary
In the rabbit carotid model, the Results compare 2-week graft patency between the new micro-nano structured grafts (MnVGs, 18 of 20 patent) and electrospinning-only control grafts (3 of 6 patent), and give "p < 0.01, Fisher’s exact test". Fisher's exact test on these counts gives p = 0.062, two-sided and one-sided alike. This is above the paper's own significance level of 0.05. None of the usual alternative tests for a 2 × 2 table gets below 0.01 either; the smallest, a one-sided two-proportion z-test, gives 0.015. The comparison is the basis for the paragraph's conclusion that MnVGs "significantly outperformed conventional electrospinning grafts". The counts show a large observed difference (90% vs 50%), but with six control grafts the stated test does not reach significance.
What the paper reports
Results, section "MnVGs maintained high patency rates and rapid neo-adventitia Ingrowth", first paragraph:
15 mm segmental defect was surgically created in the left carotid artery of each rabbit, with MnVGs implanted in the experimental group (n = 20) and electrospinning grafts as controls (n = 6) (Fig. 2a, b). At 2 weeks after implantation, graft performance diverged markedly between groups.
Doppler ultrasound revealed a 90% patency rate (18/20) in the MnVGs group whereas controls exhibited 50% patency (3/6) and significant regurgitation despite of administration of anticoagulating agents (p < 0.01, Fisher’s exact test).
The last sentence of the same paragraph:
Collectively, these results demonstrated that MnVGs significantly outperformed conventional electrospinning grafts, with structural openness critically enhancing patency and regenerative efficiency in 2 weeks.
Abstract:
When implanted in rabbits, MnVGs rapidly regenerated neo-adventitia within the highly porous graft walls and achieved endothelialization as early as 2 weeks postoperatively, resulting in 90% patency rate (18/20).
Discussion, fourth paragraph:
The key finding of this study is that MnVGs achieved rapid endothelialization within 2 weeks postoperatively, resulting in a 90% patency rate (18/20) in 1.5 cm carotid artery reconstruction in rabbits.
Methods, "Animal experiments":
Male New Zealand white rabbits (6 months old, weighing 3.5-4.5 kg; n = 32) underwent implantation of 15 mm long MnVGs interposition grafts in the left carotid artery. The experimental cohorts included a control group (Con-2W; n = 6) and a test group (MnVGs; n = 26).
Methods, "Statistical analysis":
Group comparisons were assessed using the Student’s t-test for two groups and one-way analysis of variance (ANOVA) followed by Tukey’s post-hoc test for multiple groups. Statistical significance was set at P < 0.05.
What the paper's own numbers show
Method. I built the 2 × 2 table from the counts in the Results sentence: MnVGs 18 patent and 2 occluded, controls 3 patent and 3 occluded. The Source Data file (sheet "Figure 2", "Figure 2e Number of graft") gives the same counts: 2 weeks, 18 patent and 2 occluded; con-2 weeks, 3 patent and 3 occluded. I computed Fisher's exact test by hand from the hypergeometric distribution and with scipy.stats.fisher_exact. I then computed every common alternative for a 2 × 2 table, the odds ratio with its exact confidence interval, and how much the counts would have to change for Fisher's p to fall below 0.01. Every number below is printed by recompute.py.txt.
With margins fixed (21 patent and 5 occluded among 26 grafts, 20 of them MnVGs), the number of patent MnVGs can only be 15 to 20. The observed 18 has probability 13,300/230,230 = 0.0578. Only 19 (0.0046) and 20 (0.0001) are as extreme or more so, so the p-value is (13,300 + 1,050 + 21)/230,230 = 14,371/230,230 = 0.0624. The opposite tail adds nothing, because every table with 15 to 17 patent MnVGs is more probable than the observed one. That is why the one-sided and two-sided p are the same.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| MnVG patency | Results; Abstract; Discussion | 90% (18/20) | 18/20 = 90.0% (consistent) |
| Control patency | Results | 50% (3/6) | 3/6 = 50.0% (consistent) |
| Fisher's exact p, two-sided | Results | p < 0.01 | 0.0624 |
| Fisher's exact p, one-sided | (alternative convention) | p < 0.01 | 0.0624 |
| Fisher mid-p, one-sided (doubled for two-sided) | (alternative convention) | p < 0.01 | 0.0335 (0.0671) |
| Pearson chi-square, uncorrected / Yates-corrected | (alternative test) | p < 0.01 | 0.0292 / 0.1119 |
| Two-proportion z-test (z = 2.18), two-sided / one-sided | (alternative test) | p < 0.01 | 0.0292 / 0.0146 |
| Barnard exact, two-sided | (alternative test) | p < 0.01 | 0.0384 |
| Boschloo exact, two-sided / one-sided | (alternative test) | p < 0.01 | 0.0593 / 0.0296 |
| Odds ratio (sample; conditional MLE with exact 95% CI) | not reported | not reported | 9.00; 7.97 (0.64 to 136.7) |
| Difference in patency (Newcombe 95% CI) | not reported | not reported | 0.40 (0.029 to 0.721) |
| MnVG count needed for Fisher p < 0.01, with controls at 3/6 | sensitivity | n/a | 20/20 (p = 0.0077); 19/20 gives 0.0278 |
| Control count needed for Fisher p < 0.01, with MnVGs at 18/20 | sensitivity | n/a | 1/6 or fewer (1/6: p = 0.0018); 2/6 gives 0.0129 |
| MnVGs out of the Methods' n = 26, vs 3/6 | Methods, "Animal experiments" | n/a | 24/26: p = 0.0342; 25/26: 0.0149; only 26/26 gives p < 0.01 (0.0040) |
The stated test gives p = 0.062, about six times the printed bound and above the paper's α of 0.05. The exact 95% confidence interval of the odds ratio (0.64 to 136.7) includes 1, which agrees with that test. The smallest p-value of any variant is 0.015, from a one-sided large-sample z-test. That test is not the one reported, and its normal approximation is poor with three occluded grafts among six. Some of the less conservative methods (uncorrected chi-square, mid-p, Barnard, the Newcombe interval for the difference) put the result just below 0.05. None of them supports p < 0.01.
Figure
Figure 1. P-values for the 2-week patency comparison (18 of 20 MnVGs patent vs 3 of 6 controls), on one log-scale axis. The orange arrow is the value printed in the Results, "p < 0.01, Fisher’s exact test", drawn as the interval from 0.01 downward. The blue markers are p-values recomputed from the paper's counts. The filled marker is the test the paper names, Fisher's exact test, two-sided (0.062); open markers are common alternatives. The dotted line marks 0.01 and the dashed line marks the paper's significance level of 0.05. Every recomputed value lies to the right of the printed bound. The log scale is used because p-values span orders of magnitude; it does not exaggerate the gap, which is a factor of about 6 between 0.01 and 0.062.
Figure 2. Fisher's exact two-sided p, recomputed for each possible number of patent MnVGs out of 20, with the controls held at the reported 3 of 6. The reported counts (18/20, filled marker) give 0.062. Only 20/20 would give p < 0.01 (0.008, orange line), and only 19/20 or more would give p < 0.05. Holding the MnVGs at 18/20 instead, the controls would need 1 of 6 patent or fewer (see table).
Why it matters
- The comparative claim rests on this test. The paragraph sets out to "evaluate the impact of graft wall openness" by comparing MnVGs with dense-walled electrospinning-only grafts, and concludes that MnVGs "significantly outperformed conventional electrospinning grafts, with structural openness critically enhancing patency". The only statistic given for patency is this Fisher test. At the stated test's p = 0.062 the difference is not significant at the paper's own α = 0.05, let alone at 0.01.
- The direction of the effect is not in question. 90% versus 50% patency favors MnVGs, and the counts themselves are reported consistently in the text and the Source Data. What changes is the strength of evidence. With six control grafts the estimate is imprecise: the exact 95% interval for the odds ratio runs from 0.64 to 137. The result is better read as suggestive than as established.
- The descriptive and mechanistic findings are not affected. The 90% (18/20) patency figure in the abstract, introduction and Discussion is a correct description of the MnVG group. The endothelialization, cell-coverage and lymphatic endothelial cell results rest on other measurements and tests. Later sentences that explain the "superior performance of MnVGs, as evidenced by their high patency rate" by rapid endothelialization inherit the uncertainty of the patency comparison, however.
Explanations considered
- One-sided instead of two-sided test. Does not explain it. For this table the one-sided Fisher p is also 0.0624, because the lower tail contributes no table at least as improbable as the observed one.
- A different test or convention: mid-p, chi-square with or without Yates correction, z-test, Barnard or Boschloo. Does not explain it. These give 0.015 to 0.112 (Figure 1, table). The smallest, 0.015, is a one-sided large-sample z-test, not the exact test the paper names, and it is still above 0.01.
- A different denominator. Does not explain it. The Results give n = 20 MnVGs; the Methods give n = 26 MnVG rabbits and 6 controls (n = 32). The Source Data for Fig. 2e give 18 patent and 2 occluded at 2 weeks, which matches n = 20. If the 26 were the denominator, 24/26 would give p = 0.034 and 25/26 p = 0.015. Only 26/26 would give p < 0.01, and that would contradict the stated 18/20. The difference between n = 20 and n = 26 is not explained in the text; it does not affect the conclusion here.
- A different time point or pooled time points. Does not explain it. The sentence ties the test to the 2-week Doppler result, and the control group exists only at 2 weeks (Con-2W). The Source Data list MnVG counts at 4 weeks (12 patent, 0 occluded) and 12 weeks (6 patent, 0 occluded), but these later time points have no control group, so they cannot enter a 2-week comparison with the controls.
- Adjusted or multiplicity-corrected p. Does not explain it. No adjustment is described for this test, and any adjustment would increase p, not reduce it.
- A text-conversion artifact. Ruled out. The publisher's HTML, the Europe PMC full-text XML and the PMC page all read "p < 0.01, Fisher’s exact test" with the same counts.
- A correction already issued. None found (see Status).
- Note on the screening claim. The automated screen that flagged this paper put Fisher's p at "around 0.04". That is not accurate. The two-sided Fisher p is 0.062. Values of 0.03 to 0.04 come only from the mid-p, uncorrected chi-square or Barnard tests. The finding holds, and is somewhat stronger than first stated.
- Context, not part of the finding. The rat experiment reports 100% (6/6) patency in controls versus 50% (3/6) with a VEGFR3 inhibitor, without a p-value in the text. For reference, Fisher's exact two-sided p for that table is 0.18 (one-sided 0.091). The Discussion's statement that LEC inhibition "diminishes patency" is therefore also based on a difference that the exact test does not find significant at n = 6 per group.
Status
- Version of record. The publisher's article page on nature.com ("Open access, Published: 29 December 2025") has the same Results sentence, with "(p < 0.01, Fisher’s exact test)", the same counts, the same Methods group sizes (Con-2W n = 6, MnVGs n = 26) and the same Statistical analysis section (t-test, ANOVA with Tukey, P < 0.05). The Europe PMC full-text XML and the PMC page agree. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The publisher lists nine supplementary files. The Supplementary Information PDF (MOESM1, 19 pages) has no patency statistics or Fisher test. The Description of Additional Supplementary Files (MOESM2) and the Reporting Summary (MOESM7) contain nothing relevant. The Transparent Peer Review file (MOESM8) mentions "90% patency" but not the p-value. The Source Data workbook (MOESM9, sheet "Figure 2") confirms the 2 × 2 table 18/2 versus 3/3 at 2 weeks, so the supplement confirms the discrepancy rather than resolving it. MOESM3 to MOESM6 are videos and were not opened.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by fields, and the publisher's article page shows no notice. The Europe PMC search service returned HTTP 503 ("temporarily unavailable") on repeated attempts, including on 2026-10-24, so the Europe PMC search could not be completed. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" or "correction" found no thread and no notice. - Sources. PMC article PMC12868887; publisher article page https://www.nature.com/articles/s41467-025-68075-8 and its supplementary files MOESM1 to MOESM9; Europe PMC full-text XML; Crossref; PubPeer; https://doi.org/10.1038/s41467-025-68075-8.
Suggested correction
In Results, section "MnVGs maintained high patency rates and rapid neo-adventitia Ingrowth", replace
(p < 0.01, Fisher’s exact test)
with
(p = 0.062, two-sided Fisher’s exact test)
and revise the paragraph's last sentence so that it does not describe the patency difference as significant. For example:
Collectively, these results suggested that MnVGs outperformed conventional electrospinning grafts in 2-week patency (90% vs 50%), although the difference did not reach statistical significance with six control grafts.
The authors may also wish to state the number of MnVG rabbits consistently between the Results (n = 20) and the Methods (n = 26), and to add Fisher's exact test to the Statistical analysis section, which currently lists only t-tests and ANOVA.
Recomputation script (python recompute.py.txt)
"""Recompute the graft-patency test of PMC12868887
(Nat Commun 2025, doi:10.1038/s41467-025-68075-8,
"Rapid endothelialization of printed vascular grafts by perivascular
niche-circulating endothelial progenitors crosstalk").
Run: python recompute.py.txt
Needs numpy, scipy (>= 1.10) and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Results, "MnVGs maintained high patency rates and rapid neo-adventitia Ingrowth":
MnVGs n = 20, electrospinning-only controls n = 6;
"90% patency rate (18/20) in the MnVGs group whereas controls exhibited 50% patency (3/6)
... (p < 0.01, Fisher's exact test)".
Methods, "Animal experiments": rabbits n = 32; Con-2W n = 6; MnVGs n = 26.
Methods, "Statistical analysis": "Statistical significance was set at P < 0.05."
Results (rat model, context only): "100% (6/6) patency rate in the control group compared to
only 50% (3/6) in the inhibitor group" (no p-value is given for it in the text).
Source Data (MOESM9), sheet "Figure 2", "Figure 2e Number of graft": 2 weeks 18 patent / 2
occluded; con-2 weeks 3 patent / 3 occluded.
"""
import os
import numpy as np
from scipy import stats
from scipy.stats.contingency import odds_ratio
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
A, N1 = 18, 20 # MnVG patent / implanted (2 weeks)
C, N2 = 3, 6 # electrospinning-only control patent / implanted (2 weeks)
REPORTED_BOUND = 0.01
ALPHA = 0.05
def table(a, n1, c, n2):
return np.array([[a, n1 - a], [c, n2 - c]])
def all_p(a, n1, c, n2, label):
t = table(a, n1, c, n2)
out = {}
_, out["fisher2"] = stats.fisher_exact(t, alternative="two-sided")
_, out["fisher1"] = stats.fisher_exact(t, alternative="greater")
hg = stats.hypergeom(n1 + n2, a + c, n1) # X = patent grafts in the MnVG row
out["midp1"] = hg.sf(a) + 0.5 * hg.pmf(a)
out["midp2"] = 2 * out["midp1"]
out["chi_yates"] = stats.chi2_contingency(t, correction=True)[1]
out["chi_plain"] = stats.chi2_contingency(t, correction=False)[1]
p1, p2 = a / n1, c / n2
pp = (a + c) / (n1 + n2)
z = (p1 - p2) / np.sqrt(pp * (1 - pp) * (1 / n1 + 1 / n2))
out["z"] = z
out["z2"] = 2 * stats.norm.sf(abs(z))
out["z1"] = stats.norm.sf(z)
out["barnard2"] = stats.barnard_exact(t, alternative="two-sided").pvalue
out["boschloo2"] = stats.boschloo_exact(t, alternative="two-sided").pvalue
out["boschloo1"] = stats.boschloo_exact(t, alternative="greater").pvalue
print(f"{label}: {a}/{n1} ({a/n1:.1%}) vs {c}/{n2} ({c/n2:.1%}); table {t.tolist()}")
print(f" Fisher exact, two-sided p = {out['fisher2']:.4f}")
print(f" Fisher exact, one-sided p = {out['fisher1']:.4f}")
print(f" Fisher mid-p, one-sided p = {out['midp1']:.4f} (two-sided, doubled: {out['midp2']:.4f})")
print(f" Chi-square, Yates-corrected p = {out['chi_yates']:.4f}")
print(f" Chi-square, uncorrected p = {out['chi_plain']:.4f}")
print(f" Two-proportion z = {z:.3f}: two-sided p = {out['z2']:.4f}, one-sided p = {out['z1']:.4f}")
print(f" Barnard exact, two-sided p = {out['barnard2']:.4f}")
print(f" Boschloo exact, two-sided p = {out['boschloo2']:.4f}; one-sided p = {out['boschloo1']:.4f}")
return out
# ------------------------------------------------------------------ 1. percentages
print("=== 1. Percentages ===")
print(f"MnVG {A}/{N1} = {A/N1:.3f}; control {C}/{N2} = {C/N2:.3f}; difference = {A/N1 - C/N2:.3f}")
# ------------------------------------------------------------------ 2. hypergeometric detail
print()
print("=== 2. Fisher's exact test by hand (hypergeometric, margins fixed) ===")
from math import comb # noqa: E402
tot_pat, tot = A + C, N1 + N2
denom = comb(tot, N1)
print(f"margins: patent {tot_pat}, occluded {tot - tot_pat}, N = {tot}; C({tot},{N1}) = {denom}")
probs = {}
for k in range(max(0, N1 - (tot - tot_pat)), min(N1, tot_pat) + 1):
w = comb(tot_pat, k) * comb(tot - tot_pat, N1 - k)
probs[k] = w
print(f" X = {k:2d} MnVG patent: weight {w:6d}, P = {w/denom:.5f}")
p_obs = probs[A]
two = sum(w for w in probs.values() if w <= p_obs) / denom
print(f"observed X = {A}: P = {p_obs/denom:.5f}; two-sided p = sum of P <= observed = "
f"{sum(w for w in probs.values() if w <= p_obs)}/{denom} = {two:.4f}")
print(f"one-sided P(X >= {A}) = {sum(w for k, w in probs.items() if k >= A)}/{denom} = "
f"{sum(w for k, w in probs.items() if k >= A)/denom:.4f}")
print("tables in the lower tail (X <= 14) are impossible here, so one- and two-sided p coincide")
# ------------------------------------------------------------------ 3. all tests
print()
print("=== 3. The stated test and every common alternative ===")
P = all_p(A, N1, C, N2, "Rabbit 2-week patency, MnVG vs electrospinning-only")
smallest = min(v for k, v in P.items() if k != "z")
print(f"smallest p over all variants = {smallest:.4f}; any below {REPORTED_BOUND}? {smallest < REPORTED_BOUND}")
print(f"Fisher two-sided below the paper's alpha {ALPHA}? {P['fisher2'] < ALPHA}")
# ------------------------------------------------------------------ 4. effect size
print()
print("=== 4. Effect size and its uncertainty ===")
t = table(A, N1, C, N2)
sample_or = (A * (N2 - C)) / ((N1 - A) * C)
res = odds_ratio(t, kind="conditional")
ci = res.confidence_interval(confidence_level=0.95)
print(f"sample odds ratio = {sample_or:.2f}; conditional MLE OR = {res.statistic:.2f}, "
f"exact 95% CI {ci.low:.2f} to {ci.high:.2f}")
def wilson(x, n, z=1.959963984540054):
p = x / n
d = 1 + z * z / n
c = (p + z * z / (2 * n)) / d
h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
return c - h, c + h
l1, u1 = wilson(A, N1)
l2, u2 = wilson(C, N2)
diff = A / N1 - C / N2
d_lo = diff - np.sqrt((A / N1 - l1) ** 2 + (u2 - C / N2) ** 2)
d_hi = diff + np.sqrt((u1 - A / N1) ** 2 + (C / N2 - l2) ** 2)
print(f"Wilson 95% CI MnVG {l1:.3f} to {u1:.3f}; control {l2:.3f} to {u2:.3f}")
print(f"difference {diff:.3f}, Newcombe (method 10) 95% CI {d_lo:.3f} to {d_hi:.3f}")
# ------------------------------------------------------------------ 5. other denominators
print()
print("=== 5. Alternative denominators and sensitivity ===")
print("Methods give MnVGs n = 26 (rabbits n = 32 with 6 controls):")
alt26 = {}
for a in range(20, 27):
alt26[a] = stats.fisher_exact(table(a, 26, C, N2))[1]
print(f" {a}/26 vs 3/6: Fisher two-sided p = {alt26[a]:.4f}")
print("MnVG patent count out of 20, control fixed at 3/6:")
sweep_mnvg = {}
for a in range(15, 21):
sweep_mnvg[a] = stats.fisher_exact(table(a, N1, C, N2))[1]
print(f" {a}/20 vs 3/6: Fisher two-sided p = {sweep_mnvg[a]:.4f}")
print("Control patent count out of 6, MnVG fixed at 18/20:")
sweep_ctrl = {}
for c in range(0, 7):
sweep_ctrl[c] = stats.fisher_exact(table(A, N1, c, N2))[1]
print(f" 18/20 vs {c}/6: Fisher two-sided p = {sweep_ctrl[c]:.4f}")
need = [a for a, p in sweep_mnvg.items() if p < REPORTED_BOUND]
needc = [c for c, p in sweep_ctrl.items() if p < REPORTED_BOUND]
print(f"p < 0.01 needs MnVG patent in {need} of 20, or control patent in {needc} of 6")
# ------------------------------------------------------------------ 6. rat context
print()
print("=== 6. Context: rat model, control vs VEGFR3 inhibitor (no p-value stated in text) ===")
R = all_p(6, 6, 3, 6, "Rat 2-week patency, control vs inhibitor")
# ------------------------------------------------------------------ figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans",
"text.color": FG, "axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"axes.edgecolor": FG2, "axes.linewidth": 0.8, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# Figure 1: reported bound vs recomputed p-values on one log axis
rows = [
("Reported: Fisher's exact test", None, "rep"),
("Fisher's exact, two-sided (stated test)", P["fisher2"], "main"),
("Fisher's exact, one-sided", P["fisher1"], "alt"),
("Fisher mid-p, one-sided", P["midp1"], "alt"),
("Chi-square, Yates-corrected", P["chi_yates"], "alt"),
("Chi-square, uncorrected", P["chi_plain"], "alt"),
("Barnard exact, two-sided", P["barnard2"], "alt"),
("Boschloo exact, two-sided", P["boschloo2"], "alt"),
]
XMIN, XMAX = 1e-3, 0.3
fig, ax = plt.subplots(figsize=(7.2, 3.9))
n = len(rows)
for i, (lab, val, kind) in enumerate(rows):
y = n - 1 - i
if kind == "rep":
ax.annotate("", xy=(XMIN * 1.08, y), xytext=(REPORTED_BOUND, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=14))
ax.plot([REPORTED_BOUND], [y], marker="|", ms=14, mew=2, color=ORANGE)
ax.text(REPORTED_BOUND * 1.12, y, "p < 0.01 (as printed)", va="center", ha="left",
color=ORANGE, fontsize=10.5, fontweight="bold")
else:
main = kind == "main"
ax.plot([val], [y], "o", ms=10 if main else 8, color=BLUE,
mfc=BLUE if main else BG, mew=2)
ax.text(val * 1.15, y, f"{val:.3f}", va="center", ha="left",
color=BLUE, fontsize=10.5, fontweight="bold" if main else "normal")
ax.axvline(REPORTED_BOUND, color=FG2, lw=0.8, ls=":", zorder=0)
ax.axvline(ALPHA, color=FG2, lw=0.8, ls="--", zorder=0)
ax.text(ALPHA, n - 0.35, "paper's α = 0.05", ha="center", va="bottom", color=FG2, fontsize=10)
ax.text(REPORTED_BOUND, n - 0.35, "0.01", ha="center", va="bottom", color=FG2, fontsize=10)
ax.set_yticks(range(n))
ax.set_yticklabels([r[0] for r in rows][::-1])
for tl, r in zip(ax.get_yticklabels(), rows[::-1]):
tl.set_color(ORANGE if r[2] == "rep" else FG)
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.set_ylim(-0.6, n - 0.1)
ax.set_xticks([0.001, 0.01, 0.05, 0.1, 0.3])
ax.set_xticklabels(["0.001", "0.01", "0.05", "0.1", "0.3"])
ax.set_xlabel("p-value for 18/20 vs 3/6 patent grafts (log scale)")
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.plot([], [], "o", color=BLUE, ms=8, mew=2, label="recomputed from the paper's counts (filled: the stated test)")
ax.plot([], [], color=ORANGE, lw=2, label="reported in the paper")
ax.legend(loc="lower left", bbox_to_anchor=(-0.02, -0.42), ncol=1, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: how many patent MnVG grafts would p < 0.01 require?
fig, ax = plt.subplots(figsize=(7.2, 3.2))
xs = list(sweep_mnvg)
ps = [sweep_mnvg[a] for a in xs]
ax.plot(xs, ps, color=FG2, lw=1, zorder=1)
for a, p in zip(xs, ps):
obs = a == A
ax.plot([a], [p], "o", ms=10 if obs else 8, color=BLUE, mfc=BLUE if obs else BG, mew=2, zorder=3)
below = a == 19 # keep this label clear of the alpha line
ax.text(a, p * (0.72 if below else 1.35), f"{p:.3f}", ha="center",
va="top" if below else "bottom",
color=BLUE, fontsize=10, fontweight="bold" if obs else "normal")
ax.axhline(ALPHA, color=FG2, lw=1, ls="--", zorder=2)
ax.axhline(REPORTED_BOUND, color=ORANGE, lw=2, zorder=2)
ax.text(14.6, REPORTED_BOUND * 0.78, "reported bound p < 0.01", color=ORANGE, fontsize=10.5,
va="top", ha="left", fontweight="bold")
ax.text(14.6, ALPHA * 0.82, "paper's α = 0.05", color=FG2, fontsize=10, va="top", ha="left")
ax.annotate("reported counts\n18/20 vs 3/6", xy=(A, sweep_mnvg[A]), xytext=(A + 0.75, 0.33),
color=FG, fontsize=10, ha="center",
arrowprops=dict(arrowstyle="-", color=FG2, lw=0.8))
ax.set_yscale("log")
ax.set_ylim(2e-3, 1.0)
ax.set_xlim(14.5, 20.5)
ax.set_xticks(xs)
ax.set_xticklabels([f"{a}/20" for a in xs])
ax.set_yticks([0.01, 0.05, 0.1, 0.3, 1.0])
ax.set_yticklabels(["0.01", "0.05", "0.1", "0.3", "1"])
ax.set_xlabel("Patent MnVG grafts (controls held at the reported 3/6)")
ax.set_ylabel("Fisher's exact p, two-sided")
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
save(fig, "fig2")
print()
print("figures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Abstract reports a 59.7% catalyst heat-duty reduction; the body reports 35.2% at the optimum
Nat Commun (2026) · doi:10.1038/s41467-026-69542-6 · PMC12966427
Summary
The abstract says that the optimal 1:1 iron-sulfated zirconia/silica (Fe-SZ/SiO2) catalyst "reduces relative heat duty by up to 59.7%" in the membrane vacuum regeneration (MVR) system. The Results and Discussion report something different for the same catalyst, solvent (3 M potassium glycinate, GlyK) and blank: a 35.2% reduction at 9 wt%, the loading the paper selects as optimal, and 40.3% at 12 wt%, the highest loading tested. Applying the paper's own Eq. 7 to its reported catalytic enhancements reproduces 35.2% and 40.3%. The same calculation shows that 59.7% is the relative heat duty that remains at 12 wt% (100/1.675 = 59.7%, i.e. 100 − 40.3), not a reduction. As written, the abstract's figure is 1.70 times the reduction at the stated optimum and 1.48 times the largest catalyst-only reduction in the paper. It is the abstract's only quantitative statement about the catalyst on its own.
What the paper reports
Abstract:
An optimal iron-sulfated zirconia to silica ratio of 1:1 reduces relative heat duty by up to 59.7% in the membrane vacuum system.
Finally, combining the optimized hybrid solvent, catalyst, and membrane system at 90 °C reduces thermal energy consumption by 66.8% relative to potassium glycinate, achieving an energy requirement of 2.6 GJ/tCO2.
Results, Composite solid acid catalyst › Catalytic support (SiO2 vs. Al2O3), Fig. 2g text (6 wt% 1:1 catalyst):
raising the total amount of desorbed CO2 from 244 to 307 mmol for 3 M GlyK
Results, Composite solid acid catalyst › Catalyst concentration and stability, paragraph 1 (the inputs):
the total desorbed CO2 increased from 244 to 377 mmol, corresponding to a 54.3% catalytic enhancement
the enhancement rising to 67.5%
Notably, the magnitude of improvement from 6 to 9 wt% (112%) was greater than that from 9 to 12 wt% (24.3%), suggesting a reduced catalytic effect beyond a certain concentration, indicating the presence of an optimum.
the use of 9 wt% catalyst resulted in the highest enhancement factor of 6.0 and was therefore selected as the optimal concentration for further evaluations.
Same section, paragraph 2 (text for Fig. 3g):
Accordingly, at 9 wt% and 12 wt%, the heat duty decreased by 35.2 and 40.3% as compared to the blank, respectively, reducing the thermal energy requirement from 7.8 to 5.1 and 4.7 GJ/tCO2.
Same section, paragraph 3 (stability at 9 wt%, Fig. S6b):
the RH% increased from 64.8% in cycle 1 to 68.8% in cycle 2
Results, Catalytic hybrid amino acid solvents in MVR (Fig. 5f text):
Considering GlyK as a benchmark, the catalyst reduced the RH of GlyK, SarK, and TauK to 64.8%, 90.1%, and 55.8%, respectively.
Notably, the combination of a hybrid solvent with the catalyst further reduced the RH to 33.2%, corresponding to a 66.8% reduction in energy consumption.
Discussion:
9 wt% fixed-bed catalyst was identified as the optimal concentration, achieving a 35.2% reduction in the system’s energy input.
Methods, MVR performance analysis (definition):
The relative heat duty, RH (%), was calculated using Eq. 7
where Eq. 7 is RH = Q_Cat / Q_Blank × 100.
All quotes were checked verbatim against the PMC-archived text (src/show.py; recompute.py.txt repeats the check). The string "59.7" occurs once in the archived article text, in the abstract.
What the paper's own numbers show
Method. The Methods define the catalytic enhancement CE = (M_cat − M_blank)/M_blank × 100 (Eq. 4), the thermal energy Q = m·cp·ΔT / (M_total_CO2·Mw) (Eq. 6) and RH = Q_cat/Q_blank × 100 (Eq. 7). In the catalyst-loading series the solvent (3 M GlyK), its mass and the temperature programme (90 °C) are the same with and without catalyst, so m, cp and ΔT cancel and RH = M_blank/M_cat × 100 = 100/(1 + CE/100). The heat-duty reduction is 100 − RH, and the thermal energy is 7.8 GJ/tCO2 (the blank) × RH/100. RH is computed from the reported CE values and, where the paper prints the desorbed amounts, also from the amounts in mmol. As a separate check, the reduction is recomputed from the reported, rounded energies, 1 − Q_cat/7.8. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| CE, 9 wt% 1:1 | Results, Catalyst concentration, para 1 | 54.3% (244 → 377 mmol) | 377/244 − 1 = 54.5% |
| CE, 6 wt% 1:1 | Results, Catalytic support | 25.6% (244 → 307 mmol) | 307/244 − 1 = 25.8% |
| Gain in CE, 6 → 9 wt% and 9 → 12 wt% | Results, para 1 | 112% and 24.3% | 112.1% and 24.3% |
| Enhancement factor CE/W | Results, para 1 | 6.0 at 9 wt% (highest) | 4.27 (6 wt%), 6.03 (9 wt%), 5.62 (12 wt%) |
| RH, 9 wt% (stated optimum) | Results, para 3; Fig. 5f text | 64.8% | 100/1.543 = 64.8% (from mmol: 64.7%) |
| Heat-duty reduction, 9 wt% | Results, para 2; Discussion | 35.2% | 100 − 64.8 = 35.2% (from mmol: 35.3%; from energies: 1 − 5.1/7.8 = 34.6%) |
| Thermal energy, 9 wt% | Results, para 2 | 5.1 GJ/tCO2 | 7.8 × 0.648 = 5.06 GJ/tCO2 |
| RH, 12 wt% (highest tested) | not stated in the text | — | 100/1.675 = 59.7% |
| Heat-duty reduction, 12 wt% | Results, para 2 | 40.3% | 100 − 59.7 = 40.3% (from energies: 1 − 4.7/7.8 = 39.7%) |
| Thermal energy, 12 wt% | Results, para 2 | 4.7 GJ/tCO2 | 7.8 × 0.597 = 4.66 GJ/tCO2 |
| Heat-duty reduction, 6 wt% 1:1 | not stated in the text | — | 100 − 100/1.256 = 20.4% (RH 79.6%, 6.21 GJ/tCO2) |
| Heat-duty reduction, "optimal" 1:1 catalyst | Abstract | 59.7% ("by up to") | 35.2% at the optimum; 40.3% at most (12 wt%) |
| What a 59.7% reduction would require | — | — | RH 40.3%, Q = 3.14 GJ/tCO2, CE = 148% (highest reported catalyst-only CE: 67.5%) |
| Abstract figure ÷ body figure | — | — | 59.7/35.2 = 1.70 (optimum); 59.7/40.3 = 1.48 (12 wt%) |
| Hybrid solvent + catalyst | Abstract; Results (Fig. 5f text) | RH 33.2%, 66.8% reduction, 2.6 GJ/tCO2 | 100 − 33.2 = 66.8%; 7.8 × 0.332 = 2.59 GJ/tCO2 |
The inputs in this section of the paper agree with one another to within rounding. The Results paragraph, the stability paragraph, the Fig. 5f text and the Discussion all agree on 35.2% (RH 64.8%) at 9 wt%, and the Results give 40.3% at 12 wt%. The abstract's catalyst figure is the only value that departs from them. It equals, to the printed decimal, the RH remaining at 12 wt%. The other two headline numbers in the abstract agree with the body: 66.8% and 2.6 GJ/tCO2 for the combined system, and the 69.1% desorption gain of the hybrid solvent (69.2% in the body; 589/348 − 1 = 69.3%).
Figure
Figure 1. Left: the abstract's sentence on the catalyst, which states 59.7% as a reduction and links it to the optimal configuration. Right: the reductions the Results (Fig. 3g text) and Discussion report for the same 1:1 Fe-SZ/SiO2 catalyst in 3 M GlyK against the same blank: 35.2% at the stated optimum (9 wt%) and 40.3% at 12 wt%. Orange text marks values reported in the paper. Bottom (blue): RH recomputed from the reported catalytic enhancements using the paper's Eq. 7, RH = 100/(1 + CE/100). The recomputation reproduces the body's reductions and shows that 59.7% is the share of the blank heat duty that remains at 12 wt%. A real 59.7% reduction would mean RH 40.3% and 3.14 GJ/tCO2. The paper reports neither value for the catalyst at any loading.
Figure 2. Each bar is the blank heat duty (3 M GlyK, no catalyst; 100%, 7.8 GJ/tCO2), split into the share that remains (light segment, RH) and the share removed (solid segment, reduction). Blue bars are recomputed from the paper's reported catalytic enhancements (25.6%, 54.3% and 67.5%) with Eq. 7. The column at the right gives the corresponding thermal energy, 7.8 × RH/100. The orange bar shows what the abstract's sentence states: a 59.7% reduction, which would leave 40.3% and imply about 3.14 GJ/tCO2 (marked *). The arrow links the abstract's 59.7 to the remaining-share segment at 12 wt%, which has the same value. The axis runs over the full 0 to 100% range.
Why it matters
The sentence is one of three quantitative claims in the abstract and the only one that isolates the catalyst. The catalyst is one of the paper's two named contributions, "catalytic solvent regeneration" and hybrid solvents. A reader of the abstract alone, or of a database or aggregator that reproduces it, would take the catalyst to cut heat duty by about 60% at its optimal configuration. The paper's own data show about 35% at that configuration (about 5.1 GJ/tCO2 instead of 7.8) and about 40% at the highest loading. A 60% reduction would mean about 3.1 GJ/tCO2. The abstract therefore overstates the catalyst's effect by a factor of 1.5 to 1.7. It also attaches the larger number to the word "optimal", although the paper selects 9 wt% as optimal on enhancement factor, which is the loading with the smaller reduction. The direction of the effect is unchanged: the catalyst still lowers the heat duty substantially. The body, figures and conclusions are not affected. These include the combined hybrid solvent plus catalyst result (RH 33.2%, a 66.8% reduction, 2.6 GJ/tCO2), which the abstract states correctly.
Explanations considered
- "Reduced to" written as "reduced by". 59.7 equals 100/1.675, the RH remaining at 12 wt% (equivalently 100 − 40.3). According to the version-of-record check, the publisher's full-resolution Fig. 3g, which is labelled RH (%), shows 59.7 for the 12 wt% bar and 64.8 for 9 wt%. This reading explains where the number comes from exactly. It does not make the sentence correct as published, because "reduces … by up to 59.7%" states a reduction. Even read as "to 59.7%", the value belongs to 12 wt%, not to the 9 wt% loading the paper calls optimal (RH 64.8%).
- "Up to" means the best case across loadings rather than the optimum. The best case for the 1:1 catalyst is 12 wt%, a 40.3% reduction. No loading or SZ:SiO2 ratio in the paper gives more: 20.4% at 6 wt% 1:1, 12.0% at 6 wt% 1:2 and 3.8% at 6 wt% 2:1. This reading changes which configuration is meant but not the size of the discrepancy.
- A different baseline or quantity. Measured against the 6 wt% catalyst instead of the blank, the reductions are 18.6% (9 wt%) and 25.0% (12 wt%). The gains in peak CO2 flux over the blank are 46.1% (9 wt%) and 56.5% (12 wt%), and the catalytic enhancement at 12 wt% is 67.5%. None of these equals 59.7. The closest other quantity in the paper is the hybrid solvent's reduction relative to GlyK without a catalyst: 1 − 3.2/7.8 = 59.0%, or 58.1% to 59.9% allowing for rounding of the two energies. That is a solvent effect, not the catalyst effect the sentence describes, and the paper does not state it as a percentage. The catalytic RH values for the other solvents (SarK 90.1%, TauK 55.8%) correspond to reductions of 9.9% and 44.2%, not 59.7%.
- Text conversion or archiving artefact. Ruled out. The PMC text, the Europe PMC full-text XML, the Nature Communications article page and the publisher PDF all carry the same abstract sentence and the same body values (see Status).
- Resolved in the supplement. No. The Supplementary Information does not contain 59.7. Its Fig. S6b gives RH 64.8% for cycle 1 at 9 wt%, which agrees with the body. The Transparent Peer Review file does not mention 59.7, 35.2 or 40.3. There is no separate Source Data file.
None of these makes the abstract's sentence consistent with the body. The arithmetic shows where the number most likely comes from. It says nothing about how the discrepancy arose.
Status
As of 2026-10-23:
- Version of record. The discrepancy is in the version of record. The abstract on the publisher's article page (reached through doi.org and re-checked on 2026-10-23) reads "reduces relative heat duty by up to 59.7% in the membrane vacuum system". The same page carries "the heat duty decreased by 35.2 and 40.3%" in the Results and "achieving a 35.2% reduction" in the Discussion. The publisher PDF (Nat Commun 2026, 17:2247) and the Europe PMC full-text XML carry the same text.
- Supplements. The Supplementary Information PDF (15 pages) and the Transparent Peer Review file were checked. Neither resolves the discrepancy (see above).
- Corrections. None found. Crossref lists no update-to or updated-by relations, and PMC and Europe PMC list no correction, erratum, retraction or expression of concern (
src/known.py). The publisher's article page shows no correction notice. - PubPeer and the web. Web searches for the title and DOI with "correction", "erratum" and "PubPeer", run in an earlier check on 2026-10-23, found no correction, PubPeer thread or published comment. PubPeer's own search could not be queried directly (it required human verification), and a repeat web search for this write-up was not possible because the session's search allowance was used up. A PubPeer thread therefore cannot be fully ruled out. Secondary listings, including a carbon-dioxide-removal literature aggregator and an institutional publication listing, repeat the abstract's 59.7% unchanged.
- Sources. PMC12966427 (
src/show.py); https://doi.org/10.1038/s41467-026-69542-6, which redirects to https://www.nature.com/articles/s41467-026-69542-6, and the publisher PDF; Europe PMC full-text XML; the article's supplementary files on the publisher's site; Crossref, PMC and Europe PMC notices (src/known.py); web searches as described above.
Suggested correction
In the abstract, replace
An optimal iron-sulfated zirconia to silica ratio of 1:1 reduces relative heat duty by up to 59.7% in the membrane vacuum system.
with a statement that matches the Results and Discussion, for example:
An optimal iron-sulfated zirconia to silica ratio of 1:1 at 9 wt% reduces the heat duty of the membrane vacuum system by 35.2% (by up to 40.3% at 12 wt%).
To keep the relative-heat-duty form instead, the sentence could say that the catalyst "reduces relative heat duty to 64.8% at the optimal 9 wt% loading (59.7% at 12 wt%)". No change to the body, figures or conclusions appears necessary.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12966427 (Nat Commun 2026, doi:10.1038/s41467-026-69542-6)
# "Catalytic hybrid solvent regeneration in membrane vacuum processes for direct air capture"
# Abstract vs body: size of the heat-duty reduction from the 1:1 Fe-SZ/SiO2 catalyst alone.
#
# Usage: python recompute.py.txt
# Needs: numpy, matplotlib.
# Output: prints every recomputed number used in writeup.md; writes fig1.svg/png and
# fig2.svg/png next to this script. If run from the papers repository (src/show.py
# present), it also checks each quoted statement verbatim against the PMC archive.
#
# Method. Eq. 6 of the paper: Q = m * cp * dT / (M_total_CO2 * Mw). In the catalyst-loading
# series the solvent (3 M GlyK), its mass and the temperature programme (90 C) are the same with
# and without catalyst, so Q is inversely proportional to the total CO2 desorbed in 30 min.
# Eq. 7: RH = Q_cat / Q_blank * 100, hence RH = M_blank / M_cat * 100 = 100 / (1 + CE/100),
# where CE (Eq. 4) is the catalyst enhancement. "Reduction in heat duty" = 100 - RH.
import os, sys, subprocess
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- 1. statements in the paper
QUOTES = [
("Abstract",
"An optimal iron-sulfated zirconia to silica ratio of 1:1 reduces relative heat duty by up to 59.7% in the membrane vacuum system."),
("Abstract",
"Finally, combining the optimized hybrid solvent, catalyst, and membrane system at 90 °C reduces thermal energy consumption by 66.8% relative to potassium glycinate, achieving an energy requirement of 2.6 GJ/tCO2."),
("Results, Catalytic support (SiO2 vs. Al2O3), Fig. 2g text",
"raising the total amount of desorbed CO2 from 244 to 307 mmol for 3 M GlyK"),
("Results, Catalyst concentration and stability, para 1",
"the total desorbed CO2 increased from 244 to 377 mmol, corresponding to a 54.3% catalytic enhancement"),
("Results, Catalyst concentration and stability, para 1",
"the enhancement rising to 67.5%"),
("Results, Catalyst concentration and stability, para 1",
"Notably, the magnitude of improvement from 6 to 9 wt% (112%) was greater than that from 9 to 12 wt% (24.3%), suggesting a reduced catalytic effect beyond a certain concentration, indicating the presence of an optimum."),
("Results, Catalyst concentration and stability, para 1",
"the use of 9 wt% catalyst resulted in the highest enhancement factor of 6.0 and was therefore selected as the optimal concentration for further evaluations."),
("Results, Catalyst concentration and stability, para 2 (Fig. 3g)",
"Accordingly, at 9 wt% and 12 wt%, the heat duty decreased by 35.2 and 40.3% as compared to the blank, respectively, reducing the thermal energy requirement from 7.8 to 5.1 and 4.7 GJ/tCO2."),
("Results, Catalyst concentration and stability, para 3",
"the RH% increased from 64.8% in cycle 1 to 68.8% in cycle 2"),
("Results, Catalytic hybrid amino acid solvents in MVR (Fig. 5f)",
"Considering GlyK as a benchmark, the catalyst reduced the RH of GlyK, SarK, and TauK to 64.8%, 90.1%, and 55.8%, respectively."),
("Results, Catalytic hybrid amino acid solvents in MVR (Fig. 5f)",
"Notably, the combination of a hybrid solvent with the catalyst further reduced the RH to 33.2%, corresponding to a 66.8% reduction in energy consumption."),
("Discussion",
"9 wt% fixed-bed catalyst was identified as the optimal concentration, achieving a 35.2% reduction in the system’s energy input."),
("Methods, MVR performance analysis, Eq. 7",
"The relative heat duty, RH (%), was calculated using Eq. 7"),
]
SHOW = os.path.join(HERE, "..", "..", "..", "src", "show.py")
if os.path.exists(SHOW):
print("== 1. Verbatim check against the PMC archive (src/show.py --full)")
full = subprocess.run([sys.executable, SHOW, "PMC12966427", "--full"], capture_output=True,
text=True, cwd=os.path.join(HERE, "..", "..", "..")).stdout
norm = lambda s: " ".join(s.split())
fulln = norm(full)
for loc, q in QUOTES:
print(f" [{'found' if norm(q) in fulln else 'NOT FOUND'}] {loc}: {q[:70]}...")
n597 = fulln.count("59.7")
print(f" occurrences of '59.7' in the archived text: {n597} (abstract only)")
else:
print("== 1. src/show.py not available; verbatim check skipped")
# ---------------------------------------------------------------- 2. the paper's own inputs
Q_BLANK = 7.8 # GJ/tCO2, 3 M GlyK blank (Results, Fig. 3g text)
M_BLANK = 244 # mmol CO2 desorbed in 30 min, blank
M_CAT = {6: 307, 9: 377} # mmol, 1:1 Fe-SZ/SiO2 at 6 and 9 wt%
CE_REP = {6: 25.6, 9: 54.3, 12: 67.5} # reported catalytic enhancement (%)
RED_REP = {9: 35.2, 12: 40.3} # reported heat-duty reduction (%)
Q_REP = {9: 5.1, 12: 4.7} # reported thermal energy (GJ/tCO2)
ABSTRACT_RED = 59.7 # abstract: "reduces relative heat duty by up to 59.7%"
print("\n== 2. Catalytic enhancement, Eq. 4, from the desorbed CO2 amounts")
print(f" 6 wt%: (307/244 - 1) = {(M_CAT[6]/M_BLANK - 1)*100:.1f}% (reported 25.6%)")
print(f" 9 wt%: (377/244 - 1) = {(M_CAT[9]/M_BLANK - 1)*100:.1f}% (reported 54.3%)")
print(f" 6 -> 9 wt% relative improvement: 54.3/25.6 - 1 = {(CE_REP[9]/CE_REP[6] - 1)*100:.1f}% (reported 112%)")
print(f" 9 -> 12 wt% relative improvement: 67.5/54.3 - 1 = {(CE_REP[12]/CE_REP[9] - 1)*100:.1f}% (reported 24.3%)")
print("\n== 3. Relative heat duty (Eq. 7) and reduction, two routes")
ROWS = {}
for w in (6, 9, 12):
rh_ce = 100.0 / (1 + CE_REP[w] / 100) # from reported CE
rh_m = 100.0 * M_BLANK / M_CAT[w] if w in M_CAT else None # from mmol, where printed
q = Q_BLANK * rh_ce / 100
ROWS[w] = dict(rh=rh_ce, red=100 - rh_ce, q=q, rh_m=rh_m)
s_m = f"; from mmol RH = {rh_m:.1f}%, reduction {100 - rh_m:.1f}%" if rh_m else ""
print(f" {w:>2} wt%: from CE, RH = 100/(1+{CE_REP[w]/100:.3f}) = {rh_ce:.2f}%, reduction {100 - rh_ce:.1f}%,"
f" Q = 7.8 x {rh_ce/100:.3f} = {q:.2f} GJ/t{s_m}")
print(f" reported: 9 wt% reduction 35.2% (RH 64.8%, Q 5.1); 12 wt% reduction 40.3% (Q 4.7)")
print(f" check from reported energies: 1 - 5.1/7.8 = {(1 - 5.1/7.8)*100:.1f}%; 1 - 4.7/7.8 = {(1 - 4.7/7.8)*100:.1f}%")
print(f" 100 - 35.2 = {100 - 35.2:.1f} (RH at 9 wt%, also stability cycle 1 and Fig. 5f GlyK value)")
print(f" 100 - 40.3 = {100 - 40.3:.1f} (RH at 12 wt%) <- equals the abstract's 59.7")
print("\n== 4. What the abstract's wording would imply")
rh_abs = 100 - ABSTRACT_RED
q_abs = Q_BLANK * rh_abs / 100
ce_abs = (100 / rh_abs - 1) * 100
print(f" 'reduces RH by 59.7%' -> RH = {rh_abs:.1f}%, Q = 7.8 x {rh_abs/100:.3f} = {q_abs:.2f} GJ/tCO2,"
f" requiring CE = {ce_abs:.0f}% (largest reported catalyst-only CE: 67.5%)")
print(f" ratio abstract / largest body reduction (12 wt%): 59.7/40.3 = {ABSTRACT_RED/RED_REP[12]:.2f}")
print(f" ratio abstract / optimum reduction (9 wt%): 59.7/35.2 = {ABSTRACT_RED/RED_REP[9]:.2f}")
print("\n== 5. Other headline numbers (consistency)")
print(f" hybrid + catalyst: RH 33.2% -> reduction {100 - 33.2:.1f}% (reported 66.8%);"
f" Q = 7.8 x 0.332 = {Q_BLANK*0.332:.2f} GJ/t (reported 2.6)")
print(f" SarK + catalyst RH 90.1% -> reduction {100-90.1:.1f}%; TauK + catalyst RH 55.8% -> reduction {100-55.8:.1f}%"
" (neither is a GlyK catalyst-only value, and neither equals 59.7% as a reduction)")
print(f" 6 wt% (1/2): (278/244 - 1) = {(278/244 - 1)*100:.1f}% (reported 13.7%)")
print(f" enhancement factor CE/W (Eq. 5): 6 wt% {CE_REP[6]/6:.2f}, 9 wt% {CE_REP[9]/9:.2f} (reported 6.0),"
f" 12 wt% {CE_REP[12]/12:.2f} -> 9 wt% is the maximum, as stated")
print(f" stability: 68.8/64.8 - 1 = {(68.8/64.8 - 1)*100:.1f}% (reported ~6.2%); 71.8/68.8 - 1 = {(71.8/68.8 - 1)*100:.1f}% (reported ~4.3%)")
print(f" hybrid vs TauK desorption: 589/348 - 1 = {(589/348 - 1)*100:.1f}% (abstract 69.1%, body 69.2%)")
print("\n== 6. Alternative readings of 59.7 (explanations considered)")
for lab, ce in (("6 wt% 1:1", 25.6), ("6 wt% 1:2", 13.7), ("6 wt% 2:1 (~4.0%)", 4.0)):
print(f" heat-duty reduction, {lab}: 100 - 100/(1+{ce/100:.3f}) = {100 - 100/(1 + ce/100):.1f}%")
print(f" reduction relative to 6 wt% instead of blank: 9 wt% {(1 - ROWS[9]['rh']/ROWS[6]['rh'])*100:.1f}%,"
f" 12 wt% {(1 - ROWS[12]['rh']/ROWS[6]['rh'])*100:.1f}%")
print(f" peak CO2 flux gain vs blank (19.3e-5): 9 wt% 28.2 -> {(28.2/19.3 - 1)*100:.1f}%, 12 wt% 30.2 -> {(30.2/19.3 - 1)*100:.1f}%")
lo, hi = (1 - 3.25/7.75)*100, (1 - 3.15/7.85)*100
print(f" hybrid solvent (3 M TauK + 1 M SarK) WITHOUT catalyst vs GlyK: 1 - 3.2/7.8 = {(1 - 3.2/7.8)*100:.1f}%"
f" (rounding range {lo:.1f}-{hi:.1f}%) -- a solvent effect, not the catalyst")
print(f" TauK alone vs GlyK (no catalyst): 1 - 5.5/7.8 = {(1 - 5.5/7.8)*100:.1f}%")
# ---------------------------------------------------------------- figures
BG, FG, FG2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
ORANGE_L, BLUE_L = "#f8d3c4", "#cfe0f5"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "path",
"text.color": FG, "axes.labelcolor": FG, "xtick.color": FG2,
"ytick.color": FG, "axes.edgecolor": FG2})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# --- Figure 1: abstract vs body, two text panels joined by an annotated connector
fig = plt.figure(figsize=(7.2, 5.6))
fig.patch.set_facecolor(BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 78)
ax.axis("off")
def box(x, y, w, h, ec, lw=2, ls="-"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc=BG, ec=ec, lw=lw, ls=ls))
# left: abstract
box(2.5, 30, 40, 44, ORANGE)
ax.text(4.5, 71, "Abstract", fontsize=12, weight="bold", va="top")
ax.text(4.5, 65.5, "“An optimal iron-sulfated\nzirconia to silica ratio of\n1:1 reduces relative heat\n"
"duty by up to 59.7% in\nthe membrane vacuum\nsystem.”", fontsize=10.5, va="top", linespacing=1.35)
ax.text(4.5, 38.5, "Reduction stated:", fontsize=10, color=FG2, va="center")
ax.text(4.5, 33.8, "59.7% (reported)", fontsize=12, color=ORANGE, weight="bold", va="center")
# right: results + discussion
box(57.5, 30, 40, 44, FG2)
ax.text(59.5, 71, "Results (Fig. 3g text)\nand Discussion", fontsize=12, weight="bold", va="top")
ax.text(59.5, 62.5, "“at 9 wt% and 12 wt%, the heat\nduty decreased by 35.2 and\n40.3% as compared to the\n"
"blank” (7.8 → 5.1 and 4.7 GJ/tCO2)", fontsize=10, va="top", linespacing=1.3)
ax.text(59.5, 46.5, "“9 wt% ... optimal ... achieving\na 35.2% reduction”", fontsize=10, va="top",
linespacing=1.3)
ax.text(59.5, 38.5, "Reductions stated:", fontsize=10, color=FG2, va="center")
ax.text(59.5, 33.8, "35.2% (9 wt%), 40.3% (12 wt%)", fontsize=10.5, color=ORANGE, weight="bold", va="center")
# connector
arr = FancyArrowPatch((43.5, 52), (56.5, 52), arrowstyle="<->", mutation_scale=16, lw=2, color=FG)
ax.add_patch(arr)
ax.text(50, 55, "conflict", ha="center", va="bottom", fontsize=10, color=FG, weight="bold")
ax.text(50, 49, "59.7\nvs\n35.2 / 40.3", ha="center", va="top", fontsize=10,
color=FG2, linespacing=1.3)
# bottom: recomputation that names the conflict
box(2.5, 3, 95, 21, BLUE)
ax.text(4.5, 21.5, "Recomputed from the paper's own Eq. 7 (RH = Q_cat / Q_blank = 1 / (1 + CE))",
fontsize=10.5, va="top", color=BLUE, weight="bold")
r9, r12 = ROWS[9], ROWS[12]
ax.text(4.5, 16.2,
f"9 wt% (CE 54.3%): RH = {r9['rh']:.1f}% → reduction {r9['red']:.1f}% ({r9['q']:.2f} GJ/tCO2)\n"
f"12 wt% (CE 67.5%): RH = {r12['rh']:.1f}% → reduction {r12['red']:.1f}% ({r12['q']:.2f} GJ/tCO2)\n"
f"59.7% is the heat duty that remains at 12 wt%, not the reduction. A 59.7% reduction\n"
f"would mean RH = {rh_abs:.1f}% and {q_abs:.2f} GJ/tCO2; no such value appears in the paper.",
fontsize=10, va="top", color=FG, linespacing=1.4)
save(fig, "fig1")
# --- Figure 2: heat duty remaining vs removed, by catalyst loading, plus the abstract's reading
labels = ["Blank (3 M GlyK)", "6 wt%", "9 wt% (optimum)", "12 wt%",
"Abstract as written\n(“by up to 59.7%”)"]
rh = [100.0, ROWS[6]["rh"], ROWS[9]["rh"], ROWS[12]["rh"], rh_abs]
q = [Q_BLANK, ROWS[6]["q"], ROWS[9]["q"], ROWS[12]["q"], q_abs]
is_abs = [False, False, False, False, True]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ys = np.arange(len(labels))[::-1].astype(float)
ys[-1] -= 0.35
for y, r, qq, ab in zip(ys, rh, q, is_abs):
c, cl = (ORANGE, ORANGE_L) if ab else (BLUE, BLUE_L)
ax.barh(y, r, height=0.6, color=cl, edgecolor=c, lw=1.5, zorder=2)
if r < 100:
ax.barh(y, 100 - r, left=r, height=0.6, color=c, edgecolor=c, lw=1.5, zorder=2)
ax.text(r + (100 - r) / 2, y, f"{100 - r:.1f}%", ha="center", va="center", color="white",
fontsize=10.5, weight="bold", zorder=3)
ax.text(r / 2, y, f"{r:.1f}%", ha="center", va="center", color=FG, fontsize=10.5, zorder=3)
ax.text(102, y, f"{qq:.2f}" if not ab else f"{qq:.2f}*", ha="left", va="center",
color=ORANGE if ab else BLUE, fontsize=10.5)
ax.text(102, ys[0] + 0.55, "GJ/tCO2", ha="left", va="bottom", color=FG2, fontsize=10)
ax.add_patch(FancyArrowPatch((rh_abs + (100 - rh_abs) / 2, ys[4] + 0.33), (ROWS[12]["rh"] / 2, ys[3] - 0.33),
arrowstyle="->", mutation_scale=14, lw=1.5, color=FG, zorder=4,
connectionstyle="arc3,rad=-0.15"))
ax.text(73, (ys[3] + ys[4]) / 2, "same number", ha="left", va="center",
fontsize=10, color=FG, zorder=4)
ax.set_xlim(0, 100)
ax.set_ylim(ys[-1] - 0.6, ys[0] + 0.85)
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10)
ax.set_xlabel("Share of blank heat duty (%): light = remaining (RH), solid = reduction")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
handles = [plt.Rectangle((0, 0), 1, 1, fc=BLUE, ec=BLUE,
label="Recomputed from the paper's enhancements (Eq. 7)"),
plt.Rectangle((0, 0), 1, 1, fc=ORANGE, ec=ORANGE,
label="Abstract's 59.7% read as a reduction (* implied energy)")]
ax.legend(handles=handles, loc="upper left", bbox_to_anchor=(-0.3, -0.16), ncol=1, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.27, right=0.88, top=0.93, bottom=0.27)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
The abstract reports a positive temperature association with H7 spread; the paper's own coefficients are negative
Nat Commun (2026) · doi:10.1038/s41467-026-72718-9 · PMC13351079
Summary
The abstract states that "temperature and poultry density show positive association" with the spread of H7 influenza viruses. The paper's Bayesian phylogeographic GLM reports the opposite sign for temperature: β = −0.484 (95% HPD −0.900 to −0.237) at origin and β = −0.227 (95% HPD −0.499 to −0.047) at destination. These are the estimates of the model without sample size (Fig. 9A). Both intervals lie entirely below zero, so by the paper's own rule both effects are significant and negative, and the Results and Discussion describe them that way ("a negative and supportive predictor"; "the negative correlation between temperature and transmission risk"). In the alternative models that add sample size as a predictor (Fig. 9B; Supplementary Fig. 24B), temperature is rarely included (activation rates of 2.7% or less) and its intervals span zero, so no model in the paper gives a supported positive temperature effect. The poultry-density half of the sentence agrees with the reported chicken-stock coefficient (+0.461); only the temperature direction is reversed. The error sits in the most-read part of the paper and concerns one of the named drivers of spread, but the underlying estimates and the body text are consistent with each other.
What the paper reports
In the abstract:
Geographic distance is negatively associated with the spread of the H7 virus, while temperature and poultry density show positive association.
— Abstract
The editor's summary printed with the abstract names temperature as a driver without giving a direction:
Authors show that spread of H7 avian influenza viruses is driven by geographic distance and temperature.
— Abstract (editor's summary)
In the Results, the GLM estimates are given as posterior mean and 95% HPD:
Geographic distance was strongly negatively associated with viral spread (coefficient = −0.873, 95% HPD: −0.995 to −0.753), with decisive support for its inclusion in the model (activation rate = 1) (Fig. 9A).
Chicken stock origin, defined as the number of annual chicken inventory at the source country, was consistently positively associated with virus spread (0.461, 0.111 to 0.845), while temperature origin (−0.484, −0.900 to −0.237) and temperature destination (−0.227, −0.499 to −0.047) was also considered as a negative and supportive predictor.
— Results, "Mechanisms of virus spatial spread"
In the Discussion:
Chicken stock origin consistently made a positive contribution to the spread of the virus, while temperature origin was considered as a supportively negative predictor.
Influenza viruses degrade more quickly in warm or dry conditions56, so high temperature may reduce viral persistence in the environment, which explains the negative correlation between temperature and transmission risk.
— Discussion, paragraph on the GLM
The concluding paragraph lists temperature without a direction:
Our findings highlight how geographic distance, temperature, GDP, and poultry density collectively shape viral dispersal across regions.
— Discussion, concluding paragraph
The Methods define the model and the significance rule:
This GLM approach was extended from the discrete diffusion model, which characterizes the logarithm of the transition rates within the CTMC matrix as a log-linear function dependent on multiple potential indicators of viral spread.
All predictors were log-transformed and standardized except bird migration before inclusion in the matrices.
HPD intervals (Highest Posterior Density intervals) from the posterior distribution were examined for each coefficient: if the HPD interval does not contain zero, the effect is considered significant; if it contains zero, the effect is uncertain.
— Methods, "Phylogeographic inference with epidemiological predictors"
Each quotation was checked against the article as archived in PubMed Central (src/show.py PMC13351079) and occurs exactly once; recompute.py.txt reruns that check. The archived PMC text renders minus signs as ASCII hyphens; the publisher's HTML uses the Unicode minus sign (U+2212), which is what is shown above. Footnote numerals (e.g. "56") are reproduced as they appear in the archived text.
What the paper's own numbers show
Method. The claim concerns direction only, so the check uses the paper's own estimates and its own decision rule. Under the stated model, log Λij = Σp βpδpxij,p, a negative β means that a higher (log-transformed, standardized) predictor value is associated with a lower transition rate, conditional on the predictor's inclusion. For each coefficient reported in the Results text, the sign of the 95% HPD interval was classified (entirely above zero: positive; entirely below: negative; straddling zero: uncertain) and compared with the direction the abstract states. As a descriptive aid, exp(β) gives the multiplicative change in the transition rate per one standard deviation of the predictor; its interval is the one-to-one transform of the reported HPD endpoints, which is not necessarily an HPD on that scale. No posterior samples or Fig. 9 source data are published, so the model itself cannot be re-fitted; the check is confined to the numbers the paper prints. recompute.py.txt prints every value below.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Direction of temperature association | Abstract | "positive association" | Contradicted by both temperature coefficients (rows below) |
| Temperature (origin), β and 95% HPD | Results, "Mechanisms of virus spatial spread" | −0.484 (−0.900 to −0.237); "a negative and supportive predictor" | HPD entirely below 0 (upper bound −0.237): negative, significant by the paper's rule; exp(β) = 0.616 (0.407 to 0.789), about 38% lower rate per SD |
| Temperature (destination), β and 95% HPD | Same paragraph | −0.227 (−0.499 to −0.047); same wording | HPD entirely below 0 (upper bound −0.047): negative; exp(β) = 0.797 (0.607 to 0.954), about 20% lower rate per SD |
| Direction of temperature association | Discussion | "supportively negative predictor"; "negative correlation between temperature and transmission risk" | Agrees with the coefficients |
| Chicken stock (origin), the abstract's "poultry density" | Results | +0.461 (0.111 to 0.845); abstract: positive | HPD entirely above 0: positive; exp(β) = 1.586 (1.117 to 2.328). Abstract agrees |
| Geographic distance | Results | −0.873 (−0.995 to −0.753); abstract: negative | HPD entirely below 0: negative; exp(β) = 0.418 (0.370 to 0.471). Abstract agrees |
| Abstract direction statements checked against the reported HPDs | Abstract vs Results | 4 directions (distance, poultry density, and temperature at origin and destination) | 2 of 4 agree (distance, poultry density); 2 of 4 contradicted (both temperature terms) |
Both temperature estimates printed in the text are negative, with intervals that exclude zero, and the paper's own text describes and explains a negative relationship. The abstract's "positive" cannot be derived from these numbers.
Figure
Figure 1. The conflicting statements, side by side. Left (orange): the abstract sentence, with the clause that assigns a positive association to temperature and poultry density highlighted. Right (blue): the Results sentence that reports both temperature coefficients with their 95% HPD intervals and calls temperature "a negative and supportive predictor," and the two Discussion sentences that call it "a supportively negative predictor" and explain "the negative correlation between temperature and transmission risk." The footer notes that the abstract's directions for distance and poultry density (chicken stock origin) agree with the reported coefficients; only the temperature direction conflicts.
Figure 2. Reported GLM coefficients and the direction each implies. Orange dots and lines are the posterior means and 95% HPD intervals as printed in the Results text (log scale of the transition rate; the vertical line at 0 marks no effect). At right, "Abstract says" gives the direction stated in the abstract (a dash where it states none), and the blue "Sign of reported HPD" column gives the direction implied by each interval under the paper's rule. The two temperature rows, in bold, are the only ones where the two columns disagree; both intervals lie entirely to the left of zero. The sample-size coefficients are shown for context only; according to the Fig. 9 legend they come from the model that includes sample size (Fig. 9B). The other four are from the Fig. 9A model (the distance value is cited to Fig. 9A in the text; the others match the Fig. 9A plot).
Why it matters
- The abstract reverses the direction of a named driver. Temperature is one of only two drivers named in the editor's summary ("driven by geographic distance and temperature"). A reader who sees only the abstract, as in indexing and alerting services, is told that warmer conditions go with more H7 spread, when the paper's estimates say warmer conditions go with less (about 38% lower transition rate per SD of log-transformed origin temperature, conditional on inclusion).
- The stated direction matters for the paper's own recommendations. The Discussion proposes "[I]ntegrating real-time environmental variables such as temperature monitoring into early warning systems" and explains the negative effect through faster viral degradation in warm conditions. Read with the abstract's positive sign, that mechanism would be contradicted and the direction of a temperature-based alert would be reversed.
- The main conclusions are unaffected. The coefficients themselves, the Results and the Discussion agree with one another. Nothing else in the paper depends on the abstract's sign: not the distance effect, the host-specific spread velocities, the roles of Anseriformes and Galliformes, Asia as an epicentre, or the One Health recommendations. The error is confined to one clause of one abstract sentence.
Explanations considered
- A different model reports a positive temperature effect. No model gives a supported positive effect. The paper fits a model without sample size (Fig. 9A) and one with it (Fig. 9B), and repeats both with GDP per capita added (Supplementary Fig. 24A and B, described as "similar"). In the two models without sample size, both temperature terms are negative: the text gives −0.484 and −0.227 for Fig. 9A, and Supplementary Fig. 24A plots means of about −0.45 and −0.2 with posterior draws below zero (activation rate 56.7% for Temperature (o)). In the two models with sample size, temperature has almost no inclusion support and intervals that span zero. In Fig. 9B the plotted means are about −0.33 (origin) and −0.02 (destination), with activation rates of 1.9% and 0.1%. In Supplementary Fig. 24B the Temperature (o) mean is plotted slightly above zero (about +0.15), but its median is about −0.35, its 95% HPD runs from roughly −0.8 to +2.4, and its activation rate is 2.7%; Temperature (d) sits near zero at 0.1%. Under the paper's rule, these are "uncertain" effects, not positive ones. The values for Fig. 9B and Supplementary Fig. 24 are read by eye from the publisher's figure images, as no source data are published for them. The model the text reports and the abstract summarizes is the one with significant negative temperature terms.
- "Positive association" means "supported for inclusion," not direction. This does not fit the sentence. The same sentence uses "negatively associated" for distance, which can only be a direction, and the Results keep the two ideas separate by calling temperature "a negative and supportive predictor." Even under this reading, the destination term would not qualify: the activation rate printed for Temperature (d) in Fig. 9A is 0.7%, which falls in the band the Methods describe as "<5% suggests potential unimportance." (Temperature (o) is printed at 68.0%.)
- Sign convention or transformation. The Methods specify a log-linear model of the transition rates with log-transformed, standardized predictors. Under that parameterization a negative β is a negative association, and the Discussion reads it that way. The transformation of the temperature variable does not change the sign of its association with spread.
- Confusion with a correlated predictor. The Results report a Spearman correlation of r = −0.65 between GDP per capita and temperature, and GDP was dropped from the final model for that reason. That is a correlation between two predictors, not the association of temperature with spread, and adding GDP does not produce a supported positive temperature effect (item 1): in Supplementary Fig. 24A both temperature terms remain negative, and in Supplementary Fig. 24B the origin term is included in 2.7% of samples with an interval spanning zero.
- "Poultry density" carries the positive sign and temperature was listed alongside it. The sentence does not allow that reading. "temperature and poultry density show positive association" makes the two nouns the joint subject of one plural verb, so as printed it assigns a positive direction to both. It also contrasts them ("while") with distance, which is "negatively associated". A reader cannot recover the negative temperature effect from the abstract.
- Rounding, a typo in the coefficients, or a text-conversion error. None applies. The sign is carried by the words "negative" and "positive" as well as by the minus signs, and the minus signs in the Results text agree with the Fig. 9A plot, where both temperature distributions lie below the zero line. The publisher's HTML and the PMC archive give the same abstract wording.
Limitations
- The check uses only the numbers the paper prints. No posterior samples, log files or Fig. 9 source data are published (the Source Data file covers Figs 1 and 8), so the GLM was not refitted and the HPD intervals were not recomputed.
- The values for Fig. 9B and Supplementary Fig. 24A and B are not given in the text and were read by eye from the publisher's figure images. They are approximate, but the finding does not depend on their exact values: none of them is a supported positive effect.
- exp(β) is shown only as a descriptive aid. It is conditional on the predictor's inclusion, and its interval is a transform of the reported HPD endpoints, not an HPD on that scale.
Status
As of 2026-10-23:
- Quotations. The 13 passages quoted in this write-up (listed in
recompute.py.txt) were each found exactly once in the PMC-archived text (src/show.py PMC13351079 --full);recompute.py.txtreruns that check, treating the PMC text's ASCII hyphens as the publisher's minus signs. - Numbers. Every number in the table and in Figure 2 is re-derived by
recompute.py.txtfrom the coefficients printed in the Results text. - Figures in the paper. Fig. 9 (publisher's image) and Supplementary Fig. 24 (supplementary PDF, p. 22) were inspected by eye; no values were extracted from them beyond the approximate readings in item 1.
- Refit. Not possible: no posterior samples or GLM source data are available.
- Corrections.
src/known.py(run 2026-10-23) returned empty lists: no Crossref updates and no PMC or Europe PMC correction or retraction notices.
Sources: PMC13351079 (PubMed Central archive, read with src/show.py); https://doi.org/10.1038/s41467-026-72718-9 (publisher's article page and Fig. 9 image); Supplementary Information PDF (41467_2026_72718_MOESM1_ESM.pdf), p. 22.
Suggested correction
In the Abstract, replace "while temperature and poultry density show positive association" with "while poultry density shows a positive association and temperature a negative association", so that the abstract matches the coefficients in the Results (temperature origin −0.484, 95% HPD −0.900 to −0.237; temperature destination −0.227, −0.499 to −0.047) and the Discussion. The authors could also state in the Results that, in the model that includes sample size (Fig. 9B), temperature has low inclusion support (activation rates 1.9% and 0.1%) and intervals that span zero.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13351079 (Nat Commun 2026, doi:10.1038/s41467-026-72718-9)
# Abstract vs Results/Discussion: direction of the temperature association in the
# Bayesian phylogeographic GLM of H7 influenza spread.
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib (numpy not required).
#
# Inputs
# * Quotes: copied verbatim from the article as archived in PubMed Central (PMC13351079).
# The PMC text served by src/show.py writes minus signs as ASCII '-', while the
# publisher's HTML uses U+2212; the check below treats the two as the same character.
# When run inside the /home/ubuntu/papers repo, each quote is re-checked against the
# archived full text (skipped elsewhere).
# * GLM coefficients: the posterior means and 95% HPD bounds printed in the Results
# paragraph "Mechanisms of virus spatial spread". No posterior samples or source data for
# Fig. 9 are published (the Source Data file covers Figs 1 and 8 only), so the model cannot
# be re-fitted; the check is on the paper's own reported numbers.
import math, os, subprocess
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC13351079"
# ------------------------------------------------------------------ quotes
QUOTES = {
"abstract": ("Abstract",
"Geographic distance is negatively associated with the spread of the H7 virus, while "
"temperature and poultry density show positive association."),
"editor_summary": ("Abstract (editor's summary)",
"Authors show that spread of H7 avian influenza viruses is driven by geographic distance "
"and temperature."),
"results_distance": ("Results, 'Mechanisms of virus spatial spread'",
"Geographic distance was strongly negatively associated with viral spread (coefficient = "
"-0.873, 95% HPD: -0.995 to -0.753), with decisive support for its inclusion in the model "
"(activation rate = 1) (Fig. 9A)."),
"results_temp": ("Results, 'Mechanisms of virus spatial spread'",
"Chicken stock origin, defined as the number of annual chicken inventory at the source "
"country, was consistently positively associated with virus spread (0.461, 0.111 to 0.845), "
"while temperature origin (-0.484, -0.900 to -0.237) and temperature destination (-0.227, "
"-0.499 to -0.047) was also considered as a negative and supportive predictor."),
"results_gdp": ("Results, 'Mechanisms of virus spatial spread'",
"The results incorporating GDP were similar (Supplementary Fig. 24)."),
"disc_temp": ("Discussion, paragraph on the GLM",
"Chicken stock origin consistently made a positive contribution to the spread of the virus, "
"while temperature origin was considered as a supportively negative predictor."),
"disc_mech": ("Discussion, paragraph on the GLM",
"Influenza viruses degrade more quickly in warm or dry conditions56, so high temperature may "
"reduce viral persistence in the environment, which explains the negative correlation "
"between temperature and transmission risk."),
"conclusion": ("Discussion, concluding paragraph",
"Our findings highlight how geographic distance, temperature, GDP, and poultry density "
"collectively shape viral dispersal across regions."),
"disc_ews": ("Discussion, paragraph on prevention and control",
"Integrating real-time environmental variables such as temperature monitoring into early "
"warning systems"),
"methods_glm": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
"This GLM approach was extended from the discrete diffusion model, which characterizes the "
"logarithm of the transition rates within the CTMC matrix as a log-linear function dependent "
"on multiple potential indicators of viral spread."),
"methods_std": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
"All predictors were log-transformed and standardized except bird migration before "
"inclusion in the matrices."),
"methods_hpd": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
"HPD intervals (Highest Posterior Density intervals) from the posterior distribution were "
"examined for each coefficient: if the HPD interval does not contain zero, the effect is "
"considered significant; if it contains zero, the effect is uncertain."),
"methods_act": ("Methods, 'Phylogeographic inference with epidemiological predictors'",
"Activation rate (the frequency with which a predictor is included in the model) was used "
"to gauge importance: an activation rate ≥50% indicates a highly important predictor, "
"10–50% indicates moderate importance, and <5% suggests potential unimportance."),
}
def norm(s):
return " ".join(s.replace("−", "-").split())
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.\n")
return
try:
full = subprocess.run([py, "src/show.py", PMCID, "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e: # pragma: no cover
print(f"Quote check: could not load archived text ({e}); skipped.\n")
return
full_n = norm(full)
print("Quote check against the PMC-archived full text (occurrences; >=1 means found verbatim):")
for key, (loc, q) in QUOTES.items():
print(f" {key:17s} {full_n.count(norm(q))} ({loc})")
print()
# ------------------------------------------------------------------ reported GLM numbers
# (label, mean, HPD low, HPD high, direction the abstract states or None)
# All from the Results paragraph quoted above (Fig. 9A model, except sample size, which the
# Fig. 9 legend places in the model of Fig. 9B).
COEFS = [
("Geographic distance", -0.873, -0.995, -0.753, "negative"),
("Chicken stock (origin)", 0.461, 0.111, 0.845, "positive"), # "poultry density"
("Temperature (origin)", -0.484, -0.900, -0.237, "positive"),
("Temperature (destination)", -0.227, -0.499, -0.047, "positive"),
("Sample size (origin)", 1.200, 0.777, 1.600, None),
("Sample size (destination)", 0.598, 0.402, 0.809, None),
]
def classify(lo, hi):
if lo > 0:
return "positive"
if hi < 0:
return "negative"
return "uncertain (HPD contains 0)"
def recompute():
print("Reported GLM coefficients (log scale) and the direction they imply under the paper's rule")
print("('if the HPD interval does not contain zero, the effect is considered significant'):\n")
print(f" {'predictor':27s} {'beta':>7s} {'95% HPD':>17s} {'implied':>9s} {'abstract says':>13s} agree?")
out = []
for lab, m, lo, hi, abs_dir in COEFS:
imp = classify(lo, hi)
agree = "-" if abs_dir is None else ("yes" if abs_dir == imp else "NO")
print(f" {lab:27s} {m:7.3f} [{lo:6.3f}, {hi:6.3f}] {imp:>9s} {abs_dir or '(not stated)':>13s} {agree}")
out.append((lab, m, lo, hi, abs_dir, imp, agree))
print()
print("Rate multipliers exp(beta) per 1 SD of the log-transformed, standardized predictor,")
print("conditional on inclusion (endpoints transformed one-to-one; not necessarily an HPD on")
print("the multiplier scale):")
for lab, m, lo, hi, *_ in out:
print(f" {lab:27s} exp(beta) = {math.exp(m):.3f} [{math.exp(lo):.3f}, {math.exp(hi):.3f}]")
print()
for lab, m, lo, hi, *_ in out:
if lab.startswith("Temperature"):
print(f" {lab}: upper HPD bound {hi:+.3f} < 0 -> {hi < 0}; "
f"multiplier {math.exp(m):.3f} -> {100 * (1 - math.exp(m)):.1f}% lower rate per SD")
n_temp_neg = sum(1 for r in out if r[0].startswith("Temperature") and r[5] == "negative")
print(f"\n Temperature coefficients whose HPD lies entirely below 0: {n_temp_neg} of 2")
print(f" Abstract direction statements contradicted by the reported HPDs: "
f"{sum(1 for r in out if r[6] == 'NO')} of {sum(1 for r in out if r[4])}\n")
return out
# ------------------------------------------------------------------ figures
BG, TXT, TXT2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT2,
"ytick.color": TXT})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
print(f"wrote {name}.svg and {name}.png")
def fig1():
W, H = 7.2, 5.0
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
def panel(x, w, title, sub, body, accent):
ax.add_patch(FancyBboxPatch((x, 0.95), w, 3.9, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=accent, lw=1.5))
ax.text(x + 0.14, 4.72, title, fontsize=11.5, fontweight="bold", va="top", color=TXT)
ax.text(x + 0.14, 4.42, sub, fontsize=10, va="top", color=accent)
yy = 4.08
for line, style in body:
ax.text(x + 0.14, yy, line, fontsize=10, va="top",
color={"n": TXT, "b": accent, "s": TXT2}[style],
fontweight="bold" if style == "b" else "normal",
fontstyle="italic" if style == "s" else "normal")
yy -= 0.235
left = [("“Geographic distance is negatively", "n"),
("associated with the spread of", "n"),
("the H7 virus, while", "n"),
("temperature and poultry", "b"),
("density show positive", "b"),
("association.”", "b"),
("", "n"), ("", "n"), ("", "n"), ("", "n"), ("", "n"),
("States: temperature > 0", "s"),
("(and poultry density > 0).", "s")]
right = [("Results, GLM paragraph:", "s"),
("“…temperature origin (−0.484, −0.900", "n"),
("to −0.237) and temperature destination", "n"),
("(−0.227, −0.499 to −0.047) was also", "n"),
("considered as a negative and", "b"),
("supportive predictor.”", "b"),
("Discussion:", "s"),
("“…temperature origin was considered", "n"),
("as a supportively negative predictor.”", "b"),
("“…which explains the negative", "b"),
("correlation between temperature and", "b"),
("transmission risk.”", "b"),
("Both temperature HPDs lie below 0.", "s")]
panel(0.15, 2.75, "Abstract", "reported claim", left, ORANGE)
panel(3.62, 3.43, "Results and Discussion", "the paper's own estimates and text", right, BLUE)
ax.add_patch(FancyArrowPatch((2.94, 3.55), (3.58, 3.55), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT))
ax.text(3.26, 3.85, "conflict", ha="center", fontsize=10, fontweight="bold", color=TXT)
ax.text(3.26, 3.28, "sign of\ntemper-\nature", ha="center", va="top", fontsize=10, color=TXT2)
ax.text(0.15, 0.48,
"Distance (β = −0.873) and chicken stock origin (β = +0.461; the abstract's “poultry\n"
"density”) have the signs the abstract states. Only the temperature direction differs.",
fontsize=10, color=TXT2, va="center")
save(fig, "fig1")
def fig2(rows):
n = len(rows)
fig, ax = plt.subplots(figsize=(7.2, 3.9))
fig.subplots_adjust(left=0.29, right=0.66, top=0.86, bottom=0.17)
ax.set_xlim(-1.3, 1.8)
ax.set_ylim(n - 0.4, -0.7)
ax.axvline(0.0, color=TXT2, lw=1.0, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_xticks([-1.0, -0.5, 0.0, 0.5, 1.0, 1.5])
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(TXT2)
ax.tick_params(axis="y", length=0)
ax.set_yticks(range(n))
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlabel("GLM coefficient β (log scale of transition rate)", fontsize=10, color=TXT)
for i, (lab, m, lo, hi, abs_dir, imp, agree) in enumerate(rows):
ax.plot([lo, hi], [i, i], color=ORANGE, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([m], [i], "o", ms=8, color=ORANGE, zorder=4)
fw = "bold" if agree == "NO" else "normal"
ax.text(1.03, i, abs_dir or "—", transform=ax.get_yaxis_transform(), va="center",
fontsize=10, color=TXT, fontweight=fw)
ax.text(1.36, i, imp, transform=ax.get_yaxis_transform(), va="center",
fontsize=10, color=BLUE, fontweight=fw)
ax.text(1.03, -0.95, "Abstract\nsays", transform=ax.get_yaxis_transform(), va="bottom",
fontsize=10, color=TXT, fontweight="bold")
ax.text(1.36, -0.95, "Sign of\nreported HPD", transform=ax.get_yaxis_transform(), va="bottom",
fontsize=10, color=BLUE, fontweight="bold")
ax.text(0.0, -0.95, "no effect", ha="center", va="bottom", fontsize=10, color=TXT2)
ax.plot([], [], "-o", color=ORANGE, lw=2, ms=8, label="Reported mean and 95% HPD (Results)")
ax.legend(loc="upper left", bbox_to_anchor=(-0.45, -0.14), frameon=False, fontsize=10)
save(fig, "fig2")
if __name__ == "__main__":
check_quotes()
rows = recompute()
fig1()
fig2(rows)
Abstract reports 45.6% Faradaic efficiency at 50 mA cm⁻²; the paper's own numbers give 37%
Nat Commun (2026) · doi:10.1038/s41467-026-72987-4 · PMC13377095
Summary
The Abstract, the closing paragraph of the Introduction and the Discussion all state that the CuO/Ag catalyst, in a membrane electrode assembly (MEA) reactor, gives an ethylene oxide (EO) production rate of 345 μmol cm⁻² h⁻¹ at 50 mA cm⁻² over 60 h, "with a Faradaic efficiency of 45.6%". By the paper's own Faradaic-efficiency definition (Methods Eq. 5) and the two-electron oxidation of ethylene to EO, 345 μmol cm⁻² h⁻¹ at 50 mA cm⁻² corresponds to an FE of 36.99%. The Results text reports 37% for this condition (Fig. 5g), and the Source Data lists 37.05%. The Results give 45.6% for a different operating point: the initial MEA at 3.2 V, about 10 mA cm⁻² and 100 μmol cm⁻² h⁻¹. The headline triple therefore overstates the FE of the high-rate condition by 8.6 percentage points (23% in relative terms). No single reported condition reaches both 345 μmol cm⁻² h⁻¹ and 45.6%.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py. The publisher's version of record has the same numbers (see Status).
Abstract:
In a membrane electrode assembly reactor, a stable ethylene oxide production rate of 345 μmol·cm-2·h-1 is achieved at 50 mA·cm-2 over 60 hours, with a Faradaic efficiency of 45.6% and a selectivity of 92.8%.
Introduction, closing paragraph:
When evaluated in a membrane electrode assembly (MEA) reactor, continuous direct ethylene epoxidation is enabled, with a stable EO production rate of 345 μmol cm-2 h-1 maintained over 60 hours at 50 mA cm-2, accompanied by a Faradaic efficiency of 45.6% and selectivity of 92.8%.
Discussion:
When implemented in a membrane electrode assembly reactor, a stable EO production rate of 345 μmol cm-2 h-1 is maintained over 60 h at 50 mA cm-2, with a Faradaic efficiency of 45.6% and selectivity of 92.8%.
Results, "Performance of CuO/Ag for electrochemical epoxidation of ethylene" (the two MEA conditions):
As the voltage increases, both the EO production and the FE first rise and then decline (Fig. 5f), reaching the optimal values at 3.2 V (production: 100 μmol cm-2 h-1; FE: 45.6%). However, the initial MEA configuration operated at a low current density of approximately 10 mA cm-2.
To bridge the gap toward industrial requirements, the Al foils was employed as the current collector to upgrade the MEA design (Supplementary Fig. 22), which significantly reduced the internal resistance of MEA.
The optimized system achieved an EO production rate of 345 μmol cm-2 h-1 at a current density of 50 mA cm-2, while maintaining a relatively high FE of 37% (Fig. 5g). Moreover, the CuO/Ag catalyst can operate stably in the MEA reactor for 60 h (Fig. 5h).
Results, same paragraph (comparison with earlier work):
This approach demonstrates competitive performance to other reported direct epoxidation methods in current density, production rate, and operating duration, while maintaining high FE and EO selectivity (Supplementary Table 3).
The CuO/Ag catalyst exhibits competitive direct-epoxidation performance in the MEA reactor, surpassing previously reported results in FE, production, current density and long-term stability.
Introduction, on the field's limitation:
typically result in low FE (<40%) and limited selectivity
Methods, Product analysis (Eq. 5, FE = ni z F / (I t); the equation itself is flattened in the archived text):
z is the number of electrons involved in the reaction, F is the Faraday constant, I is the reaction current, and t is the reaction time.
Methods, Membrane electrode assembly reactor:
were used as the work electrode (2 × 2 cm2) and counter electrode, respectively.
Supplementary material (publisher ESM files, values transcribed here). Supplementary Table 2, "Faradaic efficiencies, production, and selectivity of products on CuO/Ag", lists EO production 345 μmol/h, FE 45.6% and selectivity 92.8%, together with five by-products (CO2 17.8 μmol/h, FE 7.0%; CO 8.6, 2.3%; CH3OH 16.5, 2.1%; CH2O 9.8, 1.3%; CH3CHO 7.6, 1.0%). It does not state the current density. Supplementary Table 3 lists "This work" as FE 45.6%, "EO partial current density (mA/cm2)" 50, stability 60 h, selectivity 92.8% and EO production 345.16 μmol/h. In the Source Data file, sheet "Figure 5g" lists, by current density, EO production and FE: 10 mA cm⁻², 101.45, 45.54%; 25, 198.77, 42.63%; 50, 345.16, 37.05%; 75, 303.96, 21.72%; 100, 203.61, 10.91%. Sheet "Figure 5f" lists the 3.2 V point as 101.45 and 45.54%. Sheet "Figure 5h" (the 60 h stability test) contains only cell voltage against time, with no FE.
What the paper's own numbers show
Method. For a rate r (mol cm⁻² h⁻¹) and current density j (A cm⁻²) on the same area, Methods Eq. 5 becomes FE = r z F / (3600 j), with F = 96,485 C mol⁻¹. Direct water-mediated epoxidation, C2H4 + H2O → C2H4O + 2H⁺ + 2e⁻, has z = 2. The paper does not state z in the text, but its Source Data FE values for 25, 50, 75 and 100 mA cm⁻² are reproduced by this formula with z = 2 to within 0.05 points, so z = 2 is the electron count the authors used. The same formula was then applied to the headline numbers, to the 3.2 V point and to the rows of Supplementary Table 2 (with standard electron counts for each by-product formed from ethylene). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| FE at 345 μmol cm⁻² h⁻¹, 50 mA cm⁻² (z = 2) | Abstract; Introduction; Discussion | 45.6% | 36.99% |
| FE at 345 μmol cm⁻² h⁻¹, 50 mA cm⁻², Al-foil MEA | Results text (Fig. 5g) | 37% | 36.99% (consistent) |
| FE at 345.16 μmol cm⁻² h⁻¹, 50 mA cm⁻² | Source Data, sheet Figure 5g | 37.05% | 37.00% (consistent) |
| FE at 25, 75, 100 mA cm⁻² | Source Data, sheet Figure 5g | 42.63%, 21.72%, 10.91% | 42.62%, 21.72%, 10.91% (consistent) |
| EO rate that 50 mA cm⁻² at FE 45.6% would give | derived | 345 μmol cm⁻² h⁻¹ | 425 μmol cm⁻² h⁻¹ |
| Current density at which 345 μmol cm⁻² h⁻¹ is FE 45.6% | derived | 50 mA cm⁻² | 40.6 mA cm⁻² |
| Electron count that would make the Abstract triple consistent | derived | z = 2 | 2.47 (not a valid electron count for EO) |
| Current density implied by 100 μmol cm⁻² h⁻¹ at FE 45.6% (3.2 V) | Results text (Fig. 5f) | "approximately 10 mA cm-2" | 11.8 mA cm⁻² (consistent with "approximately 10") |
| Current density implied by the 3.2 V row, 101.45 at 45.54% | Source Data, sheets Figure 5f and 5g (10 mA row) | 10 mA cm⁻² (5g row label) | 11.9 mA cm⁻²; at exactly 10 mA cm⁻² this rate would be FE 54.4% |
| Total current density implied by each row of Supplementary Table 2 (EO z = 2, CO2 6, CO 4, CH2O 2, CH3CHO 2) | Supplementary Table 2 | not stated | 40.1 to 40.9 mA cm⁻² for all five rows (EO 40.55); none fits 50 |
| FEs of the Supplementary Table 2 rows if the current density is 50 mA cm⁻² | Supplementary Table 2 | 45.6, 7.0, 2.3, 1.3, 1.0% | 37.0, 5.7, 1.8, 1.1, 0.8% |
| EO partial current density carried by 345.16 μmol cm⁻² h⁻¹ | Supplementary Table 3 | 50 mA cm⁻² ("EO partial current density") | 18.5 mA cm⁻²; a 50 mA cm⁻² EO partial current would mean 933 μmol cm⁻² h⁻¹ |
| FE if 345 were per 4 cm² electrode with j per cm² | area check | — | 9.2% (cannot give 45.6%) |
| FE overstatement at the headline condition | Abstract vs Results | 45.6% vs 37% | 8.6 points; 23.2% relative |
The 45.6% figure is reproduced at only one current density, about 11.8 mA cm⁻² (the 3.2 V point), and, within Supplementary Table 2, by a charge basis of about 40.6 mA cm⁻². It is not reproduced at the 50 mA cm⁻² that the Abstract, the Introduction and the Discussion name. Supplementary Table 2 is internally consistent: each of its five FE values for products with a net electron transfer points to the same total current density, 40.1 to 40.9 mA cm⁻², which agrees with neither 50 mA cm⁻² nor the 10 to 11.8 mA cm⁻² of the 3.2 V point. The paper does not say which current the table refers to. (For CH3OH, whose formation from ethylene involves no net electron transfer, the listed FE implies about 2 electrons per molecule at that charge basis; this row was left out of the check.)
Figure
Figure 1. The two MEA conditions in the Results, and what the Abstract takes from each. Left: the headline numbers repeated in the Abstract, the Introduction and the Discussion. Right, top: the Al-foil MEA at 50 mA cm⁻² (Fig. 5g), which has the same rate and current density as the Abstract but an FE of 37%. Recomputing from 345 μmol cm⁻² h⁻¹ and 50 mA cm⁻² with z = 2 gives 36.99%. Right, bottom: the initial MEA at 3.2 V (Fig. 5f), which has the Abstract's FE of 45.6% at 100 μmol cm⁻² h⁻¹. That pair implies 11.8 mA cm⁻², consistent with the text's "approximately 10 mA cm-2". Reported values are labelled "Reported" (orange) and values recomputed from the paper's own numbers are labelled "Recomputed" (blue). The Abstract's triple would need 425 μmol cm⁻² h⁻¹ at 50 mA cm⁻², or 40.6 mA cm⁻² for 345 μmol cm⁻² h⁻¹.
Figure 2. Faradaic efficiency stated for the 345 μmol cm⁻² h⁻¹, 50 mA cm⁻² condition, by location in the paper, on one axis starting at 0. Orange circles: values reported in the paper (Abstract, Introduction and Discussion, 45.6%; Supplementary Tables 2 and 3, 45.6%; Results text, 37%; Source Data sheet Figure 5g, 50 mA cm⁻² row, 37.05%). Blue diamond: FE recomputed from the paper's rate and current density with Faraday's law and z = 2 (36.99%). The dashed line marks the 40% level that the Introduction gives for the field ("typically result in low FE (<40%)"). The reported 45.6% lies above that line, and the recomputed value lies below it.
Why it matters
- It is the paper's headline performance number. The FE of 45.6% appears in the Abstract, in the Introduction's summary of the work and in the Discussion, each time attached to the 345 μmol cm⁻² h⁻¹, 50 mA cm⁻², 60 h condition. A reader of any of these sections would conclude that high rate, practical current density and 45.6% FE were reached together. The paper's own Results, Source Data and Faraday's law show 37% for that condition, and the 45.6% point was measured at about one-fifth of the current density and less than one-third of the rate.
- It changes the position relative to the stated limitation of the field. The Introduction describes direct epoxidation as typically limited to "low FE (<40%)". At the headline condition the recomputed and Results value, 37%, falls below that threshold, while the Abstract's 45.6% lies above it.
- It affects the comparison with earlier work. Supplementary Table 3 lists "This work" with FE 45.6% alongside 50 mA cm⁻² and 345.16 μmol/h, and the Results state that the catalyst is "surpassing previously reported results in FE, production, current density and long-term stability". A comparison of FE at the 50 mA cm⁻² operating point should use 37%. Supplementary Table 3 also labels 50 mA cm⁻² as the "EO partial current density". The EO partial current density at 345.16 μmol cm⁻² h⁻¹ is 18.5 mA cm⁻², so that column entry is not consistent with the rate either.
- The 60 h stability claim does not carry an FE. The Source Data for Fig. 5h contain cell voltage only. The paper therefore reports no FE measured over the 60 h run; the "60 h with FE 45.6%" wording has no FE data behind it in the Source Data.
- What is not affected. The mechanistic and catalyst-design conclusions (strain-gradient interface, Ag-to-CuO electron transfer, OO* stabilization) do not depend on this number. The production rate of 345 μmol cm⁻² h⁻¹ at 50 mA cm⁻² is consistent throughout. The correction changes the FE attached to the headline condition, not the direction of the findings.
- A separate observation, not part of this finding. The selectivity of 92.8% quoted with the headline triple could not be reproduced. Methods Eq. 6 defines selectivity as moles of EO over moles of all ethylene-derived products. Applied to the production row of Supplementary Table 2, it gives 85.1% (ethylene glycol, which Eq. 6 includes, is not listed and would lower this further). The table does not state the condition it refers to, so this was not pursued.
Explanations considered
- A different operating condition. This is what the numbers show, but it does not make the headline correct. The Results describe two MEA conditions. The initial MEA reached its best FE, 45.6%, at 3.2 V with 100 μmol cm⁻² h⁻¹ at about 10 mA cm⁻²; that pair is self-consistent (11.8 mA cm⁻²). The Al-foil MEA gave 345 μmol cm⁻² h⁻¹ at 50 mA cm⁻² with FE 37%. The Abstract, the Introduction and the Discussion combine the rate and current density of the second with the FE of the first.
- A different electron count or FE definition. Does not account for it. Methods Eq. 5 is the standard FE = n z F / (I t). With z = 2, the same formula reproduces the Source Data FE values at 25, 50, 75 and 100 mA cm⁻² to within 0.05 points, so the authors' own calculation uses z = 2. The headline triple would require z = 2.47, which is not an integer and not an electron count for any EO pathway.
- Area normalization (per electrode rather than per cm²). Does not account for it. The working electrode is 2 × 2 cm². If 345 were μmol h⁻¹ for the whole electrode while j is per cm², FE would be 9.2%, further from 45.6%. The Source Data and Supplementary Table 2 give production in μmol/h without "cm⁻²", but the Source Data FE values are reproduced only on a per-cm² basis.
- A peak or best value during the 60 h run. Not supported. The paper never says that 45.6% was reached during the 60 h test, and the Fig. 5h Source Data contain no FE. The Results tie 45.6% to the 3.2 V point and 37% to the 50 mA cm⁻² system.
- Supplementary Table 2 as the source of the headline. It explains where the pairing appears but does not resolve it. The table pairs 345 with 45.6% and gives no current density. Its five by-product rows with net electron transfer are mutually consistent with a total current density of 40.1 to 40.9 mA cm⁻², not with 50. At 50 mA cm⁻² the same production values give EO 37.0% and by-product FEs of 5.7, 1.8, 1.1 and 0.8%. The table is therefore not consistent with the Fig. 5g Source Data for the 50 mA cm⁻² condition. The paper gives no current density near 40 mA cm⁻² for the CuO/Ag MEA.
- Rounding or typesetting. Ruled out. The gap is 8.6 points, far beyond rounding. The numbers are identical in the PMC text, the Europe PMC XML and the publisher's HTML. The supplementary tables and the Peer Review File print 45.6 as a plain decimal, and the Source Data hold the FE values as plain numbers (37.05 and 45.54).
- A published correction. None found (see Status).
Status
- Version of record. Checked on the publisher's page (nature.com, reached through doi.org; published 11 May 2026). The Abstract, Introduction, Results and Discussion sentences quoted above have the same numbers there. The Europe PMC full-text XML and the PMC archive agree. The page carries no "Change history" section and no Author or Publisher Correction notice.
- Supplements. All five ESM files on the publisher page were opened in the version-of-record check: Supplementary Information PDF (40 pp; Supplementary Tables 2 and 3 used here), description of additional files, Supplementary Data 1 (atomic coordinates only), the Peer Review File and the Source Data workbook (sheets Figure 5f, 5g and 5h used here). For this write-up the Supplementary Information, the Source Data and the Peer Review File were downloaded again and re-read. The Peer Review File reproduces Supplementary Table 2 with the same 345 / 45.6% pairing, and the reviewer comments describe the result as "345 μmol·cm-2·h-1 over 60 hours with high Faradaic efficiency and selectivity"; the discrepancy is not discussed there.
- Corrections and comments. As of 2026-10-24:
src/known.pyfinds no Crossref update, no PMC notice and no Europe PMC notice. The Crossref record has no update-to or updated-by entries and an empty relation field. The PubPeer API returned no record (HTTP 404) for the DOI, and web searches for the title with "PubPeer", "correction" or "Retraction Watch" found no thread or notice. - Sources. PMC13377095 (via
src/show.py); https://doi.org/10.1038/s41467-026-72987-4 and its ESM files (MOESM1 Supplementary Information, MOESM4 Peer Review File, MOESM5 Source Data); Europe PMC; Crossref; PubPeer.
Suggested correction
The Abstract, the closing paragraph of the Introduction and the Discussion could attach the FE measured at the stated condition, for example:
In a membrane electrode assembly reactor, a stable ethylene oxide production rate of 345 μmol·cm-2·h-1 is achieved at 50 mA·cm-2 over 60 hours, with a Faradaic efficiency of 37%. A maximum Faradaic efficiency of 45.6% is obtained at a lower current density (about 10 mA·cm-2; 100 μmol·cm-2·h-1).
The selectivity quoted alongside (92.8%) should be checked and stated for the same operating point. Supplementary Table 2 could state the current density it refers to and, if it is meant to describe the 50 mA cm⁻² condition, be recomputed with that current (EO FE 37.0%). In Supplementary Table 3, the "This work" row could give FE 37% for the 50 mA cm⁻² / 345.16 μmol cm⁻² h⁻¹ entry, and the 50 mA cm⁻² column could be relabelled as total current density (the EO partial current density is 18.5 mA cm⁻²). The Results sentence on "surpassing previously reported results in FE" could then be checked against the corrected value.
Recomputation script (python recompute.py.txt)
"""Recompute the headline Faradaic efficiency of PMC13377095
(Nat Commun 2026, doi:10.1038/s41467-026-72987-4).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg, fig1.png, fig2.svg and fig2.png next to this file.
All inputs are numbers printed in the paper (Abstract, Introduction, Results,
Discussion, Methods), its Supplementary Information (Supplementary Tables 2
and 3) and its Source Data file (sheets 'Figure 5f' and 'Figure 5g').
Faradaic efficiency, Methods Eq. 5: FE = n z F / (I t).
Per unit electrode area and per hour: FE = r z F / (3600 j),
with r in mol cm-2 h-1 and j in A cm-2.
Direct water-mediated epoxidation: C2H4 + H2O -> C2H4O + 2 H+ + 2 e-, z = 2.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
F = 96485.332 # C mol-1
Z_EO = 2
def fe(rate_umol, j_mA, z=Z_EO):
"""FE (fraction) for a rate in umol cm-2 h-1 at a current density in mA cm-2."""
return rate_umol * 1e-6 * z * F / (3600 * j_mA * 1e-3)
def j_needed(rate_umol, fe_frac, z=Z_EO):
"""Total current density (mA cm-2) at which rate_umol corresponds to fe_frac."""
return rate_umol * 1e-6 * z * F / 3600 / fe_frac * 1e3
def rate_needed(j_mA, fe_frac, z=Z_EO):
"""EO rate (umol cm-2 h-1) that j_mA at fe_frac would give."""
return fe_frac * j_mA * 1e-3 * 3600 / (z * F) * 1e6
def partial_j(rate_umol, z=Z_EO):
"""Partial current density (mA cm-2) carried by a product made at rate_umol."""
return rate_umol * 1e-6 * z * F / 3600 * 1e3
# ------------------------------------------------------------------ inputs
HEAD_RATE, HEAD_J, HEAD_FE = 345.0, 50.0, 0.456 # Abstract, Introduction, Discussion
RES_FE_5G = 0.37 # Results: "relatively high FE of 37% (Fig. 5g)"
RES_RATE_5F, RES_FE_5F, RES_J_5F = 100.0, 0.456, 10.0 # Results: 3.2 V, "approximately 10 mA cm-2"
ELECTRODE_AREA = 2 * 2 # Methods: work electrode 2 x 2 cm2
FIELD_FE = 0.40 # Introduction: "typically result in low FE (<40%)"
# Source Data, sheet 'Figure 5g': (current density mA/cm2, EO production, FE-EO %)
SD_5G = [(10, 101.45, 45.54), (25, 198.77, 42.63), (50, 345.16, 37.05),
(75, 303.96, 21.72), (100, 203.61, 10.91)]
# Source Data, sheet 'Figure 5f', 3.2 V row: (EO production, FE-EO %)
SD_5F_32V = (101.45, 45.54)
# Supplementary Table 2: product, production (umol/h), FE (%), selectivity (%), and the
# electrons per product molecule for its formation from ethylene (standard half-reactions):
# EO C2H4 + H2O -> C2H4O + 2H+ + 2e- (2 per EO)
# CO2 C2H4 + 4H2O -> 2CO2 + 12H+ + 12e- (6 per CO2)
# CO C2H4 + 2H2O -> 2CO + 8H+ + 8e- (4 per CO)
# CH2O C2H4 + 2H2O -> 2CH2O + 4H+ + 4e- (2 per CH2O)
# CH3CHO C2H4 + H2O -> CH3CHO + 2H+ + 2e- (2 per CH3CHO)
# CH3OH C2H4 + 2H2O -> 2CH3OH (0: no net electron transfer; left out)
ST2 = [("C2H4O", 345.0, 45.6, 92.8, 2), ("CO2", 17.8, 7.0, 4.78, 6), ("CO", 8.6, 2.3, 0.14, 4),
("CH3OH", 16.5, 2.1, 0.17, None), ("CH2O", 9.8, 1.3, 0.08, 2), ("CH3CHO", 7.6, 1.0, 2.03, 2)]
# Supplementary Table 3, 'This work': FE 45.6 %, 'EO partial current density' 50 mA/cm2,
# stability 60 h, selectivity 92.8 %, EO production 345.16 umol/h
ST3_FE, ST3_PARTIAL_J, ST3_RATE = 0.456, 50.0, 345.16
# ------------------------------------------------------------------ recompute
print("== Headline condition (Abstract / Introduction / Discussion)")
fe_head = fe(HEAD_RATE, HEAD_J)
print(f"FE implied by 345 umol cm-2 h-1 at 50 mA cm-2 (z = 2): {100 * fe_head:.2f} %")
print(f"Source Data rate 345.16 at 50 mA cm-2: {100 * fe(345.16, 50):.2f} % (sheet lists 37.05 %)")
print(f"Current density at which 345 umol cm-2 h-1 would be FE 45.6 %: {j_needed(HEAD_RATE, HEAD_FE):.2f} mA cm-2")
print(f"EO rate that 50 mA cm-2 at FE 45.6 % would give: {rate_needed(HEAD_J, HEAD_FE):.1f} umol cm-2 h-1")
print(f"EO rate that 50 mA cm-2 at FE 37 % would give: {rate_needed(HEAD_J, RES_FE_5G):.1f} umol cm-2 h-1")
z_req = HEAD_FE * HEAD_J * 1e-3 * 3600 / (HEAD_RATE * 1e-6 * F)
print(f"Electron count z that would make 345 / 50 / 45.6 % consistent: {z_req:.2f}")
print(f"Overstatement: {100 * (HEAD_FE - RES_FE_5G):.1f} percentage points vs 37 %; "
f"relative {100 * (HEAD_FE / RES_FE_5G - 1):.1f} % "
f"({100 * (HEAD_FE - fe_head):.1f} points vs the recomputed {100 * fe_head:.2f} %)")
print(f"EO partial current density at 345 umol cm-2 h-1: {partial_j(HEAD_RATE):.2f} mA cm-2")
print("\n== Initial MEA optimum, 3.2 V (Results, Fig. 5f)")
print(f"Current density implied by 100 umol cm-2 h-1 at FE 45.6 %: {j_needed(RES_RATE_5F, RES_FE_5F):.2f} mA cm-2 (text: ~10)")
print(f"Source Data 3.2 V row, 101.45 at 45.54 %: implies {j_needed(SD_5F_32V[0], SD_5F_32V[1] / 100):.2f} mA cm-2")
print(f"FE if 100 umol cm-2 h-1 were at exactly 10 mA cm-2: {100 * fe(RES_RATE_5F, RES_J_5F):.1f} %")
print("\n== Area normalisation (electrode 2 x 2 cm2)")
print(f"FE if 345 were umol h-1 per 4 cm2 electrode and 50 mA per cm2: {100 * fe(HEAD_RATE / ELECTRODE_AREA, HEAD_J):.1f} %")
print("\n== Source Data, sheet 'Figure 5g' (current density, EO production, listed FE -> recomputed FE)")
for j, r, fl in SD_5G:
print(f" {j:>5} mA cm-2 {r:7.2f} listed {fl:5.2f} % recomputed {100 * fe(r, j):5.2f} %"
f" (listed FE implies j = {j_needed(r, fl / 100):5.2f} mA cm-2)")
print("\n== Supplementary Table 2 (345 umol/h, FE 45.6 %): total current density implied by each row")
implied = []
for name, r, fl, sel, z in ST2:
if z is None:
z_fit = fl / 100 * j_needed(HEAD_RATE, HEAD_FE) * 1e-3 * 3600 / (r * 1e-6 * F)
print(f" {name:7s} {r:6.1f} umol/h FE {fl:4.1f} % no net electrons from C2H4; "
f"listed FE implies z = {z_fit:.2f} at 40.55 mA cm-2")
continue
jj = j_needed(r, fl / 100, z)
implied.append(jj)
print(f" {name:7s} {r:6.1f} umol/h FE {fl:4.1f} % z = {z} implies j = {jj:5.2f} mA cm-2;"
f" at 50 mA cm-2 FE would be {100 * fe(r, 50, z):4.1f} %")
print(f" range of implied total current density: {min(implied):.1f} to {max(implied):.1f} mA cm-2")
tot = sum(r for _, r, _, _, _ in ST2)
print(f" Eq. 6 selectivity of EO from the listed production row: {100 * 345.0 / tot:.1f} % (table: 92.8 %)")
print("\n== Supplementary Table 3 ('This work')")
print(f" If 50 mA cm-2 were the EO partial current density: EO rate {rate_needed(ST3_PARTIAL_J, 1.0):.0f}"
f" umol cm-2 h-1, total j at FE 45.6 % = {ST3_PARTIAL_J / ST3_FE:.1f} mA cm-2")
print(f" EO partial current density that 345.16 umol cm-2 h-1 carries: {partial_j(ST3_RATE):.2f} mA cm-2")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "svg.fonttype": "none"})
def save(fig, stem):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{stem}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: the abstract pairs numbers from two different Results conditions.
fig = plt.figure(figsize=(7.2, 4.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
def panel(x0, y0, w, h, title):
ax.add_patch(FancyBboxPatch((x0, y0), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=INK2, lw=1.0))
ax.text(x0 + 0.015, y0 + h - 0.03, title, ha="left", va="top", fontsize=10.5,
weight="bold", color=INK)
# left: Abstract
LX, LW = 0.02, 0.33
panel(LX, 0.40, LW, 0.50, "Abstract\n(same in Introduction\nand Discussion)")
ax.text(LX + 0.015, 0.73, "Reported:", fontsize=10, color=INK2, va="top")
ax.text(LX + 0.015, 0.675, "345 μmol cm⁻² h⁻¹\nat 50 mA cm⁻², 60 h", fontsize=11, color=ORANGE,
va="top", weight="bold", linespacing=1.3)
ax.text(LX + 0.015, 0.525, "FE 45.6%", fontsize=11, color=ORANGE, va="top", weight="bold")
RX, RW = 0.60, 0.38
# right top: Fig. 5g point (same rate and current as the Abstract)
panel(RX, 0.53, RW, 0.43, "Results, Fig. 5g\nAl-foil MEA, 50 mA cm⁻²")
ax.text(RX + 0.015, 0.80, "Reported:", fontsize=10, color=INK2, va="top")
ax.text(RX + 0.015, 0.755, "345 μmol cm⁻² h⁻¹, FE 37%", fontsize=10.5, color=ORANGE,
va="top", weight="bold")
ax.text(RX + 0.015, 0.69, f"Recomputed (z = 2):\nFE = {100 * fe_head:.2f}%", fontsize=10,
color=BLUE, va="top", linespacing=1.3)
# right bottom: Fig. 5f point (same FE as the Abstract)
panel(RX, 0.04, RW, 0.43, "Results, Fig. 5f\ninitial MEA, 3.2 V, ~10 mA cm⁻²")
ax.text(RX + 0.015, 0.31, "Reported:", fontsize=10, color=INK2, va="top")
ax.text(RX + 0.015, 0.265, "100 μmol cm⁻² h⁻¹, FE 45.6%", fontsize=10.5, color=ORANGE,
va="top", weight="bold")
ax.text(RX + 0.015, 0.20, f"Recomputed: FE 45.6% at 100 μmol\nneeds {j_needed(RES_RATE_5F, RES_FE_5F):.1f} mA cm⁻² (fits ~10)",
fontsize=10, color=BLUE, va="top", linespacing=1.3)
# connectors
arrow_kw = dict(arrowstyle="-|>", mutation_scale=14, lw=2, color=INK2)
ax.add_patch(FancyArrowPatch((LX + LW, 0.645), (RX, 0.745), connectionstyle="arc3,rad=-0.1", **arrow_kw))
ax.add_patch(FancyArrowPatch((LX + LW, 0.51), (RX, 0.255), connectionstyle="arc3,rad=0.1", **arrow_kw))
ax.text(0.475, 0.745, "same rate\nand current", ha="center", va="bottom", fontsize=10, color=INK2,
linespacing=1.2)
ax.text(0.495, 0.47, "same FE", ha="left", va="center", fontsize=10, color=INK2)
# conflict note
ax.text(LX, 0.355,
"Conflict: the Abstract joins the rate\nand current of Fig. 5g to the FE of\nFig. 5f. "
f"At 50 mA cm⁻², FE 45.6%\nwould need {rate_needed(HEAD_J, HEAD_FE):.0f} μmol cm⁻² h⁻¹; "
f"345 μmol\ncm⁻² h⁻¹ at 45.6% would need\n{j_needed(HEAD_RATE, HEAD_FE):.1f} mA cm⁻² "
f"(or z = {z_req:.2f}).",
fontsize=10, color=INK, va="top", linespacing=1.35)
save(fig, "fig1")
# Figure 2: FE stated for the 345 umol / 50 mA condition, by location, vs Faraday's law.
rows = [("Abstract, Introduction,\nDiscussion", 45.6, "rep"),
("Supplementary Tables 2 and 3", 45.6, "rep"),
("Results text (Fig. 5g)", 37.0, "rep"),
("Source Data, Fig. 5g,\n50 mA cm⁻² row", 37.05, "rep"),
("Faraday's law, z = 2,\n345 μmol cm⁻² h⁻¹ at 50 mA cm⁻²", round(100 * fe_head, 2), "rec")]
fig, axf = plt.subplots(figsize=(7.2, 3.5), facecolor=BG)
axf.set_facecolor(BG)
ys = list(range(len(rows)))[::-1]
for y, (lab, v, kind) in zip(ys, rows):
c, m = (ORANGE, "o") if kind == "rep" else (BLUE, "D")
axf.plot(v, y, marker=m, ms=9, color=c, zorder=3, ls="none")
left = v < 100 * FIELD_FE # keep labels clear of the 40% reference line
axf.text(v - 0.9 if left else v + 0.9, y, f"{v:g}%", va="center", ha="right" if left else "left",
fontsize=10, color=c, weight="bold")
axf.axvline(100 * FIELD_FE, color=INK2, lw=1, ls=(0, (4, 3)), zorder=0)
axf.text(100 * FIELD_FE - 0.6, len(rows) - 0.45, "Introduction: field\ntypically \"low FE (<40%)\"",
ha="right", va="top", fontsize=10, color=INK2, linespacing=1.2)
axf.set_yticks(ys)
axf.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
axf.set_xlim(0, 52)
axf.set_ylim(-0.6, len(rows) - 0.4)
axf.set_xlabel("FE for EO stated for 345 μmol cm⁻² h⁻¹ at 50 mA cm⁻² (%)", fontsize=10)
axf.grid(axis="x", color=GRID, lw=0.8)
axf.set_axisbelow(True)
for s in ("top", "right", "left"):
axf.spines[s].set_visible(False)
axf.tick_params(axis="y", length=0)
h_rep = axf.plot([], [], "o", ms=9, color=ORANGE, label="reported in the paper")[0]
h_rec = axf.plot([], [], "D", ms=9, color=BLUE, label="recomputed from the paper's\nrate and current")[0]
axf.legend(handles=[h_rep, h_rec], loc="upper center", bbox_to_anchor=(0.3, -0.2), ncol=2,
frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png")
Table 1 P values do not follow from its own mouse counts under the stated tests
Nat Commun (2017) · doi:10.1038/ncomms14382 · PMC5309857
Summary
Table 1 reports how many recipient mice were engrafted ("positive") after p190-B−/− LSK-SLAM cells were treated ex vivo: 9 of 16 untreated, 4 of 13 with rTGF-β1, and 13 of 20 with rTGF-β1 plus the p38MAPK inhibitor SB203580. It gives Fisher exact P values of 0.0006 and <0.0001 for the two pairwise comparisons, and a three-group χ² P<0.0001. With these counts, the stated tests give P = 0.264, P = 0.080 and χ² = 3.80 on 2 df (P = 0.149), so none of the three comparisons is significant at 0.05. All three reported values come out exactly if the rounded percentages (56%, 31%, 65%) are entered as counts out of 100 mice, as the footer's phrase "comparing per cent of negative and positive mice" suggests. Supplementary Table 3 shows the same pattern. Table 1 is the main-text in vivo test cited for the claim that rTGF-β1 "significantly reduced" long-term HSC activity through p38MAPK, so as printed that claim has no statistical support from these data.
What the paper reports
Table 1 ("Effect of TGFβ on p190-B−/− HSPC engraftment"). The columns are Chimera>1%, Chimera<1%, Total mice transplanted and P value:
p190-B-/- | 9 | 7 | 16 |
p190-B-/-+rTGFβ1 | 4 | 9 | 13 | 0.0006*
p190-B-/-+rTGFβ1+p38 inhi | 13 | 7 | 20 | < 0.0001†
Table 1
Table shows numbers of mice exhibiting more than 1% contribution to both myeloid and lymphoid lineages in PB (positive mice) and numbers of mice with less than 1% contribution in at least one lineage (negative mice) (2–3 independent experiments). P values were calculated by Fisher exact 2 × 2 contingency table by comparing per cent of negative and positive mice of the following groups: *p190-B-/- vs p190-B-/-+rTGF-β1, and †p190-B-/-+rTGF-β1 vs p190-B-/-+rTGF-β1+ SB203580. Difference between the three groups was analysed by χ2 test, P<0.0001.
Table 1, footer
How the Results use the table:
But, LT-HSC activity of pooled p190-B-/- LSK-SLAM cells cultured with rTGF-β1 was significantly reduced (30% of transplanted mice showing greater than 1% contribution to both myeloid and lymphoid lineages in PB versus 56% from non-treated cultures). This effect was prevented by addition of SB203580 (65% mice were engrafted (Table 1 and Supplementary Table 3).
Hence, rTGF-β1 can alter HSC activity via p38MAPK signalling pathway in vitro.
Results, section "p38MAPK pathway mediates TGF-β effect on HSPC activity"
Here, the authors show using both in vitro assays and in vivo transplantation that loss of the RhoGAP p190-B enhances self-renewal by inhibiting TGFβ/p38 signalling.
Abstract (editor's summary)
The statistics section names only the paired-daughter-cell and single-cell analyses ("In vitro paired daughter cell assay data were analysed by 2 × 2 Fisher exact test contingency table", Methods, Statistical analyses). It does not describe any other analysis of the transplantation counts.
Supplementary Table 3 (fetal-liver LSK-SLAM cells, from the Supplementary Information PDF) reports 8/4 (n = 12) untreated, 2/10 (n = 12) with rTGF-β1, P "<0.0001", and 4/6 (n = 10) with rTGF-β1 plus the p38 inhibitor, P "0.0005". Its footer reads: "P value was calculated by Fisher exact test ... P value by chi square test was < 0.0001 in both sets of transplant."
What the paper's own numbers show
Method. Each comparison named in the footer was set up as a 2 × 2 table of positive and negative mice and analysed with the two-sided Fisher exact test (scipy). As sensitivity checks I also ran the one-sided Fisher test, the χ² test with and without Yates correction, and Barnard's and Boschloo's unconditional exact tests. The three-group comparison is the Pearson χ² test on the 3 × 2 table. For each pairwise comparison I computed the difference in per cent positive with a Newcombe (hybrid Wilson score) 95% CI. To test the footer's wording, I then ran the same tests on the rounded percentages entered as if they were counts out of 100. Supplementary Table 3 was treated the same way. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Per cent positive, p190-B−/− (9/16) | Table 1; Results | 56% | 56.2% |
| Per cent positive, +rTGF-β1 (4/13) | Table 1; Results | 30% | 30.8% |
| Per cent positive, +rTGF-β1+SB203580 (13/20) | Table 1; Results | 65% | 65.0% |
| Fisher, p190-B−/− vs +rTGF-β1 (9/7 vs 4/9) | Table 1, * | P = 0.0006 | P = 0.264 two-sided (one-sided 0.160; χ² 0.170, Yates 0.319; Barnard 0.184; Boschloo 0.192) |
| Difference in per cent positive, same comparison | derived | — | +25.5 points, 95% CI −9.9 to +52.9 |
| Fisher, +rTGF-β1 vs +rTGF-β1+SB203580 (4/9 vs 13/7) | Table 1, † | P < 0.0001 | P = 0.080 two-sided (one-sided 0.058; χ² 0.055, Yates 0.117; Barnard 0.063; Boschloo 0.069) |
| Difference in per cent positive, same comparison | derived | — | −34.2 points, 95% CI −59.0 to +0.3 |
| Three-group χ², 3 × 2 | Table 1 footer | P < 0.0001 | χ² = 3.80, df 2, P = 0.149 |
| Multiplier of the real counts needed to reach the reported P | derived | — | ×7 (first comparison), ×5 (second) |
| Fisher on % as counts, 56/44 vs 31/69 | diagnostic | 0.0006 | P = 5.8 × 10⁻⁴ (prints as 0.0006) |
| Fisher on % as counts, 31/69 vs 65/35 | diagnostic | < 0.0001 | P = 2.4 × 10⁻⁶ |
| 3 × 2 χ² on % as counts | diagnostic | < 0.0001 | χ² = 24.8, P = 4.1 × 10⁻⁶ |
| Fisher, Supp. Table 3: 8/4 vs 2/10 | Supp. Table 3 | P < 0.0001 | P = 0.036 (on % as counts, 67/33 vs 17/83: 6.2 × 10⁻¹³) |
| Fisher, Supp. Table 3: 2/10 vs 4/6 | Supp. Table 3 | P = 0.0005 | P = 0.348 (on % as counts, 17/83 vs 40/60: 5.06 × 10⁻⁴, prints as 0.0005) |
| 3 × 2 χ², Supp. Table 3 | Supp. Table 3 footer | P < 0.0001 | χ² = 6.20, df 2, P = 0.045 (on % as counts: 6.1 × 10⁻¹²) |
The cells, row totals (9 + 7 = 16, 4 + 9 = 13, 13 + 7 = 20), the percentages in the Results text and the footer all agree with each other. The discrepancy lies only in the P values. Each reported value sits exactly where the stated test lands when the per cent figures are treated as counts out of 100, which treats each group as about 5 to 8 times larger than it was (100 versus 13 to 20 mice). The match is specific. For the first comparison, 4/13 rounds to 31%, and 56/44 vs 31/69 gives 0.00058, which prints as the reported 0.0006. Using the 30% from the Results text instead gives 0.00033.
Figure
Figure 1. P values on a log scale for the three tests in Table 1 (top) and the three in Supplementary Table 3 (bottom). The thin vertical line marks p = 0.05. Orange diamonds are the values the paper prints; an arrow means the paper gives only an upper bound ("<0.0001"). Filled blue circles are the stated test (two-sided Fisher exact for pairs, Pearson χ² for three groups) applied to the mouse counts printed in the same table. Open blue squares are the same test applied to the rounded per cent positive and negative entered as if each group had 100 mice. In every row the reported value coincides with, or is a correct bound for, the open square, not the filled circle. On the actual counts, all three Table 1 tests are above 0.05.
Figure 2. Calculation flow for the first Table 1 comparison (p190-B−/− vs +rTGF-β1). Upper path: the two-sided Fisher exact test on the printed numbers of mice (9 positive / 7 negative vs 4 / 9) gives p = 0.264, which is 440 times the reported P = 0.0006 (orange box). Lower path: the same test on the rounded per cent positive and negative (56 / 44 vs 31 / 69), entered as if each group had 100 mice, gives p = 0.00058, which prints as the reported 0.0006. The line at the bottom gives the same pair of values for the other two Table 1 tests, both printed in the paper as P < 0.0001. Blue marks values recomputed from the paper's own numbers; orange marks the paper's reported value.
Why it matters
Table 1 is the paper's main-text in vivo functional test of the step from TGF-β through p38MAPK to long-term HSC activity. The Results cite it, together with Supplementary Table 3, for the statements that rTGF-β1 "significantly reduced" LT-HSC activity, that the effect "was prevented by addition of SB203580", and hence that "rTGF-β1 can alter HSC activity via p38MAPK signalling pathway". The editor's summary relies on "in vivo transplantation" for the TGFβ/p38 conclusion.
On the printed counts, neither pairwise comparison in Table 1 is significant, and the three-group test gives P = 0.149. The intervals show how little the data pin down. The rTGF-β1 reduction (+25.5 percentage points) is compatible with anything from a 10-point increase to a 53-point decrease, and the SB203580 rescue (−34.2 points) has an upper CI limit just above zero. The direction of the effects (56% → 31% → 65%) is as described. What the table does not support is the statement that the differences are significant, let alone at P ≤ 0.0006.
Supplementary Table 3 does not supply the missing support at the stated level. Its rTGF-β1 effect is nominally significant on the counts (Fisher P = 0.036, against the printed <0.0001), its p38-inhibitor rescue is not (P = 0.348, against the printed 0.0005), and its three-group χ² gives P = 0.045 (printed <0.0001). Pooling the two tables, which the paper does not describe, gives P = 0.012, 0.027 and a three-group P = 0.014. These values are shown for scale only and are not a reanalysis the paper reports.
The abstract itself does not quote these P values, and the paper's p190-B/TGF-β/p38 conclusions also rest on other experiments (paired-daughter-cell assays, p-p38MAPK and bioactive TGF-β measurements, and TGFBRI-inhibitor experiments). The same wording, "comparing per cent", appears in the Fisher-test legends of Figures 3, 7 and 9. I have not checked whether those tests have the same problem.
Explanations considered
- One-sided test. The footer does not say whether the tests were one- or two-sided. One-sided Fisher P values are 0.160 and 0.058, still far from 0.0006 and <0.0001. This does not account for the numbers.
- A different 2 × 2 test. Uncorrected χ², Yates-corrected χ², Barnard's and Boschloo's tests give P between 0.055 and 0.32 for the two Table 1 comparisons. This does not account for the numbers.
- Different or pooled data. The footer ties the P values to the Table 1 groups, and the Results percentages (56%, 30%, 65%) match the Table 1 counts. Supplementary Table 3 is a separate fetal-liver experiment with its own P values. Pooling it with Table 1 gives P = 0.012 and 0.027, nowhere near the reported values. Reaching the reported P values from these proportions would need each count multiplied by about 5 to 7. This does not account for the numbers.
- Misaligned cells, lost symbols or conversion errors. The PMC archive, the Europe PMC full text and the publisher's Table 1 page carry the same cells, totals, P values and footer. The row totals add up. The P values are plain decimals with no lost minus signs, superscripts or exponents. This does not account for the numbers.
- Correction already published. None was found (see Status).
- Test run on percentages rather than counts. The footer says the test compared "per cent of negative and positive mice". Entering the rounded percentages as counts out of 100 reproduces all three Table 1 values (5.8 × 10⁻⁴, 2.4 × 10⁻⁶, 4.1 × 10⁻⁶) and both numeric Supplementary Table 3 values (0.0005, and 6.2 × 10⁻¹³ under "<0.0001"). This very likely explains how the printed numbers arose. It does not make them valid: a Fisher or χ² test on percentages treats each group as 100 independent mice instead of the 10 to 20 actually transplanted, so it overstates the evidence. It explains the reported values but does not support them.
Status
As of 2026-10-23:
- Version of record. The publisher's Table 1 page (nature.com/articles/ncomms14382/tables/1, fetched 2026-10-23) shows the same counts, P values (0.0006*, < 0.0001†) and footer text as the PMC archive. The main article page carries the same Results sentence. The discrepancy is in the published text.
- Supplements. The Supplementary Information PDF (11 pages, from static-content.springer.com) was read. Supplementary Table 3 is a separate fetal-liver experiment, and it shows the same pattern as Table 1 (see above). Supplementary Data 1 and 2 are gene-expression lists and not relevant.
- Corrections and notices. Crossref has no
update-to,updated-byorrelationentries for the DOI. Neither PMC nor Europe PMC (PMID 28176763) lists a correction, retraction or expression of concern. The publisher's article page links to no correction. - PubPeer. A web search for the title and DOI found no PubPeer thread. The PubPeer API returned HTTP 403, so no direct query of its database was possible.
- Sources: PMC5309857 (Table 1, Results, Methods, editor's summary); https://www.nature.com/articles/ncomms14382 and its Table 1 page; Supplementary Information PDF (41467_2017_BFncomms14382_MOESM1632_ESM.pdf); api.crossref.org/works/10.1038/ncomms14382; Europe PMC REST search for DOI:10.1038/ncomms14382.
Suggested correction
Recompute the Table 1 P values on the numbers of mice (9/7, 4/9 and 13/7) rather than on percentages. With two-sided Fisher exact tests this gives P = 0.26 for p190-B−/− vs +rTGF-β1 and P = 0.080 for +rTGF-β1 vs +rTGF-β1+SB203580, and the three-group χ² gives χ² = 3.80, df 2, P = 0.15. Replace the footer's "comparing per cent of negative and positive mice" with "comparing numbers of negative and positive mice". Recompute Supplementary Table 3 the same way (P = 0.036, 0.35 and χ² P = 0.045). Revise the Results sentence "was significantly reduced" and the inference that the effect was "prevented" by SB203580 to match the recomputed values, for example by reporting the observed proportions with confidence intervals. The other Fisher tests whose legends use "per cent" (Figures 3, 7 and 9) should be checked to confirm they were run on counts.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC5309857 (Nat Commun 2017, doi:10.1038/ncomms14382), Table 1
# and Supplementary Table 3. Run: python recompute.py.txt
# Prints every number used in writeup.md and writes fig1.svg / fig1.png next to this file.
import os
import numpy as np
from scipy.stats import fisher_exact, chi2_contingency, barnard_exact, boschloo_exact, norm
HERE = os.path.dirname(os.path.abspath(__file__))
# ---- Counts as printed: (positive mice "Chimera>1%", negative mice "Chimera<1%")
T1 = {"KO": (9, 7), "KO+TGF": (4, 9), "KO+TGF+p38i": (13, 7)} # Table 1
S3 = {"KO": (8, 4), "KO+TGF": (2, 10), "KO+TGF+p38i": (4, 6)} # Supplementary Table 3
PAIRS = [("KO", "KO+TGF"), ("KO+TGF", "KO+TGF+p38i")]
REPORTED = { # what the paper prints (value, is_upper_bound)
("T1", "KO", "KO+TGF"): (0.0006, False),
("T1", "KO+TGF", "KO+TGF+p38i"): (0.0001, True),
("T1", "chi2"): (0.0001, True),
("S3", "KO", "KO+TGF"): (0.0001, True),
("S3", "KO+TGF", "KO+TGF+p38i"): (0.0005, False),
("S3", "chi2"): (0.0001, True),
}
def wilson(x, n, z=1.959964):
p = x / n
c = (p + z*z/(2*n)) / (1 + z*z/n)
h = z*np.sqrt(p*(1-p)/n + z*z/(4*n*n)) / (1 + z*z/n)
return c - h, c + h
def newcombe(x1, n1, x2, n2):
"""Newcombe hybrid-score 95% CI for p1 - p2."""
p1, p2 = x1/n1, x2/n2
l1, u1 = wilson(x1, n1); l2, u2 = wilson(x2, n2)
d = p1 - p2
return d, d - np.sqrt((p1-l1)**2 + (u2-p2)**2), d + np.sqrt((u1-p1)**2 + (p2-l2)**2)
def as_pct_counts(tab):
"""Rounded per cent positive / negative, i.e. counts out of 100."""
out = {}
for k, (a, b) in tab.items():
p = round(100*a/(a+b))
out[k] = (p, 100-p)
return out
results = {}
for name, tab in [("T1", T1), ("S3", S3)]:
print("=" * 72)
print({"T1": "Table 1 (1T p190-B-/- LSK-SLAM)", "S3": "Supplementary Table 3 (fetal liver)"}[name])
for k, (a, b) in tab.items():
print(f" {k:12s} {a}/{a+b} positive = {100*a/(a+b):.1f}%")
for g1, g2 in PAIRS:
t = np.array([tab[g1], tab[g2]])
p2 = fisher_exact(t)[1]
p1 = min(fisher_exact(t, alternative="less")[1], fisher_exact(t, alternative="greater")[1])
c_u, p_u, _, _ = chi2_contingency(t, correction=False)
_, p_y, _, _ = chi2_contingency(t, correction=True)
p_b = barnard_exact(t).pvalue
p_bo = boschloo_exact(t).pvalue
a1, b1 = tab[g1]; a2, b2 = tab[g2]
d, lo, hi = newcombe(a1, a1+b1, a2, a2+b2)
pc = as_pct_counts(tab)
p_pct = fisher_exact(np.array([pc[g1], pc[g2]]))[1]
# smallest integer multiplier of the real counts that reaches the reported value
rep, _ = REPORTED[(name, g1, g2)]
mult = next(m for m in range(1, 50) if fisher_exact(t*m)[1] <= rep)
results[(name, g1, g2)] = dict(fisher=p2, pct=p_pct)
print(f"\n {g1} vs {g2}: table {t.tolist()}")
print(f" reported P : {'<' if REPORTED[(name,g1,g2)][1] else ''}{rep}")
print(f" Fisher two-sided : {p2:.4f}")
print(f" Fisher one-sided (smaller): {p1:.4f}")
print(f" chi2 uncorrected : chi2={c_u:.3f}, p={p_u:.4f}")
print(f" chi2 Yates : p={p_y:.4f}")
print(f" Barnard / Boschloo : {p_b:.4f} / {p_bo:.4f}")
print(f" difference in % positive : {100*d:+.1f} points, Newcombe 95% CI {100*lo:+.1f} to {100*hi:+.1f}")
print(f" percentages as counts/100 : {[pc[g1], pc[g2]]} -> Fisher two-sided p={p_pct:.2e}")
print(f" real counts must be multiplied by {mult} to reach p <= {rep}")
t3 = np.array(list(tab.values()))
c3, p3, df3, _ = chi2_contingency(t3)
pc = as_pct_counts(tab)
c3p, p3p, _, _ = chi2_contingency(np.array(list(pc.values())))
results[(name, "chi2")] = dict(fisher=p3, pct=p3p)
print(f"\n 3x2 chi-square on counts : chi2={c3:.2f}, df={df3}, p={p3:.3f}")
print(f" 3x2 chi-square on % as counts : {list(pc.values())} -> chi2={c3p:.1f}, p={p3p:.1e}")
# Rounding check: the Results text says 30% for 4/13 (30.8%); 30/70 instead of 31/69
print("\nRounding sensitivity, Table 1 comparison 1 with 56/44 vs 30/70:",
f"p={fisher_exact(np.array([[56,44],[30,70]]))[1]:.2e}")
# Pooled Table 1 + Supplementary Table 3 (an analysis the paper does not describe)
pool = {k: (T1[k][0]+S3[k][0], T1[k][1]+S3[k][1]) for k in T1}
print("\nPooled Table 1 + Supp. Table 3 (not an analysis the paper describes):", pool)
for g1, g2 in PAIRS:
print(f" {g1} vs {g2}: Fisher two-sided p={fisher_exact(np.array([pool[g1], pool[g2]]))[1]:.3f}")
c, p, d, _ = chi2_contingency(np.array(list(pool.values())))
print(f" 3x2 chi2={c:.2f}, df={d}, p={p:.3f}")
# ---------------------------------------------------------------- figure
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none"})
rows = [ # (key, label)
(("T1", "KO", "KO+TGF"), "Table 1: p190-B⁻/⁻ vs +rTGF-β1\n9/16 vs 4/13 positive mice"),
(("T1", "KO+TGF", "KO+TGF+p38i"), "Table 1: +rTGF-β1 vs +rTGF-β1+SB203580\n4/13 vs 13/20"),
(("T1", "chi2"), "Table 1: three groups, 3×2 χ²"),
(("S3", "KO", "KO+TGF"), "Supp. Table 3: p190-B⁻/⁻ vs +rTGF-β1\n8/12 vs 2/12"),
(("S3", "KO+TGF", "KO+TGF+p38i"), "Supp. Table 3: +rTGF-β1 vs +p38 inhibitor\n2/12 vs 4/10"),
(("S3", "chi2"), "Supp. Table 3: three groups, 3×2 χ²"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
ax.set_facecolor(BG)
ys = [5.6, 4.6, 3.6, 2.0, 1.0, 0.0]
for (key, lab), y in zip(rows, ys):
rep, ub = REPORTED[key]
r = results[key]
lo = min(rep, r["fisher"], r["pct"])
hi = max(rep, r["fisher"], r["pct"])
ax.plot([lo, hi], [y, y], color="#cfcdc8", lw=2, zorder=1)
ax.plot(r["pct"], y, marker="s", ms=12, mfc="none", mec=BLUE, mew=2, ls="none", zorder=3)
ax.plot(r["fisher"], y, marker="o", ms=9, color=BLUE, ls="none", zorder=4)
ax.plot(rep, y, marker="D", ms=8.5, color=ORANGE, ls="none", zorder=5)
if ub: # "<" bound: arrow pointing to smaller p
ax.annotate("", xy=(rep/12, y), xytext=(rep/1.6, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2), zorder=5)
ax.text(r["fisher"], y+0.24, f"{r['fisher']:.3f}", color=BLUE, fontsize=10, ha="center", va="bottom", bbox=dict(facecolor=BG, edgecolor="none", pad=0.6), zorder=6)
ax.text(rep, y+0.24, ("<" if ub else "") + f"{rep:g}", color=ORANGE, fontsize=10, ha="center", va="bottom")
ax.axvline(0.05, color=SEC, lw=0.8, zorder=0)
ax.text(0.04, -0.45, "p = 0.05", color=SEC, fontsize=10, ha="right", va="center", bbox=dict(facecolor=BG, edgecolor="none", pad=0.6))
ax.set_xscale("log")
ax.set_xlim(1e-14, 2)
ax.set_ylim(-0.6, 6.6)
ax.set_yticks(ys)
ax.set_yticklabels([lab for _, lab in rows], fontsize=10, color=TXT)
ax.axhline(2.8, color=GRID, lw=1)
ax.set_xlabel("P value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
handles = [
Line2D([], [], marker="D", ms=8.5, color=ORANGE, ls="none", label="Reported in the paper (arrow = \"<\")"),
Line2D([], [], marker="o", ms=9, color=BLUE, ls="none", label="Recomputed: stated test on the mouse counts"),
Line2D([], [], marker="s", ms=12, mfc="none", mec=BLUE, mew=2, ls="none",
label="Recomputed: same test, rounded % entered as counts of 100"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.32, -0.13), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
fig.savefig(os.path.join(HERE, "fig1.svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200, facecolor=BG)
print("\nwrote fig1.svg, fig1.png")
# ---------------------------------------------------------------- figure 2: calculation flow
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
r1 = results[("T1", "KO", "KO+TGF")]
r2 = results[("T1", "KO+TGF", "KO+TGF+p38i")]
r3 = results[("T1", "chi2")]
pc1 = as_pct_counts(T1)
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
ax.set_facecolor(BG); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")
def box(x, y, w, h, text, ec, bold=False, tc=TXT):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1.2",
fc=BG, ec=ec, lw=2))
ax.text(x + w/2, y + h/2, text, ha="center", va="center", fontsize=10.5, color=tc,
fontweight="bold" if bold else "normal", linespacing=1.4)
def arrow(x0, y0, x1, y1, color, rad=0.0):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=14,
color=color, lw=2, connectionstyle=f"arc3,rad={rad}"))
box(0.5, 26, 13, 12, "Table 1\nmouse\ncounts", SEC)
ax.text(19, 57, "Stated test on the numbers of mice", color=BLUE, fontsize=10.5, fontweight="bold", va="center")
box(19, 41, 25, 11, f"[[{T1['KO'][0]}, {T1['KO'][1]}], [{T1['KO+TGF'][0]}, {T1['KO+TGF'][1]}]]\nn = 16 and 13", BLUE)
box(48, 41, 17, 11, "Fisher exact,\ntwo-sided", SEC)
box(69, 41, 21, 11, f"Recomputed\np = {r1['fisher']:.3f}", BLUE, bold=True, tc=BLUE)
ax.text(19, 24.5, "Same test, per cent as counts of 100", color=BLUE, fontsize=10.5,
fontweight="bold", va="center")
box(19, 8, 25, 11, f"[[{pc1['KO'][0]}, {pc1['KO'][1]}], [{pc1['KO+TGF'][0]}, {pc1['KO+TGF'][1]}]]\nn = 100 and 100", BLUE)
box(48, 8, 17, 11, "Fisher exact,\ntwo-sided", SEC)
box(69, 8, 21, 11, f"Recomputed\np = {r1['pct']:.5f}", BLUE, bold=True, tc=BLUE)
for y in (46.5, 13.5):
arrow(44.6, y, 47.5, y, SEC); arrow(65.6, y, 68.5, y, SEC)
arrow(14.2, 35, 18.4, 46.5, SEC, rad=-0.3)
arrow(14.2, 29, 18.4, 13.5, SEC, rad=0.3)
box(69, 27, 21, 9, "Reported\nP = 0.0006", ORANGE, bold=True, tc=ORANGE)
ax.plot([79.5, 79.5], [36.6, 40.4], color=SEC, lw=1, ls=(0, (2, 2)))
ax.plot([79.5, 79.5], [19.6, 26.4], color=SEC, lw=1, ls=(0, (2, 2)))
ax.text(81, 38.5, f"\u2260 {r1['fisher']/0.0006:.0f}\u00d7", ha="left", va="center", fontsize=10, color=SEC)
ax.text(81, 22.6, "= rounded", ha="left", va="center", fontsize=10, color=SEC)
ax.text(50, 1.0,
f"Other Table 1 tests, mouse counts vs per cent as counts: SB203580 comparison p = {r2['fisher']:.3f} vs {r2['pct']:.1e};"
f"\n3×2 χ² p = {r3['fisher']:.3f} vs {r3['pct']:.1e}. Both reported as P < 0.0001.",
ha="center", va="center", fontsize=10, color=SEC, linespacing=1.4)
fig.savefig(os.path.join(HERE, "fig2.svg"), facecolor=BG, bbox_inches="tight", pad_inches=0.1)
fig.savefig(os.path.join(HERE, "fig2.png"), dpi=200, facecolor=BG, bbox_inches="tight", pad_inches=0.1)
print(f"ratio reported/recomputed, first comparison: {r1['fisher']/0.0006:.0f}x")
print("wrote fig2.svg, fig2.png")
The abstract reverses the open/closed conformational preference reported in the Results and figures
Nat Commun (2017) · doi:10.1038/ncomms15744 · PMC5458555
Summary
The abstract states that the GTPase domain–BSE region of MxA prefers the open conformation in the nucleotide-free, GDP·AlF4−-bound and GDP-bound forms, and that GTP loading shifts the preference to the closed state. The Results define high FRET (~0.72) as closed and low FRET (~0.35) as open. They report the high-FRET (closed) state at ~70% occupancy in those three forms, and a shift toward low FRET (open) with the GTP analogue GMPPCP. The Fig. 3c legend, the Discussion and the Fig. 6 model all use the Results direction, so the abstract is the only place where the direction is reversed. This matters because the abstract carries the paper's headline result: a reader of the abstract alone gets the main smFRET finding, and the direction of the hydrolysis-driven step in the proposed torque model, the wrong way round.
What the paper reports
Abstract:
Whereas the open conformation is preferred in nucleotide-free, GDP·AlF4--bound and GDP-bound forms, loading of GTP activates the relative movement between the two domains and alters the conformational preference to the ‘closed' state.
Results, "Conformational preference for GD-BSE region" (paragraph 1):
Surprisingly, MxAMO2-AS displayed two steadily distinguished FRET states (∼0.35/low and ∼0.72/high, Fig. 3a and Supplementary Fig. 8a). Prior structural studies identified two GD-BSE conformations: MxA/dynamin molecules in apo, GDP·AlF4--bound and GDP-bound forms display a closed conformation, while MxA/dynamin bound with non-hydrolysable GTP analogue guanosine-5′-[(β,γ)-methyleno]triphosphate (GMPPCP) adopts an open conformation where the BSE swings ∼70° against the GD
Thus, the high- and low-FRET states represent the closed and open conformations of stalkless-MxA, respectively. The FRET traces in apo, GDP·AlF4--bound and GDP-bound forms are very similar and the high-FRET state is dominant with an occupancy of ∼70%, whereas loading of GMPPCP shifts the FRET distribution towards the low-FRET state (Fig. 3c).
Figure 3 legend, panel (c):
(c) Fraction variation in the low-FRET open state (red circle) and the high-FRET closed state (green circle) during GTP hydrolysis.
Discussion (paragraph 1):
This in turn releases the intrinsic potential for conformational recovery, thereby producing a mechanistic force that drives the transition from ‘open' to ‘closed' conformation at the GD-BSE region (Figs 3c and 6a).
Figure 6 legend, panel (b):
(I) Loading of GTP promotes domain movement. The GDs of molecule 4 and A associate at open conformation. (II) Stimulated GTP hydrolysis releases the potential of conformational alternation in the GD-BSE region of molecule 4 and A, and the GDs move back to closed conformation, thereby generating latitudinal shear force.
All quotes were checked word for word against the article as archived in PubMed Central (src/show.py PMC5458555 --grep). The archive writes "AlF4-" where the publisher's page has a subscript 4 and a superscript minus.
What the paper's own numbers show
Method. I took the state mapping the Results define (high FRET = closed, low FRET = open) and applied it to the occupancies and shifts the Results report. I then compared the result with the abstract sentence, and with the Discussion and Fig. 6 statements about the GTP-loaded and post-hydrolysis states. As an independent check of the mapping, I converted the two reported FRET efficiencies to dye-to-dye distances with the standard Förster relation E = 1/(1 + (R/R0)^6). The paper does not state R0, so I used 50–60 Å, the range usually quoted for Cy3/Cy5. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own statements |
|---|---|---|---|
| FRET values of the two GD-BSE states | Results; Fig. 3a | ~0.35 (low), ~0.72 (high) | Förster distances: high FRET 42.7–51.3 Å, low FRET 55.4–66.5 Å (R0 = 50–60 Å). High FRET is the shorter distance, so it is the compact (closed) state |
| State mapping | Results; Fig. 3c legend | high FRET = closed, low FRET = open | Same; consistent with the Förster check and with the cited crystal structures (apo, GDP·AlF4−, GDP closed; GMPPCP open) |
| Occupancy in apo, GDP·AlF4−, GDP | Results | high FRET (closed) ~70% | closed 70%, open 30% (closed:open = 2.33:1), so closed is preferred |
| Preferred state in apo, GDP·AlF4−, GDP | Abstract | open | Results imply closed; the Abstract is reversed |
| Effect of GTP / GMPPCP loading | Abstract | preference altered "to the ‘closed' state" | Results: shift toward low FRET, i.e. toward open; the Abstract is reversed |
| Direction of the hydrolysis-driven step | Discussion; Fig. 6b (I–II) | GTP-loaded GDs open; hydrolysis drives open → closed | Agrees with the Results (GTP state open, post-hydrolysis closed); disagrees with the Abstract |
| Distance gap between the two states | Results | ~20 Å (crystal structures, residues 149–639) | 12.7–15.3 Å from the FRET values (R0 = 50–60 Å). Same direction; the size differs, plausibly because of dye linkers and the use of residue 639 for 634. Not part of the finding |
Figure
Figure 1. The abstract sentence (left, orange) next to the Results statements for the same conditions (right, blue). Under the mapping the Results define (high FRET ≈ 0.72 = closed, low FRET ≈ 0.35 = open), the high-FRET state at ~70% occupancy means the nucleotide-free, GDP·AlF4− and GDP forms prefer the closed state, and GMPPCP loading shifts the distribution toward open. The abstract states the opposite in both cases. The Discussion and the Fig. 6b model follow the Results. All excerpts are quoted from the paper; no numbers in the figure are new.
Figure 2. A physics check of the Results mapping. The gray curve is the Förster relation E = 1/(1 + (R/R0)^6) for R0 = 54 Å; the shaded band covers R0 = 50–60 Å. The dashed orange lines are the two FRET values reported in the Results. The blue bars are the distances recomputed from those values across the R0 band (dots at R0 = 54 Å). The high-FRET state lies at about 43–51 Å and the low-FRET state at about 55–67 Å. The high-FRET state is therefore the more compact one, which supports the Results' "closed" label and gives no support to the abstract's reversed wording.
Why it matters
The abstract is the only part of the paper most readers see, and it is what indexing services and citing papers pick up. Here it states the direction of the paper's main smFRET result the wrong way round on two counts: which conformation dominates without GTP, and which way GTP loading moves it. Read literally, it also implies that hydrolysis drives a closed-to-open change, the opposite of the open-to-closed step on which the Discussion builds the shear-force and torque model (Fig. 6). It also suggests that the full-length smFRET data disagree with the earlier stalkless-MxA and dynamin crystal structures, when the Results say they agree ("These results demonstrate that the GMPPCP-dependent BSE rearrangement observed in stalkless-MxA structures also occurs in full-length MxA").
The measurements, figures and model are consistent with one another, so the paper's actual conclusion is not affected. The rest of the abstract sentence, that GTP loading "activates the relative movement between the two domains", agrees with the body (Fig. 3g: "enhanced dynamics for GMPPCP-loading condition"). The error is confined to the two words "open" and "closed" in one sentence of the abstract.
Explanations considered
- A different region or construct. The abstract names the GTPase domain–BSE region, the same region measured with MxAMO2-AS (dyes at A149 and S634) in the Results. The BSE–stalk (hinge 1) measurements use other constructs and a low/medium/high state description with no open/closed preference, so they cannot be the source of the abstract sentence. Does not account for it.
- A different definition of open and closed. The Results define high FRET as closed and low FRET as open, in line with the cited crystal structures (apo, GDP·AlF4− and GDP closed; GMPPCP open, with the BSE swung ~70°). The Fig. 3c legend, the Discussion and the Fig. 6 legend use the same convention. No other definition appears anywhere in the paper. Does not account for it.
- Physics of FRET. Förster theory puts the higher efficiency at the shorter distance. With R0 = 50–60 Å, E = 0.72 corresponds to about 43–51 Å and E = 0.35 to about 55–67 Å, which fits the Results mapping (Fig. 2). Does not rescue the abstract.
- GTP versus GMPPCP. The abstract says "loading of GTP"; the relevant experiment uses the non-hydrolysable analogue GMPPCP, which the paper uses throughout as the GTP-loaded state. No condition in the paper shows a closed preference upon GTP loading. Does not account for it.
- Conversion or archiving artifact. The difference is a swap of the words "open" and "closed", not a number, sign, superscript or table cell. The publisher's HTML, the PMC page and the Europe PMC full text all carry the same abstract sentence. Ruled out.
- Supplementary material. The single supplementary PDF (10 pages) contains Supplementary Fig. 8a, a repeat of the Fig. 3a experiment with the same low/high two-Gaussian decomposition. It never uses the words "open" or "closed" and gives no occupancy table for MxAMO2-AS. It neither supports nor resolves the abstract wording.
None of these explains the abstract sentence. It is an internal inconsistency in the text; the data and the body of the paper agree with each other.
Status
- Version of record. Checked against the publisher's HTML at https://www.nature.com/articles/ncomms15744 (page metadata: datePublished and dateModified 2017-05-26). The abstract there contains the same sentence, "open conformation is preferred in nucleotide-free, GDP·AlF4−-bound and GDP-bound forms, ... alters the conformational preference to the ‘closed’ state", and the Results subsection carries the same mapping and occupancy text quoted above. The PMC and Europe PMC versions match.
- Supplements. The one supplementary PDF linked from the publisher page (41467_2017_BFncomms15744_MOESM590_ESM.pdf) was downloaded and searched; see above. No separate source-data file is linked.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, author correction, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices for the DOI, and the publisher's page shows no correction notice. - PubPeer. The PubPeer API requires a developer key and the search page rate-limited automated requests (HTTP 429), so PubPeer could not be searched directly. A web search for the article title together with "PubPeer" found no PubPeer thread.
- Sources. PMC article PMC5458555 (via
src/show.py); publisher version of record https://www.nature.com/articles/ncomms15744; supplementary information PDF (MOESM590); Europe PMC full-text XML; Crossref; PubPeer; https://doi.org/10.1038/ncomms15744.
Suggested correction
Swap the two conformation labels in the abstract sentence so that it matches the Results, Fig. 3c, the Discussion and Fig. 6:
Whereas the closed conformation is preferred in nucleotide-free, GDP·AlF4−-bound and GDP-bound forms, loading of GTP activates the relative movement between the two domains and alters the conformational preference to the ‘open’ state.
No change to the data, figures, Results or Discussion appears to be needed.
Recomputation script (python recompute.py.txt)
"""Recompute the abstract-vs-body check for PMC5458555
(Nat Commun 2017, doi:10.1038/ncomms15744,
"Conformational dynamics of dynamin-like MxA revealed by single-molecule FRET").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (Abstract; Results, "Conformational
preference for GD-BSE region"; Fig. 3 legend; Discussion; Fig. 6 legend).
FRET physics used (standard Foerster relation, not taken from the paper):
E = 1 / (1 + (R / R0) ** 6) -> R = R0 * (1 / E - 1) ** (1 / 6)
so a higher FRET efficiency E means a shorter dye-to-dye distance R.
The paper does not state R0; 50-60 A spans the values usually quoted for Cy3/Cy5.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
# ------------------------------------------------------------------ inputs (paper)
E_LOW, E_HIGH = 0.35, 0.72 # Results: "two steadily distinguished FRET states (~0.35/low and ~0.72/high"
OCC_HIGH = 0.70 # Results: "the high-FRET state is dominant with an occupancy of ~70%"
DR_TEXT = 20 # Results: distances "differed by ~20 A"
# Results: "the high- and low-FRET states represent the closed and open conformations ..., respectively"
MAPPING = {"high": "closed", "low": "open"}
# Abstract: "the open conformation is preferred in nucleotide-free, GDP.AlF4--bound and GDP-bound
# forms, loading of GTP ... alters the conformational preference to the 'closed' state"
ABSTRACT = {"apo / GDP.AlF4- / GDP": "open", "GTP (GMPPCP) loading": "shift toward closed"}
def distance(E, R0):
"""Dye-to-dye distance (A) for FRET efficiency E and Foerster radius R0 (A)."""
return R0 * (1.0 / E - 1.0) ** (1.0 / 6.0)
# ------------------------------------------------------------------ 1. Foerster check
print("1. Foerster check: which FRET state is the shorter distance?")
for R0 in (50, 54, 56, 60):
rh, rl = distance(E_HIGH, R0), distance(E_LOW, R0)
print(f" R0 = {R0} A: R(E={E_HIGH}) = {rh:.1f} A, R(E={E_LOW}) = {rl:.1f} A, "
f"difference = {rl - rh:.1f} A (text: ~{DR_TEXT} A)")
print(" -> high FRET is the shorter distance, i.e. the compact (closed) state, "
"matching the Results mapping.")
# ------------------------------------------------------------------ 2. occupancies
occ_low = 1 - OCC_HIGH
print("\n2. Occupancy in apo, GDP.AlF4- and GDP forms (Results)")
print(f" high FRET = {MAPPING['high']}: {OCC_HIGH:.0%}; low FRET = {MAPPING['low']}: {occ_low:.0%}")
pref = MAPPING["high"] if OCC_HIGH > 0.5 else MAPPING["low"]
print(f" -> preferred state: {pref} (ratio closed:open = {OCC_HIGH / occ_low:.2f} : 1)")
# ------------------------------------------------------------------ 3. compare with abstract
body = {"apo / GDP.AlF4- / GDP": pref,
"GTP (GMPPCP) loading": "shift toward " + MAPPING["low"]}
print("\n3. Abstract vs Results")
for k in body:
print(f" {k:24s} Results: {body[k]:22s} Abstract: {ABSTRACT[k]:22s} "
f"agree = {body[k] == ABSTRACT[k]}")
# ------------------------------------------------------------------ 4. other statements
print("\n4. Other statements in the paper, read with the Results mapping")
others = [
("Results: prior structures, apo/GDP.AlF4-/GDP closed, GMPPCP open", "closed", "open"),
("Discussion: hydrolysis 'drives the transition from open to closed'", "closed", "open"),
("Fig. 6b (I-II): GTP-loaded GDs 'associate at open', then 'move back to closed'",
"closed", "open"),
]
for text, post_hydrolysis, gtp_state in others:
agrees_body = (post_hydrolysis == body["apo / GDP.AlF4- / GDP"]
and "shift toward " + gtp_state == body["GTP (GMPPCP) loading"])
agrees_abs = (post_hydrolysis == ABSTRACT["apo / GDP.AlF4- / GDP"]
and "shift toward " + gtp_state == ABSTRACT["GTP (GMPPCP) loading"])
print(f" {text}\n GTP state = {gtp_state}, post-hydrolysis = {post_hydrolysis}; "
f"agrees with Results = {agrees_body}, agrees with Abstract = {agrees_abs}")
# ------------------------------------------------------------------ figure helpers
def box(ax, x, y, w, h, edge, face=BG, lw=1.6):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
linewidth=lw, edgecolor=edge, facecolor=face,
transform=ax.transAxes))
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), **kw)
plt.close(fig)
# ------------------------------------------------------------------ fig 1: two text panels
fig = plt.figure(figsize=(W, 4.7))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
LX, RX, PW = 0.02, 0.58, 0.40 # panel x positions and width
TOP = 0.87
ax.text(LX, TOP + 0.05, "Abstract", fontsize=12, weight="bold", color=ORANGE,
transform=ax.transAxes)
ax.text(LX, TOP + 0.01, "statement as published", fontsize=10, color=INK2,
transform=ax.transAxes)
ax.text(RX, TOP + 0.05, "Results, Fig. 3c legend", fontsize=12, weight="bold", color=BLUE,
transform=ax.transAxes)
ax.text(RX, TOP - 0.03, "high FRET (~0.72) = closed\nlow FRET (~0.35) = open",
fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.3)
rows = [ # (condition, abstract excerpt, Results excerpt, y of box bottom)
("Nucleotide-free, GDP·AlF$_4^-$, GDP",
"“open conformation\nis preferred”",
"high-FRET (closed) state dominant,\noccupancy ~70%\n→ closed preferred",
0.47),
("GTP loading (GTP analogue GMPPCP)",
"“alters the conformational\npreference to the ‘closed’ state”",
"“shifts the FRET distribution\ntowards the low-FRET state”\n→ toward open",
0.125),
]
BH = 0.24
for cond, abs_txt, res_txt, y in rows:
ax.text(0.5, y + BH + 0.03, cond, ha="center", fontsize=10.5, weight="bold", color=INK,
transform=ax.transAxes)
box(ax, LX, y, PW, BH, ORANGE)
box(ax, RX, y, PW, BH, BLUE)
ax.text(LX + 0.02, y + BH / 2, abs_txt, va="center", fontsize=10.5, color=INK,
transform=ax.transAxes, linespacing=1.35)
ax.text(RX + 0.02, y + BH / 2, res_txt, va="center", fontsize=10, color=INK,
transform=ax.transAxes, linespacing=1.35)
arr = FancyArrowPatch((LX + PW + 0.008, y + BH / 2), (RX - 0.008, y + BH / 2),
arrowstyle="<->", mutation_scale=12, linewidth=2, color=INK2,
transform=ax.transAxes)
ax.add_patch(arr)
ax.text(0.5, y + BH / 2 + 0.035, "reversed", ha="center", fontsize=10.5, weight="bold",
color=INK, transform=ax.transAxes)
ax.text(0.5, y + BH / 2 - 0.065, "open ↔ closed", ha="center", fontsize=10,
color=INK2, transform=ax.transAxes)
ax.text(LX, 0.025,
"Discussion and Fig. 6b agree with the Results: the GTP-loaded state is open,\n"
"and GTP hydrolysis drives the change from open → closed.",
fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.3)
save(fig, "fig1")
# ------------------------------------------------------------------ fig 2: Foerster curve
fig, ax = plt.subplots(figsize=(W, 4.3))
fig.subplots_adjust(left=0.10, right=0.97, bottom=0.30, top=0.96)
R = np.linspace(25, 90, 400)
R0_LO, R0_MID, R0_HI = 50, 54, 60
ax.fill_between(R, 1 / (1 + (R / R0_HI) ** 6), 1 / (1 + (R / R0_LO) ** 6), color=GRID,
alpha=0.9, linewidth=0, label=f"Förster curve, R$_0$ = {R0_LO}–{R0_HI} Å")
ax.plot(R, 1 / (1 + (R / R0_MID) ** 6), color=INK2, linewidth=2,
label=f"Förster curve, R$_0$ = {R0_MID} Å")
for E, name, state in ((E_HIGH, "high FRET", "closed"), (E_LOW, "low FRET", "open")):
ax.axhline(E, color=ORANGE, linewidth=1.2, linestyle=(0, (4, 3)), zorder=1)
rlo, rmid, rhi = distance(E, R0_LO), distance(E, R0_MID), distance(E, R0_HI)
ax.plot([rlo, rhi], [E, E], color=BLUE, linewidth=3, solid_capstyle="butt", zorder=3)
ax.plot(rmid, E, "o", color=BLUE, markersize=9, zorder=4)
ax.text(26, E + 0.025, f"reported E ≈ {E} ({name})", color=ORANGE, fontsize=10,
bbox=dict(facecolor=BG, edgecolor="none", pad=1), zorder=2)
ax.text(rhi + 1.5, E + 0.025,
f"R ≈ {rlo:.0f}–{rhi:.0f} Å → {state} (Results)", color=BLUE, fontsize=10)
ax.plot([], [], "o-", color=BLUE, markersize=9, linewidth=3,
label="distance recomputed from reported E")
ax.plot([], [], color=ORANGE, linewidth=1.2, linestyle=(0, (4, 3)), label="reported FRET value")
ax.set_xlim(25, 90)
ax.set_ylim(0, 1)
ax.set_xlabel("dye-to-dye distance R (Å)")
ax.set_ylabel("FRET efficiency E")
ax.grid(True, color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.17), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract reports reduced PKN1 and PKN4, but the Results report increased PKN2 and no PKN4
Nat Commun (2018) · doi:10.1038/s41467-018-04411-5 · PMC5998138
Summary
For the isogenic cell-line comparison, the abstract states that PCQ "was used to identify reduced levels of serine/threonine protein kinases PKN1 and PKN4 in CFBE41o- cells compared to HBE41o- cells". The Results and Fig. 7 instead describe a PKN1/PKN2/PKN3 cluster in which PKN2 was "three-fold more abundant in CFBE41o- cells" and confirmed by western blot, the evidence for PKN1 is "conflicting" even about its presence, and PKN4 is never mentioned outside the abstract. The paper's own data files agree with the Results: PKN2-specific peptides have a CFBE41o-/HBE41o- ratio of 3.000, the only peptide on the PKN1 node is shared with PKN2 and PKN3 (ratio 0.500), and no file contains PKN4. Because the abstract is the indexed summary and this sentence is one of its two biological examples, it gets both the proteins and the direction of the one validated kinase change wrong.
What the paper reports
Abstract:
PCQ was used to identify reduced levels of serine/threonine protein kinases PKN1 and PKN4 in CFBE41o- cells compared to HBE41o- cells and to elucidate that shorter proteoforms of full-length caspase-4 and ephrin B receptor are differentially expressed.
Results, "Analysis of individual protein clusters", paragraph 2. This paragraph is the only passage in the Results about the PKN proteins:
Finally, a serine/threonine-protein kinase (PKN) cluster comprising PKN1 (Q16512), PKN2 (Q16513), and PKN3 (Q6P5Z2) was identified. PKN2 (PRK2) shares 59% sequence identity with PKN1 and was three-fold more abundant in CFBE41o- cells (Fig. 7b). Western blot confirmed that PKN2 was more abundant in CFBE41o- cells when normalized to the expression levels of Na+/K+-ATPase (Fig. 7c and 7d, Supplementary Fig. 12).
A difference in PKN2 protein quantity may contribute to the altered cell morphology between HBE41o- and CFBE41o– cells in cell culture because bronchial epithelial airway cells require PKN2 to establish apical junctions32.
PKN1 is part of two incomplete protein pairs with either PKN2 or PKN3, which provide conflicting evidence for a potential presence of PKN1. While the incomplete protein pair with PKN2 shows discrepant ratio values for the peptide nodes (∆ ≥ 2-fold), the incomplete protein pair with PKN3 shows similar ratio values (∆ < 2-fold).
Fig. 7 legend, panels b and d:
b The protein cluster of serine–threonine kinase-related protein kinases PKN1, PKN2, and PKN3 is displayed.
d Western blot analysis was used to verify the difference in PKN2 expression levels between HBE41o– and CFBE41o– cells.
Data in the western blot represent independent biological replicates, CFBE41o–: n = 2, HBE41o–: n = 2
Results, "Comparison of two isogenic cell lines with PCQ", on the labelling, which fixes the direction of the ratios:
CFBE41o– and HBE41o– cells were labeled with light and heavy isobaric isotopologues, respectively, and analyzed in a single experiment (n = 1)
Methods, "Cell culture and western blot". The only antibody against a PKN protein is anti-PKN2:
PKN2 and NaK-ATPase were detected by incubation with monoclonal anti-PKN2 antibody
Discussion. The summary of the cell-line comparison mentions no kinase:
When comparing two isogenic cell lines, we found that caspase-4 and ephrin type B receptor protein fragments were differentially regulated relative to their longer proteoforms, respectively.
What the paper's own numbers show
Method. I counted every mention of PKN1 to PKN4 in the PMC-archived text, separately for the abstract and for the rest of the article (Results, Discussion, Methods, figure legends). I searched every body sentence that names a PKN protein for wording of reduction ("reduc", "decreas", "lower", "less abundant", "down"). I then opened the two supplementary data files that hold the cell-line quantification. Supplementary Data 7 (MOESM9, the 180 classified complete protein pairs) gives log2 ratios for unique and shared peptide nodes. Supplementary Data 6 (MOESM8, "Network 5" in the Results) is the XGMML peptide-to-protein network for CFBE41o- vs HBE41o-, with ion counts for each peptide node. From it I extracted the connected component that holds the three PKN accessions. I converted log2 ratios to ratios. I checked the ratio direction against the normalized light and heavy ion counts, since the Results say light = CFBE41o- and heavy = HBE41o-. I also queried UniProt for a human gene named PKN4. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own text and data |
|---|---|---|---|
| Kinases named as changed in the cell-line comparison | Abstract | PKN1 and PKN4, "reduced levels" in CFBE41o- | Results name PKN2 as changed (increased); no reduction of any PKN protein is stated |
| Mentions of PKN1 / PKN2 / PKN3 / PKN4 | Abstract vs rest of article | 1 / 0 / 0 / 1 in the abstract | 5 / 13 / 4 / 0 in the rest of the article |
| Body sentences naming a PKN protein that use reduction wording | Results, Methods, legends | — | 0 of 11 |
| Members of the PKN cluster | Results; Fig. 7b legend | PKN1 (Q16512), PKN2 (Q16513), PKN3 (Q6P5Z2) | same three in Supplementary Data 6; no PKN4 node in the network file |
| PKN2-specific peptides (3 peptides), CFBE41o-/HBE41o- | Results ("three-fold more abundant in CFBE41o- cells"); Supp. Data 7 | log2 1.58496 | ratio 3.000 (= 2^1.585; normalized light/heavy ions 4.5/1.5 in Supp. Data 6), an increase |
| PKN2 western blot | Results; Fig. 7d (n = 2 vs n = 2); Supp. Fig. 12 | "more abundant in CFBE41o- cells" | an increase, the same direction as the mass-spectrometry ratio |
| PKN3-specific peptides (3 peptides) | Supp. Data 7 only | log2 −0.678 | ratio 0.625 (1.60-fold lower; light/heavy 2.5/4.0); not discussed in the text |
| Shared peptide IADFGLCK (PKN1, PKN2, PKN3) | Supp. Data 6, 7 | log2 −1.000 | ratio 0.500 (light/heavy 0.75/1.5) |
| Peptides mapped to the PKN1 node | Supp. Data 6 | — | only the shared IADFGLCK. The node groups Q16512 with two fragment entries (H0Y429, K7EL10), and its gene-name field reads "PKN1,PKN2" |
| PKN1–PKN2 incomplete pair | Results ("∆ ≥ 2-fold") | — | unique 3.000 vs shared 0.500, a 6.00-fold discrepancy |
| PKN1–PKN3 incomplete pair | Results ("∆ < 2-fold") | — | unique 0.625 vs shared 0.500, a 1.25-fold discrepancy |
| A PKN1-specific or PKN4 ratio | article and all supplementary files | — | none exists |
| Human gene named PKN4 | UniProt (query 2026-10-23) | — | none; human PKN genes are PKN1, PKN2 (alias PRK2, PRKCL2) and PKN3 |
The body's account is internally consistent, and it reproduces from the data files: both the "∆ ≥ 2-fold" and the "∆ < 2-fold" statements match the network ratios. The abstract sentence is the outlier. It names a protein (PKN4) with no data behind it in the paper. It describes as an identified reduction a PKN1 inference that the body calls conflicting. It leaves out the one PKN change that was quantified with protein-specific peptides and checked by western blot, which is an increase.
Figure
Figure 1. Short verbatim excerpts from the abstract (left, orange) and from the Results and Fig. 7 legend (right, blue), with the supplementary-data values printed by recompute.py.txt. The abstract names PKN1 and PKN4 and says "reduced". The body names a PKN1/PKN2/PKN3 cluster and reports PKN2 three-fold more abundant in CFBE41o- cells (ratio 3.0 in Supplementary Data 6 and 7), confirmed by western blot. It calls the evidence for PKN1's presence conflicting, and its only peptide is shared with PKN2 and PKN3 (ratio 0.5). PKN4 is mentioned nowhere outside the abstract.
Figure 2. CFBE41o-/HBE41o- abundance ratios on one log2 axis; the vertical line marks no change (ratio 1). The orange hatched bands mark the abstract's claim for PKN1 and PKN4. The abstract gives a direction ("reduced") and no value, so the bands span the whole "reduced" side and carry no magnitude. Filled blue circles are ratios from protein-specific peptides in Supplementary Data 6 and 7: PKN2 at 3.0, which is the Results' "three-fold more abundant", and PKN3 at 0.625, which the text does not discuss. The hollow blue circle is the PKN1 node's only peptide, which is shared with PKN2 and PKN3 (0.5), so it is not specific to PKN1. PKN4 has no data point because the paper and its supplements contain none.
Why it matters
- The abstract misstates one of its two biological examples. The cell-line comparison is the paper's demonstration on a human proteome, and the abstract gives two results from it: the kinases and the caspase-4 and ephrin B receptor proteoforms. For the kinases, a reader of the abstract would conclude that PKN1 and PKN4 are lower in ∆F508-CFTR CFBE41o- cells. The body supports neither claim as stated. The one kinase result it quantified with protein-specific peptides and checked by western blot goes the other way: PKN2 is higher in CFBE41o- cells. The Results then offer an interpretation of that increase (apical junctions and cell morphology). The abstract's reduction cannot carry that interpretation.
- A protein name with no counterpart. PKN4 is not a human gene symbol in UniProt and has no entry in the article or its supplementary files. A reader who searches for PKN4 data, or indexes the abstract, finds nothing to match it.
- Certainty is overstated for PKN1. The abstract says PCQ "was used to identify reduced levels" of PKN1. The body says the two incomplete pairs "provide conflicting evidence for a potential presence of PKN1". The only peptide on the PKN1 node is shared with PKN2 and PKN3, and the PKN1–PKN3 pair's ratios (0.625 vs 0.500) are consistent with PKN3 alone. The mass-spectrometry comparison is a single labelled experiment (n = 1), and there is no PKN1 blot.
- What is not affected. The PCQ method, the two-species proof of principle (58% vs 21% completion of incomplete protein pairs), the caspase-4 and ephrin B receptor proteoform results, the PRDX analysis, and the body's PKN2 result do not depend on this sentence. The Discussion and the conclusions do not repeat it. The error is limited to the abstract, but it misstates which proteins changed and in which direction.
Explanations considered
- The ratios are defined the other way round (HBE41o-/CFBE41o-). This does not account for the discrepancy. The abstract fixes the comparison itself ("in CFBE41o- cells compared to HBE41o- cells"), and the Results use the same direction ("more abundant in CFBE41o- cells"). In the network file, light/heavy ion counts reproduce each stored ratio (4.5/1.5 = 3.000 for PKN2), and the Results state that CFBE41o- was the light label. So the ratios are CFBE41o-/HBE41o-, and PKN2 is increased in CFBE41o-. Even if the direction were flipped, the protein would be PKN2, not PKN1 and PKN4.
- "PKN4" is an alias of a protein in the body. This does not account for it. The aliases the paper and UniProt give for PKN2 are PRK2 and PRKCL2. UniProt lists no human gene named PKN4, and the cluster's three accessions are PKN1, PKN2 and PKN3.
- "PKN4" stands for PKN3, which is lower in the data. This would partly fit the numbers but is not supported by the text. PKN3's specific peptides have a ratio of 0.625 in Supplementary Data 7, a 1.6-fold decrease, which is below the paper's own 2-fold threshold. The Results describe PKN3 only as a member of the cluster, never as reduced. Even on this reading, the abstract would still leave out the validated PKN2 increase and overstate PKN1. The published material cannot show whether the abstract meant PKN3.
- The PKN1 reduction is a PCQ inference from the shared peptide. This is consistent with the data and could explain the PKN1 half of the sentence. The shared peptide IADFGLCK is at 0.500 while PKN2's specific peptides are at 3.000. Under the paper's logic, this 6-fold discrepancy suggests that another protein, possibly PKN1 (or the PKN2 fragment H0Y429 grouped in the same node), is present at lower levels. The body itself declines that conclusion, though, because the PKN3 pair "shows similar ratio values". The abstract therefore states as a finding what the body presents as unresolved. This does not account for PKN4, or for the omission of PKN2.
- The result is in the supplementary material. It is not. All supplementary files were opened, some in an earlier pass of this pipeline (see Status). Supplementary Information (MOESM2) holds the full western blot exposures (Supp. Fig. 12, PKN2 only) and does not otherwise discuss PKN. The PKN entries in Supplementary Data 6, 7 and 8 match the Results as tabulated above. The other network files either contain no PKN accession or belong to a different experiment, not the CFBE41o-/HBE41o- comparison. No file reports a PKN1-specific ratio, and no file mentions PKN4.
- A conversion or typesetting artifact. This is ruled out. The publisher's HTML carries the same abstract sentence, in the page body and in its metadata, as the PMC and Europe PMC versions. The discrepancy is in protein names and in the direction of change, not in symbols or digits.
Status
- Version of record. The publisher's page (nature.com/articles/s41467-018-04411-5, published 13 June 2018) has the abstract sentence word for word: PKN4 appears in the abstract text and in the page's metadata and nowhere else. The Results paragraph and the Fig. 7 legend match the PMC archive. Europe PMC's full-text XML carries the same text (prior check in this pipeline). Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py --grep. - Supplements. I downloaded Supplementary Data 7 (MOESM9, XLSX) and the network files (MOESM4–8, MOESM10–14, zipped XGMML) from the publisher's server and searched them for Q16512, Q16513, Q6P5Z2 and PKN4. The values used above come from Supplementary Data 7 and Supplementary Data 6. Supplementary Data 8 (Network 6) contains the same PKN component with identical ratios. An earlier pass in this pipeline opened the remaining files (MOESM1 PDF, MOESM2 Supplementary Information DOCX, MOESM3 XLSX) and found no PKN content apart from Supp. Fig. 12.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI. PMC and Europe PMC list no correction or comment links (PMID 29899466). The publisher's page shows no correction notice, and a web search found none. I could not check PubPeer: its API and its search page both refused automated access (Cloudflare challenge; HTTP 403).
- Sources. PMC article PMC5998138; publisher page https://www.nature.com/articles/s41467-018-04411-5; supplementary files 41467_2018_4411_MOESM8_ESM.zip (Supplementary Data 6) and 41467_2018_4411_MOESM9_ESM.xlsx (Supplementary Data 7); UniProt REST (human PKN gene query); Crossref; Europe PMC.
Suggested correction
- Revise the abstract sentence so it matches the Results. For example: "PCQ was used to identify increased levels of the serine/threonine protein kinase PKN2 in CFBE41o- cells compared to HBE41o- cells, confirmed by western blot, and to elucidate that shorter proteoforms of full-length caspase-4 and ephrin B receptor are differentially expressed."
- If a statement about PKN1 is kept, word it as the body does: the evidence for a PKN1 proteoform is inconclusive and rests on a peptide it shares with PKN2 and PKN3.
- Remove "PKN4". If another protein was meant, name it and give the supporting data.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC5998138 (Nat Commun 2018, doi:10.1038/s41467-018-04411-5)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates the figures
# (SVG + PNG at 200 dpi) in the folder that holds this script:
# fig1.*, fig2.* the figures used in writeup.md
# candidates/fig_A..fig_E.* five candidate designs for review
# Inputs: the article text as archived in PubMed Central (read through the repo's
# src/show.py when it is available) and the one PKN row of Supplementary Data 7
# (41467_2018_4411_MOESM9_ESM.xlsx), transcribed below and re-read from the
# publisher's server if it can be reached.
import io
import os
import re
import shutil
import subprocess
import textwrap
import urllib.request
import xml.etree.ElementTree as ET
import zipfile
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
REPO = "/home/ubuntu/papers"
PY = "/home/ubuntu/papers-data/venv/bin/python"
SD7_URL = ("https://media.springernature.com/original/springer-static/esm/"
"art%3A10.1038%2Fs41467-018-04411-5/MediaObjects/41467_2018_4411_MOESM9_ESM.xlsx")
# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
ORANGE_T, BLUE_T = "#fbe3d9", "#dde9f8" # light tints for panel fills
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
# ---------------------------------------------------------------- 1. text
print("=== 1. Article text (PMC archive)")
txt = None
try:
txt = subprocess.run([PY, "src/show.py", "PMC5998138", "--full"], cwd=REPO,
capture_output=True, text=True, timeout=120).stdout
except Exception as e: # repo not present
print(" show.py not available:", e)
if txt:
lines = txt.splitlines()
abstract = next(l for l in lines if l.startswith("ABSTRACT:"))
body = "\n".join(l for l in lines if not l.startswith("ABSTRACT:"))
claim = re.search(r"PCQ was used to identify reduced[^.]*\.", abstract).group(0)
print(" Abstract sentence:", claim)
counts = {}
for k in ["PKN1", "PKN2", "PKN3", "PKN4"]:
counts[k] = (len(re.findall(k + r"\b", abstract)), len(re.findall(k + r"\b", body)))
print(f" {k}: abstract {counts[k][0]}, rest of article (Results, Methods, legends) {counts[k][1]}")
sents = list(dict.fromkeys(s.strip() for s in re.findall(r"[^.]*PKN[^.]*\.", body)))
red = [s for s in sents if re.search(r"reduc|decreas|lower|less abundant|down", s)]
print(f" body sentences naming a PKN protein: {len(sents)}; of these using reduced/decreased/"
f"lower/less abundant/down: {len(red)}")
m = re.search(r"PKN2 \(PRK2\)[^.]*\.", body).group(0)
print(" PKN2 sentence:", m)
print(" cluster accessions:", re.findall(r"PKN\d \(([A-Z0-9]{6})\)", body))
else:
counts = {"PKN1": (1, 5), "PKN2": (0, 13), "PKN3": (0, 4), "PKN4": (1, 0)}
print(" using counts recorded on 2026-10-23:", counts)
# ---------------------------------------------------------------- 2. SD7
# Supplementary Data 7, sheet "180 classified protein pairs": the only row naming a
# PKN accession (Q16512 PKN1, Q16513 PKN2, Q6P5Z2 PKN3). Columns: Unique Pep1 (U1),
# Ratio (U1), Protein 1, Shared Pep (S), Ratio (S), Protein 2, Unique Pep2 (U2),
# Ratio (U2), ClassificationCase. Ratios are log2.
SD7 = {"U1": ("LLGCEQLLTAVPGR_QGAENMTHTCASGTPK_SPAAALASSPSEGWLR", -0.678071905112637, "Q6P5Z2"),
"S": ("IADFGLCK", -1.0, "shared"),
"U2": ("FLSTEAISIMR_LDFDLEPEPPPAPPR_LDNLLLDTEGFVK", 1.58496250072115, "Q16513"),
"case": "CASE3"}
def read_xlsx_rows(blob):
"""Minimal stdlib reader for the first sheet of an .xlsx file."""
ns = {"m": "http://schemas.openxmlformats.org/spreadsheetml/2006/main"}
z = zipfile.ZipFile(io.BytesIO(blob))
shared = []
if "xl/sharedStrings.xml" in z.namelist():
for si in ET.fromstring(z.read("xl/sharedStrings.xml")).findall("m:si", ns):
shared.append("".join(t.text or "" for t in si.iter("{%s}t" % ns["m"])))
sheet = sorted(n for n in z.namelist() if n.startswith("xl/worksheets/sheet"))[0]
rows = []
for r in ET.fromstring(z.read(sheet)).iter("{%s}row" % ns["m"]):
row = []
for c in r.findall("m:c", ns):
v = c.find("m:v", ns)
val = "" if v is None else v.text
if c.get("t") == "s":
val = shared[int(val)]
elif c.get("t") == "inlineStr":
val = "".join(t.text or "" for t in c.iter("{%s}t" % ns["m"]))
row.append(val)
rows.append(row)
return rows
print("\n=== 2. Supplementary Data 7 (complete protein pairs)")
try:
req = urllib.request.Request(SD7_URL, headers={"User-Agent": "Mozilla/5.0"})
rows = read_xlsx_rows(urllib.request.urlopen(req, timeout=60).read())
hits = [r for r in rows if any(a in " ".join(r) for a in ("Q16512", "Q16513", "Q6P5Z2", "PKN"))]
print(f" downloaded; {len(rows) - 1} protein-pair rows; rows naming a PKN accession: {len(hits)}")
print(" PKN1 (Q16512) present in any row:", any("Q16512" in " ".join(r) for r in rows))
for h in hits:
print(" ", h)
h = hits[0]
assert abs(float(h[1]) - SD7["U1"][1]) < 1e-9 and abs(float(h[7]) - SD7["U2"][1]) < 1e-9
print(" matches the transcribed values")
except Exception as e:
print(" could not re-read the file (", type(e).__name__, "); using transcribed values")
u1, s, u2 = SD7["U1"][1], SD7["S"][1], SD7["U2"][1]
r_pkn3, r_sh, r_pkn2 = 2 ** u1, 2 ** s, 2 ** u2
print(f" PKN2 (Q16513) unique peptides: log2 {u2:.3f} -> ratio {r_pkn2:.3f} "
f"(log2(3) = {__import__('math').log2(3):.5f})")
print(f" PKN3 (Q6P5Z2) unique peptides: log2 {u1:.3f} -> ratio {r_pkn3:.3f} (= {1 / r_pkn3:.2f}-fold lower)")
print(f" shared peptide IADFGLCK: log2 {s:.3f} -> ratio {r_sh:.3f}")
print(f" PKN2-unique vs shared: {2 ** (u2 - s):.2f}-fold apart; PKN3-unique vs shared: {2 ** (u1 - s):.2f}-fold apart")
print(" ratio direction: the PKN2 value equals the Results' 'three-fold more abundant in "
"CFBE41o- cells', so the ratios are CFBE41o-/HBE41o-")
# ---------------------------------------------------------------- 3. network file
# Supplementary Data 6 (41467_2018_4411_MOESM8_ESM.zip, XGMML, "Network 5" in the
# Results; label CFBEvsHBE_protpepCollapsed_3ionsPerNode_Rc_IDs). The connected
# component holding the PKN proteins, transcribed (log2 finalRatio; normalized ion
# counts, light = CFBE41o-, heavy = HBE41o- per the Results):
NET = {
"proteins": {"Q16513 (PKN2)": ["U_PKN2", "IADFGLCK"],
"Q6P5Z2 (PKN3)": ["IADFGLCK", "U_PKN3"],
"H0Y429 K7EL10 Q16512 (PKN1 node; genes PKN1,PKN2)": ["IADFGLCK"]},
"peptides": {"U_PKN2": (1.5849625007211563, 4.5, 1.5, 0), # log2, normLight, normHeavy, significant
"IADFGLCK": (-1.0, 0.75, 1.5, 0),
"U_PKN3": (-0.6780719051126377, 2.5, 4.0, 0)},
}
SD6_URL = SD7_URL.replace("MOESM9_ESM.xlsx", "MOESM8_ESM.zip")
print("\n=== 3. Supplementary Data 6 (peptide-to-protein network 5, CFBE41o- vs HBE41o-)")
try:
req = urllib.request.Request(SD6_URL, headers={"User-Agent": "Mozilla/5.0"})
z = zipfile.ZipFile(io.BytesIO(urllib.request.urlopen(req, timeout=120).read()))
root = ET.fromstring(z.read(z.namelist()[0]))
tag = lambda e: e.tag.split("}")[-1]
nodes = {n.get("id"): n for n in root if tag(n) == "node"}
adj = {}
for e in root:
if tag(e) == "edge":
adj.setdefault(e.get("source"), set()).add(e.get("target"))
adj.setdefault(e.get("target"), set()).add(e.get("source"))
seed = [k for k in nodes if any(a in k for a in ("Q16512", "Q16513", "Q6P5Z2"))]
comp, fr = set(seed), list(seed)
while fr:
x = fr.pop()
for y in adj.get(x, ()):
if y not in comp:
comp.add(y); fr.append(y)
print(f" downloaded; PKN component has {len(comp)} nodes; any node naming PKN4:",
any("PKN4" in ET.tostring(nodes[k], encoding="unicode") for k in nodes))
for k in sorted(comp):
a = {x.get("name"): x.get("value") for x in nodes[k].iter() if tag(x) == "att"}
if a.get("isProtein") == "1":
print(f" protein {k} | genes {a.get('GeneName')} | peptide nodes {sorted(adj[k])}")
else:
print(f" peptide {k[:44]} | log2 {float(a['finalRatio']):.3f} | normLight {a['normLightIons']}"
f" | normHeavy {a['normHeavyIons']} | significant {a['significant']}")
except Exception as e:
print(" could not re-read the file (", type(e).__name__, "); using transcribed values")
for k, (l2, li, hv, sig) in NET["peptides"].items():
print(f" {k:9s} normLight/normHeavy = {li}/{hv} = {li / hv:.3f}; 2^log2 = {2 ** l2:.3f}; significant = {sig}")
print(" -> light/heavy reproduces each ratio, and light = CFBE41o-, so ratios are CFBE41o-/HBE41o-")
print(" PKN1 node: its only peptide is the shared IADFGLCK (ratio 0.500), which also maps to PKN2 and PKN3;")
print(f" pair with PKN2: unique {r_pkn2:.3f} vs shared {r_sh:.3f} -> {2 ** (u2 - s):.2f}-fold (>= 2, 'discrepant')")
print(f" pair with PKN3: unique {r_pkn3:.3f} vs shared {r_sh:.3f} -> {2 ** (u1 - s):.2f}-fold (< 2, 'similar')")
print(" No PKN1-specific or PKN4 ratio exists in the article or its supplements.")
# ---------------------------------------------------------------- helpers
def save(fig, path_noext):
os.makedirs(os.path.dirname(path_noext), exist_ok=True)
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
print(" wrote", os.path.relpath(path_noext, HERE) + ".svg/.png")
ABS_Q = ("PCQ was used to identify reduced levels of serine/threonine protein kinases "
"PKN1 and PKN4 in CFBE41o- cells compared to HBE41o- cells")
RES_Q = [("Results", "a serine/threonine-protein kinase (PKN) cluster comprising PKN1 (Q16512), "
"PKN2 (Q16513), and PKN3 (Q6P5Z2) was identified"),
("Results", "PKN2 (PRK2) ... was three-fold more abundant in CFBE41o- cells"),
("Results", "Western blot confirmed that PKN2 was more abundant in CFBE41o- cells"),
("Results", "conflicting evidence for a potential presence of PKN1")]
# ---------------------------------------------------------------- A: text panels
def quote_block(ax, x, y, q, wrap, step=0.19):
for ln in textwrap.wrap("\u201c" + q + "\u201d", wrap):
ax.text(x, y, ln, fontsize=10, color=INK, va="top")
y -= step
return y
def fig_A(path):
H = 5.0
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
# left: abstract
lx, lw_, ltop, lh = 0.12, 2.85, H - 0.12, 2.25
ax.add_patch(FancyBboxPatch((lx, ltop - lh), lw_, lh, boxstyle="round,pad=0,rounding_size=0.08",
fc=ORANGE_T, ec=ORANGE, lw=1.5))
ax.text(lx + 0.14, ltop - 0.16, "Abstract", fontsize=11, weight="bold", va="top")
ax.text(lx + 0.14, ltop - 0.40, "the claim as reported", fontsize=10, color=INK2, va="top")
quote_block(ax, lx + 0.14, ltop - 0.72, ABS_Q, 34)
ax.text(lx + 0.14, ltop - lh + 0.14, "Named: PKN1, PKN4\nDirection: reduced", fontsize=10,
weight="bold", color=INK, va="bottom", linespacing=1.4)
# right: body
rx, rw, rtop = 3.95, 3.13, H - 0.12
rh = H - 0.24
ax.add_patch(FancyBboxPatch((rx, rtop - rh), rw, rh, boxstyle="round,pad=0,rounding_size=0.08",
fc=BLUE_T, ec=BLUE, lw=1.5))
ax.text(rx + 0.14, rtop - 0.16, "Rest of the paper", fontsize=11, weight="bold", va="top")
ax.text(rx + 0.14, rtop - 0.40, "Results, Fig. 7, Supp. Data 6 and 7", fontsize=10, color=INK2,
va="top")
yy = rtop - 0.72
for _, q in RES_Q:
yy = quote_block(ax, rx + 0.14, yy, q, 40) - 0.10
ax.plot([rx + 0.14, rx + rw - 0.14], [yy + 0.02, yy + 0.02], color=BLUE, lw=0.8)
yy -= 0.10
ax.text(rx + 0.14, yy, f"Supp. Data 6, 7: PKN2 ratio {r_pkn2:.1f};\n"
f"PKN1's only peptide is shared with PKN2\nand PKN3 (ratio {r_sh:.1f})", fontsize=10,
color=INK, va="top", linespacing=1.3)
yy -= 0.72
ax.text(rx + 0.14, yy, f"PKN4: {counts['PKN4'][1]} mentions outside the abstract", fontsize=10,
color=INK, va="top")
ax.text(rx + 0.14, rtop - rh + 0.14, "Quantified and blotted: PKN2\nDirection: increased", fontsize=10,
weight="bold", color=INK, va="bottom", linespacing=1.4)
# connector
yc = ltop - 0.95
ax.add_patch(FancyArrowPatch((lx + lw_ + 0.04, yc), (rx - 0.04, yc), arrowstyle="<->",
mutation_scale=14, lw=2, color=INK2))
ax.text((lx + lw_ + rx) / 2, yc + 0.14, "conflict", ha="center", fontsize=10, weight="bold")
# mismatch summary under the abstract panel
sy = ltop - lh - 0.35
ax.text(lx + 0.02, sy, "The two statements differ in", fontsize=10, weight="bold", va="top")
ax.text(lx + 0.02, sy - 0.30, "1. direction: \u201creduced\u201d vs \u201cthree-fold\n"
" more abundant\u201d in CFBE41o- cells", fontsize=10, va="top")
ax.text(lx + 0.02, sy - 0.86, "2. proteins: PKN1 and PKN4 vs PKN2;\n"
" PKN4 appears nowhere else", fontsize=10, va="top")
save(fig, path)
# ---------------------------------------------------------------- B: evidence matrix
def fig_B(path):
cols = ["Abstract", "Results text", "Supp. Data 6, 7\n(CFBE/HBE ratio)", "Western blot\n(Fig. 7d)"]
rows = ["PKN1", "PKN2", "PKN3", "PKN4"]
cells = {
"PKN1": ["reduced", "presence\nuncertain", f"shared peptide only\n{r_sh:.1f}\u00d7 (also PKN2, 3)", "not tested"],
"PKN2": ["not named", "three-fold more\nabundant in CFBE", f"{r_pkn2:.1f}\u00d7\n(log2 {u2:.3f})", "more abundant\nin CFBE (n = 2 vs 2)"],
"PKN3": ["not named", "cluster member;\nno level stated", f"{r_pkn3:.3f}\u00d7\n(log2 {u1:.3f})", "not tested"],
"PKN4": ["reduced", "0 mentions", "not listed", "not tested"],
}
fig = plt.figure(figsize=(W, 3.7))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, 3.7); ax.axis("off")
x0, cw, rh, ytop = 0.85, 1.55, 0.62, 2.95
for j, c in enumerate(cols):
ax.text(x0 + j * cw + cw / 2, ytop + 0.30, c, ha="center", va="center", weight="bold", fontsize=10)
for i, r in enumerate(rows):
y = ytop - (i + 1) * rh
ax.text(x0 - 0.12, y + rh / 2, r, ha="right", va="center", weight="bold", fontsize=11)
for j, v in enumerate(cells[r]):
empty = v in ("not named", "not listed", "not tested", "0 mentions")
if j == 0:
fc, ec = (BG, GRID) if empty else (ORANGE_T, ORANGE)
else:
fc, ec = (BG, GRID) if empty else (BLUE_T, BLUE)
ax.add_patch(Rectangle((x0 + j * cw + 0.03, y + 0.03), cw - 0.06, rh - 0.06,
fc=fc, ec=ec, lw=1.5))
ax.text(x0 + j * cw + cw / 2, y + rh / 2, v, ha="center", va="center", fontsize=10,
color=INK2 if empty else INK, weight="bold" if (j == 0 and not empty) else "normal")
# legend
ly = 0.18
ax.add_patch(Rectangle((0.2, ly), 0.25, 0.18, fc=ORANGE_T, ec=ORANGE, lw=1.5))
ax.text(0.52, ly + 0.09, "claim as reported in the abstract", va="center", fontsize=10)
ax.add_patch(Rectangle((3.35, ly), 0.25, 0.18, fc=BLUE_T, ec=BLUE, lw=1.5))
ax.text(3.67, ly + 0.09, "what the body and supplements show", va="center", fontsize=10)
save(fig, path)
# ---------------------------------------------------------------- C: direction plot
def fig_C(path):
fig, ax = plt.subplots(figsize=(W, 3.4))
fig.subplots_adjust(left=0.10, right=0.97, top=0.80, bottom=0.17)
ys = {"PKN1": 3, "PKN2": 2, "PKN3": 1, "PKN4": 0}
ax.set_xscale("log", base=2)
ax.set_xlim(1 / 8, 8)
ax.set_ylim(-0.6, 3.75)
ticks = [1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8]
ax.set_xticks(ticks); ax.set_xticklabels(["1/8", "1/4", "1/2", "1", "2", "4", "8"])
ax.set_yticks(list(ys.values())); ax.set_yticklabels(list(ys.keys()), fontsize=11, color=INK)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axvline(1, color=INK2, lw=1)
ax.text(1, 3.77, "no change", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_xlabel("Abundance ratio, CFBE41o- / HBE41o- (log scale)")
# abstract claim: direction only, no value -> shaded half-row on the 'reduced' side
for p in ("PKN1", "PKN4"):
y = ys[p] - (0.12 if p == "PKN1" else 0)
ax.add_patch(Rectangle((1 / 8, y - 0.22), 1 - 1 / 8, 0.44, fc=ORANGE_T, ec=ORANGE, lw=1.5,
hatch="///", alpha=1))
ax.text(0.2, y, "abstract: \u201creduced\u201d", ha="center", va="center",
fontsize=10, color=INK, bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(2.8, ys["PKN1"], "body: presence\nuncertain", ha="center", va="center", fontsize=10, color=INK2)
ax.plot([r_sh], [ys["PKN1"]], "o", ms=10, mfc=BG, mec=BLUE, mew=2, zorder=5)
ax.text(r_sh, ys["PKN1"] + 0.26, f"{r_sh:.1f} shared peptide only (also PKN2, PKN3)", ha="center",
va="bottom", fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
ax.text(2.8, ys["PKN4"], "body, supplements:\nnot mentioned", ha="center", va="center", fontsize=10, color=INK2)
# recomputed / reported in body
ax.plot([r_pkn2], [ys["PKN2"]], "o", ms=10, color=BLUE, mec=BG, mew=2)
ax.text(r_pkn2, ys["PKN2"] + 0.2, f"{r_pkn2:.1f} (Results; Supp. Data 6, 7)", va="bottom",
ha="center", fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
ax.plot([r_pkn3], [ys["PKN3"]], "o", ms=10, color=BLUE, mec=BG, mew=2)
ax.text(r_pkn3, ys["PKN3"] + 0.2, f"{r_pkn3:.3f} (Supp. Data 6, 7 only)", va="bottom", ha="center",
fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
# legend
from matplotlib.lines import Line2D
h = [Rectangle((0, 0), 1, 1, fc=ORANGE_T, ec=ORANGE, hatch="///", lw=1.5),
Line2D([], [], marker="o", ls="", ms=10, color=BLUE, mec=BG, mew=2),
Line2D([], [], marker="o", ls="", ms=10, mfc=BG, mec=BLUE, mew=2)]
ax.legend(h, ["Abstract: direction only", "Ratio, protein-specific peptides",
"Ratio, shared peptide"],
loc="lower center", bbox_to_anchor=(0.5, 1.07), ncol=3, frameon=False, fontsize=10,
handlelength=1.4, columnspacing=1.2)
save(fig, path)
# ---------------------------------------------------------------- D: mention counts
def fig_D(path):
fig, ax = plt.subplots(figsize=(W, 3.0))
fig.subplots_adjust(left=0.10, right=0.97, top=0.82, bottom=0.17)
names = ["PKN1", "PKN2", "PKN3", "PKN4"]
for i, n in enumerate(names):
y = len(names) - 1 - i
a, b = counts[n]
ax.barh(y + 0.19, a, height=0.34, color=ORANGE, edgecolor=BG, lw=2)
ax.barh(y - 0.19, b, height=0.34, color=BLUE, edgecolor=BG, lw=2)
ax.text(a + 0.2, y + 0.19, str(a), va="center", fontsize=10)
ax.text(b + 0.2, y - 0.19, str(b) + (" (named as reduced in abstract)" if n == "PKN4" else ""),
va="center", fontsize=10, color=INK)
ax.set_yticks(range(4)); ax.set_yticklabels(names[::-1], fontsize=11, color=INK)
ax.set_xlim(0, 15); ax.set_xlabel("Number of mentions in the archived article")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Rectangle((0, 0), 1, 1, fc=ORANGE), Rectangle((0, 0), 1, 1, fc=BLUE)]
ax.legend(h, ["Abstract (where the claim is made)", "Rest of article (Results, legends, Methods)"],
loc="lower center", bbox_to_anchor=(0.5, 1.02), ncol=2, frameon=False, fontsize=10)
save(fig, path)
# ---------------------------------------------------------------- E: SD7 peptide ratios
def fig_E(path):
fig, ax = plt.subplots(figsize=(W, 3.0))
fig.subplots_adjust(left=0.31, right=0.97, top=0.80, bottom=0.18)
rows = [("PKN2 (Q16513)\nunique peptides (3)", r_pkn2, u2),
("shared peptide IADFGLCK\n(PKN1 node's only peptide)", r_sh, s),
("PKN3 (Q6P5Z2)\nunique peptides (3)", r_pkn3, u1)]
ax.set_xscale("log", base=2); ax.set_xlim(1 / 8, 8); ax.set_ylim(-0.4, 2.75)
ax.set_xticks([1 / 8, 1 / 4, 1 / 2, 1, 2, 4, 8])
ax.set_xticklabels(["1/8", "1/4", "1/2", "1", "2", "4", "8"])
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axvline(1, color=INK2, lw=1)
ax.text(1, 2.77, "no change", ha="center", va="bottom", fontsize=10, color=INK2)
for i, (lab, r, l2) in enumerate(rows):
y = 2 - i
ax.plot([1, r], [y, y], color=BLUE, lw=2)
ax.plot([r], [y], "o", ms=10, color=BLUE, mec=BG, mew=2)
ax.text(r, y + 0.2, f"{r:.3f} (log2 {l2:.3f})", va="bottom", ha="center", fontsize=10,
bbox=dict(fc=BG, ec="none", pad=1))
ax.set_yticks([2, 1, 0]); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlabel("Abundance ratio, CFBE41o- / HBE41o- (log scale)")
ax.set_title("Every PKN peptide ratio in Supplementary Data 6 and 7",
fontsize=10, color=INK2, loc="left", pad=20)
save(fig, path)
print("\n=== 4. Figures")
fig_A(os.path.join(CAND, "fig_A_text_panels"))
fig_B(os.path.join(CAND, "fig_B_evidence_matrix"))
fig_C(os.path.join(CAND, "fig_C_direction_plot"))
fig_D(os.path.join(CAND, "fig_D_mention_counts"))
fig_E(os.path.join(CAND, "fig_E_peptide_ratios"))
# The figures used in writeup.md: fig1 = candidate A, fig2 = candidate C (until a design is chosen)
for src, dst in [("fig_A_text_panels", "fig1"), ("fig_C_direction_plot", "fig2")]:
for ext in (".svg", ".png"):
shutil.copyfile(os.path.join(CAND, src + ext), os.path.join(HERE, dst + ext))
print(" copied candidates A -> fig1, C -> fig2")
Abstract says host genetics explains almost half of meningitis susceptibility; the paper estimates 29%
Nat Commun (2019) · doi:10.1038/s41467-019-09976-3 · PMC6520353
Summary
The abstract states that human genetic variation "explains almost half of variation in susceptibility to pneumococcal meningitis and one-third of variation in severity". Table 4 gives 0.25 (GCTA) and 0.29 (LDAK) for susceptibility and 0.29 (GCTA) and 0.49 (LDAK) for severity; the Results and the Discussion quote only the LDAK values, as 29% ± 7% for susceptibility and 49% ± 14% for severity (the Discussion gives the severity error as ± 9%). No susceptibility estimate in the paper is near one half: 0.5 lies outside both recomputed susceptibility 95% confidence intervals. The abstract's two magnitudes match the body's headline values only if the phenotypes are swapped. The abstract also implies that host genetics matters more for susceptibility than for severity, while the paper's headline estimates rank them the other way. The significance of either estimate is unaffected, and so are the paper's other findings.
What the paper reports
Abstract:
In a genome-wide association study of human and pathogen we show that human variation explains almost half of variation in susceptibility to pneumococcal meningitis and one-third of variation in severity, identifying variants in CCDC33 associated with susceptibility.
Results, "Human genetics associates with susceptibility and severity", paragraph 3:
We then used two standard methods to calculate SNP heritability, which both showed that variation in host genetics explains 29% ± 7% of the observed variation in pneumococcal meningitis susceptibility and 49% ± 14% of the variation in meningitis severity (Table 4).
Discussion, paragraph 2:
Host genetics explained 29% ± 7% of variation in susceptibility to meningitis.
We found no evidence for pathogen genetics affecting the severity of disease, whereas human genetics explained 49% ± 9% of this variation.
Table 4, "Human SNP heritability (h2SNP) of meningitis in the MeninGene cohort" (columns: Phenotype | Method | Heritability | Error | p-value):
Susceptibility (pneumococcal) | GCTA | 0.25 | 0.05 | 2.4 × 10–6
Susceptibility (pneumococcal) | LDAK | 0.29 | 0.07 | 3.9 × 10–6
Severity (any species) | GCTA | 0.29 | 0.11 | 2.8 × 10–5
Severity (any species) | LDAK | 0.49 | 0.14 | 1.4 × 10–4
Table 4 footer:
Heritabilities are shown on the liability scale (adjusted for population prevalence and case ascertainment ratio).
What the paper's own numbers show
Method. The abstract gives words, not numbers, so I read "almost half" as about 0.50 and "one-third" as 0.333. I compared each with both Table 4 estimates for the same phenotype. I built 95% confidence intervals as estimate ± 1.96 × the Table 4 "Error" column, treating that column as a standard error, and checked whether each interval contains 0.5 or 1/3. I also checked that the Results and Discussion percentages match the Table 4 rows. Finally I tested whether the two LDAK estimates differ from each other, with a z-test that treats them as independent. That is an approximation: the two phenotypes share control samples, so the estimates are not strictly independent. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Host h² for susceptibility | Abstract | "almost half" (≈ 0.50) | Table 4 gives 0.25 (GCTA) and 0.29 (LDAK). The nearest is 0.21 below 0.50 |
| Host h² for severity | Abstract | "one-third" (≈ 0.33) | Table 4 gives 0.29 (GCTA, 0.04 away) and 0.49 (LDAK, 0.16 away). The text's headline value is 0.49 |
| Susceptibility, GCTA | Table 4 | 0.25 ± 0.05 | 95% CI 0.152–0.348, which excludes 0.5 |
| Susceptibility, LDAK | Table 4; Results p3; Discussion p2 | 0.29 ± 0.07; "29% ± 7%" | Text matches the table. 95% CI 0.153–0.427, which excludes 0.5 |
| Severity, GCTA | Table 4 | 0.29 ± 0.11 | 95% CI 0.074–0.506 |
| Severity, LDAK | Table 4; Results p3 | 0.49 ± 0.14; "49% ± 14%" | Text matches the table. 95% CI 0.216–0.764 |
| Abstract reading vs headline value, susceptibility | Abstract vs Results p3 | ≈ 0.50 vs 0.29 | 21.0 percentage points higher, 1.72 times the paper's estimate |
| Abstract reading vs headline value, severity | Abstract vs Results p3 | ≈ 0.33 vs 0.49 | 15.7 percentage points lower |
| Value nearest to "almost half" anywhere in Table 4 | Table 4 | — | Severity, LDAK, 0.49 (0.01 from 0.50) |
| Severity minus susceptibility, LDAK | Table 4 | 0.49 − 0.29 | Difference 0.20, SE 0.157, z = 1.28, two-sided p = 0.20 (GCTA: 0.04, z = 0.33, p = 0.74) |
| Severity error | Discussion p2 | "49% ± 9%" | Table 4 and the Results give ± 14%. With ± 9% the interval would be 0.314–0.666 |
The arithmetic in the body is internally consistent: the Results and Discussion quote the LDAK rows of Table 4. The inconsistency is between the abstract and everything else. For susceptibility no estimator in the paper gives a value near one half. For severity, "one-third" is close to the GCTA estimate (0.29), but the paper presents the LDAK estimate (0.49) as its headline severity figure in both the Results and the Discussion. Read together, the abstract's two magnitudes match the body's headline values (29% and 49%) with the phenotypes swapped.
Figure
Figure 1. On the left is the abstract sentence. On the right is the Results sentence that cites Table 4 (verbatim excerpts). Below each are the values each passage gives for the same two phenotypes. Orange diamonds mark the abstract's words read as fractions. Blue dots mark the Table 4 LDAK estimates quoted in the Results. For susceptibility, the abstract's "almost half" (≈ 0.50) conflicts with 0.29 ± 0.07, and 0.5 is outside both recomputed susceptibility 95% confidence intervals. For severity, the abstract's "one-third" (≈ 0.33) sits below the headline 0.49 ± 0.14, although it is close to the GCTA estimate of 0.29. All values are host SNP heritability on the liability scale.
Figure 2. All four Table 4 estimates (blue dots) with 95% confidence intervals recomputed as ± 1.96 × the printed error, on one heritability axis that starts at zero (no heritability). The orange vertical marks show where the abstract's words fall for each phenotype. The susceptibility mark at 0.5 lies to the right of both susceptibility intervals. The severity mark at 1/3 lies inside both severity intervals. The intervals are wide, so the severity wording is not ruled out statistically, but it does not match the value the paper reports as its headline severity estimate.
Why it matters
The abstract is the part of the paper most readers see, and the host-heritability sentence is its main quantitative claim about human genetics. As written it overstates the host contribution to susceptibility by about 20 percentage points (≈ 0.50 against 0.29) and understates the headline severity estimate (≈ 0.33 against 0.49). It also reverses the order of the two phenotypes. The abstract suggests that host genetics matters more for susceptibility, while the paper's headline estimates suggest the opposite. Because the two estimates are not significantly different (z = 1.28, p = 0.20 under an independence approximation), the reversal matters less for the substance than for how the result is described. Anyone who cites the abstract's figures, for example in a review comparing heritability across infectious diseases, would carry forward a susceptibility value the paper does not support.
What the discrepancy does not affect: both host estimates remain significantly above zero (Table 4 p-values 3.9 × 10–6 and 1.4 × 10–4 for LDAK). The pathogen heritability results (70% for invasiveness, 0% for severity) are reported separately and correctly. The CCDC33, UBE2U, pspC and zmpD findings and the vaccine-target discussion are not affected.
Explanations considered
- A different estimator. Table 4 reports GCTA and LDAK for each phenotype. For susceptibility both (0.25 and 0.29) are well below one half, so neither estimator supports "almost half". For severity, GCTA's 0.29 is close to "one-third", but the susceptibility half of the sentence still has no supporting estimate. The Results and Discussion use the LDAK values throughout. Does not account for the discrepancy.
- A different scale or population. All Table 4 values are on the liability scale (prevalence 1 × 10–3) in the Dutch MeninGene cohort, and the text quotes those same values. The paper reports no other host heritability, for example observed-scale or pooled invasive disease, that could be near 0.5 for susceptibility. Does not account for it.
- Confusion with a pathogen estimate. The abstract reports the pathogen figures (70% of invasive potential, no effect on severity) in a separate sentence. No pathogen estimate is presented as a host susceptibility value. Not applicable.
- The pooled severity phenotype. Severity heritability was estimated by combining severe pneumococcal cases with severe cases from any species. That changes how severity is defined, not which number belongs to which phenotype. Does not account for it.
- Rounding or uncertainty. "Almost half" cannot be read as a loose rounding of 0.29, and 0.5 lies outside both susceptibility 95% confidence intervals (upper limits 0.348 and 0.427). Does not account for it.
- A text-conversion artifact. The conflict is between words ("almost half", "one-third") and printed percentages and decimals. There are no minus signs, superscripts or column alignments that a conversion could have garbled, and the publisher's page has the same wording. Ruled out.
- Carried over from an earlier version. The bioRxiv preprint (doi:10.1101/386078, posted 2018-08-07) has the same abstract sentence ("human variation explains almost half of variation in susceptibility to pneumococcal meningitis and one-third of variation in severity"). That shows when the wording appeared, not why. It does not reconcile the abstract with the published Table 4.
- A second, minor inconsistency. The Discussion gives severity as "49% ± 9%", while Table 4 and the Results give ± 14%. This is separate from the main discrepancy and does not change the estimate itself.
Status
As of 2026-10-23:
- Version of record. The publisher's HTML (nature.com/articles/s41467-019-09976-3) has the same abstract, Results and Discussion wording as the PMC archive, including "49% ± 9%" in the Discussion. The Table 4 rows were checked in the PMC archive and in the Europe PMC full-text XML, which agree: 0.25 ± 0.05, 0.29 ± 0.07, 0.29 ± 0.11 and 0.49 ± 0.14.
- Supplements. Both electronic supplementary files on the publisher's site were searched: the Supplementary Information PDF (36 pages) and the Peer Review File (19 pages). Neither reports a host heritability value for susceptibility or severity, and neither mentions the abstract wording. The external data deposits (figshare, Zenodo and the analysis-code repository) hold summary statistics and metadata, not alternative heritability estimates.
- Corrections. Crossref shows no update relations (no correction, erratum or retraction). The only relation is to the preprint. PMC and Europe PMC list no correction or retraction notice; Europe PMC's only linked record is the preprint. The Crossref record that carries Retraction Watch data has no correction or retraction entry.
- PubPeer. PubPeer's search API returned a human-verification challenge and could not be queried automatically. A web search for the title on PubPeer found no PubPeer entry. Absence of comments there is therefore not confirmed.
- Preprint. bioRxiv doi:10.1101/386078, version 1 (2018-08-07), has the same abstract wording.
Sources: PMC6520353; publisher version; Crossref record; Europe PMC; bioRxiv preprint.
Suggested correction
Bring the abstract into line with Table 4 and the Results, for example:
… we show that human variation explains around 30% of variation in susceptibility to pneumococcal meningitis and almost half of variation in severity, …
If the authors prefer to describe severity by the GCTA estimate (0.29), the abstract should say so and give the same order of magnitude for both phenotypes ("around 30%"). In that case the Results and Discussion, which present 49% as the severity figure, would also need to change. Separately, the Discussion's "49% ± 9%" should read "49% ± 14%" to match Table 4 and the Results.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC6520353 and regenerate the figures.
Run: python recompute.py.txt
All inputs are copied from the article as archived in PubMed Central (PMC6520353):
Abstract, Results ("Human genetics associates with susceptibility and severity", paragraph 3),
Discussion paragraph 2 and Table 4 (paragraphs counted from 1). Outputs are written next to this file:
fig1.svg/png, fig2.svg/png and candidates/design_1..5.svg/png (five figure designs).
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs
# Table 4: Human SNP heritability, liability scale (estimate, error, printed p)
T4 = {
("Susceptibility", "GCTA"): (0.25, 0.05, "2.4e-6"),
("Susceptibility", "LDAK"): (0.29, 0.07, "3.9e-6"),
("Severity", "GCTA"): (0.29, 0.11, "2.8e-5"),
("Severity", "LDAK"): (0.49, 0.14, "1.4e-4"),
}
# Abstract wording, read as the fractions the words denote
ABSTRACT = {"Susceptibility": ("almost half", 0.5), "Severity": ("one-third", 1 / 3)}
# Results paragraph 3 and Discussion paragraph 2 (percent, +/- percent)
RESULTS = {"Susceptibility": (29, 7), "Severity": (49, 14)}
DISCUSSION = {"Susceptibility": (29, 7), "Severity": (49, 9)}
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- numbers
print("== Table 4 estimates with 95% CI (estimate +/- 1.96 x error)")
ci = {}
for (ph, m), (h, se, p) in T4.items():
lo, hi = h - Z * se, h + Z * se
ci[(ph, m)] = (lo, hi)
print(f"{ph:15s} {m}: {h:.2f} +/- {se:.2f} 95% CI {lo:.3f} to {hi:.3f} "
f"contains 0.5: {lo <= 0.5 <= hi} contains 1/3: {lo <= 1/3 <= hi}")
print("\n== Results and Discussion text vs Table 4 LDAK rows")
for ph in ("Susceptibility", "Severity"):
h, se, _ = T4[(ph, "LDAK")]
r, d = RESULTS[ph], DISCUSSION[ph]
print(f"{ph}: Results {r[0]}% +/- {r[1]}% Discussion {d[0]}% +/- {d[1]}% "
f"Table 4 LDAK {h*100:.0f}% +/- {se*100:.0f}% "
f"Results match: {r == (round(h*100), round(se*100))} "
f"Discussion match: {d == (round(h*100), round(se*100))}")
h, _, _ = T4[("Severity", "LDAK")]
print(f"Severity CI if the Discussion error 0.09 were used: {h-Z*0.09:.3f} to {h+Z*0.09:.3f}")
print("\n== Distance from each abstract word to each Table 4 estimate")
for ph, (word, val) in ABSTRACT.items():
print(f"Abstract '{word}' ({val:.3f}) for {ph}:")
for (ph2, m), (h, se, _) in T4.items():
print(f" {ph2:15s} {m}: |{h:.2f} - {val:.3f}| = {abs(h-val):.3f}"
f"{' <- same phenotype' if ph2 == ph else ''}")
best = {ph: min((abs(T4[(ph, m)][0] - v), m) for m in ("GCTA", "LDAK"))
for ph, (_, v) in ABSTRACT.items()}
print("Closest same-phenotype estimate:", {k: (m, round(dist, 3)) for k, (dist, m) in best.items()})
print("\n== Size of the discrepancy (abstract reading vs headline LDAK value)")
for ph, (word, val) in ABSTRACT.items():
h = T4[(ph, "LDAK")][0]
print(f"{ph}: abstract {val:.3f} vs {h:.2f}: difference {100*(val-h):+.1f} percentage points, "
f"ratio {val/h:.2f}")
print("\n== Is severity h2 different from susceptibility h2? (LDAK, treated as independent)")
(h1, s1, _), (h2, s2, _) = T4[("Susceptibility", "LDAK")], T4[("Severity", "LDAK")]
zd = (h2 - h1) / math.hypot(s1, s2)
print(f"difference {h2-h1:.2f}, SE {math.hypot(s1, s2):.4f}, z {zd:.2f}, two-sided p {2*norm.sf(abs(zd)):.2f}")
(g1, gs1, _), (g2, gs2, _) = T4[("Susceptibility", "GCTA")], T4[("Severity", "GCTA")]
zg = (g2 - g1) / math.hypot(gs1, gs2)
print(f"GCTA: difference {g2-g1:.2f}, z {zg:.2f}, two-sided p {2*norm.sf(abs(zg)):.2f}")
# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def save(fig, stem):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(f"{stem}.{ext}", bbox_inches="tight", pad_inches=0.15, **kw)
plt.close(fig)
def axis_clean(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
# ---- design 1: two text panels joined by annotated connectors
def design_1(stem):
fig = plt.figure(figsize=(W, 4.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100); ax.set_ylim(0, 62); ax.axis("off")
panels = [
(1, "Abstract", ORANGE,
"\u201chuman variation explains almost\nhalf of variation in susceptibility\n"
"to pneumococcal meningitis and\none-third of variation in severity\u201d",
[("Susceptibility", "almost half (\u2248 0.50)"), ("Severity", "one-third (\u2248 0.33)")]),
(61, "Results and Table 4", BLUE,
"\u201cvariation in host genetics explains\n29% \u00b1 7% of the observed variation\n"
"in pneumococcal meningitis\nsusceptibility and 49% \u00b1 14% of the\n"
"variation in meningitis severity\u201d",
[("Susceptibility", "0.29 \u00b1 0.07"), ("Severity", "0.49 \u00b1 0.14")]),
]
rows_y = (18, 7.5)
for x0, title, col, quote, rows in panels:
ax.add_patch(FancyBboxPatch((x0, 2), 38, 57, boxstyle="round,pad=0.3,rounding_size=1.5",
fc="white", ec=col, lw=2))
ax.text(x0 + 2, 56, title, fontsize=11.5, weight="bold", va="top", color=INK)
ax.text(x0 + 2, 49, quote, fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.plot([x0 + 2, x0 + 36], [23, 23], color=GRID, lw=1)
for (ph, val), y in zip(rows, rows_y):
ax.text(x0 + 4, y + 2.6, ph, fontsize=10.5, va="center", color=INK2)
ax.text(x0 + 4, y - 1.6, val, fontsize=11.5, va="center", weight="bold", color=INK)
ax.plot(x0 + 2.3, y - 1.6, "D" if col == ORANGE else "o", ms=8, color=col)
labels = ("0.50 vs 0.29\n(0.5 is outside\nboth 95% CIs)",
"0.33 vs 0.49\n(Table 4 GCTA: 0.29)")
for y, lab in zip(rows_y, labels):
ax.add_patch(FancyArrowPatch((40.5, y - 1.6), (60, y - 1.6), arrowstyle="<->", mutation_scale=12,
color=INK2, lw=2))
ax.text(50.2, y + 0.2, lab, ha="center", va="bottom", fontsize=10, color=INK, linespacing=1.2)
ax.text(50.2, 37, "Same\nphenotypes,\nmagnitudes\nin reverse\norder", ha="center", va="center",
fontsize=10.5, color=INK, weight="bold")
ax.text(50.2, -2.5, "Host SNP heritability, liability scale. The abstract gives words, read here as fractions. "
"LDAK values shown on the right.", ha="center", fontsize=10, color=INK2)
save(fig, stem)
# ---- design 2: dot-and-interval on one h2 axis
def design_2(stem):
fig, ax = plt.subplots(figsize=(W, 3.6))
order = [("Susceptibility", "GCTA"), ("Susceptibility", "LDAK"),
("Severity", "GCTA"), ("Severity", "LDAK")]
ys = [4.6, 3.8, 1.8, 1.0]
for (ph, m), y in zip(order, ys):
h, se, _ = T4[(ph, m)]
lo, hi = ci[(ph, m)]
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(h, y, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(0.81, y, f"{m} {h:.2f} ({lo:.2f}\u2013{hi:.2f})", va="center",
fontsize=10, color=INK2)
for ph, (yb, yt) in (("Susceptibility", (3.4, 5.0)), ("Severity", (0.6, 2.2))):
word, v = ABSTRACT[ph]
ax.plot([v, v], [yb, yt], color=ORANGE, lw=2.5)
ax.plot(v, yt, "D", ms=8, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(v, yt + 0.22, f"Abstract: \u201c{word}\u201d", ha="center", fontsize=10,
color=INK, weight="bold")
ax.axvline(0, color=INK2, lw=0.8)
ax.set_xlim(-0.02, 1.12); ax.set_ylim(0.2, 5.8)
ax.set_xticks([0, 0.2, 0.4, 0.6, 0.8])
ax.set_yticks([4.2, 1.4]); ax.set_yticklabels(["Susceptibility\n(pneumococcal)", "Severity\n(any species)"],
fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Host SNP heritability, liability scale (h\u00b2 = 0 is no heritability)", x=0.36)
axis_clean(ax)
h1 = [plt.Line2D([], [], color=BLUE, marker="o", ms=8, lw=2,
label="Table 4 estimate, 95% CI recomputed as \u00b11.96 \u00d7 error"),
plt.Line2D([], [], color=ORANGE, marker="D", ms=8, lw=2.5,
label="Abstract wording, read as a fraction")]
ax.legend(handles=h1, loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=1,
frameon=False, fontsize=10)
save(fig, stem)
# ---- design 3: dumbbell, abstract vs headline LDAK value per phenotype
def design_3(stem):
fig, ax = plt.subplots(figsize=(W, 2.9))
for ph, y in (("Susceptibility", 2), ("Severity", 1)):
word, v = ABSTRACT[ph]
h, se, _ = T4[(ph, "LDAK")]
lo, hi = ci[(ph, "LDAK")]
ax.plot([lo, hi], [y - 0.12, y - 0.12], color=BLUE, lw=2, alpha=0.45)
ax.add_patch(FancyArrowPatch((v, y), (h, y), arrowstyle="-|>", mutation_scale=14,
color=INK2, lw=2, shrinkA=6, shrinkB=6))
ax.plot(v, y, "D", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(h, y, "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(v, y + 0.2, f"“{word}”", ha="center", fontsize=10, color=INK)
ax.text(h, y + 0.2, f"{h:.2f} ± {se:.2f}", ha="center", fontsize=10, color=INK)
ax.text(0.985, y, f"{100*(v-h):+.0f} pp".replace("-", "\u2212"), ha="right", va="center", fontsize=10.5,
weight="bold", color=INK)
ax.set_xlim(0, 1.0); ax.set_ylim(0.5, 2.6)
ax.set_yticks([2, 1]); ax.set_yticklabels(["Susceptibility", "Severity"], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Host SNP heritability, liability scale")
axis_clean(ax)
hs = [plt.Line2D([], [], color=ORANGE, marker="D", ms=9, lw=0, label="Abstract (word read as fraction)"),
plt.Line2D([], [], color=BLUE, marker="o", ms=9, lw=0, label="Results / Table 4, LDAK"),
plt.Line2D([], [], color=BLUE, lw=2, alpha=0.45, label="its recomputed 95% CI")]
ax.legend(handles=hs, loc="upper center", bbox_to_anchor=(0.5, -0.3), ncol=3,
frameon=False, fontsize=10)
save(fig, stem)
# ---- design 4: slope chart, abstract -> body, lines cross
def design_4(stem):
fig, ax = plt.subplots(figsize=(W, 3.8))
for ph, ls in (("Susceptibility", "-"), ("Severity", "--")):
word, v = ABSTRACT[ph]
h, se, _ = T4[(ph, "LDAK")]
ax.plot([0, 1], [v, h], color=INK2, lw=2, ls=ls)
ax.plot(0, v, "D", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(1, h, "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(-0.04, v, f"{ph}: “{word}”", ha="right", va="center", fontsize=10.5)
ax.text(1.04, h, f"{ph}: {h:.2f} ± {se:.2f}", ha="left", va="center", fontsize=10.5)
ax.set_xlim(-1.15, 1.85); ax.set_ylim(0, 0.6)
ax.set_xticks([0, 1]); ax.set_xticklabels(["Abstract\n(orange: words read as fractions)",
"Results / Table 4, LDAK\n(blue)"], fontsize=10.5, color=INK)
ax.set_ylabel("Host SNP heritability, liability scale")
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.text(0.5, 0.56, "Lines cross: the abstract ranks the phenotypes\nthe opposite way to the paper's own estimates",
ha="center", fontsize=10, color=INK2)
save(fig, stem)
# ---- design 5: one number line, distance from each abstract word to its own estimates
def design_5(stem):
fig, ax = plt.subplots(figsize=(W, 3.6))
ax.axhline(0, color=INK2, lw=1, zorder=1)
for h, lab, ha in ((0.25, "0.25\nSusc. GCTA", "right"),
(0.29, "0.29\nSusc. LDAK,\nSev. GCTA", "left"),
(0.49, "0.49\nSev. LDAK", "left")):
ax.plot(h, 0, "o", ms=8, color=BLUE, mec=BG, mew=1.5, zorder=5) # drawn over the diamonds
ax.text(h + (0.006 if ha == "left" else -0.006), -0.12, lab, ha=ha, va="top",
fontsize=10, color=INK, linespacing=1.15)
for ph, (word, v) in ABSTRACT.items():
ax.plot(v, 0, "D", ms=13, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.plot([v, v], [0.12, 0.42], color=ORANGE, lw=2)
ax.text(v + (0.006 if v > 0.4 else -0.006), 0.48, f"Abstract, {ph.lower()}:\n\u201c{word}\u201d",
ha="left" if v > 0.4 else "right", va="bottom",
fontsize=10, color=INK)
for ph, y in (("Susceptibility", -1.45), ("Severity", -2.35)):
word, v = ABSTRACT[ph]
dist, m = best[ph]
h = T4[(ph, m)][0]
ax.annotate("", xy=(h, y), xytext=(v, y),
arrowprops=dict(arrowstyle="|-|", color=INK2, lw=2, mutation_scale=4))
ax.text(min(v, h), y + 0.13, f"{ph}: \u201c{word}\u201d is {dist:.2f} from the\nnearest "
f"{ph.lower()} estimate ({m} {h:.2f})", va="bottom", fontsize=10, color=INK2)
ax.set_xlim(0, 0.72); ax.set_ylim(-2.5, 1.15)
ax.set_xticks([0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6])
ax.set_yticks([])
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.set_xlabel("Host SNP heritability, liability scale (blue dots: Table 4; orange diamonds: abstract)",
x=0.5)
save(fig, stem)
DESIGNS = [design_1, design_2, design_3, design_4, design_5]
for i, f in enumerate(DESIGNS, 1):
f(os.path.join(CAND, f"design_{i}"))
design_1(os.path.join(HERE, "fig1"))
design_2(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)
Headline protection p-value cannot be reproduced from the paper's counts using its stated test
Nat Commun (2023) · doi:10.1038/s41467-023-36907-0 · PMC9990026
Summary
The paper reports that after nine repeated low-dose intrarectal SIVmac251 challenges, 4 of 7 vaccinated macaques and 6 of 6 controls were infected, giving a vaccine efficacy (VE) of 43%, and attaches "P < 0.0001" to that result. The paper says this comparison used a one-tailed Fisher's exact test. Applied to the paper's own counts (3/7 vs 0/6 uninfected), that test gives p = 35/286 = 0.122 (two-tailed 0.192), which is above the paper's significance threshold of 0.05. With 7 vs 6 animals, no Fisher (conditional) or permutation test can give a p-value below 1/1716 = 5.8 × 10⁻⁴. Unconditional exact tests (Barnard, Boschloo) cannot go below (7/13)⁷(6/13)⁶ = 1.3 × 10⁻⁴, and a large-sample two-sided log-rank test on any 7-vs-6 outcome cannot go below 1.7 × 10⁻⁴. Each of these floors is above 0.0001, so none of these tests can produce "P < 0.0001" with these group sizes, whatever the outcome. The 43% point estimate is arithmetically correct. The statistical significance claimed for the paper's primary protection result is not supported.
What the paper reports
Results, section "Tonsil vaccination with MVA prevented acquisition in 3/7 animals after 9 repeated low dose IR challenges with SIVmac251", first paragraph:
All 6/6 control animals were infected by the 9th low dose IR challenge whereas only 4/7 vaccinated animals were infected by the 9th challenge demonstrating a VE = 43% (Fig. 2a; P < 0.0001).
Methods, "Data analysis":
Rate of SIV acquisition after each challenge was determined using Kaplan–Meier plots and vaccine efficacy (VE) after the 9th IR challenge was determined using the formula VE = 1 – [(infection rate among vaccinated animals)/(infection rate among control animals)] × 100 and followed by Fisher’s exact test as described by Klasse et al.101.
A P < 0.05 was considered significant.
Figure 2 legend, panel (a):
(a) Study design and percentage of uninfected animals after repeated low dose intrarectal challenge with SIVmac251 in tonsil vaccinated (n = 6) and control (n = 6) groups. Statistical analysis was performed using one-tailed Fishers Exact Test as described in ref. 101 and a P < 0.05 (*) was considered significant.
Figure 2a, Kaplan–Meier panel (text printed inside the plot, read from the published figure image):
VE = 43% P = <0.0001
Methods, "Animals, vaccination, infection, and samples" (group sizes):
(unvaccinated control; n = 6 and vaccinated; n = 7)
Abstract:
43% (3/7) of vaccinated macaques remained uninfected after 9 challenges as compared to the unvaccinated control (0/6) animals.
Introduction, last paragraph:
Our results demonstrated a significant level of protection from acquisition
What the paper's own numbers show
Method. At the 9th challenge the paper's counts form a 2×2 table: vaccinated, 3 uninfected and 4 infected (n = 7); control, 0 uninfected and 6 infected (n = 6). Fisher's exact test conditions on the margins: 3 uninfected animals out of 13, 7 of them vaccinated. The one-tailed p-value in the direction of protection is the probability that all 3 uninfected animals fall in the vaccinated group if vaccination had no effect, which is C(7,3)/C(13,3) = 35/286. The full null distribution of the number of uninfected animals in the vaccinated group (0, 1, 2 or 3 of 3) is 20/286, 105/286, 126/286 and 35/286. The two-tailed p adds the 0-of-3 tail, giving (35 + 20)/286. For the lower bound, the most extreme possible result (all 7 vaccinated uninfected, all 6 controls infected) has one-tailed p = 1/C(13,7) = 1/1716. Its two-tailed p is also 1/1716, because the only opposite-tail table (just 1 of the 7 uninfected animals in the vaccinated group) has probability 7/1716 and is therefore not added. A search over every 7-vs-6 table gives the same minimum two-tailed Fisher p. Unconditional exact tests do not condition on the margins and have a lower floor here: for the most extreme table, Barnard's and Boschloo's tests both give (7/13)⁷(6/13)⁶ = 1.27 × 10⁻⁴. The Kaplan–Meier curves in Fig. 2a show when each animal became infected, which allows a log-rank (Mantel–Cox) test. Read from the published figure, approximately: controls step from 100% to 83%, 67%, 33%, 17% and 0% at challenges 1, 2, 4 (two animals), 7 and 9. Vaccinated animals step from 100% to 57% at challenge 2 (three animals) and to 43% at challenge 4, and three remain uninfected. The Fisher values were computed with scipy.stats.fisher_exact and checked against the closed-form binomial ratios, the unconditional values with scipy.stats.barnard_exact and boschloo_exact, and the log-rank values with the standard Mantel–Cox formula, all in recompute.py.txt. To find the log-rank floor, the script also enumerates all 57,257,200 possible outcomes, with each animal infected at one of challenges 1 to 9 or uninfected after the 9th.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Vaccine efficacy, 1 − (4/7)/(6/6) | Results; Abstract | 43% | 42.9% (consistent) |
| p-value for the 9th-challenge acquisition comparison | Results, first acquisition paragraph | P < 0.0001 | 0.122 with the stated test |
| One-tailed Fisher exact, 3/7 vs 0/6 uninfected | Test named in Methods and Fig. 2a legend | (not given separately) | 35/286 = 0.1224 |
| Two-tailed Fisher exact, 3/7 vs 0/6 | Same table | — | 55/286 = 0.1923 |
| One-tailed Fisher exact if n = 6 vaccinated, 3/6 vs 0/6 | Fig. 2a legend states n = 6 | — | 20/220 = 0.0909 (two-tailed 0.1818) |
| Smallest attainable Fisher or permutation p, 7 vs 6 animals | Lower bound for conditional exact tests of this design | — | one-tailed 1/1716 = 5.83 × 10⁻⁴; two-tailed also 1/1716 = 5.83 × 10⁻⁴ |
| Smallest attainable Barnard or Boschloo p, 7 vs 6 animals | Lower bound for unconditional exact tests of this design | — | (7/13)⁷(6/13)⁶ = 1.27 × 10⁻⁴ (still above 1 × 10⁻⁴) |
| Barnard exact, 3/7 vs 0/6 | Same table | — | one-sided 0.106 (two-sided 0.106) |
| Boschloo exact, 3/7 vs 0/6 | Same table | — | one-sided 0.054 (two-sided 0.108) |
| Log-rank (Mantel–Cox), Fig. 2a curves as read | Fig. 2a Kaplan–Meier panel (approximate readings) | — | χ² = 1.55, p = 0.21 (one-sided 0.11); χ² = 1.07, p = 0.30 if the first vaccinated step is read at challenge 1 |
| Largest log-rank χ² over every possible 7-vs-6 outcome | Lower bound for the large-sample log-rank test | — | χ² = 14.14, two-sided p = 1.7 × 10⁻⁴ |
| Fewest uninfected vaccinated animals (of 7, vs 0/6) needed for one-tailed p < 0.05 | Same test | — | 4 of 7 (p = 0.0490) |
| Ratio of recomputed p to reported bound | — | — | 0.1224 / 0.0001 ≈ 1,224 |
Figure
Figure 1. The p-value reported for the 9th-challenge acquisition comparison in the Results and the Fig. 2a panel (orange, "P < 0.0001", drawn as an upper bound with an arrow) and p-values recomputed from the paper's own counts and curves (blue), all on a single log-scaled axis. The dashed line marks the paper's significance threshold, α = 0.05. The one-tailed Fisher exact test named in the Methods and the Fig. 2a legend gives 0.122 for 3/7 vs 0/6 uninfected. The two-tailed version gives 0.192. If the vaccinated group had the n = 6 stated in the Fig. 2a legend, the one-tailed value is 0.091. The unconditional exact tests give one-sided 0.054 (Boschloo) and 0.106 (Barnard). A log-rank test on the Fig. 2a curves, as read approximately, gives 0.21. The shaded band marks values below 1/1716 = 5.8 × 10⁻⁴, the smallest p a Fisher or permutation test can give with 7 vs 6 animals. The dotted line marks 1.3 × 10⁻⁴, the smallest p a Barnard or Boschloo test can give. The reported bound lies below both.
Figure 2. One-tailed Fisher exact p-values for every possible outcome of the 9th-challenge comparison with 0/6 controls uninfected, as the number of uninfected vaccinated animals runs from 1/7 to 7/7 (gray). The observed 3/7 is shown in blue (p = 0.122). Even the most extreme outcome, all seven vaccinated animals uninfected, gives p = 1/1716 = 5.8 × 10⁻⁴. That value is above the reported "P < 0.0001" (orange line). The design would have needed at least 4 of 7 vaccinated animals uninfected to reach p < 0.05.
Why it matters
The 9th-challenge acquisition comparison is the paper's primary efficacy result. The paper's title says the vaccine "protects macaques", the Introduction describes "a significant level of protection from acquisition", and the Abstract presents the 3/7 vs 0/6 comparison as its main finding. "P < 0.0001" is the only p-value the paper gives for this comparison (Results text and Fig. 2a panel). Recomputed with the stated test, the comparison does not reach the paper's own α of 0.05 (one-tailed p = 0.122). A reader would therefore take a result in 13 animals that does not reach significance to be highly significant.
Some parts of the paper are not affected. The VE point estimate (42.9%, reported as 43%) and the descriptive counts are internally consistent, and so is the report that one vaccinated animal stayed uninfected through 22 challenges. The separate analyses of acute viremia, CD4 counts, cytokines and immune correlates use other tests and other data and are not addressed here. The point concerns only the inferential support for protection from acquisition.
Explanations considered
- A different test, such as a log-rank test on the Kaplan–Meier curves. The Methods say Kaplan–Meier plots were used for the rate of acquisition. However, the reported P is attached to the VE at the 9th challenge, and both the Methods and the Fig. 2a legend name Fisher's exact test for that comparison. The paper does not report a log-rank or Mantel–Cox test anywhere. The infection times can be read approximately from the Fig. 2a curves: controls infected at challenges 1, 2, 4, 4, 7 and 9; vaccinated animals infected at challenge 2 (three animals) and 4 (one animal), with three uninfected. A log-rank test on these readings gives χ² = 1.55, p = 0.21. If the first vaccinated step is read at challenge 1 instead, χ² = 1.07, p = 0.30. Neither reading comes close to 0.05. Across every possible outcome with 7 vs 6 animals, the large-sample log-rank χ² cannot exceed 14.14, so its two-sided p cannot go below 1.7 × 10⁻⁴. A one-sided version reaches below 1 × 10⁻⁴ only when all six controls are infected before any vaccinated animal (smallest value 8.5 × 10⁻⁵), which is far from the curves in Fig. 2a, where three vaccinated animals were infected when only two controls had been. An exact (permutation) version of the test cannot go below 1/1716 = 5.8 × 10⁻⁴. This explanation does not account for the reported number.
- An unconditional exact test (Barnard or Boschloo). These tests do not fix the table margins, and on 7 vs 6 animals they can reach lower values than Fisher's test, down to (7/13)⁷(6/13)⁶ = 1.27 × 10⁻⁴, which is still above 1 × 10⁻⁴. On the observed 3/7 vs 0/6 they give one-sided p = 0.106 (Barnard) and 0.054 (Boschloo). Neither is below 0.05. This explanation does not account for the number.
- Group size of 6 rather than 7. The Fig. 2a legend gives "tonsil vaccinated (n = 6)", while the Methods, Results and Abstract give 7. The post-infection panels and legends (Figs. 2b–e, 3–5, 7 and 8) also give "tonsil vaccinated (n = 6)", which matches the six vaccinated animals that eventually became infected. Before challenge, the ELISPOT legend (Fig. 6) gives n = 7, and the anti-MVA antibody panel (Fig. 1c) gives n = 6. Fig. 2a covers acquisition in all 7 vaccinated animals but carries the same n = 6 label as the post-infection panels. Even taken at face value, 3/6 vs 0/6 gives a one-tailed Fisher p of 0.091 (two-tailed 0.182). This is still not significant at 0.05 and is roughly 900 times the reported bound. This explanation does not account for the number.
- One-tailed vs two-tailed. Both were computed: 0.122 and 0.192. Neither is close to 0.0001. This explanation does not account for the number.
- Per-exposure analysis (Klasse et al.). The cited reference discusses estimating efficacy per exposure in repeated low-dose challenge studies. The paper, however, defines VE at the 9th challenge with a per-animal formula and reports 43%, which reproduces exactly from the per-animal counts. A per-exposure analysis would give a different VE than the one reported. The paper does not describe such an analysis or report its result, so it cannot be verified as the source of the p-value.
- Typographical slip, for example a misplaced decimal. No obvious single-character change links the recomputed 0.122 to "< 0.0001". The paper prints "P < 0.0001" as an inequality, which is the form statistical software uses for very small values. The source of the reported value cannot be determined from the paper.
- Conversion or rendering artifact. The sentence reads identically in the PMC archive, the Europe PMC full-text XML supplied by the publisher, and the PMC public page. This explanation is ruled out.
Status
- Version of record. The statement appears verbatim in the PubMed Central and Europe PMC archives of the published article. The DOI resolves to the publisher's article page, which redirected to an authentication page and could not be fetched directly. The archived full text is the publisher's version of record.
- Supplements. The Supplementary Information PDF (supplementary figures and captions) and the Reporting Summary are listed with the article. The Supplementary Information was read in full. It contains no statistical test for the acquisition comparison and no occurrence of "0.0001", "Fisher", "log-rank", "Mantel" or "Kaplan". No separate source-data file is linked on PMC. The supplement does not resolve the discrepancy.
- Figure 2a. The Kaplan–Meier panel was checked in the publisher's figure image. It prints "VE = 43%" and "P = <0.0001" and shows the step pattern used for the log-rank readings above.
- Corrections. No correction, erratum, retraction, expression of concern or editor's note was found in Crossref (no update-to or updated-by relations), PMC notices, or Europe PMC (no comment or correction links).
- PubPeer. PubPeer could not be queried directly (HTTP 403). A web search for the DOI and title on PubPeer and Retraction Watch found no thread or notice.
- Sources: PMC9990026; Europe PMC record PMID 36882405; api.crossref.org/works/10.1038/s41467-023-36907-0. Checked as of 2026-10-23.
Suggested correction
The authors should report the p-value for the 9th-challenge acquisition comparison that the stated method gives for the stated counts: a one-tailed Fisher's exact test of 3/7 vs 0/6 uninfected, p = 0.12 (two-tailed p = 0.19). They should state that this comparison is not significant at the paper's α of 0.05. If a different analysis produced the reported value, they should name that test, report the per-animal infection times it used, and confirm that the value is valid for 13 animals. Wording that rests on this comparison should be revised to match, specifically "significant level of protection from acquisition" in the Introduction and the claim that the vaccine "protects" in the title and Abstract. The group size in the Fig. 2a legend (n = 6 vaccinated) should be reconciled with n = 7 in the Methods, Results and Abstract. The post-infection panels use n = 6, which matches the six vaccinated animals eventually infected, but Fig. 2a covers acquisition in all 7.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC9990026 (Nat Commun 2023, doi:10.1038/s41467-023-36907-0)
# Run: python recompute.py.txt
# Prints every recomputed number in writeup.md and regenerates fig1/fig2 (.svg, .png)
# plus five candidate figure designs (A-E) in ./candidates/ for design review.
# candidates/ is review material only and is not part of the deliverable.
import os, itertools
from math import comb
import numpy as np
from scipy.stats import fisher_exact, hypergeom, barnard_exact, boschloo_exact, chi2, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------- numbers from the paper ----------------
N_VAC, N_CTL = 7, 6 # Methods: control n = 6, vaccinated n = 7
INF_VAC, INF_CTL = 4, 6 # Results: 4/7 vaccinated, 6/6 control infected by 9th challenge
UNINF_VAC, UNINF_CTL = N_VAC - INF_VAC, N_CTL - INF_CTL # 3 and 0
REPORTED_P = 1e-4 # Results and Fig. 2a panel: "P < 0.0001" (an upper bound)
ALPHA = 0.05 # Methods and Fig. 2a legend
# Approximate readings of the Fig. 2a Kaplan-Meier steps (challenge number at which
# each animal became infected; 10 = still uninfected after the 9th challenge).
# Controls: 100 -> 83 (ch 1) -> 67 (ch 2) -> 33 (ch 4, two animals) -> 17 (ch 7) -> 0 (ch 9).
# Vaccinated: 100 -> 57 (ch 2, three animals) -> 43 (ch 4); three animals uninfected.
KM_CTL = [1, 2, 4, 4, 7, 9]
KM_VAC = [2, 2, 2, 4, 10, 10, 10]
KM_VAC_ALT = [1, 1, 1, 4, 10, 10, 10] # alternative reading: first vaccinated step at challenge 1
# ---------------- recomputation ----------------
ve = 1 - (INF_VAC / N_VAC) / (INF_CTL / N_CTL)
print(f"VE = 1 - (4/7)/(6/6) = {ve:.4f} = {100*ve:.1f}% (paper: 43%)")
table = [[UNINF_VAC, INF_VAC], [UNINF_CTL, INF_CTL]] # rows vaccinated, control; cols uninfected, infected
_, p1 = fisher_exact(table, alternative="greater")
_, p2 = fisher_exact(table, alternative="two-sided")
p1_exact = comb(7, 3) / comb(13, 3)
print(f"2x2 table (uninfected, infected): vaccinated {table[0]}, control {table[1]}")
print(f"One-tailed Fisher exact p = C(7,3)/C(13,3) = {comb(7,3)}/{comb(13,3)} = {p1_exact:.4f} (scipy {p1:.4f})")
print(f"Two-tailed Fisher exact p = {p2:.4f} (= {round(p2*286)}/286)")
# Null distribution: number of the 3 uninfected animals that fall in the vaccinated group
null = {k: hypergeom.pmf(k, 13, 3, 7) for k in range(4)}
for k, v in null.items():
print(f" P(k={k} of 3 uninfected in vaccinated group) = {v:.4f} = {round(v*286)}/286")
# Fig. 2a legend gives n = 6 vaccinated: 3/6 vs 0/6
_, p1_6 = fisher_exact([[3, 3], [0, 6]], alternative="greater")
_, p2_6 = fisher_exact([[3, 3], [0, 6]], alternative="two-sided")
print(f"If n=6 vaccinated (3/6 vs 0/6): one-tailed = C(6,3)/C(12,3) = {comb(6,3)}/{comb(12,3)} = {p1_6:.4f}; two-tailed = {p2_6:.4f}")
# Floors of conditional (Fisher) / permutation tests: most extreme table [[7,0],[0,6]]
floor = 1 / comb(13, 7)
_, pf = fisher_exact([[7, 0], [0, 6]], alternative="greater")
_, pf2 = fisher_exact([[7, 0], [0, 6]], alternative="two-sided")
min2 = min(fisher_exact([[a, 7 - a], [b, 6 - b]], alternative="two-sided")[1]
for a in range(8) for b in range(7))
print(f"Smallest attainable one-tailed Fisher/permutation p, 7 vs 6 = 1/C(13,7) = 1/{comb(13,7)} = {floor:.3e} (scipy {pf:.3e})")
print(f"Two-tailed Fisher p for [[7,0],[0,6]] = {pf2:.3e} (opposite-tail table k=1 has probability "
f"{hypergeom.pmf(1, 13, 7, 7)*comb(13,7):.0f}/1716 > 1/1716, so not added); "
f"minimum two-tailed Fisher p over all 7-vs-6 tables = {min2:.3e}")
# Unconditional exact tests (Barnard, Boschloo)
bar_floor = barnard_exact([[7, 0], [0, 6]], alternative="greater").pvalue
bos_floor = boschloo_exact([[7, 0], [0, 6]], alternative="greater").pvalue
closed = (7 / 13) ** 7 * (6 / 13) ** 6
print(f"Barnard floor, one-sided, [[7,0],[0,6]] = {bar_floor:.3e}; Boschloo floor = {bos_floor:.3e}; "
f"(7/13)^7 (6/13)^6 = {closed:.3e}; above 1e-4? {closed > REPORTED_P}")
bar1 = barnard_exact(table, alternative="greater").pvalue
bar2 = barnard_exact(table, alternative="two-sided").pvalue
bos1 = boschloo_exact(table, alternative="greater").pvalue
bos2 = boschloo_exact(table, alternative="two-sided").pvalue
print(f"Observed 3/7 vs 0/6: Barnard one-sided {bar1:.4f} (two-sided {bar2:.4f}); "
f"Boschloo one-sided {bos1:.4f} (two-sided {bos2:.4f})")
# Log-rank (Mantel-Cox) test on the Fig. 2a readings
def logrank(tv, tc, n_times=9):
tv, tc = np.asarray(tv), np.asarray(tc)
oe = var = 0.0
for t in range(1, n_times + 1):
nv, nc = (tv >= t).sum(), (tc >= t).sum(); n = nv + nc
dv = (tv == t).sum(); d = dv + (tc == t).sum()
if d == 0 or n < 2:
continue
oe += dv - d * nv / n
var += d * (nv / n) * (nc / n) * (n - d) / (n - 1)
return oe ** 2 / var, oe / np.sqrt(var)
lr_c, lr_z = logrank(KM_VAC, KM_CTL)
lr_p = chi2.sf(lr_c, 1)
lr_c_alt, _ = logrank(KM_VAC_ALT, KM_CTL)
lr_p_alt = chi2.sf(lr_c_alt, 1)
print(f"Log-rank on Fig. 2a readings: chi2 = {lr_c:.2f}, p = {lr_p:.3f} (one-sided {norm.cdf(lr_z):.3f}); "
f"alternative reading (first vaccinated step at challenge 1): chi2 = {lr_c_alt:.2f}, p = {lr_p_alt:.3f}")
# Largest asymptotic log-rank chi2 over every possible 7-vs-6 outcome
# (each animal infected at challenge 1..9 or uninfected after 9).
def compositions(n, k):
rows = []
for c in itertools.combinations(range(n + k - 1), k - 1):
prev, row = -1, []
for x in c:
row.append(x - prev - 1); prev = x
row.append(n + k - 2 - prev); rows.append(row)
return np.array(rows)
SLOTS = 10
CV, CC = compositions(7, SLOTS), compositions(6, SLOTS)
riskV = np.cumsum(CV[:, ::-1], axis=1)[:, ::-1][:, :9].astype(float)
riskC = np.cumsum(CC[:, ::-1], axis=1)[:, ::-1][:, :9].astype(float)
Z1E4 = norm.isf(REPORTED_P) # one-sided normal z for p = 1e-4
first_vac = np.array([next((t for t in range(9) if r[t] > 0), 9) for r in CV]) # slot of first vaccinated infection
lr_max, n_below, n_below_other = 0.0, 0, 0
with np.errstate(divide="ignore", invalid="ignore"):
for i in range(len(CC)):
nv, nc = riskV, riskC[i][None, :]; n = nv + nc
dv = CV[:, :9]; d = dv + CC[i, :9][None, :]
e = np.where(n > 0, d * nv / n, 0.0)
v = np.where(n > 1, d * (nv / n) * (nc / n) * (n - d) / (n - 1), 0.0)
vv = v.sum(1)
oe = (dv - e).sum(1)
c2 = np.where(vv > 0, oe ** 2 / vv, 0.0)
lr_max = max(lr_max, c2.max())
z = np.where(vv > 0, oe / np.sqrt(vv), 0.0)
below = z < -Z1E4 # one-sided p < 1e-4 in the direction of protection
ctl_before = np.cumsum(CC[i])[np.maximum(first_vac - 1, 0)] * (first_vac > 0)
all_ctl_first = ctl_before == 6 # every control infected before any vaccinated animal
n_below += below.sum()
n_below_other += (below & ~all_ctl_first).sum()
lr_floor = chi2.sf(lr_max, 1)
print(f"Over all {len(CV)*len(CC):,} outcomes: max log-rank chi2 = {lr_max:.2f}, two-sided p = {lr_floor:.2e}, "
f"one-sided normal p = {norm.sf(np.sqrt(lr_max)):.2e}")
print(f" outcomes with one-sided log-rank p < 1e-4: {n_below:,}; of these, outcomes in which any vaccinated "
f"animal was infected before all 6 controls: {n_below_other}")
print(f"Reported bound 1e-4 below Fisher floor? {REPORTED_P < floor}; floor / 1e-4 = {floor/REPORTED_P:.1f}")
print(f"Recomputed one-tailed p / reported bound = {p1/REPORTED_P:.0f}x")
# Ladder: one-tailed p if k of 7 vaccinated stayed uninfected and 0/6 controls did
ladder = {}
for k in range(1, 8):
_, pk = fisher_exact([[k, 7 - k], [0, 6]], alternative="greater")
ladder[k] = pk
print(f" k={k}/7 vaccinated uninfected vs 0/6: one-tailed p = C(7,{k})/C(13,{k}) = {pk:.4g}")
k_needed = min(k for k, v in ladder.items() if v < ALPHA)
print(f"Fewest uninfected vaccinated animals (of 7, vs 0/6) giving one-tailed p < 0.05: {k_needed} (p = {ladder[k_needed]:.4f})")
# ---------------- style ----------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": "#b9b8b3", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def clean(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
# ---------- Candidate A: p-values on one log axis ----------
def design_A(path):
import matplotlib.transforms as mtrans
fig, ax = plt.subplots(figsize=(7.2, 5.0))
rows = [
("Reported (Results; Fig. 2a)", REPORTED_P, ORANGE, "P < 0.0001"),
("Fisher one-tailed, 3/7 vs 0/6\n(test named in Methods, legend)", p1, BLUE, f"{p1:.3f}"),
("Fisher two-tailed, 3/7 vs 0/6", p2, BLUE, f"{p2:.3f}"),
("Fisher one-tailed, 3/6 vs 0/6\n(n = 6 as in Fig. 2a legend)", p1_6, BLUE, f"{p1_6:.3f}"),
("Boschloo one-sided, 3/7 vs 0/6", bos1, BLUE, f"{bos1:.3f}"),
("Barnard one-sided, 3/7 vs 0/6", bar1, BLUE, f"{bar1:.3f}"),
("Log-rank on Fig. 2a curves\n(approximate readings)", lr_p, BLUE, f"{lr_p:.2f}"),
]
ys = list(range(len(rows)))[::-1]
ax.axvspan(3e-5, floor, color="#efeeea", zorder=0)
ax.axvline(bar_floor, color=SEC, lw=1.2, ls=":", zorder=1)
ax.axvline(ALPHA, color=SEC, lw=1, ls="--", zorder=1)
ax.text(ALPHA / 1.15, len(rows) - 0.45, "α = 0.05", ha="right", va="center", fontsize=10, color=SEC)
tr = mtrans.blended_transform_factory(ax.transAxes, ax.transData)
ax.text(1.03, len(rows) - 0.45, "p", transform=tr, fontsize=10, color=SEC, ha="left", va="center")
for (lab, v, c, ann), y in zip(rows, ys):
if c == ORANGE:
ax.annotate("", xy=(4e-5, y), xytext=(v, y),
arrowprops=dict(arrowstyle="-|>", color=c, lw=2, mutation_scale=14))
ax.plot(v, y, marker="|", ms=14, mew=2.5, color=c)
else:
ax.plot(v, y, "o", ms=9, color=c, zorder=3)
ax.text(1.03, y, ann, transform=tr, fontsize=10, color=c, ha="left", va="center",
weight="bold" if c == ORANGE else "normal")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
for t, r in zip(ax.get_yticklabels(), rows):
t.set_color(r[2] if r[2] == ORANGE else TXT)
ax.set_xscale("log")
ax.set_xlim(3e-5, 1)
ax.set_ylim(-0.6, len(rows) - 0.1)
ax.set_xlabel("p-value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
clean(ax)
ax.tick_params(axis="y", length=0)
fig.tight_layout(rect=(0, 0.13, 0.93, 0.94))
fig.suptitle("9th-challenge acquisition p-value: reported (orange) vs recomputed (blue)",
x=0.015, ha="left", fontsize=11, color=TXT)
fig.text(0.015, 0.075, f"Shaded band: below 1/1716 = {floor:.1e}, the smallest p a Fisher or permutation test can give.",
fontsize=9.5, color=SEC)
fig.text(0.015, 0.03, f"Dotted line: {bar_floor:.1e}, the smallest p a Barnard or Boschloo test can give (7 vs 6 animals).",
fontsize=9.5, color=SEC)
save(fig, path)
# ---------- Candidate B: null distribution of the Fisher test ----------
def design_B(path):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ks = list(null.keys())
vals = [null[k] for k in ks]
cols = [BLUE if k == 3 else "#c9c8c3" for k in ks]
ax.bar(ks, vals, color=cols, width=0.6, zorder=2)
for k, v in zip(ks, vals):
ax.text(k, v + 0.012, f"{round(v*286)}/286\n= {v:.3f}", ha="center", fontsize=10,
color=BLUE if k == 3 else SEC)
ax.annotate(f"Observed (3/7 vs 0/6)\none-tailed p = {p1:.3f}", xy=(3, null[3] + 0.115),
xytext=(2.55, 0.31), fontsize=10, color=BLUE,
arrowprops=dict(arrowstyle="-", color=BLUE, lw=1))
# The bars are point probabilities, not p-values, so the reported p-value is not drawn on this
# axis. It is stated as text: the one-tailed p is the sum of the bars at least as extreme as
# the observed result, which here is the 3-of-3 bar alone.
ax.text(-0.42, 0.70, "Reported: P < 0.0001", fontsize=10, color=ORANGE, va="top", weight="bold")
ax.text(-0.42, 0.655, "The one-tailed p is the sum of bars at least as\nextreme as observed: here the 3-of-3 bar alone,\n"
f"35/286 = {p1:.3f}, about 1,200 times the reported bound.",
fontsize=9.5, color=SEC, va="top")
ax.set_xticks(ks)
ax.set_xticklabels([f"{k} of 3" for k in ks])
ax.set_xlabel("Uninfected animals (3 in total) that fall in the vaccinated group of 7")
ax.set_ylabel("Probability if vaccine had no effect")
ax.set_ylim(0, 0.75)
ax.set_yticks([0, 0.1, 0.2, 0.3, 0.4, 0.5])
ax.grid(axis="y", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title("Fisher's exact test on the 9th-challenge counts", fontsize=11, loc="left")
fig.tight_layout()
save(fig, path)
# ---------- helpers for text diagrams ----------
def box(ax, x, y, w, h, title, body, edge, tcolor, size=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.02",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.02, y + h - 0.05, title, fontsize=10, color=SEC, va="top", weight="bold")
ax.text(x + 0.02, y + h - 0.17, body, fontsize=size, color=tcolor, va="top", linespacing=1.45)
# ---------- Candidate C: text panels, test named vs value reported ----------
def design_C(path):
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.subplots_adjust(0.01, 0.01, 0.99, 0.99)
ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
box(ax, 0.01, 0.56, 0.46, 0.42, "Methods and Fig. 2a legend",
"VE after the 9th challenge\n\"followed by Fisher\u2019s exact test\"\n\"one-tailed Fishers Exact Test\"",
"#b9b8b3", TXT, 10)
box(ax, 0.53, 0.56, 0.46, 0.42, "Results",
"4/7 vaccinated vs 6/6 control\ninfected by the 9th challenge\n\"VE = 43% (Fig. 2a; P < 0.0001)\"",
ORANGE, ORANGE, 10)
box(ax, 0.12, 0.02, 0.76, 0.34, "Recomputed: the stated test on the stated counts",
f"One-tailed Fisher, 3/7 vs 0/6 uninfected:\np = C(7,3)/C(13,3) = 35/286 = {p1:.3f} (two-tailed {p2:.3f})",
BLUE, BLUE, 10)
for x0, x1 in ((0.24, 0.36), (0.76, 0.64)):
ax.add_patch(FancyArrowPatch((x0, 0.555), (x1, 0.365), arrowstyle="-|>",
mutation_scale=14, color=SEC, lw=1.5))
ax.text(0.5, 0.46, "Orange (reported) vs blue (recomputed):\nmore than 1,000-fold apart",
ha="center", va="center", fontsize=9.5, color=SEC)
save(fig, path)
# ---------- Candidate D: calculation flow ----------
def design_D(path):
fig, ax = plt.subplots(figsize=(7.2, 2.7))
fig.subplots_adjust(0.01, 0.01, 0.99, 0.99)
ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
steps = [
("Counts", "uninfected after\n9 challenges\n(Results):\nvaccinated 3/7\ncontrol 0/6", TXT, "#b9b8b3"),
("Stated test", "one-tailed\nFisher exact\n(Methods and\nFig. 2a legend)", TXT, "#b9b8b3"),
("Recomputed", f"C(7,3)/C(13,3)\n= 35/286\n= {p1:.3f}", BLUE, BLUE),
("Reported", "P < 0.0001", ORANGE, ORANGE),
]
w = 0.215; gap = (0.98 - 4 * w) / 3
for i, (t, b, c, e) in enumerate(steps):
x = 0.01 + i * (w + gap)
box(ax, x, 0.24, w, 0.72, t, b, e, c, 10)
if i < 2:
ax.add_patch(FancyArrowPatch((x + w, 0.62), (x + w + gap, 0.62), arrowstyle="-|>",
mutation_scale=12, color=SEC, lw=1.5))
xn = 0.01 + 3 * w + 2.5 * gap
ax.text(xn, 0.61, "\u2260", ha="center", va="center", fontsize=20, color=ORANGE)
ax.text(0.5, 0.08, f"Recomputed {p1:.3f} vs reported < 0.0001: more than 1,000-fold apart", ha="center",
fontsize=10, color=SEC)
save(fig, path)
# ---------- Candidate E: ladder of attainable p-values ----------
def design_E(path):
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ks = list(ladder.keys())
ax.axhspan(3e-5, floor, color="#efeeea", zorder=0)
ax.axhline(ALPHA, color=SEC, lw=1, ls="--")
ax.text(7.85, ALPHA * 1.25, "α = 0.05", ha="right", va="bottom", fontsize=10, color=SEC)
ax.plot(ks, [ladder[k] for k in ks], "-", color="#c9c8c3", lw=2, zorder=1)
ax.plot(ks, [ladder[k] for k in ks], "o", ms=8, color="#9a9994", zorder=2,
label="Possible outcomes: k of 7 vaccinated uninfected")
ax.plot(3, p1, "o", ms=11, color=BLUE, zorder=3, label=f"Observed 3/7, recomputed p = {p1:.3f}")
ax.axhline(REPORTED_P, color=ORANGE, lw=2, label="Reported: P < 0.0001")
ax.text(0.8, 2.2e-4, f"Shaded: below the smallest p a Fisher or permutation\ntest can give (all 7 uninfected: 1/1716 = {floor:.1e})",
fontsize=9.5, color=SEC, va="center")
ax.set_yscale("log")
ax.set_ylim(3e-5, 1)
ax.set_xlim(0.7, 7.9)
ax.set_xticks(ks)
ax.set_xticklabels([f"{k}/7" for k in ks])
ax.set_xlabel("Vaccinated animals uninfected after 9 challenges (controls 0/6)")
ax.set_ylabel("One-tailed Fisher p (log scale)")
ax.grid(axis="y", color=GRID, lw=0.8)
for s_ in ("top", "right"):
ax.spines[s_].set_visible(False)
ax.legend(loc="upper right", frameon=False, fontsize=9.5)
ax.set_title("Every possible outcome of a 7-vs-6 study gives p above 0.0001", fontsize=11, loc="left")
fig.tight_layout()
save(fig, path)
design_A(os.path.join(CAND, "A_pvalue_axis"))
design_B(os.path.join(CAND, "B_null_distribution"))
design_C(os.path.join(CAND, "C_text_panels"))
design_D(os.path.join(CAND, "D_calculation_flow"))
design_E(os.path.join(CAND, "E_attainable_ladder"))
# Provisional choice, pending the user's review of candidates A-E: fig1 = A, fig2 = E.
design_A(os.path.join(HERE, "fig1"))
design_E(os.path.join(HERE, "fig2"))
print("Figures written to", HERE)
Abstract's verified-vaccination percentages and effect do not match its own counts or the Results
Nat Med (2023) · doi:10.1038/s41591-023-02670-4 · PMC10719106
Summary
The abstract of this cluster-randomized trial in rural Ghana reports that 36.6% (355 of 1,058) of verified participants in the cash-incentive arms had at least one COVID-19 vaccine dose, compared with 30.3% (439 of 1,544) in the placebo arm, a difference of 6.3% (95% CI 2.4, 10.2; P = 0.001). The counts printed in that sentence give 33.6% and 28.4%. Those are the rates the Results text and Table 2 report for the same verified sample, where the difference is 5.2% (95% CI 1.5, 8.7; P = 0.005). The abstract's 36.6%, 6.3% and 2.4–10.2 match, figure for figure, a spillover estimate that the Results give for a different subgroup. Verified vaccination is the trial's only registry-confirmed outcome, so the abstract overstates the size and precision of its headline objective result, though the effect's direction and statistical significance are unchanged.
What the paper reports
Abstract:
We also verified vaccination status of participants: in the cash treatment arm, 36.6% (355 of 1,058) of verified participants had at least one dose of the COVID-19 vaccine compared to 30.3% (439 of 1,544) for those in the placebo—a difference of 6.3% (95% CI: 2.4, 10.2; P = 0.001).
Results, "Combined low and high cash treatment effect", third paragraph:
In the cash treatment arm (both low and high cash), we verified that 33.6% of individuals had at least one dose of the COVID-19 vaccine as of April 2023 compared to 28.4% of those in the placebo arm (difference from placebo: 5.2%; 95% CI: 1.5, 8.7; P = 0.005).
In relative terms, this represents about an 18% increase over the non-treated placebo vaccination rates.
Results, "Verified vaccination status":
For our six-district sample, we verified that 29.1% of the 3,075 participants received a COVID-19 vaccination
We verified that 28.4% of individuals in the placebo arm were vaccinated (at least one dose)
Table 2, phase IV panel ("Phase IV: verified vaccination status of treated participants (30 April 2022)"; columns: total sample | placebo | health message | low cash | high cash):
Number of participants | 3075 | 1544 | 473 | 548 | 510
Actual vaccination (%) | 29.1 (45.4) | 28.4 (45.1) | 21.6 (41.2) | 40.7 (49.2) | 25.9 (43.8)
Results, "Spillover", second paragraph, which describes Extended Data Table 5 (participants who saw the placebo video, grouped by the treatment assigned to their village):
Participants in placebo-treated village clusters had an average vaccination rate of 26.8%.
Individuals receiving the placebo video in low cash treatment villages had a verified vaccination rate of 38.3% (difference from placebo: 11.5%; 95% CI: 5.2, 17.8; P < 0.001).
Placebo participants in the combined cash treatment also had a verified vaccination rate, 36.6%, that is significantly larger than the placebo treatment village participants (difference from placebo: 6.3%; 95% CI: 2.4, 10.2; P = 0.002).
What the paper's own numbers show
Method. I divided the abstract's numerators by its denominators. I then rebuilt each arm's vaccinated count from the phase IV panel of Table 2 (n × printed %) and checked that the cash-arm count and size equal the abstract's 355 and 1,058. I computed the cash-minus-placebo difference with a naive two-proportion Wald 95% interval and normal-approximation P value. This interval ignores village clustering and serves only to check the printed arithmetic. Finally I set the abstract's figures beside the spillover paragraph's figures. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Cash-arm size | Abstract; Table 2 phase IV | 1,058; low cash 548, high cash 510 | 548 + 510 = 1,058, the same sample |
| Cash-arm vaccinated | Abstract; Table 2 phase IV | 355; 40.7% of 548, 25.9% of 510 | 223.0 + 132.1 → 223 + 132 = 355, the same count |
| Placebo size and vaccinated | Abstract; Table 2 phase IV | 439 of 1,544; 28.4% of 1,544 | 1,544 × 28.4% = 438.5 (438 or 439 after rounding), consistent |
| Total verified sample | Table 2 phase IV | 3,075; 29.1% | 895/3,075 = 29.1% from the rebuilt counts, consistent |
| Cash-arm rate | Abstract | 36.6% (355 of 1,058) | 355/1,058 = 33.55% → 33.6% |
| Cash-arm rate | Results, combined cash effect | 33.6% | 33.6%, agrees |
| Placebo rate | Abstract | 30.3% (439 of 1,544) | 439/1,544 = 28.43% → 28.4% |
| Placebo rate | Results; Table 2 | 28.4% | 28.4%, agrees |
| Counts the abstract's percentages would need | Abstract | 36.6% of 1,058; 30.3% of 1,544 | 387 (not 355); 468 (not 439) |
| Cash minus placebo | Abstract | 6.3% (95% CI 2.4, 10.2; P = 0.001) | 5.12 pp, naive 95% CI (1.5, 8.7), P = 0.0057 |
| Cash minus placebo | Results, combined cash effect | 5.2% (95% CI 1.5, 8.7; P = 0.005) | 5.12 pp (33.6 − 28.4 = 5.2 from the rounded rates); same interval |
| Relative increase | Results | "about 18%" | 5.12/28.43 = 18.0%; the abstract's figures imply 6.3/30.3 = 20.8% |
| Placebo-video participants in cash villages | Results, Spillover | 36.6%; difference 6.3 (95% CI 2.4, 10.2; P = 0.002) | Identical to the abstract's 36.6%, 6.3 and 2.4–10.2; the P values differ (0.002 vs 0.001) |
| Spillover reference rate | Results, Spillover | placebo villages 26.8% | 38.3 − 11.5 = 26.8 agrees for the low-cash line; 36.6 − 6.3 = 30.3, not 26.8 (36.6 − 26.8 = 9.8) |
The abstract's counts and denominators agree with every other part of the paper. Its percentages, difference, interval and P value do not agree with those counts. The abstract's placebo rate, 30.3%, is exactly 36.6 − 6.3, the spillover paragraph's rate minus its difference.
Figure
Figure 1. The left panel gives the abstract's verified-vaccination figures and the right panel the Results text and Table 2 (phase IV) figures, both as printed (orange). Both panels describe the same 1,058 cash-arm and 1,544 placebo participants. The blue strip shows the rates and difference recomputed from the abstract's own counts, 355/1,058 and 439/1,544. These agree with the Results and Table 2, not with the abstract. The note under the strip records where the abstract's 36.6%, 6.3% and 2.4–10.2 appear in the paper: in the spillover paragraph, for a different subgroup.
Figure 2. Cash-minus-placebo difference in verified vaccination (percentage points, 95% CI) on one axis, with zero as a thin reference line. Orange marks the values printed in the abstract and in the Results. Blue marks the value recomputed from the abstract's counts with a naive, unclustered two-proportion interval. Gray marks the spillover-subgroup estimate printed in the Results. The recomputed value and interval match the Results. The abstract's estimate and interval are the same as the spillover subgroup's.
Why it matters
- This is the headline objective outcome. Intentions and reported vaccinations are self-reports. Verified vaccination comes from District Health Office records and is the only registry-confirmed measure. The abstract presents it as the confirming result.
- Size of the effect. The abstract gives 6.3 percentage points. The paper's own counts give 5.1 (5.2 in the Results), so the abstract overstates the absolute effect by about 1.1 pp and implies a relative increase of about 21% rather than the 18% the Results report. It also gives a placebo baseline of 30.3% instead of 28.4%.
- Precision and significance. The abstract gives a lower confidence bound of 2.4 pp and P = 0.001. The analysis of the same sample gives 1.5 pp and P = 0.005. The effect remains positive and its interval still excludes zero, so the qualitative conclusion that cash incentives increased verified uptake does not change.
- Comparison with self-report. The Results give a 3.7-point cash effect on reported vaccination (the abstract gives 3.5). Against that, the abstract's verified effect looks 1.70 times as large (6.3/3.7). With the Results figure the ratio is 1.41 (5.2/3.7).
- Downstream use. Abstracts are what evidence summaries, policy briefs and meta-analyses usually extract. A reader who takes the abstract's percentages together with its counts gets an inconsistent 2×2 table.
A side note for a statistician: the Results interval (1.5, 8.7) is the same as a naive interval that ignores clustering, although the Results report an intracluster correlation of 0.64 for verified vaccination. This write-up compares like with like and makes no claim about which interval is appropriate. The inconsistency described here does not depend on that question.
Explanations considered
- A different population or subsample. Ruled out for the counts. 1,058 = 548 + 510, the two cash arms in Table 2 phase IV; 1,544 is the placebo arm there; and 223 + 132 = 355 rebuilds the cash count from Table 2's percentages. The numerators and denominators therefore belong to the main verified treated sample. No subsample is described that has those sizes and the abstract's percentages.
- Adjusted rather than crude estimates. Unlikely. The abstract gives the percentages with raw counts in parentheses ("36.6% (355 of 1,058)"), the form of a crude proportion. No adjustment turns 355/1,058 into 36.6%. The Results report the crude comparison for this sample as 33.6% vs 28.4%, and the model-based results (Table 3) are reported as odds ratios, not as percentages.
- A different time point or outcome definition. Not supported. Table 2 dates phase IV to 30 April 2022, and the Results "Verified vaccination status" section uses the end of April 2022. The combined-cash sentence says "as of April 2023", which does not match the 30 April 2022 cut-off in Table 2. Its figures, however, are the Table 2 figures. No other cut-off is given for the abstract.
- Rounding. Ruled out. 355/1,058 = 33.55% and 439/1,544 = 28.43%. Reaching 36.6% would take 387 vaccinated cash participants, and reaching 30.3% would take 468 vaccinated placebo participants.
- The abstract quoting the spillover analysis. This matches the abstract's percentage, difference and interval exactly (36.6%, 6.3, 2.4–10.2), but the P value differs (0.001 vs 0.002), and the spillover analysis concerns placebo-video participants in cash villages, not the cash arm. The spillover paragraph is itself not internally consistent: 36.6 − 6.3 = 30.3, not its stated reference rate of 26.8%. So the paper does not settle where the abstract's figures came from. Whatever their source, they do not describe the 355/1,058 vs 439/1,544 comparison they are attached to.
- A conversion or archiving artifact. Ruled out. The publisher's HTML version of record, the PMC web page, the Europe PMC full-text XML and the PMC archive all carry the same numbers.
- Already corrected. No. The only correction notice, an Author Correction (doi:10.1038/s41591-023-02772-z, 22 December 2023), replaces a wrong link to the protocol in the Methods and does not touch these figures.
Status
As of 2026-10-23:
- Version of record. The publisher's page (nature.com/articles/s41591-023-02670-4, marked "This article has been updated") has the same abstract, Results and spillover text quoted above. The PMC and Europe PMC full texts agree with it word for word.
- Corrections and notices. Crossref and Europe PMC list one Author Correction (doi:10.1038/s41591-023-02772-z; PMC10957464), dated 22 December 2023. Its full text concerns only the protocol link. No retraction or expression of concern is listed.
- Supplementary material. The Supplementary Information (98 pages: Supplementary Tables S1–S5 and questionnaires) gives no verified-vaccination figures of 30.3%, 36.6%, 33.6%, 1,544 or 1,058. Extended Data Table 5 is an image in the archived text and was not read from the image. Its contents are taken here from the Results paragraph that describes it.
- Replication data. A separate check tabulated the replication file the paper deposits on Harvard Dataverse (doi:10.7910/DVN/X5MBHP; the 3,075 rows with a non-missing verified-vaccination status, grouped by individual treatment). It found placebo 439/1,544 = 28.43%, low cash 223/548 = 40.69%, high cash 132/510 = 25.88% and cash combined 355/1,058 = 33.55%, which agrees with the Results and Table 2. No grouping tried gave 30.3% with n = 1,544 or 36.6% with n = 1,058. In the same file, placebo-video participants in cash villages were 196/599 = 32.7% vaccinated and those in placebo villages 184/686 = 26.8%. The first of these differs from the 36.6% in the spillover paragraph.
- PubPeer. The PubPeer search API and site returned a human-verification challenge to automated requests, so a direct search could not be completed. A web search for the article title on PubPeer returned no PubPeer thread.
- Sources. The article as archived in PMC (PMC10719106); the publisher's HTML; Crossref and Europe PMC notice records; the Author Correction (PMC10957464); the Supplementary Information; Harvard Dataverse doi:10.7910/DVN/X5MBHP.
Suggested correction
In the abstract, change the verified-vaccination sentence so that it matches its counts and the Results, for example: "in the cash treatment arm, 33.6% (355 of 1,058) of verified participants had at least one dose of the COVID-19 vaccine compared to 28.4% (439 of 1,544) for those in the placebo—a difference of 5.2% (95% CI: 1.5, 8.7; P = 0.005)." The authors may also wish to:
- Check the spillover paragraph's combined-cash figures (36.6%; difference 6.3%; 95% CI 2.4, 10.2; P = 0.002) against Extended Data Table 5 and the replication data, since 36.6 − 6.3 does not equal the stated reference rate of 26.8%.
- Change "as of April 2023" in the combined-cash sentence to the 30 April 2022 cut-off used in Table 2, if that is the intended date.
Recomputation script (python recompute.py.txt)
"""Recompute the verified-vaccination figures of PMC10719106
(Nat Med 2023, doi:10.1038/s41591-023-02670-4).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PubMed Central:
Abstract: verified-vaccination sentence (percentages, counts, difference, CI, P)
Table 2, phase IV (verified vaccination status of treated participants, 30 April 2022):
number of participants and "Actual vaccination (%)" by arm
Results > Combined low and high cash treatment effect (33.6% vs 28.4%, 5.2, CI 1.5-8.7)
Results > Spillover, paragraph on Extended Data Table 5 (26.8%, 38.3%/11.5, 36.6%/6.3)
The naive confidence interval below is an unadjusted two-proportion Wald interval; it
ignores village clustering and is used only to check the arithmetic of the printed values.
"""
import math
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
from scipy.stats import norm # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
# Abstract
ABS = dict(cash_pct=36.6, cash_k=355, cash_n=1058, plac_pct=30.3, plac_k=439, plac_n=1544,
diff=6.3, lo=2.4, hi=10.2, p="0.001")
# Results > Combined low and high cash treatment effect
RES = dict(cash_pct=33.6, plac_pct=28.4, diff=5.2, lo=1.5, hi=8.7, p="0.005", rel="about 18%")
# Table 2, phase IV
T2_N = dict(placebo=1544, health=473, low=548, high=510)
T2_PCT = dict(placebo=28.4, health=21.6, low=40.7, high=25.9)
T2_TOTAL_N, T2_TOTAL_PCT = 3075, 29.1
# Results > Spillover (Extended Data Table 5, panel B, as described in the text)
SPILL = dict(placebo_village=26.8, lowcash_placebo=38.3, lowcash_diff=11.5,
highcash_placebo=26.3, highcash_diff=0.5,
cash_placebo=36.6, cash_diff=6.3, lo=2.4, hi=10.2, p="0.002")
# Results > Combined cash effect on reported (self-reported) vaccination
SELF_DIFF = 3.7
def pct(k, n):
return 100.0 * k / n
def wald(k1, n1, k0, n0):
p1, p0 = k1 / n1, k0 / n0
d = p1 - p0
se = math.sqrt(p1 * (1 - p1) / n1 + p0 * (1 - p0) / n0)
z = norm.ppf(0.975)
return 100 * d, 100 * (d - z * se), 100 * (d + z * se), 2 * norm.sf(abs(d) / se)
def main():
print("== 1. Table 2 phase IV: rebuild the vaccinated counts from n and %")
k = {a: round(T2_N[a] * T2_PCT[a] / 100) for a in T2_N}
for a in T2_N:
print(f" {a:8s} n={T2_N[a]:5d} {T2_PCT[a]:.1f}% -> {T2_N[a] * T2_PCT[a] / 100:.1f} "
f"-> {k[a]} vaccinated")
tot_n, tot_k = sum(T2_N.values()), sum(k.values())
print(f" total n={tot_n} (Table 2: {T2_TOTAL_N}); vaccinated {tot_k} -> "
f"{pct(tot_k, tot_n):.1f}% (Table 2: {T2_TOTAL_PCT}%)")
cash_n = T2_N["low"] + T2_N["high"]
cash_k = k["low"] + k["high"]
print(f" cash arms combined: n = {T2_N['low']} + {T2_N['high']} = {cash_n} "
f"(abstract: {ABS['cash_n']}); vaccinated = {k['low']} + {k['high']} = {cash_k} "
f"(abstract: {ABS['cash_k']})")
print(f" placebo: n = {T2_N['placebo']} (abstract: {ABS['plac_n']}); "
f"vaccinated rebuilt = {k['placebo']} (abstract: {ABS['plac_k']}; "
f"{T2_N['placebo']}*28.35% = {T2_N['placebo'] * .2835:.1f} to "
f"{T2_N['placebo']}*28.45% = {T2_N['placebo'] * .2845:.1f}, so 438 or 439)")
print("\n== 2. Abstract: percentages implied by the abstract's own counts")
c = pct(ABS["cash_k"], ABS["cash_n"])
p = pct(ABS["plac_k"], ABS["plac_n"])
print(f" cash {ABS['cash_k']}/{ABS['cash_n']} = {c:.2f}% -> {c:.1f}% "
f"(abstract prints {ABS['cash_pct']}%; Results prints {RES['cash_pct']}%)")
print(f" placebo {ABS['plac_k']}/{ABS['plac_n']} = {p:.2f}% -> {p:.1f}% "
f"(abstract prints {ABS['plac_pct']}%; Results/Table 2 print {RES['plac_pct']}%)")
print(f" counts the printed percentages would need: {ABS['cash_pct']}% of {ABS['cash_n']} = "
f"{ABS['cash_pct'] / 100 * ABS['cash_n']:.1f}; {ABS['plac_pct']}% of {ABS['plac_n']} = "
f"{ABS['plac_pct'] / 100 * ABS['plac_n']:.1f}")
# interval of counts compatible with each printed percentage after rounding to 0.1
for lab, pc, n in (("cash", ABS["cash_pct"], ABS["cash_n"]),
("placebo", ABS["plac_pct"], ABS["plac_n"])):
lo_k = math.ceil((pc - 0.05) / 100 * n)
hi_k = math.floor((pc + 0.05) / 100 * n)
print(f" {lab}: counts that round to {pc}% of {n}: {lo_k} to {hi_k}")
print("\n== 3. Difference, naive two-proportion CI and P from the counts")
d, lo, hi, pv = wald(ABS["cash_k"], ABS["cash_n"], ABS["plac_k"], ABS["plac_n"])
print(f" difference = {d:.2f} pp, naive 95% CI ({lo:.1f}, {hi:.1f}), P = {pv:.4f}")
print(f" abstract: {ABS['diff']} ({ABS['lo']}, {ABS['hi']}), P = {ABS['p']}")
print(f" Results: {RES['diff']} ({RES['lo']}, {RES['hi']}), P = {RES['p']}")
print(f" difference of the Results' rounded rates: {RES['cash_pct']} - {RES['plac_pct']} = "
f"{RES['cash_pct'] - RES['plac_pct']:.1f}")
print(f" difference of the abstract's rates: {ABS['cash_pct']} - {ABS['plac_pct']} = "
f"{ABS['cash_pct'] - ABS['plac_pct']:.1f}")
print(f" overstatement in the abstract: {ABS['diff']} - {RES['diff']} = "
f"{ABS['diff'] - RES['diff']:.1f} pp (vs recomputed: {ABS['diff'] - d:.2f} pp)")
print("\n== 4. Relative effect")
print(f" from the counts: {d:.2f}/{p:.2f} = {100 * d / p:.1f}% "
f"(Results: {RES['rel']}; {RES['diff']}/{RES['plac_pct']} = "
f"{100 * RES['diff'] / RES['plac_pct']:.1f}%)")
print(f" implied by the abstract: {ABS['diff']}/{ABS['plac_pct']} = "
f"{100 * ABS['diff'] / ABS['plac_pct']:.1f}%")
print(f" ratio of verified to self-reported effect: abstract {ABS['diff']}/{SELF_DIFF} = "
f"{ABS['diff'] / SELF_DIFF:.2f}; Results {RES['diff']}/{SELF_DIFF} = "
f"{RES['diff'] / SELF_DIFF:.2f}")
print("\n== 5. Spillover paragraph (Extended Data Table 5 as described in Results)")
print(f" abstract figures vs spillover figures: 36.6/6.3/(2.4, 10.2) vs "
f"{SPILL['cash_placebo']}/{SPILL['cash_diff']}/({SPILL['lo']}, {SPILL['hi']}); "
f"P {ABS['p']} vs {SPILL['p']}")
print(f" low-cash placebo check: {SPILL['lowcash_placebo']} - {SPILL['lowcash_diff']} = "
f"{SPILL['lowcash_placebo'] - SPILL['lowcash_diff']:.1f} "
f"(stated placebo-village rate {SPILL['placebo_village']})")
print(f" combined-cash placebo check: {SPILL['cash_placebo']} - {SPILL['cash_diff']} = "
f"{SPILL['cash_placebo'] - SPILL['cash_diff']:.1f} "
f"(stated placebo-village rate {SPILL['placebo_village']}; "
f"{SPILL['cash_placebo']} - {SPILL['placebo_village']} = "
f"{SPILL['cash_placebo'] - SPILL['placebo_village']:.1f})")
print(f" abstract placebo rate {ABS['plac_pct']} equals "
f"{SPILL['cash_placebo']} - {SPILL['cash_diff']} = "
f"{SPILL['cash_placebo'] - SPILL['cash_diff']:.1f}")
print(f" combined-cash placebo rate must lie between the two cash-village placebo rates: "
f"{SPILL['highcash_placebo']} <= {SPILL['cash_placebo']} <= {SPILL['lowcash_placebo']}")
make_fig1(c, p, d, lo, hi)
make_fig2(d, lo, hi)
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT,
"ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "font.family": "DejaVu Sans",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def panel(ax, x, y, w, h, title, lines, edge, wrap=36):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.015, y + h - 0.025, title, ha="left", va="top", fontsize=11, weight="bold",
color=TXT)
yy = y + h - 0.10
for txt, col in lines:
for ln in textwrap.wrap(txt, wrap):
ax.text(x + 0.015, yy, ln, ha="left", va="top", fontsize=10, color=col)
yy -= 0.055
yy -= 0.02
def make_fig1(c, p, d, lo, hi):
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
# Left: abstract (reported, orange)
panel(ax, 0.015, 0.47, 0.42, 0.50, "Abstract", [
("Cash: 36.6% (355 of 1,058)", ORANGE),
("Placebo: 30.3% (439 of 1,544)", ORANGE),
("Difference 6.3% (95% CI 2.4, 10.2; P = 0.001)", ORANGE),
], ORANGE)
# Right: Results / Table 2 (same sample)
panel(ax, 0.565, 0.47, 0.42, 0.50, "Results text and Table 2", [
("Table 2, phase IV n: placebo 1,544; cash 548 + 510 = 1,058", SEC),
("Cash 33.6% vs placebo 28.4%", ORANGE),
("Difference 5.2% (95% CI 1.5, 8.7; P = 0.005)", ORANGE),
], SEC)
# Connector between the two panels
ax.annotate("", xy=(0.565, 0.70), xytext=(0.437, 0.70),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.5))
ax.text(0.501, 0.73, "same n,\nsame\noutcome", ha="center", va="bottom", fontsize=10,
color=SEC)
ax.text(0.501, 0.67, "different\n% and\nestimate", ha="center", va="top", fontsize=10,
color=TXT, weight="bold")
# Recomputed strip (blue)
ax.add_patch(FancyBboxPatch((0.015, 0.20), 0.97, 0.20,
boxstyle="round,pad=0.008,rounding_size=0.015",
fc="white", ec=BLUE, lw=1.5))
ax.text(0.03, 0.375, "Recomputed from the abstract's own counts", ha="left", va="top",
fontsize=11, weight="bold")
ax.text(0.03, 0.305, f"355 / 1,058 = {c:.1f}% 439 / 1,544 = {p:.1f}%", ha="left",
va="top", fontsize=10.5, color=BLUE)
ax.text(0.03, 0.25, f"Difference {d:.1f} pp (naive unclustered 95% CI {lo:.1f}, {hi:.1f})",
ha="left", va="top", fontsize=10.5, color=BLUE)
note = ("The abstract's 36.6%, 6.3% and CI 2.4, 10.2 also appear in Results > Spillover, "
"for placebo-video participants in cash villages (a different subgroup; P = 0.002 "
"there). Orange: values as printed in the paper. Blue: recomputed.")
for i, ln in enumerate(textwrap.wrap(note, 92)):
ax.text(0.015, 0.15 - 0.05 * i, ln, ha="left", va="top", fontsize=10, color=SEC)
save(fig, "fig1")
def make_fig2(d, lo, hi):
rows = [
("Abstract", ABS["diff"], ABS["lo"], ABS["hi"], ORANGE, "o"),
("Results text", RES["diff"], RES["lo"], RES["hi"], ORANGE, "s"),
("Recomputed, 355/1,058 vs 439/1,544", d, lo, hi, BLUE, "D"),
("Spillover subgroup (Results)", SPILL["cash_diff"], SPILL["lo"],
SPILL["hi"], SEC, "^"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.4))
for i, (lab, m, l, h, col, mk) in enumerate(rows):
y = len(rows) - 1 - i
ax.plot([l, h], [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot([m], [y], marker=mk, color=col, ms=9, ls="none")
ax.text(h + 0.25, y, f"{m:.1f} ({l:.1f}, {h:.1f})", va="center", fontsize=10, color=col)
ax.axvline(0, color=SEC, lw=0.8)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows][::-1])
ax.set_xlim(-0.5, 14)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("Cash minus placebo, verified vaccination (pp, 95% CI)")
ax.set_title("Orange: as printed in the paper. Blue: recomputed. Gray: a different subgroup.",
loc="right", fontsize=10, color=SEC)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
if __name__ == "__main__":
main()
Abstract says VEGFR1 is downregulated in hyperlipidemic mice; the Results and Discussion report an increase
Nat Med (2012) · doi:10.1038/nm.2759 · PMC3959651
Summary
The Abstract states that "we observed VEGFR1 downregulation in hyperlipidemic mice", but the paper's only mouse experiments (Apoe-null and Ldlr-null versus wild-type C57BL/6 mice, Fig. 3c–e) are reported in the Results as showing "increased endothelial-specific Vegfr1 expression", and the Discussion says Ldlr-null endothelial cells "express elevated amounts of Vegfr1". Of the eight main-text statements that give VEGFR1's direction when ApoB-lipoprotein levels are raised or lowered, seven have VEGFR1 moving with the lipoproteins, as the paper's model requires; the Abstract's mouse clause is the only exception. The data and the main conclusion are unaffected, but the Abstract gives the paper's only mammalian in vivo result the wrong sign, which, read alone, implies that hyperlipidemia lowers an anti-angiogenic decoy receptor. The same wording is in the version of record and the PubMed abstract, and no correction has been issued.
What the paper reports
The Abstract sentence:
Mechanistically, we identified downregulation of vascular endothelial growth factor receptor 1 (VEGFR1), which acts as a decoy receptor for VEGF, as a key mediator of the endothelial response to lipoproteins, and we observed VEGFR1 downregulation in hyperlipidemic mice.
Abstract, second-to-last sentence
The same mouse experiments as described in the body:
In addition, hyperlipidemic Apoe- or Ldlr-null mice (reviewed in refs. 27,28) also showed increased endothelial-specific Vegfr1 expression compared to control WT mice (Fig. 3c–e), confirming conservation of this pathway in higher vertebrates.
Results, VEGFR1 paragraph (last sentence)
The finding that endothelial cells of Ldlr-null mice express elevated amounts of Vegfr1 highlights the potential role of alternative lipoprotein receptors in mediating the endothelial response.
Discussion, paragraph 2
(e) Semiquantitative RT-PCR measurement of Vegfr1 mRNA expression in aortic arches dissected from WT (n = 5) and Apoe-null (n = 5) mice. *P = 4 × 10-14 by z test.
Fig. 3 legend, panel e
All analyses were performed using mice at 8–38 weeks of age fed with a normal chow diet.
Online Methods, "Mice and immunofluorescence"
The zebrafish and cell-culture results in the same Results paragraph:
VEGFR1 protein concentrations were also strongly downregulated in stl mutants but were substantially upregulated in apoCII morphants, in which endothelial cells face high concentrations of ApoB lipoproteins (Fig. 3b). We found similar effects on VEGFR1 levels in endothelial cells in vitro. Treatment of cultured human umbilical vein endothelial cells (HUVECs) with LDL elicited an increase in VEGFR1 but not VEGFR2 concentrations (Fig. 3b).
Results, VEGFR1 paragraph
Every quotation was checked character for character against the article as archived in PubMed Central (src/show.py PMC3959651 --grep; recompute.py.txt repeats the check). The Abstract, Results and Discussion sentences and the Fig. 3e P value were also checked against a text dump of the typeset Nature Medicine PDF (see Status).
What the paper's own numbers show
Method. There is nothing to recompute numerically, because the error is in the stated direction of a change. The Fig. 3e group means are shown only as plotted bars, and the text states their direction explicitly. recompute.py.txt therefore holds a hand-curated list of every main-text statement that gives a direction for VEGFR1 (protein, mRNA or reporter) under a condition in which ApoB-lipoprotein levels are raised or lowered relative to a control. To build it, every sentence of the Abstract, Results and Discussion was read. Each statement is copied verbatim and labelled with two signs: the lipoprotein direction of the condition (+1 higher, −1 lower) and the VEGFR1 direction stated (+1 up, −1 down). The product of the two signs is +1 when VEGFR1 moves with lipoproteins and −1 when it moves against them. The script checks each excerpt against the archived text, prints the table below and draws both figures.
| Quantity | Location | Reported | Recomputed from the paper's own text |
|---|---|---|---|
| VEGFR1 in hyperlipidemic mice | Abstract | "downregulation" | Lipoproteins up, VEGFR1 down: product −1, the only one |
| Endothelial Vegfr1, Apoe- or Ldlr-null vs WT mice | Results; Fig. 3c–e | "increased" | Lipoproteins up, VEGFR1 up: +1 |
| Vegfr1, Ldlr-null endothelium | Discussion, paragraph 2 | "elevated amounts" | Lipoproteins up, VEGFR1 up: +1 |
| Vegfr1 mRNA, aortic arch, WT (n = 5) vs Apoe-null (n = 5) | Fig. 3e legend | P = 4 × 10⁻¹⁴ by z test; direction stated in Results as an increase | Not re-tested (see note below) |
| vegfr1 mRNA, mtp morphants | Results; Fig. 3a | "lower" | Lipoproteins down, VEGFR1 down: +1 |
| VEGFR1 protein, stl mutants | Results; Fig. 3b | "strongly downregulated" | Lipoproteins down, VEGFR1 down: +1 |
| VEGFR1 protein, apoCII morphants | Results; Fig. 3b | "substantially upregulated" | Lipoproteins up, VEGFR1 up: +1 |
| VEGFR1, HUVECs + LDL | Results; Fig. 3b | "an increase" | Lipoproteins up, VEGFR1 up: +1 |
| yfp (vegfr1-promoter reporter) mRNA, mtp knockdown | Results; Fig. 3i,j | "decrease" (P = 0.0248) | Lipoproteins down, VEGFR1 down: +1 |
| Condition statements in which VEGFR1 moves with lipoproteins | Main text | — | 7 of 8 |
| Mouse statements: Abstract vs Results and Discussion | Main text | — | Abstract down; Results up; Discussion up. They disagree. |
The first clause of the same Abstract sentence ("we identified downregulation of ... VEGFR1 ... as a key mediator of the endothelial response to lipoproteins") is consistent with the body. It describes the lipoprotein-depleted stl and mtp-morphant zebrafish, in which VEGFR1 falls. The second clause makes a separate claim about the mice, and that claim has the opposite sign to the Results.
Related observation (not part of the main finding). One concluding phrase in the Results also states the relation with the opposite sign to the result it summarizes. After reporting that "mtp knockdown resulted in a significant (P=0.0248) decrease in YFP expression (Fig. 3i, j)", the sentence goes on: "providing evidence that ApoB-containing lipoproteins inhibit vegfr1 expression at the transcriptional level". Removing lipoproteins lowered the reporter, so the result shows lipoproteins promoting vegfr1 transcription, not inhibiting it. This wording is also in the typeset PDF. The rest of the paragraph makes clear what is meant, so this phrase is less prominent than the Abstract sentence, but a correction could fix both.
Not evaluated. The Fig. 3e statistics (P = 4 × 10⁻¹⁴ from a two-tailed z test of the ratio of two group means against 1, n = 5 per group, as described in the Online Methods) are not part of this finding and were not re-tested. The group means and standard errors are not given in the text.
Figure
Figure 1. The left panel (orange) shows the Abstract clause that states VEGFR1 goes down in hyperlipidemic mice. The right panel (blue) shows what the Results and Discussion report for the same mice: Apoe-null and Ldlr-null mice compared with WT C57BL/6 on a chow diet (Fig. 3c,d immunostaining; Fig. 3e aortic mRNA, n = 5 per group). The connector marks the conflict: one set of experiments, two opposite directions. All text is quoted from the paper. None of the paper's figures or data are reproduced.
Figure 2. Each marker is one of the eight main-text statements that give a direction for VEGFR1 under a raised or lowered ApoB-lipoprotein level (inclusion rule under Method). The horizontal position gives the lipoprotein level of the condition relative to its control. The vertical position gives the VEGFR1 direction stated. Squares are mouse statements; circles are zebrafish or HUVEC statements. The seven blue statements lie in the two "VEGFR1 with lipoproteins" quadrants, as the paper's model requires. The orange Abstract statement is the only one in a "VEGFR1 against lipoproteins" quadrant. The other two mouse statements, for the same animals, lie in the opposite quadrant. Positions show direction only, not magnitude. Directions and locations come from recompute.py.txt.
Why it matters
- The Abstract gives the mammalian result the wrong sign. The mouse experiment is the paper's only in vivo test of the pathway outside zebrafish, and the Results present it as "confirming conservation of this pathway in higher vertebrates". The Abstract states that result in reverse.
- Read alone, the Abstract implies the opposite biology. VEGFR1 is described as a decoy receptor that restrains angiogenesis. If hyperlipidemia lowered it, hyperlipidemia would favour VEGF signalling. The paper argues the reverse: excess ApoB lipoproteins raise VEGFR1 and inhibit angiogenesis (apoCII morphants, LDL-treated HUVECs, Discussion). Its closing link to pathological hyperlipidemia ("seems to be reactivated during pathological conditions of hyperlipidemia") depends on the increase.
- The error reaches readers of the abstract. The same sentence is in the PubMed and Europe PMC record (PMID 22581286), which indexing services, text-mining pipelines and readers without journal access rely on. Crossref lists 114 citing works.
- The data and the main conclusion are not affected. No reported value, test or P value changes. Fig. 3c–e, its legend, the Results and the Discussion all give the increase, and so does every zebrafish and HUVEC experiment. The error is in one clause of the summary, not in the data or analysis.
Explanations considered
- Archiving or conversion artifact. Ruled out. The publisher's abstract page, the typeset PDF and the PubMed/Europe PMC abstract all carry the same wording as the PMC author manuscript. The conflict is in a word ("downregulation" against "increased"), not in a symbol or sign that a conversion could alter.
- A different mouse model or treatment. Ruled out. The only mouse experiments are Apoe-null and Ldlr-null mice versus WT C57BL/6 on a normal chow diet (Online Methods; Fig. 3c–e). No mouse experiment lowers lipids (for example with a statin or loss of Mtp) in a way that could produce a decrease.
- "Downregulation" describes the mechanism, not the mice. Not an explanation. The first clause of the sentence does use "downregulation" for the lipoprotein-depleted state. But the second clause, "we observed VEGFR1 downregulation in hyperlipidemic mice", is a direct observational claim about the mice, and the mice were hyperlipidemic, not depleted.
- A different measure (protein vs mRNA, endothelium vs whole tissue). Ruled out. Immunostaining of aortic endothelium (Fig. 3c,d), aortic-arch mRNA (Fig. 3e), the Results ("increased endothelial-specific Vegfr1 expression") and the Discussion ("elevated amounts of Vegfr1") all give an increase. The paper reports no mouse measure that falls. In the publisher's Fig. 3e bar chart, read by eye, relative Vegfr1 is about 1.0 for WT and about 1.7 for Apoe-null. This approximate reading is not used in any calculation or figure here.
- The supplement resolves it. It does not. The Supplementary Text and Figures (Supplementary Figs. 1–7) contain no mouse data. The mouse data that settle the direction are in main Fig. 3c–e.
- The paper's model predicts a decrease. It does not. In every zebrafish and HUVEC condition VEGFR1 moves the same way as ApoB lipoproteins (7 of 8 condition statements; Fig. 2), so hyperlipidemia should raise VEGFR1, as the Results say.
None of these explanations reconciles the Abstract clause with the paper's mouse results.
Status
As of 2026-10-23:
- Version of record. The abstract on the publisher's page (doi.org redirects to nature.com/articles/nm.2759) contains "we observed VEGFR1 downregulation in hyperlipidemic mice" word for word. The page shows no change history and no correction notice. The full text is paywalled there, so a copy of the typeset Nature Medicine PDF (vol. 18, no. 6, June 2012, p. 967 onward), posted on a university lab website, was converted to text and searched. It contains the Abstract clause, the Results and Discussion sentences, the Fig. 3e P value and the "inhibit vegfr1 expression" phrase exactly as quoted. The PMC copy is the author manuscript (NIHMS475364, pmc-version 1) and has the same wording at all of these places.
- Supplements. Nature's "Supplementary Text and Figures" PDF (Supplementary Figs. 1–7, 8 pages) was searched. It has no matches for Apoe, Ldlr, hyperlip or mice, and VEGFR1 appears only in the title. The PMC manuscript's own supplement (a .pptx file) was not opened.
- Corrections.
src/known.pyreturns no Crossref updates and no PMC or Europe PMC notices. The Crossref record has no update-to, updated-by or relation entries. The Crossref labs API, which carries the Retraction Watch data, lists no updates. The Europe PMC record (PMID 22581286) has no comment/correction list. The Nature article page has no correction link. - PubPeer and the web. The PubPeer site and API could not be reached (HTTP 403 or empty response), so a PubPeer comment cannot be ruled out. Web searches for the title or DOI together with correction, erratum, PubPeer or Retraction Watch found only the article itself and catalogue listings.
- Sources: PMC3959651 (via
src/show.py);src/known.py 10.1038/nm.2759; https://doi.org/10.1038/nm.2759 (publisher page and abstract); the typeset article PDF and Nature's supplementary PDF (41591_2012_BFnm2759_MOESM12_ESM.pdf); https://api.crossref.org/works/10.1038/nm.2759; Europe PMC REST record for PMID 22581286.
Suggested correction
Change the direction in the Abstract so that it matches the Results, Fig. 3c–e and the Discussion:
- Abstract: "...as a key mediator of the endothelial response to lipoproteins, and we observed VEGFR1 upregulation in hyperlipidemic mice."
For consistency, the concluding phrase in the Results paragraph on Fig. 3i,j could also be changed, for example to "providing evidence that ApoB-containing lipoproteins promote vegfr1 expression at the transcriptional level" (or "regulate", as in the Discussion). No change to the data, figures, statistics or conclusions is needed.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC3959651 (Nat Med 2012, doi:10.1038/nm.2759)
# Abstract vs Results / Discussion: which way does VEGFR1 change in hyperlipidemic mice?
#
# Run: python recompute.py.txt
# Prints every number and direction used in writeup.md and regenerates fig1.{svg,png} and
# fig2.{svg,png} in the folder that holds this file. Needs matplotlib.
#
# The finding is about the direction of a change stated in words; there is no numeric value
# to recompute. The script therefore:
# 1. holds a hand-curated list of every statement in the main text (Abstract, Results,
# Discussion) that reports a direction for VEGFR1 under a stated change in ApoB-lipoprotein
# levels, each copied verbatim from the article as archived in PubMed Central and labelled
# with the lipoprotein direction of the condition and the VEGFR1 direction stated;
# 2. if run inside the /home/ubuntu/papers repo, re-checks each excerpt verbatim against the
# archived full text (src/show.py PMC3959651 --full) and confirms its section; skipped
# elsewhere;
# 3. optionally re-checks the same excerpts in a text dump of the publisher's typeset PDF
# (set VOR_TXT=/path/to/vor.txt, made with pdftotext); reported as "not run" otherwise;
# 4. multiplies the two directions for each statement (+1: VEGFR1 moves with lipoproteins,
# -1: against them), tallies them and draws the figures.
# Group sizes and P values are transcribed from the Fig. 3 legend and are not re-tested.
import os, re, subprocess
from collections import OrderedDict
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
# ------------------------------------------------------------------ statements
# key: (location, section tag in show.py --full, condition, lipoprotein direction of the
# condition vs its control (+1 higher, -1 lower), VEGFR1 direction stated, verbatim excerpt)
# Section tags: "ABSTRACT" = abstract line, "BODY" = untitled main-text block (Results and
# Discussion are not separately headed in the archived XML), "FIG3" = Fig. 3 legend.
CONDITIONS = OrderedDict([
("abstract_mice", ("Abstract, last mechanistic sentence", "ABSTRACT",
"Hyperlipidemic mice", +1, -1,
"we observed VEGFR1 downregulation in hyperlipidemic mice")),
("results_mice", ("Results, VEGFR1 paragraph (Fig. 3c-e)", "BODY",
"Apoe- or Ldlr-null mice vs WT", +1, +1,
"hyperlipidemic Apoe- or Ldlr-null mice (reviewed in refs. 27,28) also showed increased "
"endothelial-specific Vegfr1 expression compared to control WT mice")),
("discussion_ldlr", ("Discussion, paragraph 2", "BODY",
"Ldlr-null mice", +1, +1,
"endothelial cells of Ldlr-null mice express elevated amounts of Vegfr1")),
("results_mtp", ("Results, VEGFR1 paragraph (Fig. 3a)", "BODY",
"mtp morphant zebrafish", -1, -1,
"the vegfr1 mRNA levels were lower in mtp morpholino oligonucleotide-injected embryos "
"than in uninjected siblings")),
("results_stl", ("Results, VEGFR1 paragraph (Fig. 3b)", "BODY",
"stl mutant zebrafish", -1, -1,
"VEGFR1 protein concentrations were also strongly downregulated in stl mutants")),
("results_apocii", ("Results, VEGFR1 paragraph (Fig. 3b)", "BODY",
"apoCII morphant zebrafish", +1, +1,
"were substantially upregulated in apoCII morphants, in which endothelial cells face high "
"concentrations of ApoB lipoproteins")),
("results_huvec", ("Results, VEGFR1 paragraph (Fig. 3b)", "BODY",
"HUVECs + LDL", +1, +1,
"with LDL elicited an increase in VEGFR1 but not VEGFR2 concentrations")),
("results_yfp", ("Results, transcription paragraph (Fig. 3i,j)", "BODY",
"mtp knockdown, flt1:YFP reporter", -1, -1,
"mtp knockdown resulted in a significant (P=0.0248) decrease in YFP expression")),
])
# General statements of the lipoprotein -> VEGFR1 relation (no single condition).
# key: (location, section tag, implied sign of the relation, verbatim excerpt)
RELATIONS = OrderedDict([
("results_rescue_logic", ("Results, rescue paragraph (Fig. 3f-h)", "BODY", +1,
"if lipoprotein depletion promotes angiogenesis by downregulation of VEGFR1")),
("results_yfp_conclusion", ("Results, transcription paragraph (Fig. 3i,j)", "BODY", -1,
"providing evidence that ApoB-containing lipoproteins inhibit vegfr1 expression at the "
"transcriptional level")),
("discussion_regulated", ("Discussion, paragraph 2", "BODY", 0,
"expression of VEGFR1 is regulated in vivo by the availability of ApoB-containing "
"lipoproteins")),
("abstract_mediator", ("Abstract, same sentence, first clause", "ABSTRACT", 0,
"we identified downregulation of vascular endothelial growth factor receptor 1 (VEGFR1), "
"which acts as a decoy receptor for VEGF, as a key mediator of the endothelial response "
"to lipoproteins")),
])
# Fig. 3 legend, panel e, and Online Methods (transcribed; not re-tested)
FIG3E = {"n_WT": 5, "n_Apoe_null": 5, "P_reported": 4e-14, "test": "two-tailed z test on ratio vs 1"}
FIG3_SNIPPETS = [
"Semiquantitative RT-PCR measurement of Vegfr1 mRNA expression in aortic arches dissected "
"from WT (n = 5) and Apoe-null (n = 5) mice. *P = 4 × 10-14 by z test.",
"Confocal images of aortic roots in WT C57BL/6 (left, c) and Apoe-null (right, c) littermates "
"and of abdominal aortas in WT C57BL/6 (left, d) and Ldlr-null (right, d) littermates",
]
METHODS_SNIPPET = ("All analyses were performed using mice at 8–38 weeks of age fed with a normal "
"chow diet.")
WORD = {1: "up", -1: "down", 0: "neutral"}
# ------------------------------------------------------------------ verification
def load_archived_text():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
return None
try:
out = subprocess.run([py, "src/show.py", "PMC3959651", "--full"], cwd=repo,
capture_output=True, text=True, timeout=120).stdout
except Exception:
return None
return out or None
def split_sections(txt):
abstract, body, fig3, methods, cur = [], [], [], [], None
for line in txt.splitlines():
if line.startswith("ABSTRACT:"):
abstract.append(line); cur = None; continue
if line.startswith("## ONLINE METHODS"):
cur = "M"; continue
if line.startswith("## "):
cur = "B"; continue
if line.startswith("[Figure 3]"):
cur = "F3"
elif line.startswith("[Figure"):
cur = "OF"
if cur == "B":
body.append(line)
elif cur == "F3":
fig3.append(line)
elif cur == "M":
methods.append(line)
return {"ABSTRACT": "\n".join(abstract), "BODY": "\n".join(body),
"FIG3": "\n".join(fig3), "METHODS": "\n".join(methods)}
def verify(txt):
secs = split_sections(txt)
ok = True
print("Verbatim check against the archived PMC full text (src/show.py PMC3959651 --full):")
items = [(k, v[1], v[5]) for k, v in CONDITIONS.items()] + \
[(k, v[1], v[3]) for k, v in RELATIONS.items()] + \
[(f"fig3_legend_{i}", "FIG3", s) for i, s in enumerate(FIG3_SNIPPETS)] + \
[("methods_mice", "METHODS", METHODS_SNIPPET)]
for key, sec, s in items:
n_all, n_sec = txt.count(s), secs.get(sec, "").count(s)
good = n_sec >= 1
ok &= good
print(f" [{'ok' if good else 'MISSING'}] {key:24s} {n_all}x in full text, {n_sec}x in {sec}")
print(" all verbatim checks passed" if ok else " SOME CHECKS FAILED")
return ok
def verify_vor():
p = os.environ.get("VOR_TXT")
print("\nPublisher typeset PDF check:", end=" ")
if not (p and os.path.exists(p)):
print("not run (set VOR_TXT to a pdftotext dump of the typeset article)")
return
t = open(p, encoding="utf-8", errors="replace").read()
t = re.sub(r"\s+", " ", re.sub(r"-\n", "", t))
norm = lambda s: re.sub(r"\s+", "", s.replace("−", "-"))
tn = norm(t)
print()
for key in ("abstract_mice", "results_mice", "discussion_ldlr"):
print(f" {key:24s} found {tn.count(norm(CONDITIONS[key][5]))}x")
print(f" {'results_yfp_conclusion':24s} found {tn.count(norm(RELATIONS['results_yfp_conclusion'][3]))}x")
print(f" {'fig3e P = 4 x 10-14':24s} found {tn.count(norm('*P = 4 × 10-14 by z test'))}x")
# ------------------------------------------------------------------ tally
def tally():
print("\nCondition statements: lipoprotein direction x VEGFR1 direction")
print(f" {'key':18s} {'location':46s} {'condition':34s} lipo VEGFR1 product")
for k, (loc, _s, cond, l, v, _q) in CONDITIONS.items():
print(f" {k:18s} {loc:46s} {cond:34s} {WORD[l]:5s} {WORD[v]:7s} {l * v:+d}")
prods = {k: v[3] * v[4] for k, v in CONDITIONS.items()}
pos = [k for k, p in prods.items() if p > 0]
neg = [k for k, p in prods.items() if p < 0]
print(f"\n statements with VEGFR1 moving WITH lipoproteins (+1): {len(pos)} of {len(prods)}")
print(f" statements with VEGFR1 moving AGAINST lipoproteins (-1): {len(neg)} of {len(prods)} -> {neg}")
mice = [k for k, v in CONDITIONS.items() if "mice" in v[2]]
print(f" mouse statements: {len(mice)}; directions: "
+ ", ".join(f"{k}={WORD[CONDITIONS[k][4]]}" for k in mice))
print(f" Abstract mouse direction agrees with Results mouse direction: "
f"{CONDITIONS['abstract_mice'][4] == CONDITIONS['results_mice'][4]}")
print(f" all non-Abstract condition statements give +1: "
f"{all(p > 0 for k, p in prods.items() if k != 'abstract_mice')}")
print("\nGeneral relation statements (implied sign of ApoB lipoproteins -> VEGFR1):")
for k, (loc, _s, sgn, _q) in RELATIONS.items():
print(f" {k:24s} {loc:46s} {WORD[sgn] if sgn == 0 else f'{sgn:+d}'}")
print(" note: results_yfp_conclusion (-1) is drawn from the result in results_yfp, "
f"whose own product is {prods['results_yfp']:+d}")
print(f"\nFig. 3e (transcribed, not re-tested): n = {FIG3E['n_WT']} WT vs "
f"{FIG3E['n_Apoe_null']} Apoe-null; P = {FIG3E['P_reported']:.0e} by {FIG3E['test']}")
return prods
# ------------------------------------------------------------------ figures
def style():
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "path", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG,
})
def panel(ax, x, y, w, h, color, title, sub, body):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
lw=2, ec=color, fc="white"))
ax.text(x + 0.02, y + h - 0.035, title, fontsize=12, fontweight="bold", color=color, va="top")
ax.text(x + 0.02, y + h - 0.095, sub, fontsize=10, color=SEC, va="top", style="italic")
yy = y + h - 0.165
for txt, kw in body:
ax.text(x + 0.02, yy, txt, fontsize=kw.get("size", 10.5), color=kw.get("color", TXT),
fontweight=kw.get("weight", "normal"), va="top", linespacing=1.3)
yy -= kw.get("step", 0.12)
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), **kw)
plt.close(fig)
def fig1():
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.98, "VEGFR1 in hyperlipidemic mice: two directions for one experiment",
fontsize=11.5, fontweight="bold", va="top")
panel(ax, 0.03, 0.27, 0.41, 0.64, ORANGE, "Abstract", "summary of the mouse result", [
("Abstract, last mechanistic sentence:", {"color": SEC, "size": 10, "step": 0.045}),
("“…and we observed VEGFR1\ndownregulation in hyperlipidemic\nmice.”", {"step": 0.26}),
("Stated: VEGFR1 goes down", {"color": ORANGE, "weight": "bold"}),
])
n_wt, n_ko = FIG3E["n_WT"], FIG3E["n_Apoe_null"]
panel(ax, 0.56, 0.27, 0.41, 0.64, BLUE, "Results and Discussion",
"Apoe-null and Ldlr-null vs WT mice", [
("Results (Fig. 3c–e):", {"color": SEC, "size": 10, "step": 0.045}),
("“hyperlipidemic Apoe- or Ldlr-null\nmice … showed increased\nendothelial-specific "
"Vegfr1\nexpression compared to control\nWT mice”", {"step": 0.20}),
("Discussion:", {"color": SEC, "size": 10, "step": 0.045}),
("“endothelial cells of Ldlr-null\nmice express elevated amounts\nof Vegfr1”",
{"step": 0.125}),
("Reported: Vegfr1 goes up", {"color": BLUE, "weight": "bold"}),
])
ax.add_patch(FancyArrowPatch((0.448, 0.59), (0.552, 0.59), arrowstyle="<|-|>",
mutation_scale=16, lw=2, color=TXT))
ax.text(0.5, 0.62, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
fontweight="bold", linespacing=1.15)
ax.text(0.03, 0.215,
"These are the paper's only mouse experiments: Apoe-null and Ldlr-null mice vs WT\n"
f"C57BL/6 on chow (Fig. 3c,d staining; Fig. 3e aortic mRNA, n = {n_wt} vs {n_ko}).",
fontsize=10, va="top", linespacing=1.35)
ax.text(0.03, 0.11,
"In zebrafish and HUVECs the paper also reports VEGFR1 rising with lipoproteins\n"
"(apoCII morphants, LDL) and falling without them (mtp morphants, stl mutants).",
fontsize=10, color=SEC, va="top", linespacing=1.35)
save(fig, "fig1")
def fig2(prods):
fig, ax = plt.subplots(figsize=(7.2, 4.9))
fig.subplots_adjust(left=0.2, right=0.98, top=0.84, bottom=0.2)
ax.set_xlim(-1.75, 1.75); ax.set_ylim(-1.75, 1.75)
for v in (0,):
ax.axhline(v, color=GRID, lw=1.2, zorder=0); ax.axvline(v, color=GRID, lw=1.2, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_xticks([-0.9, 0.9]); ax.set_xticklabels(["lower than control", "higher than control"])
ax.set_yticks([-0.9, 0.9]); ax.set_yticklabels(["VEGFR1\nlower", "VEGFR1\nhigher"])
ax.tick_params(length=0, labelsize=10)
ax.set_xlabel("ApoB-lipoprotein level in the condition", fontsize=10.5)
# quadrant notes
ax.text(-1.7, 1.68, "VEGFR1 against lipoproteins", color=SEC, fontsize=10, va="top")
ax.text(1.7, 1.68, "VEGFR1 with lipoproteins", color=SEC, fontsize=10, va="top", ha="right")
ax.text(-1.7, -1.68, "VEGFR1 with lipoproteins", color=SEC, fontsize=10, va="bottom")
ax.text(1.7, -1.68, "VEGFR1 against lipoproteins", color=SEC, fontsize=10, va="bottom",
ha="right")
short = {"abstract_mice": "Abstract:\nhyperlipidemic mice",
"results_mice": "Results: Apoe-/Ldlr-null mice",
"discussion_ldlr": "Discussion: Ldlr-null mice",
"results_apocii": "Results: apoCII morphants",
"results_huvec": "Results: HUVECs + LDL",
"results_mtp": "Results: mtp morphants",
"results_stl": "Results: stl mutants",
"results_yfp": "Results: mtp KD, YFP reporter"}
slot = {}
for k, (_loc, _s, _c, l, v, _q) in CONDITIONS.items():
i = slot.get((l, v), 0); slot[(l, v)] = i + 1
x0, y0 = 0.9 * l, 0.9 * v + 0.45 - 0.26 * i if (l, v) != (1, -1) else -0.9
col = ORANGE if prods[k] < 0 else BLUE
mk = "s" if "mice" in CONDITIONS[k][2] else "o"
ax.plot(x0 - 0.72, y0, marker=mk, ms=9, color=col, mec="white", mew=1, zorder=3)
ax.text(x0 - 0.6, y0, short[k], fontsize=10, va="center",
color=ORANGE if prods[k] < 0 else TXT,
fontweight="bold" if prods[k] < 0 else "normal")
n_pos = sum(p > 0 for p in prods.values()); n = len(prods)
ax.set_title(f"{n_pos} of {n} condition statements: VEGFR1 moves with lipoproteins (blue);\n"
f"the Abstract's mouse statement alone goes against them (orange)",
fontsize=10.5, loc="left", color=TXT)
h = [Line2D([], [], marker="s", color=SEC, ls="none", ms=8, label="mouse statement"),
Line2D([], [], marker="o", color=SEC, ls="none", ms=8, label="zebrafish or HUVEC")]
ax.legend(handles=h, loc="upper left", bbox_to_anchor=(-0.27, -0.13), ncol=2, frameon=False,
fontsize=10)
save(fig, "fig2")
if __name__ == "__main__":
txt = load_archived_text()
if txt:
verify(txt)
else:
print("(archived text not available here; verbatim check skipped)")
verify_vor()
prods = tally()
style()
fig1()
fig2(prods)
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Results text swaps BA.1 and BA.2 predicted vaccine effectiveness relative to Abstract and Fig. 4a
Nat Med (2022) · doi:10.1038/s41591-022-01877-1 · PMC9388371
Summary
The paper predicts mRNA-vaccine effectiveness (VE) against symptomatic infection for four Omicron sublineages, all from the same model (Fig. 4a). The Abstract says predicted VE ranged from 11.9% to 33.3%, highest against BA.1 and lowest against BA.2. The Results sentence that introduces Fig. 4a gives "11.9% for BA.1 and 33.3% for BA.2", which reverses that order. Measured against the axis ticks, the mRNA bars in Fig. 4a are 33.3% for BA.1 and 11.9% for BA.2, so the figure agrees with the Abstract and the Results sentence has the two lineage labels swapped. Anyone who reads only the Results text gets the direction of the BA.1 versus BA.2 comparison backwards, by 21.4 percentage points.
What the paper reports
Abstract:
Among the four sublineages of Omicron, the predicted VE varied between 11.9% and 33.3%, with the highest VE predicted against BA.1 and the lowest against BA.2, using the mRNA vaccine platform.
Results, section "Prediction for variants and Omicron sublineages without known VEs", first paragraph:
Next, we fitted the model with all available data and predicted VE against circulating variants as well as the Omicron sublineages for which there are no observed VE data at the time of writing (Fig. 4a). Interestingly, among the four sublineages of Omicron (BA.1, BA.1.1, BA.2 and BA.3), the expected VEs vary between 11.9% for BA.1 and 33.3% for BA.2, using the mRNA vaccines.
The same paragraph goes on:
This might contribute to the considerable variations in VEs for Omicron reported from observational studies, whose cohorts might have been infected by divergent Omicron sublineages, in addition to differences in immune history.
Fig. 4 legend, panel a:
Predicted VEs for specific variants/sublineages without observed VEs. Omicron sublineages: BA.1, BA.1.1, BA.2 and BA.3.
Fig. 4a is a bar chart titled on its y-axis "Estimated COVID-19 VE (%)". It has one bar per platform (mRNA, protein subunit, inactivated, viral vector) for each of 12 variants. Neither the text nor the supplement prints the plotted values.
What the paper's own numbers show
Method. Both statements describe the same quantity: predicted mRNA-platform VE from the model fitted to all data, for the same four sublineages. Both give the same two endpoints, 11.9% and 33.3%, and differ only in which lineage gets which value. To settle which pairing the model produced, I measured the mRNA bars in Fig. 4a of the version of record, using the publisher's full-resolution image (1557 × 1721 px). I calibrated the y-axis from the six tick marks from 0 to 100 (5.453 px per percentage point, largest tick residual 0.04 pp). For each bar, I located the top edge to sub-pixel precision from the anti-aliased boundary row. Two baseline conventions, the 0 tick or the bar's bottom edge, change the readings by at most 0.2 pp. recompute.py.txt holds the recorded pixel positions and prints every number below. Given the image path, it re-measures them from the image.
| Quantity | Location | Reported | Recomputed (Fig. 4a) |
|---|---|---|---|
| mRNA VE, BA.1 | Results | 11.9% | 33.3% |
| mRNA VE, BA.2 | Results | 33.3% | 11.9% |
| mRNA VE, BA.3 | Fig. 4a only | not stated | 17.5% |
| mRNA VE, BA.1.1 | Fig. 4a only | not stated | 12.5% |
| Sublineage with highest mRNA VE | Abstract | BA.1 | BA.1 |
| Sublineage with lowest mRNA VE | Abstract | BA.2 | BA.2 |
| Range across the four sublineages | Abstract and Results | 11.9% to 33.3% | 11.9% to 33.3% |
| BA.1 minus BA.2 | Results (implied) vs Abstract (positive) | −21.4 pp | +21.4 pp |
The figure's BA.1 and BA.2 bars match the Results' two values to within 0.02 pp, but paired with the opposite lineages. The range and the Abstract's ordering are both consistent with the figure. The only discrepancy is the pairing in the Results sentence.
Figure
Figure 1. The two text passages that report the predicted mRNA VE for Omicron sublineages, quoted from the version of record (top). The orange lines give the pairing that each passage states or implies. The bottom panel gives the mRNA bar heights measured from Fig. 4a of the same paper (blue; reading uncertainty about ±0.2 pp). The Abstract's ordering (BA.1 highest, BA.2 lowest) agrees with the figure. The Results sentence attaches the same two numbers to the opposite lineages.
Figure 2. Predicted mRNA VE for each Omicron sublineage, as measured from the bar heights in Fig. 4a (blue dots), and the values the Results sentence gives for BA.1 and BA.2 (orange diamonds). The arrows run from the figure's value to the text's value for the same lineage. The text's 11.9% sits exactly on the figure's BA.2 value, and its 33.3% on the figure's BA.1 value. Axis from 0; the lineages are in the order of Fig. 4a.
Why it matters
The BA.1 versus BA.2 contrast is the paper's forward-looking claim about which Omicron sublineage existing mRNA vaccines would protect against least. It is one of only two variant-specific predictions in the Abstract. In the Results, the swapped sentence directly precedes the argument that differences between sublineages "might contribute to the considerable variations in VEs for Omicron reported from observational studies". It is also the only place in the text that attaches a number to a named sublineage. A reader who takes the numbers from the Results, for example to compare predicted protection as BA.2 replaced BA.1 in early 2022, gets the direction reversed: 33.3% against BA.2 and 11.9% against BA.1 instead of the reverse, a 2.8-fold ratio pointing the wrong way.
The paper's main findings are unaffected. These include the VE–genetic-distance relationship (R² 86.3% and 87.9%), the validation concordance (CCC 0.95), the Delta prediction (82.8%) and the 11.9–33.3% range. The Abstract and Fig. 4a are correct as published.
Explanations considered
- Different platform, model or time point. Does not apply. Both passages say mRNA vaccines, refer to the same four sublineages and describe the Fig. 4a prediction from the model fitted to all data.
- "Between X for A and Y for B" read as a bare range, with no pairing. Does not apply. The Results sentence explicitly writes "11.9% for BA.1 and 33.3% for BA.2". The Abstract explicitly names BA.1 as highest and BA.2 as lowest. The two cannot both hold.
- The Abstract is the passage in error. Ruled out by Fig. 4a. The mRNA bar for BA.1 measures 33.3% and the bar for BA.2 measures 11.9%. The x-axis order of the sublineages (BA.1, BA.3, BA.1.1, BA.2) also runs from highest to lowest mRNA VE.
- Bars mislabelled in the figure rather than the text. Not supported. The figure and the Abstract agree independently of each other, and the figure gives values for all four sublineages with BA.1 clearly separated from the other three (33.3% vs 17.5% or less). Only the single Results sentence departs from both.
- Rounding or measurement error. Does not apply. The measured values match the reported numbers to within 0.02 pp, and the disagreement is 21.4 pp.
- PMC conversion artefact. Ruled out. The publisher's HTML version of record has the same wording in both places.
Status
As of 2026-10-24:
- Version of record. The publisher's HTML (nature.com/articles/s41591-022-01877-1, fetched directly) has both passages word for word as quoted above. So do the PMC text and the Europe PMC full-text XML. Fig. 4a was measured from the publisher's full-resolution image.
- Supplements. Supplementary Information 1 (VE data tables, mutation tables, representative strains) and the reporting summary list no predicted VE values by sublineage. The code and data repository named in the paper (github.com/VaccineEffectivenessPrediction/COVID19-Vaccine-Effectiveness) returned 404. There is no separate Source Data file.
- Corrections. I found no correction, erratum or retraction. Crossref has no update relations, apart from a link to the medRxiv preprint 10.1101/2021.04.22.21254079. PMC and Europe PMC have no linked notices. The article page has no correction notice, and Retraction Watch has no entry for the DOI.
- PubPeer. It could not be queried directly (it requires human verification). A web search found no PubPeer thread for the article.
Sources: PMC9388371 (archived text); doi:10.1038/s41591-022-01877-1 (publisher HTML, Fig. 4 full-resolution PNG, Supplementary Information); Crossref and PMC/Europe PMC notice queries.
Suggested correction
In the Results, section "Prediction for variants and Omicron sublineages without known VEs", change "the expected VEs vary between 11.9% for BA.1 and 33.3% for BA.2, using the mRNA vaccines" to "the expected VEs vary between 33.3% for BA.1 and 11.9% for BA.2, using the mRNA vaccines". Alternatively, change it to "between 11.9% for BA.2 and 33.3% for BA.1". The Abstract and Fig. 4a need no change.
Recomputation script (python recompute.py.txt)
"""Recompute the Omicron-sublineage predictions in PMC9388371
(Nat Med 2022, doi:10.1038/s41591-022-01877-1).
Run: python recompute.py.txt (uses the pixel positions recorded below)
python recompute.py.txt <Fig4.png> (also re-measures them from the image)
Needs numpy and matplotlib (plus Pillow for the optional re-measurement).
Prints every recomputed number used in writeup.md and writes fig1.svg/png and
fig2.svg/png next to this file.
Input: Fig. 4a of the version of record, the publisher's full-resolution image
https://media.springernature.com/full/springer-static/image/art%3A10.1038%2Fs41591-022-01877-1/MediaObjects/41591_2022_1877_Fig4_HTML.png
(1557 x 1721 px, sha256 b893163d037fabbfa30fdb9fddfc15e3e0facf8ae825db960fa5d98c09be807b).
Panel a ("Estimated COVID-19 VE (%)") is a bar chart with one mRNA bar (salmon, RGB
about 248,118,109) per variant. The plotted values are not printed in the text or
supplement, so they were measured: the y-axis was calibrated from the tick marks
(darkness-weighted centre of each tick in pixel column 121), and the top edge of each
mRNA bar was taken as its first fully coloured row minus the coloured fraction of the
anti-aliased row above it. Pixel row r covers [r, r+1), so a tick centred on row r
sits at r + 0.5.
"""
import os
import sys
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Fig. 4a pixels)
TICK_ROW = np.array([42.75, 152.0, 260.75, 370.0, 479.25, 588.0]) # tick centroids, column 121
TICK_VAL = np.array([100.0, 80.0, 60.0, 40.0, 20.0, 0.0]) # tick labels, %
BAR_BOTTOM_EDGE = 588.0 # last coloured row of every mRNA bar is 587
# variant (x-axis order) -> (first fully coloured row, coloured fraction of the row above)
BARS = {
"B.1": (75, 0.20), "B.1.526": (96, 0.87), "B.1.1.318": (95, 0.66),
"C.37": (126, 0.28), "B.1.621": (157, 0.74), "C.1.2": (171, 0.66),
"B.1.640.1": (210, 0.73), "B.1.640.2": (237, 0.86), "BA.1": (407, 0.08),
"BA.3": (494, 0.67), "BA.1.1": (521, 0.29), "BA.2": (524, 0.29),
}
OMICRON = ["BA.1", "BA.1.1", "BA.2", "BA.3"]
# ---------------------------------------------------------------- inputs (text, verbatim)
Q_ABSTRACT = ("Among the four sublineages of Omicron, the predicted VE varied between "
"11.9% and 33.3%, with the highest VE predicted against BA.1 and the lowest "
"against BA.2, using the mRNA vaccine platform.")
Q_RESULTS = ("Interestingly, among the four sublineages of Omicron (BA.1, BA.1.1, BA.2 and "
"BA.3), the expected VEs vary between 11.9% for BA.1 and 33.3% for BA.2, "
"using the mRNA vaccines.")
REPORTED_RESULTS = {"BA.1": 11.9, "BA.2": 33.3} # pairing given in the Results
REPORTED_RANGE = (11.9, 33.3) # range given in Abstract and Results
ABSTRACT_ORDER = ("BA.1", "BA.2") # (highest, lowest) per the Abstract
def remeasure(path):
"""Re-measure tick centroids and mRNA bar tops from the publisher's Fig. 4 PNG."""
from PIL import Image
im = np.asarray(Image.open(path).convert("RGB")).astype(float)[:700]
dk = 765 - im[:, 121].sum(1)
print(f"re-measured from {os.path.basename(path)}:")
cents = []
for t in [43, 152, 261, 370, 479, 588]:
w = np.arange(t - 3, t + 4)
cents.append((dk[w] * w).sum() / dk[w].sum())
print(" tick centroids:", [round(float(c), 2) for c in cents])
bar = np.array([248, 118, 109.0])
bg = np.array([255, 255, 255.0])
sal = ((abs(im[:, :, 0] - 248) < 15) & (abs(im[:, :, 1] - 118) < 20)
& (abs(im[:, :, 2] - 109) < 20))
xs = np.where(sal.any(0))[0]
groups, s, p = [], xs[0], xs[0]
for x in xs[1:]:
if x != p + 1:
groups.append((s, p))
s = x
p = x
groups.append((s, p))
groups = [g for g in groups if g[1] - g[0] > 8 and g[0] < 1250]
for name, (g0, g1) in zip(BARS, groups):
col = sal[:, (g0 + g1) // 2]
top = int(np.where(col)[0].min())
px = im[top - 1, g0 + 2:g1 - 1].mean(0)
frac = float(np.clip(((bg - px) @ (bg - bar)) / ((bg - bar) @ (bg - bar)), 0, 1))
print(f" {name:10s} x={g0}-{g1} top row {top} fraction above {frac:.2f}"
f" last row {int(np.where(col)[0].max())}")
print()
# ---------------------------------------------------------------- calculation
centre = TICK_ROW + 0.5
slope, icpt = np.polyfit(centre, TICK_VAL, 1)
resid = TICK_VAL - (slope * centre + icpt)
px_per_pct = -1 / slope
def ve_at(edge):
return slope * edge + icpt
ve = {k: float(ve_at(top - frac)) for k, (top, frac) in BARS.items()}
# alternative baseline: bar bottom edge instead of the 0 tick (half-pixel convention check)
ve_alt = {k: (BAR_BOTTOM_EDGE - (top - frac)) / px_per_pct for k, (top, frac) in BARS.items()}
om = {k: ve[k] for k in OMICRON}
hi = max(om, key=om.get)
lo = min(om, key=om.get)
def main():
if len(sys.argv) > 1:
remeasure(sys.argv[1])
print("Quotes used:\n Abstract: " + Q_ABSTRACT + "\n Results: " + Q_RESULTS + "\n")
print(f"Axis calibration: {px_per_pct:.4f} px per percentage point; "
f"max tick residual {np.abs(resid).max():.3f} pp")
print(f"Half a pixel = {0.5 / px_per_pct:.2f} pp (measurement resolution)\n")
print("mRNA VE read from Fig. 4a (tick baseline / bar-bottom baseline):")
for k in BARS:
print(f" {k:10s} {ve[k]:5.1f}% ({ve_alt[k]:5.1f}%)")
print("\nOmicron sublineages, mRNA:",
", ".join(f"{k} {om[k]:.1f}%" for k in OMICRON))
print(f"Figure: highest = {hi} ({om[hi]:.1f}%), lowest = {lo} ({om[lo]:.1f}%)")
print(f"Figure range across the four sublineages: {om[lo]:.1f}% to {om[hi]:.1f}%"
f" (reported {REPORTED_RANGE[0]}% to {REPORTED_RANGE[1]}%)")
print("Abstract ordering (highest BA.1, lowest BA.2) matches figure:",
(hi, lo) == ABSTRACT_ORDER)
r = REPORTED_RESULTS
print(f"Results pairing: BA.1 {r['BA.1']}%, BA.2 {r['BA.2']}% -> "
f"BA.1 minus BA.2 = {r['BA.1'] - r['BA.2']:+.1f} pp")
print(f"Figure: BA.1 {om['BA.1']:.1f}%, BA.2 {om['BA.2']:.1f}% -> "
f"BA.1 minus BA.2 = {om['BA.1'] - om['BA.2']:+.1f} pp")
print(f"Results ratio BA.2/BA.1 = {r['BA.2'] / r['BA.1']:.2f}; "
f"figure ratio BA.1/BA.2 = {om['BA.1'] / om['BA.2']:.2f}")
for k, v in r.items():
near = min(om, key=lambda s: abs(om[s] - v))
print(f" Results value {v}% (labelled {k}) is nearest to the figure's {near} "
f"({om[near]:.1f}%, diff {om[near] - v:+.2f} pp); "
f"figure {k} = {om[k]:.1f}% (diff {om[k] - v:+.1f} pp)")
print(f"Swapped pairing BA.1 33.3% / BA.2 11.9% vs figure: "
f"{om['BA.1'] - 33.3:+.2f} / {om['BA.2'] - 11.9:+.2f} pp")
make_fig1()
make_fig2()
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5,
"text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC})
def box(ax, x, y, w, h, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=edge, lw=1.5, transform=ax.transAxes))
def make_fig1():
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
# Abstract panel
box(ax, 0.03, 0.56, 0.44, 0.38, SEC)
ax.text(0.05, 0.905, "Abstract", fontsize=12, weight="bold", va="top")
ax.text(0.05, 0.835, "“the predicted VE varied between\n11.9% and 33.3%, with the highest\n"
"VE predicted against BA.1 and the\nlowest against BA.2”",
fontsize=10.5, va="top", color=TXT, linespacing=1.35)
ax.text(0.05, 0.595, "implies BA.1 33.3%, BA.2 11.9%", fontsize=10.5,
color=ORANGE, weight="bold")
# Results panel
box(ax, 0.53, 0.56, 0.44, 0.38, SEC)
ax.text(0.55, 0.905, "Results (Fig. 4a section)", fontsize=12, weight="bold", va="top")
ax.text(0.55, 0.835, "“the expected VEs vary between\n11.9% for BA.1 and 33.3% for\n"
"BA.2, using the mRNA vaccines”", fontsize=10.5, va="top", linespacing=1.35)
ax.text(0.55, 0.595, "states BA.1 11.9%, BA.2 33.3%", fontsize=10.5,
color=ORANGE, weight="bold")
# conflict connector
ax.add_patch(FancyArrowPatch((0.47, 0.75), (0.53, 0.75), arrowstyle="<->",
mutation_scale=14, lw=2, color=TXT, transform=ax.transAxes))
ax.text(0.5, 0.975, "Same quantity, same model, opposite assignment of BA.1 and BA.2",
ha="center", va="top", fontsize=10.5, color=SEC)
# Figure panel
box(ax, 0.20, 0.05, 0.60, 0.34, BLUE)
ax.text(0.22, 0.355, "Fig. 4a, mRNA bar heights measured", fontsize=12,
weight="bold", va="top")
ax.text(0.22, 0.28, f"BA.1 {om['BA.1']:.1f}% BA.3 {om['BA.3']:.1f}%\n"
f"BA.1.1 {om['BA.1.1']:.1f}% BA.2 {om['BA.2']:.1f}%",
fontsize=11, va="top", color=BLUE, weight="bold", linespacing=1.5,
family="DejaVu Sans Mono")
ax.text(0.22, 0.10, "highest BA.1, lowest BA.2 (reading uncertainty about ±0.2 pp)",
fontsize=10, color=SEC)
a1 = FancyArrowPatch((0.30, 0.39), (0.22, 0.56), arrowstyle="-|>", mutation_scale=14,
lw=2, color=BLUE, transform=ax.transAxes)
a2 = FancyArrowPatch((0.70, 0.39), (0.78, 0.56), arrowstyle="-|>", mutation_scale=14,
lw=2, color=ORANGE, linestyle=(0, (4, 2)), transform=ax.transAxes)
ax.add_patch(a1)
ax.add_patch(a2)
ax.text(0.155, 0.47, "agrees", fontsize=10.5, color=BLUE, weight="bold", ha="center")
ax.text(0.86, 0.47, "labels\nswapped", fontsize=10.5, color=ORANGE, weight="bold",
ha="center", va="center")
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
def make_fig2():
fig, ax = plt.subplots(figsize=(7.2, 3.4), facecolor=BG)
ax.set_facecolor(BG)
order = ["BA.1", "BA.3", "BA.1.1", "BA.2"] # x-axis order of Fig. 4a
y = np.arange(len(order))[::-1]
for yi, k in zip(y, order):
ax.plot([om[k]], [yi], "o", ms=10, color=BLUE, zorder=3,
label="Fig. 4a bar height (measured)" if k == "BA.1" else None)
ax.text(om[k], yi + 0.27, f"{om[k]:.1f}%", color=BLUE, ha="center", fontsize=10)
if k in REPORTED_RESULTS:
v = REPORTED_RESULTS[k]
ax.plot([v], [yi], "D", ms=9, mfc="none", mew=2, color=ORANGE, zorder=4,
label="Results text" if k == "BA.1" else None)
ax.annotate("", xy=(v, yi), xytext=(om[k], yi),
arrowprops=dict(arrowstyle="-|>", lw=2, color=ORANGE,
shrinkA=7, shrinkB=7))
ax.text(v, yi - 0.33, f"{v}%", color=ORANGE, ha="center", fontsize=10)
ax.set_yticks(y)
ax.set_yticklabels(order, fontsize=11, color=TXT)
ax.set_xlim(0, 40)
ax.set_ylim(-0.7, 3.7)
ax.set_xlabel("Predicted VE against symptomatic infection, mRNA vaccines (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
ax.legend(loc="center right", frameon=False, fontsize=10)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
if __name__ == "__main__":
main()
Between-subject SD of 2.8 Hz for alpha peak frequency is incompatible with the paper's Table 1
Neuroimage (2014) · doi:10.1016/j.neuroimage.2014.01.049 · PMC4013551
Summary
This MEG study of 51 subjects reports in its Abstract, Results and Discussion that posterior alpha peak frequency averaged 10.3 Hz, with a within-subject SD of 0.9 Hz and a between-subject SD of 2.8 Hz. The paper's own Table 1 lists the mean, SD and N of every detected peak for each of its 10 condition × ROI cells. Its Ns add up to the 499 detected observations, and its grand mean reproduces 10.3 Hz. Pooled exactly, these cells give an SD of 1.27 Hz across all 499 peaks. No between-subject SD can be larger than that, and once the reported within-subject SD of 0.9 Hz is taken out, about 0.90 Hz is left. The 2.8 Hz figure is used to say that between-subject variability is higher than the roughly 1 Hz reported earlier and about three times the within-subject SD. By the paper's own numbers it is neither.
What the paper reports
Abstract:
The average alpha peak frequency across all subjects, conditions, and regions of interest was 10.3 Hz with a within-subject SD of 0.9 Hz and a between-subject SD of 2.8 Hz.
Results, "Between-subject variability":
Taking all conditions and ROIs into account, detected alpha peak frequencies spread the full 7–14 Hz search range. The bin with the highest number of observations was centered at 10.8 Hz. The mean alpha peak frequency across subjects was 10.3 Hz with a between-subject SD of 2.8 Hz (compare with a within-subject SD of 0.9 Hz), and the median was 10.4 Hz.
Discussion, opening paragraph:
The average alpha peak frequency across subjects, conditions, and ROIs was 10.3 Hz with a within-subject SD of 0.9 Hz and a between-subject SD of 2.8 Hz.
Discussion, "Inter-subject variability":
As expected, inter-subject variability exceeded intra-subject variability. Individual alpha peak frequency values were found including the entire search range of 7–14 Hz, with a between-subject SD of 2.8 Hz (note that the fact that we find a higher SD than previously reported may be due to the use of various tasks and more subjects).
Introduction (the earlier value that the Discussion compares against):
The reasoning is that because of substantial inter-individual variability in alpha frequency (a mean SD of 1 Hz is reported, cf. Klimesch, 1997), significant portions of alpha power will fall outside a fixed frequency window
Table 1 caption, and the table:
Individual alpha peak frequency. Showing per condition and ROI: mean ± standard deviation (N). N indicates number of subjects that had a detectable peak for that particular condition ∗ ROI combination.
| Occipital ROI | Parietal ROI | |
|---|---|---|
| Rest | 9.981 ± 1.088 (47) | 10.143 ± 1.147 (49) |
| Base | 9.916 ± 1.303 (50) | 10.075 ± 1.107 (51) |
| Stim | 10.102 ± 1.249 (51) | 10.108 ± 1.092 (51) |
| 0-back | 10.716 ± 1.412 (49) | 10.304 ± 1.344 (51) |
| 2-back | 11.118 ± 1.132 (50) | 10.816 ± 1.311 (50) |
Fig. 4 caption, panel D:
Total number of observations was 510 (i.e., 2 ROIs, 5 conditions, 51 subjects). Note: number of observations of undetectable peak frequency was 11, not shown in histogram.
Results, "Beta peak frequency in relation to alpha" (parietal ROI, rest, N = 32):
we multiplied each individual's alpha peak frequency with factor 2 to get the estimated harmonic beta peak (mean, 20.3 Hz; SD, 2.3 Hz)
What the paper's own numbers show
Method. Each Table 1 cell holds one alpha peak frequency per subject with a detectable peak. The ten cells together therefore hold every detected observation, and their combined mean and SD follow exactly from the cell means m, SDs s and Ns n (law of total variance): the total sum of squares is Σ(n−1)s² + Σn(m − M)², where M is the N-weighted grand mean, and the SD of all observations is √(SS / (N − 1)). A between-subject SD is the spread of subjects around the grand mean, so it cannot exceed the spread of all observations. If the within-subject SD of 0.9 Hz is taken as given, the between-subject variance is what is left of the total variance after 0.9² is subtracted. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Number of detected observations | Table 1 Ns; Fig. 4D caption | 510 − 11 undetectable | Σn = 499: Table 1 covers every detected peak |
| Grand mean | Abstract; Results; Discussion | 10.3 Hz | 10.328 Hz (N-weighted mean of the cells): matches |
| SD within each cell (one value per subject) | Table 1 | 1.088–1.412 Hz | pooled within-cell SD 1.224 Hz |
| Spread of the 10 cell means | Table 1 | 9.916–11.118 Hz | N-weighted SD 0.387 Hz |
| SD of all 499 detected peaks | derived from Table 1 | not reported | 1.273 Hz (variance 1.621 Hz²; within-cell SS 732.71, between-cell SS 74.68) |
| Between-subject SD | Abstract; Results; Discussion (×2) | 2.8 Hz | at most 1.27 Hz; 0.90 Hz after removing within-subject 0.9 Hz (√(1.621 − 0.81)) |
| Reported vs recomputed between-subject SD | derived | 2.8 Hz | 2.20 × the upper bound; 3.11 × the implied value |
| Total variance the text implies | derived | 2.8² + 0.9² = 8.65 Hz² | 5.34 × the 1.62 Hz² that Table 1 allows (2.8² alone is 4.84 ×) |
| Alpha SD in the beta section (parietal, rest) | Results, beta section | 2 × alpha: SD 2.3 Hz, mean 20.3 Hz | alpha SD 1.15 Hz, mean 10.15 Hz (Table 1 parietal rest: 10.143 ± 1.147) |
| Ratio of between- to within-subject SD | derived | 2.8 / 0.9 = 3.1 | 1.00 (implied); at most 1.41 |
| Largest SD possible for values in 7–14 Hz | Methods (search range) | — | 3.50 Hz (half the range; Bhatia–Davis bound with mean 10.33 is also 3.50) |
The 2.8 Hz figure lies about two-thirds of the way from the observed spread to the largest SD that values confined to 7–14 Hz can have at all. Each of the ten Table 1 SDs is itself a between-subject SD for one condition and ROI. All ten lie between 1.09 and 1.41 Hz, and the beta section gives 1.15 Hz for the parietal rest condition in a further subset of subjects.
Figure
Figure 1. Standard deviations of alpha peak frequency (Hz). Orange: values printed in the paper, namely the between-subject SD of 2.8 Hz given in the Abstract, Results and Discussion, and the ten per-cell SDs of Table 1 (open circles), each of which is an SD across subjects for one condition and ROI. Blue: values recomputed from the paper's own numbers, namely the alpha SD implied by the beta section (2.3 Hz ÷ 2), the exact SD of all 499 detected peaks pooled from Table 1 (1.27 Hz, an upper bound for any between-subject SD), and the between-subject SD left once the reported within-subject SD of 0.9 Hz is removed (0.90 Hz). The dashed line marks the roughly 1 Hz reported in earlier work, as cited in the Introduction. The solid line at 3.5 Hz is the largest SD that any set of values within the 7–14 Hz search range can have.
Figure 2. The same comparison on the variance scale (Hz²). Upper bar: the variance the text implies, a between-subject 2.8² = 7.84 plus a within-subject 0.9² = 0.81, for 8.65 Hz² in total. Lower bar: the exact variance of all 499 detected peaks computed from Table 1, 1.62 Hz². With the same within-subject 0.81 removed, 0.81 Hz² is left for between-subject variation (SD 0.90 Hz). The text requires 5.3 times the total variance present in the paper's own table.
Why it matters
The between-subject SD is one of the headline results of a paper about inter- and intra-individual variability. It appears in the Abstract and twice in the Discussion, and three interpretive claims rest on it.
- Comparison with earlier work. The Discussion says the paper finds "a higher SD than previously reported" and attributes this to "the use of various tasks and more subjects". The earlier value cited in the Introduction is about 1 Hz. The paper's own data give 0.90 Hz (at most 1.27 Hz), which agrees with that value and does not exceed it.
- Between versus within. The text contrasts 2.8 Hz with 0.9 Hz, a ratio of about 3. The recomputed between-subject SD is about the same as the within-subject SD (ratio 1.00, at most 1.41). The Discussion's statement that "inter-subject variability exceeded intra-subject variability" may still hold, but it cannot rest on the reported ratio, and the paper gives no other test of it.
- Fixed 8–12 Hz band. The argument that a fixed band "would bias against those with an alpha peak frequency outside of that range" is right in kind: the paper reports detected peaks over the whole 7–14 Hz search range. Its size is overstated, though. As a rough guide only (the distribution is described as skewed in some conditions), a normal distribution with mean 10.3 Hz and SD 2.8 Hz would put 47.8% of values outside 8–12 Hz and 21.2% outside the 7–14 Hz search range, where no detected peak can lie. With the table's SD of 1.27 Hz the figures are 12.8% and 0.6%.
The condition effects do not depend on the 2.8 Hz value: the repeated-measures ANOVA, the increase from rest and visual stimulation to N-back, and the 2-back versus 0-back difference. Nor does the recommendation to use individual alpha frequencies, which follows from the observed range of peaks. Readers who cite the paper for the size of between-subject variability in alpha peak frequency would, however, carry forward a value about 2.2 to 3.1 times too large. Crossref records more than 600 citations of the article.
Explanations considered
- The 2.8 Hz figure describes different observations from Table 1. This does not account for the numbers. The Table 1 Ns add up to 499, exactly the 510 observations minus the 11 undetectable peaks in the Fig. 4D caption, and the table's grand mean (10.328 Hz) reproduces the reported 10.3 Hz. The Results paragraph that gives 2.8 Hz cites "Table 1, Fig. 4D" itself.
- The Table 1 entries are standard errors, not SDs. Excluded. The caption says "mean ± standard deviation (N)". As SEMs they would imply SDs of 7.5–9.9 Hz, above the 3.5 Hz ceiling for values within 7–14 Hz. Fig. 4C shows SEMs separately.
- "Between-subject SD" means the SD of each subject's average over conditions and ROIs. This does not account for the numbers. Subject averages vary less than the individual observations they summarise, so their SD cannot exceed the 1.27 Hz of all observations (apart from a negligible effect of the 11 missing values). With the reported within-subject SD of 0.9 Hz it is about 0.90 Hz.
- The 11 undetectable peaks were coded as a number and left in. This does not account for the numbers. Coded as 0 Hz they raise the SD to 1.96 Hz but lower the mean to 10.11 Hz, which no longer matches the reported 10.3 Hz. Coded as 7 or 14 Hz they give SDs of 1.35 and 1.37 Hz. None reaches 2.8 Hz. The Methods also state that undetected peaks were set to NaN.
- 2.8 is a variance, not an SD. Excluded. The total variance of the observations is 1.62 Hz², and the between-subject variance implied by the text and table together is 0.81 Hz².
- 2.8 is a simple multiple of the pooled SD (for example a confidence half-width or a range). This is not supported. 1.96 × 1.27 = 2.50 and 2 × 1.27 = 2.55, neither of which rounds to 2.8; √5 × 1.27 = 2.85 comes close but corresponds to no quantity the paper names. The paper does not say how 2.8 Hz or 0.9 Hz were computed, so the source of the value cannot be identified from the published material.
- Rounding. Excluded. The table gives means and SDs to three decimals, and 2.8 is more than twice the largest SD any of them allows.
- A conversion or typesetting error in the archived text. Excluded. The value appears four times in identical form in the publisher-deposited PMC text, in Europe PMC's full-text XML and on the live PMC page, and Table 1 is native XML table markup with the same entries in every rendering.
Status
- Version of record. The PMC copy is the publisher's own deposit (collection "Elsevier Sponsored Documents", not an author manuscript; CC BY 3.0; PMC version 1). An earlier check compared three renderings, the live PMC page, the Europe PMC full-text XML and the locally archived PMC text: all carry the four 2.8 Hz sentences and the same Table 1. ScienceDirect could not be fetched automatically (the doi.org link resolves to a redirect stub, and the article page returned HTTP 403), and the PDF links on PMC and Europe PMC returned HTML, so the publisher's PDF was not compared directly. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. None. The PMC record has no supplementary material, and Europe PMC lists none (hasSuppl = N). No public subject-level data were found from which the between-subject SD could be recomputed directly.
- Corrections and comments. As of 2026-10-24 no correction, erratum, corrigendum, expression of concern or retraction was found. Crossref lists no update relations for the DOI, and a Crossref search for works updating it (which includes Retraction Watch data) returned none. PMC and Europe PMC list no notices, and the PubMed record (PMID 24508648) has no comments or corrections linked. A web search for corrections or comments on the article found none. PubPeer could not be checked directly: its search page returned a Cloudflare challenge (HTTP 403) and its publication API returned HTTP 404.
- Sources. PMC article PMC4013551 (archived and live); Europe PMC full-text XML and record for PMID 24508648; Crossref work record for doi:10.1016/j.neuroimage.2014.01.049; PubMed.
Suggested correction
- Recompute the between-subject SD of alpha peak frequency from the subject-level data, state how it is defined (for example the SD across subjects of each subject's mean over conditions and ROIs, or a variance component from the repeated-measures model) and how the within-subject SD of 0.9 Hz is defined, and replace 2.8 Hz in the Abstract, Results and Discussion. Table 1 implies a value of about 0.9 Hz, and at most 1.27 Hz.
- Revise the Discussion statement that the study finds "a higher SD than previously reported": the value implied by Table 1 is in line with the roughly 1 Hz cited in the Introduction.
- Restate the comparison between inter- and intra-subject variability with the corrected figures, and support the claim that inter-subject variability exceeds intra-subject variability with a stated test or variance-component estimate, if it still holds.
- Optionally, report the share of detected peaks that fall outside 8–12 Hz, taken directly from the Fig. 4D histogram, so that the argument against a fixed band rests on the observed proportion.
Recomputation script (python recompute.py.txt)
"""Recompute the between-subject SD of alpha peak frequency in PMC4013551
(Neuroimage 2014, doi:10.1016/j.neuroimage.2014.01.049,
"Inter- and intra-individual variability in alpha peak frequency").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 1 (legend: "mean ± standard deviation (N). N indicates number of subjects that had a
detectable peak for that particular condition * ROI combination"):
Occipital ROI Parietal ROI
Rest 9.981 ± 1.088 (47) 10.143 ± 1.147 (49)
Base 9.916 ± 1.303 (50) 10.075 ± 1.107 (51)
Stim 10.102 ± 1.249 (51) 10.108 ± 1.092 (51)
0-back 10.716 ± 1.412 (49) 10.304 ± 1.344 (51)
2-back 11.118 ± 1.132 (50) 10.816 ± 1.311 (50)
Fig. 4D legend: 510 observations (2 ROIs x 5 conditions x 51 subjects), 11 undetectable.
Abstract / Results / Discussion: mean 10.3 Hz, within-subject SD 0.9 Hz, between-subject SD 2.8 Hz.
Methods: peaks searched within 7-14 Hz.
Results, beta section (parietal ROI, rest, N = 32): 2 x alpha peak frequency has mean 20.3 Hz, SD 2.3 Hz.
Introduction: prior literature "a mean SD of 1 Hz is reported".
Method: the Table 1 cells together hold every detected observation. The exact mean and SD of
all of them follow from the cell means, SDs and Ns (law of total variance):
SS_total = sum (n_j - 1) s_j^2 + sum n_j (m_j - M)^2, SD_all = sqrt(SS_total / (N - 1)).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import Patch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
CELLS = [ # (condition, ROI, mean, SD, N) -- Table 1
("Rest", "Occipital", 9.981, 1.088, 47), ("Rest", "Parietal", 10.143, 1.147, 49),
("Base", "Occipital", 9.916, 1.303, 50), ("Base", "Parietal", 10.075, 1.107, 51),
("Stim", "Occipital", 10.102, 1.249, 51), ("Stim", "Parietal", 10.108, 1.092, 51),
("0-back", "Occipital", 10.716, 1.412, 49), ("0-back", "Parietal", 10.304, 1.344, 51),
("2-back", "Occipital", 11.118, 1.132, 50), ("2-back", "Parietal", 10.816, 1.311, 50),
]
REP_MEAN, REP_WITHIN, REP_BETWEEN, REP_MEDIAN = 10.3, 0.9, 2.8, 10.4
TOTAL_OBS, UNDETECTED = 510, 11
LO, HI = 7.0, 14.0 # peak search range
BETA_2ALPHA_MEAN, BETA_2ALPHA_SD, BETA_N = 20.3, 2.3, 32
PRIOR_SD = 1.0 # Introduction, cf. literature
m = np.array([c[2] for c in CELLS]); s = np.array([c[3] for c in CELLS]); n = np.array([c[4] for c in CELLS])
print("=== 1. Table 1 covers the same observations as the 2.8 Hz figure ===")
N = int(n.sum())
print(f"sum of the 10 cell Ns = {N}; 510 - 11 undetected = {TOTAL_OBS - UNDETECTED}")
M = (n * m).sum() / N
print(f"N-weighted grand mean = {M:.3f} Hz (reported {REP_MEAN}); unweighted mean of cell means = {m.mean():.3f}")
print(f"cell means range {m.min():.3f}-{m.max():.3f} Hz; cell SDs range {s.min():.3f}-{s.max():.3f} Hz")
print("\n=== 2. Exact SD of all detected observations (law of total variance) ===")
SSw = ((n - 1) * s ** 2).sum()
SSb = (n * (m - M) ** 2).sum()
var_all = (SSw + SSb) / (N - 1)
sd_all = np.sqrt(var_all)
sd_within_cell = np.sqrt(SSw / (N - len(CELLS)))
sd_cellmeans = np.sqrt(SSb / N)
print(f"within-cell SS = {SSw:.2f}; between-cell SS = {SSb:.2f}")
print(f"variance of all {N} observations = {var_all:.3f} Hz^2; SD = {sd_all:.3f} Hz")
print(f"pooled within-cell SD = {sd_within_cell:.3f} Hz; N-weighted SD of the 10 cell means = {sd_cellmeans:.3f} Hz")
print("\n=== 3. Between-subject SD the table allows ===")
var_between_implied = var_all - REP_WITHIN ** 2
sd_between_implied = np.sqrt(var_between_implied)
print(f"upper bound (no within-subject variation at all): {sd_all:.2f} Hz")
print(f"after removing the reported within-subject SD {REP_WITHIN}: sqrt({var_all:.3f} - {REP_WITHIN**2:.2f}) "
f"= sqrt({var_between_implied:.3f}) = {sd_between_implied:.2f} Hz")
print(f"reported between-subject SD {REP_BETWEEN} is {REP_BETWEEN / sd_all:.2f} x the upper bound and "
f"{REP_BETWEEN / sd_between_implied:.2f} x the implied value")
print(f"variance for SD {REP_BETWEEN}: {REP_BETWEEN**2:.2f} Hz^2 = {REP_BETWEEN**2 / var_all:.2f} x the total variance {var_all:.2f}")
need_total = REP_BETWEEN ** 2 + REP_WITHIN ** 2
print(f"between {REP_BETWEEN}^2 + within {REP_WITHIN}^2 = {need_total:.2f} Hz^2 (SD {np.sqrt(need_total):.2f}) "
f"= {need_total / var_all:.2f} x the total variance the table allows")
print(f"ratio between/within: reported {REP_BETWEEN / REP_WITHIN:.1f}; implied {sd_between_implied / REP_WITHIN:.2f}; "
f"upper bound {sd_all / REP_WITHIN:.2f}")
print("\n=== 4. Other bounds and cross-checks inside the paper ===")
bd = np.sqrt((M - LO) * (HI - M))
print(f"largest SD possible for values confined to {LO:.0f}-{HI:.0f} Hz: (14-7)/2 = {(HI-LO)/2:.2f} Hz; "
f"with mean {M:.2f} (Bhatia-Davis): {bd:.2f} Hz")
alpha_sd_beta = BETA_2ALPHA_SD / 2
print(f"beta section, parietal rest (N={BETA_N}): 2 x alpha has SD {BETA_2ALPHA_SD} -> alpha SD {alpha_sd_beta:.2f} Hz; "
f"mean {BETA_2ALPHA_MEAN}/2 = {BETA_2ALPHA_MEAN/2:.2f} Hz (Table 1 parietal rest: 10.143 ± 1.147)")
print(f"if Table 1 entries were SEMs, the SDs would be SEM x sqrt(N) = {np.round(s * np.sqrt(n), 1).min()}-"
f"{np.round(s * np.sqrt(n), 1).max()} Hz, above the {(HI-LO)/2:.1f} Hz ceiling")
print("\n=== 5. The 11 undetected peaks coded as a number and left in ===")
alt = {}
for code in (0.0, 7.0, 14.0):
N2 = N + UNDETECTED
M2 = (N * M + UNDETECTED * code) / N2
SS2 = SSw + (n * (m - M2) ** 2).sum() + UNDETECTED * (code - M2) ** 2
alt[code] = (M2, np.sqrt(SS2 / (N2 - 1)))
print(f"coded as {code:4.1f} Hz: mean {M2:.2f} Hz, SD {alt[code][1]:.2f} Hz")
print("\n=== 6. Simple transformations of the pooled SD (none is named in the paper) ===")
print(f"sqrt(2.8) = {np.sqrt(2.8):.2f}; 2 x SD_all = {2*sd_all:.2f}; 1.96 x SD_all = {1.96*sd_all:.2f}; "
f"sqrt(5) x SD_all = {np.sqrt(5)*sd_all:.2f}; sqrt(10) x SD_all = {np.sqrt(10)*sd_all:.2f}")
print("\n=== 7. Share of a normal distribution outside 8-12 Hz and outside 7-14 Hz ===")
share = {}
for lab, mu, sd in (("reported (10.3, 2.8)", REP_MEAN, REP_BETWEEN), ("all peaks (10.33, 1.27)", M, sd_all),
("implied between-subject (10.33, 0.90)", M, sd_between_implied)):
d = stats.norm(mu, sd)
o812 = d.cdf(8) + d.sf(12); o714 = d.cdf(7) + d.sf(14)
share[lab] = (o812, o714)
print(f"{lab:40s}: outside 8-12 Hz {100*o812:4.1f}%; outside 7-14 Hz {100*o714:4.1f}%")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
LW, MS = 2, 9
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"axes.spines.top": False, "axes.spines.right": False, "axes.spines.left": False,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
# Figure 1: every SD in or derived from the paper, on one Hz axis
fig, ax = plt.subplots(figsize=(7.2, 3.9))
rows = [
"Text: between-subject SD\n(Abstract, Results, Discussion)",
"Table 1: SD across subjects\nin each condition × ROI cell",
"Beta section: SD of 2 × alpha\n(parietal, rest) ÷ 2",
"All 499 detected peaks pooled\n(exact, from Table 1)",
"Between-subject SD implied after\nremoving within-subject SD 0.9",
]
ys = np.arange(len(rows))[::-1]
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.axvline((HI - LO) / 2, color=SEC, lw=1, zorder=1)
ax.text((HI - LO) / 2 - 0.04, ys[-1] - 0.45, "largest SD possible\nwithin the 7–14 Hz\nsearch range (3.5)",
ha="right", va="bottom", fontsize=10, color=SEC)
ax.axvline(PRIOR_SD, color=SEC, lw=1, ls=(0, (3, 3)), zorder=1)
ax.text(PRIOR_SD + 0.04, ys[0] + 0.42, "≈1 Hz, prior literature\n(Introduction)", ha="left", va="top", fontsize=10,
color=SEC)
# row 0: reported 2.8
ax.plot(REP_BETWEEN, ys[0], "o", ms=MS, color=ORANGE, zorder=4)
ax.annotate(f"{REP_BETWEEN:.1f}", (REP_BETWEEN, ys[0]), xytext=(0, 9), textcoords="offset points", ha="center",
fontsize=10.5, fontweight="bold")
# row 1: ten reported cell SDs
ax.plot([s.min(), s.max()], [ys[1]] * 2, color=ORANGE, lw=LW, zorder=3)
ax.plot(s, [ys[1]] * len(s), "o", ms=MS, mfc=BG, mec=ORANGE, mew=2, zorder=4)
ax.annotate(f"{s.min():.2f}–{s.max():.2f} (10 cells)", (s.max(), ys[1]), xytext=(10, 0), textcoords="offset points",
ha="left", va="center", fontsize=10)
# rows 2-4: recomputed
for y, v, lab in ((ys[2], alpha_sd_beta, f"{alpha_sd_beta:.2f} (N = 32)"),
(ys[3], sd_all, f"{sd_all:.2f}: upper bound for any between-subject SD"),
(ys[4], sd_between_implied, f"{sd_between_implied:.2f}")):
ax.plot(v, y, "o", ms=MS, color=BLUE, zorder=4)
ax.annotate(lab, (v, y), xytext=(10, 0), textcoords="offset points", ha="left", va="center", fontsize=10)
ax.set_yticks(ys); ax.set_yticklabels(rows, fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_xlim(0, 3.8); ax.set_ylim(ys[-1] - 0.6, ys[0] + 0.6)
ax.set_xlabel("Standard deviation of alpha peak frequency (Hz)")
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=ORANGE, label="reported in the paper"),
Line2D([], [], marker="o", ls="", ms=MS, color=BLUE,
label="recomputed from the paper's own numbers")],
loc="upper center", bbox_to_anchor=(0.32, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: variance budget in Hz^2
fig, ax = plt.subplots(figsize=(7.2, 2.6))
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
w2 = REP_WITHIN ** 2
bars = [ # (y, between part, colour, label)
(1, REP_BETWEEN ** 2, ORANGE, "Text: between 2.8² + within 0.9²"),
(0, var_between_implied, BLUE, "Table 1: variance of all 499 peaks"),
]
for y, b, col, _ in bars:
ax.barh(y, w2, height=0.5, color=BG, edgecolor=ORANGE, hatch="///", lw=1.5, zorder=3)
ax.barh(y, b, left=w2, height=0.5, color=col, zorder=3)
ax.text(w2 + REP_BETWEEN ** 2 + 0.12, 1, f"{need_total:.2f} Hz² ({need_total / var_all:.1f} × the table's total)",
va="center", fontsize=10)
ax.text(var_all + 0.12, 0, f"{var_all:.2f} Hz² (SD {sd_all:.2f})", va="center", fontsize=10)
ax.text(w2 + REP_BETWEEN ** 2 / 2, 1, f"between-subject 7.84", ha="center", va="center", fontsize=10, color="white",
fontweight="bold")
ax.annotate(f"left for between-subject: {var_between_implied:.2f}", (w2 + var_between_implied / 2, -0.25),
xytext=(w2 + var_between_implied / 2 - 0.15, -0.62), ha="left", va="center", fontsize=10, color=SEC,
arrowprops=dict(arrowstyle="-", color=SEC, lw=1, shrinkA=6, shrinkB=0))
ax.set_yticks([b[0] for b in bars]); ax.set_yticklabels([b[3] for b in bars], fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_xlim(0, 12); ax.set_ylim(-0.85, 1.45)
ax.set_xlabel("Variance of alpha peak frequency (Hz²)")
ax.legend(handles=[Patch(facecolor=BG, edgecolor=ORANGE, hatch="///", label="within-subject 0.9² = 0.81 (reported)"),
Patch(color=ORANGE, label="between-subject (reported)"),
Patch(color=BLUE, label="between-subject (recomputed)")],
loc="upper center", bbox_to_anchor=(0.4, -0.32), ncol=3, frameon=False, fontsize=10, handlelength=1.6)
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract's CI and p-value for the coprimary fatigue score match a secondary endpoint, not the Results
Neurology (2023) · doi:10.1212/WNL.0000000000207402 · PMC10382259
Summary
The abstract reports the between-group difference on the coprimary PMMSA total fatigue score (TFS) as −0.07 (95% CI −0.10 to 0.26; p = 0.37). The Results give the same estimate with a 95% CI of −0.69 to 0.54 and p = 0.81, and those values follow from the arm LS means and SEs printed next to them (SE of the difference 0.311, which gives a 95% CI of −0.69 to 0.53 and p = 0.80). The abstract interval is centred on +0.08, not on −0.07. Its CI and p-value are identical to those the supplementary eTable 1 gives for a secondary endpoint, the PMMSA Most Bothersome Symptom score (0.08; −0.10 to 0.26; p 0.37). The trial is negative under either version, but the abstract's interval is 3.4 times narrower than the correct one. It would rule out any fatigue benefit larger than 0.10 points, whereas the paper's data are compatible with a benefit of up to 0.69 points.
What the paper reports
All quotations are from the article as archived in PubMed Central. The publisher's typeset PDF gives the same numbers, with true minus signs.
Abstract, Results paragraph:
Between the participants receiving elamipretide and those receiving placebo, the difference in the least squares mean (SE) from baseline to week 24 on distance walked on the 6MWT was -3.2 (95% CI -18.7 to 12.3; p = 0.69) meters, and on the PMMSA, the total fatigue score was -0.07 (95% CI -0.10 to 0.26; p = 0.37).
Results, "Primary Endpoints of Overall ITT", second paragraph:
The LS mean (SE) of change from baseline to week 24 on the PMMSA TFS was -1.13 (±0.22) for participants receiving elamipretide and -1.05 (±0.22) for participants receiving placebo, a -0.07 difference between the 2 groups (95% CI -0.69 to 0.54; p = 0.81) (Figure 2B). The PP participant analysis demonstrated a 0.09 difference between the 2 groups (95% CI -0.54 to 0.72; p = 0.78).
Methods, "Efficacy Assessments", which defines the endpoint:
Coprimary endpoints evaluated the effect of elamipretide for 24 weeks including the distance walked (in meters) on the 6MWT and the total fatigue score on the Primary Mitochondrial Myopathy Symptom Assessment (PMMSA), previously described in detail.
Methods, "Statistical Analysis", which describes the model:
Analyses of continuous endpoints were conducted using a mixed-model repeated measures approach, with fixed effects for treatment, visit, the treatment-by-visit interaction, and participant as a random effect.
Results, which points to the supplement:
Secondary endpoint results are summarized in eTable 1 (links.lww.com/WNL/C852).
Supplementary eTable 1, "Secondary Endpoints Results at Week 24", first row (transcribed from the one-page PDF):
| PMMSA Most Bothersome Symptom Score | Elamipretide 40 mg (n = 98) | Placebo (n = 101) |
|---|---|---|
| LS Mean (SE) | ‒0.21 (±0.06) | –0.29 (+0.06) |
| LSM Diff (95% CI) | 0.08 (‒0.10, 0.26) | |
| p-value | 0.37 |
What the paper's own numbers show
Method. The paper reports MMRM estimates. Without the model's covariance matrix, I approximated the SE of each between-group difference as √(SE₁² + SE₂²) from the two arms' LS-mean SEs, since the arms are independent. I then formed Wald 95% intervals (±1.96 SE) and two-sided p-values. For each interval the paper prints, I also worked backwards: a symmetric Wald interval has its estimate at the midpoint and an SE of (upper − lower)/3.92. Using a t reference with about 200 df instead of z changes nothing at two decimals. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| TFS difference, elamipretide − placebo | Results, ITT para 2 | −0.07 | −0.08 from −1.13 − (−1.05); CI midpoint −0.075 |
| SE of the TFS difference | Results, ITT para 2 | arm SEs 0.22 and 0.22 | 0.311 from the arms; 0.314 implied by the Results CI |
| TFS 95% CI | Results, ITT para 2 | −0.69 to 0.54 | −0.69 to 0.53 (agrees) |
| TFS p | Results, ITT para 2 | 0.81 | 0.80 from the arms; 0.81 from the CI (agrees) |
| TFS 95% CI | Abstract | −0.10 to 0.26 | should be −0.69 to 0.54. The stated interval has midpoint +0.08 and implied SE 0.092 |
| Position of −0.07 in the abstract CI | Abstract | — | 0.03 above the lower bound and 0.33 below the upper bound. The midpoint is 0.15 above the estimate |
| TFS p | Abstract | 0.37 | should be 0.81. The abstract CI implies p = 0.38 for +0.08, and p = 0.45 for the stated −0.07 |
| Estimate that p = 0.37 would need at SE 0.311 | Abstract | — | about ±0.28 (|z| = 0.896) |
| Per-arm SE needed to produce the abstract CI | Abstract | — | 0.065; the Results give 0.22 |
| Ratio of CI widths, Results / abstract | — | — | 3.4 |
| Most Bothersome Symptom difference | eTable 1 | 0.08 (‒0.10, 0.26); p 0.37 | +0.08 from −0.21 − (−0.29); SE 0.085; CI −0.09 to 0.25; p 0.35 (agrees) |
| Abstract TFS CI and p vs eTable 1 MBS CI and p | Abstract, eTable 1 | −0.10 to 0.26, p 0.37 in both | identical |
| TFS per-protocol | Results | 0.09 (−0.54 to 0.72); p 0.78 | p 0.78 (agrees); does not match the abstract |
| TFS mtDNA subgroup | Results | −0.21 (−0.9 to 0.5); p 0.55 | p 0.56 (agrees); does not match the abstract |
| TFS nDNA subgroup | Results | 0.03; p 0.93 | +0.03, CI −0.65 to 0.71, p 0.93 (agrees); does not match the abstract |
| 6MWT ITT | Abstract and Results | −3.2 (−18.7 to 12.3); p 0.69 | −3.2, CI −19.0 to 12.6, p 0.69. Abstract and Results agree |
| Unadjusted week-24 TFS change (external) | ClinicalTrials.gov NCT03323749 | −1.18 (SD 2.132, n 98) vs −1.09 (SD 2.443, n 101) | difference −0.09, CI −0.73 to 0.55, Welch p 0.78. Supports the Results version |
The Results values for the TFS are internally consistent, and the abstract values are not. The abstract's interval and p-value fit together, since an estimate of +0.08 with SE 0.092 gives p ≈ 0.38. But they fit an estimate of +0.08, not the stated −0.07, and they need an SE about one third of the SE the arm statistics give. A difference of +0.08 on (−0.10, 0.26) with p = 0.37 is exactly what eTable 1 reports for the Most Bothersome Symptom score. That endpoint's arm SEs (0.06 each) reproduce the interval to within rounding.
Figure
Figure 1. Between-group differences (elamipretide minus placebo) in change from baseline to week 24, with 95% confidence intervals, all on one points scale. Orange circles are values printed in the paper. Blue squares are values recomputed by recompute.py.txt from the arm LS means and SEs that the paper prints next to them. Top three rows: the coprimary PMMSA total fatigue score (TFS) as given in the abstract, as given in the Results text, and as recomputed. In the abstract row the stated estimate (−0.07) sits near the lower end of its interval, and a tick marks the interval's midpoint (+0.08). Bottom two rows: the secondary PMMSA Most Bothersome Symptom (MBS) score from supplementary eTable 1, as printed and as recomputed. The dashed connector joins the two rows whose 95% CI (−0.10 to 0.26) and p-value (0.37) are identical. The vertical line marks no difference. The right-hand columns give each estimate, its interval and its p-value.
Why it matters
- It is a coprimary endpoint, and the error is in the abstract. The TFS is one of the trial's two coprimary endpoints. Abstracts are what most readers, indexers and systematic reviewers extract. Anyone taking the abstract's interval would record a standard error of about 0.09 instead of about 0.31, which gives the trial about 11.5 times its actual weight in an inverse-variance meta-analysis ((0.311 / 0.092)² = 11.5).
- Precision is overstated, though the conclusion is not reversed. Both versions are non-significant and the direction of the estimate is the same. Under the Hochberg rule in the Methods, the smaller of the two coprimary p-values had to be ≤ 0.025. That is 0.37 with the abstract value and 0.69 with the Results values, so neither version changes the formal outcome. The abstract's interval would exclude any benefit larger than 0.10 points, while the correct interval extends to a 0.69-point benefit. The abstract therefore presents a much tighter null than the trial measured.
- It bears on the paper's own discussion. The Discussion says that participants given elamipretide "did report slightly less total fatigue" and that future studies should test whether this is a true effect. The correct, wider interval is compatible with that hypothesis to a degree that the abstract's interval is not.
Explanations considered
- A different analysis population for the TFS. This does not account for the numbers. The per-protocol TFS result (0.09; −0.54 to 0.72; p 0.78), the mtDNA subgroup (−0.21; −0.9 to 0.5; p 0.55) and the nDNA subgroup (0.03; p 0.93) all have intervals 3 to 4 times wider than the abstract's, and none has p = 0.37.
- A different model or adjustment for the same endpoint. This is unlikely. Any TFS analysis of these participants must reflect the observed spread of the TFS change. The posted ClinicalTrials.gov summary (SDs 2.13 and 2.44, n 98 and 101) gives an unadjusted SE of 0.325, close to the MMRM-based 0.311. Reaching SE 0.092 would require per-arm SEs of about 0.065 rather than 0.22, and nothing in the paper reports variability that small for the TFS.
- An asymmetric interval, for example from a transformation or bootstrap. This does not fit. The Results interval for the same contrast is symmetric about −0.075, so the model produces symmetric Wald-type intervals. An interval with its estimate 0.03 from one bound and 0.33 from the other would not come from the same method.
- A one-sided p-value or a per-item rescaling. Neither reproduces 0.37. Half of 0.81 is 0.405, and dividing the Results interval by the number of TFS items (4) gives −0.172 to 0.135, not −0.10 to 0.26.
- A conversion artefact in PMC. This is ruled out. The publisher's typeset PDF, the live PMC page and the Europe PMC full-text XML all give the same abstract values.
- Values carried over from the secondary Most Bothersome Symptom row of eTable 1. This accounts for every number. The abstract's CI (−0.10 to 0.26) and p (0.37) are identical to eTable 1's MBS values. The MBS estimate (0.08) is the exact midpoint of that interval, and the MBS arm SEs reproduce it (−0.09 to 0.25; p 0.35). The abstract kept the TFS point estimate (−0.07), which matches the Results. The paper does not say how the abstract values were produced. The match is identified here from the numbers alone.
Status
- Version of record. The neurology.org page returned HTTP 403. The publisher's typeset open-access PDF (Neurology 2023;101(3):e238–e252, 16 pages, as held in an institutional repository), the live PMC page and the Europe PMC full-text XML were read instead. All three carry the same abstract (−0.07; −0.10 to 0.26; p = 0.37) and Results (−0.07; −0.69 to 0.54; p = 0.81) values.
- Supplements. I downloaded and read eTable 1 (links.lww.com/WNL/C852, one page, "Secondary Endpoints Results at Week 24"). It holds the Most Bothersome Symptom row quoted above. eFigure 1 (links.lww.com/WNL/C851, genotype distribution) was also read and contains no TFS estimates. The coinvestigator list (C860) is not relevant.
- Trial registry. I read the ClinicalTrials.gov posted results for NCT03323749 (API v2). They give week-24 TFS change −1.18 (SD 2.132, n 98) for elamipretide and −1.09 (SD 2.443, n 101) for placebo.
- Corrections. I found no correction, erratum, retraction or expression of concern. Crossref has no update-to or updated-by relation, there are no PMC or Europe PMC notices (
src/known.py, rerun on 2026-10-24), and the PubMed record (PMID 37268435) lists no linked comments or errata. Web searches found no notice for this paper. The only elamipretide correction in Neurology concerns an earlier (2018) trial. - Post-publication discussion. A web search found no PubPeer thread or Retraction Watch entry. PubPeer's API required human verification, so its own search could not be run directly, and the Europe PMC search API was unavailable (HTTP 503) on three attempts.
- Status as of 2026-10-24.
Suggested correction
In the abstract's Results paragraph, replace
on the PMMSA, the total fatigue score was −0.07 (95% CI −0.10 to 0.26; p = 0.37)
with
on the PMMSA, the total fatigue score was −0.07 (95% CI −0.69 to 0.54; p = 0.81)
which matches the Results section, Figure 2B and the arm LS means and SEs (−1.13 ± 0.22 vs −1.05 ± 0.22). The authors should confirm these values against the MMRM output. No other sentence needs to change: the abstract's statements that the study did not meet its primary endpoints, and the Class I evidence statement, hold under the corrected values.
Recomputation script (python recompute.py.txt)
"""Recompute the PMMSA total fatigue score (TFS) result of PMC10382259
(Neurology 2023;101(3):e238-e252, doi:10.1212/WNL.0000000000207402).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg and fig1.png next to this file.
Inputs are copied from the paper as archived in PMC (abstract and Results text) and from the
paper's supplementary eTable 1 (links.lww.com/WNL/C852). The external check uses the
summary statistics posted on ClinicalTrials.gov for NCT03323749 (week-24 TFS change).
Approximation: the paper's estimates come from a mixed-model repeated-measures (MMRM)
analysis. Without the model covariance, the SE of a between-group difference is approximated
as sqrt(SE1^2 + SE2^2) from the two arms' LS-mean SEs (the arms are independent), and Wald
intervals use z = 1.96. A t reference with about 200 df changes nothing at two decimals.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
def p_two_sided(est, se):
return 2 * stats.norm.sf(abs(est) / se)
def from_ci(lo, hi):
"""Midpoint, implied SE and implied p of a symmetric Wald 95% CI."""
mid = (lo + hi) / 2
se = (hi - lo) / (2 * Z)
return mid, se
def from_arms(m1, se1, m2, se2):
d = m1 - m2
se = float(np.hypot(se1, se2))
return d, se, d - Z * se, d + Z * se, p_two_sided(d, se)
# ------------------------------------------------------------------ inputs
ABS_TFS = dict(est=-0.07, lo=-0.10, hi=0.26, p=0.37) # Abstract, Results paragraph
RES_TFS = dict(est=-0.07, lo=-0.69, hi=0.54, p=0.81) # Results, Primary Endpoints of Overall ITT, para 2
RES_ARMS = (-1.13, 0.22, -1.05, 0.22) # LS mean (SE): elamipretide, placebo
ETAB_MBS = dict(est=0.08, lo=-0.10, hi=0.26, p=0.37) # eTable 1, PMMSA Most Bothersome Symptom score
ETAB_ARMS = (-0.21, 0.06, -0.29, 0.06) # eTable 1 LS mean (SE): elamipretide, placebo
OTHER = [ # other TFS / 6MWT results in the Results text: (label, est, lo, hi, reported p)
("TFS per-protocol", 0.09, -0.54, 0.72, 0.78),
("TFS mtDNA subgroup", -0.21, -0.9, 0.5, 0.55),
("6MWT ITT (m)", -3.2, -18.7, 12.3, 0.69),
("6MWT per-protocol (m)", -2.2, -16.9, 12.5, 0.77),
]
NDNA_TFS = (-0.45, 0.25, -0.48, 0.24, 0.93) # nDNA subgroup arms + reported p (no CI given)
MWT_ARMS = (14.1, 5.7, 17.3, 5.7)
CTGOV = (-1.18, 2.132, 98, -1.09, 2.443, 101) # mean, SD, n: elamipretide, placebo
print("== 1. Results text, ITT TFS: LS means -1.13 (SE 0.22) vs -1.05 (SE 0.22)")
r_d, r_se, r_lo, r_hi, r_p = from_arms(*RES_ARMS)
print(f" difference = {r_d:.2f}; SE of difference = {r_se:.3f}; 95% CI {r_lo:.2f} to {r_hi:.2f}; p = {r_p:.2f}")
mid, se = from_ci(RES_TFS["lo"], RES_TFS["hi"])
print(f" reported CI -0.69 to 0.54: midpoint {mid:.3f}, implied SE {se:.3f}, "
f"p at midpoint {p_two_sided(mid, se):.2f}, p at -0.07 {p_two_sided(-0.07, se):.2f}")
for df in (190, 200, 210):
t = stats.t.ppf(0.975, df)
s = (RES_TFS["hi"] - RES_TFS["lo"]) / (2 * t)
print(f" t reference, df = {df}: SE {s:.3f}, p at midpoint {2 * stats.t.sf(abs(mid) / s, df):.2f}")
print("== 2. Abstract, ITT TFS: -0.07 (95% CI -0.10 to 0.26; p = 0.37)")
a_mid, a_se = from_ci(ABS_TFS["lo"], ABS_TFS["hi"])
print(f" midpoint {a_mid:+.2f}; implied SE {a_se:.3f}; ratio of SEs (Results / abstract) {r_se / a_se:.1f}; "
f"ratio of CI widths {(RES_TFS['hi'] - RES_TFS['lo']) / (ABS_TFS['hi'] - ABS_TFS['lo']):.1f}")
print(f" distance from -0.07 to the bounds: {ABS_TFS['est'] - ABS_TFS['lo']:.2f} below, "
f"{ABS_TFS['hi'] - ABS_TFS['est']:.2f} above; midpoint minus estimate {a_mid - ABS_TFS['est']:.2f}")
print(f" p implied at the midpoint: {p_two_sided(a_mid, a_se):.2f}; p for -0.07 with SE {a_se:.3f}: "
f"{p_two_sided(-0.07, a_se):.2f}")
zp = stats.norm.isf(0.37 / 2)
print(f" |z| for p = 0.37: {zp:.3f}; estimate needed at SE {r_se:.3f}: {zp * r_se:.2f}; at SE {a_se:.3f}: {zp * a_se:.2f}")
print(f" SE per arm needed for the abstract CI (equal SEs): {a_se / np.sqrt(2):.3f} (Results: 0.22)")
print("== 3. eTable 1, PMMSA Most Bothersome Symptom score (secondary): 0.08 (-0.10, 0.26); p 0.37")
m_d, m_se, m_lo, m_hi, m_p = from_arms(*ETAB_ARMS)
print(f" from arms -0.21 (0.06) vs -0.29 (0.06): difference {m_d:+.2f}; SE {m_se:.3f}; "
f"95% CI {m_lo:.2f} to {m_hi:.2f}; p = {m_p:.2f}")
e_mid, e_se = from_ci(ETAB_MBS["lo"], ETAB_MBS["hi"])
print(f" reported CI midpoint {e_mid:+.2f} (reported estimate {ETAB_MBS['est']:+.2f}); implied SE {e_se:.3f}; "
f"p at the estimate {p_two_sided(ETAB_MBS['est'], e_se):.2f}")
print(f" abstract TFS CI and p identical to eTable 1 MBS CI and p: "
f"{(ABS_TFS['lo'], ABS_TFS['hi'], ABS_TFS['p']) == (ETAB_MBS['lo'], ETAB_MBS['hi'], ETAB_MBS['p'])}")
print("== 4. Other analyses in the Results (do any match the abstract's CI and p?)")
for name, e, lo, hi, p in OTHER:
mid_, se_ = from_ci(lo, hi)
print(f" {name}: est {e}; CI {lo} to {hi} (midpoint {mid_:.2f}, SE {se_:.3f}); "
f"p from est {p_two_sided(e, se_):.2f}; reported p {p}")
nd, nse, nlo, nhi, npv = from_arms(*NDNA_TFS[:4])
print(f" TFS nDNA subgroup from arms: diff {nd:+.2f}; CI {nlo:.2f} to {nhi:.2f}; p {npv:.2f}; reported p {NDNA_TFS[4]}")
w_d, w_se, w_lo, w_hi, w_p = from_arms(*MWT_ARMS)
print(f" 6MWT ITT from arms 14.1 (5.7) vs 17.3 (5.7): diff {w_d:.1f}; CI {w_lo:.1f} to {w_hi:.1f}; p {w_p:.2f} "
f"(abstract and Results agree: -3.2, -18.7 to 12.3, p 0.69)")
aw = ABS_TFS["hi"] - ABS_TFS["lo"]
print(f" CI width ratios vs the abstract (width {aw:.2f}): "
+ ", ".join(f"{n} {(hi - lo) / aw:.1f}" for n, e, lo, hi, p in OTHER[:2])
+ f", TFS nDNA (recomputed CI) {(nhi - nlo) / aw:.1f}")
print(f" inverse-variance weight ratio, abstract SE vs Results SE: ({r_se:.3f}/{a_se:.3f})^2 = {(r_se / a_se) ** 2:.1f}")
print(f" one-sided p from two-sided 0.81: {0.81 / 2:.3f}; Results CI divided by 4 TFS items: "
f"{RES_TFS['lo'] / 4:.3f} to {RES_TFS['hi'] / 4:.3f}")
print("== 5. External check: ClinicalTrials.gov NCT03323749, unadjusted week-24 TFS change")
m1, s1, n1, m2, s2, n2 = CTGOV
dd = m1 - m2
sd = np.sqrt(s1 ** 2 / n1 + s2 ** 2 / n2)
welch = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2, equal_var=False)
print(f" -1.18 (SD 2.132, n 98) vs -1.09 (SD 2.443, n 101): diff {dd:.2f}; SE {sd:.3f}; "
f"CI {dd - Z * sd:.2f} to {dd + Z * sd:.2f}; Welch p {welch.pvalue:.2f}")
print("== 6. Hochberg rule for the coprimary family (Methods): smaller p must be <= 0.025")
print(f" with abstract TFS p: smaller p = min(0.69, 0.37) = {min(0.69, 0.37)}; "
f"with Results TFS p: min(0.69, 0.81) = {min(0.69, 0.81)}; neither <= 0.025")
# ------------------------------------------------------------------ figure
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
rows = [ # (label, est, lo, hi, p, kind)
("Abstract\nTFS (coprimary)", ABS_TFS["est"], ABS_TFS["lo"], ABS_TFS["hi"], ABS_TFS["p"], "rep"),
("Results text\nTFS (coprimary)", RES_TFS["est"], RES_TFS["lo"], RES_TFS["hi"], RES_TFS["p"], "rep"),
("TFS from Results\narm LS means and SEs", r_d, r_lo, r_hi, r_p, "rec"),
("eTable 1\nMost Bothersome Symptom\n(secondary)", ETAB_MBS["est"], ETAB_MBS["lo"], ETAB_MBS["hi"],
ETAB_MBS["p"], "rep"),
("MBS from eTable 1\narm LS means and SEs", m_d, m_lo, m_hi, m_p, "rec"),
]
ys = [5.0, 4.0, 3.0, 1.6, 0.6]
fig, ax = plt.subplots(figsize=(7.2, 5.0))
fig.subplots_adjust(left=0.285, right=0.655, top=0.97, bottom=0.2)
XL, XR = -0.8, 0.6
ax.set_xlim(XL, XR)
ax.set_ylim(0.1, 6.3)
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.axhline(2.3, color=GRID, lw=1.0)
def fmt(v):
return f"{v:+.2f}".replace("-", "\u2212")
trans = ax.get_yaxis_transform() # x in axes fraction, y in data
for (lab, e, lo, hi, p, kind), y in zip(rows, ys):
col = ORANGE if kind == "rep" else BLUE
mk = "o" if kind == "rep" else "s"
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([lo, lo], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot([hi, hi], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot([e], [y], marker=mk, ms=9, color=col, mec=BG, mew=1.0, zorder=4, ls="none")
ax.text(1.04, y, f"{fmt(e)} ({fmt(lo)}, {fmt(hi)})", va="center", ha="left", fontsize=10,
color=INK, transform=trans, clip_on=False)
ax.text(1.80, y, f"{p:.2f}", va="center", ha="right", fontsize=10, color=INK,
transform=trans, clip_on=False)
ax.text(1.04, 6.1, "estimate (95% CI)", fontsize=10, color=INK2, ha="left", va="center", transform=trans)
ax.text(1.80, 6.1, "p", fontsize=10, color=INK2, ha="right", va="center", transform=trans)
# stated estimate vs midpoint of the abstract interval
ax.plot([a_mid], [ys[0]], marker="|", ms=14, mew=2, color=ORANGE, zorder=4)
ax.annotate("stated estimate \u22120.07", xy=(-0.07, ys[0] + 0.14), xytext=(-0.13, ys[0] + 0.5),
fontsize=10, color=INK2, ha="right", va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8, shrinkA=1, shrinkB=0))
ax.annotate("interval midpoint +0.08", xy=(a_mid, ys[0] + 0.14), xytext=(0.13, ys[0] + 0.5),
fontsize=10, color=INK2, ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8, shrinkA=1, shrinkB=0))
# connector: abstract interval = eTable 1 MBS interval
ax.annotate("", xy=(0.36, ys[3] + 0.18), xytext=(0.36, ys[0] - 0.18),
arrowprops=dict(arrowstyle="<->", color=INK, lw=1.2, linestyle=(0, (3, 2))))
ax.text(0.33, 2.3, "same CI\nand p", fontsize=10, color=INK, ha="right", va="center",
bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"))
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.tick_params(axis="y", length=0)
ticks = [-0.8, -0.4, 0, 0.4]
ax.set_xticks(ticks)
ax.set_xticklabels([("0" if t == 0 else f"{t:+.1f}".replace("-", "\u2212").replace("+", "")) for t in ticks])
ax.set_xlabel("Elamipretide minus placebo, change to week 24 (points)\n"
"negative values favour elamipretide", fontsize=10, color=INK, x=0.5)
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
h1, = ax.plot([], [], marker="o", ms=9, color=ORANGE, lw=2, label="Reported in the paper")
h2, = ax.plot([], [], marker="s", ms=9, color=BLUE, lw=2,
label="Recomputed from the paper's arm LS means and SEs")
fig.legend(handles=[h1, h2], loc="lower left", bbox_to_anchor=(0.01, 0.005), ncol=2, frameon=False,
fontsize=10, handlelength=2.2, columnspacing=1.5)
fig.savefig(os.path.join(HERE, "fig1.svg"))
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200)
print("wrote fig1.svg, fig1.png")
Abstract's baseline cognitive-component estimates differ from the Results and Table 2, changing one component's significance
Neurology (2026) · doi:10.1212/WNL.0000000000214804 · PMC13067375
Summary
The abstract reports cross-sectional estimates from the hurdle ("zero-inflation") model of CDR-SB, adjusted for EYO, for each of the three cognitive components (reserve CogR, demographic CogD, biomarker CogB). There are three estimates for asymptomatic carriers and two for symptomatic carriers. The Results text and Table 2 describe the same model and outcome and agree with each other, but all five of the abstract's estimates or intervals differ from them. The largest difference is for the demographic component in asymptomatic carriers: the abstract gives an odds ratio of 2.60 (95% CI 1.10–6.16), which excludes 1, while the Results and Table 2 give 1.24 (0.76–2.02), which includes 1. The abstract also understates the biomarker odds ratio (5.16 against 7.77). The headline reserve finding keeps its direction and significance in both versions. However, the abstract's statement that the demographic component is associated with being unimpaired at baseline is not supported by the paper's own Table 2.
What the paper reports
Abstract, "Result":
A total of 710 Dominantly Inherited Alzheimer Network (DIAN) participants were included in the analysis, comprising 271 non-DIAD carriers (nMC), 284 asymptomatic DIAD carriers (aMC), and 155 symptomatic DIAD carriers.
In asymptomatic carriers, using a zero-inflation model adjusted for EYO showed that a 1 SD increase in the reserve component (CogR) was associated with a 4.06-fold increase in the odds of being clinically unimpaired (CDR-SB = 0; 95% CI 1.84–8.95). Similarly, a 1 SD increase in the demographic (CogD) and biomarker (CogB) components increased the odds of being CDR-SB = 0 by 2.60 (95% CI 1.10–6.16) and 5.16 (95% CI 2.00–13.33), respectively.
Among symptomatic carriers, only the reserve and the biomarker components were significant. A 1 SD increase in CogR was associated with a 0.81-fold reduction in baseline CDR-SB score (95% CI 0.72–0.92), and a 1 SD increase in CogB was associated with a 0.60-fold reduction in CDR-SB (95% CI 0.50–0.71).
Results, "Cognitive Reserve, Clinical Status, and Disease Progression", first paragraph (the quotation keeps the original wording, including "multiple"):
For the asymptomatic DIAD carrier, using the zero-inflation model and after adjusting by EYO, 1 SD increase in the reserve component would multiply the odds of being CDR-SB = 0 by 3.66 (95% CI 2.11–6.38), while 1 SD increase in demographic component and biomarker component will multiple the odds by 1.24 (95% CI 0.76–2.02) and 7.77 (95% CI 3.40–17.80), respectively (Table 2).
Once symptomatic, only the reserve and the biomarker components were significant. One SD increase in reserve and in biomarker corresponds to a 0.83-fold (95% CI 0.75–0.92) and 0.60-fold (95% CI 0.52–0.69) decrease in baseline CDR-SB, respectively (Table 2).
Table 2 ("Relationships of Cognitive Components With Baseline Clinical Diagnosis in DIAD Carriers"), CDR-SB = 0 part (columns: characteristic | odds ratio | 95% CI):
Reserve component (cogR) | 3.66 | 2.11–6.38
Demographic component (cogD) | 1.24 | 0.76–2.02
Biomarker component (cogB) | 7.77 | 3.40–17.80
cogR:EYO | 1.07 | 1.02–1.13
cogD:EYO | 0.99 | 0.95–1.04
cogB:EYO | 1.06 | 0.98–1.14
Table 2, "Cognition decline" part:
Reserve component (cogR) | 0.83 | 0.75–0.92
Demographic component (cogD) | 0.94 | 0.86–1.04
Biomarker component (cogB) | 0.60 | 0.52–0.69
Methods, "Statistical Analysis":
Both cross-sectional and longitudinal models included fixed effects of EYO, the 3 cognitive components, and their interaction with EYO.
Results, same section, second paragraph. These are the longitudinal estimates in Table 3, a different model:
One SD increase in reserve, demographic, and biomarker components corresponds to a 3.29-fold (95% CI 2.06–5.26), 1.52-fold (95% CI 1.04–2.23), and 11.82-fold (95% CI 5.67–24.63) increase in odds, respectively.
Discussion, fourth paragraph:
Our study also indicates that demographic factors, for example., age, sex, education, and SES, play a significant role in maintaining asymptomatic status in individuals with DIAD.
However, after symptom onset, the demographic component's effect diminishes, with only the reserve (CogR) and biomarker (CogB) components remaining significantly associated with disease progression.
What the paper's own numbers show
Method. I set each estimate in the abstract beside the matching estimate in the Results text and Table 2. The two body sources print identical values. I checked each printed interval for internal consistency: a log-symmetric 95% CI should have its geometric midpoint, √(lower × upper), at the point estimate. From each interval I derived an approximate standard error on the log scale, SE = (ln upper − ln lower)/3.92, and a Wald z = ln(estimate)/SE with its two-sided p. These p-values are reconstructions from rounded printed values; the paper does not print p-values for these rows. I then tested three ways the two sets could both be right: rounding, evaluation of the component effects at a different EYO through the printed EYO interactions, and a mix-up with the longitudinal Table 3. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CogR, odds of CDR-SB = 0, asymptomatic | Abstract | 4.06 (1.84–8.95) | geometric midpoint 4.06; SE 0.404; z = 3.47, p ≈ 0.0005 |
| CogR, odds of CDR-SB = 0, asymptomatic | Results text; Table 2 | 3.66 (2.11–6.38) | midpoint 3.67; SE 0.282; z = 4.60, p < 0.0001; abstract ÷ Table 2 = 1.11 |
| CogD, odds of CDR-SB = 0, asymptomatic | Abstract | 2.60 (1.10–6.16) | midpoint 2.60; SE 0.439; z = 2.17, p ≈ 0.030; CI excludes 1 |
| CogD, odds of CDR-SB = 0, asymptomatic | Results text; Table 2 | 1.24 (0.76–2.02) | midpoint 1.24; SE 0.249; z = 0.86, p ≈ 0.39; CI includes 1; abstract ÷ Table 2 = 2.10 |
| CogB, odds of CDR-SB = 0, asymptomatic | Abstract | 5.16 (2.00–13.33) | midpoint 5.16; SE 0.484; z = 3.39, p ≈ 0.0007 |
| CogB, odds of CDR-SB = 0, asymptomatic | Results text; Table 2 | 7.77 (3.40–17.80) | midpoint 7.78; SE 0.422; z = 4.85, p < 0.0001; abstract ÷ Table 2 = 0.66 |
| CogR, ratio on CDR-SB, symptomatic | Abstract vs Results text; Table 2 | 0.81 (0.72–0.92) vs 0.83 (0.75–0.92) | SE 0.063 vs 0.052; ratio 0.98; both p < 0.001 |
| CogB, ratio on CDR-SB, symptomatic | Abstract vs Results text; Table 2 | 0.60 (0.50–0.71) vs 0.60 (0.52–0.69) | same point estimate; SE 0.089 vs 0.072 (different intervals) |
| Size of the gaps against rounding | all five pairs | — | OR-scale differences 0.40, 1.36, 2.61, 0.02 and 0.00, with interval limits differing in every pair. Two-decimal rounding can move a value by at most 0.01 |
| EYO shift that would turn Table 2 into the abstract value, OR × interactionᵉ | Table 2 interaction rows (1.07, 0.99, 1.06) | — | CogR +1.5 y, CogD −73.7 y, CogB −7.0 y: no single shift fits |
| Abstract vs longitudinal estimates | Table 3; Results paragraph 2 | 3.29, 1.52, 11.82 (asymptomatic); 0.92, 0.73 (symptomatic) | none equals the abstract's 4.06, 2.60, 5.16, 0.81, 0.60 |
| Abstract SE ÷ Table 2 SE | derived | — | 1.43, 1.76, 1.15 (asymptomatic); 1.20, 1.24 (symptomatic): every abstract interval is wider |
Each set is internally consistent: every interval centres on its own point estimate. So the abstract values are not a body estimate printed with the wrong interval, or the reverse. The two sets are distinct estimates of what the paper presents as the same quantity.
Figure
The left panel lists the abstract's five estimates (95% CI) and the right panel the matching values in the Results text and Table 2, which print identical numbers. Both panels quote the model description each source gives. Each connector gives the ratio of the abstract estimate to the Table 2 estimate, computed in recompute.py.txt. Only the demographic component (CogD) in asymptomatic carriers changes significance: its interval excludes 1 in the abstract and includes 1 in Table 2. The other four pairs differ in point estimate, in interval, or in both, but keep the same direction and significance.
The same ten estimates on one logarithmic axis. Orange marks the abstract and blue the Results text and Table 2, each labelled with its printed value; the thin vertical line marks 1 (no association). The plot shows the sizes of the gaps that the text panels state as ratios. The biomarker odds ratio is a third lower in the abstract. The Table 2 interval for the demographic component crosses 1, while the abstract's does not.
Why it matters
The abstract is the part of the paper most readers see and the part that indexing services reproduce. In it, the demographic component (age, education, sex, race, socioeconomic status and vascular burden score, per the Methods) is presented as significantly associated with being clinically unimpaired at baseline: OR 2.60, 95% CI 1.10–6.16. The paper's own cross-sectional model gives 1.24 (0.76–2.02), an approximate p of 0.39, so the association the abstract reports is not supported by the paper's own analysis. The Discussion says that demographic factors "play a significant role in maintaining asymptomatic status". That sentence matches the abstract's figure. Among the paper's own results, it is supported only by the separate longitudinal estimate in Table 3 (1.52, 1.04–2.23; approximate p 0.031), not by the baseline Table 2 that the abstract describes.
The relative sizes of the components also change. In the abstract, the reserve odds ratio (4.06) is close to the biomarker odds ratio (5.16). In Table 2, the biomarker effect (7.77) is about twice the reserve effect (3.66). A reader comparing the protective weight of cognitive reserve with pathology burden would draw a different picture from each source.
What is not affected: the reserve component is significant in the same direction in every version (abstract 4.06, Table 2 3.66, Table 3 3.29), so the paper's central conclusion, that higher cognitive reserve goes with remaining unimpaired, does not depend on which set is correct. The two symptomatic-carrier estimates differ only slightly and stay significant.
Explanations considered
- Rounding or transcription of one digit. This does not explain the numbers. All five pairs differ, by up to 2.61 on the odds-ratio scale, and every interval in the abstract centres exactly on its own point estimate. A single mistyped value would leave the rest of the set unchanged and would usually break this internal consistency.
- A different reference EYO. Because the model contains component × EYO interactions, a component's odds ratio depends on the EYO at which it is evaluated. If the abstract used a different centring, one shift would have to convert all three Table 2 estimates to the abstract values. The printed interactions require +1.5 years for CogR, −73.7 years for CogD and −7.0 years for CogB. These shifts are incompatible with one another, and −73.7 years lies outside any plausible EYO range, so re-centring does not account for the gap.
- The abstract quotes the longitudinal model. It does not. Table 3 gives 3.29, 1.52 and 11.82 for asymptomatic carriers and 0.92 and 0.73 for symptomatic carriers; none equals an abstract value.
- Body and table disagree, so the abstract follows one of them. They do not disagree. The Results text cites Table 2 explicitly and prints the same five values, and the PMC HTML table shows the same rows. Misaligned cells, a missing minus sign or a lost superscript cannot apply: every value is a positive ratio, and the table and the prose agree.
- A conversion artefact in the archived text. This is ruled out. The quoted passages match the publisher-deposited JATS full text, read both as the live PMC page and as the Europe PMC XML.
- An earlier analysis. This is offered as context only and is not a conclusion about how the discrepancy arose. The abstract's estimates are identical, at the earlier abstract's printed precision, to those of a 2024 conference abstract on the same topic (Alzheimer's & Dementia 2024, doi:10.1002/alz.086295, PMC11710562). That abstract reports aMC n = 204 and sMC n = 115, against 284 and 155 here, and adjusts "by mutation mean AAO". Under simple 1/√n scaling, its standard errors would be about 1.18 and 1.16 times those of the larger sample. Every abstract interval is indeed wider than its Table 2 counterpart (1.15 to 1.76 times), which agrees with this, although it does not prove it. Whichever analysis produced them, the abstract's values are not the estimates that the paper's Results and Table 2 report for the 284/155-carrier sample.
Status
- Version of record. The publisher page and PDF on neurology.org returned HTTP 403 to automated access, and the PMC PDF could not be fetched headlessly. The quoted passages were therefore checked in the publisher-deposited JATS full text, read two ways: the live PMC HTML (pmc.ncbi.nlm.nih.gov/articles/PMC13067375/) and the Europe PMC fullTextXML. Both match the archived text used here exactly, and every quote above was re-checked against it by
recompute.py.txt. - Supplements. The Europe PMC supplementary-files bundle holds only the two figure images. The text cites eFigures 1–2 and eTable 1, but the publisher's supplement page returned 403. By their descriptions, these cover the tertile plots and the longitudinal CogR analyses, not the baseline zero-part estimates, so they would not decide which set is correct. Data are available on request; the paper states that the component-decomposition code is public.
- Corrections. Crossref lists no updates, and PMC and Europe PMC hold no correction, erratum, expression of concern or retraction notice for this DOI.
- PubPeer. Direct PubPeer search returned HTTP 403. A web search for the title together with PubPeer found no thread.
- Secondary coverage. A news summary of the paper (Neurology Advisor) quotes the body values (OR 3.66; 0.83).
- Sources: the PMC archive of PMC13067375; Crossref and PMC/Europe PMC notice checks for doi:10.1212/WNL.0000000000214804; Europe PMC full text of PMC11710562 (doi:10.1002/alz.086295). Status as of 2026-10-23.
Suggested correction
The authors should confirm which analysis is final and bring the abstract into line with it. If the Results text and Table 2 are correct, the abstract's Result paragraph should read:
… a 1 SD increase in the reserve component (CogR) was associated with a 3.66-fold increase in the odds of being clinically unimpaired (CDR-SB = 0; 95% CI 2.11–6.38). A 1 SD increase in the biomarker component (CogB) increased the odds of being CDR-SB = 0 by 7.77 (95% CI 3.40–17.80); the demographic component (CogD) was not significantly associated (1.24, 95% CI 0.76–2.02). … A 1 SD increase in CogR was associated with a 0.83-fold reduction in baseline CDR-SB score (95% CI 0.75–0.92), and a 1 SD increase in CogB was associated with a 0.60-fold reduction in CDR-SB (95% CI 0.52–0.69).
The Discussion sentence on demographic factors playing "a significant role in maintaining asymptomatic status" should then name the result that supports it, the longitudinal Table 3 estimate, or else be qualified. If instead the abstract's values are the intended ones, Table 2 and the Results text need correcting, along with an explanation of the sample and adjustment that produced them.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13067375 (Neurology 2026, doi:10.1212/WNL.0000000000214804)
# Abstract vs Results text / Table 2: baseline estimates for the cognitive components.
#
# Run: python recompute.py.txt (prints every number in writeup.md,
# writes fig1.* and fig2.*)
# python recompute.py.txt --candidates (also writes five candidate figure designs,
# candidates/cand_A..E.{svg,png})
# Needs numpy, scipy and matplotlib. Figures are written next to this file.
#
# Inputs
# * Every estimate below is transcribed from the article as archived in PubMed Central
# (PMC13067375): Abstract (Results paragraph), Results section "Cognitive Reserve,
# Clinical Status, and Disease Progression", Table 2 and Table 3.
# * For context only (not part of the paper): the 2024 conference abstract
# "Cognitive reserve influences symptom onset and longitudinal decline in Dominantly
# Inherited Alzheimer's Disease", Alzheimer's & Dementia 2024, doi:10.1002/alz.086295,
# PMC11710562, as given in its Europe PMC full-text XML.
# * When run inside the /home/ubuntu/papers repo, the script re-checks each quote against the
# archived full text with src/show.py (skipped elsewhere).
# * Approximate Wald statistics assume each printed 95% CI is symmetric on the log scale:
# SE(log OR) = (ln hi - ln lo) / (2 x 1.959964), z = ln(OR) / SE, p = 2 x (1 - Phi(|z|)).
# These are reconstructions from rounded printed values, not values the paper reports.
import math, os, subprocess, sys
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
# ------------------------------------------------------------------ quotes
QUOTES = {
"abs_n": ("Abstract, Result",
"A total of 710 Dominantly Inherited Alzheimer Network (DIAN) participants were included in "
"the analysis, comprising 271 non-DIAD carriers (nMC), 284 asymptomatic DIAD carriers (aMC), "
"and 155 symptomatic DIAD carriers."),
"abs_aMC": ("Abstract, Result",
"In asymptomatic carriers, using a zero-inflation model adjusted for EYO showed that a 1 SD "
"increase in the reserve component (CogR) was associated with a 4.06-fold increase in the odds "
"of being clinically unimpaired (CDR-SB = 0; 95% CI 1.84–8.95). Similarly, a 1 SD increase in "
"the demographic (CogD) and biomarker (CogB) components increased the odds of being CDR-SB = 0 "
"by 2.60 (95% CI 1.10–6.16) and 5.16 (95% CI 2.00–13.33), respectively."),
"abs_sMC": ("Abstract, Result",
"Among symptomatic carriers, only the reserve and the biomarker components were significant. "
"A 1 SD increase in CogR was associated with a 0.81-fold reduction in baseline CDR-SB score "
"(95% CI 0.72–0.92), and a 1 SD increase in CogB was associated with a 0.60-fold reduction in "
"CDR-SB (95% CI 0.50–0.71)."),
"res_aMC": ("Results, 'Cognitive Reserve, Clinical Status, and Disease Progression', para 1",
"For the asymptomatic DIAD carrier, using the zero-inflation model and after adjusting by EYO, "
"1 SD increase in the reserve component would multiply the odds of being CDR-SB = 0 by 3.66 "
"(95% CI 2.11–6.38), while 1 SD increase in demographic component and biomarker component will "
"multiple the odds by 1.24 (95% CI 0.76–2.02) and 7.77 (95% CI 3.40–17.80), respectively (Table 2)."),
"res_sMC": ("Results, 'Cognitive Reserve, Clinical Status, and Disease Progression', para 1",
"Once symptomatic, only the reserve and the biomarker components were significant. One SD "
"increase in reserve and in biomarker corresponds to a 0.83-fold (95% CI 0.75–0.92) and "
"0.60-fold (95% CI 0.52–0.69) decrease in baseline CDR-SB, respectively (Table 2)."),
"res_long": ("Results, 'Cognitive Reserve, Clinical Status, and Disease Progression', para 2",
"One SD increase in reserve, demographic, and biomarker components corresponds to a 3.29-fold "
"(95% CI 2.06–5.26), 1.52-fold (95% CI 1.04–2.23), and 11.82-fold (95% CI 5.67–24.63) increase "
"in odds, respectively."),
"methods": ("Methods, Statistical Analysis",
"Both cross-sectional and longitudinal models included fixed effects of EYO, the 3 cognitive "
"components, and their interaction with EYO."),
"disc_demo": ("Discussion, paragraph 4",
"Our study also indicates that demographic factors, for example., age, sex, education, and SES, "
"play a significant role in maintaining asymptomatic status in individuals with DIAD."),
"disc_after": ("Discussion, paragraph 4",
"However, after symptom onset, the demographic component's effect diminishes, with only the "
"reserve (CogR) and biomarker (CogB) components remaining significantly associated with disease "
"progression."),
"t2_cogR": ("Table 2, CDR-SB = 0 part", "Reserve component (cogR) | 3.66 | 2.11–6.38"),
"t2_cogD": ("Table 2, CDR-SB = 0 part", "Demographic component (cogD) | 1.24 | 0.76–2.02"),
"t2_cogB": ("Table 2, CDR-SB = 0 part", "Biomarker component (cogB) | 7.77 | 3.40–17.80"),
"t2_int": ("Table 2, CDR-SB = 0 part",
"cogR:EYO | 1.07 | 1.02–1.13\ncogD:EYO | 0.99 | 0.95–1.04\ncogB:EYO | 1.06 | 0.98–1.14"),
"t2_dec": ("Table 2, Cognition decline part",
"Reserve component (cogR) | 0.83 | 0.75–0.92\nDemographic component (cogD) | 0.94 | 0.86–1.04\n"
"Biomarker component (cogB) | 0.60 | 0.52–0.69"),
"t3_dec": ("Table 3, Longitudinal decline part",
"Reserve component (cogR) | 0.92 | 0.85–1.01\nDemographic component (cogD) | 0.94 | 0.87–1.03\n"
"Biomarker component (cogB) | 0.73 | 0.64–0.83"),
}
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.")
return
try:
full = subprocess.run([py, "src/show.py", "PMC13067375", "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e: # pragma: no cover
print(f"Quote check: could not load archived text ({e}); skipped.")
return
nrm = lambda s: " ".join(s.split())
full_n = nrm(full)
print("Quote check against the PMC-archived full text (occurrences; 0 = not found):")
for key, (loc, q) in QUOTES.items():
print(f" {key:10s} {full_n.count(nrm(q))} ({loc})")
# ------------------------------------------------------------------ estimates (OR, lo, hi)
ROWS = ["aMC CogR", "aMC CogD", "aMC CogB", "sMC CogR", "sMC CogB"]
ABSTRACT = {"aMC CogR": (4.06, 1.84, 8.95), "aMC CogD": (2.60, 1.10, 6.16),
"aMC CogB": (5.16, 2.00, 13.33), "sMC CogR": (0.81, 0.72, 0.92),
"sMC CogB": (0.60, 0.50, 0.71)}
TABLE2 = {"aMC CogR": (3.66, 2.11, 6.38), "aMC CogD": (1.24, 0.76, 2.02),
"aMC CogB": (7.77, 3.40, 17.80), "sMC CogR": (0.83, 0.75, 0.92),
"sMC CogD": (0.94, 0.86, 1.04), "sMC CogB": (0.60, 0.52, 0.69)}
RESULTS_TEXT = {k: TABLE2[k] for k in ROWS} # the Results prose prints the same values
TABLE3 = {"aMC CogR": (3.29, 2.06, 5.26), "aMC CogD": (1.52, 1.04, 2.23),
"aMC CogB": (11.82, 5.67, 24.63), "sMC CogR": (0.92, 0.85, 1.01),
"sMC CogD": (0.94, 0.87, 1.03), "sMC CogB": (0.73, 0.64, 0.83)}
T2_INTERACTION = {"CogR": 1.07, "CogD": 0.99, "CogB": 1.06} # cogX:EYO, CDR-SB = 0 part
# Context only: 2024 conference abstract (aMC n = 204, sMC n = 115), printed precision
CONF2024 = {"aMC CogR": (4.06, 1.8, 8.9), "aMC CogD": (2.6, 1.1, 6.2),
"aMC CogB": (5.16, 2.0, 13.3), "sMC CogR": (0.81, 0.72, 0.92),
"sMC CogB": (0.60, 0.50, 0.71)}
N_PAPER = {"aMC": 284, "sMC": 155}
N_CONF = {"aMC": 204, "sMC": 115}
def se(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z975)
def wald(o, lo, hi):
s = se(lo, hi)
z = math.log(o) / s
return math.sqrt(lo * hi), s, z, 2 * norm.sf(abs(z))
def report():
print("\n=== 1. Internal consistency and approximate Wald statistics ===")
print(f"{'source':14s} {'estimate':9s} {'OR':>6s} {'95% CI':>13s} {'geomean':>8s} "
f"{'SE':>6s} {'z':>6s} {'p':>8s} excl.1")
for name, src in [("Abstract", ABSTRACT), ("Table 2", TABLE2), ("Table 3", TABLE3)]:
for k, (o, lo, hi) in src.items():
gm, s, z, p = wald(o, lo, hi)
print(f"{name:14s} {k:9s} {o:6.2f} {lo:6.2f}-{hi:<6.2f} {gm:8.2f} {s:6.3f} "
f"{z:6.2f} {p:8.4f} {lo > 1 or hi < 1}")
print("\n=== 2. Abstract vs Results text / Table 2 ===")
for k in ROWS:
a, t = ABSTRACT[k], TABLE2[k]
print(f"{k:9s} abstract {a[0]:.2f} ({a[1]:.2f}-{a[2]:.2f}) | Table 2 {t[0]:.2f} "
f"({t[1]:.2f}-{t[2]:.2f}) | ratio abstract/Table2 = {a[0]/t[0]:.3f} | "
f"SE ratio = {se(*a[1:]) / se(*t[1:]):.2f} | identical? {a == t}")
print("Results text equals Table 2 for all five:", all(RESULTS_TEXT[k] == TABLE2[k] for k in ROWS))
print("\n=== 3. Rounding: largest change the printing precision could cause ===")
for k in ROWS:
a, t = ABSTRACT[k], TABLE2[k]
print(f"{k:9s} |abstract - Table 2| = {abs(a[0]-t[0]):.2f} on the OR scale "
f"(two-decimal rounding of both: at most 0.01)")
print("\n=== 4. Re-centring EYO: OR_at_e = OR_Table2 x interaction^e ===")
for c in ["CogR", "CogD", "CogB"]:
k = "aMC " + c
e = math.log(ABSTRACT[k][0] / TABLE2[k][0]) / math.log(T2_INTERACTION[c])
print(f"{c}: interaction {T2_INTERACTION[c]:.2f}; EYO shift needed to reach abstract value "
f"= {e:+.1f} y")
print("\n=== 5. Does the abstract match Table 3 (longitudinal)? ===")
for k in ROWS:
print(f"{k:9s} abstract {ABSTRACT[k][0]:.2f} vs Table 3 {TABLE3[k][0]:.2f}: "
f"equal? {ABSTRACT[k][0] == TABLE3[k][0]}")
print("\n=== 6. Context: abstract vs 2024 conference abstract, at the conference precision ===")
def rnd(x, ref):
d = len(f"{ref}".split(".")[1]) if "." in f"{ref}" else 0
return round(x + 1e-9, d)
for k in ROWS:
a, c = ABSTRACT[k], CONF2024[k]
same_pt = rnd(a[0], c[0]) == c[0]
ci_gap = max(abs(a[1] - c[1]), abs(a[2] - c[2]))
print(f"{k:9s} Neurology abstract {a[0]:.2f} ({a[1]:.2f}-{a[2]:.2f}) | conference "
f"{c[0]} ({c[1]}-{c[2]}) | point estimate same? {same_pt} | largest CI-limit "
f"difference {ci_gap:.2f} (within 1-decimal printing: {ci_gap <= 0.05 + 1e-9})")
for g in ["aMC", "sMC"]:
print(f"{g}: n = {N_CONF[g]} (conference) vs {N_PAPER[g]} (paper); "
f"sqrt(n ratio) = {math.sqrt(N_PAPER[g] / N_CONF[g]):.2f}")
# ------------------------------------------------------------------ style
BG, TXT, TXT2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT2, "ytick.color": TXT2,
"axes.edgecolor": "#b9b8b3",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
LABEL = {"aMC CogR": "Reserve (CogR)", "aMC CogD": "Demographic (CogD)",
"aMC CogB": "Biomarker (CogB)", "sMC CogR": "Reserve (CogR)", "sMC CogB": "Biomarker (CogB)"}
def save(fig, stem, sub=""):
d = os.path.join(HERE, sub)
os.makedirs(d, exist_ok=True)
fig.savefig(os.path.join(d, stem + ".svg"))
fig.savefig(os.path.join(d, stem + ".png"), dpi=200)
plt.close(fig)
print(f"wrote {os.path.join(sub, stem)}.svg and .png")
def ci_txt(t):
return f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})"
def log_axis(ax, lo, hi, ticks):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# ------------------------------------------------------------------ A: text panels
def cand_A(stem="cand_A", sub="candidates"):
W, H = 7.2, 5.8
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
px = {"L": 0.10, "R": 4.40}; pw = 2.70
top, bot = 5.68, 0.82
def panel(x, title, sub, accent, src, model):
ax.add_patch(FancyBboxPatch((x, bot), pw, top - bot, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=accent, lw=1.5))
ax.text(x + 0.12, top - 0.12, title, fontsize=11.5, fontweight="bold", va="top")
ax.text(x + 0.12, top - 0.42, sub, fontsize=10, va="top", color=accent)
ax.text(x + 0.12, top - 0.70, model, fontsize=10, va="top", color=TXT2, fontstyle="italic",
linespacing=1.15)
ys = {}
y = top - 1.62
for grp, keys in [("Asymptomatic: odds of CDR-SB = 0", ROWS[:3]),
("Symptomatic: ratio on CDR-SB", ROWS[3:])]:
ax.text(x + 0.12, y + 0.06, grp, fontsize=10, color=TXT2, va="bottom")
y -= 0.30
for k in keys:
ax.text(x + 0.12, y, LABEL[k].split(" (")[1][:-1], fontsize=10, va="center",
fontweight="bold")
ax.text(x + pw - 0.12, y, ci_txt(src[k]), fontsize=10, va="center", ha="right",
color=accent, fontweight="bold")
ys[k] = y
y -= 0.62
y -= 0.0
return ys
ysL = panel(px["L"], "Abstract", "Result paragraph", ORANGE, ABSTRACT,
"“…using a zero-inflation model\nadjusted for EYO…”")
panel(px["R"], "Results text and Table 2", "both print the same values", BLUE, TABLE2,
"“…using the zero-inflation model\nand after adjusting by EYO…”")
notes = {}
for k in ROWS:
a, t = ABSTRACT[k], TABLE2[k]
r = a[0] / t[0]
notes[k] = f"×{r:.2f}" if abs(r - 1) > 1e-9 else "×1.00, CI differs"
notes["aMC CogD"] = f"×{ABSTRACT['aMC CogD'][0]/TABLE2['aMC CogD'][0]:.2f}"
x0, x1 = px["L"] + pw + 0.04, px["R"] - 0.04
for k in ROWS:
y = ysL[k]
strong = k == "aMC CogD"
ax.add_patch(FancyArrowPatch((x0, y), (x1, y), arrowstyle="<|-|>", mutation_scale=11,
lw=2 if strong else 1.2, color=TXT if strong else GRAY))
ax.text((x0 + x1) / 2, y + 0.06, notes[k], ha="center", va="bottom", fontsize=10,
color=TXT if strong else TXT2, fontweight="bold" if strong else "normal",
linespacing=1.1)
if strong:
ax.text((x0 + x1) / 2, y - 0.07, "CI excludes 1\n→ includes 1", ha="center", va="top",
fontsize=10, color=TXT, fontweight="bold", linespacing=1.05)
ax.text((x0 + x1) / 2, top - 0.12, "conflict", ha="center", va="top", fontsize=11, fontweight="bold")
ax.text((x0 + x1) / 2, top - 0.45, "same model\nand outcome;\nall 5 estimates\ndiffer",
ha="center", va="top", fontsize=10, color=TXT2, linespacing=1.15)
ax.text(0.12, 0.42,
"Estimates (95% CI) per 1 SD of each component, as printed. Ratios abstract ÷ Table 2 are\n"
"computed in recompute.py.txt. Only the demographic component changes significance; the\n"
"symptomatic CogB estimate (0.60) is the same point value with a different interval.",
fontsize=10, color=TXT2, va="center", linespacing=1.25)
save(fig, stem, sub)
# ------------------------------------------------------------------ B: forest plot
def cand_B(stem="cand_B", sub="candidates"):
fig, ax = plt.subplots(figsize=(7.2, 4.8))
fig.subplots_adjust(left=0.27, right=0.78, top=0.90, bottom=0.13)
ypos, labels, y = {}, [], 0
heads = []
for grp, keys in [("Asymptomatic carriers\nodds of CDR-SB = 0", ROWS[:3]),
("Symptomatic carriers\nfold change in CDR-SB", ROWS[3:])]:
heads.append((y, grp)); y += 0.9
for k in keys:
ypos[k] = y; y += 1.25
y += 0.1
for k in ROWS:
for src, col, off, name in [(ABSTRACT, ORANGE, -0.17, "Abstract"),
(TABLE2, BLUE, 0.17, "Results text and Table 2")]:
o, lo, hi = src[k]
yy = ypos[k] + off
ax.plot([lo, hi], [yy, yy], color=col, lw=2, solid_capstyle="butt")
ax.plot(o, yy, "o", ms=8, color=col, label=name if k == ROWS[0] else None)
ax.text(1.03, yy, ci_txt(src[k]), transform=ax.get_yaxis_transform(), va="center",
fontsize=10, color=col, fontweight="bold")
ax.axvline(1, color=TXT2, lw=0.8)
log_axis(ax, 0.4, 25, [0.5, 1, 2, 5, 10, 20])
ax.set_yticks([ypos[k] for k in ROWS]); ax.set_yticklabels([LABEL[k] for k in ROWS], fontsize=10,
color=TXT)
for yh, g in heads:
ax.text(-0.36, yh + 0.1, g, transform=ax.get_yaxis_transform(), fontsize=10, color=TXT2,
va="center", ha="left", fontweight="bold", linespacing=1.1)
ax.set_ylim(y - 0.2, -0.6)
ax.set_xlabel("Odds ratio or fold change per 1 SD (log scale); thin line = 1 (no association)")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, 1.0), ncol=2, frameon=False, fontsize=10)
ax.text(0.42, ypos["aMC CogD"] + 0.62, "Table 2 CogD interval crosses 1", fontsize=10, color=TXT,
va="center", fontweight="bold")
save(fig, stem, sub)
# ------------------------------------------------------------------ C: dumbbell (shift)
def cand_C(stem="cand_C", sub="candidates"):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.subplots_adjust(left=0.34, right=0.97, top=0.86, bottom=0.17)
ys = {k: i + (0.4 if i >= 3 else 0) for i, k in enumerate(ROWS)}
for k in ROWS:
a, t = ABSTRACT[k][0], TABLE2[k][0]
y = ys[k]
if a != t:
ax.annotate("", xy=(t, y), xytext=(a, y),
arrowprops=dict(arrowstyle="-", color=GRAY, lw=2))
ax.plot(a, y, "o", ms=9, color=ORANGE, label="Abstract" if k == ROWS[0] else None, zorder=3)
ax.plot(t, y, "D", ms=8, color=BLUE, label="Results text and Table 2" if k == ROWS[0] else None,
zorder=3)
lbl = f"×{a/t:.2f}" if a != t else "same point value; CI differs"
ax.text(max(a, t) * 1.2, y, lbl, va="center", fontsize=10,
color=TXT if k == "aMC CogD" else TXT2,
fontweight="bold" if k == "aMC CogD" else "normal")
ax.axvline(1, color=TXT2, lw=0.8)
log_axis(ax, 0.4, 25, [0.5, 1, 2, 5, 10, 20])
ax.set_yticks([ys[k] for k in ROWS])
ax.set_yticklabels([("Asympt. " if k.startswith("a") else "Sympt. ") + LABEL[k] for k in ROWS],
color=TXT)
ax.set_ylim(max(ys.values()) + 0.6, -0.6)
ax.set_xlabel("Point estimate per 1 SD (log scale); thin line = 1")
ax.legend(loc="lower center", bbox_to_anchor=(0.45, 1.0), ncol=2, frameon=False)
save(fig, stem, sub)
# ------------------------------------------------------------------ D: provenance matrix
def cand_D(stem="cand_D", sub="candidates"):
W, H = 7.2, 4.3
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
cols = [("Neurology 2026\nabstract", ABSTRACT, "A"),
("2024 conference\nabstract*", CONF2024, "A"),
("Neurology 2026\nResults text\nand Table 2", TABLE2, "T"),
("Neurology 2026\nTable 3\n(longitudinal)", TABLE3, "O")]
x0, cw, rh, ytop = 1.27, 1.47, 0.42, 3.22
for j, (h, _, _) in enumerate(cols):
ax.text(x0 + j * cw + cw / 2, ytop + 0.08, h, ha="center", va="bottom", fontsize=10,
fontweight="bold", linespacing=1.1)
fill = {"A": "#fbe1d6", "T": "#d9e7f8", "O": "#f0efec"}
edge = {"A": ORANGE, "T": BLUE, "O": "#c9c8c3"}
for i, k in enumerate(ROWS):
y = ytop - (i + 1) * rh - (0.14 if i >= 3 else 0)
ax.text(0.10, y + rh / 2, ("Asympt. " if i < 3 else "Sympt. ") + k.split()[1], va="center",
fontsize=10, fontweight="bold")
for j, (_, src, grp) in enumerate(cols):
o, lo, hi = src[k]
ax.add_patch(Rectangle((x0 + j * cw + 0.03, y + 0.03), cw - 0.06, rh - 0.06,
fc=fill[grp], ec=edge[grp], lw=1))
txt = f"{o:g} ({lo:g}–{hi:g})" if src is CONF2024 else ci_txt(src[k])
ax.text(x0 + j * cw + cw / 2, y + rh / 2, txt, ha="center", va="center", fontsize=10)
yl = 0.74
for xx, grp, t in [(0.10, "A", "matches the Neurology abstract"),
(3.30, "T", "the paper's body"), (5.25, "O", "a different model")]:
ax.add_patch(Rectangle((xx, yl - 0.09), 0.22, 0.18, fc=fill[grp], ec=edge[grp], lw=1))
ax.text(xx + 0.30, yl, t, va="center", fontsize=10, color=TXT2)
ax.text(0.10, 0.30, "*Alzheimer's & Dementia 2024, doi:10.1002/alz.086295 (aMC n = 204, sMC n = 115;\n"
"adjusted by mutation mean AAO). The Neurology paper reports aMC n = 284, sMC n = 155.",
fontsize=10, color=TXT2, va="center", linespacing=1.2)
save(fig, stem, sub)
# ------------------------------------------------------------------ E: Wald z
def cand_E(stem="cand_E", sub="candidates"):
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.subplots_adjust(left=0.34, right=0.97, top=0.86, bottom=0.17)
ys = {k: i + (0.4 if i >= 3 else 0) for i, k in enumerate(ROWS)}
ax.axvspan(-Z975, Z975, color="#efeeea", zorder=0)
ax.axvline(0, color=TXT2, lw=0.8)
for k in ROWS:
y = ys[k]
for src, col, mk, off, name in [(ABSTRACT, ORANGE, "o", -0.14, "Abstract"),
(TABLE2, BLUE, "D", 0.14, "Results text and Table 2")]:
_, _, z, p = wald(*src[k])
ax.plot(z, y + off, mk, ms=8, color=col, label=name if k == ROWS[0] else None)
ax.text(z + (0.25 if z > 0 else -0.25), y + off, f"p≈{p:.3f}" if p >= 0.001 else "p<0.001",
va="center", ha="left" if z > 0 else "right", fontsize=9.5, color=col)
ax.text(0, -0.45, "|z| < 1.96\n(p > 0.05)", ha="center", va="top", fontsize=10,
color=TXT2, linespacing=1.05)
ax.set_xlim(-10.5, 7.5)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_yticks([ys[k] for k in ROWS])
ax.set_yticklabels([("Asympt. " if k.startswith("a") else "Sympt. ") + LABEL[k] for k in ROWS],
color=TXT)
ax.set_ylim(max(ys.values()) + 0.6, -1.0)
ax.set_xlabel("Approximate Wald z = ln(estimate) / SE, SE from the printed 95% CI")
ax.legend(loc="lower center", bbox_to_anchor=(0.45, 1.0), ncol=2, frameon=False)
save(fig, stem, sub)
if __name__ == "__main__":
check_quotes()
report()
print()
cand_A("fig1", "")
cand_B("fig2", "")
if "--candidates" in sys.argv:
for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
f()
Abstract reports 6-month BMI difference versus prednisone as 12-month result, and reuses it for height
Neurology (2026) · doi:10.1212/WNL.0000000000214756 · PMC13225239
Summary
The abstract gives the 12-month least-squares-mean (LSM) difference in BMI Z-score between vamorolone 6 mg/kg/d and prednisone as 0.57 [95% CI 0.24–0.90]. In the paper's Table 3, those three numbers are the 6-month value. The 12-month value is 0.24 (−0.04 to 0.52; p = 0.09), and its confidence interval includes zero. The abstract then gives the same triple, 0.57 [0.24–0.90], as the 12-month height Z-score difference versus prednisone, where Table 3 reports 0.44 (0.25 to 0.64). The deflazacort values in the same two sentences match Table 3 exactly, and the Results text and the Discussion agree with Table 3. So the error is confined to the two prednisone figures in the abstract. Those figures make the 12-month BMI excess on vamorolone look 2.4 times larger than the paper's own estimate, and statistically significant when it is not.
What the paper reports
Abstract, Results (BMI):
Body mass index (BMI) Z-scores increased in all groups, greatest with vamorolone (12-month LSM difference [95% CI]: vamorolone vs prednisone 0.57 [0.24–0.90]; vamorolone vs deflazacort 0.29 [0.03–0.56])
Abstract, Results (height), same sentence:
(12-month LSM difference in height Z-scores [95% CI]: vamorolone vs prednisone 0.57 [0.24–0.90]; vamorolone vs deflazacort 0.72 [0.53–0.91])
Table 3, "Comparisons of LSM Changes From Baseline Between Treatments at 6 and 12 Months for Anthropometric Outcomes Performed Using Weighted Data". Columns: n; LSM difference; 95% CI; p, at 6 mo and then at 12 mo:
BMI Z-score | Vam 6/Pred | 26/48 | 0.57; 0.24, 0.90 | 0.009 | 26/48 | 0.24; -0.04, 0.52 | 0.09
BMI Z-score | Vam 6/DFZ | 26/54 | 0.32; 0.00, 0.64 | 0.05 | 26/51 | 0.29; 0.03, 0.56 | 0.03
Height Z-score | Vam 6/Pred | 26/48 | 0.25; 0.14, 0.36 | <0.0001 | 26/48 | 0.44; 0.25, 0.64 | <0.0001
Height Z-score | Vam 6/DFZ | 26/51 | 0.50; 0.39, 0.61 | <0.0001 | 26/51 | 0.72; 0.53, 0.91 | <0.0001
Results, "Anthropometric Analyses" (labels 0.57 as the 6-month value):
LSM estimates of BMI Z-scores increased in all treatment groups at 12 months, with a decrease in BMI observed for prednisone at 6 months (prednisone vs vamorolone 6 mg/kg/d at 6 months, BMI Z-score LSM difference 0.57; 95% CI 0.24–0.90; p = 0.009) and a smaller increase for deflazacort compared with vamorolone at 12 months (LSM difference 0.29; 95% CI 0.03–0.56; p = 0.03) (Table 3; Figure 3A).
Discussion, paragraph 4:
To this end, we found that the increase in the BMI Z-score from baseline was greater on vamorolone than prednisone at 6 but not at 12 months.
What the paper's own numbers show
Method: each abstract triple (estimate, lower, upper) was compared for exact equality with all 12 cells of Table 3. Each cell was then checked for internal consistency. The standard error was recovered from the CI width as (upper − lower) / (2 × 1.96), and the two-sided p was recomputed from z = estimate / SE under a normal approximation. This is a check only: the paper's mixed model uses Kenward-Roger degrees of freedom, which give slightly larger p values. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| BMI Z-score, vamorolone 6 vs prednisone, "12-month" | Abstract | 0.57 [0.24–0.90] | Table 3 at 12 mo: 0.24 (−0.04, 0.52). The triple equals only the 6-mo BMI cell |
| BMI Z-score, vamorolone 6 vs prednisone, 12 mo | Table 3 | 0.24; −0.04, 0.52; p = 0.09 | SE 0.143, z 1.68, p = 0.093 (consistent); CI includes 0 |
| BMI Z-score, vamorolone 6 vs prednisone, 6 mo | Table 3; Results text | 0.57; 0.24, 0.90; p = 0.009 | SE 0.168, z 3.39, p ≈ 0.0007 (see note) |
| Abstract minus Table 3, BMI vs prednisone, 12 mo | Abstract vs Table 3 | 0.57 vs 0.24 | Difference 0.33; ratio 2.38 |
| Height Z-score, vamorolone 6 vs prednisone, "12-month" | Abstract | 0.57 [0.24–0.90] | Table 3 at 12 mo: 0.44 (0.25, 0.64), and at 6 mo: 0.25. The triple equals the 6-mo BMI cell |
| Height Z-score, vamorolone 6 vs prednisone, 12 mo | Table 3 | 0.44; 0.25, 0.64; p < 0.0001 | SE 0.099, z 4.42, p ≈ 1e−5 (consistent); abstract overstates by 0.13 |
| BMI Z-score, vamorolone 6 vs deflazacort, 12 mo | Abstract; Table 3 | 0.29 [0.03–0.56]; p = 0.03 | Abstract equals Table 3 12-mo cell; p = 0.032 (consistent) |
| Height Z-score, vamorolone 6 vs deflazacort, 12 mo | Abstract; Table 3 | 0.72 [0.53–0.91]; p < 0.0001 | Abstract equals Table 3 12-mo cell; p ≈ 1e−13 (consistent) |
Every Table 3 CI midpoint equals its point estimate to rounding (for example 0.240 for 0.24, and 0.445 for 0.44). The table's columns are therefore not misaligned. Of the four comparisons the abstract labels "12-month", the two deflazacort ones match their 12-month cells and the two prednisone ones match neither 12-month cell.
Note on a separate, minor point: for the 6-month BMI cell, the CI 0.24–0.90 implies p ≈ 0.0007 under a normal approximation. With a t reference distribution, it implies p ≤ 0.003 for any df of 10 or more. The reported p = 0.009 would need about 4 degrees of freedom. This affects only the precision of a p value whose result is significant either way, and it is not the subject of this write-up.
Figure
Figure 1. The two abstract values labelled "12-month" for vamorolone 6 mg/kg/d versus prednisone (left, orange) set against Table 3 of the same paper (right, blue). Solid arrows join identical numbers: both abstract values (BMI and height) equal the 6-month BMI cell, 0.57; 0.24, 0.90. Dashed arrows lead to the 12-month cells the abstract label refers to: BMI 0.24 (−0.04 to 0.52, p = 0.09) and height 0.44 (0.25 to 0.64). All numbers are quoted from the article.
Figure 2. The same contrasts drawn on one LSM-difference axis, with a reference line at 0 (no difference). For BMI, the abstract's "12-month" interval (orange) lies exactly on the Table 3 6-month interval (blue diamond). The Table 3 12-month interval (blue dot) is shifted left and crosses 0. For height, the abstract interval is wider than and offset from both Table 3 height intervals. Values are from the paper; recomputed p values are given in the table above.
Why it matters
Steroid-related weight gain is a main safety concern in Duchenne muscular dystrophy, and comparing weight status with prednisone is one of the paper's key anthropometric results. Read alone, the abstract implies that at 12 months vamorolone raised BMI Z-score clearly more than prednisone did (0.57, CI excluding 0). The paper's own 12-month estimate is less than half that size (0.24) and is not statistically significant (CI −0.04 to 0.52, p = 0.09). That matches the Discussion's statement that the excess was present "at 6 but not at 12 months". Many readers see only the abstract, and abstract values are what get indexed and carried into reviews and meta-analyses, so the overstatement can spread.
The height error leaves the direction and significance of the growth finding intact: Table 3 gives 0.44 (0.25 to 0.64), p < 0.0001. Only its size is overstated, by 0.13 Z-score units. The paper's headline conclusions are unaffected: similar motor efficacy, growth protection with vamorolone, and BMI increases on all three treatments. Vamorolone's 12-month BMI excess over deflazacort (0.29, 0.03 to 0.56) is correctly reported.
Explanations considered
- A different model, population or weighting in the abstract. This does not account for the numbers. The abstract describes the same entropy-balanced, weighted mixed model for repeated measures as Table 3, and its two deflazacort values in the same sentences match the Table 3 12-month cells exactly. Only the prednisone values differ, and they equal another cell of the same table.
- Unweighted data (Figure 3B). This does not account for the numbers. Figure 3B shows unweighted arithmetic mean changes per group with t-based CIs, not between-treatment LSM differences, so it cannot be the source of a 0.57 [0.24–0.90] contrast. That triple is reproduced exactly by one Table 3 cell.
- Misaligned or mislabelled Table 3 columns. Ruled out. Each Table 3 CI is centred on its estimate, each cell's p value matches its CI apart from the 6-month BMI cell (see note above), and the Results text labels 0.57 (0.24–0.90; p = 0.009) explicitly as the 6-month BMI comparison. The Discussion's "at 6 but not at 12 months" fits the table's 12-month value of 0.24 with p = 0.09.
- Different time points across the two trials (weeks 24/48 vs months 6/12). This does not account for the numbers. The paper aligns these visits as 6 and 12 months for all comparisons, and the deflazacort contrasts, built the same way, match.
- PMC conversion artefact. Ruled out. The publisher's typeset PDF (Neurology 2026;107:e214756) has the same abstract and Table 3 values, and the Europe PMC JATS XML shows proper minus signs in "0.24; −0.04, 0.52".
- A height value that is genuinely 0.57. Not supported. Table 3 gives the vamorolone-vs-prednisone height Z-score difference as 0.25 at 6 months and 0.44 at 12 months. The abstract's height triple is identical, digit for digit, to the 6-month BMI cell.
Status
As of 2026-10-23:
- Version of record: checked. The publisher's typeset PDF (14 pp., CC BY 4.0) was obtained through the Newcastle University ePrints repository, record 312342, because neurology.org returned 403. Its abstract and Table 3 match the PMC text quoted above. The PMC archive copy was read with the project's
show.py. - Supplement: the PMC/Europe PMC archive includes no supplementary files. According to the text, eTable 1 and eFigures 1–4 cover cohort balancing and the vamorolone 2 mg/kg/d vs intermittent prednisone comparison, not these contrasts. The supplement could not be opened (neurology.org returned 403). It is not needed, because Table 3 of the main text holds the values in question.
- Corrections: a publisher correction exists for this article: doi:10.1212/WNL.0000000000218498, Neurology 2026;107(5):e218498, online 2026-07-30 (Crossref "update-to" this article; PubMed type "Published Erratum", PMID 42531535; PMC records a correction-forward link). Its text could not be retrieved: neurology.org returned 403, PubMed and Europe PMC list no abstract, and the notice is not in PMC. It is therefore unverified whether this correction addresses the abstract values described here. The PMC copy of the article still shows the values quoted above.
- PubPeer: a search for the DOI was attempted, but pubpeer.com required human verification and returned no results to this check. A web search found no PubPeer thread or other public comment on these values.
- Sources: PMC13225239 (PubMed Central JATS); Europe PMC JATS XML; publisher PDF via eprints.ncl.ac.uk/312342; Crossref metadata for 10.1212/WNL.0000000000214756 and 10.1212/WNL.0000000000218498.
Suggested correction
Unless the existing correction (e218498) already covers it, the abstract's Results sentence could be amended to the 12-month values in Table 3:
Body mass index (BMI) Z-scores increased in all groups, greatest with vamorolone (12-month LSM difference [95% CI]: vamorolone vs prednisone 0.24 [−0.04 to 0.52]; vamorolone vs deflazacort 0.29 [0.03–0.56]), whereas growth slowdown occurred in prednisone- and deflazacort-treated boys but not vamorolone (12-month LSM difference in height Z-scores [95% CI]: vamorolone vs prednisone 0.44 [0.25–0.64]; vamorolone vs deflazacort 0.72 [0.53–0.91]).
Because the 12-month BMI difference versus prednisone is not statistically significant, the phrase "greatest with vamorolone" could also be qualified, for example "greater with vamorolone than deflazacort at 12 months, and than prednisone at 6 months". That wording would match the paper's own Discussion. Separately, the authors could check the 6-month BMI p value (0.009) in Table 3 and the Results text against its CI (0.24–0.90).
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13225239 write-up and regenerate its figures.
Paper: "Vamorolone for Duchenne Muscular Dystrophy", Neurology 2026,
doi:10.1212/WNL.0000000000214756 (PMC13225239).
All inputs are copied from the article as archived in PubMed Central: Table 3
(LSM differences, 95% CI and p values at 6 and 12 months) and the Abstract
(Results paragraph). Run:
python recompute.py.txt
Figures are written next to this file: five candidate designs (cand_A ... cand_E,
SVG + PNG at 200 dpi), and fig1/fig2, which are copies of the currently chosen
candidates (see CHOSEN below).
"""
import os
import shutil
import textwrap
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.ppf(0.975)
# ---------------------------------------------------------------------------
# Inputs reported in the paper
# ---------------------------------------------------------------------------
# Table 3: (outcome, comparison, month) -> (LSM difference, CI low, CI high, reported p)
T3 = {
("BMI", "Pred", 6): (0.57, 0.24, 0.90, "0.009"),
("BMI", "Pred", 12): (0.24, -0.04, 0.52, "0.09"),
("BMI", "DFZ", 6): (0.32, 0.00, 0.64, "0.05"),
("BMI", "DFZ", 12): (0.29, 0.03, 0.56, "0.03"),
("Weight", "Pred", 6): (0.61, 0.34, 0.89, "<0.0001"),
("Weight", "Pred", 12): (0.58, 0.33, 0.82, "<0.0001"),
("Weight", "DFZ", 6): (0.60, 0.33, 0.87, "<0.0001"),
("Weight", "DFZ", 12): (0.78, 0.54, 1.02, "<0.0001"),
("Height", "Pred", 6): (0.25, 0.14, 0.36, "<0.0001"),
("Height", "Pred", 12): (0.44, 0.25, 0.64, "<0.0001"),
("Height", "DFZ", 6): (0.50, 0.39, 0.61, "<0.0001"),
("Height", "DFZ", 12): (0.72, 0.53, 0.91, "<0.0001"),
}
# Abstract, Results: values the abstract labels "12-month LSM difference [95% CI]"
ABSTRACT = {
("BMI", "Pred"): (0.57, 0.24, 0.90),
("BMI", "DFZ"): (0.29, 0.03, 0.56),
("Height", "Pred"): (0.57, 0.24, 0.90),
("Height", "DFZ"): (0.72, 0.53, 0.91),
}
NAME = {"Pred": "prednisone", "DFZ": "deflazacort"}
def wald(est, lo, hi):
"""SE from the CI width (normal approximation), z, two-sided p, CI midpoint."""
se = (hi - lo) / (2 * Z975)
z = est / se
return se, z, 2 * stats.norm.sf(abs(z)), (lo + hi) / 2
def fmt_p(p):
return f"{p:.2g}" if p >= 1e-4 else f"{p:.1e}"
# ---------------------------------------------------------------------------
# 1. Internal consistency of every Table 3 cell
# ---------------------------------------------------------------------------
print("1. Table 3 internal consistency (SE from CI width, normal approximation)")
print(f" {'cell':24s} {'est':>5s} {'mid':>6s} {'SE':>6s} {'z':>6s} {'p_recomp':>9s} reported p")
for (o, c, m), (e, lo, hi, p) in T3.items():
se, z, pr, mid = wald(e, lo, hi)
print(f" {o+' vam6/'+c+' '+str(m)+' mo':24s} {e:5.2f} {mid:6.3f} {se:6.3f} {z:6.2f} {fmt_p(pr):>9s} {p}")
# ---------------------------------------------------------------------------
# 2. Which Table 3 cell does each abstract "12-month" triple equal?
# ---------------------------------------------------------------------------
print("\n2. Abstract '12-month' values vs Table 3")
for (o, c), trip in ABSTRACT.items():
hits = [f"{k[0]} vs {k[1]} {k[2]} mo" for k, v in T3.items() if v[:3] == trip]
t12 = T3[(o, c, 12)][:3]
status = "matches 12-mo cell" if trip == t12 else "DOES NOT match 12-mo cell"
print(f" {o} vs {NAME[c]:11s}: abstract {trip}; Table 3 12-mo {t12} -> {status}; "
f"identical to Table 3 cell(s): {hits}")
a, t12, t6 = ABSTRACT[("BMI", "Pred")], T3[("BMI", "Pred", 12)], T3[("BMI", "Pred", 6)]
print("\n3. Size of the discrepancies")
print(f" BMI vs prednisone: abstract {a[0]} - Table 3 12-mo {t12[0]} = {a[0]-t12[0]:.2f}; "
f"ratio {a[0]/t12[0]:.2f}")
print(f" BMI vs prednisone, 12-mo CI ({t12[1]}, {t12[2]}) contains 0: {t12[1] < 0 < t12[2]}; "
f"abstract CI ({a[1]}, {a[2]}) contains 0: {a[1] < 0 < a[2]}")
ah, h12, h6 = ABSTRACT[("Height", "Pred")], T3[("Height", "Pred", 12)], T3[("Height", "Pred", 6)]
print(f" Height vs prednisone: abstract {ah[0]} - Table 3 12-mo {h12[0]} = {ah[0]-h12[0]:.2f}; "
f"Table 3 6-mo height value is {h6[0]}, so the abstract height triple equals neither height cell")
se12, z12, p12, _ = wald(*t12[:3])
print(f" BMI vs prednisone 12 mo: SE {se12:.3f}, z {z12:.2f}, recomputed p {p12:.3f} (reported {t12[3]})")
sea, za, pa, _ = wald(*a)
print(f" Abstract triple 0.57 [0.24-0.90] taken at face value: SE {sea:.3f}, z {za:.2f}, p {pa:.5f}")
# ---------------------------------------------------------------------------
# 4. Secondary observation: 6-month BMI vs prednisone, reported p = 0.009
# ---------------------------------------------------------------------------
print("\n4. 6-month BMI vs prednisone: what would reconcile p = 0.009 with 0.57 (0.24, 0.90)?")
e, lo, hi, _ = T3[("BMI", "Pred", 6)]
half = (hi - lo) / 2
for df in (3, 4, 5, 10, 20, 30, 50, 70):
tc = stats.t.ppf(0.975, df)
se = half / tc
t = e / se
print(f" t-based CI with df={df:3d}: SE {se:.3f}, t {t:.2f}, two-sided p {2*stats.t.sf(t, df):.4f}")
print(" (p = 0.009 needs roughly df = 4; with df of 10 or more (26 + 48 boys) p is at most"
" 0.003. This does not affect the abstract error.)")
# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
LW = 1.5 # points; 2 px at 96 px/in
MS = 8 # marker size in points (>= 8 px)
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def ci_str(v):
e, lo, hi = v[:3]
f = lambda x: f"{x:.2f}".replace("-", "−")
return f"{f(e)} [{f(lo)} to {f(hi)}]"
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
plt.close(fig)
def recessive(ax, axis="x"):
ax.grid(axis=axis, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
# --- A: two text panels joined by annotated connectors ----------------------
def cand_A():
fig = plt.figure(figsize=(7.2, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
def panel(x, y, w, h, title, sub, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.006,rounding_size=0.012",
fc="white", ec=edge, lw=LW))
ax.text(x + 0.018, y + h - 0.03, title, fontsize=11, weight="bold", va="top")
ax.text(x + 0.018, y + h - 0.085, sub, fontsize=10, color=INK2, va="top")
PX0, PX1, PW = 0.02, 0.58, 0.40
panel(PX0, 0.17, PW, 0.79, "Abstract (Results)", "Values labelled \u201c12-month\u201d", ORANGE)
q1 = ("\u201cBMI Z-scores increased in all groups, greatest with vamorolone "
"(12-month LSM difference [95% CI]: vamorolone vs prednisone")
q2 = "\u201c(12-month LSM difference in height Z-scores [95% CI]: vamorolone vs prednisone"
ax.text(0.038, 0.79, textwrap.fill(q1, 40), fontsize=10, va="top", style="italic")
ax.text(0.038, 0.615, "0.57 [0.24\u20130.90]\u201d", fontsize=11, va="center", color=ORANGE, weight="bold")
ax.text(0.038, 0.53, textwrap.fill(q2, 40), fontsize=10, va="top", style="italic")
ax.text(0.038, 0.405, "0.57 [0.24\u20130.90]\u201d", fontsize=11, va="center", color=ORANGE, weight="bold")
ax.text(0.038, 0.31, textwrap.fill("The paired deflazacort values (0.29 [0.03\u20130.56] and "
"0.72 [0.53\u20130.91]) match Table 3 at 12 months.", 40),
fontsize=10, color=INK2, va="top")
panel(PX1, 0.17, PW, 0.79, "Table 3 (weighted MMRM)", "Vamorolone 6 vs prednisone:\nLSM difference; 95% CI; p", BLUE)
rows = [("BMI Z-score, 6 mo", T3[("BMI", "Pred", 6)], 0.75),
("BMI Z-score, 12 mo", T3[("BMI", "Pred", 12)], 0.59),
("Height Z-score, 12 mo", T3[("Height", "Pred", 12)], 0.43)]
ycen = {}
for lab, v, y in rows:
ax.text(0.598, y, lab, fontsize=10, color=INK2, va="top")
p = "< " + v[3][1:] if v[3].startswith("<") else "= " + v[3]
ax.text(0.598, y - 0.065, f"{v[0]:.2f}; {v[1]:.2f}, {v[2]:.2f}".replace("-", "\u2212")
+ f" p {p}", fontsize=11, va="center", color=BLUE, weight="bold")
ycen[lab] = y - 0.065
ax.text(0.598, 0.31, textwrap.fill("The Results text and the Discussion agree with Table 3: "
"the BMI excess was present \u201cat 6 but not at 12 months\u201d.", 40),
fontsize=10, color=INK2, va="top")
def conn(y0, y1, rad, ls="-"):
ax.add_patch(FancyArrowPatch((PX0 + PW + 0.012, y0), (PX1 - 0.002, y1),
connectionstyle=f"arc3,rad={rad}", arrowstyle="-|>",
mutation_scale=13, lw=LW, color=INK, ls=ls, shrinkA=0, shrinkB=0))
ax.plot([0.25, PX0 + PW + 0.012], [y0, y0], color=INK, lw=LW, ls=ls)
conn(0.615, ycen["BMI Z-score, 6 mo"], -0.2)
conn(0.615, ycen["BMI Z-score, 12 mo"], 0.0, ls=(0, (4, 3)))
conn(0.405, ycen["BMI Z-score, 6 mo"], -0.45)
conn(0.405, ycen["Height Z-score, 12 mo"], 0.0, ls=(0, (4, 3)))
ax.text(0.02, 0.115, "Solid arrows: identical numbers. Both abstract values equal the 6-month BMI cell.",
fontsize=10)
ax.text(0.02, 0.07, "Dashed arrows: the 12-month cell each abstract label points to. BMI: 0.57 vs 0.24,",
fontsize=10)
ax.text(0.02, 0.025, "and the 12-month CI includes 0. Height: 0.57 vs 0.44.", fontsize=10)
save(fig, "cand_A")
# --- B: dot-and-interval, one axis, the two prednisone contrasts --------------
def cand_B():
fig, ax = plt.subplots(figsize=(7.2, 3.9))
items = [
("BMI", "Abstract, labelled 12 mo", ABSTRACT[("BMI", "Pred")], ORANGE, "o", None),
("BMI", "Table 3, 12 mo", T3[("BMI", "Pred", 12)], BLUE, "o", T3[("BMI", "Pred", 12)][3]),
("BMI", "Table 3, 6 mo", T3[("BMI", "Pred", 6)], BLUE, "D", T3[("BMI", "Pred", 6)][3]),
("Height", "Abstract, labelled 12 mo", ABSTRACT[("Height", "Pred")], ORANGE, "o", None),
("Height", "Table 3, 12 mo", T3[("Height", "Pred", 12)], BLUE, "o", T3[("Height", "Pred", 12)][3]),
("Height", "Table 3, 6 mo", T3[("Height", "Pred", 6)], BLUE, "D", T3[("Height", "Pred", 6)][3]),
]
ys = [6.2, 5.4, 4.6, 2.6, 1.8, 1.0]
for (o, lab, v, col, mk, p), y in zip(items, ys):
ax.plot([v[1], v[2]], [y, y], color=col, lw=LW, solid_capstyle="round")
ax.plot(v[0], y, marker=mk, ms=MS, color=col, mec=BG, mew=1.2,
mfc=col if mk == "o" else BG, zorder=3)
if mk == "D":
ax.plot(v[0], y, marker=mk, ms=MS - 1, color=col, mfc=BG, mew=LW, zorder=4)
txt = ci_str(v) + ((", p < " + p[1:] if p.startswith("<") else f", p = {p}") if p else "")
ax.text(1.02, y, txt, va="center", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.8)
ax.set_yticks(ys)
ax.set_yticklabels([i[1] for i in items], fontsize=10)
for t, it in zip(ax.get_yticklabels(), items):
t.set_color(INK)
ax.text(-0.42, 6.85, "BMI Z-score, vamorolone 6 vs prednisone", fontsize=10.5, weight="bold",
transform=ax.transData, ha="left")
ax.text(-0.42, 3.25, "Height Z-score, vamorolone 6 vs prednisone", fontsize=10.5, weight="bold",
transform=ax.transData, ha="left")
ax.set_xlim(-0.42, 1.0); ax.set_ylim(0.4, 7.2)
ax.set_xlabel("LSM difference in Z-score from baseline (95% CI); 0 = no difference")
recessive(ax)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Reported in Abstract as 12-month"),
Line2D([], [], color=BLUE, marker="o", ms=MS, lw=LW, label="Paper's Table 3, 12 months"),
Line2D([], [], color=BLUE, marker="D", ms=MS - 1, mfc=BG, lw=LW, label="Paper's Table 3, 6 months")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=3, frameon=False,
fontsize=10, handlelength=1.6, columnspacing=1.2)
save(fig, "cand_B")
# --- C: lookup map, each abstract triple wired to the Table 3 cell it equals --
def cand_C():
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.955, "Abstract: \u201c12-month LSM difference\u201d", weight="bold", fontsize=11)
ax.text(0.55, 0.955, "Table 3: LSM difference (95% CI)", weight="bold", fontsize=11)
cols = {12: 0.64, 6: 0.82} # 12-month column first, next to the abstract
W, H = 0.15, 0.12
rows = [("BMI", "Pred"), ("BMI", "DFZ"), ("Height", "Pred"), ("Height", "DFZ")]
ry = {r: 0.72 - i * 0.15 for i, r in enumerate(rows)}
for m, x in cols.items():
ax.text(x, 0.215 - 0.06, f"{m} months", ha="center", fontsize=10, color=INK2)
cell = {}
for r, y in ry.items():
for m, x in cols.items():
v = T3[(r[0], r[1], m)]
ax.add_patch(Rectangle((x - W / 2, y - H / 2), W, H, fc="white", ec=GRID, lw=1))
ax.text(x, y + 0.02, f"{v[0]:.2f}", ha="center", va="center", fontsize=10.5,
color=BLUE, weight="bold")
ax.text(x, y - 0.03, f"{v[1]:.2f}, {v[2]:.2f}".replace("-", "\u2212"), ha="center",
va="center", fontsize=10, color=BLUE)
cell[(r, m)] = (x, y)
for r in (("BMI", "Pred"), ("Height", "Pred")):
x, y = cell[(r, 12)]
ax.add_patch(Rectangle((x - W / 2, y - H / 2), W, H, fc="none", ec=INK, lw=LW, ls=(0, (3, 2))))
ax.text(0.64, 0.80, "expected", ha="center", fontsize=10, color=INK)
top = 0.905 # routing lane above the table
lane = {("BMI", "Pred"): 0.47, ("Height", "Pred"): 0.50}
for r, y in ry.items():
v = ABSTRACT[r]
ax.text(0.03, y + 0.022, f"{r[0]} vs {NAME[r[1]]}", fontsize=10, color=INK2, va="center")
ax.text(0.03, y - 0.028, ci_str(v), fontsize=10.5, color=ORANGE, weight="bold", va="center")
if T3[(r[0], r[1], 12)][:3] == v:
x, yy = cell[(r, 12)]
ax.add_patch(FancyArrowPatch((0.32, y), (x - W / 2 - 0.004, yy), arrowstyle="-|>",
mutation_scale=12, lw=LW, color=INK2))
ax.text(0.45, y + 0.018, "match", ha="center", fontsize=10, color=INK2)
else:
src = [k for k, t in T3.items() if t[:3] == v][0]
x, yy = cell[((src[0], src[1]), src[2])]
lx = lane[r]
ax.plot([0.32, lx, lx, x], [y, y, top, top], color=INK, lw=LW, solid_joinstyle="round")
ax.add_patch(FancyArrowPatch((x, top), (x, yy + H / 2 + 0.004), arrowstyle="-|>",
mutation_scale=12, lw=LW, color=INK, shrinkA=0))
ax.text(0.03, 0.09, textwrap.fill("Arrows lead to the Table 3 cell whose three numbers equal the "
"abstract value. Both prednisone values equal the 6-month BMI cell, "
"not the 12-month cells (dashed) that the label implies.", 80),
fontsize=10, va="center")
save(fig, "cand_C")
# --- D: all four abstract 12-month contrasts vs Table 3 12-month values ------
def cand_D():
fig, ax = plt.subplots(figsize=(7.2, 3.8))
order = [("BMI", "Pred"), ("BMI", "DFZ"), ("Height", "Pred"), ("Height", "DFZ")]
base = [4.0, 3.0, 2.0, 1.0]
for r, y in zip(order, base):
a, t = ABSTRACT[r], T3[(r[0], r[1], 12)]
for v, col, dy in ((a, ORANGE, 0.17), (t, BLUE, -0.17)):
ax.plot([v[1], v[2]], [y + dy] * 2, color=col, lw=LW, solid_capstyle="round")
ax.plot(v[0], y + dy, "o", ms=MS, color=col, mec=BG, mew=1.2, zorder=3)
d = a[0] - t[0]
note = "identical" if a == t[:3] else f"differ by {d:.2f}"
ax.text(1.12, y, note, va="center", fontsize=10, color=INK if d else INK2,
weight="bold" if d else "normal")
ax.axvline(0, color=INK2, lw=0.8)
ax.set_yticks(base)
ax.set_yticklabels([f"{r[0]} Z-score\nvs {NAME[r[1]]}" for r in order], fontsize=10, color=INK)
ax.set_xlim(-0.2, 1.1); ax.set_ylim(0.5, 4.6)
ax.set_xlabel("12-month LSM difference in Z-score, vamorolone 6 minus comparator (95% CI)")
recessive(ax); ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Abstract (labelled 12-month)"),
Line2D([], [], color=BLUE, marker="o", ms=MS, lw=LW, label="Table 3, 12-month cell")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_D")
# --- E: trajectory 6 -> 12 months, with the abstract's 12-month claim overlaid -
def cand_E():
fig, ax = plt.subplots(figsize=(7.2, 4.0))
xs = {("BMI", 6): 0, ("BMI", 12): 1, ("Height", 6): 3.0, ("Height", 12): 4.0}
for o in ("BMI", "Height"):
v6, v12 = T3[(o, "Pred", 6)], T3[(o, "Pred", 12)]
ax.plot([xs[(o, 6)], xs[(o, 12)]], [v6[0], v12[0]], color=BLUE, lw=1.0, alpha=0.6)
for m, v in ((6, v6), (12, v12)):
x = xs[(o, m)]
ax.plot([x, x], [v[1], v[2]], color=BLUE, lw=LW)
ax.plot(x, v[0], "o", ms=MS, color=BLUE, mec=BG, mew=1.2, zorder=3)
ax.text(x - 0.07, v[0], f"{v[0]:.2f}", ha="right", va="center", fontsize=10, color=INK)
a = ABSTRACT[(o, "Pred")]
xa = xs[(o, 12)] + 0.75
ax.plot([xa, xa], [a[1], a[2]], color=ORANGE, lw=LW)
ax.plot(xa, a[0], "o", ms=MS, color=ORANGE, mec=BG, mew=1.2, zorder=3)
ax.text(xa + 0.07, a[0], f"{a[0]:.2f}", ha="left", va="center", fontsize=10, color=INK)
# the copied value: dotted guide from the 6-month BMI estimate
ax.plot([xs[("BMI", 6)], xs[("Height", 12)] + 0.75], [0.57, 0.57], color=INK2, lw=0.9, ls=(0, (2, 2)))
ax.text(2.4, 0.60, "0.57 [0.24–0.90] = 6-month BMI cell", ha="center", fontsize=10, color=INK2)
ax.axhline(0, color=INK2, lw=0.8)
ax.set_xticks([0, 1, 1.75, 3.0, 4.0, 4.75])
ax.set_xticklabels(["6 mo", "12 mo", "Abstract\n“12 mo”", "6 mo", "12 mo",
"Abstract\n“12 mo”"], fontsize=10)
ax.text(0.875, -0.36, "BMI Z-score", ha="center", fontsize=10.5, weight="bold", transform=ax.transData)
ax.text(3.875, -0.36, "Height Z-score", ha="center", fontsize=10.5, weight="bold", transform=ax.transData)
ax.set_xlim(-0.5, 5.3); ax.set_ylim(-0.12, 1.05)
ax.set_ylabel("LSM difference, vamorolone 6\nminus prednisone (95% CI)")
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="x", length=0, pad=4)
from matplotlib.lines import Line2D
h = [Line2D([], [], color=BLUE, marker="o", ms=MS, lw=LW, label="Paper's Table 3"),
Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Reported in Abstract as 12-month")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.3), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(bottom=0.2)
save(fig, "cand_E")
for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
f()
CHOSEN = {"fig1": "cand_A", "fig2": "cand_B"}
for fig, cand in CHOSEN.items():
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(HERE, f"{cand}.{ext}"), os.path.join(HERE, f"{fig}.{ext}"))
print("\nFigures written:", ", ".join(f"{c}.svg/.png" for c in "cand_A cand_B cand_C cand_D cand_E".split()),
"; fig1 =", CHOSEN["fig1"], "; fig2 =", CHOSEN["fig2"])
Abstract reports 73.5% of trials used biomarkers; the Results and phase counts give 56.3%
Neurology (2026) · doi:10.1212/WNL.0000000000218373 · PMC13445500
Summary
This systematic review of 238 phase II–IV Alzheimer disease drug trials states in its abstract that "73.5% (175/238) used at least one biomarker as efficacy measures". The Results section gives 56.3% (134/238) for the same quantity. The phase-stratified counts in the Results (79/135 + 55/103 = 134/238) and the Discussion's "more than half" agree with 134, not 175. The figure 175/238 (73.5%) appears in the Results as the count of trials that used clinical outcome measures as primary measures, so the abstract seems to carry that count in the biomarker position. The abstract overstates one of the review's headline descriptive results by 41 trials (17.2 percentage points). The paper's qualitative conclusion that biomarker use is widespread still holds at 56.3%.
What the paper reports
Abstract, Results:
Among 238 included trials, 95.4% (227/238) used at least one clinical outcome and 73.5% (175/238) used at least one biomarker as efficacy measures.
Abstract, Discussion:
while biomarker use is widespread
Results, "Efficacy Measures in AD Drug Trials" (two consecutive sentences):
Among 238 included trials, 95.4% (227/238) used at least one clinical outcome measure as an efficacy measure, of which 73.5% (175/238) used clinical outcome measures as primary measures and 84.5% (201/238) as secondary measures. In addition, 56.3% (134/238) of trials used at least one biomarker as an efficacy measure, of which 11.3% (27/238) used at least one biomarker as a primary measure and 41.2% (98/238) as a secondary measure.
The next sentence in the same paragraph:
Overall, 18.1% (43/238) of trials included neither a clinical outcome measure nor a biomarker as their primary measure.
Results, "Sample Characteristics":
Of 238 included trials, 56.7% (135/238) were phase II and 43.3% (103/238) were phase III or IV.
Results, "Clinical Relevance of Efficacy Measures", phase II and phase III–IV (domains assessed overall):
functional ability and dependency (66.7%, 90/135), and biomarkers (58.5%, 79/135).
functional ability and dependency (77.7%, 80/103), and biomarkers (53.4%, 55/103).
The same paragraph, restricted to primary measures (phase II, then phase III–IV):
followed by biomarkers (14.8%, 20/135)
and biomarkers (5.8%, 6/103)
Results, same section, and Abstract, Results:
Notably, 8.4% (20/238) used biomarkers as the only primary efficacy measures.
whereas 8.4% (20/238) used biomarkers as the sole primary efficacy measure
Discussion, first paragraph and third paragraph:
Moreover, more than half of the trials evaluated biomarkers, predominantly as secondary measures
more than half of trials used biomarkers as efficacy measures
Methods, "Data Extraction" and "Data Analysis":
We also extracted whether outcome measures were evaluated as primary and secondary outcome measures or any exploratory outcome measures.
We determined the proportion of trials that used clinical outcome measures or biomarkers as any efficacy measures, primary or secondary.
What the paper's own numbers show
Method. All counts share one denominator: the 238 included trials, split into 135 phase II and 103 phase III–IV trials. I recomputed every printed percentage from its printed count. I then rebuilt the number of trials with any biomarker use from the phase-stratified counts, compared it with the abstract and the Results, and checked which other Results quantity equals 175/238. As a further check I used the primary-measure counts: the trials with a clinical or biomarker primary measure (238 − 43), the clinical-primary count (175) and the biomarker-primary count (27). Together they imply an overlap, and from it a biomarker-only primary count that can be compared with the paper's 20/238. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Trials using ≥ 1 biomarker as an efficacy measure | Abstract, Results | 73.5% (175/238) | 175/238 = 73.53%: the arithmetic is right, but the count conflicts with the body |
| Trials using ≥ 1 biomarker as an efficacy measure | Results, Efficacy Measures | 56.3% (134/238) | 134/238 = 56.30% |
| Abstract minus Results | derived | — | 41 trials, 17.2 percentage points; the abstract count is 1.31 × the Results count |
| Trials using clinical outcomes as primary measures | Results, Efficacy Measures | 73.5% (175/238) | 175/238 = 73.53%, the same count and percentage the abstract gives for biomarkers |
| Phase II, any biomarker use | Results, Clinical Relevance | 58.5% (79/135) | 58.52% |
| Phase III–IV, any biomarker use | Results, Clinical Relevance | 53.4% (55/103) | 53.40% |
| Sum of the phase strata | derived | — | 135 + 103 = 238 trials; 79 + 55 = 134 biomarker trials (56.3%), equal to the Results and 41 below the abstract |
| Discussion: "more than half" used biomarkers | Discussion | — | half of 238 = 119; 134 fits "more than half". 175 is close to three-quarters (178.5) |
| Biomarker primary + secondary | Results, Efficacy Measures | 27/238 (11.3%) and 98/238 (41.2%) | 27 + 98 = 125, the most trials that could use a biomarker as a primary or secondary measure. 175 exceeds it by 50 |
| Trials with a clinical or biomarker primary measure | Results, Efficacy Measures | 43/238 (18.1%) with neither | 238 − 43 = 195 |
| Implied trials with both clinical and biomarker primary measures | derived | — | 175 + 27 − 195 = 7 |
| Implied biomarker-only primary measures | derived vs Results and Abstract | 8.4% (20/238) | 27 − 7 = 20, which matches. So 175 fits the paper's own figures as the clinical-primary count |
| Clinical outcome, any, primary and secondary use | Results, Efficacy Measures | 95.4% (227), 73.5% (175), 84.5% (201) | 95.38%, 73.53%, 84.45%; 175 + 201 − 227 = 149 trials with both |
Two smaller inconsistencies inside the Results came up. They do not bear on the choice between 134 and 175:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Any biomarker use vs primary + secondary use | Results, Efficacy Measures | 134 vs 27 and 98 | 134 is 9 more than 27 + 98 = 125. The Methods say exploratory measures were also extracted, which could account for this, but the paper does not say so |
| Biomarker as primary measure, phase strata vs overall | Results, Clinical Relevance vs Efficacy Measures | 20/135 + 6/103 vs 27/238 | 20 + 6 = 26, one less than 27 |
Figure
Figure 1. The two conflicting passages, quoted from the paper. Left, the abstract's Results sentence. Right, the Results paragraph "Efficacy Measures in AD Drug Trials", shortened with ellipses. Reported values are in orange. The gray connector links the abstract's biomarker figure to the identical Results figure for clinical outcomes used as primary measures. The orange connector links it to the Results figure for biomarkers, which it contradicts. The blue panel gives the value recomputed from the paper's phase-stratified counts. It matches the Results (134/238, 56.3%), not the abstract.
Figure 2. The counts behind the conflict, on one axis of trials from 0 to 238. Orange circles are counts printed in the paper. Blue diamonds are counts recomputed from the paper's other figures: the sum of the phase II and phase III–IV biomarker counts (79 + 55 = 134), and the upper bound on primary or secondary biomarker use (27 + 98 = 125). The dashed line marks half of the 238 trials, the threshold of the Discussion's "more than half". The abstract's 175 equals the clinical-primary count and lies 41 trials above every other count of biomarker use in the paper.
Why it matters
The abstract's Results section opens with this figure, and for many readers the abstract is the only part of a review they read or quote. A secondary summary of the article, whose URL carries the date 25 July 2026, already repeats the abstract's figure ("73.5% of analyzed trials incorporated at least one biomarker as an efficacy endpoint").
- Size. The abstract overstates biomarker use by 41 trials, from just over half of trials to about three-quarters, a relative overstatement of 31%.
- Comparison inside the abstract. The abstract puts the two headline figures side by side: 95.4% of trials used clinical outcomes and 73.5% used biomarkers. Read together, they make the two kinds of efficacy measure look much closer in use than the body reports (95.4% vs 56.3%).
- Conclusion. The abstract's Discussion says "biomarker use is widespread". That remains a fair description at 56.3%, and the body and Discussion carry the correct value. The error changes the magnitude of a key descriptive result, not the direction of the paper's conclusions. None of the review's other headline figures (domains assessed, co-primary measures, 20/238 biomarker-only primaries, counts of distinct measures) depends on it.
Explanations considered
- The two figures measure different things. This does not account for the numbers. Both refer to the same 238 trials and use the same wording ("at least one biomarker as an efficacy measure"). Neither refers to a subgroup, a time point or primary-only use.
- The abstract counts exploratory measures and the Results do not. This does not account for the numbers. The Methods define the analysis as "any efficacy measures, primary or secondary", and both passages use the same phrase. The phase-stratified Results, which add up to 134, also cover all domains assessed. No passage gives a broader count than 134.
- The Results figure is the wrong one, and 175 is correct. This is contradicted by three independent figures in the paper. The phase counts sum to 134. The Discussion twice says "more than half", which would understate 73.5%. Primary plus secondary biomarker use is at most 125. Separately, 175 fits the clinical-primary count: with 195 trials having a clinical or biomarker primary measure, it implies 20 trials with biomarker-only primaries, exactly the 20/238 the paper reports.
- A conversion or typesetting artifact in the PMC copy. This is excluded as far as the available copies allow. The PMC article page, the Europe PMC full-text XML and the local PMC archive all carry the same wording and numbers. A secondary summary whose URL is dated 25 July 2026, a month before the 25 August 2026 issue date, also repeats 73.5%. This suggests the figure was already in the online-first version, although the page shows no explicit date.
- The supplement resolves it. This could not be checked. eTable 2 may list counts by type of efficacy measure, but the supplement was not available (see Status). The main text alone points to 134.
The smaller inconsistencies (134 vs 125; 26 vs 27) leave some uncertainty about the exact correct value, perhaps by a few trials. They do not bring it near 175.
Status
As of 2026-10-23:
- Version of record. The publisher's site (www.neurology.org, reached from the DOI) returned HTTP 403 for the article HTML, the PDF and the supplement, so I could not read the publisher's own rendering. The PMC article page (pmc.ncbi.nlm.nih.gov/articles/PMC13445500/), the Europe PMC full-text XML and the local PMC archive all carry the same abstract sentence (73.5%, 175/238) and the same Results sentence (56.3%, 134/238). The PMC-hosted publisher PDF (WNL-2026-300706.pdf) could not be downloaded, because the server returned an interstitial page. Every quotation above was checked verbatim against the PMC-archived text.
- Supplement. The article cites eTables and eFigures. Neither PMC nor Europe PMC includes the supplementary material, the publisher's supplement page returned 403, and a web search found no public copy. The Data Availability statement says aggregated data are available on request.
- Corrections and comments. I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices (PMID 42492033). PubMed links only to a companion editorial (PMID 42492030, doi:10.1212/WNL.0000000000218468), whose text I could not open. The PubPeer API returned no comments for the DOI, though its web search page refused automated access (HTTP 403). A web search for the title with "correction", "erratum" or "PubPeer" found nothing relevant.
- Sources. PMC article PMC13445500; Europe PMC full-text XML; Crossref; PubMed; PubPeer API; secondary summary at omnicuris.com (URL dated 25 July 2026).
Suggested correction
- In the abstract's Results, replace "73.5% (175/238) used at least one biomarker as efficacy measures" with "56.3% (134/238) used at least one biomarker as efficacy measures", if the authors' data confirm the Results figure.
- Reconcile the any-use count (134) with the primary and secondary counts (27 and 98, total 125). For example, state whether exploratory measures are included in "any" use. Also reconcile the phase-level biomarker-primary counts (20 + 6 = 26) with the overall 27.
- Check any figure or supplementary table that reports overall biomarker use (for example eTable 2) against the corrected value.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13445500 write-up and regenerate its figures.
Paper: "Efficacy Measures Used in Alzheimer Disease Clinical Trials Between 2015
and 2025", Neurology 2026, doi:10.1212/WNL.0000000000218373 (PMC13445500).
Run: python recompute.py.txt
All inputs below are counts printed in the paper (location given for each).
Outputs: fig1.svg/png, fig2.svg/png and five candidate figure designs in
candidates/ (cand_A ... cand_E), all written next to this file.
"""
import os
import shutil
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs
N = 238 # included trials (Abstract; Results)
N_PH2, N_PH34 = 135, 103 # Results, Sample Characteristics
ABS_BIO_ANY = 175 # Abstract: "73.5% (175/238) used at least one biomarker"
ABS_BIO_PCT = 73.5
RES_BIO_ANY = 134 # Results: "56.3% (134/238) ... at least one biomarker"
RES_BIO_PCT = 56.3
RES_CLIN_ANY = 227 # Results: 95.4% (227/238) at least one clinical outcome
RES_CLIN_PRIM = 175 # Results: 73.5% (175/238) clinical outcomes as primary
RES_CLIN_SEC = 201 # Results: 84.5% (201/238) clinical outcomes as secondary
RES_BIO_PRIM = 27 # Results: 11.3% (27/238) biomarker as primary
RES_BIO_SEC = 98 # Results: 41.2% (98/238) biomarker as secondary
RES_NEITHER_PRIM = 43 # Results: 18.1% (43/238) neither as primary
RES_BIO_ONLY_PRIM = 20 # Results & Abstract: 8.4% (20/238) biomarkers only primary
PH2_BIO_ANY, PH34_BIO_ANY = 79, 55 # Results: 58.5% (79/135), 53.4% (55/103)
PH2_BIO_PRIM, PH34_BIO_PRIM = 20, 6 # Results: 14.8% (20/135), 5.8% (6/103)
def pct(k, n=N):
return 100.0 * k / n
def show(label, value, reported=None):
s = f"{label:<64s} {value}"
if reported is not None:
s += f" (paper: {reported})"
print(s)
print("=" * 100)
print("1. Percentages as printed")
show("Abstract biomarker any use 175/238", f"{pct(ABS_BIO_ANY):.2f}%", "73.5%")
show("Results biomarker any use 134/238", f"{pct(RES_BIO_ANY):.2f}%", "56.3%")
show("Results clinical outcome any use 227/238", f"{pct(RES_CLIN_ANY):.2f}%", "95.4%")
show("Results clinical outcome as primary 175/238", f"{pct(RES_CLIN_PRIM):.2f}%", "73.5%")
show("Results clinical outcome as secondary 201/238", f"{pct(RES_CLIN_SEC):.2f}%", "84.5%")
show("Results biomarker as primary 27/238", f"{pct(RES_BIO_PRIM):.2f}%", "11.3%")
show("Results biomarker as secondary 98/238", f"{pct(RES_BIO_SEC):.2f}%", "41.2%")
show("Results neither as primary 43/238", f"{pct(RES_NEITHER_PRIM):.2f}%", "18.1%")
show("Results biomarkers as only primary 20/238", f"{pct(RES_BIO_ONLY_PRIM):.2f}%", "8.4%")
show("Results phase II biomarker any use 79/135", f"{pct(PH2_BIO_ANY, N_PH2):.2f}%", "58.5%")
show("Results phase III-IV biomarker any use 55/103", f"{pct(PH34_BIO_ANY, N_PH34):.2f}%", "53.4%")
show("Results phase II biomarker primary 20/135", f"{pct(PH2_BIO_PRIM, N_PH2):.2f}%", "14.8%")
show("Results phase III-IV biomarker primary 6/103", f"{pct(PH34_BIO_PRIM, N_PH34):.2f}%", "5.8%")
print("\n2. Abstract vs Results")
gap = ABS_BIO_ANY - RES_BIO_ANY
show("Gap in trials (175 - 134)", gap)
show("Gap in percentage points", f"{pct(gap):.2f}")
show("Ratio abstract / Results (175/134)", f"{ABS_BIO_ANY / RES_BIO_ANY:.3f}")
show("Abstract biomarker count == Results clinical-primary count?",
ABS_BIO_ANY == RES_CLIN_PRIM)
print("\n3. Phase subtotals")
show("Phase denominators 135 + 103", N_PH2 + N_PH34)
ph_sum = PH2_BIO_ANY + PH34_BIO_ANY
show("Phase biomarker any-use 79 + 55", ph_sum)
show(" as % of 238", f"{pct(ph_sum):.2f}%")
show(" equals Results 134?", ph_sum == RES_BIO_ANY)
show(" equals Abstract 175?", ph_sum == ABS_BIO_ANY)
show("Phase biomarker primary 20 + 6", PH2_BIO_PRIM + PH34_BIO_PRIM)
show(" vs overall biomarker primary 27, difference",
RES_BIO_PRIM - (PH2_BIO_PRIM + PH34_BIO_PRIM))
print("\n4. Bounds from primary/secondary counts")
ub = RES_BIO_PRIM + RES_BIO_SEC
show("Biomarker primary + secondary (27 + 98), max primary-or-secondary use", ub)
show(" Abstract 175 exceeds this by", ABS_BIO_ANY - ub)
show(" Results 134 exceeds this by", RES_BIO_ANY - ub)
show("Clinical: primary + secondary - any (175 + 201 - 227) = both", RES_CLIN_PRIM + RES_CLIN_SEC - RES_CLIN_ANY)
any_prim = N - RES_NEITHER_PRIM
show("Trials with a clinical or biomarker primary measure (238 - 43)", any_prim)
both_prim = RES_CLIN_PRIM + RES_BIO_PRIM - any_prim
show(" implied both clinical and biomarker primary (175 + 27 - 195)", both_prim)
show(" implied biomarker-only primary (27 - 7)", RES_BIO_PRIM - both_prim, "20/238")
show("Half of 238 (Discussion: 'more than half')", N / 2)
show("Three-quarters of 238", 0.75 * N)
# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834" # values as reported in the paper
BLUE = "#2a78d6" # values recomputed from the paper's own numbers
GRID = "#e4e3df"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans",
"font.size": 10.5,
"text.color": INK,
"axes.labelcolor": INK2,
"axes.edgecolor": INK2,
"xtick.color": INK2,
"ytick.color": INK2,
"figure.facecolor": BG,
"axes.facecolor": BG,
"savefig.facecolor": BG,
"lines.linewidth": 2,
"svg.hashsalt": "PMC13445500",
})
def save(fig, name, folder=CAND):
for ext in ("svg", "png"):
fig.savefig(os.path.join(folder, f"{name}.{ext}"), dpi=200,
bbox_inches="tight", pad_inches=0.12,
metadata={"Date": None} if ext == "svg" else None)
plt.close(fig)
def box(ax, x, y, w, h, edge=INK2, lw=1.2, fill="#ffffff"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc=fill, ec=edge, lw=lw, transform=ax.transAxes))
def arrow(ax, p0, p1, color=INK2, rad=0.0, style="-|>", lw=2, ls="-"):
ax.add_patch(FancyArrowPatch(p0, p1, transform=ax.transAxes, arrowstyle=style,
mutation_scale=14, color=color, lw=lw, ls=ls,
connectionstyle=f"arc3,rad={rad}", shrinkA=2, shrinkB=2))
def legend_handles():
return [Line2D([], [], marker="o", ls="", ms=9, mfc=ORANGE, mec=ORANGE,
label="Reported in the paper"),
Line2D([], [], marker="D", ls="", ms=8, mfc=BLUE, mec=BLUE,
label="Recomputed from the paper's own counts")]
# ======================================================= Candidate A: text panels
def cand_A():
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.02, 0.975, "The abstract's biomarker count is the Results' clinical-primary count",
fontsize=12, weight="bold", va="top")
step = 0.052
def lines(x, y0, items):
"""items: (text, kind); kind 'h' header, 'q' quote, 'v' reported value."""
y = y0
pos = {}
for t, kind in items:
if kind == "h":
ax.text(x, y, t, fontsize=10.5, weight="bold", color=INK2, va="top")
elif kind == "v":
ax.text(x, y + 0.004, t, fontsize=12, weight="bold", color=ORANGE, va="top")
pos[t] = y - 0.022
else:
ax.text(x, y, t, fontsize=10.5, color=INK, va="top")
y -= step
return pos
PT, PB = 0.91, 0.30
box(ax, 0.02, PB, 0.37, PT - PB)
box(ax, 0.61, PB, 0.37, PT - PB)
pl = lines(0.04, 0.885, [("Abstract, Results", "h"), ("", "q"),
("\u201c95.4% (227/238) used at least", "q"),
("one clinical outcome and", "q"),
("73.5% (175/238)", "v"),
("used at least one biomarker", "q"),
("as efficacy measures.\u201d", "q")])
pr = lines(0.63, 0.885, [("Results, Efficacy Measures", "h"),
("in AD Drug Trials", "h"),
("\u201c... of which", "q"),
("73.5% (175/238)", "v"),
("used clinical outcome measures", "q"),
("as primary measures ...", "q"),
("In addition,", "q"),
("56.3% (134/238)", "v"),
("of trials used at least one bio-", "q"),
("marker as an efficacy measure\u201d", "q")])
yl = 0.885 - 4 * step - 0.022 # left orange value
yr1 = 0.885 - 3 * step - 0.022 # right 175
yr2 = 0.885 - 7 * step - 0.022 # right 134
arrow(ax, (0.30, yl + 0.012), (0.625, yr1), color=INK2, rad=-0.2)
ax.text(0.50, yr1 + 0.075, "same count,\ndifferent measure", fontsize=10, color=INK2,
ha="center", va="bottom", linespacing=1.2)
arrow(ax, (0.30, yl - 0.012), (0.625, yr2), color=ORANGE, rad=0.2, style="<|-|>")
ax.text(0.50, yr2 - 0.075, "conflict:\n175 vs 134", fontsize=10.5, weight="bold",
color=ORANGE, ha="center", va="top", linespacing=1.2)
# recomputation strip
box(ax, 0.02, 0.03, 0.96, 0.22, edge=BLUE, lw=1.6)
ax.text(0.04, 0.225, "Recomputed from the paper's own counts", fontsize=10.5, weight="bold",
color=BLUE, va="top")
ax.text(0.04, 0.165,
f"Phase II {PH2_BIO_ANY}/135 + phase III\u2013IV {PH34_BIO_ANY}/103 = "
f"{ph_sum}/238 = {pct(ph_sum):.1f}%, matching the Results",
fontsize=10.5, va="top")
ax.text(0.04, 0.105,
f"Abstract minus Results: {gap} trials, {pct(gap):.1f} percentage points",
fontsize=10.5, va="top")
save(fig, "cand_A_text_panels")
# ======================================================= Candidate B: dot comparison
def cand_B():
rows = [ # label, value, kind
("Abstract: ≥ 1 biomarker", ABS_BIO_ANY, "rep"),
("Results: ≥ 1 biomarker", RES_BIO_ANY, "rep"),
("Phase II + phase III–IV biomarker use (79 + 55)", ph_sum, "rec"),
("Biomarker primary + secondary (27 + 98), maximum", ub, "rec"),
("Results: clinical outcome as primary", RES_CLIN_PRIM, "rep"),
]
fig, ax = plt.subplots(figsize=(W, 3.5))
ys = list(range(len(rows)))[::-1]
for (lab, v, kind), y in zip(rows, ys):
c = ORANGE if kind == "rep" else BLUE
m = "o" if kind == "rep" else "D"
ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1, solid_capstyle="round")
ax.plot(v, y, marker=m, ms=10 if m == "o" else 9, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(v + 4, y, f"{v} ({pct(v):.1f}%)", va="center", fontsize=10.5, color=INK)
ax.axvline(N / 2, color=INK2, lw=1, ls=(0, (3, 3)), zorder=0)
ax.text(N / 2 - 2, -0.42, "half of 238 (Discussion: “more than half”)",
fontsize=10, color=INK2, va="center", ha="right",
bbox=dict(fc=BG, ec="none", pad=1))
ax.set_yticks(ys, [r[0] for r in rows], fontsize=10.5)
ax.set_xlim(0, N); ax.set_ylim(-0.7, len(rows) - 0.4)
ax.set_xticks([0, 50, 100, 150, 200, 238])
ax.set_xlabel("Trials (of 238)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.annotate("", xy=(RES_BIO_ANY, ys[1] + 0.5), xytext=(ABS_BIO_ANY, ys[1] + 0.5),
arrowprops=dict(arrowstyle="<|-|>", color=ORANGE, lw=1.6))
ax.text((ABS_BIO_ANY + RES_BIO_ANY) / 2, ys[1] + 0.56, "41 trials", ha="center",
va="bottom", fontsize=10, color=INK, weight="bold")
ax.legend(handles=legend_handles(), loc="upper center", bbox_to_anchor=(0.35, -0.2),
ncol=2, frameon=False, fontsize=10)
save(fig, "cand_B_dot_comparison")
# ======================================================= Candidate C: calculation flow
def cand_C():
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
def node(x, y, w, h, title, value, color, sub=None):
box(ax, x, y, w, h, edge=color, lw=1.8)
ax.text(x + w / 2, y + h - 0.045, title, ha="center", va="top", fontsize=10, color=INK2)
ax.text(x + w / 2, y + h / 2 - (0.02 if sub else 0.04), value, ha="center", va="center",
fontsize=12.5, weight="bold", color=INK)
if sub:
ax.text(x + w / 2, y + 0.05, sub, ha="center", va="bottom", fontsize=10, color=INK2)
node(0.02, 0.62, 0.22, 0.28, "Results, phase II", "79 / 135", ORANGE, "58.5%")
node(0.02, 0.22, 0.22, 0.28, "Results, phase III–IV", "55 / 103", ORANGE, "53.4%")
node(0.34, 0.42, 0.22, 0.28, "Recomputed sum", f"{ph_sum} / {N_PH2 + N_PH34}", BLUE,
f"{pct(ph_sum):.1f}%")
node(0.70, 0.62, 0.28, 0.28, "Results: ≥ 1 biomarker", "134 / 238", ORANGE, "56.3%")
node(0.70, 0.16, 0.28, 0.28, "Abstract: ≥ 1 biomarker", "175 / 238", ORANGE, "73.5%")
arrow(ax, (0.24, 0.76), (0.34, 0.60))
arrow(ax, (0.24, 0.36), (0.34, 0.52))
ax.text(0.29, 0.56, "+", fontsize=14, ha="center", va="center", color=INK2)
arrow(ax, (0.56, 0.60), (0.70, 0.74), color=BLUE)
ax.text(0.615, 0.715, "matches", fontsize=10.5, color=BLUE, weight="bold", ha="center")
arrow(ax, (0.56, 0.50), (0.70, 0.32), color=ORANGE, ls="--")
ax.text(0.555, 0.30, "does not match\n(41 trials fewer)", fontsize=10.5, color=ORANGE,
weight="bold", ha="center", va="top", linespacing=1.2)
ax.text(0.84, 0.035, "175/238 is the Results count of trials using\nclinical outcomes as primary measures",
fontsize=10, color=INK2, ha="center", va="bottom", linespacing=1.2)
ax.text(0.02, 0.975, "Blue: recomputed from the paper's counts. Orange: as printed in the paper.",
fontsize=10, color=INK2, va="top")
save(fig, "cand_C_calculation_flow")
# ======================================================= Candidate D: stacked bars
def cand_D():
fig, ax = plt.subplots(figsize=(W, 2.9))
rows = ["Abstract: ≥ 1 biomarker", "Results: ≥ 1 biomarker",
"Phase subtotals (recomputed)"]
y = [2, 1, 0]
h = 0.55
ax.barh(y[0], ABS_BIO_ANY, height=h, color=ORANGE)
ax.barh(y[1], RES_BIO_ANY, height=h, color=ORANGE)
ax.barh(y[2], PH2_BIO_ANY, height=h, color=BLUE, edgecolor=BG, linewidth=2)
ax.barh(y[2], PH34_BIO_ANY, left=PH2_BIO_ANY, height=h, color=BLUE, alpha=0.6,
edgecolor=BG, linewidth=2)
ax.text(PH2_BIO_ANY / 2, y[2], "phase II 79", ha="center", va="center", color="white",
fontsize=10, weight="bold")
ax.text(PH2_BIO_ANY + PH34_BIO_ANY / 2, y[2], "III–IV 55", ha="center", va="center",
color=INK, fontsize=10, weight="bold")
for yy, v in zip(y, [ABS_BIO_ANY, RES_BIO_ANY, ph_sum]):
ax.text(v + 3, yy, f"{v} ({pct(v):.1f}%)", va="center", fontsize=10.5)
ax.axvline(N / 2, color=INK2, lw=1, ls=(0, (3, 3)))
ax.text(N / 2 + 1.5, 2.55, "half of 238", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(y, rows, fontsize=10.5)
ax.set_xlim(0, N); ax.set_ylim(-0.5, 2.9)
ax.set_xticks([0, 50, 100, 150, 200, 238]); ax.set_xlabel("Trials (of 238)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
hs = [plt.Rectangle((0, 0), 1, 1, color=ORANGE, label="Reported in the paper"),
plt.Rectangle((0, 0), 1, 1, color=BLUE, label="Recomputed from the paper's own counts")]
ax.legend(handles=hs, loc="upper center", bbox_to_anchor=(0.35, -0.28), ncol=2,
frameon=False, fontsize=10)
save(fig, "cand_D_stacked_bars")
# ======================================================= Candidate E: percentage strip
def cand_E():
fig, ax = plt.subplots(figsize=(W, 2.8))
yb, yc = 1, 0
ax.plot([0, 100], [yb, yb], color=GRID, lw=2, zorder=0)
ax.plot([0, 100], [yc, yc], color=GRID, lw=2, zorder=0)
ax.plot(ABS_BIO_PCT, yb, "o", ms=11, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(RES_BIO_PCT, yb, "o", ms=11, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.plot(pct(ph_sum), yb - 0.17, "D", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.plot(pct(RES_CLIN_PRIM), yc, "o", ms=11, color=ORANGE, mec=BG, mew=1.5, zorder=3)
ax.text(ABS_BIO_PCT - 1, yb + 0.16, "Abstract 73.5%", ha="left", va="bottom", fontsize=10.5)
ax.text(RES_BIO_PCT + 1, yb + 0.16, "Results 56.3%", ha="right", va="bottom", fontsize=10.5,
bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
ax.text(pct(ph_sum) - 1.8, yb - 0.17, f"phases 79 + 55 = {pct(ph_sum):.1f}%",
ha="right", va="center", fontsize=10.5, color=INK,
bbox=dict(fc=BG, ec="none", pad=1), zorder=4)
ax.text(pct(RES_CLIN_PRIM), yc - 0.18, "Results 73.5% (175/238)", ha="center", va="top",
fontsize=10.5)
ax.plot([ABS_BIO_PCT, ABS_BIO_PCT], [yc + 0.1, yb - 0.1], color=INK2, lw=1.2,
ls=(0, (2, 2)))
ax.text(ABS_BIO_PCT + 1.2, 0.5, "identical: 175/238", fontsize=10, color=INK2, va="center")
ax.axvline(50, color=INK2, lw=1, ls=(0, (3, 3)), zorder=0)
ax.text(49, -0.45, "50%: Discussion's “more than half”", ha="right", fontsize=10,
color=INK2, va="center")
ax.set_yticks([yb, yc], ["≥ 1 biomarker,\nany use", "Clinical outcome\nas primary"],
fontsize=10.5)
ax.set_xlim(0, 100); ax.set_ylim(-0.65, 1.55)
ax.set_xticks(range(0, 101, 25)); ax.set_xticklabels([f"{t}%" for t in range(0, 101, 25)])
ax.set_xlabel("Share of the 238 included trials")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=legend_handles(), loc="upper center", bbox_to_anchor=(0.4, -0.3),
ncol=2, frameon=False, fontsize=10)
save(fig, "cand_E_percentage_strip")
for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
f()
# Provisional choice for the write-up (pending the reader's pick):
# fig1 = candidate A (text panels), fig2 = candidate B (dot comparison).
FIG1, FIG2 = "cand_A_text_panels", "cand_B_dot_comparison"
for src, dst in ((FIG1, "fig1"), (FIG2, "fig2")):
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"{src}.{ext}"), os.path.join(HERE, f"{dst}.{ext}"))
print("\nFigures written to", HERE, "and", CAND)
Abstract swaps which CEACAM1 variant the Results report as strongly overexpressed in lung tumors
Nucleic Acids Res (2008) · doi:10.1093/nar/gkn697 · PMC2582617
Summary
The abstract states "overexpression of CEACAM1 variant 1 but not variant 2 in lung tumors". The paper's Results report the opposite emphasis for the same 35 paired lung adenocarcinomas: variant 2 was "highly and significantly overexpressed" (tumor/normal ratio 36.42, P = 0.000002), and variant 1 was "only slightly overexpressed" (ratio 3.53, P = 0.0014). By the paper's own numbers both variants are significantly higher in tumor, and the variant 2 ratio is 10.3 times the variant 1 ratio. The exon-array pattern, the survival hypothesis in the Results and the Discussion's "switch in abundance from the L form (variant 1) to the S form (variant 2)" all point the same way as the Results. The abstract therefore misstates which CEACAM1 isoform rises in lung tumors. This matters because the paper presents the isoforms as having different biology, with the L form described as tumor-suppressive.
What the paper reports
Abstract:
We found overexpression of ERG variant 2 but not variant 1 in lung tumors and overexpression of CEACAM1 variant 1 but not variant 2 in lung tumors but not in breast or colon tumors.
Results, "Quantification of CEACAM1 transcript variants in NSCLC", first paragraph (variant definitions and exon-array pattern):
CEACAM1 is encoded by nine exons with two known variants in the RefSeq database. Variant 1 uses all nine exons while variant 2 is missing exon 7 (Figure 4c). Our array data demonstrated higher expression of exons 1 through 6 as well as 8 and 9 in tumors versus normal while expression of exon 7 was essentially equal (Supplementary Figures).
Results, same section, second paragraph (variant-specific qRT-PCR):
In 35 lung adenocarcinoma patients we found that CEACAM1 variant 2 was highly and significantly overexpressed in paired tumor versus normal (tumor/normal ratio = 36.42; P = 0.000002) while variant 1 was only slightly overexpressed in tumor (tumor/normal ratio = 3.53; P = 0.0014).
In the breast and colon tissues however, we found no significant differences in expression of either variant 1 or variant 2 in tumor versus normal (unmatched) samples (Figure 4b).
Given this data we postulated that overexpression of CEACAM1 variant 2 in lung cancer may be responsible for the survival differences observed between tumor types.
Figure 4 legend, panel (c):
Exon location and PCR primers locations for qPCR of variant 1 (NM_001712; primers 1 and 2), variant 2 (NM_001024912; primers 3 and 4).
Discussion, CEACAM1 paragraph:
Two common variants are known for CEACAM1; one with a long cytoplasmic domain (L form or variant 1) and one with a short cytoplasmic domain (S form or variant 2).
Our quantitative analysis clearly demonstrates a switch in abundance from the L form (variant 1) to the S form (variant 2) in NSCLC and we also demonstrate that no such switch appears to occur in breast cancer or colon cancer.
For comparison, the ERG clause of the same abstract sentence against its own Results ("Differential expression of ERG splice variants in lung adenocarcinoma"):
We observed that ERG variant 2 was significantly overexpressed (mean tumor/normal ratio 4.73; paired t-test P-value = 0.0005) in tumor samples compared to paired normal while ERG variant 1 was slightly under-expressed in tumor samples compared to normal (ratio = 0.74; P = 0.0186).
What the paper's own numbers show
Method. The individual qPCR measurements are not published, so the paired tests cannot be re-run. The check reads the reported tumor/normal ratios and P-values against the abstract's statement. A variant counts as "overexpressed" when its ratio is above 1 and P < 0.05, the threshold the paper uses throughout. I then compared the size of the two CEACAM1 changes (ratio of ratios and log2 fold change), checked which variant the exon-array pattern implies, and ran the same check on the ERG clause in the same abstract sentence as a control. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CEACAM1 variant 2 (S form, lacks exon 7), lung, 35 pairs | Results, CEACAM1 section; Fig. 4a | ratio 36.42, P = 0.000002 | log2 fold change +5.19; −log10 P = 5.70; significantly overexpressed. The abstract says "not variant 2": conflicts |
| CEACAM1 variant 1 (L form, all nine exons), lung, 35 pairs | Results, CEACAM1 section; Fig. 4a | ratio 3.53, P = 0.0014 ("only slightly overexpressed") | log2 fold change +1.82; −log10 P = 2.85; significantly overexpressed. The abstract says overexpressed: agrees |
| Size of the variant 2 change relative to variant 1 | Results, CEACAM1 section | not stated | 36.42 / 3.53 = 10.32 (log2 difference 3.37); P-values differ by a factor of 700 |
| Exon-array pattern | Results, CEACAM1 section, first paragraph | exons 1–6, 8, 9 higher in tumor; exon 7 "essentially equal" | exon 7 occurs only in variant 1, so a flat exon 7 with rising shared exons points to a rise in variant 2: agrees with the Results, not with the abstract |
| Direction stated elsewhere | Results (survival hypothesis); Discussion | "overexpression of CEACAM1 variant 2"; "switch … from the L form (variant 1) to the S form (variant 2)" | both say variant 2 rises in lung tumors |
| Breast and colon, both variants | Results; Fig. 4b | no significant difference | consistent with the abstract's "but not in breast or colon tumors" |
| ERG variant 2, lung, 35 pairs (control) | Results, ERG section | ratio 4.73, P = 0.0005 | log2 +2.24; significantly overexpressed. The abstract says overexpressed: agrees |
| ERG variant 1, lung, 35 pairs (control) | Results, ERG section | ratio 0.74, P = 0.0186 | log2 −0.43; significantly under-expressed. The abstract says not overexpressed: agrees |
The ratio of ratios compares the size of each variant's tumor/normal change. It does not compare the absolute abundance of the two transcripts.
Figure
Figure 1. The left panel holds the abstract's CEACAM1 clause and what it states for lung tumors. The right panel holds the Results and Discussion sentences and the variant-specific tumor/normal ratios with their P-values, as printed (orange). The connector marks the conflict: the abstract names variant 1 as the overexpressed isoform and excludes variant 2, while the body reports variant 2 as the strongly overexpressed isoform. The blue lines are read from the paper's own numbers. Both variants have P < 0.05, the variant 2 ratio is 10.3 times the variant 1 ratio, and the exon-array pattern also points to variant 2.
Figure 2. Tumor/normal expression ratios reported in the Results for the 35 paired lung samples (orange dots, log scale; the thin vertical line at 1 means no change), with the reported P-values. The right column gives what the abstract says for each variant and whether that agrees with the reported ratio and P-value (blue; overexpressed means ratio > 1 and P < 0.05). Three of the four clauses in the abstract sentence agree with the body. The one that conflicts is the statement that CEACAM1 variant 2, the variant with the largest and most significant increase, is not overexpressed.
Why it matters
- It names the wrong isoform. The abstract identifies the L form (variant 1) as the CEACAM1 isoform overexpressed in lung tumors and excludes the S form (variant 2). The body reports that the S form has the large increase (36.42-fold, P = 0.000002) and the L form a modest one (3.53-fold, P = 0.0014). The abstract's "not variant 2" is wrong, and its "variant 1" is only the smaller of two significant increases.
- The isoforms are presented as biologically different. The Discussion describes the L form as having "a tumor suppressive phenotype" and frames the finding as an isoform switch toward the S form in lung cancer, consistent with the earlier semi-quantitative report it cites. A reader of the abstract alone would conclude the reverse: that the tumor-suppressive form predominates in the increase in lung tumors.
- It is one of the abstract's few gene-level findings. The abstract highlights four genes (ERG, CEACAM1, CDH3, P16). Readers, indexing services and later citations that rely on the abstract inherit the reversed CEACAM1 statement, and nothing in the abstract signals the conflict.
- What it does not affect. The genome-wide result (2369 genes with differentially expressed alternative transcripts), the ERG, CDH3 and P16 findings, the breast and colon CEACAM1 result, and the CEACAM1 numbers, Figure 4 and the Discussion are all internally consistent. Only the abstract sentence is affected.
Explanations considered
- A different variant numbering. Ruled out. The Results define variant 1 as using all nine exons and variant 2 as lacking exon 7. The Figure 4c legend gives variant 1 as NM_001712 and variant 2 as NM_001024912. The Discussion gives variant 1 as the L form and variant 2 as the S form. The labels are used the same way in every part of the body, and under that numbering the body says variant 2 is the strongly overexpressed one.
- The abstract refers to a different population. Not supported. The abstract clause is about lung tumors, and so are the 35 paired adenocarcinomas in the Results. The additional 43 unpaired lung tumors are described as "indistinguishable from the original 35 adenocarcinoma samples". Breast and colon were analyzed separately, and the abstract gets that comparison right.
- The ratio is defined the other way round (normal/tumor). Ruled out. The Results call the quantity a "tumor/normal ratio" and say variant 2 "was highly and significantly overexpressed in paired tumor". The exon-array evidence (exon 7 flat, the other exons up in tumor) gives the same direction independently.
- "Not variant 2" means "not significantly". Ruled out. The variant 2 P-value, 0.000002, is the smallest of the four variant-specific P-values reported for lung, so variant 2 is not a non-significant result under any threshold the paper uses.
- The whole abstract sentence is garbled. Not supported. The ERG clause of the same sentence ("ERG variant 2 but not variant 1") matches its own Results (4.73, P = 0.0005; 0.74, P = 0.0186). The mismatch is specific to the CEACAM1 clause, where exchanging "variant 1" and "variant 2" would make the clause match the body.
- A conversion or archiving artifact. Ruled out. The publisher's version of record carries the same abstract wording and the same Results numbers as the PMC archive.
- Already corrected. No correction, erratum or other notice was found (see Status).
Status
As of 2026-10-24:
- Version of record. The DOI resolves to the publisher's version of record (academic.oup.com). Its abstract has the same sentence ("overexpression of CEACAM1 variant 1 but not variant 2 in lung tumors but not in breast or colon tumors"), and its Results have the same CEACAM1 numbers (36.42, P = 0.000002; 3.53, P = 0.0014). The PMC archive used here matches the publisher page word for word at both places. The PMC web page (behind a reCAPTCHA) and the Europe PMC full-text XML (HTTP 500) could not be opened.
- Supplementary material. The publisher's supplementary archive (four files: Supplementary Methods and primer design, Tables S1 and S2, and a 12-page Supplementary Figures PDF) was read. For CEACAM1 it holds only the exon-level array data, the exon 7 versus exon 8 qPCR verification and the primer design. The variant-specific tumor/normal results appear only in main-text Figure 4. Nothing in the supplement supports the abstract's wording. The array data are deposited in GEO (GSE12236) and were not needed, because the conflict is between two statements of the same qPCR result.
- Corrections and notices. Crossref (no update-to or updated-by relations), PMC and Europe PMC notice records list no correction, erratum, expression of concern or retraction. The publisher's article page shows no correction notice. Web searches found no erratum.
- PubPeer. The PubPeer API returned 403 to automated requests, so a direct search could not be completed. Web searches found no PubPeer thread on this article.
- Sources. The article as archived in PMC (PMC2582617); the publisher's version of record and supplementary archive; Crossref, PMC and Europe PMC records.
Suggested correction
In the abstract, change the CEACAM1 clause so that it names the variant reported in the Results, Figure 4 and the Discussion, for example:
We found overexpression of ERG variant 2 but not variant 1 in lung tumors and overexpression of CEACAM1 variant 2 but not variant 1 in lung tumors but not in breast or colon tumors.
Because variant 1 was also significantly, though modestly, overexpressed (3.53-fold, P = 0.0014), a more exact wording would be "marked overexpression of CEACAM1 variant 2 (S form), with only slight overexpression of variant 1 (L form), in lung tumors but not in breast or colon tumors". No change to the numbers, figures or Discussion appears to be needed.
Recomputation script (python recompute.py.txt)
"""Recompute the CEACAM1 variant results in PMC2582617
(Nucleic Acids Res 2008, doi:10.1093/nar/gkn697).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PubMed Central:
Abstract: "overexpression of ERG variant 2 but not variant 1 in lung tumors and
overexpression of CEACAM1 variant 1 but not variant 2 in lung tumors but not in breast
or colon tumors."
Results > Quantification of CEACAM1 transcript variants in NSCLC: 35 paired lung
adenocarcinomas, variant 2 tumor/normal = 36.42 (P = 0.000002), variant 1 = 3.53
(P = 0.0014); breast and colon (10 unmatched each): no significant difference.
Results > Differential expression of ERG splice variants: 35 pairs, variant 2 mean
tumor/normal = 4.73 (paired t-test P = 0.0005), variant 1 = 0.74 (P = 0.0186).
Results / Figure 4c legend: variant 1 = NM_001712, all nine exons; variant 2 =
NM_001024912, lacks exon 7. Discussion: variant 1 = L form, variant 2 = S form.
Individual qPCR values are not published, so the tests themselves cannot be re-run. The
checks below read the reported ratios and P-values against the abstract's statement.
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.transforms import blended_transform_factory # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ALPHA = 0.05
# ---------------------------------------------------------------- inputs (from the paper)
# gene -> variant -> (tumor/normal ratio, P), 35 paired lung tumor/normal samples
RES = {
"CEACAM1": {"v1": (3.53, 0.0014), "v2": (36.42, 0.000002)},
"ERG": {"v1": (0.74, 0.0186), "v2": (4.73, 0.0005)},
}
# what the abstract says for lung tumors: True = "overexpressed", False = "not"
ABSTRACT = {
"CEACAM1": {"v1": True, "v2": False},
"ERG": {"v1": False, "v2": True},
}
CEACAM1_EXONS = {"v1": set(range(1, 10)), "v2": set(range(1, 10)) - {7}}
ARRAY_UP = set(range(1, 7)) | {8, 9} # exons higher in tumor (Results, p0)
ARRAY_FLAT = {7} # exon 7 "essentially equal"
def call(r, p):
if p < ALPHA and r > 1:
return "significantly over-expressed"
if p < ALPHA and r < 1:
return "significantly under-expressed"
return "not significant"
def main():
R = {}
print("== 1. Reported tumor/normal ratios in 35 paired lung samples, read at alpha = 0.05")
for g, d in RES.items():
for v, (r, p) in d.items():
l2 = math.log2(r)
mlp = -math.log10(p)
over = r > 1 and p < ALPHA
R[(g, v)] = (r, p, l2, mlp, over)
ab = "overexpressed" if ABSTRACT[g][v] else "not overexpressed"
agree = over == ABSTRACT[g][v]
print(f" {g:8s} {v}: ratio {r:6.2f} log2 {l2:+.2f} P {p:.6f} -log10 P {mlp:.2f}"
f" -> {call(r, p):30s} abstract: {ab:18s} {'AGREES' if agree else 'CONFLICTS'}")
for g in RES:
(r1, p1), (r2, p2) = RES[g]["v1"], RES[g]["v2"]
print(f" {g}: v2 ratio / v1 ratio = {r2 / r1:.2f}; log2 difference = "
f"{math.log2(r2) - math.log2(r1):.2f}; P(v1)/P(v2) = {p1 / p2:.0f}")
print(" (The ratio of fold changes compares the size of the tumor/normal change for each")
print(" variant; it does not compare the absolute abundance of the two transcripts.)")
print("\n== 2. What the abstract's CEACAM1 statement would require")
r1, p1 = RES["CEACAM1"]["v1"]
r2, p2 = RES["CEACAM1"]["v2"]
print(f" 'v1 overexpressed': needs v1 ratio > 1 and P < 0.05 -> {r1} and {p1}: "
f"{r1 > 1 and p1 < ALPHA}")
print(f" 'not v2': needs v2 ratio <= 1 or P >= 0.05 -> {r2} and {p2}: "
f"{not (r2 > 1 and p2 < ALPHA)}")
print(" Swapping the variant numbers ('variant 2 but not variant 1') matches the larger,")
print(f" more significant change ({r2} vs {r1}); variant 1 is still significant (P = {p1}),")
print(" which the Results describe as 'only slightly overexpressed'.")
print("\n== 3. Exon-array pattern (Results): which variant can carry the tumor increase?")
for v, ex in CEACAM1_EXONS.items():
print(f" {v}: exons {sorted(ex)}; contains exon 7: {7 in ex}")
print(f" Exons higher in tumor: {sorted(ARRAY_UP)}; exon essentially equal: {sorted(ARRAY_FLAT)}")
print(" Exon 7 occurs only in v1, so its signal tracks v1; it is flat, while exons shared by")
print(" both variants rise. The pattern points to a rise in the exon-7-lacking v2.")
print(" Direction implied by the array: v2 up -> agrees with Results, not with the abstract.")
print("\n== 4. Other sentences in the paper, as direction checks")
print(" Results: 'overexpression of CEACAM1 variant 2 in lung cancer' (survival hypothesis) -> v2 up")
print(" Discussion: 'switch in abundance from the L form (variant 1) to the S form (variant 2)' -> v2 up")
print(" Breast and colon: no significant difference for either variant -> abstract's tissue contrast holds")
save(fig1_text_panels(R), os.path.join(HERE, "fig1"))
save(fig2_ratios(R), os.path.join(HERE, "fig2"))
print("\nFigures written: fig1, fig2 (.svg, .png)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
PANEL = "#f3f2ef"
plt.rcParams.update({
"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT,
"ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "font.family": "DejaVu Sans",
})
W = 7.2
def save(fig, stem):
fig.savefig(stem + ".svg")
fig.savefig(stem + ".png", dpi=200)
plt.close(fig)
def box(ax, x, y, w, h, edge=SEC, face=PANEL, lw=1.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=face, ec=edge, lw=lw, transform=ax.transAxes))
def lines(ax, x, y, items, dy=0.05):
"""items: list of (text, color, weight, size)."""
for txt, col, wt, sz in items:
ax.text(x, y, txt, color=col, fontweight=wt, fontsize=sz, va="top", ha="left",
transform=ax.transAxes)
y -= dy * sz / 10.5
return y
def fig1_text_panels(R):
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.03, 0.968, "The abstract names the opposite CEACAM1 variant to the Results",
fontsize=12, fontweight="bold", va="top")
ax.text(0.03, 0.915, "Lung tumors, 35 tumor/normal pairs, variant-specific qRT-PCR",
fontsize=10, color=SEC, va="top")
# left panel: abstract
box(ax, 0.03, 0.285, 0.385, 0.585)
lines(ax, 0.05, 0.845, [
("ABSTRACT", SEC, "bold", 10),
("“…overexpression of", TXT, "normal", 10.5),
("CEACAM1 variant 1 but", TXT, "bold", 10.5),
("not variant 2 in lung", TXT, "bold", 10.5),
("tumors but not in breast", TXT, "normal", 10.5),
("or colon tumors.”", TXT, "normal", 10.5),
])
lines(ax, 0.05, 0.505, [
("Stated for lung tumors:", SEC, "normal", 10),
("variant 1 (L form): up", ORANGE, "bold", 10.5),
("variant 2 (S form): not up", ORANGE, "bold", 10.5),
])
# right panel: results + discussion
box(ax, 0.55, 0.285, 0.425, 0.585)
lines(ax, 0.57, 0.845, [
("RESULTS (FIG. 4) AND DISCUSSION", SEC, "bold", 10),
("“CEACAM1 variant 2 was highly", TXT, "bold", 10.5),
("and significantly overexpressed”", TXT, "bold", 10.5),
("“variant 1 was only slightly", TXT, "normal", 10.5),
("overexpressed”", TXT, "normal", 10.5),
])
lines(ax, 0.57, 0.585, [
("Tumor/normal ratio (P):", SEC, "normal", 10),
("variant 2 36.42 (0.000002)", ORANGE, "bold", 10.5),
("variant 1 3.53 (0.0014)", ORANGE, "bold", 10.5),
("Discussion: “a switch in abundance", TXT, "normal", 10),
("from the L form (variant 1) to the", TXT, "normal", 10),
("S form (variant 2) in NSCLC”", TXT, "normal", 10),
])
ar = FancyArrowPatch((0.42, 0.62), (0.545, 0.62), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT, transform=ax.transAxes)
ax.add_patch(ar)
ax.text(0.4825, 0.665, "variants", ha="center", fontsize=10, fontweight="bold")
ax.text(0.4825, 0.595, "swapped", ha="center", va="top", fontsize=10, fontweight="bold")
# recomputed strip
r1, p1, l1, m1, o1 = R[("CEACAM1", "v1")]
r2, p2, l2, m2, o2 = R[("CEACAM1", "v2")]
ax.text(0.03, 0.245, "Read from the paper's own numbers:", fontsize=10, color=SEC, va="top")
ax.text(0.03, 0.195, f"Both variants P < 0.05; variant 2 ratio = {r2 / r1:.1f}\u00d7 variant 1 ratio "
f"(log2 {l2:+.2f} vs {l1:+.2f})", fontsize=10, color=BLUE,
fontweight="bold", va="top")
ax.text(0.03, 0.145, "Exon 7 (present only in variant 1) is flat on the array while the "
"shared exons rise,", fontsize=10, color=BLUE, va="top")
ax.text(0.03, 0.105, "which also points to variant 2. The ERG clause in the same abstract "
"sentence agrees.", fontsize=10, color=BLUE, va="top")
ax.text(0.97, 0.012, "orange = printed in the paper blue = derived from its numbers",
fontsize=10, color=SEC, ha="right", va="bottom")
return fig
def fig2_ratios(R):
rows = [("CEACAM1", "v2", "CEACAM1 variant 2\n(S form, lacks exon 7)"),
("CEACAM1", "v1", "CEACAM1 variant 1\n(L form, all 9 exons)"),
("ERG", "v2", "ERG variant 2"),
("ERG", "v1", "ERG variant 1")]
PSTR = {0.000002: "0.000002", 0.0014: "0.0014", 0.0005: "0.0005", 0.0186: "0.0186"}
fig, ax = plt.subplots(figsize=(W, 3.9))
fig.subplots_adjust(left=0.235, right=0.56, top=0.78, bottom=0.16)
ys = [3.3, 2.3, 1.0, 0.0]
ax.axvline(1, color=SEC, lw=0.8)
ax.axhline(1.65, color=GRID, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.4, 100)
ax.set_ylim(-0.5, 3.95)
tr = blended_transform_factory(ax.transAxes, ax.transData)
for (g, v, lab), y in zip(rows, ys):
r, p = R[(g, v)][:2]
ax.plot([r], [y], "o", color=ORANGE, ms=9, zorder=3)
ax.text(r, y + 0.22, f"{r}", ha="center", va="bottom", fontsize=10, color=ORANGE,
fontweight="bold")
ab = "overexpressed" if ABSTRACT[g][v] else "not overexpressed"
agree = R[(g, v)][4] == ABSTRACT[g][v]
ax.text(1.05, y, f"P = {PSTR[p]}", va="center", fontsize=10, color=ORANGE,
transform=tr, clip_on=False)
ax.text(1.47, y + 0.17, ab, va="center", fontsize=10, color=TXT, transform=tr,
clip_on=False)
ax.text(1.47, y - 0.22, "agrees" if agree else "CONFLICTS", va="center", fontsize=10,
color=BLUE, fontweight=("normal" if agree else "bold"), transform=tr,
clip_on=False)
ax.text(1.05, 3.95, "Results", fontsize=10, color=SEC, fontweight="bold", va="bottom",
transform=tr, clip_on=False)
ax.text(1.47, 3.95, "Abstract / check", fontsize=10, color=SEC, fontweight="bold",
va="bottom", transform=tr, clip_on=False)
ax.set_yticks(ys)
ax.set_yticklabels([r[2] for r in rows], fontsize=10)
ax.set_xticks([0.5, 1, 2, 5, 10, 20, 50, 100])
ax.set_xticklabels(["0.5", "1", "2", "5", "10", "20", "50", "100"])
ax.minorticks_off()
ax.set_xlabel("Tumor/normal ratio, 35 lung pairs\n(log scale; 1 = no change)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.text(0.03, 0.965, "Reported lung ratios against the abstract's statement",
fontsize=12, fontweight="bold", va="top")
fig.text(0.03, 0.905, "Orange: printed in the Results. Blue: check of the abstract "
"against them (P < 0.05, ratio > 1).", fontsize=10, color=SEC, va="top")
return fig
if __name__ == "__main__":
main()
Abstract pairs Ala510 with dA and Asn513 with dC; the paper's Results and Conclusion pair them the other way round
Nucleic Acids Res (2015) · doi:10.1093/nar/gkv760 · PMC4652769
Summary
The abstract states that "Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dA and dC, respectively". The body of the paper states the reverse pairing three times. The N513A paragraph of the Results concludes that "Asn513 plays an important role in facilitating 8-oxo-dGMP incorporation opposite dA". The Conclusion states that "Ala510 selectively enhances the incorporation of 8-oxo-dGMP opposite dC". The double-mutant paragraph describes "the opposite effects of Ala510 and Asn513, facilitating incorporation of the anti- and syn-conformations, respectively", and the paper ties the syn conformation to dA and the anti conformation to dC. The contradiction is between the abstract and the paper's own text, and that internal contradiction is the main basis of this report.
Table 1 supports the body text's assignment for Ala510 under both ways of summarising the data that I checked. For Asn513 the data support the body text (dA) on the absolute metric and are about even on the alternative metric; on neither metric do they support the abstract's dC assignment. The underlying kinetics and the Results text are internally consistent; the error is confined to one sentence of the abstract.
What the paper reports
Abstract (as archived in PMC):
Steady-state kinetics reveal that two residues lining the dNTP binding pocket, Ala510 and Asn513, play differential roles in dNTP selectivity. Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dA and dC, respectively.
Results and Discussion, N513A paragraph:
However, the N513A substitution resulted in a much more drastic effect in the context of 8-oxo-dGMP incorporation opposite dA (25-fold, Figure 3D). Also, the catalytic efficiencies of dGMP and 8-oxo-dGMP incorporation opposite dC were reduced by 1.3 (Figure 3E) and 6.4-fold (Figure 3F) respectively. Taken together, these results suggest that Asn513 plays an important role in facilitating 8-oxo-dGMP incorporation opposite dA, implying that this residue selectively stabilizes the syn-conformation of 8-oxo-dGTP.
Results and Discussion, A510D paragraph:
Notably, the A510D substitution had the largest effect on the incorporation of 8-oxo-dGTP opposite dC (7.6-fold, Figure 3E) as opposed to dGTP (2.8-fold, Figure 3F). This suggests that Ala510 may preferentially stabilize the anti-conformation of 8-oxo-dGTP, promoting its incorporation opposite dC.
Results and Discussion, double-mutant paragraph:
Interestingly, the drop in catalytic efficiency was nearly identical for 8-oxo-dGMP incorporation opposite dA and dC. This is consistent with the opposite effects of Ala510 and Asn513, facilitating incorporation of the anti- and syn-conformations, respectively.
Conclusion:
As a consequence, Ala510 selectively enhances the incorporation of 8-oxo-dGMP opposite dC.
So the Results, the double-mutant interpretation and the Conclusion all assign Asn513 to the dA (syn) context and Ala510 to the dC (anti) context. The abstract's sentence assigns them the other way round.
What the paper's own numbers show
Method. Table 1 gives the catalytic efficiency kcat/KM (min⁻¹ µM⁻¹, mean ± SD of three independent experiments) for WT and each mutant, for the correct dNTP and for 8-oxo-dGTP, opposite both dA and dC. From those values I recomputed the fold loss of kcat/KM caused by each substitution (WT ÷ mutant) and compared it with the fold values printed in the Results text. I summarise each residue's template preference two ways, because the answer for Asn513 depends on which one is used:
- Metric 1, absolute fold loss. WT kcat/KM ÷ mutant kcat/KM for 8-oxo-dGTP, in each template context. This is the metric the abstract's wording ("facilitate incorporation ... opposite dA / dC") most directly implies, and the one on which the "reversed pairing" holds cleanly.
- Metric 2, loss relative to the correct dNTP. The 8-oxo-dGTP fold loss divided by the fold loss for the correct nucleotide (dTTP opposite dA, dGTP opposite dC) in the same context. This is the comparison the paper itself makes when it argues each residue's role, e.g. A510D 7.6-fold on 8-oxo-dGTP versus 2.8-fold on dGTP opposite dC.
Approximate SDs are first-order (delta-method) propagations of the Table 1 SDs on the log scale, treating WT and mutant as independent; with n = 3 per estimate they are a rough guide only. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from Table 1 |
|---|---|---|---|
| A510D fold loss, 8-oxo-dGTP opposite dA | Results | 5.1-fold | 5.11× (SD ~0.65) |
| A510D fold loss, 8-oxo-dGTP opposite dC | Results | 7.6-fold | 7.57× (SD ~1.67) |
| N513A fold loss, 8-oxo-dGTP opposite dA | Results | 25-fold | 24.74× (SD ~3.19) |
| N513A fold loss, 8-oxo-dGTP opposite dC | Results | 6.4-fold | 6.39× (SD ~0.71) |
| A510D fold loss, dTTP opposite dA | Results | 4.8-fold | 4.78× |
| A510D fold loss, dGTP opposite dC | Results | 2.8-fold | 2.76× |
| N513A fold loss, dTTP opposite dA | Results | 5.8-fold | 5.82× |
| N513A fold loss, dGTP opposite dC | Results | 1.3-fold | 1.33× |
The Results-text values and the values recomputed from Table 1 agree throughout, so the contradiction is not between the Results and the table; it is between the abstract and the rest of the paper.
Which context does each residue affect more?
| Residue | Metric 1 (absolute): dA vs dC | larger at | Metric 2 (÷ correct dNTP): dA vs dC | larger at |
|---|---|---|---|---|
| Ala510 (A510D) | 5.11× vs 7.57× | dC (ratio 1.48; |log diff|/SD ~1.5) | 1.07 vs 2.74 | dC (ratio 2.57; |log diff|/SD ~2.6) |
| Asn513 (N513A) | 24.74× vs 6.39× | dA (ratio 3.87; |log diff|/SD ~7.9) | 4.25 vs 4.79 | about equal (ratio 1.13; |log diff|/SD ~0.4) |
Reading. On the absolute metric the pairing is the clean reverse of the abstract: Ala510's larger effect is opposite dC, Asn513's is opposite dA (and the Asn513 gap is large, ~8 approximate SDs). On the relative metric Ala510 is again larger opposite dC, by a clear margin, while Asn513 is essentially even between the two contexts (4.25 vs 4.79, well under one approximate SD apart). So:
- The abstract's Ala510 → dA assignment is contradicted on both metrics.
- The abstract's Asn513 → dC assignment is contradicted on the absolute metric and is, at best, not supported on the relative metric (the point estimate there is slightly larger opposite dC, but the difference is within noise). It is the body text's Asn513 → dA assignment, not the abstract's, that the absolute metric supports.
A consistency check favours the body text. For the double mutant the fold loss is nearly identical opposite dA (55.8×) and dC (58.9×), which the paper reads as Ala510 and Asn513 having opposite template preferences that cancel. That reading requires Asn513 → dA and Ala510 → dC, i.e. the body text, not the abstract. The Results also note that removing Asn513 cripples 8-oxo-dGTP opposite dA (25-fold) far more than it does the correct dTTP in the same context (5.8-fold), which is the paper's own argument that Asn513 selectively helps the dA (syn) mispair.
Figure
Figure 1. The abstract's sentence (left, orange) pairs Ala510 with the dA context and Asn513 with the dC context. The Results, Conclusion and double-mutant paragraphs (right) pair them the other way: Asn513 with dA (syn), Ala510 with dC (anti). The strip below lists, recomputed from Table 1, the fold loss of kcat/KM for 8-oxo-dGTP in each context, both as the absolute loss and divided by the loss for the correct dNTP (blue). Blue values are recomputed by recompute.py.txt; the abstract gives no numbers. The Results text prints the same absolute values, rounded.
Design candidates for review.
recompute.py.txtalso writes five candidate layouts incandidates/(cand1–cand5).fig1is currently a copy ofcand1(the two-panel text diagram) andfig2a copy ofcand3(a one-axis log plot of the fold losses). The final figure choice is open until the reader picks from the five candidates.
Why it matters
- The error is in the abstract, and inverts a headline mechanistic claim. The abstract's differential-role sentence is the paper's one-line summary of its kinetics. Readers, indexers and citing authors who see only the abstract take away the reverse of what the Results, Conclusion and Table 1 support: that Ala510 helps 8-oxo-dGMP opposite dA and Asn513 opposite dC.
- The directions connect to different conformations. The paper links the dA mispair to the syn conformation (stabilised by the Asn513 hydrogen bond to the C8-carbonyl) and the dC pairing to the anti conformation (where the A510D paragraph invokes an unfavourable electrostatic contact). Swapping the residues swaps which conformation each is said to stabilise, which is a mechanistic claim, not just a labelling slip.
- The data and the analysis are not affected. Table 1, the Results text and the Conclusion are internally consistent. Only the abstract sentence is wrong, so the correction is a one-sentence fix.
Explanations considered
- A reader could pick the abstract's pairing out of the numbers on some metric. Not supported. On the absolute fold loss the pairing is the clean reverse of the abstract. On the loss relative to the correct dNTP, Ala510 is still clearly larger opposite dC, and Asn513 is about even (4.25 vs 4.79) rather than larger opposite dC. Neither metric gives Ala510 → dA, and neither gives a clear Asn513 → dC. The abstract's own body text states the reverse pairing explicitly in three places.
- "Respectively" was meant to map to "dC and dA". Not supported. English "A and B ... opposite dA and dC, respectively" maps A→dA, B→dC, and the sentence names Ala510 before Asn513. Reading it the other way would make it agree with the body, but that is not what the sentence says.
- A different measure of "facilitate". The abstract gives no numbers, so "facilitate ... opposite dA / dC" can only be read as the fold effect of each substitution in each context. Both the metrics I checked place Ala510's larger effect opposite dC; neither supports Ala510 → dA.
- A correction already issued. None found (see Status).
Status
- Checks performed.
- (1) Recompute: the 10 fold values printed in the Results text all match the values recomputed from Table 1 (see the table above);
recompute.py.txtprints them. - (2) Numbers in this write-up: every figure and number drawn here comes from Table 1 or from
recompute.py.txt; none is a new claim. - (4) Tone: the write-up reports the internal contradiction and the metric dependence without speculating about cause or naming any person.
- (5) This Status section records each check and its date.
- (6) Markdown and image links: the figure link points to
fig1.svg, which exists in this folder; the candidate layouts are incandidates/. - Version of record. On 2026-10-23 the abstract sentence and the 16 kcat/KM values of Table 1 were compared with the publisher's HTML page (academic.oup.com, via the DOI) and with the Europe PMC core record (PMID 26220180); the abstract sentence and all 16 values match. Every quotation above was also checked verbatim against the PMC-archived text with
src/show.py. - Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. A web search for the title together with "PubPeer", "correction" and "erratum" returned only the paper and its PDB entries. - Figure-panel citations. This write-up cites Table 1 for all numbers and does not rely on figure-panel letters. The paper's Results label some Figure 3 panels inconsistently between the A510D and N513A paragraphs (e.g. 8-oxo-dGMP opposite dC is cited as Figure 3E in one paragraph and the dGMP/8-oxo-dGMP panels are cited as 3E/3F in another); because this write-up cites Table 1 only, that inconsistency does not affect it.
- Sources. PMC article PMC4652769; publisher page academic.oup.com (doi:10.1093/nar/gkv760); Europe PMC core record (PMID 26220180); Crossref; PMC and Europe PMC notice feeds; https://doi.org/10.1093/nar/gkv760.
Suggested correction
In the abstract, replace
Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dA and dC, respectively.
with
Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP opposite dC and dA, respectively.
(or, equivalently, "Asn513 and Ala510 ... opposite dA and dC, respectively"), so that the abstract agrees with the Results, the Conclusion, the double-mutant interpretation and Table 1.
Recomputation script (python recompute.py.txt)
"""Recompute the Ala510 / Asn513 fold effects of PMC4652769
(Nucleic Acids Res 2015, doi:10.1093/nar/gkv760).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file:
fig1.svg / fig1.png abstract vs body text (two text panels), with Table 1 numbers
fig2.svg / fig2.png fold reductions on one log axis, two metrics, approximate SDs
candidates/cand1..cand5.svg/png five candidate figure designs, for the reader to choose from
All inputs are copied from Table 1 of the article as archived in PubMed Central:
catalytic efficiency kcat/KM in min^-1 uM^-1, mean +/- SD of three independent
experiments. No raw data are used. (The check of these values against the publisher's
page is recorded in the Status section of writeup.md, not here.)
Colour code, the same in every figure:
orange = the pairing stated in the abstract (the abstract gives no numbers)
blue = recomputed from Table 1 by this script
grey = a fold value printed in the Results / Conclusion text (these agree with Table 1)
fig1 and fig2 are provisional copies of candidates 1 and 3 until a design is chosen.
"""
import os
import textwrap
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Rectangle # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (Table 1)
# (mean, SD) of kcat/KM
T1 = {
"A": { # template dA
"dTTP": {"WT": (32, 4), "A510D": (6.7, 0.6), "N513A": (5.5, 1.0), "A510D/N513A": (0.62, 0.07)},
"8oxo": {"WT": (2.4, 0.3), "A510D": (0.47, 0.01), "N513A": (0.097, 0.003), "A510D/N513A": (0.043, 0.005)},
},
"C": { # template dC
"dGTP": {"WT": (80, 10), "A510D": (29, 5), "N513A": (60, 4), "A510D/N513A": (11, 2)},
"8oxo": {"WT": (0.53, 0.05), "A510D": (0.070, 0.014), "N513A": (0.083, 0.005), "A510D/N513A": (0.009, 0.002)},
},
}
NORMAL = {"A": "dTTP", "C": "dGTP"}
# Fold values printed in the Results / Conclusion text
REPORTED = {
("A510D", "A", "8oxo"): 5.1, ("A510D", "C", "8oxo"): 7.6,
("N513A", "A", "8oxo"): 25, ("N513A", "C", "8oxo"): 6.4,
("A510D", "A", "dTTP"): 4.8, ("N513A", "A", "dTTP"): 5.8, ("A510D/N513A", "A", "dTTP"): 52,
("A510D", "C", "dGTP"): 2.8, ("N513A", "C", "dGTP"): 1.3, ("A510D/N513A", "C", "dGTP"): 7.3,
}
ABSTRACT = ("Specifically, Ala510 and Asn513 facilitate incorporation of 8-oxo-dGMP "
"opposite dA and dC, respectively.")
Q_N513 = ("Asn513 plays an important role in facilitating 8-oxo-dGMP incorporation opposite dA")
Q_A510 = ("Ala510 selectively enhances the incorporation of 8-oxo-dGMP opposite dC")
Q_DBL = ("the opposite effects of Ala510 and Asn513, facilitating incorporation of the anti- "
"and syn-conformations, respectively")
def fold(t, nt, mut):
"""Fold decrease in kcat/KM relative to WT, and an approximate SD.
The SD is a first-order (delta-method) propagation of the Table 1 SDs on the log scale,
treating WT and mutant as independent: sd(ratio) ~ ratio * sqrt(cv_wt^2 + cv_mut^2).
With n = 3 per estimate this is a rough guide only."""
(w, sw), (m, sm) = T1[t][nt]["WT"], T1[t][nt][mut]
r = w / m
return r, r * np.hypot(sw / w, sm / m)
def rel(t, mut):
"""Fold loss for 8-oxo-dGTP divided by fold loss for the correct dNTP in the same template
context (the comparison the paper itself makes, e.g. 7.6-fold vs 2.8-fold for A510D at dC).
Approximate SD by the same first-order propagation over the four Table 1 entries."""
r8, s8 = fold(t, "8oxo", mut)
rn, sn = fold(t, NORMAL[t], mut)
r = r8 / rn
return r, r * np.hypot(s8 / r8, sn / rn)
# ------------------------------------------------------------------ 1. fold reductions
print("1. Fold reduction in kcat/KM vs WT (Table 1), with text values")
print(f" {'mutant':12} {'template':8} {'dNTP':10} {'reported':>5} {'recomputed':>11} {'approx SD':>10}")
F = {}
for (mut, t, nt), rep in REPORTED.items():
r, s = fold(t, nt, mut)
F[(mut, t, nt)] = (r, s)
print(f" {mut:12} d{t:7} {('8-oxo-dGTP' if nt == '8oxo' else nt):10} {rep:>5} {r:11.2f} {s:10.2f}")
for t in "AC":
r, s = fold(t, "8oxo", "A510D/N513A")
F[("A510D/N513A", t, "8oxo")] = (r, s)
print(f" {'A510D/N513A':12} d{t:7} {'8-oxo-dGTP':10} {'-':>5} {r:11.2f} {s:10.2f}")
# ------------------------------------------------------------------ 2. WT comparisons
print("\n2. WT comparisons in the Results text")
wt = lambda t, nt: T1[t][nt]["WT"][0] # noqa: E731
print(f" dGMP vs 8-oxo-dGMP opposite dC: reported 150-fold, recomputed {wt('C','dGTP')/wt('C','8oxo'):.2f}")
print(f" dTMP vs 8-oxo-dGMP opposite dA: reported 13-fold, recomputed {wt('A','dTTP')/wt('A','8oxo'):.2f}")
print(f" dA/dC selectivity for 8-oxo-dGTP: reported 5:1, recomputed {wt('A','8oxo')/wt('C','8oxo'):.2f}:1")
# ------------------------------------------------------------------ 3. the pairing, absolute metric
print("\n3. Metric 1 (absolute): fold loss of kcat/KM for 8-oxo-dGTP vs WT, dA vs dC")
pair_abs = {}
for mut, res in (("A510D", "Ala510"), ("N513A", "Asn513")):
a, sa = F[(mut, "A", "8oxo")]
c, sc = F[(mut, "C", "8oxo")]
pair_abs[res] = "dA" if a > c else "dC"
# log-scale difference in units of its approximate SD
z = abs(np.log(a) - np.log(c)) / np.hypot(sa / a, sc / c)
print(f" {res} ({mut}): dA {a:.2f}x (SD ~{sa:.2f}) vs dC {c:.2f}x (SD ~{sc:.2f}); "
f"ratio {max(a, c)/min(a, c):.2f}; larger at {pair_abs[res]}; |log diff|/SD ~ {z:.1f}")
print(f" Larger effect: Ala510 -> {pair_abs['Ala510']}, Asn513 -> {pair_abs['Asn513']}")
# ------------------------------------------------------------------ 4. the pairing, relative metric
print("\n4. Metric 2 (relative): 8-oxo-dGTP fold loss / correct-dNTP fold loss in the same context")
print(" (dTTP opposite dA, dGTP opposite dC; the comparison the paper makes for each mutant)")
pair_rel = {}
REL = {}
for mut, res in (("A510D", "Ala510"), ("N513A", "Asn513")):
(a, sa), (c, sc) = rel("A", mut), rel("C", mut)
REL[(mut, "A")], REL[(mut, "C")] = (a, sa), (c, sc)
z = abs(np.log(a) - np.log(c)) / np.hypot(sa / a, sc / c)
pair_rel[res] = ("dA" if a > c else "dC") if z >= 1 else "about equal"
print(f" {res} ({mut}): dA {a:.2f} (SD ~{sa:.2f}) vs dC {c:.2f} (SD ~{sc:.2f}); "
f"ratio {max(a, c)/min(a, c):.2f}; larger: {pair_rel[res]}; |log diff|/SD ~ {z:.1f}")
print(f" Larger effect: Ala510 -> {pair_rel['Ala510']}, Asn513 -> {pair_rel['Asn513']}")
print("\n Abstract: Ala510 -> dA, Asn513 -> dC (\"... opposite dA and dC, respectively\")")
print(" Paper text: Ala510 -> dC (anti), Asn513 -> dA (syn)")
print(f" Ala510 abstract assignment supported by metric 1: {pair_abs['Ala510'] == 'dA'}, "
f"by metric 2: {pair_rel['Ala510'] == 'dA'}")
print(f" Asn513 abstract assignment supported by metric 1: {pair_abs['Asn513'] == 'dC'}, "
f"by metric 2: {pair_rel['Asn513']} (point estimate larger at dC, |log diff|/SD < 1)")
# ------------------------------------------------------------------ 5. double mutant
print("\n5. Double mutant, 8-oxo-dGTP (Results: 'nearly identical' for dA and dC)")
print(f" opposite dA {F[('A510D/N513A','A','8oxo')][0]:.1f}x, opposite dC {F[('A510D/N513A','C','8oxo')][0]:.1f}x")
# ================================================================== figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, BOX = "#eb6834", "#2a78d6", "#e4e3df", "#c9c8c3"
GREY = "#7a7975" # Results-printed values
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 11, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": BOX, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
W = 7.2
a510 = {t: F[("A510D", t, "8oxo")] for t in "AC"}
n513 = {t: F[("N513A", t, "8oxo")] for t in "AC"}
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def panel(ax, x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=BOX, lw=1.2))
ax.text(x + 0.025, y + h - 0.035, title, fontsize=11.5, weight="bold", color=SEC, va="top")
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
# ---------- candidate 1: two text panels joined by a connector (also fig1)
def cand1(path):
H = 7.4
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
rend = fig.canvas.get_renderer()
def height(t): # height of a drawn text object in axes fraction
bb = t.get_window_extent(renderer=rend)
return bb.height / (fig.dpi * H)
ax.text(0.03, 0.975, "Which residue facilitates 8-oxo-dGMP incorporation opposite dA, which opposite dC?",
fontsize=10.5, weight="bold", va="top")
top, bot = 0.93, 0.335
# left: abstract
panel(ax, 0.03, bot, 0.43, top - bot, "Abstract")
ax.text(0.055, 0.87, wrap("\u201c" + ABSTRACT + "\u201d", 36), fontsize=10.5, va="top", linespacing=1.4)
ax.text(0.055, 0.465, "Pairing as written (\u201crespectively\u201d):", fontsize=10, color=SEC, va="top")
ax.text(0.055, 0.425, "Ala510 \u2192 opposite dA", fontsize=12, weight="bold", color=ORANGE, va="top")
ax.text(0.055, 0.38, "Asn513 \u2192 opposite dC", fontsize=12, weight="bold", color=ORANGE, va="top")
# right: body text
panel(ax, 0.54, bot, 0.43, top - bot, "Results and Conclusion")
y = 0.87
for lab, q in (("Results, N513A paragraph:", Q_N513), ("Conclusion:", Q_A510),
("Results, double-mutant paragraph:", Q_DBL)):
t = ax.text(0.565, y, lab, fontsize=10, color=SEC, va="top")
y -= height(t) + 0.006
t = ax.text(0.565, y, wrap("\u201c\u2026" + q + "\u201d", 37), fontsize=10, va="top", linespacing=1.3)
y -= height(t) + 0.018
assert y > 0.48, y
ax.text(0.565, 0.465, "Pairing in the body text:", fontsize=10, color=SEC, va="top")
ax.text(0.565, 0.425, "Asn513 \u2192 opposite dA (syn)", fontsize=12, weight="bold", color=TXT, va="top")
ax.text(0.565, 0.38, "Ala510 \u2192 opposite dC (anti)", fontsize=12, weight="bold", color=TXT, va="top")
ax.add_patch(FancyArrowPatch((0.445, 0.40), (0.555, 0.40), arrowstyle="<->", mutation_scale=14,
lw=2, color=SEC))
ax.text(0.50, 0.385, "pairing\nreversed", ha="center", va="top", fontsize=10, style="italic", color=SEC)
# bottom strip: Table 1, both metrics
ax.text(0.03, 0.295, "Table 1, recomputed: loss of kcat/KM for 8-oxo-dGTP vs WT", fontsize=10.5,
weight="bold", va="top")
rows = [("absolute fold loss", a510, n513, "{:.1f}\u00d7"),
("\u00f7 loss for correct dNTP", {t: REL[("A510D", t)] for t in "AC"},
{t: REL[("N513A", t)] for t in "AC"}, "{:.2f}")]
ax.text(0.52, 0.250, "Ala510 (A510D)", fontsize=10, color=SEC, va="top", ha="center")
ax.text(0.82, 0.250, "Asn513 (N513A)", fontsize=10, color=SEC, va="top", ha="center")
for k, (lab, A, N, fmt) in enumerate(rows):
yy = 0.210 - k * 0.042
ax.text(0.03, yy, lab, fontsize=10, va="top")
ax.text(0.52, yy, f"dA {fmt.format(A['A'][0])} vs dC {fmt.format(A['C'][0])}", fontsize=10.5,
weight="bold", color=BLUE, va="top", ha="center")
ax.text(0.82, yy, f"dA {fmt.format(N['A'][0])} vs dC {fmt.format(N['C'][0])}", fontsize=10.5,
weight="bold", color=BLUE, va="top", ha="center")
ax.text(0.03, 0.112, "Correct dNTP: dTTP opposite dA, dGTP opposite dC. The Results text prints the same\n"
"absolute values, rounded: 5.1 and 7.6-fold (A510D), 25 and 6.4-fold (N513A).",
fontsize=10, color=SEC, va="top", linespacing=1.35)
ax.text(0.03, 0.040, "orange = pairing stated in the abstract; blue = recomputed from Table 1 by recompute.py.txt",
fontsize=10, color=SEC, va="top")
save(fig, path)
# ---------- candidate 2: two side-by-side bipartite mappings
def cand2(path):
fig = plt.figure(figsize=(W, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.03, 0.96, "Residue-to-template pairing for 8-oxo-dGMP incorporation", fontsize=12, weight="bold", va="top")
def block(x0, span, title, edges, color, ls):
ax.text(x0 + (span + 0.12) / 2, 0.85, title, ha="center", fontsize=11, weight="bold", color=SEC)
L = {"Ala510": 0.70, "Asn513": 0.34}
R = {"dA": 0.70, "dC": 0.34}
for k, y in L.items():
ax.text(x0 + 0.05, y, k, ha="center", va="center", fontsize=11.5, weight="bold",
bbox=dict(boxstyle="round,pad=0.35", fc="white", ec=BOX))
rl = {"dA": "opposite dA\n(syn)", "dC": "opposite dC\n(anti)"}
for k, y in R.items():
ax.text(x0 + span + 0.07, y, rl[k], ha="center", va="center", fontsize=10.5,
bbox=dict(boxstyle="round,pad=0.35", fc="white", ec=BOX))
for (l, r, lab, rep_lab) in edges:
x1, y1, x2, y2 = x0 + 0.105, L[l], x0 + span, R[r]
ax.plot([x1, x2], [y1, y2], color=color, lw=2.2, ls=ls)
if not lab:
continue
frac = 0.25
xm, ym = x1 + frac * (x2 - x1), y1 + frac * (y2 - y1)
ax.text(xm, ym, lab + "\n" + rep_lab, ha="center", va="center", fontsize=10.5, color=BG,
linespacing=1.3, bbox=dict(boxstyle="round,pad=0.3", fc=BG, ec=color, lw=1), zorder=5)
ax.text(xm, ym + 0.026, lab, ha="center", va="center", fontsize=10.5, color=color, weight="bold", zorder=6)
ax.text(xm, ym - 0.026, rep_lab, ha="center", va="center", fontsize=10, color=GREY, zorder=6)
block(0.03, 0.26, "Abstract (“respectively”)",
[("Ala510", "dA", "", ""), ("Asn513", "dC", "", "")], ORANGE, "--")
block(0.49, 0.36, "Table 1: larger absolute fold loss",
[("Ala510", "dC", f"{a510['C'][0]:.2f}×", f"Results: {REPORTED[('A510D', 'C', '8oxo')]}"),
("Asn513", "dA", f"{n513['A'][0]:.2f}×", f"Results: {REPORTED[('N513A', 'A', '8oxo')]}")],
BLUE, "-")
ax.plot([0.46, 0.46], [0.24, 0.82], color=BOX, lw=1)
ax.text(0.03, 0.185, f"Smaller effects: A510D {a510['A'][0]:.2f}× opposite dA; N513A {n513['C'][0]:.2f}× opposite dC "
f"(Results: 5.1 and 6.4).", fontsize=10, color=SEC)
ax.text(0.03, 0.135, f"Relative to the correct dNTP: A510D {REL[('A510D','A')][0]:.2f} (dA) vs "
f"{REL[('A510D','C')][0]:.2f} (dC); N513A {REL[('N513A','A')][0]:.2f} (dA) vs {REL[('N513A','C')][0]:.2f} (dC).",
fontsize=10, color=SEC)
ax.legend(handles=[Line2D([], [], color=ORANGE, ls="--", lw=2, label="pairing stated in the abstract"),
Line2D([], [], color=BLUE, lw=2, label="recomputed from Table 1"),
Line2D([], [], color=GREY, lw=0, marker="s", ms=8, label="printed in the Results")],
loc="lower center", bbox_to_anchor=(0.5, 0.01), ncol=3, frameon=False, fontsize=10,
columnspacing=1.0, handlelength=1.6, handletextpad=0.5)
save(fig, path)
# ---------- candidate 3: fold reductions on one log axis, two metrics, approximate SDs (also fig2)
def cand3(path):
fig, ax = plt.subplots(figsize=(W, 4.6))
fig.subplots_adjust(left=0.27, right=0.97, top=0.82, bottom=0.27)
rows = [("N513A", "C"), ("N513A", "A"), ("A510D", "C"), ("A510D", "A")]
labs = {("A510D", "A"): "A510D, opposite dA", ("A510D", "C"): "A510D, opposite dC",
("N513A", "A"): "N513A, opposite dA", ("N513A", "C"): "N513A, opposite dC"}
abstract_rows = {("A510D", "A"): "abstract: Ala510 → dA", ("N513A", "C"): "abstract: Asn513 → dC"}
off = 0.17
def bar(r, s, y, marker, filled):
lo, hi = np.exp(np.log(r) - s / r), np.exp(np.log(r) + s / r)
ax.plot([lo, hi], [y, y], color=BLUE, lw=2)
ax.plot(r, y, marker, ms=8, color=BLUE, mfc=BLUE if filled else BG, mew=2, zorder=3)
return hi
for i, (m, t) in enumerate(rows):
r, s = F[(m, t, "8oxo")]
hi = bar(r, s, i + off, "D", True)
ax.plot(REPORTED[(m, t, "8oxo")], i + off, "o", ms=14, mfc="none", mec=GREY, mew=1.6, zorder=4)
ax.text(hi * 1.08, i + off, f"{r:.1f}×", va="center", fontsize=10, color=BLUE)
r2, s2 = REL[(m, t)]
hi2 = bar(r2, s2, i - off, "s", False)
ax.text(hi2 * 1.08, i - off, f"{r2:.2f}", va="center", fontsize=10, color=BLUE)
if (m, t) in abstract_rows:
ax.text(58, i, abstract_rows[(m, t)], ha="right", va="center", fontsize=10, color=ORANGE,
weight="bold")
ax.set_yticks(range(4)); ax.set_yticklabels([labs[k] for k in rows], fontsize=10.5, color=TXT)
ax.set_xscale("log"); ax.set_xlim(0.8, 60)
ax.set_xticks([1, 2, 5, 10, 20, 50]); ax.set_xticklabels(["1", "2", "5", "10", "20", "50"])
ax.axvline(1, color=SEC, lw=0.8)
ax.axhline(1.5, color=BOX, lw=1)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
ax.set_xlabel("Fold loss of kcat/KM vs WT (log scale; 1 = no effect)", fontsize=10.5)
ax.set_ylim(-0.55, 3.55)
fig.text(0.02, 0.97, "8-oxo-dGTP: effect of each substitution opposite dA and opposite dC", fontsize=11.5,
weight="bold", va="top")
fig.text(0.02, 0.915, wrap("Absolute loss is larger at dC for A510D and at dA for N513A. Relative to the "
"correct dNTP, A510D still favours dC; N513A is about equal.", 95),
fontsize=10, color=SEC, va="top")
ax.legend(handles=[Line2D([], [], marker="D", ls="-", lw=2, ms=8, color=BLUE,
label="absolute fold loss, Table 1 (±1 approx. SD)"),
Line2D([], [], marker="s", ls="-", lw=2, ms=8, color=BLUE, mfc=BG, mew=2,
label="÷ fold loss for the correct dNTP, Table 1"),
Line2D([], [], marker="o", ls="none", ms=12, mfc="none", mec=GREY, mew=1.6,
label="absolute value printed in the Results"),
Line2D([], [], ls="none", marker="$→$", ms=10, color=ORANGE,
label="pairing stated in the abstract")],
loc="upper center", bbox_to_anchor=(0.36, -0.17), ncol=2, frameon=False, fontsize=10,
columnspacing=1.2, handletextpad=0.5)
save(fig, path)
# ---------- candidate 4: 2 x 2 grid with outlined cells
def cand4(path):
fig = plt.figure(figsize=(W, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.03, 0.96, "Effect on 8-oxo-dGTP incorporation, by residue and template",
fontsize=12, weight="bold", va="top")
x0, y0, cw, ch = 0.30, 0.20, 0.30, 0.27
cols, rowsr = ["A", "C"], [("A510D", "Ala510 (A510D)"), ("N513A", "Asn513 (N513A)")]
abstract_cells = {("A510D", "A"), ("N513A", "C")}
for j, t in enumerate(cols):
ax.text(x0 + (j + 0.5) * cw, y0 + 2 * ch + 0.04, f"opposite d{t}", ha="center", fontsize=11, weight="bold", color=SEC)
for i, (m, lab) in enumerate(rowsr):
y = y0 + (1 - i) * ch
ax.text(x0 - 0.02, y + ch / 2, lab, ha="right", va="center", fontsize=11, weight="bold")
vals = {t: F[(m, t, "8oxo")][0] for t in cols}
for j, t in enumerate(cols):
x = x0 + j * cw
ax.add_patch(Rectangle((x, y), cw, ch, fc="white", ec=BOX, lw=1))
ax.text(x + cw / 2, y + ch * 0.68, f"{vals[t]:.1f}×", ha="center", va="center",
fontsize=16, weight="bold", color=BLUE)
ax.text(x + cw / 2, y + ch * 0.44, f"÷ correct dNTP: {REL[(m, t)][0]:.2f}",
ha="center", va="center", fontsize=10, color=BLUE)
ax.text(x + cw / 2, y + ch * 0.23, f"Results text: {REPORTED[(m, t, '8oxo')]}-fold",
ha="center", va="center", fontsize=10, color=GREY)
if vals[t] == max(vals.values()):
ax.add_patch(Rectangle((x + 0.008, y + 0.012), cw - 0.016, ch - 0.024, fc="none", ec=BLUE, lw=2.5))
if (m, t) in abstract_cells:
ax.add_patch(Rectangle((x + 0.022, y + 0.030), cw - 0.044, ch - 0.060, fc="none", ec=ORANGE,
lw=2, ls="--"))
ax.text(0.03, 0.135, "Large number: absolute fold loss of kcat/KM vs WT. Second line: that loss divided by the "
"loss\nfor the correct dNTP (dTTP at dA, dGTP at dC). Both recomputed from Table 1.",
fontsize=10, color=SEC, va="center")
ax.legend(handles=[Rectangle((0, 0), 1, 1, fc="none", ec=BLUE, lw=2.5, label="larger absolute loss per residue"),
Rectangle((0, 0), 1, 1, fc="none", ec=ORANGE, lw=2, ls="--", label="pairing stated in the abstract")],
loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
save(fig, path)
# ---------- candidate 5: slope chart dA -> dC
def cand5(path):
fig, ax = plt.subplots(figsize=(W, 5.0))
fig.subplots_adjust(left=0.13, right=0.62, top=0.89, bottom=0.30)
xs = [0, 1]
for m, res, ls, mk in (("N513A", "Asn513 (N513A)", "-", "o"), ("A510D", "Ala510 (A510D)", "--", "s")):
v = [F[(m, t, "8oxo")][0] for t in "AC"]
ax.plot(xs, v, ls=ls, marker=mk, ms=8, lw=2, color=BLUE, zorder=3)
for x, t, val in zip(xs, "AC", v):
ax.plot(x, REPORTED[(m, t, "8oxo")], "o", ms=17, mfc="none", mec=GREY, mew=1.6, zorder=4)
dy = 1.0 if x == 0 else (1.13 if val == max(F[(mm, "C", "8oxo")][0] for mm in ("A510D", "N513A")) else 0.88)
ax.text(x + (0.09 if x else -0.09), val * dy, f"{val:.2f}×", ha="left" if x else "right",
va="center", fontsize=10, color=BLUE)
mid = np.sqrt(v[0] * v[1])
ax.text(0.5, mid * (1.75 if m == "N513A" else 0.68), res, ha="center", va="center",
fontsize=10.5, weight="bold")
ax.set_xticks(xs); ax.set_xticklabels(["opposite dA\n(syn)", "opposite dC\n(anti)"], fontsize=10.5, color=TXT)
ax.set_xlim(-0.35, 1.3)
ax.set_yscale("log"); ax.set_ylim(1, 40)
ax.set_yticks([1, 2, 5, 10, 20]); ax.set_yticklabels(["1", "2", "5", "10", "20"])
ax.set_ylabel("Absolute fold loss of kcat/KM,\n8-oxo-dGTP (log scale; 1 = no effect)", fontsize=10.5)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
ax.set_title("Each residue's effect in the two template contexts", fontsize=11.5, weight="bold", loc="left")
fig.text(0.67, 0.83, "Table 1, absolute loss:", fontsize=10.5, weight="bold", color=BLUE)
fig.text(0.67, 0.79, "Asn513 larger opposite dA\nAla510 larger opposite dC", fontsize=10.5, color=BLUE, va="top")
fig.text(0.67, 0.66, "÷ correct-dNTP loss:", fontsize=10.5, weight="bold", color=BLUE)
fig.text(0.67, 0.62, f"Ala510 {REL[('A510D','A')][0]:.2f} (dA), {REL[('A510D','C')][0]:.2f} (dC)\n"
f"Asn513 {REL[('N513A','A')][0]:.2f} (dA), {REL[('N513A','C')][0]:.2f} (dC)",
fontsize=10.5, color=BLUE, va="top")
fig.text(0.67, 0.47, "Abstract states:", fontsize=10.5, weight="bold", color=ORANGE)
fig.text(0.67, 0.43, "Ala510 → dA\nAsn513 → dC", fontsize=10.5, color=ORANGE, va="top")
ax.legend(handles=[Line2D([], [], marker="o", ls="-", lw=2, ms=8, color=BLUE,
label="Asn513 (N513A), recomputed from Table 1"),
Line2D([], [], marker="s", ls="--", lw=2, ms=8, color=BLUE,
label="Ala510 (A510D), recomputed from Table 1"),
Line2D([], [], marker="o", ls="none", ms=13, mfc="none", mec=GREY, mew=1.6,
label="value printed in the Results")],
loc="upper left", bbox_to_anchor=(-0.05, -0.17), ncol=1, frameon=False, fontsize=10)
save(fig, path)
for i, f in enumerate((cand1, cand2, cand3, cand4, cand5), 1):
f(os.path.join(CAND, f"cand{i}"))
cand1(os.path.join(HERE, "fig1"))
cand3(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)
Abstract's age pattern for all STIs except HSV-2 is contradicted by the paper's syphilis estimates
PLOS Medicine (2018) · doi:10.1371/journal.pmed.1002511 · PMC5828349
Summary
This individual participant data meta-analysis of 18 HIV prevention studies reports STI prevalence among women in sub-Saharan Africa by age, region and population type. The abstract states that, in all three primary region/population groups, prevalence was higher among 15–24-year-old than 25–49-year-old women "for all STIs except HSV-2". The author summary repeats this, adding "regardless of region or population type". The paper's own syphilis estimates do not show this pattern. In South African clinic/community populations, overall syphilis prevalence was 4.3% (95% CI 2.5–7.2) at ages 25–49 and 1.5% (1.0–2.2) at 15–24. The intervals do not overlap, and the Results text itself says the older group's prevalence "was higher". In the other two groups, prevalence was higher or similar in older women. High-titer syphilis in South Africa (1.0% vs 0.4%) and trichomoniasis in South Africa (8.6% vs 7.9%) also have higher point estimates in older women. The age pattern does hold for chlamydia and gonorrhea. As written, though, the headline statement misdescribes syphilis, the STI most directly tied to antenatal screening and congenital-syphilis programs.
What the paper reports
Abstract, "Methods and findings". The sentence before it reads "Despite substantial study heterogeneity for some STIs/populations, several patterns emerged."
Across the three primary region/population groups (South Africa community-based, Southern/Eastern Africa community-based, and Eastern Africa higher-risk), prevalence was higher among 15–24-year-old than 25–49-year-old women for all STIs except HSV-2.
Author summary, "What did the researchers do and find?":
STI prevalence was highest among young women aged 15–24 years for all STIs except HSV-2, regardless of region or population type.
Discussion, opening paragraph:
We observed age-, region- and population-specific patterns in the prevalence of these infections, including higher prevalence of STIs among younger (15–24-year-old) than older (25–49-year-old) women, …
Discussion, the paragraph that links the findings to the WHO Global Strategy for STIs. Unlike the abstract, this sentence says "generally":
In terms of the “who,” across all populations and regions, STI prevalence was generally higher among younger than older women, except for the prevalence of antibodies to HSV-2, which increased with age.
Results, "Syphilis, overall (high, low, and unknown titer infection)", first paragraph:
Among 15–24-year-old women, the summary estimate of syphilis prevalence overall (i.e., proportion of participants with a positive RPR and positive treponemal test) was less than 2% in clinic/community populations in both South Africa (1.5% [95% CI: 1.0%, 2.2%]; I2 = 57.7%) and Southern/Eastern Africa (1.4% [95% CI: 1.0%, 1.9%]; I2 = 25.8%), and higher among higher-risk populations in Eastern Africa (8.4% [95% CI: 6.8%, 10.3%]; I2 = 46.3%) (Fig 5).
Results, same section, second paragraph:
Among 25–49-year-olds in clinic/community populations in South Africa, the summary estimate of syphilis prevalence overall among 25–49-year-old women (4.3% [95% CI: 2.5%, 7.2%]; I2 = 94.8%) was higher than that among younger women (1.5%) (Fig 6). In clinic/community populations in Southern/Eastern Africa, the summary estimate of syphilis prevalence overall (2.0% [95% CI: 1.4%, 2.6%]; I2 = 61.8%) was similar to the summary estimate among younger women (1.4%). In the higher-risk populations in Eastern Africa, the summary estimate among older women (7.5% [95% CI: 5.5%, 10.1%]; I2 = 85.8%) was similar to prevalence among younger women (8.4%).
Results, "Syphilis, high-titer infection":
Point estimates for women aged 25–49 years were similar to those in younger populations, but were more heterogeneous (Fig 8).
The "Overall prevalence" rows of Table 4 (15–24 years) and Table 5 (25–49 years), South African clinic/community populations, as archived in PMC:
Table 4: Syphilis, high-titer infection — Overall prevalence: 0.4% (0.3, 0.5); Trichomoniasis — Overall prevalence: 7.9% (5.9, 10.5)
Table 5: Syphilis, high-titer infection — Overall prevalence: 1.0% (0.5, 2.1); Trichomoniasis — Overall prevalence: 8.6% (6.5, 11.5)
Methods, "Statistical analysis". This describes how the strata were compared. No formal test between age groups is described.
We generated forest plots in R and inspected them visually to compare prevalence estimates across strata, and combined strata if estimates were similar.
What the paper's own numbers show
Method. I collected the paper's 15–24 and 25–49 summary prevalence estimates, with their 95% CIs, for each STI in each of the three primary groups. Overall syphilis comes from the Results text, which reports Figs 5 and 6 in numbers. Every other infection comes from the "Overall prevalence" row of Tables 4–9. Where the Results text also states a value, the table rows agree with it: chlamydia SA 15–24, trichomoniasis SA 25–49, trichomoniasis E Africa 15–24 and high-titer syphilis SA 15–24 match exactly, and HSV-2 SA 15–24 differs by 0.1 in one CI bound.
The paper pools study log odds with a random-effects model and back-transforms the pooled estimate and its CI. Its CIs are therefore symmetric on the logit scale, so the standard error of each pooled log odds can be recovered as (logit(upper) − logit(lower)) / 3.92. I then computed the older-versus-younger odds ratio, its 95% CI and an approximate z statistic, treating the two age groups' pooled estimates as independent, since they come from different women. This is a descriptive check of direction and size using only the published numbers. It is not a re-analysis of the participant data. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Direction of the age difference, all STIs except HSV-2, three primary groups | Abstract; Author summary | higher in 15–24 than 25–49 in every group | 9 of 12 infection × group cells (chlamydia, gonorrhea, overall syphilis, trichomoniasis) have the higher point estimate in 15–24, and 3 do not |
| Overall syphilis, SA clinic/community | Results text (Figs 5, 6) | 15–24: 1.5% (1.0, 2.2); 25–49: 4.3% (2.5, 7.2), "higher than that among younger women" | ratio 2.87; CIs do not overlap (2.5 > 2.2); OR 2.95 (1.49–5.84), z = +3.10, p ≈ 0.002, older higher |
| Overall syphilis, S/E Africa clinic/community | Results text | 15–24: 1.4% (1.0, 1.9); 25–49: 2.0% (1.4, 2.6), "similar" | OR 1.44 (0.91–2.26), z = +1.57, p ≈ 0.12; older point estimate higher, not significant |
| Overall syphilis, E Africa higher-risk | Results text | 15–24: 8.4% (6.8, 10.3); 25–49: 7.5% (5.5, 10.1), "similar" | OR 0.88 (0.59–1.32), z = −0.60, p ≈ 0.55; no clear difference |
| High-titer syphilis, SA clinic/community | Table 4 vs Table 5 | 15–24: 0.4% (0.3, 0.5); 25–49: 1.0% (0.5, 2.1); text: point estimates "similar" | ratio 2.50; CIs touch at 0.5; OR 2.52 (1.16–5.43), z = +2.35, p ≈ 0.02, older higher |
| High-titer syphilis, S/E Africa; E Africa higher-risk | Tables 6/7, 8/9 | 1.1% vs 0.9%; 4.5% vs 2.8% | OR 0.82 (0.20–3.39); OR 0.61 (0.38–0.97) |
| Trichomoniasis, SA clinic/community | Table 4 vs Table 5 | 15–24: 7.9% (5.9, 10.5); 25–49: 8.6% (6.5, 11.5) | OR 1.10 (0.70–1.71), z = +0.41, p ≈ 0.68; older point estimate slightly higher, no clear difference |
| Trichomoniasis, S/E Africa; E Africa higher-risk | Tables 6/7, 8/9 | 6.7% vs 5.8%; 12.7% vs 10.0% | OR 0.86 (0.47–1.57); OR 0.76 (0.33–1.78); younger point estimate higher, not significant |
| Chlamydia, three groups | Tables 4–9; Results | 15.1 vs 7.0; 2.7 vs 1.2; 10.3 vs 6.2 | younger higher in 3/3; OR 0.42, 0.44, 0.58, all with p < 0.05 |
| Gonorrhea, three groups | Tables 4–9; Results | 4.6 vs 2.5; 1.7 vs 0.9; 8.2 vs 5.2 | younger higher in 3/3; OR 0.53, 0.53, 0.61 (p ≈ 0.03, 0.07, 0.05) |
| HSV-2, three groups (excluded by the abstract) | Tables 4–9 | 39.3 vs 77.8; 46.8 vs 70.0; 56.3 vs 83.3 | older higher in 3/3, as the paper says |
Of the 12 non-HSV-2 cells, younger women's prevalence is higher with approximate p < 0.05 in 4 cells (all chlamydia, plus gonorrhea in South Africa). Older women's prevalence is higher with p < 0.05 in 1 cell, overall syphilis in South Africa. That cell has non-overlapping CIs, so the paper's own numbers support the reverse of the abstract statement there, as the paper's own Results text says. Across the three groups, syphilis does not show the pattern the abstract describes: the difference is reversed in South Africa, the point estimate is higher in older women in Southern/Eastern Africa, and the estimates are similar in Eastern Africa.
A separate, minor slip turned up in the Results trichomoniasis paragraph: "among 25–29-year-olds in the South African clinic/community-based populations … (summary estimate: 8.6% …)". The value matches the 25–49 row of Table 5, so "25–29" appears to be a typographical error for "25–49".
Figure
Figure 1. On the left, the abstract and author-summary statements, quoted verbatim. On the right, the paper's Results text on overall syphilis (Figs 5 and 6), with the 95% CIs as printed. The connector marks the conflict. In South African clinic/community women, the paper reports 4.3% (2.5–7.2) at 25–49 and 1.5% (1.0–2.2) at 15–24, and its Results text calls the older group's prevalence higher. The orange-marked line restates the reported interval bounds, which do not overlap. The blue-marked line is the odds ratio recomputed from those reported CIs on the logit scale: 2.95 (1.49–5.84), z = +3.10. The abstract statement implies an odds ratio below 1.
Figure 2. For each STI the abstract statement covers (all except HSV-2) and each of the three primary groups, the blue dot and line are the odds ratio, 25–49 versus 15–24 years, with its 95% CI. Each is recomputed from the paper's reported summary prevalence and 95% CI (overall syphilis from the Results text; other infections from the "Overall prevalence" rows of Tables 4–9). The axis is logarithmic, and the thin vertical line marks OR = 1 (no age difference). The shaded orange band is where the abstract statement says every point estimate should lie (younger higher). Chlamydia and gonorrhea lie inside it. Overall syphilis in South Africa lies entirely outside it, and so does high-titer syphilis in South Africa. Overall syphilis in Southern/Eastern Africa and trichomoniasis in South Africa have point estimates on the "older higher" side.
Why it matters
The age statement is one of the few cross-cutting patterns the abstract reports. It is the first finding listed after "several patterns emerged". The author summary repeats it in stronger form ("regardless of region or population type"), and the Discussion uses it to answer the "who" of the WHO Global Health Sector Strategy on STIs. Abstracts and author summaries are what most readers, guideline writers and secondary reviews take from a prevalence paper. A reader relying on them would conclude that syphilis, like chlamydia and gonorrhea, is concentrated in women under 25 in all three settings. The paper's own estimates show that it is not. In South African clinic/community populations, syphilis prevalence is roughly three times higher in women aged 25–49. Elsewhere it is higher or similar in older women.
For syphilis this difference has practical weight. Antenatal syphilis screening and congenital-syphilis elimination programs cover women of all reproductive ages, so where seroprevalence peaks matters for targeting. The underlying estimates, the Results text and the forest plots appear to be correct and internally consistent. The error is in how they are summarized. The paper's other conclusions, on chlamydia and gonorrhea age gradients, regional differences, higher-risk populations, and HSV-2 and BV burden, are not affected.
Explanations considered
- The abstract means "generally", not "in every case". This does not account for the numbers. The Discussion does say "generally", but the abstract and author summary do not hedge, and the author summary adds "regardless of region or population type". In any case the exception is not borderline. It is a full infection (syphilis) whose age difference is null or reversed in all three groups, and significantly reversed in one.
- "Higher" refers to statistically significant differences, and the syphilis differences are not significant. This does not account for the numbers. The paper did not test age differences; it compared forest plots visually (Methods). The one syphilis difference that the published CIs show clearly, in South Africa (non-overlapping CIs; recomputed p ≈ 0.002), runs in the opposite direction to the statement. Under this reading the trichomoniasis and gonorrhea cells would not support the statement either, since several are not significant.
- "Syphilis" in the abstract refers to high-titer (active) infection rather than overall seroprevalence. This does not account for the numbers. High-titer syphilis in South Africa is also higher in older women: 1.0% (0.5, 2.1) against 0.4% (0.3, 0.5), recomputed OR 2.52 (1.16–5.43). The Results describe the high-titer point estimates as "similar" across age groups, which is not "higher among 15–24-year-olds" either.
- The two age groups draw on different sets of studies, so the comparison is not like-for-like. This may be partly true for some strata (study inclusion was applied per infection, and some strata may lack a study). But the abstract statement compares exactly these published summary estimates, and the paper's own Results text makes the same comparison and finds the older group higher. It does not reconcile the abstract with the Results.
- High heterogeneity (I² = 94.8% for SA 25–49) makes the older-women estimate unreliable. This lowers the precision of that estimate, but the random-effects CI already reflects the heterogeneity, and it still excludes the younger estimate. The paper reports and interprets the estimate without reservation in its Results.
- The approximate test assumes independence of the two age-specific pooled estimates. The two estimates come from different women, so their sampling errors are independent. Many of the same studies contribute to both, and any positive correlation between a study's deviation at ages 15–24 and at 25–49 would make the difference more precise than assumed here, not less. For a few strata with zero-count studies, the paper used exact binomial limits rather than a logit standard error (Methods), so the back-calculated standard errors are approximate there (for example, high-titer syphilis in Southern/Eastern Africa). None of this affects the central fact: the reported SA overall-syphilis CIs do not overlap.
- Text conversion or archiving error. Ruled out. The abstract, author summary and Results sentences, including the 4.3% (2.5%, 7.2%) estimate, appear identically on the publisher's version of record. The table values were read from the publisher-deposited PMC XML.
Status
- Version of record. I checked the publisher's article page (journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002511). The abstract sentence, the author-summary sentence, the overall-syphilis Results paragraph, the high-titer sentence, the Discussion "who" sentence and the "25–29-year-olds" slip all appear there verbatim. Tables 4–9 are shown as images on that page, so their values were taken from the publisher-deposited PMC XML. Every quote in this write-up was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The supporting-information list includes S5 and S6 Figs (overall syphilis by initial region and population type, 15–24 and 25–49) and S9 and S10 Figs (overall syphilis restricted to the 17 studies with titer information). I did not open them. They are forest plots stratifying the same estimates and do not change what the main-text numbers show. S1–S4 Tables and S1–S3 Text (checklists and protocol) are not relevant to this point.
- Corrections. One correction exists, doi:10.1371/journal.pmed.1002608 (PMC6019393), dated 2018-06-26. It changes only the copyright license, from CC0 to CC BY 4.0, and does not address the age statement. Crossref, PMC and Europe PMC list no other notices, and none of them lists a retraction or expression of concern.
- Comments. The PubPeer API returned no feedback for the DOI. The PubPeer web search page blocked automated access. The publisher page shows "Reader Comments (0)". A web search for the title with "syphilis", "erratum" and "PubPeer" found only copies of the article and repository records, no comment or correction.
- As of 2026-10-23.
Suggested correction
- Abstract: replace "prevalence was higher among 15–24-year-old than 25–49-year-old women for all STIs except HSV-2" with a statement that matches the Results. For example: "chlamydia and gonorrhea prevalence was higher among 15–24-year-old than 25–49-year-old women in all three groups; syphilis prevalence was similar or higher among 25–49-year-olds (e.g., 4.3% vs 1.5% in South African clinic/community populations); HSV-2 prevalence increased with age."
- Author summary: revise "STI prevalence was highest among young women aged 15–24 years for all STIs except HSV-2, regardless of region or population type" in the same way, naming syphilis (and HSV-2) as exceptions.
- Discussion: qualify the opening summary and the "who" paragraph to state that syphilis did not follow the younger-higher pattern, notably in South African clinic/community populations.
- Results, "Syphilis, high-titer infection": consider stating the South African 25–49 estimate (1.0% [0.5, 2.1] vs 0.4% [0.3, 0.5]) rather than describing the point estimates as "similar".
- Minor: in the trichomoniasis paragraph, change "25–29-year-olds" to "25–49-year-olds".
Recomputation script (python recompute.py.txt)
"""Recompute the age comparisons behind the abstract's age statement in PMC5828349
(PLOS Medicine 2018, doi:10.1371/journal.pmed.1002511).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are the paper's own summary prevalence estimates, % (95% CI):
* overall syphilis: Results text, "Syphilis, overall" paragraphs (Figs 5 and 6);
* all other infections: the "Overall prevalence" row of the co-infection tables
(Table 4/5 South Africa clinic/community 15-24/25-49, Table 6/7 Southern/Eastern
Africa clinic/community, Table 8/9 Eastern Africa higher-risk).
The paper pools study log odds with a random-effects model and back-transforms the pooled
log odds and its 95% CI (Methods), so each CI is symmetric on the logit scale. The standard
error of each pooled log odds is therefore recovered as (logit(hi) - logit(lo)) / (2*1.96).
The 15-24 and 25-49 estimates come from different women, so the two pooled estimates are
treated as independent when comparing them (see writeup.md for the caveat).
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
import matplotlib.ticker
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
GROUPS = ["South Africa clinic/community", "Southern/Eastern Africa clinic/community",
"Eastern Africa higher-risk"]
SHORT = {GROUPS[0]: "SA clinic/community", GROUPS[1]: "S/E Africa clinic/community",
GROUPS[2]: "E Africa higher-risk"}
# (infection, group): {age: (point, lo, hi)}, source
D = {
("Chlamydia", GROUPS[0]): ({"15-24": (15.1, 12.7, 17.8), "25-49": (7.0, 5.5, 8.8)}, "Tables 4/5"),
("Chlamydia", GROUPS[1]): ({"15-24": (2.7, 1.5, 3.9), "25-49": (1.2, 0.8, 1.7)}, "Tables 6/7"),
("Chlamydia", GROUPS[2]): ({"15-24": (10.3, 7.4, 14.1), "25-49": (6.2, 4.7, 8.2)}, "Tables 8/9"),
("Gonorrhea", GROUPS[0]): ({"15-24": (4.6, 3.3, 6.4), "25-49": (2.5, 1.6, 4.0)}, "Tables 4/5"),
("Gonorrhea", GROUPS[1]): ({"15-24": (1.7, 1.2, 2.6), "25-49": (0.9, 0.5, 1.6)}, "Tables 6/7"),
("Gonorrhea", GROUPS[2]): ({"15-24": (8.2, 5.8, 11.6), "25-49": (5.2, 3.9, 7.0)}, "Tables 8/9"),
("Syphilis, overall", GROUPS[0]): ({"15-24": (1.5, 1.0, 2.2), "25-49": (4.3, 2.5, 7.2)}, "Results text (Figs 5/6)"),
("Syphilis, overall", GROUPS[1]): ({"15-24": (1.4, 1.0, 1.9), "25-49": (2.0, 1.4, 2.6)}, "Results text (Figs 5/6)"),
("Syphilis, overall", GROUPS[2]): ({"15-24": (8.4, 6.8, 10.3), "25-49": (7.5, 5.5, 10.1)}, "Results text (Figs 5/6)"),
("Syphilis, high-titer", GROUPS[0]): ({"15-24": (0.4, 0.3, 0.5), "25-49": (1.0, 0.5, 2.1)}, "Tables 4/5"),
("Syphilis, high-titer", GROUPS[1]): ({"15-24": (1.1, 0.3, 1.9), "25-49": (0.9, 0.2, 1.7)}, "Tables 6/7"),
("Syphilis, high-titer", GROUPS[2]): ({"15-24": (4.5, 3.6, 5.5), "25-49": (2.8, 1.9, 4.2)}, "Tables 8/9"),
("Trichomoniasis", GROUPS[0]): ({"15-24": (7.9, 5.9, 10.5), "25-49": (8.6, 6.5, 11.5)}, "Tables 4/5"),
("Trichomoniasis", GROUPS[1]): ({"15-24": (6.7, 4.3, 10.1), "25-49": (5.8, 4.0, 8.4)}, "Tables 6/7"),
("Trichomoniasis", GROUPS[2]): ({"15-24": (12.7, 7.5, 20.6), "25-49": (10.0, 5.7, 17.1)}, "Tables 8/9"),
("HSV-2", GROUPS[0]): ({"15-24": (39.3, 34.4, 44.6), "25-49": (77.8, 75.6, 79.9)}, "Tables 4/5"),
("HSV-2", GROUPS[1]): ({"15-24": (46.8, 38.2, 55.6), "25-49": (70.0, 61.8, 77.1)}, "Tables 6/7"),
("HSV-2", GROUPS[2]): ({"15-24": (56.3, 49.2, 63.1), "25-49": (83.3, 79.3, 86.6)}, "Tables 8/9"),
}
def logit(p_pct):
p = p_pct / 100.0
return np.log(p / (1 - p))
def se_from_ci(lo, hi):
return (logit(hi) - logit(lo)) / (2 * Z975)
def compare(v):
a, b = v["15-24"], v["25-49"]
se = np.hypot(se_from_ci(a[1], a[2]), se_from_ci(b[1], b[2]))
d = logit(b[0]) - logit(a[0])
z = d / se
p = 2 * norm.sf(abs(z))
orr, lo, hi = np.exp(d), np.exp(d - Z975 * se), np.exp(d + Z975 * se)
overlap = not (a[2] < b[1] or b[2] < a[1])
return dict(ratio=b[0] / a[0], OR=orr, lo=lo, hi=hi, z=z, p=p, overlap=overlap,
younger_higher=a[0] > b[0])
# --------------------------------------------------------------- 1. all cells
print("=== 1. Older (25-49) vs younger (15-24) summary prevalence, every infection and group ===")
print("OR = odds ratio older/younger, recomputed from the reported estimates and CIs (logit scale)\n")
hdr = f"{'Infection':21s} {'Group':41s} {'15-24 %':>17s} {'25-49 %':>17s} {'ratio':>6s} {'OR (95% CI)':>20s} {'z':>6s} {'p':>7s} CIs overlap younger higher"
print(hdr)
R = {}
for (inf, grp), (v, src) in D.items():
c = compare(v)
R[(inf, grp)] = c
a, b = v["15-24"], v["25-49"]
print(f"{inf:21s} {grp:41s} {a[0]:5.1f} ({a[1]:.1f}, {a[2]:.1f}) {b[0]:5.1f} ({b[1]:.1f}, {b[2]:.1f}) "
f"{c['ratio']:6.2f} {c['OR']:6.2f} ({c['lo']:.2f}, {c['hi']:.2f}) {c['z']:+6.2f} {c['p']:7.2g} "
f"{str(c['overlap']):11s} {c['younger_higher']}")
# --------------------------------------------------------------- 2. the abstract statement
print("\n=== 2. Abstract: 'prevalence was higher among 15-24-year-old than 25-49-year-old women "
"for all STIs except HSV-2' (three primary groups) ===")
main_inf = ["Chlamydia", "Gonorrhea", "Syphilis, overall", "Trichomoniasis"]
cells = [(i, g) for i in main_inf for g in GROUPS]
yh = [k for k in cells if R[k]["younger_higher"]]
print(f"Non-HSV-2 STIs x groups (overall syphilis): {len(cells)} cells; younger point estimate higher in {len(yh)}")
sig_y = [k for k in cells if R[k]["younger_higher"] and R[k]["p"] < 0.05]
sig_o = [k for k in cells if (not R[k]["younger_higher"]) and R[k]["p"] < 0.05]
print(f" younger higher with approx. p < 0.05: {len(sig_y)}; older higher with approx. p < 0.05: {len(sig_o)}")
print("Cells where the 15-24 estimate is NOT higher (including high-titer syphilis):")
for k in cells + [("Syphilis, high-titer", g) for g in GROUPS]:
c = R[k]
if not c["younger_higher"]:
v = D[k][0]
print(f" {k[0]:21s} {SHORT[k[1]]:28s} 15-24 {v['15-24'][0]}% vs 25-49 {v['25-49'][0]}% "
f"OR {c['OR']:.2f} ({c['lo']:.2f}-{c['hi']:.2f}), z = {c['z']:+.2f}, p = {c['p']:.3f}, "
f"CIs overlap: {c['overlap']}")
for inf in ["Chlamydia", "Gonorrhea"]:
print(f"{inf}: younger higher in {sum(R[(inf, g)]['younger_higher'] for g in GROUPS)}/3 groups")
k = ("Syphilis, overall", GROUPS[0])
v = D[k][0]
print(f"\nKey cell, overall syphilis, SA clinic/community: 25-49 lower CI bound {v['25-49'][1]} > "
f"15-24 upper CI bound {v['15-24'][2]}: {v['25-49'][1] > v['15-24'][2]}")
print(f" ratio of prevalences 4.3/1.5 = {v['25-49'][0] / v['15-24'][0]:.2f}")
print(f" SE(logit) 15-24 = {se_from_ci(1.0, 2.2):.3f}, 25-49 = {se_from_ci(2.5, 7.2):.3f}")
k2 = ("Syphilis, high-titer", GROUPS[0])
v2 = D[k2][0]
print(f"High-titer syphilis, SA: 25-49 lower bound {v2['25-49'][1]} vs 15-24 upper bound {v2['15-24'][2]} "
f"(touch: {v2['25-49'][1] == v2['15-24'][2]}); ratio {v2['25-49'][0] / v2['15-24'][0]:.2f}")
# --------------------------------------------------------------- 3. consistency of table rows with text
print("\n=== 3. Table 'Overall prevalence' rows vs Results text (same estimates?) ===")
checks = [
("Chlamydia SA 15-24", "Table 4", (15.1, 12.7, 17.8), "Results", (15.1, 12.7, 17.8)),
("Trichomoniasis SA 25-49", "Table 5", (8.6, 6.5, 11.5), "Results", (8.6, 6.5, 11.5)),
("Trichomoniasis E Africa HR 15-24", "Table 8", (12.7, 7.5, 20.6), "Results", (12.7, 7.5, 20.6)),
("High-titer syphilis SA 15-24", "Table 4", (0.4, 0.3, 0.5), "Results", (0.4, 0.3, 0.5)),
("HSV-2 SA 15-24", "Table 4", (39.3, 34.4, 44.6), "Results", (39.3, 34.3, 44.6)),
]
for lab, s1, x1, s2, x2 in checks:
print(f" {lab:34s} {s1}: {x1} {s2}: {x2} identical: {x1 == x2}")
# --------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- fig1: abstract vs results text panels
c = R[("Syphilis, overall", GROUPS[0])]
W, H = 7.2, 4.75
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
def panel(x, y, w, h, title, body):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=INK2, lw=1.2))
ax.text(x + 0.14, y + h - 0.16, title, fontsize=10.5, fontweight="bold", va="top", color=INK,
linespacing=1.3)
ax.text(x + 0.14, y + h - 0.78, body, fontsize=10, va="top", color=INK, linespacing=1.45)
PY, PH = 1.5, 3.15
panel(0.1, PY, 2.95, PH, "Abstract (Methods and findings)\nand Author summary",
"“… prevalence was higher\namong 15–24-year-old than\n25–49-year-old women for\n"
"all STIs except HSV-2.”\n(three primary groups)\n\n"
"“STI prevalence was highest\namong young women aged\n15–24 years for all STIs except\n"
"HSV-2, regardless of region\nor population type.”")
panel(4.15, PY, 2.95, PH, "Results, “Syphilis, overall”\n(Figs 5 and 6)",
"South Africa clinic/community\n 15–24 y: 1.5% (1.0, 2.2)\n 25–49 y: 4.3% (2.5, 7.2)\n"
" “… was higher than that\n among younger women”\n\n"
"S/E Africa clinic/community\n 25–49 vs 15–24: 2.0% vs 1.4%\n"
"E Africa higher-risk\n 25–49 vs 15–24: 7.5% vs 8.4%")
ym = PY + PH / 2 + 0.2
ax.annotate("", xy=(4.12, ym), xytext=(3.08, ym),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(3.6, ym + 0.12, "contradicts", ha="center", va="bottom", fontsize=10, color=INK, fontweight="bold")
ax.text(3.6, ym - 0.12, "for syphilis\nin South\nAfrica", ha="center", va="top", fontsize=10, color=INK2)
# legend-style lines: reported (orange) and recomputed (blue)
ax.plot([0.12, 0.42], [1.08, 1.08], color=ORANGE, lw=2)
ax.plot([0.27], [1.08], "o", color=ORANGE, ms=8)
ax.text(0.55, 1.08, "Reported in the paper: SA 25–49 lower CI bound 2.5% > 15–24 upper CI\n"
"bound 2.2%, so the two intervals do not overlap.", fontsize=10, color=INK, va="center")
ax.plot([0.12, 0.42], [0.48, 0.48], color=BLUE, lw=2)
ax.plot([0.27], [0.48], "o", color=BLUE, ms=8)
ax.text(0.55, 0.48, f"Recomputed from those CIs (logit scale): odds ratio 25–49 vs 15–24\n"
f"= {c['OR']:.2f} (95% CI {c['lo']:.2f}–{c['hi']:.2f}), z = {c['z']:+.2f}, two-sided p ≈ "
f"{c['p']:.3f}.", fontsize=10, color=INK, va="center")
save(fig, "fig1")
# ---- fig2: OR older/younger for every non-HSV-2 cell, log axis
order = ["Chlamydia", "Gonorrhea", "Trichomoniasis", "Syphilis, overall", "Syphilis, high-titer"]
fig, ax = plt.subplots(figsize=(7.2, 7.4), facecolor=BG)
ax.set_facecolor(BG)
ys, labels, rows, heads = [], [], [], []
y = 0.0
for inf in order:
heads.append((y, inf))
y -= 0.85
for g in GROUPS:
ys.append(y); labels.append(SHORT[g]); rows.append((inf, g))
y -= 0.8
y -= 0.35
ax.axvspan(0.1, 1, color=ORANGE, alpha=0.08, lw=0)
ax.axvline(1, color=INK2, lw=0.8)
for yy, k in zip(ys, rows):
c = R[k]
ax.plot([c["lo"], c["hi"]], [yy, yy], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot([c["OR"]], [yy], "o", color=BLUE, ms=8, mec=BG, mew=1.5)
ax.text(1.02, yy, f"{c['OR']:.2f} ({c['lo']:.2f}–{c['hi']:.2f})", va="center",
fontsize=10, color=INK2, transform=ax.get_yaxis_transform())
for yy, inf in heads:
ax.text(-0.02, yy, inf, ha="right", va="center", fontsize=10.5, fontweight="bold", color=INK,
transform=ax.get_yaxis_transform())
ax.set_xscale("log")
ax.set_xlim(0.15, 7)
ax.set_xticks([0.2, 0.5, 1, 2, 5])
ax.set_xticklabels(["0.2", "0.5", "1", "2", "5"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys); ax.set_yticklabels(labels, fontsize=10, color=INK)
top = 2.2
ax.set_ylim(min(ys) - 0.6, top)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, 25–49 vs 15–24 years (log scale)", color=INK)
ax.text(0.95, top - 0.05, "← younger higher\n(abstract: every row\nbelongs in this band)", ha="right",
va="top", fontsize=10, color=INK)
ax.text(1.05, top - 0.05, "older higher →", ha="left", va="top", fontsize=10, color=INK2)
ax.text(1.02, top - 0.05, "OR (95% CI)", ha="left", va="top", fontsize=10, color=INK, fontweight="bold",
transform=ax.get_yaxis_transform())
h1, = ax.plot([], [], "o-", color=BLUE, lw=2, ms=8)
h2 = plt.Rectangle((0, 0), 1, 1, color=ORANGE, alpha=0.25, lw=0)
ax.legend([h1, h2], ["Recomputed from the paper's estimates and 95% CIs",
"Region implied by the abstract statement"],
loc="upper left", bbox_to_anchor=(-0.55, -0.075), frameon=False, fontsize=10, ncol=1)
fig.subplots_adjust(left=0.36, right=0.77, top=0.98, bottom=0.15)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract reports 27% of neurons frequency-tuned; the paper's own counts give 27 of 357 (8%)
PLoS Biol (2026) · doi:10.1371/journal.pbio.3003818 · PMC13289887
Summary
The abstract states that 27% of the 357 recorded human single neurons showed frequency tuning. The Results give the same finding as "8% of all neurons (n = 27/357)", and every breakdown in the paper adds up to 27 tuned neurons: by region (7 + 8 + 7 + 3 + 2), by modulation pattern (18 + 4 + 5) and by amplitude regime (16 + 6 + 5). 27/357 is 7.56%. A 27% share would need about 96 neurons, and no count or denominator in the paper gives 27%, so the abstract appears to have printed the count of tuned neurons as their percentage. The abstract gives no other prevalence figure, and this one is about 3.6 times too high. It also reverses the paper's own comparison: in the Results and Discussion, phase tuning (54/357, 15%) is about twice as common as frequency tuning. The existence of frequency tuning above chance is not affected.
What the paper reports
Abstract:
We analyzed 357 single units recorded simultaneously with local field potentials in 19 neurosurgical patients during awake resting.
We found 27% of neurons exhibited increased or decreased firing within specific frequencies, most commonly within the low-frequency range (<10 Hz).
Results, "A different subset of human single neurons displays frequency tuning", paragraph 1:
Our analysis revealed that 8% of all neurons (n = 27/357) exhibited significant frequency-tuning (binomial test, chance level = 0.01, p < 0.0001).
Same section, paragraph 3 (modulation patterns, Fig 5):
The most common pattern was a bidirectional modulation observed in 18 neurons (67% of the neurons displaying frequency tuning), characterized by increased firing within a specific frequency range and decreased firing in another. Less commonly, frequency-tuned neurons exhibited either a selective enhancement (4 neurons, 15%) or selective suppression (5 neurons, 18%) of firing within a frequency range, without a statistically significant opposing effect.
Same section (amplitude stratification):
Of the 27 neurons previously identified as frequency tuned, 16 remained significant in both amplitude regimes, six showed significant tuning in only one regime, and five were not significant after stratification
Same section (regional breakdown; Table 1 lists 147, 70, 100, 17 and 23 single units for these regions, 357 in total):
we identified neurons modulated by frequency across all recorded regions: 5% in Hipp (n = 7/147), 11% in EC (n = 8/70), 7% in ACC (n = 7/100), 17% in PCC (n = 3/17), and 8% in OFC (n = 2/23).
Results, "A subset of human single neurons displays phase tuning", paragraph 1:
Our phase tuning analyses revealed that 15% of neurons (n = 54/357) displayed significant phase tuning (binomial test, chance level = 0.01, p < 0.0001).
Results, "Phase and frequency tuning are independent phenomena":
Only 1% of neurons (3/357) were significantly modulated by both the instantaneous frequency and phase of the local LFP
Discussion, "A subset of human single neurons displays frequency-tuning to slow oscillations independent of the phase of oscillation":
Frequency tuning, like phase tuning, was observed across multiple brain regions and phase tuning was twice as prevalent.
What the paper's own numbers show
Method. All inputs are counts printed in the article (Abstract, Results and Table 1). I recomputed each printed share from its numerator and denominator, added up every breakdown of the tuned neurons, and computed the share for every other count or denominator in the paper that could plausibly have produced "27%". I also repeated the paper's binomial test against its 1% chance level. No raw data are needed. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Frequency-tuned share of all units | Abstract | 27% | 27/357 = 7.56% |
| Frequency-tuned share of all units | Results, frequency-tuning section, para 1 | 8% (n = 27/357) | 7.56%, rounds to 8% |
| Neurons needed for a 27% share | — | — | 0.27 × 357 = 96.4; any count from 95 to 98 rounds to 27% |
| Abstract value ÷ recomputed value | — | — | 27 / 7.56 = 3.57 (about 1 in 3.7 neurons vs 1 in 13.2) |
| Regional tuned counts, summed | Results, regional paragraph; Table 1 | 7/147, 8/70, 7/100, 3/17, 2/23 | 27 tuned of 357 units = 7.56% |
| Modulation patterns, summed | Results, Fig 5 paragraph | 18 + 4 + 5 | 27 |
| Amplitude strata, summed | Results, amplitude paragraph | 16 + 6 + 5 | 27 |
| Phase-tuned share | Results, phase-tuning section | 15% (54/357) | 15.13% |
| Phase ÷ frequency prevalence | Discussion | "twice as prevalent" | 54/27 = 2.00; with the Abstract's 27%, the ratio would be 15.13/27 = 0.56 |
| Tuned to both | Results, independence section | 1% (3/357) | 0.84% |
| Phase- or frequency-tuned | derived | — | (54 + 27 − 3)/357 = 78/357 = 21.85% |
| Frequency-tuned only | derived | — | 24/357 = 6.72% |
| Tuned neurons per macroelectrode | derived (84 macroelectrodes, Results para 1) | — | 27/84 = 32.14% |
| One-direction patterns among the tuned | derived | — | (4 + 5)/27 = 33.33% |
| Pattern shares of the 27 | Results, Fig 5 paragraph | 67%, 15%, 18% | 66.7%, 14.8%, 18.5% |
| Binomial test, 27/357 vs chance 0.01 | Results | p < 0.0001 | one-sided p = 1.15 × 10⁻¹⁵ |
| Binomial test, 54/357 vs chance 0.01 | Results | p < 0.0001 | one-sided p = 2.23 × 10⁻⁴⁵ |
Smaller points, separate from the main finding. (i) Three printed shares are truncated rather than rounded: PCC 3/17 = 17.65% (printed 17%), OFC 2/23 = 8.70% (printed 8%) and suppression-only 5/27 = 18.5% (printed 18%). (ii) The regional phase-tuned counts in the Results (17 + 17 + 14 + 1 + 4) sum to 53, not the 54 given as the total. (iii) The frequency chi-square across Hipp, EC and ACC reproduces from the printed counts as X² = 3.26 with 2 degrees of freedom, p = 0.196, which matches the printed p = 0.19; the printed "[3]" degrees of freedom would give p = 0.353. The same test on the printed phase counts gives X² = 6.12, df = 2, p = 0.047, not the printed X² = 5.49, p = 0.06. The phase test may have used counts that the text does not show, so I have not looked into this further. None of these points touches the abstract's 27%.
Figure
Figure 1. The left panel holds the Abstract's sentence on frequency tuning and the number of neurons it implies (orange, as printed). The right panel holds the Results sentence and the paper's breakdowns of the tuned neurons by region and by modulation pattern, all of which add up to 27 of the same 357 units. The blue value is the share recomputed from those counts, 27/357 = 7.56%. The connector marks the conflict: the Abstract's figure is 3.6 times the recomputed share and equals the count of tuned neurons.
Figure 2. Each blue dot is a share recomputed from counts printed in the paper. The orange vertical line is the Abstract's 27%. The axis starts at zero. The frequency-tuned share of all units (bold) is 7.6%. The other rows test denominators and subsets that might explain the Abstract's figure. None of them comes to 27%: neurons tuned to phase or frequency come to 21.8%, and the two shares above 27% (tuned neurons per macroelectrode, and one-direction patterns among the 27 tuned) are not shares of neurons in the sense of the Abstract's sentence.
Why it matters
- This is the only prevalence the abstract gives, and it is too high by a factor of about 3.6. Single-unit prevalences are often compared across studies and cited from abstracts. A reader of the abstract alone would conclude that about 1 neuron in 4 is frequency-tuned. The paper's data show about 1 in 13, a minority that the paper's own section headings call "a subset".
- The comparison with phase tuning is reversed. The abstract does not give the phase-tuning prevalence (15%, 54/357). With the abstract's 27%, frequency tuning would look almost twice as common as phase tuning. The Results ("The proportion of phase-tuned neurons was approximately twice that of frequency-tuned neurons in all regions, except for the posterior cingulate cortex") and the Discussion ("phase tuning was twice as prevalent") say the opposite, which matches 54/27 = 2.00.
- The translational framing rests on how widespread the effect is. The abstract ends by proposing frequency tuning as "an additional mechanism for information processing in the human brain, opening new avenues for frequency-targeted neural stimulation". A 27% prevalence makes that look like a broad property of human neurons. The data show it in 8%.
- What does not change. At 27/357 the tuned fraction is still far above the 1% chance level (one-sided binomial p ≈ 10⁻¹⁵). The paper also reports frequency tuning in 13 of 19 patients and in all five regions. So the existence of frequency tuning, and its significance, stand; only its stated size in the abstract is wrong. The Results and Discussion are consistent with 8%, so the error is confined to the abstract.
Explanations considered
- A different denominator. The sentence before it in the abstract names the denominator as the 357 single units. Table 1 gives 357 units across the five regions, and the Results state "n = 27/357". There are 84 macroelectrodes, and 27/84 = 32.1%, but that is neither 27% nor a share of neurons. Does not account for it.
- A different subset of neurons. Neurons tuned to phase or frequency: 78/357 = 21.8%. Frequency-only: 24/357 = 6.7%. Phase-tuned: 15.1%. The pattern shares of the 27 tuned neurons: 66.7%, 14.8%, 18.5%, and 33.3% for the two one-direction patterns together. Neurons surviving the ISI-preserving shuffle: 19/27 = 70.4% of the tuned, or 19/357 of all units. None of these is 27%. Does not account for it.
- A different analysis or time point. The Results report a single frequency-tuning prevalence. The control analyses (ISI shuffle, split halves, amplitude stratification) each start from the same 27 neurons and keep fewer, not more. Does not account for it.
- A rounding or typesetting artifact. A share of 7.56% cannot round to 27%. The version-of-record HTML on the publisher's site, the PMC copy and the Europe PMC full-text XML all carry the same "27%" word for word, so this is not a conversion artifact. Does not account for it.
- A change between preprint and publication. The bioRxiv preprint (Europe PMC PMC12424639, full-text XML retrieved 2026-10-24) has the same mismatch. Its abstract says "We found 27% neurons exhibited increased or decreased firing within specific frequencies", and its Results say "8% of all neurons (n = 27/357)". The mismatch therefore survived revision; no corrected version exists. Does not account for it.
- Supplementary material or source data giving 27%. According to the version-of-record check, S2 Table sums to 27 frequency-tuned units across 13 subjects, 54 phase-tuned units, and 3 units tuned to both. S3 Table lists 22 neurons with increased firing and 5 with suppression only (22 + 5 = 27). In the Zenodo source data (doi:10.5281/zenodo.19860065), I checked two files myself.
frequency_tuned_frequencies.matholds 27 entries.count_result_per_region.mat, decoded from its stored doubles, gives TotalNeurons 100/70/147/23/17 (= 357) and N_Frequency 7/8/7/2/3 (= 27) for ACC/EC/Hipp/OFC/PCC, with Pct_Frequency 7.0/11.43/4.76/8.70/17.65. Its N_Phase is 14/17/17/4/1 (= 53, matching the text's regional counts), and N_Either is 21/23/24/6/4 (= 78). Pooled, none of these is 27%. Does not account for it; the source data support about 8%.
What remains is that the abstract's "27%" matches the number of frequency-tuned neurons, 27. That fits a count printed in place of a percentage, but the paper does not say how the figure arose.
Status
As of 2026-10-24:
- Version of record checked. The publisher's page (journals.plos.org, reached from doi.org), the PMC page and the Europe PMC full-text XML all carry the abstract's "27% of neurons" and the Results' "8% of all neurons (n = 27/357)". Every quote above was checked word for word against the PMC archive copy.
- Supplements and data. S2 and S3 Tables (as summarized in the version-of-record check) and the Zenodo source data (doi:10.5281/zenodo.19860065; regional count table and frequency-tuned list re-read for this write-up) all give 27 frequency-tuned units of 357. None of them resolves the abstract figure.
- Corrections. None found. Crossref shows no
update-to/updated-byrelation, and a Crossrefupdates:filter search returned 0 results. PMC and Europe PMC list no notices. The Crossref/Retraction Watch record shows no updates. The PLOS article page shows no notice, and its reader comments number 0. The linked PLOS Biology commentary (doi:10.1371/journal.pbio.3003846, PMC13293611) gives no percentage. The published peer-review history quotes the 54/357 vs 27/357 comparison but does not mention the abstract's 27%. - PubPeer. A web search found no PubPeer thread or other public discussion. PubPeer itself could not be queried directly (blocked by a 403 / Cloudflare challenge), and the Europe PMC search endpoint returned 503 errors, so these two checks are incomplete.
- Preprint. bioRxiv preprint (PMC12424639): same discrepancy.
Suggested correction
In the Abstract, replace
We found 27% of neurons exhibited increased or decreased firing within specific frequencies, most commonly within the low-frequency range (<10 Hz).
with
We found 8% of neurons (27/357) exhibited increased or decreased firing within specific frequencies, most commonly within the low-frequency range (<10 Hz).
If the authors want the abstract to reflect the paper's comparison with phase tuning, they could add that 15% of neurons (54/357) showed phase tuning. Separately, they may wish to check the truncated regional percentages (PCC 17.6%, OFC 8.7%), the regional phase-tuned counts (which sum to 53 against a total of 54), and the degrees of freedom and phase statistic in the regional chi-square tests.
Recomputation script (python recompute.py.txt)
"""Recompute the frequency-tuning prevalence in PMC13289887
(PLoS Biol 2026, doi:10.1371/journal.pbio.3003818).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PubMed Central:
Abstract: 357 single units; "27% of neurons" frequency-tuned.
Results, frequency-tuning section: 8% (n = 27/357); regional counts 7/147, 8/70, 7/100,
3/17, 2/23; modulation patterns 18 / 4 / 5; amplitude strata 16 / 6 / 5; ISI shuffle 19/27.
Results, phase-tuning section: 15% (n = 54/357); regional phase counts 17/147, 17/70,
14/100, 1/17, 4/23.
Results, independence section: 1% (3/357) tuned to both.
Results, first paragraph and Table 1: 84 macroelectrodes; units per region 147/70/100/17/23.
No raw data are used; the checks are arithmetic on the paper's own counts.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from scipy.stats import binomtest, chi2, chi2_contingency # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
# ---------------------------------------------------------------- inputs (from the paper)
N = 357 # single units (Abstract, Results, Table 1)
ABSTRACT_PCT = 27.0 # Abstract: "We found 27% of neurons ..."
RESULTS_PCT = 8.0 # Results: "8% of all neurons (n = 27/357)"
FREQ, PHASE, BOTH = 27, 54, 3
MACROS = 84
REGIONS = ["Hipp", "EC", "ACC", "PCC", "OFC"]
UNITS = {"Hipp": 147, "EC": 70, "ACC": 100, "PCC": 17, "OFC": 23}
FREQ_REG = {"Hipp": 7, "EC": 8, "ACC": 7, "PCC": 3, "OFC": 2}
FREQ_REG_PCT = {"Hipp": 5, "EC": 11, "ACC": 7, "PCC": 17, "OFC": 8} # as printed
PHASE_REG = {"Hipp": 17, "EC": 17, "ACC": 14, "PCC": 1, "OFC": 4}
PHASE_REG_PCT = {"Hipp": 12, "EC": 24, "ACC": 14, "PCC": 6, "OFC": 17} # as printed
PATTERNS = {"bidirectional": (18, 67), "enhancement only": (4, 15), "suppression only": (5, 18)}
AMP_STRATA = (16, 6, 5) # both regimes / one regime / neither
ISI_SURVIVE = 19
def pct(a, b):
return 100.0 * a / b
def main():
print("== Headline prevalence")
p_freq = pct(FREQ, N)
print(f"Frequency-tuned share 27/357 = {p_freq:.2f}% (Results print {RESULTS_PCT:.0f}%; Abstract prints {ABSTRACT_PCT:.0f}%)")
need = ABSTRACT_PCT / 100 * N
lo, hi = 0.265 * N, 0.275 * N
print(f"27% of 357 = {need:.1f} neurons; a share that rounds to 27% needs {int(-(-lo // 1))}-{int(hi // 1)} neurons")
print(f"Abstract / recomputed = {ABSTRACT_PCT / p_freq:.2f}x")
print(f"1 in {N / FREQ:.1f} neurons (recomputed) vs 1 in {100 / ABSTRACT_PCT:.1f} (Abstract)")
print("\n== Counts that sum to the 27 tuned neurons")
s_t = sum(FREQ_REG.values()); s_u = sum(UNITS.values())
print(f"Regional tuned counts sum = {s_t}; regional units sum = {s_u}; pooled {pct(s_t, s_u):.2f}%")
print(f"Modulation patterns 18+4+5 = {sum(v[0] for v in PATTERNS.values())}")
print(f"Amplitude strata 16+6+5 = {sum(AMP_STRATA)}")
print(f"ISI-preserving shuffle: {ISI_SURVIVE}/27 = {pct(ISI_SURVIVE, FREQ):.1f}% remain significant")
print("\n== Regional frequency-tuned shares (printed vs recomputed)")
for r in REGIONS:
v = pct(FREQ_REG[r], UNITS[r])
print(f"{r}: {FREQ_REG[r]}/{UNITS[r]} = {v:.2f}% printed {FREQ_REG_PCT[r]}% (rounds to {round(v)}%)")
print("\n== Phase tuning and the 'twice as prevalent' statement")
p_phase = pct(PHASE, N)
print(f"Phase-tuned 54/357 = {p_phase:.2f}% (printed 15%)")
print(f"Phase / frequency = {PHASE / FREQ:.2f} using counts; at the Abstract's 27%: 15.13/27 = {p_phase / ABSTRACT_PCT:.2f}")
print(f"Both 3/357 = {pct(BOTH, N):.2f}% (printed 1%)")
print("\n== Other denominators or subsets that could have produced 27%")
cands = candidates()
for lab, num, den, v in cands:
print(f"{lab}: {num}/{den} = {v:.2f}%")
for lab, (n, printed) in PATTERNS.items():
print(f"pattern {lab}: {n}/27 = {pct(n, FREQ):.1f}% (printed {printed}%)")
print("\n== Significance against the 1% chance level (one-sided binomial)")
print(f"27/357 vs 0.01: p = {binomtest(FREQ, N, 0.01, alternative='greater').pvalue:.3g}")
print(f"54/357 vs 0.01: p = {binomtest(PHASE, N, 0.01, alternative='greater').pvalue:.3g}")
print("\n== Secondary observations (not part of the main finding)")
s_p = sum(PHASE_REG.values())
print(f"Regional phase-tuned counts sum = {s_p} (headline count is {PHASE})")
for r in REGIONS:
v = pct(PHASE_REG[r], UNITS[r])
print(f" {r}: {PHASE_REG[r]}/{UNITS[r]} = {v:.2f}% printed {PHASE_REG_PCT[r]}%")
for name, reg in [("frequency", FREQ_REG), ("phase", PHASE_REG)]:
tab = [[reg[r], UNITS[r] - reg[r]] for r in ("Hipp", "EC", "ACC")]
x2, p, df, _ = chi2_contingency(tab, correction=False)
print(f"Chi-square Hipp/EC/ACC, {name}: X2 = {x2:.2f}, df = {df}, p = {p:.3f}; "
f"same X2 with df = 3: p = {chi2.sf(x2, 3):.3f}")
fig1(p_freq)
fig2(cands, p_freq)
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
def candidates():
rows = [
("Frequency-tuned / all units", FREQ, N),
("Frequency-only / all units", FREQ - BOTH, N),
("Phase-tuned / all units", PHASE, N),
("Phase- or frequency-tuned / all units", PHASE + FREQ - BOTH, N),
("Frequency-tuned / macroelectrodes", FREQ, MACROS),
("One-direction patterns / 27 tuned", 4 + 5, FREQ),
]
return [(lab, a, b, pct(a, b)) for lab, a, b in rows]
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def panel(ax, x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.02, y + h - 0.035, title, fontsize=11.5, weight="bold", va="top", color=INK)
yy = y + h - 0.115
for txt, col, wt, size in lines:
ax.text(x + 0.02, yy, txt, fontsize=size, va="top", color=col, weight=wt)
yy -= 0.058 if size >= 10.5 else 0.052
def fig1(p_freq):
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.965, "The Abstract and the Results give different frequency-tuning prevalences",
fontsize=12, weight="bold", va="top")
ax.text(0.03, 0.905, "Same 357 single units; excerpts as printed in the version of record",
fontsize=10, color=INK2, va="top")
a_lines = [
("“We analyzed 357 single units ...”", INK2, "normal", 10),
("“We found 27% of neurons exhibited", INK, "normal", 10.5),
("increased or decreased firing within", INK, "normal", 10.5),
("specific frequencies ...”", INK, "normal", 10.5),
("Reported: 27% of 357", ORANGE, "bold", 11),
("= about 96 neurons", ORANGE, "bold", 11),
]
b_lines = [
("Frequency-tuning section:", INK2, "normal", 10),
("“8% of all neurons (n = 27/357)", INK, "normal", 10.5),
("exhibited significant frequency-tuning”", INK, "normal", 10.5),
("Regions: 7/147 + 8/70 + 7/100", INK2, "normal", 10),
("+ 3/17 + 2/23 = 27/357", INK2, "normal", 10),
("Patterns 18 + 4 + 5 = 27", INK2, "normal", 10),
("Recomputed: 27/357 = %.2f%%" % p_freq, BLUE, "bold", 11),
]
panel(ax, 0.03, 0.24, 0.40, 0.59, "Abstract", a_lines, ORANGE)
panel(ax, 0.57, 0.24, 0.40, 0.59, "Results", b_lines, BLUE)
arr = FancyArrowPatch((0.435, 0.50), (0.565, 0.50), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=INK2)
ax.add_patch(arr)
ax.text(0.50, 0.565, "conflict", ha="center", fontsize=10.5, weight="bold", color=INK)
ax.text(0.50, 0.445, "27% vs\n7.6%", ha="center", va="top", fontsize=10, color=INK2)
ax.text(0.50, 0.335, "%.1f×" % (ABSTRACT_PCT / p_freq), ha="center", va="top",
fontsize=10, color=INK2)
ax.text(0.03, 0.17, "Orange: value printed in the Abstract. Blue: value recomputed from the "
"paper's counts.", fontsize=10, color=INK2, va="top")
ax.text(0.03, 0.11, "The Results print 8%. The Abstract's 27 equals the count of tuned neurons.",
fontsize=10, color=INK2, va="top")
save(fig, "fig1")
def fig2(cands, p_freq):
order = sorted(cands, key=lambda c: c[3])
labels = [c[0] for c in order]
vals = [c[3] for c in order]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
fig.subplots_adjust(left=0.37, right=0.97, top=0.80, bottom=0.15)
y = list(range(len(order)))
ax.axvline(ABSTRACT_PCT, color=ORANGE, lw=2, zorder=1)
ax.text(ABSTRACT_PCT, len(order) - 0.05, "Abstract: 27%", color=INK, fontsize=10,
va="center", ha="center", zorder=5, bbox=dict(fc=BG, ec="none", pad=2))
for yi, (lab, a, b, v) in zip(y, order):
main = lab == "Frequency-tuned / all units"
ax.plot([0, v], [yi, yi], color=GRID, lw=2, zorder=1)
ax.plot(v, yi, "o", ms=9 if main else 8, color=BLUE, mec=BG, mew=1.5, zorder=3)
ax.text(v + 0.8, yi, f"{a}/{b} = {v:.1f}%", va="center", fontsize=10,
color=INK, weight="bold" if main else "normal", zorder=4,
bbox=dict(fc=BG, ec="none", pad=1.5))
ax.set_yticks(y); ax.set_yticklabels(labels, fontsize=10)
for t in ax.get_yticklabels():
if t.get_text() == "Frequency-tuned / all units":
t.set_color(INK); t.set_weight("bold")
ax.set_xlim(0, 45); ax.set_ylim(-0.6, len(order) + 0.15)
ax.set_xlabel("Percent")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.text(0.02, 0.965, "No count or denominator in the paper gives 27%", fontsize=12,
weight="bold", va="top")
fig.text(0.02, 0.90, "Blue dots: shares recomputed from the paper's counts. "
"Orange line: the Abstract's 27%.", fontsize=10, color=INK2, va="top")
save(fig, "fig2")
if __name__ == "__main__":
main()
GHF-ACL F1 scores in Tables 5, 6 and 9 exceed what their precision and recall allow
PLoS Comput Biol (2026) · doi:10.1371/journal.pcbi.1014461 · PMC13327522
Summary
The paper proposes GHF-ACL, a graph contrastive learning model for herb–disease association prediction, and reports precision, recall and F1 for it and for baseline models under five-fold cross-validation. On the Ethnobotany dataset (Table 6) it reports precision 0.7217, recall 0.9955 and F1 0.9885. F1 is the harmonic mean of precision and recall, so these inputs give 0.8368, and no averaging of folds can lift the mean F1 above that. The same check finds a second large gap on Cdata (Table 5: reported 0.9241, at most 0.7625) and smaller ones on LRSSL and in two ablation rows, while all 28 baseline rows match to within 0.0006 on the high side. With values consistent with the paper's own precision and recall, GHF-ACL does not have the best F1 on Ethnobotany or Cdata, so the claim of F1 superiority across all datasets (Abstract, Introduction, Results, Conclusion) is not supported. The AUPR results, the paper's stated primary metric, are not affected.
What the paper reports
Table 6 (Ethnobotany), header and two of its four rows:
DataSet | Model | AUPR | Precision | Recall | F1 Score | AUC Ethnobotany | HGHDA | 0.9033 | 0.8196 | 0.8618 | 0.8400 | 0.9028 | GHF-ACL | 0.9324 | 0.7217 | 0.9955 | 0.9885 | 0.9296
Table 5 (LRSSL, Ldata, Cdata), header and, for each dataset, the best-F1 baseline (DD-HGNN+) and the GHF-ACL row. Rows are excerpted. In the paper the dataset label appears only on each block's first row (LAGCN), so the dataset cells here are blank, as in the original, and each pair is introduced by its dataset name.
DataSet | Model | AUPR | Accuracy | Precision | Recall | F1 Score | AUC
LRSSL:
| DD-HGNN+ | 0.7460 | 0.9996 | 0.9082 | 0.7624 | 0.8289 | 0.9991 | GHF-ACL | 0.7821 | 0.9988 | 0.7247 | 0.9922 | 0.8510 | 0.9347
Ldata:
| DD-HGNN+ | 0.6886 | 0.9820 | 0.6527 | 0.6128 | 0.6325 | 0.9780 | GHF-ACL | 0.6979 | 0.8140 | 0.6719 | 0.8865 | 0.5809 | 0.9273
Cdata:
| DD-HGNN+ | 0.8954 | 0.9995 | 0.8356 | 0.9336 | 0.8819 | 0.9996 | GHF-ACL | 0.9295 | 0.9996 | 0.6355 | 0.9528 | 0.9241 | 0.8157
Table 9 (ablation on LRSSL), header and two of its five rows:
Model Variant | Loss | Accuracy | Precision | Recall | F1 Score | AUC w/o Structural Interaction Alignment | 0.84918 | 0.37407 | 0.2855 | 0.9329 | 0.5456 | 0.8847 w/o Cluster-level Contrastive Loss | 0.82725 | 0.46601 | 0.3781 | 0.9453 | 0.5901 | 0.8867
Abstract:
Extensive experiments on five datasets demonstrate that GHF-ACL achieves superior or competitive performance over six state-of-the-art models across most metrics, with significant improvements over the best-performing baseline model in AUPR (+4.8% on LRSSL, + 3.81% on Cdata), F1 score, and Recall.
Introduction, list of contributions:
Performance Superiority: Extensive experiments on five datasets demonstrate that GHF-ACL consistently outperforms baselines across AUPR, Recall, and F1 metrics, exhibiting remarkable robustness particularly in imbalanced prediction scenarios.
Section 3.1, Experimental setup (how the table values were obtained):
We employed a five-fold cross-validation strategy to ensure robustness and assess generalization capabilities. The dataset was randomly partitioned into five equal subsets; in each iteration, one subset served as the test set while the remaining four constituted the training set. This procedure was repeated five times, and the results were averaged to mitigate randomness and enhance statistical reliability.
Given the highly imbalanced nature of the herb–disease association prediction task, AUPR is adopted as the primary evaluation metric, as it emphasizes the model’s ability to correctly identify positive associations.
Section 3.2, Experimental and comparative results:
The results in Tables 5 and 6 present a comparative analysis of various methods under five-fold cross-validation. GHF-ACL consistently outperforms the other models in AUPR, Recall, and F1 Score across all evaluated datasets, while AUC performance varies across datasets due to differences in class imbalance severity.
Additionally, the model attained a Recall of 0.9922 and an F1 Score of 0.8510, indicating high sensitivity to true positives while maintaining a balanced performance.
While GHF-ACL did not consistently yield the highest Accuracy or Precision, it exhibited stable overall performance. Notably, it secured particularly strong results in AUPR and F1 Score, which are considered more reliable indicators of model efficacy in imbalanced scenarios.
Additionally, the slightly lower precision observed on the Cdata dataset can be attributed to its extreme sparsity. In such cases, GHF-ACL tends to emphasize higher recall to ensure broader coverage of potential associations.
Conclusion:
Extensive experiments demonstrate that GHF-ACL achieves superior performance in AUPR, Recall, and F1 Score on both herbal and cross-domain datasets, demonstrating particular strength in identifying true positive associations under class-imbalanced conditions.
What the paper's own numbers show
Method. For one set of thresholded predictions, F1 = 2PR/(P+R), the harmonic mean of precision P and recall R. It can never exceed the arithmetic mean (P+R)/2. With P = 0.7217 it cannot exceed 2P/(1+P) = 0.8384 even at perfect recall. The paper's metrics are averages over five folds. The harmonic mean is concave and increasing in both arguments, so by Jensen's inequality the mean of the fold F1 values cannot exceed the harmonic mean of the mean precision and mean recall: mean(F1ᵢ) ≤ 2·P̄·R̄/(P̄+R̄). That makes 2PR/(P+R) of the reported values the ceiling for a reported mean F1. A mean F1 below the ceiling is possible when folds differ. I checked the ceiling three ways. A constrained optimisation over five folds reaches exactly the ceiling and never more. A random search over 200,000 five-fold sets never exceeds it. A third variant computes F1 once on predictions pooled over folds while P and R stay fold means. Assuming folds of equal size, as the paper describes (Section 3.1: "randomly partitioned into five equal subsets"), this raises the ceiling by at most 0.0020. Every one of the 37 F1 values in Tables 5, 6, 7 and 9 was tested, allowing ±0.00005 for four-decimal rounding of each input. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| GHF-ACL F1, Ethnobotany | Table 6 | 0.9885 (P 0.7217, R 0.9955) | ceiling 2PR/(P+R) = 0.8368; (P+R)/2 = 0.8586 |
| Rounding margin, Ethnobotany | Table 6 | F1 ≥ 0.98845 | ceiling ≤ 0.83682 over the rounding range of P and R |
| Highest five-fold mean F1, Ethnobotany | Table 6 | 0.9885 | 0.8368 (numerical optimum); 0.8368 if F1 is pooled over folds |
| Precision needed for F1 0.9885 at R 0.9955 | Table 6 | P 0.7217 | 0.9816 |
| F1 at P 0.7217 with recall 1.0 | Table 6 | 0.9885 | 0.8384 |
| GHF-ACL F1, Cdata | Table 5 | 0.9241 (P 0.6355, R 0.9528) | ceiling 0.7625; (P+R)/2 = 0.7941; pooled-F1 ceiling 0.7637 |
| Precision needed for F1 0.9241 at R 0.9528 | Table 5 | P 0.6355 | 0.8971 |
| GHF-ACL F1, LRSSL | Table 5; Table 9 (Full Model); Results text | 0.8510 (P 0.7247, R 0.9922) | ceiling 0.8376 (pooled-F1 ceiling 0.8376) |
| Ablation w/o Structural Interaction Alignment, F1 | Table 9 | 0.5456 (P 0.2855, R 0.9329) | ceiling 0.4372 (pooled 0.4392) |
| Ablation w/o Cluster-level Contrastive Loss, F1 | Table 9 | 0.5901 (P 0.3781, R 0.9453) | ceiling 0.5402 (pooled 0.5416) |
| GHF-ACL F1, Ldata | Table 5 | 0.5809 (P 0.6719, R 0.8865) | 0.7644; reported value is below the ceiling, which is possible if folds differed widely |
| GHF-ACL F1, Hdata cold start | Table 7 | 0.8572 (P 0.8281, R 0.8880) | 0.8570; consistent |
| Ablation w/o High-order / w/o Hierarchical CL, F1 | Table 9 | 0.4345 / 0.2459 | 0.4345 / 0.3158 (the second is below the ceiling, possible) |
| All 28 baseline rows | Tables 5, 6, 7 | e.g. HGHDA 0.8400; DD-HGNN+ Cdata 0.8819 | e.g. 0.8402; 0.8819. Largest excess over the ceiling +0.0006; largest shortfall −0.0112 |
| Macro (two-class) F1 ceiling, Ethnobotany / Cdata | Tables 6, 5 | 0.9885 / 0.9241 | (ceiling + 1)/2 = 0.9184 / 0.8812 |
| Cdata F1 under other averaging conventions | Table 5 (accuracy 0.9996) | 0.9241 | prevalence implied by accuracy, P and R 0.00067; macro 0.8811, weighted 0.9996, micro = accuracy 0.9996 |
| Best F1 with consistent values, Ethnobotany | Table 6 | GHF-ACL 0.9885 (first) | GHF-ACL ≤ 0.8368 < HGHDA 0.8400 |
| Best F1 with consistent values, Cdata | Table 5 | GHF-ACL 0.9241 (first) | GHF-ACL ≤ 0.7625 < DD-HGNN+ 0.8819 |
| Best F1, LRSSL | Table 5 | GHF-ACL 0.8510 (first) | GHF-ACL ≤ 0.8376, still above DD-HGNN+ 0.8289 |
| Best F1, Ldata | Table 5 | GHF-ACL 0.5809 | already below DD-HGNN+ 0.6325 as reported |
The baseline rows show that the method fits these tables. Their F1 values sit on the ceiling or just below it, which is what fold averaging produces. The GHF-ACL rows on Ethnobotany and Cdata sit 0.15 and 0.16 above it (0.1517 and 0.1616). Each excess is larger than the margin by which the reported GHF-ACL F1 leads the best baseline (0.1485 over HGHDA on Ethnobotany, 0.0422 over DD-HGNN+ on Cdata). The LRSSL row is 0.0134 above its ceiling, and the two ablation rows are 0.05 and 0.11 above theirs. Neither rounding nor any of the averaging schemes above closes these gaps.
Figure
Figure 1. One row per GHF-ACL result. Orange dots are the F1 values reported in the paper. Blue dots are 2PR/(P+R) computed from the precision and recall in the same row, which is the highest F1 that a five-fold mean can reach given those mean precision and recall values. Black ticks mark the best F1 reported for any other model on the same dataset; the ablation rows have no comparator. Shaded rows have a reported F1 above the ceiling. On Ethnobotany and Cdata the reported F1 ranks GHF-ACL first, while the ceiling ranks it below HGHDA (0.8400) and DD-HGNN+ (0.8819). On Ldata the reported F1 is below the ceiling, which is possible. On Hdata (Table 7) the two values agree.
Figure 2. Every F1 value in Tables 5, 6, 7 and 9 (the Table 9 full model duplicates the LRSSL row of Table 5 and is drawn once), plotted against 2PR/(P+R) of the precision and recall in the same row. The line marks reported = recomputed. Points above it are F1 values larger than their own precision and recall allow. All 28 baseline rows (open gray circles) lie on or just below the line. The points far above it are GHF-ACL rows: Ethnobotany, Cdata and two ablation variants, with LRSSL just above. This indicates that the problem is specific to these rows rather than a convention applied to the whole table.
Why it matters
F1 is one of the three metrics on which the Abstract claims "significant improvements over the best-performing baseline model", along with AUPR and recall. The contributions list, the Results ("consistently outperforms the other models in AUPR, Recall, and F1 Score across all evaluated datasets") and the Conclusion repeat the claim. The Results also give F1 as one of two metrics on which GHF-ACL "secured particularly strong results". With F1 values consistent with the paper's own precision and recall, GHF-ACL would not rank first on Ethnobotany (at most 0.8368 against HGHDA's 0.8400) or on Cdata (at most 0.7625 against DD-HGNN+'s 0.8819); because these are upper bounds, it would at best rank second on each. On Ldata the paper's own reported F1 (0.5809) is already below DD-HGNN+ (0.6325). That leaves LRSSL among the Table 5 and 6 datasets, plus the cold-start Hdata comparison in Table 7, as the settings where an F1 lead for GHF-ACL remains possible. The statement that GHF-ACL leads on F1 across all evaluated datasets is therefore not supported by the paper's tables.
The ablation conclusions drawn from F1 in Table 9 are also less secure, because two of the four ablation F1 values exceed their own ceilings. For example, the F1 drop from removing structural alignment is 0.3054 as reported but 0.4004 if the ceilings of both rows are used. The direction of the ablation effects does not change.
The AUPR values, which the paper names as its primary metric, and the headline AUPR gains (+4.8% on LRSSL, +3.81% on Cdata) do not depend on these F1 values. The conclusion that GHF-ACL performs better than or as well as the baselines on AUPR stands. The error concerns a key secondary result and the claims built on it, not the paper's main conclusion.
Explanations considered
- Rounding. This does not account for the numbers. Over the full rounding range of the four-decimal inputs, the Ethnobotany ceiling is at most 0.83682, while the smallest value that rounds to 0.9885 is 0.98845. The LRSSL gap (0.0134) is also more than a hundred times the rounding margin.
- Averaging over folds or repeated runs. This does not account for the numbers. If P, R and F1 are each averaged over the same folds, the mean F1 cannot exceed 2P̄R̄/(P̄+R̄), and the numerical optimum reaches exactly that value. Computing F1 once on predictions pooled over folds, while P and R are fold means, raises the ceiling by at most 0.0020 (Cdata 0.7637, against a reported 0.9241), assuming the five folds are of equal size as Section 3.1 states. Averaging can explain an F1 below the ceiling, as on Ldata, but not one above it.
- Choosing the best epoch or threshold for each metric separately. For P, R and F1 that all come from one fold's predictions, this cannot raise the reported F1 above the ceiling. F1 at any evaluation point is at most the harmonic mean of that point's P and R, and the harmonic mean only grows when P or R grows. The ceiling can be exceeded only if the reported precision and recall were taken from a different evaluation than the F1: another epoch, threshold or run. The paper describes no such procedure. The released code does not settle the question (see Status). If this is the explanation, the GHF-ACL rows combine metrics from different evaluations, and the F1 comparison with the baselines, whose rows are internally consistent, is not like-for-like.
- A different F1 convention (macro, weighted or micro averaging over classes). This does not account for Cdata. Using the accuracy, precision and recall in the same row, positives make up about 0.067% of evaluated pairs. That gives macro F1 0.8811, weighted F1 0.9996 and micro F1 0.9996, and none of them is 0.9241. For Ethnobotany, macro F1 could be at most 0.9184. A support-weighted F1 would equal 0.9885 only if positives were 3.21% of the evaluated pairs, which the paper does not report. That convention would also differ from the one used for every baseline row and for the other GHF-ACL rows, all of which match the positive-class formula. The released code computes the positive-class F1, 2PR/(P+R).
- A typing error in precision rather than in F1. This is possible for Ethnobotany, where the reported F1 would fit a precision of 0.9816. That would make GHF-ACL also best on precision there, which conflicts with the Results statement that it "did not consistently yield the highest Accuracy or Precision". For Cdata, F1 0.9241 would need a precision of 0.8971, above DD-HGNN+'s 0.8356. The Results, however, specifically discuss "the slightly lower precision observed on the Cdata dataset", which fits the printed 0.6355 and not 0.8971. Either way, at least one value in each of these rows is wrong.
- Conversion or typesetting error in the archived text. This is excluded. The values are the same in the PMC text, the Europe PMC XML, the publisher's table images and the publisher's PDF, and in both publisher versions (see Status).
- A disclosed alternative in a supplement. This is excluded. The article has no data supplement. Its only supplementary files are the two response-to-reviewers letters (see Status).
Status
- Version of record. The publisher's page (journals.plos.org/ploscompbiol, reached via doi.org) presents Tables 5 and 6 as images. The full-size image of Table 6 shows the Ethnobotany GHF-ACL row as 0.9324 | 0.7217 | 0.9955 | 0.9885 | 0.9296 under AUPR | Precision | Recall | F1 Score | AUC, with the cells aligned under their headers. The publisher's 28-page PDF (Table 6 on p. 18) gives the same text. The Cdata GHF-ACL row of Table 5 reads the same in the PDF and in the Europe PMC full-text XML. The publisher lists two versions, an uncorrected proof (rev=1) and the current version (rev=2). Crossref records the new version on 2026-07-02. The table image files are byte-identical across the two versions, and the text differences between them are limited to affiliation names and whitespace, with no changed numbers. Every passage quoted above was confirmed verbatim against the PMC-archived text with
src/show.py. - Code. The Data Availability statement points to a public GitHub repository (the repository "TCM"; checked at its 2025-12-02 state). Its
main2.pycomputes precision and recall with Keras metrics on predictions thresholded at 0.5, and F1 as2 * precision_value * recall_value / (precision_value + recall_value)from those two values. That is the standard positive-class F1, which is subject to the ceiling used here. In the released script this function is applied to the training matrix during training. The cross-validation loop does not compute or aggregate test-set metrics: its metric array is initialised and never filled. The repository holds no per-fold results, so it shows neither how the reported table values were produced nor what the correct F1 values are. - Supplements and peer review. The only supplementary files are two response-to-reviewers letters (pcbi.1014461.s002.docx and s003.docx). Both letters and the published peer-review history were read. Neither the reviewers nor the authors raise the inconsistency between F1, precision and recall.
- Corrections and comments. As of 2026-10-24 there is no correction, erratum, expression of concern or retraction. Crossref lists only the 2026-07-02 new-version record, and PMC and Europe PMC list no notices. A web search found no PubPeer thread or other published comment. PubPeer's own search refused automated access (HTTP 403, human verification required), so PubPeer could not be checked directly.
- Sources. PMC article PMC13327522; Europe PMC full-text XML; the publisher's article page, table images, PDF and peer-review history at journals.plos.org; Crossref metadata for doi:10.1371/journal.pcbi.1014461; the GitHub code repository named in the Data Availability statement.
Suggested correction
- Recompute the GHF-ACL F1 scores for Ethnobotany (Table 6), Cdata, LRSSL (Table 5) and the two Table 9 ablation rows above their ceilings (w/o Structural Interaction Alignment and w/o Cluster-level Contrastive Loss) from the same fold-level predictions as the reported precision and recall, or correct the precision and recall if those are the wrong values. Report the per-fold precision, recall and F1 in a supplement so that readers can check the averages.
- Apply the same check to the remaining GHF-ACL and ablation rows (Ldata in Table 5; w/o Hierarchical Contrastive Loss and w/o High-order Heterogeneous Structure in Table 9), and state how the fold averages were formed. The reported F1 for Ldata and for w/o Hierarchical Contrastive Loss is well below the ceiling (0.5809 against 0.7644; 0.2459 against 0.3158), which fold averaging allows only when folds differ widely.
- Revise the F1 claims in the Abstract ("significant improvements … F1 score"), the contributions list, Section 3.2 ("consistently outperforms the other models in AUPR, Recall, and F1 Score across all evaluated datasets"; "particularly strong results in AUPR and F1 Score") and the Conclusion so that they match the corrected values. On values consistent with the paper's own precision and recall, GHF-ACL does not have the highest F1 on Ethnobotany or Cdata, and on Ldata its reported F1 (0.5809) is already below DD-HGNN+ (0.6325).
- Update the released code so that it includes the test-set evaluation and fold aggregation that produced Tables 5–9.
Recomputation script (python recompute.py.txt)
"""Check the F1 scores of PMC13327522 against their own precision and recall
(PLoS Comput Biol 2026, doi:10.1371/journal.pcbi.1014461, GHF-ACL herb-disease association prediction).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (Tables 5, 6, 7 and 9), as
(precision, recall, reported F1). Accuracy is kept where the table gives it (Table 5).
Facts used:
F1 = 2PR/(P+R), the harmonic mean HM(P, R), for one set of thresholded predictions.
HM(P, R) <= (P+R)/2 (arithmetic mean, AM), and HM(P, R) <= 2P/(1+P) for any recall <= 1.
HM is concave and increasing in each argument, so if P, R and F1 are each averaged over the same
k folds, mean(F1_i) <= HM(mean P_i, mean R_i) (Jensen). This is checked numerically below.
"""
import os
import numpy as np
from scipy.optimize import minimize, brentq
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import Patch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
# (table, dataset, model): (P, R, F1 reported, accuracy or None)
ROWS = {
("T5", "LRSSL", "LAGCN"): (0.0091, 0.6875, 0.0168, 0.3955),
("T5", "LRSSL", "HDGAT"): (0.0787, 0.0855, 0.0797, 0.9976),
("T5", "LRSSL", "DRWBNCF"): (0.3114, 0.3027, 0.2958, 0.9693),
("T5", "LRSSL", "DD-HGNN+"): (0.9082, 0.7624, 0.8289, 0.9996),
("T5", "LRSSL", "AdaDR"): (0.5786, 0.4038, 0.4716, 0.9947),
("T5", "LRSSL", "DRAGNN"): (0.4943, 0.5654, 0.5274, 0.9940),
("T5", "LRSSL", "GHF-ACL"): (0.7247, 0.9922, 0.8510, 0.9988),
("T5", "Ldata", "LAGCN"): (0.2892, 0.3482, 0.3140, 0.9616),
("T5", "Ldata", "HDGAT"): (0.2762, 0.3221, 0.2964, 0.9614),
("T5", "Ldata", "DRWBNCF"): (0.3794, 0.5413, 0.4460, 0.8460),
("T5", "Ldata", "DD-HGNN+"): (0.6527, 0.6128, 0.6325, 0.9820),
("T5", "Ldata", "AdaDR"): (0.4932, 0.5594, 0.5228, 0.8831),
("T5", "Ldata", "DRAGNN"): (0.3922, 0.5190, 0.4463, 0.8524),
("T5", "Ldata", "GHF-ACL"): (0.6719, 0.8865, 0.5809, 0.8140),
("T5", "Cdata", "LAGCN"): (0.0142, 0.2354, 0.0269, 0.9679),
("T5", "Cdata", "HDGAT"): (0.0172, 0.1122, 0.0296, 0.9861),
("T5", "Cdata", "DRWBNCF"): (0.3854, 0.3029, 0.3336, 0.9883),
("T5", "Cdata", "DD-HGNN+"): (0.8356, 0.9336, 0.8819, 0.9995),
("T5", "Cdata", "AdaDR"): (0.7385, 0.5375, 0.6216, 0.9939),
("T5", "Cdata", "DRAGNN"): (0.6848, 0.5549, 0.6122, 0.9934),
("T5", "Cdata", "GHF-ACL"): (0.6355, 0.9528, 0.9241, 0.9996),
("T6", "Ethnobotany", "HGHDA"): (0.8196, 0.8618, 0.8400, None),
("T6", "Ethnobotany", "HTINet(KNN)"): (0.5016, 0.9956, 0.6671, None),
("T6", "Ethnobotany", "HTINet(RF)"): (0.5566, 0.8918, 0.6852, None),
("T6", "Ethnobotany", "GHF-ACL"): (0.7217, 0.9955, 0.9885, None),
("T7", "Hdata", "GHF-ACL"): (0.8281, 0.8880, 0.8572, None),
("T7", "Hdata", "GeNNius"): (0.5751, 0.7940, 0.6650, None),
("T7", "Hdata", "HDCTI"): (0.7034, 0.8040, 0.7435, None),
("T7", "Hdata", "LAGCN"): (0.3820, 0.7652, 0.5102, None),
("T7", "Hdata", "HDGAT"): (0.3129, 0.7281, 0.4362, None),
("T7", "Hdata", "DD-HGNN+"): (0.7215, 0.8127, 0.7646, None),
("T7", "Hdata", "AdaDR"): (0.3452, 0.6893, 0.4596, None),
("T7", "Hdata", "DRAGNN"): (0.5230, 0.7617, 0.6203, None),
("T9", "LRSSL", "Full Model"): (0.7247, 0.9922, 0.8510, 0.99877),
("T9", "LRSSL", "w/o High-order Heterogeneous Structure"): (0.2808, 0.9597, 0.4345, 0.63682),
("T9", "LRSSL", "w/o Structural Interaction Alignment"): (0.2855, 0.9329, 0.5456, 0.37407),
("T9", "LRSSL", "w/o Hierarchical Contrastive Loss"): (0.1914, 0.9022, 0.2459, 0.28183),
("T9", "LRSSL", "w/o Cluster-level Contrastive Loss"): (0.3781, 0.9453, 0.5901, 0.46601),
}
H = 5e-5 # half a unit in the 4th decimal
def hm(p, r):
p = np.asarray(p, float)
r = np.asarray(r, float)
return 2 * p * r / (p + r)
def classify(p, r, f):
"""Compare a reported F1 with the bounds that its own P and R impose, allowing for rounding."""
hm_hi = float(hm(p + H, r + H)) # largest HM the rounded P, R allow
am_hi = (p + r) / 2 + H # largest AM the rounded P, R allow
f_lo = f - H # smallest F1 that rounds to the reported value
if f_lo > am_hi:
return "ABOVE AM (impossible)"
if f_lo > hm_hi + 1e-3:
return "ABOVE HM (impossible if same-fold means)"
if f_lo > hm_hi:
return "at the ceiling (above it by < 0.001)"
if f < float(hm(p, r)) - 0.02:
return "below HM (possible under fold averaging)"
return "consistent"
R = {}
print("=== 1. Every F1 in Tables 5, 6, 7 and 9 against HM(P, R) and AM(P, R) ===")
print(f"{'tab':4}{'data':12}{'model':40}{'P':>7}{'R':>7}{'F1 rep':>8}{'HM':>8}{'AM':>8}{'rep-HM':>8} verdict")
for (t, d, m), (p, r, f, acc) in ROWS.items():
h = float(hm(p, r))
a = (p + r) / 2
v = classify(p, r, f)
R[(t, d, m)] = dict(P=p, R=r, F1=f, HM=h, AM=a, verdict=v, acc=acc)
print(f"{t:4}{d:12}{m:40}{p:7.4f}{r:7.4f}{f:8.4f}{h:8.4f}{a:8.4f}{f - h:+8.4f} {v}")
base = [k for k in R if not k[2].startswith(("GHF-ACL", "Full", "w/o"))]
dev = [abs(R[k]["F1"] - R[k]["HM"]) for k in base]
print(f"\nBaseline rows (other methods): n = {len(base)}, max |reported F1 - HM| = {max(dev):.4f}, "
f"rows within 0.005: {sum(x <= 0.005 for x in dev)}")
print(f"Baseline rows: largest excess of reported F1 over HM = {max(R[k]['F1'] - R[k]['HM'] for k in base):+.4f}; "
f"largest shortfall = {min(R[k]['F1'] - R[k]['HM'] for k in base):+.4f}")
for k in base:
if abs(R[k]["F1"] - R[k]["HM"]) > 0.005:
print(" baseline row off by > 0.005:", k, f"{R[k]['F1'] - R[k]['HM']:+.4f}")
print("\n=== 2. Rounding margins for the rows above the bounds ===")
for k in [("T6", "Ethnobotany", "GHF-ACL"), ("T5", "Cdata", "GHF-ACL"), ("T5", "LRSSL", "GHF-ACL"),
("T9", "LRSSL", "w/o Structural Interaction Alignment"), ("T9", "LRSSL", "w/o Cluster-level Contrastive Loss"),
("T7", "Hdata", "GHF-ACL")]:
p, r, f = R[k]["P"], R[k]["R"], R[k]["F1"]
print(f"{k[1]:12}{k[2]:40} max HM over rounding = {float(hm(p + H, r + H)):.5f}; "
f"max AM = {(p + r) / 2 + H:.5f}; min F1 that rounds to {f:.4f} = {f - H:.5f}")
print("\n=== 3. What the reported F1 would need ===")
for k in [("T6", "Ethnobotany", "GHF-ACL"), ("T5", "Cdata", "GHF-ACL"), ("T5", "LRSSL", "GHF-ACL")]:
p, r, f = R[k]["P"], R[k]["R"], R[k]["F1"]
p_need = f * r / (2 * r - f)
f_r1 = 2 * p / (1 + p)
R[k].update(p_need=p_need, f_r1=f_r1)
print(f"{k[1]:12} precision needed for F1 = {f:.4f} at R = {r:.4f}: {p_need:.4f} (reported {p:.4f}); "
f"F1 at reported P = {p:.4f} even with recall 1.0: {f_r1:.4f}")
print("\n=== 4. Fold averaging: highest mean-of-5-folds F1 given the reported mean P and mean R ===")
def max_mean_f1(P, Rm, k=5, starts=40, seed=0):
rng = np.random.default_rng(seed)
cons = [{"type": "eq", "fun": lambda x: x[:k].mean() - P},
{"type": "eq", "fun": lambda x: x[k:].mean() - Rm}]
best = 0.0
for _ in range(starts):
x0 = np.r_[np.clip(rng.normal(P, 0.2, k), 0.01, 1), np.clip(rng.normal(Rm, 0.2, k), 0.01, 1)]
res = minimize(lambda x: -np.mean(hm(x[:k], x[k:])), x0, bounds=[(1e-6, 1)] * (2 * k),
constraints=cons, method="SLSQP")
if res.success:
best = max(best, -res.fun)
return best
for k in [("T6", "Ethnobotany", "GHF-ACL"), ("T5", "Cdata", "GHF-ACL"), ("T5", "LRSSL", "GHF-ACL"),
("T9", "LRSSL", "w/o Structural Interaction Alignment"), ("T9", "LRSSL", "w/o Cluster-level Contrastive Loss")]:
mx = max_mean_f1(R[k]["P"], R[k]["R"])
R[k]["maxcv"] = mx
print(f"{k[1]:12}{k[2]:40} numerical max = {mx:.4f}; HM(mean P, mean R) = {R[k]['HM']:.4f}; reported {R[k]['F1']:.4f}")
rng = np.random.default_rng(1)
Ps = rng.uniform(0.01, 1, (200000, 5))
Rs = rng.uniform(0.01, 1, (200000, 5))
gap = hm(Ps, Rs).mean(axis=1) - hm(Ps.mean(axis=1), Rs.mean(axis=1))
print(f"Jensen check over 200,000 random 5-fold sets: max of [mean F1_i - HM(mean P, mean R)] = {gap.max():.2e} (never positive)")
def max_pooled_f1(P, Rm, k=5, starts=40, seed=0):
"""P and R reported as fold means, F1 computed once on the predictions pooled over the k folds.
Folds hold equal numbers of test positives (KFold on the positive pairs), so TP_i is proportional to R_i,
predicted positives are TP_i / P_i, pooled precision = sum TP_i / sum(TP_i / P_i), pooled recall = mean R_i."""
rng = np.random.default_rng(seed)
cons = [{"type": "eq", "fun": lambda x: x[:k].mean() - P},
{"type": "eq", "fun": lambda x: x[k:].mean() - Rm}]
def neg(x):
p, r = x[:k], x[k:]
pp = r.sum() / (r / p).sum()
return -float(hm(pp, r.mean()))
best = 0.0
for _ in range(starts):
x0 = np.r_[np.clip(rng.normal(P, 0.2, k), 0.01, 1), np.clip(rng.normal(Rm, 0.05, k), 0.01, 1)]
res = minimize(neg, x0, bounds=[(1e-3, 1)] * (2 * k), constraints=cons, method="SLSQP")
if res.success:
best = max(best, -res.fun)
return best
print("F1 pooled over folds while P and R are fold means (highest value possible):")
for k in [("T6", "Ethnobotany", "GHF-ACL"), ("T5", "Cdata", "GHF-ACL"), ("T5", "LRSSL", "GHF-ACL"),
("T9", "LRSSL", "w/o Structural Interaction Alignment"), ("T9", "LRSSL", "w/o Cluster-level Contrastive Loss")]:
mx = max_pooled_f1(R[k]["P"], R[k]["R"])
R[k]["maxpool"] = mx
print(f"{k[1]:12}{k[2]:40} max pooled F1 = {mx:.4f}; reported {R[k]['F1']:.4f}")
rise = max(R[k]["maxpool"] - R[k]["HM"] for k in R if "maxpool" in R[k])
print(f"Largest rise of the pooled-F1 ceiling over 2PR/(P+R), assuming five folds with equal numbers of test positives: {rise:.4f}")
print("\n=== 5. Other F1 conventions ===")
e = R[("T6", "Ethnobotany", "GHF-ACL")]
c = R[("T5", "Cdata", "GHF-ACL")]
for name, row in [("Ethnobotany", e), ("Cdata", c)]:
print(f"{name:12} macro (two-class) F1 upper bound (HM + 1)/2 = {(row['HM'] + 1) / 2:.4f}; reported {row['F1']:.4f}")
def two_class(p, r, prev):
"""Per unit of positives: TP = r, FN = 1 - r, FP = r(1-p)/p, negatives = (1-prev)/prev."""
tp, fn, fp = r, 1 - r, r * (1 - p) / p
neg = (1 - prev) / prev
tn = neg - fp
f1n = 2 * tn / (2 * tn + fn + fp)
f1p = 2 * tp / (2 * tp + fn + fp)
acc = (tp + tn) / (1 + neg)
return f1p, f1n, acc
# Cdata: the prevalence implied by the reported accuracy, P and R
err_per_pos = (1 - c["R"]) + c["R"] * (1 - c["P"]) / c["P"]
prev_c = (1 - c["acc"]) / err_per_pos
f1p, f1n, acc = two_class(c["P"], c["R"], prev_c)
print(f"Cdata: prevalence of positives implied by accuracy {c['acc']} = {prev_c:.6f}; "
f"negative-class F1 = {f1n:.4f}; macro F1 = {(f1p + f1n) / 2:.4f}; "
f"weighted F1 = {prev_c * f1p + (1 - prev_c) * f1n:.4f}; micro F1 = accuracy = {acc:.4f}; reported {c['F1']:.4f}")
# Ethnobotany: prevalence at which a support-weighted F1 would equal 0.9885 (no accuracy reported)
g = lambda pi: pi * two_class(e["P"], e["R"], pi)[0] + (1 - pi) * two_class(e["P"], e["R"], pi)[1] - e["F1"]
prev_e = brentq(g, 1e-4, 0.5)
print(f"Ethnobotany: a support-weighted F1 would equal {e['F1']} only if positives were {100 * prev_e:.2f}% of evaluated pairs")
print("\n=== 6. F1 ranking with consistent values ===")
for d, tab in [("Ethnobotany", "T6"), ("Cdata", "T5"), ("LRSSL", "T5"), ("Ldata", "T5"), ("Hdata", "T7")]:
rows = {k[2]: R[k] for k in R if k[0] == tab and k[1] == d}
others = {m: v["F1"] for m, v in rows.items() if not m.startswith("GHF")}
best = max(others, key=others.get)
g_ = rows[[m for m in rows if m.startswith("GHF")][0]]
print(f"{d:12} GHF-ACL reported F1 {g_['F1']:.4f}, HM {g_['HM']:.4f}; best other model {best} {others[best]:.4f}")
for d, tab, bm in [("Ethnobotany", "T6", "HGHDA"), ("Cdata", "T5", "DD-HGNN+")]:
g_ = R[(tab, d, "GHF-ACL")]
b_ = R[(tab, d, bm)]
print(f"{d:12} excess of reported GHF-ACL F1 over its ceiling = {g_['F1'] - g_['HM']:.4f}; "
f"reported F1 lead of GHF-ACL over best baseline {bm} = {g_['F1'] - b_['F1']:.4f}")
full = R[("T9", "LRSSL", "Full Model")]
for m in ("w/o Structural Interaction Alignment", "w/o Cluster-level Contrastive Loss"):
a = R[("T9", "LRSSL", m)]
print(f"Ablation {m}: F1 drop from full model as reported = {full['F1'] - a['F1']:.4f}; "
f"using both ceilings = {full['HM'] - a['HM']:.4f}")
print(f"F1 values checked: {len(R)} rows, {len(R) - 1} distinct (Table 9 Full Model repeats Table 5 LRSSL)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": "#b9b8b3",
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: GHF-ACL rows, reported F1 vs the ceiling HM(P, R), with the best other model for context
F1ROWS = [
(("T6", "Ethnobotany", "GHF-ACL"), "Ethnobotany (Table 6)", ("T6", "Ethnobotany", "HGHDA"), "HGHDA"),
(("T5", "Cdata", "GHF-ACL"), "Cdata (Table 5)", ("T5", "Cdata", "DD-HGNN+"), "DD-HGNN+"),
(("T5", "LRSSL", "GHF-ACL"), "LRSSL (Tables 5, 9)", ("T5", "LRSSL", "DD-HGNN+"), "DD-HGNN+"),
(("T9", "LRSSL", "w/o Structural Interaction Alignment"), "LRSSL ablation, w/o\nstructural alignment (Table 9)", None, None),
(("T9", "LRSSL", "w/o Cluster-level Contrastive Loss"), "LRSSL ablation, w/o\ncluster-level CL (Table 9)", None, None),
(("T5", "Ldata", "GHF-ACL"), "Ldata (Table 5)", ("T5", "Ldata", "DD-HGNN+"), "DD-HGNN+"),
(("T7", "Hdata", "GHF-ACL"), "Hdata cold start (Table 7)", ("T7", "Hdata", "DD-HGNN+"), "DD-HGNN+"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.9))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
n = len(F1ROWS)
for i, (k, lab, bk, bname) in enumerate(F1ROWS):
y = n - 1 - i
v = R[k]
ax.axhspan(y - 0.42, y + 0.42, xmin=0, xmax=1, color="#e9e8e3" if v["F1"] > v["HM"] + 1e-3 else BG, zorder=0, lw=0)
ax.plot([v["HM"], v["F1"]], [y, y], color="#9a9893", lw=2, zorder=2)
ax.plot(v["HM"], y, "o", ms=9, color=BLU, zorder=4)
ax.plot(v["F1"], y, "o", ms=9, color=ORA, zorder=5)
if bk is not None:
ax.plot(R[bk]["F1"], y, marker="|", ms=16, mew=2, color=TXT, zorder=3)
off = 0.012
if abs(v["F1"] - v["HM"]) < 0.01:
ax.text(v["F1"] + off, y + 0.2, f"{v['F1']:.4f}", ha="left", va="bottom", color=ORA, fontsize=10)
ax.text(v["HM"] - off, y + 0.2, f"{v['HM']:.4f}", ha="right", va="bottom", color=BLU, fontsize=10)
elif v["F1"] >= v["HM"]:
ax.text(v["HM"] - off, y + 0.2, f"{v['HM']:.4f}", ha="right", va="bottom", color=BLU, fontsize=10)
ax.text(v["F1"] + off, y + 0.2, f"{v['F1']:.4f}", ha="left", va="bottom", color=ORA, fontsize=10)
else:
ax.text(v["HM"] + off, y + 0.2, f"{v['HM']:.4f}", ha="left", va="bottom", color=BLU, fontsize=10)
ax.text(v["F1"] - off, y + 0.2, f"{v['F1']:.4f}", ha="right", va="bottom", color=ORA, fontsize=10)
ax.set_yticks(range(n))
ax.set_yticklabels([r[1] for r in F1ROWS][::-1], fontsize=10)
ax.set_xlim(0.30, 1.0)
ax.set_xticks(np.arange(0.3, 1.01, 0.1))
ax.set_ylim(-0.6, n - 0.4)
ax.set_xlabel("F1 score")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=ORA, label="F1 reported for GHF-ACL"),
Line2D([], [], marker="o", ls="", ms=9, color=BLU, label="Highest F1 its own P and R allow, 2PR/(P+R)"),
Line2D([], [], marker="|", ls="", ms=14, mew=2, color=TXT, label="Best F1 reported for another model"),
Patch(facecolor="#e9e8e3", edgecolor="none", label="Shaded: reported F1 above the ceiling")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(-0.47, 1.0), ncol=2, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.0)
fig.subplots_adjust(left=0.33, right=0.90, top=0.85, bottom=0.10)
save(fig, "fig1")
# Figure 2: every row in Tables 5, 6, 7, 9: reported F1 against 2PR/(P+R)
fig, ax = plt.subplots(figsize=(7.2, 5.4))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ax.fill_between([0, 1], [0, 1], [1, 1], color="#f1f0ec", lw=0, zorder=0)
ax.plot([0, 1], [0, 1], color=SEC, lw=1, zorder=1)
ax.text(0.04, 0.93, "Region above the line: F1 larger than\n2PR/(P+R) of its own P and R", color=SEC, fontsize=10, va="top")
xb = [R[k]["HM"] for k in base]
yb = [R[k]["F1"] for k in base]
ax.plot(xb, yb, "o", ms=8, mfc="none", mec="#7c7b77", mew=1.5, ls="", zorder=3)
gk = [k for k in R if k not in base and k != ("T9", "LRSSL", "Full Model")]
ax.plot([R[k]["HM"] for k in gk], [R[k]["F1"] for k in gk], "o", ms=9, color=ORA, ls="", zorder=4)
LAB = {("T6", "Ethnobotany", "GHF-ACL"): ("Ethnobotany", (-0.03, 0.0), "right"),
("T5", "Cdata", "GHF-ACL"): ("Cdata", (-0.03, 0.0), "right"),
("T5", "LRSSL", "GHF-ACL"): ("LRSSL", (-0.03, 0.025), "right"),
("T7", "Hdata", "GHF-ACL"): ("Hdata", (0.03, -0.01), "left"),
("T5", "Ldata", "GHF-ACL"): ("Ldata", (0.025, 0.0), "left"),
("T9", "LRSSL", "w/o Structural Interaction Alignment"): ("ablation: w/o alignment", (-0.025, -0.012), "right"),
("T9", "LRSSL", "w/o Cluster-level Contrastive Loss"): ("ablation: w/o cluster CL", (-0.025, 0.018), "right"),
("T9", "LRSSL", "w/o Hierarchical Contrastive Loss"): ("ablation: w/o hierarchical CL", (0.03, -0.03), "left"),
("T9", "LRSSL", "w/o High-order Heterogeneous Structure"): ("ablation: w/o high-order", (0.03, -0.03), "left")}
for k, (t, (dx, dy), ha) in LAB.items():
ax.text(R[k]["HM"] + dx, R[k]["F1"] + dy, t, ha=ha, va="center", fontsize=10, color=TXT)
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.set_aspect("equal")
ax.set_xlabel("F1 recomputed from the reported precision and recall, 2PR/(P+R)")
ax.set_ylabel("F1 reported in the table")
ax.grid(color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
handles = [Line2D([], [], marker="o", ls="", ms=8, mfc="none", mec="#7c7b77", mew=1.5,
label=f"Other models (n = {len(base)})"),
Line2D([], [], marker="o", ls="", ms=9, color=ORA, label=f"GHF-ACL and its ablations (n = {len(gk)})"),
Line2D([], [], color=SEC, lw=1, label="Reported = recomputed")]
ax.legend(handles=handles, loc="lower right", frameon=False, fontsize=10)
fig.subplots_adjust(left=0.10, right=0.98, top=0.98, bottom=0.10)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract reverses the direction of the paper's exposed-versus-occluded skin connectivity result
PLoS Comput Biol (2018) · doi:10.1371/journal.pcbi.1006099 · PMC5927471
Summary
The abstract says that skin sites exposed to the external environment have less connected phage-bacteria networks and are "likely more susceptible to network degradation". The Results say the opposite: occluded sites were less connected than exposed and intermittently occluded sites (p < 0.001, Fig 4B). The Discussion also says exposed and intermittently exposed sites were more connected and predicted to be more resilient. Fig 4B as published agrees with the Results and Discussion: the median eigen centrality is about 0.176 for exposed sites and 0.138 for occluded sites, and the paper reads higher values as "more connected" throughout. The body of the paper is internally consistent, so the analysis is unaffected. But the abstract, which is what most readers and indexing services see, states one of the paper's three headline findings in the reverse direction, and so does its claim about vulnerability to degradation.
What the paper reports
Abstract:
Network structure differed between skin sites, with those exposed to the external environment being less connected and likely more susceptible to network degradation by microbial extinction events.
Results, "Network structures across the human skin landscape", second paragraph:
The average centrality of each sample was quantified using the weighted eigenvector centrality metric. Intermittently moist skin sites (dynamic sites that fluctuate between being moist and dry) were significantly more connected than the moist and sebaceous environments (p-value < 0.001, Fig 4A). Also, skin sites that were occluded from the environment were less connected than those that were constantly exposed to the environment or only intermittently occluded (p-value < 0.001, Fig 4B).
Discussion, third paragraph:
Network structure differed between environmentally exposed and occluded skin sites. The sites under greater environmental fluctuation and exposure (the exposed and intermittently exposed sites) were more connected and therefore were predicted to have a higher resilience against network degradation when random nodes are removed from the network.
Together these data suggested that body sites under greater degrees of fluctuation harbored more highly connected microbiomes that are potentially more robust to network disruption by extinction events.
Fig 4 legend:
(A) Notched box-plot depicting differences in average eigenvector centrality between moist, intermittently moist, and sebaceous skin sites and (B) occluded, intermittently occluded, and exposed sites.
Sample sizes for each group were: Moist = 81, Sebaceous = 56, IntMoist = 56, Occluded = 106, Exposed = 61, IntOccluded = 26.
What the paper's own numbers show
Method. The paper prints no group values for connectivity in text or tables. They appear only as box plots in Fig 4A and 4B. I took the Fig 4 raster (3800 × 3729 px) embedded on page 11 of the publisher's PDF. For each box I located the median line and the two hinges (quartiles) by pixel row, and converted rows to eigen-centrality units with a straight line fitted to the panel's own y-axis ticks. The fit is exact to within 1e-4, and one pixel is about 0.0003 units, so each value is good to about ±0.002. I then checked the group sizes in the legend. I also took the pairwise Wilcoxon p-values that the authors' public code repository writes for this figure (rtables/skinboxsig.tsv; R's pairwise.wilcox.test, Holm adjustment by default). Finally I compared the direction each passage states with the direction the figure shows. The paper's convention, which the Results apply to both panels, is that a higher eigen centrality means "more connected". Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's figure and data |
|---|---|---|---|
| Exposed vs occluded connectivity | Abstract | Exposed sites "less connected" and "more susceptible to network degradation" | Exposed median 0.176 > occluded median 0.138: exposed sites are the more connected |
| Occluded vs exposed and intermittently occluded | Results; Fig 4B | Occluded "less connected" (p < 0.001) | Occluded 0.138 < exposed 0.176 and IntOccluded 0.185, which agrees |
| Exposed and intermittently exposed | Discussion | "more connected", "higher resilience" | Agrees with Fig 4B |
| Fig 4B medians (Q1–Q3) | Fig 4B | Box plots only | Exposed 0.176 (0.147–0.196); IntOccluded 0.185 (0.172–0.194); Occluded 0.138 (0.094–0.173) |
| Median differences | Fig 4B | Not printed | Exposed − Occluded = +0.038; IntOccluded − Occluded = +0.047; IntOccluded − Exposed = +0.009 |
| Separation of the groups | Fig 4B | Not printed | The exposed median (0.176) lies above the occluded upper quartile (0.173) |
| Fig 4A medians (Q1–Q3), the convention check | Fig 4A; Results | IntMoist "more connected" than Moist and Sebaceous | IntMoist 0.187 (0.175–0.197) > Sebaceous 0.145 (0.122–0.167) and Moist 0.137 (0.078–0.176), so higher means "more connected" |
| Group sizes | Fig 4 legend | 81 + 56 + 56; 106 + 61 + 26 | Both groupings sum to 193 samples, which is consistent |
| Occluded vs exposed p | Results; repository table | p < 0.001 | Holm-adjusted Wilcoxon p = 1.38e-05, which agrees |
| Occluded vs IntOccluded p | Results; repository table | p < 0.001 | p = 1.38e-05, which agrees |
| IntOccluded vs exposed p | Repository table | Not stated in text | p = 0.277 (no difference) |
The Results, the Discussion, the figure and the repository's p-values agree with each other. The abstract is the only place in the paper where exposed sites are described as less connected.
Figure
Figure 1. The left panel gives the abstract's sentence on exposed skin sites. The right panel gives the Results sentence on Fig 4B and the Discussion sentence on the same comparison. All three are quoted as printed and framed in orange. The connector marks the conflict: the abstract says exposed sites are "less connected", while the Results and the Discussion say they are more connected than occluded sites. The blue strip gives the Fig 4B medians that recompute.py.txt reads from the published figure. In those medians, exposed (0.176) and intermittently occluded (0.185) sites are above occluded (0.138) sites, which agrees with the body of the paper.
Figure 2. Median eigen centrality (dot) and interquartile range (bar) for the six skin-site groups. All values are read by recompute.py.txt from the published Fig 4A and 4B and are shown on one axis that starts at zero. The moisture panel (top) fixes the paper's convention: the Results call IntMoist, the highest group, "more connected". The occlusion panel (bottom) shows exposed sites above occluded sites. The orange label gives the abstract's statement, which runs the other way. Group sizes are from the Fig 4 legend.
Why it matters
- It is a headline finding. The abstract gives three findings from this study: networks are person-specific, high-fat diets go with less connected networks, and skin networks depend on environmental exposure. The third is printed with the wrong sign.
- The ecological inference is reversed too. The abstract says exposed sites are "likely more susceptible to network degradation by microbial extinction events". The Discussion draws the opposite conclusion from the same result: exposed and intermittently exposed sites are "predicted to have a higher resilience against network degradation". It builds its broader thesis on that conclusion, that "body sites under greater degrees of fluctuation harbored more highly connected microbiomes". A reader of the abstract alone would take away the opposite of the paper's conclusion.
- Abstracts travel further than full texts. Indexing services, citing papers and reviews usually quote the abstract. A study citing this paper for "exposed skin sites have fragile phage-bacteria networks" would be citing the reverse of what the data show.
- What is not affected. The analysis, Fig 4, the p-values, the Results and the Discussion are consistent with one another. The abstract's general framing still holds: network structure differs between skin sites, and environment may shape phage-bacteria interactions. Only the direction of the skin example is wrong.
Explanations considered
- The abstract compares exposed sites with a different group. It cannot. The occlusion grouping has only three levels. Exposed sites are above occluded sites (0.176 vs 0.138, p = 1.38e-05) and do not differ from intermittently occluded sites (p = 0.277). No comparison in the paper makes exposed sites less connected. The moisture grouping is a separate classification, and the abstract's sentence is explicitly about exposure to the external environment.
- Lower centrality means more connected. The paper never reads its values that way. The Results call the highest group in Fig 4A (IntMoist) "more connected" and the lowest group in Fig 4B (Occluded) "less connected". In the gut analysis, "reduced degree centrality" and "decreased closeness centrality" go with "less connected" networks. The authors' script (
bin/CompareSkin.R) computes the plotted value as igraph'scentr_eigen(...)$centralization, a graph-level eigenvector centralization score. I did not re-run that analysis. However that metric is best interpreted, the paper reads it one way in the Results and the Discussion, and the abstract states the opposite under that same reading. - The abstract summarises the supplementary dissimilarity analysis (Fig 4D and 4F). That analysis says only that "Occluded sites were significantly different from exposed and intermittently occluded sites". It compares the spread of network dissimilarities (ANOVA and Tukey test on group dispersion) and has no direction of connectedness. It cannot support "less connected".
- A conversion or typesetting artifact. Ruled out. The same abstract wording appears in the publisher's HTML and PDF, in the PMC copy and in the Europe PMC record.
- An error in reading the figure. It cannot change the conclusion. The direction comes from the paper's own Results and Discussion text. The figure reading only corroborates it. The exposed and occluded medians sit about 120 pixel rows apart on the 3800-pixel raster, a gap of 0.038 units against a reading precision of about 0.002.
- A later correction fixed it. None was found (see Status). Crossref has one update, a publisher "new version" entry dated 2018-04-30. It matches the switch from the uncorrected proof to the final version and does not change the abstract.
Status
As of 2026-10-23:
- Version of record. The abstract wording quoted above is in the publisher's HTML and PDF (24 pages, dated April 18, 2018), the PMC copy (PMC5927471) and the Europe PMC record (PMID 29668682). For this write-up, Fig 4 was read from the publisher PDF as held in the PMC open-data collection (
pmc-oa-opendata/PMC5927471.1/PMC5927471.1.pdf). - Supplements. S1–S14 Fig and S1–S2 Table are listed on the publisher page. S14 Fig only tests whether the Fig 4A and 4B p-values survive added edge noise, and gives no group values. The authors' public code repository, linked from the article, contains the analysis script (
bin/CompareSkin.R) and the p-value table used above (rtables/skinboxsig.tsv). It has no per-sample centrality values. - Corrections. No correction, erratum, expression of concern or retraction was found. Crossref has only the 2018-04-30 "new version" entry and a link to the bioRxiv preprint (10.1101/144642). PMC and Europe PMC list no notices. Europe PMC's comment/correction list holds only the preprint link.
- PubPeer. The PubPeer search API and site returned a human-verification challenge to automated requests, so a direct search could not be done. A web search for the article title on PubPeer found no PubPeer thread.
Suggested correction
In the Abstract, replace
Network structure differed between skin sites, with those exposed to the external environment being less connected and likely more susceptible to network degradation by microbial extinction events.
with
Network structure differed between skin sites, with those occluded from the external environment being less connected and likely more susceptible to network degradation by microbial extinction events.
or, stated from the exposed side,
Network structure differed between skin sites, with those exposed to the external environment being more connected and likely more resilient to network degradation by microbial extinction events.
Either wording matches the Results, Fig 4B and the Discussion. No other part of the paper needs to change.
Recomputation script (python recompute.py.txt)
"""Recompute the skin-site connectivity check for PMC5927471
(PLoS Comput Biol 2018, doi:10.1371/journal.pcbi.1006099).
Run: python recompute.py.txt
Needs numpy, matplotlib and Pillow; PyMuPDF (pymupdf / fitz) to read the figure out of
the published PDF. Prints every recomputed number used in writeup.md and writes
fig1.svg/png, fig2.svg/png and candidates/cand1..cand5.svg/png next to this file.
Inputs
* Text quoted from the article as archived in PubMed Central (Abstract; Results,
"Network structures across the human skin landscape"; Discussion; Fig 4 legend).
* Fig 4 as printed in the publisher's PDF (page 11, embedded 3800 x 3729 px raster).
The PDF is fetched from the public PMC open-data bucket, which holds the same
publisher PDF (24 pages):
https://pmc-oa-opendata.s3.amazonaws.com/PMC5927471.1/PMC5927471.1.pdf
Box-plot medians and hinges (quartiles) of panels 4A and 4B are read off that
raster by pixel position against the panel's own y-axis ticks.
* The authors' public code repository (linked from the article's data-availability
statement), table rtables/skinboxsig.tsv: pairwise Wilcoxon p-values from R's
pairwise.wilcox.test (default Holm adjustment). Its six values are transcribed
below as REPO_P.
If the PDF download fails, the script falls back to the values it read on 2026-10-23
(stored below as FALLBACK_FIG) and says so.
Reading convention used by the paper itself: a higher eigen-centrality value is
described as "more connected" (Results: IntMoist, the highest box in Fig 4A, is "more
connected" than Moist and Sebaceous; Occluded, the lowest box in Fig 4B, is "less
connected").
"""
import io
import os
import tempfile
import textwrap
import urllib.request
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
PDF_URL = "https://pmc-oa-opendata.s3.amazonaws.com/PMC5927471.1/PMC5927471.1.pdf"
# ------------------------------------------------------------------ text (verbatim)
Q_ABSTRACT = ("Network structure differed between skin sites, with those exposed to the "
"external environment being less connected and likely more susceptible to "
"network degradation by microbial extinction events.")
Q_RESULTS = ("Also, skin sites that were occluded from the environment were less connected "
"than those that were constantly exposed to the environment or only "
"intermittently occluded (p-value < 0.001, Fig 4B).")
Q_RESULTS_A = ("Intermittently moist skin sites (dynamic sites that fluctuate between being "
"moist and dry) were significantly more connected than the moist and "
"sebaceous environments (p-value < 0.001, Fig 4A).")
Q_DISCUSSION = ("The sites under greater environmental fluctuation and exposure (the exposed "
"and intermittently exposed sites) were more connected and therefore were "
"predicted to have a higher resilience against network degradation when "
"random nodes are removed from the network.")
# Fig 4 legend
N = {"Moist": 81, "Sebaceous": 56, "IntMoist": 56,
"Occluded": 106, "Exposed": 61, "IntOccluded": 26}
# ------------------------------------------------------------------ fallbacks (2026-10-23)
FALLBACK_FIG = { # group: (Q1, median, Q3) in eigen-centrality units
"IntMoist": (0.175, 0.187, 0.197), "Moist": (0.078, 0.137, 0.176),
"Sebaceous": (0.122, 0.145, 0.167), "Exposed": (0.147, 0.176, 0.196),
"IntOccluded": (0.172, 0.185, 0.194), "Occluded": (0.094, 0.138, 0.173)}
REPO_P = { # rtables/skinboxsig.tsv, transcribed 2026-10-23
("IntOccluded", "Exposed"): 0.277498657068735,
("Occluded", "Exposed"): 1.38010591458678e-05,
("Occluded", "IntOccluded"): 1.38010591458678e-05,
("Sebaceous", "Moist"): 0.132875880147664,
("Moist", "IntMoist"): 2.66084000742917e-08,
("Sebaceous", "IntMoist"): 4.21495886720462e-08}
# ------------------------------------------------------------------ figure digitising
def fetch(url, timeout=60):
req = urllib.request.Request(url, headers={"User-Agent": "recompute.py"})
with urllib.request.urlopen(req, timeout=timeout) as r:
return r.read()
def load_fig4_raster():
"""Return Fig 4 as a greyscale numpy array, taken from the publisher PDF."""
try:
import pymupdf as fitz
except ImportError:
import fitz
from PIL import Image
pdf = fetch(PDF_URL)
with tempfile.TemporaryDirectory() as td:
path = os.path.join(td, "paper.pdf")
with open(path, "wb") as f:
f.write(pdf)
doc = fitz.open(path)
print(f"Publisher PDF: {doc.page_count} pages")
for pno in range(doc.page_count):
for img in doc[pno].get_images():
if (img[2], img[3]) == (3800, 3729):
raw = doc.extract_image(img[0])["image"]
print(f"Fig 4 raster found on PDF page {pno + 1}: 3800 x 3729 px")
return np.asarray(Image.open(io.BytesIO(raw)).convert("L")).astype(int)
raise RuntimeError("Fig 4 raster not found in PDF")
def axis_ticks(a, x0, x1):
"""y-axis column of a panel and the row centres of its tick marks (top to bottom)."""
dark = a < 80
sub = dark[:1150, x0:x1]
ax = int(sub.sum(0).argmax())
rows = [y for y in range(1150) if sub[y, ax - 12:ax - 2].all()]
groups = []
for y in rows:
if groups and y - groups[-1][-1] <= 1:
groups[-1].append(y)
else:
groups.append([y])
return ax + x0, [float(np.mean(g)) for g in groups]
def dark_runs(a, col, ylo=150, yhi=1100, thr=80):
out, cur = [], []
for y in range(ylo, yhi):
if a[y, col] < thr:
cur.append(y)
elif cur:
out.append(cur)
cur = []
if cur:
out.append(cur)
return out
def digitise(a):
res = {}
panels = [("A", (0, 1900), ["IntMoist", "Moist", "Sebaceous"], [158, 323, 488]),
("B", (1900, 3800), ["Exposed", "IntOccluded", "Occluded"], [763, 928, 1093])]
for panel, (x0, x1), labels, centres in panels:
axcol, tk = axis_ticks(a, x0, x1)
vals = [0.3, 0.2, 0.1, 0.0][:len(tk)]
slope, icpt = np.polyfit(tk, vals, 1)
resid = np.max(np.abs(np.polyval([slope, icpt], tk) - vals))
print(f"Fig 4{panel}: y-axis at x={axcol}px, ticks at rows "
f"{[round(t, 1) for t in tk]} = {vals}; 1 px = {abs(slope):.5f}; "
f"max tick misfit {resid:.1e}")
for lab, cx in zip(labels, centres):
xc = int(cx * 3800 / 1200) - 60 # just off the whisker, inside the notch
runs = dark_runs(a, xc)
imed = max(range(len(runs)), key=lambda i: len(runs[i])) # median: thickest line
r_med = float(np.mean(runs[imed]))
r_q3 = float(np.mean(runs[imed - 1])) # upper hinge line
r_q1 = float(np.mean(runs[imed + 1])) # lower hinge line
q1, med, q3 = (float(slope * r + icpt) for r in (r_q1, r_med, r_q3))
res[lab] = (q1, med, q3)
print(f" {lab:12s} n={N[lab]:3d} rows Q3/med/Q1 = {r_q3:6.1f}/{r_med:6.1f}/"
f"{r_q1:6.1f} (median line {len(runs[imed])} px thick) -> "
f"Q1 {q1:.3f} median {med:.3f} Q3 {q3:.3f}")
return res
try:
FIG = digitise(load_fig4_raster())
FIG_SRC = "read from the published Fig 4 (publisher PDF, p. 11)"
except Exception as e: # network or library problem
print(f"!! could not read the figure ({e}); using values read on 2026-10-23")
FIG, FIG_SRC = FALLBACK_FIG, "values read on 2026-10-23 (fallback)"
for k, (q1, m, q3) in FIG.items():
print(f" {k:12s} Q1 {q1:.3f} median {m:.3f} Q3 {q3:.3f}")
P, P_SRC = REPO_P, "authors' repository table rtables/skinboxsig.tsv"
med = {k: v[1] for k, v in FIG.items()}
print()
print("Group sizes (Fig 4 legend):")
print(f" moisture {N['Moist']} + {N['Sebaceous']} + {N['IntMoist']} = "
f"{N['Moist'] + N['Sebaceous'] + N['IntMoist']}")
print(f" occlusion {N['Occluded']} + {N['Exposed']} + {N['IntOccluded']} = "
f"{N['Occluded'] + N['Exposed'] + N['IntOccluded']}")
print()
print(f"Pairwise Wilcoxon p-values, Holm-adjusted ({P_SRC}):")
for (g1, g2), v in P.items():
print(f" {g1:11s} vs {g2:11s} p = {v:.3g} < 0.001: {v < 1e-3}")
d_eo = med["Exposed"] - med["Occluded"]
d_io = med["IntOccluded"] - med["Occluded"]
print()
print("Median differences read from Fig 4B (eigen-centrality units):")
print(f" Exposed - Occluded = {med['Exposed']:.3f} - {med['Occluded']:.3f} = {d_eo:+.3f}")
print(f" IntOccluded - Occluded = {med['IntOccluded']:.3f} - {med['Occluded']:.3f} = {d_io:+.3f}")
print(f" IntOccluded - Exposed = {med['IntOccluded'] - med['Exposed']:+.3f}")
print(f" Exposed median lies above the Occluded upper hinge (Q3 {FIG['Occluded'][2]:.3f}): "
f"{med['Exposed'] > FIG['Occluded'][2]}")
print()
print("Direction checks (paper's convention: higher centrality = more connected):")
print(" Fig 4A: IntMoist highest (Results: 'more connected') ->",
med["IntMoist"] > max(med["Moist"], med["Sebaceous"]))
print(" Fig 4B: Occluded lowest (Results: 'less connected') ->",
med["Occluded"] < min(med["Exposed"], med["IntOccluded"]))
print(" Discussion 'exposed ... more connected' agrees with Fig 4B ->",
med["Exposed"] > med["Occluded"])
print(" Abstract 'exposed ... less connected' agrees with Fig 4B ->",
med["Exposed"] < med["Occluded"])
# ================================================================== figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
W = 7.2
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
def panel(ax, x, y, w, h, edge, title, body, n=46, body_size=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=edge, lw=2, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.035, title, transform=ax.transAxes, fontsize=11,
fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.115, body, transform=ax.transAxes, fontsize=body_size,
va="top", color=INK, linespacing=1.35)
def blank(h):
fig = plt.figure(figsize=(W, h))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
return fig, ax
SHORT_ABS = ("“…those exposed to the external environment being less connected "
"and likely more susceptible to network degradation by microbial extinction "
"events.”")
SHORT_RES = ("“…skin sites that were occluded from the environment were less "
"connected than those that were constantly exposed to the environment or only "
"intermittently occluded (p-value < 0.001, Fig 4B).”")
SHORT_DIS = ("“…(the exposed and intermittently exposed sites) were more connected "
"and therefore were predicted to have a higher resilience against network "
"degradation…”")
def legend_row(ax, y, items):
x = 0.03
for color, label in items:
ax.add_patch(FancyBboxPatch((x, y - 0.012), 0.03, 0.024, boxstyle="round,pad=0,"
"rounding_size=0.004", fc="white", ec=color, lw=2, transform=ax.transAxes))
ax.text(x + 0.04, y, label, transform=ax.transAxes, va="center", fontsize=10,
color=INK2)
x += 0.06 + 0.0105 * len(label)
# ---------- candidate 1: two text panels + connector + figure strip (used as fig1)
def cand1():
fig, ax = blank(5.0)
panel(ax, 0.01, 0.35, 0.39, 0.62, ORANGE, "Abstract", wrap(SHORT_ABS, 33))
panel(ax, 0.60, 0.35, 0.39, 0.62, ORANGE, "Results and Discussion",
wrap(SHORT_RES, 35) + "\n\n" + wrap(SHORT_DIS, 35), body_size=10)
ax.add_patch(FancyArrowPatch((0.405, 0.665), (0.595, 0.665), arrowstyle="<|-|>",
mutation_scale=14, lw=2, color=INK2, transform=ax.transAxes))
ax.text(0.5, 0.70, "Opposite\ndirection", transform=ax.transAxes, ha="center",
va="bottom", fontsize=10.5, color=INK, fontweight="bold")
ax.text(0.5, 0.63, "exposed sites:\n“less connected”\nvs\n“more connected”",
transform=ax.transAxes, ha="center", va="top", fontsize=10, color=INK2)
ax.add_patch(FancyBboxPatch((0.01, 0.10), 0.98, 0.20, boxstyle="round,pad=0,"
"rounding_size=0.012", fc="white", ec=BLUE, lw=2, transform=ax.transAxes))
ax.text(0.03, 0.245, "Fig 4B medians read from the published figure (eigen centrality;"
" higher = more connected)", transform=ax.transAxes,
fontsize=10, color=INK2, va="center")
ax.text(0.03, 0.155, f"Exposed {med['Exposed']:.3f} · IntOccluded "
f"{med['IntOccluded']:.3f} · Occluded {med['Occluded']:.3f}"
" \u2192 exposed above occluded",
transform=ax.transAxes, fontsize=10.5, color=INK, va="center", fontweight="bold")
legend_row(ax, 0.04, [(ORANGE, "Text as printed in the paper"),
(BLUE, "Values read from the paper's Fig 4B")])
return fig
# ---------- candidate 2: direction ledger
def cand2():
fig, ax = blank(3.6)
rows = [("Abstract", "Exposed sites less connected; more\nsusceptible to network degradation",
"<", ORANGE),
("Results (Fig 4B)", "Occluded less connected than exposed\nand int. occluded (p < 0.001)",
">", ORANGE),
("Discussion", "Exposed and intermittently exposed\nmore connected; higher resilience",
">", ORANGE),
("Fig 4B medians", f"Exposed {med['Exposed']:.3f} vs Occluded {med['Occluded']:.3f}\n"
f"(IntOccluded {med['IntOccluded']:.3f})", ">", BLUE)]
ax.text(0.03, 0.93, "Source", transform=ax.transAxes, fontweight="bold", fontsize=10.5)
ax.text(0.25, 0.93, "What it says", transform=ax.transAxes, fontweight="bold",
fontsize=10.5)
ax.text(0.71, 0.93, "Exposed vs occluded", transform=ax.transAxes, fontweight="bold",
fontsize=10.5)
for i, (src, txt, sign, col) in enumerate(rows):
y = 0.76 - i * 0.19
ax.plot([0.02, 0.98], [y + 0.09, y + 0.09], color=GRID, lw=1, transform=ax.transAxes)
ax.plot([0.015, 0.015], [y - 0.07, y + 0.07], color=col, lw=4, transform=ax.transAxes,
solid_capstyle="butt")
ax.text(0.03, y, src, transform=ax.transAxes, va="center", fontsize=10.5)
ax.text(0.25, y, txt, transform=ax.transAxes, va="center", fontsize=10, color=INK2)
word = "▼ LESS connected" if sign == "<" else "▲ MORE connected"
ax.text(0.71, y, word, transform=ax.transAxes, va="center", fontsize=10.5,
fontweight="bold", color=INK)
legend_row(ax, 0.03, [(ORANGE, "Stated in the paper's text"),
(BLUE, "Read from the paper's Fig 4B")])
return fig
# ---------- candidate 3: dot + IQR plot of medians (used as fig2)
def cand3():
order = ["IntMoist", "Moist", "Sebaceous", "Exposed", "IntOccluded", "Occluded"]
fig, ax = plt.subplots(figsize=(W, 4.0))
ys = [6.4, 5.4, 4.4, 2.6, 1.6, 0.6]
for g, y in zip(order, ys):
q1, m, q3 = FIG[g]
ax.plot([q1, q3], [y, y], color=BLUE, lw=2, solid_capstyle="round", zorder=2)
ax.plot(m, y, "o", ms=9, color=BLUE, mec="white", mew=1.5, zorder=3)
ax.text(q3 + 0.004, y, f"{m:.3f}", va="center", fontsize=10, color=INK)
ax.set_yticks(ys)
ax.set_yticklabels([f"{g} (n={N[g]})" for g in order])
ax.set_xlim(0, 0.32)
ax.set_xticks([0, 0.05, 0.10, 0.15, 0.20, 0.25])
ax.set_ylim(0, 7.3)
ax.set_xlabel("Eigen centrality: median (dot) and interquartile range (bar)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axhline(3.5, color=GRID, lw=1)
ax.text(0.003, 7.0, "Fig 4A, moisture", fontsize=10, color=INK2, fontweight="bold")
ax.text(0.003, 3.15, "Fig 4B, occlusion", fontsize=10, color=INK2, fontweight="bold")
ax.text(0.232, 5.4, "Results: IntMoist\n“more connected”", fontsize=10,
color=INK2, va="center")
ax.text(0.232, 2.15, "Abstract: exposed\nsites “less\nconnected”", fontsize=10,
color=ORANGE, fontweight="bold", va="center")
ax.text(0.232, 0.75, "Results, Discussion:\noccluded less,\nexposed more", fontsize=10,
color=INK2, va="center")
h1, = ax.plot([], [], "o-", color=BLUE, ms=9, lw=2)
fig.legend([h1], ["Values read from the paper's Fig 4 (publisher PDF)"],
loc="lower center", bbox_to_anchor=(0.5, -0.04), frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.05, 1, 1))
return fig
# ---------- candidate 4: two-item slope chart, ordering implied vs ordering shown
def cand4():
fig, ax = plt.subplots(figsize=(W, 3.6))
left = {"Occluded": 2, "Exposed": 1}
right = {"Exposed": 2 if med["Exposed"] > med["Occluded"] else 1,
"Occluded": 1 if med["Exposed"] > med["Occluded"] else 2}
for g in ("Exposed", "Occluded"):
ax.plot([0, 1], [left[g], right[g]], color=INK2, lw=2, zorder=1)
ax.plot(0, left[g], "o", ms=11, color=ORANGE, mec="white", mew=1.5, zorder=3)
ax.plot(1, right[g], "o", ms=11, color=BLUE, mec="white", mew=1.5, zorder=3)
ax.text(-0.05, left[g], g, ha="right", va="center", fontsize=11)
ax.text(1.05, right[g], f"{g} (median {med[g]:.3f})", ha="left", va="center",
fontsize=11)
ax.text(0, 2.45, "Abstract\n(order it implies)", ha="center", fontsize=10.5,
fontweight="bold", color=INK)
ax.text(1, 2.45, "Fig 4B\n(order of medians)", ha="center", fontsize=10.5,
fontweight="bold", color=INK)
ax.text(0.5, 1.5, "lines cross:\nthe order is reversed", ha="center", va="center",
fontsize=10, color=INK2, bbox=dict(fc=BG, ec="none", pad=2))
ax.text(-0.62, 2.0, "more\nconnected", ha="center", va="center", fontsize=10, color=INK2)
ax.text(-0.62, 1.0, "less\nconnected", ha="center", va="center", fontsize=10, color=INK2)
ax.set_xlim(-0.8, 1.75)
ax.set_ylim(0.6, 2.9)
ax.axis("off")
ax.plot([], [], "o", color=ORANGE, ms=10, label="Abstract's statement (ranks only)")
ax.plot([], [], "o", color=BLUE, ms=10, label="Read from the paper's Fig 4B")
ax.legend(loc="lower center", bbox_to_anchor=(0.5, -0.12), ncol=2, frameon=False,
fontsize=10)
return fig
# ---------- candidate 5: three stacked quotes with direction tags + compact value strip
def cand5():
fig = plt.figure(figsize=(W, 5.0))
axt = fig.add_axes([0.0, 0.115, 0.62, 0.885])
axt.set_xlim(0, 1)
axt.set_ylim(0, 1)
axt.axis("off")
items = [("Abstract", SHORT_ABS, "▼ exposed less connected"),
("Results (Fig 4B)", SHORT_RES, "▲ exposed more connected"),
("Discussion", SHORT_DIS, "▲ exposed more connected")]
for i, (t, q, tag) in enumerate(items):
y0 = 0.68 - i * 0.335
axt.add_patch(FancyBboxPatch((0.01, y0), 0.97, 0.31, boxstyle="round,pad=0,"
"rounding_size=0.015", fc="white", ec=ORANGE, lw=2,
transform=axt.transAxes))
axt.text(0.04, y0 + 0.28, f"{t}: {tag}", fontsize=10.5, fontweight="bold",
va="top", transform=axt.transAxes)
axt.text(0.04, y0 + 0.205, wrap(q, 54), fontsize=10, va="top", color=INK2,
transform=axt.transAxes, linespacing=1.3)
ax = fig.add_axes([0.76, 0.20, 0.23, 0.70])
order = ["Exposed", "IntOccluded", "Occluded"]
for i, g in enumerate(order):
q1, m, q3 = FIG[g]
ax.plot([i, i], [q1, q3], color=BLUE, lw=2)
ax.plot(i, m, "o", ms=9, color=BLUE, mec="white", mew=1.5)
ax.text(i + 0.13, m, f"{m:.3f}", va="center", fontsize=10)
ax.set_xticks(range(3))
ax.set_xticklabels(["Exp.", "Int.\nOcc.", "Occ."])
ax.set_xlim(-0.5, 2.9)
ax.set_ylim(0, 0.25)
ax.set_ylabel("Eigen centrality, Fig 4B")
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title("Median and IQR", fontsize=10.5, color=INK, loc="left")
fig.text(0.01, 0.0, "Orange frames: text as printed in the paper. Blue: values read "
"from the paper's Fig 4B.\nHigher eigen centrality = more connected, as the "
"paper uses it.", fontsize=10, color=INK2, va="bottom")
return fig
save(cand1(), os.path.join(HERE, "fig1"), os.path.join(CAND, "cand1_text_panels"))
save(cand2(), os.path.join(CAND, "cand2_direction_ledger"))
save(cand3(), os.path.join(HERE, "fig2"), os.path.join(CAND, "cand3_median_dotplot"))
save(cand4(), os.path.join(CAND, "cand4_rank_slope"))
save(cand5(), os.path.join(CAND, "cand5_quotes_plus_values"))
print()
print("Wrote fig1.svg/png, fig2.svg/png and candidates/cand1..cand5 (.svg, .png)")
Abstract says LabMol-149 has EC50 below 500 nM; Table 3 gives 1450 and 509 nM
PLoS Comput Biol (2020) · doi:10.1371/journal.pcbi.1007025 · PMC7048302
Summary
The Abstract reports that two of the five compounds chosen by the paper's deep-learning screen, LabMol-149 and LabMol-152, showed antiplasmodial activity "at low nanomolar concentrations (EC50 <500 nM)". The final paragraph of the Discussion says both compounds "completely satisfy" a criterion of EC50 <500 nM against sensitive and resistant strains. The paper's own Table 3 gives LabMol-149 EC50 values of 1.450 ± 0.524 μM against the sensitive 3D7 strain and 0.509 ± 0.177 μM against the resistant W2 strain. These are 1450 nM and 509 nM, so the compound is above the stated threshold on both strains, by a factor of 2.9 on 3D7. Only LabMol-152 (49 and 78 nM) meets the criterion. The headline count of qualifying compounds is therefore one, not two.
What the paper reports
Abstract:
Among them, two compounds, LabMol-149 and LabMol-152, showed potent antiplasmodial activity at low nanomolar concentrations (EC50 <500 nM) and low cytotoxicity in mammalian cells. Therefore, the computational approach employing deep learning developed here allowed us to discover two new families of potential next generation antimalarial agents, which are in compliance with the guidelines and criteria for antimalarial target candidates.
Author summary:
This allowed us to discover two new potential antimalarial candidates.
Discussion, final paragraph:
Moreover, two of these compounds (LabMol-149 and LabMol-152) completely satisfy the guidelines and criteria for discovery of new antimalarial drugs, i.e., activity at low nanomolar concentrations (EC50 <500nM) against sensitive and multiple resistant strains of Plasmodium spp. and SI greater than 10 folds [12,42].
Table 3, header and LabMol-149 row (columns: compound | EC50 (Pf3D7)(μM) | EC50 (PfW2)(μM) | CC50 (COS-7) (μM) | SIa | SIb):
LabMol-149 | 1.450 ± 0.524 | 0.509 ± 0.177 | >100 | >71.4 | >196
Table 3, the other two active candidate compounds:
LabMol-151 | 1.911 ± 0.292 | 1.616 ± 0.321 | 140.8 ± 11.2 | 73.7 | 87.1
LabMol-152 | 0.049 ± 0.029 | 0.078 ± 0.014 | 16.7 ± 10.3 | 340.8 | 214.1
Table 3 footer:
SIa: Selectivity index calculated by COS7 CC50/3D7 EC50; and SIb: Selectivity index calculated by COS7 CC50/W2 EC50; The data are expressed as mean ± SD of three independent assays.
Two related passages use different wording. Results, "Experimental validation":
The EC50 for each compound (Table 3) indicate that two compounds, 2-(4,6-diphenyl-1,2-dihydro-1,3,5-triazin-2-yl)phenol (LabMol-149), 4-{N-[3-(morpholin-4-yl)-1,4-dioxo-1,4-dihydronaphthalen-2-yl]acetamido}benzoic acid (LabMol-151) and N2-(3-fluorophenyl)-N4-[(oxolan-2-yl)methyl]quinazoline-2,4-diamine (LabMol-152), were potent at inhibiting the parasite growth, showing activities in submicromolar and low nanomolar range against both 3D7 and W2 strains.
Discussion, third paragraph:
Two compounds (LabMol-149 and LabMol-151) showed activity at submicromolar concentrations against asexual blood stages and low cytotoxicity in mammalian cells.
What the paper's own numbers show
Method. I converted every Table 3 EC50 from μM (the unit in the column header) to nM and compared each one with the 500 nM threshold stated in the Abstract and the Discussion. The Discussion applies that threshold to both sensitive and resistant strains, so a compound qualifies only if both its 3D7 and W2 values are below 500 nM. For LabMol-149 I also looked at mean − 1 SD, using the footer's statement that the values are mean ± SD of three assays. I recomputed every selectivity index from the footer's definitions (CC50/EC50), and checked the "submicromolar" (below 1 μM) wording against the same values. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| LabMol-149 EC50, 3D7 | Table 3; Abstract and Discussion say <500 nM | 1.450 ± 0.524 μM | 1450 nM, 2.90 × the 500 nM threshold; mean − 1 SD = 926 nM |
| LabMol-149 EC50, W2 | Table 3; Abstract and Discussion say <500 nM | 0.509 ± 0.177 μM | 509 nM, 1.018 × the threshold (9 nM above it) |
| LabMol-152 EC50, 3D7 / W2 | Table 3 | 0.049 / 0.078 μM | 49 / 78 nM, both below 500 nM |
| LabMol-151 EC50, 3D7 / W2 | Table 3 | 1.911 / 1.616 μM | 1911 / 1616 nM, both above 500 nM |
| Tested LabMol compounds with EC50 <500 nM on both strains | Abstract; Discussion, final paragraph | two (LabMol-149 and LabMol-152) | one (LabMol-152) |
| LabMol-149 SIa | Table 3 | >71.4 | 100 / 1.450 = >69.0; 71.4 corresponds to an EC50 of 1.401 μM |
| LabMol-149 SIb | Table 3 | >196 | 100 / 0.509 = >196.5 |
| LabMol-151 SIa / SIb | Table 3 | 73.7 / 87.1 | 73.7 / 87.1 |
| LabMol-152 SIa / SIb | Table 3 | 340.8 / 214.1 | 340.8 / 214.1 |
| Chloroquine SIa / SIb | Table 3 | 4,545 / 289 | >4545.5 / >289.0 |
| Pyrimethamine SIa / SIb | Table 3 | 2,702 / 5.5 | >2702.7 / >5.5 |
| LabMol-149 "submicromolar" | Discussion, third paragraph | submicromolar | 3D7 1450 nM, not below 1 μM; W2 509 nM, below 1 μM |
| LabMol-151 "submicromolar" | Discussion, third paragraph | submicromolar | 1911 / 1616 nM, below 1 μM on neither strain |
The table's arithmetic is internally consistent. Every selectivity index recomputes from the printed EC50 and CC50 values, except LabMol-149's SIa, which is slightly high (71.4 against 69.0). The problem is the text. It counts LabMol-149 among the compounds below 500 nM, and the table does not support that for either strain. The Results passage also says "two compounds" and then names three.
Figure
Figure 1. On the left (orange), the two passages that state the EC50 <500 nM criterion and name LabMol-149 and LabMol-152 as meeting it, quoted from the Abstract and the final paragraph of the Discussion. On the right (blue), the Table 3 values for the same two compounds, converted from μM to nM. LabMol-149 is 1450 nM against 3D7 (2.90 times the threshold) and 509 nM against W2 (1.02 times). LabMol-152 is 49 and 78 nM. Only LabMol-152 meets the criterion as the text states it.
Figure 2. Table 3 EC50 values (blue dots, mean; blue lines, ± 1 SD of three assays) for the three active candidate compounds on one log-scaled nM axis. The solid vertical line marks 500 nM and the dashed line 1 μM. The orange markers show what the text says about each compound: below 500 nM for LabMol-149 and LabMol-152 (Abstract and final paragraph of the Discussion), and "submicromolar" for LabMol-151 (Discussion, third paragraph; that sentence also calls LabMol-149 submicromolar). LabMol-149's 3D7 interval lies entirely above 500 nM, and both LabMol-151 values lie above 1 μM. The W2 value for LabMol-149 sits at the threshold, and its ± 1 SD interval spans it.
Why it matters
The count of compounds meeting the potency criterion is the paper's headline experimental result. It appears in the Abstract ("two compounds ... (EC50 <500 nM)"), in the Author summary ("two new potential antimalarial candidates") and in the final paragraph of the Discussion, which says the two compounds "completely satisfy" the criteria for new antimalarial drugs. The Abstract also concludes that the approach found "two new families of potential next generation antimalarial agents, which are in compliance with the guidelines and criteria".
By the paper's own Table 3, one compound (LabMol-152, a quinazoline) meets the stated criterion. LabMol-149 fails it on the sensitive strain by a wide margin. Its 3D7 EC50 of 1450 nM is also above the 1 μM cut-off that the paper itself uses to label training compounds "active" (Methods, "Datasets": "1,058 active compounds with EC50 ≤ 1 μM"). On the resistant strain it is essentially at the threshold, at 509 nM. LabMol-149 remains an active and selective hit (SI above about 69 on 3D7 and 196 on W2), and nothing here questions the measurements themselves. But it is a low-micromolar hit, not a compound that satisfies the sub-500 nM criterion.
Readers who rely on the Abstract, for example to compile hit lists or to judge how well deep-learning virtual screening performs in antimalarial discovery, would count two qualifying compounds from two scaffold families where the data show one. The rest of the paper's conclusions, such as the QSAR models, the screen and LabMol-152's potency, do not depend on this. These are in vitro preclinical results, so there is no direct clinical consequence.
Explanations considered
- A μM/nM unit mix-up. This does not account for the numbers. The Table 3 header gives μM in the PMC text, the publisher's XML, its PDF and Europe PMC. Read in μM, LabMol-152's values (0.049 and 0.078) agree with the Results text ("EC50 = 0.049 μM and 0.078 μM"), and the reference drugs' values (chloroquine 0.011 μM, pyrimethamine 0.037 μM) are plausible in that unit. Read as nM, they would not be.
- A decimal slip in the 3D7 value (0.145 μM printed as 1.450 μM). This does not account for the numbers. With CC50 >100 μM, an EC50 of 0.145 μM would give SIa >689.7. The printed SIa (>71.4) corresponds to about 1.40 μM, so the table's own selectivity column supports a micromolar EC50. The W2 value (0.509 μM) also agrees with its SIb (100/196 = 0.510 μM).
- Assay variation. This does not account for the 3D7 value. The footer gives mean ± SD of three assays, and the 3D7 mean − 1 SD is still 926 nM, nearly twice the threshold. For W2 the mean is 9 nM above 500 nM, and its ± 1 SD interval (332–686 nM) spans the threshold, so W2 alone could be called borderline. The stated criterion requires activity on both strains, however, and 3D7 is not borderline.
- The claim refers to only one strain. This does not rescue it. The Discussion applies the criterion "against sensitive and multiple resistant strains". Even the more potent of LabMol-149's two values (509 nM, W2) is above 500 nM.
- A different, looser criterion was intended. The Results passage uses vaguer wording ("submicromolar and low nanomolar range"). The Abstract and the Discussion, however, state the 500 nM cut-off explicitly, citing references 12 and 42, and attribute it to LabMol-149. A looser 1 μM cut-off would still not be met on 3D7 (1450 nM).
- The compound names were swapped in the text. Substituting LabMol-151 for LabMol-149 does not help: LabMol-151 is 1911 and 1616 nM. LabMol-152 is the only tested compound below 500 nM on both strains.
- A conversion artifact in the PMC copy. This is excluded. The publisher's article page (Version 2, the final version), its JATS XML and its printable PDF carry the same text and the same Table 3 values as the PMC archive.
- The EC50 lies outside the assay range. This is not the case. The antiplasmodial assay used a 12-point series from 0.019 to 40 μM (Methods, "Antiplasmodial assay"), and 1.450 μM lies within it.
Status
- Version of record. The publisher's article page (journals.plos.org, reached from the DOI; shown as Version 2, with Version 1 as the uncorrected proof), its JATS XML and its printable PDF (21 pages, dated February 18, 2020) show the same Abstract sentence, the same Discussion sentences and the same Table 3 rows, with μM in the column header, as the PMC archive. The Europe PMC full-text XML matches them as well. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements. The article lists seven supporting files: S1–S4 File (XLSX QSAR training datasets) and S1–S3 Fig (TIF images). S1 File (1163 rows, SMILES and Outcome) and S3 File (1247 rows, SMILES and pEC50) were opened; both contain only literature training data, with no LabMol entries. None of the supporting files holds assay-level dose-response data for the tested compounds, so none of them can change the Table 3 EC50 values.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. PMC and Europe PMC list no notices. Crossref's only update record is the article's own DOI with type "new_version", dated 2020-02-28, which is PLOS's routine record for replacing the uncorrected proof with the final version; a Crossref relation search returned only the article's own components. The PLOS comments page shows "Reader Comments (0)". The PubPeer API returned no feedback for the DOI (the PubPeer web page itself refused automated access), and a web search found no discussion of the issue.
- Sources. PMC article PMC7048302; publisher page https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007025 and its comments page; PLOS JATS XML and printable PDF; Europe PMC full text; Crossref; PubPeer.
Suggested correction
- In the Abstract, change "two compounds, LabMol-149 and LabMol-152, showed potent antiplasmodial activity at low nanomolar concentrations (EC50 <500 nM)" so that it names LabMol-152 alone as meeting EC50 <500 nM. Describe LabMol-149 as active in the low-micromolar to high-nanomolar range (1.450 μM on 3D7, 0.509 μM on W2). Revise "two new families ... in compliance with the guidelines and criteria" and the Author summary's "two new potential antimalarial candidates" to match.
- In the final paragraph of the Discussion, restrict "completely satisfy the guidelines and criteria ... (EC50 <500nM)" to LabMol-152.
- In the third paragraph of the Discussion, replace "submicromolar" for LabMol-149 and LabMol-151, which are 1.450 / 0.509 μM and 1.911 / 1.616 μM, with "low micromolar" or the actual values.
- Minor: in the Results, "two compounds" is followed by three names; change it to "three compounds". Change LabMol-149's SIa from >71.4 to >69.0 (100/1.450), or state which EC50 value was used to compute it.
Recomputation script (python recompute.py.txt)
"""Recompute the potency claims of PMC7048302
(PLoS Comput Biol 2020, doi:10.1371/journal.pcbi.1007025) from the paper's own Table 3.
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from Table 3 of the paper as archived in PMC (identical in the
publisher's version of record). Table 3 header: EC50 and CC50 in uM; footer: "The data
are expressed as mean +/- SD of three independent assays"; SIa = CC50/3D7 EC50,
SIb = CC50/W2 EC50.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Table 3, uM)
# cc_censored: CC50 printed as ">x", so the SI is a lower bound
T3 = {
"LabMol-149": dict(e3=1.450, sd3=0.524, ew=0.509, sdw=0.177, cc=100.0, cc_censored=True, SIa=71.4, SIb=196.0),
"LabMol-151": dict(e3=1.911, sd3=0.292, ew=1.616, sdw=0.321, cc=140.8, cc_censored=False, SIa=73.7, SIb=87.1),
"LabMol-152": dict(e3=0.049, sd3=0.029, ew=0.078, sdw=0.014, cc=16.7, cc_censored=False, SIa=340.8, SIb=214.1),
"Chloroquine": dict(e3=0.011, sd3=0.001, ew=0.173, sdw=0.020, cc=50.0, cc_censored=True, SIa=4545, SIb=289),
"Pyrimethamine": dict(e3=0.037, sd3=0.007, ew=18.240, sdw=4.537, cc=100.0, cc_censored=True, SIa=2702, SIb=5.5),
}
# LabMol-148 and LabMol-150: EC50 > 40 uM on both strains (inactive at the highest concentration)
THRESH_NM = 500.0 # Abstract and Discussion (final paragraph): "EC50 <500 nM"
SUBMICRO_NM = 1000.0 # "submicromolar" = below 1 uM
MODEL_ACTIVE_UM = 1.0 # Methods, Datasets: training compounds "active" if EC50 <= 1 uM
ASSAY_RANGE_UM = (0.019, 40.0) # Methods, Antiplasmodial assay: 12-point dilution series
# What the text says about each compound
CLAIM_LT500 = {"LabMol-149", "LabMol-152"} # Abstract; Discussion final paragraph
CLAIM_SUBMICRO = {"LabMol-149", "LabMol-151"} # Discussion, third paragraph
def nm(um):
return um * 1000.0
print("=== 1. Table 3 EC50 values in nM, against the stated <500 nM criterion ===")
for k, v in T3.items():
n3, nw = nm(v["e3"]), nm(v["ew"])
print(f"{k:14s} 3D7 {n3:7.0f} nM ({n3 / THRESH_NM:6.3f} x 500) W2 {nw:7.0f} nM ({nw / THRESH_NM:6.3f} x 500) "
f"<500 nM on both strains: {n3 < THRESH_NM and nw < THRESH_NM}")
print("\n=== 2. LabMol-149 in detail ===")
v = T3["LabMol-149"]
for strain, m, s in (("3D7", v["e3"], v["sd3"]), ("W2", v["ew"], v["sdw"])):
print(f"{strain}: mean {nm(m):.0f} nM, SD {nm(s):.0f} nM; mean - 1 SD = {nm(m - s):.0f} nM, "
f"mean + 1 SD = {nm(m + s):.0f} nM; mean exceeds 500 nM by {nm(m) - THRESH_NM:.0f} nM")
print(f"3D7 mean - 1 SD above 500 nM: {nm(v['e3'] - v['sd3']) > THRESH_NM}; "
f"above 1 uM (the paper's own model 'active' cut-off, EC50 <= {MODEL_ACTIVE_UM} uM): {v['e3'] > MODEL_ACTIVE_UM}")
print(f"3D7 EC50 {v['e3']} uM lies inside the tested range {ASSAY_RANGE_UM[0]}-{ASSAY_RANGE_UM[1]} uM: "
f"{ASSAY_RANGE_UM[0] < v['e3'] < ASSAY_RANGE_UM[1]}")
print("\n=== 3. Selectivity indices recomputed (CC50 / EC50) ===")
for k, val in T3.items():
sia, sib = val["cc"] / val["e3"], val["cc"] / val["ew"]
pre = ">" if val["cc_censored"] else ""
print(f"{k:14s} SIa reported {pre}{val['SIa']:<7} recomputed {pre}{sia:.1f} "
f"SIb reported {pre}{val['SIb']:<6} recomputed {pre}{sib:.1f}")
print(f"LabMol-149: 3D7 EC50 implied by SIa = 71.4 with CC50 = 100 uM: {100 / 71.4:.3f} uM")
print(f"LabMol-149: SIa if the 3D7 EC50 were a decimal slip for 0.145 uM: {100 / 0.145:.1f}")
print(f"LabMol-149: W2 EC50 implied by SIb = 196 with CC50 = 100 uM: {100 / 196:.3f} uM (Table 3: 0.509 uM)")
print("\n=== 4. Text descriptions checked against Table 3 ===")
for k in ("LabMol-149", "LabMol-151", "LabMol-152"):
val = T3[k]
n3, nw = nm(val["e3"]), nm(val["ew"])
print(f"{k}: called '<500 nM' in Abstract/Discussion: {k in CLAIM_LT500}; actually <500 nM on both: "
f"{n3 < THRESH_NM and nw < THRESH_NM} | called 'submicromolar' in Discussion: {k in CLAIM_SUBMICRO}; "
f"actually <1 uM on 3D7: {n3 < SUBMICRO_NM}, on W2: {nw < SUBMICRO_NM}")
n_meet = sum(nm(T3[k]["e3"]) < THRESH_NM and nm(T3[k]["ew"]) < THRESH_NM for k in ("LabMol-149", "LabMol-151", "LabMol-152"))
print(f"Tested LabMol compounds meeting EC50 <500 nM on both strains: {n_meet} (text claims 2)")
print("Results text: 'two compounds' followed by three names (LabMol-149, LabMol-151, LabMol-152).")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: text-vs-table diagram
v = T3["LabMol-149"]
w = T3["LabMol-152"]
fig = plt.figure(figsize=(7.2, 4.8))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.8)
ax.axis("off")
def panel(x, y, wd, ht, edge):
ax.add_patch(FancyBboxPatch((x, y), wd, ht, boxstyle="round,pad=0,rounding_size=0.08",
fc="#ffffff", ec=edge, lw=1.5))
# left panel: what the text says (reported, orange)
panel(0.15, 0.95, 3.15, 3.7, ORANGE)
ax.text(0.30, 4.50, "Abstract and Discussion", fontsize=10.5, fontweight="bold", va="top")
ax.text(0.30, 4.20, "Reported in the text", fontsize=10, color=SEC, va="top")
ax.add_line(Line2D([0.30, 0.30], [3.84, 2.62], color=ORANGE, lw=2))
ax.text(0.42, 3.84, "Abstract:\n\u201ctwo compounds, LabMol-149 and\nLabMol-152, showed potent\n"
"antiplasmodial activity at low\nnanomolar concentrations\n(EC50 <500 nM)\u201d",
fontsize=10, va="top", linespacing=1.25)
ax.add_line(Line2D([0.30, 0.30], [2.30, 1.08], color=ORANGE, lw=2))
ax.text(0.42, 2.30, "Discussion, final paragraph:\n\u201c(LabMol-149 and LabMol-152)\ncompletely satisfy [\u2026] (EC50\n"
"<500nM) against sensitive and\nmultiple resistant strains\u201d", fontsize=10, va="top", linespacing=1.25)
# right panel: Table 3 converted (recomputed, blue)
panel(3.95, 0.95, 3.1, 3.7, BLUE)
ax.text(4.10, 4.50, "Table 3 (\u03bcM), converted to nM", fontsize=10.5, fontweight="bold", va="top")
ax.text(4.10, 4.20, "Recomputed from the paper\u2019s table", fontsize=10, color=SEC, va="top")
rows = [
("LabMol-149", "3D7", v["e3"], v["sd3"]),
("LabMol-149", "W2", v["ew"], v["sdw"]),
("LabMol-152", "3D7", w["e3"], w["sd3"]),
("LabMol-152", "W2", w["ew"], w["sdw"]),
]
for (cpd, strain, m, s), yy in zip(rows, (3.74, 3.08, 2.22, 1.56)):
n = nm(m)
ok = n < THRESH_NM
ax.text(4.10, yy, f"{cpd}, {strain}: {m:.3f} \u00b1 {s:.3f} \u03bcM", fontsize=10, va="center",
fontweight="normal" if ok else "bold")
ax.text(4.10, yy - 0.25, f"= {n:.0f} nM", fontsize=10, va="center", color=BLUE, fontweight="bold")
verdict = f"{n / THRESH_NM:.2f} \u00d7 500 nM: fails" if not ok else "below 500 nM: meets"
ax.text(5.15, yy - 0.25, verdict, fontsize=10, va="center", color=TXT if not ok else SEC)
ax.add_line(Line2D([4.10, 6.90], [2.52, 2.52], color=GRID, lw=1))
# connector
ax.add_patch(FancyArrowPatch((3.33, 3.25), (3.92, 3.25), arrowstyle="<|-|>", mutation_scale=14, color=SEC, lw=2))
ax.text(3.625, 3.40, "conflict", fontsize=10, ha="center", va="bottom", color=SEC, fontweight="bold")
ax.text(3.6, 0.48, f"LabMol-149 is {nm(v['e3']):.0f} nM (3D7) and {nm(v['ew']):.0f} nM (W2): above 500 nM on both strains.\n"
f"Only LabMol-152 ({nm(w['e3']):.0f} and {nm(w['ew']):.0f} nM) meets the criterion the text states.",
fontsize=10, ha="center", va="center", linespacing=1.3)
save(fig, "fig1")
# Figure 2: EC50 mean +/- SD (nM, log scale) against the text's thresholds
cpds = ["LabMol-149", "LabMol-151", "LabMol-152"]
claims = {"LabMol-149": "Abstract/Discussion: <500 nM", "LabMol-151": "Discussion: “submicromolar”",
"LabMol-152": "Abstract/Discussion: <500 nM"}
fig, ax = plt.subplots(figsize=(7.2, 4.0))
fig.subplots_adjust(left=0.20, right=0.97, top=0.84, bottom=0.15)
ypos, ylabels = [], []
y = 0
for c in cpds:
val = T3[c]
limit = THRESH_NM if c in CLAIM_LT500 else SUBMICRO_NM
# claim: orange bracket from the claimed upper limit leftwards
ax.annotate("", xy=(limit * 0.55, y + 0.5), xytext=(limit, y + 0.5),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=12))
ax.plot([limit, limit], [y + 0.32, y + 0.68], color=ORANGE, lw=2)
ax.text(limit * 1.06, y + 0.5, claims[c], color=TXT, fontsize=10, va="center", ha="left")
for j, (strain, m, s) in enumerate((("3D7", val["e3"], val["sd3"]), ("W2", val["ew"], val["sdw"]))):
yy = y - 0.18 - 0.42 * j
ax.plot([nm(m - s), nm(m + s)], [yy, yy], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(nm(m), yy, "o", color=BLUE, ms=8, mec=BG, mew=1.5, zorder=3)
ax.text(nm(m + s) * 1.08, yy, f"{strain}: {nm(m):.0f} nM", fontsize=10, va="center", color=TXT)
ypos.append(yy)
ylabels.append(f"{c} {strain}")
y -= 1.75
ax.axvline(THRESH_NM, color=SEC, lw=0.9, zorder=0)
ax.axvline(SUBMICRO_NM, color=SEC, lw=0.9, ls=(0, (3, 3)), zorder=0)
ax.text(THRESH_NM * 0.96, 1.12, "500 nM", ha="right", va="bottom", fontsize=10, color=SEC)
ax.text(SUBMICRO_NM * 1.04, 1.12, "1 μM", ha="left", va="bottom", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(10, 20000)
ax.set_ylim(y + 0.85, 1.05)
ax.set_yticks(ypos)
ax.set_yticklabels(ylabels, fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("EC50 (nM, log scale)")
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda x, _: f"{x:,.0f}"))
handles = [Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, mec=BG,
label="Table 3 mean ± SD (n = 3), converted to nM"),
Line2D([], [], color=ORANGE, lw=2, marker=4, ms=8,
label="Text\u2019s claim: EC50 below this value")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract names March and October as best twice-yearly spraying months; body, table and data give September–November
PLoS Comput Biol (2020) · doi:10.1371/journal.pcbi.1007743 · PMC7200023
Summary
The abstract says the best ultra-low volume (ULV) strategy was spraying twice a year "in March and October", with a median of 172,000 infections [IQR 158,000–183,000], a 52% reduction from no spraying. The Results text, Table 2, the Discussion and the legends of Figs 4, 8 and 10 give these same figures, and the same 549 spray days, for a different month pair: September and November. One Results sentence repeats the abstract's "October and March". The authors' public source data reproduce the reported 172,000 [158,000–183,000] for September–November, which ranks 1st of the 66 month pairs tested. March–October has a median of 219,402 infections, ranks 16th, and gives a 39% reduction, not 52%. The paper's stated question is when to spray, so the abstract gives the wrong answer to it, although every number it reports is correct.
What the paper reports
The abstract's ULV headline:
"The ULV strategy with the fewest median infections was spraying twice yearly, in March and October, which led to a median of 172,000 infections [IQR: 158,000–183,000], a 52% reduction from baseline." (Abstract)
The abstract also states the paper's question:
"The optimal time to deploy these interventions and their relative epidemiological effects are, however, not well understood." (Abstract)
The Results give the same numbers to a different pair, and describe the general timing pattern:
"The best strategy for spraying twice yearly was spraying in September and November (172,000; IQR: 158,000–183,000) (Fig 4C). Generally, undertaking the first spray in August or September (typically, just before the dengue season) and the second in October or November (typically, near the start of the dengue season) led to the fewest infections (Fig 4C). In this case, the strategy that led to the fewest infections also led to the lowest average mosquito abundance: spraying in September and November (S4 Fig)." (Results, "Ultra-low volume spraying", paragraph 1)
The next paragraph uses the abstract's pair:
"Comparing the best strategies, spraying twice yearly (in October and March) averted the most infections, but required the most spraying campaigns: 22 campaigns of spraying, spending a median of 549 days spraying in total over 11 years." (Results, "Ultra-low volume spraying", paragraph 2)
Table 2 (ULV rows, columns: None | Adaptive threshold | Yearly | Twice yearly):
"Best strategy | N/A | When monthly incidence is 1σ above mean | September | September & November" "Number of infections, 2000–2010 (1,000s) [IQR] | 361 [347, 383] | 254 [210, 277] | 261 [250, 277] | 172 [158, 183]" "Days spent spraying [IQR] | 0 | 351 [341, 371] | 274 [272, 277] | 549 [545, 552]" (Table 2)
The Discussion and three figure legends agree with Table 2:
"The best strategy for ULV spraying was to spray twice per year, in September and November." (Discussion, "Summary of main findings")
"twice yearly corresponds to spraying in September and November." (Fig 4 legend, panel d; Fig 8 legend, panel c has "and twice yearly corresponds to spraying in September and November;")
"the bottom row the best twice yearly strategy (spraying in September and November)" (Fig 10 legend)
The Methods show that the two pairs are separate strategies, each with its own result:
"We tested yearly spraying starting in each month (12 strategies) and twice-yearly spraying in each pair of months (66 strategies) (Table 1)." (Methods, "Experiments")
"Comparison of the median predicted infections for twice yearly spraying strategies, beginning on the first days of the shown month." (Fig 4 legend, panel c)
What the paper's own numbers show
Method. I first listed every statement in the paper that names the best twice-yearly ULV pair, and checked each against the PMC archive. Seven statements name September and November (Results paragraph 1, twice; Table 2; the Discussion; the Fig 4, 8 and 10 legends). Two name March and October (the Abstract and Results paragraph 2). I then checked the arithmetic of the reported values. Finally, I used the source data from the GitHub repository given in the paper's Data Availability Statement. The file figures/scripts/spray_analysis_paired_all_years.RData holds 400 simulations × 3 outputs × 66 month pairs. I mapped pairs to columns with the pair-index formula from the authors' own Fig 4 script, and took infections from output column 3, the column that script uses for Fig 4. For each pair I computed the median, the IQR (R's default quantile, which numpy also uses) and the median number of spray days. These data reproduce the paper's no-spraying, yearly-September and best-threshold results, so they match the published analysis. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Month pair with the fewest median infections (twice yearly, ULV) | Abstract; Results para 2 | March and October | September–November (rank 1 of 66) |
| Same | Results para 1; Table 2; Discussion; Fig 4d, 8c, 10 legends | September and November | September–November (rank 1 of 66) |
| Median infections [IQR], September–November | Results para 1; Table 2 | 172,000 [158,000–183,000] | 171,650 [158,131–183,442] (consistent) |
| Median infections [IQR], March–October | Abstract (as the best pair) | 172,000 [158,000–183,000] | 219,402 [208,280–230,152]; rank 16 of 66 |
| Reduction from no spraying, best twice-yearly pair | Abstract | 52% | 1 − 172/361 = 52.4%; from source data, Sep–Nov 52.4%, Mar–Oct 39.2% |
| No spraying, median [IQR] | Abstract; Table 2 | 361,000 [347,000–383,000] | 360,831 [346,748–383,264] (consistent) |
| Yearly September, median [IQR]; days | Results; Table 2 | 261,000 [250,000–277,000]; 274 | 260,708 [249,589–276,903]; 274 (consistent) |
| Spray days, twice yearly | Results para 2; Table 2 | 549 | 22 campaigns × 25 days = 550; median 549 for both Sep–Nov and Mar–Oct (median across all 66 pairs is 548–549, so this figure does not identify a pair) |
| Best ULV vs best TIRS | Results, TIRS | 17-fold | 172,000 / 9,900 = 17.4 (consistent) |
| Ranks of the four pairs with first spray Aug/Sep and second Oct/Nov | Results para 1 ("Generally, ...") | fewest infections | ranks 1–4 of 66 (consistent with the body) |
Two further checks need the RData file. Mar–Oct is not the best pair on the file's other outcome column either (rank 14 of 66, best is Sep–Nov), and not on mean infections (rank 15 of 66, best is Sep–Nov).
A secondary inconsistency, unrelated to the month pair. For the best TIRS strategy, the abstract gives "(9,900; [IQR: 8,720–11,400], a 94% reduction)". The Results give "9,900 infections (IQR: 8,720–11,300)", Table 2 gives "9.90 [8.72, 11.3]", and 1 − 9.9/361 = 97.3%. The Discussion's "97% reduction ... for repeated TIRS campaigns" is consistent with 97.3% (yearly TIRS gives 1 − 11.8/361 = 96.7%, also about 97%). The abstract's 94% and 11,400 do not match the body.
Figure
Figure 1. The two orange-bordered panels quote the paper. On the left, the abstract (and Results paragraph 2) name March and October as the best twice-yearly ULV pair. On the right, the Results, Table 2 and the Discussion (and the Fig 4d, 8c and 10 legends) name September and November. Both give the same 172,000 [158,000–183,000] infections and 549 spray days. The blue strip below gives values recomputed in recompute.py.txt from the authors' source data. September–November has median 171,650 [158,131–183,442], rank 1 of 66, a 52% reduction, and matches the reported result. March–October has median 219,402 [208,280–230,152], rank 16 of 66, a 39% reduction, and does not match.
Figure 2. Median and IQR of total infections over 2000–2010 (400 simulations each) for the 20 best of the 66 twice-yearly ULV month pairs, recomputed from the authors' source data. The top row, in orange, is the paper's reported best result, 172 [158, 183] thousand. The two pairs the paper names are in blue. September–November, named in Table 2 and most of the body, sits at the reported value. March–October, named in the abstract, ranks 16th, with an IQR that does not overlap the reported one. The thin vertical line marks the no-spraying median (361,000). The axis starts at zero.
Why it matters
The month pair is the result the paper sets out to find. The abstract describes timing as "not well understood", and the title is about optimizing deployment. A reader of the abstract alone would plan the first of two yearly ULV campaigns for March. The body puts the best first spray in August or September, "just before the dengue season", and the Author summary advises starting non-residual spraying "close to the start of the dengue transmission season". On the authors' own data, March–October gives a median of about 47,800 more infections over 11 years than September–November (219,402 vs 171,650). That is a 39% reduction instead of 52%, and fifteen pairs do better.
The error does not change the infection counts, the 52% figure (which belongs to September–November), the ranking of twice-yearly over yearly and threshold ULV strategies, the advantage of TIRS, or the recommendation of threshold-based TIRS. The paper's conclusions in the body stand. The problem is that the abstract, the part most often read, indexed and cited, gives a timing the paper's own analysis does not support. Results paragraph 2 repeats it, so it also appears in the main text.
Explanations considered
Typesetting or archiving artifact. Ruled out. The conflicting items are plain month names, not symbols, decimals or table alignment. An earlier audit stage found the same wording in the PLOS XML, the PLOS PDF and the PLOS article page, matching the PMC copy word for word. On 2026-10-23 I fetched the PLOS article page and confirmed that the abstract still reads "in March and October".
March–October and September–November as the same strategy under a different calendar. Ruled out. The Methods treat twice-yearly spraying as 66 unordered month pairs, each starting "on the first days of the shown month" every calendar year. March–October and September–November are separate strategies with separate results. In the source data they have different distributions (medians 219,402 vs 171,650, non-overlapping IQRs).
The abstract refers to a different outcome or summary statistic. Not supported. The abstract says "fewest median infections" and gives the same 172,000 [158,000–183,000] as Table 2. In the source data, March–October is not the best pair on the other output column (rank 14) or on mean infections (rank 15). The Results also say September–November gave the lowest mosquito abundance (S4 Fig).
The 549 spray days as evidence for March–October. Not informative. Every twice-yearly pair has 22 campaigns of 25 days. The median number of spray days is 548–549 for all 66 pairs, so "549 days" in Results paragraph 2 fits either wording.
Two statements against seven. The abstract is not the only place with March and October: Results paragraph 2 uses it too. But the paper reports one best twice-yearly strategy, attaches one set of numbers to it, and those numbers belong to September–November in both the paper's table and the source data. The paper does not explain where the March/October wording came from, and nothing here depends on how it arose.
Status
As of 2026-10-23:
- Version of record. The PLOS article page (journals.plos.org, reached via doi.org) was fetched on 2026-10-23. Its abstract still says "spraying twice yearly, in March and October", and no correction notice was found. An earlier audit stage compared the PLOS XML, the PLOS PDF (28 pages, abstract on p. 1) and the Europe PMC full text with the PMC archive and found them identical, including Results paragraph 2's "in October and March", Table 2 and the figure legends.
- Supplements and source data. The Data Availability Statement points to a public GitHub repository. Its
figures/scripts/spray_analysis_paired_all_years.RDataand the R script that draws Fig 4 were used above. The script picks the best strategy per category as the minimum median in output column 3 (max.col(-x)), which gives September–November. The Fig 5c panel code draws campaign start lines at September and November. The supporting-information figures (e.g. S4 Fig) were not re-read for this write-up. - Corrections. Crossref lists one update for the DOI, a "new version" record dated 2020-05-05 that points to the article's own DOI and is not a correction notice. PMC and Europe PMC list no correction, retraction or expression of concern (
src/known.py). - PubPeer and the wider web. An earlier audit stage's web searches for the DOI or title with "correction", "erratum" or "PubPeer" found no relevant notice or thread. The PubPeer API returned HTTP 403, and the PLOS reader-comments page showed 0 comments. I could not repeat the web search on 2026-10-23 because the session's search quota was used up. The medRxiv preprint (10.1101/19007971) returned HTTP 403, so it is unknown whether the preprint has the same wording. The published peer-review history was not examined.
Sources: PMC7200023 (PubMed Central archive); https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007743; Crossref and PMC notice records for 10.1371/journal.pcbi.1007743; source data from the repository named in the paper's Data Availability Statement.
Suggested correction
Change the abstract sentence to: "The ULV strategy with the fewest median infections was spraying twice yearly, in September and November, which led to a median of 172,000 infections [IQR: 158,000–183,000], a 52% reduction from baseline." In Results ("Ultra-low volume spraying", paragraph 2), change "spraying twice yearly (in October and March)" to "spraying twice yearly (in September and November)". In the same notice, the authors could also reconcile the abstract's TIRS figures ("[IQR: 8,720–11,400], a 94% reduction") with the Results, Table 2 and the Discussion (IQR upper bound 11,300; 1 − 9.9/361 = 97%), or explain the basis of the 94%.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC7200023 (PLoS Comput Biol 2020, doi:10.1371/journal.pcbi.1007743)
# Abstract vs body: which month pair is the best twice-yearly ULV spraying strategy?
#
# Usage: python recompute.py.txt [path/to/spray_analysis_paired_all_years.RData]
# Needs: numpy, matplotlib. Optional: the 'rdata' package (pip install rdata) plus the
# source-data file figures/scripts/spray_analysis_paired_all_years.RData from the
# GitHub repository named in the paper's Data Availability Statement.
# Output: prints every recomputed number used in writeup.md; writes fig1.svg/png and
# fig2.svg/png next to this script.
#
# Sections
# 1. Every statement in the paper that names the best twice-yearly ULV month pair
# (checked verbatim against the PMC archive when src/show.py is available).
# 2. Arithmetic from the paper's own reported numbers.
# 3. Per-pair results from the authors' source data. PAIR_STATS below holds, for each of
# the 66 month pairs, the median, 25th and 75th percentile of total infections over
# 400 simulations (output column 3, as in the authors' Fig 4 script) and the median
# number of spray days (output column 1). The table was produced by this script's
# from_rdata() function; when the RData path is given, it is recomputed and checked.
# 4. Figures.
import os, sys, subprocess
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
MON = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec".split()
# ---------------------------------------------------------------- 1. statements in the paper
STATEMENTS = [ # (location, verbatim text, month pair named)
("Abstract",
"The ULV strategy with the fewest median infections was spraying twice yearly, in March and October, "
"which led to a median of 172,000 infections [IQR: 158,000–183,000], a 52% reduction from baseline.",
"Mar-Oct"),
("Results, ULV para 1",
"The best strategy for spraying twice yearly was spraying in September and November (172,000; "
"IQR: 158,000–183,000) (Fig 4C).", "Sep-Nov"),
("Results, ULV para 1",
"Generally, undertaking the first spray in August or September (typically, just before the dengue "
"season) and the second in October or November (typically, near the start of the dengue season) "
"led to the fewest infections (Fig 4C).", "Aug/Sep then Oct/Nov"),
("Results, ULV para 2",
"Comparing the best strategies, spraying twice yearly (in October and March) averted the most "
"infections, but required the most spraying campaigns: 22 campaigns of spraying, spending a "
"median of 549 days spraying in total over 11 years.", "Mar-Oct"),
("Table 2, ULV best strategy row",
"ULV | Best strategy | N/A | When monthly incidence is 1σ above mean | September | September & November",
"Sep-Nov"),
("Table 2, ULV infections row",
"ULV | Number of infections, 2000–2010 (1,000s) [IQR] | 361 [347, 383] | 254 [210, 277] | "
"261 [250, 277] | 172 [158, 183]", "-"),
("Table 2, ULV days row",
"ULV | Days spent spraying [IQR] | 0 | 351 [341, 371] | 274 [272, 277] | 549 [545, 552]", "-"),
("Discussion, Summary of main findings",
"The best strategy for ULV spraying was to spray twice per year, in September and November.", "Sep-Nov"),
("Fig 4 legend, panel (d)",
"twice yearly corresponds to spraying in September and November.", "Sep-Nov"),
("Fig 8 legend, panel (c)",
"and twice yearly corresponds to spraying in September and November;", "Sep-Nov"),
("Fig 10 legend",
"the bottom row the best twice yearly strategy (spraying in September and November)", "Sep-Nov"),
("Methods, Experiments",
"We tested yearly spraying starting in each month (12 strategies) and twice-yearly spraying in "
"each pair of months (66 strategies) (Table 1).", "-"),
("Author summary",
"For non-residual insecticides, we find that it is best to begin spraying close to the start of "
"the dengue transmission season, as mosquito populations can rebound quickly and resume previous "
"levels of transmission.", "-"),
("Abstract (TIRS)",
"(9,900; [IQR: 8,720–11,400], a 94% reduction)", "-"),
("Results, TIRS para 1",
"resulting in 9,900 infections (IQR: 8,720–11,300)", "-"),
("Results, TIRS para 1",
"The result was many fewer infections (17-fold) than the best ULV strategy, which led to a median "
"of 172,000 infections.", "-"),
("Abstract (aim)",
"The optimal time to deploy these interventions and their relative epidemiological effects are, "
"however, not well understood.", "-"),
("Results, ULV para 1",
"In this case, the strategy that led to the fewest infections also led to the lowest average "
"mosquito abundance: spraying in September and November (S4 Fig).", "Sep-Nov"),
("Fig 4 legend, panel (c)",
"Comparison of the median predicted infections for twice yearly spraying strategies, beginning on "
"the first days of the shown month.", "-"),
("Discussion, Comparison to other studies",
"Our prediction of a 97% reduction in the number of infections for repeated TIRS campaigns", "-"),
]
def check_quotes():
show = os.path.join("/home/ubuntu/papers", "src", "show.py")
py = "/home/ubuntu/papers-data/venv/bin/python"
print("1) Statements in the paper (verbatim check against the PMC archive)")
full = None
if os.path.exists(show) and os.path.exists(py):
r = subprocess.run([py, show, "PMC7200023", "--full"], cwd="/home/ubuntu/papers",
capture_output=True, text=True)
full = r.stdout if r.returncode == 0 else None
for loc, q, pair in STATEMENTS:
tag = "not checked (archive tool unavailable)" if full is None else \
("found verbatim" if q in full else "NOT FOUND")
print(" %-40s pair named: %-22s %s" % (loc, pair, tag))
if full is not None:
assert all(q in full for _, q, _ in STATEMENTS), "a quote was not found verbatim"
# ---------------------------------------------------------------- 2. paper's own arithmetic
REP = dict(baseline=361000, ulv2=172000, ulv2_lo=158000, ulv2_hi=183000, ulv2_days=549,
tirs=9900, tirs_lo_abs=8720, tirs_hi_abs=11400, tirs_hi_res=11300,
tirs_yearly=11800, campaigns=22, days_per_campaign=25, years=11)
def arithmetic():
b = REP["baseline"]
out = {}
out["red_ulv2"] = 100 * (1 - REP["ulv2"] / b)
out["red_tirs"] = 100 * (1 - REP["tirs"] / b)
out["red_tirs_yearly"] = 100 * (1 - REP["tirs_yearly"] / b)
out["fold"] = REP["ulv2"] / REP["tirs"]
out["days"] = REP["campaigns"] * REP["days_per_campaign"]
print("\n2) Arithmetic from the paper's reported numbers")
print(" ULV twice-yearly reduction vs no spraying: 1 - 172/361 = %.1f%% (abstract: 52%%)" % out["red_ulv2"])
print(" 22 campaigns = 2 per year x %d years: %d" % (REP["years"], 2 * REP["years"]))
print(" 22 campaigns x 25 days = %d days (Table 2: median 549 [545, 552])" % out["days"])
print(" Best ULV / best TIRS: 172,000 / 9,900 = %.1f-fold (Results: 17-fold)" % out["fold"])
print(" TIRS adaptive threshold reduction: 1 - 9.9/361 = %.1f%% (abstract: 94%%; Discussion: 97%%)" % out["red_tirs"])
print(" TIRS yearly (Sep) reduction: 1 - 11.8/361 = %.1f%%" % out["red_tirs_yearly"])
print(" TIRS IQR upper bound: abstract 11,400 vs Results 11,300 and Table 2 11.3 (thousand)")
return out
# ---------------------------------------------------------------- 3. authors' source data
# (i, j) 1-based months, i < j : (median, q25, q75 infections; median spray days)
PAIR_STATS = {
(1, 2): (441708.00, 422530.50, 454094.25, 549.0), # Jan-Feb
(1, 3): (432794.00, 416724.00, 445201.75, 549.0), # Jan-Mar
(1, 4): (424317.50, 406502.25, 434176.75, 548.0), # Jan-Apr
(1, 5): (402984.50, 388331.75, 414365.25, 549.0), # Jan-May
(1, 6): (386051.00, 375338.75, 395504.25, 549.0), # Jan-Jun
(1, 7): (359459.50, 348058.50, 369248.25, 549.0), # Jan-Jul
(1, 8): (312679.00, 301817.75, 327553.00, 549.0), # Jan-Aug
(1, 9): (272079.00, 260225.00, 285449.25, 549.0), # Jan-Sep
(1, 10): (244997.50, 234358.25, 251953.75, 549.0), # Jan-Oct
(1, 11): (277347.00, 261847.25, 290349.25, 549.0), # Jan-Nov
(1, 12): (385329.00, 367194.00, 399237.00, 549.0), # Jan-Dec
(2, 3): (430668.50, 406035.50, 449033.00, 548.0), # Feb-Mar
(2, 4): (421545.00, 397439.75, 439812.00, 548.0), # Feb-Apr
(2, 5): (408950.50, 389504.75, 421645.25, 549.0), # Feb-May
(2, 6): (390143.00, 376047.50, 402808.00, 549.0), # Feb-Jun
(2, 7): (363220.50, 350213.75, 374916.50, 549.0), # Feb-Jul
(2, 8): (308439.00, 292948.00, 321977.75, 549.0), # Feb-Aug
(2, 9): (259513.00, 244273.75, 272738.75, 549.0), # Feb-Sep
(2, 10): (234932.00, 227319.50, 245102.25, 549.0), # Feb-Oct
(2, 11): (273375.50, 261919.75, 285630.00, 548.0), # Feb-Nov
(2, 12): (384562.50, 366859.00, 396050.25, 548.0), # Feb-Dec
(3, 4): (410443.00, 380868.50, 429001.00, 549.0), # Mar-Apr
(3, 5): (397934.50, 377044.50, 414733.25, 549.0), # Mar-May
(3, 6): (386533.00, 371343.00, 399381.25, 549.0), # Mar-Jun
(3, 7): (361814.50, 344152.50, 375015.50, 549.0), # Mar-Jul
(3, 8): (309186.00, 294559.75, 321067.00, 549.0), # Mar-Aug
(3, 9): (241259.00, 223740.00, 253670.50, 549.0), # Mar-Sep
(3, 10): (219402.50, 208280.50, 230152.00, 549.0), # Mar-Oct
(3, 11): (247274.00, 235556.50, 257797.25, 549.0), # Mar-Nov
(3, 12): (367927.00, 349634.25, 380767.75, 549.0), # Mar-Dec
(4, 5): (391103.00, 371128.25, 405921.00, 549.0), # Apr-May
(4, 6): (382233.00, 365624.50, 396880.50, 549.0), # Apr-Jun
(4, 7): (354168.50, 341128.75, 366589.25, 548.0), # Apr-Jul
(4, 8): (300416.50, 284996.50, 317564.00, 548.5), # Apr-Aug
(4, 9): (225956.00, 210163.00, 242784.25, 548.0), # Apr-Sep
(4, 10): (207010.00, 194652.50, 219858.50, 549.0), # Apr-Oct
(4, 11): (233842.50, 224488.50, 243371.25, 549.0), # Apr-Nov
(4, 12): (353418.00, 340148.00, 367024.50, 548.0), # Apr-Dec
(5, 6): (378949.00, 359401.25, 395366.00, 548.0), # May-Jun
(5, 7): (355846.00, 339660.00, 374676.00, 548.5), # May-Jul
(5, 8): (306893.50, 290227.25, 327889.75, 548.0), # May-Aug
(5, 9): (230192.50, 211397.25, 253140.75, 549.0), # May-Sep
(5, 10): (197582.00, 186184.00, 213469.75, 549.0), # May-Oct
(5, 11): (216630.00, 209058.00, 223355.25, 549.0), # May-Nov
(5, 12): (334639.00, 322462.75, 344777.25, 549.0), # May-Dec
(6, 7): (364683.00, 346838.50, 386625.75, 549.0), # Jun-Jul
(6, 8): (317711.00, 293881.75, 340245.75, 549.0), # Jun-Aug
(6, 9): (234496.00, 214429.00, 256260.50, 549.0), # Jun-Sep
(6, 10): (199118.00, 182987.75, 217445.00, 549.0), # Jun-Oct
(6, 11): (206853.50, 196577.75, 214035.00, 548.5), # Jun-Nov
(6, 12): (311791.00, 299312.75, 322937.25, 549.0), # Jun-Dec
(7, 8): (321920.00, 297495.75, 344139.50, 548.0), # Jul-Aug
(7, 9): (233935.50, 216610.50, 255196.75, 549.0), # Jul-Sep
(7, 10): (195357.50, 180265.75, 210540.25, 549.0), # Jul-Oct
(7, 11): (190111.00, 181210.75, 200435.50, 549.0), # Jul-Nov
(7, 12): (280071.00, 268760.75, 292520.75, 549.0), # Jul-Dec
(8, 9): (218566.50, 206763.50, 235713.50, 548.0), # Aug-Sep
(8, 10): (183435.00, 168294.25, 200184.25, 549.0), # Aug-Oct
(8, 11): (175365.50, 163921.25, 187042.25, 549.0), # Aug-Nov
(8, 12): (239777.00, 228854.50, 250519.25, 549.0), # Aug-Dec
(9, 10): (176823.50, 160305.75, 192265.75, 549.0), # Sep-Oct
(9, 11): (171650.50, 158131.00, 183441.50, 549.0), # Sep-Nov
(9, 12): (211443.50, 200702.00, 222217.50, 549.0), # Sep-Dec
(10, 11): (188972.50, 165478.00, 203092.00, 549.0), # Oct-Nov
(10, 12): (211302.00, 194398.25, 221180.25, 548.0), # Oct-Dec
(11, 12): (266319.00, 252818.25, 283819.75, 549.0), # Nov-Dec
}
NONE_STATS = (360831.0, 346748.0, 383264.5) # no spraying: median, q25, q75 (none.output col 3)
YEARLY_SEP = (260707.5, 249588.75, 276903.0, 274.0) # yearly September: median, q25, q75, days
THRESH_BEST = (254206.0, 351.0) # best adaptive threshold (1 sd, monthly): median, days
def pair_index(i, j):
"""Column of tot.output[['spray_twice_yearly_all']] for months i<j (1-based), from the authors'
Fig 4 script: pairs.by.month[i,j] <- -0.5*(i^2-25*i+24)+(j-i)."""
return int(-0.5 * (i * i - 25 * i + 24) + (j - i))
def from_rdata(path):
import rdata # pip install rdata
conv = rdata.conversion.convert(rdata.parser.parse_file(path), default_encoding="utf8")
tw = np.asarray(conv["tot.output"]["spray_twice_yearly_all"]) # 400 sims x 3 outputs x 66 pairs
yr = np.asarray(conv["tot.output"]["spray_yearly_all"])
th = np.asarray(conv["tot.output"]["spray_threshold_mean"])
no = np.asarray(conv["none.output"])
assert tw.shape == (400, 3, 66)
stats = {}
for i in range(1, 12):
for j in range(i + 1, 13):
k = pair_index(i, j) - 1
x = tw[:, 2, k]
q25, med, q75 = np.quantile(x, [0.25, 0.5, 0.75]) # = R quantile type 7
stats[(i, j)] = (round(med, 2), round(q25, 2), round(q75, 2), round(np.median(tw[:, 0, k]), 1))
none = tuple(np.quantile(no[:, 2], [0.5, 0.25, 0.75]))
ysep = tuple(np.quantile(yr[:, 2, 8], [0.5, 0.25, 0.75])) + (np.median(yr[:, 0, 8]),)
tbest = (np.median(th[:, 2, 0]), np.median(th[:, 0, 0]))
# Is Mar-Oct the best pair under any other summary in the file? (only with the RData file)
pairs = list(stats)
for label, f in [("output column 2, median", lambda k: np.median(tw[:, 1, k])),
("output column 3 (infections), mean", lambda k: np.mean(tw[:, 2, k]))]:
v = {p: f(pair_index(*p) - 1) for p in pairs}
order = sorted(pairs, key=v.get)
print(" [RData] %s: best pair %s-%s; Mar-Oct ranks %d of 66"
% (label, MON[order[0][0] - 1], MON[order[0][1] - 1], order.index((3, 10)) + 1))
return stats, none, ysep, tbest
def source_data(rpath):
print("\n3) Authors' source data (400 simulations per strategy)")
if rpath:
stats, none, ysep, tbest = from_rdata(rpath)
for k, v in PAIR_STATS.items():
assert np.allclose(v, stats[k], atol=0.01), (k, v, stats[k])
assert np.allclose(none, NONE_STATS) and np.allclose(ysep, YEARLY_SEP) and np.allclose(tbest, THRESH_BEST)
print(" recomputed from %s: matches the embedded table for all 66 pairs" % os.path.basename(rpath))
else:
print(" (no RData path given: using the embedded table produced by from_rdata())")
ranked = sorted(PAIR_STATS.items(), key=lambda kv: kv[1][0])
rank = {k: r + 1 for r, (k, _) in enumerate(ranked)}
b = NONE_STATS[0]
print(" No spraying: median %.0f [IQR %.0f-%.0f] (paper: 361 [347, 383] thousand)" % NONE_STATS)
print(" Yearly Sep: median %.0f [IQR %.0f-%.0f], %.0f days (paper: 261 [250, 277]; 274 days)" % YEARLY_SEP)
print(" Best threshold: median %.0f, %.0f days (paper: 254; 351 days)" % THRESH_BEST)
print(" Five best twice-yearly pairs:")
for (i, j), v in ranked[:5]:
print(" %2d. %s-%s median %.0f [IQR %.0f-%.0f], %.0f spray days" % ((rank[(i, j)], MON[i-1], MON[j-1]) + v))
out = {}
for (i, j), name in [((9, 11), "Sep-Nov"), ((3, 10), "Mar-Oct")]:
m, lo, hi, d = PAIR_STATS[(i, j)]
red = 100 * (1 - m / b)
out[name] = (m, lo, hi, d, rank[(i, j)], red)
print(" %s: median %.1f [IQR %.1f-%.1f], %.0f spray days, rank %d of 66, %.1f%% fewer than no spraying"
% (name, m, lo, hi, d, rank[(i, j)], red))
print(" Mar-Oct minus Sep-Nov median infections: %.0f; pairs better than Mar-Oct: %d"
% (out["Mar-Oct"][0] - out["Sep-Nov"][0], out["Mar-Oct"][4] - 1))
print(" IQRs overlap? %s" % (out["Mar-Oct"][1] <= out["Sep-Nov"][2]))
days = sorted(set(v[3] for v in PAIR_STATS.values()))
print(" Median spray days across all 66 pairs: %s (so 549 days does not identify a pair)" % days)
print(" Pairs with first spray in Aug/Sep and second in Oct/Nov, ranks: %s" %
sorted(rank[(i, j)] for i in (8, 9) for j in (10, 11)))
return ranked, rank, out
# ---------------------------------------------------------------- 4. figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#b9b7b1"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": FG,
"axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
"axes.edgecolor": FG2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def fig1(src):
sn, mo = src["Sep-Nov"], src["Mar-Oct"]
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.9); ax.axis("off")
def panel(x, y, w, h, edge, title, body, sub):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.14, y + h - 0.16, title, ha="left", va="top", fontsize=11, weight="bold", color=FG)
ax.text(x + 0.14, y + h - 0.50, body, ha="left", va="top", fontsize=10, color=FG, linespacing=1.4)
ax.text(x + 0.14, y + 0.14, sub, ha="left", va="bottom", fontsize=10, color=FG2, linespacing=1.3)
panel(0.12, 1.95, 3.05, 2.85, ORANGE, "Abstract",
"“spraying twice yearly, in\n"
"March and October, which led to\n"
"a median of 172,000 infections\n"
"[IQR: 158,000–183,000], a 52%\n"
"reduction from baseline”",
"Same pair in Results, ULV para 2:\n“in October and March” … 549 days")
panel(4.03, 1.95, 3.05, 2.85, ORANGE, "Results, Table 2, Discussion",
"“The best strategy for spraying\n"
"twice yearly was spraying in\n"
"September and November (172,000;\n"
"IQR: 158,000–183,000)”\n"
"Table 2: 172 [158, 183]; 549 days",
"Same pair in the legends of\nFig 4d, Fig 8c and Fig 10")
ax.add_patch(FancyArrowPatch((3.19, 3.45), (4.01, 3.45), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=FG))
ax.text(3.6, 3.68, "conflict", ha="center", va="bottom", fontsize=10, weight="bold", color=FG)
ax.text(3.6, 3.22, "same\nnumbers,\ndifferent\nmonths", ha="center", va="top", fontsize=10, color=FG2)
# recomputed strip
ax.add_patch(FancyBboxPatch((0.12, 0.12), 6.96, 1.5, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="#eef4fb", ec=BLUE, lw=1.5))
ax.text(0.26, 1.48, "Recomputed from the authors' source data (66 month pairs, 400 simulations each)",
ha="left", va="top", fontsize=10, weight="bold", color=BLUE)
ax.text(0.26, 1.16, "Mar–Oct: median %s [IQR %s–%s], rank %d of 66, %.0f%% reduction"
% (f"{mo[0]:,.0f}", f"{mo[1]:,.0f}", f"{mo[2]:,.0f}", mo[4], mo[5]),
ha="left", va="top", fontsize=10, color=FG)
ax.text(0.26, 0.86, "Sep–Nov: median %s [IQR %s–%s], rank %d of 66, %.0f%% reduction"
% (f"{sn[0]:,.0f}", f"{sn[1]:,.0f}", f"{sn[2]:,.0f}", sn[4], sn[5]),
ha="left", va="top", fontsize=10, color=FG)
ax.text(0.26, 0.56, "Both pairs: median %.0f spray days. The reported 172,000 [158,000–183,000]\n"
"and 52%% reduction fit Sep–Nov only." % sn[3], ha="left", va="top", fontsize=10, color=FG2,
linespacing=1.3)
# connectors from strip to the panels
ax.annotate("", xy=(1.65, 1.93), xytext=(1.65, 1.64),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2, linestyle=(0, (3, 2))))
ax.text(1.75, 1.785, "does not match", ha="left", va="center", fontsize=10, color=BLUE)
ax.annotate("", xy=(5.55, 1.93), xytext=(5.55, 1.64),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2))
ax.text(5.65, 1.785, "matches", ha="left", va="center", fontsize=10, color=BLUE)
save(fig, "fig1")
def fig2(ranked, rank, src, n_show=20):
rows = ranked[:n_show]
fig, ax = plt.subplots(figsize=(7.2, 6.4))
fig.subplots_adjust(left=0.22, right=0.97, top=0.86, bottom=0.09)
y_rep = 0
ax.errorbar(REP["ulv2"] / 1e3, y_rep,
xerr=[[(REP["ulv2"] - REP["ulv2_lo"]) / 1e3], [(REP["ulv2_hi"] - REP["ulv2"]) / 1e3]],
fmt="o", ms=8, color=ORANGE, lw=2, capsize=0, zorder=3)
labels = ["Reported best pair\n172 [158, 183]"]
for r, ((i, j), (m, lo, hi, d)) in enumerate(rows, start=1):
hl = (i, j) in [(9, 11), (3, 10)]
c = BLUE if hl else GRAY
ax.errorbar(m / 1e3, r + 0.5, xerr=[[(m - lo) / 1e3], [(hi - m) / 1e3]], fmt="o",
ms=8 if hl else 6, color=c, lw=2 if hl else 1.5, capsize=0, zorder=3)
labels.append("%d. %s–%s" % (r, MON[i - 1], MON[j - 1]))
if hl:
note = ("Pair named in the Abstract" if (i, j) == (3, 10) else "Pair named in Table 2")
left = (i, j) == (3, 10) # place the label where the row has free space
ax.text((lo / 1e3 - 6) if left else (hi / 1e3 + 6), r + 0.5, "%s: %.1f k" % (note, m / 1e3),
va="center", ha="right" if left else "left",
fontsize=10, color=BLUE)
ax.set_yticks([0] + [r + 0.5 for r in range(1, n_show + 1)])
ax.set_yticklabels(labels, fontsize=10)
for t in ax.get_yticklabels()[1:]:
t.set_color(FG2)
ax.get_yticklabels()[0].set_color(ORANGE)
ax.invert_yaxis()
ax.axvline(NONE_STATS[0] / 1e3, color=FG2, lw=1)
ax.text(NONE_STATS[0] / 1e3 - 4, n_show + 0.9, "no spraying\n%.0f k" % (NONE_STATS[0] / 1e3),
ha="right", va="bottom", fontsize=10, color=FG2)
ax.set_xlim(0, 400)
ax.set_ylim(n_show + 1.6, -0.8)
ax.set_xlabel("Total infections 2000–2010, thousands (median and IQR, 400 simulations)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.text(0.02, 0.975, "Twice-yearly ULV month pairs ranked by median infections (best %d of 66)" % n_show,
fontsize=11, weight="bold", ha="left", va="top")
fig.text(0.02, 0.938, "Orange: the paper's reported best twice-yearly result (Abstract and Table 2).\n"
"Blue: the two named pairs, recomputed from the authors' source data. Gray: other pairs.",
fontsize=10, color=FG2, ha="left", va="top", linespacing=1.3)
save(fig, "fig2")
if __name__ == "__main__":
check_quotes()
arithmetic()
rpath = sys.argv[1] if len(sys.argv) > 1 else None
ranked, rank, src = source_data(rpath)
fig1(src)
fig2(ranked, rank, src)
print("\nFigures written to", HERE)
Abstract reports SF2/SRSF1 decrease in astrocytes; the paper's data show it only in PNG and SPG
PLoS Genet (2023) · doi:10.1371/journal.pgen.1010973 · PMC10553832
Summary
The abstract says that after glial TDP-43 induction "SF2/SRSF1 levels were found to be decreased in both PNG and in astrocytes." The paper's own expression data, S8 Table, report a significant decrease only in perineurial glia (PNG; log2 fold change −1.31, p<0.0001) and subperineurial glia (SPG; −0.85, p<0.001). For astrocyte-like glia (ALG), the cell type the paper calls astrocytes, the table gives "n/a", and the Results say outright that TDP-43 induction in astrocytes "does not on its own cause a decrease in SF2/SRSF1 expression." So the abstract names the one cell type in which the body says there is no decrease and leaves out SPG, where there is one. The lifespan rescue by SF2/SRSF1 knockdown (Fig 5) comes from separate experiments and is not affected, but the abstract misstates the only specific transcriptomic result it names.
What the paper reports
Abstract:
Notably, SF2/SRSF1 levels were found to be decreased in both PNG and in astrocytes. We found that further knockdown of SF2/SRSF1 in either PNG or astrocytes lessens the detrimental effects of TDP-43 pathology on lifespan, but extends survival of the glial cells.
Results, section "Knockdown of SF2/SRSF1 in astrocytes or PNG ameliorates the toxicity of TDP-43", first paragraph:
Our differential expression analysis identified SF2/SRSF1 as being down regulated in response to TDP-43 expression in PNG (-1.31 log2 fold change, p<0.0001) and SPG (-0.85 log2 fold change, p < .001), but not in CG, ALG, or EG (S8 Table).
Same section, second paragraph:
The fact that SF2/SRSF1 levels are reduced in SPG and PNG, but not other glial cell types, is intriguing given that pan-glial knockdown of SF2/SRSF1 can ameliorate the lifespan effects of pan-glial TDP-43 pathology (Fig 5A) [55]. We observe a substantial impact on lifespan from pathological TDP-43 induction in PNG and ALG (Fig 1), but only see down regulation of SF2/SRSF1 in PNG (S8 Table).
And although TDP-43 induction in astrocytes does not on its own cause a decrease in SF2/SRSF1 expression, knock-down in astrocytes of SF2/SRSF1 by RNAi does ameliorate the effects on lifespan from astrocytic induction of TDP-43 pathology.
Introduction, last paragraph:
We observe that in response to TDP-43 pathology, SF2/SRSF1 is down-regulated in both the PNG and SPG specialized BBB glia.
Discussion, paragraph on the expression profiling:
Notably, the SF2/SRSF1 gene exhibits decreased expression in PNG and SPG, but not other glial cell types (S8 Table).
How the paper identifies astrocytes. Introduction:
the cortex glia (CG), astrocyte-like glia (ALG), and ensheathing glia (EG) provide many of the functional characteristics of mammalian astrocytes and microglia
The strain table (Table 2) lists "R86E01-Gal4 (ALG)". The Fig 5C genotype in the Methods is "Tubulin-Gal80 ts / UAS-SF2-IR; UAS-TDP-43/ R86E01-Gal4", and the Fig 5 legend labels that panel "ALGts + TDP-43 + SF2-IR". The Results describe the same experiment as "the PNG or Astrocyte Gal4 lines" and "TDP-43 induction in either PNG or ALG (Fig 5B and 5C)". In this paper "astrocytes" and ALG are the same cells.
S8 Table (supplementary file pgen.1010973.s011), legend:
SF2 expression was significantly reduced at day 2 post induction of TDP-43 in SPG and PNG day 2.
The table's body, paraphrased from the PDF (row label "SF2 expression (MRN)"; values per group, given here as control → +TDP-43, then log2 fold change and p):
- PNG: 899 → 363; −1.31; p<0.0001
- SPG: 1090 → 594; −0.85; p<0.001
- CG: 780 → 727; n/a; n/a
- ALG: 591 → 453; n/a; n/a
- EG: 780 → 669; n/a; n/a
The column headers give n = 3 libraries per group, except n = 2 for the CG and ALG controls. The table does not say whether the per-group values are means, and it does not define "n/a"; the Results' "but not in CG, ALG, or EG" supports reading "n/a" as not significant. The Methods say the analysis used DESeq2 with a Benjamini–Hochberg FDR threshold of p < 0.05.
What the paper's own numbers show
Method. I took the five cell types' per-group SF2/SRSF1 expression values (control and TDP-43) and the reported log2 fold changes and p values from S8 Table. For each cell type I computed the raw log2 ratio of the two group values, treating them as group-level summaries. This is not the DESeq2 estimate, which is model-based and works on per-library counts, so it can only check that the table's group values and its reported fold changes agree; it is not a test. I then listed which cell types the Abstract, Introduction, Results, Discussion and S8 Table each say are decreased, and compared the lists. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| PNG, SF2/SRSF1 change | S8 Table; Results | 899 → 363 MRN; log2FC −1.31, p<0.0001 | log2(363/899) = −1.308; group values fall 59.6% (2^−1.31 = 0.403, a 59.7% fall) |
| SPG, SF2/SRSF1 change | S8 Table; Results | 1090 → 594 MRN; log2FC −0.85, p<0.001 | log2(594/1090) = −0.876; group values fall 45.5% (2^−0.85 = 0.555, a 44.5% fall). The 0.026 gap from −0.85 is consistent with model-based estimation |
| ALG (astrocyte-like glia), SF2/SRSF1 change | S8 Table; Results | 591 → 453 MRN; log2FC n/a, p n/a | log2(453/591) = −0.384 (−23.4%); not reported as significant ("n/a", read with the Results' "but not in ... ALG") |
| CG, SF2/SRSF1 change | S8 Table | 780 → 727 MRN; n/a | log2(727/780) = −0.102 (−6.8%) |
| EG, SF2/SRSF1 change | S8 Table | 780 → 669 MRN; n/a | log2(669/780) = −0.221 (−14.2%) |
| Cell types with a significant decrease | S8 Table and legend | PNG, SPG | PNG, SPG |
| Cell types with a decrease, per the text | Introduction; Results; Discussion | PNG and SPG (Results and Discussion add: not CG, ALG or EG) | agrees with S8 Table |
| Cell types with a decrease, per the abstract | Abstract | PNG and astrocytes (ALG) | ALG is not significant in S8 Table; SPG, which is significant, is not named |
Figure
Figure 1. The abstract sentence (left) and the Results sentences and S8 Table entry for the same cell type (right), quoted verbatim from the article as archived in PMC. The abstract names astrocytes, which the paper identifies as ALG, as a cell type in which SF2/SRSF1 decreased. The Results and S8 Table report no significant change in ALG (591 to 453 MRN, log2 fold change "n/a"), and the Results state that TDP-43 induction in astrocytes does not on its own decrease SF2/SRSF1. The Introduction and Discussion give the decrease as PNG and SPG.
Figure 2. SF2/SRSF1 log2 fold change after TDP-43 induction (day 2) in each glial cell type. Orange diamonds are the DESeq2 estimates reported in S8 Table and the Results, which exist only for PNG and SPG; the other three cell types are given as "n/a", which the Results describe as not down-regulated. Blue circles are the raw log2 ratio of the S8 Table group values, computed by recompute.py.txt as a consistency check. The vertical line marks no change. The ALG point, which the abstract describes as decreased, is a non-significant −0.38.
Why it matters
The abstract is what most readers and indexing services see, and this sentence is the only specific transcriptomic result the abstract names (it otherwise says only that the authors "identified numerous glial cell-type specific transcriptional changes"). As written, it tells readers that TDP-43 lowers SF2/SRSF1 in the two cell types with the strongest lifespan effects (PNG and astrocytes), and the next sentence's "further knockdown" builds on that. The body's account is different, and the Results call it "intriguing": the decrease is confined to the two blood-brain-barrier glia (PNG and SPG), it is absent in astrocytes, and knockdown in astrocytes still extends lifespan. The Results draw exactly this contrast ("although TDP-43 induction in astrocytes does not on its own cause a decrease..."). A reader who relies on the abstract is told there is a decrease in astrocytes, where the paper reports no significant change, and does not learn about the SPG decrease.
What is not affected: the lifespan extension from SF2/SRSF1 knockdown in pan-glial, PNG and ALG TDP-43 models (Fig 5A–C, log-rank tests), the delayed glial loss with knockdown (Fig 5D–E), and the reaper ablation result in PNG. These are separate experiments and do not depend on the expression data. The title's claim stands.
Explanations considered
"Astrocytes" in the abstract means a different cell type, such as SPG. This does not fit. The paper uses "astrocytes" for ALG throughout: the Results section title is "Knockdown of SF2/SRSF1 in astrocytes or PNG", the experiment uses "the PNG or Astrocyte Gal4 lines" and reports the outcome as "PNG or ALG (Fig 5B and 5C)", and the Fig 5C driver is R86E01-Gal4, listed in Table 2 as the ALG line. SPG is described as a blood-brain-barrier glial type, alongside PNG, and is never called astrocytes.
A different time point or dataset shows an ALG decrease. None found. S8 Table covers one time point (day 2 after induction), the same as the RNA-seq design described in the Results, and the text cites no other SF2/SRSF1 expression data.
The abstract describes a non-significant trend. The ALG group values do fall (raw log2 ratio −0.38, from n = 2 control libraries), but the paper reports this as n/a, and its own Results rule out this reading by saying ALG is not down-regulated and that astrocytic TDP-43 "does not on its own cause a decrease." The abstract's wording ("were found to be decreased", the same wording used for PNG) presents it as a finding.
Text conversion or archiving error. Not supported. The PLOS article page, the Europe PMC full text and the PMC archive have the same abstract sentence, and the S8 Table PDF from the publisher has the values above.
The small SPG discrepancy (−0.876 vs −0.85). A ratio of group values need not equal a DESeq2 model estimate. This gap is small and does not bear on the finding; the PNG value agrees to within 0.002.
A narrower reading of one Results sentence. "...only see down regulation of SF2/SRSF1 in PNG" agrees with the table: it compares only PNG and ALG, the two cell types with the strongest lifespan effects, and SPG is not part of that comparison.
Status
As of 2026-10-23:
- Version of record. The PLOS Genetics article page (the current "Version 2"; a "Version 1 Uncorrected Proof" is also linked) has the same abstract and Results wording as the PMC archive. The Europe PMC full text has the identical abstract sentence.
- Supplements. S8 Table (doi:10.1371/journal.pgen.1010973.s011), a one-page PDF, was downloaded from the publisher and its text extracted; the values match those above. It is the table the text cites for this result. The other supplementary items (S1–S3 Figs, S1–S7 Tables) were not needed for this check.
- Corrections. Crossref lists only a publisher "new version" relation dated 2023-10-05, ten days after online publication on 2023-09-25. This fits the replacement of the Version 1 Uncorrected Proof by Version 2. It has no correction DOI. PMC and Europe PMC list no correction, erratum, expression of concern or retraction, and Europe PMC's record links only the bioRxiv preprint (doi:10.1101/2023.05.04.539439). The PLOS comments page shows no reader comments.
- PubPeer. Web searches found no PubPeer thread or other public discussion of this point. PubPeer itself could not be searched directly (it returned a human-verification page), so a thread there cannot be ruled out.
Sources: the article as archived in PMC (PMC10553832); journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1010973 and its supplementary file s011; Crossref and Europe PMC records for doi:10.1371/journal.pgen.1010973.
Suggested correction
In the abstract, replace
Notably, SF2/SRSF1 levels were found to be decreased in both PNG and in astrocytes. We found that further knockdown of SF2/SRSF1 in either PNG or astrocytes lessens...
with wording that matches the Results and S8 Table, for example:
Notably, SF2/SRSF1 levels were found to be decreased in PNG and SPG, but not in astrocytes. We found that knockdown of SF2/SRSF1 in either PNG or astrocytes lessens...
The word "further" would then apply only to PNG. If the authors prefer to keep the sentence about the two cell types tested in Fig 5, it could instead say that SF2/SRSF1 was decreased in PNG but not in astrocytes.
Recomputation script (python recompute.py.txt)
"""Recompute the SF2/SRSF1 cell-type result of PMC10553832
(PLoS Genet 2023, doi:10.1371/journal.pgen.1010973).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file, plus five candidate figure designs
(design_A ... design_E, .svg and .png) in ./candidates/.
Inputs are copied from the paper:
S8 Table (supplementary file pgen.1010973.s011, "SF2 expression across glial cell
types"): the per-group SF2 expression value (row label "SF2 expression (MRN)") for
control and +TDP-43 libraries at day 2 after induction, the reported log2 fold change
and p value. The table does not define "n/a"; the Results' "but not in CG, ALG, or EG"
supports reading it as not significant. The table does not say whether the per-group
values are means; they are treated here as group-level summaries.
Text excerpts: Abstract, Introduction, Results ("Knockdown of SF2/SRSF1 in astrocytes
or PNG ameliorates the toxicity of TDP-43"), Discussion, as archived in PMC.
The raw log2 ratio of two group values is not the DESeq2 estimate the paper reports
(which is model-based and uses per-library normalized counts); it is shown only as a
check that the reported values and the table's group values agree.
"""
import math
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
# cell type: (n control, MRN control, n TDP-43, MRN TDP-43, reported log2FC, reported p)
S8 = {
"PNG": (3, 899, 3, 363, -1.31, "<0.0001"),
"SPG": (3, 1090, 3, 594, -0.85, "<0.001"),
"CG": (2, 780, 3, 727, None, "n/a"),
"ALG": (2, 591, 3, 453, None, "n/a"),
"EG": (3, 780, 3, 669, None, "n/a"),
}
LONG = {
"PNG": "PNG (perineurial glia)",
"SPG": "SPG (subperineurial glia)",
"CG": "CG (cortex glia)",
"ALG": "ALG (astrocyte-like glia)",
"EG": "EG (ensheathing glia)",
}
ORDER = ["PNG", "SPG", "CG", "ALG", "EG"]
ABSTRACT = ("Notably, SF2/SRSF1 levels were found to be decreased in both PNG "
"and in astrocytes.")
RESULTS_1 = ("Our differential expression analysis identified SF2/SRSF1 as being down "
"regulated in response to TDP-43 expression in PNG (-1.31 log2 fold change, "
"p<0.0001) and SPG (-0.85 log2 fold change, p < .001), but not in CG, ALG, "
"or EG (S8 Table).")
RESULTS_2 = ("And although TDP-43 induction in astrocytes does not on its own cause a "
"decrease in SF2/SRSF1 expression, knock-down in astrocytes of SF2/SRSF1 by "
"RNAi does ameliorate the effects on lifespan from astrocytic induction of "
"TDP-43 pathology.")
# What each part of the paper says about an SF2/SRSF1 decrease in each cell type.
# "dec" = says decreased, "not" = says not decreased, None = cell type not named.
SAYS = {
"Abstract": {"PNG": "dec", "SPG": None, "CG": None, "ALG": "dec", "EG": None},
"Introduction": {"PNG": "dec", "SPG": "dec", "CG": None, "ALG": None, "EG": None},
"Results": {"PNG": "dec", "SPG": "dec", "CG": "not", "ALG": "not", "EG": "not"},
"Discussion": {"PNG": "dec", "SPG": "dec", "CG": "not", "ALG": "not", "EG": "not"},
}
# ---------------------------------------------------------------- style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
REP = "#eb6834" # values as reported by the paper
REC = "#2a78d6" # values recomputed from the paper's own numbers
GRID = "#e4e3df"
PANEL = "#f3f2ee"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans",
"font.size": 10,
"axes.edgecolor": INK2,
"axes.labelcolor": INK,
"xtick.color": INK2,
"ytick.color": INK2,
"text.color": INK,
"svg.fonttype": "none",
"lines.linewidth": 2,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", facecolor=BG)
fig.savefig(path_noext + ".png", dpi=200, facecolor=BG)
plt.close(fig)
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
# ---------------------------------------------------------------- recomputation
print("S8 Table, SF2/SRSF1 expression (MRN) per group, day 2 after induction")
print(f"{'cell type':28s}{'ctrl':>6s}{'TDP43':>7s}{'ratio':>8s}{'raw log2':>10s}"
f"{'change':>9s}{'reported':>10s}{'p':>9s}")
RAW = {}
for k in ORDER:
nc, c, nt, t, r, p = S8[k]
ratio = t / c
RAW[k] = math.log2(ratio)
print(f"{LONG[k]:28s}{c:6d}{t:7d}{ratio:8.3f}{RAW[k]:10.3f}{100 * (ratio - 1):8.1f}%"
f"{('' if r is None else f'{r:.2f}'):>10s}{p:>9s}")
print("\nReported log2 fold changes converted to percent change")
for k in ("PNG", "SPG"):
r = S8[k][4]
print(f" {k}: 2^({r}) = {2 ** r:.3f} -> {100 * (2 ** r - 1):.1f}%")
print(f" SPG: reported -0.85 vs raw log2 ratio of group values {RAW['SPG']:.3f} "
f"(difference {RAW['SPG'] - (-0.85):.3f})")
print(f" PNG: reported -1.31 vs raw log2 ratio of group values {RAW['PNG']:.3f} "
f"(difference {RAW['PNG'] - (-1.31):.3f})")
sig = [k for k in ORDER if S8[k][5] != "n/a"]
print("\nCell types with a significant SF2/SRSF1 decrease in S8 Table:", ", ".join(sig))
abstract_types = ["PNG", "ALG"] # 'astrocytes' = ALG, see writeup
print("Cell types the abstract says are decreased: PNG, astrocytes (= ALG)")
print("Named in the abstract but not significant in S8 Table:",
", ".join(k for k in abstract_types if k not in sig))
print("Significant in S8 Table but not named in the abstract:",
", ".join(k for k in sig if k not in abstract_types))
print("\nWhat each section says (dec = decreased, not = not decreased, - = not named)")
print(f"{'':14s}" + "".join(f"{k:>6s}" for k in ORDER))
for sec, d in SAYS.items():
print(f"{sec:14s}" + "".join(f"{(d[k] or '-'):>6s}" for k in ORDER))
print(f"{'S8 Table':14s}" + "".join(f"{('dec' if k in sig else 'n/a'):>6s}" for k in ORDER))
# ================================================================ design A: two text panels
def design_a(path):
fig = plt.figure(figsize=(W, 4.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
L = (0.025, 0.385)
R = (0.585, 0.975)
top, bot = 0.875, 0.17
for (x0, x1) in (L, R):
ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot,
boxstyle="round,pad=0,rounding_size=0.015",
fc=PANEL, ec=GRID, lw=1))
ax.text(L[0] + 0.015, top - 0.04, "Abstract", fontsize=12, weight="bold", va="top")
ax.text(L[0] + 0.015, top - 0.12, wrap("“" + ABSTRACT + "”", 33),
fontsize=10, va="top", linespacing=1.35)
ax.text(L[0] + 0.015, bot + 0.16, "Cell types said to decrease:", fontsize=10,
color=INK2, va="top")
ax.text(L[0] + 0.015, bot + 0.095, "PNG, astrocytes (ALG)", fontsize=10.5,
weight="bold", color=INK, va="top")
ax.text(R[0] + 0.015, top - 0.04, "Results and S8 Table", fontsize=12,
weight="bold", va="top")
ax.text(R[0] + 0.015, top - 0.12,
wrap("“…down regulated in response to TDP-43 expression in PNG "
"(-1.31 log2 fold change, p<0.0001) and SPG (-0.85 log2 fold change, "
"p < .001), but not in CG, ALG, or EG (S8 Table).”", 35),
fontsize=10, va="top", linespacing=1.35)
ax.text(R[0] + 0.015, top - 0.405,
wrap("“…TDP-43 induction in astrocytes does not on its own cause a "
"decrease in SF2/SRSF1 expression…”", 35),
fontsize=10, va="top", linespacing=1.35)
ax.text(R[0] + 0.015, bot + 0.16, "S8 Table, ALG: 591 → 453 MRN,", fontsize=10,
color=INK2, va="top")
ax.text(R[0] + 0.015, bot + 0.095, "log2FC n/a (not significant)", fontsize=10.5,
weight="bold", va="top")
ymid = 0.52
ax.annotate("", xy=(R[0] - 0.008, ymid), xytext=(L[1] + 0.008, ymid),
arrowprops=dict(arrowstyle="<|-|>", color=INK, lw=2,
shrinkA=0, shrinkB=0, mutation_scale=14))
ax.text((L[1] + R[0]) / 2, ymid + 0.035, "Conflict:\nastrocytes (ALG)",
ha="center", va="bottom", fontsize=10, weight="bold", linespacing=1.3)
ax.text((L[1] + R[0]) / 2, ymid - 0.035, "“decreased”\nvs\nnot decreased",
ha="center", va="top", fontsize=10, color=INK2, linespacing=1.3)
ax.text(0.025, 0.95,
"SF2/SRSF1 after TDP-43 induction: what the abstract says vs the paper's data",
fontsize=11, weight="bold", va="top")
ax.text(0.025, 0.13,
"Introduction and Discussion agree with the Results: the decrease is in PNG "
"and SPG.\nSPG is not named in the abstract.",
fontsize=10, color=INK2, va="top")
ax.text(0.025, 0.05, "Quotes verbatim from the article as archived in PMC; "
"S8 Table = supplementary file s011.", fontsize=10, color=INK2, va="top")
save(fig, path)
# ================================================================ design B: section x cell-type matrix
def design_b(path):
secs = list(SAYS) + ["S8 Table"]
fig, ax = plt.subplots(figsize=(W, 3.9), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.985, top=0.81, bottom=0.17)
ax.set_facecolor(BG)
ax.set_xlim(0, len(secs))
ax.set_ylim(len(ORDER), 0)
for j, sec in enumerate(secs):
for i, k in enumerate(ORDER):
if sec == "S8 Table":
nc, c, nt, t, r, p = S8[k]
txt = f"{r:.2f}\np{p}" if r is not None else "n/a"
state = "dec" if r is not None else "not"
col = REP
else:
state = SAYS[sec][k]
txt = {"dec": "decreased", "not": "not\ndecreased", None: "—"}[state]
col = INK if state else INK2
if sec == "Abstract" and k == "ALG":
txt = "decreased\n(conflict)"
fc = "#e9e7e1" if state == "dec" else BG
ax.add_patch(Rectangle((j + 0.04, i + 0.06), 0.92, 0.88, fc=fc, ec=GRID, lw=1))
ax.text(j + 0.5, i + 0.5, txt, ha="center", va="center", fontsize=10,
color=col, linespacing=1.15,
weight="bold" if (sec == "S8 Table" and state == "dec")
or (sec == "Abstract" and k == "ALG") else "normal")
# conflict cell
i = ORDER.index("ALG")
ax.add_patch(Rectangle((0.04, i + 0.06), 0.92, 0.88, fc="none", ec=INK, lw=2.5))
ax.set_xticks([j + 0.5 for j in range(len(secs))])
ax.set_xticklabels(secs, fontsize=10)
ax.xaxis.tick_top()
ax.set_yticks([i + 0.5 for i in range(len(ORDER))])
ax.set_yticklabels([LONG[k] for k in ORDER], fontsize=10)
ax.tick_params(length=0)
for s in ax.spines.values():
s.set_visible(False)
fig.text(0.015, 0.965, "Where the paper says SF2/SRSF1 decreases after TDP-43 induction",
fontsize=11, weight="bold", va="top")
fig.text(0.015, 0.06, "Shaded = says decreased; orange = S8 Table's reported log2 fold "
"change and p; \u2014 = not named.", fontsize=10, color=INK2)
fig.text(0.015, 0.015, "Astrocytes in the abstract are the ALG row (the paper's "
"astrocyte Gal4 line, R86E01, drives ALG).", fontsize=10, color=INK2)
save(fig, path)
# ================================================================ design C: log2FC dot plot
def design_c(path):
fig, ax = plt.subplots(figsize=(W, 3.9), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.72, top=0.79, bottom=0.215)
ax.set_facecolor(BG)
ys = {k: i for i, k in enumerate(ORDER)}
ax.axvline(0, color=INK2, lw=1)
for k in ORDER:
y = ys[k]
ax.plot(RAW[k], y + 0.13, "o", ms=9, color=REC, mec=BG, mew=1.5, zorder=3)
r = S8[k][4]
if r is not None:
ax.plot(r, y - 0.13, "D", ms=8.5, color=REP, mec=BG, mew=1.5, zorder=3)
lab = (f"reported {r:.2f}, p{S8[k][5]}" if r is not None
else "reported n/a (not sig.)")
ax.text(1.03, y, lab, transform=ax.get_yaxis_transform(), va="center",
fontsize=10, color=INK if r is not None else INK2, clip_on=False)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([LONG[k] for k in ORDER])
ax.set_ylim(len(ORDER) - 0.5, -0.6)
ax.set_xlim(-1.5, 0.25)
ax.set_xticks([-1.5, -1.0, -0.5, 0.0])
ax.set_xlabel("SF2/SRSF1 log2 fold change, TDP-43 vs control (day 2)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(length=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.text(0.03, 0, "no change", rotation=90, fontsize=10, color=INK2,
transform=ax.get_xaxis_transform(), ha="left", va="bottom")
yalg = ys["ALG"]
ax.annotate("abstract: “decreased”\n(astrocytes)",
xy=(RAW["ALG"] - 0.04, yalg + 0.13), xytext=(-1.48, yalg + 0.2),
fontsize=10, weight="bold", va="center",
arrowprops=dict(arrowstyle="-|>", color=INK, lw=1.5, mutation_scale=10))
handles = [Line2D([], [], marker="D", ls="", color=REP, ms=8.5,
label="reported in S8 Table / Results (DESeq2)"),
Line2D([], [], marker="o", ls="", color=REC, ms=9,
label="recomputed: log2(TDP-43 / control), S8 Table group values")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.92),
frameon=False, fontsize=10, ncol=1, handletextpad=0.4)
fig.text(0.015, 0.975, "SF2/SRSF1 change by glial cell type: only PNG and SPG are "
"significant", fontsize=11, weight="bold", va="top")
fig.text(0.015, 0.025, "The recomputed ratio of group values is a consistency check, "
"not a substitute for the DESeq2 test.", fontsize=10, color=INK2)
save(fig, path)
# ================================================================ design D: dumbbell of group MRN
def design_d(path):
fig, ax = plt.subplots(figsize=(W, 3.9), facecolor=BG)
fig.subplots_adjust(left=0.27, right=0.73, top=0.79, bottom=0.215)
ax.set_facecolor(BG)
for i, k in enumerate(ORDER):
nc, c, nt, t, r, p = S8[k]
ax.plot([t, c], [i, i], color=REP, lw=2, alpha=0.55, zorder=2)
ax.plot(c, i, "o", ms=9, mfc=BG, mec=REP, mew=2, zorder=3)
ax.plot(t, i, "o", ms=9, color=REP, mec=REP, zorder=3)
pctc = 100 * (t / c - 1)
sigtxt = f"S8: p{p}" if r is not None else "S8: n/a"
ax.text(1.03, i, f"{pctc:+.0f}% ", transform=ax.get_yaxis_transform(),
va="center", ha="left", fontsize=10, color=REC, weight="bold")
ax.text(1.17, i, sigtxt, transform=ax.get_yaxis_transform(), va="center",
ha="left", fontsize=10, color=INK if r is not None else INK2)
ax.set_yticks(range(len(ORDER)))
ax.set_yticklabels([LONG[k] for k in ORDER])
ax.set_ylim(len(ORDER) - 0.5, -0.6)
ax.set_xlim(0, 1200)
ax.set_xlabel("SF2/SRSF1 expression (MRN), S8 Table group values")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(length=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ia = ORDER.index("ALG")
ax.annotate("abstract: “decreased”", xy=(453, ia), xytext=(40, ia + 0.42),
fontsize=10, weight="bold", va="center",
arrowprops=dict(arrowstyle="-|>", color=INK, lw=1.5, mutation_scale=10))
handles = [Line2D([], [], marker="o", ls="", mfc=BG, mec=REP, mew=2, ms=9,
label="control (reported)"),
Line2D([], [], marker="o", ls="", color=REP, ms=9,
label="+TDP-43 (reported)"),
Line2D([], [], ls="", marker="$\\%$", color=REC, ms=10,
label="change (recomputed)")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.92),
frameon=False, fontsize=10, ncol=3, handletextpad=0.3, columnspacing=1.2)
fig.text(0.015, 0.975, "SF2/SRSF1 expression, control vs TDP-43, by glial cell type",
fontsize=11, weight="bold", va="top")
fig.text(0.015, 0.025, "Axis starts at zero. Controls: n = 2 libraries for CG and ALG, "
"3 otherwise; TDP-43: n = 3.", fontsize=10, color=INK2)
save(fig, path)
# ================================================================ design E: sentence revision
def runs(ax, x, y, parts, fs=10.5, fig_w=W):
"""Draw one line of text runs in axes-fraction coordinates.
parts = [(text, dict(color=..., weight=..., strike=bool, under=bool))]."""
fig = ax.figure
fig.set_dpi(200) # measure at the PNG resolution (Agg glyph advances depend on dpi)
r = fig.canvas.get_renderer()
for txt, st in parts:
wt = st.get("weight", "normal")
t = ax.text(x, y, txt.replace(" ", "\u00a0"), fontsize=fs,
color=st.get("color", INK), weight=wt, va="baseline")
bb = t.get_window_extent(renderer=r)
x1 = x + bb.width / (fig.dpi * fig_w)
if st.get("strike"):
ax.plot([x, x1], [y + 0.010, y + 0.010], color=st.get("color", INK), lw=1.8)
if st.get("under"):
ax.plot([x, x1], [y - 0.014, y - 0.014], color=st.get("color", INK), lw=1.8)
x = x1
return x
def design_e(path):
fig = plt.figure(figsize=(W, 3.8), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
fig.canvas.draw()
ax.text(0.025, 0.95, "The abstract sentence, and what the paper's own results support",
fontsize=11, weight="bold", va="top")
ax.text(0.025, 0.80, "As published (Abstract)", fontsize=10, color=INK2)
runs(ax, 0.025, 0.72, [("“…SF2/SRSF1 levels were found to be decreased in both PNG and in ", {}),
("astrocytes", {"strike": True, "weight": "bold"}),
(".”", {})])
ax.text(0.025, 0.58, "Consistent with Results, Introduction, Discussion and S8 Table",
fontsize=10, color=INK2)
runs(ax, 0.025, 0.50, [("SF2/SRSF1 levels were found to be decreased in both PNG and in ", {}),
("SPG", {"under": True, "weight": "bold"}),
(", not in astrocytes.", {"under": True, "weight": "bold"})])
ax.plot([0.025, 0.975], [0.40, 0.40], color=GRID, lw=1)
ax.text(0.025, 0.375, "S8 Table. Orange: reported MRN per group (control → TDP-43), log2FC and p.\n"
"Blue: recomputed raw log2 ratio of the two group values.", fontsize=10, color=INK2,
va="top")
cols = [0.025, 0.215, 0.405, 0.575, 0.80]
for x, k in zip(cols, ORDER):
nc, c, nt, t, r, p = S8[k]
ax.text(x, 0.245, k + (" (astrocytes)" if k == "ALG" else ""), fontsize=10,
weight="bold", va="top")
ax.text(x, 0.18, f"{c} → {t}", fontsize=10, color=REP, va="top")
ax.text(x, 0.115, (f"{r:.2f}, p{p}" if r is not None else "n/a"), fontsize=10,
color=REP, va="top")
ax.text(x, 0.05, f"raw {RAW[k]:+.2f}", fontsize=10, color=REC, va="top")
save(fig, path)
design_a(os.path.join(CAND, "design_A_two_panels"))
design_b(os.path.join(CAND, "design_B_section_matrix"))
design_c(os.path.join(CAND, "design_C_log2fc_dots"))
design_d(os.path.join(CAND, "design_D_expression_dumbbell"))
design_e(os.path.join(CAND, "design_E_sentence_revision"))
# The write-up uses design A as fig1 and design C as fig2.
design_a(os.path.join(HERE, "fig1"))
design_c(os.path.join(HERE, "fig2"))
print("\nwrote fig1, fig2 and candidates/design_A..E (.svg, .png)")
Abstract says three LIPG variants lower plasma endothelial lipase; Table 6 and Results show increases
PLoS Genet (2011) · doi:10.1371/journal.pgen.1002393 · PMC3234219
Summary
The abstract says that minor alleles of a common LIPG regulatory haplotype and of the noncoding GWAS SNPs "were associated with reduced plasma levels" of endothelial lipase (EL). The paper's own Table 6, the Results and the Discussion all report the opposite for the same three variants: plasma EL goes up with the minor allele (Table 6 up arrows, P = 0.041, 1.43×10⁻³ and 3.48×10⁻³). The genotype means in Table 6 rise with each minor-allele copy, and a per-allele trend recomputed from those means is positive for all three (+45.9, +74.0 and +76.3 ng/mL). The underlying data and the paper's main conclusion are not affected. But the abstract states the direction of a key mechanistic finding backwards, and as written it does not fit the paper's own argument that these alleles lower HDL cholesterol by raising the enzyme that breaks it down.
What the paper reports
Abstract:
Minor alleles of a common regulatory haplotype and noncoding GWAS SNPs were associated with reduced plasma levels of the LIPG gene product endothelial lipase (EL), consistent with its role in HDL-C catabolism.
Results, "Functional analysis of common LIPG regulatory variants", paragraph 3:
Consistent with these findings, the minor alleles of these variants were also associated with increased plasma EL (P<0.002 and P<0.004, respectively) (Table 6). The minor allele of the -1309 A>G variant was moderately associated with increased plasma EL (P<0.05), consistent with its role in decreasing plasma HDL-C, as suggested by the GLGC and FHS association studies.
("These variants" are rs4299883 and rs2156552, named in the previous sentence. The -1309 A>G variant is rs3829632, which Table 6 uses for the combined haplotype.)
Discussion, subsection "Putative haplotype involving a causal regulatory and a nonfunctional coding variant of LIPG is associated with HDL-C levels", paragraph 4 (haplotype) and paragraph 6 (GWAS SNPs):
Minor alleles of the haplotype variants were associated with decreased HDL-C in the FHS and GLGC GWAS studies, and the minor allele of 1 variant was associated with increased plasma EL.
In addition to being associated with decreased HDL-C, the minor alleles of these variants are associated with increased plasma EL in humans.
Table 6, "Association of common LIPG variants with plasma EL concentrations in SIRCA" (column "Effect of minor allele on plasma EL (P value)"; footnote h: "Association of log-transformed mean plasma EL concentration with minor allele for each variant in SIRCA"):
Variant Genotype (N) Mean plasma EL (ng/mL, mean ± SD) Effect of minor allele on plasma EL Combined Haplotype AA (537) / AG (202) / GG (21) 483.5±342.1 / 538.4±368.9 / 540.2±367.9 ↑ (0.041) rs4939883 (C>T) CC (542) / CT (203) / TT (10) 481.8±353.5 / 538.8±342.0 / 772.9±452.8 ↑ (1.43×10-3) rs2156552 (T>A) TT (419) / TA (146) / AA (5) 486.2±362.1 / 545.1±346.5 / 852.1±565.7 ↑ (3.48×10-3) rs34474737 (229 T>G) TT (391) / TG (305) / GG (70) 520.1±338.3 / 505.7±379.3 / 417.6±302.0 ↓ (3.38×10-3) rs2000813 (584 C>T; Thr111Ile) CC (352) / CT (330) / TT (79) 521.2±343.5 / 492.7±362.9 / 417.4±319.9 ↓ (7.00×10-4)
A second, smaller reversal appears in the same Discussion subsection, paragraph 1:
For example, the 229 T>G (rs34474737) variant in the 5′ UTR was found to decrease LIPG promoter activity in vitro and to raise plasma EL in humans.
Compare Results, same subsection, paragraph 4:
The minor allele of the rs344747347 (229 T>G) variant was highly associated with decreased plasma EL (P<0.004), consistent with the luciferase reporter assay results.
What the paper's own numbers show
Method. From the genotype counts, means and SDs in Table 6, I fitted an additive trend, the ordinary least-squares slope of EL on minor-allele count (0, 1, 2), using the group summaries, and computed each genotype's difference from the reference homozygote. The paper's P values come from regression of log-transformed individual EL values, which are not published. So the raw-scale P values below are approximations used only to check direction, and they are not expected to match. To check that the log transform cannot reverse the sign, I also approximated each group's mean log EL under a lognormal assumption (ln mean − ½ ln(1 + CV²)) and took the trend of those. Last, I compared the N stated in the Results with the Table 6 genotype counts. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Direction for the haplotype and GWAS-SNP minor alleles | Abstract | "reduced plasma levels" of EL | Increased for all three (rows below) |
| Combined haplotype (rs3829632 A>G), effect on EL | Table 6 | ↑ (P = 0.041) | +45.9 ng/mL per allele (raw-scale approx. P = 0.059); AG − AA = +54.9 (+11.4%), GG − AA = +56.7 (+11.7%) |
| rs4939883 (C>T), effect on EL | Table 6; Results p3 ("P<0.002") | ↑ (P = 1.43×10⁻³) | +74.0 ng/mL per allele (approx. P = 0.005); CT − CC = +57.0 (+11.8%), TT − CC = +291.1 (+60.4%) |
| rs2156552 (T>A), effect on EL | Table 6; Results p3 ("P<0.004") | ↑ (P = 3.48×10⁻³) | +76.3 ng/mL per allele (approx. P = 0.019); TA − TT = +58.9 (+12.1%), AA − TT = +365.9 (+75.3%) |
| Same three, trend of approximate mean log EL | Table 6 (footnote h: log scale) | ↑ ↑ ↑ | +0.099, +0.180, +0.184 per allele: all up |
| HDL-C per minor allele, same three variants (FHS) | Table 5 | ΔSD −0.15, −0.16, −0.18 | −1.98, −2.11, −2.38 mg/dL (1 SD = 13.2 mg/dL) |
| rs34474737 (229 T>G), effect on EL | Table 6; Results p4 vs Discussion p1 | Table/Results: ↓ (P = 3.38×10⁻³); Discussion: "raise plasma EL" | −36.8 ng/mL per allele; GG − TT = −102.5 (−19.7%); log-scale trend −0.106 |
| rs2000813 (Thr111Ile), effect on EL | Table 6; Results p4 | ↓ (P = 7.0×10⁻⁴) | −43.6 ng/mL per allele; TT − CC = −103.8 (−19.9%) |
| N genotyped for rs34474737 | Results p2 vs Table 6 | n = 761 | 391 + 305 + 70 = 766 |
| N genotyped, the other four variants | Results p2, p4 vs Table 6 | 760, 755, 570, 761 | Table 6 sums match exactly |
"Results pN" is paragraph N of the Results subsection "Functional analysis of common LIPG regulatory variants"; "Discussion p1" is paragraph 1 of the Discussion subsection quoted above.
The Table 6 means, the Table 6 arrows, the Results text and the Discussion agree that these three minor alleles go with higher EL. The abstract is the only place in the paper that says "reduced". The two variants whose minor alleles do lower EL (rs34474737 and rs2000813) are covered in separate sentences of the abstract and are not the ones this sentence names.
Two smaller labeling slips do not change any direction. The Results and Discussion call the GWAS SNP "rs4299883", while Table 6 lists "rs4939883"; the counts match (755), so they are the same data. The Results also write "rs344747347" for rs34474737.
Figure
Figure 1. On the left is the abstract sentence, with its direction word ("reduced plasma levels", orange: the paper's reported claim at issue). On the right is the Results sentence on the same variants ("increased plasma EL") above the Table 6 mean EL for 0, 1 and 2 copies of the minor allele and the Table 6 arrow and P value, all copied from the paper. The lower panel (blue) gives the per-allele slope recomputed from those Table 6 group means. For all three variants the abstract's direction is opposite to the paper's own table, text and recomputed trend. Means are rounded to whole ng/mL for display; the exact values are in the table above.
Figure 2. Table 6 mean plasma EL (orange dots) by genotype, with the number of people in each genotype under the axis, and the least-squares linear trend recomputed from those group summaries (blue line, slope printed at the top of each panel). The first three panels are the variants named in the abstract sentence, which calls their effect "reduced". In all three, EL rises with minor-allele count, as the Table 6 arrows (↑) say. The last two panels, rs34474737 and Thr111Ile, are the variants that do lower EL (↓), shown for contrast; the Discussion describes the first of these as raising EL. The rare homozygote groups are small (5 to 79 people), so their means are imprecise, but the heterozygote means alone already point the same way as the arrows. The y-axis starts at zero. The trend is on the raw scale; the paper's tests are on log EL.
Why it matters
The abstract is the part of the paper most readers and indexers use. Here it presents the EL result as a mechanism: the minor alleles change the amount of the enzyme, which explains their association with HDL-C. Read as written, the mechanism points the wrong way. Lower levels of an enzyme that breaks down HDL would predict higher HDL-C, yet the same paper reports that these minor alleles lower HDL-C (Table 5, about 2 mg/dL per allele in Framingham; GLGC P = 8.64×10⁻¹⁰ to 4.34×10⁻⁴⁹ in Table 6). The abstract's own phrase "consistent with its role in HDL-C catabolism" fits only the increased direction the body reports. A reader citing the abstract for the direction of the LIPG haplotype or of rs4939883/rs2156552 on EL levels would get the sign wrong.
The paper's main conclusion does not depend on this sentence. That conclusion is that rare and common noncoding regulatory variants contribute to the LIPG allelic spectrum, and the abstract states the rs34474737/rs2000813 linkage-disequilibrium argument correctly. So this is a wrong key result in the abstract, not a wrong headline conclusion. The Discussion's "raise plasma EL" for rs34474737 is the same kind of slip, in the opposite direction, about the variant the paper proposes as causal for the Thr111Ile signal.
Explanations considered
- "Reduced" might describe something other than EL. It cannot. The sentence names its object: "reduced plasma levels of the LIPG gene product endothelial lipase (EL)".
- The sentence might mean the variants that do lower EL. It does not. The variants with ↓ in Table 6 are rs34474737 and rs2000813, and the abstract covers them in its next two sentences. This sentence names the "common regulatory haplotype" (Table 6: rs3829632, ↑) and the "noncoding GWAS SNPs" (rs4939883/rs4299883 and rs2156552, ↑).
- The effect might be coded against the reference allele. The Table 6 column header is "Effect of minor allele on plasma EL", and the genotype means rise with minor-allele count (for example 481.8, 538.8, 772.9 ng/mL), so the direction does not depend on the coding convention.
- The log transform might reverse the sign. It cannot plausibly do so here. The approximate log-scale group means rise with minor-allele count for all three variants (trend +0.099 to +0.184 per allele), and the paper's own arrows, computed on the log scale, are ↑.
- The direction might differ between the publisher's version and the PMC copy, for instance through a lost symbol. In the publisher's PDF the arrows are set in a symbol font. Their mapping was checked against the HDL-C column and against rs34474737, whose ↓ matches the text's "decreased plasma EL"; the up arrows for these three rows are the same in the PDF, the publisher HTML, PMC and Europe PMC. The word "reduced" is in the abstract of every version, and the body text says "increased" in every version.
- The abstract might report a different analysis from Table 6. No other EL analysis appears in the paper or its supplement. Table 6 is the only source of EL by genotype, and the Methods describe a single analysis (log-transformed EL, linear regression).
None of these explains the difference. The most economical reading is a wording error in that one abstract sentence: "reduced" where the body says "increased".
Status
As of 2026-10-24:
- Version of record. The publisher HTML (journals.plos.org, reached through the DOI) and the publisher's 13-page PDF carry the same abstract wording ("reduced plasma levels"), the same Results and Discussion wording ("increased plasma EL") and the same Table 6 arrows and means as the PMC copy. Europe PMC's full-text XML agrees.
- Supplementary material. Six items (Figures S1 to S3, Tables S1 to S3) are listed: a promoter activity figure, a rare-variant luciferase figure, an LD plot, carrier characteristics, GLGC HDL-C associations for the haplotype variants, and mutagenesis primers. None reports plasma EL by genotype, so none can bear on the direction.
- Corrections. No correction, erratum, expression of concern or retraction was found. Crossref has no update relations, PMC and Europe PMC list no notices (the Europe PMC article record has no comment/correction list), and the PLOS article page shows "Reader Comments (0)".
- PubPeer. A web search found no PubPeer thread for the title or the DOI. PubPeer's own search refused automated access (HTTP 403, human verification required), so I could not query it directly.
- Sources. The article as archived in PubMed Central (PMC3234219); the publisher HTML and PDF at doi:10.1371/journal.pgen.1002393; Crossref and Europe PMC (MED/22174694) records.
Suggested correction
In the abstract, replace "were associated with reduced plasma levels of the LIPG gene product endothelial lipase (EL)" with "were associated with increased plasma levels of the LIPG gene product endothelial lipase (EL)". That matches Table 6, the Results and the Discussion, and makes the sentence's closing clause ("consistent with its role in HDL-C catabolism") follow. In the Discussion subsection "Putative haplotype involving a causal regulatory and a nonfunctional coding variant of LIPG is associated with HDL-C levels", paragraph 1, replace "and to raise plasma EL in humans" with "and to lower plasma EL in humans". Smaller fixes: in the Results, paragraph 2 of "Functional analysis of common LIPG regulatory variants", the N for rs34474737 should be 766 (391 + 305 + 70 in Table 6), not 761, unless the table counts are the ones in error. The GWAS SNP name should match between the text ("rs4299883") and Table 6 ("rs4939883"). And "rs344747347" in the Results should read rs34474737.
Recomputation script (python recompute.py.txt)
"""Recompute the plasma-EL directions in PMC3234219
(PLoS Genet 2011;7(12):e1002393, doi:10.1371/journal.pgen.1002393).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the article as archived in PubMed Central (identical
wording and arrows in the publisher's HTML and PDF):
Table 6: genotype N, mean +/- SD of pre-heparin plasma EL (ng/mL), and the
arrow and P value for "Effect of minor allele on plasma EL"
Results ("Functional analysis of common LIPG regulatory variants", p1, p3):
the n genotyped per variant
Table 5 (Framingham): Delta SD of HDL per copy of the minor allele
The published P values come from linear regression of log-transformed
individual EL values, which are not available. The raw-scale trend below is
computed from the group summaries only to check the direction; its P values
are approximations and are not expected to match the published ones.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Table 6)
# name: (genotype labels, N, mean, SD, Table 6 arrow, Table 6 P, named in abstract sentence?)
T6 = {
"Combined haplotype\n(rs3829632 A>G)": (("AA", "AG", "GG"), (537, 202, 21),
(483.5, 538.4, 540.2), (342.1, 368.9, 367.9), "up", 0.041, True),
"rs4939883\n(C>T)": (("CC", "CT", "TT"), (542, 203, 10),
(481.8, 538.8, 772.9), (353.5, 342.0, 452.8), "up", 1.43e-3, True),
"rs2156552\n(T>A)": (("TT", "TA", "AA"), (419, 146, 5),
(486.2, 545.1, 852.1), (362.1, 346.5, 565.7), "up", 3.48e-3, True),
"rs34474737\n(229 T>G)": (("TT", "TG", "GG"), (391, 305, 70),
(520.1, 505.7, 417.6), (338.3, 379.3, 302.0), "down", 3.38e-3, False),
"rs2000813\n(Thr111Ile)": (("CC", "CT", "TT"), (352, 330, 79),
(521.2, 492.7, 417.4), (343.5, 362.9, 319.9), "down", 7.0e-4, False),
}
# Results p1 and p3: n stated in the text
TEXT_N = {"Combined haplotype\n(rs3829632 A>G)": 760, "rs4939883\n(C>T)": 755,
"rs2156552\n(T>A)": 570, "rs34474737\n(229 T>G)": 761, "rs2000813\n(Thr111Ile)": 761}
# Table 5 (Framingham Heart Study): Delta SD of HDL per minor allele
T5_HDL = {"-1309 A>G (rs3829632)": (-0.15, 0.0002), "rs4939883": (-0.16, 2.28e-7),
"rs2156552": (-0.18, 1.08e-8)}
def trend(n, m, s):
"""OLS of EL on minor-allele count (0/1/2) from group N, mean, SD."""
n, m, s = (np.asarray(v, float) for v in (n, m, s))
g = np.array([0., 1., 2.])
N = n.sum()
gbar = (n * g).sum() / N
ybar = (n * m).sum() / N
sxx = (n * (g - gbar) ** 2).sum()
sxy = (n * (g - gbar) * (m - ybar)).sum()
b = sxy / sxx
sst = ((n - 1) * s ** 2).sum() + (n * (m - ybar) ** 2).sum()
sse = sst - b * sxy
se = np.sqrt(sse / (N - 2) / sxx)
p = 2 * stats.t.sf(abs(b / se), N - 2)
a = ybar - b * gbar
return int(N), a, b, p
def lognormal_log_mean(m, s):
"""Mean of log(EL) in a group if EL were lognormal with mean m and SD s."""
m, s = np.asarray(m, float), np.asarray(s, float)
return np.log(m) - 0.5 * np.log1p((s / m) ** 2)
def short(k):
return k.replace("\n", " ")
print("== Table 6: raw-scale additive trend per minor allele (from group summaries) ==")
print(f"{'variant':36s} {'N':>4s} {'slope ng/mL':>11s} {'dir':>5s} {'approx P':>9s}"
f" {'T6 arrow':>8s} {'T6 P':>9s}")
FIT = {}
for k, (lab, n, m, s, arrow, P, _) in T6.items():
N, a, b, p = trend(n, m, s)
FIT[k] = (a, b)
print(f"{short(k):36s} {N:4d} {b:+11.1f} {'up' if b > 0 else 'down':>5s} {p:9.4f}"
f" {arrow:>8s} {P:9.2e}")
print("\n== Table 6: genotype means relative to the reference homozygote ==")
for k, (lab, n, m, s, arrow, P, _) in T6.items():
d1, d2 = m[1] - m[0], m[2] - m[0]
print(f"{short(k):36s} {lab[1]}-{lab[0]} {d1:+7.1f} ({100 * d1 / m[0]:+5.1f}%) "
f"{lab[2]}-{lab[0]} {d2:+7.1f} ({100 * d2 / m[0]:+5.1f}%)")
print("\n== Direction on the log scale (lognormal approximation from mean and SD) ==")
for k, (lab, n, m, s, arrow, P, _) in T6.items():
lm = lognormal_log_mean(m, s)
_, _, bl, _ = trend(n, lm, np.zeros(3)) # slope of group log-means only
print(f"{short(k):36s} log-means {np.round(lm, 3).tolist()} slope {bl:+.3f}"
f" dir {'up' if bl > 0 else 'down'} (Table 6: {arrow})")
print("\n== Variants named in the abstract sentence ==")
for k, (lab, n, m, s, arrow, P, named) in T6.items():
if named:
print(f"{short(k):36s} abstract: reduced EL | Table 6: {arrow} (P={P:g}) |"
f" recomputed slope {FIT[k][1]:+.1f} ng/mL per allele")
print("\n== Table 5 (FHS): HDL per minor allele, same variants ==")
for k, (d, p) in T5_HDL.items():
print(f"{k:24s} Delta SD {d:+.2f} (P={p:g}) -> {d * 13.2:+.2f} mg/dL (1 SD = 13.2 mg/dL)")
print("\n== N genotyped: Results text vs Table 6 sum ==")
for k, (lab, n, *_r) in T6.items():
print(f"{short(k):36s} text {TEXT_N[k]:4d} Table 6 {' + '.join(map(str, n))} = {sum(n):4d}"
f" {'match' if TEXT_N[k] == sum(n) else 'MISMATCH'}")
# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, NEUTRAL = "#eb6834", "#2a78d6", "#b9b7b1"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "path", "lines.linewidth": 2,
})
W = 7.2
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def panel(ax, x, y, w, h, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=edge, lw=2, transform=ax.transAxes))
# ---------------------------------------------------------------- figure 1: abstract vs body
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
# left: abstract
panel(ax, 0.01, 0.30, 0.34, 0.68, ORANGE)
ax.text(0.03, 0.95, "Abstract", fontsize=11, fontweight="bold", va="top")
ax.text(0.03, 0.885, "“Minor alleles of a common\nregulatory haplotype and\nnoncoding GWAS SNPs were\n"
"associated with", fontsize=10, va="top", linespacing=1.45)
ax.text(0.03, 0.70, "reduced plasma levels", fontsize=10.5, va="top", color=ORANGE,
fontweight="bold")
ax.text(0.03, 0.64, "of the LIPG gene product\nendothelial lipase (EL),\nconsistent with its role in\n"
"HDL-C catabolism.”", fontsize=10, va="top", linespacing=1.45)
# right: Results p2 + Table 6
panel(ax, 0.46, 0.30, 0.53, 0.68, INK2)
ax.text(0.48, 0.95, "Results (p3) and Table 6", fontsize=11, fontweight="bold", va="top")
ax.text(0.48, 0.885, "“the minor alleles of these variants were\nalso associated with", fontsize=10,
va="top", linespacing=1.45)
ax.text(0.48, 0.80, "increased plasma EL", fontsize=10.5, va="top", fontweight="bold", color=INK)
ax.text(0.48, 0.745, "(P<0.002 and P<0.004, respectively)”", fontsize=10, va="top")
ax.text(0.48, 0.665, "Table 6, mean EL (ng/mL) by minor-allele copies", fontsize=10, va="top",
color=INK2)
ax.text(0.48, 0.615, f"{'':11s}{'0':>6s}{'1':>6s}{'2':>6s} arrow (P)", fontsize=9.6, va="top",
family="DejaVu Sans Mono", color=INK2)
yy = 0.56
for k, (lab, n, m, s, arrow, P, named) in T6.items():
if not named:
continue
nm = k.split("\n")[0].replace("Combined haplotype", "Haplotype")
ax.text(0.48, yy, f"{nm:11s}{m[0]:6.0f}{m[1]:6.0f}{m[2]:6.0f} ↑ ({P:g})",
fontsize=9.6, va="top", family="DejaVu Sans Mono")
yy -= 0.055
ax.text(0.48, 0.375, "Discussion (p6): “associated with increased\nplasma EL in humans”",
fontsize=9.6, va="top", color=INK2, linespacing=1.4)
# connector
ax.annotate("", xy=(0.46, 0.64), xytext=(0.35, 0.64),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=2))
ax.text(0.405, 0.70, "opposite\ndirection", ha="center", va="bottom", fontsize=10.5,
fontweight="bold", linespacing=1.3)
ax.text(0.405, 0.60, "same\nvariants", ha="center", va="top", fontsize=10, color=INK2,
linespacing=1.3)
# bottom: recomputed slopes
panel(ax, 0.01, 0.02, 0.98, 0.23, BLUE)
ax.text(0.03, 0.22, "Recomputed from Table 6 group means (raw scale, per minor allele)",
fontsize=10.5, fontweight="bold", va="top", color=BLUE)
xs = 0.03
for k, (lab, n, m, s, arrow, P, named) in T6.items():
if not named:
continue
nm = k.split("\n")[0].replace("Combined haplotype", "Haplotype")
ax.text(xs, 0.13, f"{nm}", fontsize=10, va="top", color=INK2)
ax.text(xs, 0.08, f"{FIT[k][1]:+.1f} ng/mL", fontsize=10.5, va="top", color=BLUE,
fontweight="bold")
xs += 0.32
save(fig, "fig1")
# ---------------------------------------------------------------- figure 2: genotype means
keys = list(T6)
fig, axs = plt.subplots(1, 5, figsize=(W, 3.5), sharey=True)
for i, (ax, k) in enumerate(zip(axs, keys)):
lab, n, m, s, arrow, P, named = T6[k]
g = np.array([0, 1, 2])
a, b = FIT[k]
ax.set_facecolor(BG)
ax.grid(axis="y", color=GRID, lw=0.8, zorder=0)
ax.plot(g, a + b * g, color=BLUE, lw=2, zorder=2,
label="Recomputed linear trend" if i == 0 else None)
ax.plot(g, m, "o", ms=8, color=ORANGE, mec="white", mew=0.8, zorder=3,
label="Table 6 genotype mean" if i == 0 else None)
ax.set_xticks(g)
ax.set_xticklabels([f"{lab[j]}\n{n[j]}" for j in range(3)], fontsize=10)
ax.set_xlim(-0.5, 2.5)
ax.set_ylim(0, 1000)
ax.set_title(k, fontsize=10, color=INK, pad=6)
ax.text(1.0, 960, f"{b:+.0f}/allele", ha="center", va="top", fontsize=10, color=BLUE)
ax.text(1.0, 80, f"Table 6: {'↑' if arrow == 'up' else '↓'}", ha="center",
va="bottom", fontsize=10, color=ORANGE)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
if named:
ax.spines["bottom"].set_color(INK)
axs[0].set_ylabel("Mean plasma EL (ng/mL)")
fig.text(0.37, -0.08, "In the abstract sentence (stated as ‘reduced’)", ha="center", fontsize=10,
color=INK2)
fig.text(0.83, -0.08, "Not in that sentence", ha="center", fontsize=10, color=INK2)
fig.text(0.545, 0.02, "genotype and N (0, 1, 2 copies of the minor allele)", ha="center",
fontsize=10, color=INK2)
fig.legend(loc="upper center", bbox_to_anchor=(0.5, 1.07), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(bottom=0.22, wspace=0.12, left=0.1, right=0.99, top=0.84)
save(fig, "fig2")
print("\nwrote fig1.svg/png, fig2.svg/png in", HERE)
Table 2 odds ratios for haplogroup B4 and other zero-cell rows cannot come from the counts
PLoS Genet (2013) · doi:10.1371/journal.pgen.1003696 · PMC3744426
Summary
The paper reports that mitochondrial haplogroup B4 is less common in biliary atresia (BA) than in controls, with an odds ratio (OR) of 0.71 (95% CI 0.66–0.77), and an age- and sex-adjusted OR of 0.82 (0.71–0.85) in the Abstract. Its own Table 2 shows that none of the 71 BA patients and 23 of the 200 controls carry B4. With an empty cell the sample odds ratio is 0. The exact 95% CI is 0 to 0.45, and adding 0.5 to each cell gives 0.053 (0.003–0.881). The printed 0.71 (0.66–0.77) instead equals, to two decimals, a different statistic: the proportion of B4-negative participants who are controls divided by the proportion of B4-positive participants who are controls, 0.714 (0.660–0.772). Three other zero-cell rows of Table 2 (M8&Z, Y and D) show the same pattern. The direction of the B4 association and its p-value (Fisher exact p = 0.0009) stand, but the effect size and its precision are misreported for the paper's headline genetic finding, and the adjusted estimate cannot be finite for these counts.
What the paper reports
Results, "Association of BA with specific mitochondrial haplogroups", second paragraph:
Evaluation of haplogroup distributions revealed that the haplogroup B4 was less prevalent in patients with BA than in the control group (odds ratio [OR], 0.71; 95% confidence interval [CI], 0.66–0.77; p = 0.001).
The same paragraph, adjusted analysis:
Multivariate logistic-regression analysis of haplogroups associated with BA with adjustment for age and sex, showing the risk for haplogroup B4 was p = 0.007, OR 0.82, 95%CI 0.71–0.85
Abstract:
The haplogroup B4 and E prevalence were significantly lower and higher respectively, in the patients with BA than in the controls (odds ratios, 0.82 [p = 0.007] and 7.36 [p = 0.032] respectively) in multivariate logistic-regression analysis.
Table 2, "Association between biliary atresia (BA) and the mitochondrial haplogroups". The columns are BA positive | BA negative | control positive | control negative | OR | 95% CI | p:
B4 | 0 | 71 | 23 | 177 | 0.71 | 0.66–0.77 | 0.001*
M8&Z | 0 | 71 | 5 | 195 | 1.36 | 1.27–1.47 | 0.331
Y | 0 | 71 | 1 | 199 | 1.36 | 1.26–1.46 | 1.000
D | 2 | 69 | 0 | 200 | 3.90 | 3.18–4.78 | 0.068
Z | 1 | 70 | 1 | 199 | 2.84 | 0.18–46.06 | 0.456
C | 1 | 70 | 1 | 199 | 2.84 | 0.18–46.07 | 1.456
Table 1, "Descriptive data of studied population". The columns are BA | control (children) | control (adult):
haplogroup B4 | 0 | 5 | 18
Materials and Methods, "Subjects", and "Statistical analysis":
The genetic background was similar in both control groups and was put together for statistical calculation.
Frequencies of all mitochondrial haplogroups in the BA patients and controls were tested for independency using Pearson chi-square statistics and Fisher's exact test as appropriate.
The statistical analyses were performed using the Statistical Package for Social Science (SPSS, version 12) software package.
The conclusions that rest on the B4 result. Author Summary:
The prevalence of haplogroup B4 and E was significantly lower and higher respectively, in the patients with BA than in the controls.
Discussion, first paragraph:
We found that whereas the haplogroup B4 was significantly less likely to occur in the patients with BA, the haplogroup E was more likely to occur in the patients with BA than in the healthy controls.
Conclusion:
The use of the cybrid models helped to establish the association of haplogroups B4 and E with BA
What the paper's own numbers show
Method. I first checked the B4 counts against Table 1: 5 child and 18 adult controls carry B4, so 23 of the 200 pooled controls, and 0 of 71 BA patients do. I then recomputed every row of Table 2 from its four counts: the sample odds ratio with a Woolf (log-Wald) CI, the odds ratio with 0.5 added to each cell (Haldane–Anscombe; for a single binary exposure this equals the Firth-penalised logistic estimate, which the script confirms), the exact conditional odds ratio and CI for B4, Fisher's exact test and Pearson's chi-square test. For the rows that did not reproduce, I tried simple ratios of proportions (with Katz log CIs) to find a statistic that does match the printed values. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| B4 carriers, BA vs controls | Table 1, Table 2 | 0/71 vs 23/200 (5 + 18) | consistent: 23 = 5 + 18; 177 = 200 − 23 |
| B4 odds ratio | Table 2; Results | 0.71 | sample OR (0 × 177)/(71 × 23) = 0; exact conditional MLE 0 |
| B4 95% CI | Table 2; Results | 0.66–0.77 | exact conditional CI 0–0.45; with 0.5 added to each cell, OR 0.053 (0.003–0.881) |
| B4 CI width on the log scale | Table 2 | 0.154 | 5.63 for the 0.5-corrected OR (36 times wider) |
| Statistic that does match 0.71 (0.66–0.77) | Table 2 | — | P(control | B4-negative) / P(control | B4-positive) = (177/248)/(23/23) = 0.714 (0.660–0.772) |
| B4 p | Table 2; Results | 0.001 | Fisher exact two-sided 0.0009 (matches); Pearson 0.0028; Yates 0.0062 |
| B4 adjusted OR | Abstract; Results | 0.82 (0.71–0.85), p = 0.007 | no finite maximum-likelihood estimate: all 23 B4 carriers are controls, so the B4 coefficient is quasi-completely separated with or without age and sex in the model. The printed CI is also off-centre on the log scale (geometric midpoint 0.777, not 0.82) |
| M8&Z | Table 2 | OR 1.36 (1.27–1.47) | sample OR 0 (0/71 vs 5/200); 0.5-corrected 0.249 (0.014–4.552). Matching statistic: P(control | positive)/P(control | negative) = 1.364 (1.269–1.467) |
| Y | Table 2 | OR 1.36 (1.26–1.46) | sample OR 0 (0/71 vs 1/200); 0.5-corrected 0.930 (0.037–23.092). Matching statistic: P(control | positive)/P(control | negative) = 1.357 (1.263–1.457) |
| D | Table 2 | OR 3.90 (3.18–4.78), p = 0.068 | sample OR infinite (2/71 vs 0/200); 0.5-corrected 14.424 (0.684–304.150). Matching statistic: P(BA | positive)/P(BA | negative) = 3.899 (3.181–4.778). The printed CI excludes 1 while the printed p (Fisher 0.0679) does not reach 0.05 |
| C p | Table 2 | 1.456 | Fisher 0.456; row Z, with identical counts, prints 0.456. A probability cannot exceed 1 |
| The 14 rows without a zero cell | Table 2 | ORs and CIs | all 14 ORs reproduce to two decimals and the CIs to within rounding, e.g. E 11.88 (1.30–108.16), Fisher p 0.0177 vs 0.018 |
| E adjusted OR (side note) | Abstract; Results | 7.36 (1.22–94.6) | CI geometric midpoint 10.74, not 7.36; cannot be checked further without individual data |
The ratios of proportions that match the zero-cell rows are not oriented the same way: for B4 the B4-negative group is the numerator, whereas for M8&Z, Y and D the carrier group is the numerator. In the opposite orientation M8&Z would give 0.733 (0.682–0.788). For the two rows whose p-values differ from Fisher's test, F (0.324) and D5 (0.372), the printed values equal Pearson's chi-square p (0.3240 and 0.3723), which the Methods allow ("as appropriate"), so I do not count them as errors.
Figure
Figure 1. Haplogroup B4, 0 of 71 BA patients vs 23 of 200 controls, on one logarithmic ratio axis with the null value 1 marked by a thin vertical line. Orange marks the values the paper prints: the "OR" 0.71 (0.66–0.77) of Table 2 and the Results, and the adjusted "OR" 0.82 (0.71–0.85) of the Abstract and Results. Blue marks values recomputed from the Table 2 counts. The sample odds ratio is 0, which a log axis cannot show, so its exact 95% CI is drawn as an arrow from 0.45 towards zero. The odds ratio with 0.5 added to each cell is 0.053, with a CI (0.003–0.881) spanning more than two orders of magnitude. The bottom row is the ratio of the proportion of controls among B4-negative participants to that among B4-positive participants, 0.714 (0.660–0.772), which reproduces the printed 0.71 (0.66–0.77). The printed intervals are narrow enough to sit almost inside their markers. All values come from Table 2 or from recompute.py.txt.
Figure 2. The four rows of Table 2 with an empty cell (B4, M8&Z, Y and D), on one logarithmic axis with the null value 1 marked. For each row, the top orange marker is the printed "OR" with its 95% CI, the blue diamond is the odds ratio with 0.5 added to each cell with its Woolf CI, and the hollow blue square is the ratio of proportions that matches the printed value. In every row the printed value and interval coincide with the ratio of proportions, not with any odds ratio, and the printed intervals are one to two orders of magnitude narrower than the counts allow. For M8&Z and Y the printed value lies above 1 although no BA patient carries the haplogroup.
Why it matters
B4 is the paper's headline genetic result. It appears in the title, the Abstract, the Author Summary, the Discussion and the Conclusion, and it is the reason the in vitro work compares a B4 cybrid with an E cybrid. The printed crude OR of 0.71 describes a 29% reduction in odds with a tight interval (0.66–0.77). The counts show something different in both size and certainty: no BA patient carried B4, so the data are compatible with anything from no carriers at all to an odds ratio of about 0.45 (exact CI), or 0.003 to 0.88 with a 0.5 correction. A reader using 0.71 (0.66–0.77) in a meta-analysis or a power calculation would carry over an effect measure of the wrong kind with a standard error about 36 times too small on the log scale.
The adjusted estimate in the Abstract, OR 0.82 (0.71–0.85), p = 0.007, is affected more seriously. With no B4 carrier among the cases, the logistic-regression likelihood keeps increasing as the B4 odds ratio goes to 0 (the script prints the profile log-likelihood at 0.1, 0.01 and 0.001), and adding age and sex as covariates cannot change that. Standard software then reports either no estimate or a very large negative coefficient with a very large standard error, not a finite OR with a narrow interval. Table 1 also shows that the BA patients are aged 0.1–11 years and the adult controls 18–60 years, so age alone separates the cases from three-quarters of the controls; any age-adjusted model of these data is fragile for that reason too.
What survives: the direction of the B4 association and its crude significance (Fisher exact p = 0.0009, matching the printed 0.001). The E association (crude OR 11.88) recomputes correctly, as do the other 13 rows without a zero cell. The cybrid experiments do not depend on Table 2's effect sizes. The finding is therefore a misreported key effect estimate and an adjusted estimate that cannot exist as stated, not an overturned conclusion. The errors in the M8&Z, Y and D rows matter less on their own, but the M8&Z and Y values point the wrong way (above 1 with no exposed cases), and the D interval contradicts its own p-value.
Explanations considered
- A different control group. If B4 had been compared against only one control group, the counts would differ. Table 1 gives 5 + 18 = 23 carriers among the 200 pooled controls, matching Table 2, and the Methods say the two control groups were "put together for statistical calculation". No subset of the controls gives an OR near 0.71 with 0 exposed cases, since any odds ratio with 0 exposed cases is 0.
- A zero-cell correction or an exact method. Adding 0.5 to each cell gives 0.053 (0.003–0.881); the exact conditional estimate is 0 with CI 0–0.45; the Firth estimate equals the 0.5-corrected one. None comes near 0.71, and none can give a CI as narrow as 0.66–0.77 from a table with an empty cell.
- A text-conversion or typesetting error. The publisher's PDF of the version of record prints the same Table 2 values as PMC and Europe PMC, cell for cell, and the Results text repeats 0.71 (0.66–0.77) independently of the table. The values are not a conversion artefact.
- A copying slip from the neighbouring row F, whose correct OR is also 0.71. Row F's CI is 0.36–1.41, so this would explain the B4 point estimate but not its interval. The ratio of proportions reproduces both the point estimate and both CI limits for B4, and it also reproduces M8&Z, Y and D, so it accounts for all four rows where a copying slip accounts for one number.
- A ratio of proportions reported under the heading OR. This fits every zero-cell row exactly. SPSS Crosstabs, the package named in the Methods, prints such ratios as "For cohort" risk estimates next to the odds ratio. This explains the numbers but does not make them odds ratios: they answer a different question (the relative probability of being a control, or a case, by carrier status), their orientation differs between rows, and they are not the measure the text and table headings name.
- The adjusted model used individual data that differ from Table 2. I cannot rule this out without the data. But Tables 1 and 2 agree that no BA patient carries B4, and with that the adjusted B4 odds ratio has no finite maximum-likelihood value whatever the other covariates are.
- Row C's p-value. A p-value of 1.456 is not a probability; row Z, with the same counts, prints 0.456, which matches Fisher's test. This reads as a typographical error and does not change any conclusion.
Status
- Version of record. Table 2 in the publisher's PDF (journals.plos.org, 11 pages, Table 2 on page 3) matches the PMC and Europe PMC full text for every row cited here. The PLOS HTML Abstract and Results give the same B4 and E numbers.
- Supplements. The article has two supplements: Figure S1 (characterisation of the ρ0 cells and the B4 and E cybrids) and Table S1 (full mitochondrial sequences of haplogroups B4 and E). Neither holds case-control counts or regression output, and no individual-level data are available.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI; PMC and Europe PMC list no correction or comment links (PMID 23966875); the PLOS article page shows no correction notice and no reader comments. The PubPeer API refused automated access (HTTP 403), so PubPeer could not be checked directly; a web search for the article with "correction", "erratum" and "PubPeer" found no thread or notice for this paper.
- Sources. PMC article PMC3744426; publisher page and PDF at journals.plos.org/plosgenetics (doi:10.1371/journal.pgen.1003696); Europe PMC record and full-text XML; Crossref; PubPeer (blocked); web search.
Suggested correction
- Replace the B4 effect estimate in Table 2 and the Results with an estimate that suits a zero cell, for example the exact conditional OR (0; 95% CI 0–0.45) or a 0.5-corrected or Firth OR (0.053; 95% CI 0.003–0.88), and state the method. Alternatively, report the carrier proportions (0/71 vs 23/200) with Fisher's exact p = 0.0009.
- Withdraw or replace the adjusted B4 estimate in the Abstract and Results (OR 0.82, 0.71–0.85, p = 0.007). If an adjusted analysis is kept, use a method that handles separation (Firth-penalised or exact logistic regression) and report its estimate and interval. Recheck the adjusted E estimate (7.36, 1.22–94.6), whose interval is not centred on it on the log scale.
- Correct the OR and CI columns of Table 2 for rows M8&Z, Y and D in the same way, and correct the p-value for row C from 1.456 to 0.456.
- State in the Methods which test produced each p-value in Table 2 (Fisher's exact test or Pearson's chi-square).
Recomputation script (python recompute.py.txt)
"""Recompute the Table 2 effect estimates of PMC3744426
(PLoS Genet 2013, doi:10.1371/journal.pgen.1003696).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png, fig2.svg/png and five candidate figure designs (candidates/)
next to this file.
Inputs are copied from the article as archived in PMC (identical to the publisher PDF):
Table 1: B4 carriers by group (BA 0; child controls 5; adult controls 18)
Table 2: BA positive / BA negative / control positive / control negative, OR, 95% CI, p
Results text and Abstract: adjusted (age, sex) ORs for B4 and E
"""
import os
import numpy as np
from scipy.stats import chi2_contingency, fisher_exact
from scipy.stats.contingency import odds_ratio
from scipy.special import expit
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = 1.959964
# ------------------------------------------------------------------ inputs (Table 2)
# name: (BA+, BA-, Ctl+, Ctl-, reported OR, reported CI low, high, reported p)
T2 = {
"M10": (2, 69, 4, 196, 1.42, 0.25, 7.93, 0.654),
"M7": (11, 60, 32, 168, 0.96, 0.45, 2.03, 1.000),
"M8": (5, 66, 10, 190, 1.44, 0.48, 4.37, 0.549),
"E": (4, 67, 1, 199, 11.88, 1.31, 108.16, 0.018),
"G": (1, 70, 10, 190, 0.27, 0.03, 2.16, 0.298),
"M9": (4, 67, 4, 196, 2.93, 0.71, 12.02, 0.213),
"M8&Z": (0, 71, 5, 195, 1.36, 1.27, 1.47, 0.331),
"Z": (1, 70, 1, 199, 2.84, 0.18, 46.06, 0.456),
"C": (1, 70, 1, 199, 2.84, 0.18, 46.07, 1.456),
"B4": (0, 71, 23, 177, 0.71, 0.66, 0.77, 0.001),
"B5": (2, 69, 14, 186, 0.39, 0.09, 1.74, 0.253),
"F": (13, 58, 48, 152, 0.71, 0.36, 1.41, 0.324),
"A": (4, 67, 13, 187, 0.86, 0.27, 2.73, 1.000),
"Y": (0, 71, 1, 199, 1.36, 1.26, 1.46, 1.000),
"D": (2, 69, 0, 200, 3.90, 3.18, 4.78, 0.068),
"D4": (6, 65, 24, 176, 0.68, 0.27, 1.73, 0.512),
"D5": (4, 67, 18, 182, 0.60, 0.20, 1.85, 0.372),
"N9": (2, 69, 7, 193, 0.80, 0.16, 3.94, 1.000),
}
TABLE1_B4 = {"BA": 0, "control children": 5, "control adults": 18}
N_BA, N_CTL = 71, 200
ADJ = {"B4": (0.82, 0.71, 0.85, 0.007), "E": (7.36, 1.22, 94.6, 0.032)} # Abstract / Results
# ------------------------------------------------------------------ helpers
def woolf(a, b, c, d):
"""Sample OR ad/bc with Woolf (log-Wald) CI; None if a cell is zero."""
if 0 in (a, b, c, d):
return None
o = a * d / (b * c)
s = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
return o, o * np.exp(-Z * s), o * np.exp(Z * s)
def haldane(a, b, c, d):
return woolf(a + .5, b + .5, c + .5, d + .5)
def ratio_of_proportions(x1, n1, x2, n2):
"""(x1/n1)/(x2/n2) with Katz log CI."""
r = (x1 / n1) / (x2 / n2)
s = np.sqrt(1 / x1 - 1 / n1 + 1 / x2 - 1 / n2)
return r, r * np.exp(-Z * s), r * np.exp(Z * s)
def firth_or(a, b, c, d, iters=200):
"""Firth-penalised logistic regression of BA on carrier status (individual rows)."""
x = np.r_[np.ones(a), np.zeros(b), np.ones(c), np.zeros(d)]
y = np.r_[np.ones(a + b), np.zeros(c + d)]
X = np.c_[np.ones_like(x), x]
beta = np.zeros(2)
for _ in range(iters):
p = expit(X @ beta)
W = p * (1 - p)
info = X.T @ (X * W[:, None])
inv = np.linalg.inv(info)
h = W * np.einsum("ij,jk,ik->i", X, inv, X)
step = inv @ (X.T @ (y - p + h * (0.5 - p)))
beta = beta + step
if np.max(np.abs(step)) < 1e-12:
break
return float(np.exp(beta[1]))
def fmt(t, k=2):
return f"{t[0]:.{k}f} ({t[1]:.{k}f}–{t[2]:.{k}f})"
# ------------------------------------------------------------------ 1. B4 consistency with Table 1
print("== 1. B4 counts")
print("Table 1 B4 carriers:", TABLE1_B4, "-> controls", 5 + 18, "of", N_CTL,
"; BA", 0, "of", N_BA, "; B4-negative controls", N_CTL - 23)
# ------------------------------------------------------------------ 2. B4 odds ratio, recomputed
a, b, c, d = T2["B4"][:4]
print("\n== 2. B4 (BA 0/71 vs controls 23/200)")
print("sample OR (0*177)/(71*23) =", a * d / (b * c))
cond = odds_ratio([[a, b], [c, d]])
ci = cond.confidence_interval()
print(f"exact conditional MLE OR = {cond.statistic:.3f}, exact 95% CI {ci.low:.3f}–{ci.high:.3f}")
B4_HAL = haldane(a, b, c, d)
print("Haldane-Anscombe (+0.5) OR, Woolf CI =", fmt(B4_HAL, 3))
print(f"Firth-penalised logistic OR (no covariates) = {firth_or(a, b, c, d):.4f} (equals the +0.5 OR)")
print(f"Fisher exact two-sided p = {fisher_exact([[a, b], [c, d]])[1]:.4f} (reported 0.001)")
print(f"Pearson chi-square p (no correction) = {chi2_contingency([[a, b], [c, d]], correction=False)[1]:.4f}; "
f"Yates p = {chi2_contingency([[a, b], [c, d]], correction=True)[1]:.4f}")
B4_RR = ratio_of_proportions(d, b + d, c, a + c) # P(control | B4-neg) / P(control | B4-pos)
print("P(control | B4-negative) / P(control | B4-positive) = (177/248)/(23/23) =", fmt(B4_RR, 3),
" <- reported 'OR' 0.71 (0.66–0.77)")
print("width of reported CI on log scale:", round(np.log(0.77 / 0.66), 3),
"; width of +0.5 OR CI:", round(np.log(B4_HAL[2] / B4_HAL[1]), 3),
"; ratio:", round(np.log(B4_HAL[2] / B4_HAL[1]) / np.log(0.77 / 0.66), 1))
print(f"+0.5 OR to 4 dp = {B4_HAL[0]:.4f}")
# ------------------------------------------------------------------ 3. adjusted (age, sex) estimates
print("\n== 3. adjusted estimates (Abstract, Results)")
for k, (o, lo, hi, p) in ADJ.items():
print(f"{k}: reported OR {o} ({lo}–{hi}); geometric centre of CI = {np.sqrt(lo * hi):.3f}")
print("B4: every B4-positive participant is a control, so the B4 coefficient in any logistic model "
"of BA is quasi-completely separated; the likelihood keeps rising as OR -> 0, whatever "
"covariates are added. Profile of the log-likelihood (no covariates) for OR = 0.1, 0.01, 0.001:")
for orv in (0.1, 0.01, 0.001):
# intercept for B4-negative fixed at its MLE log(71/177); B4-positive group: 23 controls, 0 cases
p_pos = expit(np.log(71 / 177) + np.log(orv))
ll = 71 * np.log(71 / 248) + 177 * np.log(177 / 248) + 23 * np.log(1 - p_pos)
print(f" OR={orv:<6} loglik={ll:.4f}")
print("Table 1 age ranges: BA 0.1–11 y; child controls 0.3–16 y; adult controls 18–60 y")
print("adult controls (all older than every BA patient):", 148, "of", N_CTL, "=", f"{148 / N_CTL:.0%}")
# ------------------------------------------------------------------ 4. every Table 2 row
print("\n== 4. all Table 2 rows: reported vs recomputed")
print(f"{'row':5s} {'reported OR (CI)':22s} {'Woolf OR (CI)':22s} {'+0.5 OR (CI)':24s} "
f"{'rep p':>6s} {'Fisher':>7s} {'Pearson':>7s}")
REC = {}
for k, (a, b, c, d, ro, rl, rh, rp) in T2.items():
w = woolf(a, b, c, d)
h = haldane(a, b, c, d)
pf = fisher_exact([[a, b], [c, d]])[1]
pp = chi2_contingency([[a, b], [c, d]], correction=False)[1] if min(a, b, c, d) >= 0 else np.nan
REC[k] = dict(woolf=w, hal=h, fisher=pf, pearson=pp)
ws = fmt(w) if w else "0 or inf (zero cell)"
print(f"{k:5s} {ro:.2f} ({rl:.2f}–{rh:.2f}){'':5s} {ws:22s} {fmt(h, 3):24s} "
f"{rp:6.3f} {pf:7.4f} {pp:7.4f}")
print("\nzero-cell rows: the printed value equals a ratio of proportions")
ZERO = {
"B4": ("P(control | negative) / P(control | positive)", ratio_of_proportions(177, 248, 23, 23)),
"M8&Z": ("P(control | positive) / P(control | negative)", ratio_of_proportions(5, 5, 195, 266)),
"Y": ("P(control | positive) / P(control | negative)", ratio_of_proportions(1, 1, 199, 270)),
"D": ("P(BA | positive) / P(BA | negative)", ratio_of_proportions(2, 2, 69, 269)),
}
for k, (lab, r) in ZERO.items():
ro, rl, rh = T2[k][4:7]
print(f"{k:5s} reported {ro:.2f} ({rl:.2f}–{rh:.2f}) {lab} = {fmt(r, 3)} "
f"+0.5 OR = {fmt(REC[k]['hal'], 3)}")
print("M8&Z opposite orientation: P(control|neg)/P(control|pos) =",
fmt(ratio_of_proportions(195, 266, 5, 5), 3))
print("D: reported CI 3.18–4.78 excludes 1, reported p = 0.068; Fisher p =", round(REC["D"]["fisher"], 4))
print("C and Z have identical counts (1/70 vs 1/199): reported p 1.456 and 0.456; Fisher p =",
round(REC["C"]["fisher"], 4))
print("F and D5 reported p match Pearson chi-square without correction:",
round(REC["F"]["pearson"], 4), round(REC["D5"]["pearson"], 4))
print("E: Woolf OR", fmt(REC["E"]["woolf"]), "Fisher p", round(REC["E"]["fisher"], 4))
print("F row OR (13*152)/(58*48) =", round(13 * 152 / (58 * 48), 3), "with CI", fmt(REC["F"]["woolf"]))
nonzero_ok = [k for k in T2 if REC[k]["woolf"] and abs(REC[k]["woolf"][0] - T2[k][4]) < 0.006]
print("rows without a zero cell whose OR reproduces to 2 dp:", len(nonzero_ok), "of",
sum(1 for k in T2 if REC[k]["woolf"]))
# ================================================================== figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK, "axes.edgecolor": INK2, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def clean(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
# ---- candidate A: B4 dot-and-interval, all estimates on one log axis
def design_a(path):
XMIN, XMAX = 0.001, 3
rows = [
("Table 2, Results:\n“OR” 0.71 (0.66–0.77)", 0.71, 0.66, 0.77, "rep"),
("Abstract, Results (adjusted):\n“OR” 0.82 (0.71–0.85)", 0.82, 0.71, 0.85, "rep"),
(f"Odds ratio (0×177)/(71×23) = 0,\nexact 95% CI 0–{ci.high:.2f}",
None, None, ci.high, "rec0"),
("Odds ratio, 0.5 added to cells:\n" + fmt(B4_HAL, 3), *B4_HAL, "rec"),
("Share of controls, B4− / B4+:\n" + fmt(B4_RR, 3),
*B4_RR, "rec"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.arange(len(rows))[::-1]
for y, (lab, o, lo, hi, kind) in zip(ys, rows):
col = ORANGE if kind == "rep" else BLUE
if kind == "rec0":
ax.annotate("", xy=(XMIN, y), xytext=(hi, y),
arrowprops=dict(arrowstyle="-|>", color=col, lw=2, mutation_scale=14))
ax.plot([hi, hi], [y - .12, y + .12], color=col, lw=2)
ax.text(XMIN * 1.15, y + 0.2, "estimate 0 (off the log axis)", color=INK2, fontsize=10,
va="bottom")
continue
ax.plot([lo, hi], [y, y], color=col, lw=2, zorder=3)
for e in (lo, hi):
ax.plot([e, e], [y - .14, y + .14], color=col, lw=2, zorder=3)
mk = "o" if kind == "rep" else "D"
ax.plot(o, y, mk, color=col, ms=9, mec=BG, mew=1.5, zorder=4)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(0.93, ys[0] + 0.5, "no association", color=INK2, fontsize=10, va="center", ha="right")
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xticks([0.001, 0.01, 0.1, 0.5, 1, 2])
ax.set_xticklabels(["0.001", "0.01", "0.1", "0.5", "1", "2"])
ax.set_xlabel("Ratio, log scale (B4: 0/71 BA vs 23/200 controls)")
clean(ax)
ax.plot([], [], "o-", color=ORANGE, ms=8, label="reported in the paper")
ax.plot([], [], "D-", color=BLUE, ms=8, label="recomputed from Table 2 counts")
fig.legend(loc="upper left", bbox_to_anchor=(0.01, 0.99), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.36, right=0.97, top=0.88, bottom=0.14)
save(fig, path)
# ---- candidate B: Cleveland dot plot over every Table 2 row
def design_b(path):
order = sorted(T2, key=lambda k: T2[k][4])
fig, ax = plt.subplots(figsize=(7.2, 5.6))
for i, k in enumerate(order):
ro = T2[k][4]
w, h = REC[k]["woolf"], REC[k]["hal"]
zero = w is None
rec = h[0] if zero else w[0]
ax.plot([min(ro, rec), max(ro, rec)], [i, i], color=GRID if not zero else INK2,
lw=2, zorder=2)
ax.plot(ro, i, "o", color=ORANGE, ms=11, mec=BG, mew=1.5, zorder=4)
if zero:
ax.plot(rec, i, "D", color=BLUE, ms=8, mfc=BG, mew=2, zorder=5)
ax.text(max(ro, rec) * 1.25, i, "zero cell", color=INK2, fontsize=10, va="center")
else:
ax.plot(rec, i, "D", color=BLUE, ms=5.5, zorder=6)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.03, 40)
ax.set_yticks(range(len(order)))
ax.set_yticklabels(order)
ax.set_ylim(-0.7, len(order) - 0.3)
ax.set_xticks([0.05, 0.1, 0.5, 1, 2, 5, 10, 20])
ax.set_xticklabels(["0.05", "0.1", "0.5", "1", "2", "5", "10", "20"])
ax.set_xlabel("Odds ratio, BA vs controls (log scale; 1 = no association)")
clean(ax)
ax.plot([], [], "o", color=ORANGE, ms=11, label="“OR” printed in Table 2")
ax.plot([], [], "D", color=BLUE, ms=5.5, label="OR recomputed from the row's counts (inside the orange dot when they agree)")
ax.plot([], [], "D", color=BLUE, ms=8, mfc=BG, mew=2,
label="zero-cell row: OR with 0.5 added to each cell")
ax.legend(loc="lower left", bbox_to_anchor=(0, 1.0), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- shared box helpers for the diagram designs
def box(ax, x, y, w, h, text, edge, fs=10.5, ha="left", weight="normal", fill=BG):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.012,rounding_size=0.015",
fc=fill, ec=edge, lw=1.6))
tx = x + 0.02 if ha == "left" else x + w / 2
ax.text(tx, y + h / 2, text, ha=ha, va="center", fontsize=fs, color=INK, weight=weight,
linespacing=1.45, family="DejaVu Sans")
def arrow(ax, p, q, col=INK2, text=None, toff=(0, 0.03)):
ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=14, color=col, lw=1.6))
if text:
ax.text((p[0] + q[0]) / 2 + toff[0], (p[1] + q[1]) / 2 + toff[1], text, ha="center",
va="bottom", fontsize=10, color=INK2)
def canvas(h):
fig = plt.figure(figsize=(7.2, h))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
return fig, ax
# ---- candidate C: calculation-flow diagram for the B4 row
def design_c(path):
fig, ax = canvas(4.4)
ax.text(0.03, 0.95, "Haplogroup B4, Table 2 counts", fontsize=11, weight="bold", va="center")
box(ax, 0.03, 0.60, 0.30, 0.28,
" B4+ B4−\nBA 0 71\ncontrols 23 177",
INK2, fs=10.5)
box(ax, 0.40, 0.66, 0.57, 0.22,
f"Odds ratio = (0 × 177) / (71 × 23) = 0\nexact 95% CI 0–{ci.high:.2f}\n"
f"with 0.5 added to each cell: {fmt(B4_HAL, 3)}", BLUE, fs=10.5)
box(ax, 0.40, 0.36, 0.57, 0.22,
"Proportion of controls, B4− / B4+\n"
"= (177/248) / (23/23) = " + fmt(B4_RR, 3), BLUE)
arrow(ax, (0.335, 0.77), (0.395, 0.77))
arrow(ax, (0.335, 0.66), (0.395, 0.47))
box(ax, 0.40, 0.05, 0.57, 0.20,
"Printed in Table 2 and Results as the odds ratio:\n“OR” 0.71, 95% CI 0.66–0.77, p = 0.001",
ORANGE)
ax.add_patch(FancyArrowPatch((0.695, 0.355), (0.695, 0.255), arrowstyle="<|-|>",
mutation_scale=14, color=INK2, lw=1.6))
ax.text(0.71, 0.305, "same value to two decimals", fontsize=10, color=INK2, va="center")
ax.text(0.03, 0.45, "blue = recomputed from\nthe paper's counts\n\norange = reported\nin the paper",
fontsize=10, color=INK2, va="top")
save(fig, path)
# ---- candidate D: text panels, Table 2 rows vs what the counts give
def design_d(path):
fig, ax = canvas(4.6)
ax.text(0.03, 0.955, "Table 2 (as published)", fontsize=11, weight="bold", va="center",
color=INK)
ax.text(0.46, 0.955, "Recomputed from the same counts", fontsize=11, weight="bold",
va="center", color=INK)
lines = [
("B4", "0/71 vs 23/200", "“OR” 0.71 (0.66–0.77)",
"OR 0; ratio of proportions 0.714 (0.66–0.77)"),
("M8&Z", "0/71 vs 5/200", "“OR” 1.36 (1.27–1.47)",
"OR 0; ratio of proportions 1.364 (1.27–1.47)"),
("Y", "0/71 vs 1/200", "“OR” 1.36 (1.26–1.46)",
"OR 0; ratio of proportions 1.357 (1.26–1.46)"),
("D", "2/71 vs 0/200", "“OR” 3.90 (3.18–4.78)",
"OR infinite; ratio of proportions 3.899 (3.18–4.78)"),
("C", "1/71 vs 1/200", "p = 1.456",
"Fisher p = %.3f (row Z, same counts: 0.456)" % REC["C"]["fisher"]),
]
top = 0.86
hgt = 0.135
for i, (k, cnt, rep, rec) in enumerate(lines):
y = top - i * (hgt + 0.03) - hgt
box(ax, 0.03, y, 0.33, hgt, f"{k} {cnt}\n{rep}", ORANGE, fs=10.5)
box(ax, 0.46, y, 0.51, hgt, rec.replace("; ", ";\n").replace(" (row Z", "\n(row Z"),
BLUE, fs=10.5)
arrow(ax, (0.365, y + hgt / 2), (0.455, y + hgt / 2))
ax.text(0.03, 0.025, "Counts: carriers among 71 BA patients vs among 200 controls.",
fontsize=10, color=INK2, ha="left")
save(fig, path)
# ---- candidate E: intervals for every zero-cell row on one log axis
def design_e(path):
rows = ["B4", "M8&Z", "Y", "D"]
fig, ax = plt.subplots(figsize=(7.2, 4.6))
yt, yl = [], []
for i, k in enumerate(rows):
base = (len(rows) - 1 - i) * 3.4
ro, rl, rh = T2[k][4:7]
h = REC[k]["hal"]
r = ZERO[k][1]
trio = [(base + 1.0, ro, rl, rh, ORANGE, "o", BLUE),
(base + 0.0, h[0], h[1], h[2], BLUE, "D", BLUE),
(base - 1.0, r[0], r[1], r[2], BLUE, "s", BLUE)]
for y, o, lo, hi, col, mk, _ in trio:
ax.plot([lo, hi], [y, y], color=col, lw=2, zorder=3)
ax.plot(o, y, mk, color=col, ms=8, mec=BG, mew=1.5, zorder=4,
mfc=col if mk != "s" else BG)
if mk == "s":
ax.plot(o, y, mk, color=col, ms=8, mfc=BG, mew=2, zorder=4)
a_, b_, c_, d_ = T2[k][:4]
yt.append(base)
yl.append(f"{k}\n{a_}/71 vs {c_}/200")
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.002, 400)
ax.set_yticks(yt)
ax.set_yticklabels(yl, fontsize=10)
ax.set_ylim(-1.8, (len(rows) - 1) * 3.4 + 1.8)
ax.set_xticks([0.01, 0.1, 1, 10, 100])
ax.set_xticklabels(["0.01", "0.1", "1", "10", "100"])
ax.set_xlabel("Ratio, BA vs controls (log scale; 1 = no association)")
clean(ax)
ax.plot([], [], "o-", color=ORANGE, ms=8, label="“OR” (95% CI) printed in Table 2")
ax.plot([], [], "D-", color=BLUE, ms=8, label="OR with 0.5 added to each cell")
ax.plot([], [], "s-", color=BLUE, ms=8, mfc=BG, mew=2,
label="ratio of proportions matching the printed value")
ax.legend(loc="lower left", bbox_to_anchor=(0, 1.0), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
design_a(os.path.join(CAND, "A_b4_intervals"))
design_b(os.path.join(CAND, "B_all_rows_dotplot"))
design_c(os.path.join(CAND, "C_b4_calculation_flow"))
design_d(os.path.join(CAND, "D_zero_cell_text_panels"))
design_e(os.path.join(CAND, "E_zero_cell_intervals"))
# provisional choice for the write-up: A as fig1, E as fig2
design_a(os.path.join(HERE, "fig1"))
design_e(os.path.join(HERE, "fig2"))
print("\nfigures written to", HERE)
Abstract swaps the insured and uninsured labels on the key outpatient odds ratios
PLoS Med (2026) · doi:10.1371/journal.pmed.1005087 · PMC13160435
Summary
For people with physical-psychological-cognitive multimorbidity, the Abstract reports an outpatient odds ratio of 6.22 (95% CI 5.33 to 7.25) among insured individuals and 3.40 (3.03 to 3.82) among uninsured individuals. The Results report the same two estimates and intervals with the groups reversed: 6.22 without insurance and 3.40 with insurance. Fig 5's source data (S1 Data) and the interaction model in S1 Appendix agree with the Results. The Abstract's labels also contradict the Abstract's own preceding sentence, which says the association was "attenuated" among the insured. As printed, the only number the Abstract gives for the insurance analysis points the opposite way from the paper's finding: the association with outpatient use looks 0.55 times as strong among the uninsured, when the Results give 1.83 times as strong.
What the paper reports
Abstract, Methods and findings:
Among individuals with health insurance coverage, the association between multimorbidity and outpatient care utilisation was generally attenuated. This was especially evident for those with physical-psychological-cognitive multimorbidity: insured individuals had an OR of 6.22 (95% CI [5.33, 7.25]; p < 0.001), compared with 3.40 (95% CI [3.03, 3.82]; p < 0.001) among uninsured individuals.
Results, subsection "The role of health insurance", first paragraph:
Specifically, among individuals with health insurance coverage, the association between multimorbidity and outpatient care use was generally weaker. This was particularly evident for those with physical-psychological-cognitive multimorbidity (without insurance: OR 6.22, 95% CI [5.33, 7.25]; p < 0.001; with insurance: OR 3.40, 95% CI [3.03, 3.82]; p < 0.001), physical-psychological multimorbidity (without insurance: OR 8.32, 95% CI [7.76, 8.93]; p < 0.001; with insurance: OR 3.01, 95% CI [2.76, 3.29]; p < 0.001), and single physical conditions (without insurance: OR 5.67, 95% CI [5.39, 5.97]; p < 0.001; with insurance: OR 3.53, 95% CI [3.23, 3.85]; p < 0.001).
Discussion, fifth paragraph:
In this study, the association between multimorbidity and healthcare use was less pronounced among insured individuals than among those without coverage, especially regarding outpatient services.
Author summary:
Health insurance appeared to buffer the association between multimorbidity and healthcare utilisation.
Methods, "Health insurance":
we treated health insurance at the individual level as a binary variable (covered versus not covered), encompassing public, private, or any other form of coverage.
What the paper's own numbers show
Method. I compared the Abstract's estimates with those in the Results sentence, checked each OR against its own confidence interval (the geometric midpoint of a symmetric log-scale interval should equal the point estimate), and compared both with the unrounded source data for Fig 5 in S1 Data (sheet "Figure_5"). To express the direction of the insurance difference as a single number, I computed the ratio OR(uninsured) / OR(insured) for each multimorbidity pattern. A value above 1 means the association is weaker among the insured. Its 95% CI uses the standard errors implied by the printed intervals, treating the two strata as independent; they are disjoint groups of participants, so this is a reasonable approximation. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| OR, outpatient, phy-psy-cog, insured | Abstract | 6.22 [5.33, 7.25] | 3.40 [3.03, 3.82] per Results and S1 Data (3.4033) |
| OR, outpatient, phy-psy-cog, uninsured | Abstract | 3.40 [3.03, 3.82] | 6.22 [5.33, 7.25] per Results and S1 Data (6.2227) |
| OR, outpatient, phy-psy-cog, without / with insurance | Results, "The role of health insurance" | 6.22 [5.33, 7.25] / 3.40 [3.03, 3.82] | geometric CI midpoints 6.22 / 3.40 (internally consistent) |
| Abstract vs Results estimates | Abstract; Results | same two estimates and CIs | identical to two decimals, with the group labels exchanged |
| S1 Data, outpatient, no / yes insurance | S1 Data, sheet Figure_5 | phy-psy-cog 6.2227 / 3.4033; phy-psy 8.3225 / 3.0145; single physical 5.6723 / 3.5259 | round to the Results values 6.22 / 3.40, 8.32 / 3.01, 5.67 / 3.53 |
| OR(uninsured) / OR(insured), phy-psy-cog | derived from Results | "generally weaker" among insured | 1.83 [1.51, 2.22], z = 6.1 |
| Same ratio, using the Abstract's labels | derived from Abstract | "generally attenuated" among insured | 0.55 [0.45, 0.66]: stronger among insured, contradicting the Abstract's own sentence |
| OR(uninsured) / OR(insured), phy-psy | derived from Results | 8.32 vs 3.01 | 2.76 [2.47, 3.09] (same direction as the Results) |
| OR(uninsured) / OR(insured), single physical | derived from Results | 5.67 vs 3.53 | 1.61 [1.45, 1.78] (same direction as the Results) |
| Multimorbidity × insurance interaction, phy-psy-cog, outpatient | S1 Appendix, Table N | OR 0.66, p < 0.001 | below 1, so weaker among the insured (agrees with the Results); stratified ratio insured / uninsured = 3.4033 / 6.2227 = 0.547 |
A minor side note that is not part of the finding: the S1 Data bounds for the uninsured phy-psy-cog estimate, 5.3362 and 7.2565, round to [5.34, 7.26]. The paper prints [5.33, 7.25] in both the Abstract and the Results, which looks like truncation rather than rounding. This does not affect any conclusion.
Figure
Figure 1. The two passages that report the insurance-stratified outpatient odds ratios for physical-psychological-cognitive multimorbidity, reduced to their key words and numbers as printed. Left (orange border): the Abstract, which says the association was "attenuated" among the insured and then gives the insured group the larger OR (6.22) and the uninsured group the smaller one (3.40). Right (blue border): the Results subsection "The role of health insurance", which gives the same estimates and CIs to two decimals with the groups the other way round: 6.22 without insurance and 3.40 with insurance. The connectors join identical estimates. Only the Results assignment fits the attenuation sentence that both passages contain, and it is the one that matches Fig 5's source data (S1 Data), the interaction model (S1 Appendix Table N) and the Discussion.
Figure 2. Ratio of the outpatient odds ratio among the uninsured to that among the insured, on a log scale. The thin vertical line at 1 marks no difference by insurance status; values to the right mean the association is weaker among the insured. Blue circles are recomputed from the ORs and CIs in the Results for the three patterns the paper reports. All three lie to the right of 1 (1.83, 2.76 and 1.61). The orange diamond takes the Abstract's labels at face value for physical-psychological-cognitive multimorbidity and gives 0.55 [0.45, 0.66], the mirror image of the blue estimate for the same pattern. The Abstract's labelling is the only one of the four that would mean a stronger association among the insured. Intervals assume independent strata.
Why it matters
The Abstract contains a single numerical result for the insurance analysis, and the Abstract's conclusion rests on it: "Our study highlights the potential role of health insurance in reducing socioeconomic disparities in healthcare utilisation associated with multimorbidity". As printed, those numbers say that the association between the most complex multimorbidity pattern and outpatient use was stronger, by a factor of about 1.8, among people with insurance. That is the opposite of the buffering finding the paper reports. Readers who see only the Abstract, along with indexing services, evidence summaries and systematic reviews that extract numbers from abstracts, would record the effect in the wrong direction, or would see a sentence that contradicts its own numbers and be unable to tell from the Abstract which one is right.
The paper's main conclusion is not affected. The Results, the Fig 5 source data, the interaction model in S1 Appendix, the Discussion and the Author summary all use the correct labels, and the two other patterns reported in the Results (physical-psychological, single physical) show the same direction. The error is in labelling, not in the analysis. The title, Table 1 and the Conclusions do not repeat the swapped numbers.
Explanations considered
- The Abstract reports a different model or population. This does not account for the numbers. The outcome (outpatient use), the pattern (physical-psychological-cognitive), the point estimates, the CIs and the p-values are identical in both places. The Methods describe a single set of insurance-stratified models (pooled across countries with country fixed effects), and no other insurance-stratified estimates appear in the paper.
- The insurance variable is coded the other way somewhere. Ruled out. The Methods define insurance as a binary variable, "covered versus not covered", and nothing in the paper reverses that coding. S1 Data labels the Fig 5 source rows Health_Insurance = No (6.2227) and Yes (3.4033), which matches the Results.
- The Results, not the Abstract, contain the swap. Ruled out. The Results assignment agrees with S1 Data, with Fig 5 (in the outpatient panel, the "No insurance" marker for this pattern is at about 6.2 and the "Yes" marker at about 3.4), with the S1 Appendix interaction OR of 0.66 (below 1, meaning weaker among the insured), with the other two patterns, and with the attenuation sentence that the Abstract and the Results share.
- Rounding or interval arithmetic. Not relevant. Each printed OR is the geometric midpoint of its CI, so the numbers are internally consistent; only the group words differ.
- A conversion or typesetting artifact in PMC. Ruled out. The publisher's HTML article page and printable PDF and the Europe PMC full-text XML carry the same Abstract wording ("insured individuals had an OR of 6.22 ... among uninsured individuals") and the same Results wording. The difference is between the words "insured" and "uninsured", not a lost character or a misaligned table cell.
Status
- Version of record. Checked on the PLOS Medicine article page (journals.plos.org). The page lists Version 1, marked "Uncorrected Proof", and Version 2. Both carry the swapped Abstract and the Results as quoted, and so does the printable PDF (text extracted with PyMuPDF). Europe PMC's full-text XML and the PMC-archived text match. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements. S1 Appendix (DOCX), S1 Checklist (STROBE) and S1 Data (XLSX, "Minimal dataset") are listed. S1 Data, sheet "Figure_5", and S1 Appendix Table N were opened and agree with the Results labels (values in the table above). They resolve which assignment is correct.
- Corrections. None found. Crossref lists one update, a publisher "new version" record dated 2026-05-11 that points to the article's own DOI. This is the move from Version 1 to Version 2, not a correction notice, and Version 2 still has the swapped labels. No correction, erratum, expression of concern or retraction was found in Crossref, PMC or Europe PMC, and the publisher page shows no correction notice.
- Comments. PubPeer's search page and API returned a Cloudflare block (HTTP 403) in this session, so PubPeer could not be checked directly. A web search for the DOI together with "correction" returned only the article and index records.
- Sources. PMC13160435; https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1005087 (HTML and printable PDF); S1 Data and S1 Appendix from the same page; Crossref; Europe PMC. As of 2026-10-24.
Suggested correction
In the Abstract (Methods and findings), exchange the group labels so that the sentence agrees with the Results, Fig 5 and S1 Data:
This was especially evident for those with physical-psychological-cognitive multimorbidity: uninsured individuals had an OR of 6.22 (95% CI [5.33, 7.25]; p < 0.001), compared with 3.40 (95% CI [3.03, 3.82]; p < 0.001) among insured individuals.
Optionally, the authors could also check the printed bounds of the uninsured CI (S1 Data gives 5.3362 and 7.2565, which round to 5.34 and 7.26).
Recomputation script (python recompute.py.txt)
"""Recompute the insurance-stratified outpatient odds ratios of PMC13160435
(PLoS Med 2026, doi:10.1371/journal.pmed.1005087) and check the Abstract's labels.
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (Abstract; Results, subsection
"The role of health insurance", paragraph 1) and from the paper's S1 Data workbook,
sheet "Figure_5" (source data for Fig 5). No individual-level data are used.
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
Z = 1.959964
# ------------------------------------------------------------------ inputs
# Results, "The role of health insurance": outpatient care, OR [95% CI]
RESULTS = {
"Physical-psychological-cognitive": {"no": (6.22, 5.33, 7.25), "yes": (3.40, 3.03, 3.82)},
"Physical-psychological": {"no": (8.32, 7.76, 8.93), "yes": (3.01, 2.76, 3.29)},
"Single physical": {"no": (5.67, 5.39, 5.97), "yes": (3.53, 3.23, 3.85)},
}
# Abstract, Methods and findings: "insured individuals had an OR of 6.22 (95% CI [5.33, 7.25] ...),
# compared with 3.40 (95% CI [3.03, 3.82] ...) among uninsured individuals."
ABSTRACT_PPC = {"yes": (6.22, 5.33, 7.25), "no": (3.40, 3.03, 3.82)}
# S1 Data, sheet Figure_5, outpatient (unrounded source data for Fig 5)
S1DATA = {
"Physical-psychological-cognitive": {"no": (6.2227, 5.3362, 7.2565), "yes": (3.4033, 3.0325, 3.8194)},
"Physical-psychological": {"no": (8.3225, None, None), "yes": (3.0145, None, None)},
"Single physical": {"no": (5.6723, None, None), "yes": (3.5259, None, None)},
}
# S1 Appendix, Table N: multimorbidity x health insurance interaction, outpatient, Phy+Psy+Cog
INTERACTION_PPC = 0.66
def se_from_ci(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z)
def ratio_ci(a, b):
"""Ratio of two ORs a/b with a 95% CI, treating the two strata as independent."""
(oa, la, ua), (ob, lb, ub) = a, b
d = math.log(oa / ob)
se = math.hypot(se_from_ci(la, ua), se_from_ci(lb, ub))
return oa / ob, math.exp(d - Z * se), math.exp(d + Z * se), d / se
# ------------------------------------------------------------------ 1. internal consistency
print("1. Each reported OR against its own CI (geometric midpoint of the CI)")
for k, v in RESULTS.items():
for g in ("no", "yes"):
o, lo, hi = v[g]
mid = math.sqrt(lo * hi)
print(f" {k:34s} insured={g:3s} OR {o:.2f} geo-mid {mid:.2f} SE(logOR) {se_from_ci(lo, hi):.3f}")
# ------------------------------------------------------------------ 2. label swap
print("\n2. Abstract vs Results, physical-psychological-cognitive, outpatient")
r = RESULTS["Physical-psychological-cognitive"]
same_swapped = ABSTRACT_PPC["yes"] == r["no"] and ABSTRACT_PPC["no"] == r["yes"]
print(f" Abstract insured = {ABSTRACT_PPC['yes']} Results without insurance = {r['no']}")
print(f" Abstract uninsured = {ABSTRACT_PPC['no']} Results with insurance = {r['yes']}")
print(f" Identical estimates with the group labels exchanged: {same_swapped}")
# ------------------------------------------------------------------ 3. S1 Data
print("\n3. S1 Data (sheet Figure_5), outpatient, rounded to 2 decimals")
for k, v in S1DATA.items():
no, yes = v["no"][0], v["yes"][0]
print(f" {k:34s} no insurance {no:.4f} -> {no:.2f} insurance {yes:.4f} -> {yes:.2f} "
f"(Results: {RESULTS[k]['no'][0]:.2f} / {RESULTS[k]['yes'][0]:.2f})")
lo_no, hi_no = S1DATA["Physical-psychological-cognitive"]["no"][1:]
lo_y, hi_y = S1DATA["Physical-psychological-cognitive"]["yes"][1:]
print(f" PPC CI bounds in S1 Data: no [{lo_no}, {hi_no}], yes [{lo_y}, {hi_y}]")
print(f" rounded: no [{lo_no:.2f}, {hi_no:.2f}], yes [{lo_y:.2f}, {hi_y:.2f}] "
f"(printed: [5.33, 7.25], [3.03, 3.82])")
# ------------------------------------------------------------------ 4. direction
print("\n4. Ratio OR(uninsured) / OR(insured), outpatient; > 1 means weaker among the insured")
RATIOS = {}
for k, v in RESULTS.items():
RATIOS[k] = ratio_ci(v["no"], v["yes"])
q, lo, hi, z = RATIOS[k]
print(f" {k:34s} Results labels : {q:.2f} [{lo:.2f}, {hi:.2f}] z = {z:.1f}")
ABS_RATIO = ratio_ci(ABSTRACT_PPC["no"], ABSTRACT_PPC["yes"])
q, lo, hi, z = ABS_RATIO
print(f" {'Physical-psychological-cognitive':34s} Abstract labels: {q:.2f} [{lo:.2f}, {hi:.2f}] z = {z:.1f}")
s = S1DATA["Physical-psychological-cognitive"]
print(f" S1 Data, PPC, insured / uninsured = {s['yes'][0]}/{s['no'][0]} = {s['yes'][0] / s['no'][0]:.3f}")
print(f" S1 Appendix Table N interaction OR (insured x PPC) = {INTERACTION_PPC} (< 1: weaker among insured)")
# ------------------------------------------------------------------ figure 1: text panels
# Axis units are inches (7.2 x 4.4) so that text positions are predictable.
fig = plt.figure(figsize=(W, 4.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, 4.4)
ax.axis("off")
BW, GAP = 3.1, 0.8
LX = (W - 2 * BW - GAP) / 2
RX = LX + BW + GAP
BY, BH = 0.55, 3.7
PANELS = [
(LX, ORANGE, "Abstract", "Methods and findings", "attenuated",
("insured individuals", "uninsured individuals"),
"As labelled: stronger among the insured,\ncontradicting the sentence above"),
(RX, BLUE, "Results", "‘The role of health insurance’", "weaker",
("without insurance", "with insurance"),
"Weaker among the insured; agrees with\nFig 5, S1 Data and the Discussion"),
]
ROW_Y = (2.05, 1.35) # y of the OR line in each row; label sits 0.27 in above
VALUES = ("OR 6.22 [5.33, 7.25]", "OR 3.40 [3.03, 3.82]")
for x0, col, title, sub, word, labs, note in PANELS:
ax.add_patch(FancyBboxPatch((x0, BY), BW, BH, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=col, lw=1.8))
tx = x0 + 0.18
ax.text(tx, BY + BH - 0.22, title, fontsize=12.5, fontweight="bold", va="top")
ax.text(tx, BY + BH - 0.52, sub, fontsize=10, color=INK2, va="top")
ax.text(tx, BY + BH - 0.92, "“Among individuals with health\ninsurance coverage, the association\n"
f"… was generally {word}.”", fontsize=10, va="top", style="italic", linespacing=1.3)
for lab, val, y in zip(labs, VALUES, ROW_Y):
ax.text(tx, y + 0.27, lab, fontsize=10.5, color=col, fontweight="bold", va="center")
ax.text(tx, y, val, fontsize=11.5, va="center")
ax.text(tx, BY + 0.18, note, fontsize=9.5, color=col, va="bottom", linespacing=1.3)
for y in ROW_Y:
ax.add_patch(FancyArrowPatch((LX + BW - 0.25, y), (RX + 0.12, y), arrowstyle="<|-|>",
mutation_scale=11, lw=1.4, color=INK2, shrinkA=0, shrinkB=0))
ax.text(W / 2, (ROW_Y[0] + ROW_Y[1]) / 2, "same OR\nand CI,\nlabels\nswapped", ha="center",
va="center", fontsize=9.5, color=INK, linespacing=1.2)
ax.text(W / 2, 0.22, "Outpatient care use, physical-psychological-cognitive multimorbidity "
"vs no condition; values as printed.", ha="center", fontsize=9.5, color=INK2)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200)
plt.close(fig)
# ------------------------------------------------------------------ figure 2: ratio plot
fig, ax = plt.subplots(figsize=(W, 3.5))
rows = [("Phy-psy-cog, Abstract labels", ABS_RATIO, ORANGE, "D", 4)]
rows += [(n, RATIOS[k], BLUE, "o", y) for n, k, y in
(("Phy-psy-cog", "Physical-psychological-cognitive", 3),
("Phy-psy", "Physical-psychological", 2),
("Single physical", "Single physical", 1))]
for name, (q, lo, hi, _), col, mk, y in rows:
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot(q, y, mk, ms=9, color=col)
ax.text(1.03, y, f"{q:.2f} [{lo:.2f}, {hi:.2f}]", va="center", fontsize=10, color=col,
transform=ax.get_yaxis_transform())
ax.axvline(1, color=INK2, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.35, 4.5)
ticks = [0.5, 1, 2, 4]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.set_yticks([r[4] for r in rows])
ax.set_yticklabels([r[0] for r in rows])
ax.set_ylim(0.4, 4.6)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("OR(uninsured) / OR(insured), outpatient care (log scale)")
ax.text(0.95, 0.5, "← stronger if insured", ha="right", va="center", fontsize=9.5, color=INK2)
ax.text(1.05, 0.5, "weaker if insured →", ha="left", va="center", fontsize=9.5, color=INK2)
handles = [Line2D([], [], color=ORANGE, marker="D", ms=8, lw=2, label="Abstract, as labelled"),
Line2D([], [], color=BLUE, marker="o", ms=8, lw=2, label="Recomputed from the Results ORs")]
fig.legend(handles=handles, loc="upper center", ncol=2, frameon=False, fontsize=9.5)
fig.subplots_adjust(left=0.30, right=0.79, bottom=0.17, top=0.88)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200)
plt.close(fig)
print("\nWrote fig1.svg/png and fig2.svg/png")
Table 3's Z statistics imply p-values of 0.16 to 0.60, not 0.001 to 0.15
PLoS Med (2026) · doi:10.1371/journal.pmed.1005132 · PMC13313369
Summary
Table 3 reports Wilcoxon Z statistics and p-values for three patient-journey intervals of a pre-post training programme, and the Results repeat the p-values as evidence that patient delay (p = 0.02) and diagnostic delay (p < 0.001) fell. Under the stated Wilcoxon test, whose Z has a standard-normal reference, the printed Z values (0.53, 1.41, −0.55) give two-sided p-values of 0.60, 0.16 and 0.58, while the printed p-values (0.02, 0.001, 0.15) would need |Z| of about 2.33, 3.29 and 1.44. The other 11 Z/p pairs in the paper's S1 Table follow the normal relation, so only these three rows are inconsistent, and no rounding, one-sided, exact or adjusted test reconciles them. If the Z values are correct, neither the headline 54% drop in diagnostic delay nor the 9% drop in patient delay is statistically significant, which removes the stated statistical support for the abstract's "reduced diagnostic delays"; the published information does not show which column is wrong.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py. The table rows were also checked against the publisher's XML and PDF (see Status).
Table 3, "Main indicators change of intervention" (columns: interval; pre-intervention median (IQR); post-intervention median (IQR); % change; Z; P-value):
Symptom onset → first medical contact (Patient Delay) | 27 (IQR = 49) | 24.5 (IQR = 44.5) | -9.3% | 0.53 | 0.02
Symptom onset → diagnosis at first medical contact (Diagnostic Delay) | 56.5 (IQR = 87.5) | 25.8 (IQR = 40.3) | -54.2% | 1.41 | 0.001
Diagnosis at oncology center → treatment initiation (Treatment Delay)a | 69 (IQR = 91.5) | 15.9 (IQR = 21.1) | -76.9%a | -0.55 | 0.15
The only footnote (a) concerns the definition of treatment delay.
Methods, Analysis:
Stata 17 was used for quantitative data analysis. Descriptive statistics (medians, interquartile ranges) characterized delays and KAP distributions. The Wilcoxon signed-rank test compared pre- and post-training KAP scores and interval changes.
Methods, Study of the intervention:
Patient-journey intervals (symptom onset to first contact; first contact to confirmed diagnosis; diagnosis to treatment) were extracted from 100 pediatric oncology charts at baseline and 6 months post-implementation, which are all available charts during the study period.
Results, Process measures and outcomes:
Six months post-intervention, patient delay decreased by 9.3% (to 24.5 days, p = 0.02), the diagnostic interval decreased by 54.3% (to 25.8 days, p < 0.001), and the interval from diagnosis to treatment initiation increased by 11.9% (to 15.9 days, p = 0.15). Treatment initiation at the oncology center accelerated from a median of 69.0 to 15.9 days (-76.9%) (Tables 3 and S1).
Abstract, Methods and findings:
and the time from first contact to confirmed diagnosis decreased by 54.2%. Treatment initiation interval increased by 11.9%, reflecting ongoing infrastructural constraints.
Abstract, Conclusions:
A context-tailored, multilevel training and mentorship model was associated with improved provider capacity and reduced diagnostic delays in Northwest Ethiopia.
Author summary:
The median interval from symptom onset to diagnosis decreased by 54.2%, and patient delay decreased by 10.3%.
Discussion:
Compared with similar efforts in Tanzania and Uganda, which achieved 30%–40% reductions in diagnostic delays through clinician workshops alone, our 54% decrease likely reflects the added impact of engaging community health workers and formalizing referral protocols [16,23–25].
The intervention’s success is driven by concurrent gains in provider capacity, shortened diagnostic timelines, and strengthened interdisciplinary collaboration, which collectively demonstrate the model’s operational viability.
S1 Table (supporting file .s005, "Mean difference between stages of cancer and types of interventions"), column headed "Wilcoxon ranked-sum test Groups of intervention (pre & post)", gives the same three pairs (Patient delay 0.53 / 0.02; Diagnostic Delay 1.41 / 0.001; Treatment Delay −0.55 / 0.15) and 11 further Z / Sig. pairs for other intervals.
What the paper's own numbers show
Method. Stata's signrank and ranksum commands, like the Wilcoxon tests in other packages, report a standardized statistic Z and the two-sided p-value p = 2[1 − Φ(|Z|)], where Φ is the standard normal distribution function. I applied this relation to each printed Z, and inverted it to find the |Z| each printed p requires. To test robustness, I allowed for rounding of Z (±0.005) and of p, replaced the normal reference with t distributions (df 5 to 99), computed exact (permutation) Wilcoxon p-values for signed-rank samples of 20, 50 and 100 and rank-sum samples of 10 + 10 and 50 + 50, and applied the same relation to the other 11 pairs in S1 Table as a control. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Patient delay: p for Z = 0.53 | Table 3, row 1; S1 Table; Results | 0.02 | 0.596 two-sided (0.593 to 0.600 over Z rounding); one-sided 0.298 |
| Diagnostic delay: p for Z = 1.41 | Table 3, row 2; S1 Table; Results ("p < 0.001") | 0.001 | 0.159 two-sided (0.157 to 0.160); one-sided 0.079 |
| Treatment delay: p for Z = −0.55 | Table 3, row 3; S1 Table; Results | 0.15 | 0.582 two-sided (0.579 to 0.586); one-sided 0.291 |
| |Z| needed for the printed p | — | 0.02; 0.001; 0.15 | 2.33 (2.24 to 2.43 over rounding of p); 3.29 (3.17 to 3.48; "p < 0.001" needs > 3.29); 1.44 (1.42 to 1.46) |
| Ratio, recomputed / printed p | — | — | 29.8; 158.5; 3.9 |
| p with a t reference, df = 5, 30, 99 | — | — | 0.619, 0.600, 0.597; 0.218, 0.169, 0.162; 0.606, 0.586, 0.584 |
| Exact Wilcoxon p, signed-rank n = 20, 50, 100 | — | — | 0.596, 0.598, 0.597; 0.165, 0.160, 0.159; 0.596, 0.585, 0.585 |
| Exact Wilcoxon p, rank-sum 10 + 10 and 50 + 50 | — | — | 0.579, 0.599; 0.165, 0.159; 0.579, 0.585 |
| Bonferroni over the three rows | — | — | 1.0; 0.476; 1.0 |
| Other 11 Z / Sig. pairs in S1 Table | S1 Table | e.g. 1.24 / 0.22; 2.64 / 0.01; −0.77 / 0.44; −0.98 / 0.33; −0.40 / 0.69; −0.13 / 0.90; five with |Z| 6.47 to 12.10 / < 0.001 | 0.215; 0.008; 0.441; 0.327; 0.689; 0.897; all < 10−10: all 11 consistent |
| Patient delay % change, 27 → 24.5 | Table 3; Results; Author summary and Discussion | −9.3%; "10.3%" | −9.26% |
| Diagnostic delay % change, 56.5 → 25.8 | Table 3; Results | −54.2%; "54.3%" | −54.34% |
| Treatment delay % change, 69 → 15.9 | Table 3; Results; Abstract | −76.9%; "increased by 11.9%" | −76.96% |
The medians and percentage changes in Table 3 reproduce to rounding (with a 0.1-point discrepancy between −54.2% in the table and 54.3% in the text). The inferential columns do not: all three Z/p pairs fall far off the normal relation that the paper's other 11 Wilcoxon pairs follow. For the diagnostic-delay row the printed p is 159 times smaller than the p implied by its Z. The paper gives no patient-level interval data (S4 File holds only clinician knowledge, attitude and practice scores), so the Wilcoxon tests cannot be re-run, and the correct Z or p cannot be recovered.
Figure
Figure 1. For each Table 3 row, the p-value printed in the table (orange diamonds) and the p-value implied by the printed Z under the stated Wilcoxon test with its standard-normal reference (blue: filled circle two-sided, open circle one-sided), on one log-scale axis. The thin dashed line marks p = 0.05. The printed values are 0.02, 0.001 and 0.15; the two-sided values implied by Z = 0.53, 1.41 and −0.55 are 0.60, 0.16 and 0.58, and the one-sided values 0.30, 0.08 and 0.29. Rounding of Z to two decimals moves the two-sided values by at most 0.004, less than a marker width. Patient delay and diagnostic delay lie on opposite sides of the 0.05 line depending on which column is believed.
Figure 2. All Wilcoxon Z / p pairs printed in the paper's S1 Table, plotted against the relation p = 2[1 − Φ(|Z|)] (blue curve); p on a log scale. Open orange circles are the six printed pairs with |Z| below 3.6 outside Table 3; each lies on the curve to within rounding. Five further pairs with |Z| from 6.47 to 12.10 are printed as p < 0.001, also consistent, and are not drawn. The three Table 3 pairs (orange diamonds, with the p their Z implies in blue) are the only ones off the curve, by factors of 3.9 to 159. The thin dashed line marks p = 0.05.
Why it matters
- It is the paper's only test of the patient-journey outcomes. Table 3 carries the sole significance tests for the intervals from symptom onset to first contact, to diagnosis and to treatment. The Methods list these intervals as secondary outcomes (the primary outcomes are provider knowledge, attitude and practice scores), but the 54% reduction in diagnostic delay is reported in the abstract, the author summary, the Results and the Discussion.
- The conclusions on diagnostic delay depend on which column is right. If the Z values are correct, the diagnostic-delay reduction has p = 0.16 and the patient-delay reduction p = 0.60, and neither is significant at the 5% level. That would leave the abstract conclusion that the model "was associated with ... reduced diagnostic delays", the title's "accelerate early childhood cancer diagnosis", the Discussion comparison in which "our 54% decrease" exceeds the 30–40% reductions reported in Tanzania and Uganda, and the concluding statement that success is "driven by ... shortened diagnostic timelines" without statistical support from the paper. If instead the p-values are correct, the Z column is wrong and the reported significance may stand. A reader cannot tell which from the published material.
- What it does not affect. The medians, IQRs and percentage changes are not in question, and the provider knowledge and practice results, the other half of the main conclusion, rest on separate analyses. The study is a single-arm pre-post design, which the paper itself notes limits causal inference; this issue concerns only the reported statistical evidence for the change.
- The same outcomes carry further, separate inconsistencies. The Abstract and Results say the diagnosis-to-treatment interval "increased by 11.9%", while Table 3 and the next sentence of the Results report a fall from 69.0 to 15.9 days (−76.9%); the baseline 69.0 (IQR 91.5) is also the value the Results give for the "median total delay". The Author summary and Discussion give the patient-delay reduction as 10.3%, against 9.3% in Table 3, the Abstract and the Results. These do not bear on the Z/p mismatch but show that the reporting of these outcomes needs checking as a whole.
Explanations considered
- Rounding. Does not account for it. Allowing for rounding of Z to two decimals, the two-sided p-values range only over 0.593 to 0.600, 0.157 to 0.160 and 0.579 to 0.586. Allowing for rounding of the printed p, the required |Z| ranges are 2.24 to 2.43, 3.17 to 3.48 and 1.42 to 1.46, none of which contains the printed 0.53, 1.41 or 0.55.
- A one-sided test. Does not account for it. The one-sided p-values (0.298, 0.079, 0.291) are also far from 0.02, 0.001 and 0.15, and the paper does not state a one-sided test.
- A t or other heavier-tailed reference distribution. Makes the discrepancy larger: with any df from 5 to 99 the p-values rise (for example 0.162 to 0.218 for Z = 1.41).
- An exact (permutation) Wilcoxon test or a small sample. Does not account for it. With about 100 charts the normal approximation is close to exact; even at n = 20 pairs or 10 + 10 patients, the exact p for Z = 1.41 is 0.165 and for Z = 0.53 at least 0.579. Ties are already absorbed in the standardized Z that Stata reports.
- Signed-rank versus rank-sum. The Methods name the signed-rank test, while S1 Table heads the column "Wilcoxon ranked-sum test", and a reviewer in the published peer-review history asked which was used. Both tests report a standard-normal Z, so the label does not change the mapping from Z to p (the exact checks above cover both).
- A multiplicity adjustment. None is stated, and any such adjustment (for example Bonferroni: 1.0, 0.476, 1.0) would make the p-values larger, not smaller.
- The "Z" column is an effect size r = Z/√N. Not possible: 1.41 exceeds the maximum of 1 for r, and the reported p-values would imply N of about 19, 5 and 7, not about 100 charts.
- Row or column misalignment. Not supported. No Z anywhere in S1 Table is near 2.33 or 3.29, the values the printed 0.02 and 0.001 require, and the Table 3 and S1 Table values are identical, so the pairs are not shifted from neighbouring rows.
- A typesetting or conversion artifact. Ruled out. The PLOS JATS XML, the PLOS PDF, Europe PMC and PMC all carry the same values, and S1 Table, a separate Word file, repeats them. The Results text repeats the p-values.
- The software uses a different p convention. Not supported by the paper itself. The 11 other Wilcoxon pairs reported in the same column of S1 Table follow p = 2[1 − Φ(|Z|)] to rounding.
- Direction of the Z signs. Not an explanation but a further oddity: all three medians fell, yet two Z values are positive and one negative. The sign does not affect a two-sided p.
Status
- Version of record. Checked. The PLOS JATS XML (article/file?type=manuscript), the PLOS printable PDF (page 9, Table 3, doi:10.1371/journal.pmed.1005132.t003), the Europe PMC full-text XML and the PMC archive (via
src/show.py) all give the Table 3 values quoted above, and the PDF Results text repeats "p = 0.02", "p < 0.001" and "p = 0.15". Published 29 June 2026 (Crossref). - Supplements. Five supporting files are listed. S1 Table (.s005, DOCX) was opened and its 14 Wilcoxon pairs are analysed above; it repeats the three Table 3 pairs. As a side check, its Kruskal–Wallis column (χ², df = 5) mostly matches, but three of the four entries printed "< 0.001" (χ² 18.82, 18.04 and 17.16) recompute to 0.002 to 0.004; none crosses 0.05. S4 File (.s004, XLS) holds only clinician pre/post scores. S1 File (protocol), S2 File (COREQ checklist) and S3 File (assessment tool) were listed but not opened; none is described as containing patient-level interval data. The supplements do not resolve the discrepancy.
- Peer review. The published peer-review history asks why a rank-sum test was used for paired interval data, and notes that the "increased by 11.9 percent" sentence differs from Table 3. No reviewer or editor comment mentions the Z statistics or their p-values.
- Corrections and comments. As of 2026-10-24, no correction, erratum, expression of concern or retraction was found:
src/known.pyreturns no Crossref updates and no PMC or Europe PMC notices; the Crossref work record has no update-to, updated-by or relation entries. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title with "PubPeer" found no thread. - Sources. PMC13313369 (via
src/show.py); https://doi.org/10.1371/journal.pmed.1005132 (article, PDF, supporting files .s004 and .s005, peer-review history); Europe PMC; Crossref; PubPeer.
Suggested correction
The authors may wish to re-run the interval comparisons in Table 3 and S1 Table and report, for each row, the test actually used (signed-rank for paired data, or rank-sum if baseline and follow-up charts are from different patients, with the Methods amended to match), the Z statistic and the p-value that the software gives for it. If the printed Z values are correct, Table 3 would read approximately: patient delay Z = 0.53, p = 0.60; diagnostic delay Z = 1.41, p = 0.16; treatment delay Z = −0.55, p = 0.58. The Results sentence would then need its p-values changed, and the Abstract conclusion, Author summary, Discussion and concluding paragraph would need to describe the reductions in patient and diagnostic delay as not statistically significant. If instead the p-values are correct, the Z column should be corrected so that each Z matches its p. In either case, the authors may also wish to reconcile the treatment-delay direction ("increased by 11.9%" in the Abstract and Results against −76.9% in Table 3) and the patient-delay figure (10.3% in the Author summary and Discussion against 9.3% elsewhere), and to report the number of charts in each period.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC13313369 (PLoS Med 2026, doi:10.1371/journal.pmed.1005132)
# Recomputes every number used in writeup.md from values printed in the paper
# (Table 3, Results text, S1 Table) and regenerates fig1.svg/png and fig2.svg/png
# in this script's own folder.
# Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
import os
import numpy as np
from scipy.stats import norm, t, chi2
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------------------
# 1. Table 3 as printed (identical in S1 Table for the Z / Sig. columns)
# name, pre median, post median, % change printed, Z printed, p printed
# ---------------------------------------------------------------------------
T3 = [
("Patient delay", 27.0, 24.5, -9.3, 0.53, 0.02),
("Diagnostic delay", 56.5, 25.8, -54.2, 1.41, 0.001),
("Treatment delay", 69.0, 15.9, -76.9, -0.55, 0.15),
]
def p2(z):
"""Two-sided standard-normal p, as Stata's signrank/ranksum report it."""
return 2 * norm.sf(abs(z))
print("=" * 78)
print("Table 3: Z vs p under the stated Wilcoxon test (standard-normal reference)")
print("=" * 78)
print(f"{'row':17s} {'Z':>6s} {'p rep':>6s} {'p 2-sided':>10s} {'Z-rounding range':>18s} "
f"{'p 1-sided':>9s} {'|Z| needed':>10s} {'ratio':>6s}")
rec = {}
for name, pre, post, pc, z, p in T3:
lo, hi = sorted((p2(abs(z) + 0.005), p2(abs(z) - 0.005)))
zneed = norm.isf(p / 2)
rec[name] = dict(p2=p2(z), lo=lo, hi=hi, p1=norm.sf(abs(z)), zneed=zneed)
print(f"{name:17s} {z:6.2f} {p:6.3f} {p2(z):10.3f} {lo:8.3f} - {hi:6.3f} "
f"{norm.sf(abs(z)):9.3f} {zneed:10.2f} {p2(z)/p:6.1f}")
print("\nt reference distribution instead of normal (only makes p larger):")
for name, pre, post, pc, z, p in T3:
vals = [round(float(2 * t.sf(abs(z), d)), 3) for d in (5, 30, 99)]
print(f" {name:17s} df=5, 30, 99 -> {vals}")
print("\n|Z| needed for each printed p, allowing for rounding of the printed p")
for name, p, lo, hi in [("Patient delay", 0.02, 0.015, 0.025),
("Diagnostic delay", 0.001, 0.0005, 0.0015),
("Treatment delay", 0.15, 0.145, 0.155)]:
print(f" {name:17s} p={p:<6g} (range {lo}-{hi}) -> |Z| {norm.isf(hi/2):.2f} to {norm.isf(lo/2):.2f}")
print(" Diagnostic-delay text value 'p < 0.001' needs |Z| >", f"{norm.isf(0.0005):.2f}")
print(" p = 0.05 (two-sided) needs |Z| >=", f"{norm.isf(0.025):.2f}")
# Exact (permutation) null distributions, no ties: does any plausible sample size
# turn the printed Z into the printed p? Exact two-sided p = P(|S - E S| >= |Z| * SD).
def signed_rank_exact(n):
m = n * (n + 1) // 2
f = np.zeros(m + 1); f[0] = 1.0
for k in range(1, n + 1):
g = f.copy(); g[k:] += f[:-k]; f = g / 2.0
return np.arange(m + 1), f, m / 2, np.sqrt(n * (n + 1) * (2 * n + 1) / 24)
def rank_sum_exact(n1, n2):
# distribution of the Mann-Whitney U by the standard recursion, as probabilities
prev = [None] * (n2 + 1)
for i in range(n1 + 1):
cur = []
for j in range(n2 + 1):
if i == 0 or j == 0:
d = np.zeros(i * j + 1); d[0] = 1.0
else:
d = np.zeros(i * j + 1)
a = prev[j] # (i-1, j): shift by j
d[j:j + len(a)] += i / (i + j) * a
b = cur[j - 1] # (i, j-1)
d[:len(b)] += j / (i + j) * b
cur.append(d)
prev = cur
f = prev[n2]
return np.arange(len(f)), f, n1 * n2 / 2, np.sqrt(n1 * n2 * (n1 + n2 + 1) / 12)
def exact_p(dist, z):
s, f, mu, sd = dist
return float(f[np.abs(s - mu) >= abs(z) * sd - 1e-9].sum())
print("\nExact Wilcoxon p for the printed |Z| (no ties), against the printed p")
dists = {"signed-rank n=20": signed_rank_exact(20),
"signed-rank n=50": signed_rank_exact(50),
"signed-rank n=100": signed_rank_exact(100),
"rank-sum 10 vs 10": rank_sum_exact(10, 10),
"rank-sum 50 vs 50": rank_sum_exact(50, 50)}
for name, pre, post, pc, z, p in T3:
vals = ", ".join(f"{k}: {exact_p(d, z):.3f}" for k, d in dists.items())
print(f" {name:17s} printed p={p:g} | {vals}")
print("\nBonferroni over the 3 Table 3 tests (only makes p larger):",
[round(float(min(1, 3 * p2(z))), 3) for *_, z, p in T3])
print("\nIf the 'Z' column were an effect size r = Z/sqrt(N): N implied by printed p")
for name, pre, post, pc, z, p in T3:
print(f" {name:17s} N = {(norm.isf(p/2)/abs(z))**2:5.1f} (|r| = {abs(z)} {'> 1, impossible' if abs(z) > 1 else '<= 1'})")
print("\nPercent changes from the printed medians")
for name, pre, post, pc, z, p in T3:
print(f" {name:17s} ({post} - {pre}) / {pre} = {100*(post-pre)/pre:7.2f}% printed {pc}%")
print(" Text values: patient delay '10.3%' (author summary, Discussion), "
"diagnostic '54.3%' (Results), treatment 'increased by 11.9%' (abstract, Results)")
print(f" Treatment delay printed medians 69.0 -> 15.9 is a change of "
f"{100*(15.9-69.0)/69.0:.1f}%, not +11.9%")
# ---------------------------------------------------------------------------
# 2. S1 Table (supporting file .s005): all 14 Wilcoxon Z / Sig. pairs and the
# Kruskal-Wallis chi-square (df = 5) / Sig. pairs, as printed
# ---------------------------------------------------------------------------
S1 = [ # name, KW chi2, KW sig printed, Z, Z sig printed (None = "<0.001")
("Patient delay", 18.82, "<0.001", 0.53, 0.02),
("Diagnostic Delay", 14.23, "0.01", 1.41, 0.001),
("Referral Delay", 15.10, "0.01", 1.24, 0.22),
("Tertiary Care Diagnostic Delay", 16.58, "0.01", -0.13, 0.90),
("Primary Care Delay", 18.04, "<0.001", 2.64, 0.01),
("Physician Delay", 21.11, "<0.001", -0.77, 0.44),
("Healthcare System Delay", 17.16, "<0.001", -0.98, 0.33),
("Post-Diagnostic Referral Interval", 0.18, "1.00", 12.10, None),
("Secondary Diagnostic Interval", 3.69, "0.59", 11.15, None),
("Primary Care to Treatment Interval", 11.49, "0.04", 6.47, None),
("Tertiary Care Diagnostic Delay (2)", 5.03, "0.41", -8.74, None),
("Referral to Treatment Interval", 6.51, "0.26", -6.69, None),
("Treatment Delay", 22.01, "0.00", -0.55, 0.15),
("Total delay", 16.32, "0.01", -0.40, 0.69),
]
T3_NAMES = {"Patient delay", "Diagnostic Delay", "Treatment Delay"}
print("\n" + "=" * 78)
print("S1 Table: Wilcoxon Z vs printed Sig. (two-sided normal p, rounded to 2 dp)")
print("=" * 78)
n_ok = 0
for name, c, cp, z, zp in S1:
pr = p2(z)
if zp is None:
ok = pr < 0.001
shown = "<0.001"
else:
# allow for Z printed to 2 dp (+/-0.005) and p printed to 2 dp (+/-0.005)
lo, hi = sorted((p2(abs(z) + 0.005), p2(abs(z) - 0.005)))
ok = (lo - 0.005) <= zp <= (hi + 0.005)
shown = f"{zp}"
flag = "consistent" if ok else "INCONSISTENT"
n_ok += (ok and name not in T3_NAMES)
print(f" {name:36s} Z={z:6.2f} printed {shown:>7s} recomputed {pr:9.3g} {flag}")
print(f" Pairs outside Table 3 consistent: {n_ok} of {len(S1) - 3}")
print("\nS1 Table: Kruskal-Wallis chi-square (df = 5) vs printed Sig. (side check)")
for name, c, cp, z, zp in S1:
print(f" {name:36s} chi2={c:6.2f} printed {cp:>7s} recomputed {chi2.sf(c, 5):.4f}")
# ---------------------------------------------------------------------------
# 3. Figures
# ---------------------------------------------------------------------------
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, TXT, TXT2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT2, "ytick.color": TXT,
"axes.edgecolor": TXT2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def style(ax):
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(True, which="major", axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# ---- Figure 1: reported vs recomputed p per Table 3 row --------------------
fig, ax = plt.subplots(figsize=(7.2, 3.9))
style(ax)
ys = [2, 1, 0]
ax.axvline(0.05, color=TXT2, lw=0.9, ls=(0, (4, 3)), zorder=1)
ax.text(0.05, 2.62, "p = 0.05", ha="center", va="bottom", color=TXT2, fontsize=10)
for y, (name, pre, post, pc, z, p) in zip(ys, T3):
r = rec[name]
ax.plot([p, r["p2"]], [y, y], color="#b9b8b3", lw=2, zorder=2)
ax.plot(r["p1"], y, "o", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=4)
ax.plot(r["p2"], y, "o", ms=10, color=BLUE, zorder=5)
ax.plot(p, y, "D", ms=9, color=ORANGE, zorder=5)
ax.annotate(f"{p:g}", (p, y), xytext=(0, 9), textcoords="offset points",
ha="center", va="bottom", color=ORANGE, fontsize=10.5, fontweight="bold")
ax.annotate(f"{r['p2']:.2f}", (r["p2"], y), xytext=(0, 9), textcoords="offset points",
ha="center", va="bottom", color=BLUE, fontsize=10.5, fontweight="bold")
ax.annotate(f"{r['p1']:.2f}", (r["p1"], y), xytext=(0, -11), textcoords="offset points",
ha="center", va="top", color=BLUE, fontsize=10)
ax.set_yticks(ys)
ax.set_yticklabels([f"{n}\nZ = {z:g}" for n, _, _, _, z, _ in T3])
ax.set_xscale("log")
ax.set_xlim(4e-4, 1.6)
ax.set_ylim(-0.6, 2.95)
ax.set_xticks([0.001, 0.01, 0.05, 0.1, 0.5, 1])
ax.set_xticklabels(["0.001", "0.01", "0.05", "0.1", "0.5", "1"])
ax.set_xlabel("p-value (log scale)")
ax.tick_params(axis="y", length=0)
h = [
plt.Line2D([], [], marker="D", ls="", ms=9, color=ORANGE, label="p printed in Table 3"),
plt.Line2D([], [], marker="o", ls="", ms=10, color=BLUE,
label="two-sided p from printed Z"),
plt.Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=BLUE, mew=2,
label="one-sided p from printed Z"),
]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=2,
frameon=False, fontsize=10, handletextpad=0.4, columnspacing=1.2)
fig.tight_layout()
save(fig, "fig1")
# ---- Figure 2: every Z / p pair in S1 Table against the normal relation ----
fig, ax = plt.subplots(figsize=(7.2, 4.2))
style(ax)
ax.grid(True, which="major", axis="y", color=GRID, lw=0.8)
zz = np.linspace(0, 3.6, 400)
ax.plot(zz, 2 * norm.sf(zz), color=BLUE, lw=2, zorder=2,
label="p = 2[1 − Φ(|Z|)]")
ax.axhline(0.05, color=TXT2, lw=0.9, ls=(0, (4, 3)), zorder=1)
ax.text(3.58, 0.045, "p = 0.05", ha="right", va="top", color=TXT2, fontsize=10)
other = [(abs(z), zp) for n, c, cp, z, zp in S1 if n not in T3_NAMES and zp is not None]
ax.plot([a for a, _ in other], [b for _, b in other], "o", ms=8, mfc=BG, mec=ORANGE,
mew=2, ls="", zorder=4, label="other S1 Table pairs, printed")
first = True
for name, pre, post, pc, z, p in T3:
a = abs(z)
ax.plot([a, a], [p, p2(z)], color="#b9b8b3", lw=2, zorder=3)
ax.plot(a, p2(z), "o", ms=9, color=BLUE, zorder=5,
label="Table 3 rows, p from Z" if first else None)
ax.plot(a, p, "D", ms=9, color=ORANGE, zorder=5,
label="Table 3 rows, printed p" if first else None)
first = False
below = name == "Treatment delay" # keep clear of the Z = 1.24 marker above it
ax.annotate(f"{name}\nZ = {z:g}\np = {p:g}", (a, p), xytext=(9, -2 if below else 0),
textcoords="offset points", ha="left", va="top" if below else "center",
fontsize=10, color=TXT, zorder=6,
bbox=dict(boxstyle="square,pad=0.1", fc=BG, ec="none"))
ax.set_yscale("log")
ax.set_ylim(4e-4, 1.5)
ax.set_xlim(0, 3.6)
ax.set_yticks([0.001, 0.01, 0.05, 0.1, 0.5, 1])
ax.set_yticklabels(["0.001", "0.01", "0.05", "0.1", "0.5", "1"])
ax.set_xlabel("|Z| as printed")
ax.set_ylabel("p-value (log scale)")
ax.text(3.58, 1.0, "Not drawn: 5 pairs with |Z| from 6.47 to 12.10,\n"
"printed p < 0.001 (consistent)", ha="right", va="top", fontsize=10, color=TXT2)
ax.legend(loc="upper right", bbox_to_anchor=(1.0, 0.86), frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written to", HERE)
Abstract gives the earlier cohort's 70% accuracy as this trial's result; recomputed value is 63%
PLoS Med (2019) · doi:10.1371/journal.pmed.1002813 · PMC6546206
Summary
The abstract of this multi-institutional validation trial (MULTI-IMPROD, 338 men analysed) reports that "the accuracy and NPV of IMPROD bpMRI for SPCa were 70% (113/161) and 95% (71/75)". The NPV of 71/75 belongs to this trial. The accuracy of 113/161, however, is the figure printed in Table 3 for the earlier single-centre IMPROD trial (161 men). From the paper's own Table 2 counts, accuracy in the 338-man trial is (142 + 71)/338 = 213/338 = 63.0% (Wilson 95% CI 57.8%–68.0%). Table 3 prints this as 63%, but with the count 217/338, which is 64.2%. Overall accuracy is part of the trial's stated primary outcome, so the abstract overstates a primary-outcome result by 7.2 percentage points, by a value outside its 95% CI. It also hides the fall in accuracy between the earlier trial and the validation. The paper's headline conclusion, a high NPV, does not depend on this figure and stands.
What the paper reports
Abstract, Methods and findings (the primary outcome and the cohort):
The primary outcome of the trial was diagnostic accuracy—including overall accuracy, sensitivity, specificity, negative predictive value (NPV), and positive predictive value—of IMPROD bpMRI for clinically significant PCa (SPCa), which was defined as a Gleason score ≥ 3 + 4 (Gleason grade group 2 or higher). In total, 338 (338/364, 93%) prospectively enrolled men completed the trial.
Abstract, Methods and findings, the next sentence:
The accuracy and NPV of IMPROD bpMRI for SPCa were 70% (113/161) and 95% (71/75) (95% CI 87%–98%), respectively.
Methods, "Study end points":
The primary outcome of the trial was diagnostic accuracy—including overall accuracy, sensitivity, specificity, NPV, and positive predictive value—of IMPROD bpMRI for SPCa, which was defined as a Gleason score ≥ 3 + 4.
Table 3, title and rows (columns: Measure | MULTI-IMPROD trial | IMPROD trial):
Comparison between the current trial (MULTI-IMPROD trial) and the pre-validation cohort (IMPROD trial).
Patient cohort in total | 338 | 161
Sensitivity* | 97% (142/146) [93%–99%] | 95% (79/83) [88%–98%]
Specificity* | 37% (71/192) [31%–44%] | 44% (34/78) [33%–55%]
NPV* | 95% (71/75) [87%–98%] | 90% (34/38) [76%–96%]
PPV* | 54% (141/263) [48%–60%] | 64% (79/123) [55%–72%]
Accuracy* | 63% (217/338) | 70% (113/161)
Table 3, footnote:
*Sensitivity, specificity, NPV, PPV, and accuracy values are based on binary classification (IMPROD bpMRI Likert score 1–2 versus 3–5) for predicting prostate cancer with Gleason score ≥ 3 + 4.
Table 2, title (the table gives men per Likert score 1–5 by biopsy outcome; its cells are used below):
Prevalence of prostate cancer and clinically significant prostate cancer in different IMPROD bpMRI Likert score groups based on a cohort of 338 men.
Results, "Diagnostic accuracy", first paragraph (relevant to a secondary finding below):
SPCa was diagnosed in 5% (2/34), 5% (2/41), 12% (8/66), 39% (23/59), and 88% (121/138) of men with IMPROD bpMRI Likert scores of 1, 2, 3, 4, and 5, respectively (Table 2).
What the paper's own numbers show
Method. Table 2 gives, for each Likert score from 1 to 5, the number of men with no cancer, Gleason 3+3, Gleason 3+4 and Gleason > 3+4. I checked that each column sums to its printed total (338 in all). Using the Table 3 footnote's definitions (test positive = Likert 3–5; reference positive = SPCa, Gleason ≥ 3+4), I built the 2 × 2 table for the 338 men. From it I computed sensitivity, specificity, NPV, PPV and overall accuracy, (TP + TN)/N, each with a Wilson 95% CI, and compared them with Table 3 and the abstract. I did the same for the earlier IMPROD trial from its Table 3 counts. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| 2 × 2 cells, this trial | Table 2 | (cell counts by Likert score) | TP = 8 + 23 + 111 = 142, FN = 2 + 2 = 4, FP = 263 − 142 = 121, TN = 75 − 4 = 71; N = 338 |
| Sensitivity, this trial | Table 3 | 97% (142/146) [93%–99%] | 142/146 = 97.3% [93.2–98.9]; matches |
| Specificity, this trial | Table 3 | 37% (71/192) [31%–44%] | 71/192 = 37.0% [30.5–44.0]; matches (lower bound differs by under one point) |
| NPV, this trial | Table 3; Abstract | 95% (71/75) [87%–98%] | 71/75 = 94.7% [87.1–97.9]; matches |
| PPV, this trial | Table 3 | 54% (141/263) [48%–60%] | 142/263 = 54.0% [48.0–59.9]; percentage matches, numerator printed one too low (141/263 = 53.6%) |
| Overall accuracy, this trial | Table 3 | 63% (217/338) | (142 + 71)/338 = 213/338 = 63.0% [57.8–68.0]; percentage matches, but 217/338 = 64.2% |
| Overall accuracy, as given in the abstract | Abstract | 70% (113/161) | 113/161 = 70.2%. Denominator 161 is not this trial's N (338); 70.2% lies outside the 57.8–68.0% CI of the recomputed value |
| Overall accuracy, earlier IMPROD trial | Table 3, IMPROD column | 70% (113/161) | (79 + 34)/161 = 113/161 = 70.2% [62.7–76.7]; identical to the abstract figure |
| NPV, earlier IMPROD trial | Table 3, IMPROD column | 90% (34/38) | 34/38 = 89.5% |
| Gap between abstract accuracy and this trial's accuracy | derived | — | 70.2% − 63.0% = 7.2 percentage points |
| SPCa in Likert 5 (secondary finding) | Results, "Diagnostic accuracy" | 88% (121/138) | Table 2: 37 + 74 = 111/138 = 80.4%. With 121 the per-score SPCa counts would total 156, not the 146 used throughout |
The abstract's single sentence therefore combines two cohorts: its NPV (71/75) comes from the 338 men of this trial, and its accuracy (113/161) from the 161 men of the earlier trial. Accuracy for this trial, from its own cells, is 63.0%, which is the percentage Table 3 prints.
Figure
Figure 1. Where each number in the abstract sentence comes from. Top: the two consecutive abstract sentences, quoted verbatim; the accuracy figure (orange outline, as reported) and the NPV figure (blue outline) are boxed. Bottom: the two columns of Table 3, with each printed value and the value recomputed from that column's own counts (TP + TN over the cohort N for accuracy; TN over test negatives for NPV). The abstract's accuracy, 70% (113/161), is the earlier single-centre cohort's value; its NPV, 95% (71/75), is this trial's. The recomputed accuracy for this trial's 338 men is 213/338 = 63.0%. All numbers are from the paper's Abstract, Table 2 and Table 3, or from recompute.py.txt.
Figure 2. Overall accuracy for SPCa on one 0–100% axis. The hollow orange dot is the abstract's reported value (70%; 113/161 = 70.2%). The blue dot and line are the accuracy recomputed for this trial's 338 men from Table 2 (213/338 = 63.0%, Wilson 95% CI 57.8–68.0%, also shaded). The gray dot and line are the earlier IMPROD trial (113/161 = 70.2%, 62.7–76.7%), shown for context: the abstract value coincides with it and falls outside the CI for this trial.
Why it matters
- It is a primary-outcome figure in the abstract. The abstract and Methods both name overall accuracy as part of the primary outcome. The abstract gives 70% for this trial; the trial's own data give 63.0%. The abstract is the part of the paper most readers, indexers and guideline reviewers see.
- It hides the change between cohorts. Read correctly, accuracy fell from 70.2% in the earlier single-centre trial to 63.0% in the multi-institutional validation, in line with the falls in specificity (43.6% to 37.0%) and PPV (64.2% to 54.0%) shown in Table 3. The aim stated in the abstract is to validate the protocol in a multi-institutional setting, so this change is what a reader of a validation study needs to see. Quoting the earlier figure as the result makes the two cohorts look identical on this measure.
- What is not affected. The abstract's conclusion rests on the NPV, "IMPROD bpMRI demonstrated a high NPV for SPCa", and that value (95%, 71/75, CI 87%–98%) is reported correctly. So are the 97% sensitivity and the 22% (75/338) reduction in biopsies with 4/146 SPCa missed. The direction of the findings does not change; the magnitude of overall accuracy does.
- Smaller count errors. Table 3's accuracy count (217/338) and PPV count (141/263) do not match the 2 × 2 cells (213 and 142); the printed percentages are right. The Results text gives SPCa in Likert 5 as 88% (121/138), whereas Table 2 gives 111/138 = 80.4%. This last figure enters the description of how SPCa prevalence rises with Likert score, which readers use when weighing whether to biopsy a Likert 5 finding.
Explanations considered
- The abstract deliberately quotes the earlier cohort. Does not account for it. The preceding sentence introduces "338 (338/364, 93%) prospectively enrolled men", and the same sentence pairs the accuracy with this trial's NPV (71/75) using "respectively". Nothing in the abstract tells the reader that 113/161 comes from the earlier IMPROD trial (NCT01864135).
- A different definition, threshold or subgroup. Does not account for it. The Table 3 footnote states that all five metrics use the same cut (Likert 1–2 versus 3–5) and the same reference (Gleason ≥ 3+4). No subgroup of the 338 men with 161 members is reported; 161 is the total of the earlier cohort in Table 3. The 2 × 2 cells rebuilt from Table 2 reproduce Table 3's sensitivity, specificity and NPV and their CIs, and give an accuracy of 213/338 = 63.0%.
- Alternative definitions of SPCa in the supplements. Does not account for it. S1 Table and S2 Table (two alternative SPCa definitions) give MULTI-IMPROD accuracies of 62% (210/338) and 48% (163/338). In both, accuracy equals TP + TN over 338, and neither is 70% or uses 161.
- Rounding. Does not account for it. 70% against 63% is 7.2 percentage points.
- A text-conversion error in the PMC copy. Ruled out. The publisher's HTML and PDF print the same abstract sentence and the same Table 3 values; the PMC and Europe PMC XML carry each value in its own correctly aligned table cell.
- Is Table 3's 217/338 the true count, with 63% the slip? No. 217 is inconsistent with the table's own sensitivity and specificity numerators (142 + 71 = 213) and with 63% (217/338 = 64.2%). The percentage is right and the numerator is off by 4. Likewise the PPV numerator 141 should be 142 (TP), with no effect on the printed 54%.
- Likert 5 count of 121. Table 2 gives 37 + 74 = 111 men with SPCa among 138 at Likert 5, and only 111 makes the per-score counts sum to the 146 SPCa cases used throughout. The figure 121 does appear elsewhere in the paper as the number of SPCa cases detected by targeted biopsy (121/338 and 121/263), but that is a different quantity with different denominators, so it does not reconcile 121/138.
Status
- Version of record. The publisher's HTML (journals.plos.org/plosmedicine, article 10.1371/journal.pmed.1002813) and the publisher's PDF (abstract on page 2, Table 3 on page 8) print the abstract sentence and Table 3 exactly as quoted. The PMC and Europe PMC full-text XML match. Every quote above was checked verbatim against the PMC-archived text.
- Supplements. The paper lists three supplementary files: S1 CONSORT checklist, S1 Table and S2 Table (alternative SPCa definitions). S1 and S2 Table were downloaded from PLOS and parsed; they are discussed under explanation 3 and do not reconcile the abstract figure. The protocols.io record linked from Crossref (doi:10.17504/protocols.io.ynifvce) is the imaging protocol and holds no outcome data; it was not opened.
- Corrections and comments. As of 2026-10-23 no correction, erratum, expression of concern or retraction was found. Crossref lists no update relations for the DOI; PMC and Europe PMC list no correction notices (the PMC record links three related articles of type "commentary", none a correction); a Crossref search for a "Correction:" item with this title under the PLOS prefix found none; the PLOS article page shows no correction notice; a web search found only the original article. PubPeer could not be checked: its API and search page both returned an HTTP 403 (Cloudflare challenge) to automated access.
- Sources. PMC article PMC6546206; publisher page and PDF at journals.plos.org; S1 and S2 Table (PLOS supplementary files); Crossref; Europe PMC; ClinicalTrials.gov registrations NCT02241122 and NCT01864135 as cited in the paper.
Suggested correction
- Abstract, Methods and findings: replace "The accuracy and NPV of IMPROD bpMRI for SPCa were 70% (113/161) and 95% (71/75) (95% CI 87%–98%), respectively." with "The accuracy and NPV of IMPROD bpMRI for SPCa were 63% (213/338) and 95% (71/75) (95% CI 87%–98%), respectively." If a CI for accuracy is wanted, the Wilson 95% CI is 58%–68%. If the authors wish to compare with the earlier trial, say so explicitly (for example, "compared with 70% (113/161) in the pre-validation IMPROD trial").
- Table 3, MULTI-IMPROD column: change "Accuracy 63% (217/338)" to "63% (213/338)" and "PPV 54% (141/263)" to "54% (142/263)".
- Results, "Diagnostic accuracy", first paragraph: change "88% (121/138)" to "80% (111/138)" for Likert score 5, as in Table 2.
Recomputation script (python recompute.py.txt)
"""Recompute the diagnostic-accuracy figures of PMC6546206
(PLoS Med 2019, doi:10.1371/journal.pmed.1002813, the MULTI-IMPROD trial).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
fig1.svg/png provenance diagram: which cohort each number in the abstract sentence comes from
fig2.svg/png accuracy: abstract value vs the value recomputed for the 338-man cohort, with 95% CIs
candidates/cand1..cand5.svg/png five alternative figure designs for review
(cand1 = fig1, cand2 = fig2)
Inputs are copied from the paper as archived in PMC:
Table 2: men per IMPROD bpMRI Likert score (1..5) by biopsy outcome, MULTI-IMPROD cohort (n = 338)
Table 3: diagnostic metrics for MULTI-IMPROD (n = 338) and the earlier IMPROD trial (n = 161)
Test positive = Likert 3-5; reference positive = Gleason score >= 3+4 (SPCa), as the Table 3 footnote states.
"""
import os
from math import sqrt
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from scipy.stats import beta # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (from the paper)
# Table 2, columns = Likert 1, 2, 3, 4, 5
NO_CANCER = [31, 33, 42, 19, 6]
GG1 = [1, 6, 16, 17, 21] # Gleason 3+3
GG2 = [2, 2, 4, 13, 37] # Gleason 3+4
GG3P = [0, 0, 4, 10, 74] # Gleason > 3+4
TOTAL = [34, 41, 66, 59, 138]
# Table 3, as printed: (numerator, denominator, printed %)
T3_MULTI = {"Sensitivity": (142, 146, 97), "Specificity": (71, 192, 37), "NPV": (71, 75, 95),
"PPV": (141, 263, 54), "Accuracy": (217, 338, 63)}
T3_IMPROD = {"Sensitivity": (79, 83, 95), "Specificity": (34, 78, 44), "NPV": (34, 38, 90),
"PPV": (79, 123, 64), "Accuracy": (113, 161, 70)}
T3_MULTI_CI = {"Sensitivity": (93, 99), "Specificity": (31, 44), "NPV": (87, 98), "PPV": (48, 60)}
# Abstract, Methods and findings
ABS_ACC = (113, 161, 70)
ABS_NPV = (71, 75, 95)
ABS_NPV_CI = (87, 98)
# Results, "Diagnostic accuracy", paragraph 1 (Likert 5)
TEXT_L5 = (121, 138, 88)
TB_SPCA = 121 # Results, "Comparison of TB with SB": TB detected SPCa in 121/338
# ------------------------------------------------------------------ helpers
Z = 1.959964
def wilson(a, n):
p = a / n
c = (p + Z * Z / (2 * n)) / (1 + Z * Z / n)
h = Z * sqrt(p * (1 - p) / n + Z * Z / (4 * n * n)) / (1 + Z * Z / n)
return c - h, c + h
def exact(a, n):
lo = beta.ppf(0.025, a, n - a + 1) if a > 0 else 0.0
hi = beta.ppf(0.975, a + 1, n - a) if a < n else 1.0
return lo, hi
def pct(a, n):
return 100.0 * a / n
def line(label, a, n, ci=True):
s = f" {label:<40s} {a:>4d}/{n:<4d} = {pct(a, n):5.1f}%"
if ci:
w = wilson(a, n)
e = exact(a, n)
s += f" Wilson [{100*w[0]:.1f}-{100*w[1]:.1f}] Clopper-Pearson [{100*e[0]:.1f}-{100*e[1]:.1f}]"
print(s)
# ------------------------------------------------------------------ 1. Table 2 consistency
print("1. Table 2 (MULTI-IMPROD, n = 338)")
for i in range(5):
assert NO_CANCER[i] + GG1[i] + GG2[i] + GG3P[i] == TOTAL[i]
N = sum(TOTAL)
SPCA = [GG2[i] + GG3P[i] for i in range(5)]
print(f" rows sum to column totals; N = {N}; SPCa per Likert 1..5 = {SPCA}, total {sum(SPCA)}")
for i in range(5):
line(f"SPCa, Likert {i+1}", SPCA[i], TOTAL[i], ci=False)
# ------------------------------------------------------------------ 2. 2x2 table, MULTI-IMPROD
TP = sum(SPCA[2:])
FN = sum(SPCA[:2])
FP = sum(TOTAL[2:]) - TP
TN = sum(TOTAL[:2]) - FN
assert TP + FN + FP + TN == N == 338
print(f"\n2. MULTI-IMPROD 2x2 (Likert 3-5 positive, SPCa reference): TP={TP} FN={FN} FP={FP} TN={TN} N={N}")
line("Sensitivity TP/(TP+FN)", TP, TP + FN)
line("Specificity TN/(TN+FP)", TN, TN + FP)
line("NPV TN/(TN+FN)", TN, TN + FN)
line("PPV TP/(TP+FP)", TP, TP + FP)
line("Accuracy (TP+TN)/N", TP + TN, N)
ACC_M = (TP + TN, N)
print(" As printed in Table 3, MULTI-IMPROD column:")
for k, (a, n, p) in T3_MULTI.items():
print(f" {k:<12s} printed {p}% ({a}/{n}); {a}/{n} = {pct(a, n):.1f}%")
print(f" Accuracy numerator printed 217, recomputed {TP+TN}: difference {217-(TP+TN)}")
print(f" PPV numerator printed 141, recomputed {TP}: difference {TP-141}")
# ------------------------------------------------------------------ 3. IMPROD trial column
TP2, FN2 = 79, 83 - 79
TN2, FP2 = 34, 78 - 34
N2 = TP2 + FN2 + TN2 + FP2
assert N2 == 161 and TP2 + FP2 == 123 and TN2 + FN2 == 38
print(f"\n3. IMPROD trial (Table 3 right-hand column): TP={TP2} FN={FN2} FP={FP2} TN={TN2} N={N2}")
line("Accuracy (TP+TN)/N", TP2 + TN2, N2)
line("NPV", TN2, TN2 + FN2)
line("Specificity", TN2, TN2 + FP2)
line("PPV", TP2, TP2 + FP2)
# ------------------------------------------------------------------ 4. The abstract sentence
print("\n4. Abstract: 'accuracy and NPV ... were 70% (113/161) and 95% (71/75) (95% CI 87%-98%)'")
print(f" 113/161 = {pct(113,161):.1f}% = IMPROD trial accuracy ({TP2}+{TN2} = {TP2+TN2}; N = {N2})")
print(f" 71/75 = {pct(71,75):.1f}% = MULTI-IMPROD NPV (Wilson {100*wilson(71,75)[0]:.0f}-{100*wilson(71,75)[1]:.0f}%)")
lo, hi = wilson(*ACC_M)
print(f" MULTI-IMPROD accuracy {ACC_M[0]}/{ACC_M[1]} = {pct(*ACC_M):.1f}%, Wilson 95% CI {100*lo:.1f}-{100*hi:.1f}%;"
f" 70.2% inside CI? {lo <= 113/161 <= hi}")
print(f" Overstatement: {pct(113,161) - pct(*ACC_M):.1f} percentage points")
lo2, hi2 = wilson(113, 161)
print(f" IMPROD accuracy Wilson CI {100*lo2:.1f}-{100*hi2:.1f}%")
# ------------------------------------------------------------------ 5. Side finding (Likert 5)
print("\n5. Results text, Likert 5: printed 88% (121/138)")
print(f" 121/138 = {pct(121,138):.1f}%; Table 2 gives {GG2[4]}+{GG3P[4]} = {SPCA[4]}/138 = {pct(SPCA[4],138):.1f}%")
print(f" With 121 the Likert-group SPCa counts would sum to {sum(SPCA[:4]) + 121}, not {sum(SPCA)}")
print(f" 121 equals the TB-detected SPCa count ({TB_SPCA}/338 = {pct(TB_SPCA,338):.1f}%; {TB_SPCA}/263 = {pct(TB_SPCA,263):.1f}%)")
# ================================================================== figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRAY, GRID = "#eb6834", "#2a78d6", "#8a8984", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"lines.linewidth": 1.5, "svg.fonttype": "path",
})
W = 7.2
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def box(ax, x, y, w, h, edge=GRAY, lw=1.2, fill=BG):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc=fill, ec=edge, lw=lw, transform=ax.transAxes))
def arrow(ax, p0, p1, color, rad=0.0, lw=1.5):
ax.add_patch(FancyArrowPatch(p0, p1, transform=ax.transAxes, arrowstyle="-|>", mutation_scale=13,
color=color, lw=lw, connectionstyle=f"arc3,rad={rad}"))
def blank(h):
fig = plt.figure(figsize=(W, h))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
return fig, ax
m_acc = f"{ACC_M[0]}/{ACC_M[1]} = {pct(*ACC_M):.1f}%"
# ---------------------------------------------------------- candidate 1: provenance diagram (fig1)
def segments(fig, ax, x, y, parts, fs=10.5):
"""Place text pieces left to right; parts = [(text, edgecolor or None)]. Returns x-centres."""
r = fig.canvas.get_renderer()
inv = ax.transAxes.inverted()
centres = []
for txt, ec in parts:
kw = dict(fontsize=fs, va="top", transform=ax.transAxes)
if ec:
kw.update(weight="bold", bbox=dict(boxstyle="round,pad=0.22", fc=BG, ec=ec, lw=2))
t = ax.text(x, y, txt, **kw)
bb = t.get_window_extent(r)
(x0, _), (x1, _) = inv.transform([(bb.x0, bb.y0), (bb.x1, bb.y1)])
centres.append((x0 + x1) / 2)
x = x1 + (0.012 if ec else 0.0)
if ec:
x += 0.004
return centres
def cand1():
fig, ax = blank(4.7)
box(ax, 0.02, 0.68, 0.96, 0.29, edge=INK2)
ax.text(0.04, 0.945, "Abstract (Methods and findings)", fontsize=10.5, weight="bold", va="top")
ax.text(0.04, 0.875, "\u201cIn total, 338 (338/364, 93%) prospectively enrolled men completed the trial.",
fontsize=10.5, va="top")
ax.text(0.04, 0.815, "The accuracy and NPV of IMPROD bpMRI for SPCa were", fontsize=10.5, va="top")
c = segments(fig, ax, 0.04, 0.745, [("70% (113/161)", ORANGE), (" and ", None), ("95% (71/75)", BLUE),
(" (95% CI 87%\u201398%), respectively.\u201d", None)])
acc_x, npv_x = c[0], c[2]
panels = [
(0.52, "Table 3, MULTI-IMPROD column", "this trial: 4 centres, n = 338",
[("NPV", "95% (71/75)", f"recomputed {TN}/{TN+FN} = {pct(TN,TN+FN):.1f}%", True),
("Accuracy", "63% (217/338)", f"recomputed ({TP}+{TN})/{N} = {pct(*ACC_M):.1f}%", False)]),
(0.02, "Table 3, IMPROD trial column", "earlier single-centre cohort, n = 161",
[("NPV", "90% (34/38)", f"recomputed {TN2}/{TN2+FN2} = {pct(TN2,TN2+FN2):.1f}%", False),
("Accuracy", "70% (113/161)", f"recomputed ({TP2}+{TN2})/{N2} = {pct(TP2+TN2,N2):.1f}%", True)]),
]
for x0, title, sub, rows in panels:
box(ax, x0, 0.10, 0.46, 0.40, edge=INK2)
ax.text(x0 + 0.02, 0.47, title, fontsize=10.5, weight="bold", va="top")
ax.text(x0 + 0.02, 0.415, sub, fontsize=10, color=INK2, va="top")
for j, (k, pr, rc, hit) in enumerate(rows):
y = 0.335 - 0.115 * j
ax.text(x0 + 0.02, y, f"{k}: {pr}", fontsize=10.5, va="top", weight="bold" if hit else "normal")
ax.text(x0 + 0.02, y - 0.05, rc, fontsize=10, va="top", color=INK2)
arrow(ax, (acc_x, 0.69), (acc_x, 0.505), ORANGE)
arrow(ax, (npv_x, 0.69), (0.70, 0.505), BLUE, rad=0.1)
ax.text(acc_x + 0.02, 0.59, "accuracy: from the earlier\ntrial (n = 161)", fontsize=10, ha="left", va="center")
ax.text(0.97, 0.59, "NPV: from this trial\n(n = 338)", fontsize=10, ha="right", va="center")
ax.text(0.02, 0.045, f"Accuracy of this trial's 338 men, from its own cells: {m_acc}. "
f"(Table 3 prints 63%, but 217/338 = {pct(217,338):.1f}%.)", fontsize=10, color=INK2, va="center")
return fig
# ---------------------------------------------------------- candidate 2: dot-and-interval (fig2)
def cand2():
from matplotlib.lines import Line2D
rows = [ # label, a, n, kind
("Abstract, presented as\nthis trial: \u201c70% (113/161)\u201d", 113, 161, "rep"),
("This trial (n = 338),\nrecomputed (142 + 71)/338", ACC_M[0], ACC_M[1], "rec"),
("Earlier IMPROD trial\n(n = 161), Table 3", 113, 161, "ctx"),
]
col = {"rep": ORANGE, "rec": BLUE, "ctx": GRAY}
fig, ax = plt.subplots(figsize=(W, 2.9))
ys = list(range(len(rows)))[::-1]
lo_m, hi_m = wilson(*ACC_M)
ax.axvspan(100 * lo_m, 100 * hi_m, color=BLUE, alpha=0.08, lw=0)
for y, (lab, a, n, kind) in zip(ys, rows):
p = pct(a, n)
c = col[kind]
if kind == "rep":
ax.plot(p, y, "o", ms=10, mfc=BG, mec=c, mew=2.2)
ax.text(p + 2.5, y, f"70% as printed (113/161 = {p:.1f}%)", va="center", fontsize=10)
else:
lo, hi = wilson(a, n)
ax.plot([100 * lo, 100 * hi], [y, y], color=c, lw=2, solid_capstyle="round")
ax.plot(p, y, "o", ms=9, color=c)
ax.text(100 * hi + 1.5, y, f"{p:.1f}% [{100*lo:.1f}\u2013{100*hi:.1f}]", va="center", fontsize=10)
ax.text(100 * lo_m + 0.5, len(rows) - 0.45, "95% CI, this trial", fontsize=10, color=INK2, va="top", ha="right")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlim(0, 100)
ax.set_ylim(-0.55, len(rows) - 0.4)
ax.set_xlabel("Overall accuracy for SPCa, % (Likert 3\u20135 vs 1\u20132; lines: Wilson 95% CI)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=ORANGE, mew=2, label="reported in the abstract"),
Line2D([], [], marker="o", ls="-", ms=9, color=BLUE, lw=2, label="recomputed, this trial"),
Line2D([], [], marker="o", ls="-", ms=9, color=GRAY, lw=2, label="earlier cohort (context)")]
ax.legend(handles=h, loc="lower left", bbox_to_anchor=(-0.02, 1.0), frameon=False, fontsize=10, ncol=3,
handletextpad=0.4, columnspacing=1.2)
return fig
# ---------------------------------------------------------- candidate 3: all metrics, both cohorts
def cand3():
metrics = [("Sensitivity", (TP, TP + FN), (TP2, TP2 + FN2)),
("Specificity", (TN, TN + FP), (TN2, TN2 + FP2)),
("NPV", (TN, TN + FN), (TN2, TN2 + FN2)),
("PPV", (TP, TP + FP), (TP2, TP2 + FP2)),
("Accuracy", ACC_M, (TP2 + TN2, N2))]
fig, ax = plt.subplots(figsize=(W, 3.4))
ys = list(range(len(metrics)))[::-1]
for y, (k, m, i) in zip(ys, metrics):
pm, pi = pct(*m), pct(*i)
ax.plot([pm, pi], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(pi, y, "s", ms=8, color=GRAY, zorder=2)
ax.plot(pm, y, "o", ms=9, color=BLUE, zorder=3)
# abstract values
ax.plot(pct(*ABS_NPV[:2]), ys[2], "o", ms=16, mfc="none", mec=ORANGE, mew=2, zorder=4)
ax.plot(pct(*ABS_ACC[:2]), ys[4], "o", ms=16, mfc="none", mec=ORANGE, mew=2, zorder=4)
ax.annotate("abstract NPV 95% (71/75)\n= this trial's value", (pct(71, 75), ys[2]), xytext=(72, ys[2] - 0.45),
fontsize=10, color=INK, ha="right", va="center",
arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.5))
ax.annotate("abstract accuracy 70% (113/161)\n= the earlier trial's value", (pct(113, 161), ys[4]),
xytext=(84, ys[4] + 0.55), fontsize=10, color=INK, ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1.5))
ax.set_yticks(ys)
ax.set_yticklabels([m[0] for m in metrics], color=INK)
ax.set_xlim(0, 100)
ax.set_ylim(-0.6, len(metrics) - 0.3)
ax.set_xlabel("% (Likert 3–5 vs 1–2, SPCa = Gleason ≥ 3+4)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D
h = [Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="MULTI-IMPROD (this trial, n = 338), recomputed"),
Line2D([], [], marker="s", ls="", ms=8, color=GRAY, label="IMPROD trial (earlier, n = 161), Table 3"),
Line2D([], [], marker="o", ls="", ms=12, mfc="none", mec=ORANGE, mew=2, label="value in the abstract")]
ax.legend(handles=h, loc="lower left", bbox_to_anchor=(0, 1.0), frameon=False, fontsize=10, ncol=2)
return fig
# ---------------------------------------------------------- candidate 4: calculation flow
def cand4():
fig, ax = blank(4.6)
top = [
(0.02, "1. Table 2 (n = 338)", ["SPCa, Likert 1\u20135:", " " + ", ".join(map(str, SPCA)),
"men, Likert 1\u20135:", " " + ", ".join(map(str, TOTAL))], INK2),
(0.355, "2. Cut at Likert \u2265 3", [f"TP {TP} FN {FN}", f"FP {FP} TN {TN}",
"reproduces Table 3 sens,", "spec, NPV and their CIs"], INK2),
(0.69, "3. Accuracy", [f"({TP} + {TN}) / {N}", f"= {TP+TN}/{N} = {pct(TP+TN,N):.1f}%",
"Wilson 95% CI", f"{100*wilson(TP+TN,N)[0]:.1f}\u2013{100*wilson(TP+TN,N)[1]:.1f}%"], BLUE),
]
for x0, t, body, c in top:
box(ax, x0, 0.56, 0.29, 0.41, edge=c, lw=2.2 if c == BLUE else 1.2)
ax.text(x0 + 0.02, 0.935, t, weight="bold", va="top", fontsize=10.5)
for j, b in enumerate(body):
ax.text(x0 + 0.02, 0.85 - 0.068 * j, b, va="top", fontsize=10,
color=INK if c == BLUE or j % 2 else INK2)
arrow(ax, (0.312, 0.765), (0.353, 0.765), INK2)
arrow(ax, (0.647, 0.765), (0.688, 0.765), INK2)
bottom = [
(0.355, "Table 3 prints", ["63% (217/338)", f"but 217/338 = {pct(217,338):.1f}%",
f"count {217-(TP+TN)} too high"]),
(0.69, "Abstract prints", ["70% (113/161)", f"= earlier trial's {TP2}+{TN2}",
f"{pct(113,161)-pct(TP+TN,N):.1f} points higher"]),
]
for x0, t, body in bottom:
box(ax, x0, 0.13, 0.29, 0.33, edge=ORANGE, lw=2.2)
ax.text(x0 + 0.02, 0.425, t, weight="bold", va="top", fontsize=10.5)
for j, b in enumerate(body):
ax.text(x0 + 0.02, 0.34 - 0.07 * j, b, va="top", fontsize=10, color=INK if j == 0 else INK2)
arrow(ax, (0.80, 0.56), (0.55, 0.465), INK2, rad=-0.1)
arrow(ax, (0.835, 0.56), (0.835, 0.465), INK2)
ax.text(0.02, 0.40, "compare step 3\nwith the printed\nvalues (orange)", fontsize=10, color=INK2, va="top")
ax.text(0.02, 0.04, "Blue: recomputed from the paper's own Table 2 counts. Orange: values as printed.",
fontsize=10, color=INK2, va="bottom")
return fig
# ---------------------------------------------------------- candidate 5: the two 2x2 tables
def cand5():
fig, axs = plt.subplots(1, 2, figsize=(W, 3.6))
for ax, (title, tp, fn, fp, tn, col, note) in zip(axs, [
("MULTI-IMPROD (this trial), n = 338", TP, FN, FP, TN, BLUE, f"accuracy {TP+TN}/{TP+FN+FP+TN} = {pct(TP+TN,TP+FN+FP+TN):.1f}%"),
("IMPROD trial (earlier), n = 161", TP2, FN2, FP2, TN2, GRAY, f"accuracy {TP2+TN2}/{N2} = {pct(TP2+TN2,N2):.1f}%")]):
ax.set_xlim(0, 2)
ax.set_ylim(0, 2.75)
ax.set_axis_off()
cells = {(0, 1): ("TP", tp), (1, 1): ("FP", fp), (0, 0): ("FN", fn), (1, 0): ("TN", tn)}
for (cx, cy), (k, v) in cells.items():
diag = k in ("TP", "TN")
ax.add_patch(FancyBboxPatch((cx + 0.05, cy + 0.05), 0.9, 0.9, boxstyle="round,pad=0,rounding_size=0.04",
fc=col if diag else BG, ec=col, lw=2, alpha=0.18 if diag else 1))
ax.add_patch(FancyBboxPatch((cx + 0.05, cy + 0.05), 0.9, 0.9, boxstyle="round,pad=0,rounding_size=0.04",
fc="none", ec=col, lw=2))
ax.text(cx + 0.5, cy + 0.6, k, ha="center", fontsize=10, color=INK2)
ax.text(cx + 0.5, cy + 0.3, str(v), ha="center", fontsize=13, weight="bold")
ax.text(0.5, 2.03, "SPCa", ha="center", fontsize=10, color=INK2)
ax.text(1.5, 2.03, "no SPCa", ha="center", fontsize=10, color=INK2)
ax.text(1.0, 2.35, title, ha="center", fontsize=10.5, weight="bold")
ax.text(1.0, -0.12, note, ha="center", va="top", fontsize=10.5)
axs[0].text(-0.05, 1.5, "Likert 3–5", ha="right", va="center", fontsize=10, color=INK2)
axs[0].text(-0.05, 0.5, "Likert 1–2", ha="right", va="center", fontsize=10, color=INK2)
fig.text(0.5, -0.12, "Abstract, for this trial: “accuracy … 70% (113/161)” — the right-hand "
"table's figure, not the left-hand table's.", ha="center", fontsize=10.5, color=INK,
bbox=dict(boxstyle="round,pad=0.35", fc=BG, ec=ORANGE, lw=2))
fig.subplots_adjust(wspace=0.25)
return fig
save(cand1(), os.path.join(HERE, "fig1"), os.path.join(CAND, "cand1_provenance"))
save(cand2(), os.path.join(HERE, "fig2"), os.path.join(CAND, "cand2_dot_interval"))
save(cand3(), os.path.join(CAND, "cand3_all_metrics"))
save(cand4(), os.path.join(CAND, "cand4_calculation_flow"))
save(cand5(), os.path.join(CAND, "cand5_two_by_two"))
print("\nWrote fig1, fig2 and candidates/cand1..cand5 (.svg, .png) in", HERE)
The abstract's univariate diabetes odds ratio is inverted, and its p-value disagrees with Fig 5
PLoS Med (2019) · doi:10.1371/journal.pmed.1002961 · PMC6822721
Summary
The abstract and the Results say that type 2 diabetes is a risk factor for being a tuberculosis transmitter. They report the univariate result as "odds ratio 0.19 [95% CI 0.02–1.10], p < 0.003". Fig 5 of the same paper gives the counts behind this comparison: 5 of 23 transmitters and 3 of 61 other clustered cases had diabetes. It prints OR 5.14 (0.90–36.45) and p = 0.03 for them. Recomputed from those counts, the OR for transmitters vs others is 5.24 (exact 95% CI 0.92–37.08) and the two-sided Fisher p is 0.032. The text's 0.19 (0.02–1.10) is that estimate with the groups swapped (0.191, 0.027–1.089). So the text gives an OR pointing the opposite way from its own word "enriched", and a p-value ten times smaller than the counts support. This is the only risk factor named in the abstract. At the univariate level, the counts support a borderline association (p = 0.032, exact CI just including 1), not one with p < 0.003.
What the paper reports
Abstract, "Methods and findings":
Our data suggest that type 2 diabetes mellitus is a risk factor associated with being a transmitter (odds ratio 0.19 [95% CI 0.02–1.10], p < 0.003).
Results, section "Identification of transmitters allows association of risk factors to transmission", second paragraph:
Transmitters were significantly enriched in diabetic patients in both univariate (Fisher’s exact test; OR 0.19 [95% CI 0.02–1.10], p < 0.003) and multivariate (logistic regression; OR 23.77 [95% CI 2.53–339.69], p < 0.009) statistical analyses.
The same section, first paragraph, defines the comparison:
We combined likely transmitter cases together with the index cases predicted by TransPhylo (n = 23) and compared them to the other clustered cases (n = 61).
Methods, "Statistical analysis":
We selected the index cases and the samples with higher than 0.6 posterior probability of being transmitters as predicted by TransPhylo (23 transmitters compared to the remaining 84 clustered cases), with sensitivity analysis of the later threshold in S1 Text and S1 Table. Then, we computed the odds ratio (OR) and 95% confidence intervals (Fisher’s exact test) to explore epidemiological variables associated with being a transmitter.
Fig 5, Diabetes row. Fig 5 is an image, so this row was read from the publisher's figure file (journal.pmed.1002961.g005). Its columns are All Patients (n=84), Transmitter (n=23), Other samples (n=61), Odd ratio (95% IC), p-value*:
Diabetes | 8 (9%) | 5 (22%) | 3 (5%) | 5.14 (0.90-36.45) | 0.03
Fig 5 caption:
The figure shows estimated odd ratios for each risk factor tested. *Fisher’s exact test. Comparisons were made between transmitter cases and the rest of the clustered samples.
Discussion, fourth paragraph:
Still, our data suggest that certain risk and epidemiological factors are enriched among the transmitters, while others are depleted.
What the paper's own numbers show
Method. I took the diabetes counts from Fig 5 and rebuilt the 2×2 table: 5 of 23 transmitters and 3 of 61 other clustered cases had diabetes, 8 of 84 in all. For transmitters vs others, and again with the groups swapped, I computed the conditional maximum-likelihood OR with its central exact (Cornfield) 95% CI. These are the quantities R's fisher.test prints. I also computed the two-sided Fisher exact p. As a separate check that does not use the counts, I back-calculated a log-Wald p from the text's own interval, 0.02–1.10. I searched every 23-vs-61 table with more diabetes among transmitters for tables that would give Fisher p < 0.003. I did the same Wald back-calculation for the multivariate result, which I cannot recompute without individual data. Every number below is printed by recompute.py.txt (scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Diabetic transmitters | Fig 5 | 5 (22%) of 23 | 5/23 = 21.7% |
| Diabetic other clustered cases | Fig 5 | 3 (5%) of 61 | 3/61 = 4.9% |
| OR, transmitters vs others | Fig 5 | 5.14 (0.90–36.45) | conditional MLE 5.24 (exact 95% CI 0.92–37.08); sample OR 5.37. The median-unbiased estimate is 5.15, closer to the printed 5.14 |
| OR as given in the text | Abstract; Results | 0.19 (0.02–1.10) | Groups swapped (others vs transmitters): 0.191 (0.027–1.089). Equivalently 1/5.14 = 0.195, 1/36.45 = 0.027, 1/0.90 = 1.11 |
| Direction implied by the text's OR | Abstract; Results | "risk factor", "significantly enriched" | With others at 4.9%, an OR of 0.19 would mean 0.97% of transmitters had diabetes (0.22 of 23 people), not 21.7% |
| Fisher exact p | Fig 5 | 0.03 | two-sided 0.0324; one-sided 0.0324 |
| Fisher exact p | Abstract; Results | p < 0.003 | 0.0324 from the Fig 5 counts, about ten times larger |
| p compatible with the text's own CI | Abstract; Results | p < 0.003 | log-Wald p from 0.19 (0.02–1.10) ≈ 0.10. A central exact 95% CI that includes 1 means each one-sided exact p is at least 0.025, so the two-sided Fisher p is at least 0.025 |
| Counts needed for Fisher p < 0.003 | derived | — | With 3 of 61 diabetic others, at least 8 of 23 transmitters (Fig 5: 5). With 5 diabetic transmitters, 0 of 61 others (Fig 5: 3) |
| Multivariate logistic OR | Results | 23.77 (2.53–339.69), p < 0.009 | Not recomputable. Wald p from the CI: 0.011 using 23.77, or 0.007 using the CI's geometric midpoint (29.32). Consistent with the reported p and with OR > 1 |
| Size of the comparison group | Methods vs Results, Fig 5 | "remaining 84" vs n = 61 | 23 + 61 = 84, so 84 is the total, not the comparison group |
Figure
Odds ratios for diabetes with 95% confidence intervals on a log scale; the thin vertical line marks OR = 1 (no association). Orange: values printed in the paper (abstract and Results text, Fig 5, and the multivariate model). Blue: values recomputed from Fig 5's counts (5/23 transmitters vs 3/61 other clustered cases), once with transmitters as the index group and once with the groups swapped. The text's univariate interval coincides with the swapped-group recomputation. Fig 5's interval coincides with the transmitters-vs-others recomputation. The p-value printed next to the text's interval (p < 0.003) matches neither.
The same comparison on a p-value scale (log scale); the thin vertical line marks 0.05. Orange: the text's "p < 0.003" (the arrow shows that the bound is an upper bound) and Fig 5's p = 0.03. Blue: the two-sided Fisher p recomputed from Fig 5's counts (0.0324), and the log-Wald p implied by the text's own interval 0.02–1.10 (≈ 0.10). The shaded region (p < 0.025) is excluded for a Fisher test whose exact 95% CI includes 1. The text's value lies inside it.
Why it matters
The diabetes result is the only risk factor for transmission named in the abstract. It also serves as the example for the abstract's wider claim that "factors associated with likely transmitters are different to those of simply being in a transmission cluster". As printed, the univariate statistics cannot all be right together:
- An OR of 0.19 means transmitters had lower odds of diabetes, which is the opposite of the claimed enrichment.
- Its interval includes 1.
- p < 0.003 is incompatible with that interval and with the counts in Fig 5.
A reader who takes the abstract at face value gets a wrong direction and a strength of evidence about ten times greater than the data support. Recomputed from the paper's own counts, the univariate result is OR 5.24, two-sided Fisher p = 0.032, with an exact CI (0.92–37.08) that just includes 1. That is a borderline association resting on 5 diabetic transmitters, not a result with p < 0.003. The text describes the univariate analysis as showing transmitters "significantly enriched". That description holds only at p = 0.032, and the matching exact interval does not exclude 1.
What is not affected:
- Direction. The direction of the association is supported by the Fig 5 counts (22% vs 5%) and by the separate multivariate model (OR 23.77, 2.53–339.69). That model's reported p < 0.009 is close to the 0.011 back-calculated from its interval.
- Main conclusions. The paper's main conclusions do not rest on this number: index cases are often misidentified (14 of 21 clusters), and some transmission occurs before symptom onset (5 of 14).
Explanations considered
- Reversed reference group. This fully accounts for the OR and its interval. Swapping the groups in the Fig 5 table gives 0.191 (0.027–1.089), which matches "0.19 (0.02–1.10)". So does taking the reciprocal of Fig 5's 5.14 (0.90–36.45). It does not make the sentence correct: the text pairs this inverted OR with "risk factor" and "significantly enriched", and Fig 5 itself reports the comparison in the transmitters-vs-others direction.
- A different two-sided or one-sided convention for p. This does not account for p < 0.003. From the Fig 5 counts, the two-sided Fisher p is 0.0324, the one-sided p is 0.0324, and doubling the one-sided p gives 0.065. None is below 0.003. Fig 5 prints 0.03. One possibility is that "0.003" is a misplaced decimal for 0.03, but the paper gives no explanation, so this is an inference.
- A different comparison group (the Methods' "remaining 84"). This does not account for it either. The Results and Fig 5 both use 61 others, and 23 + 61 = 84 is Fig 5's "All Patients" total, so "remaining 84" appears to be a wording slip in the Methods. More generally, any table whose exact 95% CI includes 1, as the printed 0.02–1.10 does, has a Fisher p of at least 0.025. No choice of comparison group can then produce p < 0.003 together with that interval.
- Fisher's p and the exact CI disagreeing near 0.05. This is real, but it does not rescue the text. Fisher's two-sided p (summing tables no more likely than the one observed) and the central exact CI can disagree near the 0.05 boundary. That explains why Fig 5 shows p = 0.03 alongside an interval that just includes 1. It cannot produce a p ten times smaller.
- Small differences between Fig 5 and the recomputation (5.14 vs 5.24; 0.90–36.45 vs 0.92–37.08). These are minor and not part of the claim. They may reflect different software or estimators: the median-unbiased estimate is 5.15. Either way, the interval includes 1 and the estimate is above 1.
- The multivariate result. This is a separate logistic model, consistent in direction and in its own p-value. I cannot recompute it from the paper, and it is not part of the claim.
- Conversion artefact. Ruled out. The same text appears in the publisher's HTML, the publisher's manuscript XML, PMC and Europe PMC, and the Fig 5 values come from the publisher's own image file.
Two further observations, which do not affect the claim:
- Fig 5's HIV row repeats the Diabetes row's counts, 5 (22%) and 3 (5%), but with OR 1.01, and its "All Patients" entry for HIV is 7 rather than 8.
- In the comparison with non-clustered cases, the Results give transmitters as 27% diabetic. Neither 5/23 (21.7%) nor 6/23 (26.1%) rounds to 27%.
Status
- Version of record. The errors are in the version of record. I checked the PLOS Medicine article page (journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002961), the publisher's manuscript XML, PMC (PMC6822721), Europe PMC full-text XML and the locally archived PMC text. All carry the same abstract and Results wording. The Fig 5 values were read from the publisher's figure file (journal.pmed.1002961.g005).
- Supplements. I listed all 19 supplementary files (S1–S13 Figs, S1–S4 Tables, S1–S2 Text) and opened S1–S4 Tables and S1 Text. None gives the diabetes 2×2 table or an odds ratio for diabetes. S3 Table lists diabetes in 13 of the 109 cases with epidemiological data. S1 Table is a meta-analysis of clock rates, although the Methods cite it for the threshold sensitivity analysis.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no notices. The PLOS article page shows "Reader Comments (0)". The PubPeer API asked for human verification, and the PubPeer search page refused automated access (HTTP 403), so I could not check PubPeer directly. A web search found no PubPeer page for the article.
- Sources. PMC6822721; https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002961 (HTML, manuscript XML, Fig 5 image g005, supplementary files); Europe PMC; Crossref; PubPeer.
Suggested correction
In the Abstract and in the Results ("Identification of transmitters..."), the univariate result should be reported in the same direction as the sentence and as Fig 5. Using Fig 5's own values, the replacement would be "odds ratio 5.14 [95% CI 0.90–36.45], p = 0.03". From the Fig 5 counts, the conditional maximum-likelihood estimate is "OR 5.24 [95% CI 0.92–37.08], p = 0.032".
Because that interval includes 1, the Results' "significantly enriched ... in both univariate ... and multivariate" analyses could be reworded. For example: diabetes was more frequent among transmitters (5/23 vs 3/61; Fisher's exact p = 0.03, OR 5.14, 95% CI 0.90–36.45) and was significantly associated in the multivariate model.
In the Methods, "the remaining 84 clustered cases" should read "the remaining 61 clustered cases (84 in total)".
The authors hold the individual data and could confirm these values, and the Fig 5 HIV row, against it.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the diabetes risk-factor statistics in PMC6822721
(PLoS Med 2019, doi:10.1371/journal.pmed.1002961) from the paper's own Fig 5 counts,
and regenerate the figures in this folder.
Run: python recompute.py.txt
Needs numpy, scipy, matplotlib.
Inputs (all from the version of record):
Fig 5, Diabetes row: transmitters 5 (22%) of n = 23; other clustered samples 3 (5%) of n = 61;
odds ratio 5.14 (0.90-36.45); Fisher's exact p = 0.03
Abstract and Results text: OR 0.19 [95% CI 0.02-1.10], p < 0.003 (Fisher's exact test)
Results text, multivariate logistic regression: OR 23.77 [95% CI 2.53-339.69], p < 0.009
"""
import os
import numpy as np
from scipy.stats import fisher_exact, nchypergeom_fisher as NCH, norm
from scipy.optimize import brentq
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs
TX_DIAB, TX_N = 5, 23 # Fig 5, transmitters
OT_DIAB, OT_N = 3, 61 # Fig 5, other clustered samples
FIG5 = dict(or_=5.14, lo=0.90, hi=36.45, p=0.03)
TEXT = dict(or_=0.19, lo=0.02, hi=1.10, p=0.003) # p printed as "p < 0.003"
MULTI = dict(or_=23.77, lo=2.53, hi=339.69, p=0.009) # p printed as "p < 0.009"
# ---------------------------------------------------------------- helpers
def margins(a, b, c, d):
return a + b + c + d, a + b, a + c # total, row-1 total, column-1 total
def cond_mle(a, b, c, d):
"""Conditional maximum-likelihood OR (the estimate R's fisher.test prints)."""
M, n, N = margins(a, b, c, d)
return np.exp(brentq(lambda t: NCH.mean(M, n, N, np.exp(t)) - a, -20, 20))
def exact_ci(a, b, c, d, alpha=0.05):
"""Central exact (Cornfield) CI, as printed by R's fisher.test."""
M, n, N = margins(a, b, c, d)
lo = np.exp(brentq(lambda t: NCH.sf(a - 1, M, n, N, np.exp(t)) - alpha / 2, -20, 20))
hi = np.exp(brentq(lambda t: NCH.cdf(a, M, n, N, np.exp(t)) - alpha / 2, -20, 20))
return lo, hi
def median_unbiased(a, b, c, d):
M, n, N = margins(a, b, c, d)
f = lambda t: (NCH.cdf(a, M, n, N, np.exp(t)) - 0.5 * NCH.pmf(a, M, n, N, np.exp(t))) - 0.5
return np.exp(brentq(f, -20, 20))
def wald_p_from_ci(est, lo, hi):
se = (np.log(hi) - np.log(lo)) / (2 * norm.ppf(0.975))
z = np.log(est) / se
return se, z, 2 * norm.sf(abs(z))
def r(x, k=3):
return float(np.round(x, k))
# ---------------------------------------------------------------- numbers
print("=== Fig 5 diabetes counts ===")
a, b = TX_DIAB, TX_N - TX_DIAB
c, d = OT_DIAB, OT_N - OT_DIAB
print(f"transmitters diabetic {a}/{TX_N} = {a/TX_N:.1%}; other clustered {c}/{OT_N} = {c/OT_N:.1%}; "
f"all {a+c}/{TX_N+OT_N} = {(a+c)/(TX_N+OT_N):.1%}")
print(f"group sizes: {TX_N} + {OT_N} = {TX_N+OT_N} (Fig 5 'All Patients (n=84)')")
print("\n=== Transmitters vs others (direction stated in the text: 'enriched') ===")
sample_or = (a * d) / (b * c)
mle = cond_mle(a, b, c, d)
lo, hi = exact_ci(a, b, c, d)
mue = median_unbiased(a, b, c, d)
p2 = fisher_exact([[a, b], [c, d]])[1]
p_hi = fisher_exact([[a, b], [c, d]], alternative="greater")[1]
print(f"sample OR = {sample_or:.3f}")
print(f"conditional MLE OR = {mle:.3f}, exact 95% CI {lo:.3f}-{hi:.3f}")
print(f"median-unbiased OR = {mue:.3f} (Fig 5 prints 5.14)")
print(f"Fisher exact p: two-sided = {p2:.4f}; one-sided (greater) = {p_hi:.4f}; doubled one-sided = {min(1,2*p_hi):.4f}")
print(f"exact CI includes 1: {lo < 1 < hi}")
print("\n=== Reversed coding: others vs transmitters ===")
mle_r = cond_mle(c, d, a, b)
lo_r, hi_r = exact_ci(c, d, a, b)
print(f"conditional MLE OR = {mle_r:.3f}, exact 95% CI {lo_r:.3f}-{hi_r:.3f} (text: 0.19 [0.02-1.10])")
print(f"reciprocals of Fig 5: 1/{FIG5['or_']} = {1/FIG5['or_']:.3f}; 1/{FIG5['hi']} = {1/FIG5['hi']:.4f}; "
f"1/{FIG5['lo']} = {1/FIG5['lo']:.3f}")
print(f"reciprocals of text: 1/{TEXT['or_']} = {1/TEXT['or_']:.2f}; 1/{TEXT['hi']} = {1/TEXT['hi']:.3f}; "
f"1/{TEXT['lo']} = {1/TEXT['lo']:.0f}")
print("\n=== What the text's OR would imply ===")
odds_ot = c / d
odds_tx_implied = TEXT["or_"] * odds_ot
pct_implied = odds_tx_implied / (1 + odds_tx_implied)
print(f"with others at {c}/{OT_N} ({c/OT_N:.1%}), OR 0.19 implies {pct_implied:.2%} of transmitters diabetic "
f"= {pct_implied*TX_N:.2f} of {TX_N} people; Fig 5 reports {a}/{TX_N} = {a/TX_N:.1%}")
print("\n=== p-values compatible with the text's CI ===")
se_t, z_t, p_t = wald_p_from_ci(TEXT["or_"], TEXT["lo"], TEXT["hi"])
print(f"log-Wald back-calculation from 0.19 (0.02-1.10): se = {se_t:.3f}, z = {z_t:.3f}, p = {p_t:.3f}")
print("an exact central 95% CI that includes 1 means each one-sided exact p >= 0.025,")
print("so Fisher's two-sided p (which contains a full tail) is >= 0.025; text p < 0.003 cannot hold")
print("\n=== Which 23 vs 61 tables would give Fisher p < 0.003 with more diabetes in transmitters? ===")
hits = []
for x in range(TX_N + 1):
for y in range(OT_N + 1):
if x / TX_N > y / OT_N:
pp = fisher_exact([[x, TX_N - x], [y, OT_N - y]])[1]
if pp < 0.003:
hits.append((x, y, pp))
min_x_given_y3 = min(x for x, y, _ in hits if y == OT_DIAB)
print(f"with 3 diabetic others, transmitters would need at least {min_x_given_y3} of {TX_N} diabetic "
f"(Fig 5: {TX_DIAB}); with {TX_DIAB} diabetic transmitters, others would need "
f"{max(y for x, y, _ in hits if x == TX_DIAB)} of {OT_N}")
print("\n=== Multivariate result (reported only; not recomputable) ===")
se_m, z_m, p_m = wald_p_from_ci(MULTI["or_"], MULTI["lo"], MULTI["hi"])
gm = np.exp((np.log(MULTI["lo"]) + np.log(MULTI["hi"])) / 2)
_, _, p_m2 = wald_p_from_ci(gm, MULTI["lo"], MULTI["hi"])
print(f"Wald from CI with reported point 23.77: se = {se_m:.3f}, p = {p_m:.4f}; "
f"with CI geometric midpoint {gm:.2f}: p = {p_m2:.4f} (reported p < 0.009)")
print("\n=== Non-clustered comparison sentence (observation only) ===")
for k in (5, 6):
print(f"{k}/{TX_N} = {k/TX_N:.1%}", end="; ")
print("text says transmitters 27% diabetic in that comparison")
# ---------------------------------------------------------------- figure style
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
LW, MS = 2.0, 9
def save(fig, name, also=None):
for ext in ("svg", "png"):
kw = {"dpi": 200} if ext == "png" else {}
fig.savefig(os.path.join(CAND, f"{name}.{ext}"), bbox_inches="tight", **kw)
if also:
fig.savefig(os.path.join(HERE, f"{also}.{ext}"), bbox_inches="tight", **kw)
plt.close(fig)
def fmt(x):
return f"{x:.3g}" if x < 1 else (f"{x:.2f}" if x < 100 else f"{x:.0f}")
# ---------------------------------------------------------------- A: OR dot-and-interval
def cand_A():
rows = [ # label, est, lo, hi, color, kind, p-label
("Abstract & Results text\n(univariate, Fisher)", TEXT["or_"], TEXT["lo"], TEXT["hi"], ORANGE, "Reported",
"0.19 (0.02–1.10) p < 0.003"),
("Recomputed, others vs transmitters\n(reversed coding)", mle_r, lo_r, hi_r, BLUE, "Recomputed",
f"{mle_r:.3f} ({lo_r:.3f}–{hi_r:.3f}) p = {p2:.3f}"),
("Fig 5, Diabetes row\n(univariate, Fisher)", FIG5["or_"], FIG5["lo"], FIG5["hi"], ORANGE, "Reported",
"5.14 (0.90–36.45) p = 0.03"),
(f"Recomputed, transmitters vs others\n(Fig 5 counts {a}/{TX_N} vs {c}/{OT_N})", mle, lo, hi, BLUE, "Recomputed",
f"{mle:.2f} ({lo:.2f}–{hi:.2f}) p = {p2:.3f}"),
("Results text\n(multivariate logistic)", MULTI["or_"], MULTI["lo"], MULTI["hi"], ORANGE, "Reported",
"23.77 (2.53–339.69) p < 0.009"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.3))
ys = np.arange(len(rows))[::-1]
for y, (lab, e, l, h, col, kind, pl) in zip(ys, rows):
ax.plot([l, h], [y, y], color=col, lw=LW, solid_capstyle="butt")
ax.plot([e], [y], "o", color=col, ms=MS, mec=BG, mew=1.5,
label=kind if kind not in ax.get_legend_handles_labels()[1] else None)
ax.text(h * 1.25, y, pl, va="center", fontsize=10, color=INK)
ax.axvline(1, color=INK2, lw=0.8, zorder=0)
ax.set_xscale("log")
ax.set_xlim(0.01, 20000)
ax.set_xticks([0.01, 0.1, 1, 10, 100, 1000])
ax.set_xticklabels(["0.01", "0.1", "1", "10", "100", "1000"])
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.7, len(rows) - 0.3)
ax.grid(axis="x", color=GRID, lw=0.6, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio for diabetes, 95% CI (log scale)")
ax.text(0.85, len(rows) - 0.45, "← fewer diabetic transmitters", ha="right", fontsize=10, color=INK2)
ax.text(1.18, len(rows) - 0.45, "more diabetic transmitters →", ha="left", fontsize=10, color=INK2)
ax.legend(loc="lower left", frameon=False, fontsize=10, handletextpad=0.3)
return fig
# ---------------------------------------------------------------- B: p-value scale
def cand_B():
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ax.axvspan(1e-4, 0.025, color="#f1f0ec", zorder=0)
ax.text(1.2e-4, -0.55, "Shaded: Fisher p below 0.025, impossible when the exact\n95% CI of the same test includes 1 (as 0.02–1.10 does)",
fontsize=10, color=INK2, va="center")
rows = [
("Abstract & Results text", TEXT["p"], ORANGE, "p < 0.003", True),
("Fig 5, Diabetes row", FIG5["p"], ORANGE, "p = 0.03", False),
(f"Recomputed Fisher, two-sided\n(Fig 5 counts {a}/{TX_N} vs {c}/{OT_N})", p2, BLUE, f"p = {p2:.4f}", False),
("Recomputed log-Wald p implied\nby the text's CI 0.02–1.10", p_t, BLUE, f"p ≈ {p_t:.2f}", False),
]
ys = np.arange(len(rows))[::-1]
seen = set()
for y, (lab, p, col, pl, less) in zip(ys, rows):
kind = "Reported" if col == ORANGE else "Recomputed"
ax.plot([p], [y], "o", color=col, ms=MS, mec=BG, mew=1.5, zorder=3,
label=None if kind in seen else kind)
seen.add(kind)
if less:
ax.annotate("", xy=(p / 3.5, y), xytext=(p / 1.15, y),
arrowprops=dict(arrowstyle="-|>", color=col, lw=LW))
ax.text(p * 1.35, y + 0.02, pl, va="center", fontsize=10)
ax.axvline(0.05, color=INK2, lw=0.8, zorder=1)
ax.text(0.053, 3.45, "0.05", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(1e-4, 1)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-1.0, len(rows) - 0.3)
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("p-value for diabetes, transmitters vs other clustered cases (log scale)")
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.1, 1]); ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.1", "1"])
ax.legend(loc="lower left", bbox_to_anchor=(0.0, 1.0), ncol=2, frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- C: text vs figure panels
def box(ax, x, y, w, h, title, body, edge, fc="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=fc, ec=edge, lw=1.5))
ax.text(x + 0.02, y + h - 0.025, title, fontsize=10.5, weight="bold", va="top")
ax.text(x + 0.02, y + h - 0.085, body, fontsize=10, va="top", color=INK, linespacing=1.4)
def cand_C():
fig = plt.figure(figsize=(7.2, 5.0)); ax = fig.add_axes([0.01, 0.01, 0.98, 0.98])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
box(ax, 0.0, 0.545, 0.45, 0.445, "Abstract & Results text (printed)",
"\"type 2 diabetes mellitus is a risk\nfactor associated with being a\ntransmitter\"\n"
"\"Transmitters were significantly\nenriched in diabetic patients\"\n\n"
"OR 0.19 [95% CI 0.02–1.10]\np < 0.003", ORANGE)
box(ax, 0.55, 0.545, 0.45, 0.445, "Fig 5, Diabetes row (printed)",
"Transmitters (n = 23): 5 (22%)\nOther samples (n = 61): 3 (5%)\n\n"
"Odds ratio 5.14 (0.90–36.45)\np = 0.03", ORANGE)
ax.add_patch(FancyArrowPatch((0.45, 0.77), (0.55, 0.77), arrowstyle="<|-|>", mutation_scale=14,
color=INK2, lw=LW))
ax.text(0.5, 0.80, "conflict", ha="center", fontsize=10, color=INK2)
box(ax, 0.0, 0.27, 1.0, 0.245, "Recomputed: the direction is reversed",
f"0.19 (0.02–1.10) is the reciprocal of Fig 5: 1/5.14 = {1/FIG5['or_']:.3f},\n"
f"1/36.45 = {1/FIG5['hi']:.3f}, 1/0.90 = {1/FIG5['lo']:.2f}. Others vs transmitters: {mle_r:.3f} ({lo_r:.3f}–{hi_r:.3f}).\n"
"An OR below 1 means fewer diabetic transmitters, not \"enriched\".", BLUE, fc="#eef4fc")
box(ax, 0.0, 0.0, 1.0, 0.245, "Recomputed: the text's p-value is ten times too small",
f"Fisher exact two-sided p from the Fig 5 counts = {p2:.4f}\n"
"(Fig 5 prints 0.03; the text prints p < 0.003). An exact 95% CI that\n"
"includes 1, as 0.02–1.10 does, rules out a Fisher p below 0.025.", BLUE, fc="#eef4fc")
return fig
# ---------------------------------------------------------------- D: what OR 0.19 would imply
def cand_D():
fig, ax = plt.subplots(figsize=(7.2, 2.9))
rows = [
("Other clustered cases\n(Fig 5)", c / OT_N * 100, ORANGE, f"{c}/{OT_N} = {c/OT_N:.1%}"),
("Transmitters\n(Fig 5)", a / TX_N * 100, ORANGE, f"{a}/{TX_N} = {a/TX_N:.1%}"),
("Transmitters, if the text's\nOR 0.19 were correct", pct_implied * 100, BLUE,
f"{pct_implied:.1%} (≈ {pct_implied*TX_N:.1f} of {TX_N} people)"),
]
ys = np.arange(len(rows))[::-1]
seen = set()
for y, (lab, v, col, txt) in zip(ys, rows):
kind = "Reported" if col == ORANGE else "Recomputed (implied by OR 0.19 and the 4.9% in others)"
ax.plot([0, v], [y, y], color=col, lw=LW, alpha=0.5)
ax.plot([v], [y], "o", color=col, ms=MS, mec=BG, mew=1.5, label=None if kind in seen else kind)
seen.add(kind)
ax.text(v + 0.7, y, txt, va="center", fontsize=10)
ax.set_xlim(0, 30)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.grid(axis="x", color=GRID, lw=0.6); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Share with diabetes (%)")
ax.legend(loc="lower left", bbox_to_anchor=(0.0, 1.0), ncol=2, frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- E: calculation flow
def cand_E():
fig = plt.figure(figsize=(7.2, 4.8)); ax = fig.add_axes([0.01, 0.01, 0.98, 0.98])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
def b(x, y, w, h, title, body, edge, fc="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=fc, ec=edge, lw=1.5))
ax.text(x + w / 2, y + h - 0.03, title, ha="center", va="top", fontsize=10.5, weight="bold")
ax.text(x + w / 2, y + h - 0.095, body, ha="center", va="top", fontsize=10, linespacing=1.35)
def arrow(p, q):
ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=13, color=INK2, lw=1.6))
b(0.29, 0.79, 0.42, 0.20, "Fig 5 counts (as printed)",
f"Transmitters: {a} of {TX_N} diabetic\nOther clustered: {c} of {OT_N} diabetic", ORANGE)
b(0.0, 0.43, 0.44, 0.22, "Transmitters vs others",
f"recomputed OR {mle:.2f} ({lo:.2f}–{hi:.2f})\nFig 5 prints 5.14 (0.90–36.45)", BLUE, fc="#eef4fc")
b(0.56, 0.43, 0.44, 0.22, "Others vs transmitters",
f"recomputed OR {mle_r:.3f} ({lo_r:.3f}–{hi_r:.3f})\ntext prints 0.19 (0.02–1.10)", BLUE, fc="#eef4fc")
b(0.29, 0.04, 0.42, 0.25, "Fisher exact, two-sided",
f"recomputed p = {p2:.4f}\nFig 5 prints 0.03\ntext prints p < 0.003", BLUE, fc="#eef4fc")
arrow((0.36, 0.79), (0.22, 0.65)); arrow((0.64, 0.79), (0.78, 0.65)); arrow((0.5, 0.79), (0.5, 0.29))
ax.text(0.27, 0.73, "matches Fig 5", fontsize=10, color=INK2, ha="right", va="center")
ax.text(0.73, 0.73, "matches the text's OR, but\nthe text says \"enriched\"",
fontsize=10, color=INK2, ha="left", va="center")
ax.text(0.515, 0.36, "matches Fig 5, not the text", fontsize=10, color=INK2, ha="left")
ax.text(0.0, 0.0, "Orange outline: as printed. Blue: recomputed in this script.", fontsize=10, color=INK2)
return fig
save(cand_A(), "A_or_intervals", also="fig1")
save(cand_B(), "B_p_values", also="fig2")
save(cand_C(), "C_text_vs_fig5")
save(cand_D(), "D_implied_share")
save(cand_E(), "E_calculation_flow")
print("\nfigures written to", HERE, "and", CAND)
End-study hemoglobin confidence interval is nine times wider than the paper's own SDs allow
PLoS Med (2020) · doi:10.1371/journal.pmed.1003430 · PMC7707571
Summary
This randomized trial of daily vitamin B12 in 600 Nepalese infants reports an end-study difference in hemoglobin, one of its three primary outcomes, of 0.02 g/dL (95% CI −1.33 to 1.37, P = 0.978). The figure appears in the Abstract, in the Results text and in Table 3. With the group SDs (0.8 and 1.0 g/dL), group sizes (283 and 290) and equal-variance Student t test that Table 3 itself reports, the interval is −0.13 to 0.17 and P = 0.79. The printed interval is 9.07 times too wide, while the same calculation reproduces the other 22 rows of Tables 2 and 3. The paper's conclusion of no effect on hemoglobin stands and becomes more precise. But the printed interval cannot exclude effects of more than 1 g/dL in either direction, and any meta-analysis that takes it at face value gives this trial about 1/82 of its proper weight.
What the paper reports
Abstract, Methods and findings:
Furthermore, at the end of the study, the mean difference in hemoglobin concentration was 0.02 g/dL (95% CI: -1.33 to 1.37, P = 0.978), and the difference in the cognitive scaled scores was 0.16 (95% CI: -0.54 to 0.87, P = 0.648).
Results, "End points":
The mean difference in hemoglobin concentration between the groups was 0.02 g/dL (95% CI: -1.33 to 1.37, P = 0.978).
Table 3, column headers and the two hemoglobin rows (vitamin B12 group n = 283, placebo group n = 290; mean ± SD; mean difference (95% CI) (P value)):
Change from baseline … Hemoglobin (g/dL) | 1.0 ± 1.1 | 1.0 ± 1.2 | -0.02 (-0.20 to 0.17) (0.876)
End study … Hemoglobin (g/dL) | 11.6 ± 0.8 | 11.6 ± 1.0 | 0.02 (-1.33 to 1.37) (0.978)
Table 3 footer (the Table 2 footer is identical):
The mean differences, the corresponding 95% CI, and P values were calculated using Student t test assuming equal variances.
Participants and methods, "Statistical analyses":
For the other outcome variables (Bayley-III raw scores, growth, and hemoglobin concentrations), the effects of the intervention were estimated by comparing the changes from baseline to end study between the study groups. The precision of the effect estimates and corresponding P values were calculated using the Student t test assuming equal variances.
Results, first paragraph:
We were able to collect and analyze blood samples for biomarker assessments at end study from 567 children.
Discussion:
As a result, we have precise effect estimates of our primary outcomes, and thus, sufficient power to detect clinically meaningful differences between the treatment groups.
Table 1 gives baseline hemoglobin as 10.6 ± 0.96 g/dL (vitamin B12) and 10.6 ± 0.91 g/dL (placebo).
What the paper's own numbers show
Method. I used the test the paper names: a pooled-SD (equal-variance) Student t test, with sp = √(((n₁−1)s₁² + (n₂−1)s₂²)/(n₁+n₂−2)), SE = sp·√(1/n₁+1/n₂), 95% CI = d ± t₀.₉₇₅,df·SE, and two-sided P from d/SE. The SE implied by a printed interval is its width divided by 2·t₀.₉₇₅,df. I applied this to every row of Table 3 (n = 283/290, df = 571) and Table 2 (n = 283/289, df = 570), using each printed mean difference, so the other rows serve as controls for the method. I then tested rounding of the SDs, a smaller analysed sample, and a g/L-versus-g/dL unit mix-up. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| End-study Hb, mean difference (95% CI), P | Table 3; Abstract; Results | 0.02 (−1.33 to 1.37), P = 0.978 | 0.02 (−0.13 to 0.17), P = 0.792 |
| End-study Hb, SE of the difference | Table 3 (derived) | 0.687 (from the printed CI) | 0.0758 (pooled SD 0.907); ratio 9.07 |
| End-study Hb, unequal-variance (Welch) SE | derived | – | 0.0756, essentially the same |
| End-study Hb, P from the printed CI | derived | P = 0.978 | 0.977: the printed CI and P fit each other |
| Pooled SD that the printed CI would need | derived | SDs 0.8 and 1.0 g/dL | 8.23 g/dL |
| Group size that the printed CI would need | derived | n = 283 and 290 | about 3.5 children per group |
| Largest half-width over SD rounding (0.75–0.85, 0.95–1.05) and n 567 or 573 | derived | 1.35 | 0.158 |
| Change in Hb, mean difference (95% CI), P | Table 3 | −0.02 (−0.20 to 0.17), P = 0.876 | −0.02 (−0.21 to 0.17), P = 0.835; SE ratio 0.98 |
| End-study Hb mean = baseline + change | Tables 1 and 3 | 11.6 | 10.6 + 1.0 = 11.6 in both groups |
| Other 7 rows of Table 3 (length, weight, z-scores) | Table 3 | CIs as printed | SE ratios 0.96–1.05 |
| 14 rows of Table 2 (Bayley-III) | Table 2 | CIs as printed | SE ratios 0.99–1.02, and 1.10 for the socio-emotional raw change |
| Inverse-variance weight, printed vs recomputed SE | derived | – | the recomputed SE gives 82 times the weight |
The method reproduces 22 of the 23 printed intervals in the two tables, to within the rounding of the SDs and mean differences. (The socio-emotional raw-score change, at 1.10, is the only other row outside 0.96–1.05. It is consistent with somewhat fewer children completing that scale, and it is not part of this finding.) The end-study hemoglobin row is off by a factor of nine, although its SDs are similar to those of the change-score row just above it, which does reproduce. Its printed P value is consistent with the printed interval, so the CI and P appear to come from the same wrong standard error.
Figure
Figure 1. Mean difference in hemoglobin (placebo minus vitamin B12, g/dL) with 95% CI, for the end-study comparison (top) and the change from baseline (bottom). Orange: as printed in Table 3, the Abstract and the Results. Blue: recomputed from the group SDs and sample sizes printed in Table 3 with the equal-variance t test the table footer names. The thin vertical line marks no difference. For the change from baseline the two intervals coincide. For the end-study comparison, whose group SDs (0.8 and 1.0) are slightly smaller than the change-score SDs (1.1 and 1.2), the printed interval is nine times wider than the recomputed one.
Figure 2. For each of the 23 rows of Tables 2 and 3, the standard error implied by the printed 95% CI divided by the standard error recomputed from that row's printed SDs and group sizes (log scale; the thin line at 1 marks agreement). The 22 gray points lie between 0.96 and 1.10, which shows that the method and inputs reproduce the paper's own calculations. The end-study hemoglobin row (orange) lies at 9.07.
Why it matters
Hemoglobin is one of the trial's three primary outcomes, and the wrong interval is the one the Abstract quotes. As printed, the interval runs from −1.33 to 1.37 g/dL. It therefore cannot exclude an effect of vitamin B12 on hemoglobin of more than 1 g/dL in either direction, a clinically large difference in a population where 61% to 67% of the infants were anemic at baseline (Table 1). The interval that the paper's own SDs and sample sizes give, −0.13 to 0.17 g/dL, excludes any effect larger than about 0.15 g/dL. The printed interval also undercuts the Discussion's statement that the trial produced "precise effect estimates of our primary outcomes". The corrected interval supports that statement.
The paper's conclusions do not change. The Statistical analyses section says the hemoglobin effect was estimated from change scores, and that row (−0.02, −0.20 to 0.17) is correct. The Abstract's conclusion that supplementation "did not affect neurodevelopment, growth, or hemoglobin concentration" stands, and its policy statement ("Our results do not support widespread vitamin B12 supplementation in marginalized infants from low-income countries") is, for hemoglobin, strengthened rather than weakened by the correct interval.
The readers most likely to be misled are meta-analysts and guideline reviewers who extract the end-study mean difference and its CI from the Abstract or Table 3. The SE they derive (0.687 g/dL) is nine times too large, which gives this 573-child trial about 1/82 of its proper inverse-variance weight. The trial, one of the larger infant B12 trials, would then contribute almost nothing to a pooled estimate of the effect of B12 on hemoglobin. A reader who instead recomputes the SE from the SDs gets a different interval from the one printed, with no way to tell which is right. Crossref lists 37 citing works.
A separate, minor point: the Abstract and the Results both say "Children in both groups grew on an average 12.5 cm (SD: 1.8), and the mean difference was 0.20 cm (95% CI: -0.23 to 0.63, P = 0.354)." In Table 3, 0.20 (−0.23 to 0.63) (0.354) is the end-study length row. The change-in-length row, which the sentence describes, is 0.09 (−0.21 to 0.39) (0.574). Both are null, so the conclusion is unaffected.
Explanations considered
- A different test from the one stated. This does not account for the numbers. Both table footers and the Statistical analyses section name the equal-variance Student t test. The unequal-variance (Welch) SE is 0.0756 against a pooled 0.0758, so the choice of variance assumption does not matter here. The same pooled test reproduces the other 22 rows of Tables 2 and 3.
- Rounding of the printed SDs or mean difference. This does not account for the numbers. Letting the SDs range over 0.75–0.85 and 0.95–1.05 gives a largest half-width of 0.158 g/dL, against the printed 1.35. Rounding of the means (both 11.6) moves the centre of the interval, not its width.
- Fewer children with end-study blood samples. This does not account for the numbers. The Results give 567 children with end-study blood samples, against 573 in the Table 3 header; the half-width becomes 0.158 at most. Even 50 children per group would give 0.379. The printed width would need about 3.5 children per group at the printed SDs.
- A different population or time point. This is unlikely. The row is labelled "End study", and its means are consistent with the rest of the paper: baseline 10.6 (Table 1) plus change 1.0 (Table 3) equals the end-study 11.6 in both groups.
- SDs in g/L (ten times larger) used for the SE while the interval is printed in g/dL. This comes close but does not reproduce the numbers. SDs of 8 and 10 g/L give a half-width of 1.488, and the rounding range 7.5–8.5 and 9.5–10.5 g/L gives 1.41 to 1.57, all above the printed 1.35. The observed ratio is 9.07, not 10. I could not identify a calculation from the printed inputs that gives exactly 1.35; without the trial data, the source of the wrong standard error cannot be determined.
- A conversion or typesetting error in the archived text. This is excluded. The PLOS Medicine article page, the publisher's PDF (Abstract on p. 2; Results and Table 3 on p. 9), the publisher's article XML, the PMC page and the Europe PMC full-text XML all print −1.33 to 1.37 and P = 0.978, with proper minus signs and aligned table cells.
- The value is explained in a supplement. This is excluded; see Status.
Status
- Version of record. The values above are identical in the PLOS Medicine article page (reached from the DOI), the publisher's 15-page PDF, the publisher's article XML, the PMC-archived text and the Europe PMC full-text XML. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py(the PMC text renders the minus signs as hyphens). - Supplements. The article lists 10 supporting files: S1 Fig (Bayley subscales by subgroup), S2 Fig (described as growth and hemoglobin across subgroups, but it shows only height-for-age and weight-for-age Z-score forest plots), S1 Table (B12 status markers), S2 Table (compliance), S3 Table (adverse events), S1 and S2 Text (protocols), S3 and S4 Text (analysis plans) and the CONSORT checklist. None reports the end-study hemoglobin comparison. The data are available only on request with ethics approval, so the interval cannot be recomputed from individual data. The published peer-review history does not raise the hemoglobin interval.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, PMC and Europe PMC list no notices, and the PLOS Medicine article page shows no reader comments and no linked correction. A web search for corrections or comments found none. The PubPeer API and site refused automated access (Cloudflare challenge, HTTP 403), and a web search found no PubPeer thread for the DOI, so PubPeer could not be checked directly.
- Sources. PMC article PMC7707571; Europe PMC full-text XML; the PLOS Medicine article page, PDF, XML, supporting files and comments page for doi:10.1371/journal.pmed.1003430; Crossref; trial registration NCT02272842 (cited in the paper).
Suggested correction
- Recompute the end-study hemoglobin comparison from the trial data and correct the 95% CI and P value in Table 3, in the Abstract and in the Results "End points" paragraph. From the printed SDs and group sizes these are about 0.02 g/dL (95% CI −0.13 to 0.17, P = 0.79).
- Because the Statistical analyses section defines the hemoglobin effect as the change from baseline, consider quoting the change-score result (−0.02 g/dL, 95% CI −0.20 to 0.17, P = 0.876) in the Abstract and Results, or say which of the two is the primary estimate.
- Minor: in the Abstract and Results sentence on growth, either report the change-in-length difference (0.09 cm, 95% CI −0.21 to 0.39, P = 0.574) or state that 0.20 cm is the end-study difference in length.
Recomputation script (python recompute.py.txt)
"""Recompute the end-study hemoglobin confidence interval of PMC7707571
(PLoS Med 2020, doi:10.1371/journal.pmed.1003430, vitamin B12 supplementation in Nepalese infants).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (and identical in the publisher's PDF and XML):
Table 3 (n = 283 vitamin B12, n = 290 placebo), mean +/- SD per group and
"Mean differences (95% CI) (P value)"; footer: "Student t test assuming equal variances".
Table 2 (n = 283 vitamin B12, n = 289 placebo), same layout and footer.
Table 1 baseline hemoglobin: 10.6 +/- 0.96 (B12), 10.6 +/- 0.91 (placebo), n = 300 each.
Results: blood samples at end study from 567 children.
Differences in both tables are placebo minus vitamin B12 (e.g. end-study length 78.4 - 78.2 = 0.20).
Method: pooled-SD (equal-variance) Student t test, as the table footers state:
sp = sqrt(((n1-1) s1^2 + (n2-1) s2^2) / (n1+n2-2)), SE = sp * sqrt(1/n1 + 1/n2),
95% CI = d +/- t(0.975, n1+n2-2) * SE, P = 2 * P(T > |d| / SE).
The SE implied by a printed CI is (hi - lo) / (2 * t(0.975, df)).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
import matplotlib.transforms
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (from the paper)
# (table, label, m_b12, sd_b12, m_plac, sd_plac, diff, lo, hi, P)
T3 = [
("Table 3", "Change: length (cm)", 12.5, 1.8, 12.5, 1.8, 0.09, -0.21, 0.39, 0.574),
("Table 3", "Change: weight (kg)", 2.1, 0.5, 2.1, 0.6, -0.01, -0.10, 0.09, 0.919),
("Table 3", "Change: hemoglobin (g/dL)", 1.0, 1.1, 1.0, 1.2, -0.02, -0.20, 0.17, 0.876),
("Table 3", "End: length (cm)", 78.2, 2.6, 78.4, 2.6, 0.20, -0.23, 0.63, 0.354),
("Table 3", "End: weight (kg)", 9.4, 0.9, 9.4, 1.0, -0.02, -0.18, 0.13, 0.780),
("Table 3", "End: length-for-age z", -1.8, 0.7, -1.7, 0.7, 0.05, -0.06, 0.16, 0.411),
("Table 3", "End: weight-for-height z", -0.7, 0.8, -0.8, 0.8, -0.08, -0.22, 0.05, 0.238),
("Table 3", "End: weight-for-length z", -1.4, 0.7, -1.4, 0.8, -0.04, -0.16, 0.08, 0.529),
("Table 3", "End: hemoglobin (g/dL)", 11.6, 0.8, 11.6, 1.0, 0.02, -1.33, 1.37, 0.978),
]
T2 = [
("Table 2", "Change: cognitive raw", 20.9, 4.3, 20.7, 4.2, 0.16, -0.54, 0.87, 0.648),
("Table 2", "Change: expressive raw", 14.4, 4.7, 14.8, 5.0, 0.35, -0.44, 1.14, 0.387),
("Table 2", "Change: receptive raw", 10.9, 4.0, 11.2, 3.6, -0.31, -0.94, 0.32, 0.334),
("Table 2", "Change: fine motor raw", 13.8, 3.3, 13.7, 2.9, 0.10, -0.41, 0.60, 0.712),
("Table 2", "Change: gross motor raw", 21.4, 5.1, 21.7, 4.7, -0.28, -1.08, 0.52, 0.491),
("Table 2", "Change: socio-emotional raw", 29.4, 14.3, 31.0, 12.0, -1.57, -3.96, 0.82, 0.197),
("Table 2", "End: cognitive composite", 90.5, 8.2, 91.2, 7.3, 0.73, -0.55, 2.02, 0.261),
("Table 2", "End: language composite", 93.0, 12.8, 92.6, 12.6, -0.42, -2.51, 1.66, 0.692),
("Table 2", "End: expressive scaled", 8.6, 2.6, 8.5, 2.4, -0.01, -0.43, 0.41, 0.965),
("Table 2", "End: receptive scaled", 9.0, 2.3, 8.9, 2.5, -0.13, -0.53, 0.27, 0.517),
("Table 2", "End: motor composite", 99.9, 8.7, 100.2, 8.3, 0.32, -1.08, 1.73, 0.652),
("Table 2", "End: fine motor scaled", 10.7, 1.6, 10.9, 1.8, 0.22, -0.06, 0.50, 0.131),
("Table 2", "End: gross motor scaled", 9.2, 2.0, 9.1, 1.7, -0.12, -0.42, 0.19, 0.457),
("Table 2", "End: socio-emotional composite", 104.3, 16.7, 103.4, 17.1, -0.92, -3.70, 1.86, 0.518),
]
N = {"Table 3": (283, 290), "Table 2": (283, 289)}
def pooled(s1, n1, s2, n2):
df = n1 + n2 - 2
sp = np.sqrt(((n1 - 1) * s1 ** 2 + (n2 - 1) * s2 ** 2) / df)
return sp, sp * np.sqrt(1 / n1 + 1 / n2), df
def ttest(d, s1, n1, s2, n2):
sp, se, df = pooled(s1, n1, s2, n2)
t = stats.t.ppf(0.975, df)
return se, d - t * se, d + t * se, 2 * stats.t.sf(abs(d) / se, df), df
def se_from_ci(lo, hi, df):
return (hi - lo) / 2 / stats.t.ppf(0.975, df)
# ------------------------------------------------------------------ 1. every row of Tables 2 and 3
print("=== 1. Pooled-SD t test from the printed SDs and n, every row of Tables 3 and 2 ===")
print(f"{'row':34s} {'printed SE':>10s} {'recomp SE':>9s} {'ratio':>6s} {'recomputed 95% CI':>18s} {'P recomp':>8s} {'P print':>7s}")
RATIOS = []
for tab, lab, m1, s1, m2, s2, d, lo, hi, p in T3 + T2:
n1, n2 = N[tab]
se, l, h, pp, df = ttest(d, s1, n1, s2, n2)
rse = se_from_ci(lo, hi, df)
RATIOS.append((tab, lab, rse / se))
print(f"{tab + ' ' + lab:34s} {rse:10.3f} {se:9.3f} {rse / se:6.2f} ({l:6.2f} to {h:5.2f}) {pp:8.3f} {p:7.3f}")
others = [r for t, lab, r in RATIOS if lab != "End: hemoglobin (g/dL)"]
print(f"All rows except end-study hemoglobin: ratio range {min(others):.2f} to {max(others):.2f} ({len(others)} rows)")
# ------------------------------------------------------------------ 2. end-study hemoglobin in detail
print("\n=== 2. End-study hemoglobin (Table 3; Abstract; Results > End points) ===")
n1, n2 = 283, 290
d_hb, lo_hb, hi_hb, p_hb = 0.02, -1.33, 1.37, 0.978
sp, se_hb, df = pooled(0.8, n1, 1.0, n2)
tcrit = stats.t.ppf(0.975, df)
_, l_hb, h_hb, pr_hb, _ = ttest(d_hb, 0.8, n1, 1.0, n2)
print(f"df = {df}, t(0.975) = {tcrit:.4f}")
print(f"pooled SD = {sp:.3f} g/dL, SE = {se_hb:.4f} g/dL")
print(f"recomputed 95% CI = {l_hb:.3f} to {h_hb:.3f} (half-width {tcrit * se_hb:.3f}, width {2 * tcrit * se_hb:.3f}), P = {pr_hb:.3f}")
welch = np.sqrt(0.8 ** 2 / n1 + 1.0 ** 2 / n2)
print(f"unequal-variance (Welch) SE = {welch:.4f}, half-width {1.96 * welch:.3f}")
rse_hb = se_from_ci(lo_hb, hi_hb, df)
print(f"printed CI {lo_hb} to {hi_hb}: half-width {(hi_hb - lo_hb) / 2:.2f}, midpoint {(hi_hb + lo_hb) / 2:.2f}, implied SE = {rse_hb:.4f}")
print(f"ratio printed SE / recomputed SE = {rse_hb / se_hb:.2f}")
print(f"P implied by the printed CI (d = 0.02, SE {rse_hb:.3f}) = {2 * stats.t.sf(d_hb / rse_hb, df):.3f} (printed P = {p_hb})")
print(f"pooled SD that the printed CI would need = {rse_hb / np.sqrt(1 / n1 + 1 / n2):.2f} g/dL (printed SDs 0.8 and 1.0)")
print(f"inverse-variance weight ratio (recomputed / printed) = {(rse_hb / se_hb) ** 2:.0f}")
# ------------------------------------------------------------------ 3. rounding and sample size
print("\n=== 3. Rounding of the SDs and the number of children with end-study blood samples ===")
max_hw = 0
for a in (0.75, 0.85):
for b in (0.95, 1.05):
for (m, k) in ((283, 290), (280, 287)):
s = pooled(a, m, b, k)[1] * stats.t.ppf(0.975, m + k - 2)
max_hw = max(max_hw, s)
print(f"SDs {a:.2f}/{b:.2f}, n {m}/{k} (total {m + k}): half-width {s:.3f}")
print(f"largest half-width over the rounding range = {max_hw:.3f} (printed 1.35)")
for m in (100, 50):
s = pooled(0.85, m, 1.05, m)[1] * stats.t.ppf(0.975, 2 * m - 2)
print(f"even with only {m} children per group and SDs 0.85/1.05: half-width {s:.3f}")
s_eff = 2 / (rse_hb / sp) ** 2
print(f"group size (equal groups, pooled SD {sp:.3f}) that the printed CI would imply: about {s_eff:.1f} per group")
# ------------------------------------------------------------------ 4. g/L hypothesis
print("\n=== 4. Hypothesis: SE computed with SDs in g/L (10x) while the CI is printed in g/dL ===")
for a, b in ((7.5, 9.5), (8.0, 10.0), (8.5, 10.5)):
s = pooled(a, n1, b, n2)[1]
print(f"SDs {a}/{b} g/L: SE {s:.3f}, half-width {s * tcrit:.3f} (printed 1.35)")
print(f"ratio printed/recomputed SE {rse_hb / se_hb:.2f}, not 10")
# ------------------------------------------------------------------ 5. consistency of the hemoglobin rows
print("\n=== 5. Internal consistency of the other hemoglobin numbers ===")
print("baseline 10.6 (Table 1) + change 1.0 (Table 3) = 11.6 = end-study mean, in both groups")
_, lc, hc, pc, _ = ttest(-0.02, 1.1, n1, 1.2, n2)
print(f"change-score row recomputed: -0.02 ({lc:.3f} to {hc:.3f}), P = {pc:.3f}; printed -0.02 (-0.20 to 0.17), P = 0.876")
# ------------------------------------------------------------------ 6. separate minor observation
print("\n=== 6. Separate, minor: the Abstract growth sentence ===")
print("Abstract/Results: 'grew on an average 12.5 cm (SD: 1.8), and the mean difference was 0.20 cm (-0.23 to 0.63, P = 0.354)'")
print("Table 3: change in length 0.09 (-0.21 to 0.39) (0.574); end-study length 0.20 (-0.23 to 0.63) (0.354)")
print("-> the quoted difference is the end-study length row, not the change in length")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: printed vs recomputed intervals for the two hemoglobin comparisons
rows = [
("End-study hemoglobin", "printed", d_hb, lo_hb, hi_hb, p_hb, ORANGE),
("End-study hemoglobin", "recomputed", d_hb, l_hb, h_hb, pr_hb, BLUE),
("Change from baseline", "printed", -0.02, -0.20, 0.17, 0.876, ORANGE),
("Change from baseline", "recomputed", -0.02, lc, hc, pc, BLUE),
]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
ys = [3.6, 3.0, 1.2, 0.6]
for y, (grp, kind, d, lo, hi, p, c) in zip(ys, rows):
ax.plot([lo, hi], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([lo, lo], [y - 0.09, y + 0.09], color=c, lw=2, zorder=3)
ax.plot([hi, hi], [y - 0.09, y + 0.09], color=c, lw=2, zorder=3)
ax.plot(d, y, "o", ms=8, color=c, mec=BG, mew=1.5, zorder=4)
lab = "printed in paper" if kind == "printed" else "recomputed from printed SDs and n"
ax.text(1.62, y, f"{lab}\n{d:.2f} ({lo:.2f} to {hi:.2f}), P = {p:.3f}".replace("-", "−"), va="center", ha="left",
fontsize=9.5 + 0.5, color=INK if kind == "printed" else INK, linespacing=1.25)
ax.text(-1.95, 4.15, "End study (11.6 ± 0.8 vs 11.6 ± 1.0 g/dL)", fontsize=10.5, color=INK, fontweight="bold")
ax.text(-1.95, 1.75, "Change from baseline (1.0 ± 1.1 vs 1.0 ± 1.2 g/dL)", fontsize=10.5, color=INK, fontweight="bold")
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.set_xlim(-1.95, 1.6)
ax.set_ylim(0.2, 4.4)
ax.set_yticks([])
ax.set_xticks(np.arange(-1.5, 1.51, 0.5))
ax.set_xlabel("Placebo − vitamin B12, hemoglobin (g/dL), mean and 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("left", "right", "top"):
ax.spines[s].set_visible(False)
ax.legend(handles=[Line2D([0], [0], color=ORANGE, lw=2, marker="o", ms=8, label="Printed in the paper (Table 3)"),
Line2D([0], [0], color=BLUE, lw=2, marker="o", ms=8, label="Recomputed, equal-variance t test")],
loc="lower left", bbox_to_anchor=(0.0, -0.42), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.03, right=0.62, top=0.97, bottom=0.27)
save(fig, "fig1")
# Figure 2: printed SE / recomputed SE for every row of Tables 2 and 3
order = [r for r in RATIOS if r[0] == "Table 3"] + [r for r in RATIOS if r[0] == "Table 2"]
fig, ax = plt.subplots(figsize=(7.2, 6.4))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
y = np.arange(len(order))[::-1].astype(float)
y[len([r for r in order if r[0] == "Table 3"]):] -= 1.3
for yi, (tab, lab, r) in zip(y, order):
out = lab == "End: hemoglobin (g/dL)"
ax.plot(r, yi, "o", ms=8 if not out else 9, color=ORANGE if out else INK2, mec=BG, mew=1.2, zorder=3)
if out:
ax.annotate(f"{r:.2f}×\nprinted CI −1.33 to 1.37\nrecomputed −0.13 to 0.17", (r, yi), xytext=(-14, 0),
textcoords="offset points", ha="right", va="center", fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.set_yticks(y)
ax.set_yticklabels([lab for _, lab, _ in order], fontsize=10)
for t in ax.get_yticklabels():
if t.get_text() == "End: hemoglobin (g/dL)":
t.set_color(INK); t.set_fontweight("bold")
ax.set_xscale("log")
ax.set_xlim(0.5, 14)
ax.set_xticks([0.5, 1, 2, 5, 10])
ax.set_xticklabels(["0.5", "1", "2", "5", "10"])
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(1.03, y[0] + 0.9, "printed = recomputed", fontsize=10, color=INK2, va="bottom")
n3 = len([r for r in order if r[0] == "Table 3"])
TR = matplotlib.transforms.blended_transform_factory(ax.transAxes, ax.transData)
ax.text(-0.45, y[0] + 0.9, "Table 3 (growth, hemoglobin)", transform=TR, ha="left", va="bottom", fontsize=10, color=INK, fontweight="bold")
ax.text(-0.45, y[n3] + 0.9, "Table 2 (Bayley-III)", transform=TR, ha="left", va="bottom", fontsize=10, color=INK, fontweight="bold")
ax.set_ylim(y[-1] - 0.8, y[0] + 1.6)
ax.set_xlabel("SE implied by printed 95% CI ÷ SE from printed SDs and n (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("left", "right", "top"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([0], [0], ls="", marker="o", ms=8, color=INK2, label="Other 22 rows (0.96–1.10×)"),
Line2D([0], [0], ls="", marker="o", ms=9, color=ORANGE, label="End-study hemoglobin, as printed")],
loc="lower right", frameon=False, fontsize=10)
fig.subplots_adjust(left=0.33, right=0.97, top=0.98, bottom=0.09)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Abstract and Table 3 assign the same two adjusted hazard ratios to opposite subgroups
PLoS Med (2021) · doi:10.1371/journal.pmed.1003661 · PMC8205172
Summary
In this cohort of 5,300 adults in Taiwan grouped by body mass index (BMI) and serum albumin (SA), the Abstract and the Results report that the lean–malnourished group had a significant adjusted excess risk of heart failure (HF) hospitalization or death (HR 1.78, 95% CI 1.04 to 3.04, p = 0.034) and the obese–well-nourished group did not (HR 1.41, 0.77 to 2.58, p = 0.27). Table 3 prints these same two estimates, digit for digit, on the opposite rows; each HR, CI and p-value is internally consistent (the Wald p recomputed from the CIs is 0.035 and 0.265), so one of the two places attaches them to the wrong groups. The swap decides which intermediate group carries a statistically significant risk, and with it the Abstract's risk ranking and the Discussion statement that low albumin was associated with increased HF events "in both lean and obese individuals", while the main finding (obese–malnourished HR 2.49, 1.43 to 4.34) is the same in both places. The paper's crude counts, the order of its Kaplan–Meier curves and the text's "30.1% and 29.5%" (which equal 47/156 and 46/156, with 156 the event total in the title of S3 Table) fit the text's assignment better than Table 3's, but only the authors' model output can settle which is right.
What the paper reports
Abstract, Methods and findings:
Over a median 3.6 years (interquartile range 2.5 to 4.8 years) of follow-up, the obese–malnourished group had the highest multivariable-adjusted risk of the composite outcome (hazard ratio [HR] 2.49, 95% CI 1.43 to 4.34, p = 0.001), followed by the lean–malnourished (HR 1.78, 95% CI 1.04 to 3.04, p = 0.034) and obese–well-nourished (HR 1.41, 95% CI 0.77 to 2.58, p = 0.27) groups (with lean–well-nourished group as reference). Results were similar when indexed by other anthropometric indices (waist circumference and body fat) and other measures of nutritional status (PNI and GLIM criteria).
Results, Outcomes:
The lean–malnourished (low BMI, low SA) and obese–malnourished (high BMI, low SA) groups had equally high crude event rates (30.1% and 29.5%; Table 3). Compared to the lean–well-nourished group (low BMI, high SA), the obese–malnourished group (high BMI, low SA) showed the highest multivariable-adjusted risk of the composite outcome (hazard ratio [HR] 2.49, 95% CI 1.43 to 4.34), followed by the lean–malnourished (HR 1.78, 95% CI 1.04 to 3.04, p = 0.034) and obese–well-nourished (HR 1.41, 95% CI 0.77 to 2.58; p = 0.27) groups (Table 3). Results were similar when analyzed by other anthropometric indices (waist circumference and body fat) (Tables 3 and 4).
Results, Outcomes, and the titles of Table 3 and of S3 Table (the PNI sensitivity analysis), on the number of composite events:
In the median 3.6 years (IQR 2.5 to 4.8 years) of follow-up, there were 145 events of the composite outcome of HF hospitalization or all-cause mortality.
Association of subgroups of obesity measures and SA with the composite outcome (heart failure hospitalization and all-cause mortality) (n = 145).
S3: Association of subgroups of BMI and PNI on composite outcomes (heart failure hospitalization and all cause mortality) (n=156)
Table 3, "Combined subgroups of BMI and SA" (columns: subgroup | crude events | p interaction | univariate HR (95% CI) | p | multivariate HR (95% CI) | p; the stray comma after the first multivariate CI is in the original):
Lean–malnourished | 26 | | 1.46 (0.84, 2.53) | 0.18 | 1.41 (0.77, 2.58), | 0.27
Obese–well-nourished | 47 | | 3.04 (1.88, 4.91) | <0.001 | 1.78 (1.04, 3.04) | 0.034
Obese–malnourished | 46 | | 5.07 (3.13, 8.22) | <0.001 | 2.49 (1.43, 4.34) | 0.001
The waist circumference (WC) and body fat (BF) blocks of Table 3 list their rows in the same order (low–well-nourished, low–malnourished, high–well-nourished, high–malnourished). Table 1, Table 2 and the supplementary tables use a different column order, with the obese–well-nourished group before the lean–malnourished group.
Table 1, group sizes (lean–well-nourished | obese–well-nourished | lean–malnourished | obese–malnourished):
Total n (N = 5,300) | 2,096 (39%) | 1,369 (25.8%) | 1,154 (21.8%) | 681 (12.8%)
Table 2, N row and LVH row, under the same column headings as Table 1:
N | 2,096 | 1,154 | 1,369 | 681
LVH, percent | 81 (3.9%) | 84 (6.1%) | 102 (8.8%) | 99 (14.5%)
Discussion. The first sentence relies on the lean stratum showing an excess risk. The second comes from a paragraph comparing this cohort with older cohorts (Health ABC, CHS) and is about age and the albumin cutoff:
Of note, malnourished status (low SA) in our study was associated with increased HF events in both lean and obese individuals, suggesting a prognostic role of protein-energy nutrition, even in the general population
Notably, our study adds that nutritional status was an independent predictor of HF outcomes, even in younger individuals (mean age 49.6 years) and with a higher SA cutoff (45 g/L)
Fig 2 of the paper (Kaplan–Meier curves, an image in the article; legend text as printed in the image): panel A has four curves, labeled "Lean well-nourished", "Obese well-nourished", "Lean malnourished" and "Obese malnourished", with numbers at risk at time 0 of 1710, 1122, 887 and 540. Panels B (WC) and C (BF) have the same structure.
What the paper's own numbers show
Method. (1) For each hazard ratio in Table 3 I recovered the standard error of ln(HR) from the printed 95% CI, SE = (ln upper − ln lower)/(2 × 1.96), and computed the two-sided Wald p = 2·(1 − Φ(|ln HR|/SE)), as a check that each HR, CI and p belong together. (2) I compared the text's and Table 3's group labels for each estimate. (3) To see which labeling the paper's counts support, I divided each Table 3 row's crude events by the size of the group it is labeled as (Table 1 for BMI; S1 and S2 Tables for WC and BF), formed the crude risk ratio against the reference group, and compared it with the row's univariate HR. I repeated this with the two middle labels exchanged. With event risks of 1% to 7%, a univariate Cox HR should lie close to the crude risk ratio. (4) I checked the Results' "30.1% and 29.5%" against every Table 3 event count and every denominator from 20 to 6,000, and against the composite-event totals the paper states (145 in the Results, Table 3 and S6 Table; 156 in the title of S3 Table). (5) I checked the Table 2 N row against the Table 2 LVH percentages. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Adjusted HR, lean–malnourished | Abstract; Results › Outcomes vs Table 3 | Text: 1.78 (1.04–3.04), p = 0.034. Table 3: 1.41 (0.77, 2.58), p 0.27 | The two places disagree; the text's tuple is the one Table 3 prints on the obese–well-nourished row |
| Adjusted HR, obese–well-nourished | Abstract; Results › Outcomes vs Table 3 | Text: 1.41 (0.77–2.58), p = 0.27. Table 3: 1.78 (1.04, 3.04), p 0.034 | The two places disagree; the text's tuple is the one Table 3 prints on the lean–malnourished row |
| Adjusted HR, obese–malnourished | Abstract; Results; Table 3 | 2.49 (1.43–4.34); p = 0.001 (Abstract, Table 3; the Results give no p) | Text and table agree; Wald p = 0.0013 |
| p from CI, HR 1.78 (1.04, 3.04) | Table 3 | 0.034 | 0.035 (CI log-midpoint 1.78): consistent |
| p from CI, HR 1.41 (0.77, 2.58) | Table 3 | 0.27 | 0.265 (CI log-midpoint 1.41): consistent |
| p from CI, all other 15 HRs in Table 3 | Table 3 | as printed | all agree with the printed p within rounding |
| Crude risk ratio, row with 26 events (univariate HR 1.46) | Table 3 row 2; Table 1 N | HR 1.46 | 1.82 if this row is lean–malnourished (26/1,154 = 2.25% vs 26/2,096 = 1.24%); 1.53 if obese–well-nourished |
| Crude risk ratio, row with 47 events (univariate HR 3.04) | Table 3 row 3; Table 1 N | HR 3.04 | 2.77 if this row is obese–well-nourished (47/1,369 = 3.43%); 3.28 if lean–malnourished (47/1,154 = 4.07%) |
| Same check, WC block, rows with 26 and 49 events | Table 3; S1 Table N | HR 2.35 and 3.41 | as labeled: RR 1.96 and 4.58; labels exchanged: RR 2.43 and 3.70 |
| Same check, BF block, rows with 24 and 55 events | Table 3; S2 Table N | HR 1.58 and 3.55 | as labeled: RR 1.67 and 3.95; labels exchanged: RR 1.72 and 3.83 |
| HR ÷ crude RR over the 9 non-reference rows | derived | — | 0.74 to 1.20 with Table 3's labels; 0.90 to 0.97 with the middle labels exchanged. The three high-adiposity–malnourished rows, whose labels are not in question, give 0.93, 0.95 and 0.90 |
| "equally high crude event rates (30.1% and 29.5%; Table 3)", lean– vs obese–malnourished | Results › Outcomes | 30.1% and 29.5% | Not event rates under Table 3's labels (26/1,154 = 2.25%, 46/681 = 6.75%), nor shares of the 145 events (17.9% and 31.7%). They equal 47/156 = 30.13% and 46/156 = 29.49%, and 156 is the composite-event total in S3 Table's title. This is the only pair of Table 3 event counts with a common denominator that gives both. As shares of 156, they put 47 events in the group the text calls lean–malnourished; Table 3 gives 47 events to its obese–well-nourished row (and 26, 16.7% of 156, to its lean–malnourished row) |
| Total composite events | Results text; Table 3, S6 Table and S3 Table titles | 145 (Results, Table 3, S6 Table); 156 (S3 Table) | The paper gives two totals 11 events apart and does not explain the difference; Table 3's BMI block sums to 145 |
| N, obese–well-nourished and lean–malnourished columns | Table 2 N row vs Table 1 | Table 2: 1,154 and 1,369. Table 1: 1,369 and 1,154 | Table 2's own LVH percentages need Table 1's sizes (84/1,369 = 6.1%, 102/1,154 = 8.8%; with the Table 2 N row they would be 7.3% and 7.5%), so the Table 2 N row has the two sizes interchanged |
| Fig 2 numbers at risk at time 0 ÷ group N | Fig 2A, 2B; Table 1, S1 Table | 1122 and 887 (BMI); 810 and 927 (WC) | 0.82 and 0.77 (BMI) and 0.82 and 0.75 (WC) with the legend as printed, in line with the other two groups (0.79 to 0.82); 0.97 and 0.65 (BMI) if the legend's middle labels were exchanged. The figure's legend therefore follows the group sizes |
The conflict itself is not in doubt: the text and Table 3 give opposite answers to the question of which group has a significant adjusted HR. For which version is right, the checks that are independent of Table 3's labels point the same way: the text's assignment. Under Table 3's labels, the obese–well-nourished group has a higher crude event risk than the lean–malnourished group (3.43% vs 2.25%), so its Kaplan–Meier curve should lie below the lean–malnourished curve. In the paper's Fig 2A the curve labeled lean malnourished lies below the obese well-nourished curve from about year 1 to about year 5.5, second only to obese malnourished. Its panels 2B and 2C show the same order for the low-adiposity malnourished and high-adiposity well-nourished groups, which is the opposite of what Table 3's event counts for those rows imply. With the middle labels exchanged, the crude risk ratios fit the univariate HRs in all three blocks about as closely as they do for the undisputed rows (Figure 2 below). The text's "30.1% and 29.5%" equal 47/156 and 46/156, where 156 is the composite-event total that S3 Table's title gives, so they read as shares of 156 events rather than event rates; read that way, they put 47 events in the lean–malnourished group and 46 in the obese–malnourished group, while Table 3 gives 47 events to its obese–well-nourished row and 26 to its lean–malnourished row. And in the supplementary analysis that defines malnutrition by the prognostic nutritional index (PNI, which is built mostly from albumin; S3 Table), the lean–malnourished group again has the higher HR: univariate 1.81 (1.09, 3.00) against 1.15 (0.63, 2.12), adjusted 1.45 (0.81, 2.58) against 1.14 (0.56, 2.32). None of these is proof. The KM curves cover the full follow-up while the Cox model was censored at 1 year, crude risks ignore censoring, and the PNI groups are not the SA groups. The 156 total belongs to the PNI analysis, which does not print event counts by group, so 47 and 46 could also be counts for the PNI groups; in that case the two percentages would say nothing about Table 3's labels.
Figures
Figure 1. Left: the adjusted hazard ratios that the Abstract and the Results › Outcomes paragraph give for the two intermediate groups. Right: the multivariate HRs that Table 3 prints for the same group labels. Orange values are printed in the paper. The blue values are two-sided Wald p-values recomputed from Table 3's HR and 95% CI, and they reproduce the printed p-values (0.035 vs 0.034, 0.265 vs 0.27). The crossing arrows connect identical HR, CI and p triplets. The obese–malnourished estimate (bottom) is the same in both places.
Figure 2. Each row is one Table 3 row, named as Table 3 labels it, with its crude event count. Orange: the univariate HR printed in Table 3. Filled blue: the crude risk ratio, (events / group N) ÷ (reference events / reference N), with N taken for the group named by Table 3 (Table 1, S1 Table and S2 Table). Open blue squares: the same ratio if each block's two middle labels are exchanged; the high-adiposity–malnourished rows have no alternative. Log scale; the axis starts at 1, the null value. For the high-adiposity–malnourished rows, whose labels are not in question, the HR lies 5% to 10% below the crude ratio. With Table 3's labels, the BMI and WC middle rows have HR/RR from 0.74 to 1.20 (HR up to 26% below or 20% above the crude ratio). With the labels exchanged, all six middle rows fall in the same 3% to 8% band. The BF block cannot tell the two labelings apart.
Why it matters
The Abstract's ranking of adjusted risk, obese–malnourished > lean–malnourished (significant) > obese–well-nourished (not significant), comes from the text's assignment. Under Table 3's assignment the ranking of the two middle groups reverses: obese–well-nourished becomes significant (1.78, p = 0.034) and lean–malnourished does not (1.41, p = 0.27). The directions are unchanged, since every HR is above 1, but the paper's statistical claims change. The Discussion statement that low albumin "was associated with increased HF events in both lean and obese individuals" depends on the lean stratum showing a significant excess risk; under Table 3's labels, the only significant adjusted association with low albumin among lean participants disappears. The claim that nutritional status "was an independent predictor of HF outcomes" in this younger population may also be affected, depending on which analysis it rests on (the Results also cite an SA-quartile analysis, in which higher SA quartiles had a significantly lower risk).
The question also reaches the waist circumference and body fat analyses, which the Abstract calls "similar". With Table 3's labels as printed, those two blocks show the Table 3 pattern: the low-adiposity malnourished group is not significant (WC 1.78, 0.98–3.24, p = 0.059; BF 1.62, 0.88–2.98, p = 0.12), and the high-adiposity well-nourished group is (WC 1.87, p = 0.025; BF 2.01, p = 0.007). That contradicts the Abstract's pattern for BMI. If Table 3's two middle labels are reversed in all three blocks, as Fig 2 of the paper suggests for all three blocks, and the crude risks for the BMI and WC blocks, the WC and BF blocks would read the other way as well. Either way, a reader cannot currently tell from the paper which intermediate group carries the excess risk under any of the three adiposity measures.
The paper's central result stands in both versions: the obese–malnourished group has the highest adjusted risk (HR 2.49, 1.43–4.34) and is identical in the text and in Table 3. So are the baseline and echocardiographic comparisons of the obese–malnourished group with the lean–well-nourished group. The title, the Author summary and the Conclusions mention only the obese–malnourished group.
Explanations considered
- A conversion or typesetting error in the archived copy. Excluded. The publisher's PDF (Table 3 on page 9), the PLOS HTML text, the Europe PMC full-text XML and the PMC copy carry the same sentences and the same Table 3 cells.
- Rounding, or a slightly different model behind the text. Excluded. The two triplets match to every printed digit, including both CI limits and the p-value, and each is internally consistent (recomputed p 0.035 and 0.265). A different model would not reproduce both triplets exactly with only their labels exchanged.
- The text quotes a sensitivity analysis rather than Table 3. Excluded. The Results sentence cites Table 3. The GLIM analysis (S6 Table) gives lean–malnourished an adjusted HR of 1.78, but with a different CI (1.09, 2.92) and p (0.021), and against a different reference group (all well-nourished). The PNI analysis (S3 Table) gives 1.45 and 1.14. The triplet "1.41 [0.77, 2.58], p=0.27" does appear in S3 Table, but as the univariate HR for the WC–PNI high-waist well-nourished group, not as an adjusted BMI–SA estimate.
- Table 3 is right and the text transposed the two estimates. Possible. Table 3's labels agree with themselves across its three blocks. That agreement is not independent evidence, though: a single row-order error in the table template would affect all three blocks the same way.
- The text is right and Table 3's middle labels are reversed (possibly in all three blocks). Better supported by the paper's other material, but not proven: the Kaplan–Meier order in the paper's Fig 2A–C, the closer fit of crude risk ratios to univariate HRs (Figure 2 of this write-up), the text's "30.1% and 29.5%" (47/156 and 46/156), and the PNI-based ordering in S3 Table. Table 3 is also the only results table with one row or column per group that lists the lean–malnourished group before the obese–well-nourished group (Table 1, Table 2's headings and S1–S3 Tables list obese–well-nourished first).
- A wider confusion between the two group orders. Plausible as a mechanism, and present elsewhere. The Table 2 N row lists the two middle group sizes in Table 3's order under Table 1's headings, which Table 2's own LVH percentages show to be an error. A rough heuristic finds the same pattern in Table 2's multivariable-adjusted means. For 22 of 25 measures, the adjusted values printed for these two groups lie closer to the unadjusted values of the other group. For example, LV mass is 161.0 g unadjusted and 139.8 g adjusted in the obese–well-nourished column, and 134.1 g and 154.1 g in the lean–malnourished column. Adjustment can legitimately move means, so this is noted only as a possibility for the authors to check, not as a finding.
- The "30.1% and 29.5%" are shares of events, not event rates. Likely. S3 Table's title gives n = 156 composite events, while the Results, Table 3 and S6 Table give 145. The text's 30.1% and 29.5% equal 47/156 and 46/156, so they appear to be shares of 156 events rather than event rates, and they put 47 events in the group the text calls lean–malnourished. Table 3 gives 47 events to its obese–well-nourished row. Two points stay open: which analysis has 156 events (the paper states 145 for the whole cohort, and S3 Table prints no event counts by group), and whether 47 and 46 are counts from the SA grouping or from the PNI grouping.
Status
- Version of record. Checked. Table 3 in the publisher's PDF (journals.plos.org printable PDF, page 9, doi:10.1371/journal.pmed.1003661.t003) has the same row labels and values as the PMC copy. The PLOS HTML page has the Abstract and Results sentences word for word; on that page Table 3 is an image. The Europe PMC full-text XML matches both. All quotations above were confirmed verbatim against the PMC-archived text with
src/show.py, except the Fig 2 legend and numbers at risk, which were read from the publisher's Fig 2 image, and the S3 Table title and cells, which were read from the publisher's supplement file (journal.pmed.1003661.s005). - Supplements. All Supporting Information files were opened (S1 STROBE checklist, S1 Fig, S1–S6 Tables). None repeats the BMI × SA Cox model, so none settles which labeling is correct. Three side observations from S3 Table. Its title gives the composite-event total as "(n=156)", against 145 in the Results text and in the titles of Table 3 and S6 Table (see the "30.1% and 29.5%" row above). It contains the triplet "1.41 [0.77, 2.58], p=0.27" in an unrelated cell (see above). And its adjusted entry for BMI–PNI obese–well-nourished, "1.14 [0.56, 2.32], p=0.034", does not agree with itself: the recomputed p is 0.72. The Data Availability statement says the data are available on request from the hospital's ethics committee, so no individual-level data could be checked.
- Corrections. One correction exists: doi:10.1371/journal.pmed.1003784 (PMC8437276), published 2021-09-13. It changes only the first sentence of the Abstract's Methods and findings (removing "prospectively") and the consent statement in the Methods. It does not address Table 3 or the hazard ratios. Crossref and Europe PMC list no other notice, and no retraction or expression of concern.
- Comments. A web search found no PubPeer thread or published comment on this discrepancy. PubPeer's own search and API refused automated access (Cloudflare challenge), so PubPeer could not be checked directly.
- Sources. PMC8205172 (via
src/show.py); publisher PDF, HTML, Fig 2 image and Supporting Information files at journals.plos.org; Europe PMC full-text XML; Crossref update metadata; correction PMC8437276. - As of 2026-10-24.
Suggested correction
- Check the two intermediate-group rows of the BMI × SA Cox model against the original output. Then correct either Table 3's row labels or the Abstract and Results sentences, so that HR 1.78 (1.04–3.04, p = 0.034) and HR 1.41 (0.77–2.58, p = 0.27) are attached to the right groups.
- Check the WC × SA and BF × SA blocks of Table 3 in the same way. If the middle row labels were reversed in one block, they may be reversed in all three, which would change the interpretation of the WC and BF results.
- Revise the Abstract's risk ranking and the Discussion statement on lean participants ("increased HF events in both lean and obese individuals") to match the corrected attribution, and check whether the statement that nutritional status was "an independent predictor of HF outcomes" rests on this analysis or on the SA-quartile analysis.
- Correct the Table 2 N row, which gives the obese–well-nourished and lean–malnourished sizes as 1,154 and 1,369 instead of 1,369 and 1,154. Confirm that the multivariable-adjusted means of Table 2 are printed under the right column headings.
- Reconcile the two composite-event totals (156 in the title of S3 Table, 145 in the Results text and in the titles of Table 3 and S6 Table). State the denominator of the "30.1% and 29.5%", which equal 47/156 and 46/156 and so read as shares of events rather than "crude event rates", say which groups the 47 and 46 events belong to, or replace the figures with event counts and percentages that match Table 3.
- Minor: the p-value for the BMI–PNI obese–well-nourished adjusted HR in S3 Table (1.14 [0.56, 2.32], p = 0.034) should be about 0.72.
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind writeup.md for PMC8205172
(PLoS Med 2021, doi:10.1371/journal.pmed.1003661, obesity x serum albumin and HF hospitalization / death).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (and checked against the publisher PDF):
Abstract and Results > Outcomes: lean-malnourished HR 1.78 (1.04-3.04), p = 0.034;
obese-well-nourished HR 1.41 (0.77-2.58), p = 0.27;
obese-malnourished HR 2.49 (1.43-4.34), p = 0.001
Results > Outcomes: lean-malnourished and obese-malnourished "equally high crude event rates (30.1% and 29.5%; Table 3)";
"there were 145 events of the composite outcome"
Composite-event totals in table titles: Table 3 "(n = 145)"; S6 Table "(n=145)"; S3 Table "(n=156)"
Table 3 (three blocks, rows in the order printed; crude events, univariate HR, multivariate HR)
Table 1 group sizes (BMI x SA); S1 Table (WC x SA) and S2 Table (BF x SA) group sizes
Table 2 N row and LVH counts
S3 Table (PNI x BMI) hazard ratios
Fig 2 (Kaplan-Meier) numbers at risk at time 0, read from the publisher's image
Method for p from CI: for a Cox model, the 95% Wald CI is exp(b +/- 1.96 SE), so
SE = (ln hi - ln lo) / (2 * 1.96), z = ln(HR) / SE and p = 2 * (1 - Phi(|z|)).
"""
import math
import os
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
import matplotlib.ticker as mticker # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.isf(0.025)
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
def wald(hr, lo, hi):
se = (math.log(hi) - math.log(lo)) / (2 * Z975)
z = math.log(hr) / se
return 2 * stats.norm.sf(abs(z)), math.exp((math.log(lo) + math.log(hi)) / 2), se, z
# ------------------------------------------------------------------ inputs from the paper
# Table 3: block -> list of (row label as printed, crude events, univariate (HR, lo, hi, p), multivariate (HR, lo, hi, p))
T3 = {
"BMI": [("Lean–well-nourished", 26, None, None),
("Lean–malnourished", 26, (1.46, 0.84, 2.53, "0.18"), (1.41, 0.77, 2.58, "0.27")),
("Obese–well-nourished", 47, (3.04, 1.88, 4.91, "<0.001"), (1.78, 1.04, 3.04, "0.034")),
("Obese–malnourished", 46, (5.07, 3.13, 8.22, "<0.001"), (2.49, 1.43, 4.34, "0.001"))],
"WC": [("Low WC–well-nourished", 27, None, None),
("Low WC–malnourished", 26, (2.35, 1.36, 4.04, "0.002"), (1.78, 0.98, 3.24, "0.059")),
("High WC–well-nourished", 49, (3.41, 2.13, 5.46, "<0.001"), (1.87, 1.08, 3.23, "0.025")),
("High WC–malnourished", 45, (6.34, 3.93, 10.23, "<0.001"), (2.94, 1.65, 5.25, "<0.001"))],
"BF": [("Low BF–well-nourished", 25, None, None),
("Low BF–malnourished", 24, (1.58, 0.90, 2.78, "0.11"), (1.62, 0.88, 2.98, "0.12")),
("High BF–well-nourished", 55, (3.55, 2.21, 5.70, "<0.001"), (2.01, 1.22, 3.34, "0.007")),
("High BF–malnourished", 26, (3.82, 2.20, 6.63, "<0.001"), (2.20, 1.21, 4.01, "0.01"))],
}
# Group sizes in the order (low-well, low-mal, high-well, high-mal) = Table 3 row order.
# BMI: Table 1 / Abstract; WC: S1 Table; BF: S2 Table.
N = {"BMI": (2096, 1154, 1369, 681), "WC": (2503, 1228, 992, 625), "BF": (2161, 1241, 1205, 529)}
# Fig 2 numbers at risk at time 0, legend order (low-well, high-well, low-mal, high-mal)
AT_RISK = {"BMI": (1710, 1122, 887, 540), "WC": (2021, 810, 927, 500), "BF": (1753, 963, 936, 417)}
TEXT = {"Lean–malnourished": (1.78, 1.04, 3.04, "0.034"), "Obese–well-nourished": (1.41, 0.77, 2.58, "0.27"),
"Obese–malnourished": (2.49, 1.43, 4.34, "0.001")}
out = {}
print("=== 1. Abstract / Results text vs Table 3, multivariable HRs (BMI x SA) ===")
for g, (hr, lo, hi, p) in TEXT.items():
t3 = [r for r in T3["BMI"] if r[0] == g][0][3]
print(f" {g:22s} text: {hr} ({lo}-{hi}), p = {p:6s} | Table 3: {t3[0]} ({t3[1]}, {t3[2]}), p = {t3[3]}"
f" -> {'agree' if (hr, lo, hi) == t3[:3] else 'DIFFERENT'}")
print("\n=== 2. Wald p back-calculated from each printed HR and 95% CI ===")
for b, rows in T3.items():
for lab, ev, uni, mul in rows[1:]:
for kind, t in (("univariate", uni), ("multivariate", mul)):
p, mid, se, z = wald(*t[:3])
out[(b, lab, kind)] = p
print(f" {b:3s} {lab:24s} {kind:12s} {t[0]} ({t[1]}, {t[2]}): p from CI = {p:.3g} "
f"(printed {t[3]}); CI log-midpoint {mid:.2f}")
P178, P141 = wald(1.78, 1.04, 3.04)[0], wald(1.41, 0.77, 2.58)[0]
print(f" -> the 1.78 block gives p = {P178:.3f}, the 1.41 block p = {P141:.3f}: each block is internally consistent")
print("\n=== 3. Crude risk ratio (events / N, vs reference) under each labelling of the two middle rows ===")
print(" Table 3 labels: rows as printed; exchanged: the two middle rows' events belong to the other group")
crude = {}
for b, rows in T3.items():
n_lw, n_lm, n_hw, n_hm = N[b]
ref = rows[0][1] / n_lw
ev_r2, ev_r3, ev_r4 = rows[1][1], rows[2][1], rows[3][1]
uni = [rows[i][2][0] for i in (1, 2, 3)]
printed = [ev_r2 / n_lm / ref, ev_r3 / n_hw / ref, ev_r4 / n_hm / ref]
exch = [ev_r2 / n_hw / ref, ev_r3 / n_lm / ref, ev_r4 / n_hm / ref]
crude[b] = (uni, printed, exch, [rows[i][1] for i in (1, 2, 3)], [rows[i][0] for i in (1, 2, 3)])
print(f" {b}: reference {rows[0][1]}/{n_lw} = {100 * ref:.2f}%")
for i in range(3):
e = (ev_r2, ev_r3, ev_r4)[i]
print(f" row '{rows[i + 1][0]}' ({e} events, univariate HR {uni[i]}): "
f"RR with Table 3 label = {printed[i]:.2f} (HR/RR {uni[i] / printed[i]:.2f}); "
f"RR with labels exchanged = {exch[i]:.2f} (HR/RR {uni[i] / exch[i]:.2f})")
for nm, n_g, e in (("low-mal (T3 label)", n_lm, ev_r2), ("high-well (T3 label)", n_hw, ev_r3)):
print(f" {nm}: {e}/{n_g} = {100 * e / n_g:.2f}%")
allp = [crude[b][0][i] / crude[b][1][i] for b in crude for i in range(3)]
alle = [crude[b][0][i] / crude[b][2][i] for b in crude for i in range(3)]
print(f" HR/RR range over the 9 non-reference rows: Table 3 labels {min(allp):.2f}-{max(allp):.2f}; "
f"labels exchanged {min(alle):.2f}-{max(alle):.2f}")
und = [crude[b][0][2] / crude[b][1][2] for b in crude]
print(f" undisputed high-adiposity-malnourished rows alone: HR/RR = " + ", ".join(f"{x:.2f}" for x in und))
print("\n=== 4. 'equally high crude event rates (30.1% and 29.5%; Table 3)' ===")
EVENT_TOTALS = {"Results text": 145, "Table 3 title": 145, "S6 Table title": 145, "S3 Table title": 156}
print(" composite-event totals stated in the paper: " + "; ".join(f"{k} n = {v}" for k, v in EVENT_TOTALS.items()))
print(f" Table 3 BMI block events sum: {sum(r[1] for r in T3['BMI'])}; WC block: {sum(r[1] for r in T3['WC'])}; "
f"BF block: {sum(r[1] for r in T3['BF'])}")
print(f" as shares of S3 Table's 156: 47/156 = {100 * 47 / 156:.2f}%, 46/156 = {100 * 46 / 156:.2f}%, "
f"26/156 = {100 * 26 / 156:.2f}%; S3 minus Table 3 total = {156 - 145}")
evs = sorted({r[1] for rows in T3.values() for r in rows})
hits = sorted({(a, b, d) for a in evs for b in evs if a != b for d in range(20, 6001)
if round(100 * a / d, 1) == 30.1 and round(100 * b / d, 1) == 29.5})
print(f" pairs of Table 3 event counts (a, b) and common denominators d with a/d = 30.1% and b/d = 29.5%: {hits}")
print(f" as percent of group N: 26/1154 = {100 * 26 / 1154:.2f}%, 47/1154 = {100 * 47 / 1154:.2f}%, "
f"47/1369 = {100 * 47 / 1369:.2f}%, 46/681 = {100 * 46 / 681:.2f}%")
print(f" as percent of the 145 events: 26 -> {100 * 26 / 145:.1f}%, 47 -> {100 * 47 / 145:.1f}%, 46 -> {100 * 46 / 145:.1f}%")
print(f" ratio 30.1/29.5 = {30.1 / 29.5:.4f} (rounding range {30.05 / 29.55:.4f}-{30.15 / 29.45:.4f}); "
f"47/46 = {47 / 46:.4f}; 26/46 = {26 / 46:.4f}")
print("\n=== 5. Fig 2 numbers at risk at time 0 / group N (does each legend label track its group's size?) ===")
for b in AT_RISK:
lw, lm, hw, hm = N[b]
a_lw, a_hw, a_lm, a_hm = AT_RISK[b]
print(f" {b}: low-well {a_lw}/{lw} = {a_lw / lw:.2f}; high-well {a_hw}/{hw} = {a_hw / hw:.2f}; "
f"low-mal {a_lm}/{lm} = {a_lm / lm:.2f}; high-mal {a_hm}/{hm} = {a_hm / hm:.2f} | if the two middle "
f"labels were exchanged: {a_hw}/{lm} = {a_hw / lm:.2f}, {a_lm}/{hw} = {a_lm / hw:.2f}")
print("\n=== 6. Table 2 N row vs Table 1 ===")
print(" Table 1: obese-well 1,369, lean-mal 1,154; Table 2 N row: obese-well column 1,154, lean-mal column 1,369")
print(f" LVH, obese-well column, 84 (printed 6.1%): 84/1369 = {100 * 84 / 1369:.1f}%, 84/1154 = {100 * 84 / 1154:.1f}%")
print(f" LVH, lean-mal column, 102 (printed 8.8%): 102/1154 = {100 * 102 / 1154:.1f}%, 102/1369 = {100 * 102 / 1369:.1f}%")
print(f" LVH, lean-well 81 (3.9%): 81/2096 = {100 * 81 / 2096:.1f}%; obese-mal 99 (14.5%): 99/681 = {100 * 99 / 681:.1f}%")
print("\n=== 7. Table 2: multivariable-adjusted vs unadjusted means for the two middle columns (heuristic) ===")
# (unadjusted obese-well, unadjusted lean-mal, adjusted col 'obese-well', adjusted col 'lean-mal')
T2 = {"IVS": (9.48, 8.78, 8.92, 9.28), "LVPW": (9.44, 8.75, 8.87, 9.25), "IVSi": (4.63, 5.08, 4.94, 4.69),
"LVPWi": (4.61, 5.06, 4.92, 4.67), "LVIDD": (48.4, 45.9, 46.5, 47.8), "LVIDS": (30.5, 28.7, 29.1, 30.0),
"LV EDV": (82.6, 72.3, 75.1, 80.2), "LV ESV": (31.4, 26.7, 28.0, 30.2), "LV EDVi": (40.2, 41.6, 41.4, 40.3),
"LV ESVi": (15.3, 15.4, 15.4, 15.1), "LVEF": (62.2, 63.2, 62.9, 62.5), "LV mass": (161.0, 134.1, 139.8, 154.1),
"LVMI (BSA)": (78.4, 77.0, 76.9, 77.3), "LVMI (Ht2.7)": (39.9, 36.6, 35.9, 39.7), "RWT": (0.39, 0.38, 0.38, 0.39),
"Decel. time": (204.8, 204.7, 203.4, 205.0), "IVRT": (90.8, 89.9, 90.0, 90.2), "TDI-e'": (8.61, 9.20, 9.38, 8.79),
"TDI-s'": (8.28, 8.10, 8.33, 8.21), "E/A": (1.13, 1.26, 1.24, 1.17), "E/e'": (8.04, 8.27, 7.87, 8.10),
"Tau": (39.3, 41.1, 39.5, 40.4), "TRV": (2.09, 2.18, 2.13, 2.11), "LAV": (36.1, 29.7, 29.6, 34.9),
"LAVi": (17.6, 17.2, 16.3, 17.5)}
closer = 0
for k, (uo, ul, ao, al) in T2.items():
same, sw = abs(ao - uo) + abs(al - ul), abs(ao - ul) + abs(al - uo)
closer += sw < same
print(f" measures where the adjusted pair sits closer to the unadjusted pair read in the other order: {closer} of {len(T2)}")
for k in ("LV mass", "LAV", "IVS"):
uo, ul, ao, al = T2[k]
print(f" {k}: unadjusted obese-well {uo}, lean-mal {ul}; adjusted 'obese-well' {ao}, 'lean-mal' {al}")
print("\n=== 8. S3 Table (PNI instead of SA), BMI block ===")
S3 = {"Obese-well-nourished uni": (1.15, 0.63, 2.12, "0.64"), "Lean-malnourished uni": (1.81, 1.09, 3.00, "0.021"),
"Obese-well-nourished multi": (1.14, 0.56, 2.32, "0.034"), "Lean-malnourished multi": (1.45, 0.81, 2.58, "0.21"),
"Obese-malnourished multi": (2.21, 1.22, 4.01, "0.009")}
for k, t in S3.items():
print(f" {k:28s} {t[0]} [{t[1]}, {t[2]}], printed p = {t[3]}: p from CI = {wald(*t[:3])[0]:.3g}")
# ------------------------------------------------------------------ figures
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "svg.fonttype": "none",
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: text-vs-table diagram
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")
ax.text(2, 61, "The same two adjusted hazard ratios, assigned to opposite groups", fontsize=12.5,
weight="bold", va="top")
ax.text(2, 56.8, "Composite of HF hospitalization or death; reference: lean–well-nourished",
fontsize=10, color=INK2, va="top")
panels = [(2, "Abstract; Results › Outcomes"), (57, "Table 3, BMI × albumin rows")]
for x0, title in panels:
ax.add_patch(FancyBboxPatch((x0, 6), 41, 44, boxstyle="round,pad=0,rounding_size=1.5",
fc="white", ec="#c9c8c3", lw=1))
ax.text(x0 + 2, 47.5, title, fontsize=10.5, weight="bold", va="top")
rows_left = [("lean–malnourished", "HR 1.78 (1.04–3.04), p = 0.034", 36),
("obese–well-nourished", "HR 1.41 (0.77–2.58), p = 0.27", 20)]
rows_right = [("Lean–malnourished", "1.41 (0.77, 2.58), p 0.27", 36, P141),
("Obese–well-nourished", "1.78 (1.04, 3.04), p 0.034", 20, P178)]
for lab, val, y in rows_left:
ax.text(4, y + 4, lab, fontsize=10.5, va="center")
ax.text(4, y - 0.5, val, fontsize=10.5, va="center", color=ORANGE, weight="bold")
for lab, val, y, p in rows_right:
ax.text(59, y + 4, lab, fontsize=10.5, va="center")
ax.text(59, y - 0.5, val, fontsize=10.5, va="center", color=ORANGE, weight="bold")
ax.text(59, y - 5, f"p recomputed from CI: {p:.3f}", fontsize=10, va="center", color=BLUE)
ax.text(4, 9.5, "obese–malnourished: 2.49 (1.43–4.34)", fontsize=10, color=INK2, va="center")
ax.text(59, 9.5, "Obese–malnourished: 2.49 (1.43, 4.34)", fontsize=10, color=INK2, va="center")
for (ya, yb) in ((35.5, 19.5), (19.5, 35.5)):
ax.add_patch(FancyArrowPatch((43.5, ya), (56.5, yb), arrowstyle="-|>", mutation_scale=12,
lw=1.5, color=INK2, shrinkA=2, shrinkB=2))
ax.text(50, 44, "same HR,\nCI and p;\nlabels\nswapped", ha="center", va="center", fontsize=10, color=INK)
ax.text(50, 9.5, "agree", ha="center", va="center", fontsize=10, color=INK2)
ax.text(2, 2.2, "Orange: printed in the paper. Blue: two-sided Wald p recomputed from the printed HR and CI.",
fontsize=10, color=INK2, va="center")
save(fig, "fig1")
# Figure 2: univariate HR vs crude risk ratio under each labelling
fig, ax = plt.subplots(figsize=(7.2, 5.4), facecolor=BG)
fig.subplots_adjust(left=0.37, right=0.97, top=0.80, bottom=0.12)
ax.set_facecolor(BG)
ylabels, y = [], 0
ypos = []
for b in ("BMI", "WC", "BF"):
uni, printed, exch, evs3, labs = crude[b]
for i in range(3):
yy = -y
ypos.append(yy)
short = labs[i].replace("Lean–", "lean–").replace("Obese–", "obese–").replace(f"Low {b}–", "low–").replace(f"High {b}–", "high–")
ylabels.append(f"{b} {short} ({evs3[i]})")
ax.plot([uni[i]], [yy], "o", ms=9, color=ORANGE, zorder=3)
ax.plot([printed[i]], [yy], "o", ms=9, mfc=BLUE, mec=BLUE, zorder=4)
if i < 2:
ax.plot([exch[i]], [yy], "s", ms=9, mfc="white", mec=BLUE, mew=2, zorder=4)
y += 1
y += 0.6
ax.set_xscale("log")
ax.set_xlim(1, 8)
ax.xaxis.set_major_locator(mticker.FixedLocator([1, 1.5, 2, 3, 4, 5, 6, 8]))
ax.xaxis.set_major_formatter(mticker.FixedFormatter(["1", "1.5", "2", "3", "4", "5", "6", "8"]))
ax.xaxis.set_minor_locator(mticker.NullLocator())
ax.axvline(1, color=INK2, lw=0.8)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_yticks(ypos)
ax.set_yticklabels(ylabels, fontsize=10)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Ratio vs low-adiposity, well-nourished group (log scale)")
fig.text(0.02, 0.965, "Univariate HR in Table 3 vs crude risk ratio from the paper's counts", fontsize=12.5,
weight="bold", va="top")
fig.text(0.02, 0.92, "Each row is one Table 3 row, named as Table 3 labels it; (n) = crude events in that row", fontsize=10, color=INK2, va="top")
handles = [Line2D([], [], ls="", marker="o", ms=9, color=ORANGE, label="univariate HR printed in Table 3"),
Line2D([], [], ls="", marker="o", ms=9, color=BLUE, label="crude RR, group as labeled in Table 3"),
Line2D([], [], ls="", marker="s", ms=9, mfc="white", mec=BLUE, mew=2,
label="crude RR, middle two labels exchanged")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.895), ncol=2, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract labels the 39.65 million prevented CVD events as deaths; Results report 13.86 million deaths
PLoS Med (2022) · doi:10.1371/journal.pmed.1004111 · PMC9725126
Summary
This microsimulation study compares hypertension screening scenarios in low- and middle-income countries. Its abstract states that almost 40 million (39,650,363) HTN-induced deaths over 10 years could be prevented by increasing screening coverage and completion (scenario S4). The Results give that same number, 39,650,363 (14.2%), as prevented CVD events, and give S4 prevented deaths as 13,856,507 (13.7%); the Author summary agrees ("almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths"). The same abstract sentence also places 13,856,507 beside the 0.6% result for improved test accuracy (S2). By the paper's own percentages, 0.6% corresponds to roughly 0.61 million deaths. So the abstract's mortality headline is about 2.9 times the Results figure, and its absolute death count for the accuracy scenario is about 23 times what the Results imply. The version of record carries this text, and no correction notice was found (as of 2026-10-23).
What the paper reports
Abstract, "Methods and findings":
The study showed that while improving test accuracy avoids 0.6% of HTN-induced deaths over 10 years (13,856,507 [9,382,742; 17,395,833]), almost 40 million (39,650,363 [31,34,233, 49,298,921], i.e., 12.7% [9.9, 15.8]) of the HTN-induced deaths could be prevented by increasing coverage and completion of a screening event in the same time frame.
Abstract, the next sentence:
Doubling the coverage only would still prevent 3,304,212 million ([2,274,664; 4,164,180], 12.1% [8.3, 15.2]) CVD events 10 years after the rollout of the program.
Author summary, "What did the researchers do and find?":
While improving test accuracy avoids only 0.6% of HTN-induced deaths over 10 years, scaling up test coverage and completion can avoid almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths.
Results, paragraph 3 (CVD events, Fig 3, scenario S4):
In fact, it leads to a reduction of over 7 million (7,031,120; 26%) CVD events in year 10 compared to current standard of care (scenario S0). In sum, this equals 39,650,363 (14.2%) prevented CVD events over the period of 10 years.
Results, paragraph 4 (deaths, Fig 4):
Whereas survival does not improve much by increasing the test accuracy with 0.6% of HTN-induced deaths avoided over 10 years, we observe a high impact by doubling the coverage of the first screen (S3) with 6.6%. In year 10 alone, more than 2 million more lives (2,392,109) can be saved by additionally increasing completing probability in scenario S4, equalling to a total of 13,856,507 (13.7%) of the HTN-induced deaths over the 10 years.
Discussion, paragraph 1:
Whereas the improvement of the test accuracy had only a small potential impact on CVD events and related mortality over 10 years (reduction of 0.5% and 0.6%, respectively), by doubling of the coverage of a first screen, 6.6% of HTN-induced deaths could be avoided over a period of 10 years, and by increasing screen completion in addition, even 13.7% of deaths would not occur.
Discussion, paragraph 2:
Moreover, according to a review on available data on HTN in the literature [30], in 2015, an estimated 7.5 million deaths were attributable to systolic blood pressure in LMICs. This number coincides with the number of deaths in our model for the baseline (S0).
Methods, "Simulation study design", which treats events and deaths as separate endpoints:
We investigated the effect of blood pressure screening on the total number of CVD events and the resulting deaths within the population.
What the paper's own numbers show
Method. Each prevented count in the Results comes with its relative reduction against the base case S0. Dividing the count by that share gives the 10-year S0 total the paper is working from: S0 CVD events = 39,650,363 / 0.142 and S0 deaths = 13,856,507 / 0.137. These implied totals let us check each abstract number. For each one we ask which outcome and which percentage it fits. We also ask what S0 total the abstract's own label and percentage would require. The printed percentages are rounded to 0.1 points, so the implied totals are accurate to within about 1%. That is far smaller than any discrepancy below. All arithmetic is in recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| S4 prevented CVD events, 10 y | Results para. 3; Author summary | 39,650,363 (14.2%); "almost 40 million" events | Implied S0 CVD events = 279,227,908 (27.9 M per year). Cross-check: year-10 figures imply 27.04 M (S4) and 27.31 M (S3) S0 events in year 10 |
| S4 prevented deaths, 10 y | Results para. 4; Author summary; Discussion | 13,856,507 (13.7%); "14 million (13.7%)" deaths | Implied S0 deaths = 101,142,387 (10.11 M per year); deaths/events in S0 = 0.362 |
| S4 prevented deaths, 10 y | Abstract | 39,650,363, "12.7% [9.9, 15.8]" of HTN-induced deaths | 2.86 times the Results figure; 25,793,856 more deaths than the Results report |
| S0 deaths implied by the abstract's S4 sentence | Abstract | (39,650,363 at 12.7%) | 312,207,583 deaths (31.2 M per year). That is 1.118 times the implied S0 CVD events, so more deaths than events; and 4.16 times the 7.5 M per year the Discussion says the baseline matches (Results reading: 1.35 times) |
| S4 relative reduction | Abstract vs Results | 12.7% vs 14.2% (events) and 13.7% (deaths) | 39,650,363 / 0.127 = 312.2 M matches neither implied denominator (279.2 M or 101.1 M); gaps of 1.5 and 1.0 points |
| S2 (improved accuracy) prevented deaths, 10 y | Abstract | 0.6% "(13,856,507 [9,382,742; 17,395,833])" | 0.6% × 101,142,387 = 606,854 deaths. 13,856,507 / 101,142,387 = 0.137, the S4 share. Read as 0.6%, 13,856,507 would need 2.31 billion S0 deaths (22.8 times the implied baseline) |
| Lower bound of the S4 interval | Abstract | "31,34,233" | 7 digits; a value near 31 million needs 8. In the publisher's JATS XML, "31" and "34" are tagged as citations of references 31 and 34 |
| S3 prevented CVD events | Abstract vs Results para. 3 | "3,304,212 million" vs 3,304,212 | Read literally, the abstract says 3.30 × 10^12; the Results value is 3,304,212 |
The abstract's mortality sentence takes the S4 CVD-event count and labels it as deaths. It then moves the S4 death count next to the S2 accuracy percentage. The Author summary, Results and Discussion all agree with one another; only the abstract departs from them.
Figure
Figure 1. Left, in orange: the two clauses of the abstract's "Methods and findings" sentence, quoted verbatim. Right: the Results paragraphs (counted from the first Results paragraph as 1) and Author summary that report the same numbers. The upper connector links 39,650,363. The abstract calls it HTN-induced deaths (12.7%), while Results paragraph 3 calls it prevented CVD events (14.2%). Recomputed (blue), it is 2.86 times the S4 death count given in Results paragraph 4. The lower connector links 13,856,507. The abstract places it beside the 0.6% improved-accuracy result (S2), while Results paragraph 4 gives it as the S4 death total (13.7%). Recomputed from the paper's implied baseline of 101.1 million S0 deaths over 10 years, the S2 figure is 0.61 million. The Author summary (bottom right) gives about 40 million CVD events and 14 million deaths for S4, which agrees with the Results.
Figure 2. Deaths prevented over 10 years relative to S0, in millions, on a linear axis starting at zero. Orange: the counts the abstract attaches to each scenario's death result. For S4 (top), the open blue marker is the count in Results paragraph 4 (13.86 M); the abstract's 39.65 M is 2.9 times larger. For S2 (bottom), the filled blue marker is 0.6% of the S0 deaths implied by the Results (0.61 M); the abstract's 13.86 M is 22.8 times larger. The interval the abstract prints with 13,856,507 (9.38–17.40 M) excludes the S2 value implied by the Results.
Why it matters
The abstract's "Methods and findings" sentence is the paper's headline mortality estimate, and it is the figure most likely to be quoted. It says almost 40 million deaths could be prevented under S4. The paper's own Results put this at about 13.9 million, so the abstract overstates deaths averted by about 25.8 million. The same sentence gives the improved-accuracy scenario about 13.9 million averted deaths in absolute terms, when its 0.6% corresponds to about 0.6 million. A reader of the abstract alone would therefore overestimate the absolute mortality benefit of both interventions. The ratio between them would also be off: the abstract's numbers give about 2.9 to 1 (39.65 M vs 13.86 M), while the Results imply about 23 to 1 (13.86 M vs 0.61 M).
The direction and ranking of the scenarios do not change. Coverage plus completion still clearly outperforms improved accuracy, and the Author summary, Results and Discussion state the percentages consistently. The paper's qualitative conclusion, that screening completion and coverage matter more than test accuracy, is therefore unaffected. The error concerns the absolute numbers in the abstract and, for S4, its percentage. The abstract's 12.7% [9.9, 15.8] cannot be matched to any figure in the main text, so it is not clear what that interval refers to.
Explanations considered
- "Deaths" in the abstract might mean the same as "CVD events" in the Results. This does not fit. The Methods treat CVD events and "the resulting deaths" as separate endpoints. The Results report them in separate paragraphs with separate figures (Fig 3 events, Fig 4 deaths). The Author summary gives both numbers side by side ("almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths"). If 39,650,363 were deaths at 12.7%, the implied baseline would be 312.2 million deaths over 10 years. That exceeds the 279.2 million baseline CVD events implied by the Results. Deaths that follow CVD events cannot outnumber the events.
- A different population, subgroup or time window. The abstract sentence says "over 10 years" and "in the same time frame", matching the 10-year totals in the Results. No income-group or other subset is named. The abstract's own S3 clause ("10 years after the rollout") matches the Results' year-10 count, 3,304,212 (12.1%). The implied year-10 S0 events (27.3 M) agree with the S4 year-10 figure (27.0 M), so the abstract draws on the same model output as the Results.
- The 13,856,507 might really be the S2 death count. This does not fit. The Results give 13,856,507 explicitly as the S4 total (13.7%). At the S0 baseline implied by the Results, the S2 share of 0.6% is about 0.61 million. For 13,856,507 to be 0.6%, the S0 baseline would have to be about 2.31 billion deaths, against a simulated population of 3.15 billion adults.
- A different run or a revised analysis. No sensitivity analysis or alternative run in the article or S1 Text reports 39,650,363 as deaths or 12.7% as the S4 reduction. S1 Text gives no 10-year absolute totals, so it can neither confirm nor reconcile the 12.7%.
- A text-conversion artefact in PMC. This is ruled out for the labelling. The publisher's HTML and the JATS XML (via Europe PMC) carry the same wording. The malformed lower bound "31,34,233" does come from typesetting: in the XML, "31" and "34" are marked up as reference citations. That markup is unrelated to the deaths-versus-events labelling and the misplaced 13,856,507.
- The Discussion's 7.5 million deaths per year. This is a loose calibration check, not a precise target. The Results reading implies about 10.1 million S0 deaths per year, the same order as 7.5 million. The abstract reading implies about 31.2 million per year, about four times larger. This check is supportive only; the conclusion rests on the explicit Results labels.
Status
- Version of record. I checked the publisher's article page at journals.plos.org (via the DOI), the PMC record and the publisher's JATS XML from the Europe PMC full-text service. All carry the quoted text unchanged, including the abstract sentence, the Author summary and Results paragraphs 3–4.
- Supplements. The three supporting files are the S1 STRESS Checklist, S1 Text and S1 HPACC Consortium. S1 Text covers model methods, population scaling (3.15 billion), Table A, Fig A (yearly percentage heatmaps for events and deaths, as images) and Fig B (income-class stratification). It gives no 10-year absolute totals or denominators and does not resolve the discrepancy. No source data or code is linked.
- Corrections.
src/known.pyreturned no Crossref updates, no PMC notices and no Europe PMC notices. The Crossref record has noupdate-to,updated-byor relation entries. I found no correction, erratum, retraction or expression of concern. - PubPeer. The PubPeer search page and API returned HTTP 403 from this machine, so PubPeer could not be checked directly. A web search for the article title with "pubpeer" found no PubPeer thread.
- Sources. https://doi.org/10.1371/journal.pmed.1004111; https://pmc.ncbi.nlm.nih.gov/articles/PMC9725126/; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC9725126/fullTextXML; https://api.crossref.org/works/10.1371/journal.pmed.1004111; S1 Text (journal.pmed.1004111.s002.docx).
- As of 2026-10-23.
Suggested correction
In the abstract's "Methods and findings", align the mortality sentence with the Results. For example: "while improving test accuracy avoids 0.6% of HTN-induced deaths over 10 years, 13,856,507 [9,382,742; 17,395,833] (13.7%) of the HTN-induced deaths and almost 40 million CVD events (39,650,363 [lower bound; 49,298,921], i.e., 14.2%) could be prevented by increasing coverage and completion of a screening event in the same time frame." Here "lower bound" stands for the correct lower credible bound, which is not printed anywhere in the article. The authors should also confirm:
- The absolute number of deaths avoided under S2 (about 0.6 million by the paper's own percentages), with its credible interval.
- Which percentage and interval belong to the S4 CVD-event reduction (the Results give 14.2%; the abstract's 12.7% [9.9, 15.8] matches no reported figure).
- The S4 lower credible bound, garbled as "31,34,233".
- "3,304,212 million" corrected to "3,304,212" CVD events (S3, year 10).
Recomputation script (python recompute.py.txt)
"""Recompute the abstract-vs-Results figures of PMC9725126
(PLoS Med 2022, doi:10.1371/journal.pmed.1004111).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PMC (identical in the
publisher's JATS XML):
Abstract (Methods and findings), Author summary, Results paragraphs 3-4 (Fig 3 events, Fig 4 deaths),
Discussion paragraphs 1-2.
Scenario labels: S0 base case, S2 improved accuracy, S3 increased coverage,
S4 increased coverage + improved completion. All totals are over 10 years.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
# Results paragraph 3 (CVD events, Fig 3)
S4_EVENTS_10Y = 39_650_363 # "39,650,363 (14.2%) prevented CVD events over the period of 10 years"
S4_EVENTS_PCT = 0.142
S4_EVENTS_Y10 = 7_031_120 # "reduction of over 7 million (7,031,120; 26%) CVD events in year 10"
S4_EVENTS_Y10_PCT = 0.26
S3_EVENTS = 3_304_212 # "3,304,212 prevented CVD events (12.1%) ... 10 years after a rollout"
S3_EVENTS_PCT = 0.121
# Results paragraph 4 (deaths, Fig 4)
S4_DEATHS_10Y = 13_856_507 # "a total of 13,856,507 (13.7%) of the HTN-induced deaths over the 10 years"
S4_DEATHS_PCT = 0.137
S2_DEATHS_PCT = 0.006 # "0.6% of HTN-induced deaths avoided over 10 years"
S3_DEATHS_PCT = 0.066 # "(S3) with 6.6%"
# Abstract
ABS_S4_DEATHS = 39_650_363 # "almost 40 million (39,650,363 [...], i.e., 12.7% [9.9, 15.8]) of the HTN-induced deaths"
ABS_S4_PCT = 0.127
ABS_S4_LOWER_TEXT = "31,34,233" # printed lower bound of that interval
ABS_S4_UPPER = 49_298_921
ABS_S2_DEATHS = 13_856_507 # "improving test accuracy avoids 0.6% ... (13,856,507 [9,382,742; 17,395,833])"
ABS_S2_CI = (9_382_742, 17_395_833)
ABS_S3_TEXT_MILLION = 3_304_212 # "3,304,212 million ([2,274,664; 4,164,180], 12.1% [8.3, 15.2]) CVD events"
# Author summary: "almost 40 million HTN-induced CVD events and 14 million (13.7%) related deaths"
SUMMARY_EVENTS_M, SUMMARY_DEATHS_M, SUMMARY_DEATHS_PCT = 40, 14, 0.137
# Discussion paragraph 2: 7.5 million deaths attributable to SBP in LMICs (2015),
# which "coincides with the number of deaths in our model for the baseline (S0)"
DISC_ANNUAL_DEATHS = 7.5e6
M = 1e6
print("=== 1. Denominators implied by the Results (baseline S0, 10 years) ===")
base_events = S4_EVENTS_10Y / S4_EVENTS_PCT
base_deaths = S4_DEATHS_10Y / S4_DEATHS_PCT
print(f"S0 CVD events = 39,650,363 / 0.142 = {base_events:,.0f} ({base_events / 10 / M:.1f} M per year on average)")
print(f"S0 deaths = 13,856,507 / 0.137 = {base_deaths:,.0f} ({base_deaths / 10 / M:.2f} M per year on average)")
print(f"S0 deaths / S0 CVD events = {base_deaths / base_events:.3f}")
print(f"cross-check, year 10: S4 7,031,120 / 0.26 = {S4_EVENTS_Y10 / S4_EVENTS_Y10_PCT:,.0f} S0 events in year 10; "
f"S3 3,304,212 / 0.121 = {S3_EVENTS / S3_EVENTS_PCT:,.0f}")
print(f"Author summary rounding: 39,650,363 -> {S4_EVENTS_10Y / M:.1f} M ('almost {SUMMARY_EVENTS_M} million' events); "
f"13,856,507 -> {S4_DEATHS_10Y / M:.1f} M ('{SUMMARY_DEATHS_M} million', {SUMMARY_DEATHS_PCT:.1%} deaths)")
print("\n=== 2. Abstract S4 sentence: 39,650,363 labelled as deaths ===")
ratio = ABS_S4_DEATHS / S4_DEATHS_10Y
diff = ABS_S4_DEATHS - S4_DEATHS_10Y
print(f"abstract deaths / Results deaths = 39,650,363 / 13,856,507 = {ratio:.2f}")
print(f"difference = {diff:,} deaths")
implied_deaths_abs = ABS_S4_DEATHS / ABS_S4_PCT
print(f"if 39,650,363 were deaths at 12.7%: implied S0 deaths = {implied_deaths_abs:,.0f} "
f"({implied_deaths_abs / 10 / M:.1f} M per year)")
print(f" -> exceeds the implied S0 CVD events ({base_events:,.0f}) by a factor {implied_deaths_abs / base_events:.3f}")
print(f" -> per-year value vs Discussion's 7.5 M: x{implied_deaths_abs / 10 / DISC_ANNUAL_DEATHS:.2f}; "
f"Results reading: x{base_deaths / 10 / DISC_ANNUAL_DEATHS:.2f}")
print(f"abstract 12.7% vs Results 14.2% (events) and 13.7% (deaths): "
f"gaps {100 * (S4_EVENTS_PCT - ABS_S4_PCT):.1f} and {100 * (S4_DEATHS_PCT - ABS_S4_PCT):.1f} points")
print(f"39,650,363 / 0.127 = {S4_EVENTS_10Y / ABS_S4_PCT:,.0f} matches neither {base_events:,.0f} nor {base_deaths:,.0f}")
print("\n=== 3. Abstract S2 sentence: 13,856,507 beside 0.6% ===")
s2_deaths = S2_DEATHS_PCT * base_deaths
print(f"0.6% x {base_deaths:,.0f} = {s2_deaths:,.0f} deaths (S2, implied)")
print(f"13,856,507 / {base_deaths:,.0f} = {S4_DEATHS_10Y / base_deaths:.3f} (the S4 share, 13.7%)")
print(f"13,856,507 as 0.6% would imply S0 deaths = {ABS_S2_DEATHS / S2_DEATHS_PCT:,.0f} "
f"({ABS_S2_DEATHS / S2_DEATHS_PCT / base_deaths:.1f} x the Results-implied baseline)")
print(f"abstract S2 count / implied S2 count = {ABS_S2_DEATHS / s2_deaths:.1f}")
print(f"for reference, S3 deaths at 6.6% = {S3_DEATHS_PCT * base_deaths:,.0f}")
print(f"abstract interval printed with 13,856,507: {ABS_S2_CI[0]:,}-{ABS_S2_CI[1]:,}; "
f"its lower end is {ABS_S2_CI[0] / s2_deaths:.1f} x the implied S2 point value")
print("\n=== 4. Typesetting / unit slips in the same abstract paragraph ===")
digits = len(ABS_S4_LOWER_TEXT.replace(",", ""))
print(f"lower bound '{ABS_S4_LOWER_TEXT}': {digits} digits (a value near 31 million needs 8); "
f"upper bound {ABS_S4_UPPER:,}")
print(f"'3,304,212 million' read literally = {ABS_S3_TEXT_MILLION * M:,.0f}; Results value {S3_EVENTS:,}")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# fig1: abstract vs Results/Author summary, two text panels with connectors
fig = plt.figure(figsize=(7.2, 5.6))
fig.patch.set_facecolor(BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def box(x, y, w, h, edge, title, body, title_col):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
facecolor="white", edgecolor=edge, linewidth=1.5))
ax.text(x + 1.5, y + h - 1.8, title, fontsize=10, fontweight="bold", color=title_col, va="top")
ax.text(x + 1.5, y + h - 7.0, body, fontsize=10, color=INK, va="top", linespacing=1.35)
ax.text(21, 96, "Abstract (Methods and findings)", ha="center", fontsize=11, fontweight="bold", color=ORANGE)
ax.text(79, 96, "Results and Author summary", ha="center", fontsize=11, fontweight="bold", color=INK)
LX, RX, W = 1.5, 59.5, 39
box(LX, 64, W, 27, ORANGE, "S4 sentence",
"“almost 40 million (39,650,363\n[31,34,233, 49,298,921], i.e.,\n12.7% [9.9, 15.8]) of the\n"
"HTN-induced deaths could be\nprevented”", ORANGE)
box(RX, 64, W, 27, INK2, "Results paragraph 3 (S4)",
"“In sum, this equals 39,650,363\n(14.2%) prevented CVD events\nover the period of 10 years.”", INK)
box(LX, 30, W, 27, ORANGE, "S2 sentence",
"“while improving test accuracy\navoids 0.6% of HTN-induced\ndeaths over 10 years\n"
"(13,856,507 [9,382,742;\n17,395,833])”", ORANGE)
box(RX, 30, W, 27, INK2, "Results paragraph 4 (S4)",
"“equalling to a total of\n13,856,507 (13.7%) of the\nHTN-induced deaths over the\n10 years.”", INK)
box(RX, 3, W, 20, INK2, "Author summary (S4)",
"“almost 40 million HTN-induced\nCVD events and 14 million\n(13.7%) related deaths”", INK)
def connector(y, line1, line2, line3):
ax.annotate("", xy=(RX - 0.6, y), xytext=(LX + W + 0.6, y),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
ax.text(50, y + 1.5, line1, ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
ax.text(50, y - 1.5, line2, ha="center", va="top", fontsize=10, color=INK2)
ax.text(50, y - 9.0, line3, ha="center", va="top", fontsize=10, color=BLUE)
connector(80, "same number", "deaths vs\nCVD events", f"recomputed:\n{ratio:.2f}× the\nS4 deaths")
connector(46, "same number", "S2 (0.6%) vs\nS4 (13.7%)", f"recomputed:\nS2 = {s2_deaths / M:.2f} M\ndeaths")
ax.text(21, 18, f"Abstract quotes in orange.\nBlue: recomputed from the\npaper's own Results numbers\n"
f"(S0 deaths = 13,856,507 / 0.137\n= {base_deaths / M:.1f} M over 10 years).",
ha="center", va="center", fontsize=10, color=INK2, linespacing=1.35)
save(fig, "fig1")
# fig2: deaths prevented over 10 years, abstract vs Results / recomputed
fig, ax = plt.subplots(figsize=(7.2, 3.5))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
rows = [("S2: improved test accuracy", ABS_S2_DEATHS, s2_deaths, "recomputed: 0.6% × S0 deaths"),
("S4: coverage + completion", ABS_S4_DEATHS, S4_DEATHS_10Y, "Results / Author summary")]
for i, (lab, rep, alt, altlab) in enumerate(rows):
y = i
ax.plot([alt / M, rep / M], [y, y], color=GRID, lw=2, zorder=1)
ax.scatter(rep / M, y, s=90, color=ORANGE, zorder=3, label="Abstract" if i == 0 else None)
if i == 0:
ax.scatter(alt / M, y, s=90, color=BLUE, zorder=3, label="Recomputed: 0.6% of S0 deaths")
else:
ax.scatter(alt / M, y, s=90, facecolor="white", edgecolor=BLUE, linewidth=2, zorder=3,
label="Results paragraph 4")
ax.text(rep / M, y + 0.2, f"{rep / M:.2f} M", ha="center", va="bottom", color=ORANGE, fontsize=10)
ax.text(alt / M, y + 0.2, f"{alt / M:.2f} M", ha="center", va="bottom", color=BLUE, fontsize=10)
ax.text((rep + alt) / 2 / M, y - 0.3, f"×{rep / alt:.1f}", ha="center", va="top", color=INK2, fontsize=10)
ax.set_yticks([0, 1])
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_ylim(-0.7, 1.55)
ax.set_xlim(0, 45)
ax.set_xlabel("HTN-induced deaths prevented over 10 years vs S0 (millions)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
fig.tight_layout(rect=(0, 0, 1, 0.88))
fig.legend(loc="upper center", ncol=3, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.99))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract reverses the direction of the Alpha versus B.1.177 viral-burden difference reported in Table 1
PLoS Pathog (2023) · doi:10.1371/journal.ppat.1011461 · PMC10449197
Summary
The Abstract states that, among unvaccinated people with no known prior exposure, viral burden was "44% lower among Alpha variant infections" than among B.1.177 infections, and one Results sentence says the same. The paper's own coefficient points the other way. In Table 1, B.1.177 has a Beta score of +1.00 Ct against the Alpha reference, so B.1.177 samples had higher Ct values. Under the paper's stated rule (3 Ct = 10-fold change in viral load), Alpha burden is 10^(1/3) = 2.15 times that of B.1.177: 115% higher, or equivalently B.1.177 is 54% lower. The Results subsection heading, the sensitivity analysis and the Discussion ("115% higher among Alpha") all agree with this. The Abstract's first quantitative finding therefore has the wrong sign and the wrong size, and the publisher correction issued in October 2023 does not address it. A second sentence, in the sensitivity analysis, reverses the Delta versus Alpha direction among vaccinated people, and several smaller conversions in the text also disagree with Table 1.
What the paper reports
Abstract:
By analysing the adjusted Ct values using partial least squares regression, we found that among unvaccinated individuals with no known prior exposure, viral burden was 44% lower among Alpha variant infections, compared to those with the predecessor strain, B.1.177.
Results, subsection headed "Among unvaccinated individuals with no known prior exposure, viral burden was higher among Alpha compared to B.1.177 infections":
For samples sequenced at Oxford, Ct values in this group were higher for B.1.177 samples (Beta score = 1.00, VIP = 1.05) compared to Alpha. This corresponds to a 44% reduction in viral loads with Alpha.
Table 1, block "Variant amongst unvaccinated individuals with no known prior exposure" (columns: Oxford N | Beta score (viral load factor change*) | VIP):
Ref = Alpha (Oxford), Delta (Northumbria) | 3526
B.1.177 | 658 | 1.00 (2.15) | 1.05
Table 1 caption footnote:
*based upon a Ct value decrease of 3 being equivalent to a 10-fold increase in viral load [34].
Results, sensitivity analysis (subsection "The association between novel variants and higher viral burden is robust to assumptions about the shape of the within-host viral trajectory"):
It is noteworthy that the VIP value remained greater than unity across plausible parameter combinations, providing support for the observation that viral burden is higher in infections with the Alpha-variant compared to B.1.177, among unvaccinated individuals with no known prior exposure.
The second paragraph of the same subsection, on Delta versus Alpha among vaccinated individuals:
This analysis therefore provides support for the finding that samples with the Delta variant had a lower viral burden compared to samples with the Alpha variant among vaccinated individuals. In summary, our sensitivity analysis supports the finding that both the shift from B.1.177 to Alpha and the shift from Alpha to Delta was associated with an increase in viral burden amongst the population (mostly unvaccinated and mostly vaccinated, respectively) that prevailed at each stage.
Discussion, third paragraph:
Among individuals with no known prior immunity, viral burden was, on average, 115% higher among Alpha-variant compared to B.1.177 samples.
The same paragraph and the Abstract also give the other conversions checked below:
We found that viral burden decreased over time and was higher among older individuals (13% and 17% higher for every 10 year age increase, in the samples sequenced in Oxford and Northumbria, respectively).
Vaccination reduced viral burden, with the average reduction being 75% and 30% amongst the samples sequenced at Oxford and Northumbria, respectively.
Abstract:
Vaccination reduced viral burden by 67%, and among vaccinated individuals, viral burden was 286% higher among Delta variant, compared to Alpha variant, infections.
Results, subsection "Viral burden is higher among older individuals":
Based upon a 3-point decrease in Ct value being equivalent to a 10-fold increase in viral load (see [34]), these age effects equate to an approximate 14% and 17% increase in viral load for every 10 year age increase, respectively.
What the paper's own numbers show
Method. A Beta score is the difference in (epidemiologically adjusted) Ct between a category and its reference. The paper converts it to a viral-load factor with the rule in the Table 1 footnote and the Results: a Ct decrease of 3 is a 10-fold increase, so factor (category / reference) = 10^(−Beta/3). I applied this rule to all 27 Beta scores in Table 1 and compared the result with the factor printed beside each one. I then converted every percentage in the text back to the Beta it implies. The PLS model itself cannot be re-run: the authors' public repository holds MATLAB code but no Ct data. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Ct(B.1.177) − Ct(Alpha), unvaccinated, Oxford | Table 1; Results | Beta = 1.00 ("Ct values ... were higher for B.1.177") | B.1.177 has higher Ct, so lower viral burden |
| Viral load, Alpha / B.1.177 | Table 1 | factor 2.15 | 10^(1.00/3) = 2.154 |
| Alpha relative to B.1.177 | Discussion | "115% higher among Alpha" | +115.4% (matches) |
| Alpha relative to B.1.177 | Abstract; Results sentence | "44% lower among Alpha"; "44% reduction in viral loads with Alpha" | +115.4%: opposite direction |
| B.1.177 relative to Alpha | (the reverse reading) | — | −53.6%, not −44% |
| Beta implied by "44% lower among Alpha" | derived | — | 3·log10(0.56) = −0.76, opposite sign to the printed +1.00 |
| Table 1 factors that follow 10^(−Beta/3) | Table 1, all rows | 27 printed factors | 26 of 27 match to two decimals; the exception is this B.1.177 row, where 0.46 is expected and 2.15 = 1/0.464 is printed |
The other conversions in the text were checked the same way. These are smaller and do not change a conclusion:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Vaccinated vs no known prior exposure, Oxford (Beta 1.44) | Table 1; Abstract; Results; Discussion | 0.33; "67%" reduction (Abstract, Results); "75%" (Discussion) | factor 0.331, a 66.9% reduction; the Discussion's 75% does not match |
| Vaccinated vs no known prior exposure, Northumbria (Beta 0.34) | Table 1; Discussion | 0.77; "30%" reduction | factor 0.770, a 23.0% reduction |
| Delta vs Alpha among vaccinated, Oxford (Beta −1.37) | Table 1; Abstract; Results; Discussion | 2.86; "286% higher" | factor 2.862, which is 186% higher (286% of Alpha's level) |
| Age per 10 years, Oxford (Beta −0.016 per year) | Table 1; Results; Discussion | 1.13; "14%" (Results); "13%" (Discussion) | factor 1.131, +13.1% |
| Age per 10 years, Northumbria (Beta −0.021 per year) | Table 1; Abstract; Results; Discussion | 1.17; "17%" | factor 1.175, +17.5% (consistent) |
| Direction of Delta vs Alpha among vaccinated, Oxford (Beta −1.37) | Results, sensitivity-analysis subsection, second paragraph | "samples with the Delta variant had a lower viral burden compared to samples with the Alpha variant among vaccinated individuals" | factor 2.862: Delta higher. Table 1, the Abstract ("286% higher among Delta"), the Results subsection on vaccinated individuals, the Discussion and the next sentence of the same paragraph ("the shift from Alpha to Delta was associated with an increase in viral burden") all say higher |
Figure
Why it matters
The Alpha versus B.1.177 comparison is the first quantitative finding in the Abstract, which is the part of the paper most readers and indexing services see. As printed, it tells readers that the Alpha variant was associated with lower within-host viral burden than its predecessor among unvaccinated people. The paper's coefficient, its Table 1 factor, its section heading, its sensitivity analysis and its Discussion all report the opposite: Alpha was associated with about 2.15-fold (115%) higher burden. A reader who relies on the Abstract, or who cites the Results sentence, would carry forward the reversed direction and a magnitude (44%) that no conversion of the published coefficient under the paper's rule produces.
The paper's broader narrative, that each new variant brought higher viral burden in the population then prevailing (B.1.177 to Alpha among the mostly unvaccinated, Alpha to Delta among the mostly vaccinated), is supported by its own analysis and survives once the Abstract is corrected. The Abstract's closing summary ("is influenced by the interplay of vaccination status and viral variant") states no direction and is not affected. The secondary discrepancies matter less. The sensitivity-analysis sentence that Delta had "a lower viral burden" than Alpha among vaccinated people contradicts the summary sentence that immediately follows it and every other statement of that result, so a careful reader can resolve it, but a reader quoting that sentence alone would carry the reversed direction. In addition, "286% higher" for a 2.86-fold factor overstates the Delta effect by 100 percentage points in the Abstract, and the Discussion's vaccination reductions (75% and 30%) are larger than the Table 1 values (67% and 23%).
Explanations considered
- The sentence names the wrong variant as the lower one, and 44% describes B.1.177 relative to Alpha. This accounts for the direction but not the size. Under the paper's rule B.1.177 is 53.6% lower than Alpha, not 44%. A 44% deficit for B.1.177 would need a Beta of +0.76, not +1.00.
- A different Ct-to-load rule. Only a 4-Ct-per-10-fold rule gives about 44% from a 1.00 Ct difference (1 − 10^(−1/4) = 43.8%), and even then it is B.1.177 that is lower. The paper states and uses the 3-Ct rule throughout: with 4 Ct the vaccination effect would be a 56.3% reduction (text: 67%), the Delta factor 2.20 (Table 1: 2.86) and the Northumbria age effect 12.8% (text: 17%). Other common rules (3.3 Ct per 10-fold, or 1 Ct per doubling) give 50.2% and 50.0%. None of them makes Alpha the lower one.
- The Beta sign convention is the reverse of what I assumed. This does not account for the numbers. The same Results sentence says "Ct values in this group were higher for B.1.177 samples (Beta score = 1.00 ...) compared to Alpha", and every other positive Beta in Table 1 (for example Vaccinated, 1.44) is printed with a factor below 1, meaning lower burden.
- The Abstract reports a different model or sensitivity setting. No such result appears in the paper. The sensitivity analysis states that the Beta for B.1.177 decreased under some trajectory assumptions, but that the evidence remained "support for the observation that viral burden is higher in infections with the Alpha-variant compared to B.1.177"; it reports no setting in which Alpha is lower.
- Carry-over from the preprint. The bioRxiv preprint abstract (v1, 2 December 2022, read through the bioRxiv API) states that "the average Ct value was 0.94 lower among Alpha variant infections, compared those with the predecessor strain, B.1.177", which is the same direction as Table 1 (Alpha 2.06-fold higher under the 3-Ct rule). The preprint abstract therefore does not contain the reversed statement. The preprint's full text could not be downloaded in this session, so the origin of the 44% figure could not be traced.
- A conversion or typesetting artifact in PMC. Excluded. The publisher's HTML and printable PDF and the Europe PMC XML carry the same Abstract, Results sentence and Table 1 row as the PMC archive.
Status
- Version of record. Checked on the publisher's site (journals.plos.org, HTML and the 24-page printable PDF) and in Europe PMC's full-text XML. All carry the Abstract sentence, the Results sentence ("44% reduction in viral loads with Alpha"), the Table 1 row "B.1.177 658 1.00 (2.15) 1.05" and the Discussion's "115% higher". Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements and code. S1 Table (review of earlier studies), S2 Table (VIF values), S1 Fig (PLS error and loading plots), S2 Fig (distributions of the Fig 2 data), S1 Data (ENA accession numbers) and S1 Text (consortium members) contain no variant coefficients or viral-burden conversions. The code repository linked in the paper's Methods (Implementation of analysis) holds MATLAB scripts and a README, without Ct data or results.
- Corrections. One correction exists: doi:10.1371/journal.ppat.1011706 (PMC10558069, published 2023-10-06), listed by Crossref and Europe PMC. It corrects formatting errors in the sixth and ninth Methods equations, in-text expressions on page 20, errors in the Fig 4 caption, and a figure reference (Fig 4C/4F to 4E/4F) in the Results. It does not mention the Abstract, the Alpha versus B.1.177 percentage or Table 1. No retraction or expression of concern was found.
- Comments. PubPeer's API and search page both returned a Cloudflare block (HTTP 403) in this session, so PubPeer could not be checked. A web search for the DOI with "44%" and "B.1.177" returned only the article and repository mirrors repeating the Abstract.
- Sources. PMC10449197; https://journals.plos.org/plospathogens/article?id=10.1371/journal.ppat.1011461; correction PMC10558069; Crossref; Europe PMC; bioRxiv API record for doi:10.1101/2022.12.02.518847. As of 2026-10-23.
Suggested correction
- Abstract: replace "viral burden was 44% lower among Alpha variant infections, compared to those with the predecessor strain, B.1.177" with a statement consistent with Table 1, for example "viral burden was 115% higher (2.15-fold) among Alpha variant infections, compared to those with the predecessor strain, B.1.177".
- Results: change "This corresponds to a 44% reduction in viral loads with Alpha" to "This corresponds to a 115% increase in viral loads with Alpha (equivalently, a 54% reduction with B.1.177)".
- Table 1: print the B.1.177 factor in the same orientation as the other rows (0.46, B.1.177 relative to Alpha), or footnote that 2.15 is Alpha relative to B.1.177.
- Abstract, Results and Discussion: describe the Delta versus Alpha effect as "186% higher" or "2.86-fold", not "286% higher".
- Results, sensitivity-analysis subsection, second paragraph: change "samples with the Delta variant had a lower viral burden compared to samples with the Alpha variant among vaccinated individuals" to "... had a higher viral burden ...".
- Discussion: change the vaccination reductions from 75% and 30% to 67% and 23%, and align the Oxford age effect (14% in the Results, 13% in the Discussion; 13.1% from Table 1).
Recomputation script (python recompute.py.txt)
"""Recompute the Alpha vs B.1.177 viral-burden statements of PMC10449197
(PLoS Pathog 2023, doi:10.1371/journal.ppat.1011461).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
five candidate figure designs (candidates/cand_A..E.svg/png) next to this file, for the user
to choose from. No design is embedded until one is chosen:
python recompute.py.txt --pick A,E # writes fig1 = cand_A, fig2 = cand_E and prints captions
The candidates/ folder is review material only and is not part of what goes to the journal.
Inputs are copied from the paper as archived in PMC (Table 1 and the text).
Conversion rule stated in the paper (Table 1 footnote; Results, "Viral burden is higher
among older individuals"): a Ct decrease of 3 is a 10-fold increase in viral load, so a
Beta score b (Ct of the category minus Ct of the reference) corresponds to a viral-load
factor (category / reference) of 10 ** (-b / 3).
"""
import math
import os
import sys
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def factor(beta):
"""Viral-load factor (category / reference) for a Ct difference beta."""
return 10 ** (-beta / 3)
# ------------------------------------------------------------------ inputs (Table 1)
# (label, site, beta as printed, multiplier to the printed unit, printed factor)
TABLE1 = [
("Sample date (per year)", "Oxford", 0.008, 1, 0.99),
("Sample date (per year)", "Northumbria", 0.013, 1, 0.99),
("Age (per 10 years)", "Oxford", -0.016, 10, 1.13),
("Age (per 10 years)", "Northumbria", -0.021, 10, 1.17),
("Known prior exposure", "Oxford", 1.56, 1, 0.30),
("Known prior exposure", "Northumbria", -0.12, 1, 1.10),
("Vaccinated", "Oxford", 1.44, 1, 0.33),
("Vaccinated", "Northumbria", 0.34, 1, 0.77),
("B.1.177 vs Alpha, unvaccinated", "Oxford", 1.00, 1, 2.15),
("Delta vs Alpha, unvaccinated", "Oxford", -0.01, 1, 1.01),
("BA.1 vs Delta, unvaccinated", "Northumbria", -0.95, 1, 2.07),
("B.1.177 vs Alpha, vaccinated", "Oxford", 1.82, 1, 0.25),
("Delta vs Alpha, vaccinated", "Oxford", -1.37, 1, 2.86),
("BA.1 vs Delta, vaccinated", "Northumbria", -0.59, 1, 1.57),
("B.1.177 vs Alpha, prior exposure", "Oxford", 5.17, 1, 0.02),
("Delta vs Alpha, prior exposure", "Oxford", -3.05, 1, 10.39),
("BA.1 vs Delta, prior exposure", "Northumbria", -0.35, 1, 1.31),
("AstraZeneca vs Pfizer", "Oxford", -0.19, 1, 1.16),
("AstraZeneca vs Pfizer", "Northumbria", -0.10, 1, 1.08),
(">=2 doses vs 1", "Oxford", -0.09, 1, 1.07),
(">=2 doses vs 1", "Northumbria", 0.07, 1, 0.95),
("Other ethnicities vs White", "Oxford", 0.24, 1, 0.83),
("Other ethnicities vs White", "Northumbria", 0.46, 1, 0.70),
("Male vs female", "Oxford", -0.11, 1, 1.09),
("Male vs female", "Northumbria", 0.50, 1, 0.68),
("Health care worker", "Oxford", 0.44, 1, 0.71),
("Health care worker", "Northumbria", 0.28, 1, 0.81),
]
BETA_B177 = 1.00 # Table 1: B.1.177 vs Alpha (ref), unvaccinated, no known prior exposure, Oxford
ABSTRACT_PCT_LOWER = 44 # Abstract and Results: "44% lower among Alpha" / "44% reduction ... with Alpha"
DISCUSSION_PCT_HIGHER = 115
# ------------------------------------------------------------------ 1. main comparison
print("== 1. B.1.177 vs Alpha, unvaccinated, no known prior exposure (Oxford) ==")
b177_over_alpha = factor(BETA_B177)
alpha_over_b177 = 1 / b177_over_alpha
print(f"Beta (Ct B.1.177 - Ct Alpha) = {BETA_B177:+.2f}")
print(f"B.1.177 / Alpha viral load = 10^(-1/3) = {b177_over_alpha:.4f}")
print(f"Alpha / B.1.177 viral load = 10^(+1/3) = {alpha_over_b177:.4f} (Table 1 prints 2.15)")
print(f"Alpha relative to B.1.177 = {100 * (alpha_over_b177 - 1):+.1f}% (Discussion: +115%)")
print(f"B.1.177 relative to Alpha = {100 * (b177_over_alpha - 1):+.1f}%")
abs_ratio = 1 - ABSTRACT_PCT_LOWER / 100
# Alpha/B.1.177 = 0.56 -> Ct(Alpha) - Ct(B.1.177) = -3*log10(0.56) -> beta = 3*log10(0.56)
beta_implied_abs = 3 * math.log10(abs_ratio)
print(f"Abstract: Alpha / B.1.177 = {abs_ratio:.2f}")
print(f" implied Beta (Ct B.1.177 - Ct Alpha) = {beta_implied_abs:+.3f} (Table 1: +1.00)")
beta_implied_disc = -3 * math.log10(1 / (1 + DISCUSSION_PCT_HIGHER / 100))
print(f"Discussion: Alpha / B.1.177 = {1 + DISCUSSION_PCT_HIGHER / 100:.2f}; implied Beta = {beta_implied_disc:+.3f}")
beta_b177_44_lower = -3 * math.log10(abs_ratio)
print(f"Beta needed for 'B.1.177 44% lower than Alpha' = {beta_b177_44_lower:+.3f}")
print("\n-- Conversions that might give 44% from Beta = 1.00 --")
cands = [
("3 Ct per 10-fold, B.1.177 lower than Alpha (paper's rule)", 1 - 10 ** (-1 / 3)),
("3.3 Ct per 10-fold", 1 - 10 ** (-1 / 3.3)),
("1 Ct = 2-fold", 1 - 2 ** -1),
("4 Ct per 10-fold", 1 - 10 ** (-1 / 4)),
]
for lab, v in cands:
print(f" {lab:58s} -> {100 * v:.1f}% lower")
print(" (none of these makes Alpha the lower one; only the 4-Ct rule gives ~44%)")
print(" Other text percentages under a 4-Ct rule (check whether the paper used it):")
print(f" Vaccinated Oxford, Beta 1.44 -> {100 * (1 - 10 ** (-1.44 / 4)):.1f}% reduction (text: 67%)")
print(f" Delta vs Alpha vacc, Beta -1.37 -> factor {10 ** (1.37 / 4):.2f} (Table 1: 2.86)")
print(f" Age Northumbria, -0.21 per 10y -> {100 * (10 ** (0.21 / 4) - 1):.1f}% (text: 17%)")
# preprint (bioRxiv v1, 2022-12-02) abstract: Ct "0.94 lower among Alpha ... compared [to] B.1.177"
pre = 0.94
print(f"\nPreprint abstract: Ct 0.94 lower with Alpha -> Alpha/B.1.177 = {10 ** (pre / 3):.2f} "
f"(+{100 * (10 ** (pre / 3) - 1):.0f}%); B.1.177 {100 * (1 - 10 ** (-pre / 3)):.1f}% lower")
# ------------------------------------------------------------------ 2. Table 1 consistency
print("\n== 2. Table 1: printed factor vs 10^(-Beta/3) for every entry ==")
rows = []
for lab, site, b, mult, printed in TABLE1:
calc = factor(b * mult)
recip = abs(printed - round(1 / calc, 2)) <= 0.011
ok = abs(printed - round(calc, 2)) <= 0.011
rows.append((lab, site, b * mult, calc, printed, ok))
flag = "matches" if ok else ("RECIPROCAL" if recip else "MISMATCH")
print(f" {lab:34s} {site:11s} beta={b * mult:+.3f} 10^(-b/3)={calc:7.3f} printed={printed:6.2f} {flag}")
n_ok = sum(r[5] for r in rows)
print(f" {n_ok} of {len(rows)} entries match 10^(-Beta/3) to 2 decimals; the exception is "
f"B.1.177 vs Alpha, unvaccinated (expected {factor(BETA_B177):.2f}, printed 2.15 = 1/{factor(BETA_B177):.3f})")
# ------------------------------------------------------------------ 3. other text percentages
print("\n== 3. Other percentages in the text ==")
pcts = [
("Vaccination, Oxford (reduction)", 1.44, "Abstract/Results 67%; Discussion 75%", "red"),
("Vaccination, Northumbria (reduction)", 0.34, "Discussion 30%", "red"),
("Delta vs Alpha, vaccinated, Oxford", -1.37, "Abstract/Results/Discussion '286% higher'", "inc"),
("Age per 10 y, Oxford", -0.16, "Results 14%; Discussion 13%", "inc"),
("Age per 10 y, Northumbria", -0.21, "Abstract/Results/Discussion 17%", "inc"),
]
for lab, b, txt, kind in pcts:
f = factor(b)
if kind == "red":
print(f" {lab:38s} factor {f:.3f} -> {100 * (1 - f):.1f}% reduction [{txt}]")
else:
print(f" {lab:38s} factor {f:.3f} -> {100 * (f - 1):.1f}% higher [{txt}]")
print("\n== 4. Direction of the Delta vs Alpha effect among vaccinated (Oxford) ==")
f_delta = factor(-1.37)
print(f" Beta (Ct Delta - Ct Alpha) = -1.37 -> Delta / Alpha viral load = {f_delta:.3f} (Table 1: 2.86)")
print(f" Delta is {'HIGHER' if f_delta > 1 else 'LOWER'} than Alpha by {100 * (f_delta - 1):.0f}%; "
"the sensitivity-analysis sentence says Delta had a 'lower viral burden'")
# ================================================================== figures
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
def box(ax, x, y, w, h, edge, lw=1.4, fc="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=fc, ec=edge, lw=lw, transform=ax.transAxes))
def fig_A():
"""Two text panels joined by an annotated connector."""
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.03, 0.965, "Alpha vs B.1.177: unvaccinated, no known prior exposure (Oxford)",
fontsize=11.5, weight="bold", va="top")
LQ, RQ = "\u201c", "\u201d"
def panel(x, title, items, verdict):
box(ax, x, 0.29, 0.41, 0.6, ORANGE)
t = ax.text(x + 0.02, 0.865, title, fontsize=10, weight="bold", va="top")
fig.canvas.draw() # keep every panel title inside its box
bb = t.get_window_extent().transformed(ax.transAxes.inverted())
assert bb.x1 < x + 0.41 - 0.01, f"panel title overflows: {title}"
y = 0.79
for where, txt in items:
ax.text(x + 0.02, y, where, fontsize=10, va="top", color=INK2)
body = wrap(txt, 36)
ax.text(x + 0.02, y - 0.045, body, fontsize=10, va="top", color=INK)
y -= 0.045 + 0.042 * (body.count("\n") + 1) + 0.025
ax.text(x + 0.02, 0.325, verdict, fontsize=10.5, weight="bold", color=ORANGE)
panel(0.03, "Abstract and a Results sentence", [
("Abstract", LQ + "viral burden was 44% lower among Alpha variant infections, compared to those "
"with the predecessor strain, B.1.177" + RQ),
("Results", LQ + "This corresponds to a 44% reduction in viral loads with Alpha." + RQ),
], "States: Alpha LOWER, by 44%")
panel(0.56, "Table 1, heading, Discussion", [
("Table 1, B.1.177 row (ref = Alpha)", "Beta score 1.00 (2.15)"),
("Results subsection heading", LQ + "viral burden was higher among Alpha compared to B.1.177 "
"infections" + RQ),
("Discussion", LQ + "viral burden was, on average, 115% higher among Alpha-variant compared to "
"B.1.177 samples" + RQ),
], "States: Alpha HIGHER, by 115%")
ax.add_patch(FancyArrowPatch((0.44, 0.55), (0.56, 0.55), arrowstyle="<|-|>", mutation_scale=14,
color=INK, lw=2, transform=ax.transAxes))
ax.text(0.5, 0.58, "opposite\ndirection", ha="center", va="bottom", fontsize=10, weight="bold")
# recomputation strip
box(ax, 0.03, 0.03, 0.94, 0.21, BLUE, fc="#f3f7fd")
ax.text(0.05, 0.215, "Recomputed with the paper's rule (3 Ct = 10-fold change):", fontsize=10.5,
weight="bold", va="top", color=INK)
ax.text(0.05, 0.15, f"Beta = +1.00 Ct \u2192 Alpha / B.1.177 = 10^(1/3) = {alpha_over_b177:.2f} \u2192 "
f"Alpha {100 * (alpha_over_b177 - 1):.0f}% higher\n(equivalently B.1.177 "
f"{100 * (1 - b177_over_alpha):.0f}% lower). '44% lower with Alpha' would need Beta = "
f"{beta_implied_abs:+.2f}.", fontsize=10, va="top", color=BLUE, linespacing=1.5)
return fig
def fig_B():
"""One log axis: the Alpha/B.1.177 ratio implied by each statement."""
fig, ax = plt.subplots(figsize=(W, 3.3))
items = [
("Abstract: “44% lower among Alpha”", abs_ratio, ORANGE, "o"),
("Results: “44% reduction ... with Alpha”", abs_ratio, ORANGE, "o"),
("Discussion: “115% higher among Alpha”", 1 + DISCUSSION_PCT_HIGHER / 100, ORANGE, "o"),
("Table 1: Beta = +1.00 Ct, via 10^(Beta/3)", alpha_over_b177, BLUE, "D"),
]
ys = list(range(len(items)))[::-1]
for (lab, v, c, m), y in zip(items, ys):
ax.plot([v], [y], marker=m, ms=10, color=c, mec=BG, mew=1.5, ls="none", zorder=3)
ax.text(v * (1.06 if v > 1 else 0.94), y, f"{v:.2f}", va="center",
ha="left" if v > 1 else "right", fontsize=10, color=INK)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log"); ax.set_xlim(0.35, 3.5)
ax.minorticks_off()
ax.set_xticks([0.4, 0.5, 0.7, 1, 1.5, 2, 3]); ax.set_xticklabels(["0.4", "0.5", "0.7", "1", "1.5", "2", "3"])
ax.set_yticks(ys); ax.set_yticklabels([i[0] for i in items]); ax.set_ylim(-0.6, len(items) - 0.4)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.set_xlabel("Viral load, Alpha / B.1.177 (log scale)")
ax.text(0.97, len(items) - 0.45, "← Alpha lower", ha="right", fontsize=10, color=INK2, va="top")
ax.text(1.03, len(items) - 0.45, "Alpha higher →", ha="left", fontsize=10, color=INK2, va="top")
ax.plot([], [], "o", color=ORANGE, ms=9, label="stated in the paper")
ax.plot([], [], "D", color=BLUE, ms=9, label="recomputed from Table 1 Beta")
ax.legend(loc="lower center", bbox_to_anchor=(0.35, 1.0), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
return fig
def fig_C():
"""Calculation-flow diagram."""
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
def node(x, y, w, h, txt, edge, color=INK, bold=False, fc="white"):
box(ax, x, y, w, h, edge, fc=fc)
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=10, color=color,
weight="bold" if bold else "normal")
def arrow(p, q):
ax.add_patch(FancyArrowPatch(p, q, arrowstyle="-|>", mutation_scale=12, color=INK2, lw=1.6,
transform=ax.transAxes))
node(0.02, 0.62, 0.24, 0.26, "Table 1\nB.1.177 vs Alpha (ref)\nBeta = +1.00 Ct", ORANGE)
node(0.37, 0.62, 0.26, 0.26, "Paper's rule:\n3 Ct = 10-fold\nfactor = 10^(-Beta/3)", INK2)
node(0.73, 0.62, 0.25, 0.26, f"Alpha / B.1.177\n= 10^(1/3) = {alpha_over_b177:.2f}", BLUE, color=BLUE, bold=True)
arrow((0.26, 0.75), (0.37, 0.75)); arrow((0.63, 0.75), (0.73, 0.75))
node(0.55, 0.12, 0.2, 0.3, f"Alpha\n{100 * (alpha_over_b177 - 1):.0f}% higher\n(Discussion: 115%,\nmatches)",
BLUE, color=BLUE)
node(0.78, 0.12, 0.2, 0.3, f"B.1.177\n{100 * (1 - b177_over_alpha):.0f}% lower\nthan Alpha", BLUE, color=BLUE)
arrow((0.80, 0.62), (0.68, 0.42)); arrow((0.88, 0.62), (0.88, 0.42))
node(0.02, 0.12, 0.42, 0.3, f"Abstract and Results: Alpha 44% lower\n= Alpha / B.1.177 of {abs_ratio:.2f}\n"
f"would need Beta = {beta_implied_abs:+.2f} Ct\n(opposite sign to Table 1)", ORANGE, color=INK)
ax.text(0.495, 0.27, "≠", fontsize=22, ha="center", va="center", color=INK)
ax.text(0.02, 0.965, "From the Table 1 coefficient to a percentage", fontsize=11.5, weight="bold", va="top")
return fig
def fig_D():
"""Ct-difference number line: the Beta each statement implies."""
fig, ax = plt.subplots(figsize=(W, 2.9))
items = [
("Abstract and Results: “44% lower among Alpha”", beta_implied_abs, BLUE, "D",
"implied by the text"),
("Discussion: “115% higher among Alpha”", beta_implied_disc, BLUE, "D", "implied by the text"),
("Table 1: Beta score as printed", BETA_B177, ORANGE, "o", "printed"),
]
ys = [2, 1, 0]
for (lab, v, c, m, _), y in zip(items, ys):
ax.plot([0, v], [y, y], color=c, lw=2, zorder=2)
ax.plot([v], [y], marker=m, ms=10, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(v + (0.06 if v > 0 else -0.06), y + 0.22, f"{v:+.2f}", ha="left" if v > 0 else "right",
fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.8)
ax.set_xlim(-1.5, 1.5); ax.set_ylim(-0.6, 2.7)
ax.set_yticks(ys); ax.set_yticklabels([i[0] for i in items])
ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Ct(B.1.177) − Ct(Alpha), Ct units")
ax.text(-0.05, -0.5, "← Alpha lower burden", ha="right", fontsize=10, color=INK2)
ax.text(0.05, -0.5, "Alpha higher burden →", ha="left", fontsize=10, color=INK2)
ax.plot([], [], "o", color=ORANGE, ms=9, label="printed in Table 1")
ax.plot([], [], "D", color=BLUE, ms=9, label="Beta implied by the text")
ax.legend(loc="lower center", bbox_to_anchor=(0.3, 1.0), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
return fig
def fig_E():
"""Every Table 1 entry: printed factor vs 10^(-Beta/3)."""
fig, ax = plt.subplots(figsize=(W, 4.6))
lo, hi = 0.015, 15
ax.plot([lo, hi], [lo, hi], color=BLUE, lw=2, zorder=1, label="printed = 10^(−Beta/3) (paper's rule)")
xs = [r[3] for r in rows]; ys = [r[4] for r in rows]
okx = [x for x, r in zip(xs, rows) if r[5]]; oky = [y for y, r in zip(ys, rows) if r[5]]
ax.plot(okx, oky, "o", ms=8, color=ORANGE, mec=BG, mew=1.2, zorder=3,
label=f"Table 1 entries ({n_ok} of {len(rows)} on the line)")
bad = [r for r in rows if not r[5]]
for r in bad:
ax.plot([r[3]], [r[4]], "o", ms=11, mfc="none", mec=ORANGE, mew=2.2, zorder=4)
ax.plot([r[3]], [r[3]], "D", ms=9, color=BLUE, mec=BG, mew=1.2, zorder=4)
ax.annotate("", xy=(r[3], r[4] * 0.93), xytext=(r[3], r[3] * 1.08),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.4))
ax.text(r[3] * 0.8, r[4], f"B.1.177 vs Alpha, unvaccinated\nprinted {r[4]:.2f}", ha="right", va="center",
fontsize=10)
ax.text(r[3] * 1.2, r[3] * 0.9, f"expected {r[3]:.2f}", ha="left", va="top", fontsize=10, color=INK)
ax.set_xscale("log"); ax.set_yscale("log")
ax.set_xlim(lo, hi); ax.set_ylim(lo, hi)
ticks = [0.02, 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.set_yticks(ticks); ax.set_yticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Viral-load factor recomputed from the printed Beta, 10^(−Beta/3)")
ax.set_ylabel("Viral-load factor printed in Table 1")
ax.legend(loc="upper left", frameon=False, fontsize=10)
fig.tight_layout()
return fig
CAPTIONS = {
"A": "The two sets of statements the paper makes about the same comparison (B.1.177 vs Alpha, "
"unvaccinated with no known prior exposure, Oxford samples), quoted verbatim. The orange-outlined "
"panels hold the paper's text and Table 1 entry. On the left, the Abstract and one Results sentence "
"say viral burden was 44% lower with Alpha. On the right, the Table 1 coefficient, the heading of the "
"Results subsection that reports it, and the Discussion say Alpha was higher, by 115%. The blue strip "
"at the bottom is the recomputation from the printed Beta score with the paper's own 3-Ct-per-10-fold "
"rule; it agrees with the right-hand panel. A 44% reduction with Alpha would need a Beta of -0.76.",
"B": "The Alpha / B.1.177 viral-load ratio implied by each statement in the paper (orange dots), and the "
"ratio recomputed from the Table 1 Beta score of +1.00 with the paper's 3-Ct-per-10-fold rule (blue "
"diamond), on one log axis. The vertical line marks no difference. The Abstract and the Results "
"sentence imply 0.56 (Alpha lower); the Discussion and the recomputation give 2.15 (Alpha higher).",
"C": "From the Table 1 coefficient to a percentage. The printed Beta of +1.00 Ct (orange), converted with "
"the paper's rule, gives Alpha / B.1.177 = 2.15: Alpha 115% higher, or B.1.177 54% lower (blue). The "
"Abstract and Results statement that Alpha was 44% lower would need a Beta of -0.76 Ct, the opposite "
"sign.",
"D": "The Ct difference, Ct(B.1.177) - Ct(Alpha), printed in Table 1 (orange) and the difference each text "
"statement implies under the paper's 3-Ct-per-10-fold rule (blue). The Discussion implies +1.00, "
"matching Table 1; the Abstract and Results sentence imply -0.76, the opposite sign.",
"E": "Each orange dot is one of the 27 viral-load factors printed in Table 1, plotted against the factor "
"recomputed from the Beta score printed beside it, 10^(-Beta/3) (both axes log scale; age is per 10 "
"years, sample date per year). The blue line is where the two agree. Twenty-six entries lie on it. "
"The exception (hollow orange circle) is B.1.177 vs Alpha among unvaccinated people with no known "
"prior exposure: its Beta of +1.00 implies 0.46 (blue diamond), but 2.15, the reciprocal, is printed. "
"The printed 2.15 is the Alpha / B.1.177 ratio, consistent with the Discussion; the Abstract's "
"\"44% lower among Alpha\" matches neither orientation.",
}
FIGS = {"A": fig_A, "B": fig_B, "C": fig_C, "D": fig_D, "E": fig_E}
for name, fn in FIGS.items():
save(fn(), os.path.join(CAND, f"cand_{name}"))
print("\nwrote candidates/cand_A..E (.svg, .png)")
if "--pick" in sys.argv:
picks = sys.argv[sys.argv.index("--pick") + 1].upper().split(",")
for i, name in enumerate(picks, 1):
save(FIGS[name](), os.path.join(HERE, f"fig{i}"))
print(f"\nfig{i} = cand_{name}\nFigure {i}. {CAPTIONS[name]}")
The abstract and the body disagree on nitric oxide after arg1 knockdown; which wording is right depends on the comparison
PLoS Pathog (2012) · doi:10.1371/journal.ppat.1002417 · PMC3261917
Summary
The abstract reports that miRNAi-mediated knockdown of host arginase 1 (arg1) in BHK cells "led to increased generation of nitric oxide and reduced parasite burden (p<0.005)." The other three places in the body that describe the nitric oxide (NO) result of the same Fig. 6I experiment use the opposite wording: the Results subsection heading reads "promotes parasite killing without enhanced NO production," the Results prose says the enhanced killing "was not accompanied by greater parasite-induced NO production (Fig. 6I)," the Fig. 6I legend says "Arg1 knockdown did not induce NO production," and the Discussion says the effect "did not appear to be driven by enhanced production of NO." This is a wording inconsistency about the direction of a single result.
It can be resolved only by naming the comparison, because the two descriptions refer to different control groups and both can be literally true. The paper prints no NO numbers — they appear only inside the Fig. 6I bar chart — so we digitised the four bars from the publisher's figure image. Relative to infected non-targeted control cells, knockdown NO is about 1.37 times higher (≈5.5 vs ≈4.0 µM); the figure draws no significance marker for that difference, and this is the comparison the abstract's "increased generation of nitric oxide" fits. Relative to uninfected cells, knockdown NO is essentially unchanged (≈5.5 vs ≈5.4 µM); that is the comparison the heading, prose, legend and Discussion fit when they say NO was not induced or not enhanced. So the inconsistency is in how the finding is phrased and against which reference, not a contradiction of the underlying measurement. It matters because the abstract and the body read as opposite directional claims about the same experiment, and the paper's stated mechanism is NO-independent (polyamine-mediated), so a reader of the abstract alone could take the opposite of what the body concludes.
What the paper reports
The abstract sentence (the two claims are "increased NO" and "reduced parasite burden"):
"miRNAi-mediated selective knockdown of hamster arginase 1 (arg1) in BHK cells led to increased generation of nitric oxide and reduced parasite burden (p<0.005)." (Abstract)
The Results subsection heading for the same experiment:
"Knockdown of host arg1 promotes parasite killing without enhanced NO production" (Results, subsection heading)
The Results prose, citing Fig. 6I directly:
"This enhancement of parasite killing by knockdown of arg1 in BHK cells was not accompanied by greater parasite-induced NO production (Fig. 6I), even though BHK cells were fully capable of generating NO (see Fig. 5B). Thus it would appear that parasite-induced macrophage arg1 contributes to L. donovani replication through mechanisms other than reduction of NO production." (Results, same subsection)
The Fig. 6I legend:
"Arg1 knockdown did not induce NO production in BHK cells infected with L. donovani." (Figure 6, legend, panel I)
The Discussion:
"This increased control of infection mediated by targeted arg1 knockdown did not appear to be driven by enhanced production of NO by the isolated infected macrophage, suggesting that the parasite-induced arginase has a pathological effect through the increase in polyamines that promote parasite growth." (Discussion)
What the paper's own figure shows
Method. No per-cell NO values are archived in text; the Fig. 6I bars are the only source. We downloaded the publisher's Figure 6 image (journals.plos.org, image g006, size=large) and measured the four bars in pixels against the panel's y-axis ticks (0 to 8 µM), using python. The extraction — axis calibration, bar tops, SD caps, and the position of the one significance bracket — is in recompute.py.txt, section 2, and reruns from the downloaded image. Reading precision is about ±0.05 µM (3 px), so the values below are estimates, not the authors' numbers.
Digitised Fig. 6I values (nitrites + nitrates, mean ± SD, µM):
| Uninfected | Infected (48 h) | |
|---|---|---|
| Arg1 knockdown | ≈5.4 ± 0.3 | ≈5.5 ± 0.4 |
| Non-targeted control | ≈6.3 ± 0.5 | ≈4.0 ± 0.5 |
The single significance bracket (**) runs between the two non-targeted bars (uninfected vs infected): infection lowered NO in control cells (≈6.3 → ≈4.0 µM). Knockdown cells held their baseline (≈5.4 → ≈5.5 µM). So:
- Relative to infected non-targeted control cells, knockdown NO is higher: ≈5.5 vs ≈4.0 µM, a ratio of about 1.37. The figure shows no significance marker for this difference.
- Relative to uninfected cells, infection did not induce NO in knockdown cells: ≈5.5 vs ≈5.4 µM (about +0.1 µM). Knockdown NO is also not above the uninfected non-targeted level (≈5.5 vs ≈6.3 µM).
Both the abstract and the body can therefore be read as literally true, each against a different control. The body's heading, prose and legend speak of "induction" and of NO relative to the uninfected or baseline state; the abstract's "increased generation of nitric oxide" fits a comparison of knockdown with the infected non-targeted control.
The second claim in the abstract sentence (reduced parasite burden) is supported by the Results, which give p<0.01 at 48 h and p<0.001 at 72 h (Fig. 6H); the abstract's bound p<0.005 is satisfied by the 72 h value. The knockdown efficiency (>90% reduction in arg1 mRNA, p=0.002; Fig. 6G) is not in dispute.
| Quantity | Location | Reported | Digitised / cross-check |
|---|---|---|---|
| NO after arg1 knockdown (wording) | Abstract | "increased generation of nitric oxide" | Fits KD vs infected non-targeted control: ≈5.5 vs ≈4.0 µM (×1.37; no significance marker drawn) |
| NO after arg1 knockdown (wording) | Results heading | "without enhanced NO production" | Fits KD infected vs uninfected: ≈5.5 vs ≈5.4 µM (not induced) |
| NO after arg1 knockdown (wording) | Results prose (Fig. 6I) | "not accompanied by greater parasite-induced NO production" | Infection did not raise NO in KD cells above its uninfected level |
| NO after arg1 knockdown (wording) | Fig. 6I legend | "did not induce NO production" | KD infected ≈5.5 µM vs KD uninfected ≈5.4 µM (no induction) |
| NO after arg1 knockdown (wording) | Discussion | "did not appear to be driven by enhanced production of NO" | Hedged; consistent with the no-induction reading |
| Only significance marker in Fig. 6I (**) | Fig. 6I | between non-targeted bars | Non-targeted uninfected ≈6.3 vs infected ≈4.0 µM (infection lowered control NO) |
| Reduced parasite burden p-value | Abstract | p<0.005 | Consistent: Results give p<0.01 (48 h), p<0.001 (72 h); the 72 h value satisfies p<0.005 |
| arg1 mRNA knockdown efficiency | Results / Fig. 6G | >90% reduction, p=0.002 | Stated once; not in dispute |
Figure
Figure 1. The abstract's wording (top left) and the body's wording (top right) describe the same Fig. 6I knockdown experiment but fit different reference groups. Below, the four Fig. 6I bars are shown as means ± SD on one axis, digitised from the publisher's figure image by pixel measurement (filled = Arg1 knockdown, open = non-targeted; the paper prints no NO numbers). Comparison (a): knockdown vs non-targeted among infected cells, ≈5.5 vs ≈4.0 µM (×1.37), which has no significance marker in the figure and matches the abstract's "increased" wording. Comparison (b): knockdown infected vs uninfected, ≈5.5 vs ≈5.4 µM, essentially unchanged, which matches the body's "not induced" wording. The paper's only significance mark (**) is between the two non-targeted bars (uninfected vs infected). All quotes are verbatim from the PMC-archived text; the blue values are digitised estimates, not the authors' numbers, and the paper's own figure is not reproduced.
Why it matters
The two statements sit in different parts of the paper — one in the abstract, which databases index, and the rest in the Results, legend and Discussion — and they read as opposite directional claims about the same measurement. The abstract's "increased generation of nitric oxide," taken on its own, suggests the canonical arginase-versus-NOS2 story in which inhibiting host arginase restores NO and that NO kills the parasite. The body instead concludes that enhanced parasite control was not driven by induction of NO and attributes it to reduced polyamine synthesis, explicitly NO-independent. Read against the Fig. 6I values, neither statement is simply wrong: the abstract's wording fits a comparison with infected control cells (for which the figure shows no significance marker), while the body's wording fits a comparison with uninfected or baseline cells. If a reader reconciles the abstract and the body by the wrong comparison, they could take away a mechanism the paper does not claim. The paper's principal conclusions are unaffected: arg1 knockdown reduces parasite burden, STAT6 knockdown reduces arg1 and enhances parasite control, and parasite-induced STAT6/arg1 drives progressive disease; these depend on the NO-independent reading. A secondary, cosmetic point: the abstract gives p<0.005 for the burden effect while the Results give p<0.01 at 48 h and p<0.001 at 72 h; the bound is satisfied by the 72 h value and changes no conclusion.
Explanations considered
- The two statements compare with different control groups. This is the resolution supported by the figure. "Increased generation of nitric oxide" is true relative to the infected non-targeted control (≈5.5 vs ≈4.0 µM; no significance marker shown). "Without enhanced NO production" / "did not induce NO" is true relative to the uninfected or baseline state (≈5.5 vs ≈5.4 µM). The inconsistency is which reference each text names, not the measurement. The appropriate fix is to make the comparison explicit in both places (see Suggested correction); intent is not assessed.
- Two separate NO measurements (different cells, time points, or stimuli). No. All five locations name the same model: miRNAi arg1 knockdown in BHK cells infected with L. donovani, NO measured in Fig. 6I by Griess assay at 48 h. The Results prose and the Fig. 6I legend describe exactly the experiment the abstract summarises.
- A word slip in the abstract (e.g. "increased" was meant to be "without increased"). Possible, but not required: the digitised values show that "increased" is defensible against the infected control, so a simple reversal of the abstract is not the only reconciliation and could itself misdescribe the data relative to that control. The write-up does not assert which phrasing the authors intended.
- A typesetting or PMC conversion artifact. No. The publisher's version of record carries the identical abstract sentence and the identical body statements; the wording difference is present in both.
- The supplements resolve it. No. The four supporting figures cover polyamine content, BHK infection, the nor-NOHA arginase inhibitor, and STAT6 activation. None reports NO after arg1 knockdown, which appears only in main Fig. 6I. There is no source-data file.
Two smaller, cosmetic inconsistencies, noted for completeness; neither changes any conclusion:
- The Fig. 6I legend ends "(p<0.001)" for the one significance bracket, but the same legend's key defines ", p<0.01" (and "*, p<0.001"). The two uses of "" disagree.
- The Results prose cites "see Fig. 5B" for BHK cells being able to make NO, but Fig. 5B is the arg1/NOS2 mRNA ratio in splenic macrophages; the BHK NO data are in Fig. 6B.
Status
As of 2026-10-23. Version of record checked at the publisher (journals.plos.org, PLoS Pathogens): the abstract sentence, the Results heading, the Results prose, and the Fig. 6I legend match the PMC archive word for word, and the Discussion sentence matches. The Fig. 6I values were digitised from the publisher's Figure 6 image (image g006, size=large); the SHA-256 of the downloaded image is printed by recompute.py.txt. Supplements S1-S4 were identified from the publisher page and none covers NO after arg1 knockdown; there is no source-data file. Corrections: none found. Crossref lists no update/correction relation; src/known.py returned no PMC or Europe PMC notices; the Europe PMC record for PMID 22275864 has no comment/correction list and no erratum citing the DOI; the publisher page shows no correction notice and zero reader comments. PubPeer could not be checked (API behind Cloudflare, search page returned HTTP 403); web searches found no PubPeer thread or Retraction Watch coverage. Sources: src/show.py PMC3261917 --full (PMC-archived abstract, Results, Fig. 5/6 legends, Discussion, Methods); publisher HTML and Figure 6 image; Crossref and Europe PMC via src/known.py.
Suggested correction
Reconcile the abstract and the body by stating, in both, the comparison being made. For example, the abstract and the Fig. 6I-related text could say that after infection, arg1-knockdown cells retained their baseline NO while non-targeted control cells lost NO, so that knockdown NO was higher than in infected controls but was not induced above uninfected levels. We do not recommend simply reversing the abstract to "without increased generation of nitric oxide," because against the infected non-targeted control that wording could itself be inaccurate; the correct phrasing depends on which comparison the authors intend. The two cosmetic points above (the "**" p-value in the Fig. 6I legend, and the "see Fig. 5B" vs Fig. 6B citation) could be corrected in the same notice. A published author correction is the appropriate mechanism; no data or main conclusions need revision.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""
Checks for PMC3261917
"Progressive Visceral Leishmaniasis Is Driven by Dominant Parasite-induced
STAT6 Activation and STAT6-dependent Host Arginase 1 Expression"
PLoS Pathog 2012, doi:10.1371/journal.ppat.1002417
The issue is a wording inconsistency about nitric oxide (NO) after miRNAi
knockdown of host arginase 1 (arg1) in BHK cells. The abstract says knockdown
"led to increased generation of nitric oxide"; the Results heading, Results
prose, Fig. 6I legend and Discussion say NO was not enhanced / not induced.
Whether each statement is true depends on which control the knockdown is
compared with, so this script also reads the Fig. 6I bars from the
publisher's published figure image.
Sections
1. An ENCODING (not a computation) of what each quoted location says.
2. Digitisation of Fig. 6I from the publisher's Figure 6 image
(downloaded at run time to a temporary folder; the image is not stored
or redrawn here).
3. Computed comparisons on the digitised values and on stated p-values.
4. Figures: fig1.svg/png (the layout used in writeup.md) and five
alternative layouts in ./candidates/ (cand1..cand5).
Run: python recompute.py.txt
Needs numpy and matplotlib, and network access to journals.plos.org.
"""
import os, sys, hashlib, tempfile, textwrap, urllib.request
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
from matplotlib.patches import FancyBboxPatch, ConnectionPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ----------------------------------------------------------------------
# 1. ENCODING of quoted statements (hand-coded, not computed)
# ----------------------------------------------------------------------
# Each statement is quoted verbatim from the PMC-archived text
# (src/show.py PMC3261917 --full). The second field is the comparison the
# wording implies, as read by the write-up; it is an encoding, not a result.
statements = [
("Abstract",
"led to increased generation of nitric oxide and reduced parasite burden (p<0.005)",
"NO increased (reference not stated)"),
("Results heading",
"Knockdown of host arg1 promotes parasite killing without enhanced NO production",
"NO not enhanced"),
("Results text",
"was not accompanied by greater parasite-induced NO production (Fig. 6I)",
"no greater parasite-induced NO (i.e. no induction by infection)"),
("Fig. 6I legend",
"Arg1 knockdown did not induce NO production in BHK cells infected with L. donovani.",
"NO not induced"),
("Discussion",
"did not appear to be driven by enhanced production of NO",
"NO not enhanced (hedged)"),
]
print("=" * 72)
print("1. ENCODING OF QUOTED STATEMENTS (hand-coded reading, not a computation)")
print("=" * 72)
for loc, q, reading in statements:
print(" %-16s %s" % (loc, reading))
n_up = sum(1 for s in statements if s[2].startswith("NO increased"))
print(" Locations whose wording says NO increased: %d of %d" % (n_up, len(statements)))
# ----------------------------------------------------------------------
# 2. DIGITISE Fig. 6I from the publisher's Figure 6 image
# ----------------------------------------------------------------------
FIG_URL = ("https://journals.plos.org/plospathogens/article/figure/image"
"?size=large&id=10.1371/journal.ppat.1002417.g006")
tmpdir = tempfile.mkdtemp(prefix="pmc3261917_")
img_path = os.path.join(tmpdir, "g006.png")
try:
req = urllib.request.Request(FIG_URL, headers={"User-Agent": "Mozilla/5.0"})
with urllib.request.urlopen(req, timeout=60) as r, open(img_path, "wb") as f:
f.write(r.read())
except Exception as e:
sys.exit("Could not download Figure 6 image (%s); digitisation needs it." % e)
sha = hashlib.sha256(open(img_path, "rb").read()).hexdigest()
img = mpimg.imread(img_path)[..., :3].mean(axis=2)
dark = img < 0.5
print("\n" + "=" * 72)
print("2. DIGITISATION OF Fig. 6I (publisher image g006, size=large)")
print("=" * 72)
print(" image %d x %d px, sha256 %s" % (img.shape[1], img.shape[0], sha))
assert img.shape[:2] == (2556, 2042), "image size changed; re-check pixel ranges"
def runs(v, off=0):
out, s = [], None
for i, b in enumerate(v):
if b and s is None: s = i
if not b and s is not None: out.append((s + off, i - 1 + off)); s = None
if s is not None: out.append((s + off, len(v) - 1 + off))
return out
# Panel I sits at x 1520-2042, y 1930-2440 of the image. The y axis line is at
# x = 1541-1542; tick marks extend left of it. Read the tick rows at x = 1535.
tick_runs = runs(dark[1930:2440, 1535], 1930)
tick_y = np.array([(a + b) / 2 for a, b in tick_runs])
tick_val = np.array([8, 6, 4, 2, 0], float) # axis labels, top to bottom
assert len(tick_y) == 5, tick_runs
slope, icpt = np.polyfit(tick_y, tick_val, 1) # uM per pixel row
resid = tick_val - (slope * tick_y + icpt)
px_per_uM = -1 / slope
print(" tick rows (px):", ", ".join("%.1f" % y for y in tick_y), "-> 0..8 uM")
print(" scale %.2f px per uM; max tick residual %.4f uM" % (px_per_uM, abs(resid).max()))
to_uM = lambda y: slope * y + icpt
# Bar column ranges (px), read from the image by eye; tops and caps are
# then found automatically. Black bars = Arg1 knockdown, white = non-targeted.
bars = {
("uninfected", "knockdown"): (1578, 1652),
("uninfected", "non-targeted"): (1652, 1727),
("infected", "knockdown"): (1836, 1910),
("infected", "non-targeted"): (1910, 1984),
}
Y_MIN = 2035 # below the significance bracket (row ~2031) and the legend key
D = {}
for key, (x0, x1) in bars.items():
tops = []
for x in (x0 + 6, x1 - 6): # near the bar edges, away from the SD stem
r = [a for a, b in runs(dark[Y_MIN:2424, x], Y_MIN)]
tops.append(r[0] + 0.5) # centre of the 2-px top outline
top = float(np.mean(tops))
xc = (x0 + x1) / 2
caps = []
for x in range(int(xc) - 8, int(xc) + 9):
if abs(x - xc) <= 2: continue # skip the vertical SD stem
short = [a for a, b in runs(dark[2008:int(top) - 3, x], 2008) if b - a <= 4]
if short: caps.append(max(short) + 0.5) # lowest short run above the bar = SD cap
cap = float(np.median(caps))
D[key] = dict(mean=to_uM(top), sd=to_uM(cap) - to_uM(top), xc=xc)
print(" %-10s %-13s mean %.2f uM SD %.2f uM (top row %.1f, cap row %.1f)"
% (key[0], key[1], D[key]["mean"], D[key]["sd"], top, cap))
# The one significance bracket: a horizontal 2-px line at row ~2031 above the bars.
br = [r for r in runs(dark[2031, 1560:2020], 1560) if r[1] - r[0] > 40]
assert len(br) == 1, br
bx0, bx1 = br[0]
def bar_at(x):
for k, (a, b) in bars.items():
if a <= x <= b: return k
print(" significance bracket (**) spans x %d-%d: from %s %s to %s %s"
% (bx0, bx1, *bar_at(bx0), *bar_at(bx1)))
print(" reading precision: about +/-0.05 uM (3 px); values are estimates.")
# ----------------------------------------------------------------------
# 3. COMPUTED COMPARISONS
# ----------------------------------------------------------------------
print("\n" + "=" * 72)
print("3. COMPARISONS ON THE DIGITISED Fig. 6I MEANS")
print("=" * 72)
ku = D[("uninfected", "knockdown")]["mean"]; nu = D[("uninfected", "non-targeted")]["mean"]
ki = D[("infected", "knockdown")]["mean"]; ni = D[("infected", "non-targeted")]["mean"]
cmp = {
"KD infected vs non-targeted infected (abstract-type comparison)": (ki, ni),
"KD infected vs KD uninfected (induction by infection)": (ki, ku),
"KD infected vs non-targeted uninfected": (ki, nu),
"non-targeted infected vs non-targeted uninfected (** bracket)": (ni, nu),
}
R = {}
for name, (a, b) in cmp.items():
R[name] = (a - b, a / b)
print(" %-66s diff %+.2f uM ratio %.2f" % (name, a - b, a / b))
ratio_kd_nt_inf = ki / ni
print(" -> Relative to infected non-targeted cells, knockdown NO is higher (x%.2f);"
% ratio_kd_nt_inf)
print(" no significance marker is drawn for this difference.")
print(" -> Relative to uninfected knockdown cells, infection changed NO by %+.2f uM"
% (ki - ku))
print(" (not induced).")
print("\n Stated p-value checks")
abs_bound = 0.005
results_burden = {"48 h": 0.01, "72 h": 0.001} # Results: p<0.01, p<0.001
for t, p in results_burden.items():
print(" parasite burden %s: stated p<%g; within the abstract's p<%g? %s"
% (t, p, abs_bound, p <= abs_bound))
legend_key = {"*": 0.05, "**": 0.01, "***": 0.001} # Fig. 6 legend key
panel_I_note = ("**", 0.001) # panel I text: "(**p<0.001)"
print(" Fig. 6I legend: '%s' given as p<%g; key defines '%s' as p<%g; same? %s"
% (panel_I_note[0], panel_I_note[1], panel_I_note[0],
legend_key[panel_I_note[0]], panel_I_note[1] == legend_key[panel_I_note[0]]))
print(" arg1 mRNA knockdown efficiency: >90% reduction, p = 0.002 (Fig. 6G); stated once")
# ----------------------------------------------------------------------
# 4. FIGURES
# ----------------------------------------------------------------------
BG = "#fcfcfb"; INK = "#0b0b0b"; SEC = "#52514e"; ORANGE = "#eb6834"; BLUE = "#2a78d6"
GRID = "#e4e3e0"; OR_BG = "#fdf0ea"; BL_BG = "#edf3fb"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": SEC, "axes.labelcolor": INK, "xtick.color": SEC, "ytick.color": SEC,
"figure.facecolor": BG, "savefig.facecolor": BG, "axes.facecolor": BG,
"xtick.labelsize": 10, "ytick.labelsize": 10, "lines.linewidth": 1.5})
W = 7.2
os.makedirs(os.path.join(HERE, "candidates"), exist_ok=True)
def wrap(s, n): return "\n".join(textwrap.wrap(s, n))
def save(fig, path_stem):
fig.savefig(path_stem + ".svg")
fig.savefig(path_stem + ".png", dpi=200)
plt.close(fig)
def panel(ov, x, y, w, h, fc, ec):
ov.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.008",
fc=fc, ec=ec, lw=1.5, transform=ov.transAxes, zorder=1))
def overlay(fig):
ov = fig.add_axes([0, 0, 1, 1]); ov.axis("off"); ov.set_xlim(0, 1); ov.set_ylim(0, 1)
ov.set_zorder(-1); return ov
Q_ABS = '"...knockdown of hamster arginase 1 (arg1) in BHK cells led to increased generation of nitric oxide and reduced parasite burden (p<0.005)."'
Q_BODY = [("Results heading", '"...promotes parasite killing without enhanced NO production"'),
("Results text", '"...was not accompanied by greater parasite-induced NO production (Fig. 6I)"'),
("Fig. 6I legend", '"Arg1 knockdown did not induce NO production..."'),
("Discussion", '"...did not appear to be driven by enhanced production of NO..."')]
CONDS = [("uninfected", "knockdown"), ("uninfected", "non-targeted"),
("infected", "knockdown"), ("infected", "non-targeted")]
LAB_ABS = "true if compared with infected\nnon-targeted cells: %.1f vs %.1f µM\n(×%.2f; no significance marker)" % (ki, ni, ratio_kd_nt_inf)
LAB_BODY = "true if compared with uninfected\ncells: %.1f vs %.1f µM (not induced)" % (ki, ku)
SRC = ("Quotes verbatim from the PMC-archived text. Blue values digitised by pixel measurement from the "
"publisher's Fig. 6I image (mean ± SD, ±0.05 µM reading error); the paper prints no NO numbers.")
def dot_axis(ax, xpos, show_legend=True, ylabel=True):
"""Digitised Fig. 6I means ± SD as dots on one 0-8 uM axis."""
ax.set_ylim(0, 8); ax.set_yticks([0, 2, 4, 6, 8])
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"): ax.spines[s].set_visible(False)
for k in CONDS:
m, sd = D[k]["mean"], D[k]["sd"]; x = xpos[k]
filled = k[1] == "knockdown"
ax.errorbar(x, m, yerr=sd, fmt="o", ms=9, color=BLUE, mfc=BLUE if filled else BG,
mec=BLUE, mew=2, elinewidth=2, capsize=4, zorder=3)
if ylabel: ax.set_ylabel("Nitrites + nitrates (µM)")
if show_legend:
h = [Line2D([], [], marker="o", ls="", ms=9, color=BLUE, mfc=BLUE, mew=2, label="Arg1 knockdown"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, mfc=BG, mew=2, label="Non-targeted")]
ax.legend(handles=h, title="Digitised from Fig. 6I", frameon=False, loc="lower left",
fontsize=10, title_fontsize=10)
def bracket(ax, x0, x1, y, text, color=INK, dy=0.15, va="bottom", fs=10):
ax.plot([x0, x0, x1, x1], [y - dy, y, y, y - dy], color=color, lw=1.5, zorder=4)
ax.text((x0 + x1) / 2, y + 0.08, text, ha="center", va=va, color=color, fontsize=fs)
# ---- Layout A: text panels on top, dot plot below with the comparisons ----
def layout_A(stem):
fig = plt.figure(figsize=(W, 8.2)); ov = overlay(fig)
fig.text(0.02, 0.982, "Arg1 knockdown and NO (Fig. 6I): the abstract and the body fit different reference groups",
fontsize=10, color=SEC, va="top")
panel(ov, 0.02, 0.60, 0.40, 0.35, OR_BG, ORANGE)
fig.text(0.04, 0.932, "ABSTRACT (paper's text)", color=ORANGE, fontweight="bold", va="top")
fig.text(0.04, 0.895, wrap(Q_ABS, 36), va="top")
fig.text(0.04, 0.615, "says NO increased: fits (a)", color=ORANGE, fontweight="bold")
panel(ov, 0.44, 0.60, 0.54, 0.35, OR_BG, ORANGE)
fig.text(0.46, 0.932, "BODY, 4 locations (paper's text)", color=ORANGE, fontweight="bold", va="top")
y = 0.895
for loc, q in Q_BODY:
t = wrap(loc + ": " + q, 50)
fig.text(0.46, y, t, va="top"); y -= 0.0265 * (t.count("\n") + 1) + 0.010
fig.text(0.46, 0.615, "says NO not enhanced / not induced: fits (b)", color=ORANGE, fontweight="bold")
fig.add_artist(ConnectionPatch(xyA=(0.22, 0.598), xyB=(0.71, 0.598), coordsA="figure fraction",
coordsB="figure fraction", arrowstyle="<->", color=INK, lw=2, connectionstyle="arc3,rad=0.12"))
fig.text(0.465, 0.548, "opposite wording, same experiment", ha="center", va="top", fontweight="bold")
ax = fig.add_axes([0.10, 0.12, 0.88, 0.37])
xpos = {CONDS[0]: 0.8, CONDS[1]: 1.2, CONDS[2]: 2.3, CONDS[3]: 2.7}
dot_axis(ax, xpos)
ax.set_xlim(0.4, 4.1); ax.set_xticks([1.0, 2.5]); ax.set_xticklabels(["Uninfected", "Infected (48 h)"])
# (a) knockdown vs non-targeted, infected
ax.plot([2.85, 2.95, 2.95, 2.85], [ki, ki, ni, ni], color=INK, lw=1.5)
ax.text(3.02, (ki + ni) / 2, "(a) ×%.2f\nno significance\nmarker" % ratio_kd_nt_inf, va="center")
# (b) knockdown infected vs uninfected
ax.plot([0.8, 2.3], [ku, ki], color=SEC, lw=1.5, ls=":")
ax.text(1.55, 4.95, "(b) %+.1f µM:\nnot induced" % (ki - ku), ha="center", va="top", color=INK)
# the paper's only significance bracket
bracket(ax, 1.2, 2.7, 7.25, "** (the paper's only mark): non-targeted, uninfected vs infected",
color=SEC, dy=0.2)
fig.text(0.02, 0.012, wrap(SRC, 98), fontsize=10, color=SEC, va="bottom")
save(fig, stem)
# ---- Layout B: slope chart, quotes as side callouts ----
def layout_B(stem):
fig = plt.figure(figsize=(W, 5.8)); ov = overlay(fig)
ax = fig.add_axes([0.10, 0.22, 0.40, 0.68])
for grp, mfc, ls in (("knockdown", BLUE, "-"), ("non-targeted", BG, "--")):
m = [D[(c, grp)]["mean"] for c in ("uninfected", "infected")]
s = [D[(c, grp)]["sd"] for c in ("uninfected", "infected")]
ax.errorbar([0, 1], m, yerr=s, color=BLUE, ls=ls, lw=2, marker="o", ms=9, mfc=mfc, mew=2,
capsize=4, elinewidth=1.5, zorder=3)
ax.text(1.08, m[1], ("Arg1 knockdown" if grp == "knockdown" else "Non-targeted") + "\n%.1f µM" % m[1],
color=BLUE, va="center")
ax.text(-0.08, m[0], "%.1f" % m[0], color=BLUE, va="center", ha="right")
ax.set_ylim(0, 8); ax.set_yticks([0, 2, 4, 6, 8]); ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_xlim(-0.35, 1.55); ax.set_xticks([0, 1]); ax.set_xticklabels(["Uninfected", "Infected"])
for s in ("top", "right"): ax.spines[s].set_visible(False)
ax.set_ylabel("Nitrites + nitrates (µM)")
ax.set_title("Fig. 6I, digitised (mean ± SD)", fontsize=10, color=BLUE, loc="left")
panel(ov, 0.60, 0.57, 0.38, 0.36, OR_BG, ORANGE)
fig.text(0.62, 0.91, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
fig.text(0.62, 0.87, wrap('"...led to increased generation of nitric oxide..."', 34), va="top")
fig.text(0.62, 0.78, wrap("Fits knockdown vs non-targeted, infected: %.1f vs %.1f µM (×%.2f), no significance marker."
% (ki, ni, ratio_kd_nt_inf), 34), va="top", color=SEC)
panel(ov, 0.60, 0.14, 0.38, 0.40, OR_BG, ORANGE)
fig.text(0.62, 0.52, "RESULTS, LEGEND, DISCUSSION", color=ORANGE, fontweight="bold", va="top")
fig.text(0.62, 0.48, wrap('"without enhanced NO production" · "...not accompanied by greater parasite-induced NO production" · '
'"did not induce NO production"', 34), va="top")
fig.text(0.62, 0.27, wrap("Fits knockdown, infected vs uninfected: %.1f vs %.1f µM, not induced." % (ki, ku), 34),
va="top", color=SEC)
fig.text(0.02, 0.975, "Same experiment, two references: which comparison each text describes", fontsize=10, color=SEC, va="top")
fig.text(0.02, 0.012, wrap(SRC, 98), fontsize=10, color=SEC)
save(fig, stem)
# ---- Layout C: three columns, Abstract | digitised values | Body ----
def layout_C(stem):
fig = plt.figure(figsize=(W, 5.0)); ov = overlay(fig)
fig.text(0.02, 0.975, "Abstract vs body on NO after arg1 knockdown, read against the Fig. 6I values",
fontsize=10, color=SEC, va="top")
panel(ov, 0.02, 0.14, 0.26, 0.77, OR_BG, ORANGE)
fig.text(0.035, 0.89, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
fig.text(0.035, 0.845, wrap(Q_ABS, 23), va="top")
fig.text(0.035, 0.165, "NO increased", color=ORANGE, fontweight="bold")
panel(ov, 0.72, 0.14, 0.26, 0.77, OR_BG, ORANGE)
fig.text(0.735, 0.89, "BODY (4 places)", color=ORANGE, fontweight="bold", va="top")
fig.text(0.735, 0.845, wrap('"without enhanced NO production"; "...not accompanied by greater parasite-induced NO production"; '
'"did not induce NO production"; "did not appear to be driven by enhanced production of NO"', 23), va="top")
fig.text(0.735, 0.165, wrap("NO not enhanced / not induced", 18), color=ORANGE, fontweight="bold")
panel(ov, 0.30, 0.14, 0.40, 0.77, BL_BG, BLUE)
fig.text(0.50, 0.89, "Fig. 6I, digitised", color=BLUE, fontweight="bold", va="top", ha="center")
fig.text(0.50, 0.845, "mean ± SD, µM", color=SEC, va="top", ha="center")
rows = [("", "Uninfected", "Infected"),
("Knockdown", "%.1f ± %.1f" % (ku, D[CONDS[0]]["sd"]), "%.1f ± %.1f" % (ki, D[CONDS[2]]["sd"])),
("Non-targeted", "%.1f ± %.1f" % (nu, D[CONDS[1]]["sd"]), "%.1f ± %.1f" % (ni, D[CONDS[3]]["sd"]))]
for i, r in enumerate(rows):
yy = 0.77 - i * 0.075
fig.text(0.315, yy, r[0], va="top")
fig.text(0.515, yy, r[1], va="top", ha="center", color=BLUE if i else SEC)
fig.text(0.625, yy, r[2], va="top", ha="center", color=BLUE if i else SEC)
fig.text(0.315, 0.53, wrap("(a) knockdown vs non-targeted, infected: %.1f vs %.1f µM, ×%.2f, no significance marker"
% (ki, ni, ratio_kd_nt_inf), 36), va="top")
fig.text(0.315, 0.39, wrap("(b) knockdown, infected vs uninfected: %.1f vs %.1f µM, not induced" % (ki, ku), 36), va="top")
fig.text(0.315, 0.27, wrap("The paper's only mark (**): non-targeted, uninfected vs infected", 40), va="top", color=SEC)
fig.add_artist(ConnectionPatch(xyA=(0.28, 0.50), xyB=(0.31, 0.50), coordsA="figure fraction",
coordsB="figure fraction", arrowstyle="->", color=INK, lw=2))
fig.add_artist(ConnectionPatch(xyA=(0.72, 0.36), xyB=(0.69, 0.36), coordsA="figure fraction",
coordsB="figure fraction", arrowstyle="->", color=INK, lw=2))
fig.text(0.02, 0.012, wrap("Quotes verbatim (PMC text). Abstract fits (a); body fits (b). Values digitised from the "
"publisher's Fig. 6I image (±0.05 µM reading error); the paper prints no NO numbers.", 98),
fontsize=10, color=SEC)
save(fig, stem)
# ---- Layout D: horizontal dot-interval rows on one uM axis, quotes above ----
def layout_D(stem):
fig = plt.figure(figsize=(W, 5.8)); ov = overlay(fig)
panel(ov, 0.02, 0.75, 0.47, 0.21, OR_BG, ORANGE)
fig.text(0.035, 0.94, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
fig.text(0.035, 0.895, wrap('"...knockdown... led to increased generation of nitric oxide..."', 42), va="top")
fig.text(0.035, 0.77, "NO increased: fits (a)", color=ORANGE, fontweight="bold")
panel(ov, 0.51, 0.75, 0.47, 0.21, OR_BG, ORANGE)
fig.text(0.525, 0.94, "RESULTS, Fig. 6I LEGEND, DISCUSSION", color=ORANGE, fontweight="bold", va="top")
fig.text(0.525, 0.895, wrap('"without enhanced NO production"; "did not induce NO production"', 42), va="top")
fig.text(0.525, 0.77, "NO not induced: fits (b)", color=ORANGE, fontweight="bold")
ax = fig.add_axes([0.30, 0.25, 0.60, 0.44])
order = [CONDS[0], CONDS[2], CONDS[3], CONDS[1]] # top to bottom
names = {CONDS[0]: "Knockdown, uninfected", CONDS[2]: "Knockdown, infected",
CONDS[3]: "Non-targeted, infected", CONDS[1]: "Non-targeted, uninfected"}
yv = {k: 3 - i for i, k in enumerate(order)}
for k in order:
m, sd = D[k]["mean"], D[k]["sd"]; f = k[1] == "knockdown"
ax.errorbar(m, yv[k], xerr=sd, fmt="o", ms=9, color=BLUE, mfc=BLUE if f else BG, mew=2, elinewidth=2, capsize=4)
ax.text(m, yv[k] + 0.22, "%.1f" % m, color=BLUE, ha="center", va="bottom")
ax.set_yticks([yv[k] for k in order]); ax.set_yticklabels([names[k] for k in order])
ax.set_xlim(0, 8); ax.set_xticks([0, 2, 4, 6, 8]); ax.set_ylim(-0.5, 3.7)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"): ax.spines[s].set_visible(False)
ax.set_xlabel("Nitrites + nitrates (µM), digitised from Fig. 6I (mean ± SD)", color=BLUE)
def vbr(y0, y1, txt, col):
ax.plot([7.35, 7.5, 7.5, 7.35], [y0 - 0.05, y0 - 0.05, y1 + 0.05, y1 + 0.05], color=col, lw=1.5, clip_on=False)
ax.text(7.6, (y0 + y1) / 2, txt, va="center", color=col, clip_on=False)
vbr(3, 2, "(b) %+.1f" % (ki - ku), INK)
vbr(2 - 0.1, 1 + 0.1, "(a) ×%.2f" % ratio_kd_nt_inf, INK)
vbr(1 - 0.1, 0, "** paper", SEC)
fig.text(0.02, 0.015, wrap("(a) knockdown vs non-targeted, infected: higher, no significance marker. (b) knockdown, "
"infected vs uninfected: not induced. ** is the paper's only mark. Quotes verbatim (PMC text); "
"values digitised from the publisher's Fig. 6I image (±0.05 µM reading error).", 98),
fontsize=10, color=SEC)
save(fig, stem)
# ---- Layout E: stacked text panels left, dot plot right, connectors to the comparisons ----
def layout_E(stem):
fig = plt.figure(figsize=(W, 5.9)); ov = overlay(fig)
panel(ov, 0.02, 0.55, 0.40, 0.38, OR_BG, ORANGE)
fig.text(0.035, 0.91, "ABSTRACT", color=ORANGE, fontweight="bold", va="top")
fig.text(0.035, 0.865, wrap(Q_ABS, 36), va="top")
fig.text(0.035, 0.575, "NO increased", color=ORANGE, fontweight="bold")
panel(ov, 0.02, 0.12, 0.40, 0.40, OR_BG, ORANGE)
fig.text(0.035, 0.50, "BODY (4 locations)", color=ORANGE, fontweight="bold", va="top")
fig.text(0.035, 0.455, wrap('"without enhanced NO production" · "...not accompanied by greater parasite-induced NO production" · '
'"did not induce NO production" · "did not appear to be driven by enhanced production of NO"', 36), va="top")
fig.text(0.035, 0.145, "NO not enhanced / not induced", color=ORANGE, fontweight="bold")
ax = fig.add_axes([0.53, 0.12, 0.45, 0.78])
xpos = {CONDS[0]: 0.8, CONDS[1]: 1.2, CONDS[2]: 2.0, CONDS[3]: 2.4}
dot_axis(ax, xpos, ylabel=False)
ax.set_xlim(0.45, 2.75); ax.set_xticks([1.0, 2.2]); ax.set_xticklabels(["Uninfected", "Infected"])
ax.set_title("Fig. 6I digitised, µM (mean ± SD)", fontsize=10, color=BLUE, loc="left")
bracket(ax, 2.0, 2.4, 6.5, "×%.2f\nno mark" % ratio_kd_nt_inf)
ax.plot([0.8, 0.8, 2.0, 2.0], [4.85, 4.65, 4.65, 4.85], color=INK, lw=1.5)
ax.text(1.4, 4.5, "%+.1f µM\nnot induced" % (ki - ku), ha="center", va="top")
fig.add_artist(ConnectionPatch(xyA=(0.42, 0.74), coordsA="figure fraction", xyB=(1.98, 7.2), coordsB=ax.transData,
arrowstyle="->", color=ORANGE, lw=2, connectionstyle="arc3,rad=-0.15"))
fig.add_artist(ConnectionPatch(xyA=(0.42, 0.32), coordsA="figure fraction", xyB=(1.0, 3.95), coordsB=ax.transData,
arrowstyle="->", color=ORANGE, lw=2, connectionstyle="arc3,rad=0.15"))
fig.text(0.02, 0.975, "Two texts, two reference groups (Fig. 6I knockdown experiment)", fontsize=10, color=SEC, va="top")
fig.text(0.02, 0.012, wrap(SRC, 98), fontsize=10, color=SEC)
save(fig, stem)
LAYOUTS = [layout_A, layout_B, layout_C, layout_D, layout_E]
for i, f in enumerate(LAYOUTS, 1):
f(os.path.join(HERE, "candidates", "cand%d" % i))
FIG1 = layout_A # layout used in writeup.md
FIG1(os.path.join(HERE, "fig1"))
print("\nWrote fig1.svg/png (layout %s) and candidates/cand1-cand5 in %s"
% (FIG1.__name__[-1], HERE))
Results text assigns each group's cAMP mean to the other group, contradicting its own "higher" claim
PLoS Pathog (2013) · doi:10.1371/journal.ppat.1003319 · PMC3635970
Summary
The Results sentence for Figure 6B says that CD4+ T cells from HIV+ patients have "higher" intracellular cAMP than controls. The means it gives point the other way: 548.3 ± 9.1 fmol/million cells for HIV+ and 761.6 ± 29.4 for controls, which puts HIV+ 28% lower. Reading the six points per group off the published Figure 6B gives 547.8 ± 9.1 for the HIV− squares and 760.5 ± 29.4 for the HIV+ circles. The printed numbers are the figure's numbers with the two group labels swapped, so the figure and the paper's direction of effect (HIV+ higher) stand, and the sentence as printed contradicts itself. Separately, the Figure 6B legend gives 8 controls where the text gives 6, and the figure and the printed SEM both fit 6. The higher-cAMP result is one step of the mechanism the abstract and conclusions put forward, so the sentence should be corrected.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py.
Results, section "In vitro activated CD4+ cells from HIV+ patients were not able to demethylate il-2 CpG site 1 ...", first paragraph:
Ex vivo CD4+ T cells from HIV+ patients (n = 6) exhibit also higher levels of intra-cytoplasmic cAMP (mean 548.3±9.1 fmol/million cells) as compared to controls (n = 6; mean 761.6±29.4 fmol/million cells) (P = 0.002; Figure 6B).
Same paragraph, last sentence:
All together these results demonstrate a constitutive high expression of A2AR and cAMP resulting in a clear inhibitory effect on CpG demethylation accompanied by the lack of IL-2 production in HIV+ART- upon anti-CD3/CD28 activation.
Figure 6 title and legend, panel B:
High levels of A2AR and endogenous cAMP in CD4+ T cells from HIV infected patients and lack of IL-2 production.
(B) CD4+ T cells were purified from the blood of ART-naive HIV-infected patients (n = 6) and healthy controls (n = 8). Intra-cellular cAMP is measured using the cAMP direct enzyme immunoassay from GE healthcare Biosciences.
Abstract:
We show here that CD4+ T-cells from HIV-1 infected individuals express high levels of A2AR and intracellular cAMP.
Author Summary:
We found that T cells from HIV patients have a higher expression on A2AR as well as intra-cellular cAMP and a lesser capacity to produce IL-2 upon stimulation than healthy subjects.
Introduction, third paragraph:
We also found that CD4+ T cells from HIV infected patients express high level of cytoplasmic cAMP
Discussion, third paragraph:
In accordance with this, we show significant increased intracellular cAMP levels in CD4+ T cells from HIV+ patients compared to healthy controls.
Discussion, concluding paragraph:
This defect is caused by increased intracellular cAMP, due in part to increased hydrolysis of inflammatory ATP by both expanded Treg/CD39+ and increased A2AR expression levels.
Materials and Methods, "Statistical analysis":
The non-parametric Mann-Whitney U, Fisher's exact and paired T tests were used for statistical analyses (GraphPad Prism 5.0 statistical software).
What the paper's own numbers show
Method. First, I compared the two means printed in the Results sentence. Second, I read the individual points of Figure 6B off the publisher's image (PLOS "large" figure g006, 2776 × 2063 px). I calibrated the y-axis on its five tick marks (500 to 900; one pixel = 0.67 fmol/million cells; tick residuals ≤ 0.13). I located each marker's centroid after a morphological opening that removes the thin mean and error-bar lines, and I located the drawn mean bars and error-bar caps from their pixel rows. Third, I recomputed the mean, SD and SEM of each group of points, and the Mann-Whitney P values that the possible sample sizes allow. The pixel positions and every calculation are in recompute.py.txt, which prints every number below. One marker (a square behind the HIV− mean bar) is partly hidden; its centroid is taken from its visible rows. Read-off values are accurate to about ±1 fmol/million cells.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Direction of the difference | Results sentence | "higher" in HIV+ | 548.3 − 761.6 = −213.3; HIV+/control = 0.720 (HIV+ 28.0% lower) |
| HIV+ mean ± SEM | Results sentence | 548.3 ± 9.1 | Figure 6B HIV+ circles: 760.5 ± 29.4 (n = 6; drawn bar 761.9 ± 30.1) |
| Control mean ± SEM | Results sentence | 761.6 ± 29.4 | Figure 6B HIV− squares: 547.8 ± 9.1 (n = 6; drawn bar 548.0 ± 8.9) |
| Individual values, HIV− | Figure 6B | not stated | 574.8, 572.8, 553.0, 532.8, 528.8, 524.8 (SD 22.3) |
| Individual values, HIV+ | Figure 6B | not stated | 818.7, 808.9, 792.9, 768.9, 750.8, 622.8 (SD 72.0) |
| Control n | Results text vs Figure 6B legend | 6 vs 8 | 6 squares visible; SD/√6 of those six = 9.12, matching the printed ±9.1 |
| SEM if two hidden controls lay at the mean (n = 8) | hypothetical | — | 6.68 (would need SD 25.7 over 8 values to give 9.1) |
| P value, 6 vs 6, complete separation | Results (P = 0.002) | 0.002 | exact Mann-Whitney 2/924 = 0.00216; normal approximation 0.0039 |
| P value, 6 vs 8, complete separation | if the legend's n were right | — | exact 2/3003 = 0.00067; normal approximation 0.0019 |
| Exact P, 6 vs 8, U = 1 and U = 2 | if not completely separated | — | 0.00133 and 0.00266 |
| Size of the difference, corrected labels | from printed means and SEMs | — | +213.3 (HIV+ 38.9% higher); Welch t = 6.93, P = 4.6 × 10-4 |
Taken at its word, the Results sentence reports HIV+ cAMP below the control level, the opposite of what the same sentence says. Each printed mean and SEM matches, to within read-off accuracy, the mean and SEM of the other group's points in Figure 6B. The figure's HIV+ circles all lie above its HIV− squares, which is the direction the text states. Because the twelve points are completely separated, the exact Mann-Whitney test gives P = 0.00216, which rounds to the reported 0.002.
Figure
Figure 1. Left: the Results sentence and the means ± SEM it prints for each group (orange). Right: the mean ± SEM of the six points per group read off the published Figure 6B (blue), computed in recompute.py.txt. The connectors show that the value printed for HIV+ matches the figure's HIV− group and the value printed for controls matches the figure's HIV+ group. Bottom: the control sample size is 6 in the text and 8 in the legend. Six control squares are visible, and the printed ±9.1 equals the SD of those six divided by √6. The diagram reproduces no part of the paper's figure; it shows only summary numbers.
Figure 2. The four means ± SEM on one axis from zero, in fmol/million cells. Orange circles are printed in the Results text; blue squares are computed from the six points per group read off Figure 6B. The printed HIV+ value (548.3) coincides with the figure's control value (547.8), and the printed control value (761.6) with the figure's HIV+ value (760.5). The interval widths match in the same crossed way (±9.1 and ±29.4).
Why it matters
- The sentence contradicts itself on a mechanistic result. Higher intracellular cAMP in HIV+ CD4+ T cells is the step that links the CD39/A2AR pathway to the impaired IL-2 response in patients. The abstract, author summary, introduction, Figure 6 title, the Results summary sentence and the Discussion all rely on it. A reader who takes the means from the Results sentence, for example for a meta-analysis or to plan a study, gets the direction wrong and an HIV+ mean about 212 fmol/million cells too low.
- The finding itself is not affected. The published figure shows HIV+ above HIV−, and the printed numbers, once assigned to the right groups, give a 38.9% increase that is clearly significant (exact Mann-Whitney P = 0.00216 on the read-off values). The error is in how the sentence reports the data, not in the data or the conclusion.
- The control sample size is stated two ways. The legend's n = 8 and the text's n = 6 cannot both be right for the same measurement. This matters for anyone weighing or pooling the result, and for a reader checking the P value.
Explanations considered
- Different units, normalisation or transformation. Ruled out. Both groups are given in fmol/million cells from the same enzyme immunoassay on ex vivo CD4+ T cells, and the figure's axis has the same units. No rescaling turns 548.3 into the larger of the two values.
- The printed numbers are right and the direction words are wrong. Not supported. The published Figure 6B places the HIV+ circles (about 623 to 819) above all the HIV− squares (about 525 to 575). The direction "higher in HIV+" is repeated throughout the paper, and the Discussion cites earlier reports of higher cAMP in T cells from HIV+ patients.
- The text describes a different data set from Figure 6B. Not supported. The sentence cites Figure 6B, and the means and SEMs of the plotted points reproduce the printed values to within 1.1 fmol/million cells (means) and 0.03 (SEMs), but only with the group labels exchanged. It would be a remarkable coincidence for a different data set to produce both numbers.
- "±" is an SD rather than an SEM. Not supported. The SDs of the plotted points are 22.3 and 72.0. The printed 9.1 and 29.4 equal their SDs divided by √6, and the drawn error bars (±8.9 and ±30.1) are SEM bars.
- The legend's n = 8 is right and two control points are hidden by overlap. Unlikely, though it cannot be fully excluded from an image. Two extra values would have to keep the mean at about 548 and raise the SD from 22.3 to about 25.7 so that SD/√8 still equals 9.1. If they sat at the mean, the SEM would be 6.68. The P value does not settle the question alone. Under an exact test, which the Prism 5 documentation says it uses for small samples without ties, 6 vs 6 gives 0.00216 (reported 0.002), while 6 vs 8 can give only 0.00067, 0.00133 or 0.00266 near that value. Under the normal approximation, however, 6 vs 8 gives 0.0019, which would also round to 0.002. The next sentence of the Results gives "healthy controls (n = 8)" for the Figure 6C IL-2 experiment, which may be where the legend's 8 came from; that is an inference only.
- A conversion or typesetting artifact in PMC. Ruled out. The publisher's article page gives the same sentence and the same legend, word for word.
- A correction already issued. None found (see Status).
Status
- Version of record. The PLOS Pathogens article page (journals.plos.org, the doi.org target), retrieved on 2026-10-24, gives the Results sentence and the Figure 6B legend exactly as quoted above. The Figure 6B values were read from the publisher's own figure image (g006, large size), not from the PMC rendering.
- Supplements. The publisher lists four supporting figures: S1 (ATPase activity of Treg/CD39+ vs Treg/CD39−), S2 (CD73 expression), S3 (HPLC of AMP conversion to adenosine) and S4 (anti-CD39 effect on a cell line). None covers the cAMP measurement in patients, and no source data are posted. The question of control n = 6 vs 8 therefore cannot be settled beyond what the figure shows.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by fields and an empty relation field. The PLOS article page shows no correction notice. The PubPeer API returned no record (HTTP 404) for the DOI, and PubPeer's search page could not be queried automatically. A web search for the title together with "PubPeer", "correction" or "erratum" found no thread or notice. - Sources. PMC article PMC3635970; PLOS Pathogens article page and Figure 6 image (https://doi.org/10.1371/journal.ppat.1003319); Crossref; Europe PMC; PubPeer; GraphPad knowledgebase article #416 on Prism 5's Mann-Whitney P values.
Suggested correction
In the Results, replace
Ex vivo CD4+ T cells from HIV+ patients (n = 6) exhibit also higher levels of intra-cytoplasmic cAMP (mean 548.3±9.1 fmol/million cells) as compared to controls (n = 6; mean 761.6±29.4 fmol/million cells) (P = 0.002; Figure 6B).
with
Ex vivo CD4+ T cells from HIV+ patients (n = 6) exhibit also higher levels of intra-cytoplasmic cAMP (mean 761.6±29.4 fmol/million cells) as compared to controls (n = 6; mean 548.3±9.1 fmol/million cells) (P = 0.002; Figure 6B).
The authors should also confirm the number of controls in the cAMP assay from their records and make the Figure 6B legend agree with the text. If the six plotted controls are the full set, the legend should read "healthy controls (n = 6)". It would help to state that the ± values are SEMs.
Recomputation script (python recompute.py.txt)
"""Recompute the Figure 6B cAMP comparison of PMC3635970
(PLoS Pathog 2013, doi:10.1371/journal.ppat.1003319,
"Regulatory T Cells Negatively Affect IL-2 Production of Effector T Cells through
CD39/Adenosine Pathway in HIV Infection").
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
python recompute.py.txt g006.png (optional: re-measure the marker positions
from the publisher's Figure 6 image)
Prints every recomputed number used in writeup.md and writes fig1.svg/png and
fig2.svg/png next to this file. The figures draw no part of the paper's own image;
they show only summary numbers.
Inputs copied from the paper as archived in PMC:
Results: "Ex vivo CD4+ T cells from HIV+ patients (n = 6) exhibit also higher levels of
intra-cytoplasmic cAMP (mean 548.3±9.1 fmol/million cells) as compared to controls
(n = 6; mean 761.6±29.4 fmol/million cells) (P = 0.002; Figure 6B)".
Figure 6B legend: "ART-naive HIV-infected patients (n = 6) and healthy controls (n = 8)".
Methods: Mann-Whitney U test, GraphPad Prism 5.0.
Inputs read off Figure 6B of the publisher's image
https://journals.plos.org/plospathogens/article/figure/image?size=large&id=10.1371/journal.ppat.1003319.g006
(2776 x 2063 px, grayscale). Pixel rows below are in that image's coordinates.
y-axis ticks (centre row of each 5-6 px tick mark, at x = 1192-1207):
900 -> 219.0, 800 -> 367.5, 700 -> 516.5, 600 -> 665.5, 500 -> 814.5
Marker centroids: dark pixels (< 110) in x 1225-1720, y 300-808, after a binary
opening with an 11 x 11 square, which removes the thin mean and error-bar lines.
The third HIV- square sits behind the mean bar; its centroid is the middle of its
visible rows 723-748.
Mean bars and error-bar caps: rows where a long horizontal dark run crosses the
group's x position (mean bar 5-6 rows thick, caps 8-9 rows thick).
"""
import os
import sys
from math import comb
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs: text
TEXT = {"HIV+": dict(n=6, mean=548.3, pm=9.1), "control": dict(n=6, mean=761.6, pm=29.4)}
LEGEND_N_CONTROL = 8
P_REPORTED = 0.002
# ------------------------------------------------------------------ inputs: Figure 6B pixels
TICK_ROWS = np.array([219.0, 367.5, 516.5, 665.5, 814.5])
TICK_VALS = np.array([900, 800, 700, 600, 500])
HIVNEG_SQUARES_ROWS = [703.0, 706.0, 735.5, 765.5, 771.5, 777.5] # x = 1365, 1304, 1334, 1304, 1365, 1334
HIVPOS_CIRCLES_ROWS = [339.9, 354.5, 378.3, 414.0, 441.0, 631.5] # x = 1583, 1614, 1583, 1552, 1583, 1583
MEANBAR_ROWS = {"HIV-": (741, 745), "HIV+": (422, 427)}
CAP_ROWS = {"HIV-": ((726, 734), (753, 760)), "HIV+": ((377, 385), (467, 474))}
def remeasure(path):
"""Optional: re-derive the marker centroids from the publisher's image."""
from PIL import Image
from scipy import ndimage as nd
a = np.array(Image.open(path).convert("L")).astype(int)
x0, y0, x1, y1 = 1225, 300, 1720, 808
m = nd.binary_opening(a[y0:y1, x0:x1] < 110, structure=np.ones((11, 11)))
lab, n = nd.label(m)
out = []
for i in range(1, n + 1):
cy, cx = nd.center_of_mass(m, lab, i)
out.append((round(cx + x0), round(cy + y0, 1)))
print(f" re-measured {n} markers (x, y):", sorted(out))
print(" (x < 1450 are HIV- squares, x > 1450 are HIV+ circles)")
slope, intercept = np.polyfit(TICK_ROWS, TICK_VALS, 1)
def to_val(y):
return intercept + slope * np.asarray(y, dtype=float)
print("=== 0. Axis calibration of Figure 6B ===")
print(f" {abs(1 / slope):.4f} px per fmol/million cells; one pixel = {abs(slope):.3f} units; "
f"max tick residual {np.max(np.abs(to_val(TICK_ROWS) - TICK_VALS)):.2f} units")
if len(sys.argv) > 1:
remeasure(sys.argv[1])
# ------------------------------------------------------------------ 1. the text's own numbers
print("\n=== 1. The Results sentence, taken at its word ===")
h, c = TEXT["HIV+"]["mean"], TEXT["control"]["mean"]
print(f" HIV+ mean {h} vs control mean {c}: HIV+ higher? {h > c}")
print(f" difference HIV+ - control = {h - c:.1f} fmol/million cells")
print(f" ratio HIV+/control = {h / c:.3f} -> HIV+ {100 * (h - c) / c:.1f}% relative to controls")
print(f" if the labels are swapped: HIV+ {c} vs control {h}, difference +{c - h:.1f}, "
f"HIV+ {100 * (c - h) / h:.1f}% higher")
# ------------------------------------------------------------------ 2. Figure 6B read off the image
print("\n=== 2. Figure 6B, values read off the published image ===")
neg = to_val(HIVNEG_SQUARES_ROWS)
pos = to_val(HIVPOS_CIRCLES_ROWS)
FIG = {}
for name, x in (("HIV- (squares)", neg), ("HIV+ (circles)", pos)):
sd = x.std(ddof=1)
FIG[name] = dict(n=len(x), mean=x.mean(), sd=sd, sem=sd / np.sqrt(len(x)))
print(f" {name}: n visible = {len(x)}, values = {np.round(x, 1).tolist()}")
print(f" mean {x.mean():.1f}, SD {sd:.1f}, SEM (SD/sqrt(6)) {sd / np.sqrt(len(x)):.1f}")
BAR = {}
for g in ("HIV-", "HIV+"):
r0, r1 = MEANBAR_ROWS[g]
(u0, u1), (l0, l1) = CAP_ROWS[g]
mean_bar = float(to_val((r0 + r1) / 2))
half = float((to_val((u0 + u1) / 2) - to_val((l0 + l1) / 2)) / 2)
BAR[g] = dict(mean=mean_bar, half=half)
print(f" drawn {g} mean bar = {mean_bar:.1f}; drawn error bar = +/- {half:.1f}")
print("\n Match of the text's numbers with the figure groups:")
print(f" text 'HIV+' 548.3 +/- 9.1 vs figure HIV- {FIG['HIV- (squares)']['mean']:.1f} +/- "
f"{FIG['HIV- (squares)']['sem']:.1f} (bar {BAR['HIV-']['mean']:.1f} +/- {BAR['HIV-']['half']:.1f})")
print(f" text 'control' 761.6 +/- 29.4 vs figure HIV+ {FIG['HIV+ (circles)']['mean']:.1f} +/- "
f"{FIG['HIV+ (circles)']['sem']:.1f} (bar {BAR['HIV+']['mean']:.1f} +/- {BAR['HIV+']['half']:.1f})")
print(f" text 'HIV+' 548.3 vs figure HIV+ {FIG['HIV+ (circles)']['mean']:.1f}: off by "
f"{548.3 - FIG['HIV+ (circles)']['mean']:.1f}")
# ------------------------------------------------------------------ 3. control n: 6 or 8?
print("\n=== 3. Control sample size: text n = 6, legend n = 8 ===")
ss = float(((neg - neg.mean()) ** 2).sum())
print(f" SEM of the 6 visible squares with n = 6: {np.sqrt(ss / 5) / np.sqrt(6):.2f} (text: 9.1)")
print(f" same 6 values plus 2 hidden values exactly at their mean, n = 8: "
f"SEM {np.sqrt(ss / 7) / np.sqrt(8):.2f}")
sd_needed = 9.1 * np.sqrt(8)
print(f" for SEM = 9.1 with n = 8 the SD would have to be {sd_needed:.1f} "
f"(6 visible squares: SD {np.sqrt(ss / 5):.1f})")
mw = stats.mannwhitneyu(pos, neg, alternative="two-sided", method="exact")
print(f" exact Mann-Whitney on the 12 read-off values: U = {mw.statistic:.0f}, P = {mw.pvalue:.5f}")
P_ALL = {}
for n1, n2 in ((6, 6), (6, 8)):
exact = 2 / comb(n1 + n2, n1)
mu, sd = n1 * n2 / 2, np.sqrt(n1 * n2 * (n1 + n2 + 1) / 12)
z = (n1 * n2 - mu) / sd
p_norm = 2 * stats.norm.sf(z)
p_cc = 2 * stats.norm.sf((n1 * n2 - mu - 0.5) / sd)
P_ALL[(n1, n2)] = dict(exact=exact, norm=p_norm, cc=p_cc)
print(f" {n1} vs {n2}, complete separation: exact P = 2/C({n1 + n2},{n1}) = 2/{comb(n1 + n2, n1)} = "
f"{exact:.5f}; normal approx. z = {z:.3f}, P = {p_norm:.5f} (with continuity corr. {p_cc:.5f})")
# smallest exact P values for 6 vs 8 with U = 1 and U = 2 (from the null distribution of U)
from itertools import combinations
counts = np.zeros(6 * 8 + 1, dtype=int)
for ranks in combinations(range(14), 6):
u = sum(r - i for i, r in enumerate(ranks)) # number of (group-2 < group-1) pairs
counts[u] += 1
cdf = np.cumsum(counts) / counts.sum()
print(" 6 vs 8, exact two-sided P for U = 0, 1, 2: "
+ ", ".join(f"{2 * cdf[k]:.5f}" for k in range(3)))
# ------------------------------------------------------------------ 4. size of the difference
print("\n=== 4. Size of the difference under the corrected labelling ===")
for nc in (6, 8):
t, p = stats.ttest_ind_from_stats(761.6, 29.4 * np.sqrt(6), 6, 548.3, 9.1 * np.sqrt(nc), nc,
equal_var=False)
print(f" Welch t from the printed means and SEMs, HIV+ n = 6 vs control n = {nc}: "
f"t = {t:.2f}, P = {p:.1e}")
# ------------------------------------------------------------------ figures
INK, INK2, BG, GRID = "#0b0b0b", "#52514e", "#fcfcfb", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, stem):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{stem}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- Figure 1: text-vs-figure diagram
fig = plt.figure(figsize=(7.2, 4.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 100); ax.axis("off")
ax.set_facecolor(BG)
def panel(x, y, w, hgt, title):
ax.add_patch(FancyBboxPatch((x, y), w, hgt, boxstyle="round,pad=0,rounding_size=1.5",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(x + 2, y + hgt - 4.2, title, fontsize=11, weight="bold", va="center")
panel(2, 37, 41, 55, "Results text (Figure 6B sentence)")
panel(57, 37, 41, 55, "Figure 6B (points read off)")
ax.text(4, 80, "“HIV+ patients (n = 6) exhibit\nalso higher levels of\nintra-cytoplasmic cAMP”",
fontsize=10, color=INK2, va="top", style="italic")
ax.text(4, 61.5, "HIV+:", fontsize=10.5, va="center")
ax.text(16, 61.5, "548.3 ± 9.1", fontsize=11.5, color=ORANGE, weight="bold", va="center")
ax.text(4, 46.5, "controls:", fontsize=10.5, va="center")
ax.text(16, 46.5, "761.6 ± 29.4", fontsize=11.5, color=ORANGE, weight="bold", va="center")
ax.text(4, 40.0, "fmol/million cells; P = 0.002", fontsize=10, color=INK2, va="center")
fn, fp = FIG["HIV- (squares)"], FIG["HIV+ (circles)"]
ax.text(59, 80, "HIV+ circles lie above\nHIV− squares (** in figure)", fontsize=10,
color=INK2, va="top", style="italic")
ax.text(59, 61.5, "HIV+:", fontsize=10.5, va="center")
ax.text(72, 61.5, f"{fp['mean']:.1f} ± {fp['sem']:.1f}", fontsize=11.5, color=BLUE,
weight="bold", va="center")
ax.text(59, 46.5, "HIV−:", fontsize=10.5, va="center")
ax.text(72, 46.5, f"{fn['mean']:.1f} ± {fn['sem']:.1f}", fontsize=11.5, color=BLUE,
weight="bold", va="center")
ax.text(59, 40.0, "mean ± SEM of 6 points per group", fontsize=10, color=INK2, va="center")
# crossing connectors: each printed value matches the OTHER group
kw = dict(arrowstyle="-|>", mutation_scale=13, lw=2, color=INK2, shrinkA=2, shrinkB=2)
ax.add_patch(FancyArrowPatch((31.5, 61.5), (70.5, 46.5), connectionstyle="arc3,rad=0", **kw))
ax.add_patch(FancyArrowPatch((32.5, 46.5), (70.5, 61.5), connectionstyle="arc3,rad=0", **kw))
ax.text(50, 70.5, "same numbers,\ngroup labels swapped", ha="center", va="center", fontsize=10,
weight="bold", bbox=dict(fc=BG, ec="none", pad=1.5))
# n conflict strip
panel(2, 9, 96, 22, "Control sample size")
ax.text(4, 19.5, "Text: controls", fontsize=10.5, va="center")
ax.text(20.5, 19.5, "n = 6", fontsize=11.5, color=ORANGE, weight="bold", va="center")
ax.text(29, 19.5, "Legend: controls", fontsize=10.5, va="center")
ax.text(47.5, 19.5, "n = 8", fontsize=11.5, color=ORANGE, weight="bold", va="center")
ax.text(58, 19.5, "Figure: 6 squares;", fontsize=10.5, va="center")
ax.text(77.5, 19.5, "SD/√6 = " + f"{np.sqrt(ss / 5) / np.sqrt(6):.1f}", fontsize=11.5,
color=BLUE, weight="bold", va="center")
ax.text(4, 13.0, "The printed ±9.1 equals the SEM of the six visible squares computed with n = 6.",
fontsize=10, color=INK2, va="center")
ax.text(2, 3.5, "Orange: printed in the paper. Blue: computed from Figure 6B in recompute.py.txt.",
fontsize=10, color=INK2, va="center")
save(fig, "fig1")
# ---- Figure 2: dot-and-interval, one axis
rows = [("HIV+ — Results text", 548.3, 9.1, ORANGE, "o"),
("HIV+ — Figure 6B", fp["mean"], fp["sem"], BLUE, "s"),
("Controls (HIV−) — Results text", 761.6, 29.4, ORANGE, "o"),
("Controls (HIV−) — Figure 6B", fn["mean"], fn["sem"], BLUE, "s")]
fig, ax = plt.subplots(figsize=(7.2, 3.3), facecolor=BG)
ax.set_facecolor(BG)
ys = [3.6, 3.0, 1.6, 1.0]
for (lab, m, e, col, mk), y in zip(rows, ys):
ax.errorbar(m, y, xerr=e, fmt=mk, ms=8, color=col, ecolor=col, elinewidth=2, capsize=4,
capthick=2, zorder=3)
ax.text(m + e + 14, y, f"{m:.1f} ± {e:.1f}", va="center", fontsize=10, color=INK)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlim(0, 1000); ax.set_xticks(range(0, 901, 100)); ax.set_ylim(0.4, 4.2)
ax.set_xlabel("Intracellular cAMP, fmol/million cells (mean ± SEM)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#bdbcb7")
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D # noqa: E402
fig.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=2, ms=8, label="printed in Results text"),
Line2D([], [], marker="s", color=BLUE, lw=2, ms=8,
label="computed from Figure 6B (6 points per group)")],
loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=2, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0, 1, 0.9))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
The abstract reverses the direction of the paper's hypoxia result
PLoS Pathog (2019) · doi:10.1371/journal.ppat.1007745 · PMC6513112
Summary
The paper reports that STAT2-dependent type I interferon signaling oxygenates the infected gut, so that removing STAT2 restores hypoxia. Its abstract, however, states that the gut tissue of Stat2−/− mice showed "decreased hypoxia," while the Results, the author summary, the Discussion and the paper's own mechanistic thesis all state the opposite: Stat2−/− tissue was more hypoxic (higher pimonidazole staining, lower oxygenation) than wild-type. Fig 6F is a qualitative immunohistochemistry image with no reported staining intensities or statistics, so there is no number to recompute; what can be checked is that one sentence assigns the reverse direction to the same comparison that four other passages describe. Because the abstract is the most-read and most-indexed part of a paper, a reader who takes this result from the abstract alone records the direction backwards, even though the body and the main conclusions are correct and unchanged.
What the paper reports
Here, we show that STAT2 dependent type I IFN signaling contributes to the inflammatory environment by disrupting hypoxia enabling the pathogenic S. Typhimurium to outgrow the microbiota.
— Abstract (opening; the mechanism: type I IFN removes hypoxia in wild-type mice)
Although inflammatory pathology was similar between wild-type and Stat2−/− mice, we observed decreased hypoxia in the gut tissue of Stat2−/− mice.
— Abstract (the sentence in question: Stat2−/− = less hypoxic)
We determined that the intestinal environment in Stat2−/− mice was more hypoxic than in wild-type mice as shown by higher levels of pimonidazole staining (red) (Fig 6F). Both wild-type and Stat2−/− mice showed comparable hypoxia staining without infection (S3B Fig).
— Results, "Neutrophil-associated oxygenation contributes to low bacterial burdens" (Stat2−/− = more hypoxic)
In the absence of STAT2-dependent type I IFN, production of neutrophil reactive oxygen species was impaired, and epithelial metabolism returned to homeostatic hypoxia.
— Author summary (loss of STAT2 restores hypoxia, i.e. more hypoxic)
The competition experiments between wild-type S. Typhimurium and cyxA mutant as well as hypoxia staining confirmed that oxygenation of the gut lumen of Stat2−/− mice was lower compared to that of wild-type mice.
— Discussion, paragraph 1 (lower oxygenation = more hypoxic)
It was previously reported that hypoxia decreases in the intestine during S. Typhimurium infection [45].
— Results (the baseline biology: infection removes hypoxia; retaining hypoxia is the Stat2−/− phenotype)
One hour prior to euthanasia, wild-type and Stat2−/− infected mice were injected with 100 mg/kg of PMDZ … [PMDZ is] a marker of hypoxia.
— Materials and methods, "Hypoxia staining"; Results (pimonidazole is a hypoxia probe: higher signal = more hypoxia)
What the paper's own numbers show
Method. This is a text-versus-text contradiction, not an arithmetic slip. Fig 6F reports pimonidazole immunohistochemistry images; the paper gives no staining intensities, group means or statistics for the wild-type versus Stat2−/− comparison, so there is no value to re-add. What is objectively checkable is (i) the direction each section assigns to the same headline comparison — gut-tissue hypoxia in Stat2−/− versus wild-type, infected animals, Fig 6F — and whether those directions agree; (ii) that the probe runs in the direction the argument assumes (pimonidazole adducts form only at low O₂, so higher staining means more hypoxia, as the Methods state); and (iii) that the pimonidazole dose, stated two ways, is internally consistent and so is not the source of the discrepancy. Direction is encoded as +1 (Stat2−/− more hypoxic) or −1 (Stat2−/− less hypoxic). Every value below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own statements |
|---|---|---|---|
| Hypoxia direction, Stat2−/− vs WT | Abstract | "decreased hypoxia in Stat2−/−" | direction −1 (less hypoxic) |
| Hypoxia direction, Stat2−/− vs WT | Results; Fig 6F | "more hypoxic … higher pimonidazole staining" | direction +1 (more hypoxic) |
| Hypoxia direction, Stat2−/− vs WT | Author summary | "returned to homeostatic hypoxia" | direction +1 (more hypoxic) |
| Hypoxia direction, Stat2−/− vs WT | Discussion, para 1 | oxygenation "lower" in Stat2−/− | direction +1 (more hypoxic) |
| Hypoxia direction, Stat2−/− vs WT | Title / thesis | IFN oxygenates gut; absence restores hypoxia | direction +1 (more hypoxic) |
| Four non-abstract sections agree with each other | cross-section | — | True (shared value +1) |
| Abstract matches the rest of the paper | cross-section | — | False (abstract −1 vs others +1) |
| Pimonidazole dose | Results | 2.0 mg / 20 g body weight | 100.0 mg/kg |
| Pimonidazole dose | Methods | 100 mg/kg | 100.0 mg/kg — consistent |
| Uninfected baseline (WT vs Stat2−/−) | Results; S3B Fig | "comparable hypoxia staining" | no baseline difference that could flip the infected sign |
Figure
Figure 1. The left panel holds the abstract's sentence and the direction it states (Stat2−/− less hypoxic than wild-type); the right panel holds the Results sentence describing the same Fig 6F pimonidazole comparison and its direction (Stat2−/− more hypoxic), which the Discussion, author summary and thesis share. The two directions are opposite, as the labelled connector marks. Orange marks the direction as printed in the abstract and blue the direction printed everywhere else, so the contrast never rests on colour alone. Because pimonidazole marks hypoxia (higher staining = more hypoxic) and the uninfected controls in S3B Fig were comparable, no change of probe direction or baseline reconciles the two sentences.
Why it matters
- Direction, not magnitude. The error reverses the stated sign of a key mechanistic result. There is no magnitude or significance to get wrong, because Fig 6F is qualitative; the body states the correct direction in words three separate times.
- It sits in the most-read section. Readers, databases and evidence syntheses lean on the abstract. Anyone who tabulates a genotype-versus-hypoxia effect from the abstract alone — for a microbiome or enteric-infection review, say — records Stat2−/− as less hypoxic, the opposite of the finding. Nothing in the abstract signals the conflict.
- The abstract contradicts itself. The same abstract opens by saying type I IFN acts "by disrupting hypoxia" in wild-type mice, which implies that Stat2−/− mice retain hypoxia. The later "decreased hypoxia in … Stat2−/−" sentence is at odds with its own framing as well as with the Results.
- What it does not affect. The paper's central model — type I IFN/STAT2 signaling oxygenates the gut lumen, letting S. Typhimurium respire via cytochrome bd-II and expand over the microbiota, so Stat2−/− mice keep homeostatic hypoxia and resist expansion — is stated correctly throughout the body and is unchanged. Correcting the abstract restores internal consistency rather than overturning any conclusion. No clinical or policy decision depends on it.
Explanations considered
- Different experiments, time points or models. Ruled out. The abstract, Results, Discussion and author summary all refer to the same infected wild-type-versus-Stat2−/− comparison shown in Fig 6F (streptomycin-pretreated mice, 1:1 wild-type:cyxA infection, colons collected 4 days post infection).
- Different measurement or units. Ruled out. Both the abstract ("hypoxia in the gut tissue") and the Results ("intestinal environment … pimonidazole staining") describe tissue hypoxia measured with the same pimonidazole probe. There are no numeric values, so no unit, scaling or adjusted-versus-unadjusted difference is in play. The dose is stated consistently (2.0 mg/20 g = 100 mg/kg).
- The probe runs the other way. Ruled out. Pimonidazole (Hypoxyprobe) forms adducts only under low oxygen; the Methods call it "a marker of hypoxia." Higher staining means more hypoxia, so the Results' "higher levels of pimonidazole staining" unambiguously means Stat2−/− is more hypoxic.
- A baseline difference flips the infected sign. Ruled out. S3B Fig shows comparable hypoxia staining in uninfected wild-type and Stat2−/− mice, so there is no baseline asymmetry that could invert the infected comparison.
- The table/figure is the typo and the abstract is right. Not supported. There is no table of values; the direction is stated in words in four places (Results, author summary, Discussion, and the thesis of title and abstract opening), all agreeing that Stat2−/− is more hypoxic. The lone outlier is the one abstract word "decreased."
- A conversion or archiving artefact. Ruled out. The publisher's version of record, the Europe PMC full-text XML and the PMC archive used here all carry the same two sentences with the same directions.
- Already corrected. No correction, erratum or other notice was found (see Status).
Status
As of 2026-10-24:
- Version of record. The DOI resolves to the PLoS Pathogens version of record (the typeset "Version 2"). Its abstract reads "we observed decreased hypoxia in the gut tissue of Stat2−/− mice," while its Results read "the intestinal environment in Stat2−/− mice was more hypoxic than in wild-type mice." The Europe PMC full-text XML and the PMC archive (
src/show.py PMC6513112) carry the identical sentences. The error is in the published paper, not introduced downstream. - Supplementary material. No numeric hypoxia data exist to recover: Fig 6F is qualitative immunohistochemistry, and the only related supplement is S3B Fig (comparable staining in uninfected wild-type and Stat2−/− mice). The Fig 6F legend and the Methods "Hypoxia staining" section were read in full; neither bears on the abstract-versus-Results wording.
- Corrections and notices.
src/known.py 10.1371/journal.ppat.1007745returns only a self-referential publisher "new version" (2019-05-13) and no PMC or Europe PMC correction/retraction notices. The Crossref record carries no is-corrected-by/updated-by relation beyond that self-reference. The PLoS article page shows a version selector but no correction or erratum banner. A web search for a correction, erratum or PubPeer thread on the DOI and title found none. - Sources. The article as archived in PMC (PMC6513112): abstract, Results, author summary, Discussion, Methods "Hypoxia staining," Fig 6 legend;
src/known.pyfor the DOI; the PLoS version-of-record page; Crossref, PMC and Europe PMC notice records; a web/PubPeer search.
Suggested correction
In the abstract, change the one sentence so its direction matches the Results, Discussion, author summary and the abstract's own framing, for example:
Although inflammatory pathology was similar between wild-type and Stat2−/− mice, we observed increased hypoxia in the gut tissue of Stat2−/− mice.
No change to the figures, Methods or conclusions appears to be needed; the correction removes an internal contradiction rather than altering any finding.
Recomputation script (python recompute.py.txt)
"""Directional-consistency check for the hypoxia claim in PMC6513112
(PLoS Pathog 2019, doi:10.1371/journal.ppat.1007745).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed value used in writeup.md and writes
fig1.svg / fig1.png next to this file.
The finding is a text-vs-text contradiction, not a numerical error: Fig 6F is
qualitative pimonidazole immunohistochemistry and the paper reports no staining
intensities, group means or statistics for it. There is therefore no value to
re-add. What can be checked objectively is the DIRECTION that each section of the
paper assigns to the same headline comparison (Stat2-/- vs wild-type intestine),
and whether those directions agree.
Convention used below:
+1 => Stat2-/- intestine MORE hypoxic than wild-type (less oxygenated)
-1 => Stat2-/- intestine LESS hypoxic than wild-type (more oxygenated)
All strings are copied verbatim from the article as archived in PubMed Central
(src/show.py PMC6513112) and confirmed against the PLoS version of record.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
# section -> (verbatim excerpt, direction code, location label)
STATEMENTS = {
"Abstract": (
"we observed decreased hypoxia in the gut tissue of Stat2-/- mice",
-1, "Abstract"),
"Results": (
"the intestinal environment in Stat2-/- mice was more hypoxic than in "
"wild-type mice as shown by higher levels of pimonidazole staining (Fig 6F)",
+1, "Results > Neutrophil-associated oxygenation; Fig 6F"),
"Author summary": (
"in the absence of STAT2-dependent type I IFN ... epithelial metabolism "
"returned to homeostatic hypoxia",
+1, "Author summary"),
"Discussion": (
"oxygenation of the gut lumen of Stat2-/- mice was lower compared to that "
"of wild-type mice",
+1, "Discussion, paragraph 1"),
"Thesis/Abstract framing": (
"STAT2 dependent type I IFN signaling ... disrupt[s] hypoxia (oxygenates the "
"wild-type gut); absence of STAT2 therefore restores hypoxia",
+1, "Title / Abstract opening / Discussion"),
}
# Methods: pimonidazole (PMDZ) is a hypoxia probe; adducts form under low O2, so
# HIGHER staining = MORE hypoxia. Dose is stated two ways; check they agree.
DOSE_RESULTS_MG = 2.0 # "2.0 mg/20 g body weight"
DOSE_RESULTS_BODY_G = 20.0
DOSE_METHODS_MG_PER_KG = 100.0 # "100 mg/kg of PMDZ"
def main():
print("== 1. Direction each section assigns to Stat2-/- vs wild-type hypoxia")
print(" (+1 = Stat2-/- MORE hypoxic; -1 = Stat2-/- LESS hypoxic)\n")
for sec, (txt, d, loc) in STATEMENTS.items():
print(f" {sec:24s} dir = {d:+d} [{loc}]")
print(f" \"{txt}\"")
dirs = {s: d for s, (t, d, l) in STATEMENTS.items()}
abstract = dirs["Abstract"]
others = {s: d for s, d in dirs.items() if s != "Abstract"}
others_set = set(others.values())
print("\n== 2. Internal agreement")
print(f" Abstract direction: {abstract:+d}")
print(f" All other sections agree with each other: {len(others_set) == 1} "
f"(shared value {others_set.pop() if len(others_set) == 1 else others_set:+})"
if len(others_set) == 1 else
f" All other sections agree with each other: {len(others_set) == 1}")
body_dir = list(set(others.values()))[0]
print(f" Body/abstract-framing direction: {body_dir:+d}")
print(f" Abstract matches the rest of the paper: {abstract == body_dir}")
print(f" ABSTRACT CONTRADICTS RESULTS: {abstract != dirs['Results']}")
print("\n== 3. Probe direction (so 'higher staining' is unambiguously 'more hypoxic')")
print(" Pimonidazole (Hypoxyprobe) forms protein adducts only at low O2 tension;")
print(" Methods call it 'a marker of hypoxia'. Higher pimonidazole signal =>")
print(" more hypoxia. Results report HIGHER staining in Stat2-/- => Stat2-/- more hypoxic.")
print("\n== 4. Pimonidazole dose stated two ways -- are they consistent?")
d1 = DOSE_RESULTS_MG / (DOSE_RESULTS_BODY_G / 1000.0)
print(f" Results: {DOSE_RESULTS_MG} mg / {DOSE_RESULTS_BODY_G} g = {d1:.1f} mg/kg")
print(f" Methods: {DOSE_METHODS_MG_PER_KG:.1f} mg/kg")
print(f" Consistent: {abs(d1 - DOSE_METHODS_MG_PER_KG) < 1e-9} "
"(dosing is internally consistent; not the source of the discrepancy)")
print("\n== 5. Innocent explanations that could let both sentences be correct")
print(" Same experiment (all refer to Fig 6F pimonidazole): True")
print(" Same comparison axis (Stat2-/- vs wild-type, infected): True")
print(" Uninfected baseline (S3B Fig) comparable WT vs Stat2-/-: True")
print(" -> no baseline difference that could flip the infected sign")
print(" Conclusion: no innocent reading reconciles the two sentences;")
print(" the Abstract word 'decreased' is the reversed outlier.")
fig = make_fig()
fig.savefig(os.path.join(HERE, "fig1.svg"))
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200)
plt.close(fig)
print("\nFigure written: fig1.svg, fig1.png")
# ---------------------------------------------------------------- figure
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, PANEL = "#eb6834", "#2a78d6", "#e4e3df", "#f3f2ef"
plt.rcParams.update({
"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": TXT, "ytick.color": TXT, "axes.edgecolor": SEC,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "font.family": "DejaVu Sans",
})
W = 7.2
def box(ax, x, y, w, h, edge=SEC, face=PANEL, lw=1.1):
ax.add_patch(FancyBboxPatch((x, y), w, h,
boxstyle="round,pad=0,rounding_size=0.012",
fc=face, ec=edge, lw=lw, transform=ax.transAxes, clip_on=False))
def put(ax, x, y, items, dy=0.050):
for txt, col, wt, sz in items:
ax.text(x, y, txt, color=col, fontweight=wt, fontsize=sz,
va="top", ha="left", transform=ax.transAxes)
y -= dy * sz / 10.5
return y
def make_fig():
"""Two text panels holding the conflicting excerpts, joined by a labelled
connector that names the conflict; a direction key keeps colour from carrying
meaning on its own."""
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.035, 0.965,
"Abstract and results give opposite directions for one comparison",
fontsize=12, fontweight="bold", va="top")
ax.text(0.035, 0.905,
"Gut-tissue hypoxia in Stat2−/− vs wild-type mice (pimonidazole staining, Fig 6F)",
fontsize=10, color=SEC, va="top")
# left panel: abstract
box(ax, 0.035, 0.30, 0.40, 0.555)
put(ax, 0.055, 0.825, [
("ABSTRACT", SEC, "bold", 9.5),
("“…we observed", TXT, "normal", 10.5),
("decreased hypoxia in the", TXT, "bold", 10.5),
("gut tissue of Stat2−/− mice.”", TXT, "bold", 10.5),
], dy=0.052)
put(ax, 0.055, 0.49, [
("Direction stated:", SEC, "normal", 9.5),
("Stat2−/− LESS hypoxic", ORANGE, "bold", 11),
("than wild-type", ORANGE, "bold", 11),
], dy=0.052)
# right panel: results + supporting
box(ax, 0.565, 0.30, 0.40, 0.555)
put(ax, 0.585, 0.825, [
("RESULTS (Fig 6F)", SEC, "bold", 9.5),
("“…the intestinal environment", TXT, "normal", 10.5),
("in Stat2−/− mice was more", TXT, "normal", 10.5),
("hypoxic than in wild-type", TXT, "bold", 10.5),
("mice… higher levels of", TXT, "bold", 10.5),
("pimonidazole staining.”", TXT, "bold", 10.5),
], dy=0.0475)
put(ax, 0.585, 0.455, [
("Direction stated:", SEC, "normal", 9.5),
("Stat2−/− MORE hypoxic", BLUE, "bold", 11),
("(also: Discussion, author", SEC, "normal", 9),
("summary, thesis)", SEC, "normal", 9),
], dy=0.046)
# connector naming the conflict
arr = FancyArrowPatch((0.435, 0.57), (0.565, 0.57),
arrowstyle="<|-|>", mutation_scale=16,
lw=2, color=SEC, transform=ax.transAxes)
ax.add_patch(arr)
ax.text(0.50, 0.615, "direction\nreversed", color=SEC, fontsize=9.5,
fontstyle="italic", ha="center", va="bottom", transform=ax.transAxes,
linespacing=1.0)
# direction key (so the finding never rests on colour alone)
ax.text(0.035, 0.235,
"Key: orange = direction printed in the abstract;",
fontsize=9.5, color=SEC, va="top")
ax.text(0.035, 0.190,
"blue = direction printed everywhere else in the paper.",
fontsize=9.5, color=SEC, va="top")
ax.text(0.035, 0.130,
"Pimonidazole marks hypoxia: higher staining = more hypoxic. "
"Uninfected controls",
fontsize=9.5, color=SEC, va="top")
ax.text(0.035, 0.085,
"(S3B Fig) were comparable, so no baseline difference explains the "
"reversal. No",
fontsize=9.5, color=SEC, va="top")
ax.text(0.035, 0.040,
"intensities are reported; the check is only that four sections agree "
"and the abstract does not.",
fontsize=9.5, color=SEC, va="top")
return fig
if __name__ == "__main__":
main()
Combined odds ratio interval for HLA-C variant rs114598080 is incompatible with its p-value and components
PLoS Pathog (2020) · doi:10.1371/journal.ppat.1008565 · PMC7259797
Summary
The abstract, the Results text and Table 2 report the combined association of rs114598080 with leprosy as OR = 1.47 [1.46–1.48] with p = 8.77x10-13. An interval that narrow implies a test statistic of about z = 111 (about 75.5 under the interval convention the other rows of Table 2 follow, and no less than about 50 at any rounding), while p = 8.77x10-13 implies z = 7.15 and an interval of about [1.32–1.63]. The combined OR of 1.47 also lies below both of its components (discovery 1.85, replication 2.06). Pooling those two components gives about 1.97 [1.65–2.36], with p ≈ 6x10-14, close to the printed p-value. rs114598080 is one of the three independent HLA signals the abstract highlights. As printed, its effect size is probably too low and its precision is overstated about 10–15-fold in standard error (roughly 110–240 times in statistical information), depending on the interval convention.
What the paper reports
Abstract:
We identified three independently associated SNPs, two located in the HLA class I region (rs1265048: OR = 0.69 [0.58–0.80], combined p-value = 5.53x10-11; and rs114598080: OR = 1.47 [1.46–1.48], combined p-value = 8.77x10-13), and one located in the HLA class II region (rs3187964 (OR = 1.67 [1.55–1.80], combined p-value = 8.35x10-16).
Results, "Replication study > Within the MHC region", third paragraph:
The rs114598080 SNP, located in a CTCF binding site (ENSR00000262841) 15.6 kb upstream of the HLA-C gene, gave a univariate p-value of 8.8x10-13 in the combined analysis, with an OR for developing leprosy for AG vs. GG (or AA vs. AG) of 1.47 [1.46–1.48].
Table 2, row rs114598080. The columns are: SNP, m/M, gene, MAF, then P value and OR [95% CI] for the discovery GWAS, the replication (one-tailed, "#") and the combined GWAS and replication sample:
rs114598080 (6:31255442) | A/G | HLA-C | 0.10 | 3.90x10-06 | 1.85 [1.39–2.44] | 4.39x10-11 | 2.06 [1.7 –Inf] | 8.77x10-13 | 1.47 [1.46–1.48]
Table 2 footer (excerpt):
* with respect to the minor allele# one-tailed test
Materials and methods, "Statistical methods", first paragraph, on the combined analysis:
Conditional logistic regression was also used for a combined analysis of the discovery family and replication case-control samples. In this combined analysis, all cases and controls from the case-control study were grouped together in a stratum for analysis, in combination with the strata consisting of the affected offspring and their pseudosibs, as previously described [34, 35].
Results, same section, second paragraph, on the multivariable model:
We found that three SNPs (rs3187964, rs114598080 and rs1265048) were required to capture the full association signal in the HLA region (S7 Table).
The strength of association was much lower for rs114598080 (multivariable p-value = 3.75x10-5), and even lower for rs1265048, (multivariable p-value = 0.011).
What the paper's own numbers show
Method. For a Wald estimate, the 95% interval and the p-value both follow from the same standard error of the log OR. So each can be recovered from the other:
- From the interval: SE = (ln upper − ln lower) / (2 × 1.96) and z = ln OR / SE.
- From the p-value: z = Φ⁻¹(1 − p/2) and SE = ln OR / z.
The other four combined rows of Table 2 have intervals that are symmetric on the OR scale (for example 0.69 [0.58–0.80]). Read as OR ± 1.96·SE(log OR), they reproduce each row's printed p-value to within rounding. I therefore apply both the standard log-scale reading and this reading to rs114598080. Separately, I pool the SNP's discovery and replication estimates by inverse variance. Their SEs come from their printed intervals; the replication interval is a one-sided 95% bound, so its SE is (ln 2.06 − ln 1.7) / 1.645. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| rs114598080 combined OR [95% CI] | Abstract; Results; Table 2 | 1.47 [1.46–1.48] | — |
| rs114598080 combined p | Abstract and Table 2 (8.77x10-13); Results (8.8x10-13) | 8.77x10-13 | — |
| SE(log OR) implied by the CI, log scale | from Table 2 | — | 0.00347 → z = 111.0, p ≈ 10^-2678 |
| Same log-scale reading, at the widest rounding [1.455–1.485] | from Table 2 | — | SE 0.00521 → z = 74.0, p ≈ 10^-1191 |
| Same, under the other rows' OR ± 1.96·SE reading | from Table 2 | — | SE 0.0051 → z = 75.5, p ≈ 10^-1240 |
| Same reading, at the widest rounding [1.455–1.485] | from Table 2 | — | SE 0.00765 → z = 50.3 at OR 1.47 (49.9 at OR 1.465), p ≈ 10^-542 |
| z and SE implied by the reported p | from Table 2 | p = 8.77x10-13 | z = 7.15, SE = 0.0539: 15.5x the interval's SE under the log-scale reading (241x less information), 10.6x under the OR ± 1.96·SE reading (112x), 7.0x at that reading's widest rounding (50x) |
| 95% CI implied by the reported p, at OR 1.47 | from Table 2 | [1.46–1.48] | [1.32–1.63] log scale; [1.36–1.58] under the OR ± 1.96·SE reading |
| Other four combined rows: |z| from p vs from CI (OR ± 1.96·SE reading) | Table 2 | — | 6.56 vs 6.61; 8.05 vs 8.04; 7.53 vs 7.46; 5.41 vs 5.36 (all agree) |
| Combined OR relative to its components | Table 2 | 1.47 vs discovery 1.85, replication 2.06 | outside both; the other four rows lie between their components |
| Discovery component | Table 2 | 1.85 [1.39–2.44], p 3.90x10-06 | SE 0.1435, z 4.29 |
| Replication component (one-sided bound) | Table 2 | 2.06 [1.7–Inf], one-tailed p 4.39x10-11 | SE 0.1168, z 6.19 |
| Inverse-variance pool of the two components | from Table 2 | — | OR 1.97 [1.65–2.36], SE 0.0906, z 7.51, p ≈ 6.1x10-14 |
| Same, with the bound "1.7" read as 1.65 or 1.75 | from Table 2 | — | 1.96 [1.62–2.37], p 8x10-12; 1.99 [1.70–2.33], p 3.4x10-17 |
| SE implied by the reported p if the OR were 1.97 | from Table 2 | — | 0.0951 (pooled component SE 0.0906) |
| Rough allelic SE from sample size: 1803 cases, ≤ 2534 controls + pseudosibs | Abstract; Methods; MAF from Table 2 (0.10) or S5 Table (0.13) | — | 0.073 (MAF 0.10) or 0.065 (MAF 0.13); an SE of 0.0035 would need a sample about 350–440x larger |
Three things follow:
- The printed interval and the printed p-value cannot both be right. No rounding or interval convention brings the interval's z below about 50, and the p-value's z is 7.15.
- The printed OR of 1.47 cannot be a pooled estimate of 1.85 and 2.06. A common-OR likelihood that is the sum of the two parts has its maximum between the maxima of the two parts. All four other rows of Table 2 behave this way.
- The printed p-value fits the pooled components. Pooling gives z = 7.51 (p ≈ 6x10-14), against z = 7.15 printed. At an OR near 1.97, the printed p implies an SE of 0.095, close to the pooled SE of 0.091. This suggests that the p-value is correct and that the OR and its interval are not, although only the original model output can show which value is wrong.
Figure
Figure 1. Odds ratios for rs114598080 on one log-scale axis; the thin vertical line marks OR = 1. Orange values are printed in Table 2: discovery 1.85 [1.39–2.44], replication 2.06 with a one-sided 95% lower bound of 1.7 (arrow = open upper end), and the combined estimate 1.47 [1.46–1.48], whose interval is barely wider than its marker. Blue values are recomputed from Table 2's own numbers by recompute.py.txt. The first is the 95% interval that the printed p = 8.77x10-13 implies around the printed OR, [1.32–1.63]. The second is the inverse-variance pool of the two printed components, 1.97 [1.65–2.36]. The printed combined estimate lies below both component point estimates, and its SE is about 10–15 times smaller than its own p-value allows (15.5x under a log-scale reading of the interval, 10.6x under the OR ± 1.96·SE reading the other rows follow).
Figure 2. For each combined row of Table 2, the test statistic |z| implied by the printed combined p-value (orange ring) and by the printed combined interval read as OR ± 1.96·SE(log OR) (blue diamond), on a log axis. For four SNPs the two agree to within 0.07, so this reading of the intervals is consistent with the table. For rs114598080 the p-value gives 7.15 and the interval gives 75.5 (111 under a standard log-scale reading). All values are computed by recompute.py.txt from Table 2.
Why it matters
- It is a headline result. rs114598080 is one of the three independent HLA signals named in the abstract, and the abstract gives its effect as OR 1.47 [1.46–1.48]. That figure is likely to be carried into reviews, meta-analyses and fine-mapping comparisons across populations.
- The magnitude is probably understated. The paper's own component estimates point to a combined OR near 1.9–2.0, not 1.47.
- The precision is greatly overstated. The printed interval claims roughly 110–240 times the statistical information that the p-value supports (112x under the OR ± 1.96·SE reading, 241x under the log-scale reading). It is also far more than a sample of about 1800 cases could give at this allele frequency: an SE of 0.0035 would need a sample about 350–440 times larger. A meta-analysis that weights studies by the printed interval would give this estimate almost all the weight.
- What is not affected. The direction of the association (the A allele raises risk), its genome-wide significance, its replication in the case/control sample, and the conclusion that three SNPs capture the HLA signal all stand. That conclusion rests on the multivariable case/control model (rs114598080: multivariable p = 3.75x10-5, OR 1.64 [1.29–2.09] in S7 Table), not on the combined univariate estimate.
Explanations considered
- Rounding. Does not account for it. The widest endpoints that round to 1.46 and 1.48 (1.455 and 1.485) still give SE 0.0052 and z = 74 under the log-scale reading, and SE 0.0077 and z ≈ 50 (49.9 at OR 1.465) under the OR ± 1.96·SE reading. Both are far above the p-value's z of 7.15.
- A different interval convention (OR-scale symmetric, profile or score intervals). Does not account for it. The other four combined rows follow OR ± 1.96·SE(log OR) and match their p-values. Under that same rule rs114598080 gives z = 75.5. Profile and score intervals differ from Wald intervals only slightly at this sample size, not by a factor of 10–15 in width.
- A one-sided p-value. Does not account for it. The footer's "#" (one-tailed) marks only the replication columns, and a one-sided reading would move z only from 7.15 to 7.05.
- A different confidence level. Does not account for it. The column header says 95% CI. An interval as narrow as [1.46–1.48] at z = 7.15 would correspond to a confidence level of about 10%.
- A different contrast or genetic model. Does not account for it. The text describes the OR per minor allele ("AG vs. GG (or AA vs. AG)"), the same additive coding as the other rows. A different contrast would not explain why the interval is incompatible with the p-value from the same model.
- Different samples in the combined row. Not supported. The combined analysis uses the same discovery families and replication cases and controls as the other rows (Methods). The two components' SEs (0.14 and 0.12) cannot combine to anything below about 0.09.
- Different models for the components (FBAT and SNPTEST p-values versus conditional logistic regression in the combined analysis). Accounts for small differences only. The component p-values differ from their own intervals by about 0.3 in z, which is enough to blur the pooled p slightly but not to move a pooled OR from about 1.97 to 1.47, or an SE from about 0.09 to 0.0035.
- A conversion or typesetting artifact. Ruled out. The same numbers appear in the publisher's PDF (page 1 abstract, page 7 Table 2, page 8 Results), on the PLOS HTML page, in the PMC archive and in the medRxiv preprint abstract (doi:10.1101/2020.01.30.20018432).
Status
As of 2026-10-24:
- Version of record. I checked the publisher's PDF (journals.plos.org,
file?type=printable, 16 pages, text extracted with PyMuPDF). The abstract (p. 1), Table 2 (p. 7) and the Results text (p. 8) all read 1.47 [1.46–1.48]. The PLOS HTML page, PMC and Europe PMC agree, and so does the medRxiv preprint. - Supplements. S5 Table (HLA SNP statistics) and S7 Table (multivariable analysis), downloaded from the PLOS article page, give rs114598080 discovery p 3.9016x10-06, replication p 4.39264x10-11 and replication OR "2.055 [1.7 - Inf[". S7 also repeats Table 2's discovery and replication univariate values and gives the multivariable OR 1.64 [1.29 -2.09]. Neither table reports the combined-sample OR or interval, and no model output for the combined analysis is published, so the supplements cannot settle which combined value is wrong.
- Corrections. Crossref lists only the publisher's "new version" record (2020-05-29) and the preprint relation. PMC and Europe PMC have no correction, expression of concern or retraction notice. The PLOS article page shows "Reader Comments (0)".
- PubPeer. The PubPeer search API required human verification and could not be queried directly. Web searches for the title, DOI and SNP returned no PubPeer entry.
Sources: PLOS article, PMC7259797, medRxiv preprint, Crossref REST API (relation.object and update records for the DOI).
Suggested correction
The authors would need to re-extract the combined conditional logistic regression output for rs114598080 and correct the OR and 95% CI in three places: the abstract, Table 2 (combined columns) and the Results text ("1.47 [1.46–1.48]"). They should also confirm the p-value of 8.77x10-13, which is consistent with the component estimates. Based on the paper's own numbers, the corrected estimate is expected near OR 1.9–2.0, with a 95% interval on the order of [1.65–2.36], rather than [1.46–1.48]. If the OR of 1.47 is in fact correct, the interval implied by the printed p-value would be about [1.32–1.63].
The same table has a second, smaller slip. The discovery interval for rs1265048 is printed as 0.63 [0.52–0.63], with the upper limit equal to the estimate, in both Table 2 and S7 Table. The printed p = 8.72x10-06 implies about [0.51–0.77], so the upper limit is probably about 0.76–0.77.
Recomputation script (python recompute.py.txt)
"""Recompute the combined rs114598080 estimate of PMC7259797
(PLoS Pathog 2020, doi:10.1371/journal.ppat.1008565).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from Table 2 of the paper (identical in the abstract, the Results
text, the PMC archive and the publisher's PDF), and from the sample sizes in the
Abstract / Results / Table 2 footer.
"""
import math
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z95 = norm.isf(0.025) # 1.959964
Z95_1 = norm.isf(0.05) # 1.644854 (one-sided 95% bound)
L = np.log
# ---------------------------------------------------------------- inputs (Table 2)
# name: (MAF, discovery p, discovery OR, lo, hi, replication one-tailed p, repl OR, repl bound,
# combined p, combined OR, lo, hi)
T2 = {
"rs1265048": (0.49, 8.72e-6, 0.63, 0.52, 0.63, 1.58e-6, 0.72, 0.81, 5.53e-11, 0.69, 0.58, 0.80),
"rs114598080": (0.10, 3.90e-6, 1.85, 1.39, 2.44, 4.39e-11, 2.06, 1.70, 8.77e-13, 1.47, 1.46, 1.48),
"rs3187964": (0.41, 1.28e-6, 1.52, 1.25, 1.85, 5.28e-13, 1.72, 1.51, 8.35e-16, 1.67, 1.55, 1.80),
"rs3764147": (0.35, 6.56e-5, 1.41, 1.18, 1.67, 2.36e-11, 1.61, 1.49, 5.06e-14, 1.52, 1.41, 1.63),
"rs6871626": (0.35, 5.12e-4, 0.71, 0.58, 0.86, 1.37e-5, 0.74, 0.86, 6.44e-8, 0.73, 0.61, 0.84),
}
# Sample sizes: 622 affected offspring (discovery), up to 3 pseudosibs each;
# replication 1181 cases and 668 controls.
N_OFF, N_CASE_R, N_CTRL_R = 622, 1181, 668
def p2(z):
return 2 * norm.sf(abs(z))
def log10_p2(z):
"""log10 of a two-sided normal p-value, safe for very large |z|."""
return (math.log(2) + norm.logsf(abs(z))) / math.log(10)
def z_from_p2(p):
return norm.isf(p / 2)
def se_logci(lo, hi):
"""SE of log OR from a standard log-scale 95% CI."""
return (L(hi) - L(lo)) / (2 * Z95)
def se_linci(lo, hi):
"""SE of log OR if the CI was printed as OR +/- 1.96*SE(log OR) (see section 2)."""
return (hi - lo) / (2 * Z95)
def h(t):
print("\n" + "=" * 8 + " " + t + " " + "=" * 8)
S = T2["rs114598080"]
OR_C, LO_C, HI_C, P_C = S[9], S[10], S[11], S[8]
b_c = L(OR_C)
# ---------------------------------------------------------------- 1. log-scale reading
h("1. rs114598080 combined: OR 1.47 [1.46-1.48], p = 8.77e-13 (standard log-scale CI)")
se_ci = se_logci(LO_C, HI_C)
z_ci = b_c / se_ci
print(f"SE(log OR) implied by CI = {se_ci:.5f}")
print(f"z implied by CI = {z_ci:.1f}; log10 two-sided p = {log10_p2(z_ci):.0f}")
z_p = z_from_p2(P_C)
se_p = b_c / z_p
ci_p = (OR_C * math.exp(-Z95 * se_p), OR_C * math.exp(Z95 * se_p))
print(f"z implied by reported p = {z_p:.3f}; SE(log OR) = {se_p:.4f}")
print(f"95% CI implied by p (log scale) = [{ci_p[0]:.2f}, {ci_p[1]:.2f}]")
print(f"ratio of SEs (p-implied / CI-implied) = {se_p / se_ci:.1f}x; information ratio = {(se_p / se_ci) ** 2:.0f}x")
se_w = se_logci(1.455, 1.485)
print(f"widest rounding [1.455, 1.485]: SE = {se_w:.5f}, z = {b_c / se_w:.1f}, log10 p = {log10_p2(b_c / se_w):.0f}")
z_1s = norm.isf(P_C)
print(f"if p were one-sided: z = {z_1s:.3f} (vs {z_p:.3f} two-sided)")
zc = (L(HI_C) - L(LO_C)) / 2 / se_p
print(f"confidence level at which SE {se_p:.4f} gives an interval as narrow as [1.46, 1.48]: "
f"{100 * (2 * norm.cdf(zc) - 1):.0f}%")
# ---------------------------------------------------------------- 2. the table's own CI convention
h("2. Convention of the combined CIs: symmetric OR +/- 1.96*SE(log OR)")
print("row | CI half-widths (lower, upper) | z from CI (linear rule) | z from reported p | p from CI (linear) | reported p | z from CI (log rule) | p from CI (log rule)")
fig2_rows = []
for name, r in T2.items():
p, orr, lo, hi = r[8], r[9], r[10], r[11]
zl = abs(L(orr)) / se_linci(lo, hi)
zg = abs(L(orr)) / se_logci(lo, hi)
zr = z_from_p2(p)
fig2_rows.append((name, p, zl, zr))
print(f"{name:12s} | {orr - lo:.2f}, {hi - orr:.2f} | {zl:6.2f} | {zr:5.2f} | 10^{log10_p2(zl):.1f} | {p:.3g} | {zg:6.2f} | 10^{log10_p2(zg):.1f}")
se_lin = se_linci(LO_C, HI_C)
print(f"rs114598080 under the linear rule: SE = {se_lin:.4f}, z = {b_c / se_lin:.1f}, log10 p = {log10_p2(b_c / se_lin):.0f}")
print(f"interval the reported p implies under the linear rule: "
f"[{OR_C - Z95 * se_p:.2f}, {OR_C + Z95 * se_p:.2f}]")
print(f"ratio of SEs under the linear rule (p-implied / CI-implied) = {se_p / se_lin:.1f}x; "
f"information ratio = {(se_p / se_lin) ** 2:.0f}x")
se_lw = se_linci(1.455, 1.485)
print(f"linear rule, widest rounding [1.455, 1.485]: SE = {se_lw:.5f}, z = {b_c / se_lw:.1f} at OR 1.47, "
f"z = {L(1.465) / se_lw:.1f} at OR 1.465; log10 p = {log10_p2(L(1.465) / se_lw):.0f}; "
f"SE ratio = {se_p / se_lw:.1f}x, information ratio = {(se_p / se_lw) ** 2:.0f}x")
print(f"log rule, widest rounding, at OR 1.465: z = {L(1.465) / se_w:.1f}; "
f"SE ratio = {se_p / se_w:.1f}x, information ratio = {(se_p / se_w) ** 2:.0f}x")
# ---------------------------------------------------------------- 3. combined vs components
h("3. Combined OR versus its two component ORs (Table 2)")
for name, r in T2.items():
d, rp, c = r[2], r[6], r[9]
inside = min(d, rp) <= c <= max(d, rp)
print(f"{name:12s}: discovery {d:.2f}, replication {rp:.2f}, combined {c:.2f} -> "
f"{'between' if inside else 'OUTSIDE'} the component ORs")
h("4. Inverse-variance combination of the rs114598080 components")
se_d = se_logci(S[3], S[4])
z_d = L(S[2]) / se_d
print(f"discovery 1.85 [1.39-2.44]: SE = {se_d:.4f}, z = {z_d:.2f}, two-sided p = {p2(z_d):.2g} "
f"(reported 3.90e-06, from FBAT; z = {z_from_p2(S[1]):.2f})")
se_r = (L(S[6]) - L(S[7])) / Z95_1
z_r = L(S[6]) / se_r
print(f"replication 2.06 [1.7-Inf], one-sided 95% bound: SE = {se_r:.4f}, z = {z_r:.2f}, "
f"one-sided p = {norm.sf(z_r):.2g} (reported 4.39e-11; z = {norm.isf(S[5]):.2f})")
w = np.array([1 / se_d ** 2, 1 / se_r ** 2])
b = np.array([L(S[2]), L(S[6])])
bm = float((w * b).sum() / w.sum())
sem = float(w.sum() ** -0.5)
IV = (math.exp(bm), math.exp(bm - Z95 * sem), math.exp(bm + Z95 * sem))
print(f"inverse-variance combination: OR = {IV[0]:.2f} [{IV[1]:.2f}-{IV[2]:.2f}], SE = {sem:.4f}, "
f"z = {bm / sem:.2f}, two-sided p = {p2(bm / sem):.2g}")
print(f"SE of the combination / SE implied by the printed CI = {sem / se_ci:.0f}x")
for bound in (1.65, 1.75): # rounding of the one-sided bound "1.7"
s_r = (L(S[6]) - L(bound)) / Z95_1
ww = np.array([1 / se_d ** 2, 1 / s_r ** 2])
bb = float((ww * b).sum() / ww.sum())
ss = float(ww.sum() ** -0.5)
print(f" sensitivity, replication bound {bound}: OR = {math.exp(bb):.2f} "
f"[{math.exp(bb - Z95 * ss):.2f}-{math.exp(bb + Z95 * ss):.2f}], z = {bb / ss:.2f}, p = {p2(bb / ss):.2g}")
se_at_iv = math.log(IV[0]) / z_p
print(f"SE implied by the reported p if the OR were {IV[0]:.2f}: {se_at_iv:.4f} "
f"(vs {sem:.4f} from the components)")
# ---------------------------------------------------------------- 5. sample size
h("5. Rough SE from sample size (allelic, MAF 0.10)")
maf = S[0]
nc = N_OFF + N_CASE_R
nk = 3 * N_OFF + N_CTRL_R
se_n = math.sqrt(1 / (2 * nc * maf * (1 - maf)) + 1 / (2 * nk * maf * (1 - maf)))
print(f"cases {nc}, controls + pseudosibs <= {nk}: SE(log OR) ~ {se_n:.4f}")
print(f"sample would need to be ~{(se_n / se_ci) ** 2:.0f}x larger to give SE {se_ci:.4f}")
m2 = 0.13 # S5 Table gives MAF 0.133 (discovery) / 0.130 (replication) for rs114598080
se_n2 = math.sqrt(1 / (2 * nc * m2 * (1 - m2)) + 1 / (2 * nk * m2 * (1 - m2)))
print(f"with MAF 0.13 (S5 Table): SE(log OR) ~ {se_n2:.4f}; ~{(se_n2 / se_ci) ** 2:.0f}x larger sample needed")
# ---------------------------------------------------------------- 6. second slip in Table 2
h("6. rs1265048 discovery interval 0.63 [0.52-0.63]")
r = T2["rs1265048"]
zd = z_from_p2(r[1])
sd = abs(L(r[2])) / zd
print(f"upper limit equals the OR. Log-symmetric upper limit = {r[2] ** 2 / r[3]:.2f}; "
f"from p = 8.72e-06: z = {zd:.2f}, SE = {sd:.4f}, CI = [{r[2] * math.exp(-Z95 * sd):.2f}, {r[2] * math.exp(Z95 * sd):.2f}]")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
LW, MS = 2.0, 9
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# fig1: dot-and-interval plot of the rs114598080 estimates on one log OR axis
rows = [
("Discovery (families)", "reported", S[2], S[3], S[4], f"{S[2]:.2f} [{S[3]:.2f}–{S[4]:.2f}]", False),
("Replication (case/control)", "reported", S[6], S[7], None, f"{S[6]:.2f} [{S[7]:.1f}–Inf], one-sided", True),
("Combined, as printed", "reported", OR_C, LO_C, HI_C, f"{OR_C:.2f} [{LO_C:.2f}–{HI_C:.2f}]", False),
("Combined: interval implied\nby printed p = 8.77×10⁻¹³", "recomputed", OR_C, ci_p[0], ci_p[1],
f"{OR_C:.2f} [{ci_p[0]:.2f}–{ci_p[1]:.2f}]", False),
("Combined: inverse-variance\npool of the two components", "recomputed", IV[0], IV[1], IV[2],
f"{IV[0]:.2f} [{IV[1]:.2f}–{IV[2]:.2f}]", False),
]
XMIN, XMAX = 0.9, 3.2
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.arange(len(rows))[::-1].astype(float)
ys[3:] -= 0.35 # small gap between printed and recomputed rows
for (lab, kind, est, lo, hi, txt, open_hi), y in zip(rows, ys):
col = ORANGE if kind == "reported" else BLUE
if open_hi:
ax.annotate("", xy=(XMAX, y), xytext=(lo, y),
arrowprops=dict(arrowstyle="-|>", color=col, lw=LW, mutation_scale=12))
ax.plot([lo, lo], [y - 0.13, y + 0.13], color=col, lw=LW)
else:
ax.plot([lo, hi], [y, y], color=col, lw=LW, solid_capstyle="butt")
ax.plot([lo, lo], [y - 0.13, y + 0.13], color=col, lw=LW)
ax.plot([hi, hi], [y - 0.13, y + 0.13], color=col, lw=LW)
ax.plot(est, y, "o", color=col, ms=MS, mec=BG, mew=1.2, zorder=3)
ax.text(XMAX * 1.03, y, txt, va="center", ha="left", color=col, fontsize=10)
ax.axvline(1.0, color=INK2, lw=0.8, zorder=0)
ax.text(1.0, ys[0] + 0.62, "OR = 1", ha="center", va="bottom", color=INK2, fontsize=10)
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.set_ylim(ys[-1] - 0.6, ys[0] + 0.9)
ticks = [1.0, 1.25, 1.5, 2.0, 2.5, 3.0]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(plt.NullLocator())
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10.5)
for tl, r in zip(ax.get_yticklabels(), rows):
tl.set_color(ORANGE if r[1] == "reported" else BLUE)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Odds ratio per A allele, rs114598080 (log scale), with 95% CI")
from matplotlib.lines import Line2D # noqa: E402
ax.legend(handles=[
Line2D([], [], color=ORANGE, lw=LW, marker="o", ms=MS, mec=BG, label="Printed in Table 2"),
Line2D([], [], color=BLUE, lw=LW, marker="o", ms=MS, mec=BG, label="Recomputed from Table 2's numbers"),
], loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.335, right=0.70, top=0.97, bottom=0.27)
save(fig, "fig1")
# fig2: for each combined row, |z| from the printed p vs |z| from the printed CI
order = ["rs6871626", "rs1265048", "rs3764147", "rs3187964", "rs114598080"]
dd = {n: (p, zl, zr) for n, p, zl, zr in fig2_rows}
fig, ax = plt.subplots(figsize=(7.2, 3.3))
for i, n in enumerate(order):
p, zl, zr = dd[n]
y = i
ax.plot([min(zl, zr), max(zl, zr)], [y, y], color=GRID, lw=LW, zorder=1)
ax.plot(zr, y, "o", mfc="none", mec=ORANGE, ms=MS + 6, mew=2, zorder=3)
ax.plot(zl, y, "D", color=BLUE, ms=MS - 2, mec=BG, mew=0.8, zorder=4)
if n == "rs114598080":
ax.text(zr, y + 0.32, f"{zr:.2f}", ha="center", va="bottom", color=ORANGE, fontsize=10)
ax.text(zl, y + 0.32, f"{zl:.1f}", ha="center", va="bottom", color=BLUE, fontsize=10)
else:
ax.text(max(zl, zr) * 1.1, y, f"p: {zr:.2f} CI: {zl:.2f}", va="center", ha="left",
color=INK2, fontsize=10)
ax.set_xscale("log")
ax.set_xlim(4, 110)
xt = [5, 7, 10, 20, 50, 100]
ax.set_xticks(xt)
ax.set_xticklabels([str(t) for t in xt])
ax.xaxis.set_minor_locator(plt.NullLocator())
ax.set_yticks(range(len(order)))
ax.set_yticklabels(order)
ax.set_ylim(-0.6, len(order) - 0.2)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("|z| of the combined association (log scale)")
ax.legend(handles=[
Line2D([], [], lw=0, marker="o", mfc="none", mec=ORANGE, ms=MS + 6, mew=2,
label="From the printed combined p-value (p)"),
Line2D([], [], color=BLUE, lw=0, marker="D", ms=MS - 2, mec=BG,
label="From the printed combined CI, read as OR ± 1.96·SE(log OR) (CI)"),
], loc="upper left", bbox_to_anchor=(0.0, -0.22), ncol=1, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.16, right=0.97, top=0.93, bottom=0.36)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Zarnekow fen 2018 CO2 components imply a net source; the paper reports a net sink
Philos Trans R Soc Lond B Biol Sci (2020) · doi:10.1098/rstb.2019.0685 · PMC7485093
Summary
The paper reports eddy-covariance CO2 budgets for two rewetted fens, Hütelmoor (Hte) and Zarnekow (Zrk), during the 2018 drought and the year after. Its Abstract states that the drought "stopped the persistent net CO2 emissions of Zrk", which rests on a reported 2018 net balance (NEE) of −0.03 kg CO2 m⁻², a small net sink. The paper defines NEE as the sum of gross ecosystem productivity (GEP, negative) and ecosystem respiration (Reco), and its own Zrk 2018 sums give 3.88 − 3.72 = +0.16 kg CO2 m⁻², a net source. The Zrk 2019 sums do not add up either (+0.33 against a reported +0.06), while the Hte budget closes exactly, and the curves in Figure 3 show the same mismatch as the text. So the reported Zrk numbers cannot all be right, and which ones are wrong decides whether Zrk was a sink or a source in 2018, and how much its emissions fell in 2019.
What the paper reports
Abstract:
Altogether, the drought reduced the net CO2 sink in Hte, while it stopped the persistent net CO2 emissions of Zrk.
Methods, "Data acquisition" (the definitions that link the three quantities):
The measured net ecosystem exchange of CO2 (NEE) was partitioned into the two opposed component fluxes gross ecosystem productivity (GEP, here indicated with a negative sign) and ecosystem respiration (Reco) by modelling. Reco, the CO2 released by autotrophic and heterotrophic respiration into the atmosphere, was obtained from night-time NEE fluxes. GEP, the photosynthetic sequestration of CO2 from the atmosphere into the canopy, was then calculated from the difference between NEE and Reco.
Results, "Drought effects on carbon dioxide and methane exchange", first paragraph:
The summer drought 2018 caused new maxima in annual GEP sums of 3.58 and 3.72 kg CO2 m-2 in Hte and Zrk (figure 3a,b). These exceeded the average annual GEP totals of the reference years 2014–2017 by 22 and 86%, respectively.
GEP in Zrk, on the other hand, remained high in 2019, reaching annual sums of 3.39 kg m-2.
Same section, second paragraph:
The warm and dry conditions in 2018 also induced new maxima in the annual Reco sums, which amounted to 3.19 and 3.88 kg CO2 m-2 in Hte and Zrk (figure 3c,d). This corresponded to an increase of 35 and 56%, respectively, in comparison to reference years.
In Zrk, Reco remained at the same level in 2019 (3.72 kg CO2 m-2).
Same section, third paragraph (the net balances):
In Hte, the resulting net CO2 balance amounted to -0.39 kg CO2 m-2. Hence, Hte remained a CO2 sink though net CO2 uptake was 0.18 kg m-2 lower than in previous years (figure 3e). In Zrk, photosynthetic CO2 sequestration almost doubled in 2018 and thereby offset respiratory CO2 losses by 0.03 kg CO2 m-2 (figure 3f). Indeed, net CO2 uptake was comparatively low, but the fen had previously acted as a net CO2 source with average NEE balances of +0.34 kg CO2 m-2 [14]. In 2019, Zrk became a net CO2 source of 0.06 kg m-2 again, though CO2 emissions were considerably lower than in the years before the drought.
Same section, last paragraph:
In addition, the year after the drought revealed marked carry-over effects in Zrk: here, the newly formed vegetation persisted under climate-normal conditions with beneficial consequences for GEP and the net CO2 balance.
Figure 3 caption (excerpt):
Cumulative fluxes of CO2 and CH4 for the year of drought 2018, the post-drought year 2019 and the reference period 2014–2017. Negative signs indicate CO2 uptake from the atmosphere into the ecosystem.
Conclusion:
The drought-induced spread in vegetation can increase photosynthetic CO2 sequestration and thereby move the net CO2 budget closer towards a sink.
What the paper's own numbers show
Method. With GEP negative, the Methods' definition "GEP … calculated from the difference between NEE and Reco" means GEP = NEE − Reco, so annual NEE = GEP + Reco = Reco − |GEP|. I applied this identity to every site-year for which the text gives all three annual sums, and used Hte as a control. Each value is printed to 0.01, so rounding can move GEP + Reco − NEE by at most 3 × 0.005 = 0.015. I also back-calculated the reference-period (2014–2017) means implied by the stated percentage increases. To check whether the text matches the figure, I read the end-of-year values of the cumulative GEP, Reco and NEE curves in Figure 3 from the publisher's image (PMC open-access copy, rstb20190685-g3.jpg). I calibrated each panel's y axis from its tick marks and recorded the pixel row of each curve at its last drawn column. One pixel is 0.017 (GEP), 0.019 (Reco) and 0.009 (NEE) kg CO2 m⁻², so the summed reading error is about ±0.045. Every number below is printed by recompute.py.txt. Units are kg CO2 m⁻² yr⁻¹, and negative values mean net uptake.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Hte 2018 NEE (control) | Results, 3rd para.; Fig. 3a,c,e | GEP −3.58, Reco 3.19, NEE −0.39 | 3.19 − 3.58 = −0.39; closes exactly |
| Hte 2018 drop in uptake vs reference (control) | Results, 3rd para. | 0.18 lower | reference NEE −0.571 (from +22% and +35%), so 0.181 lower; consistent |
| Zrk 2018 NEE | Results, 1st–3rd para.; Fig. 3b,d,f; Abstract | GEP −3.72, Reco 3.88, NEE −0.03 (sink) | 3.88 − 3.72 = +0.16 (source); gap +0.19 |
| Zrk 2018, rounding range | derived | — | GEP + Reco between +0.15 and +0.17; cannot round to −0.03 |
| Zrk 2018, value needed to give NEE −0.03 | derived | Reco 3.88 or GEP −3.72 | Reco 3.69 (with GEP −3.72), or GEP −3.91 (with Reco 3.88) |
| Zrk 2019 NEE | Results, 1st–3rd para. | GEP −3.39, Reco 3.72, NEE +0.06 | 3.72 − 3.39 = +0.33; gap +0.27 |
| Zrk 2019, rounding range | derived | — | +0.32 to +0.34 |
| Zrk 2019, value needed to give NEE +0.06 | derived | Reco 3.72 or GEP −3.39 | Reco 3.45 (with GEP −3.39), or GEP −3.66 (with Reco 3.72) |
| Zrk reference GEP and Reco implied by +86% and +56% | Results, 1st–2nd para. | "almost doubled" | reference |GEP| 2.000, reference Reco 2.487; 2018 GEP is ×1.86 (consistent with "almost doubled") |
| Zrk reference NEE implied by the percentages | derived | +0.34 (cited from ref. [14]) | +0.487; ref. [14] was published in 2016, so its period cannot be the full 2014–2017 reference |
| Zrk 2018 Reco if the reference NEE were +0.34 | derived | 3.88 (+56%) | reference Reco 2.340, so +56% gives 3.65, and 3.88 would be +66% |
| Figure 3 end-of-year values, Hte 2018 | Fig. 3a,c,e | — | GEP −3.59, Reco +3.18, sum −0.41; NEE curve −0.41; non-closure 0.00 |
| Figure 3 end-of-year values, Hte 2019 | Fig. 3a,c,e | GEP −2.56 (text) | GEP −2.56, Reco +2.40, sum −0.16; NEE curve −0.14; non-closure −0.02 |
| Figure 3 end-of-year values, Zrk 2018 | Fig. 3b,d,f | GEP −3.72, Reco 3.88, NEE −0.03 | GEP −3.71, Reco +3.87, sum +0.16; NEE curve −0.03; non-closure +0.19 |
| Figure 3 end-of-year values, Zrk 2019 | Fig. 3b,d,f | GEP −3.39, Reco 3.72, NEE +0.06 | GEP −3.39, Reco +3.70, sum +0.32; NEE curve +0.04; non-closure +0.28 |
| Zrk 2019 vs the earlier average of +0.34 | Results, 3rd para. | "considerably lower" | stated +0.06 is 0.28 lower; GEP + Reco +0.33 is 0.01 lower (0.16 lower than the percentage-implied reference +0.49) |
The 2019 dashed curves in Figure 3 end about one to two weeks before 31 December, so the 2019 readings are slightly early. The Zrk 2019 NEE curve is still rising at that point, which is why it reads +0.04 rather than the stated +0.06.
The Hte budget closes to within 0.02: in 2018 in both the text and the figure, and in 2019 in the figure (the text gives no 2019 Hte Reco or NEE). The Zrk budget fails to close by about 0.2 to 0.3 kg CO2 m⁻² in both years, and Figure 3 reproduces the same mismatch as the text. The mismatch is therefore not a misprint in one sentence: the cumulative Zrk GEP and Reco curves, as drawn, do not add up to the Zrk NEE curve drawn below them.
Figure
Figure 1. For each site-year in which the text states all three annual sums, the orange dot is the net CO2 balance (NEE) stated in the paper. The blue dot is Reco − |GEP| computed from the GEP and Reco sums stated in the same section. The vertical black line at zero separates a net sink (left) from a net source (right). The dotted line is the earlier Zrk average of +0.34 kg CO2 m⁻² that the paper cites from ref. [14]. For Hte 2018 the two values coincide (−0.39), so the paper's identity holds there. For Zrk 2018 the stated −0.03 is a small sink, but the stated components give +0.16, a source. For Zrk 2019 the stated +0.06 is well below the earlier average, but the stated components give +0.33, close to it.
Figure 2. The non-closure GEP + Reco − NEE for each site-year, which is zero when the budget adds up. Filled blue dots use the annual sums stated in the text, where all three are given. Hollow blue dots use the end-of-year values read from the cumulative curves of the paper's Figure 3, with a ±1-pixel reading error on each curve, summed over the three. The grey band marks ±0.015, the most that rounding to two decimals can explain. Hte closes in both years. Zrk misses by +0.19 (2018) and +0.27 to +0.28 (2019), and the text and the figure agree on this.
Why it matters
The Zrk 2018 result is one of the paper's two headline CO2 findings. The Abstract says the drought "stopped the persistent net CO2 emissions of Zrk", and the Results say the GEP gain "offset respiratory CO2 losses by 0.03 kg CO2 m-2". These statements hold only if the stated NEE of −0.03 is the correct annual balance. If the stated GEP and Reco sums are correct, Zrk stayed a net CO2 source in 2018 (+0.16), although a weaker one than the +0.34 the paper cites for earlier years. The 0.19 difference changes the sign of the balance, and so whether the drought "stopped" Zrk's emissions or only reduced them.
The 2019 statement is affected by more. The paper says Zrk "became a net CO2 source of 0.06 kg m-2 again, though CO2 emissions were considerably lower than in the years before the drought", and describes "beneficial consequences for GEP and the net CO2 balance" as a carry-over effect. With the stated 2019 components (+0.33), the 2019 balance is almost equal to the cited earlier average of +0.34. Compared with the reference implied by the paper's own percentages (+0.49), it is still 0.16 lower. Which Zrk numbers are correct therefore decides whether there is a post-drought reduction in net emissions at Zrk, and how large it is.
The direction of the broader conclusion is not affected. Both candidate 2018 values (−0.03 and +0.16) lie below the earlier Zrk average, so the drought moved the Zrk budget "closer towards a sink", as the Conclusion says. The Hte results, the GEP and Reco percentage changes and the CH4 results are not affected.
Explanations considered
- Sign convention. This does not account for the numbers. The Methods give GEP a negative sign, and the Figure 3 caption says negative values mean uptake. Under either reading, NEE = Reco − |GEP| = 3.88 − 3.72 = +0.16 for Zrk 2018. The same identity reproduces the Hte 2018 balance exactly (3.19 − 3.58 = −0.39).
- Rounding. This does not account for the numbers. Rounding three values to 0.01 can explain at most 0.015, not the gaps of 0.19 and 0.27.
- A typographical error in one sentence. This does not account for the numbers. Figure 3 shows the same values as the text: the Zrk 2018 curves end at about GEP −3.71, Reco +3.87 and NEE −0.03. The mismatch also occurs in a second year (2019), in both text and figure.
- A conversion or typesetting error in the archived copy. This is excluded. The PMC-archived text, the Europe PMC full-text XML and the publisher PDF deposited in PMC all carry the same numbers, and Figure 3 is the publisher's image.
- Annual sums that do not close because of how gaps were filled. This is possible, but the paper does not say so. Supplement S1 says NEE gaps were filled with artificial neural networks (ANNs): per-year ANNs for gaps under 24 h and a single day-aggregated ANN over 2009–2018 for longer gaps. A separate ANN predicted Reco from night-time NEE, and GEP was calculated "from the difference between the measured daytime NEE and modelled Reco". If the gap-filled NEE series and the modelled Reco differ systematically at night, or if GEP is not defined over gap periods in the same way as NEE, annual GEP + Reco need not equal annual NEE. This could produce a non-closure of this size at one site. Three things count against it as a full account. The main-text Methods define GEP as the difference between NEE and Reco, which implies closure. Hte, processed in the same way, closes to within 0.02 in both years. And no legend, footnote or supplement mentions any non-closure. Even if this is the cause, the paper presents numbers that do not add up without saying so, and the difference is large enough to change Zrk's 2018 source/sink status. Which estimate (gap-filled NEE or the sum of the components) is the better annual budget would then need to be stated and justified.
- The stated percentages identify the misprinted value. This is not decisive. If the reference NEE were the +0.34 cited from ref. [14], the stated +86% and +56% would imply a 2018 Reco of about 3.65, close to the 3.69 that would give the stated −0.03. That points to Reco. But the percentages taken alone imply a 2014–2017 reference NEE of +0.49, not +0.34. Ref. [14] was published in 2016 and so cannot cover the full 2014–2017 reference period. And the Figure 3 Reco curve itself ends at about 3.87. The material available does not settle which Zrk quantity is wrong.
Status
- Version of record. The publisher's site and PDF (royalsocietypublishing.org) returned HTTP 403 to automated requests, and the PMC article page returned a challenge page. I checked the text instead in the PMC-archived copy (
src/show.py), the Europe PMC full-text XML (the publisher's JATS deposit) and the article PDF in the PMC open-access dataset (PMC7485093.1.pdf). All three carry the quoted numbers, including "3.19 and 3.88", "3.58 and 3.72", "−0.39", "0.03 kg CO2 m−2", "3.39", "3.72" and "0.06 kg m−2". Every quotation above was confirmed verbatim against the PMC-archived text. Figure 3 values were read from the publisher's image in the PMC open-access dataset (rstb20190685-g3.jpg). - Supplements. The article has one supplementary file, rstb20190685supp1.docx (PMC open-access dataset; also on figshare as rstb20190685_si_001.docx, doi:10.6084/m9.figshare.12896846). I read it. It contains S1 (flux processing, gap-filling and partitioning, meteorology, vegetation maps, MODIS EVI, Fig. S1-1), S2 (temperature and precipitation figure) and S3 (seasonal flux figure). It has no table of annual sums and no source data, and it does not say whether, or how, annual NEE, GEP and Reco are reconciled.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. A web search for corrections or comments found none. PubPeer could not be checked directly: its API returned HTTP 404 for the DOI query and its search page returned HTTP 403.
- Sources. PMC article PMC7485093; Europe PMC full-text XML for PMC7485093; PMC open-access dataset (s3://pmc-oa-opendata/PMC7485093.1/: PDF, Figure 3 image, supplement); figshare item 10.6084/m9.figshare.12896846; Crossref; Europe PMC.
Suggested correction
- Recheck the Zrk annual sums for 2018 and 2019 (GEP, Reco and NEE in the Results text and the cumulative curves in Figure 3b, d and f), and publish a consistent set. If the components are correct, the 2018 balance is about +0.16 and the 2019 balance about +0.33 kg CO2 m⁻²; if NEE is correct, Reco or GEP must change, for example Reco ≈ 3.69 in 2018 and ≈ 3.45 in 2019 at the stated GEP.
- If the non-closure is a real result of gap-filling NEE separately from the modelled Reco and GEP, state that in the Methods and the Figure 3 legend. Report its size for each site-year, say which quantity is used as the annual net balance, and explain why it is preferred.
- Revise the Abstract and Results statements that depend on the Zrk balance accordingly: "it stopped the persistent net CO2 emissions of Zrk", "offset respiratory CO2 losses by 0.03 kg CO2 m-2", and the 2019 statement that emissions "were considerably lower than in the years before the drought".
- State the period and value of the pre-drought Zrk NEE used as the comparison. The cited +0.34 (ref. [14]) differs from the +0.49 implied by the stated percentage increases over 2014–2017.
Recomputation script (python recompute.py.txt)
"""Recompute the annual CO2 budgets of PMC7485093
(Philos Trans R Soc B 2020, doi:10.1098/rstb.2019.0685, drought 2018 in two rewetted fens, Hte and Zrk).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Sign convention, as the paper states it (Methods, "Data acquisition"): GEP carries a negative
sign, Reco is positive, and GEP was "calculated from the difference between NEE and Reco",
so annual NEE = GEP + Reco = Reco - |GEP|. Negative NEE = net uptake (sink).
Inputs from the text (Results, "Drought effects on carbon dioxide and methane exchange"), kg CO2 m-2 yr-1:
2018 GEP Hte -3.58, Zrk -3.72; +22% and +86% vs the 2014-2017 reference mean
2018 Reco Hte 3.19, Zrk 3.88; +35% and +56% vs the reference mean
2018 NEE Hte -0.39 ("0.18 kg m-2 lower [uptake] than in previous years");
Zrk -0.03 ("offset respiratory CO2 losses by 0.03")
2019 GEP Hte -2.56, Zrk -3.39; 2019 Reco Zrk 3.72; 2019 NEE Zrk +0.06 ("net CO2 source of 0.06")
Zrk earlier average NEE +0.34 (cited from ref. [14], published 2016)
Inputs from Figure 3 (publisher image rstb20190685-g3.jpg, 760 x 1349 px, PMC open-access copy):
pixel rows of the axis tick marks and of each cumulative curve at its last drawn column
(2018 solid lines: x = 368 (Hte) / 747 (Zrk); 2019 dashed lines: last dash, x = 366 / 741).
One pixel is 0.017 (GEP), 0.019 (Reco) and 0.009 (NEE) kg CO2 m-2.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs from the text
TXT = {
("Hte", 2018): dict(GEP=-3.58, Reco=3.19, NEE=-0.39),
("Zrk", 2018): dict(GEP=-3.72, Reco=3.88, NEE=-0.03),
("Hte", 2019): dict(GEP=-2.56, Reco=None, NEE=None),
("Zrk", 2019): dict(GEP=-3.39, Reco=3.72, NEE=+0.06),
}
PCT = {"Hte": dict(GEP=22, Reco=35), "Zrk": dict(GEP=86, Reco=56)}
ZRK_PRIOR_NEE = 0.34
# ------------------------------------------------------------------ inputs from Figure 3 (pixel rows)
# y-axis tick rows found in the image (left and right panels share them):
# GEP panels a,b: 0 at 61.5, -3 at 236.0 Reco panels c,d: 0 at 611.5, 4 at 396.5
# NEE panels e,f: +1 at 734.5, 0 at 842.0, -1 at 949.5
gep_px = lambda y: -(y - 61.5) / ((236.0 - 61.5) / 3)
reco_px = lambda y: (611.5 - y) / ((611.5 - 396.5) / 4)
nee_px = lambda y: (842.0 - y) / ((842.0 - 734.5) / 1.0)
PIX = { # curve rows at the last drawn column
("Hte", 2018): dict(GEP=270.5, Reco=440.5, NEE=886.0),
("Hte", 2019): dict(GEP=210.5, Reco=482.5, NEE=857.5),
("Zrk", 2018): dict(GEP=277.5, Reco=403.5, NEE=845.5),
("Zrk", 2019): dict(GEP=258.5, Reco=412.5, NEE=838.0),
}
FIG = {k: dict(GEP=gep_px(v["GEP"]), Reco=reco_px(v["Reco"]), NEE=nee_px(v["NEE"])) for k, v in PIX.items()}
# one-pixel reading error on each of the three values, added in absolute terms
READ_ERR = 3 / (236.0 - 61.5) + 4 / (611.5 - 396.5) + 1 / (842.0 - 734.5)
R = {}
print("=== 1. Closure of the text values: NEE = GEP + Reco ===")
for k, v in TXT.items():
if v["Reco"] is None or v["NEE"] is None:
continue
s = v["GEP"] + v["Reco"]
gap = s - v["NEE"]
R[k] = dict(sum=s, gap=gap, reco_needed=v["NEE"] - v["GEP"], gep_needed=v["NEE"] - v["Reco"])
print(f"{k[0]} {k[1]}: GEP + Reco = {v['GEP']:+.2f} + {v['Reco']:.2f} = {s:+.2f}; stated NEE {v['NEE']:+.2f}; "
f"gap {gap:+.2f}")
print(f" to give the stated NEE: Reco would have to be {R[k]['reco_needed']:.2f} "
f"(with GEP {v['GEP']:+.2f}) or |GEP| {-R[k]['gep_needed']:.2f} (with Reco {v['Reco']:.2f})")
print("Rounding: each value printed to 0.01, so GEP + Reco - NEE can be off by at most 3 x 0.005 = 0.015.")
lo, hi = 3.875 - 3.725, 3.885 - 3.715
print(f"Zrk 2018, GEP + Reco over the rounding range: {lo:+.2f} to {hi:+.2f}; Zrk 2019: "
f"{3.715 - 3.395:+.2f} to {3.725 - 3.385:+.2f}")
print("\n=== 2. Reference-period means implied by the stated percentage increases ===")
ref = {}
for site in ("Hte", "Zrk"):
g = -TXT[(site, 2018)]["GEP"] / (1 + PCT[site]["GEP"] / 100)
r = TXT[(site, 2018)]["Reco"] / (1 + PCT[site]["Reco"] / 100)
ref[site] = dict(GEP=g, Reco=r, NEE=r - g)
print(f"{site}: reference |GEP| = {-TXT[(site, 2018)]['GEP']:.2f}/{1 + PCT[site]['GEP'] / 100:.2f} = {g:.3f}; "
f"reference Reco = {TXT[(site, 2018)]['Reco']:.2f}/{1 + PCT[site]['Reco'] / 100:.2f} = {r:.3f}; "
f"reference NEE = {r - g:+.3f}")
d = TXT[("Hte", 2018)]["NEE"] - ref["Hte"]["NEE"]
print(f"Hte: 2018 NEE -0.39 vs reference {ref['Hte']['NEE']:+.3f} -> net uptake lower by {d:.3f} (text: 0.18)")
print(f"Zrk: 2018 |GEP| / reference = {-TXT[('Zrk', 2018)]['GEP'] / ref['Zrk']['GEP']:.2f} (text: 'almost doubled')")
alt_reco_ref = ref["Zrk"]["GEP"] + ZRK_PRIOR_NEE
print(f"Zrk: if the reference NEE were the +0.34 cited from [14], reference Reco = {alt_reco_ref:.3f}, "
f"and +56% gives 2018 Reco = {alt_reco_ref * 1.56:.2f}; "
f"Reco 3.88 would then be +{(3.88 / alt_reco_ref - 1) * 100:.0f}%")
print(f"Zrk: Reco needed for NEE -0.03 ({R[('Zrk', 2018)]['reco_needed']:.2f}) is "
f"+{(R[('Zrk', 2018)]['reco_needed'] / ref['Zrk']['Reco'] - 1) * 100:.0f}% over the percentage-implied "
f"reference Reco {ref['Zrk']['Reco']:.2f}")
print("\n=== 3. End-of-year values read from Figure 3 ===")
print(f"(one-pixel reading error, summed over GEP, Reco and NEE: +/-{READ_ERR:.3f})")
for k, v in FIG.items():
s = v["GEP"] + v["Reco"]
v["sum"], v["gap"] = s, s - v["NEE"]
print(f"{k[0]} {k[1]}: GEP {v['GEP']:+.2f}, Reco {v['Reco']:+.2f}, GEP + Reco {s:+.2f}; "
f"NEE curve {v['NEE']:+.2f}; non-closure {v['gap']:+.2f}")
print("(2019 dashed lines are read at their last dash, about one to two weeks before 31 December)")
print("\n=== 4. Zrk 2019 against the earlier Zrk average ===")
print(f"stated 2019 NEE {TXT[('Zrk', 2019)]['NEE']:+.2f} vs earlier average +{ZRK_PRIOR_NEE:.2f}: "
f"{ZRK_PRIOR_NEE - TXT[('Zrk', 2019)]['NEE']:.2f} lower")
print(f"GEP + Reco 2019 {R[('Zrk', 2019)]['sum']:+.2f} vs earlier average +{ZRK_PRIOR_NEE:.2f}: "
f"{ZRK_PRIOR_NEE - R[('Zrk', 2019)]['sum']:.2f} lower; vs percentage-implied reference "
f"{ref['Zrk']['NEE']:+.2f}: {ref['Zrk']['NEE'] - R[('Zrk', 2019)]['sum']:.2f} lower")
# ------------------------------------------------------------------ figures
BG, INK, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK, "ytick.color": INK, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def f2(x):
"""Signed two-decimal label with a true minus sign and no '-0.00'."""
x = 0.0 if abs(x) < 0.005 else x
return f"{x:+.2f}".replace("-", "\u2212") if x else "0.00"
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", **kw)
plt.close(fig)
def style(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
# Figure 1: stated NEE vs GEP + Reco from the stated sums, per site-year
rows = [("Hte", 2018), ("Zrk", 2018), ("Zrk", 2019)]
labels = {("Hte", 2018): "Hte 2018\n(closes)", ("Zrk", 2018): "Zrk 2018", ("Zrk", 2019): "Zrk 2019"}
fig, ax = plt.subplots(figsize=(7.2, 3.5))
ys = [2, 1, 0]
for y, k in zip(ys, rows):
rep, rec = TXT[k]["NEE"], R[k]["sum"]
ax.plot([rep, rec], [y, y], color=SEC, lw=2, zorder=1)
ax.plot(rep, y, "o", ms=10, color=ORANGE, zorder=3)
closes = abs(rec - rep) <= 0.02
ax.plot(rec, y, "o", ms=16 if closes else 10, color=BLUE, zorder=2 if closes else 4,
mfc="none" if closes else BLUE, mew=2)
if abs(rec - rep) > 0.02:
ax.annotate(f2(rep), (rep, y), xytext=(0, 10), textcoords="offset points", ha="center",
fontsize=10, color=ORANGE)
ax.annotate(f2(rec), (rec, y), xytext=(0, 10), textcoords="offset points", ha="center",
fontsize=10, color=BLUE)
ax.annotate(f"gap {f2(rec - rep)}", ((rep + rec) / 2, y), xytext=(0, -17), textcoords="offset points",
ha="center", fontsize=10, color=SEC)
else:
ax.annotate(f"{f2(rep)} = {TXT[k]['Reco']:.2f} − {-TXT[k]['GEP']:.2f}", (rep, y), xytext=(0, 13),
textcoords="offset points", ha="center", fontsize=10, color=INK)
ax.axvline(0, color=INK, lw=1)
ax.axvline(ZRK_PRIOR_NEE, color=SEC, lw=1, ls=":")
ax.text(ZRK_PRIOR_NEE + 0.01, 2.55, "Zrk earlier average\n+0.34 (cited, ref. [14])", fontsize=10, color=SEC,
va="top", ha="left")
ax.text(-0.02, -0.75, "← net sink", ha="right", va="center", fontsize=10, color=SEC)
ax.text(0.02, -0.75, "net source →", ha="left", va="center", fontsize=10, color=SEC)
ax.set_yticks(ys)
ax.set_yticklabels([labels[k] for k in rows])
ax.set_ylim(-1.0, 2.65)
ax.set_xlim(-0.6, 0.6)
ax.set_xlabel("Annual net CO$_2$ balance, NEE (kg CO$_2$ m$^{-2}$)")
style(ax)
ax.legend(handles=[
Line2D([], [], marker="o", ls="", ms=10, color=ORANGE, label="NEE stated in the paper"),
Line2D([], [], marker="o", ls="", ms=10, color=BLUE, label="Reco − |GEP| from the paper's stated sums"),
], loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: non-closure (GEP + Reco - NEE) from the text and from the Figure 3 curves
order = [("Hte", 2018), ("Hte", 2019), ("Zrk", 2018), ("Zrk", 2019)]
fig, ax = plt.subplots(figsize=(7.2, 3.3))
ys = [3, 2, 1, 0]
ax.axvspan(-0.015, 0.015, color=GRID, alpha=0.9, lw=0, zorder=0)
ax.text(0.022, 3.55, "shaded: rounding limit ±0.015", ha="left", va="bottom", fontsize=10, color=SEC)
for y, k in zip(ys, order):
if k in R:
ax.plot(R[k]["gap"], y + 0.12, "o", ms=10, color=BLUE, zorder=3)
ax.annotate(f2(R[k]['gap']), (R[k]["gap"], y + 0.12), xytext=(10, 0), textcoords="offset points",
va="center", fontsize=10, color=BLUE)
g = FIG[k]["gap"]
ax.errorbar(g, y - 0.12, xerr=READ_ERR, fmt="o", ms=9, mfc=BG, mec=BLUE, mew=2, ecolor=BLUE, elinewidth=1.5,
capsize=0, zorder=3)
ax.annotate(f2(g), (g + READ_ERR, y - 0.12), xytext=(6, 0), textcoords="offset points", va="center",
fontsize=10, color=SEC)
ax.axvline(0, color=INK, lw=1)
ax.set_yticks(ys)
ax.set_yticklabels([f"{s} {yr}" + ("\n(no NEE in text)" if (s, yr) not in R else "") for s, yr in order])
ax.set_ylim(-0.6, 3.8)
ax.set_xlim(-0.12, 0.42)
ax.set_xlabel("GEP + Reco − NEE, annual sums (kg CO$_2$ m$^{-2}$); 0 = the budget closes")
style(ax)
ax.legend(handles=[
Line2D([], [], marker="o", ls="", ms=10, color=BLUE, label="from the sums stated in the text"),
Line2D([], [], marker="o", ls="-", lw=1.5, ms=9, mfc=BG, mec=BLUE, mew=2, color=BLUE,
label="from the Figure 3 curve ends (±1 px per curve)"),
], loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Settlement ANOVA reports F(12,80) = 0.771 with p < 0.001; that F gives p = 0.68
Proc Biol Sci (2025) · doi:10.1098/rspb.2025.0729 · PMC12187397
Summary
The Results report the host × tile interaction for zygote settlement as "ANOVA, F12,80 = 0.771, p < 0.001". The degrees of freedom match the stated 4 × 5 crossed design with n = 5 tiles per combination. On (12, 80) df, however, F = 0.771 has an upper-tail p of 0.678, and p < 0.001 would need F ≥ 3.16. Supplementary Table S6 prints the same F, and that F equals the table's own mean-square ratio (11000 / 14271), so the inconsistent entry is the p-value, not the F. The same table also prints p < 0.001 for the tile main effect, F(4,80) = 1.638, which gives p = 0.173. This is the only reported omnibus test of the settlement experiment, and the abstract and Discussion describe an "interactive effect" of host- and benthic-associated microbiota on settlement.
What the paper reports
All quotations are verbatim from the article as archived in PMC (checked with src/show.py), except the Table S6 excerpt, which is from the electronic supplementary material.
Methods, "Effect of disruption of host and benthic microbiota on zygote settlement", second paragraph:
Zygotes, released and fertilized by the macroalgal individuals used in the host experiment described above, were introduced to the tiles in an experimental design in which all host and tile treatments were crossed (20 treatment combinations; n = 5 tiles and females per treatment combination).
Methods, "Statistical analyses":
to test for the effect of host and tile microbial disruption on zygote settlement we used a two-factor ANOVA (host and tile factors, fixed, crossed, 4 and 5 levels, respectively) using the GAD package in R (v.4.0.3).
Post hoc contrasts (emmeans R package) were used to examine significant main effects or interactions, as appropriate.
Results, "Effect of microbial disruption on settlement of zygotes", first sentence:
Disruption to the host- and tile-associated microbiota caused significant, but complex and interactive, effects on the settlement of Hormosira’s zygotes (ANOVA, F12,80 = 0.771, p < 0.001, electronic supplementary material, table S6, figure 4).
Same section, second paragraph:
Furthermore, post hoc tests showed that all tiles where the microbiota was disrupted by autoclaving had significantly lower levels of settlement (by approximately 7%) than those that were not autoclaved, which was generally consistent across all host treatments (electronic supplementary material, table S6).
Abstract:
Disruption of host- and benthic-associated microbiota, in combination, caused a significant decrease in settlement of H. banksii zygotes, with the combined disruption having the greatest impact on settlement success. Our results demonstrate the importance of host-associated microbiota in macroalgal reproduction and an interactive effect of host- and benthic-associated microbiota on settlement
Discussion, first paragraph:
Furthermore, microbiota associated with both the adult host and the benthic substratum had a strong interactive effect on zygote settlement and morphogenesis.
Electronic supplementary material (rspb20250729_si_001.docx), Supplementary Table 6, ANOVA rows (columns Df | Sum of Sq | Mean Sq | F | P-value):
Host Treatment | 3 | 34724483 | 11574828 | 811.063 | <0.001
Tile Treatment | 4 | 93486 | 23371 | 1.638 | <0.001
Host Treatment * Tile Treatment | 12 | 132002 | 11000 | 0.771 | <0.001
Residual | 80 | 1213050 | 14271 | |
What the paper's own numbers show
Method. I derived the degrees of freedom from the design stated in the Methods (4 host levels × 5 tile levels × 5 tiles = 100 observations) and compared them with the printed df. I then computed the upper-tail p-value of each printed F on its printed df with scipy.stats.f. For each row of Table S6 I also checked SS / df against the printed mean square and MS / MSresidual against the printed F. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| df of the host × tile interaction; residual df | Results; Table S6; design in Methods | 12; 80 | (4 − 1)(5 − 1) = 12; 100 − 20 = 80 (match) |
| p for host × tile, F(12,80) = 0.771 | Results, settlement; Table S6 | < 0.001 | 0.678 (upper tail); lower tail 0.322 |
| F(12,80) needed for p < 0.001 (for p < 0.05) | derived | — | ≥ 3.162 (≥ 1.875) |
| Host × tile F from Table S6 mean squares | Table S6 | 0.771 | 11000 / 14271 = 0.771 (match) |
| p for tile, F(4,80) = 1.638 | Table S6 | < 0.001 | 0.173; p < 0.001 would need F ≥ 5.123 |
| Tile F from Table S6 mean squares | Table S6 | 1.638 | 23371 / 14271 = 1.638 (match) |
| p for host, F(3,80) = 811.063 | Table S6 | < 0.001 | 9 × 10-60 (consistent) |
| Residual mean square | Table S6 | 14271 | 1213050 / 80 = 15163.1 (the printed value equals SS / 85.00) |
| F and p with residual MS = SS / 80 | derived from Table S6 | — | host × tile F = 0.725, p = 0.723; tile F = 1.541, p = 0.198; host F = 763.4, p = 1 × 10-58 |
| Same F = 0.771 on the other df of this design | derived | — | (3,80): p = 0.514; (4,80): p = 0.547; (19,80): p = 0.733 |
| Reproductive output, F(3,96) = 812.7 | Results, reproductive output | p < 0.001 | df match 4 groups × 25; p = 4.5 × 10-68 (consistent) |
The df, the F values and the mean squares of Table S6 agree with one another, so the F of 0.771 is what the tabulated sums of squares produce. The entries that do not fit are the p-values for the interaction and for the tile main effect. An F of 0.771 lies below the mean of the F(12,80) distribution (1.026) and below its median (0.953); it is a value expected when the interaction has no effect at all. The residual mean square printed in Table S6 does not equal its own SS / df. Using SS / 80 makes the residual larger and both F values smaller, so it widens the gap rather than closing it.
Figure
Figure 1. p-values on one log axis for the three terms of the settlement ANOVA in Table S6. Orange squares with leftward arrows are the printed values, all "< 0.001" (upper bounds). Filled blue circles are the upper-tail p-values recomputed from the printed F and df. Open blue circles use the residual mean square recomputed as SS / 80 = 15163.1 in place of the printed 14271. The dashed line marks α = 0.05 and the dotted line 0.001. For the host term the recomputed p (9 × 10-60) is off the left edge of the axis and agrees with the printed bound. For the host × tile interaction (p = 0.678) and the tile main effect (p = 0.173) the recomputed values lie above 0.05 whichever residual mean square is used.
Figure 2. Density of the F(12, 80) distribution, which is the reference distribution for the host × tile test when there is no interaction. The orange line is the printed F = 0.771; the shaded area to its right, 0.678, is the recomputed p-value. The dashed blue line at 3.16 is the smallest F that would give p < 0.001 on these df. The figure shows that the printed F is an unremarkable value under the null hypothesis of no interaction, not one in the extreme tail.
Why it matters
- The interaction claim rests on this test. The Results present the host × tile ANOVA as evidence of "significant, but complex and interactive, effects" on settlement, and the abstract and Discussion state "an interactive effect" and "a strong interactive effect" of host- and benthic-associated microbiota. The printed F and Table S6 sums of squares instead give p = 0.678 for the interaction, which does not reject additivity.
- The tile main effect is not significant on the printed numbers either. Table S6 gives F(4,80) = 1.638 for tile treatment, p = 0.173. The paper's statements that autoclaved tiles had lower settlement, and that AB2 tiles mattered only in combination with host disruption, are drawn from post hoc contrasts. By the paper's own rule, post hoc contrasts "were used to examine significant main effects or interactions, as appropriate"; on the printed numbers neither the tile effect nor the interaction meets that condition.
- The host effect is unaffected. Host treatment, F(3,80) = 811.063, has p ≈ 9 × 10-60, consistent with the printed bound. The finding that AB2-treated hosts gave lower settlement, which Table S6 reports as consistent across every tile treatment, is supported by this term. The separate reproductive-output ANOVA, F(3,96) = 812.7, is also internally consistent.
- What changes. If the Table S6 sums of squares are correct, settlement differs by host treatment, but the printed ANOVA gives no evidence for a tile effect or a host × tile interaction. The headline that combined host and benthic disruption acts interactively would then not be supported by the reported test. If instead the p-values are correct, then the F values, mean squares and sums of squares for the tile and interaction terms are all misprinted, and the analysis would need to be reported again. Either way, the printed statistics cannot all be right.
Explanations considered
- The df belong to a different term. Not supported. (12, 80) are exactly the interaction and residual df of the stated 4 × 5 design with n = 5, and Table S6 assigns 12 df to the interaction. Putting F = 0.771 on the host (3,80), tile (4,80) or whole-model (19,80) df gives p between 0.51 and 0.73.
- A lower-tail or one-sided reading. Not able to explain it. The lower-tail p of F = 0.771 on (12,80) is 0.322. No convention for an ANOVA F test gives p < 0.001 for F below 1 on these df.
- A decimal slip in F (for example 7.71, which would give p ≈ 3 × 10-9). Ruled out by Table S6. The printed F equals MSinteraction / MSresidual = 11000 / 14271 = 0.771, and the mean square equals SS / df = 132002 / 12. The F is the value the table's sums of squares produce, so the p-value is the inconsistent entry.
- A pseudo-F or permutation test. Not applicable. The Methods describe a parametric two-factor ANOVA in the GAD package, and the paper reports PERMANOVA pseudo-F values separately for the community analyses (Tables S1 and S2). In any case, a permutation p-value for a statistic below its null mean would not fall below 0.001.
- The residual mean square. Not able to explain it. The printed residual MS (14271) is SS / 85 rather than SS / 80; correcting it lowers the interaction and tile F values (to 0.725 and 1.541) and raises their p-values (to 0.723 and 0.198).
- A text-conversion or typesetting artifact. Ruled out. The PMC page, the Europe PMC full-text XML (
F<sub>12,80</sub> = 0.771, <italic>p</italic> < 0.001) and the publisher's supplementary file all carry the same values. - A correction already issued. None found (see Status).
- Side observations, not part of the main finding.
- Supplementary Table 5 (reproductive output) prints Sum of Sq 347244 and Mean Sq 11574 for host (SS / 3 = 115748.0) and 143853 and 14243 for the residual (SS / 96 = 1498.5); neither the printed mean-square ratio (0.813) nor the SS-based ratio (77.2) gives the printed F of 812.7. The main-text F(3,96) = 812.7 and its p-value agree with each other.
- Table S6 is captioned as an analysis of the number of zygotes that settled, and Figure 4 states that 200 eggs were added per tile. The largest total sum of squares 100 counts between 0 and 200 can produce is 1,000,000, while the printed host SS is 34,724,483. The table may therefore have been computed on a differently scaled response, such as the predicted number of settled eggs described in the Methods; this cannot be checked without the data, and it does not affect the F-to-p inconsistency.
- The Results cite "table S2" for the host post hoc contrasts; Table S2 is the PERMANOVA of tile communities, and these contrasts appear in Table S6.
Status
- Version of record. The publisher's article page and PDF (royalsocietypublishing.org, reached through doi.org) returned HTTP 403 to automated requests. The published text was checked in three other copies: the live PMC page (Results section 3c), the Europe PMC full-text XML, and the publisher's own supplementary file. All agree with the passages quoted here. Every quotation from the article was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The electronic supplementary material (Royal Society figshare collection 10.6084/m9.figshare.c.7834312, one file, rspb20250729_si_001.docx, figshare file 54729154, record modified 2026-02-17) was downloaded and parsed; its Table 6 is quoted above and confirms the main-text numbers without resolving the discrepancy. The Dryad dataset doi:10.5061/dryad.1rn8pk13s lists Experiment_3_Response_Variables_200.xlsx, an R script for plots and statistics, and a README, but every file download returned HTTP 401 or 403, so the ANOVA could not be recomputed from raw data.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by fields; its relations list only peer-review items. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC12187397; Europe PMC full-text XML for PMC12187397; Royal Society figshare file 54729154 (rspb20250729_si_001.docx); Dryad dataset doi:10.5061/dryad.1rn8pk13s (file listing only); Crossref; Europe PMC; PubPeer; https://doi.org/10.1098/rspb.2025.0729.
Suggested correction
The authors may wish to re-run the two-factor settlement ANOVA from the archived data and report the full table. If the sums of squares in Table S6 are correct, the first Results sentence on settlement would read, for example,
(ANOVA, host × tile F12,80 = 0.771, p = 0.68; host F3,80 = 811.1, p < 0.001; tile F4,80 = 1.64, p = 0.17)
with the P-value column of Table S6 corrected to match (and the residual mean square to SS / 80 = 15163, with F values recomputed from it). The abstract's and Discussion's statements of an "interactive effect" of host- and benthic-associated microbiota on settlement, and the post hoc interpretation of tile and interaction contrasts, would then need to be revised in line with the corrected test. If instead the reported p-values are confirmed, the F values, mean squares and sums of squares for the tile and interaction terms would need correcting. The authors may also wish to check the mean squares and F in Supplementary Table 5 and the "table S2" citation in the settlement Results.
Recomputation script (python recompute.py.txt)
"""Recompute the settlement ANOVA of PMC12187397
(Proc Biol Sci 2025, doi:10.1098/rspb.2025.0729,
"Disruption of host-associated and benthic microbiota affects reproductive output and
settlement of a habitat-forming macroalga").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC and from its electronic supplementary
material (rspb20250729_si_001.docx, Royal Society figshare, file 54729154):
Methods: two-factor ANOVA, host and tile fixed and crossed, 4 and 5 levels;
20 treatment combinations, n = 5 tiles per combination.
Results, settlement: "ANOVA, F12,80 = 0.771, p < 0.001".
Results, reproductive output: "ANOVA, F3,96 = 812.7, p < 0.001".
Table S6 (settled zygotes): rows df, Sum of Sq, Mean Sq, F, P-value.
Table S5 (reproductive output): rows df, Sum of Sq, Mean Sq, F, P-value.
Figure 4 legend: settlement out of 200 eggs added per tile.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
H, T, n = 4, 5, 5 # Methods
F_TEXT, DF_TEXT = 0.771, (12, 80) # Results, settlement
F_REPRO, DF_REPRO = 812.7, (3, 96) # Results, reproductive output
S6 = { # term: (df, SS, MS, F) ; every printed P-value is "<0.001"
"Host": (3, 34724483, 11574828, 811.063),
"Tile": (4, 93486, 23371, 1.638),
"Host x Tile": (12, 132002, 11000, 0.771),
}
S6_RES = (80, 1213050, 14271) # Residual: df, SS, MS
S5 = {"Host": (3, 347244, 11574, 812.7)}
S5_RES = (96, 143853, 14243)
EGGS_PER_TILE = 200 # Figure 4 legend
P_REPORTED = 0.001 # all printed as "< 0.001"
# ---------------------------------------------------------------- 1. design
N = H * T * n
df_h, df_t, df_ht, df_res = H - 1, T - 1, (H - 1) * (T - 1), N - H * T
print("=== 1. Degrees of freedom implied by the stated design ===")
print(f"N = {H}*{T}*{n} = {N}; host df = {df_h}; tile df = {df_t}; "
f"host x tile df = {df_ht}; residual df = {N} - {H*T} = {df_res}")
print(f"whole-model df = {H*T-1}")
assert (df_ht, df_res) == DF_TEXT
# ---------------------------------------------------------------- 2. the printed pair
print("\n=== 2. p for the printed F12,80 = 0.771 ===")
p_up = stats.f.sf(F_TEXT, *DF_TEXT)
p_lo = stats.f.cdf(F_TEXT, *DF_TEXT)
c001 = stats.f.isf(0.001, *DF_TEXT)
c05 = stats.f.isf(0.05, *DF_TEXT)
print(f"upper-tail p = {p_up:.4f}; lower-tail p = {p_lo:.4f}")
print(f"critical F(12,80): alpha 0.05 -> {c05:.3f}; alpha 0.001 -> {c001:.3f}")
print(f"mean of F(12,80) = 80/78 = {stats.f.mean(*DF_TEXT):.3f}; "
f"median = {stats.f.median(*DF_TEXT):.3f}")
print("same F on other df of this design:")
for d1 in (df_h, df_t, H * T - 1):
print(f" F({d1},80) = 0.771 -> p = {stats.f.sf(F_TEXT, d1, 80):.4f}")
for alt in (7.71, 77.1):
print(f" hypothetical F(12,80) = {alt} -> p = {stats.f.sf(alt, 12, 80):.2g}")
# ---------------------------------------------------------------- 3. Table S6
print("\n=== 3. Table S6, recomputed row by row ===")
res_df, res_ss, res_ms = S6_RES
res_ms_80 = res_ss / res_df
print(f"Residual: SS/df = {res_ss}/{res_df} = {res_ms_80:.1f}; printed MS = {res_ms}; "
f"SS/printed MS = {res_ss/res_ms:.2f}")
rows = {}
for term, (df, ss, ms, f) in S6.items():
ms_r = ss / df
f_r = ms / res_ms
p_r = stats.f.sf(f, df, res_df)
f_80 = ms_r / res_ms_80
p_80 = stats.f.sf(f_80, df, res_df)
rows[term] = dict(df=df, f=f, p=p_r, f80=f_80, p80=p_80)
print(f"{term:12s} df={df:2d} SS/df={ms_r:12.1f} (printed MS {ms}) "
f"MS/MS_res={f_r:9.3f} (printed F {f}) p(printed F)={p_r:.3g} "
f"| with MS_res=SS/80: F={f_80:.3f}, p={p_80:.3g}")
print("terms whose printed F would need to be at least this for p < 0.001:")
for term, (df, *_r) in S6.items():
print(f" {term:12s} F({df},80) >= {stats.f.isf(0.001, df, 80):.3f}")
# ---------------------------------------------------------------- 4. main-text reproductive ANOVA
print("\n=== 4. Reproductive output, main text F3,96 = 812.7 ===")
print(f"df check: 4 groups x 25 = 100 -> ({4-1}, {100-4}); "
f"p = {stats.f.sf(F_REPRO, *DF_REPRO):.2g} (consistent with p < 0.001)")
df5, ss5, ms5, f5 = S5["Host"]
r5df, r5ss, r5ms = S5_RES
print(f"Table S5 side check: host SS/df = {ss5/df5:.1f} (printed MS {ms5}); "
f"residual SS/df = {r5ss/r5df:.1f} (printed MS {r5ms}); "
f"printed MS ratio = {ms5/r5ms:.3f} (printed F {f5}); "
f"SS-based F = {(ss5/df5)/(r5ss/r5df):.1f}")
# ---------------------------------------------------------------- 5. scale of the S6 sums of squares
print("\n=== 5. Scale of Table S6 sums of squares (side observation) ===")
max_total_ss = N * (EGGS_PER_TILE / 2) ** 2
print(f"largest possible total SS for {N} values in [0, {EGGS_PER_TILE}] = {max_total_ss:.0f}; "
f"printed host SS = {S6['Host'][1]} ({S6['Host'][1]/max_total_ss:.1f} x that bound)")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans",
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# Figure 1: printed vs recomputed p, one log axis
XMIN, XMAX = 1e-5, 1.0
BOX = dict(facecolor=BG, edgecolor="none", pad=1.0)
items = [
("Host × Tile\nF(12,80) = 0.771\nmain text and Table S6", rows["Host x Tile"]),
("Tile\nF(4,80) = 1.638\nTable S6", rows["Tile"]),
("Host\nF(3,80) = 811.063\nTable S6", rows["Host"]),
]
fig, ax = plt.subplots(figsize=(7.2, 4.8))
ax.set_xscale("log")
ax.set_xlim(XMIN, XMAX)
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
ax.axvline(0.05, color=INK2, lw=0.8, ls="--", zorder=0)
ax.text(0.05, 3.62, "α = 0.05", ha="center", va="bottom", fontsize=10, color=INK2)
ax.axvline(P_REPORTED, color=INK2, lw=0.8, ls=":", zorder=0)
ax.text(P_REPORTED, 3.62, "0.001", ha="center", va="bottom", fontsize=10, color=INK2)
ys = [3, 2, 1]
for y, (lab, r) in zip(ys, items):
yo, yb, yw = y + 0.25, y - 0.02, y - 0.29
# printed: "< 0.001" drawn as an upper bound with a leftward arrow
ax.annotate("", xy=(P_REPORTED / 12, yo), xytext=(P_REPORTED, yo),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2))
ax.plot([P_REPORTED], [yo], marker="s", ms=9, color=ORANGE, ls="none")
ax.text(P_REPORTED * 1.4, yo, "printed p < 0.001", ha="left", va="center",
fontsize=10, color=ORANGE, bbox=BOX)
if r["p"] >= XMIN:
ax.plot([r["p"]], [yb], marker="o", ms=9, color=BLUE, ls="none")
ax.text(r["p"] / 1.4, yb, f"p = {r['p']:.3f}", ha="right", va="center",
fontsize=10, color=BLUE, bbox=BOX)
ax.plot([r["p80"]], [yw], marker="o", ms=9, mfc=BG, mec=BLUE, mew=2, ls="none")
ax.text(r["p80"] / 1.4, yw, f"p = {r['p80']:.3f}", ha="right", va="center",
fontsize=10, color=BLUE, bbox=BOX)
else:
ax.annotate("", xy=(XMIN, yb), xytext=(XMIN * 8, yb),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2))
ax.text(XMIN * 9, yb, f"p = {r['p']:.0e} (off scale; consistent with < 0.001)",
ha="left", va="center", fontsize=10, color=BLUE, bbox=BOX)
ax.set_yticks(ys)
ax.set_yticklabels([i[0] for i in items], fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_ylim(0.5, 3.9)
ax.set_xlabel("p-value (log scale)")
handles = [
Line2D([], [], marker="s", ms=9, color=ORANGE, ls="none", label="printed in paper (< 0.001)"),
Line2D([], [], marker="o", ms=9, color=BLUE, ls="none", label="recomputed from printed F and df"),
Line2D([], [], marker="o", ms=9, mfc=BG, mec=BLUE, mew=2, ls="none",
label="recomputed with residual MS = SS/80"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.42, -0.16), ncol=2,
frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: where F = 0.771 sits in the F(12,80) reference distribution
x = np.linspace(0, 4.2, 800)
pdf = stats.f.pdf(x, *DF_TEXT)
fig, ax = plt.subplots(figsize=(7.2, 3.4))
for s_ in ("top", "right"):
ax.spines[s_].set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.plot(x, pdf, color=INK2, lw=2)
ax.fill_between(x, 0, pdf, where=x >= F_TEXT, color=BLUE, alpha=0.16, lw=0)
ytop = pdf.max()
ax.axvline(F_TEXT, color=ORANGE, lw=2)
ax.text(F_TEXT + 0.05, ytop * 1.22, f"printed F = {F_TEXT}", ha="left", va="top",
fontsize=10, color=ORANGE)
ax.text(1.55, ytop * 0.62, f"shaded area = recomputed p\nfor F ≥ {F_TEXT}: {p_up:.3f}",
ha="left", va="center", fontsize=10, color=BLUE, bbox=dict(facecolor=BG, edgecolor="none"))
ax.axvline(c001, color=BLUE, lw=2, ls="--")
ax.text(c001 - 0.05, ytop * 0.3, f"F needed for the\nprinted p < 0.001:\n≥ {c001:.2f}",
ha="right", va="center", fontsize=10, color=BLUE, bbox=dict(facecolor=BG, edgecolor="none"))
ax.set_xlim(0, 4.2)
ax.set_ylim(0, ytop * 1.25)
ax.set_xlabel("F statistic")
ax.set_ylabel("density, F(12, 80)")
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Abstract and Table 1 report mean perivascular flow rate ten times below the per-mouse values' mean
Proc Natl Acad Sci U S A (2023) · doi:10.1073/pnas.2217744120 · PMC10083563
Summary
The abstract and the summary rows of Table 1 give the volume flow rate in the perivascular spaces of N = 4 mice as 2.22 ± 1.983 × 103 µm3/s. The four per-mouse values in the same table (2.18 × 104, 5.02 × 104, 1.13 × 104 and 5.57 × 103 µm3/s) have a mean of 2.222 × 104 and a sample SD of 1.983 × 104 µm3/s. The mantissas match exactly and only the power of ten differs, so the printed mean and spread are each ten times too small. The printed mean is also 2.5 times smaller than the smallest of the four values it averages. Flow rate is one of the four headline quantities in the abstract, and the paper offers these numbers as "immediately useful in modeling". The per-mouse values and the Discussion's whole-brain estimate are on the correct 104 scale.
What the paper reports
The PMC text flattens superscripts ("× 10 4"); below they are written as superscripts, as in the Europe PMC JATS markup and the typeset PDF. Wording is otherwise verbatim.
Abstract, last part:
In PVSs of N = 4 wild-type mice we find mean flow speed 16.33 ± 11.09 µm/s, volume flow rate 2.22 ± 1.983 × 103 µm3/s, axial pressure gradient ( - 2.75 ± 2.01)×10-4 Pa/µm (-2.07 ± 1.51 mmHg/m), and wall shear stress (3.00 ± 1.45)×10-3 Pa (all mean ± SE).
Table 1, "Quantities of interest for each mouse", excerpt of the Mouse and Q (µm3/s) columns:
1 | 2.18 × 104
2 | 5.02 × 104
3 | 1.13 × 104
4 | 5.57 × 103
mean | 2.22 × 103
σ | 1.983 × 103
Table 1 footer:
All values except the shear rate are temporally and spatially averaged. The shear rate is the median of all points in time and space. The values reported for mouse 1 are the average of the predictions from G1 to G6.
Results, section "Volume Flow Rate":
Among N = 4 mice, the time-averaged volume flow rate at the inlet ranged from 0.5 to 5 × 104 µm3/s, and the difference between the time-averaged flow rates at inlet and outlet ranged from -12 to 33%, as reported in Table 1.
Discussion, fourth paragraph:
Using AIV, we infer a volume flow rate of around 2.25 × 104 µm3/s (or 1.35 × 10-3 µL/min) in one side of the pial PVS in the territory of the middle cerebral artery of mouse 1.
This half of the PVS has a cross-sectional area of 1.1 × 103 µm2. Ray et al. estimate that the total periarterial cross-sectional area in the murine brain is 0.2 mm2, or approximately 182 times larger than the section we measured (5). If we assume that the flow rate in the section we measure is typical, we can approximate a total PVS flow rate of 0.25 µL/min, or 0.625 µL/(g-min) for a 0.4-g mouse brain, which agrees well with the estimates of 0.5µL/(g-min) inflow in mice from Ray et al. (5).
Discussion, last paragraph:
These numbers are immediately useful in modeling flow in PVSs
What the paper's own numbers show
Method. I took the four per-mouse rows of Table 1 and recomputed the mean and spread of every column, then compared them with the printed "mean" and "σ" rows. I also checked that the per-mouse Q values are really in µm3/s, in two ways that use other columns of the same table: dividing Q by the mean flow speed Q/A gives the cross-sectional area, and dividing the pressure gradient by Q gives the hydraulic resistance R. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Mean volume flow rate Q, N = 4 | Table 1 mean row; Abstract | 2.22 × 103 µm3/s | 2.222 × 104 µm3/s (sum 88,870 ÷ 4 = 22,217.5); ratio 10.01 |
| Spread of Q | Table 1 σ row; Abstract | 1.983 × 103 µm3/s | 1.983 × 104 (sample SD, n − 1); ratio 10.00 |
| SE of Q, if "SE" is meant | Abstract "(all mean ± SE)" | 1.983 × 103 | 9.91 × 103 (SD ÷ √4) |
| Smallest and largest per-mouse Q | Table 1 rows | 5.57 × 103 and 5.02 × 104 | the printed mean is 0.40 times the smallest value |
| Range of inlet Q | Results, "Volume Flow Rate" | 0.5 to 5 × 104 µm3/s | recomputed mean lies inside it; printed mean lies below it |
| Mouse 1 Q | Discussion vs Table 1 | 2.25 × 104 (= 1.35 × 10-3 µL/min) vs 2.18 × 104 | 2.25 × 104 µm3/s = 1.35 × 10-3 µL/min; same scale |
| Implied cross-section Q ÷ (Q/A) | Table 1; Discussion gives 1.1 × 103 µm2 | — | 1082, 1674, 1089, 1160 µm2 |
| R = |∂P/∂n| ÷ Q | Table 1 R column | 2.49 × 10-8, 6.47 × 10-9, 1.51 × 10-8, 1.30 × 10-8 | 2.45 × 10-8, 6.47 × 10-9, 1.50 × 10-8, 1.30 × 10-8 |
| Mean ± σ of the other columns | Table 1 mean and σ rows | e.g. Q/A 16.33 ± 11.09; ∂P/∂n −2.75 ± 2.01 × 10-4; R 1.49 × 10-8 ± 7.63 × 10-9; wall shear stress 3.00 ± 1.45 × 10-3 | all agree with the plain mean and sample SD of the four rows (e.g. 16.33 ± 11.09; −2.754 ± 2.013 × 10-4; 1.487 × 10-8 ± 7.63 × 10-9; 2.998 ± 1.453 × 10-3) |
The per-mouse Q values are self-consistent in µm3/s: the implied cross-sections of about 1.1 to 1.7 × 103 µm2 match the area the Discussion gives for mouse 1, and dividing the pressure gradient by Q reproduces the tabulated resistances to within 1.6%. The Q column is the only one whose summary rows disagree with its own entries, and the disagreement is exactly one power of ten in both rows. A mean of four positive values cannot be smaller than the smallest of them, so the printed 2.22 × 103 cannot be the mean of the printed rows.
Figure
Figure 1. Volume flow rate on a log scale. Top row: the four per-mouse values printed in Table 1 (open black circles, numbered by mouse); the grey band is the range stated in the Results text (0.5 to 5 × 104 µm3/s). Middle row (blue): the mean ± sample SD recomputed from those four values, 2.222 × 104 ± 1.983 × 104 µm3/s. Bottom row (orange): the mean ± σ printed in Table 1 and the abstract, 2.22 × 103 ± 1.983 × 103 µm3/s. The printed summary is shifted by a factor of 10.0 from the recomputed one and lies outside the range of its own inputs. The log scale is used because the error is a ratio; it does not exaggerate the difference.
Figure 2. The Discussion's scale-up (Q × 182 for total periarterial area, ÷ 0.4 g brain mass) applied to each value, in µL/(g·min) on a log scale. Open black circles are numbers printed in the Discussion: the cited prior estimate of 0.5 (dashed line) and the paper's own 0.625, which it derived from mouse 1. Applied to the recomputed mean (blue) the scale-up gives 0.607; applied to the printed mean (orange) it gives 0.061, about one-eighth of the prior estimate. The Discussion used mouse 1, not the mean, so its conclusion stands; the figure shows only that the abstract value, taken at face value, would not support it.
Why it matters
- A headline number is wrong by a factor of 10. Flow rate is one of four quantities the abstract reports for the N = 4 mice, and the paper describes these numbers as "immediately useful in modeling flow in PVSs". A reader or modeler who takes the mean from the abstract or from the Table 1 summary row gets a perivascular flow rate, and its spread, ten times too small.
- The printed value contradicts the paper's own comparison with prior work. The abstract says "Pressure gradients, flow rates, and resistances agree with prior predictions." The correct mean scales up to about 0.24 µL/min, or 0.61 µL/(g·min), close to the cited prior 0.5 µL/(g·min). The printed mean would scale up to about 0.024 µL/min, or 0.061 µL/(g·min) (Figure 2).
- The analysis itself is not affected. The per-mouse flow rates, the Results range, the Discussion's whole-brain estimate (0.25 µL/min, 0.625 µL/(g·min)), the hydraulic resistances (which reproduce from the per-mouse Q values) and the validation of the method were all computed on the correct 104 scale. No statistical test depends on the mean flow rate. The error is confined to the reported summary statistic.
- The dispersion label is also inaccurate. The abstract labels all four ± values "mean ± SE", but each equals the sample SD of the four mice (for example 11.09 for flow speed). The SEs would be half as large (5.55 µm/s for flow speed and 0.99 × 104 µm3/s for flow rate). Table 1 labels the same row "σ", which fits an SD.
Explanations considered
- A different unit for the summary rows. Not supported. The Q column header is µm3/s and covers every row, including mean and σ. No conversion factor of exactly 10 links any plausible pair of volume-flow units.
- The per-mouse values, not the mean, are the ones in error. Not supported. The per-mouse values agree with three independent statements in the paper: the Results range (0.5 to 5 × 104), the Discussion's mouse 1 value (2.25 × 104 µm3/s, which the paper converts correctly to 1.35 × 10-3 µL/min), and the Discussion's half-PVS area (1.1 × 103 µm2, matching Q ÷ (Q/A)). They also reproduce the tabulated resistances R = |∂P/∂n| ÷ Q.
- A different population, subset or statistic. Not supported. The footer describes only spatial and temporal averaging and how mouse 1 is formed from G1 to G6. Every other column's mean and σ rows equal the plain mean and sample SD of the same four rows, and the Q mantissas (2.222 and 1.983) match the recomputed mean and SD exactly. Only the exponent differs.
- Population SD, SE or another spread measure. Not able to explain it. The population SD (n) of Q is 1.717 × 104 and the SE is 9.91 × 103; neither gives 1.983 × 103. Only the sample SD divided by exactly 10 does.
- A text-conversion or typesetting artifact. Ruled out. In the typeset PNAS PDF (text spans read with PyMuPDF), the exponent after "1.983 × 10" in the abstract and after "2.22 × 10" and "1.983 × 10" in the Table 1 summary rows is a superscript "3", while the rows for mice 1 to 3 carry a superscript "4". The Europe PMC JATS XML has the same markup (
10<sup>3</sup>), and the PMC HTML page shows the same values. - A correction already issued. None found (see Status).
- Side observation, not part of the main finding. For mouse 3, the pressure gradient −1.70 × 10-4 Pa/µm converts to −1.275 mmHg/m, while Table 1 gives −1.23 (a 3.7% difference; the other three mice agree within 0.4%). The printed mmHg/m mean and σ (−2.07, 1.51) equal the converted Pa/µm mean and SD (−2.066, 1.510) rather than the mean of the mmHg/m column (−2.059, 1.524). These differences are small and do not affect any conclusion.
Status
- Version of record. The publisher's page (pnas.org, reached through doi.org) returned HTTP 403 to automated requests. I checked three other copies of the published text. (1) The typeset PNAS PDF, whose metadata identifies it as Proc. Natl. Acad. Sci. U.S.A. 2023, 120:e2217744120, as hosted on the corresponding laboratory's website: the abstract (p. 1), Table 1 (p. 5) and Discussion (p. 7) carry the exponents described above. (2) The Europe PMC full-text JATS XML. (3) The PMC HTML page. All three agree with the passages quoted here. Every quotation was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The SI Appendix (
pnas.2217744120.sapp.pdf, 18 pages, from the Europe PMC supplementary bundle, which also holds Movies S1–S5) has no per-mouse or mean flow-rate table; it covers methods, Reynolds and Womersley estimates and validation errors. The data record cited in the Data Availability statement (Zenodo 7723381) holds raw imaging and geometry for mouse 1 only, and cannot bear on a four-mouse mean, which is in any case fully determined by the four values printed in Table 1. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has empty update-to and updated-by fields, and the PubMed record (PMID 36989300) has no CommentsCorrections entries. A Crossref search for a correction notice with this title found none. The PubPeer API returned no record (HTTP 404) to an automated DOI query, and a web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC10083563; Europe PMC full-text XML and supplementary files for PMC10083563; typeset PNAS PDF; SI Appendix
pnas.2217744120.sapp.pdf; Zenodo record 7723381; Crossref; PubMed; Europe PMC; PubPeer; https://doi.org/10.1073/pnas.2217744120.
Suggested correction
In the abstract, replace
volume flow rate 2.22 ± 1.983 × 103 µm3/s
with
volume flow rate 2.22 ± 1.983 × 104 µm3/s
and in Table 1, change the Q entries of the mean row from 2.22 × 103 to 2.22 × 104 and of the σ row from 1.983 × 103 to 1.983 × 104. The authors may also wish to change "(all mean ± SE)" in the abstract to "(all mean ± SD)", since every ± value there equals the sample SD of the four mice. Alternatively they could report SEs, which for N = 4 are half the SDs (for flow rate, 0.99 × 104 µm3/s). They may also wish to check the mmHg/m entry for mouse 3 (−1.23) against its Pa/µm value (−1.70 × 10-4, which converts to −1.28).
Recomputation script (python recompute.py.txt)
"""Recompute the Table 1 summary rows of PMC10083563
(Proc Natl Acad Sci U S A 2023, doi:10.1073/pnas.2217744120,
"Artificial intelligence velocimetry reveals in vivo flow rates, pressure gradients,
and shear stresses in murine perivascular flows").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 1, rows for mice 1-4 (all nine numeric columns), and its "mean" and "sigma" rows.
Abstract: "volume flow rate 2.22 ± 1.983 × 10^3 µm3/s ... (all mean ± SE)".
Results, "Volume Flow Rate": inlet flow rate "ranged from 0.5 to 5 × 10^4 µm3/s".
Discussion: "around 2.25 × 10^4 µm3/s (or 1.35 × 10^-3 µL/min)" for mouse 1; half-PVS area
1.1 × 10^3 µm2; total periarterial area "approximately 182 times larger"; total
"0.25 µL/min, or 0.625 µL/(g-min) for a 0.4-g mouse brain"; prior estimate 0.5 µL/(g-min).
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Table 1, mice 1-4)
COLS = {
# name: (unit, per-mouse values, printed mean, printed sigma)
"PTV mean speed": ("µm/s", [19.76, 14.76, 7.78, 6.38], 12.17, 6.25),
"Q/A (mean flow speed)": ("µm/s", [20.15, 29.98, 10.38, 4.80], 16.33, 11.09),
"Q (volume flow rate)": ("µm3/s", [2.18e4, 5.02e4, 1.13e4, 5.57e3], 2.22e3, 1.983e3),
"(Qin-Qout)/Qout": ("", [-0.12, 0.33, 0.00, 0.33], 0.14, 0.23),
"shear rate": ("1/s", [4.53, 2.12, 1.69, 1.00], 2.33, 1.53),
"dP/dn": ("Pa/µm", [-5.34e-4, -3.25e-4, -1.70e-4, -7.26e-5], -2.75e-4, 2.01e-4),
"dP/dn (mmHg/m)": ("mmHg/m", [-4.02, -2.44, -1.23, -0.546], -2.07, 1.51),
"R (hydraulic resistance)": ("Pa·s/µm4", [2.49e-8, 6.47e-9, 1.51e-8, 1.30e-8], 1.49e-8, 7.63e-9),
"wall shear stress": ("Pa", [4.95e-3, 3.16e-3, 2.30e-3, 1.58e-3], 3.00e-3, 1.45e-3),
}
N = 4
RANGE_TEXT = (0.5e4, 5e4) # Results, "Volume Flow Rate"
Q_MOUSE1_DISC = 2.25e4 # Discussion, µm3/s
AREA_DISC = 1.1e3 # Discussion, half-PVS area, µm2
SCALE = 182 # Discussion, total periarterial / measured area
BRAIN_G = 0.4 # Discussion, g
TOTAL_DISC_ULMIN = 0.25 # Discussion, µL/min
TOTAL_DISC_PERG = 0.625 # Discussion, µL/(g·min)
RAY_PERG = 0.5 # Discussion, prior estimate, µL/(g·min)
UM3_PER_UL = 1e9
# ---------------------------------------------------------------- 1. summary rows
print("=== 1. Table 1 summary rows: printed vs recomputed from the four mouse rows ===")
print(f"{'column':26s} {'printed mean':>13s} {'mean':>11s} {'ratio':>7s} | {'printed σ':>10s} {'SD(n-1)':>10s} {'SD(n)':>10s} {'SE':>10s}")
S = {}
for name, (unit, v, pm, ps) in COLS.items():
x = np.array(v)
m, sd1, sd0 = x.mean(), x.std(ddof=1), x.std(ddof=0)
se = sd1 / np.sqrt(N)
S[name] = dict(mean=m, sd=sd1, sd0=sd0, se=se, pm=pm, ps=ps)
print(f"{name:26s} {pm:13.4g} {m:11.4g} {m/pm:7.3f} | {ps:10.4g} {sd1:10.4g} {sd0:10.4g} {se:10.4g}")
q = np.array(COLS["Q (volume flow rate)"][1])
Sq = S["Q (volume flow rate)"]
print()
print("=== 2. Volume flow rate Q in detail ===")
print(f"per-mouse Q (µm3/s): {q.tolist()}")
print(f"sum = {q.sum():.1f}; mean = {Sq['mean']:.1f} = {Sq['mean']:.4e}")
print(f"sample SD (n-1) = {Sq['sd']:.1f} = {Sq['sd']:.4e}; population SD (n) = {Sq['sd0']:.4e}")
print(f"SE = SD/sqrt(4) = {Sq['se']:.1f} = {Sq['se']:.4e}")
print(f"recomputed mean / printed mean = {Sq['mean']/Sq['pm']:.4f}; recomputed SD / printed σ = {Sq['sd']/Sq['ps']:.4f}")
print(f"mantissas: mean {Sq['mean']/1e4:.4f} (printed 2.22), SD {Sq['sd']/1e4:.4f} (printed 1.983)")
print(f"smallest mouse Q = {q.min():.3e}; printed mean / smallest = {Sq['pm']/q.min():.3f}")
print(f"Results range {RANGE_TEXT[0]:.1e} to {RANGE_TEXT[1]:.1e}; observed range {q.min():.3e} to {q.max():.3e}")
print(f"printed mean inside Results range? {RANGE_TEXT[0] <= Sq['pm'] <= RANGE_TEXT[1]}; recomputed mean inside? {RANGE_TEXT[0] <= Sq['mean'] <= RANGE_TEXT[1]}")
# mean of four positive values can never be below the smallest one
print(f"any 10^3 mean would need sum of four values = {4*Sq['pm']:.3e}, less than the largest single value {q.max():.3e}")
print()
print("=== 3. Internal consistency of the per-mouse Q values (shows they are in µm3/s) ===")
qa = np.array(COLS["Q/A (mean flow speed)"][1])
area = q / qa
print("implied cross-section Q/(Q/A) (µm2):", np.round(area, 0).tolist(), f"; Discussion gives {AREA_DISC:.1e} for mouse 1")
dp = np.array(COLS["dP/dn"][1])
r_tab = np.array(COLS["R (hydraulic resistance)"][1])
r_calc = np.abs(dp) / q
print("R = |dP/dn| / Q (Pa·s/µm4):", [f"{v:.3e}" for v in r_calc], "; Table 1:", [f"{v:.3e}" for v in r_tab])
print("ratio computed/tabulated R:", np.round(r_calc / r_tab, 3).tolist())
print(f"mouse 1 Discussion value {Q_MOUSE1_DISC:.3e} µm3/s = {Q_MOUSE1_DISC*60/UM3_PER_UL:.3e} µL/min (paper: 1.35e-3)")
print(f"mouse 1 Table 1 value {q[0]:.3e}; Discussion/Table = {Q_MOUSE1_DISC/q[0]:.3f}")
print()
print("=== 4. Pressure-gradient unit conversion (side check) ===")
mm = np.array(COLS["dP/dn (mmHg/m)"][1])
conv = dp * 1e6 / 133.322
print("Pa/µm -> mmHg/m per mouse:", np.round(conv, 3).tolist(), "; Table 1:", mm.tolist())
print("relative difference:", np.round(conv / mm - 1, 3).tolist())
Sd = S["dP/dn"]
print(f"mean dP/dn converted: {Sd['mean']*1e6/133.322:.3f} mmHg/m (printed -2.07); SD converted {Sd['sd']*1e6/133.322:.3f} (printed 1.51)")
print()
print("=== 5. Abstract label 'mean ± SE' vs the printed ± values ===")
for name in ["Q/A (mean flow speed)", "Q (volume flow rate)", "dP/dn", "wall shear stress"]:
s = S[name]
print(f"{name:26s} printed ± {s['ps']:.4g}; SD(n-1) {s['sd']:.4g}; SE {s['se']:.4g}")
print()
print("=== 6. Discussion scale-up applied to each value (×182, per 0.4 g brain) ===")
def scale_up(qv):
total_ul_min = qv * SCALE * 60 / UM3_PER_UL
return total_ul_min, total_ul_min / BRAIN_G
SU = {}
for lab, qv in [("Discussion, mouse 1 (2.25e4)", Q_MOUSE1_DISC),
("recomputed mean (2.222e4)", Sq["mean"]),
("printed mean (2.22e3)", Sq["pm"])]:
t, pg = scale_up(qv)
SU[lab] = pg
print(f"{lab:30s} total {t:.4f} µL/min; {pg:.4f} µL/(g·min); ratio to prior 0.5 = {pg/RAY_PERG:.3f}")
print(f"paper's own figures: {TOTAL_DISC_ULMIN} µL/min, {TOTAL_DISC_PERG} µL/(g·min) (0.25/0.4 = {TOTAL_DISC_ULMIN/BRAIN_G})")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def fmt_sci(v):
e = int(np.floor(np.log10(abs(v))))
return f"{v/10**e:.2f} × 10$^{{{e}}}$"
# ---- fig1: printed mean lies below every value it averages
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ax.set_xscale("log")
ax.set_xlim(1.5e2, 1.2e5)
rows = {"Table 1 rows\n(mice 1–4)": 3.0, "Recomputed\nmean ± SD": 2.0, "Printed mean ± σ\n(Table 1, Abstract)": 1.0}
ax.set_ylim(0.4, 3.75)
ax.set_yticks(list(rows.values()))
ax.set_yticklabels(list(rows.keys()))
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.grid(axis="x", which="minor", color=GRID, lw=0.4, alpha=0.6)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# Results range as a light band
ax.axvspan(*RANGE_TEXT, color="#efeeea", zorder=0)
ax.text(np.sqrt(RANGE_TEXT[0]*RANGE_TEXT[1]), 3.62, "Results text: “ranged from 0.5 to 5 × 10$^4$ µm$^3$/s”",
ha="center", va="center", fontsize=10, color=INK2)
# per-mouse values
for i, v in enumerate(q):
ax.plot(v, 3.0, "o", ms=9, mfc=BG, mec=INK, mew=1.6, zorder=3)
ax.text(v, 3.0 + (0.2 if i % 2 == 0 else -0.24), f"{i+1}", ha="center", va="center", fontsize=10, color=INK2)
# recomputed
m, sd = Sq["mean"], Sq["sd"]
ax.plot([m - sd, m + sd], [2.0, 2.0], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(m, 2.0, "o", ms=10, color=BLUE, zorder=3)
ax.text(m, 2.24, f"{fmt_sci(m)} (SD {fmt_sci(sd)})", ha="center", va="bottom", fontsize=10, color=BLUE)
# printed
pm, ps = Sq["pm"], Sq["ps"]
ax.plot([pm - ps, pm + ps], [1.0, 1.0], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(pm, 1.0, "o", ms=10, color=ORANGE, zorder=3)
ax.text(pm, 1.24, f"{fmt_sci(pm)} (σ {fmt_sci(ps)})", ha="center", va="bottom", fontsize=10, color=ORANGE)
# connector: factor 10
ax.annotate("", xy=(pm*1.12, 1.55), xytext=(m/1.12, 1.55),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.2))
ax.text(np.sqrt(pm*m), 1.62, f"mean × {m/pm:.2f}, SD × {sd/ps:.2f}", ha="center", va="bottom", fontsize=10, color=INK2)
ax.text(2.0e2, 0.62, f"printed mean is {q.min()/pm:.1f}× below the smallest of the four values it averages ({fmt_sci(q.min())})",
ha="left", va="center", fontsize=10, color=INK2)
ax.set_xlabel("Volume flow rate Q (µm$^3$/s, log scale)")
handles = [Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=INK, mew=1.6, label="per-mouse value printed in Table 1"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=9, label="recomputed from those four values"),
Line2D([], [], marker="o", color=ORANGE, lw=2, ms=9, label="summary printed in Table 1 and Abstract")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# ---- fig2: Discussion scale-up applied to each value
fig, ax = plt.subplots(figsize=(7.2, 2.9))
ax.set_xscale("log")
ax.set_xlim(0.03, 1.5)
items = [
("Prior estimate cited\nin Discussion", RAY_PERG, INK, "o", "0.5 (cited)"),
("Discussion, from\nmouse 1 (paper)", TOTAL_DISC_PERG, INK, "o", "0.625 (printed)"),
("From recomputed\nmean Q", SU["recomputed mean (2.222e4)"], BLUE, "o", None),
("From printed\nmean Q", SU["printed mean (2.22e3)"], ORANGE, "o", None),
]
ys = np.arange(len(items))[::-1]
for (lab, v, c, mk, txt), y in zip(items, ys):
filled = c != INK
ax.plot(v, y, mk, ms=10, color=c, mfc=c if filled else BG, mec=c, mew=1.6, zorder=3)
ax.text(v*1.12, y + 0.02, txt if txt else f"{v:.3f}", ha="left", va="center", fontsize=10, color=c if filled else INK2)
ax.axvline(RAY_PERG, color=INK2, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.set_yticks(ys)
ax.set_yticklabels([i[0] for i in items])
ax.set_ylim(-0.6, len(items) - 0.4)
ax.grid(axis="x", which="both", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xticks([0.03, 0.05, 0.1, 0.2, 0.5, 1.0])
ax.set_xticklabels(["0.03", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_xlabel("Whole-brain periarterial flow, µL/(g·min), log scale (Q × 182 ÷ 0.4 g)")
handles = [Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=INK, mew=1.6, label="value printed in the Discussion"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="scale-up of recomputed mean"),
Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="scale-up of printed mean")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.45, -0.28), ncol=3, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
save(fig, "fig2")
print()
print("wrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract says vitamin D3 rose in Scribble knockout mice; Results and Fig. 3E show a fall
Proc Natl Acad Sci U S A (2023) · doi:10.1073/pnas.2220851120 · PMC10266016
Summary
The abstract states that knocking out Scribble in renal proximal tubules "caused significant plasma phosphate elevation and increased aggregate vitamin D3 levels". The paper defines a single aggregate vitamin D3 measure: three metabolites, each normalized to littermate controls and then combined. Fig. 3E is the only place it is plotted. The Results text and the Fig. 3 legend both describe that measure as decreased, and the points plotted in Fig. 3E agree: digitized, the six tamoxifen-induced knockout values have a mean of 63.2% of control, and 5 of the 6 lie below 100%. Vitamin D is one of the two in vivo findings the abstract reports, so a reader of the abstract alone gets the direction of that finding reversed.
What the paper reports
Abstract (as archived in PMC, after the Significance statement):
The loss of Scribble impacted serum phosphate and vitamin D levels and caused significant plasma phosphate elevation and increased aggregate vitamin D3 levels, whereas blood glucose levels remained unchanged.
Results, section "Loss of Scribble Impacts Renal Phosphate Homeostasis and Serum Vitamin D Levels", second paragraph:
Combined 1,25(OH)2D3 + 24,25(OH)2D3 + 24(OH)D3 levels, as indicated in Methods, decreased only in Tam-injected animals and not in corresponding Cre littermates (Fig. 3E).
Fig. 3 legend:
(B) Targeted Scribble KO mice display increased serum phosphate (C), normal levels of calcium and glucose (D), decreased 1,25[OH]2-vitamin D (1,25[OH]2D) levels (E), and normal levels of PTH and FGF23 (F and G).
Materials and Methods, "Mice.", where the aggregate is defined:
Vitamin D3 was measured as described (52) on littermate pairs. 1,25 vitamin D3, 24,25 vitamin D3, and 25 vitamin D3 forms were normalized to control levels and combined.
Fig. 3E (an image; the axis title and group labels were read from the archived figure file pnas.2220851120fig03.jpg): the y-axis is "Plasma Vitamin D % control", the groups are Veh, Tam and Cre, and a bracket marks Veh vs Tam with "*" and Veh vs Cre with "ns".
Statements elsewhere that give no direction. Significance statement:
The loss of this interaction impacts serum phosphate and vitamin D levels in Scribble KO mice.
Discussion, second paragraph:
Targeted loss of Scribble in proximal tubules resulted in pronounced alterations to serum phosphate and vitamin D levels, whereas glucose, calcium, PTH and FGF23 levels remained unaffected.
Discussion, third paragraph, on mechanism:
PTHR activates the 25-hydroxyvitamin D3-1α-hydroxylase catalyzing the formation of biologically active 1,25(OH)2-vitamin D3.
What the paper's own numbers show
Method. The text gives no vitamin D values, means or test statistics, so the direction can be checked in two ways: by comparing what the text says in each place, and by reading the plotted points in Fig. 3E. For the second, I took the figure as archived in PMC and Europe PMC (708 × 927 px). I calibrated the y-axis from its five tick marks (0, 50, 100, 150 and 200%; largest residual 0.5 percentage points) and located each dot as the centroid of its connected block of dark pixels. As a check, the six Veh (control) dots, which are fixed at 100% by the normalization, come out at 99.8%. The mean computed from the digitized dots matches the mean bar drawn in the figure to within 0.7 percentage points for Tam and exactly for Cre. The paper does not say which test produced the "*". Because each knockout value is already a ratio to its own littermate control, a one-sample test against 100% is the natural analogue, and I report it only to show that the plotted points are consistent with the asterisk. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own figure |
|---|---|---|---|
| Direction of the aggregate vitamin D3 measure | Abstract | "increased" | decrease (opposite sign) |
| Direction of the same measure | Results, second paragraph | "decreased only in Tam-injected animals" | decrease (agrees) |
| Direction of panel E | Fig. 3 legend | "decreased … levels (E)" | decrease (agrees) |
| Control (Veh) dots | Fig. 3E | fixed at 100% | 99.8% (calibration check) |
| Tam (knockout) dots, % of control | Fig. 3E | plotted only | 109.2, 78.0, 55.8, 49.2, 44.0, 42.7 |
| Tam mean (median) | Fig. 3E | mean bar drawn, no value given | 63.2% (52.5%); drawn bar reads 62.5% |
| Change vs control | Fig. 3E | — | −36.8 percentage points |
| Tam dots below 100% | Fig. 3E | — | 5 of 6 |
| Tam 95% CI of the mean (t, df = 5) | Fig. 3E | — | 35.9% to 90.4%; excludes 100% |
| Test of Tam vs control | Fig. 3E | "*" | one-sample t = −3.47, two-sided p = 0.018; exact Wilcoxon p = 0.0625 |
| Cre-only littermates, % of control | Fig. 3E | "ns" | 150.8, 133.4, 126.0, 109.9, 78.5, 69.2; mean 111.3% (95% CI 77.8 to 144.9; p = 0.43) |
| Digitizing error needed to move the Tam mean to 100% | — | — | 21.9 px for every dot; a dot is about 5 px across |
The direction does not depend on how precisely the dots were located: every Tam dot would have to be misplaced by about four dot diameters for the mean to reach 100%. The only group whose mean lies above 100% is Cre, and that difference is marked "ns" and is not attributed to loss of Scribble.
Figure
Figure 1. Top: the two text statements about the same aggregate vitamin D measure. The abstract (orange outline) says "increased"; the Results text and Fig. 3 legend (grey outline) say "decreased". Bottom: the six knockout (Tam) values read from Fig. 3E, as % of littermate control (open blue circles), with their mean of 63.2% (filled blue circle) and its t-based 95% interval of 35.9% to 90.4% (blue line); the vertical line marks the control level of 100%. Blue values are recomputed by recompute.py.txt from the paper's own figure; they are not values printed in the paper. The axis starts at zero.
Design candidates for review.
recompute.py.txtalso writes five candidate layouts of this figure,fig1_Atofig1_E(fig1 is currently design D). Only the chosen design will be kept as fig1.
Why it matters
- The abstract reverses one of its two in vivo findings. Plasma phosphate and vitamin D are the only in vivo results in the abstract. Many readers, indexers and citing authors see only the abstract, and from it they will take an increase in vitamin D3 that the paper's data do not show.
- The direction matters physiologically. High plasma phosphate with lower vitamin D metabolites and unchanged PTH and FGF23, which is what the Results and Fig. 3 report, fits reduced PTHR signaling in the proximal tubule. The paper's own Discussion notes that PTHR activates the 1α-hydroxylase that forms 1,25(OH)2D3. Higher vitamin D, as the abstract states, would point the other way.
- The overall conclusion is not affected. The paper's broader claim, that Scribble loss alters phosphate and vitamin D homeostasis, holds whichever way vitamin D moved. The error is in how the abstract describes the result, not in the data or the analysis.
Explanations considered
- A different vitamin D measure. Not supported. Methods define one aggregate ("normalized to control levels and combined"), Fig. 3E is the only vitamin D panel, and the SI Appendix (4 pages: Figs. S1–S2 and two movie legends) contains no vitamin D data. "Aggregate vitamin D3" in the abstract can only refer to the Fig. 3E measure, which the Results call "Combined" levels.
- A different comparison group. Not supported. The Cre-only littermates have a mean slightly above 100% (111.3%), but the figure marks that difference "ns", and the abstract attributes the change to "the loss of Scribble", which is the Tam group. The Results say the change occurred "only in Tam-injected animals and not in corresponding Cre littermates".
- An inverted scale, such as control divided by knockout. Not supported. The axis is "% control" with the controls fixed at 100, and both the Results and the legend use "decreased" for the same panel.
- A conversion or typesetting artifact. Not possible here. The conflict is between the words "increased" and "decreased", not a sign, superscript or digit. The Europe PMC full-text XML deposited for this article carries the same abstract, Results and legend wording.
- Digitizing error. Not able to explain it. The calibration reproduces the control row (99.8%) and both drawn mean bars, and a reversal would need every Tam dot to be misplaced by about 22 px.
- A correction already issued. None found (see Status).
- Side inconsistencies that do not change the direction. The Results name the third metabolite "24(OH)D3", while Methods list "25 vitamin D3". The legend calls panel E "1,25[OH]2-vitamin D" although the panel shows the combined measure. The legend's panel letters follow the items rather than preceding them ("(B) … increased serum phosphate (C)"), while the Results cite phosphate as Fig. 3B. Panel E is labelled as vitamin D in the figure itself, so none of these affects the direction.
Status
- Version of record. The publisher's page (pnas.org, reached through the DOI) returned HTTP 403 to automated requests. I checked the Europe PMC full-text XML deposited for the article, and its abstract, Results paragraph, Fig. 3 legend and Methods sentence match the PMC archive word for word. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. The Fig. 3E axis title, group labels and significance marks were read from the published figure image. - Supplements. I downloaded the Europe PMC supplementary bundle. It holds the SI Appendix (
pnas.2220851120.sapp.pdf, 4 pages), two molecular dynamics movies and the three main figure images. A text search of the SI Appendix finds "vitamin" only in the title and no vitamin D data. The Data Availability statement places all data in the article or SI, and there is no source-data file. - Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Europe PMC record (PMID 37252981) lists no comments or corrections, and a Europe PMC search for an erratum on this title returned none. PubPeer's search page returned HTTP 403 and its API returned no record to automated requests. A web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC10266016; Europe PMC full-text XML and supplementary files for PMC10266016; figure image
pnas.2220851120fig03.jpg; SI Appendixpnas.2220851120.sapp.pdf; Crossref; Europe PMC; PubPeer; https://doi.org/10.1073/pnas.2220851120.
Suggested correction
In the abstract, replace
caused significant plasma phosphate elevation and increased aggregate vitamin D3 levels
with
caused significant plasma phosphate elevation and decreased aggregate vitamin D3 levels
so that the abstract agrees with the Results text, the Fig. 3 legend and the data in Fig. 3E. The authors may also wish to make the third metabolite the same in Results ("24(OH)D3") and Methods ("25 vitamin D3"), to describe panel E in the legend as the combined measure, and to give the test used for Fig. 3E with its n and P value.
Recomputation script (python recompute.py.txt)
"""Recompute the direction of the vitamin D result in PMC10266016
(Proc Natl Acad Sci U S A 2023, doi:10.1073/pnas.2220851120).
Run: python recompute.py.txt (uses the pixel positions recorded below)
python recompute.py.txt <fig03.jpg> (also re-measures them from the image)
Needs numpy, scipy and matplotlib (plus Pillow for the optional re-measurement).
Prints every recomputed number used in writeup.md and writes fig1.svg/png and the
five candidate designs fig1_A ... fig1_E (.svg/.png) next to this file.
Input: Fig. 3E of the paper ("Plasma Vitamin D % control"), as archived in PMC /
Europe PMC (pnas.2220851120fig03.jpg, 708 x 927 px). Panel E is the paper's only
vitamin D data. The plotted values are not printed anywhere in the text, so they were
read off the image: dark pixels (grey < 110) in the panel were grouped into connected
blobs and each dot's centre taken as the blob centroid; two Cre dots that touch were
split by their row profile, and one Cre dot drawn on top of its mean bar was read from
the row profile too (the re-measurement prints the blobs it can separate automatically).
The y-axis was calibrated from the five tick marks.
"""
import os
import sys
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Fig. 3E pixels)
TICK_ROW = np.array([448.0, 478.0, 508.0, 537.5, 567.0]) # tick marks on the y-axis (x = 71-75)
TICK_VAL = np.array([200.0, 150.0, 100.0, 50.0, 0.0]) # tick labels, % control
VEH_ROW = 507.8 # six Veh dots, one merged row
TAM_ROWS = np.array([502.2, 520.8, 534.0, 537.9, 541.0, 541.8]) # six Tam dots
TAM_BAR_ROW = 530.0 # Tam mean bar as drawn
CRE_ROWS = np.array([477.5, 487.8, 492.2, 501.8, 520.5, 526.0]) # six Cre dots
CRE_BAR_ROW = 501.0 # Cre mean bar as drawn
# ---------------------------------------------------------------- inputs (text, verbatim)
Q_ABSTRACT = ("caused significant plasma phosphate elevation and increased "
"aggregate vitamin D3 levels")
Q_RESULTS = ("Combined 1,25(OH)2D3 + 24,25(OH)2D3 + 24(OH)D3 levels ... "
"decreased only in Tam-injected animals")
Q_LEGEND = "decreased 1,25[OH]2-vitamin D (1,25[OH]2D) levels (E)"
Q_METHODS = ("1,25 vitamin D3, 24,25 vitamin D3, and 25 vitamin D3 forms were "
"normalized to control levels and combined.")
def remeasure(path):
"""Re-detect the Fig. 3E dots from the archived JPEG and print their rows."""
from PIL import Image
from scipy import ndimage
im = np.asarray(Image.open(path).convert("L")).astype(float)
y0, x0 = 440, 60
sub = im[y0:580, x0:220] < 110
lab, n = ndimage.label(sub)
print(f"re-measured from {os.path.basename(path)} ({im.shape[1]}x{im.shape[0]} px):")
for i, sl in enumerate(ndimage.find_objects(lab)):
area = int((lab[sl] == i + 1).sum())
cy, cx = ndimage.center_of_mass(lab == i + 1)
w = sl[1].stop - sl[1].start
h = sl[0].stop - sl[0].start
if 15 <= area <= 50 and w <= 7 and 135 <= cx + x0 <= 205 and cy + y0 > 470:
grp = "Tam" if cx + x0 < 170 else "Cre"
note = " (two touching dots)" if h > 8 else ""
print(f" {grp} dot x={cx + x0:6.1f} row={cy + y0:6.1f} area={area}{note}")
print()
# ---------------------------------------------------------------- calculation
slope, icpt = np.polyfit(TICK_ROW, TICK_VAL, 1)
resid = TICK_VAL - (slope * TICK_ROW + icpt)
def pct(row):
return slope * np.asarray(row, dtype=float) + icpt
veh = float(pct(VEH_ROW))
tam = pct(TAM_ROWS)
cre = pct(CRE_ROWS)
tam_bar = float(pct(TAM_BAR_ROW))
cre_bar = float(pct(CRE_BAR_ROW))
n = len(tam)
tam_mean = tam.mean()
tam_sd = tam.std(ddof=1)
tam_se = tam_sd / np.sqrt(n)
tcrit = stats.t.ppf(0.975, n - 1)
tam_ci = (tam_mean - tcrit * tam_se, tam_mean + tcrit * tam_se)
tt = stats.ttest_1samp(tam, 100.0)
wx = stats.wilcoxon(tam - 100.0)
cre_tt = stats.ttest_1samp(cre, 100.0)
cre_ci = (cre.mean() - tcrit * cre.std(ddof=1) / np.sqrt(n),
cre.mean() + tcrit * cre.std(ddof=1) / np.sqrt(n))
# how far the digitizing would have to be off for the sign to flip
px_per_pct = 1.0 / abs(slope)
shift_to_100 = (100.0 - tam_mean) * px_per_pct
def report():
print("Fig. 3E, plasma vitamin D (% control), digitized")
print(f" axis calibration: {slope:.4f} % per px (max tick residual {abs(resid).max():.2f} %)")
print(f" Veh row -> {veh:.1f} % (should be 100: the controls are the normalizing base)")
print(" Tam points:", ", ".join(f"{v:.1f}" for v in tam))
print(f" Tam mean {tam_mean:.1f} %, median {np.median(tam):.1f} %, SD {tam_sd:.1f}")
print(f" Tam mean bar as drawn: {tam_bar:.1f} %")
print(f" Tam change vs control: {tam_mean - 100:+.1f} percentage points")
print(f" Tam points below 100 %: {(tam < 100).sum()} of {n}")
print(f" Tam 95% CI of mean (t, df={n - 1}): {tam_ci[0]:.1f} to {tam_ci[1]:.1f} %")
print(f" one-sample t vs 100: t = {tt.statistic:.2f}, two-sided p = {tt.pvalue:.4f}")
print(f" Wilcoxon signed-rank vs 100: p = {wx.pvalue:.4f} (exact, n = 6)")
print(" Cre points:", ", ".join(f"{v:.1f}" for v in cre))
print(f" Cre mean {cre.mean():.1f} %, mean bar as drawn {cre_bar:.1f} %, "
f"95% CI {cre_ci[0]:.1f} to {cre_ci[1]:.1f} %, t vs 100 p = {cre_tt.pvalue:.3f}")
print(f" digitizing error needed to move the Tam mean up to 100 %: "
f"{shift_to_100:.1f} px for every dot (dot diameter about 5 px)")
print()
print("Direction of the same aggregate measure, as stated in the paper")
rows = [("Abstract", "increased", "+"),
("Results, 'Loss of Scribble Impacts ...' p1", "decreased", "-"),
("Fig. 3 legend, panel E", "decreased", "-"),
("Fig. 3E plotted data (digitized)",
f"mean {tam_mean:.0f} % of control", "-" if tam_mean < 100 else "+")]
for loc, word, sign in rows:
print(f" {sign} {loc:44s} {word}")
print(" Abstract agrees with the plotted data:", rows[0][2] == rows[3][2])
# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834" # the paper's reported statement / value
BLUE = "#2a78d6" # recomputed from the paper's own figure
GRID = "#e4e3df"
PANEL = "#f3f2ef"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "axes.facecolor": BG, "figure.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
SUB = str.maketrans("23", "₂₃")
def vd(s):
"""Typeset vitamin D names with subscripts."""
return (s.replace("(OH)2D3", "(OH)₂D₃").replace("(OH)D3", "(OH)D₃")
.replace("[OH]2", "[OH]₂").replace("D3 ", "D₃ ").replace("D3\n", "D₃\n"))
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200)
plt.close(fig)
def textbox(ax, x, y, w, h, title, body, hl=None, hl_color=ORANGE, edge=INK2, hl2=None):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=PANEL, ec=edge, lw=1.0, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.05, title, transform=ax.transAxes, fontsize=10.5,
fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.16, body, transform=ax.transAxes, fontsize=10.5,
va="top", color=INK, linespacing=1.45)
if hl2:
ax.text(x + 0.02, y + 0.035, hl2, transform=ax.transAxes, fontsize=10,
fontweight="bold", va="bottom", color=BLUE)
if hl:
ax.text(x + 0.02, y + (0.11 if hl2 else 0.05), hl, transform=ax.transAxes, fontsize=11,
fontweight="bold", va="bottom", color=hl_color)
def blank(figsize):
fig = plt.figure(figsize=figsize)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
return fig, ax
# ---------------------------------------------------------------- A: two text panels
def design_A():
fig, ax = blank((W, 3.7))
ax.text(0.03, 0.95, "The same aggregate vitamin D measure, described in opposite directions",
fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
textbox(ax, 0.03, 0.12, 0.40, 0.70, "Abstract",
vd("“…caused significant plasma\nphosphate elevation and\n"
"increased aggregate vitamin D3 \nlevels…”"),
hl="▲ increased", hl_color=ORANGE, edge=ORANGE)
textbox(ax, 0.57, 0.12, 0.40, 0.70, "Results text and Fig. 3 legend",
vd("Results: “Combined 1,25(OH)2D3 +\n24,25(OH)2D3 + 24(OH)D3 levels …\n"
"decreased only in Tam-injected\nanimals”\n"
"Legend: “decreased 1,25[OH]2-\nvitamin D … levels (E)”"),
hl="▼ decreased", hl_color=INK2,
hl2=f"Fig. 3E: Tam mean {tam_mean:.0f}% of control")
ax.add_patch(FancyArrowPatch((0.435, 0.47), (0.565, 0.47), transform=ax.transAxes,
arrowstyle="<->", mutation_scale=14, lw=2, color=INK2))
ax.text(0.5, 0.55, "opposite\nsign", ha="center", va="bottom", fontsize=10.5,
color=INK, transform=ax.transAxes, linespacing=1.2)
ax.text(0.5, 0.40, "same\nmeasure\n(Methods)", ha="center", va="top", fontsize=10,
color=INK2, transform=ax.transAxes, linespacing=1.2)
ax.text(0.03, 0.03, "Orange: the abstract's statement. Blue: recomputed from the "
"points plotted in Fig. 3E.", fontsize=10, color=INK2, transform=ax.transAxes)
return fig
# ---------------------------------------------------------------- B: direction ledger
def design_B():
fig, ax = blank((W, 3.2))
ax.text(0.03, 0.95, "Every place the paper gives the direction of the vitamin D change",
fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
rows = [("Abstract", vd("“increased aggregate vitamin D3 levels”"), "up", ORANGE,
"stated in the paper"),
("Results, p1", vd("“…decreased only in Tam-injected animals”"), "down",
INK2, "stated in the paper"),
("Fig. 3 legend", vd("“decreased 1,25[OH]2-vitamin D … levels (E)”"),
"down", INK2, "stated in the paper"),
("Fig. 3E data", f"Tam mean {tam_mean:.1f}% of control; "
f"{(tam < 100).sum()} of {n} dots below 100%", "down", BLUE,
"recomputed (digitized)")]
ax.text(0.80, 0.81, "increase", ha="center", fontsize=10, color=INK2, transform=ax.transAxes)
ax.text(0.92, 0.81, "decrease", ha="center", fontsize=10, color=INK2, transform=ax.transAxes)
for i, (loc, txt, d, col, src) in enumerate(rows):
y = 0.68 - i * 0.165
ax.plot([0.03, 0.97], [y - 0.075, y - 0.075], color=GRID, lw=1, transform=ax.transAxes)
ax.text(0.03, y + 0.02, loc, fontweight="bold", fontsize=10.5, va="center",
transform=ax.transAxes)
ax.text(0.03, y - 0.04, src, fontsize=10, color=INK2, va="center",
transform=ax.transAxes)
ax.text(0.22, y, txt, fontsize=10.5, va="center", transform=ax.transAxes)
xm = 0.80 if d == "up" else 0.92
ax.text(xm, y, "▲" if d == "up" else "▼", fontsize=15, ha="center",
va="center", color=col, transform=ax.transAxes)
ax.text(0.03, 0.02, "Orange: the abstract. Blue: recomputed from the plotted points. "
"Grey: the other text.", fontsize=10, color=INK2,
transform=ax.transAxes)
return fig
# ---------------------------------------------------------------- C: one-axis direction scale
def design_C():
fig, ax = plt.subplots(figsize=(W, 2.9))
fig.subplots_adjust(left=0.04, right=0.97, top=0.80, bottom=0.25)
ax.set_xlim(0, 160)
ax.set_ylim(-1, 1.55)
ax.set_yticks([])
for s in ("left", "right", "top"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.axvline(100, color=INK2, lw=1)
ax.text(100, 1.50, "control = 100%", ha="center", va="bottom", fontsize=10, color=INK2)
# abstract: direction only (no number given) -> arrow into the >100 region
ax.annotate("", xy=(150, 0.65), xytext=(101, 0.65),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=14))
ax.text(103, 0.78, "Abstract: “increased”\n(direction only, no value)",
fontsize=10.5, color=INK, va="bottom", linespacing=1.25)
# recomputed
ax.scatter(tam, np.full(n, -0.35), s=42, facecolor=BG, edgecolor=BLUE, lw=1.5, zorder=3)
ax.plot(tam_ci, [-0.35, -0.35], color=BLUE, lw=2, zorder=2)
ax.scatter([tam_mean], [-0.35], s=80, color=BLUE, zorder=4)
ax.text(tam_mean, -0.05, f"Fig. 3E Tam mean {tam_mean:.1f}% (95% CI {tam_ci[0]:.0f}–"
f"{tam_ci[1]:.0f}), recomputed", ha="center", fontsize=10.5, va="bottom")
ax.text(2, -0.85, "Results and legend: “decreased”", fontsize=10, color=INK2)
ax.set_xlabel("Combined plasma vitamin D in Scribble KO (Tam) mice, % of littermate control",
fontsize=10.5)
fig.text(0.04, 0.93, "The abstract's direction lies on the wrong side of the control line",
fontsize=11.5, fontweight="bold")
return fig
# ---------------------------------------------------------------- D: text + strip
def design_D():
fig = plt.figure(figsize=(W, 3.9))
ax = fig.add_axes([0, 0.34, 1, 0.66])
ax.set_axis_off()
ax.text(0.03, 0.95, "Abstract versus the paper's own text and data on vitamin D",
fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
textbox(ax, 0.03, 0.04, 0.40, 0.74, "Abstract",
vd("“…plasma phosphate elevation\nand increased aggregate\nvitamin D3 "
"levels…”"), hl="▲ increased", hl_color=ORANGE, edge=ORANGE)
textbox(ax, 0.57, 0.04, 0.40, 0.74, "Results text and Fig. 3 legend",
vd("“…decreased only in Tam-\ninjected animals” (Results)\n"
"“decreased 1,25[OH]2-vitamin D\n… levels (E)” (legend)"),
hl="▼ decreased", hl_color=INK2)
ax.add_patch(FancyArrowPatch((0.435, 0.41), (0.565, 0.41), transform=ax.transAxes,
arrowstyle="<->", mutation_scale=14, lw=2, color=INK2))
ax.text(0.5, 0.47, "opposite\nsign", ha="center", va="bottom", fontsize=10.5,
transform=ax.transAxes, linespacing=1.2)
s = fig.add_axes([0.08, 0.16, 0.86, 0.12])
s.set_xlim(0, 160)
s.set_ylim(-1, 1)
s.set_yticks([])
for sp in ("left", "right", "top"):
s.spines[sp].set_visible(False)
s.grid(axis="x", color=GRID, lw=0.8)
s.set_axisbelow(True)
s.axvline(100, color=INK2, lw=1)
s.scatter(tam, np.zeros(n), s=40, facecolor=BG, edgecolor=BLUE, lw=1.5, zorder=3)
s.plot(tam_ci, [0, 0], color=BLUE, lw=2)
s.scatter([tam_mean], [0], s=80, color=BLUE, zorder=4)
s.text(103, 0.15, "control = 100%", fontsize=10, color=INK2, va="bottom")
s.set_xlabel(f"Fig. 3E, Tam mice: digitized points, mean {tam_mean:.1f}% and 95% CI "
f"({tam_ci[0]:.0f}–{tam_ci[1]:.0f}), % of control (recomputed)",
fontsize=10, color=INK)
return fig
# ---------------------------------------------------------------- E: flow of the measure
def design_E():
fig, ax = blank((W, 3.0))
ax.text(0.03, 0.95, "One measure, traced from its definition to the abstract",
fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
boxes = [(0.03, "Methods", "1,25 + 24,25 +\n25 vitamin D₃,\nnormalized to\ncontrol, combined", INK2, None),
(0.27, "Fig. 3E data", f"Tam mean\n{tam_mean:.1f}% of control\n"
f"{(tam < 100).sum()} of {n} below 100", BLUE, "▼"),
(0.51, "Results, legend", "“decreased”\n“decreased”", INK2,
"▼"),
(0.75, "Abstract", "“increased\naggregate\nvitamin D₃\nlevels”", ORANGE,
"▲")]
for x, t, b, c, mark in boxes:
ax.add_patch(FancyBboxPatch((x, 0.22), 0.21, 0.58,
boxstyle="round,pad=0,rounding_size=0.015", fc=PANEL,
ec=c, lw=1.4 if c != INK2 else 1.0, transform=ax.transAxes))
ax.text(x + 0.015, 0.75, t, fontweight="bold", fontsize=10.5, va="top",
transform=ax.transAxes)
ax.text(x + 0.015, 0.63, b, fontsize=10.5, va="top", transform=ax.transAxes,
linespacing=1.35)
if mark:
ax.text(x + 0.015, 0.26, mark, fontsize=16, ha="left", va="bottom", color=c,
transform=ax.transAxes)
for x0 in (0.24, 0.48, 0.72):
ok = x0 < 0.7
ax.add_patch(FancyArrowPatch((x0, 0.51), (x0 + 0.03, 0.51), transform=ax.transAxes,
arrowstyle="-|>", mutation_scale=12, lw=2,
color=INK2 if ok else ORANGE))
ax.text(0.735, 0.14, "sign flips here", ha="center", fontsize=10.5, color=INK,
fontweight="bold", transform=ax.transAxes)
ax.text(0.03, 0.03, "Blue: recomputed from the plotted points. Orange: the abstract's "
"statement.", fontsize=10, color=INK2, transform=ax.transAxes)
return fig
DESIGNS = {"A": design_A, "B": design_B, "C": design_C, "D": design_D, "E": design_E}
CHOSEN = "D" # the design used as fig1 in writeup.md
if __name__ == "__main__":
if len(sys.argv) > 1:
remeasure(sys.argv[1])
report()
for k, f in DESIGNS.items():
save(f(), f"fig1_{k}")
save(DESIGNS[CHOSEN](), "fig1")
print(f"\nwrote fig1_A..fig1_E (.svg/.png) and fig1 = design {CHOSEN} to {HERE}")
Abstract states interaction explains 16.4% and 35.1% of AF risk; Table 3 gives 6.2% and 12.7%
Proc Natl Acad Sci U S A (2023) · doi:10.1073/pnas.2302708120 · PMC10410743
Summary
The abstract and Discussion state that around 16.4% (PM10) and 35.1% (NO2) of atrial fibrillation (AF) risk is attributable to the interaction between high air pollution and high genetic risk. Those two numbers are the relative excess risks due to interaction (RERI = 0.164 and 0.351 in Table 3) written as percentages. The measure that gives the share of risk attributable to interaction is the attributable proportion (AP), which Table 3 reports as 0.062 (95% CI 0.011 to 0.113) and 0.127 (0.055 to 0.200), that is 6.2% and 12.7%. Recomputing AP from the reported RERI and synergy index reproduces those table values, so the table is internally consistent and the abstract overstates the attributable share about 2.6-fold (PM10) and 2.8-fold (NO2). The direction and statistical significance of the additive interaction are unaffected.
What the paper reports
Abstract:
Moreover, the significant additive interactions between PM10 and NO2 and genetic risk on AF risk were observed, with around 16.4% and 35.1% of AF risk could be attributable to the interactive effects.
Discussion, paragraph 4:
Approximately 16.4 to 35.1% of AF risk can be explained by the additive effects of co-exposure to high air pollutants and high genetic risk, meaning that the combined effects of air pollutants and genetic risk were much greater than simply adding the two effects together. This finding has far-reaching implications for the field of environmental epidemiology, such as the provision of targeted preventive or medical care for the group of people at high genetic risk and exposed to severe levels of air pollutants.
Results, paragraph 2:
Notably, for participants with high air pollutants exposure and high genetic risk, the RERIs were 0.164 (0.027, 0.302) for PM10 and 0.351 (0.133, 0.568) for NO2, indicating the relative excess risk of AF onset in the context of genetic–air pollutants interactions of 0.164 and 0.351, respectively.
Materials and Methods, Statistical Analysis, paragraph 2:
To examine whether genetic predisposition modifies the associations of air pollutants with AF incidence, we fitted models on an additive scale and calculated the RERI, the AP, and the SI. If 0 is outside the CIs of RERI and AP, or 1 is outside the CIs of SI, it means that there is an additive interaction.
Table 3, "Additive interactions between air pollutants and PRS on the risk of incident AF" (excerpt, reformatted: the "High pollution" row of each pollutant, high genetic risk columns only; the medium genetic risk columns are left out). Per the table footnote, pollution is split at the WHO guideline values for PM2.5, PM10 and NO2 and at the median (42.20 μg/m3) for NOx. The table does not state the reference group; the standard additive-interaction reference, low genetic risk and low pollution, is assumed here.
Air pollutants (High pollution row) RERIs (95% CIs) APs (95% CIs) SIs (95% CIs) PM2.5 0.057 (-0.064, 0.178) 0.023 (-0.026, 0.073) 1.041 (0.954, 1.135) PM10 0.164 (0.027, 0.302) 0.062 (0.011, 0.113) 1.111 (1.015, 1.216) NO2 0.351 (0.133, 0.568) 0.127 (0.055, 0.200) 1.250 (1.091, 1.431) NOx 0.056 (-0.065, 0.177) 0.023 (-0.026, 0.072) 1.040 (0.954, 1.134)
Table 3 footnote (abbreviations):
AP, attributable proportion due to interaction
RERI, relative excess risk due to interaction; SI, synergy index.
What the paper's own numbers show
Method. With RR10, RR01 and RR11 the relative risks of the pollution-only, gene-only and doubly exposed groups against the doubly unexposed group, the standard additive-interaction measures are RERI = RR11 − RR10 − RR01 + 1, AP = RERI / RR11 and SI = (RR11 − 1) / [(RR10 − 1) + (RR01 − 1)]. RERI is an excess relative risk, in units of the reference group's risk. AP is the fraction of the doubly exposed group's risk that is due to the interaction, which is what the abstract's wording describes. The paper does not report RR11 for the binary groups used in Table 3, but it follows from the reported RERI and SI: the SI denominator equals (RR11 − 1) − RERI, so RR11 = 1 + SI · RERI / (SI − 1). I recovered RR11 this way for every Table 3 cell, recomputed AP = RERI / RR11, and compared both the tabled AP and the recomputed AP with the abstract's percentages. All numbers are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Share of AF risk attributable to interaction, PM10 × high PRS | Abstract; Discussion para 4 | 16.4% | 6.2% (Table 3 AP 0.062; RERI / RR11 = 0.164 / 2.641 = 0.0621) |
| Share of AF risk attributable to interaction, NO2 × high PRS | Abstract; Discussion para 4 | 35.1% | 12.7% (Table 3 AP 0.127; RERI / RR11 = 0.351 / 2.755 = 0.1274) |
| RERI, PM10 × high PRS | Table 3; Results para 2 | 0.164 (0.027, 0.302) | 100 × RERI = 16.4%, identical to the abstract figure |
| RERI, NO2 × high PRS | Table 3; Results para 2 | 0.351 (0.133, 0.568) | 100 × RERI = 35.1%, identical to the abstract figure |
| Implied joint RR11, PM10 / NO2 (binary WHO guideline cut-offs) | From Table 3 RERI and SI | not reported | 2.641 / 2.755 |
| AP, PM10 × high PRS | Table 3 | 0.062 (0.011, 0.113) | 0.0621; consistent |
| AP, NO2 × high PRS | Table 3 | 0.127 (0.055, 0.200) | 0.1274; consistent |
| AP, PM2.5 and NOx × high PRS | Table 3 | 0.023, 0.023 | 0.0233, 0.0228; consistent |
| AP, medium PRS, four pollutants | Table 3 | 0.038, 0.024, 0.032, 0.032 | 0.0377, 0.0244, 0.0316, 0.0314; consistent to input rounding |
| Abstract figure / Table 3 AP | Abstract vs Table 3 | — | 2.65 (PM10), 2.76 (NO2) |
| Abstract figure vs upper 95% limit of AP | Abstract vs Table 3 | 16.4%, 35.1% | Both lie above the AP upper limits (11.3%, 20.0%) |
All three Table 3 statistics in each cell agree with one another, and the abstract's percentages agree only with RERI. The share of AF risk attributable to the gene-pollution interaction, as the paper itself estimates it, is 6.2% for PM10 and 12.7% for NO2.
Figure
Figure 1. Left: the abstract's statement that "16.4% and 35.1%" of AF risk is attributable to the interaction (orange: the value as reported), with the matching Discussion sentence. Right: the Table 3 high-genetic-risk cells for PM10 and NO2. The connector shows that the abstract's figures repeat the RERI column digit for digit (orange outline), not the AP column, which the table footnote defines as the attributable proportion due to interaction. The blue outline marks the AP column, which the recomputation reproduces. Bottom: the joint relative risk RR11 recovered from the reported RERI and SI, and AP = RERI / RR11 (blue: recomputed from the paper's own numbers), which gives 6.2% and 12.7% against the abstract's 16.4% and 35.1%. All values are from the paper or printed by recompute.py.txt.
Figure 2. For each pollutant, the blue dot is the attributable proportion recomputed from Table 3's RERI and SI (6.2%, 12.7%) and the blue line is the 95% CI Table 3 reports for the AP (1.1 to 11.3%; 5.5 to 20.0%). The orange diamond is the share stated in the abstract (16.4%, 35.1%), which equals 100 × RERI. Both abstract figures lie outside the paper's own confidence intervals for the quantity they describe. The thin line marks 0, the null value for AP.
Why it matters
The attributable share is one of the abstract's headline quantitative results and is repeated in the Discussion, where it supports the argument for targeted prevention in people with high genetic risk who are highly exposed. Readers who take the figure from the abstract will think that roughly a sixth to a third of AF risk in the doubly exposed group comes from gene-pollution synergy, when the paper's own estimates are about a sixteenth to an eighth. Narrative reviews, risk communication and burden calculations that cite the abstract would carry the overstatement forward.
The qualitative findings stand. Table 3 shows positive additive interactions for PM10 and NO2 with high genetic risk, and the RERI, AP and SI intervals all exclude their null values, as the Methods' criterion requires. The per-IQR hazard ratios, the joint-effect estimates and the title, Significance statement and concluding paragraph, which mention additive interaction without these percentages, are not affected.
Explanations considered
- The percentages are APs from another model, group or category. They match the high-PRS × high-pollution RERIs in Table 3 to every digit (0.164, 0.351). The only APs Table 3 gives for those cells are 0.062 and 0.127, and the medium-PRS APs are smaller (0.024, 0.032). No other table in the paper or in the SI Appendix (Tables S1 to S17) reports additive-interaction measures. This does not account for the numbers.
- The Table 3 AP column is the one in error. AP rebuilt from the reported RERI and SI is 0.0621 and 0.1274, matching the tabled 0.062 and 0.127; the same check holds for all eight cells. The column order (RERI, AP, SI) is also confirmed in the publisher's XML. The table is internally consistent, so the AP column is not the error.
- "RERI × 100%" is a stated convention for the share of risk. The Methods and Table 3 footnote define RERI as the relative excess risk due to interaction and AP as the attributable proportion due to interaction, and the Results read 0.164 and 0.351 as relative excess risks. RERI is measured in units of the reference group's risk; it is not a fraction of the doubly exposed group's risk. The abstract's "of AF risk could be attributable" and the Discussion's "of AF risk can be explained" describe AP. This does not account for the numbers.
- Another denominator. The proportion of the joint excess risk due to interaction, RERI / (RR11 − 1), is 10.0% and 20.0%. Dividing RERI by the Fig. 2 joint hazard ratios for high genetic risk and the highest pollution quintile (2.817, 2.702), which use a different pollution grouping, gives 5.8% and 13.0%, or 9.0% and 20.6% with HR − 1. Using the Results text's PM10 value (2.871) instead gives 5.7% and 8.8%. None gives 16.4% and 35.1%.
- A text-conversion artifact. The publisher-supplied full text in PMC and the Europe PMC JATS XML carry the same abstract and Discussion sentences and the same Table 3 cells. The figures are not a conversion artifact.
A separate inconsistency lies outside this claim. Results paragraph 2 gives the PM10 joint hazard ratio (high pollution, high genetic risk) as 2.871 (2.579, 3.076), which would be 187.1% higher risk. The paper's own Fig. 2 (PM10 panel, row "High genetic risk, Q5") prints 2.817 (2.579, 3.076), which matches the abstract's 181.7% and the geometric midpoint of the interval (2.817). For PM2.5, NO2 and NOx the Results text, Fig. 2 and the abstract agree exactly. Only the Results text differs from the paper's own figure for PM10; 2.871 looks like a transposition of 2.817.
Status
As of 2026-10-24:
- Version of record. The PNAS page returned HTTP 403 to automated access after the doi.org redirect, so the publisher-supplied text was read from the PMC article and the Europe PMC JATS full-text XML. Both match the PMC archive in the abstract sentence, the Discussion sentence, the Results RERIs and every Table 3 cell. The publisher-deposited full text in PMC and Europe PMC carries the error; the PNAS page itself could not be accessed.
- Supplements. The SI Appendix (Appendix 01, 23 pages, Tables S1 to S17) has no RERI, AP or SI results and no other version of the 16.4% / 35.1% figures. It does not change or explain the discrepancy.
- Corrections. Crossref update metadata (no update-to or updated-by relations; a search for works updating this DOI returned none), PMC and Europe PMC notices (
src/known.py), and the PubMed record (PMID 37523535, no CommentsCorrections links) show no correction, erratum, expression of concern or retraction. The Europe PMC search API was unavailable (HTTP 503) on three tries; its full-text XML was readable. - PubPeer and other discussion. PubPeer refused direct fetches (HTTP 403); web searches found no PubPeer thread and no Retraction Watch coverage. The only outside commentary found, a Science Media Centre expert reaction, concerns exposure modelling and does not address this issue.
- Sources. doi.org/10.1073/pnas.2302708120 and pnas.org (403); PMC10410743 (text, Table 3, Fig. 2 image pnas.2302708120fig02.jpg, SI Appendix); Europe PMC full-text XML and supplementary files; Crossref REST API; PubMed; web searches for PubPeer and Retraction Watch.
Suggested correction
Abstract: replace "with around 16.4% and 35.1% of AF risk could be attributable to the interactive effects" with "with around 6.2% and 12.7% of AF risk among participants with high exposure and high genetic risk attributable to the interactive effects (relative excess risks due to interaction 0.164 and 0.351)". Discussion paragraph 4: replace "Approximately 16.4 to 35.1% of AF risk can be explained by the additive effects" with "Approximately 6.2 to 12.7% of AF risk in this group can be explained by the additive interaction". Results paragraph 2: replace the PM10 joint hazard ratio "2.871 (2.579, 3.076)" with "2.817 (2.579, 3.076)", the value shown in Fig. 2 and implied by the abstract's 181.7%.
Recomputation script (python recompute.py.txt)
"""Recompute the "share of AF risk attributable to interaction" stated in the abstract
and Discussion of PMC10410743 (Proc Natl Acad Sci U S A 2023, doi:10.1073/pnas.2302708120).
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes, next to
this file, fig1.svg/png and fig2.svg/png.
Inputs are copied from the paper as archived in PubMed Central (identical in the
publisher's full text):
Abstract: "with around 16.4% and 35.1% of AF risk could be attributable to the
interactive effects" (PM10, NO2)
Discussion para 4: "Approximately 16.4 to 35.1% of AF risk can be explained by the
additive effects ..."
Table 3: RERI, AP and SI (95% CIs), high genetic risk x high pollution (WHO guideline
cut-offs for PM2.5, PM10 and NO2; median for NOx). The table does not state the
reference group; the standard additive-interaction reference (low genetic risk x
low pollution) is assumed.
Fig. 2 (forest plot, row "High genetic risk, Q5" of each pollutant panel): joint HRs,
highest pollution quintile x high genetic risk, versus low genetic risk x Q1.
Fig. 2 prints PM10 as 2.817 (2.579, 3.076).
Results para 2: the same joint HRs in the text, where PM10 is given as
2.871 (2.579, 3.076); the other three pollutants match Fig. 2.
Abstract: "% higher risk" for the same joint groups
Additive-interaction measures (Rothman), with RR10, RR01, RR11 the relative risks of the
exposed-only, gene-only and doubly exposed groups versus the doubly unexposed group:
RERI = RR11 - RR10 - RR01 + 1
AP = RERI / RR11 (attributable proportion due to interaction)
SI = (RR11 - 1) / [(RR10 - 1) + (RR01 - 1)]
Since (RR10 - 1) + (RR01 - 1) = (RR11 - 1) - RERI, SI = (RR11 - 1) / (RR11 - 1 - RERI),
so RR11 = 1 + SI * RERI / (SI - 1). The paper does not report RR11 for the Table 3
binary groups; it is recovered here from the reported RERI and SI.
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
# Table 3, high genetic risk x high pollution: (estimate, lower, upper)
T3_HIGH = {
"PM2.5": {"RERI": (0.057, -0.064, 0.178), "AP": (0.023, -0.026, 0.073), "SI": (1.041, 0.954, 1.135)},
"PM10": {"RERI": (0.164, 0.027, 0.302), "AP": (0.062, 0.011, 0.113), "SI": (1.111, 1.015, 1.216)},
"NO2": {"RERI": (0.351, 0.133, 0.568), "AP": (0.127, 0.055, 0.200), "SI": (1.250, 1.091, 1.431)},
"NOx": {"RERI": (0.056, -0.065, 0.177), "AP": (0.023, -0.026, 0.072), "SI": (1.040, 0.954, 1.134)},
}
# Table 3, medium genetic risk x high pollution
T3_MED = {
"PM2.5": {"RERI": 0.058, "AP": 0.038, "SI": 1.121},
"PM10": {"RERI": 0.040, "AP": 0.024, "SI": 1.067},
"NO2": {"RERI": 0.050, "AP": 0.032, "SI": 1.094},
"NOx": {"RERI": 0.048, "AP": 0.032, "SI": 1.100},
}
ABSTRACT_SHARE = {"PM10": 16.4, "NO2": 35.1} # percent, abstract and Discussion
# Fig. 2, row "High genetic risk, Q5": joint HR, highest pollution quintile x high genetic risk
JOINT_HR = {"PM2.5": (2.492, 2.290, 2.712), "PM10": (2.817, 2.579, 3.076),
"NO2": (2.702, 2.475, 2.949), "NOx": (2.572, 2.358, 2.805)}
# Results para 2 text: identical to Fig. 2 except PM10, which the text gives as 2.871
RESULTS_TEXT_HR = {**JOINT_HR, "PM10": (2.871, 2.579, 3.076)}
ABSTRACT_HIGHER = {"PM2.5": 149.2, "PM10": 181.7, "NO2": 170.2, "NOx": 157.2} # percent
# ---------------------------------------------------------------- recomputation
def rr11_from(reri, si):
return 1 + si * reri / (si - 1)
print("1. Table 3, high genetic risk x high pollution: implied RR11 and AP")
print(f"{'':6s} {'RERI':>6s} {'SI':>6s} {'RR11':>6s} {'AP tab':>7s} {'RERI/RR11':>9s} "
f"{'RR10+RR01-2':>11s}")
REC = {}
for k, d in T3_HIGH.items():
reri, si, ap = d["RERI"][0], d["SI"][0], d["AP"][0]
rr11 = rr11_from(reri, si)
ap_rec = reri / rr11
sep = rr11 - 1 - reri # (RR10-1)+(RR01-1)
REC[k] = dict(rr11=rr11, ap=ap_rec)
print(f"{k:6s} {reri:6.3f} {si:6.3f} {rr11:6.3f} {ap:7.3f} {ap_rec:9.4f} {sep:11.3f}")
print("\n Same check, medium genetic risk x high pollution (Table 3)")
for k, d in T3_MED.items():
rr11 = rr11_from(d["RERI"], d["SI"])
print(f" {k:6s} RERI={d['RERI']:.3f} SI={d['SI']:.3f} RR11={rr11:.3f} "
f"AP table={d['AP']:.3f} RERI/RR11={d['RERI'] / rr11:.4f}")
print("\n2. Abstract 'share of AF risk attributable to the interactive effects'")
for k, v in ABSTRACT_SHARE.items():
reri, ap = T3_HIGH[k]["RERI"][0], T3_HIGH[k]["AP"][0]
lo, hi = T3_HIGH[k]["AP"][1:]
print(f" {k}: stated {v:.1f}% | 100 x RERI = {100 * reri:.1f}% | Table 3 AP = "
f"{100 * ap:.1f}% (95% CI {100 * lo:.1f} to {100 * hi:.1f}%) | recomputed AP = "
f"{100 * REC[k]['ap']:.2f}% | stated / Table 3 AP = {v / (100 * ap):.2f}x | "
f"stated above AP upper limit: {v > 100 * hi}")
print("\n3. Alternative readings of 'share of risk' (none gives 16.4% / 35.1%)")
for k in ABSTRACT_SHARE:
reri = T3_HIGH[k]["RERI"][0]
rr11 = REC[k]["rr11"]
hr_fig2 = JOINT_HR[k][0]
print(f" {k}: RERI/(RR11-1) [share of joint excess] = {100 * reri / (rr11 - 1):.1f}% ; "
f"RERI/Fig. 2 joint HR ({hr_fig2:.3f}) = {100 * reri / hr_fig2:.1f}% ; "
f"RERI/(Fig. 2 HR - 1) = {100 * reri / (hr_fig2 - 1):.1f}%")
print("\n Same, using the PM10 value in the Results text (2.871) instead of Fig. 2")
reri, hr_txt = T3_HIGH["PM10"]["RERI"][0], RESULTS_TEXT_HR["PM10"][0]
print(f" PM10: RERI/{hr_txt:.3f} = {100 * reri / hr_txt:.1f}% ; "
f"RERI/({hr_txt:.3f} - 1) = {100 * reri / (hr_txt - 1):.1f}%")
print("\n4. Side note: joint HRs in Fig. 2 and the Results text vs abstract '% higher risk'")
for k, (hr, lo, hi) in JOINT_HR.items():
hr_txt = RESULTS_TEXT_HR[k][0]
print(f" {k:6s} Fig. 2 HR {hr:.3f} ({lo:.3f}, {hi:.3f}) -> {100 * (hr - 1):.1f}% higher; "
f"Results text HR {hr_txt:.3f} -> {100 * (hr_txt - 1):.1f}% higher; "
f"abstract {ABSTRACT_HIGHER[k]:.1f}%; geometric CI midpoint {math.sqrt(lo * hi):.3f}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: abstract text vs Table 3 columns, with the recomputation
fig = plt.figure(figsize=(7.2, 4.8), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.8)
ax.axis("off")
fig.canvas.draw()
REND = fig.canvas.get_renderer()
def seq(x, y, parts, **kw):
"""Draw text pieces left to right, each starting where the previous one ended."""
inv = ax.transData.inverted()
for s, style in parts:
t = ax.text(x, y, s, va="center", **{**kw, **style})
bb = t.get_window_extent(renderer=REND)
x = inv.transform((bb.x1, bb.y0))[0]
return x
def panel(x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec="#c9c8c3", lw=1))
ax.text(x + 0.15, y + h - 0.2, title, fontsize=10.5, fontweight="bold", va="top")
# left panel: abstract and Discussion
panel(0.2, 1.85, 2.75, 2.75, "Abstract")
ax.text(0.35, 4.05, "\u201c... with around", fontsize=10, va="top", color=SEC)
ax.text(0.35, 3.72, "16.4% and 35.1%", fontsize=13, fontweight="bold", va="top", color=ORANGE)
ax.text(0.35, 3.32, "of AF risk could be attributable\nto the interactive effects\u201d",
fontsize=10, va="top", color=SEC, linespacing=1.4)
ax.text(0.35, 2.62, "Discussion para 4: \u201c16.4 to 35.1%\nof AF risk can be explained by\n"
"the additive effects ...\u201d", fontsize=10, va="top", color=SEC, linespacing=1.4)
# right panel: Table 3
panel(4.05, 1.85, 2.95, 2.75, "Table 3, high genetic risk")
ax.text(4.2, 4.15, "\u00d7 high pollution", fontsize=10, va="top", color=SEC)
cols = [("RERI", 5.15), ("AP", 5.85), ("SI", 6.55)]
for name, cx in cols:
ax.text(cx, 3.72, name, ha="center", fontsize=10, fontweight="bold", va="top")
rows = [("PM\u2081\u2080", 3.2, "PM10"), ("NO\u2082", 2.8, "NO2")]
for lab, ry, key in rows:
d = T3_HIGH[key]
ax.text(4.2, ry, lab, fontsize=10.5, va="center")
ax.text(5.15, ry, f"{d['RERI'][0]:.3f}", ha="center", va="center", fontsize=10.5,
color=ORANGE, fontweight="bold")
ax.text(5.85, ry, f"{d['AP'][0]:.3f}", ha="center", va="center", fontsize=10.5)
ax.text(6.55, ry, f"{d['SI'][0]:.3f}", ha="center", va="center", fontsize=10.5)
ax.text(4.2, 2.42, "AP = attributable proportion due\nto interaction (table footnote)",
fontsize=10, va="top", color=SEC, linespacing=1.4)
ax.add_patch(FancyBboxPatch((4.83, 2.6), 0.64, 0.8, boxstyle="round,pad=0,rounding_size=0.05",
fc="none", ec=ORANGE, lw=1.5))
ax.add_patch(FancyBboxPatch((5.55, 2.6), 0.6, 0.8, boxstyle="round,pad=0,rounding_size=0.05",
fc="none", ec=BLUE, lw=1.5))
# connector abstract -> RERI column
ax.add_patch(FancyArrowPatch((2.3, 3.6), (4.83, 3.15), arrowstyle="-|>", mutation_scale=14,
color=ORANGE, lw=2, connectionstyle="arc3,rad=-0.2"))
ax.text(3.5, 3.05, "same digits\nas RERI,\nnot AP", ha="center", va="top", fontsize=10,
color=ORANGE, fontweight="bold", linespacing=1.3)
# bottom band: recomputation
ax.text(0.2, 1.55, "Recomputed from Table 3: RR\u2081\u2081 = 1 + SI\u00b7RERI / (SI \u2212 1), "
"AP = RERI / RR\u2081\u2081", fontsize=10, fontweight="bold", va="top")
for i, key in enumerate(("PM10", "NO2")):
d, r = T3_HIGH[key], REC[key]
lab = "PM\u2081\u2080" if key == "PM10" else "NO\u2082"
y = 1.05 - 0.4 * i
plain, sec = dict(fontsize=10.5), dict(fontsize=10.5, color=SEC)
seq(0.35, y, [(f"{lab}: RR\u2081\u2081 = {r['rr11']:.3f}, AP = {d['RERI'][0]:.3f} / "
f"{r['rr11']:.3f} = ", plain),
(f"{100 * r['ap']:.1f}%", dict(fontsize=11.5, fontweight="bold", color=BLUE)),
(" abstract: ", sec),
(f"{ABSTRACT_SHARE[key]:.1f}%", dict(fontsize=10.5, fontweight="bold",
color=ORANGE)),
(f" ({ABSTRACT_SHARE[key] / (100 * d['AP'][0]):.2f}\u00d7)", sec)])
ax.text(0.2, 0.2, "Orange: value as reported in the abstract. Blue: recomputed from the "
"paper's own Table 3.", fontsize=10, color=SEC, va="center")
save(fig, "fig1")
# Figure 2: dot-and-interval plot on a percentage axis
fig, ax = plt.subplots(figsize=(7.2, 3.0), facecolor=BG)
ax.set_facecolor(BG)
keys = ["PM10", "NO2"]
labels = {"PM10": "PM₁₀ × high PRS", "NO2": "NO₂ × high PRS"}
for i, k in enumerate(keys):
y = len(keys) - 1 - i
ap, lo, hi = (100 * v for v in T3_HIGH[k]["AP"])
ax.plot([lo, hi], [y - 0.13] * 2, color=BLUE, lw=2, solid_capstyle="butt")
ax.plot([REC[k]["ap"] * 100], [y - 0.13], "o", ms=9, color=BLUE, zorder=3)
ax.plot([ABSTRACT_SHARE[k]], [y + 0.13], "D", ms=9, color=ORANGE, zorder=3)
ax.text(hi + 0.6, y - 0.13, f"AP {100 * REC[k]['ap']:.1f}% (Table 3 95% CI {lo:.1f}–{hi:.1f}%)",
va="center", fontsize=10, color=BLUE)
ax.text(ABSTRACT_SHARE[k] + 1.2, y + 0.13, f"abstract {ABSTRACT_SHARE[k]:.1f}% (= 100 × RERI)",
va="center", fontsize=10, color=ORANGE)
ax.axvline(0, color=SEC, lw=0.8)
ax.set_yticks(range(len(keys)))
ax.set_yticklabels([labels[k] for k in reversed(keys)], fontsize=10.5, color=TXT)
ax.set_ylim(-0.6, len(keys) - 0.4)
ax.set_xlim(-1, 60)
ax.set_xlabel("Share of AF risk in the doubly exposed group attributable to interaction (%)",
fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
fig.tight_layout()
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Table 3's extreme-rainfall columns duplicate the coldwave columns and contradict the text's Cox's Bazar finding
Proc Natl Acad Sci U S A (2023) · doi:10.1073/pnas.2206189120 · PMC10801877
Summary
Table 3 reports, for each of 20 refugee settlements, the national average, the settlement value and their difference for three rapid-onset indicators: heatwaves, coldwaves and extreme rainfall. In the version archived in PubMed Central, every cell of the Extreme rainfall block is identical to the matching cell of the Coldwaves block (20 of 20 rows, 60 of 60 cells). The block is only labelled extreme rainfall. Its values are the coldwave values. As printed, they also run against the paper's text: Cox's Bazar is 0.33 days below the national average (rank 17 of 20), while the abstract, Significance statement, Results and Discussion describe extreme-rainfall exposure there as relatively high (SNR +0.33). The rapid-onset finding singled out in the abstract therefore has no valid settlement-level descriptive data behind it in the paper.
What the paper reports
The headline claim about extreme rainfall:
"We find that refugee populations in the settlements are highly exposed to extreme weather conditions, particularly low rainfall, high temperatures, and extreme rainfall (in the case of Bangladesh)."
— Significance statement
"Our findings for rapid-onset events—heatwaves, coldwaves, and extreme rainfall—are less conclusive compared to country trends, although we find relatively high exposure to extreme rainfall in Cox’s Bazar, Bangladesh."
— Abstract
The text describing the descriptive statistics in Table 3:
"Finally, several settlement locations had extreme rainfall patterns, which were higher than the population-weighted national average, with particularly high scores in Bangladesh."
— Results, Descriptive Statistics
The signal-to-noise results for extreme rainfall:
"Extreme rainfall, which can be used as a proxy for flooding, is most present in Bangladesh, even though we record an SNR of only 0.33. [...] For the other locations, we observe negative SNRs between -2.36 (Jordan) and -5.28 (Ethiopia), indicating that these settlements are in relatively dry, albeit hot regions as compared to the population-weighted national average."
— Results, Signal-to-Noise Analyses
"For extreme rainfall, which with poor water management infrastructure may result in flooding, we identified a significant and notable problem for refugees in Cox’s Bazaar, Bangladesh."
— Discussion
How the SNR's sign is defined:
"We calculate the SNR, a single numeric value, as a ratio between the mean and the SD of the difference: SNR = µ/σ. The SNR can be positive, negative, or equal to zero. A negative SNR means that the national average of a specific indicator is higher than that of the settlement ones. A positive SNR implies the opposite."
— Materials and Methods, Analyses
Table 3, as archived (one row of the 20; the full table is in recompute.py.txt):
| Country | Settlement | Heatwaves: national / settlement / difference | Coldwaves: national / settlement / difference | Extreme rainfall: national / settlement / difference |
|---|---|---|---|---|
| Bangladesh | Cox's Bazar | 1.56 / 1.36 / -0.20 | 1.62 / 1.29 / -0.33 | 1.62 / 1.29 / -0.33 |
"Notes: 1Mean annual max seasonal WSDI or CSDI from 1981-2010. 2 Mean annual number of days with extreme rainfall from 1981-2018."
— Table 3, footnote
What the paper's own numbers show
Method. All 20 rows of Table 3 were transcribed and the Extreme rainfall block was compared with the Coldwaves block cell by cell. In every block, Difference was recomputed as Settlement minus National average, which is the sign convention given in the Methods. Signs and ranks of the extreme-rainfall differences were then read off. Under the paper's definition, SNR = µ/σ with σ > 0, so an SNR always has the same sign as the mean difference. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Extreme rainfall block vs Coldwaves block | Table 3 | Two separately labelled blocks | Identical in 20 of 20 rows, 60 of 60 cells (national average, settlement and difference columns each 20/20) |
| Heatwaves block vs Coldwaves block, and vs Extreme rainfall block | Table 3 | — | 0 of 20 rows identical, so the table as a whole was not duplicated |
| Difference = settlement − national | Table 3, all three blocks | Difference column | Matches within 0.01 (rounding) in all 20 rows of each block, so each block is internally consistent |
| Cox's Bazar, extreme rainfall | Table 3 | 1.62 / 1.29 / -0.33 | 1.29 − 1.62 = −0.33: settlement 20.4% below national |
| Cox's Bazar rank by extreme-rainfall difference | Table 3 | "particularly high scores in Bangladesh" (Results) | 17th of 20 |
| Largest extreme-rainfall differences | Table 3 | — | Dadaab +0.85, Jamjang +0.78, Maban +0.55 |
| Direction for Bangladesh | Results, SNR paragraph; Abstract; Discussion | SNR +0.33, settlement above national | Table difference −0.33. A positive SNR needs a positive mean difference |
| Direction for the other locations | Results, SNR paragraph | All negative, −2.36 (Jordan) to −5.28 (Ethiopia) | 10 of the 19 settlements outside Bangladesh have positive differences. Kenya (+0.85, +0.30) and South Sudan (+0.55, +0.78) are positive at every settlement |
Figure
Figure 1. On the left, the paper's text: the abstract reports relatively high extreme-rainfall exposure in Cox's Bazar, and the Results give a Bangladesh SNR of +0.33 (settlement above national) and negative SNRs from −2.36 to −5.28 for all other locations. On the right, the Extreme rainfall block of Table 3 as printed (orange), with values recomputed from it (blue). Cox's Bazar is below the national average (−0.33, rank 17 of 20), 10 of the 19 other settlements are above it, and every row of the block equals the Coldwaves block. All numbers come from the paper's text and Table 3.
Figure 2. The Difference column of each Table 3 block, one row per settlement. The filled orange dots (extreme rainfall) sit exactly inside the open orange circles (coldwaves) in all 20 rows. The grey squares (heatwaves) are a different block of the same table, shown for contrast: they do not coincide, so only the extreme-rainfall block repeats another. Values are as printed in Table 3, plotted on an untruncated common scale.
Why it matters
- The abstract's only positive rapid-onset finding loses its tabulated support. The Significance statement and Abstract single out extreme rainfall in Cox's Bazar. Table 3 is the paper's only settlement-level descriptive table for rapid-onset events, and its extreme-rainfall block holds coldwave values. Read at face value, it shows the opposite sign for Cox's Bazar.
- Readers cannot recover the correct values from the paper. The paper gives no other settlement-level extreme-rainfall figures. Fig. 3 reports SNRs aggregated by country. A reader who uses Table 3, for example to compare flood-proxy exposure across settlements, gets coldwave durations.
- Several text statements cannot be checked against the table. These include "several settlement locations had extreme rainfall patterns, which were higher than the population-weighted national average" and the claim that the non-Bangladesh settlements are relatively dry. As printed, the table contradicts both.
- What is not affected. The slow-onset results (temperature and rainfall, Table 2) and the heatwave and coldwave blocks of Table 3 are distinct and internally consistent, so the paper's main conclusion about slow-onset exposure does not depend on this error. The country-level SNRs quoted in the text for extreme rainfall may well be right (see below). The problem is that Table 3 does not support them.
Explanations considered
- Text extraction or conversion artifact. Ruled out. The publisher-supplied JATS XML in Europe PMC (fullTextXML, fetched 2026-10-24) has the same duplicated values as the local PMC copy. Minus signs are present, and there are no misaligned cells and no decimal or superscript problems. The duplication is also already present upstream, in the results file of the authors' code repository (below), so it was not introduced in typesetting or archiving.
- The two indicators share a definition, so equal values are expected. Ruled out by the paper itself. Table 4 defines CSDI as the "Number of days per season with at least 6 consecutive days when TN < TN10th" and extreme rainfall as the "Number of days per 10 days when RR > 20mm". The Methods aggregate them differently: "yearly population-weighted maxima (WSDI and CSDI), and yearly population-weighted sums (extreme rainfall [...])". A temperature-based and a precipitation-based index, aggregated differently, cannot agree to two decimals in 60 of 60 cells.
- The text's SNRs and the table measure different things, so a sign difference is possible. This explains only part of the problem. The SNRs are aggregated by country, and the repository figures show that the table's Difference and the figure's mean difference are not computed identically (for coldwaves in Bangladesh, the table has −0.33 and the repository figure has −0.65, but both are negative and the settlement means agree, 1.29 vs 1.28). It does not explain why two different indicators have identical values. Nor does it explain a Bangladesh row, a single settlement, whose extreme-rainfall difference is negative while its SNR is positive.
- The table is right and the text is wrong. Unlikely. The repository's extreme-rainfall panel figure,
figures/floods.pdf, produced by the same plotting function as the other indicator figures, prints the SNRs quoted in the text: Bangladesh +0.33, Jordan −2.36, Ethiopia −5.28. For Bangladesh it gives a mean settlement value of 1.96 days and a mean difference of +0.12 days, not the 1.29 and −0.33 in Table 3. The text is therefore consistent with the authors' extreme-rainfall output. It is Table 3 that departs from it.
Status
As of 2026-10-24:
- Version of record. The publisher pages (pnas.org, reached via doi.org) returned HTTP 403 to automated requests, so the typeset HTML and PDF were not read directly. The publisher-supplied article XML deposited in PMC and served by Europe PMC carries Table 3 with the duplicated block. The PMC copy is the one quoted here.
- Supplements and data. The Data Availability statement points to a 4TU.ResearchData/figshare private link and to a public GitHub code repository (exposure-refugee-camps). The figshare link returned HTTP 202 (browser verification), and the figshare API returned 404 for the private link, so it could not be opened. In the GitHub repository (master branch),
results/descriptive_statistics.csvhas the same structure as Tables 2 and 3. Its Extreme rainfall columns equal its Coldwaves columns in all 60 cells, unrounded: for Bangladesh, 1.6177 / 1.2875 / −0.3302 in both.python recompute.py.txt path/to/descriptive_statistics.csvchecks this. The repository'sfigures/floods.pdfholds the distinct extreme-rainfall values described above. - Corrections. No correction, erratum or retraction notice was found in Crossref, PMC or Europe PMC.
- PubPeer. The PubPeer search API required human verification, so it could not be queried directly. A web search for the title found no PubPeer thread.
Sources: PMC10801877; Europe PMC full-text XML; doi:10.1073/pnas.2206189120; the code repository linked in the paper's Data Availability statement.
Suggested correction
- Replace the Extreme rainfall block of Table 3 (national average, settlement and difference for all 20 settlements) with the values from the extreme-rainfall computation that produced Fig. 3.
- Re-export the corresponding columns of
results/descriptive_statistics.csv. - Re-check the text statements that rest on Table 3 against the corrected values: the Descriptive Statistics paragraph ("several settlement locations [...] higher than the population-weighted national average, with particularly high scores in Bangladesh"), the Abstract and Significance statements on Cox's Bazar, and the Discussion.
- While doing this, reconcile the extreme-rainfall period. Table 3's footnote says "1981-2018" and Table 4 says "1981-2010".
Recomputation script (python recompute.py.txt)
"""Recompute the Table 3 checks for PMC10801877
(Proc Natl Acad Sci U S A 2023, doi:10.1073/pnas.2206189120,
"Refugee settlements are highly exposed to extreme weather conditions").
Run: python recompute.py.txt (paper numbers only)
python recompute.py.txt path/to/descriptive_statistics.csv
(also checks the CSV from the code
repository linked in the paper)
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PubMed Central:
Table 3 "Descriptive statistics rapid onset events": for 20 settlements, three blocks
(Heatwaves, Coldwaves, Extreme rainfall) of National average | Settlement | Difference.
Results, Signal-to-Noise Analyses: extreme-rainfall SNR 0.33 for Bangladesh; other
locations "negative SNRs between -2.36 (Jordan) and -5.28 (Ethiopia)".
Method: compare the Extreme rainfall block with the Coldwaves block cell by cell; recompute
Difference = Settlement - National average in every block; read off signs and ranks.
Under the paper's definition SNR = mu/sigma (sigma > 0), an SNR has the sign of the mean
difference.
"""
import csv
import os
import sys
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# Country | Settlement | HW nat | HW settl | HW diff | CW nat | CW settl | CW diff | ER nat | ER settl | ER diff
TABLE3 = """Bangladesh|Cox's Bazar|1.56|1.36|-0.20|1.62|1.29|-0.33|1.62|1.29|-0.33
Jordan|Zaatari|0.86|1.03|0.18|1.12|0.91|-0.21|1.12|0.91|-0.21
Rwanda|Mahama|0.20|0.71|0.50|0.14|0.10|-0.04|0.14|0.10|-0.04
Ethiopia|Nguenyyiel|1.29|0.65|-0.63|0.80|0.24|-0.56|0.80|0.24|-0.56
Ethiopia|Dolo Ado|1.29|1.23|-0.06|0.80|1.10|0.30|0.80|1.10|0.30
Kenya|Dadaab|1.20|0.81|-0.39|0.31|1.16|0.85|0.31|1.16|0.85
Kenya|Kakuma|1.20|2.39|1.19|0.31|0.61|0.30|0.31|0.61|0.30
Pakistan|Khyber Pakhtunkhwa|1.17|1.32|0.14|1.83|0.81|-1.02|1.83|0.81|-1.02
Pakistan|Balochistan|1.17|1.57|0.40|1.83|1.09|-0.74|1.83|1.09|-0.74
South Sudan|Maban|0.45|1.04|0.59|0.31|0.86|0.55|0.31|0.86|0.55
South Sudan|Jamjang|0.45|1.30|0.86|0.31|1.09|0.78|0.31|1.09|0.78
Sudan|White Nile|0.86|1.58|0.71|1.38|1.43|0.04|1.38|1.43|0.04
Sudan|East Dafur|0.86|1.20|0.34|1.38|1.38|-0.01|1.38|1.38|-0.01
Tanzania|Nyarugusu|1.49|0.58|-0.91|0.30|0.17|-0.13|0.30|0.17|-0.13
Tanzania|Nduta|1.49|0.88|-0.61|0.30|0.25|-0.05|0.30|0.25|-0.05
Uganda|Bidi Bidi|0.85|1.03|0.19|0.08|0.32|0.24|0.08|0.32|0.24
Uganda|Kyaka 2 & Rwamwanja|0.85|0.74|-0.10|0.08|0.12|0.04|0.08|0.12|0.04
Uganda|Nakivale|0.85|0.63|-0.22|0.08|0.04|-0.05|0.08|0.04|-0.05
Uganda|Kiryandongo|0.85|0.86|0.02|0.08|0.27|0.19|0.08|0.27|0.19
Uganda|Kyangwali|0.85|0.94|0.09|0.08|0.20|0.12|0.08|0.20|0.12"""
TEXT_SNR_BANGLADESH = 0.33 # Results, SNR paragraph
TEXT_SNR_OTHERS = (-2.36, -5.28) # Jordan, Ethiopia
countries, names, rows = [], [], []
for line in TABLE3.splitlines():
p = line.split("|")
countries.append(p[0])
names.append(p[1])
rows.append([float(x) for x in p[2:]])
A = np.array(rows)
HW, CW, ER = A[:, 0:3], A[:, 3:6], A[:, 6:9]
n = len(names)
ib = names.index("Cox's Bazar")
print("=== 1. Is the Extreme rainfall block a copy of the Coldwaves block? ===")
same_rows = np.all(ER == CW, axis=1)
print(f"settlements in Table 3: {n}")
print(f"rows where all 3 Extreme rainfall cells equal the Coldwaves cells: {int(same_rows.sum())} of {n}")
print(f"matching cells: {int((ER == CW).sum())} of {ER.size}")
for j, lab in enumerate(["National average", "Settlement", "Difference"]):
print(f" column '{lab}': {int((ER[:, j] == CW[:, j]).sum())} of {n} identical")
print(f"rows where Heatwaves block equals Coldwaves block: {int(np.all(HW == CW, axis=1).sum())} of {n}")
print(f"rows where Heatwaves block equals Extreme rainfall block: {int(np.all(HW == ER, axis=1).sum())} of {n}")
print("\n=== 2. Internal arithmetic: Difference = Settlement - National average ===")
for lab, B in [("Heatwaves", HW), ("Coldwaves", CW), ("Extreme rainfall", ER)]:
dev = np.abs((B[:, 1] - B[:, 0]) - B[:, 2])
print(f"{lab:17s}: max |(settl - nat) - printed diff| = {dev.max():.2f} "
f"(rows off by > 0.011: {int((dev > 0.011).sum())})")
print("\n=== 3. Cox's Bazar, extreme rainfall as printed ===")
nat, settl, diff = ER[ib]
print(f"national {nat:.2f}, settlement {settl:.2f}, printed difference {diff:+.2f}")
print(f"recomputed settlement - national = {settl - nat:+.2f} -> settlement BELOW national average")
print(f"settlement / national = {settl / nat:.3f} ({(settl / nat - 1) * 100:+.1f}%)")
order = np.argsort(-ER[:, 2], kind="stable")
rank = int(np.where(order == ib)[0][0]) + 1
print(f"rank of Cox's Bazar among {n} settlements by extreme-rainfall difference (1 = largest): {rank}")
print("three largest extreme-rainfall differences:",
", ".join(f"{names[k]} {ER[k, 2]:+.2f}" for k in order[:3]))
print(f"text: Bangladesh SNR = {TEXT_SNR_BANGLADESH:+.2f}; with SNR = mu/sigma and sigma > 0, "
f"a positive SNR needs a positive mean difference; Table 3 prints {diff:+.2f}")
print("\n=== 4. Other settlements, extreme rainfall as printed ===")
pos = ER[:, 2] > 0
print(f"settlements with a positive extreme-rainfall difference: {int(pos.sum())} of {n}")
print(" " + ", ".join(f"{names[k]} {ER[k, 2]:+.2f}" for k in np.where(pos)[0]))
others = [k for k in range(n) if k != ib]
print(f"positive among the {len(others)} settlements outside Bangladesh: {int(pos[others].sum())}")
print(f"text: other locations have negative SNRs from {TEXT_SNR_OTHERS[0]} to {TEXT_SNR_OTHERS[1]}")
for c in ["Kenya", "South Sudan", "Uganda", "Ethiopia"]:
ks = [k for k in range(n) if countries[k] == c]
print(f" {c}: printed differences {[f'{ER[k, 2]:+.2f}' for k in ks]}, "
f"mean {ER[ks, 2].mean():+.2f}")
# Optional: the repository CSV (results/descriptive_statistics.csv)
if len(sys.argv) > 1:
print("\n=== 5. Repository file results/descriptive_statistics.csv ===")
with open(sys.argv[1], newline="") as fh:
rdr = list(csv.reader(fh))
head = rdr[0]
cw_cols = [i for i, h in enumerate(head) if h == "Coldwaves"]
er_cols = [i for i, h in enumerate(head) if h == "Extreme rainfall"]
data = [r for r in rdr[3:] if len(r) > max(er_cols)]
eq = sum(float(r[a]) == float(r[b]) for r in data for a, b in zip(cw_cols, er_cols))
print(f"data rows: {len(data)}; Extreme rainfall cells equal to Coldwaves cells "
f"(unrounded): {eq} of {len(data) * len(er_cols)}")
bd = data[0]
print("first row (" + bd[0] + "): Coldwaves " + " / ".join(f"{float(bd[i]):.4f}" for i in cw_cols)
+ "; Extreme rainfall " + " / ".join(f"{float(bd[i]):.4f}" for i in er_cols))
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
# Figure 1: text versus Table 3
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 64)
ax.axis("off")
def panel(x, w, title, sub):
ax.add_patch(FancyBboxPatch((x, 7), w, 49, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(x + 1.5, 53.2, title, fontsize=11.5, weight="bold", va="top")
ax.text(x + 1.5, 49.6, sub, fontsize=10, color=INK2, va="top")
panel(1, 40, "Text", "Abstract; Results; Discussion")
panel(57, 42, "Table 3, Extreme rainfall block", "as printed (days), and recomputed")
# left panel: claims (reported, orange)
ax.text(2.5, 45.2, "Abstract:", fontsize=10, color=INK2, va="top")
ax.text(2.5, 42.2, "“relatively high exposure to extreme\nrainfall in Cox’s Bazar, Bangladesh”",
fontsize=10, color=ORANGE, va="top")
ax.text(2.5, 34.0, "Results, SNR paragraph:", fontsize=10, color=INK2, va="top")
ax.text(2.5, 31.0, f"Bangladesh SNR = {TEXT_SNR_BANGLADESH:+.2f}", fontsize=10.5,
color=ORANGE, weight="bold", va="top")
ax.text(2.5, 27.2, "(positive: settlement above national)", fontsize=10, color=INK2, va="top")
ax.text(2.5, 21.0, "Other locations: negative SNRs,", fontsize=10.5, color=ORANGE, va="top")
ax.text(2.5, 17.6, f"{TEXT_SNR_OTHERS[0]} to {TEXT_SNR_OTHERS[1]}", fontsize=10.5,
color=ORANGE, weight="bold", va="top")
ax.text(2.5, 13.8, "(settlements below national)", fontsize=10, color=INK2, va="top")
# right panel: table values (orange) and recomputation (blue)
ax.text(58.5, 45.2, "Cox’s Bazar: national / settlement / diff.", fontsize=10, color=INK2, va="top")
ax.text(58.5, 42.0, f"{nat:.2f} / {settl:.2f} / {diff:+.2f}", fontsize=10.5, color=ORANGE,
weight="bold", va="top")
ax.text(58.5, 38.4, f"{settl:.2f} − {nat:.2f} = {settl - nat:+.2f}: below national;",
fontsize=10, color=BLUE, va="top")
ax.text(58.5, 35.2, f"rank {rank} of {n} settlements", fontsize=10, color=BLUE, va="top")
ax.text(58.5, 29.0, "Other settlements, printed differences:", fontsize=10, color=INK2, va="top")
ax.text(58.5, 25.8, "largest: Dadaab +0.85, Jamjang +0.78",
fontsize=10, color=ORANGE, va="top")
ax.text(58.5, 22.4, f"{int(pos[others].sum())} of {len(others)} positive (outside Bangladesh)",
fontsize=10, color=BLUE, va="top")
ax.text(58.5, 16.6, "Every row equals the Coldwaves block:", fontsize=10, color=INK2, va="top")
ax.text(58.5, 13.4, f"{int(same_rows.sum())} of {n} rows, {int((ER == CW).sum())} of {ER.size} cells",
fontsize=10.5, color=BLUE, weight="bold", va="top")
def connector(y0, y1, label):
ax.add_patch(FancyArrowPatch((41.8, y0), (56.2, y1), arrowstyle="<|-|>", mutation_scale=11,
lw=1.5, color=INK, shrinkA=0, shrinkB=0, zorder=2))
ax.text(49, (y0 + y1) / 2, label, ha="center", va="center", fontsize=10, weight="bold",
zorder=3, bbox=dict(boxstyle="round,pad=0.25", fc=BG, ec="none"))
connector(29.6, 41.0, "opposite\nsign")
connector(19.0, 25.0, "opposite\nsign")
ax.text(1, 62.5, "The text and Table 3 disagree on the direction of extreme-rainfall exposure",
fontsize=12, weight="bold", va="top")
ax.text(1, 58.9, "Difference = settlement − population-weighted national average; "
"SNR = mean difference / SD", fontsize=10, color=INK2, va="top")
handles = [Line2D([], [], marker="s", ls="", ms=9, color=ORANGE, label="reported in the paper"),
Line2D([], [], marker="s", ls="", ms=9, color=BLUE,
label="recomputed from the paper’s own Table 3")]
ax.legend(handles=handles, loc="lower center", bbox_to_anchor=(0.5, -0.01), ncol=2,
frameon=False, fontsize=10, handletextpad=0.4, columnspacing=2.0)
save(fig, "fig1")
# Figure 2: the three blocks of Table 3, row by row
fig, ax = plt.subplots(figsize=(7.2, 6.4))
y = np.arange(n)[::-1]
ax.axvline(0, color=INK2, lw=1, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.scatter(HW[:, 2], y, marker="s", s=60, color="#a9a8a3", zorder=2,
label="Heatwaves block, Difference (for contrast)")
ax.scatter(CW[:, 2], y, marker="o", s=190, facecolors="none", edgecolors=ORANGE, linewidths=2,
zorder=3, label="Coldwaves block, Difference")
ax.scatter(ER[:, 2], y, marker="o", s=60, color=ORANGE, zorder=4,
label="Extreme rainfall block, Difference")
ax.set_yticks(y)
ax.set_yticklabels([f"{nm} ({c})" if k == ib else nm for k, (nm, c) in enumerate(zip(names, countries))])
for t in ax.get_yticklabels():
if t.get_text().startswith("Cox"):
t.set_fontweight("bold")
ax.set_xlim(-1.3, 1.3)
ax.set_ylim(-0.8, n - 0.2)
ax.set_xlabel("Settlement − national average, as printed in Table 3 (days)")
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.annotate(f"Cox’s Bazar: {diff:+.2f} in both blocks\n(text: above national, SNR {TEXT_SNR_BANGLADESH:+.2f})",
xy=(diff, y[ib]), xytext=(0.12, y[ib] - 0.9), fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.text(1.28, y[names.index("Nduta")] + 0.5, f"Extreme rainfall = Coldwaves\nin {int(same_rows.sum())} of {n} rows "
f"({int((ER == CW).sum())}/{ER.size} cells)", ha="right", va="center", fontsize=10,
color=BLUE, weight="bold")
ax.legend(loc="upper center", bbox_to_anchor=(0.42, 1.10), ncol=2, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Abstract's tissue-culture counts (1/6, 6/6) conflict with the paper's Results and design (2/8, 8/8)
Proc Natl Acad Sci U S A (2024) · doi:10.1073/pnas.2318627121 · PMC11331133
Summary
The abstract reports the study's postmortem tissue-culture result as MAP-positive tissue in "1 out of 6" phage-treated calves and "6 out of 6" infected control calves. The Methods and the Fig. 1 legend assign 8 calves to each infected group, and the Results, using the same 49-day tissue culture and duplex qPCR at the single necropsy (16 weeks post-inoculation), report 8 MAP-positive calves in the MAP-only group and 2 in the phage-treated group, which is 8/8 and 2/8, with no calf reported as excluded. The direction and significance of the protective effect are unchanged (recomputed Fisher exact p = 0.015 for the abstract's counts, 0.0070 for the Results'), but the abstract understates the infected calves in each group as 6 rather than 8, and puts residual infection in the phage-treated group at 16.7% (1 calf) rather than 25.0% (2 calves).
What the paper reports
Abstract, the postmortem result:
Postmortem tissue culture analysis further supported the effectiveness of MPs, with only 1 out of 6 animals in the phage-treated group testing positive for MAP colonized tissues compared to 6 out of 6 animals in the infected control group.
Results, "Oral Administration of Phages Prevented Establishment of MAP Infection in Intestinal Tissues", first paragraph:
To quantify the prevention of infection, MAP infection status was measured in intestinal tissues with high sensitivity (20) via duplex qPCR after samples were cultured for 49 d.
Within the MAP-only group, there were eight MAP-positive calves, each with 3 to 6 positive tissue types (Fig. 4A). However, in the phage-treated group, only two calves were MAP positive, with only one positive tissue type each.
Therefore, MP administration strongly reduced MAP-positive tissues (P value > 0.01) and provided almost complete protection against MAP infection.
Materials and Methods, "Study Design":
The first eight calves were designated the MAP-inoculated control group (MAP-only group). The next eight calves to arrive were designated the phage-treated MAP infection group (phage-treated group). The last three calves were designated the uninfected control (control).
Fig. 1 legend:
Newborn dairy calves (n = 19) were collected within 12 h of birth and randomly assigned into three groups: MAP-only (n = 8), phage-treated (n = 8), and control (n = 3).
Materials and Methods, "Inoculum":
Pellets were prepared per group (n = 8) per day.
Materials and Methods, "Necropsies and Tissue Cultures":
Calves were killed 16 wk postinoculation
Six tissue samples per calf were obtained: ileum, distal jejunum, ileocecal valve, and their associated lymph nodes.
The only "6 out of 8" in the paper concerns phage recovery, not MAP. Results, "Phages Were Persistent in the Small Intestine...":
MPs were detected in both the ileum and jejunum of 6 out of 8 of the phage-treated calves. The inability to recover MPs from the tissues of two calves was attributed to a documented sampling error with these specific samples
The only exclusions reported are for the blood assay. Results, first section:
Two samples were excluded from our results based on cell viability assessments (19).
The conclusion that rests on the tissue result, Discussion, first sentence:
This study demonstrated the ability of MPs to provide almost complete protection against MAP infection and the ability to eliminate fecal shedding by calves.
What the paper's own numbers show
Method. I took the group sizes from the Methods and the Fig. 1 legend and the numbers of MAP-positive calves from the Results text. I computed animal-level positivity for each group, with exact (Clopper–Pearson) 95% intervals, and did the same for the abstract's counts. For both versions I computed the risk difference, the relative reduction and a two-sided Fisher exact test on the 2×2 table. The paper itself reports only per-tissue chi-square tests, so the Fisher test is my addition and serves only to compare the two versions. I worked out which calves would have to be left out to turn the Results' counts into the abstract's. As a cross-check on the per-calf counts, I read the bar heights of the paper's Fig. 4A, which gives the number of positive animals for each tissue type. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Calves per group | Methods; Fig. 1 legend; Inoculum | MAP-only 8, phage-treated 8, control 3; total 19 | 8 + 8 + 3 = 19, consistent; 16 infected calves |
| Phage-treated calves with MAP-positive tissue | Abstract | 1 out of 6 | 16.7% (exact 95% CI 0.4–64.1%) |
| Phage-treated calves with MAP-positive tissue | Results, tissue paragraph | "only two calves were MAP positive" | 2/8 = 25.0% (exact 95% CI 3.2–65.1%) |
| Infected-control calves with MAP-positive tissue | Abstract | 6 out of 6 | 100% (exact 95% CI 54.1–100%) |
| MAP-only calves with MAP-positive tissue | Results, tissue paragraph | "eight MAP-positive calves" | 8/8 = 100% (exact 95% CI 63.1–100%) |
| Risk difference, phage minus control | derived | — | abstract −83.3 points; Results −75.0 points |
| Relative reduction in positive calves | derived | — | abstract 83.3%; Results 75.0% |
| Fisher exact test, two-sided | derived (not in the paper) | — | abstract [[1, 5], [6, 0]] p = 0.0152; Results [[2, 6], [8, 0]] p = 0.0070 |
| Calves that would have to be left out to get the abstract's counts | derived | — | phage-treated: 1 positive and 1 negative; MAP-only: 2 positives |
| Most MAP-only animals positive in any one tissue type | Fig. 4A (read from image) | 8 (jejunum LN) | consistent with 8 positive calves of 8 |
| MAP-only positive tissues, summed over the six types | Fig. 4A (read from image) | 7, 7, 6, 7, 8, 3 | 38 over 8 calves = 4.75 per calf, consistent with "3 to 6 positive tissue types" each |
| Phage-treated positive tissues | Fig. 4A (read from image) | jejunum 1, ileocecal valve 1 | 2 tissues in 2 different tissue types, consistent with "two calves ... one positive tissue type each" |
The Results text, the Methods group sizes and Fig. 4A agree with each other: 8 of 8 MAP-only calves and 2 of 8 phage-treated calves had at least one MAP-positive tissue. The abstract is the only place where the result appears as 1/6 and 6/6.
Three smaller inconsistencies turned up during the check. None affects the claim above or any conclusion.
| Quantity | Location | Reported | Recomputed / note |
|---|---|---|---|
| MAP-only fecal shedding "increased from 60 to 100%" | Results, fecal shedding | 60% | 60% of 8 = 4.8 calves, not a whole number. Fig. 3 shows 3/8 = 37.5% (F57) or 5/8 = 62.5% (IS900) at 10 wk PI, and 8/8 at 16 wk |
| Significance of the tissue reduction | Results, tissue paragraph | "(P value > 0.01)" | inconsistent with "strongly reduced"; Fig. 4A marks the significant tissue types with asterisks, so "<" was probably meant |
| qPCR positivity cutoff | Methods "qPCR Procedure"; Fig. 4 legend; Fig. 3 legend | "Ct values ≤ 37 considered positive"; "Ct < 37"; "Ct ... >37.0" | the Fig. 3 legend reverses the direction of the cutoff |
Figure
Figure 1. The abstract's sentence (left, orange) and the corresponding statements in the Results and Methods (right, blue), with the proportions each implies. The two panels refer to the same assay (49-day tissue culture followed by duplex F57/IS900 qPCR) and the same single necropsy at 16 weeks post-inoculation, and the paper reports no calf excluded from it. The numerators disagree (1 vs 2 positive phage-treated calves; 6 vs 8 positive infected controls), and so do the denominators (6 vs the 8 calves enrolled per group). The Fisher exact p-values at the bottom were recomputed from each pair of counts; the paper does not report them.
Figure 2. One square per calf. Filled squares are calves with MAP-positive tissue, hollow squares are calves without, and dashed outlines are calves enrolled in the group (n = 8, Methods and Fig. 1 legend) that the abstract's count leaves out. Orange rows are the abstract's counts. Blue rows are the Results' counts set against the Methods group size. Going from the Results to the abstract takes away one positive and one negative calf in the phage-treated group and two positive calves in the infected control group.
Why it matters
The postmortem tissue result is the study's direct measure of infection, and the abstract is where most readers, and abstracting services, will take it from. As published, the abstract:
- Misstates the sample size. It describes two groups of 6 animals (12 in total) where the study enrolled two infected groups of 8 (16 in total). A reader using the abstract would take the study to be smaller than it was.
- Overstates the protective effect somewhat. It puts residual infection in the phage-treated group at 1 calf (16.7%) instead of 2 (25.0%), and the relative reduction at 83.3% instead of 75.0%. With groups this small, each calf moves the proportion by 12.5 points.
- Does not change the qualitative conclusion. Both versions show a large reduction in the same direction, and both are significant by Fisher's exact test (p = 0.015 and p = 0.0070). The exact 95% intervals for the phage-treated proportion are wide either way (0.4–64.1% and 3.2–65.1%). The title, the Significance statement, the Discussion and the Conclusion state the effect qualitatively and are not affected. The fecal-shedding result is separate and is also not affected.
So this is an error in a headline number, not in the study's main finding. It matters for anyone who quotes the abstract's figures, pools them in a meta-analysis or uses them to plan a follow-up study's sample size.
Explanations considered
- Some calves were excluded from the tissue analysis. This does not account for the numbers. The paper reports no death, removal or exclusion of any calf. All calves were killed at 16 weeks post-inoculation, and six tissues were taken from each. The only exclusions mentioned are two or three blood samples for the IFN-γ assay. Even if exclusions had happened, getting from 2/8 to 1/6 means leaving out exactly one positive and one negative phage-treated calf, and getting from 8/8 to 6/6 means leaving out two positive controls. Nothing in the paper describes such a selection, and the Results text itself counts 8 and 2 positive calves.
- The abstract refers to the 6 phage-treated calves from which phages were recovered. This does not account for the numbers. That "6 out of 8" is phage recovery from tissue, with two calves lost to a sampling error. It cannot apply to the infected control group, which received no phage, and the paper does not report MAP status for that subset.
- The abstract uses a different assay. This does not account for the numbers. The abstract says "postmortem tissue culture analysis", which is the 49-day culture plus duplex qPCR behind the Results' 8 and 2 positive calves (Fig. 4A). The direct-extraction F57 data (Fig. 4B and 4C) show a similar per-tissue pattern and give no per-calf count that would give 1/6 or 6/6.
- The abstract uses a different time point. This does not account for the numbers. There is a single necropsy, at 16 weeks post-inoculation.
- The 6 refers to the six tissue types, not to animals. This partly matches the numbers. Fig. 4A shows positive tissue in all 6 tissue types in the MAP-only group, which would give "6 out of 6". In the phage-treated group, however, the two positive tissues were in two different tissue types (jejunum and ileocecal valve), so the same reading gives 2/6, not 1/6. The abstract also says "animals" explicitly. This reading therefore reconciles neither figure as written.
- A conversion or typesetting artifact. This is excluded. The numbers are plain inline integers and words ("1 out of 6", "eight", "two"). They appear identically in the JATS XML from Europe PMC, the PMC HTML page and the PMC archive used here.
- The error was corrected later. As of 2026-10-23 no correction has been issued (see Status).
Status
- Version of record. The abstract sentence and the Results sentences quoted above appear word for word in the Europe PMC full-text XML (the JATS version of record) and on the PMC HTML page, and they match the PMC archive used for this check. The publisher's page on pnas.org returned HTTP 403 to automated access. Every quote above was checked verbatim against the PMC-archived text. The Fig. 3A/3B and 4A counts were read from the PMC copies of the figure images (pnas.2318627121fig03.jpg and pnas.2318627121fig04.jpg).
- Supplements. The SI Appendix (pnas.2318627121.sapp.pdf, 2 pages, from the Europe PMC supplementary-files bundle) holds Supplementary Table 1, which lists phage presence in fecal and tissue samples for each animal. It numbers the calves 1–8 (MAP-only), 9–16 (phage-treated) and 17–19 (uninfected control), which confirms the group sizes of 8. It gives no MAP tissue results, so it does not reconcile the abstract with the Results.
- Corrections and comments. Crossref lists no update relations for the DOI. PMC and Europe PMC (PMID 39102547) list no correction, erratum, expression of concern or retraction. The PubPeer API and search page refused automated access (HTTP 403 / Cloudflare challenge), so PubPeer could not be checked directly. A web search for the title with "pubpeer" returned no PubPeer page.
- Sources. PMC article PMC11331133; Europe PMC full-text XML and supplementary files; Crossref; Europe PMC record PMID 39102547; web search. Checked as of 2026-10-23.
Suggested correction
- Change the abstract sentence to match the Results and the design, for example: "Postmortem tissue culture analysis further supported the effectiveness of MPs, with only 2 out of 8 animals in the phage-treated group testing positive for MAP colonized tissues compared to 8 out of 8 animals in the infected control group." If the abstract's figures came from a different analysis or subset, state that analysis and its denominators in the Methods and Results instead.
- Minor: correct "(P value > 0.01)" in the tissue paragraph, presumably to "<"; give the MAP-only fecal-shedding proportions as counts out of 8 (for example 5/8 at 10 weeks by IS900, or 3/8 by F57), since 60% is not a possible proportion of 8 calves; and change "Ct value of >37.0" in the Fig. 3 legend to match the Methods ("≤ 37").
Recomputation script (python recompute.py.txt)
"""Recompute the animal-level tissue-culture result of PMC11331133
(Proc Natl Acad Sci U S A 2024, doi:10.1073/pnas.2318627121).
Run: python recompute.py.txt
Needs scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract: "only 1 out of 6 animals in the phage-treated group testing positive
for MAP colonized tissues compared to 6 out of 6 animals in the
infected control group"
Results (tissues): "Within the MAP-only group, there were eight MAP-positive calves ...
in the phage-treated group, only two calves were MAP positive"
Methods, Fig. 1: MAP-only n = 8, phage-treated n = 8, control n = 3 (n = 19)
Results (feces): MAP-only shedding "increased from 60 to 100%"
"""
import os
import warnings
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from scipy.stats import beta, fisher_exact # noqa: E402
warnings.filterwarnings("ignore", message=".*findfont.*")
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
GROUP_N = {"MAP-only": 8, "phage-treated": 8, "control": 3} # Methods; Fig. 1 legend
TOTAL_STATED = 19 # Methods; Fig. 1 legend
ABSTRACT = {"phage-treated": (1, 6), "MAP-only": (6, 6)} # (positive, total)
RESULTS_POS = {"phage-treated": 2, "MAP-only": 8} # positive calves
RESULTS = {g: (RESULTS_POS[g], GROUP_N[g]) for g in RESULTS_POS} # total = group size
SHEDDING_START_PCT = 60 # Results, fecal shedding
# Read from the bar heights of the paper's Fig. 4A image (pnas.2318627121fig04.jpg, PMC):
# number of MAP-positive animals per tissue type after 49-d culture + duplex qPCR.
TISSUES = ("ileum", "jejunum", "ileocecal valve", "ileum LN", "jejunum LN", "ileocecal valve LN")
FIG4A = {"MAP-only": (7, 7, 6, 7, 8, 3), "phage-treated": (0, 1, 1, 0, 0, 0)}
# Read from the paper's Fig. 3A/3B image (pnas.2318627121fig03.jpg): MAP-only calves with
# positive fecal culture at 10 wk PI (first week of active shedding) and 16 wk PI.
FIG3_MAP_ONLY = {"F57": {10: 3, 16: 8}, "IS900": {10: 5, 16: 8}}
def cp_ci(k, n, a=0.05):
"""Clopper-Pearson exact 95% interval for k/n."""
lo = 0.0 if k == 0 else beta.ppf(a / 2, k, n - k + 1)
hi = 1.0 if k == n else beta.ppf(1 - a / 2, k + 1, n - k)
return lo, hi
def summarise(label, d):
kp, np_ = d["phage-treated"]
kc, nc = d["MAP-only"]
pp, pc = kp / np_, kc / nc
table = [[kp, np_ - kp], [kc, nc - kc]]
p = fisher_exact(table)[1]
print(f"\n{label}")
for g, (k, n) in (("phage-treated", (kp, np_)), ("MAP-only / infected control", (kc, nc))):
lo, hi = cp_ci(k, n)
print(f" {g:28s} {k}/{n} = {100 * k / n:5.1f}% exact 95% CI {100 * lo:4.1f}-{100 * hi:5.1f}%")
print(f" risk difference (phage - control) = {100 * (pp - pc):.1f} percentage points")
print(f" relative reduction 1 - p_phage/p_control = {100 * (1 - pp / pc):.1f}%")
print(f" 2x2 table {table}: Fisher exact two-sided p = {p:.4f}")
return p
print("=== Design (Methods, Fig. 1 legend)")
s = sum(GROUP_N.values())
print(f"Group sizes {GROUP_N}; sum = {s} (stated {TOTAL_STATED}); consistent = {s == TOTAL_STATED}")
print(f"Infected calves (MAP-only + phage-treated) = {GROUP_N['MAP-only'] + GROUP_N['phage-treated']}")
print("\n=== Animal-level tissue-culture positivity")
p_abs = summarise("Abstract", ABSTRACT)
p_res = summarise("Results, with the Methods group sizes", RESULTS)
print("\n=== What the abstract's figures would require")
for g in ("phage-treated", "MAP-only"):
ka, na = ABSTRACT[g]
kr, nr = RESULTS[g]
drop_pos, drop_neg = kr - ka, (nr - kr) - (na - ka)
print(f"{g}: from {kr}/{nr} to {ka}/{na} requires leaving out {drop_pos} positive "
f"and {drop_neg} negative calves ({nr - na} calves in total)")
print("Calves in the abstract's two groups: 6 + 6 = 12, versus 8 + 8 = 16 infected calves enrolled")
print("\n=== Tissue-type reading (Fig. 4A, read from the image)")
for g in ("MAP-only", "phage-treated"):
c = FIG4A[g]
types = sum(1 for x in c if x > 0)
print(f"{g}: positive tissue types {types}/6; most animals positive in one tissue type = "
f"{max(c)}; positive tissues summed over types = {sum(c)}")
print(f"MAP-only: {sum(FIG4A['MAP-only'])} positive tissues over 8 calves = "
f"{sum(FIG4A['MAP-only']) / 8:.2f} per calf (Results: 3 to 6 each)")
print("Phage-treated: 2 positive tissues in 2 different tissue types (jejunum, ileocecal valve),"
" so a tissue-type count would be 2/6, not 1/6")
print("\n=== Side check: fecal shedding 'from 60 to 100%' in a group of 8")
print("Possible percentages of 8 calves:", ", ".join(f"{k}/8 = {100 * k / 8:.1f}%" for k in range(9)))
print(f"{SHEDDING_START_PCT}% of 8 = {SHEDDING_START_PCT / 100 * 8:.1f} calves (not a whole number); "
f"nearest are 5/8 = 62.5% and 4/8 = 50.0%")
for gene, d in FIG3_MAP_ONLY.items():
print(f"Fig. 3 {gene}: MAP-only calves shedding at 10 wk PI = {d[10]}/8 = {100 * d[10] / 8:.1f}%, "
f"at 16 wk PI = {d[16]}/8 = {100 * d[16] / 8:.1f}%")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
print("wrote", name + ".svg and", name + ".png")
# Figure 1: abstract vs Results/Methods, two text panels joined by a connector
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.9)
ax.axis("off")
def panel(x, w, colour, title, sub, lines):
y0, h = 1.2, 3.25
ax.add_patch(FancyBboxPatch((x, y0), w, h, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=colour, lw=1.5))
ax.add_patch(Rectangle((x, y0 + h - 0.08), w, 0.08, fc=colour, ec="none"))
ax.text(x + 0.15, y0 + h - 0.32, title, fontsize=12, fontweight="bold", va="center")
ax.text(x + 0.15, y0 + h - 0.58, sub, fontsize=10, color=INK2, va="center")
y = y0 + h - 0.9
for kind, txt in lines:
if kind == "q":
ax.text(x + 0.15, y, txt, fontsize=10, color=INK, va="top", style="italic",
linespacing=1.35)
y -= 0.2 * (txt.count("\n") + 1) + 0.2
else:
ax.text(x + 0.15, y, txt, fontsize=11.5, color=colour, va="top", fontweight="bold")
y -= 0.36
panel(0.2, 3.0, ORANGE, "Abstract", "as reported",
[("q", "“only 1 out of 6 animals in the\nphage-treated group testing positive\n"
"for MAP colonized tissues compared\nto 6 out of 6 animals in the\n"
"infected control group”"),
("n", "phage-treated: 1/6 = 16.7%"),
("n", "infected control: 6/6 = 100%")])
panel(4.0, 3.0, BLUE, "Results + Methods", "paper's own counts",
[("q", "“there were eight MAP-positive\ncalves” (MAP-only group)\n"
"“only two calves were MAP positive”\n(phage-treated group)\n"
"Group sizes (Methods, Fig. 1):\nMAP-only n = 8, phage-treated n = 8"),
("n", "phage-treated: 2/8 = 25.0%"),
("n", "MAP-only: 8/8 = 100%")])
ax.annotate("", xy=(4.0, 2.85), xytext=(3.2, 2.85),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2, shrinkA=0, shrinkB=0))
ax.text(3.6, 3.05, "conflict", ha="center", fontsize=10, fontweight="bold")
ax.text(3.6, 0.6, "Same assay (49-d tissue culture + duplex qPCR), same single necropsy at 16 wk PI,\n"
"no exclusions reported. Numerators differ (1 vs 2; 6 vs 8) and denominators differ (6 vs 8).\n"
f"Both versions are significant: Fisher exact p = {p_abs:.3f} (abstract) vs {p_res:.4f} (Results).",
ha="center", va="center", fontsize=10, color=INK2, linespacing=1.4)
ax.text(0.2, 4.68, "Calves with MAP-positive tissue at necropsy: abstract vs body of the paper",
fontsize=12, fontweight="bold", va="center")
save(fig, "fig1")
# Figure 2: one square per calf, so numerator and denominator are both visible
rows = [
("Phage-treated, abstract", ABSTRACT["phage-treated"], ORANGE),
("Phage-treated, Results", RESULTS["phage-treated"], BLUE),
("Infected control, abstract", ABSTRACT["MAP-only"], ORANGE),
("MAP-only, Results", RESULTS["MAP-only"], BLUE),
]
fig, ax = plt.subplots(figsize=(7.2, 3.2))
fig.subplots_adjust(left=0.30, right=0.80, top=0.80, bottom=0.17)
size, gap = 0.8, 0.2
for i, (lab, (k, n), col) in enumerate(rows):
y = [4.0, 3.0, 1.4, 0.4][i]
for j in range(8):
if j < n:
filled = j < k
ax.add_patch(Rectangle((j, y - size / 2), size, size, fc=col if filled else "white",
ec=col, lw=2))
else:
ax.add_patch(Rectangle((j, y - size / 2), size, size, fc="none", ec="#b9b8b3",
lw=1.2, ls=(0, (2, 2))))
ax.text(-0.3, y, lab, ha="right", va="center", fontsize=10.5)
ax.text(8.2, y, f"{k}/{n} = {100 * k / n:.1f}%", ha="left", va="center", fontsize=10.5,
fontweight="bold", color=INK)
ax.set_xlim(-0.1, 8.0)
ax.set_ylim(-0.2, 4.6)
ax.axis("off")
fig.text(0.02, 0.94, "One square per calf: MAP-positive tissue at necropsy", fontsize=12,
fontweight="bold")
fig.text(0.02, 0.87, "Filled = positive, hollow = negative, dashed = calf enrolled (n = 8) "
"but absent from the abstract's count", fontsize=10, color=INK2)
fig.legend(handles=[Rectangle((0, 0), 1, 1, fc=ORANGE, ec=ORANGE),
Rectangle((0, 0), 1, 1, fc=BLUE, ec=BLUE)],
labels=["Reported in the abstract", "Paper's own Results counts, Methods group size"],
loc="lower center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.0))
save(fig, "fig2")
Abstract reports significant tDCS effects in three brain regions; the Results find significance in one
Proc Natl Acad Sci U S A (2025) · doi:10.1073/pnas.2416541122 · PMC12168024
Summary
The abstract states that significant neuromodulation of face-selective intracranial EEG activity was found in the anterior temporal lobe (ATL), the posterior temporal lobe (PTL) and the occipital lobe (OCC), with increases of "3% and 4%, 16% and 13%, and 36% and 33%". The paper's own Results, Fig. 4 legend and Discussion report significance in the PTL only. The ATL change (+3.8% during and +2.7% after stimulation, recomputed from the printed means) is described as "no significant amplitude change", and the OCC change as "a trend toward increase", which the Discussion attributes to too few contacts. The source data the paper deposited reproduce every printed mean and SEM, and with them no post hoc test reaches significance in the ATL (P ≥ 0.17). In the OCC, P lies between 0.022 and 0.23, depending on the test and on whether it is corrected for multiple comparisons. The pooled result, a significant increase across the right ventral occipito-temporal cortex (VOTC), is not affected. But the abstract tells readers that the effect reached all three regions, and the paper's own analysis shows it in one.
What the paper reports
Abstract:
Significant neuromodulations of face-selective iEEG activity in anterior & posterior temporal lobe and in the occipital lobe were found, with amplitude increases of 3% and 4%, 16% and 13%, and 36% and 33%, during and after tDCS, respectively.
Abstract, next sentence:
Interestingly, despite a unique tDCS session, the face-selective neuromodulation in the right visual occipito-temporal cortex remained significant (P = 0.015) after tDCS (P3 vs. P1).
Significance statement:
The neuromodulations were observed in the PTL.
Results, section "HD Electrodes Spatially Targeted Posterior Temporal Lobe (PTL).", paragraph on face-selective responses:
For face-selective responses, the baseline-corrected amplitudes for the iEEG contacts located in the ATL (n = 114) were 9.79 ± 1.20, 10.16 ± 1.23 (i.e., +4% with respect to P1) and 10.05 ± 1.22 µV (i.e., +3% with respect to P1), during the three phases P1, P2, and P3, respectively (no significant amplitude change). At the opposite, in the PTL (n = 51), the baseline corrected amplitudes were 21.43 ± 4.82, 24.83 ± 5.14 (i.e., +16% with respect to P1) and 24.15 ± 5.14 µV (i.e., +13% with respect to P1) during the three phases, respectively. The amplitude of the face-selective responses significantly varied from P1 to P2 i.e., during the tDCS (P = 0.003), as well as from P1 to P3 i.e., after the tDCS (P = 0.015) (Fig. 4B). In the OCC (n = 16), the baseline corrected amplitudes were 14.55 ± 3.56, 19.86 ± 5.41 (i.e., +36% with respect to P1), and 19.41 ± 5.24 µV (i.e., +33% with respect to P1) during the three phases, respectively. The variation of the amplitudes between P2 and P1 and between P3 and P1 showed a trend toward increase.
Results, section "TDCS Significantly Neuromodulates Face Selective Responses.", the pooled VOTC result:
For face-selective visual responses, the average baseline-corrected amplitudes recorded from 181 contacts during phases P1, P2, and P3 were 13.49 ± 1.62, 15.15 ± 1.77 (i.e., +12% with respect to P1), and 14.85 ± 1.76 µV (i.e., +10% with respect to P1), respectively. A significant baseline-corrected amplitude difference was found between P2 and P1 (P = 0.002) and between P3 and P1. (P = 0.003, Repeated measures ANOVA).
Fig. 4 legend:
A significant difference in the baseline-corrected amplitudes of the iEEG responses was observed in the PTL during the tDCS (P2) and maintained afterward (P3) in comparison to before tDCS (P1) (P = 0.003 & P = 0.015, respectively; repeated measure ANOVA).
Discussion, "Functional Targeting and Specific Neuromodulation.":
Brain structures in the PTL of the VOTC were significantly impacted by tDCS but the largest amplitude neuromodulation was observed in the OCC, where face-selective activity (“occipital face area”) is typically localized according to the literature (39–42) and where a high EF magnitude was observed. The lack of significance in the OCC during and after tDCS is likely due to insufficient statistical power, as there were fewer iEEG contacts in this region (clinical constraints) compared to the ATL and PTL.
Discussion, "Significant Neuromodulation Using Low-Intensity, -Duration, and -Resolution TDCS Configuration.":
The quantification of iEEG neuromodulation in terms of amplitude showed significant but moderate increases (from 10 to 36%) which could be related to these routine tDCS parameters set in the study.
Finally, the iEEG neuromodulation induced by tDCS in the posterior and OCC during our face-perception tasks could objectively explain the behavioral (27, 28) and scalp EEG (14, 15) changes and increases observed after tDCS in some studies.
Materials and Methods, the regional analysis:
Statistical comparisons were performed using repeated measure ANOVA tests with post hoc comparisons with the null hypothesis that intracranial neuromodulation will be the same in all ROI (Fig. 7).
What the paper's own numbers show
Method. I took the means printed in the Results and computed each percent change as 100 × (P2 or P3 mean / P1 mean − 1). P1 is before stimulation, P2 during and P3 after. The Data Availability statement cites deposited source data (ref. 55, Mendeley Data doi:10.17632/25z97kkndg.1). I downloaded two files from it and checked them against Mendeley's published SHA-256 hashes: per-contact baseline-corrected amplitudes for P1–P3 for all 302 face-selective contacts, and the region labels of the 181 VOTC contacts. One block of 8 contacts is labelled with one patient code in the localisation file and appears under the next patient code in the response file, with identical contact names. With that one match made, the counts are exactly ATL 114, PTL 51 and OCC 16, and every printed mean and SEM is reproduced. The paper does not say which post hoc test followed its repeated-measures ANOVA. So I computed four parametric versions, each for P2 vs P1 and P3 vs P1: paired t, paired t with Bonferroni correction for the three pairwise comparisons, a t test on the pooled ANOVA error (Fisher LSD), and Tukey HSD. A Wilcoxon signed-rank test is a non-parametric check. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Regions with significant face-selective change | Abstract | ATL, PTL and OCC | — |
| Regions with significant face-selective change | Results, Fig. 4 legend, Discussion | PTL only | PTL only (deposited data, all four parametric post hoc tests) |
| ATL change, during / after | Results | +4% / +3%, "no significant amplitude change" | +3.8% / +2.7% |
| ATL change, during / after | Abstract | "3% and 4%" | +3.8% / +2.7%: order reversed in the abstract |
| ATL post hoc P, during / after | Results | not given ("no significant") | uncorrected 0.167–0.325; corrected 0.350–0.889; Wilcoxon 0.375 / 0.664 |
| ATL omnibus repeated-measures ANOVA | — | — | F(2,226) = 1.02, P = 0.363 |
| PTL change, during / after | Results; Abstract | +16% / +13% | +15.9% / +12.7% |
| PTL post hoc P, during / after | Results; Fig. 4 legend | P = 0.003 / P = 0.015 | 0.0009–0.0177 / 0.0074–0.0331 (exact test unknown) |
| OCC change, during / after | Results; Abstract | +36% / +33%, "trend toward increase" | +36.5% / +33.4% |
| OCC post hoc P, during / after | Results; Discussion | not given ("lack of significance") | uncorrected 0.022–0.075; corrected 0.055–0.225; Wilcoxon 0.044 / 0.130 |
| OCC omnibus repeated-measures ANOVA | — | — | F(2,30) = 3.60, P = 0.040; Greenhouse–Geisser ε = 0.558, P = 0.072 |
| VOTC (all three ROIs) change, during / after | Results | +12% / +10%, P = 0.002 / P = 0.003 | +12.3% / +10.1%; post hoc P 0.00003–0.0015 / 0.0005–0.0031 |
| Right hemisphere, during / after | Results | +13% / +10%, P < 0.001 / P < 0.001 | +13.1% / +9.6% |
| ROI contacts | Results | ATL 114, PTL 51, OCC 16; VOTC 181 | 114 + 51 + 16 = 181 (the ROIs partition the VOTC) |
| P value "after" attached to the right VOTC | Abstract | P = 0.015 | Results give 0.003 for the VOTC; 0.015 is the PTL value |
| Range of "significant" increases | Discussion | "from 10 to 36%" | the largest significant regional increase is 15.9% (PTL); 36% is the non-significant OCC value |
All the printed percentages are correct rounding of the printed means. The problem is the significance attached to them in the abstract. Under every test I ran, the ATL is far from significance. The OCC is borderline: some uncorrected tests fall just below 0.05, but every multiplicity-corrected test, and the Greenhouse–Geisser-corrected omnibus test, does not. The paper itself classifies the OCC as not significant in the Results, the Fig. 4 legend and the Discussion. The PTL is significant under all four parametric post hoc tests.
Figure
Figure 1. Left: the abstract sentence (orange outline), split by region. Right: what the Results, Fig. 4 legend and Discussion say about the same three regions. Text in orange is printed in the paper. Text in blue is the percent change recomputed from the printed means by recompute.py.txt. Solid connectors mark the two regions where the abstract's claim of significance conflicts with the body of the paper (for the ATL, the abstract also reverses the order of the during and after values). The dashed connector marks the one region where abstract and body agree.
Figure 2. Post hoc P values against before stimulation (P1), on a log scale, for the during (P2) and after (P3) phases. Orange diamonds: P values printed in the paper. The right-hemisphere values are printed only as "P < 0.001" and are drawn at 0.001 with a leftward arrow. The paper prints no P value for the ATL or OCC. Blue: P values recomputed from the paper's deposited per-contact data by recompute.py.txt. The filled circle is a paired t test with Bonferroni correction for three comparisons, and the line with open circles spans four parametric post hoc tests (paired t, paired t with Bonferroni correction, pooled-error t, Tukey HSD). The vertical line marks P = 0.05. The right-hand column gives the paper's own classification in the Results. It agrees with the recomputed values and contradicts the abstract for the ATL and OCC.
Why it matters
- The abstract misstates where the effect was found. Where in the brain tDCS acts is the question of the regional analysis, and the paper's title, Significance statement and Results all present spatial targeting as a key result. The Results section heading is "HD Electrodes Spatially Targeted Posterior Temporal Lobe (PTL)". A reader of the abstract alone will conclude that significant effects were shown in all three ventral occipito-temporal regions, when the paper shows one: the PTL, the region under the anode.
- The effect sizes quoted in the abstract and Discussion are mostly not significant. The abstract's largest figures, 36% and 33% for the OCC, come from 16 contacts and are not significant by the paper's own analysis. The Discussion's "significant but moderate increases (from 10 to 36%)" also carries this 36% as its upper bound, when the largest significant regional increase is about 16% (PTL). Readers comparing tDCS effect sizes across studies could take the larger figure.
- Downstream interpretation leans on the occipital effect. The Discussion suggests that "the iEEG neuromodulation induced by tDCS in the posterior and OCC" could explain earlier behavioural and scalp EEG findings. On the paper's own statistics, that interpretation rests on an occipital effect that did not reach significance.
- The central finding holds. Pooled over the VOTC (181 contacts) and over the right hemisphere (302 contacts), the face-selective increase is significant in the paper and in the deposited data. The error lies in how the abstract reports the regional breakdown, not in the main result.
Explanations considered
- A different level of analysis. Not supported. The three regions together are exactly the VOTC set (114 + 51 + 16 = 181 contacts), and the VOTC as a whole was significant. So the abstract sentence might be a loose way of saying "significant in the VOTC, which spans ATL, PTL and OCC". But the sentence names each region and gives each one its own percentages. The next sentence gives the VOTC its own P value. The Significance statement says "The neuromodulations were observed in the PTL", and the Fig. 4 legend reports significance for the PTL only.
- A different test or correction. Not able to explain the ATL. The Methods describe repeated-measures ANOVA with post hoc comparisons and no other test. On the deposited data, the ATL omnibus ANOVA gives P = 0.36 and no post hoc variant gives P below 0.16. For the OCC, an uncorrected pooled-error t test (P = 0.022 / 0.034), an uncorrected Wilcoxon test during stimulation (P = 0.044) and the uncorrected omnibus ANOVA (P = 0.040) fall below 0.05. But the paper reports none of these, and it states "lack of significance in the OCC". The abstract therefore conflicts with the paper's own reported analysis, whatever might be argued for another one.
- A typesetting or conversion artifact. Not possible. The conflict is between words ("Significant" in the abstract, "no significant amplitude change" and "lack of significance" in the body), not between digits or signs. The publisher-deposited JATS full text on Europe PMC carries the same wording.
- A reversed during/after convention. This covers only the ATL order swap. The abstract gives the PTL and OCC values in the order during, after, which matches the Results. For the ATL it gives "3% and 4%", the reverse of the Results' +4% during and +3% after. It reads as an ordering slip, and the change is small either way.
- A correction already issued. None found (see Status).
Status
- Version of record. The publisher's site (pnas.org, reached through the DOI) returned HTTP 403 to automated requests for both the HTML and the PDF. I checked the publisher-deposited JATS full text on Europe PMC (PMC12168024) and the PMC article. The abstract, Results, Fig. 4 legend and Discussion wording quoted above is the same in both. In the Fig. 4B image supplied with the article, significance marks appear only on the PTL series. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. SI Appendix (
pnas.2416541122.sapp.pdf, 16 pages; Figs. S1–S4, Table S5): Fig. S3 shows regional SNR only, with the caption "No significant difference was found in any of the 3 regions in the VOTC for both types of responses", and gives no regional amplitude statistics. Deposited source data, Mendeley Data doi:10.17632/25z97kkndg.1 (4 .xlsx files, deposited 19 May 2025): the two face-selective files were downloaded, checked against Mendeley's SHA-256 hashes, and reanalysed as described above. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Europe PMC record (PMID 40460123) lists no comments or corrections. PubPeer returned HTTP 403 to automated requests. A web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC12168024; Europe PMC full-text XML and supplementary files for PMC12168024; SI Appendix
pnas.2416541122.sapp.pdf; Mendeley Data doi:10.17632/25z97kkndg.1; Crossref; Europe PMC; PubPeer; https://doi.org/10.1073/pnas.2416541122.
Suggested correction
In the abstract, replace
Significant neuromodulations of face-selective iEEG activity in anterior & posterior temporal lobe and in the occipital lobe were found, with amplitude increases of 3% and 4%, 16% and 13%, and 36% and 33%, during and after tDCS, respectively.
with wording that keeps the significance claim to the PTL, for example
Significant neuromodulation of face-selective iEEG activity was found in the posterior temporal lobe, with amplitude increases of 16% and 13% during and after tDCS, respectively; nonsignificant changes of 4% and 3% were observed in the anterior temporal lobe and of 36% and 33% in the occipital lobe.
The authors may also wish to check the P value in the next abstract sentence: P = 0.015 is the PTL value, while the Results give P = 0.003 for the right VOTC after tDCS. In the Discussion, the range "from 10 to 36%" could be limited to significant effects (about 10 to 16%), and the sentence on "the posterior and OCC" could note that the occipital change was not significant. Naming the post hoc test, and any correction for multiple comparisons, would let readers reproduce the reported P values.
Recomputation script (python recompute.py.txt)
"""Recompute the regional tDCS results of PMC12168024
(Proc Natl Acad Sci U S A 2025, doi:10.1073/pnas.2416541122).
Run: python recompute.py.txt (downloads the deposited data from Mendeley Data)
python recompute.py.txt <folder> (uses the two .xlsx files already saved in <folder>)
Needs numpy, scipy, matplotlib and openpyxl. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Part 1 uses only numbers printed in the paper's Results text (means +/- SEM in uV of the
baseline-corrected face-selective amplitudes, by phase P1 before, P2 during, P3 after tDCS).
Part 2 uses the source data the paper cites in its Data Availability statement (ref. 55,
Mendeley Data doi:10.17632/25z97kkndg.1, deposited 19 May 2025):
"Face-selective iEEG responses.xlsx" 302 contacts, SNR and baseline-corrected amplitude, P1-P3
"Anatomical localisation of face-selective response contacts.xlsx" 181 VOTC contacts with ROI
The files are checked against the SHA-256 hashes Mendeley publishes for them.
In the localisation file, the 8 contacts of the block labelled PAT_8 that are absent from
PAT_8 in the response file all appear, with the same contact names, under PAT_9 (which has
no block of its own in the localisation file); they are matched there.
"""
import hashlib
import io
import os
import sys
import textwrap
import urllib.request
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ORANGE, BLUE, INK, INK2, BG, GRID = "#eb6834", "#2a78d6", "#0b0b0b", "#52514e", "#fcfcfb", "#e4e3df"
# ------------------------------------------------------------------ Part 1: the paper's text
# region: (n, means P1-P3, SEMs P1-P3, % printed in Results for P2, P3, significance in Results)
TEXT = {
"RH": (302, (9.79, 11.07, 10.73), (1.07, 1.19, 1.17), (13, 10), "P < 0.001 ; P < 0.001"),
"VOTC": (181, (13.49, 15.15, 14.85), (1.62, 1.77, 1.76), (12, 10), "P = 0.002 ; P = 0.003"),
"ATL": (114, (9.79, 10.16, 10.05), (1.20, 1.23, 1.22), (4, 3), "'no significant amplitude change'"),
"PTL": (51, (21.43, 24.83, 24.15), (4.82, 5.14, 5.14), (16, 13), "P = 0.003 ; P = 0.015"),
"OCC": (16, (14.55, 19.86, 19.41), (3.56, 5.41, 5.24), (36, 33), "'trend toward increase'"),
}
ABSTRACT_PCT = {"ATL": (3, 4), "PTL": (16, 13), "OCC": (36, 33)} # "during and after tDCS, respectively"
REPORTED_P = {("RH", 2): "<0.001", ("RH", 3): "<0.001", ("VOTC", 2): 0.002, ("VOTC", 3): 0.003,
("PTL", 2): 0.003, ("PTL", 3): 0.015}
GENERAL_VISUAL_N = {"ATL": 91, "PTL": 40, "OCC": 16}
print("=" * 78)
print("PART 1. Percent changes from the means printed in the Results")
print("=" * 78)
pct = {}
print(f"{'region':6s} {'n':>4s} {'P2 vs P1':>9s} {'P3 vs P1':>9s} Results prints abstract prints Results significance")
for r, (n, m, s, rep, sig) in TEXT.items():
p2, p3 = 100 * (m[1] / m[0] - 1), 100 * (m[2] / m[0] - 1)
pct[r] = (p2, p3)
ab = ABSTRACT_PCT.get(r)
abs_txt = f"{ab[0]}% / {ab[1]}%" if ab else "-"
print(f"{r:6s} {n:4d} {p2:+8.1f}% {p3:+8.1f}% +{rep[0]}% / +{rep[1]}% {abs_txt:15s} {sig}")
print()
for r, (a2, a3) in ABSTRACT_PCT.items():
q2, q3 = (round(x) for x in pct[r])
status = "same order" if (a2, a3) == (q2, q3) else "ORDER REVERSED (abstract during/after = Results after/during)"
print(f"abstract {r}: during {a2}%, after {a3}% | means give {q2}% / {q3}% -> {status}")
print()
print("ROI contact counts (face-selective): 114 + 51 + 16 =", 114 + 51 + 16, "(VOTC n = 181)")
print("ROI contact counts (general visual):", " + ".join(str(v) for v in GENERAL_VISUAL_N.values()),
"=", sum(GENERAL_VISUAL_N.values()), "; with face-selective 181 ->", 181 + sum(GENERAL_VISUAL_N.values()),
"(text: '328 iEEG contacts in the VOTC for both iEEG responses')")
print("Largest regional increase reported as significant (PTL, P2):", f"{pct['PTL'][0]:.1f}%")
print("Discussion range 'from 10 to 36%': 36% is the OCC P2 value, which Results/Discussion call not significant")
print("Abstract 'remained significant (P = 0.015) after tDCS' for the right VOTC; Results give VOTC P3 vs P1",
"P = 0.003 and PTL P3 vs P1 P = 0.015")
# ------------------------------------------------------------------ Part 2: deposited data
MENDELEY = "https://data.mendeley.com/public-files/datasets/25z97kkndg/files/{}/file_downloaded"
FILES = {
"Face-selective iEEG responses.xlsx":
("6669e878-1bab-421d-998e-8ce8a2878bcd", "77177beeed122f93b02e48efdeae4b775d0e2636b25cb3e180609b007352a279"),
"Anatomical localisation of face-selective response contacts.xlsx":
("c823f672-0432-44c6-befa-d3eacc5a84ab", "15229a5fcc65c37698e56fb10a34d0c3ba302718c9fa69308d28921735429341"),
}
def load(name, folder):
fid, sha = FILES[name]
if folder:
with open(os.path.join(folder, name), "rb") as fh:
raw = fh.read()
else:
req = urllib.request.Request(MENDELEY.format(fid), headers={"User-Agent": "Mozilla/5.0"})
raw = urllib.request.urlopen(req, timeout=60).read()
got = hashlib.sha256(raw).hexdigest()
if got != sha:
sys.exit(f"{name}: SHA-256 {got} does not match the published {sha}")
import openpyxl
wb = openpyxl.load_workbook(io.BytesIO(raw), read_only=True, data_only=True)
return list(next(iter(wb)).iter_rows(values_only=True))
folder = sys.argv[1] if len(sys.argv) > 1 else None
resp, pat = {}, None
for row in load("Face-selective iEEG responses.xlsx", folder)[2:]:
if row[0]:
pat = row[0].strip().upper()
if row[1] is None:
continue
key = (pat, row[1].strip())
assert key not in resp
resp[key] = np.array(row[5:8], float) # baseline-corrected amplitude P1, P2, P3
loc, pat = [], None
for row in load("Anatomical localisation of face-selective response contacts.xlsx", folder)[1:]:
if row[0]:
pat = row[0].strip().upper()
if row[1] is None:
continue
loc.append((pat, str(row[1]).strip(), row[2].strip()))
remapped = 0
groups = {"RH": np.array(list(resp.values()))}
votc = []
for p, c, roi in loc:
if (p, c) in resp:
votc.append((roi, resp[(p, c)]))
else:
assert p == "PAT_8" and ("PAT_9", c) in resp
remapped += 1
votc.append((roi, resp[("PAT_9", c)]))
groups["VOTC"] = np.array([v for _, v in votc])
for r in ("ATL", "PTL", "OCC"):
groups[r] = np.array([v for roi, v in votc if roi == r])
print()
print("=" * 78)
print("PART 2. Deposited data (Mendeley Data doi:10.17632/25z97kkndg.1)")
print("=" * 78)
print(f"contacts in response file: {len(resp)}; in localisation file: {len(loc)}; "
f"matched under PAT_9: {remapped}")
print(f"{'region':6s} {'n':>4s} means P1/P2/P3 (uV) SEMs P1/P2/P3 printed in paper")
for r, A in groups.items():
m = A.mean(0)
se = A.std(0, ddof=1) / np.sqrt(len(A))
tm, ts = TEXT[r][1], TEXT[r][2]
print(f"{r:6s} {len(A):4d} {m[0]:6.3f} {m[1]:6.3f} {m[2]:6.3f} {se[0]:5.2f} {se[1]:5.2f} {se[2]:5.2f} "
f"{tm} +/- {ts}")
def rm_anova(A):
n, k = A.shape
gm = A.mean()
ss_c = n * ((A.mean(0) - gm) ** 2).sum()
ss_s = k * ((A.mean(1) - gm) ** 2).sum()
ss_e = ((A - gm) ** 2).sum() - ss_c - ss_s
df1, df2 = k - 1, (k - 1) * (n - 1)
mse = ss_e / df2
F = (ss_c / df1) / mse
# Greenhouse-Geisser epsilon from orthonormal contrasts
C = np.linalg.qr(np.array([[1, -1, 0], [0, 1, -1]], float).T)[0]
Sc = C.T @ np.cov(A.T) @ C
eps = np.trace(Sc) ** 2 / (df1 * np.trace(Sc @ Sc))
return F, df1, df2, stats.f.sf(F, df1, df2), eps, stats.f.sf(F, eps * df1, eps * df2), mse
# The paper's post hoc tests follow a (parametric) repeated-measures ANOVA, so the figure's range
# spans four parametric post hoc variants; the Wilcoxon signed-rank test is printed as a check only.
VARIANTS = ["paired t", "paired t, Bonf. x3", "pooled-error t (LSD)", "Tukey HSD"]
pvals, wil = {}, {}
print()
print("Omnibus one-way repeated-measures ANOVA over P1, P2, P3:")
for r, A in groups.items():
F, d1, d2, p, eps, pgg, mse = rm_anova(A)
print(f" {r:5s} F({d1},{d2}) = {F:5.2f}, P = {p:.4f}; Greenhouse-Geisser eps = {eps:.3f}, P = {pgg:.4f}")
n = len(A)
m = A.mean(0)
for j in (1, 2):
pt = stats.ttest_rel(A[:, j], A[:, 0]).pvalue
pw = stats.wilcoxon(A[:, j], A[:, 0]).pvalue
t_l = (m[j] - m[0]) / np.sqrt(2 * mse / n)
pl = 2 * stats.t.sf(abs(t_l), d2)
ph = stats.studentized_range.sf(abs(m[j] - m[0]) / np.sqrt(mse / n), 3, d2)
pvals[(r, j + 1)] = [pt, min(1, 3 * pt), pl, ph]
wil[(r, j + 1)] = pw
print()
print("Post hoc P values, P2 vs P1 and P3 vs P1 (the paper does not say which post hoc test it used):")
print(f" {'':9s}" + "".join(f"{v:>21s}" for v in VARIANTS) + f"{'range':>16s}{'Wilcoxon':>10s} reported")
for (r, ph), ps in pvals.items():
rep = REPORTED_P.get((r, ph), "n.s." if r in ("ATL", "OCC") else "-")
print(f" {r:5s} P{ph} " + "".join(f"{p:21.4f}" for p in ps) + f" {min(ps):.2g}-{max(ps):.2g}"
f"{wil[(r, ph)]:10.4f} {rep}")
print()
for r in ("ATL", "OCC"):
unc = [pvals[(r, k)][i] for k in (2, 3) for i in (0, 2)]
cor = [pvals[(r, k)][i] for k in (2, 3) for i in (1, 3)]
print(f"{r}: post hoc P, uncorrected (paired t, LSD) {min(unc):.4f}-{max(unc):.4f}; "
f"multiplicity-corrected (Bonferroni, Tukey) {min(cor):.4f}-{max(cor):.4f}; "
f"Wilcoxon {wil[(r, 2)]:.4f} / {wil[(r, 3)]:.4f}")
print("PTL: all four parametric variants below 0.05:", all(max(pvals[("PTL", k)]) < 0.05 for k in (2, 3)),
f"(largest {max(pvals[('PTL', 2)] + pvals[('PTL', 3)]):.4f})")
# ------------------------------------------------------------------ figures
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "svg.fonttype": "none",
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2})
def save(fig, stem):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{stem}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: abstract vs body
W, H = 7.2, 5.6
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.axis("off")
def box(x0, y0, x1, y1, edge):
ax.add_patch(FancyBboxPatch((x0, y0), x1 - x0, y1 - y0, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
LX0, LX1, RX0, RX1, TOP, BOT = 0.10, 2.70, 4.06, 7.10, 5.48, 0.75
PAD = 0.13
# left panel: abstract
box(LX0, BOT, LX1, TOP, ORANGE)
ax.text(LX0 + PAD, TOP - 0.12, "Abstract", fontsize=11, fontweight="bold", va="top")
quote = ("“Significant neuromodulations of face-selective iEEG activity in anterior & posterior "
"temporal lobe and in the occipital lobe were found, with amplitude increases of 3% and 4%, "
"16% and 13%, and 36% and 33%, during and after tDCS, respectively.”")
ax.text(LX0 + PAD, TOP - 0.45, textwrap.fill(quote, 31), fontsize=10, va="top", color=ORANGE, linespacing=1.3)
ax.plot([LX0 + PAD, LX1 - PAD], [3.02, 3.02], color=GRID, lw=1)
ax.text(LX0 + PAD, 2.90, "region by region:", fontsize=10, color=INK2, va="top")
# right panel: Results / Fig. 4 legend / Discussion
box(RX0, BOT, RX1, TOP, INK2)
ax.text(RX0 + PAD, TOP - 0.12, "Results, Fig. 4 legend, Discussion", fontsize=10.5, fontweight="bold", va="top")
LS = 0.22 # line step
rows = [
("ATL", "anterior temporal lobe", 2.40, 4.95,
["+4% during, +3% after;", "“(no significant amplitude change)”"],
f"recomputed: {pct['ATL'][0]:+.1f}% / {pct['ATL'][1]:+.1f}%",
"conflict:\nsignificance;\nduring/after\nswapped"),
("PTL", "posterior temporal lobe", 1.75, 3.75,
["+16% / +13%; “significantly varied”", "P = 0.003 / P = 0.015"],
f"recomputed: {pct['PTL'][0]:+.1f}% / {pct['PTL'][1]:+.1f}%",
"agrees"),
("OCC", "occipital lobe", 1.10, 2.65,
["+36% / +33%; “showed a trend toward", "increase” (Results); “The lack of", "significance in the OCC” (Discussion)"],
f"recomputed: {pct['OCC'][0]:+.1f}% / {pct['OCC'][1]:+.1f}%",
"conflict:\nsignificance"),
]
ab_txt = {"ATL": "significant; 3% during, 4% after", "PTL": "significant; 16% during, 13% after",
"OCC": "significant; 36% during, 33% after"}
for code, name, ly, ry, rtxt, rec, verdict in rows:
ax.text(LX0 + PAD, ly + 0.22, f"{code} ({name})", fontsize=10, fontweight="bold", va="center")
ax.text(LX0 + PAD, ly, ab_txt[code], fontsize=10, color=ORANGE, va="center")
ax.text(RX0 + PAD, ry, f"{code} (n = {TEXT[code][0]} contacts)", fontsize=10, fontweight="bold", va="center")
for i, line in enumerate(rtxt):
ax.text(RX0 + PAD, ry - LS * (i + 1), line, fontsize=10, color=ORANGE, va="center")
ax.text(RX0 + PAD, ry - LS * (len(rtxt) + 1), rec, fontsize=10, color=BLUE, va="center")
conflict = verdict != "agrees"
y0, y1 = ly + 0.11, ry - LS * (len(rtxt) + 1) / 2 # aim at the middle of the right-hand block
ax.add_patch(FancyArrowPatch((LX1 + 0.03, y0), (RX0 - 0.03, y1), arrowstyle="-|>", mutation_scale=14,
lw=2 if conflict else 1.5, color=INK if conflict else INK2,
linestyle="-" if conflict else (0, (3, 2))))
t = 0.6 if code == "ATL" else 0.5 # keeps the ATL label clear of the PTL connector
ax.text(LX1 + 0.03 + t * (RX0 - LX1 - 0.06), y0 + t * (y1 - y0), verdict, fontsize=10, ha="center",
va="center", color=INK if conflict else INK2, fontweight="bold" if conflict else "normal",
linespacing=1.15, bbox=dict(boxstyle="round,pad=0.2", fc=BG, ec="none"))
handles = [Line2D([], [], color=ORANGE, lw=0, marker="s", ms=8, label="as printed in the paper"),
Line2D([], [], color=BLUE, lw=0, marker="s", ms=8,
label="recomputed from the printed means")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(0.06 / W, 0.0), frameon=False, ncol=2,
fontsize=10, handletextpad=0.3, columnspacing=1.0)
save(fig, "fig1")
# Figure 2: post hoc P values, reported vs recomputed from deposited data
order = [("RH", 2), ("RH", 3), ("VOTC", 2), ("VOTC", 3), ("ATL", 2), ("ATL", 3),
("PTL", 2), ("PTL", 3), ("OCC", 2), ("OCC", 3)]
label_r = {"RH": "Right hemisphere", "VOTC": "VOTC (all ROIs)", "ATL": "ATL", "PTL": "PTL", "OCC": "OCC"}
paper_says = {"RH": "significant", "VOTC": "significant", "ATL": "not significant", "PTL": "significant",
"OCC": "not significant (“trend”)"}
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
fig.subplots_adjust(left=0.30, right=0.76, top=0.86, bottom=0.14)
ax.set_facecolor(BG)
ys = []
y = 0
for i, (r, ph) in enumerate(order):
if i and order[i - 1][0] != r:
y += 0.6
ys.append(y)
y += 1
ys = -np.array(ys)
for (r, ph), yy in zip(order, ys):
ps = pvals[(r, ph)]
ax.plot([min(ps), max(ps)], [yy, yy], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.scatter(ps, [yy] * len(ps), s=28, facecolor="white", edgecolor=BLUE, lw=1.2, zorder=3)
ax.scatter([ps[1]], [yy], s=64, color=BLUE, zorder=4)
rep = REPORTED_P.get((r, ph))
if rep is not None:
if rep == "<0.001":
ax.annotate("", xy=(4e-4, yy + 0.28), xytext=(1e-3, yy + 0.28),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=10))
ax.scatter([1e-3], [yy + 0.28], marker="D", s=56, color=ORANGE, zorder=5)
else:
ax.scatter([rep], [yy + 0.28], marker="D", s=56, color=ORANGE, zorder=5)
ax.axvline(0.05, color=INK2, lw=1, zorder=1)
ax.text(0.042, ys[0] + 0.95, "P = 0.05 ", ha="right", va="bottom", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(1e-7, 1.5)
ax.set_xticks([1e-6, 1e-4, 1e-2, 1])
ax.set_xticklabels(["10⁻⁶", "10⁻⁴", "0.01", "1"])
ax.set_xlabel("Post hoc P value vs before tDCS (P1), log scale")
ax.set_yticks(ys)
ax.set_yticklabels([f"{label_r[r]} {'during' if ph == 2 else 'after'}" for r, ph in order], fontsize=10)
ax.set_ylim(ys[-1] - 0.7, ys[0] + 1.3)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
seen = set()
for (r, ph), yy in zip(order, ys):
if r in seen:
continue
seen.add(r)
ax.text(1.03, yy - 0.5, textwrap.fill(paper_says[r], 18), transform=ax.get_yaxis_transform(), fontsize=10,
va="center", color=INK2)
ax.text(1.03, ys[0] + 0.95, "Paper's Results", transform=ax.get_yaxis_transform(), fontsize=10,
va="bottom", fontweight="bold")
handles = [Line2D([], [], color=ORANGE, marker="D", lw=0, ms=8, label="reported in the paper"),
Line2D([], [], color=BLUE, marker="o", lw=0, ms=8, label="paired t, Bonferroni ×3 (deposited data)"),
Line2D([], [], color=BLUE, marker="o", mfc="white", lw=2, ms=6,
label="range over four post hoc tests (deposited data)")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=2, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.0)
save(fig, "fig2")
print()
print("wrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Title says the 2025–2030 guidelines raise water use; the paper's own results show lower water use
Proc Natl Acad Sci U S A (2026) · doi:10.1073/pnas.2604814123 · PMC13273247
Summary
This PNAS brief report compares diets that follow the 2025–2030 Dietary Guidelines for Americans (no ultraprocessed foods, protein raised to 1.2–1.6 g/kg) with the current Mean American Diet (MAD) on greenhouse gas (GHG) emissions, freshwater use, land use and nitrogen fertilizer use. Its title says the guidelines are "associated with higher land, water and nitrogen use, and greenhouse gas emissions", but its abstract, Results and Discussion all report that water use is "lower by 7 to 19% than the MAD". The paper's own Dataset S1 gives −6.6% to −19.5% for those scenarios (the text's range, to the nearest percent) and lower water use than the MAD in every scenario modelled, while the other three outcomes rise by up to 31.8%. The title thus states the wrong direction for one of the four outcomes it names, and a reader who sees only the title, as indexes, citations and news coverage present it, gets the water result backwards.
What the paper reports
Title:
The 2025–2030 Dietary Guidelines for Americans are associated with higher land, water and nitrogen use, and greenhouse gas emissions
Abstract, which states the comparator and the water result:
Here, we quantified the greenhouse gas (GHG) emissions, freshwater use, land use, and nitrogen fertilizer use of adherence to these new guidelines in comparison to the current Mean American Diet (MAD).
We find that although eliminating ultraprocessed foods from the MAD could yield substantial environmental benefits—given their large share of total intake and animal-based makeup—the recommended increase in animal-based proteins, will more than offset this environmental dividend in GHG, land use, and nitrogen fertilizer use by as much as 32%, but not in water use (i.e., water use is lower by 7 to 19% than the MAD).
Overall, adherence to the 2025–2030 Dietary Guidelines for Americans would confer net environmental harm across most key metrics relative to the prevailing U.S. diet.
Introduction, last paragraph:
We then quantified greenhouse gas (GHG) emissions, freshwater use, land use, and nitrogen fertilizer use for each diet and compared them against the MAD.
Results, second paragraph:
Our results indicate that GHG, land use, and nitrogen fertilizer dividends of eliminating UPFs from the MAD are outweighed by the impacts associated with increased protein intake (1.2 to 1.6 g/kg) by as much as 32%. In contrast, eliminating UPF and increasing protein levels to 2025–2030 DGAs reduced the water footprint compared to MAD by 7 to 19%.
Discussion, first paragraph:
For land use, GHG emissions, and nitrogen fertilizer use the net effects are adverse. For water use, the net effect is beneficial, but would be even more so were the avoidance of UPFs combined with a lower protein threshold, and the sourcing of protein preferentially from plants.
The paper has no tables. Fig. 1 panel B ("freshwater use (panel B)") plots the water results. All quotes above were checked verbatim against the PMC-archived article text.
What the paper's own numbers show
Method. Dataset S1 (pnas.2604814123.sd01.xlsx) has a "summary" sheet. Its rows 5–11 give annual per-capita GHG (kg CO2e), water (litres), land (m²·yr) and nitrogen fertilizer (g Nr), plus daily kcal and protein, for the MAD, the ultraprocessed-food (UPF) component of the MAD, and five UPF-free scenarios: 0.8, 1.2 and 1.6 g/kg, and 1.6 g/kg weighted toward plant or animal sources. I took the cached cell values and computed each scenario's percentage change against the MAD. I then compared the results with the ranges stated in the text. The sheet is identical to the one in the data file that the paper links under Data Availability. The sheet's own "Range" cells (K15:L15) pool GHG, land and nitrogen for the 1.2, 1.6 and 1.6 g/kg animal scenarios, and leave water out. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Direction of water use, guideline diets vs MAD | Title | "higher … water … use" | Lower in all 5 scenarios (−6.6% to −27.7%) |
| Water, 1.2 g/kg vs MAD | Abstract; Results ¶2; Fig. 1B | "lower by 7 to 19%" (upper end) | −19.45% |
| Water, 1.6 g/kg vs MAD | Abstract; Results ¶2; Fig. 1B | "lower by 7 to 19%" (lower end) | −6.58% |
| Water, 1.6 g/kg animal vs MAD | Fig. 1B | within 7–19% lower | −13.14% |
| Water, 1.6 g/kg plant vs MAD | Fig. 1B | lower (not quantified) | −24.92% |
| Water, 0.8 g/kg vs MAD | Fig. 1B | lower (not quantified) | −27.73% |
| Net water effect | Discussion ¶1 | "beneficial" | Consistent: every scenario is below the MAD |
| GHG, land, N fertilizer vs MAD, 1.2–1.6 g/kg scenarios | Abstract; Results ¶2 | "by as much as 32%" | −2.5% (GHG, 1.2 g/kg) to +31.8% (land, 1.6 g/kg) |
| 1.6 g/kg vs MAD: GHG / land / N | Fig. 1A, C, D | higher | +21.5% / +31.8% / +7.7% |
| UPF share of MAD impacts (GHG, water, land, N) | Results ¶2 | "40 to 58%" | 46.1%, 52.4%, 40.3%, 57.6% |
| Protein, 0.8 / 1.2 / 1.6 g/kg × 84 kg | Results ¶1; Methods | 67 / – / 134 g per day | 67.2 / 100.8 / 134.4 g (modelled 67.3 / 100.8 / 134.3) |
Every number in the abstract, Results and Discussion reproduces from Dataset S1. The title is the only statement that disagrees with the data, and it disagrees only on the direction of water use.
Figure
Figure 1. The title clause (left, orange) lists water among the outcomes that are "higher" under the 2025–2030 guidelines. On the right are the three places where the text reports the water result, each against the Mean American Diet (MAD) (gray): the abstract ("lower by 7 to 19% than the MAD"), the second paragraph of Results ("reduced the water footprint compared to MAD by 7 to 19%") and the first paragraph of Discussion ("the net effect is beneficial"). Below them (blue) are the water changes recomputed from the summary sheet of Dataset S1 for the three scenarios in the guidelines' protein range that the text's 7–19% covers. The 1.6 g/kg plant scenario (−24.9%) and the 0.8 g/kg scenario (−27.7%) are also below the MAD and are not shown in this panel.
Figure 2. Change against the MAD for each of the four outcomes named in the title, in the four UPF-free scenarios at 1.2–1.6 g/kg protein (blue markers, recomputed from Dataset S1; the hollow marker is the plant-weighted 1.6 g/kg diet). The thin vertical line marks no change. Orange marks show what the paper states: the "lower by 7 to 19%" interval for water, the "as much as 32%" maximum for GHG, land and nitrogen (reached for land), and the direction that the title gives for water (arrow pointing to higher values). Land, GHG and nitrogen fertilizer rise in the 1.6 g/kg scenarios. Every water value is below zero, so none matches the direction the title gives.
Why it matters
The title is the paper's headline claim. It names four outcomes and says all four are higher under the guidelines. Against the comparator the paper states throughout (the Mean American Diet), one of those four, freshwater use, goes the other way in every scenario the paper models: it falls by 6.6–19.5% at the guidelines' protein levels and by up to 27.7% overall. The body of the paper reports this correctly and calls the net water effect "beneficial". The abstract's closing sentence is qualified accordingly ("net environmental harm across most key metrics").
The title is what bibliographic databases index, what citing papers and reference lists reproduce, and what headlines and social media posts pick up. A reader who relies on the title, which is the usual case for a widely covered policy paper, comes away with the water result reversed. One news item that links to this PMC record says the guidelines are linked to "16 percent more water consumption", alongside 33% higher emissions and 28% greater land use, and gives no comparator (Species Unite). In Dataset S1 those three figures coincide with the 1.6 g/kg animal versus 1.6 g/kg plant contrast (+32.6% GHG, +27.7% land, +15.7% water). That is a comparison between two guideline-range diets, not between the guidelines and the current diet.
The following results are not affected: the GHG, land-use and nitrogen-fertilizer increases (up to +31.8% against the MAD), the finding that plant-sourced protein does better than animal-sourced protein on every metric, the UPF shares, and the paper's main conclusion of net environmental harm on most metrics. The paper reports scenario point estimates without inferential statistics, so no test or interval changes.
Explanations considered
- A different comparator. The title names no comparator. Within the UPF-free scenarios, water use does rise with protein: the 1.2 g/kg and 1.6 g/kg diets use 11.4% and 29.3% more water than the 0.8 g/kg diet. Read against a UPF-free 0.8 g/kg diet, "higher water" would be true. The paper, however, states its comparator explicitly in the abstract ("in comparison to the current Mean American Diet (MAD)"), the Introduction ("compared them against the MAD") and the abstract's conclusion ("relative to the prevailing U.S. diet"). In the title, "higher" applies to land, nitrogen and GHG on the MAD comparison. The abstract singles water out as the exception to that comparison ("but not in water use"). This does not account for the title.
- The marginal effect of the added protein. The Discussion notes that water use would be lower still "were the avoidance of UPFs combined with a lower protein threshold". This describes the protein increase as eating into a water saving, not as producing higher water use than the current diet. The same paragraph calls the net effect "beneficial". This does not account for the title.
- Animal-weighted versus plant-weighted sourcing. The 1.6 g/kg animal diet uses 15.7% more water than the 1.6 g/kg plant diet. That is a comparison between two guideline-range diets. The title describes the guidelines as a whole, and even the animal-weighted diet uses 13.1% less water than the MAD. This does not account for the title.
- A scenario with higher water use. None exists. All five modelled scenarios are below the MAD on water (−6.6% to −27.7%). This does not account for the title.
- Normalisation. All diets are isocaloric (about 2,500 kcal/day), so water per 1,000 kcal has the same sign as the totals. Per gram of protein, every guideline-range diet is also lower than the MAD (−15.4% to −40.8%). This does not account for the title.
- Uncertainty. The paper gives no intervals, so it cannot be judged whether the smallest reduction (−6.6%) is distinguishable from zero. The paper itself, though, states the direction as lower in three places and never as higher or as uncertain. This does not account for the title.
- Text-conversion artifact. The conflict is between words ("higher" against "lower", "reduced", "beneficial"). It does not depend on a sign, symbol or table cell. The same title is in the Crossref metadata deposited by the publisher, the Europe PMC record and the live PMC page. An archiving artifact is ruled out.
Status
As of 2026-10-23:
- Version of record. The publisher's site (pnas.org, the DOI target) returned HTTP 403 from this machine, so its HTML and PDF could not be read. The copies checked were the PMC-archived text (via
src/show.py), the live PMC page, the Europe PMC full-text XML and record (PMID 42224503), and the Crossref metadata title. All carry the same title and the same abstract, Results and Discussion sentences. - Supplements. Dataset S1 (pnas.2604814123.sd01.xlsx, downloaded via Europe PMC) was read. Its "summary" sheet gives the scenario totals used above, and the same sheet appears unchanged in the authors' linked data file. It confirms the text's 7–19% water reduction and shows no scenario with higher water use. The supplement supports the body text and does not resolve the conflict with the title.
- Corrections. No correction, erratum, expression of concern or retraction was found.
src/known.py: no Crossref updates, no PMC notices, no Europe PMC notices. In the Crossref REST record, update-to and updated-by are absent. The Crossreffilter=updates:query returned 0 items in an earlier check today, and a repeat query was rate-limited (HTTP 429). Europe PMC lists one linked item, a "Comment in" PNAS commentary (PMID 42475589, doi:10.1073/pnas.2617208123). It is not a correction, and only its metadata was available. - PubPeer. The PubPeer search API returned HTTP 403. A web search for the title together with "PubPeer", "correction" or "erratum" found no PubPeer thread and no correction, only news coverage.
Suggested correction
Change the title to give the direction that the paper's results report for water, for example:
The 2025–2030 Dietary Guidelines for Americans are associated with higher land and nitrogen use and greenhouse gas emissions, but lower water use
or, if the title should keep to the adverse outcomes only:
The 2025–2030 Dietary Guidelines for Americans are associated with higher land and nitrogen use, and greenhouse gas emissions
Either wording matches the abstract ("but not in water use (i.e., water use is lower by 7 to 19% than the MAD)"), the Results and the Discussion. No numbers, figures or conclusions in the body need to change.
Recomputation script (python recompute.py.txt)
"""Recompute the direction of each environmental outcome in PMC13273247
(Proc Natl Acad Sci U S A 2026; doi:10.1073/pnas.2604814123) from the paper's own
data, and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy and matplotlib)
Inputs are the cached cell values of the "summary" sheet (rows 5-11, columns F-K)
of Dataset S1 (pnas.2604814123.sd01.xlsx), the supporting-information workbook of
the article. The same sheet, with identical values, is in the data file the paper
links under Data Availability (NOVA+PROTEIN_DGA25-30_ver5.xlsx). Units are annual
per-capita totals, as in the paper's Fig. 1 (GHG kg CO2e, water litres, land m2-yr,
nitrogen fertilizer g Nr), plus daily kcal and protein (g).
Quoted text is copied verbatim from the PMC-archived article.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
METRICS = ["GHG", "Water", "Land", "N fertilizer"]
# Dataset S1, sheet "summary": GHG (F), water (G), land (H), N fertilizer (I), kcal (J), protein (K)
S1 = {
"MAD": [1620.3738784771374, 1884982.1709933979, 16238.053625377528, 48733.303782843548, 2500.3896000000004, 105.88118154726627],
"UPF (in MAD)": [747.61481329437038, 987787.62738541316, 6537.7528699819459, 28078.039550923815, 1636.7655999999999, 50.996616966708991],
"0.8 g/kg": [1253.3556489623356, 1362294.9058782607, 12576.133868282266, 47734.201604786555, 2499.6447058832632, 67.285004777165909],
"1.2 g/kg": [1579.418981096685, 1518270.9459631008, 16646.023128729423, 49705.242645896185, 2499.6957642603857, 100.84518830094257],
"1.6 g/kg": [1969.1244734977586, 1760953.6181536382, 21407.659406569564, 52485.30446813177, 2499.516060583017, 134.26708806627164],
"1.6 g/kg PLANT": [1497.7793450200566, 1415155.5062714668, 16597.693788935609, 43883.534702103032, 2499.6400527792202, 134.37530603776761],
"1.6 g/kg ANIMAL": [1985.3613800760761, 1637370.2722960121, 21199.247774193638, 52751.029784245409, 2500.187619501251, 134.21656241614866],
}
SCEN = ["0.8 g/kg", "1.2 g/kg", "1.6 g/kg", "1.6 g/kg PLANT", "1.6 g/kg ANIMAL"]
# Scenarios the authors' own "Range" cells (summary!K15:L15) pool: 1.2, 1.6 and 1.6 ANIMAL
DGA = ["1.2 g/kg", "1.6 g/kg", "1.6 g/kg ANIMAL"]
# What the paper states
REPORTED_WATER = (-19.0, -7.0) # Abstract: "water use is lower by 7 to 19% than the MAD"
REPORTED_MAX_OTHER = 32.0 # Abstract/Results: "by as much as 32%" (GHG, land, N fertilizer)
REPORTED_UPF_SHARE = (40.0, 58.0) # Results: "(40 to 58%)"
BODY_WEIGHT = 84.0 # Methods: average adult weight, kg
TITLE = ("The 2025–2030 Dietary Guidelines for Americans are associated with higher land, "
"water and nitrogen use, and greenhouse gas emissions")
ABSTRACT_Q = "but not in water use (i.e., water use is lower by 7 to 19% than the MAD)"
RESULTS_Q = "reduced the water footprint compared to MAD by 7 to 19%"
DISCUSSION_Q = "For water use, the net effect is beneficial"
def pc(a, b):
return 100.0 * (a - b) / b
mad = np.array(S1["MAD"])
ch = {k: pc(np.array(S1[k][:4]), mad[:4]) for k in SCEN}
print("=== 1. Percent change versus the Mean American Diet (MAD), Dataset S1 summary sheet ===")
print(f"{'scenario':18s}" + "".join(f"{m:>14s}" for m in METRICS) + f"{'kcal/d':>9s}{'protein g/d':>13s}")
for k in ["MAD"] + SCEN:
v = S1[k]
row = "".join(f"{x:+13.2f}%" for x in ch[k]) if k != "MAD" else "".join(f"{'(baseline)':>14s}" for _ in METRICS)
print(f"{k:18s}{row}{v[4]:9.1f}{v[5]:13.1f}")
print("\n=== 2. Water: the paper's stated range and the title's direction ===")
w_dga = [ch[k][1] for k in DGA]
print("Water change vs MAD, DGA-range scenarios (1.2, 1.6, 1.6 ANIMAL):", [f"{x:+.2f}%" for x in w_dga])
print(f" reductions run from {-max(w_dga):.1f}% to {-min(w_dga):.1f}% "
f"(paper: 'lower by {-REPORTED_WATER[1]:.0f} to {-REPORTED_WATER[0]:.0f}%') -> "
f"matches after rounding: {round(-max(w_dga)) == -REPORTED_WATER[1] and round(-min(w_dga)) == -REPORTED_WATER[0]}")
w_all = {k: ch[k][1] for k in SCEN}
print("Water change vs MAD, every modelled scenario:", {k: f"{v:+.1f}%" for k, v in w_all.items()})
print(" any scenario with HIGHER water use than the MAD?", any(v > 0 for v in w_all.values()))
print(" smallest reduction:", f"{max(w_all.values()):+.1f}%", " largest reduction:", f"{min(w_all.values()):+.1f}%")
print("\n=== 3. GHG, land and N fertilizer (the other three title outcomes) ===")
other = [(k, METRICS[j], ch[k][j]) for k in DGA for j in (0, 2, 3)]
lo = min(other, key=lambda t: t[2]); hi = max(other, key=lambda t: t[2])
print(f" range over 1.2, 1.6, 1.6 ANIMAL: {lo[2]:+.1f}% ({lo[1]}, {lo[0]}) to {hi[2]:+.1f}% ({hi[1]}, {hi[0]})")
print(f" paper: 'as much as {REPORTED_MAX_OTHER:.0f}%' -> max rounds to {round(hi[2])}: {round(hi[2]) == REPORTED_MAX_OTHER}")
for k in DGA:
print(f" {k:16s} GHG {ch[k][0]:+.1f}% land {ch[k][2]:+.1f}% N {ch[k][3]:+.1f}% | water {ch[k][1]:+.1f}%")
print("\n=== 4. Other numbers in the text ===")
share = [100 * S1["UPF (in MAD)"][j] / mad[j] for j in range(4)]
print("UPF share of MAD impacts (GHG, water, land, N):", [f"{x:.1f}%" for x in share],
f" range {min(share):.1f}-{max(share):.1f}% (paper: 40 to 58%)")
print(f"UPF share of MAD calories: {100*S1['UPF (in MAD)'][4]/mad[4]:.1f}% (paper: 'over 50%')")
print("Protein targets 0.8/1.2/1.6 g/kg x 84 kg =", [round(p * BODY_WEIGHT, 1) for p in (0.8, 1.2, 1.6)],
" modelled:", [round(S1[k][5], 1) for k in ("0.8 g/kg", "1.2 g/kg", "1.6 g/kg")])
print(f"MAD protein: {mad[5]:.1f} g/d = {mad[5]/BODY_WEIGHT:.2f} g/kg (paper: 'MAD protein intake level is 1.2 g/kg')")
print("\n=== 5. Alternative comparators examined as explanations for the title ===")
for k in ["1.2 g/kg", "1.6 g/kg", "1.6 g/kg ANIMAL"]:
print(f" water, {k:16s} vs 0.8 g/kg UPF-free diet: {pc(S1[k][1], S1['0.8 g/kg'][1]):+.1f}%")
print(f" water, 1.6 g/kg ANIMAL vs 1.6 g/kg PLANT: {pc(S1['1.6 g/kg ANIMAL'][1], S1['1.6 g/kg PLANT'][1]):+.1f}%")
print(" 1.6 g/kg ANIMAL vs 1.6 g/kg PLANT, (GHG, water, land, N):",
[f"{pc(S1['1.6 g/kg ANIMAL'][j], S1['1.6 g/kg PLANT'][j]):+.1f}%" for j in range(4)])
print(" water per gram of daily protein vs MAD:",
{k: f"{pc(S1[k][1]/S1[k][5], mad[1]/mad[5]):+.1f}%" for k in SCEN})
print(" water per 1,000 kcal vs MAD (all diets ~2,500 kcal):",
{k: f"{pc(S1[k][1]/S1[k][4], mad[1]/mad[4]):+.1f}%" for k in SCEN})
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# Figure 1: title vs body text (text-vs-text contradiction)
fig = plt.figure(figsize=(7.2, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.9); ax.axis("off")
def panel(x, y, w, h, head, body, edge, lw=2, ls="-", body_color=INK, size=10):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.06",
fc="white", ec=edge, lw=lw, ls=ls))
ax.text(x + 0.12, y + h - 0.12, head, ha="left", va="top", fontsize=10, color=INK2, weight="bold")
ax.text(x + 0.12, y + h - 0.42, body, ha="left", va="top", fontsize=size, color=body_color,
linespacing=1.35)
panel(0.15, 2.55, 2.5, 1.6, "Title",
"\u201c\u2026are associated with\nhigher land, water and\nnitrogen use, and\ngreenhouse gas emissions\u201d",
ORANGE, lw=2.5)
ax.text(0.27, 2.65, "Water: higher", color=ORANGE, fontsize=11, weight="bold", ha="left", va="bottom")
wd = {k: ch[k][1] for k in DGA}
fmt = lambda x: f"{x:+.1f}%".replace("-", "\u2212")
rx, rw = 4.0, 3.05
panel(rx, 3.62, rw, 0.98, "Abstract", "\u201c\u2026but not in water use (i.e., water\nuse is lower by 7 to 19% than the MAD)\u201d",
INK2, lw=1.2, size=9.5)
panel(rx, 2.66, rw, 0.86, "Results, 2nd paragraph", "\u201c\u2026reduced the water footprint\ncompared to MAD by 7 to 19%\u201d",
INK2, lw=1.2, size=9.5)
panel(rx, 1.90, rw, 0.66, "Discussion, 1st paragraph", "\u201cFor water use, the net effect is beneficial\u201d",
INK2, lw=1.2, size=9.5)
panel(rx, 0.55, rw, 1.25, "Recomputed from Dataset S1", "", BLUE, lw=2.5)
for i, k in enumerate(DGA):
yy = 1.38 - 0.19 * i
ax.text(rx + 0.12, yy, k.replace("ANIMAL", "animal"), color=BLUE, fontsize=10, ha="left", va="top")
ax.text(rx + 2.0, yy, fmt(wd[k]), color=BLUE, fontsize=10, ha="right", va="top")
ax.text(rx + 0.12, 0.64, "Water: lower in all scenarios", color=BLUE, fontsize=10, weight="bold",
ha="left", va="bottom")
# bracket grouping the right-hand panels
bx = rx - 0.12
ax.plot([bx, bx], [0.55, 4.60], color=INK2, lw=1.2)
for yy in (0.55, 4.60):
ax.plot([bx, bx + 0.07], [yy, yy], color=INK2, lw=1.2)
ax.add_patch(FancyArrowPatch((2.7, 3.35), (bx - 0.05, 2.6), arrowstyle="<|-|>", mutation_scale=14,
color=INK, lw=2, connectionstyle="arc3,rad=0.0"))
ax.text(3.27, 3.55, "Opposite\ndirection", ha="center", va="bottom", fontsize=10.5, weight="bold", color=INK)
ax.text(3.27, 2.55, "Paper's stated\ncomparator:\nMean American\nDiet (MAD)", ha="center", va="top", fontsize=9.5,
color=INK2, linespacing=1.3)
ax.text(0.15, 4.72, "Freshwater use under the 2025\u20132030 guideline diets, relative to the MAD",
fontsize=11, weight="bold", ha="left", va="center")
ax.text(0.15, 2.2, "Orange: the paper's headline\nclaim (title).\nGray: the paper's own text.\n"
"Blue: recomputed from the\npaper's Dataset S1\n(summary sheet).", fontsize=9.5, color=INK2,
ha="left", va="top", linespacing=1.35)
save(fig, "fig1")
# Figure 2: percent change vs MAD for all four title outcomes
fig, ax = plt.subplots(figsize=(7.2, 4.1))
fig.subplots_adjust(left=0.17, right=0.97, top=0.80, bottom=0.14)
order = ["GHG", "Water", "Land", "N fertilizer"]
ypos = {m: i for i, m in enumerate(reversed(order))}
marks = {"1.2 g/kg": "o", "1.6 g/kg": "s", "1.6 g/kg ANIMAL": "^", "1.6 g/kg PLANT": "v"}
offs = {"1.2 g/kg": 0.12, "1.6 g/kg": 0.04, "1.6 g/kg ANIMAL": -0.04, "1.6 g/kg PLANT": -0.12}
ax.axvline(0, color=INK2, lw=1)
for m in order:
j = METRICS.index(m); y = ypos[m]
vals = [ch[k][j] for k in marks]
ax.plot([min(vals), max(vals)], [y, y], color=BLUE, lw=2, alpha=0.35, zorder=1)
for k, mk in marks.items():
filled = k != "1.6 g/kg PLANT"
ax.plot(ch[k][j], y + offs[k], mk, ms=8.5, mec=BLUE, mfc=BLUE if filled else BG, mew=2, zorder=3)
# reported values
yw = ypos["Water"] + 0.33
ax.plot(REPORTED_WATER, [yw, yw], color=ORANGE, lw=2)
for x in REPORTED_WATER:
ax.plot([x, x], [yw - 0.07, yw + 0.07], color=ORANGE, lw=2)
ax.text(np.mean(REPORTED_WATER), yw + 0.1, "Abstract/Results: “lower by 7 to 19%”", color=INK,
fontsize=9.5, ha="center", va="bottom")
ax.annotate("", xy=(13, yw - 0.02), xytext=(0.6, yw - 0.02),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=13))
ax.text(13.8, yw - 0.02, "Title: “higher … water”", color=INK, fontsize=9.5, ha="left", va="center")
yl = ypos["Land"] + 0.33
ax.plot(REPORTED_MAX_OTHER, yl, "D", ms=8.5, color=ORANGE, zorder=3)
ax.text(REPORTED_MAX_OTHER - 1.2, yl + 0.02, "Text: “as much as 32%”", color=INK, fontsize=9.5,
ha="right", va="center")
ax.set_yticks([ypos[m] for m in order]); ax.set_yticklabels(order, fontsize=10.5, color=INK)
ax.set_ylim(-0.55, 3.62)
ax.set_xlim(-32, 40)
ax.set_xlabel("Change versus the Mean American Diet (%)", fontsize=10.5)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], ls="", marker=mk, ms=8, mec=BLUE, mew=2, mfc=BLUE if k != "1.6 g/kg PLANT" else BG,
label=k.replace("ANIMAL", "animal").replace("PLANT", "plant")) for k, mk in marks.items()]
h += [Line2D([], [], color=ORANGE, lw=2, marker="D", ms=7, label="reported in the paper")]
leg = ax.legend(handles=h, ncol=5, loc="lower left", bbox_to_anchor=(-0.2, 1.01), frameon=False,
fontsize=9, handletextpad=0.3, columnspacing=0.9)
fig.text(0.02, 0.965, "Blue: recomputed from Dataset S1. Orange: values and direction stated in the paper.",
fontsize=10, color=INK2, ha="left", va="top")
save(fig, "fig2")
print("\nFigures written:", [os.path.join(HERE, f"fig{i}.{e}") for i in (1, 2) for e in ("svg", "png")])
Printed hiPSC-CM telomere means contradict the stated 52% and 58% decreases and Figure 3C
Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1714538115 · PMC6140486
Summary
The Results report that cardiomyocytes derived from patient induced pluripotent stem cells (hiPSC-CMs) show "significant 52% and 58% decreases" in telomere signal for hypertrophic (HCM) and dilated (DCM) cardiomyopathy; the same sentence gives means of 6.16 ± 0.65 TFU for HCM and 5.70 ± 0.65 TFU for DCM, against a control mean of 5.83 ± 0.65 TFU. Those means differ from control by +5.7% and −2.2%, and as two-group comparisons they give P ≈ 0.73 and 0.89, not the stated P < 0.0005 and P < 0.0001. Figure 3C, which the sentence cites, shows group means of about 2.9 (HCM) and 2.3 (DCM) TFU against 5.83 for controls, decreases of about 50% and 60%, so the stated percentages and the figure agree and only the printed HCM and DCM means (and the shared ± 0.65) do not. This Q-FISH result is one of the "two independent assays" on which the Abstract's in vitro claim rests.
What the paper reports
Results, second paragraph (hiPSC-CM Q-FISH):
Importantly, in all cases, we measured telomeres in hiPSC-CMs at day 30 of differentiation. The five HCM hiPSC-CMs (6.16 ± 0.65 TFU, P < 0.0005; Table 1, blue) and eight DCM hiPSC-CMs (5.70 ± 0.65 TFU, P < 0.0001; Table 1, red) exhibited significant 52% and 58% decreases in telomere levels, respectively, compared with the control hiPSC-CMs of diverse etiology (5.83 ± 0.65 TFU; Fig. 3C, gray triangle and gray squares; Table 1, gray). These data are also shown for each patient line separately (Fig. 3C).
Abstract:
Telomere shortening was recapitulated in DCM and HCM CMs differentiated from patient-derived human-induced pluripotent stem cells (hiPSCs) measured by two independent assays.
Significance statement:
HCM and DCM CMs differentiated from human-induced pluripotent stem cells (hiPSCs) also exhibit significant telomere shortening relative to healthy controls.
Figure 3 legend (title, panel C, and final sentence):
HCM and DCM hiPSC-derived CMs recapitulate telomere shortening.
Telomere signal intensity per DAPI-stained nucleus is shown for HCM, DCM, and control hiPSCs (day 0) and hiPSC-CMs (day 30).
All data plotted as mean ± SEM (*P < 0.05, ***P < 0.001, and ****P < 0.0001).
Discussion:
We corroborated this shortening for patient hiPSC-CMs by two independent methods, Q-FISH analysis of single CMs (telomere signal relative to nuclear signal) and qPCR of FACS-enriched CM populations (telomere signal relative to single-copy gene signal).
Materials and Methods, statistics:
Statistical differences in primary patient samples and between the hiPSC and hiPSC-CM groups were analyzed by one-way ANOVA tests by comparing the mean of each group with the mean of every other group, followed by Holm–Šídák multiple comparison test.
All data are shown as the mean ± SEM.
The two other percentage statements used below as controls for the method (Results, first and third paragraphs):
the telomere levels of Troponin-T+ CMs in HCM and DCM hearts were significantly reduced by, on average, 25% and 40%, respectively, compared with age- and sex-matched healthy controls (HCM, 4.64 ± 0.46 TFU, P < 0.05; DCM, 3.70 ± 1.05 TFU, P < 0.0001; control, 6.21 ± 0.16 TFU; Fig. 1C).
The T/S ratios in HCM and DCM TMRM+ CMs were significantly reduced by, on average, 21% and 39%, respectively, compared with control CMs (HCM, 2.67 ± 0.18, P < 0.05; DCM, 2.06 ± 0.16, P < 0.0005; control, 3.40 ± 0.24; Fig. 3F).
What the paper's own numbers show
Method. I computed the percent change of each printed group mean against the printed control mean, 100 × (group / control − 1), and the reverse: the group mean that each stated decrease implies at the printed control (5.83 × (1 − d)), and the control that each printed group mean would need. I tested whether the printed means and SEMs could give the stated P values, using two-group Welch t tests and a one-way ANOVA built from the summary statistics with the line counts the text gives (5, 5 and 8). I then digitized the published Fig. 3C by pixel analysis: the axis ticks, the mean line and SEM caps of each group in the left panel (one dot per line), and the top edge of each of the 18 per-line bars in the right panel. One pixel is about 0.05–0.06 TFU, so these readings are approximate to that level; the pixel rows are listed in recompute.py.txt, which converts them. As a check on the method, I applied the same percent arithmetic to the paper's tissue Q-FISH and hiPSC-CM qPCR statements. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| HCM hiPSC-CM vs control | Results ¶2 | 6.16 vs 5.83 TFU; "52% decrease" | 6.16 / 5.83 = 1.057, +5.7% (an increase) |
| DCM hiPSC-CM vs control | Results ¶2 | 5.70 vs 5.83 TFU; "58% decrease" | 5.70 / 5.83 = 0.978, −2.2% |
| Mean implied by the stated decrease at control 5.83 | derived | 52%, 58% | HCM 2.80 TFU, DCM 2.45 TFU |
| Control implied by the stated decrease at the printed group mean | derived | 5.83 TFU | 12.83 (from HCM) and 13.57 (from DCM): no single control value fits both |
| HCM vs control P from printed means ± SEM | Results ¶2 | P < 0.0005 | Welch t = +0.36, P = 0.73 |
| DCM vs control P from printed means ± SEM | Results ¶2 | P < 0.0001 | Welch t = −0.14, P = 0.89 |
| One-way ANOVA from printed means ± SEM (5, 5, 8 lines) | Methods; Results ¶2 | significant | F(2,15) = 0.12, P = 0.89 |
| Control group mean ± SEM | Fig. 3C left panel | 5.83 ± 0.65 (text) | 5.83 ± 0.32 (read from figure) |
| HCM group mean ± SEM | Fig. 3C left panel | 6.16 ± 0.65 (text) | 2.94 ± 0.44 (read from figure) |
| DCM group mean ± SEM | Fig. 3C left panel | 5.70 ± 0.65 (text) | 2.32 ± 0.35 (read from figure) |
| Per-line bars, mean (range) | Fig. 3C right panel | — | control 5.80 (4.75–6.64); HCM 2.90 (1.55–4.17); DCM 2.31 (1.13–3.86) |
| Decrease vs control from the figure | Fig. 3C | 52% (HCM), 58% (DCM) | left panel −49.5% and −60.2%; bars −49.9% and −60.2% |
| ANOVA + Holm–Šídák on the per-line bar values | Fig. 3C; Methods | P < 0.0005 (HCM), P < 0.0001 (DCM); stars ***, **** | F(2,15) = 23.1; adjusted P = 1.8 × 10⁻⁴ (HCM), 1.6 × 10⁻⁵ (DCM) |
| Tissue Q-FISH decrease (control for the method) | Results ¶1; Abstract | 25% (Results) and 26% (Abstract) HCM; 40% DCM | 25.3% and 40.4% |
| hiPSC-CM qPCR T/S decrease (control for the method) | Results ¶3 | 21% HCM, 39% DCM | 21.5% and 39.4% |
The same arithmetic reproduces the paper's other four percentages to within rounding (the Abstract's 26% for tissue HCM is a minor rounding difference from the Results' 25%; 25.3% is exact). For the hiPSC-CM Q-FISH result the printed HCM and DCM means give the wrong sign for HCM and a negligible change for DCM, and they cannot produce the stated P values. The figure's group means, its per-line bars and its significance stars all fit the stated 52% and 58% decreases and P values. The figure's control mean matches the printed 5.83, but its SEMs (0.32, 0.44, 0.35) differ from the ± 0.65 printed identically for all three groups. No average of the plotted HCM or DCM lines, however weighted, can reach 6.16 or 5.70: the highest HCM bar is 4.17 and the highest DCM bar 3.86 TFU.
Figure
Figure 1. hiPSC-CM telomere signal (TFU) for the control, HCM and DCM groups, on one axis starting at zero. Orange circles: the mean ± SEM printed in the Results text. Blue diamonds: the group mean that the stated 52% and 58% decreases imply against the printed control mean of 5.83 (5.83 × 0.48 = 2.80 and 5.83 × 0.42 = 2.45). Blue open squares: the group mean ± SEM read from the published Fig. 3C left panel by pixel digitization (approximate to about 0.06 TFU). The dashed line marks the printed control mean, 5.83. The printed control matches the figure; the printed HCM and DCM means sit at the control level, while the stated decreases and the figure both place those groups near 2.3–2.9 TFU.
Why it matters
The Abstract states that telomere shortening "was recapitulated" in patient hiPSC-CMs "measured by two independent assays", the Significance statement says these cells "also exhibit significant telomere shortening", and the Discussion repeats that the shortening was corroborated by Q-FISH and qPCR. The Q-FISH sentence is the only place in the text where the hiPSC-CM Q-FISH effect is quantified. A reader who works from the printed means would find no shortening at all: HCM above control and DCM 2% below, with differences far inside one SEM. The text therefore does not support the claim it makes, and a reader cannot tell from it which numbers are correct.
The figure resolves most of this. Its group means and per-line bars give decreases of about 50% and 60%, close to the stated 52% and 58%, and P values of the stated order. On that reading the in vitro Q-FISH finding stands and the error is in the printed HCM and DCM means and in the SEM given for all three groups. The size of the in vitro effect also matters for the paper's comparisons: the hiPSC-CM decreases (about 50–60%) are larger than the tissue decreases (25% and 40%), which the printed means would hide.
Not affected: the tissue Q-FISH results (25.3% and 40.4%, reproduced), the VSMC null results, and the hiPSC-CM qPCR result (21.5% and 39.4%, reproduced), which is the second of the two assays.
Explanations considered
- Different time point or cell type (for example, day-0 hiPSC values instead of day-30 hiPSC-CMs). This cannot be excluded, but it cannot be confirmed either. The text says "in all cases, we measured telomeres in hiPSC-CMs at day 30 of differentiation" and names the control hiPSC-CMs as the reference. The Fig. 3 legend says panel C shows both "hiPSCs (day 0) and hiPSC-CMs (day 30)", but the published panel shows one set of values per line and does not label the day; it also has no gray triangle, which the text cites. If the printed 6.16, 5.70 and 5.83 belong to another data set, that would explain the numbers, but it would still leave the text's hiPSC-CM means wrong.
- A wrong control value. This does not account for the numbers. The stated decreases would need a control of 12.83 TFU (from HCM) and 13.57 TFU (from DCM), which no single value satisfies, and the figure's control mean matches the printed 5.83.
- Different definition, units or weighting of the means. This does not account for the numbers. All three means are in TFU, in one sentence, against one control. The per-line bars cap the possible group means: weighting the lines by the nucleus counts in Table 1 gives 6.05 (control), 3.34 (HCM) and 2.51 (DCM), still nowhere near 6.16 and 5.70.
- Rounding. This does not account for the numbers. The gap is a change of sign for HCM and a factor of about 25 in the size of the DCM change.
- Conversion or typesetting error in the archive. This is unlikely. The numbers are plain characters in the publisher-supplied JATS XML at Europe PMC and in the PMC text, with no superscripts, minus signs or table cells involved, and both copies agree. The publisher's HTML page could not be fetched (see Status).
- A correction already published. None was found (see Status).
Status
- Version of record. I compared the PMC-archived text (via
src/show.py) with the publisher-supplied JATS XML at Europe PMC (PMC6140486/fullTextXML). Both carry the quoted sentences with identical values. The doi.org link resolves to pnas.org, which returned HTTP 403 to automated requests, so the publisher's HTML and PDF were not read directly. Every quotation above was confirmed verbatim against the PMC-archived text. Figure 3 was read from the PMC copy of the published image (pnas.1714538115fig03.jpg); the figure readings are mine, by pixel digitization, and are approximate. - Supplements. The JATS XML has no supplementary-material element, and the PMC record lists none. The PNAS supplement URL returned HTTP 403, so the existence of an SI appendix could not be confirmed. No source data are deposited, so the exact group means cannot be computed from published material.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. A web search for corrections or comments found none. PubPeer refused automated access (HTTP 403), so it could not be checked directly.
- Sources. PMC article PMC6140486; Europe PMC full-text XML for PMC6140486; the PMC copy of Figure 3; Crossref; Europe PMC.
Suggested correction
- Replace the hiPSC-CM Q-FISH means in Results ¶2 with the values that underlie Fig. 3C and the stated decreases. The figure indicates about 2.9 TFU (HCM) and 2.3 TFU (DCM) against 5.83 TFU (control). Report each group's own SEM (the figure indicates about 0.44, 0.35 and 0.32), not one shared value.
- Confirm that the stated 52% and 58% decreases and the P values were computed from those corrected means, and state the unit of analysis for the ANOVA (lines or nuclei).
- If 6.16, 5.70 and 5.83 TFU are the day-0 hiPSC values mentioned in the Fig. 3 legend, report them as such, and either show the day-0 data in Fig. 3C or remove the reference to them from the legend; also reconcile the text's reference to a "gray triangle", which does not appear in the panel.
- Minor: reconcile the tissue HCM decrease given as 26% in the Abstract and 25% in the Results (4.64 / 6.21 gives 25.3%).
Recomputation script (python recompute.py.txt)
"""Recompute the hiPSC-CM Q-FISH numbers of PMC6140486
(Proc Natl Acad Sci U S A 2018, doi:10.1073/pnas.1714538115,
"Telomere shortening is a hallmark of genetic cardiomyopathies").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg and fig1.png next to this file.
Inputs copied from the paper as archived in PMC:
Results, hiPSC-CM paragraph (Q-FISH, day 30):
HCM 6.16 +/- 0.65 TFU (5 lines), DCM 5.70 +/- 0.65 TFU (8 lines), control 5.83 +/- 0.65 TFU (5 lines);
"significant 52% and 58% decreases", P < 0.0005 (HCM) and P < 0.0001 (DCM)
Results, tissue paragraph (Q-FISH): HCM 4.64, DCM 3.70, control 6.21 TFU; "25% and 40%" (Abstract: 26% and 40%)
Results, qPCR paragraph: T/S HCM 2.67, DCM 2.06, control 3.40; "21% and 39%"
Table 1: Q-FISH nucleus counts (n) per hiPSC line
Methods: one-way ANOVA, Holm-Sidak multiple comparisons, data shown as mean +/- SEM
Figure 3C readings. The published Fig. 3C (PMC file pnas.1714538115fig03.jpg, 757 x 539 px) was
digitized by pixel analysis; only the pixel rows below are used here, and the image itself is not
reproduced. One pixel is about 0.05 TFU in the right panel and 0.06 TFU in the left panel.
Right panel (one bar per line): y-axis ticks 0, 2, 4, 6, 8 at rows 477.5, 439.5, 401.5, 363, 325;
top edge of each bar (row of the horizontal bar outline) listed in BAR_ROWS.
Left panel (one dot per line, group mean +/- SEM): ticks 0..10 every 2 at rows 479.5 ... 310;
mean line and SEM caps of each group listed in DOT_ROWS.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs from the text
TEXT = {"Control": (5.83, 0.65, 5), "HCM": (6.16, 0.65, 5), "DCM": (5.70, 0.65, 8)} # mean, SEM, lines
STATED_DECREASE = {"HCM": 0.52, "DCM": 0.58}
STATED_P = {"HCM": "P < 0.0005", "DCM": "P < 0.0001"}
TISSUE = {"Control": 6.21, "HCM": 4.64, "DCM": 3.70}
TISSUE_STATED = {"HCM": "25% (Results), 26% (Abstract)", "DCM": "40%"}
QPCR = {"Control": 3.40, "HCM": 2.67, "DCM": 2.06}
QPCR_STATED = {"HCM": "21%", "DCM": "39%"}
# ---------------------------------------------------------------- Fig. 3C pixel readings
BAR_TICKS_ROW = np.array([477.5, 439.5, 401.5, 363.0, 325.0])
BAR_TICKS_VAL = np.array([0, 2, 4, 6, 8])
BAR_ROWS = { # line: (row of bar top, Table 1 Q-FISH n)
"Control": {"Con#1": (368, 92), "Con#2": (371, 40), "Con#3": (387, 44), "Con#4": (351, 100), "Con#5": (358, 210)},
"HCM": {"MYH7#2": (422, 79), "MYH7#4": (448, 37), "MYBPC3#1": (412, 519), "MYH7#1": (398, 115), "MYH7#3": (431, 54)},
"DCM": {"DMD#1": (428, 129), "DMD#2": (438, 48), "DMD#3": (430, 164), "TNNT2#1": (449, 117),
"TNNT2#2": (456, 131), "TNNT2#3": (452, 50), "TTN#1": (404, 184), "TTN#2": (412, 133)},
}
DOT_TICKS_ROW = np.array([479.5, 445.5, 411.5, 377.5, 343.5, 310.0])
DOT_TICKS_VAL = np.array([0, 2, 4, 6, 8, 10])
DOT_ROWS = {"Control": (380.5, 375, 386), "HCM": (429.5, 422, 437), "DCM": (440, 434, 446)} # mean, upper cap, lower cap
kb, bb = np.polyfit(BAR_TICKS_ROW, BAR_TICKS_VAL, 1)
kd, bd = np.polyfit(DOT_TICKS_ROW, DOT_TICKS_VAL, 1)
GROUPS = ["Control", "HCM", "DCM"]
def pct(x, ref):
return 100 * (x / ref - 1)
R = {}
print("=== 1. Percent change implied by the text's own Q-FISH means (hiPSC-CM, day 30) ===")
c = TEXT["Control"][0]
for g in ("HCM", "DCM"):
m = TEXT[g][0]
R[g + "_textpct"] = pct(m, c)
print(f"{g}: {m:.2f} / {c:.2f} = {m / c:.3f} -> {pct(m, c):+.1f}% (stated: {100 * STATED_DECREASE[g]:.0f}% decrease)")
print("\n=== 2. What the stated decreases imply ===")
for g in ("HCM", "DCM"):
d = STATED_DECREASE[g]
R[g + "_implied_mean"] = c * (1 - d)
R[g + "_implied_ctrl"] = TEXT[g][0] / (1 - d)
print(f"{g}: {100 * d:.0f}% below control 5.83 -> mean {c * (1 - d):.2f} TFU; "
f"or, keeping {TEXT[g][0]:.2f}, a control of {TEXT[g][0] / (1 - d):.2f} TFU")
print("A single wrong control value cannot fit both: the two implied controls differ by "
f"{R['DCM_implied_ctrl'] - R['HCM_implied_ctrl']:.2f} TFU.")
print("\n=== 3. Can the printed means and SEMs give the printed P values? (two-group checks) ===")
for g in ("HCM", "DCM"):
m1, s1, n1 = TEXT[g]
m0, s0, n0 = TEXT["Control"]
se = np.hypot(s1, s0)
t = (m1 - m0) / se
sd1, sd0 = s1 * np.sqrt(n1), s0 * np.sqrt(n0)
df = se ** 4 / ((s1 ** 2) ** 2 / (n1 - 1) + (s0 ** 2) ** 2 / (n0 - 1))
p = 2 * stats.t.sf(abs(t), df)
R[g + "_text_p"] = p
print(f"{g} vs control, Welch t from the printed values: t = {t:+.2f}, df = {df:.1f}, P = {p:.2f} (stated {STATED_P[g]})")
# one-way ANOVA from summary statistics (pooled variance), as in the Methods
ns = np.array([TEXT[g][2] for g in GROUPS])
ms = np.array([TEXT[g][0] for g in GROUPS])
sds = np.array([TEXT[g][1] * np.sqrt(TEXT[g][2]) for g in GROUPS])
N, k = ns.sum(), len(ns)
grand = (ns * ms).sum() / N
ssb = (ns * (ms - grand) ** 2).sum()
ssw = ((ns - 1) * sds ** 2).sum()
F = (ssb / (k - 1)) / (ssw / (N - k))
pF = stats.f.sf(F, k - 1, N - k)
R["text_anova_p"] = pF
print(f"One-way ANOVA from the printed means/SEMs (n = 5, 5, 8 lines): F({k - 1},{N - k}) = {F:.2f}, P = {pF:.2f}")
print("\n=== 4. Fig. 3C, left panel: group means and SEMs (one dot per line) ===")
FIGDOT = {}
for g in GROUPS:
mr, up, lo = DOT_ROWS[g]
mean = kd * mr + bd
sem = (kd * up - kd * lo) / 2
FIGDOT[g] = (mean, sem)
print(f"{g}: mean {mean:.2f} TFU, SEM {sem:.2f} (text: {TEXT[g][0]:.2f} +/- {TEXT[g][1]:.2f})")
print("\n=== 5. Fig. 3C, right panel: per-line bars ===")
FIGBAR = {}
for g in GROUPS:
vals = np.array([kb * r + bb for r, _ in BAR_ROWS[g].values()])
nn = np.array([n for _, n in BAR_ROWS[g].values()])
FIGBAR[g] = vals
print(f"{g}: " + ", ".join(f"{name} {v:.2f}" for name, v in zip(BAR_ROWS[g], vals)))
print(f" n = {len(vals)} lines, mean {vals.mean():.2f}, SEM {vals.std(ddof=1) / np.sqrt(len(vals)):.2f}, "
f"range {vals.min():.2f}-{vals.max():.2f}, mean weighted by Table 1 nuclei {np.average(vals, weights=nn):.2f}")
for g in ("HCM", "DCM"):
R[g + "_figpct"] = pct(FIGBAR[g].mean(), FIGBAR["Control"].mean())
R[g + "_figpct_txtctrl"] = pct(FIGBAR[g].mean(), c)
print(f"{g} vs control from the bars: {R[g + '_figpct']:+.1f}% (vs the text's control 5.83: {R[g + '_figpct_txtctrl']:+.1f}%; "
f"vs the left-panel means: {pct(FIGDOT[g][0], FIGDOT['Control'][0]):+.1f}%; stated -{100 * STATED_DECREASE[g]:.0f}%)")
print(f"Highest HCM bar {FIGBAR['HCM'].max():.2f} and highest DCM bar {FIGBAR['DCM'].max():.2f}: "
"no average of the plotted lines can reach 6.16 or 5.70.")
print("\n=== 6. ANOVA + Holm-Sidak on the per-line bar values ===")
F2, pF2 = stats.f_oneway(*[FIGBAR[g] for g in GROUPS])
dfw = sum(len(FIGBAR[g]) for g in GROUPS) - 3
msw = sum(((FIGBAR[g] - FIGBAR[g].mean()) ** 2).sum() for g in GROUPS) / dfw
print(f"F(2,{dfw}) = {F2:.1f}, P = {pF2:.1e}")
raw = {}
for g in ("HCM", "DCM"):
a, b = FIGBAR[g], FIGBAR["Control"]
t = (a.mean() - b.mean()) / np.sqrt(msw * (1 / len(a) + 1 / len(b)))
raw[g] = 2 * stats.t.sf(abs(t), dfw)
print(f"{g} vs control: pooled t = {t:.2f}, unadjusted P = {raw[g]:.1e}")
order = sorted(raw, key=raw.get)
adj, prev = {}, 0
for i, g in enumerate(order): # Holm-Sidak step-down over the two comparisons with control
a = 1 - (1 - raw[g]) ** (len(order) - i)
prev = max(prev, a)
adj[g] = prev
for g in ("HCM", "DCM"):
R[g + "_fig_p"] = adj[g]
print(f"{g} vs control: Holm-Sidak adjusted P = {adj[g]:.1e} (stated {STATED_P[g]}; figure stars "
f"{'***' if g == 'HCM' else '****'})")
print("(The paper's ANOVA compares every group with every other and its unit of analysis is not stated, "
"so these P values show order of magnitude only.)")
print("\n=== 7. Other percentages in the paper, for comparison ===")
for g in ("HCM", "DCM"):
R[g + "_tissue"] = -pct(TISSUE[g], TISSUE["Control"])
R[g + "_qpcr"] = -pct(QPCR[g], QPCR["Control"])
print(f"Tissue Q-FISH {g}: 1 - {TISSUE[g]:.2f}/{TISSUE['Control']:.2f} = {R[g + '_tissue']:.1f}% (stated {TISSUE_STATED[g]})")
print(f"hiPSC-CM qPCR {g}: 1 - {QPCR[g]:.2f}/{QPCR['Control']:.2f} = {R[g + '_qpcr']:.1f}% (stated {QPCR_STATED[g]})")
# ---------------------------------------------------------------- figure
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": TXT, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
fig, ax = plt.subplots(figsize=(7.2, 4.3))
ypos = {"Control": 2.7, "HCM": 1.35, "DCM": 0}
off = {"text": 0.27, "implied": 0.0, "fig": -0.27}
for g in GROUPS:
y = ypos[g]
m, s, _ = TEXT[g]
ax.errorbar(m, y + off["text"], xerr=s, fmt="o", ms=9, color=ORANGE, ecolor=ORANGE, elinewidth=2, capsize=4, zorder=3)
fm, fs = FIGDOT[g]
ax.errorbar(fm, y + off["fig"], xerr=fs, fmt="s", ms=8.5, mfc=BG, mec=BLUE, mew=2, ecolor=BLUE,
elinewidth=2, capsize=4, zorder=3)
if g != "Control":
im = R[g + "_implied_mean"]
ax.plot(im, y + off["implied"], "D", ms=8.5, color=BLUE, zorder=3)
ax.text(m + s + 0.15, y + off["text"], f"printed mean: {R[g + '_textpct']:+.1f}% vs control".replace("-", "\u2212"),
va="center", ha="left", fontsize=10, color=ORANGE)
ax.text(im + 0.2, y + off["implied"], f"stated \u2212{100 * STATED_DECREASE[g]:.0f}% implies {im:.2f}",
va="center", ha="left", fontsize=10, color=BLUE,
bbox=dict(boxstyle="square,pad=0.1", fc=BG, ec="none"))
ax.text(fm + fs + 0.15, y + off["fig"], f"Fig. 3C: {pct(fm, FIGDOT['Control'][0]):+.0f}%".replace("-", "\u2212"),
va="center", ha="left", fontsize=10, color=BLUE)
else:
ax.text(m + s + 0.15, y + off["text"], "printed 5.83 \u00b1 0.65", va="center", ha="left", fontsize=10, color=ORANGE)
ax.text(fm + fs + 0.15, y + off["fig"], f"Fig. 3C control {fm:.2f}", va="center", ha="left", fontsize=10, color=BLUE)
ax.axvline(c, color=SEC, lw=0.8, ls="--", zorder=1)
ax.set_yticks([2.7, 1.35, 0])
ax.set_yticklabels(["Control\n(5 lines)", "HCM\n(5 lines)", "DCM\n(8 lines)"])
ax.set_ylim(-0.55, 3.3)
ax.tick_params(axis="y", length=0)
ax.set_xlim(0, 10)
ax.set_xlabel("hiPSC-CM telomere signal, day 30 (TFU), mean ± SEM")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
handles = [
Line2D([], [], marker="o", color=ORANGE, ms=9, lw=2, label="Mean ± SEM printed in the Results text"),
Line2D([], [], marker="D", color=BLUE, ms=8.5, lw=0, label="Mean implied by the stated 52% / 58% decrease from 5.83"),
Line2D([], [], marker="s", color=BLUE, mfc=BG, mew=2, ms=8.5, lw=2, label="Group mean ± SEM read from Fig. 3C"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, -0.17), ncol=1, frameon=False, fontsize=10)
ax.set_title("hiPSC-CM Q-FISH: printed means vs stated decreases and Fig. 3C",
fontsize=11, loc="left", color=TXT)
fig.tight_layout()
fig.savefig(os.path.join(HERE, "fig1.svg"))
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200)
print("\nwrote fig1.svg and fig1.png")
The abstract names the inhibited RNA strand as negative, but the paper's own data show positive
Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1805195115 · PMC6176606
Summary
The paper studies a dengue virus genome carrying an NS3-G459L mutation that blocks an internal self-cleavage and acts as a trans-dominant inhibitor of wild-type virus in the same cell. The abstract states that this variant "dominantly inhibited negative-strand RNA synthesis." The Results section, using strand-specific RT-qPCR (Fig. 5 F and G), reports the opposite direction: positive-strand synthesis was strongly inhibited, while negative-strand synthesis was allowed (negative-strand RNA was significantly greater than the polymerase-dead control). The Results conclusion and the Discussion both state that positive-strand synthesis is the inhibited step. The single word "negative-strand" in the abstract therefore reverses the direction of the paper's key mechanistic result, and should read "positive-strand."
What the paper reports
an NS3-G459L variant that is incapable of cleavage at the internal NS3 junction dominantly inhibited negative-strand RNA synthesis of wild-type virus present in the same cell
— Abstract
To identify whether the dominant NS3-G459L mutant genome inhibited negative-strand synthesis (and therefore positive-strand synthesis as well) or just positive-strand synthesis, we performed strand-specific RT-qPCR to measure both strands of WT and GDD replicons (Fig. 5 F and G). ... The amount of positive-strand replicon RNA detected when cotransfected with the NS3-G459L dominant mutant was indistinguishable from that detected for the GDD replicon alone, indicating that positive-strand synthesis was strongly inhibited (Fig. 5F). By contrast, the amount of negative-strand replicon RNA was significantly greater than that for the GDD replicon alone (Fig. 5G), indicating that the NS3-G459L dominant mutant allows negative-strand synthesis and therefore specifically inhibits positive-strand synthesis.
— Results, "NS3-G459L Is a Dominant Inhibitor of WT Viral Growth"
Most importantly, viral genomes that abrogate this self-processing event trans-dominantly inhibit positive-strand RNA synthesis of WT genomes present in the same cell
— Discussion, "NS3 Internal Cleavage"
(F) Positive-strand replicon RNA was detected by strand-specific RT-qPCR. (G) Negative-strand replicon RNA was detected by strand-specific RT-qPCR (n = 2).
— Legend to Fig. 5
What the paper's own numbers show
There is no arithmetic to redo. The discrepancy is a reversed qualitative direction between two parts of the same paper describing one experiment: a strand-specific RT-qPCR of the luciferase replicon cotransfected with the NS3-G459L variant, measured at a single time point and normalized to the wild-type replicon alone, with the polymerase-dead GDD replicon as the no-synthesis reference. The two panels of that one experiment (Fig. 5F, positive strand; Fig. 5G, negative strand) are each reported relative to the GDD control. Taking those two relations as stated fixes which strand is inhibited.
| quantity | location | reported | recomputed (from the paper's own numbers) |
|---|---|---|---|
| Strand whose synthesis G459L inhibits | Abstract | negative-strand | positive-strand |
| Positive-strand replicon RNA, G459L vs GDD-dead | Results / Fig. 5F | = GDD → positive synthesis strongly inhibited | positive-strand inhibited (consistent with body) |
| Negative-strand replicon RNA, G459L vs GDD-dead (n = 2) | Results / Fig. 5G | > GDD → negative synthesis allowed | negative-strand allowed, not inhibited |
| Strand inhibited (stated) | Results conclusion | positive-strand | positive-strand |
| Strand inhibited (stated) | Discussion | positive-strand | positive-strand |
Four statements in the body (Fig. 5F, Fig. 5G, the Results conclusion, and the Discussion) place the inhibition on the positive strand; one statement in the abstract places it on the negative strand. recompute.py.txt encodes the two reported relations and prints VERDICT: claim CONFIRMED.
Figure
The abstract (left, orange, as reported) states that negative-strand synthesis is inhibited. The Results and Discussion (right, blue, from the paper's own numbers) state that positive-strand synthesis is inhibited while negative-strand synthesis is allowed, based on the strand-specific RT-qPCR in Fig. 5F/G of the same cotransfection experiment. The connector names the conflict: the two descriptions assign the inhibition to opposite strands of one assay. Colors are reinforced by the panel labels and legend so the distinction does not rely on color alone.
Why it matters
Which RNA strand's synthesis is blocked is the mechanistic heart of this result. A reader using the abstract alone would conclude that the dominant-negative genome blocks negative-strand (template) synthesis, an early step of replication. The paper's data support the reverse: negative-strand synthesis proceeds and positive-strand (progeny genome) synthesis is specifically inhibited. Flavivirus and antiviral-mechanism researchers who cite the abstract without reading the Results could adopt an inverted picture of where in the replication cycle the trans-dominant block acts. The paper's broader conclusion — that abrogating the internal NS3 self-cleavage yields trans-dominant inhibition of viral replication, supporting the proposed antiviral strategy — is unaffected in direction, magnitude, or significance, because it does not depend on which strand is affected. The error is confined to one word of one abstract sentence and is contradicted correctly elsewhere in the same paper.
Explanations considered
- A different experiment or population in the abstract vs. the body. No. Both refer to the same strand-specific RT-qPCR of the replicon cotransfected with NS3-G459L; the abstract summarizes exactly the experiment reported in Fig. 5 F and G.
- A different time point, normalization, or unit. No. It is one assay (single time point, normalized to the wild-type replicon alone); Fig. 5F and 5G are simply the positive- and negative-strand measurements of that one experiment.
- Loose reading: inhibiting positive-strand synthesis indirectly lowers negative-strand RNA, so "inhibited negative" is approximately true. No. Fig. 5G reports negative-strand RNA as significantly greater than the polymerase-dead GDD control, and the text explicitly calls negative-strand synthesis "allowed," not reduced.
- A logical-framing rescue in which "inhibited negative" is a defensible summary. No. The Results pose an explicit dichotomy — the mutant either "inhibited negative-strand synthesis (and therefore positive-strand synthesis as well) or just positive-strand synthesis" — and answer "just positive-strand." The abstract asserts the alternative the experiment rejected.
- A one-sided test, an adjusted-vs-unadjusted value, or a stated convention. Not applicable. This is a reversed qualitative direction, not a statistical-threshold or reporting-convention issue.
- A published correction or erratum that resolves it. None.
known.pyreturns empty Crossref, PMC, and EuropePMC notices, and no PubPeer thread was found.
Status
- Version of record checked. The error is present in the version of record, not an archiving artifact. The abstract sentence reading "negative-strand RNA synthesis" appears identically in the Europe PMC publisher-deposited full-text XML and in the PMC HTML; the PNAS publisher page could not be read directly (HTTP 403), but a third-party citation page independently quotes the abstract with "negative-strand RNA synthesis." The Results and Discussion text quoted above comes from the PMC archive via
src/show.py. - Supplements. The SI Appendix (primer sequences for the strand-specific RT-qPCR and supporting figures) does not bear on the discrepancy, which is a single word choice in the abstract rather than a numeric or table value. The main-text Results, the Fig. 5F/G legend, and the Discussion already give the correct, consistent direction.
- Corrections and PubPeer. No correction, erratum, or retraction notice found via Crossref, PMC, or EuropePMC (
known.py); no PubPeer thread located by search. - Sources. Abstract, Results, Discussion, and Fig. 5 legend as archived in PubMed Central (PMC6176606); Crossref/PMC/EuropePMC correction records via
known.py. As of 2026-10-23.
Suggested correction
In the abstract, change "dominantly inhibited negative-strand RNA synthesis of wild-type virus present in the same cell" to "dominantly inhibited positive-strand RNA synthesis of wild-type virus present in the same cell," making the abstract agree with the Results (Fig. 5 F and G), the Results conclusion, and the Discussion. No figure, table, or numeric value needs to change.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""
Recompute / re-derivation for the documented error in PMC6176606
(Proc Natl Acad Sci U S A 2018, doi:10.1073/pnas.1805195115).
The error is a reversed qualitative direction, not an arithmetic slip:
the Abstract names one RNA strand (negative) as the one whose synthesis the
NS3-G459L trans-dominant variant inhibits, while the Results (strand-specific
RT-qPCR, Fig. 5 F and G) and the Discussion both report the opposite --
positive-strand synthesis is inhibited and negative-strand synthesis is allowed.
Running this file:
python recompute.py.txt
prints every value used in the write-up and regenerates fig1.svg / fig1.png
in the folder that holds this script.
No data are taken from outside the paper. The RT-qPCR panels (Fig. 5F, 5G) are
reported in the paper only qualitatively relative to the polymerase-dead GDD
control ("indistinguishable from" / "significantly greater than"); those two
relations are the inputs, encoded verbatim below.
"""
import os
# ----------------------------------------------------------------------
# Inputs: the paper's own statements (strand-specific RT-qPCR, one assay,
# 26 h, normalized to the WT replicon alone). GDD = polymerase-dead control.
# ----------------------------------------------------------------------
statements = {
"abstract": {
"text": ("an NS3-G459L variant that is incapable of cleavage at the "
"internal NS3 junction dominantly inhibited NEGATIVE-strand "
"RNA synthesis of wild-type virus present in the same cell"),
"inhibited_strand": "negative",
},
"results_Fig5F_positive": {
"observation": "positive-strand RNA (G459L) == GDD-dead control",
"meaning": "positive-strand synthesis strongly INHIBITED",
"inhibited": "positive",
},
"results_Fig5G_negative": {
"observation": "negative-strand RNA (G459L) > GDD-dead control",
"meaning": "negative-strand synthesis ALLOWED (not inhibited)",
"allowed": "negative",
"n": 2,
},
"results_conclusion": {
"text": ("the NS3-G459L dominant mutant allows negative-strand "
"synthesis and therefore specifically inhibits "
"positive-strand synthesis"),
"inhibited_strand": "positive",
"allowed_strand": "negative",
},
"discussion": {
"text": ("viral genomes that abrogate this self-processing event "
"trans-dominantly inhibit positive-strand RNA synthesis of "
"WT genomes present in the same cell"),
"inhibited_strand": "positive",
},
}
# ----------------------------------------------------------------------
# Re-derivation
# ----------------------------------------------------------------------
body_inhibited = "positive" # Fig. 5F + Results conclusion + Discussion
body_allowed = "negative" # Fig. 5G + Results conclusion
abstract_inhibited = statements["abstract"]["inhibited_strand"] # "negative"
contradiction = (abstract_inhibited != body_inhibited)
reversed_direction = (abstract_inhibited == body_allowed)
def main():
print("Source: PMC6176606, doi:10.1073/pnas.1805195115")
print("Error type: abstract-vs-body contradiction (reversed strand direction)")
print("-" * 68)
print("Fig. 5F positive strand : G459L == GDD(dead) -> positive synthesis INHIBITED")
print("Fig. 5G negative strand : G459L > GDD(dead) -> negative synthesis ALLOWED (n=2)")
print()
print(f"BODY: strand inhibited = {body_inhibited!r}")
print(f"BODY: strand allowed = {body_allowed!r}")
print(f"ABSTRACT: strand stated inhibited = {abstract_inhibited!r}")
print()
print(f"Abstract contradicts body? {contradiction}")
print(f"Direction reversed (abstract names ALLOWED strand as {reversed_direction}")
print(f" the inhibited one)?")
print("-" * 68)
# Table mirrored in the write-up
rows = [
("Strand G459L inhibits", "Abstract",
"negative-strand", "positive-strand"),
("Positive-strand RNA, G459L vs GDD-dead", "Results / Fig. 5F",
"= GDD => positive INHIBITED", "positive INHIBITED (consistent)"),
("Negative-strand RNA, G459L vs GDD-dead (n=2)", "Results / Fig. 5G",
"> GDD => negative ALLOWED", "negative ALLOWED, not inhibited"),
("Strand inhibited (stated)", "Results conclusion",
"positive-strand", "positive-strand"),
("Strand inhibited (stated)", "Discussion",
"positive-strand", "positive-strand"),
]
w = [44, 20, 30, 32]
hdr = ("quantity", "location", "reported", "recomputed")
print(" | ".join(h.ljust(wi) for h, wi in zip(hdr, w)))
print("-+-".join("-" * wi for wi in w))
for r in rows:
print(" | ".join(str(c).ljust(wi) for c, wi in zip(r, w)))
print("-" * 68)
# Innocent explanations checked
checks = {
"Different experiment/population?":
"No - same strand-specific RT-qPCR, same G459L cotransfection.",
"Different time/normalization/units?":
"No - one assay (26 h, normalized to WT-alone); 5F & 5G are the two strands.",
"Inhibiting + strand loosely lowers - strand?":
"No - 5G shows negative SIGNIFICANTLY GREATER than GDD; text says ALLOWED.",
"Logical framing rescues abstract?":
"No - Results pose 'inhibit negative (hence both) OR just positive' and answer 'just positive'.",
"One-sided/adjusted test / stated convention?":
"N/A - reversed qualitative direction, not a statistical threshold.",
"Correction/erratum?":
"No Crossref/PMC/EuropePMC notice; no PubPeer thread.",
}
for q, a in checks.items():
print(f"- {q} {a}")
print("-" * 68)
verdict = "CONFIRMED" if (contradiction and reversed_direction) else "NOT CONFIRMED"
print(f"VERDICT: claim {verdict}")
make_figure()
# ----------------------------------------------------------------------
# Figure: two text panels (Abstract vs Results/Discussion) joined by an
# annotated connector naming the conflict.
# ----------------------------------------------------------------------
def make_figure():
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
BG = "#fcfcfb"
INK = "#0b0b0b"
SUB = "#52514e"
ORANGE = "#eb6834" # the paper's reported value (abstract)
BLUE = "#2a78d6" # value recomputed from the paper's own numbers
GRID = "#d8d7d4"
here = os.path.dirname(os.path.abspath(__file__))
fig, ax = plt.subplots(figsize=(7.2, 4.5))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
# Title
ax.text(50, 96, "Abstract names the allowed strand as the inhibited one",
ha="center", va="top", fontsize=13, fontweight="bold", color=INK)
ax.text(50, 90.5,
"PMC6176606 - strand-specific RT-qPCR (Fig. 5 F and G)",
ha="center", va="top", fontsize=10, color=SUB)
# ---- Left panel: Abstract (reported, orange) ----
lx, lw = 2.5, 38
ly, lh = 28, 50
ax.add_patch(FancyBboxPatch((lx, ly), lw, lh,
boxstyle="round,pad=0.6,rounding_size=2.2",
linewidth=2, edgecolor=ORANGE, facecolor="#fdece4"))
ax.text(lx + lw / 2, ly + lh - 4, "ABSTRACT", ha="center", va="top",
fontsize=11, fontweight="bold", color=ORANGE)
ax.text(lx + lw / 2, ly + lh - 11,
"\"...dominantly inhibited\n"
"NEGATIVE-strand RNA\n"
"synthesis of wild-type virus\n"
"present in the same cell\"",
ha="center", va="top", fontsize=9.5, color=INK, style="italic")
ax.text(lx + lw / 2, ly + 8,
"Reads: NEGATIVE-strand\nsynthesis is inhibited",
ha="center", va="center", fontsize=10, color=ORANGE,
fontweight="bold")
# ---- Right panel: Results / Discussion (recomputed, blue) ----
rx, rw = 59.5, 38
ax.add_patch(FancyBboxPatch((rx, ly), rw, lh,
boxstyle="round,pad=0.6,rounding_size=2.2",
linewidth=2, edgecolor=BLUE, facecolor="#e7f0fb"))
ax.text(rx + rw / 2, ly + lh - 4, "RESULTS / DISCUSSION",
ha="center", va="top", fontsize=11, fontweight="bold", color=BLUE)
ax.text(rx + rw / 2, ly + lh - 11,
"Fig. 5F positive = GDD-dead\n"
" -> positive INHIBITED\n"
"Fig. 5G negative > GDD-dead (n=2)\n"
" -> negative ALLOWED",
ha="center", va="top", fontsize=9, color=INK)
ax.text(rx + rw / 2, ly + 8,
"States: POSITIVE-strand\nsynthesis is inhibited",
ha="center", va="center", fontsize=10, color=BLUE,
fontweight="bold")
# ---- Connector naming the conflict (in the central gap only) ----
gap_l, gap_r = lx + lw, rx # 40.5 -> 59.5
cy = ly + lh / 2
con = FancyArrowPatch((gap_l + 0.5, cy), (gap_r - 0.5, cy),
arrowstyle="<|-|>", mutation_scale=14,
linewidth=2, color=INK,
connectionstyle="arc3,rad=0.0")
ax.add_patch(con)
mid = (gap_l + gap_r) / 2
ax.text(mid, cy + 6, "DIRECTION", ha="center", va="bottom",
fontsize=9.5, fontweight="bold", color=INK)
ax.text(mid, cy + 2.2, "REVERSED", ha="center", va="bottom",
fontsize=9.5, fontweight="bold", color=INK)
ax.text(mid, cy - 4, "same assay,", ha="center", va="top",
fontsize=8.5, color=SUB)
ax.text(mid, cy - 7.5, "opposite strand", ha="center", va="top",
fontsize=8.5, color=SUB)
# ---- Footer legend (color never the sole signal) ----
ax.plot([8, 12], [12, 12], color=ORANGE, linewidth=2)
ax.text(13, 12, "as reported in the paper (abstract)",
ha="left", va="center", fontsize=9.5, color=SUB)
ax.plot([8, 12], [6, 6], color=BLUE, linewidth=2)
ax.text(13, 6, "from the paper's own numbers (Results / Discussion)",
ha="left", va="center", fontsize=9.5, color=SUB)
# faint recessive grid baseline (single horizontal)
for gy in (22,):
ax.axhline(gy, color=GRID, linewidth=0.8, zorder=0)
fig.tight_layout()
svg = os.path.join(here, "fig1.svg")
png = os.path.join(here, "fig1.png")
fig.savefig(svg, facecolor=BG)
fig.savefig(png, dpi=200, facecolor=BG)
plt.close(fig)
print(f"wrote {svg}")
print(f"wrote {png}")
if __name__ == "__main__":
main()
Significance statement says tracer chromosome goes "exclusively" with X; Results report 28 of 42 cells
Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1809583115 · PMC6310853
Summary
The paper's Significance statement says that male Ts65Dn translocation mice carrying a maternally derived supernumerary chromosome place this "tracer" chromosome "exclusively" in the haploid set that contains the X chromosome. The only experiment behind that sentence is in Results (Fig. 4 H–L). It reports the tracer in the X-containing hemisphere in "the majority of cases": n = 28/42 cells, P = 0.022. That is 66.7% (exact 95% CI 50.5% to 80.4%), with 14 of 42 cells (33.3%) not counted on the X side. The reported P reproduces as a one-sided binomial test against 0.5 (0.0218; two-sided 0.0436). The paper's own Fig. 4L also shows some tracer markers below the chromosome 16 midpoint, which the legend defines as the Y-containing hemisphere. The Significance statement thus describes as all-or-none a modest majority that only just reaches significance, in the experiment that the Results present as the direct test of the parental-origin model. The paper's main finding, that homologs are kept apart in mitosis, rests on other experiments and is not affected.
What the paper reports
Male translocation mice with a maternal-derived supernumerary chromosome display the tracer chromosome exclusively to the haploid set containing the X chromosome.
— Abstract, Significance statement
Using 3D chromosome imaging of a translocation mouse with a supernumerary chromosome, we found that this maternally derived chromosome is positioned by parental origin. These data, taken together, support the identity of haploid sets by parental origin.
— Abstract (main)
In the majority of cases we examined, the hemisphere containing the translocated chromosome 1716 also contained the maternally derived X chromosome (n = 28/42 cells, P = 0.022; Fig. 4 H–L and Movie S7).
— Results › In Vitro and in Vivo Segregation of Mouse Chromosomes, paragraph 2 (the plain-text archive renders 1716 as "1716")
To identify which X-, or Y-containing haploid set was most likely associated with the translocation chromosome, we first determined the approximate boundary between the haploid sets by finding the midpoint of the pair of homologous chromosome 16. We then mapped the position of the translocated chromosome (Fig. 4L).
— Same paragraph, immediately before
A maternally derived supernumerary chromosome segregates with the X chromosome.
— Fig. 4 legend, title sentence
The X- or Y-containing hemisphere domains are defined as being above/below the plus sign (+), respectively.
— Fig. 4 legend, panel L (n = 42)
Movie S7. The translocated chromosome in the Ts65Dn MEFs was consistently detected in the same hemisphere as the maternal-derived X chromosome.
— SI Appendix, movie legends (the Results cite Movie S7 alongside n = 28/42)
Center of mass calculations were used for hemisphere localization; however, chromosomes that were positioned with the majority of their staining along the boundary of the hemispheres, due to the limits of resolution, were scored to support the null hypothesis, which goes against our model, in the calculation of the one-tailed P value which gives rise to an underestimation of statistical significance.
— Materials and Methods › Statistical Analysis
Although our data support the paradigm (13, 14, 47) that haploid sets are derived by parental origin, there are limitations to our analysis.
— Discussion, paragraph 3
The same paragraph adds that in the reciprocal (paternal) cross the tracer "localized to the boundary between two nuclear hemispheres, making it difficult to determine its association to either X- or Y-containing hemispheres (SI Appendix, Fig. S5 C–I)."
What the paper's own numbers show
Method. The Methods state that hemisphere assignments were tested with a binomial distribution against p = 0.5, using a one-tailed P value. I recomputed every count that the paper reports with a P value, as one-sided P(X ≥ x | n, 0.5), as two-sided exact P, and as the single-term binomial probability b(x; n, 0.5) that the Methods formula writes out. For the tracer result I computed the exact (Clopper–Pearson) 95% CI for the proportion. As a separate, approximate check, I read the positions of the tracer circles and the chromosome 16 midpoint markers ("+") off the published Fig. 4 image (757 × 449 px) by pixel colour. Each circle was classed as above, within ±3 axis units of, or below its local midpoint. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Tracer 1716 in the maternal-X hemisphere | Significance statement | "exclusively", i.e. 42/42 (100%) | — |
| Same result | Results, mouse section para 2; Fig. 4 H–L | n = 28/42 cells, P = 0.022, "the majority of cases" | 66.7%; exact 95% CI 50.5% to 80.4% (excludes 100%); one-sided P = 0.0218, two-sided P = 0.0436 |
| Cells not counted in the X hemisphere | Same | — | 14/42 (33.3%) |
| Margin over the significance threshold | Same | — | smallest count out of 42 with one-sided P < 0.05 is 27; the reported 28 is one cell above it |
| Compatibility with a near-exclusive effect | Same | — | if the true share were 90%, P(≤ 28 of 42) = 3.5 × 10⁻⁵; if 95%, 8.5 × 10⁻⁹ |
| Tracer circles vs midpoint, read from the image (approximate) | Fig. 4L (n = 42) | no count given in the legend | 41 of 42 circles resolved: 35 above (X side), 2 within ±3 units, 4 below (Y side), all 4 among the nuclei at the right of the plot |
| XY one per hemisphere, WT male MEFs | Results, mouse section para 1; Fig. 4 E, F | 28/42, P = 0.022 | one-sided P = 0.0218 (match) |
| Chr 16 homologs in separate hemispheres, Ts65Dn MEFs | Results, mouse section para 2; SI Fig. S5 A, B | 28/42, P = 0.022 | one-sided P = 0.0218 (match) |
| Chr 17 homologs, WT MEF anaphase | Results, mouse section para 1 | 28/36, P = 0.001 | one-sided 0.0006 (match) |
| Chr 4, HUVEC metaphase / anaphase | Results, Homologous and XY section para 1 | 15/20, P = 0.021; 28/38, P = 0.003 | 0.0207; 0.0025 (match) |
| XX, female HUVEC metaphase / anaphase | Same section, para 2 | 18/24, P = 0.011; 32/42, P = 0.001 | 0.0113; 0.0005 (match) |
| XY, male HUVEC metaphase | Same section, para 2 | 19/28, P = 0.044 | 0.0436 (match) |
| XY, male HUVEC anaphase (side finding) | Same section, para 2; Fig. 2 D–F | 20/28, P = 0.001 | one-sided 0.0178, two-sided 0.0357, b(x) 0.0116; none gives 0.001 (that would need about 23/28) |
Two things follow. First, every reported P except one reproduces as a one-sided binomial tail probability, so the 28/42 count is the paper's own figure for this result, not a typo. Second, a count of 28/42 is a majority, not an exclusive pattern, and the CI rules out anything near 100%.
The image read-out is resolution-limited and is not meant as a recount. It suggests more circles above the midpoint than the 28 counted in the text. One possible reason is the Methods' rule that boundary chromosomes were scored against the model. I do not treat this difference as a second error. The read-out matters here only because it shows tracer markers on the Y side of the boundary in the paper's own figure.
Figure
Figure 1. The same experiment described in two places. Left: the Significance statement, whose word "exclusively" implies the tracer is in the X-containing set in all cells (orange, the paper's wording). Right: the Results sentence that reports the experiment, n = 28/42 cells, which is 66.7% (blue, recomputed from the paper's own count). The connector marks that both sentences describe the same 42 cells. Below: the 14 cells not counted on the X side, and the exact 95% CI, which excludes 100%.
Figure 2. Share of the 42 male Ts65Dn MEF cells with the maternal-origin tracer chromosome in the X-containing hemisphere, on one 0–100% axis. Orange: the share implied by "exclusively" in the Significance statement (100%). Blue: the share reported in Results, 28/42 = 66.7%, with its exact 95% CI (50.5% to 80.4%), computed in recompute.py.txt. The dashed line marks the chance value of 50% used as the null in the paper's binomial test. The lower CI limit lies just above that line, and the upper limit lies well below 100%.
Why it matters
- It is the summary sentence for the paper's only direct test of parental origin. The Results open this experiment with "To directly test whether each haploid set is derived by parental origin". The main abstract then draws the conclusion that the data "support the identity of haploid sets by parental origin". The Significance statement is the version most readers see, and it gives an all-or-none result.
- The size of the effect is overstated. The data show a tracer on the X side in two of three cells. The evidence against chance is modest: one-sided P = 0.022, two-sided P = 0.044, and one cell fewer on the X side (27/42, P = 0.044) would still be significant, but 26/42 (P = 0.082) would not. The CI lower limit (50.5%) sits just above the null.
- The figure title and the movie legend repeat the categorical form. The Fig. 4 legend title ("segregates with the X chromosome") gives no qualifier, and the SI legend for Movie S7 says "consistently detected in the same hemisphere as the maternal-derived X chromosome". Meanwhile the panel the Fig. 4 title describes (Fig. 4L) shows tracer markers below the midpoint.
- The supporting evidence is otherwise limited. The authors' Discussion notes that the reciprocal paternal cross could not assign the tracer to either hemisphere. The Results qualify the interpretation ("If the 1716 chromosome behaves similarly to other wild-type autosomes"). Neither qualification reaches the Significance statement.
- Repeated count. The identical "n = 28/42 cells, P = 0.022" is given for three different measurements in the mouse section: XY separation in WT MEFs, chromosome 16 homolog separation in Ts65Dn MEFs, and tracer-with-X co-localization. The n = 42 given for the XY and tracer measurements matches the Fig. 4 legend, and identical counts can occur by chance. The per-cell data are not in the paper or the SI Appendix, so the three counts cannot be checked independently.
- What does not change. The direction of the tracer result (more often with the X than chance), the antipairing of homologs, and the haploid-set separation in human and mouse cells all rest on the other counts above, which reproduce. One exception: the XY HUVEC anaphase P (reported 0.001, recomputed 0.018 one-sided) is misreported but is still below 0.05.
Explanations considered
- "Exclusively" means the tracer is in one set rather than split across both. The sentence names the set ("the haploid set containing the X chromosome"), so it is a claim about which set, and the Results give that as 28/42.
- Loose wording in a lay summary. The main abstract avoids the word, saying only that the chromosome "is positioned by parental origin". But the Significance statement is part of the published abstract, and the word has a clear meaning that the reported count does not support. Whatever the reason for the wording, it does not change what the numbers show.
- The 14 remaining cells may include boundary cells rather than Y-side cells. The Methods say that boundary chromosomes were "scored to support the null hypothesis". So some of the 14 may be ambiguous rather than clearly on the Y side. This could explain why the image read-out shows more than 28 circles above the midpoint. It does not rescue "exclusively": the paper's own count is 28/42, and Fig. 4L, read by the legend's definition, has about 4 tracer markers below the midpoint, that is, on the Y side.
- A different population or experiment behind the Significance sentence. The Significance statement refers to "translocation mice with a maternal-derived supernumerary chromosome" (male). That is the Ts65Dn maternal × WT paternal cross in Fig. 4 G–L (n = 42). The in vivo embryo panels (Fig. 4 M–P) are painted for chromosomes 16 and 17 and report no tracer-with-X count. The Results cite Movie S7 for the same n = 28/42. No other tracer count appears in the paper.
- A conversion or typesetting artefact. The wording is the same in the publisher's XML archived in PMC, in Europe PMC's full text and on the live PMC page. The companion "PNAS Plus Significance Statements" record (PMC6310825) repeats the clause with "exclusively". The disputed numbers (28/42, 0.022) are plain text, with no superscripts or table cells involved. The only conversion artefact is the chromosome label 1716, which the plain-text archive renders as "1716".
- A typo in the count. The count 28/42 and its P value are mutually consistent (one-sided 0.0218), and the text describes it as "the majority of cases", so the body is internally coherent. The discrepancy lies between the body and the Significance statement.
None of these accounts for the gap between "exclusively" and 28 of 42 cells.
Status
As of 2026-10-23:
- Version of record. The publisher pages (pnas.org, reached through the DOI redirect, and the PDF and SI URLs) returned HTTP 403 to automated access. I therefore checked the publisher-deposited text: the JATS XML archived in PubMed Central (PMC6310853, read with
src/show.py), the Europe PMC full-text XML (title and DOI checked; 115(52):E12235–E12244, PMID 30530674) and the live PMC article page. Every quote from the article above was checked verbatim against the PMC-archived text, and all three sources carry the same wording and numbers. The companion "PNAS Plus Significance Statements" record (PMC6310825) was read on the live PMC page; its Europe PMC full text returned HTTP 500. - Supplements. The SI Appendix (pnas.1809583115.sapp.pdf, from the Europe PMC supplementary-files archive; its md5, 629446949a72863dfbeb0748dcfc62b9, matches the PMC listing) was read in full as text. It contains Figs. S1–S5 and legends for Movies S1–S7, but no per-cell table and no separate count for the tracer-with-X assignment. Fig. S5 A–B covers the maternal cross (n = 42) and Fig. S5 C–I the reciprocal paternal cross (n = 34). The Movie S7 legend is quoted above. The movies were not viewed; Movie S7 was not in the archive.
- Figure image. Fig. 4 was read from the published image file (link below) for the approximate Fig. 4L read-out. No part of it is reproduced here.
- Corrections and notices.
src/known.pyreturned no Crossref update relations and no PMC or Europe PMC notices. A Europe PMC search for the DOI returns only the article and the companion record. - PubPeer and the wider web. The PubPeer API and search page both returned HTTP 403, so PubPeer could not be queried directly. Web searches made earlier in this verification, for the title together with correction, erratum or PubPeer, and for PubPeer together with the DOI, found no correction and no PubPeer thread. They could not be repeated for this write-up because the session's web-search quota was used up. A PubPeer thread therefore cannot be fully ruled out.
- Sources. https://pmc.ncbi.nlm.nih.gov/articles/PMC6310853/ ; https://pmc.ncbi.nlm.nih.gov/articles/PMC6310825/ ; Europe PMC REST (search, full text and supplementary files for PMC6310853); https://api.crossref.org/works/10.1073/pnas.1809583115 ; published Fig. 4 image, https://cdn.ncbi.nlm.nih.gov/pmc/blobs/c3f5/6310853/c1c7e28b198f/pnas.1809583115fig04.jpg .
Suggested correction
In the Significance statement, replace "display the tracer chromosome exclusively to the haploid set containing the X chromosome" with wording that matches the Results, for example: "display the tracer chromosome more often in the haploid set containing the X chromosome (28 of 42 cells, 67%; one-sided binomial P = 0.022)". Correspondingly, the Fig. 4 legend title could read "A maternally derived supernumerary chromosome segregates preferentially with the X chromosome", and "consistently" in the Movie S7 legend could be replaced by "in most cells (28/42)". The authors may also wish to:
- state the boundary-scoring rule next to the 28/42 count, and give the numbers of cells on the X side, on the Y side and on the boundary in Fig. 4L;
- confirm the three separate "n = 28/42 cells, P = 0.022" counts in the mouse section against the source data; and
- correct the XY HUVEC anaphase P value (20/28 gives one-sided P = 0.018, not 0.001).
Recomputation script (python recompute.py.txt)
"""Recompute the numbers behind the parental-origin (Ts65Dn tracer) result in PMC6310853
(Proc Natl Acad Sci U S A 2018;115(52):E12235-E12244, doi:10.1073/pnas.1809583115),
and regenerate fig1/fig2 in this folder.
Run: python recompute.py.txt [path/to/pnas.1809583115fig04.jpg]
Needs numpy, scipy, matplotlib; Pillow only for the optional Fig. 4L read-out (part 5).
If no image path is given, part 5 downloads the published Fig. 4 image from the PMC CDN
into a temporary directory that is deleted afterwards; if that fails, part 5 is skipped and
the figures are still produced. The image is only read, never copied into this folder.
"""
import os
import shutil
import sys
import tempfile
import urllib.request
from pathlib import Path
import numpy as np
from scipy.stats import binom, binomtest
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = Path(__file__).resolve().parent
# ------------------------------------------------------------------ inputs (paper)
# (label, x, n, reported P, location)
ROWS = [
("Chr 4, HUVEC metaphase", 15, 20, "0.021", "Results, Homologous and XY ..., para 1"),
("Chr 4, HUVEC anaphase", 28, 38, "0.003", "Results, Homologous and XY ..., para 1"),
("XX, female HUVEC metaphase", 18, 24, "0.011", "Results, Homologous and XY ..., para 2"),
("XX, female HUVEC anaphase", 32, 42, "0.001", "Results, Homologous and XY ..., para 2"),
("XY, male HUVEC metaphase", 19, 28, "0.044", "Results, Homologous and XY ..., para 2"),
("XY, male HUVEC anaphase", 20, 28, "0.001", "Results, Homologous and XY ..., para 2"),
("Chr 17, WT MEF anaphase", 28, 36, "0.001", "Results, Mouse section, para 1"),
("XY, WT male MEF", 28, 42, "0.022", "Results, Mouse section, para 1"),
("Chr 16 homologs apart, Ts65Dn MEF", 28, 42, "0.022", "Results, Mouse section, para 2"),
("Tracer 17(16) with maternal X, Ts65Dn MEF", 28, 42, "0.022", "Results, Mouse section, para 2"),
]
X_KEY, N_KEY = 28, 42
print("=" * 78)
print("1. Binomial tests against p = 0.5 for every count reported with a P value")
print(" Methods: binomial null, 'one-tailed P value'.")
print(f" {'comparison':42s} {'x/n':>7s} {'%':>6s} {'P1(>=x)':>9s} {'P2':>8s} {'b(x)':>8s} {'reported':>8s}")
results = {}
for lab, x, n, rep, loc in ROWS:
p1 = binom.sf(x - 1, n, 0.5)
p2 = binomtest(x, n, 0.5).pvalue
pmf = binom.pmf(x, n, 0.5)
results[lab] = (p1, p2, pmf)
print(f" {lab:42s} {x:>3d}/{n:<3d} {100*x/n:6.1f} {p1:9.4f} {p2:8.4f} {pmf:8.4f} {rep:>8s}")
print("\n2. The parental-origin result (tracer chromosome 17(16) in the maternal-X hemisphere)")
frac = X_KEY / N_KEY
ci = binomtest(X_KEY, N_KEY, 0.5).proportion_ci(confidence_level=0.95, method="exact")
p1, p2, _ = results["Tracer 17(16) with maternal X, Ts65Dn MEF"]
print(f" reported: n = {X_KEY}/{N_KEY} cells, P = 0.022")
print(f" fraction in X hemisphere = {X_KEY}/{N_KEY} = {frac:.4f} ({100*frac:.1f}%)")
print(f" cells not counted as X hemisphere = {N_KEY - X_KEY}/{N_KEY} ({100*(N_KEY - X_KEY)/N_KEY:.1f}%)")
print(f" exact (Clopper-Pearson) 95% CI for the fraction = {ci.low:.3f} to {ci.high:.3f} "
f"({100*ci.low:.1f}% to {100*ci.high:.1f}%)")
print(f" one-sided binomial P(X >= {X_KEY} | n = {N_KEY}, p = 0.5) = {p1:.4f}; two-sided = {p2:.4f}")
print(f" 'exclusively' corresponds to {N_KEY}/{N_KEY} = 100%; 100% lies outside the CI "
f"(upper limit {100*ci.high:.1f}%)")
for ptrue in (0.90, 0.95):
print(f" if the true fraction were {ptrue:.2f}: P(X <= {X_KEY} | n = {N_KEY}) = {binom.cdf(X_KEY, N_KEY, ptrue):.1e}")
xmin = int(min(k for k in range(N_KEY + 1) if binom.sf(k - 1, N_KEY, 0.5) < 0.05))
print(f" smallest count out of {N_KEY} with one-sided P < 0.05: {xmin} "
f"(P = {binom.sf(xmin - 1, N_KEY, 0.5):.4f}); the reported 28 is {X_KEY - xmin} above it")
print(f" one-sided P for 26/42 = {binom.sf(25, N_KEY, 0.5):.4f} (not < 0.05)")
print("\n3. Identical count reported for three different measurements")
for lab in ("XY, WT male MEF", "Chr 16 homologs apart, Ts65Dn MEF", "Tracer 17(16) with maternal X, Ts65Dn MEF"):
print(f" {lab:42s} 28/42, P = 0.022 reported; one-sided P recomputed {results[lab][0]:.4f}")
print("\n4. Side finding: XY male HUVEC anaphase")
p1, p2, pmf = results["XY, male HUVEC anaphase"]
print(f" reported 20/28, P = 0.001; recomputed one-sided {p1:.4f}, two-sided {p2:.4f}, pmf {pmf:.4f}")
for x in range(20, 29):
if binom.sf(x - 1, 28, 0.5) < 0.0015:
print(f" smallest x out of 28 whose one-sided P rounds to <= 0.001: {x} (P = {binom.sf(x-1, 28, 0.5):.4f})")
break
# ------------------------------------------------------------------ part 5: Fig. 4L read-out
FIG_URL = ("https://cdn.ncbi.nlm.nih.gov/pmc/blobs/c3f5/6310853/c1c7e28b198f/"
"pnas.1809583115fig04.jpg")
TMPDIR = None
def get_fig4():
global TMPDIR
if len(sys.argv) > 1 and os.path.exists(sys.argv[1]):
return sys.argv[1]
try:
d = TMPDIR = tempfile.mkdtemp(prefix="pmc6310853_")
path = os.path.join(d, "fig4.jpg")
req = urllib.request.Request(FIG_URL, headers={"User-Agent": "Mozilla/5.0"})
with urllib.request.urlopen(req, timeout=60) as r, open(path, "wb") as f:
f.write(r.read())
return path
except Exception as e: # network not available
print(f" (could not download Fig. 4: {e})")
return None
print("\n5. Fig. 4L read from the published image (757 x 449 px; tracer circles vs chr16 midpoint '+')")
fig4 = get_fig4()
if fig4 is None:
print(" skipped")
else:
from PIL import Image
from scipy import ndimage as ndi
im = np.asarray(Image.open(fig4).convert("RGB")).astype(int)
if im.shape[:2] != (449, 757):
print(f" unexpected image size {im.shape[:2]}; skipped")
else:
y0, x0 = 336, 64 # inside the panel L frame (frame rows 334-424, cols 62-182)
sub = im[y0:423, x0:181]
r, g, b = sub[..., 0], sub[..., 1], sub[..., 2]
TOP, BOT = 334, 424 # frame lines = +50 / -50 on the y axis
pct = lambda y: 50 - 100 * (y - TOP) / (BOT - TOP)
yel = (r > 170) & (g > 170) & (b < 160)
circ = ndi.binary_dilation(yel, iterations=3)
gry = (abs(r - g) < 25) & (abs(g - b) < 25) & (r > 100) & (r < 225) & ~circ
gry[:15, :] = False # drop the 'n=42' label
ys, xs = np.nonzero(gry)
p = np.polyfit(xs + x0, ys + y0, 1)
for _ in range(3):
keep = abs(ys + y0 - np.polyval(p, xs + x0)) < 3
p = np.polyfit(xs[keep] + x0, ys[keep] + y0, 1)
def mid(X):
m = keep & (abs(xs + x0 - X) <= 5)
if m.sum() < 3:
return pct(np.polyval(p, X))
return pct(np.median(ys[m] + y0))
lab, nb = ndi.label(yel)
cs = ndi.center_of_mass(yel, lab, range(1, nb + 1))
sz = ndi.sum(yel, lab, range(1, nb + 1))
cnt = {"above": 0, "near": 0, "below": 0}
tot = 0
for (cy, cx), s in sorted(zip(cs, sz), key=lambda t: t[0][1]):
if s < 4:
continue
k = 2 if s > 20 else 1 # two overlapping circles merged into one blob
tot += k
d = pct(cy + y0) - mid(cx + x0)
t = "above" if d >= 3 else ("below" if d <= -3 else "near")
cnt[t] += k
if t != "above":
print(f" x = {cx + x0:5.1f} px: tracer {pct(cy + y0):5.1f}, midpoint {mid(cx + x0):5.1f}, "
f"difference {d:5.1f} -> {t}")
print(f" circles resolved: {tot} of 42; above midpoint (X side) {cnt['above']}, "
f"within +/-3 units {cnt['near']}, below (Y side) {cnt['below']}")
print(" (approximate: low resolution, overlapping circles; boundary threshold 3 axis units)")
if TMPDIR:
shutil.rmtree(TMPDIR, ignore_errors=True)
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(HERE / f"{name}.svg")
fig.savefig(HERE / f"{name}.png", dpi=200)
plt.close(fig)
print(f" wrote {name}.svg / {name}.png")
print("\n6. Figures")
# ---- fig1: Significance statement vs Results (abstract-vs-body contradiction)
fig = plt.figure(figsize=(7.2, 4.1))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.1); ax.axis("off")
def panel(x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.15, y + h - 0.18, title, fontsize=10.5, fontweight="bold", color=INK, va="top")
ax.text(x + 0.15, y + h - 0.55, body, fontsize=10, color=INK, va="top", linespacing=1.45)
panel(0.12, 1.35, 2.85, 2.55, "Significance statement",
"“Male translocation mice with a\nmaternal-derived supernumerary\n"
"chromosome display the tracer\nchromosome exclusively to the\n"
"haploid set containing the X\nchromosome.”", INK2)
panel(4.23, 1.35, 2.85, 2.55, "Results, mouse section, para 2",
"“In the majority of cases we\nexamined, the hemisphere\ncontaining "
"the translocated\nchromosome 17¹⁶ also contained\nthe maternally derived X\n"
"chromosome (n = 28/42 cells,\nP = 0.022 ...)”", INK2)
ax.text(1.545, 1.55, "implies 42/42 = 100%", ha="center", fontsize=10.5, color=ORANGE, fontweight="bold")
ax.text(5.655, 1.55, f"{X_KEY}/{N_KEY} = {100*frac:.1f}%", ha="center", fontsize=10.5, color=BLUE, fontweight="bold")
arr = FancyArrowPatch((3.0, 2.62), (4.2, 2.62), arrowstyle="<->", mutation_scale=14, lw=2, color=INK2)
ax.add_patch(arr)
ax.text(3.6, 2.80, "same 42\ncells", ha="center", va="bottom", fontsize=10, color=INK2)
ax.text(3.6, 2.45, "“exclusively”\nvs\n“majority”", ha="center", va="top", fontsize=10, color=INK)
ax.text(3.6, 1.02, "Both sentences describe one experiment: Fig. 4 H–L, male Ts65Dn MEFs, n = 42.",
ha="center", fontsize=10, color=INK2)
ax.text(3.6, 0.62,
f"{N_KEY - X_KEY} of {N_KEY} cells ({100*(N_KEY - X_KEY)/N_KEY:.1f}%) not counted on the X side; "
f"exact 95% CI {100*ci.low:.1f}–{100*ci.high:.1f}% excludes 100%",
ha="center", fontsize=10, color=INK)
ax.text(3.6, 0.25, "orange = paper's wording (Significance); blue = recomputed from the paper's own count",
ha="center", fontsize=9.5, color=INK2, style="italic")
save(fig, "fig1")
# ---- fig2: proportion on one axis, claim vs data with exact CI, null reference
fig, ax = plt.subplots(figsize=(7.2, 2.9))
fig.subplots_adjust(left=0.30, right=0.97, top=0.86, bottom=0.22)
ax.axvline(50, color=INK2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(49, 0.45, "chance (p = 0.5)", ha="right", va="bottom", fontsize=9.5, color=INK2)
ax.plot([100], [2], "o", ms=10, color=ORANGE, zorder=3)
ax.annotate("100%", (100, 2), xytext=(-10, 0), textcoords="offset points", ha="right", va="center",
fontsize=10, color=INK)
ax.plot([100 * ci.low, 100 * ci.high], [1, 1], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
for v in (ci.low, ci.high):
ax.plot([100 * v, 100 * v], [0.88, 1.12], color=BLUE, lw=2, zorder=2)
ax.plot([100 * frac], [1], "o", ms=10, color=BLUE, zorder=3)
ax.annotate(f"{100*frac:.1f}% (95% CI {100*ci.low:.1f}–{100*ci.high:.1f}%)", (52, 1),
xytext=(0, -14), textcoords="offset points", ha="left", va="top", fontsize=10, color=INK)
ax.set_yticks([2, 1])
ax.set_yticklabels(["Significance statement:\n“exclusively” (paper)",
"Results: 28/42 cells\n(recomputed, exact CI)"], fontsize=10, color=INK)
ax.set_ylim(0.35, 2.6)
ax.set_xlim(0, 105)
ax.set_xticks([0, 25, 50, 75, 100])
ax.set_xticklabels(["0%", "25%", "50%", "75%", "100%"])
ax.set_xlabel("Cells with the tracer chromosome in the X-containing hemisphere", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_title("Share of 42 male Ts65Dn MEF cells, maternal-origin tracer (Fig. 4 H–L)",
fontsize=10.5, color=INK, loc="left", x=-0.39)
save(fig, "fig2")
Figure 3 legend calls a transduction P value of 0.096 statistically significant
Proc Natl Acad Sci U S A (2022) · doi:10.1073/pnas.1910844119 · PMC8812682
Summary
The Figure 3 legend reports a Friedman/exact test of CAR transduction across three activation conditions (CD3, CD3/CD28, CD3/CD28/CD81) and states that the data "reject the null hypothesis, indicating a statistically significant difference in transduction (P = 0.096)". At the conventional 0.05 level, which the paper's parallel Figure 2 statement also uses (P = 0.038, called significant), a P value of 0.096 does not reject the null hypothesis. No other significance level, one-sided test or adjustment is stated anywhere in the article. This P value is the only inferential statistic the article gives for any transduction comparison, and transduction is the outcome named in the title. The observed direction of the effect is unaffected, but its stated statistical significance is not supported by the number printed.
What the paper reports
Figure 3 legend ("CD81 costimulation improves CAR transduction efficiency"):
These data are representative of three experiments, with transduction efficiency across experiments illustrated. For CAR transduction, we employed the nonparametric Friedman test and exact test for repeated-measures experiments. Our null hypothesis (H0) is that there is no difference in median transduction between groups. Our data reject the null hypothesis, indicating a statistically significant difference in transduction (P = 0.096) for cells treated with additional CD81 costimulation.
Figure 2 legend ("CD81 costimulation enhances activation of isolated naive T cells"), the parallel statement for activation:
These data are representative of four experiments with average CD69 expression prior to CAR transduction across experiments illustrated. For activation, we employed the nonparametric Friedman test and exact test for repeated-measures experiments. Our null hypothesis (H0) is that there is no difference in median activation between groups. Our data reject the null hypothesis, indicating a statistically significant difference in activation (P = 0.038) for cells treated with additional CD81 costimulation.
Results, "CD81 Costimulation Enhances CAR Transduction in Naive T Cells":
In specific, activating naive T cells with additional CD81 costimulation achieved greater transduction efficiency as compared with CD3 or CD3/CD28 activation conditions (Fig. 3A and SI Appendix, Fig. S2A).
Abstract (Significance statement):
We show that CD81 costimulation of naive T cells prior to CAR transduction can lead to enhanced CAR expression in this T cell subset.
Discussion, second paragraph:
Importantly, the T cell product derived from these CD81-costimulated naive T cells achieved greater transduction efficiency, with greater levels of CD19 CAR expression.
The article has no statistics subsection in its Methods and states no significance level. The only P values in the article are the two above and the three in the Figure 1C legend (CD4 "<0.0001", CD8 "<0.0002", naive "<0.0001"; seven experiments; no test named).
What the paper's own numbers show
Method. I compared each reported P value with the conventional significance level of 0.05, checked whether rounding, halving (a one-sided reading) or a 0.10 level would change the comparison, and confirmed that the parallel Figure 2 statement follows the same template with a P value below 0.05. To see whether a P value below 0.05 was reachable with the stated design, I enumerated the exact null distribution of the Friedman statistic for three conditions and three, four and seven experiments (all (3!)^n equally likely within-experiment rankings, assuming no ties). The per-experiment transduction values are not in the article, so the test itself cannot be recomputed. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Transduction P value | Fig. 3 legend | P = 0.096, "reject the null hypothesis", "statistically significant" | 0.096 > 0.05, so H0 is not rejected at 0.05; P is 1.92 times the threshold |
| Unrounded range of the transduction P value | Fig. 3 legend | 0.096 | any value in [0.0955, 0.0965) rounds to 0.096; all are above 0.05 |
| One-sided reading | derived | not stated | 0.096 / 2 = 0.048 (< 0.05), but the legend prints 0.096 as the P it calls significant |
| 0.10 level | derived | not stated | 0.096 ≤ 0.10, but no 0.10 level is stated anywhere in the article |
| Activation P value | Fig. 2 legend | P = 0.038, "statistically significant" | 0.038 < 0.05, so H0 is rejected at 0.05; consistent |
| Smallest attainable exact Friedman P, 3 experiments × 3 conditions | Fig. 3 design | not stated | 0.0278 (6/216), reached only when all three experiments rank the conditions identically; the next value is 0.1944 |
| Is 0.096 an attainable exact P for 3 × 3? | Fig. 3 design | 0.096 | no; the nearest chi-square-approximation P is 0.0970 (Q = 4.667), whose exact P is 0.1944 |
| Smallest attainable exact Friedman P, 4 experiments × 3 conditions | Fig. 2 design | not stated | 0.0046; 0.038 is not attainable exactly (nearest: exact 0.0417, chi-square 0.0388 at Q = 6.5) |
| Smallest attainable exact Friedman P, 7 experiments × 3 conditions | Fig. 1C design | "<0.0001", "<0.0002", "<0.0001" | 0.000021, so values below 0.0001 are attainable; consistent if the same test was used |
The central finding needs no reconstruction: the legend's own P value is above 0.05, and the legend's own conclusion is that the null hypothesis is rejected. The attainability rows are a secondary observation. Neither 0.096 nor 0.038 matches the exact Friedman distribution for the simplest reading of the stated designs, which suggests that the printed values come from a different computation (for example the chi-square approximation, ties, or a different number of blocks) or contain a transcription slip. The article does not give enough detail to tell which.
Figure
P values on a log scale, with the 0.05 level as a vertical reference line. Filled orange points are the P values printed in the Figure 2 (activation, four experiments) and Figure 3 (transduction, three experiments) legends; both legends describe their result as statistically significant. Open blue points are the exact Friedman P values that three conditions and the stated number of experiments can produce (no ties), computed by recompute.py.txt. The Figure 2 value falls to the left of the line, where H0 is rejected. The Figure 3 value falls to the right of it, where H0 is not rejected. For the Figure 3 design an exact P below 0.05 (0.028) was attainable, and 0.096 itself is not one of the attainable values.
Why it matters
The article's title states that CD81 costimulation "skews CAR transduction toward naive T cells", and the Significance statement and Discussion describe enhanced transduction and CAR expression. The Figure 3 legend contains the only inferential test the article reports for any transduction comparison: the transduction results in Figures 4 and 5 carry no P values. The legend presents that test as a rejection of the null hypothesis. Read at the conventional 0.05 level, the test does not reject it. With three experiments the result is better described as a non-significant difference in the stated direction. The observed direction and size of the transduction difference are unchanged, and the activation results (Fig. 1C; Fig. 2, P = 0.038) are not affected. The paper calls itself a proof of principle, and its qualitative message survives. What does not survive is the claim that the transduction difference in Figure 3 is statistically significant. A reader or a meta-analyst taking the legend at its word would count as significant a result that is not significant by the paper's own figure.
Explanations considered
- Rounding. Any unrounded value that prints as 0.096 lies between 0.0955 and 0.0965, all above 0.05. Rounding does not account for it.
- A one-sided test. Halving gives 0.048, which is below 0.05. But the legend prints 0.096 as the value it calls significant, and it states a non-directional null ("no difference in median transduction between groups"). The Friedman statistic is an omnibus test across three groups whose P is already an upper-tail probability, so it has no natural one-sided version. A one-sided convention would have been reported as 0.048. This does not account for the sentence as written.
- A significance level of 0.10. P = 0.096 would meet a 0.10 level. No such level is stated anywhere in the article, and the parallel Figure 2 sentence uses the same wording for P = 0.038. A 0.10 level would make the sentence consistent only if it had been declared, and it was not.
- Multiplicity adjustment. Adjustment can only raise a P value, so an adjusted or unadjusted reading cannot bring 0.096 below 0.05.
- A different quantity or population. A P value has no units. The 0.038 in Figure 2 belongs to a separate test of activation in four experiments, not to an alternative reading of the transduction result.
- Significance at 0.05 unreachable with three experiments. This is ruled out by enumeration. With three conditions and three experiments, the exact Friedman test can produce P = 0.028 when every experiment orders the conditions the same way, so a significant result was possible with this design.
- A misprinted P value. This cannot be excluded. Because 0.096 is not an attainable exact P for three experiments and three conditions, the printed value may not be the value the stated test produced. If the experiments all ranked the conditions identically, the exact P would be 0.028 and the significance statement would be correct with a misprinted number. If they did not, the exact P would be 0.194 or larger, and the result would be further from significance. The per-experiment values (cited as SI Appendix Fig. S2A) are not reproduced in the retrievable text, so this cannot be resolved from the article. Either way, the published sentence pairs a conclusion with a number that does not support it, and only the authors can say which of the two is right.
Status
- Version of record. As of 2026-10-23, the sentence appears verbatim in the Figure 3 legend of the PubMed Central version (PMC8812682), in the Europe PMC full-text XML, and in the local PMC archive used here. The value 0.096 occurs only in the Figure 3 legend. The legend is published as text, not as part of an image, and all three copies agree, so a conversion artifact is not plausible.
- Supplement. The SI Appendix is cited for the transduction data (Fig. S2A). Per-experiment transduction measurements or ranks are not available in the retrievable text. They would settle the misprint question above, but they would not change the inconsistency between the printed P value and the printed conclusion.
- Corrections and comments. The Crossref record for the DOI has no update-to, updated-by or relation entries. PMC and Europe PMC list no correction, erratum, expression of concern or retraction, and the PMC page has no notice. Searches of PubPeer and Retraction Watch found no entry.
- Sources checked. The PMC article page and archive (
src/show.py PMC8812682),src/known.py 10.1073/pnas.1910844119(Crossref, PMC and Europe PMC notices: none), the Crossref works API, the Europe PMC full-text XML and core search, PubPeer and Retraction Watch.
Suggested correction
Either correct the conclusion or correct the number, whichever the underlying data support:
- If P = 0.096 is the correct result of the stated test, the Figure 3 legend could read: "Our data do not reject the null hypothesis at the 0.05 level (P = 0.096); the difference in transduction with additional CD81 costimulation was not statistically significant." Any statement elsewhere implying a statistically demonstrated transduction advantage in Figure 3 should be worded descriptively to match.
- If 0.096 is a misprint, the corrected P value should be given, together with the number of experiments (blocks) and conditions entering the test, whether the exact or the chi-square P is reported, and the per-experiment transduction values (or ranks) so that readers can verify it.
In either case, stating the significance level used for all tests (presumably 0.05, as in Figure 2) would remove the ambiguity.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC8812682 and regenerate fig1.svg / fig1.png.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Inputs are only numbers printed in the article:
Fig. 3 legend: transduction, Friedman test and exact test, "three experiments", P = 0.096
Fig. 2 legend: activation, Friedman test and exact test, "four experiments", P = 0.038
Fig. 1C legend: activation, "seven experiments", P < 0.0001 / < 0.0002 / < 0.0001
Three activation conditions throughout: CD3, CD3/CD28, CD3/CD28/CD81 (k = 3).
"""
import itertools
import os
from collections import Counter
from fractions import Fraction
from scipy.stats import chi2
HERE = os.path.dirname(os.path.abspath(__file__))
ALPHA = 0.05
P_FIG3 = 0.096 # transduction, called "statistically significant"
P_FIG2 = 0.038 # activation, called "statistically significant"
K = 3 # conditions: CD3, CD3/CD28, CD3/CD28/CD81
def friedman_exact(n, k):
"""Exact null distribution of the Friedman statistic Q (no ties).
Returns list of (Q, exact upper-tail P, chi-square approximation P), Q ascending.
Every within-block ranking is equally likely under H0: (k!)^n configurations.
"""
perms = list(itertools.permutations(range(1, k + 1)))
sums = Counter({tuple([0] * k): 1})
for _ in range(n):
nxt = Counter()
for s, m in sums.items():
for p in perms:
nxt[tuple(a + b for a, b in zip(s, p))] += m
sums = nxt
total = len(perms) ** n
qc = Counter()
for s, m in sums.items():
q = Fraction(12, n * k * (k + 1)) * sum(r * r for r in s) - 3 * n * (k + 1)
qc[q] += m
qs = sorted(qc)
out = []
for q in qs:
tail = sum(qc[x] for x in qs if x >= q)
out.append((float(q), tail / total, float(chi2.sf(float(q), k - 1))))
return out, total
print("=== 1. The significance statement in the Fig. 3 legend ===")
print(f"Reported P (Fig. 3, transduction) = {P_FIG3}")
print(f"Conventional alpha = {ALPHA}")
print(f"P <= alpha ? = {P_FIG3 <= ALPHA} -> H0 not rejected at 0.05")
print(f"P / alpha = {P_FIG3 / ALPHA:.2f}")
print(f"P - alpha = {P_FIG3 - ALPHA:.3f}")
lo, hi = P_FIG3 - 0.0005, P_FIG3 + 0.0005
print(f"Range of unrounded P that rounds to 0.096 = [{lo:.4f}, {hi:.4f}); lower end > 0.05: {lo > ALPHA}")
print(f"Half of reported P (one-sided reading) = {P_FIG3 / 2:.3f} (< 0.05: {P_FIG3 / 2 < ALPHA})")
print(f"Reported P <= 0.10 ? = {P_FIG3 <= 0.10}")
print()
print("=== 2. The parallel statement in the Fig. 2 legend ===")
print(f"Reported P (Fig. 2, activation) = {P_FIG2}")
print(f"P <= alpha ? = {P_FIG2 <= ALPHA} -> H0 rejected at 0.05 (consistent)")
print()
print("=== 3. Exact Friedman P values attainable with k = 3 conditions (no ties) ===")
designs = {}
for n, note in [(3, "Fig. 3: three experiments"), (4, "Fig. 2: four experiments"),
(7, "Fig. 1C: seven experiments")]:
dist, total = friedman_exact(n, K)
designs[n] = dist
smallest = min(p for _, p, _ in dist)
print(f"n = {n} blocks x k = {K} ({note}); {total} equally likely rank configurations")
print(f" smallest attainable exact P = {smallest:.6f} (< 0.05: {smallest < ALPHA})")
if n <= 4:
print(" attainable values (Q: exact P | chi-square approx P):")
for q, p, a in sorted(dist, key=lambda t: -t[0]):
print(f" Q = {q:6.3f}: exact {p:.4f} | chi2 {a:.4f}")
else:
print(f" smallest attainable exact P < 0.0001: {smallest < 0.0001}")
print()
d3 = designs[3]
print("Fig. 3 design (n = 3): is 0.096 an attainable exact P?",
any(abs(p - P_FIG3) < 0.0005 for _, p, _ in d3))
q_near = min(d3, key=lambda t: abs(t[2] - P_FIG3))
print(f" nearest chi-square-approximation P: Q = {q_near[0]:.3f}, chi2 P = {q_near[2]:.4f}, "
f"exact P for the same Q = {q_near[1]:.4f}")
d4 = designs[4]
print("Fig. 2 design (n = 4): is 0.038 an attainable exact P?",
any(abs(p - P_FIG2) < 0.0005 for _, p, _ in d4))
q_near4 = min(d4, key=lambda t: abs(t[2] - P_FIG2))
print(f" nearest chi-square-approximation P: Q = {q_near4[0]:.3f}, chi2 P = {q_near4[2]:.4f}, "
f"exact P for the same Q = {q_near4[1]:.4f}")
print()
# ---------------------------------------------------------------- figure
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e3e2de"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"svg.fonttype": "none"})
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xscale("log")
XMIN, XMAX = 0.003, 1.0
ax.set_xlim(XMIN, XMAX)
rows = [ # (y, label)
(3.0, "Fig. 2 (activation)\nreported P"),
(2.3, "Fig. 2 design\nattainable exact P"),
(1.0, "Fig. 3 (transduction)\nreported P"),
(0.3, "Fig. 3 design\nattainable exact P"),
]
ax.set_ylim(-0.25, 3.55)
ax.set_yticks([y for y, _ in rows])
ax.set_yticklabels([l for _, l in rows], fontsize=10)
# alpha reference line and regions
ax.axvline(ALPHA, color=INK2, lw=1, zorder=1)
ax.text(ALPHA * 0.92, -0.2, "\u2190 P \u2264 \u03b1 = 0.05: reject H\u2080", ha="right", va="bottom",
color=INK2, fontsize=10)
ax.text(ALPHA * 1.08, -0.2, "P > 0.05: do not reject H\u2080 \u2192", ha="left", va="bottom",
color=INK2, fontsize=10)
# attainable exact P values (blue, recomputed); label the ones that bracket 0.05
labels = {2.3: {0.0046: -1, 0.0417: -1, 0.0694: 1}, 0.3: {0.0278: 1, 0.1944: 1}}
for y, dist in [(2.3, designs[4]), (0.3, designs[3])]:
ps = sorted({round(p, 4) for _, p, _ in dist if p >= XMIN})
ax.plot(ps, [y] * len(ps), ls="none", marker="o", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=3)
for p, side in labels[y].items():
ax.annotate(f"{p:.3f}", (p, y), xytext=(0, 11 * side), textcoords="offset points",
ha="center", va="bottom" if side > 0 else "top", color=INK2, fontsize=10,
bbox=dict(facecolor=BG, edgecolor="none", pad=1.0), zorder=5)
# reported P values (orange, from the paper)
for y, p, verdict in [(3.0, P_FIG2, "described as significant"),
(1.0, P_FIG3, "described as significant")]:
ax.plot([p], [y], ls="none", marker="o", ms=10, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.annotate(f"P = {p:.3f}, {verdict}", (p, y), xytext=(0, 11),
textcoords="offset points", ha="center", va="bottom", color=INK, fontsize=10,
bbox=dict(facecolor=BG, edgecolor="none", pad=1.0), zorder=5)
ax.set_xticks([0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1.0])
ax.set_xticklabels(["0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_xlabel("P value (log scale)", color=INK)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.tick_params(colors=INK2, axis="x")
handles = [
Line2D([], [], ls="none", marker="o", ms=10, color=ORANGE, label="Reported in the paper"),
Line2D([], [], ls="none", marker="o", ms=8, mfc=BG, mec=BLUE, mew=2,
label="Recomputed: exact Friedman P values the stated design can produce"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.4, -0.2), ncol=1,
frameon=False, fontsize=10, labelcolor=INK)
fig.tight_layout()
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
out = os.path.join(HERE, f"fig1.{ext}")
fig.savefig(out, facecolor=BG, bbox_inches="tight", **kw)
print("wrote", out)
Chi-square P = 0.0379 for casein B-cell responders does not follow from 19/45 vs 10/35
Proc Natl Acad Sci U S A (2022) · doi:10.1073/pnas.2117034119 · PMC8916005
Summary
The paper reports that more patients with multiple sclerosis (MS) than patients with other neurologic diseases (OND) had casein-specific B cells in an ELISPOT assay: 19/45 (42%) vs 10/35 (28%), "P = 0.0379; χ2 test". A chi-square test on those counts gives χ² = 1.59, P = 0.208. Fisher's exact test gives P = 0.246, and none of the eight other standard two-group tests tried comes below P = 0.10. The reported P is reproduced exactly (P = 0.03794) only when the printed percentages are entered as if they were counts out of 100 patients per group. This comparison is the only between-group support the paper gives for the B-cell half of the abstract's statement that patients with MS had "significantly higher B cell and antibody reactivity to bovine casein". The mouse experiments and the separate serum IgG ELISA comparison are not affected.
What the paper reports
The result itself:
"While the number of casein-specific B cell spots/patient was similar in the two patient groups, the number of patients positive for casein-specific B cells was higher in the MS (19/45; 42%) than in the OND group (10/35; 28%) (P = 0.0379; χ2 test)." (Results, "Patients with MS Harbor Bovine Casein-Specific Antibodies and B Cells", paragraph 3)
The group sizes are stated in the same paragraph, in the Methods and in Table 2:
"we tested the CNS- and casein-specific B cell response in patients with MS (n = 45) and those with ONDs (n = 35)" (Results, same paragraph)
"PBMCs from 45 patients with MS and 35 patients with ONDs were tested by ELISPOT assay." (Materials and Methods, "Patient Sample ELISPOT")
"ELISPOT | 35 | 45" and "ELISPOT casein responders, %† | 28 | 42" (Table 2, columns "Patients with ONDs" | "Patients with MS")
The claims that rest on this comparison:
"The results obtained from the mouse model were complemented by clinical data showing that serum samples from patients with MS contained significantly higher B cell and antibody reactivity to bovine casein than those from patients with other neurologic diseases." (Abstract)
"Patients with MS show elevated numbers of B cell responses and IgG titers to bovine casein in the blood." (Fig. 4 title)
The Statistical Analysis section lists the tests used and the significance level; it does not mention a chi-square test:
"differences between two parametric groups were assessed using an unpaired t test or paired t test while ANOVA was used for greater than two groups. For nonparametric datasets, a Mann–Whitney U test was used. For both parametric and nonparametric datasets, a significance level of 5% was chosen." (Materials and Methods, "Statistical Analysis")
What the paper's own numbers show
Method. The stated counts give the 2×2 table MS 19 positive / 26 negative and OND 10 positive / 25 negative (80 patients). recompute.py.txt runs the usual two-group tests for a difference in proportions on this table with scipy: Pearson chi-square with and without Yates' continuity correction, its one-sided version, the likelihood-ratio G-test, the N−1 chi-square, and Fisher's, Barnard's and Boschloo's exact tests. It then searches every 2×2 table with row totals 45 and 35 for one whose chi-square P (uncorrected or Yates) equals 0.0379. Finally it tries tests that are not two-group comparisons and a chi-square on the printed percentages entered as counts.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| MS responders | Results para 3; Table 2 | 19/45; 42% | 42.22% |
| OND responders | Results para 3; Table 2 | 10/35; 28% | 28.57% (29% with conventional rounding) |
| Difference in responder proportion | derived | 42% vs 28% | 13.7 percentage points; Newcombe 95% CI −7.5 to 32.6 |
| Pearson χ², no correction | Results para 3 | P = 0.0379 (χ2 test) | χ² = 1.588, df 1, P = 0.2077 |
| Pearson χ², Yates correction | Results para 3 | P = 0.0379 | χ² = 1.052, P = 0.3051 |
| Pearson χ², one-sided | Results para 3 | P = 0.0379 | P = 0.1038 |
| G-test / N−1 χ² | Results para 3 | P = 0.0379 | P = 0.2050 / 0.2105 |
| Fisher exact, two-sided / one-sided | Results para 3 | P = 0.0379 | P = 0.2463 / 0.1525 |
| Barnard / Boschloo exact, two-sided | Results para 3 | P = 0.0379 | P = 0.2326 / 0.2175 |
| χ² needed for P = 0.0379 (df 1) | derived | — | 4.31; observed 1.59 |
| Any table with row totals 45 and 35 giving P = 0.0379 | derived | — | none (closest: Yates P = 0.0373 for 17/45 vs 5/35, and 0.0384 for 23/45 vs 9/35) |
| Percentages entered as counts, 42/100 vs 28/100, Pearson χ² | derived | P = 0.0379 | χ² = 4.308, P = 0.037940 (exact match) |
| Same, 42/100 vs 29/100 (conventional rounding) | derived | — | P = 0.0547 |
| Same, with Yates correction | derived | — | P = 0.0539 |
| MS rate against a fixed rate of 10/35 or 0.28 (goodness of fit; binomial) | derived | — | P = 0.043 or 0.034; binomial P = 0.048 or 0.045 |
| ELISA OD > 1.0 rates, 16/39 vs 5/23 | Table 2 (no P given) | 41.02% vs 21.7% | χ² P = 0.1211; Fisher P = 0.1673 |
| Group sizes needed at the same proportions for P < 0.05 | derived | — | about 3 times larger (135/105: P = 0.029; 90/70: P = 0.075) |
The expected numbers of responders under no group difference are 16.3 (MS) and 12.7 (OND), against 19 and 10 observed. The four cell contributions to Pearson's statistic sum to 1.59, well short of the 4.31 needed for P = 0.0379. The 2×2 table is fully determined by the text and Table 2, and the table's row totals equal the totals in the Methods, so there is no unstated count that could be varied.
Figure
Figure 1. P values for the comparison of casein-specific B-cell responders, 19/45 (MS) vs 10/35 (OND), on one log-scaled axis. The orange point is the value reported in the Results (P = 0.0379, "χ2 test"). The filled blue points are nine standard two-group tests recomputed from the stated counts. All lie between 0.10 and 0.31, on the far side of the dashed line at the paper's stated 5% significance level. The open blue point in the shaded row is a Pearson chi-square run on the printed percentages as if each group had 100 patients (42/100 vs 28/100). It gives P = 0.0379, the reported value, to four decimals. All values come from recompute.py.txt.
Why it matters
The comparison changes from significant (as reported) to not significant (as recomputed) at the 5% level the paper sets. The recomputed 95% CI for the difference in responder proportions runs from −7.5 to +32.6 percentage points and includes zero. The same paragraph says that "the number of casein-specific B cell spots/patient was similar in the two patient groups", so the paper's ELISPOT data provide no between-group difference in B-cell reactivity that is statistically significant.
This affects the following statements:
- Abstract. "significantly higher B cell and antibody reactivity to bovine casein than those from patients with other neurologic diseases". The antibody half rests on the serum IgG ELISA (39 MS vs 23 OND, "P = 0.032; Mann–Whitney U test"), which this check does not touch. The B-cell half rests on the chi-square comparison alone.
- Fig. 4 title. "Patients with MS show elevated numbers of B cell responses". Fig. 4 itself shows the ELISA (panels A and B) and an MS-only correlation (panel C); the B-cell group contrast appears only in the text and Table 2.
- Table 2. The "ELISPOT casein responders, %" row (28 vs 42) is presented as a group contrast, without a P value.
- Translational conclusions. The paragraph closes with "these findings corroborated the link between casein- and MAG-specific antibodies and MS", and the Discussion calls for "personalized dietary restrictions". Without a significant B-cell difference, the human evidence that patients with MS differ from OND controls rests on the single ELISA comparison.
The direction of the observed difference is unchanged (42% vs 29%), and the data are compatible with a modest difference. They are equally compatible with none. At the observed proportions, about three times as many patients per group would have been needed for P < 0.05.
The main mouse findings do not depend on this test: casein immunization with demyelination and IgG deposition, MAG cross-reactivity shown with antibodies and adsorption, and complement dependence. Neither do the ELISA comparison (P = 0.032) or the MS-only correlation between CNS- and casein-specific spot counts (r = 0.605).
Explanations considered
- Continuity correction, or a one-sided test. Yates' correction raises P to 0.305. A one-sided Pearson test gives 0.104 and a one-sided Fisher test 0.153. None is near 0.0379.
- An exact or likelihood-based test labelled "χ2". The G-test, N−1 chi-square and Fisher's, Barnard's and Boschloo's tests give P = 0.205 to 0.246.
- Different or misprinted counts. The denominators 45 and 35 appear in the Results, the Methods and Table 2. The percentages match the counts (19/45 = 42.2%; 10/35 = 28.6%, printed as 28), so a misprint in the counts would also have to appear in the percentages. Across all 2×2 tables with row totals 45 and 35, no table gives P = 0.0379 to four decimals with or without Yates' correction. The nearest values (0.0373, 0.0384) need responder counts far from 19 and 10, such as 17/45 vs 5/35 with Yates' correction.
- The P value belongs to a different comparison. The other proportion contrast in Table 2, ELISA OD > 1.0 (41.02% of 39 vs 21.7% of 23, i.e. 16 vs 5), gives P = 0.121 (Fisher 0.167). The ELISA titer comparison has its own reported P (0.032), and the spot counts per patient are described as "similar". None of these accounts for 0.0379.
- A one-sample test against the OND rate. Comparing the MS count with a fixed rate of 10/35 or 0.28 gives P = 0.034 to 0.048 (goodness of fit or binomial), never 0.0379. Such a test also treats the OND estimate as known without error, so it is not a valid two-group comparison.
- Percentages entered as counts. A Pearson chi-square without correction on 42/100 vs 28/100 gives χ² = 4.308, P = 0.037940, which matches the reported value to all four printed decimals. This also fits the printed 28% (with 29%, P = 0.0547). The paper does not describe how the test was run, so this is a numerical match, not a documented procedure. Even so, the calculation treats 80 patients as 200 and is not a valid analysis of the stated data. It explains where the number could come from; it does not make the number correct.
- Text-conversion artifact. The counts and P value are plain text with ordinary decimal points. They read identically in the PMC archive and the Europe PMC full-text XML, with no minus signs, superscripts or table cells involved.
Status
As of 2026-10-23:
- Version of record. The publisher's page (pnas.org, reached through doi.org) returned HTTP 403 to automated access and could not be read. The archived full text in PubMed Central (PMC8916005) and the Europe PMC full-text XML carry the identical sentence, counts and P value, and Table 2 matches.
- Supplement. The SI Appendix (pnas.2117034119.sapp.pdf, from the Europe PMC supplementary-files archive) was converted to text and searched. Tables S3 and S4, which the paragraph cites, list patient characteristics (diagnosis, sex, age, disease duration, EDSS, dairy intake) but not per-patient ELISPOT results or responder status. The SI contains no "ELISPOT", "chi" or "χ". The data availability statement points to the article and SI; no source-data file was found. The supplement does not resolve the discrepancy.
- Corrections. None found. Crossref update and relation metadata are empty, and a Crossref
updates:filter on the DOI returns 0 items. Crossref carries Retraction Watch entries, and none appear. The PMC and Europe PMC notice lists are empty, Europe PMC has no comment/correction list, and PubMed (PMID 35235454) has no CommentsCorrections entries. - PubPeer. PubPeer's site and API refused automated access (HTTP 403 and 429), so it could not be searched directly. An earlier web search found no PubPeer thread for the article. A PubPeer comment therefore cannot be fully ruled out.
Sources: PMC full text via src/show.py PMC8916005; src/known.py 10.1073/pnas.2117034119; api.crossref.org; Europe PMC REST (search, fullTextXML, supplementaryFiles); PubMed E-utilities.
Suggested correction
In Results, "Patients with MS Harbor Bovine Casein-Specific Antibodies and B Cells", paragraph 3, the sentence could read: "the proportion of patients positive for casein-specific B cells was 42% (19/45) in the MS group and 29% (10/35) in the OND group; this difference was not statistically significant (P = 0.21, χ2 test; Fisher's exact test P = 0.25)." Reporting the difference with its 95% CI (13.7 percentage points, −7.5 to 32.6) would show the uncertainty.
Consequential changes:
- Abstract. Limit "significantly higher" to the antibody (ELISA) result, e.g. "...contained significantly higher antibody reactivity to bovine casein..., and a numerically, but not significantly, higher proportion had casein-specific B cells."
- Fig. 4 title. Remove "elevated numbers of B cell responses" or qualify it.
- Table 2. Optionally add counts to the "ELISPOT casein responders" row (10/35, 29%; 19/45, 42%).
- Statistical Analysis. Add the chi-square (or Fisher) test, since the section currently lists only t tests, ANOVA, Mann–Whitney and Pearson correlation.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC8916005 and regenerate fig1.
Run: python recompute.py.txt (writes fig1.svg and fig1.png next to this file)
Inputs come from the article as archived in PubMed Central (PMC8916005):
Results, "Patients with MS Harbor Bovine Casein-Specific Antibodies and B Cells",
paragraph 3: "the number of patients positive for casein-specific B cells was higher
in the MS (19/45; 42%) than in the OND group (10/35; 28%) (P = 0.0379; chi2 test)".
Table 2: ELISPOT n = 35 (OND), 45 (MS); "ELISPOT casein responders, %" 28 (OND), 42 (MS);
ELISA n = 23 (OND), 39 (MS); "ELISA OD value > 1.0, %" 21.7 (OND), 41.02 (MS).
Requires numpy, scipy (>= 1.7 for barnard_exact/boschloo_exact) and matplotlib.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
REPORTED_P = 0.0379
MS_POS, MS_N = 19, 45 # MS patients positive for casein-specific B cells
OND_POS, OND_N = 10, 35 # OND patients positive
REPORTED_PCT = (42, 28) # percentages as printed in the text and in Table 2
tab = np.array([[MS_POS, MS_N - MS_POS], [OND_POS, OND_N - OND_POS]])
print("== Inputs ==")
print("2x2 table [[pos, neg] MS; [pos, neg] OND]:", tab.tolist())
print(f"MS responders: {MS_POS}/{MS_N} = {100*MS_POS/MS_N:.2f}% (reported 42%)")
print(f"OND responders: {OND_POS}/{OND_N} = {100*OND_POS/OND_N:.2f}% (reported 28%; "
f"conventional rounding gives {round(100*OND_POS/OND_N)}%)")
diff = MS_POS/MS_N - OND_POS/OND_N
print(f"Difference in proportions: {100*diff:.2f} percentage points")
exp = stats.contingency.expected_freq(tab)
print("Expected counts under no difference:", np.round(exp, 2).tolist())
contrib = (tab - exp) ** 2 / exp
print("Pearson contributions:", np.round(contrib.ravel(), 3).tolist())
print("\n== Two-group tests on the stated counts 19/45 vs 10/35 ==")
res = {}
c, p, _, _ = stats.chi2_contingency(tab, correction=False)
res["Pearson chi2, no correction"] = (c, p)
c, p, _, _ = stats.chi2_contingency(tab, correction=True)
res["Pearson chi2, Yates correction"] = (c, p)
res["Pearson chi2, one-sided"] = (res["Pearson chi2, no correction"][0],
res["Pearson chi2, no correction"][1] / 2)
g, p, _, _ = stats.chi2_contingency(tab, correction=False, lambda_="log-likelihood")
res["G-test (likelihood ratio)"] = (g, p)
N = tab.sum()
cN1 = res["Pearson chi2, no correction"][0] * (N - 1) / N
res["N-1 chi2"] = (cN1, stats.chi2.sf(cN1, 1))
_, p = stats.fisher_exact(tab)
res["Fisher exact, two-sided"] = (np.nan, p)
_, p = stats.fisher_exact(tab, alternative="greater")
res["Fisher exact, one-sided"] = (np.nan, p)
r = stats.barnard_exact(tab)
res["Barnard exact, two-sided"] = (r.statistic, r.pvalue)
r = stats.boschloo_exact(tab)
res["Boschloo exact, two-sided"] = (r.statistic, r.pvalue)
for k, (s, p) in res.items():
s_txt = "" if np.isnan(s) else f"stat = {s:.4f}"
print(f" {k:32s} {s_txt:16s} P = {p:.4f}")
chi_needed = stats.chi2.isf(REPORTED_P, 1)
print(f"\nChi-square (df 1) needed for P = {REPORTED_P}: {chi_needed:.4f}; "
f"observed {res['Pearson chi2, no correction'][0]:.4f}")
# Newcombe (Wilson score) 95% CI for the difference in proportions
def wilson(x, n, z=1.959964):
ph = x / n
centre = (ph + z*z/(2*n)) / (1 + z*z/n)
half = z*np.sqrt(ph*(1-ph)/n + z*z/(4*n*n)) / (1 + z*z/n)
return centre - half, centre + half
l1, u1 = wilson(MS_POS, MS_N)
l2, u2 = wilson(OND_POS, OND_N)
p1, p2 = MS_POS/MS_N, OND_POS/OND_N
lo = diff - np.sqrt((p1-l1)**2 + (u2-p2)**2)
hi = diff + np.sqrt((u1-p1)**2 + (p2-l2)**2)
print(f"Newcombe 95% CI for the difference: {100*lo:.1f} to {100*hi:.1f} percentage points")
print("\n== Exhaustive search: all 2x2 tables with row totals 45 and 35 ==")
best = []
for a in range(MS_N + 1):
for b in range(OND_N + 1):
t = np.array([[a, MS_N - a], [b, OND_N - b]])
if (t.sum(0) == 0).any():
continue
for corr in (False, True):
_, p, _, _ = stats.chi2_contingency(t, correction=corr)
best.append((abs(p - REPORTED_P), a, b, corr, p))
best.sort()
print("Closest five (uncorrected or Yates) to P = 0.0379:")
for d, a, b, corr, p in best[:5]:
print(f" {a}/45 vs {b}/35, Yates={corr}: P = {p:.4f}")
n_match = sum(1 for d, *_ in best if d < 0.00005)
print(f"Tables giving P = 0.0379 to 4 decimals: {n_match}; "
f"among tables within 4 responders of 19 and 10: "
f"{sum(1 for d,a,b,c,p in best if d < 0.00005 and abs(a-19)<=4 and abs(b-10)<=4)}")
print("\n== Percentages entered as counts out of 100 ==")
pct_res = {}
for a, b in [(42, 28), (42.22, 28.57), (42, 29)]:
t = np.array([[a, 100 - a], [b, 100 - b]])
c, p, _, _ = stats.chi2_contingency(t, correction=False)
pct_res[(a, b)] = (c, p)
print(f" {a}/100 vs {b}/100, uncorrected: chi2 = {c:.4f}, P = {p:.6f}")
c, p, _, _ = stats.chi2_contingency(np.array([[42, 58], [28, 72]]), correction=True)
print(f" 42/100 vs 28/100, Yates: chi2 = {c:.4f}, P = {p:.6f}")
print("\n== Other tests that do not compare two groups ==")
for lab, p0 in [("MS vs fixed p0 = 10/35", 10/35), ("MS vs fixed p0 = 0.28", 0.28)]:
o = np.array([MS_POS, MS_N - MS_POS]); e = np.array([p0*MS_N, (1-p0)*MS_N])
c, p = stats.chisquare(o, e)
pb = stats.binomtest(MS_POS, MS_N, p0, alternative="two-sided").pvalue
print(f" {lab}: goodness-of-fit chi2 = {c:.4f}, P = {p:.4f}; binomial two-sided P = {pb:.4f}")
c, p = stats.chisquare([MS_POS, OND_POS])
print(f" 19 vs 10 responders, equal expected: chi2 = {c:.4f}, P = {p:.4f}")
print("\n== ELISA OD > 1.0 rates in Table 2 (no P value given there) ==")
ms_e, ond_e = 0.4102*39, 0.217*23
print(f" 41.02% of 39 = {ms_e:.2f}; 21.7% of 23 = {ond_e:.2f} -> 16/39 vs 5/23")
te = np.array([[16, 23], [5, 18]])
c, p, _, _ = stats.chi2_contingency(te, correction=False)
print(f" Pearson chi2 = {c:.4f}, P = {p:.4f}")
_, p = stats.fisher_exact(te)
print(f" Fisher exact two-sided P = {p:.4f}")
print("\n== Sample size needed at the same proportions (19/45 vs 10/35) ==")
for k in range(1, 5):
t = tab * k
c, p, _, _ = stats.chi2_contingency(t, correction=False)
print(f" x{k}: n = {MS_N*k}/{OND_N*k}, P = {p:.4f}")
# ------------------------------------------------------------------ figure 1
BG, TXT, SUB = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans",
"text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SUB, "ytick.color": TXT,
"svg.fonttype": "none"})
rows = [
("Reported in the paper (χ² test)", REPORTED_P, "rep"),
("Pearson χ², no correction", res["Pearson chi2, no correction"][1], "re"),
("Pearson χ², Yates correction", res["Pearson chi2, Yates correction"][1], "re"),
("Pearson χ², one-sided", res["Pearson chi2, one-sided"][1], "re"),
("G-test (likelihood ratio)", res["G-test (likelihood ratio)"][1], "re"),
("N−1 χ²", res["N-1 chi2"][1], "re"),
("Fisher exact, two-sided", res["Fisher exact, two-sided"][1], "re"),
("Fisher exact, one-sided", res["Fisher exact, one-sided"][1], "re"),
("Barnard exact, two-sided", res["Barnard exact, two-sided"][1], "re"),
("Boschloo exact, two-sided", res["Boschloo exact, two-sided"][1], "re"),
("Percentages entered as counts\n(42/100 vs 28/100), Pearson χ²", pct_res[(42, 28)][1], "pct"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.9), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1].astype(float)
ys[-1] -= 0.5 # separate the "percentages as counts" row
ys[1:-1] -= 0.25 # separate the reported row from the recomputed block
for (lab, p, kind), y in zip(rows, ys):
if kind == "rep":
ax.plot(p, y, "o", ms=10, color=ORANGE, zorder=3)
ax.annotate(f"P = {p:.4f}", (p, y), xytext=(-10, 0), textcoords="offset points",
va="center", ha="right", color=ORANGE, fontsize=10, fontweight="bold")
elif kind == "re":
ax.plot(p, y, "o", ms=9, color=BLUE, zorder=3)
ax.annotate(f"{p:.3f}", (p, y), xytext=(9, 0), textcoords="offset points",
va="center", color=BLUE, fontsize=10)
else:
ax.plot(p, y, "o", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=3)
ax.annotate(f"P = {p:.4f}", (p, y), xytext=(-10, 0), textcoords="offset points",
va="center", ha="right", color=BLUE, fontsize=10)
ax.text(0.058, y, "matches the reported value,\nbut assumes 100 patients per group",
va="center", ha="left", color=BLUE, fontsize=10)
ax.axvline(0.05, color=SUB, lw=1, ls="--", zorder=1)
ax.text(0.05, ys[0] + 0.75, "P = 0.05", color=SUB, fontsize=10, ha="center", va="bottom",
bbox=dict(facecolor=BG, edgecolor="none", pad=1.5), zorder=2)
ax.axhspan(ys[-1] - 0.5, ys[-1] + 0.6, color="#efeeea", zorder=0)
ax.set_xscale("log")
ax.set_xlim(0.01, 1)
ax.set_xticks([0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1])
ax.set_xticklabels(["0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_ylim(ys[-1] - 0.7, ys[0] + 1.3)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
for t, (_, _, kind) in zip(ax.get_yticklabels(), rows):
t.set_color(ORANGE if kind == "rep" else TXT)
ax.set_xlabel("P value (log scale)", fontsize=10)
ax.grid(axis="x", color="#e4e3df", lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SUB)
from matplotlib.lines import Line2D
handles = [Line2D([], [], marker="o", ls="", ms=10, color=ORANGE, label="Reported in the paper"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE,
label="Recomputed from the stated counts 19/45 vs 10/35"),
Line2D([], [], marker="o", ls="", ms=9, mfc=BG, mec=BLUE, mew=2,
label="Recomputed from the printed percentages as counts")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=1,
frameon=False, fontsize=10)
fig.text(0.01, 0.80, "MS vs OND casein-specific B-cell responders (Results, ELISPOT)",
fontsize=10, color=SUB)
fig.subplots_adjust(left=0.37, right=0.97, top=0.78, bottom=0.10)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg and fig1.png to", HERE)
Maternal 2-AG/1-AG rise: printed p = 0.005, yet its own t and CI imply p ≈ 0.07
Psychol Med (2023) · doi:10.1017/S0033291723001204 · PMC10719681
Summary
Table 3 and the Results text report that hair 2-AG/1-AG levels in mothers "significantly increased" from late pregnancy to one year postpartum, with t = 1.83 and p = 0.005. The same Table 3 row gives b = 10.52 and a 95% CI of −0.77 to 21.81. These three numbers agree with each other: the CI implies a standard error of 5.76 and z = 1.83. They imply a two-sided p of 0.067 to 0.068, and the interval includes zero. Under the paper's stated method (p values from z, two-tailed, α = 0.05), the Time effect is therefore not significant. This increase is the first finding in the abstract's Results and the basis of an argument in the Discussion. Other columns of Table 3 also do not fit together, so the table as printed cannot show which number is wrong. But the significance claim rests only on the p value of 0.005.
What the paper reports
Table 3, "Results of robust linear mixed effects models for endocannabinoid concentrations in mothers (Nt0 = 150; N t2 = 148)", 2-AG/1-AG block, Time row (columns: Predictor | b | 95% CI (b) | β | η2p (95% CI) | t | p):
Time | 10.52 | -0.77–21.81 | 0.26 | 0.01 (0.00–0.05) | 1.83 | 0.005**
Table 3 footer:
Note. * p < 0.050, ** p < 0.010, *** p < 0.001, two-tailed.
Results, "Alterations in eCB and NAE levels within the first year postpartum", Mothers paragraph:
2-AG/1-AG levels significantly increased in maternal hair over the first year postpartum (tTime = 1.83, p = 0.005, η2p = 0.01; Table 3), which did not depend on maternal CM load (tInteraction = -0.92, p = 0.357, η2p < 0.01).
Materials and methods, Statistical analyses:
P values of model predictors and post hoc tests were calculated based on z values.
All reported analyses were performed two-tailed with the significance level set at p < 0.05.
Abstract, Results:
Maternal hair levels of 2-AG/1-AG increased and SEA levels decreased from late pregnancy to one year postpartum.
Discussion, "eCB and NAE levels in maternal hair at the perinatal and postpartum", first and second paragraphs:
Independent of maternal CM, 2-AG/1-AG concentrations in maternal hair were lower in late pregnancy than one year postpartum, while SEA concentrations decreased from late pregnancy to one year postpartum.
Our study indicates lowered 2-AG/1-AG levels during late pregnancy which may contribute to reduced negative feedback in the HPA axis, and thus to an increase in tonic cortisol levels in the last pregnancy trimester. Increasing 2-AG/1-AG hair levels from pregnancy to one year postpartum might reflect the restoration of a tighter 2-AG-mediated HPA-axis regulation that returns glucocorticoid secretion back to pre-pregnancy levels.
Discussion, "Influence of maternal CM exposure on eCB and NAE levels", first paragraph:
While all women showed decreasing SEA and increasing 2-AG/1-AG levels from late pregnancy to one year later, women with a history of CM exhibited significantly lower SEA levels in late pregnancy as compared with women without CM.
What the paper's own numbers show
Method. The models are robust linear mixed models (robustlmm). The Methods state that p values were computed from z values, and the table footer says two-tailed. For a Wald-type 95% CI, SE = (upper − lower) / (2 × 1.96) and z = b / SE. I computed z from the printed b and CI, then computed the two-sided normal p from that z and from the printed t. As a check that does not depend on the normal reference, I also used a t distribution with 294 df (298 observations minus 4 fixed effects). For the effect size I used η²p = t² / (t² + df) with the same df. The paper does not state its η²p formula or df, but with df = 294 this formula reproduces the printed η²p of the intercept row (0.050 vs 0.05). I applied the same checks to every row of Table 3 and to the matching row of Supplementary Table S10, which repeats the model for the 63 mothers with data at both time points. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Midpoint of the 95% CI | Table 3, 2-AG/1-AG Time | b = 10.52 | (−0.77 + 21.81) / 2 = 10.520 |
| SE implied by the CI | Table 3 | not printed | 22.58 / 3.92 = 5.760 |
| Test statistic | Table 3; Results | t = 1.83 | z = 10.52 / 5.760 = 1.83 (1.826) |
| Two-sided p | Table 3; Results | 0.005** | 0.067 from t = 1.83 as z (Methods); 0.068 from b and CI; 0.068 with t(294) |
| p for any t that rounds to 1.83 (1.825 to 1.835) | — | — | 0.0665 to 0.0680 |
| One-sided p | footer says two-tailed | — | 0.034 |
| |z| needed for two-sided p = 0.005 | — | — | 2.81, which is 1.53 times the printed t |
| 95% CI that p = 0.005 would need around b = 10.52 | — | −0.77 to 21.81 | 3.17 to 17.87 (SE 3.75) |
| Does the 95% CI include 0? | Table 3 | −0.77 to 21.81 | yes, so p > 0.05 |
| η²p | Table 3; Results | 0.01 | 0.011 from t = 1.83 (df 294); |t| = 2.81 would give 0.026 |
| Same effect, complete cases (N = 63) | Suppl. Table S10 | b = 11.50, CI −5.87 to 28.88, t = 1.30, p = .004** | SE 8.865, z = 1.30, p = 0.19 (0.195 from CI); η²p 0.014 (df 122), consistent with the printed 0.01 |
| 2-AG/1-AG Intercept, CM, Time × CM (same block) | Table 3 | t 3.94, 1.58, −0.92; p < 0.001, 0.113, 0.357 | p from t: 0.0001, 0.114, 0.358 (consistent) |
The b, CI, t and η²p of the Time row all agree with one another, and the p value does not agree with any of them. The other three rows of the 2-AG/1-AG block give p values consistent with their t. A two-sided p of 0.005 would need a statistic about 1.5 times larger, or a CI about two-thirds as wide that excludes zero. Supplementary Table S10 shows the same pattern for the subsample: t = 1.30 is printed next to p = .004, while the t implies p = 0.19.
The rest of Table 3. The same table has further defects, listed here because they bear on which cell is wrong (see Explanations considered):
| Rows | What is printed | What the printed b and CI imply |
|---|---|---|
| SEA: Intercept, Time, CM, Time × CM | t = 5.18, 0.14, −0.47, −0.77 (the same four values as the OEA block) | z = 9.98, −3.81, −3.46, 1.52 |
| PEA: Intercept, Time, CM, Time × CM | t = 9.98, −3.81, −3.46, 1.50 (nearly the SEA rows' CI-implied z) | z = 5.84, 0.09, −0.96, −0.51 |
| SEA and PEA η²p | 0.25, 0.05, 0.04, 0.00 and 0.10, 0.00, 0.00, 0.00 | 0.253, 0.047, 0.039, 0.008 and 0.104, 0.000, 0.003, 0.001 from the CI-implied z (consistent) |
| p of OEA, SEA and PEA Intercept | 0.008, 0.054, 0.002 | all below 10-6 |
| p of OEA and PEA Time | 0.080, 0.250 | 0.89, 0.93 |
| p of SEA Time × CM ("marginally significant" in Results) | 0.054 | 0.13 |
| p of SEA Time and SEA CM | < 0.001, 0.001 | 0.0001, 0.0005 (consistent) |
So in Table 3 the b, CI and η²p columns agree with one another, the t column is misplaced in the SEA and PEA blocks, and seven of the sixteen p values do not match the b and CI printed in their own rows. For the 2-AG/1-AG Time row, the t value agrees with the b, the CI and η²p, so the error lies in the p value.
Figure
Figure 1. Two-sided p values for the maternal 2-AG/1-AG Time effect on a log scale. The orange circle is the p printed in Table 3 and the Results (0.005). The blue squares are recomputed from the same row: from the printed t = 1.83 used as z, as the Methods specify (0.067); from the printed b and 95% CI, SE = 5.76 (0.068); and from t = 1.83 with a t(294) reference (0.068). The dashed line is the paper's significance level, α = 0.05. All three recomputations agree, and all fall on the non-significant side of the line. The printed value is 13 times smaller. A log scale is used because p values span orders of magnitude.
Figure 2. The maternal 2-AG/1-AG Time coefficient b with its 95% CI, on a linear axis with the null value b = 0 as a vertical line. Orange: the values printed in Table 3 (b = 10.52, −0.77 to 21.81) and in Supplementary Table S10 for the 63 complete cases (b = 11.50, −5.87 to 28.88). Each is printed with a p below 0.01, yet each interval crosses zero. Blue: for comparison, the 95% CI that the same b = 10.52 would need for a two-sided p of 0.005 (3.17 to 17.87, |z| = 2.81). The interval printed in the paper is about 1.5 times that width.
Why it matters
- A result in the abstract is not supported by its own statistics. "Maternal hair levels of 2-AG/1-AG increased" is the first sentence of the abstract's Results. The coefficient, interval and test statistic behind it give p ≈ 0.07 under the paper's own α = 0.05 and two-tailed z test. The estimate is positive, but the paper does not show that the increase is statistically significant. The complete-case analysis that the footer cites as support ("Considering only these cases did not change the pattern of results") gives p ≈ 0.19 from its own t.
- A Discussion argument is built on it. The Discussion treats lower late-pregnancy 2-AG/1-AG as an established result. It proposes that this "may contribute to reduced negative feedback in the HPA axis" and that the rise to one year postpartum "might reflect the restoration of a tighter 2-AG-mediated HPA-axis regulation". The second Discussion section states that "all women showed ... increasing 2-AG/1-AG levels".
- The broader conclusion stands. The Conclusion says that "Mothers and children showed alterations in the eCB system from the last trimester of pregnancy to one year postpartum". It does not depend on this row. The maternal SEA decrease is consistent with its own CI (b = −926.03, −1402.21 to −449.84, z = −3.81, p = 0.0001). The infant 2-AG/1-AG Time effect in Table 4 stays significant on recomputation (z = −3.02, p = 0.0025, printed < 0.001).
- Table 3 cannot be relied on as printed. Seven of the sixteen p values and nine of the t values (the eight in the SEA and PEA blocks, plus 2-AG/1-AG Time × CM) do not match the b and CI in their rows. One example is the SEA Time × CM interaction, which the Results call "marginally significant" (p = 0.054). From its CI it has p = 0.13, which falls outside the paper's own trend band (p < .100).
Explanations considered
- The p value refers to a different test, such as a post hoc contrast or a likelihood-ratio test. Not supported. The Results cite "tTime = 1.83, p = 0.005" as one test of the Time predictor. The Methods say predictor p values were computed from z. Post hoc tests are listed separately in Table 3 and were run only for significant interactions, and the 2-AG/1-AG block has none.
- A t distribution with few degrees of freedom. Cannot explain it. Any t reference gives a larger p than the normal reference, not a smaller one: 0.068 with 294 df. No choice of df turns t = 1.83 into p = 0.005.
- One-sided testing. Cannot explain it. The footer and Methods say two-tailed, and a one-sided p would be 0.034, not 0.005.
- The t value or the CI is the wrong cell and p = 0.005 is correct. Not supported for this row. Three independent columns agree with t = 1.83: the CI-derived z (1.826), the CI midpoint (equal to b), and η²p (0.011 from t = 1.83, against 0.026 if |t| were 2.81). Elsewhere in Table 3, the t column is misplaced only in the SEA and PEA blocks. In the 2-AG/1-AG block, t matches the CI for the Intercept, Time and CM rows.
- Robust or non-Wald confidence intervals. Not able to explain it. A CI that is not a symmetric Wald interval could make the SE estimate inexact, but this interval is centred exactly on b. When the CI and the p value come from the same Wald z test, as the Methods describe, a 95% CI that includes zero cannot go with p < 0.05.
- A conversion or typesetting artifact. Ruled out. The publisher's PDF (Cambridge Core), the Cambridge HTML Results text, the Europe PMC JATS XML and the PMC HTML all carry the same values in the same columns. The Results sentence repeats the table's t and p. The supplement shows the same pattern (Table S10, t = 1.30 with p = .004).
- A simple typo in the p value, for example 0.065 written as 0.005. Possible, but the table cannot confirm it, and a typo would not rescue the claim. Any p consistent with the row is about 0.067 and is not significant. Whatever the cause, the significance statement in the text and abstract does not follow from the printed b, CI and t.
Side observations (not part of the main finding). In the 2-AG/1-AG Time × CM row, the CI (−0.45 to 0.16) is centred at −0.145, not at b = −0.09, and it implies z = −0.58 rather than the printed t = −0.92. Neither value approaches significance. For the children, the abstract says "In the children's hair, levels of 2-AG/1-AG increased while levels of SEA, OEA, and PEA decreased from late pregnancy to one year later". The Results say "Infants exhibited an inversed pattern of change as compared to their mothers", with 2-AG/1-AG decreasing and OEA, PEA and SEA increasing. Table 4 matches the Results: b = −47.02 for 2-AG/1-AG and +511.49 and +417.36 for OEA and PEA. The abstract therefore states the opposite direction from the Results and Table 4. The Results also call the infant SEA increase significant, but Table 4 gives SEA Time p = 0.058, and its CI (−189.20 to 257.33) implies p = 0.76.
Status
- Version of record. I checked the publisher's PDF from Cambridge Core, reached through the doi.org redirect. Its Results sentence and its Table 3 2-AG/1-AG Time row ("Time 10.52 −0.77–21.81 0.26 0.01 (0.00–0.05) 1.83 0.005**") match the PMC text quoted above. So do the Cambridge HTML Results text, the Europe PMC full-text XML (table-wrap 3) and the PMC HTML page. The misplaced SEA/PEA t values and the other p mismatches are also the same in every copy. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. There is one supplementary file,
S0033291723001204sup001.pdf, from static.cambridge.org (md5 609e36b272137d97bf8ac33543d29a77, identical to the file referenced in the PMC record). I read Supplementary Table S10, the complete-case model for N = 63 mothers. Its 2-AG/1-AG Time row is quoted above and shows the same kind of inconsistency. S10 also prints the same t and η²p columns (t = 2.94, 1.30, 0.41, −0.26) in the 2-AG/1-AG, OEA and PEA blocks. The supplement holds no raw data or model output that could recover the correct p. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref updates, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by relations. The Cambridge Core article metadata lists no corrections, retractions or addenda. The PMC page carries no erratum notice. The Europe PMC REST search returned "Search service is temporarily unavailable" on repeated attempts. The PubPeer API returned no record (HTTP 404) for the DOI, the PubPeer search page could not be queried automatically, and a web search for the title together with "correction", "erratum" or "PubPeer" found no notice or thread. - Sources. PMC article PMC10719681; Europe PMC full-text XML for PMC10719681; the Cambridge Core article page and PDF at https://doi.org/10.1017/S0033291723001204; supplementary file S0033291723001204sup001.pdf; Crossref; PubPeer; web search.
Suggested correction
The correct p value can only come from the authors' model output. Based on the printed b, CI and t, the 2-AG/1-AG Time row of Table 3 would read
Time | 10.52 | -0.77–21.81 | 0.26 | 0.01 (0.00–0.05) | 1.83 | 0.067
and the Results sentence would become, for example,
2-AG/1-AG levels in maternal hair did not change significantly over the first year postpartum (tTime = 1.83, p = 0.067, η2p = 0.01; Table 3)
with matching changes to the abstract ("Maternal hair levels of SEA decreased from late pregnancy to one year postpartum") and to the two Discussion passages that treat the 2-AG/1-AG increase as established. Supplementary Table S10 (Time, p = .004 alongside t = 1.30) needs the same check.
If the authors' output instead shows that p = 0.005 is correct, then b, the CI, t and η²p in this row are all wrong and should be replaced together. In either case, the authors may wish to re-issue Table 3 from the model output. The t column of the SEA and PEA blocks and seven p values (OEA, SEA and PEA Intercept; 2-AG/1-AG, OEA and PEA Time; SEA Time × CM) do not match the b and CI in their rows. The "marginally significant" SEA Time × CM interaction in the Results should be rechecked as well. Separately, the abstract sentence on the direction of change in children's hair should be reconciled with the Results and Table 4.
Recomputation script (python recompute.py.txt)
"""Recompute the maternal 2-AG/1-AG Time effect in Table 3 of PMC10719681
(Psychol Med 2023, doi:10.1017/S0033291723001204, "Longitudinal course of
endocannabinoids and N-acylethanolamines in hair of mothers and their children in
the first year postpartum: investigating the relevance of maternal childhood
maltreatment experiences").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (and the publisher PDF):
Table 3 (mothers, N t0 = 150, N t2 = 148): b, 95% CI, eta2p, t and p of every row.
Table 4 (children): 2-AG/1-AG Time row.
Methods: "P values of model predictors and post hoc tests were calculated based on
z values." Table 3 footer: "two-tailed".
Supplementary Table S10 (mothers with complete data, N = 63): 2-AG/1-AG rows.
Assumption for eta2p only: eta2p = t^2 / (t^2 + df), with df = observations minus
fixed effects (150 + 148 - 4 = 294 for Table 3; 2 x 63 - 4 = 122 for Table S10).
"""
import os
import numpy as np
from scipy.stats import norm, t as tdist
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.ppf(0.975)
DF3 = 150 + 148 - 4
DF_S10 = 2 * 63 - 4
def p2(z):
return 2 * norm.sf(abs(z))
def se_ci(lo, hi):
return (hi - lo) / (2 * Z975)
def eta(tval, df):
return tval ** 2 / (tval ** 2 + df)
# ---------------------------------------------------------------- 1. the claim
print("=== 1. Table 3, maternal 2-AG/1-AG, Time row ===")
b, lo, hi, t_rep, p_rep, eta_rep = 10.52, -0.77, 21.81, 1.83, 0.005, 0.01
se = se_ci(lo, hi)
z_ci = b / se
print(f"printed: b = {b}, 95% CI {lo} to {hi}, t = {t_rep}, p = {p_rep}, eta2p = {eta_rep}")
print(f"CI midpoint = {(lo + hi) / 2:.3f} (printed b = {b})")
print(f"SE from CI = ({hi} - ({lo})) / (2 x {Z975:.4f}) = {se:.3f}")
print(f"z = b / SE = {z_ci:.3f} (printed t = {t_rep})")
p_t = p2(t_rep)
p_ci = p2(z_ci)
p_tdf = 2 * tdist.sf(t_rep, DF3)
print(f"two-sided p from printed t = 1.83 as z (Methods) = {p_t:.4f}")
print(f"two-sided p from b and CI = {p_ci:.4f}")
print(f"two-sided p from t = 1.83 with t({DF3}) = {p_tdf:.4f}")
lo_r, hi_r = p2(1.835), p2(1.825)
print(f"range for t rounded to 1.83 (1.825-1.835) = {lo_r:.4f} to {hi_r:.4f}")
print(f"one-sided p from t = 1.83 = {norm.sf(t_rep):.4f}")
z_need = norm.isf(p_rep / 2)
print(f"|z| needed for two-sided p = 0.005 = {z_need:.3f}")
print(f" ratio to printed t = {z_need / t_rep:.2f}")
se_need = b / z_need
ci_need = (b - Z975 * se_need, b + Z975 * se_need)
print(f"95% CI around b = 10.52 that p = 0.005 would need = {ci_need[0]:.2f} to {ci_need[1]:.2f} (SE {se_need:.3f})")
print(f"CI includes 0: {lo < 0 < hi}")
print(f"eta2p from t = 1.83, df {DF3} = {eta(t_rep, DF3):.4f} (printed {eta_rep})")
print(f"eta2p from |t| = {z_need:.2f}, df {DF3} = {eta(z_need, DF3):.4f}")
print(f"significance at 0.05: printed p -> {p_rep < 0.05}; recomputed p -> {p_t < 0.05}")
# ---------------------------------------------------------------- 2. whole Table 3
print("\n=== 2. Table 3 (mothers), every row: printed t and p vs CI-implied z and p ===")
T3 = [
# outcome, predictor, b, lo, hi, eta2p, t, p (printed; '<0.001' kept as 0.001 bound)
("2-AG/1-AG", "Intercept", 15.02, 7.55, 22.50, 0.05, 3.94, "<0.001"),
("2-AG/1-AG", "Time", 10.52, -0.77, 21.81, 0.01, 1.83, "0.005"),
("2-AG/1-AG", "CM", 0.16, -0.04, 0.36, 0.00, 1.58, "0.113"),
("2-AG/1-AG", "Time x CM", -0.09, -0.45, 0.16, 0.00, -0.92, "0.357"),
("OEA", "Intercept", 2436.32, 1514.46, 3358.19, 0.08, 5.18, "0.008"),
("OEA", "Time", 98.61, -1293.57, 1490.78, 0.00, 0.14, "0.080"),
("OEA", "CM", -5.89, -30.45, 18.68, 0.00, -0.47, "0.639"),
("OEA", "Time x CM", -14.89, -52.59, 22.80, 0.00, -0.77, "0.439"),
("SEA", "Intercept", 1605.40, 1290.08, 1920.72, 0.25, 5.18, "0.054"),
("SEA", "Time", -926.03, -1402.21, -449.84, 0.05, 0.14, "<0.001"),
("SEA", "CM", -14.82, -23.22, -6.42, 0.04, -0.47, "0.001"),
("SEA", "Time x CM", 9.99, -2.90, 22.88, 0.00, -0.77, "0.054"),
("PEA", "Intercept", 2855.75, 1897.47, 3814.03, 0.10, 9.98, "0.002"),
("PEA", "Time", 65.19, -1381.97, 1512.36, 0.00, -3.81, "0.250"),
("PEA", "CM", -12.55, -38.08, 12.99, 0.00, -3.46, "0.335"),
("PEA", "Time x CM", -10.13, -49.31, 29.06, 0.00, 1.50, "0.612"),
]
print(f"{'outcome':10s} {'predictor':10s} {'z(CI)':>7s} {'t prt':>6s} {'p(CI)':>9s} {'p(t)':>9s} {'p prt':>7s} "
f"{'eta(zCI)':>8s} {'eta prt':>7s}")
ZCI = {}
for o, pr, bb, l, h, e, tt, pp in T3:
z = bb / se_ci(l, h)
ZCI[(o, pr)] = z
print(f"{o:10s} {pr:10s} {z:7.2f} {tt:6.2f} {p2(z):9.2g} {p2(tt):9.2g} {pp:>7s} {eta(z, DF3):8.3f} {e:7.2f}")
print("t column of SEA rows == t column of OEA rows:",
[r[6] for r in T3[8:12]] == [r[6] for r in T3[4:8]])
print("PEA printed t: ", [r[6] for r in T3[12:16]])
print("SEA CI-implied z: ", [float(round(ZCI[("SEA", p)], 2)) for p in ("Intercept", "Time", "CM", "Time x CM")])
print("PEA CI-implied z: ", [float(round(ZCI[("PEA", p)], 2)) for p in ("Intercept", "Time", "CM", "Time x CM")])
print("OEA printed t: ", [r[6] for r in T3[4:8]])
print("OEA CI-implied z: ", [float(round(ZCI[("OEA", p)], 2)) for p in ("Intercept", "Time", "CM", "Time x CM")])
print("2-AG/1-AG Time x CM: CI midpoint", round((-0.45 + 0.16) / 2, 3), "vs printed b -0.09")
print("2-AG/1-AG rows with |z(CI) - t| <= 0.02:",
[r[1] for r in T3[:4] if abs(ZCI[(r[0], r[1])] - r[6]) <= 0.02])
# ---------------------------------------------------------------- 3. Table S10 and Table 4
print("\n=== 3. Supplementary Table S10 (mothers, N = 63), 2-AG/1-AG Time row ===")
b10, l10, h10, t10, p10 = 11.50, -5.87, 28.88, 1.30, 0.004
z10 = b10 / se_ci(l10, h10)
print(f"printed: b = {b10}, CI {l10} to {h10}, t = {t10}, p = .004")
print(f"SE from CI = {se_ci(l10, h10):.3f}; z = {z10:.3f}; p from z(CI) = {p2(z10):.4f}; p from t as z = {p2(t10):.4f}")
print(f"eta2p from t = 1.30, df {DF_S10} = {eta(t10, DF_S10):.4f} (printed 0.01)")
b10i, l10i, h10i = 18.44, 6.15, 30.72
print(f"S10 Intercept: z(CI) = {b10i / se_ci(l10i, h10i):.3f} (printed t 2.94, p <.001)")
print("\n=== 4. Table 4 (children), 2-AG/1-AG Time row ===")
b4, l4, h4 = -47.02, -77.51, -16.52
z4 = b4 / se_ci(l4, h4)
print(f"z(CI) = {z4:.3f} (printed t -3.02); p = {p2(z4):.4f} (printed <0.001); "
f"p from t = -3.02 as z = {p2(-3.02):.4f}")
print("Table 4 Time rows (side observation: sign of b vs the abstract's direction words):")
for o, bb, l, h, tt, pp in [("2-AG/1-AG", -47.02, -77.51, -16.52, -3.02, "<0.001"),
("OEA", 511.49, 343.86, 679.12, 5.98, "<0.001"),
("SEA", 34.06, -189.20, 257.33, 0.30, "0.058"),
("PEA", 417.36, 238.00, 596.71, 4.56, "<0.001")]:
z = bb / se_ci(l, h)
print(f" {o:10s} b = {bb:8.2f} ({'rise' if bb > 0 else 'fall'} t0->t2) z(CI) = {z:5.2f} "
f"p(CI) = {p2(z):.2g} printed t = {tt}, p = {pp}")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"axes.edgecolor": INK2, "axes.spines.top": False, "axes.spines.right": False,
"axes.spines.left": False, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# Figure 1: two-sided p values on one log axis
rows = [
("Printed in Table 3 and Results", p_rep, ORANGE, "o", "p = 0.005 (printed)"),
("From printed t = 1.83 as z\n(Methods: p from z values)", p_t, BLUE, "s", f"p = {p_t:.3f}"),
("From printed b and 95% CI\n(SE = 5.76, z = 1.83)", p_ci, BLUE, "s", f"p = {p_ci:.3f}"),
(f"From t = 1.83 with t({DF3})", p_tdf, BLUE, "s", f"p = {p_tdf:.3f}"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.1))
ys = np.arange(len(rows))[::-1]
for y, (lab, p, c, m, txt) in zip(ys, rows):
ax.plot([p], [y], marker=m, ms=10, color=c, mec=c, ls="none", zorder=3)
ax.annotate(txt, (p, y), xytext=(10, 0), textcoords="offset points", va="center",
color=c if c == BLUE else "#b84a1e", fontsize=10.5)
ax.axvline(0.05, color=INK2, lw=0.9, ls=(0, (3, 3)), zorder=1)
ax.text(0.05, ys[0] + 0.62, "α = 0.05", ha="center", va="bottom", color=INK2, fontsize=10)
ax.set_xscale("log")
ax.set_xlim(0.002, 0.5)
ax.set_ylim(-0.6, len(rows) - 0.2)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.tick_params(axis="y", length=0)
ax.set_xticks([0.002, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5])
ax.set_xticklabels(["0.002", "0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.5"])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.set_xlabel("Two-sided p value, maternal 2-AG/1-AG Time effect (log scale)")
ax.plot([], [], "o", color=ORANGE, ms=9, label="reported in the paper")
ax.plot([], [], "s", color=BLUE, ms=9, label="recomputed from the paper's own numbers")
ax.legend(loc="upper center", bbox_to_anchor=(0.4, -0.25), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: the coefficient and its 95% CI on one linear axis
rows2 = [
("Table 3 (N = 150 / 148)\nprinted b and 95% CI", b, lo, hi, ORANGE, "o",
f"b = {b}, CI {lo} to {hi}; printed p = 0.005"),
("95% CI that p = 0.005\nwould require (same b)", b, ci_need[0], ci_need[1], BLUE, "s",
f"CI {ci_need[0]:.2f} to {ci_need[1]:.2f} (|z| = {z_need:.2f})"),
("Suppl. Table S10 (N = 63)\nprinted b and 95% CI", b10, l10, h10, ORANGE, "o",
f"b = {b10:.2f}, CI {l10} to {h10}; printed p = .004"),
]
fig, ax = plt.subplots(figsize=(7.2, 2.9))
ys = np.arange(len(rows2))[::-1]
for y, (lab, bb, l, h, c, m, txt) in zip(ys, rows2):
ax.plot([l, h], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([bb], [y], marker=m, ms=9, color=c, ls="none", zorder=3)
ax.text(bb, y + 0.22, txt, ha="center", va="bottom", fontsize=10,
color=c if c == BLUE else "#b84a1e",
bbox=dict(facecolor=BG, edgecolor="none", pad=1.0), zorder=4)
ax.axvline(0, color=INK2, lw=0.9, zorder=1)
ax.text(-0.4, -0.38, "no change (b = 0)", ha="right", va="center", color=INK2, fontsize=10)
ax.set_xlim(-12, 34)
ax.set_ylim(-0.5, len(rows2) - 0.25)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows2])
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.set_xlabel("Time effect b on maternal hair 2-AG/1-AG, t0 to t2 (with 95% CI)")
ax.plot([], [], "o-", color=ORANGE, ms=8, lw=2, label="reported in the paper")
ax.plot([], [], "s-", color=BLUE, ms=8, lw=2, label="computed from the paper's numbers")
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract reports PGS-ADHD odds ratios above 1 as decreased odds; Results report them below 1
Psychol Med (2024) · doi:10.1017/S0033291724002186 · PMC11578915
Summary
The study fits multinomial regressions of four treatment trajectories on polygenic scores (PGS) in 10,577 people with early-onset depression, with the brief contact trajectory as the reference. The Abstract says higher PGS-ADHD was linked to "decreased odds" of the prolonged initial contact and persistent contact trajectories, but it prints odds ratios above 1 for both: 1.06 (1.01–1.11) and 1.12 (1.03–1.21). For the same comparisons the Results text and Supplementary Table 2 give 0.91 (0.87–0.96), p = 0.0002, and 0.90 (0.83–0.97), p = 0.007. All four intervals exclude 1, so the Abstract's numbers disagree with the body in direction and value, and the two Abstract triples are identical to the PGS-AN rows of Supplementary Table 2. The Abstract is the paper's most widely read summary, and its printed figures reverse the direction of the result the Discussion calls the strongest.
What the paper reports
Relative to the brief contact trajectory, higher PGS for ADHD was associated with a decreased odds of membership in the prolonged initial contact (odds ratio = 1.06, 95% confidence interval = 1.01–1.11) and persistent contact (1.12, 1.03–1.21) trajectories, while PGS-AN was associated with increased odds of membership in the persistent contact trajectory (1.12, 1.03–1.21).
— Abstract, Results
PGS-ADHD was associated with decreased odds of membership in the prolonged initial contact (OR = 0.91 [0.87–0.96], p = 0.0002) and persistent contact classes (OR = 0.90 [0.83–0.97], p = 0.007). PGS-AN was associated with increased odds of membership in the persistent contact class (OR = 1.12 [1.03–1.21], p = 0.005).
— Results, "PGS and trajectory patterns in secondary care", second paragraph
For each PGS and family history variable, we calculated the odds ratio (OR), 95% confidence intervals (CIs), and p-value for trajectory class membership relative to the brief contact class using multinomial regression models adjusted for sex, age-at-onset, and calendar year-at-diagnosis in three-year bands.
— Methods, "Statistical analysis"
Statistical significance was assessed at the Bonferroni-adjusted alpha of 0.0083 for the PGS (0.05/6 PGSs), and 0.017 for parental history (0.05/3 parental history variables).
— Methods, "Statistical analysis"
The brief contact class (class 1) was used as the reference category in the multinomial regressions.
— Figure 2 legend
The strongest results were for PGS-ADHD, which was associated with increased likelihood of a brief contact pattern relative to the prolonged initial contact and persistent contact patterns, particularly for younger individuals.
— Discussion
Supplementary Table 2 ("Associations between continuous polygenic scores (PGSs) and trajectory classes"; brief contact is the reference) lists, as OR, SE, p-value, lower CI, upper CI:
Prolonged initial contact | PGS-ADHD | 0.91 | 0.02 | 2.12x10-04 | 0.87 | 0.96 Persistent contact | PGS-ADHD | 0.90 | 0.04 | 6.99x10-03 | 0.83 | 0.97 Prolonged initial contact | PGS-AN | 1.06 | 0.02 | 3.06x10-02 | 1.01 | 1.11 Persistent contact | PGS-AN | 1.12 | 0.04 | 5.00x10-03 | 1.03 | 1.21
— Supplementary material (S0033291724002186sup001.docx), STable 2, rows for PGS-ADHD and PGS-AN
What the paper's own numbers show
Method. In a multinomial logistic model with brief contact as the reference, the odds ratio for class k is exp(β), where β is the change in log[P(k)/P(brief)] per SD of PGS. An OR above 1 therefore means increased odds of class k relative to brief contact, and an OR below 1 means decreased odds. A Wald 95% CI is exp(β ± 1.96 SE), so SE = (ln hi − ln lo)/3.92, z = ln(OR)/SE and the two-sided p follows. I applied this to every Abstract and Results triple. I also took reciprocals of the Results values, to test whether a swapped reference category could produce the Abstract's numbers, and searched all 18 rows of Supplementary Table 2, and the other trajectory-class tables of the supplement (STables 3, 4, 5 and 18), for the Abstract's exact OR and CI bounds and for any PGS-ADHD estimate above 1 for these two classes. Every number below is printed by recompute.py.txt (Python, numpy/scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| PGS-ADHD, prolonged initial contact | Abstract | 1.06 (1.01–1.11), "decreased odds" | OR > 1, so increased odds (+6% per SD); z = +2.42, p = 0.0155 |
| PGS-ADHD, prolonged initial contact | Results text; STable 2 | 0.91 [0.87–0.96], p = 0.0002 | OR < 1, decreased odds (−9% per SD); z = −3.76, p = 0.0002 |
| PGS-ADHD, persistent contact | Abstract | 1.12 (1.03–1.21), "decreased odds" | OR > 1, so increased odds (+12% per SD); z = +2.76, p = 0.0058 |
| PGS-ADHD, persistent contact | Results text; STable 2 | 0.90 [0.83–0.97], p = 0.007 | OR < 1, decreased odds (−10% per SD); z = −2.65, p = 0.0081 |
| PGS-AN, persistent contact | Abstract; Results text; STable 2 | 1.12 [1.03–1.21] (p = 0.005 in Results text and STable 2) | z = +2.76, p = 0.0058; all three places agree |
| Reciprocal of Results PGS-ADHD, prolonged | derived | Abstract: 1.06 (1.01–1.11) | 1.10 [1.04–1.15]; does not match |
| Reciprocal of Results PGS-ADHD, persistent | derived | Abstract: 1.12 (1.03–1.21) | 1.11 [1.03–1.20]; does not match |
| Supplement rows equal to the Abstract's "ADHD, prolonged" triple | STable 2; STable 5 | 1.06 (1.01–1.11) | PGS-AN, prolonged initial contact, in STable 2 (printed p = 0.0306) and in STable 5 (printed p = 0.0281); no PGS-ADHD row |
| STable 2 rows equal to the Abstract's "ADHD, persistent" triple | STable 2 | 1.12 (1.03–1.21) | one row: PGS-AN, persistent contact |
| Abstract prolonged-contact value against the Bonferroni α | Abstract; Methods | α = 0.05/6 = 0.0083 | p = 0.0155 from the CI (0.0306 as printed in STable 2 for the matching PGS-AN row); neither passes |
| PGS-ADHD in the mutually adjusted model | STable 5 | 0.90 [0.86–0.95] and 0.91 [0.84–0.99] | both below 1 |
| PGS-ADHD for these two classes in the age strata and top-decile/5% contrasts | STable 18; STables 3–4 | 18 estimates | 16 below 1; 2 point estimates above 1 (age 21–25, persistent contact, 1.13 [0.98–1.31], p = 0.0968; top 5% vs bottom 5%, persistent contact, 1.04 [0.62–1.74]), both with CIs that include 1; none has a CI wholly above 1 and none matches an Abstract triple |
The Results-text and STable 2 values are internally consistent: the p values recovered from their CIs (0.0002 and 0.0081) match the printed 0.0002 and 0.007 to within rounding of the two-decimal bounds. The Abstract's two "ADHD" triples are not consistent with the word "decreased" in the same sentence, with the Results text, or with any reciprocal of it. Each matches, digit for digit, a PGS-AN row of Supplementary Table 2.
Figure
Figure 1. The two PGS-ADHD comparisons against the brief contact trajectory, as printed in the Abstract (left) and in the Results text (right). All numbers are copied from the paper (orange: Abstract and Results text as printed); no recomputed values are drawn. The Abstract prints OR 1.06 (1.01–1.11) for prolonged initial contact and 1.12 (1.03–1.21) for persistent contact, both above 1. The Results text, which agrees with Supplementary Table 2, prints 0.91 [0.87–0.96] (p = 0.0002) and 0.90 [0.83–0.97] (p = 0.007), both below 1. Each pair of intervals excludes 1 on opposite sides, so the two sources disagree in direction, not only in value. Both sources describe the association in words as "decreased odds", which fits only the Results values. The Abstract's two triples are identical to the PGS-AN rows of Supplementary Table 2 for the same classes.
Why it matters
The PGS-ADHD association is the result the Discussion singles out as "the strongest". The Abstract is what most readers, indexers and citing authors see, and it is the only part of the paper whose numbers point the other way. A reader who takes the Abstract's numbers at face value would conclude that higher PGS-ADHD raises the odds of the longer-treatment trajectories, by about 6% and 12% per SD. The body says the opposite: it lowers them, by about 9% and 10% per SD. The Abstract's prolonged-contact value would also fail the paper's own multiple-testing threshold (p = 0.0155 from its CI, against α = 0.0083), whereas the body's value passes (p = 0.0002). Anyone pooling these estimates in a meta-analysis or citing them from the Abstract would carry the wrong sign.
The error is confined to the Abstract's numbers. The Abstract's wording, the Results text, Figure 2 (brief contact as reference), Supplementary Tables 2 and 5, the Discussion and the Conclusions all agree with ORs below 1, and the age-stratified results describe "the negative association between PGS-ADHD and the persistent contact class" as driven by ages 10–20 (STable 18: 0.81 [0.71–0.94] at 10–15 and 0.84 [0.74–0.95] at 16–20). The paper's main conclusions are therefore not affected.
Explanations considered
- A different reference category. The Abstract states the comparison as "relative to the brief contact trajectory", the same as the Methods, the Results text and the Figure 2 legend. If the reference were swapped, the Results values would become 1.10 [1.04–1.15] and 1.11 [1.03–1.20]. Neither matches the Abstract's 1.06 (1.01–1.11), and a swap would also make the Abstract's word "decreased" wrong. This does not account for the numbers.
- A different model or subgroup. The mutually adjusted model (STable 5) gives PGS-ADHD ORs of 0.90 [0.86–0.95] and 0.91 [0.84–0.99]. The age-stratified analyses are described as a "negative association". No model or subgroup in the paper gives a PGS-ADHD OR for these two classes whose CI lies entirely above 1, or that matches the Abstract's triples. The only point estimates above 1 (age 21–25, persistent contact, 1.13 [0.98–1.31], STable 18; top 5% vs bottom 5%, persistent contact, 1.04 [0.62–1.74], STable 4) have CIs that include 1. This does not account for the numbers.
- A text-conversion or rendering artifact. The same values appear in the PMC archive, the Europe PMC full text, the publisher's abstract deposited with Crossref, and the Cambridge Core version of record. They are complete OR–CI triples in running text, with no minus signs, superscripts or table cells involved. This does not account for the numbers.
- The Abstract being correct and the body wrong. The Results text, Supplementary Table 2 (with p values that match its CIs), Supplementary Table 5, the Discussion, and the Abstract's own wording all agree on ORs below 1. Only the Abstract's two numeric triples disagree. The body values are the ones supported throughout the paper.
- Values carried from another row. Both Abstract triples equal, OR and both CI bounds, the PGS-AN rows of Supplementary Table 2 for the same two classes (1.06 [1.01–1.11], p = 0.0306; 1.12 [1.03–1.21], p = 0.005). The same 1.06 (1.01–1.11) triple is also the PGS-AN, prolonged initial contact row of the mutually adjusted model (STable 5, p = 0.0281), and it appears nowhere in the main text. This match is consistent with the Abstract reporting the PGS-AN rows under the PGS-ADHD label. It describes what the numbers show, not how they came to be there.
No explanation found reconciles the Abstract's numbers with the rest of the paper.
Status
- Version of record. Checked on Cambridge Core (doi.org resolves to cambridge.org/core/product/identifier/S0033291724002186/type/journal_article). The Abstract and Results text carry the same values as the PMC archive. The publisher's abstract in the Crossref record and the Europe PMC full text also carry 1.06 (1.01–1.11) and 1.12 (1.03–1.21).
- Supplement. S0033291724002186sup001.docx, downloaded through Europe PMC. STable 2 and STable 5 support the Results text, as shown above; STables 3, 4 and 18 contain no PGS-ADHD estimate for these two classes that matches the Abstract or has a CI wholly above 1. A side observation: the STable 2 and STable 5 footnotes give the Bonferroni alpha as "0.00083", while the Methods give 0.0083 (0.05/6). This does not bear on the Abstract discrepancy.
- Corrections. As of 2026-10-23 no correction, erratum, corrigendum, expression of concern or retraction was found. Crossref lists no update relations for the DOI, and a Crossref
updatesfilter search returned none. PMC and Europe PMC list no notices. - PubPeer. The PubPeer API and search page refused automated access (Cloudflare challenge). A web search for the title with "pubpeer" returned no PubPeer thread.
- Sources. PMC11578915 (local archive via
src/show.py); Europe PMC full text and supplementary files; Crossref works record for 10.1017/S0033291724002186; Cambridge Core version of record; PubPeer;src/known.py.
Suggested correction
In the Abstract, Results paragraph, replace "(odds ratio = 1.06, 95% confidence interval = 1.01–1.11)" with "(odds ratio = 0.91, 95% confidence interval = 0.87–0.96)" and "persistent contact (1.12, 1.03–1.21) trajectories" with "persistent contact (0.90, 0.83–0.97) trajectories", matching the Results text and Supplementary Table 2. The PGS-AN estimate in the same sentence (1.12, 1.03–1.21) is correct and needs no change. Separately, the Bonferroni alpha in the footnotes of Supplementary Tables 2 and 5 could be corrected from 0.00083 to 0.0083 to match the Methods.
Recomputation script (python recompute.py.txt)
"""Recompute the PGS-ADHD abstract-versus-Results discrepancy in PMC11578915
(Psychol Med 2024, doi:10.1017/S0033291724002186) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md. Writes, into the folder that
holds this file:
candidates/cand_A..cand_E (.svg/.png) five alternative designs for Figure 1
fig1.svg / fig1.png a copy of the design named in FIG1 below
All inputs are copied from the paper: the Abstract, the Results section
("PGS and trajectory patterns in secondary care"), the Methods (Bonferroni
alpha) and Supplementary Table 2 (STable 2; continuous PGS vs trajectory class,
brief contact = reference). STable 5 (mutually adjusted model), STable 18
(age-stratified models) and STables 3-4 (top 10% / top 5% contrasts) are used only as
further checks.
Method: Wald inference on the log-odds scale. A 95% CI for an odds ratio is
exp(ln OR +/- 1.96 SE), so SE = (ln hi - ln lo) / (2 * 1.959964),
z = ln(OR) / SE and two-sided p = 2 * (1 - Phi(|z|)).
"""
import os
import shutil
import numpy as np
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
FIG1 = "A" # which candidate design is copied to fig1.*
Z = norm.ppf(0.975)
ALPHA_BONF = 0.05 / 6 # Methods: "0.0083 for the PGS (0.05/6 PGSs)"
def wald(o, lo, hi):
se = (np.log(hi) - np.log(lo)) / (2 * Z)
z = np.log(o) / se
return se, z, 2 * norm.sf(abs(z))
line = "-" * 78
# ------------------------------------------------------------------ inputs
ABSTRACT = { # label: (OR, lo, hi) -- Abstract, Results paragraph
"ADHD, prolonged initial contact": (1.06, 1.01, 1.11),
"ADHD, persistent contact": (1.12, 1.03, 1.21),
"AN, persistent contact": (1.12, 1.03, 1.21),
}
RESULTS = { # label: (OR, lo, hi, printed p) -- Results text
"ADHD, prolonged initial contact": (0.91, 0.87, 0.96, 0.0002),
"ADHD, persistent contact": (0.90, 0.83, 0.97, 0.007),
"AN, persistent contact": (1.12, 1.03, 1.21, 0.005),
}
# STable 2: (class, PGS): (OR, SE, p, lo, hi)
ST2 = {
("Prolonged initial contact", "MDD"): (0.96, 0.02, 6.60e-02, 0.91, 1.00),
("Later re-entry", "MDD"): (1.09, 0.04, 1.47e-02, 1.02, 1.17),
("Persistent contact", "MDD"): (0.95, 0.04, 1.89e-01, 0.88, 1.03),
("Prolonged initial contact", "ADHD"): (0.91, 0.02, 2.12e-04, 0.87, 0.96),
("Later re-entry", "ADHD"): (1.00, 0.04, 9.32e-01, 0.93, 1.07),
("Persistent contact", "ADHD"): (0.90, 0.04, 6.99e-03, 0.83, 0.97),
("Prolonged initial contact", "ASD"): (1.01, 0.02, 5.79e-01, 0.97, 1.06),
("Later re-entry", "ASD"): (1.08, 0.04, 3.15e-02, 1.01, 1.16),
("Persistent contact", "ASD"): (0.95, 0.04, 2.20e-01, 0.88, 1.03),
("Prolonged initial contact", "BD"): (1.03, 0.02, 2.00e-01, 0.98, 1.08),
("Later re-entry", "BD"): (0.99, 0.04, 8.68e-01, 0.92, 1.07),
("Persistent contact", "BD"): (0.99, 0.04, 7.15e-01, 0.91, 1.07),
("Prolonged initial contact", "SCZ"): (0.99, 0.02, 6.37e-01, 0.94, 1.04),
("Later re-entry", "SCZ"): (1.05, 0.04, 2.31e-01, 0.97, 1.12),
("Persistent contact", "SCZ"): (0.99, 0.04, 7.83e-01, 0.91, 1.07),
("Prolonged initial contact", "AN"): (1.06, 0.02, 3.06e-02, 1.01, 1.11),
("Later re-entry", "AN"): (1.02, 0.04, 5.99e-01, 0.95, 1.10),
("Persistent contact", "AN"): (1.12, 0.04, 5.00e-03, 1.03, 1.21),
}
ST5_ADHD = { # mutually adjusted model, STable 5
"prolonged initial contact": (0.90, 0.86, 0.95, 1.93e-04),
"persistent contact": (0.91, 0.84, 0.99, 3.28e-02),
}
ST5_AN_PRO = (1.06, 1.01, 1.11, 2.81e-02) # STable 5, PGS-AN, prolonged initial contact
# Every other PGS-ADHD estimate in the supplement for the two classes at issue
# (source table, model/subgroup, class): (OR, lo, hi, printed p)
OTHER_ADHD = {
("STable 18", "age 10-15", "prolonged initial contact"): (0.85, 0.77, 0.95, 2.83e-03),
("STable 18", "age 10-15", "persistent contact"): (0.81, 0.71, 0.94, 3.78e-03),
("STable 18", "age 16-20", "prolonged initial contact"): (0.95, 0.88, 1.01, 1.17e-01),
("STable 18", "age 16-20", "persistent contact"): (0.84, 0.74, 0.95, 5.03e-03),
("STable 18", "age 21-25", "prolonged initial contact"): (0.90, 0.82, 0.99, 3.01e-02),
("STable 18", "age 21-25", "persistent contact"): (1.13, 0.98, 1.31, 9.68e-02),
("STable 3", "top 10% vs bottom 10%", "prolonged initial contact"): (0.70, 0.56, 0.87, 1.60e-03),
("STable 3", "top 10% vs bottom 10%", "persistent contact"): (0.71, 0.50, 1.00, 5.17e-02),
("STable 3", "top 10% vs bottom 90%", "prolonged initial contact"): (0.83, 0.70, 0.99, 3.49e-02),
("STable 3", "top 10% vs bottom 90%", "persistent contact"): (0.88, 0.67, 1.15, 3.35e-01),
("STable 3", "top 10% vs bottom 50%", "prolonged initial contact"): (0.79, 0.66, 0.94, 8.95e-03),
("STable 3", "top 10% vs bottom 50%", "persistent contact"): (0.80, 0.61, 1.06, 1.22e-01),
("STable 4", "top 5% vs bottom 5%", "prolonged initial contact"): (0.70, 0.51, 0.96, 2.80e-02),
("STable 4", "top 5% vs bottom 5%", "persistent contact"): (1.04, 0.62, 1.74, 8.77e-01),
("STable 4", "top 5% vs bottom 95%", "prolonged initial contact"): (0.82, 0.65, 1.04, 1.04e-01),
("STable 4", "top 5% vs bottom 95%", "persistent contact"): (0.97, 0.68, 1.39, 8.82e-01),
("STable 4", "top 5% vs bottom 50%", "prolonged initial contact"): (0.77, 0.61, 0.99, 3.80e-02),
("STable 4", "top 5% vs bottom 50%", "persistent contact"): (0.88, 0.61, 1.28, 5.12e-01),
}
# ------------------------------------------------------------------ 1. abstract values
print(line)
print(f"Bonferroni alpha for the PGS = 0.05/6 = {ALPHA_BONF:.5f}")
print(line)
print("1. Abstract values: direction and p implied by each printed CI")
abs_calc = {}
for k, (o, lo, hi) in ABSTRACT.items():
se, z, p = wald(o, lo, hi)
abs_calc[k] = (se, z, p)
print(f" {k:32s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] direction "
f"{'increased' if o > 1 else 'decreased'}; SE {se:.4f}; z {z:+.2f}; "
f"p {p:.4f}; passes Bonferroni: {p < ALPHA_BONF}; "
f"change in odds per SD {100 * (o - 1):+.0f}%")
# ------------------------------------------------------------------ 2. Results values
print(line)
print("2. Results-text values: p implied by each printed CI vs printed p")
res_calc = {}
for k, (o, lo, hi, pr) in RESULTS.items():
se, z, p = wald(o, lo, hi)
res_calc[k] = (se, z, p)
print(f" {k:32s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr} | "
f"SE {se:.4f}; z {z:+.2f}; implied p {p:.4f}; direction "
f"{'increased' if o > 1 else 'decreased'}; change in odds per SD {100 * (o - 1):+.0f}%")
# ------------------------------------------------------------------ 3. reference swap
print(line)
print("3. Reciprocals of the Results ADHD values (what a swapped reference category would give)")
recip = {}
for k in ("ADHD, prolonged initial contact", "ADHD, persistent contact"):
o, lo, hi, _ = RESULTS[k]
r = (1 / o, 1 / hi, 1 / lo)
recip[k] = r
a = ABSTRACT[k]
print(f" {k:32s} 1/OR = {r[0]:.2f} [{r[1]:.2f}-{r[2]:.2f}] abstract {a[0]:.2f} "
f"[{a[1]:.2f}-{a[2]:.2f}] matches to 2 dp: "
f"{tuple(round(x, 2) for x in r) == a}")
# ------------------------------------------------------------------ 4. matching to STable 2
print(line)
print("4. Which STable 2 rows carry exactly the abstract's OR and CI (2 dp)?")
matches = {}
for k, (o, lo, hi) in ABSTRACT.items():
m = [f"{c} / PGS-{g}" for (c, g), v in ST2.items() if (v[0], v[3], v[4]) == (o, lo, hi)]
matches[k] = m
print(f" abstract {k:32s} {o:.2f} [{lo:.2f}-{hi:.2f}] -> {m}")
print(" STable 2 rows labelled PGS-ADHD for the same classes:")
for c in ("Prolonged initial contact", "Persistent contact"):
v = ST2[(c, "ADHD")]
print(f" {c:28s} OR {v[0]:.2f} [{v[3]:.2f}-{v[4]:.2f}] p {v[2]:.2e}")
print(" STable 2 printed SE vs SE recovered from its CI, and printed p vs implied p:")
for (c, g), (o, se_p, p_p, lo, hi) in ST2.items():
if g in ("ADHD", "AN") and c != "Later re-entry":
se, z, p = wald(o, lo, hi)
print(f" {c:28s} PGS-{g:4s} printed SE {se_p:.2f} vs {se:.3f}; printed p {p_p:.2e} vs {p:.2e}")
n_any = sum(1 for v in ST2.values() if (v[0], v[3], v[4]) == ABSTRACT["ADHD, prolonged initial contact"])
print(f" rows in STable 2 equal to 1.06 [1.01-1.11]: {n_any}")
# ------------------------------------------------------------------ 5. mutually adjusted model
print(line)
print("5. STable 5 (mutually adjusted) PGS-ADHD rows")
for k, (o, lo, hi, pr) in ST5_ADHD.items():
se, z, p = wald(o, lo, hi)
print(f" {k:28s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr:.2e}; implied p {p:.2e}; below 1: {hi < 1}")
o, lo, hi, pr = ST5_AN_PRO
print(f" STable 5 PGS-AN, prolonged initial contact: OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr:.2e}; "
f"equals the Abstract's 'ADHD, prolonged' triple: {(o, lo, hi) == ABSTRACT['ADHD, prolonged initial contact']}")
# ------------------------------------------------------------------ 6. other PGS-ADHD estimates
print(line)
print("6. Other PGS-ADHD estimates for the two classes (STable 18 age strata, STable 3 and 4 top-10%/5% contrasts)")
abs_triples = {ABSTRACT["ADHD, prolonged initial contact"], ABSTRACT["ADHD, persistent contact"]}
for (tab, sub, c), (o, lo, hi, pr) in OTHER_ADHD.items():
se, z, p = wald(o, lo, hi)
print(f" {tab:9s} {sub:22s} {c:26s} OR {o:.2f} [{lo:.2f}-{hi:.2f}] printed p {pr:.2e}; implied p {p:.3f}; "
f"OR>1 {o > 1}; CI wholly above 1 {lo > 1}; equals an Abstract triple {(o, lo, hi) in abs_triples}")
n_gt1 = sum(v[0] > 1 for v in OTHER_ADHD.values())
n_ci_gt1 = sum(v[1] > 1 for v in OTHER_ADHD.values())
n_match = sum(v[:3] in abs_triples for v in OTHER_ADHD.values())
print(f" point estimates above 1: {n_gt1}; CIs wholly above 1: {n_ci_gt1}; equal to an Abstract triple: {n_match}")
print(line)
# ================================================================== figures
import matplotlib # noqa: E402
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
LW, MS = 1.5, 8 # 1.5 pt = 2 px; marker 8 pt
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
W = 7.2
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(CAND, f"{name}.{ext}"), bbox_inches="tight", **kw)
plt.close(fig)
def fmt(t):
return f"{t[0]:.2f} [{t[1]:.2f}–{t[2]:.2f}]"
def log_axis(ax, lo=0.78, hi=1.30, ticks=(0.8, 0.85, 0.9, 0.95, 1.0, 1.05, 1.1, 1.15, 1.2)):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ticks = [t for t in ticks if lo <= t <= hi]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
A_PRO, A_PER = ABSTRACT["ADHD, prolonged initial contact"], ABSTRACT["ADHD, persistent contact"]
R_PRO, R_PER = RESULTS["ADHD, prolonged initial contact"], RESULTS["ADHD, persistent contact"]
AN_PRO = ST2[("Prolonged initial contact", "AN")]
AN_PER = ST2[("Persistent contact", "AN")]
# ------------------------------------------------------------------ A: two text panels
def cand_A():
fig = plt.figure(figsize=(W, 4.3))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off()
ax.set_xlim(0, 100); ax.set_ylim(0, 60)
def panel(x, w, title, sub):
ax.add_patch(FancyBboxPatch((x, 4), w, 50, boxstyle="round,pad=0,rounding_size=1.2",
fc="white", ec=INK2, lw=0.8))
ax.text(x + 2, 50.5, title, fontsize=11.5, weight="bold", va="top")
ax.text(x + 2, 46.3, sub, fontsize=10, color=INK2, va="top")
panel(1, 38, "Abstract", "Results paragraph")
panel(57, 42, "Results text", "“PGS and trajectory patterns”")
ax.text(50, 58.5, "PGS-ADHD vs the brief contact trajectory (reference): the same two comparisons",
ha="center", fontsize=10.5, va="top")
rows = [("Prolonged initial contact", A_PRO, (R_PRO[0], R_PRO[1], R_PRO[2]), f"p = {R_PRO[3]}", 34),
("Persistent contact", A_PER, (R_PER[0], R_PER[1], R_PER[2]), f"p = {R_PER[3]}", 19)]
for lab, a, r, ptxt, y in rows:
ax.text(3, y + 4.2, lab, fontsize=10, color=INK2)
ax.text(3, y, f"OR {fmt(a)}", fontsize=12.5, color=ORANGE, weight="bold")
ax.text(3, y - 4, "OR > 1: increased odds", fontsize=10, color=INK2)
ax.text(59.3, y + 4.2, lab, fontsize=10, color=INK2)
ax.text(59.3, y, f"OR {fmt(r)}", fontsize=12.5, color=ORANGE, weight="bold")
ax.text(59.3, y - 4, f"{ptxt}; OR < 1: decreased odds", fontsize=10, color=INK2)
ax.annotate("", xy=(56.6, y + 1), xytext=(39.4, y + 1),
arrowprops=dict(arrowstyle="<->", color=INK, lw=LW))
ax.text(48, y + 2.6, "opposite\ndirection", ha="center", va="bottom", fontsize=10, weight="bold", linespacing=1.1)
ax.text(48, y - 1.0, "both CIs\nexclude 1", ha="center", va="top", fontsize=10, color=INK2, linespacing=1.1)
ax.text(3, 9.8, "Wording: “decreased odds”", fontsize=10)
ax.text(3, 6.0, "Values equal the PGS-AN rows", fontsize=10, color=INK2)
ax.text(59.3, 9.8, "Wording: “decreased odds”", fontsize=10)
ax.text(59.3, 6.0, "Matches PGS-ADHD rows of STable 2", fontsize=10, color=INK2)
ax.text(1, 1.0, "Orange: values as printed in the paper.", fontsize=10, color=INK2)
save(fig, "cand_A")
# ------------------------------------------------------------------ B: dot-and-interval, 2 rows
def cand_B():
fig, ax = plt.subplots(figsize=(W, 3.4))
log_axis(ax, 0.80, 1.36)
items = [ # y, (o, lo, hi), style
(3.25, A_PRO, "abs"), (2.95, (R_PRO[0], R_PRO[1], R_PRO[2]), "res"), (2.65, recip["ADHD, prolonged initial contact"], "rec"),
(1.35, A_PER, "abs"), (1.05, (R_PER[0], R_PER[1], R_PER[2]), "res"), (0.75, recip["ADHD, persistent contact"], "rec"),
]
for y, (o, lo, hi), st in items:
col = BLUE if st == "rec" else ORANGE
mk = {"abs": "D", "res": "o", "rec": "s"}[st]
fc = "white" if st == "abs" else col
ax.plot([lo, hi], [y, y], color=col, lw=LW, solid_capstyle="round", zorder=2)
ax.plot(o, y, mk, ms=MS, mfc=fc, mec=col, mew=LW, zorder=3)
ax.text(1.232, y, fmt((o, lo, hi)), va="center", fontsize=10, color=INK)
ax.set_yticks([2.95, 1.05])
ax.set_yticklabels(["Prolonged\ninitial contact", "Persistent\ncontact"], fontsize=10.5, color=INK)
ax.set_ylim(0.4, 3.75)
ax.set_xlabel("Odds ratio per SD of PGS-ADHD vs brief contact (log scale)")
ax.text(0.995, 3.68, "← decreased odds", ha="right", va="top", fontsize=10, color=INK2)
ax.text(1.005, 3.68, "increased odds →", ha="left", va="top", fontsize=10, color=INK2)
h = [Line2D([], [], color=ORANGE, marker="D", mfc="white", ms=MS, lw=LW, mew=LW, label="Abstract (printed)"),
Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Results text and STable 2 (printed)"),
Line2D([], [], color=BLUE, marker="s", ms=MS, lw=LW, label="1/OR of Results values (recomputed)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_B")
# ------------------------------------------------------------------ C: source-matching diagram
def cand_C():
fig = plt.figure(figsize=(W, 4.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off()
ax.set_xlim(0, 100); ax.set_ylim(0, 56)
ax.text(2, 53, "Abstract, as printed", fontsize=11.5, weight="bold", va="top")
ax.text(58, 53, "Supplementary Table 2, as printed", fontsize=11.5, weight="bold", va="top")
left = [("PGS-ADHD, prolonged", A_PRO, 40), ("PGS-ADHD, persistent", A_PER, 28), ("PGS-AN, persistent", ABSTRACT["AN, persistent contact"], 16)]
right = {("Prolonged initial contact", "ADHD"): 44, ("Persistent contact", "ADHD"): 35,
("Prolonged initial contact", "AN"): 23, ("Persistent contact", "AN"): 14}
for (c, g), y in right.items():
v = ST2[(c, g)]
ax.add_patch(FancyBboxPatch((58, y - 3.2), 40, 6.4, boxstyle="round,pad=0,rounding_size=0.8",
fc="white", ec=INK2, lw=0.8))
ax.text(60, y + 0.9, f"PGS-{g}, {c.split()[0].lower()}", fontsize=10, color=INK2, va="center")
ax.text(60, y - 1.5, f"OR {v[0]:.2f} [{v[3]:.2f}–{v[4]:.2f}]", fontsize=10.5, color=ORANGE, weight="bold", va="center")
for lab, t, y in left:
ax.add_patch(FancyBboxPatch((2, y - 3.2), 36, 6.4, boxstyle="round,pad=0,rounding_size=0.8",
fc="white", ec=INK2, lw=0.8))
ax.text(4, y + 0.9, lab, fontsize=10, color=INK2, va="center")
ax.text(4, y - 1.5, f"OR {fmt(t)}", fontsize=10.5, color=ORANGE, weight="bold", va="center")
# identical-value links
for yl, yr in ((40, 23), (28, 14), (16, 14)):
ax.annotate("", xy=(57.6, yr), xytext=(38.4, yl),
arrowprops=dict(arrowstyle="-|>", color=INK, lw=LW, shrinkA=0, shrinkB=0))
# rows with the same label
for yl, yr in ((40, 44), (28, 35)):
ax.annotate("", xy=(57.6, yr), xytext=(38.4, yl),
arrowprops=dict(arrowstyle="-", color=INK2, lw=1.0, ls=(0, (3, 2)), shrinkA=0, shrinkB=0))
h = [Line2D([], [], color=INK, lw=LW, label="identical OR and both CI bounds"),
Line2D([], [], color=INK2, lw=1.0, ls=(0, (3, 2)), label="same PGS and class label")]
ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.01), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_C")
# ------------------------------------------------------------------ D: forest of STable 2 with abstract overlay
def cand_D():
order = ["MDD", "ADHD", "ASD", "BD", "SCZ", "AN"]
classes = ["Prolonged initial contact", "Persistent contact"]
fig, axs = plt.subplots(1, 2, figsize=(W, 3.9), sharey=True)
for ax, c in zip(axs, classes):
log_axis(ax, 0.80, 1.26, ticks=(0.8, 0.9, 1.0, 1.1, 1.2))
for i, g in enumerate(order):
y = len(order) - i
o, _, p, lo, hi = ST2[(c, g)]
ax.plot([lo, hi], [y, y], color=ORANGE, lw=LW, solid_capstyle="round", zorder=2)
ax.plot(o, y, "o", ms=MS, color=ORANGE, zorder=3)
a = A_PRO if c.startswith("Prolonged") else A_PER
yA = len(order) - order.index("ADHD") + 0.38
ax.plot([a[1], a[2]], [yA, yA], color=ORANGE, lw=LW, ls=(0, (2, 1.5)), zorder=2)
ax.plot(a[0], yA, "D", ms=MS, mfc="white", mec=ORANGE, mew=LW, zorder=3)
yAN = len(order) - order.index("AN")
ax.annotate("", xy=(a[0], yAN + 0.22), xytext=(a[0], yA - 0.2),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.0, ls=(0, (3, 2))))
ax.set_title(c, fontsize=10.5, color=INK)
ax.set_xlabel("OR per SD vs brief contact (log)")
axs[0].set_yticks(range(len(order), 0, -1))
axs[0].set_yticklabels([f"PGS-{g}" for g in order], fontsize=10, color=INK)
axs[0].set_ylim(0.4, 6.9)
h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="STable 2 (printed)"),
Line2D([], [], color=ORANGE, marker="D", mfc="white", ms=MS, lw=LW, ls=(0, (2, 1.5)), mew=LW,
label="Abstract's PGS-ADHD value (printed)"),
Line2D([], [], color=INK2, lw=1.0, ls=(0, (3, 2)), label="lands on the PGS-AN row")]
fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.07), ncol=3, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.06, 1, 1))
save(fig, "cand_D")
# ------------------------------------------------------------------ E: p-value strip
def cand_E():
fig, ax = plt.subplots(figsize=(W, 3.2))
rows = [
("Prolonged, Abstract\n1.06 [1.01–1.11]", None, abs_calc["ADHD, prolonged initial contact"][2]),
("Prolonged, Results\n0.91 [0.87–0.96]", R_PRO[3], res_calc["ADHD, prolonged initial contact"][2]),
("Persistent, Abstract\n1.12 [1.03–1.21]", None, abs_calc["ADHD, persistent contact"][2]),
("Persistent, Results\n0.90 [0.83–0.97]", R_PER[3], res_calc["ADHD, persistent contact"][2]),
]
ax.set_xscale("log"); ax.set_xlim(5e-5, 0.2)
ax.set_xticks([1e-4, 1e-3, 1e-2, 1e-1]); ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.1"]); ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.6); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axvline(ALPHA_BONF, color=INK2, lw=0.8, ls=(0, (4, 2)))
ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (1, 2)))
ax.text(ALPHA_BONF / 1.08, 4.62, "Bonferroni α 0.0083", fontsize=10, color=INK2, va="top", ha="right")
ax.text(0.05 * 1.08, 4.62, "0.05", fontsize=10, color=INK2, va="top")
for i, (lab, pr, pc) in enumerate(rows):
y = len(rows) - i
if pr is not None:
ax.plot(pr, y, "o", ms=MS, color=ORANGE, zorder=3)
ax.plot(pc, y, "D", ms=MS, color=BLUE, zorder=4, mec="white", mew=1.0)
ax.text(max(pc, pr or 0) * 1.35, y - 0.05, f"{pc:.4f}", fontsize=10, va="center", color=INK)
ax.set_yticks(range(len(rows), 0, -1)); ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_ylim(0.4, 4.7)
ax.set_xlabel("Two-sided p for PGS-ADHD vs brief contact (log scale)")
h = [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=0, label="p printed in Results"),
Line2D([], [], color=BLUE, marker="D", ms=MS, lw=0, label="p recomputed from the printed CI")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2, frameon=False, fontsize=10)
save(fig, "cand_E")
# Which values each design draws, so the Figure 1 caption can name them correctly
CAPTION_KEY = {
"A": "All numbers are copied from the paper (orange: Abstract and Results text as printed); no recomputed values are drawn.",
"B": "Orange: odds ratios and 95% CIs as printed (Abstract; Results text and STable 2). Blue: reciprocals of the Results values, recomputed here to show what a swapped reference category would give.",
"C": "All numbers are copied from the paper (orange: Abstract and STable 2 as printed); connectors mark identical OR-CI triples and shared labels.",
"D": "All numbers are copied from the paper (orange: STable 2 rows, and the Abstract's PGS-ADHD values as open diamonds).",
"E": "Orange: p values as printed in the Results text (the Abstract prints none). Blue: two-sided p recomputed from each printed 95% CI.",
}
print(f"Figure 1 caption key for candidate {FIG1}: {CAPTION_KEY[FIG1]}")
for f in (cand_A, cand_B, cand_C, cand_D, cand_E):
f()
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"cand_{FIG1}.{ext}"), os.path.join(HERE, f"fig1.{ext}"))
print(f"figures written to {CAND}; fig1 = candidate {FIG1}")
Cohort 2 is reported as 361,258 participants in Table 2 but 32,220 in the flowchart
Psychol Med (2026) · doi:10.1017/S0033291726103328 · PMC12902173
Summary
The paper analyses two matched cohorts from Taiwan's one-million-person Longitudinal Health Insurance Database (LHID): psychological distress (PD, meaning depression or anxiety) followed for incident IBS (Cohort 1), and IBS followed for incident PD (Cohort 2). For Cohort 2, the Results text and Table 2 describe 361,258 participants (180,629 per arm), while the sample-selection flowchart (Figure 1) gives a final follow-up N of 32,220 (16,110 per arm), 11.2 times smaller. The paper's own follow-up event percentages (2,031 = 12.6% and 1,308 = 8.1%) fit only the flowchart's 16,110 per arm, and the width of the headline hazard-ratio interval fits the roughly 3,300 events in that smaller sample. Table 2 therefore cannot describe the cohort that was analysed. It also reports 45.77% of the IBS arm "With PD" at baseline in a cohort defined by excluding prior PD. Cohort 1 shows the same pattern (Table 1: 234,361 per arm; flowchart: 18,402 per arm).
What the paper reports
Results, "Baseline characteristics: Cohort 2 (IBS predicting psychological distress)":
Table 2 summarizes the baseline characteristics of 361,258 participants, divided evenly into IBS (n = 180,629) and non-IBS controls (n = 180,629).
Table 2, column headers and the first block (rows "Without PD" and "With PD"):
Variable | IBS(n = 180,629) | Without IBS(n = 180,629) | p-value
Without PD | 97,948 (54.23%) | 148,971 (82.47%) |
With PD | 82,681 (45.77%) | 31,658 (17.53%) |
Results, the same section, "Psychological distress outcomes":
The prevalence of psychological distress was substantially higher among individuals with IBS (45.77%) compared with controls (17.53%, p < .001).
Methods, "Cohort 2: IBS predicting psychological distress", on the definition of the cohort:
Those with any depressive or anxiety disorder before the index date were excluded to ensure incident outcome assessment.
A consolidated flowchart including all exclusions (prior IBS, prior psychological distress, and missing data) and final analytic sample sizes is shown in Figure 1.
Figure 1 flowchart, Cohort 2 (list items, rendered as text in PMC and in the publisher's HTML):
Newly diagnosed IBS cases (2002–2012): 16,110
Excluded prior depressive or anxiety disorders: 3,231 excluded
IBS cohort: 16,110
Matched controls: 16,110 Final follow-up N for PD outcomes: 32,220
Results, "Cohort 2: IBS predicting incident depression or anxiety":
In the reverse analysis, during a mean follow-up of 6.9 years, 2,031 IBS patients (12.6%) developed either depression or anxiety, compared to 1,308 (8.1%) in the control group.
Cohort 1 has the same kind of discrepancy. Results, "Baseline characteristics: Cohort 1":
A total of 468,722 participants were included, with 234,361 individuals in each group after matching.
Figure 1 flowchart, Cohort 1:
Newly diagnosed depressive/anxiety disorders (2002–2012): 18,402
Psychological distress group: 18,402
Matched controls: 18,402 Final follow-up N for IBS outcome: 36,804
Results, Cohort 1 follow-up, and the Cohort 1 baseline IBS prevalence:
During a mean follow-up period of 7.3 years, 1,498 individuals (8.1%) in the psychological distress group developed IBS, compared to 921 (5.0%) in the control group.
The prevalence of IBS at baseline was significantly higher in the psychological distress group compared to controls (32.11% vs. 12.04%, p < .001)
The Cohort 1 definition (Methods) and the Discussion:
Individuals with a prior diagnosis of IBS (ICD-9-CM: 564.1) before the index date were excluded to ensure incident outcome assessment.
Our findings apply only to incident cases, as individuals with any history of IBS or prior depressive/anxiety disorders were excluded from both cohorts.
The large sample enabled robust subgroup and interaction analyses.
What the paper's own numbers show
Method. I took every sample size, count and percentage for both cohorts from the Results text, the Figure 1 flowchart text, Tables 1–4 and the Methods description of the LHID. I checked (i) each stated total against twice the per-arm N; (ii) that each block of Table 2 sums to its column header; (iii) which denominators reproduce the printed follow-up event percentages, by finding every per-arm N for which the printed event count divided by N rounds to the printed percentage; (iv) the flowchart's own subtraction; (v) the number of events implied by the width of each overall adjusted hazard ratio's 95% CI. For (v) I used the standard approximation for a 1:1 comparison, var(log HR) ≈ 4/D, and also computed the interval expected from the reported event counts, SE ≈ √(1/d₁ + 1/d₂), and the interval expected if the Table N had produced events at the printed rates. Finally, I compared the cohort sizes with the size of the LHID. Patient-level data are not available, so the models cannot be refitted. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Cohort 2 total N | Results text; Table 2 headers | 361,258 (180,629 per arm) | 180,629 × 2 = 361,258, internally consistent |
| Cohort 2 final follow-up N | Figure 1 flowchart | 32,220 (16,110 per arm) | 16,110 × 2 = 32,220, internally consistent |
| Ratio of the two Cohort 2 totals | — | — | 361,258 / 32,220 = 11.21; difference 329,038 (164,519 per arm) |
| Table 2 block sums (PD status, sex, age group, urbanization, income) | Table 2 | n = 180,629 per column | every block sums to 180,629 in both columns |
| Depression + anxiety = "With PD" | Table 2 | 82,681 and 31,658 | 15,806 + 66,875 = 82,681; 7,339 + 24,319 = 31,658 |
| IBS-arm follow-up events | Results | 2,031 (12.6%) | 2,031 / 16,110 = 12.61%; 2,031 / 180,629 = 1.12% |
| Control-arm follow-up events | Results | 1,308 (8.1%) | 1,308 / 16,110 = 8.12%; 1,308 / 180,629 = 0.72% |
| Per-arm N that reproduces both printed percentages | derived | — | 16,056 to 16,183: contains 16,110, excludes 180,629 |
| IBS cohort after excluding prior PD | Figure 1 flowchart | 16,110 new cases, 3,231 excluded, cohort 16,110 | 16,110 − 3,231 = 12,879, not 16,110 |
| IBS arm "With PD" at baseline | Table 2; Results | 82,681 (45.77%) | 82,681 / 180,629 = 45.77%, in a cohort that excluded prior PD; 25.6 times the flowchart's 3,231 exclusions and 40.7 times the 2,031 follow-up events |
| Cohort 2 aHR 1.46 (1.37–1.56): events implied by CI width | Results; Table 4 | — | SE(log HR) = 0.0331; 4/SE² ≈ 3,644 events (3,288–4,061 allowing for rounding of the limits); reported events 2,031 + 1,308 = 3,339, inside that range |
| CI expected from 3,339 events | derived | 1.37–1.56 | 1.36–1.57 (matches) |
| CI expected if 180,629 per arm had the printed rates | derived | — | 22,759 + 14,631 = 37,390 events; CI 1.43–1.49 (does not match; the reported interval is 3.1 times as wide) |
| Cohort 1 total N | Results text; Table 1 headers | 468,722 (234,361 per arm) | 234,361 × 2 = 468,722 |
| Cohort 1 final follow-up N | Figure 1 flowchart | 36,804 (18,402 per arm) | 18,402 × 2 = 36,804; ratio 468,722 / 36,804 = 12.74 |
| Cohort 1 follow-up events | Results | 1,498 (8.1%) and 921 (5.0%) | /18,402 = 8.14% and 5.00%; /234,361 = 0.64% and 0.39%; N reproducing both: 18,381–18,606 |
| Cohort 1 aHR 1.55 (1.42–1.69): events implied by CI width | Results; Table 3 | — | ≈ 2,028 (1,885–2,188), slightly fewer than the 2,419 reported (1,498 + 921), i.e. the reported interval is a little wider than the events alone imply; both are far from the 30,701 implied by the Table N. CI from 2,419 events 1.43–1.68; from Table N at printed rates (30,701 events) 1.51–1.59, 3.8 times narrower than reported |
| PD arm with IBS at baseline | Table 1; Results | 75,244 (32.11%) | 75,244 / 234,361 = 32.11%, in a cohort that excluded prior IBS |
| Adults in the LHID | Methods | 1,000,000 − 1,982 − 2,614 | 995,404 |
| Cohort 1 eligible population | Figure 1 flowchart | "990,0xx" | 995,404 − 5,412 = 989,992, which is not of the form 990,0xx either |
| Exposed arms of both cohorts | Tables 1 and 2 | 234,361 + 180,629 | 414,990 = 41.7% of LHID adults |
| Exposed arms of both cohorts | Figure 1 flowchart | 18,402 + 16,110 | 34,512 = 3.5% of LHID adults |
The two exposed groups essentially cannot share people. Cohort 1 excludes anyone with IBS before the PD diagnosis, and Cohort 2 excludes anyone with PD before the IBS diagnosis, so each person can be exposed in at most one cohort (apart from anyone first diagnosed with both on the same date). Taken at face value, Tables 1 and 2 would mean that 41.7% of all adults in the database received a first diagnosis of PD or of IBS between 2002 and 2012, with the other condition absent beforehand. The flowchart's figures correspond to 3.5%.
Both totals for each cohort are internally consistent. Table 2's blocks all sum to 180,629, and the flowchart's numbers multiply out to 32,220. The two descriptions cannot both refer to the same matched sample. The follow-up percentages and the CI widths fit the flowchart, so the survival analyses appear to have been run on about 16,110 patients per arm in Cohort 2 (and about 18,402 in Cohort 1). It is not stated what population Tables 1 and 2 describe.
Figure
Figure 1. Participants per arm for each cohort on one linear axis starting at zero. Orange markers are printed in the paper: circles are the Table and Results-text values (234,361 for Cohort 1, 180,629 for Cohort 2), and squares are the Figure 1 flowchart values (18,402 and 16,110). The grey bar and the label on it give the ratio between the two printed values (12.7 and 11.2). The blue diamond marks the range of per-arm N (narrower than the marker at this scale) for which both printed follow-up percentages are reproduced from the printed event counts (Cohort 2: 2,031 at 12.6% and 1,308 at 8.1% give 16,056–16,183; Cohort 1: 1,498 at 8.1% and 921 at 5.0% give 18,381–18,606), computed by recompute.py.txt. In both cohorts this range contains the flowchart value and excludes the Table value.
Figure 2. Overall adjusted hazard ratios on one log-scale axis; the vertical line marks HR = 1. Orange circles are the reported aHRs and 95% CIs (Table 4: 1.46, 1.37–1.56; Table 3: 1.55, 1.42–1.69). Filled blue diamonds are the intervals expected from the reported numbers of events (3,339 and 2,419), with SE(log HR) = √(1/d₁ + 1/d₂); they agree closely with the reported intervals (for Cohort 1 the reported interval is slightly wider than 2,419 events alone imply, which corresponds to about 2,028 events). Open blue diamonds are the intervals expected if the Table sample sizes had produced events at the printed rates (37,390 and 30,701 events); they are about three to four times narrower than reported (3.1x for Cohort 2, 3.8x for Cohort 1, measured as SE(log HR) and identical for the linear width). The reported precision is what the flowchart-sized cohorts would give, not what the Table-sized cohorts would give.
Why it matters
Cohort 2 is one of the paper's two primary analyses. Its sample size is given in the Results text and in Table 2 as 361,258, about eleven times the 32,220 in the flowchart that the Methods identify as showing the "final analytic sample sizes". The follow-up events and the precision of the hazard ratio fit the smaller number, so the hazard ratios were most likely estimated on about 32,000 people. Their direction and statistical significance are probably not affected by the misstated N.
What is affected is the description of the study population. Table 2, the Results paragraphs that summarize it ("well matched", the income and comorbidity contrasts, the 45.77% vs 17.53% PD prevalence) and the corresponding Table 1 material for Cohort 1 cannot describe the analysed cohorts. They report N values 11–13 times too large and a large prevalence of the very condition that each cohort was defined to exclude: 82,681 IBS patients "With PD" in Cohort 2 and 75,244 PD patients "With IBS" in Cohort 1. A reader cannot use these tables to judge baseline balance, comorbidity burden or generalizability, and the Discussion's "The large sample enabled robust subgroup and interaction analyses" rests in part on the larger figures. Anyone extracting the sample size for a systematic review or meta-analysis would take 361,258 (or 468,722) from the text, overstating the size of these cohorts eleven- to thirteen-fold.
The flowchart has its own problem. It lists 16,110 newly diagnosed IBS cases, then 3,231 exclusions for prior PD, and then an IBS cohort of 16,110 again, where 12,879 would be expected. So it is not clear whether 16,110 is the number before or after the exclusion.
A separate inconsistency adds to the uncertainty. The abstract gives "IBS was associated with increased risks of depression (adjusted hazard ratio [aHR] = 1.55) and anxiety (aHR = 1.68)", but 1.55 and 1.68 are the overall and female HRs for the opposite direction (PD → IBS) in Table 3. The body reports no depression- or anxiety-specific HRs, and the abstract's 1.45 and 1.51 for "depression and anxiety ... developing IBS" do not appear as point estimates anywhere in the body. Table 4's overall HR for IBS → PD is 1.46.
Explanations considered
- Table 2 describes the population before exclusions or before matching. This does not account for the numbers. Table 2 shows identical sex, age, age-group and urbanization counts in both columns, each with p = 1.000, so it describes a 1:1 matched sample, the same stage as the flowchart's "After matching 1:1 with controls". The Results text also says the participants were "divided evenly", and the Cohort 1 text says "234,361 individuals in each group after matching".
- Table 2 comes from a larger database (for example the full NHIRD rather than the 1-million-person LHID). Nothing in the paper says so. The Methods state that the cohorts were drawn from the LHID, a one-million-person subset of the NHIRD (as does Supplementary Table S1), and no table note gives another source. Even so, the event percentages in the Results fit only the flowchart denominators, so the paper's own analyses use the smaller cohorts.
- The rows count visits, claims or person-years rather than people. This does not account for the numbers. The rows are counts of people by sex, age group, urbanization level and income, each block sums exactly to the column N, and the percentages are of that N.
- The follow-up percentages use a different denominator from Table 2 on purpose. Nothing in the paper says so. The text gives "2,031 IBS patients (12.6%)" with no other denominator, and 12.6% is 2,031/16,110. A different denominator would mean that the analysed cohort is not the one in Table 2, which is the inconsistency itself.
- The "With PD" row in Table 2 (and "With IBS" in Table 1) reports outcomes during follow-up, not baseline status. This does not resolve the problem either. The Results place these figures under "Baseline characteristics" and call the Table 1 figure "The prevalence of IBS at baseline". Read as outcomes, 82,681 would contradict the 2,031 events reported for the same arm (40.7 times larger).
- The flowchart is wrong and the Tables are right. This is not consistent with the rest of the paper. The event percentages and the width of the hazard-ratio intervals both fit the flowchart sizes. With 180,629 per arm and events at the printed rates, the interval would be about 1.43–1.49, not 1.37–1.56. The Table sizes would also mean that 41.7% of the database's adults were incident cases of one condition without the other.
- A conversion or typesetting artifact in PMC. This is excluded. The publisher's HTML on cambridge.org and the Europe PMC JATS XML carry the same numbers: 361,258 and 180,629 in the text and Table 2 headers, and 16,110 / 3,231 / 16,110 / 16,110 / 32,220 in the flowchart.
- The supplement clarifies the cohorts. It does not (see Status). It gives no cohort sizes, and the "complete flow diagram (Figure S1)" that the supplementary text refers to is a cumulative-incidence plot.
Status
- Version of record. The publisher's HTML article page (doi.org redirects to cambridge.org/core) carries the same Results text ("361,258 participants, divided evenly into IBS (n = 180,629) and non-IBS controls (n = 180,629)"), the same flowchart text (16,110 / 3,231 excluded / 16,110 / 16,110 / 32,220) and the same follow-up sentence (2,031, 12.6%; 1,308, 8.1%). On that page Table 2 is an image. The Europe PMC full-text XML has Table 2 headers "IBS(n = 180,629)" and "Without IBS(n = 180,629)" and the same flowchart items. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements. One supplementary file (doi:10.1017/S0033291726103328.sm001, a .docx) was downloaded and read. It holds Supplementary Table S1 (LHID sampling and attrition: 1,000,000 sampled, 1,982 excluded for missing demographics, 2,614 under 18), Tables S2–S3 (interaction aHRs, no Ns) and Figures S1–S5 (cumulative-incidence curves without numbers at risk, and forest plots). It gives no cohort sizes. Its text refers to a "complete flow diagram (Figure S1)", but Figure S1 is a cumulative-incidence plot. Data are available from the authors on request.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI. PMC and Europe PMC (PMID 41667938) list no correction or comment links. The Cambridge article page shows no correction notice, and the article page's metadata lists no corrections, retractions or addenda. A web search for the title with "correction", "erratum" or "PubPeer" returned nothing relevant. PubPeer could not be checked directly: its API returned HTTP 403 to automated access.
- Sources. PMC article PMC12902173; publisher page at cambridge.org (doi:10.1017/S0033291726103328); supplementary file S0033291726103328sup001.docx; Europe PMC full-text XML; Crossref; PubPeer (not reachable).
Suggested correction
- State which population Tables 1 and 2 describe. If the analysed cohorts are those in Figure 1 (18,402 and 16,110 per arm, as the event percentages and hazard-ratio precision indicate), replace Tables 1 and 2 and the Results paragraphs that summarize them with the baseline characteristics of those cohorts, and correct "468,722" / "234,361" and "361,258" / "180,629" in the Results text.
- Explain why Tables 1 and 2 show 75,244 PD patients with IBS and 82,681 IBS patients with PD at baseline in cohorts defined by excluding those conditions, or remove these rows if they do not describe baseline status.
- Reconcile the Cohort 2 flowchart: 16,110 newly diagnosed IBS cases minus 3,231 exclusions should not leave an IBS cohort of 16,110. Give the counts before and after exclusion, and give the Cohort 1 eligible population as a number instead of "990,0xx" (995,404 − 5,412 = 989,992, which does not fit "990,0xx" either).
- Correct the abstract's hazard ratios so that each one matches a reported estimate in the body for the stated direction and outcome (for IBS → PD, Table 4 gives 1.46 overall), or add the depression- and anxiety-specific estimates to the Results.
- Reword statements that rest on the larger sample sizes, such as "The large sample enabled robust subgroup and interaction analyses", if the analysed cohorts are the smaller ones.
Recomputation script (python recompute.py.txt)
"""Recompute the cohort sizes of PMC12902173
(Psychol Med 2026, doi:10.1017/S0033291726103328).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file, plus five design candidates in candidates/.
All inputs are copied from the article as archived in PubMed Central (the same numbers are
in the publisher's HTML and the Europe PMC XML): Methods (data source and the Figure 1
flowchart text), Results, the Abstract and Tables 1-4. No patient-level data are used.
"""
import math
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (from the paper)
LHID = 1_000_000 # Methods, data source
MISSING_DEMO = 1_982 # Methods / flowchart
UNDER_18 = 2_614 # Methods / flowchart
PRIOR_IBS_C1 = 5_412 # flowchart, Cohort 1
# Cohort 2: IBS -> psychological distress (PD)
C2_TABLE_ARM = 180_629 # Table 2 headers and Results text ("divided evenly")
C2_TEXT_TOTAL = 361_258 # Results text
C2_FC_NEW = 16_110 # flowchart: newly diagnosed IBS cases
C2_FC_EXCL = 3_231 # flowchart: excluded prior depressive or anxiety disorders
C2_FC_ARM = 16_110 # flowchart: IBS cohort / matched controls
C2_FC_TOTAL = 32_220 # flowchart: final follow-up N for PD outcomes
C2_EVENTS = (2_031, 1_308) # Results: IBS group, control group
C2_EVENT_PCT = (12.6, 8.1) # Results
C2_AHR = (1.46, 1.37, 1.56) # Results / Table 4, overall
TABLE2 = { # (IBS, without IBS) counts per block of Table 2
"Psychological distress (without PD, with PD)": ((97_948, 82_681), (148_971, 31_658)),
"Gender (male, female)": ((91_509, 89_120), (91_509, 89_120)),
"Age group (18-24, 25-44, 45-64, >=65)": ((17_211, 67_837, 66_504, 29_077),) * 2,
"Urbanization (levels 1-4)": ((55_995, 68_639, 28_900, 27_095),) * 2,
"Monthly income (3 bands)": ((43_194, 85_106, 52_329), (99_860, 55_902, 24_867)),
}
T2_WITH_PD = (82_681, 31_658)
T2_PD_PCT = (45.77, 17.53)
T2_DEP_ANX = ((15_806, 66_875), (7_339, 24_319))
# Cohort 1: PD -> IBS
C1_TABLE_ARM = 234_361 # Table 1 headers and Results text
C1_TEXT_TOTAL = 468_722
C1_FC_ARM = 18_402 # flowchart
C1_FC_TOTAL = 36_804
C1_EVENTS = (1_498, 921)
C1_EVENT_PCT = (8.1, 5.0)
C1_AHR = (1.55, 1.42, 1.69) # Results / Table 3, overall
T1_WITH_IBS = (75_244, 28_210)
T1_IBS_PCT = (32.11, 12.04)
TABLE3 = {"Overall": 1.55, "Female": 1.68, "Male": 1.39}
TABLE4 = {"Overall": 1.46, "Female": 1.52, "Male": 1.38}
ABSTRACT_HR = {"IBS -> depression": 1.55, "IBS -> anxiety": 1.68,
"depression -> IBS": 1.45, "anxiety -> IBS": 1.51}
Z = 1.959964
def pct(a, b, d=2):
return round(100 * a / b, d)
def n_range(events, pct_printed, dec=1):
"""Denominators N for which 100*events/N rounds to pct_printed at `dec` decimals."""
h = 0.5 * 10 ** (-dec)
return math.floor(events / ((pct_printed + h) / 100)) + 1, math.floor(events / ((pct_printed - h) / 100))
def se_from_ci(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z)
# ------------------------------------------------------------------ 1. Cohort 2 sizes
print("1. Cohort 2 (IBS -> psychological distress)")
print(f" Table 2 / text: {C2_TABLE_ARM:,} x 2 = {2 * C2_TABLE_ARM:,} (text says {C2_TEXT_TOTAL:,})")
print(f" Flowchart: {C2_FC_ARM:,} x 2 = {2 * C2_FC_ARM:,} (flowchart says {C2_FC_TOTAL:,})")
c2_ratio = C2_TEXT_TOTAL / C2_FC_TOTAL
print(f" Ratio text/flowchart = {c2_ratio:.2f}; difference = {C2_TEXT_TOTAL - C2_FC_TOTAL:,} "
f"({C2_TABLE_ARM - C2_FC_ARM:,} per arm)")
c2_after_excl = C2_FC_NEW - C2_FC_EXCL
print(f" Flowchart internal: {C2_FC_NEW:,} new IBS - {C2_FC_EXCL:,} excluded = {c2_after_excl:,}, "
f"but IBS cohort printed as {C2_FC_ARM:,}")
print(" Table 2 block sums (IBS column, control column):")
for k, (a, b) in TABLE2.items():
print(f" {k}: {sum(a):,}, {sum(b):,}")
print(f" With PD: {T2_WITH_PD[0]:,}/{C2_TABLE_ARM:,} = {pct(T2_WITH_PD[0], C2_TABLE_ARM)}%, "
f"{T2_WITH_PD[1]:,}/{C2_TABLE_ARM:,} = {pct(T2_WITH_PD[1], C2_TABLE_ARM)}% "
f"(printed {T2_PD_PCT[0]}% and {T2_PD_PCT[1]}%)")
for lab, (d, a), wp in zip(("IBS", "control"), T2_DEP_ANX, T2_WITH_PD):
print(f" {lab}: depression + anxiety = {d:,} + {a:,} = {d + a:,} (= 'With PD' {wp:,}); "
f"depression share {pct(d, wp)}%")
print(" Follow-up events:")
c2_pct_fc, c2_pct_tab = [], []
for lab, e, rp in zip(("IBS", "control"), C2_EVENTS, C2_EVENT_PCT):
a, b = 100 * e / C2_FC_ARM, 100 * e / C2_TABLE_ARM
c2_pct_fc.append(a); c2_pct_tab.append(b)
lo, hi = n_range(e, rp)
print(f" {lab}: {e:,}/{C2_FC_ARM:,} = {a:.2f}%; {e:,}/{C2_TABLE_ARM:,} = {b:.2f}% "
f"(printed {rp}%); N giving {rp}% after rounding: {lo:,}-{hi:,}")
c2_lo = max(n_range(e, p)[0] for e, p in zip(C2_EVENTS, C2_EVENT_PCT))
c2_hi = min(n_range(e, p)[1] for e, p in zip(C2_EVENTS, C2_EVENT_PCT))
print(f" Both percentages hold only for N per arm in {c2_lo:,}-{c2_hi:,}: contains "
f"{C2_FC_ARM:,}, excludes {C2_TABLE_ARM:,}")
print(f" Prevalent PD at baseline in a cohort that excluded prior PD: {T2_WITH_PD[0]:,} "
f"vs {C2_FC_EXCL:,} excluded in the flowchart ({T2_WITH_PD[0] / C2_FC_EXCL:.1f}x); "
f"vs {C2_EVENTS[0]:,} follow-up events ({T2_WITH_PD[0] / C2_EVENTS[0]:.1f}x)")
# ------------------------------------------------------------------ 2. Cohort 1 sizes
print("\n2. Cohort 1 (psychological distress -> IBS)")
print(f" Table 1 / text: {C1_TABLE_ARM:,} x 2 = {2 * C1_TABLE_ARM:,} (text says {C1_TEXT_TOTAL:,})")
print(f" Flowchart: {C1_FC_ARM:,} x 2 = {2 * C1_FC_ARM:,} (flowchart says {C1_FC_TOTAL:,})")
c1_ratio = C1_TEXT_TOTAL / C1_FC_TOTAL
print(f" Ratio text/flowchart = {c1_ratio:.2f}")
c1_pct_fc, c1_pct_tab = [], []
for lab, e, rp in zip(("PD", "control"), C1_EVENTS, C1_EVENT_PCT):
a, b = 100 * e / C1_FC_ARM, 100 * e / C1_TABLE_ARM
c1_pct_fc.append(a); c1_pct_tab.append(b)
lo, hi = n_range(e, rp)
print(f" {lab}: {e:,}/{C1_FC_ARM:,} = {a:.2f}%; {e:,}/{C1_TABLE_ARM:,} = {b:.2f}% "
f"(printed {rp}%); N giving {rp}% after rounding: {lo:,}-{hi:,}")
c1_lo = max(n_range(e, p)[0] for e, p in zip(C1_EVENTS, C1_EVENT_PCT))
c1_hi = min(n_range(e, p)[1] for e, p in zip(C1_EVENTS, C1_EVENT_PCT))
print(f" Both percentages hold only for N per arm in {c1_lo:,}-{c1_hi:,}: contains "
f"{C1_FC_ARM:,}, excludes {C1_TABLE_ARM:,}")
print(f" With IBS at baseline (Table 1): {T1_WITH_IBS[0]:,}/{C1_TABLE_ARM:,} = "
f"{pct(T1_WITH_IBS[0], C1_TABLE_ARM)}%, {T1_WITH_IBS[1]:,}/{C1_TABLE_ARM:,} = "
f"{pct(T1_WITH_IBS[1], C1_TABLE_ARM)}% (printed {T1_IBS_PCT[0]}%, {T1_IBS_PCT[1]}%), in a "
f"cohort that excluded prior IBS (flowchart: {PRIOR_IBS_C1:,} excluded)")
# ------------------------------------------------------------------ 3. source population
print("\n3. Source population (LHID)")
adults = LHID - MISSING_DEMO - UNDER_18
elig_c1 = adults - PRIOR_IBS_C1
print(f" {LHID:,} - {MISSING_DEMO:,} - {UNDER_18:,} = {adults:,} adults with complete data")
print(f" minus {PRIOR_IBS_C1:,} prior IBS = {elig_c1:,} (flowchart prints '990,0xx'; "
f"{elig_c1:,} is {'' if 990_000 <= elig_c1 < 990_100 else 'not '}of that form)")
exp_tab = C1_TABLE_ARM + C2_TABLE_ARM
exp_fc = C1_FC_ARM + C2_FC_ARM
print(f" Exposed arms, Tables 1+2: {C1_TABLE_ARM:,} + {C2_TABLE_ARM:,} = {exp_tab:,} = "
f"{pct(exp_tab, adults, 1)}% of adults")
print(f" Exposed arms, flowchart: {C1_FC_ARM:,} + {C2_FC_ARM:,} = {exp_fc:,} = "
f"{pct(exp_fc, adults, 1)}% of adults")
all_tab = C1_TEXT_TOTAL + C2_TEXT_TOTAL
print(f" All four Table arms: {C1_TEXT_TOTAL:,} + {C2_TEXT_TOTAL:,} = {all_tab:,} = "
f"{pct(all_tab, adults, 1)}% of adults (controls may overlap between cohorts)")
print(" The two exposed groups essentially cannot overlap: Cohort 1 excludes prior IBS and"
" Cohort 2 excludes prior PD, so a person is exposed in at most one of them (apart from"
" anyone first diagnosed with both on the same date).")
# ------------------------------------------------------------------ 4. CI width vs events
print("\n4. Number of events implied by the width of the overall aHR interval")
print(" Schoenfeld approximation for 1:1 allocation: var(log HR) = 4 / D")
ci_rows = []
for lab, (hr, lo, hi), ev, ev_pct, arm_tab in (
("Cohort 2 (Table 4)", C2_AHR, C2_EVENTS, C2_EVENT_PCT, C2_TABLE_ARM),
("Cohort 1 (Table 3)", C1_AHR, C1_EVENTS, C1_EVENT_PCT, C1_TABLE_ARM)):
se = se_from_ci(lo, hi)
se_min = se_from_ci(lo - 0.005, hi + 0.005)
se_max = se_from_ci(lo + 0.005, hi - 0.005)
d_impl = 4 / se ** 2
d_rep = sum(ev)
se_rep = math.sqrt(1 / ev[0] + 1 / ev[1])
ev_big = [p / 100 * arm_tab for p in ev_pct]
se_big = math.sqrt(1 / ev_big[0] + 1 / ev_big[1])
ci_rep = (hr * math.exp(-Z * se_rep), hr * math.exp(Z * se_rep))
ci_big = (hr * math.exp(-Z * se_big), hr * math.exp(Z * se_big))
print(f" {lab}: aHR {hr} ({lo}-{hi}); SE(log HR) = {se:.4f} "
f"(rounding range {se_max:.4f}-{se_min:.4f})")
print(f" implied events 4/SE^2 = {d_impl:,.0f} (range {4 / se_min ** 2:,.0f}-"
f"{4 / se_max ** 2:,.0f}); reported events {ev[0]:,} + {ev[1]:,} = {d_rep:,}")
print(f" 95% CI expected from the reported events (SE = sqrt(1/d1 + 1/d2) = {se_rep:.4f}): "
f"{ci_rep[0]:.2f}-{ci_rep[1]:.2f}")
print(f" if {arm_tab:,} per arm had the printed event rates: events "
f"{ev_big[0]:,.0f} + {ev_big[1]:,.0f} = {sum(ev_big):,.0f}; SE {se_big:.4f}; "
f"95% CI {ci_big[0]:.2f}-{ci_big[1]:.2f}")
d_lo, d_hi = 4 / se_min ** 2, 4 / se_max ** 2
inside = d_lo <= d_rep <= d_hi
print(f" reported events {d_rep:,} {'lie inside' if inside else 'lie outside'} the implied "
f"range {d_lo:,.0f}-{d_hi:,.0f}")
w_rep = hi - lo
w_big = ci_big[1] - ci_big[0]
print(f" reported interval vs interval expected from Table N: SE {se:.4f} / {se_big:.4f} "
f"= {se / se_big:.2f}x; linear width {w_rep:.2f} / {w_big:.4f} = {w_rep / w_big:.2f}x")
ci_rows.append((lab, hr, (lo, hi), ci_rep, ci_big, d_rep, sum(ev_big)))
# ------------------------------------------------------------------ 5. abstract HRs
print("\n5. Abstract hazard ratios vs Tables 3 and 4")
for k, v in ABSTRACT_HR.items():
t3 = [s for s, x in TABLE3.items() if x == v]
t4 = [s for s, x in TABLE4.items() if x == v]
print(f" Abstract {k} = {v}: Table 3 (PD -> IBS) rows {t3 or 'none'}; "
f"Table 4 (IBS -> PD) rows {t4 or 'none'}")
print(f" Table 4 overall (IBS -> PD) = {TABLE4['Overall']}; Table 3 overall (PD -> IBS) = "
f"{TABLE3['Overall']}. No depression- or anxiety-specific HRs are reported in the body.")
# ================================================================== figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e3e2de"
W = 7.2
LW = 1.5 # points; 1.5 pt = 2 px at 96 dpi
plt.rcParams.update({
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"font.size": 10, "axes.edgecolor": INK2, "axes.labelcolor": INK, "text.color": INK,
"xtick.color": INK2, "ytick.color": INK2, "axes.spines.top": False,
"axes.spines.right": False, "font.family": "DejaVu Sans", "svg.fonttype": "none",
})
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def wrap(s, w):
return "\n".join(textwrap.wrap(s, w))
def fmt(n):
return f"{n:,.0f}"
# ---------------------------------------------------------------- candidate A: N per arm dot plot
def cand_A():
"""Per-arm N: Tables/text vs flowchart (both printed), with the N range implied by the
printed event counts and percentages (recomputed)."""
fig, ax = plt.subplots(figsize=(W, 3.3))
rows = [("Cohort 1\nPD \u2192 IBS", C1_TABLE_ARM, C1_FC_ARM, (c1_lo, c1_hi), C1_EVENTS,
C1_EVENT_PCT, "Table 1"),
("Cohort 2\nIBS \u2192 PD", C2_TABLE_ARM, C2_FC_ARM, (c2_lo, c2_hi), C2_EVENTS,
C2_EVENT_PCT, "Table 2")]
ys = [1, 0]
for y, (lab, tab, fc, (lo, hi), ev, ep, tname) in zip(ys, rows):
ax.plot([fc, tab], [y, y], color=GRID, lw=4, zorder=1, solid_capstyle="round")
ax.scatter(tab, y, s=150, color=ORANGE, zorder=3, edgecolor="white", lw=0.8)
ax.scatter(fc, y, s=150, color=ORANGE, marker="s", zorder=3, edgecolor="white", lw=0.8)
ax.plot([lo, hi], [y - 0.2, y - 0.2], color=BLUE, lw=LW * 2, zorder=2)
ax.scatter((lo + hi) / 2, y - 0.2, s=70, color=BLUE, marker="D", zorder=3,
edgecolor="white", lw=0.6)
ax.text(tab, y + 0.17, f"{tname} / text: {fmt(tab)}", ha="center", va="bottom",
color=ORANGE, fontsize=10)
ax.text(fc, y + 0.17, f"Figure 1 flowchart: {fmt(fc)}", ha="left", va="bottom",
color=ORANGE, fontsize=10)
ax.text(hi + 4000, y - 0.2,
f"N implied by {fmt(ev[0])} ({ep[0]}%) and {fmt(ev[1])} ({ep[1]}%) events: "
f"{fmt(lo)}\u2013{fmt(hi)}", ha="left", va="center", color=BLUE, fontsize=10)
ax.text((fc + tab) / 2, y, f"\u00d7{tab / fc:.1f}", ha="center", va="center",
color=INK2, fontsize=10, zorder=4,
bbox=dict(boxstyle="square,pad=0.25", fc=BG, ec="none"))
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.55, 1.55); ax.set_xlim(0, 255_000)
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{v / 1000:.0f}k"))
ax.set_xlabel("Participants per arm (exposed group; controls are matched 1:1)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.spines["left"].set_visible(False); ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="reported: Table / Results text"),
Line2D([], [], marker="s", ls="", color=ORANGE, ms=9, label="reported: flowchart"),
Line2D([], [], marker="D", ls="", color=BLUE, ms=7,
label="recomputed: N consistent with the printed event %")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.25), ncol=2,
frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- candidate B: text panels
def cand_B():
fig = plt.figure(figsize=(W, 5.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 80); ax.axis("off")
def panel(x, y, w, h, title, lines):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=ORANGE, lw=LW))
ax.text(x + 1.5, y + h - 1.5, title, fontsize=11, fontweight="bold", va="top")
ax.text(x + 1.5, y + h - 6.5, lines, fontsize=10, va="top", linespacing=1.4)
panel(2, 30, 43, 46, "Results text and Table 2",
wrap("\u201cTable 2 summarizes the baseline characteristics of 361,258 participants, "
"divided evenly into IBS (n = 180,629) and non-IBS controls (n = 180,629).\u201d", 40)
+ "\n\nTable 2 column headers:\n\u201cIBS(n = 180,629)\u201d\n\u201cWithout IBS(n = 180,629)\u201d"
+ "\n\nTable 2, \u201cWith PD\u201d:\n\u201c82,681 (45.77%) | 31,658 (17.53%)\u201d")
panel(55, 30, 43, 46, "Figure 1 flowchart, Cohort 2",
"\u201cNewly diagnosed IBS cases\n(2002\u20132012): 16,110\u201d\n"
"\u201cExcluded prior depressive or\nanxiety disorders: 3,231 excluded\u201d\n"
"\u201cIBS cohort: 16,110\u201d\n\u201cMatched controls: 16,110\u201d\n"
"\u201cFinal follow-up N for PD\noutcomes: 32,220\u201d\n\nResults, follow-up:\n"
+ wrap("\u201c2,031 IBS patients (12.6%) \u2026 compared to 1,308 (8.1%)\u201d", 40))
ax.annotate("", xy=(54.2, 53), xytext=(45.8, 53),
arrowprops=dict(arrowstyle="<->", color=BLUE, lw=LW))
ax.text(50, 56, "\u00d711.2", ha="center", color=BLUE, fontsize=11, fontweight="bold")
ax.text(2, 23, "Recomputed from the paper\u2019s own numbers", fontsize=10.5,
fontweight="bold", color=BLUE)
notes = [f"180,629 \u00d7 2 = {2 * C2_TABLE_ARM:,}, but 16,110 \u00d7 2 = {2 * C2_FC_ARM:,}"
f" (ratio {c2_ratio:.2f})",
f"2,031 / 16,110 = {c2_pct_fc[0]:.1f}% and 1,308 / 16,110 = {c2_pct_fc[1]:.1f}%"
f" (as printed); over 180,629: {c2_pct_tab[0]:.2f}% and {c2_pct_tab[1]:.2f}%",
f"16,110 \u2212 3,231 = {c2_after_excl:,}, not 16,110",
f"82,681 IBS patients \u201cWith PD\u201d in a cohort that excluded prior PD"]
for i, s in enumerate(notes):
ax.text(3, 18 - i * 4.4, "\u2022 " + s, fontsize=10, color=BLUE, va="center")
return fig
# ---------------------------------------------------------------- candidate C: calculation flow
def cand_C():
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")
def box(x, y, w, h, s, ec, fs=10, bold=False):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1",
fc="white", ec=ec, lw=LW))
ax.text(x + w / 2, y + h / 2, s, ha="center", va="center", fontsize=fs,
fontweight="bold" if bold else "normal")
def arrow(x0, y0, x1, y1, c=INK2):
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=c, lw=LW))
ax.text(1, 61, "Cohort 2 as printed in the Methods and the Figure 1 flowchart", fontsize=10.5,
fontweight="bold")
box(1, 44, 21, 12, "Methods: LHID\n1,000,000 \u2212 1,982\n\u2212 2,614", ORANGE)
box(27, 44, 20, 12, "new IBS cases\n2002\u20132012\n16,110", ORANGE)
box(52, 44, 20, 12, "excluded prior\nPD: 3,231\nIBS cohort: 16,110", ORANGE)
box(77, 44, 22, 12, "matched 1:1\n16,110 + 16,110\n= 32,220", ORANGE)
arrow(22.6, 50, 26.4, 50); arrow(47.6, 50, 51.4, 50); arrow(72.6, 50, 76.4, 50)
ax.text(24, 38.5, "Recomputed from the paper\u2019s own numbers", fontsize=10.5,
fontweight="bold", color=BLUE, va="center")
box(1, 22, 21, 11, f"{adults:,}\nadults", BLUE)
box(52, 22, 20, 11, f"16,110 \u2212 3,231\n= {c2_after_excl:,}", BLUE)
box(77, 22, 22, 11, f"2,031 (12.6%), 1,308\n(8.1%) events imply\nN = {c2_lo:,}\u2013{c2_hi:,}", BLUE)
ax.plot([11.5, 11.5], [43.4, 33.6], color=BLUE, lw=LW, ls=(0, (3, 2)))
ax.plot([62, 62], [43.4, 41.0], color=BLUE, lw=LW, ls=(0, (3, 2)))
ax.plot([62, 62], [36.0, 33.6], color=BLUE, lw=LW, ls=(0, (3, 2)))
ax.plot([88, 88], [43.4, 33.6], color=BLUE, lw=LW, ls=(0, (3, 2)))
ax.text(1, 15, "Cohort 2 as printed in the Results text and Table 2", fontsize=10.5,
fontweight="bold")
box(52, 1, 47, 11, f"180,629 + 180,629 = 361,258\n({c2_ratio:.1f} times the flowchart\u2019s 32,220)",
ORANGE, bold=True)
box(1, 1, 46, 11, "82,681 (45.77%) of the IBS arm \u201cWith PD\u201d\nalthough prior PD was "
"an exclusion", ORANGE)
return fig
# ---------------------------------------------------------------- candidate D: event % dot plot
def cand_D():
fig, ax = plt.subplots(figsize=(W, 3.5))
rows = [("Cohort 1, PD arm\n1,498 events", C1_EVENT_PCT[0], c1_pct_fc[0], c1_pct_tab[0]),
("Cohort 1, controls\n921 events", C1_EVENT_PCT[1], c1_pct_fc[1], c1_pct_tab[1]),
("Cohort 2, IBS arm\n2,031 events", C2_EVENT_PCT[0], c2_pct_fc[0], c2_pct_tab[0]),
("Cohort 2, controls\n1,308 events", C2_EVENT_PCT[1], c2_pct_fc[1], c2_pct_tab[1])]
ys = [3, 2, 1, 0]
for y, (lab, rep, fc, tab) in zip(ys, rows):
ax.plot([tab, rep], [y, y], color=GRID, lw=4, zorder=1)
ax.scatter(rep, y, s=170, color=ORANGE, zorder=2, edgecolor="white", lw=0.8)
ax.scatter(fc, y, s=60, color=BLUE, marker="D", zorder=3, edgecolor="white", lw=0.6)
ax.scatter(tab, y, s=110, facecolor=BG, edgecolor=BLUE, marker="D", lw=LW, zorder=3)
ax.text(rep + 0.3, y, f"{rep}%", va="center", color=ORANGE, fontsize=10)
ax.text(tab + 0.3, y + 0.22, f"{tab:.2f}%", va="bottom", ha="left", color=BLUE,
fontsize=10)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 14.5); ax.set_ylim(-0.6, 3.6)
ax.set_xlabel("Share of the arm with an outcome event during follow-up (%)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.spines["left"].set_visible(False); ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", ls="", color=ORANGE, ms=10, label="reported %"),
Line2D([], [], marker="D", ls="", color=BLUE, ms=7,
label="recomputed: events / flowchart N"),
Line2D([], [], marker="D", ls="", mfc=BG, mec=BLUE, mew=LW, ms=9,
label="recomputed: events / Table N")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=3,
frameon=False, fontsize=10)
return fig
# ---------------------------------------------------------------- candidate E: CI width check
def cand_E():
fig, ax = plt.subplots(figsize=(3.4, 3.8))
y = 0
yt, yl = [], []
for lab, hr, rep_ci, ci_rep, ci_big, d_rep, d_big in ci_rows:
tname = "Table 2" if "Cohort 2" in lab else "Table 1"
items = [("reported aHR, 95% CI", rep_ci, ORANGE, "o", 0.28),
(f"recomputed: {d_rep:,} reported events", ci_rep, BLUE, "D", 0.0),
(f"recomputed: {tname} N at printed rates\n({d_big:,.0f} events)",
ci_big, BLUE, "D", -0.28)]
for txt, (lo, hi), c, m, dy in items:
mfc = c if dy >= 0 else BG
ax.plot([lo, hi], [y + dy, y + dy], color=c, lw=LW)
ax.plot([hr], [y + dy], marker=m, color=c, mfc=mfc, mec=c, mew=LW, ms=8, ls="")
ax.text(1.02, y + dy, f"{lo:.2f}\u2013{hi:.2f} {txt}", va="center", fontsize=10,
color=c, transform=ax.get_yaxis_transform(), clip_on=False)
yt.append(y); yl.append(lab.replace(" (", "\n("))
y -= 1.2
ax.axvline(1, color=INK2, lw=0.8)
ax.set_xscale("log")
ax.set_xticks([1, 1.2, 1.4, 1.6, 1.8])
ax.get_xaxis().set_major_formatter(matplotlib.ticker.FormatStrFormatter("%.1f"))
ax.get_xaxis().set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_xlim(0.95, 1.8)
ax.minorticks_off()
ax.set_yticks(yt); ax.set_yticklabels(yl, fontsize=10)
ax.set_ylim(y + 0.75, 0.55)
ax.set_xlabel("Overall adjusted hazard ratio\n(log scale; line at 1 = no association)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.spines["left"].set_visible(False); ax.tick_params(axis="y", length=0)
return fig
for name, fn in (("A_n_per_arm", cand_A), ("B_text_panels", cand_B),
("C_calc_flow", cand_C), ("D_event_pct", cand_D), ("E_ci_width", cand_E)):
save(fn(), os.path.join(CAND, name))
save(cand_A(), os.path.join(HERE, "fig1"))
save(cand_E(), os.path.join(HERE, "fig2"))
print("\nFigures written: fig1 (= candidate A), fig2 (= candidate E), candidates/A-E")
Abstract says mediators explain 37% of the association; the paper's coefficients give 23%
Psychol Med (2026) · doi:10.1017/S0033291725102936 · PMC13092991
Summary
The abstract states that childhood internalizing symptoms, externalizing behaviors and peer problems account for "37% of the total association" between prenatal tobacco exposure and adolescent suicide attempt, split as 16%, 12% and 9%. The paper defines the proportion mediated as the indirect association divided by the total association, β = 0.731. The paper's own coefficients give an internalizing share of 1.6% (0.012 / 0.731), not 16%, and a combined share of 23.1% (0.169 / 0.731). The Results state 23% for the combined share, and the Discussion says "more than one-fifth". The abstract's 37% is the sum 16 + 12 + 9, so it carries the ten-fold internalizing figure. The abstract's headline result therefore overstates the mediated share by about 14 percentage points. It also presents a non-significant pathway (OR 1.01, 95% CI 0.99 to 1.03) as the largest of the three mediators.
What the paper reports
Abstract, Results:
In models adjusted for key covariates, we found a significant association between prenatal tobacco exposure and increased risk of adolescent suicide attempts (odds ratio = 2.08, 95% confidence interval = [1.68, 2.56]), partly mediated through internalizing problems, externalizing behaviors, and peer problems from ages 3 to 14 years (accounting for 37% of the total association, that is, 16%, 12%, and 9%, respectively).
Methods, "Mediation analysis", which defines the quantities:
Indirect associations were calculated using the product of the coefficient approach (i.e. the product of the coefficient for the association between exposure and mediator, and between mediator and outcome).
The remaining unexplained part was represented by the direct association (total association = indirect + direct associations). The proportion of the association mediated was also calculated as the ratio between the indirect association and total association, to better understand the relative importance of each mediator.
Results, "Association between maternal smoking during pregnancy and childhood psychosocial difficulties" (the exposure-to-mediator paths):
Children whose mothers smoked during pregnancy were more likely to exhibit internalizing symptoms (β = 0.205, SE = 0.031, p < 0.001), externalizing behaviors (β = 0.477, SE = 0.031, p < 0.001), and peer problems (β = 0.279, SE = 0.031, p < 0.001) across their childhood.
Results, "Association between childhood psychosocial difficulties and adolescent suicide attempt" (the mediator-to-outcome paths):
We found that externalizing behaviors (β = 0.188, SE = 0.051, p < 0.001; OR = 1.21, 95% CI = [1.09, 1.33]) and peer problems (β = 0.241, SE = 0.052, p < 0.001; OR = 1.27, 95% CI = [1.15, 1.41]) from ages 3 to 14 increased the likelihood of later suicide attempt. Internalizing symptoms were not significantly associated with increased likelihood of suicide attempt (β = 0.057, SE = 0.052, p = 0.278; OR = 1.06, 95% CI = [0.96, 1.17]).
Results, "Pathways of association between maternal smoking during pregnancy and adolescent suicide attempt":
Structural equation models demonstrated that prenatal smoking was significantly associated with an increased risk of suicide attempt in adolescence (total association: β = 0.731, SE = 0.107, OR = 2.08, 95% CI = [1.68, 2.56]) after adjusting for the selected covariates.
Mediation analysis showed that 12.3% of the association was explained by externalizing behaviors (indirect association: β = 0.90, SE = 0.025, OR = 1.09, 95% CI = [1.04, 1.15]), and 9.2% by peer problems (indirect association: β = 0.057, SE = 0.025, OR = 1.07, 95% CI = [1.03, 1.11]). Internalizing symptoms explained 16% of the association, although the indirect association was not statistically significant (β = 0.012, SE = 0.011, OR = 1.01, 95% CI = [0.99, 1.03]) (Figure 2).
The proportion of the total effect mediated through all three mediators was estimated at 23% (overall indirect effect: β = 0.169, SE = 0.023, OR = 1.18, 95% CI = [1.13, 1.24]), suggesting that ~23% of the association between prenatal smoking and adolescent suicide attempts is attributed to these mediators. The direct association of prenatal tobacco exposure on suicide attempts remained significant (β = 0.562, SE = 0.107, OR = 1.75, 95% CI = [1.42, 2.16]).
Discussion, fourth paragraph:
Externalizing behaviors and peer problems accounted for 12.3% and 9.2% of the association between prenatal smoking exposure and suicide attempt, respectively. This indicates that psychosocial difficulties collectively mediated more than one-fifth of this association.
Table 2, whole sample, adjusted columns (OR, 95% CI): total association 2.08 (1.68–2.56); direct 1.75 (1.42–2.16); indirect via internalizing symptoms 1.01 (0.99–1.03); via externalizing behaviors 1.09 (1.04–1.15); via peer problems 1.07 (1.03–1.11); total indirect 1.18 (1.13–1.24).
What the paper's own numbers show
Method. The Methods give three rules, and the paper prints every input they need. (1) Each indirect association is the product a × b of the exposure-to-mediator coefficient and the mediator-to-outcome coefficient. (2) Total = direct + indirect. (3) The proportion mediated is indirect / total. All coefficients are on the log-odds scale: exp(β) reproduces each printed OR. I applied the three rules to the printed coefficients. As independent checks I also took the log-scale centre of each printed 95% CI and the log of each printed OR. No raw data are available, so the model was not refitted. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Total association, β (OR) | Results; Table 2 | 0.731 (2.08) | exp(0.731) = 2.077, consistent |
| Direct association, β (OR) | Results; Table 2 | 0.562 (1.75) | exp(0.562) = 1.754, consistent |
| Overall indirect, β (OR) | Results; Table 2 | 0.169 (1.18) | 0.731 − 0.562 = 0.169; sum of a × b = 0.1686; CI centre 0.1687 |
| Internalizing indirect, β | Results | 0.012 (OR 1.01, CI 0.99–1.03) | 0.205 × 0.057 = 0.0117; CI centre 0.0098; consistent |
| Externalizing indirect, β | Results | 0.90 (OR 1.09, CI 1.04–1.15) | 0.477 × 0.188 = 0.0897; CI centre 0.0895. exp(0.90) = 2.46, not 1.09, so 0.90 is a misprint of 0.090 |
| Peer problems indirect, β | Results | 0.057 (OR 1.07, CI 1.03–1.11) | 0.279 × 0.241 = 0.0672; CI centre 0.0670; exp(0.057) = 1.059, not 1.07, so 0.057 is a misprint of about 0.067 |
| Share mediated, internalizing | Results; Abstract | 16% | 0.0117 / 0.731 = 1.6% (from printed 0.012: 1.64%) |
| Share mediated, externalizing | Results; Abstract | 12.3%; 12% | 0.0897 / 0.731 = 12.3%, consistent |
| Share mediated, peer problems | Results; Abstract | 9.2%; 9% | 0.0672 / 0.731 = 9.2%, consistent |
| Share mediated, all three | Results; Discussion | 23%; "more than one-fifth" | 0.169 / 0.731 = 23.1%; 1.6 + 12.3 + 9.2 = 23.1%, consistent |
| Share mediated, all three | Abstract | 37% (16 + 12 + 9) | 23.1% |
| Sum of the components printed in the Results | Results | 16 + 12.3 + 9.2 = 37.5% | contradicts the 23% printed in the same paragraph |
| Indirect β the 16% would require | — | — | 0.16 × 0.731 = 0.117 (OR 1.12, about 9.7 × the printed 0.012; 95% CI with the printed SE 1.10 to 1.15) |
| Overall indirect and direct β the 37% would require | — | — | indirect 0.270 (OR 1.31); direct 0.461 (OR 1.58), against the printed 0.169 (OR 1.18) and 0.562 (OR 1.75) |
| Direct share of the total | implied | 63% if the abstract held | 0.562 / 0.731 = 76.9% |
Three independent routes give the same decomposition: the product of coefficients, the total − direct identity, and the CI centres. All three put the overall indirect association at 0.169 and the internalizing component at about 0.01. The 12.3%, 9.2% and 23% in the Results agree with them. The only figures that disagree are the internalizing 16%, which appears in both the Results and the Abstract, and the abstract's 37%, which is built from it.
Two smaller slips, noted in passing, do not affect the finding. The Results give "1,229 (17%)" children as exposed, but 1,229 / 8,861 = 13.9%. And the sentence after the total association (OR 2.08) reads "1.75 times higher odds", which is the direct OR.
Figure
Figure 1. Share of the total association between prenatal tobacco exposure and adolescent suicide attempt (β = 0.731) that is mediated by each mediator and by all three together. Orange circles are the values printed in the Results and the Abstract. Blue diamonds are recomputed by recompute.py.txt as indirect / total, with each indirect association taken as the product of the paper's printed exposure-to-mediator and mediator-to-outcome coefficients. The overall share is the printed 0.169 / 0.731. Externalizing behaviors, peer problems and the Results' 23% agree with the recomputed values. The internalizing share (16% printed, 1.6% recomputed) and the abstract's combined 37% (23.1% recomputed) do not. The axis starts at zero.
Figure 2. The same total association split into its direct and indirect parts. Top (orange): the split stated in the abstract, 16% + 12% + 9% = 37% mediated. The 63% direct share is not printed; it is implied as 100 − 37. Bottom (blue): the split given by the paper's own coefficients, direct 0.562 / 0.731 = 76.9% and indirect 1.6% + 12.3% + 9.2% = 23.1%. Because total = direct + indirect, the abstract's 37% cannot be squared with the direct association the paper reports: it would require a direct β of 0.461 (OR 1.58) instead of the printed 0.562 (OR 1.75).
Why it matters
- The abstract's headline number is wrong. The abstract gives one quantitative mediation result, and that result is the 37%. The paper's own coefficients, its Results paragraph and its Discussion all support about 23%. Readers who see only the abstract will take away a mediated share roughly 60% larger, in relative terms, than the paper's analysis supports.
- A non-significant pathway is presented as the largest mediator. In the abstract, internalizing symptoms carry the largest share (16%), ahead of externalizing behaviors (12%) and peer problems (9%). By the paper's own estimates the internalizing indirect association is the smallest (OR 1.01, 95% CI 0.99 to 1.03) and is reported as not significant. Its mediator-to-outcome path is not significant either (p = 0.278). The Discussion and the closing paragraph name only externalizing behaviors and peer problems as the mediating pathways.
- The abstract's recommendation leans on that figure. The abstract concludes that "interventions targeting mental health symptoms and peer problems may maximize suicide prevention efforts". The body of the paper supports targeting externalizing behaviors and peer problems.
- What is not affected. The total association (adjusted OR 2.08), the direct association (OR 1.75), the significant indirect paths through externalizing behaviors and peer problems, and the conclusion that mediation is partial all stand as reported. Table 2 and the sex-stratified and continuous-exposure analyses are consistent with the corrected decomposition.
Explanations considered
- A different model, with internalizing symptoms fitted alone. The Methods say each mediator was first analyzed separately, then all three together. For 16%, the internalizing indirect β would have to be about 0.117 (OR 1.12). With a = 0.205, that needs a mediator-to-outcome β of about 0.571 (OR 1.77), but the paper reports 0.057 (OR 1.06, p = 0.278). The 16% sits in the same sentence as β = 0.012 and OR 1.01, and next to the 12.3% and 9.2% that match the joint model. No single-mediator estimates are reported anywhere. This explanation is not supported.
- A different scale for the proportion. The Methods define the proportion as indirect / total. On the log-odds scale, which reproduces 12.3%, 9.2% and 23%, the internalizing share is 1.6%. On the rounded Table 2 ORs it is 1.4% (log scale) or 0.9% ((OR − 1) / (OR − 1)). No scale gives 16%.
- A different sample or exposure coding. Using Table 2's ORs, the internalizing share is 3.3% in the unadjusted model, 1.2% for females, 1.8% for males, and 1.3% with smoking as a continuous variable (Supplementary Table S2). None is near 16%.
- The 16% is right and the 0.012 is wrong. This is ruled out by the paper's own numbers. The OR 1.01, the CI 0.99 to 1.03 (log-scale centre 0.0098), the product 0.205 × 0.057 = 0.0117, the overall indirect 0.169 and the total − direct identity all independently support an internalizing indirect association of about 0.012. A β of 0.117 with the printed SE would give a CI of about 1.10 to 1.15, which is clearly significant.
- Rounding in the abstract. Rounding explains 12.3% → 12% and 9.2% → 9%. It does not explain 16% or 37%.
- A conversion artifact in the archived text. Ruled out. The same strings appear in the Cambridge Core version of record, the PMC HTML page and the Europe PMC full-text XML.
The pattern fits a shifted decimal (1.6% printed as 16%) that was then carried into the abstract's sum. The printed externalizing β of 0.90 for 0.090 is a misprint of the same kind. The cause cannot be determined from the paper.
Status
As of 2026-10-24:
- Version of record: checked on Cambridge Core (via doi.org). The abstract's "accounting for 37% of the total association, that is, 16%, 12%, and 9%", the Results' "Internalizing symptoms explained 16% of the association" and "β = 0.90", and the 23% overall share appear there exactly as in PMC.
- Supplement: the supplementary file (doi:10.1017/S0033291725102936.sm001) contains Tables S1 and S2 and Figure S1. Table S2 (continuous exposure) gives adjusted ORs of total 2.10, direct 1.79, internalizing indirect 1.01 (0.99–1.03), externalizing 1.08, peer problems 1.07 and total indirect 1.17, the same pattern as the main analysis. It reports no proportions mediated, so it does not resolve the discrepancy.
- Corrections: none found. Crossref shows no update-to or updated-by relation. PMC and Europe PMC notice checks returned nothing. The PubMed record (PMID 41480736) is typed "Journal Article" only, with no linked comments or corrections. The Cambridge Core page shows no correction notice.
- PubPeer: no discussion found. The PubPeer API refused automated queries (HTTP 403), the PubPeer search page did not resolve to an entry for this DOI, and web searches for the title together with "PubPeer" or "correction" found no thread or notice.
Sources: version of record; PMC13092991; Crossref record.
Suggested correction
- Abstract, Results: replace "accounting for 37% of the total association, that is, 16%, 12%, and 9%, respectively" with "accounting for 23% of the total association, that is, 1.6%, 12%, and 9%, respectively". Because the internalizing indirect association is not significant, the authors may also wish to revise "partly mediated through internalizing problems, externalizing behaviors, and peer problems" so that it names the significant pathways.
- Results, "Pathways" paragraph: "Internalizing symptoms explained 16% of the association" should read "1.6%". "(indirect association: β = 0.90" for externalizing behaviors should read "β = 0.090". The authors should also check the peer problems indirect β (printed 0.057). Its OR (1.07), its CI centre (0.067) and the product of its paths (0.067) all point to about 0.067, which is also the value that gives the reported 9.2%.
- Abstract, Conclusions: the authors may wish to reconsider "interventions targeting mental health symptoms and peer problems" in light of the corrected internalizing share. The Discussion's own wording is "externalizing behaviors and peer problems".
- Minor: "1,229 (17%)" should read 13.9% if 8,861 is the denominator. The sentence after the total OR of 2.08 should give 2.08, not 1.75, times higher odds.
Recomputation script (python recompute.py.txt)
"""Recompute the proportions mediated in PMC13092991
(Psychol Med 2026, doi:10.1017/S0033291725102936).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PubMed Central:
Results, "Association between maternal smoking during pregnancy and childhood
psychosocial difficulties": exposure -> mediator coefficients (a paths).
Results, "Association between childhood psychosocial difficulties and adolescent
suicide attempt": mediator -> suicide attempt coefficients (b paths).
Results, "Pathways of association ...": total, direct and indirect coefficients
(log-odds scale), their ORs and 95% CIs, and the proportions mediated.
Abstract, Results: "accounting for 37% of the total association, that is, 16%,
12%, and 9%, respectively".
Methods, "Mediation analysis": indirect = product of coefficients; proportion
mediated = indirect association / total association; total = indirect + direct.
No raw data are available; the checks use only the printed coefficients.
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = 1.959964
# ---------------------------------------------------------------- inputs (from the paper)
MED = ["intern", "extern", "peer"]
NAME = {"intern": "Internalizing symptoms", "extern": "Externalizing behaviors",
"peer": "Peer problems"}
A = {"intern": 0.205, "extern": 0.477, "peer": 0.279} # exposure -> mediator
B = {"intern": 0.057, "extern": 0.188, "peer": 0.241} # mediator -> attempt
TOTAL, DIRECT, IND_ALL = 0.731, 0.562, 0.169 # log-odds scale
SE_TOTAL, SE_DIRECT, SE_IND_ALL = 0.107, 0.107, 0.023
OR_TOTAL, OR_DIRECT, OR_IND_ALL = 2.08, 1.75, 1.18
CI_DIRECT = (1.42, 2.16)
CI_IND_ALL = (1.13, 1.24)
IND_PRINTED = {"intern": 0.012, "extern": 0.90, "peer": 0.057} # beta as printed
SE_IND = {"intern": 0.011, "extern": 0.025, "peer": 0.025}
OR_IND = {"intern": 1.01, "extern": 1.09, "peer": 1.07}
CI_IND = {"intern": (0.99, 1.03), "extern": (1.04, 1.15), "peer": (1.03, 1.11)}
PCT_BODY = {"intern": 16.0, "extern": 12.3, "peer": 9.2} # Results text
PCT_BODY_ALL = 23.0 # Results text
PCT_ABS = {"intern": 16, "extern": 12, "peer": 9} # Abstract
PCT_ABS_ALL = 37 # Abstract
# Supplementary Table S2 (smoking as a continuous variable), adjusted ORs
S2_TOTAL_OR, S2_INTERN_OR = 2.10, 1.01
# side checks (Results)
N, N_EXPOSED, N_ATTEMPT, N_ATT_EXPOSED = 8861, 1229, 627, 167
# ---------------------------------------------------------------- styling
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{stem}.{ext}"), bbox_inches="tight", **kw)
plt.close(fig)
def main():
R = {}
print("== 1. Indirect associations by the product of coefficients (a x b)")
ab = {k: A[k] * B[k] for k in MED}
for k in MED:
print(f" {NAME[k]:24s} a={A[k]:.3f} x b={B[k]:.3f} = {ab[k]:.4f} "
f"printed beta = {IND_PRINTED[k]}")
s = sum(ab.values())
print(f" sum of a x b = {s:.4f} printed overall indirect = {IND_ALL}")
print(f" total - direct = {TOTAL - DIRECT:.3f} printed overall indirect = {IND_ALL}")
R["ab"], R["sum_ab"] = ab, s
print("\n== 2. Printed indirect betas vs their own ORs and 95% CIs")
print(" (log-OR implied by the OR; centre and SE implied by the CI on the log scale)")
for k in MED:
lo, hi = CI_IND[k]
mid = (math.log(lo) + math.log(hi)) / 2
se = (math.log(hi) - math.log(lo)) / (2 * Z)
print(f" {NAME[k]:24s} ln(OR {OR_IND[k]}) = {math.log(OR_IND[k]):.4f} "
f"CI centre = {mid:.4f} (SE {se:.4f}) exp(printed beta {IND_PRINTED[k]}) = "
f"{math.exp(IND_PRINTED[k]):.3f} exp(a x b) = {math.exp(ab[k]):.3f}")
lo, hi = CI_IND_ALL
print(f" {'Overall indirect':24s} ln(OR {OR_IND_ALL}) = {math.log(OR_IND_ALL):.4f} "
f"CI centre = {(math.log(lo) + math.log(hi)) / 2:.4f}")
b = IND_PRINTED["intern"]
print(f" internalizing CI from printed beta and SE: exp({b} +/- 1.96 x {SE_IND['intern']}) "
f"= [{math.exp(b - Z * SE_IND['intern']):.3f}, {math.exp(b + Z * SE_IND['intern']):.3f}]"
f" (printed [0.99, 1.03])")
b = 0.090
print(f" externalizing CI with beta 0.090: [{math.exp(b - Z * 0.025):.3f}, "
f"{math.exp(b + Z * 0.025):.3f}] (printed [1.04, 1.15]); with beta 0.90: "
f"[{math.exp(0.90 - Z * 0.025):.3f}, {math.exp(0.90 + Z * 0.025):.3f}]")
print("\n== 3. Log-OR consistency of total, direct and overall indirect")
for lab, beta, OR in (("total", TOTAL, OR_TOTAL), ("direct", DIRECT, OR_DIRECT),
("overall indirect", IND_ALL, OR_IND_ALL)):
print(f" {lab:17s} exp({beta}) = {math.exp(beta):.3f} printed OR {OR}")
print("\n== 4. Proportion mediated = indirect / total (Methods definition)")
pct = {k: 100 * ab[k] / TOTAL for k in MED}
for k in MED:
print(f" {NAME[k]:24s} a x b / total = {pct[k]:5.1f}% Results {PCT_BODY[k]}% "
f"Abstract {PCT_ABS[k]}%")
print(f" internalizing from printed beta 0.012: {100 * 0.012 / TOTAL:.2f}%")
print(f" externalizing from 0.090: {100 * 0.090 / TOTAL:.2f}%; from printed 0.90: "
f"{100 * 0.90 / TOTAL:.1f}%")
print(f" peer problems from printed beta 0.057: {100 * 0.057 / TOTAL:.2f}%; from 0.067: "
f"{100 * 0.067 / TOTAL:.2f}%")
p_all = 100 * IND_ALL / TOTAL
print(f" all three: overall indirect / total = {p_all:.1f}% Results {PCT_BODY_ALL:.0f}% "
f"Abstract {PCT_ABS_ALL}%")
print(f" sum of recomputed components = {sum(pct.values()):.1f}%")
print(f" sum of Results components 16 + 12.3 + 9.2 = {sum(PCT_BODY.values()):.1f}%")
print(f" sum of Abstract components 16 + 12 + 9 = {sum(PCT_ABS.values())}%")
print(f" direct share = direct / total = {100 * DIRECT / TOTAL:.1f}% "
f"(check: 100 - {p_all:.1f} = {100 - p_all:.1f}%)")
R["pct"], R["p_all"] = pct, p_all
print("\n== 5. What the printed 16% and 37% would require")
need_i = 0.16 * TOTAL
print(f" internalizing indirect beta for 16%: 0.16 x {TOTAL} = {need_i:.3f} -> OR "
f"{math.exp(need_i):.2f}; ratio to printed 0.012 = {need_i / 0.012:.1f}x")
print(f" with printed SE 0.011 its 95% CI would be [{math.exp(need_i - Z * 0.011):.2f}, "
f"{math.exp(need_i + Z * 0.011):.2f}] (printed OR 1.01, CI [0.99, 1.03])")
need_ind = 0.37 * TOTAL
need_dir = TOTAL - need_ind
print(f" overall indirect beta for 37%: {need_ind:.3f} -> OR {math.exp(need_ind):.2f} "
f"(printed 0.169, OR 1.18, CI [1.13, 1.24])")
print(f" direct beta then left: {TOTAL} - {need_ind:.3f} = {need_dir:.3f} -> OR "
f"{math.exp(need_dir):.2f} (printed 0.562, OR 1.75, CI [1.42, 2.16])")
print(f" internalizing b path needed for 16% with a = 0.205: {need_i / 0.205:.3f} "
f"-> OR {math.exp(need_i / 0.205):.2f} (printed b = 0.057, OR 1.06, p = 0.278)")
print("\n== 6. Other scales and models (does any of them give 16%?)")
print(f" OR scale (OR_ind - 1)/(OR_tot - 1): intern {(1.01 - 1) / (2.08 - 1) * 100:.1f}%, "
f"extern {(1.09 - 1) / (2.08 - 1) * 100:.1f}%, peer {(1.07 - 1) / (2.08 - 1) * 100:.1f}%")
print(f" log of rounded Table 2 ORs: intern {100 * math.log(1.01) / math.log(2.08):.1f}%, "
f"extern {100 * math.log(1.09) / math.log(2.08):.1f}%, "
f"peer {100 * math.log(1.07) / math.log(2.08):.1f}%, "
f"overall {100 * math.log(1.18) / math.log(2.08):.1f}%")
print(f" unadjusted model (Table 2 ORs 1.03 / 2.47): intern "
f"{100 * math.log(1.03) / math.log(2.47):.1f}%")
print(f" Supplementary Table S2 (continuous smoking, ORs {S2_INTERN_OR} / {S2_TOTAL_OR}): "
f"intern {100 * math.log(S2_INTERN_OR) / math.log(S2_TOTAL_OR):.1f}%")
print(f" female-only adjusted (1.01 / 2.23): {100 * math.log(1.01) / math.log(2.23):.1f}%; "
f"male-only adjusted (1.01 / 1.76): {100 * math.log(1.01) / math.log(1.76):.1f}%")
print("\n== 7. Side checks (not part of the main finding)")
print(f" exposed share {N_EXPOSED}/{N} = {100 * N_EXPOSED / N:.1f}% (text: 17%)")
print(f" attempt share {N_ATTEMPT}/{N} = {100 * N_ATTEMPT / N:.1f}% (text: 7%)")
print(f" exposed share of attempters {N_ATT_EXPOSED}/{N_ATTEMPT} = "
f"{100 * N_ATT_EXPOSED / N_ATTEMPT:.2f}% (text: 26.63%)")
crude = (N_ATT_EXPOSED / (N_EXPOSED - N_ATT_EXPOSED)) / (
(N_ATTEMPT - N_ATT_EXPOSED) / (N - N_EXPOSED - (N_ATTEMPT - N_ATT_EXPOSED)))
print(f" crude OR from counts = {crude:.2f} (text, unadjusted: 2.47)")
fig1(R)
fig2(R)
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png")
def fig1(R):
"""Dot comparison: printed proportion mediated vs a x b / total."""
rows = [
("Internalizing symptoms", [("Abstract and Results", 16.0)], R["pct"]["intern"]),
("Externalizing behaviors", [("Results", 12.3), ("Abstract", 12)], R["pct"]["extern"]),
("Peer problems", [("Results", 9.2), ("Abstract", 9)], R["pct"]["peer"]),
("All three mediators", [("Abstract", 37), ("Results", 23)], R["p_all"]),
]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ys = list(range(len(rows)))[::-1]
for y, (lab, reps, rec) in zip(ys, rows):
xs = [v for _, v in reps] + [rec]
ax.plot([min(xs), max(xs)], [y, y], color=GRID, lw=2, zorder=1)
for _, v in reps:
ax.scatter(v, y + 0.13, s=70, color=ORANGE, zorder=3)
lab_txt = ", ".join(f"{v:g}% ({src})" for src, v in reps)
if len(reps) == 2 and abs(reps[0][1] - reps[1][1]) > 5:
for src, v in reps:
ax.annotate(f"{v:g}% ({src})", (v, y + 0.13), xytext=(0, 7),
textcoords="offset points", ha="center", va="bottom",
fontsize=9.5, color=ORANGE)
else:
ax.annotate(lab_txt, (max(v for _, v in reps), y + 0.13), xytext=(0, 7),
textcoords="offset points", ha="center", va="bottom",
fontsize=9.5, color=ORANGE)
ax.scatter(rec, y - 0.13, s=70, marker="D", color=BLUE, zorder=3)
ax.annotate(f"{rec:.1f}%", (rec, y - 0.13), xytext=(0, -8),
textcoords="offset points", ha="center", va="top",
fontsize=9.5, color=BLUE)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.set_xlim(0, 42)
ax.set_ylim(-0.6, len(rows) - 0.35)
ax.set_xlabel("Share of the total association mediated (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
h1 = ax.scatter([], [], s=70, color=ORANGE, label="Printed in the paper")
h2 = ax.scatter([], [], s=70, marker="D", color=BLUE,
label="Recomputed: indirect / total (0.731), from the paper's own coefficients")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.42, -0.2), ncol=1,
frameon=False, fontsize=10)
save(fig, "fig1")
def fig2(R):
"""100% bar of the total association (beta 0.731): abstract's split vs the paper's betas."""
pct = R["pct"]
bars = [
("Abstract\n(16 + 12 + 9 = 37%)", 100 - 37, [16, 12, 9], ORANGE),
("Paper's own coefficients\n(0.562 direct + 0.169 indirect)",
100 * DIRECT / TOTAL, [pct["intern"], pct["extern"], pct["peer"]], BLUE),
]
fig, ax = plt.subplots(figsize=(7.2, 2.9))
seg_names = ["Int.", "Ext.", "Peer"]
alphas = [1.0, 0.7, 0.45]
for y, (lab, direct, parts, col) in zip([1, 0], bars):
ax.barh(y, direct, color="#d9d8d3", edgecolor=BG, height=0.55)
ax.text(direct / 2, y, f"Direct {direct:.0f}% (implied: 100 − 37)" if col == ORANGE else f"Direct {direct:.1f}%",
ha="center", va="center", fontsize=10, color=INK)
left = direct
for nm, p, al in zip(seg_names, parts, alphas):
ax.barh(y, p, left=left, color=col, alpha=al, edgecolor=BG, height=0.55)
if p >= 5:
ax.text(left + p / 2, y, f"{nm}\n{p:g}%" if col == ORANGE else f"{nm}\n{p:.1f}%",
ha="center", va="center", fontsize=9.5, color=INK)
else:
ax.annotate(f"{nm} {p:.1f}%", (left + p / 2, y + 0.275), xytext=(0, 4),
textcoords="offset points", ha="center", va="bottom",
fontsize=9.5, color=INK)
left += p
ax.set_yticks([1, 0])
ax.set_yticklabels([b[0] for b in bars])
for t, b in zip(ax.get_yticklabels(), bars):
t.set_color(b[3])
ax.set_xlim(0, 100)
ax.set_ylim(-0.5, 1.6)
ax.set_xlabel("Share of the total association, β = 0.731 (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
save(fig, "fig2")
if __name__ == "__main__":
main()
A 99% specificity cannot arise from 14 controls and conflicts with the reported 72% accuracy
Psychol Med 2014;44(3):519–532 (first published online 5 June 2013) · doi:10.1017/S0033291713001013 · PMC3880067
Summary
The abstract and Results report that the grey-matter (GM) classifier in cohort 2 had a sensitivity of 64%, a specificity of 99% and an overall accuracy of 72%. Cohort 2 has 14 patients and 14 controls, and every subject is tested once, so specificity can only be a multiple of 1/14: the values nearest 99% are 13/14 = 92.9% and 14/14 = 100%. With equal groups, accuracy must also equal the mean of sensitivity and specificity, and (64 + 99)/2 = 81.5%, not 72%. The printed 72% is not exactly attainable either: with 28 subjects the nearest accuracies are 20/28 = 71.4% and 21/28 = 75.0%. The counts nearest to the printed 64% sensitivity and 72% accuracy are 9 of 14 patients (64.3%) and 11 of 14 controls (78.6%) classified correctly, giving 20/28 = 71.4%, which would be printed as 72% if the accuracy was taken as the mean of the rounded sensitivity and specificity, (64 + 79)/2 = 71.5, with halves rounded to even, as the printed 78% for cohort 2 WM also requires (see below). If the reported sensitivity and accuracy are correct, the specificity was 11/14 = 78.6%, and the printed 99% overstates a headline diagnostic figure in the abstract by about 20 points. The paper's accuracy-based conclusions are unaffected.
What the paper reports
Abstract, Results:
The diagnostic accuracy of the GPC for GM was 73% in cohort 1 and 72% in cohort 2; the sensitivity and specificity of the GM classification were respectively 69% and 77% in cohort 1 and 64% and 99% in cohort 2. The diagnostic accuracy of the GPC for WM was 69% in cohort 1 and 78% in cohort 2; the sensitivity and specificity of the WM classification were both 69% in cohort 1 and 71% and 86% respectively in cohort 2.
Results, "Prediction accuracy":
For cohort 1, classification accuracy using GPC analysis of GM images was 73% with a sensitivity and specificity of 77% and 69% respectively.
For cohort 2, the sensitivity and specificity of the GM classification were 64% and 99% respectively and the overall accuracy was 72%. In the same cohort, GPC analysis using WM images yielded an accuracy of 78% with a sensitivity of 71% and specificity of 86%. For both cohorts, the models were significant at p < 0.001.
Method, "Samples" (Table 1 also gives n = 14 patients and n = 14 healthy controls for cohort 2; the in-text citation at the end is replaced by "[citation]"):
Cohort 2 comprised 14 patients fulfilling DSM-IV criteria for BP-I and 14 healthy controls derived from participants in the Vulnerability Indicators to Bipolar Disorder Study (VIBES; [citation]).
Method, "Pattern classification analysis":
To achieve this, we used nested (three-way) cross-validation where we first excluded a matched pair of subjects (one from each group) to comprise the test set, and then performed a second split where we repeatedly repartitioned the remaining subject pairs into a validation and a training set.
We thresholded the probabilistic predictions at 0.5 to convert the probabilistic predictions to class labels and computed the proportion of subjects having the correct label across all test splits to estimate the classification accuracy.
Method, "Cross-validation":
Second, we quantified the sensitivity and specificity of each classifier defined as: sensitivity = TP/(TP + FN) and specificity = TN/(TN + FP), where TP is the number of true positives (number of images of patients correctly classified), TN is the number of true negatives (number of images of controls correctly classified), FP is the number of false positives (number of images of controls classified as patients) and FN is the number of false negatives (number of images of patients classified as controls).
Discussion, opening paragraph, and Discussion > Prediction accuracy of GPCs applied to sMRI in BD, second paragraph:
We found that GPCs applied to GM reliably achieved above chance discriminative power solely on the basis of anatomical data, with classification accuracy ranging between 69% and 78%.
The application of GPC analysis to anatomical scans in BD provided diagnostic accuracy in the range 69–78%.
What the paper's own numbers show
Method. Leave-one-matched-pair-out cross-validation tests each subject exactly once, with a hard 0.5 threshold. In cohort 2 there are therefore 14 patient predictions and 14 control predictions, so sensitivity = TP/14, specificity = TN/14 and accuracy = (TP + TN)/28. Because the groups are equal, accuracy is exactly the mean of sensitivity and specificity. For each of the paper's four classifiers I searched all integer pairs (TP, TN) for those whose sensitivity, specificity and accuracy each round to the printed whole percentage (within 0.5 points). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Cohort 2 GM specificity | Abstract; Results | 99% | Not attainable with 14 controls. The values near it are 13/14 = 92.9% and 14/14 = 100%. The value nearest to consistency with the reported sensitivity and accuracy is 11/14 = 78.6% |
| Cohort 2 GM accuracy vs mean of sensitivity and specificity | Abstract; Results | 72% | (64 + 99)/2 = 81.5% (between 81.0% and 82.0% allowing for rounding of both inputs); gap 9.5 points |
| Cohort 2 GM sensitivity | Abstract; Results | 64% | 9/14 = 64.3%, the only count that rounds to 64% (consistent) |
| Cohort 2 GM accuracy | Abstract; Results | 72% | Not exactly attainable with 28 subjects: no k/28 rounds to 72%. The nearest values are 20/28 = 71.4% (rounds to 71%) and 21/28 = 75.0%. With TP = 9, 20/28 needs TN = 11. The 72% fits 20/28 only if it was computed from rounded inputs (see the rounding paragraph below) |
| Cohort 2 WM accuracy / sensitivity / specificity | Abstract; Results | 78% / 71% / 86% | 22/28 = 78.6% / 10/14 = 71.4% / 12/14 = 85.7%; consistent apart from 78.6% being printed as 78% |
| Cohort 1 GM accuracy / sensitivity / specificity | Results | 73% / 77% / 69% | 38/52 = 73.1% / 20/26 = 76.9% / 18/26 = 69.2% (consistent) |
| Cohort 1 GM sensitivity / specificity | Abstract | 69% / 77% | The reverse of the Results (77% / 69%). Both orders fit 38/52 = 73.1% accuracy, so the text cannot settle which is right |
| Cohort 1 WM accuracy / sensitivity / specificity | Abstract; Results | 69% / 69% / 69% | 36/52 = 69.2% / 18/26 = 69.2% / 18/26 = 69.2% (consistent) |
Under the strict rounding rule above, no integer counts reproduce the cohort 2 GM triple at all, because neither 99% nor 72% can occur; recompute.py.txt finds no (TP, TN) pair. The analysis therefore looks for the counts nearest to the printed values. Other values for cohort 2 GM fit worse. With TP = 9, a specificity of 13/14 or 14/14 gives an accuracy of 22/28 = 78.6% or 23/28 = 82.1%, not 72%. Keeping the 72% accuracy (20/28) with a specificity of 13/14 or 14/14 would need a sensitivity of 50.0% or 42.9%, so the printed 64% would also have to be wrong. The single change that brings all three values into line (64%, 79% and an accuracy of 71.4%, printed as 72% under the rounding pattern below) is a specificity of 11/14 = 78.6% in place of 99%.
The paper's accuracies may have been computed from the rounded sensitivity and specificity rather than from the counts. Cohort 2 WM is printed as 78%, which matches (71 + 86)/2 = 78.5 rounded down, while 22/28 = 78.6% would round to 79%. For cohort 2 GM, (64 + 79)/2 = 71.5 would match the printed 72% if rounded up, while 20/28 = 71.4% would round to 71%. The two halves go in opposite directions, so they fit one rule only if halves are rounded to the nearest even number (78.5 to 78, 71.5 to 72); otherwise the rounding direction is not consistent. This leaves a printed "99%" standing for 79% as a possible explanation, but the text alone cannot confirm it.
Figure
Figure 1. Each row shows one cohort 2 GM performance measure on a common percentage axis. Grey tick marks show every value that can occur with the paper's group sizes (k/14 for sensitivity and specificity, k/28 for accuracy). Orange dots are the values printed in the abstract and Results. Filled blue dots are the nearest counts consistent with the printed sensitivity and accuracy, recomputed by recompute.py.txt: 9/14 for 64%, and 20/28 = 71.4%, the attainable accuracy closest to 72% (no count out of 28 gives exactly 72%); with those counts, specificity is 11/14 = 78.6%. The hollow blue circle is the accuracy implied by the printed 64% and 99% (81.5%), which does not match the printed 72%. Neither the printed 99% nor the printed 72% lies on a tick mark. The axis starts at 50% because this is a dot plot, where values are read from position, not from bar length.
Figure 2. For each of the four classifiers, the reported accuracy (orange) and the mean of the reported sensitivity and specificity (hollow blue), which must be equal when the two groups are the same size. Three classifiers agree to within 0.5 points. Cohort 2 GM differs by 9.5 points. Values are from the abstract and Results (cohort 1 GM in the order given in the Results) and from recompute.py.txt.
Why it matters
- A headline diagnostic figure is overstated. Sensitivity and specificity are the measures a clinical reader uses to judge a diagnostic test, and the abstract presents them for each classifier. A specificity of 99% says the cohort 2 GM classifier almost never labels a healthy person as having bipolar disorder. The value consistent with the paper's own counts, about 79% (3 of 14 controls misclassified), is in line with the other three classifiers (69% to 86%).
- It distorts the comparison between classifiers. As printed, cohort 2 GM looks far better at identifying controls than at identifying patients (99% vs 64%) and better than every other classifier on specificity. With 78.6%, its specificity is below that of the cohort 2 WM classifier (85.7%).
- The paper's main conclusions are unaffected. The conclusions rest on accuracy: GM accuracy of 72% to 73% in both cohorts, the 69% to 78% range stated in the Discussion, above-chance performance (p < 0.001 by permutation), and the comparison with case-finding questionnaires whose "sensitivity and specificity are about 70%". The printed 72% accuracy is close to the accuracy implied by the corrected specificity (20/28 = 71.4%), so none of these change.
Explanations considered
- Different group sizes or exclusions. Not supported. Method > Samples and Table 1 both give 14 patients and 14 controls for cohort 2, and no exclusions from the classification are described.
- Averaging over folds or repeated cross-validation. Not able to explain it. Each test split is one matched pair, and accuracy is "the proportion of subjects having the correct label across all test splits", so each subject is classified once. Averaging per-fold specificity over 14 one-control folds also gives a multiple of 1/14. No repeated or resampled cross-validation is described; the permutation test is used only for p-values.
- A different definition of specificity or accuracy. Not supported. Specificity is defined as TN/(TN + FP), and accuracy as the proportion of correctly classified subjects. No balanced or weighted accuracy is described, and with equal groups such variants coincide with plain accuracy anyway.
- Probabilistic rather than thresholded predictions. Not supported. The Methods state that predictions were thresholded at 0.5 before computing accuracy.
- An error in the accuracy or the sensitivity instead. Less consistent with the paper. The 99% cannot be right in any case. Keeping 64% and the nearest attainable accuracy to 72% (20/28) gives a specificity of 78.6%. Keeping 72% with a specificity of 13/14 or 14/14 would require the 64% sensitivity to be wrong too (50.0% or 42.9%). Keeping 64% with 13/14 or 14/14 would require an accuracy of 78.6% or 82.1%; the Discussion's 69% to 78% accuracy range is consistent with the printed 72% and not with 78.6% (printed as 79%) or 82.1%.
- A typographical slip of 99 for 79. Plausible but not confirmable from the text. It is the single change that brings all three values into line, and the printed 72% then fits if halves were rounded to even, as for cohort 2 WM (see above). It does not make the published 99% correct.
- A conversion or typesetting artifact in PMC. Ruled out. The same figure appears in the publisher's version of record and in the Europe PMC full text (see Status).
- Side issue: cohort 1 GM sensitivity and specificity. The abstract gives 69% and 77%, and the Results give 77% and 69%, explaining the latter as a 0.77 probability of correct classification for a patient. Both orders give the same accuracy (38/52 = 73.1%), so the counts cannot say which order is right; the two sections disagree.
Status
- Version of record. The publisher's page on Cambridge Core (reached through the DOI) has the same abstract and Results values: 64% and 99% for cohort 2 GM with 72% accuracy, and 69% and 77% for cohort 1 GM in the abstract against 77% and 69% in the Results. The PMC page and the Europe PMC full-text XML match. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. Both supplementary files were downloaded and opened:
S0033291713001013sup001.doc(Supplementary Tables S1 to S4, regional maxima from voxel-based morphometry) andS0033291713001013sup002.pdf(4 pages of classifier methods). Neither contains classification counts, a confusion matrix or performance values. Tables 1 to 5 of the paper give demographics and discriminative regions only. - Corrections and comments. As of 2026-10-24 no correction, erratum, corrigendum, retraction or expression of concern was found.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by relations. The Europe PMC record (PMID 23734914) lists no comments or corrections. Neither the Cambridge Core page nor the PMC page shows a notice. A direct PubPeer API query returned HTTP 403, and web searches for PubPeer, Retraction Watch or erratum discussion of this article found nothing. - Sources. PMC article PMC3880067; Europe PMC full-text XML and supplementary files; Cambridge Core version of record via https://doi.org/10.1017/S0033291713001013; Crossref; Europe PMC; PubPeer; web search.
Suggested correction
In the abstract and in Results > Prediction accuracy, the cohort 2 GM specificity of "99%" should be replaced by the value from the classifier's actual predictions. If 9 of 14 patients and 11 of 14 controls were classified correctly, the counts nearest to the reported 64% sensitivity and 72% accuracy, the sentences would read
the sensitivity and specificity of the GM classification were respectively 69% and 77% in cohort 1 and 64% and 79% in cohort 2
and
For cohort 2, the sensitivity and specificity of the GM classification were 64% and 79% respectively and the overall accuracy was 72%.
The authors may also wish to make the order of cohort 1 GM sensitivity and specificity the same in the abstract and Results, to report accuracies computed from the counts (20/28 = 71.4% and 22/28 = 78.6% for cohort 2), and to give the confusion-matrix counts for each classifier so readers can check the percentages.
Recomputation script (python recompute.py.txt)
"""Recompute the cohort 2 grey-matter (GM) classifier metrics in PMC3880067
(Psychol Med 2014;44(3):519-532, first published online 5 June 2013,
doi:10.1017/S0033291713001013).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs, all from the paper:
* group sizes (Table 1; Method > Samples): cohort 1 has 26 patients + 26 controls,
cohort 2 has 14 patients + 14 controls;
* evaluation (Method > Pattern classification analysis): leave-one-matched-pair-out
cross-validation, predictions thresholded at 0.5, accuracy = proportion of subjects
with the correct label across all test splits, so every subject is tested once;
* definitions (Method > Cross-validation): sensitivity = TP/(TP+FN),
specificity = TN/(TN+FP);
* reported percentages (Abstract; Results > Prediction accuracy).
"""
import os
from itertools import product
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# label, n per group, reported accuracy, sensitivity, specificity (%), where reported
ROWS = [
("Cohort 1 GM (Results)", 26, 73, 77, 69, "Results"),
("Cohort 1 GM (Abstract)", 26, 73, 69, 77, "Abstract"),
("Cohort 1 WM", 26, 69, 69, 69, "Abstract; Results"),
("Cohort 2 GM", 14, 72, 64, 99, "Abstract; Results"),
("Cohort 2 WM", 14, 78, 71, 86, "Abstract; Results"),
]
def near(value, reported, tol=0.5):
"""True if value rounds to the reported whole percentage (+-0.5 points)."""
return abs(value - reported) <= tol + 1e-9
def pct(k, n):
return 100.0 * k / n
print("=" * 78)
print("1. Integer counts consistent with each reported (accuracy, sensitivity, specificity)")
print(" With n patients and n controls, accuracy = (TP+TN)/2n = (sens+spec)/2 exactly.")
print("=" * 78)
for lab, n, acc, se, sp, where in ROWS:
sols = [(tp, tn) for tp, tn in product(range(n + 1), repeat=2)
if near(pct(tp, n), se) and near(pct(tn, n), sp) and near(pct(tp + tn, 2 * n), acc)]
print(f"\n{lab} [{where}] n = {n} + {n}; reported acc {acc}%, sens {se}%, spec {sp}%")
print(f" mean of reported sens and spec: ({se} + {sp})/2 = {(se + sp) / 2:.1f}%")
if sols:
for tp, tn in sols:
print(f" consistent counts: TP = {tp}/{n} ({pct(tp, n):.1f}%), TN = {tn}/{n} "
f"({pct(tn, n):.1f}%), correct = {tp + tn}/{2 * n} ({pct(tp + tn, 2 * n):.1f}%)")
else:
tps = [tp for tp in range(n + 1) if near(pct(tp, n), se)]
tns = [tn for tn in range(n + 1) if near(pct(tn, n), sp)]
print(" NO integer (TP, TN) reproduces all three values.")
print(f" TP that round to {se}%: {[(t, round(pct(t, n), 1)) for t in tps]}")
print(f" TN that round to {sp}%: {[(t, round(pct(t, n), 1)) for t in tns] or 'none'}")
if tps and tns:
for tp in tps:
for tn in tns:
print(f" using TP = {tp} and TN = {tn}: accuracy = {tp + tn}/{2 * n} = "
f"{pct(tp + tn, 2 * n):.1f}% (reported {acc}%)")
print("\n" + "=" * 78)
print("2. Cohort 2 GM in detail (14 patients, 14 controls)")
print("=" * 78)
n = 14
print(" attainable specificities k/14 near the top:",
", ".join(f"{k}/14 = {pct(k, n):.1f}%" for k in range(10, 15)))
print(" attainable specificities that round to 99%:",
[k for k in range(n + 1) if near(pct(k, n), 99)] or "none")
print(" attainable accuracies k/28 near 72%:",
", ".join(f"{k}/28 = {pct(k, 28):.2f}%" for k in range(19, 23)))
print(" attainable accuracies k/28 that round to 72%:",
[k for k in range(29) if near(pct(k, 28), 72)] or "none")
print(" so neither the printed 99% nor the printed 72% can occur exactly; the search below"
" looks for the counts nearest to the printed values")
TP = 9
print(f" sensitivity 64% -> TP = {TP} ({TP}/14 = {pct(TP, n):.1f}%), the only count that rounds to 64%")
for k in (20, 21):
print(f" if {k}/28 ({pct(k, 28):.1f}%) were correct: TN = {k - TP}, "
f"specificity = {k - TP}/14 = {pct(k - TP, n):.1f}%")
for tn in (13, 14):
print(f" if TN = {tn} ({pct(tn, n):.1f}%) with TP = 9: accuracy = {TP + tn}/28 = {pct(TP + tn, 28):.1f}%")
for tn in (13, 14):
tp = 20 - tn
print(f" if TN = {tn} and accuracy = 20/28: TP = {tp}, sensitivity = {pct(tp, n):.1f}% (reported 64%)")
TN = 11
print(f" single-change solution: TP = 9, TN = {TN}: sens {pct(TP, n):.1f}%, spec {pct(TN, n):.1f}%, "
f"accuracy {TP + TN}/28 = {pct(TP + TN, 28):.1f}%")
print(f" gap between printed and consistent specificity: 99 - {pct(TN, n):.1f} = {99 - pct(TN, n):.1f} points")
print(f" accuracy implied by the printed 64% and 99%: {(64 + 99) / 2:.1f}% "
f"(gap to reported 72%: {(64 + 99) / 2 - 72:.1f} points)")
print(f" bounds: sens in [63.5, 64.5], spec in [98.5, 99.5] -> accuracy in "
f"[{(63.5 + 98.5) / 2:.1f}, {(64.5 + 99.5) / 2:.1f}]%")
print("\n" + "=" * 78)
print("3. Rounding pattern: accuracy as the mean of the rounded sens and spec")
print(" (Python round() rounds halves to even; shown for comparison only)")
print("=" * 78)
for lab, se, sp, acc in [("Cohort 1 GM", 77, 69, 73), ("Cohort 1 WM", 69, 69, 69),
("Cohort 2 WM", 71, 86, 78), ("Cohort 2 GM if spec = 79", 64, 79, 72)]:
m = (se + sp) / 2
print(f" {lab}: ({se} + {sp})/2 = {m:.1f} -> round half to even {round(m)} (reported accuracy {acc}%)")
print(f" from counts instead: cohort 2 WM 22/28 = {pct(22, 28):.2f}% (reported 78%), "
f"cohort 2 GM 20/28 = {pct(20, 28):.2f}% (reported 72%)")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def style_axis(ax, lo=50):
ax.set_xlim(lo - 2, 102)
ax.set_xticks(range(lo, 101, 10))
ax.set_xticklabels([f"{t}%" for t in range(lo, 101, 10)])
ax.spines["bottom"].set_bounds(lo, 100)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D # noqa: E402
# Figure 1: cohort 2 GM, reported vs count-consistent values on the attainable grid.
# A dot plot: position, not length, carries the value, so the axis starts at 50%.
fig, ax = plt.subplots(figsize=(7.2, 5.0))
D = 0.2
rows = [ # y, label, denominator, reported, consistent value, consistent label
(2.6, "Specificity\ncontrols, n = 14", 14, 99, pct(11, 14), "11/14 = 78.6%"),
(1.3, "Sensitivity\npatients, n = 14", 14, 64, pct(9, 14), "9/14 = 64.3%"),
(0.0, "Accuracy\nall, n = 28", 28, 72, pct(20, 28), "20/28 = 71.4%"),
]
for y, lab, d, rep, cons, clab in rows:
xs = [pct(k, d) for k in range(d + 1) if pct(k, d) >= 49.9]
ax.vlines(xs, y - 0.08, y + 0.08, color="#b9b8b3", lw=1.2, zorder=1)
ax.plot([rep], [y + D], "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.annotate(f"reported {rep}%", (rep, y + D), xytext=(-9, 0), textcoords="offset points",
ha="right", va="center", fontsize=10, color=INK)
ax.plot([cons], [y - D], "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.annotate(clab, (cons, y - D), xytext=(9, 0), textcoords="offset points",
ha="left", va="center", fontsize=10, color=INK)
imp = (64 + 99) / 2 # accuracy implied by the printed 64% and 99%
ax.plot([imp], [D], "o", ms=9, mfc="none", mec=BLUE, mew=2, zorder=4)
ax.annotate(f"(64 + 99)/2 = {imp:.1f}%,\nimplied by printed values", (imp, D), xytext=(9, 0),
textcoords="offset points", ha="left", va="center", fontsize=10, color=INK2)
ax.text(50, 2.6 + 0.62, "No count out of 14 gives 99% (nearest: 13/14 = 92.9%, 14/14 = 100%).\n"
"No count out of 28 gives 72% (nearest: 20/28 = 71.4%, 21/28 = 75.0%).",
ha="left", va="center", fontsize=10, color=INK2)
ax.set_yticks([r[0] for r in rows])
ax.set_yticklabels([r[1] for r in rows])
ax.set_ylim(-0.5, 3.55)
style_axis(ax)
ax.set_title("Cohort 2 GM classifier: printed values vs nearest consistent counts",
loc="left", fontsize=11.5, color=INK, pad=8)
handles = [
Line2D([], [], ls="", marker="o", ms=9, color=ORANGE, label="Reported in the paper"),
Line2D([], [], ls="", marker="o", ms=9, color=BLUE, label="Recomputed: nearest counts\nconsistent with 64% and 72%"),
Line2D([], [], ls="", marker="o", ms=9, mfc="none", mec=BLUE, mew=2, label="Recomputed: mean of\nprinted sens. and spec."),
Line2D([], [], ls="", marker="|", ms=10, mew=1.2, color="#b9b8b3", label="Attainable values (k/14, k/28)"),
]
fig.legend(handles=handles, loc="lower left", bbox_to_anchor=(0.01, 0.0), ncol=2,
frameon=False, fontsize=10, handletextpad=0.3, columnspacing=1.2)
fig.subplots_adjust(left=0.17, right=0.98, top=0.93, bottom=0.24)
save(fig, "fig1")
# Figure 2: accuracy identity check across the four classifiers
cls = [("Cohort 1 GM\nn = 26 + 26", 73, 77, 69), ("Cohort 1 WM\nn = 26 + 26", 69, 69, 69),
("Cohort 2 GM\nn = 14 + 14", 72, 64, 99), ("Cohort 2 WM\nn = 14 + 14", 78, 71, 86)]
fig, ax = plt.subplots(figsize=(7.2, 3.5))
for i, (lab, acc, se, sp) in enumerate(cls):
y = len(cls) - 1 - i
m = (se + sp) / 2
if abs(m - acc) > 1:
ax.plot([acc, m], [y, y], color=INK2, lw=2, zorder=2)
ax.annotate(f"gap {m - acc:.1f} points", ((acc + m) / 2, y), xytext=(0, 9),
textcoords="offset points", ha="center", va="bottom", fontsize=10, color=INK)
ax.plot([acc], [y], "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.plot([m], [y], "o", ms=13, mfc="none", mec=BLUE, mew=2, zorder=5)
ax.text(1.03, y, f"{acc}% vs ({se} + {sp})/2 = {m:.1f}%", transform=ax.get_yaxis_transform(),
ha="left", va="center", fontsize=10, color=INK)
ax.text(1.03, len(cls) - 0.45, "reported vs recomputed", transform=ax.get_yaxis_transform(),
ha="left", va="center", fontsize=10, color=INK2)
ax.set_yticks(range(len(cls)))
ax.set_yticklabels([c[0] for c in cls][::-1])
ax.set_ylim(-0.6, len(cls) - 0.3)
style_axis(ax)
ax.set_title("Equal groups: accuracy must equal the mean of sensitivity and specificity",
loc="left", fontsize=11.5, color=INK, pad=10)
handles = [
Line2D([], [], ls="", marker="o", ms=9, color=ORANGE, label="Reported accuracy"),
Line2D([], [], ls="", marker="o", ms=12, mfc="none", mec=BLUE, mew=2,
label="Recomputed: mean of reported sensitivity and specificity"),
]
fig.legend(handles=handles, loc="lower left", bbox_to_anchor=(0.01, 0.0), ncol=2,
frameon=False, fontsize=10, handletextpad=0.3)
fig.subplots_adjust(left=0.17, right=0.66, top=0.88, bottom=0.22)
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
24-month remission advantage over conventional psychoeducation: printed NNT, RR and p contradict the rates
Psychol Med (2017) · doi:10.1017/S0033291717000526 · PMC5551382
Summary
This three-arm randomized trial in schizophrenia-spectrum disorders reports that, at 24 months, complete remission was significantly more frequent with the mindfulness-based psychoeducation group (MBPEG) than with conventional psychoeducation (CPEG): 38.9% v. 27.0%, χ² = 4.1, p = 0.010, RR = 1.9, 95% CI 0.9–3.0, NNT = 2.0. These statistics do not agree with the two rates printed next to them, or with each other. The rates give an NNT of 8.4, not 2.0. They give a relative risk of 1.44, not 1.9. A χ² of 4.1 on 1 df gives p = 0.043, not 0.010, and a 95% CI from 0.9 to 3.0 contains 1, so it cannot go with p = 0.010. A 2×2 test on the counts nearest to the printed rates (43/111 v. 30/110) gives χ² = 3.28 and p = 0.070. This comparison is the only test of MBPEG against the active comparator on remission. The Abstract's conclusion that MBPEG improves remission "compared with TAU and CPEG" relies on it in part.
What the paper reports
Results, 'Complete remission' paragraph (Results, paragraph 4):
Complete remissions (4-month simultaneous ratings of all PANSS item scores ⩽3) were not significantly more likely in the MBPEG and CPEG than in the TAU group at times 2 and 3, but significant differences emerged during times 4 and 5. In addition, there was a significant difference on remission rate at the latest post-test between the MBPEG and CPEG [at 24 months: 38.9% v. 27.0%, χ2 = 4.1, p = 0.010, relative risk (RR) = 1.9, 95% confidence interval (CI) 0.9–3.0, number needed to treat (NNT) = 2.0].
Results, next paragraph (the model-based results):
The differences on the estimated odds between the two intervention groups, and between the MBPEG and TAU-alone group, were significant at 12 months (Δ = 1.6 and 1.3, 95% CI 0.58–2.83 and 0.45–2.31, p = 0.022 and 0.039) and 24 months follow-up (Δ = 2.9 and 1.9, 95% CI 0.98–4.38 and 0.65–2.89, p = 0.001 and 0.010). Mixed-effects logistic regression indicated a significant differential change in proportional ORs over the 24-month follow-up; 37 and 26% of the MBPEG and CPEG participants, respectively, but only 7.2% of the TAU group were in remission (χ2 = 8.9 and 8.0, p = 0.001 and 0.003, RR = 3.5 and 3.1, 95% CI 2.0–7.2 and 1.6–6.3, NNT = 3.2 and 4.6).
Abstract, Results and Conclusions:
The MBPEG and CPEG participants had significant differential changes in proportional odds ratios of complete remission (all individual PANSS items <3) over the 24-month follow-up (37 and 26%, respectively), as opposed to only 7.2% of the TAU group (χ2 = 8.9 and 8.0, p = 0.001 and 0.003, relative risk = 3.5 and 3.1, 95% confidence interval 2.0–7.2 and 1.6–6.3).
Compared with TAU and CPEG, MBPEG improves remission and hospitalization rates of people with schizophrenia spectrum disorders over 24 months.
Discussion, paragraph 1:
Nevertheless, significant greater symptom reduction and thus complete remission rates were observed among the participants receiving MBPEG when compared with the other two study groups at both 12- and 24-month follow-up. At 24-month follow-up, more than one-third of the participants receiving MBPEG were in complete remission, compared with about 26 and 7% of those receiving CPEG and TAU alone, respectively.
Methods, 'Statistical analyses':
Mixed-effects multilevel models were used to analyse the overall significance of the model (Wald χ2 statistic) and modelled (intention-to-treat) group differences on remission rates at 24-month follow-up (i.e. whether MBPEG and/or CPEG plus TAU was better or worse than TAU alone at the last follow-up time-point).
With only a few missing data over the 24-month follow-ups, the last data were brought forward for data imputation.
Results, paragraph 1:
Table 2 summarizes the demographic and clinical characteristics of the 342 patients in the three study groups (n = 114 in each group)
Table 3, remission rows (columns are times 2–5; time 5 is 24 months; column headings MBPEG n = 111, CPEG n = 110, TAU n = 112). The footnotes read "eχ2 Frequency between MBPEG and TAU, CPEG and TAU.fRR using TAU as reference."
| Row | MBPEG t2 | t3 | t4 | t5 | CPEG t2 | t3 | t4 | t5 | TAU t2 | t3 | t4 | t5 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Remission rate, % | 9.5 | 15.1 | 29.7 | 38.9 | 8.9 | 13.0 | 24.8 | 27.0 | 6.9 | 8.7 | 9.8 | 7.4 |
| χ² (p) v. TAU | 1.02 (0.50) | 3.0 (0.09) | 4.9 (0.01) | 5.9 (0.001) | 0.94 (0.55) | 2.8 (0.09) | 4.3 (0.03) | 4.0 (0.01) | – | – | – | – |
| RR v. TAU (95% CI) | 1.3 (0.5–2.4) | 1.7 (0.7–3.0) | 2.5 (1.3–4.9) | 2.0 (1.1–4.2) | 1.2 (0.4–0.2) | 1.6 (0.6–2.8) | 2.9 (1.1–4.1) | 2.8 (0.7–3.1) | – | – | – | – |
| NNT | 35 | 9.6 | 5.4 | 3.4 | 36 | 10.3 | 6.5 | 5.0 | – | – | – | – |
What the paper's own numbers show
Method. I took the two rates printed in the MBPEG v. CPEG sentence (38.9% and 27.0%, the same as Table 3 at time 5) and computed the standard quantities that the sentence reports. These are the risk difference and NNT = 1/(risk difference), the rate ratio and the odds ratio. I also checked the p value against the printed χ² (upper tail of χ² with 1 df, and with 2 df), and the printed CI against the printed RR and p value. For the Wald check, SE(log RR) = (ln 3.0 − ln 0.9)/(2 × 1.96), and p comes from z = ln(RR)/SE. The paper gives no remission counts, so I took the integer counts nearest to the printed rates for the Table 3 denominators (43/111 and 30/110) and for the randomized groups of 114 (44/114 and 31/114). I ran Pearson, Yates-corrected and Fisher exact tests on both 2×2 tables. I then found the number of patients per group that the printed rates would need to reach χ² = 4.1 or p = 0.010. Rounding of the printed rates (±0.05 points) is carried through. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| NNT, MBPEG v. CPEG, 24 months | Results ¶4 | 2.0 | 1/(0.389 − 0.270) = 8.40 (8.33–8.47 over the rounding range); 9.09 from the Abstract's 37% v. 26% |
| Risk difference that NNT 2.0 implies | derived | NNT 2.0 | 0.50 (50 points); with CPEG at 27.0%, MBPEG would need 77.0% |
| Relative risk, MBPEG v. CPEG | Results ¶4 | 1.9 | 0.389/0.270 = 1.44 (1.436–1.445); odds ratio 1.72; RR 1.9 would need MBPEG at 51.3% |
| Point estimate implied by the CI 0.9–3.0 | Results ¶4 | 1.9 | log-scale midpoint 1.64 |
| p for the printed χ² = 4.1 | Results ¶4 | 0.010 | 0.043 (df = 1); 0.129 (df = 2); p = 0.010 needs χ² ≥ 6.63 |
| p implied by the printed CI 0.9–3.0 | Results ¶4 | 0.010 | Wald p = 0.106 centred on 1.64 (0.037 if centred on 1.9); the lower limit is below 1 under any rounding (≤ 0.95) |
| CI that p = 0.010 needs at RR 1.9 | derived | 0.9–3.0 | 1.17–3.10 |
| 2×2 test, 43/111 (38.7%) v. 30/110 (27.3%) | Table 3 n | χ² 4.1, p 0.010 | Pearson χ² = 3.28, p = 0.070; Yates χ² = 2.79, p = 0.095; Fisher p = 0.086; RR 1.42 (95% CI 0.97–2.09); NNT 8.7 |
| 2×2 test, 44/114 (38.6%) v. 31/114 (27.2%) | ITT n = 114 | χ² 4.1, p 0.010 | Pearson χ² = 3.36, p = 0.067; Yates 2.86, p = 0.091; Fisher p = 0.090; RR 1.42 (0.97–2.07); NNT 8.8 |
| Largest χ² the printed rates allow with 114 per group | derived | χ² 4.1 | 3.65 (p = 0.056); 3.72 (p = 0.054) at the most favourable rounding |
| Patients per group the printed rates need | derived | 114 randomized per group | 120 for p = 0.05; 128 for χ² = 4.1; 207 for p = 0.010 |
| Integer counts matching 38.9%, 27.0% and 7.4% | Table 3 n 111 / 110 / 112 | 38.9 / 27.0 / 7.4% | none exists; nearest 43/111 = 38.7%, 30/110 = 27.3%, 8/112 = 7.1% |
The NNT is wrong by a factor of about four, which rounding cannot explain. The RR does not fit the rates and is not centred in its own interval. The p value does not fit the χ² printed beside it or the interval printed beside it. Every test reconstructed from the printed rates gives p between 0.054 and 0.095, and a Wald test centred on the printed interval gives 0.106. Even at the paper's full randomized size of 114 per group, rates of 38.9% and 27.0% do not reach p < 0.05 with a Pearson χ² test.
Context: the other remission statistics follow the same pattern. These are secondary, but they bear on whether the rates or the statistics are the problem. In Table 3 the 24-month RRs against TAU (2.0 and 2.8) do not match the rate ratios (38.9/7.4 = 5.26 and 27.0/7.4 = 3.65). The CPEG interval at 24 months, 0.7–3.1, includes 1 next to "4.0 (0.01)". At time 2 the CPEG RR is printed as "1.2 (0.4–0.2)", where the upper limit is below both the lower limit and the estimate. Several χ²–p pairs disagree. For 1 df, 5.9 gives p = 0.015 (printed 0.001), 4.9 gives 0.027 (printed 0.01) and 4.0 gives 0.046 (printed 0.01). The same 24-month comparisons with TAU appear as χ² 5.9 and 4.0, RR 2.0 and 2.8 in Table 3, and as χ² 8.9 and 8.0, RR 3.5 and 3.1 in the Results and Abstract. The NNTs against TAU fit the rates more closely: 3.2 and 5.1 from the table rates (table: 3.4 and 5.0), and 3.36 and 5.32 from the text rates (text: 3.2 and 4.6). The 2.0 for MBPEG v. CPEG is the outlier.
Figure
Figure 1. The relative risk of complete remission at 24 months, MBPEG v. CPEG, on a log scale. The thin vertical line is RR = 1 (no difference). Orange shows what the Results report: RR 1.9 with 95% CI 0.9–3.0. Blue shows three values recomputed from the paper's own numbers. The first is the log-scale midpoint of the reported interval (1.64). The second is the ratio of the two printed rates (1.44). The third is the RR and 95% CI from the integer counts nearest to the printed rates, 43/111 v. 30/110: 1.42 (0.97–2.09). The reported point sits off-centre in its own interval, and the rates give a smaller ratio. Every interval drawn crosses RR = 1, which does not fit a reported p of 0.010.
Figure 2. Left: the two-sided p value for MBPEG v. CPEG remission at 24 months, on a log scale, with p = 0.05 marked. Orange is the reported p = 0.010. Blue values are recomputed: from the printed χ² = 4.1 with 1 df (0.043); from 2×2 tests on 43/111 v. 30/110 (Pearson 0.070, Fisher exact 0.086, Yates 0.095); and from a Wald test on the printed CI 0.9–3.0 (0.106). Right: the number needed to treat, on a linear axis starting at 0. The reported NNT is 2.0. The NNT from the Table 3 rates is 8.4, from the reconstructed counts 8.7, and from the Abstract's 37% v. 26% 9.1.
Why it matters
The trial has an active comparator, so the MBPEG v. CPEG contrast is the one that separates the mindfulness component from psychoeducation as such. On remission, the Results sentence quoted above is the only direct test of that contrast. Its printed statistics make the difference look both significant and large. An NNT of 2.0 would mean one extra remission for every two patients given MBPEG instead of CPEG. The paper's own rates give one extra remission for every eight to nine patients, about a quarter of the stated benefit. They give a relative risk near 1.4 with a 95% interval that reaches below 1. Every test reconstructed from the rates gives p between 0.054 and 0.095, and a Wald test centred on the printed interval gives 0.106. Only the printed χ² of 4.1, taken alone, would give p below 0.05 (0.043), and no reconstruction from the rates reaches that χ². On the printed rates, this comparison is not statistically significant at the paper's stated α of 0.05.
The Abstract's Conclusions say "Compared with TAU and CPEG, MBPEG improves remission", and the Discussion says remission was significantly greater with MBPEG "when compared with the other two study groups at both 12- and 24-month follow-up". The Abstract's own remission statistics compare each intervention only with TAU. The 'versus CPEG' part of the remission claim therefore rests on the sentence examined here, and on a model-based odds difference (Δ = 2.9, 95% CI 0.98–4.38, p = 0.001). That model result cannot be checked from what is published: no coefficients or counts are given, and Δ is not defined as a difference or a ratio.
Other results are not affected. The superiority of both interventions over TAU on remission survives any reconstruction: the nearest counts give χ² = 31.6 for MBPEG v. TAU (43/111 v. 8/112) and χ² = 15.9 for CPEG v. TAU (30/110 v. 8/112). The length-of-rehospitalization and PANSS results are not involved either. However, the other remission statistics (described above) show that the reported χ², p, RR and CI values throughout the remission analysis cannot be relied on as printed.
Explanations considered
- Rounding of the printed rates. This does not account for the numbers. Over the full rounding range (38.85–38.95% v. 26.95–27.05%), the NNT stays between 8.33 and 8.47 and the RR between 1.436 and 1.445.
- The NNT, RR, CI and p come from a model, not the raw rates. This does not account for the numbers. The sentence gives the statistics in one bracket with the raw rates and calls them χ², RR and NNT, and it names no model. An adjusted model could move the RR somewhat. It could not make a 95% CI of 0.9–3.0 go with p = 0.010, or a χ² of 4.1 on 1 df give p = 0.010. An NNT of 2.0 needs a risk difference of 50 points, which no adjustment of an 11.9-point raw difference would plausibly produce. The paper does report model-based remission rates (37% v. 26%), but those give an NNT of 9.1 and an RR of 1.42, which is no closer.
- A different number of degrees of freedom or a one-sided test. This does not account for the numbers. χ² = 4.1 gives p = 0.129 on 2 df. A one-sided version of the 1-df test would give 0.021, not 0.010. And the CI includes 1 whichever way it is read.
- NNT computed against the wrong group (TAU). This does not account for the numbers. Against TAU, the 24-month NNT is 3.2 from the table rates and 3.4 from the text rates, not 2.0.
- A different denominator. This does not account for the numbers. The paper gives three denominators: 342 randomized (114 per group) in the Results, n = 111/110/112 in the Table 3 headings, and n = 265 in the Table 3 caption. None of 38.9%, 27.0% or 7.4% is an exact integer count of the Table 3 group sizes, so the rates may rest on yet other denominators. With the printed rates, however, a Pearson χ² stays below 3.84 for any group size up to the 114 randomized per group. Reaching χ² = 4.1 would take about 128 per group, and p = 0.010 about 207 per group.
- A conversion or typesetting error in the archived text. This is excluded. The publisher's PDF has the same sentence with the same values, and so do the Cambridge Core HTML and the Europe PMC XML (see Status).
- Explained in a supplement or later corrected. This is excluded. There is no supplementary data file with remission counts, and the only correction notice concerns a different issue (see Status).
Status
- Version of record. The publisher's PDF from Cambridge (S0033291717000526a.pdf, 16 pages) has the same MBPEG v. CPEG sentence (38.9% v. 27.0%, χ2 = 4.1, p = 0.010, RR 1.9, 95% CI 0.9–3.0, NNT 2.0). It also has the same Abstract remission sentence and Conclusions. I read it with PyMuPDF on 2026-10-24. The workflow's version-of-record check found the same values in the Cambridge Core HTML, the Europe PMC full-text XML and the publisher's Table 3 image. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. The Table 3 rows are transcribed from the PMC table. - Supplements. The article has no supplementary data file. The Europe PMC supplementary files hold only the publisher's figure and table images. The appendix table at the end of the publisher's PDF gives means and SDs of the continuous outcomes by clinic and time. It has no remission counts, RRs or NNTs, so the correct statistics cannot be computed from published material.
- Corrections and comments. One notice exists: a Corrigendum, doi:10.1017/S0033291723000041 (PMC10123818), published online 2023-01-20. It states that this multi-site trial shared part of its data and results with an earlier single-site (Hong Kong) report of the same registered trial (NCT01667601). It does not change any remission statistic. Crossref lists no other update, and PMC and Europe PMC list no other notice. A web search found no comment on these statistics. PubPeer refused automated access (HTTP 403, Cloudflare challenge), so PubPeer could not be checked directly. Status as of 2026-10-24.
- Sources. The PMC article PMC5551382; the publisher's PDF and the Cambridge Core page via doi:10.1017/S0033291717000526; the Europe PMC full text and supplementary files for PMC5551382; Crossref; the Corrigendum PMC10123818; ClinicalTrials.gov NCT01667601 (cited in the paper).
Suggested correction
- Report the remission counts and denominators for each group at each follow-up, and state which population (completers, LOCF-imputed intention-to-treat, or other) the Table 3 rates use.
- For MBPEG v. CPEG at 24 months, report a test, RR, 95% CI and NNT that match those counts. With the printed rates 38.9% v. 27.0%, these are approximately RR 1.4 (95% CI about 0.97–2.1), NNT 8–9 and χ² ≈ 3.3 (p ≈ 0.07). On that basis, remove "significant" from the Results sentence.
- Revise the Abstract's Conclusions and the Discussion so that MBPEG is not described as improving remission compared with CPEG, unless a correctly specified and fully reported model supports it. If the model-based Δ = 2.9 (p = 0.001) is meant to support that claim, define Δ and report the model estimates it comes from.
- Recheck the χ², p, RR and CI entries of the Table 3 remission rows and the corresponding Results and Abstract values against the counts. The values that need it include the RR "1.2 (0.4–0.2)", the χ²–p pairs that do not match on 1 df, the RRs against TAU that differ from the rate ratios, and the table-versus-text differences for the same 24-month comparisons (χ² 5.9 v. 8.9, 4.0 v. 8.0; RR 2.0 v. 3.5, 2.8 v. 3.1).
- Reconcile the sample sizes: 342 randomized (114 per group), n = 111/110/112 in the Table 3 headings, and n = 265 in the Table 3 caption.
Recomputation script (python recompute.py.txt)
"""Recompute the 24-month remission statistics of PMC5551382
(Psychol Med 2017, doi:10.1017/S0033291717000526; MBPEG v. CPEG v. TAU in schizophrenia).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs, copied from the paper as archived in PMC (identical in the publisher PDF):
Results, 'Complete remission' paragraph, MBPEG v. CPEG at 24 months:
38.9% v. 27.0%, chi2 = 4.1, p = 0.010, RR = 1.9, 95% CI 0.9-3.0, NNT = 2.0
Results (next paragraph) and Abstract, model-based: 37% (MBPEG), 26% (CPEG), 7.2% (TAU);
v. TAU: chi2 = 8.9 and 8.0, p = 0.001 and 0.003, RR = 3.5 and 3.1,
95% CI 2.0-7.2 and 1.6-6.3, NNT = 3.2 and 4.6
Table 3 (column n: MBPEG 111, CPEG 110, TAU 112), remission rate %, times 2-5:
MBPEG 9.5 15.1 29.7 38.9 | CPEG 8.9 13.0 24.8 27.0 | TAU 6.9 8.7 9.8 7.4
chi2 (p) v. TAU: MBPEG 1.02 (0.50) 3.0 (0.09) 4.9 (0.01) 5.9 (0.001)
CPEG 0.94 (0.55) 2.8 (0.09) 4.3 (0.03) 4.0 (0.01)
RR v. TAU (95% CI): MBPEG 1.3 (0.5-2.4) 1.7 (0.7-3.0) 2.5 (1.3-4.9) 2.0 (1.1-4.2)
CPEG 1.2 (0.4-0.2) 1.6 (0.6-2.8) 2.9 (1.1-4.1) 2.8 (0.7-3.1)
NNT: MBPEG 35 9.6 5.4 3.4 | CPEG 36 10.3 6.5 5.0
Results: 342 randomized, n = 114 in each group (intention-to-treat).
"""
import math
import os
import numpy as np
from scipy import stats
from scipy.stats import chi2 as CHI2, norm, chi2_contingency, fisher_exact
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.isf(0.025)
# ------------------------------------------------------------------ inputs from the paper
P_MB, P_CP, P_TAU = 0.389, 0.270, 0.074 # Table 3, time 5 (and Results p4 for MB, CP)
T_MB, T_CP, T_TAU = 0.37, 0.26, 0.072 # Abstract / Results p5 (model-based)
N_MB, N_CP, N_TAU = 111, 110, 112 # Table 3 column headings
N_ITT = 114 # per group, Results
REP = dict(chi2=4.1, p=0.010, rr=1.9, lo=0.9, hi=3.0, nnt=2.0)
R = {}
print("=== 1. MBPEG v. CPEG at 24 months, from the printed rates 38.9% v. 27.0% ===")
rd = P_MB - P_CP
R["rd"], R["nnt"] = rd, 1 / rd
R["rr"] = P_MB / P_CP
R["or"] = (P_MB / (1 - P_MB)) / (P_CP / (1 - P_CP))
print(f" risk difference = {rd:.3f} -> NNT = 1/RD = {1/rd:.2f} (reported NNT 2.0)")
lo_rd, hi_rd = (P_MB - 0.0005) - (P_CP + 0.0005), (P_MB + 0.0005) - (P_CP - 0.0005)
print(f" rounding range of RD = {lo_rd:.3f}-{hi_rd:.3f} -> NNT {1/hi_rd:.2f}-{1/lo_rd:.2f}")
print(f" RD that NNT 2.0 needs = {1/REP['nnt']:.3f} (50 percentage points)")
print(f" MBPEG rate NNT 2.0 needs with CPEG at 27.0% = {P_CP + 0.5:.3f}")
print(f" rate ratio (RR) = {R['rr']:.3f} (reported RR 1.9)")
rr_lo, rr_hi = (P_MB - 0.0005) / (P_CP + 0.0005), (P_MB + 0.0005) / (P_CP - 0.0005)
print(f" rounding range of RR = {rr_lo:.3f}-{rr_hi:.3f}")
print(f" odds ratio (OR) = {R['or']:.3f}")
print(f" MBPEG rate RR 1.9 needs with CPEG at 27.0% = {1.9 * P_CP:.3f}")
print(f" from the Abstract/Results-p5 rates 37% v. 26%: RD = {T_MB-T_CP:.3f}, NNT = {1/(T_MB-T_CP):.2f}, "
f"RR = {T_MB/T_CP:.3f}, OR = {(T_MB/(1-T_MB))/(T_CP/(1-T_CP)):.3f}")
R["nnt_text"] = 1 / (T_MB - T_CP)
print("\n=== 2. The printed chi2, p and CI against one another ===")
R["p_chi41_df1"] = CHI2.sf(REP["chi2"], 1)
R["p_chi41_df2"] = CHI2.sf(REP["chi2"], 2)
R["chi_for_p010"] = CHI2.isf(REP["p"], 1)
print(f" one-sided p for chi2 = 4.1 (z = {math.sqrt(4.1):.3f}): {norm.sf(math.sqrt(4.1)):.4f}")
print(f" p for chi2 = 4.1, df = 1: {R['p_chi41_df1']:.4f}; df = 2: {R['p_chi41_df2']:.4f} (reported p = 0.010)")
print(f" chi2 (df = 1) needed for p = 0.010: {R['chi_for_p010']:.2f}")
lo, hi = REP["lo"], REP["hi"]
gm = math.sqrt(lo * hi)
se = (math.log(hi) - math.log(lo)) / (2 * Z975)
R["gm"], R["se_ci"] = gm, se
print(f" 95% CI 0.9-3.0: geometric (log-scale) midpoint = {gm:.3f} (reported point 1.9); SE(log RR) = {se:.3f}")
for est, key in ((gm, "p_ci_mid"), (1.9, "p_ci_19")):
z = math.log(est) / se
R[key] = 2 * norm.sf(abs(z))
print(f" Wald test centred on {est:.2f}: z = {z:.2f}, two-sided p = {R[key]:.3f}")
lo_max = 0.95
print(f" largest lower limit that rounds to 0.9 is {lo_max} < 1, so the interval contains 1 under any rounding;")
print(f" a 95% CI that contains 1 corresponds to a two-sided p > 0.05 for the matching test")
se_need = math.log(1.9) / norm.isf(0.005)
print(f" for p = 0.010 at RR 1.9 the 95% CI would have to be "
f"{math.exp(math.log(1.9)-Z975*se_need):.2f}-{math.exp(math.log(1.9)+Z975*se_need):.2f}")
R["ci_need"] = (math.exp(math.log(1.9) - Z975 * se_need), math.exp(math.log(1.9) + Z975 * se_need))
print("\n=== 3. Integer counts that reproduce the printed rates ===")
def counts_for(rate_pct, n):
return [a for a in range(n + 1) if round(100 * a / n, 1) == rate_pct]
for lab, pct, n in (("MBPEG 38.9%", 38.9, N_MB), ("CPEG 27.0%", 27.0, N_CP), ("TAU 7.4%", 7.4, N_TAU)):
exact = counts_for(pct, n)
near = sorted(range(n + 1), key=lambda a: abs(a / n - pct / 100))[:2]
print(f" {lab} of n = {n}: exact integer count {'none' if not exact else exact}; nearest "
+ ", ".join(f"{a}/{n} = {100*a/n:.1f}%" for a in near))
for lab, pct in (("38.9%", 38.9), ("27.0%", 27.0), ("7.4%", 7.4)):
ns = [n for n in range(90, 115) if counts_for(pct, n)]
print(f" denominators 90-114 for which {lab} is an exact count: {ns}")
print("\n=== 4. 2x2 tests on reconstructed counts (MBPEG v. CPEG, 24 months) ===")
R["recon"] = []
for a, n1, c, n2 in ((43, N_MB, 30, N_CP), (44, N_ITT, 31, N_ITT)):
t = np.array([[a, n1 - a], [c, n2 - c]])
x2, p, _, _ = chi2_contingency(t, correction=False)
x2y, py, _, _ = chi2_contingency(t, correction=True)
pf = fisher_exact(t)[1]
r1, r2 = a / n1, c / n2
rr = r1 / r2
se_c = math.sqrt(1 / a - 1 / n1 + 1 / c - 1 / n2)
ci = (math.exp(math.log(rr) - Z975 * se_c), math.exp(math.log(rr) + Z975 * se_c))
R["recon"].append(dict(a=a, n1=n1, c=c, n2=n2, x2=x2, p=p, x2y=x2y, py=py, pf=pf, rr=rr, ci=ci, nnt=1 / (r1 - r2)))
print(f" {a}/{n1} ({100*r1:.1f}%) v. {c}/{n2} ({100*r2:.1f}%): Pearson chi2 = {x2:.2f}, p = {p:.3f}; "
f"Yates chi2 = {x2y:.2f}, p = {py:.3f}; Fisher p = {pf:.3f}; RR = {rr:.2f} (95% CI {ci[0]:.2f}-{ci[1]:.2f}); "
f"NNT = {1/(r1-r2):.1f}")
def pearson_equal_n(p1, p2, n):
pb = (p1 + p2) / 2
return (p1 - p2) ** 2 / (pb * (1 - pb) * (2 / n))
x2_114 = pearson_equal_n(P_MB, P_CP, N_ITT)
n_41 = 4.1 / pearson_equal_n(P_MB, P_CP, 1)
n_663 = R["chi_for_p010"] / pearson_equal_n(P_MB, P_CP, 1)
n_384 = CHI2.isf(0.05, 1) / pearson_equal_n(P_MB, P_CP, 1)
x2_114_max = pearson_equal_n(P_MB + 0.0005, P_CP - 0.0005, N_ITT)
R.update(x2_114=x2_114, n_41=n_41, n_663=n_663, n_384=n_384, x2_114_max=x2_114_max)
print(f" exact rates 0.389 v. 0.270 with n = 114 per group (all randomized): chi2 = {x2_114:.2f}, p = {CHI2.sf(x2_114,1):.3f}")
print(f" most favourable rounding (0.3895 v. 0.2695), n = 114 per group: chi2 = {x2_114_max:.2f}, "
f"p = {CHI2.sf(x2_114_max,1):.3f}")
print(f" n per group at which these rates give chi2 = 3.84 (p = 0.05): {n_384:.0f}; chi2 = 4.1: {n_41:.0f}; "
f"chi2 = 6.63 (p = 0.010): {n_663:.0f} (114 were randomized per group)")
print("\n=== 5. Context: other remission statistics in Table 3 and Results p5 ===")
tab = {
"MBPEG": dict(rate=[9.5, 15.1, 29.7, 38.9], chi=[1.02, 3.0, 4.9, 5.9], p=[0.50, 0.09, 0.01, 0.001],
rr=[1.3, 1.7, 2.5, 2.0], ci=[(0.5, 2.4), (0.7, 3.0), (1.3, 4.9), (1.1, 4.2)], nnt=[35, 9.6, 5.4, 3.4]),
"CPEG": dict(rate=[8.9, 13.0, 24.8, 27.0], chi=[0.94, 2.8, 4.3, 4.0], p=[0.55, 0.09, 0.03, 0.01],
rr=[1.2, 1.6, 2.9, 2.8], ci=[(0.4, 0.2), (0.6, 2.8), (1.1, 4.1), (0.7, 3.1)], nnt=[36, 10.3, 6.5, 5.0]),
}
tau = [6.9, 8.7, 9.8, 7.4]
for g, d in tab.items():
for i, t in enumerate((2, 3, 4, 5)):
nnt = 100 / (d["rate"][i] - tau[i])
rr = d["rate"][i] / tau[i]
print(f" {g} v. TAU time {t}: rates {d['rate'][i]} v. {tau[i]} -> NNT {nnt:.1f} (table {d['nnt'][i]}), "
f"RR {rr:.2f} (table {d['rr'][i]} {d['ci'][i]}), p for chi2 {d['chi'][i]} = {CHI2.sf(d['chi'][i],1):.3f} "
f"(table {d['p'][i]})")
print(" Results p5 / Abstract (model-based rates 37, 26 v. 7.2%):")
for g, r, x2, p, rr, nnt in (("MBPEG", T_MB, 8.9, 0.001, 3.5, 3.2), ("CPEG", T_CP, 8.0, 0.003, 3.1, 4.6)):
print(f" {g}: NNT {1/(r-T_TAU):.2f} (printed {nnt}), RR {r/T_TAU:.2f} (printed {rr}), "
f"p for chi2 {x2} = {CHI2.sf(x2,1):.4f} (printed {p})")
print(" same 24-month comparison, Table 3 v. text: MBPEG v. TAU chi2 5.9 v. 8.9, RR 2.0 v. 3.5, NNT 3.4 v. 3.2; "
"CPEG v. TAU chi2 4.0 v. 8.0, RR 2.8 v. 3.1, NNT 5.0 v. 4.6")
print(" Table 3 CPEG time-2 RR '1.2 (0.4-0.2)': upper limit below the lower limit and below the point estimate")
for lab, a, n1 in (("MBPEG", 43, N_MB), ("CPEG", 30, N_CP)):
t = np.array([[a, n1 - a], [8, N_TAU - 8]])
x2, p, _, _ = chi2_contingency(t, correction=False)
pf = fisher_exact(t)[1]
R[f"tau_{lab}"] = (x2, p, pf)
print(f" reconstructed {lab} {a}/{n1} v. TAU 8/{N_TAU} ({100*8/N_TAU:.1f}%): chi2 = {x2:.1f}, p = {p:.1e}, Fisher p = {pf:.1e}")
# ------------------------------------------------------------------ figures
BG, TXT, SEC, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": TXT, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight")
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight")
plt.close(fig)
def style(ax):
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# Figure 1: relative risk, reported v. recomputed, one log axis
rc = R["recon"][0]
rows = [
("Reported (Results)\nRR 1.9, 95% CI 0.9–3.0", 1.9, (0.9, 3.0), ORANGE, "o"),
("Log-scale midpoint of the\nreported CI", R["gm"], None, BLUE, "D"),
("Ratio of the printed rates\n38.9% / 27.0%", R["rr"], None, BLUE, "o"),
(f"From counts {rc['a']}/{rc['n1']} v. {rc['c']}/{rc['n2']}\n(nearest to the printed rates)", rc["rr"], rc["ci"], BLUE, "o"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.3))
for i, (lab, est, ci, col, mk) in enumerate(rows):
y = len(rows) - 1 - i
if ci:
ax.plot(ci, [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([ci[0]] * 2, [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot([ci[1]] * 2, [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot(est, y, marker=mk, ms=9, color=col, zorder=4, ls="none")
txt = f"{est:.2f}" if not ci else f"{est:.2f} ({ci[0]:.2f}–{ci[1]:.2f})"
if i == 0:
txt = "1.9 (0.9–3.0)"
ax.annotate(txt, (est, y), xytext=(0, 9), textcoords="offset points", ha="center", va="bottom", fontsize=10, color=TXT)
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.text(1.015, len(rows) - 0.35, "RR = 1 (no difference)", ha="left", va="bottom", fontsize=10, color=SEC,
transform=ax.transData)
ax.set_xscale("log")
ax.set_xlim(0.75, 3.6)
ticks = [0.8, 1, 1.5, 2, 3]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows[::-1]], fontsize=10)
ax.set_ylim(-0.6, len(rows) - 0.1)
ax.set_xlabel("Relative risk of complete remission at 24 months, MBPEG v. CPEG (log scale)")
style(ax)
leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper"),
Line2D([], [], color=BLUE, marker="o", ms=8, lw=2, label="Recomputed from the paper's own numbers")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.3, -0.24), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: p value and NNT, reported v. recomputed (two panels, each with one x axis)
fig, (a1, a2) = plt.subplots(1, 2, figsize=(7.2, 3.4), gridspec_kw=dict(width_ratios=[1.25, 1], wspace=0.08))
prow = [
("Reported", REP["p"], ORANGE),
("Printed χ² = 4.1, df = 1", R["p_chi41_df1"], BLUE),
("Counts, Pearson χ²", rc["p"], BLUE),
("Counts, Fisher exact", rc["pf"], BLUE),
("Counts, Yates χ²", rc["py"], BLUE),
("Printed CI 0.9–3.0 (Wald)", R["p_ci_mid"], BLUE),
]
for i, (lab, v, col) in enumerate(prow):
y = len(prow) - 1 - i
a1.plot(v, y, "o", ms=9, color=col, zorder=4)
a1.annotate(f"{v:.3f}", (v, y), xytext=(8, 0), textcoords="offset points", va="center", fontsize=10, color=TXT)
a1.axvline(0.05, color=SEC, lw=0.8, zorder=1)
a1.text(0.053, len(prow) - 0.45, "p = 0.05", ha="left", va="bottom", fontsize=10, color=SEC)
a1.set_xscale("log")
a1.set_xlim(0.006, 0.3)
pt = [0.01, 0.02, 0.05, 0.1, 0.2]
a1.set_xticks(pt)
a1.set_xticklabels([f"{t:g}" for t in pt])
a1.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
a1.set_yticks(range(len(prow)))
a1.set_yticklabels([r[0] for r in prow[::-1]], fontsize=10)
a1.set_ylim(-0.6, len(prow) - 0.0)
a1.set_xlabel("Two-sided p, MBPEG v. CPEG (log scale)")
a1.set_title("p value", fontsize=10.5, loc="left", color=TXT)
style(a1)
nrow = [
("Reported", REP["nnt"], ORANGE),
("Table 3 rates\n1 / (0.389 − 0.270)", R["nnt"], BLUE),
(f"Counts {rc['a']}/{rc['n1']} v. {rc['c']}/{rc['n2']}", rc["nnt"], BLUE),
("Abstract rates\n1 / (0.37 − 0.26)", R["nnt_text"], BLUE),
]
for i, (lab, v, col) in enumerate(nrow):
y = len(nrow) - 1 - i
a2.plot(v, y, "o", ms=9, color=col, zorder=4)
a2.annotate(f"{v:.1f}", (v, y), xytext=(0, 8), textcoords="offset points", ha="center", va="bottom", fontsize=10, color=TXT)
a2.set_xlim(0, 10.5)
a2.set_xticks([0, 2, 4, 6, 8, 10])
a2.set_yticks(range(len(nrow)))
a2.set_yticklabels([r[0] for r in nrow[::-1]], fontsize=10)
a2.yaxis.tick_right()
a2.set_ylim(-0.6, len(nrow) - 0.3)
a2.set_xlabel("Number needed to treat (from 0)")
a2.set_title("NNT", fontsize=10.5, loc="left", color=TXT)
style(a2)
leg = [Line2D([], [], color=ORANGE, marker="o", ms=8, ls="none", label="Reported in the paper"),
Line2D([], [], color=BLUE, marker="o", ms=8, ls="none", label="Recomputed from the paper's own numbers")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, -0.02), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png")
The logistic-regression "odds ratios", including those in the abstract, appear to be log-odds coefficients
Psychol Med 2022;52(14):3289-3296 (published online 18 March 2021) · doi:10.1017/S0033291721000088 · PMC9693673
Summary
The paper reports two multivariable logistic regressions for clinically elevated hsCRP (⩾3 mg/L) and labels every estimate an odds ratio (OR). One of them is negative ("OR −4.39, 95% CI −7.04 to −1.97"), and six more have negative lower confidence limits (four distinct intervals; the age and gender intervals appear in both models). An odds ratio cannot be negative. Read as log-odds coefficients, the printed values hang together: every printed CI agrees with its printed p-value on significance at 0.05 (the interval excludes 0 exactly when p < 0.05), and the paper's own Table 1 counts reproduce the estimates closely (unadjusted log-odds 2.39 against the printed 2.44). Read as ORs, the two main-effect intervals in model 2 contradict their p-values. On the coefficient reading, the abstract's headline odds ratios are understated about 2.7- to 4.7-fold: "OR 2.44" corresponds to an OR of about 11.5 (95% CI 3.6 to 51.4), "OR 1.52" to about 4.6 and "OR 1.09" to about 3.0. The direction and significance of each finding are unchanged, but the effect sizes as printed are on the wrong scale, and read literally as ORs, the CI given for depression (0.27 to 2.01) includes 1 and would contradict the stated significance.
What the paper reports
Methods, "Statistical analysis":
To address the second hypothesis, two models of multivariable logistic regression were used. The first assessed how much risk comorbid depression and overweight contributed to the hsCRP levels ⩾3 mg/L relative to other participant groups, with age and gender as covariates. The second assessed how much risk depression and overweight status (BMI ⩾ 25) separately contributed to hsCRP levels ⩾3 mg/L, with age and gender as covariates.
Results, "High sensitivity C-Reactive Protein", first logistic paragraph (model 1):
Overweight patients with depression were at a significantly increased risk of having hsCRP ⩾3 mg/L [OR 2.44, 95% confidence interval (CI) 1.28 to 3.94, p < 0.001], overweight controls did not have a significantly increased risk of hsCRP ⩾3 mg/L (OR 1.51, 95% CI −0.33 to 2.87, p = 0.13), normal-weight patients with depression did not have a significantly increased risk of hsCRP ⩾3 mg/L (OR 1.08, 95% CI −0.6 to 2.4, p = 0.28), while normal-weight controls were at a significantly decreased risk of hsCRP ⩾3 mg/L (OR −4.39, 95% CI −7.04 to −1.97, p < 0.001). Age and gender were not significantly associated with hsCRP ⩾3 mg/L [OR 0.01, 95% CI −0.04 to 0.06, p = 0.621, OR 0.61, 95% CI −0.19 to 1.47, p = 0.145, respectively].
Results, same section, second logistic paragraph (model 2):
Depression status was significantly associated with hsCRP ⩾3 mg/L (OR 1.09, 95% CI 0.27 to 2.01, p = 0.013) and overweight status was significantly associated with hsCRP ⩾3 mg/L (OR 1.52, 95% CI 0.7 to 2.41, p < 0.001). Age and gender were not significantly associated with hsCRP ⩾3 mg/L [OR 0.62, 95% CI −0.04 to 0.06, p = 0.647, OR 0.01, 95% CI −0.19 to 1.47, p = 0.137, respectively].
Abstract, "Results":
Multivariable logistic regression showed that comorbid depression and overweight significantly increased the risk of clinically elevated hsCRP levels ⩾3 mg/L (OR 2.44, 1.28–3.94). In a separate multivariable logistic regression model, overweight status contributed most to the risk of having hsCRP levels ⩾3 mg/L (OR 1.52, 0.7–2.41), while depression also contributed a significant risk (OR 1.09, 0.27–2).
Table 1, row "hsCRP ⩾3 mg/L (%)". The columns are overweight & depressed (N = 69), overweight control (N = 35), normal weight & depressed (N = 55), normal weight control (N = 57), statistic, p:
hsCRP ⩾3 mg/L (%) | 26 (37.7%) | 5 (14.3%) | 6 (10.9%) | 3 (5.3%) | x2 = 25.96 | <0.001
The conclusions that rest on these models. Abstract, "Conclusion":
Comorbid depression and overweight status are associated with increased hsCRP, and the coexistence of these conditions amplified the risk of clinically elevated hsCRP levels. Overweight status contributed most to the risk of clinically elevated hsCRP levels, but depression also contributed to a significant risk.
Discussion, first paragraph:
Of clinical relevance, overweight patients with depression had the highest risk for hsCRP levels ⩾3 mg/L, indicating clinically elevated peripheral inflammation.
What the paper's own numbers show
Method. In logistic regression a coefficient β is a log odds ratio, and the odds ratio is exp(β). An OR must be positive and its null is 1, while a coefficient can take any sign and its null is 0. I made three checks:
- Sign. I checked each printed estimate and CI limit for values ⩽ 0, which no OR can take.
- CI against p-value. I took each printed CI as a 95% Wald interval and back-calculated the two-sided p-value it implies under two readings: as a coefficient (null 0) and as an OR (null 1, computed on the log scale). I also checked, more simply, whether each interval excludes the null exactly when the printed p is below 0.05. The printed intervals are not symmetric about their estimates, and most look like profile-likelihood intervals (see check 3), so a Wald back-calculation from them is only approximate. The significance check does not depend on that assumption.
- Refit from Table 1. I refitted both models without the age and gender covariates, using only the grouped counts in Table 1 (40 cases with hsCRP ⩾3 mg/L among 216 participants). The paper's covariate-adjusted models need individual data, so the refits show the scale and approximate size of the estimates, not their exact values. I report both Wald and profile-likelihood intervals for the refits, because most printed intervals are asymmetric about their estimates in the way profile-likelihood intervals are (2.44 lies 1.16 above its lower limit of 1.28 and 1.50 below its upper limit of 3.94).
Every number below is printed by recompute.py.txt. Quoted passages keep the paper's minus signs (U+2212); elsewhere minus signs are typed as hyphens.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| hsCRP ⩾3 mg/L by group | Table 1 | 26/69, 5/35, 6/55, 3/57; χ² = 25.96, p < 0.001 | 37.7%, 14.3%, 10.9%, 5.3%; Pearson χ² = 25.96, df 3, p = 9.7 × 10⁻⁶ (matches) |
| Overweight + depressed v. reference | Abstract; Results, model 1 | "OR 2.44 (1.28 to 3.94)", p < 0.001 | As a coefficient, the CI implies p = 0.0003 (matches p < 0.001); as an OR, it implies p = 0.002. Unadjusted refit: log-odds 2.39 (profile 1.26 to 3.87; Wald 1.13 to 3.65), p = 0.0002. Crude OR = (26/43)/(3/54) = 10.9. If the printed value is a coefficient, OR = exp(2.44) = 11.5 (3.6 to 51.4) |
| Overweight controls v. reference | Results, model 1 | "OR 1.51 (−0.33 to 2.87)", p = 0.13 | Negative lower limit is impossible for an OR. As a coefficient, the CI includes 0 (not significant, agreeing with p = 0.13), but a Wald back-calculation gives p = 0.064, about half the printed 0.13. Refit: log-odds 1.10 (profile -0.37 to 2.74; Wald -0.40 to 2.60), Wald p = 0.15. exp(printed) = 4.5 (0.72 to 17.6) |
| Normal-weight depressed v. reference | Results, model 1 | "OR 1.08 (−0.6 to 2.4)", p = 0.28 | Negative lower limit is impossible for an OR. As a coefficient, the CI includes 0 (not significant, agreeing with p = 0.28), but a Wald back-calculation gives p = 0.16, about half the printed 0.28. Refit: log-odds 0.79 (profile -0.60 to 2.39; Wald -0.65 to 2.23), Wald p = 0.28. exp(printed) = 2.9 (0.55 to 11.0) |
| "Normal-weight controls" | Results, model 1 | "OR −4.39 (−7.04 to −1.97)", p < 0.001 | Impossible as an OR. With four groups and an intercept, one group must be the reference and gets no coefficient of its own. Normal-weight controls are the reference implied by the other three contrasts. Their observed log-odds is ln(3/54) = -2.89, and the refit intercept is -2.89 (profile -4.30 to -1.89), so −4.39 is most consistent with the model intercept, which in the adjusted model refers to age 0 |
| Depression | Abstract; Results, model 2 | "OR 1.09 (0.27 to 2.01)", p = 0.013 | Read as an OR, the CI contains 1 and implies p = 0.87, contradicting p = 0.013. Read as a coefficient, it implies p = 0.014 (matches). Refit: log-odds 1.12 (profile 0.31 to 2.03), p = 0.010. exp(printed) = 3.0 (1.31 to 7.46) |
| Overweight | Abstract; Results, model 2 | "OR 1.52 (0.7 to 2.41)", p < 0.001 | Read as an OR, the CI contains 1 and implies p = 0.18, contradicting p < 0.001. Read as a coefficient, it implies p = 0.0005 (matches). Refit: log-odds 1.46 (profile 0.68 to 2.32), p = 0.00045. exp(printed) = 4.6 (2.01 to 11.1) |
| Age, gender | Results, model 1 | "OR 0.01 (−0.04 to 0.06)", p = 0.621; "OR 0.61 (−0.19 to 1.47)", p = 0.145 | Negative lower limits are impossible for ORs. As coefficients, implied p = 0.70 and 0.15. exp(printed) = 1.01 (0.96 to 1.06) per year and 1.84 (0.83 to 4.35) |
| Age, gender | Results, model 2 | "OR 0.62 (−0.04 to 0.06)", p = 0.647; "OR 0.01 (−0.19 to 1.47)", p = 0.137 | The age estimate 0.62 lies outside its own CI (-0.04 to 0.06). The two point estimates appear to be transposed relative to their CIs and to model 1 (0.01 with -0.04 to 0.06; 0.61 with -0.19 to 1.47). Both CIs include 0 and both p-values are above 0.05, so significance still agrees on the coefficient reading |
| Size of understatement | Abstract | 2.44, 1.52, 1.09 | exp(printed) / printed = 4.7×, 3.0×, 2.7× |
In the refit of model 1, the printed intervals and the profile-likelihood intervals line up closely: 1.28 to 3.94 against 1.26 to 3.87, -0.33 to 2.87 against -0.37 to 2.74, and -0.6 to 2.4 against -0.60 to 2.39. For overweight + depressed, depression and overweight, the printed intervals also share the refit's upward skew (for example 1.16 below and 1.50 above the estimate, against 1.13 and 1.48 in the refit). That pattern is what R's confint() gives for a glm fit on the coefficient scale. The Methods state that all analyses used R 4.0.0.
The Wald back-calculation is less exact. On the coefficient reading it matches the printed p for overweight + depressed (0.0003 v. < 0.001), "normal-weight controls" (0.0007 v. < 0.001), depression (0.014 v. 0.013), overweight (0.0005 v. < 0.001) and model 1 gender (0.15 v. 0.145), and is close to it for model 1 age (0.70 v. 0.621). For overweight controls (0.064 v. 0.13) and normal-weight depressed (0.16 v. 0.28) it gives about half the printed value, a gap a reader rerunning the script will see. In these two rows the printed limits are close to the refit's limits, but the printed estimates (1.51, 1.08) are higher than the refit estimates (1.10, 0.79). The printed estimates therefore sit nearer the upper limit (1.84 below and 1.36 above; 1.68 below and 1.32 above). The back-calculation divides the estimate by an SE taken from the interval width, so a larger estimate with the same width gives a larger z and a smaller p-value. The printed p-values for these two rows are close to the unadjusted refit Wald p-values (0.15 and 0.28). Because the printed intervals look like asymmetric profile intervals, a Wald back-calculation from them is only approximate, and these two rows cannot be resolved further without the model output. The gap is in size only: on the coefficient reading, no printed interval contradicts its printed p-value on significance at 0.05.
Figure
Each row is one group or main-effect estimate the paper prints with the label "OR"; the age and gender rows are not shown. Orange circles and lines are the printed estimate and 95% CI, on the linear scale on which they are printed. Blue squares and lines are log-odds coefficients refitted, without the age and gender covariates, from the hsCRP ⩾3 mg/L counts in Table 1, with profile-likelihood 95% intervals. The shaded region holds values below 0, which no odds ratio can take. The solid line at 0 is the null for a log-odds coefficient, and the dashed line at 1 is the null for an odds ratio. Whether each printed p-value (right) is below 0.05 matches whether its interval crosses 0, not whether it crosses 1. That is clearest for depression and overweight in model 2, whose intervals cross 1 but are reported as significant. The row labelled "normal-weight controls" has no blue refit because the reference group of a categorical predictor has no contrast of its own.
The same estimates on a log odds-ratio axis. Orange is the printed value taken literally as an OR. Lower limits below 0 cannot be placed on this axis and are drawn as arrows. Blue is exp(printed value), the odds ratio implied if the printed numbers are coefficients, labelled with its point estimate. The intercept-like "normal-weight controls" row is omitted because its printed value is negative. The age and gender rows of both models are not shown either.
Why it matters
The three figures in the abstract are the paper's quantitative answer to its second aim, "what extent overweight status and depression status contributed to clinically elevated hsCRP levels". As printed, they suggest modest associations: odds roughly 2.4, 1.5 and 1.1 times higher. If they are coefficients, the implied odds ratios are about 11.5, 4.6 and 3.0, a 2.7- to 4.7-fold difference. A reader citing these ORs, or pooling them in a meta-analysis, would carry the wrong magnitude. Read literally as ORs, the printed intervals also contradict the paper's own significance statements: the depression interval (0.27 to 2.01) and the overweight interval (0.7 to 2.41) both include 1. A reader who takes the label at face value could therefore conclude that neither main effect is significant, which is the opposite of what the abstract states.
The qualitative conclusions do not depend on the scale. Overweight-and-depressed participants have the highest odds, overweight contributes more than depression, and both are significant in model 2. Those orderings and significance calls are the same on either scale, and the Table 1 refits support them. The second-paragraph age and gender estimates are a separate, smaller transcription problem.
Explanations considered
- Conversion or typesetting artefact (lost or added minus signs). This does not account for the numbers. The evidence check found the same values, with true minus signs (U+2212), in the publisher's HTML and PDF on Cambridge Core and in the PMC and Europe PMC XML. Removing the signs would not help either: an OR of 4.39 would mean increased risk, while the text says normal-weight controls were at "significantly decreased risk".
- The printed values are ORs and only the negative ones are typos. This does not account for them. The positive-only intervals for depression (0.27 to 2.01) and overweight (0.7 to 2.41) would then contain 1 and imply p = 0.87 and p = 0.18, not the printed p = 0.013 and p < 0.001. Separately, the unadjusted crude OR for overweight-depressed v. normal-weight controls is 10.9, far from 2.44.
- The values are coefficients, correctly computed, and labelled "OR" in error. This accounts for everything checked. On the coefficient reading, every printed CI agrees with its printed p-value on significance at 0.05. Of the eight rows that are not transposed, a Wald back-calculation also matches the printed p-value closely for five and is close to it for a sixth, model 1 age (0.70 v. 0.621). It is about a factor of 2 low for overweight controls (0.064 v. 0.13) and normal-weight depressed (0.16 v. 0.28), whose printed p-values are instead close to the unadjusted refit Wald p-values (0.15 and 0.28). Since the printed intervals look like asymmetric profile intervals, the back-calculation is only approximate, and this gap does not point to a different scale: on the OR reading those two rows have impossible negative limits. The unadjusted refits from Table 1 give 2.39, 1.10, 0.79, 1.12 and 1.46 against the printed 2.44, 1.51, 1.08, 1.09 and 1.52, and their profile intervals closely match the printed intervals. The covariates would be expected to shift the estimates somewhat (age and gender differ between groups, Table 1), so small differences are expected.
- The "−4.39" is a genuine group contrast. Unlikely. With an intercept, only three of the four groups can have coefficients, and the three positive coefficients are the contrasts against normal-weight controls. −4.39 fits an intercept: the unadjusted intercept is -2.89, and an adjusted intercept refers to age 0, so a more negative value is expected. Its exact meaning cannot be confirmed without the model output.
- The supplement holds the correct output. It does not. The only supplementary file contains a non-smoker version of Table 1 and a consortium member list, with no regression output.
Status
As of 2026-10-23:
- Version of record. Checked on Cambridge Core (HTML and PDF, S0033291721000088a.pdf) and in the PMC and Europe PMC full-text XML. All four sources carry the same values, including the negative "OR" and the negative limits, so the problem is in the version of record and is not introduced by conversion.
- Supplement. The only supplementary file is S0033291721000088sup001.docx. It holds Supplementary Table 1 (non-smokers: hsCRP ⩾3 mg/L in 23/58, 4/32, 5/44 and 3/51; χ² = 24.405) and the consortium list. It has no logistic-regression output, so it cannot settle whether the values are coefficients or ORs.
- Corrections and notices. No correction, erratum, expression of concern, retraction or comment was found. Crossref shows no
update-to/updated-byrelations. The Crossref Labs updates feed, which carries the Retraction Watch data, is empty. PMC and Europe PMC show no linked notices (Europe PMC record PMID 33731235, no comment/correction list), and the Cambridge Core page shows none either. - Repository copy. The Cardiff ORCA repository has a record for the paper (orca.cardiff.ac.uk/id/eprint/138378). The site refused automated access (HTTP 403), so I did not check its full text or which version it holds. It could show whether the "OR" labels were already in the accepted manuscript.
- PubPeer. PubPeer refused direct access (HTTP 403), and a web search for the title with "PubPeer" found no PubPeer entry. So PubPeer was searched only indirectly.
- Sources. doi.org/10.1017/S0033291721000088; PMC9693673; Crossref REST API
api.crossref.org/works/10.1017/S0033291721000088; Europe PMC REST search.
Suggested correction
- State that the values reported in the logistic-regression paragraphs of the Results and in the abstract are log-odds coefficients (β), or replace them with odds ratios, exp(β), and their exponentiated CIs. If the printed values are the model coefficients, the abstract would read approximately: - "OR 11.5, 95% CI 3.6–51.4" for comorbid depression and overweight; - "OR 4.6, 2.0–11.1" for overweight; - "OR 3.0, 1.3–7.5" for depression.
These values should be confirmed from the original model output. 2. In model 1, report the reference group explicitly (apparently normal-weight controls), and either drop the "−4.39" or label it as the intercept rather than as a risk estimate for normal-weight controls. 3. In the model 2 paragraph, check the age and gender estimates, which appear transposed (0.62 is printed with the interval -0.04 to 0.06). 4. Consider giving the full model output (coefficients, SEs, intercept and coding) in a table or supplement.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC9693673 (Psychol Med 2022;52(14):3289-3296, online 18 March 2021;
# doi:10.1017/S0033291721000088)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2
# (SVG + PNG) plus five candidate figure designs in ./candidates/.
# Inputs are only numbers printed in the paper (Results text and Table 1).
import os
import numpy as np
from scipy import stats, optimize
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs
# Table 1, row "hsCRP >=3 mg/L (%)": events / group N
groups = {"OW_dep": (26, 69), "OW_ctl": (5, 35), "NW_dep": (6, 55), "NW_ctl": (3, 57)}
# Results, hsCRP paragraphs 1 and 2: (estimate, CI low, CI high, reported p) as printed with label "OR"
rep1 = {
"OW_dep": (2.44, 1.28, 3.94, "<0.001"),
"OW_ctl": (1.51, -0.33, 2.87, "0.13"),
"NW_dep": (1.08, -0.6, 2.4, "0.28"),
"NW_ctl": (-4.39, -7.04, -1.97, "<0.001"),
"age": (0.01, -0.04, 0.06, "0.621"),
"gender": (0.61, -0.19, 1.47, "0.145"),
}
rep2 = {
"depression": (1.09, 0.27, 2.01, "0.013"),
"overweight": (1.52, 0.7, 2.41, "<0.001"),
"age (as printed)": (0.62, -0.04, 0.06, "0.647"),
"gender (as printed)": (0.01, -0.19, 1.47, "0.137"),
}
def pnum(s):
return 0.001 if s.startswith("<") else float(s)
def fmt(x, d=2):
return f"{x:.{d}f}"
# ---------------------------------------------------------------- 1. Table 1 arithmetic
print("1. Table 1 hsCRP >=3 mg/L row")
for g, (k, n) in groups.items():
print(f" {g}: {k}/{n} = {100*k/n:.1f}%")
obs = np.array([[k, n - k] for k, n in groups.values()])
chi2, p_chi, dof, _ = stats.chi2_contingency(obs, correction=False)
print(f" Pearson chi2 = {chi2:.2f}, df = {dof}, p = {p_chi:.2g} (reported x2 = 25.96, p < 0.001)")
print(f" total events = {obs[:,0].sum()}, non-events = {obs[:,1].sum()}, N = {obs.sum()}")
# ---------------------------------------------------------------- 2. reading the printed values two ways
def wald_from_ci(b, lo, hi):
se = (hi - lo) / (2 * Z)
return se, 2 * stats.norm.sf(abs(b / se))
def as_or(b, lo, hi):
"""p implied if (b, lo, hi) were an OR and its CI (Wald on log scale)."""
if b <= 0 or lo <= 0:
return None
se = (np.log(hi) - np.log(lo)) / (2 * Z)
return 2 * stats.norm.sf(abs(np.log(b) / se))
readings = {}
for title, rep in (("Model 1 (group status + age + gender)", rep1), ("Model 2 (depression + overweight + age + gender)", rep2)):
print(f"\n2. {title}")
print(" name | printed 'OR' (CI), p | impossible as OR? | p if log-odds | p if OR | exp(printed) = OR if log-odds")
for k, (b, lo, hi, pr) in rep.items():
_, pl = wald_from_ci(b, lo, hi)
po = as_or(b, lo, hi)
imp = (b <= 0) or (lo <= 0)
ci_has_1 = (lo < 1 < hi)
readings[k] = dict(b=b, lo=lo, hi=hi, pr=pr, p_logodds=pl, p_or=po, exp=(np.exp(b), np.exp(lo), np.exp(hi)))
e = f"{np.exp(b):.3g} ({np.exp(lo):.3g} to {np.exp(hi):.3g})"
pos = "impossible" if po is None else f"{po:.3f} (CI contains 1: {ci_has_1})"
print(f" {k}: {b} ({lo} to {hi}), p {pr} | {'YES' if imp else 'no'} | {pl:.3g} | {pos} | {e}")
print(" Significance at 0.05 on the coefficient reading: does 'CI excludes 0' agree with 'printed p < 0.05'?")
for k, (b, lo, hi, pr) in rep.items():
excl0 = not (lo < 0 < hi)
sig = pnum(pr) < 0.05
print(f" {k}: CI excludes 0 = {excl0}; printed p < 0.05 = {sig}; agree = {excl0 == sig}")
print(" Point estimate inside its own printed CI?",
{k: bool(v[1] <= v[0] <= v[2]) for k, v in rep.items()})
# CI asymmetry of the printed headline interval (Wald would be symmetric)
b, lo, hi, _ = rep1["OW_dep"]
print(f"\n OW_dep printed interval: distance below = {b-lo:.2f}, above = {hi-b:.2f} (asymmetric; midpoint {(lo+hi)/2:.2f})")
# ---------------------------------------------------------------- 3. unadjusted logistic fits from Table 1 counts
names = list(groups)
y = np.array([groups[g][0] for g in names], float)
n = np.array([groups[g][1] for g in names], float)
def nll(beta, X):
eta = X @ beta
return -np.sum(y * eta - n * np.logaddexp(0, eta))
def fit(X, fixed=None):
if fixed is None:
r = optimize.minimize(nll, np.zeros(X.shape[1]), args=(X,), method="BFGS", options={"gtol": 1e-10})
return r.x, r.fun
j, val = fixed
free = [i for i in range(X.shape[1]) if i != j]
def f(bf):
bb = np.empty(X.shape[1]); bb[free] = bf; bb[j] = val
return nll(bb, X)
r = optimize.minimize(f, np.zeros(len(free)), method="BFGS", options={"gtol": 1e-10})
return None, r.fun
def summarize(X, labels):
beta, f0 = fit(X)
p = 1 / (1 + np.exp(-X @ beta))
W = n * p * (1 - p)
se = np.sqrt(np.diag(np.linalg.inv(X.T @ (W[:, None] * X))))
out = {}
crit = stats.chi2.ppf(0.95, 1) / 2
for j, lab in enumerate(labels):
g = lambda t: fit(X, (j, t))[1] - f0 - crit
plo = optimize.brentq(g, beta[j] - 12 * se[j], beta[j])
phi = optimize.brentq(g, beta[j], beta[j] + 12 * se[j])
pw = 2 * stats.norm.sf(abs(beta[j] / se[j]))
out[lab] = dict(b=beta[j], se=se[j], wlo=beta[j] - Z * se[j], whi=beta[j] + Z * se[j], plo=plo, phi=phi, p=pw)
print(f" {lab}: log-odds {beta[j]:.2f} (Wald {beta[j]-Z*se[j]:.2f} to {beta[j]+Z*se[j]:.2f}; "
f"profile {plo:.2f} to {phi:.2f}), p = {pw:.2g}; OR = {np.exp(beta[j]):.2f} "
f"(profile {np.exp(plo):.2f} to {np.exp(phi):.2f})")
return out
print("\n3a. Unadjusted model 1 from Table 1 counts (reference = normal-weight controls)")
X1 = np.array([[1, 1, 0, 0], [1, 0, 1, 0], [1, 0, 0, 1], [1, 0, 0, 0]], float)
m1 = summarize(X1, ["intercept", "OW_dep", "OW_ctl", "NW_dep"])
print("\n3b. Unadjusted model 2 from Table 1 counts (main effects, no interaction)")
X2 = np.array([[1, 1, 1], [1, 0, 1], [1, 1, 0], [1, 0, 0]], float) # intercept, depression, overweight
m2 = summarize(X2, ["intercept", "depression", "overweight"])
print("\n3b2. Interval skew: distance from estimate to lower / upper limit, printed v. unadjusted refit (profile)")
for k_rep, k_fit, mm in (("OW_dep", "OW_dep", m1), ("OW_ctl", "OW_ctl", m1), ("NW_dep", "NW_dep", m1),
("depression", "depression", m2), ("overweight", "overweight", m2)):
b_, lo_, hi_, _ = {**rep1, **rep2}[k_rep]
f_ = mm[k_fit]
print(f" {k_rep}: printed {b_-lo_:.2f} below / {hi_-b_:.2f} above; "
f"refit {f_['b']-f_['plo']:.2f} below / {f_['phi']-f_['b']:.2f} above")
print("\n3c. Crude ORs v. normal-weight controls, (a/b)/(c/d)")
crude = {}
for g in ["OW_dep", "OW_ctl", "NW_dep"]:
k, N = groups[g]
crude[g] = (k / (N - k)) / (3 / 54)
print(f" {g}: OR = {crude[g]:.2f}, log OR = {np.log(crude[g]):.2f}")
# intercept plausibility: NW controls' observed log-odds and a rough age-0 shift
lo_nw = np.log(3 / 54)
shift = lo_nw - 0.01 * 30 - 0.61 * 44 / 57
print(f"\n3d. Normal-weight controls: observed log-odds ln(3/54) = {lo_nw:.2f}; "
f"shifted to age 0 and the other gender using printed age 0.01/yr (median age 30) "
f"and gender 0.61 (77.2% female): about {shift:.2f} (printed '-4.39', CI -7.04 to -1.97)")
print("\n3e. Headline ratios, exp(printed) / printed")
for k, r in (("OW_dep", rep1["OW_dep"]), ("overweight", rep2["overweight"]), ("depression", rep2["depression"])):
print(f" {k}: exp({r[0]}) = {np.exp(r[0]):.2f}; ratio to printed = {np.exp(r[0])/r[0]:.1f}x")
# ================================================================ figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "axes.edgecolor": FG2, "axes.labelcolor": FG,
"xtick.color": FG2, "ytick.color": FG, "text.color": FG, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
def style(ax, grid_axis="x"):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis=grid_axis, color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
ROWS = [ # (label, printed tuple, Table-1 refit dict or None)
("Overweight + depressed\n(model 1)", rep1["OW_dep"], m1["OW_dep"]),
("Overweight controls\n(model 1)", rep1["OW_ctl"], m1["OW_ctl"]),
("Normal-weight depressed\n(model 1)", rep1["NW_dep"], m1["NW_dep"]),
("'Normal-weight controls'\n(model 1)", rep1["NW_ctl"], None),
("Depression\n(model 2)", rep2["depression"], m2["depression"]),
("Overweight\n(model 2)", rep2["overweight"], m2["overweight"]),
]
# ---- design A: printed values on their own (linear) scale, both candidate nulls
def design_A(path):
fig, ax = plt.subplots(figsize=(7.2, 4.9))
ypos = np.arange(len(ROWS))[::-1].astype(float)
ax.axvspan(-7.6, 0, color="#f1efea", zorder=0.5, lw=0)
ax.text(-7.45, len(ROWS) - 0.35, "values < 0: impossible for an OR", color=FG2, fontsize=10, va="center")
ax.axvline(0, color=FG, lw=1, zorder=1)
ax.axvline(1, color=FG2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(-0.12, -0.75, "0 = null for a\nlog-odds coefficient", fontsize=10, color=FG, ha="right", va="center")
ax.text(1.12, -0.75, "1 = null for\nan odds ratio", fontsize=10, color=FG2, ha="left", va="center")
for yy, (lab, (b, lo, hi, pr), m) in zip(ypos, ROWS):
ax.plot([lo, hi], [yy + 0.12] * 2, color=ORANGE, lw=2, zorder=3, solid_capstyle="butt")
ax.plot(b, yy + 0.12, "o", ms=8, color=ORANGE, zorder=4)
if m is None:
ax.text(-7.45, yy - 0.42, "no refit: a reference group\nhas no contrast of its own",
fontsize=10, color=BLUE, va="center", ha="left", linespacing=1.1,
bbox=dict(fc="#f1efea", ec="none", pad=1.5), zorder=2)
if m is not None:
ax.plot([m["plo"], m["phi"]], [yy - 0.14] * 2, color=BLUE, lw=2, zorder=3, solid_capstyle="butt")
ax.plot(m["b"], yy - 0.14, "s", ms=7.5, color=BLUE, zorder=4)
ax.text(4.75, yy, f"p {pr}" if pr.startswith("<") else f"p = {pr}", fontsize=10, color=FG2, va="center")
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in ROWS], fontsize=10)
ax.set_xlim(-7.6, 5.6)
ax.set_ylim(-1.6, len(ROWS) - 0.1)
ax.set_xticks(range(-7, 5))
ax.set_xlabel("Value on the scale the paper prints (labelled 'OR', 95% CI)")
style(ax)
ax.text(4.75, len(ROWS) - 0.35, "printed p", fontsize=10, color=FG2, va="center", weight="bold")
h1 = ax.errorbar([], [], xerr=[], fmt="o", color=ORANGE, ms=8, label="Printed in the paper as 'OR (95% CI)'")
h2 = ax.errorbar([], [], xerr=[], fmt="s", color=BLUE, ms=7.5, label="Log-odds refitted from Table 1 counts (unadjusted)")
ax.legend(handles=[h1, h2], loc="lower left", bbox_to_anchor=(-0.36, 1.0), frameon=False, ncol=1, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- design B: odds-ratio (log) axis, printed-as-OR v. exp(printed)
def design_B(path):
rows = [r for r in ROWS if r[1][0] > 0]
fig, ax = plt.subplots(figsize=(7.2, 4.4))
ypos = np.arange(len(rows))[::-1].astype(float)
left = 0.1
ax.axvline(1, color=FG2, lw=1, zorder=1)
for yy, (lab, (b, lo, hi, pr), m) in zip(ypos, rows):
y0 = yy + 0.15
if lo > 0:
ax.plot([lo, hi], [y0] * 2, color=ORANGE, lw=2, zorder=3)
else:
ax.annotate("", xy=(left * 1.03, y0), xytext=(hi, y0),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=12), zorder=3)
ax.text(left * 1.12, y0 + 0.2, f"lower limit {str(lo).replace('-', '\u2212')} < 0", fontsize=10, color=ORANGE, va="center")
ax.plot(b, y0, "o", ms=8, color=ORANGE, zorder=4)
e, elo, ehi = np.exp(b), np.exp(lo), np.exp(hi)
y1 = yy - 0.15
ax.plot([elo, ehi], [y1] * 2, color=BLUE, lw=2, zorder=3)
ax.plot(e, y1, "s", ms=7.5, color=BLUE, zorder=4)
ax.text(ehi * 1.08, y1, f"{e:.1f}", fontsize=10, color=BLUE, va="center")
ax.set_xscale("log")
ax.set_xlim(left, 120)
ax.set_xticks([0.1, 0.3, 1, 3, 10, 30, 100])
ax.set_xticklabels(["0.1", "0.3", "1", "3", "10", "30", "100"])
ax.set_yticks(ypos)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.7, len(rows) - 0.3)
ax.set_xlabel("Odds ratio for hsCRP ⩾3 mg/L (log scale, 95% CI)")
style(ax)
h1 = ax.errorbar([], [], xerr=[], fmt="o", color=ORANGE, ms=8, label="Printed value read as an OR")
h2 = ax.errorbar([], [], xerr=[], fmt="s", color=BLUE, ms=7.5, label="exp(printed value): the OR if it is a log-odds coefficient")
ax.legend(handles=[h1, h2], loc="lower left", bbox_to_anchor=(-0.33, 1.0), frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- design C: p-value consistency under the two readings
def design_C(path):
keys = [("Overweight + depressed (m1)", "OW_dep"), ("Overweight controls (m1)", "OW_ctl"),
("Normal-weight depressed (m1)", "NW_dep"), ("'Normal-weight controls' (m1)", "NW_ctl"),
("Depression (m2)", "depression"), ("Overweight (m2)", "overweight")]
fig, ax = plt.subplots(figsize=(7.2, 4.3))
ypos = np.arange(len(keys))[::-1].astype(float)
ax.axvline(0.05, color=FG2, lw=1, ls=(0, (4, 3)))
ax.text(0.052, len(keys) - 0.45, "p = 0.05", fontsize=10, color=FG2)
for yy, (lab, k) in zip(ypos, keys):
r = readings[k]
pr = pnum(r["pr"])
ax.plot(pr, yy + 0.18, "o", ms=9, color=ORANGE, zorder=4)
if r["pr"].startswith("<"):
ax.annotate("", xy=(pr / 3, yy + 0.18), xytext=(pr, yy + 0.18),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=10))
ax.plot(r["p_logodds"], yy, "s", ms=8, color=BLUE, zorder=4)
if r["p_or"] is None:
ax.text(1.5e-5, yy - 0.3, "OR reading impossible", fontsize=10, color=FG2, va="center", ha="left")
else:
ax.plot(r["p_or"], yy - 0.2, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=4)
ax.set_xscale("log")
ax.set_xlim(1e-5, 1.5)
ax.set_yticks(ypos)
ax.set_yticklabels([k[0] for k in keys], fontsize=10)
ax.set_xlabel("Two-sided p-value (log scale)")
style(ax)
h = [plt.Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="p printed in the paper"),
plt.Line2D([], [], marker="s", ls="", color=BLUE, ms=8, label="p implied by the printed CI read as log-odds"),
plt.Line2D([], [], marker="D", ls="", mfc=BG, mec=BLUE, mew=2, ms=8, label="p implied by the printed CI read as an OR")]
ax.legend(handles=h, loc="lower left", bbox_to_anchor=(-0.4, 1.0), frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- design D: calculation flow for the headline abstract figure
def box(ax, x, y, w, h, txt, ec, fs=10.5, weight="normal"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.02",
fc=BG, ec=ec, lw=2))
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=fs, color=FG, weight=weight)
def arrow(ax, a, b, txt=None, off=(0, 0.03)):
ax.annotate("", xy=b, xytext=a, arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.6, mutation_scale=14))
if txt:
ax.text((a[0] + b[0]) / 2 + off[0], (a[1] + b[1]) / 2 + off[1], txt, ha="center", fontsize=10, color=FG2)
def design_D(path):
fig = plt.figure(figsize=(7.2, 4.0))
ax = fig.add_axes([0.02, 0.02, 0.96, 0.96])
ax.set_xlim(-0.02, 1.02); ax.set_ylim(0, 1); ax.axis("off")
lr = np.log(crude["OW_dep"])
b, lo, hi, _ = rep1["OW_dep"]
X = [(0.0, 0.34), (0.40, 0.25), (0.71, 0.29)]
top = ["Table 1, hsCRP ⩾3 mg/L\noverweight + depressed: 26/69\nnormal-weight controls: 3/57",
f"log odds ratio\nln[(26/43)/(3/54)]\n= {lr:.2f}",
f"crude odds ratio\n= {crude['OW_dep']:.1f}"]
bot = ["Abstract and Results print\n'OR 2.44 (1.28–3.94)'",
f"read as a log-odds\ncoefficient: {b:.2f}\n({lo:.2f} to {hi:.2f})",
f"odds ratio = exp(2.44)\n= {np.exp(b):.1f}\n(95% CI {np.exp(lo):.1f} to {np.exp(hi):.1f})"]
for (x, w), t in zip(X, top):
box(ax, x, 0.62, w, 0.27, t, BLUE, fs=10)
for (x, w), t, ec in zip(X, bot, [ORANGE, ORANGE, BLUE]):
box(ax, x, 0.08, w, 0.27, t, ec, fs=10)
for yy in (0.755, 0.215):
arrow(ax, (0.355, yy), (0.385, yy))
arrow(ax, (0.665, yy), (0.695, yy), "exp", off=(0, 0.035))
ax.annotate("", xy=(0.525, 0.365), xytext=(0.525, 0.605),
arrowprops=dict(arrowstyle="<|-|>", color=FG2, lw=1.6, mutation_scale=14))
ax.text(0.545, 0.485, f"{lr:.2f} v. {b:.2f}: same (log-odds) scale;\nthe printed value is also adjusted\nfor age and gender",
fontsize=10, color=FG2, va="center")
ax.text(0.0, 0.985, "Blue boxes: computed from the paper's own numbers. Orange boxes: as printed.", fontsize=10, color=FG2, va="top")
save(fig, path)
# ---- design E: dumbbell, printed headline 'ORs' v. exp(printed)
def design_E(path):
rows = [("Overweight + depressed v. normal-weight\ncontrols (abstract, model 1)", rep1["OW_dep"]),
("Overweight (abstract, model 2)", rep2["overweight"]),
("Depression (abstract, model 2)", rep2["depression"])]
fig, ax = plt.subplots(figsize=(7.2, 3.3))
ypos = np.arange(len(rows))[::-1].astype(float)
ax.axvline(1, color=FG2, lw=1)
for yy, (lab, (b, lo, hi, pr)) in zip(ypos, rows):
e = np.exp(b)
ax.annotate("", xy=(e / 1.08, yy), xytext=(b * 1.06, yy),
arrowprops=dict(arrowstyle="-|>", color=FG2, lw=1.4, mutation_scale=12))
ax.plot(b, yy, "o", ms=10, color=ORANGE, zorder=4)
ax.plot(e, yy, "s", ms=9.5, color=BLUE, zorder=4)
ax.text(b, yy + 0.27, f"{b:.2f}", ha="center", fontsize=10, color=ORANGE)
ax.text(e, yy + 0.27, f"{e:.1f}", ha="center", fontsize=10, color=BLUE)
ax.text(np.sqrt(b * e), yy - 0.33, f"x{e/b:.1f}", ha="center", fontsize=10, color=FG2)
ax.set_xscale("log")
ax.set_xlim(0.8, 20)
ax.set_xticks([1, 2, 5, 10, 20]); ax.set_xticklabels(["1", "2", "5", "10", "20"])
ax.set_yticks(ypos); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.7, len(rows) - 0.3)
ax.set_xlabel("Odds ratio (log scale)")
style(ax)
h = [plt.Line2D([], [], marker="o", ls="", color=ORANGE, ms=10, label="'OR' printed in the abstract"),
plt.Line2D([], [], marker="s", ls="", color=BLUE, ms=9.5, label="exp(printed value): the OR if it is a log-odds coefficient")]
ax.legend(handles=h, loc="lower left", bbox_to_anchor=(-0.45, 1.0), frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
designs = [("A_printed_scale_two_nulls", design_A), ("B_odds_ratio_axis", design_B),
("C_pvalue_consistency", design_C), ("D_calculation_flow", design_D),
("E_headline_dumbbell", design_E)]
for i, (nm, f) in enumerate(designs, 1):
f(os.path.join(CAND, f"cand{i}_{nm}"))
design_A(os.path.join(HERE, "fig1"))
design_B(os.path.join(HERE, "fig2"))
# contact sheet of the five candidates
imgs = [plt.imread(os.path.join(CAND, f"cand{i}_{nm}.png")) for i, (nm, _) in enumerate(designs, 1)]
fig, axs = plt.subplots(3, 2, figsize=(14.4, 15))
for i, ax in enumerate(axs.flat):
ax.axis("off")
if i < len(imgs):
ax.imshow(imgs[i])
ax.set_title(f"Candidate {i+1}: {designs[i][0][2:].replace('_', ' ')}", fontsize=13, loc="left")
fig.tight_layout()
fig.savefig(os.path.join(CAND, "contact_sheet.png"), dpi=110)
plt.close(fig)
print("\nFigures written: fig1.svg/png, fig2.svg/png, candidates/cand1..5 (.svg/.png), candidates/contact_sheet.png")
Reported positive predictive value of 55% conflicts with the paper's own sensitivity, specificity and prevalence
Sci Adv (2023) · doi:10.1126/sciadv.adg7545 · PMC10732528
Summary
The paper reports that its serial panel of three bivariate miRNA ratios separates preeclampsia (PE) cases from controls with sensitivity 93%, specificity 79%, positive predictive value (PPV) 55% and negative predictive value (NPV) 89%. It states these figures in the Abstract, the Results and the Discussion, and the Results and Discussion add "given a 57.7% prevalence of PE in our study" (71 of 123 samples). At that prevalence, a sensitivity of 93% and a specificity of 79% give a PPV of 85.8%. The per-sample classifications in the paper's data file S5 give 66 true positives and 11 false positives, so PPV = 66/77 = 85.7%. The reported NPV and both likelihood ratios agree with the same inputs; only the PPV does not. The error understates the panel's performance in a headline figure, and it affects the paper's comparison with the sFlt1:PlGF ratio, whose main limitation the paper gives as a low PPV.
What the paper reports
Abstract:
Applying an iterative machine learning method identified three bivariate miRNA biomarkers (miR-522-3p/miR-4732-5p, miR-516a-5p/miR-144-3p, and miR-27b-3p/let-7b-5p) that, when applied serially, distinguished between PE cases of different severity and differentiated cases from controls with a sensitivity of 93%, specificity of 79%, positive predictive value (PPV) of 55%, and negative predictive value (NPV) of 89%.
Results, "Identification of a small panel of candidate bivariate biomarkers":
We then re-ran the algorithm an additional two times, correctly identifying all but five PE cases, three of which were diagnosed as superimposed PE and one mild PE (Fig. 7A).
This iterative panel of three bivariate biomarkers achieved a PPV of 55% at a sensitivity of 93%, specificity of 79%, a positive likelihood ratio (+LR) of 4.43, and a negative likelihood ratio (-LR) of 0.09, given a 57.7% prevalence of PE in our study.
Discussion:
Overall, this iterative process was able to identify over 90% of cases with a false-positive rate of less than 10% at each iteration and achieved a PPV of 55% at a sensitivity of 93% and specificity of 79% (+LR = 4.43, -LR = 0.09) given a prevalence of PE of 57.7% in our study.
Results, "Characteristics of the study population" (the cohort the panel was applied to):
Cases were defined as pregnancies with an adjudicated PE diagnosis (n = 71), and controls (n = 52) were from participants with either hypertension that did not meet the criteria for PE or normal (nonhypertensive) pregnancy outcomes. The pass-filter 71 cases and 52 controls were subjected to further analysis.
Materials and Methods, "Candidate bivariate marker selection" (the definition used):
PPV was calculated as PPV = True Positives/(True Positives + False Positives), and NPV was calculated as NPV = True Negatives/(True Negatives + False Negatives).
Introduction (the comparison the PPV feeds into):
The primary limitation of the sFlt1:PlGF ratio is its relatively low positive predictive value (PPV), which numerous studies have reported to be between 30 and 65% (9–12).
What the paper's own numbers show
Method. I recomputed the PPV in two independent ways. (1) From the paper's rounded rates, using Bayes' rule, PPV = Se·p / (Se·p + (1 − Sp)(1 − p)), with Se = 0.93, Sp = 0.79 and p = 0.577. The same formula is stated in the medRxiv preprint of this paper. (2) From counts. Data file S5 ("Refers to Figure 7") lists all 123 samples with the iteration (1, 2 or 3) at which each one was flagged as PE, or "Not Removed". Flagged cases are true positives, unflagged cases false negatives, flagged controls false positives and unflagged controls true negatives. I applied the Methods definitions to those counts. I also computed what a PPV of 55% would require, both in prevalence and in false positives. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Prevalence | Results; Discussion | 57.7% | 71/123 = 57.72% |
| Classification counts | Data file S5 | cases flagged at iterations 1/2/3: 40/15/11, not flagged 5; controls flagged 3/4/4, not flagged 41 | TP = 66, FN = 5, FP = 11, TN = 41 |
| Sensitivity | Abstract; Results; Discussion | 93% | 66/71 = 92.96% |
| Specificity | Abstract; Results; Discussion | 79% | 41/52 = 78.85% |
| PPV | Abstract; Results; Discussion | 55% | 85.8% from the rounded rates; 66/77 = 85.71% from the counts |
| NPV | Abstract | 89% | 89.2% from the rounded rates; 41/46 = 89.13% from the counts |
| +LR | Results; Discussion | 4.43 | 0.93/0.21 = 4.43; 4.39 from the counts |
| −LR | Results; Discussion | 0.09 | 0.07/0.79 = 0.089; 0.089 from the counts |
| PPV if specificity were anywhere from 38/52 to 44/52 (TP fixed at 66) | derived | — | 82.5% to 89.2% |
| Prevalence at which Se 93% and Sp 79% give PPV 55% | derived | — | 21.6%; the NPV there would be 97.6%, not 89% |
| False positives needed for PPV 55% with 66 true positives | derived | — | 66/0.55 − 66 = 54, more than the 52 controls |
| PPV of the miRNA panel alone in the 70-patient sFlt1:PlGF subset (42 TP, 6 FP) | Results | not stated | 42/48 = 87.5% |
| Combined miRNA then sFlt1:PlGF (42 TP, 2 FP, 21 TN, 5 FN) | Abstract; Results | Se 89.4%, Sp 91.3%, PPV 95.5%, NPV 80.8% | 89.36%, 91.3%, 95.5%, 80.8% (consistent) |
| Preprint inputs (prevalence 57.8%, Sp 78.8%) | medRxiv preprint, Methods | PPV 55% | 85.7% with the preprint's own formula |
Every other metric of the panel agrees with its inputs to the printed precision. The PPV is the only one that does not, and it is about 31 percentage points below the value the paper's own numbers give.
Figure
Figure 1. The blue curve is the PPV that a test with the paper's reported sensitivity (93%) and specificity (79%) has at each prevalence (Bayes' rule). The thin vertical line marks the study prevalence the paper states, 71/123 = 57.7%. At that prevalence the curve gives 85.8% (blue diamond), and the counts in data file S5 give 66/77 = 85.7%. The paper's reported 55% (orange circle) lies well below the curve. The curve reaches 55% only at a prevalence of 21.6% (open circle), and at that prevalence the NPV would be 97.6%, not the 89% the paper reports. All values come from the paper or from recompute.py.txt.
Figure 2. The count reconstruction from data file S5. Summing the cases and controls flagged at each of the three iterations gives TP 66, FN 5, FP 11 and TN 41. These counts reproduce the reported sensitivity, specificity and NPV, but give a PPV of 85.7% against the reported 55%. The lower panel shows that a 55% PPV would need 54 false positives, more than the 52 controls in the cohort.
Why it matters
- Headline figure. The PPV is one of four performance figures in the Abstract for the paper's main diagnostic result, and it is repeated in the Results and the Discussion. Readers who cite the panel's performance will cite 55%.
- Comparison with sFlt1:PlGF. The Introduction gives the sFlt1:PlGF ratio's "relatively low positive predictive value", 30 to 65%, as its main limitation. At 55% the miRNA panel appears no better on this measure. At about 86% it appears better, although PPVs from cohorts with different prevalence cannot be compared directly, and this cohort's prevalence (57.7%) is high for a triage population.
- Gain from combining assays. The paper reports that applying sFlt1:PlGF after the miRNA panel gives a PPV of 95.5%. Against a stated baseline of 55% this looks like a large gain. The actual baseline is about 86% in the full cohort and 87.5% in the 70-patient subset that has sFlt1:PlGF data, so the gain is about 8 to 10 percentage points.
- What is not affected. The reported sensitivity, specificity, NPV and likelihood ratios are consistent with the data. So are the combined-assay counts and metrics, the identities of the three biomarkers and the call for larger validation. The error makes the panel look worse than the paper's own data show, not better.
Explanations considered
- A different prevalence. A PPV is sometimes recalculated for an external, lower prevalence. This would explain 55% if the paper had used a prevalence of about 21.6%. But the paper explicitly attaches the 55% to "a 57.7% prevalence of PE in our study", which is 71/123, the full cohort the panel was applied to. At 21.6% the NPV would be 97.6%, not the reported 89%, so the reported NPV and PPV cannot both come from the same adjusted prevalence. This does not account for the numbers.
- A different definition of PPV. The Methods define PPV as TP/(TP + FP), the standard formula. I tried other ratios that could be mislabeled: TP/(TP + FP + FN) = 80.5%, accuracy = 87.0%, TN/(TN + FP + FN) = 71.9%, TP/N = 53.7%, 1 − PPV = 14.3% and the false-positive rate = 21.2%. None gives 55%. TP/N (53.7%) is the closest, but it rounds to 54% and is not a predictive value. This does not account for the numbers.
- Different integer counts. The sensitivity is pinned by the text ("all but five PE cases", so 66 of 71). Any specificity count from 38/52 to 44/52 gives a PPV between 82.5% and 89.2%. A PPV of 55% with 66 true positives would need 54 false positives, more than the 52 controls. Data file S5 gives the exact counts (FP = 11). This does not account for the numbers.
- A subset or a different stage of the iteration. In the 70-patient subset with sFlt1:PlGF data, the miRNA panel alone has 42 true and 6 false positives (PPV 87.5%). The three iterations in data file S5, taken one at a time, give PPVs of 40/43 = 93.0%, 15/19 = 78.9% and 11/15 = 73.3%. None of these is near 55%. This does not account for the numbers.
- A conversion or typesetting artifact. The 55% appears in running text in three places, in the PMC archive, the Europe PMC full-text XML and the medRxiv preprint. It is not in a table, so a misaligned column is ruled out, and it is not a lost minus sign or decimal. The preprint states the Bayes formula and its inputs (prevalence 57.8%, specificity 78.8%). Those inputs give 85.7% with that formula, so the 55% was already present in the preprint and was carried into the published version unchanged. This does not account for the numbers.
None of these explanations reconciles 55% with the paper's other figures. I have not found what calculation produced it.
A related observation (not part of the main claim). For the 18-patient independent validation cohort (11 cases, 7 controls), the Results report "a PPV of 85% at a sensitivity of 91% and specificity of 57%". A sensitivity of 91% with 11 cases is 10/11, which agrees with the Discussion's "we found all but one PE case". A specificity of 57% with 7 controls is 4/7, which leaves 3 false positives. That gives PPV = 10/13 = 76.9%. A PPV of 85% (11/13 = 84.6%) would require all 11 cases to be detected. Data file S6 gives only the cohort's demographics, not per-sample calls, so I could not check this from counts.
Status
As of 2026-10-23:
- Version of record. science.org returned HTTP 403 to automated requests, so I checked the publisher-deposited text instead. The PMC article page, the Europe PMC full-text XML and the local PMC archive all contain the three sentences quoted above word for word.
- Supplements. I checked the supplementary PDF (Figs. S1 to S3) and data files S1 to S11 (Excel). Data file S5 gives the per-sample classifications used above. It does not mention a PPV.
- Preprint. The medRxiv preprint (doi:10.1101/2023.01.26.23285072, posted 2023-01-29) also reports a PPV of 55% at sensitivity 93% and prevalence 57.7%. Its Methods give the Bayes formula, which yields about 86% with its stated inputs.
- Corrections and retractions. Crossref update metadata, PMC notices and Europe PMC notices list no correction, expression of concern or retraction.
- PubPeer. PubPeer's publication lookup for the DOI returned no comments. Its web search page refused automated access (HTTP 403). A general web search found no discussion of this error.
Sources: article (DOI); PMC10732528; medRxiv preprint; supplementary files via the Europe PMC supplementaryFiles service.
Suggested correction
In the Abstract, the Results ("Identification of a small panel of candidate bivariate biomarkers") and the Discussion, the PPV of the three-bivariate panel in the 123-sample cohort should read 86% (66/77 = 85.7% from data file S5; 85.8% from the rounded sensitivity, specificity and prevalence) instead of 55%. Sensitivity, specificity, NPV, +LR and −LR need no change. If the 55% was meant to describe a different population or prevalence, the text should say which one, and the NPV should then be recomputed for it. The authors may also wish to check the validation-cohort PPV of 85% in the Results. With the stated sensitivity (10/11) and specificity (4/7) it computes to 10/13 = 77%.
Recomputation script (python recompute.py.txt)
# Recompute the diagnostic metrics of the three-bivariate miRNA panel in
# Sci Adv 2023, doi:10.1126/sciadv.adg7545 (PMC10732528), and draw the figures.
#
# Run: python recompute.py.txt
# Every input below is a number printed in the article or in its data file S5
# (per-sample iteration at which each of the 123 samples was flagged as PE).
# Outputs: fig1.svg/png, fig2.svg/png and five candidate designs in candidates/.
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
import numpy as np
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs
CASES, CONTROLS = 71, 52 # Results, "Characteristics of the study population"
N = CASES + CONTROLS
REP = dict(se=0.93, sp=0.79, ppv=0.55, npv=0.89, plr=4.43, nlr=0.09, prev=0.577) # Abstract/Results/Discussion
# Data file S5: iteration at which each sample was flagged (1, 2, 3) or "Not Removed"
S5 = {"case": {1: 40, 2: 15, 3: 11, "not": 5}, "control": {1: 3, 2: 4, 3: 4, "not": 41}}
TP = sum(v for k, v in S5["case"].items() if k != "not")
FN = S5["case"]["not"]
FP = sum(v for k, v in S5["control"].items() if k != "not")
TN = S5["control"]["not"]
assert TP + FN == CASES and FP + TN == CONTROLS
def ppv_f(se, sp, p):
return se * p / (se * p + (1 - sp) * (1 - p))
def npv_f(se, sp, p):
return sp * (1 - p) / (sp * (1 - p) + (1 - se) * p)
def pct(x, d=1):
return f"{100 * x:.{d}f}%"
print("=== 1. Prevalence")
prev = CASES / N
print(f"prevalence = {CASES}/{N} = {pct(prev, 2)} (reported 57.7%)")
print("\n=== 2. From the paper's rounded rates (Se 93%, Sp 79%, prevalence 57.7%)")
se, sp, p = REP["se"], REP["sp"], REP["prev"]
ppv_r, npv_r = ppv_f(se, sp, p), npv_f(se, sp, p)
plr_r, nlr_r = se / (1 - sp), (1 - se) / sp
print(f"PPV = {se}*{p} / ({se}*{p} + {1-sp:.2f}*{1-p:.3f}) = {pct(ppv_r)} (reported 55%)")
print(f"NPV = {pct(npv_r)} (reported 89%) +LR = {plr_r:.2f} (reported 4.43) -LR = {nlr_r:.3f} (reported 0.09)")
print("\n=== 3. From the counts in data file S5")
se_c, sp_c = TP / CASES, TN / CONTROLS
ppv_c, npv_c = TP / (TP + FP), TN / (TN + FN)
plr_c, nlr_c = se_c / (1 - sp_c), (1 - se_c) / sp_c
print(f"cases flagged at iterations 1/2/3 = {S5['case'][1]}/{S5['case'][2]}/{S5['case'][3]}, not flagged = {FN}")
print(f"controls flagged at iterations 1/2/3 = {S5['control'][1]}/{S5['control'][2]}/{S5['control'][3]}, not flagged = {TN}")
print(f"TP={TP} FN={FN} FP={FP} TN={TN}")
print(f"Se = {TP}/{CASES} = {pct(se_c, 2)} Sp = {TN}/{CONTROLS} = {pct(sp_c, 2)}")
print(f"PPV = {TP}/{TP+FP} = {pct(ppv_c, 2)} NPV = {TN}/{TN+FN} = {pct(npv_c, 2)}")
print(f"+LR = {plr_c:.2f} -LR = {nlr_c:.3f}")
print(f"gap between recomputed and reported PPV = {100*(ppv_c-REP['ppv']):.1f} percentage points")
for it in (1, 2, 3):
t, f_ = S5["case"][it], S5["control"][it]
print(f" iteration {it} alone: PPV = {t}/{t+f_} = {pct(t/(t+f_))}")
print("\n=== 4. Sensitivity of the PPV to the exact specificity count (TP fixed at 66)")
sweep = []
for tn in range(38, 45):
fp = CONTROLS - tn
sweep.append(TP / (TP + fp))
print(f" TN={tn} FP={fp}: Sp={pct(tn/CONTROLS)} PPV={pct(TP/(TP+fp))}")
print(f"lowest PPV in this range = {pct(min(sweep))}")
print("\n=== 5. What would a PPV of 55% require?")
target = REP["ppv"]
p55 = target * (1 - sp) / (se * (1 - target) + target * (1 - sp))
print(f"prevalence giving PPV 55% at Se 93%/Sp 79% = {pct(p55)}; NPV at that prevalence = {pct(npv_f(se, sp, p55))} (reported 89%)")
fp55 = TP / target - TP
print(f"false positives needed with TP={TP}: {fp55:.1f} (only {CONTROLS} controls exist)")
print("\n=== 6. Other ratios tried, none gives 55%")
print(f"TP/(TP+FP+FN) = {pct(TP/(TP+FP+FN))} accuracy = {pct((TP+TN)/N)} TN/(TN+FP+FN) = {pct(TN/(TN+FP+FN))}")
print(f"1 - PPV = {pct(1-ppv_c)} FP/(FP+TN) = {pct(FP/CONTROLS)} TP/N = {pct(TP/N)}")
print("\n=== 7. Preprint inputs (medRxiv 10.1101/2023.01.26.23285072: prevalence 57.8%, Sp 78.8%)")
print(f"PPV formula stated in the preprint, applied to Se 0.93, Sp 0.788, p 0.578 = {pct(ppv_f(0.93, 0.788, 0.578))}")
print("\n=== 8. sFlt1:PlGF subset (n = 70), Results")
subset = {"miRNA panel alone": (42, 6, 17, 5),
"miRNA then sFlt1:PlGF": (42, 2, 21, 5),
"sFlt1:PlGF only on miRNA positives": (39, 0, 23, 8)}
for lab, (tp, fp, tn, fn) in subset.items():
print(f" {lab}: Se={pct(tp/(tp+fn), 2)} Sp={pct(tn/(tn+fp))} PPV={pct(tp/(tp+fp))} NPV={pct(tn/(tn+fn))}")
print(f" PPV gain from adding sFlt1:PlGF: {100*(0.955-ppv_c):.1f} points vs full-cohort {pct(ppv_c)}, "
f"{100*(42/44-42/48):.1f} points vs subset {pct(42/48)}")
print(" (reported for 'then': Se 89.4%, Sp 91.3%, PPV 95.5%, NPV 80.8%; for 'only on positives': Se 82.9%, Sp 100%)")
print("\n=== 9. Side observation: independent validation cohort (11 cases, 7 controls)")
vtp, vfn, vtn, vfp = 10, 1, 4, 3 # Se 91% = 10/11 ('all but one PE case'), Sp 57% = 4/7
print(f"Se={pct(vtp/11)} Sp={pct(vtn/7)} PPV={vtp}/{vtp+vfp} = {pct(vtp/(vtp+vfp))} (reported 85%); 11/13 = {pct(11/13)}")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "axes.edgecolor": INK2, "xtick.color": INK2, "ytick.color": INK2,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"axes.titlesize": 11.5, "axes.titleweight": "bold", "axes.titlelocation": "left",
"svg.fonttype": "none", "lines.linewidth": 2,
})
def style_axes(ax, grid_axis="both"):
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(True, axis=grid_axis, color=GRID, linewidth=0.8)
ax.set_axisbelow(True)
def save(fig, path):
fig.savefig(path + ".svg")
fig.savefig(path + ".png", dpi=200)
plt.close(fig)
print("wrote", os.path.relpath(path, HERE) + ".svg/.png")
print("\n=== figures")
# A. Dot comparison: reported vs recomputed for the four percentage metrics
def fig_A(path):
fig, ax = plt.subplots(figsize=(W, 3.8))
rows = [("Sensitivity", REP["se"], se_c, f"{TP}/{CASES}"),
("Specificity", REP["sp"], sp_c, f"{TN}/{CONTROLS}"),
("NPV", REP["npv"], npv_c, f"{TN}/{TN+FN}"),
("PPV", REP["ppv"], ppv_c, f"{TP}/{TP+FP}")]
d = 0.14
for i, (lab, r, c, frac) in enumerate(rows):
y = len(rows) - 1 - i
ax.scatter(r * 100, y + d, s=80, color=ORANGE, marker="o", zorder=3, edgecolor=BG, linewidth=1.2)
ax.scatter(c * 100, y - d, s=80, color=BLUE, marker="D", zorder=4, edgecolor=BG, linewidth=1.2)
if lab == "PPV":
ax.text(r * 100 - 2.5, y + d, f"reported {r*100:.0f}%", ha="right", va="center", fontsize=10)
ax.text(c * 100 - 2.5, y - d, f"{frac} = {c*100:.1f}%", ha="right", va="center", fontsize=10)
else:
ax.text(min(r, c) * 100 - 2.5, y, f"reported {r*100:.0f}%; {frac} = {c*100:.1f}%",
ha="right", va="center", fontsize=10, color=INK2)
ax.set_yticks(range(len(rows)))
ax.set_yticklabels([r[0] for r in rows][::-1])
ax.set_xlim(0, 100)
ax.set_ylim(-0.5, len(rows) - 0.5)
ax.set_xlabel("Percent (whole cohort: 71 cases, 52 controls)")
style_axes(ax, "x")
ax.scatter([], [], s=70, color=ORANGE, marker="o", label="Reported in the paper")
ax.scatter([], [], s=70, color=BLUE, marker="D", label="Recomputed from data file S5 counts")
ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.2), ncol=2, frameon=False, fontsize=10)
ax.set_title("Only the PPV disagrees with the panel's own classification counts")
fig.tight_layout()
save(fig, path)
# B. PPV as a function of prevalence at the reported Se/Sp
def fig_B(path):
fig, ax = plt.subplots(figsize=(W, 4.3))
ps = np.linspace(0.005, 0.995, 400)
ax.plot(ps * 100, ppv_f(se, sp, ps) * 100, color=BLUE, lw=2, zorder=2,
label="PPV implied by Se 93% and Sp 79%")
ax.axvline(p * 100, color=INK2, lw=0.8, zorder=1)
ax.text(p * 100 + 1, 3, f"study prevalence\n71/123 = {prev*100:.1f}%", color=INK2, fontsize=10, va="bottom")
ax.plot([p55 * 100, p * 100], [55, 55], color=INK2, lw=1, ls=(0, (3, 3)), zorder=1)
ax.scatter(p55 * 100, 55, s=64, facecolor=BG, edgecolor=BLUE, linewidth=2, zorder=4)
ax.annotate(f"the curve reaches 55% only at a prevalence\nof {p55*100:.1f}%, where NPV would be "
f"{npv_f(se, sp, p55)*100:.1f}%, not 89%",
(p55 * 100, 55), xytext=(8, -10), textcoords="offset points", ha="left", va="top",
fontsize=10, color=INK2)
ax.scatter(p * 100, ppv_r * 100, s=110, color=BLUE, marker="D", zorder=5, edgecolor=BG, linewidth=1.5,
label="Recomputed PPV at 57.7%")
ax.annotate(f"recomputed {ppv_r*100:.1f}%\n(counts: {TP}/{TP+FP} = {ppv_c*100:.1f}%)", (p * 100, ppv_r * 100),
xytext=(12, -8), textcoords="offset points", ha="left", va="top", fontsize=10)
ax.scatter(p * 100, 55, s=110, color=ORANGE, marker="o", zorder=5, edgecolor=BG, linewidth=1.5,
label="Reported PPV at 57.7%")
ax.annotate("reported 55%", (p * 100, 55), xytext=(10, 0), textcoords="offset points",
ha="left", va="center", fontsize=10)
ax.set_xlim(0, 100); ax.set_ylim(0, 100)
ax.set_xlabel("Prevalence of preeclampsia (%)")
ax.set_ylabel("Positive predictive value (%)")
style_axes(ax)
ax.legend(loc="upper left", frameon=False, fontsize=10)
ax.set_title("At the study's prevalence, Se 93% and Sp 79% give a PPV of 86%")
fig.tight_layout()
save(fig, path)
def box(ax, x, y, w, h, text, edge, fs=10.5, weight="normal", fc=BG, color=INK):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.006,rounding_size=0.012",
linewidth=2, edgecolor=edge, facecolor=fc))
if text:
ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", fontsize=fs, fontweight=weight,
color=color, linespacing=1.3)
def arrow(ax, a, b, color=INK2):
ax.add_patch(FancyArrowPatch(a, b, arrowstyle="-|>", mutation_scale=13, lw=1.5, color=color))
# C. Calculation flow from data file S5 counts
def fig_C(path):
fig = plt.figure(figsize=(W, 4.2))
ax = fig.add_axes([0.02, 0.02, 0.96, 0.96])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
box(ax, 0.005, 0.56, 0.19, 0.34, "Data file S5\n123 samples,\neach with the\niteration that\nflagged it as PE", INK2, fs=10)
box(ax, 0.235, 0.75, 0.30, 0.19,
f"71 cases\nflagged {S5['case'][1]}+{S5['case'][2]}+{S5['case'][3]} = {TP} (TP)\nnot flagged: {FN} (FN)", BLUE, fs=10)
box(ax, 0.235, 0.52, 0.30, 0.19,
f"52 controls\nflagged {S5['control'][1]}+{S5['control'][2]}+{S5['control'][3]} = {FP} (FP)\nnot flagged: {TN} (TN)", BLUE, fs=10)
arrow(ax, (0.20, 0.78), (0.23, 0.83)); arrow(ax, (0.20, 0.68), (0.23, 0.62))
arrow(ax, (0.545, 0.73), (0.575, 0.73))
ax.text(0.59, 0.96, "Recomputed", fontsize=10, color=BLUE, fontweight="bold", va="center")
ax.text(0.995, 0.96, "Reported", fontsize=10, color=ORANGE, fontweight="bold", va="center", ha="right")
outs = [(0.87, f"Se {TP}/{CASES} = {se_c*100:.1f}%", "93% (matches)", True),
(0.77, f"Sp {TN}/{CONTROLS} = {sp_c*100:.1f}%", "79% (matches)", True),
(0.67, f"NPV {TN}/{TN+FN} = {npv_c*100:.1f}%", "89% (matches)", True),
(0.57, f"PPV {TP}/{TP+FP} = {ppv_c*100:.1f}%", "55% (no match)", False)]
for y, t, r, ok in outs:
ax.text(0.59, y, t, fontsize=10, color=INK, va="center")
ax.text(0.995, y, r, fontsize=10, ha="right", va="center", color=INK2 if ok else INK)
ax.add_patch(FancyBboxPatch((0.585, 0.53), 0.405, 0.08, boxstyle="round,pad=0.004,rounding_size=0.01",
linewidth=1.5, edgecolor=ORANGE, facecolor="none"))
ax.plot([0.005, 0.995], [0.42, 0.42], color=GRID, lw=1)
ax.text(0.005, 0.34, "What a PPV of 55% would require", fontsize=10.5, fontweight="bold", color=INK, va="center")
ax.text(0.005, 0.23, f"With the same {TP} true positives: {TP}/0.55 - {TP} = {fp55:.0f} false positives, "
f"but there are only {CONTROLS} controls.", fontsize=10, color=INK, va="center")
ax.text(0.005, 0.13, f"With Se 93% and Sp 79%: a prevalence of {p55*100:.1f}% (the paper states "
f"{REP['prev']*100:.1f}%),", fontsize=10, color=INK, va="center")
ax.text(0.005, 0.05, f"at which the NPV would be {npv_f(se, sp, p55)*100:.1f}% rather than the reported 89%.",
fontsize=10, color=INK, va="center")
save(fig, path)
# D. False positives observed vs required
def fig_D(path):
fig, ax = plt.subplots(figsize=(W, 2.8))
vals = [("Observed\n(data file S5)", FP, BLUE, None, f"{FP} PPV {TP}/{TP+FP} = {ppv_c*100:.1f}%"),
("Needed for the\nreported PPV of 55%", fp55, ORANGE, "//", f"{fp55:.0f} PPV {TP}/{TP+fp55:.0f} = 55%")]
for i, (lab, v, c, h, txt) in enumerate(vals):
y = 1 - i
ax.barh(y, v, height=0.46, color=c if h is None else BG, edgecolor=c, hatch=h, linewidth=2)
ax.text(v + 1.5, y, txt, va="center", ha="left", fontsize=10)
ax.axvline(CONTROLS, color=INK, lw=1)
ax.text(CONTROLS - 1, 1.48, f"all {CONTROLS} controls", ha="right", va="center", fontsize=10, color=INK2)
ax.set_yticks([1, 0]); ax.set_yticklabels([v[0] for v in vals])
ax.set_xlim(0, 100); ax.set_ylim(-0.45, 1.7)
ax.set_xlabel(f"Controls classified as preeclampsia (true positives fixed at {TP})")
style_axes(ax, "x")
fig.tight_layout(rect=(0, 0, 1, 0.9))
fig.suptitle(f"A 55% PPV would need {fp55:.0f} false positives; the cohort has only {CONTROLS} controls",
x=0.01, ha="left", fontsize=11.5, fontweight="bold")
save(fig, path)
# E. Text panels: the three reported locations vs the paper's own inputs
def fig_E(path):
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0.02, 0.02, 0.96, 0.96])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
quotes = [("Abstract", "“...sensitivity of 93%, specificity of 79%,\npositive predictive value (PPV) of 55%,\nand negative predictive value (NPV) of 89%.”"),
("Results (Fig. 7 paragraph)", "“...achieved a PPV of 55% at a sensitivity of\n93%, specificity of 79%, ... (+LR) of 4.43, ...\n(-LR) of 0.09, given a 57.7% prevalence...”"),
("Discussion", "“...achieved a PPV of 55% at a sensitivity of\n93% and specificity of 79% (+LR = 4.43,\n-LR = 0.09) given a prevalence of PE of 57.7%...”")]
ys = [0.65, 0.345, 0.04]
for (h, q), y in zip(quotes, ys):
box(ax, 0.005, y, 0.53, 0.26, "", ORANGE)
ax.text(0.02, y + 0.215, h, fontsize=10.5, fontweight="bold", va="center", color=INK)
ax.text(0.02, y + 0.095, q, fontsize=10, va="center", color=INK, linespacing=1.3)
bx, by, bw, bh = 0.60, 0.22, 0.385, 0.56
box(ax, bx, by, bw, bh, "", BLUE)
ax.text(bx + 0.02, by + bh - 0.05, "The paper's own inputs", fontsize=10.5, fontweight="bold", va="center")
lines = [("Se 0.93, Sp 0.79,", False), ("prevalence 71/123 = 0.577", False),
(f"PPV = {ppv_r*100:.1f}% (reported 55%)", True),
(f"NPV = {npv_r*100:.1f}% (reported 89%)", False),
(f"+LR = {plr_r:.2f} (reported 4.43)", False),
(f"-LR = {nlr_r:.3f} (reported 0.09)", False),
(f"S5 counts: PPV {TP}/{TP+FP} = {ppv_c*100:.1f}%", True)]
for k, (t, b) in enumerate(lines):
ax.text(bx + 0.02, by + bh - 0.12 - k * 0.062, t, fontsize=10, va="center",
fontweight="bold" if b else "normal", color=INK)
for y in ys:
ax.add_patch(FancyArrowPatch((0.54, y + 0.13), (bx - 0.005, 0.50), arrowstyle="-|>", mutation_scale=12,
lw=1.3, color=INK2))
ax.text(bx + bw / 2, 0.86, "same 55% in all three places;\nonly the PPV fails the check", fontsize=10,
color=INK2, ha="center", va="center")
ax.text(0.005, 0.975, "Orange boxes: the paper's text", fontsize=10, color=INK2, va="center")
ax.text(0.995, 0.975, "Blue box: recomputed", fontsize=10, color=INK2, va="center", ha="right")
save(fig, path)
fig_A(os.path.join(CAND, "A_metric_dots"))
fig_B(os.path.join(CAND, "B_ppv_vs_prevalence"))
fig_C(os.path.join(CAND, "C_count_flow"))
fig_D(os.path.join(CAND, "D_false_positives_needed"))
fig_E(os.path.join(CAND, "E_text_panels"))
# provisional choice for the write-up, pending a pick among candidates/A-E
fig_B(os.path.join(HERE, "fig1"))
fig_C(os.path.join(HERE, "fig2"))
Mediation effect called significant although its reported 95% CI includes zero and P = 0.146
Sci Adv (2022) · doi:10.1126/sciadv.abq3211 · PMC10936045
Summary
The Results state that a mediation analysis "revealed a significant indirect effect" of the CDA set-size effect on the link between 1064-nm tPBM and working-memory capacity, but the statistics printed with that sentence are 0.300, 95% CI −0.107 to 0.706, P = 0.146. The interval contains zero and agrees with the printed P (a Wald P recomputed from it is 0.148), so by the paper's own test (is zero inside the 95% bootstrap CI?) the indirect effect is not significant; the numbers also match the Fig. 3A paths in a way suggesting they belong to the direct path c′, which would leave the indirect effect (about 0.13 to 0.15) untested. Either way, no statistic in the paper supports the mediation claim stated in the abstract, a Results heading, the Fig. 3 title and the Discussion. The main behavioral result, that K values rose after 1064-nm tPBM, rests on separate tests and is not affected.
What the paper reports
Abstract:
The CDA set-size effect during retention mediated the effect between the 1064-nm tPBM and subsequent WM capacity.
Results, section heading:
CDA mediates the WM improvement with tPBM (1064 nm) applied to the right PFC
Results, same section (the PMC text renders the minus sign as a hyphen; the archived JATS XML has a true minus sign, U+2212, before 0.107):
This mediation analysis revealed a significant indirect effect of the CDA set-size effect (indirect effect: 0.300, 95% confidence interval: -0.107 to 0.706, P = 0.146; direct effect: 0.245, 95% confidence interval: 0.021 to 0.844, P = 0.040).
The mediation analysis demonstrated the indirect effect of 1064-nm tPBM on the behavioral K values through an increase in the amount of information maintained in visual WM, as reflected by increases in the CDA set-size effect.
Results, same section, next sentences:
(r = 0.404, P < 0.001, confidence interval: 0.154 to 0.654; Fig. 3B)
Materials and Methods, "Statistical analysis":
The direct and indirect effects were then obtained by contrasting these two models. The null hypothesis was tested by examining whether zero was within the 95% bootstrap confidence intervals.
Fig. 3 title and legend:
The CDA set-size effect mediated behavioral K values by 1064-nm tPBM.
a, b, and c denote standardized beta coefficients of the direct path strength. c′ denotes the beta coefficient of path strength after controlling for changes in the CDA set-size effect.
*P < 0.05 and **P < 0.01.
Fig. 3A path labels (read from the archived figure image, not reproduced here): a = 0.370*, b = 0.404**, c = 0.433*, c′ = 0.300 (no asterisk).
Discussion, first paragraph:
A mediation analysis revealed that the behavioral enhancements with tPBM were mediated by CDA set-size effects.
Discussion, "Improvement of CDA-identified WM capacity by 1064-nm tPBM on the right PFC":
We further established neurophysiological links between 1064-nm tPBM and subsequent WM capacity, by demonstrating that the CDA during the retention served as a mediator.
What the paper's own numbers show
Method. I first applied the paper's own decision rule to each printed interval: is zero inside the 95% CI? As a consistency check, I then read each interval as estimate ± 1.96 SE, recovered the implied SE, and computed a two-sided Wald P to compare with the printed P. A bootstrap interval need not be symmetric, so this check can only confirm agreement among the numbers; the paper's rule does not depend on it. I also compared the printed estimates and interval midpoints with the path coefficients in Fig. 3A, and computed the indirect effect those paths imply (a × b, and c − c′). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Is the indirect effect significant? | Results, mediation paragraph | "a significant indirect effect" | No: 0 lies inside −0.107 to 0.706, and P = 0.146 > 0.05 |
| Indirect effect: CI midpoint | same | 0.300 (−0.107 to 0.706) | 0.2995; the interval is symmetric about the estimate (0.407 below, 0.406 above) |
| Indirect effect: P | same | 0.146 | implied SE 0.2074, z = 1.446, Wald P = 0.148 (agrees with the printed P) |
| Indirect effect, read one-sided | derived; no one-sided test is stated | — | P = 0.073; 90% CI −0.041 to 0.641 (still contains 0) |
| Direct effect: significant? | same | 0.245, CI 0.021 to 0.844, P = 0.040 | yes: 0 lies outside the CI, and P < 0.05 |
| Direct effect: CI midpoint vs estimate | same | 0.245 | midpoint 0.4325; the estimate is 0.224 above the lower bound and 0.599 below the upper |
| Direct effect: Wald P | same | 0.040 | 0.243 for the stated estimate 0.245; 0.039 for the CI midpoint 0.4325 |
| Printed "indirect" estimate vs Fig. 3A | Results vs Fig. 3A | 0.300 | equals c′ = 0.300, the path after controlling for the mediator (no asterisk) |
| Printed "direct" interval vs Fig. 3A | Results vs Fig. 3A | CI 0.021 to 0.844, P = 0.040 | midpoint 0.4325 equals c = 0.433 (marked *); Wald P 0.039 |
| Indirect effect implied by Fig. 3A | Fig. 3A | not reported | a × b = 0.370 × 0.404 = 0.1495; c − c′ = 0.133; no CI or P for either |
Two readings are possible, and both lead to the same conclusion.
- Take the labels as printed. The indirect effect is 0.300 with a 95% CI that contains zero and P = 0.146. Under the stated rule, the null of no indirect effect is not rejected. The direct effect is significant. A significant direct effect with a non-significant indirect effect does not show mediation.
- Match the numbers to Fig. 3A. The "indirect" estimate and its symmetric interval equal c′ = 0.300, which the figure also leaves unstarred. The "direct" interval and P fit c = 0.433: its midpoint is 0.4325, and a Wald P for 0.433 on that interval is 0.039, against the printed 0.040. The printed point estimate 0.245 lies inside the "direct" interval but far from its center (midpoint 0.4325), and matches no path in Fig. 3A. On this reading, the two sets of statistics belong to the direct path c′ and the total effect c, and the indirect effect itself (a × b ≈ 0.149, or c − c′ = 0.133) is never tested in the paper. The claim of a "significant indirect effect" then has no reported statistic behind it.
Figure
Figure 1. Standardized effects of 1064-nm tPBM on K values, on one linear axis, with zero (no effect) as the thin vertical line. Orange: the estimate and 95% CI printed in the Results for the "indirect effect" (top row) and the "direct effect" (middle row), with the printed P values. Blue diamonds: the interval midpoints recomputed from the printed bounds, with the Wald P implied by each interval. Black open triangles: the path labels printed in Fig. 3A, placed on the row whose interval they match (c′ = 0.300 with the "indirect" interval, c = 0.433 with the "direct" interval). Bottom row, blue squares: the indirect effect implied by the Fig. 3A paths, a × b = 0.149 (filled) and c − c′ = 0.133 (open), for which no CI or P is reported. The "indirect effect" interval crosses zero; its estimate and midpoint coincide with c′; and the "direct effect" estimate 0.245 sits far from its own interval's midpoint, which equals c.
Figure 2. The two statistic sets printed in the Results (left panel) set beside the Fig. 3A path labels (right panel). Orange text is printed in the paper; blue text is recomputed from it. The "indirect effect" estimate equals c′, the tPBM → K path after controlling for the mediator, and neither carries a significance mark. The "direct effect" interval is centered on 0.4325, and a Wald P for c = 0.433 on that interval is 0.039, against the printed P = 0.040. The indirect effect the paths imply, a × b = 0.149 or c − c′ = 0.133, appears nowhere in the paper with an interval or P value.
Why it matters
- A mechanistic claim in the abstract has no statistical support in the paper. The abstract states that the CDA set-size effect "mediated" the effect of 1064-nm tPBM on WM capacity. The same claim appears in a Results heading, the Fig. 3 title, and two places in the Discussion, where it is presented as having "established neurophysiological links" between tPBM and WM capacity. The only indirect-effect statistic printed (CI −0.107 to 0.706, P = 0.146) fails the paper's own test.
- The significance call is reversed, not just loosely worded. Under reading 1 the paper reports a non-significant indirect effect and calls it significant. Under reading 2 it reports no test of the indirect effect and calls it significant. Neither reading allows the conclusion that mediation was shown.
- What is not affected. The behavioral improvement in K values after 1064-nm tPBM (experiment 1: t22 = 2.841, P = 0.009; experiment 2: t17 = 2.760, P = 0.013), the increase in the CDA set-size effect, the correlations between the two, and the null results at 852 nm and over the left PFC come from separate tests that this issue does not touch. The title's conclusion stands on those. A correlation between changes in CDA and changes in K is consistent with mediation, but it is not a test of it.
- The labels in the text also need checking. If reading 2 is right, the numbers labelled "direct effect" are the total effect c, and the stated estimate 0.245 does not belong to the interval printed with it (a symmetric interval of the same width around 0.245 would run from −0.166 to 0.656 and include zero).
Explanations considered
- A typo in the interval or the P value. Ruled out. The interval is centered on 0.300 (midpoint 0.2995), and the Wald P it implies (0.148) matches the printed 0.146. Estimate, interval and P are consistent with each other, and all three indicate a non-significant effect. A dropped minus sign (0.107 instead of −0.107) would not fit: the interval would then not be centered on 0.300, and P would not be near 0.15.
- A one-sided test. No one-sided test is mentioned anywhere. Even one-sided, P = 0.073 and the corresponding 90% interval (−0.041 to 0.641) still contains zero.
- A different significance convention. Ruled out by the paper itself. Methods states that "The null hypothesis was tested by examining whether zero was within the 95% bootstrap confidence intervals." Zero is within the printed indirect-effect interval.
- The words "direct" and "indirect" swapped. Not supported. A swap would make the indirect effect 0.245 with CI 0.021 to 0.844, P = 0.040, which would be significant. But 0.245 matches neither a × b (0.149) nor c − c′ (0.133), and that interval's midpoint and implied P match the total effect c = 0.433 instead. The numbers fit reading 2 (the statistics belong to c′ and c) better than a simple swap.
- Mediation inferred from the pattern of c and c′. Fig. 3A marks c = 0.433 as significant and c′ = 0.300 as not, which some older approaches read as evidence of mediation. The paper does not argue this. Its stated test is the bootstrap CI of the indirect effect, and the change from significant to non-significant is not itself a test of c − c′ (a c′ of 0.300 is 69% of c).
- Joint significance of paths a and b. Fig. 3A marks a (P < 0.05) and b (P < 0.01) as significant, and testing a and b jointly is one recognized way to test mediation. The paper does not use or report it, and its stated criterion is different. If the authors intend this test, it should be stated and reported, with the exact P values for a and b. As a side observation, b = 0.404 is the same value as the zero-order correlation r = 0.404 between CDA and K values reported for Fig. 3B, though in a mediation model b should be the mediator's coefficient after adjusting for session. The two can differ, so the authors may wish to confirm which value Fig. 3A shows.
- A text-conversion artifact. Ruled out. The archived JATS XML carries a true minus sign (U+2212) before 0.107, and the bioRxiv preprint (version 2) prints the same sentence and numbers.
- A correction already issued. None found (see Status).
Status
- Version of record. The publisher's page (science.org, reached through doi.org) returned HTTP 403 to automated requests. I checked the publisher-deposited text in the PMC archive (JATS XML and the PMC HTML page) and the Europe PMC full-text XML, which agree with all passages quoted here. Every quotation was checked verbatim against the PMC-archived text with
src/show.py. The bioRxiv preprint (doi:10.1101/2022.02.24.481703, version 2) has the same mediation sentence and numbers. - Supplements and data. The Supplementary Materials PDF (
sciadv.abq3211_sm.pdf, 4 pages, Figs. S1 to S3 on theta, alpha and beta band activity) does not mention mediation, the indirect effect or a bootstrap. The Fig. 3 image in the same Europe PMC bundle gives the path labels quoted above. The OSF project cited by the paper (doi:10.17605/OSF.IO/67YUS) holds behavioral data, preprocessed EEG files and MATLAB EEG analysis code. It has no mediation syntax or output, so the mediation cannot be rerun from the deposited materials without reprocessing the EEG. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations and no PMC or Europe PMC notices. The Crossref record has empty update-to and updated-by fields, and the PubMed record (PMID 36459562) has no CommentsCorrections entries. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title with "PubPeer", "correction" or "erratum" found no thread or notice. Crossref links one post-publication peer review on ScienceOpen (doi:10.14293/S2199-1006.1.SOR-UNCAT.A10936045.v1.RIGKMC, deposited 2026-03-16). Its page returned HTTP 403, so I could not tell whether it raises this point. - Sources. https://pmc.ncbi.nlm.nih.gov/articles/PMC10936045 ; Europe PMC full-text XML and supplementary files for PMC10936045; https://www.biorxiv.org/content/10.1101/2022.02.24.481703v2 ; https://doi.org/10.17605/OSF.IO/67YUS ; Crossref; PubMed; PubPeer; https://doi.org/10.1126/sciadv.abq3211.
Suggested correction
In the Results, the sentence
This mediation analysis revealed a significant indirect effect of the CDA set-size effect (indirect effect: 0.300, 95% confidence interval: −0.107 to 0.706, P = 0.146; direct effect: 0.245, 95% confidence interval: 0.021 to 0.844, P = 0.040).
should be checked against the original SPSS output. If those numbers are the indirect and direct effects, the sentence could read "This mediation analysis did not reveal a significant indirect effect of the CDA set-size effect (indirect effect: 0.300, 95% confidence interval: −0.107 to 0.706, P = 0.146), whereas the direct effect was significant (…)". If they are instead the c′ and c paths of Fig. 3A, the labels should be corrected, and the indirect effect (a × b) should be reported with its bootstrap 95% CI. In either case the abstract sentence ("The CDA set-size effect during retention mediated the effect…"), the Results heading, the Fig. 3 title, and the two Discussion sentences on mediation should be revised to match the corrected result. If the bootstrap CI of a × b excludes zero, the mediation claim could be kept with that statistic cited. If it does not, these passages should say that the CDA set-size effect correlated with the behavioral gain but that mediation was not shown. The authors may also wish to confirm that Fig. 3A's b is the mediator's coefficient adjusted for session rather than the zero-order correlation r = 0.404.
Recomputation script (python recompute.py.txt)
"""Recheck the mediation statistics of PMC10936045
(Sci Adv 2022, doi:10.1126/sciadv.abq3211,
"Transcranial photobiomodulation enhances visual working memory capacity in humans").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg / fig1.png next to this file.
Inputs, copied from the article as archived in PMC:
Results, "CDA mediates the WM improvement with tPBM (1064 nm) applied to the right PFC":
"indirect effect: 0.300, 95% confidence interval: -0.107 to 0.706, P = 0.146;
direct effect: 0.245, 95% confidence interval: 0.021 to 0.844, P = 0.040"
correlation "r = 0.404, P < 0.001, confidence interval: 0.154 to 0.654; Fig. 3B"
Fig. 3A path labels (read from the archived figure image sciadv.abq3211-f3.jpg):
a = 0.370*, b = 0.404**, c = 0.433*, c' = 0.300 (no asterisk); legend "*P < 0.05 and **P < 0.01".
Methods, Statistical analysis: "The null hypothesis was tested by examining whether zero
was within the 95% bootstrap confidence intervals."
The Wald-type checks below treat each 95% CI as estimate +/- 1.96 SE. That is only a
consistency check (a bootstrap CI need not be symmetric); the paper's own decision rule,
"is zero inside the 95% CI", needs no such assumption.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.ppf(0.975)
Z95 = stats.norm.ppf(0.95)
# ---------------------------------------------------------------- inputs
REP = {
"indirect": dict(est=0.300, lo=-0.107, hi=0.706, p=0.146),
"direct": dict(est=0.245, lo=0.021, hi=0.844, p=0.040),
}
FIG3A = dict(a=0.370, b=0.404, c=0.433, c_prime=0.300)
CORR = dict(r=0.404, lo=0.154, hi=0.654)
def wald(est, lo, hi):
se = (hi - lo) / (2 * Z975)
z = est / se
return se, z, 2 * stats.norm.sf(abs(z))
print("=== 1. Reported intervals: zero inside? midpoint, implied SE, Wald P ===")
R = {}
for name, d in REP.items():
mid = (d["lo"] + d["hi"]) / 2
se, z, p = wald(d["est"], d["lo"], d["hi"])
se_m, z_m, p_m = wald(mid, d["lo"], d["hi"])
R[name] = dict(mid=mid, se=se, z=z, p=p, p_mid=p_m, z_mid=z_m)
print(f"{name:9s} est {d['est']:.3f} CI [{d['lo']:.3f}, {d['hi']:.3f}] reported P {d['p']:.3f}")
print(f" zero inside 95% CI? {d['lo'] < 0 < d['hi']} reported P < 0.05? {d['p'] < 0.05}")
print(f" CI midpoint {mid:.4f}; est - lo = {d['est']-d['lo']:.3f}, hi - est = {d['hi']-d['est']:.3f}")
print(f" implied SE {se:.4f}; Wald z for the stated estimate {z:.3f}, P {p:.4f}")
print(f" Wald z for the CI midpoint {z_m:.3f}, P {p_m:.4f}")
ind = REP["indirect"]
se_i = R["indirect"]["se"]
print()
print("=== 2. Indirect effect under a one-sided reading ===")
print(f"one-sided P = reported P / 2 = {ind['p']/2:.3f}")
print(f"one-sided Wald P from CI = {stats.norm.sf(R['indirect']['z']):.4f}")
print(f"90% Wald CI = {ind['est'] - Z95*se_i:.3f} to {ind['est'] + Z95*se_i:.3f} (still contains 0)")
print()
print("=== 3. 'Direct effect' 0.245 with a symmetric interval of the reported width ===")
se_d = R["direct"]["se"]
print(f"0.245 +/- 1.96 x {se_d:.4f} = {REP['direct']['est'] - Z975*se_d:.3f} to {REP['direct']['est'] + Z975*se_d:.3f}")
print()
print("=== 4. Matching the reported statistics to the Fig. 3A path coefficients ===")
print(f"'indirect' estimate {ind['est']:.3f} and CI midpoint {R['indirect']['mid']:.4f} vs c' = {FIG3A['c_prime']:.3f} (no asterisk)")
print(f"'direct' CI midpoint {R['direct']['mid']:.4f} vs c = {FIG3A['c']:.3f} (*); "
f"Wald P for c on that CI = {wald(FIG3A['c'], REP['direct']['lo'], REP['direct']['hi'])[2]:.4f} (reported 0.040)")
ab = FIG3A["a"] * FIG3A["b"]
cdiff = FIG3A["c"] - FIG3A["c_prime"]
print(f"indirect effect implied by the paths: a x b = {ab:.4f}; c - c' = {cdiff:.4f}")
print(f"c' + a x b = {FIG3A['c_prime'] + ab:.4f} (c = {FIG3A['c']:.3f})")
print(f"reported 'indirect' 0.300 / a x b = {ind['est']/ab:.2f}")
print(f"c' / c = {FIG3A['c_prime']/FIG3A['c']:.3f}")
print()
print("=== 5. Side check: Fig. 3A path b vs Fig. 3B correlation ===")
print(f"b = {FIG3A['b']:.3f}; r = {CORR['r']:.3f}; r CI midpoint = {(CORR['lo']+CORR['hi'])/2:.3f}")
# ---------------------------------------------------------------- figure
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
fig, ax = plt.subplots(figsize=(7.2, 4.1))
ax.set_xlim(-0.35, 1.25)
Y = {"ind": 3.0, "dir": 2.0, "ab": 1.0}
ax.set_ylim(0.45, 3.65)
ax.set_yticks([Y["ind"], Y["dir"], Y["ab"]])
ax.set_yticklabels(["Text: “indirect\neffect”",
"Text: “direct\neffect”",
"Indirect effect\nfrom Fig. 3A paths"])
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axvline(0, color=INK2, lw=0.9, zorder=1)
ax.text(0.012, 0.55, "no effect (0)", ha="left", va="center", fontsize=10, color=INK2)
for key, row in [("indirect", "ind"), ("direct", "dir")]:
d, y = REP[key], Y[row]
ax.plot([d["lo"], d["hi"]], [y, y], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
for x in (d["lo"], d["hi"]):
ax.plot([x, x], [y - 0.07, y + 0.07], color=ORANGE, lw=2, zorder=2)
ax.plot(d["est"], y, "o", ms=10, color=ORANGE, zorder=4)
ax.plot(R[key]["mid"], y, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=5)
ax.text(d["hi"] + 0.03, y + 0.02, f"P = {d['p']:.3f} (reported)", ha="left", va="bottom",
fontsize=10, color=ORANGE)
ax.text(REP["indirect"]["hi"] + 0.03, 3.0 - 0.06, f"P = {R['indirect']['p']:.3f} (Wald, from CI)", ha="left", va="top",
fontsize=10, color=BLUE)
ax.text(REP["direct"]["hi"] + 0.03, 2.0 - 0.06, f"P = {R['direct']['p_mid']:.3f} (Wald,\nfor CI midpoint)",
ha="left", va="top", fontsize=10, color=BLUE)
# Fig. 3A path labels on the matching rows
ax.plot(FIG3A["c_prime"], Y["ind"] + 0.30, "v", ms=9, mfc=BG, mec=INK, mew=1.6, zorder=4)
ax.text(FIG3A["c_prime"] + 0.03, Y["ind"] + 0.30, "Fig. 3A c′ = 0.300 (no star)", ha="left", va="center",
fontsize=10, color=INK)
ax.plot(FIG3A["c"], Y["dir"] + 0.30, "v", ms=9, mfc=BG, mec=INK, mew=1.6, zorder=4)
ax.text(FIG3A["c"] + 0.03, Y["dir"] + 0.30, "Fig. 3A c = 0.433*", ha="left", va="center",
fontsize=10, color=INK)
ax.text(REP["indirect"]["est"], Y["ind"] - 0.22, "estimate 0.300 = midpoint", ha="center", va="top", fontsize=10, color=ORANGE)
ax.text(REP["direct"]["est"], Y["dir"] - 0.22, "0.245", ha="right", va="top", fontsize=10, color=ORANGE)
ax.text(R["direct"]["mid"], Y["dir"] - 0.22, f"midpoint {R['direct']['mid']:.4f}", ha="left", va="top",
fontsize=10, color=BLUE)
# a*b and c - c'
ax.plot(ab, Y["ab"] + 0.12, "s", ms=9, color=BLUE, zorder=4)
ax.text(ab + 0.04, Y["ab"] + 0.12, f"a×b = {ab:.3f}", ha="left", va="center", fontsize=10, color=BLUE)
ax.plot(cdiff, Y["ab"] - 0.12, "s", ms=9, mfc=BG, mec=BLUE, mew=2, zorder=4)
ax.text(cdiff + 0.04, Y["ab"] - 0.12, f"c − c′ = {cdiff:.3f}", ha="left", va="center", fontsize=10, color=BLUE)
ax.text(0.56, Y["ab"], "no CI or P for the indirect\neffect is reported anywhere", ha="left", va="center",
fontsize=10, color=INK2)
ax.set_xlabel("Standardized effect of 1064-nm tPBM on K values")
handles = [
Line2D([], [], marker="o", color=ORANGE, lw=2, ms=9, label="reported estimate and 95% CI (Results)"),
Line2D([], [], marker="D", ls="", ms=8, mfc=BG, mec=BLUE, mew=2, label="CI midpoint (recomputed)"),
Line2D([], [], marker="v", ls="", ms=9, mfc=BG, mec=INK, mew=1.6, label="path label printed in Fig. 3A"),
Line2D([], [], marker="s", ls="", ms=9, color=BLUE, label="a×b (filled), c − c′ (open), recomputed"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.45, -0.17), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
print()
print("wrote fig1.svg, fig1.png in", HERE)
# ---- fig2: which printed statistics match which Fig. 3A path
from matplotlib.patches import FancyBboxPatch # noqa: E402
fig, ax = plt.subplots(figsize=(7.2, 4.4))
ax.set_xlim(0, 10)
ax.set_ylim(0, 7.0)
ax.axis("off")
BW = 3.8
for x0, title in [(0.05, "Results text"), (6.15, "Fig. 3A path labels")]:
ax.add_patch(FancyBboxPatch((x0, 0.8), BW, 5.6, boxstyle="round,pad=0.02,rounding_size=0.08",
fc=BG, ec=INK2, lw=1.2))
ax.text(x0 + 0.15, 6.05, title, ha="left", va="center", fontsize=11, weight="bold", color=INK)
L, Rx = 0.2, 6.3
TAG = dict(ha="left", va="center", fontsize=10)
rows = [
(5.05, "printed", ORANGE, "“indirect effect” 0.300\n95% CI −0.107 to 0.706\nP = 0.146",
"c′ = 0.300 (no star)\ntPBM → K, controlling\nfor the CDA mediator"),
(3.35, "printed", ORANGE, "“direct effect” 0.245\n95% CI 0.021 to 0.844\nP = 0.040",
"c = 0.433*\ntPBM → K, without\nthe mediator"),
]
for y, tag, col, ltxt, rtxt in rows:
ax.text(L, y, ltxt, color=col, linespacing=1.35, **TAG)
ax.text(Rx, y, rtxt, color=col, linespacing=1.35, **TAG)
ax.text(L, 1.6, f"recomputed:\nCI midpoint {R['direct']['mid']:.4f}\n"
f"Wald P for 0.433 = {wald(FIG3A['c'], REP['direct']['lo'], REP['direct']['hi'])[2]:.3f}",
color=BLUE, linespacing=1.35, **TAG)
ax.text(Rx, 1.6, f"recomputed indirect effect:\na × b = {ab:.3f}\nc − c′ = {cdiff:.3f}",
color=BLUE, linespacing=1.35, **TAG)
for y, lab in [(5.05, "same estimate,\nboth unstarred"), (3.35, "interval\ncentred on 0.433")]:
ax.annotate("", xy=(6.05, y - 0.35), xytext=(3.95, y - 0.35),
arrowprops=dict(arrowstyle="<->", color=INK, lw=1.6))
ax.text(5.0, y - 0.2, lab, ha="center", va="bottom", fontsize=10, color=INK)
ax.text(5.0, 0.35, "No interval or P is reported for the indirect effect itself (about 0.13 to 0.15).",
ha="center", va="center", fontsize=10, color=INK2)
ax.text(5.0, 6.75, "orange = printed in the paper; blue = recomputed from those numbers",
ha="center", va="center", fontsize=10, color=INK2)
save(fig, "fig2")
print("wrote fig2.svg, fig2.png in", HERE)
Rapa Nui population estimates in Table 3 use 760 ha, but 0.76 km² is 76 ha
Sci Adv (2024) · doi:10.1126/sciadv.ado1459 · PMC11192071
Summary
The paper maps 0.76 km² of rock gardens on Rapa Nui and uses this area in Table 3 to estimate how many people sweet potatoes grown in the gardens could have fed. Since 1 km² = 100 ha, 0.76 km² is 76 ha, but Table 3 enters 760 ha (380 ha and 126.67 ha for the two shifting-cultivation scenarios). The table's own arithmetic reproduces every printed population from these hectare figures, so all six "this study" estimates are ten times what the paper's stated area implies. With 76 ha, the maximum falls from 3,901 to about 390 people, and the "~3000" that the paper says "matches historical accounts" becomes about 300. The abstract's conclusion, that gardening could not support the large populations assumed before, is unaffected and, if anything, stronger.
What the paper reports
Results, after the model output was cleaned by hand:
The resulting dataset contains a total of 0.76 km2 of rock gardens across Rapa Nui (Fig. 1).
Discussion, first paragraph:
Ladefoged et al.’s (26) mulching estimates ranged from 4.3 to 21.1 km2. Our study estimates indicate that the total area of rock gardening is 0.76 km2, approximately one-fifth of the most conservative previous estimates, including many misclassified colluvium areas, lava flows, and roadways.
Discussion, "Reevaluating population estimates", first paragraph, giving the conversion factors:
Puleston et al. (6) rely on estimates of sweet potato yields to approximate the pre-European contact population size of Rapa Nui and report that 1 ton/year of sweet potato produces a yield of 2809 kcal/day.
(maximum of 2785 kcal per day per person)
Table 3 (column header, then the six "This study" rows):
Cultivation frequency | N availability | Sweet potato yield (tons/ha/year) | Rock garden area estimate | Land available for rock gardening (ha) | Sweet potato yield (kcal/day) | Number of individuals supported
Continuous | Low | 1.46 | This study | 760 | 3,116,886.4 | 1,119 Continuous | High | 5.09 | This study | 760 | 10,866,335.6 | 3,901 Shifting (5/5) | Low | 2.38 | This study | 380 | 2,540,459.6 | 912 Shifting (5/5) | High | 8.00 | This study | 380 | 8,539,360.0 | 3,066 Shifting (15/3) | Low | 5.61 | This study | 126.67 | 1,996,075.4 | 716 Shifting (15/3) | High | 17.60 | This study | 126.67 | 6,262,384.6 | 2,248
Table 3 caption:
We use the same ratios as Puleston et al. (6) (100% for continuous, 50% for shifting 5 years off 5 years on, and 25% for shifting 15 years off 3 years on).
Discussion, conclusions drawn from Table 3:
In contrast, baseline estimates of the population size using our new rock gardening data suggest that population sizes could not have exceeded 4000 (Table 3).
The average carrying capacity of rock gardening calculated in this study is ~2000. If we add 50% to this estimate to account for marine and additional terrestrial foods from nonmulched areas, we arrive at an estimated carrying capacity of ~3000 people. This value matches historical accounts of population size at the time of European arrival in the 18th century (44, 45).
Our estimates suggest that the maximum population supported by rock gardening is not ~17,000 as claimed through Ladefoged et al.’s (26) rock gardening calculations but just 3901 using our measurements.
Introduction, paragraph setting out the study's aims:
We demonstrate that the extent of rock gardening cultivation found in the occupied coastal areas comports with estimates of the population from observations made by early European visitors to the island in the 18th century, which is about 3000 (36).
Abstract, final sentence:
We show that the extent of this agricultural infrastructure is substantially less than previously claimed and likely could not have supported the large population sizes that have been assumed.
What the paper's own numbers show
Method. Table 3 follows the model the Discussion describes: daily kcal = yield (tons/ha/year) × area (ha) × 2809, and people supported = kcal ÷ 2785. I first applied this to all twelve printed rows, using the printed hectares, to establish which area each row actually used. I then recomputed the six "this study" rows with the paper's stated 0.76 km² = 76 ha, keeping the yields and the area fractions the table applies (1, 1/2 and 1/6 for continuous, 5/5 and 15/3 cultivation). Finally I checked which of the two areas fits the paper's own verbal comparisons with earlier estimates. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Mapped rock-garden area | Results; Discussion | 0.76 km² | = 76 ha (1 km² = 100 ha) |
| Land available, continuous | Table 3 | 760 ha | 76 ha; 760 ha would be 7.6 km² |
| Land available, shifting 5/5 and 15/3 | Table 3 | 380 ha; 126.67 ha | 38 ha; 12.67 ha (380/760 = 1/2; 126.67/760 = 1/6) |
| Printed populations reproduced from printed hectares | Table 3, all 12 rows | e.g. 1,119; 3,901; 16,089 | 1,119.2; 3,901.7; 16,089.0 (every row agrees to rounding) |
| People supported, continuous, low N / high N | Table 3 | 1,119 / 3,901 | 111.9 / 390.2 |
| People supported, shifting 5/5, low N / high N | Table 3 | 912 / 3,066 | 91.2 / 306.6 |
| People supported, shifting 15/3, low N / high N | Table 3 | 716 / 2,248 | 71.7 / 224.9 |
| Ratio printed / recomputed, each row | Table 3 | — | 10.00 (9.99 for one row) |
| Upper bound | Discussion | "could not have exceeded 4000" | about 390 |
| Maximum | Discussion | "just 3901" | 390.2 |
| Average of the six rows | Discussion | "~2000" | 1,993.7 as printed; 199.4 with 76 ha |
| Average + 50% for other foods | Discussion | "~3000", "matches historical accounts" | 2,990.5 as printed; 299.1 with 76 ha (10% of ~3000) |
| Previous minimum / this study | Discussion: "approximately one-fifth" | ~1/5 | 4.3/0.76 = 5.7 (fits 0.76 km²); 4.3/7.6 = 0.57 (7.6 km² would exceed the previous minimum) |
| Previous range / this study | Discussion, last section: "between 5 and 20 times too high" | 5 to 20 | 5.7 to 27.8 with 0.76 km²; 0.57 to 2.8 with 7.6 km² |
Three smaller points, none of which affects the factor of ten:
- The continuous low-N row prints 3,116,886.4 kcal/day; 1.46 × 760 × 2809 = 3,116,866.4, a difference of 20.0 kcal/day. The 15/3 rows were evidently computed with 760/6 = 126.6667 ha, which reproduces the printed 1,996,075.4 exactly.
- The caption says 25% of the area for 15/3 shifting cultivation, but both blocks of the table apply 1/6 (522.3/3,133.9 = 0.1667; 126.67/760 = 0.1667).
- The lower bound of the earlier estimate appears as 4.9 km² in the Introduction ("between 4.9 and 21.1 km2 of the island’s total 163.6 km2 area") and as 4.3 km² in the Discussion. The comparison above holds with either.
Figure
Figure 1. The calculation chain from the mapped area to the carrying capacity the paper compares with historical accounts. The left column (orange outline) gives the values printed in Table 3 and the Discussion; the right column (blue outline) gives the values recompute.py.txt obtains from the same formula and the paper's stated 0.76 km². The two columns diverge at one step only, the conversion of km² to hectares (shaded): 0.76 km² is 76 ha, and Table 3 uses 760 ha. Every later value differs by the same factor of ten. Maximum: continuous cultivation, high soil nitrogen (5.09 t/ha/yr × ha × 2809 ÷ 2785). Average: mean of the six "This study" rows of Table 3.
Figure 2. People supported in each of the six "This study" scenarios of Table 3 (cultivation frequency × soil nitrogen), and the average of the six with and without the paper's 50% addition for marine and other foods. Orange: the table's printed values, computed from 760, 380 and 126.67 ha (for the two summary rows, the mean of the printed values, which the text rounds to ~2000 and ~3000). Blue: the same formula with 76, 38 and 12.67 ha. The vertical line marks the ~3000 of historical accounts that the paper says its adjusted estimate matches. The horizontal axis is logarithmic, so the constant factor of ten appears as equal-length connectors; all values are drawn from recompute.py.txt.
Why it matters
- The paper's headline numbers on population depend on Table 3. The maximum of 3,901, the bound of 4,000, the average of ~2,000 and the adjusted ~3,000 all come from the six "this study" rows. Each is ten times what the paper's own mapped area gives.
- The claimed agreement with historical observations does not hold under the stated area. The paper presents its ~3,000 as matching the population observed by Europeans in the 18th century, and the Introduction states that the extent of rock gardening "comports with" that figure. With 76 ha, the same reasoning gives about 300 people, about a tenth of the observed figure. Under the paper's model, rock gardens of 0.76 km² would then feed only a small share of the population recorded at contact, and the remainder would have to come from other sources.
- The error travelled beyond the paper. The institutional press release (Columbia Climate School, State of the Planet, 21 June 2024) gives the area correctly as "about 188 acres" (76.1 ha) but repeats the table-derived "about 2,000 people" and "about 3,000".
- The direction of the main argument is unaffected. The abstract says only that gardening "likely could not have supported the large population sizes that have been assumed". A smaller carrying capacity strengthens rather than weakens that statement. The mapped area itself, as stated in the text, is not affected by this inconsistency.
Explanations considered
- The table is right and the text should say 7.6 km². Not supported by the paper. The area is given as 0.76 km² twice (Results and Discussion). The Discussion calls it "approximately one-fifth of the most conservative previous estimates" (4.3 km²): 0.76 km² is about 1/5.7 of that, while 7.6 km² would be larger than the previous minimum. The final section says previous estimates were "between 5 and 20 times too high", which fits 0.76 km² (5.7 to 27.8 times) and not 7.6 km² (0.57 to 2.8 times). The press release's 188 acres is 76.1 ha.
- "Land available" means something other than the mapped area, such as a buffer around the gardens. Not supported. The column is headed "Land available for rock gardening (ha)", the row is labelled "This study", the caption says the rows use "our revised estimates for rock garden areas", and the shifting rows are exactly 1/2 and 1/6 of 760, as the (26) rows are of 3,133.9 ha. No other area of 7.6 km² appears in the paper.
- A rounding or typesetting slip in the table only. Not supported. If 760 were a mistyped 76, the kcal and population columns would have been computed from 76 ha. They were not: yield × 760 × 2809 ÷ 2785 reproduces every printed population, and the Discussion's ~2,000 (mean of the printed rows 1,993.7), ~3,000 (2,990.5) and 3,901 all follow from the 760-ha values.
- The version archived in PMC differs from the version of record. Not supported. The PMC text, the Europe PMC full-text XML and a copy of the publisher's typeset PDF all give 0.76 km² in the text and 760 / 380 / 126.67 ha in Table 3 (see Status).
- A text extraction or conversion artefact (for example a lost decimal point). Not supported: the typeset PDF has the same numbers, and the kcal column is consistent with the 760-ha figure, not with 76 ha.
Status
- Version of record. The publisher's site (science.org, reached through the DOI) returned HTTP 403 to automated requests. The PMC article (PMC11192071), the Europe PMC full-text XML (PMC11192071, version 1) and a copy of the publisher-typeset PDF hosted on a Columbia University laboratory site (metadata "Sci. Adv. 2024.10:eado1459"; pages stamped as downloaded from science.org on 21 June 2024) give the same text and numbers, including the Table 3 rows quoted above. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements and data. Crossref lists no journal-hosted supplementary material. The data statement points to Dryad (doi:10.5061/dryad.gqnk98swd) and a GitHub repository; the Dryad record has versions dated 2024-05-15, 2024-06-13 and 2024-11-15 with identical files (the later two are metadata changes). I downloaded the GitHub copy.
Supplemental_Code.RandREADME.mdcontain no area total and no Table 3 calculation.Mulch_Estimation.zipholdsMulch_Cleaned.tif, a rock-garden raster with 3.7 m cells (EPSG:32712) and 750,880 garden cells, which is 10.28 km² (1,028 ha) at 13.69 m² per cell. That matches neither the text's 0.76 km² (76 ha) nor the table's 760 ha. The deposited files do not state how 0.76 km² was obtained, so this observation does not settle what the mapped area is; it is one more reason for the area to be stated explicitly (see below). - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations and no PMC notices; its Europe PMC query was unavailable at the time of the check. A Crossref search for works related to this DOI returned none. PubPeer's API required human verification and could not be queried automatically; web searches for the title and DOI together with "PubPeer", and for published comments or replies, found no PubPeer thread, technical comment or author response. - Sources. PMC article PMC11192071; Europe PMC full-text XML; publisher-typeset PDF (copy at obtlab.climate.columbia.edu); Dryad doi:10.5061/dryad.gqnk98swd; GitHub repository named in the data statement (
Mulch_Estimation.zip,Supplemental_Code.R,README.md); Crossref; PubPeer; Columbia Climate School State of the Planet, "Study Challenges Popular Idea That Easter Islanders Committed 'Ecocide'", 21 June 2024; https://doi.org/10.1126/sciadv.ado1459.
Suggested correction
If the mapped area is 0.76 km², Table 3's "This study" rows should read 76, 38 and 12.67 ha, with kcal/day and people supported of about 311,687 and 112 (continuous, low N), 1,086,634 and 390 (continuous, high N), 254,046 and 91 (5/5, low N), 853,936 and 307 (5/5, high N), 199,608 and 72 (15/3, low N), and 626,220 and 225 (15/3, high N). The Discussion would then read "could not have exceeded ~400" instead of 4000, an average of ~200 instead of ~2000, ~300 instead of ~3000 after the 50% addition, and a maximum of 390 instead of 3901. The sentences on agreement with the ~3000 of historical accounts, in the Introduction and the Discussion, would need to be revised accordingly.
Because the deposited raster sums to a third value (10.28 km² at its native cell size), the authors may wish to state how the 0.76 km² total was computed and confirm it before revising Table 3. Whatever the confirmed area, the text and Table 3 should use the same figure. The caption's "25%" for 15/3 shifting cultivation should be changed to the 1/6 the table applies (or the table recomputed with 25%), and the minimum of the earlier estimate should be given consistently as 4.3 or 4.9 km².
Recomputation script (python recompute.py.txt)
"""Recompute Table 3 of PMC11192071
(Sci Adv 2024, doi:10.1126/sciadv.ado1459): rock-garden area and population estimates.
Run: python recompute.py.txt (uses the inputs recorded below)
python recompute.py.txt <Mulch_Cleaned.tif> (also re-counts the deposited raster;
needs rasterio)
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PubMed Central:
Results: "The resulting dataset contains a total of 0.76 km2 of rock gardens ..."
Discussion: 1 ton/year of sweet potato = 2809 kcal/day; 2785 kcal per person per day
Table 3: yields (tons/ha/year), land available (ha), kcal/day and people, all 12 rows
"""
import os
import sys
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (paper)
AREA_KM2 = 0.76 # Results and Discussion
KCAL_PER_TON_YR = 2809.0 # kcal/day per ton/year of sweet potato (Discussion)
KCAL_PER_PERSON = 2785.0 # kcal per person per day (Discussion)
ISLAND_KM2 = 163.6 # Introduction
LADEFOGED_KM2 = {"Discussion minimum": 4.3, "Introduction minimum": 4.9, "maximum": 21.1}
HISTORICAL = 3000 # "about 3000" (Introduction), "~3000 ... matches historical accounts"
# Table 3 rows: (frequency, N availability, yield t/ha/yr, source, ha, kcal/day, people)
TABLE3 = [
("Continuous", "Low", 1.46, "(26)", 3133.9, 12852562.6, 4614),
("Continuous", "High", 5.09, "(26)", 3133.9, 44807906.8, 16089),
("Shifting (5/5)", "Low", 2.38, "(26)", 1566.9, 10475384.6, 3761),
("Shifting (5/5)", "High", 8.00, "(26)", 1566.9, 35211376.8, 12643),
("Shifting (15/3)", "Low", 5.61, "(26)", 522.3, 8230659.3, 2955),
("Shifting (15/3)", "High", 17.60, "(26)", 522.3, 25821676.3, 9271),
("Continuous", "Low", 1.46, "This study", 760.0, 3116886.4, 1119),
("Continuous", "High", 5.09, "This study", 760.0, 10866335.6, 3901),
("Shifting (5/5)", "Low", 2.38, "This study", 380.0, 2540459.6, 912),
("Shifting (5/5)", "High", 8.00, "This study", 380.0, 8539360.0, 3066),
("Shifting (15/3)", "Low", 5.61, "This study", 126.67, 1996075.4, 716),
("Shifting (15/3)", "High", 17.60, "This study", 126.67, 6262384.6, 2248),
]
# fraction of the area cultivated in each year, as Table 3 applies it (see check 3)
FRACTION = {"Continuous": 1.0, "Shifting (5/5)": 0.5, "Shifting (15/3)": 1.0 / 6.0}
# Deposited raster (Dryad doi:10.5061/dryad.gqnk98swd / GitHub copy, Mulch_Estimation.zip):
RASTER_GARDEN_CELLS = 750880 # cells with value 5 in Mulch_Cleaned.tif
RASTER_CELL_M = 3.7 # cell size, metres (EPSG:32712)
# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def line(s=""):
print(s)
# ---------------------------------------------------------------- 1. unit conversion
line("1. Area conversion (1 km2 = 100 ha)")
AREA_HA = AREA_KM2 * 100
line(f" 0.76 km2 = {AREA_HA:.0f} ha ; Table 3 uses 760 ha = {760 / 100:.1f} km2 "
f"(ratio {760 / AREA_HA:.0f})")
line(f" share of island: 0.76 km2 = {AREA_KM2 / ISLAND_KM2 * 100:.2f}% ; "
f"7.6 km2 = {7.6 / ISLAND_KM2 * 100:.2f}% of {ISLAND_KM2} km2")
ACRE_HA = 0.40468564224
line(f" press release '188 acres' = {188 * ACRE_HA:.1f} ha ; "
f"'7,700 acres, or 19% of the island' (2017 study) = {7700 * ACRE_HA:,.0f} ha")
line(f" (26) rows base: 3133.9 ha = {3133.9 / 100:.2f} km2 = "
f"{3133.9 / 100 / ISLAND_KM2 * 100:.1f}% of the island")
# ---------------------------------------------------------------- 2. Table 3 internal check
line("\n2. Table 3 internal check: kcal = yield x ha x 2809 ; people = kcal / 2785")
line(f" {'source':10} {'frequency':16} {'N':4} {'ha':>8} {'kcal printed':>14} "
f"{'kcal recomputed':>16} {'people printed':>15} {'recomputed':>10}")
for f, n, y, s, ha, kc, pp in TABLE3:
k = y * ha * KCAL_PER_TON_YR
line(f" {s:10} {f:16} {n:4} {ha:8.2f} {kc:14,.1f} {k:16,.1f} {pp:15,d} {k / KCAL_PER_PERSON:10.1f}")
line(" -> every printed population is reproduced (to rounding) from the printed hectares,")
line(" so the 'This study' rows were computed with 760 / 380 / 126.67 ha.")
k1 = 1.46 * 760 * KCAL_PER_TON_YR
line(f" minor digit slip: 1.46 x 760 x 2809 = {k1:,.1f} (printed 3,116,886.4; "
f"difference {3116886.4 - k1:.1f})")
for y, kc in ((5.61, 1996075.4), (17.60, 6262384.6)):
line(f" 15/3 row, yield {y}: area implied by printed kcal = {kc / y / KCAL_PER_TON_YR:.4f} ha "
f"(printed 126.67; 760/6 = {760 / 6:.4f})")
# ---------------------------------------------------------------- 3. area fractions
line("\n3. Area fractions used in Table 3")
line(f" (26): 1566.9/3133.9 = {1566.9 / 3133.9:.4f} ; 522.3/3133.9 = {522.3 / 3133.9:.4f}")
line(f" This study: 380/760 = {380 / 760:.4f} ; 126.67/760 = {126.67 / 760:.4f}")
line(" caption says 25% for 15/3; the table applies 1/6 in both blocks")
# ---------------------------------------------------------------- 4. corrected rows
line(f"\n4. 'This study' rows recomputed with {AREA_HA:.0f} ha (the paper's 0.76 km2), same fractions")
rep, cor, labels = [], [], []
for f, n, y, s, ha, kc, pp in TABLE3[6:]:
a = AREA_HA * FRACTION[f]
k = y * a * KCAL_PER_TON_YR
p = k / KCAL_PER_PERSON
rep.append(pp)
cor.append(p)
labels.append(f"{f}, {n} N")
line(f" {f:16} {n:4} ha={a:6.2f} kcal/day={k:12,.1f} people={p:7.1f} "
f"(printed {pp:,d}; printed/recomputed = {pp / p:.2f})")
rep, cor = np.array(rep, float), np.array(cor)
mean_rep, mean_cor = rep.mean(), cor.mean()
line(f" mean of six printed values = {mean_rep:.1f} (text: ~2000); x1.5 = {mean_rep * 1.5:.1f} (text: ~3000)")
line(f" mean of six recomputed = {mean_cor:.1f}; x1.5 = {mean_cor * 1.5:.1f}")
line(f" maximum printed = {rep.max():.0f} (text: 'just 3901'); recomputed = {cor.max():.1f}")
line(f" recomputed maximum as share of 'about 3000': {cor.max() / HISTORICAL * 100:.0f}% ; "
f"recomputed mean x1.5 as share: {mean_cor * 1.5 / HISTORICAL * 100:.0f}%")
# ---------------------------------------------------------------- 5. text comparisons
line("\n5. The paper's own comparisons of area, under each reading")
for name, v in LADEFOGED_KM2.items():
line(f" Ladefoged {name} {v} km2: / 0.76 = {v / AREA_KM2:.1f}x ; / 7.6 = {v / 7.6:.2f}x")
line(" 'approximately one-fifth' and 'between 5 and 20 times too high' fit 0.76 km2;")
line(" 7.6 km2 would be larger than both stated minima (4.3 and 4.9 km2).")
line(f" Table 3 ratio of bases 3133.9/760 = {3133.9 / 760:.2f} ; 3133.9/76 = {3133.9 / 76:.1f}")
# ---------------------------------------------------------------- 6. deposited raster
line("\n6. Deposited raster Mulch_Cleaned.tif")
cells, cell = RASTER_GARDEN_CELLS, RASTER_CELL_M
if len(sys.argv) > 1:
import rasterio # optional
with rasterio.open(sys.argv[1]) as r:
a = r.read(1)
cells, cell = int((a == 5).sum()), float(r.res[0])
line(f" re-counted from {os.path.basename(sys.argv[1])}")
area_r = cells * cell * cell / 1e6
line(f" {cells:,d} garden cells x {cell} m x {cell} m = {area_r:.2f} km2 = {area_r * 100:,.0f} ha")
line(f" {cells:,d} cells x 1 m2 = {cells / 1e6:.2f} km2 (numerical observation only)")
# ---------------------------------------------------------------- figure 1: calculation flow
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 5.6)
ax.axis("off")
XL, XO, XB, W = 0.18, 2.45, 5.0, 2.0 # step-label x, orange-column x, blue-column x, box width
rows_y = [4.55, 3.55, 2.55, 1.55, 0.55] # box centres (top to bottom)
H = 0.62
def box(x, y, w, text, edge, lw=2.0, fill=BG, weight="normal", size=10.5):
ax.add_patch(FancyBboxPatch((x, y - H / 2), w, H, boxstyle="round,pad=0.02,rounding_size=0.08",
fc=fill, ec=edge, lw=lw))
ax.text(x + w / 2, y, text, ha="center", va="center", fontsize=size, color=INK, weight=weight)
def arrow(x, y0, y1, color):
ax.annotate("", xy=(x, y1), xytext=(x, y0),
arrowprops=dict(arrowstyle="-|>", color=color, lw=2, mutation_scale=12,
shrinkA=0, shrinkB=0))
# column headers (they act as the legend)
ax.text(XO + W / 2, 5.38, "As printed in the paper", ha="center", va="center",
fontsize=10.5, weight="bold", color=INK)
ax.add_patch(Rectangle((XO + 0.05, 5.24), W - 0.1, 0.035, color=ORANGE, lw=0))
ax.text(XB + W / 2, 5.38, "Recomputed from 0.76 km²", ha="center", va="center",
fontsize=10.5, weight="bold", color=INK)
ax.add_patch(Rectangle((XB + 0.05, 5.24), W - 0.1, 0.035, color=BLUE, lw=0))
# highlighted divergence band (row 2)
ax.add_patch(Rectangle((0.08, rows_y[1] - 0.47), 7.04, 0.94, fc="#fbe9e1", ec="none", zorder=0))
steps = [
("Mapped rock-garden area\n(Results; Discussion)", None, None),
("Converted to hectares\n(Table 3, land available)", "760 ha\n(= 7.6 km²)",
f"{AREA_HA:.0f} ha\n(0.76 × 100)"),
("Most people supported\n(continuous, high N)",
f"{rep.max():,.0f}", f"{cor.max():.0f}"),
("Average of the six\n'This study' rows", "~2000", f"{mean_cor:.0f}"),
("Average + 50% for\nmarine and other foods", "~3000",
f"{mean_cor * 1.5:.0f}"),
]
for i, (lab, o, b) in enumerate(steps):
y = rows_y[i]
ax.text(XL, y, lab, ha="left", va="center", fontsize=10, color=INK2, linespacing=1.25)
if i == 0:
box(XO, y, XB + W - XO, "0.76 km²", INK2, lw=1.5, weight="bold")
continue
box(XO, y, W, o, ORANGE, weight="bold", size=10.5 if "\n" not in o else 10)
box(XB, y, W, b, BLUE, weight="bold", size=10.5 if "\n" not in b else 10)
arrow(XO + W / 2, rows_y[i - 1] - H / 2 - 0.02, y + H / 2 + 0.02, ORANGE)
arrow(XB + W / 2, rows_y[i - 1] - H / 2 - 0.02, y + H / 2 + 0.02, BLUE)
# annotation on the divergent step
ax.text(XO + W + 0.02 + (XB - XO - W) / 2 - 0.02, rows_y[1], "×10", ha="center", va="center",
fontsize=11, weight="bold", color=INK)
ax.text(XO + W / 2, rows_y[4] - H / 2 - 0.17, "paper: “matches historical accounts”", ha="center",
va="center", fontsize=10, color=INK2, style="italic")
ax.text(XL, rows_y[1] - 0.40, "the factor of 10 enters here", ha="left", va="center",
fontsize=10, color=INK, style="italic")
fig.savefig(os.path.join(HERE, "fig1.svg"))
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200)
plt.close(fig)
# ---------------------------------------------------------------- figure 2: all six rows
names = labels + ["Average of six rows", "Average + 50%"]
r_vals = list(rep) + [mean_rep, mean_rep * 1.5]
c_vals = list(cor) + [mean_cor, mean_cor * 1.5]
fig, ax = plt.subplots(figsize=(7.2, 4.3))
fig.subplots_adjust(left=0.30, right=0.95, top=0.86, bottom=0.14)
ypos = np.arange(len(names))[::-1].astype(float)
ypos[-2:] -= 0.0
ypos[:6] += 0.5 # small gap above the two summary rows
for y, rv, cv in zip(ypos, r_vals, c_vals):
ax.plot([cv, rv], [y, y], color=GRID, lw=2, zorder=1, solid_capstyle="butt")
ax.scatter(r_vals, ypos, s=64, color=ORANGE, zorder=3, label="As printed (from 760 / 380 / 126.67 ha)",
edgecolors=BG, linewidths=1.5)
ax.scatter(c_vals, ypos, s=64, color=BLUE, zorder=3, label="Recomputed (from 76 / 38 / 12.67 ha)",
edgecolors=BG, linewidths=1.5)
ax.axvline(HISTORICAL, color=INK2, lw=1, zorder=0)
ax.text(HISTORICAL / 1.04, ypos[0] + 0.62, "historical accounts cited by the paper: ~3000",
fontsize=10, color=INK2, va="center", ha="right")
ax.set_xscale("log")
ax.set_xlim(50, 10000)
ax.set_ylim(-0.7, ypos[0] + 1.1)
ax.set_xticks([50, 100, 200, 500, 1000, 2000, 5000, 10000])
ax.set_xticklabels(["50", "100", "200", "500", "1,000", "2,000", "5,000", "10,000"])
ax.minorticks_off()
ax.set_yticks(ypos)
ax.set_yticklabels(names, fontsize=10, color=INK)
ax.set_xlabel("People supported by rock gardens (log scale)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
for y, rv, cv in zip(ypos, r_vals, c_vals):
ax.text(rv * 1.12, y, f"{rv:,.0f}", va="center", ha="left", fontsize=10, color=INK)
ax.text(cv / 1.12, y, f"{cv:,.0f}", va="center", ha="right", fontsize=10, color=INK)
ax.legend(loc="lower left", bbox_to_anchor=(-0.42, 1.0), ncol=2, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
fig.savefig(os.path.join(HERE, "fig2.svg"))
fig.savefig(os.path.join(HERE, "fig2.png"), dpi=200)
plt.close(fig)
line("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png")
Reported CO2 storage capacities per km³ are about 920 times what the paper's inputs give
Sci Adv (2024) · doi:10.1126/sciadv.adq0515 · PMC11566992
Summary
The paper's only quantitative estimate of reservoir storage capacity is a first-order calculation. It fills the 5.7% effective porosity of a basalt reservoir with a fluid carrying 0.65 M CO2 (6.5 M once the chelating agent GLDA has fully degraded) and reports capacities of 1.5 and 15 Gt CO2 per km³. Multiplying the paper's own inputs (10⁹ m³ × 0.057 × 650 mol/m³ × 44.01 g/mol) gives 0.0016 and 0.016 Gt per km³ (1.6 and 16 Mt). The reported values are about 920 times larger, which fits a megatonne-to-gigatonne unit slip, and they would require a pore volume 52 times the volume of the rock. The same inflated figure produces the claim that "up to 20 Gt of CO2 annually" fits in 1.3 km³ (1/25,000 of Iceland's active rift zone): with the paper's inputs, 1.3 km³ holds about 0.021 Gt, and the stated "two orders of magnitude higher" comparison with the cited 0.18 Gt/km³ becomes about 11 times lower.
What the paper reports
All passages are from the article as archived in PubMed Central. The archive text writes km3 and CO2 for km³ and CO₂. Where the paper names the cited study (its reference 17) by author, the name is replaced here by "[the cited study]"; everything else is verbatim.
The inputs:
"Specifically, the total porosity ranged from 10.99 to 15.23%, of which only 18.7 to 57.5% were connected porosity, resulting in a mean effective porosity of 4.8%. Assuming an enhancement in pore properties comparable to that observed in the experiment with injection of the acidic GLDA solution, the effective porosity was anticipated to increase from 4.8 to 5.7%." (Results, "CO2 mineralization with GLDA biodegradation and overall evaluation", para 2)
"if a solution containing 20 wt % GLDA-Na4 (C9H9NNa4O8) was used for injection, then CO2 concentration is expected to increase 10-fold, from 0.65 M to 0.65 + 0.65 × 9 (number of carbon atoms in GLDA) = 6.5 M." (Same subsection, para 1)
The capacity calculation and the claims built on it:
"When filling all effective porosity with a CO2-charged 20 wt % GLDA-Na4 seawater solution (CO2 concentration estimated as 0.65 M), the reservoir exhibited a CO2 storage capacity of 1.5 Gt/km3." (Same subsection, para 2)
"With complete degradation of GLDA, the concentration of CO2 in the fluid would attain a value of 6.5 M, resulting in a reservoir capable of storing 15 Gt of CO2/km3, two orders of magnitude higher than the value reported by [the cited study] (17) based on CO2 capture and storage using fresh water (a mean value of 0.18 Gt of CO2/km3)." (Same subsection, para 2)
"Up to 20 Gt of CO2 annually could potentially be sequestered in a 1.3 km3 reservoir volume, which is 1/25,000 of the active rift zone in Iceland (area of 33,000 km2, thickness of 1 km)." (Same subsection, para 2)
The abstract's closing claim:
"Promising CO2 storage and mineralization capacities two orders higher than current techniques, this approach reduces required reservoir volume while enhancing efficiency." (Abstract)
The paper uses M for mol/L throughout. For example, the Methods give "~0.0325, 0.065, 0.162, 0.325, and 0.65 M" for 1 to 20 wt % GLDA solutions (Materials and Methods, "Powder experiments", para 2).
What the paper's own numbers show
Method. The paper's stated rule is to fill all effective porosity of the rock with fluid at the stated CO2 concentration. For 1 km³ of rock (10⁹ m³) this gives:
capacity = 10⁹ m³ × effective porosity × concentration (mol/L × 1,000 L/m³) × 44.01 g/mol ÷ 10¹⁵ g/Gt.
The only constant in the capacity calculation not taken from the paper is the molar mass of CO2. One ceiling uses an assumed dense-CO2 density of 700 kg/m³, and one comparison uses an assumed basalt density of 2,950 kg/m³. recompute.py.txt evaluates the expression, inverts it to find the porosity the reported values would need, and recomputes the 1.3 km³ and 1/25,000 figures. It also takes the cited 0.18 Gt/km³ back to the cited study's own tables.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Pore fluid in 1 km³ at 5.7% effective porosity | Results, "CO2 mineralization …", para 2 | not stated | 5.7 × 10⁷ m³ |
| CO2 in that fluid at 0.65 M | same | not stated | 3.7 × 10¹⁰ mol = 1.63 × 10¹² g |
| Storage capacity at 0.65 M | same | 1.5 Gt/km³ | 0.0016 Gt/km³ (1.6 Mt/km³); reported is 920× larger |
| Concentration after full GLDA degradation | same, para 1 | 0.65 + 0.65 × 9 = 6.5 M | 6.5 M (consistent) |
| Storage capacity at 6.5 M | same, para 2 | 15 Gt/km³ | 0.016 Gt/km³ (16 Mt/km³); reported is 920× larger |
| Effective porosity the reported values would need | derived | 5.7% stated | 5,244% (52 m³ of fluid per m³ of rock), at either concentration |
| CO2 mass per m³ of rock implied by the reported values | derived | 1.5 / 15 Gt/km³ | 1,500 / 15,000 kg per m³ of rock |
| Ceiling: 6.5 M fluid filling 100% of the rock | derived | – | 0.29 Gt/km³ |
| Ceiling: dense CO2 (700 kg/m³, assumed) filling all 5.7% pores | derived | – | 0.040 Gt/km³ |
| Reservoir volume for 20 Gt | same, para 2 | 1.3 km³ = 1/25,000 of 33,000 km³ | 20 / 15 = 1.33 km³ and 33,000 / 25,000 = 1.32 km³ (the reported chain is self-consistent with 15 Gt/km³) |
| CO2 held in 1.3 km³ at 6.5 M | same | "up to 20 Gt" | 0.021 Gt (21 Mt) |
| Reservoir volume for 20 Gt at the recomputed capacity | derived | 1.3 km³ | 1,227 km³ = 1/27 of the 33,000 km³ rift zone |
| Cited comparator, from the cited study's own tables | cited study, Tables 7 and 8 | 0.18 Gt/km³ | 0.33 Gt / 1.8 km³ = 0.183; 6,100 Gt / 33,000 km³ = 0.185 Gt/km³ |
| Ratio of the 6.5 M capacity to the comparator | same, para 2; Abstract | "two orders of magnitude higher" (15 / 0.18 = 83×) | 0.016 / 0.18 = 0.09, about 11× lower |
Each later number follows arithmetically from the 15 Gt/km³ figure: 1.3 km³ × 15 Gt/km³ = 19.5 Gt ("up to 20 Gt"), and 1.3 km³ is 1/25,000 of the 33,000 km³ rift zone. The error therefore enters once, at the step that converts grams of CO2 to gigatonnes, and is carried through the rest of the paragraph.
Figure
Figure 1. The top row gives the paper's stated inputs, multiplied in sequence for 1 km³ of rock: 5.7% effective porosity gives 5.7 × 10⁷ m³ of pore fluid, 0.65 M (650 mol/m³) gives 3.7 × 10¹⁰ mol of CO2, and 44.01 g/mol gives 1.6 × 10¹² g, or 1.63 Mt. The dashed box marks the conversion from grams to gigatonnes, where the reported values become about 920 times larger than the product of the inputs. The table below gives each downstream result twice: recomputed from the paper's inputs (blue marker, middle column) and as reported in the paper (orange marker, right column). All recomputed numbers come from recompute.py.txt.
Figure 2. Storage capacity in Gt CO2 per km³ of rock, on a log scale. For each scenario, the orange dot is the value reported in the paper and the blue dot is the value recomputed from the paper's own porosity and concentration; the labelled factor is the ratio of the two. The dotted lines are two ceilings, also computed in recompute.py.txt. Ceiling A (0.040 Gt/km³) is pure dense CO2 at an assumed 700 kg/m³ filling all of the 5.7% effective porosity. Ceiling B (0.29 Gt/km³) is the 6.5 M fluid filling 100% of the rock volume. Both reported values lie above both ceilings. The grey diamond is the 0.18 Gt/km³ comparator the paper cites, a mineral-carbonation estimate that the cited study derives as 0.33 Gt for a 1.8 km³ formation. The recomputed 6.5 M value lies below it, not two orders of magnitude above it.
Why it matters
The two capacities are the paper's only quantitative estimate of storage at reservoir scale, and they support the paper's scale-up claims: that 20 Gt of CO2 per year (the 2100 removal target the Introduction cites) could be stored in 1.3 km³, a small fraction of Iceland's rift zone, and, in the abstract, that the approach offers capacities "two orders higher than current techniques" and "reduces required reservoir volume". With the paper's own inputs, the reservoir volume needed for 20 Gt is about 1,230 km³ rather than 1.3 km³, roughly 1/27 of the rift zone rather than 1/25,000. The comparison with the cited value is not two orders of magnitude higher: in the same units it is about 11 times lower.
The abstract gives no numbers and does not say which comparison its "two orders higher" rests on. The only quantitative comparison in the body is the one recomputed here, so with the paper's inputs the abstract claim is not supported by the body.
The experimental results do not depend on this calculation. These are the measured changes in basalt cores after acidic GLDA flooding (16% gain in effective porosity, 26-fold gain in permeability), the CO2 absorption of GLDA seawater solutions, and the batch and flooding chemistry. The stoichiometric step from 0.65 M to 6.5 M after GLDA degradation is arithmetically consistent and is not in question here.
Explanations considered
A different meaning of "M". The paper uses M as mol/L, and 1 mol/L equals 1 kmol/m³, which is what the recomputation uses. The value 0.65 M for 20 wt % GLDA-Na4 (molar mass 351.1 g/mol, so 0.570 mol per kg of solution) implies a solution density of 1.14 kg/L, which is plausible. No reading of M makes the concentration 1,000 times larger.
A different porosity. To give 1.5 Gt/km³ at 0.65 M, or 15 Gt/km³ at 6.5 M, the pore fluid would have to be 52 times the volume of the rock (an effective porosity of about 5,200%). The paper's stated values are 4.8% and 5.7%. In its own experiments, the flooding core's effective porosity rose from 12.4 to 14.4%, and its basalt samples' total porosity is 14–20%. The cited study gives 10.99–15.23% total. None of these comes close. Even if the whole rock volume were 6.5 M fluid, the capacity would be 0.29 Gt/km³.
A megatonne/gigatonne slip. The correct results are 1.63 and 16.3 Mt/km³. Reporting them as 1.5 and 15 Gt/km³ fits a factor-of-1,000 unit slip combined with rounding or slightly different inputs (the remaining factor is 0.92). This is the most economical arithmetic explanation of how the numbers arose. It does not make the Gt figures correct, and every downstream statement in the paragraph inherits the factor.
A different basis (per km³ of pore space, or including mineralized CO2). The text says the calculation fills "all effective porosity" with the solution at the stated concentration. Even 1 km³ of pure pore fluid at 6.5 M holds only 0.29 Gt, not 15. Mineralization converts CO2 already in the fluid into carbonate and adds no CO2. Filling all 5.7% of the pore space with dense CO2 is capped at about 0.04 Gt/km³. The reported 15 Gt/km³ means 15,000 kg of CO2 per m³ of rock, about five times (5.1×) the density of basalt itself (taking a typical 2,950 kg/m³, an assumed value).
"Annually" meaning repeated injection. The paper gives no injection rate, pore-fluid turnover or number of cycles per year. The 20 Gt figure equals the static capacity times the volume (1.3 km³ × 15 Gt/km³ = 19.5 Gt), so "annually" adds no factor to the calculation.
The comparator is in Mt as well. If the cited 0.18 Gt/km³ were also inflated by a factor of 1,000, the paper's ratio would survive. The cited study's own tables rule this out. Its Table 7 gives a mean CO2 sequestration potential of 0.33 Gt for a CarbFix formation of 1.8 km³ (0.4 km × 2.5 km × 1.8 km), which is 0.18 Gt/km³. Its Table 8 gives 6,100 Gt for the 33,000 km³ rift zone, which is 0.185 Gt/km³. The paper takes its 33,000 km² × 1 km rift-zone dimensions from the same table. The value is computed from reactive surface area and a mineral-carbonation fixation rate per unit area, not from CO2 dissolved in pore fluid. It is in gigatonnes in the source, and it rests on a different basis from the paper's fluid-filling calculation. With consistent units, the paper's 0.016 Gt/km³ is about 11 times below it.
Conversion or archiving artifact. Ruled out. The publisher-typeset PDF (Science Advances, deposited in the PMC Open Access collection) carries the same numbers and units as the PMC text and the Europe PMC XML, with km³ correctly marked up. The quantities are not affected by formatting.
A derivation in the supplement. The only supplementary file (Tables S1–S2, Figs. S1–S11) contains rock compositions, the seawater recipe and experimental figures. It contains no capacity calculation, no Gt or km³ values, and no alternative units.
Status
As of 2026-10-23:
- Version of record. science.org, reached through doi.org, returned HTTP 403 to automated fetches. An earlier audit stage therefore checked the publisher-typeset PDF from the PMC Open Access bucket (
s3://pmc-oa-opendata/PMC11566992.1/, a Science Advances InDesign PDF created 2024-11-14). Its Results subsection "CO2 mineralization with GLDA biodegradation and overall evaluation" carries every quoted number unchanged (4.8 to 5.7%; 0.65 M, 1.5 Gt/km³; 6.5 M, 15 Gt/km³; 0.18 Gt/km³; 20 Gt, 1.3 km³, 1/25,000). The abstract sentence is also unchanged. The quotes above were re-checked verbatim against the PMC archive text on 2026-10-23. - Supplements.
sciadv.adq0515_sm.pdf(13 pages) was read in full at an earlier audit stage. It has no capacity derivation. No source-data files are deposited. - Cited comparator. The cited study's accepted manuscript (OSTI 1538337), Tables 7 and 8, was read on 2026-10-23 to establish the basis and units of 0.18 Gt/km³.
- Corrections.
src/known.py(run 2026-10-23) lists no Crossref updates and no PMC or Europe PMC correction or retraction notices. A Crossreffilter=updates:query for the DOI returns 0 results. The PMC record lists no related notices. Crossref Labs (Retraction Watch data) and a Europe PMC search for the DOI showed no correction at an earlier audit stage. - PubPeer. The PubPeer API returned HTTP 403 on 2026-10-23, so PubPeer could not be checked directly. A web search on 2026-10-23 for the DOI or title together with PubPeer, correction and erratum found no relevant thread or notice.
Sources: PMC11566992 (PubMed Central archive, read with src/show.py); https://doi.org/10.1126/sciadv.adq0515; publisher PDF and supplement from s3://pmc-oa-opendata/PMC11566992.1/; cited study, doi:10.1016/j.ijggc.2017.12.008 (accepted manuscript https://www.osti.gov/servlets/purl/1538337); https://api.crossref.org/works?filter=updates:10.1126/sciadv.adq0515.
Suggested correction
In the Results paragraph:
- Replace "a CO2 storage capacity of 1.5 Gt/km3" with "a CO2 storage capacity of about 1.6 Mt/km3 (0.0016 Gt/km3)".
- Replace "15 Gt of CO2/km3" with "about 16 Mt of CO2/km3 (0.016 Gt/km3)".
- Rewrite the comparison with the cited value of 0.18 Gt/km3. In the same units the corrected figure is about one-eleventh of it, not two orders of magnitude higher, and the cited value is a mineral-carbonation estimate rather than a dissolved-CO2 capacity.
- Restate the 20 Gt sentence. Under the stated assumptions, 1.3 km³ holds about 21 Mt of CO2, and 20 Gt would need about 1,230 km³, roughly 1/27 of the 33,000 km³ rift zone. If "annually" is meant to reflect repeated injection, state the injection rate or turnover that supports it.
In the abstract, revise "capacities two orders higher than current techniques" and "reduces required reservoir volume" to match the corrected calculation, or name the comparison and basis that support them.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC11566992 and regenerate fig1/fig2.
Run: python recompute.py.txt (writes fig1.svg/png and fig2.svg/png next to this file)
Inputs are taken from the article as archived in PubMed Central (PMC11566992),
Results subsection "CO2 mineralization with GLDA biodegradation and overall
evaluation", paragraphs 1-2:
- effective porosity after GLDA treatment: 5.7 % (from 4.8 %)
- CO2 concentration of the injected fluid: 0.65 M; 6.5 M after complete GLDA degradation
- reported capacities: 1.5 and 15 Gt CO2 per km3; up to 20 Gt in 1.3 km3
- active rift zone in Iceland: 33,000 km2 x 1 km; comparator 0.18 Gt CO2/km3
The comparator's own basis is taken from the cited study (ref. 17 of the paper,
Int. J. Greenh. Gas Control 70:146-156 (2018), doi:10.1016/j.ijggc.2017.12.008, accepted manuscript, Tables 7 and 8).
Three constants are not from either paper: the molar mass of CO2 (used in the
capacity calculation); for one physical ceiling, an assumed dense-CO2 density of
700 kg/m3; and, for one comparison only, a typical basalt density of 2,950 kg/m3
(both assumed values are labelled as such).
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (paper)
PHI0, PHI = 0.048, 0.057 # effective porosity before / after GLDA treatment
C1, C2 = 0.65, 6.5 # mol/L CO2 (GLDA-loaded; after full degradation)
REP1, REP2 = 1.5, 15.0 # reported Gt CO2 per km3
REP_ANNUAL, REP_VOL = 20.0, 1.3 # "Up to 20 Gt ... in a 1.3 km3 reservoir volume"
RIFT_KM3 = 33000 * 1 # 33,000 km2 x 1 km
RIFT_FRACTION_REP = 25000 # "1/25,000 of the active rift zone"
CITED = 0.18 # Gt CO2 per km3, as quoted from the cited study
# Cited study (ref. 17), Table 7 (mean row) and Table 8, as printed in that study
CITED_SITE_GT, CITED_SITE_KM3 = 0.33, 1.8 # CarbFix site: 0.4 km x 2.5 km x 1.8 km
CITED_RIFT_GT = 6100 # active rift zone, 33,000 km3
# ---------------------------------------------------------------- constants
M_CO2 = 44.01 # g/mol
KM3 = 1e9 # m3
G_PER_GT = 1e15 # g per Gt
RHO_DENSE_CO2 = 700 # kg/m3, assumed (generous) dense/supercritical CO2 density
RHO_BASALT = 2950 # kg/m3, assumed typical basalt grain/bulk density (~2,900-3,000)
def gt_per_km3(conc_M, phi):
"""CO2 (Gt) held in 1 km3 of rock whose effective pores are filled with fluid at conc_M mol/L."""
mol = conc_M * 1000 * phi * KM3 # mol/L -> mol/m3, times pore volume
return mol * M_CO2 / G_PER_GT
pore_m3 = PHI * KM3
mol1 = C1 * 1000 * pore_m3
g1 = mol1 * M_CO2
cap1, cap2 = gt_per_km3(C1, PHI), gt_per_km3(C2, PHI)
ratio1, ratio2 = REP1 / cap1, REP2 / cap2
print("== Capacity per km3 of rock, paper's own inputs ==")
print(f"pore fluid in 1 km3 at {PHI:.1%} effective porosity: {pore_m3:.3g} m3")
print(f"0.65 M: {mol1:.4g} mol CO2 = {g1:.4g} g = {g1/1e12:.3f} Mt = {cap1:.5f} Gt/km3; "
f"reported {REP1} Gt/km3; reported/recomputed = {ratio1:.0f}x")
print(f"6.5 M: {cap2*1e3:.2f} Mt = {cap2:.4f} Gt/km3; reported {REP2} Gt/km3; "
f"reported/recomputed = {ratio2:.0f}x")
print(f"10x step from 0.65 M to 6.5 M (0.65 + 0.65 x 9): {0.65 + 0.65*9:.2f} M (paper's arithmetic is consistent)")
print()
print("== What the reported values would require ==")
for c, rep in [(C1, REP1), (C2, REP2)]:
phi_needed = rep * G_PER_GT / (c * 1000 * M_CO2 * KM3)
print(f"{rep} Gt/km3 at {c} M needs effective porosity {phi_needed:.1%}; "
f"implied CO2 mass per m3 of rock = {rep*1e12/KM3:,.0f} kg/m3")
print(f"15 Gt/km3 = {REP2*1e12/KM3:,.0f} kg CO2 per m3 of rock = "
f"{REP2*1e12/KM3/RHO_BASALT:.1f}x a basalt density of {RHO_BASALT:,} kg/m3 (assumed)")
print()
print("== Physical ceilings (per km3 of rock) ==")
ceil_100 = gt_per_km3(C2, 1.0)
ceil_dense = PHI * KM3 * RHO_DENSE_CO2 / 1e12
print(f"6.5 M fluid filling 100% of the rock volume: {ceil_100:.3f} Gt/km3")
print(f"pure dense CO2 ({RHO_DENSE_CO2} kg/m3, assumed) filling all 5.7% pores: {ceil_dense:.4f} Gt/km3")
print()
print("== 20 Gt in 1.3 km3 ==")
print(f"reported chain: 20 / 15 = {REP_ANNUAL/REP2:.2f} km3; 33,000 / 25,000 = {RIFT_KM3/RIFT_FRACTION_REP:.2f} km3; "
f"1.3 x 15 = {REP_VOL*REP2:.1f} Gt")
held_13 = REP_VOL * cap2
vol_20 = REP_ANNUAL / cap2
print(f"CO2 held in 1.3 km3 at 6.5 M (recomputed): {held_13:.4f} Gt = {held_13*1e3:.1f} Mt")
print(f"volume needed for 20 Gt at the recomputed 6.5 M capacity: {vol_20:,.0f} km3 "
f"= 1/{RIFT_KM3/vol_20:.0f} of the {RIFT_KM3:,} km3 rift zone")
print()
print("== Comparison with the cited value (0.18 Gt/km3) ==")
print(f"cited study, CarbFix site: {CITED_SITE_GT} Gt / {CITED_SITE_KM3} km3 = {CITED_SITE_GT/CITED_SITE_KM3:.3f} Gt/km3")
print(f"cited study, rift zone: {CITED_RIFT_GT} Gt / {RIFT_KM3:,} km3 = {CITED_RIFT_GT/RIFT_KM3:.3f} Gt/km3")
print(f"reported 15 / 0.18 = {REP2/CITED:.0f}x ('two orders of magnitude higher')")
print(f"recomputed {cap2:.4f} / 0.18 = {cap2/CITED:.3f} (about {CITED/cap2:.0f}x lower)")
print()
print("== Side check: 0.65 M for 20 wt % GLDA-Na4 ==")
MW_GLDA_NA4 = 9*12.011 + 9*1.008 + 14.007 + 4*22.990 + 8*15.999 # C9H9NNa4O8
molal = 200 / MW_GLDA_NA4
print(f"MW C9H9NNa4O8 = {MW_GLDA_NA4:.1f} g/mol; 20 wt % = {molal:.3f} mol/kg solution; "
f"0.65 M implies a solution density of {C1/molal:.2f} kg/L (plausible; M is mol/L)")
# ---------------------------------------------------------------- figure style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, HL = "#eb6834", "#2a78d6", "#e4e3df", "#fbe9e1"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK, "svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def fmt_sci(x):
e = int(f"{x:e}".split("e")[1])
m = x / 10**e
return f"{m:.2g}×10$^{{{e}}}$"
# ---------------------------------------------------------------- fig1: arithmetic chain
W, H = 7.2, 6.65
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, W); ax.set_ylim(0, H); ax.axis("off")
def box(x, y, w, h, text, ec=INK2, fc="white", color=INK, weight="normal", lw=1.5, ls="-", size=10):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.06",
fc=fc, ec=ec, lw=lw, ls=ls))
ax.text(x + w/2, y + h/2, text, ha="center", va="center", fontsize=size, color=color,
weight=weight, linespacing=1.35)
def arrow(x0, y0, x1, y1, color=INK2):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=12,
color=color, lw=1.5, shrinkA=0, shrinkB=0))
ax.text(0.15, H - 0.18, "Capacity of 1 km³ of rock, using only the paper's inputs",
ha="left", va="top", fontsize=11, weight="bold")
ax.text(0.15, H - 0.45, "Shared steps (the paper's stated inputs)", ha="left", va="top",
fontsize=10, color=INK2)
bw, bh, gap, y0 = 1.55, 0.95, 0.22, H - 1.65
xs = [0.15 + i*(bw + gap) for i in range(4)]
steps = ["Rock volume\n1 km³ = 10$^{9}$ m³",
f"× 5.7% effective\nporosity\n= {fmt_sci(pore_m3)} m³ fluid",
f"× 0.65 M\n(650 mol/m³)\n= {fmt_sci(mol1)} mol",
f"× 44.01 g/mol\n= {fmt_sci(g1)} g CO$_2$\n= {g1/1e12:.2f} Mt"]
for x, s in zip(xs, steps):
box(x, y0, bw, bh, s)
for i in range(3):
arrow(xs[i] + bw, y0 + bh/2, xs[i+1], y0 + bh/2)
# divergence step
dx, dy, dw, dh = xs[2] - 0.9, y0 - 1.05, W - 0.15 - (xs[2] - 0.9), 0.62
ax.add_patch(FancyBboxPatch((dx, dy), dw, dh, boxstyle="round,pad=0.02,rounding_size=0.06",
fc=HL, ec=ORANGE, lw=2, ls="--"))
ax.text(dx + dw/2, dy + dh/2 + 0.11, "Unit step: grams → gigatonnes (÷10$^{15}$)",
ha="center", va="center", fontsize=10, weight="bold")
ax.text(dx + dw/2, dy + dh/2 - 0.15, f"the reported values diverge here by ≈{ratio1:.0f}×",
ha="center", va="center", fontsize=10, color=INK2)
arrow(xs[3] + bw/2, y0, xs[3] + bw/2, dy + dh)
# outcome table
ty = dy - 0.42
cx_lab, cx_b, cx_o = 0.15, 3.75, 5.95
ax.text(cx_lab, ty, "Result", ha="left", va="center", fontsize=10, color=INK2, weight="bold")
ax.plot([cx_b - 1.05], [ty], "o", ms=8, color=BLUE)
ax.plot([cx_o - 0.85], [ty], "o", ms=8, color=ORANGE)
ax.text(cx_b, ty, "Recomputed from\nthe paper's inputs", ha="center", va="center", fontsize=10,
color=BLUE, weight="bold")
ax.text(cx_o, ty, "Reported in\nthe paper", ha="center", va="center", fontsize=10,
color=ORANGE, weight="bold")
rows = [("Capacity per km³, 0.65 M", f"{cap1:.4f} Gt ({cap1*1e3:.1f} Mt)", "1.5 Gt"),
("Capacity per km³, 6.5 M\n(after GLDA degradation, ×10)", f"{cap2:.3f} Gt ({cap2*1e3:.0f} Mt)", "15 Gt"),
("CO$_2$ held in 1.3 km³ at 6.5 M", f"{held_13:.3f} Gt ({held_13*1e3:.0f} Mt)", "up to 20 Gt"),
("Rock volume needed for 20 Gt", f"{vol_20:,.0f} km³\n(1/{RIFT_KM3/vol_20:.0f} of the rift zone)",
"1.3 km³\n(1/25,000 of the rift zone)"),
("Ratio to the cited\n0.18 Gt/km³ (ref. 17)", f"{cap2/CITED:.2f}×\n(about {CITED/cap2:.0f}× lower)",
"“two orders of\nmagnitude higher”")]
rh = 0.62
for i, (lab, b, o) in enumerate(rows):
yy = ty - 0.55 - i*rh
ax.plot([0.15, W - 0.15], [yy + rh/2, yy + rh/2], color=GRID, lw=1)
ax.text(cx_lab, yy, lab, ha="left", va="center", fontsize=10, linespacing=1.2)
ax.text(cx_b, yy, b, ha="center", va="center", fontsize=10, color=INK, linespacing=1.2)
ax.text(cx_o, yy, o, ha="center", va="center", fontsize=10, color=INK, linespacing=1.2)
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: log-scale comparison
fig, ax = plt.subplots(figsize=(7.2, 3.7))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.17)
rows2 = [("0.65 M fluid in\n5.7% porosity", REP1, cap1),
("6.5 M fluid in 5.7% porosity\n(after GLDA degradation)", REP2, cap2)]
yp = [2, 1]
for y, (lab, rep, rec) in zip(yp, rows2):
ax.plot([rec, rep], [y, y], color=INK2, lw=1, zorder=1)
ax.plot(rep, y, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(rec, y, "o", ms=9, color=BLUE, zorder=3)
gm = 0.11 if y == 2 else 1.6
ax.text(gm, y + 0.13, f"×{rep/rec:.0f}", ha="center", va="bottom", fontsize=10, color=INK2)
ax.text(rep * 1.25, y, f"{rep:g}", ha="left", va="center", fontsize=10, color=INK)
ax.text(rec / 1.25, y, f"{rec:.4f}" if rec < 0.01 else f"{rec:.3f}", ha="right", va="center",
fontsize=10, color=INK)
ax.plot(CITED, 0, "D", ms=8, color=INK2, zorder=3)
ax.text(CITED * 1.3, 0, "0.18 (cited mineral-carbonation\nestimate: 0.33 Gt / 1.8 km³)",
ha="left", va="center", fontsize=10, color=INK)
ax.set_yticks([2, 1, 0])
ax.set_yticklabels([r[0] for r in rows2] + ["Comparator cited in\nthe paper (ref. 17)"])
ax.set_ylim(-0.6, 2.6)
ax.set_xscale("log")
ax.set_xlim(2e-4, 2e2)
for xv, lab, ha in [(ceil_dense, f"A: {ceil_dense:.3f} ", "right"), (ceil_100, f" B: {ceil_100:.2f}", "left")]:
ax.axvline(xv, color=INK2, lw=1, ls=":", zorder=0)
ax.text(xv, 2.62, lab, ha=ha, va="bottom", fontsize=10, color=INK2)
ax.set_xlabel("Gt CO$_2$ per km³ of rock (log scale)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8, which="major")
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0, labelsize=10)
ax.tick_params(axis="x", labelsize=10, colors=INK2)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported in the paper"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed from the paper's inputs"),
Line2D([], [], ls=":", color=INK2, lw=1, label="Ceilings (see caption)")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=3, frameon=False,
fontsize=10, handletextpad=0.4, columnspacing=1.2)
save(fig, "fig2")
print("\nfigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Results report two incompatible sets of Fig. 3A means, including PD putamen 0.243 versus 0.143
Sci Adv (2025) · doi:10.1126/sciadv.ads3051 · PMC12143374
Summary
The first Results subsection reports the mitochondrial DJ-1–β-subunit PLA signal (Manders' coefficient, mean ± SEM) for six nigrostriatal cells: control and PD, each in SNpc somata, SNpc proximal neurites and putamen (Put) distal neurites. It reports them twice. Both passages cite Fig. 3A, and the two sets disagree in every cell. For PD putamen, the cell behind the paper's emphasis on a reduction "particularly in distal neurites", the values are 0.243 ± 0.030 (n = 18) and 0.143 ± 0.034 (n = 17). Fig. 3A itself plots 18 PD putamen points with a mean of 0.241 ± 0.030, and its six mean bars match the first set to within 0.004. The second set, which is the one the text uses to describe the within-pathway pattern and a "significantly higher" proximal-versus-somata signal in PD, does not match the plotted data. A third paragraph, introduced as comparing SNpc and VTA and citing Fig. 3B, repeats the first set's SNpc/Put numbers verbatim, and its stated ordering does not match them.
What the paper reports
All passages are from the Results subsection "DJ-1-β-sub association is reduced in SNpc DA neurons of patients with PD". Paragraphs are counted from the start of that subsection, so ¶1 begins "Loss of function mutations", ¶2 "To examine the association" and ¶3 "We found that".
Paragraph 3 (begins "We found that"):
The DA axons in the Put also showed decreased DJ-1-β-sub association in brains of patients with PD compared to control cases (PD: somata M = 0.357 ± 0.025 R, n = 20; proximal neurites M = 0.408 ± 0.028 R, n = 20; Put M = 0.243 ± 0.030 R, n = 18; control: somata M = 0.420 ± 0.032 R, n = 20; proximal neurites M = 0.538 ± 0.028 R, n = 20; Put M = 0.377 ± 0.032 R, n = 19) (Fig. 3A).
Paragraph 4 (begins "The intracellular heterogeneity"):
To determine whether the subcellular distribution of DJ-1-β-sub association levels within SNpc and VTA neurons aligns with or deviates from their expected energetic demand and expenditure, we compared the DJ-1-β-sub PLA signal levels within the SNpc and VTA DA neurons across the three subcellular regions. We observed the highest PLA signal within the mitochondria of the proximal neurites (M = 0.538 ± 0.028, n = 20), followed by distal neurites (M = 0.377 ± 0.032, n = 19) and somata (M = 0.420 ± 0.032, n = 20). A similar trend appeared in PD cases, where proximal neurites showed the highest signal (M = 0.408 ± 0.028, n = 20), followed by distal neurites (M = 0.243 ± 0.030, n = 18) and somata (M = 0.357 ± 0.025, n = 20), suggesting a correlation with subcellular metabolic demand (Fig. 3B).
Paragraph 5 (begins "In control SNpc neurons"):
In control SNpc neurons, the PLA signal does not follow the same trend observed in VTA neurons. While the PLA signal is notably higher in the proximal neurites (M = 0.490 ± 0.040 R, n = 20) compared to the somata (M = 0.351 ± 0.038 R, n = 20), it declines in the distal axons in the Put (M = 0.332 ± 0.047 R, n = 19) (Fig. 3A). In PD cases, this pattern is retained, with proximal neurites (M = 0.404 ± 0.046 R, n = 20) showing significantly higher PLA signal than the somata (M = 0.299 ± 0.032 R, n = 20), while the distal axons in the Put exhibit the lowest levels of DJ-1-β-sub association (M = 0.143 ± 0.034 R, n = 17).
The trends observed in both control and PD cases indicate a mismatch in metabolic compensation along the nigrostriatal pathway, potentially contributing to the selective vulnerability of SNpc neurons in PD.
Fig. 3 legend:
(C and D) The data in (A) and (B) are rearranged to allow for the comparison of PLA signal within the subcellular compartments (somata, proximal neurites, and distal neurites) of mesolimbic and nigrostriatal pathways in control (C) and PD (D) cases. Each data point represents the mean of three confocal images per section (one section from each region per case).
To maintain consistency across striatal and midbrain sections, the 40 sections were processed in two independent batches of 20 sections, the maximum feasible per experimental run. Comparable trends and statistical significance were observed across both datasets. The figure presents combined data from all sections. Statistical analysis: two-way ANOVA with Tukey’s post hoc test.
Methods, "Statistical analyses":
The data were analyzed using GraphPad Prism and are presented as the means ± SEM.
Where the finding is stated in summary form. Abstract:
In PD brains, DJ-1-β-sub-PLA was lower than control in substantia nigra pars compacta (SNpc) somata and neurites but unchanged in ventral tegmental area (VTA) neurons.
Introduction, last paragraph:
Here, we show a reduction in DJ-1-β-sub association in the SNpc of brains of patients with PD, particularly in distal neurites in the dorsal striatum, while in the VTA, this association is unchanged between control and PD brains.
Discussion, first paragraph:
Here, we demonstrate that DJ-1-β-sub association is selectively reduced in the subcellular compartments of SNpc of brains of patients with PD, particularly in distal neurites in the Put, while remaining unchanged in the VTA. In control brains, VTA neurons exhibit a gradient of DJ-1-β-sub association that increases from somata to proximal and distal neurites, whereas SNpc neurons lack this gradient.
What the paper's own numbers show
Method. I tabulated the six SNpc/Put summaries (mean, SEM, n) from paragraphs 3, 4 and 5 and compared them cell by cell. I converted each SEM to the SD it implies (SEM × √n) and tested whether the n = 17 summary could be the n = 18 summary with one case dropped, using the fact that a Manders' coefficient lies between 0 and 1. I then read Fig. 3A from the published image (sciadv.ads3051-f3.jpg, Europe PMC supplementary bundle). I calibrated the y-axis on the 1.0, 0.5 and 0.0 tick labels (the 0.5 tick reads back as 0.498) and located each horizontal mean bar. In the PD putamen column all points are separable, so I also located each point and recomputed its mean and SEM. In the other five columns some points are hidden behind the mean and SEM bars, so only their spread is used, and only as a secondary check. Every number below is printed by recompute.py.txt, which fetches the figure image itself.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| PD, Put distal neurites | ¶3 and ¶4 vs ¶5 | 0.243 ± 0.030, n = 18 vs 0.143 ± 0.034, n = 17 | difference 0.100 (z = 2.21 if the two were independent estimates); Fig. 3A mean bar 0.244; 18 plotted points, mean 0.241, SD 0.128, SEM 0.030 |
| PD, SNpc somata | ¶3 vs ¶5 | 0.357 ± 0.025 vs 0.299 ± 0.032 (both n = 20) | Fig. 3A bar 0.354 |
| PD, SNpc proximal | ¶3 vs ¶5 | 0.408 ± 0.028 vs 0.404 ± 0.046 (both n = 20) | Fig. 3A bar 0.409 (means near-equal; SEMs differ, implied SD 0.125 vs 0.206) |
| Control, SNpc somata | ¶3 vs ¶5 | 0.420 ± 0.032 vs 0.351 ± 0.038 (both n = 20) | Fig. 3A bar 0.416 |
| Control, SNpc proximal | ¶3 vs ¶5 | 0.538 ± 0.028 vs 0.490 ± 0.040 (both n = 20) | Fig. 3A bar 0.540 |
| Control, Put distal | ¶3 vs ¶5 | 0.377 ± 0.032 vs 0.332 ± 0.047 (both n = 19) | Fig. 3A bar 0.375 |
| Pattern of the ¶5 discrepancies | ¶3 vs ¶5 | — | ¶5 mean lower in all 6 cells, by 0.004 to 0.100 (ratios 0.588 to 0.990: no common offset or factor); ¶5 SEM larger in all 6 cells |
| One case dropped from the n = 18 set? | ¶3 vs ¶5 | 0.243 (n = 18) → 0.143 (n = 17) | sum of 18 values 4.374; the dropped value would have to be 1.943 (1.926 to 1.961 allowing for rounding); dropping any value in [0, 1] leaves a mean of 0.199 to 0.257; dropping the largest plotted point (0.436) leaves 0.229 |
| Spread of plotted points | Fig. 3A vs ¶3 and ¶5 | implied SDs 0.112 to 0.143 (¶3) and 0.140 to 0.206 (¶5) | SD of the visible points in each column 0.121 to 0.154; closer to the ¶3 SD in all 6 cells |
| ¶4 values | ¶4 vs ¶3 | ¶4 cites Fig. 3B and is introduced as comparing SNpc and VTA | all six ¶4 values are identical to the ¶3 SNpc/Put values |
| ¶4 stated ordering | ¶4 | "highest ... proximal neurites ..., followed by distal neurites ... and somata" | by ¶4's own numbers the order is proximal > somata > distal in both groups (control 0.538 > 0.420 > 0.377; PD 0.408 > 0.357 > 0.243) |
| VTA/NAc values | ¶3 vs Fig. 3B | control 0.401, 0.588, 0.620; PD 0.489, 0.628, 0.592 | Fig. 3B bars 0.402, 0.588, 0.615; 0.485, 0.629, 0.588 (agree) |
| PD proximal vs somata, "significantly higher" | ¶5; Fig. 3A asterisk | significant | context only: unpaired Welch t = 1.36, p = 0.18 with the ¶3 values; t = 1.87, p = 0.070 with the ¶5 values. The two compartments come from the same cases and the paper used two-way ANOVA with Tukey's test, so summary statistics cannot check the asterisk |
Three things follow. First, paragraphs 3 and 5 cannot both describe the Fig. 3A data: they give different means for the same group and compartment in all six cells, and different SEMs at the same n. Second, the n = 17 summary is not a subset of the n = 18 summary, because the case dropped would need a Manders' coefficient of about 1.94. Third, the published figure agrees with paragraph 3. Its six mean bars lie within 0.004 of the paragraph 3 means and 0.043 to 0.101 from the paragraph 5 means. The one exception is PD proximal, where the two text means differ by only 0.004. The PD putamen column has 18 points, not 17, and their mean and SEM (0.241 ± 0.030) reproduce 0.243 ± 0.030.
Figure
Figure 1. The six nigrostriatal cells of Fig. 3A (Manders' coefficient, mean ± SEM, on a full 0 to 0.65 axis). Filled orange circles are the values reported in Results paragraph 3, which paragraph 4 repeats verbatim. Open orange squares are the values reported in paragraph 5. Both paragraphs cite Fig. 3A. Blue diamonds are the mean bars read from the published Fig. 3A image by recompute.py.txt. For PD putamen, the blue whisker is the SEM of the 18 plotted points, recomputed from their positions. The blue values sit on the paragraph 3 values in every cell. Paragraph 5 is lower in every cell, and for PD putamen it gives 0.143 (n = 17) where the figure shows 18 points averaging 0.241.
Figure 2. Short excerpts of the PD values in the three Results passages, with the figure each one cites. Paragraph 4 is introduced as a comparison of SNpc and VTA neurons and cites Fig. 3B, yet reproduces the six SNpc/Put values of paragraph 3. Paragraph 5 cites Fig. 3A, like paragraph 3, and gives different values. The blue box is what the Fig. 3A image shows for PD putamen. The orange box gives the size of the conflict and the value one dropped case would need to have.
Why it matters
- The headline compartment carries the largest discrepancy. The Introduction and Discussion single out a PD-related reduction "particularly in distal neurites in the Put". The PD-minus-control difference in putamen is 0.134 (0.243 vs 0.377) by paragraph 3 and 0.189 (0.143 vs 0.332) by paragraph 5, which is 41% larger. PD putamen is 64% of control by paragraph 3 and 43% by paragraph 5. A reader using paragraph 5 overstates the reduction relative to the plotted data.
- The within-pathway pattern is described with numbers that do not match the figure. Paragraph 5 supports the "mismatch in metabolic compensation along the nigrostriatal pathway" and the PD proximal-versus-somata difference it calls significant. In paragraph 5 that difference is 0.105; in paragraph 3 and in the figure it is about 0.05. The ordering (proximal > somata > distal in both groups) is the same in both sets, so the qualitative pattern stands. The magnitudes in paragraph 5 do not.
- The VTA-gradient paragraph has no VTA numbers. Paragraph 4 is introduced as comparing SNpc and VTA neurons and cites Fig. 3B, but it reports only SNpc/Put values, in an order its own numbers contradict. The VTA gradient claimed in the Introduction and Discussion (somata < proximal < distal in controls) is consistent with the control VTA/NAc values in paragraph 3 (0.401, 0.588, 0.620) and with Fig. 3B. So the claim has support, but not from the paragraph written to present it.
- The direction of the main findings does not change. Under either set, PD is lower than control in all three SNpc/Put compartments, and putamen is the lowest compartment in PD. The abstract's qualitative statements hold. The error concerns which descriptive statistics belong to the plotted data, and how large the reported effects are.
Explanations considered
- Rounding or transcription of a single value. Does not account for it. All six cells differ, the SEMs differ at identical n (for example control proximal 0.028 vs 0.040, both n = 20), and the PD putamen means differ by 0.100.
- One case excluded from the PD putamen set (n = 18 → 17). Ruled out. Reaching 0.143 from 0.243 by dropping one case would require that case to have a Manders' coefficient of 1.943, outside the 0 to 1 range. It also would not explain the changes in the five other cells, four of which keep the same n.
- A different batch or subset. Not supported by the stated n. The legend says the sections were run in two batches and "The figure presents combined data from all sections". Paragraph 5 gives n = 20, 20 and 19 for control and 20, 20 and 17 for PD, essentially the full cohort, so it is not a single batch of about half the cases.
- A different metric or normalisation, such as an earlier threshold setting. Cannot be checked from the paper and does not match the figure. The paragraph 5 means are lower in every cell, but not by a common offset (0.004 to 0.100) or a common factor (0.59 to 0.99). Both paragraphs use the same unit label ("R") and cite the same panel. The paper describes only one PLA quantification. Whatever its origin, the paragraph 5 set is not the data plotted in Fig. 3A.
- Paragraph 3, not paragraph 5, is the one in error. Not supported. The Fig. 3A mean bars match paragraph 3 within 0.004 in all six cells. The PD putamen column has 18 separable points with mean 0.241 and SEM 0.030. The visible-point spread in every column is closer to the paragraph 3 SDs.
- Paragraph 4 describing VTA with the right numbers. Ruled out. Its six values are the paragraph 3 SNpc/Put values, not the paragraph 3 VTA/NAc values (control 0.401, 0.588, 0.620; PD 0.489, 0.628, 0.592), which Fig. 3B also shows.
- A conversion or archiving artifact. Ruled out. The same paragraph 3 and paragraph 5 values appear in the live PMC page, in the Europe PMC full-text XML (the publisher's JATS) and in the local PMC archive.
- Side observation, not part of the main finding. The Results give the control cohort as "11 males and 9 females"; Table 1 lists 9 males and 11 females, as do the Methods. The Methods give mean ages of 78.6 ± 6.8 (PD) and 83.5 ± 5.7 (control). Table 1 gives 78.95 (SD 7.22) and 83.20 (SD 8.39), which agree, to rounding, with the Results values of 78.95 ± 1.61 and 83.20 ± 1.87 (SEM; Table 1 implies 1.61 and 1.88). These do not affect the PLA results.
Status
- Version of record. The publisher's page (science.org, reached through doi.org) returned HTTP 403 to automated requests. Paragraphs 3, 4 and 5 read the same in three other copies of the published text: the live PMC page, the Europe PMC full-text XML (the publisher's JATS) and the local PMC archive. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The Europe PMC supplementary bundle holds the six main-figure images and
sciadv.ads3051_sm.pdf(4 pages). The PDF contains Fig. S1 (a different assay, DJ-1–14-3-3β PLA puncta), Table S1 (antibodies and reagents) and Table S2 (PLA optimisation). It has no DJ-1–β-subunit values or source data, so it cannot settle which set is correct. The Fig. 3 image is the only further evidence, and it matches paragraph 3. - Corrections and comments. As of 2026-10-24, no correction, erratum, retraction or expression of concern was found.
src/known.pyreturns no Crossref update relations and no PMC notices. The Crossref record (published 2025-06-06) has empty update-to and updated-by fields and an empty relation field. The PMC article page shows no notice. The PubPeer API returned no record (HTTP 404) for the DOI, and web searches for the title or DOI with "correction", "erratum" or "PubPeer" found nothing relevant. Not reachable: the Europe PMC search API (HTTP 503), and the science.org page and PubPeer search page (both HTTP 403). - Sources. PMC article PMC12143374; Europe PMC full-text XML and supplementary files for PMC12143374; Crossref; PubPeer; https://doi.org/10.1126/sciadv.ads3051.
Suggested correction
In Results paragraph 5, replace the six values with the ones that match Fig. 3A, as given in paragraph 3. Control: proximal neurites M = 0.538 ± 0.028 R, n = 20; somata M = 0.420 ± 0.032 R, n = 20; Put M = 0.377 ± 0.032 R, n = 19. PD: proximal neurites M = 0.408 ± 0.028 R, n = 20; somata M = 0.357 ± 0.025 R, n = 20; Put M = 0.243 ± 0.030 R, n = 18. If paragraph 5 is instead meant to report a different analysis, that analysis should be described and the panel it refers to identified. Either way, the authors may wish to confirm that the "significantly higher" proximal-versus-somata statement for PD, and the Fig. 3A asterisk, refer to the plotted data.
In Results paragraph 4, the values and the figure citation should describe the comparison the paragraph introduces. If the paragraph is meant to present the VTA gradient claimed in the Introduction and Discussion, the corresponding Fig. 3B values from paragraph 3 are control somata 0.401 ± 0.019, proximal 0.588 ± 0.028, NAc 0.620 ± 0.019; PD somata 0.489 ± 0.027, proximal 0.628 ± 0.030, NAc 0.592 ± 0.027. The stated order ("followed by distal neurites ... and somata") should match whichever numbers are given.
Separately, the control sex split in Results paragraph 2 ("11 males and 9 females") could be aligned with Table 1 (9 males, 11 females), and the Methods mean ages with Table 1.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12143374 (Sci Adv 2025, doi:10.1126/sciadv.ads3051)
# Run: python recompute.py.txt [path/to/sciadv.ads3051-f3.jpg]
#
# Prints every recomputed number in writeup.md and regenerates fig1/fig2 (.svg, .png)
# in the folder that holds this script.
#
# Inputs are the summary statistics printed in the Results section (first Results
# subsection, paragraphs 3, 4 and 5) and Table 1. The Fig. 3A read-off needs the
# published figure image sciadv.ads3051-f3.jpg. If no path is given, the script tries
# to fetch it from the Europe PMC supplementary-files bundle into a temporary folder
# (nothing is saved next to this script). If that fails, it prints the values
# recorded from a run on 2026-10-24 and says so.
import io, os, sys, tempfile, urllib.request, zipfile
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
CELLS = [("CTL", "SNpc soma"), ("CTL", "SNpc prox"), ("CTL", "Put distal"),
("PD", "SNpc soma"), ("PD", "SNpc prox"), ("PD", "Put distal")]
# ---- values printed in the paper: (mean, SEM, n) -------------------------------
# Results para 3 ("We found that ...", cites Fig. 3A)
p3 = {("PD", "SNpc soma"): (0.357, 0.025, 20), ("PD", "SNpc prox"): (0.408, 0.028, 20),
("PD", "Put distal"): (0.243, 0.030, 18), ("CTL", "SNpc soma"): (0.420, 0.032, 20),
("CTL", "SNpc prox"): (0.538, 0.028, 20), ("CTL", "Put distal"): (0.377, 0.032, 19)}
# Results para 4 ("The intracellular heterogeneity ...", cites Fig. 3B)
p4 = {("CTL", "SNpc prox"): (0.538, 0.028, 20), ("CTL", "Put distal"): (0.377, 0.032, 19),
("CTL", "SNpc soma"): (0.420, 0.032, 20), ("PD", "SNpc prox"): (0.408, 0.028, 20),
("PD", "Put distal"): (0.243, 0.030, 18), ("PD", "SNpc soma"): (0.357, 0.025, 20)}
# Results para 5 ("In control SNpc neurons ...", cites Fig. 3A)
p5 = {("CTL", "SNpc prox"): (0.490, 0.040, 20), ("CTL", "SNpc soma"): (0.351, 0.038, 20),
("CTL", "Put distal"): (0.332, 0.047, 19), ("PD", "SNpc prox"): (0.404, 0.046, 20),
("PD", "SNpc soma"): (0.299, 0.032, 20), ("PD", "Put distal"): (0.143, 0.034, 17)}
# Results para 3, VTA/NAc values (cites Fig. 3B)
vta = {("CTL", "VTA soma"): 0.401, ("CTL", "VTA prox"): 0.588, ("CTL", "NAc distal"): 0.620,
("PD", "VTA soma"): 0.489, ("PD", "VTA prox"): 0.628, ("PD", "NAc distal"): 0.592}
print("=" * 78)
print("1. Same group and compartment: paragraph 3 vs paragraph 5 (both cite Fig. 3A)")
print("=" * 78)
print(f"{'cell':16s} {'para 3':>20s} | {'para 5':>20s} | diff ratio z(diff)")
for k in CELLS:
m2, s2, n2 = p3[k]; m4, s4, n4 = p5[k]
print(f"{k[0]:4s}{k[1]:12s} {m2:.3f} +/- {s2:.3f} n={n2:2d} | {m4:.3f} +/- {s4:.3f} n={n4:2d} |"
f" {m2 - m4:+.3f} {m4 / m2:.3f} {(m2 - m4) / np.hypot(s2, s4):.2f}")
print("para 4 identical to para 3 for all six SNpc/Put cells:", all(p4[k] == p3[k] for k in CELLS))
d = [p3[k][0] - p5[k][0] for k in CELLS]
r = [p5[k][0] / p3[k][0] for k in CELLS]
print(f"differences range {min(d):.3f} to {max(d):.3f}; ratios range {min(r):.3f} to {max(r):.3f}"
" (no common offset or factor)")
print("cells where para 5 SEM > para 3 SEM:", sum(p5[k][1] > p3[k][1] for k in CELLS), "of 6")
print("\nImplied SD = SEM * sqrt(n):")
for k in CELLS:
print(f" {k[0]:4s}{k[1]:12s} para 3 SD {p3[k][1] * np.sqrt(p3[k][2]):.3f} para 5 SD {p5[k][1] * np.sqrt(p5[k][2]):.3f}")
print("\nPD vs control in Put distal neurites under each set:")
dput = {}
for name, dd in [("para 3", p3), ("para 5", p5)]:
c, p = dd[("CTL", "Put distal")][0], dd[("PD", "Put distal")][0]
dput[name] = c - p
print(f" {name}: control {c:.3f} - PD {p:.3f} = {c - p:.3f}; PD as % of control {100 * p / c:.0f}%")
print(f" para 5 difference is {100 * (dput['para 5'] / dput['para 3'] - 1):.0f}% larger than para 3")
print("\n" + "=" * 78)
print("2. Can 0.143 (n = 17) be the 0.243 (n = 18) set with one case removed?")
print("=" * 78)
m18, n18 = 0.243, 18
lo, hi = (m18 * n18 - 1.0) / 17, (m18 * n18 - 0.0) / 17
print(f"sum of 18 values = {m18 * n18:.3f}; dropping one value in [0, 1] leaves a mean in [{lo:.4f}, {hi:.4f}]")
need = m18 * n18 - 0.143 * 17
print(f"value that would have to be dropped to reach 0.143: {need:.3f} (outside the 0-1 range of a Manders coefficient)")
print(f"allowing rounding of both means: {18 * 0.2425 - 17 * 0.1435:.4f} to {18 * 0.2435 - 17 * 0.1425:.4f}")
print("\n" + "=" * 78)
print("3. Paragraph 4: stated ordering vs its own numbers")
print("=" * 78)
for g in ["CTL", "PD"]:
o = sorted([(p4[(g, c)][0], c) for c in ["SNpc soma", "SNpc prox", "Put distal"]], reverse=True)
print(f" {g}: numbers order high->low: {[c for _, c in o]} (text: proximal > distal > somata)")
for g in ["CTL", "PD"]:
o = sorted([(vta[(g, c)], c) for c in ["VTA soma", "VTA prox", "NAc distal"]], reverse=True)
print(f" {g} VTA/NAc values from para 3, high->low: {[c for _, c in o]}")
print("\n" + "=" * 78)
print("4. PD proximal vs somata ('significantly higher', para 5) from summary statistics")
print("=" * 78)
for name, dd in [("para 3", p3), ("para 5", p5)]:
(m1, s1, n1), (m0, s0, n0) = dd[("PD", "SNpc prox")], dd[("PD", "SNpc soma")]
t = (m1 - m0) / np.hypot(s1, s0)
df = (s1 ** 2 + s0 ** 2) ** 2 / (s1 ** 4 / (n1 - 1) + s0 ** 4 / (n0 - 1))
print(f" {name}: diff = {m1 - m0:.3f}, unpaired Welch t = {t:.2f}, df = {df:.1f}, p = {2 * stats.t.sf(abs(t), df):.3f}")
print(" (somata and proximal neurites come from the same cases; an unpaired test cannot check a")
print(" repeated-measures or ANOVA/Tukey result, so this is context only)")
print("\n" + "=" * 78)
print("5. Side observation: Table 1 demographics vs Results and Methods")
print("=" * 78)
pd_age = [69, 72, 80, 83, 69, 76, 78, 71, 79, 71, 80, 68, 87, 86, 86, 93, 79, 80, 84, 88]
ct_age = [80, 87, 89, 66, 87, 94, 70, 82, 92, 88, 84, 70, 84, 81, 68, 89, 90, 84, 91, 88]
pd_sex, ct_sex = "MFMMFFMFFMFMMFMFFMMF", "FFMFFFMMFMFMFFFMMMFM"
for name, a, sx, res, meth in [("PD", pd_age, pd_sex, "78.95 +/- 1.61 SEM, 10 M/10 F", "78.6 +/- 6.8, 10 M/10 F"),
("Control", ct_age, ct_sex, "83.20 +/- 1.87 SEM, 11 M/9 F", "83.5 +/- 5.7, 9 M/11 F")]:
a = np.array(a)
print(f" {name}: Table 1 mean {a.mean():.2f}, SD {a.std(ddof=1):.2f}, SEM {a.std(ddof=1) / np.sqrt(20):.2f},"
f" {sx.count('M')} M/{sx.count('F')} F | Results: {res} | Methods: {meth}")
# ---- Fig. 3A / 3B read-off ------------------------------------------------------
RECORDED = {
"bars_A": {("CTL", "SNpc soma"): 0.416, ("CTL", "SNpc prox"): 0.540, ("CTL", "Put distal"): 0.375,
("PD", "SNpc soma"): 0.354, ("PD", "SNpc prox"): 0.409, ("PD", "Put distal"): 0.244},
"bars_B": {("CTL", "VTA soma"): 0.402, ("CTL", "VTA prox"): 0.588, ("CTL", "NAc distal"): 0.615,
("PD", "VTA soma"): 0.485, ("PD", "VTA prox"): 0.629, ("PD", "NAc distal"): 0.588},
"visible_sd": {("CTL", "SNpc soma"): (17, 0.154), ("CTL", "SNpc prox"): (16, 0.142),
("CTL", "Put distal"): (18, 0.142), ("PD", "SNpc soma"): (17, 0.121),
("PD", "SNpc prox"): (19, 0.127), ("PD", "Put distal"): (18, 0.128)},
"pdput_points": [0.034, 0.044, 0.086, 0.134, 0.143, 0.155, 0.160, 0.196, 0.223, 0.284, 0.285,
0.292, 0.335, 0.338, 0.381, 0.402, 0.402, 0.436],
}
def get_figure(path):
if path and os.path.exists(path):
return path, None
try:
url = "https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12143374/supplementaryFiles"
data = urllib.request.urlopen(url, timeout=60).read()
z = zipfile.ZipFile(io.BytesIO(data))
tmp = tempfile.mkdtemp()
out = os.path.join(tmp, "f3.jpg")
with open(out, "wb") as f:
f.write(z.read("sciadv.ads3051-f3.jpg"))
return out, tmp
except Exception as e: # network or archive problem
print(f" (could not fetch Fig. 3 image: {e})")
return None, None
def read_figure(path):
from PIL import Image
from scipy import ndimage
a = np.asarray(Image.open(path).convert("RGB")).astype(int)
s = a.sum(2)
# y-axis tick labels (1.0, 0.5, 0.0) of panel A, found as dark text rows left of the axis
rows = [y for y in range(40, 215) if (s[y, 30:52] < 300).sum() > 0]
grp, cur = [], [rows[0]]
for y in rows[1:]:
if y == cur[-1] + 1:
cur.append(y)
else:
grp.append(cur); cur = [y]
grp.append(cur)
y10, y05, y00 = [(g[0] + g[-1]) / 2 for g in grp]
ppu = y00 - y10
print(f" panel A tick centres (px): 1.0 -> {y10}, 0.5 -> {y05}, 0.0 -> {y00}; "
f"linearity check: 0.5 tick reads {(y00 - y05) / ppu:.3f}")
def bar_value(xc, x0=None):
# mean bar = the longest horizontal dark run (about 31 px) near the column centre
best = None
for y in range(60, 207):
row = s[y, xc - 25:xc + 25] < 250
run = m = 0
for v in row:
run = run + 1 if v else 0
m = max(m, run)
if best is None or m > best[1]:
best = (y, m)
return (y00 - best[0]) / ppu
colsA = dict(zip(CELLS, [86, 127, 168, 241, 282, 322]))
colsB = dict(zip([("CTL", "VTA soma"), ("CTL", "VTA prox"), ("CTL", "NAc distal"),
("PD", "VTA soma"), ("PD", "VTA prox"), ("PD", "NAc distal")],
[488, 529, 570, 642, 683, 724]))
bars_A = {k: round(bar_value(x), 3) for k, x in colsA.items()}
bars_B = {k: round(bar_value(x), 3) for k, x in colsB.items()}
# individual points: blue fill (control) or tan fill (PD), one connected blob per point
R, G, B = a[..., 0], a[..., 1], a[..., 2]
masks = {"CTL": (B > 150) & (R < 140) & (B - R > 60), "PD": (R > 180) & (G > 150) & (B < 160) & (R - B > 40)}
visible = {}
for k, xc in colsA.items():
sub = masks[k[0]][60:210, xc - 22:xc + 22]
lab, nl = ndimage.label(sub)
idx = range(1, nl + 1)
cm = ndimage.center_of_mass(sub, lab, idx); sz = ndimage.sum(sub, lab, idx)
visible[k] = sorted(round((y00 - (cy + 60)) / ppu, 3) for (cy, cx), z in zip(cm, sz) if z >= 5)
return {"bars_A": bars_A, "bars_B": bars_B, "pdput_points": visible[("PD", "Put distal")],
"visible_sd": {k: (len(v), round(float(np.std(v, ddof=1)), 3)) for k, v in visible.items()}}
print("\n" + "=" * 78)
print("6. Fig. 3A and 3B read from the published image (pixel positions against the y-axis)")
print("=" * 78)
figpath, tmpdir = get_figure(sys.argv[1] if len(sys.argv) > 1 else None)
if figpath:
FIG = read_figure(figpath)
print(" source: image read in this run")
else:
FIG = RECORDED
print(" source: values RECORDED from a run on 2026-10-24 (image not available in this run)")
print(f" {'cell':16s} {'Fig. 3A bar':>11s} {'para 3':>7s} {'para 5':>7s} closer to")
for k in CELLS:
v = FIG["bars_A"][k]
closer = "para 3" if abs(v - p3[k][0]) < abs(v - p5[k][0]) else "para 5"
print(f" {k[0]:4s}{k[1]:12s} {v:11.3f} {p3[k][0]:7.3f} {p5[k][0]:7.3f} {closer}"
f" (|bar - para 3| = {abs(v - p3[k][0]):.3f}, |bar - para 5| = {abs(v - p5[k][0]):.3f})")
print(" Fig. 3B bars vs para 3 VTA/NAc values:")
for k, v in FIG["bars_B"].items():
print(f" {k[0]:4s}{k[1]:12s} {v:.3f} vs {vta[k]:.3f}")
pts = np.array(FIG["pdput_points"])
sd = pts.std(ddof=1)
print(f" PD Put points in Fig. 3A: {len(pts)} points; mean {pts.mean():.3f}, SD {sd:.3f}, "
f"SEM {sd / np.sqrt(len(pts)):.3f}; max {pts.max():.3f}")
print(" spread of the points visible in each Fig. 3A column (points hidden behind the mean/SEM bars")
print(" are missed, so counts can fall short of n) vs SD implied by each paragraph:")
for k in CELLS:
nv, sv = FIG["visible_sd"][k]
sd2, sd4 = p3[k][1] * np.sqrt(p3[k][2]), p5[k][1] * np.sqrt(p5[k][2])
print(f" {k[0]:4s}{k[1]:12s} {nv:2d} visible, SD {sv:.3f} | para 3 SD {sd2:.3f} | para 5 SD {sd4:.3f}"
f" closer to {'para 3' if abs(sv - sd2) < abs(sv - sd4) else 'para 5'}")
print(f" dropping the largest plotted point leaves a mean of {(pts.sum() - pts.max()) / (len(pts) - 1):.3f}")
# ---- figures -------------------------------------------------------------------
BG, INK, INK2, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e3e2de"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK,
"ytick.color": INK, "text.color": INK})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: the six SNpc/Put cells, two text versions vs the plotted mean bars
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.patch.set_facecolor(BG); ax.set_facecolor(BG)
xs = np.array([0, 1, 2, 3.6, 4.6, 5.6])
off = 0.22
for i, k in enumerate(CELLS):
m, s, n = p3[k]
ax.errorbar(xs[i] - off, m, yerr=s, fmt="o", ms=8, color=ORANGE, mfc=ORANGE, mec=ORANGE,
elinewidth=2, capsize=3, capthick=2, zorder=3)
m, s, n = p5[k]
ax.errorbar(xs[i], m, yerr=s, fmt="s", ms=8, color=ORANGE, mfc=BG, mec=ORANGE, mew=2,
elinewidth=2, capsize=3, capthick=2, zorder=3)
ax.plot(xs[i] + off, FIG["bars_A"][k], marker="D", ms=8, color=BLUE, ls="none", zorder=4)
# PD Put: plotted points reconstructed from the figure, mean +/- SEM
i = 5
ax.errorbar(xs[i] + off, pts.mean(), yerr=sd / np.sqrt(len(pts)), fmt="none", ecolor=BLUE,
elinewidth=2, capsize=3, capthick=2, zorder=3)
ax.annotate(f"PD Put distal:\n¶3: {p3[CELLS[5]][0]:.3f} (n = {p3[CELLS[5]][2]})\n"
f"¶5: {p5[CELLS[5]][0]:.3f} (n = {p5[CELLS[5]][2]})\n"
f"Fig. 3A: {len(pts)} points, mean\n{pts.mean():.3f} ± {sd / np.sqrt(len(pts)):.3f} (blue whisker)",
xy=(3.3, 0.125), fontsize=10, color=INK, va="center")
ax.set_xticks(xs)
ax.set_xticklabels(["SNpc\nsomata", "SNpc\nproximal", "Put\ndistal"] * 2)
ax.set_xlim(-0.6, 6.2)
ax.set_ylim(0, 0.65)
ax.set_yticks(np.arange(0, 0.61, 0.1))
ax.set_ylabel("PLA signal (Manders' coefficient)\nmean ± SEM")
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right"]:
ax.spines[sp].set_visible(False)
ax.text(1, 0.625, "Control", ha="center", fontsize=11, fontweight="bold")
ax.text(4.6, 0.625, "PD", ha="center", fontsize=11, fontweight="bold")
ax.axvline(2.8, color=GRID, lw=1)
handles = [Line2D([], [], marker="o", color=ORANGE, ms=8, lw=0, label="Reported: Results ¶3 (= ¶4)"),
Line2D([], [], marker="s", color=ORANGE, mfc=BG, mew=2, ms=8, lw=0, label="Reported: ¶5"),
Line2D([], [], marker="D", color=BLUE, ms=8, lw=0, label="Recomputed: Fig. 3A mean bar")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(-0.02, 1.0), ncol=3, frameon=False,
fontsize=10, handletextpad=0.2, columnspacing=0.9)
fig.subplots_adjust(left=0.13, right=0.99, top=0.91, bottom=0.12)
save(fig, "fig1")
# Figure 2: the three passages side by side
fig = plt.figure(figsize=(7.2, 3.9))
fig.patch.set_facecolor(BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 55); ax.axis("off")
def panel(x, y, w, h, title, body, edge):
ax.add_patch(matplotlib.patches.FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=edge, lw=2))
ax.text(x + 1.2, y + h - 1.6, title, fontsize=10.5, fontweight="bold", va="top", color=INK)
ax.text(x + 1.2, y + h - 7.2, body, fontsize=10, va="top", color=INK, linespacing=1.35)
panel(2, 32, 30, 20, "Results ¶3 (cites Fig. 3A)",
"PD: somata 0.357 ± 0.025\nproximal 0.408 ± 0.028\nPut 0.243 ± 0.030, n = 18\n\"(Fig. 3A)\"", ORANGE)
panel(35, 32, 30, 20, "Results ¶4 (cites Fig. 3B)",
"introduced as SNpc vs VTA;\nPD: proximal 0.408 ± 0.028\ndistal 0.243 ± 0.030, n = 18\nsomata 0.357 ± 0.025", ORANGE)
panel(68, 32, 30, 20, "Results ¶5 (cites Fig. 3A)",
"PD: proximal 0.404 ± 0.046\nsomata 0.299 ± 0.032\nPut 0.143 ± 0.034, n = 17\n\"(Fig. 3A)\"", ORANGE)
panel(2, 3, 30, 17, "Fig. 3A, plotted data",
f"PD Put: {len(pts)} points\nmean bar {FIG['bars_A'][CELLS[5]]:.3f}\npoints' mean {pts.mean():.3f} ± {sd / np.sqrt(len(pts)):.3f}", BLUE)
# connectors
ax.text(33.5, 42.5, "=", ha="center", va="center", fontsize=18, fontweight="bold", color=INK2)
ax.text(50, 30.2, "¶4 = ¶3 for all 6 SNpc/Put\nvalues, though ¶4 cites Fig. 3B", ha="center", va="top",
fontsize=10, color=INK2)
ax.annotate("", xy=(17, 20.8), xytext=(17, 31.2), arrowprops=dict(arrowstyle="-", color=BLUE, lw=2))
ax.text(18.2, 26, "matches ¶3", fontsize=10, color=INK2, va="center")
ax.annotate("", xy=(83, 20.6), xytext=(83, 31.2), arrowprops=dict(arrowstyle="-", color=ORANGE, lw=2, ls="--"))
panel(52, 3, 46, 17, "Conflict between ¶3 and ¶5",
"PD Put 0.243 vs 0.143 (n = 18 vs 17);\nthe 5 other SNpc/Put cells also differ.\n"
f"One dropped case would need {need:.3f} (max 1)", ORANGE)
save(fig, "fig2")
print("\nfigures written to", HERE)
The abstract gives the top tolerated monkey dose in mg/kg; the body gives μg/kg
Sci Adv (2021) · doi:10.1126/sciadv.abi8193 · PMC8357232
Summary
The abstract states that the T cell–engaging bispecific CCW702 "was well tolerated up to 34.1 mg/kg per dose" in cynomolgus monkeys. The Results, the Methods and the Fig. 6 legend all describe a single repeat-dose monkey study with doses of 2, 9.8 and 34.1 μg/kg, and they put the no-observed-adverse-effect level (NOAEL) at 34.1 μg/kg. Read literally, the abstract's 34.1 mg/kg is 34,100 μg/kg, 1,000 times the highest dose tested. The paper's own exposure value for that group (day-1 AUC 1.02 h·μg/mL) gives an apparent clearance of 33 mL/h/kg for a μg/kg dose. For a mg/kg dose it gives about 33,400 mL/h/kg, which is 2.6 times total cardiac output and physically impossible. This is the abstract's only primate safety number. Read as printed, it puts the tolerated monkey dose level with the highest mouse efficacy dose (35.2 mg/kg), when the tested dose was 33 to 1,000 times lower than the mouse efficacy doses.
What the paper reports
Abstract:
In cynomolgus monkeys, CCW702 was well tolerated up to 34.1 mg/kg per dose, with near-complete subcutaneous bioavailability and a PK profile supporting testing of a weekly dosing regimen in patients.
Results, "Pharmacokinetics and pharmacodynamics in nonhuman primate", second paragraph:
Doses of 2, 9.8, and 34.1 μg/kg were administered to animals QAD for 10 days (five total doses).
CCW702 administration was generally well tolerated at these dose levels with no changes in animal body weight, food consumption, or body temperature and the no-observed-adverse-effect level (NOAEL) determined to be the highest dose in the study of 34.1 μg/kg.
This occurred despite similar exposure of CCW702 on day 1 and on day 9 (1.02 hours·μg/ml versus 0.705 hours·μg/ml for the 34.1 μg/kg group) (fig. S17).
Same section, first paragraph (single-dose PK studies):
CCW702 exposure was detectable by electrochemiluminescence bioanalytical assay at doses of ≥9.8 μg/kg.
Subcutaneous dosing had near complete bioavailability (F = 108 ± 20.2%) with a maximum concentration (Cmax) and half-life (t1/2) of 1.5 (ng/ml)/(mg/kg) and 11.4 hours compared with the intravenous route of administration Cmax of 28.3 (ng/ml)/(mg/kg) and t1/2 of 1.6 hours, respectively (fig. S17).
Fig. 6 legend, panel A (panel B gives the same doses):
Heatmaps of cytokines measured from peripheral blood of animals administrated CCW702 at 2, 9.8, and 34.1 μg/kg, every other day for 10 days (a total of five total doses).
Methods, "Pharmacokinetic and pharmacodynamic 10-day study in cyno":
Nine naïve male cynomolgus monkeys (Macaca fascicularis, Chinese origin), age 2 to 4 years, weighing 2.5 to 3.2 kg were used for the study.
The NOAEL was determined to be 34.1 μg/kg under the study conditions.
For context, Results, "In vivo efficacy in xenograft and patient-derived xenograft models":
every 7 days (Q7D) at four-dose levels (1.13, 3.5, 11.1, and 35.2 mg/kg; groups G to J)
What the paper's own numbers show
Method. I converted the abstract's figure to μg/kg and compared it with the doses tested in the body. To find which unit the data support, I divided the top dose by the paper's day-1 AUC for that group. That gives the apparent clearance CL/F, and since F ≈ 1 (108%) it is close to systemic clearance. I compared CL/F under each reading with standard physiological values for the monkey (Davies & Morris, Pharm Res 1993;10:1093–5): cardiac output 217 mL/min/kg (13,020 mL/h/kg), hepatic blood flow 43.6 mL/min/kg (2,616 mL/h/kg) and plasma volume 44.8 mL/kg. Plasma clearance cannot exceed the blood flow that carries the drug to the organs that eliminate it, so it cannot exceed cardiac output. The AUC interval is given in fig. S17, which I could not access. A partial AUC would make CL/F an overestimate, but by much less than the factor of 1,000 at issue here. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Highest tolerated monkey dose | Abstract | 34.1 mg/kg per dose | = 34,100 μg/kg |
| Monkey repeat-dose levels | Results NHP para 2; Fig. 6 legend | 2, 9.8, 34.1 μg/kg, QAD × 5 | highest = 34.1 μg/kg = 0.0341 mg/kg |
| NOAEL | Results NHP para 2; Methods | 34.1 μg/kg | equals the top dose tested |
| Abstract value / top dose tested | derived | (implied equal) | 1,000 |
| Cumulative dose, top group | derived | — | 5 × 34.1 = 170.5 μg/kg = 0.171 mg/kg |
| Amount per dose, 2.5–3.2 kg animal | Methods (weights) | — | 0.085–0.109 mg if μg/kg; 85–109 mg if mg/kg |
| CL/F if top dose = 34.1 μg/kg | Results NHP para 2 (AUC 1.02 h·μg/mL) | — | 33.4 mL/h/kg, 0.26% of cardiac output (day 9: 48.4 mL/h/kg) |
| CL/F if top dose = 34.1 mg/kg | same | — | 33,431 mL/h/kg, 2.6 × total cardiac output |
| SC Cmax and AUC at top dose | Results NHP para 1 (1.5 per unit dose, t1/2 11.4 h) | AUC 1.02 h·μg/mL | Cmax 51.2 ng/mL; crude AUC ≈ Cmax/k = 841 ng·h/mL, close to the reported 1,020. This check holds under either reading, so it does not settle the unit on its own |
| Mouse Q7D efficacy doses / top monkey dose | Results (mouse efficacy) | 1.13–35.2 mg/kg | 33× to 1,032× the top monkey dose tested |
| Abstract value / top mouse dose | derived | — | 34.1 / 35.2 = 0.97 |
Figure
Figure 1. The two statements of the top cynomolgus dose. The left panel (orange, the value as printed in the abstract) quotes the abstract's "34.1 mg/kg per dose", which equals 34,100 μg/kg. The right panel (blue) shows the doses used in the Results, Methods and Fig. 6 legend, with the NOAEL at 34.1 μg/kg. The connector marks the factor of 1,000 between the two. The bottom line of the right panel gives the apparent clearance recomputed from the paper's own day-1 AUC, which is plausible only for the μg/kg reading. All values come from the paper or from recompute.py.txt.
Figure 2. Where the factor of 1,000 enters. The dose label "34.1" for the top group is read in the abstract's unit (orange, mg/kg) and in the body's unit (blue, μg/kg). Each reading is divided by the paper's own day-1 AUC for that group (1,020 ng·h/mL). The mg/kg reading gives an apparent clearance of 33,431 mL/h/kg, 2.6 times the total cardiac output of a monkey, which is not possible. The μg/kg reading gives 33.4 mL/h/kg, which is plausible. The cardiac output value is an external reference (Davies & Morris 1993); all other numbers come from the paper or from recompute.py.txt.
Why it matters
The abstract sentence is the paper's only summary of primate safety. It is also the evidence the abstract offers just before it says the drug is in a first-in-human trial (NCT04077021). The data in the body are reported consistently, and the qualitative finding holds: CCW702 was well tolerated at every dose tested, and the NOAEL was the top dose. The error is in magnitude. Read as printed, the abstract suggests that monkeys tolerated about 34 mg/kg per dose, roughly the highest mouse efficacy dose (35.2 mg/kg Q7D), which implies a wide tolerability margin over the efficacious range. In fact the highest dose tested in monkeys was 0.0341 mg/kg per dose, 33 to 1,000 times below the mouse Q7D efficacy doses. Mouse and monkey doses in mg/kg are not directly interchangeable across species, but that is the comparison a reader of the abstract would make. No dose above 34.1 μg/kg was tested in monkeys, so the study established a NOAEL but not a maximum tolerated dose. The Methods also report clinical pathology changes and microscopic findings, judged not adverse, at doses above 2 μg/kg. A reader who relies on the abstract, or a database or review that cites it, will carry a tolerated dose that is 1,000 times too high.
Explanations considered
- A different, higher-dose monkey study. This does not account for the numbers. The only monkey tolerability study described is the 10-day repeat-dose study (nine animals, three groups), and its doses are 2, 9.8 and 34.1 μg/kg. The single-dose PK studies report no dose in mg/kg; their only dose statement is detectability "at doses of ≥9.8 μg/kg". The numeral 34.1 in the abstract matches the body's top dose exactly, unit aside.
- The body is wrong and the abstract is right. This is ruled out by the paper's own data. With the reported day-1 AUC, a 34.1 mg/kg dose implies a clearance of 33,431 mL/h/kg, 2.6 times total cardiac output. The μg/kg reading gives 33.4 mL/h/kg. The body uses μg/kg in six separate places (two Results paragraphs, both Fig. 6 legend panels, the Methods), and the version-of-record check found the same unit in the panel labels of the Fig. 6 image itself ("34.1 μg/kg/dose").
- A text-conversion artifact (μ rendered as m). This is ruled out. The same "34.1 mg/kg" appears in the publisher's JATS XML at Europe PMC, on the live PMC page and in the PubMed abstract, which the publisher supplies separately. The same files render μ correctly everywhere else.
- Mixing up mouse and monkey units. The mouse doses really are in mg/kg (0.15 to 35.2 mg/kg), and this may be how mg/kg came to appear. It does not change the result, because the abstract sentence names cynomolgus monkeys explicitly.
- A related unit question (side note, not part of this claim). The dose-normalized Cmax values are labeled "(ng/ml)/(mg/kg)". An IV Cmax of 28.3 ng/mL per mg/kg would require an initial distribution volume of about 35 L/kg for a roughly 50 kDa protein. Per μg/kg it gives 35 mL/kg, close to plasma volume (44.8 mL/kg, which predicts about 22 ng/mL per μg/kg). So these labels may also be 1,000-fold off. I list this only as an observation, because I could not check it against fig. S17.
Status
As of 2026-10-23. Version of record: the publisher's site (science.org, the target of the DOI redirect) returned HTTP 403, so I could not read its HTML or PDF directly. Four publisher-derived sources carry the same text: the live PMC page, the Europe PMC full-text XML (publisher JATS), the PubMed record (PMID 34380625), and the publisher's Fig. 6 image file, whose panel labels read "2 μg/kg/dose", "9.8 μg/kg/dose" and "34.1 μg/kg/dose". The quotations above were checked verbatim against the PMC-archived text. Supplement: the supplementary PDF (abi8193_sm.pdf, figs. S1 to S18, including fig. S17 with the monkey PK) is hosted only on science.org, which returned 403. Europe PMC's supplementary bundle holds only the main figures, so I could not check figs. S17 and S18. Corrections: I found none. Crossref has no update-to or updated-by relation, and the updates: filter returns 0 results. Crossref Labs (which includes Retraction Watch data) lists no updates. Europe PMC and PubMed show no errata or comment/correction links, and two web searches for an erratum or correction found none. PubPeer: the search page and the API returned 403 or not found, so I could not check PubPeer directly. A general web search found no PubPeer thread.
Suggested correction
In the abstract, replace "In cynomolgus monkeys, CCW702 was well tolerated up to 34.1 mg/kg per dose" with "In cynomolgus monkeys, CCW702 was well tolerated up to 34.1 μg/kg per dose (the highest dose tested)". The authors may also want to check the unit labels of the dose-normalized Cmax values in the first NHP Results paragraph and in fig. S17 ("(ng/ml)/(mg/kg)"), which the paper's own values suggest should be per μg/kg.
Recomputation script (python recompute.py.txt)
"""Recompute the cynomolgus dose-unit check for PMC8357232
(Sci Adv 2021, doi:10.1126/sciadv.abi8193).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file, five candidate figure designs (candidate1..5.svg/.png)
plus fig1 (= candidate 1) and fig2 (= candidate 2).
Inputs are copied from the article as archived in PubMed Central:
Abstract: "well tolerated up to 34.1 mg/kg per dose"
Results (NHP para 2), Methods (10-day cyno study), Fig. 6 legend: doses 2, 9.8, 34.1 ug/kg; NOAEL 34.1 ug/kg
Results (NHP para 2): day-1 exposure 1.02 h*ug/mL, day-9 0.705 h*ug/mL, 34.1 ug/kg group
Results (NHP para 1): SC F = 108%, SC Cmax 1.5 and IV Cmax 28.3 "(ng/ml)/(mg/kg)", SC t1/2 11.4 h
Methods: monkeys weighed 2.5 to 3.2 kg
Results (mouse efficacy): Q7D doses 1.13, 3.5, 11.1, 35.2 mg/kg
External reference values (not from the paper), cynomolgus/rhesus monkey, Davies & Morris,
Pharm Res 1993;10:1093-5: plasma volume 44.8 mL/kg, cardiac output 217 mL/min/kg,
hepatic blood flow 43.6 mL/min/kg.
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (paper)
ABSTRACT_TOP_MG_KG = 34.1
CYNO_DOSES_UG_KG = [2.0, 9.8, 34.1]
NOAEL_UG_KG = 34.1
N_DOSES = 5
AUC_DAY1_H_UG_ML = 1.02
AUC_DAY9_H_UG_ML = 0.705
F_SC = 1.08
CMAX_SC_PER_UNIT = 1.5 # printed as (ng/ml)/(mg/kg)
CMAX_IV_PER_UNIT = 28.3 # printed as (ng/ml)/(mg/kg)
T_HALF_SC_H = 11.4
WEIGHT_KG = (2.5, 3.2)
MOUSE_Q7D_MG_KG = [1.13, 3.5, 11.1, 35.2]
# ------------------------------------------------------------------ external reference
PLASMA_VOL_ML_KG = 44.8
CARDIAC_OUTPUT_ML_MIN_KG = 217.0
HEPATIC_FLOW_ML_MIN_KG = 43.6
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none"})
def section(t):
print("\n== " + t)
# ------------------------------------------------------------------ 1. unit conversion
section("1. Abstract dose vs. dose tested")
abstract_ug = ABSTRACT_TOP_MG_KG * 1000
top_tested = max(CYNO_DOSES_UG_KG)
ratio = abstract_ug / top_tested
print(f"abstract 34.1 mg/kg = {abstract_ug:,.0f} ug/kg")
print(f"highest dose tested (Results/Methods/Fig. 6) = {top_tested} ug/kg = {top_tested/1000:.4f} mg/kg; NOAEL = {NOAEL_UG_KG} ug/kg")
print(f"ratio abstract / tested = {ratio:,.0f}")
print(f"cumulative over {N_DOSES} doses at top dose = {N_DOSES*top_tested:.1f} ug/kg = {N_DOSES*top_tested/1000:.3f} mg/kg")
for w in WEIGHT_KG:
print(f" per-dose amount for a {w} kg monkey: {top_tested*w/1000:.3f} mg if ug/kg, {ABSTRACT_TOP_MG_KG*w:.1f} mg if mg/kg")
# ------------------------------------------------------------------ 2. clearance check
section("2. Apparent clearance from the paper's own day-1 AUC (34.1 group)")
auc_ng_h_ml = AUC_DAY1_H_UG_ML * 1000
cl = {}
for label, dose_ng_kg in [("34.1 ug/kg", 34.1e3), ("34.1 mg/kg", 34.1e6)]:
cl[label] = dose_ng_kg / auc_ng_h_ml # mL/h/kg = CL/F; F ~ 1 (108%)
print(f"CL/F if dose = {label}: {cl[label]:,.1f} mL/h/kg = {cl[label]/1000:,.2f} L/h/kg")
co_ml_h_kg = CARDIAC_OUTPUT_ML_MIN_KG * 60
hep_ml_h_kg = HEPATIC_FLOW_ML_MIN_KG * 60
print(f"reference: cardiac output {co_ml_h_kg:,.0f} mL/h/kg, hepatic blood flow {hep_ml_h_kg:,.0f} mL/h/kg")
print(f"mg/kg reading: CL/F = {cl['34.1 mg/kg']/co_ml_h_kg:.1f} x total cardiac output")
print(f"ug/kg reading: CL/F = {cl['34.1 ug/kg']/co_ml_h_kg*100:.2f}% of cardiac output")
print(f"day-9 CL/F (ug/kg reading): {34.1e3/(AUC_DAY9_H_UG_ML*1000):.1f} mL/h/kg")
# ------------------------------------------------------------------ 3. side note: Cmax labels
section("3. Side note: the '(ng/ml)/(mg/kg)' Cmax labels")
c0_per_ug = 1000 / PLASMA_VOL_ML_KG # ng/mL per ug/kg if dose stays in plasma volume
print(f"IV dose of 1 ug/kg in {PLASMA_VOL_ML_KG} mL/kg plasma -> C0 about {c0_per_ug:.1f} ng/mL "
f"(1 mg/kg -> {c0_per_ug*1000:,.0f} ng/mL); reported IV Cmax {CMAX_IV_PER_UNIT} per unit dose")
print(f"implied initial volume if per mg/kg: {1e6/CMAX_IV_PER_UNIT/1000:,.1f} L/kg; if per ug/kg: {1e3/CMAX_IV_PER_UNIT:.1f} mL/kg")
cmax_sc = CMAX_SC_PER_UNIT * 34.1
k = math.log(2) / T_HALF_SC_H
print(f"SC Cmax at top dose = {cmax_sc:.1f} ng/mL; crude AUC ~ Cmax/k = {cmax_sc/k:.0f} ng*h/mL vs reported {auc_ng_h_ml:.0f}")
print(" (this pairing is consistent under either reading, so it does not by itself fix the unit)")
# ------------------------------------------------------------------ 4. context
section("4. Context: mouse Q7D efficacy doses vs. highest cyno dose tested")
for d in MOUSE_Q7D_MG_KG:
print(f" mouse {d} mg/kg = {d*1000:,.0f} ug/kg = {d*1000/top_tested:,.0f} x the top cyno dose")
print(f"abstract figure 34.1 mg/kg vs top mouse dose 35.2 mg/kg: ratio {ABSTRACT_TOP_MG_KG/35.2:.2f}")
# ================================================================== figures
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), bbox_inches="tight", pad_inches=0.12, **kw)
def box(ax, x, y, w, h, edge, lw=2.0, fill=BG):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
lw=lw, ec=edge, fc=fill, transform=ax.transData, clip_on=False))
def cand1():
"""Two text panels: Abstract vs body, joined by an annotated connector."""
fig, ax = plt.subplots(figsize=(7.2, 3.6)); ax.axis("off")
ax.set_xlim(-0.01, 1.01); ax.set_ylim(0, 1); ax.set_position([0.01, 0.01, 0.98, 0.98])
box(ax, 0.0, 0.04, 0.38, 0.90, ORANGE)
ax.text(0.02, 0.89, "Abstract (as printed)", fontsize=11.5, weight="bold", va="top")
ax.text(0.02, 0.77, "\u201cIn cynomolgus monkeys,\nCCW702 was well\ntolerated up to", va="top", fontsize=10.5, linespacing=1.35)
ax.text(0.02, 0.48, "34.1 mg/kg", va="top", fontsize=15, weight="bold")
ax.text(0.02, 0.36, "per dose\u201d", va="top", fontsize=10.5)
ax.text(0.02, 0.08, "34.1 mg/kg = 34,100 \u03bcg/kg", va="bottom", fontsize=10, color=INK2)
box(ax, 0.58, 0.04, 0.42, 0.90, BLUE)
ax.text(0.60, 0.89, "Results, Methods, Fig. 6", fontsize=11.5, weight="bold", va="top")
ax.text(0.60, 0.77, "Doses tested: 2, 9.8, 34.1 \u03bcg/kg\n(every other day, five doses)", va="top", fontsize=10, linespacing=1.35)
ax.text(0.60, 0.59, "\u201c\u2026 NOAEL \u2026 the highest dose\nin the study of", va="top", fontsize=10, linespacing=1.35)
ax.text(0.60, 0.43, "34.1 \u03bcg/kg\u201d", va="top", fontsize=15, weight="bold")
ax.text(0.60, 0.08, "Day-1 AUC 1.02 h\u00b7\u03bcg/mL gives\nCL/F 33 mL/h/kg only for \u03bcg/kg", va="bottom", fontsize=9.8, color=INK2)
ax.add_patch(FancyArrowPatch((0.385, 0.42), (0.575, 0.42), arrowstyle="<->", mutation_scale=14,
lw=2, color=INK2, transform=ax.transData))
ax.text(0.48, 0.47, "same dose,\nunit differs", ha="center", va="bottom", fontsize=10.5)
ax.text(0.48, 0.37, f"\u00d7{ratio:,.0f}", ha="center", va="top", fontsize=15, weight="bold")
save(fig, "candidate1"); save(fig, "fig1"); plt.close(fig)
def cand2():
"""Arithmetic flow: dose / AUC -> clearance; divergence at the unit step."""
fig, ax = plt.subplots(figsize=(7.2, 3.6)); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
box(ax, 0.00, 0.38, 0.18, 0.24, INK2)
ax.text(0.09, 0.50, "Dose label\n\u201c34.1\u201d\n(top group)", ha="center", va="center", fontsize=10.5)
rows = [(0.68, ORANGE, "Abstract: mg/kg", "34,100,000 ng/kg", f"{cl['34.1 mg/kg']:,.0f} mL/h/kg",
f"{cl['34.1 mg/kg']/co_ml_h_kg:.1f}\u00d7 total cardiac\noutput: not possible"),
(0.08, BLUE, "Body: \u03bcg/kg", "34,100 ng/kg", f"{cl['34.1 ug/kg']:,.1f} mL/h/kg",
f"{cl['34.1 ug/kg']/co_ml_h_kg*100:.1f}% of cardiac\noutput: plausible")]
for y, col, unit, ng, clv, verdict in rows:
box(ax, 0.235, y, 0.25, 0.24, col, lw=3)
ax.text(0.36, y + 0.12, f"{unit}\n= {ng}", ha="center", va="center", fontsize=9.8, linespacing=1.15)
box(ax, 0.53, y, 0.18, 0.24, INK2)
ax.text(0.62, y + 0.12, "\u00f7 day-1 AUC\n1,020\nng\u00b7h/mL", ha="center", va="center", fontsize=9.8)
ax.text(0.74, y + 0.17, "CL/F = " + clv, fontsize=10.5, weight="bold", va="center")
ax.text(0.74, y + 0.07, verdict, fontsize=9.8, color=INK2, va="center", linespacing=1.2)
ax.add_patch(FancyArrowPatch((0.485, y + 0.12), (0.53, y + 0.12), arrowstyle="-|>", mutation_scale=12, lw=2, color=INK2, transform=ax.transData))
ax.add_patch(FancyArrowPatch((0.18, 0.50), (0.235, y + 0.12), arrowstyle="-|>", mutation_scale=12, lw=2, color=col, transform=ax.transData))
ax.text(0.36, 0.50, "the factor of 1,000\nenters at this step", ha="center", va="center", fontsize=10.5, weight="bold")
ax.text(0.0, -0.04, "AUC: Results, 34.1 group, day 1. F \u2248 1 (108%), so CL/F \u2248 CL. "
"Cardiac output 13,020 mL/h/kg (monkey reference value).", fontsize=9.5, color=INK2, va="top")
save(fig, "candidate2"); save(fig, "fig2"); plt.close(fig)
def cand3():
"""Dose ladder on one log axis (ug/kg)."""
fig, ax = plt.subplots(figsize=(7.2, 3.0))
rows = ["Mouse efficacy, Q7D\n(Results)", "Cyno, as stated\nin Abstract", "Cyno, doses tested\n(Results, Methods, Fig. 6)"]
ax.scatter([d * 1000 for d in MOUSE_Q7D_MG_KG], [0] * 4, s=70, marker="s", facecolor="white", edgecolor=INK2, lw=2, zorder=3, label="Mouse doses (context)")
ax.scatter([abstract_ug], [1], s=90, color=ORANGE, zorder=3, label="Abstract: 34.1 mg/kg")
ax.scatter(CYNO_DOSES_UG_KG, [2] * 3, s=90, color=BLUE, zorder=3, label="Body: 2, 9.8, 34.1 μg/kg")
ax.annotate("", xy=(abstract_ug, 1.5), xytext=(34.1, 1.5), arrowprops=dict(arrowstyle="<->", color=INK2, lw=2))
ax.text(math.sqrt(abstract_ug * 34.1), 1.62, "×1,000", ha="center", va="bottom", fontsize=11, weight="bold")
ax.text(abstract_ug, 0.75, "34,100", ha="center", va="top", fontsize=10)
for d in CYNO_DOSES_UG_KG:
ax.text(d, 2.25, f"{d:g}", ha="center", va="bottom", fontsize=10)
for d in MOUSE_Q7D_MG_KG:
ax.text(d * 1000, -0.25, f"{d*1000:,.0f}", ha="center", va="top", fontsize=10)
ax.set_xscale("log"); ax.set_xlim(1, 1e5); ax.set_ylim(-0.7, 2.7)
ax.set_yticks([0, 1, 2]); ax.set_yticklabels(rows); ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.set_xlabel("Dose per administration (μg/kg, log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
ax.legend(loc="upper left", frameon=False, fontsize=9.5, bbox_to_anchor=(1.0, 1.0))
save(fig, "candidate3"); plt.close(fig)
def cand4():
"""Implied clearance under each reading vs physiological ceilings, one log axis."""
fig, ax = plt.subplots(figsize=(7.2, 2.6))
ax.axvline(hep_ml_h_kg, color=INK2, lw=1, ls=":")
ax.axvline(co_ml_h_kg, color=INK2, lw=1, ls="--")
ax.text(hep_ml_h_kg, 1.55, f"hepatic blood flow\n{hep_ml_h_kg:,.0f}", ha="right", va="top", fontsize=9.5, color=INK2)
ax.text(co_ml_h_kg * 1.1, 1.55, f"total cardiac\noutput {co_ml_h_kg:,.0f}", ha="left", va="top", fontsize=9.5, color=INK2)
ax.scatter([cl["34.1 mg/kg"]], [1], s=90, color=ORANGE, zorder=3)
ax.scatter([cl["34.1 ug/kg"]], [0], s=90, color=BLUE, zorder=3)
ax.text(cl["34.1 mg/kg"] * 1.25, 1, f"{cl['34.1 mg/kg']:,.0f}", ha="left", va="center", fontsize=10.5, weight="bold")
ax.text(cl["34.1 ug/kg"] * 1.25, 0, f"{cl['34.1 ug/kg']:.1f}", ha="left", va="center", fontsize=10.5, weight="bold")
ax.set_yticks([0, 1]); ax.set_yticklabels(["Dose read as 34.1 μg/kg\n(Results, Methods)", "Dose read as 34.1 mg/kg\n(Abstract)"])
ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.set_xscale("log"); ax.set_xlim(1, 1e6); ax.set_ylim(-0.6, 1.7)
ax.set_xlabel("Apparent clearance CL/F = dose / day-1 AUC (mL/h/kg, log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
save(fig, "candidate4"); plt.close(fig)
def cand5():
"""Where the dose appears, and in which unit: a location-by-unit dot table."""
locs = ["Abstract", "Results, NHP para 1 (detectable ≥ 9.8)", "Results, NHP para 2 (dose groups)",
"Results, NHP para 2 (NOAEL)", "Results, NHP para 2 (cytokines, AUC)",
"Fig. 6 legend, panels A and B", "Methods, 10-day cyno study (NOAEL)"]
unit = ["mg", "ug", "ug", "ug", "ug", "ug", "ug"]
fig, ax = plt.subplots(figsize=(7.2, 3.2))
for i, (l, u) in enumerate(zip(locs, unit)):
y = len(locs) - 1 - i
x = 0 if u == "mg" else 1
ax.scatter([x], [y], s=110, color=ORANGE if u == "mg" else BLUE, zorder=3)
ax.scatter([1 - x], [y], s=60, facecolor="none", edgecolor=GRID, lw=1.5, zorder=2)
ax.set_yticks(range(len(locs))); ax.set_yticklabels(locs[::-1]); ax.tick_params(axis="y", length=0, labelcolor=INK)
ax.set_xticks([0, 1]); ax.set_xticklabels(["mg/kg", "μg/kg"], fontsize=11, color=INK)
ax.xaxis.tick_top(); ax.tick_params(axis="x", length=0)
ax.set_xlim(-0.6, 2.6); ax.set_ylim(-0.6, len(locs) - 0.4)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ax.spines.values(): s.set_visible(False)
ax.text(1.35, len(locs) - 1, "34.1 mg/kg (as printed)\n= 34,100 μg/kg", va="center", fontsize=10, color=INK2)
ax.text(1.35, 2.5, "All six body locations:\n2, 9.8, 34.1 μg/kg\n(filled dot = unit used;\nopen = not used)", va="center", fontsize=10, color=INK2)
save(fig, "candidate5"); plt.close(fig)
for f in (cand1, cand2, cand3, cand4, cand5):
f()
print("\nfigures written to", HERE)
Abstract reports Table 2's other-respiratory-infection hazard ratios as the COVID-19 severity and mortality estimates
eLife (2024) · doi:10.7554/eLife.94973 · PMC11251724
Summary
This UK Biobank cohort study (160,923 participants) relates regular proton pump inhibitor (PPI) use to respiratory infections. Its abstract gives the hazard ratios for severe COVID-19 and COVID-19 mortality as 1.19 (95% CI 1.11–1.27) and 1.37 (1.29–1.46). Those two triplets match, digit for digit, the fully adjusted Table 2 estimates for other upper and other lower respiratory infections. The paper's Results text and Supplementary file 1e report the COVID-19 outcomes as 1.33 (1.09–1.61) and 1.46 (1.05–2.03). The paper's p-values, event counts and severity E-value fit the Results figures and not the abstract's. The direction and significance stay the same, but the abstract understates both effects and makes the two estimates look far more precise than they are: its intervals are 2.9 and 5.3 times too narrow on the log scale. The error has been in every version since the 2022 preprint, and as of 2026-10-24 no correction has been published.
What the paper reports
Abstract, Results:
In contrast, the risk of COVID-19 infection was not significant with regular PPI use (HR 1.08, 95% CI 0.99–1.17), while the risks of severe COVID-19 (HR 1.19. 95% CI 1.11–1.27) and mortality (HR 1.37. 95% CI 1.29–1.46) were increased.
Results, "Proton pump inhibitor use and other respiratory infections, COVID-19 severity, and COVID-19 mortality":
For other upper and lower respiratory infections, the risks among regular PPI users were increased by 19% (aHR 1.19, 95% CI 1.11–1.27, p<0.001; e-value 1.67) and 37% (aHR 1.37, 95% CI 1.29–1.46, p<0.001; e-value 2.08), respectively. In contrast, the risks of developing severe COVID-19 (aHR 1.33, 95% CI 1.09–1.61, p=0.004; e value 1.99) and mortality due to COVID-19 (aHR 1.46, 95% CI 1.05–2.03, p=0.024; e value 2.03) were significantly increased among PPI users compared to those among PPI non-users (Supplementary file 1e).
Table 2, regular PPI use rows. The columns are Case/person-years | non-adjusted HR (95% CI) | p | age/sex-adjusted HR (95% CI) | p | fully adjusted HR (95% CI) | p:
Other upper respiratory infections: Regular PPI use | 1 118/3 988 | 1.30 (1.22–1.38) | <0.001 | 1.31 (1.23–1.39) | <0.001 | 1.19 (1.11–1.27) | <0.001
Other lower respiratory infections: Regular PPI use | 1 486/5 598 | 1.78 (1.67–1.88) | <0.001 | 1.65 (1.56–1.74) | <0.001 | 1.37 (1.29–1.46) | <0.001
Supplementary file 1e, "Associations of PPI use with COVID-19 severity and mortality", regular PPI use rows (same column layout):
COVID-19 severity: Regular PPI use | 138/100 | 2.32 (1.94-2.77) | <0.001 | 1.95 (1.64-2.34) | <0.001 | 1.33 (1.09-1.61) | 0.004
COVID-19 mortality: Regular PPI use | 48/36 | 2.61 (1.93-3.54) | <0.001 | 1.90 (1.40-2.58) | <0.001 | 1.46(1.05-2.03) | 0.024
The matching non-regular-use rows have 1 104 and 337 cases.
Methods, "Definition of outcomes":
The secondary outcomes included other upper or lower respiratory infections, COVID-19 mortality, and COVID-19 severity.
Severe COVID-19 cases were defined as being hospitalized for COVID-19. COVID-19 mortality was defined as the underlying recorded cause of death due to COVID-19 (ICD-10 U07.1 and U07.2).
Abstract, Conclusions, and Discussion, "Conclusion":
PPI users are associated with increased risks of influenza, pneumonia, as well as COVID-19 severity and mortality compared to non-users, while the effects on pneumonia or COVID-19-related outcomes under PPI use were attenuated when compared to the use of H2RAs.
In conclusion, compared to non-users, people regularly taking PPIs are associated with increased susceptibility to influenza, pneumonia, as well as COVID-19 severity and mortality, while their association with pneumonia and COVID-19-related outcomes is diminished after comparison with H2RA use and remains to be further explored.
What the paper's own numbers show
Method. I compared each hazard ratio and confidence interval in the abstract with every fully adjusted row of Table 2, and with every COVID-19 severity and mortality estimate in Supplementary file 1. That covers table 1e (all three models), 1j (multiple imputation), 1m (propensity-score matched) and 1n (versus H2RA users). I treated each 95% CI as a Wald interval, symmetric on the log-HR scale as Cox output is, so SE = (ln U − ln L)/(2 × 1.96) and p = 2[1 − Φ(|ln HR|/SE)]. I recomputed each E-value from its point estimate as E = HR + √(HR(HR − 1)), which is the formula the paper's other printed E-values follow. To check how plausible each interval's width is, I compared its SE with √(1/d₁ + 1/d₀), where d₁ and d₀ are the event counts in the two groups. This is the approximate information-based SE of a log hazard ratio for a binary exposure, and covariate adjustment usually makes the actual SE larger, not smaller. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Severe COVID-19, HR (95% CI) | Abstract | 1.19 (1.11–1.27) | Identical to Table 2, other upper respiratory infections, fully adjusted |
| COVID-19 mortality, HR (95% CI) | Abstract | 1.37 (1.29–1.46) | Identical to Table 2, other lower respiratory infections, fully adjusted |
| Severe COVID-19, aHR (95% CI) | Results text; Supp. 1e | 1.33 (1.09–1.61) | Differs from the abstract |
| COVID-19 mortality, aHR (95% CI) | Results text; Supp. 1e | 1.46 (1.05–2.03) | Differs from the abstract |
| Any other COVID-19 severity or mortality estimate equal to the abstract's values | Supp. 1e (non- and age/sex-adjusted), 1j, 1m, 1n; Table 2 COVID-19 positivity (3 models) | — | None |
| Influenza, pneumonia, COVID-19 positivity HRs | Abstract vs Table 2 | 1.32 (1.12–1.56); 1.42 (1.26–1.59); 1.08 (0.99–1.17) | Identical to Table 2 (the other abstract estimates are transcribed correctly) |
| Severe COVID-19: p from CI | Results | p = 0.004 | 0.0042 |
| COVID-19 mortality: p from CI | Results | p = 0.024 | 0.0244 |
| Severe COVID-19: E-value | Results | 1.99 | 1.99 for HR 1.33 (1.67 for HR 1.19) |
| COVID-19 mortality: E-value | Results | 2.03 | 2.28 for HR 1.46 (2.08 for HR 1.37); 2.03 corresponds to HR 1.347, nearer 1.37 than 1.46; it supports neither figure and is a separate minor inconsistency |
| E-values for influenza, pneumonia, other upper, other lower (Table 2 HRs) | Results | 1.97, 2.19, 1.67, 2.08 | 1.97, 2.19, 1.67, 2.08 |
| E-values vs H2RA: influenza, other upper, other lower | Results | 2.87, 1.88, 1.99 | 2.87, 1.88, 1.99 |
| Severe COVID-19: SE(ln HR) | Abstract vs Results | — | 0.0344 vs 0.0995 (ratio 2.90) |
| COVID-19 mortality: SE(ln HR) | Abstract vs Results | — | 0.0316 vs 0.1682 (ratio 5.33) |
| CI width on the HR scale | Abstract vs Results | 0.16 vs 0.52; 0.17 vs 0.98 | ratios 3.25 and 5.76 |
| Event-based SE, severe COVID-19 (138 / 1 104 events) | Supp. 1e | — | 0.0903. Results SE is 1.10 × this; abstract SE is 0.38 × this |
| Event-based SE, COVID-19 mortality (48 / 337 events) | Supp. 1e | — | 0.1543. Results SE is 1.09 × this; abstract SE is 0.20 × this |
| Event-based SE, other upper (1 118 / 14 449) and other lower (1 486 / 14 494) | Table 2 | — | 0.0310 and 0.0272. The abstract SEs (0.0344, 0.0316) are 1.11 × and 1.16 × these |
| Excess hazard, 100 × (HR − 1) | Abstract vs Results | 19% and 37% | 33% and 46% |
| All four intervals exclude HR = 1 | Abstract and Results | — | Yes |
| Separate minor point: COVID-19 mortality vs H2RA users, p from CI | Supp. 1n; Results | 0.83 (0.45–1.56), p = 0.745 | p = 0.557 (0.539–0.574 within 2-decimal rounding); not significant either way |
Four independent features of the paper point to the Results figures and away from the abstract. First, the HR and both CI limits of each abstract value match a different outcome's row exactly. Second, the printed p-values (0.004, 0.024) are those of the Results intervals. Third, the printed severity E-value (1.99) is that of HR 1.33, not of HR 1.19 (1.67, which the paper itself prints for the other upper respiratory row). Fourth, the abstract's interval widths are not attainable with 138 and 48 events. They need SEs of about 0.03, which is what roughly 1,100–1,500 events give, and those are the event counts of the two Table 2 rows they match. The printed mortality E-value (2.03) does not help decide the question. It matches neither HR 1.46 (2.28) nor HR 1.37 (2.08); it corresponds to HR 1.347, slightly nearer the abstract's 1.37 than the Results' 1.46. It is therefore treated as a separate minor inconsistency, not as evidence either way.
Figure
Figure 1. The left panel shows the two COVID-19 estimates printed in the abstract (orange circles). The right panel shows the fully adjusted Table 2 rows for other upper and other lower respiratory infections, and the fully adjusted severe-COVID-19 and COVID-19-mortality estimates in the Results text and Supplementary file 1e (blue squares), each with the E-value printed in the Results. The Table 2 rows carry the same orange marker as the abstract because their numbers are identical to the abstract's. Solid orange connectors join each abstract value to the Table 2 row with identical numbers but a different outcome. Dashed blue connectors join it to the estimate for the same outcome, which has different numbers. The printed severity E-value (1.99) follows the right-hand estimate. The asterisk marks the printed mortality E-value (2.03), which matches neither HR 1.46 (recomputed 2.28) nor HR 1.37 (2.08) and so supports neither figure. All numbers are printed in the paper or by recompute.py.txt. The paper's own figures are not reproduced.
Figure 2. Hazard ratios for regular PPI use, on a log axis with a reference line at HR = 1. For each COVID-19 outcome, the orange circle and interval are the values printed in the abstract and the blue square and interval are the fully adjusted estimate in the Results text and Supplementary file 1e. The intervals in the Results, based on 138 and 48 events among regular PPI users, are 2.9 and 5.3 times wider on the log scale than the ones in the abstract. All four exclude 1, so the error changes the size and the apparent precision of the associations but not their direction or significance.
Why it matters
The abstract is where most readers, and most citing papers and evidence summaries, take the numbers from. For two of its five reported non-user comparisons, it gives effect sizes that belong to different outcomes. The COVID-19 severity and mortality associations are understated (19% vs 33% and 37% vs 46% excess hazard). More consequentially, they look about three to five times more precise than the data allow: the abstract implies a mortality HR known to within 1.29–1.46, when the 48 deaths among regular users support only 1.05–2.03. In a meta-analysis that took the abstract's figures, the inverse-variance weight of this study's mortality estimate would be inflated by a factor of about 28 (5.33²) and that of the severity estimate by about 8 (2.90²). Either estimate would then carry far more weight than the data justify.
The qualitative claims survive. All four intervals exclude 1, so "increased" is correct for both outcomes in both places. The conclusions of the abstract and the Discussion, which give no numbers, are unaffected. The comparison with H2RA users (Supplementary file 1n), on which the abstract bases its statement that COVID-19-related risks "were not evident", is a separate analysis and is not involved. Of the non-user comparisons, the influenza, pneumonia and COVID-19 positivity figures in the abstract match Table 2. The error is confined to the two secondary COVID-19 outcomes.
Explanations considered
- A different model (unadjusted, age/sex-adjusted, fully adjusted). This is ruled out. Supplementary file 1e gives all three models for both COVID-19 outcomes (2.32, 1.95, 1.33 for severity; 2.61, 1.90, 1.46 for mortality). None is 1.19 or 1.37, and none has an interval close to the abstract's.
- A sensitivity analysis (multiple imputation, propensity-score matching, CYP2C19 strata, H2RA comparison). This is ruled out. Tables 1j (1.47 (1.33–1.64); 1.53 (1.27–1.86)), 1m (1.10, 1.25; 1.15, 1.45) and 1n (0.91; 0.83) give no match. The CYP2C19 tables (1g, 1h) have no matching COVID-19 severity or mortality estimate either. The only places in the paper where 1.19 (1.11–1.27) and 1.37 (1.29–1.46) appear are the Table 2 upper and lower respiratory rows and the Results sentence that describes those rows.
- The abstract is right and the Results / Supplementary file 1e are wrong. This does not fit. The Results p-values and the severity E-value are consistent with the Results intervals. The Results SEs are 1.09–1.10 times the event-based approximation from the paper's own case counts (138 and 48), which is what a fully adjusted Cox model would give. The abstract SEs are 0.38 and 0.20 times that approximation, which a model fitted to those events cannot produce. They are 1.11–1.16 times the approximation for the upper and lower respiratory rows, which have 1,118 and 1,486 events.
- "Severe COVID-19" and "mortality" in the abstract mean something broader, such as respiratory infections in general. This does not fit the text. The sentence opens with COVID-19 infection, and the Methods define severe COVID-19 as hospitalisation for COVID-19 and COVID-19 mortality by ICD-10 U07.1/U07.2. The upper and lower respiratory outcomes are defined separately (J00–J06, J20–J22). They are not mentioned anywhere in the abstract.
- Conversion or typesetting artefact. This is ruled out. The same abstract numbers appear in the PMC text, in both eLife Version-of-Record versions (v1, 16 July 2024; v2, 18 July 2024) retrieved from the eLife API, and in all three medRxiv versions of the preprint (22 August 2022, 9 January 2024, 7 May 2024).
Status
As of 2026-10-24:
- Version of record. Both Version-of-Record versions in the eLife API (https://api.elifesciences.org/articles/94973, v1 dated 2024-07-16 and v2 dated 2024-07-18) carry the abstract with "HR 1.19. 95% CI 1.11–1.27" and "HR 1.37. 95% CI 1.29–1.46". Each also carries the Results sentences with "aHR 1.33, 95% CI 1.09" and "aHR 1.46, 95% CI 1.05". The v2 update therefore left the discrepancy in place. The PMC-archived text and Table 2 agree, and every quotation above from the article was checked verbatim against the PMC copy. The medRxiv preprint (doi:10.1101/2022.08.21.22279047), versions 1–3, has the same abstract figures, so the discrepancy goes back to the first preprint.
- Supplement. Supplementary file 1 (https://cdn.elifesciences.org/articles/94973/elife-94973-supp1-v2.docx, tables 1a–1n), the article's only supplementary data file, was downloaded and read. The other additional file listed in the Version of Record is the MDAR checklist (elife-94973-mdarchecklist1-v2.pdf). Table 1e confirms the Results values and contains no estimate equal to the abstract's. Tables 1j, 1m and 1n were also checked (see above). The supplement supports the Results text, so it resolves which figures are correct. The abstract is unchanged.
- Corrections. None found. Crossref update metadata, PMC notices and Europe PMC notices list no correction, erratum, retraction or expression of concern. The public peer reviews and the author response on the eLife site do not mention the discrepancy.
- PubPeer. The PubPeer search page and API refused automated access (HTTP 403 and a "human verification required" response). A web search for the title together with "PubPeer" found no PubPeer thread, only the article and its preprint versions. That there is no PubPeer thread is therefore likely but not confirmed.
- Sources. PMC11251724; https://elifesciences.org/articles/94973; https://api.elifesciences.org/articles/94973 (versions 1 and 2); Supplementary file 1 (elife-94973-supp1-v2.docx); medRxiv doi:10.1101/2022.08.21.22279047 (versions 1–3); https://elifesciences.org/articles/94973/peer-reviews.
Suggested correction
In the Abstract, Results, replace "while the risks of severe COVID-19 (HR 1.19. 95% CI 1.11–1.27) and mortality (HR 1.37. 95% CI 1.29–1.46) were increased" with "while the risks of severe COVID-19 (HR 1.33, 95% CI 1.09–1.61) and COVID-19 mortality (HR 1.46, 95% CI 1.05–2.03) were increased". These are the fully adjusted estimates in the Results and Supplementary file 1e. The authors may also wish to check two minor points. First, the E-value printed for COVID-19 mortality (2.03), which would be 2.28 for HR 1.46. Second, the p-value in Supplementary file 1n for COVID-19 mortality versus H2RA users (printed 0.745; the printed interval 0.45–1.56 implies about 0.56). Neither changes a conclusion. A typographic slip in Table 2 could be fixed at the same time: the influenza interval is printed as "1.32 (1,12–1.56)".
Recomputation script (python recompute.py.txt)
"""Recompute the checks behind the write-up for PMC11251724
(eLife 2024, doi:10.7554/eLife.94973).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
fig1.svg / fig1.png abstract vs Results / Table 2 / Supplementary file 1e
fig2.svg / fig2.png the four hazard ratios and 95% CIs on one log axis
Inputs are copied from the article as archived in PMC (abstract, Results text,
Table 2) and from Supplementary file 1 (elife-94973-supp1-v2.docx, tables 1e,
1j, 1m and 1n).
Method. Each 95% CI is taken to be a Wald interval, symmetric on the log-HR
scale, as Cox output is:
SE = (ln U - ln L) / (2 * 1.959964), z = ln(HR) / SE, P = 2 * (1 - Phi(|z|)).
E-value for a point estimate HR >= 1 (VanderWeele & Ding 2017):
E = HR + sqrt(HR * (HR - 1)).
Approximate SE of a Cox log-HR for a binary exposure from the event counts:
SE ~ sqrt(1/d_exposed + 1/d_unexposed) (information from events; covariate
adjustment usually makes the actual SE larger, not smaller).
"""
import math
import os
import numpy as np
from scipy.optimize import brentq
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ------------------------------------------------------------------ inputs
# Abstract, Results: labelled as COVID-19 outcomes
ABSTRACT = {
"severe COVID-19": (1.19, 1.11, 1.27),
"COVID-19 mortality": (1.37, 1.29, 1.46),
}
# Abstract, Results: the other three non-user comparisons
ABSTRACT_OTHER = {"Influenza": (1.32, 1.12, 1.56), "Pneumonia": (1.42, 1.26, 1.59),
"COVID-19 positivity": (1.08, 0.99, 1.17)}
# Table 2, regular PPI use, fully adjusted model; cases in regular / non-regular users
TABLE2 = {
"Influenza": ((1.32, 1.12, 1.56), 183, 2009),
"Pneumonia": ((1.42, 1.26, 1.59), 378, 2904),
"COVID-19 positivity": ((1.08, 0.99, 1.17), 1440, 23989),
"Other upper respiratory infections": ((1.19, 1.11, 1.27), 1118, 14449),
"Other lower respiratory infections": ((1.37, 1.29, 1.46), 1486, 14494),
}
# Table 2, COVID-19 positivity, all three models (to rule out a model mix-up)
T2_COVIDPOS_ALL_MODELS = [(1.18, 1.09, 1.26), (1.07, 0.99, 1.15), (1.08, 0.99, 1.17)]
# Results text + Supplementary file 1e, regular PPI use, fully adjusted;
# (HR, lo, hi), printed p, printed E-value, cases regular, cases non-regular
RESULTS = {
"severe COVID-19": ((1.33, 1.09, 1.61), 0.004, 1.99, 138, 1104),
"COVID-19 mortality": ((1.46, 1.05, 2.03), 0.024, 2.03, 48, 337),
}
# Supplementary file 1e, other models (non-adjusted, age/sex-adjusted)
S1E_OTHER = {
"severe COVID-19": [(2.32, 1.94, 2.77), (1.95, 1.64, 2.34)],
"COVID-19 mortality": [(2.61, 1.93, 3.54), (1.90, 1.40, 2.58)],
}
# Supplementary files 1j (multiple imputation) and 1m (propensity-score matched)
S1J = {"severe COVID-19": (1.47, 1.33, 1.64), "COVID-19 mortality": (1.53, 1.27, 1.86)}
S1M = {"severe COVID-19": [(1.10, 0.91, 1.34), (1.25, 1.02, 1.52)],
"COVID-19 mortality": [(1.15, 0.84, 1.59), (1.45, 1.04, 2.02)]}
# Supplementary file 1n (vs regular H2RA users): (HR, lo, hi), printed p
S1N = {"severe COVID-19": ((0.91, 0.64, 1.30), 0.608),
"COVID-19 mortality": ((0.83, 0.45, 1.56), 0.745)}
# E-values printed in the Results for other estimates
EVALUES_PRINTED = [("Influenza (Table 2)", 1.32, 1.97), ("Pneumonia (Table 2)", 1.42, 2.19),
("Other upper resp. (Table 2)", 1.19, 1.67),
("Other lower resp. (Table 2)", 1.37, 2.08),
("Influenza vs H2RA (Table 3)", 1.74, 2.87),
("Other upper vs H2RA (Table 3)", 1.28, 1.88),
("Other lower vs H2RA (Table 3)", 1.33, 1.99)]
def se_from_ci(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z)
def p_from_ci(hr, lo, hi):
se = se_from_ci(lo, hi)
return 2 * norm.sf(abs(math.log(hr) / se))
def evalue(hr):
hr = hr if hr >= 1 else 1 / hr
return hr + math.sqrt(hr * (hr - 1))
def fmt(t):
return f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})"
# ------------------------------------------------------------------ 1. exact matches
print("1. Abstract 'COVID-19' estimates vs every fully adjusted Table 2 row")
MATCH = {}
for k, v in ABSTRACT.items():
m = [row for row, (t, _, _) in TABLE2.items() if t == v]
MATCH[k] = m
print(f" abstract {k:20s} {fmt(v)} -> identical (HR and both limits) to: {m}")
for k, v in ABSTRACT.items():
r = RESULTS[k][0]
print(f" abstract {k:20s} {fmt(v)} vs Results/Supp 1e {fmt(r)} identical: {v == r}")
other = [t for lst in S1E_OTHER.values() for t in lst] + list(S1J.values()) \
+ [t for lst in S1M.values() for t in lst] + [t for t, _ in S1N.values()] \
+ T2_COVIDPOS_ALL_MODELS
hits = [t for t in other if t in ABSTRACT.values()]
print(f" any other reported COVID-19 model (1e non/age-sex adjusted, 1j, 1m, 1n, "
f"Table 2 COVID positivity x3) equal to an abstract value: {hits if hits else 'none'}")
print(" other abstract estimates equal to their Table 2 rows: " + ", ".join(
f"{k} {ABSTRACT_OTHER[k] == TABLE2[k][0]}" for k in ABSTRACT_OTHER))
# ------------------------------------------------------------------ 2. Results internal consistency
print("\n2. Results / Supp 1e COVID-19 estimates: p and E-value recomputed from the printed numbers")
REC = {}
for k, ((hr, lo, hi), p_pr, e_pr, d1, d0) in RESULTS.items():
se = se_from_ci(lo, hi)
p = p_from_ci(hr, lo, hi)
gm = math.sqrt(lo * hi)
e = evalue(hr)
REC[k] = dict(se=se, p=p, e=e)
print(f" {k:20s} {fmt((hr, lo, hi))}: SE(lnHR) {se:.4f}, CI geometric midpoint {gm:.3f}, "
f"p {p:.4f} (printed {p_pr}), E-value {e:.2f} (printed {e_pr})")
E203 = brentq(lambda h: evalue(h) - 2.03, 1.0001, 3)
print(f" HR whose E-value is 2.03: {E203:.3f}")
print(f" E-value of the mortality lower CI limit 1.05: {evalue(1.05):.2f}")
# ------------------------------------------------------------------ 3. E-values elsewhere
print("\n3. E-values printed elsewhere in the Results")
for lab, hr, e_pr in EVALUES_PRINTED:
print(f" {lab:32s} HR {hr:.2f}: E {evalue(hr):.2f} (printed {e_pr})")
print(" E-values of the abstract's two numbers: "
+ ", ".join(f"HR {v[0]:.2f} -> {evalue(v[0]):.2f}" for v in ABSTRACT.values()))
# ------------------------------------------------------------------ 4. precision
print("\n4. Precision implied by each interval vs the number of events behind it")
SEROWS = []
for k, v in ABSTRACT.items():
se_a = se_from_ci(v[1], v[2])
(hr, lo, hi), _, _, d1, d0 = RESULTS[k]
se_r = se_from_ci(lo, hi)
se_ev = math.sqrt(1 / d1 + 1 / d0)
print(f" {k:20s} abstract SE {se_a:.4f} | Results SE {se_r:.4f} | ratio {se_r / se_a:.2f}"
f" | events {d1}/{d0}, sqrt(1/d1+1/d0) = {se_ev:.4f}"
f" | abstract SE / event-based SE = {se_a / se_ev:.2f}; Results SE / event-based SE = {se_r / se_ev:.2f}")
print(f" {'':20s} CI width on HR scale: abstract {v[2] - v[1]:.2f}, Results {hi - lo:.2f}, "
f"ratio {(hi - lo) / (v[2] - v[1]):.2f}")
print(f" {'':20s} inverse-variance weight inflation if the abstract SE were used: "
f"{(se_r / se_a) ** 2:.1f}x")
SEROWS.append((k, se_a, se_r, se_ev))
for row in ("Other upper respiratory infections", "Other lower respiratory infections"):
(hr, lo, hi), d1, d0 = TABLE2[row]
se = se_from_ci(lo, hi)
se_ev = math.sqrt(1 / d1 + 1 / d0)
print(f" {row:36s} Table 2 SE {se:.4f} | events {d1}/{d0}, sqrt(1/d1+1/d0) = {se_ev:.4f}"
f" | ratio {se / se_ev:.2f}")
# ------------------------------------------------------------------ 5. magnitudes
print("\n5. Excess hazard, 100 x (HR - 1)")
for k, v in ABSTRACT.items():
r = RESULTS[k][0]
print(f" {k:20s} abstract {100 * (v[0] - 1):.0f}% vs Results {100 * (r[0] - 1):.0f}%;"
f" all four CIs exclude 1: {v[1] > 1 and r[1] > 1}")
# ------------------------------------------------------------------ 6. a separate minor point (Supp 1n)
print("\n6. Supplementary file 1n (PPI vs H2RA), p recomputed from the printed CI")
for k, ((hr, lo, hi), p_pr) in S1N.items():
p = p_from_ci(hr, lo, hi)
ps = [p_from_ci(h, l, u) for h in (hr - .005, hr + .005) for l in (lo - .005, lo + .005)
for u in (hi - .005, hi + .005)]
print(f" {k:20s} {fmt((hr, lo, hi))}: p {p:.3f} (range within 2-decimal rounding "
f"{min(ps):.3f}-{max(ps):.3f}); printed {p_pr}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
def box(ax, x, y, w, h):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=INK2, lw=1.2, transform=ax.transAxes))
# ---- Figure 1: text panels
fig = plt.figure(figsize=(7.2, 5.4), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off"); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
box(ax, 0.02, 0.37, 0.33, 0.57)
box(ax, 0.53, 0.15, 0.45, 0.79)
ax.text(0.04, 0.90, "Abstract, Results", fontsize=11, weight="bold", va="top")
ax.text(0.04, 0.83, "“… while the risks of\nsevere COVID-19 … and\nmortality … were increased”",
fontsize=10, style="italic", va="top", linespacing=1.35)
YA = {"severe COVID-19": 0.62, "COVID-19 mortality": 0.48}
for k, y in YA.items():
ax.text(0.04, y + 0.03, "Severe COVID-19" if k.startswith("severe") else "COVID-19 mortality",
fontsize=10, va="center")
ax.plot([0.045], [y - 0.025], "o", ms=8, color=ORANGE, transform=ax.transAxes)
ax.text(0.07, y - 0.025, f"HR {fmt(ABSTRACT[k])}", fontsize=10, va="center", color=INK)
ax.text(0.55, 0.90, "Results text, Table 2 and\nSupplementary file 1e", fontsize=11,
weight="bold", va="top", linespacing=1.2)
ax.text(0.55, 0.785, "fully adjusted HR (95% CI)", fontsize=10, color=INK2, va="center")
ax.text(0.955, 0.785, "printed\nE-value", fontsize=10, color=INK2, va="center", ha="right",
linespacing=1.1)
ax.plot([0.55, 0.96], [0.745, 0.745], color=GRID, lw=1, transform=ax.transAxes)
ROWS = [ # label, source, value, E printed, y, marker colour, key in ABSTRACT matched
("Other upper resp. inf.", "Table 2", TABLE2["Other upper respiratory infections"][0],
"1.67", 0.685, ORANGE),
("Other lower resp. inf.", "Table 2", TABLE2["Other lower respiratory infections"][0],
"2.08", 0.575, ORANGE),
("Severe COVID-19", "Results, Supp. 1e", RESULTS["severe COVID-19"][0], "1.99", 0.445, BLUE),
("COVID-19 mortality", "Results, Supp. 1e", RESULTS["COVID-19 mortality"][0], "2.03*", 0.33,
BLUE),
]
for lab, src, v, e, y, c in ROWS:
ax.text(0.55, y + 0.03, f"{lab} · {src}", fontsize=10, va="center")
ax.plot([0.555], [y - 0.025], "o" if c == ORANGE else "s", ms=8, color=c,
transform=ax.transAxes)
ax.text(0.58, y - 0.025, fmt(v), fontsize=10, va="center", color=INK)
ax.text(0.955, y - 0.025, e, fontsize=10, va="center", ha="right")
ax.plot([0.55, 0.96], [0.515, 0.515], color=GRID, lw=1, transform=ax.transAxes)
ax.text(0.55, 0.225, f"* recomputed: {REC['COVID-19 mortality']['e']:.2f} for HR 1.46, "
f"{evalue(ABSTRACT['COVID-19 mortality'][0]):.2f} for\n HR 1.37; 2.03 is the E-value of HR {E203:.2f},"
f"\n so it supports neither figure", fontsize=10, color=INK2, va="center",
linespacing=1.25)
# connectors: solid = identical numbers, dashed = same outcome
for k, (yr_same_num, yr_same_out) in {"severe COVID-19": (0.685, 0.445),
"COVID-19 mortality": (0.575, 0.33)}.items():
ya = YA[k] - 0.025
ax.add_patch(FancyArrowPatch((0.355, ya), (0.54, yr_same_num - 0.025), arrowstyle="-|>",
mutation_scale=11, lw=2, color=ORANGE,
transform=ax.transAxes, shrinkA=0, shrinkB=2))
ax.add_patch(FancyArrowPatch((0.355, ya), (0.54, yr_same_out - 0.025), arrowstyle="-|>",
mutation_scale=11, lw=2, color=BLUE, ls=(0, (3, 2)),
transform=ax.transAxes, shrinkA=0, shrinkB=2))
ax.text(0.02, 0.35, "Conflict: the abstract's COVID-19\n"
"estimates are identical, HR and both\n"
"limits, to the Table 2 rows for other\n"
"upper and lower respiratory infections",
fontsize=10, va="top", linespacing=1.3, weight="bold")
ax.legend(handles=[Line2D([], [], color=ORANGE, marker="o", ms=8, ls="none",
label="numbers printed in the abstract\n(= Table 2 upper/lower resp. rows)"),
Line2D([], [], color=BLUE, marker="s", ms=8, ls="none",
label="COVID-19 estimates in Results / Supp. 1e"),
Line2D([], [], color=ORANGE, lw=2, label="same numbers, different outcome"),
Line2D([], [], color=BLUE, lw=2, ls=(0, (3, 2)),
label="same outcome, different numbers")],
loc="lower left", bbox_to_anchor=(0.015, 0.0), frameon=False, fontsize=10,
handlelength=2.6, ncol=2, columnspacing=1.5)
save(fig, "fig1")
# ---- Figure 2: forest-style comparison on one log axis
items = [
("Severe COVID-19\nas printed in the abstract", ABSTRACT["severe COVID-19"], ORANGE, "o"),
("Severe COVID-19\nResults text / Supp. 1e", RESULTS["severe COVID-19"][0], BLUE, "s"),
("COVID-19 mortality\nas printed in the abstract", ABSTRACT["COVID-19 mortality"], ORANGE, "o"),
("COVID-19 mortality\nResults text / Supp. 1e", RESULTS["COVID-19 mortality"][0], BLUE, "s"),
]
events = [None, 138, None, 48]
fig, ax = plt.subplots(figsize=(7.2, 3.6), facecolor=BG)
fig.subplots_adjust(left=0.30, right=0.80, top=0.93, bottom=0.27)
ax.set_facecolor(BG)
ys = [3.6, 2.9, 1.1, 0.4]
for (lab, (hr, lo, hi), c, mk), y in zip(items, ys):
ax.plot([lo, hi], [y, y], color=c, lw=2, solid_capstyle="butt")
ax.plot([hr], [y], mk, ms=8, color=c)
ax.text(2.25, y, fmt((hr, lo, hi)), va="center", fontsize=10, color=INK,
transform=ax.transData, clip_on=False)
ax.axvline(1, color=INK2, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.95, 2.2)
ticks = [1.0, 1.2, 1.4, 1.6, 1.8, 2.0]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:.1f}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys); ax.set_yticklabels([i[0] for i in items], fontsize=10)
ax.set_ylim(-0.1, 4.1)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Hazard ratio, regular PPI use vs non-regular (log scale)", fontsize=10)
ax.text(2.25, 4.05, "HR (95% CI)", fontsize=10, color=INK2, va="center", clip_on=False)
ax.text(1.005, 4.05, "HR = 1", fontsize=10, color=INK2, va="center")
ax.legend(handles=[Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2,
label="abstract (values of the Table 2 upper/lower respiratory rows)"),
Line2D([], [], color=BLUE, marker="s", ms=8, lw=2,
label="Results text and Supplementary file 1e (138 and 48 events)")],
loc="upper center", bbox_to_anchor=(0.25, -0.2), frameon=False, fontsize=10, ncol=1)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png in", HERE)
Abstract says Serpin A1c rose in all three tissues; the paper's data show liver only
eLife (2026) · doi:10.7554/eLife.107332 · PMC13090031
Summary
The abstract says that under intermittent fasting "Serpin A1c emerged as the only protein commonly upregulated across all three tissues." The paper's Results and Conclusion, the Figure 3B inset table and both supplementary data files report something else: Serpin A1c goes up in liver and down in skeletal muscle and cerebral cortex. In the per-animal values of Supplementary file 3 the IF16 minus AL differences are +0.76, −0.51 and −0.55 log2 units. Supplementary file 1 reports limma logFCs of +0.78, −0.47 and −0.55. Serpin A1c is the only protein differentially expressed in all three tissues, but it is upregulated in one of them, not three. The abstract presents it as a shared, systemic response, which is the opposite of the paper's own reading in the Results, where the pattern indicates "a tissue-specific regulatory role".
What the paper reports
Abstract:
Notably, Serpin A1c emerged as the only protein commonly upregulated across all three tissues, highlighting its potential role in systemic adaptation to IF.
Results and discussion, section "Identification of shared and tissue-specific biological processes", first paragraph:
Our analysis revealed that Serpin A1c was the only protein commonly dysregulated across all three organs, suggesting a potentially central role for this protein in the systemic response to IF. The observation that Serpin A1c levels increase in the liver but decrease in muscle and cerebral cortex in response to IF suggests a tissue-specific regulatory role for this protein during metabolic adaptation to IF (Figure 3C, Supplementary file 3).
Same paragraph:
In the liver, the upregulation of Serpin A1c, a serine protease inhibitor, may reflect a protective mechanism to modulate inflammation or protein degradation as the body shifts toward fat metabolism and ketogenesis during fasting (Okagawa et al., 2024). Conversely, its reduction in muscle and cerebral cortex could indicate a reduced need for protease inhibition in these tissues, possibly to facilitate protein breakdown for energy or to utilize resources under fasting conditions (Sanfeliu et al., 2019; Pattamaprapanont et al., 2024).
The results confirmed that IF upregulates the expression levels of Serpin A1c in the liver (Figure 3D).
Conclusion, second paragraph:
Notably, our findings reveal that while some molecular responses to IF are conserved across tissues, such as the upregulation of Serpin A1c in the liver and its downregulation in muscle and cortex, others are highly tissue-specific.
Figure 3 legend:
(C) Expression levels of Serpin A1c in the liver, muscle, and cerebral cortex. (D) Alpha1-antitrypsin staining in green in the liver of both AL and IF mice. n=3 mice in each group.
The Figure 3B inset table, read from the figure image, lists Q00896 / SERPINA1C as "Up in liver", "Down in muscle" and "Down in cortex".
What the paper's own numbers show
Method. I took three data sources from the paper. (1) The per-animal log2 normalized Serpin A1c (UniProt Q00896) intensities in Supplementary file 3, sheet "Liver+Muscle+Cortex_Common DEP": n = 7 per group, labelled only "Group1" and "Group2". (2) The raw label-free intensities for the same protein in Supplementary file 1 (sheets Liver_raw, Muscle_raw, Cortex_raw), where each sample is labelled "AL" or "IF16". (3) The reported logFC and P value for Q00896 in the Supplementary file 1 DE sheets. I identified Group1 as AL by correlating its values with the log2 raw AL intensities (r = 0.999, 0.986 and 0.984 in liver, muscle and cortex). For each tissue I then computed the IF16 minus AL mean difference with a Welch 95% CI and p value. These two-group tests check direction only. They are not a re-run of the paper's limma (FlexStat) pipeline. Finally, I checked that the Figure 3A and 3B Venn segments add up to the stated totals. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Tissues in which Serpin A1c is upregulated | Abstract | 3 of 3 ("commonly upregulated across all three tissues") | 1 of 3 (liver) |
| Serpin A1c direction (liver / muscle / cortex) | Results p. 1; Conclusion; Fig 3B inset table | up / down / down | up / down / down |
| Serpin A1c, liver, IF16 − AL | Supp. file 1 Liver_DE; Supp. file 3 | logFC +0.7837, P = 0.0175 | mean 24.600 → 25.360, +0.760 [95% CI +0.217, +1.303], Welch p = 0.012; raw log2 +0.699 |
| Serpin A1c, skeletal muscle, IF16 − AL | Supp. file 1 Muscle_DE; Supp. file 3 | logFC −0.4667, P = 0.0169 | 23.570 → 23.062, −0.508 [−0.938, −0.078], Welch p = 0.025; raw log2 −0.395 |
| Serpin A1c, cerebral cortex, IF16 − AL | Supp. file 1 Cortex_DE; Supp. file 3 | logFC −0.5507, P = 0.0137 | 21.699 → 21.149, −0.551 [−1.032, −0.069], Welch p = 0.030; raw log2 −0.621 |
| Group medians, AL → IF16 (liver / muscle / cortex) | Fig 3C (box plots); Supp. file 3 | shown graphically | 24.67 → 25.39 / 23.55 → 22.93 / 21.80 → 21.21 |
| DEPs shared by all three tissues | Fig 3B; Results p. 1 | 1 (Serpin A1c, "dysregulated") | 1, consistent |
| Proteins per tissue (liver / muscle / cortex) | Fig 3A | 4777 / 2709 / 6332 | Venn segments sum to 4777 / 2709 / 6332 |
| DEPs per tissue (liver / muscle / cortex) | Fig 3B | 835 / 155 / 294 | Venn segments sum to 835 / 155 / 294 |
The three sources agree on the sign in every tissue. Normalized means, raw intensities and the reported limma logFC all give a rise in liver and a fall in muscle and cortex. The abstract's "upregulated across all three tissues" disagrees with all of them in two of the three tissues.
Figure
Figure 1. The abstract sentence (left) and the Results and Conclusion sentences on the same protein (right), quoted verbatim from the article as archived in PMC ("[...]" marks omitted text). The table below gives the direction of Serpin A1c change, IF16 versus AL, as each part of the paper states it. The orange values are the logFCs reported in Supplementary file 1. The blue values are mean differences recomputed by recompute.py.txt from the per-animal values in Supplementary file 3. The abstract is the only source that puts muscle and cortex as "up".
Figure 2. Serpin A1c, IF16 minus AL, in log2 intensity units for each tissue. Orange diamonds are the limma logFCs reported in Supplementary file 1. Blue circles are the mean differences recomputed from the 7 + 7 per-animal values in Supplementary file 3, with Welch 95% confidence intervals. The vertical line marks no change. The shaded region to the right is where the abstract's "upregulated across all three tissues" would place every tissue. Only liver lies there. The muscle and cortex estimates and their whole intervals lie below zero.
Why it matters
Serpin A1c is the only protein from the proteomic profiling that the abstract names, and the abstract uses it as its one concrete example of a shared, "systemic adaptation to IF". It also stands behind the closing claim of "both conserved and tissue-specific responses". As written, readers of the abstract and indexing services are told that one protein rises in liver, muscle and brain under intermittent fasting, which would make it a candidate systemic marker. The paper's data and its own Results describe the opposite pattern. The protein is significantly changed in all three tissues, but it rises in liver and falls in muscle and cortex, and the Results use this to argue for "a tissue-specific regulatory role". The orthogonal validation (immunohistochemistry, Fig 3D, n = 3 per group) covers liver only, so it cannot support a rise in the other two tissues.
The Conclusion has a related wording problem. It gives "the upregulation of Serpin A1c in the liver and its downregulation in muscle and cortex" as an example of a response "conserved across tissues". It reports the directions correctly, but a change in opposite directions is not usually described as conserved.
What is not affected: the metabolic phenotype (glucose, HbA1c, cholesterol, ketone bodies), the per-tissue differential expression and pathway enrichment results, the Venn counts in Figure 3A and 3B, the RNA–protein concordance analysis, and the shared "synaptic vesicle localization" process. The error is confined to the direction the abstract states for one protein in two tissues. The body text, figure and supplements are internally consistent.
Explanations considered
"Upregulated" is used loosely to mean "changed". The Results use the neutral word "dysregulated" for the same finding, which would have been accurate. But "commonly upregulated across all three tissues" states a direction, and the same paper uses "upregulation" and "downregulation" for opposite directions of this very protein in the Results and Conclusion. The abstract's wording cannot be read as direction-neutral.
A different measure, tissue or time point. All three tissues come from one comparison: IF16 versus ad libitum (AL), after 4 months, in male C57BL/6 mice, n = 7 per group. The paper reports no other Serpin A1c protein measurement in muscle or cortex. The immunohistochemistry covers liver only.
Transcript rather than protein. The abstract sentence is about protein ("the only protein"). The transcriptomic sections do not report Serpina1c rising in muscle or cortex. The only serpin they name for cortex is Serpina1e, which is a different gene and is described as downregulated.
Group labels swapped in Supplementary file 3. That file labels the groups only "Group1" and "Group2". The values of Group1 track the log2 raw intensities that Supplementary file 1 labels AL (r ≥ 0.98 in every tissue), and Supplementary file 1's own DE sheets, labelled IF16 versus AL, give the same signs. A swap would also flip liver to "down", contradicting the paper's liver immunohistochemistry.
Choice of statistical test. The sign of a mean difference does not depend on the test. Normalized means, raw intensities and the reported limma estimates all give the same three signs, and the muscle and cortex confidence intervals exclude zero.
Text conversion or archiving error. Not possible here. The conflict is in words ("upregulated across all three tissues"), not in symbols or numbers, and the identical sentence appears in the publisher's version-of-record HTML and PDF, in Europe PMC and in PMC.
Status
As of 2026-10-24:
- Version of record. eLife version of record doi:10.7554/eLife.107332.3, published 17 April 2026, after reviewed preprints v1 (7 October 2025) and v2 (20 March 2026). The publisher's HTML page and PDF (text extracted) have the same abstract, Results and Conclusion wording as the PMC archive and Europe PMC.
- Supplements. Supplementary files 1 and 3 (elife-107332-supp1.xlsx, supp3.xlsx) were downloaded through Europe PMC and read in Python. Their Serpin A1c values are those listed above, and the values used are copied into
recompute.py.txt. - Corrections. Crossref lists no update-to or updated-by relations for this DOI, only the preprint, version and dataset links (PXD063124, GSE290224). PMC and Europe PMC list no correction, erratum, expression of concern or retraction. The eLife page shows no notice.
- Peer review. The public reviews do not discuss the direction of the Serpin A1c change. One author response describes Serpin A1C as "a key protein common to all three organs" when introducing the liver immunohistochemistry.
- PubPeer. Web searches found no PubPeer thread or other public discussion of this point. PubPeer itself could not be searched directly because it returned a human-verification page, so a thread there cannot be ruled out.
Sources: the article as archived in PMC (PMC13090031); elifesciences.org/articles/107332 and its PDF; elifesciences.org/reviewed-preprints/107332/reviews; Supplementary files 1 and 3; Crossref, PMC and Europe PMC records for doi:10.7554/eLife.107332.
Suggested correction
In the abstract, replace
Notably, Serpin A1c emerged as the only protein commonly upregulated across all three tissues, highlighting its potential role in systemic adaptation to IF.
with wording that matches the Results and supplementary data, for example:
Notably, Serpin A1c emerged as the only protein differentially expressed in all three tissues, increasing in the liver but decreasing in muscle and cerebral cortex, suggesting tissue-specific regulation during adaptation to IF.
In the Conclusion, the opposite-direction Serpin A1c pattern would be better given as an example of a response shared across tissues in occurrence but tissue-specific in direction, rather than as a response "conserved across tissues".
Recomputation script (python recompute.py.txt)
"""Recompute the Serpin A1c direction-of-change result of PMC13090031
(eLife 2026, doi:10.7554/eLife.107332).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper's supplementary files (eLife / Europe PMC):
Supplementary file 3 (elife-107332-supp3.xlsx), sheet "Liver+Muscle+Cortex_Common DEP":
per-animal log2 normalized intensities of SERPINA1C (UniProt Q00896), Group1 and
Group2, n = 7 each, for liver, muscle and cortex.
Supplementary file 1 (elife-107332-supp1.xlsx):
sheets Liver_raw / Muscle_raw / Cortex_raw: raw label-free intensities of Q00896,
with the group of each sample labelled "AL" or "IF16" in the second header row;
sheets Liver_DE / Muscle_DE / Cortex_DE: the reported logFC and P.Value for Q00896.
Figure 3A/3B Venn segment counts, read from the figure.
Text excerpts: Abstract, Results ("Identification of shared and tissue-specific
biological processes", first paragraph) and Conclusion, as archived in PMC.
Supplementary file 3 labels its groups only "Group1"/"Group2". Group1 is identified as
AL below by matching its values against the log2 raw intensities that Supplementary
file 1 labels AL (same sample order, numbers 1, 3-8).
The paper's Methods describe limma (FlexStat) for differential expression; the t tests
below are simple two-group checks of direction, not a re-run of that pipeline.
"""
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
TISSUES = ["Liver", "Muscle", "Cortex"]
LABEL = {"Liver": "Liver", "Muscle": "Skeletal muscle", "Cortex": "Cerebral cortex"}
# ---------------------------------------------------------------- inputs (from the paper)
# Supplementary file 3, "Liver+Muscle+Cortex_Common DEP": log2 normalized intensity
SUPP3_G1 = { # Group1_1, _3, _4, _5, _6, _7, _8
"Liver": [24.24397, 24.08873, 24.6831, 23.93125, 24.97634, 24.67132, 25.60579],
"Muscle": [23.68303985, 23.54329062, 24.11543772, 23.55292401, 23.15937914,
23.32285014, 23.61199835],
"Cortex": [21.90583486, 21.42777107, 21.80104356, 21.8559396, 21.95307153,
21.54484944, 21.40764559],
}
SUPP3_G2 = { # Group2_2 ... _8
"Liver": [25.45565, 25.32452, 25.0918, 25.63992, 25.38545, 25.0458, 25.57684],
"Muscle": [22.92942279, 23.60975694, 23.30590172, 23.51469058, 22.53804146,
22.88256333, 22.65270119],
"Cortex": [21.8905763, 21.30969849, 21.08189548, 21.2120679, 20.58894065,
20.43997248, 21.51816713],
}
# Supplementary file 1, *_raw sheets, row Q00896 (Serpina1c); first 7 = AL, last 7 = IF16
SUPP1_RAW = {
"Liver": [19800000, 17300000, 26600000, 15500000, 34700000, 26500000, 51700000,
33300000, 42300000, 36700000, 52500000, 42900000, 34700000, 49400000],
"Muscle": [13300000, 11700000, 18800000, 11400000, 9280000, 10500000, 13300000,
11300000, 12100000, 9740000, 11500000, 7690000, 8530000, 6320000],
"Cortex": [3950000, 2810000, 3680000, 3820000, 3790000, 3110000, 2740000,
2490000, 2630000, 2260000, 2460000, 1580000, 1460000, 2950000],
}
# Supplementary file 1, *_DE sheets, row Q00896: (logFC, P.Value), IF16 vs AL
SUPP1_DE = {"Liver": (0.7837, 0.0175), "Muscle": (-0.4667, 0.0169),
"Cortex": (-0.5507, 0.0137)}
# What each part of the paper says about the direction (IF16 vs AL)
SAYS = {
"Abstract": {"Liver": "up", "Muscle": "up", "Cortex": "up"},
"Results p0": {"Liver": "up", "Muscle": "down", "Cortex": "down"},
"Conclusion p1": {"Liver": "up", "Muscle": "down", "Cortex": "down"},
"Fig 3B table": {"Liver": "up", "Muscle": "down", "Cortex": "down"},
}
# Figure 3A (all proteins) and 3B (DEPs) Venn segments
VENN_A = {"L only": 1125, "M only": 219, "C only": 2385, "L&M": 99, "L&C": 1556,
"M&C": 394, "L&M&C": 1997}
VENN_A_TOT = {"Liver": 4777, "Muscle": 2709, "Cortex": 6332}
VENN_B = {"L only": 776, "M only": 117, "C only": 248, "L&M": 25, "L&C": 33,
"M&C": 12, "L&M&C": 1}
VENN_B_TOT = {"Liver": 835, "Muscle": 155, "Cortex": 294}
ABSTRACT = ("Notably, Serpin A1c emerged as the only protein commonly upregulated "
"across all three tissues, highlighting its potential role in systemic "
"adaptation to IF.")
RESULTS = ("Our analysis revealed that Serpin A1c was the only protein commonly "
"dysregulated across all three organs [...] The observation that Serpin A1c "
"levels increase in the liver but decrease in muscle and cerebral cortex in "
"response to IF suggests a tissue-specific regulatory role [...]")
CONCLUSION = ("[...] such as the upregulation of Serpin A1c in the liver and its "
"downregulation in muscle and cortex [...]")
# ---------------------------------------------------------------- style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
REP = "#eb6834" # values as reported by the paper
REC = "#2a78d6" # values recomputed from the paper's own numbers
GRID = "#e4e3df"
PANEL = "#f3f2ee"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "axes.edgecolor": INK2,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "svg.fonttype": "none", "lines.linewidth": 2,
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n))
def direction(x):
return "up" if x > 0 else "down"
# ---------------------------------------------------------------- recomputation
print("1. Which Supplementary file 3 group is AL?")
for t in TISSUES:
raw = np.log2(np.array(SUPP1_RAW[t], float))
al, if16 = raw[:7], raw[7:]
g1, g2 = np.array(SUPP3_G1[t]), np.array(SUPP3_G2[t])
r1 = np.corrcoef(g1, al)[0, 1]
print(f" {t:7s} corr(Group1, log2 raw AL) = {r1:.3f}; mean Group1 {g1.mean():.3f} vs "
f"log2 raw AL {al.mean():.3f}; mean Group2 {g2.mean():.3f} vs log2 raw IF16 "
f"{if16.mean():.3f}")
print("\n2. Serpin A1c (Q00896), IF16 minus AL, three sources")
print(f" {'tissue':7s} {'S3 AL':>7s} {'S3 IF16':>8s} {'S3 diff':>8s} {'95% CI (Welch)':>17s}"
f" {'Welch p':>8s} {'t p':>7s} | {'raw diff':>8s} {'Welch p':>8s} | "
f"{'S1 logFC':>8s} {'S1 P':>7s}")
REC_D = {}
for t in TISSUES:
g1, g2 = np.array(SUPP3_G1[t]), np.array(SUPP3_G2[t])
d = g2.mean() - g1.mean()
v1, v2 = g1.var(ddof=1) / 7, g2.var(ddof=1) / 7
se = np.sqrt(v1 + v2)
df = (v1 + v2) ** 2 / (v1 ** 2 / 6 + v2 ** 2 / 6)
q = stats.t.ppf(0.975, df)
lo, hi = d - q * se, d + q * se
pw = stats.ttest_ind(g2, g1, equal_var=False).pvalue
ps = stats.ttest_ind(g2, g1).pvalue
raw = np.log2(np.array(SUPP1_RAW[t], float))
dr = raw[7:].mean() - raw[:7].mean()
prw = stats.ttest_ind(raw[7:], raw[:7], equal_var=False).pvalue
fc, p = SUPP1_DE[t]
REC_D[t] = (d, lo, hi)
print(f" {t:7s} {g1.mean():7.3f} {g2.mean():8.3f} {d:+8.3f} [{lo:+.3f}, {hi:+.3f}]"
f" {pw:8.4f} {ps:7.4f} | {dr:+8.3f} {prw:8.4f} | {fc:+8.4f} {p:7.4f}")
print(f" medians: AL {np.median(g1):.3f}, IF16 {np.median(g2):.3f}; "
f"range AL {g1.min():.2f} to {g1.max():.2f}, "
f"IF16 {g2.min():.2f} to {g2.max():.2f}")
print("\n3. Direction by source")
src = dict(SAYS)
src["Supp 3 mean diff"] = {t: direction(REC_D[t][0]) for t in TISSUES}
src["Supp 1 logFC"] = {t: direction(SUPP1_DE[t][0]) for t in TISSUES}
print(f" {'source':18s}" + "".join(f"{t:>8s}" for t in TISSUES) + " n up")
for s, dct in src.items():
print(f" {s:18s}" + "".join(f"{dct[t]:>8s}" for t in TISSUES)
+ f" {sum(v == 'up' for v in dct.values())} of 3")
print(" Abstract agrees with Supp 1 logFC in every tissue:",
all(SAYS["Abstract"][t] == src["Supp 1 logFC"][t] for t in TISSUES))
print("\n4. Venn arithmetic")
for name, V, T in (("Fig 3A proteins", VENN_A, VENN_A_TOT), ("Fig 3B DEPs", VENN_B, VENN_B_TOT)):
sums = {"Liver": V["L only"] + V["L&M"] + V["L&C"] + V["L&M&C"],
"Muscle": V["M only"] + V["L&M"] + V["M&C"] + V["L&M&C"],
"Cortex": V["C only"] + V["L&C"] + V["M&C"] + V["L&M&C"]}
for t in TISSUES:
print(f" {name}: {t:7s} segments sum {sums[t]:5d} (reported {T[t]})")
print(f" {name}: in all three tissues = {V['L&M&C']}")
# ================================================================ figure 1: text panels
fig = plt.figure(figsize=(W, 6.4), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
top, bot = 0.975, 0.445
L, R = (0.025, 0.375), (0.555, 0.975)
for (x0, x1), edge in ((L, REP), (R, INK2)):
ax.add_patch(FancyBboxPatch((x0, bot), x1 - x0, top - bot,
boxstyle="round,pad=0,rounding_size=0.012",
fc=PANEL, ec=edge, lw=1.6))
ax.text(L[0] + 0.02, top - 0.025, "Abstract", fontsize=11.5, weight="bold", va="top")
ax.text(L[0] + 0.02, top - 0.075, wrap("\u201c" + ABSTRACT + "\u201d", 30), fontsize=10,
va="top", linespacing=1.35)
ax.text(L[0] + 0.02, bot + 0.035, "Direction stated:\nup in all 3 tissues", fontsize=10,
color=REP, weight="bold", linespacing=1.3)
ax.text(R[0] + 0.02, top - 0.025, "Results", fontsize=11.5, weight="bold", va="top")
ax.text(R[0] + 0.02, top - 0.065, wrap("\u201c" + RESULTS + "\u201d", 38), fontsize=10,
va="top", linespacing=1.3)
ax.text(R[0] + 0.02, bot + 0.20, "Conclusion", fontsize=11.5, weight="bold", va="top")
ax.text(R[0] + 0.02, bot + 0.16, wrap("\u201c" + CONCLUSION + "\u201d", 38),
fontsize=10, va="top", linespacing=1.3)
ax.add_patch(FancyArrowPatch((L[1] + 0.01, 0.73), (R[0] - 0.01, 0.73),
arrowstyle="<->", mutation_scale=16, lw=2, color=INK))
ax.text(0.465, 0.765, "Conflict", ha="center", fontsize=10.5, weight="bold")
ax.text(0.465, 0.71, "direction in\nmuscle and\ncortex", ha="center", va="top",
fontsize=10, color=INK2, linespacing=1.3)
# direction strip
y0 = 0.375
ax.text(0.025, y0 + 0.03, "Serpin A1c, IF16 vs AL (log2 difference)", fontsize=10.5,
weight="bold")
cols = [0.40, 0.62, 0.84]
for c, t in zip(cols, TISSUES):
ax.text(c, y0 - 0.02, LABEL[t], ha="center", fontsize=10, weight="bold")
rows = [
("Abstract", {t: "up" for t in TISSUES}, None, REP),
("Results, Conclusion,\nFig 3B inset table", SAYS["Results p0"], None, INK2),
("Supp. file 1 logFC\n(reported, limma)", None, {t: SUPP1_DE[t][0] for t in TISSUES}, REP),
("Supp. file 3 animals\n(recomputed mean diff.)", None, {t: REC_D[t][0] for t in TISSUES}, REC),
]
for i, (name, dirs, vals, col) in enumerate(rows):
yy = y0 - 0.085 - i * 0.073
ax.plot([0.025, 0.975], [yy + 0.034, yy + 0.034], color=GRID, lw=1)
ax.text(0.025, yy, name, fontsize=10, va="center", color=INK, linespacing=1.15)
for c, t in zip(cols, TISSUES):
if dirs is not None:
s = "↑ up" if dirs[t] == "up" else "↓ down"
else:
v = vals[t]
s = ("↑ " if v > 0 else "↓ ") + f"{v:+.2f}".replace("-", "−")
ax.text(c, yy, s, ha="center", va="center", fontsize=10.5, color=col,
weight="bold")
save(fig, "fig1")
# ================================================================ figure 2: effect plot
fig, ax = plt.subplots(figsize=(W, 3.3), facecolor=BG)
fig.subplots_adjust(left=0.20, right=0.97, top=0.80, bottom=0.18)
ax.set_facecolor(BG)
ys = {t: i for i, t in enumerate(reversed(TISSUES))}
ax.axvspan(0, 1.4, color=REP, alpha=0.07, lw=0)
ax.text(0.70, 2.62, "Abstract: “upregulated” in all three\n(the side where it places every tissue)",
ha="center", va="center", fontsize=10, color=REP)
ax.axvline(0, color=INK2, lw=1)
for t in TISSUES:
y = ys[t]
d, lo, hi = REC_D[t]
ax.plot([lo, hi], [y - 0.12, y - 0.12], color=REC, lw=2, solid_capstyle="butt")
ax.plot(d, y - 0.12, "o", ms=9, color=REC, mec=BG, mew=1)
ax.plot(SUPP1_DE[t][0], y + 0.12, "D", ms=8.5, color=REP, mec=BG, mew=1)
ax.set_yticks([ys[t] for t in TISSUES], [LABEL[t] for t in TISSUES])
ax.set_ylim(-0.6, 2.95)
ax.set_xlim(-1.4, 1.4)
ax.set_xlabel("Serpin A1c, IF16 minus AL (log2 intensity)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(-1.38, -0.5, "← lower under IF", fontsize=10, color=INK2, va="center")
ax.text(1.38, -0.5, "higher under IF →", fontsize=10, color=INK2, va="center",
ha="right")
from matplotlib.lines import Line2D # noqa: E402
h = [Line2D([], [], marker="D", ls="", ms=8.5, color=REP,
label="Reported logFC (Supp. file 1, limma)"),
Line2D([], [], marker="o", ls="-", ms=9, color=REC,
label="Recomputed mean difference, 95% CI (Supp. file 3, n=7+7)")]
fig.legend(handles=h, loc="upper center", ncol=1, frameon=False, fontsize=10,
bbox_to_anchor=(0.55, 1.0))
save(fig, "fig2")
print("\nwrote fig1.svg/png, fig2.svg/png in", HERE)
Figure 5A's key drug-effect statistic, F(1,6) = 3.7, gives p = 0.10, not the printed 0.024
eLife (2019) · doi:10.7554/eLife.47870 · PMC6728144
Summary
The Figure 5A legend reports that the Kv1.4 antagonist UK-78,282 suppresses spiking in nucleus accumbens neurons from low-motivation (lowS) rats, with "Main drug effect: F(1,6) = 3.7, p=0.024". An F of 3.7 on (1, 6) degrees of freedom has an upper-tail P of 0.103, and p = 0.024 would need F ≈ 9.0. With 1 numerator degree of freedom, no choice of error df brings P below 0.054. The matching highS statistic in the same legend, and every other F test in the Figure 5 section, recompute correctly. This is the only statistic with values reported for the within-lowS drug effect on firing (the asterisk in Figure 5B appears to mark the lowS vs highS comparison under drug). That effect underpins the claim that the drug suppresses firing "selectively" in lowS neurons, which the Figure 5 title, a Results subheading and the Discussion all rely on.
What the paper reports
Figure 5 legend, title and panel A:
NAc neurons from lowS animals are uniquely sensitive to Kv1.4 antagonist, UK-78,282.(A) Action potential frequency was measured after application of UK-78,282 (100 nM) and expressed as percent change from frequency before UK-78,282 in the same cell (n = 7, N = 4/group). UK-72,282 suppresses firing in the lowS group to levels significantly different from baseline (Main drug effect: F(1,6) = 3.7, p=0.024, two-way RM ANOVA). In the highS group, firing after UK-72,282 application is not significantly different from baseline (Main drug effect: F(1,6) = 0.99, p=0.36, two-way RM ANOVA).
(The legend spells the compound "UK-72,282" in these two sentences and "UK-78,282" everywhere else. This looks like a separate typo and does not affect the statistics.)
Figure 5 legend, panel B:
(B) Firing frequency-current relationship highlights decreased action potential output in the lowS group after UK-78,282 application (*, p<0.05, two-way RM ANOVA).
This legend gives no F, df or exact P. The Results text (below) gives the Figure 5B statistic as a between-group effect under drug, "Main group effect: F(1,12)=5.819, p=0.032". Its error df of 12 fits 7 + 7 cells in two groups, not a within-lowS comparison of 7 cells (error df 6). So the Figure 5B asterisk most likely marks lowS vs highS in the presence of the drug, not lowS before vs after the drug. It is not a second test with reported values for the lowS drug effect.
Results, section "Kv1.4 channels modulate SPN excitability selectively in lowS animals":
The overall effect of acute application of UK-78,282 (100 nM) was to suppress action potential output in lowS, but not highS, SPNs (Figure 5A). This differential effect reversed the excitability profile of lowS relative to highS neurons such that in the presence of UK-78,282, lowS SPNs fired fewer action potentials than highS SPNs (Figure 5B, [Main group effect: F(1,12)=5.819, p=0.032] two-way RM ANOVA); cf. Figure 4A).
Discussion, "Selective effect of UK-78,282 on lowS Kv1.4":
The compound selectively suppressed action potential firing in the lowS group, but was equally efficacious at blocking the peak of A-type currents in both lowS and highS animals.
Discussion, opening paragraph:
Suppression of Kv1.4 activity by UK-78,282 selectively decreases SPN spiking and facilitates motivated behavior in subjects with a lower baseline motivation for reward.
Materials and methods, Statistics:
Data were reported as mean ± standard error of the mean and statistical significance thresholds were set at p<0.05.
What the paper's own numbers show
Method. An ANOVA F statistic on (d1, d2) degrees of freedom has P = Pr(F_{d1,d2} ≥ F), the upper tail of the F distribution. With d1 = 1 this equals the two-sided t-test P for t = √F on d2 df. The legend gives n = 7 cells per group, so a within-cell main drug effect in one group's two-way repeated-measures ANOVA has df (1, 6), which is the df printed. I computed P for every F statistic printed in Figure 5, its supplements and the Kv1.4 Results section, plus the other F tests in the paper as a check on the convention (scipy, recompute.py.txt).
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| lowS main drug effect, F(1,6) = 3.7 | Figure 5 legend (A) | p = 0.024 | P = 0.1028 |
| Range of P for any F that rounds to 3.7 (3.65–3.75), df (1,6) | Figure 5 legend (A) | — | 0.101 to 0.105 |
| F needed for p = 0.024 at df (1,6) | Figure 5 legend (A) | F = 3.7 | F = 9.00 (8.91–9.10 for p 0.0235–0.0245) |
| Critical F(1,6) at α = 0.05 | — | — | 5.99 |
| Smallest possible P for F = 3.7 with d1 = 1, any error df | — | — | 0.0544 (chi-square limit) |
| One-sided P for F(1,6) = 3.7 | Figure 5 legend (A) | — | 0.0514 |
| highS main drug effect, F(1,6) = 0.99 | Figure 5 legend (A) | p = 0.36 | 0.358 (consistent) |
| Group effect under drug, F(1,12) = 5.819 | Results, Kv1.4 section (Fig 5B) | p = 0.032 | 0.0328 (consistent) |
| IA lowS drug effect, F(1,12) = 14.03 | Results, Kv1.4 section (Fig 5D) | p = 0.0028 | 0.0028 (consistent) |
| IA highS drug effect, F(1,13) = 17.19 | Results, Kv1.4 section (Fig 5D) | p = 0.001 | 0.00115 (consistent) |
| IA midS drug effect, F(1,6) = 6.39 | Figure 5—figure supplement 2 legend (F) | p = 0.045 | 0.0448 (consistent) |
| τ lowS, t(17) = 3.5 | Results, Kv1.4 section (Fig 5E) | p = 0.003 | 0.0027 (consistent) |
| τ highS, t(13) = 0.35 | Results, Kv1.4 section (Fig 5E) | p = 0.73 | 0.732 (consistent) |
| Firing group effect, F(1,215) = 22.52 (not part of this claim) | Results, spike-output section (Fig 4A) | p = 0.01 | 3.8 × 10⁻⁶ (inconsistent; conclusion unchanged) |
| Total K⁺ group effect, F(1,13) = 0.12 (not part of this claim) | Results, spike-output section (Fig 4F) | p = 0.31 | 0.735 (inconsistent; conclusion unchanged) |
| FR group effect, F(1,42) = 2.13 (not part of this claim) | Results, Figure 1F | p = 0.1 | 0.152 (inconsistent at printed precision; conclusion unchanged) |
The behavioural F tests for Figure 6 recompute to within rounding: breakpoints lowS F(2,21) = 9.5, P = 0.0012 (reported 0.001); midS F(2,25) = 0.198, P = 0.822 (0.82); highS F(2,21) = 0.5, P = 0.614 (0.64); interaction F(4,32) = 5.1, P = 0.0027 (0.003). Small rounding mismatches occur elsewhere: besides Figure 1F above, Figure 4E prints F(1,12) = 0.04 with p = 0.83, while any F that rounds to 0.04 gives P between 0.836 and 0.855 (recomputed 0.845). These do not change any conclusion. So the paper otherwise uses standard upper-tail F P values with the printed df, and the lowS Figure 5A pair is the outlier in its section.
Figure
Figure 1. Each row is an F test printed in the paper. The orange ring is the P value as printed and the blue dot is the P recomputed from the printed F and degrees of freedom. The dashed line marks the paper's p < 0.05 threshold. In the top six rows (Figure 5, its supplement and the Kv1.4 Results text) the two values coincide except for the lowS main drug effect in Figure 5A, where the printed p = 0.024 sits below the threshold and the recomputed P = 0.103 sits above it. The two lower rows are other mismatches in the Figure 4 statistics. They do not change any conclusion and are shown only to complete the picture.
Figure 2. The blue curve is P for F = 3.7 on 1 numerator df as the error (denominator) df varies from 1 to 10,000. At the printed df (1, 6), P = 0.103. As the error df grows, P falls toward 0.0544, the chi-square limit, and it never reaches the printed p = 0.024 (orange ring). So a mistyped denominator df cannot reconcile the printed F and p.
Why it matters
The lowS statistic in Figure 5A is the only statistic with values reported for the statement that UK-78,282 significantly suppresses spiking in lowS neurons. The Figure 5B legend also mentions "decreased action potential output in the lowS group after UK-78,282 application" with an asterisk, but the only statistic the paper gives for Figure 5B is the between-group effect under drug, F(1,12) = 5.819 (see above). The lowS suppression is half of the "selective" effect that the paper draws from Figure 5. The Figure 5 title ("uniquely sensitive"), the Results subheading ("modulate SPN excitability selectively in lowS animals"), the Discussion ("selectively suppressed action potential firing in the lowS group") and the opening Discussion summary ("selectively decreases SPN spiking") all rely on it. The Abstract makes the link between low motivation and faster Kv1.4 inactivation more generally. It does not quote this value.
If F = 3.7 is the correct statistic, the drug effect in lowS neurons is not significant at the paper's own threshold (P = 0.10). The contrast "lowS, but not highS," would then compare two non-significant within-group results. The paper reports no group × drug interaction test for firing. Some supporting evidence does not depend on this statistic: the between-group difference under drug in Figure 5B (F(1,12) = 5.819, P = 0.033), the lowS-only change in IA inactivation time constant in Figure 5E (t(17) = 3.5, P = 0.003), and the behavioural microinfusion results in Figure 6. So the paper's broader conclusion about Kv1.4 and motivation does not stand or fall on this one number. But as printed, the specific claim that the drug significantly suppresses lowS firing rests on a statistic whose F and P cannot both be right.
Explanations considered
- Different denominator df. n = 7 cells per group gives error df 6 for a within-group main drug effect, which is what is printed. The highS test next to it, F(1,6) = 0.99, p = 0.36, recomputes exactly (0.358) with the same df. Even if the error df were 12 (both groups pooled), 60 or effectively infinite, F = 3.7 on 1 df gives P = 0.079, 0.059 or 0.054. No error df gives 0.024 (Figure 2). This does not explain the numbers.
- Post hoc P printed next to the main-effect F. The Methods say the two-way RM ANOVAs were "followed by Bonferroni post hoc tests". p = 0.024 could in principle be a post hoc comparison, for example at one current step. The label "Main drug effect" and its pairing with F(1,6) argue against this, and the highS statistic beside it is a matching main-effect pair. Even if it were a post hoc P, the legend would still attribute it to the main drug effect, and the main-effect P for F(1,6) = 3.7 would be 0.10. This would not rescue the main-effect claim as printed.
- One-sided test. Halving gives P = 0.051. That is not 0.024, and it is still above 0.05. The Methods do not mention one-sided tests. This does not explain the numbers.
- Interaction term reported in place of the main effect. The label says "Main drug effect". I scanned df (k, 6k) for a drug × current-step interaction within one group and (k, 12k) for both groups pooled, for k = 1 to 15. Only df (3, 36) comes within 0.004 of 0.024, at P = 0.020. That still does not round to 0.024, and it contradicts both the printed df and the label. This is unlikely to explain the numbers.
- Mistyped F value. If the true F were about 9.0 (8.9–9.1), p = 0.024 would be correct at df (1, 6). A typo such as 3.7 for 8.7 gives P = 0.026, close but not exact. This explanation is possible and would leave the significance claim intact. Only the underlying analysis can show it. In that case the error is in the F value, and the printed record still needs correcting.
- Mistyped P value. If F = 3.7 is correct, the true P is about 0.10 and the lowS drug effect is not significant. This explanation is equally possible from the published numbers.
- Text-conversion artefact. The same values appear in the eLife HTML version of record, the publisher PDF, the PMC page and the Europe PMC XML, where the subscript holds only the df. This is not a conversion error.
- Computation from the supplements. No source data are provided for Figure 5. The only source-data file (Figure 3—source data 1) holds RNA-seq gene lists, so the ANOVA cannot be recomputed from published material.
Status
- Version of record. The Figure 5A legend reads the same in four renderings: the eLife Version of Record (https://elifesciences.org/articles/47870, v1, 2019-09-05), the publisher PDF (https://cdn.elifesciences.org/articles/47870/elife-47870-v1.pdf, text extracted with PyMuPDF), the PMC page, and the Europe PMC full-text XML. The eLife versions API lists only one version (VoR v1). Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The supplementary items are Figure 3—source data 1 (
elife-47870-fig3-data1-v1.xlsx, RNA-seq differential expression lists), image-only figure supplements (Figure 1—S1, 4—S1/S2, 5—S1/S2, 6—S1) and a transparent reporting form. The data availability statement says the data are in the manuscript and supporting files, and no external repository is listed. None of these gives the Figure 5A cell-level data. The published peer review (https://elifesciences.org/articles/47870/peer-reviews) does not mention this statistic. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record'srelationfield holds only the two peer-review links. A web search for the title with "pubpeer" or "correction" returned only the article's own pages and unrelated corrections. The PubPeer API and search page refused automated requests (Cloudflare challenge and HTTP 403), so PubPeer could not be checked directly. - Sources. PMC article PMC6728144; Europe PMC full-text XML; eLife article page, PDF, versions API and peer-review page; Crossref; Europe PMC; PubPeer (blocked); https://doi.org/10.7554/eLife.47870.
Suggested correction
- Check the original ANOVA output for the lowS main drug effect in Figure 5A and correct whichever of F(1,6) = 3.7 or p = 0.024 is wrong. If F = 3.7 is right, P = 0.10, and the legend should no longer call the suppression significant.
- If P = 0.10 is correct, revise the statements that UK-78,282 suppresses firing selectively in lowS neurons (Figure 5 title, Results subheading and first paragraph, Discussion) so that they rest on the tests that support them, mainly the Figure 5B group difference under drug. Ideally, also report a group × drug interaction test for firing.
- Check and correct the two other mismatched pairs in the Figure 4 Results text: "F(1,215)=22.52, p=0.01" (F and df imply P ≈ 4 × 10⁻⁶; F(1,215) also looks inconsistent with n = 9 cells per group) and "F(1,13)=0.12, p=0.31" (implies P ≈ 0.73). Neither changes a conclusion.
- Minor: correct "UK-72,282" to "UK-78,282" in the Figure 5A legend.
Recomputation script (python recompute.py.txt)
"""Recompute the F-test P values of PMC6728144
(eLife 2019, doi:10.7554/eLife.47870, "Altered gating of Kv1.4 in the nucleus accumbens
suppresses motivation for reward").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (and identical in the eLife version of record):
Figure 5 legend, panel A: lowS "Main drug effect: F(1,6) = 3.7, p=0.024, two-way RM ANOVA"
highS "Main drug effect: F(1,6) = 0.99, p=0.36, two-way RM ANOVA"
n = 7 cells, N = 4 animals per group
Results, Kv1.4 section: Fig 5B group F(1,12)=5.819 p=0.032
IA lowS drug F(1, 12)=14.03 p=0.0028; IA highS drug F(1, 13)=17.19 p=0.001
tau lowS t(17)=3.5 p=0.003; highS t(13)=0.35 p=0.73; midS t(6)=0.35 p=0.74
Fig 5-figure supplement 2F: midS IA drug F(1,6)=6.39 p=0.045
Results, Figure 1F/4/6 (convention check only): see OTHER below.
Method: an ANOVA F statistic with (d1, d2) degrees of freedom has upper-tail P = P(F_{d1,d2} >= F).
For d1 = 1, F = t^2 and the same P is the two-sided t-test P on d2 df.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
def pF(F, d1, d2):
return stats.f.sf(F, d1, d2)
# ------------------------------------------------------------------ 1. the claim
print("=== 1. Figure 5A legend: main drug effect within each group ===")
p_low = pF(3.7, 1, 6)
p_high = pF(0.99, 1, 6)
print("lowS F(1,6) = 3.7 -> P = %.4f (reported p = 0.024)" % p_low)
print("highS F(1,6) = 0.99 -> P = %.4f (reported p = 0.36)" % p_high)
print("same lowS P via two-sided t(6) = sqrt(3.7) = %.3f: P = %.4f" % (np.sqrt(3.7), 2 * stats.t.sf(np.sqrt(3.7), 6)))
lo, hi = pF(3.75, 1, 6), pF(3.65, 1, 6)
print("any F printed as 3.7 (3.65 to 3.75) at df(1,6): P from %.4f to %.4f" % (lo, hi))
F_need = stats.f.isf(0.024, 1, 6)
F_lo, F_hi = stats.f.isf(0.0245, 1, 6), stats.f.isf(0.0235, 1, 6)
print("F needed for P = 0.024 at df(1,6): %.2f (P printed as 0.024, i.e. 0.0235-0.0245, needs F %.2f to %.2f)"
% (F_need, F_lo, F_hi))
print("F(1,6) critical value at alpha = 0.05: %.2f" % stats.f.isf(0.05, 1, 6))
print("\n=== 2. Could another denominator df explain it? (numerator df = 1) ===")
for d2 in (6, 12, 13, 60, 1000):
print(" F = 3.7 at df(1,%d): P = %.4f" % (d2, pF(3.7, 1, d2)))
chi_lim = stats.chi2.sf(3.7, 1)
print(" limit as d2 -> infinity (chi-square, 1 df): P = %.4f" % chi_lim)
d2s = np.arange(1, 100001)
print(" smallest P over d2 = 1..100000: %.4f" % pF(3.7, 1, d2s).min())
print("\n=== 3. One-sided test ===")
print(" one-sided P = %.4f" % (p_low / 2))
print("\n=== 4. Interaction-type df: (k, 6k) for one group of 7 cells, (k, 12k) for both groups pooled ===")
near = []
n_checked = 0
for k in range(1, 16):
for d2 in (6 * k, 12 * k):
n_checked += 1
p = pF(3.7, k, d2)
if abs(p - 0.024) < 0.004:
near.append((k, d2, p))
print(" F = 3.7 at df(%d,%d): P = %.4f (within 0.004 of 0.024)" % (k, d2, p))
rounds_to = [m for m in near if round(m[2], 3) == 0.024]
print(" %d of %d df pairs (k,6k) or (k,12k), k = 1..15, come within 0.004 of 0.024; %d give a P that rounds to 0.024"
% (len(near), n_checked, len(rounds_to)))
if near:
print(" df pair(s) within 0.004: " + ", ".join("(%d,%d)" % (k, d2) for k, d2, _ in near)
+ ("; no other (k,6k) or (k,12k) comes that close" if len(near) < n_checked else ""))
else:
print(" no (k,6k) or (k,12k), k = 1..15, comes within 0.004 of 0.024")
print("\n=== 5. F values that would match p = 0.024 at df(1,6) ===")
for F in (3.7, 7.3, 8.7, 9.0, 9.7, 13.7):
print(" F(1,6) = %4.1f -> P = %.4f" % (F, pF(F, 1, 6)))
# ------------------------------------------------------------------ 6. same section
print("\n=== 6. Other tests in Figure 5 and the Kv1.4 Results section (same convention) ===")
SECTION = [
# key, label, F, d1, d2, reported P, location
("f5a_low", "Fig 5A lowS drug", 3.7, 1, 6, 0.024, "Figure 5 legend (A)"),
("f5a_high", "Fig 5A highS drug", 0.99, 1, 6, 0.36, "Figure 5 legend (A)"),
("f5b", "Fig 5B group under drug", 5.819, 1, 12, 0.032, "Results, Kv1.4 section"),
("ia_low", "Fig 5D IA lowS drug", 14.03, 1, 12, 0.0028, "Results, Kv1.4 section"),
("ia_high", "Fig 5D IA highS drug", 17.19, 1, 13, 0.001, "Results, Kv1.4 section"),
("ia_mid", "Fig 5-S2F IA midS drug", 6.39, 1, 6, 0.045, "Figure 5-figure supplement 2 legend (F)"),
]
R = {}
for key, lab, F, a, b, prep, loc in SECTION:
R[key] = pF(F, a, b)
print(" %-24s F(%d,%d) = %-6g reported P = %-7g recomputed P = %.4g" % (lab, a, b, F, prep, R[key]))
for lab, t, df, prep in [("tau lowS", 3.5, 17, 0.003), ("tau highS", 0.35, 13, 0.73), ("tau midS", 0.35, 6, 0.74)]:
print(" %-24s t(%d) = %-5g reported P = %-7g recomputed P = %.4f" % (lab, df, t, prep, 2 * stats.t.sf(t, df)))
print("\n=== 7. Elsewhere in the paper (not part of the Figure 5A claim) ===")
OTHER = [
("f4a", "Fig 4A group (firing)", 22.52, 1, 215, 0.01, "Results, spike-output section"),
("f4f", "Fig 4F total K+ group", 0.12, 1, 13, 0.31, "Results, spike-output section"),
("f1f", "Fig 1F FR group", 2.13, 1, 42, 0.1, "Results, Figure 1F"),
("f4c", "Fig 4C IA group", 2.64, 1, 28, 0.12, "Results, spike-output section"),
("f4e", "Fig 4E IBK group", 0.04, 1, 12, 0.83, "Results, spike-output section"),
("pr_conc", "Fig 6 PR concentration", 3.9, 1.97, 31.5, 0.03, "Results, Figure 6"),
("pr_int", "Fig 6 PR interaction", 5.1, 4, 32, 0.003, "Results, Figure 6"),
("bp_low", "Fig 6Ai lowS breakpoint", 9.5, 2, 21, 0.001, "Results, Figure 6"),
("bp_mid", "Fig 6Bi midS breakpoint", 0.198, 2, 25, 0.82, "Results, Figure 6"),
("bp_high", "Fig 6Ci highS breakpoint", 0.5, 2, 21, 0.64, "Results, Figure 6"),
]
for key, lab, F, a, b, prep, loc in OTHER:
R[key] = pF(F, a, b)
print(" %-24s F(%g,%g) = %-6g reported P = %-6g recomputed P = %.3g" % (lab, a, b, F, prep, R[key]))
print(" Fig 6 PR group F(2,16) = 153.6, reported p<0.0001: recomputed P = %.2g" % pF(153.6, 2, 16))
# printed-precision checks for the small mismatches
print(" Fig 1F: any F printed as 2.13 (2.125-2.135) at df(1,42): P from %.4f to %.4f; printed p = 0.1 would need 0.05-0.15"
% (pF(2.135, 1, 42), pF(2.125, 1, 42)))
print(" Fig 4E: any F printed as 0.04 (0.035-0.045) at df(1,12): P from %.4f to %.4f; printed p = 0.83 would need 0.825-0.835"
% (pF(0.045, 1, 12), pF(0.035, 1, 12)))
print(" Fig 6Ci: any F printed as 0.5 (0.45-0.55) at df(2,21): P from %.3f to %.3f; printed p = 0.64 lies in this range"
% (pF(0.55, 2, 21), pF(0.45, 2, 21)))
# ------------------------------------------------------------------ figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, "%s.%s" % (name, ext)), dpi=200, facecolor=BG)
plt.close(fig)
def fmtp(p):
if p < 0.001:
return "%.1e" % p
if p < 0.01:
return "%.4f" % p
return ("%.3f" % p) if p < 0.2 else ("%.2f" % p)
# fig1: reported vs recomputed P for each F test, one log axis
rows = [
# label, printed P, recomputed P, printed P consistent with the recomputed one at its printed precision?
("Fig 5A lowS, drug F(1,6) = 3.7", 0.024, R["f5a_low"], False),
("Fig 5A highS, drug F(1,6) = 0.99", 0.36, R["f5a_high"], True),
("Fig 5B group F(1,12) = 5.819", 0.032, R["f5b"], True),
("Fig 5D IA lowS F(1,12) = 14.03", 0.0028, R["ia_low"], True),
("Fig 5D IA highS F(1,13) = 17.19", 0.001, R["ia_high"], True),
("Fig 5-S2F IA midS F(1,6) = 6.39", 0.045, R["ia_mid"], True),
None,
("Fig 4A group F(1,215) = 22.52", 0.01, R["f4a"], False),
("Fig 4F total K+ F(1,13) = 0.12", 0.31, R["f4f"], False),
]
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.patch.set_facecolor(BG)
style(ax)
ys, labels = [], []
y = 0
for r in rows:
if r is None:
y -= 0.7
continue
lab, prep, prec, same = r
if y == 0:
ax.axhspan(y - 0.45, y + 0.45, color="#f3ece6", zorder=0)
ax.plot([prep, prec], [y, y], color=SEC, lw=1.0, zorder=1)
ax.plot(prep, y, "o", ms=12, mfc="none", mec=ORANGE, mew=2, zorder=3)
ax.plot(prec, y, "o", ms=8, mfc=BLUE, mec=BLUE, zorder=4)
tr = ax.get_yaxis_transform() # x in axes fraction, y in data
ax.text(1.03, y, "%g" % prep, transform=tr, va="center", ha="left", fontsize=10,
color=ORANGE if not same else SEC, fontweight="normal" if same else "bold")
ax.text(1.27, y, fmtp(prec), transform=tr, va="center", ha="left", fontsize=10,
color=BLUE if not same else SEC, fontweight="normal" if same else "bold")
ys.append(y)
labels.append(lab)
y -= 1
ax.axvline(0.05, color=SEC, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(0.043, 0.62, "p = 0.05", ha="right", va="bottom", fontsize=10, color=SEC,
bbox=dict(boxstyle="square,pad=0.1", fc=BG, ec="none"), zorder=5)
ax.text(1.03, 0.62, "printed", transform=ax.get_yaxis_transform(), ha="left", va="bottom", fontsize=10, color=SEC)
ax.text(1.27, 0.62, "recomputed", transform=ax.get_yaxis_transform(), ha="left", va="bottom", fontsize=10,
color=SEC)
ax.set_xscale("log")
ax.set_xlim(1e-6, 1.0)
ax.set_ylim(y + 0.4, 1.1)
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("P value (log scale)")
ax.text(1.2e-6, -5.85, "Elsewhere, not part of the Figure 5A claim", fontsize=10, color=SEC, ha="left",
va="center", style="italic", bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"), zorder=5)
leg = [Line2D([], [], ls="none", marker="o", ms=12, mfc="none", mec=ORANGE, mew=2, label="P printed in the paper"),
Line2D([], [], ls="none", marker="o", ms=8, mfc=BLUE, mec=BLUE, label="P recomputed from the printed F and df")]
fig.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 0.995), ncol=2, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.5)
fig.subplots_adjust(left=0.36, right=0.72, top=0.88, bottom=0.12)
save(fig, "fig1")
# fig2: P for F = 3.7 (numerator df 1) as a function of the error df
d2 = np.unique(np.round(np.logspace(np.log10(1), np.log10(10000), 400)).astype(int))
fig, ax = plt.subplots(figsize=(7.2, 3.6))
fig.patch.set_facecolor(BG)
style(ax)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.plot(d2, pF(3.7, 1, d2), color=BLUE, lw=2, label="P for F = 3.7 on (1, error df)")
ax.axhline(chi_lim, color=BLUE, lw=1, ls=(0, (2, 2)))
ax.text(10000, chi_lim + 0.004, "lower limit %.4f (infinite error df)" % chi_lim, ha="right", va="bottom",
fontsize=10, color=BLUE)
ax.axhline(0.05, color=SEC, lw=0.8, ls=(0, (3, 3)))
ax.text(1.05, 0.05 - 0.004, "p = 0.05", ha="left", va="top", fontsize=10, color=SEC)
ax.plot(6, p_low, "o", ms=8, color=BLUE)
ax.annotate("df (1,6) as printed: P = %.3f" % p_low, (6, p_low), xytext=(12, p_low + 0.025),
fontsize=10, color=BLUE, arrowprops=dict(arrowstyle="-", color=BLUE, lw=1))
ax.plot(6, 0.024, "o", ms=12, mfc="none", mec=ORANGE, mew=2)
ax.annotate("printed p = 0.024", (6, 0.024), xytext=(12, 0.012), fontsize=10, color=ORANGE,
arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1))
ax.set_xscale("log")
ax.set_xlim(1, 10000)
ax.set_ylim(0, 0.32)
ax.set_xlabel("Error (denominator) degrees of freedom, log scale")
ax.set_ylabel("P value")
fig.subplots_adjust(left=0.10, right=0.97, top=0.95, bottom=0.16)
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Age-by-haplogroup interaction p-value is 0.002 in abstract and text but 0.02 by Table 3's own statistic
Ann Neurol (2014) · doi:10.1002/ana.24116 · PMC4112718
Summary
The abstract and the Results give the interaction between mtDNA haplogroup and age on 6-month Glasgow Outcome Score as p = 0.002, and the Results call it "a strong interaction". Table 3 reports the same model term with chi-square 13.439 on 5 df, F 2.688 and p = 0.02. The upper-tail chi-square probability of 13.439 on 5 df is 0.0196, which matches the table and is 9.8 times the value in the text; a p of 0.002 on 5 df would need a chi-square of about 18.9. Every other p-value in Table 3 that can be checked agrees with its statistic, so the text value is the outlier. The interaction stays nominally significant at the paper's 0.05 threshold, but the evidence for it is roughly ten times weaker than the abstract states.
What the paper reports
All quotations were checked verbatim against the article as archived in PubMed Central.
Abstract, Results:
There was also a significant interaction between mitochondrial genome and age (p = 0.002), with a strong protective effect of both haplogroups T (p = 0.015) and K (p = 0.017) with advancing age.
Results, section "Interaction between mtDNA and Aging":
The inclusion of the interaction term "age by haplogroup" in the model revealed a strong interaction between age and mtDNA haplogroups (p = 0.002). Possessing either haplogroup K or T significantly mitigated the negative effect of aging on outcome (p = 0.017 and p = 0.015, respectively), as shown by Figure 1A.
Table 3, "Model Effects" (columns Parameter | Category | Chi-Square | df | F | OR (95% CI) | p):
Interaction between mtDNA haplogroup and age | | 13.439 | 5 | 2.688 | | 0.02
Table 3, neighbouring rows for the other two mtDNA tests:
mtDNA haplogroup, overall effect | | 15.601 | 5 | 3.120 | | 0.008
Interaction between mtDNA haplogroup and APOE ε4 | | 21.35 | 5 | 4.27 | | 0.001
Patients and Methods, Statistical Analysis:
We also analyzed the interaction between mtDNA haplogroups and age, and between mtDNA haplogroup and APOE by fitting the interaction terms "age by mtDNA haplogroup" and "APOE genotype by mtDNA haplogroup" into the model.
A p value of <0.05 was considered significant.
What the paper's own numbers show
Method. For every row of Table 3, I took the printed chi-square and df and computed the upper-tail probability of the chi-square distribution (scipy.stats.chi2.sf). As a cross-check I also tested the printed F on (df, 1000) degrees of freedom; the denominator df is not reported, but for a cohort of 1,094 any plausible value from 500 upward changes p only in the fourth decimal. I compared each result with the printed p and the rounding range its printed decimals imply. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Haplogroup × age interaction, p | Abstract; Results, "Interaction between mtDNA and Aging" | 0.002 | 0.0196 (χ² 13.439, df 5); 9.8 times the reported value |
| Haplogroup × age interaction, p | Table 3 | 0.02 (χ² 13.439, df 5, F 2.688) | 0.0196 from χ²; 0.0201 from F on (5, 1000) df; 0.0201 to 0.0207 for denominator df 1094 down to 500 |
| F for the age interaction | Table 3 | 2.688 | 13.439 / 5 = 2.6878 |
| χ² needed for p = 0.002 on 5 df | derived | — | 18.907 (18.386 for p = 0.0025, the upper rounding edge of 0.002) |
| F needed for p = 0.002 on (5, 1000) df | derived | — | 3.812 (printed F is 2.688) |
| p for χ² 13.439 at other df | derived | — | df 2: 0.0012; df 3: 0.0038; df 4: 0.0093; df 6: 0.0366. No integer df gives 0.002 |
| Haplogroup, overall effect, p | Table 3; Abstract | 0.008 | 0.0081 (χ² 15.601, df 5) |
| Haplogroup × APOE interaction, p | Table 3; Abstract; Results | 0.001 | 0.0007 (χ² 21.35, df 5) |
| Traumatic SAH, p | Table 3 | 0.001 | 0.0014 (χ² 10.149, df 1) |
| Pupillary reactivity, p | Table 3 | 0.011 | 0.0105 (χ² 9.112, df 2) |
| Hypotension, p | Table 3 | 0.046 | 0.0454 (χ² 4.002, df 1) |
| Hypoxia, p | Table 3 | 0.331 | 0.3310 (χ² 0.945, df 1) |
| ApoE genotype, p | Table 3 | 0.476 | 0.4764 (χ² 0.507, df 1) |
| Age, GCS motor, CT Marshall, p | Table 3 | 0.000 | < 10-5 for each |
Table 3 is internally consistent: its printed F equals chi-square divided by df, and its printed p agrees with the chi-square on that row (the hypotension row differs by one unit in the last digit: 0.0454 would round to 0.045, not 0.046). The only disagreement is between Table 3 and the text. The text's 0.002 differs from the table's 0.02 by exactly one decimal place, and the table's value is the one that follows from the reported statistic.
Figure
Figure 1. p-values on a log scale for each Table 3 term with a printed non-zero p. Orange markers are values printed in the paper, with a horizontal bar spanning the range that rounds to the printed digits; open blue circles are the upper-tail chi-square probabilities recomputed from the chi-square and df printed in Table 3. The shaded top two rows show the age-by-haplogroup interaction twice: once with the value given in the Abstract and Results (diamond, 0.002) and once with the value in Table 3 (square, 0.02). The recomputed value, 0.0196, lies within the Table 3 rounding range and 9.8 times above the text value. The dashed line marks the paper's significance threshold, p = 0.05. The log scale is used because the discrepancy is a ratio.
Figure 2. The two passages that state p = 0.002 (left) and the Table 3 row for the same interaction term (right). Orange text is printed in the paper; blue text is recomputed from Table 3. The Methods describe a single proportional-odds model into which the interaction terms were fitted, and Table 3 is that model's table of effects, so the two locations describe the same test.
Why it matters
- The abstract overstates the strength of a reported result about tenfold. The interaction with age is one of four quantitative findings in the abstract's Results, and the Results text describes it as "a strong interaction". The paper's own statistic gives p = 0.0196. That is still below the stated threshold of 0.05, so the direction and nominal significance are unchanged, but it is a much weaker result than p = 0.002.
- The corrected value is more fragile to multiplicity. The paper tests three omnibus mtDNA terms in the same model (overall effect, interaction with age, interaction with APOE) and reports them without adjustment. As an illustration only, a Bonferroni factor of 3 gives 0.0588 for the corrected age interaction, above 0.05, whereas it would give 0.006 for the stated 0.002. The overall effect (0.0081 × 3 = 0.024) and the APOE interaction (0.0007 × 3 = 0.002) would remain below 0.05.
- Downstream claims lean on this test. The haplogroup-specific age effects for T (p = 0.015) and K (p = 0.017) are follow-up tests of this omnibus interaction. The Discussion states "We found a strong mitigating effect of haplogroups K and T on the effect of aging on TBI outcome", builds a mechanistic interpretation on it, and proposes that "mtDNA genomics should be included in future outcome prediction models". The abstract's Interpretation ("an interplay between mitochondrial DNA, pathophysiology of TBI, and aging") also draws on it. These claims rest on an omnibus interaction with p of about 0.02, not 0.002.
- What is not affected. The overall mtDNA association (p = 0.008), the haplogroup × APOE interaction (p = 0.001) and the other covariate effects all agree with their printed statistics. The per-haplogroup p-values for T and K (0.015, 0.017) cannot be checked from the paper, since their statistics are not reported.
Explanations considered
- A different model for the text value. Does not account for it. The Statistical Analysis section describes one SPSS proportional-odds model with the two interaction terms fitted "into the model", and Table 3 is that model's table of effects. Neither the text nor the table footer mentions a second model, and the Results sentence refers to "the inclusion of the interaction term 'age by haplogroup' in the model".
- A different test statistic (Wald chi-square vs F). Does not account for it. Table 3 gives both, and they agree: F = 13.439 / 5 = 2.688, and the two give p = 0.0196 and about 0.020. Both match the table's 0.02.
- A degrees-of-freedom slip. Does not account for it. No integer df turns 13.439 into 0.002 (df 2 gives 0.0012, df 3 gives 0.0038). The table's df of 5 matches a six-category haplogroup term (H, J, T, U, K, other), as for the other two mtDNA rows.
- One-sided testing or adjustment. Does not account for it. Neither is mentioned. Chi-square and F tests already use the upper tail only; halving 0.0196 would give 0.0098, not 0.002, and any multiplicity adjustment would raise p.
- A per-haplogroup value. Does not account for it. The text gives the T and K slope p-values separately (0.015 and 0.017) in the same sentences, so 0.002 is presented as the overall interaction p.
- Confusion with the APOE interaction or the K-in-ε4 odds ratio. Not supported. The APOE interaction is 0.001 in the abstract, the Results and Table 3, and recomputes to 0.0007. The Results do give p = 0.002 for the odds ratio of haplogroup K among APOE ε4 carriers, but that is a separate, labelled estimate in a separate section.
- A conversion or typesetting artifact. Ruled out. The publisher-typeset PDF of the version of record (see Status), the PMC HTML and the Europe PMC full-text XML all carry p = 0.002 in the abstract and Results and 0.02 in Table 3.
- Most likely explanation. The table, its F column and its chi-square are mutually consistent, and the other omnibus p-values given in the text (0.008 and 0.001) match the table. A transcription slip from 0.02 to 0.002 in the text is consistent with all the numbers, but the paper does not let one confirm this; only the original model output can.
- Side observations, not part of the main finding. In Table 3 the hypotension F is printed as 4.4002 against a chi-square of 4.002 on 1 df, where every other row has F = chi-square / df; this looks like a typographical slip and does not change the p-value. The CT Marshall F (11.856) differs slightly from 59.238 / 5 = 11.848. Neither affects any conclusion.
Status
- Version of record. The Wiley landing page and PDF (reached through doi.org) returned HTTP 403 to automated requests. I checked the publisher-typeset PDF deposited in the University of Cambridge institutional repository (handle 1810/291579), which has the Ann Neurol 2014;75:186–195 layout. Its abstract, Results and Table 3 carry the same values as PMC: 0.002 in the abstract and Results, and 13.439, 5, 2.688, 0.02 in the Table 3 row. The PMC HTML page and the Europe PMC full-text XML agree. Every quotation here was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The PMC/Europe PMC record declares no supplementary material, and neither the PMC page nor the typeset PDF lists supporting information. No patient-level data or model output is available to settle which value the software produced.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, PMC notices or Europe PMC notices. The Crossref record has no update-to or updated-by fields, and the PubMed record (PMID 24523223) has no CommentsCorrections entries. A web search for the title with "erratum", "correction" or "PubPeer" returned only the article and repository copies. The PubPeer API returned no record (HTTP 404) for the DOI. Two checks were inconclusive: a Europe PMC DOI search returned HTTP 503 on repeated tries, and the publisher's page could not be checked directly for notices. - Sources. PMC article PMC4112718; Europe PMC full-text XML; typeset PDF in the University of Cambridge repository (handle 1810/291579); Crossref; PubMed; Europe PMC; PubPeer; https://doi.org/10.1002/ana.24116.
Suggested correction
In the abstract, replace
There was also a significant interaction between mitochondrial genome and age (p = 0.002)
with
There was also a significant interaction between mitochondrial genome and age (p = 0.02)
and in the Results, section "Interaction between mtDNA and Aging", replace "(p = 0.002)" with "(p = 0.02)", subject to confirmation against the original model output. Since the corrected value is close to the significance threshold, the authors may also wish to reconsider the description "a strong interaction" and to say whether the three mtDNA omnibus tests were adjusted for multiplicity. In Table 3 the hypotension F value should probably read 4.002 rather than 4.4002.
Recomputation script (python recompute.py.txt)
"""Recompute the Table 3 p-values of PMC4112718
(Ann Neurol 2014;75:186-195, doi:10.1002/ana.24116,
"Mitochondrial DNA and traumatic brain injury").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 3 "Model Effects": Chi-Square, df, F and p for every model term.
Abstract and Results ("Interaction between mtDNA and Aging"): the age-by-haplogroup
interaction is given as p = 0.002.
Methods: cohort of 1,094 patients (used only to bracket the F-test denominator df).
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (Table 3)
# (term, chi-square, df, F as printed, p as printed)
ROWS = [
("mtDNA haplogroup, overall effect", 15.601, 5, 3.120, "0.008"),
("Age", 132.52, 1, 132.52, "0.000"),
("Motor score of GCS", 46.739, 6, 7.790, "0.000"),
("CT Marshall grade", 59.238, 5, 11.856, "0.000"),
("Traumatic SAH", 10.149, 1, 10.149, "0.001"),
("Hypotension", 4.002, 1, 4.4002, "0.046"),
("Hypoxia", 0.945, 1, 0.945, "0.331"),
("Pupillary reactivity", 9.112, 2, 4.556, "0.011"),
("ApoE genotype", 0.507, 1, 0.507, "0.476"),
("mtDNA haplogroup x age", 13.439, 5, 2.688, "0.02"),
("mtDNA haplogroup x APOE e4", 21.35, 5, 4.27, "0.001"),
]
P_TEXT_AGE = 0.002 # Abstract and Results text, age-by-haplogroup interaction
N_PATIENTS = 1094
DEN_DF = 1000 # nominal F denominator df; bracketed below
def half_unit(pstr):
"""Rounding interval implied by the number of printed decimals."""
dec = len(pstr.split(".")[1])
h = 0.5 * 10 ** (-dec)
v = float(pstr)
return max(v - h, 0.0), v + h
# ---------------------------------------------------------------- 1. every Table 3 row
print("=== 1. Table 3: printed p vs p recomputed from the printed statistic ===")
print(f"{'term':34s} {'chi2':>8s} {'df':>3s} {'chi2/df':>8s} {'F_print':>8s} "
f"{'p_chi2':>9s} {'p_F(df,1000)':>12s} {'p_print':>7s} {'within rounding?':>17s}")
REC = {}
for name, c, df, F, p in ROWS:
pc = stats.chi2.sf(c, df)
pf = stats.f.sf(F, df, DEN_DF)
lo, hi = half_unit(p)
ok = lo <= pc < hi
REC[name] = dict(chi2=c, df=df, F=F, p_print=p, p_chi2=pc, p_F=pf, lo=lo, hi=hi, ok=ok)
print(f"{name:34s} {c:8.3f} {df:3d} {c/df:8.3f} {F:8.4f} {pc:9.5f} {pf:12.5f} {p:>7s} {str(ok):>17s}")
# ---------------------------------------------------------------- 2. the age interaction
print("\n=== 2. Age-by-haplogroup interaction ===")
c, df = 13.439, 5
pc = stats.chi2.sf(c, df)
print(f"chi2 = {c}, df = {df}: p = {pc:.5f} (Table 3 prints 0.02; Abstract/Results print {P_TEXT_AGE})")
print(f"F = chi2/df = {c/df:.4f} (Table 3 prints 2.688)")
for d2 in (500, 800, 1000, N_PATIENTS, 10**6):
print(f" F = 2.688 on (5, {d2}) df: p = {stats.f.sf(2.688, 5, d2):.5f}")
print(f"chi2 needed for p = 0.002 at df 5: {stats.chi2.isf(0.002, 5):.3f}")
print(f"chi2 needed for p = 0.0025 (upper rounding edge of 0.002) at df 5: {stats.chi2.isf(0.0025, 5):.3f}")
print(f"F needed for p = 0.002 on (5, {DEN_DF}) df: {stats.f.isf(0.002, 5, DEN_DF):.3f}")
print(f"ratio stated p / recomputed p: {P_TEXT_AGE / pc:.3f} (recomputed is {pc / P_TEXT_AGE:.1f} times the stated value)")
print("p for chi2 = 13.439 at other df (is there a df that yields 0.002?):")
for d in range(1, 7):
print(f" df {d}: p = {stats.chi2.sf(c, d):.5f}")
print("Two-sided vs one-sided: chi-square and F tests are upper-tail only; halving would give "
f"{pc/2:.4f}, still not 0.002.")
# ---------------------------------------------------------------- 3. context numbers
print("\n=== 3. Context ===")
print(f"Bonferroni x3 (three mtDNA omnibus tests: overall, x age, x APOE) on recomputed p: "
f"{min(1, 3 * pc):.4f}; on stated 0.002: {3 * P_TEXT_AGE:.4f}")
for k in ("mtDNA haplogroup, overall effect", "mtDNA haplogroup x APOE e4"):
print(f"Bonferroni x3 on {k}: {3 * REC[k]['p_chi2']:.4f}")
print(f"CT Marshall: chi2/df = {59.238/5:.4f}, Table 3 F = 11.856")
print(f"Hypotension: chi2/df = {4.002/1:.4f}, Table 3 F = 4.4002; "
f"p(chi2 4.002, 1) = {stats.chi2.sf(4.002, 1):.4f}, p(F 4.4002; 1, {DEN_DF}) = {stats.f.sf(4.4002, 1, DEN_DF):.4f}")
lo_c, hi_c = 4.0015, 4.0025
print(f"Hypotension p over chi2 rounding range [{lo_c}, {hi_c}): {stats.chi2.sf(hi_c, 1):.5f} to {stats.chi2.sf(lo_c, 1):.5f} (rounds to 0.045; printed 0.046)")
n_ok = sum(1 for r in REC.values() if r["ok"])
print(f"Rows whose printed p is within rounding of chi2 p: {n_ok} of {len(REC)}")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f"wrote {stem}.svg and {stem}.png")
# ---- Figure 1: dot-and-interval of printed vs recomputed p, log scale
# The age interaction is drawn on two rows: the value printed in the Abstract/Results,
# and the value printed in Table 3 for the same model term.
order = [
("TEXT", "Haplogroup \u00d7 age: Abstract, Results"),
("mtDNA haplogroup x age", "Haplogroup \u00d7 age: Table 3"),
("mtDNA haplogroup, overall effect", "Haplogroup, overall effect"),
("mtDNA haplogroup x APOE e4", "Haplogroup \u00d7 APOE interaction"),
("Traumatic SAH", "Traumatic SAH"),
("Pupillary reactivity", "Pupillary reactivity"),
("Hypotension", "Hypotension"),
("Hypoxia", "Hypoxia"),
("ApoE genotype", "APOE genotype"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.0))
ax.set_xscale("log")
ax.set_xlim(2e-4, 1.0)
ys = np.arange(len(order))[::-1].astype(float)
ys[2:] -= 0.4 # gap between the two focus rows and the rest of Table 3
ax.axhspan(ys[1] - 0.45, ys[0] + 0.45, color="#f0efeb", zorder=0)
r_age = REC["mtDNA haplogroup x age"]
for (key, label), y in zip(order, ys):
if key == "TEXT":
lo2, hi2 = half_unit("0.002")
ax.plot([lo2, hi2], [y, y], color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(P_TEXT_AGE, y, marker="D", ms=9, color=ORANGE, ls="none")
ax.plot(r_age["p_chi2"], y, marker="o", ms=8, mfc="none", mec=BLUE, mew=2, ls="none")
ax.annotate("", xy=(r_age["p_chi2"] * 0.86, y), xytext=(hi2 * 1.08, y),
arrowprops=dict(arrowstyle="->", color=INK2, lw=1.2))
ax.text(np.sqrt(hi2 * r_age["p_chi2"]), y + 0.18,
f"{r_age['p_chi2'] / P_TEXT_AGE:.1f}\u00d7",
ha="center", va="bottom", fontsize=10, color=INK)
continue
r = REC[key]
ax.plot([r["lo"], r["hi"]], [y, y], color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(float(r["p_print"]), y, marker="s", ms=8, color=ORANGE, ls="none")
ax.plot(r["p_chi2"], y, marker="o", ms=8, mfc="none", mec=BLUE, mew=2, ls="none")
ax.set_yticks(ys)
ax.set_yticklabels([lab for _, lab in order])
ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (4, 3)), zorder=1)
ax.text(0.05 * 1.08, ys[-1] - 0.55, "p = 0.05", color=INK2, fontsize=10, va="center", ha="left")
ax.set_ylim(ys[-1] - 0.8, ys[0] + 0.8)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s_ in ("top", "right", "left"):
ax.spines[s_].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("p-value (log scale)")
handles = [
Line2D([], [], color=ORANGE, marker="D", ms=9, lw=2, label="p printed in Abstract and Results (bar = rounding range)"),
Line2D([], [], color=ORANGE, marker="s", ms=8, lw=2, label="p printed in Table 3 (bar = rounding range)"),
Line2D([], [], color=BLUE, marker="o", ms=8, mfc="none", mew=2, lw=0,
label="p recomputed from Table 3 chi-square and df"),
]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.38, -0.13), ncol=1,
frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig1")
# ---- Figure 2: text-vs-table diagram
fig = plt.figure(figsize=(7.2, 4.3))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.3)
ax.axis("off")
def panel(x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.15, y + h - 0.15, title, fontsize=10.5, fontweight="bold", va="top", color=INK)
yy = y + h - 0.5
for txt, col, bold in lines:
ax.text(x + 0.15, yy, txt, fontsize=10, va="top", color=col,
fontweight="bold" if bold else "normal")
yy -= 0.27
panel(0.15, 2.55, 3.3, 1.6, "Abstract (Results)", [
("\u201cThere was also a significant", INK2, False),
("interaction between mitochondrial", INK2, False),
("genome and age", INK2, False),
("(p = 0.002)\u201d", ORANGE, True),
], ORANGE)
panel(0.15, 0.35, 3.3, 1.95, "Results", [
("section \u2018Interaction between mtDNA", INK2, False),
("and Aging\u2019:", INK2, False),
("\u201c\u2026revealed a strong interaction", INK2, False),
("between age and mtDNA haplogroups", INK2, False),
("(p = 0.002)\u201d", ORANGE, True),
], ORANGE)
panel(4.0, 1.55, 3.05, 2.45, "Table 3, same model term", [
("Interaction between mtDNA", INK2, False),
("haplogroup and age", INK2, False),
("Chi-square 13.439 df 5 F 2.688", INK, False),
("p = 0.02", ORANGE, True),
("Recomputed:", BLUE, True),
(f"P(\u03c7\u00b2\u2085 \u2265 13.439) = {pc:.4f}", BLUE, True),
(f"p = 0.002 would need \u03c7\u00b2 = {stats.chi2.isf(0.002, 5):.1f}", BLUE, False),
], INK2)
for ystart in (3.35, 1.3):
ax.add_patch(FancyArrowPatch((3.47, ystart), (3.98, 2.75), arrowstyle="<->", mutation_scale=12,
color=INK, lw=1.5, connectionstyle="arc3,rad=0"))
ax.text(5.52, 1.2, "Text and table disagree tenfold;", ha="center", fontsize=10.5,
fontweight="bold", color=INK)
ax.text(5.52, 0.92, "the table agrees with its statistic", ha="center", fontsize=10.5,
fontweight="bold", color=INK)
ax.text(5.52, 0.55, "Orange: printed. Blue: recomputed.", ha="center",
fontsize=10, color=INK2)
save(fig, "fig2")
The abstract's anxiety odds ratio (5.80) lies outside its own 95% CI (1.19 to 4.22)
Ann Rheum Dis (2017) · doi:10.1136/annrheumdis-2017-212000 · PMC5867422
Summary
The abstract reports that high fear scores on the FAIR questionnaire were associated with high HADS anxiety, "OR 5.80, 95% CI (1.19 to 4.22)". The point estimate is above the upper limit of its own interval, which a Wald interval cannot produce. The geometric centre of the printed interval is 2.24, while the abstract's other four odds ratios sit at the centres of their intervals to within rounding. Figure 3, the paper's forest plot, splits HADS anxiety into two categories. Read by pixel, its 8–10 vs <8 row has an interval of about 1.20 to 4.18, which matches the abstract's interval, and its >10 vs <8 row has an OR of about 5.97, which matches the abstract's 5.80. The abstract therefore appears to pair the estimate of one category with the interval of the other, without naming either. Under either pairing the association remains positive and its interval excludes 1, but the size of the effect given in the abstract is not a valid estimate-interval pair.
What the paper reports
Abstract, Results (identical in the PMC archive and in the publisher's typeset PDF):
High fear scores were associated with high Arthritis Helplessness Index scores (OR 6.85, 95% CI (3.95 to 11.87)); high Hospital Anxiety and Depression Scale anxiety (OR 5.80, 95% CI (1.19 to 4.22)) and depression (OR 2.37, 95% CI (1.29 to 4.37)) scores; low education level (OR 3.48, 95% CI (1.37 to 8.83)); and high perceived disease activity (OR 2.36, 95% CI (1.10 to 5.04)).
Methods, statistical analysis:
Univariate, then multivariate logistic regression was used to identify patient variables (including demographic, social and economic characteristics; disease status, and anxiety/depression and helplessness levels) independently associated with the highest compared with the lowest fear score cluster.
Results, "Factors associated with fear scores" (the text gives no numbers and refers to Figure 3):
Cluster 1 (high fear scores) was associated with higher global rating of disease activity by the patient, high AHI helplessness scores and high HADS anxiety and depression scores.
Figure 3 caption:
High scores indicate those >75 (shown in red); moderate scores, 50–75 (shown in blue); and low scores, <50. Data are presented as ORs (95% CIs).
Figure 3 itself is an image. Its HADS anxiety block has two rows, labeled "8-10 vs <8" and ">10 vs <8". It shows markers on a log axis from 0.1 to 10 and prints no numbers.
Discussion, second and third paragraphs:
In contrast, a strong association was observed between FAIR scores and scores on the AHI (≥20) or HADS (≥10 for anxiety and ≥8 for depression), all of which are non-specific markers of psychological distress.
In agreement with this, we observed a robust association between fears and non-specific measures of psychological distress, such as the AHI, the HADS anxiety score and, to a lesser extent, the HADS depression score.
What the paper's own numbers show
Method. An odds ratio from logistic regression with a Wald interval satisfies CI = exp(b ∓ 1.96·SE) with OR = exp(b). So the OR must lie inside its interval and equal the interval's geometric centre, √(lower × upper), up to rounding. Each arm of the interval implies an SE(log OR): ln(OR/lower)/1.96 for the lower arm and ln(upper/OR)/1.96 for the upper arm. The two must agree and both must be positive. I applied this to all five odds ratios in the abstract.
Figure 3 gives the HADS anxiety association in two categories, so I also measured its red markers (high vs low fear cluster) by pixel position. I calibrated the log axis on its tick marks at 0.1, 1 and 10, which are exactly 194 px per decade apart, so 1 px is about 1.2% of an OR. As a check on the method, the four abstract ORs that correspond to a single Figure 3 row read 0.4% to 2.1% high, and their lower limits match to within 0.02. Upper ends beyond 10 are cut off at the edge of the plot, as the AHI row shows: its printed upper limit is 11.87, but the line stops at about 9.8. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| HADS anxiety OR and 95% CI | Abstract | 5.80 (1.19 to 4.22) | OR lies outside its CI: 1.58 above the upper limit (a factor of 1.37) |
| Geometric centre of the HADS anxiety CI | Abstract | — | √(1.19 × 4.22) = 2.24; reported OR / centre = 2.59 |
| SE(log OR) implied by each arm, HADS anxiety | derived | — | lower arm 0.808, upper arm −0.162 (a negative SE is impossible); symmetric SE 0.323 |
| Interval of the printed width centred on 5.80 | derived | — | 3.08 to 10.92 |
| AHI OR and 95% CI | Abstract | 6.85 (3.95 to 11.87) | centre 6.85; arms 0.281 / 0.280 (consistent) |
| HADS depression OR and 95% CI | Abstract | 2.37 (1.29 to 4.37) | centre 2.37; arms 0.310 / 0.312 (consistent) |
| Low education OR and 95% CI | Abstract | 3.48 (1.37 to 8.83) | centre 3.48; arms 0.476 / 0.475 (consistent) |
| High perceived activity OR and 95% CI | Abstract | 2.36 (1.10 to 5.04) | centre 2.35; arms 0.389 / 0.387 (consistent) |
| HADS anxiety 8–10 vs <8 | Figure 3 (image, no numbers) | marker and line | OR ≈ 2.28, line ≈ 1.20 to ≈ 4.18: matches the abstract's interval and its centre (2.24) |
| HADS anxiety >10 vs <8 | Figure 3 (image, no numbers) | marker and line | OR ≈ 5.97, line ≈ 3.03 to ≥ 9.94 (cut off at 10): matches the abstract's point estimate (5.80) |
| Figure-reading check, AHI / depression / education / activity | Figure 3 vs Abstract | 6.85 / 2.37 / 3.48 / 2.36 | read as 6.88 / 2.42 / 3.52 / 2.41 (ratio to printed 1.004 to 1.021) |
| Significance, if the OR is 2.24 with the printed CI | derived | — | z = 2.50, p ≈ 0.013 |
| Significance, if the OR is 5.80 with an SE like the printed CI's | derived (illustration only) | — | 3.08 to 10.92; z = 5.44 |
The lower end of the Figure 3 >10 row (≈ 3.03) lies close to the lower limit of an interval of the printed width centred on 5.80 (3.08). Its lower arm implies an SE of 0.346, against 0.323 for the printed interval. That fits the reading that 5.80 is the >10 estimate and that its own interval was replaced by the 8–10 interval.
A separate, smaller inconsistency turned up in the same abstract. It does not affect the odds ratios:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Mean FAIR score, RA | Abstract vs Results | Abstract: "51.2 (±25.4) in RA"; Results: "51.8±25.4" | Overall mean 54.9 with axSpA 60.5 (n = 432 and 240) implies an RA mean of 51.79, so 51.8 fits. With 51.2, the overall mean would be 54.52, not 54.9 |
Figure
Figure 1. HADS anxiety odds ratios for the high vs low fear cluster, on one log axis with the null value (OR = 1) as a thin vertical line. Orange marks are numbers printed in the abstract. Blue marks are recomputed from them: the geometric centre of the printed interval (2.24) and an interval of the printed width centred on the printed OR (3.08 to 10.92). Gray hollow diamonds are approximate readings of the two HADS anxiety rows of the paper's Figure 3, measured by pixel position; the >10 row's line runs to the edge of that plot at 10, so its upper end is a lower bound. The printed pair (top row) is impossible because 5.80 lies above 4.22. Each of the two consistent pairings agrees with one Figure 3 row, and both lie wholly above 1.
Figure 2. The abstract's estimate and interval trace to different rows of Figure 3. The printed interval, 1.19 to 4.22, matches the 8–10 vs <8 row (≈ 1.20 to ≈ 4.18). The printed point estimate, 5.80, matches the >10 vs <8 row (≈ 5.97). The abstract names neither category. Figure 3 values are approximate readings, because the figure prints no numbers.
Why it matters
The HADS anxiety estimate is one of five headline associations in the abstract, and for most readers the abstract is the only place it appears as a number. The Results text gives no odds ratios, and Figure 3 shows them only graphically.
- The printed pair cannot be used as published. A reader or meta-analyst who takes the SE from the printed interval and the estimate from the printed OR gets a log OR that is about one SE above the interval's upper limit. A reader who back-calculates from either arm alone gets an SE of 0.808 or a negative one.
- The size of the anxiety effect depends on which number is right. With the >10 category the OR is about 5.8, close to the AHI estimate (6.85) and well above depression (2.37). With the 8–10 category it is about 2.2–2.3, about the same as depression. The Discussion's statement that the association is with "the HADS anxiety score and, to a lesser extent, the HADS depression score" holds for the >10 category but not for the 8–10 category. The abstract does not say which category its number refers to, and readers have no way to tell.
- Direction and significance are not affected. Both consistent pairings lie wholly above 1 (z = 2.50 for 2.24 with the printed interval). Figure 3, which appears to be drawn correctly, shows the same. The abstract's conclusion that "High fear scores were associated with psychological distress" is unaffected.
Explanations considered
- Conversion or typesetting artifact in PMC. This is excluded. The publisher's typeset PDF (InDesign file, created 2018-01-11) has the same numbers, as do the PMC HTML, the Europe PMC XML and the PubMed record. The numbers involve no minus signs, superscripts or table cells that a conversion could garble.
- An asymmetric (profile-likelihood or exact) interval. This does not account for the numbers. An asymmetric interval still contains its own estimate, and 5.80 lies outside 1.19 to 4.22. The other four abstract intervals are symmetric on the log scale (their two arms differ by at most 0.002 in SE units), which indicates that Wald intervals were used.
- A different confidence level (90% or 99%). This does not account for the numbers. Any interval for an estimate contains that estimate, whatever its level.
- A single-digit misprint in the OR. This does not fit. With the printed interval, an OR of 2.80, 3.80, 4.80 or 5.30 gives arms of 0.437/0.209, 0.592/0.053, 0.712/−0.066 and 0.762/−0.116. Only a value near 2.24 makes the arms agree, and 2.24 is not one keystroke away from 5.80.
- A misprint in the interval. This is possible but less likely than the pairing explanation. The printed interval is not arbitrary: it matches the Figure 3 8–10 row to within the precision of the reading at both ends.
- The abstract's estimate and interval come from the two HADS anxiety categories of Figure 3. This accounts for all the numbers. The interval matches the 8–10 vs <8 row, and the point estimate matches the >10 vs <8 row. Which category the abstract meant to report cannot be confirmed without the regression output. The words "high ... anxiety" and the Discussion's cut-off of "≥10 for anxiety" suggest the upper category. Even so, ≥10 is not the same as the figure's >10, and a score of 10 falls in the figure's 8–10 row.
- Imprecise figure reading. This does not change the conclusion. The impossibility of the printed pair follows from the abstract's numbers alone. The figure readings agree with the four unambiguous abstract ORs to within about 2%, while the two HADS anxiety rows differ from each other by a factor of about 2.6.
Status
- Version of record. I checked the publisher's typeset PDF (6 pages, Adobe InDesign CS6, created 2018-01-11, from the PMC open-access bucket,
PMC5867422.1.pdf). Its abstract reads "Scale anxiety (or 5.80, 95% CI (1.19 to 4.22))" and "51.2 (±25.4) in rA", where "or" and "rA" are the PDF's small-caps text, and its Results read "(51.8±25.4; 52 (33–71))". The publisher's web pages (ard.bmj.com, ard.eular.org) returned HTTP 403 to automated access. The PMC HTML, Europe PMC full-text XML and the PubMed record (PMID 29102955) carry the same values. The article appeared online in 2017 and in print in 2018 (77(2):258–263). Every quote above was confirmed verbatim against the PMC-archived text. - Figure 3. I measured the publisher's image
annrheumdis-2017-212000f03.jpg(703 × 1015 px) from Europe PMC.recompute.py.txt --figure <file>repeats the measurement, and the script records the file's SHA-256 hash. - Supplements. The four supplementary files (a flow chart, a correlation figure, ROC curves and tables of the questionnaire items) contain no odds ratios and do not resolve the inconsistency.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, PMC and Europe PMC list no correction or comment links, PubMed has no CommentsCorrections entry, and the typeset PDF carries no notice. PubPeer's API and search page refused automated access (Cloudflare challenge and HTTP 403). A web search found no PubPeer entry and no discussion of these numbers.
- Sources. PMC article PMC5867422; publisher PDF via the PMC open-access bucket; Figure 3 and supplementary files via Europe PMC; Crossref; Europe PMC; PubMed; PubPeer (attempted).
Suggested correction
- In the abstract, replace "OR 5.80, 95% CI (1.19 to 4.22)" with an estimate and interval from the same row of the regression output, and name the category. For example: "HADS anxiety >10 vs <8 (OR 5.80, 95% CI (x to y))", or the 8–10 vs <8 estimate with 1.19 to 4.22, whichever was intended. Reporting both categories would remove the ambiguity.
- Give the Figure 3 odds ratios and intervals as numbers, in the figure or in a table (online if necessary), so that readers do not have to read them off the plot. Extending the axis past 10 would show the full AHI and HADS >10 intervals.
- Make the HADS anxiety cut-off consistent between the Discussion ("≥10 for anxiety") and Figure 3 (">10").
- Minor: change the abstract's RA mean FAIR score from 51.2 to 51.8, which agrees with the Results and with the overall mean of 54.9.
Recomputation script (python recompute.py.txt)
"""Recompute the internal consistency of the odds ratios in the abstract of PMC5867422
(Ann Rheum Dis 2017, doi:10.1136/annrheumdis-2017-212000, the FAIR questionnaire paper).
Run: python recompute.py.txt # prints all numbers, writes fig1/fig2 (.svg, .png)
python recompute.py.txt --candidates # also writes five candidate figure designs
# to candidates/ (A-E) for review
python recompute.py.txt --figure f03.jpg # re-measures Figure 3 from the publisher's image
Needs numpy and matplotlib.
Inputs are copied from the paper as archived in PMC (identical in the publisher's typeset PDF):
Abstract (Results): the five odds ratios with 95% CIs, the RA and axSpA mean scores;
Results ("Distribution of scores"): overall mean 54.9, RA 51.8, axSpA 60.5; n = 432 RA, 240 axSpA.
Method: an odds ratio from logistic regression with a Wald interval satisfies
CI = exp(b -/+ 1.959964 * SE), OR = exp(b),
so the OR must lie inside its CI and equal the geometric centre sqrt(lower * upper)
(up to rounding), and SE(log OR) = (ln upper - ln lower) / (2 * 1.959964).
Figure 3 (the paper's forest plot) gives the HADS anxiety OR in two categories (8-10 vs <8 and
>10 vs <8) but prints no numbers. Its red markers (high vs low fear cluster) were read by pixel
position on the log axis, calibrated on the tick marks at 0.1, 1 and 10 (x = 145.5, 339.5, 533.5 px;
194 px per decade, so 1 px is about 1.2% of an OR). The readings stored below were produced by
read_figure3() on the image file annrheumdis-2017-212000f03.jpg (703 x 1015 px, SHA-256
903d9a0e66f16d94a68154971c8494fc339ad26b9623da0327d18fab51888df0), downloaded from
https://europepmc.org/api/fulltextRepo?pprId=PMC5867422&type=FILE&fileName=annrheumdis-2017-212000f03.jpg&mimeType=image/jpeg
Pass --figure <path> to repeat the measurement. Interval ends drawn past OR = 10 are cut off at
the edge of the plot, so an upper end read near 10 is a lower bound for the true limit.
"""
import os
import sys
from math import exp, log, sqrt, erf
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.ticker import FixedLocator, NullLocator, FuncFormatter # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = 1.959964
# ---------------------------------------------------------------- inputs from the paper
ABSTRACT = { # label: (OR, lower, upper), Abstract (Results)
"Helplessness (AHI)": (6.85, 3.95, 11.87),
"HADS anxiety": (5.80, 1.19, 4.22),
"HADS depression": (2.37, 1.29, 4.37),
"Low education": (3.48, 1.37, 8.83),
"High perceived activity": (2.36, 1.10, 5.04),
}
# Figure 3 rows matching each abstract OR (red markers, cluster 1 vs cluster 3)
FIG_ROW_FOR = {
"Helplessness (AHI)": "AHI >=20 vs <20",
"HADS depression": "HADS depression >=8 vs <8",
"Low education": "Primary vs tertiary education",
"High perceived activity": "Patient-rated activity >60 vs <=30",
}
FIG_LABELS = ["AHI >=20 vs <20", "HADS anxiety 8-10 vs <8", "HADS anxiety >10 vs <8",
"HADS depression >=8 vs <8", "Patient-rated activity 30-60 vs <=30",
"Patient-rated activity >60 vs <=30", "Foreign vs France born",
"Primary vs tertiary education", "Secondary vs tertiary education",
"Employed vs retired", "Unemployed vs retired", "Living alone vs with others"]
# Stored readings (point, left end, right end), produced by read_figure3() on the file above
FIG3_STORED = {
"AHI >=20 vs <20": (6.88, 3.94, 9.82),
"HADS anxiety 8-10 vs <8": (2.28, 1.20, 4.18),
"HADS anxiety >10 vs <8": (5.97, 3.03, 9.94),
"HADS depression >=8 vs <8": (2.42, 1.31, 4.23),
"Patient-rated activity 30-60 vs <=30": (1.85, 1.02, 3.30),
"Patient-rated activity >60 vs <=30": (2.41, 1.11, 4.99),
"Foreign vs France born": (2.30, 1.09, 4.65),
"Primary vs tertiary education": (3.52, 1.37, 8.72),
"Secondary vs tertiary education": (1.67, 0.90, 2.89),
"Employed vs retired": (3.54, 1.78, 6.72),
"Unemployed vs retired": (3.02, 1.31, 6.64),
"Living alone vs with others": (2.34, 1.20, 4.49),
}
N_RA, N_AXSPA = 432, 240
MEAN_ALL, MEAN_AXSPA = 54.9, 60.5
MEAN_RA_ABSTRACT, MEAN_RA_RESULTS = 51.2, 51.8
def read_figure3(path):
"""Measure the red markers of Figure 3 by pixel. Returns {label: (OR, left, right)}."""
import matplotlib.image as mimg
img = mimg.imread(path).astype(int)
R, G, B = img[..., 0], img[..., 1], img[..., 2]
red = (R > 170) & (G < 120) & (B < 120)
red[:, 560:] = False # drop the legend
x1, ppd = 339.5, 194.0 # OR = 1 tick, pixels per decade (ticks 145.5/339.5/533.5)
to_or = lambda x: 10 ** ((x - x1) / ppd) # noqa: E731
rows = np.where(red.sum(1) > 0)[0]
groups, cur = [], [rows[0]]
for r in rows[1:]:
if r - cur[-1] <= 3:
cur.append(r)
else:
groups.append(cur)
cur = [r]
groups.append(cur)
out = {}
for lab, g in zip(FIG_LABELS, groups):
cc = red[g[0]:g[-1] + 1].sum(0)
xs = np.where(cc > 0)[0]
dot = np.where(cc >= 0.8 * cc.max())[0]
out[lab] = (round(to_or(dot.mean()), 2), round(to_or(xs.min()), 2),
round(to_or(xs.max() + 1), 2))
return out
def p_two_sided(z):
return 2 * (1 - 0.5 * (1 + erf(abs(z) / sqrt(2))))
# ---------------------------------------------------------------- 1. abstract ORs
print("=== 1. Abstract ORs: does each OR sit at the geometric centre of its 95% CI? ===")
RES = {}
for k, (o, lo, hi) in ABSTRACT.items():
gm = sqrt(lo * hi)
se_lo = (log(o) - log(lo)) / Z
se_hi = (log(hi) - log(o)) / Z
se_sym = (log(hi) - log(lo)) / (2 * Z)
RES[k] = dict(gm=gm, se_lo=se_lo, se_hi=se_hi, se=se_sym, inside=lo <= o <= hi)
print(f"{k:25s} OR {o:5.2f} CI ({lo:.2f}, {hi:5.2f}) inside={str(lo <= o <= hi):5s} "
f"centre sqrt(lo*hi) = {gm:5.2f} ratio OR/centre = {o / gm:5.3f} "
f"SE from lower arm = {se_lo:6.3f}, upper arm = {se_hi:6.3f}, symmetric = {se_sym:.3f}")
o, lo, hi = ABSTRACT["HADS anxiety"]
a = RES["HADS anxiety"]
hw = (log(hi) - log(lo)) / 2
same_width = (exp(log(o) - hw), exp(log(o) + hw))
print("\n=== 2. HADS anxiety pair ===")
print(f"5.80 exceeds the upper limit 4.22 by {o - hi:.2f} (a factor of {o / hi:.3f}); "
f"on the log scale by {(log(o) - log(hi)) / a['se']:.2f} SE (SE = {a['se']:.3f} from the printed CI)")
print(f"OR / centre of printed CI = {o / a['gm']:.2f}")
print(f"Interval of the printed width centred on 5.80: ({same_width[0]:.2f}, {same_width[1]:.2f})")
zc = log(a["gm"]) / a["se"]
zo = log(o) / a["se"]
print(f"Pairing 1, OR {a['gm']:.2f} (1.19 to 4.22): z = {zc:.2f}, p = {p_two_sided(zc):.4f}")
print(f"Pairing 2, OR 5.80 ({same_width[0]:.2f} to {same_width[1]:.2f}) if SE were the same: "
f"z = {zo:.2f}, p = {p_two_sided(zo):.1e}")
print("Single-digit misprints of the OR, checked against the printed CI (arms must agree):")
for cand in (2.80, 3.80, 4.80, 5.30):
print(f" OR {cand:.2f}: SE from lower arm {(log(cand) - log(lo)) / Z:.3f}, "
f"upper arm {(log(hi) - log(cand)) / Z:.3f}")
# ---------------------------------------------------------------- 3. Figure 3
fig_path = sys.argv[sys.argv.index("--figure") + 1] if "--figure" in sys.argv else None
FIG3 = read_figure3(fig_path) if fig_path else FIG3_STORED
print(f"\n=== 3. Figure 3 red markers ({'measured from ' + fig_path if fig_path else 'stored readings'}) ===")
for lab in FIG_LABELS:
p, l_, r_ = FIG3[lab]
print(f"{lab:38s} OR ~{p:5.2f} line from ~{l_:4.2f} to ~{r_:5.2f}")
print("\nCalibration: abstract value vs Figure 3 reading for the four unambiguous ORs")
ratios = []
for k, lab in FIG_ROW_FOR.items():
o_, l_, h_ = ABSTRACT[k]
p, fl, fr = FIG3[lab]
ratios.append(p / o_)
clip = " (true upper limit beyond the plot edge at 10)" if h_ > 10 else ""
print(f"{k:25s} abstract {o_:5.2f} ({l_:.2f}, {h_:5.2f}) figure {p:5.2f} ({fl:.2f}, {fr:5.2f}) "
f"point ratio {p / o_:.3f}{clip}")
print(f"mean point ratio figure/abstract = {np.mean(ratios):.3f} (range {min(ratios):.3f}-{max(ratios):.3f})")
f810, f10 = FIG3["HADS anxiety 8-10 vs <8"], FIG3["HADS anxiety >10 vs <8"]
print("\nHADS anxiety: which Figure 3 row does each abstract number match?")
print(f"abstract CI (1.19, 4.22) vs 8-10 row line ({f810[1]:.2f}, {f810[2]:.2f}); "
f"vs >10 row line ({f10[1]:.2f}, {f10[2]:.2f})")
print(f"abstract OR 5.80 vs >10 row point {f10[0]:.2f} (ratio {f10[0] / 5.80:.3f}); "
f"vs 8-10 row point {f810[0]:.2f}")
print(f"centre of abstract CI {a['gm']:.2f} vs 8-10 row point {f810[0]:.2f} (ratio {f810[0] / a['gm']:.3f})")
print(f"interval of printed width centred on 5.80 ({same_width[0]:.2f}, {same_width[1]:.2f}) vs "
f">10 row ({f10[1]:.2f}, >= {f10[2]:.2f}, cut off at the plot edge)")
se_fig10 = (log(f10[0]) - log(f10[1])) / Z
print(f">10 row: SE(log OR) from the figure's lower arm = {se_fig10:.3f}; implied upper limit "
f"{exp(log(f10[0]) + Z * se_fig10):.2f}")
# ---------------------------------------------------------------- 4. side check
print("\n=== 4. Side check: RA mean FAIR score (abstract 51.2 vs Results 51.8) ===")
ra_implied = (MEAN_ALL * (N_RA + N_AXSPA) - N_AXSPA * MEAN_AXSPA) / N_RA
print(f"RA mean implied by overall 54.9 and axSpA 60.5: {ra_implied:.2f}")
for m in (MEAN_RA_ABSTRACT, MEAN_RA_RESULTS):
print(f"overall mean if RA = {m}: {(N_RA * m + N_AXSPA * MEAN_AXSPA) / (N_RA + N_AXSPA):.2f}")
# ================================================================= figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GRAY = "#eb6834", "#2a78d6", "#52514e"
plt.rcParams.update({
"font.family": ["DejaVu Sans"], "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path", "lines.linewidth": 2,
})
MS = 9 # marker size in points (>= 8 px)
def save(fig, stem):
fig.savefig(stem + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(stem + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def log_axis(ax, lo=0.7, hi=16, ticks=(1, 2, 3, 5, 10)):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ax.xaxis.set_major_locator(FixedLocator(list(ticks)))
ax.xaxis.set_minor_locator(NullLocator())
ax.xaxis.set_major_formatter(FuncFormatter(lambda v, _: f"{v:g}"))
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.axvline(1, color=INK2, lw=0.9, zorder=1)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, high vs low fear cluster (log scale; thin line = no association)")
def interval(ax, y, pt, l_, r_, ptc, linec, marker="o", hollow=False, arrow=False):
ax.plot([l_, r_], [y, y], color=linec, lw=2, solid_capstyle="butt", zorder=2)
for x in (l_, r_):
ax.plot([x, x], [y - 0.12, y + 0.12], color=linec, lw=2, zorder=2)
if arrow:
ax.annotate("", xy=(r_ * 1.16, y), xytext=(r_, y),
arrowprops=dict(arrowstyle="-|>", color=linec, lw=2, shrinkA=0, shrinkB=0))
ax.plot([pt], [y], marker=marker, ms=MS + (1 if marker == "D" else 2), zorder=3,
mfc=BG if hollow else ptc, mec=ptc, mew=2, ls="none")
def legend_handles(items):
hs = []
for lab, kw in items:
hs.append(plt.Line2D([], [], label=lab, **kw))
return hs
# ---- Candidate A: the printed pair and the centre of its own interval
def fig_core(stem):
fig, ax = plt.subplots(figsize=(7.2, 2.4))
interval(ax, 1, 5.80, 1.19, 4.22, ORANGE, ORANGE)
interval(ax, 0, a["gm"], 1.19, 4.22, BLUE, ORANGE)
ax.set_yticks([1, 0])
ax.set_yticklabels(["Abstract as printed:\nOR 5.80 (1.19 to 4.22)",
f"Centre of the printed CI:\nsqrt(1.19 x 4.22) = {a['gm']:.2f}"], color=INK)
ax.set_ylim(-0.6, 1.7)
log_axis(ax)
ax.annotate("5.80 lies above its own\nupper limit (4.22)", xy=(5.80, 1.12), xytext=(7.4, 1.45),
fontsize=10, color=INK, va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.9))
ax.legend(handles=legend_handles([
("printed in the abstract", dict(color=ORANGE, marker="o", ms=MS, lw=2)),
("recomputed from the printed CI", dict(color=BLUE, marker="o", ms=MS, ls="none"))]),
loc="upper center", bbox_to_anchor=(0.45, -0.32), ncol=2, frameon=False, fontsize=10)
save(fig, stem)
# ---- Candidate B: all five abstract ORs, reported point vs geometric centre
def fig_all_ors(stem):
keys = list(ABSTRACT)
fig, ax = plt.subplots(figsize=(7.2, 3.3))
for i, k in enumerate(keys):
y = len(keys) - 1 - i
o_, l_, h_ = ABSTRACT[k]
ax.plot([l_, h_], [y, y], color=ORANGE, lw=2, zorder=2)
for x in (l_, h_):
ax.plot([x, x], [y - 0.12, y + 0.12], color=ORANGE, lw=2)
ax.plot([RES[k]["gm"]], [y], marker="o", ms=MS + 5, mfc="none", mec=BLUE, mew=2, zorder=4)
ax.plot([o_], [y], marker="o", ms=MS, color=ORANGE, zorder=3)
txt = (f"OR {o_:.2f}, centre {RES[k]['gm']:.2f}" if abs(o_ / RES[k]["gm"] - 1) < 0.01
else f"OR {o_:.2f}, centre {RES[k]['gm']:.2f}: OR outside its CI")
ax.text(17.5, y, txt, va="center", fontsize=10,
color=INK if abs(o_ / RES[k]["gm"] - 1) < 0.01 else INK, weight="normal"
if abs(o_ / RES[k]["gm"] - 1) < 0.01 else "bold")
ax.set_yticks(range(len(keys)))
ax.set_yticklabels(keys[::-1], color=INK)
ax.set_ylim(-0.7, len(keys) - 0.3)
log_axis(ax, 0.7, 16)
ax.legend(handles=legend_handles([
("OR and 95% CI as printed in the abstract", dict(color=ORANGE, marker="o", ms=MS, lw=2)),
("geometric centre of the printed CI (recomputed)",
dict(color=BLUE, marker="o", ms=MS + 5, mfc="none", mew=2, ls="none"))]),
loc="upper center", bbox_to_anchor=(0.5, -0.28), ncol=1, frameon=False, fontsize=10)
save(fig, stem)
# ---- Candidate C: the two arms of each interval, expressed as an implied SE
def fig_arms(stem):
keys = list(ABSTRACT)
fig, ax = plt.subplots(figsize=(7.2, 3.2))
for i, k in enumerate(keys):
y = len(keys) - 1 - i
r = RES[k]
ax.plot([r["se_lo"], r["se_hi"]], [y + 0.13, y - 0.13], color=GRID, lw=2, zorder=1)
ax.plot([r["se_lo"]], [y + 0.13], marker="o", ms=MS, color=BLUE, ls="none", zorder=3)
ax.plot([r["se_hi"]], [y - 0.13], marker="s", ms=MS, mfc=BG, mec=BLUE, mew=2, ls="none", zorder=3)
ax.text(0.95, y, f"{r['se_lo']:.3f} / {r['se_hi']:.3f}", va="center", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.9)
ax.set_xlim(-0.3, 1.2)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_yticks(range(len(keys)))
ax.set_yticklabels(keys[::-1], color=INK)
ax.set_ylim(-0.7, len(keys) - 0.3)
ax.set_xlabel("SE(log OR) implied by each arm of the printed CI\n"
"(lower arm = ln(OR/lower)/1.96; upper arm = ln(upper/OR)/1.96; must agree and be > 0)")
ax.legend(handles=legend_handles([
("from the lower arm", dict(color=BLUE, marker="o", ms=MS, ls="none")),
("from the upper arm", dict(color=BLUE, marker="s", ms=MS, mfc=BG, mew=2, ls="none"))]),
loc="lower right", bbox_to_anchor=(1.0, 1.0), ncol=2, frameon=False, fontsize=10)
save(fig, stem)
# ---- Candidate D: text panels, abstract excerpt vs the two Figure 3 rows
def fig_text_panels(stem):
from matplotlib.patches import FancyBboxPatch
fig = plt.figure(figsize=(7.2, 3.0))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
for x0, w, title in ((0.01, 0.38, "Abstract (Results), as printed"),
(0.60, 0.39, "Figure 3, high vs low fear\n(red markers), read by pixel")):
ax.add_patch(FancyBboxPatch((x0, 0.13), w, 0.84, boxstyle="round,pad=0.0,rounding_size=0.015",
fc="#f3f2ee", ec=GRID, lw=1))
ax.text(x0 + 0.02, 0.88, title, fontsize=10.5, weight="bold", va="center", color=INK)
ax.text(0.03, 0.70, "\u201chigh Hospital Anxiety and\nDepression Scale anxiety", fontsize=10.5,
va="center", color=INK)
ax.text(0.03, 0.53, "(OR 5.80,", fontsize=12, va="center", color=ORANGE, weight="bold")
ax.text(0.03, 0.38, "95% CI (1.19 to 4.22))\u201d", fontsize=12, va="center", color=ORANGE,
weight="bold")
ax.text(0.03, 0.22, "no HADS category is named", fontsize=10, va="center", color=INK2, style="italic")
p1, p2 = FIG3["HADS anxiety 8-10 vs <8"], FIG3["HADS anxiety >10 vs <8"]
ax.text(0.62, 0.66, "HADS anxiety >10 vs <8", fontsize=10.5, va="center", color=INK, weight="bold")
ax.text(0.62, 0.58, f"OR \u2248{p2[0]:.2f} (\u2248{p2[1]:.2f} to \u226510;", fontsize=10.5,
va="center", color=GRAY)
ax.text(0.62, 0.50, "line cut off at the plot edge)", fontsize=10.5, va="center", color=GRAY)
ax.text(0.62, 0.34, "HADS anxiety 8\u201310 vs <8", fontsize=10.5, va="center", color=INK, weight="bold")
ax.text(0.62, 0.26, f"OR \u2248{p1[0]:.2f} (\u2248{p1[1]:.2f} to \u2248{p1[2]:.2f})", fontsize=10.5,
va="center", color=GRAY)
kw = dict(arrowstyle="-|>", color=INK2, lw=1.4, shrinkA=4, shrinkB=4)
ax.annotate("", xy=(0.605, 0.62), xytext=(0.175, 0.53), arrowprops=kw)
ax.annotate("", xy=(0.605, 0.30), xytext=(0.355, 0.38), arrowprops=kw)
ax.text(0.495, 0.60, "point estimate\nmatches", fontsize=10, ha="center", va="center", color=INK,
bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(0.495, 0.27, "interval\nmatches", fontsize=10, ha="center", va="center", color=INK,
bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(0.01, 0.05, "Orange: numbers printed in the abstract. Gray: approximate values read from "
"Figure 3, which prints no numbers.", fontsize=10, color=INK2, va="center")
save(fig, stem)
# ---- Candidate E: printed pair, the two internally consistent pairings, and Figure 3
def fig_pairings(stem):
fig, ax = plt.subplots(figsize=(7.2, 4.1))
p1, p2 = FIG3["HADS anxiety 8-10 vs <8"], FIG3["HADS anxiety >10 vs <8"]
rows = [
(6.0, "Abstract as printed", lambda y: interval(ax, y, 5.80, 1.19, 4.22, ORANGE, ORANGE)),
(4.3, f"Centre of printed CI ({a['gm']:.2f})\nwith the printed CI",
lambda y: interval(ax, y, a["gm"], 1.19, 4.22, BLUE, ORANGE)),
(3.3, "Figure 3: 8–10 vs <8",
lambda y: interval(ax, y, p1[0], p1[1], p1[2], GRAY, GRAY, marker="D", hollow=True)),
(1.6, "Printed OR 5.80 with\nthe printed CI’s width",
lambda y: interval(ax, y, 5.80, *same_width, ORANGE, BLUE)),
(0.6, "Figure 3: >10 vs <8",
lambda y: interval(ax, y, p2[0], p2[1], p2[2], GRAY, GRAY, marker="D", hollow=True, arrow=True)),
]
for y, _, draw in rows:
draw(y)
ax.set_yticks([r[0] for r in rows])
ax.set_yticklabels([r[1] for r in rows], color=INK)
ax.text(0.72, 5.15, "If the interval is right: HADS anxiety 8–10 vs <8", fontsize=10,
weight="bold", color=INK, va="center", bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(0.72, 2.45, "If the point estimate is right: HADS anxiety >10 vs <8", fontsize=10,
weight="bold", color=INK, va="center", bbox=dict(fc=BG, ec="none", pad=1.5))
ax.annotate("5.80 lies above its own\nupper limit (4.22)", xy=(5.80, 6.15), xytext=(7.3, 6.45),
fontsize=10, color=INK, va="center", arrowprops=dict(arrowstyle="-", color=INK2, lw=0.9))
ax.text(11.6, 1.6, f"{same_width[0]:.2f} to\n{same_width[1]:.2f}", fontsize=10, va="center", color=INK)
ax.text(12.2, 0.6, "cut off\nat 10", fontsize=10, va="center", color=INK2)
ax.text(4.6, 4.3, "1.19 to 4.22", fontsize=10, va="center", color=INK)
ax.set_ylim(0.0, 6.9)
log_axis(ax, 0.7, 20, ticks=(1, 2, 3, 5, 10, 20))
ax.legend(handles=legend_handles([
("printed in the abstract", dict(color=ORANGE, marker="o", ms=MS, lw=2)),
("recomputed from the printed numbers", dict(color=BLUE, marker="o", ms=MS, lw=2)),
("read from Figure 3 (approximate)",
dict(color=GRAY, marker="D", ms=MS, mfc=BG, mew=2, lw=2))]),
loc="upper center", bbox_to_anchor=(0.42, -0.2), ncol=3, frameon=False, fontsize=10,
handlelength=2.2, columnspacing=1.2)
save(fig, stem)
fig_pairings(os.path.join(HERE, "fig1"))
fig_text_panels(os.path.join(HERE, "fig2"))
print("\nwrote fig1.svg/png (printed pair vs consistent pairings) and fig2.svg/png (abstract vs Figure 3 rows)")
if "--candidates" in sys.argv:
cdir = os.path.join(HERE, "candidates")
os.makedirs(cdir, exist_ok=True)
for tag, fn in (("A_core_interval", fig_core), ("B_all_abstract_ors", fig_all_ors),
("C_interval_arms", fig_arms), ("D_text_panels", fig_text_panels),
("E_two_pairings", fig_pairings)):
fn(os.path.join(cdir, tag))
print("wrote candidates/A-E (.svg, .png)")
Primary-outcome 95% CI in the TICOSPA trial is incompatible with its reported p-values and sample size
Ann Rheum Dis (2021) · doi:10.1136/annrheumdis-2020-219585 · PMC8522451
Summary
The Results section gives the primary-outcome difference between arms (11.2 percentage points, 47.3% vs 36.1% of patients with at least 30% improvement in ASAS-HI) with a 95% CI of 8.5% to 13.9%. The same sentence calls the difference not statistically significant, with p=0.094 and p=0.079. That interval excludes zero. On the usual normal approximation it implies a standard error of 1.38 pp and p ≈ 4×10⁻¹⁶. The reported p-values instead imply standard errors of 6.4 to 6.7 pp and 95% CIs of about −1.3 to 23.7 pp and −1.9 to 24.3 pp. Three other sources agree with the p-values, not with the printed interval: the trial's size (80 patients per arm), the per-arm 95% CIs the authors later published in a reply to correspondence, and the post-hoc power the authors reported there. The non-significance conclusion is therefore supported. The printed interval, which is the only interval given for the primary outcome, overstates the precision of the estimate about five-fold.
What the paper reports
The estimated percentages of patients achieving an improvement ≥30% on the ASAS-HI at the 1 year visit with the ‘cluster and imbalance-adjusted model’ was 47.3% in the TC/T2T arm and 36.1% in the UC arm. The estimated difference (11.2%, 95% CI 8.5% to 13.9%) was not statistically significant in either model (p=0.094 and p=0.079 for the ‘cluster-adjusted’ and ‘cluster and imbalance-adjusted’ models, respectively).
— Results, Efficacy outcomes, paragraph 1
ASAS-HI significant improvement | 47.3% | 36.1% | NS | NS
— Table 2, primary-outcome row (columns: TC/T2T, UC, cluster-adjusted model, cluster and imbalance-adjusted model)
160 patients were included (80/group). [...] ASAS-HI improved by ≥30% in 47.3% of the TC/T2T arm and in 36.1% of those receiving UC (non-significant).
— Abstract
For an α risk of 5% and a β risk of 80%, with a bilateral test, 77 patients per arm were needed to demonstrate a 20-percentage-point difference among responders [...] For a uniform number of patients per centre, we aimed for a total sample of 160 patients (80 per arm).
— Methods, Sample size estimation, paragraph 1
The inflation factor for our study was 1.45, and thus the sample needed to take the cluster-randomised design into account was 232 patients (116 patients per arm).
— Methods, Sample size estimation, paragraph 2
In total, 144 patients (72 per arm) completed the last visit
— Results, Participant flow
The authors' published reply to correspondence (doi:10.1136/annrheumdis-2021-220913, first published 29 June 2021) adds per-arm intervals and a power calculation:
ASAS HI significant improvement | 47.3% (95% CI 36.2% to 59.0%) | 36.1% (95% CI 25.6% to 46.6%)
— Reply, table "Estimated outcomes at week 48% and 95% CI"
with p1=0.47 and p2=0.36 [...] and an α risk of 5%, the post-hoc-calculated power is 29.9%.
— Reply, paragraph 2
What the paper's own numbers show
Method. I treated the printed interval as a Wald interval: SE = half-width / 1.96. I backed each two-sided p-value out to z = Φ⁻¹(1 − p/2) and SE = 11.2 / z, and rebuilt the 95% CI as 11.2 ± 1.96·SE. As independent checks I used three more inputs: the binomial SE of a difference in proportions at the reported rates and arm sizes, the per-arm CIs in the authors' reply (SE taken from each half-width, arms combined as independent because clusters are nested within arms), and the reply's stated post-hoc power. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Difference in responders | Results, Efficacy outcomes | 11.2% | 47.3 − 36.1 = 11.2 pp (consistent) |
| Midpoint and half-width of the 95% CI | Results, Efficacy outcomes | 8.5% to 13.9% | midpoint 11.2, half-width 2.7 pp (symmetric, risk-difference scale) |
| SE, z and p implied by the printed CI | derived | — | SE 1.38 pp, z = 8.13, p ≈ 4.3×10⁻¹⁶ |
| Same, if the interval were ±1 SE / 90% / 99% | derived | "95%" stated | p = 3.4×10⁻⁵ / 8.9×10⁻¹² / 1.2×10⁻²⁶ (all significant) |
| 95% CI implied by p, cluster and imbalance-adjusted model | Results; Table 2 "NS" | p = 0.079 | z = 1.757, SE 6.38 pp, CI −1.3 to 23.7 pp |
| 95% CI implied by p, cluster-adjusted model | Results; Table 2 "NS" | p = 0.094 | z = 1.675, SE 6.69 pp, CI −1.9 to 24.3 pp |
| Ratio of p-implied SE to CI-implied SE | derived | — | 4.6 (p = 0.079) and 4.9 (p = 0.094) |
| Difference CI from the per-arm 95% CIs | Authors' reply, table | 36.2–59.0% and 25.6–46.6% | SE 5.82 and 5.36 pp → SE 7.91 pp, CI −4.3 to 26.7 pp, p ≈ 0.16 |
| Width of each per-arm CI vs the difference CI | Reply vs Results | 22.8 and 21.0 pp vs 5.4 pp | each arm's interval is about four times wider than the interval for the difference |
| Binomial SE, 80 per arm (no clustering) | Abstract (80/group) | — | SE 7.75 pp, CI −4.0 to 26.4 pp, p = 0.148 |
| Binomial SE, 72 per arm (completers) | Participant flow | — | SE 8.16 pp, CI −4.8 to 27.2 pp, p = 0.170 |
| Binomial SE × inflation factor 1.45 | Sample size estimation | — | SE 9.33 pp, CI −7.1 to 29.5 pp |
| Patients per arm needed for SE 1.38 pp | derived | 80 per arm | about 2529 per arm (5058 in total, against 160 randomised) |
| Post-hoc power, 80 per arm, p1 = 0.47, p2 = 0.36 | Authors' reply | 29.9% | 29.5% with binomial SE 7.74 pp; at SE 1.38 pp, power would be 1 − 3×10⁻¹⁰ |
| Post-hoc power, 116 per arm | Authors' reply | 41% | 40.2% with binomial SE 6.43 pp |
The two printed p-values, the per-arm intervals and the power figures in the reply, and the trial's size all point to an SE of roughly 6–8 pp for the difference. Only the printed 95% CI implies 1.38 pp.
Figure
Figure 1. The primary-outcome difference (TC/T2T minus usual care, ≥30% ASAS-HI improvement, 11.2 pp) on one axis, with the null value 0 as a thin vertical line. Orange: the 95% CI printed in the Results (8.5 to 13.9 pp), which excludes zero and implies p ≈ 4×10⁻¹⁶. Blue, recomputed: the 95% CIs that correspond to the paper's own p-values, 0.079 (cluster and imbalance-adjusted model, −1.3 to 23.7 pp) and 0.094 (cluster-adjusted model, −1.9 to 24.3 pp); the CI obtained by combining the per-arm 95% CIs in the authors' reply (−4.3 to 26.7 pp); and an individual-level binomial check at 80 per arm with no allowance for clustering (−4.0 to 26.4 pp). Every recomputed interval includes zero and is 4.6 to 5.8 times as wide as the printed one.
Figure 2. Standard error of a difference between 47.3% and 36.1% as a function of patients per arm (blue, binomial, no clustering), against the SE implied by the printed 95% CI (orange, 1.38 pp). TICOSPA had 80 patients per arm (SE 7.75 pp). An SE of 1.38 pp would need about 2529 per arm, before any inflation for cluster randomisation.
Why it matters
- The interval belongs to the trial's primary outcome, and it is the only interval reported for that outcome in the article (Table 2 gives only "NS").
- As printed, it describes a precisely estimated benefit of at least 8.5 pp. The intervals supported by the paper's own p-values run from about −1 to 24 pp, so the data are compatible with no effect and with a benefit about twice the point estimate. A reader who extracts the interval on its own, for example for a meta-analysis or a guideline evidence table, would take a non-significant result as a highly significant and precise one. In inverse-variance pooling it would also receive about 21 times its correct weight ((6.38/1.38)² ≈ 21).
- The Discussion compares the observed difference ("differed by 11% in favour of TC/T2T") with the 12% margin it cites from head-to-head trials. Read alongside the printed interval, the 11% difference looks tightly estimated. With an interval of roughly −1 to 24 pp, the data neither confirm nor exclude a difference of that size.
- What is not affected: the point estimate, its direction, and the headline conclusion that TC/T2T "was not significantly superior to UC for the primary outcome". The abstract, Table 2 and the conclusion follow the p-values, and the p-values agree with everything else.
Explanations considered
- A different confidence level. The text says 95%, and no other level appears for this interval. Read as ±1 SE, 90%, 99% or 99.9%, the interval still excludes zero with p of 3×10⁻⁵ or smaller. No confidence level reconciles it with p = 0.079 or 0.094.
- One-sided test. The sample-size section specifies a bilateral test. A one-sided p would also be smaller, which would widen the conflict, not resolve it.
- A different scale (odds ratio, risk ratio, log scale). The interval is centred exactly on 11.2, the difference 47.3 − 36.1, so it is on the risk-difference scale. An interval for a ratio would not be centred on the risk difference, and it would be compared with 1, not 0.
- The two models give different intervals. The interval follows the cluster and imbalance-adjusted estimates (47.3% and 36.1%). That model's own p = 0.079 implies an SE of 6.38 pp, 4.6 times the SE implied by the interval. The cluster-adjusted model is further away still.
- Covariate adjustment or clustering could narrow the interval. Cluster randomisation widens intervals; the paper's own inflation factor is 1.45. Covariate adjustment can narrow an interval modestly, but going from the unadjusted binomial SE of 7.75 pp to 1.38 pp would require the covariates to explain about 97% of the outcome variance. That model's own p-value (0.079) shows no such gain.
- A different quantity (for example a per-arm interval, or the interval for another outcome). The per-arm intervals the authors published later are 21–23 pp wide, not 5.4 pp, and the interval is centred on the difference. I found no other outcome in the article whose reported numbers produce 8.5 to 13.9.
- Conversion or typesetting error. The publisher's typeset PDF shows the same text. At 200 dpi there is no minus sign before 8.5, and the decimal points are ordinary. The PMC HTML, Europe PMC full text and PMC XML carry the same numbers. A lost minus sign would not explain it either: −8.5 to 13.9 is not centred on the reported 11.2.
- The p-values are the error and the CI is correct. This would make the trial strongly positive. It is ruled out by the trial's size (an SE of 1.38 pp would need about 2500 patients per arm), by the authors' own per-arm CIs, and by their post-hoc power of 29.9%, which reproduces only with an SE of about 7.7 pp.
None of these accounts for the printed interval. The p-values, the sample size and the later per-arm intervals agree with one another. The printed 95% CI is the outlier.
Status
As of 2026-10-24:
- Version of record: checked. The publisher's typeset PDF (Ann Rheum Dis 2021;80:1436–1444), obtained from the PMC open-access bucket, prints the same sentence on page 4 (Results, Efficacy outcomes) and "NS NS" in Table 2. The PMC HTML, Europe PMC full-text XML and the PMC XML match it. The publisher's site (ard.bmj.com) returned HTTP 403 to automated requests.
- Supplements: all seven online supplemental files were checked (treatment algorithm, cluster-level flow diagram, baseline imbalance table, economic evaluation, NSAID score and bDMARD initiation, observed secondary outcomes over time, protocol v5.0). None reports the model output, SE or CI for the primary outcome, so none of them settles the question.
- Corrections and retractions: none found. Crossref lists no update-to or updated-by relation; its only relation is an F1000 review (has-review). The PMC page shows no correction notice, and PubMed CommentsCorrections lists only "Comment in" links. The project's automated check (Crossref, PMC and Europe PMC notices) returned none.
- Published correspondence: the journal published two letters on this article (doi:10.1136/annrheumdis-2021-220904, Ann Rheum Dis 2023;82:e166; doi:10.1136/annrheumdis-2021-220938, 2023;82:e222) and an authors' reply (doi:10.1136/annrheumdis-2021-220913, 2023;82:e167). I read the reply in full from the open publisher's version in the Leiden University repository (hdl:1887/3247824). It gives per-arm 95% CIs and post-hoc power, but it does not mention the 8.5% to 13.9% interval and gives no CI for the difference. I could not open the text of the two letters (publisher HTTP 403), so I cannot say whether either one raises this interval. The reply was first published online on 29 June 2021. The final issue PDF (created 24 September 2021) still carries the original interval.
- PubPeer: searched. The PubPeer API returned a human-verification challenge, and web searches for the DOI and title found no PubPeer thread.
Suggested correction
Replace the interval in Results, Efficacy outcomes, paragraph 1, with the 95% CI for the difference that comes from the cluster and imbalance-adjusted mixed model, and state which model it comes from. Ideally, report the interval from the cluster-adjusted model as well. Going by the reported p-values, the corrected interval should be close to −1.3% to 23.7% (cluster and imbalance-adjusted, p = 0.079) and −1.9% to 24.3% (cluster-adjusted, p = 0.094). The exact limits must come from the model output, which only the authors hold. For example:
The estimated difference (11.2%, 95% CI [model-based lower limit, ≈ −1.3%] to [model-based upper limit, ≈ 23.7%]) was not statistically significant in either model (p=0.094 and p=0.079 ...).
No change is needed to the abstract, Table 2 or the conclusions. The Discussion's comparison of the 11% difference with a 12% margin could note that the interval includes both zero and differences of about twice that margin.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC8522451 write-up and redraw the figures.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Inputs are values printed in the paper (Ann Rheum Dis 2021;80:1436-1444,
doi:10.1136/annrheumdis-2020-219585):
- Results, Efficacy outcomes: 47.3% (TC/T2T) vs 36.1% (UC); "11.2%, 95% CI 8.5% to 13.9%";
p=0.094 (cluster-adjusted) and p=0.079 (cluster and imbalance-adjusted)
- Abstract / Participant flow: 160 patients, 80 per arm; 144 (72 per arm) completed
- Methods, Sample size estimation: inflation factor 1.45 (m=10, rho=0.05); 116 per arm targeted
and in the authors' published reply to correspondence (doi:10.1136/annrheumdis-2021-220913,
Ann Rheum Dis 2023;82:e167, first published 29 June 2021):
- table: ASAS-HI significant improvement 47.3% (95% CI 36.2% to 59.0%) vs 36.1% (95% CI 25.6% to 46.6%)
- text: post-hoc power 29.9% (p1=0.47, p2=0.36, alpha 5%, trial sample) and 41% (116 per arm)
Figures are written next to this script.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
Z95 = norm.isf(0.025)
# ---- values printed in the paper -------------------------------------------
p1, p0 = 0.473, 0.361 # estimated responder proportions
lo, hi = 0.085, 0.139 # printed 95% CI of the difference
p_cl, p_cli = 0.094, 0.079 # printed p-values
n_arm, n_comp, n_target = 80, 72, 116
deff = 1.45 # inflation factor from the sample-size section
# ---- values printed in the authors' reply to correspondence ----------------
ci_t2t = (0.362, 0.590) # per-arm 95% CI, TC/T2T
ci_uc = (0.256, 0.466) # per-arm 95% CI, usual care
pp = lambda x: 100 * x
d = p1 - p0
print("== Point estimate")
print(f"difference 47.3 - 36.1 = {pp(d):.1f} pp")
print("\n== What the printed CI implies (Wald, 95%)")
mid, half = (lo + hi) / 2, (hi - lo) / 2
se_ci = half / Z95
z_ci = d / se_ci
print(f"CI midpoint {pp(mid):.2f} pp, half-width {pp(half):.2f} pp (symmetric about 11.2: {abs(mid - d) < 1e-9})")
print(f"implied SE {pp(se_ci):.2f} pp, z = {z_ci:.2f}, two-sided p = {2 * norm.sf(z_ci):.1e}")
print("\n== If the printed interval were at another level")
for lab, zc in [("+-1 SE (68%)", 1.0), ("90%", norm.isf(0.05)), ("99%", norm.isf(0.005)), ("99.9%", norm.isf(0.0005))]:
se = half / zc
print(f" {lab:13s}: SE {pp(se):.2f} pp, z {d / se:.2f}, p = {2 * norm.sf(d / se):.1e}")
print("\n== What the printed p-values imply (Wald, two-sided)")
implied = {}
for lab, p in [("cluster-adjusted", p_cl), ("cluster and imbalance-adjusted", p_cli)]:
z = norm.isf(p / 2)
se = d / z
implied[lab] = (p, se, d - Z95 * se, d + Z95 * se)
print(f" {lab}: p={p} -> z={z:.3f}, SE={pp(se):.2f} pp, 95% CI {pp(d - Z95 * se):.1f} to {pp(d + Z95 * se):.1f} pp,"
f" SE ratio to CI-implied SE {se / se_ci:.2f}")
print(f" 95% CI half-width {pp(Z95 * se):.1f} pp vs printed {pp(half):.1f} pp")
print("\n== Per-arm 95% CIs in the authors' reply, combined (arms treated as independent)")
se_t = (ci_t2t[1] - ci_t2t[0]) / (2 * Z95)
se_u = (ci_uc[1] - ci_uc[0]) / (2 * Z95)
se_r = np.hypot(se_t, se_u)
reply_ci = (d - Z95 * se_r, d + Z95 * se_r)
print(f" SE TC/T2T {pp(se_t):.2f} pp, SE UC {pp(se_u):.2f} pp -> SE of difference {pp(se_r):.2f} pp")
print(f" 95% CI of difference {pp(reply_ci[0]):.1f} to {pp(reply_ci[1]):.1f} pp, p = {2 * norm.sf(d / se_r):.3f}")
print(f" each per-arm CI alone is wider than the printed difference CI: "
f"{pp(ci_t2t[1] - ci_t2t[0]):.1f} and {pp(ci_uc[1] - ci_uc[0]):.1f} pp vs {pp(hi - lo):.1f} pp")
print("\n== Scale checks from the trial's size (individual-level binomial)")
def binom_se(n, a=p1, b=p0):
return np.sqrt(a * (1 - a) / n + b * (1 - b) / n)
rows = {}
for lab, n, infl in [("80 per arm, no clustering", n_arm, 1.0),
("72 per arm (completers), no clustering", n_comp, 1.0),
("80 per arm x inflation factor 1.45", n_arm, deff)]:
se = binom_se(n) * np.sqrt(infl)
rows[lab] = (se, d - Z95 * se, d + Z95 * se, 2 * norm.sf(d / se))
print(f" {lab}: SE {pp(se):.2f} pp, 95% CI {pp(d - Z95 * se):.1f} to {pp(d + Z95 * se):.1f} pp, p = {2 * norm.sf(d / se):.3f}")
n_need = (p1 * (1 - p1) + p0 * (1 - p0)) / se_ci ** 2
print(f" n per arm for SE {pp(se_ci):.2f} pp (binomial, no clustering): {n_need:.0f} (total {2 * n_need:.0f} vs 160 randomised)")
r2 = 1 - (se_ci / binom_se(n_arm)) ** 2
print(f" share of outcome variance covariates would have to explain to shrink SE "
f"from {pp(binom_se(n_arm)):.2f} to {pp(se_ci):.2f} pp: {100 * r2:.1f}%")
print(f" inverse-variance weight of printed CI relative to p=0.079-implied SE: "
f"{(implied['cluster and imbalance-adjusted'][1] / se_ci) ** 2:.1f}x")
w = [b - a for a, b in [implied['cluster and imbalance-adjusted'][2:], implied['cluster-adjusted'][2:], reply_ci,
rows['80 per arm, no clustering'][1:3]]]
print(" recomputed CI widths / printed width: " + ", ".join(f"{x / (hi - lo):.2f}" for x in w))
print("\n== Post-hoc power reported in the reply, reproduced with the binomial SE")
def power(se, dd):
return norm.cdf(dd / se - Z95) + norm.cdf(-dd / se - Z95)
for n, rep in [(n_arm, "29.9%"), (n_target, "41%")]:
se = binom_se(n, 0.47, 0.36)
print(f" {n} per arm, p1=0.47, p2=0.36: SE {pp(se):.2f} pp, power {100 * power(se, 0.11):.1f}% (reply: {rep})")
print(f" power if the SE were {pp(se_ci):.2f} pp (printed CI): 1 - {1 - power(se_ci, d):.1e}")
# ---- figures ------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "axes.edgecolor": INK2,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
# Figure 1: printed CI vs intervals implied by the paper's own p-values, the reply's CIs and the trial size
items = [
("Printed 95% CI\n(Results, Efficacy outcomes)", lo, hi, f"implies p ≈ {2 * norm.sf(z_ci):.0e}", ORANGE, "o"),
("From printed p = 0.079\n(cluster and imbalance-adj.)", *implied["cluster and imbalance-adjusted"][2:], "p = 0.079", BLUE, "o"),
("From printed p = 0.094\n(cluster-adjusted)", *implied["cluster-adjusted"][2:], "p = 0.094", BLUE, "o"),
("From per-arm 95% CIs\n(authors' reply, 2021)", *reply_ci, f"p ≈ {2 * norm.sf(d / se_r):.2f}", BLUE, "D"),
("Binomial check,\n80 per arm, no clustering", *rows["80 per arm, no clustering"][1:3],
f"p ≈ {rows['80 per arm, no clustering'][3]:.2f}", BLUE, "s"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(items))[::-1]
for y, (lab, a, b, note, col, mk) in zip(ys, items):
ax.plot([pp(a), pp(b)], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
for x in (a, b):
ax.plot([pp(x)] * 2, [y - 0.13, y + 0.13], color=col, lw=2, zorder=3)
ax.plot(pp(d), y, marker=mk, ms=9, color=col, mec=BG, mew=1.5, zorder=4)
ax.text(pp(b) + 0.8, y + 0.02, f"{pp(a):.1f} to {pp(b):.1f}\n{note}", va="center", ha="left",
fontsize=10, color=INK2, linespacing=1.15)
ax.axvline(0, color=INK2, lw=1, zorder=2)
ax.text(0.4, len(items) - 0.45, "no difference", ha="left", va="bottom", fontsize=10, color=INK2)
ax.set_yticks(ys)
ax.set_yticklabels([it[0] for it in items], fontsize=10, color=INK)
ax.set_xlim(-10, 40)
ax.set_ylim(-0.55, len(items) - 0.15)
ax.set_xticks(np.arange(-10, 31, 5))
ax.spines["bottom"].set_bounds(-10, 30)
ax.set_xlabel("Difference in ≥30% ASAS-HI responders,\nTC/T2T − UC (percentage points)", x=0.4)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
leg = [Line2D([0], [0], color=ORANGE, lw=2, marker="o", ms=8, mec=BG, label="Reported in the paper"),
Line2D([0], [0], color=BLUE, lw=2, marker="o", ms=8, mec=BG, label="Recomputed from the paper's p-values"),
Line2D([0], [0], color=BLUE, lw=2, marker="D", ms=8, mec=BG, label="Recomputed from the reply's per-arm CIs"),
Line2D([0], [0], color=BLUE, lw=2, marker="s", ms=8, mec=BG, label="Recomputed from the trial size")]
fig.legend(handles=leg, loc="lower center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.0))
fig.tight_layout(rect=(0, 0.12, 1, 1))
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 2: standard error vs patients per arm
ns = np.logspace(np.log10(20), np.log10(5000), 300)
fig, ax = plt.subplots(figsize=(7.2, 3.6), facecolor=BG)
ax.set_facecolor(BG)
ax.plot(ns, pp(binom_se(ns)), color=BLUE, lw=2,
label="Binomial SE of the difference at 47.3% vs 36.1%\n(no clustering; recomputed)")
ax.axhline(pp(se_ci), color=ORANGE, lw=2, label=f"SE implied by the printed 95% CI ({pp(se_ci):.2f} pp)")
ax.plot(n_arm, pp(binom_se(n_arm)), "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.annotate(f"TICOSPA: 80 per arm\nSE {pp(binom_se(n_arm)):.2f} pp", (n_arm, pp(binom_se(n_arm))), xytext=(110, 9.2),
fontsize=10, color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.plot(n_need, pp(se_ci), "o", ms=9, color=ORANGE, mec=BG, mew=1.5, zorder=4)
ax.annotate(f"≈{n_need:.0f} per arm\nneeded", (n_need, pp(se_ci)), xytext=(n_need * 0.95, 4.2), ha="center",
fontsize=10, color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.set_xscale("log")
ax.set_xlim(20, 5000)
ax.set_ylim(0, 16)
ax.set_xticks([20, 50, 100, 200, 500, 1000, 2000, 5000])
ax.set_xticklabels(["20", "50", "100", "200", "500", "1000", "2000", "5000"])
ax.set_xlabel("Patients per arm (log scale)")
ax.set_ylabel("SE of the difference (pp)")
ax.grid(color=GRID, lw=0.8, zorder=0)
for s in ["top", "right"]:
ax.spines[s].set_visible(False)
ax.legend(loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nfigures written to", HERE)
Abstract reports a same-day A&E increase after locum consultations; Table 3 and Results report none
BMC Med (2024) · doi:10.1186/s12916-024-03332-z · PMC10967114
Summary
The abstract says consultations with locum GPs were "4% more likely to be followed by an A&E visit on the same day (OR = 1.04 (1.01 to 1.08))", an interval that excludes 1. The paper's own model output gives a different result for the same outcome and the same adjusted model. Table 3 reports OR 1.02 (0.98 to 1.07), p = 0.331, Additional file 2 (Table S19) reports 1.023 (0.977 to 1.073), and the Results text says there was "no difference in rates of the same day A&E visits". The interval in the abstract implies p ≈ 0.022, while the interval in Table 3 implies p ≈ 0.38, consistent with the printed 0.331. No table in the paper or its supplement contains 1.04 (1.01 to 1.08). As a result, the abstract lists a harm signal that the analysis does not support. The paper's overall conclusion of no systematic difference is not affected.
What the paper reports
Abstract, Results:
Consultations with locum GPs were 22% more likely to involve a prescription for an antibiotic (OR = 1.22 (1.21 to 1.22)), 8% more likely to involve a prescription for a strong opioid (OR = 1.08 (1.06 to 1.09)), 4% more likely to be followed by an A&E visit on the same day (OR = 1.04 (1.01 to 1.08)) and 5% more likely to be followed by an A&E visit within 1 to 7 days (OR = 1.05 (1.02 to 1.08)).
Table 3, "Mixed effects logistic regression for patient outcomes and potentially hazardous prescribing indicators over time, odds ratios", rows 7 and 8:
A&E visits, same day | 1.02 (0.98 to 1.07), < 0.331 [0.028]
A&E visits, within 1 to 7 days | 1.05 (1.02 to 1.08), < 0.001 [0.015]
Results, paragraph 3:
In terms of hospital-related outcomes, a consultation with a locum was 5% more likely to be followed by an A&E visit within 1 to 7 days (OR = 1.05, 95% CI 1.02 to 1.08) but there was no difference in rates of the same day A&E visits, emergency admissions or ACSC emergency admissions.
Results, paragraph 4, which points to the full model output:
The full output from the multilevel regressions is presented in Table S18–S22 in Additional file 2.
Additional file 2, Table S19, "Locum consultations" row, column "A&E visits, same day". The docx prints the < before the p-value as in Table 3:
1.023 (0.977 to 1.073),<0.331[0.028]
Additional file 2, Table S23, sensitivity analysis excluding 2020–2022:
A&E visits, same day | 1.02 (0.97 to 1.07), <0.408 [0.026]
Fig. 1 caption. The figure plots the patient-outcome ORs on a linear axis. Read from pixel positions in the publisher's image (calibrated on the .8 to 1.2 gridlines, about ±0.003), same-day A&E is about 1.021 (0.972 to 1.072), with an interval crossing the dashed line at 1.
Coefficient plot for locum consultations across all outcomes. Note: Results are expressed as odd ratios (OR) and corresponding confidence intervals (CI). When the corresponding CIs cross the dashed vertical line coefficients are not statistically significant
Methods, "Model covariates":
Stata v17 was used for data cleaning, management and analyses and an α level of 1% was used throughout [35].
Abstract, Conclusions:
Despite existing concerns, the clinical practice and performance of locum GPs did not appear to be systematically different from that of permanent GPs.
What the paper's own numbers show
Method. For each printed version of the same-day A&E estimate, I recovered the log-scale standard error from the 95% CI, SE = (ln upper − ln lower) / (2 × 1.959964). I then computed z = ln(OR) / SE and the two-sided Wald p-value. I compared these with the p-values the paper prints, with the Results text and with the paper's stated α of 1%. I also compared every OR in the abstract with Table 3, the Results text and the three-decimal model output in Additional file 2 (Tables S18–S20 and S23). I read selected estimates in the paper's Fig. 1 from pixel positions in the publisher's image (linear axis calibrated on its gridlines; precision about ±0.003). As a final check, I computed crude odds ratios from the counts in Table 1. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Same-day A&E, adjusted OR | Abstract (Results) | 1.04 (1.01 to 1.08), "4% more likely" | interval excludes 1; SE(log) 0.0171, z 2.29, p ≈ 0.022; not significant at the paper's α = 1% |
| Same-day A&E, adjusted OR | Table 3, row 7 | 1.02 (0.98 to 1.07), p 0.331 | interval includes 1; SE(log) 0.0224, z 0.88, p ≈ 0.38 |
| Same-day A&E, adjusted OR (3 dp) | Additional file 2, Table S19 | 1.023 (0.977 to 1.073), p 0.331 | p ≈ 0.34; rounds to Table 3's 1.02 (0.98 to 1.07) |
| Same-day A&E, excluding 2020–2022 | Additional file 2, Table S23 | 1.02 (0.97 to 1.07), p 0.408 | p ≈ 0.43; not the abstract value either |
| Same-day A&E | Results, paragraph 3 | "no difference" | agrees with Table 3 and S19, contradicts the abstract |
| Same-day A&E | Fig. 1 | digitised ≈ 1.021 (0.972 to 1.072), interval crosses OR = 1 | agrees with Table 3 and S19, not with the abstract |
| A&E within 1–7 days | Abstract; Table 3; Results; S19 | 1.05 (1.02 to 1.08) everywhere; S19 1.052 (1.022 to 1.082) | z 3.48, p ≈ 5.0 × 10-4 (printed < 0.001); consistent in all places |
| Same-day A&E, crude | Table 1 | 37,861 (0.31%) permanent vs 4,723 (0.35%) locum | implied denominators 12,334,156 and 1,367,257; crude OR 1.126, which is neither 1.04 nor 1.02 (the abstract describes an adjusted model) |
| Absolute difference at the permanent-GP rate (0.307%) | derived | — | OR 1.04: +0.0122 percentage points (12.2 per 100,000 consultations); OR 1.02: +0.0061 (6.1 per 100,000) |
| Antibiotics | Abstract vs Table 3 vs text vs S18 | 1.22 (1.21 to 1.22) vs 1.21 (1.21 to 1.22) vs 1.21 (1.20 to 1.22) vs 1.213 (1.207 to 1.220) | abstract point estimate is not the rounded model output; the text's lower limit differs from Table 3 |
| Practice revisits | Abstract vs Table 3 vs text vs S18 vs S23 vs Fig. 1 | 0.88 (0.87 to 0.88) vs 0.88 (0.88 to 0.89) vs 0.88 (0.88 to 0.91) vs 0.884 (0.878 to 0.891) vs 0.87 (0.87 to 0.88) vs digitised ≈ 0.877 (0.871 to 0.884) | abstract interval matches none of the main-analysis tables; it matches the S23 interval and Fig. 1 as plotted |
| Hypnotics | Abstract vs Table 3 vs S18 vs Fig. 1 | 0.96 (0.94 to 0.98) vs 0.97 (0.95 to 0.99) vs 0.967 (0.946 to 0.988) vs digitised ≈ 0.962 (0.941 to 0.984) | abstract does not match the rounded model output; Fig. 1 sits closer to the abstract |
| Tests | Abstract vs Table 3 vs S20 | 0.81 (0.80 to 0.82) vs 0.80 (0.80 to 0.81) vs 0.805 (0.796 to 0.814) | 0.805 is a rounding tie for the point; the abstract's upper limit 0.82 does not match 0.814 |
| Prescribing safety indicators | Abstract vs Methods and Table 3 | "eight out of the eleven" not different | Methods define 10 indicators and Table 3 lists A–J; 3 (A, D, H) have p < 0.01, so 7 of 10 are not different |
The same-day A&E estimate is printed in five places. Four of them (Table 3, Table S19, Fig. 1, the Results text) agree with each other: the estimate is about 1.02, its interval spans 1, and p is about 0.33 to 0.38. The abstract alone gives 1.04 with an interval that excludes 1. No version of the model in the paper or its supplement, including the sensitivity analysis, produces 1.04 (1.01 to 1.08). Rounding the three-decimal supplement values reproduces Table 3 for every clinical-practice outcome except tests, where 0.805 is a rounding tie. The abstract differs from those values for five of the eight outcomes (practice revisits, antibiotics, hypnotics, tests and same-day A&E), and only for same-day A&E does the difference change whether the interval includes 1.
Figure
Figure 1. The two conflicting statements about the same outcome (same-day A&E visits after a locum consultation) under the same adjusted model. Left (orange): the abstract's reported OR, whose 95% interval excludes 1. Right: Table 3 (same-day A&E row; the row label is outside the quotation marks), Additional file 2 Table S19 and the Results text, whose intervals include 1 and which describe no difference. Bottom (blue): the two-sided p-values implied by each printed interval on the log scale, recomputed by recompute.py.txt (0.022 and 0.38). The paper prints p = 0.331 for Table 3. All excerpts are verbatim.
Figure 2. ORs and 95% CIs for locum vs permanent GP consultations, as reported in the abstract (orange, filled) and in Table 3 (gray, open), on a log scale with a dashed line at OR = 1. For same-day A&E, the abstract's interval lies entirely above 1 and Table 3's spans it. The blue labels give the p-values implied by each same-day A&E interval, recomputed, each level with its own marker; Table 3 prints p = 0.331. The abstract's point estimates for antibiotics, hypnotics and tests, and its revisit interval, also differ slightly from Table 3, but none of these differences changes which side of 1 an interval falls on. Same-day A&E is the only outcome where the significance reading changes. A&E within 1–7 days agrees in both places. The figure shows only numbers printed in the paper and p-values computed from them; it does not reproduce the paper's Fig. 1.
Why it matters
- The abstract lists a harm signal that the analysis does not support. The abstract counts both same-day and 1–7 day A&E visits among the outcomes more frequent after locum consultations. Only the 1–7 day effect (OR 1.05, 1.02 to 1.08, p < 0.001) is supported by Table 3, Table S19, Fig. 1 and the Results text. Many readers, including indexers and those who cite the paper, see only the abstract.
- On the paper's own criterion the abstract's figure would not qualify even at face value. The Methods set α at 1%. The interval in the abstract implies p ≈ 0.022, so the paper's threshold would not count it as a difference. The Table 3 value (p = 0.331) clearly does not meet it either.
- The size involved is small and the main conclusion is unchanged. At the permanent-GP rate of 0.307%, OR 1.04 corresponds to about 12 extra same-day A&E visits per 100,000 consultations, and OR 1.02 to about 6. The paper's conclusion that locum and permanent GPs are not systematically different does not rest on this outcome. Correcting it moves one secondary outcome from "higher after locum" to "no difference", so if anything the conclusion becomes slightly stronger.
- The abstract was not reconciled with the final model output. The other abstract values that do not match Table 3 (antibiotics, practice revisits, hypnotics, tests, and "eight out of the eleven" safety indicators) are each small, and none changes an interpretation. They do mean that the abstract's numbers cannot be relied on to the second decimal.
Explanations considered
- A different model, subset or outcome definition. Not supported. The abstract and Table 3 both describe the multi-level mixed-effects logistic regression adjusted for patient and practice characteristics. The Methods define a single same-day A&E variable ("Two binary variables were created indicating whether there was an A&E visit on the same day or within 1 to 7 days"). The only other model reported is the sensitivity analysis excluding 2020–2022 (Table S23), which gives 1.02 (0.97 to 1.07), p 0.408.
- An earlier version of the analysis. Possible, but it cannot rescue the abstract value. The abstract's practice-revisit interval (0.87 to 0.88) matches Table S23 rather than Table 3, and the "eleven" indicators conflict with the 10 defined in Methods. The paper's Fig. 1 also appears to plot output other than Tables 3 and S18: read from pixel positions, its hypnotics estimate is about 0.962 (0.941 to 0.984), close to the abstract's 0.96 (0.94 to 0.98) rather than Table 3's 0.97 (0.95 to 0.99), and its practice-revisit estimate is about 0.877 (0.871 to 0.884), close to the abstract's 0.88 (0.87 to 0.88) rather than S18's 0.884 (0.878 to 0.891). These point to the abstract and Fig. 1 drawing at least in part on output other than the final main models. But even Fig. 1, which shares these abstract values, shows same-day A&E at about 1.02 (0.97 to 1.07), not 1.04 (1.01 to 1.08). No reported output, table or figure, gives the abstract's same-day A&E value.
- Crude rather than adjusted estimate. Not supported. The crude OR from Table 1 counts is 1.126, which is far from 1.04, and the abstract says the models control for patient and practice characteristics.
- Rounding. Not able to explain it. The three-decimal output 1.023 (0.977 to 1.073) rounds to 1.02 (0.98 to 1.07). No rounding turns a lower limit of 0.977 into 1.01.
- Table 3 is the value in error rather than the abstract. Not supported. Table 3 agrees with the full regression output in Table S19, with the same-day A&E estimate plotted in Fig. 1 (about 1.021, 0.972 to 1.072) and with the Results text, and its printed p = 0.331 is consistent with its own interval (implied p ≈ 0.34 from the three-decimal interval). The abstract is the only place where the 1.04 value appears.
- A text-conversion artifact in PMC. Ruled out. A previous check opened the publisher's version of record on Springer Nature Link, the publisher's Table 3 page and the Europe PMC full-text XML. All of them have the same abstract wording, the same Results sentence and the same Table 3 row.
- Side observations, not part of the main finding. The Table 3 footer says results "are reported as incidence rate ratios (IRR)", while the table title, Fig. 1 and the abstract say odds ratios. The p-values are printed with a leading "<" (for example "< 0.331"), although they appear to be exact values. The Discussion gives the A&E visit rate as "0.15%", while Table 1 gives 0.31% (same day) and 0.39% (1–7 days) for permanent GPs; 0.15% is close to the rates Table 1 prints for emergency and ACSC admissions (0.14% to 0.16%). None of these affects a conclusion.
Status
- Version of record. A previous check opened the publisher's article page (Springer Nature Link / BMC Medicine, vol 22, article 126, published 27 March 2024) in headless Chromium, because plain requests met a JavaScript challenge. The abstract has "4% more likely to be followed by an A&E visit on the same day (OR = 1.04 (1.01 to 1.08))". The Results text has "no difference in rates of the same day A&E visits". The publisher's Table 3 page has "A&E visits, same day 1.02 (0.98 to 1.07), < 0.331 [0.028]". So the mismatch is in the version of record itself. I checked every quotation above verbatim against the PMC-archived text with
src/show.py. - Supplements. I downloaded Additional file 2 (
12916_2024_3332_MOESM2_ESM.docx) from Springer static content and extracted its text. Table S19 gives same-day A&E 1.023 (0.977 to 1.073), p 0.331, and Table S23 gives 1.02 (0.97 to 1.07), p 0.408. Tables S18–S20 give the three-decimal values used above. The abstract's values 1.04 (1.01 to 1.08), 1.22, 0.96 (0.94 to 0.98) and 0.81 (0.80 to 0.82) appear nowhere in the file, nor in the paper's tables. The paper's Fig. 1, as plotted, does sit close to the abstract's hypnotics value 0.96 (0.94 to 0.98) and revisit interval; it does not show 1.04 (1.01 to 1.08) for same-day A&E. A previous check also opened Additional file 1 (12916_2024_3332_MOESM1_ESM.xlsx), which holds code lists only. - Corrections and comments. As of 2026-10-24, I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. A Crossrefupdates:filter on the DOI returns 0 works, and the Crossref record has no update-to or updated-by relations. The PubMed record (PMID 38532468) has no CommentsCorrections links, and the publisher's article page shows no change history or correction notice. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search found no PubPeer thread or correction. The Europe PMC REST search returned errors on repeated tries, so I could not query it directly; its notices were covered throughsrc/known.py. - Sources. PMC article PMC10967114; https://doi.org/10.1186/s12916-024-03332-z (Springer Nature Link article page and Table 3 page); Additional files 1 and 2 from static-content.springer.com; Europe PMC full-text XML; Crossref API; PubMed; PubPeer.
Suggested correction
The final model output (Table 3, Table S19 and the Results text), together with the same-day A&E estimate plotted in Fig. 1, supports the following edit to the abstract's Results. Replace
8% more likely to involve a prescription for a strong opioid (OR = 1.08 (1.06 to 1.09)), 4% more likely to be followed by an A&E visit on the same day (OR = 1.04 (1.01 to 1.08)) and 5% more likely to be followed by an A&E visit within 1 to 7 days (OR = 1.05 (1.02 to 1.08)).
with
8% more likely to involve a prescription for a strong opioid (OR = 1.08 (1.06 to 1.09)) and 5% more likely to be followed by an A&E visit within 1 to 7 days (OR = 1.05 (1.02 to 1.08)).
and replace
We found no evidence that emergency admissions, ACSC admissions and eight out of the eleven prescribing safety indicators were different
with
We found no evidence that same-day A&E visits, emergency admissions, ACSC admissions and seven out of the ten prescribing safety indicators were different
The authors may also wish to bring the abstract's other estimates in line with Table 3 (antibiotics 1.21 (1.21 to 1.22), with the percentage changed to 21%; practice revisits 0.88 (0.88 to 0.89); hypnotics 0.97 (0.95 to 0.99), 3%; tests 0.80 (0.80 to 0.81), 20%). They may also wish to align the Results text's revisit interval (0.88 to 0.91), antibiotic interval (1.20 to 1.22) and hypnotic interval (0.94 to 0.99) with Table 3, and to check that Fig. 1 plots the final model output for hypnotics and practice revisits. If instead the authors' final model gives 1.04 (1.01 to 1.08) for same-day A&E, then Table 3, Table S19, the same-day A&E estimate in Fig. 1 and the Results sentence would all need correcting, and the abstract should say that the estimate does not meet the paper's 1% α.
Recomputation script (python recompute.py.txt)
"""Recompute the same-day A&E result of PMC10967114
(BMC Med 2024, doi:10.1186/s12916-024-03332-z, "Comparing the clinical practice and
prescribing safety of locum and permanent doctors: observational study of primary care
consultations in England").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC and from Additional file 2
(12916_2024_3332_MOESM2_ESM.docx):
Abstract (Results): ORs and 95% CIs for eight clinical-practice outcomes.
Table 3: ORs, 95% CIs, p-values for the same outcomes ("< 0.331" printed as such).
Results, paragraph 3: ORs and CIs quoted in the text.
Fig. 1 (publisher image 12916_2024_3332_Fig1_HTML): selected ORs and CIs read from pixel
positions (linear axis calibrated on the .8/.9/1/1.1/1.2 gridlines; about +/-0.003).
Additional file 2, Tables S18-S20: the same locum ORs to three decimals.
Additional file 2, Table S23: sensitivity analysis excluding 2020-2022.
Table 1: counts and percentages of consultations with each outcome.
Methods: "an alpha level of 1% was used throughout".
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ALPHA = 0.01 # Methods, "Model covariates"
Z975 = norm.ppf(0.975)
def from_ci(or_, lo, hi):
"""Log-scale SE, z and two-sided p implied by an OR and its 95% CI (Wald, log scale)."""
se = (np.log(hi) - np.log(lo)) / (2 * Z975)
z = np.log(or_) / se
p = 2 * norm.sf(abs(z))
return se, z, p, np.sqrt(lo * hi)
# ---------------------------------------------------------------- inputs
# outcome: (abstract, Table 3, Results text, Additional file 2 S18-S20 (3 dp), Table S23)
OUT = {
"Practice revisits": ((0.88, 0.87, 0.88), (0.88, 0.88, 0.89), (0.88, 0.88, 0.91), (0.884, 0.878, 0.891), (0.87, 0.87, 0.88)),
"Antibiotics": ((1.22, 1.21, 1.22), (1.21, 1.21, 1.22), (1.21, 1.20, 1.22), (1.213, 1.207, 1.220), (1.21, 1.21, 1.22)),
"Strong opioids": ((1.08, 1.06, 1.09), (1.08, 1.06, 1.09), (1.08, 1.06, 1.09), (1.077, 1.063, 1.091), (1.08, 1.07, 1.09)),
"Hypnotics": ((0.96, 0.94, 0.98), (0.97, 0.95, 0.99), (0.97, 0.94, 0.99), (0.967, 0.946, 0.988), (0.97, 0.94, 0.99)),
"Referrals": ((0.85, 0.84, 0.86), (0.85, 0.84, 0.86), (0.85, 0.84, 0.86), (0.850, 0.842, 0.858), (0.85, 0.85, 0.86)),
"Tests": ((0.81, 0.80, 0.82), (0.80, 0.80, 0.81), (0.80, 0.80, 0.81), (0.805, 0.796, 0.814), (0.80, 0.80, 0.81)),
"A&E same day": ((1.04, 1.01, 1.08), (1.02, 0.98, 1.07), None, (1.023, 0.977, 1.073), (1.02, 0.97, 1.07)),
"A&E within 1-7 days": ((1.05, 1.02, 1.08), (1.05, 1.02, 1.08), (1.05, 1.02, 1.08), (1.052, 1.022, 1.082), (1.05, 1.02, 1.08)),
}
P_REPORTED = {"Table 3": 0.331, "Table S19": 0.331, "Table S23": 0.408}
# Table 1: (count permanent, count locum, printed % permanent, printed % locum)
T1 = {
"Practice revisits within 7 days": (1008934, 94204, 8.18, 6.89),
"Same day A&E visits": (37861, 4723, 0.31, 0.35),
"A&E visits within 1-7 days": (48545, 6132, 0.39, 0.45),
}
N_TOTAL = 13696455
# ---------------------------------------------------------------- 1. same-day A&E versions
print("=== 1. Same-day A&E visits: every version the paper prints ===")
versions = [
("Abstract (Results)", OUT["A&E same day"][0], None),
("Table 3 row 7", OUT["A&E same day"][1], P_REPORTED["Table 3"]),
("Table S19 (3 dp)", OUT["A&E same day"][3], P_REPORTED["Table S19"]),
("Table S23 excl. 2020-22", OUT["A&E same day"][4], P_REPORTED["Table S23"]),
]
IMPLIED = {}
for lab, (o, lo, hi), prep in versions:
se, z, p, gm = from_ci(o, lo, hi)
IMPLIED[lab] = p
print(f"{lab:24s} OR {o} ({lo} to {hi}) SE(log)={se:.4f} z={z:.2f} implied p={p:.3f}"
f" reported p={prep} CI includes 1: {lo <= 1 <= hi}"
f" p<0.05: {p < 0.05} p<alpha(0.01): {p < ALPHA} geometric midpoint={gm:.3f}")
print("Results paragraph 3 text: 'there was no difference in rates of the same day A&E visits'")
# z needed to reproduce the reported p=0.331 from Table S19's point estimate
z331 = norm.isf(0.331 / 2)
print(f"reported p=0.331 corresponds to |z|={z331:.3f}; S19 OR 1.023 then gives SE(log)={np.log(1.023)/z331:.4f}"
f" vs {from_ci(1.023, 0.977, 1.073)[0]:.4f} from its CI")
print()
print("=== 2. Same-day vs 1-7 day A&E (Table S19) ===")
se, z, p, _ = from_ci(1.052, 1.022, 1.082)
print(f"A&E within 1-7 days 1.052 (1.022 to 1.082): z={z:.2f}, implied p={p:.1e} (reported <0.001)")
print()
print("=== 3. Crude rates from Table 1 ===")
n_perm = T1["Practice revisits within 7 days"][0] / (T1["Practice revisits within 7 days"][2] / 100)
n_loc = T1["Practice revisits within 7 days"][1] / (T1["Practice revisits within 7 days"][3] / 100)
print(f"implied consultations: permanent {n_perm:,.0f}, locum {n_loc:,.0f}, total {n_perm + n_loc:,.0f}"
f" (Table 1 header N = {N_TOTAL:,}; difference from rounding of the printed %)")
CRUDE = {}
for lab in ("Same day A&E visits", "A&E visits within 1-7 days"):
a, b, pa, pb = T1[lab]
p1, p2 = a / n_perm, b / n_loc
cor = (p2 / (1 - p2)) / (p1 / (1 - p1))
CRUDE[lab] = (p1, p2, cor)
print(f"{lab:28s} permanent {100*p1:.3f}% (printed {pa}%), locum {100*p2:.3f}% (printed {pb}%), crude OR {cor:.3f}")
print()
print("=== 4. Absolute size at the permanent-GP same-day A&E rate ===")
p0 = CRUDE["Same day A&E visits"][0]
for o in (1.04, 1.02):
odds = p0 / (1 - p0) * o
p_ = odds / (1 + odds)
print(f"OR {o}: rate {100*p_:.4f}% vs {100*p0:.4f}% difference {100*(p_-p0):.4f} percentage points"
f" = {1e5*(p_-p0):.1f} per 100,000 consultations")
print()
print("=== 5. Abstract vs Table 3 vs Results text vs Additional file 2 (same model claimed) ===")
print(f"{'outcome':20s} {'abstract':20s} {'Table 3':20s} {'text':20s} {'S18-S20 (3dp)':24s} {'S23 (excl 2020-22)':20s} abs==T3")
fmt = lambda t: "-" if t is None else f"{t[0]} ({t[1]}-{t[2]})"
for lab, (ab, t3, tx, s, s23) in OUT.items():
print(f"{lab:20s} {fmt(ab):20s} {fmt(t3):20s} {fmt(tx):20s} {fmt(s):24s} {fmt(s23):20s} {ab == t3}")
print("Abstract values found in no other table: same-day A&E 1.04 (1.01-1.08), antibiotics 1.22,"
" hypnotics 0.96 (0.94-0.98), tests 0.81 (0.80-0.82).")
print("Only abstract value matching S23 but not Table 3: practice revisit CI 0.87-0.88.")
print("Rounding the 3-dp S18-S20 values to 2 dp reproduces Table 3 except for tests, where 0.805 is a rounding tie:")
for lab, (ab, t3, tx, s, s23) in OUT.items():
r = tuple(round(v + 1e-9, 2) for v in s)
print(f" {lab:20s} round(S)={r} Table 3={t3} match={r == t3}")
print()
print("=== 6. Prescribing safety indicators (Table 3, indicators A-J) ===")
IND = {"A": "<0.001", "B": 0.547, "C": 0.349, "D": 0.007, "E": 0.675, "F": 0.852,
"G": 0.894, "H": "<0.001", "I": 0.613, "J": 0.128}
sig = [k for k, v in IND.items() if v == "<0.001" or v < ALPHA]
print(f"indicators in Table 3: {len(IND)} (Methods: 'We adapted 10 indicators')")
print(f"significant at alpha=1%: {sig} -> {len(IND) - len(sig)} of {len(IND)} not different;"
" abstract says 'eight out of the eleven'")
print()
print("=== 7. Paper's Fig. 1, digitised from pixel positions (about +/-0.003) ===")
FIG1 = {"Practice revisits": (0.877, 0.871, 0.884), "Antibiotics": (1.215, 1.207, 1.222),
"Hypnotics": (0.962, 0.941, 0.984), "Tests": (0.805, 0.796, 0.814),
"A&E same day": (1.021, 0.972, 1.072), "A&E within 1-7 days": (1.052, 1.022, 1.084)}
for lab, d in FIG1.items():
ab, t3, s = OUT[lab][0], OUT[lab][1], OUT[lab][3]
dist = lambda v: max(abs(a - b) for a, b in zip(d, v))
print(f" {lab:20s} Fig.1 {fmt(d):22s} abstract {fmt(ab):18s} (max diff {dist(ab):.3f})"
f" Table 3 {fmt(t3):18s} (max diff {dist(t3):.3f}) 3dp {fmt(s):22s} (max diff {dist(s):.3f})")
print("Fig. 1 hypnotics sit nearer the abstract's 0.96 (0.94-0.98) than Table 3's 0.97 (0.95-0.99);"
" same-day A&E sits at about 1.02 (0.97-1.07), not the abstract's 1.04 (1.01-1.08).")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"svg.fonttype": "none", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# Figure 1: two text panels joined by an annotated connector
fig = plt.figure(figsize=(7.2, 4.25))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.25)
ax.axis("off")
def panel(x, y, w, h, title, edge, lines):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.15, y + h - 0.18, title, fontsize=11, fontweight="bold", va="top", color=INK)
yy = y + h - 0.58
for txt, col, style in lines:
ax.text(x + 0.15, yy, txt, fontsize=10, va="top", color=col, style=style, linespacing=1.3)
yy -= 0.23 * (txt.count("\n") + 1) + 0.11
panel(0.15, 1.45, 3.0, 2.68, "Abstract (Results)", ORANGE, [
("“...4% more likely to be followed\nby an A&E visit on the same day", INK, "italic"),
("(OR = 1.04 (1.01 to 1.08))”", ORANGE, "italic"),
("interval excludes 1", INK2, "normal"),
])
panel(4.05, 1.45, 3.0, 2.68, "Table 3, Table S19, Results", INK2, [
("Table 3, same-day A&E row:\n“1.02 (0.98 to 1.07), < 0.331”", INK, "italic"),
("Table S19: “1.023 (0.977 to 1.073),\n<0.331”", INK, "italic"),
("Results: “no difference in rates of\nthe same day A&E visits”", INK, "italic"),
("intervals include 1", INK2, "normal"),
])
ax.add_patch(FancyArrowPatch((3.2, 2.8), (4.0, 2.8), arrowstyle="<->", mutation_scale=14,
lw=2, color=INK))
ax.text(3.6, 3.0, "conflict", ha="center", fontsize=10, fontweight="bold")
ax.text(0.15, 1.27,
"Same outcome, same adjusted model. Two-sided p implied by each 95% CI\n(log-scale Wald), recomputed:",
fontsize=10, color=INK2, va="top")
ax.text(0.15, 0.7, f"abstract interval → p = {IMPLIED['Abstract (Results)']:.3f}", fontsize=10.5,
color=BLUE, va="top", fontweight="bold")
ax.text(4.05, 0.7, f"Table 3 interval → p = {IMPLIED['Table 3 row 7']:.2f}", fontsize=10.5,
color=BLUE, va="top", fontweight="bold")
ax.text(0.15, 0.4, "Orange: value reported in the abstract. Blue: recomputed from the paper's own\nintervals. The paper's stated alpha is 1%.", fontsize=10, color=INK2, va="top")
save(fig, "fig1")
# Figure 2: abstract vs Table 3 for the eight clinical-practice outcomes, one log axis
labels = list(OUT.keys())
fig, ax = plt.subplots(figsize=(7.2, 4.6))
fig.subplots_adjust(left=0.24, right=0.97, top=0.88, bottom=0.14)
off = 0.2
for i, lab in enumerate(labels):
y = len(labels) - 1 - i
(ao, al, ah), (to, tl, th) = OUT[lab][0], OUT[lab][1]
ax.plot([al, ah], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(ao, y + off, "o", ms=8, color=ORANGE)
ax.plot([tl, th], [y - off] * 2, color=INK2, lw=2, solid_capstyle="butt")
ax.plot(to, y - off, "o", ms=8, mfc=BG, mec=INK2, mew=2)
ax.axvline(1, color=INK, lw=0.9, ls="--", zorder=1)
ax.set_xscale("log")
ticks = [0.8, 0.85, 0.9, 0.95, 1.0, 1.05, 1.1, 1.15, 1.2, 1.25]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.set_xlim(0.77, 1.55)
ax.set_yticks(range(len(labels)))
ax.set_yticklabels(labels[::-1])
ax.set_ylim(-0.6, len(labels) - 0.4)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, locum vs permanent GP (log scale; dashed line = 1)")
yA = len(labels) - 1 - labels.index("A&E same day")
ax.text(1.095, yA + off, f"implied p = {IMPLIED['Abstract (Results)']:.3f}",
va="center", fontsize=10, color=BLUE)
ax.text(1.095, yA - off, f"implied p = {IMPLIED['Table 3 row 7']:.2f} (printed 0.331)",
va="center", fontsize=10, color=BLUE)
handles = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="Abstract (reported)"),
Line2D([], [], color=INK2, lw=2, marker="o", ms=8, mfc=BG, mec=INK2, mew=2,
label="Table 3 (reported)"),
Line2D([], [], color=BLUE, lw=0, marker="$p$", ms=9, label="implied p (recomputed)")]
ax.legend(handles=handles, loc="lower center", bbox_to_anchor=(0.42, 1.0), ncol=3, frameon=False,
fontsize=10, handlelength=1.8)
save(fig, "fig2")
print()
print("wrote fig1.svg/png and fig2.svg/png to", HERE)
Printed 95% CI for a mortality predictor implies P ≈ 0.98, not the reported 0.006
BMC Med (2024) · doi:10.1186/s12916-024-03812-2 · PMC11656579
Summary
The paper fits a multivariable logistic model of mortality in patients who developed immune checkpoint inhibitor-related endocrinopathies (IREs; N = 173). It reports the pre-treatment platelet-to-lymphocyte ratio (PLR) as OR 1.004, 95% CI 0.7–1.4, P = 0.006, and names PLR as one of two significant predictors. The same three figures appear in the Abstract, in the Results text and in Table 5, and they cannot all be correct. A 95% CI of 0.7–1.4 contains 1 and implies a two-sided P of 0.98 (no lower than 0.73 under any rounding of the printed values). An OR of 1.004 with P = 0.006 would need a 95% CI of about 1.0011–1.0069. A reader therefore cannot tell from the paper whether PLR independently predicts mortality in this subgroup.
What the paper reports
After MVR, factors influencing mortality in the subgroup with IREs included a higher age (OR 1.1, 95% CI 1.04–1.2, P = 0.001) and platelet-to-lymphocyte ratio (OR 1.004, 95% CI 0.7–1.4, P = 0.006).
— Abstract, Results
According to the MVR analysis, only age at ICI initiation (OR 1.1, 95% CI: 1.04–1.2; P = 0.001) and PLR (OR 1.004, 95% CI: 0.7–1.4; P = 0.006) were found to be statistically significant factors for predicting mortality.
— Results, "Factors influencing mortality among patients who developed endocrinopathy post-ICI"
PLR | 1.001 (1.0001-1.003) | 0.03 | 1.004 (0.7-1.4) | 0.006
— Table 5, "Univariate and multivariate logistic regression analysis of factors influencing mortality following endocrinopathies post immune checkpoint inhibitor therapy", PLR row. The columns are univariate OR (95% CI), univariate P, multivariate OR (95% CI) and multivariate P. In the version of record the multivariate P of 0.006 is set in bold. The table footer says "P values in bold indicate statistical significance".
Similarly, mortality was predicted by age at ICI initiation and a higher pre-ICI PLR.
— Discussion, first paragraph
A P value of < 0.05 indicated statistical significance.
— Methods, "Statistical analysis"
Two related passages bear on how PLR is scaled. The first gives the definition (the archived text renders the superscript in "× 10³" as "103"):
Platelet-to-lymphocyte ratio (PLR) was calculated for the included patients using the formula PLR = Lymphocyte Count (× 103/μL) ÷ Platelet Count (× 103/μL).
— Methods, "Data sources and measures"
PLR | 145.3 (64.2-229.2) | 143.9 (27-259.6) | 0.7
— Table 3, PLR row (median and IQR, IRE vs no IRE)
What the paper's own numbers show
Method. A Wald 95% CI for a logistic-regression odds ratio is exp(ln OR ± 1.96 × SE), so it is symmetric on the log scale. The standard error can therefore be recovered from the printed bounds as SE = (ln hi − ln lo)/3.92, and the two-sided P follows from z = ln(OR)/SE. Run backwards, the 95% CI that must go with a given OR and P is exp(ln OR ± 1.96 × ln OR / z_P). To allow for rounding, each printed number was varied across its whole rounding interval (0.7 means 0.65–0.75, 1.004 means 1.0035–1.0045), and the estimate was also allowed to sit at the geometric centre of the CI. As a check on the method, the same calculation was run on nine other rows from Tables 4 and 5. Every number below is printed by recompute.py.txt (Python, numpy/scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Does the 95% CI contain 1? | Abstract; Results; Table 5 | 0.7–1.4 | Yes |
| SE of ln(OR) implied by the CI | Table 5 | 0.7–1.4 | (ln 1.4 − ln 0.7)/3.92 = 0.177 |
| Geometric centre of the CI | Table 5 | OR 1.004 | 0.990 |
| Two-sided P, multivariable PLR | Abstract; Results; Table 5 | 0.006 | 0.98 with OR 1.004 (z = 0.023); 0.95 with the CI centre; one-sided 0.49 |
| Smallest P under any rounding of the printed values | Table 5 | 0.006 | 0.976 with the estimate rounding to 1.004; 0.73 if the estimate is taken as the CI centre |
| P if 0.7–1.4 were a 90%, 99% or 99.9% CI | Table 5 | 0.006 | 0.985, 0.976, 0.970 |
| 95% CI required by OR 1.004 with P = 0.006 | Abstract; Results; Table 5 | 0.7–1.4 | 1.0011–1.0069 (SE 0.00145, 122 times smaller than the SE implied by the printed CI) |
| Required CI over rounding of OR and P | derived | 0.7–1.4 | lower bound 1.0010–1.0013, upper bound 1.0060–1.0078 |
| OR per 100 PLR units, if 1.004 is per unit | derived | 1.004 | 1.004^100 = 1.49, which lies outside 0.7–1.4 |
| Printed CI read as per 100 units, converted to per unit | derived | 0.7–1.4 | 0.9964–1.0034, which still contains 1 |
| Control: univariable PLR, 1.001 (1.0001–1.003) | Table 5 | P = 0.03 | 0.036 (0.027–0.044 over rounding of the bounds) |
| Control: multivariable age, 1.1 (1.04–1.2) | Abstract; Results; Table 5 | P = 0.001 | 0.0024 (0.0002–0.0075) |
| Control: univariable HbA1c, 1.3 (1.01–1.6) | Table 5 | P = 0.03 | 0.041 (0.036–0.046) |
| Control: IRE model, pembrolizumab, 2.6 (1.05–6.3) | Abstract; Table 4 | P = 0.04 | 0.039 (0.038–0.040) |
| PLR scale: Methods formula, lymphocytes ÷ platelets | Methods; Table 3 | medians 145.3 and 143.9 | Using the reference ranges in the Table 3 footer, lymphocytes ÷ platelets falls between 0.0024 and 0.020, and platelets ÷ lymphocytes between 50 and 410. The Table 3 values fit platelets ÷ lymphocytes |
Figure 2 and the script's output list the other control rows: univariable age, univariable haemoglobin, and IRE-model age, WBC and nivolumab. In those nine control rows the recomputed P agrees with the printed P to within rounding, or within about a factor of two where the printed estimate is coarsely rounded. In the multivariable PLR row, the printed P and the P implied by the printed CI differ by more than two orders of magnitude and fall on opposite sides of the paper's 0.05 threshold.
Figure
Figure 1. Odds ratio for mortality per unit PLR in the multivariable model of the IRE subgroup, on one log-scale axis. The thin vertical line marks OR = 1. The orange interval and dot show the paper's printed result: OR 1.004, 95% CI 0.7–1.4, stated P = 0.006. The interval straddles OR = 1. Its geometric centre is 0.990 (open blue diamond), and recomputed from its width it gives a two-sided P of 0.98 for an OR of 1.004. The filled blue diamond marks the 95% CI that an OR of 1.004 with P = 0.006 would actually require, 1.0011–1.0069. At this scale that interval is narrower than the marker. The printed interval and the printed P value cannot describe the same estimate.
Figure 2. Two-sided P values printed in Tables 4 and 5 (orange circles; the arrow marks a value printed as "<0.001") compared with P values recomputed from each row's printed 95% CI (blue diamonds, with the estimate at the CI's geometric centre). The light-blue bars show how far the recomputed P moves when each printed CI bound varies across its rounding interval. The dashed line marks P = 0.05. In nine rows the printed and recomputed values are close, which shows that the method recovers the paper's own reporting. The multivariable PLR row (top) is the exception. Its printed P is 0.006, while its printed CI gives 0.73–1.0.
Why it matters
Mortality in the IRE subgroup is one of the paper's stated secondary outcomes. PLR is one of only two variables the Abstract lists as multivariable predictors of it. The Results text and Table 5 repeat the result, and the Discussion's opening summary restates it ("mortality was predicted by age at ICI initiation and a higher pre-ICI PLR"). The printed figures support two incompatible readings:
- If the CI is correct, PLR is not a significant independent predictor (P ≈ 0.98). The claim in the Abstract, Results and Discussion would then be reversed.
- If the OR and the P value are correct, the CI is misprinted and would be about 1.001–1.007 per unit, roughly 1.12–1.98 per 100 units of PLR. The significance claim would stand, but the printed interval misstates its precision by a factor of about 120 on the log scale.
Nothing in the paper shows which reading is correct. The supplement contains no regression output that could settle it (see Status).
How PLR is defined adds a second difficulty with this variable. As written in Methods, the formula (lymphocytes ÷ platelets) would give values of about 0.002–0.02, not the medians near 145 shown in Table 3. If the values in Table 3 are platelets ÷ lymphocytes, as their size suggests, the formula in Methods is inverted. Either way, a reader cannot be sure what unit the per-unit OR refers to.
The paper's main conclusions do not depend on this row. These are the IRE incidence of 26.7%, the commonest IRE types, the predictors of IRE development, and the statement that "The development of IREs did not seem to influence mortality" (Abstract, Conclusions). The error affects a secondary, prognostic result, but that result is printed in the Abstract.
Explanations considered
- Rounding. This does not account for the numbers. With every printed value varied across its rounding interval, the smallest two-sided P that can be reached is 0.976 if the estimate rounds to 1.004, or 0.73 if the estimate is taken as the CI centre. In the other direction, the CI that would fit P = 0.006 (1.0011–1.0069) rounds to "1.0–1.0" at one decimal place, not to "0.7–1.4".
- A one-sided test. This does not account for the numbers. The one-sided P from the printed CI is 0.49. Methods mention no one-sided tests and set significance at P < 0.05.
- A confidence level other than 95%. This does not account for the numbers. Read as a 90%, 99% or 99.9% interval, 0.7–1.4 gives P between 0.970 and 0.985.
- A rescaled PLR unit (for example, OR per 10 or per 100 units). This does not account for the numbers. Neither Table 5 nor Methods states any rescaling. The OR and its CI are printed together, so they should be on the same scale, and on that scale the CI is centred at 0.990, not 1.004. If only the OR had been rescaled, 1.004 per unit would be 1.49 per 100 units, which lies outside 0.7–1.4. If the CI were per 100 units, it would be 0.9964–1.0034 per unit, which still contains 1.
- A non-Wald interval (profile likelihood or bootstrap). This does not account for the numbers. Such intervals can be somewhat asymmetric, but a 95% interval that runs from 0.7 to 1.4 around an estimate of 1.004 cannot accompany a test that rejects OR = 1 at P = 0.006. An interval obtained by inverting a test with P = 0.006 excludes 1 at the 95% level. An unstable fit with a very large SE would also give a large P, not a small one.
- Adjusted and unadjusted figures mixed together. This does not account for the numbers. All three figures sit in the multivariate columns of Table 5, and the Abstract and Results text label them as multivariable (MVR) results. The univariable PLR row (1.001, 1.0001–1.003, P = 0.03) is internally consistent (recomputed P 0.036).
- A conversion or typesetting artefact in the archived copy. This was ruled out. The publisher's HTML (link.springer.com, including the standalone Table 5 page), the PMC text and the Europe PMC full-text XML all give "1.004 (0.7-1.4)" and "0.006" in separate table cells. The Abstract and the Results text repeat the same figures in prose.
- A general reporting pattern. This does not account for the numbers. In nine other rows of Tables 4 and 5, the recomputed P values agree with the printed ones (Figure 2). The discrepancy is specific to this row.
From the paper's numbers alone, it is not possible to tell which of the three printed values is in error.
Status
As of 2026-10-23:
- Version of record: checked. The publisher's HTML at link.springer.com (published 18 December 2024) was rendered in headless Chromium. It shows the same Abstract sentence and the same Results sentence. The standalone Table 5 page gives the PLR row as "1.004 (0.7-1.4)" with "0.006" in bold. The PMC HTML and the Europe PMC full-text XML match. The publisher PDF could not be retrieved: the request returned an HTML interstitial page.
- Supplement: checked. Additional file 1 (12916_2024_3812_MOESM1_ESM.docx, Tables S1–S5) was downloaded, and its MD5 checksum matches the one listed in the PMC XML. It gives laboratory values (including platelets, lymphocytes and the platelet/lymphocyte ratio), follow-up endocrine profiles and subgroup characteristics. It contains no regression coefficients, standard errors or model output, so it does not resolve the discrepancy.
- Corrections: none found. Crossref lists no updates (no update-to or updated-by entries) and PMC lists no notices, both checked with
src/known.py. Europe PMC (PMID 39695571) shows no comment or correction links and no erratum records. The Retraction Watch database has no entry for the DOI. The publisher page shows no correction notice. - PubPeer: not reachable by automated query. The API returned HTTP 403, and the search page served a bot-verification challenge. Earlier web searches during this audit found no PubPeer thread, erratum or published comment. Fresh web searches could not be run today because the session's search quota was used up. A manual search at pubpeer.com is still advisable.
Sources: doi.org/10.1186/s12916-024-03812-2 · link.springer.com article · Table 5 page · PMC11656579 · Europe PMC record · Crossref works record
Suggested correction
The authors could re-check the multivariable logistic model of mortality in the IRE subgroup against the original software output. They could then report the PLR odds ratio, its 95% CI and its P value exactly as produced, so that the three values agree, and state the PLR unit that the OR refers to.
- If the CI is the misprinted value, the corrected interval should replace "0.7–1.4" in the Abstract, the Results text and Table 5.
- If the P value or the estimate is in error, the statement that PLR is a significant independent predictor of mortality should be revised in the Abstract, the Results and the Discussion.
Separately, the PLR formula in Methods ("Lymphocyte Count ÷ Platelet Count") should be checked against the values in Table 3, which fit platelets ÷ lymphocytes.
Recomputation script (python recompute.py.txt)
"""Recompute the multivariable PLR mortality result of PMC11656579
(BMC Med 2024, doi:10.1186/s12916-024-03812-2) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md and writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file.
All inputs are copied from the paper (Abstract, Results, Methods, Tables 3, 4, 5).
Method: Wald test on the log-odds scale. A 95% CI for an odds ratio is
exp(ln OR +/- 1.96 SE), so SE = (ln hi - ln lo) / (2 * 1.959964) and
z = ln(OR) / SE, two-sided P = 2 * (1 - Phi(|z|)).
"""
import itertools
import os
import numpy as np
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975) # 1.959964
def se_from_ci(lo, hi, z=Z):
return (np.log(hi) - np.log(lo)) / (2 * z)
def p_two_sided(est, lo, hi, z=Z):
se = se_from_ci(lo, hi, z)
zz = np.log(est) / se
return se, zz, 2 * norm.sf(abs(zz))
def half_unit(s):
"""Half of the last printed digit ('0.7' -> 0.05; '1.0001' -> 0.00005).
A bound printed without decimals ('1') is treated as one decimal (1.0 +/- 0.05)."""
d = len(s.split(".")[1]) if "." in s else 1
return 0.5 * 10 ** (-d)
def interval(s, n=41):
v, h = float(s), half_unit(s)
return np.linspace(v - h, v + h - 1e-12, n)
def p_range(est_s, lo_s, hi_s, use_printed_est=True):
"""Min and max two-sided P over every value that rounds to the printed numbers.
The estimate is taken as the geometric midpoint of the (unrounded) CI and,
if use_printed_est, also anywhere inside its own rounding interval."""
ps = []
for lo, hi in itertools.product(interval(lo_s), interval(hi_s)):
if not (0 < lo < hi):
continue
mid = np.sqrt(lo * hi)
ps.append(p_two_sided(mid, lo, hi)[2])
if use_printed_est:
for est in interval(est_s, 11):
ps.append(p_two_sided(est, lo, hi)[2])
return min(ps), max(ps)
line = "-" * 78
# ---------------------------------------------------------------- 1. the PLR row
print(line)
print("1. Multivariable PLR, mortality model in the IRE subgroup (Abstract; Results; Table 5)")
print(" printed: OR 1.004, 95% CI 0.7-1.4, P = 0.006")
OR, LO, HI, P_REP = 1.004, 0.7, 1.4, 0.006
se, zz, p = p_two_sided(OR, LO, HI)
mid = np.sqrt(LO * HI)
print(f" CI contains 1: {LO < 1 < HI}")
print(f" SE(ln OR) from CI = {se:.4f}")
print(f" z = ln(1.004)/SE = {zz:.4f}")
print(f" two-sided P = {p:.4f} one-sided P = {p / 2:.4f}")
print(f" geometric midpoint of CI = {mid:.4f}")
_, zm, pm = p_two_sided(mid, LO, HI)
print(f" P using the CI midpoint = {pm:.4f} (z = {zm:.4f})")
# rounding: estimate held to values that round to 1.004
pmin_est = min(p_two_sided(e, lo, hi)[2]
for lo in interval("0.7") for hi in interval("1.4") for e in interval("1.004", 11))
pmin_all, pmax_all = p_range("1.004", "0.7", "1.4")
print(f" min P, estimate rounding to 1.004, bounds rounding to 0.7 / 1.4 = {pmin_est:.4f}")
print(f" min P, also allowing the estimate to be the CI midpoint = {pmin_all:.4f}")
for level in (0.90, 0.99, 0.999):
zl = norm.ppf(0.5 + level / 2)
print(f" if 0.7-1.4 were a {level * 100:g}% CI: P = {p_two_sided(OR, LO, HI, zl)[2]:.4f}")
# ---------------------------------------------------------------- 2. CI required by OR and P
print(line)
print("2. 95% CI that OR 1.004 with P = 0.006 would require")
z_req = norm.isf(P_REP / 2)
se_req = np.log(OR) / z_req
req_lo, req_hi = np.exp(np.log(OR) - Z * se_req), np.exp(np.log(OR) + Z * se_req)
print(f" z = {z_req:.4f}; SE = {se_req:.6f}; CI = {req_lo:.4f}-{req_hi:.4f}")
print(f" ratio of SE implied by printed CI to SE required: {se / se_req:.0f}")
los, his = [], []
for e in interval("1.004", 11):
for pp in np.linspace(0.0055, 0.0065, 11):
s = np.log(e) / norm.isf(pp / 2)
los.append(np.exp(np.log(e) - Z * s)); his.append(np.exp(np.log(e) + Z * s))
print(f" over OR 1.0035-1.0045 and P 0.0055-0.0065: lower {min(los):.4f}-{max(los):.4f}, "
f"upper {min(his):.4f}-{max(his):.4f}")
print(f" required CI at 1 decimal: {req_lo:.1f}-{req_hi:.1f}")
# ---------------------------------------------------------------- 3. scaling checks
print(line)
print("3. Rescaling checks")
print(f" 1.004^10 = {OR ** 10:.4f}; 1.004^100 = {OR ** 100:.4f}")
print(f" 1.004^100 inside 0.7-1.4: {LO < OR ** 100 < HI}")
print(f" 0.7-1.4 read as per 100 units -> per unit {LO ** 0.01:.5f}-{HI ** 0.01:.5f} (contains 1)")
print(f" per-100-unit OR implied by the required CI: {req_lo ** 100:.3f}-{req_hi ** 100:.3f}")
# ---------------------------------------------------------------- 4. PLR definition vs Table 3
print(line)
print("4. PLR definition (Methods) vs values (Table 3)")
print(" Methods: PLR = lymphocyte count / platelet count")
print(" Table 3 median PLR: 145.3 (IRE), 143.9 (no IRE)")
print(" Table 3 footer reference ranges: platelets 150-410, lymphocytes 1-3 (x10^3/uL)")
print(f" lymphocytes/platelets at the reference limits: {1 / 410:.4f}-{3 / 150:.4f}")
print(f" platelets/lymphocytes at the reference limits: {150 / 3:.0f}-{410 / 1:.0f}")
# ---------------------------------------------------------------- 5. control rows
print(line)
print("5. Same calculation on other printed rows. P from the CI midpoint; 'range' lets the CI bounds")
print(" vary over their rounding intervals (estimate = CI midpoint); 'wide' also lets the")
print(" estimate vary over its own rounding interval")
CONTROLS = [
# label, source, est, lo, hi, printed P (string), printed P value for plotting
("Mortality MVR: PLR", "Table 5", "1.004", "0.7", "1.4", "0.006", 0.006),
("Mortality UV: PLR", "Table 5", "1.001", "1.0001", "1.003", "0.03", 0.03),
("Mortality MVR: age", "Table 5", "1.1", "1.04", "1.2", "0.001", 0.001),
("Mortality UV: age", "Table 5", "1.04", "1.02", "1.07", "<0.001", 0.001),
("Mortality UV: HbA1c", "Table 5", "1.3", "1.01", "1.6", "0.03", 0.03),
("Mortality UV: hemoglobin", "Table 5", "0.8", "0.7", "1", "0.03", 0.03),
("IRE MVR: age", "Table 4", "1.02", "1.003", "1.03", "0.02", 0.02),
("IRE MVR: WBC", "Table 4", "0.94", "0.89", "0.99", "0.04", 0.04),
("IRE MVR: pembrolizumab", "Table 4", "2.6", "1.05", "6.3", "0.04", 0.04),
("IRE MVR: nivolumab", "Table 4", "2.6", "1.04", "6.6", "0.04", 0.04),
]
results = []
for lab, src, e, lo, hi, prs, prv in CONTROLS:
lo_f, hi_f = float(lo), float(hi)
_, zc, pc = p_two_sided(np.sqrt(lo_f * hi_f), lo_f, hi_f)
a, b = p_range(e, lo, hi, use_printed_est=False)
wa, wb = p_range(e, lo, hi)
results.append((lab, src, prs, prv, pc, a, b))
print(f" {lab:24s} {src}: {e} ({lo}-{hi}) printed P {prs:>6s} | "
f"recomputed {pc:.4f}; range {a:.4f}-{b:.4f}; wide {wa:.4f}-{wb:.4f}")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": FG,
"axes.edgecolor": FG2, "axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def style(ax):
ax.grid(axis="x", which="both", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# Figure 1: printed interval vs required interval, one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.7))
style(ax)
ax.set_xscale("log")
ax.axvline(1, color=FG2, lw=0.9, zorder=1)
ax.text(1, 2.62, "OR = 1 (no association)", ha="center", va="bottom", color=FG2, fontsize=10)
y_p, y_r = 1.75, 0.75
ax.plot([LO, HI], [y_p, y_p], color=ORANGE, lw=2, solid_capstyle="butt", zorder=3)
for x in (LO, HI):
ax.plot([x, x], [y_p - 0.08, y_p + 0.08], color=ORANGE, lw=2, zorder=3)
ax.plot(OR, y_p, "o", ms=9, color=ORANGE, zorder=4)
ax.plot(mid, y_p, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=5)
ax.text(LO, y_p + 0.17, "Printed: OR 1.004, 95% CI 0.7–1.4, stated P = 0.006",
ha="left", va="bottom", color=ORANGE, fontsize=10.5, fontweight="bold")
ax.text(HI, y_p - 0.17, f"Recomputed from this CI: centre {mid:.3f}; P = {p:.2f} for OR 1.004",
ha="right", va="top", color=BLUE, fontsize=10)
ax.plot([req_lo, req_hi], [y_r, y_r], color=BLUE, lw=2, zorder=3)
ax.plot(OR, y_r, "D", ms=8, color=BLUE, zorder=4)
ax.text(LO, y_r + 0.17,
f"Recomputed: 95% CI needed for OR 1.004 with P = 0.006: {req_lo:.4f}–{req_hi:.4f}",
ha="left", va="bottom", color=BLUE, fontsize=10.5, fontweight="bold")
ax.text(1.03, y_r, "(narrower than the marker at this scale)",
ha="left", va="center", color=FG2, fontsize=10)
ax.set_xlim(0.62, 1.6)
ax.set_ylim(0.25, 2.9)
ticks = [0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.4]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks([])
ax.set_xlabel("Odds ratio for mortality per unit PLR (log scale)")
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper"),
Line2D([], [], color=BLUE, marker="D", ms=8, lw=2, label="Recomputed from the paper's numbers")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.5, -0.24), ncol=2,
frameon=False, fontsize=10)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)
# Figure 2: printed vs recomputed P values across rows, one log axis
fig, ax = plt.subplots(figsize=(7.2, 5.0))
style(ax)
ax.set_xscale("log")
n = len(results)
ys = np.arange(n)[::-1]
ax.axvline(0.05, color=FG2, lw=0.9, ls=(0, (4, 3)), zorder=1)
ax.text(0.05, n - 0.35, "P = 0.05", ha="center", va="bottom", color=FG2, fontsize=10)
for y, (lab, src, prs, prv, pc, a, b) in zip(ys, results):
ax.plot([a, b], [y, y], color=BLUE, alpha=0.35, lw=7, solid_capstyle="butt", zorder=2)
ax.plot(pc, y, "D", ms=8, color=BLUE, zorder=4)
ax.plot(prv, y, "o", ms=9, mfc=ORANGE, mec=ORANGE, zorder=3)
if prs.startswith("<"):
ax.annotate("", xy=(prv * 0.55, y), xytext=(prv, y),
arrowprops=dict(arrowstyle="->", color=ORANGE, lw=2))
ax.set_yticks(ys)
ax.set_yticklabels([f"{r[0]} ({r[1]})" for r in results])
for t, r in zip(ax.get_yticklabels(), results):
if r[0] == "Mortality MVR: PLR":
t.set_fontweight("bold"); t.set_color(FG)
ax.set_xlim(1e-4, 1.5)
ax.set_ylim(-0.7, n - 0.1)
ax.set_xticks([1e-4, 1e-3, 1e-2, 0.05, 1])
ax.set_xticklabels(["0.0001", "0.001", "0.01", "0.05", "1"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_xlabel("Two-sided P value (log scale)")
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=0, label="P printed in the paper"),
Line2D([], [], color=BLUE, marker="D", ms=8, lw=0, label="P recomputed from the printed 95% CI"),
Line2D([], [], color=BLUE, alpha=0.35, lw=7, label="Range over rounding of the printed CI bounds")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.3, -0.14), ncol=1,
frameon=False, fontsize=10)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
plt.close(fig)
print(line)
print("Figures written to", HERE)
The abstract's Y-maze P = 0.065, called significant, conflicts with the Results' P = 0.015
BMC Med (2013) · doi:10.1186/1741-7015-11-90 · PMC3616817
Summary
The paper reports the same Y-maze spatial-novelty comparison (control IgG vs 16/6-Id-injected mice) three ways. The abstract gives 42% vs 9% at P = 0.065 and calls the difference "significantly higher". The Results text gives 46% vs 9% at P = 0.015, and the Figure 2 legend gives 0.46 vs 0.09 at P = 0.02. The paper prints no Y-maze SEMs, so I measured the means and SEMs from the Figure 2 bars. A t-test rebuilt from those values, with the group sizes in Methods, gives P = 0.015. That matches the Results text and the legend; it does not match the abstract's 42% or P = 0.065. The abstract also attaches a P-value above the paper's own 0.05 threshold to a claim of significance.
What the paper reports
Abstract, Results:
Similarly, the preference for spatial novelty in the Y-maze test was significantly higher in the control group compared to the 16/6-Id-injected mice (42% vs. 9%, respectively, P = 0.065).
Results, "Cognitive and behavioral performance":
The control IgG mice spent 46% additional time in the new lane while the mice injected with 16/6-Id spent 9% additional time in the new lane (P = 0.015 by t-test).
Figure 2 legend:
Results are presented as the proportion of time (mean ± SEM) spent in the new arm introduced by the 16/6-Id (gray bars) and IgG control (black bars) injected mice.
There was a significant difference in additional time spent in the new lane between the 16/6 and IgG group (0.46 vs. 0.09, P = 0.02 respectively). * Statistically significant (P <0.05).
Methods, "Experimental design":
Twenty-one CH3 mice were injected ICV to the right hemisphere, 11 with human 16/6-Id antibodies and 10 with commercial human IgG antibodies (control).
Methods, "Statistical analysis":
The differences in mean for average immobility time in the FST, the staircase test parameters (number of rearing and stair-climbing events), novel object recognition and Y-maze tests were evaluated by T-test. Significant results were determined as P <0.05.
What the paper's own numbers show
Method. The text gives no Y-maze SEMs and no per-mouse data. I therefore measured the two bars of Figure 2 from the pixels of the open-access image in PubMed Central (1741-7015-11-90-2.jpg, 600 × 430 px). The axis ticks sit at row 378 (0%) and row 25 (100%). The intermediate ticks check the scale: they read 60.1 and 39.9. One pixel equals 0.28 percentage points. I took the bar tops as the means and the error-bar caps as mean + SEM. From these I rebuilt a two-sample t-test (SD = SEM × √n) with n = 10 control and n = 11 16/6-Id from Methods. I then varied every measured edge by ±1 px, the group sizes, the test (Student vs Welch) and one- vs two-sided P. recompute.py.txt downloads the image, checks its SHA-256, and prints every number below.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Control preference | Abstract | 42% | 46.7% (Fig. 2 bar) |
| Control preference | Results text / Fig. 2 legend | 46% / 0.46 | 46.7% (Fig. 2 bar) |
| 16/6-Id preference | Abstract / Results / legend | 9% / 9% / 0.09 | 10.5% (Fig. 2 bar) |
| SEM, control / 16/6-Id | Fig. 2 error bars | not stated in numbers | 12.5 / 6.2 points |
| Between-group P, Student t, n = 10/11 | Abstract | 0.065 | 0.015 (±1 px on every edge: 0.011–0.020) |
| Between-group P, Student t, n = 10/11 | Results text | 0.015 | 0.015: matches |
| Between-group P | Fig. 2 legend | 0.02 | 0.015 rounds to 0.02: consistent |
| Between-group P, Welch t, n = 10/11 | — | — | 0.022 (±1 px: 0.016–0.028) |
| P with the abstract's means (42 vs 9) and the figure's SEMs | Abstract | 0.065 | 0.025 (Student) – 0.034 (Welch) |
| P with other group sizes (11/10, 10/10, 9/11, 8/8) | — | — | 0.013–0.026 |
| P with only 5 mice per group | — | — | 0.032 (Student) – 0.041 (Welch) |
| SE of the difference: figure / needed for P = 0.065 | derived | — | 13.5 / 18.9 points (difference 37) or 16.8 (difference 33) |
| |t| needed, df 19: P = 0.065 vs P = 0.015 | derived | — | 1.96 vs 2.67 |
| Ratio 0.065 / 0.015 | derived | — | 4.33 (a one- vs two-sided mix-up gives exactly 2) |
No variant I tried reproduces P = 0.065. Every variant built from the figure, including those that use the abstract's own 42% vs 9%, falls below 0.05. With the figure's means and SEMs, Student P reaches 0.065 only with groups of 2 to 3 mice (n = 3 gives 0.060; n = 2 gives 0.121), and the study injected 10 and 11.
Figure
The conflicting excerpts, quoted from the paper (printed numbers in orange). The horizontal connector marks the conflict between the abstract (42% vs 9%, P = 0.065) and the Results text (46% vs 9%, P = 0.015). The left vertical connector marks the conflict inside the abstract: P = 0.065 is not below the Methods threshold of P < 0.05, yet the result is called "significantly higher". The right vertical connector shows that the legend's P = 0.02 agrees with the Results text after rounding. The blue box gives the means and SEMs measured from the bars of the paper's Figure 2 and the t-test rebuilt from them, which support the Results text.
Two-sided P values for the control vs 16/6-Id Y-maze comparison on one log axis. Orange dots are the three values printed in the paper. Blue dots are recomputed from the Figure 2 bars. Their lines show the range when every measured edge moves by ±1 px (Student and Welch rows), or the span between the Student and Welch tests (other rows). The dashed line is the paper's threshold, P = 0.05. Only the abstract's P = 0.065 lies above it. Every recomputed value lies below it, including the one that uses the abstract's own means.
Why it matters
The abstract is the part of the paper most readers see, and it is what indexing services reproduce. As written, it reports a non-significant P-value under a claim of significance. A reader of the abstract alone cannot tell whether the spatial-memory effect met the paper's threshold. The direction of the effect is the same in all three versions: the controls prefer the new arm by 33 or 37 percentage points more than the injected mice.
The Y-maze is the paper's only test of spatial memory. It is the sole support for the "spatial" half of the Discussion's statement that the antibody caused "a selective cognitive impairment, expressed as visual recognition and spatial memory deficits". It also supports the Discussion's statement that "the 16/6-Id is related to spatial novelty and visual recognition memory impairments in mice", which the paper uses to argue for hippocampal involvement. The headline conclusion of cognitive impairment rests also on the novel object recognition test (P = 0.012) and on immunohistochemistry. If the Results text and Figure 2 are correct, as the measured bars indicate, the conclusions stand and only the abstract needs correcting. If the abstract's P = 0.065 were correct, the spatial-memory claim would not meet the paper's own criterion. Without per-mouse data that cannot be ruled out with certainty, although the figure argues against it.
Explanations considered
- Rounding. P = 0.015 rounds to 0.02, which accounts for the legend. No rounding turns 0.015 into 0.065, and none turns 46% (or the measured 46.7%) into 42%.
- One-sided vs two-sided test. Halving or doubling a P-value gives a ratio of 2. The ratio 0.065 / 0.015 is 4.33. The figure gives a one-sided P of 0.007 and a two-sided P of 0.015, so neither matches 0.065.
- Student vs Welch t-test. From the figure, Student gives 0.015 and Welch gives 0.022. Both are below 0.05 and both are far from 0.065.
- Fewer mice in the Y-maze (exclusions). The Methods for the object recognition test allow animals to be excluded, so the Y-maze groups might be smaller than 10/11. With the figure's means and SEMs, P is 0.013–0.026 for 8–11 mice per group and 0.032–0.041 for 5 per group. It reaches 0.065 only at 2–3 mice per group. Nothing in the paper suggests groups that small.
- A different scale for the preference measure. The Methods define the index as (new − old)/(new + old), while the legend speaks of the "proportion of time" in the new arm. Converting between these scales is linear, so t and P do not change. A change of scale also cannot move one mean (46 to 42) while leaving the other (9) the same.
- The abstract's numbers come from a different analysis or an earlier dataset. This cannot be checked from the paper. Even taking the abstract's 42% vs 9% with the figure's SEMs gives P = 0.025–0.034. To reach 0.065, the standard error of the difference would have to be 16.8 points; the figure shows 13.5.
- A non-parametric test. A rank test on 10 and 11 mice could give a larger P than the t-test. The paper, however, states that the Y-maze was analysed by t-test, and reports no other test. This cannot be ruled out without per-mouse data, but nothing in the paper supports it.
- A typographical slip (0.015 printed as 0.065, 46 as 42). This would account for the numbers but cannot be confirmed from the paper. Even so, it would not account for the abstract calling P = 0.065 significant.
- A conversion or archiving error. Ruled out. The publisher's version of record carries the same text as PMC and Europe PMC.
Separately, and outside the claim above, the novel object recognition result for the 16/6-Id group is given as P = 0.655 in the Results text but P = 0.5 in the Figure 1 legend. The Figure 1 bars for that group read 51.4% (new) and 39.5% (old), against 56% and 44% in the text. The control bars (63.9% and 36.1%) match the text's 64% and 36%. Neither value changes that the 16/6-Id group showed no significant preference.
Status
As of 2026-10-24:
- Version of record. The publisher's page (link.springer.com/article/10.1186/1741-7015-11-90, reached from doi.org through bmcmedicine.biomedcentral.com; "Published: 04 April 2013") carries the same abstract, Results, Figure 2 legend and Methods wording as PMC and Europe PMC. The discrepancy is in the version of record.
- Supplements. The article has no supplementary material. The publisher lists only the authors' original image files. Figure 2 is the nearest thing to source data, and I measured it as described above.
- Corrections and notices. None found. Crossref has no update-to or updated-by relation; its only relation is an F1000 recommendation (has-review). PMC and Europe PMC list no correction, erratum, expression of concern or retraction. Europe PMC lists only a "Comment in" link to the related commentary (BMC Med 2013;11:91), which is an editorial, not a correction. Crossref's Retraction Watch data show no entry, and the publisher's page has no update notice.
- PubPeer. The PubPeer API required human verification, so it could not be queried directly. A web search found no PubPeer thread about this article.
- Sources. The PMC archive of the article (PMC3616817); PMC open-access image 1741-7015-11-90-2.jpg (s3://pmc-oa-opendata/PMC3616817.1/); the Springer/BMC article page; Europe PMC full-text XML and annotations; Crossref metadata for doi:10.1186/1741-7015-11-90.
Suggested correction
If the Results text and Figure 2 are correct, as the measured bars indicate, the abstract's sentence could read:
Similarly, the preference for spatial novelty in the Y-maze test was significantly higher in the control group compared to the 16/6-Id-injected mice (46% vs. 9%, respectively, P = 0.015).
If the abstract's P = 0.065 is the correct value, the word "significantly" should be removed from the abstract, and the Results text and Figure 2 legend should be brought into line with it. The Discussion and Conclusions should then be qualified where they rely on a spatial-memory deficit. Either way, stating the Y-maze group sizes and SEMs, or depositing the per-mouse preference indices, would settle which value is correct. The same would resolve the Figure 1 legend's P = 0.5 against the Results text's P = 0.655 for the 16/6-Id group in the object recognition test.
Recomputation script (python recompute.py.txt)
"""Recompute the Y-maze comparison of PMC3616817
(BMC Med 2013, doi:10.1186/1741-7015-11-90, 16/6-Id antibodies injected into mouse brain).
Run: python recompute.py.txt
Needs numpy, scipy, matplotlib and Pillow. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs copied from the paper as archived in PMC (same text in the publisher's version):
Abstract: control vs 16/6-Id preference 42% vs 9%, P = 0.065, "significantly higher"
Results: 46% vs 9%, P = 0.015 by t-test
Figure 2 legend: 0.46 vs 0.09, P = 0.02; "* Statistically significant (P <0.05)"; mean +/- SEM
Methods: 11 mice injected with 16/6-Id, 10 with control IgG; t-test; significance P < 0.05
Figure 1 legend / Results (side check): novel object recognition, control 64% vs 36%,
16/6-Id 56% vs 44% (P = 0.655 in Results, P = 0.5 in legend)
The paper prints no SEM for the Y-maze. The group means and SEMs are therefore measured from the
pixels of the Figure 2 bar chart (PMC open-access image 1741-7015-11-90-2.jpg, 600 x 430 px).
The script downloads that image into memory (it is not saved) and checks its SHA-256; if the
download fails it uses the pixel rows recorded below, which it found in the image on 2026-10-24.
"""
import hashlib
import io
import os
import urllib.request
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
BASE = "https://pmc-oa-opendata.s3.amazonaws.com/PMC3616817.1/"
IMG = {
"fig2": ("1741-7015-11-90-2.jpg", "8ff743cab73943cc7848b119473b00ae647efb71f76b52c48e51fb76856ee2dc"),
"fig1": ("1741-7015-11-90-1.jpg", "6b6cdb0450bf66ceb26de2e823b809a19f57a4ac03f57b017f8d76840344123f"),
}
# Pixel rows found in the images on 2026-10-24 (used only if the download fails)
RECORDED = {
"fig2": {"ticks": [25, 166, 237, 378], "ctrl_top": 213, "ctrl_cap": 169, "id_top": 341, "id_cap": 319},
"fig1": {"ticks": [24, 165, 235, 305, 376], "ctrl_old": 249, "id_old": 237, "ctrl_new": 151, "id_new": 195},
}
N_CTRL, N_ID = 10, 11 # Methods
REP = { # reported values: (control, 16/6-Id, P)
"Abstract": (42, 9, 0.065),
"Results text": (46, 9, 0.015),
"Fig. 2 legend": (46, 9, 0.02), # printed as 0.46 vs 0.09
}
ALPHA = 0.05 # Methods: "Significant results were determined as P <0.05."
def load(key):
name, sha = IMG[key]
try:
raw = urllib.request.urlopen(BASE + name, timeout=30).read()
if hashlib.sha256(raw).hexdigest() != sha:
print(f"[{key}] downloaded image has a different SHA-256; using recorded pixel rows")
return None
from PIL import Image
return np.asarray(Image.open(io.BytesIO(raw)).convert("L")).astype(float)
except Exception as e: # offline
print(f"[{key}] download failed ({e.__class__.__name__}); using recorded pixel rows")
return None
def first_dark(a, col, start, thr=100):
c = a[:, col]
return next(i for i in range(start, len(c)) if c[i] < thr <= c[i - 1])
def tick_rows(a, col, thr=100):
c = a[:, col]
return [i for i in range(1, len(c)) if c[i] < thr <= c[i - 1]]
# ---------------------------------------------------------------- 1. digitise Figure 2
print("=== 1. Figure 2 (Y-maze) measured from its pixels ===")
a = load("fig2")
if a is not None:
f2 = {"ticks": tick_rows(a, 80),
"ctrl_top": first_dark(a, 230, 130), # top of the black control bar (below the bracket at row 121)
"ctrl_cap": first_dark(a, 215, 130), # control error-bar cap
"id_top": first_dark(a, 445, 130), # top outline of the grey 16/6-Id bar
"id_cap": first_dark(a, 438, 130)} # 16/6-Id error-bar cap
assert f2 == RECORDED["fig2"], f2
else:
f2 = RECORDED["fig2"]
y100, y60, y40, y0 = f2["ticks"]
ppu = (y0 - y100) / 100.0
def val(r):
return (y0 - r) / ppu
print(f"axis: 0% at row {y0}, 100% at row {y100} ({ppu:.2f} px per percentage point); "
f"check ticks: row {y60} -> {val(y60):.1f}, row {y40} -> {val(y40):.1f}")
m_c, m_i = val(f2["ctrl_top"]), val(f2["id_top"])
s_c, s_i = val(f2["ctrl_cap"]) - m_c, val(f2["id_cap"]) - m_i
print(f"control: mean {m_c:.1f}%, SEM {s_c:.1f} 16/6-Id: mean {m_i:.1f}%, SEM {s_i:.1f}")
print(f"difference {m_c - m_i:.1f} points; reading uncertainty +/-1 px = +/-{1 / ppu:.2f} points")
def tt(m1, s1, n1, m2, s2, n2):
sd1, sd2 = s1 * np.sqrt(n1), s2 * np.sqrt(n2)
st = stats.ttest_ind_from_stats(m1, sd1, n1, m2, sd2, n2, equal_var=True)
we = stats.ttest_ind_from_stats(m1, sd1, n1, m2, sd2, n2, equal_var=False)
return st.statistic, st.pvalue, we.statistic, we.pvalue
# ---------------------------------------------------------------- 2. t-tests from the figure
print("\n=== 2. Two-sample t-tests rebuilt from the measured means and SEMs (n = 10 / 11) ===")
R = {}
ts, ps, tw, pw = tt(m_c, s_c, N_CTRL, m_i, s_i, N_ID)
R["fig_student"], R["fig_welch"] = ps, pw
print(f"Student t = {ts:.2f}, two-sided P = {ps:.4f} (one-sided {ps / 2:.4f})")
print(f"Welch t = {tw:.2f}, two-sided P = {pw:.4f} (one-sided {pw / 2:.4f})")
# sensitivity: every bar top and cap moved by -1, 0, +1 px
pst, pwe = [], []
for d1 in (-1, 0, 1):
for d2 in (-1, 0, 1):
for d3 in (-1, 0, 1):
for d4 in (-1, 0, 1):
mc, mi_ = val(f2["ctrl_top"] + d1), val(f2["id_top"] + d2)
sc, si = val(f2["ctrl_cap"] + d3) - mc, val(f2["id_cap"] + d4) - mi_
_, p1, _, p2 = tt(mc, sc, N_CTRL, mi_, si, N_ID)
pst.append(p1)
pwe.append(p2)
R["fig_student_rng"], R["fig_welch_rng"] = (min(pst), max(pst)), (min(pwe), max(pwe))
print(f"+/-1 px on all four edges: Student P {min(pst):.4f}-{max(pst):.4f}; Welch P {min(pwe):.4f}-{max(pwe):.4f}")
print("\nOther group sizes (same means and SEMs):")
alt = {}
for n1, n2 in [(11, 10), (10, 10), (9, 11), (8, 8), (5, 5)]:
_, p1, _, p2 = tt(m_c, s_c, n1, m_i, s_i, n2)
alt[(n1, n2)] = (p1, p2)
print(f" n = {n1}/{n2}: Student P = {p1:.4f}, Welch P = {p2:.4f}")
R["n55"] = alt[(5, 5)]
print("\nReported means with the measured SEMs (n = 10 / 11):")
for lab, (c, i, _) in REP.items():
if lab == "Fig. 2 legend":
continue
_, p1, _, p2 = tt(c, s_c, N_CTRL, i, s_i, N_ID)
R[lab] = (p1, p2)
print(f" {lab} {c}% vs {i}%: Student P = {p1:.4f}, Welch P = {p2:.4f}")
# ---------------------------------------------------------------- 3. what P = 0.065 would need
print("\n=== 3. What the abstract's P = 0.065 would require ===")
df = N_CTRL + N_ID - 2
for p in (0.065, 0.02, 0.015):
print(f"P = {p}: |t| needed (two-sided, df {df}) = {stats.t.isf(p / 2, df):.2f}; one-sided = {stats.t.isf(p, df):.2f}")
print(f"ratio 0.065 / 0.015 = {0.065 / 0.015:.2f} (a one- vs two-sided mix-up gives exactly 2)")
print(f"0.015 rounded to 2 decimals = {round(0.015 + 1e-12, 2)}")
sp = np.sqrt(((N_CTRL - 1) * (s_c * np.sqrt(N_CTRL)) ** 2 + (N_ID - 1) * (s_i * np.sqrt(N_ID)) ** 2) / df)
se_pooled = sp * np.sqrt(1 / N_CTRL + 1 / N_ID)
print(f"SE of the difference from the figure (pooled, Student) = {se_pooled:.1f} points")
for d in (46 - 9, 42 - 9):
print(f"difference {d}: SE needed for P = 0.015 is {d / stats.t.isf(0.015 / 2, df):.1f}; "
f"for P = 0.065 it is {d / stats.t.isf(0.065 / 2, df):.1f}")
# smallest equal group size at which the figure's t statistic would give P >= 0.065
t_fig = ts
for n in range(10, 1, -1):
_, p1, _, _ = tt(m_c, s_c, n, m_i, s_i, n)
if p1 >= 0.065:
_, p_prev, _, _ = tt(m_c, s_c, n + 1, m_i, s_i, n + 1)
print(f"with the figure's means and SEMs, Student P first reaches 0.065 at n = {n} per group "
f"(P = {p1:.3f}; n = {n + 1} gives {p_prev:.3f})")
break
print(f"monotone rescaling check: proportion = (1 + index)/2 maps 0.46 -> {(1 + 0.46) / 2:.3f}, "
f"0.09 -> {(1 + 0.09) / 2:.3f}; a linear change of scale leaves t and P unchanged")
# ---------------------------------------------------------------- 4. Figure 1 side check
print("\n=== 4. Side check: Figure 1 (novel object recognition) measured from its pixels ===")
b = load("fig1")
if b is not None:
f1 = {"ticks": tick_rows(b, 79),
"ctrl_old": first_dark(b, 146, 200), "id_old": first_dark(b, 226, 200),
"ctrl_new": first_dark(b, 386, 130), "id_new": first_dark(b, 466, 180)}
assert f1 == RECORDED["fig1"], f1
else:
f1 = RECORDED["fig1"]
z0, z100 = f1["ticks"][-1], f1["ticks"][0]
v1 = lambda r: (z0 - r) / ((z0 - z100) / 100.0) # noqa: E731
F1 = {k: v1(f1[k]) for k in ("ctrl_new", "ctrl_old", "id_new", "id_old")}
print(f"control new {F1['ctrl_new']:.1f}% / old {F1['ctrl_old']:.1f}% (sum {F1['ctrl_new'] + F1['ctrl_old']:.1f}); "
f"text 64 / 36")
print(f"16/6-Id new {F1['id_new']:.1f}% / old {F1['id_old']:.1f}% (sum {F1['id_new'] + F1['id_old']:.1f}); "
f"text 56 / 44; P = 0.655 (Results) vs P = 0.5 (legend)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "axes.edgecolor": "#bdbcb7",
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), bbox_inches="tight", pad_inches=0.15)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.15)
plt.close(fig)
# --- fig1: text panels holding the conflicting excerpts, joined by annotated connectors
fig = plt.figure(figsize=(7.2, 5.8))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 104)
ax.axis("off")
RED = "#b23b12"
def panel(x, y, w, h, title, lines):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="#ffffff", ec="#bdbcb7", lw=1.5))
ax.text(x + 2, y + h - 2.5, title, fontsize=11, fontweight="bold", va="top", color=TXT)
yy = y + h - 9
for txt, col, wt in lines:
ax.text(x + 2, yy, txt, fontsize=10, va="top", color=col, fontweight=wt)
yy -= 5.6
panel(2, 62, 38, 34, "Abstract (Results)", [
("“significantly higher in the", TXT, "normal"),
("control group”", TXT, "normal"),
("42% vs. 9%, P = 0.065", ORA, "bold"),
])
panel(60, 62, 38, 34, "Results text", [
("“additional time in the", TXT, "normal"),
("new lane”", TXT, "normal"),
("46% vs 9%, P = 0.015", ORA, "bold"),
("by t-test", TXT, "normal"),
])
panel(60, 23, 38, 30, "Figure 2 legend", [
("“significant difference”", TXT, "normal"),
("0.46 vs. 0.09, P = 0.02", ORA, "bold"),
("* Statistically significant", TXT, "normal"),
("(P <0.05)", TXT, "normal"),
])
panel(2, 23, 38, 30, "Methods", [
("“Significant results were", TXT, "normal"),
("determined as P <0.05.”", TXT, "normal"),
("t-test; n = 10 control,", TXT, "normal"),
("11 16/6-Id", TXT, "normal"),
])
def conn(p1, p2, label, col, lx, ly, ha="center"):
ax.add_patch(FancyArrowPatch(p1, p2, arrowstyle="<->", mutation_scale=12, lw=2, color=col))
ax.text(lx, ly, label, fontsize=10, ha=ha, va="center", color=col, linespacing=1.3)
conn((41.5, 70), (58.5, 70), "", RED, 50, 70)
ax.text(50, 85, "CONFLICT", fontsize=10, ha="center", va="center", color=RED, fontweight="bold")
ax.text(50, 78.5, "control\n42% vs 46%;\nP 0.065 vs 0.015", fontsize=10, ha="center", va="center", color=RED,
linespacing=1.3)
conn((8, 61), (8, 54), "CONFLICT: P = 0.065 is not < 0.05,\nyet called “significantly higher”", RED, 10, 57.5,
ha="left")
conn((66, 61), (66, 54), "consistent: 0.015\nrounds to 0.02", SEC, 68, 57.5, ha="left")
ax.add_patch(FancyBboxPatch((2, 2), 96, 15, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="#eef4fc", ec=BLU, lw=1.5))
ax.text(4, 15.8, "Measured from the bars of the paper’s Figure 2 (recomputed)", fontsize=10.5, fontweight="bold",
va="top", color=TXT)
ax.text(4, 10.6, f"control {m_c:.1f}% ± {s_c:.1f}, 16/6-Id {m_i:.1f}% ± {s_i:.1f} (mean ± SEM)",
fontsize=10, va="top", color=BLU, fontweight="bold")
ax.text(4, 5.6, f"t-test with n = 10 / 11: P = {ps:.3f} (Student), {pw:.3f} (Welch)",
fontsize=10, va="top", color=BLU, fontweight="bold")
ax.text(98, 103.5, "orange: printed in the paper blue: recomputed", fontsize=10, ha="right", va="top", color=SEC)
save(fig, "fig1")
# --- fig2: reported vs recomputed P values on one log axis
rows = [
("Abstract (42% vs 9%)", "rep", 0.065, None),
("Results text (46% vs 9%)", "rep", 0.015, None),
("Fig. 2 legend (0.46 vs 0.09)", "rep", 0.02, None),
("Fig. 2 bars, Student t, n = 10/11", "rec", R["fig_student"], R["fig_student_rng"]),
("Fig. 2 bars, Welch t, n = 10/11", "rec", R["fig_welch"], R["fig_welch_rng"]),
("Abstract means 42 vs 9, Fig. 2 SEMs", "rec", R["Abstract"][0], (R["Abstract"][0], R["Abstract"][1])),
("Fig. 2 bars, n = 5/5 (Student–Welch)", "rec", R["n55"][0], (R["n55"][0], R["n55"][1])),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
ys = np.arange(len(rows))[::-1]
for y, (lab, kind, p, rng) in zip(ys, rows):
col = ORA if kind == "rep" else BLU
if rng is not None and rng[1] > rng[0] * 1.02:
ax.plot(rng, [y, y], color=col, lw=2, solid_capstyle="butt")
ax.plot([p], [y], "o", color=col, ms=9, zorder=3)
txt = f"{p:.3f}" if kind == "rec" else f"{p:g}"
if rng is not None and rng[1] > rng[0] * 1.02:
txt = f"{rng[0]:.3f}–{rng[1]:.3f}"
xt = rng[1] * 1.08
else:
xt = p * 1.08
ax.text(xt, y, txt, va="center", ha="left", fontsize=10, color=col,
bbox=dict(boxstyle="square,pad=0.1", fc=BG, ec="none"), zorder=4)
ax.axvline(ALPHA, color=SEC, lw=1, ls="--", zorder=1)
ax.text(ALPHA * 1.03, ys[0] + 0.55, "P = 0.05 (paper’s threshold)", fontsize=10, color=SEC, va="center")
ax.set_xscale("log")
ax.set_xlim(0.006, 0.2)
ax.set_xticks([0.01, 0.02, 0.05, 0.1, 0.2])
ax.set_xticklabels(["0.01", "0.02", "0.05", "0.1", "0.2"])
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.7, len(rows) - 0.2)
ax.set_xlabel("Two-sided P value, control vs 16/6-Id, Y-maze (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], marker="o", ls="", color=ORA, ms=9, label="reported in the paper"),
Line2D([], [], marker="o", ls="-", lw=2, color=BLU, ms=9,
label="recomputed from Fig. 2 (line: ±1 px reading range or Student–Welch)")],
loc="upper center", bbox_to_anchor=(0.3, -0.2), ncol=1, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract's risk-factor-adjusted hazard ratios for three fat measures match a BMI-adjusted supplementary model, not Table 2
BMC Med (2022) · doi:10.1186/s12916-022-02505-y · PMC9484252
Summary
The abstract says that "after adjustments for cardiovascular risk factors", total fat mass, gynoid fat mass and the android-to-gynoid fat ratio had hazard ratios for new-onset atrial fibrillation (AF) of 1.33 (1.05–1.68), 1.36 (1.12–1.65) and 0.81 (0.70–0.94). Table 2 (Model 2) and the Results text describe the same adjustment, and both give 1.22 (1.09–1.36), 1.26 (1.13–1.41) and 0.85 (0.74–0.98). The three abstract values match exactly, to every printed digit, the Model 2 column of Supplementary Table S6, which adds body mass index (BMI) to the cardiovascular-risk-factor model. The fourth estimate in the same abstract sentence, epicardial fat at 1.27 (1.09–1.48), comes from the model without BMI. So the abstract labels two different models with one description. As a result it overstates the risk-factor-adjusted hazard ratios for total and gynoid fat by about 8–9% and gives intervals 1.75 to 2.1 times wider than the model it names. No direction or significance changes, and the paper's lean-mass-adjusted results and its conclusions are not affected.
What the paper reports
Abstract, Results, second sentence:
After adjustments for cardiovascular risk factors, absolute fat mass (HR; 95% CI 1.33; 1.05–1.68), gynoid fat mass (HR; 95% CI 1.36; 1.12–1.65), epicardial fat mass (HR; 95% CI 1.27; 1.09-1.48), and android-to-gynoid fat ratio (HR; 95% CI 0.81; 0.70-0.94) were independently associated with new-onset AF.
Abstract, Results, third sentence:
After further adjustment for lean mass, associations between fat mass (HR; 95% CI 1.17; 1.04-1.32), gynoid fat mass (HR; 95% CI 1.21; 1.08–1.37), and android-to-gynoid fat ratio (HR; 95% CI 0.84; 0.72–0.97) remained statistically significant.
Methods, Statistical analysis, which defines the model the abstract describes:
Model 2 was additionally adjusted for cardiovascular risk factors including total and HDL cholesterol, history of hypertension, history of diabetes mellitus (DM), history of coronary heart disease (CHD), history of heart failure (HF), history of left ventricular hypertrophy, smoking status, total alcohol intake, use of lipid-lowering medication, and use of cardiac medication. Moreover, we additionally adjusted for total lean mass in Model 3 and for total fat mass in Model 4, respectively.
Results, "Association of different fat depots with incident AF":
After adjustment for cardiovascular risk factors (model 2), higher levels of fat depots were significantly associated with new-onset AF with the exception for liver fat. The multivariable-adjusted HRs and 95% CIs were 1.22 (1.09–1.36) for fat mass, 1.14 (1.03–1.27) for android mass, 1.26 (1.13–1.41) for gynoid fat mass, and 1.27 (1.09–1.48) for epicardial fat. Notably, android-to-gynoid fat mass ratio showed an inverse association with incident AF [0.85 (0.74–0.98)].
Table 2 ("Associations between various DXA-based fat depots and incident atrial fibrillation", N = 3468), columns Model 1 | Model 2 | Model 3 | Model 4:
Total fat mass | 1.28 (1.15–1.42) | 1.22 (1.09–1.36) | 1.17 (1.04–1.32) | –
Gynoid fat mass | 1.31 (1.18–1.46) | 1.26 (1.13–1.41) | 1.21 (1.08–1.37) | 1.36 (0.96–1.93)
Android-to-gynoidfat ratio | 0.93 (0.81–1.06) | 0.85 (0.74–0.98) | 0.84 (0.72–0.97) | 0.82 (0.70–0.95)
Table 3 (CT-based), Model 2 column:
Epicardial fat | 1.31 (1.13–1.51) | 1.27 (1.09–1.48) | 1.16 (0.95–1.42) | 1.09 (0.87–1.38)
Additional file 1, Table S6, "Associations between various fat depots and incident atrial fibrillation, adjusting for body mass index and waist-to-hip ratio", columns Model 1 | Model 2 | Model 3 (text taken from the supplementary .docx; hyphens as printed there):
Total fat mass | 1.22 (1.09-1.36) | 1.33 (1.05-1.68) | 1.27 (1.13-1.42)
Gynoid fat mass | 1.26 (1.13-1.41) | 1.36 (1.12-1.65) | 1.29 (1.15-1.44)
Android-to-gynoid fat ratio | 0.85 (0.74-0.98) | 0.81 (0.70-0.94) | 0.77 (0.65-0.91)
Epicardial fat | 1.27 (1.09-1.48) | 1.18 (0.99-1.41) | 1.32 (1.11-1.57)
Table S6 footnote, excerpt:
Model 1 was adjusted for age, high-density lipoprotein cholesterol, total cholesterol, smoking, total alcohol intake, lipid lowering medication and cardiac medication, history of hypertension, left ventricular hypertrophy, diabetes mellitus, heart failure and coronary heart disease. Model 2: Model 1 + body mass index. Model 3: Model 1 + waist-to-hip ratio.
Discussion, "Study strengths and limitations", which cites Table S6:
Of note, findings indicated a robust and independent association between gynoid fat and android-to-gynoid fat ratio with AF after adjusting for BMI, WHR, or total lean mass, (Additional file 1: Table. S6)
What the paper's own numbers show
Method. I matched each hazard ratio (95% CI) in the abstract's "after adjustments for cardiovascular risk factors" sentence against every column of Tables 2 and 3 and of Supplementary Table S6. Assuming Wald intervals on the log scale, I took the implied standard error of each log hazard ratio as SE = [ln(upper) − ln(lower)] / (2 × 1.96). To make sure the differences are not a rounding effect, I also bounded each SE by moving the printed limits ±0.005. Finally, as a rough benchmark, I compared each SE with the precision the event count allows: for a standardized exposure, SE ≈ 1/√events. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Total fat mass, adjusted for CV risk factors | Abstract vs Table 2 Model 2 and Results | 1.33 (1.05–1.68) vs 1.22 (1.09–1.36) | Differ by +0.11 (+9.0%). Abstract equals Table S6 Model 2 (+BMI) exactly |
| Gynoid fat mass, adjusted for CV risk factors | Abstract vs Table 2 Model 2 and Results | 1.36 (1.12–1.65) vs 1.26 (1.13–1.41) | Differ by +0.10 (+7.9%). Abstract equals Table S6 Model 2 (+BMI) exactly |
| Android-to-gynoid ratio, adjusted for CV risk factors | Abstract vs Table 2 Model 2 and Results | 0.81 (0.70–0.94) vs 0.85 (0.74–0.98) | Differ by −0.04 (−4.7%). Abstract equals Table S6 Model 2 (+BMI) exactly |
| Epicardial fat, same abstract sentence | Abstract vs Table 3 Model 2 | 1.27 (1.09–1.48) vs 1.27 (1.09–1.48) | Identical. Table S6 Model 2 (+BMI) gives 1.18 (0.99–1.41), p ≈ 0.067 |
| Table S6 Model 1 vs Table 2/3 Model 2 | Supplement vs main tables | 1.22, 1.26, 0.85, 1.27 with the same CIs | Identical for all four, so S6 Model 1 is the main-text Model 2 |
| Implied SE of log HR, total fat | Abstract vs Table 2 Model 2 | — | 0.120 vs 0.056. Ratio 2.12 (2.01–2.24 allowing for rounding) |
| Implied SE of log HR, gynoid fat | Abstract vs Table 2 Model 2 | — | 0.099 vs 0.056. Ratio 1.75 (1.66–1.85) |
| Implied SE of log HR, A/G ratio | Abstract vs Table 2 Model 2 | — | 0.075 vs 0.072. Ratio 1.05 (0.96–1.14) |
| Benchmark SE for 395 events | Abstract and Results (395 AF cases, DXA) | — | 1/√395 = 0.050. Table 2 Model 2 SE 0.056 implies variance inflation of 1.26. Abstract SE 0.120 implies 5.68, which fits adding a covariate as collinear as BMI |
| Abstract values in any Table 2 column | Table 2, Models 1–4 | — | No exact match. Only the gynoid point estimate 1.36 recurs, in Model 4, with CI 0.96–1.93 |
| Lean-mass-adjusted values | Abstract vs Table 2 Model 3 | 1.17 (1.04–1.32); 1.21 (1.08–1.37); 0.84 (0.72–0.97) | Identical in both places |
All three discordant abstract estimates reproduce the corresponding Table S6 Model 2 (+BMI) entries digit for digit. Their intervals are internally consistent: the geometric midpoints are 1.328, 1.359 and 0.811. Their widths are the ones expected after adding BMI, a covariate strongly correlated with total and gynoid fat. The SE for total fat roughly doubles, from 0.056 to 0.120, which no rounding of two-decimal limits can produce. The abstract sentence therefore reports three estimates from the BMI-adjusted model and one from the model without BMI, and describes all four with the wording that the Methods and Results use for Model 2.
Figure
Figure 1. Three panels give the hazard ratios (95% CI) per 1 SD for the four fat measures in the abstract's "after adjustments for cardiovascular risk factors" sentence. Centre (orange): the abstract. Left (blue): Table 2/Table 3 Model 2 and the Results text, which the Methods define as the cardiovascular-risk-factor model. Right (dark grey): Supplementary Table S6 Model 2, which is that model plus body mass index. Each connector is labelled "same" (solid) or "differs" (dashed). For total fat mass, gynoid fat mass and the android-to-gynoid ratio, the abstract differs from Table 2 and equals Table S6 Model 2 exactly. For epicardial fat, the abstract equals Table 3 and differs from Table S6 Model 2.
Figure 2. The same estimates on one log-scale hazard-ratio axis, with a thin reference line at 1 (no association). Orange circles: abstract. Blue squares: Table 2/3 Model 2 (cardiovascular risk factors). Dark grey open diamonds: Table S6 Model 2 (cardiovascular risk factors + BMI). Each value is also printed at the right. For total and gynoid fat the abstract interval lies exactly on the BMI-adjusted one, and it is about twice as wide on the log scale as the Model 2 interval. For epicardial fat the abstract lies on Model 2, and the BMI-adjusted interval would cross 1. All estimates stay on the same side of 1 in every model except BMI-adjusted epicardial fat.
Why it matters
- The abstract misstates the size and precision of a named model. A reader of the abstract would take 1.33 and 1.36 as the risk-factor-adjusted hazard ratios per SD of total and gynoid fat. The model the paper defines with those words gives 1.22 and 1.26, with intervals about half as wide on the log scale. Anyone who pools or compares effect sizes from the abstract, for example in a meta-analysis of fat mass and AF, would carry over the wrong estimate and the wrong variance. The abstract SE for total fat is 2.1 times the Model 2 SE, so the inverse-variance weight would be about 4.5 times too small.
- One sentence mixes two models. Under the BMI-adjusted model that supplies three of its four numbers, epicardial fat would be 1.18 (0.99–1.41). That interval includes 1, so the sentence's claim that all four depots "were independently associated with new-onset AF" holds only because epicardial fat is taken from the other model.
- The headline conclusions are unaffected. The corrected Model 2 estimates keep the same direction and all remain significant (p ≈ 0.0004, < 0.0001 and 0.023). The abstract's lean-mass-adjusted estimates match Table 2 Model 3. Its other results (body fat score 1.10, subcutaneous-pattern 1.21) are not involved. Table S6 itself, cited in the Discussion, supports associations of total fat, gynoid fat and the ratio after BMI adjustment. The BMI-adjusted results are therefore legitimate analyses of the paper. The error is that the abstract labels them as the cardiovascular-risk-factor model.
- Side observation. The abstract's Conclusions say "Total fat mass and gynoid fat mass were independently associated with AF after adjustment for body size". The main-text Conclusions say only "Gynoid fat mass was independently associated with AF after taking body size into account." Both versions are supported by the paper's Model 3 or Table S6 estimates, and this difference is not part of the numerical finding above.
Explanations considered
- Rounding or transcription noise. Ruled out. The three abstract estimates differ from Model 2 by 0.04 to 0.11 on the HR scale, and their intervals are not compatible: allowing for two-decimal rounding of the limits, the SE ratio is 2.01–2.24 for total fat and 1.66–1.85 for gynoid fat. Every digit of all three values equals a single other column of the paper, which noise would not produce.
- A different column of Table 2 (Model 1, 3 or 4). Ruled out. None of Models 1–4 gives any of the three hazard ratio–CI pairs. The gynoid point estimate 1.36 also appears in Model 4, but there its interval is 0.96–1.93, not 1.12–1.65, and Model 4 is not described as the cardiovascular-risk-factor model.
- A subgroup or sensitivity analysis described elsewhere as risk-factor-adjusted. Ruled out as the source. Table S3, the analysis excluding prevalent cardiovascular disease, gives 1.23 (1.08–1.40), 1.27 (1.11–1.45) and 0.84 (0.71–0.99) for Model 2. A text search of the whole supplement finds each of the three abstract hazard ratio–CI pairs exactly once, so the sex- and BMI-stratified results do not match either. The only exact match is Table S6 Model 2, whose footnote defines it as "Model 1 + body mass index".
- "Cardiovascular risk factors" was meant to include BMI. Not supported by the paper's own definitions. The Methods list of Model 2 covariates, and the footnotes of Tables 2, 3 and 4, do not include BMI. The Results text gives the non-BMI values under the same wording. The abstract's epicardial estimate in the same sentence is the non-BMI value. Table S6 itself sets "+ body mass index" apart from the risk-factor model.
- A conversion or typesetting artifact. Ruled out. The same abstract numbers appear on the PMC page, in the Europe PMC full-text XML (the publisher's JATS file), and in the abstract on the publisher's springermedicine.com platform (see Status). In the JATS file the Table 2 cells sit in their expected columns.
- A correction already issued. None found (see Status).
Status
- Version of record. As of 2026-10-24. The publisher's article page (doi.org → bmcmedicine.biomedcentral.com → link.springer.com) redirected automated requests to a sign-in page, and the publisher's PDF links returned a challenge page instead of a PDF. Three copies of the published text were compared instead, and all carry the numbers quoted here: (1) the PMC page for PMC9484252; (2) the Europe PMC full-text JATS XML (publication date 19 September 2022); (3) the abstract on springermedicine.com. Every quotation from the article was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. Additional file 1 (
12916_2022_2505_MOESM1_ESM.docx, the only supplementary file, from static-content.springer.com) was downloaded and its text extracted. Tables S3 and S6 are quoted or summarized above. S6 Model 1 reproduces the main-text Model 2 for every depot, and S6 Model 2 (+BMI) supplies the three discordant abstract values. The supplement explains where the abstract numbers come from, but it does not remove the mislabelling in the abstract. - Corrections and comments. I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by entry and an empty relation field. The Europe PMC record (PMID 36117169) has no comment or correction links. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title with "PubPeer" found no thread. - Sources. PMC article PMC9484252; Europe PMC full-text XML and record; Additional file 1 from static-content.springer.com; springermedicine.com abstract page; Crossref; PubPeer; https://doi.org/10.1186/s12916-022-02505-y.
Suggested correction
In the abstract, either report the Model 2 estimates that the stated adjustment refers to:
After adjustments for cardiovascular risk factors, absolute fat mass (HR; 95% CI 1.22; 1.09–1.36), gynoid fat mass (HR; 95% CI 1.26; 1.13–1.41), epicardial fat mass (HR; 95% CI 1.27; 1.09–1.48), and android-to-gynoid fat ratio (HR; 95% CI 0.85; 0.74–0.98) were independently associated with new-onset AF.
or, if the BMI-adjusted estimates are intended, label them as such, for example "After adjustment for cardiovascular risk factors and body mass index, absolute fat mass (HR 1.33; 1.05–1.68), gynoid fat mass (HR 1.36; 1.12–1.65) and android-to-gynoid fat ratio (HR 0.81; 0.70–0.94) …". In that case the epicardial fat estimate from the same model, 1.18 (0.99–1.41), should replace 1.27 (1.09–1.48) or be reported separately. Nothing in Tables 2–4, the Results text or the supplement needs to change.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC9484252 write-up and redraw its figures.
Run: python recompute.py.txt
All inputs are hazard ratios (95% CI) printed in the article (abstract, Results text,
Tables 2 and 3) and in Additional file 1, Table S6. Figures are written next to this file.
"""
import math
import os
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ---------------------------------------------------------------------------
# Inputs, transcribed from the paper. (HR, lower, upper)
# ---------------------------------------------------------------------------
ABSTRACT_M2 = { # Abstract, Results: "After adjustments for cardiovascular risk factors, ..."
"Total fat mass": (1.33, 1.05, 1.68),
"Gynoid fat mass": (1.36, 1.12, 1.65),
"Android-to-gynoid ratio": (0.81, 0.70, 0.94),
"Epicardial fat": (1.27, 1.09, 1.48),
}
TABLE_M2 = { # Table 2 / Table 3, Model 2 (= Results text)
"Total fat mass": (1.22, 1.09, 1.36),
"Gynoid fat mass": (1.26, 1.13, 1.41),
"Android-to-gynoid ratio": (0.85, 0.74, 0.98),
"Epicardial fat": (1.27, 1.09, 1.48),
}
TABLE_M1 = { # Table 2 / Table 3, Model 1 (age and sex)
"Total fat mass": (1.28, 1.15, 1.42),
"Gynoid fat mass": (1.31, 1.18, 1.46),
"Android-to-gynoid ratio": (0.93, 0.81, 1.06),
"Epicardial fat": (1.31, 1.13, 1.51),
}
TABLE_M3 = { # Table 2, Model 3 (Model 2 + total lean mass)
"Total fat mass": (1.17, 1.04, 1.32),
"Gynoid fat mass": (1.21, 1.08, 1.37),
"Android-to-gynoid ratio": (0.84, 0.72, 0.97),
}
TABLE_M4 = { # Table 2, Model 4 (Model 2 + total fat mass)
"Gynoid fat mass": (1.36, 0.96, 1.93),
"Android-to-gynoid ratio": (0.82, 0.70, 0.95),
}
ABSTRACT_M3 = { # Abstract: "After further adjustment for lean mass, ..."
"Total fat mass": (1.17, 1.04, 1.32),
"Gynoid fat mass": (1.21, 1.08, 1.37),
"Android-to-gynoid ratio": (0.84, 0.72, 0.97),
}
S6_M1 = { # Additional file 1, Table S6, Model 1 (cardiovascular risk factors)
"Total fat mass": (1.22, 1.09, 1.36),
"Gynoid fat mass": (1.26, 1.13, 1.41),
"Android-to-gynoid ratio": (0.85, 0.74, 0.98),
"Epicardial fat": (1.27, 1.09, 1.48),
}
S6_M2_BMI = { # Table S6, Model 2 = Model 1 + body mass index
"Total fat mass": (1.33, 1.05, 1.68),
"Gynoid fat mass": (1.36, 1.12, 1.65),
"Android-to-gynoid ratio": (0.81, 0.70, 0.94),
"Epicardial fat": (1.18, 0.99, 1.41),
}
S6_M3_WHR = { # Table S6, Model 3 = Model 1 + waist-to-hip ratio
"Total fat mass": (1.27, 1.13, 1.42),
"Gynoid fat mass": (1.29, 1.15, 1.44),
"Android-to-gynoid ratio": (0.77, 0.65, 0.91),
"Epicardial fat": (1.32, 1.11, 1.57),
}
EVENTS_DXA = 395 # incident AF cases, DXA analysis (Abstract and Results)
EVENTS_CT = 172 # incident AF cases, CT analysis
def se(lo, hi):
"""SE of log HR implied by a symmetric Wald 95% CI on the log scale."""
return (math.log(hi) - math.log(lo)) / (2 * Z)
def se_range(lo, hi):
"""Smallest and largest SE compatible with the CI limits rounded to 2 decimals."""
return se(lo + 0.005, hi - 0.005), se(lo - 0.005, hi + 0.005)
def geo_mid(lo, hi):
return math.exp((math.log(lo) + math.log(hi)) / 2)
def pval(hr, lo, hi):
return 2 * norm.sf(abs(math.log(hr)) / se(lo, hi))
def fmt(t):
return f"{t[0]:.2f} ({t[1]:.2f}-{t[2]:.2f})"
print("=" * 100)
print("1. Abstract 'adjusted for cardiovascular risk factors' values vs the paper's Model 2 and Table S6")
print("=" * 100)
hdr = f"{'measure':24s} {'source':28s} {'HR (95% CI)':>18s} {'geo-mid':>8s} {'SE logHR':>9s} {'SE range*':>15s} {'p':>8s}"
print(hdr)
sources = [("Abstract", ABSTRACT_M2), ("Table 2/3 Model 2 = Results", TABLE_M2),
("Table S6 Model 1 (CV RF)", S6_M1), ("Table S6 Model 2 (+BMI)", S6_M2_BMI),
("Table S6 Model 3 (+WHR)", S6_M3_WHR), ("Table 2/3 Model 1", TABLE_M1)]
for k in ABSTRACT_M2:
for name, d in sources:
h, l, u = d[k]
a, b = se_range(l, u)
print(f"{k:24s} {name:28s} {fmt(d[k]):>18s} {geo_mid(l, u):8.3f} {se(l, u):9.4f} {a:7.4f}-{b:.4f} {pval(h, l, u):8.4f}")
A, T, B = ABSTRACT_M2[k], TABLE_M2[k], S6_M2_BMI[k]
print(f" abstract == Table 2/3 Model 2: {A == T}; abstract == Table S6 Model 2 (+BMI): {A == B};"
f" Table S6 Model 1 == Table 2/3 Model 2: {S6_M1[k] == T}")
print(f" abstract - Model 2 HR = {A[0] - T[0]:+.2f} ({100 * (A[0] / T[0] - 1):+.1f}%);"
f" SE ratio abstract / Model 2 = {se(A[1], A[2]) / se(T[1], T[2]):.2f}"
f" (range {se_range(A[1], A[2])[0] / se_range(T[1], T[2])[1]:.2f}-{se_range(A[1], A[2])[1] / se_range(T[1], T[2])[0]:.2f});"
f" CI width ratio on log scale = {math.log(A[2] / A[1]) / math.log(T[2] / T[1]):.2f};"
f" inverse-variance weight ratio Model 2 / abstract = {(se(A[1], A[2]) / se(T[1], T[2])) ** 2:.2f}")
print()
print("* SE range: smallest and largest SE compatible with CI limits rounded to two decimals.")
print()
print("=" * 100)
print("2. Can the abstract values be any printed column of Table 2?")
print("=" * 100)
cols = {"Model 1": TABLE_M1, "Model 2": TABLE_M2, "Model 3": TABLE_M3, "Model 4": TABLE_M4}
for k in ["Total fat mass", "Gynoid fat mass", "Android-to-gynoid ratio"]:
hits = [c for c, d in cols.items() if d.get(k) == ABSTRACT_M2[k]]
pe = [c for c, d in cols.items() if k in d and d[k][0] == ABSTRACT_M2[k][0]]
print(f"{k:24s} exact HR+CI match in Table 2: {hits or 'none'}; point-estimate-only match: {pe or 'none'}")
if k in TABLE_M4:
print(f"{'':24s} Table 2 Model 4 = {fmt(TABLE_M4[k])}")
print()
print("=" * 100)
print("3. Abstract lean-mass-adjusted values vs Table 2 Model 3")
print("=" * 100)
for k in ABSTRACT_M3:
print(f"{k:24s} abstract {fmt(ABSTRACT_M3[k])} Table 2 Model 3 {fmt(TABLE_M3[k])} equal: {ABSTRACT_M3[k] == TABLE_M3[k]}")
print()
print("=" * 100)
print("4. Precision expected from the number of events")
print("=" * 100)
se0 = 1 / math.sqrt(EVENTS_DXA)
print(f"SE(log HR per 1 SD) for an unadjusted standardized exposure ~ 1/sqrt(events) = 1/sqrt({EVENTS_DXA}) = {se0:.4f}")
for k in ["Total fat mass", "Gynoid fat mass", "Android-to-gynoid ratio"]:
for name, d in [("Table 2 Model 2", TABLE_M2), ("Abstract / S6 +BMI", ABSTRACT_M2)]:
s = se(d[k][1], d[k][2])
print(f"{k:24s} {name:20s} SE {s:.4f} implied variance inflation (SE/{se0:.4f})^2 = {(s / se0) ** 2:.2f}")
print(f"CT analysis: 1/sqrt({EVENTS_CT}) = {1 / math.sqrt(EVENTS_CT):.4f}; epicardial Model 2 SE = {se(1.09, 1.48):.4f}")
print()
print("=" * 100)
print("5. Epicardial fat: what the same abstract sentence would say under the BMI-adjusted model")
print("=" * 100)
for name, d in [("Abstract", ABSTRACT_M2), ("Table 3 Model 2", TABLE_M2), ("Table S6 Model 2 (+BMI)", S6_M2_BMI)]:
h, l, u = d["Epicardial fat"]
print(f"{name:24s} {fmt(d['Epicardial fat'])} p = {pval(h, l, u):.4f} CI excludes 1: {l > 1 or u < 1}")
# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAYM = "#eb6834", "#2a78d6", "#e4e3df", "#52514e"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none"})
ROWS = ["Total fat mass", "Gynoid fat mass", "Android-to-gynoid ratio", "Epicardial fat"]
SHORT = {"Total fat mass": "Total fat mass", "Gynoid fat mass": "Gynoid fat mass",
"Android-to-gynoid ratio": "Android/gynoid ratio", "Epicardial fat": "Epicardial fat"}
def ci_txt(t):
return f"{t[0]:.2f} ({t[1]:.2f}–{t[2]:.2f})"
# ---- Figure 1: three text panels joined by labelled connectors -------------
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.6)
ax.axis("off")
ax.set_facecolor(BG)
panels = [ # x0, width, title, subtitle, data, colour
(0.10, 2.05, "Table 2/3, Model 2\nand Results text", "“adjusted for cardiovascular\nrisk factors (model 2)”", TABLE_M2, BLUE),
(2.58, 2.05, "Abstract, Results", "“After adjustments for cardio-\nvascular risk factors”", ABSTRACT_M2, ORANGE),
(5.06, 2.05, "Suppl. Table S6, Model 2", "“Model 1 + body mass\nindex”", S6_M2_BMI, GRAYM),
]
top, bottom = 3.98, 0.62
row_y = [2.78, 2.20, 1.62, 1.04]
for x0, w, title, sub, data, col in panels:
ax.add_patch(FancyBboxPatch((x0, bottom), w, top - bottom, boxstyle="round,pad=0.02,rounding_size=0.06",
facecolor="white", edgecolor=col, linewidth=1.5))
ax.text(x0 + w / 2, top - 0.08, title, ha="center", va="top", fontsize=10.5, weight="bold", color=col)
ax.text(x0 + w / 2, top - 0.52, sub, ha="center", va="top", fontsize=10, color=SEC, style="italic")
for k, y in zip(ROWS, row_y):
ax.text(x0 + w / 2, y + 0.11, SHORT[k], ha="center", va="center", fontsize=10, color=SEC)
ax.text(x0 + w / 2, y - 0.12, ci_txt(data[k]), ha="center", va="center", fontsize=11, color=col, weight="bold")
for k, y in zip(ROWS, row_y):
for xa, xb, other in [(2.15, 2.58, TABLE_M2), (4.63, 5.06, S6_M2_BMI)]:
same = ABSTRACT_M2[k] == other[k]
xm = (xa + xb) / 2
ax.plot([xa + 0.02, xb - 0.02], [y - 0.12, y - 0.12], color=SEC if same else ORANGE,
lw=2, ls="-" if same else (0, (2, 1.5)))
ax.text(xm, y + 0.06, "same" if same else "differs", ha="center", va="center", fontsize=10,
color=SEC if same else ORANGE, weight="bold",
bbox=dict(boxstyle="round,pad=0.08", fc=BG, ec="none"))
ax.text(3.6, 4.36, "Same label, different model: three abstract estimates are the BMI-adjusted ones",
ha="center", va="center", fontsize=10.5, weight="bold", color=TXT)
ax.text(3.6, 0.30, "Hazard ratio (95% CI) per 1 SD. Implied SE of log HR, abstract vs Model 2:\n"
f"total fat {se(1.05, 1.68):.3f} vs {se(1.09, 1.36):.3f}; gynoid fat {se(1.12, 1.65):.3f} vs {se(1.13, 1.41):.3f}.",
ha="center", va="center", fontsize=10, color=SEC)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ---- Figure 2: dot-and-interval plot on one log axis ------------------------
fig, ax = plt.subplots(figsize=(7.2, 4.4), facecolor=BG)
ax.set_facecolor(BG)
series = [("Abstract (reported as CV-risk-adjusted)", ABSTRACT_M2, ORANGE, "o", "full"),
("Table 2/3 Model 2 = Results text (CV risk factors)", TABLE_M2, BLUE, "s", "full"),
("Suppl. Table S6 Model 2 (CV risk factors + BMI)", S6_M2_BMI, GRAYM, "D", "none")]
offs = [0.22, 0.0, -0.22]
yt = []
for i, k in enumerate(ROWS):
y0 = len(ROWS) - 1 - i
yt.append(y0)
for (lab, d, col, mk, fill), off in zip(series, offs):
h, l, u = d[k]
ax.plot([l, u], [y0 + off, y0 + off], color=col, lw=2, solid_capstyle="butt")
ax.plot([h], [y0 + off], marker=mk, ms=8, color=col, mfc=(col if fill == "full" else "white"),
mew=2, ls="none", label=lab if i == 0 else None)
ax.text(2.02, y0 + off, ci_txt(d[k]), va="center", ha="left", fontsize=10, color=col)
ax.axvline(1, color=SEC, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.6, 2.0)
ticks = [0.6, 0.7, 0.8, 0.9, 1.0, 1.2, 1.4, 1.6, 1.8, 2.0]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.set_yticks(yt)
ax.set_yticklabels([SHORT[k] for k in ROWS], color=TXT)
ax.set_ylim(-0.6, len(ROWS) - 0.4)
ax.grid(axis="x", color=GRID, lw=0.8)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Hazard ratio per 1 SD (log scale); line at 1 = no association")
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.16), ncol=1, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.22, right=0.76, top=0.97, bottom=0.36)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print()
print("Figures written to", HERE)
Abstract's 95% CI for the duration effect of tricyclics excludes its own point estimate
BMJ (2010) · doi:10.1136/bmj.c5222 · PMC2958257
Summary
The abstract reports that the benefit of tricyclic antidepressants for headache grew with treatment duration, with a metaregression slope of β = −0.11 and a 95% confidence interval of −0.63 to −0.15 (P < 0.0005). The point estimate −0.11 lies 0.04 above the upper limit of its own interval, which cannot happen for a 95% CI of any estimate. The interval's midpoint is −0.39, and its width implies a standard error under which −0.11 would have P ≈ 0.37, not P < 0.0005. The Results give only subgroup slopes, both internally consistent: tension-type −0.16 (−0.23 to −0.10) and migraine −0.09 (−0.14 to −0.04). These support the direction and significance of the abstract's finding, so the error concerns the printed precision of one abstract result, not the conclusion's direction.
What the paper reports
The PMC-archived text prints minus signs as hyphens; the quotations below keep that form. Every quotation was checked verbatim against the PMC text with src/show.py.
Abstract, Results:
The effect of tricyclics increased with longer duration of treatment (β=-0.11, 95% confidence interval -0.63 to -0.15; P<0.0005).
Abstract, Conclusions:
The effectiveness of tricyclics seems to increase over time.
Results, "Tricyclics versus placebo", second paragraph:
For both headache types this beneficial effect increased over time (tension-type: β=-0.16, 95% confidence interval -0.23 to -0.10, P<0.0005, migraine: β=-0.09, -0.14 to -0.04, P=0.001, fig 3). Therefore for every additional week of treatment with a tricyclic, the number of headaches were reduced by 0.16 standard deviations.
Fig 3 legend:
Fig 3 Effect of tricyclic antidepressants compared with placebo over time. SMD=standardised mean difference
Results, "Assessment of bias and sensitivity analyses":
Metaregression found no effect of any other variables abstracted on summary outcomes, with the exception of study duration. Duration explained 32.4% of the heterogeneity for continuous outcomes and 27.3% of the variance for dichotomous outcomes.
Methods, "Statistical analysis":
We pooled data using the DerSimonian and Laird random effects model
Because of the large number of analyses, we used P=0.01 as our threshold for significance.
All analyses were done using Stata (v 11.2).
Discussion, "Strengths and weaknesses of the study":
This analysis should be viewed as exploratory rather than definitive.
The average duration of the trials in our study was 11 weeks and the longest only 26 weeks; consequently our timeline was constrained to four to 26 weeks.
What the paper's own numbers show
Method. A 95% confidence interval produced by metaregression (as in Stata) is estimate ± 1.96 SE, so it must contain the estimate, and its half-width gives the SE. I checked whether each printed estimate lies inside its interval, took the midpoint and implied SE of each interval, and converted estimate ÷ SE into a two-sided P value to compare with the printed P. The paper reports no overall (pooled) duration slope outside the abstract. As a plausibility check on the abstract's −0.11, I pooled the two Results subgroup slopes by inverse variance (fixed effect). This is an approximation, not a re-run of the authors' metaregression, which used study-level data I do not have. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Overall duration slope: is the point inside its CI? | Abstract, Results | β = −0.11 (−0.63 to −0.15) | No. −0.11 is 0.04 above the upper limit −0.15 |
| Midpoint of the abstract interval | Abstract, Results | — | −0.39 |
| SE implied by the abstract interval width (0.48) | Abstract, Results | — | 0.1225 |
| P for β = −0.11 with that SE | Abstract, Results | P < 0.0005 | z = −0.90, P = 0.37 |
| Interval symmetric about −0.11 keeping the printed upper limit | Abstract, Results | — | lower limit −0.07, above the upper limit −0.15 (impossible) |
| Tension-type slope | Results, "Tricyclics versus placebo", para 2; Fig 3 | −0.16 (−0.23 to −0.10), P < 0.0005 | inside CI; midpoint −0.165; SE 0.0332; z = −4.82; P = 1.4 × 10−6. Consistent |
| Migraine slope | Results, para 2; Fig 3 | −0.09 (−0.14 to −0.04), P = 0.001 | inside CI; midpoint −0.090; SE 0.0255; z = −3.53; P = 0.0004. Consistent (printed P slightly conservative) |
| Inverse-variance pool of the two subgroup slopes | Derived from Results | (abstract: −0.11) | −0.116 (−0.156 to −0.076), z = −5.74, P = 9.6 × 10−9; width 0.079, one-sixth of the abstract's 0.48 |
| Heterogeneity between the two slopes | Derived from Results | — | Cochran Q = 2.80 on 1 df, P = 0.09 |
| One possible reading: a dropped zero, −0.063 to −0.15 (hypothesis) | Abstract, Results | — | contains −0.11; midpoint −0.1065; SE 0.0222; z = −4.96; P ≈ 7 × 10−7, which matches P < 0.0005 |
| Total change over the 11-week average trial at the printed lower limit | Abstract; Discussion (11 weeks) | −0.63 per week | −6.93 SD, against an overall SMD of −0.96 for tricyclics vs placebo (Results) |
Three separate checks point the same way. The printed point estimate is outside its interval. The printed interval width is inconsistent with the printed P value: if the SE were 0.12, as the width implies, −0.11 would be far from significant. And the printed interval is six times wider than the pooled interval of the two subgroups and wider than either subgroup interval, whereas an estimate that uses both subgroups would normally be more precise than either one alone. The point estimate and the P value, by contrast, agree with the subgroup results.
Figure
Figure 1. Metaregression slopes of the standardised mean difference (SMD) per additional week of treatment, with 95% confidence intervals, on one linear axis; the vertical line at 0 marks no effect of duration. Orange: values printed in the paper, namely the abstract's overall slope −0.11 (−0.63 to −0.15) and the Results subgroup slopes for tension-type −0.16 (−0.23 to −0.10) and migraine −0.09 (−0.14 to −0.04). Blue: values recomputed from those printed numbers, namely the midpoint of the abstract's interval (−0.39, diamond) and the inverse-variance pool of the two subgroup slopes, −0.116 (−0.156 to −0.076). The abstract's point estimate sits outside its interval; the subgroup estimates sit inside theirs, and the pooled subgroup estimate lies close to the abstract's −0.11 with a much narrower interval.
Why it matters
- An impossible interval in the abstract. The abstract is the part of the paper most often read and quoted, and this sentence is the only overall estimate of the duration effect anywhere in the paper. A reader who takes the interval at face value would conclude that the slope could be as steep as −0.63 SD per week, which over the 11-week average trial would be about 6.9 SD of extra benefit, seven times the overall treatment effect of −0.96 SD. The printed interval also cannot be reconciled with the printed P value.
- The supported conclusion is unaffected in direction. The direction (benefit grows with duration) and the statistical significance rest on the Results subgroup slopes, which are internally consistent, both below the paper's P = 0.01 threshold, and close to the abstract's −0.11 when pooled. The abstract conclusion "The effectiveness of tricyclics seems to increase over time" is therefore not reversed.
- The magnitude and its precision are not recoverable from the paper. Because no overall slope is reported in the body, the correct interval for −0.11 cannot be confirmed from the published text. A dropped leading zero (−0.063 to −0.15) would fit the point estimate and the P value, but that is only one possibility.
- Scope. The authors already describe the duration analysis as exploratory. The primary efficacy and adverse-effect results do not depend on this number.
Explanations considered
- A different model or population for the abstract's β. Partly plausible for the point estimate: the abstract's −0.11 may be an overall slope across both headache types rather than either subgroup, and the inverse-variance pool of the subgroups gives −0.116. This cannot explain the interval. Whatever the model, population or outcome, a 95% CI must contain the estimate it belongs to.
- A transformed or asymmetric scale. Not applicable. Metaregression coefficients on SMD are reported on the linear scale (the Results define β as SMD per additional week). Even an asymmetric interval on a log or ratio scale still contains its point estimate.
- A different confidence level, or a one-sided interval. Not able to explain it. Changing the level widens or narrows an interval around the estimate; it never moves the estimate outside. The Methods use P = 0.01 as a significance threshold but do not describe a different interval convention, and the subgroup intervals in the same paragraph behave as ordinary 95% intervals.
- A reversed-sign or reordered interval. Not able to explain it. Reading the limits in either order gives the same set, −0.63 to −0.15, which excludes −0.11. Changing signs would put the interval on the positive side and contradict both the point estimate and the stated direction of effect.
- A text-conversion artifact. Ruled out. The same string appears in the PMC article page, in the publisher-deposited Europe PMC JATS XML (which uses true Unicode minus signs on all three numbers) and in the PubMed abstract (PMID 20961988). The publisher's own page could not be reached (see Status), but three independent renderings of the publisher's text agree.
- A correct value given elsewhere. Not found. The Results text and the Fig 3 legend give only the two subgroup slopes; there are no supplementary tables (see Status).
- A correction already issued. None found (see Status).
- Side observation, outside the main finding. For tension-type headache versus SSRIs, the abstract gives −0.80 (−2.63 to 0.02) while the Results give −0.80 (−1.63 to 0.02, four studies). The lower limits differ by exactly 1.00, and only the Results interval is centred near −0.80 (midpoint −0.805 vs −1.305). The abstract's placebo comparisons (tension-type −1.29, −2.18 to −0.39; migraine −0.70, −0.93 to −0.48) also differ from the Results' "burden" estimates (−0.99, −1.66 to −0.32; −1.00, −1.52 to −0.48). The abstract describes these as headache frequency outcomes, so they may come from a different outcome definition; I did not assess them further.
Status
- Version of record. As of 2026-10-24, bmj.com (reached through doi.org) returned HTTP 403 to automated requests, and the publisher PDF could not be downloaded from bmj.com, PMC or Europe PMC. The abstract sentence was checked in three renderings of the publisher's text, which agree: the PMC article page, the Europe PMC full-text JATS XML, and the PubMed record (PMID 20961988).
- Supplements. The JATS XML has no supplementary-material elements and no web-extra or appendix links. The Europe PMC supplementary bundle holds only the seven figure images. Fig 3, the duration metaregression plot, is labelled only with the two subgroup slopes and shows no overall slope or interval. The paper states that it is based on a Cochrane review; that review could not be reached through the publisher sites.
- Corrections and comments. No correction, erratum, retraction or expression of concern was found.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by entries, and a Crossref filter for updates to this DOI returns nothing. The PubMed CommentsCorrections entries are two comments (the accompanying BMJ editorial, c5250, and an Evidence-Based Medicine commentary, doi:10.1136/ebm1176), neither an erratum. A Retraction Watch site search and web searches for the title together with "correction", "erratum" and "PubPeer" found nothing relevant. The PubPeer API could not be queried (Cloudflare block or HTTP 404 to automated requests), and the Europe PMC REST search was unavailable (HTTP 503) on several attempts, so those two sources are not fully confirmed. - Sources. PMC article PMC2958257; Europe PMC full-text XML and supplementary files for PMC2958257; PubMed 20961988; Crossref; https://doi.org/10.1136/bmj.c5222; Retraction Watch; PubPeer (attempted).
Suggested correction
The authors may wish to check the overall duration metaregression output and replace, in the abstract,
(β=-0.11, 95% confidence interval -0.63 to -0.15; P<0.0005)
with the interval that the analysis actually produced for β = −0.11. If the intended interval is −0.15 to −0.063 (one possible reading, consistent with the point estimate and P < 0.0005), the corrected text would read "(β=−0.11, 95% confidence interval −0.15 to −0.063; P<0.0005)". It would also help readers if the overall slope and its interval were reported in the Results, next to the subgroup slopes, so that the abstract figure can be traced. The authors may also wish to check the abstract's interval for tension-type headache versus SSRIs (−2.63 to 0.02), which the Results give as −1.63 to 0.02.
Recomputation script (python recompute.py.txt)
"""Recompute the duration metaregression numbers of PMC2958257
(BMJ 2010;341:c5222, doi:10.1136/bmj.c5222,
"Tricyclic antidepressants and headaches: systematic review and meta-analysis").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg and fig1.png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract, Results: "(β=-0.11, 95% confidence interval -0.63 to -0.15; P<0.0005)"
Results, Tricyclics versus placebo, para 2: "tension-type: β=-0.16, 95% confidence interval
-0.23 to -0.10, P<0.0005, migraine: β=-0.09, -0.14 to -0.04, P=0.001, fig 3"
Results, Tricyclics versus placebo, para 1: overall SMD -0.96 (-1.39 to -0.53)
Discussion, Strengths and weaknesses: "The average duration of the trials in our study was
11 weeks and the longest only 26 weeks"
Context only (abstract vs Results SMDs): see section 5.
Intervals are treated as normal-theory 95% intervals (estimate ± 1.96 SE), the usual output
of Stata metaregression; the paper names Stata v11.2.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ---------------------------------------------------------------- inputs
ABS = dict(b=-0.11, lo=-0.63, hi=-0.15, p="<0.0005") # abstract
SUB = { # Results, para 2
"tension-type": dict(b=-0.16, lo=-0.23, hi=-0.10, p="<0.0005"),
"migraine": dict(b=-0.09, lo=-0.14, hi=-0.04, p="0.001"),
}
OVERALL_SMD = -0.96 # Results, para 1 (both headache types combined)
MEAN_WEEKS, MAX_WEEKS = 11, 26 # Discussion
def p2(z):
return 2 * stats.norm.sf(abs(z))
# ---------------------------------------------------------------- 1. abstract interval
print("=== 1. Abstract: beta = %.2f, 95%% CI %.2f to %.2f, P%s ===" % (ABS["b"], ABS["lo"], ABS["hi"], ABS["p"]))
inside = ABS["lo"] <= ABS["b"] <= ABS["hi"]
print("Point estimate inside its interval? %s" % inside)
print("Distance of point above upper limit: %.2f" % (ABS["b"] - ABS["hi"]))
mid = (ABS["lo"] + ABS["hi"]) / 2
se_w = (ABS["hi"] - ABS["lo"]) / (2 * Z)
print("Midpoint of printed interval: %.3f" % mid)
print("Width of printed interval: %.2f" % (ABS["hi"] - ABS["lo"]))
print("SE implied by printed width: %.4f" % se_w)
print(" z, p for the midpoint with that SE: z=%.2f, p=%.2g" % (mid / se_w, p2(mid / se_w)))
print(" z, p for beta=-0.11 with that SE: z=%.2f, p=%.2f (paper: P<0.0005)" % (ABS["b"] / se_w, p2(ABS["b"] / se_w)))
print("Interval centred on -0.11 keeping lower -0.63: %.2f to %.2f" % (ABS["lo"], 2 * ABS["b"] - ABS["lo"]))
print("Interval centred on -0.11 keeping upper -0.15: %.2f to %.2f (lower > upper: impossible)" % (2 * ABS["b"] - ABS["hi"], ABS["hi"]))
# ---------------------------------------------------------------- 2. subgroup slopes
print("\n=== 2. Results: subgroup slopes (SMD per additional week) ===")
for k, d in SUB.items():
se = (d["hi"] - d["lo"]) / (2 * Z)
d["se"] = se
print("%-13s beta=%.2f (%.2f to %.2f): inside=%s, midpoint=%.3f, width=%.2f, SE=%.4f, z=%.2f, p=%.2g (paper P%s)"
% (k, d["b"], d["lo"], d["hi"], d["lo"] <= d["b"] <= d["hi"], (d["lo"] + d["hi"]) / 2,
d["hi"] - d["lo"], se, d["b"] / se, p2(d["b"] / se), d["p"] if d["p"].startswith("<") else "=" + d["p"]))
# ---------------------------------------------------------------- 3. pooled slope
print("\n=== 3. Inverse-variance (fixed-effect) pool of the two subgroup slopes ===")
w = np.array([1 / d["se"] ** 2 for d in SUB.values()])
b = np.array([d["b"] for d in SUB.values()])
bp = float((w * b).sum() / w.sum())
sp = float(w.sum() ** -0.5)
POOL = dict(b=bp, lo=bp - Z * sp, hi=bp + Z * sp)
print("Pooled beta = %.3f (95%% CI %.3f to %.3f), SE=%.4f, z=%.2f, p=%.2g"
% (bp, POOL["lo"], POOL["hi"], sp, bp / sp, p2(bp / sp)))
print("Pooled width %.3f vs printed abstract width %.2f (ratio %.1f)"
% (POOL["hi"] - POOL["lo"], ABS["hi"] - ABS["lo"], (ABS["hi"] - ABS["lo"]) / (POOL["hi"] - POOL["lo"])))
qstat = float((w * (b - bp) ** 2).sum())
print("Cochran Q for the two slopes: %.2f on 1 df, p=%.2f" % (qstat, stats.chi2.sf(qstat, 1)))
# ---------------------------------------------------------------- 4. one possible reading
print("\n=== 4. One possible reading (hypothesis only): a dropped zero, -0.063 to -0.15 ===")
H = dict(b=-0.11, lo=-0.15, hi=-0.063)
se_h = (H["hi"] - H["lo"]) / (2 * Z)
print("Contains -0.11? %s; midpoint=%.4f; SE=%.4f; z=%.2f; p=%.1g"
% (H["lo"] <= H["b"] <= H["hi"], (H["lo"] + H["hi"]) / 2, se_h, H["b"] / se_h, p2(H["b"] / se_h)))
# ---------------------------------------------------------------- 4b. magnitude implied
print("\n=== 4b. Implied change over the %d-week average trial ===" % MEAN_WEEKS)
for lab, v in [("printed lower limit -0.63", ABS["lo"]), ("printed upper limit -0.15", ABS["hi"]),
("point -0.11", ABS["b"]), ("pooled lower %.3f" % POOL["lo"], POOL["lo"])]:
print("%-28s x %d weeks = %.2f SD" % (lab, MEAN_WEEKS, v * MEAN_WEEKS))
print("Overall SMD tricyclic vs placebo (Results): %.2f" % OVERALL_SMD)
# ---------------------------------------------------------------- 5. context only
print("\n=== 5. Context only: abstract vs Results SMDs ===")
pairs = [("TTH vs placebo", (-1.29, -2.18, -0.39), (-0.99, -1.66, -0.32)),
("Migraine vs placebo", (-0.70, -0.93, -0.48), (-1.00, -1.52, -0.48)),
("TTH vs SSRI", (-0.80, -2.63, 0.02), (-0.80, -1.63, 0.02)),
("Migraine vs SSRI", (-0.20, -0.60, 0.19), (-0.22, -0.75, 0.31))]
for name, a, r in pairs:
print("%-20s abstract %.2f (%.2f to %.2f), midpoint %.3f | Results %.2f (%.2f to %.2f), midpoint %.3f"
% (name, a[0], a[1], a[2], (a[1] + a[2]) / 2, r[0], r[1], r[2], (r[1] + r[2]) / 2))
print("TTH vs SSRI lower limits differ by %.2f" % (pairs[2][2][1] - pairs[2][1][1]))
# ---------------------------------------------------------------- figure
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "svg.fonttype": "none",
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
})
LW = 2.0 * 72 / 96 # 2 px in points
MS = 9 # marker size in points (>= 8 px)
def f(v, nd=2):
return ("%%.%df" % nd % v).replace("-", "\u2212")
rows = [
# label, estimate, lo, hi, colour
("Abstract, overall\n%s (%s to %s)" % (f(ABS["b"]), f(ABS["lo"]), f(ABS["hi"])), ABS["b"], ABS["lo"], ABS["hi"], ORANGE),
("Results, tension-type\n%s (%s to %s)" % tuple(f(SUB["tension-type"][k]) for k in ("b", "lo", "hi")),
SUB["tension-type"]["b"], SUB["tension-type"]["lo"], SUB["tension-type"]["hi"], ORANGE),
("Results, migraine\n%s (%s to %s)" % tuple(f(SUB["migraine"][k]) for k in ("b", "lo", "hi")),
SUB["migraine"]["b"], SUB["migraine"]["lo"], SUB["migraine"]["hi"], ORANGE),
("Pool of the two\nResults slopes\n%s (%s to %s)" % tuple(f(POOL[k], 3) for k in ("b", "lo", "hi")),
POOL["b"], POOL["lo"], POOL["hi"], BLUE),
]
fig, ax = plt.subplots(figsize=(7.2, 3.8))
ys = np.arange(len(rows))[::-1].astype(float)
for (lab, est, lo, hi, col), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=LW * 1.6, solid_capstyle="butt", zorder=3)
for e in (lo, hi):
ax.plot([e, e], [y - 0.12, y + 0.12], color=col, lw=LW, zorder=3)
ax.plot(est, y, "o", ms=MS, color=col, mec=col, zorder=4)
y0 = ys[0]
ax.plot(mid, y0, "D", ms=MS - 1, color=BLUE, mec=BLUE, zorder=5)
ax.text(mid, y0 + 0.22, "midpoint of printed interval: %s (recomputed)" % f(mid), fontsize=10, color=BLUE,
ha="center", va="bottom")
ax.text(ABS["b"] + 0.018, y0, "point estimate lies\n%s above upper limit" % f(ABS["b"] - ABS["hi"]),
fontsize=10, color=ORANGE, ha="left", va="center")
ax.axvline(0, color=INK2, lw=0.8, zorder=1)
ax.text(0.006, ys[-1] - 0.45, "no effect", fontsize=10, color=INK2, ha="left", va="center")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(-0.68, 0.24)
ax.set_xticks([-0.6, -0.5, -0.4, -0.3, -0.2, -0.1, 0.0, 0.1, 0.2])
ax.set_ylim(ys[-1] - 0.7, ys[0] + 0.7)
ax.set_xlabel("Metaregression slope β: change in SMD per additional week of treatment")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([0], [0], color=ORANGE, lw=LW * 1.6, marker="o", ms=MS, label="reported in the paper (estimate and 95% CI)"),
Line2D([0], [0], color=BLUE, lw=LW * 1.6, marker="o", ms=MS, label="recomputed from the paper's own numbers")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.4, -0.2), ncol=2, frameon=False, fontsize=10,
handlelength=2.2)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, "fig1." + ext), bbox_inches="tight", pad_inches=0.12, **kw)
print("\nwrote fig1.svg and fig1.png in", HERE)
Prediabetes meta-analysis reports mortality risk difference 7.36 outside its own 95% CI (9.59 to 12.51)
BMJ (2020) · doi:10.1136/bmj.m2297 · PMC7362233
Summary
The abstract and the Results both give the absolute risk difference (ARD) for all cause mortality with prediabetes in the general population as "7.36 (95% confidence interval 9.59 to 12.51)" per 10 000 person years, so the estimate lies outside its own interval. The paper derives each ARD as comparator risk × (RR − 1). Its supplement gives the comparator risk as 73.6 per 10 000 person years, and the pooled relative risk is 1.13 (1.10 to 1.17), so the ARD works out to 9.57 (7.36 to 12.51). The printed estimate and the printed lower limit have swapped places. The other seven ARDs in the paper reproduce exactly by the same method. The absolute excess mortality the abstract gives is therefore about 23% too low (7.36 rather than about 9.6 per 10 000 person years), while the pooled relative risk and the paper's conclusions are unaffected.
What the paper reports
Abstract, Results, and again word for word in Results, section "All cause mortality and cardiovascular disease associated with prediabetes in the general population", third paragraph:
Compared with normoglycaemia, the absolute risk difference in prediabetes for all cause mortality, composite cardiovascular disease, coronary heart disease, and stroke was 7.36 (95% confidence interval 9.59 to 12.51), 8.75 (6.41 to 10.49), 6.59 (4.53 to 8.65), and 3.68 (2.10 to 5.26) per 10 000 person years, respectively.
Abstract, Results, with the relative risks these ARDs are derived from:
In the general population, prediabetes was associated with an increased risk of all cause mortality (relative risk 1.13, 95% confidence interval 1.10 to 1.17), composite cardiovascular disease (1.15, 1.11 to 1.18), coronary heart disease (1.16, 1.11 to 1.21), and stroke (1.14, 1.08 to 1.20) in a median follow-up time of 9.8 years.
Methods, "Statistical analysis", which defines the calculation:
We calculated the absolute risk difference for all cause mortality and cardiovascular outcomes associated with prediabetes by multiplying the assumed comparator risk of each outcome of interest by the estimated RR-1, according to the recommendation of the Cochrane guidelines.25 We used the median risks of outcomes in individuals with normoglycaemia across studies as the assumed comparator risks. Absolute risk differences were expressed as events per 10 000 person years.
Supplementary file 15 (supplement PDF, p. 54), "Men and women" row for all cause mortality. The columns are "Event rate in normoglycaemia (No of studies)" and "Absolute risk difference":
All cause mortality 73.6 (44) 7.36 (95% CI 9.59 to 12.51)
What the paper's own numbers show
Method. Under the stated method, the estimate and both 95% limits of an ARD are the comparator risk C multiplied by (RR − 1), (RR lower − 1) and (RR upper − 1), always with the same C. Two checks follow. (1) Multiply the supplement's C by the printed relative risks. (2) Divide each printed ARD number by its (RR − 1) and see whether it gives back the supplement's C. Because the RRs are printed to two decimals, each implied C carries a tolerance, worked out by moving the RR by ±0.005. I ran both checks on all eight ARD triples in the paper: four for the general population (supplementary file 15) and four for patients with atherosclerotic cardiovascular disease (ASCVD; supplementary file 21). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Comparator risk C, all cause mortality, general population | Supplementary file 15 | 73.6 per 10 000 PY (44 studies) | input |
| RR, all cause mortality, general population | Abstract; Results | 1.13 (1.10 to 1.17) | input |
| ARD, all cause mortality, general population | Abstract; Results; Supp. file 15 | 7.36 (9.59 to 12.51) | 9.57 (7.36 to 12.51) = 73.6 × (0.13; 0.10; 0.17) |
| Is the estimate inside its own CI? | same | 7.36 < 9.59 | no; 2.23 below the printed lower limit |
| C implied by printed estimate 7.36 / 0.13 | same | — | 56.6 (rounding range 54.5 to 58.9); does not fit 73.6 |
| C implied by printed lower limit 9.59 / 0.10 | same | — | 95.9 (rounding range 91.3 to 100.9); does not fit 73.6 |
| C implied by printed upper limit 12.51 / 0.17 | same | — | 73.6; fits |
| C implied by transposed triple 9.59 (7.36 to 12.51) | — | — | 73.8 / 73.6 / 73.6; all fit |
| RR implied by 9.59 with C = 73.6 | — | — | 1.1303, i.e. an unrounded RR that prints as 1.13 |
| ARD, composite CVD, general population (C = 58.3) | Abstract; Results | 8.75 (6.41 to 10.49) | 8.74 (6.41 to 10.49) |
| ARD, coronary heart disease, general population (C = 41.2) | Abstract; Results | 6.59 (4.53 to 8.65) | 6.59 (4.53 to 8.65) |
| ARD, stroke, general population (C = 26.3) | Abstract; Results | 3.68 (2.10 to 5.26) | 3.68 (2.10 to 5.26) |
| ARD, all cause mortality, ASCVD (C = 183.8) | Abstract; Results | 66.19 (38.60 to 99.25) | 66.17 (38.60 to 99.25) |
| ARD, composite CVD, ASCVD (C = 512.9) | Abstract; Results | 189.77 (117.97 to 271.84) | 189.77 (117.97 to 271.84) |
| ARD, coronary heart disease, ASCVD (C = 270.8) | Abstract; Results | 40.62 (5.42 to 78.53) | 40.62 (5.42 to 78.53) |
| ARD, stroke, ASCVD (C = 170.7) | Results; Supp. file 21 | 8.54 (−32.43 to 61.45) | 8.54 (−32.43 to 61.45) |
| Same, as printed in the abstract | Abstract | 8.54 (32.43 to 61.45) | lower limit is −32.43; the minus sign is missing |
| Printed numbers consistent with the supplement's C | all 8 triples | 24 numbers | 22 of 24; the two exceptions are the general-population mortality estimate and lower limit |
The method reproduces 22 of the 24 printed ARD numbers to within 0.02, which confirms that the paper's ARDs were computed exactly as its Methods describe. The only two that fail are the estimate and lower limit of a single triple, and exchanging those two makes that triple consistent too.
Figure
Figure 1. Absolute risk difference of prediabetes against normoglycaemia in the general population, per 10 000 person years, for the four outcomes in the abstract. Orange: the estimate and 95% CI as printed in the abstract and Results. Blue: recomputed by recompute.py.txt as the comparator event rate from supplementary file 15 × (RR − 1), using the printed relative risks and their limits. The numbers are given at the right. For composite CVD, coronary heart disease and stroke the printed and recomputed intervals coincide. For all cause mortality the printed estimate (7.36) lies to the left of its own interval, and the recomputed estimate, 9.57, lies inside the same interval, which runs from 7.36 to 12.51. The vertical line at 0 marks no difference; the axis starts at that line.
Figure 2. Each printed ARD number divided by its (RR − 1) gives the comparator risk it implies. The figure plots that implied risk as a ratio to the event rate stated in supplementary files 15 and 21, for all eight outcomes (circle: estimate; left triangle: lower limit; right triangle: upper limit). Grey markers fit the stated rate within the tolerance allowed by two-decimal rounding of the RR; all 22 of them lie at 1.00 (range 0.998 to 1.001). The two orange markers are printed values that do not fit: the general-population mortality estimate 7.36 implies 0.77 times the stated rate, and its lower limit 9.59 implies 1.30 times. Exchanging those two values puts both at 1.00.
Why it matters
- The headline absolute effect for a primary outcome is wrong in the abstract. All cause mortality is one of the two primary outcomes, and the ARD is the only absolute-scale figure the abstract gives for it. The printed estimate of 7.36 excess deaths per 10 000 person years is 23.1% lower than the 9.57 the paper's own method and inputs give. Readers who quote the absolute excess mortality from the abstract, for example in guidelines, decision aids or number-needed-to-screen calculations, will understate it.
- The printed triple cannot be used as it stands. An estimate outside its own interval shows a careful reader that something is wrong, but the text gives no way to tell which number is wrong. Two readings are plausible from the main text alone: the estimate is right and the interval is wrong (about 5.66 to 9.62), or the estimate is wrong. Only the comparator rate in supplementary file 15 settles it, in favour of the second.
- What is not affected. The pooled relative risk, 1.13 (1.10 to 1.17), its significance, and the paper's conclusions ("prediabetes was associated with an increased risk of all cause mortality and cardiovascular disease", and the remarks on screening) do not depend on this ARD. The correct interval, 7.36 to 12.51, is the one already printed, and it excludes zero.
- A second, smaller discrepancy. In the abstract, the ASCVD stroke ARD reads "8.54 (32.43 to 61.45)", which again puts the estimate outside its interval. The Results and supplementary file 21 give "−32.43", and 170.7 × (0.81 − 1) = −32.43. This one is easy to read correctly from the body text, and it matches the abstract's statement that there was "no difference" for stroke.
Explanations considered
- The estimate is right and the interval is wrong (the interval should be about 5.66 to 9.62). Not supported. This would require a comparator risk of 7.36 / 0.13 = 56.6. Supplementary file 15 gives 73.6, in the same row as the printed triple, and 56.6 lies outside the range that RR rounding allows (54.5 to 58.9).
- The estimate should be the midpoint of the printed interval (about 11.05). Not supported. ARDs are not built from symmetric intervals here, and 11.05 would require a comparator risk of 85.0, which does not match the supplement either.
- Rounding of the relative risk. Not sufficient. Varying the RR and its limits by ±0.005 moves the implied comparator risk by a few percent at most. It cannot reconcile the printed estimate (implied 56.6) or lower limit (implied 95.9) with 73.6, and the other 22 numbers fit to within 0.2%.
- A different population, model or subgroup. Not supported. The sentence and the supplement row are both "men and women" in the general population, and the sex-specific ARDs are shown separately in supplementary file 15 (men 15.80 (9.72 to 23.09); women 11.15 (6.27 to 16.03)), each with its estimate inside its interval. No other analysis in the paper reports an all cause mortality ARD of 7.36 or 9.59.
- A one-sided interval, different units or a different convention. Not supported. All ARDs are two-sided 95% CIs in events per 10 000 person years, and under the stated method an ARD for an RR above 1 lies between its limits by construction.
- A conversion artefact in the archived text. Ruled out. The same triple, without a minus sign, appears in the publisher-supplied JATS XML at Europe PMC, on the PMC page, and in the publisher's own supplement PDF. (For the ASCVD stroke value, the abstract in the publisher XML also lacks the minus sign that the Results carry.)
- Transposition of the estimate and the lower limit. This accounts for all the numbers. The triple 9.59 (7.36 to 12.51) implies comparator risks of 73.8, 73.6 and 73.6, matching the supplement. The small difference between 9.59 and 9.57 matches an unrounded RR of 1.1303, the same size as the 66.19 against 66.17 difference seen for ASCVD mortality. It describes the form the error takes; it says nothing about how it arose.
Status
As of 2026-10-24.
- Version of record. bmj.com returned HTTP 403 to automated requests, so the publisher's HTML page and PDF could not be read directly. The publisher-supplied JATS XML archived at Europe PMC (which links
bmj.m2297.pdfas its self-URI), the live PMC page and the PMC text all carry the same wording in the abstract and the Results. The same triple appears in the publisher's supplement PDF (caix056552.ww.pdf, 64 pages, from the Europe PMC supplementary-files bundle), supplementary file 15, p. 54. The error is therefore in the published source, not in conversion. - Supplements. Supplementary file 15 (p. 54) gives the general-population comparator rates, 73.6 / 58.3 / 41.2 / 26.3 per 10 000 person years, and repeats the printed ARDs. Supplementary file 21 (p. 61) gives the ASCVD rates, 183.8 / 512.9 / 270.8 / 170.7, and ARDs including stroke 8.54 (95% CI −32.43 to 61.45). The supplement resolves which number is wrong.
- Corrections. None found. Crossref shows no update-to, updated-by or relation entries, and a relation filter and a bibliographic search for a BMJ correction found nothing. PMC and Europe PMC list no notices (checked with
src/known.py), PubMed (PMID 32669282) has no CommentsCorrections entry, and the PMC page shows no correction notice. Web searches for a correction or erratum found none. - PubPeer. Web searches found no thread. The PubPeer API requires a human-verification step and could not be queried directly.
- Not checked. The bmj.com corrections and rapid-responses panels (HTTP 403), and the Europe PMC search API, which returned 503 on four attempts; the Europe PMC article XML and supplementary-file endpoints did respond. Retraction Watch was covered only through Crossref's updated-by field.
Suggested correction
In the abstract (Results) and in the Results section, "All cause mortality and cardiovascular disease associated with prediabetes in the general population", paragraph 3, and in supplementary file 15 (men and women, all cause mortality):
- Replace "7.36 (95% confidence interval 9.59 to 12.51)" with "9.59 (95% confidence interval 7.36 to 12.51)". This is the value implied by the paper's method, comparator rate 73.6 × (RR − 1) with RR 1.13 (1.10 to 1.17), which gives 9.57 from the rounded RR. The authors can confirm the exact second decimal from the unrounded RR.
In the abstract only:
- Replace "8.54 (32.43 to 61.45)" with "8.54 (−32.43 to 61.45)", matching the Results and supplementary file 21.
Recomputation script (python recompute.py.txt)
"""Recompute the absolute risk differences (ARDs) in PMC7362233
(BMJ 2020;370:m2297, doi:10.1136/bmj.m2297).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Method, as stated in the paper (Methods, Statistical analysis):
"We calculated the absolute risk difference ... by multiplying the assumed comparator
risk of each outcome of interest by the estimated RR-1 ... We used the median risks of
outcomes in individuals with normoglycaemia across studies as the assumed comparator
risks. Absolute risk differences were expressed as events per 10 000 person years."
So ARD = C x (RR - 1), and its 95% CI limits are C x (RR_lower - 1) and C x (RR_upper - 1),
with the same comparator risk C for the estimate and both limits.
Inputs, all from the paper:
RR (95% CI): Abstract / Results text.
Printed ARD (95% CI): Abstract / Results text (= supplementary files 15 and 21).
C, "Event rate in normoglycaemia" per 10 000 person years: supplementary file 15
(general population, p. 54 of caix056552.ww.pdf) and file 21 (ASCVD patients, p. 61).
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# label, RR, RR_lo, RR_hi, printed ARD, ARD_lo, ARD_hi, comparator rate C (supplement)
ROWS = [
("General population: all cause mortality", 1.13, 1.10, 1.17, 7.36, 9.59, 12.51, 73.6),
("General population: composite CVD", 1.15, 1.11, 1.18, 8.75, 6.41, 10.49, 58.3),
("General population: CHD", 1.16, 1.11, 1.21, 6.59, 4.53, 8.65, 41.2),
("General population: stroke", 1.14, 1.08, 1.20, 3.68, 2.10, 5.26, 26.3),
("ASCVD: all cause mortality", 1.36, 1.21, 1.54, 66.19, 38.60, 99.25, 183.8),
("ASCVD: composite CVD", 1.37, 1.23, 1.53, 189.77, 117.97, 271.84, 512.9),
("ASCVD: CHD", 1.15, 1.02, 1.29, 40.62, 5.42, 78.53, 270.8),
("ASCVD: stroke", 1.05, 0.81, 1.36, 8.54, -32.43, 61.45, 170.7),
]
HALF = 0.005 # RRs are printed to 2 decimals, so each lies within +/- 0.005 of its true value
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
def f2(x):
return f"{x:.2f}"
print("1. Is each printed estimate inside its own printed 95% CI?")
for lab, r, rl, rh, a, al, ah, c in ROWS:
print(f" {lab:42s} {f2(a):>7s} ({f2(al)} to {f2(ah)}) inside = {al <= a <= ah}")
print("\n2. ARD recomputed as C x (RR - 1), with C from the supplement")
print(f" {'quantity':42s} {'C':>6s} {'printed':>24s} {'recomputed':>24s}")
recomputed = {}
for lab, r, rl, rh, a, al, ah, c in ROWS:
p, lo, hi = c * (r - 1), c * (rl - 1), c * (rh - 1)
recomputed[lab] = (p, lo, hi)
print(f" {lab:42s} {c:6.1f} {f2(a):>7s} ({f2(al):>6s} to {f2(ah):>6s})"
f" {f2(p):>7s} ({f2(lo):>6s} to {f2(hi):>6s})")
print("\n3. Comparator risk implied by each printed number, C = ARD / (RR - 1),")
print(" as a ratio to the supplement's C; range allowed by RR rounding (+/- 0.005) in []")
ratios = []
for lab, r, rl, rh, a, al, ah, c in ROWS:
out = []
for name, val, rr in (("estimate", a, r), ("lower", al, rl), ("upper", ah, rh)):
b = val / (rr - 1)
bmin, bmax = sorted([val / (rr + HALF - 1), val / (rr - HALF - 1)])
ok = bmin <= c <= bmax
ratios.append((lab, name, b / c, ok))
out.append(f"{name} {b:6.1f} ({b / c:.3f}) [{bmin:.1f}-{bmax:.1f}] {'fits' if ok else 'DOES NOT FIT'}")
print(f" {lab}\n " + "\n ".join(out))
n_fit = sum(ok for *_, ok in ratios)
print(f" printed numbers consistent with the supplement's C: {n_fit} of {len(ratios)}")
fit_r = [x for *_, x, ok in ratios if ok]
print(f" ratio range of the fitting numbers: {min(fit_r):.3f} to {max(fit_r):.3f}")
print("\n4. General-population all cause mortality, explanations tested")
lab, r, rl, rh, a, al, ah, c = ROWS[0]
print(f" (a) printed 7.36 is right, CI wrong: C = 7.36/0.13 = {a / (r - 1):.2f}; "
f"CI would be {f2(a / (r - 1) * (rl - 1))} to {f2(a / (r - 1) * (rh - 1))}; "
f"but supplement C = {c}")
print(f" (b) estimate and lower limit transposed, 9.59 (7.36 to 12.51): implied C = "
f"{9.59 / 0.13:.2f} / {7.36 / 0.10:.2f} / {12.51 / 0.17:.2f}")
print(f" recomputed with C = {c}: {f2(c * (r - 1))} ({f2(c * (rl - 1))} to {f2(c * (rh - 1))})")
print(f" unrounded RR implied by printed 9.59 with C = {c}: {1 + 9.59 / c:.4f} "
f"(within 1.13 +/- 0.005: {abs(1 + 9.59 / c - 1.13) <= HALF})")
print(f" (c) midpoint of printed CI: {(al + ah) / 2:.2f}; implied C = {(al + ah) / 2 / 0.13:.1f}")
print(f" printed 7.36 vs recomputed {f2(c * 0.13)}: printed is {100 * (1 - a / (c * 0.13)):.1f}% lower")
print(f" printed 7.36 is {f2(al - a)} below its own printed lower limit")
print("\n5. ASCVD stroke, abstract vs Results")
lab, r, rl, rh, a, al, ah, c = ROWS[7]
print(f" recomputed lower limit {c} x ({rl} - 1) = {f2(c * (rl - 1))}; "
f"abstract prints 32.43 (no minus), Results print -32.43")
print(f" abstract triple 8.54 (32.43 to 61.45): estimate inside = {32.43 <= 8.54 <= 61.45}")
# ------------------------------------------------------------------ figure 1
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(colors=INK2, labelsize=10, length=0)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
gp = ROWS[:4]
short = ["All cause mortality", "Composite CVD", "Coronary heart disease", "Stroke"]
fig, ax = plt.subplots(figsize=(7.2, 4.7), facecolor=BG)
style(ax)
ax.axvline(0, color=INK2, lw=0.8)
yt, yl = [], []
for i, (row, name) in enumerate(zip(gp, short)):
lab, r, rl, rh, a, al, ah, c = row
p, lo, hi = recomputed[lab]
y0 = -i * 1.0
for y, (pt, l, h), col in ((y0 + 0.17, (a, al, ah), ORANGE), (y0 - 0.17, (p, lo, hi), BLUE)):
ax.plot([l, h], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([l, l], [y - 0.07, y + 0.07], color=col, lw=2, zorder=2)
ax.plot([h, h], [y - 0.07, y + 0.07], color=col, lw=2, zorder=2)
ax.plot(pt, y, "o", ms=8, color=col, mec=BG, mew=1.5, zorder=3)
ax.text(14.3, y, f"{pt:.2f} ({l:.2f} to {h:.2f})", va="center", ha="left",
fontsize=10, color=INK)
yt.append(y0)
yl.append(name)
# annotation on the mortality row
ax.annotate("printed estimate lies\noutside its own interval",
xy=(7.36, 0.17), xytext=(1.0, 0.62), fontsize=10, color=INK2,
ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8, shrinkA=2, shrinkB=6))
ax.set_yticks(yt)
ax.set_yticklabels(yl, fontsize=10, color=INK)
ax.set_xlim(-0.5, 14)
ax.set_ylim(-3.55, 0.95)
ax.set_xticks(range(0, 15, 2))
ax.set_xlabel("Absolute risk difference, prediabetes vs normoglycaemia\n"
"(events per 10 000 person years; 0 = no difference)", fontsize=10, color=INK2)
ax.text(14.3, 0.78, "estimate (95% CI)", fontsize=10, color=INK2, ha="left", va="center")
handles = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, mec=BG,
label="Printed in abstract and Results"),
Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, mec=BG,
label="Recomputed: supplement comparator rate × (RR − 1)")]
fig.legend(handles=handles, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=1,
frameon=False, fontsize=10, labelcolor=INK, handlelength=2.2)
fig.text(0.99, 0.965, "General population", fontsize=11, color=INK, ha="right", va="top")
fig.subplots_adjust(left=0.27, right=0.70, top=0.84, bottom=0.15)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ------------------------------------------------------------------ figure 2
labels2 = ["Gen. pop.: all cause mortality", "Gen. pop.: composite CVD", "Gen. pop.: CHD",
"Gen. pop.: stroke", "ASCVD: all cause mortality", "ASCVD: composite CVD",
"ASCVD: CHD", "ASCVD: stroke"]
markers = {"estimate": "o", "lower": "<", "upper": ">"}
offs = {"estimate": 0.0, "lower": 0.22, "upper": -0.22}
fig, ax = plt.subplots(figsize=(7.2, 4.9), facecolor=BG)
style(ax)
ax.axvline(1.0, color=INK2, lw=0.8)
for k, (lab, name, ratio, ok) in enumerate(ratios):
i = k // 3
y = -i + offs[name]
col = ORANGE if not ok else INK2
ax.plot(ratio, y, markers[name], ms=8, color=col, mec=BG, mew=1.0, zorder=3)
if not ok:
val = {"estimate": 7.36, "lower": 9.59}[name]
if ratio > 1:
ax.text(ratio + 0.03, y, f"lower limit {val:.2f}\nimplies {ratio:.2f}×",
fontsize=10, color=INK, va="center", ha="left")
else:
ax.text(0.52, y - 0.55, f"estimate {val:.2f}\nimplies {ratio:.2f}×",
fontsize=10, color=INK, va="center", ha="left")
ax.set_yticks([-i for i in range(8)])
ax.set_yticklabels(labels2, fontsize=10, color=INK)
ax.set_xlim(0.5, 1.8)
ax.set_xticks(np.arange(0.6, 1.81, 0.2))
ax.set_ylim(-7.5, 0.5)
ax.set_xlabel("Comparator risk implied by each printed number, C = ARD / (RR − 1),\n"
"relative to the rate given in the supplement (1 = consistent)",
fontsize=10, color=INK2)
handles = [Line2D([], [], ls="", marker="o", ms=8, color=INK2, label="estimate"),
Line2D([], [], ls="", marker="<", ms=8, color=INK2, label="lower limit"),
Line2D([], [], ls="", marker=">", ms=8, color=INK2, label="upper limit"),
Line2D([], [], ls="", marker="s", ms=8, color=INK2,
label="fits the stated rate (within RR rounding)"),
Line2D([], [], ls="", marker="s", ms=8, color=ORANGE,
label="printed value that does not fit")]
fig.legend(handles=handles[:3], loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=3,
frameon=False, fontsize=10, labelcolor=INK, columnspacing=1.2)
fig.legend(handles=handles[3:], loc="upper left", bbox_to_anchor=(0.01, 0.945), ncol=2,
frameon=False, fontsize=10, labelcolor=INK, columnspacing=1.2)
fig.subplots_adjust(left=0.33, right=0.97, top=0.84, bottom=0.15)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract reports a 90% education effect at p<0.01; Table 5 gives OR 1.06, p=0.50
BMJ Glob Health (2025) · doi:10.1136/bmjgh-2025-020195 · PMC12699593
Summary
This cross-sectional survey of 1659 households in two Nigerian states examines how social capital relates to corruption when people use public health facilities. The Abstract states that higher education "was associated with a 90% increased chance of engaging in corrupt practices (p<0.01)". The paper's only education estimate is in Table 5, where years of formal education has a pooled odds ratio of 1.06 (p=0.50), and the Findings text says education "was not significantly associated with corruption in the pooled model". So the Abstract overstates the size of the effect (about 6% higher odds per year, not 90%) and reports significance where the paper's own table shows none. A second, smaller inconsistency is that the Northwestern-state paragraph quotes two odds ratios that appear in Table 5 only in the Southeastern column. The paper's main conclusion about social capital rests on other estimates and is not affected.
What the paper reports
Abstract, Results:
Higher education was associated with a 90% increased chance of engaging in corrupt practices (p<0.01).
Table 5 ("Influence of social capital on corrupt behaviour during care in public facilities"), columns pooled / Southeastern state / Northwestern state, each headed OR (P>|t|), with the footnote "*Significant at 5%.":
Years spent on formal education | 1.06 (0.50) | 1.05 (0.02)* | 1.01 (0.90)
Findings, paragraph on the pooled model:
Years of formal education were not significantly associated with corruption in the pooled model.
Findings, Southeastern-state paragraph:
In the Southeastern state, none of the social capital, wealth, gender or age variables were significantly associated with corruption. However, education emerged as a predictor: each additional year of formal schooling was associated with a 5% increase in the odds of engaging in corrupt practices (OR=1.05, p<0.05).
Findings, Northwestern-state paragraph:
Poor households, compared with the extremely poor, were more likely to engage in corrupt practices (OR=1.58, p<0.05).
Furthermore, years of formal education were associated with a 5% increase in the odds of corruption (OR=1.05, p<0.05).
Table 5, wealth row cited in that paragraph:
Poor | 1.21 (0.24) | 1.58 (0.04)* | 0.72 (0.18)
Discussion:
Education and wealth influenced engagement in corrupt practices. Higher levels of education and wealth enable individuals to leverage social networks more effectively, increasing their opportunities to engage in corrupt practices.
Methods, Data analysis, which identifies education as a control variable entered as a continuous measure:
The independent variables included in the regression models were selected based on the previous literature, showing an association with engaging in corruption while receiving care, while gender, age and years of completed education of the household member were added as control variables.
What the paper's own numbers show
Method. In a logistic regression, an odds ratio r means the odds change by 100 × (r − 1)% per unit of the predictor; here the unit is one year of formal education. A "90% increased chance" therefore corresponds to an odds ratio of about 1.90 for whatever contrast the Abstract has in mind. I compared this with every education estimate in the paper, checked whether any Table 5 cell (or its reciprocal) is near 1.90, and checked how many years of education the per-year odds ratio would need to compound to 1.90. To show the uncertainty, I back-calculated approximate 95% Wald intervals from each printed OR and p value (SE of ln OR = |ln OR| / z for p/2), including the full rounding range of the printed values. I also matched every OR quoted in the Findings text to the Table 5 column it appears in. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Education effect | Abstract, Results | "90% increased chance" (p<0.01) | implies OR ≈ 1.90 at p<0.01; no estimate in the paper matches |
| Years of education, pooled model | Table 5 | 1.06 (0.50) | +6% odds per year, not significant; approx. 95% CI 0.89–1.26 (0.88–1.28 over the rounding range or with a t reference) |
| Years of education, Southeastern | Table 5 | 1.05 (0.02)* | +5% per year; significant at 5%, not at 1%; approx. 95% CI 1.01–1.09 |
| Years of education, Northwestern | Table 5 | 1.01 (0.90) | +1% per year, not significant; approx. 95% CI 0.86–1.18 |
| Pooled education, text | Findings, pooled paragraph | "not significantly associated" | agrees with Table 5 (p=0.50); contradicts the Abstract |
| Is OR 1.90 inside any of the three intervals? | derived | n/a | no |
| Education cells with p<0.01 | Table 5 | n/a | none (smallest is p=0.02) |
| Table 5 ORs, or reciprocals, between 1.85 and 1.95 | Table 5, all 24 cells | n/a | none |
| Years needed for OR 1.06 per year to reach 1.90 | derived | n/a | 11.0 (13.2 at 1.05; 64.5 at 1.01); 1.06^10 = 1.79. A rescaled contrast keeps p=0.50 |
| Northwestern education, text | Findings, Northwestern paragraph | OR=1.05, p<0.05 | Table 5 Northwestern is 1.01 (0.90); 1.05 (0.02) is the Southeastern value |
| Northwestern "poor" quintile, text | Findings, Northwestern paragraph | OR=1.58, p<0.05 | Table 5 Northwestern is 0.72 (0.18); 1.58 (0.04) is the Southeastern value |
| Southeastern wealth, text | Findings, Southeastern paragraph | no wealth variable significant | Table 5 Southeastern "poor" is 1.58 (0.04)*, the only Southeastern cell besides education with p<0.05 |
| Other ORs in the Findings text | Findings, pooled and Northwestern paragraphs | 2.24, 1.66, 0.45, 1.01; 2.04, 0.35, 1.02 | each matches its own Table 5 column |
| Share with any schooling | Abstract; Table 2 | "over 90% had formal education" | 100 × (1 − 144/1659) = 91.3%: consistent, and descriptive only |
The pooled estimate is a small, non-significant positive association, and the paper's own text describes it that way. The Abstract's 90% effect at p<0.01 cannot be obtained from any reported model.
Figure
Figure 1. Left (orange, the value reported in the Abstract): the Abstract's sentence on education, which corresponds to an odds ratio of about 1.90 at p<0.01. Right: the education row of Table 5 for the three models, and the Findings sentence on the pooled model. Below (blue, recomputed from Table 5): the pooled OR of 1.06 is a 6% change in odds per year with p=0.50. No education estimate reaches p<0.01, no Table 5 odds ratio lies between 1.85 and 1.95, and even a contrast of 11 years, which would compound 1.06 to 1.90, would keep p=0.50.
Figure 2. The Abstract's claim, read as an odds ratio of 1.90 (orange, reported), and the three Table 5 odds ratios per year of education (blue), each with an approximate 95% interval back-calculated from its printed OR and p value. The axis is logarithmic; the thin vertical line marks OR = 1 (no association). The value 1.90 lies far outside all three intervals. The intervals are approximate because the printed p values are rounded to two decimals; over the full rounding range the pooled upper limit stays between 1.23 and 1.28.
Why it matters
The Abstract is the part of the paper most readers see, and it presents education as a strong, highly significant predictor of corruption: a 90% increase at p<0.01. The paper's results show a pooled odds ratio of 1.06 per year that is not significant (p=0.50), a small significant effect in the Southeastern state only (1.05 per year, p=0.02), and no effect in the Northwestern state (1.01, p=0.90). A reader relying on the Abstract would draw the wrong conclusion about both the size and the strength of the evidence for an education effect. The Discussion's statement that "Education and wealth influenced engagement in corrupt practices" is weaker than it reads for the same reason: education was significant in one of the three models, at the 5% level.
The text-to-table mismatches in the state paragraphs compound this. The Northwestern paragraph credits that state with a significant education effect (OR=1.05) and a significant "poor" quintile effect (OR=1.58), but Table 5 shows both of these values in the Southeastern column; the Northwestern column has 1.01 (p=0.90) and 0.72 (p=0.18). Meanwhile the Southeastern paragraph says no wealth variable was significant there, although Table 5 marks the Southeastern "poor" estimate as significant. Either the paragraphs or the columns are misattributed for these two rows, and a reader cannot tell from the paper which is correct.
The scope is limited. Education was entered as a control variable, and the paper's hypothesis and main conclusion concern social capital, which rest on other estimates (pooled OR 2.24, p<0.01; Northwestern OR 2.04, p<0.01) that the text reports in agreement with Table 5. The other Abstract findings were not part of this check.
Explanations considered
- A different model or population. This does not account for the numbers. Education appears only in Table 5, as a continuous variable (years of formal education), in three models: pooled, Southeastern and Northwestern. None gives an OR near 1.90 and none gives p<0.01. No table reports education as categories (for example tertiary vs none).
- A multi-year contrast. This does not account for the numbers. At 1.06 per year, about 11 years would compound to 1.90 (roughly no schooling vs completed secondary school), but the paper reports no such contrast, and rescaling the coefficient does not change its p value of 0.50, so "p<0.01" would remain unsupported.
- "Chance" meaning probability rather than odds. This does not help. A 6% increase in odds per year implies a still smaller relative increase in probability, and no reading of the table gives a 90% increase in either.
- A descriptive percentage. This is excluded. The only figures near 90% are "over 90% had formal education" (91.3%, from Table 2) and 89.19% (Northwestern professional-association members who engaged in corruption, Table 4); neither is an effect of education. Table 2's "Engaged in corruption (%)" column for education levels sums to 100.1, so it is a breakdown of those who engaged, not a rate per education level, and none of its values is 90.
- A reciprocal or a "reduction" phrasing. This is excluded. No Table 5 odds ratio or reciprocal lies between 1.85 and 1.95.
- A conversion or typesetting error in the archived text. This is excluded. The PMC text, the publisher-supplied JATS XML in Europe PMC and the PubMed abstract (PMID 41365626) all carry the same Abstract sentence, and in the XML the three Table 5 education values are separate cells.
- An analysis in a supplement. This is excluded. The two supplementary files hold the principal component analysis behind the social capital index and the questionnaire; neither contains regression output (see Status).
- Misplaced state columns in Table 5. This cannot explain the Abstract and does not fully explain the state paragraphs. Whatever the column order, the pooled education estimate is 1.06 (p=0.50). For the state paragraphs, the social capital, gender and age values that the Northwestern text quotes already match the Northwestern column as printed, and both state paragraphs quote the same education value (OR=1.05, p<0.05), while Table 5 has 1.05 in one state and 1.01 in the other. Swapping the state columns for the education row would therefore fix one paragraph and break the other. At least one paragraph or one cell is misattributed, and the paper does not show which.
Status
- Version of record. The publisher's pages (doi.org redirect to gh.bmj.com, the article HTML and the PDF) returned HTTP 403 to automated access. I used the publisher-deposited JATS full text in Europe PMC (PMC12699593/fullTextXML; BMJ Glob Health vol 10, issue 12, e020195), which has the same text as the PMC archive, and the PubMed abstract, which has the same Abstract sentence. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. Europe PMC holds two supplementary files: bmjgh-10-12-s001.docx (principal component analysis results for the social capital index: eigenvalues, loadings, uniqueness) and bmjgh-10-12-s002.docx (the study questionnaire). Neither contains regression results, education contrasts or alternative models. No data or code are deposited, so the correct estimates cannot be recomputed from published material.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, PMC and Europe PMC list no notices, and the PubMed record has no linked correction. A web search found no correction or published comment. The PubPeer API refused automated access (HTTP 403), so PubPeer could not be checked directly.
- Sources. PMC article PMC12699593; Europe PMC full-text XML and supplementary files for PMC12699593; PubMed PMID 41365626; Crossref record for 10.1136/bmjgh-2025-020195.
Suggested correction
- Revise the Abstract's education sentence to match Table 5, for example: "Years of formal education were not significantly associated with corruption in the pooled model (OR 1.06 per year, p=0.50); in the Southeastern state each additional year was associated with 5% higher odds (OR 1.05, p=0.02)." If the 90% figure comes from an analysis not shown in the paper, report that analysis, its contrast and its p value.
- Reconcile the Northwestern-state paragraph with Table 5. Its education (OR=1.05) and "poor" quintile (OR=1.58) values appear in Table 5 only in the Southeastern column. Either correct the paragraph or, if the Table 5 columns are misplaced for these rows, correct the table, and then align the Southeastern paragraph's statement that no wealth variable was significant.
- Temper the Discussion's statement that education "influenced engagement in corrupt practices" to reflect that education was significant at the 5% level in one state model only.
- Optionally, report confidence intervals alongside the odds ratios in Table 5 and print p values below 0.005 as "<0.01" or "<0.001" rather than "0.00".
Recomputation script (python recompute.py.txt)
"""Recompute the education result of PMC12699593
(BMJ Glob Health 2025, doi:10.1136/bmjgh-2025-020195, social capital and corruption in Nigerian health facilities).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract, Results: "Higher education was associated with a 90% increased chance of engaging in
corrupt practices (p<0.01)."
Table 5 (logistic regression, OR (P>|t|); pooled / Southeastern state / Northwestern state),
row "Years spent on formal education": 1.06 (0.50) | 1.05 (0.02)* | 1.01 (0.90);
footnote "*Significant at 5%."
Findings text: the ORs quoted for the pooled, Southeastern and Northwestern models (see TEXT below).
Table 2: education level frequencies and the "Engaged in corruption (%)" column.
"""
import math
import os
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = stats.norm.isf(0.025)
# ---------------------------------------------------------------- inputs (from the paper)
COLS = ["pooled", "Southeastern", "Northwestern"]
T5 = { # Table 5: variable -> [(OR, p) pooled, Southeastern, Northwestern]
"Used social capital to access care": [(2.24, 0.00), (1.16, 0.63), (2.04, 0.00)],
"Poor": [(1.21, 0.24), (1.58, 0.04), (0.72, 0.18)],
"Average": [(1.18, 0.34), (1.24, 0.35), (0.89, 0.66)],
"Rich": [(1.26, 0.19), (1.18, 0.50), (0.97, 0.92)],
"Extreme rich": [(1.66, 0.00), (1.30, 0.33), (1.22, 0.42)],
"Gender (female)": [(0.45, 0.00), (0.75, 0.07), (0.35, 0.00)],
"Age": [(1.01, 0.00), (1.04, 0.19), (1.02, 0.00)],
"Years spent on formal education": [(1.06, 0.50), (1.05, 0.02), (1.01, 0.90)],
}
EDU = "Years spent on formal education"
ABSTRACT_PCT = 90 # "90% increased chance"
ABSTRACT_P = "<0.01"
# ORs quoted in the Findings text, by the paragraph they appear in
TEXT = {
"pooled": [("Used social capital to access care", 2.24, "p<0.01"), ("Extreme rich", 1.66, "p<0.01"),
("Gender (female)", 0.45, "p<0.01"), ("Age", 1.01, "p<0.01"),
(EDU, None, "not significant")],
"Southeastern": [(EDU, 1.05, "p<0.05")],
"Northwestern": [("Used social capital to access care", 2.04, "p<0.01"), ("Poor", 1.58, "p<0.05"),
("Gender (female)", 0.35, "p<0.01"), ("Age", 1.02, "p<0.01"), (EDU, 1.05, "p<0.05")],
}
T2_EDU = { # Table 2: level -> (frequency, "Engaged in corruption (%)")
"I didn't go to school": (144, 7.7), "Primary": (146, 9.8), "Junior secondary": (89, 5.1),
"Senior secondary": (702, 39.5), "University": (242, 15.2), "Polytechnics": (130, 10),
"Educational colleges": (162, 11.3), "Others": (44, 1.5),
}
N_TOTAL = 1659
R = {}
print("=== 1. The abstract's education claim as an odds ratio ===")
or_abs = 1 + ABSTRACT_PCT / 100
R["or_abs"] = or_abs
print(f" '{ABSTRACT_PCT}% increased chance' read as an odds ratio: OR = {or_abs:.2f}; claimed p {ABSTRACT_P}")
print("\n=== 2. Table 5 education row: change in odds, significance, implied 95% CI ===")
print(" (Wald CI back-calculated from the rounded OR and p: SE = |ln OR| / z(p/2); approximate)")
for col, (o, p) in zip(COLS, T5[EDU]):
b = math.log(o)
z = stats.norm.isf(p / 2)
se = abs(b) / z
lo, hi = math.exp(b - Z975 * se), math.exp(b + Z975 * se)
R[col] = dict(o=o, p=p, se=se, lo=lo, hi=hi)
print(f" {col:13s} OR {o:.2f} -> {100 * (o - 1):+.0f}% odds per year; p = {p}; p < 0.01? {p < 0.01}; "
f"p < 0.05? {p < 0.05}; SE(ln OR) = {se:.3f}; 95% CI {lo:.2f}-{hi:.2f}; "
f"1.90 inside CI? {lo <= or_abs <= hi}")
print("\n=== 3. Rounding range for the pooled CI (OR 1.055-1.065, p 0.495-0.505) ===")
his, los = [], []
for o in (1.055, 1.065):
for p in (0.495, 0.505):
b = math.log(o); se = b / stats.norm.isf(p / 2)
los.append(math.exp(b - Z975 * se)); his.append(math.exp(b + Z975 * se))
R["pooled_hi_range"] = (min(his), max(his))
print(f" pooled 95% CI lower {min(los):.2f}-{max(los):.2f}, upper {min(his):.2f}-{max(his):.2f}")
print(f" t instead of z (Table 5 heads its p column P>|t|; illustrative 9 df): ", end="")
b = math.log(1.06); se_t = b / stats.t.isf(0.25, 9)
print(f"SE {se_t:.3f}, 95% CI {math.exp(b - stats.t.isf(0.025, 9) * se_t):.2f}-"
f"{math.exp(b + stats.t.isf(0.025, 9) * se_t):.2f}")
print("\n=== 4. Could a multi-year contrast give OR 1.90? ===")
for o in (1.06, 1.05, 1.01):
print(f" years for {o}^n = 1.90: {math.log(or_abs) / math.log(o):.1f}")
print(f" 1.06^10 = {1.06 ** 10:.2f}; 1.06^12 = {1.06 ** 12:.2f} (a rescaled contrast keeps p = 0.50)")
print("\n=== 5. Is any Table 5 OR (or its reciprocal) near 1.90? ===")
near = [(k, COLS[i], o) for k, v in T5.items() for i, (o, _) in enumerate(v) if 1.85 <= o <= 1.95]
near_r = [(k, COLS[i], round(1 / o, 2)) for k, v in T5.items() for i, (o, _) in enumerate(v)
if 1.85 <= 1 / o <= 1.95]
print(f" ORs within 1.85-1.95: {near}; reciprocals within 1.85-1.95: {near_r}")
p01 = [(k, COLS[i], o, p) for k, v in T5.items() for i, (o, p) in enumerate(v) if p < 0.01]
print(f" cells with printed p < 0.01: {p01}")
print("\n=== 6. Which Table 5 column does each OR quoted in the Findings text match? ===")
for para, items in TEXT.items():
for var, o, ptxt in items:
cells = dict(zip(COLS, T5[var]))
own = cells[para]
if o is None:
ok = own[1] >= 0.05
print(f" [{para:12s}] {var}: '{ptxt}' -> own column {own}: {'consistent' if ok else 'MISMATCH'}")
continue
match = [c for c in COLS if abs(cells[c][0] - o) < 1e-9]
flag = "consistent" if para in match else "MISMATCH"
print(f" [{para:12s}] {var}: text OR {o} ({ptxt}); own column {own}; value found in {match} -> {flag}")
sig_se = [(k, o, p) for k, v in T5.items() for (o, p) in [v[1]] if p < 0.05 and k != EDU]
print(f" Southeastern text: 'none of the social capital, wealth, gender or age variables were significant';"
f" Table 5 Southeastern cells with p < 0.05 besides education: {sig_se}")
print("\n=== 7. Table 2 checks (descriptive figures, ruled out as a source of '90%') ===")
nsum = sum(n for n, _ in T2_EDU.values())
esum = sum(e for _, e in T2_EDU.values())
R["pct_school"] = 100 * (1 - T2_EDU["I didn't go to school"][0] / N_TOTAL)
print(f" education frequencies sum to {nsum} of {N_TOTAL}; % with any schooling = {R['pct_school']:.1f}"
" (abstract: 'over 90% had formal education')")
print(f" 'Engaged in corruption (%)' column sums to {esum:.1f}: a breakdown of those engaged, not a rate per level")
# ---------------------------------------------------------------- figures
BG, TXT, TXT2, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": TXT2, "ytick.color": TXT, "axes.edgecolor": TXT2, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: abstract vs Results/Table 5 text panels
fig = plt.figure(figsize=(7.2, 4.3))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 60); ax.axis("off")
def panel(x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=edge, lw=1.5))
ax.text(x + 1.5, y + h - 2.2, title, fontsize=11, fontweight="bold", va="top", color=TXT)
panel(2, 21, 38, 34, "Abstract, Results", ORANGE)
ax.text(3.5, 47, "“Higher education was\nassociated with a 90%\nincreased chance of engaging\n"
"in corrupt practices\n(p<0.01).”", fontsize=10.5, va="top", color=TXT, linespacing=1.35)
ax.text(3.5, 25.5, f"Reported: +90%, p<0.01\n(OR ≈ {or_abs:.2f})", fontsize=10.5, va="bottom",
color=ORANGE, fontweight="bold")
panel(55, 21, 43, 34, "Table 5 and Findings text", TXT2)
ax.text(56.5, 47.2, "Years spent on formal education,\nOR (p):", fontsize=10.5, va="top", color=TXT)
rows = [("Pooled", R["pooled"]), ("Southeastern", R["Southeastern"]), ("Northwestern", R["Northwestern"])]
for i, (lab, d) in enumerate(rows):
y = 40.0 - 3.6 * i
star = "*" if d["p"] < 0.05 else ""
ax.text(58, y, lab, fontsize=10.5, va="top", color=TXT)
ax.text(77, y, f"{d['o']:.2f} ({d['p']:.2f}){star}", fontsize=10.5, va="top", color=TXT,
fontweight="bold" if lab == "Pooled" else "normal")
ax.text(56.5, 28.6, "“Years of formal education were\nnot significantly associated with\n"
"corruption in the pooled model.”", fontsize=10, va="top", color=TXT2, style="italic",
linespacing=1.3)
arr = FancyArrowPatch((41.2, 38), (54, 38), arrowstyle="<->", mutation_scale=16, lw=2, color=TXT)
ax.add_patch(arr)
ax.text(47.6, 40.5, "conflict", ha="center", fontsize=10, color=TXT, fontweight="bold")
ax.text(50, 15.5, "Recomputed from Table 5 (pooled): OR 1.06 = +6% odds per year, p = 0.50",
ha="center", fontsize=10.5, color=BLUE, fontweight="bold")
ax.text(50, 11.0, f"No education estimate in any model reaches p<0.01; no Table 5 OR lies between 1.85 and 1.95.\n"
f"OR 1.06 per year would need {math.log(or_abs) / math.log(1.06):.0f} years to compound to 1.90, "
"and p would stay 0.50.", ha="center", va="top", fontsize=10, color=TXT2, linespacing=1.35)
save(fig, "fig1")
# Figure 2: dot-and-interval on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.3))
fig.subplots_adjust(left=0.30, right=0.97, top=0.86, bottom=0.2)
labels = ["Abstract claim\n(+90%, p<0.01)", "Table 5, pooled", "Table 5, Southeastern", "Table 5, Northwestern"]
ys = [3, 2, 1, 0]
ax.axvline(1, color=TXT2, lw=0.8, zorder=1)
ax.plot([or_abs], [3], "o", ms=9, color=ORANGE, zorder=3)
ax.text(or_abs, 3.32, f"OR ≈ {or_abs:.2f}", ha="center", fontsize=10, color=ORANGE)
for y, col in zip([2, 1, 0], COLS):
d = R[col]
ax.plot([d["lo"], d["hi"]], [y, y], color=BLUE, lw=2, zorder=2, solid_capstyle="butt")
ax.plot([d["o"]], [y], "o", ms=9, color=BLUE, zorder=3)
ax.text(d["hi"] * 1.025, y, f"{d['o']:.2f} ({d['lo']:.2f}–{d['hi']:.2f}), p = {d['p']:.2f}",
va="center", fontsize=10, color=TXT)
ax.set_xscale("log")
ax.set_xlim(0.8, 2.6)
ticks = [0.8, 1, 1.25, 1.5, 2, 2.5]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks]); ax.minorticks_off()
ax.set_yticks(ys); ax.set_yticklabels(labels)
ax.set_ylim(-0.6, 3.7)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio for engaging in corruption (log scale; 1 = no association)")
fig.text(0.02, 0.975, "Orange: abstract claim read as an odds ratio (reported). Blue: Table 5 OR per year of\n"
"education, with 95% CI back-calculated from its printed p value (recomputed).",
fontsize=10, color=TXT2, va="top")
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract's subgroup gains in contraceptive use disagree with the paper's own Table 2
BMJ Glob Health (2026) · doi:10.1136/bmjgh-2024-018775 · PMC13250227
Summary
The abstract reports 2008–2019 gains in modern contraceptive prevalence (mCPR) for four subgroups: +12.6 and +10.0 percentage points for married women aged 15–24 and 25–49, and +19.5 and +16.9 for unmarried, sexually active women of the same ages. It states that among married women the gains "were larger in those aged 15–24 years". The paper's Table 2, the main text's only table of mCPR by age and marital status, gives +13.5 and +14.2 for the married groups and +31.5 and +21.0 for the unmarried groups. On those figures, older married women gained slightly more, the reverse of the abstract's ranking. The headline gain for all women (+15.7) and the married total both match Table 2. The paper's conclusions about drivers of the increase do not depend on these subgroup figures, but they are the only subgroup results in the abstract.
What the paper reports
All quotations are verbatim from the article as archived in PMC.
Abstract, Results:
Between 2008 and 2019, modern contraceptive prevalence among women aged 15–49 rose from 8.2% to 23.9% (+15.7 percentage points). Among married women, gains were larger in those aged 15–24 years (+12.6) than in women aged 25–49 years (+10.0). Unmarried women maintained substantially higher use, with gains of +19.5 and +16.9 points, respectively.
Table 2, "Modern contraceptive prevalence rate (mCPR, %) by age group and marital status, Sierra Leone, Demographic Health Survey (DHS) 2008–2019", excerpt (columns: population group | 2008 mCPR | N2008 | 2013 mCPR | N2013 | 2019 mCPR | N2019):
All women (15–49 years) | 8.2 | 7374 | 20.9 | 16 658 | 23.9 | 15 574
Married/in-union (15–49 years) | 6.7 | 5525 | 15.6 | 10 903 | 20.9 | 9715
Married/in-union, age 15–24 | 3.7 | 1171 | 11.8 | 2299 | 17.2 | 1842
Married/in-union, age 25–49 | 7.6 | 4353 | 16.6 | 8603 | 21.8 | 7873
Unmarried/sexually active, age 15–24† | 26.6 | 343 | 57.5 | 1472 | 58.1 | 1309
Unmarried/sexually active, age 25–49† | 21.1 | 208 | 53.3 | 586 | 42.1 | 678
Table 2 footer, excerpt:
All prevalence rates and denominators are from DHS 2008 (Table 5.4), DHS 2013 (Table 7.2) and DHS 2019 (Table 7.3), or computed as described in Methods.
Calculated values for age subgroups represent weighted averages of individual 5-year cohorts; see Methods for details.
Results, "Modern contraceptive prevalence and demand satisfied", first paragraph:
Between 2008 and 2019, Sierra Leone’s mCPR increased, from 8.2% to 23.9% among all women and from 6.7% to 20.9% among married or in-union women (DHS table 2).
Gains were observed across both married and unmarried women, with mCPR among sexually active unmarried women aged 15–24 reaching 57.5% by 2013 and remaining high in 2019 (58.1%).
Table 2 presents mCPR by marital status and age, with subgroup N calculated from original DHS reports or as weighted averages of DHS age and marital groups.
Methods, "Trend analysis":
These data sources were used to describe national and subnational trends in mCPR, with results presented as simple proportions over time.
Methods, "Decomposition analysis" (the sample of the separate decomposition model):
Analysis included women aged 15–49 who had sexual intercourse in the previous 12 months, were not pregnant or menopausal and did not self-report as infecund, regardless of parity.
The figures +12.6, +10.0, +19.5 and +16.9 do not appear as mCPR values or gains anywhere in the main text outside the abstract.
What the paper's own numbers show
Method. For each subgroup named in the abstract I subtracted the Table 2 mCPR for 2008 from the value for 2019, giving the absolute change in percentage points (pp). That is the measure the abstract uses for the all-women figure, where its "+15.7 percentage points" equals 23.9 − 8.2. I then checked whether other readings of Table 2 could produce the abstract values: the 2008–2013 and 2013–2019 windows, relative change, rounding of each table cell (±0.05), and every ordered pair of cells in the table. I also checked that Table 2 is internally consistent: each 15–49 row should equal the N-weighted average of its two age rows. Finally, I compared the abstract values with the only other subgroup mCPR figures in the paper, the descriptive tables of online supplemental file 4. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Gain, all women 15–49, 2008→2019 | Abstract vs Table 2 row 1 | +15.7 (8.2% → 23.9%) | 23.9 − 8.2 = +15.7 (matches) |
| Gain, married 15–49, 2008→2019 | Results text vs Table 2 | 6.7% → 20.9% (no gain stated) | 20.9 − 6.7 = +14.2 (levels match) |
| Gain, married 15–24, 2008→2019 | Abstract vs Table 2 | +12.6 | 17.2 − 3.7 = +13.5 |
| Gain, married 25–49, 2008→2019 | Abstract vs Table 2 | +10.0 | 21.8 − 7.6 = +14.2 |
| Younger minus older married gain | Abstract ("gains were larger in those aged 15–24 years") | +2.6 pp (15–24 larger) | −0.7 pp (25–49 larger) |
| Gain, unmarried sexually active 15–24 | Abstract vs Table 2 | +19.5 | 58.1 − 26.6 = +31.5 |
| Gain, unmarried sexually active 25–49 | Abstract vs Table 2 | +16.9 | 42.1 − 21.1 = +21.0 |
| Range of each gain if every cell is rounded to ±0.05 | Table 2 | — | married 13.4–13.6 and 14.1–14.3; unmarried 31.4–31.6 and 20.9–21.1 |
| Married gains, other windows (15–24; 25–49) | Table 2 | — | 2008→2013: +8.1, +9.0; 2013→2019: +5.4, +5.2 |
| Relative change 2008→2019, married (15–24; 25–49) | Table 2 | — | +364.9%, +186.8% |
| N-weighted average of age rows, married, 2008/2013/2019 | Table 2 internal check | 6.7 / 15.6 / 20.9 | 6.8 / 15.6 / 20.9 (table self-consistent; all-women and unmarried rows also agree within 0.1) |
| Gain, decomposition sample, partnered 15–24; 25–49 | Supplemental file 4, descriptive table 2 | 5.7 → 28.5; 9.5 → 30 | +22.8; +20.5 |
| Gain, decomposition sample, all women 15–49 | Supplemental file 4, descriptive table 1 | 11.7 → 36 | +24.3 |
Table 2 is internally coherent: each 15–49 row equals, within 0.1 pp, the N-weighted average of its 15–24 and 25–49 rows, and the Ns add up within one woman. The abstract's all-women gain is read off this table exactly. The four subgroup gains in the next two sentences of the abstract are not. Rounding moves each table gain by at most 0.1 pp, so it cannot account for differences of 0.9, 4.2, 12.0 and 4.1 pp. Searching all ordered pairs of Table 2 cells turns up only scattered coincidences that mix unrelated rows, years or populations. For example, 10.0 equals the married 25–49 value in 2019 minus the married 15–24 value in 2013. No pair at all gives 19.5.
Figure
Figure 1. Left panel: the four subgroup gains in mCPR, 2008→2019, and the ranking statement as printed in the abstract (orange). Right panel: the same quantities computed from the 2008 and 2019 cells of Table 2 (blue). The dashed connectors give the size of each discrepancy in percentage points. The double-headed connector marks the married-women comparison: the abstract has the 15–24 group gaining more, Table 2 the 25–49 group (+14.2 vs +13.5). The line under the panels shows the one gain in the same abstract passage that does match Table 2, for all women aged 15–49.
Figure 2. The 2008→2019 gain in mCPR, in percentage points, for each group named in the abstract. Open orange circles are the abstract values. Filled blue circles are recomputed from Table 2. Open grey diamonds are the gains in the decomposition sample of online supplemental file 4 (sexually active, not pregnant, not menopausal and fecund women; no unmarried-only group is reported there). The axis starts at zero. For all women aged 15–49, the abstract and Table 2 values coincide at 15.7 (the blue dot is drawn inside the orange ring). For every subgroup, the abstract value matches neither Table 2 nor the supplement.
Why it matters
- The abstract's only subgroup finding is not supported by the table it summarises. The comparison "gains were larger in those aged 15–24 years (+12.6) than in women aged 25–49 years (+10.0)" is the abstract's sole statement about age differences. Measured the way the abstract measures them, in percentage points, Table 2 gives +13.5 and +14.2: the older group gained slightly more. The 0.7 pp difference is small, and the paper reports no confidence intervals for these subgroups (the Table 2 footer says they cannot be computed from published DHS tables). The table therefore does not support a ranking in either direction with any stated precision. It does not support the abstract's 2.6 pp lead for the younger group.
- The size of the gains among unmarried women is understated. The abstract's +19.5 and +16.9 compare with +31.5 and +21.0 in Table 2. The gains among unmarried women came almost entirely between 2008 and 2013 (+30.9 and +32.2); between 2013 and 2019 the 15–24 group rose only 0.6 pp and the 25–49 group fell 11.2 pp, from 53.3% to 42.1%.
- In relative terms the younger married group did gain more. mCPR among married women aged 15–24 rose 4.65-fold (3.7% to 17.2%), against 2.87-fold for those aged 25–49. A reader could reach the abstract's qualitative conclusion that way, but not with the absolute figures the abstract prints.
- The main conclusions are not affected. The headline rise from 8.2% to 23.9% (+15.7 pp), the married total of 6.7% to 20.9%, the Oaxaca–Blinder decomposition (supplemental file 4), the qualitative findings and the policy conclusions do not use the four abstract subgroup values. The main text cites the overall and married totals and the unmarried 15–24 levels (57.5% in 2013, 58.1% in 2019); all of these match Table 2.
- The abstract is the most widely read part of the paper. Readers and indexing services that rely on it will carry forward subgroup gains that the paper's own table contradicts.
Explanations considered
- A different time window. Not supported. The abstract frames all figures as "Between 2008 and 2019", and the all-women figure in the same passage is the 2008→2019 difference. The 2008→2013 gains (married +8.1 and +9.0; unmarried +30.9 and +32.2) and the 2013→2019 gains (married +5.4 and +5.2; unmarried +0.6 and −11.2) do not reproduce the abstract values either.
- Relative rather than absolute change. Not supported. The abstract prints the values with a "+" and as "points", like the all-women "+15.7 percentage points". The relative changes are +364.9%, +186.8%, +118.4% and +99.5%, none of which resembles the printed values.
- Rounding. Ruled out. With each Table 2 cell uncertain by ±0.05, each recomputed gain is fixed to within ±0.1 pp, much less than the 0.9 to 12.0 pp discrepancies.
- A different population or sample, such as the decomposition sample. Not able to explain it. Supplemental file 4 reports mCPR for the decomposition sample (sexually active in the past 12 months, not pregnant, not menopausal, fecund). Its gains are +22.8 (partnered 15–24), +20.5 (partnered 25–49) and +24.3 (all women 15–49), and the supplement has no unmarried-only group. In that sample the younger partnered group does gain more, which agrees with the abstract's direction but not with its numbers. The abstract does not name this sample. Its all-women figure in the same passage comes from Table 2, not from the supplement.
- Annualised gains. Not supported. Per-year gains over the 11 years are 1.23, 1.29, 2.86 and 1.91 pp.
- A mix-up of rows or cells in Table 2. Not supported as a single consistent error. An exhaustive search over pairs of Table 2 cells finds matches for 12.6, 10.0 and 16.9 only by combining unrelated rows or years, and no pair gives 19.5.
- An error in Table 2 rather than the abstract. Not supported by the paper's internal checks. Table 2's age rows reproduce its 15–49 rows as N-weighted averages to within 0.1 pp. The 15–49 rows are the figures the Results text and the abstract's own all-women sentence cite. Without the DHS source tables, an independent check of each subgroup cell is outside the scope of this note. Whichever is correct, the abstract and Table 2 cannot both be.
- Values taken from an external source. Not determinable from the paper. The Methods also mention UN Population Division estimates, but the paper reports none by age and marital status, and the abstract does not cite a source for the subgroup gains. Nothing in the paper or its supplements yields the four values.
- A text-conversion artifact. Ruled out. The PMC HTML, the Europe PMC full-text XML (parsed cell by cell) and a secondary copy of the abstract all carry the same values. Table 2 has no stray superscripts, signs or misaligned cells.
Status
- Version of record. The publisher's pages (gh.bmj.com, through doi.org, the article page and the PDF) returned HTTP 403 to automated access. I checked the publisher-deposited full text in two places, the PMC HTML page and the Europe PMC full-text XML. A secondary copy of the abstract (an institutional repository record) has the same wording and values. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. All seven supplementary files were downloaded as one bundle from Europe PMC: a PRISMA figure, search strategies, operationalisation of variables, a grey-literature table, the decomposition analysis (file 4, PDF), interview guides and a policy mapping. Only file 4 contains subgroup mCPR; its values are given above and do not match the abstract.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices, and the Crossref record has no update-to or updated-by fields. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC13250227; Europe PMC full-text XML and supplementary files for PMC13250227; Crossref; PubPeer; https://doi.org/10.1136/bmjgh-2024-018775.
Suggested correction
If Table 2 is correct, the abstract's Results sentences
Among married women, gains were larger in those aged 15–24 years (+12.6) than in women aged 25–49 years (+10.0). Unmarried women maintained substantially higher use, with gains of +19.5 and +16.9 points, respectively.
could read
Among married women, gains were similar in those aged 15–24 years (+13.5) and 25–49 years (+14.2), although prevalence among the younger group rose more in relative terms (from 3.7% to 17.2%). Unmarried, sexually active women maintained substantially higher use, with gains of +31.5 and +21.0 points, respectively.
If the abstract values were instead drawn from a different source or sample, the authors could name that source in the abstract and explain why its subgroup gains differ from Table 2. Either way, the abstract's comparison of married age groups should be stated so that it agrees with the reported data.
Recomputation script (python recompute.py.txt)
"""Recompute the subgroup mCPR gains reported in the abstract of PMC13250227
(BMJ Glob Health 2026, doi:10.1136/bmjgh-2024-018775,
"Progress in family planning in Sierra Leone: a mixed-methods case study").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 2, all nine rows (mCPR % and N for 2008, 2013, 2019).
Abstract, Results: "+15.7 percentage points" (all women 15-49), "+12.6" (married 15-24),
"+10.0" (married 25-49), "+19.5 and +16.9 points, respectively" (unmarried).
Online supplemental file 4 (s005.pdf), descriptive tables, "Outcome: Current use of any
modern contraceptive method (%)", 2008 and 2019, for the decomposition sample (had sex in
past 12 months, not pregnant, not menopausal, fecund).
"""
import itertools
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
YEARS = (2008, 2013, 2019)
# ------------------------------------------------------------------ inputs: Table 2
# name: (mCPR 2008, 2013, 2019, N 2008, 2013, 2019)
T = {
"All 15-49": (8.2, 20.9, 23.9, 7374, 16658, 15574),
"All 15-24": (7.9, 23.7, 24.9, 2384, 6561, 6056),
"All 25-49": (8.3, 19.1, 23.4, 4989, 10096, 9519),
"Married 15-49": (6.7, 15.6, 20.9, 5525, 10903, 9715),
"Married 15-24": (3.7, 11.8, 17.2, 1171, 2299, 1842),
"Married 25-49": (7.6, 16.6, 21.8, 4353, 8603, 7873),
"Unmarried SA 15-49": (24.5, 56.3, 52.6, 551, 2058, 1987),
"Unmarried SA 15-24": (26.6, 57.5, 58.1, 343, 1472, 1309),
"Unmarried SA 25-49": (21.1, 53.3, 42.1, 208, 586, 678),
}
# ------------------------------------------------------------------ inputs: abstract
ABSTRACT = {"All 15-49": 15.7, "Married 15-24": 12.6, "Married 25-49": 10.0,
"Unmarried SA 15-24": 19.5, "Unmarried SA 25-49": 16.9}
# ------------------------------------------------------------------ inputs: supplemental file 4
# (2008, 2019) current use of any modern method, %, decomposition sample
SUPP = {"All 15-24": (14.9, 44.7), "All 25-49": (10.6, 31.8), "All 15-49": (11.7, 36.0),
"Married 15-24": (5.7, 28.5), "Married 25-49": (9.5, 30.0), "Married 15-49": (8.8, 29.8)}
SUPP_TEXT = {"All 15-49": 24.3, "All 15-24": 29.8, "All 25-49": 21.2,
"Married 15-49": 20.9, "Married 15-24": 22.8, "Married 25-49": 20.5}
def r1(x):
return round(x + 1e-9, 1)
# ------------------------------------------------------------------ 1. row-wise changes
print("=== 1. Table 2 changes in percentage points (pp) vs abstract ===")
print(f"{'row':20s} {'08->13':>7s} {'13->19':>7s} {'08->19':>7s} {'rel%08->19':>11s} abstract diff")
G = {}
for k, v in T.items():
a, b, c = v[:3]
G[k] = dict(d1=r1(b - a), d2=r1(c - b), d=r1(c - a), rel=(c - a) / a * 100)
ab = ABSTRACT.get(k)
diff = f"{r1(G[k]['d'] - ab):+5.1f}" if ab is not None else ""
print(f"{k:20s} {G[k]['d1']:7.1f} {G[k]['d2']:7.1f} {G[k]['d']:7.1f} {G[k]['rel']:10.1f}% "
f"{'' if ab is None else ab:>8} {diff}")
# ------------------------------------------------------------------ 2. ranking
print("\n=== 2. Ranking of married age groups ===")
gm1, gm2 = G["Married 15-24"]["d"], G["Married 25-49"]["d"]
print(f"Table 2: married 15-24 {gm1} vs 25-49 {gm2}; younger minus older = {r1(gm1 - gm2):+.1f} pp "
f"-> 15-24 larger? {gm1 > gm2}")
print(f"Abstract: 12.6 vs 10.0; younger minus older = {r1(12.6 - 10.0):+.1f} pp")
print(f"Relative (Table 2): married 15-24 {G['Married 15-24']['rel']:.1f}% vs 25-49 "
f"{G['Married 25-49']['rel']:.1f}% (ratio 2019/2008: "
f"{T['Married 15-24'][2] / T['Married 15-24'][0]:.2f} vs {T['Married 25-49'][2] / T['Married 25-49'][0]:.2f})")
gu1, gu2 = G["Unmarried SA 15-24"]["d"], G["Unmarried SA 25-49"]["d"]
print(f"Unmarried (Table 2): 15-24 {gu1} vs 25-49 {gu2}")
# ------------------------------------------------------------------ 3. rounding bounds
print("\n=== 3. Rounding bounds (each Table 2 cell is +/-0.05) ===")
for k in ABSTRACT:
d = T[k][2] - T[k][0]
print(f"{k:20s} gain in [{d - 0.1:.1f}, {d + 0.1:.1f}] abstract {ABSTRACT[k]} "
f"inside? {d - 0.1 - 1e-9 <= ABSTRACT[k] <= d + 0.1 + 1e-9}")
# ------------------------------------------------------------------ 4. any pair of Table 2 cells
print("\n=== 4. Every ordered pair of Table 2 mCPR cells whose difference equals an abstract value (+/-0.05) ===")
cells = [(k, y, v[i]) for k, v in T.items() for i, y in enumerate(YEARS)]
for tgt in (12.6, 10.0, 19.5, 16.9):
hits = [(f"{a[0]} {a[1]}", f"{b[0]} {b[1]}") for a, b in itertools.permutations(cells, 2)
if abs((b[2] - a[2]) - tgt) < 0.051]
print(f"target {tgt}: {len(hits)} pair(s): {hits}")
# ------------------------------------------------------------------ 5. internal consistency of Table 2
print("\n=== 5. Table 2 internal check: N-weighted mean of the two age rows vs the 15-49 row ===")
for grp, (s1, s2, tot) in {"All": ("All 15-24", "All 25-49", "All 15-49"),
"Married": ("Married 15-24", "Married 25-49", "Married 15-49"),
"Unmarried": ("Unmarried SA 15-24", "Unmarried SA 25-49", "Unmarried SA 15-49")}.items():
out = []
for i, y in enumerate(YEARS):
p1, p2 = T[s1][i], T[s2][i]
n1, n2 = T[s1][3 + i], T[s2][3 + i]
out.append(f"{y}: {(p1 * n1 + p2 * n2) / (n1 + n2):.1f} vs {T[tot][i]} (N {n1 + n2} vs {T[tot][3 + i]})")
print(f"{grp:9s} " + "; ".join(out))
# ------------------------------------------------------------------ 6. supplemental file 4
print("\n=== 6. Supplemental file 4 (decomposition sample), 2008->2019 ===")
for k, (a, b) in SUPP.items():
print(f"{k:14s} {a:5.1f} -> {b:5.1f} gain {r1(b - a):5.1f} (text: {SUPP_TEXT[k]})")
# ------------------------------------------------------------------ 7. annualised
print("\n=== 7. Annualised gains 2008->2019 (11 years), pp/year ===")
for k in ABSTRACT:
print(f"{k:20s} {G[k]['d'] / 11:.2f}")
# ================================================================== figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": FG,
"axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
"svg.fonttype": "none"})
def save(fig, stem):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{stem}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---------------------------------------------------------------- fig 1: two text panels
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.6)
ax.axis("off")
ax.set_facecolor(BG)
L = (0.15, 2.85) # left panel x0, x1
R = (4.35, 7.05) # right panel x0, x1
TOP, BOT = 4.0, 0.7
for (x0, x1), title, sub in ((L, "Abstract, Results", "values as printed"),
(R, "Table 2 (2019 − 2008)", "recomputed from its own cells")):
ax.add_patch(FancyBboxPatch((x0, BOT), x1 - x0, TOP - BOT, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(x0 + 0.12, TOP - 0.22, title, fontsize=12, fontweight="bold", va="center")
ax.text(x0 + 0.12, TOP - 0.5, sub, fontsize=10, color=FG2, va="center")
rows = [ # (label, abstract text, table key, annotation)
("Married 15–24", "+12.6", "Married 15-24"),
("Married 25–49", "+10.0", "Married 25-49"),
("Unmarried 15–24", "+19.5", "Unmarried SA 15-24"),
("Unmarried 25–49", "+16.9", "Unmarried SA 25-49"),
]
ys = [3.05, 2.55, 1.55, 1.05]
for (lab, abtxt, key), y in zip(rows, ys):
a, c = T[key][0], T[key][2]
ax.text(L[0] + 0.15, y, lab, fontsize=10.5, va="center")
ax.text(L[1] - 0.15, y, abtxt, fontsize=11.5, va="center", ha="right", color=ORANGE, fontweight="bold")
ax.text(R[0] + 0.15, y + 0.13, lab, fontsize=10, va="center", color=FG2)
ax.text(R[0] + 0.15, y - 0.12, f"{a} → {c} = +{G[key]['d']}", fontsize=11, va="center",
color=BLUE, fontweight="bold")
ax.annotate("", xy=(R[0] - 0.05, y), xytext=(L[1] + 0.05, y),
arrowprops=dict(arrowstyle="-", color=FG2, lw=1.4, ls=(0, (3, 2))))
ax.text((L[1] + R[0]) / 2, y + 0.11, f"differs by {abs(r1(G[key]['d'] - ABSTRACT[key])):.1f} pp",
fontsize=10, ha="center", va="bottom", color=FG2)
# ranking statements (married)
ax.text(L[0] + 0.15, 2.12, "“gains were larger in those\naged 15–24”", fontsize=10, va="center",
color=ORANGE, style="italic")
ax.text(R[0] + 0.15, 2.12, f"25–49 gained more\n(+{gm2} vs +{gm1})", fontsize=10, va="center",
color=BLUE, style="italic")
ax.annotate("", xy=(R[0] - 0.05, 2.12), xytext=(L[1] + 0.05, 2.12),
arrowprops=dict(arrowstyle="<->", color=FG, lw=2))
ax.text((L[1] + R[0]) / 2, 2.21, "ranking reversed", fontsize=10.5, ha="center", va="bottom",
fontweight="bold")
# check row
ax.text((L[0] + R[1]) / 2, 0.45,
f"Check that matches: all women 15–49, abstract +15.7; Table 2: {T['All 15-49'][0]} → "
f"{T['All 15-49'][2]} = +{G['All 15-49']['d']}",
fontsize=10, ha="center", va="center", color=FG2)
ax.text(0.15, 4.35, "Orange: value printed in the paper's abstract. Blue: value recomputed from Table 2.",
fontsize=10, color=FG2, va="center")
ax.text((L[0] + R[1]) / 2, 0.2, "pp = percentage points of mCPR; unmarried = sexually active, not in union",
fontsize=10, color=FG2, va="center", ha="center")
save(fig, "fig1")
# ---------------------------------------------------------------- fig 2: dot comparison
order = [("All women 15–49", "All 15-49"), ("Married 15–24", "Married 15-24"),
("Married 25–49", "Married 25-49"), ("Unmarried 15–24", "Unmarried SA 15-24"),
("Unmarried 25–49", "Unmarried SA 25-49")]
fig, ax = plt.subplots(figsize=(7.2, 3.6), facecolor=BG)
ax.set_facecolor(BG)
for i, (lab, key) in enumerate(order):
y = len(order) - 1 - i
rep, rec = ABSTRACT[key], G[key]["d"]
ax.plot([min(rep, rec), max(rep, rec)], [y, y], color="#bdbcb7", lw=2, zorder=1)
if abs(rep - rec) < 0.3:
# coincident values: orange ring drawn larger and unfilled, blue dot on top, both visible
ax.plot(rep, y, "o", ms=15, mfc="none", mec=ORANGE, mew=2.2, zorder=3)
ax.plot(rec, y, "o", ms=8, color=BLUE, zorder=4)
else:
ax.plot(rec, y, "o", ms=10, color=BLUE, zorder=3)
ax.plot(rep, y, "o", ms=10, mfc=BG, mec=ORANGE, mew=2.2, zorder=4)
if key in SUPP:
s = r1(SUPP[key][1] - SUPP[key][0])
ax.plot(s, y, "D", ms=8, mfc="none", mec=FG2, mew=1.6, zorder=2)
if abs(rep - rec) < 0.3:
ax.text(rec + 0.9, y, f"{rec} (both)", va="center", fontsize=10, color=FG)
else:
lo, hi = sorted([(rep, ORANGE), (rec, BLUE)])
ax.text(lo[0] - 0.9, y, f"{lo[0]}", va="center", ha="right", fontsize=10, color=FG)
ax.text(hi[0] + 0.9, y, f"{hi[0]}", va="center", ha="left", fontsize=10, color=FG)
ax.set_yticks(range(len(order)))
ax.set_yticklabels([lab for lab, _ in order][::-1], fontsize=10.5, color=FG)
ax.set_xlim(0, 36)
ax.set_ylim(-0.6, len(order) - 0.4)
ax.set_xlabel("Gain in mCPR, 2008 → 2019 (percentage points)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#9a9994")
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="o", ls="", ms=10, mfc=BG, mec=ORANGE, mew=2.2, label="Reported in abstract"),
Line2D([], [], marker="o", ls="", ms=10, color=BLUE, label="Recomputed from Table 2"),
Line2D([], [], marker="D", ls="", ms=8, mfc="none", mec=FG2, mew=1.6,
label="Suppl. file 4 (different sample)")]
ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.42, 1.0), ncol=3, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.2)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract reverses the direction of both age effects on cortical thickness reported in Table 3
Brain (2023) · doi:10.1093/brain/awad100 · PMC10473572
Summary
In its brain-resilience results, the abstract says that "older age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau". The paper's own Table 3 reports a negative age coefficient on AD-signature cortical thickness (standardized β = −0.49, 95% CI −0.613 to −0.366, P < 0.001) and a negative Time × Age coefficient (standardized β = −0.051, 95% CI −0.083 to −0.020, P = 0.002). Both mean that older age goes with thinner cortex and faster thinning. The Results, both relevant Discussion passages and Supplementary Table 5 all give that direction, and the printed intervals reproduce the printed t values, so the signs in the table are not typesetting slips. Only the abstract sentence states the reverse. Because the abstract is the part most often read and indexed, readers who rely on it get the direction of a secondary finding backwards; the estimates themselves and the paper's main conclusions are unaffected.
What the paper reports
Abstract, results paragraph:
Higher education also modified the negative impact of tau-PET on cortical thinning, while older age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau.
For contrast, the same abstract paragraph gives the cognitive-resilience result with the favourable direction attached to younger age:
Younger age, higher education and greater cortical thickness were associated with better cognitive performance at baseline.
Results, "Brain resilience", first paragraph:
In models that estimated main effects (i.e. after removing the three-way interaction term), older age was related to thinner cross-sectional AD-signature cortex [stβ (95%CI) = -0.49 (-0.613,-0.366), P < 0.001] and to accelerated cortical thinning [stβ (95%CI) = -0.051 (-0.083,-0.02), P < 0.01] independent of temporal meta-ROI tau.
Results of analyses with global tau burden were consistent with these findings (Supplementary Table 5).
Table 3 ("Results of linear mixed-effect models investigating determinants of brain resilience to tau burden in the temporal meta-ROI"), Age row. Columns: Time × Tau × Variable β (unstandardized | standardized | t | P), Time × Variable β (same four), Variable β (same four):
Agea | 0.001 (-0.001,0.002) | 0.009 (-0.021,0.038) | 0.566 | 0.572 | -0.001 (-0.002,0) | -0.051 (-0.083,-0.02) | -3.182 | 0.002 | -0.012 (-0.015,-0.009) | -0.49 (-0.613,-0.366) | -7.774 | <0.001
Table 3 footer, on coding of the outcome and predictors:
Time (years), temporal meta-ROI tau (SUVR), age at baseline (years), educational level (years of education) and ICV (dm3) were centred and used in continuous form
The ‘Time × Variable β’ and ‘Variable β’ coefficients are main effects from models where the non-significant interaction term was removed.
Methods, "Determinants":
For the current study, age (in years) was defined as the age at the time of the tau-PET scan
Discussion, first paragraph:
While there was no evidence for age as a moderator in brain resilience models, older age was associated with lower cortical thickness at the time of the tau-PET scan, and with faster cortical thinning over time.
Discussion, "Age and cortical thickness":
we found that younger age was associated with both higher baseline cortical thickness and slower rate of cortical thinning at similar levels of tau burden.
Conclusion:
we found that age, education, ICV and cortical thickness play a role in cognitive resilience, while age and education contribute to brain resilience.
What the paper's own numbers show
Method. The outcome of the Table 3 models is AD-signature cortical thickness in mm (a higher value is a thicker cortex), and age is continuous, centred, in years. A negative Age coefficient therefore means a thinner cortex at the tau-PET scan for older participants, and a negative Time × Age coefficient means a more negative thickness slope, i.e. faster thinning, for older participants. The abstract's sentence requires both coefficients to be positive. To check that the printed signs belong to the printed statistics, I recomputed each standard error from its 95% CI (SE = (upper − lower)/(2 × 1.96)), the t value and P value that follow, and conversely the CI implied by the printed β and t. I also checked that the unstandardized and standardized rows agree, and repeated the sign check on Supplementary Table 5 (global tau, global thickness). Raw data are not available, so the models themselves cannot be refitted. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Age → baseline thickness, standardized β | Table 3; Results | −0.49 (−0.613, −0.366), t = −7.774, P < 0.001 | CI midpoint −0.4895; SE 0.0630; t −7.78; P ≈ 7.5 × 10⁻¹⁵; negative, CI excludes 0 |
| Time × Age → rate of thinning, standardized β | Table 3; Results | −0.051 (−0.083, −0.02), t = −3.182, P = 0.002 (Results: P < 0.01) | CI midpoint −0.0515; SE 0.0161; t −3.17; P ≈ 0.0015; negative, CI excludes 0 |
| CI rebuilt from printed β and t | Table 3 | as above | Age −0.614 to −0.366; Time × Age −0.082 to −0.020 (match the printed CIs) |
| Age, unstandardized (mm per year of age) | Table 3 | −0.012 (−0.015, −0.009) | t −7.84 from the CI; negative; 10 years older ≈ 0.12 mm thinner at the tau-PET scan |
| Time × Age, unstandardized | Table 3 | −0.001 (−0.002, 0) | negative; t from the CI is only −1.96 because the values are rounded to 3 decimals |
| Agreement of standardized and unstandardized rows | Table 3 | — | −0.49 / −0.012 = 40.8; a printed −0.001 (−0.0015 to −0.0005) × 40.8 = −0.061 to −0.020, which contains the printed −0.051 |
| Expected thickness gap, 1 SD older vs average age, average tau | Table 3 model without the three-way term | — | −0.49 SD at the tau-PET scan; −0.567 SD at the median MRI follow-up (18 months); −0.758 SD at the longest (63 months): the gap widens |
| Age as moderator of the tau effect (Time × Tau × Age) | Table 3 | 0.009 (−0.021, 0.038), P = 0.572 | t 0.60 from the CI; not significant, consistent with "independent of tau" |
| Age → baseline thickness, global analysis | Supplementary Table 5 | −0.374 (−0.522 – −0.227), t = −5.001, P < 0.001 | t −4.97 from the CI; negative |
| Time × Age → rate of thinning, global analysis | Supplementary Table 5 | −0.044 (−0.084 – −0.003), t = −2.112, P = 0.036 | t −2.13 from the CI; negative |
| Direction stated in the abstract | Abstract | older → higher baseline thickness, slower thinning | requires Age β > 0 and Time × Age β > 0; both printed coefficients are < 0 |
Figure
Figure 1. The left panel holds the abstract's sentence on age and brain resilience and the direction it states. The right panel holds the Results sentence and the Table 3 Age row (standardized β with 95% CI and t), all as printed (orange). The connector marks the conflict: the two panels give opposite directions for both the baseline (cross-sectional) effect and the longitudinal (Time × Age) effect. The blue lines give the t and P values recomputed from the printed confidence intervals; they match the printed t values, so the negative signs are internally consistent within the table.
Figure 2. Each panel shows one Table 3 coefficient on its own symmetric axis, with zero as a thin reference line; the shaded half is the direction the abstract states (older age with thicker cortex, or with slower thinning). Orange circles and bars are the printed estimates and 95% CIs; blue squares and bars are the intervals rebuilt from the printed β and t (SE = β/t). Both intervals lie entirely on the negative side, away from the direction stated in the abstract.
Why it matters
- Direction, not magnitude. The error reverses the stated direction of two associations. The size and significance are printed correctly in the body: about 0.49 SD thinner AD-signature cortex per SD of age (about 0.012 mm per year of age), and a thickness slope about 0.051 SD per year more negative per SD of age. Both intervals exclude zero.
- It changes the reading of a resilience finding. As written, the abstract says older age is associated with preserved brain structure, which a reader would take as older age conferring brain resilience. The paper's actual finding is the conventional one, which its Discussion states: younger age goes with thicker cortex and slower thinning at similar tau burden, as an additive effect rather than a moderator.
- Where it sits. The abstract is the part of the paper that most readers, databases and evidence syntheses use. Someone who takes the age-resilience association from the abstract alone gets the sign wrong, and nothing in the abstract signals the conflict. The Conclusion's "age and education contribute to brain resilience" gives no direction, so read with the abstract it carries the reversed direction too.
- What it does not affect. The headline result (education as the most robust determinant), the education moderation of tau-related thinning, the cognitive-resilience results and the age results in the body are not affected. Age did not moderate the tau–thinning relationship (P = 0.572), and the abstract's "independent of tau" is consistent with that.
Explanations considered
- The outcome is an atrophy score, so a negative coefficient means less atrophy. Ruled out. The Table 3 header names the outcome "AD-signature cortical thickness", the Methods compute cortical thickness with FreeSurfer as "the distance from the grey matter–white matter boundary to the corresponding pial surface", the Table 2 footer gives AD-signature cortical thickness in mm, and the Results describe tau uptake as "significantly negatively associated with cortical thinning". A negative coefficient is a thinner cortex.
- Age is reverse-coded or entered as a "younger" indicator. Ruled out. The Methods define age as age in years at the tau-PET scan, and the Table 3 footer says it was centred and used in continuous form.
- The abstract describes a different model (full model, global tau, sensitivity analysis). Not supported. The Table 3 footer and the Results say the age main effects come from the model without the non-significant three-way term, which matches the abstract's "independent of tau". The global-tau analysis (Supplementary Table 5) gives negative age coefficients as well (−0.374 and −0.044). No analysis in the paper or the supplement reports a positive age effect on thickness or a slower thinning rate with age.
- The table signs are typos and the abstract is right. Ruled out. The printed CIs reproduce the printed t values (−7.78 vs −7.774; −3.17 vs −3.182), the unstandardized and standardized rows agree in sign and scale, and the Results and two Discussion passages describe the same negative direction in words.
- The sentence means "younger age" was associated with thicker cortex and slower thinning. That is what the body reports, and it is also how the Discussion phrases it ("younger age was associated with both higher baseline cortical thickness and slower rate of cortical thinning"). The abstract, however, says "older age". As printed, the abstract sentence states the opposite of the paper's results.
- A conversion or archiving artifact. Ruled out. The publisher's version of record, the PMC page, the Europe PMC full-text XML and the PMC archive used here all carry the same abstract wording.
- Already corrected. No correction, erratum or other notice was found (see Status).
Status
As of 2026-10-23:
- Version of record. The DOI resolves to the publisher's version of record (Brain 146(9):3719). Its abstract carries the same sentence ("older age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau"), and its Results and Table 3 carry the same negative coefficients quoted above. The PMC page, the Europe PMC full-text XML and the PMC archive agree word for word.
- Supplementary material. The single supplementary file (awad100_supplementary_data.pdf, retrieved from Europe PMC) was read. Supplementary Table 5 (global tau, global cortical thickness) reports negative age coefficients for both baseline thickness and thinning rate, and nothing in the supplement supports the abstract's direction.
- Corrections and notices. Crossref (no update-to or updated-by relations; Crossref now includes Retraction Watch data), PMC and Europe PMC notice records, and PubMed CommentsCorrections list no correction, erratum, expression of concern or retraction. The publisher's article page shows none; its "Corrected and typeset" date is a production date. None of the 51 citing records in Europe PMC is a correction, comment or reply.
- PubPeer. The PubPeer API returned 403 to automated requests, so a direct search could not be completed. Web searches for a correction, erratum or PubPeer thread on this article found none.
- Sources. The article as archived in PMC (PMC10473572); the publisher's version of record; the supplementary data file; Crossref, PMC, Europe PMC and PubMed records.
Suggested correction
In the abstract, change the sentence so that it states the direction reported in Table 3, the Results and the Discussion, for example:
Higher education also modified the negative impact of tau-PET on cortical thinning, while older age was associated with lower baseline cortical thickness and faster rate of cortical thinning independent of tau.
Equivalently, "younger age was associated with higher baseline cortical thickness and slower rate of cortical thinning independent of tau", which matches the wording of the Discussion. No change to the estimates, tables or conclusions appears to be needed.
Recomputation script (python recompute.py.txt)
"""Recompute the age effects on brain resilience in PMC10473572
(Brain 2023, doi:10.1093/brain/awad100).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file, plus five candidate figure
designs (cand1..cand5, .svg and .png) in the candidates/ subfolder.
All inputs are copied from the article as archived in PubMed Central:
Table 3, Age row (brain resilience, outcome = AD-signature cortical thickness, mm):
Time x Tau x Age, Time x Age and Age coefficients, unstandardized and standardized,
with 95% CIs, t and P.
Results > Brain resilience (the same standardized values, quoted in the text).
Methods / Results: median MRI follow-up 18 months (range 9-63 months).
Supplementary Table 5, Age row (global tau, global cortical thickness as outcome).
No raw data are available, so the model itself cannot be refitted. The checks below test
(i) that the printed CIs, t and P belong together, so the printed signs are not typos, and
(ii) what direction the printed coefficients imply.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from scipy.stats import norm # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
# Table 3, Age row: (beta, lo, hi, t, P as printed)
T3 = {
"age_std": (-0.49, -0.613, -0.366, -7.774, "<0.001"),
"timexage_std": (-0.051, -0.083, -0.020, -3.182, "0.002"),
"age_unstd": (-0.012, -0.015, -0.009, -7.774, "<0.001"),
"timexage_unstd": (-0.001, -0.002, 0.0, -3.182, "0.002"),
"3way_std": (0.009, -0.021, 0.038, 0.566, "0.572"),
}
# Supplementary Table 5, Age row (global tau, global cortical thickness)
S5 = {
"age_std": (-0.374, -0.522, -0.227, -5.001, "<0.001"),
"timexage_std": (-0.044, -0.084, -0.003, -2.112, "0.036"),
}
FU_MEDIAN_Y, FU_MAX_Y = 18 / 12, 63 / 12 # brain resilience subsample, MRI follow-up
LABELS = {
"age_std": "Age, standardized (baseline thickness)",
"timexage_std": "Time x Age, standardized (rate of thinning)",
"age_unstd": "Age, unstandardized (mm per year of age)",
"timexage_unstd": "Time x Age, unstandardized (mm/yr per year of age)",
"3way_std": "Time x Tau x Age, standardized (moderation)",
}
def from_ci(b, lo, hi):
se = (hi - lo) / (2 * Z)
t = b / se
return (lo + hi) / 2, se, t, 2 * norm.sf(abs(t))
def main():
R = {}
print("== 1. Table 3, Age row: SE, t and P recomputed from the printed 95% CIs")
print(" (normal approximation; SE = (upper - lower) / (2 x 1.96))")
for k, (b, lo, hi, t, p) in T3.items():
mid, se, tr, pr = from_ci(b, lo, hi)
R[k] = (mid, se, tr, pr)
print(f" {LABELS[k]}")
print(f" printed beta {b} ({lo}, {hi}), t = {t}, P = {p}")
print(f" CI midpoint {mid:.4f}; SE {se:.4f}; t from CI {tr:.2f}; P from CI {pr:.2g}; "
f"sign {'negative' if b < 0 else 'positive'}; CI excludes 0: {hi < 0 or lo > 0}")
print("\n== 2. 95% CI rebuilt from the printed beta and t (SE = beta / t)")
CI_T = {}
for k in ("age_std", "timexage_std"):
b, lo, hi, t, p = T3[k]
se = b / t
CI_T[k] = (b - Z * se, b + Z * se)
print(f" {LABELS[k]}: SE = {se:.4f}; CI {b - Z * se:.3f} to {b + Z * se:.3f} "
f"(printed {lo} to {hi})")
print("\n== 3. Do the unstandardized and standardized rows agree in sign and scale?")
ratio = T3["age_std"][0] / T3["age_unstd"][0]
print(f" Age: standardized / unstandardized = {T3['age_std'][0]} / {T3['age_unstd'][0]} "
f"= {ratio:.1f} (= SD of age / SD of thickness, in years per mm)")
lo_u, hi_u = -0.0015, -0.0005 # values that round to -0.001
print(f" Time x Age unstandardized printed as -0.001, i.e. between {lo_u} and {hi_u};")
print(f" times {ratio:.1f} gives a standardized value between {lo_u * ratio:.3f} and "
f"{hi_u * ratio:.3f}; printed standardized value {T3['timexage_std'][0]} lies inside")
print(" (time is in years in both, so the same factor applies)")
print(f" Unstandardized Age: -0.012 mm per year of age -> 10 years older = "
f"{10 * T3['age_unstd'][0]:.2f} mm thinner AD-signature cortex at the tau-PET scan")
print("\n== 4. What the Table 3 coefficients imply (model without the 3-way term, average tau)")
b0, b1 = T3["age_std"][0], T3["timexage_std"][0]
for yrs, lab in ((0, "baseline (tau-PET)"), (FU_MEDIAN_Y, "median MRI follow-up, 18 months"),
(FU_MAX_Y, "longest MRI follow-up, 63 months")):
print(f" 1 SD older, at {lab}: expected thickness difference = {b0} + ({b1} x {yrs:.2f}) "
f"= {b0 + b1 * yrs:.3f} SD")
print(" Both terms are negative: the older participant starts thinner and the gap widens,")
print(" i.e. older age -> lower baseline thickness and faster thinning.")
print(" The abstract ('older age ... higher baseline cortical thickness and slower rate of")
print(" cortical thinning') would need Age beta > 0 and Time x Age beta > 0.")
b, lo, hi, t, p = T3["3way_std"]
print(f" Age as moderator of the tau effect: {b} ({lo}, {hi}), P = {p}: not significant,")
print(" consistent with the abstract's 'independent of tau'.")
print("\n== 5. Supplementary Table 5 (global tau, global cortical thickness), Age row")
for k, (b, lo, hi, t, p) in S5.items():
mid, se, tr, pr = from_ci(b, lo, hi)
print(f" {LABELS[k]}: printed {b} ({lo}, {hi}), t = {t}, P = {p}; "
f"t from CI {tr:.2f}; P from CI {pr:.2g}; sign {'negative' if b < 0 else 'positive'}")
os.makedirs(CAND, exist_ok=True)
designs = [cand1_text_panels, cand2_intervals, cand3_grid, cand4_trajectory, cand5_combined]
for i, f in enumerate(designs, 1):
fig = f(R, CI_T)
save(fig, os.path.join(CAND, f"cand{i}"))
plt.close(fig)
# chosen designs for the write-up
save(cand1_text_panels(R, CI_T), os.path.join(HERE, "fig1"))
save(cand2_intervals(R, CI_T), os.path.join(HERE, "fig2"))
print("\nFigures written: fig1, fig2 and candidates/cand1..cand5 (.svg, .png)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
PANEL = "#f3f2ef"
plt.rcParams.update({
"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT, "xtick.color": TXT,
"ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "font.family": "DejaVu Sans",
})
W = 7.2
def save(fig, stem):
fig.savefig(stem + ".svg")
fig.savefig(stem + ".png", dpi=200)
def box(ax, x, y, w, h, edge=SEC, face=PANEL, lw=1.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=face, ec=edge, lw=lw, transform=ax.transAxes))
def lines(ax, x, y, items, dy=0.058):
"""items: list of (text, color, weight, size)."""
for txt, col, wt, sz in items:
ax.text(x, y, txt, color=col, fontweight=wt, fontsize=sz, va="top", ha="left",
transform=ax.transAxes)
y -= dy * sz / 10.5
return y
def fmt(v, nd=3):
return f"{v:.{nd}f}".replace("-", "−")
# 1 ---- two text panels joined by an annotated connector
def cand1_text_panels(R, CI_T):
fig = plt.figure(figsize=(W, 4.9))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.03, 0.965, "The abstract and the body give opposite directions for both age effects",
fontsize=12, fontweight="bold", va="top")
ax.text(0.03, 0.91, "Brain resilience models; outcome = AD-signature cortical thickness (mm)",
fontsize=10, color=SEC, va="top")
# left panel
box(ax, 0.03, 0.255, 0.385, 0.605)
lines(ax, 0.05, 0.835, [
("ABSTRACT", SEC, "bold", 10),
("\u201c\u2026older age was associated", TXT, "normal", 10.5),
("with higher baseline cortical", TXT, "bold", 10.5),
("thickness and slower rate of", TXT, "bold", 10.5),
("cortical thinning independent", TXT, "bold", 10.5),
("of tau.\u201d", TXT, "normal", 10.5),
], dy=0.052)
lines(ax, 0.05, 0.43, [
("Direction stated:", SEC, "normal", 10),
("older \u2192 thicker at baseline", ORANGE, "bold", 10.5),
("older \u2192 slower thinning", ORANGE, "bold", 10.5),
], dy=0.052)
# right panel
box(ax, 0.55, 0.255, 0.425, 0.605)
lines(ax, 0.57, 0.835, [
("RESULTS AND TABLE 3", SEC, "bold", 10),
("\u201colder age was related to", TXT, "normal", 10.5),
("thinner cross-sectional", TXT, "bold", 10.5),
("AD-signature cortex [\u2026] and", TXT, "normal", 10.5),
("to accelerated cortical", TXT, "bold", 10.5),
("thinning\u201d", TXT, "bold", 10.5),
], dy=0.052)
lines(ax, 0.57, 0.50, [
("Table 3, Age row, std. \u03b2 (95% CI):", SEC, "normal", 10),
("Age \u22120.49 (\u22120.613, \u22120.366)", ORANGE, "bold", 10),
("Time\u00d7Age \u22120.051 (\u22120.083, \u22120.02)", ORANGE, "bold", 10),
("t = \u22127.774 and \u22123.182", ORANGE, "bold", 10),
("older \u2192 thinner, faster thinning", TXT, "bold", 10.5),
], dy=0.052)
# connector
ar = FancyArrowPatch((0.42, 0.56), (0.545, 0.56), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT, transform=ax.transAxes)
ax.add_patch(ar)
ax.text(0.4825, 0.605, "opposite", ha="center", fontsize=10, fontweight="bold")
ax.text(0.4825, 0.535, "sign for\nboth", ha="center", va="top", fontsize=10)
# recomputed strip
_, se0, t0, p0 = R["age_std"]; _, se1, t1, p1 = R["timexage_std"]
ax.text(0.03, 0.215, "Recomputed from the printed 95% CIs:", fontsize=10, color=SEC, va="top")
ax.text(0.03, 0.165, f"Age t \u2248 {fmt(t0, 2)} (P \u2248 {p0:.0e}); Time\u00d7Age t \u2248 "
f"{fmt(t1, 2)} (P \u2248 {p1:.4f})", fontsize=10, color=BLUE, fontweight="bold", va="top")
ax.text(0.03, 0.115, "Both intervals exclude 0, and the t values match the printed ones,",
fontsize=10, color=BLUE, va="top")
ax.text(0.03, 0.078, "so the negative signs in Table 3 are not typesetting slips.",
fontsize=10, color=BLUE, va="top")
ax.text(0.97, 0.008, "orange = printed in the paper blue = recomputed from its numbers",
fontsize=10, color=SEC, ha="right", va="bottom")
return fig
# 2 ---- dot-and-interval plots, one panel per coefficient, zero line, abstract direction shaded
def cand2_intervals(R, CI_T):
fig, axs = plt.subplots(2, 1, figsize=(W, 4.2), gridspec_kw=dict(hspace=0.75))
spec = [("age_std", "Age → baseline thickness (standardized β)", 0.7,
"older → thinner at baseline", "abstract: older → thicker at baseline"),
("timexage_std", "Time × Age → rate of thinning (standardized β per year)", 0.1,
"older → faster thinning", "abstract: older → slower thinning")]
for ax, (k, title, lim, lneg, lpos) in zip(axs, spec):
b, lo, hi, t, p = T3[k]
ax.axvspan(0, lim, color=ORANGE, alpha=0.08, lw=0)
ax.axvline(0, color=SEC, lw=0.8)
ax.errorbar([b], [1], xerr=[[b - lo], [hi - b]], fmt="o", color=ORANGE, ms=8, lw=2,
capsize=0)
cl, ch = CI_T[k]
ax.errorbar([b], [0], xerr=[[b - cl], [ch - b]], fmt="s", color=BLUE, ms=8, lw=2,
capsize=0)
ax.set_yticks([1, 0])
ax.set_yticklabels(["Table 3, printed", "rebuilt from β and t"], fontsize=10)
ax.set_ylim(-0.7, 2.3)
ax.set_xlim(-lim, lim)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_title(title, loc="left", fontsize=10.5, fontweight="bold", pad=6)
ax.text(lim * 0.97, 2.15, lpos, ha="right", va="top", fontsize=10, color=SEC)
ax.text(-lim * 0.97, 2.15, lneg, ha="left", va="top", fontsize=10, color=SEC)
ax.annotate(f"{fmt(b, 3 if abs(b) < 0.1 else 2)} ({fmt(lo)}, {fmt(hi)})", (hi, 1),
xytext=(8, 0), textcoords="offset points", va="center", fontsize=10,
color=ORANGE, fontweight="bold")
ax.annotate(f"({fmt(cl)}, {fmt(ch)})", (ch, 0), xytext=(8, 0),
textcoords="offset points", va="center", fontsize=10, color=BLUE,
fontweight="bold")
fig.subplots_adjust(left=0.22, right=0.97, top=0.9, bottom=0.08)
return fig
# 3 ---- consistency grid: every place the paper states the direction
def cand3_grid(R, CI_T):
rows = [
("Abstract", "thicker", "slower", True),
("Results › Brain resilience", "thinner", "faster (accelerated)", False),
("Table 3, Age row", "−0.49 (thinner)", "−0.051 (faster)", False),
("Discussion, 1st paragraph", "lower", "faster", False),
("Discussion › Age and cortical thickness*", "lower", "faster", False),
("Supplementary Table 5 (global)", "−0.374 (thinner)", "−0.044 (faster)", False),
]
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off"); ax.set_xlim(0, 1); ax.set_ylim(0, 1)
ax.text(0.03, 0.95, "Where the paper states the direction of the age effects on thickness",
fontsize=11.5, fontweight="bold", va="top")
x0, x1, x2 = 0.03, 0.46, 0.73
yh = 0.83
ax.text(x1, yh, "Older age → baseline\nthickness", fontsize=10, color=SEC, va="center",
fontweight="bold")
ax.text(x2, yh, "Older age → rate\nof thinning", fontsize=10, color=SEC, va="center",
fontweight="bold")
dy = 0.095
for i, (src, a, b, bad) in enumerate(rows):
y = yh - 0.12 - i * dy
if bad:
ax.add_patch(FancyBboxPatch((0.02, y - dy / 2 + 0.008), 0.96, dy - 0.016,
boxstyle="round,pad=0,rounding_size=0.01",
fc="#fbe6dd", ec=ORANGE, lw=1.5, transform=ax.transAxes))
else:
ax.plot([0.02, 0.98], [y - dy / 2, y - dy / 2], color=GRID, lw=0.8,
transform=ax.transAxes)
col = ORANGE if bad else TXT
ax.text(x0 + 0.01, y, src, fontsize=10.5, va="center", fontweight="bold" if bad else "normal")
ax.text(x1, y, a, fontsize=10.5, va="center", color=col, fontweight="bold")
ax.text(x2, y, b, fontsize=10.5, va="center", color=col, fontweight="bold")
ax.text(0.03, 0.025, "Highlighted: the only statement with the opposite direction.\n"
"*Phrased as 'younger age → higher baseline thickness, slower thinning'.",
fontsize=10, color=SEC, va="bottom")
return fig
# 4 ---- implied expected difference over follow-up, from the Table 3 coefficients
def cand4_trajectory(R, CI_T):
import numpy as np
b0, b1 = T3["age_std"][0], T3["timexage_std"][0]
t = np.linspace(0, FU_MAX_Y, 50)
fig, ax = plt.subplots(figsize=(W, 3.6))
ax.axhspan(0, 0.9, color=ORANGE, alpha=0.08, lw=0)
ax.axhline(0, color=SEC, lw=0.8)
ax.plot(t, b0 + b1 * t, color=BLUE, lw=2)
ax.plot([0], [b0], "o", color=ORANGE, ms=8)
ax.annotate(f"Table 3 Age β = {fmt(b0, 2)}\n(95% CI {fmt(T3['age_std'][1])}, "
f"{fmt(T3['age_std'][2])})", (0, b0), xytext=(8, -20), textcoords="offset points",
va="top", fontsize=10, color=ORANGE, fontweight="bold")
for yrs, lab in ((FU_MEDIAN_Y, "median follow-up"), (FU_MAX_Y, "longest follow-up")):
v = b0 + b1 * yrs
ax.plot([yrs], [v], "s", color=BLUE, ms=8)
ax.annotate(f"{lab}: {fmt(v, 2)}", (yrs, v), xytext=(0, 10), textcoords="offset points",
ha="center" if yrs < 4 else "right", fontsize=10, color=BLUE,
fontweight="bold")
ax.text(0.1, 0.8, "region the abstract implies (older participant thicker, gap closing or "
"reversing)", fontsize=10, color=SEC, va="top")
ax.text(FU_MAX_Y * 0.98, -0.2, f"slope = Time×Age β = {fmt(b1)} SD/year\n(gap widens: "
"faster thinning)", fontsize=10, color=BLUE, ha="right", va="top")
ax.set_xlim(-0.1, FU_MAX_Y + 0.1); ax.set_ylim(-0.9, 0.9)
ax.set_xlabel("Years from tau-PET")
ax.set_ylabel("Thickness difference, 1 SD older\nvs average age (SD units)")
ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title("Expected AD-signature thickness gap implied by Table 3 (average tau)",
loc="left", fontsize=11, fontweight="bold")
fig.tight_layout()
return fig
# 5 ---- compact: abstract claim on the left, the two printed intervals on the right
def cand5_combined(R, CI_T):
fig = plt.figure(figsize=(W, 3.4))
axt = fig.add_axes([0, 0, 0.36, 1]); axt.axis("off")
box(axt, 0.06, 0.34, 0.9, 0.5)
lines(axt, 0.11, 0.80, [
("ABSTRACT", SEC, "bold", 10),
("“older age was associated", TXT, "normal", 10.5),
("with higher baseline", TXT, "bold", 10.5),
("cortical thickness and", TXT, "bold", 10.5),
("slower rate of cortical", TXT, "bold", 10.5),
("thinning”", TXT, "bold", 10.5),
("implies both β > 0", ORANGE, "bold", 10.5),
])
axt.text(0.06, 0.95, "Claim vs Table 3", fontsize=11.5, fontweight="bold", va="top")
ax = fig.add_axes([0.47, 0.2, 0.5, 0.62])
ks = [("age_std", "Age\n(baseline)"), ("timexage_std", "Time×Age\n(slope)")]
ax.axvspan(0, 0.7, color=ORANGE, alpha=0.08, lw=0)
ax.axvline(0, color=SEC, lw=0.8)
for y, (k, lab) in zip((1, 0), ks):
b, lo, hi, t, p = T3[k]
ax.errorbar([b], [y], xerr=[[b - lo], [hi - b]], fmt="o", color=ORANGE, ms=8, lw=2)
ax.annotate(f"{fmt(b, 3 if abs(b) < 0.1 else 2)}", (b, y), xytext=(0, 9),
textcoords="offset points", ha="center", fontsize=10, color=ORANGE,
fontweight="bold")
ax.set_yticks([1, 0]); ax.set_yticklabels([k[1] for k in ks], fontsize=10)
ax.set_ylim(-0.6, 1.6); ax.set_xlim(-0.7, 0.7)
ax.set_xlabel("Standardized β, Table 3 (95% CI)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.68, 1.5, "abstract's\ndirection", ha="right", va="top", fontsize=10, color=SEC)
ax.set_title("Table 3 (orange, printed): both < 0", loc="left", fontsize=10.5,
fontweight="bold")
_, _, t0, _ = R["age_std"]; _, _, t1, _ = R["timexage_std"]
fig.text(0.47, 0.03, f"Recomputed from CIs: t ≈ {fmt(t0, 2)} and {fmt(t1, 2)}",
fontsize=10, color=BLUE, fontweight="bold")
return fig
if __name__ == "__main__":
main()
A Discussion passage reverses the lesion–donation directions stated in the Abstract and Results
Brain (2018) · doi:10.1093/brain/awy064 · PMC7341482
Summary
The paper maps penetrating brain injuries in 94 veterans against how much money each gave away on an altruistic decision task, and its Abstract, Results, Discussion summary and Figure 3 legend agree that lesions of the dorsomedial parietal cortex (precuneus, paracentral lobule) increased donations while lesions of the right posterior middle temporal gyrus (MTG) and superior temporal sulcus decreased them. The Discussion subsection that interprets this result says the opposite for both regions ("Damage to the left medial parietal lobe decreased donations, whereas damage to the right middle temporal gyrus increased donations"), and calls left-sided a parietal effect that the Abstract calls "bilateral" and Table 3 lists for both hemispheres (12 right, 11 left subjects). The paragraph's closing interpretation follows the reversed direction, so the paper's functional reading of one of its two main findings conflicts with that finding as stated, and the figures' colour bars do not independently settle which statement is correct. No statistic depends on the wording, but no correction has been published.
What the paper reports
The direction given in the Abstract (Abstract, main finding (ii)):
altruistic donations were increased by bilateral lesions of the dorsomedial parietal cortex, whereas lesions of the right posterior superior temporal sulcus and middle temporal gyri decreased donations
The same direction in the Results (Results > Donation, paragraph 1, sentences 1–2):
Lesions that led to increased donations were restricted to the paracentral lobule and precuneus, as well as to small areas in the right dorsomedial prefrontal cortex (Fig. 3).
Injuries leading to a decrease in donations were restricted to small sectors of the right ventrolateral occipital cortex and posterior middle temporal gyrus (Fig. 3).
The same direction in the Discussion's summary of main findings (Discussion, paragraph 1):
altruistic (costly) donations were increased by bilateral lesions of the dorsomedial parietal cortex, whereas lesions of the right posterior superior temporal sulcus and middle temporal gyri had the opposite effect
The colour code of the donation map (Figure 3 legend):
Statistical β-maps of lesions that decreased (red) or increased (blue) donations on the Altruistic Decision Task.
The reversed statement (Discussion > Brain regions implicated in donations, paragraph 1, sentence 3):
Damage to the left medial parietal lobe decreased donations, whereas damage to the right middle temporal gyrus increased donations.
The interpretation that closes the same paragraph:
Overall, our findings suggest that the superomedial parietal cortex is somehow involved in the facilitation of donations, whereas the ventrolateral occipitotemporal cortex is necessary for decisions that reduce the probability of donating.
The definition of the sign of the lesion maps (Materials and methods, lesion analysis paragraph):
each with a positive or a negative sign (i.e. the variable of interest is positively or negatively correlated with the presence of a brain lesion at a specific location
What the paper's own numbers show
Method. The error concerns the direction of an effect, not a computed value. For each passage that describes the lesion–donation result, I recorded which way it says a lesion moves donations in the two regions involved: medial parietal (precuneus/paracentral) and right posterior temporal (posterior MTG, pSTS, ventrolateral occipital). I then compared each passage with Results > Donation. The closing sentence of the Discussion paragraph speaks of the function of intact regions, so I translated it with the usual lesion-deficit reading: if a region "facilitates" donations, damage to it should lower them. I checked the laterality claim against Table 3, and checked that the Table 2 subgroups add up and that their means fit their labels. To see whether the figures follow the colour code their legends give, I counted strongly red and strongly blue pixels in the relevant panels of the paper's Figs 2 and 3 as archived in PMC (the figures are not reproduced here). As a side check, I tested whether each Table 2 column sums to $60, which the task design requires (see below). Every quote and number is checked or computed in recompute.py.txt.
| Quantity | Location | Reported | Recomputed / checked |
|---|---|---|---|
| Effect of medial parietal (precuneus/paracentral) lesions on donations | Abstract (ii); Results > Donation; Discussion summary (ii) | increased | Same direction in all three passages |
| Same effect | Discussion > Brain regions implicated in donations, sentence 3 | "decreased" | Opposite to the three passages above |
| Effect of right posterior MTG/pSTS lesions on donations | Abstract (ii); Results > Donation; Discussion summary (ii) | decreased | Same direction in all three passages |
| Same effect | Discussion > Brain regions implicated in donations, sentence 3 | "increased" | Opposite to the three passages above |
| Lesion effects implied by the closing interpretation (parietal "facilitation"; occipitotemporal cortex needed to "reduce" donating) | Same paragraph, last sentence | — | Parietal lesion → fewer donations, occipitotemporal lesion → more: the same as the reversed sentence |
| Passages agreeing with Results / reversing both regions | All six passages above, Fig. 3 legend included | — | 4 agree (Results included) / 2 reverse |
| Laterality of the medial parietal effect | Abstract and Discussion summary: "bilateral"; Discussion subsection: "left" | left (subsection) | Table 3, "Dorsomedial parietal cortex (precuneus)": side R/L, 12 right / 11 left subjects (23 in all) |
| Fig. 3, medial parietal panels A, B, G: red / blue pixels | Fig. 3 | blue = increased (legend) | 0 / 562, 0 / 664, 0 / 148: blue only, so "increased" under the legend |
| Fig. 3, lateral occipitotemporal panels E, F: red / blue pixels | Fig. 3 | red = decreased (legend) | 313 / 50 and 419 / 40: mostly red, so "decreased" under the legend |
| Fig. 2 legend check: right perisylvian (A) vs medial dorsomedial prefrontal (E–G), red / blue pixels | Fig. 2; Results > Punishment (perisylvian decreased, dorsomedial prefrontal increased punishment) | red = decreased, blue = increased | 3817 / 312 and 241 / 3092: the legend code matches the Results text for punishment as well |
| Fig. 3 colour bar | Fig. 3 (labelled "Beta") | — | The end labelled 10 is dark red, RGB (140, 20, 29); the end labelled −10 is dark blue, RGB (0, 49, 96) |
| Lesion subgroup sizes (P / D / S) | Table 2 | 18+34+22+20; 43+23+28(+0); 32+25+32+5 | 94 / 94 / 94, equal to the 94 patients |
| Mean donations, D+ (n = 23) / D− (n = 28) / controls (n = 28) | Table 2 | $20.0 / $9.7 / $12.9 | D+ is 7.1 above controls and D− 3.2 below: the means fit the subgroup labels. Table 2 does not say which regions define D+ and D− |
| Side check: mean donations + punishments + savings | Table 2, P− column | 13.5 + 2.6 + 39.4 | 55.5 (55.35–55.65 under rounding), not 60 |
| Side check: same sum | Table 2, D− column | 9.7 + 4.5 + 43.8 | 58.0 (57.85–58.15), not 60 |
| Side check: same sum | Table 2, D0 and S+ columns (savings printed without decimals) | 14.7 + 4.6 + 40; 11.4 + 2.6 + 45 | 59.3 (58.70–59.90) and 59.0 (58.40–59.60): just short of 60 even allowing ±0.5 for the whole-dollar savings |
| Side check: same sum | Table 2, the other 8 columns, controls included | — | 59.6–60.0, all consistent with 60 under rounding |
Four sources agree on the direction: the Abstract, the Results, the Discussion's summary, and the Figure 3 legend read against the map colours. The Discussion subsection reverses both regions, and its closing interpretation follows that reversal.
The Table 2 sums are a separate matter and do not bear on which direction is correct. The task gives each participant $60, with 30 opportunities to donate and 30 to punish at $1 each, and pays out "$60 minus the amount you spent on donations and punishments" (supplementary instructions). Donations + punishments + savings therefore equal 60 for every participant, and so do subgroup means taken over the same participants. The P− and D− columns are well outside what rounding allows.
Figure
Figure 1. Short excerpts quoted from the paper. The blue panel holds the direction stated in the Abstract (finding ii) and in Results > Donation; the Discussion's opening summary and the Figure 3 legend give the same direction. The orange panel holds the Discussion subsection "Brain regions implicated in donations": its third sentence, and the closing sentence that interprets the finding. The connector names the conflict. In the subsection, lesions in both regions have the opposite effect on donations, and the parietal effect, "bilateral" in the Abstract, becomes "left".
Figure 2. The direction each passage gives for the effect of a lesion on donations, by region. Blue marks the direction stated in the Abstract and Results; orange marks the reverse. The first four rows agree. The Discussion sentence reverses both regions, and so does the paragraph's closing interpretation when read in the usual lesion-deficit way (italic, because the direction is inferred rather than stated). The grey bottom row is conditional: it shows what Figure 3 would say if its colour bar (red = +10, blue = −10, labelled "Beta") were read with the Methods' definition, under which a positive sign means that a lesion goes with more donations. That reading would agree with the Discussion sentence and not with the legend. The paper does not say how β was signed for display (see Explanations considered).
Why it matters
- It concerns one of the paper's two main findings. The donation map is finding (ii) in the Abstract and in the Discussion summary. The subsection "Brain regions implicated in donations" is the only place where the paper interprets this finding, and there the reader is told the opposite of the Abstract and Results for both regions.
- The functional interpretation follows the reversed version. The closing statement, that the superomedial parietal cortex is "involved in the facilitation of donations" and the ventrolateral occipitotemporal cortex is "necessary for decisions that reduce the probability of donating", matches the reversed sentence, not the Results. If parietal lesions increase donations, as the Abstract says, the standard lesion-deficit inference is that the intact parietal region restrains giving, the opposite of facilitating it. As printed, the paper's conclusion about what these regions do and its headline result point in opposite directions, and a reader citing either one cannot tell from the text which is intended.
- The laterality also changes. The subsection calls the parietal effect left-sided. The Abstract and Discussion summary say "bilateral", and Table 3 lists the precuneus region with 12 right-hemisphere and 11 left-hemisphere subjects.
- What does not change. No statistic, p-value or effect size depends on these sentences. The paper's broader conclusion, that "altruistic punishment and donation are determined by largely non-overlapping cerebral regions", holds whichever direction is correct. The punishment findings are stated consistently in the Abstract, the Results and the Fig. 2 legend.
Explanations considered
- A different sign convention in the subsection. The reversed sentence uses the same wording as the Results ("Damage to ... decreased donations"), with the same outcome (number of donations), the same patients and the same analysis. No passage introduces a convention under which the words would change meaning, such as describing intact tissue or a deficit score. Does not account for it.
- An isolated slip in one sentence. If only one sentence were wrong, the closing interpretation in the same paragraph would be expected to follow the Results. It follows the reversed sentence instead: "facilitation of donations" by the superomedial parietal cortex implies that parietal damage lowers donations. Two statements in the subsection therefore share the reversed direction. Does not account for it as a one-word typo, although the problem is confined to this subsection.
- The figures settle which version is right. Read with their legends (red = decreased, blue = increased), Figs 2 and 3 agree with the Abstract and Results. For punishment the legend code also matches the Results text: right perisylvian clusters are red ("decreased punishment") and dorsomedial prefrontal clusters blue ("increased"). The legends of Supplementary Figs 1 and 2 use the same code ("red and blue indicate less and more, respectively"; "increased donations (blue) and decreased donations (red)"). However, the colour bars of Figs 2 and 3 are labelled "Beta" and run from red at +10 to blue at −10, and the Methods say the sign of the maps reflects whether the score "is positively or negatively correlated with the presence of a brain lesion". Read literally, that makes red mean "lesion associated with more donations", which is the opposite of the legend. On that reading the Discussion sentence would match the maps, and the Abstract, Results and legends would be reversed for donations and punishment alike. The paper does not say whether the displayed β was sign-flipped. The figures do not independently settle the question. All the text and every legend point one way; only the colour-bar label, combined with the Methods' sign definition, points the other.
- Table 2 settles it. The subgroup means fit their labels (D+ $20.0, D− $9.7, controls $12.9), but Table 2 does not say which regions define D+ and D−, and Table 3 lists the donation regions without a direction. Neither table resolves it.
- A conversion artefact in the PMC copy. The publisher's HTML and typeset PDF and the Europe PMC full-text XML all contain the same sentences. Only words are involved, so minus signs, superscripts and table alignment play no part. Does not account for it.
- For the Table 2 side observation: rounding or differing denominators. One-decimal rounding can move a three-term sum by at most 0.15, or by up to 0.6 when a savings mean is printed as a whole number. The P− (55.5) and D− (58.0) columns are outside this range. The sums could differ from 60 if some participants were missing one of the three measures, so that the means were taken over different sets of people, but the paper reports a single n per subgroup and does not mention missing data. A transcription error in one cell per affected column would also explain it; the paper gives no way to tell which cell. Not resolved by the published material.
Status
As of 2026-10-23:
- Version of record. The publisher's page (doi:10.1093/brain/awy064, which resolves to academic.oup.com/brain/article/141/5/1558/4953518) and the publisher-typeset PDF ("BRAIN 2018: 141; 1558–1569") contain the same Abstract finding (ii), Results > Donation text, Discussion summary, reversed Discussion sentence and closing interpretation as the PMC archive. Table 2 in the typeset PDF has the same values. The Europe PMC full-text XML has identical wording. The discrepancy is in the version of record.
- Supplements. The single supplementary file (task instructions, list of organisations, Supplementary Tables 1–2, legends of Supplementary Figs 1–2) does not describe the lesion–donation directions in words beyond the subgroup labels ("D + : lesions increased donations"; "D ‒ : lesions decreased donations"). Its figure legends use the same colour code as the main figures. The instructions confirm that payout equals $60 minus donations and punishments.
- Corrections. Crossref lists no update relations. PubMed Central and Europe PMC list no correction, erratum or retraction notices. The publisher's page shows none.
- PubPeer and the web. The PubPeer search page returned HTTP 403 (checked again on 2026-10-23), so PubPeer could not be checked directly. Web searches made during this audit for the title or DOI combined with "PubPeer", "correction", "erratum" or "corrigendum" returned only the article itself, institutional listings and press coverage. A further search could not be run on 2026-10-23 because the search quota was exhausted.
- Sources. PMC7341482 (archived full text, figure images); publisher HTML and typeset PDF (version of record); Europe PMC full-text XML; Crossref metadata; the supplementary PDF (awy064_brain-2017-01334-file008.pdf).
Suggested correction
Ask the authors to state the sign convention used to display the β-maps in Figs 2 and 3, check it against the underlying maps, and correct whichever passages disagree with it.
If the Abstract, Results and figure legends are correct, the Discussion subsection "Brain regions implicated in donations" could read, for example: "Damage to the medial parietal lobe (bilaterally) increased donations, whereas damage to the right posterior middle temporal gyrus decreased donations." The closing interpretation would then need to be revised to match: on the usual lesion-deficit reading, the intact superomedial parietal cortex would be associated with restraining donations, and the right ventrolateral occipitotemporal cortex with supporting them.
If instead the colour bars are correct and the legends reversed, the Abstract (findings i and ii), the Results, the Discussion summary and the figure legends would all need correcting.
Separately, the authors could check the P−, D−, D0 and S+ columns of Table 2, whose mean donations, punishments and savings should sum to $60.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC7341482 and regenerate the figures.
Run: python recompute.py.txt
(writes fig1.svg/png and fig2.svg/png next to this file)
Paper: "Altruistic decisions following penetrating traumatic brain injury",
Brain 2018;141(5):1558-1569, doi:10.1093/brain/awy064, PMC7341482.
All inputs are transcribed from the article (Abstract, Results > Donation,
Discussion, Tables 2 and 3, legends of Figs 2 and 3) and from its single
supplementary file (task instructions, Supplementary Table 2 and figure legends).
Optional parts, skipped with a message if unavailable:
part 0 checks every quote used in the write-up verbatim against the PMC archive
(needs the papers repository and its src/show.py);
part 4 downloads the paper's Fig. 2 and Fig. 3 images from the PMC CDN into a
temporary directory, checks their MD5, and counts strongly red and strongly
blue pixels in a few panels. Nothing from the images is saved or redrawn.
"""
import hashlib
import os
import subprocess
import tempfile
import textwrap
import urllib.request
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------------------
# 0. Quotes used in writeup.md, checked verbatim against the PMC archive
# ---------------------------------------------------------------------------
QUOTES = {
"Abstract (ii)":
"altruistic donations were increased by bilateral lesions of the dorsomedial "
"parietal cortex, whereas lesions of the right posterior superior temporal "
"sulcus and middle temporal gyri decreased donations",
"Results > Donation (1)":
"Lesions that led to increased donations were restricted to the paracentral "
"lobule and precuneus, as well as to small areas in the right dorsomedial "
"prefrontal cortex (Fig. 3).",
"Results > Donation (2)":
"Injuries leading to a decrease in donations were restricted to small sectors "
"of the right ventrolateral occipital cortex and posterior middle temporal "
"gyrus (Fig. 3).",
"Discussion summary (ii)":
"altruistic (costly) donations were increased by bilateral lesions of the "
"dorsomedial parietal cortex, whereas lesions of the right posterior superior "
"temporal sulcus and middle temporal gyri had the opposite effect",
"Fig. 3 legend":
"Statistical β-maps of lesions that decreased (red) or increased (blue) "
"donations on the Altruistic Decision Task.",
"Discussion > donations, sentence 3":
"Damage to the left medial parietal lobe decreased donations, whereas damage "
"to the right middle temporal gyrus increased donations.",
"Discussion > donations, closing sentence":
"Overall, our findings suggest that the superomedial parietal cortex is "
"somehow involved in the facilitation of donations, whereas the ventrolateral "
"occipitotemporal cortex is necessary for decisions that reduce the "
"probability of donating.",
"Methods, sign of maps":
"each with a positive or a negative sign (i.e. the variable of interest is "
"positively or negatively correlated with the presence of a brain lesion at a "
"specific location",
"Methods, task":
"each participant had the opportunity to keep a maximum of $60, donate a "
"maximum of $30, or spend a maximum of $30 in punishments",
"Results > Punishment":
"In the right hemisphere, the lesions leading to decreased punishment followed "
"a perisylvian distribution",
"Results > Punishment (increase)":
"Lesions that increased punishments were found (i) bilaterally in the "
"dorsomedial prefrontal cortex",
}
REPO = "/home/ubuntu/papers"
PY = "/home/ubuntu/papers-data/venv/bin/python"
print("== 0. Verbatim check of quotes against the PMC archive ==")
try:
full = subprocess.run([PY, "src/show.py", "PMC7341482", "--full"], cwd=REPO,
capture_output=True, text=True, timeout=120).stdout
if not full:
raise RuntimeError("show.py returned nothing")
for k, q in QUOTES.items():
print(f" {k:42s} {'found verbatim' if q in full else 'NOT FOUND'}")
except Exception as e: # repository not available
print(f" skipped ({e})")
# ---------------------------------------------------------------------------
# 1. Direction of the lesion effect on donations, passage by passage
# "+" = lesion increased donations, "-" = lesion decreased donations
# ---------------------------------------------------------------------------
PAR = "medial parietal"
TMP = "right posterior temporal"
passages = [
# (location, kind, {region: sign}, parietal laterality)
("Abstract, finding (ii)", "stated", {PAR: "+", TMP: "-"}, "bilateral"),
("Results > Donation", "stated", {PAR: "+", TMP: "-"}, "not stated"),
("Discussion, opening summary (ii)", "stated", {PAR: "+", TMP: "-"}, "bilateral"),
("Fig. 3 legend read with map colours", "stated", {PAR: "+", TMP: "-"}, "n/a"),
("Discussion > donations, sentence 3", "stated", {PAR: "-", TMP: "+"}, "left"),
("Discussion > donations, closing sentence", "implied (lesion-deficit reading)",
{PAR: "-", TMP: "+"}, "n/a"),
]
print("\n== 1. Direction of lesion effect on donations, by passage ==")
ref = passages[1][2] # Results > Donation
n_agree = n_rev = 0
for loc, kind, d, lat in passages:
flips = [r for r in d if d[r] != ref[r]]
tag = ("REVERSES Results for both regions" if len(flips) == 2 else
"agrees with Results" if not flips else f"reverses {flips}")
n_agree += not flips
n_rev += len(flips) == 2
print(f" {loc:42s} parietal {d[PAR]} temporal {d[TMP]} "
f"parietal side: {lat:10s} [{kind}] -> {tag}")
print(f" passages agreeing with Results: {n_agree} (including Results itself); "
f"reversing both regions: {n_rev}")
# ---------------------------------------------------------------------------
# 2. Table 2 (lesion subgroups)
# Methods and supplementary instructions: $60 to start, 30 donate and 30 punish
# opportunities at $1 each; payout = $60 minus donations minus punishments.
# So for each participant D + P + S = 60, and the same holds for subgroup means
# computed over the same participants.
# ---------------------------------------------------------------------------
T2 = { # (donations, punishments, savings, n) as printed
"P0": ("13.2", "3.8", "42.9", 18), "P+": ("16.1", "6.6", "37.3", 34),
"P-": ("13.5", "2.6", "39.4", 22), "P±": ("14.1", "4.0", "41.8", 20),
"D0": ("14.7", "4.6", "40", 43), "D+": ("20.0", "4.7", "35.2", 23),
"D-": ("9.7", "4.5", "43.8", 28),
"S0": ("14.3", "4.4", "41", 32), "S+": ("11.4", "2.6", "45", 25),
"S-": ("17.9", "6.7", "35", 32), "S±": ("14.2", "2.4", "43.4", 5),
"Controls": ("12.9", "5.8", "41.2", 28),
}
N_PATIENTS = 94
def halfwidth(s):
"""Largest rounding error of a printed value (0.05 for one decimal, 0.5 for none)."""
return 0.5 * 10 ** (-(len(s.split(".")[1]) if "." in s else 0))
print("\n== 2. Table 2 ==")
for g in "PDS":
ks = [k for k in T2 if k[0] == g]
print(f" {g} subgroups: n = " + " + ".join(str(T2[k][3]) for k in ks)
+ f" = {sum(T2[k][3] for k in ks)} (patients: {N_PATIENTS})")
d_plus, d_minus, ctl = (float(T2[k][0]) for k in ("D+", "D-", "Controls"))
print(f" mean donations D+ {d_plus:.1f} D- {d_minus:.1f} controls {ctl:.1f}; "
f"D+ minus controls = {d_plus - ctl:+.1f}, D- minus controls = {d_minus - ctl:+.1f}")
print(" mean donations + punishments + savings (each participant's total is 60):")
offenders = []
for k, (d, p, s, n) in T2.items():
tot = float(d) + float(p) + float(s)
hw = halfwidth(d) + halfwidth(p) + halfwidth(s)
ok = tot - hw <= 60 + 1e-9 <= tot + hw + 2e-9
if not ok:
offenders.append((k, tot, tot - hw, tot + hw))
print(f" {k:9s} n={n:3d} {d:>5s} + {p:>4s} + {s:>5s} = {tot:5.1f} "
f"(range under rounding {tot - hw:.2f} to {tot + hw:.2f}) "
f"{'consistent with 60' if ok else 'NOT consistent with 60'}")
print(" columns whose sum cannot be 60 under rounding: "
+ "; ".join(f"{k} {t:.1f} ({lo:.2f}-{hi:.2f})" for k, t, lo, hi in offenders))
# ---------------------------------------------------------------------------
# 3. Table 3 (donation rows), as printed; laterality of the parietal effect
# ---------------------------------------------------------------------------
T3_DON = [("Middle temporal sulcus (posterior half)", "R", "37", "6"),
("Dorsomedial parietal cortex (precuneus)", "R/L", "5/7m", "12/11"),
("Medial orbitofrontal/gyrus rectus", "R/L", "11", "6/7"),
("Inferior frontal gyrus", "L", "45/46/47", "17")]
print("\n== 3. Table 3, donation rows ==")
for r in T3_DON:
print(f" {r[0]:42s} side {r[1]:4s} BA {r[2]:9s} subjects (R/L) {r[3]}")
prec_r, prec_l = map(int, T3_DON[1][3].split("/"))
print(f" precuneus: {prec_r} right + {prec_l} left subjects = {prec_r + prec_l}; "
"listed for both hemispheres, not left only")
print(" Table 3 gives no direction (increase/decrease) for any donation region")
# ---------------------------------------------------------------------------
# 4. Optional: colour check of the paper's Fig. 2 and Fig. 3 (not redrawn)
# ---------------------------------------------------------------------------
FIGS = {
"f2": ("https://cdn.ncbi.nlm.nih.gov/pmc/blobs/e84c/7341482/bd9cea49707e/awy064f2.jpg",
"bd9cea49707e56866080625dd2ce2009"),
"f3": ("https://cdn.ncbi.nlm.nih.gov/pmc/blobs/e84c/7341482/af8bc4065be1/awy064f3.jpg",
"af8bc4065be1d710719d4406426873f8"),
}
# panel boxes (x0, x1, y0, y1) in pixels of the 764 x 512 PMC images
BOX3 = {"A medial parietal (precuneus/SPL)": (40, 260, 60, 270),
"B medial (precuneus/dmPFC)": (255, 480, 0, 160),
"G medial": (490, 690, 300, 512),
"E lateral (occipital/pMTG)": (0, 255, 300, 512),
"F lateral (occipital/pMTG)": (255, 490, 300, 512)}
BOX2 = {"A right lateral perisylvian": (0, 240, 20, 215),
"E/F/G medial dmPFC": (250, 700, 215, 512)}
def counts(img, box):
x0, x1, y0, y1 = box
a = img[y0:y1, x0:x1, :3].astype(float)
if a.max() <= 1:
a = a * 255
R, G, B = a[..., 0], a[..., 1], a[..., 2]
red = ((R - B) > 60) & ((R - G) > 50)
blue = ((B - R) > 60) & (B > 110)
return int(red.sum()), int(blue.sum())
print("\n== 4. Colour check of the paper's figures "
"(legends: red = decreased, blue = increased) ==")
try:
from matplotlib.image import imread
imgs = {}
with tempfile.TemporaryDirectory() as td:
for k, (url, md5) in FIGS.items():
req = urllib.request.Request(url, headers={"User-Agent": "Mozilla/5.0"})
data = urllib.request.urlopen(req, timeout=30).read()
if hashlib.md5(data).hexdigest() != md5:
raise RuntimeError(f"{k}: image differs from the one checked")
path = os.path.join(td, k + ".jpg")
with open(path, "wb") as fh:
fh.write(data)
imgs[k] = imread(path)
print(" Fig. 3 (donations):")
for k, b in BOX3.items():
r, bl = counts(imgs["f3"], b)
print(f" {k:36s} red {r:5d} blue {bl:5d}")
print(" Fig. 2 (punishment), legend convention checked against Results > Punishment:")
for k, b in BOX2.items():
r, bl = counts(imgs["f2"], b)
print(f" {k:36s} red {r:5d} blue {bl:5d}")
col = imgs["f3"][:, 722, :3].astype(int) # colour bar column of Fig. 3
top, bot = tuple(int(v) for v in col[60]), tuple(int(v) for v in col[460])
print(f" Fig. 3 colour bar (labelled 'Beta'): end labelled 10 has RGB {top} (red); "
f"end labelled -10 has RGB {bot} (blue)")
except Exception as e: # network or image problems: skip this optional part
print(f" skipped ({e})")
# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#d9d8d4"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"text.color": TXT, "svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- fig1: two text panels joined by an annotated connector ---------------
W, H = 7.2, 5.0
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.axis("off")
PX, PW, PY0, PH = (0.12, 3.78), 3.3, 1.55, 3.33
def panel(x, color, title, sub, blocks):
ax.add_patch(FancyBboxPatch((x, PY0), PW, PH,
boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=color, lw=2))
ax.text(x + 0.15, PY0 + PH - 0.14, title, fontsize=11, fontweight="bold",
color=color, va="top")
ax.text(x + 0.15, PY0 + PH - 0.42, sub, fontsize=10, color=SEC, va="top")
yy = PY0 + PH - 0.80
for head, body in blocks:
ax.text(x + 0.15, yy, head, fontsize=10, color=SEC, va="top", style="italic")
yy -= 0.23
wrapped = textwrap.fill(body, 40)
ax.text(x + 0.15, yy, wrapped, fontsize=10, color=TXT, va="top",
linespacing=1.3)
yy -= 0.205 * (wrapped.count("\n") + 1) + 0.2
left = [("Abstract, finding (ii)",
"“...donations were increased by bilateral lesions of the dorsomedial "
"parietal cortex, whereas lesions of the right posterior superior temporal "
"sulcus and middle temporal gyri decreased donations”"),
("Results > Donation",
"“Lesions that led to increased donations were restricted to the "
"paracentral lobule and precuneus ...”")]
right = [("Paragraph 1, third sentence",
"“Damage to the left medial parietal lobe decreased donations, whereas "
"damage to the right middle temporal gyrus increased donations.”"),
("Same paragraph, closing sentence",
"“...the superomedial parietal cortex is somehow involved in the "
"facilitation of donations ...”")]
panel(PX[0], BLUE, "Abstract and Results",
"(also Discussion summary, Fig. 3 legend)", left)
panel(PX[1], ORANGE, "Discussion subsection",
"“Brain regions implicated in donations”", right)
# connector: a bracket from the bottom of each panel to a labelled box
lx, rx = PX[0] + PW / 2, PX[1] + PW / 2
ybot, ymid = PY0 - 0.03, 1.12
ax.plot([lx, lx, rx, rx], [ybot, ymid, ymid, ybot], color=TXT, lw=2,
solid_joinstyle="miter", zorder=1)
ax.add_patch(FancyBboxPatch((W / 2 - 1.85, 0.86), 3.7, 0.52,
boxstyle="round,pad=0.02,rounding_size=0.06",
fc=BG, ec=TXT, lw=2, zorder=2))
ax.text(W / 2, 1.12, "Conflict: opposite direction for both regions",
ha="center", va="center", fontsize=10.5, fontweight="bold", color=TXT, zorder=3)
rows = [("medial parietal lesions:", "increased", "decreased"),
("right MTG lesions:", "decreased", "increased"),
("parietal laterality:", "“bilateral”", "“left”")]
for i, (lab, a, b) in enumerate(rows):
y = 0.55 - 0.21 * i
ax.text(2.85, y, lab, ha="right", va="center", fontsize=10, color=SEC)
ax.text(2.95, y, a, ha="left", va="center", fontsize=10, color=BLUE,
fontweight="bold")
ax.text(4.0, y, "→", ha="center", va="center", fontsize=10, color=SEC)
ax.text(4.15, y, b, ha="left", va="center", fontsize=10, color=ORANGE,
fontweight="bold")
save(fig, "fig1")
# ---- fig2: direction grid, passage x region --------------------------------
rows = [
("Abstract, finding (ii)", "+", "-", "stated"),
("Results > Donation", "+", "-", "stated"),
("Discussion, opening summary (ii)", "+", "-", "stated"),
("Fig. 3 legend + map colours", "+", "-", "stated"),
("Discussion > donations, sentence 3", "-", "+", "stated"),
("Same paragraph, closing sentence*", "-", "+", "implied"),
("Fig. 3 colour bar, if +β = more†", "-", "+", "conditional"),
]
nr = len(rows)
fig, ax = plt.subplots(figsize=(7.2, 5.4), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 7.2)
ax.set_ylim(-3.95, nr + 0.85)
ax.axis("off")
cols = {1: (3.85, "Medial parietal\n(precuneus/paracentral)"),
2: (5.9, "Right posterior MTG/\npSTS/occipital")}
ax.text(0.08, nr + 0.35, "Passage", va="center", fontsize=10, fontweight="bold")
for _, (cx, lab) in cols.items():
ax.text(cx, nr + 0.35, lab, ha="center", va="center", fontsize=10,
fontweight="bold")
ax.plot([0.05, 7.15], [nr - 0.5] * 2, color=SEC, lw=1)
for i, (lab, a, b, kind) in enumerate(rows):
y = nr - 1 - i
ax.plot([0.05, 7.15], [y - 0.5] * 2, color=GRID, lw=0.8)
ax.text(0.08, y, lab, va="center", fontsize=10,
color=TXT if kind == "stated" else SEC,
style="normal" if kind == "stated" else "italic")
for j, s in ((1, a), (2, b)):
cx = cols[j][0]
color = BLUE if s == rows[1][j] else ORANGE
if kind == "conditional":
color = SEC
word = "▲ increased" if s == "+" else "▼ decreased"
ax.text(cx, y, word, ha="center", va="center", fontsize=10.5,
color=color, fontweight="bold" if kind == "stated" else "normal")
leg = [(BLUE, "same direction as Abstract and Results"),
(ORANGE, "reverse direction"),
(SEC, "depends on a sign convention the paper does not state")]
for k, (c, t) in enumerate(leg):
yy = -0.95 - 0.42 * k
ax.plot([0.15], [yy], marker="s", ms=9, color=c, ls="none")
ax.text(0.32, yy, t, va="center", fontsize=10, color=TXT)
foot = ("* Lesion-deficit reading: a region that “facilitates” donations "
"should lower them when damaged.\n"
"† The colour bar, labelled “Beta”, runs from red (+10) to blue "
"(−10). The Methods\n define the sign as a positive or negative "
"correlation with lesion presence;\n the legend says red = decreased.")
ax.text(0.08, -2.25, foot, fontsize=10, color=SEC, va="top", linespacing=1.35)
fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.01)
save(fig, "fig2")
print("\nfigures written: fig1.svg/png, fig2.svg/png in", HERE)
By the paper's own model, odds of death rise 2.7-fold per decade of age, not 11-fold
Brain (2022) · doi:10.1093/brain/awab466 · PMC9336587
Summary
The abstract and Results state that ageing by 10 years increased the odds of death in intensive care after traumatic brain injury 11-fold. The same Results sentence gives the model's estimate as an odds ratio of 1.10 per year of age (95% CI 1.05–1.16), and the authors' R output in the supplement gives the coefficient behind it: 0.09914 log-odds per year. In a logistic model, odds ratios compound over years, so 10 years corresponds to exp(10 × 0.09914) = 2.70 (95% CI 1.59–4.57). The 11-fold figure, which lies far outside that interval, equals 10 × 1.104: the per-year odds ratio multiplied by 10 rather than raised to the 10th power. The direction and significance of the age effect are unchanged, but the abstract overstates its size about fourfold, and Fig. 5D plots the same overstated value.
What the paper reports
Abstract:
A warmer mean brain temperature was associated with survival (P = 0.035), however, and ageing by 10 years increased the odds of death 11-fold (P = 0.0002).
Results, section "Daily brain temperature rhythms predict patient survival":
Applying a generalized linear mixed model (Fig. 5), we found that lack of a daily TBr rhythm, or an age increase of 10 years, increased the odds of death in intensive care 21-fold and 11-fold, respectively (odds ratio for death with rhythm 0.05; 95% CI 0.004–0.57, P = 0.016 and odds ratio for death with ageing by 1 year 1.10; 1.05–1.16, P = 0.0002).
Materials and methods, "Statistical analysis", the model behind both statements:
For the final analysis of patient data, a generalized linear mixed model for logit binomial distribution of patient outcome was chosen
where fixed effects include: Age (patient age in intensive care);
Fig. 5 legend:
(D) Odds of death in intensive care transformed from the data in A–C; the results for these three predictors are significant since the 95% CIs (double-ended error bars) do not include 1.
Fig. 5D itself (an image, read from the archived file awab466f5.jpg) has a row labelled "Age (per decade)" on a logarithmic "Odds of death" axis. Its marker sits at about 11, with a narrow interval.
Supplementary File016 (the authors' R code with its output) fits glmer(Outcome ~ Age +Sex + BrainMean +BrainRange +Daily + (1|Subject), ..., family=binomial) on 98 patients. Its data listing describes Age as "Patient age in years" (integer). It reports:
Age 0.09914 0.02696 3.677 0.000236 ***
exp(0.09914) # calculating Age odds for death [1] 1.104221
exp(c(0.0462984, 0.1519816)) [1] 1.047387 1.164139
What the paper's own numbers show
Method. In a logit-link model, the coefficient for age, β, is the change in log-odds per year. The odds ratio for a k-year difference is therefore exp(k × β) = (per-year odds ratio)^k, and its Wald 95% CI is exp(k × (β ± 1.96 × SE)). Rescaling age from years to decades multiplies β and SE by the same factor, so z and P do not change. I used the authors' unrounded coefficient and SE from Supplementary File016. As a check, I also used the rounded values printed in the Results, and the two agree. To read Fig. 5D, I calibrated its log x-axis from the six major tick marks on the axis line (80.07 px per decade; largest residual 0.24 px; the minor ticks at 2 and 3 fall within 0.2 px of where a log scale puts them). I then took the centre of the Age ring marker and its two interval end caps. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Odds ratio per 1 year of age | Results | 1.10 (1.05–1.16), P = 0.0002 | exp(0.09914) = 1.104 (1.047–1.164); z = 3.677, P = 0.000236 (File016) |
| Internal consistency of the rounded per-year figures | Results | 1.10 (1.05–1.16), P = 0.0002 | P implied by the CI = 0.00018: consistent |
| Odds ratio per 10 years of age | Abstract; Results | 11-fold | exp(10 × 0.09914) = 2.70 |
| 95% CI per 10 years | not reported | — | 1.59–4.57 (from the rounded text: 1.63–4.41) |
| Per 10 years, from the rounded 1.10 | Results | — | 1.10^10 = 2.59; any value from 1.095 to 1.105 gives 2.48–2.71 |
| Per-year odds ratio that 11-fold over 10 years would need | derived | — | 11^(1/10) = 1.271, i.e. β = 0.240, 5.2 SE above the fitted β |
| Per-year odds ratio multiplied by 10 | derived | — | 1.10 × 10 = 11.00; 10 × 1.104 = 11.04 |
| Per-year CI multiplied by 10 | derived | — | 10.47–11.64 |
| "Age (per decade)" marker and interval | Fig. 5D | plotted, no number given | ≈11.1 (10.4–11.7) digitized; matches 10 × the per-year values, not 2.70 (1.59–4.57) |
| P per decade | Abstract | 0.0002 | unchanged by rescaling (z = 3.677): consistent |
| Stated / recomputed per 10 years | — | — | 11 / 2.70 = 4.1 (4.2 using 1.10^10) |
| Lack of a daily rhythm, fold increase in odds | Abstract; Results | 21-fold (OR 0.05; 0.004–0.57; P = 0.016) | 1/exp(−3.03024) = 20.7 (95% CI 1.76–243); P from the CI = 0.016: consistent |
| Mean brain temperature, OR per 1 °C | Results | 0.42 (0.18–0.94), P = 0.035 | exp(−0.87808) = 0.416, P = 0.035 (File016): consistent |
Only the 11-fold age figure, in the text and in Fig. 5D, disagrees with the model output. Every other odds ratio, interval and P value in this paragraph reproduces.
Figure
Figure 1. Odds ratios for death in intensive care on one logarithmic axis; the thin vertical line marks no effect (1). Orange: values the paper reports. These are the per-year odds ratio in the Results (filled; its interval of 1.05–1.16 is narrower than the marker), the "11-fold" per 10 years in the Abstract and Results (filled; no interval is given), and the "Age (per decade)" marker in Fig. 5D as digitized from the archived image (open ring, ≈11 with interval 10.4–11.7). Blue: the 10-year odds ratio recomputed from the authors' own coefficient, exp(10 × 0.09914) = 2.70, with its 95% CI of 1.59–4.57. The printed values lie well outside the recomputed interval.
Figure 2. The arithmetic chain. Both routes start from the same model output and the same per-year odds ratio of 1.104. Compounding over 10 years (blue, left) gives 2.70 (1.59–4.57). Multiplying the per-year odds ratio and its interval by 10 (orange, right) gives 11.04 (10.47–11.64), which matches both the printed "11-fold" and the position and narrow interval of the Fig. 5D marker. All values are printed by recompute.py.txt.
Why it matters
- The abstract states the wrong magnitude. Age is one of the three outcome predictors whose effects the abstract reports. A reader of the abstract alone will take away an 11-fold rise in the odds of death per decade of age. The paper's model supports about 2.7-fold (95% CI 1.6–4.6).
- Fig. 5D shows the same overstated value. The legend presents panel D as the odds of death "transformed from the data in A–C". The age row is plotted at about 11 with an interval of roughly 10.4–11.7, both 10 times the per-year values. Its interval is therefore also misstated (correct: 1.59–4.57), although it still excludes 1.
- The comparison between predictors changes. As printed, lack of a daily rhythm (21-fold) and a decade of age (11-fold) look similar in size, a ratio of about 1.9. With the corrected age figure the ratio is 20.7 / 2.70 = 7.7. The paper's description of the rhythm as "one of the strongest single predictors of survival" is, if anything, strengthened.
- What is unaffected. The direction of the age effect (older patients at higher risk), its significance (P = 0.0002; rescaling does not change the test), the per-year odds ratio, the rhythm result in the title and the mean-temperature result are all unaffected. The statement that "Older TBI patients lacking a daily TBr rhythm are at greatest risk of death" still holds qualitatively.
Explanations considered
- Age entered per decade, or in other units. Not supported. The supplementary R listing describes
Ageas "Patient age in years" (integer values such as 57, 65, 62), the Methods define it as "patient age in intensive care" with no rescaling, and the Results label 1.10 as the odds ratio "with ageing by 1 year". If 1.10 were per decade, the 10-year figure would be 1.10, not 11. - Rounding of 1.10. Not able to explain it. The unrounded estimate, 1.104, gives 2.70. Every per-year value that rounds to 1.10 gives 2.48–2.71, and even the upper confidence limit, 1.164, gives only 4.57 over 10 years. Reaching 11 needs 1.271 per year, 5.2 standard errors above the estimate.
- The per-year estimate is the wrong number. Not supported. The per-year odds ratio, its CI and its P agree with each other (P from the rounded CI = 0.00018, against 0.0002 reported) and agree exactly with the R output in File016 (exp(0.09914) = 1.104221; CI 1.047387–1.164139; P = 0.000236).
- Probability rather than odds, or a different contrast such as oldest versus youngest. Not supported. Both statements say "odds" and "10 years", and Fig. 5D says "Age (per decade)". No other contrast is described.
- A different model or population. Not supported. The abstract, the Results sentence and Fig. 5D all refer to the same generalized linear mixed model on 98 patients, with the same P = 0.0002.
- A conversion or typesetting artifact. Not possible here. The publisher's version of record carries the same wording in the Abstract and Results, and the figure image plots the same value.
- Other fold changes computed the same way. No. The companion "21-fold" for lack of a rhythm was obtained correctly: File016 shows 1/0.04830404 = 20.7022. Only the age conversion reproduces as 10 × the per-year odds ratio.
- A correction already issued. None found (see Status).
Status
- Version of record. The DOI resolves to the publisher's article page (academic.oup.com/brain/article/145/6/2031/6604351). It carries the same Abstract sentence ("ageing by 10 years increased the odds of death 11-fold") and the same Results sentence and numbers. The PMC HTML page and the Europe PMC full-text XML match. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. The Fig. 5D values were read from the published figure imageawab466f5.jpg. - Supplements. I downloaded the Europe PMC supplementary bundle (
awab466_Supplementary_Data.zip). It contains videos, the participant information sheet, the study, scan and case-report protocols, File016 (analytic R code with model output), File017 (the 30-page Supplementary Material), File018 (a spreadsheet of rhythm analyses) and an investigator list. File016 contains the outcome model's coefficient table and the authors' own exp() conversions quoted above. Neither File016 nor File017 mentions "10 years", "decade" or "11-fold". The supplement does not resolve the discrepancy in favour of 11-fold; it confirms the per-year coefficient from which 2.70 follows. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturned no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by relations. The PubMed record (PMID 35691613) lists no linked comments or errata, and the publisher page shows no notice. Retraction Watch has no entry for the paper, and a web search found no erratum or PubPeer thread. PubPeer's site returned HTTP 403 to automated requests, and the Europe PMC search endpoint returned HTTP 503 on four attempts, so those two checks are incomplete. A web search also found the medRxiv preprint (doi:10.1101/2021.01.23.21250327), which reportedly gives the same 11-fold figure; I did not check its wording myself. - Sources. PMC article PMC9336587; Europe PMC full-text XML and supplementary files for PMC9336587 (File016
brain-2021-00914-File016.pdf, File017brain-2021-00914-File017.pdf); figure imageawab466f5.jpg; Crossref; PubMed; Retraction Watch; PubPeer; https://doi.org/10.1093/brain/awab466.
Suggested correction
In the Abstract, replace
ageing by 10 years increased the odds of death 11-fold (P = 0.0002)
with
ageing by 10 years increased the odds of death 2.7-fold (P = 0.0002)
In the Results, replace "increased the odds of death in intensive care 21-fold and 11-fold, respectively" with "increased the odds of death in intensive care 21-fold and 2.7-fold, respectively", and optionally add the 10-year interval (95% CI 1.6–4.6). In Fig. 5D, either plot the "Age (per decade)" row at 2.70 (1.59–4.57), or relabel the row "per year" and plot 1.10 (1.05–1.16). The per-year odds ratio, its confidence interval and all P values need no change.
Recomputation script (python recompute.py.txt)
"""Recompute the age effect on death in intensive care in PMC9336587
(Brain 2022, doi:10.1093/brain/awab466).
Run: python recompute.py.txt (uses the numbers and pixel positions recorded below)
python recompute.py.txt <awab466f5.jpg> (also re-measures Fig. 5D from the archived image)
Needs numpy, scipy and matplotlib (plus Pillow for the optional re-measurement).
Prints every recomputed number used in writeup.md and writes fig1.svg/png and
fig2.svg/png next to this file.
Inputs
* Results text (verbatim): odds ratio for death with ageing by 1 year 1.10; 1.05-1.16,
P = 0.0002; odds ratio for death with rhythm 0.05; 95% CI 0.004-0.57, P = 0.016;
mean TBr 0.42 for 1 degree C increase, 0.18-0.94, P = 0.035; "11-fold" and "21-fold".
* Supplementary File016 (the authors' R code and output, brain-2021-00914-File016.pdf),
glmer(Outcome ~ Age + Sex + BrainMean + BrainRange + Daily + (1|Subject),
family = binomial), n = 98: Age estimate 0.09914 (SE 0.02696, z 3.677, P 0.000236);
BrainMean -0.87808 (SE 0.41734, P 0.035378); DailyYes -3.03024 (SE 1.25685, P 0.015910).
The str() listing in the same file describes Age as "Patient age in years" (integer).
* Fig. 5D (awab466f5.jpg, 794 x 784 px, as archived in PMC / Europe PMC): log x-axis
"Odds of death", row "Age (per decade)". Major tick marks were located on the axis
line at x = 381, 461.5, 541.5, 621.5, 701.5, 781.5 px (0.001 ... 100; 80 px per
decade; the minor ticks at 2 and 3 sit at 645.5 and 659.5 px, as a log scale
requires). The Age marker is a ring spanning x = 690-720 px with interval end caps
at x = 703 and 707 px.
"""
import os
import sys
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (paper text)
OR1_REP, LO1_REP, HI1_REP, P1_REP = 1.10, 1.05, 1.16, 0.0002 # per 1 year, Results
FOLD10_REP = 11.0 # "11-fold", Abstract + Results
ORR_REP, LOR_REP, HIR_REP, PR_REP = 0.05, 0.004, 0.57, 0.016 # rhythm present, Results
FOLDR_REP = 21.0 # "21-fold"
ORT_REP, LOT_REP, HIT_REP, PT_REP = 0.42, 0.18, 0.94, 0.035 # mean TBr per 1 C, Results
# ------------------------------------------------------------------ inputs (Supplementary File016)
B_AGE, SE_AGE, Z_AGE, P_AGE = 0.09914, 0.02696, 3.677, 0.000236
B_TBR, SE_TBR, P_TBR = -0.87808, 0.41734, 0.035378
B_DAY, SE_DAY, P_DAY = -3.03024, 1.25685, 0.015910
# ------------------------------------------------------------------ inputs (Fig. 5D pixels)
TICK_X = np.array([381.0, 461.5, 541.5, 621.5, 701.5, 781.5])
TICK_V = np.array([-3, -2, -1, 0, 1, 2]) # log10 of 0.001 ... 100
AGE_RING_X = (690.0, 720.0)
AGE_CAP_X = (703.0, 707.0)
# colours / style
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def remeasure(path):
"""Re-locate the Fig. 5D axis ticks and the Age marker in the archived JPEG."""
from PIL import Image
im = np.asarray(Image.open(path).convert("L")).astype(int)
rows = [y for y in range(700, 770) if (im[y, 380:790] < 100).sum() > 200]
print(f" axis line rows: {rows}")
y = rows[-1] + 7
ticks = (np.where(im[y, 375:790] < 120)[0] + 375).tolist()
print(f" dark pixels just below the axis (tick marks), row {y}: {ticks}")
sub = im[648:684, 688:730] < 80 # right of the word "Younger"
cols = np.where(sub.any(axis=0))[0] + 688
print(f" Age marker (ring) dark columns: {cols.min()}-{cols.max()}")
inner = np.where((im[662:674, 698:712] < 80).all(axis=0))[0] + 698
print(f" interval end caps (dark in every row 662-673): {inner.tolist()}")
def main():
out = {}
print("=" * 72)
print("1. Per-year odds ratio for age (Results) and its internal consistency")
b = np.log(OR1_REP)
se = (np.log(HI1_REP) - np.log(LO1_REP)) / (2 * Z)
z = b / se
p = 2 * norm.sf(z)
print(f" from the rounded text: beta = ln(1.10) = {b:.4f}, SE from CI = {se:.4f}, "
f"z = {z:.2f}, P = {p:.5f} (reported P = {P1_REP})")
print(f" geometric midpoint of 1.05-1.16 = {np.sqrt(LO1_REP*HI1_REP):.4f}")
or1 = np.exp(B_AGE)
lo1, hi1 = np.exp(B_AGE - 1.96 * SE_AGE), np.exp(B_AGE + 1.96 * SE_AGE)
z_s = B_AGE / SE_AGE
print(f" from File016: exp(0.09914) = {or1:.6f}; 95% CI exp(0.09914 +/- 1.96*0.02696) = "
f"{lo1:.6f}-{hi1:.6f}")
print(f" File016 z = 0.09914/0.02696 = {z_s:.3f} (printed {Z_AGE}); "
f"P = {2*norm.sf(z_s):.6f} (printed {P_AGE})")
out.update(or1=or1, lo1=lo1, hi1=hi1)
print("=" * 72)
print("2. Odds ratio for 10 years of age under the logit model: OR(10) = exp(10*beta)")
or10_txt = OR1_REP ** 10
print(f" from the rounded text: 1.10^10 = {or10_txt:.3f}; CI 1.05^10-1.16^10 = "
f"{LO1_REP**10:.3f}-{HI1_REP**10:.3f}")
print(f" rounding range of 1.10 (1.095-1.105): {1.095**10:.3f}-{1.105**10:.3f}")
or10 = np.exp(10 * B_AGE)
lo10, hi10 = np.exp(10 * (B_AGE - 1.96 * SE_AGE)), np.exp(10 * (B_AGE + 1.96 * SE_AGE))
print(f" from File016: exp(10 x 0.09914) = {or10:.3f}; 95% CI {lo10:.3f}-{hi10:.3f}")
print(f" Wald z for a per-decade coding: (10*beta)/(10*SE) = {(10*B_AGE)/(10*SE_AGE):.3f} "
"(unchanged)")
need = FOLD10_REP ** 0.1
print(f" per-year OR needed for 11-fold over 10 years: 11^(1/10) = {need:.4f} "
f"(beta {np.log(need):.4f}, i.e. {np.log(need)/B_AGE:.2f} x the fitted beta)")
zn = (np.log(need) - B_AGE) / SE_AGE
print(f" distance of that per-year value from the estimate: {zn:.1f} SE")
print(f" ratio stated / recomputed: 11 / {or10:.3f} = {FOLD10_REP/or10:.2f}; "
f"11 / {or10_txt:.3f} = {FOLD10_REP/or10_txt:.2f}")
out.update(or10=or10, lo10=lo10, hi10=hi10, need=need)
print("=" * 72)
print("3. The arithmetic that reproduces '11-fold'")
m10 = 10 * or1
print(f" 1.10 x 10 = {10*OR1_REP:.2f}; 10 x exp(0.09914) = {m10:.3f}; "
f"10 x CI = {10*lo1:.2f}-{10*hi1:.2f}")
out.update(m10=m10, m10lo=10 * lo1, m10hi=10 * hi1)
print("=" * 72)
print("4. Fig. 5D, row 'Age (per decade)', read from the archived image")
fit = np.polyfit(TICK_X, TICK_V, 1)
resid = TICK_V - np.polyval(fit, TICK_X)
print(f" axis calibration: {1/fit[0]:.2f} px per decade, largest residual "
f"{np.abs(resid).max()*1/fit[0]:.2f} px")
print(f" minor ticks 2 and 3 predicted at {np.polyval([1/fit[0], -fit[1]/fit[0]], np.log10(2)):.1f} "
f"and {np.polyval([1/fit[0], -fit[1]/fit[0]], np.log10(3)):.1f} px (measured 645.5, 659.5)")
cx = np.mean(AGE_RING_X)
v = 10 ** np.polyval(fit, cx)
vlo, vhi = (10 ** np.polyval(fit, x) for x in AGE_CAP_X)
print(f" marker centre x = {cx:.1f} px -> odds ratio {v:.1f}; interval caps "
f"{vlo:.1f}-{vhi:.1f} (+/- 1 px changes the centre by a factor "
f"{10**(fit[0]):.3f})")
print(f" compare 10 x exp(beta) = {m10:.2f} ({10*lo1:.2f}-{10*hi1:.2f}) and "
f"exp(10 beta) = {or10:.2f} ({lo10:.2f}-{hi10:.2f})")
out.update(fig_v=v, fig_lo=vlo, fig_hi=vhi)
print("=" * 72)
print("5. The other fold changes in the same sentence and paragraph")
mid = np.sqrt(LOR_REP * HIR_REP)
seR = (np.log(HIR_REP) - np.log(LOR_REP)) / (2 * Z)
zR = np.log(mid) / seR
print(f" rhythm, text: 1/0.05 = {1/ORR_REP:.1f}; 1/geometric midpoint of CI = {1/mid:.2f}; "
f"P from CI = {2*norm.sf(abs(zR)):.4f} (reported {PR_REP})")
fr = 1 / np.exp(B_DAY)
frlo, frhi = 1 / np.exp(B_DAY + 1.96 * SE_DAY), 1 / np.exp(B_DAY - 1.96 * SE_DAY)
print(f" rhythm, File016: 1/exp(-3.03024) = {fr:.2f} (95% CI {frlo:.2f}-{frhi:.0f}); "
f"stated '21-fold'")
seT = (np.log(HIT_REP) - np.log(LOT_REP)) / (2 * Z)
zT = np.log(ORT_REP) / seT
print(f" mean TBr, text: P from CI = {2*norm.sf(abs(zT)):.4f} (reported {PT_REP}); "
f"File016 exp(-0.87808) = {np.exp(B_TBR):.4f}, P = {P_TBR}")
print(f" ratio of rhythm fold change to age (per decade): stated 21/11 = {21/11:.1f}; "
f"recomputed {fr:.1f}/{or10:.2f} = {fr/or10:.1f}")
out.update(fr=fr)
if len(sys.argv) > 1:
print("=" * 72)
print("Re-measurement of Fig. 5D:")
remeasure(sys.argv[1])
fig1(out)
fig2(out)
print("=" * 72)
print("wrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
def fig1(o):
rows = [
# label, value, lo, hi, colour, filled, value text
("Per 1 year of age\nreported, Results", OR1_REP, LO1_REP, HI1_REP, ORANGE, True,
"1.10 (1.05–1.16)"),
("Per 10 years\nstated, Abstract and Results", FOLD10_REP, None, None, ORANGE, True,
"“11-fold”"),
("Per decade\nplotted, Fig. 5D (digitized)", o["fig_v"], o["fig_lo"], o["fig_hi"], ORANGE,
False, f"≈{o['fig_v']:.0f} ({o['fig_lo']:.1f}–{o['fig_hi']:.1f})"),
("Per 10 years\nrecomputed: exp(10 × 0.09914)", o["or10"], o["lo10"], o["hi10"], BLUE,
True, f"{o['or10']:.2f} ({o['lo10']:.2f}–{o['hi10']:.2f})"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9))
n = len(rows)
for i, (lab, v, lo, hi, c, filled, txt) in enumerate(rows):
y = n - 1 - i
if lo is not None:
ax.plot([lo, hi], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=2)
for e in (lo, hi):
ax.plot([e, e], [y - 0.09, y + 0.09], color=c, lw=2, zorder=2)
ax.plot(v, y, "o", ms=9, mew=2, color=c, mfc=c if filled else BG, zorder=3)
xt = (hi if hi is not None else v) * 1.35
ax.text(xt, y, txt, va="center", ha="left", color=c if c == BLUE else TXT, fontsize=10.5)
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.text(0.97, -0.5, "no effect", ha="right", va="center", color=SEC, fontsize=10)
ax.set_xscale("log")
ax.set_xlim(0.45, 100)
ax.set_ylim(-0.6, n - 0.25)
ticks = [0.5, 1, 2, 5, 10, 20, 50, 100]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.set_yticks(range(n))
ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10.5)
for t, r in zip(ax.get_yticklabels(), rows[::-1]):
t.set_color(r[4] if r[4] == BLUE else TXT)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_xlabel("Odds ratio for death in intensive care (log scale)")
# legend by marker style
from matplotlib.lines import Line2D
h = [Line2D([], [], marker="o", ls="-", lw=2, ms=9, color=ORANGE, label="reported by the paper"),
Line2D([], [], marker="o", ls="-", lw=2, ms=9, color=BLUE,
label="recomputed from the paper's own model output")]
fig.tight_layout(rect=(0, 0, 1, 0.9))
fig.legend(handles=h, loc="upper left", bbox_to_anchor=(0.01, 0.995), ncol=2,
frameon=False, fontsize=10, handlelength=2.2)
save(fig, "fig1")
def box(ax, x, y, w, h, text, ec, lw=1.2, fc=BG, tc=TXT, fs=10.5, weight="normal"):
ax.add_patch(FancyBboxPatch((x - w / 2, y - h / 2), w, h,
boxstyle="round,pad=0.02,rounding_size=0.06",
fc=fc, ec=ec, lw=lw))
ax.text(x, y, text, ha="center", va="center", fontsize=fs, color=tc,
linespacing=1.35, weight=weight)
def arrow(ax, x0, y0, x1, y1, c=SEC, lw=1.6):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=14,
color=c, lw=lw, shrinkA=0, shrinkB=0))
def fig2(o):
fig, ax = plt.subplots(figsize=(7.2, 4.5))
ax.set_xlim(0, 10)
ax.set_ylim(0, 6.25)
ax.axis("off")
box(ax, 5, 5.55, 8.4, 0.95,
"Model output (Supplementary File016, n = 98)\n"
f"Age: β = {B_AGE} log-odds per year of age (SE {SE_AGE}, P = {P_AGE})", SEC)
arrow(ax, 5, 5.07, 5, 4.62)
box(ax, 5, 4.15, 8.4, 0.9,
f"Per year: exp(β) = {o['or1']:.3f} (95% CI {o['lo1']:.3f}–{o['hi1']:.3f})\n"
"Results text: “ageing by 1 year 1.10; 1.05–1.16”", SEC)
ax.text(5, 3.33, "Scaling to 10 years:\nthe two routes diverge here",
ha="center", va="center", fontsize=10.5, color=TXT, weight="bold", linespacing=1.3)
arrow(ax, 1.4, 3.7, 1.4, 2.95, c=BLUE, lw=2)
arrow(ax, 8.6, 3.7, 8.6, 2.95, c=ORANGE, lw=2)
box(ax, 2.5, 2.2, 4.5, 1.45,
"Recomputed: odds ratios compound\n"
f"exp(10 × β) = {o['or1']:.3f}¹⁰ = {o['or10']:.2f}\n"
f"95% CI {o['lo10']:.2f}–{o['hi10']:.2f}",
BLUE, lw=2, tc=BLUE)
box(ax, 7.5, 2.2, 4.5, 1.45,
"As printed: per-year OR multiplied by 10\n"
f"10 × {o['or1']:.3f} = {o['m10']:.2f}\n"
f"10 × CI = {o['m10lo']:.2f}–{o['m10hi']:.2f}",
ORANGE, lw=2)
arrow(ax, 7.5, 1.47, 7.5, 1.12, c=ORANGE)
box(ax, 7.5, 0.6, 4.5, 0.95,
"Printed: “11-fold” (Abstract, Results)\n"
f"Fig. 5D marker ≈ {o['fig_v']:.0f} ({o['fig_lo']:.1f}–{o['fig_hi']:.1f})", ORANGE, lw=1.2)
arrow(ax, 2.5, 1.47, 2.5, 1.12, c=BLUE)
box(ax, 2.5, 0.6, 4.5, 0.95,
"Implied by the model per 10 years:\n"
f"about {o['or10']:.1f}-fold higher odds of death", BLUE, lw=1.2, tc=BLUE)
fig.tight_layout(pad=0.3)
save(fig, "fig2")
if __name__ == "__main__":
main()
Abstract reports 18 PDX models for the co-clinical trial; the paper's data contain 15
Cancer Res (2022) · doi:10.1158/0008-5472.CAN-22-0198 · PMC9478530
Summary
The abstract says the combination of MEK and CDK4/6 inhibition was tested "across 18 PDX models" and that 60% of tumors regressed, meeting the co-clinical trial's primary endpoint. Every other count in the paper gives 15 models. The Results subgroups add up to 6 + 2 + 3 + 4 = 15, the Results report "9 of 15 models (60%)", the Fig. 1 legends say 15 models, and Supplementary Tables S1 and S3 each list the same 15 model IDs. Every model used in the later experiments is one of those 15. Of the two counts, only 15 gives exactly 60% (9/15); with 18 the nearest values are 55.6% (10/18) and 61.1% (11/18), 60% of 18 is 10.8 models, and the 9 responders reported in the Results give 50%. The response rate and the "endpoint met" conclusion hold at 9 of 15. What the abstract gets wrong is the size of the evidence base: it overstates it by 3 models (20%), and it sits in the place readers and meta-analysts most often take sample sizes from.
What the paper reports
Abstract:
Across 18 PDX models undergoing dual inhibition of MEK and CDK4/6, 60% of tumors regressed, meeting the co-clinical trial primary endpoint.
Results, "Combined inhibition of MEK and CDK4/6 in colorectal cancer PDX models", first paragraph:
we evaluated typical RAS models that include a mixture of exon 2 KRASG12/13 (n = 6) P-loop domain mutated PDX models, extended RAS models that include NRASQ61 switch II domain mutated tumors (n = 2) along with KRASA146T nucleotide exchange promoting domain mutated tumors (n = 3; refs. 23, 24), and wild-type RAS tumors (n = 4).
Results, same section, second paragraph:
The PDX co-clinical trial of MEK (trametinib) and CDK4/6 (palbociclib) inhibition met its prespecified primary endpoint of tumor regression of ≥30% in more than 30% of the models. As shown in Fig. 1B and C, the combination of MEK and CDK4/6 inhibition produced ≥30% regression in 9 of 15 models (60%) with significantly greater tumor growth inhibition when compared with single agent–treated PDXs.
Figure 1 legend, panels A to D:
Six PDX models harboring codon 12 or 13 (“typical” RAS), 5 with codons 61 or 149 mutations (“extended” RAS), and 4 wild-type RAS PDX models were analyzed by RPPA
Trametinib (0.25 mpk, daily; green), palbociclib (75 mpk, daily; blue), and their combination (red) were evaluated in 15 mCRC PDX models.
C, Waterfall representation of 15 PDX models’ responses, per PDX model results.
D, Effect of treatments on mice body weight. Right, Tukey plot, after 21 days of treatment. Summarized data for 15 PDX models.
Results, "Mechanisms of acquired resistance":
Intriguingly, elevated phospho-SHP2 level was detected in 11 of 15 PDX models treated with the combination
Materials and Methods, "In vivo studies and PDXs":
PDX model details are provided in Supplementary Table S1.
Discussion, first paragraph:
The co-clinical trial met its primary endpoint of demonstrating tumor regressions in greater than 30% of models
What the paper's own numbers show
Method. I added up the RAS subgroup counts given in the Results text and in the Fig. 1A legend. I then built the list of PDX model IDs from Supplementary Table S1 (sheet "PDX details"), Supplementary Table S3 (the ranking of combination responses, grouped by RAS status), and the Fig. 1B panel titles. I added every model named for the follow-on experiments: the dose study (Fig. 2, Supp. Table S6), the durability cohort (Fig. 4, Supp. Table S7), the SHP2 triple combination (Fig. 5) and Supp. Fig. S2. Last, I checked which denominators can produce the abstract's 60%. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| PDX models given MEK + CDK4/6 inhibition | Abstract | 18 | 15; no set of 18 models appears anywhere in the paper or supplement |
| RAS subgroups, text | Results, paragraph 1 | n = 6, 2, 3, 4 | 6 + 2 + 3 + 4 = 15 |
| RAS subgroups, figure legend | Fig. 1A legend | 6 typical, 5 extended, 4 wild type | 6 + 5 + 4 = 15; the text's 2 + 3 = 5 extended models match |
| Models in the co-clinical trial | Results, paragraph 2; Fig. 1B–D legends | "9 of 15 models"; "15 mCRC PDX models" | 15 |
| Model panels | Fig. 1B | – | 15 |
| Model rows | Supp. Table S1 ("PDX details") | – | 15 unique IDs |
| Model rows by RAS group | Supp. Table S3 | – | 15 (6 typical, 5 extended, 4 wild type); same IDs as S1 and Fig. 1B |
| Models in follow-on experiments | Figs. 2, 4, 5; Supp. Fig. S2; Supp. Tables S6, S7 | 3, 8, 2 and 2 models | 9 distinct models, 0 outside the 15 |
| All distinct models named anywhere | Paper and supplement | – | 15, so 18 − 15 = 3 models are unaccounted for |
| Durability cohort | Results; Fig. 4 | "eight PDX models (4 of each RASmut subtype)" | 8, all among the 15 (4 typical, 4 extended) |
| Regression rate, 15 models | Abstract; Results | 60%; "9 of 15 models (60%)" | 9/15 = 60.0% |
| Same 9 responders over 18 models | – | – | 9/18 = 50.0% |
| Responders implied by 60% of 18 | Abstract | 60% | 0.6 × 18 = 10.8, not a whole number; 10/18 = 55.6%, 11/18 = 61.1% |
| Values of k for which k/n rounds to 60% | – | – | n = 15: k = 9 only; n = 18: none |
| Primary endpoint (more than 30% of models with ≥30% regression) | Results | met | 9/15 = 60% > 30%. Exact one-sided binomial test against 30% (my calculation; the paper reports no test): P = 0.015. Clopper–Pearson 95% CI 32.3% to 83.7%, one-sided 95% lower bound 36.0% |
Figure
Figure 1. Two text panels hold short excerpts from the article as archived in PMC. The left panel (orange, the value reported in the abstract) gives the abstract's sentence with its count of 18 models and its 60% response rate. The right panel (blue, counts found in or recomputed from the paper's own body and supplement) gives the Results' "9 of 15 models (60%)", the Results' subgroup counts, which add up to 15, the Fig. 1B legend's "15 mCRC PDX models", and the 15 rows in each of Supplementary Tables S1 and S3. The connector names the conflict: no set of 18 models is described anywhere in the paper, and of the two counts only 15 gives exactly 60% (9/15). The arithmetic is in recompute.py.txt.
Figure 2. Each gray dot is a response rate that can occur with n models (k/n, k = 0 to n), on a full 0–100% axis. The orange vertical line marks the 60% reported in both the abstract and the Results. The dotted line marks the primary endpoint's 30% threshold. With n = 15, 60% falls exactly on 9/15 (filled blue). With n = 18, no attainable value equals 60%: the same 9 responders give 9/18 = 50.0% (filled blue), and the two nearest values are 10/18 = 55.6% and 11/18 = 61.1% (open blue). Of the two counts in the paper, the reported 60% therefore matches the 15-model denominator exactly, and not the 18 stated in the abstract.
Why it matters
The abstract is the only place in the paper that says 18, and the abstract is where readers, indexers and meta-analysts usually take the number of models. Anyone counting PDX models across preclinical studies of MEK plus CDK4/6 inhibition, or citing this study as an example of a co-clinical trial design, would record 3 models (20%) that the paper does not document.
The efficacy conclusion is unaffected. The correct figure is 9 of 15 models with at least 30% regression (60%), well above the prespecified 30% threshold. The exact 95% interval for 9/15 runs from 32.3% to 83.7%. A reader who combined the abstract's 18 with the 9 responders given in the Results would get 50%, which still exceeds the 30% threshold but does not match the abstract's own 60%. The subgroup comparisons (typical vs. extended RAS, TP53, CMS) already rest on small groups, and the 15-model count is the right basis for judging them.
Explanations considered
- A wider set of models, such as the co-clinical trial plus the dose, durability or SHP2 experiments. This does not explain the 18. The models in Fig. 2 and Supp. Table S6 (C1138, B1011, C1174), Fig. 4 and Supp. Table S7 (8 models: C1138, C1047, B8093, B1011, B1008, F3008, C1012, C0997), Fig. 5 (B1008, F3008) and Supp. Fig. S2 (F3008, B1011) are all among the 15. The union of every model named in the paper and supplement is 15.
- A different unit, such as tumors or mice instead of models. This does not fit. The abstract says "18 PDX models" outright. Fig. 1B reports n = 6–9 tumors per arm for each model, so a count of tumors would be roughly 90 to 135 in the combination arm alone (15 models × 6–9 tumors), not 18.
- Another denominator that gives 60%. Not with 18. 0.6 × 18 = 10.8 is not a whole number, and no k/18 rounds to a whole-percent 60% (10/18 = 55.6%, 11/18 = 61.1%). Other denominators such as 5, 10 or 20 could give exactly 60%, but no such model count appears in the paper. Of the two counts the paper gives, only 15 yields exactly 60% (9/15), which is what the Results report.
- Models in the supplement that the main text omits. Supplementary Table S1 (PDX details) and Supplementary Table S3 (response ranking) each list exactly 15 models, with the same IDs as the 15 panels of Fig. 1B, split 6 typical, 5 extended and 4 wild type. The Supplementary Materials and Methods describe no other cohort.
- A text-conversion or archiving artifact. Ruled out. "18" is a spelled-out integer in running text, so it cannot be a lost sign, a misread superscript or a shifted table cell. The same wording appears on the live PMC page, in the Europe PMC full-text XML, in the PubMed abstract (PMID 35913398), and in the abstract on the publisher's figshare collection for the article.
- A later correction. None found (see Status).
- Side observation, not part of the claim. The Fig. 1A legend describes the extended group as "codons 61 or 149", while the Results text names KRAS A146T. The Fig. 1B panel titles label the three KRAS codon-146 models as B1011 A146T, C1151 A146T and C1012 A146V, so the text's "KRASA146T (n = 3)" is also slightly off. None of this changes any count: the 5 extended models equal the text's 2 NRAS Q61 plus 3 KRAS codon-146 models.
Status
As of 2026-10-23:
- Version of record. The publisher page (aacrjournals.org, Cancer Res 82(18):3335) and its PDF returned HTTP 403 to automated access and could not be read directly. The abstract sentence with "18" and the Results' "9 of 15 models (60%)" are identical on the live PMC page (PMC9478530), in the Europe PMC full-text XML, in the PubMed record (PMID 35913398) and in the abstract on AACR's figshare collection for the article (doi:10.1158/0008-5472.c.6513959). The figshare collection is published by the journal, so the error is very likely in the version of record too.
- Supplements. I took the full supplementary package (Figures S1–S5, Supplementary Materials and Methods, Tables S1–S9) from Europe PMC's supplementaryFiles endpoint and read Table S1, Table S3, Table S6 ("Results of alternative trametinib/palbociclib regimens study"; models C1138, B1011 and C1174), Table S7 ("TP53 mutational status in mCRC PDX models") and the Supplementary Materials and Methods. All are consistent with 15 models, as described above.
- Corrections. Crossref has no update-to or updated-by relations for the DOI. PMC and Europe PMC list no erratum, correction, expression of concern or retraction.
- PubPeer. The PubPeer publications API, queried with the DOI, returned no comments, and a web search found no PubPeer thread for the article.
Suggested correction
In the abstract, replace "Across 18 PDX models undergoing dual inhibition of MEK and CDK4/6, 60% of tumors regressed" with "Across 15 PDX models undergoing dual inhibition of MEK and CDK4/6, 9 (60%) regressed". This brings the abstract in line with the Results, the Fig. 1 legends and Supplementary Tables S1 and S3. The journal could also check the Fig. 1A legend's "codons 61 or 149" and the Results' "KRASA146T (n = 3)" against the Fig. 1B panel labels, which show two A146T models and one A146V model.
Recomputation script (python recompute.py.txt)
"""Recompute the PDX model count behind the abstract of PMC9478530
(Cancer Res 2022;82(18):3335, doi:10.1158/0008-5472.CAN-22-0198).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md,
writes fig1.svg/png and fig2.svg/png next to this file, and writes five candidate
figure designs (A-E) to ./candidates/.
Inputs are copied from the article as archived in PubMed Central and from its
supplementary files (Europe PMC supplementaryFiles package):
Abstract: "Across 18 PDX models ... 60% of tumors regressed"
Results, first section: subgroup counts n = 6 (KRAS G12/13), 2 (NRAS Q61),
3 (KRAS A146T), 4 (RAS wild type); "9 of 15 models (60%)"
Fig. 1 legend: 6 typical + 5 extended + 4 wild type; "15 mCRC PDX models"
Supplementary Table S1: sheet "PDX details", one row per model (model IDs only used here)
Supplementary Table S3: combination-response ranking, one row per model, by RAS group
Figs. 2, 4, 5, Supp. Fig. S2, Supp. Tables S6 and S7: models named for the follow-on experiments
Model IDs are laboratory PDX codes, not patient identifiers.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
from scipy import stats # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
ABSTRACT_N = 18 # Abstract: "Across 18 PDX models"
ABSTRACT_PCT = 60 # Abstract: "60% of tumors regressed"
RESULTS_RESPONDERS = 9 # Results: "≥30% regression in 9 of 15 models (60%)"
RESULTS_N = 15
RESULTS_PCT = 60
ENDPOINT_THRESHOLD = 0.30 # Results: primary endpoint "≥30% in more than 30% of the models"
SUBGROUPS_TEXT = {"KRAS G12/13 (typical)": 6, "NRAS Q61 (extended)": 2,
"KRAS A146T (extended)": 3, "RAS wild type": 4} # Results paragraph 1
SUBGROUPS_FIG1A = {"typical": 6, "extended": 5, "wild type": 4} # Fig. 1A legend
FIG1_LEGEND_N = 15 # Fig. 1B, 1C and 1D legends: "15 mCRC PDX models" / "15 PDX models"
SHP2_OF = 15 # Results: "elevated phospho-SHP2 ... in 11 of 15 PDX models treated with the combination"
S3 = { # Supplementary Table S3, model -> RAS group
"B1008": "typical", "C1035": "typical", "C1047": "typical",
"C1138": "typical", "C1177": "typical", "F3008": "typical",
"B1011": "extended", "B8093": "extended", "C0997": "extended",
"C1012": "extended", "C1151": "extended",
"B1025": "wild type", "B8178": "wild type", "C1023": "wild type", "C1174": "wild type",
}
S1 = ["C1035", "C1047", "B1008", "F3008", "C1138", "C1177", "B8093", "C0997",
"C1151", "C1012", "C1023", "B1025", "C1174", "B8178", "B1011"] # Supp. Table S1, "PDX details"
FIG1B = ["C1035", "C1047", "B1008", "F3008", "C1138", "C1177", "B8093", "C0997",
"C1151", "B1011", "C1012", "C1023", "B1025", "C1174", "B8178"] # Fig. 1B panel titles
FIG2 = ["C1138", "B1011", "C1174"] # Fig. 2 / Supp. Table S6 (dose study)
FIG4 = ["C1138", "C1047", "B8093", "B1011", "B1008", "F3008", "C1012", "C0997"] # Fig. 4 / Supp. Table S7 (TP53 status, durability cohort)
FIG5 = ["B1008", "F3008"] # Fig. 5 (SHP2 triple)
SUPP_S2 = ["F3008", "B1011"] # Supp. Fig. S2
# ---------------------------------------------------------------- recomputation
def line(label, value):
print(f"{label:<62} {value}")
print("== Model counts")
sum_text = sum(SUBGROUPS_TEXT.values())
sum_fig1a = sum(SUBGROUPS_FIG1A.values())
line("Results subgroups 6 + 2 + 3 + 4", sum_text)
line("Fig. 1A legend 6 + 5 + 4", sum_fig1a)
line("Extended group in text (NRAS Q61 + KRAS A146T) 2 + 3", 2 + 3)
s3_groups = {g: sum(v == g for v in S3.values()) for g in ("typical", "extended", "wild type")}
line("Supp. Table S3 models", f"{len(S3)} {s3_groups}")
line("Supp. Table S1 models (unique)", len(set(S1)))
line("Fig. 1B model panels (unique)", len(set(FIG1B)))
line("S1, S3 and Fig. 1B list the same models", set(S1) == set(S3) == set(FIG1B))
followon = set(FIG2) | set(FIG4) | set(FIG5) | set(SUPP_S2)
line("Models in Figs. 2, 4, 5, Supp. Fig. S2, Supp. Tables S6, S7 (unique)", len(followon))
line("... of which outside the 15", len(followon - set(S3)))
union = set(S3) | set(S1) | set(FIG1B) | followon
line("Union of all models named in paper and supplement", len(union))
line("Abstract count minus union", ABSTRACT_N - len(union))
line("Abstract count relative to 15", f"{ABSTRACT_N / len(union):.2f} x (+{ABSTRACT_N / len(union) - 1:.0%})")
fig4_groups = {g: sum(S3[m] == g for m in FIG4) for g in ("typical", "extended", "wild type")}
line("Fig. 4 durability cohort by group", fig4_groups)
print("\n== Response rate")
r15 = RESULTS_RESPONDERS / RESULTS_N
r18 = RESULTS_RESPONDERS / ABSTRACT_N
line("9/15", f"{r15:.1%}")
line("9/18", f"{r18:.1%}")
line("60% of 15", f"{0.60 * RESULTS_N:.1f}")
line("60% of 18", f"{0.60 * ABSTRACT_N:.1f} (not a whole number)")
for k in range(10, 13):
line(f"{k}/18", f"{k / 18:.1%}")
hits15 = [k for k in range(16) if round(100 * k / 15) == 60]
hits18 = [k for k in range(19) if round(100 * k / 18) == 60]
line("k with k/15 rounding to 60%", hits15)
line("k with k/18 rounding to 60%", hits18 if hits18 else "none")
line("(for contrast) n <= 20 with some k/n exactly 60%",
[n for n in range(1, 21) if any(100 * k == 60 * n for k in range(n + 1))])
print("\n== Primary endpoint (>30% of models with >=30% regression)")
p15 = stats.binom.sf(RESULTS_RESPONDERS - 1, RESULTS_N, ENDPOINT_THRESHOLD)
lb15 = stats.beta.ppf(0.05, RESULTS_RESPONDERS, RESULTS_N - RESULTS_RESPONDERS + 1)
lo15 = stats.beta.ppf(0.025, RESULTS_RESPONDERS, RESULTS_N - RESULTS_RESPONDERS + 1)
hi15 = stats.beta.ppf(0.975, RESULTS_RESPONDERS + 1, RESULTS_N - RESULTS_RESPONDERS)
line("Exact one-sided binomial P(X >= 9 | n=15, p=0.30)", f"{p15:.4f}")
line("Clopper-Pearson one-sided 95% lower bound, 9/15", f"{lb15:.1%}")
line("Clopper-Pearson two-sided 95% CI, 9/15", f"{lo15:.1%} to {hi15:.1%}")
p18 = stats.binom.sf(RESULTS_RESPONDERS - 1, ABSTRACT_N, ENDPOINT_THRESHOLD)
line("(for contrast) P(X >= 9 | n=18, p=0.30)", f"{p18:.4f}")
for k in (10, 11):
lo = stats.beta.ppf(0.025, k, ABSTRACT_N - k + 1)
hi = stats.beta.ppf(0.975, k + 1, ABSTRACT_N - k)
line(f"(for contrast) Clopper-Pearson two-sided 95% CI, {k}/18", f"{lo:.1%} to {hi:.1%}")
# ---------------------------------------------------------------- figure style
ORANGE, BLUE = "#eb6834", "#2a78d6"
INK, INK2, BG, GRID = "#0b0b0b", "#52514e", "#fcfcfb", "#e6e5e1"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "path", "lines.linewidth": 1.5,
"figure.dpi": 200, "text.hinting": "none", "text.hinting_factor": 1,
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def runs(ax, x, y, parts, size=10.5, ha_box=None):
"""Draw text runs [(text, color, weight)] left to right starting at axes coords (x, y)."""
fig = ax.figure
r = fig.canvas.get_renderer()
axw = ax.get_window_extent(renderer=r).width
def width(s, weight):
t = ax.text(0, 0, s, fontsize=size, fontweight=weight, transform=ax.transAxes)
w = t.get_window_extent(renderer=r).width
t.remove()
return w / axw
for text, color, weight in parts:
core = text.lstrip(" ")
x += width("|" + text[:len(text) - len(core)] + "|", "normal") - width("||", "normal")
ax.text(x, y, core, color=color, fontsize=size, fontweight=weight,
transform=ax.transAxes, va="baseline", ha="left")
# advance by the run's width, with the bounding-box padding cancelled out
x += width("|" + core + "|", weight) - width("||", weight)
return x
def box(ax, x, y, w, h, edge, lw=1.5, ls="-", fill=BG):
p = FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
transform=ax.transAxes, fc=fill, ec=edge, lw=lw, ls=ls)
ax.add_patch(p)
def blank(h):
fig = plt.figure(figsize=(W, h))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
return fig, ax
# ---------------------------------------------------------------- A: two text panels
def cand_A(path):
fig, ax = blank(3.9)
ax.text(0.03, 0.93, "The abstract's model count disagrees with every count in the body",
fontsize=12, fontweight="bold", transform=ax.transAxes)
# left panel: abstract
box(ax, 0.03, 0.12, 0.38, 0.70, ORANGE)
ax.text(0.05, 0.755, "ABSTRACT", fontsize=10, color=ORANGE, fontweight="bold",
transform=ax.transAxes)
ax.text(0.05, 0.695, "(reported)", fontsize=10, color=ORANGE, transform=ax.transAxes)
runs(ax, 0.05, 0.62, [("“Across ", INK, "normal"), ("18", ORANGE, "bold"),
(" PDX models", INK, "normal")])
runs(ax, 0.05, 0.54, [("undergoing dual inhibition of", INK, "normal")])
runs(ax, 0.05, 0.46, [("MEK and CDK4/6, ", INK, "normal"), ("60%", ORANGE, "bold"),
(" of tumors", INK, "normal")])
runs(ax, 0.05, 0.38, [("regressed…”", INK, "normal")])
ax.text(0.05, 0.20, f"With 18 models: 9/18 = {r18:.0%};\n60% of 18 = {0.6 * ABSTRACT_N:.1f} models",
fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.4)
# right panel: body
box(ax, 0.59, 0.12, 0.38, 0.70, BLUE)
ax.text(0.61, 0.755, "BODY & SUPPLEMENT", fontsize=10, color=BLUE, fontweight="bold",
transform=ax.transAxes)
ax.text(0.61, 0.695, "(reported / recomputed)", fontsize=10, color=BLUE, transform=ax.transAxes)
runs(ax, 0.61, 0.62, [("“≥30% regression in ", INK, "normal"), ("9 of 15", BLUE, "bold")])
runs(ax, 0.61, 0.54, [("models (60%)”", INK, "normal"), (" Results", INK2, "normal")])
runs(ax, 0.61, 0.45, [("n = 6 + 2 + 3 + 4 = ", INK, "normal"), (f"{sum_text}", BLUE, "bold"),
(" Results", INK2, "normal")])
runs(ax, 0.61, 0.36, [("“evaluated in ", INK, "normal"), ("15", BLUE, "bold"),
(" mCRC PDX", INK, "normal")])
runs(ax, 0.61, 0.28, [("models”", INK, "normal"), (" Fig. 1B legend", INK2, "normal")])
runs(ax, 0.61, 0.19, [("Supp. Tables S1, S3: ", INK, "normal"), (f"{len(S3)}", BLUE, "bold"),
(" rows each", INK, "normal")])
# connector
ax.annotate("", xy=(0.585, 0.47), xytext=(0.415, 0.47), xycoords="axes fraction",
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
ax.text(0.50, 0.55, "18 ≠ 15", ha="center", fontsize=12, fontweight="bold",
transform=ax.transAxes)
ax.text(0.50, 0.33, "no set of 18\nmodels is\ndescribed; of\nthe two counts,\nonly 9/15 = 60%", ha="center",
va="center", fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.3)
ax.text(0.03, 0.04, "Excerpts from the PMC-archived article and its supplement; "
"arithmetic from recompute.py.txt.", fontsize=10, color=INK2, transform=ax.transAxes)
save(fig, path)
# ---------------------------------------------------------------- B: count by location dot plot
def cand_B(path):
rows = [
("Abstract: “Across 18 PDX models”", ABSTRACT_N, "abs"),
("Results: subgroups 6 + 2 + 3 + 4", sum_text, "body"),
("Results: “9 of 15 models (60%)”", RESULTS_N, "body"),
("Fig. 1A legend: 6 + 5 + 4", sum_fig1a, "body"),
("Fig. 1B–D legends: “15 … PDX models”", FIG1_LEGEND_N, "body"),
("Fig. 1B: model panels", len(set(FIG1B)), "body"),
("Results: phospho-SHP2 “in 11 of 15”", SHP2_OF, "body"),
("Supp. Table S1: model rows", len(set(S1)), "body"),
("Supp. Table S3: model rows", len(S3), "body"),
("All models named anywhere", len(union), "body"),
]
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0.42, 0.12, 0.54, 0.70])
ys = list(range(len(rows)))[::-1]
for (lab, v, kind), y in zip(rows, ys):
c = ORANGE if kind == "abs" else BLUE
ax.plot([0, v], [y, y], color=GRID, lw=1.5, zorder=1)
ax.plot(v, y, "o", ms=9, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(v + 0.5, y, f"{v}", va="center", fontsize=10.5, color=INK)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10.5, color=INK)
ax.set_xlim(0, 20.5)
ax.set_xticks(range(0, 21, 5))
ax.set_xlabel("Number of PDX models")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.axvspan(15, 18, color=ORANGE, alpha=0.08, lw=0)
ax.set_ylim(-0.6, len(rows) - 1 + 1.3)
ax.text(16.5, ys[0] + 0.75, "3 models\nnot found", ha="center", va="center", fontsize=10,
color=INK2, linespacing=1.1)
h1, = ax.plot([], [], "o", ms=9, color=ORANGE, label="Abstract (reported)")
h2, = ax.plot([], [], "o", ms=9, color=BLUE, label="Body and supplement (stated or counted)")
fig.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(0.005, 0.93), ncol=2,
frameon=False, fontsize=10)
fig.text(0.01, 0.945, "Model count for the dual MEK + CDK4/6 co-clinical trial, by location",
fontsize=11.5, fontweight="bold")
save(fig, path)
# ---------------------------------------------------------------- C: percentage lattice
def cand_C(path):
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0.12, 0.2, 0.84, 0.56])
for n, y in ((15, 1), (18, 0)):
xs = [100 * k / n for k in range(n + 1)]
ax.plot(xs, [y] * len(xs), "o", ms=5, color="#b9b8b3", zorder=2)
ax.plot([0, 100], [y, y], color=GRID, lw=1.5, zorder=1)
ax.axvline(100 * ENDPOINT_THRESHOLD, color=INK2, lw=0.8, ls=(0, (2, 2)), zorder=0)
ax.text(100 * ENDPOINT_THRESHOLD + 0.8, 1.55, "endpoint threshold 30%", fontsize=10, color=INK2)
ax.axvline(ABSTRACT_PCT, color=ORANGE, lw=1.5, zorder=1)
ax.text(ABSTRACT_PCT + 0.8, 1.55, "Abstract and Results: 60%", fontsize=10, color=ORANGE,
fontweight="bold")
ax.plot(100 * r15, 1, "o", ms=10, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.annotate("9/15 = 60.0%", (100 * r15, 1), xytext=(100 * r15 + 4, 1.28), fontsize=10.5,
color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.plot(100 * r18, 0, "o", ms=10, color=BLUE, mec=BG, mew=1.5, zorder=4)
ax.annotate(f"9/18 = {r18:.1%}\n(same 9 responders)", (100 * r18, 0), xytext=(100 * r18 - 24, -0.62),
fontsize=10, color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
for k in (10, 11):
ax.plot(100 * k / 18, 0, "o", ms=9, mfc=BG, mec=BLUE, mew=1.5, zorder=4)
ax.annotate(f"10/18 = {10 / 18:.1%}", (100 * 10 / 18, 0), xytext=(38, 0.42), fontsize=10,
color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.annotate(f"11/18 = {11 / 18:.1%}\n60% of 18 = 10.8 models",
(100 * 11 / 18, 0), xytext=(66, -0.62), fontsize=10, color=INK,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.set_yticks([1, 0])
ax.set_yticklabels(["n = 15", "n = 18"], fontsize=10.5, color=INK)
ax.set_ylim(-0.9, 1.85)
ax.set_xlim(0, 100)
ax.set_xticks(range(0, 101, 10))
ax.set_xticklabels([f"{t}%" for t in range(0, 101, 10)])
ax.set_xlabel("Share of models regressing ≥30% (every attainable value k/n shown in gray)")
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.text(0.01, 0.92, "Exactly 60% is attainable with 15 models (9/15), not with 18",
fontsize=11.5, fontweight="bold")
save(fig, path)
# ---------------------------------------------------------------- D: unit chart of models
def cand_D(path):
fig, ax = blank(4.0)
ax.text(0.02, 0.93, "The abstract says 18 models; the paper names 15", fontsize=12,
fontweight="bold", transform=ax.transAxes)
groups = [("Typical RAS\n(KRAS G12/13)", [m for m in S3 if S3[m] == "typical"]),
("Extended RAS\n(NRAS Q61, KRAS A146T)", [m for m in S3 if S3[m] == "extended"]),
("RAS wild type", [m for m in S3 if S3[m] == "wild type"]),
("Abstract's extra\nmodels", [None, None, None])]
bw, bh, gap, x0 = 0.092, 0.13, 0.011, 0.29
for i, (lab, models) in enumerate(groups):
y = 0.70 - i * 0.175
ax.text(0.02, y + bh / 2, lab, va="center", fontsize=10, color=INK if models[0] else ORANGE,
transform=ax.transAxes, linespacing=1.2)
for j, m in enumerate(models):
x = x0 + j * (bw + gap)
if m:
box(ax, x, y, bw, bh, BLUE, fill="#e9f1fb")
ax.text(x + bw / 2, y + bh / 2, m, ha="center", va="center", fontsize=10,
transform=ax.transAxes)
else:
box(ax, x, y, bw, bh, ORANGE, ls=(0, (3, 2)))
ax.text(x + bw / 2, y + bh / 2, "?", ha="center", va="center", fontsize=11,
color=ORANGE, transform=ax.transAxes)
ax.text(x0 + len(models) * (bw + gap) + 0.005, y + bh / 2,
f"{len(models)}" if models[0] else "3 (none named)", va="center", fontsize=10.5,
fontweight="bold", color=BLUE if models[0] else ORANGE, transform=ax.transAxes)
ax.text(0.02, 0.025, f"Blue: models listed in Supp. Table S3 (6 + 5 + 4 = {len(S3)}); "
"the same 15 appear in Supp. Table S1\nand Fig. 1B. Orange dashed: the 3 further models "
"the abstract's “18” implies;\nnone is described in the paper or supplement.",
fontsize=10, color=INK2, transform=ax.transAxes, linespacing=1.35)
save(fig, path)
# ---------------------------------------------------------------- E: calculation flow
def cand_E(path):
fig, ax = blank(3.6)
ax.text(0.02, 0.93, "Following the paper's own counts to the abstract's 60%", fontsize=12,
fontweight="bold", transform=ax.transAxes)
def node(x, y, w, h, txt, edge, bold=False, fill=BG, ls="-"):
box(ax, x, y, w, h, edge, fill=fill, ls=ls)
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=10,
fontweight="bold" if bold else "normal", transform=ax.transAxes, linespacing=1.3)
def arrow(x1, y1, x2, y2):
ax.annotate("", xy=(x2, y2), xytext=(x1, y1), xycoords="axes fraction",
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.5, mutation_scale=12))
ax.text(0.02, 0.80, "From the Results and Fig. 1 (recomputed)", fontsize=10, color=BLUE,
fontweight="bold", transform=ax.transAxes)
box(ax, 0.02, 0.48, 0.25, 0.26, BLUE)
sub = "\n".join(f"{k:<12}{v:>2}" for k, v in
zip(("KRAS G12/13", "NRAS Q61", "KRAS A146T", "RAS WT"), SUBGROUPS_TEXT.values()))
ax.text(0.145, 0.61, sub, ha="center", va="center", fontsize=10, family="DejaVu Sans Mono",
transform=ax.transAxes, linespacing=1.3)
arrow(0.27, 0.61, 0.32, 0.61)
node(0.32, 0.52, 0.15, 0.18, f"{sum_text} models", BLUE, bold=True)
arrow(0.47, 0.61, 0.52, 0.61)
node(0.52, 0.52, 0.17, 0.18, f"{RESULTS_RESPONDERS} regressed\n≥30%", BLUE)
arrow(0.69, 0.61, 0.74, 0.61)
node(0.74, 0.52, 0.23, 0.18, f"9/15 = {r15:.1%}\nmatches 60%", BLUE, bold=True)
ax.text(0.02, 0.38, "From the abstract (reported)", fontsize=10, color=ORANGE, fontweight="bold",
transform=ax.transAxes)
node(0.32, 0.10, 0.15, 0.18, f"{ABSTRACT_N} models", ORANGE, bold=True)
arrow(0.47, 0.22, 0.52, 0.255)
arrow(0.47, 0.15, 0.52, 0.10)
node(0.52, 0.20, 0.45, 0.11, f"same 9 responders: 9/18 = {r18:.1%}, not 60%", ORANGE)
node(0.52, 0.02, 0.45, 0.15, f"keep 60%: 0.6 × 18 = {0.6 * ABSTRACT_N:.1f} models\n(not a whole number)",
ORANGE)
save(fig, path)
cand_A(os.path.join(CAND, "A_text_panels"))
cand_B(os.path.join(CAND, "B_count_by_location"))
cand_C(os.path.join(CAND, "C_percentage_lattice"))
cand_D(os.path.join(CAND, "D_model_grid"))
cand_E(os.path.join(CAND, "E_calculation_flow"))
# Provisional choice for the write-up, pending review of the candidates:
cand_A(os.path.join(HERE, "fig1"))
cand_C(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)
Primary-endpoint confidence interval matches 13 of 16 responders, not the 11 reported
Clin Cancer Res (2025) · doi:10.1158/1078-0432.CCR-24-3528 · PMC11995005
Summary
The paper reports its primary endpoint, objective response rate (ORR), as 68.8% (11 of 16 patients) with a 95% CI of 54.3–95.9, and the disease control rate (DCR) as 87.5% (14 of 16) with a 95% CI of 69.7–99.8. The point estimates agree with the best-response counts in Table 2. The intervals do not: the exact (Clopper–Pearson) 95% CI for 11/16 is 41.3–89.0 and for 14/16 is 61.7–98.4. The printed intervals are instead the exact intervals for 13/16 and 15/16, to the last printed digit, and Table 2 prints 13 in the ORR count cell beside the 68.75% that 11/16 gives. The point estimate is unaffected, but the stated lower bound of the primary endpoint is 13 points too high: the correct lower bound sits 1.3 points above the 40% level the design treats as a lack of activity, not 14.3 points above it.
What the paper reports
Results, "Clinical outcomes", first paragraph:
All patients had measurable lesions, and the ORR (primary endpoint) was 68.8% [11 of 16 patients; 95% confidence interval (CI), 54.3–95.9], with a DCR of 87.5% (14 of 16 patients; 95% CI, 69.7–99.8; Fig. 2B; Table 2).
Table 2, "Clinical activity of pembrolizumab plus trastuzumab and chemotherapy in patients with HER2-positive gastric cancer" (columns: Numbers | %, N = 16):
Best response Numbers % CR — — PR 11 68.75 SD 3 18.75 PD 2 12.5 ORRa (95% CI) 13 68.75 (54.3–95.9) DCRb (95% CI) 14 87.50 (69.7–99.8)
Table 2 footnotes define ORR as CR or PR and DCR as CR, PR or SD as best overall response.
Patients and Methods, "Response evaluation":
The ORR was defined as the percentage of patients who achieved a CR or PR among all patients.
Patients and Methods, "Statistical analysis":
A single-stage study design was used to determine the sample size based on the hypothesis that an observed ORR <40% suggests a lack of activity and an improvement in the ORR to ≥60% is of clinical interest.
Other places that give the responder count. Results, "Clinical outcomes":
The 11 patients who achieved PR (responders) had longer PFS (log-rank test, 12.1 vs. 4.0 months, P < 0.01)
Eleven (84.6%) of thirteen patients who had PD-L1+ tumors were responders, whereas all patients with PD-L1– (N = 3, 100%) were nonresponders (Fig. 2E).
Abstract, Results:
Sixteen patients were enrolled. The objective response rate was 69%, and the median progression-free survival was 11.9 months.
What the paper's own numbers show
Method. The inputs are the best-response counts in Table 2 (CR 0, PR 11, SD 3, PD 2; N = 16) and the ORR and DCR definitions in Methods and the Table 2 footnotes. The paper names no CI method; neither do the Methods or the trial protocol in the supplement. I therefore computed seven standard two-sided 95% binomial intervals (Clopper–Pearson exact, Wilson, Wilson with continuity correction, Wald, Agresti–Coull, Jeffreys and mid-p) for the stated counts. To find what the printed intervals correspond to, I searched every count x of every n from 5 to 40, with all seven methods at 95% and 90% confidence, for an interval within 0.15 percentage points of both printed bounds. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Best-response counts | Table 2 | CR 0, PR 11, SD 3, PD 2 | sum to 16; 68.75%, 18.75%, 12.50% (agree) |
| ORR point estimate | Results; Table 2 | 68.8% (11 of 16); 68.75 | (0 + 11)/16 = 68.75% (agrees) |
| ORR count | Table 2, ORR row | 13 | 11 (CR + PR); 13/16 would be 81.25% |
| Responders, by PD-L1 status | Results | 11 of 13 PD-L1+; 0 of 3 PD-L1− | 11 + 0 = 11 of 16 (agrees with 11) |
| ORR 95% CI | Results; Table 2 | 54.3–95.9 | 41.3–89.0 (exact, 11/16) |
| ORR 95% CI, other methods | — | — | Wilson 44.4–85.8; Wilson-cc 41.5–87.9; Wald 46.0–91.5; Agresti–Coull 44.2–86.1; Jeffreys 44.4–86.9; mid-p 43.7–87.5 |
| Exact 95% CI for 13/16 | — | — | 54.354–95.953; truncated to one decimal, 54.3–95.9 |
| DCR point estimate | Results; Table 2 | 87.5% (14 of 16); 87.50 | (0 + 11 + 3)/16 = 87.50% (agrees) |
| DCR 95% CI | Results; Table 2 | 69.7–99.8 | 61.7–98.4 (exact, 14/16) |
| Exact 95% CI for 15/16 | — | — | 69.768–99.842; truncated to one decimal, 69.7–99.8 |
| Intervals in the search that match the printed ORR CI | — | — | 1: exact 95%, 13/16 |
| Intervals in the search that match the printed DCR CI | — | — | 1: exact 95%, 15/16 |
| Midpoint of the printed ORR CI | Results | — | 75.1%, vs a point estimate of 68.75% |
| Exact lower bound for ORR minus 40% | — | 54.3 − 40 = 14.3 pp | 41.3 − 40 = 1.3 pp |
| One-sided exact binomial test, 11/16 vs 0.40 | — | not reported | P = 0.019 |
None of the seven methods gives an interval near 54.3–95.9 for 11 of 16, or near 69.7–99.8 for 14 of 16. Every method puts the ORR lower bound between 41.3% and 46.0%. The exact interval for 13/16 reproduces the printed ORR interval, and the exact interval for 15/16 reproduces the printed DCR interval. Both match when the bounds are truncated to one decimal, as the paper appears to do. Under the same truncation the correct intervals would read 41.3–88.9 and 61.6–98.4; rounded, they read 41.3–89.0 and 61.7–98.4.
Figure
Figure 1. ORR (top) and DCR (bottom) for the 16 patients, as percentages on one axis from 0 to 100. Orange: point estimate and 95% CI as printed in the Results and Table 2. Blue: the exact (Clopper–Pearson) 95% CI recomputed from the paper's own counts, 11/16 for ORR and 14/16 for DCR, with the same point estimates. Grey, dashed, open circle: the exact 95% CI for 13/16 (the count printed in Table 2's ORR row) and for 15/16; the dotted verticals show that these coincide with the printed intervals. The thin vertical line at 40% is the ORR level that the design's Statistical analysis section says "suggests a lack of activity". All blue and grey values come from recompute.py.txt.
Why it matters
- It is the primary endpoint. The ORR is the trial's stated primary endpoint, and the interval is the only measure of its precision in the paper. A reader of the Results or Table 2 is told that the response rate is at least 54.3% with 95% confidence. The paper's own counts support a lower bound of 41.3%.
- The margin over the design threshold shrinks from 14.3 to 1.3 points. The design treats an ORR below 40% as a lack of activity and an ORR of 60% or more as of clinical interest. The printed interval lies entirely above 54%. The correct interval, 41.3–89.0, still excludes 40% (one-sided exact P = 0.019 against 0.40), so the reading that the regimen is active does not reverse. But the interval also takes in most of the 40–60% band between "lack of activity" and "of clinical interest", which the printed interval does not.
- The interval's upward shift is carried into the table. Table 2 prints 13 as the ORR count. Anyone extracting the table for a pooled analysis or a comparison with KEYNOTE-811 could take 13 responders (81%) instead of 11 (69%).
- What is not affected. The point estimates (ORR 68.8%, DCR 87.5%), the abstract (which gives ORR 69% with no interval), the survival results and the paper's translational and biomarker findings do not depend on these intervals.
Explanations considered
- A different CI method. Not able to account for it. Seven standard methods at 95% give ORR intervals for 11/16 whose lower bounds lie between 41.3% and 46.0%, and none of them comes near 54.3–95.9. A 90% interval does not match either (exact 90% for 11/16: 45.2–86.8). In the search over all counts, n up to 40, seven methods and two confidence levels, the only interval that matches is the exact 95% interval for 13/16. The same holds for the DCR interval and 15/16.
- A different denominator or population. Not supported. The ORR is defined over all patients, the text says "11 of 16 patients", and all 16 had measurable lesions. The number of PD-L1+ patients is also 13, but the paper attributes the interval to the whole cohort, and the exact interval for the PD-L1+ subgroup (11 of 13) is 54.6–98.1, which does not match.
- A different responder definition, such as target-lesion response. Not supported. The paper defines ORR by RECIST best overall response (CR or PR). Table 2 lists 11 PR and no CR, and the text names 11 responders in three places. In any case, a numerator of 13 would make the point estimate 81.25%, not the printed 68.75%, so the printed estimate and interval could not both refer to it.
- A misprint in a single number. Not able to explain it. Both bounds of both intervals match the exact intervals for other counts to the last printed digit. A misprint would not be expected to produce two internally consistent exact intervals. Table 2's ORR count cell shows the same 13.
- A conversion or typesetting artifact. Not possible here. The Europe PMC full-text XML deposited by the publisher for the version of record carries the same text and the same Table 2 cells, including the 13.
- A correction already issued. None found (see Status).
- Unexplained difference between ORR and DCR. The ORR interval corresponds to a count two higher than reported (13 vs 11) and the DCR interval to a count one higher (15 vs 14). The paper gives no information on how the intervals were computed, so the reason the two differ cannot be determined from the article.
Status
- Version of record. The publisher's article page (aacrjournals.org, reached through the DOI) and its PDF returned HTTP 403 to automated requests. I checked the Europe PMC full-text XML instead. This is the publisher-deposited JATS of the final article, Clin Cancer Res 31(8):1476–1490. Its Results sentence and Table 2 cells, including the ORR count of 13, are the same as in the PMC archive. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. I downloaded all supplementary files from Europe PMC: Supplementary Figures S1–S7, Tables S1–S5 and Trial Protocol S1 (74 pages). These match the 13 supplementary items registered in Crossref. Protocol section 11.1.1.1 states that ORR "was calculated with two-sided 95% confidence intervals for the overall population" and defines it as (CR + PR) over the ITT population. It names no CI method. No supplement gives a different responder count or a method that would yield 54.3–95.9 from 11 of 16.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by relation. The Europe PMC search service returned HTTP 503 when I tried to repeat the check on 2026-10-24. PubPeer's search page returned HTTP 403 and its publications API returned 404 for this DOI. A web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC11995005; Europe PMC full-text XML and supplementary files for PMC11995005; Crossref; PubPeer; https://doi.org/10.1158/1078-0432.CCR-24-3528.
Suggested correction
In the Results ("Clinical outcomes"), replace
68.8% [11 of 16 patients; 95% confidence interval (CI), 54.3–95.9], with a DCR of 87.5% (14 of 16 patients; 95% CI, 69.7–99.8
with
68.8% [11 of 16 patients; 95% confidence interval (CI), 41.3–89.0], with a DCR of 87.5% (14 of 16 patients; 95% CI, 61.7–98.4
In Table 2, change the ORR row to "11 | 68.75 (41.3–89.0)" and the DCR row to "14 | 87.50 (61.7–98.4)". These are exact Clopper–Pearson intervals, rounded. If the authors used a different method, the intervals should be recomputed from 11/16 and 14/16 with that method and the method stated in the Statistical analysis section. The authors may also wish to add a sentence noting that the corrected ORR interval excludes the 40% inactivity level only narrowly.
Recomputation script (python recompute.py.txt)
"""Recompute the ORR and DCR confidence intervals of PMC11995005 (Clin Cancer Res 2025,
doi:10.1158/1078-0432.CCR-24-3528) from the paper's own counts, and redraw fig1.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
All inputs are copied from the paper: Results 'Clinical outcomes' and Table 2.
"""
import math, os
import numpy as np
from scipy.stats import beta, norm, binom
from scipy.optimize import brentq
HERE = os.path.dirname(os.path.abspath(__file__))
# ---- inputs from the paper -------------------------------------------------
N = 16 # "16 patients with HER2+ gastric cancer were enrolled and treated"
CR, PR, SD, PD = 0, 11, 3, 2 # Table 2 best response (CR shown as an em dash)
REP = { # reported (Results text and Table 2)
"ORR": dict(x_text=11, x_table=13, pct=68.75, ci=(54.3, 95.9)),
"DCR": dict(x_text=14, x_table=14, pct=87.50, ci=(69.7, 99.8)),
}
PDL1_POS, PDL1_NEG, RESP_PDL1_POS, RESP_PDL1_NEG = 13, 3, 11, 0
NULL_ORR = 0.40 # "an observed ORR <40% suggests a lack of activity"
# ---- interval methods --------------------------------------------------------
def cp(x, n, a=0.05):
lo = 0.0 if x == 0 else beta.ppf(a / 2, x, n - x + 1)
hi = 1.0 if x == n else beta.ppf(1 - a / 2, x + 1, n - x)
return lo, hi
def wilson(x, n, a=0.05, cc=False):
z = norm.ppf(1 - a / 2); p = x / n
if not cc:
c = (p + z * z / (2 * n)) / (1 + z * z / n)
h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / (1 + z * z / n)
return c - h, c + h
lo = (2*n*p + z*z - 1 - z*math.sqrt(z*z - 2 - 1/n + 4*p*(n*(1-p)+1))) / (2*(n+z*z))
hi = (2*n*p + z*z + 1 + z*math.sqrt(z*z + 2 - 1/n + 4*p*(n*(1-p)-1))) / (2*(n+z*z))
return (0.0 if x == 0 else max(0, lo)), (1.0 if x == n else min(1, hi))
def wald(x, n, a=0.05):
z = norm.ppf(1 - a / 2); p = x / n; s = math.sqrt(p * (1 - p) / n)
return p - z * s, p + z * s
def agresti_coull(x, n, a=0.05):
z = norm.ppf(1 - a / 2); nt = n + z * z; pt = (x + z * z / 2) / nt
s = math.sqrt(pt * (1 - pt) / nt); return pt - z * s, pt + z * s
def jeffreys(x, n, a=0.05):
return beta.ppf(a / 2, x + .5, n - x + .5), beta.ppf(1 - a / 2, x + .5, n - x + .5)
def midp(x, n, a=0.05):
g = lambda p: binom.sf(x, n, p) + 0.5 * binom.pmf(x, n, p) - a / 2
f = lambda p: binom.cdf(x - 1, n, p) + 0.5 * binom.pmf(x, n, p) - a / 2
lo = 0.0 if x == 0 else brentq(g, 1e-12, 1 - 1e-12)
hi = 1.0 if x == n else brentq(f, 1e-12, 1 - 1e-12)
return lo, hi
METHODS = {"Clopper-Pearson": cp, "Wilson": wilson,
"Wilson-cc": lambda x, n, a=0.05: wilson(x, n, a, cc=True),
"Wald": wald, "Agresti-Coull": agresti_coull, "Jeffreys": jeffreys, "mid-p": midp}
pc = lambda t: (100 * t[0], 100 * t[1])
f1 = lambda t: "%.1f-%.1f" % pc(t)
trunc = lambda v: math.floor(1000 * v) / 10 # truncate a proportion to 1 dp in %
# ---- 1. internal arithmetic ---------------------------------------------------
print("1. Internal arithmetic")
print(f" CR+PR+SD+PD = {CR+PR+SD+PD} (N = {N})")
print(f" PR % = {100*PR/N:.2f}; SD % = {100*SD/N:.2f}; PD % = {100*PD/N:.2f}")
print(f" ORR = (CR+PR)/N = {CR+PR}/{N} = {100*(CR+PR)/N:.2f}% (Table 2 count cell prints 13 -> {100*13/N:.2f}%)")
print(f" DCR = (CR+PR+SD)/N = {CR+PR+SD}/{N} = {100*(CR+PR+SD)/N:.2f}%")
print(f" responders: {RESP_PDL1_POS}/{PDL1_POS} PD-L1+ = {100*RESP_PDL1_POS/PDL1_POS:.1f}%, "
f"{RESP_PDL1_NEG}/{PDL1_NEG} PD-L1-; total {RESP_PDL1_POS+RESP_PDL1_NEG}/{N}")
# ---- 2. intervals for the stated counts --------------------------------------
print("\n2. 95% intervals for the stated counts (%), reported ORR 54.3-95.9, DCR 69.7-99.8")
for name, x in (("ORR", 11), ("DCR", 14)):
for m, f in METHODS.items():
lo, hi = pc(f(x, N))
print(f" {name} {x}/{N} {m:16s} {lo:6.2f}-{hi:6.2f} width {hi-lo:5.1f}")
r = REP["ORR"]["ci"]; print(f" reported ORR width {r[1]-r[0]:.1f}; reported DCR width {REP['DCR']['ci'][1]-REP['DCR']['ci'][0]:.1f}")
print(f" reported ORR interval midpoint {(r[0]+r[1])/2:.2f} vs point estimate 68.75")
# ---- 3. which count reproduces the printed intervals -------------------------
print("\n3. Exact (Clopper-Pearson) 95% interval for every x out of 16")
for x in range(9, 17):
lo, hi = cp(x, N)
print(f" {x:2d}/16 = {100*x/N:6.2f}% CP {100*lo:7.3f}-{100*hi:7.3f} truncated {trunc(lo):.1f}-{trunc(hi):.1f}")
for name, x in (("ORR", 13), ("DCR", 15)):
lo, hi = cp(x, N)
print(f" {name}: CP {x}/16 truncated to 1 dp = {trunc(lo):.1f}-{trunc(hi):.1f}; reported {REP[name]['ci'][0]}-{REP[name]['ci'][1]}")
print("\n Search: every x<=n, 5<=n<=40, 7 methods, alpha 0.05 or 0.10, within 0.15 pp of both printed bounds")
for name in ("ORR", "DCR"):
tgt = REP[name]["ci"]; hits = []
for n in range(5, 41):
for x in range(0, n + 1):
for m, f in METHODS.items():
for a in (0.05, 0.10):
lo, hi = pc(f(x, n, a))
if abs(lo - tgt[0]) < 0.15 and abs(hi - tgt[1]) < 0.15:
hits.append(f"{m} alpha={a} {x}/{n} -> {lo:.2f}-{hi:.2f}")
print(f" {name} ({tgt[0]}-{tgt[1]}): {len(hits)} match(es)")
for h in hits: print(" ", h)
print("\n Other candidate populations")
print(f" CP 11/13 (responders among the 13 PD-L1+ patients) = {f1(cp(11,13))}")
print(f" correct intervals reported in the paper's own style (truncated): ORR {trunc(cp(11,N)[0]):.1f}-{trunc(cp(11,N)[1]):.1f}, "
f"DCR {trunc(cp(14,N)[0]):.1f}-{trunc(cp(14,N)[1]):.1f}; rounded: ORR {f1(cp(11,N))}, DCR {f1(cp(14,N))}")
# ---- 4. consequence for the design threshold ----------------------------------
p_one = binom.sf(10, N, NULL_ORR) # P(X >= 11 | p = 0.40)
print("\n4. Design threshold (ORR < 40% suggests lack of activity)")
print(f" correct exact lower bound {100*cp(11,N)[0]:.1f}% vs reported 54.3%: margin above 40% "
f"{100*cp(11,N)[0]-40:.1f} pp vs {54.3-40:.1f} pp")
print(f" one-sided exact binomial P(X>=11 | n=16, p=0.40) = {p_one:.4f}")
print(f" one-sided 95% exact lower bound for 11/16 (alpha 0.10 two-sided) = {100*cp(11,N,0.10)[0]:.1f}%")
# ---- figure -------------------------------------------------------------------
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, INK, INK2, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "svg.fonttype": "none",
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK})
orr_cp, dcr_cp = pc(cp(11, N)), pc(cp(14, N))
orr13, dcr15 = pc(cp(13, N)), pc(cp(15, N))
rows = [ # (y, label, point, (lo,hi), style)
(5.4, "Reported (Results, Table 2)\n68.75%, 95% CI 54.3–95.9", 68.75, REP["ORR"]["ci"], "rep"),
(4.6, f"Recomputed, exact, 11/16\n68.75%, 95% CI {orr_cp[0]:.1f}–{orr_cp[1]:.1f}", 68.75, orr_cp, "rec"),
(3.8, f"Exact interval for 13/16 (Table 2 count)\n81.25%, {orr13[0]:.2f}–{orr13[1]:.2f}", 81.25, orr13, "alt"),
(2.0, "Reported (Results, Table 2)\n87.50%, 95% CI 69.7–99.8", 87.5, REP["DCR"]["ci"], "rep"),
(1.2, f"Recomputed, exact, 14/16\n87.50%, 95% CI {dcr_cp[0]:.1f}–{dcr_cp[1]:.1f}", 87.5, dcr_cp, "rec"),
(0.4, f"Exact interval for 15/16\n93.75%, {dcr15[0]:.2f}–{dcr15[1]:.2f}", 93.75, dcr15, "alt"),
]
fig, ax = plt.subplots(figsize=(7.2, 5.0), facecolor=BG)
ax.set_facecolor(BG)
ax.axvline(40, color=INK2, lw=0.9, zorder=1)
ax.text(41.0, 6.1, "40%: an ORR below this \"suggests a lack\nof activity\" (Statistical analysis)", fontsize=10, color=INK2, va="center", ha="left")
for y, lab, pt, (lo, hi), st in rows:
if st == "rep":
ax.plot([lo, hi], [y, y], color=ORANGE, lw=2, solid_capstyle="butt", zorder=3)
ax.plot(pt, y, "o", ms=9, color=ORANGE, zorder=4)
elif st == "rec":
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt", zorder=3)
ax.plot(pt, y, "o", ms=9, color=BLUE, zorder=4)
else:
ax.plot([lo, hi], [y, y], color=INK2, lw=2, ls=(0, (3, 2)), zorder=3)
ax.plot(pt, y, "o", ms=8, mfc=BG, mec=INK2, mew=1.6, zorder=4)
for v in (lo, hi):
ax.plot([v, v], [y - 0.12, y + 0.12], color={"rep": ORANGE, "rec": BLUE, "alt": INK2}[st], lw=2, zorder=3)
# bracket showing the reported interval coincides with the 13/16 and 15/16 exact intervals
for ytop, ybot, (lo, hi) in ((5.4, 3.8, REP["ORR"]["ci"]), (2.0, 0.4, REP["DCR"]["ci"])):
for v in (lo, hi):
ax.plot([v, v], [ybot + 0.14, ytop - 0.14], color=INK2, lw=0.8, ls=":", zorder=2)
ax.set_yticks([y for y, *_ in rows])
ax.set_yticklabels([r[1] for r in rows], fontsize=10)
ax.set_xlim(0, 100); ax.set_ylim(-0.1, 6.5)
ax.set_xticks(range(0, 101, 20))
ax.set_xlabel("Percentage of the 16 patients")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"): ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
from matplotlib.transforms import blended_transform_factory
tr = blended_transform_factory(ax.transAxes, ax.transData)
for yh, txt in ((6.0, "ORR (primary endpoint)"), (2.6, "DCR")):
ax.text(-0.02, yh, txt, transform=tr, ha="right", va="center", fontsize=11, fontweight="bold")
handles = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="Reported in the paper"),
Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, label="Recomputed from the paper's counts"),
Line2D([], [], color=INK2, lw=2, ls=(0, (3, 2)), marker="o", ms=8, mfc=BG, mec=INK2,
label="Exact interval for the count that reproduces the printed CI")]
ax.legend(handles=handles, loc="upper left", bbox_to_anchor=(-0.55, -0.16), ncol=1, frameon=False, fontsize=10,
handlelength=2.2, columnspacing=1.2)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, bbox_inches="tight", **kw)
print("\nwrote fig1.svg and fig1.png")
Fisher exact P reported as 0.0005 for 1/6 versus 7/18 remissions; the counts give 0.62
Clin Cancer Res (2012) · doi:10.1158/1078-0432.CCR-12-2442 · PMC3596113
Summary
The Discussion of this phase I trial compares the CR/CRi rate (complete remission plus complete remission with incomplete recovery) between dose groups of SCH 900776: 1 of 6 patients (17%) below 40 mg/m² and 7 of 18 (39%) at 40 mg/m² or above, reported as "P = 0.0005, Fisher exact test". The counts agree with Table 3, Table 4, the Results text and the abstract. A Fisher exact test on them gives a two-sided P of 0.62 (one-sided 0.32), and with 6 and 18 patients and 8 responders no table in the reported direction can reach a P below 0.066. This is the only P value in the paper, and the Discussion uses it as statistical support for a dose-response that would point to a contribution of SCH 900776 itself; the trial's toxicity-based dose-finding conclusions do not depend on it.
What the paper reports
The comparison and its P value (Discussion, paragraph 1):
Indeed, the CR/CRi rate was 1 of 6 (17%) for patients who received <40 mg/m2 SCH 900776 versus 7 of 18 (39%) for those receiving ≥ 40 mg/m2 (P = 0.0005, Fisher exact test).
The sentence immediately before it, which states the claim the P value supports (Discussion, paragraph 1; the PMC text prints "doseresponse", the publisher's HTML "dose-response"):
The notion that part of the responses might relate to adding SCH 900776 to the timed sequential cytarabine infusions is supported by the presence of a doseresponse curve, with clinical responses to SCH 900776 beginning at 40 mg/m2, in conjunction with Cmax exceeding 1,000 ng/mL.
The same counts, without a P value (Results, "Clinical outcome"):
Similarly, the overall response rate (CR plus CRi) was 8 of 24 (33%), with 1 CRi (17%) of 6 patients treated with SCH 900776 doses of ≤ 20 mg/m2 and 7 (39%; 5 CR, 2 CRi) of 18 patients treated at dose level 3 (40 mg/m2) or higher, with at least 2 of 6 (33%) in each of dose levels 3 through 5.
Abstract, Results:
Complete remissions occurred in 8 of 24 (33%) patients, with 7 of 8 at 40 mg/m2 or higher.
Table 3 ("Dose escalation and clinical outcome for 24 adults with relapsed or refractory acute leukemia"), dose-level labels and the "Response (CR/CRi)" column, condensed to one line with the archived typography kept:
- 10 mg/m2 (n = 3): 1 · 2. 20 mg/m2 (n = 3): 0 · 3. 40 (mg/m2 (n = 6): 3 · 4. 56 (mg/m2 (n = 6): 2 · 5. 140 mg (n = 6): 2 · Total: 8
Table 4 lists the 8 responders individually, at dose levels 1, 3, 3, 3, 4, 4, 5 and 5.
The authors' caveats, later in the same paragraph (Discussion, paragraph 1):
we cannot exclude the possibility that some or all of the responders might have responded to timed sequential cytarabine without SCH 900776
Moreover, the small sample size and heterogeneous nature of this patient population preclude any ability to draw accurate conclusions about additive or synergistic clinical effects of adding SCH 900776 to cytarabine
The statistics subsection of the Methods names no hypothesis test (Methods, "Statistical considerations"):
The primary objective was to determine the combination-recommended phase II dose (RP2D) by assessing the safety, tolerability, DLT, and antileukemic activity (CR rate) of the combination of cytarabine and SCH 900776 in adult patients with relapsed and/or refractory acute leukemias.
What the paper's own numbers show
Method. I rebuilt the 2×2 table from Table 3. Dose levels 1–2 (10 and 20 mg/m², 3 patients each) form the group below 40 mg/m²: 1 + 0 = 1 responder of 6. Dose levels 3–5 (40 mg/m², 56 mg/m² and a flat 140 mg that the paper states is equivalent to 80 mg/m²; 6 patients each) form the group at or above 40 mg/m²: 3 + 2 + 2 = 7 responders of 18. Table 4's responder dose levels and the abstract's "7 of 8" give the same split. The table is therefore [[1 responder, 5 non-responders], [7, 11]]. I ran Fisher's exact test with scipy and cross-checked it by enumerating the hypergeometric distribution directly. I also ran the usual alternatives for a 2×2 table, two tests across all five dose levels, and enumerated every table with the same margins (6 vs 18 patients, 8 responders). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| CR/CRi below 40 mg/m² | Discussion ¶1; Results; Table 3 | 1 of 6 (17%) | 1/6 = 16.7% (Table 3: 1 + 0 of 3 + 3) |
| CR/CRi at or above 40 mg/m² | Discussion ¶1; Results; Table 3; abstract | 7 of 18 (39%); "7 of 8" | 7/18 = 38.9% (Table 3: 3 + 2 + 2 of 6 + 6 + 6); Table 4: 7 of 8 responders at level ≥ 3 |
| Fisher exact test, below vs at or above 40 mg/m² | Discussion ¶1 | P = 0.0005 | two-sided P = 0.6214, 1,243 times larger |
| Fisher exact, one-sided (fewer responses below 40 mg/m²) | – | – | P = 0.3191 |
| Odds ratio, ≥ 40 vs < 40 mg/m² | – | – | sample 3.18; conditional MLE 3.05, exact 95% CI 0.25 to 171.6 |
| Exact 95% CIs for the two response rates | – | – | 16.7% (0.4% to 64.1%) and 38.9% (17.3% to 64.3%) |
| Pearson chi-square / Yates-corrected chi-square | – | – | P = 0.3173 / 0.6171 |
| Barnard / Boschloo exact unconditional tests | – | – | P = 0.4090 / 0.3761 |
| Cochran-Armitage trend over the 5 dose levels | – | – | P = 0.7423 (scores 1–5); 0.7159 (mg/m²); 0.6541 (log dose) |
| Fisher-Freeman-Halton exact test, 2×5 dose levels | – | – | P = 0.8265 |
| Smallest two-sided Fisher P in the reported direction, same margins | – | – | P = 0.0664, at 0/6 vs 8/18 |
| Tables with these margins giving P ≤ 0.0005 | – | – | only 6/6 vs 2/18 (P = 0.00021), the reverse of the reported pattern |
| Responders needed at ≥ 40 mg/m² for P ≤ 0.0005 if the low-dose group stays at 1/6 | – | – | 18 of 18 (P = 0.00014) |
The arithmetic of the counts and percentages is correct; only the P value is not. It is 1,243 times smaller than the two-sided Fisher P for the stated counts and lies below the smallest value that any table with these margins can produce in the reported direction (0.066). With 8 responders among 24 patients, 2.0 responders would be expected in the low-dose group if response did not depend on dose; 1 was observed.
Figure
Figure 1. P values for CR/CRi below vs at or above 40 mg/m² (1/6 vs 7/18, Table 3) on one logarithmic axis. The orange point is the value printed in the Discussion (P = 0.0005, Fisher exact test). The blue points are recomputed from the paper's own counts: the stated test (Fisher exact, two-sided, and one-sided in the reported direction), common alternatives for a 2×2 table (Pearson and Yates chi-square, Barnard and Boschloo exact tests), and two tests across the five dose levels (Cochran-Armitage trend, Fisher-Freeman-Halton). The dashed line marks P = 0.05 for reference. The shaded region (P < 0.066) contains P values that no 2×2 table with the paper's margins can produce in the reported direction. All recomputed values lie between 0.32 and 0.83.
Figure 2. With the margins fixed by the paper (6 patients below 40 mg/m², 18 at or above, 8 responders), only seven 2×2 tables are possible. Each is labelled on the horizontal axis by its responder counts, below 40 mg/m² on the top line and at or above 40 mg/m² on the bottom line. Blue markers give the two-sided Fisher P for each table; the filled marker is the observed table (P = 0.62). The orange line is the reported P = 0.0005. Only the table in which all 6 low-dose patients respond, the opposite of the reported pattern, falls below it. The most extreme table in the reported direction (0/6 vs 8/18) gives P = 0.066.
Why it matters
The P value is the only formal statistical result in the paper. The Discussion presents it as evidence for a dose-response, and uses the dose-response to support "the notion that part of the responses might relate to adding SCH 900776" to cytarabine. As printed, it reads as strong evidence that response rates differ between the dose groups. Computed from the paper's own counts, the two-group comparison gives P = 0.62, the trend test across all five dose levels gives P = 0.65 to 0.74 depending on scoring, and the exact 95% CI for the odds ratio runs from 0.25 to 172. With 6 patients in the lower-dose group, the data are consistent both with no dose effect and with a large one; they do not establish a dose-response either way.
The error is limited in scope. It is not in the title, the abstract or any table. The abstract gives only correct counts ("8 of 24 (33%)", "7 of 8 at 40 mg/m2 or higher"), although its emphasis on the 7 of 8 carries the same dose-response reading, now without statistical support. The phase I conclusions (feasibility and tolerability, the dose-limiting toxicities at 140 mg, the recommended phase II dose of 100 mg, about 56 mg/m², the pharmacokinetics and the H2Ax observations) rest on other data and are unaffected. The raw counts are printed next to the P value, and the authors state in the same paragraph that the small, heterogeneous sample precludes firm conclusions about added effect. Even so, a reader who takes the P value at face value, for example when judging whether the dose-response signal justifies a randomized trial, or when citing the paper as evidence of activity attributable to the Chk1 inhibitor, would overestimate that signal considerably.
Three smaller inconsistencies, separate from the P value and not affecting any conclusion, turned up during the check:
- Results, "Clinical outcome": day-14 complete tumor clearance at the two lowest doses is given as "2 of 6 (17%)"; 2/6 is 33% (Table 3: 1 + 1 of 3 + 3).
- The CR/CRi split differs between tables. Table 3 and the Results text give 5 CR and 3 CRi, with "3 (2/1)" at dose level 3. Table 4 lists three CRs at dose level 3 and 6 CR and 2 CRi overall. The combined CR/CRi counts, which the P value concerns, are the same in both.
- Discussion, paragraph 5: "2 of 10 (20%) patients with a complex karyotype achieved CR/CRi versus 5 of 14 (36%)" covers all 24 patients but sums to 7 of the 8 responders.
Explanations considered
- A different test, or a one-sided test. Fisher one-sided gives 0.32 and two-sided 0.62; Pearson chi-square 0.32 and Yates-corrected 0.62; Barnard 0.41 and Boschloo 0.38; the Cochran-Armitage trend test 0.65 to 0.74 and the 2×5 Freeman-Halton test 0.83. None comes within two orders of magnitude of 0.0005, and the Methods describe no other analysis. Does not account for the value.
- Different counts, grouping or cut-point. The 1/6 vs 7/18 split is confirmed independently by Table 3, Table 4, the Results text and the abstract, so the 2×2 table is fixed. Placing the 140 mg flat dose anywhere at or above 40 mg/m² does not change it. Other cut-points on the same data (below 20, 56 or 80 mg/m²) each give a Fisher P of 1.00. Keeping 1/6 in the low-dose group, the high-dose group would need 18 of 18 responders to reach P ≤ 0.0005. Does not account for the value.
- The P value belongs to another comparison. The nearest is CR/CRi in patients with day-14 tumor clearance against those with residual leukemia (8 of 12 vs 0 of 12, from the Results text): two-sided P = 0.0013, one-sided 0.00067. That is the same order of magnitude but not the same number, the paper attaches no P value to that comparison, and the sentence clearly refers to the dose groups. Other comparisons in the paper give larger values: day-14 clearance by dose group (2/6 vs 10/18) 0.64, AML vs other leukemias (8/21 vs 0/3) 0.53, complex karyotype vs not (2/10 vs 5/14) 0.65. A one-sample test of 7/18 against an assumed historical response rate gives 0.00002 (against 5%) or 0.0012 (against 10%); neither matches, and the sentence describes a two-group Fisher test. Does not account for the value; the source of 0.0005 cannot be identified from the paper.
- A typographical slip, such as misplaced digits. Cannot be excluded, but neither "0.05" nor "0.5" matches the correct two-sided value of 0.62, so a simple digit shift does not reconcile the numbers. Does not reconcile the numbers.
- An error introduced in the archived manuscript. The PMC copy is the NIH author manuscript. An archived copy of the publisher's full-text page (see Status) contains the identical sentence, "(P = 0.0005, Fisher exact test)", together with the same Results counts and abstract. The value is plain text with no superscripts or special characters that a conversion could alter. Ruled out for the sentence.
- Patient-level or supplementary data that would change the table. The paper has no supplementary material. Table 3's per-level counts and Table 4's per-patient dose levels fully determine the 2×2 table. Ruled out.
Status
As of 2026-10-23:
- Version of record. The live publisher site (aacrjournals.org, reached through the doi.org redirect) returned HTTP 403 and a browser-verification page to automated access, as did the publisher PDF link listed in Crossref. I therefore checked an Internet Archive capture of the publisher's full-text HTML (clincancerres.aacrjournals.org/content/18/24/6723.full, captured 2021-10-19; citation metadata Clin Cancer Res 18(24):6723–6731). It contains the Discussion sentence word for word, including "(P = 0.0005, Fisher exact test)", the same Results paragraph and the same abstract. Table 3 is not rendered inline in that capture, so the table itself was checked only in the PMC copy, where it agrees with the Results text and Table 4. The PMC copy (NIHMS author manuscript) matches the publisher text in every passage compared, apart from the hyphen in "dose-response".
- Supplements. None. The PMC record has no supplementary material, Europe PMC lists none, and the archived publisher page links none.
- Corrections, retractions, expressions of concern. None found. The repository's
known.pycheck returned no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to, updated-by or relation entries, and the PubMed record has no comment/correction links. - PubPeer. Could not be checked directly: both the PubPeer search page and its browser-extension API returned HTTP 403 / Cloudflare verification to automated access. Earlier web searches for the DOI and title with "PubPeer", "erratum" and "correction" found no thread or notice; a fresh web search could not be run for this write-up.
- Sources. PMC3596113 (as archived in PubMed Central); doi:10.1158/1078-0432.CCR-12-2442; Internet Archive capture of the publisher's full-text page dated 2021-10-19; Crossref and Europe PMC records; ClinicalTrials.gov NCT00907517 (no per-patient results that bear on these counts).
Suggested correction
The sentence in Discussion paragraph 1 could read: "Indeed, the CR/CRi rate was 1 of 6 (17%) for patients who received <40 mg/m2 SCH 900776 versus 7 of 18 (39%) for those receiving ≥ 40 mg/m2 (P = 0.62, two-sided Fisher exact test)." Because neither this comparison nor a trend test across the five dose levels (P ≈ 0.7) supports a dose-response at this sample size, the preceding sentence's statement that the responses are "supported by the presence of a dose-response curve" could be qualified as a descriptive observation. If the reported value came from a different comparison, the correction should say which one. Separately, "2 of 6 (17%)" for day-14 tumor clearance at ≤ 20 mg/m² should read "2 of 6 (33%)", and the CR/CRi breakdown at dose level 3 and the karyotype counts could be reconciled between Table 3, Table 4 and the Discussion.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC3596113 write-up and regenerate its figures.
Paper: "Phase I and Pharmacologic Trial of Cytosine Arabinoside with the Selective
Checkpoint 1 Inhibitor Sch 900776 in Refractory Acute Leukemias",
Clin Cancer Res 2012;18(24):6723-31, doi:10.1158/1078-0432.CCR-12-2442 (PMC3596113).
All inputs are copied from Table 3, Table 4 and the Results/Discussion text.
Run: python recompute.py.txt (needs numpy, scipy >= 1.7, matplotlib)
Writes fig1.svg/png and fig2.svg/png next to this file.
"""
import os
from math import comb
from itertools import product
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Table 3, per dose level: (level, dose in mg/m2, n, day-14 CTC, CR/CRi, CR, CRi).
# Level 5 was a flat 140 mg, which the paper states is "equivalent to 80 mg/m2".
TABLE3 = [
(1, 10, 3, 1, 1, 0, 1),
(2, 20, 3, 1, 0, 0, 0),
(3, 40, 6, 4, 3, 2, 1),
(4, 56, 6, 3, 2, 2, 0),
(5, 80, 6, 3, 2, 1, 1),
]
# Table 4, one row per responder: (dose level, response)
TABLE4 = [(1, "CRi"), (3, "CR"), (3, "CR"), (3, "CR"), (4, "CR"), (4, "CR"), (5, "CRi"), (5, "CR")]
REPORTED_P = 0.0005 # Discussion, paragraph 1: "(P = 0.0005, Fisher exact test)"
lev = np.array([r[0] for r in TABLE3]); dose = np.array([r[1] for r in TABLE3], float)
ns = np.array([r[2] for r in TABLE3]); ctc = np.array([r[3] for r in TABLE3])
rs = np.array([r[4] for r in TABLE3])
cr3 = sum(r[5] for r in TABLE3); cri3 = sum(r[6] for r in TABLE3)
def fmt(p):
return f"{p:.4f}" if p >= 1e-3 else f"{p:.5f}"
def section(t):
print("\n" + "=" * 72 + "\n" + t + "\n" + "=" * 72)
# ------------------------------------------------- 1. counts
section("1. Counts from Table 3 (checked against Table 4 and the text)")
N, R = int(ns.sum()), int(rs.sum())
lo = dose < 40
lo_n, lo_r = int(ns[lo].sum()), int(rs[lo].sum())
hi_n, hi_r = int(ns[~lo].sum()), int(rs[~lo].sum())
print(f"Total CR/CRi: {R}/{N} = {R/N:.1%}")
print(f"<40 mg/m2 (levels 1-2): {lo_r}/{lo_n} = {lo_r/lo_n:.1%} (paper: '1 of 6 (17%)')")
print(f">=40 mg/m2 (levels 3-5): {hi_r}/{hi_n} = {hi_r/hi_n:.1%} (paper: '7 of 18 (39%)')")
t4_levels = [lv for lv, _ in TABLE4]
print(f"Table 4 responder dose levels: {t4_levels}; below level 3: {sum(l < 3 for l in t4_levels)}, "
f"level 3 or above: {sum(l >= 3 for l in t4_levels)} (abstract: '7 of 8 at 40 mg/m2 or higher')")
tab = np.array([[lo_r, lo_n - lo_r], [hi_r, hi_n - hi_r]])
print("2x2 table [responders, non-responders], rows <40 / >=40:", tab.tolist())
# ------------------------------------------------- 2. the stated test
section("2. Fisher exact test of <40 vs >=40 mg/m2 (the test the paper names)")
or_lo_vs_hi, p2 = stats.fisher_exact(tab, alternative="two-sided")
p_less = stats.fisher_exact(tab, alternative="less")[1]
p_greater = stats.fisher_exact(tab, alternative="greater")[1]
print(f"Sample odds ratio (<40 vs >=40) = {or_lo_vs_hi:.3f}; (>=40 vs <40) = {1/or_lo_vs_hi:.2f}")
print(f"Two-sided P = {fmt(p2)}")
print(f"One-sided P, fewer responses below 40 mg/m2 (the reported direction) = {fmt(p_less)}")
print(f"One-sided P, opposite direction = {fmt(p_greater)}")
print(f"Reported P / recomputed two-sided P: {REPORTED_P} vs {p2:.4f} -> recomputed is {p2/REPORTED_P:,.0f} times larger")
# manual hypergeometric enumeration as a cross-check of scipy
pmf = {x: comb(R, x) * comb(N - R, lo_n - x) / comb(N, lo_n) for x in range(0, min(R, lo_n) + 1)}
p2_manual = sum(v for v in pmf.values() if v <= pmf[lo_r] * (1 + 1e-7))
print("Hypergeometric P(X = x responders below 40 mg/m2):", {k: round(v, 4) for k, v in pmf.items()})
print(f"Manual two-sided P = {fmt(p2_manual)}; manual P(X <= {lo_r}) = {fmt(sum(pmf[x] for x in range(lo_r + 1)))}")
ci_or = stats.contingency.odds_ratio(tab[::-1], kind="conditional")
lo_ci, hi_ci = ci_or.confidence_interval(0.95)
print(f"Conditional MLE odds ratio (>=40 vs <40) = {ci_or.statistic:.2f}, exact 95% CI {lo_ci:.2f} to {hi_ci:.1f}")
for lab, r, n in (("<40", lo_r, lo_n), (">=40", hi_r, hi_n)):
ci = stats.binomtest(r, n).proportion_ci(0.95, method="exact")
print(f"Clopper-Pearson 95% CI for CR/CRi rate {lab}: {r}/{n} = {r/n:.1%} ({ci.low:.1%} to {ci.high:.1%})")
# ------------------------------------------------- 3. other tests
section("3. Other common tests on the same 2x2 table")
results = {
"Fisher exact, two-sided\n(the stated test)": p2,
"Fisher exact, one-sided\n(reported direction)": p_less,
"Pearson chi-square": stats.chi2_contingency(tab, correction=False)[1],
"Chi-square, Yates": stats.chi2_contingency(tab, correction=True)[1],
"Barnard exact": stats.barnard_exact(tab).pvalue,
"Boschloo exact": stats.boschloo_exact(tab).pvalue,
}
for k in ("Pearson chi-square", "Chi-square, Yates", "Barnard exact", "Boschloo exact"):
print(f"{k}: P = {fmt(results[k])}")
# ------------------------------------------------- 4. trend / all levels
section("4. Tests across the five dose levels")
def ca_trend(ns, rs, s):
s = np.asarray(s, float); n_ = ns.sum(); pbar = rs.sum() / n_
T = np.sum(s * (rs - ns * pbar))
V = pbar * (1 - pbar) * (np.sum(ns * s**2) - np.sum(ns * s) ** 2 / n_)
z = T / np.sqrt(V)
return z, 2 * stats.norm.sf(abs(z))
ca = {}
for name, sc in (("level 1-5", lev), ("dose mg/m2", dose), ("log dose", np.log(dose))):
z, p = ca_trend(ns, rs, sc); ca[name] = p
print(f"Cochran-Armitage, scores = {name:10s}: z = {z:.3f}, two-sided P = {fmt(p)}")
results["Cochran-Armitage trend,\n5 dose levels"] = ca["level 1-5"]
def prob(rr):
return np.prod([comb(int(n_), int(r_)) for r_, n_ in zip(rr, ns)]) / comb(N, R)
p_obs = prob(rs)
ffh = sum(prob(rr) for rr in product(*[range(int(n_) + 1) for n_ in ns])
if sum(rr) == R and prob(rr) <= p_obs * (1 + 1e-9))
print(f"Fisher-Freeman-Halton exact test, 2x5 table: P = {fmt(ffh)}")
results["Freeman-Halton exact,\n2x5 dose levels"] = ffh
# ------------------------------------------------- 5. attainable P values
section("5. Every 2x2 table with the paper's margins (6 vs 18 patients, 8 responders)")
fisher_by_x = {}
for x in range(0, lo_n + 1):
t = [[x, lo_n - x], [R - x, hi_n - (R - x)]]
fisher_by_x[x] = stats.fisher_exact(t)[1]
print(f" {x}/6 below 40 vs {R-x}/18 at or above: two-sided Fisher P = {fmt(fisher_by_x[x])}")
expected_lo = R * lo_n / N
print(f"Expected responders below 40 mg/m2 under no difference: {expected_lo:.1f}")
p_min_dir = min(fisher_by_x[x] for x in range(0, lo_r + 1))
print(f"Smallest P in the reported direction (fewer responders below 40 mg/m2 than expected): {fmt(p_min_dir)} (at 0/6 vs 8/18)")
print(f"Smallest P for any table: {fmt(min(fisher_by_x.values()))} at 6/6 vs 2/18 (the reverse pattern)")
print(f"Tables with P <= 0.0005: {[f'{x}/6 vs {R-x}/18' for x, p in fisher_by_x.items() if p <= REPORTED_P]}")
print("\nKeeping 1/6 below 40 mg/m2 and 18 patients at or above (margins free):")
for k in range(hi_r, hi_n + 1):
p = stats.fisher_exact([[1, 5], [k, hi_n - k]])[1]
if p <= REPORTED_P:
print(f" first k with P <= 0.0005: {k}/18 ({k/18:.0%}), P = {fmt(p)}")
break
print("Keeping 7/18 at or above 40 mg/m2: P for 0/6 below 40 =", fmt(stats.fisher_exact([[0, 6], [7, 11]])[1]))
# ------------------------------------------------- 6. alternative cut-points
section("6. Other dose cut-points (two-sided Fisher)")
for cut in (20, 56, 80):
m = dose < cut
t = [[int(rs[m].sum()), int(ns[m].sum() - rs[m].sum())], [int(rs[~m].sum()), int(ns[~m].sum() - rs[~m].sum())]]
print(f" <{cut} vs >={cut} mg/m2: {t[0][0]}/{sum(t[0])} vs {t[1][0]}/{sum(t[1])}: P = {fmt(stats.fisher_exact(t)[1])}")
# ------------------------------------------------- 7. other comparisons in the paper
section("7. Other comparisons in the paper (could 0.0005 belong to one of them?)")
other = {
"CR/CRi with day-14 CTC (8/12) vs residual leukemia (0/12)": [[8, 4], [0, 12]],
"Day-14 CTC <40 (2/6) vs >=40 (10/18)": [[2, 4], [10, 8]],
"CR/CRi AML (8/21) vs other leukemias (0/3)": [[8, 13], [0, 3]],
"CR/CRi complex karyotype (2/10) vs not (5/14)": [[2, 8], [5, 9]],
}
for k, t in other.items():
p_two = stats.fisher_exact(t)[1]
p_one = min(stats.fisher_exact(t, alternative=a)[1] for a in ("less", "greater"))
print(f" {k}: two-sided P = {fmt(p_two)}, smaller one-sided P = {fmt(p_one)}")
for p0 in (0.05, 0.10):
print(f" Binomial test of 7/18 against a {p0:.0%} response rate: one-sided P = "
f"{fmt(stats.binomtest(7, 18, p0, alternative='greater').pvalue)}")
# ------------------------------------------------- 8. side observations
section("8. Side observations (minor, separate from the P value)")
c = int(ctc[lo].sum())
print(f"Day-14 CTC at <=20 mg/m2: {c}/{lo_n} = {c/lo_n:.1%} (Results text: '2 of 6 (17%)')")
cr4 = sum(r == "CR" for _, r in TABLE4); cri4 = sum(r == "CRi" for _, r in TABLE4)
cr4_l3 = sum(1 for lv, r in TABLE4 if lv == 3 and r == "CR")
print(f"CR / CRi split: Table 3 total {cr3}/{cri3} (Results text: 1 CRi + 5 CR, 2 CRi = 5/3); "
f"Table 4 rows {cr4}/{cri4}; level 3: Table 3 '3 (2/1)', Table 4 {cr4_l3} CR")
print(f"Complex karyotype: 2 of 10 + 5 of 14 = {2+5} responders among {10+14} patients (total responders {R}); "
f"2/10 = {2/10:.0%}, 5/14 = {5/14:.0%}")
print(f"Total day-14 CTC: {ctc.sum()}/{N} = {ctc.sum()/N:.0%}")
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e6e5e1"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK,
"axes.edgecolor": "#bdbcb7", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 1.5,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def plabel(p):
return f"P = {p:.4f}" if p < 0.01 else (f"P = {p:.3f}" if p < 0.1 else f"P = {p:.2f}")
# Figure 1: reported P vs P values recomputed from the paper's own counts, one log axis
rows = [("Reported in the paper\n(Fisher exact test)", REPORTED_P, ORANGE)]
rows += [(k, v, BLUE) for k, v in results.items()]
fig, ax = plt.subplots(figsize=(7.2, 5.0))
ys = np.arange(len(rows))[::-1].astype(float)
ax.axvspan(1e-4, p_min_dir, color="#efeeea", zorder=0, lw=0)
ax.text(1.2e-4, ys[0] + 0.95, f"Shaded: P values no table with these margins\ncan reach in the reported direction (P < {p_min_dir:.3f})",
fontsize=10, color=INK2, va="center", ha="left")
ax.axvline(0.05, color=INK2, lw=1, ls=(0, (3, 3)), zorder=1)
ax.text(0.05 * 0.93, ys[-1] - 0.62, "P = 0.05", ha="right", va="center", color=INK2, fontsize=10)
for y, (lab, p, col) in zip(ys, rows):
ax.plot([p], [y], "o", ms=9, color=col, mec=BG, mew=1.2, zorder=3)
if p < 0.01:
ax.text(p * 1.4, y, plabel(p), va="center", ha="left", fontsize=10)
else:
ax.text(p * 0.8, y, plabel(p), va="center", ha="right", fontsize=10)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xscale("log"); ax.set_xlim(1e-4, 1.6)
ax.set_ylim(ys[-1] - 1.0, ys[0] + 1.6)
ax.grid(axis="x", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("P value (log scale)")
ax.set_title("CR/CRi below vs at or above 40 mg/m²: 1/6 vs 7/18 (Table 3)", loc="left", fontsize=11.5)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported (Discussion)"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE, label="Recomputed from Table 3 counts"),
],
loc="upper center", bbox_to_anchor=(0.3, -0.13), ncol=2, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.2)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: two-sided Fisher P for every table with the paper's margins
fig, ax = plt.subplots(figsize=(7.2, 4.3))
xs = np.array(sorted(fisher_by_x)); ps = np.array([fisher_by_x[x] for x in xs])
for x, p in zip(xs, ps):
obs = (x == lo_r)
ax.plot([x], [p], "o", ms=11 if obs else 8, color=BLUE if obs else BG, mec=BLUE, mew=2, zorder=3)
ax.axhline(REPORTED_P, color=ORANGE, lw=2, zorder=2)
ax.text(2.0, REPORTED_P * 1.3, "Reported P = 0.0005", color=INK, ha="left", va="bottom", fontsize=10)
ax.annotate(f"Observed: 1/6 vs 7/18\n{plabel(fisher_by_x[lo_r])}", xy=(lo_r, fisher_by_x[lo_r]),
xytext=(1.45, 0.07), fontsize=10, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.annotate(f"Most extreme table in the\nreported direction: 0/6 vs 8/18\n{plabel(fisher_by_x[0])}",
xy=(0, fisher_by_x[0]), xytext=(0.1, 0.0028), fontsize=10,
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.annotate(f"All 6 low-dose patients respond\n(reverse pattern), {plabel(fisher_by_x[6])}",
xy=(6, fisher_by_x[6]), xytext=(2.9, 0.00018), fontsize=10, va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.set_yscale("log"); ax.set_ylim(1e-4, 2)
ax.set_xlim(-0.4, 6.4); ax.set_xticks(xs)
ax.set_xticklabels([f"{x}/6\n{R-x}/18" for x in xs], fontsize=10)
ax.set_xlabel("CR/CRi below 40 mg/m² (top line) and at or above 40 mg/m² (bottom line)")
ax.set_ylabel("Two-sided Fisher P (log scale)")
ax.grid(axis="y", which="major", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.set_title("Every possible split of 8 responders between 6 and 18 patients", loc="left", fontsize=11.5)
ax.legend(handles=[Line2D([], [], color=ORANGE, lw=2, label="Reported in the paper"),
Line2D([], [], marker="o", ls="", ms=10, color=BLUE, label="Recomputed, observed table"),
Line2D([], [], marker="o", ls="", ms=8, mfc=BG, mec=BLUE, mew=2, label="Recomputed, other tables")],
loc="upper right", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Abstract reverses the paper's own stiffness ordering of intermediate EMT and mesenchymal cells
Commun Biol (2022) · doi:10.1038/s42003-022-03358-0 · PMC9061773
Summary
The abstract credits the intermediate (nest-like, partial-EMT) phenotype with "increased stiffness (and by inference invasiveness) ... compared to mesenchymal cells". The paper's atomic force microscopy (AFM) result says the opposite, and so do its Introduction, Results and Discussion: epithelial cells are the softest, mesenchymal cells the stiffest, and intermediate cells lie between them. The individual Young's modulus values in the paper's Supplementary Data 1 give means of 212, 560 and 980 Pa at 0 h (epithelial), 14 h (intermediate) and 48 h (mesenchymal), reproduce the three Fig. 5 P values, and put the intermediate-minus-mesenchymal difference at −420 Pa (95% CI −700 to −140). The measurements and the body text are consistent with each other; the abstract, whose only stiffness or invasiveness statement this is, reverses the direction of the measured effect and uses it as the basis for its translational statement.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py.
Abstract:
Further, owing to the increased stiffness (and by inference invasiveness) of the intermediate EMT phenotype compared to mesenchymal cells, our work can be instrumental in aiding the search for future treatment strategies that combat metastasis by specifically targeting the fiber alignment process.
Introduction, last paragraph:
Because the final aligned state is reported to have enhanced invasiveness, it would be advantageous to intervene at the intermediate states in order to prevent invasion.
We have also correlated the intermediate phenotype with lower stiffness compared to mesenchymal cells (stiffer than epithelial cells) supporting their partial EMT properties.
Results, section "The early EMT phenotype demonstrates a partial EMT nature and has different elastic properties compared to mesenchymal cells":
Due to the extensive interrelationship between the actin cytoskeleton and elastic properties of cells, the nest-like phenotype can also be expected to demonstrate a partial epithelial nature and be more compliant compared to the mesenchymal phenotype.
The epithelial cells had the lowest Young’s modulus and the late EMT mesenchymal phenotype had the highest Young’s modulus. The Young’s modulus of the nest-like phenotype was intermediate between the two (Fig. 5).
Fig. 5 legend:
Young’s Modulus values at 0 (blue) (n = 15), 14 (brown) (n = 13), and 48 (green) (n = 14) hours after EMT induction using TGFβ1 showing their mean and standard errors along with a plot of the mean values.
P1 = 0.0012, P2 = 0.0048, P3 = 3.6 × 10-9.
Methods, "Statistics and reproducibility":
We have carried out the two-sample t tests using the “ttest2” function in the Statistics and Machine Learning Toolbox in Matlab.
Discussion, first paragraph:
We demonstrated that the nest-like phenotype occupies an intermediate elastic niche between epithelial and mesenchymal cells.
As the cytoskeleton extensively affects the elasticity and motility of cells, we can expect the motility and invasiveness of the intermediate phenotype to be in between that of the epithelial and mesenchymal phenotypes.
Discussion, closing summary:
To summarize, in this work, we discovered a partial EMT phenotype in lung cancer cells with a unique cytoskeletal signature which are consistent with decreased stiffness compared to mesenchymal cells.
What the paper's own numbers show
Method. The main text gives no numeric moduli, so I used the paper's Supplementary Data 1 (42003_2022_3358_MOESM5_ESM.xlsx), whose sheet "Figure 5" lists each cell's Young's modulus in Pa at 0, 14 and 48 h after TGFβ1. For each time point I computed the mean, SD and SE. I then ran pooled-variance two-sample t tests (the default of Matlab's ttest2, named in the Methods) and pooled-variance 95% confidence intervals for the differences in means, and compared the results with the Fig. 5 legend and with the direction each passage states. recompute.py.txt holds the 42 values and prints every number below; given the .xlsx path, it also re-reads the file and checks the values against its copy.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Direction, intermediate vs mesenchymal | Abstract | "increased stiffness ... compared to mesenchymal cells" (EI > EM) | EI < EM: 560.0 vs 980.2 Pa |
| Direction, intermediate vs mesenchymal | Introduction; Results; Discussion (twice) | lower; more compliant; intermediate; decreased (EI < EM) | EI < EM (agrees) |
| Sample sizes | Fig. 5 legend | n = 15, 13, 14 | 15, 13, 14 values in Supplementary Data 1 |
| Young's modulus, 0 h (epithelial) | Fig. 5 (bars only) | not given in text | mean 212.4 ± 15.5 Pa (SE; SD 60.0) |
| Young's modulus, 14 h (intermediate, nest-like) | Fig. 5 (bars only) | not given in text | mean 560.0 ± 101.0 Pa (SE; SD 364.0) |
| Young's modulus, 48 h (mesenchymal) | Fig. 5 (bars only) | not given in text | mean 980.2 ± 91.5 Pa (SE; SD 342.4) |
| P1, 0 h vs 14 h | Fig. 5 legend | 0.0012 | 0.00115 (t = −3.65, df 26); EI − EE = +347.6 Pa, 95% CI 151.9 to 543.2 |
| P2, 14 h vs 48 h | Fig. 5 legend | 0.0048 | 0.00485 (t = −3.09, df 25); EI − EM = −420.2 Pa, 95% CI −700.2 to −140.2 |
| P3, 0 h vs 48 h | Fig. 5 legend | 3.6 × 10-9 | 3.6 × 10-9 (t = −8.56, df 27); EM − EE = +767.8 Pa, 95% CI 583.7 to 951.9 |
| Ratio EI / EM | — | abstract implies > 1 | 0.571 (the intermediate mean is 57% of the mesenchymal mean) |
| Intermediate cells below the mesenchymal mean (980.2 Pa) | — | — | 12 of 13 |
The source data reproduce all three legend P values to the printed precision, so they are the data behind Fig. 5. Unequal-variance (Welch) tests give P = 0.0049, 0.0050 and 1.05 × 10-6, and Mann-Whitney tests give P = 2.3 × 10-5, 0.0015 and 5.1 × 10-6. Under every test the intermediate phenotype is significantly less stiff than the mesenchymal phenotype, and the 95% CI of the difference lies entirely below zero. Of the five passages that state a direction, the abstract is the only one that contradicts these data.
Figure
Figure 1. The conflicting statements. Left (orange frame): the abstract excerpt, which states that the intermediate phenotype is stiffer than mesenchymal cells. Right: short excerpts from the Introduction, Results and Discussion, which all place the intermediate phenotype below mesenchymal and above epithelial cells. The orange "Stated" lines give the direction each side reports. Bottom right (blue): values recomputed by recompute.py.txt from the paper's Supplementary Data 1. The mean moduli rise from 212 to 560 to 980 Pa, and the 14 h minus 48 h difference is −420 Pa with a pooled t test P value of 0.0048, the legend's P2. The source data agree with the body text and contradict the abstract.
Figure 2. Differences in mean Young's modulus with pooled-variance 95% confidence intervals, recomputed from Supplementary Data 1 (blue), on one linear axis in Pa with zero as the reference line. Top row: intermediate minus epithelial, +348 Pa (95% CI +152 to +543), consistent with "stiffer than epithelial cells". Bottom row: intermediate minus mesenchymal, −420 Pa (95% CI −700 to −140). The orange arrow marks the side of zero the abstract's statement requires (difference > 0); the recomputed interval lies wholly on the other side.
Why it matters
- The abstract reverses a measured effect. This is the only stiffness or invasiveness statement in the abstract. It says the intermediate phenotype is stiffer than mesenchymal cells, but the AFM data show it is significantly softer (P = 0.0048), at 57% of the mesenchymal mean. A reader who sees only the abstract, or an indexing service or citing paper that relies on it, gets the opposite of the paper's finding.
- The reversed direction carries an inference about invasiveness. The abstract presents the intermediate phenotype as stiffer "and by inference" more invasive than mesenchymal cells. The paper's own reasoning in the Discussion runs the other way: invasiveness is expected "to be in between that of the epithelial and mesenchymal phenotypes", that is, lower than in mesenchymal cells. Corrected, the abstract's inference about invasiveness also changes direction. Invasiveness itself was not measured.
- The translational rationale rests on the reversed statement. The abstract gives the stiffness and invasiveness claim as the reason ("owing to") the work could help target fiber alignment to combat metastasis. The Introduction gives a rationale consistent with the data: the final aligned state "is reported to have enhanced invasiveness", so intervening at the intermediate states would be advantageous.
- The paper's main contributions are not affected. These are the SPOCC tool, the orientational order parameter as an EMT reporter, the split of EMT into stress-fiber formation followed by alignment, and the existence of a non-aligned intermediate phenotype. None of them depends on the abstract sentence. The AFM measurements, the statistics and the body text are internally consistent. The error is confined to the wording of the abstract.
Explanations considered
- A different population, time point or dataset. Does not account for it. There is a single AFM dataset: one TGFβ1 time course at 0 h (epithelial, n = 15), 14 h (nest-like or intermediate, n = 13) and 48 h (mesenchymal, n = 14). Supplementary Data 1 holds exactly these 42 values and reproduces the legend P values. No other comparison in the paper could make the intermediate phenotype stiffer than mesenchymal cells.
- "Intermediate EMT phenotype" means something else in the abstract. Does not account for it. The abstract uses the term for the intermediate EMT state it introduces in its second sentence, which the body calls the I or nest-like state. The abstract's words "compared to mesenchymal cells" rule out a comparison with epithelial cells, against which the intermediate phenotype is indeed stiffer.
- Stiffness read as compliance (an inverse quantity). Does not account for it. The abstract says "stiffness" outright and ties it to invasiveness. The Results use "compliant" only for the expectation that the nest-like phenotype is softer than mesenchymal cells, which again means EI < EM.
- The body text, not the abstract, is wrong. Ruled out by the source data. The individual moduli give EI < EM with a 95% CI of −700 to −140 Pa, and 12 of the 13 intermediate cells lie below the mesenchymal mean.
- A text-conversion or typesetting artifact. Ruled out. The discrepancy is in words ("increased" versus "lower" or "decreased"), not in a sign, exponent or table cell. The publisher's HTML, the PMC page and the Europe PMC full-text XML carry the same wording.
- Wording carried over from an earlier version. This bears on how the sentence arose, not on whether it is correct. The bioRxiv preprint (doi:10.1101/2021.06.17.448784, version 1, 17 June 2021) has the same sentence with "hence" in place of "by inference" and "intermediate phenotype" in place of "intermediate EMT phenotype". In the Peer Review File, a reviewer quotes the sentence and asks only that the invasiveness inference be reworded; the revised abstract changed "hence" to "by inference", and the comparison direction stayed as it was. In the same file, a reviewer's summary describes the intermediate phenotype as "associated with decreased invasiveness", which fits the Results, not the abstract.
- A correction already issued. None found (see Status).
Status
- Version of record. The abstract, Introduction, Results and Discussion passages quoted above are worded identically in the publisher's HTML (nature.com/articles/s42003-022-03358-0, read after its cookie redirect), the PMC page and the Europe PMC full-text XML. Every quotation was also checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The PMC XML lists six supplementary items: the Peer Review File, Supplementary Information, a description of additional files, Supplementary Video 1, Supplementary Data 1 (XLSX) and the Reporting Summary. Supplementary Data 1 and the Peer Review File were downloaded from static-content.springer.com and read. Supplementary Data 1, sheet "Figure 5", resolves the question numerically, as shown above. Its SHA-256 (b991028077911ce5b8d6259b1c87ee5f1e92e5362830d8e5fdca7f94afc5bf7e) is recorded in
recompute.py.txtso the copy can be checked. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by entries; its relations list only the bioRxiv preprint and a Faculty Opinions review. The publisher's page shows no correction notice. On PubPeer, an automated DOI query to the API returned no record (HTTP 404), and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC9061773; Europe PMC full-text XML; publisher page https://doi.org/10.1038/s42003-022-03358-0; Supplementary Data 1 (
42003_2022_3358_MOESM5_ESM.xlsx) and Peer Review File (42003_2022_3358_MOESM1_ESM.pdf); bioRxiv preprint 10.1101/2021.06.17.448784 (via api.biorxiv.org); Crossref; Europe PMC; PubPeer.
Suggested correction
In the abstract, replace
Further, owing to the increased stiffness (and by inference invasiveness) of the intermediate EMT phenotype compared to mesenchymal cells,
with wording that matches Fig. 5 and the body text, for example
Further, as the intermediate EMT phenotype has decreased stiffness compared to mesenchymal cells (and by inference lower invasiveness),
The rest of the sentence may then need rephrasing so that its rationale (targeting fiber alignment to combat metastasis) follows from the corrected direction; the Introduction already gives such a rationale, that the aligned mesenchymal state is reported to be more invasive and that intervening at the intermediate stage would therefore be advantageous. Stating the group means in the text would also help readers, for example mean ± SE Young's modulus of 212 ± 15, 560 ± 101 and 980 ± 92 Pa at 0, 14 and 48 h, as computed here from Supplementary Data 1.
Recomputation script (python recompute.py.txt)
"""Recompute the stiffness comparison of PMC9061773
(Commun Biol 2022, doi:10.1038/s42003-022-03358-0,
"Statistical parametrization of cell cytoskeleton reveals lung cancer cytoskeletal
phenotype with partial EMT signature").
Run: python recompute.py.txt [path/to/42003_2022_3358_MOESM5_ESM.xlsx]
With the optional .xlsx path (needs openpyxl), the values below are re-read from the
file and checked against this copy (SHA-256 of the file used:
b991028077911ce5b8d6259b1c87ee5f1e92e5362830d8e5fdca7f94afc5bf7e).
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper:
* Supplementary Data 1 (42003_2022_3358_MOESM5_ESM.xlsx), sheet "Figure 5":
individual Young's modulus values in Pa at 0 h (epithelial), 14 h (intermediate,
nest-like) and 48 h (mesenchymal) after TGF-beta1 induction.
* Fig. 5 legend: n = 15, 13, 14; "P1 = 0.0012, P2 = 0.0048, P3 = 3.6 × 10-9".
* Methods: two-sample t tests with Matlab "ttest2" (equal variances).
* Text excerpts (Abstract, Introduction, Results, Discussion), checked verbatim
against the PMC-archived text.
"""
import os
import sys
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Supplementary Data 1, sheet "Figure 5", Young's modulus (Pa)
E0 = [286.1, 366.3, 115, 269.4, 184.2, 169.3, 159.3, 189.2, 170.2, 227, 220.9, 210.5,
210.7, 192.8, 214.9] # 0 h, epithelial
I14 = [527.5, 544.7, 274.3, 354.9, 942.3, 1618, 307, 413.6, 649.2, 461.7, 294.3, 451.1,
441.1] # 14 h, intermediate
M48 = [899.8, 1804, 561.7, 817, 758.6, 888.5, 1256, 1364, 744.2, 936.1, 900.9, 538.8,
960.8, 1292] # 48 h, mesenchymal
GROUPS = {"epithelial (0 h)": np.array(E0),
"intermediate (14 h)": np.array(I14),
"mesenchymal (48 h)": np.array(M48)}
N_LEGEND = {"epithelial (0 h)": 15, "intermediate (14 h)": 13, "mesenchymal (48 h)": 14}
P_LEGEND = {"P1": 0.0012, "P2": 0.0048, "P3": 3.6e-9}
if len(sys.argv) > 1:
import hashlib
import openpyxl
print("SHA-256:", hashlib.sha256(open(sys.argv[1], "rb").read()).hexdigest())
ws = openpyxl.load_workbook(sys.argv[1], data_only=True)["Figure 5"]
cols = [[], [], []]
for row in ws.iter_rows(min_row=3, values_only=True):
for j in range(3):
if isinstance(row[2 + j], (int, float)):
cols[j].append(float(row[2 + j]))
for j, (k, x) in enumerate(GROUPS.items()):
assert np.allclose(cols[j], x), k
print("Values re-read from the .xlsx match the copy in this script.\n")
# Direction of (E_intermediate - E_mesenchymal) asserted by each passage:
# +1 = intermediate stiffer, -1 = mesenchymal stiffer
STATED_SIGN = {
"Abstract": +1, # "increased stiffness ... of the intermediate EMT phenotype compared to mesenchymal cells"
"Introduction p5": -1, # "lower stiffness compared to mesenchymal cells (stiffer than epithelial cells)"
"Results (Fig. 5)": -1, # "mesenchymal phenotype had the highest Young's modulus ... nest-like ... intermediate"
"Discussion p0": -1, # "intermediate elastic niche between epithelial and mesenchymal cells"
"Discussion p8": -1, # "decreased stiffness compared to mesenchymal cells"
}
# ---------------------------------------------------------------- 1. group summaries
print("=== 1. Young's modulus by group (Supplementary Data 1, sheet 'Figure 5') ===")
S = {}
for k, x in GROUPS.items():
n = len(x)
m, sd = x.mean(), x.std(ddof=1)
se = sd / np.sqrt(n)
t = stats.t.ppf(0.975, n - 1)
S[k] = dict(n=n, mean=m, sd=sd, se=se, lo=m - t * se, hi=m + t * se, med=np.median(x))
print(f"{k:22s} n={n:2d} (legend n={N_LEGEND[k]}) mean={m:7.1f} Pa SD={sd:6.1f} "
f"SE={se:6.1f} 95% CI {m - t * se:6.1f} to {m + t * se:6.1f} median={np.median(x):6.1f}")
e, i, mz = (S[k]["mean"] for k in GROUPS)
print(f"Ordering of means: E < I < M ? {e < i < mz}")
print(f"Intermediate mean / mesenchymal mean = {i / mz:.3f} ({100 * i / mz:.0f}%)")
print(f"Intermediate mean / epithelial mean = {i / e:.2f}")
nbelow = int((GROUPS['intermediate (14 h)'] < mz).sum())
print(f"Intermediate cells below the mesenchymal mean: {nbelow} of {len(I14)}")
nabove = int((GROUPS['intermediate (14 h)'] > S['mesenchymal (48 h)']['med']).sum())
print(f"Intermediate cells above the mesenchymal median ({S['mesenchymal (48 h)']['med']:.1f} Pa): "
f"{nabove} of {len(I14)}")
# ---------------------------------------------------------------- 2. reproduce the legend P values
print("\n=== 2. Fig. 5 legend P values: reported vs two-sample t test (equal variances) ===")
pairs = {"P1": ("epithelial (0 h)", "intermediate (14 h)"),
"P2": ("intermediate (14 h)", "mesenchymal (48 h)"),
"P3": ("epithelial (0 h)", "mesenchymal (48 h)")}
DIFF = {}
for lab, (a, b) in pairs.items():
xa, xb = GROUPS[a], GROUPS[b]
tt = stats.ttest_ind(xa, xb) # pooled variance, as ttest2
w = stats.ttest_ind(xa, xb, equal_var=False)
mw = stats.mannwhitneyu(xa, xb, alternative="two-sided")
na, nb = len(xa), len(xb)
sp = np.sqrt(((na - 1) * xa.var(ddof=1) + (nb - 1) * xb.var(ddof=1)) / (na + nb - 2))
se_d = sp * np.sqrt(1 / na + 1 / nb)
d = xa.mean() - xb.mean()
tc = stats.t.ppf(0.975, na + nb - 2)
DIFF[lab] = (d, d - tc * se_d, d + tc * se_d)
print(f"{lab}: {a} vs {b}: reported {P_LEGEND[lab]:.2g}; t test P = {tt.pvalue:.3g} "
f"(t = {tt.statistic:.2f}, df {na + nb - 2}); Welch P = {w.pvalue:.3g}; Mann-Whitney P = {mw.pvalue:.3g}")
print(f" difference {a.split()[0]} - {b.split()[0]} = {d:.1f} Pa, "
f"95% CI {d - tc * se_d:.1f} to {d + tc * se_d:.1f} (pooled)")
dIM = DIFF["P2"] # intermediate - mesenchymal
dIE = -DIFF["P1"][0], -DIFF["P1"][2], -DIFF["P1"][1] # intermediate - epithelial
print(f"Ratio intermediate / mesenchymal mean = {i / mz:.3f}")
print(f"\nIntermediate - mesenchymal = {dIM[0]:.1f} Pa (95% CI {dIM[1]:.1f} to {dIM[2]:.1f})")
print(f"Intermediate - epithelial = {dIE[0]:.1f} Pa (95% CI {dIE[1]:.1f} to {dIE[2]:.1f})")
# ---------------------------------------------------------------- 3. direction check across the text
print("\n=== 3. Direction of (intermediate - mesenchymal) stated in each passage ===")
data_sign = int(np.sign(dIM[0]))
for k, s in STATED_SIGN.items():
agree = "agrees with" if s == data_sign else "CONTRADICTS"
print(f"{k:18s} states {'> 0 (intermediate stiffer)' if s > 0 else '< 0 (mesenchymal stiffer)'}"
f" -> {agree} Supplementary Data 1")
print("Passages contradicting the data:", [k for k, s in STATED_SIGN.items() if s != data_sign])
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: abstract vs body text diagram
fig = plt.figure(figsize=(7.2, 4.4), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.4)
ax.axis("off")
def panel(x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.14, y + h - 0.16, title, fontsize=11, fontweight="bold", va="top", color=TXT)
yy = y + h - 0.52
for txt, col, style in lines:
ax.text(x + 0.14, yy, txt, fontsize=10, va="top", color=col, style=style)
yy -= 0.235 * (txt.count("\n") + 1) + 0.09
panel(0.15, 2.0, 2.9, 2.25, "Abstract (as published)", [
("“owing to the increased stiffness\n(and by inference invasiveness)\nof the "
"intermediate EMT\nphenotype compared to\nmesenchymal cells”", TXT, "italic"),
("Stated: intermediate > mesenchymal", ORANGE, "normal"),
], ORANGE)
panel(3.8, 1.25, 3.25, 3.0, "Introduction, Results, Discussion", [
("Intro: “lower stiffness compared to\nmesenchymal cells (stiffer than\n"
"epithelial cells)”", TXT, "italic"),
("Results: epithelial “lowest”, mesenchymal\n“highest”, nest-like "
"“intermediate”", TXT, "italic"),
("Discussion: “decreased stiffness\ncompared to mesenchymal cells”", TXT, "italic"),
("Stated: intermediate < mesenchymal", ORANGE, "normal"),
], SEC)
ax.add_patch(FancyBboxPatch((3.8, 0.12), 3.25, 0.95, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=BLUE, lw=1.5))
ax.text(3.94, 0.95, "Recomputed (Supplementary Data 1)", fontsize=10, fontweight="bold",
va="top", color=BLUE)
ax.text(3.94, 0.66, f"means {e:.0f} < {i:.0f} < {mz:.0f} Pa at 0, 14, 48 h\n"
f"14 h − 48 h: −{-dIM[0]:.0f} Pa, t test P = {stats.ttest_ind(I14, M48).pvalue:.4f}",
fontsize=10, va="top", color=BLUE)
arr = FancyArrowPatch((3.07, 3.0), (3.78, 3.0), arrowstyle="<->", mutation_scale=14,
lw=2, color=TXT)
ax.add_patch(arr)
ax.text(3.425, 3.12, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
color=TXT)
ax.text(0.15, 1.75, "The abstract is the only passage\nthat puts the intermediate phenotype\n"
"above mesenchymal cells. The body\ntext and the source data agree\nwith each other.",
fontsize=10, va="top", color=SEC)
save(fig, "fig1")
# Figure 2: the intermediate - mesenchymal contrast on one axis
fig, ax = plt.subplots(figsize=(7.2, 3.1), facecolor=BG)
fig.subplots_adjust(left=0.20, right=0.97, top=0.80, bottom=0.17)
ax.set_facecolor(BG)
rows = [("intermediate\n− epithelial", dIE, 1.0), ("intermediate\n− mesenchymal", dIM, 0.0)]
for lab, (d, lo, hi), y in rows:
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(d, y, "o", ms=9, color=BLUE, zorder=3)
fm = lambda v: f"{v:+.0f}".replace("-", "−")
ax.text(lo, y + 0.2, f"{fm(d)} Pa (95% CI {fm(lo)} to {fm(hi)})", ha="left",
va="bottom", fontsize=10, color=BLUE)
ax.axvline(0, color=SEC, lw=0.8)
ax.annotate("", xy=(700, 0.0), xytext=(40, 0.0),
arrowprops=dict(arrowstyle="->", color=ORANGE, lw=2))
ax.text(60, -0.12, "Abstract: intermediate stiffer than\nmesenchymal, i.e. difference > 0", va="top",
fontsize=10, color=ORANGE)
ax.set_yticks([1.0, 0.0])
ax.set_yticklabels([r[0] for r in rows], fontsize=10.5, color=TXT)
ax.set_ylim(-0.75, 1.5)
ax.set_xlim(-800, 1000)
ax.set_xlabel("Difference in mean Young's modulus (Pa)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
fig.text(0.20, 0.92, "Blue: recomputed from Supplementary Data 1 (difference in means, 95% CI)\n"
"Orange: direction the abstract states for intermediate − mesenchymal",
fontsize=10, color=SEC, va="center")
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract and Results report different median CCK iAUC values for the same before–after comparison
Diabetes Care (2012) · doi:10.2337/dc11-2050 · PMC3329824
Summary
The paper reports the meal-test incremental area under the curve (iAUC) for cholecystokinin (CCK) in 20 subjects before (day 14) and at the end (day 70) of colesevelam treatment, once in the Abstract and once in the Results text. The two passages give different numbers for the same comparison: 43.2 [0–130.1] to 127.1 [47.2–295.2] pmol/L·min in the Abstract, and 27.5 [2.4–109.2] to 200.3 [40.3–300] pmol/L·min in the Results. All six medians and quartiles differ, and the implied ratio of medians is about 2.9-fold in one version and about 7.3-fold in the other. The three other iAUCs reported in both places (glucose, insulin, FGF-19) match digit for digit. Both versions show a significant increase, so the direction of the finding holds either way, but at least one set of CCK figures is wrong, and the CCK increase is the mechanism the Abstract's conclusion invokes.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py.
Abstract, Results:
iAUC for cholecystokinin (CCK) increased (from 43.2 [0–130.1] to 127.1 [47.2–295.2] pmol/L ⋅ min, P < 0.01)
Results, "Effect of colesevelam on the release of gastrointestinal tract–related peptides":
Treatment with colesevelam was associated with increases in CCK during the MTT at 60, 90, and 120 min, the result being a significant increase in iAUC (from 27.5 [2.4–109.2] to 200.3 [40.3–300] pmol/L ⋅ min, P = 0.001). Basal MTT CCK concentrations did not differ before and after treatment.
Abstract, Conclusions:
This effect may be at least partially explained by the colesevelam-induced increase in CCK.
Methods, Calculations (one CCK iAUC is defined):
In a similar manner, iAUCs were calculated for the gastrointestinal peptides GLP-1, GIP, PYY, and CCK, as well as for FGF-19 and FGF-21.
Methods, MTT (one sampling window):
Blood samples were obtained at 5 and 1 min before the meal and every 30 min for 120 min after starting the meal.
Methods, Statistical analysis:
Unless otherwise specified, all data are mean ± SEM for normally distributed variables and median (interquartile range) for nonnormally distributed data.
For comparison, two other iAUCs that appear in both places. Results, "Effect of colesevelam on glucose tolerance, insulin sensitivity, and ..." and "... gastrointestinal tract–related peptides":
colesevelam administration was associated with decreases in the iAUC for both insulin (from 20,130 [13,542–35,292] to 13,086 [9,804–21,138] pmol/L ⋅ min, P < 0.05)
The iAUC for FGF-19 was significantly lower with colesevelam treatment (from 11,185 [1,346–17,661] to 2,093 [673–6,707] pg/mL ⋅ min, P < 0.01)
The Abstract gives the same insulin and FGF-19 values, and the same glucose iAUC (249.3 ± 28.5 to 198.8 ± 23.6 mmol/L ⋅ min, P < 0.01) as the Results.
What the paper's own numbers show
Method. No individual-level data are available, so the check is a comparison of the reported summaries. For each iAUC reported in both the Abstract and the Results, I compared the six reported numbers (median and quartiles, or mean and SEM, at day 14 and day 70) field by field. For CCK, I computed the ratio and the difference of the reported medians in each version, the Results-to-Abstract ratio for each field (a unit or scaling difference would make these ratios equal), and whether each version's medians fall inside the other's IQR. Ratios and differences of medians describe the reported group summaries; they are not the median paired change, which the paper does not report. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| CCK iAUC, day 14, median [IQR] (pmol/L·min) | Abstract | 43.2 [0–130.1] | differs from Results in all 3 numbers: median −15.7, Q1 +2.4, Q3 −20.9 |
| CCK iAUC, day 14, median [IQR] (pmol/L·min) | Results | 27.5 [2.4–109.2] | — |
| CCK iAUC, day 70, median [IQR] (pmol/L·min) | Abstract | 127.1 [47.2–295.2] | differs from Results in all 3 numbers: median +73.2, Q1 −6.9, Q3 +4.8 |
| CCK iAUC, day 70, median [IQR] (pmol/L·min) | Results | 200.3 [40.3–300] | — |
| CCK ratio of medians, day 70 / day 14 | Abstract | — | ×2.94 |
| CCK ratio of medians, day 70 / day 14 | Results | — | ×7.28 (2.48 times the Abstract's ratio) |
| CCK difference of medians | Abstract | — | +83.9 pmol/L·min |
| CCK difference of medians | Results | — | +172.8 pmol/L·min (2.06 times the Abstract's) |
| Results / Abstract, per CCK field | both | — | 0.637 (day-14 median), 0.839 (day-14 Q3), 1.576 (day-70 median), 0.854 (day-70 Q1), 1.016 (day-70 Q3); day-14 Q1 is 0 in the Abstract. No common factor |
| CCK P value | Abstract; Results | P < 0.01; P = 0.001 | compatible with each other |
| Glucose iAUC, mean ± SEM (mmol/L·min) | Abstract; Results | 249.3 ± 28.5 to 198.8 ± 23.6 (both) | 0 of 6 fields differ; ratio of means ×0.80 in both |
| Insulin iAUC, median [IQR] (pmol/L·min) | Abstract; Results | 20,130 [13,542–35,292] to 13,086 [9,804–21,138] (both) | 0 of 6 fields differ; ratio of medians ×0.65 in both |
| FGF-19 iAUC, median [IQR] (pg/mL·min) | Abstract; Results | 11,185 [1,346–17,661] to 2,093 [673–6,707] (both) | 0 of 6 fields differ; ratio of medians ×0.19 in both |
| Mean CCK increment above basal implied by the median iAUC over 120 min | Abstract | — | 0.36 pmol/L (day 14), 1.06 pmol/L (day 70) |
| Same | Results | — | 0.23 pmol/L (day 14), 1.67 pmol/L (day 70) |
Each version's day-14 and day-70 medians fall inside the other version's IQR at the same time point, and both versions describe a plausible range for a 120-minute iAUC given the assay's stated detection limit of 0.3 pmol/L. So neither version is internally impossible, and the reported summaries cannot show which one is correct. They do show that the two cannot both be right: the same 20 paired measurements have one median and one pair of quartiles at each time point.
Figure
Figure 1. Top: the two passages that report the meal-test CCK iAUC, quoted verbatim from the Abstract and from the Results subsection on gastrointestinal peptides. Both describe the same endpoint in the same 20 subjects, day 14 (end of placebo run-in) against day 70 (end of colesevelam). Below each passage, in blue, is the ratio of day-70 to day-14 medians recomputed from its numbers: ×2.9 for the Abstract and ×7.3 for the Results. Bottom: the reported medians (markers) and interquartile ranges (lines) on one linear axis starting at 0, in orange because all are values printed in the paper. Filled circles are the Abstract's values and open squares the Results' values. At each time point the two versions give different medians and different quartiles.
Figure 2. Ratio of day-70 to day-14 summaries (log scale; the thin vertical line marks no change) for every meal-test iAUC that the paper reports in both the Abstract and the Results, recomputed from the reported values (blue). Filled circles use the Abstract's values; open diamonds use the Results' values. For glucose (ratio of means), insulin and FGF-19 (ratios of medians), the two sources give identical numbers, so the markers coincide at ×0.80, ×0.65 and ×0.19. For CCK they give ×2.94 and ×7.28. The mismatch is confined to CCK.
Why it matters
- The CCK result is the paper's proposed mechanism. The Abstract concludes that the improvement in oral glucose tolerance "may be at least partially explained by the colesevelam-induced increase in CCK", and the Discussion builds on it: delayed gastric emptying, suppression of hepatic glucose output, and the statement "we favor the change in CCK being the primary event". The numbers behind that increase are given two different ways.
- The size of the effect is uncertain by a factor of about 2.5. The ratio of medians is ×2.94 in the Abstract and ×7.28 in the Results; the difference of medians is +83.9 and +172.8 pmol/L·min. A reader comparing the CCK rise with physiological CCK infusion studies, as the Discussion does, or using the figure in a review or meta-analysis, cannot tell which magnitude the data support.
- The two versions circulate separately. The PubMed abstract, which indexing services reproduce, carries the Abstract's values (43.2 to 127.1), while the full text carries the other set (27.5 to 200.3).
- What is not affected. Both versions show an increase that is significant (P = 0.001 also satisfies P < 0.01), so the qualitative statement that postprandial CCK rose with colesevelam holds under either. The glucose, insulin and FGF-19 iAUCs, the null FSIGT results, and the title claim (independence from insulin sensitivity and β-cell function) are reported consistently and do not depend on the CCK figures. CCK is a secondary, mechanistic measure in this study.
Explanations considered
- Rounding or a typographical slip in one digit. Does not account for it. All six numbers differ, by up to 73.2 pmol/L·min, which is far beyond rounding at one decimal place. A single transposed digit would change one number, not six.
- Different units or a scaling factor. Does not account for it. Both passages state pmol/L·min. The Results-to-Abstract ratios differ across fields (0.637 for the day-14 median, 1.576 for the day-70 median, 0.839 to 1.016 for the quartiles), whereas a unit conversion would multiply every number by the same factor.
- A different population or subset. Not supported. The paper reports one analysed cohort (21 enrolled, 20 completed) and no CCK-specific exclusions, missing samples or subset analysis.
- Different time points. Not supported. Both passages compare the same two visits; the Methods and the Figure 3 legend define them as day 14 (before colesevelam) and day 70 (end of treatment).
- A different iAUC definition or time window. Not supported. The Methods define one iAUC per analyte by the trapezoidal method, and the MTT has one sampling schedule (−5 and −1 min, then every 30 min to 120 min). No alternative window, baseline definition or adjusted value is described.
- Mean versus median, or log-transformed values. Not supported. Both passages are in median [IQR] form, as the Methods specify for non-normal data. Log transformation does not change the ordering of the observations, so back-transformed medians and quartiles could differ from the raw ones only through interpolation between neighbouring observations (n = 20), not by 36% to 58% at both time points.
- The P values identify the correct version. They do not. "P < 0.01" and "P = 0.001" are compatible, so they cannot discriminate between the two sets.
- A conversion or typesetting artifact in the archived copy. Ruled out. The same Abstract values appear in the live PMC page, the Europe PMC full-text XML and the publisher-supplied PubMed abstract (PMID 22446171), and the same Results values appear in the PMC page and the Europe PMC XML.
- A systematic difference between the Abstract and the body. Not supported. Every other iAUC reported in both places (glucose, insulin, FGF-19) matches exactly, so the discrepancy is confined to CCK.
- The paper's other material resolves it. It does not. Table 1 has no CCK values, Figure 3D shows the CCK curves without numbers (and plots time courses, not iAUC medians), no supplementary material is listed, and the ClinicalTrials.gov record (NCT00990184) posts results only for AIRg, SI and Kg.
Status
- Version of record. The publisher's page (doi.org redirects to diabetesjournals.org) returned HTTP 403 to automated fetches, as did its PDF links. The live PMC page and the Europe PMC full-text XML carry the publisher's typeset text, and both contain the two CCK passages exactly as quoted. The PubMed abstract (PMID 22446171) has the Abstract's values (43.2 [0-130.1] to 127.1 [47.2-295.2], P < 0.01).
- Supplements. None listed on PMC or in the JATS XML; the Europe PMC supplementary-files endpoint returned 404. ClinicalTrials.gov NCT00990184 has posted results but no CCK data.
- Corrections and comments. As of 2026-10-24, no correction, erratum, retraction or expression of concern was found.
src/known.pyreturns no Crossref updates, PMC notices or Europe PMC notices; the Crossref record has no update relations; the PubMed record has no CommentsCorrections entries; the Retraction Watch database (Crossref Labs export) has no entry for the DOI or PMID. The PubPeer API returned no record (HTTP 404) for the DOI, and web searches for the title with "erratum", "correction" or "PubPeer" found none. - Sources. PMC3329824 (via
src/show.py); https://pmc.ncbi.nlm.nih.gov/articles/PMC3329824/; Europe PMC full-text XML for PMC3329824; PubMed PMID 22446171; https://doi.org/10.2337/dc11-2050; Crossref; Retraction Watch database; PubPeer; https://clinicaltrials.gov/study/NCT00990184.
Suggested correction
The authors may wish to check the CCK iAUC summaries against the source data and correct whichever passage is wrong, so that the Abstract and the Results report the same values. If the Results values are correct, the Abstract would read:
iAUC for cholecystokinin (CCK) increased (from 27.5 [2.4–109.2] to 200.3 [40.3–300] pmol/L ⋅ min, P = 0.001)
If the Abstract values are correct, the Results sentence would read "... a significant increase in iAUC (from 43.2 [0–130.1] to 127.1 [47.2–295.2] pmol/L ⋅ min, P = ...)", with the exact P value. A correction notice could also give the median of the paired within-subject change (or the Wilcoxon test statistic), which would let readers judge the size of the CCK increase that the mechanistic conclusion rests on.
Recomputation script (python recompute.py.txt)
"""Compare the two CCK iAUC results reported in PMC3329824
(Diabetes Care 2012, doi:10.2337/dc11-2050).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/fig1.png and fig2.svg/fig2.png next to this file.
All inputs are typed from the paper's text (PMC archive, checked with src/show.py):
the Abstract (Results paragraph) and the Results subsections "Effect of colesevelam
on glucose tolerance, insulin sensitivity, and ..." and "Effect of colesevelam on the
release of gastrointestinal tract-related peptides". Values are median [IQR]
except glucose iAUC, which is mean +/- SEM. Before = day 14 (end of placebo run-in),
after = day 70 (end of 8 weeks of colesevelam), n = 20.
No individual-level data are available, so only the reported summaries are compared.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
# (before centre, before lo, before hi, after centre, after lo, after hi, P as printed)
# For glucose, lo/hi hold the SEM (centre +/- SEM is not used further).
REPORTED = {
"Glucose iAUC (mmol/L·min, mean ± SEM)": {
"Abstract": (249.3, 28.5, None, 198.8, 23.6, None, "P < 0.01"),
"Results": (249.3, 28.5, None, 198.8, 23.6, None, "P < 0.01"),
},
"Insulin iAUC (pmol/L·min, median [IQR])": {
"Abstract": (20130, 13542, 35292, 13086, 9804, 21138, "P < 0.05"),
"Results": (20130, 13542, 35292, 13086, 9804, 21138, "P < 0.05"),
},
"FGF-19 iAUC (pg/mL·min, median [IQR])": {
"Abstract": (11185, 1346, 17661, 2093, 673, 6707, "P < 0.01"),
"Results": (11185, 1346, 17661, 2093, 673, 6707, "P < 0.01"),
},
"CCK iAUC (pmol/L·min, median [IQR])": {
"Abstract": (43.2, 0.0, 130.1, 127.1, 47.2, 295.2, "P < 0.01"),
"Results": (27.5, 2.4, 109.2, 200.3, 40.3, 300.0, "P = 0.001"),
},
}
CCK = "CCK iAUC (pmol/L·min, median [IQR])"
FIELDS = ["before median", "before Q1", "before Q3", "after median", "after Q1", "after Q3"]
MTT_MINUTES = 120 # Methods: samples at -5 and -1 min, then every 30 min to 120 min
# ------------------------------------------------------------------ 1. field-by-field comparison
print("1. Abstract vs Results, field by field")
mismatch_counts = {}
for name, d in REPORTED.items():
a, r = d["Abstract"][:6], d["Results"][:6]
n_diff = sum(1 for x, y in zip(a, r) if x != y)
mismatch_counts[name] = n_diff
print(f" {name:42s} fields differing: {n_diff} P: {d['Abstract'][6]} | {d['Results'][6]}")
print()
a, r = REPORTED[CCK]["Abstract"][:6], REPORTED[CCK]["Results"][:6]
print(" CCK, field by field (Abstract -> Results):")
for f, x, y in zip(FIELDS, a, r):
rel = f"{(y - x) / x * 100:+.0f}%" if x else "n/a (Abstract value 0)"
print(f" {f:14s} {x:7.1f} vs {y:7.1f} difference {y - x:+7.1f} relative to Abstract {rel}")
print(f" largest absolute difference: {max(abs(y - x) for x, y in zip(a, r)):.1f} pmol/L·min")
# ------------------------------------------------------------------ 2. implied size of the CCK change
print("\n2. Size of the CCK change implied by each version (from the reported medians)")
sizes = {}
for src in ("Abstract", "Results"):
b, _, _, e, _, _, p = REPORTED[CCK][src]
sizes[src] = (e / b, e - b)
print(f" {src:8s}: median {b} -> {e}; ratio of medians x{e / b:.2f}; "
f"difference of medians {e - b:+.1f} pmol/L·min ({p})")
print(f" difference of medians, Results / Abstract: {sizes['Results'][1] / sizes['Abstract'][1]:.2f}")
print(f" ratio of medians, Results / Abstract: {sizes['Results'][0] / sizes['Abstract'][0]:.2f}")
print(" (ratios and differences of medians are not the median paired change; the paired data are not given)")
# ------------------------------------------------------------------ 3. ratio of each version's values
print("\n3. Abstract-to-Results ratio for each CCK field (one common unit or scale factor would give one ratio)")
for f, x, y in zip(FIELDS, a, r):
print(f" {f:14s} Results/Abstract = {y / x:.3f}" if x else f" {f:14s} Abstract value is 0")
# ------------------------------------------------------------------ 4. cross-containment
print("\n4. Does each version's median fall inside the other version's IQR?")
print(f" Abstract day-14 median 43.2 within Results day-14 IQR [2.4, 109.2]: {2.4 <= 43.2 <= 109.2}")
print(f" Results day-14 median 27.5 within Abstract day-14 IQR [0, 130.1]: {0 <= 27.5 <= 130.1}")
print(f" Abstract day-70 median 127.1 within Results day-70 IQR [40.3, 300]: {40.3 <= 127.1 <= 300}")
print(f" Results day-70 median 200.3 within Abstract day-70 IQR [47.2, 295.2]: {47.2 <= 200.3 <= 295.2}")
# ------------------------------------------------------------------ 5. P values
print("\n5. P values: Abstract 'P < 0.01', Results 'P = 0.001'; compatible:", 0.001 < 0.01)
# ------------------------------------------------------------------ 6. plausibility scale (per minute)
print(f"\n6. Mean increment above basal implied by each median iAUC over the {MTT_MINUTES}-min MTT")
for src in ("Abstract", "Results"):
b, _, _, e, _, _, _ = REPORTED[CCK][src]
print(f" {src:8s}: day 14 {b / MTT_MINUTES:.2f} pmol/L, day 70 {e / MTT_MINUTES:.2f} pmol/L "
f"(assay detection limit 0.3 pmol/L)")
# ------------------------------------------------------------------ 7. ratios for the control endpoints
print("\n7. After/before ratio for every iAUC reported in both places (fig2)")
RATIOS = {}
for name, d in REPORTED.items():
RATIOS[name] = {}
for src in ("Abstract", "Results"):
b, e = d[src][0], d[src][3]
RATIOS[name][src] = e / b
print(f" {name:42s} Abstract x{RATIOS[name]['Abstract']:.3f} Results x{RATIOS[name]['Results']:.3f}")
# ================================================================== figures
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834" # values as reported in the paper
BLUE = "#2a78d6" # values recomputed from the paper's own numbers
GRID = "#e4e3df"
PANEL = "#f3f2ef"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "axes.facecolor": BG, "figure.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200)
plt.close(fig)
def textbox(ax, x, y, w, h, title, body, foot):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=PANEL, ec=INK2, lw=1.0, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.05, title, transform=ax.transAxes, fontsize=10.5,
fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.20, body, transform=ax.transAxes, fontsize=10.5,
va="top", color=INK, linespacing=1.45)
ax.text(x + 0.02, y + 0.05, foot, transform=ax.transAxes, fontsize=10.5,
fontweight="bold", va="bottom", color=BLUE)
# ------------------------------------------------------------------ fig1: two text panels + strip
fig = plt.figure(figsize=(W, 5.0))
ax = fig.add_axes([0, 0.42, 1, 0.58])
ax.set_axis_off()
ax.text(0.03, 0.96, "Abstract and Results give different CCK values for the same comparison",
fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
ra, rr = sizes["Abstract"], sizes["Results"]
textbox(ax, 0.03, 0.03, 0.37, 0.78, "Abstract",
"“iAUC for cholecystokinin (CCK)\nincreased (from 43.2 [0–130.1]\nto 127.1 [47.2–295.2]\n"
"pmol/L ⋅ min, P < 0.01)”",
f"ratio of medians ×{ra[0]:.1f}\n(recomputed)")
textbox(ax, 0.60, 0.03, 0.37, 0.78, "Results text (GI peptides)",
"“…a significant increase in\niAUC (from 27.5 [2.4–109.2]\nto 200.3 [40.3–300]\n"
"pmol/L ⋅ min, P = 0.001)”",
f"ratio of medians ×{rr[0]:.1f}\n(recomputed)")
ax.add_patch(FancyArrowPatch((0.41, 0.42), (0.59, 0.42), transform=ax.transAxes,
arrowstyle="<->", mutation_scale=14, lw=2, color=INK2))
ax.text(0.5, 0.50, "same\nendpoint,\nn = 20,\nday 14 vs\nday 70", ha="center", va="bottom",
fontsize=10, transform=ax.transAxes, linespacing=1.2, color=INK2)
ax.text(0.5, 0.36, f"{mismatch_counts[CCK]} of 6\nnumbers\ndiffer", ha="center", va="top",
fontsize=10.5, fontweight="bold", transform=ax.transAxes, linespacing=1.2)
s = fig.add_axes([0.20, 0.13, 0.76, 0.24])
s.set_xlim(0, 320)
s.set_ylim(-0.6, 3.6)
rows = [("Abstract, day 14", REPORTED[CCK]["Abstract"][0:3], "o"),
("Results, day 14", REPORTED[CCK]["Results"][0:3], "s"),
("Abstract, day 70", REPORTED[CCK]["Abstract"][3:6], "o"),
("Results, day 70", REPORTED[CCK]["Results"][3:6], "s")]
for i, (lab, (m, lo, hi), mk) in enumerate(rows):
y = 3 - i
s.plot([lo, hi], [y, y], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
filled = lab.startswith("Abstract")
s.scatter([m], [y], s=70, marker=mk, zorder=3, lw=2,
facecolor=ORANGE if filled else BG, edgecolor=ORANGE)
s.text(m, y + 0.28, f"{m:g}", ha="center", va="bottom", fontsize=10, color=INK)
s.set_yticks([3, 2, 1, 0])
s.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
s.tick_params(axis="y", length=0)
for sp in ("left", "right", "top"):
s.spines[sp].set_visible(False)
s.grid(axis="x", color=GRID, lw=0.8)
s.set_axisbelow(True)
s.set_xlabel("CCK iAUC during the meal test, pmol/L·min: reported median (marker) and IQR (line)",
fontsize=10)
save(fig, "fig1")
# ------------------------------------------------------------------ fig2: ratios, all endpoints
fig, ax = plt.subplots(figsize=(W, 3.4))
fig.subplots_adjust(left=0.27, right=0.97, top=0.80, bottom=0.18)
labels = ["Glucose (ratio of means)", "Insulin (ratio of medians)",
"FGF-19 (ratio of medians)", "CCK (ratio of medians)"]
names = list(REPORTED)
for i, (lab, name) in enumerate(zip(labels, names)):
y = len(names) - 1 - i
ab, rs = RATIOS[name]["Abstract"], RATIOS[name]["Results"]
ax.scatter([ab], [y + 0.12], s=70, marker="o", color=BLUE, zorder=3)
ax.scatter([rs], [y - 0.12], s=70, marker="D", facecolor=BG, edgecolor=BLUE, lw=2, zorder=3)
if ab == rs:
ax.text(ab * 1.12, y, f"×{ab:.2f}, identical in both places", va="center", fontsize=10,
color=INK2)
else:
ax.plot([ab, rs], [y + 0.12, y - 0.12], color=BLUE, lw=2, alpha=0.35, zorder=2)
ax.text(ab * 0.9, y + 0.12, f"Abstract ×{ab:.2f}", ha="right", va="center", fontsize=10)
ax.text(rs * 1.1, y - 0.12, f"Results ×{rs:.2f}", ha="left", va="center", fontsize=10)
ax.axvline(1, color=INK2, lw=1)
ax.text(1.04, -0.42, "no change", ha="left", va="bottom", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.1, 20)
ax.set_xticks([0.1, 0.2, 0.5, 1, 2, 5, 10, 20])
ax.set_xticklabels(["0.1", "0.2", "0.5", "1", "2", "5", "10", "20"])
ax.set_ylim(-0.5, 3.5)
ax.set_yticks(range(len(names)))
ax.set_yticklabels(labels[::-1], color=INK)
ax.tick_params(axis="y", length=0)
for sp in ("left", "right", "top"):
ax.spines[sp].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.set_xlabel("Day 70 / day 14, recomputed from the reported summaries (log scale)", fontsize=10)
ax.scatter([], [], s=70, marker="o", color=BLUE, label="from Abstract values")
ax.scatter([], [], s=70, marker="D", facecolor=BG, edgecolor=BLUE, lw=2, label="from Results values")
ax.legend(loc="lower left", bbox_to_anchor=(-0.02, 1.0), ncol=2, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.5)
fig.text(0.03, 0.95, "Only the CCK result differs between Abstract and Results",
fontsize=11.5, fontweight="bold", va="top")
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Hypertension–knowledge p = 0.007 cannot be reproduced from Table 1; its summary statistics give p ≈ 0.59
Diabetologia (2024) · doi:10.1007/s00125-024-06304-3 · PMC11732930
Summary
The paper surveys 1050 Jordanian adults. In its Abstract, Results and Table 1 it reports that diabetes knowledge scores differ significantly by hypertension status: 46.46 ± 13.11 with hypertension (n = 72) against 47.31 ± 12.87 without (n = 978), p = 0.007. The paper's own means, SDs and group sizes do not give that result. Student's t test, which the Table 1 footer names, gives t = −0.540 and p = 0.589 (Welch p = 0.596), with a 95% CI for the difference of −3.94 to 2.24 points. The companion behaviour comparison (p = 0.017) recomputes to p = 0.794. Both are printed as significant associations. The recomputed values are clearly non-significant, so the unadjusted link between hypertension and diabetes knowledge that the Abstract lists has no support in the paper's own numbers.
What the paper reports
Significant associations were found between diabetes knowledge and the following factors: […] hypertension (46.46 ± 13.11 for present vs 47.31 ± 12.87 for absent; p=0.007)
— Abstract, Results. The bracket marks four omitted factors (age, sex, marital status, smoking status). The sentence ends with diabetes status.
Behaviour scores correlated similarly with these factors except for diabetes and smoking status.
— Abstract, Results (this sentence presents the hypertension–behaviour association as significant as well)
Non-hypertensive individuals had higher knowledge scores (47.31 ± 12.87) than those with hypertension (46.46 ± 13.11, p=0.007). Behaviour scores were also higher in non-hypertensive individuals (85.10 ± 19.90) compared with hypertensive individuals (84.46 ± 21.75, p=0.017).
— Results, second paragraph
Yes | 72 (6.86) | 46.46 ± 13.11 | 0.007 | 84.46 ± 21.75 | 0.017 No | 978 (93.14) | 47.31 ± 12.87 | | 85.10 ± 19.90 |
— Table 1, "Distribution of diabetes knowledge and behaviour scores across demographic and clinical categories", Hypertension rows. The columns are n (%), knowledge score, p value, behaviour score, p value.
based on Student’s t tests for variables with two categories and one-way ANOVA for variables with three or more categories
— Table 1 footer. The footer also says the scores are "presented as means ± SD".
Differences in knowledge and behaviour scores across categories were tested using unpaired t test for binary factors (e.g. sex or marital status) and one-way ANOVA for multi-level factors (e.g. age or BMI), with statistical significance set at p<0.05.
— Methods, "Statistical analysis"
Variables that demonstrated significant associations (p<0.05) in preliminary t tests and ANOVA were included in the regression models.
— Methods, "Statistical analysis"
Two later passages describe hypertension's adjusted effect. It points the opposite way from the unadjusted means:
Hypertension status had a significant positive coefficient (4.5346, p<0.0001), indicating higher adjusted knowledge scores among individuals with hypertension.
— Results, third paragraph (Table 2: "Hypertension | 4.5346 | 0.558 | <0.0001")
Participants with hypertension exhibited higher diabetes knowledge and behaviour scores, possibly reflecting a broader understanding of chronic disease management.
— Discussion, third paragraph
What the paper's own numbers show
Method. An unpaired two-sample t test needs only the group sizes, means and SDs, and all of them are printed in Table 1. Student's pooled-variance test (df = n₁ + n₂ − 2 = 1048) is the test that Table 1's footer names. Welch's unequal-variance test and a z test serve as sensitivity checks. For each printed p, the t statistic it requires is found from the inverse t distribution. From that follow the mean difference, the standard error and the 95% interval that the p value implies. To test whether the method recovers the paper's own reporting, the same calculation was applied to every comparison in Table 1: 20 two-group t tests, and 6 one-way ANOVAs computed from group n, mean and SD. Every number below is printed by recompute.py.txt (Python, numpy/scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Knowledge, mean difference (hypertension yes − no) | Abstract; Results; Table 1 | 46.46 vs 47.31 | −0.85 points |
| Knowledge, Student's t test (pooled SD 12.89, SE 1.574, df 1048) | Abstract; Results; Table 1 | p = 0.007 | t = −0.540, two-sided p = 0.589 |
| Knowledge, Welch t test (SE √(2.387 + 0.169) = 1.599, df 81.4) | sensitivity | — | t = −0.532, p = 0.596 |
| Knowledge, z test / one-sided Student p | sensitivity | — | 0.595 / 0.295 |
| Knowledge, 95% CI for the difference | derived | — | −3.94 to 2.24 points |
| Knowledge, |t| needed for p = 0.007 (df 1048) | derived | p = 0.007 | 2.702 |
| Knowledge, difference needed for p = 0.007 / p = 0.05 with the printed SDs and n | derived | 0.85 | 4.25 / 3.09 points |
| Knowledge, SE implied by a 0.85 difference with p = 0.007 | derived | — | 0.315, one-fifth of the SE the printed SDs give |
| Knowledge, common SD that p = 0.007 would need with n = 72 and 978 | derived | 13.11 and 12.87 | 2.58 |
| Knowledge, smallest p with each mean and SD moved anywhere within its rounding interval | derived | p = 0.007 | 0.585 |
| Knowledge, largest |t| reachable with 978 controls, however large the hypertensive group | derived | — | 2.07 (p = 0.039), still short of 2.70 |
| Knowledge, equal group size needed for p ≤ 0.007 | derived | 72 and 978 | 3400 per group (study total 1050) |
| Behaviour, mean difference (yes − no) | Results; Table 1 | 84.46 vs 85.10 | −0.64 points |
| Behaviour, Student's t test (SE 2.446) | Results; Table 1 | p = 0.017 | t = −0.262, p = 0.794 |
| Behaviour, Welch t test (SE 2.641, df 80.0) | sensitivity | — | t = −0.242, p = 0.809 |
| Behaviour, 95% CI for the difference | derived | — | −5.44 to 4.16 points |
| Behaviour, difference needed for p = 0.017 | derived | 0.64 | 5.85 points |
| Control: smoking, knowledge (45.78 vs 48.22, n = 452 / 598) | Abstract; Results; Table 1 | p = 0.003 | t = −3.10, p = 0.0020 |
| Control: sex, marital status (knowledge), age (both scores) | Abstract; Results; Table 1 | p < 0.0001 | 2.0 × 10⁻⁵ to 8.5 × 10⁻¹² |
| Control: education, employment and diabetes status (behaviour) | Table 1 | 0.913, 0.429, 0.294 | 0.91, 0.43, 0.29 |
| All 26 comparisons in Table 1 | Table 1 | — | 14 reproduce; 12 do not; 3 of the 12 cross p = 0.05: hypertension knowledge, hypertension behaviour, physical-activity behaviour |
| Hypertension coefficient, adjusted model | Table 2 / Table 3 | +4.5346 (SE 0.558) / +0.5410 (SE 0.133), p < 0.0001 | Opposite sign to the unadjusted −0.85 / −0.64; a separate analysis, not a source of 0.007 or 0.017 |
"Reproduce" means that a printed "<" bound holds, or that the recomputed p lies within 10% of the printed value or within one unit of its last printed digit. Many comparisons reproduce closely, among them the knowledge comparisons for four of the five other factors that the Abstract lists next to hypertension (age, sex, marital status, smoking). The fifth, diabetes status, does not reproduce: its knowledge comparison recomputes to p = 0.0052 (Welch 0.0084) against a printed "<0.001", although it remains significant at p < 0.05. So the paper's summary statistics and the stated test do recover many of its own p values. The hypertension rows are among those that do not, and they are two of the three where the recomputed value lands on the other side of the paper's p < 0.05 threshold. The other non-reproducing rows are listed in the script's output and shown in Figure 2.
Figures
Figure 1. Mean score difference between participants with and without hypertension (yes minus no), in points on the 0–100 scale, on one axis. The thin vertical line marks no difference. Orange circles and intervals show the printed differences (−0.85 for knowledge, −0.64 for behaviour), each with the 95% interval its printed p value implies (p = 0.007: −1.47 to −0.23; p = 0.017: −1.17 to −0.11). Neither orange interval includes zero. Blue diamonds and intervals are the 95% CIs recomputed from the Table 1 SDs and group sizes with Student's t test: −3.94 to 2.24 (p = 0.59) and −5.44 to 4.16 (p = 0.79). Both blue intervals contain zero. For the printed p values to hold, the standard errors would have to be about one-fifth (knowledge) to one-ninth (behaviour) of what the printed SDs and group sizes give.
Figure 2. Two-sided p values printed in Table 1 (orange circles; an arrow marks a value printed as a "<" bound) against p values recomputed from each row's n, mean and SD (blue diamonds; Student's t for two-category variables, one-way ANOVA for three-category variables, as the Table 1 footer specifies). The dashed line marks p = 0.05. Where a printed value is reproduced, the two markers sit together (often the diamond covers the circle) and the row label is in grey. For rows printed as a "<" bound, a reproduced value can lie far to the left of the printed bound; the thin grey line then marks agreement, not a gap. Row labels in bold mark comparisons whose printed p is not reproduced. The shaded rows are the hypertension comparisons, two of the three rows whose recomputed p falls on the other side of p = 0.05 (the third is physical-activity behaviour). Many rows reproduce, including age, sex, marital status and smoking (knowledge), four of the five other factors listed in the Abstract; the fifth, diabetes status, does not reproduce (0.0052 vs printed <0.001). The method therefore recovers the paper's reporting in general. Gaps also occur in rows other than hypertension (for example dyslipidaemia and physical-activity behaviour).
Why it matters
The hypertension result is printed in the Abstract, as one of six factors with "Significant associations" with diabetes knowledge, and the Results text repeats it. The Abstract's next sentence ("Behaviour scores correlated similarly with these factors except for diabetes and smoking status") presents the behaviour association as significant too. Recomputed from the paper's own Table 1, neither comparison is significant: p = 0.59 for knowledge and p = 0.79 for behaviour. The 95% CI for the knowledge difference, −3.94 to 2.24 points, is compatible with no difference. The direction of the printed means is unchanged, with hypertensive participants about 0.85 points lower on a 0–100 scale. What changes is the conclusion that the difference is statistically detectable.
The error has two further consequences.
- Selection into the regression models. Methods state that variables significant at p < 0.05 in the preliminary t tests and ANOVA were entered into the regression models. Hypertension was also included on the strength of prior literature ("covariate adjustment was guided by previous research … hypertension [36]"), so its inclusion does not rest on the unadjusted test alone.
- The Discussion's account of hypertension. The Discussion says participants with hypertension "exhibited higher diabetes knowledge and behaviour scores". That rests on the adjusted coefficients (+4.5346 and +0.5410, both p < 0.0001). The unadjusted means point slightly the other way, and their difference is not significant. So the unadjusted comparison gives no support to the adjusted finding. A reader cannot tell from the paper how an unadjusted difference of −0.85 points becomes an adjusted +4.53 points.
The paper's central conclusions do not depend on these rows. These are the association between knowledge and behaviour (the knowledge–attitude–behaviour framing) and the differences by age and sex, which do reproduce. The error affects a secondary result that is printed in the Abstract.
A related observation, separate from the claim above: the Table 1 rows for different variables imply different overall knowledge means for the same 1050 participants. About 47.2 follows from most variables (for example sex 47.23, hypertension 47.25, smoking 47.17), but 41.33 from age, 63.09 from diabetes status and 75.98 from location. This does not change the hypertension recomputation, which uses only the hypertension rows. It does show that Table 1 has other internal inconsistencies.
Explanations considered
- Rounding of the printed means and SDs. This does not account for the numbers. Moving every mean and SD to the edge of its rounding interval gives a smallest p of 0.585. The difference would have to be 4.25 points, not 0.85, to give p = 0.007.
- A different version of the t test (Welch, z, one-sided). This does not account for the numbers. Welch gives p = 0.596, a z test 0.595 and a one-sided Student test 0.295. Methods set significance at p < 0.05 with no mention of one-sided tests.
- The ± values being standard errors rather than SDs. This does not account for the numbers, and the paper rules it out. The Table 1 footer says "means ± SD". Read as SEs, the values would make the test weaker (p = 0.96) and would imply an SD of about 111 in the hypertensive group on a 0–100 score.
- The p value coming from the adjusted model. This does not account for the numbers. Tables 2 and 3 give the hypertension coefficient as +4.5346 and +0.5410 with p < 0.0001. Those values have the opposite sign to the unadjusted differences, and their p values do not match 0.007 or 0.017. Table 1 is presented as the unadjusted comparison.
- A rank-based or other non-parametric test on the raw data. This cannot be fully excluded from summary statistics alone. However, the Table 1 footer and Methods both name the t test, and the paper gives no other test for these comparisons. With near-identical means and SDs, a distribution-free test would need markedly different distribution shapes in the two groups to reach p = 0.007. Nothing in the paper describes such shapes.
- Values moved between rows. This is not established. The recomputed p for the diabetes-status knowledge comparison is 0.0052 (Student) or 0.0084 (Welch), which is close to 0.007, but that row is printed as "<0.001". No row recomputes to 0.017. The coincidence alone cannot show that p values were shifted between rows.
- A conversion or typesetting artefact in the archived copy. This is very unlikely: the publisher page gives the same Abstract and Results text, and PMC and Europe PMC give identical Table 1 cells; the publisher's PDF table could not be retrieved automatically. The PMC HTML and the Europe PMC full-text XML give the Table 1 hypertension cells as separate
<td>elements with "0.007" and "0.017". The neighbouring smoking row reproduces from the same layout (p = 0.0020 against a printed 0.003).
From the paper's numbers alone it is not possible to tell whether the p values, the means or the SDs in the hypertension rows are in error. Taken together, the printed means, SDs and group sizes are consistent with each other and with the Table 1 overall mean of about 47.2, and they give p ≈ 0.59 and p ≈ 0.79.
Status
As of 2026-10-23:
- Version of record: checked. On the publisher's article page (link.springer.com, reached after the cookie redirect), the Abstract reads "hypertension (46.46 ± 13.11 for present vs 47.31 ± 12.87 for absent; p=0.007)" and the Results sentence matches the one quoted above. The Table 1 cells did not come through in the publisher-page capture, and the standalone table page and the PDF returned a bot check. The PMC HTML page and the Europe PMC full-text XML give identical Table 1 cells, with the Student's t footer. The figures agree across the Abstract, Results and Table 1 in every source.
- Supplement: none exists. No electronic supplementary material is listed on the publisher page or on PMC, and Europe PMC records no supplementary files (hasSuppl = N, no
<supplementary-material>element). The data availability statement says the data are available from the corresponding author on request. No public data could settle the discrepancy. - Corrections: none found.
src/known.pyfound no Crossref updates and no PMC or Europe PMC notices. The Crossref works record has no update-to or updated-by entries and no relations, and a Crossref relation search for the DOI returned nothing. The Europe PMC record (PMID 39505766) has an empty comment/correction list. A Europe PMC search for a correction or erratum to this article in Diabetologia found none, and the captured publisher page shows no correction notice. - PubPeer: not reachable by automated query, because the API returned a human-verification check. Web searches (including one today for the title with "pubpeer", "correction" and "erratum") found no PubPeer thread, erratum or published comment. A manual search at pubpeer.com is still advisable.
Sources: doi.org/10.1007/s00125-024-06304-3 · link.springer.com article · PMC11732930 · Europe PMC record · Crossref works record
Suggested correction
The authors could re-run the unadjusted comparisons by hypertension status from the original data and report the group sizes, means, SDs and p values exactly as produced, so that they agree with each other.
- If the printed means, SDs and group sizes are correct, the hypertension p values should read about 0.59 (knowledge) and 0.79 (behaviour). Hypertension should then be removed from the Abstract's list of factors significantly associated with knowledge, and from the behaviour statement that follows it. The Results sentences should say the differences were not significant.
- If the p values are correct, the corrected means or SDs should replace those in the Abstract, the Results and Table 1.
It would also help to reconcile the other Table 1 p values that do not reproduce from their rows (Figure 2, script output section 3), and the differing overall knowledge means implied by the diabetes-status, location and age rows. The text could explain how the near-zero unadjusted difference relates to the adjusted hypertension coefficients in Tables 2 and 3.
Recomputation script (python recompute.py.txt)
"""Recompute the unadjusted hypertension comparisons of PMC11732930
(Diabetologia 2024, doi:10.1007/s00125-024-06304-3) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md. Writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file, and five candidate figure
designs (A-E) into its candidates/ subfolder (fig1 = design A, fig2 = design B).
All inputs are copied from Table 1 of the paper (n, mean +/- SD, printed p).
The Table 1 footer states that two-category variables were compared with
Student's t tests and variables with three or more categories with one-way
ANOVA. Student's t is computed with pooled variance (df = n1 + n2 - 2); Welch's
t is shown as a sensitivity check.
"""
import os
import shutil
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------------------
# Table 1 inputs: label, (n, mean, SD) group 1, (n, mean, SD) group 2, printed p
# Differences are always group 1 minus group 2, in the order of Table 1.
# ---------------------------------------------------------------------------
TWO_GROUP = [
("Sex (male vs female)", "knowledge", (474, 45.09, 13.27), (576, 49.00, 12.35), "<0.0001"),
("Sex (male vs female)", "behaviour", (474, 81.81, 22.00), (576, 88.00, 18.14), "<0.0001"),
("Marital status", "knowledge", (652, 45.39, 13.10), (398, 50.92, 11.69), "<0.0001"),
("Marital status", "behaviour", (652, 83.47, 20.91), (398, 88.53, 18.11), "0.0003"),
("Education", "knowledge", (176, 47.04, 14.58), (874, 47.27, 12.56), "0.457"),
("Education", "behaviour", (176, 85.06, 21.33), (874, 85.24, 19.98), "0.913"),
("Employment", "knowledge", (431, 47.70, 12.35), (619, 46.91, 13.29), "0.745"),
("Employment", "behaviour", (431, 85.80, 20.42), (619, 84.80, 20.05), "0.429"),
("Hypertension (yes vs no)", "knowledge", (72, 46.46, 13.11), (978, 47.31, 12.87), "0.007"),
("Hypertension (yes vs no)", "behaviour", (72, 84.46, 21.75), (978, 85.10, 19.90), "0.017"),
("Dyslipidaemia", "knowledge", (16, 47.12, 11.34), (1034, 47.25, 12.98), "0.076"),
("Dyslipidaemia", "behaviour", (16, 86.25, 22.45), (1034, 85.00, 19.98), "0.228"),
("Coronary artery disease", "knowledge", (11, 47.45, 10.58), (1039, 47.19, 12.77), "0.975"),
("Coronary artery disease", "behaviour", (11, 82.73, 20.58), (1039, 85.20, 20.00), "0.849"),
("Smoking", "knowledge", (452, 45.78, 13.22), (598, 48.22, 12.15), "0.003"),
("Smoking", "behaviour", (452, 84.09, 21.32), (598, 86.15, 19.43), "0.076"),
("Diabetes status", "knowledge", (52, 69.49, 17.35), (998, 62.76, 16.88), "<0.001"),
("Diabetes status", "behaviour", (52, 88.08, 20.39), (998, 85.06, 20.19), "0.294"),
("Location", "knowledge", (921, 75.60, 20.90), (129, 78.71, 19.35), "0.1108"),
("Location", "behaviour", (921, 84.99, 20.29), (129, 86.74, 19.53), "0.3571"),
]
MULTI_GROUP = [
("Age (3 groups)", "knowledge", [(542, 39.73, 9.95), (382, 42.61, 9.18), (126, 44.32, 9.53)], "<0.0001"),
("Age (3 groups)", "behaviour", [(542, 82.42, 21.84), (382, 88.06, 17.72), (126, 88.57, 18.27)], "<0.0001"),
("Physical activity (3 groups)", "knowledge", [(851, 46.37, 12.73), (91, 47.90, 11.88), (108, 48.35, 13.07)], "0.200"),
("Physical activity (3 groups)", "behaviour", [(851, 83.40, 20.56), (91, 87.77, 18.22), (108, 90.08, 17.95)], "0.507"),
("BMI (3 groups)", "knowledge", [(535, 46.52, 12.99), (345, 48.60, 11.82), (170, 47.13, 12.20)], "0.089"),
("BMI (3 groups)", "behaviour", [(535, 84.52, 20.12), (345, 86.70, 18.98), (170, 83.05, 21.64)], "0.108"),
]
def pooled(a, b):
(n1, m1, s1), (n2, m2, s2) = a, b
df = n1 + n2 - 2
sp = np.sqrt(((n1 - 1) * s1 ** 2 + (n2 - 1) * s2 ** 2) / df)
se = sp * np.sqrt(1 / n1 + 1 / n2)
t = (m1 - m2) / se
return dict(diff=m1 - m2, sp=sp, se=se, t=t, df=df, p=2 * stats.t.sf(abs(t), df))
def welch(a, b):
(n1, m1, s1), (n2, m2, s2) = a, b
v1, v2 = s1 ** 2 / n1, s2 ** 2 / n2
se = np.sqrt(v1 + v2)
df = (v1 + v2) ** 2 / (v1 ** 2 / (n1 - 1) + v2 ** 2 / (n2 - 1))
t = (m1 - m2) / se
return dict(diff=m1 - m2, v1=v1, v2=v2, se=se, t=t, df=df, p=2 * stats.t.sf(abs(t), df))
def anova(groups):
N = sum(g[0] for g in groups)
k = len(groups)
gm = sum(g[0] * g[1] for g in groups) / N
ssb = sum(g[0] * (g[1] - gm) ** 2 for g in groups)
ssw = sum((g[0] - 1) * g[2] ** 2 for g in groups)
F = (ssb / (k - 1)) / (ssw / (N - k))
return F, k - 1, N - k, stats.f.sf(F, k - 1, N - k)
def parse_p(s):
"""Return (value, is_upper_bound, half of last printed digit)."""
ub = s.startswith("<")
v = s.lstrip("<")
dec = len(v.split(".")[1])
return float(v), ub, 10 ** (-dec)
def agrees(printed, rec):
"""Printed p reproduces if a '<x' bound holds, or if the recomputed value,
rounded to the printed precision, is within one unit of the last digit,
or if it lies within 10% of the printed value."""
v, ub, unit = parse_p(printed)
if ub:
return rec < v
return abs(round(rec / unit) - round(v / unit)) <= 1 or abs(rec - v) <= 0.10 * v
def ci95(res):
q = stats.t.ppf(0.975, res["df"])
return res["diff"] - q * res["se"], res["diff"] + q * res["se"]
def fmt_p(p):
return f"{p:.2g}" if p >= 0.001 else f"{p:.1e}"
print("=" * 78)
print("1. Hypertension, diabetes KNOWLEDGE score (Table 1; Abstract; Results)")
print("=" * 78)
HY_K = ((72, 46.46, 13.11), (978, 47.31, 12.87))
HY_B = ((72, 84.46, 21.75), (978, 85.10, 19.90))
sk, wk = pooled(*HY_K), welch(*HY_K)
lo_k, hi_k = ci95(sk)
print(f"mean difference (yes - no) {sk['diff']:.2f}")
print(f"Welch variance components {wk['v1']:.3f} + {wk['v2']:.3f} = {wk['v1'] + wk['v2']:.3f}")
print(f"Welch SE {wk['se']:.3f}")
print(f"Welch t, df, two-sided p t = {wk['t']:.3f}, df = {wk['df']:.1f}, p = {wk['p']:.3f}")
print(f"pooled SD {sk['sp']:.2f}")
print(f"Student SE {sk['se']:.4f}")
print(f"Student t, df, two-sided p t = {sk['t']:.3f}, df = {sk['df']}, p = {sk['p']:.3f}")
print(f"Student one-sided p {sk['p'] / 2:.3f}")
print(f"z-test (Welch SE) two-sided p {2 * stats.norm.sf(abs(wk['t'])):.3f}")
print(f"95% CI for the difference (Student) {lo_k:.2f} to {hi_k:.2f}")
t_007 = stats.t.ppf(1 - 0.007 / 2, sk["df"])
t_05 = stats.t.ppf(1 - 0.05 / 2, sk["df"])
print(f"|t| needed for p = 0.007 (df 1048) {t_007:.3f}")
print(f"difference needed for p = 0.007 {t_007 * sk['se']:.2f} points")
print(f"difference needed for p = 0.05 {t_05 * sk['se']:.2f} points")
se_imp_k = abs(sk["diff"]) / t_007
print(f"SE implied by diff 0.85 and p = 0.007 {se_imp_k:.3f} (ratio to Student SE {sk['se'] / se_imp_k:.1f})")
imp_lo_k, imp_hi_k = sk["diff"] - t_05 * se_imp_k, sk["diff"] + t_05 * se_imp_k
print(f"95% CI implied by diff and p = 0.007 {imp_lo_k:.2f} to {imp_hi_k:.2f}")
# Common SD (if both groups shared one SD) needed for p = 0.007
sd_need = abs(sk["diff"]) / (t_007 * np.sqrt(1 / 72 + 1 / 978))
print(f"common SD that p = 0.007 would need {sd_need:.2f} (printed SDs 13.11 and 12.87)")
# Even an unlimited hypertensive group cannot reach p = 0.007 with 978 controls:
t_lim = abs(sk["diff"]) / (12.87 / np.sqrt(978))
print(f"largest |t| with 978 non-hypertensive and infinitely many hypertensive: {t_lim:.2f}"
f" (p = {2 * stats.norm.sf(t_lim):.3f})")
# Equal group sizes needed
for n in range(10, 200000):
if pooled((n, 46.46, 13.11), (n, 47.31, 12.87))["p"] <= 0.007:
print(f"equal n per group needed for p <= 0.007: {n} (total {2 * n}; study total 1050)")
break
# Rounding: move every printed mean and SD to the edge of its rounding interval
pmin = 1.0
for d1 in (-0.005, 0.005):
for d2 in (-0.005, 0.005):
for e1 in (-0.005, 0.005):
for e2 in (-0.005, 0.005):
r = pooled((72, 46.46 + d1, 13.11 + e1), (978, 47.31 + d2, 12.87 + e2))
pmin = min(pmin, r["p"])
print(f"smallest Student p over rounding of means and SDs {pmin:.3f}")
# If the +/- values were SEs rather than SDs
se_alt = np.sqrt(13.11 ** 2 + 12.87 ** 2)
print(f"if +/- were SEs: t = {0.85 / se_alt:.3f}, z-test p = {2 * stats.norm.sf(0.85 / se_alt):.2f};"
f" implied SD in the hypertensive group = {13.11 * np.sqrt(72):.0f} on a 0-100 score")
print()
print("=" * 78)
print("2. Hypertension, diabetes BEHAVIOUR score (Table 1; Results)")
print("=" * 78)
sb, wb = pooled(*HY_B), welch(*HY_B)
lo_b, hi_b = ci95(sb)
print(f"mean difference (yes - no) {sb['diff']:.2f}")
print(f"Student SE, t, p SE = {sb['se']:.3f}, t = {sb['t']:.3f}, p = {sb['p']:.3f}")
print(f"Welch SE, t, df, p SE = {wb['se']:.3f}, t = {wb['t']:.3f}, df = {wb['df']:.1f}, p = {wb['p']:.3f}")
print(f"95% CI for the difference (Student) {lo_b:.2f} to {hi_b:.2f}")
t_017 = stats.t.ppf(1 - 0.017 / 2, sb["df"])
print(f"difference needed for p = 0.017 {t_017 * sb['se']:.2f} points")
se_imp_b = abs(sb["diff"]) / t_017
imp_lo_b, imp_hi_b = sb["diff"] - t_05 * se_imp_b, sb["diff"] + t_05 * se_imp_b
print(f"SE implied by diff 0.64 and p = 0.017 {se_imp_b:.3f} (ratio to Student SE {sb['se'] / se_imp_b:.1f})")
print(f"95% CI implied by diff and p = 0.017 {imp_lo_b:.2f} to {imp_hi_b:.2f}")
print()
print("=" * 78)
print("3. Every Table 1 comparison: printed p vs p recomputed from Table 1 summaries")
print("=" * 78)
print(f"{'variable':30s} {'score':9s} {'diff':>6s} {'Student p':>10s} {'Welch p':>9s} {'printed':>8s} reproduces?")
ALL = []
for lab, sc, a, b, rp in TWO_GROUP:
s, w = pooled(a, b), welch(a, b)
ok = agrees(rp, s["p"])
ALL.append((lab, sc, rp, s["p"], ok))
print(f"{lab:30s} {sc:9s} {s['diff']:6.2f} {fmt_p(s['p']):>10s} {fmt_p(w['p']):>9s} {rp:>8s} {'yes' if ok else 'NO'}")
for lab, sc, gs, rp in MULTI_GROUP:
F, d1, d2, p = anova(gs)
ok = agrees(rp, p)
ALL.append((lab, sc, rp, p, ok))
print(f"{lab:30s} {sc:9s} {'F=' + format(F, '.2f'):>6s} {fmt_p(p):>10s} {'':>9s} {rp:>8s} {'yes' if ok else 'NO'}")
n_ok = sum(r[4] for r in ALL)
print(f"\nreproduce: {n_ok} of {len(ALL)}; do not reproduce: {len(ALL) - n_ok}")
flip = [r for r in ALL if (parse_p(r[2])[0] < 0.05) != (r[3] < 0.05)]
print("comparisons whose significance at 0.05 changes:", [f"{r[0]} / {r[1]}" for r in flip])
print("smoking knowledge (control): Student t = "
f"{pooled((452, 45.78, 13.22), (598, 48.22, 12.15))['t']:.2f}, p = {pooled((452, 45.78, 13.22), (598, 48.22, 12.15))['p']:.4f}")
dm = pooled((52, 69.49, 17.35), (998, 62.76, 16.88))
dmw = welch((52, 69.49, 17.35), (998, 62.76, 16.88))
print(f"diabetes-status knowledge: Student p = {dm['p']:.4f}, Welch p = {dmw['p']:.4f} (printed <0.001)")
print()
print("=" * 78)
print("4. Sample-wide knowledge mean implied by each variable's Table 1 rows")
print("=" * 78)
groups_k = {lab: [a, b] for lab, sc, a, b, rp in TWO_GROUP if sc == "knowledge"}
groups_k.update({lab: gs for lab, sc, gs, rp in MULTI_GROUP if sc == "knowledge"})
for lab, gs in groups_k.items():
N = sum(g[0] for g in gs)
print(f" {lab:30s} N = {N:5d} implied mean = {sum(g[0] * g[1] for g in gs) / N:6.2f}")
print()
print("=" * 78)
print("5. Adjusted regression rows (Tables 2 and 3), for comparison only")
print("=" * 78)
for lab, b_, se_ in [("knowledge (Table 2)", 4.5346, 0.558), ("behaviour (Table 3)", 0.5410, 0.133)]:
z = b_ / se_
print(f" hypertension, {lab}: coefficient {b_:+.4f}, SE {se_}, z = {z:.2f}, p = {2 * stats.norm.sf(z):.1e};"
f" sign {'+' if b_ > 0 else '-'} vs unadjusted difference sign -")
# ---------------------------------------------------------------------------
# Figures
# ---------------------------------------------------------------------------
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, BLUE_L = "#eb6834", "#2a78d6", "#a9c8ee"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"axes.spines.top": False, "axes.spines.right": False, "axes.linewidth": 0.8,
"svg.fonttype": "none",
})
LW, MS = 2.0, 9 # 2 px lines (at 72 pt/in) and markers >= 8 px
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(CAND, f"{name}.{ext}"), dpi=200, bbox_inches="tight")
plt.close(fig)
def grid_x(ax):
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
# Design A: dot-and-interval, mean difference with 95% CI, reported vs recomputed
def design_a():
fig, ax = plt.subplots(figsize=(7.2, 3.4))
rows = [
("Knowledge score", sk["diff"], (imp_lo_k, imp_hi_k), (lo_k, hi_k), "0.007", sk["p"]),
("Behaviour score", sb["diff"], (imp_lo_b, imp_hi_b), (lo_b, hi_b), "0.017", sb["p"]),
]
y = 3.0
yt, yl = [], []
for lab, d, imp, rec, rp, pp in rows:
ax.plot(imp, [y + 0.28] * 2, color=ORANGE, lw=LW, solid_capstyle="round", zorder=3)
ax.plot(d, y + 0.28, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=4)
ax.text(imp[1] + 0.25, y + 0.28, f"interval implied by printed p = {rp}", va="center", fontsize=10, color=INK)
ax.plot(rec, [y - 0.28] * 2, color=BLUE, lw=LW, solid_capstyle="round", zorder=3)
ax.plot(d, y - 0.28, "D", color=BLUE, ms=MS - 1, mec=BG, mew=1.5, zorder=4)
ax.text(rec[1] + 0.25, y - 0.28, f"recomputed 95% CI, p = {pp:.2f}", va="center", fontsize=10, color=INK)
yt.append(y)
yl.append(lab)
y -= 1.6
ax.plot([0, 0], [0.6, 3.6], color=INK2, lw=0.8, zorder=1)
ax.text(0, 3.65, "no difference", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_yticks(yt, yl, fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
ax.set_ylim(0.6, 3.9)
ax.set_xlim(-6.5, 10.5)
ax.set_xlabel("Mean score difference, hypertension yes minus no (points on the 0-100 scale)")
grid_x(ax)
h1, = ax.plot([], [], "o-", color=ORANGE, lw=LW, ms=MS, label="Paper: printed difference with the 95% interval its p value implies")
h2, = ax.plot([], [], "D-", color=BLUE, lw=LW, ms=MS - 1, label="Recomputed from Table 1 means, SDs and n (Student's t)")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.22), frameon=False, fontsize=10, ncol=1)
save(fig, "A_difference_intervals")
# Design B: printed vs recomputed p for every Table 1 comparison (log scale)
def design_b():
order = list(range(len(ALL)))
fig, ax = plt.subplots(figsize=(7.2, 7.6))
labels = []
for i, idx in enumerate(order):
lab, sc, rp, rec, ok = ALL[idx]
y = len(order) - 1 - i
v, ub, _ = parse_p(rp)
is_h = lab.startswith("Hypertension")
if is_h:
ax.axhspan(y - 0.5, y + 0.5, color="#f3f2ee", zorder=0)
ax.plot([min(v, rec), max(v, rec)], [y, y], color=INK2 if ok else INK, lw=0.8 if ok else 1.4, zorder=2)
if ub:
ax.annotate("", xy=(v / 3, y), xytext=(v, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=1.5, mutation_scale=10), zorder=3)
ax.plot(v, y, "o", color=ORANGE, ms=MS, mec=BG, mew=1.2, zorder=4)
ax.plot(max(rec, 1e-12), y, "D", color=BLUE, ms=MS - 1, mec=BG, mew=1.2, zorder=5)
labels.append((y, f"{lab.split(' (')[0]}: {sc}", ok))
ax.set_xscale("log")
ax.set_xlim(1e-12, 1.6)
ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (4, 3)), zorder=1)
ax.text(0.05 * 1.25, len(order) - 0.45, "p = 0.05", ha="left", va="center", fontsize=10, color=INK2)
ax.set_yticks([l[0] for l in labels], [l[1] for l in labels], fontsize=10)
for tick, (_, _, ok) in zip(ax.get_yticklabels(), labels):
tick.set_color(INK if not ok else INK2)
tick.set_fontweight("bold" if not ok else "normal")
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
ax.set_ylim(-0.7, len(order) + 0.2)
ax.set_xlabel("Two-sided p value (log scale)")
grid_x(ax)
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Printed in Table 1 (arrow: printed as '<' bound)")
h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed from Table 1 summaries (t test or ANOVA)")
ax.legend(handles=[h1, h2], loc="upper left", bbox_to_anchor=(-0.02, -0.07), frameon=False, fontsize=10, ncol=1)
ax.text(-0.02, -0.165, "Bold row label: printed p not reproduced (differs by more than 10% and by more than one unit "
"in its last printed digit).\nShaded rows: hypertension.",
transform=ax.transAxes, ha="left", va="top", fontsize=10, color=INK2)
save(fig, "B_all_table1_pvalues")
# Design C: observed difference vs the difference each p value would need
def design_c():
fig, ax = plt.subplots(figsize=(7.2, 2.9))
items = [
(abs(sk["diff"]), ORANGE, "o", "Printed difference\n|46.46 - 47.31| = 0.85", +1),
(t_05 * sk["se"], BLUE, "D", f"Needed for p = 0.05:\n{t_05 * sk['se']:.2f}", -1),
(t_007 * sk["se"], BLUE, "D", f"Needed for the printed\np = 0.007: {t_007 * sk['se']:.2f}\n"
f"({t_007 * sk['se'] / abs(sk['diff']):.1f} times 0.85)", +1),
]
ax.plot([0, 6], [1, 1], color=GRID, lw=LW, zorder=1)
ax.plot([abs(sk["diff"]), t_007 * sk["se"]], [1, 1], color=INK2, lw=1.0, zorder=2)
for x, c, m, txt, side in items:
ax.plot(x, 1, m, color=c, ms=MS + 1, mec=BG, mew=1.5, zorder=4)
ax.text(x, 1 + 0.1 * side, txt, ha="center", va="bottom" if side > 0 else "top", fontsize=10, color=INK)
ax.set_xlim(0, 6)
ax.set_ylim(0.55, 1.75)
ax.set_yticks([])
ax.spines["left"].set_visible(False)
ax.set_xlabel("Absolute mean difference in knowledge score, hypertension yes vs no (points, 0-100 scale)\n"
"with the printed SDs (13.11, 12.87) and group sizes (72, 978), Student's t")
grid_x(ax)
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="From the paper (Table 1)")
h2, = ax.plot([], [], "D", color=BLUE, ms=MS - 1, label="Recomputed from the paper's SDs and n")
ax.legend(handles=[h1, h2], loc="upper left", frameon=False, fontsize=10, ncol=2, bbox_to_anchor=(0, 1.08))
save(fig, "C_difference_needed")
# Design D: each group's mean with its own 95% CI
def design_d():
fig, ax = plt.subplots(figsize=(7.2, 3.0))
gs = [("Hypertension: yes (n = 72)", 72, 46.46, 13.11), ("Hypertension: no (n = 978)", 978, 47.31, 12.87)]
for i, (lab, n, m, s) in enumerate(gs):
y = 1 - i
h = stats.t.ppf(0.975, n - 1) * s / np.sqrt(n)
ax.plot([m - h, m + h], [y, y], color=BLUE, lw=LW, solid_capstyle="round", zorder=3)
ax.plot(m, y, "o", color=ORANGE, ms=MS, mec=BG, mew=1.5, zorder=4)
ax.text(m + h + 0.3, y, f"{m:.2f} (95% CI {m - h:.2f} to {m + h:.2f})", va="center", fontsize=10)
ax.set_yticks([1, 0], [g[0] for g in gs], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
ax.set_ylim(-0.6, 1.9)
ax.set_xlim(40, 60)
ax.text(40.6, 1.6, f"Printed: p = 0.007", color=INK, fontsize=10.5, va="center")
ax.plot(40.25, 1.6, "o", color=ORANGE, ms=8, clip_on=False)
ax.text(48.6, 1.6, f"Recomputed (Student's t): p = {sk['p']:.2f}", color=INK, fontsize=10.5, va="center")
ax.plot(48.25, 1.6, "D", color=BLUE, ms=7, clip_on=False)
ax.set_xlabel("Mean diabetes knowledge score (0-100 scale; axis shows 40-60)")
grid_x(ax)
h1, = ax.plot([], [], "o", color=ORANGE, ms=MS, label="Group mean printed in Table 1")
h2, = ax.plot([], [], "-", color=BLUE, lw=LW, label="95% CI recomputed from the printed SD and n")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.3), frameon=False, fontsize=10, ncol=2)
save(fig, "D_group_means")
# Design E: where the test statistic sits on the null t distribution
def design_e():
fig, ax = plt.subplots(figsize=(7.2, 3.3))
x = np.linspace(-4.2, 4.2, 801)
yv = stats.t.pdf(x, sk["df"])
ax.plot(x, yv, color=INK2, lw=1.2, zorder=2)
ta = abs(sk["t"])
m = np.abs(x) >= ta
ax.fill_between(x, 0, yv, where=m & (x < 0), color=BLUE_L, lw=0, zorder=1)
ax.fill_between(x, 0, yv, where=m & (x > 0), color=BLUE_L, lw=0, zorder=1)
ax.axvline(sk["t"], color=BLUE, lw=LW, zorder=3)
ax.plot(sk["t"], stats.t.pdf(sk["t"], sk["df"]), "D", color=BLUE, ms=MS - 1, mec=BG, zorder=4)
for s in (-1, 1):
ax.axvline(s * t_007, color=ORANGE, lw=LW, ls=(0, (5, 3)), zorder=3)
ax.text(sk["t"] - 0.08, 0.43, f"recomputed t = {sk['t']:.2f}\nshaded tails: p = {sk['p']:.2f}",
ha="right", va="top", fontsize=10, color=INK)
ax.text(-t_007 - 0.08, 0.2, f"|t| needed for\nprinted p = 0.007:\n{t_007:.2f}", ha="right", va="center",
fontsize=10, color=INK)
ax.set_ylim(0, 0.45)
ax.set_xlim(-4.2, 4.2)
ax.set_yticks([])
ax.spines["left"].set_visible(False)
ax.set_xlabel("Student's t statistic under no difference (df = 1048), hypertension knowledge comparison")
grid_x(ax)
h1, = ax.plot([], [], "-", color=BLUE, lw=LW, label="Recomputed from Table 1 summaries")
h2, = ax.plot([], [], "--", color=ORANGE, lw=LW, label="Implied by the printed p = 0.007")
ax.legend(handles=[h1, h2], loc="upper center", bbox_to_anchor=(0.5, -0.2), frameon=False, fontsize=10, ncol=2)
save(fig, "E_null_distribution")
design_a()
design_b()
design_c()
design_d()
design_e()
for src, dst in [("A_difference_intervals", "fig1"), ("B_all_table1_pvalues", "fig2")]:
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"{src}.{ext}"), os.path.join(HERE, f"{dst}.{ext}"))
print("\nfigures written: fig1 (design A), fig2 (design B), candidates/A-E (.svg, .png)")
Abstract calls a non-significant subgroup difference at seroconversion significant; Results and Table 3 report none
Diabetologia (2025) · doi:10.1007/s00125-025-06573-6 · PMC12686032
Summary
This DIPP cohort analysis compares 1334 children with islet autoimmunity in three groups: an affected first-degree relative (FDR) at birth (FDR+, n = 145), an FDR diagnosed during follow-up (FDR− FDR+, n = 87), and no affected FDR (FDR−, n = 1102). The abstract's Results state that at seroconversion both FDR+ and FDR− FDR+ children were "significantly more often positive for GADA and multiple autoantibodies" than FDR− children. The paper's own Table 3 gives non-significant results for the FDR− FDR+ group: GADA OR 1.3 (95% CI 0.9, 2.1), p = 0.201, and multipositivity OR 1.5 (0.9, 2.4), p = 0.106. The Results text and the Discussion both say this group did not differ from FDR− children at seroconversion, and a recomputation from the Table 2 counts agrees (p = 0.19 and 0.11). The abstract is the only place where the claim appears. A reader who sees only the abstract could reasonably conclude that a family history arising after birth already marks a distinct autoantibody profile at seroconversion, which the body of the paper does not show.
What the paper reports
Text is quoted from the PubMed Central archive, which writes the group label as "FDR- FDR+" with a hyphen; the version of record typesets it with a minus sign (FDR− FDR+).
Abstract, Results:
At seroconversion, both FDR+ and FDR- FDR+ children were significantly more often positive for GADA and multiple autoantibodies than children without affected FDRs (FDR-).
Results, fourth paragraph (beginning “We compared the islet autoantibody profiles”):
At seroconversion, there was a statistically significant difference between the three groups for GADA positivity and multipositivity (Table 2). In pairwise comparisons, at seroconversion, FDR+ children had a higher risk for GADA positivity and multipositivity than FDR- children, but no difference was observed between FDR- FDR+ and FDR- children (Table 3).
Table 3, "At seroconversion" rows (columns: FDR+ vs FDR− | p value | FDR− FDR+ vs FDR− | p value; footnote: "Values are OR (95% CI) by logistic regression analysis"):
GADA+ | 1.5 (1.1, 2.2) | 0.020 | 1.3 (0.9, 2.1) | 0.201
Multipositive | 1.8 (1.2, 2.6) | 0.002 | 1.5 (0.9, 2.4) | 0.106
Table 2, "At seroconversion" rows (columns: FDR+ (n=145) | FDR− FDR+ (n=87) | FDR− (n=1102) | p value; data are % (n); footnote: "The exact χ2 test was used for categorical variables"):
GADA+ | 62.8 (91) | 59.8 (52) | 52.5 (578) | 0.036
Multipositive | 35.2 (51) | 31.0 (27) | 23.3 (257) | 0.004
Discussion, seventh paragraph (beginning “In terms of the timing of type 1 diabetes in the FDRs”):
FDR- FDR+ children did not differ from FDR- children at seroconversion, but showed a significantly higher risk of being positive for IAA, GADA, IA-2A or multiple autoantibodies during the follow-up period than FDR- children (Table 3).
Methods, "Statistical analyses":
The standard normal distribution (SND) test was used for pairwise comparisons of categorical variables. Logistic regression analysis was used to estimate the risk of various islet autoantibody profiles and progression to type 1 diabetes in children with and without an FDR with the disease.
Abstract, Conclusions/interpretation (for comparison; this sentence does not mention the FDR− FDR+ group):
At seroconversion, genetically susceptible children who had one or more FDRs with type 1 diabetes, especially an affected father, were more often positive for GADA and multiple islet autoantibodies.
What the paper's own numbers show
Method. From the Table 2 counts I rebuilt each 2×2 table (group vs FDR−; positive vs not positive) at seroconversion. For each I computed the unadjusted odds ratio with a Woolf 95% CI and Wald p value. These are exactly what a logistic regression of the outcome on a single group indicator gives, and the Methods name no covariates. I also computed the pooled two-proportion z test (the "SND test" named in the Methods), Fisher's exact test, the likelihood-ratio test and the Yates-corrected χ² test, and repeated the three-group χ² tests from Table 2. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Group sizes | Abstract; Table 2 | 145 / 87 / 1102; total 1334 | 145 + 87 + 1102 = 1334 |
| GADA+ at seroconversion, FDR− FDR+ vs FDR− | Table 2; Table 3 | 59.8% (52) vs 52.5% (578); OR 1.3 (0.9, 2.1), p = 0.201 | 52/87 vs 578/1102; OR 1.35 (0.86, 2.10); Wald p = 0.189; SND p = 0.188; Fisher p = 0.220; LR p = 0.186 |
| Multipositive at seroconversion, FDR− FDR+ vs FDR− | Table 2; Table 3 | 31.0% (27) vs 23.3% (257); OR 1.5 (0.9, 2.4), p = 0.106 | 27/87 vs 257/1102; OR 1.48 (0.92, 2.38); Wald p = 0.106; SND p = 0.104; Fisher p = 0.117; LR p = 0.114 |
| GADA+ at seroconversion, FDR+ vs FDR− | Table 3 | OR 1.5 (1.1, 2.2), p = 0.020 | 91/145 vs 578/1102; OR 1.53 (1.07, 2.18); p = 0.020 |
| Multipositive at seroconversion, FDR+ vs FDR− | Table 3 | OR 1.8 (1.2, 2.6), p = 0.002 | 51/145 vs 257/1102; OR 1.78 (1.23, 2.58); p = 0.002 |
| Three-group test, GADA+ at seroconversion | Table 2 | p = 0.036 (exact χ²) | Pearson χ² = 6.71, df 2, p = 0.035 |
| Three-group test, multipositive at seroconversion | Table 2 | p = 0.004 (exact χ²) | Pearson χ² = 11.31, df 2, p = 0.004 |
| Abstract statement on FDR− FDR+ vs FDR− at seroconversion | Abstract, Results | "significantly more often positive for GADA and multiple autoantibodies" | not significant on either measure under any of the tests above; both 95% CIs include 1 |
The two disputed cells of Table 3 reproduce from Table 2: the odds ratios and intervals match to the printed precision, and the p values are close (printed 0.201, recomputed 0.189 for GADA; printed 0.106, recomputed 0.106 for multipositivity). The body of the paper is internally consistent. Only the abstract sentence departs from it.
Side observation, not part of the error. When every Table 3 cell is recomputed in the same way from the Table 2 counts, six other cells do not reproduce (rule applied in recompute.py.txt, section 7: the recomputed OR, rounded to one decimal, differs from the print by more than 0.1, or the p value differs by more than 0.05). Five are in the FDR− FDR+ vs FDR− column: IAA+ at seroconversion (printed 1.3 (0.8, 2.1), p = 0.181; recomputed 1.52 (0.97, 2.39), p = 0.070), IA-2A+ at seroconversion (printed 1.0 (0.5, 1.6), p = 0.937; recomputed 0.95 (0.52, 1.75), p = 0.869), ZnT8A+ at seroconversion (printed 0.8 (0.3, 1.5), p = 0.484; recomputed 0.89 (0.43, 1.81), p = 0.739), GADA+ during follow-up (printed 2.8 (1.1, 3.7), p = 0.021; recomputed 2.05 (1.12, 3.76), p = 0.020) and ZnT8A+ during follow-up (printed 1.2 (0.8, 1.8), p = 0.361; recomputed 1.48 (0.96, 2.30), p = 0.077). One is in the FDR+ vs FDR− column: ZnT8A+ during follow-up (printed 1.2 (0.8, 1.7), p = 0.227; recomputed 1.21 (0.86, 1.71), p = 0.278). For GADA+ during follow-up, the printed estimate 2.8 is far from the geometric midpoint of its interval (1.1, 3.7), which is 2.0, while the recomputed OR is 2.05; this suggests a typographical slip in the point estimate. None of these six changes which side of p = 0.05 a result falls on, and none bears on the abstract sentence.
Figure
Figure 1. The left panel quotes the abstract's claim that FDR− FDR+ children were significantly more often GADA-positive and multipositive at seroconversion than FDR− children. The right panel quotes the Results sentence and the Table 3 estimates for the same comparison, which are non-significant. Values the paper reports are in orange. The blue panel gives the estimates recomputed from the Table 2 counts (unadjusted logistic regression, equivalent to the 2×2 odds ratio), with Fisher exact p values for comparison. Both recomputed intervals include OR = 1.
Figure 2. Odds ratios with 95% confidence intervals for the four pairwise comparisons at seroconversion, on a log scale with a thin reference line at OR = 1 (no difference). Orange circles show Table 3 as printed; blue diamonds show the values recomputed from the Table 2 counts, with the p value for each printed at the right. The two shaded rows are the FDR− FDR+ comparisons, which the abstract calls significant but Table 3 and the Results text do not. In both, the printed and the recomputed intervals cross 1. The two unshaded FDR+ rows, which the abstract also mentions, are significant and reproduce.
Why it matters
- The error is in the most-read part of the paper. The abstract is what most readers, indexers and citing authors see. Nothing in the abstract signals that the FDR− FDR+ comparison was non-significant, so a reader of the abstract alone cannot detect the problem.
- It changes a substantive interpretation. The FDR− FDR+ group is the paper's way of asking whether a family history that appears only after birth goes with a different autoimmune process. The abstract's sentence implies that these children already differ at the very start of islet autoimmunity. The paper's own analysis, and its Discussion, say instead that they differ from FDR− children only later, during follow-up.
- The headline conclusions are not affected. The FDR+ vs FDR− differences at seroconversion (p = 0.020 and 0.002) reproduce. So do the follow-up differences, the higher progression rates (55.9% vs 57.5% vs 38.9%), the shorter time to diabetes in FDR+ children, and the paternal vs maternal comparison. The abstract's Conclusions sentence does not name the FDR− FDR+ group; its wording (“had one or more FDRs”, “especially an affected father”) matches the paper's definition of the at-birth FDR+ group and is consistent with the body under that reading.
- Direction is unchanged, significance is not. The FDR− FDR+ point estimates are above 1 (GADA 59.8% vs 52.5%; multipositive 31.0% vs 23.3%), and they are close to the FDR+ estimates (FDR− FDR+ vs FDR+: Fisher p = 0.68 and 0.57). The data are therefore compatible with a modest difference, but the study cannot show one for this group of 87 children: the intervals run from about 0.9 to over 2. "Significantly more often" overstates what the analysis supports.
Explanations considered
- An adjusted model that would make the comparison significant. The Methods describe logistic regression with no covariates, and Table 3's footnote says only "by logistic regression analysis". The unadjusted odds ratios from the Table 2 counts reproduce the two disputed Table 3 cells (printed 1.3 and 1.5, recomputed 1.35 and 1.48; printed p = 0.201 and 0.106, recomputed 0.189 and 0.106). No adjusted analysis is reported anywhere in the paper or supplement. This does not account for the claim.
- A different pairwise test. The Methods name the SND test for pairwise comparisons of categorical variables. It gives p = 0.188 and 0.104. Fisher's exact test (0.220, 0.117), the likelihood-ratio test (0.186, 0.114) and the Yates-corrected χ² test (0.228, 0.135) are all non-significant too. Even a one-sided SND test, which the paper does not use, gives p = 0.094 and 0.052. This does not account for the claim.
- The three-group tests in Table 2. The exact χ² tests across all three groups are significant (p = 0.036 and 0.004; recomputed 0.035 and 0.004). An omnibus test shows only that the three groups are not all equal. It does not establish that each family-history group differs from FDR−, and the Results text itself moves from the omnibus result to pairwise comparisons that find "no difference" for FDR− FDR+. This does not account for the sentence, which makes a claim about each group ("both … were").
- A pooled reading ("children with any affected FDR"). If the 145 FDR+ and 87 FDR− FDR+ children are pooled (n = 232), the comparison with FDR− is significant (GADA OR 1.46 (1.09, 1.95), p = 0.011; multipositive OR 1.67 (1.23, 2.26), p = 0.001). The paper reports no pooled analysis, and the sentence names both groups separately. This could explain how a looser summary arose, but it does not make the sentence as written correct.
- A text-conversion artifact. The sentence was checked against the publisher's typeset PDF (Diabetologia (2026) 69:93–102) and the Europe PMC full-text XML. It reads the same in both, and the Table 3 values and Results sentence also match. Not an artifact.
- A supplement that reports a different analysis. The electronic supplementary material (ESM Methods and ESM Tables 1–4) covers HLA genotypes, father/mother/sibling splits and the multipositive subset. It contains no FDR− FDR+ vs FDR− comparison at seroconversion for the full cohort. It does not resolve the conflict.
Status
- Version of record. The abstract sentence, the Results sentence, Table 2 and Table 3 were compared with the publisher's typeset PDF, which is headed "Diabetologia (2026) 69:93–102", carries the same DOI, and is hosted by the University of Oulu repository (https://oulurepo.oulu.fi/handle/10024/59105), and with the Europe PMC full-text XML. All quoted passages are identical to the PMC archive. The publisher's own article page could not be loaded by automated access, which met a login or verification step.
- Supplement. 125_2025_6573_MOESM1_ESM.pdf (9 pages: ESM Methods, ESM Tables 1–4) was reviewed and does not change the comparison.
- Corrections and notices. No correction, erratum, retraction or expression of concern was found. Crossref lists no updates for the DOI, PubMed Central and Europe PMC list no notices, and the Europe PMC record (PMID 41204978) has no linked comments or corrections.
- PubPeer and the web. A web search for the title and DOI with "correction", "erratum" and "PubPeer" found no correction, PubPeer thread or published comment. PubPeer's own search could not be queried directly, because it required human verification, so a PubPeer thread cannot be fully ruled out.
- Checked as of 2026-10-23.
Suggested correction
In the abstract's Results, replace
At seroconversion, both FDR+ and FDR− FDR+ children were significantly more often positive for GADA and multiple autoantibodies than children without affected FDRs (FDR−).
with
At seroconversion, FDR+ children were significantly more often positive for GADA and multiple autoantibodies than children without affected FDRs (FDR−), whereas FDR− FDR+ children did not differ significantly from FDR− children (GADA OR 1.3 [95% CI 0.9, 2.1]; multipositivity OR 1.5 [0.9, 2.4]).
No change to the Results, Tables or Discussion is needed for this point. Separately, the authors may wish to check the six cells of Table 3 listed above (five in the FDR− FDR+ column, one in the FDR+ column) that do not reproduce from the Table 2 counts. One example is GADA+ during follow-up, printed as 2.8 (1.1, 3.7), where the counts give 2.05 (1.12, 3.76).
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12686032 (Diabetologia 2025, doi:10.1007/s00125-025-06573-6)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1 and fig2
# (SVG and 200 dpi PNG) in this script's folder.
# Inputs are the counts and estimates printed in the paper's Table 2 and Table 3.
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
N = {"FDR+": 145, "FDR- FDR+": 87, "FDR-": 1102}
# Table 2, % (n): (FDR+, FDR- FDR+, FDR-)
T2 = {
("seroconversion", "IAA+"): (83, 55, 585),
("seroconversion", "GADA+"): (91, 52, 578),
("seroconversion", "IA-2A+"): (25, 13, 172),
("seroconversion", "ZnT8A+"): (17, 9, 127),
("seroconversion", "Multipositive"): (51, 27, 257),
("follow-up", "IAA+"): (104, 71, 756),
("follow-up", "GADA+"): (124, 74, 810),
("follow-up", "IA-2A+"): (98, 56, 587),
("follow-up", "ZnT8A+"): (71, 47, 487),
("follow-up", "Multipositive"): (107, 64, 622),
("follow-up", "T1D diagnosis"): (81, 50, 429),
}
T2_pct = { # printed percentages
("seroconversion", "GADA+"): (62.8, 59.8, 52.5),
("seroconversion", "Multipositive"): (35.2, 31.0, 23.3),
}
T2_p = {("seroconversion", "GADA+"): 0.036, ("seroconversion", "Multipositive"): 0.004,
("seroconversion", "IAA+"): 0.143, ("seroconversion", "IA-2A+"): 0.862,
("seroconversion", "ZnT8A+"): 0.940}
# Table 3, OR (95% CI), p: FDR+ vs FDR-, then FDR- FDR+ vs FDR-
T3 = {
("seroconversion", "IAA+"): ((1.2, 0.8, 1.7, 0.356), (1.3, 0.8, 2.1, 0.181)),
("seroconversion", "GADA+"): ((1.5, 1.1, 2.2, 0.020), (1.3, 0.9, 2.1, 0.201)),
("seroconversion", "IA-2A+"): ((1.1, 0.7, 1.7, 0.612), (1.0, 0.5, 1.6, 0.937)),
("seroconversion", "ZnT8A+"): ((1.0, 0.5, 1.7, 0.944), (0.8, 0.3, 1.5, 0.484)),
("seroconversion", "Multipositive"): ((1.8, 1.2, 2.6, 0.002), (1.5, 0.9, 2.4, 0.106)),
("follow-up", "IAA+"): ((1.2, 0.8, 1.7, 0.472), (2.0, 1.2, 3.5, 0.014)),
("follow-up", "GADA+"): ((2.1, 1.3, 3.4, 0.002), (2.8, 1.1, 3.7, 0.021)),
("follow-up", "IA-2A+"): ((1.8, 1.3, 2.7, 0.001), (1.6, 1.0, 2.5, 0.049)),
("follow-up", "ZnT8A+"): ((1.2, 0.8, 1.7, 0.227), (1.2, 0.8, 1.8, 0.361)),
("follow-up", "Multipositive"): ((2.2, 1.5, 3.2, 0.0005), (2.1, 1.3, 3.5, 0.002)), # "<0.001" drawn as 0.0005
("follow-up", "T1D diagnosis"): ((2.0, 1.4, 2.8, 0.0005), (2.1, 1.4, 3.3, 0.0005)),
}
# ---------------------------------------------------------------- methods
def woolf(a, n1, c, n0):
"""Unadjusted OR, Woolf 95% CI, Wald p. Identical to a logistic regression
of the outcome on a single group indicator (no covariates)."""
b, d = n1 - a, n0 - c
lor = np.log(a * d / (b * c))
se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
z = lor / se
return np.exp(lor), np.exp(lor - 1.96 * se), np.exp(lor + 1.96 * se), 2 * stats.norm.sf(abs(z)), z
def snd(a, n1, c, n0):
"""Pooled two-proportion z test (the 'SND test' named in the Methods)."""
p1, p0, pp = a / n1, c / n0, (a + c) / (n1 + n0)
z = (p1 - p0) / np.sqrt(pp * (1 - pp) * (1 / n1 + 1 / n0))
return z, 2 * stats.norm.sf(abs(z)), stats.norm.sf(z)
def wilson(x, n, z=1.96):
p = x / n
den = 1 + z * z / n
c = (p + z * z / (2 * n)) / den
h = z * np.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / den
return 100 * p, 100 * (c - h), 100 * (c + h)
def table22(a, n1, c, n0):
return np.array([[a, n1 - a], [c, n0 - c]])
R = {} # recomputed results reused by the figures
print("=" * 78)
print("1. Group sizes")
print(f" 145 + 87 + 1102 = {sum(N.values())} (paper: 1334); 145/1334 = {100*145/1334:.1f}% (paper: 10.9%)")
print("=" * 78)
print("2. Table 2 percentages at seroconversion (recomputed from the printed n)")
for key in [("seroconversion", "GADA+"), ("seroconversion", "Multipositive")]:
x = T2[key]
pct = [100 * xi / ni for xi, ni in zip(x, N.values())]
print(f" {key[1]:14s} {pct[0]:.1f} / {pct[1]:.1f} / {pct[2]:.1f} (paper: {T2_pct[key]})")
print("=" * 78)
print("3. Three-group tests at seroconversion (Table 2 p column)")
for key in [("seroconversion", k) for k in ["IAA+", "GADA+", "IA-2A+", "ZnT8A+", "Multipositive"]]:
x = T2[key]
tab = np.array([[xi, ni - xi] for xi, ni in zip(x, N.values())])
chi2, p3, _, _ = stats.chi2_contingency(tab, correction=False)
print(f" {key[1]:14s} Pearson chi2 = {chi2:.2f}, df 2, p = {p3:.3f} (paper, exact chi2: {T2_p[key]})")
print("=" * 78)
print("4. The disputed comparison: FDR- FDR+ (n=87) vs FDR- (n=1102), at seroconversion")
for k in ["GADA+", "Multipositive"]:
x1, x2, x0 = T2[("seroconversion", k)]
o, lo, hi, p, zw = woolf(x2, 87, x0, 1102)
zs, ps, p1s = snd(x2, 87, x0, 1102)
pf = stats.fisher_exact(table22(x2, 87, x0, 1102))[1]
pyates = stats.chi2_contingency(table22(x2, 87, x0, 1102), correction=True)[1]
plr = stats.chi2_contingency(table22(x2, 87, x0, 1102), correction=False, lambda_="log-likelihood")[1]
rep = T3[("seroconversion", k)][1]
R[("FDR- FDR+", k)] = dict(or_=o, lo=lo, hi=hi, p=p, pf=pf, ps=ps)
print(f" {k}: {x2}/87 ({100*x2/87:.1f}%) vs {x0}/1102 ({100*x0/1102:.1f}%), "
f"difference {100*(x2/87 - x0/1102):.1f} percentage points")
print(f" unadjusted OR {o:.2f} (95% CI {lo:.2f}, {hi:.2f}); Wald z = {zw:.2f}, p = {p:.3f}")
print(f" SND / two-proportion z = {zs:.2f}, two-sided p = {ps:.3f}, one-sided p = {p1s:.3f}")
print(f" Fisher exact p = {pf:.3f}; Yates chi2 p = {pyates:.3f}; likelihood-ratio p = {plr:.3f}")
print(f" Table 3 prints: OR {rep[0]} ({rep[1]}, {rep[2]}), p = {rep[3]}")
print("=" * 78)
print("5. The comparison the abstract gets right: FDR+ (n=145) vs FDR- (n=1102), at seroconversion")
for k in ["GADA+", "Multipositive"]:
x1, x2, x0 = T2[("seroconversion", k)]
o, lo, hi, p, zw = woolf(x1, 145, x0, 1102)
pf = stats.fisher_exact(table22(x1, 145, x0, 1102))[1]
zs, ps, _ = snd(x1, 145, x0, 1102)
rep = T3[("seroconversion", k)][0]
R[("FDR+", k)] = dict(or_=o, lo=lo, hi=hi, p=p, pf=pf, ps=ps)
print(f" {k}: {x1}/145 vs {x0}/1102; OR {o:.2f} ({lo:.2f}, {hi:.2f}), Wald p = {p:.3f}, "
f"SND p = {ps:.3f}, Fisher p = {pf:.3f} (Table 3: OR {rep[0]} ({rep[1]}, {rep[2]}), p = {rep[3]})")
print("=" * 78)
print("6. Alternative readings of the abstract sentence")
print(" (a) pooled 'ever FDR+' (145 + 87 = 232) vs FDR- -- not what the sentence says ('both ... were')")
for k in ["GADA+", "Multipositive"]:
x1, x2, x0 = T2[("seroconversion", k)]
o, lo, hi, p, _ = woolf(x1 + x2, 232, x0, 1102)
print(f" {k}: {x1+x2}/232 vs {x0}/1102, OR {o:.2f} ({lo:.2f}, {hi:.2f}), p = {p:.4f}")
print(" (b) FDR- FDR+ vs FDR+ (are the two family-history groups different from each other?)")
for k in ["GADA+", "Multipositive"]:
x1, x2, x0 = T2[("seroconversion", k)]
print(f" {k}: {x2}/87 vs {x1}/145, Fisher p = {stats.fisher_exact(table22(x2, 87, x1, 145))[1]:.3f}")
print(" (c) three-group test without the FDR- FDR+ group's contribution is not needed:"
" the omnibus test only says the three groups are not all equal.")
print(" (d) smallest one-sided p for FDR- FDR+ vs FDR- (GADA, multipositive):",
f"{snd(52, 87, 578, 1102)[2]:.3f}, {snd(27, 87, 257, 1102)[2]:.3f}")
print("=" * 78)
print("7. Side observations: every Table 3 cell recomputed as an unadjusted OR from Table 2 counts")
print(" (flag '<--': rounded OR differs from the print by more than 0.1, or p differs by more than 0.05;"
" none bears on the abstract claim)")
FLAGGED = []
for key, (r1, r2) in T3.items():
x1, x2, x0 = T2[key]
for lab, a, n1, r in [("FDR+ vs FDR-", x1, 145, r1), ("FDR- FDR+ vs FDR-", x2, 87, r2)]:
o, lo, hi, p, _ = woolf(a, n1, x0, 1102)
flag = " <--" if abs(round(o, 1) - r[0]) > 0.1 + 1e-9 or (r[3] > 0.001 and abs(p - r[3]) > 0.05) else ""
if flag:
FLAGGED.append((key, lab))
rp = "<0.001" if r[3] == 0.0005 else r[3]
ps_ = "<0.001" if p < 0.001 else f"{p:.3f}"
print(f" {key[0]:14s} {key[1]:14s} {lab:18s} OR {o:.2f} ({lo:.2f}, {hi:.2f}) p={ps_}"
f" printed {r[0]} ({r[1]}, {r[2]}) p={rp}{flag}")
print(f" flagged cells: {len(FLAGGED)} "
f"({sum(l == 'FDR- FDR+ vs FDR-' for _, l in FLAGGED)} FDR- FDR+ vs FDR-, "
f"{sum(l == 'FDR+ vs FDR-' for _, l in FLAGGED)} FDR+ vs FDR-)")
r = T3[("follow-up", "GADA+")][1]
print(f" GADA+ during follow-up, FDR- FDR+ vs FDR-: geometric midpoint of printed CI ({r[1]}, {r[2]}) = "
f"{np.sqrt(r[1] * r[2]):.1f}; printed OR {r[0]}")
# ================================================================= figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAYBOX = "#eb6834", "#2a78d6", "#e4e3df", "#52514e"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": FG,
"axes.labelcolor": FG, "xtick.color": FG2, "ytick.color": FG2,
"axes.edgecolor": "#b9b8b3", "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG,
})
W = 7.2
LW = 2.0 # 2 px-equivalent line weight in points
MS = 8.5 # marker size (pt)
def save(fig, path):
fig.savefig(path + ".svg", facecolor=BG)
fig.savefig(path + ".png", dpi=200, facecolor=BG)
plt.close(fig)
def box(ax, x, y, w, h, edge, lw=2.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc=BG, ec=edge, lw=lw, transform=ax.transData))
def fmt_or(o, lo, hi, nd=1):
return f"{o:.{nd}f} ({lo:.{nd}f}, {hi:.{nd}f})"
G = R[("FDR- FDR+", "GADA+")]; M = R[("FDR- FDR+", "Multipositive")]
GP = R[("FDR+", "GADA+")]; MP = R[("FDR+", "Multipositive")]
# ---------------------------------------------- Figure 1: two text panels
def fig_text_panels(path):
fig = plt.figure(figsize=(W, 5.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 78); ax.axis("off")
# left: abstract
box(ax, 2, 30, 41, 45, ORANGE)
ax.text(5, 71, "Abstract, Results", weight="bold", fontsize=12.5, va="top")
ax.text(5, 65.5, "“At seroconversion, both FDR+ and\n"
"FDR− FDR+ children were\n"
"significantly more often positive for\n"
"GADA and multiple autoantibodies\n"
"than children without affected FDRs\n"
"(FDR−).”",
fontsize=10.5, va="top", linespacing=1.45)
ax.text(5, 39.5, "Claim for FDR− FDR+ vs FDR−:", color=FG2, fontsize=10.5, va="top")
ax.text(5, 35.5, "significant (as reported)", color=ORANGE, weight="bold", fontsize=11.5, va="top")
# right: body
box(ax, 57, 30, 41, 45, GRAYBOX)
ax.text(60, 71, "Results text and Table 3", weight="bold", fontsize=12.5, va="top")
ax.text(60, 65.5, "“…but no difference was observed\n"
"between FDR− FDR+ and FDR−\n"
"children (Table 3).”", fontsize=10.5, va="top", linespacing=1.45)
ax.text(60, 52, "Table 3, FDR− FDR+ vs FDR−,\nOR (95% CI), p:", color=FG2, fontsize=10.5, va="top",
linespacing=1.3)
ax.text(60, 44.5, "GADA+: 1.3 (0.9, 2.1), p = 0.201\n"
"Multipositive: 1.5 (0.9, 2.4), p = 0.106",
fontsize=10, va="top", color=ORANGE, linespacing=1.5)
ax.text(60, 35.5, "not significant (as reported)", color=ORANGE, weight="bold", fontsize=11.5, va="top")
# connector
ax.add_patch(FancyArrowPatch((44, 52), (56, 52), arrowstyle="<|-|>", mutation_scale=16, lw=LW, color=FG))
ax.text(50, 56, "conflict", ha="center", weight="bold", fontsize=11)
ax.text(50, 47.5, "same\ncomparison,\nopposite\nverdict", ha="center", va="top", color=FG2, fontsize=9.5,
linespacing=1.3)
# bottom: recomputed
box(ax, 2, 3, 96, 23, BLUE)
ax.text(5, 22.5, "Recomputed from the Table 2 counts (unadjusted logistic regression)",
color=BLUE, weight="bold", fontsize=10.5, va="top")
ax.text(5, 16.5,
f"GADA+: 52/87 vs 578/1102, OR {fmt_or(G['or_'], G['lo'], G['hi'], 2)}, "
f"p = {G['p']:.2f} (Fisher exact {G['pf']:.2f})\n"
f"Multipositive: 27/87 vs 257/1102, OR {fmt_or(M['or_'], M['lo'], M['hi'], 2)}, "
f"p = {M['p']:.2f} (Fisher exact {M['pf']:.2f})\n"
"Both intervals include OR = 1, as Table 3 and the Results text state.",
fontsize=10, va="top", linespacing=1.6)
save(fig, path)
# ---------------------------------------------- Figure 2: forest plot (reported vs recomputed)
ROWS = [ # label, reported tuple, recomputed dict, disputed?
("GADA+\nFDR+ vs FDR−", T3[("seroconversion", "GADA+")][0], GP, False),
("GADA+\nFDR− FDR+ vs FDR−", T3[("seroconversion", "GADA+")][1], G, True),
("Multipositive\nFDR+ vs FDR−", T3[("seroconversion", "Multipositive")][0], MP, False),
("Multipositive\nFDR− FDR+ vs FDR−", T3[("seroconversion", "Multipositive")][1], M, True),
]
def forest(ax, rows, ptext=True):
ax.set_xscale("log")
ax.set_xlim(0.6, 4.0)
ticks = [0.6, 0.8, 1, 1.5, 2, 3, 4]
ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
ax.axvline(1, color=FG2, lw=0.9, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
n = len(rows)
for i, (lab, rep, rc, disp) in enumerate(rows):
y = n - 1 - i
if disp:
ax.axhspan(y - 0.45, y + 0.45, color="#f3f2ee", zorder=0)
ax.plot([rep[1], rep[2]], [y + 0.14] * 2, color=ORANGE, lw=LW, zorder=3, solid_capstyle="butt")
ax.plot(rep[0], y + 0.14, "o", color=ORANGE, ms=MS, zorder=4)
ax.plot([rc["lo"], rc["hi"]], [y - 0.14] * 2, color=BLUE, lw=LW, zorder=3, solid_capstyle="butt")
ax.plot(rc["or_"], y - 0.14, "D", color=BLUE, ms=MS - 1, zorder=4)
if ptext:
ax.text(4.25, y + 0.14, f"p = {rep[3]:.3f}", color=ORANGE, va="center", fontsize=10)
ax.text(4.25, y - 0.14, f"p = {rc['p']:.3f}", color=BLUE, va="center", fontsize=10)
ax.set_yticks(range(n)); ax.set_yticklabels([r[0] for r in rows][::-1], fontsize=10, color=FG)
ax.set_ylim(-0.6, n - 0.4)
ax.set_xlabel("Odds ratio at seroconversion (log scale); line at OR = 1 means no difference")
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def legend_handles():
from matplotlib.lines import Line2D
return [Line2D([], [], color=ORANGE, marker="o", ms=MS, lw=LW, label="Reported (Table 3), 95% CI"),
Line2D([], [], color=BLUE, marker="D", ms=MS - 1, lw=LW,
label="Recomputed from Table 2 counts, 95% CI")]
def fig_forest(path):
fig, ax = plt.subplots(figsize=(W, 4.4))
fig.subplots_adjust(left=0.27, right=0.83, top=0.80, bottom=0.14)
forest(ax, ROWS)
ax.legend(handles=legend_handles(), loc="lower left", bbox_to_anchor=(-0.36, 1.08), ncol=2,
frameon=False, fontsize=9.6, handlelength=2.2)
for y in [2, 0]:
ax.text(0.62, y + 0.36, "abstract: “significantly more often”", fontsize=9.3, color=FG2,
va="center", style="italic")
save(fig, path)
if __name__ == "__main__":
fig_text_panels(os.path.join(HERE, "fig1"))
fig_forest(os.path.join(HERE, "fig2"))
print("=" * 78)
print("figures written: fig1.svg/.png, fig2.svg/.png")
The 2013 estimate of MSM living with HIV lies above its own 90% upper limit
Epidemiology (2016) · doi:10.1097/EDE.0000000000000423 · PMC4733816
Summary
The paper's main applied result is the number of men who have sex with men (MSM) living with HIV in the UK in 2013, given in the Abstract and Results as 48,310 (90% plausibility range 39,900–45,560). The Methods define the point estimate as the median and the range limits as the 5th and 95th percentiles of one distribution of model outputs, so the median cannot exceed the upper limit. Here it does, by 2,750. The point estimate is consistent with the paper's other numbers (its four CD4 strata sum to exactly 48,310), and the paper's own Figure 3A shows the 2013 range running from about 40,000 to about 60,400. The printed upper limit is therefore wrong, and as printed it understates the upward uncertainty of the headline estimate by roughly 15,000 people, in a paper whose stated contribution is a method that produces such ranges.
What the paper reports
Abstract:
In 2013, 48,310 (90% plausibility range: 39,900–45,560) men who have sex with men were estimated to be living with HIV in the UK, of whom 10,400 (6,160–17,350) were undiagnosed.
Results, third paragraph of "Calibration to Data on MSM in the UK and Resulting Estimates for 2013":
By 2013, an estimated 48,310 (90% plausibility range: 39,900–45,560) MSM were living with HIV in the UK (Figure 3), and 2,300, 5,500, 11,830, and 28,680 individuals had CD4 count ≤200, 201–350, 351–500, and >500 cells/mm3 respectively. It is thought that 10,400 (6,160–17,350) MSM were living with undiagnosed HIV, which means the undiagnosed proportion is 22% (13%–36%, Figure 3).
Methods, third paragraph of "How the Model Is Calibrated to Country Data", which defines the estimate and the range:
The median, 5th and 95th percentiles of the distribution of these model outputs at each calendar year are considered the point estimate and plausibility range limits, respectively.
Results, first paragraph of the same subsection, which says which runs the results come from:
The following results presented are those generated using the 742 parameter sets where the calibration score was <0.2 (median: 0.178, min: 0.107, max: 0.199) among the 1,000 resampled simulations.
Figure 3 legend:
Estimates of the (A) total number of MSM living with HIV in the UK and (B) total number of MSM living with undiagnosed HIV, by calendar year. Columns and error bars: Modeled median and 90% plausibility range.
Discussion, second paragraph (point estimate only):
The number of MSM living with HIV in the UK is now estimated to be 48,310 for 2013.
Discussion, final (tenth) paragraph, on what the method contributes:
This novel method allows us to also naturally produce plausibility ranges for these estimates, which describe the uncertainty surrounding the estimates, based on the data to which the model was calibrated.
What the paper's own numbers show
Method. I checked the reported estimate against its range using the paper's definition (median between the 5th and 95th percentiles; this holds for any distribution, and recompute.py.txt confirms it on four very different simulated ones). To see which of the three numbers is at fault, I summed the CD4 strata and the undiagnosed CD4 strata given in the same paragraph and recomputed the undiagnosed proportion. I checked every other estimate-with-range in the Results the same way, and expressed each as the position of the estimate within its range, (estimate − lower)/(upper − lower), which must lie between 0 and 1. Finally I read the 2013 column of the paper's Figure 3A, which the Results cite for this estimate. I fitted a straight line to the pixel rows of the eight labelled y-axis tick marks (0 to 70,000) on the PMC image ede-27-247-g004.jpg; one pixel row is 165 people, and the largest tick-mark residual is 154. I then converted the measured rows of the grey bar and the diamond. Every number below is printed by recompute.py.txt.
In the table, Results paragraphs are counted from 1 within the subsection "Calibration to Data on MSM in the UK and Resulting Estimates for 2013".
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| MSM living with HIV, 2013: is the estimate inside its range? | Abstract; Results, 3rd paragraph | 48,310 (39,900–45,560) | No. 48,310 − 45,560 = 2,750 above the upper limit |
| Lower limit to estimate | Abstract; Results, 3rd paragraph | 39,900 | 8,410 below the estimate; possible |
| Required upper limit (95th percentile ≥ median) | Methods, 3rd paragraph of "How the Model Is Calibrated to Country Data" | 45,560 | must be ≥ 48,310 |
| Sum of CD4 strata, 2013 | Results, 3rd paragraph | 2,300 + 5,500 + 11,830 + 28,680 | 48,310, equal to the point estimate |
| Sum of undiagnosed CD4 strata, 2013 | Results, 3rd paragraph | 710 + 1,930 + 3,040 + 4,720 | 10,400, equal to the undiagnosed estimate |
| Undiagnosed proportion | Results, 3rd paragraph | 22% (13%–36%) | 10,400 / 48,310 = 21.5%, rounds to 22% |
| Undiagnosed MSM, 2013 | Abstract; Results, 3rd paragraph | 10,400 (6,160–17,350) | inside; position 0.38 |
| Infections per year, 2010–2013 | Abstract; Results, 2nd paragraph | 3,210 (1,730–5,350) | inside; position 0.41 |
| Cumulative infections by end of 2012 | Results, 2nd paragraph | 67,720 (56,470–78,800) | inside; position 0.50 |
| Asymptomatic diagnosis rate per 3 months | Results, 2nd paragraph | 0.042 (0.025–0.069) | inside; position 0.39 |
| MSM living with HIV, 2013, as printed | Abstract; Results, 3rd paragraph | 48,310 (39,900–45,560) | position 1.49 |
| Figure 3A, 2013 column, read off | Figure 3A | median and 90% range drawn, no numbers | median about 48,500; range about 40,000–60,400 (position 0.42) |
| Figure 3A, 2012 and 2011 columns, read off | Figure 3A | as above | 2012: about 46,100 (38,200–56,400); 2011: about 43,800 (36,200–52,300) |
| Figure 3A minus text, 2013 | Figure 3A vs Abstract | — | median +190, lower limit +100, upper limit +14,840 |
| Check of the reading method: Figure 3B, undiagnosed, 2013 | Figure 3B vs Results, 3rd paragraph | 10,400 (6,160–17,350) | read off as about 10,400 (6,100–17,400) |
| eAppendix row described as "the same as the main result" | eAppendix eTable 6 (threshold 0.20, weights A) | 121 parameter sets; 51,000 (41,400–61,000) | upper limit above the median; but the set count and median differ from the main text (742 sets, 48,310) |
| Upper limits in all nine sensitivity scenarios | eAppendix eTable 6 | 59,700 to 61,500 | none is below 48,310 |
The point estimate fits every other number in the Results, and the lower limit agrees with Figure 3A to within reading precision. Only the upper limit fails, and Figure 3A, which plots the same quantity from the same model runs, puts it at about 60,400, not 45,560. The figure reading is approximate (about ±200 people), but the difference from the text is about 74 times larger than that. The same reading method applied to the 2013 column of Figure 3B (undiagnosed) gives about 10,400 (6,100–17,400), against the printed 10,400 (6,160–17,350), so the method recovers a range the text prints correctly.
Figure
Figure 1. MSM living with HIV in the UK in 2013, on one axis of people (the axis starts at 30,000; no bar lengths are compared). Orange: the estimate and 90% plausibility range printed in the Abstract and Results, 48,310 (39,900–45,560). The printed upper limit lies 2,750 to the left of the estimate, which the Methods' definition (median ≤ 95th percentile) does not allow. Blue: the same quantity read from the 2013 column of the paper's Figure 3A, about 48,500 (40,000–60,400). Grey: the eAppendix eTable 6 row that the supplement describes as the main result, 51,000 (41,400–61,000), shown for context; it comes from 121 parameter sets, not the 742 the main text uses. The dashed line marks the reported estimate. The figure reading and the supplement agree that the upper limit is near 60,000.
Figure 2. For each estimate with a 90% range in the Results, the position of the point estimate within its range, (estimate − lower)/(upper − lower). A median must fall between 0 (the lower limit) and 1 (the upper limit). The five other ranges in the Results (blue) put the estimate at 0.38 to 0.50, as does the Figure 3A reading for the 2013 total (0.42). The printed 2013 total (orange) is at 1.49, outside its own range.
Why it matters
- It is the paper's headline applied number. The range appears in the Abstract and in the Results paragraph that reports the main estimate for the UK. Readers who quote the Abstract will carry an impossible interval, and the error is not obvious unless the reader checks the ordering.
- As printed, it understates uncertainty, all on one side. The printed range spans 5,660 people and ends below the estimate. The paper's own figure suggests a span of about 20,400 people, about 12,090 of them above the reported estimate. A reader who used the printed interval, for example to plan services or to compare with other UK estimates, would wrongly conclude that values above 45,560 are implausible under the model.
- The method's stated contribution is uncertainty ranges. The Discussion says the method naturally produces "plausibility ranges for these estimates, which describe the uncertainty surrounding the estimates". The main applied range is the paper's showcase of that feature.
- What is not affected. The point estimate of 48,310, the CD4 breakdown, the undiagnosed estimate of 10,400 (6,160–17,350), the incidence and diagnosis-rate estimates and their ranges, the pseudo-data validation (Figure 5), and the conclusions about data availability do not depend on the printed upper limit. Figure 3A itself appears correct.
Explanations considered
- A different definition of the estimate or the range. This does not account for the numbers. The Methods name the median and the 5th and 95th percentiles of one distribution of model outputs. A median cannot exceed its own 95th percentile under any distribution, so no reading of the definition allows the printed range. A mean outside a percentile interval would be possible in principle, but the paper calls the estimate a median, and Figure 3's legend says "Modeled median".
- The two numbers refer to different quantities, years or runs. This does not account for the numbers. Both sentences give the estimate and range together, for the same quantity (total MSM living with HIV in the UK, 2013), and both cite Figure 3, whose 2013 median matches 48,310. The "based on prior distributions" outputs and the pseudo-epidemic results (Figure 5) are separate and are not the source.
- The point estimate is the wrong number, not the upper limit. This is ruled out by the paper's other numbers. 48,310 is the exact sum of the four CD4 strata, it gives the stated 22% undiagnosed, it is repeated in the Discussion, and it matches the Figure 3A median to within reading precision. The lower limit matches Figure 3A as well. Only the upper limit disagrees.
- A conversion artifact in the PMC text. This is excluded. The same "48,310 (90% plausibility range: 39,900–45,560)" appears in the PMC-archived text, on the PMC article page and in the Europe PMC full-text XML (the last two fetched on 2026-10-23), and "48,310 (90% plausibility range:39,900-45,560)" appears in the Abstract and Results of the manuscript file ede-27-247-s001.docx, which is independent of the PMC text conversion. The publisher's typeset PDF was not re-verified for this write-up (see Status).
- A typographical slip in one place. The same range appears in the Abstract, the Results and both places in the manuscript version supplied as supplementary file ede-27-247-s001.docx. So the error was present before typesetting and was carried into both places. The published material does not show how it arose. Possible correct values include a limit near 60,400 (Figure 3A) or 61,000 (the eTable 6 row described as the main result), but the exact figure cannot be recovered from the paper.
- A side observation, not needed for the main finding. The eAppendix states that its eTable 6 row for threshold 0.20 and weights A "is the same as the main result presented in the main manuscript", yet that row reports 121 parameter sets and 51,000 (41,400–61,000), while the main text reports 742 sets and 48,310. The supplement's own description of the calibration (page 20) says "The results in the manuscript are based on the 742 runs", so eTable 6 may report parameter sets rather than resampled runs, or a separate run. Either way, its upper limit, like those of all nine sensitivity scenarios (59,700 to 61,500), is well above 48,310.
Status
- Version of record. The archived PMC text, the PMC article page and the Europe PMC full-text XML (the last two fetched on 2026-10-23) all print 48,310 (90% plausibility range: 39,900–45,560). The publisher's typeset PDF was not re-verified: the Europe PMC PDF links and journals.lww.com (where the doi.org link resolves) refused automated access on 2026-10-23. Every quote above was confirmed verbatim against the PMC-archived text with
src/show.py, except the eAppendix sentences, which I read from the supplement PDF. - Figure. I read Figure 3 from the image ede-27-247-g004.jpg (800 × 1244 px; the same file is in the Europe PMC supplementaryFiles zip) and measured the 2011–2013 columns of panel A pixel by pixel. The pixel rows are recorded in
recompute.py.txt. The image is not reproduced here. Read the same way, panel B's 2013 column gives about 10,400 (6,100–17,400), which matches the printed undiagnosed estimate 10,400 (6,160–17,350). - Supplements. (a) The eAppendix (the article's Supplemental Digital Content 1, links.lww.com/EDE/A994, 30 pages), downloaded on 2026-10-23: I read page 20 (742 runs) and eTable 6 (pages 23–24), including the sentence on page 24 that the threshold 0.20, weights A row "is the same as the main result". The eAppendix does not print 45,560 anywhere, and it does not resolve the discrepancy. (b) The PMC supplementary file ede-27-247-s001.docx is a manuscript version of the article. I checked it directly on 2026-10-23, from the zip returned by the Europe PMC supplementaryFiles service (www.ebi.ac.uk/europepmc/webservices/rest/PMC4733816/supplementaryFiles). It prints "48,310 (90% plausibility range:39,900-45,560)" in both its Abstract and its Results, and 48,310 again in its Discussion.
- Corrections. One erratum exists: Epidemiology 2016;27(3):e24, doi:10.1097/EDE.0000000000000474, PMC6855333, PMID 27043876. I read its PMC page on 2026-10-23. It corrects only the Open Access footnote, to a CC BY 4.0 licence statement, and does not mention the estimate or its range. Crossref lists no update-to, updated-by or relation entries for the article. PMC and Europe PMC list no other notice, and there is no retraction or expression of concern.
- Comments. Not re-verified for this write-up: a PubPeer search on 2026-10-23 was refused by a human-verification check, and no Retraction Watch search was recorded.
- As of 2026-10-23, the range 39,900–45,560 is uncorrected.
- Sources. PMC4733816 (archived XML and article page); Europe PMC full-text XML and supplementaryFiles zip (manuscript docx and Figure 3 image ede-27-247-g004.jpg); eAppendix (links.lww.com/EDE/A994); erratum PMC6855333; Crossref.
Suggested correction
- Replace the upper limit of the 2013 estimate in both the Abstract and the Results ("48,310 (90% plausibility range: 39,900–45,560)") with the 95th percentile of the 742 accepted runs. The paper's Figure 3A puts it at about 60,400, so the corrected text would read about "48,310 (90% plausibility range: 39,900–60,400)" if the figure is right. The authors' model output is needed for the exact value.
- Confirm that the lower limit (39,900) and the point estimate (48,310) were taken from the same output. Both match Figure 3A, so no change is expected.
- Optionally, clarify in the eAppendix how the eTable 6 row described as "the same as the main result" (121 parameter sets; 51,000, 41,400–61,000) relates to the main text's 742 runs and 48,310.
Recomputation script (python recompute.py.txt)
"""Recompute the 2013 estimate of MSM living with HIV in the UK and its 90% plausibility
range, PMC4733816 (Epidemiology 2016;27(2):247-256, doi:10.1097/EDE.0000000000000423).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png, fig2.svg/png and five candidate designs in candidates/ next to this file.
Inputs are copied from the paper as archived in PubMed Central:
Abstract and Results ("Calibration to Data on MSM in the UK ...", first to third paragraphs)
Methods ("How the Model Is Calibrated to Country Data", third paragraph): the point estimate is
the median, and the range limits are the 5th and 95th percentiles of one distribution
Figure 3A (image ede-27-247-g004.jpg, 800 x 1244 px): pixel rows measured once from the
image and entered below; the image itself is not redrawn
eAppendix (links.lww.com/EDE/A994), eTable 6: sensitivity analysis
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs from the paper
EST, LO, HI = 48310, 39900, 45560 # Abstract; Results, 3rd paragraph
CD4_TOTAL = [2300, 5500, 11830, 28680] # Results, 3rd paragraph: <=200, 201-350, 351-500, >500
CD4_UNDIAG = [710, 1930, 3040, 4720] # Results, 3rd paragraph, undiagnosed
CD4_LABELS = ["CD4 ≤200", "201–350", "351–500", ">500"]
OTHER = [ # (label, location, estimate, lower, upper)
("Undiagnosed MSM, 2013", "Abstract; Results, 3rd paragraph", 10400, 6160, 17350),
("Undiagnosed proportion, 2013 (%)", "Results, 3rd paragraph", 22, 13, 36),
("Infections per year, 2010–2013", "Abstract; Results, 2nd paragraph", 3210, 1730, 5350),
("Cumulative infections by end 2012", "Results, 2nd paragraph", 67720, 56470, 78800),
("Asymptomatic diagnosis rate per 3 months", "Results, 2nd paragraph", 0.042, 0.025, 0.069),
]
# eAppendix eTable 6, row "threshold 0.20, weight A", which the eAppendix says "is the same as
# the main result presented in the main manuscript"
ETAB6_MAIN = dict(sets=121, est=51000, lo=41400, hi=61000)
ETAB6_ALL_HI = [60000, 60700, 60400, 61000, 61200, 60400, 61500, 59700, 61200] # 9 scenarios
MAIN_SETS = 742 # Results, 1st paragraph
# Figure 3A: pixel rows of the labelled y-axis tick marks (y, value) and the 2011-2013 columns, measured on the image
GRID = [(46, 70000), (107.5, 60000), (167.5, 50000), (228.5, 40000),
(288.5, 30000), (349.5, 20000), (411.5, 10000), (471, 0)]
FIG3A = { # year: (top row of grey bar, bottom row of grey bar, diamond top row, diamond bottom row)
2011: (154, 251, 201, 210),
2012: (129, 239, 187, 196),
2013: (105, 228, 173, 181),
}
# Figure 3B (undiagnosed), same image: y-axis tick-mark rows and the 2013 column, used as a check of the
# reading method against a range the text prints (10,400; 6,160-17,350)
GRID_B = [(696.5, 20000), (801.5, 15000), (908.5, 10000), (1015.5, 5000), (1121, 0)]
FIG3B_2013 = (752, 991, 895, 905)
# ------------------------------------------------------------------ calculations
print("=== 2013 MSM living with HIV: reported value and range ===")
print(f"Reported: {EST:,} (90% range {LO:,}-{HI:,})")
print(f"Estimate inside its own range? {LO <= EST <= HI}")
print(f"Estimate minus upper limit: {EST - HI:,}")
print(f"Estimate minus lower limit: {EST - LO:,}")
print(f"Reported range width: {HI - LO:,}")
print("\n=== Order constraint implied by the Methods (median <= 95th percentile) ===")
rng = np.random.default_rng(0)
ok = True
for name, draw in [("normal", rng.normal(48310, 5000, 10**5)),
("lognormal", rng.lognormal(np.log(48310), 0.2, 10**5)),
("right-skewed exponential", rng.exponential(1, 10**5)),
("discrete mixture", rng.choice([1, 2, 50], size=10**5, p=[.6, .3, .1]))]:
p5, p50, p95 = np.percentile(draw, [5, 50, 95])
ok &= p5 <= p50 <= p95
print(f" {name:25s} P5={p5:10.2f} P50={p50:10.2f} P95={p95:10.2f} ordered={p5 <= p50 <= p95}")
print(f"Ordering holds in every case: {ok} (it holds for any distribution by definition)")
print(f"So the 95th percentile must be >= {EST:,}; the printed {HI:,} is not.")
print("\n=== Is 48,310 the sound number? ===")
s_tot, s_und = sum(CD4_TOTAL), sum(CD4_UNDIAG)
print(f"Sum of CD4 strata: {' + '.join(f'{v:,}' for v in CD4_TOTAL)} = {s_tot:,} (reported {EST:,})")
print(f"Sum of undiagnosed CD4 strata: {' + '.join(f'{v:,}' for v in CD4_UNDIAG)} = {s_und:,} (reported 10,400)")
prop = 100 * 10400 / EST
print(f"Undiagnosed proportion 10,400 / 48,310 = {prop:.1f}% (reported 22%)")
print(f"Mirror of the lower gap above the estimate: {EST:,} + {EST - LO:,} = {EST + (EST - LO):,} (indicative only)")
print("\n=== Every other estimate-with-range in the Results ===")
pos = {}
for lab, loc, e, lo, hi in OTHER:
p = (e - lo) / (hi - lo)
pos[lab] = p
print(f" {lab:42s} {e} ({lo}-{hi}) inside={lo <= e <= hi} position in range={p:.2f}")
p_main = (EST - LO) / (HI - LO)
print(f" {'MSM living with HIV, 2013':42s} {EST} ({LO}-{HI}) inside={LO <= EST <= HI} position in range={p_main:.2f}")
print("\n=== Figure 3A read-off (paper's own chart of the same quantity) ===")
gy, gv = np.array(GRID).T
slope, icpt = np.polyfit(gy, gv, 1)
resid = gv - (slope * gy + icpt)
print(f"Calibration: {slope:.1f} people per pixel row; max tick-mark residual {np.abs(resid).max():.0f}")
fig3 = {}
for yr, (top, bot, d0, d1) in FIG3A.items():
hi_v = slope * (top - 0.5) + icpt
lo_v = slope * (bot + 0.5) + icpt
med_v = slope * ((d0 + d1) / 2) + icpt
fig3[yr] = (round(med_v, -2), round(lo_v, -2), round(hi_v, -2))
print(f" {yr}: median about {fig3[yr][0]:,.0f}; 90% range about {fig3[yr][1]:,.0f}-{fig3[yr][2]:,.0f}")
f_med, f_lo, f_hi = fig3[2013]
print(f"Figure 3A 2013 upper limit minus text upper limit: about {f_hi - HI:,.0f}")
print(f"Figure 3A 2013 lower limit minus text lower limit: about {f_lo - LO:,.0f}")
print(f"Figure 3A 2013 median minus text estimate: about {f_med - EST:,.0f}")
print(f"Reading precision: one pixel row = {abs(slope):.0f} people")
print(f"Figure 3A 2013 range width: about {f_hi - f_lo:,.0f}; above the reported estimate: about {f_hi - EST:,.0f}")
print(f"Printed range width: {HI - LO:,}; text-vs-figure upper-limit gap / 200-person reading error = {(f_hi - HI) / 200:.0f}")
gyb, gvb = np.array(GRID_B).T
sb, ib = np.polyfit(gyb, gvb, 1)
t, bt, d0, d1 = FIG3B_2013
print(f"Check on Figure 3B (undiagnosed, 2013): median about {round(sb * (d0 + d1) / 2 + ib, -2):,.0f}; "
f"range about {round(sb * (bt + 0.5) + ib, -2):,.0f}-{round(sb * (t - 0.5) + ib, -2):,.0f} "
f"(text: 10,400; 6,160-17,350); one pixel row = {abs(sb):.0f} people")
print("\n=== eAppendix eTable 6 ===")
print(f"Row described as the main result: {ETAB6_MAIN['sets']} parameter sets, "
f"{ETAB6_MAIN['est']:,} ({ETAB6_MAIN['lo']:,}-{ETAB6_MAIN['hi']:,}); main text: {MAIN_SETS} sets, {EST:,} ({LO:,}-{HI:,})")
print(f"Upper limits across all 9 sensitivity scenarios: {min(ETAB6_ALL_HI):,}-{max(ETAB6_ALL_HI):,}")
print(f"Number of scenarios with upper limit below {EST:,}: {sum(h < EST for h in ETAB6_ALL_HI)}")
# ------------------------------------------------------------------ figure style
BG, INK, INK2, GRIDC = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, GREY = "#eb6834", "#2a78d6", "#9a9893"
plt.rcParams.update({
"font.family": ["Inter", "DejaVu Sans"], "font.size": 10.5,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"text.color": INK, "axes.spines.top": False, "axes.spines.right": False,
"svg.fonttype": "none",
})
W = 7.2
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", bbox_inches="tight")
fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight")
plt.close(fig)
def kfmt(ax):
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{v:,.0f}"))
# ---- Candidate A: dot-and-interval rows on one axis --------------------------------------
def cand_a(path):
fig, ax = plt.subplots(figsize=(W, 3.9))
rows = [
("Reported in Abstract and Results", ORANGE, EST, LO, HI, "o"),
("Paper's Figure 3A, 2013 column (read off)", BLUE, f_med, f_lo, f_hi, "D"),
("eAppendix eTable 6, row called the main result", GREY, ETAB6_MAIN["est"],
ETAB6_MAIN["lo"], ETAB6_MAIN["hi"], "s"),
]
ys = [3, 1.7, 0.4]
ax.axvline(EST, color=INK2, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(EST, 3.72, f"reported estimate {EST:,}", ha="center", va="bottom", color=INK2, fontsize=10,
bbox=dict(fc=BG, ec="none", pad=1))
for (lab, col, e, lo, hi, m), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=2.5, solid_capstyle="butt", zorder=2)
for x in (lo, hi):
ax.plot([x, x], [y - 0.12, y + 0.12], color=col, lw=2, zorder=2)
ax.plot(e, y, m, color=col, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(30500, y + 0.36, lab, ha="left", va="center", fontsize=10.5, color=INK, zorder=4,
bbox=dict(fc=BG, ec="none", pad=1.5))
ax.text(max(hi, e) + 900, y, f"{e:,.0f} ({lo:,.0f}\u2013{hi:,.0f})", ha="left", va="center",
fontsize=10, color=INK2)
# gap between the printed upper limit and the estimate
ax.annotate("", xy=(EST, 3 - 0.36), xytext=(HI, 3 - 0.36),
arrowprops=dict(arrowstyle="<->", color=ORANGE, lw=1.5, shrinkA=0, shrinkB=0))
ax.text(EST + 900, 3 - 0.36, f"upper limit is {EST - HI:,} below the estimate",
ha="left", va="center", fontsize=10, color=INK)
ax.set_ylim(-0.05, 4.05)
ax.set_xlim(30000, 72000)
ax.set_yticks([])
ax.spines["left"].set_visible(False)
ax.grid(axis="x", color=GRIDC, lw=0.8)
ax.set_axisbelow(True)
kfmt(ax)
ax.set_xlabel("MSM living with HIV in the UK, 2013 (median and 90% plausibility range)")
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", lw=2.5, ms=8, label="Reported in the text"),
Line2D([], [], color=BLUE, marker="D", lw=2.5, ms=8, label="Read from the paper's own Figure 3A"),
Line2D([], [], color=GREY, marker="s", lw=2.5, ms=8, label="Supplement, eTable 6 (121 sets)")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=3, frameon=False,
fontsize=10, handlelength=2.2, columnspacing=1.2)
save(fig, path)
# ---- Candidate B: one number line with the order constraint -----------------------------
def cand_b(path):
fig, ax = plt.subplots(figsize=(W, 2.6))
ax.set_xlim(36000, 52000)
ax.set_ylim(-1.3, 1.75)
ax.axhline(0, color=INK2, lw=1)
marks = [(LO, "lower limit\n(5th percentile)", 39900), (HI, "upper limit\n(95th percentile)", 45560),
(EST, "point estimate\n(median)", 48310)]
for x, lab, _ in marks:
ax.plot(x, 0, "o", color=ORANGE, ms=10, mec=BG, mew=1.5, zorder=3)
ax.text(x, 0.28, f"{x:,}", ha="center", va="bottom", fontsize=11, fontweight="bold")
ax.text(x, -0.25, lab, ha="center", va="top", fontsize=10, color=INK2)
ax.annotate("", xy=(HI, 0.95), xytext=(EST, 0.95), arrowprops=dict(arrowstyle="->", color=ORANGE, lw=2))
ax.text((HI + EST) / 2, 1.03, f"the 95th percentile is {EST - HI:,} below the median",
ha="center", va="bottom", fontsize=10.5)
ax.text(36100, -1.2, "Required by the Methods' definition, for any distribution: "
"5th percentile ≤ median ≤ 95th percentile", ha="left", va="bottom", fontsize=10, color=BLUE)
ax.text(36100, 1.6, "Reported values (Abstract; Results), in order along one axis", ha="left", va="bottom", fontsize=10, color=INK2)
ax.set_yticks([])
for s in ("left", "bottom"):
ax.spines[s].set_visible(False)
ax.set_xticks([])
save(fig, path)
# ---- Candidate C: text panels joined by an annotated connector --------------------------
def cand_c(path):
fig, ax = plt.subplots(figsize=(W, 5.0))
ax.set_xlim(0, 10)
ax.set_ylim(0, 6.85)
ax.axis("off")
def panel(x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.12",
fc="#ffffff", ec=edge, lw=1.6))
ax.text(x + 0.18, y + h - 0.2, title, ha="left", va="top", fontsize=10, color=INK2, fontweight="bold")
ax.text(x + 0.18, y + h - 0.62, body, ha="left", va="top", fontsize=10.5, color=INK, linespacing=1.4)
panel(0.1, 3.6, 4.7, 3.15, "Abstract; Results, 3rd paragraph",
"\u201cIn 2013, 48,310 (90% plausibility\nrange: 39,900\u201345,560) men who\n"
"have sex with men were estimated\nto be living with HIV in the UK\u201d", ORANGE)
panel(5.2, 3.6, 4.7, 3.15, "Methods, model calibration",
"\u201cThe median, 5th and 95th\npercentiles of the distribution of\n"
"these model outputs ... are\nconsidered the point estimate and\nplausibility range limits\u201d", INK2)
panel(0.1, 0.1, 4.7, 2.5, "Results, 3rd paragraph: CD4 strata",
"2,300 + 5,500 + 11,830 + 28,680\n= 48,310, the point estimate\n"
"10,400 / 48,310 = 21.5%,\nthe stated 22% undiagnosed", BLUE)
panel(5.2, 0.1, 4.7, 2.5, "What the definition requires",
"median \u2264 95th percentile, so the\nupper limit must be \u2265 48,310\n"
"printed upper limit 45,560\nfalls 2,750 short", BLUE)
for xc in (2.45, 7.55):
ax.plot([xc, xc], [3.6, 3.32], color=ORANGE, lw=1.6)
ax.plot([xc, xc], [2.78, 2.6], color=BLUE, lw=1.6, ls=(0, (2, 2)))
ax.text(5.0, 3.05, "conflict: median 48,310 > 95th percentile 45,560",
ha="center", va="center", fontsize=10.5, color=INK, fontweight="bold",
bbox=dict(boxstyle="round,pad=0.35", fc=BG, ec=ORANGE, lw=1.5))
save(fig, path)
# ---- Candidate D: where each estimate sits inside its own range -------------------------
def cand_d(path):
labs = [o[0] for o in OTHER] + ["MSM living with HIV, 2013 (Fig. 3A read-off)",
"MSM living with HIV, 2013 (as printed)"]
vals = [pos[o[0]] for o in OTHER] + [(f_med - f_lo) / (f_hi - f_lo), p_main]
cols = [BLUE] * (len(OTHER) + 1) + [ORANGE]
fig, ax = plt.subplots(figsize=(W, 3.6))
y = np.arange(len(labs))[::-1]
ax.axvspan(0, 1, color="#f0efeb", zorder=0)
for xv, t in ((0, "lower limit"), (1, "upper limit")):
ax.axvline(xv, color=INK2, lw=1, zorder=1)
ax.text(xv, len(labs) - 0.35, t, ha="center", va="bottom", fontsize=10, color=INK2,
bbox=dict(fc=BG, ec="none", pad=1.5))
for yi, v, c, lab in zip(y, vals, cols, labs):
ax.plot([0, 1], [yi, yi], color=GRIDC, lw=1, zorder=1)
ax.plot(v, yi, "o", color=c, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(v + 0.04, yi, f"{v:.2f}", va="center", ha="left", fontsize=10, color=INK2)
ax.set_yticks(y)
ax.set_yticklabels(labs, fontsize=10)
ax.tick_params(axis="y", length=0)
ax.spines["left"].set_visible(False)
ax.set_xlim(-0.1, 1.7)
ax.set_ylim(-0.6, len(labs) - 0.1)
ax.set_xticks([0, 0.5, 1, 1.5])
ax.set_xlabel("Position of the point estimate within its own 90% range\n"
"(estimate − lower) / (upper − lower); must lie between 0 and 1")
from matplotlib.lines import Line2D
h = [Line2D([], [], color=ORANGE, marker="o", lw=0, ms=8, label="Reported 2013 total, as printed"),
Line2D([], [], color=BLUE, marker="o", lw=0, ms=8, label="Other ranges in the paper, and the Fig. 3A read-off")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.3, -0.3), ncol=2, frameon=False, fontsize=10)
save(fig, path)
# ---- Candidate E: calculation flow, CD4 strata summed against the range -----------------
def cand_e(path):
fig, ax = plt.subplots(figsize=(W, 2.9))
left = 0
shades = ["#9cc0ea", "#6fa3e2", "#4a8cdc", BLUE]
for v, lab, c in zip(CD4_TOTAL, CD4_LABELS, shades):
ax.barh(2, v, left=left, height=0.5, color=c, ec=BG, lw=2)
if v > 10000:
ax.text(left + v / 2, 2, f"{lab}\n{v:,}", ha="center", va="center", fontsize=10,
color="#ffffff" if c in (BLUE, "#4a8cdc") else INK)
left += v
ax.text(0, 2.42, f"Sum of the four reported CD4 strata = {s_tot:,}", ha="left", va="bottom", fontsize=10.5)
ax.text(0, 1.68, f"{CD4_LABELS[0]}: {CD4_TOTAL[0]:,} {CD4_LABELS[1]}: {CD4_TOTAL[1]:,}",
ha="left", va="top", fontsize=10, color=INK2)
ax.plot([LO, HI], [1, 1], color=ORANGE, lw=2.5)
for x in (LO, HI):
ax.plot([x, x], [0.85, 1.15], color=ORANGE, lw=2)
ax.plot(EST, 1, "o", color=ORANGE, ms=9, mec=BG, mew=1.5)
ax.text(0, 1, "Reported 90% range\n39,900–45,560 and estimate", ha="left", va="center", fontsize=10.5)
ax.axvline(EST, color=INK2, lw=0.8, ls=(0, (3, 3)))
ax.text(EST + 400, 0.5, f"{EST:,}", ha="left", va="center", fontsize=10, color=INK2)
ax.set_xlim(0, 60000)
ax.set_ylim(0.3, 2.85)
ax.set_yticks([])
ax.spines["left"].set_visible(False)
ax.grid(axis="x", color=GRIDC, lw=0.8)
ax.set_axisbelow(True)
kfmt(ax)
ax.set_xlabel("MSM living with HIV in the UK, 2013")
ax.text(HI - 200, 1.22, "upper limit ends 2,750 short of the estimate", ha="right", va="bottom",
fontsize=10, color=INK)
save(fig, path)
cand_a(os.path.join(CAND, "cand_A_interval_rows"))
cand_b(os.path.join(CAND, "cand_B_order_line"))
cand_c(os.path.join(CAND, "cand_C_text_panels"))
cand_d(os.path.join(CAND, "cand_D_position_in_range"))
cand_e(os.path.join(CAND, "cand_E_cd4_flow"))
cand_a(os.path.join(HERE, "fig1"))
cand_d(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)
Abstract reports a 31% fall in endotoxin activity on the prudent diet; Results report 38%
Gastroenterology (2012) · doi:10.1053/j.gastro.2012.01.034 · PMC3978718
Summary
This short crossover study fed 8 healthy inpatients a Western-style diet and a prudent-style diet for 1 month each and measured plasma endotoxin activity before and after each diet. The abstract says the prudent-style diet reduced endotoxin activity by 31%. The Results paragraph that cites Figure 1 says the decrease was 38%. Both passages describe the same subjects, assay and before/after comparison, and both give the same +71% for the Western-style diet. Rounding cannot produce a 7-point gap. The paper prints no group means for endotoxin activity, only the individual-subject plots in Figure 1. Reading those plots by eye gives a 37.5% fall in the group mean, which supports the Results figure. The abstract's 31% matches no single summary method that also gives +71% for the Western diet. The abstract is the number most likely to be quoted, so the size of the prudent-diet effect is now ambiguous in the literature. The direction of the effect, the Western-diet result and the paper's conclusions are unaffected.
What the paper reports
Abstract:
Placing 8 healthy subjects on a Western-style diet for 1 month induced a 71% increase in plasma levels of endotoxin activity (endotoxemia), whereas a prudent-style diet reduced levels by 31%.
Main text, Results paragraph (the paragraph that cites Figure 1 and Table 1):
Our study shows that 4 weeks of feeding a Western-style diet induced highly significant increases (by 71%) in plasma endotoxin activity levels, in contrast to a major decrease (by 38%) after the prudent-style diet (Figure 1) (P < .001).
Main text, same paragraph:
Baseline endotoxin activity levels were about identical
Main text, study description. This shows that both passages concern one group of subjects and one design:
Briefly, 8 healthy subjects were fed a Western-style and a “prudent-style” diet for 1 month as inpatients hospitalized in a metabolic ward in a crossover study separated by a 1-month washout period.
Figure 1 legend:
Data from 8 individual subjects shown as endotoxin activity units for the Western-style diet (WD) and the prudent-style diet (PD). Paired t tests were used to compare effects of the diets on endotoxin activity levels. The Mann–Whitney sum test was used to compare effects of the 2 diets on endotoxin activity levels.
In the Figure 1 image, panels A (Western diet, before vs after) and B (prudent diet, before vs after) are labelled "P≤.05". Panel D, which compares the two diets after the intervention, is labelled "P≤.001". Neither the text nor the figure gives a group mean, SD or percentage for endotoxin activity. Table 1 gives cytokine concentrations only, as before and after means with a "Change (%)" column.
What the paper's own numbers show
Method. The paper reports the endotoxin results only as two percentages and as individual-subject lines in Figure 1, panels A and B (8 subjects each). I read each subject's before and after values off the enlarged figure image by eye. Each reading is uncertain by about ±0.005 activity units. These readings are approximations, not the authors' data. From them I computed the percentage change in four standard ways: - the change in the group mean (mean after / mean before − 1); - the mean of each subject's own percentage change; - the geometric mean of the after/before ratios; - the change in the median.
To carry the reading error through, I added independent uniform ±0.005 noise to every point, repeated this 20,000 times and took the 2.5–97.5 percentile band of each statistic. As a check on which method the authors used elsewhere, I recomputed Table 1's cytokine percentages from its rounded means. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Prudent diet, change in endotoxin activity | Abstract | −31% | — (same quantity as the next row) |
| Prudent diet, change in endotoxin activity | Results paragraph citing Fig. 1 | −38% | — |
| Gap between the two printed values | Abstract vs Results | — | 7 percentage points (23% of the abstract value); post/baseline ratio 0.69 vs 0.62 |
| Western diet, change in endotoxin activity | Abstract and Results | +71% (both) | — |
| Prudent diet, group mean before → after | Fig. 1B (read by eye) | not reported | 0.115 → 0.072 units |
| Western diet, group mean before → after | Fig. 1A (read by eye) | not reported | 0.120 → 0.207 units |
| Prudent diet, change of group means | Fig. 1B | −31% / −38% | −37.5% (band −39.5 to −35.4) |
| Western diet, change of group means | Fig. 1A | +71% | +72.2% (band +69.0 to +75.5) |
| Prudent diet, mean of per-subject % changes | Fig. 1B | — | −33.3% (band −35.9 to −30.4) |
| Western diet, mean of per-subject % changes | Fig. 1A | — | +80.2% (band +75.6 to +85.3) |
| Prudent / Western, geometric mean ratio | Fig. 1A–B | — | −35.7% (−38.4 to −33.1) / +71.3% (+66.9 to +75.9) |
| Prudent / Western, change of medians | Fig. 1A–B | — | −28.7% (−33.5 to −24.1) / +78.3% (+71.8 to +85.5) |
| Within-diet paired t test, PD / WD | Results text; Fig. 1A–B labels | "P < .001" (text); "P≤.05" (figure) | p ≈ 0.020 / 0.017 (from figure readings) |
| Table 1, PD cytokines: TNF-α, MCP-1, IL-6, IL-8 | Table 1 | −17, −9, −11, −28% | −16.0, −8.7, −12.5, −26.3% (change of rounded means) |
| Table 1, WD cytokines: TNF-α, MCP-1, IL-6, IL-8 | Table 1 | −1, −9, −2, −13% | 0.0, −9.1, 0.0, −11.8% (change of rounded means) |
Three findings stand out: - The change in the group mean, read off Figure 1, reproduces both the Results' −38% and the shared +71%. - The geometric mean ratio also gives values close to −38% and +71%. - The only methods whose bands reach −31% are the mean of per-subject changes and the change of medians. Both give +78% to +80% for the Western diet, not the +71% printed beside the 31% in the abstract.
Table 1's printed percentages agree, within the rounding of its means, with the change-of-means method. This fits the 38% value, although rounded means cannot rule out other methods.
Figure
The two passages that report the prudent-diet result, as printed in the archived article. In orange are the paper's values: the abstract gives a 31% decrease and the Results paragraph citing Figure 1 gives a 38% decrease. Both concern the same 8 subjects, the same neutrophil-priming assay and the same before/after comparison, and both pair the value with an identical +71% for the Western-style diet. The footer gives the change of group means computed in recompute.py.txt from Figure 1 values read by eye.
Percentage change in endotoxin activity recomputed from the individual values in Figure 1, read by eye (blue dots), under four summary methods. The bars show the 2.5–97.5 percentile range when each reading is perturbed by ±0.005 units. The dashed orange lines are the printed values: −31% (abstract), −38% (Results) and +71% (both). The left cluster is the prudent diet and the right cluster the Western diet, on one shared axis. The change of group means and the geometric mean ratio match −38% and +71% together. No method matches −31% and +71% together.
Why it matters
The abstract is where readers, reviews and meta-analyses usually take effect sizes from. Here it gives a prudent-diet reduction (31%, ratio 0.69) that differs from the body (38%, ratio 0.62) by 7 percentage points, or about 23% of the abstract value. A reader who compares the two diets gets a different contrast depending on the source: a post-diet ratio of 1.71/0.69 ≈ 2.48 between diets if the abstract is right, against 1.71/0.62 ≈ 2.76 if the body is right. Because the paper prints no group means or SDs for endotoxin activity, a reader cannot tell from the article which value is correct. Anyone sizing a follow-up study or pooling this result has to choose between two figures the paper gives for the same quantity.
The error is limited in scope: - The Western-diet +71%, the result named in the title, is the same in both places. - The prudent diet lowers endotoxin activity in both statements. - The cytokine results in Table 1 and the paper's qualitative interpretation do not depend on which value is correct.
A smaller, related point concerns the P value. The Results sentence attaches "P < .001" to the within-diet changes. The Figure 1 panels for those changes (A and B) are labelled "P≤.05", and the figure readings give paired-t p-values of about 0.02. The "P≤.001" label appears only on panel D, the between-diet comparison after the intervention. The Results sentence may be citing the panel D value. As written, though, it reads as applying to the within-diet changes.
Explanations considered
- Different populations, time points, assays or units. Ruled out. The paper describes one cohort of 8 subjects, one crossover with 4 weeks on each diet, and one assay, with endotoxin activity measured before and after each diet. Both passages give the same +71% for the Western diet, which would be unlikely if they described different analyses.
- Rounding. Ruled out. A value that prints as 31% must lie between 30.5% and 31.5%, and one that prints as 38% must lie between 37.5% and 38.5%. The two ranges do not overlap.
- Different summary methods (change of means, mean of individual changes, geometric mean, medians). These do not reconcile the values. Under the readings of Figure 1, a method that gives about 31% for the prudent diet gives about 78–80% for the Western diet. A method that gives about 71% for the Western diet gives about 36–38% for the prudent diet. No single method produces the abstract's pair (−31%, +71%). It remains possible that the abstract used one method for the prudent diet and another for the Western diet, but the paper does not say so.
- Decrease expressed relative to the post-diet level, or as a ratio. This does not fit. Taking the change of means relative to the post-diet level gives (0.115 − 0.072)/0.072 ≈ 60%, not 31%. A post/baseline ratio of 0.69 is what a 31% decrease means, which is the abstract's claim, not a different quantity.
- Reading error in the figure. This affects only which of the two printed values the figure supports, not the existence of the inconsistency. The two printed values contradict each other whatever the figure shows. The reading-error bands (about ±2 points) are much narrower than the 7-point gap.
- Text conversion artifact in the archived copy. Unlikely. Both values appear as plain text with a % sign, not in tables or superscripts. Both sentences describe the change in words ("reduced", "decrease"), so a dropped sign would not explain the gap. The abstract's 31% also appears in the PubMed record, which derives from the publisher.
- Correction already published. None found (see Status).
Status
As of 2026-10-23:
- Version of record. I could not open the typeset publisher full text. ScienceDirect (pii S0016508512001588) and gastrojournal.org returned HTTP 403 or a bot challenge to automated access. The abstract's "reduced levels by 31%" is confirmed in the PubMed record (PMID 22326433), which derives from the publisher's abstract. The Results sentence with "by 38%" was checked in the PubMed Central copy, which is the NIH author manuscript (NIHMS561466, PMC3978718), using both the archived JATS XML and the rendered PMC page. I could not confirm whether the typeset Results text also reads 38%.
- Supplement. The only file online is one publisher page, NIHMS561466-supplement-supplement_1.pdf. It holds Supplementary Table 1 (diet composition) and Supplementary Table 2 (end-of-diet glucose, insulin and lipids). It contains no endotoxin values, so it cannot settle 31% vs 38%. The "Supplementary Patients and Methods" text cited in the article was not among the files available to me.
- Corrections and retractions. None in Crossref update metadata, PubMed Central notices or Europe PMC notices.
- PubPeer. The PubPeer search page and API require human verification and could not be queried automatically. A web search found no PubPeer thread for this article. I cannot rule out a thread that is not indexed.
- Sources. The PMC author manuscript PMC3978718 (text, Table 1, Figure 1 image nihms561466f1.jpg); PubMed 22326433; Crossref record for doi:10.1053/j.gastro.2012.01.034. All numbers are recomputed in
recompute.py.txt.
Suggested correction
The authors could state which value is correct for the change in plasma endotoxin activity after the prudent-style diet. They could also give the group mean (SD) endotoxin activity before and after each diet, and say how the percentage change was calculated (for example, change of group means or mean of per-subject changes). If the Results value is correct, the abstract would read: "…whereas a prudent-style diet reduced levels by 38%." If the abstract value is correct, the Results sentence would need the same correction. The authors could also say which comparison the "P < .001" in the Results sentence refers to, since Figure 1 labels the within-diet changes "P≤.05" and only the between-diet comparison after the intervention "P≤.001".
Recomputation script (python recompute.py.txt)
"""Recompute the numbers in the PMC3978718 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
Inputs
------
* Reported values (paper): abstract -31% (prudent diet, PD), +71% (Western
diet, WD); Results paragraph citing Figure 1: -38% (PD), +71% (WD);
Table 1 cytokine means before/after and printed % changes.
* Individual endotoxin activity values: the paper gives these only as plotted
points in Figure 1 (panels A and B, 8 subjects each). The values below were
read by eye from the enlarged figure image (PMC file nihms561466f1.jpg);
reading error is roughly +/-0.005 units per point. They are approximations,
not the authors' data.
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- reported
REP = {"abstract_PD": -31, "results_PD": -38, "abstract_WD": 71, "results_WD": 71}
# ------------------------------------------------- Figure 1, read by eye
# subject order (by plotting symbol): light-blue '+', purple 'x', blue diamond,
# orange circle, green triangle, brown, cyan, red square
pd_before = np.array([0.246, 0.138, 0.132, 0.120, 0.080, 0.077, 0.067, 0.062])
pd_after = np.array([0.120, 0.089, 0.080, 0.0625, 0.094, 0.047, 0.050, 0.034])
wd_before = np.array([0.235, 0.130, 0.172, 0.125, 0.105, 0.078, 0.050, 0.068])
wd_after = np.array([0.290, 0.375, 0.210, 0.200, 0.275, 0.105, 0.100, 0.103])
CONV = {
"Change of group means": lambda b, a: (a.mean() / b.mean() - 1) * 100,
"Mean of per-subject % changes": lambda b, a: np.mean(a / b - 1) * 100,
"Geometric mean ratio": lambda b, a: (np.exp(np.mean(np.log(a / b))) - 1) * 100,
"Change of medians": lambda b, a: (np.median(a) / np.median(b) - 1) * 100,
}
def jitter_band(b, a, f, n=20000, half=0.005, seed=0):
rng = np.random.default_rng(seed)
vals = np.empty(n)
for i in range(n):
bb = b + rng.uniform(-half, half, b.size)
aa = a + rng.uniform(-half, half, a.size)
vals[i] = f(bb, aa)
return np.percentile(vals, [2.5, 97.5])
print("=== Figure 1 (read by eye), endotoxin activity units ===")
for name, b, a in [("Prudent (PD)", pd_before, pd_after), ("Western (WD)", wd_before, wd_after)]:
t = stats.ttest_rel(a, b)
print(f"{name}: mean before {b.mean():.3f}, after {a.mean():.3f}; paired t p = {t.pvalue:.3f}")
print("Paper reports: PD -31% (abstract), -38% (Results); WD +71% (both)")
RES = {}
print("\n=== % change under each summary convention (2.5-97.5% band from +/-0.005 reading error) ===")
for cname, f in CONV.items():
pdv, wdv = f(pd_before, pd_after), f(wd_before, wd_after)
pdb, wdb = jitter_band(pd_before, pd_after, f), jitter_band(wd_before, wd_after, f)
RES[cname] = (pdv, pdb, wdv, wdb)
print(f"{cname:32s} PD {pdv:+6.1f} [{pdb[0]:+.1f}, {pdb[1]:+.1f}] "
f"WD {wdv:+6.1f} [{wdb[0]:+.1f}, {wdb[1]:+.1f}]")
print("\n=== Which conventions reproduce the printed pairs? (|recomputed - reported| <= band) ===")
for cname, (pdv, pdb, wdv, wdb) in RES.items():
ok31 = pdb[0] - 0.5 <= -31 <= pdb[1] + 0.5
ok38 = pdb[0] - 0.5 <= -38 <= pdb[1] + 0.5
ok71 = wdb[0] - 0.5 <= 71 <= wdb[1] + 0.5
print(f"{cname:32s} PD=-31? {ok31!s:5s} PD=-38? {ok38!s:5s} WD=+71? {ok71!s:5s}")
print("\n=== Gap between the two printed PD values ===")
print(f"38 - 31 = {38-31} percentage points; relative to abstract value: {(38-31)/31*100:.0f}%")
print(f"post/baseline ratio: abstract {1-0.31:.2f}, Results {1-0.38:.2f}")
print(f"Diet contrast (post/baseline WD / PD): abstract {1.71/0.69:.2f}, Results {1.71/0.62:.2f}")
print(f"PD decrease expressed relative to post-diet mean (change of means): "
f"{(pd_before.mean()-pd_after.mean())/pd_after.mean()*100:.0f}%")
print(f"Rounding: any value in [-31.5, -30.5) prints as 31%, any in [-38.5, -37.5) as 38%; the intervals do not overlap.")
print("\n=== Table 1: convention check (change of rounded means vs printed % change) ===")
T1 = [("PD TNF-a", 10, 8.4, -17), ("PD MCP-1", 195, 178, -9), ("PD IL-6", 1.6, 1.4, -11), ("PD IL-8", 19, 14, -28),
("WD TNF-a", 9, 9, -1), ("WD MCP-1", 198, 180, -9), ("WD IL-6", 1.8, 1.8, -2), ("WD IL-8", 17, 15, -13)]
for n, b, a, rep in T1:
print(f"{n:9s} before {b:>5} after {a:>5}: change of means {(a/b-1)*100:+6.1f}% printed {rep:+d}%")
# ================================================================ figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
BG, INK, INK2, ORG, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2})
def fmt(v):
return f"{v:+.1f}".replace("-", "\u2212")
# ---------- fig1: abstract vs Results text panels with connector
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 64); ax.axis("off")
def panel(x, title, lines, pdtxt):
ax.add_patch(FancyBboxPatch((x, 15), 42, 42, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(x + 2, 54.5, title, fontsize=11, fontweight="bold", va="top", color=INK)
y = 48.5
for ln in lines:
ax.text(x + 2, y, ln, fontsize=10, va="top", color=INK2)
y -= 4.3
ax.text(x + 21, 23.5, pdtxt, fontsize=17, fontweight="bold", ha="center", va="center", color=ORG)
ax.text(x + 21, 18.3, "Western diet: +71%", fontsize=10.5, ha="center", va="center", color=ORG)
panel(3, "Abstract", ["\u201cPlacing 8 healthy subjects on a",
"Western-style diet for 1 month",
"induced a 71% increase \u2026",
"whereas a prudent-style diet",
"reduced levels by 31%.\u201d"],
"Prudent diet: \u221231%")
panel(55, "Results (paragraph citing Fig. 1)", ["\u201c\u2026 increases (by 71%) in plasma",
"endotoxin activity levels, in contrast",
"to a major decrease (by 38%) after",
"the prudent-style diet (Figure 1)",
"(P < .001).\u201d"],
"Prudent diet: \u221238%")
ax.annotate("", xy=(54.2, 23.5), xytext=(45.8, 23.5),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(50, 26.5, "conflict", ha="center", va="bottom", fontsize=10, fontweight="bold", color=INK)
ax.text(50, 6.8, "Same 8 subjects, same assay, same before/after comparison: the prudent-diet decrease is printed\n"
f"as 31% in the abstract and 38% in the Results (gap {38-31} percentage points); the Western-diet value agrees.\n"
f"Figure 1, read by eye, change of group means: prudent {fmt(RES['Change of group means'][0])}%, "
f"Western {fmt(RES['Change of group means'][2])}% (shown in fig. 2).",
ha="center", va="center", fontsize=9.5, color=INK, linespacing=1.5)
ax.text(50, 61, "Values in orange are as printed in the paper", ha="center", va="center", fontsize=10, color=ORG)
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)
# ---------- fig2: conventions vs printed values, one axis
fig, ax = plt.subplots(figsize=(7.2, 4.0))
names = list(RES)
ys = np.arange(len(names))[::-1].astype(float)
top = len(names) - 0.5
ax.axvline(0, color=INK2, lw=0.8)
for v, lab, ha in [(-31, "abstract \u221231%", "left"), (-38, "Results \u221238%", "right"),
(71, "abstract and\nResults +71%", "right")]:
ax.plot([v, v], [-0.5, top], color=ORG, lw=2, ls=(0, (4, 2)))
ax.text(v + (0.8 if ha == "left" else -0.8), top + 0.05, lab, color=ORG, fontsize=9.5, ha=ha, va="bottom")
ax.text(-35, top + 0.75, "Prudent diet", ha="center", va="bottom", fontsize=10.5, fontweight="bold")
ax.text(75, top + 0.75, "Western diet", ha="center", va="bottom", fontsize=10.5, fontweight="bold")
for y, n in zip(ys, names):
pdv, pdb, wdv, wdb = RES[n]
for v, b in [(pdv, pdb), (wdv, wdb)]:
ax.plot(b, [y, y], color=BLU, lw=2, solid_capstyle="butt")
ax.plot(v, y, "o", ms=8, color=BLU)
ax.text(-58, y, fmt(pdv), color=BLU, fontsize=10, ha="left", va="center")
ax.text(110, y, fmt(wdv), color=BLU, fontsize=10, ha="right", va="center")
ax.set_yticks(ys); ax.set_yticklabels(names, color=INK)
ax.set_xlim(-60, 112); ax.set_ylim(-0.6, top + 1.35)
ax.set_xlabel("% change in endotoxin activity over 4 weeks on each diet")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o-", color=BLU, ms=8, lw=2, label="recomputed from Figure 1 values read by eye (bar: 95% reading-error band)")
ax.plot([], [], color=ORG, lw=2, ls=(0, (4, 2)), label="value printed in the paper")
ax.legend(loc="upper center", bbox_to_anchor=(0.4, -0.2), ncol=1, frameon=False, fontsize=9.5)
fig.tight_layout()
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
plt.close(fig)
print("\nfigures written to", HERE)
Abstract reverses the direction of the paper's own DDIT4 and gastric cancer survival finding
Gastroenterology (2020) · doi:10.1053/j.gastro.2020.09.016 · PMC7857017
Summary
The abstract states that "Lower DDIT4 expression correlated with longer survival" in gastric cancer. The rest of the paper reports the opposite for the only gastric cancer survival cohort it analyses, a tissue microarray (TMA) of 509 patients. The Introduction, Results, Figure 4 title and Discussion all say that high DDIT4 goes with longer survival, and the Kaplan-Meier curves in Figure 4B agree: the DDIT4-positive group (n = 200) lies above the DDIT4-negative group (n = 309), log-rank P = .021. The reversed sentence is also in the copyedited abstract indexed in PubMed, and no correction has been issued, so readers who see only the abstract get the direction of the paper's main human result backwards.
What the paper reports
Abstract, Results (PMC author manuscript):
Lower DDIT4 expression correlated with longer survival of gastric cancer patients.
Abstract, Results, as deposited in PubMed (PMID 32956680) and Europe PMC, in the journal's copyedited wording:
Lower DDIT4 expression correlated with longer survival of patients with gastric cancer.
Abstract, Conclusions:
During injury-induced metaplastic proliferation, failure of licensing mTORC1 reactivation correlates with increased proliferation of cells harboring DNA damage, as well as increased tumor formation and growth in mice and humans.
Introduction, final (5th) paragraph:
Patients with low expression have poorer survival.
Results, section "Loss of DDIT4 causes cell-autonomous increased growth of human gastric…", paragraph 1:
We examined another annotated tissue microarray on which we had compiled 509 patients with gastric cancer. We saw statistically significant correlation of high DDIT4 expression with increased patient survival and more differentiated tumors, especially in patients who had received chemotherapy, consistent with DDIT4 slowing proliferation in response to DNA damage (Figure 4A–C; Table S3).
Figure 4 title:
In humans, loss of DDIT4 correlates with worse gastric cancer survival and causes cell-autonomous increased mTORC1 and proliferation despite chemotherapy
Figure 4 legend, panel B:
Overall survival determined via Kaplan-Meier curves separated by DDIT4 expression, with an event defined as a cancer-related death.
Discussion, paragraph 3:
Our results showing DDIT4 expression correlating with less proliferation in precancerous lesions and better survival of patients with gastric cancer confirm and extend on those of a previous report43.
Discussion, paragraph 4:
Paradoxically, however, studies across multiple organs have shown disparate results for DDIT4 in tumorigenesis with some studies showing higher DDIT4 decreases cancer survival44, 47–50.
Figure 4B and 4C, legend boxes printed on the image (read from the PMC image file nihms-1661019-f0004.jpg, not from the text):
4B, Overall Patients: DDIT4 negative (n=309); DDIT4 positive (n=200); P = .021
4C, Patients receiving chemotherapy: DDIT4 negative (n=263); DDIT4 positive (n=170); P = .016
What the paper's own numbers show
Method. The paper reports no hazard ratio or survival estimate in its text, so there is no estimate to recompute. Instead I recorded the direction of the DDIT4 and survival association in every place the paper describes it, and checked the group sizes printed on Figure 4B and 4C against the cohort size in the Results. All quotes from the paper's text were matched verbatim against the PMC-archived text, and recompute.py.txt repeats this check for every one of them; the PubMed wording was checked against the PubMed efetch record, and the Figure 4B and 4C counts were read from the figure image. In Figure 4B and 4C the DDIT4-positive curve lies above the DDIT4-negative curve over most of follow-up. In this paper, "positive" is the high-expression group (Figure 4A shows cores scored "negative" or "positive").
| Quantity | Location | Reported | Recomputed / what the paper's own material shows |
|---|---|---|---|
| Direction of DDIT4 and survival | Abstract, Results (PMC and PubMed) | Low DDIT4 → longer survival | Opposite to every statement below and to Figure 4B |
| Direction of DDIT4 and survival | Introduction, final (5th) paragraph | Low DDIT4 → poorer survival | High DDIT4 → longer survival |
| Direction of DDIT4 and survival | Results, 509-patient TMA | High DDIT4 → increased survival | High DDIT4 → longer survival |
| Direction of DDIT4 and survival | Figure 4 title | Loss of DDIT4 → worse survival | High DDIT4 → longer survival |
| Direction of DDIT4 and survival | Discussion, paragraph 3 | DDIT4 expression → better survival | High DDIT4 → longer survival |
| Direction of DDIT4 and survival | Figure 4B and 4C curves | Positive curve above negative curve | High DDIT4 → longer survival |
| Body statements that agree | Intro, Results, Fig 4 title, Discussion, Fig 4B | n/a | 5 of 5 agree with each other; the abstract contradicts all 5 |
| Cohort size | Results text vs Figure 4B | 509 patients; 309 + 200 | 309 + 200 = 509 (matches) |
| Chemotherapy subgroup | Figure 4C | 263 + 170 | 433 patients, 85.1% of the 509 |
| DDIT4-positive share | Figure 4B / 4C | 200 of 509; 170 of 433 | 39.3% in both |
| Log-rank P | Figure 4B / 4C | .021 / .016 | Not recomputable from published material (no individual data) |
Figure
Figure 1. On the left (orange outline) is the abstract's Results sentence, quoted from the PMC author manuscript; the same sentence, in copyedited wording ("…of patients with gastric cancer"), is in the publisher-deposited abstract in PubMed. On the right (blue outline) are short excerpts from the Introduction, the Results, the Figure 4 title and the Discussion, plus the group sizes and log-rank P value printed on Figure 4B. They describe the same 509-patient gastric cancer tissue microarray (the Introduction sentence does not name the cohort, but the 509-patient TMA is the paper's only survival analysis) and the same endpoint (overall survival, with cancer-related death as the event). The abstract states that low DDIT4 goes with longer survival. Every body statement, and the Kaplan-Meier data, state or show that high DDIT4 goes with longer survival. The counts come from recompute.py.txt, which transcribes them from the Figure 4 image. The paper's own figure is not reproduced.
Why it matters
- The human survival association is the paper's only patient-outcome evidence linking DDIT4 loss to gastric cancer. The abstract's Conclusions draw on it in part ("increased tumor formation and growth in mice and humans", which also covers the human-cell xenograft and in-vitro data), and so does the paper's mechanism (DDIT4 restrains proliferation of DNA-damaged cells, so losing it should worsen outcomes). The abstract's Results sentence contradicts both the abstract's own Conclusions and the evidence behind them.
- The abstract is what most readers, indexing services, systematic reviewers and automated literature tools see. A reader of the abstract alone would conclude that DDIT4 is a marker of poor prognosis in gastric cancer, which is the reverse of what this study found.
- The confusion is easy to compound. The Discussion itself notes that other studies found "higher DDIT4 decreases cancer survival". A reader comparing studies by their abstracts could wrongly count this paper among them.
- This write-up does not question the survival analysis itself (Kaplan-Meier curves with a log-rank test) or the numbers printed on Figure 4B and 4C. Other estimates in the paper, and Supplementary Table 3, were not checked. The error is a reversed statement in the abstract, the most widely read part of the paper.
Explanations considered
- A different population or subgroup. This does not explain it. The paper has one gastric cancer survival cohort, the 509-patient TMA in Figure 4B. The only subgroup shown, patients who received chemotherapy (Figure 4C, n = 433), has the same direction (positive curve above negative, P = .016), and the Results call the association stronger there ("especially in patients who had received chemotherapy"). The second TMA, of 218 patients with normal and precancerous lesions, has no survival analysis.
- Results from other studies. This does not explain it. Discussion paragraph 4 cites external studies (references 44 and 47–50) in which higher DDIT4 lowers survival, but it presents them as disagreeing with this study ("Paradoxically, however…"), not as part of its own findings.
- Inverted coding or scale. No evidence of this. DDIT4 is scored "negative" or "positive" by immunohistochemistry (Figure 4A), the figure legends label the groups explicitly, and the text uses "loss of DDIT4" and "low expression" in agreement with the "negative" group.
- A different endpoint. This does not explain it. The only endpoint is overall survival, with cancer-related death as the event (Figure 4B legend), and the same endpoint is used throughout.
- A conversion or manuscript-copy artifact. Ruled out. The conflict is in words ("Lower … longer" against "high … increased"), not in symbols, signs or table cells. The same sentence, in the journal's copyedited wording, appears in the abstract deposited in PubMed and Europe PMC.
- A correction already issued. None was found (see Status).
- What remains. Every part of the paper except one abstract sentence agrees on the direction, and so do the Kaplan-Meier curves. The simplest reading is that the abstract sentence was worded backwards.
Status
- Version of record. The publisher's pages could not be reached by automated access: doi.org redirects to linkinghub.elsevier.com, which returned only a redirect stub, and sciencedirect.com and gastrojournal.org returned HTTP 403. The copyedited abstract deposited in PubMed (PMID 32956680, efetch XML) and in the Europe PMC core record reads "Lower DDIT4 expression correlated with longer survival of patients with gastric cancer." Its wording differs slightly from the manuscript ("patients with gastric cancer" rather than "gastric cancer patients"), which indicates it is the journal's edited version, so the reversed sentence very likely also appears in the version of record. PMC7857017 is the NIH author manuscript (NIHMS1661019). The publisher's full-text body could not be checked; the body quotes above come from the PMC author manuscript.
- Figures and supplements. I opened Figure 4 itself (nihms-1661019-f0004.jpg, from the PMC CDN) and read the group sizes and P values in panels B and C. The PMC page lists Supplementary Tables 1–3 (.docx), Supplementary Figures 1–6, Supplementary Figure Legends and Supplementary Materials and Methods. Supplementary Table 3, the clinicopathological table cited for the survival result, could not be downloaded: PMC served a download-challenge page, and Europe PMC reported the supplementary files as not open access. Figure 4B settles the direction without it.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI. PMC and Europe PMC list no notices, and the PubMed record has no CommentsCorrections entries. Crossref Labs (which includes Retraction Watch data) shows no updates. PubPeer could not be checked directly: its API and search page were blocked by a bot-protection challenge (HTTP 403). A web search for the title with "pubpeer" returned no PubPeer page.
- Sources. PMC article PMC7857017 (author manuscript NIHMS1661019) and its Figure 4 image; PubMed 32956680 (efetch XML); Europe PMC core record; Crossref REST API; Crossref Labs; PubPeer (attempted); web search.
Suggested correction
- Change the abstract's Results sentence so that it matches the body of the paper and Figure 4B, for example: "Higher DDIT4 expression correlated with longer survival of patients with gastric cancer." The equivalent "Lower DDIT4 expression correlated with shorter survival…" would also work.
- Ideally, add the effect size to the abstract and Results (for example a hazard ratio with a 95% CI from a Cox model, alongside the log-rank P = .021 for the 509 patients), so the size and direction of the association can be read without the figure.
- Ask PubMed and Europe PMC to update their abstract records when the correction is published, since those are the copies most readers see.
Recomputation script (python recompute.py.txt)
"""Check the direction of the DDIT4-survival association stated in PMC7857017
(Gastroenterology 2020, doi:10.1053/j.gastro.2020.09.016).
Run: python recompute.py.txt
Needs matplotlib. Prints every number used in writeup.md and writes fig1.svg and
fig1.png next to this file.
There is no estimate to recompute: the paper reports no hazard ratio, only Kaplan-Meier
curves with log-rank P values (Figure 4B-C). The check is therefore (1) the direction
stated in each place the paper describes the 509-patient gastric cancer survival result,
and (2) the group sizes and P values printed on Figure 4B-C, read from the PMC image
nihms-1661019-f0004.jpg (they are not in the text, so they are typed in below).
If the repository's src/show.py is available (run from /home/ubuntu/papers), every quote
from the paper's text is also confirmed verbatim against the PMC-archived text.
"""
import os
import subprocess
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Patch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (from the paper)
# direction: +1 = high DDIT4 -> longer survival ; -1 = high (i.e. not low) DDIT4 -> shorter survival
STATEMENTS = [
("Abstract, Results",
"Lower DDIT4 expression correlated with longer survival of gastric cancer patients.", -1),
("Introduction, final (5th) paragraph",
"Patients with low expression have poorer survival.", +1),
("Results, human gastric cancer TMA",
"We saw statistically significant correlation of high DDIT4 expression with increased "
"patient survival and more differentiated tumors, especially in patients who had received "
"chemotherapy", +1),
("Figure 4 title",
"In humans, loss of DDIT4 correlates with worse gastric cancer survival", +1),
("Discussion, paragraph 3",
"Our results showing DDIT4 expression correlating with less proliferation in precancerous "
"lesions and better survival of patients with gastric cancer", +1),
]
# Other quotes from the PMC text used in writeup.md (checked verbatim, no direction coded)
OTHER_QUOTES = [
("Abstract, Conclusions",
"During injury-induced metaplastic proliferation, failure of licensing mTORC1 reactivation "
"correlates with increased proliferation of cells harboring DNA damage, as well as increased "
"tumor formation and growth in mice and humans."),
("Results, human gastric cancer TMA (full passage)",
"We examined another annotated tissue microarray on which we had compiled 509 patients with "
"gastric cancer. We saw statistically significant correlation of high DDIT4 expression with "
"increased patient survival and more differentiated tumors, especially in patients who had "
"received chemotherapy, consistent with DDIT4 slowing proliferation in response to DNA damage "
"(Figure 4A–C; Table S3)."),
("Figure 4 legend, panel B",
"Overall survival determined via Kaplan-Meier curves separated by DDIT4 expression, with an "
"event defined as a cancer-related death."),
("Discussion, paragraph 3 (full sentence)",
"Our results showing DDIT4 expression correlating with less proliferation in precancerous "
"lesions and better survival of patients with gastric cancer confirm and extend on those of "
"a previous report43."),
("Discussion, paragraph 4",
"Paradoxically, however, studies across multiple organs have shown disparate results for "
"DDIT4 in tumorigenesis with some studies showing higher DDIT4 decreases cancer survival44, "
"47–50."),
]
# The copyedited abstract deposited in PubMed (PMID 32956680) and Europe PMC:
PUBMED_ABSTRACT = "Lower DDIT4 expression correlated with longer survival of patients with gastric cancer."
TMA_N_TEXT = 509 # Results: "compiled 509 patients with gastric cancer"
# Figure 4B (overall) and 4C (patients receiving chemotherapy), legend boxes on the image
FIG4 = {
"4B overall": {"negative": 309, "positive": 200, "P": 0.021},
"4C chemotherapy": {"negative": 263, "positive": 170, "P": 0.016},
}
# On both panels the DDIT4-positive (green) curve lies above the DDIT4-negative (blue) curve
# over most of follow-up, i.e. the data show high DDIT4 -> longer survival.
FIG4_DIRECTION = +1
def check_quotes():
show = os.path.join(os.getcwd(), "src", "show.py")
if not os.path.exists(show):
print("src/show.py not found in the working directory; quote check skipped")
return
py = "/home/ubuntu/papers-data/venv/bin/python"
py = py if os.path.exists(py) else "python3"
txt = subprocess.run([py, show, "PMC7857017", "--full"], capture_output=True,
text=True).stdout
quotes = [(loc, s) for loc, s, _ in STATEMENTS] + OTHER_QUOTES
for loc, s in quotes:
print(f" verbatim in PMC text: {s in txt!s:5} ({loc})")
print(f" PMC-text quotes found: {sum(s in txt for _, s in quotes)} of {len(quotes)}"
" (the PubMed wording and the Figure 4B-C counts are not in the PMC text)")
def main():
print("== Direction of the DDIT4-survival association, by location")
word = {+1: "high DDIT4 -> longer survival", -1: "low DDIT4 -> longer survival"}
for loc, _, d in STATEMENTS:
print(f" {loc:36s} {word[d]}")
print(f" {'Figure 4B-C Kaplan-Meier curves':36s} {word[FIG4_DIRECTION]}")
body = [d for _, _, d in STATEMENTS[1:]] + [FIG4_DIRECTION]
abstract = STATEMENTS[0][2]
print(f" body statements agreeing with each other: {sum(d == body[0] for d in body)} of {len(body)}")
print(f" abstract opposite to every body statement: {all(d == -abstract for d in body)}")
print(f" PubMed/Europe PMC abstract wording: {PUBMED_ABSTRACT!r}")
check_quotes()
print("\n== Figure 4B-C group sizes")
b, c = FIG4["4B overall"], FIG4["4C chemotherapy"]
nb = b["negative"] + b["positive"]
nc = c["negative"] + c["positive"]
print(f" 4B: {b['negative']} negative + {b['positive']} positive = {nb}"
f" (Results text: {TMA_N_TEXT}; match = {nb == TMA_N_TEXT}); log-rank P = {b['P']}")
print(f" 4C: {c['negative']} negative + {c['positive']} positive = {nc}"
f" = {100 * nc / nb:.1f}% of the 4B cohort; log-rank P = {c['P']}")
print(f" DDIT4-positive share: 4B {100 * b['positive'] / nb:.1f}%, 4C {100 * c['positive'] / nc:.1f}%")
make_figure(b)
# ------------------------------------------------------------------ figure
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
def box(ax, x, y, w, h, color):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
linewidth=2, edgecolor=color, facecolor="white"))
def make_figure(b):
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "svg.fonttype": "none"})
fig = plt.figure(figsize=(7.2, 5.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off"); ax.set_facecolor(BG)
# left panel: abstract
lx, lw, ly, lh = 0.03, 0.30, 0.36, 0.48
box(ax, lx, ly, lw, lh, ORANGE)
ax.text(lx + 0.02, ly + lh - 0.035, "Abstract (Results)", fontsize=11, weight="bold",
color=INK, va="top")
ax.text(lx + 0.02, ly + lh - 0.085, "PMC author manuscript", fontsize=10,
color=INK2, va="top")
q = textwrap.fill("“Lower DDIT4 expression correlated with longer survival of gastric "
"cancer patients.”", 24)
ax.text(lx + 0.02, ly + lh - 0.135, q, fontsize=10.5, color=INK, va="top", style="italic")
ax.text(lx + 0.02, ly + 0.11, "Same sentence, in\ncopyedited wording, is\nin the PubMed abstract", fontsize=10,
color=INK2, va="bottom")
ax.text(lx + 0.02, ly + 0.03, "Stated: LOW DDIT4 →\nlonger survival", fontsize=10.5,
weight="bold", color=INK, va="bottom")
# right panel: body + figure data
rx, rw, ry, rh = 0.44, 0.53, 0.17, 0.79
box(ax, rx, ry, rw, rh, BLUE)
ax.text(rx + 0.02, ry + rh - 0.03, "Body of the paper (509-patient TMA)", fontsize=11,
weight="bold", color=INK, va="top")
items = [
("Introduction", "“Patients with low expression have poorer survival.”"),
("Results", "“high DDIT4 expression with increased patient survival”"),
("Figure 4 title", "“loss of DDIT4 correlates with worse gastric cancer survival”"),
("Discussion", "“DDIT4 expression correlating with … better survival”"),
("Figure 4B data", f"DDIT4-positive curve (n={b['positive']}) above DDIT4-negative "
f"(n={b['negative']}); log-rank P = {str(b['P'])[1:]}"),
]
y = ry + rh - 0.095
for head, body in items:
ax.text(rx + 0.02, y, head, fontsize=10, weight="bold", color=INK2, va="top")
lines = textwrap.fill(body, 47)
ax.text(rx + 0.02, y - 0.036, lines, fontsize=10, color=INK, va="top")
y -= 0.036 + 0.031 * (lines.count("\n") + 1) + 0.025
ax.text(rx + 0.02, ry + 0.025, "Stated and shown: HIGH DDIT4 →\nlonger survival",
fontsize=10.5, weight="bold", color=INK, va="bottom")
# connector
arr = FancyArrowPatch((lx + lw + 0.005, ly + lh / 2), (rx - 0.005, ly + lh / 2),
arrowstyle="<->", mutation_scale=14, linewidth=2, color=INK2)
ax.add_patch(arr)
ax.text((lx + lw + rx) / 2, ly + lh / 2 + 0.03, "opposite\ndirection", ha="center",
va="bottom", fontsize=10, color=INK, weight="bold")
ax.text((lx + lw + rx) / 2, ly + lh / 2 - 0.03, "same\ncohort,\nsame\nendpoint", ha="center",
va="top", fontsize=10, color=INK2)
ax.legend(handles=[Patch(facecolor="white", edgecolor=ORANGE, linewidth=2,
label="As reported in the abstract"),
Patch(facecolor="white", edgecolor=BLUE, linewidth=2,
label="Paper's own body text and Figure 4B data")],
loc="lower left", bbox_to_anchor=(0.03, 0.02), frameon=False, fontsize=10,
ncol=1, handlelength=1.6)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nwrote fig1.svg, fig1.png")
if __name__ == "__main__":
main()
Abstract gives the genome length as 259.28 Mbp; the main text and deposited assembly give 1.27 Gbp
Genome Biol (2025) · doi:10.1186/s13059-025-03542-5 · PMC11959729
Summary
The abstract describes the chromosome-level genome of Ophiura sarsii vadicola as "259.28 Mbp in length with a scaffold N50 length of 66.91 Mbp". The main text describes the same assembly, with the same 66.91 Mbp N50, as 1.27 Gbp, which is 4.9 times larger. Every other size-related number in the paper agrees with 1.27 Gbp and not with 259.28 Mbp: the k-mer estimate (1.21 Gbp), the genome size implied by the stated sequencing coverage (1.255 Gbp), the chromosome lengths in Supplementary Table S3 (summing to 1,268,118,785 bp with the unplaced sequence), and the assembly deposited at NCBI (1,267,624,991 bp). The abstract figure is the headline description of the paper's main genomic resource, so readers who see only the abstract or an indexed summary get a genome size about one fifth of the real one; none of the regeneration findings depend on it.
What the paper reports
Abstract, Results:
Here, we report a chromosome-level genome of Ophiura sarsii vadicola, 259.28 Mbp in length with a scaffold N50 length of 66.91 Mbp.
Main text, subsection "Genome assembly and characterization", first paragraph (the first subsection of the journal's Results; in the PMC copy it follows directly under the Background heading, see Status):
A total of 143.7 Gbp (114.47×) of PacBio SMRT long reads and 218.32 Gbp (173.92×) of Illumina short reads were generated for genome assembly (Additional file 2: Table S1). K-mer analysis estimated the size of O. sarsii vadicola genome to be 1.21 Gbp with a heterozygosity rate of 2.21% (K-mer = 17, Additional file 1: Fig. S1). Using a hybrid assembly approach with 362.02 Gbp of sequence data, we assembled a 1.27 Gbp genome, consisting of 1988 contigs with an N50 of 2.44 Mbp and GC content of 37.2% (Additional file 2: Table S2). After HI-C scaffolding, 99.62% of the assembled contigs were anchored into 19 chromosomes, yielding a final genome size of 1.27 Gbp across 216 scaffolds with an N50 of 66.91 Mbp (Fig. 1A, Additional file 1: Fig. S2 and Additional file 2: Tables S3–S4).
Methods, "Genome sequencing and assembly":
PacBio sequencing generated 143.7 Gbp of data with 114.47× coverage and an N50 length of 20,943 bp.
Contigs were polished with Quiver and aligned to 218.32 Gbp of Illumina data (173.92×) using the BWA v0.6 [81].
Discussion, "Chromosome-level genome features in the phylum Echinodermata":
Our genome showed higher completeness (94.0%) and comparable assembly quality, with a scaffold N50 of 66.91 Mbp, relative to O. sarsii (93.5%, 78.3 Mbp) and A. filiformis (92.7%, 68.8 Mbp) (Additional file 2: Table S4).
Back matter, "Data availability":
The data of genome sequencing and the genome assembly data of Ophiura sarsii vadicola in this study have been deposited at NCBI under the BioProject PRJNA732836 [119].
Additional file 2 (supplementary tables), as downloaded from the publisher: Table S2 gives a total scaffold length of 1,268,118,785 bp (216 scaffolds, scaffold N50 66,915,169 bp, L50 8); Table S3 lists 19 chromosome lengths totalling 1,263,260,924 bp (99.62%) plus 4,857,861 bp unplaced; Table S4 gives the size of this genome as 1,268.11 Mb.
What the paper's own numbers show
Method. I put every genome-size figure the paper gives, and every figure that can be derived from its inputs, on the same scale. Data volume divided by stated fold-coverage gives the genome size the authors used for coverage. The 19 chromosome lengths in Table S3 give the assembly size directly and let N50 and L50 be recomputed. The N50 also constrains the total: half the assembly must lie in scaffolds at least 66.91 Mbp long. Finally I compared these with the assembly deposited under the cited BioProject (NCBI GCA_043643665.1). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Genome length | Abstract | 259.28 Mbp | matches no other figure; 0.2045 of the Table S2 total and of the NCBI total |
| Contig assembly size | Main text, Genome assembly subsection | 1.27 Gbp, 1988 contigs | Table S2 contig total 1,267,941,585 bp |
| Final assembly size | Main text, Genome assembly subsection | 1.27 Gbp, 216 scaffolds, N50 66.91 Mbp | Table S3: 1,263,260,924 + 4,857,861 = 1,268,118,785 bp (equals Table S2) |
| Ratio, body to abstract | Abstract vs main text | 1.27 Gbp vs 259.28 Mbp | 4.898 (4.891 using Table S2) |
| Genome size behind PacBio coverage | Genome assembly subsection; Methods; Table S1 | 143.7 Gbp at 114.47× | 1.2554 Gbp; at 259.28 Mbp the coverage would be 554.2× |
| Genome size behind Illumina coverage | Genome assembly subsection; Methods; Table S1 | 218.32 Gbp at 173.92× | 1.2553 Gbp; at 259.28 Mbp the coverage would be 842.0× |
| Total sequence data | Genome assembly subsection | 362.02 Gbp | 143.7 + 218.32 = 362.02 Gbp |
| K-mer genome size | Genome assembly subsection | 1.21 Gbp | 4.67 times the abstract figure |
| Mean chromosome length | Genome assembly subsection (19 chromosomes, 99.62% anchored) | — | 66.84 Mbp at 1.27 Gbp (19 × N50 = 1271.29 Mbp); 13.65 Mbp at 259.28 Mbp |
| Scaffolds of at least N50 length | Abstract; Table S3 | N50 66.91 Mbp | Table S3: 8 chromosomes ≥ 66,915,169 bp and L50 = 8, as in Table S2; these 8 sum to 662.37 Mbp, 2.55 times 259.28 Mbp; a 259.28 Mbp assembly could hold at most 3 such scaffolds |
| Largest chromosome | Table S3 | 147,934,811 bp | 0.571 of the abstract's total genome length |
| Deposited assembly | Data availability (PRJNA732836) | — | GCA_043643665.1: 1,267,624,991 bp, scaffold N50 66,915,169 bp, L50 8, 19 chromosomes |
| Scaffold N50 | Abstract | 66.91 Mbp | consistent with the main text, Table S2 and NCBI (66.915 Mbp) |
The figures agree among themselves to within 5% (1.21 to 1.27 Gbp), and the abstract's 259.28 Mbp is the only exception. Taken with Table S3, the abstract figure cannot be right. The largest chromosome alone (147.9 Mbp) is more than half of 259.28 Mbp (129.6 Mbp), so an assembly of that size containing it would have a scaffold N50 of at least 147.9 Mbp, not the 66.91 Mbp printed next to it; and the 8 chromosomes of at least 66.91 Mbp already sum to 662.37 Mbp, 2.55 times 259.28 Mbp. The 19 chromosomes listed in Table S3 sum to 1.263 Gbp. The deposited assembly differs from Table S2 by 0.039% in total length (and has 191 rather than 216 scaffolds), which does not bear on the discrepancy.
Figure
Figure 1. The two conflicting sentences, quoted verbatim (the main-text excerpt is abridged at the start and end). The abstract (left) gives the length of the chromosome-level genome as 259.28 Mbp; the main text's Genome assembly subsection (right) gives the final assembly as 1.27 Gbp. Both state a scaffold N50 of 66.91 Mbp, so they describe the same assembly. The supplementary scaffold total and the deposited NCBI assembly, shown underneath for reference, agree with the main text.
Figure 2. Every genome-size figure in the article and Additional file 2 (orange circles), and the sizes recomputed from the paper's own inputs or read from the deposited assembly (blue diamonds), on one linear axis starting at zero. The coverage-based values are data volume divided by stated fold-coverage; the Table S3 value is the sum of the 19 chromosome lengths plus the unplaced sequence. All values except the abstract's lie between 1,210 and 1,270 Mbp.
Why it matters
The genome assembly is one of the paper's two main contributions (the title leads with it, and the abstract's Conclusions call it "Our genomic resource"), and its length is the first number in the abstract's Results. The abstract understates it by a factor of 4.9. Abstracts are what indexing services, search results and many citing papers reproduce, so the 259.28 Mbp figure is the one most likely to be repeated, for instance in comparisons of echinoderm genome sizes, where it would make O. sarsii vadicola smaller than every echinoderm genome in Table S4 (the smallest listed is 383.69 Mb) rather than placing it with the other four ophiuroids listed there (1,184.53 to 2,764.32 Mb). It could also mislead anyone planning sequencing depth or resource use from the abstract.
The error does not touch the analyses. The full text, the supplementary tables and the deposited assembly all have the correct size; the assembly quality metrics (N50, BUSCO completeness) are stated correctly; and none of the regeneration results (cell clusters, connective-tissue-derived progenitors, pseudotime ordering of muscle differentiation) depend on the genome length. A reader of the full text would notice the conflict.
Explanations considered
- A different object (contig set, subset, haplotype or an earlier stage). Does not fit. The abstract pairs 259.28 Mbp with the scaffold N50 of 66.91 Mbp, which the main text gives for the final 1.27 Gbp, 19-chromosome assembly, and the contig assembly is also 1.27 Gbp.
- A unit or decimal slip (Mbp vs Gbp). Does not fit. The ratio is 4.898, not a power of ten.
- A stated fraction of the assembly. Does not fit. 259.28 Mbp is 20.45% of the assembly. The non-repeat part would be about 574 Mbp (45.2%), and the unplaced sequence is 4.86 Mbp (0.38%). No chromosome length in Table S3 equals it (they range from 44.6 to 147.9 Mbp).
- The genome-size estimate rather than the assembly length. Does not fit. The k-mer estimate is 1.21 Gbp, and the coverage figures imply 1.255 Gbp.
- A conversion or typesetting artifact in the archived copy. Does not fit. The PMC page and the publisher-supplied JATS XML in Europe PMC both have "259.28 Mbp" as plain text with an ordinary decimal point. A rendering problem would not produce a value 4.9 times smaller.
- A size that the N50 permits. Does not fit. A 259.28 Mbp assembly with an N50 of 66.91 Mbp could contain at most 3 scaffolds of N50 length, which is incompatible with Table S3, which lists 8 chromosomes of at least that length (together 662.37 Mbp).
- A later correction. None found (see Status).
None of these accounts for the number. The body, supplement and deposited data agree with each other, so the abstract value is the one in error.
Status
As of 2026-10-24:
- Version of record. The publisher's page (genomebiology.biomedcentral.com, redirecting to link.springer.com) and its PDF could not be read; both returned a bot challenge or a login redirect. The wording was checked in the PMC article page and in the publisher-supplied JATS XML from Europe PMC, and both contain "259.28 Mbp" in the abstract and "1.27 Gbp" twice in the "Genome assembly and characterization" subsection. The local PMC archive matches. In these archived copies the only "Results" heading is in the structured abstract; the body's results subsections, including "Genome assembly and characterization", follow directly under the "Background" heading. The publisher's version may carry a Results heading there, so this write-up cites the subsection by name. Every quotation above was checked against the PMC archive; the Data availability sentence was checked in the full-text back matter.
- Supplements. Additional file 2 (13059_2025_3542_MOESM2_ESM.docx, Tables S1–S22) was downloaded from static-content.springer.com and parsed; Tables S1–S4 support 1.27 Gbp (values above). Additional file 1 (figures) and the FigShare dataset (10.6084/m9.figshare.28468406) were not needed for the size and were not examined further.
- Deposited data. NCBI Datasets record for GCA_043643665.1 (BioProject PRJNA732836, released 2024-10-24): total length 1,267,624,991 bp, 19 chromosomes, 191 scaffolds, scaffold N50 66,915,169 bp, L50 8.
- Corrections. No correction, erratum, expression of concern or retraction in Crossref (no update-to, updated-by or relation entries, which also covers the Retraction Watch data Crossref carries), PMC or Europe PMC.
- PubPeer. The PubPeer search and API returned a human-verification wall and could not be queried directly; a web search for the title and PubPeer returned no PubPeer entry.
- Side note, outside this claim. The Genome assembly subsection gives the contig N50 as "2.44 Mbp", whereas Table S2 and NCBI give 2,259,232 bp (2.26 Mbp).
Sources: PMC article; DOI; NCBI GCA_043643665.1; BioProject PRJNA732836; Crossref record; Additional file 2 from the publisher's supplementary material.
Suggested correction
In the abstract, replace "259.28 Mbp in length" with "1.27 Gbp in length", so that the sentence reads: "Here, we report a chromosome-level genome of Ophiura sarsii vadicola, 1.27 Gbp in length with a scaffold N50 length of 66.91 Mbp." The authors may also wish to check the contig N50 in the Genome assembly subsection ("2.44 Mbp") against Table S2 (2,259,232 bp).
Recomputation script (python recompute.py.txt)
# Recompute every number in writeup.md for PMC11959729 and redraw the figures.
# Run: python recompute.py.txt (numpy + matplotlib). Figures are written next to this file.
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---- Values printed in the article (PMC archive) ----
abstract_len_mbp = 259.28 # Abstract: "259.28 Mbp in length"
abstract_n50_mbp = 66.91 # Abstract: "scaffold N50 length of 66.91 Mbp"
body_contig_gbp = 1.27 # Results: "we assembled a 1.27 Gbp genome, consisting of 1988 contigs"
body_final_gbp = 1.27 # Results: "final genome size of 1.27 Gbp across 216 scaffolds"
body_n50_mbp = 66.91 # Results: "with an N50 of 66.91 Mbp"
kmer_gbp = 1.21 # Results: K-mer estimate
pacbio_gbp, pacbio_x = 143.7, 114.47
illum_gbp, illum_x = 218.32, 173.92
total_data_gbp = 362.02
n_chrom = 19
anchored_pct = 99.62
repeat_pct = 54.8
# ---- Additional file 2 (MOESM2_ESM.docx) ----
s2_contig_total = 1_267_941_585
s2_scaffold_total = 1_268_118_785
s2_scaffold_n50 = 66_915_169
s2_scaffold_l50 = 8
s2_contig_n50 = 2_259_232
s3_chrom = [147_934_811, 92_748_282, 80_758_857, 70_437_490, 68_898_335, 67_708_752,
66_966_678, 64_701_964, 52_984_193, 56_980_601, 54_509_896, 52_689_219,
52_573_618, 49_102_080, 58_363_436, 62_702_253, 66_915_169, 51_704_667,
44_580_623]
s3_placed = 1_263_260_924
s3_unplaced = 4_857_861
s4_size_mb = 1268.11
# ---- NCBI Datasets, GCA_043643665.1 (BioProject PRJNA732836) ----
ncbi_total = 1_267_624_991
ncbi_scaffold_n50 = 66_915_169
ncbi_scaffold_l50 = 8
ncbi_contig_n50 = 2_259_232
def p(label, val):
print(f"{label}: {val}")
body_mbp = body_final_gbp * 1000
p("Ratio body/abstract (1270 / 259.28)", round(body_mbp / abstract_len_mbp, 3))
p("Ratio Table S2 scaffold total / abstract", round(s2_scaffold_total / 1e6 / abstract_len_mbp, 3))
p("Abstract as fraction of Table S2 total", round(abstract_len_mbp / (s2_scaffold_total / 1e6), 4))
p("Abstract as fraction of NCBI total", round(abstract_len_mbp / (ncbi_total / 1e6), 4))
p("Genome size implied by PacBio coverage (Gbp)", round(pacbio_gbp / pacbio_x, 4))
p("Genome size implied by Illumina coverage (Gbp)", round(illum_gbp / illum_x, 4))
p("PacBio + Illumina (Gbp), reported 362.02", round(pacbio_gbp + illum_gbp, 2))
p("PacBio coverage if genome were 259.28 Mbp (x)", round(pacbio_gbp * 1000 / abstract_len_mbp, 1))
p("Illumina coverage if genome were 259.28 Mbp (x)", round(illum_gbp * 1000 / abstract_len_mbp, 1))
p("K-mer estimate / abstract", round(kmer_gbp * 1000 / abstract_len_mbp, 2))
p("Mean chromosome length at 1.27 Gbp (Mbp)", round(body_mbp / n_chrom, 2))
p("Mean chromosome length at 259.28 Mbp (Mbp)", round(abstract_len_mbp / n_chrom, 2))
p("19 x N50 (Mbp)", round(n_chrom * body_n50_mbp, 2))
p("Max number of scaffolds >= N50 that fit in 259.28 Mbp", int(abstract_len_mbp // abstract_n50_mbp))
p("Sum of Table S3 chromosome lengths (bp)", sum(s3_chrom))
p(" equals Table S3 'placed' total", sum(s3_chrom) == s3_placed)
p("Placed + unplaced (bp), Table S2 scaffold total 1,268,118,785", s3_placed + s3_unplaced)
p("Placed share (%)", round(100 * s3_placed / (s3_placed + s3_unplaced), 2))
p("Largest chromosome (Mbp)", round(max(s3_chrom) / 1e6, 2))
p("Largest chromosome / 259.28 Mbp", round(max(s3_chrom) / 1e6 / abstract_len_mbp, 3))
p("Chromosomes >= N50 (Table S3)", sum(c >= s2_scaffold_n50 for c in s3_chrom))
big = [c for c in s3_chrom if c >= s2_scaffold_n50]
p("Sum of chromosomes >= N50 (Mbp)", round(sum(big) / 1e6, 2))
p(" as a multiple of 259.28 Mbp", round(sum(big) / 1e6 / abstract_len_mbp, 2))
p("Largest chromosome > half of 259.28 Mbp, so N50 at that total would be >= it", max(s3_chrom) / 1e6 > abstract_len_mbp / 2)
srt = sorted(s3_chrom, reverse=True)
cum = 0
for i, c in enumerate(srt, 1):
cum += c
if cum >= (s3_placed + s3_unplaced) / 2:
p("L50 recomputed from Table S3 (scaffold count, chromosome reaching half)", f"{i} ({c} bp)")
break
p("NCBI total (Gbp)", round(ncbi_total / 1e9, 4))
p("NCBI vs Table S2 scaffold total, difference (%)", round(100 * (s2_scaffold_total - ncbi_total) / s2_scaffold_total, 3))
p("NCBI scaffold N50 (Mbp)", round(ncbi_scaffold_n50 / 1e6, 3))
p("Non-repeat part of assembly at 1.27 Gbp (Mbp)", round(body_mbp * (1 - repeat_pct / 100), 1))
p("Unplaced part (Mbp)", round(s3_unplaced / 1e6, 2))
s4_other_sizes = {"Ophiura sarii": 1592.18, "Amphiura filiformis": 1570.00, "Ophionereis fasciata": 1184.53,
"Ophiothrix spiculata": 2764.32, "Acanthaster planci": 383.69, "Pisaster ochraceus": 401.94,
"Asterias rubens": 417.60, "Strogylocentrotus purpuratus": 921.86, "Lytechinus variegatus": 870.40,
"Holothuria leucospilota": 1390.00, "Apostichopus japonicus": 804.62, "Anneissia japonica": 589.63}
p("Smallest other genome in Table S4 (Mb)", min(s4_other_sizes.values()))
oph = [s4_other_sizes[k] for k in ("Ophiura sarii", "Amphiura filiformis", "Ophionereis fasciata", "Ophiothrix spiculata")]
p("Other ophiuroids in Table S4, range (Mb)", (min(oph), max(oph)))
p("Smallest and largest chromosome in Table S3 (Mbp)", (round(min(s3_chrom) / 1e6, 1), round(max(s3_chrom) / 1e6, 1)))
p("Half of a 259.28 Mbp assembly (Mbp)", round(abstract_len_mbp / 2, 1))
p("Side note: body contig N50 2.44 Mbp vs Table S2/NCBI contig N50 (Mbp)", round(s2_contig_n50 / 1e6, 2))
# ---------------- Figures ----------------
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: two text panels joined by an annotated connector
fig = plt.figure(figsize=(7.2, 4.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 60); ax.axis("off")
def panel(x, w, title, lines):
ax.add_patch(FancyBboxPatch((x, 18), w, 34, boxstyle="round,pad=0,rounding_size=1.5",
fc="white", ec="#cfcdc8", lw=1.2))
ax.text(x + 2, 48, title, fontsize=11, weight="bold", va="center", color=INK)
y = 42.5
for txt, col, wt in lines:
ax.text(x + 2, y, txt, fontsize=10 if wt == "normal" else 11, va="center", color=col, weight=wt)
y -= 4.6
panel(1.5, 40, "Abstract (Results)", [
("\u201cHere, we report a chromosome-level", INK2, "normal"),
("genome of Ophiura sarsii vadicola,", INK2, "normal"),
("259.28 Mbp in length", ORANGE, "bold"),
("with a scaffold N50 length", INK2, "normal"),
("of 66.91 Mbp.\u201d", INK2, "normal"),
])
panel(58.5, 40, "Main text, Genome assembly", [
("\u201c...anchored into 19 chromosomes,", INK2, "normal"),
("yielding a final genome size of", INK2, "normal"),
("1.27 Gbp", ORANGE, "bold"),
("across 216 scaffolds with an N50", INK2, "normal"),
("of 66.91 Mbp ...\u201d", INK2, "normal"),
])
ratio = body_mbp / abstract_len_mbp
yh = 42.5 - 2 * 4.6
ax.add_patch(FancyArrowPatch((32.5, yh), (59.5, yh), arrowstyle="<->", mutation_scale=14, lw=2, color=INK))
ax.text(50, yh + 1.8, f"{ratio:.1f}-fold", ha="center", va="bottom", fontsize=11, weight="bold", color=INK)
ax.text(50, yh - 1.8, "apart", ha="center", va="top", fontsize=10.5, weight="bold", color=INK)
ax.add_patch(FancyArrowPatch((14, 22.5), (72, 22.5), connectionstyle="arc3,rad=0.25",
arrowstyle="<->", mutation_scale=12, lw=1.6, color=INK2, ls=(0, (3, 2))))
ax.text(50, 13.3, "same assembly: identical scaffold N50 (66.91 Mbp)", ha="center", va="center",
fontsize=10, color=INK2, bbox=dict(fc=BG, ec="none", pad=1.0))
ax.text(50, 7.0, f"For reference: Supplementary Table S2, scaffold total {s2_scaffold_total:,} bp",
ha="center", va="center", fontsize=10, color=INK)
ax.text(50, 3.0, f"Deposited assembly NCBI GCA_043643665.1, total length {ncbi_total:,} bp",
ha="center", va="center", fontsize=10, color=INK)
ax.text(50, 56.5, "Orange: the two conflicting values, as printed in the article",
ha="center", va="center", fontsize=10, color=INK2)
save(fig, "fig1")
# Figure 2: every genome-size figure the paper and its data give, on one linear axis from 0
rows = [
("Abstract", abstract_len_mbp, "reported"),
("Main text: contig assembly", body_contig_gbp * 1000, "reported"),
("Main text: final assembly", body_final_gbp * 1000, "reported"),
("Main text: K-mer estimate", kmer_gbp * 1000, "reported"),
("Table S4: size", s4_size_mb, "reported"),
("PacBio data / coverage (143.7 Gbp / 114.47x)", pacbio_gbp / pacbio_x * 1000, "recomputed"),
("Illumina data / coverage (218.32 Gbp / 173.92x)", illum_gbp / illum_x * 1000, "recomputed"),
("Sum of 19 chromosomes + unplaced (Table S3)", (s3_placed + s3_unplaced) / 1e6, "recomputed"),
("NCBI GCA_043643665.1 total length", ncbi_total / 1e6, "recomputed"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.1), facecolor=BG)
ax.set_facecolor(BG)
ys = list(range(len(rows)))[::-1]
for (lab, v, kind), y in zip(rows, ys):
col = ORANGE if kind == "reported" else BLUE
mk = "o" if kind == "reported" else "D"
ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1, solid_capstyle="round")
ax.plot(v, y, mk, ms=9, color=col, mec="white", mew=1.5, zorder=3)
ax.text(v + 25, y, f"{v:,.1f}" if v < 1000 or kind == "recomputed" else f"{v:,.0f}",
va="center", fontsize=10, color=INK)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 1500)
ax.set_xlabel("Genome size (Mbp)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#b9b7b1")
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o", ms=9, color=ORANGE, label="printed in the paper or its supplement")
ax.plot([], [], "D", ms=9, color=BLUE, label="recomputed from the paper's inputs / deposited data")
ax.legend(loc="lower center", bbox_to_anchor=(0.32, 1.0), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("figures written to", HERE)
Abstract reverses the omnivore-versus-plant-diet direction for Prevotella, Roseburia and Bacteroides given in the Results
Gut (2021) · doi:10.1136/gutjnl-2021-325168 · PMC9185830
Summary
The abstract states that omnivores had higher Prevotella and Roseburia and lower Bacteroides abundances than vegans and vegetarians. Every other place in the paper that gives these directions gives the opposite one. The Results and Discussion say that P. copri and Roseburia sp. CAG 182 were more abundant in vegetarians and vegans and that B. dorei prevailed in omnivores (Wilcoxon p<0.05, Figure 3A), and Figures 3A and 3C show the same thing: 0 of 13 direction statements outside the abstract agree with it. The abstract is the part of the paper most readers and indexing services see, and it reports the diet–microbiome contrast the wrong way round; the analysis and the paper's other headline results are not affected.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py (python recompute.py.txt --verify repeats the check), and against the publisher's typeset PDF (Gut 2022;71:1302–1314).
Abstract, Results paragraph:
In omnivores, higher Prevotella and Roseburia and lower Bacteroides abundances than in vegans and vegetarians were observed.
Abstract, Design paragraph (the population):
dietary recording from 120 healthy volunteers, equally distributed for the different diets and matched for sex and age
Results, section "Gut microbiome composition and functions and relation to faecal miRNAs", first paragraph:
Prevotella copri and Roseburia sp. CAG 182 showed higher abundance in VT/VN compared with O, while Bacteroides dorei (currently Phocaeicola dorei) prevailed in O (p<0.05; figure 3A).
Figure 3 legend, panels A and C:
(A) Box plots showing the relative abundance of three bacterial species differentially prevalent among the dietary groups. *Wilcoxon rank-sum p<0.05.
The colour indicates the dietary group in which the variable has the maximum level.
Discussion, paragraph on the gut microbiome:
the gut microbiome taxonomic composition varied according to the dietary patterns, with P. copri and Roseburia sp. CAG 182, having a higher abundance in VT/VN than O.
Discussion, paragraph on miR-425-3p:
Higher stool levels of miR-425-3p and miR-638 were also significantly correlated with Roseburia and R. bacterium D5, which in our cohort were significantly more abundant in VT/VN
Results, section "Integration of miRNAs, microbiome and diet" (the miRNA–taxon chain that depends on the direction):
Two DEmiRNAs, miR-425-3p and miR-638, attracted our interest since they were associated with a coherent higher expression in VT/VN compared with both the analysed O groups
Consistently, higher levels of Roseburia sp. CAG 182 and R. bacterium D5, as well as a lower abundance of A. colihominis were found in subjects within the highest expression levels of miR-638.
(O, omnivores; VT, vegetarians; VN, vegans.)
What the paper's own numbers show
Method. The paper prints no abundance values, effect sizes or exact p-values for these taxa, only the direction and "p<0.05", so there is nothing numeric to recompute. I instead coded every statement or display in the paper that says which diet group has more of a Prevotella, Roseburia or Bacteroides taxon as "higher in O" or "lower in O" and compared it with the abstract. The text statements come from the Results and Discussion. The figure entries were read from the archived Figure 3 image (Europe PMC, gutjnl-2021-325168f03.jpg). In panel A I used the box positions and the significance marks; in panel C I used the bar colour, which by the legend marks the group with the highest level (red O, green VN). recompute.py.txt holds the ledger and prints every count below.
| Quantity | Location | Reported (abstract) | What the rest of the paper shows |
|---|---|---|---|
| Prevotella direction | Results p1 (P. copri); Discussion; Fig. 3A | higher in O | lower in O: higher in VT/VN, O vs VT and O vs VN marked p<0.05 (0 of 3 agree) |
| Roseburia direction | Results p1 and Discussion (sp. CAG 182); Discussion (genus); Fig. 3A (sp. CAG 182); Fig. 3C (sp. CAG 182, R. hominis) | higher in O | lower in O: highest in VN, O vs VN marked p<0.05; both Fig. 3C bars coloured VN (0 of 6 agree) |
| Bacteroides direction | Results p1 (B. dorei); Fig. 3A (B. dorei); Fig. 3C (B. finegoldii, B. massiliensis) | lower in O | higher in O: B. dorei "prevailed in O", O vs VN marked p<0.05; both Fig. 3C bars coloured O (0 of 4 agree) |
| All direction statements | text (6) and figures (7) | — | 0 of 13 agree with the abstract |
| Indirect check | Results, "Integration..." | — | miR-425-3p and miR-638 are higher in VT/VN, and high levels go with more Roseburia sp. CAG 182; this fits Roseburia higher in VT/VN, not in O |
| Correction or erratum | Crossref, PMC, Europe PMC, PubMed | — | none found |
The thirteen entries are not thirteen independent analyses. The Results sentence, the Discussion and Figure 3A all describe the same Wilcoxon comparison, and Figure 3C is a separate (DIABLO) view of the same subjects. The point of the count is that the body of the paper is internally consistent and the abstract sentence is the only exception.
Figure
Figure 1. Top: the abstract sentence (orange) and the Results sentence it summarises (blue), both verbatim; they describe the same comparison of omnivores with vegetarians and vegans among the 120 subjects. Bottom: the direction for each genus as the abstract gives it (orange) and as the Results, Discussion and Figure 3A/3C give it (blue). The right-hand column counts how many of the paper's own statements and displays agree with the abstract for each genus (0 of 3, 0 of 6, 0 of 4; 0 of 13 in total, from recompute.py.txt). The figure shows no data values, because the paper reports none for these taxa.
Why it matters
- The abstract states the opposite of the paper's result. For all three genera the direction in the abstract is the reverse of what the Results text, the Discussion and Figure 3 report. Readers who see only the abstract, for example in PubMed or in literature databases that index abstracts, receive the reversed finding.
- It also clashes with the next sentence of the abstract. The abstract goes on to say that miR-425-3p and miR-638 were associated with "increased abundances of microbial species, such as Roseburia sp. CAG 182 ... specific of different diets". In the body those miRNAs are higher in VT/VN and track higher Roseburia sp. CAG 182, which only fits Roseburia being higher in VT/VN. Read together, the two abstract sentences imply a link between omnivory and Roseburia that the paper's data contradict.
- The body's direction matches prior literature; the abstract's does not. The paper's own data (higher Prevotella with plant-based diets, higher Bacteroides with an omnivorous diet) are in line with earlier studies linking these taxa to fibre-rich and animal-rich diets. The abstract's version runs against that literature, so a reader could take it as a new and contrary finding.
- The analysis and the main conclusions are not affected. The error sits in one abstract sentence. The 49 differentially expressed miRNAs, the miR-636 and miR-4739 associations with diet duration, the nutrient correlations, the integrated model (AUC 0.89) and the Conclusion do not depend on it. The diet–microbiome contrast is a secondary result of a paper about stool miRNAs.
Explanations considered
- Genus in the abstract, species in the body. Does not explain it. The only taxa of these genera reported as differing by diet are P. copri, Roseburia sp. CAG 182 and B. dorei (Figure 3A shows exactly these three), and the abstract's genus names summarise them. The Discussion itself uses the genus name in the body's direction ("Roseburia and R. bacterium D5, which in our cohort were significantly more abundant in VT/VN"). The other Roseburia and Bacteroides species shown in Figure 3C (R. hominis, B. finegoldii, B. massiliensis) also go the body's way.
- A different comparison or cohort. Does not explain it. The abstract compares omnivores with vegans and vegetarians, which is the O vs VT/VN comparison of Figure 3A in the same 120 subjects. The independent validation cohort of 45 omnivores contributed stool miRNA data only, and no other microbiome comparison by diet is reported.
- The miRNA-quartile analyses. Do not apply. Figure 3F and the related text group subjects by miR-425-3p or miR-638 level "regardless of the type of diet", not by diet, so they cannot be the source of a statement about omnivores versus vegans and vegetarians.
- A reference-group convention (e.g. "lower in VT/VN" phrased from the other side). Not supported. No such convention is stated in the Methods or figure legends, and the abstract sentence names its reference group explicitly ("In omnivores, higher ... than in vegans and vegetarians").
- The body, not the abstract, is wrong. Not supported. The Results text, two Discussion passages, the Figure 3A box plots with their significance marks and the Figure 3C bar colours all agree with one another, and the miRNA–Roseburia chain in the Results is coherent only with the body's direction. The abstract sentence is the only place with the reversed direction.
- A text-conversion artifact. Ruled out. The discrepancy is in words ("higher", "lower"), not in signs, superscripts or table cells, and the same sentence appears in the PMC archive, the PubMed record and the publisher's typeset PDF.
- A correction already issued. None found (see Status).
Status
- Version of record. doi.org resolves to gut.bmj.com, which returned HTTP 403 to automated requests. I checked the publisher's typeset PDF (Gut 2022;71:1302–1314, 13 pages, as deposited in an institutional repository, iris.unito.it), the PMC archive, the live PMC page, the Europe PMC full-text XML and the PubMed record (PMID 34315772). All carry the abstract sentence and the Results and Discussion passages exactly as quoted. The article was published online in 2021 and in print in July 2022 (volume 71, issue 7).
- Supplements. The Europe PMC supplementary bundle holds five supplement PDFs and the four main figures.
supp001.pdf(supplementary methods and legends, 11 pages) does not mention Prevotella, Roseburia, Bacteroides or Phocaeicola;supp002tosupp005are Supplementary Figures 1 to 4 as single images; by their legends they show food and nutrient intake, miRNA analyses and (Supplementary Figure 4) taxon abundances grouped by miRNA level, not by diet. No abundance table for these taxa by diet is supplied. The main Figure 3 image was examined and supports the body's direction (panels A and C). - Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices; the Crossref record has no update-to or updated-by entries. The PubMed record lists only two "CommentIn" entries (a Gut commentary, PMID 34479951, and a Hepatobiliary Surgery and Nutrition commentary, PMID 35464286), neither a correction. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title with "PubPeer" found no thread. - Sources. PMC article PMC9185830; Europe PMC full-text XML and supplementary files; typeset PDF of Gut 2022;71:1302–1314; PubMed 34315772; Crossref; PubPeer; https://doi.org/10.1136/gutjnl-2021-325168.
Suggested correction
In the Results paragraph of the abstract, replace
In omnivores, higher Prevotella and Roseburia and lower Bacteroides abundances than in vegans and vegetarians were observed.
with
In omnivores, lower Prevotella and Roseburia and higher Bacteroides abundances than in vegans and vegetarians were observed.
or, to match the species-level wording of the Results, "In vegans and vegetarians, higher Prevotella copri and Roseburia sp. CAG 182 and lower Bacteroides dorei abundances than in omnivores were observed." No change to the Results, Discussion, figures or conclusions is needed.
Recomputation script (python recompute.py.txt)
"""Direction check for the diet-microbiome sentence of PMC9185830
(Gut 2021, published in print as Gut 2022;71:1302-1314; doi:10.1136/gutjnl-2021-325168).
Run: python recompute.py.txt prints the direction ledger and writes fig1.svg / fig1.png
python recompute.py.txt --verify also checks every quotation below, verbatim, against the
PMC-archived text (needs this repository's src/show.py)
Needs matplotlib only (numpy is not used).
There is nothing numeric to recompute: the paper prints no abundance values, effect sizes or
p-values for these taxa beyond "p<0.05". The check is a direction ledger. Each place in the
paper that states, or shows, which diet group has more of a taxon is coded
+1 = higher in omnivores (O) than in vegetarians/vegans (VT/VN)
-1 = lower in O than in VT/VN
and compared with the abstract sentence.
Figure 3 entries were read by eye from the archived image gutjnl-2021-325168f03.jpg (Europe PMC
supplementary bundle). Panel A: box plots per diet, with "*Wilcoxon rank-sum p<0.05" marks.
Panel C: DIABLO loading bars, "The colour indicates the dietary group in which the variable has
the maximum level" (legend); red = O, green = VN.
"""
import os
import subprocess
import sys
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC9185830"
# ---------------------------------------------------------------- quotations (verbatim)
Q_ABSTRACT = ("In omnivores, higher Prevotella and Roseburia and lower Bacteroides abundances "
"than in vegans and vegetarians were observed.")
Q_RESULTS = ("Prevotella copri and Roseburia sp. CAG 182 showed higher abundance in VT/VN "
"compared with O, while Bacteroides dorei (currently Phocaeicola dorei) prevailed "
"in O (p<0.05; figure 3A).")
Q_DISC1 = ("the gut microbiome taxonomic composition varied according to the dietary patterns, "
"with P. copri and Roseburia sp. CAG 182, having a higher abundance in VT/VN than O.")
Q_DISC2 = ("Higher stool levels of miR-425-3p and miR-638 were also significantly correlated "
"with Roseburia and R. bacterium D5, which in our cohort were significantly more "
"abundant in VT/VN")
Q_LEG3A = ("(A) Box plots showing the relative abundance of three bacterial species "
"differentially prevalent among the dietary groups. *Wilcoxon rank-sum p<0.05.")
Q_LEG3C = ("The colour indicates the dietary group in which the variable has the maximum level.")
Q_MIRNA = ("Two DEmiRNAs, miR-425-3p and miR-638, attracted our interest since they were "
"associated with a coherent higher expression in VT/VN compared with both the "
"analysed O groups")
Q_QUART = ("Consistently, higher levels of Roseburia sp. CAG 182 and R. bacterium D5, as well as "
"a lower abundance of A. colihominis were found in subjects within the highest "
"expression levels of miR-638.")
Q_DESIGN = ("dietary recording from 120 healthy volunteers, equally distributed for the "
"different diets and matched for sex and age")
QUOTES = [Q_ABSTRACT, Q_RESULTS, Q_DISC1, Q_DISC2, Q_LEG3A, Q_LEG3C, Q_MIRNA, Q_QUART, Q_DESIGN]
# ---------------------------------------------------------------- the ledger
GENUS = ["Prevotella", "Roseburia", "Bacteroides"]
ABSTRACT = {"Prevotella": +1, "Roseburia": +1, "Bacteroides": -1}
# (location, taxon as named there, genus, direction, kind)
LEDGER = [
("Results, 'Gut microbiome composition...' p1", "Prevotella copri", "Prevotella", -1, "text"),
("Results, 'Gut microbiome composition...' p1", "Roseburia sp. CAG 182", "Roseburia", -1, "text"),
("Results, 'Gut microbiome composition...' p1", "Bacteroides dorei", "Bacteroides", +1, "text"),
("Discussion, microbiome paragraph", "P. copri", "Prevotella", -1, "text"),
("Discussion, microbiome paragraph", "Roseburia sp. CAG 182", "Roseburia", -1, "text"),
("Discussion, miR-425-3p paragraph", "Roseburia", "Roseburia", -1, "text"),
("Figure 3A box plots (O vs VT and VN marked *)", "Prevotella copri", "Prevotella", -1, "figure"),
("Figure 3A box plots (O vs VN marked *)", "Roseburia sp CAG 182", "Roseburia", -1, "figure"),
("Figure 3A box plots (O vs VN marked *)", "Bacteroides dorei", "Bacteroides", +1, "figure"),
("Figure 3C DIABLO bar colour (green = VN)", "Roseburia sp CAG 182", "Roseburia", -1, "figure"),
("Figure 3C DIABLO bar colour (green = VN)", "Roseburia hominis", "Roseburia", -1, "figure"),
("Figure 3C DIABLO bar colour (red = O)", "Bacteroides finegoldii", "Bacteroides", +1, "figure"),
("Figure 3C DIABLO bar colour (red = O)", "Bacteroides massiliensis", "Bacteroides", +1, "figure"),
]
LAB = {+1: "higher in O", -1: "lower in O"}
def verify_quotes():
repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
show = os.path.join(repo, "src", "show.py")
py = "/home/ubuntu/papers-data/venv/bin/python"
if not os.path.exists(show):
print("--verify: src/show.py not found; skipped\n")
return
py = py if os.path.exists(py) else sys.executable
full = subprocess.run([py, show, PMCID, "--full"], capture_output=True, text=True,
cwd=repo).stdout
ok = 0
for q in QUOTES:
hit = q in full
ok += hit
print(f" {'found ' if hit else 'MISSING'} {q[:70]}...")
print(f"quotations found verbatim in the PMC text: {ok} of {len(QUOTES)}\n")
def report():
print("Abstract sentence:", Q_ABSTRACT)
for g in GENUS:
print(f" abstract: {g:12s} {LAB[ABSTRACT[g]]}")
print()
print("Every other statement of direction in the paper")
n_agree = 0
by_genus = {g: [0, 0] for g in GENUS}
by_kind = {"text": [0, 0], "figure": [0, 0]}
for loc, taxon, g, d, kind in LEDGER:
agree = d == ABSTRACT[g]
n_agree += agree
by_genus[g][0] += agree
by_genus[g][1] += 1
by_kind[kind][0] += agree
by_kind[kind][1] += 1
print(f" {loc:47s} {taxon:25s} {LAB[d]:12s} agrees with abstract: {agree}")
print()
for g in GENUS:
a, n = by_genus[g]
print(f" {g:12s} statements agreeing with the abstract: {a} of {n}")
for k, (a, n) in by_kind.items():
print(f" {k:12s} statements agreeing with the abstract: {a} of {n}")
print(f" ALL statements agreeing with the abstract: {n_agree} of {len(LEDGER)}")
print()
print("Indirect check (not counted above): miR-425-3p and miR-638 are higher in VT/VN,")
print("and subjects with the highest levels of either have more Roseburia sp. CAG 182.")
print("That chain is coherent only if Roseburia sp. CAG 182 is higher in VT/VN (body),")
print("not higher in O (abstract).")
return by_genus, n_agree
# ---------------------------------------------------------------- figure style
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834" # what the abstract reports
BLUE = "#2a78d6" # what the paper's own Results, Discussion and Figure 3 show
GRID = "#e4e3df"
PANEL = "#f3f2ef"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"figure.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200)
plt.close(fig)
def panel(ax, x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc=PANEL, ec=edge, lw=1.5, transform=ax.transAxes))
ax.text(x + 0.018, y + h - 0.028, title, transform=ax.transAxes, fontsize=10.5,
fontweight="bold", va="top", color=edge if edge != INK2 else INK)
ax.text(x + 0.018, y + h - 0.085, body, transform=ax.transAxes, fontsize=10,
va="top", color=INK, linespacing=1.4)
def figure1(by_genus, n_agree):
fig = plt.figure(figsize=(W, 6.2))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.text(0.03, 0.975, "One comparison (omnivores vs vegetarians/vegans, n = 120),",
fontsize=11.5, fontweight="bold", va="top")
ax.text(0.03, 0.935, "described in opposite directions", fontsize=11.5,
fontweight="bold", va="top")
# two text panels
top, h = 0.885, 0.30
panel(ax, 0.03, top - h, 0.41, h, "Abstract",
"“In omnivores, higher\nPrevotella and Roseburia\nand lower Bacteroides\n"
"abundances than in vegans\nand vegetarians were\nobserved.”", ORANGE)
panel(ax, 0.56, top - h, 0.41, h, "Results (same in Discussion)",
"“Prevotella copri and\nRoseburia sp. CAG 182\nshowed higher abundance in\n"
"VT/VN compared with O, while\nBacteroides dorei … prevailed\n"
"in O (p<0.05; figure 3A).”", BLUE)
ym = top - h / 2
ax.add_patch(FancyArrowPatch((0.445, ym), (0.555, ym), arrowstyle="<->",
mutation_scale=14, lw=2, color=INK2))
ax.text(0.5, ym + 0.025, "all three\nreversed", ha="center", va="bottom", fontsize=10,
fontweight="bold", linespacing=1.2)
ax.text(0.5, ym - 0.025, "same\ncomparison", ha="center", va="top", fontsize=10,
color=INK2, linespacing=1.2)
# direction grid
gy = 0.53
ax.text(0.03, gy, "Direction, omnivores relative to vegetarians/vegans", fontsize=10.5,
fontweight="bold", va="center")
cx_abs, cx_body, cx_n = 0.47, 0.70, 0.90
hy = gy - 0.05
ax.text(cx_abs, hy, "Abstract", ha="center", va="center", fontsize=10, color=ORANGE,
fontweight="bold")
ax.text(cx_body, hy, "Results, Discussion,\nFigure 3", ha="center", va="center",
fontsize=10, color=BLUE, fontweight="bold", linespacing=1.15)
ax.text(cx_n, hy, "agree with\nabstract", ha="center", va="center", fontsize=10,
color=INK2, linespacing=1.15)
names = {"Prevotella": "Prevotella\n(P. copri)",
"Roseburia": "Roseburia\n(sp. CAG 182, R. hominis)",
"Bacteroides": "Bacteroides\n(B. dorei and others)"}
body_dir = {g: [d for _, _, gg, d, _ in LEDGER if gg == g] for g in GENUS}
for i, g in enumerate(GENUS):
y = hy - 0.085 - i * 0.08
ax.plot([0.03, 0.97], [y + 0.04, y + 0.04], color=GRID, lw=1)
ax.text(0.03, y, names[g], fontsize=10, va="center", linespacing=1.2,
style="italic")
da = ABSTRACT[g]
ax.text(cx_abs, y, ("▲ " if da > 0 else "▼ ") + LAB[da], ha="center", va="center",
fontsize=10.5, color=ORANGE, fontweight="bold")
db = body_dir[g][0]
assert all(d == db for d in body_dir[g])
ax.text(cx_body, y, ("▲ " if db > 0 else "▼ ") + LAB[db], ha="center", va="center",
fontsize=10.5, color=BLUE, fontweight="bold")
a, n = by_genus[g]
ax.text(cx_n, y, f"{a} of {n}", ha="center", va="center", fontsize=10.5)
ax.plot([0.03, 0.97], [hy - 0.085 - 2 * 0.08 - 0.04] * 2, color=GRID, lw=1)
ax.text(cx_n, hy - 0.085 - 3 * 0.08 + 0.02, f"total {n_agree} of {len(LEDGER)}",
ha="center", va="center", fontsize=10.5, fontweight="bold")
ax.text(0.03, 0.035, "Orange: the abstract's statement. Blue: direction stated in the "
"Results and Discussion\nand shown in Figure 3A/3C (counted in recompute.py.txt).",
fontsize=10, color=INK2, va="bottom", linespacing=1.3)
return fig
if __name__ == "__main__":
if "--verify" in sys.argv:
verify_quotes()
bg, na = report()
save(figure1(bg, na), "fig1")
print(f"\nwrote fig1.svg and fig1.png to {HERE}")
Abstract calls the frontoparietal TE-entropy association positive; Results, Discussion and Figure 5C show it negative
Hum Brain Mapp (2025) · doi:10.1002/hbm.70427 · PMC12720285
Summary
The abstract reports "a positive association between TE and entropy in the frontoparietal network (p = 7.03e-3)". The Results section reports the same test on the same network, with the same p-value, and says the association was negative. The Discussion also says negative and builds its interpretation on that sign. In Figure 5C, the only network bar marked significant (FPN) lies on the negative side, at a beta of about −0.5 when measured from the published image. A reader who sees only the abstract gets the direction of this result reversed. The paper's main group-difference findings do not depend on it.
What the paper reports
Abstract (results sentences):
There was a positive association between TE and entropy in the frontoparietal network (p = 7.03e-3).
Results, section "Association Between Transition Energy and Entropy":
At the network level, TE was negatively associated with entropy in the frontoparietal network (BH corrected p-value = 7.03e-3).
Discussion, section "Association Between Regional TE and Entropy":
At the network level, a negative association was observed within the frontoparietal network. A negative TE and entropy association in the frontoparietal network implies that loss of entropy (i.e., less complexity) could be linked to higher energetic demands to switch from or to the frontoparietal network, a network that is critical for rapid and accurate behavioral performance (Marek and Dosenbach 2018).
Figure 5 legend, panel C:
(C) The beta estimates from the linear model adjusted for age, sex, and ICV were used to represent the amplitude and the direction of the association between network-wise TE and network-wise entropy. The frontoparietal control network (marked with an asterisk) was significant after multiple comparison p-value corrections.
Materials and Methods, "Statistical Analysis":
The p-values obtained with multiple comparisons for the region, network, and state-wise analysis were corrected using the Benjamini-Hochberg method.
What the paper's own numbers show
Method. The text gives no coefficient for the network-level association, only its direction and a p-value. I compared three things: (1) the direction word and p-value in each of the three statements; (2) the sign of the FPN bar in Figure 5C, measured in pixels from the published image (HBM-46-e70427-g003.jpg) and calibrated against the x-axis tick labels "−0.4", "0" and "0.4" (calibrated once from each side of zero, because the minus sign shifts the "−0.4" label, and reported as a range); and (3) whether choosing which variable is the outcome could reverse the sign. For (3) I ran 2,000 simulated datasets (n = 554, with age, sex and ICV-like covariates) and fitted the regression both ways. A two-sided p-value is the same for t and −t, so p = 7.03e-3 says nothing about direction. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Direction of FPN TE-entropy association | Abstract | positive, p = 7.03e-3 | contradicted by the three rows below |
| Direction of FPN TE-entropy association | Results | negative, BH corrected p = 7.03e-3 | negative |
| Direction of FPN TE-entropy association | Discussion | negative, with an interpretation of the negative sign | negative |
| FPN network-level beta (the only asterisked bar) | Figure 5C | bar, no value printed | β ≈ −0.51 to −0.48 (from image), sign negative |
| p-value in Abstract vs Results | Abstract, Results | 7.03e-3 and 7.03e-3 | identical: the same test |
| Sign change when outcome and predictor are swapped (OLS with covariates) | simulation | — | 0 of 2,000 |
Two minor labelling points, separate from the main finding:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Group sizes | Abstract vs Figure 1 legend | Abstract: "Four hundred ninety-nine HC and 55 MCI"; Figure 1 legend: "(499 MCI and 55 HC)" | the Figure 1 legend swaps the two groups |
| Aβ-entropy association in MCI | Abstract vs Figure 6C | Abstract: "ρ = 0.632, p = 0.041"; Figure 6C: linear-model "estimate: 0.632", "p-value: 0.041" | the same number; the abstract labels a regression estimate as ρ |
Figure
The two panels hold short verbatim excerpts. The left panel, in orange, is the abstract's statement. The right panel, in blue, holds the Results and Discussion statements and the beta measured from Figure 5C. All describe the network-level association between transition energy (TE) and entropy in the frontoparietal network, from a linear model adjusted for age, sex and ICV, with the same p-value (7.03e-3). Only the abstract gives a positive direction. The Figure 5C value (β ≈ −0.51 to −0.48) was measured from the published image by recompute.py.txt. The paper's own figure is not reproduced here.
Why it matters
The abstract is the part of a paper that is indexed, quoted and used when screening for systematic reviews. As it stands, the abstract tells readers that higher transition energy goes with higher entropy (more complex activity) in the frontoparietal network. The paper's own analysis shows the reverse. The Discussion makes the point explicitly: lower entropy goes with higher energetic demand to switch to or from that network. Significance is unaffected, because changing the sign of a regression coefficient leaves its p-value unchanged. The effect size is unaffected too, since the text reports none.
Several things do not depend on this result. They are the paper's headline findings (lower limbic-network TE in MCI, adjusted p = 0.028; lower global entropy in MCI, p = 7.29e-7; the global Aβ-entropy association in MCI), the key findings listed at the start of the Discussion, and the Conclusion. The error is limited to how the abstract reports one secondary, exploratory result. It is still a reversal of direction in the abstract, though, and readers who rely on abstracts cannot detect it.
Explanations considered
- A different test, network or subgroup. Ruled out. The abstract and the Results name the same network, give the same p-value and describe the same network-level linear model. Statistical Analysis describes a single set of covariate-adjusted, BH-corrected models. Nothing in the text says the abstract refers to a subgroup (MCI only or HC only), and no other frontoparietal TE-entropy test is reported.
- Direction of the regression (TE on entropy, or entropy on TE). Ruled out. With the same covariates, both OLS slopes have the sign of the partial correlation. In 2,000 simulated datasets, swapping outcome and predictor never changed the sign.
- A sign convention in Figure 5C. Ruled out. The legend says the betas show "the amplitude and the direction of the association". The asterisked FPN bar lies on the negative side of zero, at about −0.5.
- An error in the body rather than the abstract. Unlikely. The Results text, the Discussion's interpretation and Figure 5C all agree on negative. Only the abstract differs.
- A conversion or archiving artefact. Ruled out. The same wording appears in the Europe PMC JATS XML and on the live PMC page. The bioRxiv preprint (doi:10.1101/2025.06.19.660545, PMC12262368) has the same two sentences, positive in the abstract and negative in Results, both with p = 7.03e-3. The mismatch was carried into the published version unchanged.
- Resolved in the supplement or by a correction. No. The supporting information (
HBM-46-e70427-s001.docx) has Figures S1–S8, and from their captions none covers the network-level TE-entropy association. No correction notice was found (see Status).
Status
- Version of record. The publisher's page (onlinelibrary.wiley.com, reached through the DOI) returned HTTP 403 to automated requests. I checked the publisher-supplied JATS full text in Europe PMC and the live PMC HTML page. Both contain the abstract sentence ("positive") and the Results sentence ("negatively") exactly as quoted. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. The Figure 5C bar positions and tick labels, and the Figure 6C "estimate" and "p-value" labels, were read from the published figure images on the PMC CDN. - Supplements. There is one supporting-information file,
HBM-46-e70427-s001.docx. According to its caption list, Figures S1–S8 cover clustering choices, the T parameter, PET group differences, PET associations and group TE differences. None covers the network-level TE-entropy association, which is settled by the main-text Figure 5C. The preprint's Europe PMC full text was also checked. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Europe PMC record (PMID 41427478) lists only an "Update of" link to the bioRxiv preprint. PubPeer's API returned HTTP 403 to automated requests. A web search for the title together with PubPeer, erratum or correction found no thread or notice. - Sources. PMC article PMC12720285; Europe PMC full-text XML for PMC12720285 and for the preprint PMC12262368; figure images
HBM-46-e70427-g003.jpg(Figure 5) andHBM-46-e70427-g001.jpg(Figure 6); Crossref; Europe PMC; PubPeer; https://doi.org/10.1002/hbm.70427; bioRxiv doi:10.1101/2025.06.19.660545.
Suggested correction
In the abstract, replace
There was a positive association between TE and entropy in the frontoparietal network (p = 7.03e-3).
with
There was a negative association between TE and entropy in the frontoparietal network (BH corrected p = 7.03e-3).
so that the abstract agrees with the Results, the Discussion and Figure 5C. Reporting the FPN beta and its confidence interval in the text would let readers check direction and size without measuring the figure. The authors may also wish to correct two smaller points. The Figure 1 legend should read "499 HC and 55 MCI". The Aβ-entropy result in the abstract should be labelled as a regression estimate (β = 0.632) rather than ρ, since Figure 6C reports 0.632 as the linear-model estimate.
Recomputation script (python recompute.py.txt)
"""Recompute the direction of the frontoparietal TE-entropy association in PMC12720285
(Hum Brain Mapp 2025, doi:10.1002/hbm.70427).
Run: python recompute.py.txt (uses the pixel positions recorded below)
python recompute.py.txt <g003.jpg> (also re-measures them from the image)
Needs numpy, scipy and matplotlib (plus Pillow for the optional re-measurement).
Prints every recomputed number used in writeup.md and writes fig1.svg/png next
to this file.
Inputs
* Text strings copied verbatim from the article as archived in PMC (and identical in
the Europe PMC JATS XML and the live PMC HTML page).
* Figure 5C of the article (file HBM-46-e70427-g003.jpg, 709 x 614 px, from the PMC
CDN). The network-level betas are drawn as bars but not printed, so the bar ends and
the x-axis tick-label centres were measured in pixels (bars: coloured pixels on the
bar's centre row; labels: dark-pixel extents of "-0.4", "0" and "0.4"). The label
"-0.4" includes a minus sign, which shifts its centre, so the beta is calibrated
twice, once from each side of zero, and reported as a range.
"""
import os
import re
import sys
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ verbatim text
ABSTRACT = ("There was a positive association between TE and entropy in the "
"frontoparietal network (p = 7.03e-3).")
RESULTS = ("At the network level, TE was negatively associated with entropy in the "
"frontoparietal network (BH corrected p-value = 7.03e-3).")
DISCUSSION = ("At the network level, a negative association was observed within the "
"frontoparietal network. A negative TE and entropy association in the "
"frontoparietal network implies that loss of entropy (i.e., less "
"complexity) could be linked to higher energetic demands to switch from "
"or to the frontoparietal network")
FIG5_LEGEND = ("(C) The beta estimates from the linear model adjusted for age, sex, and "
"ICV were used to represent the amplitude and the direction of the "
"association between network-wise TE and network-wise entropy. The "
"frontoparietal control network (marked with an asterisk) was "
"significant after multiple comparison p-value corrections.")
ABSTRACT_N = "Four hundred ninety-nine HC and 55 MCI patients were included."
FIG1_LEGEND_N = "The fMRI BOLD time series for all subjects (499 MCI and 55 HC)"
ABSTRACT_ABETA = "Increased global Aβ was associated with higher global entropy in MCI patients (ρ = 0.632, p = 0.041)."
# ------------------------------------------------------------------ Figure 5C pixels
ZERO_LABEL_X = (599 + 608) / 2 # centre of label "0"
POS_LABEL_X = (660 + 689) / 2 # centre of label "0.4"
NEG_LABEL_X = (509 + 546) / 2 # centre of label "-0.4" (includes the minus sign)
BAR_ZERO_X = 603 # common base of all seven bars
BARS = { # network: (left end px, right end px) on the bar's centre row
"VIS": (604, 632), "VAN": (605, 623), "SMN": (604, 698), "LIM": (603, 639),
"FPN": (512, 602), "DMN": (552, 603), "DAN": (603, 660)}
ASTERISK = "FPN" # the only bar marked significant
def remeasure(path):
from PIL import Image
im = np.asarray(Image.open(path).convert("RGB")).astype(int)
print(f"re-measured from {os.path.basename(path)} ({im.shape[1]}x{im.shape[0]} px):")
rows = {"VIS": 45, "VAN": 120, "SMN": 192, "LIM": 268, "FPN": 340, "DMN": 412, "DAN": 487}
for k, r in rows.items():
seg = im[r, 480:709]
sat = seg.max(1) - seg.min(1)
cols = np.where(sat > 40)[0] + 480
print(f" {k}: bar spans x = {cols.min()}..{cols.max()}")
lab = im[540:565, 480:709].mean(2) < 120
print(" dark tick-label columns:", (np.where(lab.any(0))[0] + 480).tolist())
def beta_from_px(left, right):
end = left if left < BAR_ZERO_X - 1 else right
d = end - BAR_ZERO_X
pos_scale = (POS_LABEL_X - ZERO_LABEL_X) / 0.4
neg_scale = (ZERO_LABEL_X - NEG_LABEL_X) / 0.4
return sorted([d / pos_scale, d / neg_scale])
def sign_word(s):
if re.search(r"\bpositive", s):
return "positive"
if re.search(r"\bnegative", s):
return "negative"
return "none"
def main():
if len(sys.argv) > 1:
remeasure(sys.argv[1])
print("== 1. Direction and p-value stated in each place")
pv = lambda s: (re.search(r"7\.03e-3", s) or [None])[0] # noqa: E731
for name, s in [("Abstract", ABSTRACT), ("Results", RESULTS), ("Discussion", DISCUSSION)]:
print(f" {name:<10} direction = {sign_word(s):<8} p = {pv(s)}")
print(" Same p-value string in Abstract and Results:", pv(ABSTRACT) == pv(RESULTS))
print("== 2. Figure 5C network-level betas read from the image")
betas = {}
for k, (l, r) in BARS.items():
lo, hi = beta_from_px(l, r)
betas[k] = (lo, hi)
star = " *" if k == ASTERISK else ""
print(f" {k}: beta ~ {lo:+.2f} to {hi:+.2f}{star}")
fpn = betas["FPN"]
print(f" FPN (the asterisked bar): beta ~ {fpn[0]:.2f} to {fpn[1]:.2f}; sign negative:", fpn[1] < 0)
print("== 3. Does reversing outcome and predictor change the sign? (OLS, covariates)")
rng = np.random.default_rng(0)
n, flips, reps = 554, 0, 2000
for _ in range(reps):
Z = np.column_stack([np.ones(n), rng.normal(70, 8, n), rng.integers(0, 2, n),
rng.normal(1500, 150, n)])
rho = rng.uniform(-0.5, 0.5)
te = rng.normal(size=n) + 0.002 * Z[:, 3]
ent = rho * te + rng.normal(size=n) + 0.01 * Z[:, 1]
b1 = np.linalg.lstsq(np.column_stack([te, Z]), ent, rcond=None)[0][0]
b2 = np.linalg.lstsq(np.column_stack([ent, Z]), te, rcond=None)[0][0]
flips += int(np.sign(b1) != np.sign(b2))
print(f" sign flips: {flips} / {reps} (both slopes share the sign of the partial correlation)")
print("== 4. Is the p-value informative about direction?")
df = 554 - 5
for t in (2.7, -2.7):
print(f" illustration: t = {t:+.1f}, df = {df}: two-sided p = {2 * stats.t.sf(abs(t), df):.4g}")
print(" -> identical p for opposite signs; p = 7.03e-3 cannot distinguish them")
print("== 5. Minor side observations")
print(" Abstract: 499 HC, 55 MCI; Figure 1 legend: '499 MCI and 55 HC' -> groups swapped")
print(" Abstract labels the Abeta-entropy result 'rho = 0.632'; Figure 6C labels 0.632 an 'estimate'"
" from the linear model (a regression beta, not a correlation)")
make_fig1(fpn)
print("wrote fig1.svg/png")
# ------------------------------------------------------------------ figures
BG, INK, INK2, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"svg.fonttype": "none", "axes.edgecolor": INK2})
def wrap(s, w):
return "\n".join(textwrap.wrap(s, w))
def make_fig1(fpn):
W, H = 7.2, 4.6
fig = plt.figure(figsize=(W, H), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.axis("off")
def box(x, y, w, h, color):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=color, lw=2))
top, bot = 4.35, 1.05
# left: abstract
box(0.15, bot, 2.7, top - bot, ORANGE)
ax.text(0.3, top - 0.17, "Abstract", color=ORANGE, fontsize=12, fontweight="bold", va="top")
ax.text(0.3, top - 0.6, wrap("\u201c" + ABSTRACT + "\u201d", 29), color=INK, va="top",
fontsize=10.5, linespacing=1.35)
ax.text(0.3, bot + 0.38, "Direction stated: POSITIVE", color=ORANGE, fontsize=10.5,
fontweight="bold", va="top")
# right: body
box(3.95, bot, 3.1, top - bot, BLUE)
ax.text(4.1, top - 0.17, "Results \u00b7 Discussion \u00b7 Figure 5C", color=BLUE,
fontsize=11, fontweight="bold", va="top")
body = [("Results:", "\u201cTE was negatively associated with entropy in the "
"frontoparietal network (BH corrected p-value = 7.03e-3).\u201d"),
("Discussion:", "\u201ca negative association was observed within the "
"frontoparietal network\u201d"),
("Figure 5C (read from image):",
f"asterisked FPN bar at \u03b2 \u2248 {fpn[0]:.2f} to {fpn[1]:.2f}".replace("-", "\u2212"))]
y = top - 0.55
for lab, txt in body:
ax.text(4.1, y, lab, color=INK2, fontsize=10, fontweight="bold", va="top")
t = wrap(txt, 40)
ax.text(4.1, y - 0.2, t, color=INK, fontsize=10, va="top", linespacing=1.3)
y -= 0.2 + 0.185 * (t.count("\n") + 1) + 0.17
ax.text(4.1, bot + 0.38, "Direction stated: NEGATIVE", color=BLUE, fontsize=10.5,
fontweight="bold", va="top")
# connector
ym = (top + bot) / 2
ax.add_patch(FancyArrowPatch((2.9, ym), (3.9, ym), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=INK))
ax.text(3.4, ym + 0.15, "opposite\nsign", ha="center", va="bottom", fontsize=10.5,
fontweight="bold", color=INK)
ax.text(3.4, ym - 0.15, "same\nnetwork,\nsame test,\nsame p", ha="center", va="top",
fontsize=10, color=INK2, linespacing=1.25)
ax.text(0.15, 0.82, wrap("Excerpts quoted verbatim from the article as archived in PMC "
"(PMC12720285). The Figure 5C beta was measured from the published "
"image (see recompute.py.txt). Orange: the abstract's statement. "
"Blue: the paper's Results, Discussion and Figure 5C.", 84),
color=INK2, fontsize=10, va="top", linespacing=1.3)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
if __name__ == "__main__":
main()
Both all-cause hazard ratios lie outside their own 95% confidence intervals in abstract, text and table
Int J Epidemiol (2023) · doi:10.1093/ije/dyad006 · PMC10114125
Summary
The paper's headline result is the all-cause mortality hazard ratio (HR) versus housed patients. It is reported as HR 4.0 (95% CI 2.0–3.3) for homeless patients and HR 2.6 (95% CI 3.4–4.8) for marginally housed patients, with the same figures in the abstract, the Results text and Table 2. Neither point estimate lies inside its own interval, and rounding cannot close the gap. The geometric midpoint of each printed interval (2.57 and 4.04) matches the other group's HR, while all 14 cause-specific rows of Table 2 are centred on their own estimates to within 0.3%. The most likely explanation is that the two intervals were transposed. The direction and significance of the findings do not change, but the uncertainty printed for the main result is impossible as it stands, and the printed intervals reverse the ordering behind the paper's comparison of the two groups.
What the paper reports
Abstract, Findings (the unmatched bracket is in the original):
Data from 6290 patients (homeless deceased = 382/1050, marginally housed deceased = 259/518, housed deceased = 1204/4722) found increased risk of mortality in homeless [hazard ratio (HR) = 4.0, 95% confidence interval (CI) = 2.0–3.3) and marginally housed (HR = 2.6, 95% CI = 3.4–4.8) patients.
Results, all-cause mortality paragraph:
Risk of all-cause mortality over the 16-year follow-up period was increased for homeless (HR = 4.0, 95% CI = 2.0–3.3) and marginally housed (HR = 2.6, 95% CI = 3.4–4.8) patients after their index (first) 2003/04 ED presentation, compared with housed patients who presented for ED care during the same period (Table 2).
Table 2, "Estimated hazard ratios for all-cause and cause-specific deaths in homeless and marginally housed patients", header and All-causes rows (columns: Cox HR | FPNP HR | 95% CI | P-value):
Homeless | | 4.0 | 2.0–3.3 | <0.0001
Marginally housed | | 2.6 | 3.4–4.8 | <0.0001
Table 2 footer:
HR, hazard ratio; FPNP, flexible non-proportional parametric hazards model; CI, confidence interval.
Methods, Statistical analysis:
Hazard ratios (HRs) and 95% confidence intervals (CIs) were generated for homeless and marginally housed patients, with housed patients as the reference category. We assessed for violations of the proportionality assumption by calculating Schoenfeld residuals. If proportionality did not hold, flexible non-proportional parametric models were created using restricted cubic splines to incorporate time-dependent effects.
Abstract, Conclusions:
Marginal housing may assuage some risk of premature mortality associated with homelessness; however, it is not equivalent to stable housing.
What the paper's own numbers show
Method. A 95% confidence interval reported with a point estimate must contain that estimate. For ratio measures estimated on the log scale, which is the standard approach for Cox and flexible parametric survival models, the interval is also near-symmetric on the log scale, so the point estimate is close to the geometric midpoint √(L×U) of the limits. The implied standard error of ln HR is (ln U − ln L)/(2 × 1.96). I applied these checks to all 16 rows of Table 2 and then repeated them with the two all-cause intervals exchanged. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Homeless all-cause HR inside its CI? | Abstract, Results, Table 2 | 4.0 (2.0–3.3) | No: 4.0 exceeds the upper limit by 0.7. Even at the rounding extremes (HR ≥ 3.95, upper < 3.35) the gap is ≥ 0.60 |
| Marginally housed all-cause HR inside its CI? | Abstract, Results, Table 2 | 2.6 (3.4–4.8) | No: 2.6 is 0.8 below the lower limit. At the rounding extremes the gap is ≥ 0.70 |
| Geometric midpoint of the homeless CI | Table 2 | 2.0–3.3 | √(2.0×3.3) = 2.57, which rounds to 2.6, the marginally housed HR |
| Geometric midpoint of the marginally housed CI | Table 2 | 3.4–4.8 | √(3.4×4.8) = 4.04, which rounds to 4.0, the homeless HR |
| Printed HR ÷ √(L×U), all-cause rows | Table 2 | – | 1.56 (homeless), 0.64 (marginally housed) |
| Printed HR ÷ √(L×U), 14 cause-specific rows | Table 2 | e.g. 6.11 (4.47–8.35), 4.69 (1.82–12.05) | 0.997 to 1.003 for every row; all 14 contain their estimate |
| P-values recomputed from the cause-specific CIs (Wald, log scale) | Table 2 | 0.550, 0.004, 0.060, 0.194, <0.001; others <0.0001 | 0.55, 0.004, 0.060, 0.19, 0.0014; others < 0.0001. All agree to rounding except respiratory/marginally housed (<0.001 vs 0.0014), a small difference that may reflect a different test and does not concern the all-cause rows |
| Swap test: homeless HR 4.0 with CI 3.4–4.8 | derived | – | Contains 4.0; √(L×U) = 4.04; log-scale asymmetry +0.020 |
| Swap test: marginally housed HR 2.6 with CI 2.0–3.3 | derived | – | Contains 2.6; √(L×U) = 2.57; log-scale asymmetry −0.024 |
| Implied SE(ln HR), homeless / marginally housed | derived | – | As printed: 0.128 / 0.088 (ratio 1.45). After swap: 0.088 / 0.128 (ratio 0.69) |
| Rough SE(ln HR) from death counts, √(1/d₁ + 1/d₀) | Abstract counts | 382, 259 and 1204 deaths | 0.059 / 0.068 (ratio 0.86): the group with more deaths should have the narrower interval, which matches the swapped pairing, not the printed one. This is a crude approximation that ignores the time-varying FPNP model, so only the direction of the ratio is informative |
| Crude proportions dead | Abstract | 382/1050, 259/518, 1204/4722 | 0.364, 0.500, 0.255 (consistent with the counts; total 6290) |
Figure
Figure 1. All-cause hazard ratios versus housed patients, on a log scale with the null (HR = 1) marked. Top two rows, in orange, show the values as printed in the abstract, the Results and Table 2. Each orange point lies outside its orange interval. The open blue diamonds mark the geometric midpoint √(L×U) of each printed interval, which is where a log-scale interval puts its point estimate: 2.57 for the homeless interval and 4.04 for the marginally housed interval, each matching the other group's printed HR. The bottom two rows, in blue, pair each printed HR with the other row's interval. Each interval then contains its estimate and is centred on it. This pairing is an inference from the paper's numbers and is not confirmed by the paper.
Figure 2. For every row of Table 2, the printed HR divided by the geometric midpoint of its own printed 95% CI, on a log scale. The 14 cause-specific rows (open circles) sit at 0.997–1.003, so the table's intervals follow the usual log-symmetric convention. Only the two all-cause rows (filled circles) depart from it, at 1.56 and 0.64, which are reciprocal deviations of about the same size. A systematic reporting convention would affect all rows, so this points to an error confined to the two all-cause rows.
Why it matters
- The all-cause HRs are the paper's primary result. The impossible intervals appear in the abstract, which is the part most often read, cited and indexed on its own, as well as in the Results text and Table 2. The Discussion repeats only the point estimates ("four times higher", "2.6 times greater"). The title and the Conclusions section do not quote the intervals.
- As printed, the homeless interval (2.0–3.3) lies entirely below the marginally housed interval (3.4–4.8). Taken at face value, that would mean marginally housed patients had significantly higher all-cause hazard than homeless patients. This is the opposite of the paper's conclusion that "Marginal housing may assuage some risk of premature mortality associated with homelessness". Under the swapped pairing, the ordering agrees with the point estimates and with that conclusion.
- Readers who extract these estimates for meta-analysis would enter either an impossible pair or an interval that belongs to the other group. A pooled analysis that derives standard errors from the printed limits would give the homeless estimate the wider interval and the marginally housed estimate the narrower one.
- Unaffected: the direction and statistical significance of both all-cause associations (every candidate interval excludes 1), the point estimates themselves, all cause-specific HRs, the cumulative incidence probabilities of death by age 55 (0.2, 0.1 and 0.02), and the crude death counts.
Explanations considered
- Non-proportional (FPNP) model. The all-cause estimates come from the flexible non-proportional parametric model, not from Cox regression. This does not explain the mismatch. Whatever model produced it, a 95% CI reported with a point estimate must contain that estimate. If a time-varying HR is summarised at one time point or averaged, its interval is computed for the same summary. Three other rows in Table 2 (circulatory, respiratory and the footnote-a category) are also FPNP estimates, and all of them are centred on their point estimates.
- The intervals refer to a different quantity (another time point, an adjusted model or another reference group). The Methods describe one unadjusted model per outcome with housed patients as the reference. Neither the table footnote nor the text says the all-cause intervals refer to anything else. The abstract and the Results text present each interval in parentheses directly after its HR.
- Rounding. The HRs and limits are given to one decimal. At the most favourable rounding extremes, the gaps are still at least 0.60 (homeless) and 0.70 (marginally housed), so rounding cannot account for them.
- Unit or scale mix-up. Every other row of Table 2 is on the ratio scale and internally consistent (Figure 2), and both all-cause intervals are plausible ratio-scale intervals. They simply sit on the wrong rows.
- Conversion or typesetting artifact in PMC. The same values appear in the publisher's HTML, the PMC HTML and the Europe PMC full-text XML, where each HR and its CI are in adjacent cells of the same table row. The same pairs also appear in two prose passages, so a table-rendering fault in one version cannot explain them.
- Clarified in the supplement. The supplementary file covers housing definitions, inter-rater reliability, sampling, ICD-10 groupings and a linkage flow chart. It contains no hazard ratios or model output.
None of these accounts for the numbers. Exchanging the two intervals between the rows explains every observation: containment, log-symmetry, and the ordering of interval widths by number of deaths. However, only the authors' model output can confirm what the correct intervals are.
Status
As of 2026-10-24:
- Version of record. The publisher's HTML (doi.org/10.1093/ije/dyad006, which resolves to academic.oup.com/ije/article/52/2/501/7031336) gives the same values in the abstract, the Results and Table 2. So do the PMC HTML, the Europe PMC full-text XML and the local PMC archive copy. The publisher PDF could not be retrieved (the download returned a block page).
- Supplement. dyad006_supplementary_data.docx, obtained through the Europe PMC supplementary-files archive, was read in full. It does not report hazard ratios and does not resolve the discrepancy.
- Corrections.
- Crossref has no update-to, updated-by or relation entries for the DOI.
- PMC and Europe PMC correction/retraction notice queries returned nothing (re-checked on 2026-10-24).
- A Crossref bibliographic search for a "Correction to" notice under the publisher prefix found only the original article.
- Neither the publisher page nor the PMC page shows a correction notice.
- Web searches found no erratum and no Retraction Watch coverage.
- The Europe PMC search endpoint returned errors on three attempts, so its commentCorrectionList could not be checked directly.
- PubPeer. The API and search page did not return parseable results, so the PubPeer check is inconclusive.
Suggested correction
In the abstract, the Results (all-cause mortality paragraph) and Table 2 (All-causes rows), state the correct 95% confidence interval for each all-cause hazard ratio from the FPNP model output. If, as the paper's own numbers suggest, the intervals were exchanged between rows, the corrected values would read: homeless HR 4.0 (95% CI 3.4–4.8) and marginally housed HR 2.6 (95% CI 2.0–3.3). Both P-values (<0.0001) and all cause-specific estimates would stay as they are. A short correction notice linked to the article would let readers and meta-analysts who have already extracted the printed intervals update them. The unmatched bracket in the abstract ("[hazard ratio (HR) = 4.0, 95% confidence interval (CI) = 2.0–3.3)") could be fixed at the same time.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC10114125 (Int J Epidemiol 2023, doi:10.1093/ije/dyad006).
# Every input below is copied from the paper (abstract, Results, Table 2).
# Run: python recompute.py.txt -> prints all numbers in writeup.md and writes fig1/fig2 (.svg, .png)
import math, os
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# Table 2: (label, HR, lower, upper, reported P)
T2 = [
("All-cause, homeless (FPNP)", 4.0, 2.0, 3.3, "<0.0001"),
("All-cause, marginally housed (FPNP)", 2.6, 3.4, 4.8, "<0.0001"),
("External, homeless", 6.11, 4.47, 8.35, "<0.0001"),
("External, marginally housed", 3.57, 2.36, 5.40, "<0.0001"),
("Circulatory, homeless", 4.92, 2.78, 8.70, "<0.0001"),
("Circulatory, marginally housed", 1.44, 0.43, 4.79, "0.550"),
("Cancer, homeless", 1.55, 1.15, 2.09, "0.004"),
("Cancer, marginally housed", 1.34, 0.99, 1.82, "0.060"),
("Respiratory, homeless", 2.14, 0.68, 6.77, "0.194"),
("Respiratory, marginally housed", 4.69, 1.82, 12.05, "<0.001"),
("Other (footnote a), homeless", 4.09, 2.78, 6.03, "<0.0001"),
("Other (footnote a), marg. housed", 3.31, 2.03, 5.38, "<0.0001"),
("Alcohol, homeless", 8.91, 6.22, 12.78, "<0.0001"),
("Alcohol, marginally housed", 4.52, 2.87, 7.13, "<0.0001"),
("Drugs, homeless", 7.75, 5.04, 11.89, "<0.0001"),
("Drugs, marginally housed", 4.21, 2.35, 7.54, "<0.0001"),
]
print("1. Containment, geometric midpoint, log-asymmetry, implied SE and P for every Table 2 row")
print(f"{'row':38s} {'HR':>5} {'CI':>12} {'in CI':>6} {'sqrt(LU)':>8} {'HR/sqrt(LU)':>11} {'log-asym':>8} {'SE(lnHR)':>8} {'P (from CI)':>11} {'P reported':>10}")
rows_out = []
for lab, h, l, u, prep in T2:
gm = math.sqrt(l * u)
se = (math.log(u) - math.log(l)) / (2 * Z)
asym = math.log(u / h) - math.log(h / l)
p = 2 * norm.sf(abs(math.log(h) / se))
rows_out.append((lab, h, l, u, gm, h / gm, asym, se, p))
print(f"{lab:38s} {h:5.2f} {f'{l:.2f}-{u:.2f}':>12} {str(l <= h <= u):>6} {gm:8.2f} {h/gm:11.3f} {asym:+8.3f} {se:8.3f} {p:11.2g} {prep:>10}")
print("\n2. Gap between each all-cause HR and its printed interval")
print(f"homeless: HR 4.0 exceeds upper limit 3.3 by {4.0-3.3:.1f}; smallest possible gap given rounding "
f"(HR >= 3.95, upper < 3.35) = {3.95-3.35:.2f}")
print(f"marginally housed: HR 2.6 is below lower limit 3.4 by {3.4-2.6:.1f}; smallest possible gap given rounding "
f"(HR < 2.65, lower >= 3.35) = {3.35-2.65:.2f}")
print("\n3. Swap test: each all-cause HR paired with the other row's interval")
for g, h, (l, u) in [("homeless", 4.0, (3.4, 4.8)), ("marginally housed", 2.6, (2.0, 3.3))]:
gm = math.sqrt(l * u)
print(f"{g}: HR {h} with CI {l}-{u}: contains={l <= h <= u}, sqrt(LU)={gm:.3f}, "
f"log-asym={math.log(u/h)-math.log(h/l):+.3f}, SE(lnHR)={(math.log(u)-math.log(l))/(2*Z):.3f}")
print("\n4. Crude counts (abstract) and a rough large-sample SE(ln HR) = sqrt(1/d1 + 1/d0)")
d = {"homeless": (382, 1050), "marginally housed": (259, 518), "housed": (1204, 4722)}
for g, (k, n) in d.items():
print(f"{g}: {k}/{n} = {k/n:.3f}")
print(f"total patients = {sum(n for _, n in d.values())}")
se_h = math.sqrt(1/382 + 1/1204); se_m = math.sqrt(1/259 + 1/1204)
print(f"rough SE(ln HR): homeless {se_h:.3f}, marginally housed {se_m:.3f}; ratio homeless/marginal = {se_h/se_m:.2f}")
se_a = (math.log(3.3)-math.log(2.0))/(2*Z); se_b = (math.log(4.8)-math.log(3.4))/(2*Z)
print(f"implied SE ratio homeless/marginal: as printed {se_a/se_b:.2f}; after swap {se_b/se_a:.2f}")
print("\n5. Do the two all-cause intervals overlap?")
print("as printed: homeless 2.0-3.3 vs marginal 3.4-4.8 -> homeless interval lies wholly below marginal:", 3.3 < 3.4)
print("after swap: homeless 3.4-4.8 vs marginal 2.0-3.3 -> marginal interval lies wholly below homeless:", 3.3 < 3.4)
# ---------------------------------------------------------------- figures
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC,
"font.family": "DejaVu Sans", "svg.fonttype": "none"})
LW, MS = 2, 9
def style(ax):
ax.set_facecolor(BG)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# Figure 1: all-cause rows as printed, with the centre each printed CI implies, and the re-paired version
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
style(ax)
rows = [ # y, label, HR, L, U, colour of interval/dot, show implied centre
(5, "Homeless, as printed", 4.0, 2.0, 3.3, ORA, True),
(4, "Marginally housed, as printed", 2.6, 3.4, 4.8, ORA, True),
(2.0, "Homeless, re-paired", 4.0, 3.4, 4.8, BLU, False),
(1.0, "Marginally housed, re-paired", 2.6, 2.0, 3.3, BLU, False),
]
ax.axvline(1.0, color=SEC, lw=0.8, zorder=1)
ax.text(1.0, 5.75, " null (HR = 1)", color=SEC, fontsize=10, va="center", ha="left")
for y, lab, h, l, u, c, show in rows:
ax.plot([l, u], [y, y], color=c, lw=LW, solid_capstyle="butt", zorder=3)
ax.plot([l, l], [y-0.12, y+0.12], color=c, lw=LW, zorder=3)
ax.plot([u, u], [y-0.12, y+0.12], color=c, lw=LW, zorder=3)
ax.plot(h, y, "o", color=c, ms=MS, zorder=4)
ax.text(h, y + 0.28, f"HR {h}", color=c, ha="center", fontsize=10)
ax.text(5.15, y, f"{l}–{u}", color=c, va="center", fontsize=10)
if show:
gm = math.sqrt(l * u)
ax.plot(gm, y, "D", mfc=BG, mec=BLU, mew=LW, ms=MS, zorder=5)
ax.text(gm, y - 0.36, f"√(L×U) = {gm:.2f}", color=BLU, ha="center", fontsize=10)
ax.axhline(3.0, color=GRID, lw=0.8)
ax.text(0.93, 2.9, "Below: each HR with the other row's interval\n(inferred pairing, not confirmed by the paper)",
color=SEC, fontsize=10, va="top", ha="left")
ax.set_yticks([r[0] for r in rows]); ax.set_yticklabels([r[1] for r in rows])
ax.set_xscale("log"); ax.set_xlim(0.9, 6.6); ax.set_ylim(0.45, 6.0)
ticks = [1, 1.5, 2, 3, 4, 5]
ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
ax.set_xlabel("All-cause hazard ratio vs housed (log scale)")
ax.text(5.15, 5.75, "95% CI", color=SEC, fontsize=10, va="center")
handles = [Line2D([], [], color=ORA, lw=LW, marker="o", ms=MS, label="As printed (HR and 95% CI)"),
Line2D([], [], color=BLU, lw=0, marker="D", mfc=BG, mec=BLU, mew=LW, ms=MS,
label="Centre implied by the printed CI, √(L×U)"),
Line2D([], [], color=BLU, lw=LW, marker="o", ms=MS, label="Re-paired HR and CI")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.3, -0.17), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.31, right=0.98, top=0.97, bottom=0.27)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 2: HR / sqrt(L*U) for all 16 rows of Table 2
fig, ax = plt.subplots(figsize=(7.2, 5.6), facecolor=BG)
style(ax)
n = len(rows_out)
for i, (lab, h, l, u, gm, r, asym, se, p) in enumerate(rows_out):
y = n - i
allc = lab.startswith("All-cause")
ax.plot(r, y, "o", color=BLU, ms=MS, mfc=(BG if not allc else BLU), mew=LW, zorder=4)
if allc:
ax.text(r * (0.96 if r > 1 else 1.04), y, f"{h} / {gm:.2f} = {r:.2f}", color=BLU, va="center",
ha="right" if r > 1 else "left", fontsize=10)
ax.axvline(1.0, color=SEC, lw=0.8, zorder=1)
ax.set_yticks(range(n, 0, -1)); ax.set_yticklabels([r[0] for r in rows_out], fontsize=10)
ax.set_xscale("log"); ax.set_xlim(0.5, 2.0)
ticks = [0.5, 0.64, 0.8, 1, 1.25, 1.56, 2.0]
ax.set_xticks(ticks); ax.set_xticklabels([str(t) for t in ticks]); ax.minorticks_off()
ax.set_xlabel("Printed HR ÷ √(L×U) of its own printed 95% CI (log scale)")
handles = [Line2D([], [], color=BLU, lw=0, marker="o", mfc=BG, mec=BLU, mew=LW, ms=MS,
label="Cause-specific rows: ratio 0.997–1.003"),
Line2D([], [], color=BLU, lw=0, marker="o", ms=MS, label="All-cause rows: ratio 1.56 and 0.64")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.3, -0.13), ncol=1, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.38, right=0.96, top=0.98, bottom=0.24)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nfigures written to", HERE)
Abstract hazard ratio for SLiM-CRAB status lies outside its own confidence interval and contradicts the Results
J Clin Oncol (2024) · doi:10.1200/JCO.23.02771 · PMC11404760
Summary
This single-arm phase II trial in high-risk smoldering myeloma (90 patients) reports in its abstract that having any SLiM-CRAB criterion predicted progression to multiple myeloma (MM), with "hazard ratio, 0.12; 95% CI, 0.14 to 1.13; P = .03". Those three numbers cannot belong together. The point estimate lies below the lower confidence limit, and the interval contains 1, so a Wald test based on it gives P ≈ 0.084, not .03. The Results section reports the same comparison as HR 0.10 (95% CI 0.01 to 0.96; P = .046). Those values are internally consistent (the interval's centre is 0.098, Wald P 0.046) and roughly match the paper's raw counts. The correct result keeps its direction and nominal significance. But the abstract figures, the version most readers see, cannot be read as printed, and the body's figures show that the estimate rests on 5 events and has an interval spanning about two orders of magnitude.
What the paper reports
Abstract, Results:
The presence of any SLiM CRAB criteria predicted progression to MM (four of the five patients; hazard ratio, 0.12; 95% CI, 0.14 to 1.13; P = .03).
Results, "Time-To-Event End Points", paragraph 4 of that section (same comparison, same outcome, same figure):
As mentioned above, 28 patients presented with any one SLiM biomarker. Although their response rate was not different from that observed in the standard high-risk–SMM population (Appendix Table A2), the presence of any SLiM biomarker predicted progression to MM (HR, 0.10 [95% CI, 0.01 to 0.96]; P = .046; Fig 3B).
The subgroup sizes. Abstract, Results:
90 patients were included, and 31% met at least one SixtyLightchain MRI (SLiM)-hypercalcemia, renal impairment, anemia, bone disease (CRAB) criterion.
Table 1:
SLiM-CRAB criteria, No. (%) | 28 (31)
Appendix Table A2 gives the columns "High Risk (n = 62)" and "Ultra High Risk (n = 28)". Its footnote b reads:
Four patients biochemically progressed and one of them progressed to myeloma.
The model. Methods, "Statistical Analysis":
The Kaplan-Meier method was used to characterize event time distributions, and the corresponding treatment hazard ratio was estimated using a stratified Cox regression model to compare outcomes based on baseline characteristics and MRD status.
The conclusion that rests on this result. Discussion, paragraph 6:
In our study, the only feature predicting progression to MM was the presence of any SLiM biomarker, now incorporated into the definition of MM, which would reinforce this hypothesis.
What the paper's own numbers show
Method. A Cox-model 95% CI is symmetric on the log scale, so the reported HR should lie close to the interval's geometric centre, sqrt(lower × upper). The standard error of log HR is ln(upper/lower)/(2 × 1.96), and the two-sided Wald P is 2Φ(−|ln centre|/SE). I applied this check to every hazard ratio printed in the paper (five in all), which shows whether the paper follows one consistent convention. I then checked whether rounding or a single misplaced digit could reconcile the abstract. As a rough check on direction and size that is independent of the Cox model, I computed a crude risk ratio from the counts (1 of 62 vs 4 of 28 progressed to MM). It ignores follow-up time and is not the paper's estimate. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Is the HR inside its 95% CI? | Abstract, Results | HR 0.12; CI 0.14 to 1.13 | No: 0.12 < 0.14 |
| Centre of the CI, sqrt(lo × hi) | Abstract, Results | HR 0.12 | 0.398 |
| Does the CI exclude 1? | Abstract, Results | P = .03 | No: 1.13 > 1, so any test matching this interval gives P ≥ .050 |
| Wald P from the CI | Abstract, Results | P = .03 | 0.084 (SE of log HR 0.533) |
| Can rounding explain HR < lower limit? | Abstract, Results | 0.12 vs 0.14 | No: values rounding to 0.12 are < 0.125; values rounding to 0.14 are ≥ 0.135 |
| Centre of the CI | Results, Time-To-Event End Points; Fig 3B | HR 0.10; CI 0.01 to 0.96 | 0.098 |
| Wald P from the CI | Results, Time-To-Event End Points; Fig 3B | P = .046 | 0.046 from the CI centre; 0.048 from HR 0.10 (SE 1.164) |
| The paper's other three HRs: CI centre and Wald P | Results, Time-To-Event End Points (Fig 2B, Appendix Fig A1A, A1B) | 0.29 (0.13 to 0.66), P = .003; 0.42 (0.22 to 0.81), P = .01; 0.40 (0.18 to 0.85), P = .018 | 0.293, P 0.003; 0.422, P 0.009; 0.391, P 0.018: all consistent |
| Abstract with the lower limit read as 0.014 | hypothetical single typo | n/a | Centre 0.126 (fits 0.12), but the CI still contains 1; Wald P 0.058 to 0.064 |
| CI that HR 0.12 with P = .03 would imply | hypothetical | n/a | 0.018 to 0.814 |
| Progression to MM by SLiM status | Abstract ("four of the five"); Table 1; Table A2 | 4 of 5 events in the SLiM group; n = 28 vs 62 | 4/28 = 14.3% vs 1/62 = 1.6% |
| Crude risk ratio, no SLiM vs SLiM | from the counts above | n/a | 0.113 (95% CI 0.013 to 0.965); Wald P 0.046; Fisher exact P 0.031 |
| Share of patients with any SLiM-CRAB criterion | Abstract; Table 1 | 31%; 28 (31) | 28/90 = 31.1% (consistent) |
Of the five hazard ratios printed in the paper, the abstract's is the only one whose point estimate, interval and P value do not fit together. The other four, including the Results-section statement of this same comparison, agree with the Wald method to within rounding.
Further observations, not part of the main finding. The following are reported here so that a correction can address them together. I have not investigated them further.
| Quantity | Location | Reported | Check |
|---|---|---|---|
| 70-month TTP | Abstract; Results, Time-To-Event End Points | 94% (95% CI, 84 to 89) | Estimate lies outside its own CI |
| 70-month OS | Abstract; Results, Time-To-Event End Points | 92% (95% CI, 82 to 89) | Estimate lies outside its own CI |
| 70-month TTBP | Results, Time-To-Event End Points | 59% (95% CI, 66 to 76) | Estimate lies outside its own CI |
| Patients with any SLiM criterion | Results, Time-To-Event End Points, paragraph 2 | "12 [38.7%] of 31 patients" vs "18 [30.5%] of 59 patients" | 31 here vs 28 in Table 1, Table A2 and the Results paragraph quoted above; 12 + 18 = 30, against 36 biochemical progressions reported in the same section |
| Sustained uMRD 4 years after ASCT | Abstract / Discussion and Table A1 / CONTEXT box | "29 patients (31%)" / "28 patients (31%)" and "28 (31)" / "40 (44%)" | 29/90 = 32.2%; 28/90 = 31.1%; 40/90 = 44.4%. The three counts differ |
Figure
Figure 1. One comparison, any SLiM-CRAB criterion vs none and progression to MM, as printed in two places and as recomputed. Orange circles and lines show values printed in the paper; blue diamonds show values recomputed from the paper's own numbers. The abstract's point estimate (0.12) lies to the left of its own interval (0.14 to 1.13), and that interval crosses the reference line at 1. The interval's own centre is 0.40, with a Wald P of 0.084. The Results-section estimate (0.10, 0.01 to 0.96) sits at the centre of its interval and implies P = 0.046, as printed. A crude risk ratio from the counts (1/62 vs 4/28), which ignores follow-up time, lands near the Results value. The horizontal axis is logarithmic, and the vertical line marks a ratio of 1.
Figure 2. Every hazard ratio in the paper, comparing the printed P value (orange circle) with the Wald P implied by its printed 95% CI (blue diamond). Within each row the circle and diamond are offset slightly up and down so that both stay visible. For four of the five, the two agree to within rounding. Only the abstract's statement crosses the P = .05 line, at .03 printed against 0.084 implied by its interval. This suggests that the paper's intervals and P values otherwise follow one convention, and that the abstract figures are an exception rather than a different method.
Why it matters
The error does not touch the trial's primary end point (uMRD 3 months after ASCT, 56 of 90) or its sustained-uMRD secondary end point. Nor, on the evidence of the body text and Fig 3B, does it reverse the reported finding: the internally consistent value (HR 0.10, 95% CI 0.01 to 0.96, P = .046) has the same direction and is nominally significant.
It matters for three reasons:
- The abstract is uninterpretable as printed. A reader who has only the abstract sees an interval (0.14 to 1.13) that includes no effect, next to a claim that SLiM status "predicted" progression and a P value of .03. Read at face value, the interval says the association is not statistically significant, which contradicts the sentence it supports.
- The body values show how fragile the result is. The finding rests on 5 progression events, 4 of them in the SLiM group. Its interval spans roughly two orders of magnitude, and its P value (.046) sits just under the threshold. The Discussion uses this result, "the only feature predicting progression to MM", to support the hypothesis that early treatment of true high-risk smoldering myeloma raises the chance of operational cure. A reader should weigh it as a hypothesis-generating signal.
- The direction convention is not stated. "Presence ... predicted progression" corresponds to an HR above 1 for SLiM present vs absent. Both printed HRs are below 1, so the reference group must be the SLiM group (absent vs present). The crude proportions (1.6% vs 14.3%) support that reading, but the paper does not state it.
Explanations considered
- Rounding. This cannot explain the result. A value that rounds to 0.12 is below 0.125, and a lower limit that rounds to 0.14 is at least 0.135, so no rounding puts 0.12 inside 0.14 to 1.13. Rounding also cannot turn an interval that crosses 1 into one compatible with P < .05.
- A different test for the P value (for example log-rank, score or exact test rather than Wald). With only 5 events these tests can differ noticeably. For instance, a Fisher exact test on the crude counts gives P = 0.031. This could account for P = .03 differing from a Wald P on its own. It cannot account for a point estimate outside its own interval, or for a different HR and CI from the Results section, which also reports P = .046 for the same comparison.
- Different model, adjustment or population. The Methods name a single model for comparisons by baseline characteristics (a stratified Cox regression). The abstract and the Results both describe any SLiM criterion vs none, progression to MM, in the ITT population, and the Results cite Fig 3B. Neither text marks one value as adjusted, landmarked or from a different time point. According to the version-of-record check, the Fig 3B panel itself prints HR 0.10 (95% CI 0.01 to 0.96), P = .046, which matches the Results.
- Direction or reference-group convention. An inverted HR (1/0.12 ≈ 8.3) does not produce the interval 0.14 to 1.13 either. Both statements share the same convention (HR < 1), so the convention does not explain why they differ.
- One-sided test. No one-sided convention puts a point estimate outside its own two-sided interval. And an interval that includes 1 at the 95% level is not compatible with a two-sided P of .03.
- A single typo in the interval. If the lower limit should read 0.014, the point estimate fits (centre 0.126). But the interval would still include 1, with a Wald P of about 0.06, so the abstract would still conflict with its own P = .03. Reconciling HR 0.12 with P = .03 would need an interval of about 0.018 to 0.81, which differs from the printed one in both limits. No single-character change makes the abstract consistent.
- Text-conversion artefact. The publisher's typeset PDF, the PMC article, the Europe PMC full-text XML and the PubMed abstract all carry the same numbers, so the discrepancy is in the version of record.
Status
As of 2026-10-23:
- Version of record. ascopubs.org returned HTTP 403. The publisher's typeset PDF (JCO 2024;42:3247-3256, 14 pages including the Appendix) was read from a copy held in an institutional repository. Its abstract and Results carry the same wording and numbers as quoted above. The "P 5" in its extracted text is the journal's font glyph for "=". The PMC article (pmc.ncbi.nlm.nih.gov/articles/PMC11404760), the Europe PMC full-text XML and the PubMed record (PMID 39038268) agree.
- Supplements. There is no separate data supplement. The Appendix (Tables A1 and A2, Fig A1) is part of the article. It gives SLiM subgroup sizes and response rates but does not restate the HR for SLiM status and progression to MM. Individual patient data needed to refit the Cox model are not published.
- Corrections and notices. The project's
known.pyreturned no Crossref update, PMC notice or Europe PMC notice. The Crossref record has noupdate-toorupdated-byrelation, the PubMed record has no CommentsCorrections entries, and the PMC page shows no correction notice. A web search for an erratum or correction citing the DOI found none. - PubPeer. It could not be checked: pubpeer.com returned HTTP 403 (Cloudflare) to both the search page and the API.
- Sources: doi:10.1200/JCO.23.02771; PMC11404760; PubMed 39038268; Crossref; Europe PMC.
Suggested correction
- In the abstract, replace "hazard ratio, 0.12; 95% CI, 0.14 to 1.13; P = .03" with the values from the fitted model. If the model is the one reported in the Results and Fig 3B, these are "hazard ratio, 0.10; 95% CI, 0.01 to 0.96; P = .046".
- State the reference group for the SLiM hazard ratio (it appears to be SLiM present as reference), so that an HR below 1 reads correctly alongside "presence ... predicted progression". Consider reporting it as SLiM present vs absent (HR ≈ 10) instead.
- Given 5 events, consider stating the number of events with the HR, and describing the association as exploratory in the abstract and Discussion.
- Separately, check the 70-month TTP, OS and TTBP intervals (each excludes its own estimate). Also check the SLiM subgroup count (31 vs 28) in the biochemical-progression paragraph, and the sustained-uMRD count at 4 years (29 in the abstract, 28 in the Discussion and Table A1, 40 in the CONTEXT box).
Recomputation script (python recompute.py.txt)
"""Recompute the SLiM-CRAB hazard ratio statements of PMC11404760
(J Clin Oncol 2024, doi:10.1200/JCO.23.02771).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
fig1.svg / fig1.png (= candidate A, dot-and-interval plot)
fig2.svg / fig2.png (= candidate C, reported vs CI-implied P for every HR)
candidates/cand_A..E.svg / .png (five candidate designs for review)
All inputs are copied from the article as archived in PubMed Central:
Abstract, Results: HR 0.12; 95% CI 0.14 to 1.13; P = .03 (any SLiM-CRAB -> MM)
Results, Time-To-Event End Points p3: HR 0.10 [95% CI 0.01 to 0.96]; P = .046 (Fig 3B)
Results, Time-To-Event End Points p0-p1: the three other HRs printed in the paper
Table 1 / Appendix Table A2: 28 of 90 patients with any SLiM-CRAB criterion (62 without)
Abstract / Appendix Table A2 footnotes: 4 of the 5 progressions to MM were in the SLiM group
Wald method: for a ratio with 95% CI (lo, hi) on the log scale,
SE = ln(hi/lo) / (2 * 1.959964), centre = sqrt(lo * hi), z = ln(centre) / SE.
"""
import math
import os
import numpy as np
from scipy.stats import norm, fisher_exact
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
# (label, location, HR, lo, hi, printed P)
HRS = [
("Any SLiM-CRAB vs none -> MM (Abstract)", "Abstract, Results", 0.12, 0.14, 1.13, 0.03),
("Any SLiM vs none -> MM (Results, Fig 3B)", "Results, TTE p3", 0.10, 0.01, 0.96, 0.046),
("uMRD vs detectable MRD -> biochem. progression", "Results, TTE p0 (Fig 2B)", 0.29, 0.13, 0.66, 0.003),
("HRCA vs none -> biochem. progression", "Results, TTE p1 (Fig A1A)", 0.42, 0.22, 0.81, 0.01),
("HRCA incl. 1q vs none -> biochem. progression", "Results, TTE p1 (Fig A1B)", 0.40, 0.18, 0.85, 0.018),
]
N_TOTAL = 90
N_SLIM = 28 # Table 1 "SLiM-CRAB criteria, No. (%) | 28 (31)"; Table A2 n = 28
N_NOSLIM = 62 # Table A2 n = 62
MM_SLIM, MM_NOSLIM = 4, 1 # abstract: "four of the five patients"; five progressed in total
# Side observations: (label, location, estimate %, lo %, hi %)
KM = [
("70-month TTP", "Abstract; Results TTE p3", 94, 84, 89),
("70-month OS", "Abstract; Results TTE p4", 92, 82, 89),
("70-month TTBP", "Results TTE p0", 59, 66, 76),
]
SUSTAINED = [("Abstract", 29), ("Discussion p1 / Table A1", 28), ("CONTEXT box", 40)]
# ---------------------------------------------------------------- helpers
def wald(hr, lo, hi):
se = math.log(hi / lo) / (2 * Z)
centre = math.sqrt(lo * hi)
p_centre = 2 * norm.sf(abs(math.log(centre)) / se)
p_hr = 2 * norm.sf(abs(math.log(hr)) / se)
return se, centre, p_centre, p_hr
def fmt_p(p):
return f"{p:.3f}" if p >= 0.001 else f"{p:.1e}"
R = []
print("=== 1. Every hazard ratio printed in the paper, checked against its own 95% CI ===")
for lab, loc, hr, lo, hi, p in HRS:
se, c, pc, ph = wald(hr, lo, hi)
R.append(dict(lab=lab, loc=loc, hr=hr, lo=lo, hi=hi, p=p, se=se, c=c, pc=pc, ph=ph))
print(f"\n{lab} [{loc}]")
print(f" printed: HR {hr}; 95% CI {lo} to {hi}; P = {p}")
print(f" HR inside its CI? {lo <= hr <= hi}; CI includes 1? {lo <= 1 <= hi}")
print(f" CI centre sqrt(lo*hi) = {c:.3f}; SE(log HR) = {se:.3f}")
print(f" Wald P from CI centre = {fmt_p(pc)}; Wald P from printed HR with that SE = {fmt_p(ph)}")
A, B = R[0], R[1]
print("\n=== 2. Can rounding explain the abstract? ===")
print(" values that round to HR 0.12 lie in [0.115, 0.125); values that round to a lower limit of 0.14 lie in [0.135, 0.145)")
print(f" the two ranges overlap: {0.125 > 0.135}")
print(f" smallest z compatible with a 95% CI that includes 1 is 1.96, i.e. P >= {2 * norm.sf(Z):.3f}")
print("\n=== 3. Single-typo hypothesis: lower limit 0.014 instead of 0.14 ===")
se, c, pc, ph = wald(0.12, 0.014, 1.13)
print(f" CI 0.014 to 1.13: centre {c:.3f}; SE {se:.3f}; Wald P from centre {pc:.3f}; from HR 0.12 {ph:.3f}; includes 1: True")
z03 = norm.isf(0.03 / 2)
se03 = abs(math.log(0.12)) / z03
print(f" for HR 0.12 and P = .03 the Wald CI would be {math.exp(math.log(0.12) - Z * se03):.3f} to {math.exp(math.log(0.12) + Z * se03):.3f}")
TYPO = dict(c=c, se=se, pc=pc, ph=ph)
print("\n=== 4. Crude check from counts (ignores follow-up time; not the Cox model) ===")
r1, r0 = MM_NOSLIM / N_NOSLIM, MM_SLIM / N_SLIM
rr = r1 / r0
se_rr = math.sqrt(1 / MM_NOSLIM - 1 / N_NOSLIM + 1 / MM_SLIM - 1 / N_SLIM)
rr_lo, rr_hi = math.exp(math.log(rr) - Z * se_rr), math.exp(math.log(rr) + Z * se_rr)
p_rr = 2 * norm.sf(abs(math.log(rr)) / se_rr)
_, p_fisher = fisher_exact([[MM_NOSLIM, N_NOSLIM - MM_NOSLIM], [MM_SLIM, N_SLIM - MM_SLIM]])
print(f" progression to MM: no SLiM {MM_NOSLIM}/{N_NOSLIM} = {100 * r1:.1f}%; SLiM {MM_SLIM}/{N_SLIM} = {100 * r0:.1f}%")
print(f" crude risk ratio (no SLiM / SLiM) = {rr:.3f}; 95% CI {rr_lo:.3f} to {rr_hi:.3f}; Wald P = {p_rr:.3f}; Fisher exact P = {p_fisher:.3f}")
print(f" N_SLIM + N_NOSLIM = {N_SLIM + N_NOSLIM} (= {N_TOTAL}); 28/90 = {100 * N_SLIM / N_TOTAL:.1f}%")
CRUDE = dict(rr=rr, lo=rr_lo, hi=rr_hi, p=p_rr)
print("\n=== 5. Side observations ===")
for lab, loc, e, lo, hi in KM:
print(f" {lab} [{loc}]: {e}% (95% CI {lo} to {hi}) -> estimate inside its CI? {lo <= e <= hi}")
print(" biochemical progression by SLiM (Results TTE p1): 12/31 = %.1f%%, 18/59 = %.1f%%; 31 + 59 = %d; 12 + 18 = %d (36 biochemical progressions reported)"
% (1200 / 31, 1800 / 59, 31 + 59, 12 + 18))
for loc, n in SUSTAINED:
print(f" sustained uMRD at 4 years [{loc}]: {n}/90 = {100 * n / 90:.1f}%")
# ---------------------------------------------------------------- figure style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
W = 7.2
MS = 9 # marker size in points (>= 8 px)
LW = 1.5 # 1.5 pt = 2 px at 96 dpi
def save(fig, name, folder=HERE):
for ext in ("svg", "png"):
fig.savefig(os.path.join(folder, f"{name}.{ext}"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def log_axis(ax, lo, hi, ticks, label):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel(label, color=INK2)
LEGEND = [
Line2D([0], [0], color=ORANGE, lw=LW, marker="o", ms=MS, label="Reported in the paper"),
Line2D([0], [0], color=BLUE, lw=LW, marker="D", ms=MS - 1, label="Recomputed from the paper's own numbers"),
]
# ---------------------------------------------------------------- candidate A: dot-and-interval
def cand_A():
rows = [
("Abstract, as printed", ORANGE, "o", A["hr"], A["lo"], A["hi"],
f"HR 0.12 (0.14 to 1.13); P = .03"),
("Abstract interval: its centre\nand Wald P", BLUE, "D", A["c"], A["lo"], A["hi"],
f"{A['c']:.2f} (0.14 to 1.13); P = {A['pc']:.3f}"),
("Results text and Fig 3B,\nas printed", ORANGE, "o", B["hr"], B["lo"], B["hi"],
"HR 0.10 (0.01 to 0.96); P = .046"),
("Results interval: its centre\nand Wald P", BLUE, "D", B["c"], B["lo"], B["hi"],
f"{B['c']:.3f} (0.01 to 0.96); P = {B['pc']:.3f}"),
("Crude risk ratio from counts\n1/62 vs 4/28 (no Cox model)", BLUE, "D", CRUDE["rr"], CRUDE["lo"], CRUDE["hi"],
f"{CRUDE['rr']:.3f} ({CRUDE['lo']:.3f} to {CRUDE['hi']:.2f}); P = {CRUDE['p']:.3f}"),
]
fig, ax = plt.subplots(figsize=(W, 3.9))
y = np.arange(len(rows))[::-1].astype(float)
y[2:] -= 0.35 # gap between abstract pair and body pair
for yi, (lab, col, mk, est, lo, hi, txt) in zip(y, rows):
ax.plot([lo, hi], [yi, yi], color=col, lw=LW, solid_capstyle="butt", zorder=2)
ax.plot(est, yi, marker=mk, ms=MS if mk == "o" else MS - 1, color=col,
mec=BG, mew=1.2, zorder=3)
ax.text(3.4, yi, txt, va="center", ha="left", fontsize=10, color=INK)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(0.93, y[0] + 0.75, "ratio = 1 (no association)", ha="right", va="bottom", fontsize=10, color=INK2)
ax.annotate("0.12 lies below its own\nlower limit (0.14)", xy=(0.12, y[0]), xytext=(0.0125, y[0] + 0.05),
fontsize=10, color=INK, va="center",
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=0.9, shrinkB=6))
ax.set_yticks(y)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
log_axis(ax, 0.004, 3.0, [0.01, 0.03, 0.1, 0.3, 1, 3], "Ratio, SLiM-CRAB absent vs present (log scale)")
ax.set_ylim(y[-1] - 0.6, y[0] + 1.25)
ax.text(3.4, y[0] + 0.75, "Estimate (95% CI); P", fontsize=10, color=INK2, va="bottom")
ax.legend(handles=LEGEND, loc="upper left", bbox_to_anchor=(0.0, -0.2), ncol=2, frameon=False, fontsize=10)
ax.set_title("Same comparison, two incompatible results: any SLiM-CRAB criterion and progression to MM",
fontsize=11, loc="left", color=INK, pad=10)
return fig
# ---------------------------------------------------------------- candidate B: two text panels
def cand_B():
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(8, 50)
ax.axis("off")
def panel(x, y, w, h, title, quote, check):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=INK2, lw=1.0))
ax.text(x + 1.5, y + h - 2.0, title, fontsize=11, fontweight="bold", va="top")
ax.add_patch(plt.Rectangle((x + 1.5, y + h - 15.8), 0.6, 10.6, color=ORANGE, lw=0))
ax.text(x + 3.2, y + h - 5.4, quote, fontsize=10, va="top", color=INK)
ax.text(x + 1.5, y + h - 17.6, "Recomputed check:", fontsize=10, va="top", color=INK2)
ax.plot(x + 2.3, y + h - 22.0, marker="D", ms=MS - 1, color=BLUE, mec=BG)
ax.text(x + 4.2, y + h - 20.2, check, fontsize=10, va="top", color=INK)
panel(1.5, 19, 42, 29, "Abstract, Results",
"“...predicted progression to MM\n(four of the five patients; hazard\nratio, 0.12; 95% CI, 0.14 to 1.13;\nP = .03).”",
f"HR 0.12 is below the lower limit 0.14\nInterval contains 1\nCI centre {A['c']:.2f}; Wald P {A['pc']:.3f}")
panel(56.5, 19, 42, 29, "Results text (Fig 3B)",
"“...predicted progression to MM\n(HR, 0.10 [95% CI, 0.01 to 0.96];\nP = .046; Fig 3B).”",
f"HR 0.10 is inside its interval\nInterval excludes 1\nCI centre {B['c']:.3f}; Wald P {B['pc']:.3f}")
ax.annotate("", xy=(55.6, 34), xytext=(44.4, 34),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.5))
ax.text(50, 35.5, "same\nanalysis", ha="center", va="bottom", fontsize=10, color=INK2)
ax.text(50, 30.5, "different\nHR, CI, P", ha="center", va="top", fontsize=10, color=INK)
ax.text(50, 10.5, "Conflict: the two statements describe the same analysis (any SLiM criterion vs none, progression to MM,\n"
"ITT population) but give different HR, CI and P. Only the Results values are internally consistent.",
ha="center", va="bottom", fontsize=10, color=INK)
return fig
# ---------------------------------------------------------------- candidate C: reported vs CI-implied P
def cand_C():
fig, ax = plt.subplots(figsize=(W, 3.4))
y = np.arange(len(R))[::-1]
for yi, r in zip(y, R):
ax.plot([r["p"], r["pc"]], [yi + 0.13, yi - 0.13], color=GRID, lw=2, zorder=1)
ax.plot(r["p"], yi + 0.13, "o", ms=MS, color=ORANGE, mec=BG, mew=1.2, zorder=3)
ax.plot(r["pc"], yi - 0.13, "D", ms=MS - 1, color=BLUE, mec=BG, mew=1.2, zorder=3)
ax.text(1.15, yi, f"{r['p']:g} vs {r['pc']:.3f}", va="center", fontsize=10)
ax.axvline(0.05, color=INK2, lw=0.8)
ax.text(0.047, y[0] + 0.6, "P = .05", ha="right", va="bottom", fontsize=10, color=INK2)
ax.text(1.15, y[0] + 0.6, "printed vs from CI", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(y)
ax.set_yticklabels([r["lab"].replace(" -> ", " → ") for r in R], fontsize=10, color=INK)
log_axis(ax, 0.001, 1.0, [0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1], "Two-sided P value (log scale)")
ax.set_ylim(-0.6, y[0] + 1.1)
leg = [Line2D([0], [0], color=ORANGE, lw=0, marker="o", ms=MS, label="P printed in the paper"),
Line2D([0], [0], color=BLUE, lw=0, marker="D", ms=MS - 1, label="Wald P implied by the printed 95% CI")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0.0, -0.22), ncol=2, frameon=False, fontsize=10)
ax.set_title("Every hazard ratio in the paper: printed P vs the P its own interval implies",
fontsize=11, loc="left", pad=10)
return fig
# ---------------------------------------------------------------- candidate D: printed HR vs CI centre, all HRs
def cand_D():
fig, ax = plt.subplots(figsize=(W, 3.4))
y = np.arange(len(R))[::-1]
for yi, r in zip(y, R):
ax.plot([r["lo"], r["hi"]], [yi, yi], color=INK2, lw=LW, alpha=0.55, zorder=1, solid_capstyle="butt")
ax.plot(r["c"], yi - 0.2, "D", ms=MS - 1, color=BLUE, mec=BG, mew=1.2, zorder=3)
ax.plot(r["hr"], yi + 0.2, "o", ms=MS, color=ORANGE, mec=BG, mew=1.2, zorder=4)
ax.text(2.4, yi, f"{r['hr']:.2f} vs {r['c']:.3f}", va="center", fontsize=10)
ax.axvline(1, color=INK2, lw=0.8)
ax.text(2.4, y[0] + 0.6, "printed vs centre", fontsize=10, color=INK2, va="bottom")
ax.set_yticks(y)
ax.set_yticklabels([r["lab"].replace(" -> ", " → ") for r in R], fontsize=10, color=INK)
log_axis(ax, 0.006, 2.0, [0.01, 0.03, 0.1, 0.3, 1], "Hazard ratio (log scale); gray line = printed 95% CI")
ax.set_ylim(-0.6, y[0] + 1.1)
leg = [Line2D([0], [0], color=ORANGE, lw=0, marker="o", ms=MS, label="HR printed in the paper"),
Line2D([0], [0], color=BLUE, lw=0, marker="D", ms=MS - 1, label="Centre of the printed CI, sqrt(lo × hi)"),
Line2D([0], [0], color=INK2, alpha=0.55, lw=LW, label="Printed 95% CI")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0.0, -0.22), ncol=3, frameon=False, fontsize=10)
ax.set_title("Every hazard ratio in the paper: only the abstract's sits off its interval's centre",
fontsize=11, loc="left", pad=10)
return fig
# ---------------------------------------------------------------- candidate E: consistency checklist
def cand_E():
checks = [
("Printed HR lies inside its 95% CI", ("No: 0.12 < 0.14", False), ("Yes: 0.01 ≤ 0.10 ≤ 0.96", True)),
("CI excludes 1, as P < .05 requires", ("No: interval contains 1", False), ("Yes: upper limit 0.96", True)),
("CI centre sqrt(lo × hi) ≈ printed HR", (f"No: centre {A['c']:.2f}", False), (f"Yes: centre {B['c']:.3f}", True)),
("Wald P from CI ≈ printed P", (f"No: {A['pc']:.3f} vs .03", False), (f"Yes: {B['pc']:.3f} vs .046", True)),
]
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 50)
ax.axis("off")
cx = [2, 38, 72]
for x, t1, t2 in ((cx[1], "Abstract", "HR 0.12 (0.14 to 1.13); P = .03"),
(cx[2], "Results text / Fig 3B", "HR 0.10 (0.01 to 0.96); P = .046")):
ax.text(x, 46, t1, fontsize=10, va="top", fontweight="bold")
ax.text(x, 42.6, t2, fontsize=10, va="top")
ax.add_patch(plt.Rectangle((x - 1.2, 38.6), 0.6, 7.4, color=ORANGE, lw=0))
ax.text(cx[0], 46, "Check, recomputed from the\npaper's own numbers", fontsize=10, va="top", color=INK2)
for i, (lab, a, b) in enumerate(checks):
yy = 32 - i * 8
ax.plot([1, 99], [yy + 4.4, yy + 4.4], color=GRID, lw=1)
ax.text(cx[0], yy, lab, fontsize=10, va="center")
for x, (txt, ok) in ((cx[1], a), (cx[2], b)):
ax.plot(x + 0.6, yy, marker="D", ms=MS - 2, color=BLUE, mec=BG)
ax.text(x + 2.2, yy, ("✓ " if ok else "✗ ") + txt, fontsize=10, va="center",
fontweight="normal" if ok else "bold")
ax.text(cx[0], 1.5, "Orange bar: values printed in the paper. Blue diamond: check recomputed from them (Wald method, log scale).",
fontsize=10, color=INK2, va="center")
return fig
for name, fn in zip("ABCDE", (cand_A, cand_B, cand_C, cand_D, cand_E)):
save(fn(), f"cand_{name}", CAND)
save(cand_A(), "fig1")
save(cand_C(), "fig2")
print("\nwrote fig1.*, fig2.* and candidates/cand_A..E.* in", HERE)
Reported two-tailed p = 0.046 for t(18) = 1.75; that statistic gives p = 0.097
J Neurosci (2021) · doi:10.1523/JNEUROSCI.3216-18.2021 · PMC7984597
Summary
The paper reports that a high-fat diet reduced surface expression of the insulin receptor β subunit (IRβ) in the nucleus accumbens (NAc) by 21.7%, tested with a "two-tailed unpaired t test: t(18) = 1.75, p = 0.046". For t = 1.75 with 18 degrees of freedom, the two-tailed p is 0.097, and no t that rounds to 1.75 gives less than 0.096. The printed 0.046 is close to the one-tailed value (0.049), but the paper calls this test two-tailed in the Results, in the Fig. 6 legend and in the Methods. If t and df are correct, the reduction is not significant at the conventional 0.05 threshold that the figure legends use ("p < 0.05"), yet the abstract reports it as a finding and the Discussion uses it as a possible mechanism for the loss of insulin's effect after obesity.
What the paper reports
Results, section "Diet-induced obesity blunts insulin receptor-mediated increases in excitatory transmission and reduces NAc insulin receptor surface expression", second paragraph:
One potential explanation for the loss of insulin-induced increases in excitatory transmission is a reduction in NAc insulin receptor expression.
In NAc tissue from chow and high-fat groups, we found a 21.7% (±7.6%) reduction in IRβ surface expression in high-fat versus chow-fed groups (Fig. 6D; two-tailed unpaired t test: t(18) = 1.75, p = 0.046) without any changes in total IRβ (Fig. 6E).
This suggests that reductions in NAc insulin receptor expression may contribute to the loss of insulin-induced increases in eEPSC amplitude in the high-fat group.
Fig. 6 legend, title and last sentence:
High-fat diet-induced obesity results in a loss of insulin-induced increases in excitatory transmission and a reduction in NAc IRβ surface expression.
Statistical differences were determined by two-tailed unpaired t tests (A,B: **p < 0.001), two-way repeated-measures ANOVA comparing BL and treatment conditions (C: *chow group, main effect of treatment; #high-fat group, main effect of treatment, p < 0.05), and two-tailed unpaired t test (D: p = 0.046).
Materials and Methods, "Experimental design and statistical analysis":
Two-tailed t tests, one-way or two-way repeated measures ANOVAs, and Sidak's post hoc multiple comparisons tests were conducted using Prism 6-8 software (GraphPad).
Fig. 1 legend:
Data in all figures are shown as the mean ± SEM.
Abstract:
Furthermore, obesity resulted in a loss of insulin receptor-mediated increases in excitatory transmission and a reduction in NAc insulin receptor surface expression, while preserving reductions in transmission mediated by IGF receptors.
Introduction, third paragraph:
Finally, diet-induced obesity resulted in a loss of insulin-induced increases in NAc excitatory transmission and a reduction in NAc insulin receptor surface expression.
Discussion, "Loss of insulin-receptor mediated effects following obesity":
The loss of insulin-induced increases may be due in part to modest reductions in NAc insulin receptor expression, as surface expression of IRβ was reduced following high-fat diet (Fig. 6D).
A reduction in insulin receptor expression is consistent with the development of insulin resistance in the face of chronic elevations in circulating insulin resulting from obesity
What the paper's own numbers show
Method. The paper gives the test (unpaired, two-tailed), the statistic (t = 1.75) and the degrees of freedom (18). From these alone, the two-tailed p is 2·P(T₁₈ ≥ 1.75), computed with scipy.stats.t. To allow for rounding, I repeated the calculation for every t that rounds to 1.75 (1.745 to 1.755). I also computed the |t| that a two-tailed p of 0.046 would need, and the lowest two-tailed p that t = 1.75 can give at any df (the normal limit). As a check on the method, I recomputed the p values of all 36 other t and F statistics printed in the Results. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Two-tailed p, t(18) = 1.75 | Results §"Diet-induced obesity…", para 2; Fig. 6 legend (D) | p = 0.046 | 0.0971 (0.0963 to 0.0980 for t of 1.745 to 1.755) |
| One-tailed p, t(18) = 1.75 | same | not reported | 0.0486 (0.0481 to 0.0490) |
| Lowest two-tailed p for t = 1.75 at any df | derived | — | 0.0801 (df → ∞) |
| Two-tailed p for t = 1.75 at df = 9 (smallest Welch df if the 20 samples were split 10 + 10; unequal splits give lower df and higher p) | derived | — | 0.1140 |
| Absolute t needed for two-tailed p = 0.046 at df = 18 | derived | t = 1.75 | 2.143 |
| Absolute t needed for two-tailed p = 0.05 at df = 18 (critical value, α = 0.05) | derived | — | 2.101 |
| t needed for one-tailed p = 0.046 at df = 18 | derived | — | 1.780 |
| Reported p ÷ recomputed two-tailed p | derived | — | 0.474 |
| SE of the difference implied by 21.7 / 1.75 | Results | "±7.6%" | 12.40 percentage points |
| t if ±7.6 were the SE of the difference (21.7 / 7.6) | Results | t = 1.75 | 2.855 (two-tailed p = 0.0105) |
| t if both group SEMs were 7.6 | Results | t = 1.75 | 2.019 (two-tailed p = 0.0586) |
| Same comparison in bioRxiv v1 (2019) and v2 (2020) | Results, both versions | "two-tailed unpaired t-test t18 =1.75, p=0.05" (v2: "t18=1.75") | 0.0971 |
| Other 34 t and F statistics in Results | Results, Figs. 1–6 | various | all agree with the printed p after rounding |
| Fig. 3J, mEPSC amplitude with HNMPA, t(5) = 7.75 | Results, "Insulin-induced changes…" | p = 0.006 | 0.00057 (p = 0.006 would need t = 4.57) |
| Fig. 5F, (–)-naloxone alone, t(9) = 0.53 | Results, "Insulin-induced increases… disinhibition" | p = 0.53 | 0.61 (p = 0.53 would need t = 0.65) |
Rounding of t cannot close the gap: the whole rounding range gives two-tailed p of 0.096 to 0.098. Nor can any choice of degrees of freedom: with t = 1.75 the two-tailed p never falls below 0.080. The reported 0.046 is not exactly the one-tailed value either (0.049; a one-tailed 0.046 would need t = 1.78), but it is far closer to it than to the two-tailed value. Of the paper's other 36 t and F statistics, 34 match their printed p after rounding. The remaining two (Fig. 3J and Fig. 5F) have printed p values that differ from the recomputed ones, but neither difference changes whether the result is significant.
The "±7.6%" cannot settle which of t, p or the test description is wrong. The paper does not say what ±7.6% is. If it were the SE of the group difference, t would be 2.86, which does not match the printed t either. If it is the SEM of one group, expressed as a percentage of the chow mean, the other group's SEM would have to be 9.80 for t = 1.75, which is plausible. The group sizes (they must total 20 for df = 18) and the group means are not given, and there are no supplementary data.
Figure
Figure 1. p values for the high-fat vs chow comparison of NAc IRβ surface expression, on a log scale. The vertical line marks 0.05. Orange: the values printed for this test, p = 0.046 in the published article (filled) and p = 0.05 in both bioRxiv versions (open), all with the same t18 = 1.75. Blue: values recomputed by recompute.py.txt from t = 1.75 and df = 18. These are the two-tailed p that the paper's stated test gives (filled), the one-tailed p (open), and the lowest two-tailed p that t = 1.75 can give at any degrees of freedom (open). The ranges in brackets cover every t that rounds to 1.75; they are narrower than the markers.
Figure 2. Where the two p values come from. The curve is the Student t density with 18 degrees of freedom. The orange solid line is the reported t = 1.75, and the orange dashed line is its mirror, −1.75. Each shaded tail beyond ±1.75 holds 0.049. The right tail alone is the one-tailed p, and both tails together are the two-tailed p, 0.097. The blue dotted line marks |t| = 2.14, the value a two-tailed p of 0.046 would need. All areas and positions come from recompute.py.txt.
Why it matters
- A finding in the abstract rests on this test. The abstract and the Introduction both list "a reduction in NAc insulin receptor surface expression" among the results of diet-induced obesity. The Results section heading and the Fig. 6 title say the same. With the stated test, t(18) = 1.75 gives p = 0.097, which does not meet the p < 0.05 criterion the figure legends use. Elsewhere the paper treats p values up to 0.04 as significant and those of 0.14 or more as not.
- It is offered as a mechanism. The Results say the reduction "may contribute to" the loss of insulin-induced increases in eEPSC amplitude after high-fat diet. The Discussion adds that it is "consistent with the development of insulin resistance". Both statements treat the reduction as established.
- The paper's other conclusions do not depend on it. The bidirectional effects of insulin receptor and IGF receptor activation, the opioid receptor-dependent disinhibition, and the loss of insulin-induced eEPSC increases after high-fat diet (Fig. 6C) all rest on separate experiments and statistics, which recompute correctly. The Discussion itself frames the receptor reduction as "modest" and as one possible cause among others ("concomitant reductions in signaling downstream of the receptor could also contribute").
- The direction is unchanged. The data still show a 21.7% lower mean in the high-fat group. Only the claim that the difference is statistically significant fails, and with it the strength of the evidence for that part of the mechanism.
Explanations considered
- Rounding of t. Does not explain it. Every t from 1.745 to 1.755 gives a two-tailed p between 0.0963 and 0.0980.
- A different degrees of freedom, or a Welch test. Does not explain it. A Welch correction can only lower df below 18, and that raises p (0.114 at df = 9, the smallest Welch df if the 20 samples were split 10 + 10; unequal splits give lower df and higher p). Even at infinite df, t = 1.75 gives a two-tailed p of 0.080.
- A one-tailed test labelled as two-tailed. This fits the numbers best: the one-tailed p is 0.049. The Results text does state a directional expectation ("One potential explanation … is a reduction"). But the paper calls this test two-tailed in three places, and a one-tailed 0.046 would need t = 1.78, so even this reading leaves a small mismatch. If the test was one-tailed, the result is significant at 0.05 only under a one-sided hypothesis that the paper never states.
- A typo in t rather than in p. Possible. A two-tailed p of 0.046 at df = 18 would need |t| = 2.14. Both bioRxiv versions print the same t18 = 1.75 with "p=0.05", and the published version changes only the p, to 0.046. So t = 1.75 appears in all three versions, while p differs between them. Without the group data this cannot be settled.
- The ±7.6% recovers the test. It does not. Read as the SE of the difference, it implies t = 2.86, which matches neither the printed t nor the printed p. Read as one group's SEM, it is compatible with t = 1.75. The paper does not say which.
- A conversion or typesetting artifact. Ruled out. The PMC text and the Europe PMC JATS XML both give "t(18) = 1.75, p = 0.046", with 18 as a subscript and ordinary decimal points, in running text rather than a table. The Fig. 6 legend repeats p = 0.046 independently. The statistic has no sign, so a lost minus could not matter for a two-tailed test.
- A general pattern of p-value errors. Not the case. The other 34 t and F statistics all recompute to their printed p. The two other mismatches (Fig. 3J and Fig. 5F, both also in bioRxiv v2; Fig. 5F's also in v1, as Figure 5D) do not change significance.
Status
- Version of record. The publisher's site (jneurosci.org, reached through the DOI) returned HTTP 403 to automated requests on 2026-10-24. The publisher-deposited full text archived in PMC (PMC7984597) and the Europe PMC JATS XML were checked instead, and both carry the same Results sentence and Fig. 6 legend. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Preprint. bioRxiv doi:10.1101/517797, v1 (2019-01-11) and v2 (2020-09-01). The JATS XML of v1 gives "two-tailed unpaired t-test t18 =1.75, p=0.05" for the same comparison, and v2 gives "t18=1.75, p=0.05"; both have the same "21.7% (± 7.6%)" and also call the test two-tailed.
- Supplements. The PMC record lists no supplementary material other than the article PDF. The Europe PMC XML has no supplementary-material element, and the Europe PMC record lists none. No group-level data for Fig. 6D are available, so the t test cannot be rebuilt independently.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by relation, and the PMC page shows no correction notice. PubPeer's search API returned HTTP 403 to automated requests, and a web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC7984597; Europe PMC full-text XML for PMC7984597 (PMID 33514676); bioRxiv JATS XML for 10.1101/517797 v1 and v2; Crossref; Europe PMC; PubPeer; https://doi.org/10.1523/JNEUROSCI.3216-18.2021.
Suggested correction
The authors should check the original analysis of Fig. 6D and correct whichever element is wrong. There are three possibilities:
- If t(18) = 1.75 is correct and the test was two-tailed, the Results should read "t(18) = 1.75, p = 0.097", and the Fig. 6 legend "(D: p = 0.097)". The abstract, the Introduction summary, the Results section heading, the Fig. 6 title and the Discussion should then describe the reduction in IRβ surface expression as a non-significant trend rather than an established effect.
- If the test was one-tailed, the Results, the Fig. 6 legend and the Methods ("Two-tailed t tests") should say so, give the one-tailed p (0.049 for t = 1.75), and state the directional hypothesis in advance.
- If t was misreported, the correct t and p should be given.
In every case, the group sizes for Fig. 6D and the meaning of "±7.6%" (SEM of which quantity) should be stated. Separately, the p values for Fig. 3J (t(5) = 7.75 gives p = 0.0006, not 0.006) and Fig. 5F (t(9) = 0.53 gives p = 0.61, not 0.53) could be corrected at the same time.
Recomputation script (python recompute.py.txt)
"""Recompute the IRbeta surface-expression test in PMC7984597
(J Neurosci 2021, doi:10.1523/JNEUROSCI.3216-18.2021).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the text of the paper as archived in PMC (Results,
section "Diet-induced obesity blunts insulin receptor-mediated increases in
excitatory transmission and reduces NAc insulin receptor surface expression",
second paragraph, and the Fig. 6 legend), and from the two bioRxiv versions of
the same work (doi:10.1101/517797, v1 2019-01-11 and v2 2020-09-01), whose
JATS XML give the same comparison as "t18 =1.75, p=0.05".
"""
import os
import numpy as np
from scipy import stats, optimize
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
T, DF = 1.75, 18 # "two-tailed unpaired t test: t(18) = 1.75"
P_REPORTED = 0.046 # "p = 0.046" (Results and Fig. 6 legend)
P_PREPRINT = 0.05 # "p=0.05" (bioRxiv v1 and v2, same t18 = 1.75)
DIFF, PM = 21.7, 7.6 # "a 21.7% (±7.6%) reduction"
T_LO, T_HI = 1.745, 1.755 # values of t that round to 1.75
# Other tests in the paper whose printed p differs from the recomputed one
# beyond rounding (each as printed in Results).
SECONDARY = [
("Fig. 3J, mEPSC amplitude with HNMPA", "two-tailed paired t", ("t", 5), 7.75, "p = 0.006"),
("Fig. 5F, (-)-naloxone alone", "two-tailed paired t", ("t", 9), 0.53, "p = 0.53"),
]
# All other t and F statistics printed in Results (statistic, df, printed p).
OTHERS = [("t", (26,), 3.65, "0.001"), ("t", (26,), 6.82, "<0.0001"), ("t", (4,), 3.32, "0.03"),
("t", (5,), 0.082, "0.94"), ("t", (5,), 0.465, "0.66"), ("t", (5,), 1.66, "0.16"),
("t", (5,), 3.45, "0.02"), ("t", (6,), 0.43, "0.68"), ("t", (6,), 0.94, "0.38"),
("t", (6,), 3.90, "0.008"), ("t", (6,), 4.67, "0.003"), ("t", (7,), 2.55, "0.04"),
("F", (1, 10), 2.58, "0.14"), ("F", (1, 3), 0.004, "0.95"), ("F", (1, 3), 1.617, "0.29"),
("F", (1, 4), 19.56, "0.01"), ("F", (1, 4), 2.7, "0.17"), ("F", (1, 4), 43.50, "0.003"),
("F", (1, 5), 0.093, "0.77"), ("F", (1, 5), 0.82, "0.41"), ("F", (1, 5), 2.86, "0.15"),
("F", (1, 5), 7.55, "0.04"), ("F", (1, 5), 8.58, "0.03"), ("F", (1, 6), 21.41, "0.003"),
("F", (1, 7), 10.55, "0.01"), ("F", (1, 8), 19.2, "0.002"), ("F", (1, 8), 24.0, "0.001"),
("F", (2, 10), 8.74, "0.01"), ("F", (2, 16), 12.5, "0.0005"), ("F", (2, 6), 33.2, "<0.001"), ("F", (2, 6), 6.61, "0.03"),
("F", (2, 8), 0.074, "0.93"), ("F", (2, 8), 5.17, "0.04"), ("F", (8, 48), 2.91, "0.01")]
def p_two(t, df):
return 2 * stats.t.sf(abs(t), df)
def p_one(t, df):
return stats.t.sf(t, df)
def pval(kind, df, v):
return p_two(v, df[0]) if kind == "t" else stats.f.sf(v, *df)
# ------------------------------------------------------------------ focal test
p2, p1 = p_two(T, DF), p_one(T, DF)
p2_lo, p2_hi = p_two(T_HI, DF), p_two(T_LO, DF)
p1_lo, p1_hi = p_one(T_HI, DF), p_one(T_LO, DF)
p2_min = 2 * stats.norm.sf(T) # limit as df -> infinity
t_need2 = stats.t.isf(P_REPORTED / 2, DF) # |t| for two-tailed 0.046
t_need1 = stats.t.isf(P_REPORTED, DF) # t for one-tailed 0.046
t_crit05 = stats.t.isf(0.025, DF) # two-tailed alpha = 0.05
t_crit10 = stats.t.isf(0.05, DF) # two-tailed alpha = 0.10
t_need2_pre = stats.t.isf(P_PREPRINT / 2, DF) # |t| for two-tailed 0.05
# degrees of freedom at which a one-tailed p for t = 1.75 equals 0.046
df_one = optimize.brentq(lambda d: p_one(T, d) - P_REPORTED, 2, 1e6)
# Welch-type df can only be smaller than 18: p rises
p2_df9 = p_two(T, 9)
print("Focal test: IRbeta surface expression, high-fat vs chow (Fig. 6D)")
print(f" reported: t({DF}) = {T}, p = {P_REPORTED} (labelled two-tailed); preprint: p = {P_PREPRINT}")
print(f" two-tailed p for t({DF}) = {T}: {p2:.4f} (t 1.745 to 1.755: {p2_lo:.4f} to {p2_hi:.4f})")
print(f" one-tailed p for t({DF}) = {T}: {p1:.4f} (t 1.745 to 1.755: {p1_lo:.4f} to {p1_hi:.4f})")
print(f" lowest two-tailed p for t = {T} at any df (normal limit): {p2_min:.4f}")
print(f" two-tailed p for t = {T} at df = 9 (smallest Welch df if groups were 10/10): {p2_df9:.4f}")
print(f" |t| needed for two-tailed p = {P_REPORTED} at df {DF}: {t_need2:.3f}")
print(f" |t| needed for two-tailed p = {P_PREPRINT} at df {DF}: {t_need2_pre:.3f}")
print(f" t needed for one-tailed p = {P_REPORTED} at df {DF}: {t_need1:.3f}")
print(f" critical |t| at df {DF}: alpha 0.05 two-tailed {t_crit05:.3f}; alpha 0.10 two-tailed {t_crit10:.3f}")
print(f" df at which one-tailed p for t = {T} equals {P_REPORTED}: {df_one:.1f}")
print(f" ratio reported p / recomputed two-tailed p: {P_REPORTED / p2:.3f}")
# ------------------------------------------------------------------ the +/- 7.6%
se_impl = DIFF / T
t_if_se = DIFF / PM
t_if_both = DIFF / (PM * np.sqrt(2))
sem_other = np.sqrt(se_impl ** 2 - PM ** 2)
print("\nThe '21.7% (±7.6%)' figure")
print(f" SE of the difference implied by 21.7 / t: {se_impl:.2f} percentage points")
print(f" if ±7.6 were the SE of the difference: t = {t_if_se:.3f}, two-tailed p = {p_two(t_if_se, DF):.4f}")
print(f" if both group SEMs were 7.6: t = {t_if_both:.3f}, two-tailed p = {p_two(t_if_both, DF):.4f}")
print(f" if ±7.6 is one group's SEM, the other group's SEM for t = 1.75 would be {sem_other:.2f}")
# ------------------------------------------------------------------ other tests
print("\nOther tests whose printed p differs beyond rounding")
sec_rows = []
for where, test, spec, v, printed in SECONDARY:
kind, *df = spec
p = pval(kind, df, v)
lab = f"t({df[0]}) = {v}" if kind == "t" else f"F({df[0]},{df[1]}) = {v}"
sec_rows.append((where, lab, printed, p))
print(f" {where}: {lab}, printed {printed}, recomputed p = {p:.2g}")
pp = float(printed.split("=")[1])
if kind == "t":
print(f" |t| that would give two-tailed {printed} at df {df[0]}: {stats.t.isf(pp / 2, df[0]):.2f}")
print(f"\nAll other {len(OTHERS)} t/F statistics in Results (printed p checked against the p range")
print(" for the statistic's own rounding interval, allowing rounding or truncation of p)")
def agrees(kind, df, v, printed, vtxt):
dec_v = len(vtxt.split(".")[1]) if "." in vtxt else 0
h = 0.5 * 10 ** (-dec_v)
ps = [pval(kind, df, max(v - h, 0)), pval(kind, df, v + h)]
lo, hi = min(ps), max(ps)
if printed.startswith("<"):
return lo < float(printed[1:])
q = float(printed)
step = 10 ** (-len(printed.split(".")[1]))
# rounding: [q - step/2, q + step/2); truncation: [q, q + step)
return hi >= q - step / 2 and lo < q + step
n_ok = 0
for kind, df, v, printed in OTHERS:
p = pval(kind, df, v)
vtxt = repr(v)
ok = agrees(kind, df, v, printed, vtxt)
n_ok += ok
lab = f"t({df[0]})={v}" if kind == "t" else f"F({df[0]},{df[1]})={v}"
print(f" {lab:16s} printed p = {printed:8s} recomputed p = {p:.4g} {'agrees' if ok else 'DIFFERS'}")
print(f" agree with the printed p: {n_ok} of {len(OTHERS)}")
print(" same check on the focal test and the two tests listed above:")
for (kind, df, v, printed, name) in [("t", (DF,), T, str(P_REPORTED), "focal t(18) = 1.75, two-tailed"),
("t", (5,), 7.75, "0.006", "t(5) = 7.75"),
("t", (9,), 0.53, "0.53", "t(9) = 0.53")]:
print(f" {name}: {'agrees' if agrees(kind, df, v, printed, repr(v)) else 'DIFFERS'}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"axes.edgecolor": INK2, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2, "text.color": INK,
"svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: p values on one log axis
fig, ax = plt.subplots(figsize=(7.2, 4.1))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
rows = [ # y, label, value, (lo, hi) or None, colour, filled, annotation
(4, "Reported, published version\n(Results; Fig. 6 legend)", P_REPORTED, None, ORANGE, True,
f"p = {P_REPORTED}"),
(3, "Reported, preprint v1 and v2\n(same t18 = 1.75)", P_PREPRINT, None, ORANGE, False,
f"p = {P_PREPRINT}"),
(2, "Recomputed two-tailed\n(the test the paper names)", p2, (p2_lo, p2_hi), BLUE, True,
f"p = {p2:.3f}\n({p2_lo:.4f} to {p2_hi:.4f})"),
(1, "Recomputed one-tailed\n(not stated in the paper)", p1, (p1_lo, p1_hi), BLUE, False,
f"p = {p1:.3f}\n({p1_lo:.4f} to {p1_hi:.4f})"),
(0, "Lowest two-tailed p for t = 1.75\nat any degrees of freedom", p2_min, None, BLUE, False,
f"p = {p2_min:.3f}"),
]
ax.axvline(0.05, color=INK2, lw=1, zorder=1)
ax.text(0.0495, -0.45, "0.05 ", color=INK2, fontsize=10, va="center", ha="right")
for y, lab, v, ci, col, filled, ann in rows:
if ci:
ax.plot(ci, [y, y], color=col, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([v], [y], "o", ms=9, mew=2, color=col, mfc=col if filled else BG, zorder=3)
ax.text(v * 1.045, y, ann, color=INK, fontsize=10, va="center", ha="left")
ax.set_yticks([r[0] for r in rows])
ax.set_yticklabels([r[1] for r in rows], fontsize=10, color=INK)
ax.set_xscale("log")
ax.set_xlim(0.03, 0.2)
ax.set_ylim(-0.6, 4.9)
ticks = [0.03, 0.04, 0.05, 0.07, 0.1, 0.15, 0.2]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.set_xlabel("p value, high-fat vs chow IRβ surface expression (log scale)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
from matplotlib.lines import Line2D # noqa: E402
fig.legend(handles=[Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Printed in the paper or its preprint"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE,
label="Recomputed from t(18) = 1.75")],
loc="upper center", bbox_to_anchor=(0.6, 0.995), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.33, right=0.97, top=0.88, bottom=0.14)
save(fig, "fig1")
# Figure 2: the t(18) distribution and the tails that make each p
fig, ax = plt.subplots(figsize=(7.2, 3.4))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
x = np.linspace(-4, 4, 1601)
y = stats.t.pdf(x, DF)
ax.plot(x, y, color=INK2, lw=2)
right = x >= T
left = x <= -T
ax.fill_between(x[right], 0, y[right], color=BLUE, alpha=0.55, lw=0)
ax.fill_between(x[left], 0, y[left], color=BLUE, alpha=0.25, lw=0)
ax.axvline(T, color=ORANGE, lw=2)
ax.axvline(-T, color=ORANGE, lw=2, ls=(0, (3, 2)))
ax.axvline(t_need2, color=BLUE, lw=2, ls=(0, (1, 2)))
ax.text(T - 0.06, 0.385, f"reported\nt = {T}", color=INK, fontsize=10, va="top", ha="right")
ax.text(-T + 0.06, 0.385, f"−{T}", color=INK, fontsize=10, va="top", ha="left")
ax.text(t_need2 + 0.06, 0.25, f"|t| = {t_need2:.2f}\nneeded for a\ntwo-tailed p = {P_REPORTED}",
color=INK, fontsize=10, va="top")
ax.annotate(f"right tail = {p1:.3f}\n(one-tailed p)", xy=(2.05, 0.03), xytext=(2.55, 0.115),
fontsize=10, color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.annotate(f"left tail = {p1:.3f}", xy=(-2.05, 0.03), xytext=(-3.95, 0.115),
fontsize=10, color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.text(-3.95, 0.30, f"two-tailed p\n= both tails = {p2:.3f}\n(paper: {P_REPORTED})",
ha="left", va="top", fontsize=10.5, color=INK)
ax.set_xlim(-4, 4)
ax.set_ylim(0, 0.42)
ax.set_xlabel("t statistic, Student t distribution with 18 degrees of freedom", fontsize=10)
ax.set_ylabel("density", fontsize=10)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png")
Abstract calls a subgroup sex difference significant (P < .001); the Results report P = .28
JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.17149 · PMC12188365
Summary
The abstract says the female share among the additional eligible people with lung cancer under the 20-year smoking-duration (IPN-Potter) criterion, 70 of 128 (55%), differed significantly from USPSTF 2021 (55% vs 48%; P < .001). The Results section reports the same comparison as not significant: 55% vs 50% [370 of 747]; P = .28. A Pearson χ² test on the paper's own counts (70/128 vs 370/747) gives P = .281, which reproduces the Results. The abstract's "48%" equals 1847/3840, the female share of all USPSTF 2021 eligibles, and "55% vs 48%; P < .001" is word for word the result of a different comparison, the one for all additional eligibles. The error is confined to one abstract sentence. It turns a non-significant subgroup result into a significant one in the most-read part of the paper. The headline conclusions rest on other, correctly reported comparisons.
What the paper reports
Abstract, Results paragraph (the sentence in question):
The additional eligible individuals with lung cancer in IPN programs who were female included 70 of 128 (55%) by IPN-Potter criteria, 89 of 208 (43%) by IPN-ACS criteria, and 143 of 304 (47%) by IPN-Potter-ACS criteria, with statistically significant differences between the IPN-Potter criteria and USPSTF 2021 criteria (55% vs 48%; P < .001).
Results, section "Screening Eligibility in Those With Lung Cancer":
The additional eligible individuals with lung cancer in IPN (IPN-Potter Extra, IPN-ACS Extra, IPN-Potter-ACS Extra) who were female included 70 of 128 (55%) in Potter criteria, 89 of 208 (43%) in ACS criteria, and 143 of 304 (47%) in Potter-ACS criteria, respectively; not significantly different between IPN-Potter Extra and IPN-USPSTF 2021 (55% vs 50% [370 of 747]; P = .28) (Figure 2).
The same Results section, preceding paragraph, gives the comparator group:
compared with 50% (370 of 747) in IPN-USPSTF 2021
The comparison for all additional eligibles, which carries the same "55% vs 48%; P < .001", appears in the sentence just before the one in question in the abstract:
There were significant differences in the number of additional eligible individuals who were female (55% vs 48%; P < .001) and Black (35% vs 22%; P < .001) between the IPN-Potter and USPSTF 2021 criteria.
and in Results, section "Additional Persons Eligible":
with significant differences in the numbers of females included between the IPN-Potter Extra and IPN-USPSTF 2021 criteria (55% vs 48%; P < .001) (Figure 2)
The test, from Methods, "Statistical Analysis" (the abstract also names the χ2 test):
using an α = .05 to determine statistical significance without adjusting for multiple comparisons. Tests were 2-sided.
What the paper's own numbers show
Method. I took the counts printed in the text and confirmed them against eTables 1 and 2 of Supplement 1. eTable 2 gives Female 370 (49.53%) of 747 for IPN-USPSTF2021 and 70 (54.69%) of 128 for IPN-Potter Extra. eTable 1 gives 1847 (48.1%) of 3840 and 606 (54.94%) of 1103. I then ran the 2-sided Pearson χ² test that Methods describes on each 2 × 2 table. As checks I also ran the Yates-corrected χ², Fisher's exact test and a pooled two-proportion z test. Proportions carry 95% Wilson intervals. recompute.py.txt prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Female, IPN-Potter Extra with lung cancer | Abstract; Results | 70 of 128 (55%) | 54.7% (95% CI 46.1–63.0) |
| Female, IPN-USPSTF 2021 with lung cancer | Results | 50% (370 of 747) | 49.5% (95% CI 46.0–53.1) |
| Comparator % in the abstract's lung-cancer sentence | Abstract | 48% | 48% is 1847/3840 = 48.1%, the female share of all IPN-USPSTF 2021 eligibles, not of those with lung cancer (49.5%) |
| P, 70/128 vs 370/747 | Results | P = .28 | P = .281 (χ² = 1.16; z = 1.08). Yates P = .326; Fisher P = .294 |
| P, the same comparison | Abstract | P < .001 | P = .281 |
| P, 70/128 vs 1847/3840 (if the 48% group was meant) | Abstract, alternative reading | P < .001 | P = .142 (Yates .168; Fisher .151) |
| Female, all additional eligibles: 606/1103 vs 1847/3840 | Abstract, previous sentence; Results "Additional Persons Eligible" | 55% vs 48%; P < .001 | 54.9% vs 48.1%; P = 6.2 × 10⁻⁵ (consistent) |
| Black, lung-cancer subgroup: 50/128 vs 162/747 | Abstract; Results | 39% vs 22%; P < .001 | 39.1% vs 21.7%; P = 2.2 × 10⁻⁵ (consistent) |
| Black, all additional eligibles: 382/1103 vs 777/3840 | Abstract (35% vs 22%); Results (35% vs 20%) | P < .001 | 34.6% vs 20.2%; P = 2.6 × 10⁻²³. The abstract's 22% matches 162/747 (21.7%), not 777/3840 |
The Results value reproduces to the second decimal place from the printed counts with the test the Methods name. With these counts, P < .001 against 70/128 would require the USPSTF 2021 lung-cancer group to be at most 292/747 (39.1%) female, not 370/747.
Figure
Figure 1. The two passages, shortened to their key clauses (full quotes above). Both describe the female share among additional eligible people with lung cancer under IPN-Potter (70 of 128) compared with IPN-USPSTF 2021. The values the paper reports are in orange. The blue box gives values recomputed from the paper's own counts with the 2-sided Pearson χ² test named in Methods. The counts reproduce the Results (P = .28), not the abstract (P < .001). The abstract's comparator (48%) is the female share of all USPSTF 2021 eligibles. "55% vs 48%; P < .001" is the correct result for all additional eligibles (606/1103 vs 1847/3840), reported in the sentence before.
Figure 2. Female share in each group. Orange diamonds are the rounded percentages the paper reports. Blue dots and lines are proportions recomputed from the printed counts, with 95% Wilson intervals. The 128 additional eligibles with lung cancer form a small group, and their interval (46.1–63.0%) overlaps both possible comparators. The difference from the USPSTF 2021 lung-cancer group is 5.2 percentage points (P = .28). Even against the abstract's 48% group it is 6.6 points (P = .14). The axis starts at 35% to show the intervals. The figure is a dot-and-interval plot with no bars, so the cut-off axis does not change how large any difference looks relative to its interval.
Why it matters
- A significance verdict is reversed in the abstract. Readers of the abstract are told that the Potter criterion significantly raised the share of women among the extra lung cancers it would catch. The paper's own data show no detectable difference: 54.7% vs 49.5%, P = .28. The direction is the same, but the gap is about 5 points rather than the 7 the abstract implies, and it is not significant.
- It bears on the equity argument as read from the abstract. The paper's premise is that relaxing smoking criteria reduces race- and sex-based disparities. With the correct numbers, the sex benefit holds for eligibility overall (55% vs 48% female among all additional eligibles, P < .001), but not among those who had lung cancer. Among those who had lung cancer, only the race difference is significant (39% vs 22%, P < .001).
- The headline conclusions are not affected. The Key Points claim that the change "reduced race- and sex-based disparities in eligibility, while maintaining diagnostic efficiency", and the Discussion limits the lung-cancer finding to race: "reduced race-based and sex-based disparities in LCS eligibility and race-based disparities in lung cancer diagnosis". Both rest on comparisons that are reported correctly. The full-text Results also give the correct value, so the error is confined to one sentence of the abstract.
- A smaller, related inconsistency. In the abstract's sentence about all additional eligibles, the USPSTF 2021 comparator for Black individuals is given as 22%, while the Results give 20%. The counts (777/3840) give 20.2%. The 22% matches the lung-cancer subgroup (162/747 = 21.7%). The test is P < .001 either way, so no conclusion changes.
Explanations considered
- A different population in the abstract. Not supported. The abstract sentence names "additional eligible individuals with lung cancer", gives 70 of 128, and lists the matching ACS (89 of 208) and Potter-ACS (143 of 304) counts. These are exactly the numbers in the Results lung-cancer paragraph. The lung-cancer comparator printed in the paper is 370 of 747 (50%).
- A different comparator group (the 48%). It explains the 48% but not the P value. 48% equals 1847/3840, the female share among all USPSTF 2021 eligibles. Testing 70/128 against that group still gives P = .14 (Fisher .15), not P < .001. Comparing 70/128 with a group of people mostly without lung cancer would also not be the comparison the sentence describes.
- A different test, correction or sidedness. Not able to explain it. The uncorrected Pearson χ², the test the Methods name, gives P = .281 and reproduces the Results' P = .28. The Yates correction (.326), Fisher's exact test (.294) and a one-sided test (.141) are all far above .001. The Methods state that tests were 2-sided with no multiplicity adjustment.
- The overall Potter lung-cancer group instead of the "Extra" group. Not able to explain it. All IPN-Potter lung cancers, 440/872 (50.5%), against 370/747 give P = .71.
- A text-conversion artifact. Ruled out. The publisher's full text, the PMC HTML page and the Europe PMC full-text XML carry the same wording and numbers in both passages.
- Which passage is correct. The Results are. Their P value reproduces exactly from the printed counts, and those counts match eTable 2 of the supplement. The abstract's "(55% vs 48%; P < .001)" is word for word the result printed for the all-additional-eligible comparison in the sentence before it and in Results, section "Additional Persons Eligible". That result reproduces from 606/1103 vs 1847/3840 (P = 6.2 × 10⁻⁵).
Status
- Version of record. doi.org resolves to the publisher's full text at jamanetwork.com (article 2835517). Both passages appear there with the same numbers as quoted above. The PMC HTML page and the Europe PMC full-text XML match word for word. Every quotation here was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. Supplement 1 (eTables 1–4, eFigures 1–3; 22 pages). eTable 2 lists Female 370 (49.53%) of 747 for IPN-USPSTF2021 eligibles with lung cancer and 70 (54.69%) of 128 for IPN-Potter Extra. eTable 1 lists 1847 (48.1%) of 3840 and 606 (54.94%) of 1103. The supplement has no P values. Supplement 2 is the data sharing statement and is not relevant. The paper's Figure 2, which also displays these proportions, was not used.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyfinds no Crossref update relations and no PMC or Europe PMC notices. The Crossref record has no update-to or updated-by entries. The PubMed record (PMID 40553471) has no CommentsCorrections entries. Neither the publisher's page nor the PMC page shows a correction notice, and a web search found none for this article. Not checked: the Europe PMC search API returned HTTP 503 on every attempt, and PubPeer could not be queried (HTTP 403 on the search page; the API returned Not Found). Retraction Watch data was checked only indirectly, through Crossref. - Sources. PMC article PMC12188365; Europe PMC full-text XML and supplementary files; publisher full text via https://doi.org/10.1001/jamanetworkopen.2025.17149; Crossref; PubMed.
Suggested correction
In the Results paragraph of the abstract, replace
with statistically significant differences between the IPN-Potter criteria and USPSTF 2021 criteria (55% vs 48%; P < .001).
with
with no statistically significant difference between the IPN-Potter criteria and USPSTF 2021 criteria (55% vs 50%; P = .28).
so that it agrees with the Results section and with eTable 2. In the sentence before it, the authors may also wish to change the USPSTF 2021 comparator for Black individuals, "(35% vs 22%; P < .001)", to "(35% vs 20%; P < .001)", matching the Results (777 of 3840).
Recomputation script (python recompute.py.txt)
"""Recompute the sex comparison among additional eligible people with lung cancer in PMC12188365
(JAMA Netw Open 2025, doi:10.1001/jamanetworkopen.2025.17149,
"Reducing Smoking Requirements for Lung Screening to Address Health Disparities in a Community Cohort").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are counts copied from the paper as archived in PMC (abstract and Results):
IPN-Potter Extra with lung cancer: 70 female of 128, 50 Black of 128.
IPN-USPSTF 2021 with lung cancer: 370 female of 747, 162 Black of 747.
IPN-Potter (all) with lung cancer: 440 female of 872, 212 Black of 872.
IPN-Potter Extra (all additional eligibles): 606 female of 1103, 382 Black of 1103.
IPN-USPSTF 2021 (all eligibles): 1847 female of 3840, 777 Black of 3840.
The same counts appear in eTables 1 and 2 of Supplement 1 (370 [49.53%] of 747 and
70 [54.69%] of 128 in eTable 2; 1847 [48.1%] of 3840 and 606 [54.94%] of 1103 in eTable 1).
Methods: "the chi2 test for categorical variables", alpha = .05, 2-sided, no multiplicity adjustment.
"""
import os
import textwrap
import numpy as np
from scipy.stats import chi2_contingency, fisher_exact, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT, "ytick.color": TXT,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
# ------------------------------------------------------------------ inputs (counts from the paper)
LC_POTTER_EXTRA_F, LC_POTTER_EXTRA_N = 70, 128
LC_USPSTF_F, LC_USPSTF_N = 370, 747
LC_POTTER_ALL_F, LC_POTTER_ALL_N = 440, 872
ALL_POTTER_EXTRA_F, ALL_POTTER_EXTRA_N = 606, 1103
ALL_USPSTF_F, ALL_USPSTF_N = 1847, 3840
LC_POTTER_EXTRA_B, LC_USPSTF_B, LC_POTTER_ALL_B = 50, 162, 212
ALL_POTTER_EXTRA_B, ALL_USPSTF_B = 382, 777
ALL_POTTER_N, ALL_USPSTF_N_ = 4905, 3840
def test(a, n1, b, n2):
t = np.array([[a, n1 - a], [b, n2 - b]])
chi0, p0, _, _ = chi2_contingency(t, correction=False)
chi1, p1, _, _ = chi2_contingency(t, correction=True)
_, pf = fisher_exact(t)
# two-proportion z test, pooled (identical to the uncorrected chi2)
pp = (a + b) / (n1 + n2)
se = np.sqrt(pp * (1 - pp) * (1 / n1 + 1 / n2))
z = (a / n1 - b / n2) / se
return dict(chi2=chi0, p=p0, chi2_yates=chi1, p_yates=p1, p_fisher=pf, z=z,
p_z=2 * norm.sf(abs(z)), p1=a / n1, p2=b / n2)
def wilson(k, n, a=0.05):
zc = norm.ppf(1 - a / 2)
p = k / n
c = (p + zc ** 2 / (2 * n)) / (1 + zc ** 2 / n)
h = zc * np.sqrt(p * (1 - p) / n + zc ** 2 / (4 * n ** 2)) / (1 + zc ** 2 / n)
return c - h, c + h
def show(label, r):
print(f"{label:62s} {100*r['p1']:5.1f}% vs {100*r['p2']:5.1f}% diff {100*(r['p1']-r['p2']):+5.1f} pp "
f"chi2={r['chi2']:7.3f} P={r['p']:.3g} | Yates P={r['p_yates']:.3g} | Fisher P={r['p_fisher']:.3g} | z={r['z']:.3f}")
# ------------------------------------------------------------------ 1. percentages
print("=== 1. Percentages from the paper's counts ===")
PCTS = [
("Potter Extra, lung cancer, female", LC_POTTER_EXTRA_F, LC_POTTER_EXTRA_N),
("USPSTF 2021, lung cancer, female", LC_USPSTF_F, LC_USPSTF_N),
("Potter (all), lung cancer, female", LC_POTTER_ALL_F, LC_POTTER_ALL_N),
("Potter Extra, all additional eligibles, female", ALL_POTTER_EXTRA_F, ALL_POTTER_EXTRA_N),
("USPSTF 2021, all eligibles, female", ALL_USPSTF_F, ALL_USPSTF_N),
("Potter Extra, lung cancer, Black", LC_POTTER_EXTRA_B, LC_POTTER_EXTRA_N),
("USPSTF 2021, lung cancer, Black", LC_USPSTF_B, LC_USPSTF_N),
("Potter Extra, all additional eligibles, Black", ALL_POTTER_EXTRA_B, ALL_POTTER_EXTRA_N),
("USPSTF 2021, all eligibles, Black", ALL_USPSTF_B, ALL_USPSTF_N),
]
CI = {}
for lab, k, n in PCTS:
lo, hi = wilson(k, n)
CI[lab] = (k / n, lo, hi)
print(f"{lab:50s} {k:5d}/{n:<5d} = {100*k/n:5.2f}% (95% Wilson CI {100*lo:.1f}% to {100*hi:.1f}%)")
# ------------------------------------------------------------------ 2. the comparison in question
print()
print("=== 2. Female share, additional eligibles WITH LUNG CANCER: Potter Extra vs USPSTF 2021 ===")
R_LC = test(LC_POTTER_EXTRA_F, LC_POTTER_EXTRA_N, LC_USPSTF_F, LC_USPSTF_N)
show("70/128 vs 370/747 (the comparison both passages describe)", R_LC)
print(f" Results text: 55% vs 50%, P = .28 -> recomputed uncorrected chi2 P = {R_LC['p']:.3f} (matches)")
print(f" Abstract: 55% vs 48%, P < .001 -> not reproducible from these counts")
print(f" pooled two-proportion z = {R_LC['z']:.3f}, P = {R_LC['p_z']:.3f}")
print()
print("=== 3. Alternative comparators that might explain the abstract ===")
R_ALT48 = test(LC_POTTER_EXTRA_F, LC_POTTER_EXTRA_N, ALL_USPSTF_F, ALL_USPSTF_N)
show("70/128 vs 1847/3840 (abstract's 48% comparator = all USPSTF eligibles)", R_ALT48)
R_OVER = test(LC_POTTER_ALL_F, LC_POTTER_ALL_N, LC_USPSTF_F, LC_USPSTF_N)
show("440/872 vs 370/747 (all Potter lung cancers vs USPSTF lung cancers)", R_OVER)
# 70/128 vs the lung cancers in USPSTF that are not ... (USPSTF LC minus none); one-sided variant
print(f" one-sided version of 70/128 vs 370/747: P = {R_LC['p']/2:.3f}")
# smallest USPSTF-LC female count that would have given P < .001 against 70/128 with n=747
k_needed = None
for k in range(LC_USPSTF_F, -1, -1):
if test(LC_POTTER_EXTRA_F, LC_POTTER_EXTRA_N, k, LC_USPSTF_N)["p"] < 0.001:
k_needed = k
break
print(f" for P < .001 against 70/128, USPSTF lung-cancer females would need to be <= {k_needed}/747 "
f"({100*k_needed/747:.1f}%), not 370/747")
print()
print("=== 4. All additional eligibles (where '55% vs 48%; P < .001' belongs) ===")
R_ALL = test(ALL_POTTER_EXTRA_F, ALL_POTTER_EXTRA_N, ALL_USPSTF_F, ALL_USPSTF_N)
show("606/1103 vs 1847/3840 (female, all additional eligibles)", R_ALL)
print()
print("=== 5. Race comparisons (for context; correctly significant) ===")
R_LC_B = test(LC_POTTER_EXTRA_B, LC_POTTER_EXTRA_N, LC_USPSTF_B, LC_USPSTF_N)
show("50/128 vs 162/747 (Black, lung-cancer subgroup)", R_LC_B)
R_ALL_B = test(ALL_POTTER_EXTRA_B, ALL_POTTER_EXTRA_N, ALL_USPSTF_B, ALL_USPSTF_N)
show("382/1103 vs 777/3840 (Black, all additional eligibles)", R_ALL_B)
print(f" USPSTF all-eligible Black share 777/3840 = {100*777/3840:.1f}% (Results: 20%; abstract: 22%)")
print(f" USPSTF lung-cancer Black share 162/747 = {100*162/747:.1f}% (the abstract's 22% matches this one)")
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# ------------------------------------------------------------------ figure 1: abstract vs Results
def wrap(s, w):
return "\n".join(textwrap.wrap(s, w))
def fig1():
fig = plt.figure(figsize=(7.2, 5.3))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
def panel(x, y, w, h, title, sub, body, key, keycolor):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=SEC, lw=1.0))
ax.text(x + 0.02, y + h - 0.035, title, fontsize=11.5, fontweight="bold", va="top")
ax.text(x + 0.02, y + h - 0.085, sub, fontsize=10, color=SEC, va="top", style="italic")
ax.text(x + 0.02, y + h - 0.165, body, fontsize=10, va="top", linespacing=1.35)
ax.text(x + 0.02, y + 0.03, key, fontsize=10.5, va="bottom", color=keycolor,
fontweight="bold", linespacing=1.3)
abs_body = wrap("“The additional eligible individuals with lung cancer in IPN programs who "
"were female included 70 of 128 (55%) by IPN-Potter criteria, … with "
"statistically significant differences between the IPN-Potter criteria and "
"USPSTF 2021 criteria", 38)
res_body = wrap("“The additional eligible individuals with lung cancer in IPN … who were "
"female included 70 of 128 (55%) in Potter criteria, … not significantly "
"different between IPN-Potter Extra and IPN-USPSTF 2021", 38)
panel(0.03, 0.40, 0.44, 0.57, "Abstract", "Results paragraph",
abs_body, "(55% vs 48%; P < .001).”\nreported: significant", ORANGE)
panel(0.53, 0.40, 0.44, 0.57, "Results", "“Screening Eligibility in Those\nWith Lung Cancer”",
res_body, "(55% vs 50% [370 of 747]; P = .28)”\nreported: not significant", ORANGE)
# connector between the two panels
arr = FancyArrowPatch((0.47, 0.50), (0.53, 0.50), arrowstyle="<->", mutation_scale=14,
lw=2, color=TXT)
ax.add_patch(arr)
ax.text(0.50, 0.555, "same\ngroups,\nsame\ntest", ha="center", va="bottom", fontsize=10, color=SEC)
# recomputation strip
ax.add_patch(FancyBboxPatch((0.03, 0.03), 0.94, 0.32, boxstyle="round,pad=0,rounding_size=0.012",
fc="#eef4fc", ec=BLUE, lw=1.5))
ax.text(0.05, 0.325, "Recomputed from the paper's own counts (Pearson χ², 2-sided, as in Methods)",
fontsize=10.5, fontweight="bold", color=BLUE, va="top")
lines = [
(f"Lung-cancer subgroup: 70/128 = {100*R_LC['p1']:.1f}% vs 370/747 = {100*R_LC['p2']:.1f}%"
f" → P = {R_LC['p']:.2f}", TXT),
(f" matches the Results (P = .28); neither Yates (P = {R_LC['p_yates']:.2f}) nor Fisher "
f"(P = {R_LC['p_fisher']:.2f}) gives P < .001", SEC),
(f"Abstract's 48% = 1847/3840 = {100*ALL_USPSTF_F/ALL_USPSTF_N:.1f}%, the female share of ALL "
f"USPSTF 2021 eligibles", TXT),
(f" 70/128 vs 1847/3840 → P = {R_ALT48['p']:.2f} (still not significant)", SEC),
(f"'55% vs 48%; P < .001' fits all additional eligibles: 606/1103 = {100*R_ALL['p1']:.1f}% vs "
f"{100*R_ALL['p2']:.1f}%, P = {R_ALL['p']:.1e}", TXT),
]
yy = 0.265
for s, c in lines:
ax.text(0.05, yy, s, fontsize=10, color=c, va="top")
yy -= 0.047
save(fig, "fig1")
# ------------------------------------------------------------------ figure 2: proportions with 95% CI
def fig2():
rows = [
("IPN-Potter Extra,\nwith lung cancer (70/128)", "Potter Extra, lung cancer, female", 55, "55% (abstract and Results)"),
("IPN-USPSTF 2021,\nwith lung cancer (370/747)", "USPSTF 2021, lung cancer, female", 50, "50% (Results)"),
("IPN-USPSTF 2021,\nall eligibles (1847/3840)", "USPSTF 2021, all eligibles, female", 48, "48% (abstract comparator)"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.5))
fig.subplots_adjust(left=0.30, right=0.97, top=0.80, bottom=0.22)
ys = [2, 1, 0]
for y, (lab, key, rep, replab) in zip(ys, rows):
p, lo, hi = CI[key]
ax.plot([100 * lo, 100 * hi], [y, y], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(100 * p, y, "o", ms=9, color=BLUE, zorder=3)
ax.plot(rep, y + 0.28, "D", ms=8, color=ORANGE, zorder=3)
ax.text(rep + 1.0, y + 0.28, replab, va="center", fontsize=10, color=ORANGE)
ax.text(100 * hi + 1.0, y - 0.02, f"{100*p:.1f}% ({100*lo:.1f}–{100*hi:.1f})",
va="center", fontsize=10, color=BLUE)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_ylim(-0.6, 2.75)
ax.set_xlim(35, 80)
ax.set_xlabel("Female, % of group")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="D", color=ORANGE, ls="none", ms=8, label="reported in the paper (rounded %)"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=9, label="recomputed from counts, with 95% Wilson CI")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(-0.42, 1.02), ncol=2, frameon=False, fontsize=10)
fig.text(0.03, 0.025, f"Recomputed Pearson χ²: 70/128 vs 370/747, P = {R_LC['p']:.2f}; "
f"70/128 vs 1847/3840, P = {R_ALT48['p']:.2f}", fontsize=10, color=BLUE)
save(fig, "fig2")
fig1()
fig2()
print()
print("figures written: fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract reports two obesity-group odds ratios as the overweight and obesity estimates for EPT use
JAMA Netw Open (2025) · doi:10.1001/jamanetworkopen.2025.19236 · PMC12235495
Summary
The study pools 31 case-control studies to estimate associations between exogenous hormone use and intrinsic-like breast cancer subtypes, stratified by body mass index (BMI). The abstract says the associations of current estrogen-progestin therapy (EPT) use with luminal A–like and luminal B–ERBB2-like tumors "were attenuated but remained" for women with overweight (OR 1.40, 95% CI 1.02–1.92) or obesity (OR 1.68, 95% CI 1.01–2.78). In the paper's own Results text and in Supplement 1 eTable 3, both of these values come from the obesity stratum: 1.40 is the luminal A–like OR and 1.68 is the luminal B–ERBB2-like OR. The overweight estimates, 2.18 (1.83–2.61) for luminal A–like and 1.83 (1.32–2.55) for luminal B–ERBB2-like, do not appear in the abstract. No direction or significance changes, but an abstract reader would take the overweight luminal A–like excess odds to be about a third of what the paper's own data show, and would see a BMI pattern (1.40 for overweight, then 1.68 for obesity) that the full results do not show.
What the paper reports
Abstract, Results:
Current EPT users with healthy weight were more likely to be diagnosed with luminal A–like (odds ratio [OR], 2.51 [95% CI, 2.26-2.80]) or luminal B–ERBB2-like (OR, 1.95 [95% CI, 1.61-2.37]) subtypes. These associations were attenuated but remained for individuals with overweight (OR, 1.40 [95% CI, 1.02-1.92]) or obesity (OR, 1.68 [95% CI, 1.01-2.78]).
Results, "Combined EPT Use and Tumor Subtypes", paragraph 2:
In women with healthy weight, current EPT users were more likely to be diagnosed with luminal A–like (OR, 2.51 [95% CI, 2.26-2.80]), luminal B–like (OR, 1.47 [95% CI, 1.17-1.86]), or luminal B–ERBB2–like (OR, 1.95 [95% CI, 1.61-2.37]) breast tumors, with significant heterogeneity across subtypes (P < .001 for heterogeneity).
Results, "Combined EPT Use and Tumor Subtypes", paragraph 3:
In women with overweight, current EPT use was positively associated with luminal A–like (OR, 2.18 [95% CI, 1.83-2.61]) and luminal B–ERBB2-like (OR, 1.83 [95% CI, 1.32-2.55]) tumors but not with other subtypes (P < .001 for heterogeneity). In women with obesity, associations for luminal A–like (OR, 1.40 [95% CI, 1.02-1.92]) and luminal B–ERBB2-like (OR, 1.68 [95% CI, 1.01-2.78]) tumors were seen, with smaller effect sizes and nonsignificant heterogeneity (P < .11 for heterogeneity).
Methods, Statistical Analysis (the only BMI definition in the paper):
To evaluate heterogeneity in associations by BMI, associations were stratified by BMI categories as follows: lean or healthy weight (hereinafter, healthy weight) (18.5-<25 [calculated as weight in kilograms divided by height in meters squared]), overweight (25-<30), or obesity (≥30).
Supplement 1, eTable 3 (EPT use by intrinsic-like subtype, stratified by BMI), "Current use" rows: Lean/normal, luminal A-like 2.51 (2.26,2.80) and luminal B-ERBB2-like 1.95 (1.61,2.37). Overweight, 2.18 (1.83,2.61) and 1.83 (1.32,2.55). Obese, 1.40 (1.02,1.92) and 1.68 (1.01,2.78).
All quotes from the article were checked verbatim against the PMC-archived text with src/show.py PMC12235495 --grep. The eTable values were read from the supplement PDF.
What the paper's own numbers show
Method. I took every current-EPT odds ratio and 95% CI for luminal A–like and luminal B–ERBB2-like tumors from the Results text, and checked that each CI is symmetric about its OR on the log scale. I checked the text values against Supplement 1 eTable 3. I then matched each (OR, lower, upper) triple in the abstract against every BMI-by-subtype cell in the Results. To size the discrepancy, I compared the abstract's "overweight" value with the paper's overweight estimates as a ratio of ORs and as excess odds (OR − 1). I computed a z statistic for the difference of log ORs, using standard errors taken from the CI widths. That z treats the overweight and obesity strata as independent, which holds because they are disjoint groups of women. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Healthy weight, luminal A–like | Abstract; Results para. 2; eTable 3 | 2.51 (2.26–2.80) | Abstract and Results identical; geometric CI midpoint 2.516 |
| Healthy weight, luminal B–ERBB2-like | Abstract; Results para. 2; eTable 3 | 1.95 (1.61–2.37) | Abstract and Results identical; midpoint 1.953 |
| Overweight, luminal A–like | Results para. 3; eTable 3 | 2.18 (1.83–2.61) | Midpoint 2.185, consistent; not in the abstract |
| Overweight, luminal B–ERBB2-like | Results para. 3; eTable 3 | 1.83 (1.32–2.55) | Midpoint 1.835, consistent; not in the abstract |
| Obesity, luminal A–like | Results para. 3; eTable 3 | 1.40 (1.02–1.92) | Midpoint 1.399, consistent |
| Obesity, luminal B–ERBB2-like | Results para. 3; eTable 3 | 1.68 (1.01–2.78) | Midpoint 1.676, consistent |
| Abstract value labelled "overweight" | Abstract, Results | 1.40 (1.02–1.92) | Equals only the obesity, luminal A–like cell; matches no overweight cell |
| Abstract value labelled "obesity" | Abstract, Results | 1.68 (1.01–2.78) | Equals only the obesity, luminal B–ERBB2-like cell |
| Overweight luminal A–like vs abstract "overweight" | Results vs Abstract | 2.18 vs 1.40 | Ratio of ORs 1.56; excess odds 1.18 vs 0.40, so the abstract shows 34% of it; z = 2.39 for the difference |
| Overweight luminal B–ERBB2-like vs abstract "overweight" | Results vs Abstract | 1.83 vs 1.40 | Ratio of ORs 1.31; excess odds 0.83 vs 0.40 (48%); z = 1.15 |
| BMI gradient, luminal A–like | Results paras. 2–3 | 2.51 → 2.18 → 1.40 | Falls steadily with BMI |
| BMI gradient, luminal B–ERBB2-like | Results paras. 2–3 | 1.95 → 1.83 → 1.68 | Falls steadily with BMI |
| BMI gradient as the abstract presents it | Abstract | overweight 1.40, obesity 1.68 | Rises from overweight to obesity, which no subtype in the Results does |
| Significance, overweight and obesity cells | Results para. 3 | All four CIs exclude 1 | z (log OR / SE) = 8.60, 3.60, 2.09, 2.01; unchanged |
| Fully adjusted model (sensitivity) | eTable 7 | Overweight 2.18 (1.82–2.61), 1.81 (1.30–2.52); obese 1.41 (1.02–1.93), 1.65 (0.99–2.75) | No overweight cell equals 1.40 (1.02–1.92) |
Every value in the Results text agrees with eTable 3, and every CI is consistent with its point estimate. The other four current-EPT values in the abstract (2.51, 1.95 and 1.47 for healthy weight, 1.68 for obesity) match Results values for the BMI group they are given under. The only mismatch is the BMI label on the first of the two higher-BMI values.
Figure
Figure 1. On the left (orange) is the abstract's sentence on attenuation, with the value attached to each BMI group as printed. On the right (blue) are the current-EPT odds ratios (95% CI) for women with overweight and obesity from Results paragraph 3, which are identical to the Supplement 1 eTable 3 "Current use" rows. Arrows link each abstract value to the only Results cell that has the same OR and CI. Both arrows end in the obesity row. The abstract's "overweight" value is the obesity, luminal A–like estimate. The shaded overweight row (2.18 and 1.83) is not reported anywhere in the abstract.
Figure 2. Odds ratios with 95% CIs for current EPT use vs never use, on a log axis with a thin reference line at OR = 1. Blue: values from the Results text and eTable 3 (circles, luminal A–like; squares, luminal B–ERBB2-like). Orange diamonds: the abstract's values, drawn under the BMI group the abstract names. In the healthy-weight and obesity groups each orange interval repeats a blue one. In the overweight group the abstract's interval (1.02–1.92) matches neither overweight estimate. It reproduces the obesity, luminal A–like interval one group lower. The blue luminal A–like estimates fall steadily from 2.51 to 2.18 to 1.40.
Why it matters
The BMI stratification is central to how the paper frames its EPT findings: the Key Points and Conclusions stress associations "particularly in participants with healthy weight", and the Discussion describes the luminal A–like and luminal B–ERBB2-like associations as "consistent across BMI groups, albeit attenuated with greater adiposity". The abstract is the only part of the paper that most readers, and most citing articles, see. A reader who relies on it would conclude that among women with overweight, current EPT use is associated with an OR of about 1.40. By the paper's own estimates the overweight luminal A–like OR is 2.18. That is roughly three times the excess odds (1.18 vs 0.40), and the difference is larger than sampling error alone would plausibly produce (z ≈ 2.4). The mislabeling also distorts the dose-response picture by BMI: the abstract's numbers rise from overweight (1.40) to obesity (1.68), while the full results fall steadily with BMI for both subtypes. Because neither value in the sentence is tied to a subtype, a reader also cannot tell that the two numbers refer to different tumor subtypes.
The consequences are limited. All four overweight and obesity estimates are above 1 with CIs that exclude 1, so the abstract's qualitative statement ("attenuated but remained") is true for both BMI groups. The title, Key Points, abstract Conclusions, body Results, Discussion, Figure 1 and the supplement are correct. The estrogen-only therapy and oral contraceptive findings, and the heterogeneity tests, are not affected. The error is a misreported secondary estimate in the abstract, not a change to any headline conclusion.
Explanations considered
- A different model, population or adjustment set in the abstract. The abstract and the Results both describe current EPT use vs nonuse in postmenopausal women, from the same polytomous logistic model stratified by BMI. No footnote, legend or Methods statement points to a different model for the abstract. The fully adjusted sensitivity model (eTable 7) gives overweight estimates of 2.18 and 1.81, not 1.40. This does not explain the numbers.
- A different BMI definition. The Methods and the abstract's own Design section use one set of cut points (overweight 25–<30, obesity ≥30), and eTable 3 uses the same. This does not explain the numbers.
- The value belongs to another overweight estimate. I checked whether 1.40 (1.02–1.92) is an overweight value for some other subtype or exposure. The overweight current-EPT estimates in the text are 2.18 and 1.83. The overweight past-EPT values mentioned in the Results are 1.50 and 1.57, both with different CIs. In the whole of Supplement 1, the exact triple 1.40 (1.02, 1.92) occurs once, in the Obese, current-use, luminal A-like cell of eTable 3. The four other ORs of 1.40 in the supplement all have different CIs. This does not explain the numbers.
- "Overweight or obesity" meant as one combined group. On this reading the two values would be the luminal A–like and luminal B–ERBB2-like ORs for women with higher BMI. The sentence's grammar does not support that reading: each value sits directly after a different BMI category, and no subtype is named. Even on this reading, both values come from the obesity stratum alone, and the overweight stratum, which has the larger estimates, is left out. This explanation reduces the problem to ambiguous wording, but the numbers would still be attributed to women with overweight when they come only from women with obesity.
- Rounding or transcription within the same cell. The abstract triples match the obesity cells to every printed digit and differ from the overweight cells by 0.78 and 0.43 on the OR scale. This is not a rounding effect.
- Text-conversion artifact in PMC. The same wording appears on the publisher's full-text page, on the PMC HTML page and in the Europe PMC full-text XML. This does not explain the numbers.
The pattern fits a mislabeled sentence in the abstract: the two obesity-row values were placed after "overweight" and "obesity" in place of an overweight value and an obesity value for the same subtype. Nothing in the paper shows how this happened.
Status
As of 2026-10-23:
- Version of record. doi.org resolves to the publisher's full-text page (jamanetwork.com/journals/jamanetworkopen/fullarticle/2836044), which was read in this workflow. Its abstract has the same "overweight (OR, 1.40 [95% CI, 1.02-1.92]) or obesity (OR, 1.68 [95% CI, 1.01-2.78])" wording. Its body Results give the same overweight (2.18, 1.83) and obesity (1.40, 1.68) values as PMC. The PMC page and the Europe PMC full-text XML carry identical text.
- Supplements. Supplement 1 (jamanetwopen-e2519236-s001.pdf: eMethods, eTables 1–29, eFigures 1–3) was downloaded via Europe PMC and read. eTable 3 agrees with the Results text in every cell used here. eTable 7, the fully adjusted model, gives similar values. Neither table has 1.40 (1.02–1.92) or 1.68 (1.01–2.78) in an overweight row. The supplement confirms the Results text and resolves which stratum the abstract values belong to.
- Corrections. None found.
src/known.pyreturned no Crossref updates and no PMC or Europe PMC notices. The Crossref works record has no update-to or updated-by relation. A Crossreffilter=updates:query returned 0 results. The Crossref Labs record, which carries the Retraction Watch data, lists no updates. Europe PMC (PMID 40622713) lists no comment or correction. The publisher page shows no correction notice. Earlier web searches in this workflow for the title together with "correction" or "erratum" found nothing relevant. - PubPeer. The PubPeer search API and search page both returned HTTP 403 and could not be queried directly. An earlier web search in this workflow for the DOI together with "PubPeer" found no thread. A repeat web search could not be run for this write-up because the session's search quota was exhausted.
Sources: publisher full text; PMC12235495; Europe PMC REST full-text XML and supplementary files for PMC12235495; Crossref REST API for doi:10.1001/jamanetworkopen.2025.19236.
Suggested correction
Abstract, Results. Replace "These associations were attenuated but remained for individuals with overweight (OR, 1.40 [95% CI, 1.02-1.92]) or obesity (OR, 1.68 [95% CI, 1.01-2.78])." with:
These associations were attenuated but remained for individuals with overweight (luminal A–like: OR, 2.18 [95% CI, 1.83-2.61]; luminal B–ERBB2-like: OR, 1.83 [95% CI, 1.32-2.55]) or obesity (luminal A–like: OR, 1.40 [95% CI, 1.02-1.92]; luminal B–ERBB2-like: OR, 1.68 [95% CI, 1.01-2.78]).
If word limits require a shorter sentence, the luminal A–like values alone (overweight 2.18 [1.83-2.61]; obesity 1.40 [1.02-1.92]) would give the BMI gradient correctly. All values are those already given in the Results text and in Supplement 1 eTable 3. No other part of the paper needs to change.
Recomputation script (python recompute.py.txt)
"""Check the BMI-stratified current-EPT odds ratios in the abstract of PMC12235495
(JAMA Netw Open 2025; doi:10.1001/jamanetworkopen.2025.19236) against the paper's own
Results text and Supplement 1 eTable 3, and redraw fig1/fig2 in this file's folder.
Run: python recompute.py.txt (needs numpy, matplotlib)
All inputs are copied from the published article (Abstract; Results, "Combined EPT Use
and Tumor Subtypes", paragraphs 2 and 3) and from Supplement 1 (eTables 3 and 7).
"""
import os
import math
import numpy as np
HERE = os.path.dirname(os.path.abspath(__file__))
Z = 1.959964
# ---------------------------------------------------------------- inputs
# Results text: current EPT use vs never use, by BMI stratum and intrinsic-like subtype.
# (OR, lower 95% CI, upper 95% CI). Identical values in Supplement 1 eTable 3, "Current use" rows.
RES = {
("healthy weight", "luminal A-like"): (2.51, 2.26, 2.80), # Results para 2
("healthy weight", "luminal B-like"): (1.47, 1.17, 1.86), # Results para 2
("healthy weight", "luminal B-ERBB2-like"): (1.95, 1.61, 2.37), # Results para 2
("overweight", "luminal A-like"): (2.18, 1.83, 2.61), # Results para 3
("overweight", "luminal B-ERBB2-like"): (1.83, 1.32, 2.55), # Results para 3
("obesity", "luminal A-like"): (1.40, 1.02, 1.92), # Results para 3
("obesity", "luminal B-ERBB2-like"): (1.68, 1.01, 2.78), # Results para 3
}
# Supplement 1, eTable 3 (base model), "Current use" rows, as printed.
ETABLE3 = {
("healthy weight", "luminal A-like"): (2.51, 2.26, 2.80),
("healthy weight", "luminal B-ERBB2-like"): (1.95, 1.61, 2.37),
("overweight", "luminal A-like"): (2.18, 1.83, 2.61),
("overweight", "luminal B-ERBB2-like"): (1.83, 1.32, 2.55),
("obesity", "luminal A-like"): (1.40, 1.02, 1.92),
("obesity", "luminal B-ERBB2-like"): (1.68, 1.01, 2.78),
}
# Supplement 1, eTable 7 (fully adjusted model), "Current use" rows.
ETABLE7 = {
("overweight", "luminal A-like"): (2.18, 1.82, 2.61),
("overweight", "luminal B-ERBB2-like"): (1.81, 1.30, 2.52),
("obesity", "luminal A-like"): (1.41, 1.02, 1.93),
("obesity", "luminal B-ERBB2-like"): (1.65, 0.99, 2.75),
}
# Abstract, Results: the BMI group each value is attached to (subtype as stated, if any).
ABS = [
("healthy weight", "luminal A-like", (2.51, 2.26, 2.80)),
("healthy weight", "luminal B-ERBB2-like", (1.95, 1.61, 2.37)),
("overweight", "(no subtype named)", (1.40, 1.02, 1.92)),
("obesity", "(no subtype named)", (1.68, 1.01, 2.78)),
("healthy weight", "luminal B-like", (1.47, 1.17, 1.86)),
]
def se_log(lo, hi):
return (math.log(hi) - math.log(lo)) / (2 * Z)
def gmid(lo, hi):
return math.exp((math.log(lo) + math.log(hi)) / 2)
def fmt(t):
return f"{t[0]:.2f} ({t[1]:.2f}-{t[2]:.2f})"
print("=== 1. Internal consistency of the Results values (CI symmetric on the log scale) ===")
for k, (o, lo, hi) in RES.items():
print(f" {k[0]:15s} {k[1]:22s} OR {o:.2f} geometric midpoint of CI {gmid(lo, hi):.3f} "
f"SE(log OR) {se_log(lo, hi):.3f}")
print("\n=== 2. Results text vs Supplement 1 eTable 3 ===")
for k, v in ETABLE3.items():
print(f" {k[0]:15s} {k[1]:22s} text {fmt(RES[k])} eTable 3 {fmt(v)} identical: {RES[k] == v}")
print("\n=== 3. Which Results estimate does each abstract value equal? (exact OR + CI match) ===")
for bmi, sub, v in ABS:
hits = [f"{b} / {s}" for (b, s), rv in RES.items() if rv == v]
same = any(b == bmi for (b, s), rv in RES.items() if rv == v)
print(f" abstract '{bmi}' {sub:22s} {fmt(v)} -> Results: {hits}; same BMI group: {same}")
ow = {s: v for (b, s), v in RES.items() if b == "overweight"}
print(f" Results overweight estimates: " + "; ".join(f"{s} {fmt(v)}" for s, v in ow.items()))
print(f" Does 1.40 (1.02-1.92) appear among any overweight estimate? "
f"{any(v == (1.40, 1.02, 1.92) for v in ow.values())}")
print(" Fully adjusted eTable 7 values for the same cells: " +
"; ".join(f"{b}/{s} {fmt(v)}" for (b, s), v in ETABLE7.items()))
print(f" Does the abstract's 'overweight' triple equal any eTable 7 overweight cell? "
f"{any(v == (1.40, 1.02, 1.92) for (b, s), v in ETABLE7.items() if b == 'overweight')}")
print("\n=== 4. Size of the discrepancy for women with overweight ===")
a = ABS[2][2]
for s, v in ow.items():
ex_rep, ex_true = a[0] - 1, v[0] - 1
d = math.log(v[0]) - math.log(a[0])
se_d = math.sqrt(se_log(v[1], v[2]) ** 2 + se_log(a[1], a[2]) ** 2)
print(f" {s:22s} abstract 1.40 vs Results {v[0]:.2f}: ratio of ORs {v[0]/a[0]:.2f}; "
f"excess odds {ex_rep:.2f} vs {ex_true:.2f} -> abstract shows {100*ex_rep/ex_true:.0f}% "
f"of the excess (Results value is {ex_true/ex_rep:.1f}x)")
print(f" {'':22s} difference of log ORs {d:.3f}, SE {se_d:.3f}, z {d/se_d:.2f} "
f"(treating the overweight and obesity strata as independent samples)")
print(f" Overweight CI lower bounds vs abstract value: luminal A-like lower bound 1.83 > 1.40: {1.83 > 1.40}; "
f"the two CIs overlap only between 1.83 and 1.92")
print("\n=== 5. BMI gradient as shown by the abstract and by the Results ===")
for s in ["luminal A-like", "luminal B-ERBB2-like"]:
seq = [RES[(b, s)][0] for b in ["healthy weight", "overweight", "obesity"]]
print(f" Results {s:22s} healthy -> overweight -> obesity: {seq}; monotone decreasing: "
f"{all(x > y for x, y in zip(seq, seq[1:]))}")
print(f" Abstract 'overweight' 1.40 < 'obesity' 1.68: looks non-monotone: {1.40 < 1.68}")
print("\n=== 6. Significance unchanged ===")
for k in [("overweight", "luminal A-like"), ("overweight", "luminal B-ERBB2-like"),
("obesity", "luminal A-like"), ("obesity", "luminal B-ERBB2-like")]:
o, lo, hi = RES[k]
print(f" {k[0]:11s} {k[1]:22s} CI excludes 1: {lo > 1} z = {math.log(o)/se_log(lo, hi):.2f}")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle, FancyArrowPatch
from matplotlib.lines import Line2D
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "font.family": "DejaVu Sans",
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
# ---- Figure 1: abstract vs Results text diagram
fig = plt.figure(figsize=(7.2, 4.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(-7, 62); ax.axis("off")
ax.add_patch(FancyBboxPatch((1.5, 6), 34.5, 50, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="white", ec=ORA, lw=2))
ax.text(4, 52.5, "Abstract, Results", fontsize=11.5, weight="bold", va="top")
ax.text(4, 47.5, "As labelled in the abstract", fontsize=10.5, color=ORA, weight="bold", va="top")
ax.text(4, 42, "\u201cThese associations were\nattenuated but remained for\nindividuals with \u2026\u201d",
fontsize=10.5, style="italic", va="top", linespacing=1.35)
ax.text(4, 27.6, "overweight", fontsize=10.5, color=SEC, va="center")
ax.text(4, 23.6, "OR 1.40 (1.02\u20131.92)", fontsize=11, color=ORA, weight="bold", va="center")
ax.text(4, 16.6, "obesity", fontsize=10.5, color=SEC, va="center")
ax.text(4, 12.6, "OR 1.68 (1.01\u20132.78)", fontsize=11, color=ORA, weight="bold", va="center")
ax.text(4, 8.4, "No subtype named for either", fontsize=10, color=SEC, va="center")
# Results grid (row labels on the right)
x0, cw, ch = 57, 15.5, 11
ax.text(x0 + cw, 58, "Results text, para. 3", fontsize=11.5, weight="bold", ha="center", va="top")
ax.text(x0 + cw, 53.5, "(identical in eTable 3)", fontsize=10.5, color=BLU, ha="center", va="top")
ax.text(x0 + cw / 2, 45, "Luminal A\u2013like", ha="center", fontsize=10.5, color=SEC)
ax.text(x0 + 1.5 * cw, 45, "Luminal B\u2013\nERBB2-like", ha="center", fontsize=10.5, color=SEC,
linespacing=1.1)
rows = [("overweight", 32), ("obesity", 21)]
for b, y in rows:
ax.text(x0 + 2 * cw + 1.2, y + ch / 2, b.capitalize(), ha="left", va="center", fontsize=10.5, color=SEC)
for j, s in enumerate(["luminal A-like", "luminal B-ERBB2-like"]):
o, lo, hi = RES[(b, s)]
ax.add_patch(Rectangle((x0 + j * cw, y), cw, ch, fc="#e8f0fb" if b == "overweight" else "white",
ec=BLU, lw=2 if b == "obesity" else 1.2))
ax.text(x0 + j * cw + cw / 2, y + ch * 0.64, f"{o:.2f}", ha="center", va="center",
fontsize=12, weight="bold", color=BLU)
ax.text(x0 + j * cw + cw / 2, y + ch * 0.28, f"({lo:.2f}\u2013{hi:.2f})", ha="center",
va="center", fontsize=10, color=SEC)
ax.text(x0 - 1.5, 37.5, "Overweight row\n(shaded): not in\nthe abstract", ha="right", va="center",
fontsize=10, color=SEC, linespacing=1.2)
ax.text(x0 + 2 * cw + 1.2, 16, "Current EPT\nvs never use", ha="left", va="top", fontsize=10,
color=SEC, linespacing=1.2)
# connectors
ax.add_patch(FancyArrowPatch((32, 23.6), (x0 - 0.3, 26.5), arrowstyle="-|>", mutation_scale=13,
lw=2, color=TXT, shrinkA=2, shrinkB=2))
ax.add_patch(FancyArrowPatch((32, 12.6), (x0 + 1.5 * cw, 21 - 0.3), arrowstyle="-|>", mutation_scale=13,
lw=2, color=TXT, connectionstyle="arc3,rad=0.3", shrinkA=2, shrinkB=2))
ax.text(50, -1.8, "Conflict: both abstract values are the obesity-row estimates. The value labelled\n"
"\u201coverweight\u201d is the obesity, luminal A\u2013like OR; the overweight luminal A\u2013like\n"
"OR is 2.18 (1.83\u20132.61) and the overweight luminal B\u2013ERBB2-like OR is 1.83 (1.32\u20132.55).",
ha="center", va="center", fontsize=10, linespacing=1.3)
save(fig, "fig1")
# ---- Figure 2: dot-and-interval plot, abstract labels vs Results values, by BMI
fig, ax = plt.subplots(figsize=(7.2, 4.8), facecolor=BG)
ax.set_facecolor(BG)
rows = [] # (label for y group, kind, value, text)
plan = [
("healthy weight", [("res", "luminal A-like"), ("res", "luminal B-ERBB2-like"),
("abs", 0, "abstract, luminal A\u2013like"), ("abs", 1, "abstract, luminal B\u2013ERBB2-like")]),
("overweight", [("res", "luminal A-like"), ("res", "luminal B-ERBB2-like"),
("abs", 2, "abstract, \u201coverweight\u201d")]),
("obesity", [("res", "luminal A-like"), ("res", "luminal B-ERBB2-like"),
("abs", 3, "abstract, \u201cobesity\u201d")]),
]
mk = {"luminal A-like": "o", "luminal B-ERBB2-like": "s"}
y, step, gap, centers = 0.0, 0.34, 0.55, {}
for b, items in plan:
ys = []
for it in items:
if it[0] == "res":
o, lo, hi = RES[(b, it[1])]
col, m, lab = BLU, mk[it[1]], f"{o:.2f} {it[1].replace('-like', '')}"
else:
o, lo, hi = ABS[it[1]][2]
col, m, lab = ORA, "D", f"{o:.2f} {it[2]}"
ax.plot([lo, hi], [-y, -y], color=col, lw=2, solid_capstyle="butt")
ax.plot(o, -y, marker=m, ms=8, color=col, ls="none")
ax.text(hi * 1.03, -y, lab, va="center", fontsize=10, color=col)
ys.append(-y); y += step
centers[b] = np.mean(ys); y += gap
ax.axvline(1, color=SEC, lw=0.8)
ax.set_xscale("log")
ax.set_xlim(0.9, 7.0)
ticks = [1, 1.5, 2, 3, 4]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks]); ax.minorticks_off()
ax.set_yticks(list(centers.values()))
ax.set_yticklabels(["Healthy\nweight", "Overweight", "Obesity"], fontsize=10.5)
ax.set_ylim(-y + gap - 0.3, 0.3)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, current EPT use vs never use (log scale; 95% CI)", fontsize=10.5)
h = [Line2D([], [], color=BLU, marker="o", ms=8, lw=2, label="Results text / eTable 3, luminal A\u2013like"),
Line2D([], [], color=BLU, marker="s", ms=8, lw=2, label="Results text / eTable 3, luminal B\u2013ERBB2-like"),
Line2D([], [], color=ORA, marker="D", ms=8, lw=2, label="Abstract, under the BMI group it names")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.45, -0.15), ncol=1, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png in", HERE)
The abstract's two interval cancer rates each pair an estimate with a mismatched interval
JAMA Netw Open (2022) · doi:10.1001/jamanetworkopen.2022.44212 · PMC9716401
Summary
The Abstract and Results give the pooled interval cancer rate, the paper's secondary outcome, as 3.73 per 1000 examinations in both groups: with a 95% CI of 0.84-3.98 after mastectomy without reconstruction, and 0.41-2.73 after mastectomy with reconstruction. The second estimate lies outside its own interval, and that interval is the one printed for the clinically occult rate in the same group (1.06; 0.41-2.73). In the first group, the printed interval, the printed odds ratio and the paper's own Figure 3 all put the estimate near 1.8, not 3.73. Figure 3 plots the with-reconstruction estimate at 3.73 with an interval of about 1.8-7.8. So the text appears to have swapped two numbers, and the paper's own figure shows the values that belong there. The paper's main conclusion, which rests on clinically occult rates, is not affected. But the secondary outcome is misreported in the Abstract, and no correction has been issued.
What the paper reports
Abstract, Results:
For mastectomy without reconstruction, the rate of clinically occult (nonpalpable) cancer per 1000 examinations (2.96; 95% CI, 1.38-6.32) and the interval cancer rate per 1000 examinations (3.73; 95% CI, 0.84-3.98) were lower than the overall cancer detection rate (including both palpable and nonpalpable lesions) per 1000 examinations (6.41; 95% CI, 3.09-13.25) across all imaging modalities.
The interval cancer rate per 1000 examinations for mastectomy with reconstruction (3.73; 95% CI, 0.41-2.73) was comparable to the pooled cancer detection rate per 1000 examinations (4.73; 95% CI, 2.32-9.63) across all imaging modalities.
Results, "Cancer Detection Rates After Mastectomy With or Without Reconstruction", first paragraph:
For mastectomy without reconstruction, the pooled overall cancer detection rate per 1000 examinations was 6.41 (95% CI, 3.09-13.25), whereas the pooled rate of interval cancer per 1000 examinations was 3.73 (95% CI, 0.84-3.98), and the rate of clinically occult (nonpalpable) cancer was 2.96 (95% CI, 1.38-6.32) per 1000 examinations.
Same section, second paragraph:
The pooled interval cancer rate per 1000 examinations (3.73; 95% CI, 0.41-2.73) was comparable to the pooled cancer detection rate observed among patients who underwent mastectomy with reconstruction. The rate of clinically occult (nonpalpable) cancer per 1000 examinations was 1.06 (95% CI, 0.41-2.73).
Results, "Comparisons Among Surveillance Methods and Outcomes":
For mastectomy without reconstruction, the OR for occult cancer was 0.46 (95% CI, 0.30-0.70; P < .001) times the overall cancer detection rate, and the OR for interval cancer was 0.28 (95% CI, 0.18-0.46; P < .001) times the overall cancer detection rate.
However, the corresponding ORs were not statistically significant for mastectomy with reconstruction; the OR for occult cancer was 0.36 (95% CI, 0.08-1.67; P = .08) times the overall cancer detection rate, and the OR for interval cancer was 2.04 (95% CI, 0.53-7.87; P = .17) times the overall cancer detection rate.
Methods, "Outcomes and Definitions" and "Data Synthesis":
The secondary outcome was the pooled interval cancer rate, defined as ipsilateral cancer detected after a normal imaging procedure but before any subsequent imaging.
We used the generalized linear mixed model framework to meta-analyze the proportion of cases across studies.
Pooled estimates and 95% CIs from the model were presented as the number of cases per 1000 examinations.
Discussion:
Considering interval cancer rates, in the clinical scenario of mastectomy with reconstruction, the interval cancer rate was similar to the overall cancer detection rate, whereas this was not the case among patients who underwent mastectomy without reconstruction, where interval cancer rates were lower than overall cancer detection rates. This difference in trends can be explained by the higher cancer detection rates among patients who underwent mastectomy without reconstruction in general rather than by a true difference in the interval cancer rates between the 2 clinical scenarios.
Figure 3 is titled "Pooled Cancer Detection Rates, Averaged Over Imaging Modality, Among Patients After Mastectomy Without or With Reconstruction". It plots the interval, clinically occult and overall rates for each group as markers with 95% CI whiskers on a linear axis from 0 to 20.
What the paper's own numbers show
Method. I used three checks, all based only on numbers the paper prints or plots.
- Interval centre. The rates are pooled with a binomial generalized linear mixed model (GLMM). Its 95% CIs are symmetric on the logit scale. At rates of a few per 1000, that makes them almost exactly symmetric on the log scale, so the estimate should sit at the CI's geometric midpoint, √(lo × hi). For the seven printed rates other than the two interval rates, this rule recovers the printed estimate to within 0.03 per 1000. On the log scale every one of them sits within 0.004 half-widths of its CI centre.
- Odds-ratio cross-check. Multiplying the odds of the overall rate by a printed odds ratio gives the rate that odds ratio implies. Without reconstruction, this reproduces the printed occult rate exactly (0.46 with 6.41 gives 2.96).
- Figure 3. I measured the plotted marker centres and whisker ends from the pixels of Figure 3 as archived in PMC. The axis was calibrated by a linear fit to the ticks at 0, 5, 10, 15 and 20; one pixel is 0.095 per 1000. For the four occult and overall rates (two per group), the measured values agree with the printed text to within about one pixel (at most 0.07).
Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Interval rate, with reconstruction: is the estimate inside its CI? | Abstract; Results p2 | 3.73 (0.41-2.73) | No. 3.73 is 1.00 above the upper limit; rounding can move each value by at most 0.005 |
| Centre of that CI, √(0.41 × 2.73) | Abstract; Results p2 | n/a | 1.06, which equals the printed occult rate for the same group, 1.06 (0.41-2.73) |
| Interval rate, with reconstruction, Figure 3 | Figure 3 | plotted | 3.73 (1.78-7.78); the centre of that whisker is 3.72 |
| Interval rate, without reconstruction: centre of CI √(0.84 × 3.98) | Abstract; Results p1 | 3.73 (0.84-3.98) | 1.83. 3.73 sits 0.92 half-widths (log scale) above the centre, next to the upper limit |
| Same rate implied by OR 0.28 × overall rate 6.41 | Results, "Comparisons" | OR 0.28 (0.18-0.46) | 1.80 |
| Same rate, Figure 3 | Figure 3 | plotted | 1.83 (0.83-3.97) |
| Occult rate, without reconstruction | Abstract; Results p1 | 2.96 (1.38-6.32) | CI centre 2.95; OR 0.46 × 6.41 gives 2.96; Figure 3 shows 2.93 (1.40-6.35) |
| Overall rate, without reconstruction | Abstract; Results p1 | 6.41 (3.09-13.25) | CI centre 6.40; Figure 3 shows 6.36 (3.02-13.21) |
| Occult rate, with reconstruction | Results p2; Additional Analyses | 1.06 (0.41-2.73) | CI centre 1.06; Figure 3 shows 1.07 (0.35-2.73) |
| Overall rate, with reconstruction | Abstract; Results p2 | 4.73 (2.32-9.63) | CI centre 4.73; Figure 3 shows 4.74 (2.35-9.69) |
| Overall rate by modality (mammography, ultrasonography, MRI) | Abstract; Results | 1.86 (1.05-3.30); 2.66 (1.48-4.76); 5.17 (1.49-17.75) | CI centres 1.86, 2.65, 5.14 (logit-scale centres 1.86, 2.66, 5.17) |
The two interval rates are the only printed rates that fail the centre check. Each fails in a different way. With reconstruction, the estimate 3.73 is confirmed by Figure 3, but the printed interval is the occult-rate interval. Without reconstruction, the printed interval is confirmed by Figure 3 (to within one pixel), but the estimate should be about 1.8. Three independent routes give that value: 1.83, 1.80 and 1.83. The value 3.73 printed for this group is the with-reconstruction estimate.
Figure
Figure 1. Pooled interval cancer rates per 1000 examinations, with 95% CIs, on one linear axis. Orange circles and bars are the rates as printed in the Abstract and Results. Blue diamonds and bars are the rates as plotted in the paper's own Figure 3, measured from its pixels (resolution about 0.1). Without reconstruction (top), the printed 3.73 sits at the top of its interval. Figure 3 and two recomputations from printed numbers put it at about 1.8: the centre of the printed CI, 1.83 (open blue circle), and the printed odds ratio 0.28 applied to the printed overall rate 6.41, which gives 1.80 (open blue triangle). With reconstruction (bottom), the printed estimate of 3.73 lies outside its printed interval. That interval matches, digit for digit, the interval printed for the clinically occult rate in the same group (grey square, 1.06; dotted guides). Figure 3 plots the interval rate at 3.73 with an interval of about 1.78-7.78. The axis starts at zero and no difference is exaggerated.
Why it matters
- The secondary outcome is misreported in the Abstract. The paper names the pooled interval cancer rate as its secondary outcome, and both group values appear in the Abstract. One of them is impossible as printed (an estimate outside its CI). The other is about twice the value that the paper's own interval, odds ratio and figure support. A reader citing the Abstract, or entering these rates in a later meta-analysis, would take a wrong number in either group: an interval of 0.41-2.73 that belongs to another outcome, or an estimate of 3.73 instead of about 1.8.
- It makes the two groups look identical. As printed, both groups have an interval cancer rate of 3.73. That fits the Discussion's reading that the different trends reflect higher detection rates without reconstruction "rather than by a true difference in the interval cancer rates between the 2 clinical scenarios". By the paper's own figure, the interval rate is about twice as high with reconstruction: 3.73 against about 1.8. The intervals overlap (about 1.8-7.8 and 0.84-3.98), so this is not evidence of a real difference. But the printed numbers show two identical estimates where the model's estimates are not identical.
- The qualitative statements survive. With the figure's values, the without-reconstruction interval rate (about 1.8) is still lower than the overall rate (6.41), and the odds ratio test (0.28, P < .001) is unchanged. With reconstruction, 3.73 against 4.73 still reads as "comparable". The headline conclusion compares clinically occult with overall detection rates, and all of those numbers check out. The error is in the reported size and precision of the secondary outcome, not in the main finding.
Explanations considered
- Rounding. Ruled out. The estimate exceeds the upper limit by 1.00 per 1000. Two-decimal rounding can account for at most 0.01.
- A different kind of interval (for example a prediction interval, or a CI on another scale). Not supported. Every other printed rate sits at the geometric centre of its CI, as a logit-scale GLMM interval should. Figure 3 plots an ordinary interval around 3.73 that contains it. And the interval 0.41-2.73 matches the occult-rate interval digit for digit.
- The text is right and Figure 3 is wrong. Not supported. For the four occult and overall rates, Figure 3 matches the text to within about one pixel. For the without-reconstruction interval rate, the figure agrees with two calculations that do not use the figure: the centre of the printed CI (1.83) and the printed odds ratio (1.80). The text estimate of 3.73 agrees with neither.
- The two groups truly have the same interval rate, 3.73. Not supported. The without-reconstruction interval 0.84-3.98 has its centre at 1.83, and the odds ratio 0.28 relative to 6.41 implies 1.80. Neither is compatible with 3.73 as the model estimate.
- A text-conversion artifact in PMC. Ruled out. The publisher's HTML page (jamanetwork.com, fullarticle/2799117) shows the same numbers in the Abstract and Results, as does the Europe PMC full-text XML.
- A correction already issued. None found (see Status).
- Side observations, not part of the main finding. (1) With reconstruction, the printed odds ratios do not reproduce the printed rates, although they do without reconstruction. The ratio 3.73/4.73 is 0.79, but the OR of 2.04 implies an interval rate of 9.60. The ratio 1.06/4.73 is 0.22, but the OR of 0.36 implies an occult rate of 1.71. Figure 3 matches the printed rates (3.73, 1.07, 4.74), not those implied by the ORs, so the figure supports correcting the interval and not the estimate. The odds ratios themselves cannot be checked from the published material. (2) The Methods say the significance tests are 2-tailed. Yet the four nonsignificant P values (.08, .17, .06, .15) are each roughly half the two-sided Wald P implied by their own odds ratio and 95% CI (0.19, 0.30, 0.15, 0.28). The 95% CIs and P values may come from different procedures. These points need the model output to resolve, and they are outside the claim of this write-up.
Status
- Version of record. Checked. The publisher's HTML (reached through doi.org; jamanetwork.com/journals/jamanetworkopen/fullarticle/2799117) gives "3.73; 95% CI, 0.84-3.98" and "3.73; 95% CI, 0.41-2.73" in the Abstract, and the same values in the Results. It carries no correction notice. The PMC text and the Europe PMC full-text XML agree. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The one supplement (jamanetwopen-e2244212-s001.pdf, 6 pages: eMethods, eReferences, eTable 1 QUADAS-2, eTable 2 per-study counts) gives no pooled estimates. eTable 2 does not mark each study's reconstruction status, so the GLMM could not be refitted from the supplement. The Figure 3 image (jamanetwopen-e2244212-g003.jpg, from the PMC/Europe PMC figure files) is the source of the plotted values.
recompute.py.txtre-measures them when given that image. - Corrections and comments. As of 2026-10-24, no correction, erratum, retraction or expression of concern was found.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has empty update-to and updated-by fields and no relations (27 citations). The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC9716401; Europe PMC full-text XML and supplementary files; publisher HTML at https://doi.org/10.1001/jamanetworkopen.2022.44212; Crossref; PubPeer.
Suggested correction
In the Abstract Results and in the first Results paragraph on reconstruction status, the interval cancer rate for mastectomy without reconstruction should read approximately
1.8 (95% CI, 0.84-3.98)
instead of 3.73 (95% CI, 0.84-3.98). The exact value should come from the model output; Figure 3, the CI centre and the odds ratio give 1.83, 1.83 and 1.80.
In the Abstract Results and in the second Results paragraph, the interval cancer rate for mastectomy with reconstruction should read
3.73 (95% CI, approximately 1.8-7.8)
instead of 3.73 (95% CI, 0.41-2.73), with the exact limits taken from the model output. The interval 0.41-2.73 should stay only with the clinically occult rate, 1.06.
The authors may also wish to check the with-reconstruction odds ratios (0.36 and 2.04) against the pooled rates they compare, and to confirm that the reported P values are two-sided, as the Methods state.
Recomputation script (python recompute.py.txt)
"""Recompute the checks in writeup.md for PMC9716401
(JAMA Netw Open 2022; doi:10.1001/jamanetworkopen.2022.44212).
Usage:
python recompute.py.txt # print every number, redraw fig1.svg / fig1.png
python recompute.py.txt path/to/g003.jpg # additionally re-measure Figure 3 from the image
The optional image is the article's Figure 3 as distributed by PMC / Europe PMC
(file jamanetwopen-e2244212-g003.jpg, 780 x 187 px). It is not redistributed here.
Without it, the script uses the values measured from that image (FIG3_MEASURED below).
Requires numpy, scipy, matplotlib (and Pillow for the optional image step).
"""
import math
import os
import sys
import numpy as np
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------------------
# 1. Pooled rates per 1000 examinations as printed in the text (point, lo, hi)
# ---------------------------------------------------------------------------
REPORTED = {
"No recon: any cancer (overall CDR)": (6.41, 3.09, 13.25),
"No recon: clinically occult": (2.96, 1.38, 6.32),
"No recon: interval cancer": (3.73, 0.84, 3.98),
"Recon: any cancer (overall CDR)": (4.73, 2.32, 9.63),
"Recon: clinically occult": (1.06, 0.41, 2.73),
"Recon: interval cancer": (3.73, 0.41, 2.73),
"Mammography: any cancer": (1.86, 1.05, 3.30),
"Ultrasonography: any cancer": (2.66, 1.48, 4.76),
"MRI: any cancer": (5.17, 1.49, 17.75),
}
# Odds ratios printed in Results, "Comparisons Among Surveillance Methods and Outcomes"
REPORTED_OR = {
"No recon: occult vs overall": (0.46, 0.30, 0.70, "<.001"),
"No recon: interval vs overall": (0.28, 0.18, 0.46, "<.001"),
"Recon: occult vs overall": (0.36, 0.08, 1.67, ".08"),
"Recon: interval vs overall": (2.04, 0.53, 7.87, ".17"),
"Ultrasonography vs mammography": (1.43, 0.88, 2.33, ".06"),
"MRI vs mammography": (2.79, 0.44, 17.84, ".15"),
}
# Values measured from the pixels of the article's Figure 3 (marker centre, whisker ends),
# axis calibrated by a least-squares fit to the tick marks at 0, 5, 10, 15, 20
# (one pixel = 0.095 per 1000 examinations).
# Re-measured (and checked against these) when the image path is passed.
FIG3_MEASURED = {
"No recon: interval cancer": (1.83, 0.83, 3.97),
"No recon: clinically occult": (2.93, 1.40, 6.35),
"No recon: any cancer (overall CDR)": (6.36, 3.02, 13.21),
"Recon: interval cancer": (3.73, 1.78, 7.78),
"Recon: clinically occult": (1.07, 0.35, 2.73),
"Recon: any cancer (overall CDR)": (4.74, 2.35, 9.69),
}
def logit(p):
return math.log(p / (1 - p))
def expit(x):
return 1 / (1 + math.exp(-x))
def logit_mid(lo, hi):
"""Centre of a CI that is symmetric on the logit scale (as a GLMM binomial CI is)."""
return 1000 * expit((logit(lo / 1000) + logit(hi / 1000)) / 2)
def implied_rate(rate_any, OR):
"""Rate per 1000 implied by an odds ratio relative to the overall rate."""
o = rate_any / 1000 / (1 - rate_any / 1000) * OR
return 1000 * o / (1 + o)
print("=" * 78)
print("1) Does each printed CI contain its point estimate, and where is its centre?")
print(" GLMM binomial CIs are symmetric on the logit scale; at rates of a few per")
print(" 1000 that is almost symmetric on the log scale, so sqrt(lo*hi) ~ estimate.")
print("=" * 78)
print(f" {'quantity':36s} {'printed':>20s} {'inside':>7s} {'sqrt(lo*hi)':>11s} {'logit mid':>9s}")
for k, (p, lo, hi) in REPORTED.items():
inside = lo <= p <= hi
print(f" {k:36s} {p:6.2f} ({lo:5.2f}-{hi:5.2f}) {str(inside):>7s} "
f"{math.sqrt(lo * hi):11.2f} {logit_mid(lo, hi):9.2f}")
p, lo, hi = REPORTED["Recon: interval cancer"]
print(f"\n Recon interval: estimate exceeds upper limit by {p - hi:.2f} per 1000.")
print(f" Largest true upper limit that rounds to {hi}: {hi + 0.005:.3f}; smallest estimate"
f" that rounds to {p}: {p - 0.005:.3f} -> rounding cannot explain it.")
p, lo, hi = REPORTED["No recon: interval cancer"]
print(f" No-recon interval: 3.73 is {math.log(p / math.sqrt(lo * hi)) / math.log(hi / math.sqrt(lo * hi)):.2f}"
f" half-widths (log scale) above the CI centre; for the seven rates other than the two interval rates it is"
f" within +/-{max(abs(math.log(v[0] / math.sqrt(v[1] * v[2])) / math.log(v[2] / math.sqrt(v[1] * v[2]))) for kk, v in REPORTED.items() if 'interval' not in kk):.3f}.")
print("\n" + "=" * 78)
print("2) Rates implied by the printed odds ratios x odds of the overall rate")
print("=" * 78)
cdr_nr = REPORTED["No recon: any cancer (overall CDR)"][0]
cdr_r = REPORTED["Recon: any cancer (overall CDR)"][0]
rows = [
("No recon: clinically occult", cdr_nr, 0.46),
("No recon: interval cancer", cdr_nr, 0.28),
("Recon: clinically occult", cdr_r, 0.36),
("Recon: interval cancer", cdr_r, 2.04),
]
IMPLIED = {}
for k, base, OR in rows:
v = implied_rate(base, OR)
IMPLIED[k] = v
print(f" {k:30s} OR {OR:4.2f} x CDR {base:5.2f} -> {v:5.2f} printed {REPORTED[k][0]:5.2f}"
f" printed/CDR = {REPORTED[k][0] / base:4.2f}")
print("\n" + "=" * 78)
print("3) Two-sided Wald P values implied by each printed OR and its 95% CI")
print("=" * 78)
for k, (o, lo, hi, P) in REPORTED_OR.items():
se = (math.log(hi) - math.log(lo)) / (2 * norm.ppf(0.975))
z = math.log(o) / se
p2 = 2 * norm.sf(abs(z))
asym = math.log(o) - (math.log(lo) + math.log(hi)) / 2
print(f" {k:32s} z = {z:5.2f} P(2-sided) = {p2:.3f} half = {p2 / 2:.3f} printed P {P:>6s}"
f" (CI log-asymmetry {asym:+.3f})")
# ---------------------------------------------------------------------------
# 4. Figure 3 (optional re-measurement from the image)
# ---------------------------------------------------------------------------
print("\n" + "=" * 78)
print("4) Values plotted in the article's Figure 3 (marker, whisker ends)")
print("=" * 78)
def measure_fig3(path):
from PIL import Image
im = np.array(Image.open(path).convert("RGB")).astype(float)
g = im.mean(axis=2)
H, W = g.shape
def runs(xs):
out = []
if len(xs) == 0:
return out
s = p = xs[0]
for x in xs[1:]:
if x != p + 1:
out.append((s, p))
s = x
p = x
out.append((s, p))
return out
# axis line: row with the most dark pixels in the lower half; ticks just below it
axis_y = max(range(H // 2, H), key=lambda y: (g[y] < 170).sum())
tick_x = [(a + b) / 2 for a, b in runs(np.where(g[axis_y + 2] < 170)[0])]
assert len(tick_x) == 10, tick_x
panels = [tick_x[:5], tick_x[5:]]
cal = [np.polyfit(t, [0, 5, 10, 15, 20], 1) for t in panels]
# markers: dark teal squares (green channel > red channel)
teal = ((im[:, :, 1] - im[:, :, 0]) > 15) & (g < 150)
bands = runs(np.where(teal.any(axis=1))[0])
names = ["interval cancer", "clinically occult", "any cancer (overall CDR)"]
out = {}
for (y0, y1), nm in zip(bands, names):
yc = (y0 + y1) // 2
for pi, (lab, t) in enumerate(zip(["No recon", "Recon"], panels)):
xlo, xhi = int(t[0]) - 2, int(t[-1]) + 2
ys, xs = np.where(teal[y0:y1 + 1, xlo:xhi])
xc = xs.mean() + xlo
# whisker: columns whose darkest pixel within the marker band is darker than
# halfway between the white background and the grey line (anti-aliased end
# pixels that are less than half covered are excluded); ends at pixel edges
line = np.where(g[y0:y1 + 1, xlo:xhi].min(axis=0) < 200)[0] + xlo
seg = max(runs(line), key=lambda r: r[1] - r[0])
f = lambda x: float(np.polyval(cal[pi], x))
out[f"{lab}: {nm}"] = (round(f(xc), 2), round(f(seg[0] - 0.5), 2), round(f(seg[1] + 0.5), 2))
return out
FIG3 = dict(FIG3_MEASURED)
if len(sys.argv) > 1:
meas = measure_fig3(sys.argv[1])
for k, v in meas.items():
ref = FIG3_MEASURED[k]
assert all(abs(a - b) < 0.06 for a, b in zip(v, ref)), (k, v, ref)
FIG3 = meas
print(" (re-measured from image; agrees with stored values to within 0.05)")
else:
print(" (stored measurements; pass the image path to re-measure)")
for k, (p, lo, hi) in FIG3.items():
rp = REPORTED[k]
print(f" {k:36s} figure {p:5.2f} ({lo:5.2f}-{hi:5.2f}) text {rp[0]:5.2f} ({rp[1]:5.2f}-{rp[2]:5.2f})"
f" figure sqrt(lo*hi) = {math.sqrt(lo * hi):5.2f}")
print(" Ratio of figure values, recon interval / recon overall: "
f"{FIG3['Recon: interval cancer'][0] / FIG3['Recon: any cancer (overall CDR)'][0]:.2f}")
# ---------------------------------------------------------------------------
# 5. Figure 1 of the write-up
# ---------------------------------------------------------------------------
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"axes.edgecolor": INK2, "xtick.color": INK2, "ytick.color": INK2})
nr_int = REPORTED["No recon: interval cancer"]
r_int = REPORTED["Recon: interval cancer"]
r_occ = REPORTED["Recon: clinically occult"]
nr_mid = math.sqrt(nr_int[1] * nr_int[2])
f_nr, f_r = FIG3["No recon: interval cancer"], FIG3["Recon: interval cancer"]
nr_or = IMPLIED["No recon: interval cancer"]
# rows: (y, row label, (point, lo, hi), colour, marker)
rows = [
(6.2, "Printed in Abstract and Results", nr_int, ORANGE, "o"),
(5.4, "Plotted in the paper's Figure 3", f_nr, BLUE, "D"),
(2.8, "Printed in Abstract and Results", r_int, ORANGE, "o"),
(2.0, "Printed for the occult rate", r_occ, INK2, "s"),
(1.2, "Plotted in the paper's Figure 3", f_r, BLUE, "D"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
ax.set_facecolor(BG)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
lab_tr = matplotlib.transforms.blended_transform_factory(fig.transFigure, ax.transData)
LX = 0.015
for y, lab, (p, lo, hi), col, mk in rows:
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=2)
for e in (lo, hi):
ax.plot([e, e], [y - 0.14, y + 0.14], color=col, lw=2, zorder=2)
ax.plot([p], [y], marker=mk, ms=9, color=col, mec=BG, mew=1.5, zorder=4)
note = ": estimate outside its CI" if not lo <= p <= hi else ""
ax.text(max(hi, p) + 0.3, y, f"{p:.2f} ({lo:.2f}–{hi:.2f}){note}", va="center", ha="left",
color=INK, fontsize=10)
ax.text(LX + 0.012, y, lab, va="center", ha="left", color=INK2, fontsize=10, transform=lab_tr)
# arithmetic cross-checks for the no-recon estimate
y = 4.6
ax.plot([nr_mid], [y], marker="o", ms=9, mfc="none", mec=BLUE, mew=2, zorder=3)
ax.plot([nr_or], [y], marker="^", ms=9, mfc="none", mec=BLUE, mew=2, zorder=3)
ax.text(2.25, y, f"{nr_mid:.2f} (CI centre); {nr_or:.2f} (OR 0.28 × rate 6.41)",
va="center", ha="left", color=INK, fontsize=10)
ax.text(LX + 0.012, y, "Recomputed from printed numbers", va="center", ha="left", color=INK2,
fontsize=10, transform=lab_tr)
# group headings
ax.text(LX, 7.0, "Mastectomy without reconstruction: interval cancer rate", ha="left", va="center",
color=INK, fontsize=10.5, fontweight="bold", transform=lab_tr)
ax.text(LX, 3.6, "Mastectomy with reconstruction: interval cancer rate", ha="left", va="center",
color=INK, fontsize=10.5, fontweight="bold", transform=lab_tr)
# the printed point estimate lies outside its printed CI
for e in (0.41, 2.73):
ax.plot([e, e], [2.0, 2.8], color=INK2, lw=0.8, ls=(0, (2, 2)), zorder=1)
ax.set_xlim(0, 12)
ax.set_ylim(0.6, 7.35)
ax.set_yticks([])
for s in ["left", "right", "top"]:
ax.spines[s].set_visible(False)
ax.set_xlabel("Pooled rate per 1000 examinations, with 95% CI", color=INK, fontsize=10)
leg = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, mec=BG, label="As printed in the text"),
Line2D([], [], color=BLUE, lw=2, marker="D", ms=8, mec=BG, label="As plotted in Figure 3 (measured)"),
Line2D([], [], color=BLUE, lw=0, marker="o", ms=8, mfc="none", mew=2, label="Recomputed: centre of printed CI"),
Line2D([], [], color=BLUE, lw=0, marker="^", ms=8, mfc="none", mew=2, label="Recomputed: printed OR × overall rate")]
fig.legend(handles=leg, loc="upper left", bbox_to_anchor=(LX - 0.005, 0.995), ncol=2, frameon=False,
fontsize=10, labelcolor=INK, handlelength=2.2, columnspacing=1.5)
fig.subplots_adjust(left=0.36, right=0.98, top=0.85, bottom=0.11)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
print("\nWrote fig1.svg and fig1.png to", HERE)
Abstract reports a diastolic blood pressure difference that the trial's own analyses do not show
JAMA Network Open (2021) · doi:10.1001/jamanetworkopen.2021.12417 · PMC8176331
Summary
This randomized trial of a smartphone lifestyle app (204 adults with type 2 diabetes in Singapore) lists diastolic blood pressure in its abstract among the outcomes with "intergroup differences favoring the intervention". The paper's Table 2 gives the diastolic between-group differences as −2.6 mm Hg (95% CI −6.2 to 0.9; P = .15) at 3 months and −2.4 mm Hg (95% CI −5.7 to 1.0; P = .17) at 6 months, and the complete-case analysis in Supplement 1 gives P = .13 and .08. All four intervals include zero, recomputation from the printed means and SDs agrees with them, and the paper's own Results text says that between-group differences in blood pressure were not observed at 6 months. The primary outcome (weight) and the glycemic results are unaffected, but the abstract credits the app with a blood pressure benefit that none of the paper's reported analyses supports at its stated threshold of 2-sided P < .05.
What the paper reports
Abstract, Results (the sentence follows the 6-month weight, HbA1c and medication results):
Intergroup differences favoring the intervention were also noted for fasting blood glucose, diastolic blood pressure, and dietary changes.
Results, "Metabolic Outcomes":
Between-group differences in blood pressure and lipids were not observed at 6 months.
Table 2, "Change in diastolic blood pressure, mm Hg" (blood pressure block: control group, 72 participants; intervention group, 67 participants). Columns: participants, control mean (SD) change, intervention mean (SD) change, mean difference (95% CI), P value, Cohen d. Superscript b is reproduced as "b":
3 mo | 139 | -1.8 (11.0) | -4.4 (9.9)b | -2.6 (-6.2 to 0.9) | .15 | 0.25
6 mo | 139 | -3.1 (9.2)b | -5.4 (11.1)b | -2.4 (-5.7 to 1.0) | .17 | 0.23
Table 2, footnote b (it marks within-group change from baseline, not between-group differences):
Statistically significant change from baseline to postintervention at P < .05, after Benjamini-Hochberg correction with false discovery rate at 0.20 and with 76 participants.
Supplement 1, eTable 1 (complete-case analysis), "∆ Diastolic blood pressure (mmHg)", as extracted from the PDF:
3 months 132 -1.6 (10.6) -4.1 (8.8)* -2.6 (-5.9 to 0.8) .13 0.26
6 months 131 -2.5 (7.9)* -5.4 (10.9)* -2.9 (-6.2 to 0.4) .08 0.30
Methods, "Statistical Analysis":
Type I error for multiple comparisons was adjusted using the Benjamini-Hochberg procedure with false discovery rate at 0.20.
Statistical significance was set at 2-sided P < .05.
Results, "Participants":
Baseline characteristics were similar between study groups, except for a significantly higher diastolic blood pressure in the control group.
What the paper's own numbers show
Method. The paper fitted generalized linear mixed models to change from baseline on 5 multiply imputed data sets, with recruitment source as a random effect. That model cannot be refitted from published summaries, so I checked the printed numbers against each other in two ways. (a) From the printed mean (SD) changes and group sizes (72 and 67 for blood pressure), I computed the intervention-minus-control difference with a pooled-variance t 95% CI and 2-sided P, a Welch P, and Cohen d. (b) I back-calculated the 2-sided P implied by each printed 95% CI (normal approximation, SE = CI width / 3.92). eTable 1 does not give the group split behind its n = 132 and 131, so its rows assume 66/66 and 66/65; across every split consistent with at most 72 control and 67 intervention participants, the recomputed P ranges only from .142 to .148 (3 mo) and .080 to .085 (6 mo). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Diastolic BP difference, 3 mo (multiple imputation) | Table 2 | −2.6 (−6.2 to 0.9); P = .15; d = 0.25 | −2.60 (−6.12 to 0.92); P = .146 (Welch .145; from printed CI .151); d = 0.25 |
| Diastolic BP difference, 6 mo (multiple imputation) | Table 2 | −2.4 (−5.7 to 1.0); P = .17; d = 0.23 | −2.30 from the rounded arm means (−5.71 to 1.11); P = .185 (Welch .188; from printed CI .160); d = 0.23 |
| Diastolic BP difference, 3 mo (complete case) | Supplement 1, eTable 1 | −2.6 (−5.9 to 0.8); P = .13 | −2.50 (−5.85 to 0.85); P = .143 (from printed CI .128) |
| Diastolic BP difference, 6 mo (complete case) | Supplement 1, eTable 1 | −2.9 (−6.2 to 0.4); P = .08 | −2.90 (−6.19 to 0.39); P = .083 (from printed CI .085) |
| One-sided P implied by the printed CIs (not the paper's convention) | Table 2; eTable 1 | — | .076 and .080 (multiple imputation, 3 and 6 mo); .064 and .042 (complete case) |
| Systolic BP difference, 3 mo / 6 mo (for comparison) | Table 2 | −4.2 (−8.8 to 0.3), P = .07 / −2.8 (−7.7 to 2.0), P = .25 | −4.30 (−8.98 to 0.38), P = .072 / −2.80 (−7.66 to 2.06), P = .257 |
| Systolic BP difference, 3 mo (complete case) | Supplement 1, eTable 1 | −4.7 (−9.3 to −0.2); P = .04 | −4.70 (−9.25 to −0.15); P = .043, the only blood pressure comparison in the paper below .05 |
| Fasting blood glucose difference, 6 mo (named in the same abstract sentence) | Table 2 | −12.6 (−23.4 to −3.6); P = .01 | −12.6 (−21.4 to −3.8); P = .005 |
| Calorie intake difference, 6 mo (named in the same abstract sentence, "dietary changes") | Table 2 | −305.8 (−441.0 to −170.7); P < .001 | −305.5 (−441.1 to −169.9); P < .001 |
| Baseline diastolic BP, control vs intervention | Table 1; Results | 85.7 (9.8) vs 82.7 (8.8); "significantly higher" in control | difference 3.0 mm Hg; t = 2.30, P = .023 |
| Participants with blood pressure data | Table 2 | 139 of 204 randomized | 68.1% |
Both multiple-imputation analyses and both complete-case analyses of diastolic blood pressure have 95% CIs that include zero, and recomputation reproduces each printed P value to within rounding. Of the outcomes named in the abstract sentence, fasting blood glucose and the dietary outcomes have between-group P values below .05 in Table 2. Diastolic blood pressure does not, in any analysis the paper reports.
Figure
Figure 1. The conflicting statements, quoted from the article. Left: the abstract lists diastolic blood pressure among the outcomes with intergroup differences favoring the intervention. Right: the Results text says between-group differences in blood pressure were not observed at 6 months, and Table 2 prints diastolic between-group differences whose 95% CIs include zero at both time points (orange: values printed in the paper). Below (blue): the 2-sided P values recomputed from Table 2's own mean (SD) changes and group sizes (.146 and .185), and those implied by the printed CIs (.151 and .160). The paper sets significance at 2-sided P < .05.
Figure 2. Between-group difference in change in diastolic blood pressure (intervention minus control, mm Hg) with 95% CIs, for every analysis the paper reports: multiple imputation (Table 2) and complete case (Supplement 1, eTable 1). Orange circles and lines are the printed values. Blue squares and lines are recomputed from the printed mean (SD) changes with a pooled-variance t interval; the complete-case rows assume a 66/66 and 66/65 group split, which eTable 1 does not print. The vertical line marks no difference. Every interval, printed or recomputed, crosses it, and the P values at right are all above .05.
Why it matters
Blood pressure is a prespecified secondary outcome (Methods, "Outcomes"). The abstract's Results paragraph is where a reader learns which secondary outcomes differed between arms, and as written it tells that reader that the app improved diastolic blood pressure relative to usual care. The paper's analyses instead show point estimates of about −2.4 to −2.9 mm Hg with intervals running from about −6 to about +1 mm Hg. That result is compatible with no effect, and the paper's own Results text describes it as no difference. Abstracts are what is indexed, screened for systematic reviews and quoted in secondary literature, so a secondary outcome reported there as a benefit can be carried forward as one. The paper has been cited about 88 times. Meta-analysts who extract from Table 2 would get the correct estimate; readers who rely on the abstract would not.
Two further features of the data argue for caution even about the point estimate. Blood pressure was analysed in 139 of 204 randomized participants (68.1%). Baseline diastolic pressure was 3.0 mm Hg higher in the control group (P = .023), an imbalance the paper reports but does not adjust for in any presented analysis.
The error is limited in scope. The title, Key Points and abstract Conclusions mention only weight, glycemic and medication outcomes. The primary outcome (weight change, −2.4 kg at 6 months, P < .001), HbA1c (−0.4%, P = .01) and fasting glucose stand on Table 2. The main-text Conclusions refer more broadly to "significant improvements in weight and multiple metabolic profiles"; of the metabolic outcomes in Table 2, only HbA1c and fasting glucose have between-group P < .05 at 6 months.
Explanations considered
-
A different time point. Table 2 and eTable 1 give diastolic results at 3 and 6 months only, with P from .08 to .17. No time point reaches P < .05. This does not account for the claim.
-
Within-group rather than between-group change. Footnote b marks the intervention arm's diastolic decrease as significant at both times, so "favoring the intervention" could describe a within-arm improvement. But the abstract says "Intergroup differences", and the control arm's 6-month decrease is also marked significant, so a within-group reading does not distinguish the arms. This does not account for the wording.
-
"Favoring" meaning only the direction of the point estimate. The other outcomes in the sentence (fasting glucose, dietary changes) are significant in Table 2. If direction alone were the criterion, systolic blood pressure (−4.2 and −2.8 mm Hg) and total and LDL cholesterol at 6 months would also qualify, and they are not listed. The paper's own Results sentence also treats the blood pressure result as no difference. This does not account for the claim.
-
A one-sided test. The Methods state 2-sided tests. One-sided P values implied by the printed CIs are .076 and .080 for the main analysis. Only the 6-month complete-case result would fall below .05 one-sided (.042), and the paper presents the complete-case analysis as a supplementary check. This does not account for the claim.
-
The paper's Benjamini-Hochberg procedure. BH adjustment can only raise a P value, so it cannot turn P = .15 or .17 into P < .05. For completeness: if all 21 between-group P values in Table 2 at a time point are treated as one family and q ≤ 0.20 is used as the criterion (the paper applies FDR 0.20 to within-group changes only), diastolic BP has q = .175 at 3 months and would pass, and q = .255 at 6 months and would not. On the same rule, systolic BP (q = .092), total cholesterol (q = .056) and LDL cholesterol (q = .173) would also pass at 3 months, and none of them appears in the abstract. The paper reports no between-group FDR analysis and sets significance at P < .05. This does not plausibly account for the claim.
-
A systolic/diastolic slip. The only blood pressure comparison in the paper with P < .05 is complete-case systolic BP at 3 months (P = .04, eTable 1). If the abstract meant systolic, it would still rest on a supplementary analysis at the earlier time point, with P = .07 in the main analysis. This would explain the word but not support the claim.
-
A baseline-adjusted analysis. Because baseline diastolic pressure was higher in the control group, an ANCOVA could give a larger estimate than the change-score analysis. As an illustration only, using the full-sample baseline gap from Table 1 (the 139-person subgroup's gap is not reported) and an assumed baseline coefficient between 0.3 and 1.0, the 6-month adjusted estimate would lie between −4.5 and −2.4 mm Hg. Its standard error cannot be derived from the published summaries. No such analysis is reported in the paper or supplement, so it cannot be the basis of the abstract statement.
-
Conversion or transcription artifact. The abstract sentence, the Results sentence and the Table 2 rows read identically in the publisher's page, the live PMC page and the PMC archive. The printed P values agree with their own CIs and with recomputation. No artifact is indicated.
Status
- Version of record. The publisher's article page (reached through doi.org, jamanetwork.com article 2780570) carries the same abstract sentence (the fetched text was cut off after "diastolic blood pressure"), the same Results sentence and the same Table 2 diastolic rows. The live PMC page and the archived PMC text match. All quotes from the article above were checked verbatim against the PMC-archived text; the eTable 1 rows were read from the supplement PDF.
- Supplements. Supplement 1 (eAppendix, eFigures 1–2, eTables 1–3), Supplement 2 (trial protocol) and Supplement 3 (data-sharing statement) were obtained from the Europe PMC supplementary-files archive. eTable 1, the complete-case analysis, does not resolve the discrepancy (diastolic P = .13 and .08). No individual-level data are posted, so the mixed model could not be rerun.
- Corrections. As of 2026-10-23 no correction, erratum, retraction or expression of concern was found: Crossref lists no update relations for the DOI, the Crossref/Retraction Watch data have no entry, Europe PMC (PMID 34081137) lists no comment or correction links, PubMed lists no linked notices, and the PMC page shows none.
- PubPeer. PubPeer's search page and API returned an access challenge (HTTP 403) to automated queries, and a web search could not be run in this session, so PubPeer comments could not be checked.
- Related observations (not the subject of this note). (i) The abstract gives P = .001 for the 6-month HbA1c difference in the subgroup with baseline HbA1c ≥ 8%; the Results text and Table 3 give P = .02 for the same comparison (−0.9%, 95% CI −1.6 to −0.1), whose CI implies P = .019 (Welch from the Table 3 summaries: .039). (ii) In Table 2 the 6-month fasting glucose CI (−23.4 to −3.6) is identical to the 3-month CI; its midpoint is −13.5 rather than the printed −12.6, and recomputation gives −21.4 to −3.8 (P = .005). Neither changes a conclusion.
Suggested correction
In the Abstract, Results, replace
Intergroup differences favoring the intervention were also noted for fasting blood glucose, diastolic blood pressure, and dietary changes.
with, for example,
Intergroup differences favoring the intervention were also noted for fasting blood glucose and dietary changes; between-group differences in blood pressure were not statistically significant (diastolic blood pressure at 6 months, −2.4 mm Hg [95% CI, −5.7 to 1.0]; P = .17).
If the authors intended a baseline-adjusted or other analysis, that analysis and its result should be reported in the Results and the supplement before it is summarized in the abstract. The editors may also wish to check the abstract's P = .001 for the HbA1c ≥ 8% subgroup against the P = .02 in the Results and Table 3, and the 6-month fasting glucose CI in Table 2.
Recomputation script (python recompute.py.txt)
"""Recompute the diastolic blood pressure (DBP) between-group results of PMC8176331
(JAMA Netw Open 2021, doi:10.1001/jamanetworkopen.2021.12417).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC and from Supplement 1:
Table 1 baseline blood pressure, control n = 105, intervention n = 99
Table 2 multiple-imputation (MI) results; blood pressure block: control 72, intervention 67
eTable 1 (Supplement 1) complete-case (CC) results; total n = 132 (3 mo) and 131 (6 mo),
split between groups not printed
The paper fitted generalized linear mixed models on 5 imputed data sets. Those models
cannot be refitted from published summaries, so two approximations are used:
(a) a two-sample t test (pooled and Welch) from the printed mean (SD) changes and group n;
(b) the 2-sided P value implied by the printed difference and its 95% CI
(normal approximation, SE = CI width / (2 x 1.96)).
Both only check the printed numbers against each other; neither is a reanalysis.
"""
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
# ------------------------------------------------------------------ inputs (from the paper)
# label: (control mean, SD, n, intervention mean, SD, n, reported diff, reported CI, reported P, reported d, location)
ROWS = {
"DBP 3 mo (MI)": (-1.8, 11.0, 72, -4.4, 9.9, 67, -2.6, (-6.2, 0.9), ".15", 0.25, "Table 2"),
"DBP 6 mo (MI)": (-3.1, 9.2, 72, -5.4, 11.1, 67, -2.4, (-5.7, 1.0), ".17", 0.23, "Table 2"),
"SBP 3 mo (MI)": (-2.2, 14.7, 72, -6.5, 13.1, 67, -4.2, (-8.8, 0.3), ".07", 0.31, "Table 2"),
"SBP 6 mo (MI)": (-5.0, 13.8, 72, -7.8, 15.2, 67, -2.8, (-7.7, 2.0), ".25", 0.19, "Table 2"),
# complete case: group split not printed; 66/66 and 66/65 assumed (sensitivity below)
"DBP 3 mo (CC)": (-1.6, 10.6, 66, -4.1, 8.8, 66, -2.6, (-5.9, 0.8), ".13", 0.26, "eTable 1"),
"DBP 6 mo (CC)": (-2.5, 7.9, 66, -5.4, 10.9, 65, -2.9, (-6.2, 0.4), ".08", 0.30, "eTable 1"),
"SBP 3 mo (CC)": (-1.4, 13.8, 66, -6.1, 12.6, 66, -4.7, (-9.3, -0.2), ".04", 0.36, "eTable 1"),
"SBP 6 mo (CC)": (-4.4, 13.2, 66, -7.9, 14.2, 65, -3.5, (-8.2, 1.3), ".14", 0.26, "eTable 1"),
# comparators named in the same abstract sentence (Table 2, n = 105 / 99)
"FBG 3 mo (MI)": (-3.6, 32.4, 105, -18.0, 37.8, 99, -14.4, (-23.4, -3.6), ".005", 0.41, "Table 2"),
"FBG 6 mo (MI)": (-1.8, 25.2, 105, -14.4, 37.8, 99, -12.6, (-23.4, -3.6), ".01", 0.39, "Table 2"),
"Calories 6 mo (MI)": (-245.8, 466.7, 105, -551.3, 515.4, 99, -305.8, (-441.0, -170.7), "<.001", 0.62, "Table 2"),
}
BASELINE_DBP = (85.7, 9.8, 105, 82.7, 8.8, 99) # Table 1, control vs intervention
BASELINE_SBP = (135.3, 13.0, 105, 134.7, 13.5, 99)
BP_N, TOTAL_N = 139, 204 # Table 2 blood pressure block vs randomized
def recompute(mc, sc, nc, mi, si, ni, rep_diff, rep_ci):
d = mi - mc
sp = np.sqrt(((nc - 1) * sc**2 + (ni - 1) * si**2) / (nc + ni - 2))
se_p = sp * np.sqrt(1 / nc + 1 / ni)
df = nc + ni - 2
q = stats.t.ppf(0.975, df)
p_pool = 2 * stats.t.sf(abs(d / se_p), df)
se_w = np.sqrt(sc**2 / nc + si**2 / ni)
dfw = se_w**4 / ((sc**2 / nc) ** 2 / (nc - 1) + (si**2 / ni) ** 2 / (ni - 1))
p_welch = 2 * stats.t.sf(abs(d / se_w), dfw)
se_ci = (rep_ci[1] - rep_ci[0]) / (2 * Z)
p_ci = 2 * stats.norm.sf(abs(rep_diff) / se_ci)
return dict(diff=d, lo=d - q * se_p, hi=d + q * se_p, p_pool=p_pool, p_welch=p_welch,
p_ci=p_ci, d=abs(d) / sp, p_one=p_pool / 2, mid_ci=(rep_ci[0] + rep_ci[1]) / 2)
R = {}
print("=== 1. Between-group differences (intervention minus control) ===")
print(" diff/CI from rounded mean (SD) changes, pooled-variance t; P also by Welch and from the printed CI\n")
for k, v in ROWS.items():
mc, sc, nc, mi, si, ni, rd, rci, rp, rdd, loc = v
r = recompute(mc, sc, nc, mi, si, ni, rd, rci)
R[k] = r
print(f"{k:20s} [{loc}] reported {rd:+.1f} ({rci[0]} to {rci[1]}), P={rp}, d={rdd}")
print(f"{'':20s} recomputed {r['diff']:+.2f} ({r['lo']:+.2f} to {r['hi']:+.2f}); "
f"P pooled={r['p_pool']:.3f}, Welch={r['p_welch']:.3f}, from printed CI={r['p_ci']:.3f}; "
f"d={r['d']:.2f}; one-sided P={r['p_one']:.3f}; midpoint of printed CI={r['mid_ci']:+.2f}")
print("\n=== 2. Complete-case rows: sensitivity to the unknown group split ===")
for k, tot in (("DBP 3 mo (CC)", 132), ("DBP 6 mo (CC)", 131)):
mc, sc, _, mi, si, _, rd, rci, *_ = ROWS[k]
ps = []
for nc in range(tot - 72, 73): # control at most 72, intervention at most 67
ni = tot - nc
if ni > 67 or ni < 1:
continue
ps.append(recompute(mc, sc, nc, mi, si, ni, rd, rci)["p_pool"])
print(f"{k}: over every split with control <= 72 and intervention <= 67, "
f"2-sided P ranges {min(ps):.3f} to {max(ps):.3f}")
print("\n=== 3. Does any DBP between-group test reach 2-sided P < .05? ===")
dbp = [k for k in ROWS if k.startswith("DBP")]
for k in dbp:
r = R[k]
print(f" {k:16s} printed CI includes 0: {ROWS[k][7][0] < 0 < ROWS[k][7][1]}; "
f"recomputed CI includes 0: {r['lo'] < 0 < r['hi']}; max P (any method) "
f"{max(r['p_pool'], r['p_welch'], r['p_ci']):.3f}, min P {min(r['p_pool'], r['p_welch'], r['p_ci']):.3f}")
print(f" any DBP row with recomputed 2-sided P < .05: {any(min(R[k]['p_pool'], R[k]['p_welch'], R[k]['p_ci']) < .05 for k in dbp)}")
print("\n=== 4. Baseline imbalance and coverage ===")
for lab, b in (("DBP", BASELINE_DBP), ("SBP", BASELINE_SBP)):
t, p = stats.ttest_ind_from_stats(*b)
print(f" baseline {lab}: control {b[0]} ({b[1]}) vs intervention {b[3]} ({b[4]}); "
f"difference {b[0] - b[3]:.1f} mm Hg; t={t:.2f}, P={p:.3f}")
print(f" participants in the blood pressure analysis: {BP_N}/{TOTAL_N} = {100 * BP_N / TOTAL_N:.1f}%")
print("\n=== 5. One-sided P implied by the printed CIs (the paper states 2-sided tests) ===")
for k in dbp:
print(f" {k:16s} one-sided P from printed CI = {R[k]['p_ci'] / 2:.3f}")
print("\n=== 6. Benjamini-Hochberg q over all 21 between-group P values printed in Table 2 ===")
# printed P values, Table 2, in row order: weight kg, weight %, BMI, HbA1c, FBG, SBP, DBP, cost,
# total chol, LDL, HDL, TG, creatinine, calories, carbohydrate, sugar, protein, total fat, sat fat, fiber, PA.
# "<.001" is entered as .001 (conservative: larger than the true value).
NAMES = ["weight kg", "weight %", "BMI", "HbA1c", "FBG", "SBP", "DBP", "cost", "total chol", "LDL",
"HDL", "TG", "creatinine", "calories", "carbohydrate", "sugar", "protein", "total fat",
"sat fat", "fiber", "physical activity"]
P_T2 = {
"3 mo": [.001, .001, .001, .006, .005, .07, .15, .04, .04, .14, .90, .48, .91, .001, .001, .009, .006, .001, .001, .02, .007],
"6 mo": [.001, .001, .001, .01, .01, .25, .17, .01, .65, .53, .76, .49, .63, .001, .001, .01, .09, .001, .001, .54, .009],
}
def bh(p):
p = np.asarray(p, float); m = len(p); o = np.argsort(p)
q = np.empty(m); run = 1.0
for rank in range(m, 0, -1):
i = o[rank - 1]; run = min(run, p[i] * m / rank); q[i] = run
return q
for tp, ps in P_T2.items():
q = bh(ps)
print(f" {tp}: q(DBP)={q[NAMES.index('DBP')]:.3f} q(SBP)={q[NAMES.index('SBP')]:.3f} "
f"q(LDL)={q[NAMES.index('LDL')]:.3f} q(total chol)={q[NAMES.index('total chol')]:.3f}")
print(f" outcomes with q <= 0.20: {[n for n, qq in zip(NAMES, q) if qq <= 0.20]}")
print("\n=== 7. Illustration only: what a baseline-adjusted (ANCOVA) estimate could be ===")
# change-score difference = (F_I - F_C) - (B_I - B_C); ANCOVA = (F_I - F_C) - b (B_I - B_C)
# => ANCOVA = change difference + (1 - b)(B_I - B_C). Baseline gap from Table 1 (all 204, not the 139).
gap = BASELINE_DBP[3] - BASELINE_DBP[0]
for b in (0.3, 0.5, 0.7, 1.0):
print(f" 6 mo, assumed b={b:.1f}: {-2.4 + (1 - b) * gap:+.1f} mm Hg (standard error not derivable)")
print("\n=== 8. Side observations (not the subject of the note) ===")
r = R["FBG 6 mo (MI)"]
print(f" FBG 6 mo: printed CI (-23.4 to -3.6) is identical to the 3-mo row; its midpoint is {r['mid_ci']:+.1f}, "
f"not the printed -12.6; recomputed {r['diff']:+.1f} ({r['lo']:.1f} to {r['hi']:.1f}), P={r['p_pool']:.3f}")
# HbA1c >= 8% subgroup, 6 mo, Table 3: control -1.0 (1.4) n=29, intervention -1.8 (1.4) n=26; -0.9 (-1.6 to -0.1), P=.02
hb = recompute(-1.0, 1.4, 29, -1.8, 1.4, 26, -0.9, (-1.6, -0.1))
print(f" HbA1c>=8% subgroup 6 mo: abstract P=.001; Results text and Table 3 P=.02; "
f"P from Table 3 CI={hb['p_ci']:.3f}; Welch from Table 3 summaries={hb['p_welch']:.3f}")
# ------------------------------------------------------------------ figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ---- Figure 1: abstract vs Results / Table 2
fig = plt.figure(figsize=(7.2, 4.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")
def panel(x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=1.5))
ax.text(x + 0.14, y + h - 0.2, title, fontsize=11, weight="bold", va="top", color=INK)
panel(0.15, 1.05, 2.85, 3.25, "Abstract, Results", INK2)
ax.text(0.29, 3.68,
"“Intergroup differences favoring\nthe intervention were also noted\n"
"for fasting blood glucose,\ndiastolic blood pressure, and\ndietary changes.”",
fontsize=10.5, va="top", color=INK, linespacing=1.45)
ax.text(0.29, 1.62, "Claim: a between-group\ndifference in diastolic BP", fontsize=10,
va="top", color=INK2, style="italic", linespacing=1.35)
panel(3.75, 1.05, 3.3, 3.25, "Results text and Table 2", INK2)
ax.text(3.89, 3.68, "“Between-group differences in blood\npressure and lipids were not\nobserved at 6 months.”",
fontsize=10.5, va="top", color=INK, linespacing=1.45)
ax.text(3.89, 2.55, "Table 2, change in diastolic BP\n(intervention − control, mm Hg)",
fontsize=10, va="top", color=INK2, linespacing=1.35)
for i, (k, lab) in enumerate((("DBP 3 mo (MI)", "3 mo"), ("DBP 6 mo (MI)", "6 mo"))):
v = ROWS[k]
yy = 1.88 - i * 0.28
ax.text(3.89, yy, f"{lab}: −{abs(v[6]):.1f} ({v[7][0]:.1f} to {v[7][1]:.1f}), P = {v[8]}".replace("-", "−"),
fontsize=10.5, va="top", color=ORANGE, weight="bold")
ax.text(3.89, 1.27, "printed in the paper", fontsize=10, va="top", color=ORANGE)
ax.annotate("", xy=(3.72, 2.7), xytext=(3.03, 2.7),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(3.375, 2.82, "conflict", ha="center", fontsize=10, color=INK, weight="bold")
r3, r6 = R["DBP 3 mo (MI)"], R["DBP 6 mo (MI)"]
ax.text(0.15, 0.78,
f"Recomputed from Table 2's own mean (SD) changes: P = {r3['p_pool']:.3f} (3 mo), {r6['p_pool']:.3f} (6 mo);",
fontsize=10, color=BLUE, va="top")
ax.text(0.15, 0.50,
f"P implied by the printed 95% CIs: {r3['p_ci']:.3f} and {r6['p_ci']:.3f}. Every CI, printed or recomputed, includes 0.",
fontsize=10, color=BLUE, va="top")
ax.text(0.15, 0.22, "Orange: printed in the paper. Blue: recomputed from the paper's numbers.",
fontsize=10, color=INK2, va="top")
save(fig, "fig1")
# ---- Figure 2: dot-and-interval, DBP between-group differences, reported vs recomputed
order = ["DBP 3 mo (MI)", "DBP 6 mo (MI)", "DBP 3 mo (CC)", "DBP 6 mo (CC)"]
names = ["3 mo, multiple imputation\n(Table 2)", "6 mo, multiple imputation\n(Table 2)",
"3 mo, complete case\n(eTable 1)", "6 mo, complete case\n(eTable 1)"]
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
ax.set_facecolor(BG)
off = 0.13
for i, k in enumerate(order):
y = len(order) - 1 - i
v, r = ROWS[k], R[k]
ax.plot(v[7], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(v[6], y + off, "o", ms=8, color=ORANGE, mec=BG, mew=1.5,
label="Reported (paper)" if i == 0 else None)
ax.plot([r["lo"], r["hi"]], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(r["diff"], y - off, "s", ms=8, color=BLUE, mec=BG, mew=1.5,
label="Recomputed from the paper's mean (SD) changes" if i == 0 else None)
ax.text(3.0, y + off, f"P = {v[8]}", va="center", fontsize=10, color=INK)
ax.text(3.0, y - off, f"P = {r['p_pool']:.3f}".replace("0.", ".", 1), va="center", fontsize=10, color=INK)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(0.08, 3.55, "no difference", fontsize=10, color=INK2, va="center")
ax.set_yticks(range(len(order)))
ax.set_yticklabels(names[::-1], fontsize=10, color=INK)
ax.set_xlim(-7, 4.2); ax.set_ylim(-0.6, 3.75)
ax.set_xticks(range(-7, 3))
ax.set_xlabel("Diastolic BP change, intervention − control (mm Hg), 95% CI")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.tight_layout(rect=(0, 0, 1, 0.9))
fig.legend(loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 0.99))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
The 2019 overall adolescent YLL is 11 494 below boys plus girls, understating the five-year total
JAMA Pediatr (2022) · doi:10.1001/jamapediatrics.2021.6032 · PMC8804970
Summary
The paper's single Table gives years of life lost (YLL) to unintentional drug overdose by year and sex. For adolescents (10-19 years) in 2019 it prints an overall YLL of 27 919.85, but the boys and girls cells in the same row add up to 26 711.99 + 12 701.86 = 39 413.85. In the other nine year-rows of the Table, the overall cell equals boys plus girls to the cent. The adolescent five-year total of 187 077.92, printed in the Total row and in the Results, is the sum of the overall column with the 2019 cell as printed. The paper's own boys and girls totals add up to 198 571.92, and its stated annual mean (SD) of 39 714.38 (2689.63) and its Discussion figure of "nearly 200 000 YLL" both match 198 571.92, not 187 077.92. The stated adolescent total is therefore 11 494.00 YLL (5.8%) too low, and the printed 2019 cell shows a 24.6% fall in adolescent YLL from 2018 to 2019 where the paper's own sex-specific numbers show a 6.4% rise. The paper's conclusions do not change.
What the paper reports
Table, "Annual Mortality Due to Unintentional Drug Overdose Among Adolescents and Young People, 2015-2019", adolescents section. Columns are Year | Boys/men deaths, No. | Boys/men YLL | Girls/women deaths, No. | Girls/women YLL | Overall deaths, No. | Overall YLL:
2018 | 407 | 23 910.59 | 207 | 13 117.02 | 614 | 37 027.61
2019 | 455 | 26 711.99 | 199 | 12 701.86 | 654 | 27 919.85
Total | 2267 | 133 023.64 | 1029 | 65 548.28 | 3296 | 187 077.92
Table footer (the only footnote):
Abbreviation: YLL, years of life lost.
Methods:
YLL were calculated as standard life expectancy minus age at death. Male and female life expectancy at each individual age was determined from the 2017 Social Security Administration Period Life Table.
Results, first paragraph:
Male adolescents outnumbered female adolescents in incident deaths (2267 [68.8%] vs 1029 [31.2%]) and YLL (133 023.64 vs 65 548.28). Annual total YLL due to unintentional drug overdose was stably elevated with a mean (SD) 39 714.38 (2689.63) annual YLL (Table). Adolescents experienced a total of 187 077.92 YLL during the study period.
Results, second paragraph:
Young people experienced a total of 1 227 223.58 YLL during the 5-year period of study.
Discussion, first sentence:
Over the 5-year period of this cross-sectional study, adolescents experienced nearly 200 000 YLL, and young people amassed greater than 1.25 million YLL.
What the paper's own numbers show
Method. YLL and deaths are counts summed over decedents, so for each year the overall cell should equal the boys/men cell plus the girls/women cell. I checked this for every year-row of both age groups, summed every column and compared it with the printed Total row, and recomputed the annual mean and SD of the adolescent overall YLL series two ways: from the overall column as printed, and from boys plus girls in each row. I also computed YLL per death for each adolescent year as a check on scale. All inputs are copied from the Table and text. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Adolescent overall YLL, 2019 | Table, 2019 row, Overall YLL | 27 919.85 | 39 413.85 (26 711.99 + 12 701.86); printed cell is 11 494.00 lower |
| Adolescent overall YLL, 2015-2018 | Table, Overall YLL | 38 464.60; 44 187.60; 39 478.26; 37 027.61 | identical to boys + girls (difference 0.00 in each year) |
| Adolescent overall deaths, 2019 | Table, 2019 row | 654 | 654 (455 + 199) |
| Adolescent five-year total YLL | Table Total row; Results | 187 077.92 | 198 571.92 (133 023.64 + 65 548.28); 187 077.92 is the sum of the printed overall column, including the 2019 cell |
| Boys and girls total YLL | Table Total row; Results | 133 023.64; 65 548.28 | 133 023.64; 65 548.28 (column sums agree) |
| Annual mean (SD) of adolescent YLL | Results | 39 714.38 (2689.63) | 39 714.38 (2689.63) from boys + girls per year (sample SD); 37 415.58 (5948.42) from the printed column |
| Adolescent total, Discussion | Discussion | "nearly 200 000 YLL" | 198 571.92 is 99.3% of 200 000; 187 077.92 is 93.5% |
| Change in adolescent YLL, 2018 to 2019 | Table | 37 027.61 to 27 919.85 (implied −24.6%) | 37 027.61 to 39 413.85 (+6.4%) |
| Adolescent YLL per death, 2019 | Table | 27 919.85 / 654 = 42.69 | 39 413.85 / 654 = 60.27; other years 60.29, 60.20, 60.18, 60.31 |
| Young people, every year and Total | Table, young people section | e.g. 2019: 236 273.30; Total 1 227 223.58 | all match men + women and the column sums |
| Male share of deaths | Results | 68.8% vs 31.2% (adolescents); 71.9% vs 28.1% (young people) | 68.8% vs 31.2%; 71.9% vs 28.1% |
The Table is internally consistent everywhere except one cell. Every other row adds across, every column adds down to its Total, and the death counts in the 2019 row add correctly. The 2019 adolescent overall YLL is the only exception. Two further checks in the paper's own text favor 39 413.85 over the printed value. The Results give an annual mean and SD that reproduce to the cent only from the boys-plus-girls series (39 714.38 × 5 = 198 571.90), and the YLL per death in 2019 is about 60.3 with the boys-plus-girls value, as in every other year, but 42.7 with the printed value. The difference between the printed and recomputed cells is a whole number, 11 494.00, and the two values share the decimals .85.
Figure
Figure 1. Adolescent (10-19 years) unintentional overdose YLL by year, on an axis starting at zero. Blue filled circles and solid line: boys YLL plus girls YLL in each row of the Table, recomputed. Orange open circles and dashed line: the Overall YLL column as printed. From 2015 to 2018 the two series are identical to the cent, so the markers overlap. In 2019 the printed cell is 27 919.85 and boys plus girls is 39 413.85, a gap of 11 494.00 YLL. Read at face value, the printed column shows a fall of 24.6% from 2018 to 2019; the paper's sex-specific cells show a rise of 6.4%. All values are from the Table or from recompute.py.txt.
Figure 2. Calculation flow. Left (orange): starting from the Overall YLL column as printed, the five-year sum is 187 077.92, which is the total in the Table and the Results, and the annual mean (SD) is 37 415.58 (5948.42). Right (blue): replacing the 2019 cell by boys plus girls (39 413.85), the five-year sum is 198 571.92, equal to the printed boys total plus the printed girls total, and the annual mean (SD) is 39 714.38 (2689.63), exactly the values in the Results. The Discussion's "nearly 200 000 YLL" also fits the right-hand total. The paper's stated total and its stated mean therefore come from different versions of the 2019 cell.
Why it matters
- A stated total is wrong in two places. The Results sentence "Adolescents experienced a total of 187 077.92 YLL during the study period" and the Table's Total row both understate the paper's own adolescent burden by 11 494.00 YLL (5.8%). The figure the paper's inputs support is 198 571.92.
- The paper contradicts itself. Within one paragraph of the Results, the annual mean of 39 714.38 implies a total of 198 571.90, while the next sentence gives 187 077.92. A reader cannot tell from the text which is right without redoing the arithmetic.
- The printed 2019 cell suggests a trend that is not in the data. Taken at face value, the overall column makes 2019 the lowest year by a wide margin, a 24.6% fall from 2018. The paper's sex-specific cells show a 6.4% rise, in line with the text's description of YLL as "stably elevated". Anyone extracting the overall series for trend analysis or for a meta-analysis would carry the false fall forward.
- The conclusions stand. The adolescent death counts (3296), the sex-specific YLL, the male predominance, the stated mean (SD), all young-people YLL figures and the Discussion's "nearly 200 000 YLL" are unaffected. The paper tests no hypotheses, so no inference changes. The error is one table cell and the total built on it.
Explanations considered
- "Overall" is computed differently from boys plus girls (for example with a pooled, sex-combined life table). Not supported. The Methods describe only sex-specific life expectancies from the 2017 Social Security Administration Period Life Table and give no separate method for the overall column. In the other nine year-rows (adolescents 2015-2018, young people 2015-2019), overall equals boys plus girls to the cent, and the young-people Total does too. A different method would not give exact additivity in nine rows and a 29% shortfall in one.
- A different population, age band or subset in the 2019 overall cell. Not supported. The 2019 overall death count is 654 = 455 + 199, so the cell covers the same decedents as the sex-specific cells. The Table's only footnote is an abbreviation and sets out no exception for 2019.
- The sex-specific 2019 cells, not the overall cell, are wrong. Not supported. The boys and girls columns sum to the printed boys and girls totals (133 023.64 and 65 548.28), which the Results also quote. Their sum, 198 571.92, is the total that the Results mean (SD) and the Discussion's "nearly 200 000" imply. YLL per death in 2019 is 60.27 with the sex-specific values, matching 60.18 to 60.31 in the other years.
- Rounding. Ruled out. The gap is 11 494.00, against differences of exactly 0.00 in every other row.
- A text-conversion or table-alignment artifact. Ruled out. The publisher's full text, the PMC HTML page and the Europe PMC JATS XML give the same values. In the XML, each value is in its own table cell in column order, so the 2019 overall YLL is not shifted from another column or row.
- A correction already issued. None found (see Status).
- Side observation, not part of the main finding. The Discussion says young people "amassed greater than 1.25 million YLL", but the Table and Results give 1 227 223.58 YLL for young people, which is less than 1.25 million. The Table's young-people figures are internally consistent, so this concerns only the wording of the Discussion.
Status
- Version of record. The publisher's full text (doi.org/10.1001/jamapediatrics.2021.6032, which resolves to jamanetwork.com/journals/jamapediatrics/fullarticle/2788490) has the same adolescent 2019 row (455 | 26 711.99 | 199 | 12 701.86 | 654 | 27 919.85), the same Total row and the same Results sentence as the PMC copy. The PMC HTML page and the Europe PMC full-text XML agree. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. None. This is a research letter (pages 415-417); the publisher page lists no supplement, and the PMC and Europe PMC records have no supplementary material. The source data are public CDC WONDER multiple-cause-of-death files, but individual-level ages at death are not published, so the YLL cannot be rebuilt independently. The check here relies only on the paper's own numbers.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations and no PMC or Europe PMC notices; the Crossref record has no update-to, updated-by or relation entries. PubMed (PMID 35099529) lists no erratum, only a linked letter (doi:10.1001/jamapediatrics.2022.1171) and reply (doi:10.1001/jamapediatrics.2022.1174), JAMA Pediatr 2022;176(7):729, which concern race-specific overdose trends; the reply does not mention the Table or revise any YLL figure. The publisher page shows no correction notice. PubPeer has no comments on the article, and Retraction Watch has no matching item. The Europe PMC search endpoint returned HTTP 503 on three attempts; the Europe PMC notices check throughsrc/known.pyreturned empty. - Sources. PMC article PMC8804970; https://pmc.ncbi.nlm.nih.gov/articles/PMC8804970/; Europe PMC full-text XML for PMC8804970; https://doi.org/10.1001/jamapediatrics.2021.6032 (publisher full text); Crossref API record; PubMed PMID 35099529; doi:10.1001/jamapediatrics.2022.1171 and doi:10.1001/jamapediatrics.2022.1174; PubPeer; Retraction Watch.
Suggested correction
In the Table, adolescents section, change the 2019 Overall YLL from
27 919.85
to
39 413.85
and the Total row Overall YLL from
187 077.92
to
198 571.92
In the Results, change
Adolescents experienced a total of 187 077.92 YLL during the study period.
to
Adolescents experienced a total of 198 571.92 YLL during the study period.
These values follow from the paper's own sex-specific cells and agree with its stated mean (SD) and with the Discussion. The authors may wish to confirm the 2019 value against their source data. They may also wish to check the Discussion's "greater than 1.25 million YLL" for young people against the 1 227 223.58 YLL in the Table and Results, for example by changing it to "more than 1.2 million YLL".
Recomputation script (python recompute.py.txt)
"""Recompute the Table of PMC8804970
(JAMA Pediatr 2022, doi:10.1001/jamapediatrics.2021.6032,
"Unintentional Drug Overdose Mortality in Years of Life Lost Among Adolescents and
Young People in the US From 2015 to 2019").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (identical in the publisher's version):
Table, "Annual Mortality Due to Unintentional Drug Overdose Among Adolescents and Young
People, 2015-2019": every year row and Total row, both age groups.
Results: "mean (SD) 39 714.38 (2689.63) annual YLL"; "a total of 187 077.92 YLL";
"2267 [68.8%] vs 1029 [31.2%]"; "15 604 [71.9%] vs 6085 [28.1%]".
Discussion: "adolescents experienced nearly 200 000 YLL, and young people amassed
greater than 1.25 million YLL".
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
YEARS = [2015, 2016, 2017, 2018, 2019]
# ---------------------------------------------------------------- inputs (Table)
# m_d, f_d, o_d: deaths (boys/men, girls/women, overall); m, f, o: YLL
TABLE = {
"Adolescents (10-19 y)": dict(
m_d=[436, 514, 455, 407, 455], m=[25578.86, 30159.22, 26662.98, 23910.59, 26711.99],
f_d=[202, 220, 201, 207, 199], f=[12885.74, 14028.38, 12815.28, 13117.02, 12701.86],
o_d=[638, 734, 656, 614, 654], o=[38464.60, 44187.60, 39478.26, 37027.61, 27919.85],
total=dict(m_d=2267, m=133023.64, f_d=1029, f=65548.28, o_d=3296, o=187077.92)),
"Young people (10-24 y)": dict(
m_d=[2694, 3574, 3465, 2871, 3000], m=[148984.31, 197270.24, 190957.17, 158439.46, 165925.24],
f_d=[1050, 1239, 1398, 1227, 1171], f=[63219.62, 74608.05, 83744.29, 73727.14, 70348.06],
o_d=[3744, 4813, 4863, 4098, 4171], o=[212203.93, 271878.29, 274701.46, 232166.60, 236273.30],
total=dict(m_d=15604, m=861576.42, f_d=6085, f=365647.16, o_d=21689, o=1227223.58)),
}
RESULTS_MEAN, RESULTS_SD = 39714.38, 2689.63 # Results, adolescents
RESULTS_TOTAL = 187077.92 # Results, adolescents
DISCUSSION_ADOL = 200000 # Discussion, "nearly 200 000"
def r2(x):
return round(float(x) + 0.0, 2)
# ---------------------------------------------------------------- 1. row additivity
print("=== 1. Overall = boys/men + girls/women, row by row ===")
for name, T in TABLE.items():
print(name)
for i, yr in enumerate(YEARS):
s = T["m"][i] + T["f"][i]
sd = T["m_d"][i] + T["f_d"][i]
print(f" {yr}: deaths {T['m_d'][i]} + {T['f_d'][i]} = {sd} (printed {T['o_d'][i]}); "
f"YLL {T['m'][i]:.2f} + {T['f'][i]:.2f} = {s:.2f} (printed {T['o'][i]:.2f}); "
f"printed - sum = {T['o'][i] - s:+.2f}")
# ---------------------------------------------------------------- 2. column totals
print("\n=== 2. Column sums vs printed Total row ===")
for name, T in TABLE.items():
print(name)
for k in ["m_d", "m", "f_d", "f", "o_d", "o"]:
print(f" sum of {k:3s} = {sum(T[k]):12.2f} printed Total = {T['total'][k]:12.2f} "
f"diff = {sum(T[k]) - T['total'][k]:+.2f}")
print(f" boys total + girls total YLL = {T['total']['m'] + T['total']['f']:.2f}")
# ---------------------------------------------------------------- 3. adolescents, 2019 cell
A = TABLE["Adolescents (10-19 y)"]
o_print = np.array(A["o"])
o_corr = np.array(A["m"]) + np.array(A["f"])
gap = o_corr[4] - o_print[4]
tot_print, tot_corr = o_print.sum(), o_corr.sum()
print("\n=== 3. Adolescents: the 2019 Overall YLL cell and what follows from it ===")
print(f" 2019 boys + girls = {o_corr[4]:.2f}; printed {o_print[4]:.2f}; shortfall {gap:.2f}")
print(f" shortfall as % of corrected = {100 * gap / o_corr[4]:.1f}%")
print(f" five-year total, printed column = {tot_print:.2f} (Total row and Results: {RESULTS_TOTAL:.2f})")
print(f" five-year total, corrected = {tot_corr:.2f}")
print(f" total understated by = {tot_corr - tot_print:.2f} ({100 * (tot_corr - tot_print) / tot_corr:.1f}% of corrected)")
print(f" 2018 -> 2019 change, printed = {o_print[3]:.2f} -> {o_print[4]:.2f} ({100 * (o_print[4] / o_print[3] - 1):+.1f}%)")
print(f" 2018 -> 2019 change, corrected = {o_corr[3]:.2f} -> {o_corr[4]:.2f} ({100 * (o_corr[4] / o_corr[3] - 1):+.1f}%)")
print(f" printed 2019 as lowest year? printed min = {YEARS[int(o_print.argmin())]}, corrected min = {YEARS[int(o_corr.argmin())]}")
# ---------------------------------------------------------------- 4. mean (SD) in Results
print("\n=== 4. Results: 'mean (SD) 39 714.38 (2689.63) annual YLL' ===")
for lab, x in [("printed Overall column", o_print), ("boys + girls per year", o_corr)]:
print(f" {lab:24s}: mean {x.mean():.2f}, sample SD (n-1) {x.std(ddof=1):.2f}, population SD (n) {x.std(ddof=0):.2f}")
print(f" Results states : mean {RESULTS_MEAN:.2f}, SD {RESULTS_SD:.2f}")
print(f" Results mean x 5 = {RESULTS_MEAN * 5:.2f}")
print(f" printed total / 5 = {RESULTS_TOTAL / 5:.2f}")
# ---------------------------------------------------------------- 5. other statements
print("\n=== 5. Other statements in the text ===")
print(f" Discussion 'nearly 200 000': corrected total {tot_corr:.2f} is {100 * tot_corr / DISCUSSION_ADOL:.1f}% of 200 000; "
f"printed {tot_print:.2f} is {100 * tot_print / DISCUSSION_ADOL:.1f}%")
Y = TABLE["Young people (10-24 y)"]
print(f" Young people total {sum(Y['o']):.2f} (> 1.25 million? {sum(Y['o']) > 1.25e6})")
print(f" Adolescent sex shares: {100 * 2267 / 3296:.1f}% vs {100 * 1029 / 3296:.1f}%")
print(f" Young-people sex shares: {100 * 15604 / 21689:.1f}% vs {100 * 6085 / 21689:.1f}%")
print(f" Adolescent YLL per death, 2019: printed {o_print[4] / 654:.2f}; corrected {o_corr[4] / 654:.2f}; "
f"other years {', '.join(f'{o_corr[i] / A['o_d'][i]:.2f}' for i in range(4))}")
print(f" Integer part of the gap: {int(o_corr[4])} vs {int(o_print[4])}; decimals {o_corr[4] % 1:.2f} vs {o_print[4] % 1:.2f}")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": SEC, "svg.fonttype": "none"})
def fmt(x):
# JAMA style: thin-space thousands separator only from 5 digits up
if abs(x) < 10000:
return f"{x:.2f}"
return f"{x:,.2f}".replace(",", " ")
# Figure 1: adolescent overall YLL by year
fig, ax = plt.subplots(figsize=(7.2, 4.2), facecolor=BG)
ax.set_facecolor(BG)
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right"]:
ax.spines[s].set_visible(False)
ax.plot(YEARS, o_corr, "-o", color=BLUE, lw=2, ms=9, zorder=3,
label="Boys + girls YLL, same row (recomputed)")
ax.plot(YEARS, o_print, "--o", color=ORANGE, lw=2, ms=9, mfc=BG, mew=2, zorder=4,
label="Overall YLL as printed in the Table")
ax.annotate("", xy=(2019, o_print[4] + 500), xytext=(2019, o_corr[4] - 500),
arrowprops=dict(arrowstyle="<->", color=SEC, lw=1.2))
ax.text(2019.08, (o_print[4] + o_corr[4]) / 2, f"printed cell\nis {fmt(gap)}\nYLL lower",
ha="left", va="center", color=TXT, fontsize=10)
ax.text(2019.1, o_corr[4], fmt(o_corr[4]), color=BLUE, va="center", fontsize=10)
ax.text(2019.1, o_print[4], fmt(o_print[4]), color=ORANGE, va="center", fontsize=10)
ax.text(2016.5, 46800, "2015-2018: printed Overall = boys + girls to the cent",
ha="center", color=SEC, fontsize=10)
ax.set_xlim(2014.6, 2020.15)
ax.set_ylim(0, 50000)
ax.set_xticks(YEARS)
ax.yaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, p: f"{v:,.0f}".replace(",", "\u2009")))
ax.set_ylabel("Adolescent YLL (10-19 years)")
ax.legend(loc="lower left", frameon=False, fontsize=10)
fig.tight_layout()
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 2: calculation flow, two routes to the five-year total and annual mean (SD)
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def box(x, y, w, h, text, edge, bold=False):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.5",
fc=BG, ec=edge, lw=2))
ax.text(x + w / 2, y + h / 2, text, ha="center", va="center", fontsize=10,
color=TXT, fontweight="bold" if bold else "normal", linespacing=1.35)
def arrow(x0, y0, x1, y1):
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=SEC, lw=1.4))
ax.text(25, 96, "Printed Overall column", ha="center", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(75, 96, "Boys + girls in each row (recomputed)", ha="center", fontsize=11, color=BLUE, fontweight="bold")
box(3, 72, 44, 16, f"2019 Overall YLL = {fmt(o_print[4])}\n(2015-2018 as printed)", ORANGE)
box(53, 72, 44, 16, f"2019: {fmt(26711.99)} + {fmt(12701.86)}\n= {fmt(o_corr[4])}", BLUE)
arrow(25, 71, 25, 61)
arrow(75, 71, 75, 61)
box(3, 44, 44, 16, f"Five-year total = {fmt(tot_print)}\nmatches Table Total row and Results", ORANGE)
box(53, 44, 44, 16, f"Five-year total = {fmt(tot_corr)}\n= boys total {fmt(133023.64)}\n+ girls total {fmt(65548.28)}", BLUE)
arrow(25, 43, 25, 33)
arrow(75, 43, 75, 33)
box(3, 16, 44, 16, f"Annual mean (SD)\n{fmt(o_print.mean())} ({fmt(o_print.std(ddof=1))})\ndoes not match Results", ORANGE)
box(53, 16, 44, 16, f"Annual mean (SD)\n{fmt(o_corr.mean())} ({fmt(o_corr.std(ddof=1))})\nmatches Results exactly", BLUE)
ax.text(50, 4, "Results states mean (SD) 39 714.38 (2689.63); Discussion states \"nearly 200 000 YLL\".",
ha="center", fontsize=10, color=SEC)
fig.tight_layout()
for ext, kw in [("svg", {}), ("png", {"dpi": 200})]:
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Hospital-stay estimate of 8.4 days lies outside its own reported 95% CI of 4.4–6.7
Kidney Int (2021) · doi:10.1016/j.kint.2021.02.026 · PMC7927648
Summary
This retrospective cohort of 152 children hospitalized with acute COVID-19 or MIS-C reports, in the Abstract, the Results and Table 4, that acute kidney injury (AKI, stages 1–3) was associated with an 8.4-day longer hospital stay, with a 95% CI of 4.4 to 6.7 and P < 0.0001, from unadjusted simple linear regression. The point estimate lies 1.7 days above the upper bound of its own interval. A regression CI is centred on its coefficient, and the stated interval is centred on 5.55. Every other row of Table 4 except one is centred on its estimate. At least one of the three printed numbers is therefore wrong, and the paper does not contain enough information to say which. The direction and statistical significance of the association survive every reconstruction considered, but the precision is overstated or the size is misstated. A second Table 4 row, for length of mechanical ventilation, is also not centred on its estimate.
What the paper reports
Abstract:
In unadjusted models, patients with acute kidney injury had 8.4 days longer hospitalizations compared to patients without acute kidney injury (95% confidence interval, 4.4-6.7).
Results, "Clinical course and outcomes", third paragraph:
In addition, patients with AKI had a significantly greater length of hospital stay of 8.4 days compared with patients without AKI (95% CI, 4.4–6.7; P < 0.0001).
Table 4, "Association of AKI with hospital outcomes", under the column header "β (95% CI)" (columns: outcome by AKI stage | β (95% CI) | P value):
Length of hospital stay, d AKI stage 1 + 2 + 3 | 8.4 (4.4 to 6.7) | <0.0001 AKI stage 2 + 3 | 5.2 (0.07 to 10.2) | 0.047 Length of PICU stay, d AKI stage 1 + 2 + 3 | 5 (0.86 to 9.1) | 0.02 AKI stage 2 + 3 | 2.3 (–3.7 to 8.3) | 0.45 Length of mechanical ventilation, d AKI stage 1 + 2 + 3 | –3.9 (–9.4 to 12.7) | 0.15 AKI stage 2 + 3 | –1.08 (–7.4 to 5.3) | 0.71
The same table gives mortality as a hazard ratio: "AKI stage 1 + 2 + 3 | 0.57 (0.03 to 11.8) | 0.72".
Methods, "Statistical analysis":
Multiple imputation was used for missing values in regression analysis. Subsequently, simple linear regression analysis was performed to assess the relationship between AKI and continuous outcomes, including PICU and hospital length of stay and length of time on mechanical ventilation.
The conclusions that draw on this result. Discussion, first paragraph:
AKI, in unadjusted models, was significantly associated with longer PICU and hospital length of stay.
Discussion, "Conclusions":
Pediatric COVID-19–related AKI, similar to reports in larger AKI studies, was associated with poor outcomes, such as increased PICU and hospital length of stay.
What the paper's own numbers show
Method. For simple linear regression the 95% CI of a coefficient is β ± t(0.975, df) × SE. It always contains β and is centred on it. A CI pooled across multiply imputed data sets with Rubin's rules has the same form, so the same holds. For each Table 4 row I checked whether the estimate lies inside its interval and computed the centre of the interval, the SE implied by its width (df = 150, from N = 152), and the P value implied by β/SE. For the hazard ratio I did the same on the log scale with a normal critical value. I then rebuilt the hospital-stay row under each assumption about which number is correct. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Hospital stay, AKI 1–3: is β inside its CI? | Abstract; Results; Table 4 | 8.4 (4.4 to 6.7) | No. 8.4 is 1.7 d above the upper bound 6.7 |
| Hospital stay, AKI 1–3: centre of the CI | Table 4 | 8.4 | 5.55 (half-width 1.15; SE from width 0.582) |
| Same row, if 8.4 and the lower bound 4.4 are right | derived | upper bound 6.7; P < 0.0001 | upper bound 12.4; SE 2.024; t = 4.15; P = 5.6 × 10⁻⁵ (consistent with P < 0.0001) |
| Same row, if 8.4 and the upper bound 6.7 are right | derived | — | lower bound 2 × 8.4 − 6.7 = 10.1, above the estimate: impossible |
| Same row, if the CI 4.4–6.7 is right | derived | β = 8.4 | β = 5.55; t = 9.54; P = 3.8 × 10⁻¹⁷ (also < 0.0001) |
| SE under the 4.4–12.4 reconstruction vs SE implied by the printed width | derived | — | 2.024 / 0.582 = 3.48 times larger |
| Hospital stay, AKI 2–3 | Table 4 | 5.2 (0.07 to 10.2), P = 0.047 | centre 5.135; P from β/SE 0.044: consistent |
| PICU stay, AKI 1–3 | Table 4 | 5 (0.86 to 9.1), P = 0.02 | centre 4.98; P 0.018: consistent |
| PICU stay, AKI 2–3 | Table 4 | 2.3 (–3.7 to 8.3), P = 0.45 | centre 2.30; P 0.45: consistent |
| Mechanical ventilation, AKI 1–3 | Table 4 | –3.9 (–9.4 to 12.7), P = 0.15 | inside, but centre +1.65; arms 5.5 below and 16.6 above; P from β/SE 0.49 (0.45 if only the 11 ventilated patients in Table 3 were analysed, df = 9): inconsistent |
| Mechanical ventilation, AKI 2–3 | Table 4 | –1.08 (–7.4 to 5.3), P = 0.71 | centre –1.05; P 0.74: consistent |
| Mortality hazard ratio, AKI 1–3 | Table 4 | 0.57 (0.03 to 11.8), P = 0.72 | geometric centre 0.595; P 0.71: consistent |
| Median hospital stay, without vs with AKI | Table 3 | acute COVID-19 2.8 vs 22.2 d; MIS-C 4.6 vs 8.2 d | differences 19.4 and 3.6 d (medians by subgroup, so they cannot reproduce the pooled β) |
Five of the seven Table 4 rows are centred on their estimates to within rounding, and their printed P values agree with the P implied by their intervals. The hospital-stay row for AKI stages 1–3 is the only one whose estimate falls outside its interval. The mechanical-ventilation row for AKI stages 1–3 is the only other row that is not centred on its estimate, and its printed P = 0.15 does not match the P implied by its interval.
A few smaller inconsistencies turned up during the comparison. None of them affects a conclusion:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| PICU stay, AKI 1–3 | Results text vs Table 4 | text "95% CI, 0.86–9; P = 0.025"; table "0.86 to 9.1", 0.02 | same estimate, different rounding of the bound and P |
| Acute COVID-19 group size | Table 3 vs Table 1 and Abstract | Table 3 "No AKI (N = 88)", "AKI (N = 8)" | 88 + 8 = 96; Table 1 gives 97 overall and No AKI N = 89 |
| Mechanical ventilation, acute COVID-19 with AKI | Table 3 | 2 (33.3) of N = 8 | 2/8 = 25.0% |
| Vasopressors, acute COVID-19 with AKI | Table 3 | 5 (32) of N = 8 | 5/8 = 62.5% |
Figure
Figure 1. The six day-scale rows of Table 4 on one axis, with no effect (0 days) as the thin vertical line. Orange dots and bars are the reported β and 95% CI. Blue rings mark the centre of each reported interval, computed from its printed bounds; where the orange dot sits inside the blue ring the estimate is centred, as it must be for a linear-regression CI. For hospital stay with AKI stages 1–3 the estimate 8.4 lies outside the interval, 1.7 days above its upper bound, and the interval's centre is 5.55. The dashed blue row below it is one possible reconstruction, not a correction: the interval centred on 8.4 that keeps the printed lower bound 4.4 (4.4 to 12.4, P = 6 × 10⁻⁵). The mechanical-ventilation row for AKI stages 1–3 contains its estimate, but its centre (+1.65) is 5.55 days from the estimate.
Why it matters
The result appears in the Abstract as one of the study's main findings, in the Results text and in Table 4, and it supports the Discussion and Conclusions statements that AKI was associated with longer hospital stays. As printed, the interval is impossible. A reader cannot tell whether the effect is about 8.4 days with a wide interval (for example 4.4 to 12.4) or about 5.6 days with the narrow interval printed. The two readings differ in effect size by almost 3 days and in standard error by a factor of about 3.5.
What does not change: every reconstruction leaves the lower bound well above zero and gives P < 0.0001, so the qualitative statement that AKI was associated with a significantly longer unadjusted hospital stay is not overturned. Table 3 medians (2.8 vs 22.2 days in acute COVID-19, 4.6 vs 8.2 days in MIS-C) and the PICU-stay result (5 days, 0.86 to 9.1) point the same way. Length of stay is a secondary outcome. The paper states that "The primary outcome of this study was the incidence rate of AKI" (11.8% overall), and that result, the albumin and white-cell odds ratios, the MIS-C systolic-dysfunction comparison and the mortality result do not depend on this row.
The mechanical-ventilation row for AKI stages 1–3 is a second, smaller problem: the printed estimate, interval and P value cannot all come from one symmetric interval. It is reported as non-significant, and every value implied by its interval is non-significant too, so the paper's statement that there was no significant association with length of mechanical ventilation is not affected.
Explanations considered
- The interval is on a transformed scale (log days) and was back-transformed. This does not account for the numbers. The Methods describe simple linear regression of length of stay, Table 4 heads the column "β (95% CI)" under "Length of hospital stay, d", and no transformation is mentioned. A back-transformed interval would also still contain its back-transformed estimate. The other day-scale rows are symmetric on the raw scale.
- A bootstrap or other asymmetric interval. No such method is described, and even a percentile bootstrap interval would almost never exclude the point estimate. The other rows are symmetric, which points to Wald-type intervals.
- Multiple imputation. The Methods state that multiple imputation was used for missing values. Rubin's rules give a pooled estimate and a t-based interval centred on it, so pooling cannot place the estimate outside the interval.
- The estimate and interval were taken from different rows or models. No other row in Table 4 has the interval 4.4 to 6.7 or the estimate 8.4. The same pair appears together in the Abstract, the Results text and Table 4, so they were not misaligned in one place only.
- A conversion or typesetting artifact. This is excluded. The PMC page and the Europe PMC full-text XML, which comes from the publisher's deposit, give the same plain decimals in all three places, with no minus signs, superscripts or mid-dots that could have been lost. The table cell holds the estimate and the interval together.
- A mistyped upper bound. This is plausible but cannot be proved. If 8.4 and 4.4 are correct, the upper bound would be about 12.4, the implied SE (about 2.0) is similar to those of the PICU-stay rows (2.1 and 3.0), and the implied P (5.6 × 10⁻⁵) fits the printed P < 0.0001. The alternative reading (estimate 5.55, interval as printed) is also compatible with P < 0.0001. The published material does not distinguish them.
- For the mechanical-ventilation row, a small analysed sample. Table 3 lists only 11 ventilated patients, so the regression may have used far fewer than 152 observations. That changes the critical value but not the centre of a symmetric interval: with df = 9 the interval still centres on +1.65 and implies P = 0.45, not 0.15.
Status
- Version of record. The PMC article page and the Europe PMC full-text XML (publisher-supplied JATS) give the same figures in the Abstract, the Results and Table 4. The publisher's own pages could not be opened: doi.org redirects to linkinghub.elsevier.com, which served only a redirect stub, and kidney-international.org returned HTTP 403. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplement. The only supplement, mmc1.pdf (4 pages, retrieved through the Europe PMC supplementary-files endpoint; its md5 matches the one in the PMC XML), contains Tables S1–S4: outcomes by AKI stage as medians, echocardiography in MIS-C, admission characteristics by unit, and logistic regression of baseline factors. It has no linear-regression output for length of stay and no individual-level data, so it does not resolve the error.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, the Crossref record that includes Retraction Watch data shows no updates, and PMC and Europe PMC list no correction notices. The only linked item is a commentary in Kidney Int 2021;100(1):16–19 (PMID 34154708, PMC8221810), which does not mention the hospital-stay estimate or its interval. A web search found no erratum and no PubPeer thread; PubPeer itself could not be queried directly (HTTP 403).
- Sources. PMC article PMC7927648; Europe PMC full text and supplementary files; Crossref REST API; PubMed; DOI resolver and publisher site (not reachable).
Suggested correction
- Re-run or re-extract the unadjusted linear regression of hospital length of stay on AKI (stages 1–3) and correct the estimate, the 95% CI or both in the Abstract, the Results text ("Clinical course and outcomes") and Table 4, together with the P value if it changes.
- Check the mechanical-ventilation row for AKI stages 1–3 in Table 4, whose estimate (–3.9), interval (–9.4 to 12.7) and P value (0.15) are mutually inconsistent, and state how many patients each length-of-ventilation model included.
- Minor: reconcile the PICU-stay interval and P value between the Results text and Table 4; correct the acute COVID-19 No AKI column size in Table 3 (88 vs 89); and check the percentages for mechanical ventilation (2 of 8) and vasopressors (5 of 8) in the acute COVID-19 AKI column of Table 3.
Recomputation script (python recompute.py.txt)
"""Recompute the Table 4 regression intervals of PMC7927648
(Kidney Int 2021, doi:10.1016/j.kint.2021.02.026).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md,
writes fig1.svg/png next to this file, and writes five candidate figure designs to
candidates/ (c1..c5 .svg/.png plus contact_sheet.png). fig1 is a copy of the candidate
named in CHOSEN below.
Inputs are copied from the paper as archived in PMC:
Table 4 (beta or hazard ratio, 95% CI, P), also quoted in the Abstract and Results
Methods: unadjusted simple linear regression (SPSS 26), multiple imputation; cohort N = 152
Table 3: medians (IQR) by AKI status, and counts used in the minor checks
"""
import math
import os
import shutil
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from scipy import stats # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
CHOSEN = "c1"
# ---------------------------------------------------------------- inputs (from the paper)
N = 152 # Abstract / Results: "The cohort included 152 patients"
DF = N - 2 # simple linear regression, one predictor
TCRIT = stats.t.ppf(0.975, DF)
# label: (estimate, lower, upper, printed P, scale) -- Table 4
ROWS = [
("Hospital stay, AKI 1-3", 8.4, 4.4, 6.7, "<0.0001", "lin"),
("Hospital stay, AKI 2-3", 5.2, 0.07, 10.2, "0.047", "lin"),
("PICU stay, AKI 1-3", 5.0, 0.86, 9.1, "0.02", "lin"),
("PICU stay, AKI 2-3", 2.3, -3.7, 8.3, "0.45", "lin"),
("Mechanical ventilation, AKI 1-3", -3.9, -9.4, 12.7, "0.15", "lin"),
("Mechanical ventilation, AKI 2-3", -1.08, -7.4, 5.3, "0.71", "lin"),
("Mortality (hazard ratio), AKI 1-3", 0.57, 0.03, 11.8, "0.72", "log"),
]
N_VENTILATED = 4 + 2 + 1 + 4 # Table 3, "Mechanical ventilation, n": 4, 2, 1, 4
R = {}
print("=== 1. Is each Table 4 estimate centred in its own 95% CI? ===")
print(f"t critical value, df = {DF}: {TCRIT:.4f}")
for lab, b, lo, hi, p, sc in ROWS:
if sc == "log":
mid = math.exp((math.log(lo) + math.log(hi)) / 2)
half = (math.log(hi) - math.log(lo)) / 2
se = half / 1.959964
pos = (math.log(b) - math.log(mid)) / half
pp = 2 * stats.norm.sf(abs(math.log(b) / se))
else:
mid = (lo + hi) / 2
half = (hi - lo) / 2
se = half / TCRIT
pos = (b - mid) / half
pp = 2 * stats.t.sf(abs(b / se), DF)
inside = lo <= b <= hi
R[lab] = dict(b=b, lo=lo, hi=hi, p=p, sc=sc, mid=mid, half=half, se=se, pos=pos, pp=pp, inside=inside)
print(f"{lab:36s} est {b:6.2f} CI {lo:6.2f} to {hi:6.2f} inside={inside!s:5s} "
f"CI centre {mid:6.3f} est - centre {b - mid if sc == 'lin' else b / mid:+.3f}"
f"{'' if sc == 'lin' else ' (ratio)'} position in CI {pos:+.2f} "
f"SE from CI {se:.3f} P from est/SE {pp:.2g} (printed {p})")
print("\n=== 2. The hospital-stay row, AKI stages 1-3 ===")
b, lo, hi = 8.4, 4.4, 6.7
print(f"8.4 within [4.4, 6.7]: {lo <= b <= hi}; 8.4 - 6.7 = {b - hi:.1f} days above the upper bound")
print(f"centre of printed CI = {(lo + hi) / 2:.2f}; half-width = {(hi - lo) / 2:.2f}")
se_ci = (hi - lo) / 2 / TCRIT
print(f"SE implied by the printed CI width = {se_ci:.3f}")
up_rec = 2 * b - lo
se_rec = (b - lo) / TCRIT
p_rec = 2 * stats.t.sf(b / se_rec, DF)
print(f"(a) if 8.4 and the lower bound 4.4 are right: upper bound = 2*8.4 - 4.4 = {up_rec:.1f}; "
f"SE = {se_rec:.3f}; t = {b / se_rec:.2f}; P = {p_rec:.1e} (printed P < 0.0001: {p_rec < 1e-4})")
print(f" SE under (a) / SE implied by printed width = {se_rec / se_ci:.2f}")
lo_b = 2 * b - hi
print(f"(b) if 8.4 and the upper bound 6.7 are right: lower bound = 2*8.4 - 6.7 = {lo_b:.1f} > 8.4, impossible")
mid = (lo + hi) / 2
p_c = 2 * stats.t.sf(mid / se_ci, DF)
print(f"(c) if the CI 4.4-6.7 is right: estimate = {mid:.2f}; t = {mid / se_ci:.2f}; P = {p_c:.1e} "
f"(also < 0.0001: {p_c < 1e-4})")
print("Every reconstruction keeps the lower bound well above 0, so the direction and significance survive.")
print("\n=== 3. Side finding: mechanical-ventilation row, AKI stages 1-3 ===")
b, lo, hi = -3.9, -9.4, 12.7
for df, lab in ((DF, f"df = {DF} (whole cohort)"), (N_VENTILATED - 2, f"df = {N_VENTILATED - 2} (only the {N_VENTILATED} ventilated patients, Table 3)")):
tc = stats.t.ppf(0.975, df)
se = (hi - lo) / 2 / tc
print(f" {lab}: CI centre {(lo + hi) / 2:+.2f}; SE {se:.3f}; P from -3.9/SE {2 * stats.t.sf(abs(b) / se, df):.2f} (printed 0.15)")
print(f" estimate - lower = {b - lo:.1f}; upper - estimate = {hi - b:.1f}")
print("\n=== 4. Plausibility of an ~8-day difference (Table 3 medians, not a regression) ===")
print(f"acute COVID-19: 22.2 - 2.8 = {22.2 - 2.8:.1f} d; MIS-C: 8.2 - 4.6 = {8.2 - 4.6:.1f} d")
print("\n=== 5. Smaller inconsistencies noted in passing ===")
print("PICU stay, AKI 1-3: Results text '95% CI, 0.86-9; P = 0.025'; Table 4 '0.86 to 9.1', P 0.02")
print(f"Table 3 acute COVID-19 columns: 88 + 8 = {88 + 8}; Table 1 and Abstract: 97 (No AKI N = 89 in Table 1)")
print(f"Table 3 mechanical ventilation, acute COVID-19 with AKI: 2/8 = {200 / 8:.1f}% (printed 33.3)")
print(f"Table 3 vasopressors, acute COVID-19 with AKI: 5/8 = {500 / 8:.1f}% (printed 32)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
LW = 1.5 # 2 px at 96 px/in is 1.5 pt
MS = 8.5
def save(fig, stem, folder=CAND):
fig.savefig(os.path.join(folder, stem + ".svg"))
fig.savefig(os.path.join(folder, stem + ".png"), dpi=200)
plt.close(fig)
def tidy(ax, xgrid=True, ygrid=False):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
if xgrid:
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
if ygrid:
ax.yaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
LIN = [r for r in ROWS if r[5] == "lin"]
FLAG = "Hospital stay, AKI 1-3"
# --- c1: all day-scale Table 4 rows, reported intervals with CI centres, plus the reconstruction
fig, ax = plt.subplots(figsize=(7.2, 4.9))
labels, ys = [], []
y = 0
for lab, b, lo, hi, p, sc in LIN:
r = R[lab]
ax.plot([lo, hi], [y, y], color=ORANGE, lw=LW, solid_capstyle="butt", zorder=2)
for e in (lo, hi):
ax.plot([e, e], [y - 0.13, y + 0.13], color=ORANGE, lw=LW, zorder=2)
ax.plot([b], [y], "o", ms=MS, color=ORANGE, zorder=4)
ax.plot([r["mid"]], [y], "o", ms=MS + 5, mfc="none", mec=BLUE, mew=2, zorder=5)
labels.append(f"{lab}\n{b:g} ({lo:g} to {hi:g}), P {p if p.startswith('<') else '= ' + p}")
ys.append(y)
if lab == FLAG:
y += 0.9
ax.plot([4.4, up_rec], [y, y], color=BLUE, lw=LW, ls=(0, (4, 2)), zorder=2)
for e in (4.4, up_rec):
ax.plot([e, e], [y - 0.13, y + 0.13], color=BLUE, lw=LW, zorder=2)
ax.plot([8.4], [y], "D", ms=MS - 1, color=BLUE, zorder=4)
labels.append(f"Same row, rebuilt from 8.4 and 4.4\n8.4 (4.4 to {up_rec:.1f}), P = {p_rec:.0e}".replace("e-0", "e-"))
ys.append(y)
ax.annotate("8.4 lies 1.7 d\nabove its own\nupper bound", xy=(8.4, 0), xytext=(10.6, -0.05),
fontsize=10, color=TXT, va="center", ha="left",
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.8))
y += 1.15
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10)
ax.invert_yaxis()
ax.set_xlim(-11, 16.5)
ax.set_xticks(range(-10, 16, 5))
ax.set_xlabel("β: difference with vs without AKI, days")
tidy(ax)
fig.legend(handles=[
Line2D([], [], marker="o", ls="-", lw=LW, ms=MS, color=ORANGE, label="Reported β and 95% CI (Table 4)"),
Line2D([], [], marker="o", ls="none", ms=MS + 5, mfc="none", mec=BLUE, mew=2,
label="Centre of the reported CI (recomputed); a ring around a dot means the estimate is centred"),
Line2D([], [], marker="D", ls=(0, (4, 2)), lw=LW, ms=MS - 1, color=BLUE,
label="Interval centred on 8.4 with lower bound 4.4 (recomputed)")],
loc="lower left", bbox_to_anchor=(0.01, 0.0), frameon=False, fontsize=10, ncol=1)
fig.tight_layout(rect=(0, 0.17, 1, 1))
save(fig, "c1")
# --- c2: the flagged row alone, three readings on one axis
fig, ax = plt.subplots(figsize=(7.2, 3.3))
items = [
("Reported (Abstract, Results, Table 4)\n8.4 (4.4 to 6.7), P < 0.0001", 8.4, 4.4, 6.7, ORANGE, "o", "-"),
(f"If the CI is right: estimate = its centre\n{(4.4 + 6.7) / 2:.2f} (4.4 to 6.7), P = {p_c:.0e}".replace("e-", "e−"),
(4.4 + 6.7) / 2, 4.4, 6.7, BLUE, "D", (0, (4, 2))),
(f"If 8.4 and 4.4 are right: symmetric CI\n8.4 (4.4 to {up_rec:.1f}), P = {p_rec:.0e}".replace("e-0", "e−"),
8.4, 4.4, up_rec, BLUE, "D", (0, (4, 2))),
]
for i, (lab, b, lo, hi, col, mk, ls) in enumerate(items):
ax.plot([lo, hi], [i, i], color=col, lw=LW, ls=ls, zorder=2)
for e in (lo, hi):
ax.plot([e, e], [i - 0.12, i + 0.12], color=col, lw=LW, zorder=2)
ax.plot([b], [i], mk, ms=MS, color=col, zorder=4)
ax.annotate("", xy=(6.7, -0.32), xytext=(8.4, -0.32),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.0, shrinkA=0, shrinkB=0))
ax.text(7.55, -0.42, "1.7 d outside", ha="center", va="bottom", fontsize=10, color=TXT)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
ax.set_yticks(range(len(items)))
ax.set_yticklabels([it[0] for it in items], fontsize=10)
ax.set_ylim(len(items) - 0.5, -0.8)
ax.set_xlim(-0.5, 14)
ax.set_xticks(range(0, 15, 2))
ax.set_xlabel("Longer hospital stay with AKI (stages 1–3), days")
tidy(ax)
fig.legend(handles=[
Line2D([], [], marker="o", ls="-", lw=LW, ms=MS, color=ORANGE, label="Reported"),
Line2D([], [], marker="D", ls=(0, (4, 2)), lw=LW, ms=MS, color=BLUE, label="Recomputed from the paper's numbers")],
loc="lower left", bbox_to_anchor=(0.01, 0.0), frameon=False, fontsize=10, ncol=2)
fig.tight_layout(rect=(0, 0.09, 1, 1))
save(fig, "c2")
# --- c3: where each estimate sits inside its own CI (-1 = lower bound, +1 = upper bound)
fig, ax = plt.subplots(figsize=(7.2, 3.9))
order = [r[0] for r in ROWS]
ax.axvspan(-1, 1, color="#efeeea", zorder=0, lw=0)
ax.text(0, -0.85, "inside the CI", ha="center", va="center", fontsize=10, color=SEC)
for x in (-1, 1):
ax.axvline(x, color=SEC, lw=0.8, ls=(0, (2, 2)), zorder=1)
ax.axvline(0, color=SEC, lw=0.8, zorder=1)
for i, lab in enumerate(order):
r = R[lab]
ax.plot([0, r["pos"]], [i, i], color=BLUE, lw=LW, zorder=2)
ax.plot([r["pos"]], [i], "o", ms=MS, color=BLUE, zorder=3)
ax.text(r["pos"] + (0.1 if r["pos"] >= 0 else -0.1), i - 0.3, f"{r['pos'] + 0.0:+.2f}".replace("-0.00", "+0.00").replace("-", "−"),
ha="left" if r["pos"] >= 0 else "right", va="center", fontsize=10, color=TXT)
ax.set_yticks(range(len(order)))
ax.set_yticklabels([f"{lab}\n{R[lab]['b']:g} ({R[lab]['lo']:g} to {R[lab]['hi']:g})" for lab in order], fontsize=10)
ax.set_ylim(len(order) - 0.5, -1.2)
ax.set_xlim(-1.6, 3.0)
ax.set_xticks([-1, 0, 1, 2, 3])
ax.set_xticklabels(["−1\nlower bound", "0\ncentre", "+1\nupper bound", "+2", "+3"])
ax.set_xlabel("Where the reported estimate sits in its reported CI (recomputed;\nhazard ratio on the log scale)")
tidy(ax)
fig.tight_layout()
save(fig, "c3")
# --- c4: symmetry check, lower arm vs upper arm of each interval
fig, ax = plt.subplots(figsize=(7.2, 5.0))
lim = (-3, 18)
ax.plot(lim, lim, color=SEC, lw=0.8, zorder=1)
ax.text(17.6, 13.6, "symmetric CI\n(arms equal)", fontsize=10, color=SEC, ha="right", va="top")
ax.axhline(0, color=SEC, lw=0.8, ls=(0, (2, 2)), zorder=1)
ax.text(17.8, 0.25, "below 0: estimate above its own upper bound", fontsize=10, color=SEC,
ha="right", va="bottom")
TPOS = {"Hospital stay, AKI 1-3": (5.2, -2.2), "Hospital stay, AKI 2-3": (9.5, 4.4),
"PICU stay, AKI 1-3": (9.5, 1.9), "PICU stay, AKI 2-3": (-2.5, 9.6),
"Mechanical ventilation, AKI 1-3": (6.4, 16.6), "Mechanical ventilation, AKI 2-3": (-2.5, 12.6)}
for lab, b, lo, hi, p, sc in LIN:
la, ua = b - lo, hi - b
ax.plot([la], [ua], "o", ms=MS, color=BLUE, zorder=3)
tx, ty = TPOS[lab]
ax.annotate(f"{lab}\n({la:.1f}, {ua:.1f})".replace("-", "−").replace("AKI 1−3", "AKI 1-3").replace("AKI 2−3", "AKI 2-3"),
xy=(la, ua), xytext=(tx, ty), fontsize=10, ha="left", va="center", color=TXT,
arrowprops=dict(arrowstyle="-", color=SEC, lw=0.7, shrinkA=2, shrinkB=5))
ax.set_xlim(*lim)
ax.set_ylim(*lim)
ax.set_aspect("equal")
ax.set_xticks(range(0, 19, 5))
ax.set_yticks(range(0, 19, 5))
ax.set_xlabel("Estimate − lower bound, days (recomputed from Table 4)")
ax.set_ylabel("Upper bound − estimate, days")
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
fig.tight_layout()
save(fig, "c4")
# --- c5: the three printed locations as text, above one number line
fig = plt.figure(figsize=(7.2, 5.0))
quotes = [
("Abstract", "“… 8.4 days longer hospitalizations compared to patients without acute kidney injury\n(95% confidence interval, 4.4-6.7).”"),
("Results, Clinical course and outcomes", "“… greater length of hospital stay of 8.4 days compared with patients without AKI\n(95% CI, 4.4–6.7; P < 0.0001).”"),
("Table 4, Length of hospital stay, AKI stage 1 + 2 + 3", "β (95% CI) “8.4 (4.4 to 6.7)” P value “<0.0001”"),
]
for i, (where, q) in enumerate(quotes):
y0 = 0.80 - i * 0.165
fig.patches.append(matplotlib.patches.FancyBboxPatch(
(0.02, y0), 0.96, 0.15, boxstyle="round,pad=0.004,rounding_size=0.01", transform=fig.transFigure,
facecolor="#ffffff", edgecolor="#d6d5d0", lw=1))
fig.text(0.035, y0 + 0.13, where, fontsize=10, fontweight="bold", va="top", color=TXT)
fig.text(0.035, y0 + 0.088, q, fontsize=10, va="top", color=TXT, linespacing=1.3)
fig.text(0.5, 0.425, "the same estimate and interval in all three places, drawn on one axis below",
ha="center", fontsize=10, color=SEC)
ax = fig.add_axes((0.05, 0.13, 0.9, 0.25))
ax.plot([4.4, 6.7], [0, 0], color=ORANGE, lw=LW * 2, solid_capstyle="butt", zorder=2)
for e in (4.4, 6.7):
ax.plot([e, e], [-0.18, 0.18], color=ORANGE, lw=LW, zorder=2)
ax.plot([8.4], [0], "o", ms=MS + 1, color=ORANGE, zorder=4)
ax.plot([5.55], [0], "o", ms=MS, mfc=BG, mec=BLUE, mew=2, zorder=5)
ax.text(5.55, 0.32, "reported 95% CI 4.4 to 6.7", ha="center", fontsize=10, color=TXT)
ax.text(8.4, 0.32, "reported β = 8.4", ha="center", fontsize=10, color=TXT)
ax.text(5.55, -0.42, "CI centre 5.55\n(recomputed)", ha="center", va="top", fontsize=10, color=TXT)
ax.annotate("", xy=(6.7, -0.25), xytext=(8.4, -0.25),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=1.0, shrinkA=0, shrinkB=0))
ax.text(7.55, -0.42, "1.7 d", ha="center", va="top", fontsize=10, color=TXT)
ax.axvline(0, color=SEC, lw=0.8)
ax.set_xlim(-0.3, 10)
ax.set_ylim(-1, 0.7)
ax.set_yticks([])
ax.set_xticks(range(0, 11, 2))
ax.set_xlabel("Longer hospital stay with AKI, days")
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
save(fig, "c5")
# contact sheet (PNG only, for choosing a design)
import matplotlib.image as mpimg # noqa: E402
fig, axs = plt.subplots(3, 2, figsize=(14, 16))
for ax, stem in zip(axs.flat, ("c1", "c2", "c3", "c4", "c5")):
ax.imshow(mpimg.imread(os.path.join(CAND, stem + ".png")))
ax.set_title(stem, fontsize=16, fontweight="bold", loc="left")
ax.axis("off")
axs.flat[-1].axis("off")
fig.tight_layout()
fig.savefig(os.path.join(CAND, "contact_sheet.png"), dpi=100)
plt.close(fig)
for ext in ("svg", "png"):
shutil.copyfile(os.path.join(CAND, f"{CHOSEN}.{ext}"), os.path.join(HERE, f"fig1.{ext}"))
print(f"\nwrote candidates c1-c5 and contact_sheet.png in {CAND}; fig1 = {CHOSEN}")
Abstract reports 58 (53%) of 113 resistant sites; Results' own counts give 57 (51%) of 112
Lancet Infect Dis (2014) · doi:10.1016/S1473-3099(14)70855-2 · PMC4178238
Summary
This systematic review reports its headline result in the abstract as chloroquine resistance in "58 (53%) of 113 assessable study sites". The Results instead give 57 (51%) of 112 site estimates, and the body's own sub-counts support that figure: 177 site estimates minus 65 that could not be categorised (37 + 15 + 4 + 4 + 5) leaves 112, and resistance categories 1, 2 and 3 (35 + 8 + 14) add up to 57. The abstract figure is also wrong on its own terms, because 58/113 = 51.3%, which rounds to 51%, not 53%. The qualitative finding, that about half of assessable sites showed resistance, is unchanged. But the abstract's one headline prevalence figure, the number most likely to be quoted, is about 2 percentage points too high, and its count and denominator match nothing in the paper.
What the paper reports
Abstract, Findings:
We identified 129 eligible clinical trials involving 21 694 patients at 179 study sites and 26 case reports describing 54 patients. Chloroquine resistance was present in 58 (53%) of 113 assessable study sites, spread across most countries that are endemic for P vivax.
Results, fifth paragraph:
Estimates of chloroquine efficacy could be derived for 177 sites from 144 treatment groups. In 65 (37%) of these sites chloroquine resistance could not be categorised for the following reasons: early administration of primaquine (37 sites); no documentation of parasitological data at day 28 (15 sites); sample size of fewer than ten patients (four sites); good response documented in asymptomatic patients (four sites); and administration of only a single dose of chloroquine (five sites). Overall P vivax was defined as chloroquine resistant for 57 (51%) of 112 site estimates, 35 (61%) of these sites fulfilled the predefined criteria for category 1 resistance, eight (14%) for category 2 resistance, and 14 (25%) for category 3 resistance.
Table 1, "Site estimates of chloroquine efficacy" column, "Overall" row:
Overall | 129 | 230 | 177
Discussion, ninth paragraph:
Our definitions were conservative, with data excluded from almost 40% of studies that we thought could be confounded. Almost half of the informative studies suggested evidence of reduced chloroquine efficacy.
What the paper's own numbers show
Method. I checked each printed percentage against its own fraction. I rebuilt the assessable denominator from the Table 1 site total and the five listed reasons for exclusion, and rebuilt the resistant count from the three resistance categories. I checked the Table 1 site total by each of its three breakdowns (location, year, origin of patients). Finally, I searched for fractions that would round to 53%. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Resistant sites, abstract | Abstract, Findings | 58 (53%) of 113 | 58/113 = 51.3%, which rounds to 51%, not 53% |
| Resistant sites, Results | Results, para 5 | 57 (51%) of 112 | 57/112 = 50.9% (51%); consistent |
| Chloroquine site estimates | Table 1; Results, para 5 | 177 | 134 + 13 + 22 + 8 = 177 by location; 8 + 41 + 80 + 48 = 177 by year; 22 + 21 + 134 = 177 by origin |
| Uncategorisable sites | Results, para 5 | 65 (37%) | 37 + 15 + 4 + 4 + 5 = 65; 65/177 = 36.7% (37%) |
| Assessable sites | Results 112; Abstract 113 | 112 / 113 | 177 − 65 = 112 |
| Resistant sites = categories 1 + 2 + 3 | Results 57; Abstract 58 | 57 / 58 | 35 + 8 + 14 = 57 |
| Category shares | Results, para 5 | 61% / 14% / 25% | of 57: 61.4% / 14.0% / 24.6% (match); of 58: 60.3% / 13.8% / 24.1% (35/58 would print as 60%, not 61%) |
| Implied sensitive sites | derived | — | 112 − 57 = 55 (Results); 113 − 58 = 55 (Abstract) |
| Fractions that round to 53% | derived | 53% | 58/109, 58/110, 60/113 or 59/112; none of these counts appears in the paper |
| Size of the discrepancy | derived | — | printed 53% is 1.67 points above 58/113 and 2.11 points above 57/112; the two fractions differ by 0.43 points |
Figure
Figure 1. The two passages that report the overall prevalence of chloroquine-resistant P vivax sites. Values printed in the abstract are in orange; values printed in the Results, which agree with the body's sub-counts, and values recomputed from the paper's numbers are in blue. The abstract's count (58) and denominator (113) are each one site higher than the Results' (57 and 112), and its percentage (53%) does not follow from its own fraction (58/113 = 51.3%). The lower panel rebuilds the Results' figures from the Table 1 site total, the five listed reasons for exclusion and the three resistance categories. All numbers are from the paper or from recompute.py.txt.
Why it matters
The abstract's sentence is the paper's single summary figure for how widespread chloroquine-resistant P vivax is, and it is the version most likely to be quoted in later reviews, guideline background documents and resistance summaries. The error is small: the figure the body supports is 51% (57/112), and the abstract's own fraction also gives 51%. Its direction and the qualitative reading ("almost half of the informative studies", in the Discussion's words) are unchanged, and so are the conclusions that resistance is geographically widespread and that monitoring needs to improve. The practical consequences are that readers who quote the abstract carry a percentage about 2 points too high, and that the counts 58 and 113 cannot be reconciled with the paper's own tables and text. The abstract and the body disagree in three places at once (count, denominator and percentage), which will puzzle anyone who tries to check or update the analysis.
Explanations considered
- A different unit of analysis. The abstract says "assessable study sites" and the Results say "site estimates". The paper gives only two site totals: 179 study sites for all drugs (Abstract) and 177 site estimates of chloroquine efficacy (Results; Table 1, whose three breakdowns each sum to 177). The only assessable denominator the paper derives is 177 − 65 = 112. No figure in the text, tables or legends gives 113. Applying the 65 exclusions to the 179 study sites would give 114, not 113. This does not account for the numbers.
- A different set of resistance categories. Resistance is defined as categories 1, 2 or 3, and these sum to 35 + 8 + 14 = 57. No further resistant subgroup is described that could add a 58th site. Both versions imply 55 sensitive sites (112 − 57 = 55 and 113 − 58 = 55). The abstract therefore has one extra site that is counted as resistant. The printed category shares also fit a base of 57 rather than 58 (35/58 = 60.3% would print as 60%, not 61%). This does not account for the numbers.
- Rounding convention. 58/113 = 51.33%. No rounding rule turns that into 53%. A printed 53% would need 58/109, 58/110 or 60/113 (or 59/112), and none of these counts appears in the paper. This does not account for the percentage.
- A conversion artefact in the archived text. The same sentences appear in the Lancet's own typeset PDF (the online-first version), in the PMC HTML and in the Europe PMC XML. This is not a conversion artefact.
- A data update between versions, such as an earlier count left in the abstract. The body is internally consistent throughout (65 = 37 + 15 + 4 + 4 + 5; 65/177 = 37%; 57/112 = 51%; the category shares of 57). That points to the abstract's figure as the inconsistent one, but the paper gives no information on how it arose, and even such a history would not explain why 53% does not equal 58/113.
- The supplement. The single supplementary appendix contains reference lists and a reference table only. It has no site-level categories or counts, so it cannot reconcile 58/113 with 57/112.
Status
- Version of record. I checked the Lancet's typeset PDF ("Published online September 9, 2014", www.thelancet.com/infection headers), held in the Oxford ORA repository. It contains both sentences word for word: "Chloroquine resistance was present in 58 (53%) of 113 assessable study sites" in the abstract, and "Overall P vivax was defined as chloroquine resistant for 57 (51%) of 112 site estimates" in the Results. The PMC copy is the publisher-deposited open-access (CC BY) version and shows the same text, as do the PMC HTML page and the Europe PMC full-text XML. The publisher's site (thelancet.com) refused automated access (HTTP 403), and the DOI resolver stopped at a redirect page, so I could not compare against the final issue version (14(10):982–91) there. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements. I read the one supplementary appendix (mmc1.pdf, 21 pages, from the Europe PMC supplementary-files bundle). It holds the reference lists for the clinical trials and case reports and a reference table, with no site-level resistance data. The authors' 2015 reply to letters (PMC4629410; Lancet Infect Dis 2015;15:630–1) says the full dataset is available through the Worldwide Antimalarial Resistance Network. I did not find a downloadable copy: the network's publication page (iddo.org/node/5315) lists the paper but offers no data files. The reply addresses other points (literature from China, follow-up duration) and does not mention the site counts.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and a Crossref query for works that update it returned none. PMC and Europe PMC list no correction notices. The Europe PMC record (PMID 25213732) links only comments: an editorial comment and two 2015 letters with the authors' reply. The reply, which answers both letters (on literature from China and on follow-up duration), does not mention the site counts. The PubPeer API was blocked by a Cloudflare challenge, and a web search found no PubPeer thread for the paper.
- Sources. PMC article PMC4178238; Lancet typeset PDF via ORA (ora.ox.ac.uk, uuid:db45d120-9689-40f3-8e61-1006e9886bc0); Europe PMC full text and supplementary files; authors' reply PMC4629410; Crossref; Europe PMC; iddo.org/node/5315; PubPeer.
Suggested correction
- In the abstract's Findings, replace "Chloroquine resistance was present in 58 (53%) of 113 assessable study sites" with "Chloroquine resistance was present in 57 (51%) of 112 assessable study sites". This matches the Results, the five listed exclusions from the 177 site estimates in Table 1, and the 35 + 8 + 14 resistance categories.
- If the authors' dataset shows that 58 of 113 is the correct count, then the Results paragraph (the 112 denominator, the 65 exclusions and their breakdown, the 57 total and the category counts and shares) needs correcting instead. The abstract percentage would then still need to change from 53% to 51%, because 58/113 = 51.3%.
Recomputation script (python recompute.py.txt)
# Recompute the site counts behind the chloroquine-resistance prevalence in
# PMC4178238 (Lancet Infect Dis 2014; doi:10.1016/S1473-3099(14)70855-2)
# and regenerate fig1.svg / fig1.png in this file's folder.
# Run: python recompute.py.txt
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
pct = lambda a, b: 100.0 * a / b
# ---- Numbers printed in the paper -------------------------------------------
abs_res, abs_den, abs_pct = 58, 113, 53 # Abstract, Findings
res_res, res_den, res_pct = 57, 112, 51 # Results, para 5
sites_all_drugs = 179 # Abstract: "179 study sites"
cq_sites = 177 # Results / Table 1
uncat = {"early primaquine": 37, "no day-28 data": 15, "n < 10": 4,
"asymptomatic, good response": 4, "single CQ dose": 5}
uncat_reported, uncat_pct_reported = 65, 37
cats = {1: 35, 2: 8, 3: 14}
cats_pct_reported = {1: 61, 2: 14, 3: 25}
table1 = {"location": [134, 13, 22, 8], "year": [8, 41, 80, 48],
"origin": [22, 21, 134]}
print("== Abstract figure checked against its own fraction")
print("58/113 = %.2f%% -> rounds to %d%% (printed %d%%)"
% (pct(abs_res, abs_den), round(pct(abs_res, abs_den)), abs_pct))
print("== Results figure checked against its own fraction")
print("57/112 = %.2f%% -> rounds to %d%% (printed %d%%)"
% (pct(res_res, res_den), round(pct(res_res, res_den)), res_pct))
print("== Table 1 site-estimate totals")
for k, v in table1.items():
print(" by %-8s: %s = %d" % (k, "+".join(map(str, v)), sum(v)))
print("== Uncategorisable sites (Results para 5)")
u = sum(uncat.values())
print(" 37+15+4+4+5 = %d (printed %d); %d/177 = %.1f%% (printed %d%%)"
% (u, uncat_reported, u, pct(u, cq_sites), uncat_pct_reported))
assess = cq_sites - u
print(" assessable = 177 - %d = %d (Results %d; Abstract %d)"
% (u, assess, res_den, abs_den))
print("== Resistant sites = categories 1+2+3")
r = sum(cats.values())
print(" 35+8+14 = %d (Results %d; Abstract %d)" % (r, res_res, abs_res))
for c, n in cats.items():
print(" category %d: %d/57 = %.1f%% (printed %d%%)"
% (c, n, pct(n, r), cats_pct_reported[c]))
print(" same shares with 58 as base: " + ", ".join(
"%d/58 = %.1f%%" % (n, pct(n, 58)) for n in cats.values()))
print(" 179 study sites - 65 = %d (not 113)" % (sites_all_drugs - u))
print("== Implied sensitive (category 4) sites")
print(" Results: 112-57 = %d; Abstract: 113-58 = %d"
% (res_den - res_res, abs_den - abs_res))
print("== Differences")
print(" printed 53%% minus 58/113: %.2f points" % (abs_pct - pct(58, 113)))
print(" printed 53%% minus 57/112: %.2f points" % (abs_pct - pct(57, 112)))
print(" 58/113 minus 57/112: %.2f points" % (pct(58, 113) - pct(57, 112)))
print("== Fractions that would round to 53%")
print(" d with round(58/d) = 53%%: %s"
% [d for d in range(90, 131) if round(pct(58, d)) == 53])
print(" n with round(n/113) = 53%%: %s"
% [n for n in range(40, 81) if round(pct(n, 113)) == 53])
print(" n with round(n/112) = 53%%: %s"
% [n for n in range(40, 81) if round(pct(n, 112)) == 53])
print(" n with round(n/177) = 53%%: %s"
% [n for n in range(60, 120) if round(pct(n, 177)) == 53])
print(" n with round(n/179) = 53%%: %s"
% [n for n in range(60, 120) if round(pct(n, 179)) == 53])
print(" 58/179 = %.1f%%, 58/177 = %.1f%%, 57/177 = %.1f%%"
% (pct(58, 179), pct(58, 177), pct(57, 177)))
# ---- Figure 1: abstract vs Results, with the body's arithmetic -------------
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#d9d8d4"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"svg.fonttype": "none"})
fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 72); ax.set_ylim(0, 50)
ax.axis("off"); ax.set_facecolor(BG)
def panel(x, y, w, h, edge, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1",
fc="white", ec=edge, lw=1.5))
ax.text(x + 1, y + h - 1.6, title, fontsize=11, fontweight="bold",
color=TXT, va="top")
# Left: Abstract
panel(1.5, 21, 28.5, 25, ORANGE, "Abstract, Findings")
ax.text(2.5, 40.5, "“Chloroquine resistance was present\nin 58 (53%) of 113 "
"assessable\nstudy sites”", fontsize=10, color=TXT, va="top",
style="italic", linespacing=1.35)
ax.text(2.5, 31.0, "Count 58", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(2.5, 28.0, "Denominator 113", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(2.5, 25.0, "Percentage 53%", fontsize=11, color=ORANGE, fontweight="bold")
ax.text(2.5, 22.0, "but 58/113 = %.1f%% → 51%%" % pct(58, 113),
fontsize=10, color=BLUE, fontweight="bold")
# Right: Results
panel(42, 21, 28.5, 25, BLUE, "Results, paragraph 5")
ax.text(43, 40.5, "“Overall P vivax was defined as\nchloroquine resistant "
"for 57 (51%)\nof 112 site estimates”", fontsize=10, color=TXT,
va="top", style="italic", linespacing=1.35)
ax.text(43, 31.0, "Count 57", fontsize=11, color=BLUE, fontweight="bold")
ax.text(43, 28.0, "Denominator 112", fontsize=11, color=BLUE, fontweight="bold")
ax.text(43, 25.0, "Percentage 51%", fontsize=11, color=BLUE, fontweight="bold")
ax.text(43, 22.0, "57/112 = %.1f%% → 51%%" % pct(57, 112),
fontsize=10, color=BLUE, fontweight="bold")
# Connector
ax.add_patch(FancyArrowPatch((31.0, 33.5), (41.3, 33.5), arrowstyle="<->",
mutation_scale=14, lw=2, color=SEC))
ax.text(36, 35.2, "conflict", ha="center", fontsize=10, color=SEC, fontweight="bold")
ax.text(36, 31.6, "one site\nhigher in\nboth count\nand\ndenominator", ha="center",
va="top", fontsize=10, color=SEC, linespacing=1.2)
# Bottom: the body's own arithmetic
panel(1.5, 1.5, 69, 15.5, GRID, "What the Results' own sub-counts give (recomputed)")
ax.text(2.5, 11.6, "Site estimates (Table 1) 177", fontsize=10, color=TXT)
ax.text(2.5, 8.6, "− uncategorisable 37+15+4+4+5 = %d" % u, fontsize=10, color=TXT)
ax.text(2.5, 5.6, "= assessable %d" % assess, fontsize=11, color=BLUE, fontweight="bold")
ax.text(37, 11.6, "Resistant = category 1 + 2 + 3", fontsize=10, color=TXT)
ax.text(37, 8.6, "35 + 8 + 14 = %d" % r, fontsize=10, color=TXT)
ax.text(37, 5.6, "= resistant %d → %d/%d = %.1f%%" % (r, r, assess, pct(r, assess)),
fontsize=11, color=BLUE, fontweight="bold")
ax.text(2.5, 2.6, "No count or subgroup in the paper gives 113 or 58; 53% needs 58/109, 58/110 or 60/113.",
fontsize=10, color=SEC)
# Legend (names the colours)
ax.text(1.8, 48.3, "■", color=ORANGE, fontsize=12, va="center")
ax.text(3.6, 48.3, "printed in the abstract", color=TXT, fontsize=10, va="center")
ax.text(26, 48.3, "■", color=BLUE, fontsize=12, va="center")
ax.text(27.8, 48.3, "recomputed from / consistent with the body's counts",
color=TXT, fontsize=10, va="center")
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, "fig1." + ext), facecolor=BG, **kw)
print("wrote fig1.svg and fig1.png in", HERE)
Abstract pairs the adjusted BNT162b2 estimate with the interval of the unadjusted ChAdOx1 estimate
Lancet Infect Dis (2021) · doi:10.1016/S1473-3099(21)00330-3 · PMC8221734
Summary
The abstract reports the adjusted one-dose effectiveness of BNT162b2 against COVID-19 hospitalisation in adults aged at least 80 years as 71·4% (95% CI 46·5–90·6). The Results text and Table 2 give the same estimate as 71·4% (43·1–86·2), and that interval follows exactly from the adjusted odds ratio printed in Table 2, 0·286 (0·138–0·569), since the paper defines effectiveness as 1 − OR. The interval in the abstract, 46·5–90·6, is identical, both as effectiveness and as odds ratio (0·094–0·535), to the interval of a different estimate: the unadjusted ChAdOx1 nCoV-19 effectiveness of 76·7%. The point estimate and the conclusion are unaffected, but a headline sentence of the abstract gives the estimate an interval that is shifted upwards by 3.4 points at the lower end and 4.4 at the upper end, wider on the log-odds scale (implied SE 23% larger), and not centred on its own point estimate. The error is in the version of record and has not been corrected.
What the paper reports
Abstract, Findings (BNT162b2):
18 (13%) of 135 people with SARS-CoV-2 infection and 90 (34%) of 269 controls received one dose of BNT162b2. The adjusted vaccine effectiveness was 71·4% (95% CI 46·5–90·6).
Abstract, Findings (ChAdOx1 nCoV-19 and the period-restricted BNT162b2 estimate):
Nine (25%) of 36 people with COVID-19 infection and 53 (59%) of 90 controls received one dose of ChAdOx1 nCoV-19. The adjusted vaccine effectiveness was 80·4% (95% CI 36·4–94·5). When BNT162b2 effectiveness analysis was restricted to the period covered by ChAdOx1 nCoV-19, the estimate was 79·3% (95% CI 47·0–92·5).
Results, third paragraph (BNT162b2):
18 (13%) of the 135 people with SARS-CoV-2 infection and 90 (34%) of 269 controls received one dose BTN162b2 (difference -20·2%), giving an unadjusted vaccine effectiveness of 69·4% (95% CI 47·7–82·9) and an adjusted effectiveness of 71·4% (43·1–86·2; table 2 ).
Results, fifth paragraph (ChAdOx1 nCoV-19):
Nine (25%) of the 36 people with SARS-CoV-2 infection and 53 (59%) of 90 controls received one dose of ChAdOx1 nCoV-19 (difference -33·9%), giving an unadjusted vaccine effectiveness of 76·7% (95% CI 46·5–90·6) and adjusted effectiveness of 80·4% (36·4–94·5; table 2).
Table 2, "One dose of BNT162b2 (Pfizer)", logistic regression model, row "One dose" (vaccine effectiveness, odds ratio, p value):
71·4 (43·1 to 86·2) | 0·286 (0·138–0·569) | <0·0001
Table 2, "One dose of ChAdOx1 nCoV-19 (Oxford–AstraZeneca)", row "Unadjusted":
76·7 (46·5 to 90·6) | 0·233 (0·094–0·535) | <0·0001
What the paper's own numbers show
Method. The paper defines vaccine effectiveness as (1 − OR) × 100, and all eight one-dose effect rows of Table 2 obey that relation to the printed precision, except for a 0.1-point rounding difference in the upper bound of the ChAdOx1 matched-conditional row (printed 93·2; 1 − 0·067 gives 93.3). So any effectiveness interval can be converted back to an odds-ratio interval and compared with the table. For the adjusted estimate the odds ratio cannot be recomputed from raw counts, but its effectiveness interval follows directly from the printed OR interval. For the unadjusted estimates, the 2 × 2 counts given in the abstract and Results allow an independent recomputation (Wald interval on the log-odds scale, and the conditional maximum-likelihood estimate with its exact interval). A further check uses the fact that a 95% Wald interval is symmetric about ln(OR): I measure how far each point estimate lies from the log-midpoint of its interval, in units of the interval's half-width. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| BNT162b2 adjusted VE (95% CI) | Abstract | 71·4% (46·5–90·6) | 71.4 (43.1–86.2) from Table 2 OR 0·286 (0·138–0·569) |
| BNT162b2 adjusted VE (95% CI) | Results ¶3; Table 2 | 71·4 (43·1 to 86·2) | 71.4 (43.1–86.2), agrees |
| Abstract BNT162b2 interval, as OR | Abstract | 46·5–90·6 | OR 0.094–0.535 |
| ChAdOx1 unadjusted VE and OR | Results ¶5; Table 2 | 76·7 (46·5 to 90·6); OR 0·233 (0·094–0·535) | 76.7 (46.5–90.6), agrees; same interval as the abstract's BNT162b2 interval |
| Table 2 rows whose VE interval is 46·5 to 90·6 | Table 2 | — | only "ChAdOx1 unadjusted" |
| Log-midpoint of the abstract BNT162b2 interval | Abstract | — | OR 0.224 (VE 77.6), vs point estimate OR 0.286 |
| Point estimate's distance from its interval's log-midpoint | Abstract, BNT162b2 adjusted | — | +0.28 half-widths |
| Same distance, all eight Table 2 effect rows and the other two abstract intervals | Table 2; Abstract | — | between 0.00 and +0.04 half-widths |
| BNT162b2 unadjusted OR from counts 18/135 vs 90/269 | Counts: Abstract, Results ¶3; reported OR/VE: Table 2 (VE also Results ¶3) | 0·306 (0·171–0·523); VE 69·4 (47·7–82·9) | Wald 0.306 (0.175–0.534), VE 69.4 (46.6–82.5); conditional MLE 0.307 (0.165–0.545), VE 69.3 (45.5–83.5) |
| ChAdOx1 unadjusted OR from counts 9/36 vs 53/90 | Counts: Abstract, Results ¶5; reported OR/VE: Table 2 (VE also Results ¶5) | 0·233 (0·094–0·535); VE 76·7 (46·5–90·6) | Wald 0.233 (0.098–0.552), VE 76.7 (44.8–90.2); conditional MLE 0.235 (0.087–0.589), VE 76.5 (41.1–91.3) |
| ChAdOx1 adjusted VE | Abstract; Table 2 | 80·4% (36·4–94·5); OR 0·196 (0·055–0·636) | 80.4 (36.4–94.5), agrees |
| BNT162b2 restricted-period VE | Abstract | 79·3% (47·0–92·5) | OR 0.207 (0.075–0.530); centred (+0.04 half-widths) |
| SE(ln OR) implied by the interval | Abstract vs Table 2 | — | 0.444 (abstract) vs 0.361 (Table 2), 23% larger |
The Table 2 conversion holds to the printed precision for every row except the 0.1-point rounding difference noted above (ChAdOx1 matched conditional, 93·2 vs 93.3), and exactly for the BNT162b2 adjusted row, so the correct interval for 71·4% is 43·1–86·2. The abstract's interval matches the ChAdOx1 unadjusted interval digit for digit, and its log-midpoint (OR 0.224) is that row's midpoint, not the midpoint of the BNT162b2 adjusted interval (OR 0.280).
Figure
Figure 1. Odds ratios for one dose versus none, on a log scale (VE = 1 − OR), with 95% intervals. Orange circles and lines are values printed in the paper (the Abstract row shows the abstract's printed VE converted to OR, 1 − VE); blue squares and lines are unadjusted odds ratios recomputed by recompute.py.txt from the paper's 2 × 2 counts (Wald intervals); the open black diamond is the log-midpoint of the abstract's interval, computed by recompute.py.txt. The right-hand column gives each row as VE % (95% CI), in the colour of its row. The abstract row (top) carries the BNT162b2 adjusted point estimate, 0.286, but its interval ends exactly where the ChAdOx1 unadjusted interval ends (dashed lines at 0.094 and 0.535), and its centre (0.224) lies well to the left of the point estimate. The Table 2 BNT162b2 rows and the interval recomputed from the BNT162b2 counts end at 0.138–0.175 at the lower OR bound (adjusted 0.138, unadjusted 0.171, Wald from counts 0.175), not at 0.094. The vertical line marks no effect (OR 1).
Figure 2. The three places in the paper that give the conflicting numbers. The abstract (orange outline) and Results/Table 2 (grey outline, top right) give the same BNT162b2 point estimate with different intervals; the abstract's interval is identical to the unadjusted ChAdOx1 nCoV-19 interval in Results paragraph 5 and Table 2 (bottom right). Orange text is printed in the paper; blue text is the 1 − OR conversion computed by recompute.py.txt.
Why it matters
- It is a headline estimate. The adjusted one-dose BNT162b2 effectiveness is the first of the paper's two primary results, and the abstract is the part most readers, indexers and citing authors use. The accompanying Comment in the same issue (PMC8221733) already repeats the abstract's interval: "Estimated vaccine efficacy for symptomatic disease 14 days after one dose of vaccine was 71·4 % (95% CI 46·5–90·6) for BNT162b2".
- It shifts the interval upwards. The upper bound is 90·6 instead of 86·2 (+4.4 points) and the lower bound 46·5 instead of 43·1 (+3.4 points), so the abstract's lower bound, the one readers use to judge the minimum plausible protection, is more favourable than the data support. The interval is not narrower: it spans 44.1 VE points against 43.1, and the SE(ln OR) it implies is larger, 0.444 against 0.361. Measured from the point estimate, it reaches 19.2 points above 71·4 (Table 2: 14.8) and 24.9 points below (Table 2: 28.3).
- It can bias re-use. A meta-analyst who back-calculates a standard error from the abstract would obtain SE(ln OR) = 0.444 instead of 0.361, giving this study about two-thirds (0.66×) of its proper inverse-variance weight, and would centre the estimate on an interval whose midpoint corresponds to VE 77.6%, not 71.4%.
- The conclusion stands. Both intervals exclude zero effectiveness, the point estimate is correct everywhere, and the Interpretation ("One dose of either BNT162b2 or ChAdOx1 nCoV-19 resulted in substantial risk reductions of COVID-19-related hospitalisation in people aged at least 80 years") is unaffected. The error is in how the abstract reports the result, not in the analysis.
Explanations considered
- A different BNT162b2 model. Not supported. Table 2 lists every BNT162b2 estimate: unadjusted 47·7 to 82·9, logistic adjusted 43·1 to 86·2, matched conditional −0·20 to 81·9, and unadjusted within 14 days −65·8 to 48·2. The period-restricted estimate, 79·3% (47·0–92·5), appears as its own sentence in the abstract. None is 46·5–90·6.
- A different confidence-interval method applied to the BNT162b2 data. Not supported. From the 18/135 and 90/269 counts, Wald and exact conditional intervals give VE of about 45–47 to 82–84, far below 90·6 at the top. The adjusted interval would differ from these, but the paper prints it (43·1–86·2), and the abstract's interval is not centred on 71·4 on the log-odds scale (+0.28 half-widths), whereas every interval in Table 2 and the other two abstract intervals are centred within 0.04 half-widths.
- Copying from the adjacent ChAdOx1 row. Consistent with all the numbers. 46·5–90·6 is the unadjusted ChAdOx1 interval, both as VE and as OR (0·094–0·535), and it is the only row of Table 2 with that interval. This is an observation about the numbers only; how the error arose is not known.
- A conversion or typesetting artifact. Not possible. The publisher's typeset PDF of the final issue version shows the same abstract sentence, Results text and Table 2 entries as the PMC XML, and the intervals are complete, internally consistent number pairs rather than garbled characters.
- Definition or units. Not an explanation. VE = (1 − OR) × 100 holds for every printed effect row of Table 2, to within a 0.1-point rounding difference in one bound (ChAdOx1 matched conditional).
- A correction already issued. None for this interval (see Status).
- Side observations, not part of the claim. The Results give the BNT162b2 difference in vaccination as −20·2%; the counts give 13.3% − 33.5% = −20.1 points (a rounding difference). The Wald p value implied by the adjusted OR and its interval is about 0.0005 (z = −3.46), while Table 2 prints <0·0001; a likelihood-ratio or other test could give a somewhat different value, so this is noted only for completeness.
Status
- Version of record. The publisher's web pages could not be retrieved by automated requests (the DOI redirect returned only a redirect stub and thelancet.com returned HTTP 403). I used the publisher's own typeset PDF from the PMC Open Access dataset (PMC8221734.1), which is the final issue version ("The Lancet Infectious Diseases, 21 (2021) 1539-1548", footer "Vol 21 November 2021", created 2021-10-18, after the July 2021 erratum). It carries 71·4% (95% CI 46·5–90·6) in the abstract (p 2), 71·4% (43·1–86·2) in the Results (p 5), and the Table 2 rows quoted above (p 7). The PMC web page and the Europe PMC full-text XML show the same numbers. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The Europe PMC supplementary bundle contains only the two article figure images. Neither the PDF nor the XML refers to a separate appendix. A supplement could not change the verdict, because Table 2 prints the adjusted OR and its interval.
- Corrections and comments. As of 2026-10-24, the only notice is an erratum (doi:10.1016/S1473-3099(21)00397-2, PMC8318405, Lancet Infect Dis 2021;21(8):e208, dated 5 July 2021), listed by Crossref and Europe PMC; it corrects the spelling of an author's name and does not mention the interval. Crossref lists no other update relations, including none from Retraction Watch. The linked Comment (PMC8221733) repeats the abstract's interval. The PubMed abstract (PMID 34174190) still reads 71·4% (95% CI 46·5-90·6). PubPeer could not be queried directly (its site and API sit behind an automated-access check); web searches for the article together with "PubPeer" found no thread, and found no Retraction Watch coverage.
- Sources. PMC article PMC8221734 and its Open Access PDF; Europe PMC full-text XML and supplementary files; erratum PMC8318405; Comment PMC8221733; PubMed 34174190; Crossref; Europe PMC; PubPeer; https://doi.org/10.1016/S1473-3099(21)00330-3.
Suggested correction
In the abstract (Findings), replace
The adjusted vaccine effectiveness was 71·4% (95% CI 46·5–90·6).
with
The adjusted vaccine effectiveness was 71·4% (95% CI 43·1–86·2).
so that the abstract agrees with the Results text, Table 2 and the adjusted odds ratio 0·286 (0·138–0·569). The same change would apply to the PubMed abstract and could be noted against the accompanying Comment, which quotes the abstract's interval. No other number in the abstract needs changing.
Recomputation script (python recompute.py.txt)
"""Recompute the one-dose vaccine-effectiveness intervals of PMC8221734
(Lancet Infect Dis 2021, doi:10.1016/S1473-3099(21)00330-3, test-negative case-control
study of BNT162b2 and ChAdOx1 nCoV-19 in adults aged at least 80 years).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC (and identical in the publisher PDF):
Abstract, Findings: BNT162b2 adjusted VE 71.4% (95% CI 46.5-90.6)
ChAdOx1 adjusted VE 80.4% (36.4-94.5)
BNT162b2 restricted to the ChAdOx1 period 79.3% (47.0-92.5)
Table 2 (VE and OR, 95% CI):
BNT162b2 unadjusted 69.4 (47.7 to 82.9) OR 0.306 (0.171-0.523)
BNT162b2 logistic, one dose 71.4 (43.1 to 86.2) OR 0.286 (0.138-0.569)
BNT162b2 matched conditional 57.4 (-0.20 to 81.9) OR 0.426 (0.181-1.002)
BNT162b2 within 14 days 6.5 (-65.8 to 48.2) OR 0.935 (0.518-1.658)
ChAdOx1 unadjusted 76.7 (46.5 to 90.6) OR 0.233 (0.094-0.535)
ChAdOx1 logistic, one dose 80.4 (36.4 to 94.5) OR 0.196 (0.055-0.636)
ChAdOx1 matched conditional 73.3 (-6.1 to 93.2) OR 0.267 (0.067-1.061)
ChAdOx1 within 14 days -12.6 (-136.9 to 46.5) OR 1.126 (0.535-2.369)
Counts (Abstract and Results): BNT162b2 vaccinated 18 of 135 cases, 90 of 269 controls;
ChAdOx1 vaccinated 9 of 36 cases, 53 of 90 controls.
Method: VE = 100 x (1 - OR) throughout the paper. An unadjusted OR and its CI can be
recomputed from the 2x2 counts (Wald on the log scale, and the conditional MLE with its
exact interval). An adjusted OR cannot be recomputed from counts, but its VE interval follows
from the reported OR interval, and a 95% Wald interval is symmetric about ln(OR).
"""
import math
import os
import numpy as np
from scipy import stats
from scipy.stats.contingency import odds_ratio
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.isf(0.025)
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"axes.edgecolor": INK2, "text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2})
def ve(o):
return 100 * (1 - o)
def to_or(v):
return 1 - v / 100
# ----------------------------------------------------------------------------- 1. counts
print("=== 1. Unadjusted OR recomputed from the 2x2 counts ===")
COUNTS = {"BNT162b2": (18, 135, 90, 269, -20.2), "ChAdOx1": (9, 36, 53, 90, -33.9)}
WALD = {}
for lab, (a, n1, c, n0, diff_rep) in COUNTS.items():
b, d = n1 - a, n0 - c
orr = a * d / (b * c)
se = math.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
lo, hi = math.exp(math.log(orr) - Z * se), math.exp(math.log(orr) + Z * se)
WALD[lab] = (orr, lo, hi)
r = odds_ratio([[a, b], [c, d]], kind="conditional")
ci = r.confidence_interval()
print(f"{lab}: cases {a}/{n1} = {100*a/n1:.1f}%, controls {c}/{n0} = {100*c/n0:.1f}%, "
f"difference {100*(a/n1-c/n0):.1f} points (reported {diff_rep})")
print(f" Wald OR {orr:.3f} ({lo:.3f}-{hi:.3f}) -> VE {ve(orr):.1f} ({ve(hi):.1f} to {ve(lo):.1f})")
print(f" conditional MLE OR {r.statistic:.3f} ({ci.low:.3f}-{ci.high:.3f}) -> "
f"VE {ve(r.statistic):.1f} ({ve(ci.high):.1f} to {ve(ci.low):.1f})")
# ---------------------------------------------------------------- 2. Table 2 OR -> VE
print("\n=== 2. Table 2: VE recomputed from each reported OR (VE = 1 - OR) ===")
TABLE = { # label: (OR, lo, hi, VE as printed) -- all eight one-dose effect rows of Table 2
"BNT162b2 unadjusted": (0.306, 0.171, 0.523, "69.4 (47.7 to 82.9)"),
"BNT162b2 logistic, one dose": (0.286, 0.138, 0.569, "71.4 (43.1 to 86.2)"),
"BNT162b2 matched conditional": (0.426, 0.181, 1.002, "57.4 (-0.20 to 81.9)"),
"BNT162b2 within 14 days": (0.935, 0.518, 1.658, "6.5 (-65.8 to 48.2)"),
"ChAdOx1 unadjusted": (0.233, 0.094, 0.535, "76.7 (46.5 to 90.6)"),
"ChAdOx1 logistic, one dose": (0.196, 0.055, 0.636, "80.4 (36.4 to 94.5)"),
"ChAdOx1 matched conditional": (0.267, 0.067, 1.061, "73.3 (-6.1 to 93.2)"),
"ChAdOx1 within 14 days": (1.126, 0.535, 2.369, "-12.6 (-136.9 to 46.5)"),
}
def offcentre(o, lo, hi):
"""Distance of ln(OR) from the log-midpoint of its CI, as a fraction of the half-width."""
mid = 0.5 * (math.log(lo) + math.log(hi))
return (math.log(o) - mid) / (0.5 * (math.log(hi) - math.log(lo)))
def printed_nums(vtxt):
v, rest = vtxt.split(" (")
l, h = rest.rstrip(")").split(" to ")
return float(v), float(l), float(h)
offs = []
for lab, (o, lo, hi, vtxt) in TABLE.items():
pv, pl, ph = printed_nums(vtxt)
gap = max(abs(ve(o) - pv), abs(ve(hi) - pl), abs(ve(lo) - ph))
offs.append(offcentre(o, lo, hi))
print(f"{lab}: OR {o} ({lo}-{hi}) -> VE {ve(o):.1f} ({ve(hi):.1f} to {ve(lo):.1f}); "
f"printed {vtxt}; largest gap {gap:.1f} points; log-midpoint OR {math.sqrt(lo*hi):.3f}; "
f"off-centre {offcentre(o, lo, hi):+.2f} half-widths")
print(f"{len(TABLE)} Table 2 effect rows; off-centre range {min(offs):+.2f} to {max(offs):+.2f} half-widths")
# ------------------------------------------------------------- 3. the abstract intervals
print("\n=== 3. Abstract intervals converted back to the OR scale ===")
ABSTRACT = {"BNT162b2 adjusted": (71.4, 46.5, 90.6),
"ChAdOx1 adjusted": (80.4, 36.4, 94.5),
"BNT162b2 restricted period": (79.3, 47.0, 92.5)}
for lab, (v, l, h) in ABSTRACT.items():
o, olo, ohi = to_or(v), to_or(h), to_or(l)
print(f"{lab}: VE {v} ({l}-{h}) -> OR {o:.3f} ({olo:.3f}-{ohi:.3f}); log-midpoint OR "
f"{math.sqrt(olo*ohi):.3f} (VE {ve(math.sqrt(olo*ohi)):.1f}); "
f"off-centre {offcentre(o, olo, ohi):+.2f} half-widths")
a_lo, a_hi = to_or(90.6), to_or(46.5)
c_o, c_lo, c_hi, _ = TABLE["ChAdOx1 unadjusted"]
print(f"Abstract BNT162b2 OR interval {a_lo:.3f}-{a_hi:.3f} vs Table 2 ChAdOx1 unadjusted "
f"{c_lo}-{c_hi}: identical = {round(a_lo,3)==c_lo and round(a_hi,3)==c_hi}")
matches = [lab for lab, (o, lo, hi, _) in TABLE.items()
if abs(ve(hi) - 46.5) < 0.05 and abs(ve(lo) - 90.6) < 0.05]
print("Table 2 rows whose VE interval is 46.5 to 90.6:", matches)
# ---------------------------------------------------------- 4. consequence for precision
print("\n=== 4. Standard error implied by each interval (95% Wald, log OR) ===")
se_tab = (math.log(0.569) - math.log(0.138)) / (2 * Z)
se_abs = (math.log(a_hi) - math.log(a_lo)) / (2 * Z)
print(f"SE(ln OR) from Table 2 interval 0.138-0.569: {se_tab:.3f}")
print(f"SE(ln OR) from abstract interval {a_lo:.3f}-{a_hi:.3f}: {se_abs:.3f} "
f"({100*(se_abs/se_tab-1):.0f}% larger)")
print(f"Inverse-variance weight from abstract interval / correct weight: {(se_tab/se_abs)**2:.2f}")
print(f"Upper VE bound 90.6 vs 86.2: +{90.6-86.2:.1f} points; lower 46.5 vs 43.1: +{46.5-43.1:.1f} points")
print(f"Distance from point estimate 71.4: abstract +{90.6-71.4:.1f} above / -{71.4-46.5:.1f} below; "
f"Table 2 +{86.2-71.4:.1f} above / -{71.4-43.1:.1f} below")
print(f"Total VE width: abstract {90.6-46.5:.1f} points vs Table 2 {86.2-43.1:.1f} points")
z = math.log(0.286) / se_tab
print(f"Wald z from Table 2 OR and CI: {z:.2f}, two-sided p = {2*stats.norm.sf(abs(z)):.1e} (Table 2: <0.0001)")
# ===================================================================== figure 1: intervals
def style_axes(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ROWS = [ # (label, OR, lo, hi, colour, VE text, kind)
("Abstract\nBNT162b2, adjusted", 0.286, a_lo, a_hi, ORANGE, "71.4 (46.5–90.6)", "rep"),
("Table 2 / Results\nBNT162b2, adjusted (logistic)", 0.286, 0.138, 0.569, ORANGE, "71.4 (43.1–86.2)", "rep"),
("Table 2\nBNT162b2, unadjusted", 0.306, 0.171, 0.523, ORANGE, "69.4 (47.7–82.9)", "rep"),
("From counts 18/135 vs 90/269\nBNT162b2, unadjusted (Wald)", *WALD["BNT162b2"], BLUE,
"{:.1f} ({:.1f}–{:.1f})".format(ve(WALD["BNT162b2"][0]), ve(WALD["BNT162b2"][2]), ve(WALD["BNT162b2"][1])), "rec"),
("Table 2 / Results\nChAdOx1, unadjusted", 0.233, 0.094, 0.535, ORANGE, "76.7 (46.5–90.6)", "rep"),
("From counts 9/36 vs 53/90\nChAdOx1, unadjusted (Wald)", *WALD["ChAdOx1"], BLUE,
"{:.1f} ({:.1f}–{:.1f})".format(ve(WALD["ChAdOx1"][0]), ve(WALD["ChAdOx1"][2]), ve(WALD["ChAdOx1"][1])), "rec"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.9), facecolor=BG)
style_axes(ax)
n = len(ROWS)
ys = np.arange(n)[::-1].astype(float)
for (lab, o, lo, hi, col, vtxt, kind), y in zip(ROWS, ys):
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([o], [y], marker="o" if kind == "rep" else "s", ms=9, color=col, zorder=4,
markeredgecolor=BG, markeredgewidth=1)
ax.text(1.55, y, vtxt, va="center", ha="left", fontsize=10, color=col)
# geometric centre of the abstract interval
gm = math.sqrt(a_lo * a_hi)
ax.plot([gm], [ys[0]], marker="D", ms=8, mfc="none", mec=INK, mew=2, zorder=5)
ax.annotate(f"centre: OR {gm:.3f}",
xy=(gm, ys[0]), xytext=(0.102, ys[0] - 0.5), fontsize=10, color=INK,
ha="left", va="center",
arrowprops=dict(arrowstyle="-", color=INK, lw=1, shrinkA=0, shrinkB=5))
# identical endpoints
for xv in (0.094, 0.535):
ax.plot([xv, xv], [ys[4], ys[0]], color=INK2, lw=0.9, ls=(0, (3, 3)), zorder=2)
ax.text(math.sqrt(0.094 * 0.535), ys[4] + 0.5, "identical endpoints",
fontsize=10, color=INK2, ha="center", va="center")
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(1.06, ys[2] - 0.5, "no effect (OR 1)", fontsize=10, color=INK2, ha="left", va="center",
rotation=90)
ax.set_xscale("log")
ax.set_xlim(0.05, 1.5)
ax.set_ylim(-0.6, n - 0.2)
ticks = [0.05, 0.1, 0.2, 0.3, 0.5, 1.0]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in ROWS], fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Odds ratio, one dose vs none (log scale; VE = 1 − OR)", color=INK)
ax.text(1.55, n - 0.2, "VE % (95% CI)", fontsize=10, color=INK2, ha="left", va="top", fontweight="bold")
leg = [Line2D([], [], color=ORANGE, lw=2, marker="o", ms=8, label="reported in the paper"),
Line2D([], [], color=BLUE, lw=2, marker="s", ms=8, label="recomputed from the paper's counts"),
Line2D([], [], ls="none", marker="D", ms=8, mfc="none", mec=INK, mew=2,
label="log-midpoint of the abstract interval")]
fig.legend(handles=leg, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10,
columnspacing=1.2, handletextpad=0.5)
fig.subplots_adjust(left=0.33, right=0.80, top=0.95, bottom=0.24)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# =================================================================== figure 2: text panels
fig = plt.figure(figsize=(7.2, 3.9), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 3.9)
ax.axis("off")
def panel(x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=edge, lw=2))
ax.text(x + 0.15, y + h - 0.17, title, fontsize=10, fontweight="bold", va="top", color=INK)
yy = y + h - 0.55
for txt, col, bold in lines:
ax.text(x + 0.15, yy, txt, fontsize=10, va="top", color=col,
fontweight="bold" if bold else "normal")
yy -= 0.29
panel(0.1, 2.0, 3.05, 1.8, "Abstract (Findings), BNT162b2",
[("“The adjusted vaccine effectiveness", ORANGE, False),
("was 71·4% (95% CI 46·5–90·6).”", ORANGE, True),
("Interval as OR: 0.094–0.535", BLUE, False)], ORANGE)
panel(4.05, 2.0, 3.05, 1.8, "Results ¶3 and Table 2, BNT162b2",
[("adjusted (logistic), one dose:", INK, False),
("71·4 (43·1 to 86·2)", ORANGE, True),
("OR 0·286 (0·138–0·569)", ORANGE, False),
("1 − OR interval: 43.1–86.2", BLUE, False)], INK2)
panel(4.05, 0.1, 3.05, 1.5, "Results ¶5 and Table 2, ChAdOx1",
[("unadjusted:", INK, False),
("76·7 (46·5 to 90·6)", ORANGE, True),
("OR 0·233 (0·094–0·535)", ORANGE, False)], INK2)
ax.add_patch(FancyArrowPatch((3.17, 3.2), (4.03, 3.2), arrowstyle="<->", mutation_scale=12,
color=INK, lw=1.5))
ax.text(3.6, 3.1, "same\nestimate,\nother CI", fontsize=10, ha="center", va="top", color=INK)
ax.add_patch(FancyArrowPatch((1.7, 2.0), (4.03, 0.85), connectionstyle="arc3,rad=0.25",
arrowstyle="<->", mutation_scale=12, color=INK, lw=1.5))
ax.text(1.45, 1.05, "identical interval,\nother vaccine and\nunadjusted model", fontsize=10,
ha="center", va="center", color=INK)
ax.text(0.15, 0.2, "orange: printed in the paper\nblue: converted by recompute.py.txt",
fontsize=10, color=INK2, va="bottom")
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
plt.close(fig)
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png")
Vaccine "prevented fraction" of critical care admissions reported as 144%; the paper's counts give 59%
Lancet Reg Health Am (2025) · doi:10.1016/j.lana.2025.101143 · PMC12173695
Summary
The abstract and the Results report that vaccination prevented "144%" of SARS-CoV-2 critical care admissions in Ontario, citing 22,017 admissions expected without vaccination against 9020 observed. Those two counts imply 12,997 averted admissions. As a share of the 22,017 expected admissions, which is the prevented fraction the Methods say was estimated, that is 59.0%. The 144% is 12,997 ÷ 9020, the excess of the counterfactual over the observed count. A prevented fraction cannot exceed 100%, so the headline figure for the paper's secondary (counterfactual) objective is impossible as worded and overstates the share of admissions prevented. The underlying counts, the forecasting results and the averted-admission count used in the economic analysis are not affected.
What the paper reports
All quotations from the main text were checked verbatim against the PMC-archived article with src/show.py.
Abstract, Summary, Findings:
Averted admissions attributable to vaccination were estimated at 144% (22,017 expected vs. 9020 observed).
Results, last paragraph (p5 in the PMC numbering):
Counterfactual modeling estimated that vaccination prevented 144% of critical care admissions during the study period (22,017 admissions expected vs. 9020 observed). Details of this analysis, along with an economic evaluation of averted admissions, are provided in the Supplementary Appendix.
Methods, "Counterfactual analysis":
We used our models to estimate the prevented fraction of SARS-CoV-2 critical care attributable to vaccination by generating model predictions with the coefficient for vaccination set to zero, as described by Zhou.19 Prevented fraction was obtained by comparing model predicted critical care admissions with and without vaccination.
Discussion, fifth paragraph:
we find that if vaccination was indeed driving the decline in per-case critical care risk, Ontario's universal SARS-CoV-2 vaccination program effectively paid for itself, simply through prevention of critical care-related resource use.
Supplementary Appendix (mmc1.pdf), "Counterfactual Estimates of Vaccine Impact", p. 9 (text extracted from the PDF):
Over the period from March 2020 to August 2022, we estimated that vaccination prevented approximately 12,977 admissions, representing a prevented fraction of 144% (22,017 expected vs. 9,020 observed).
Supplementary Figure 11 caption, p. 22: "vaccination was estimated to have prevented 12,977 critical care unit admissions, representing a prevented fraction of 144%." Supplementary Table 5, p. 30: "Critical care admissions averted 12997". The Appendix's length-of-stay section (p. 7) states that "A total of 9,020 individuals met our criteria for critical care receipt", which identifies 9,020 as the observed total. The French, Portuguese and Spanish abstracts (mmc2 to mmc4) give the same 144% with 22 017 and 9 020.
What the paper's own numbers show
Method. I took the two counts printed beside the percentage (22,017 expected without vaccination, 9020 observed) and computed the averted admissions, the prevented fraction (averted ÷ expected) and the other ratios that can be formed from the two counts. I then checked which of them reproduces 144%, and cross-checked the averted count against the Supplementary Appendix. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Expected admissions without vaccination, E | Abstract; Results p5 | 22,017 | input |
| Observed admissions, O | Abstract; Results p5; Appendix p. 7 | 9020 | input |
| Averted admissions, E − O | Appendix Table 5; Appendix p. 9 and Suppl. Fig. 11 | 12997 (Table 5); 12,977 (text, caption) | 12,997 |
| Share of admissions prevented (prevented fraction), (E − O) ÷ E | Abstract; Results p5; Appendix | 144% | 59.0% (12,997 ÷ 22,017 = 0.5903) |
| Relative excess of counterfactual over observed, (E − O) ÷ O | not stated as such | — | 144.1% (12,997 ÷ 9020 = 1.4409), the only ratio that gives 144% |
| Ratio E ÷ O | not stated | — | 2.44 |
| Observed count implied by a 144% prevented fraction, E × (1 − 1.44) | — | — | −9,687 (impossible) |
Only one of the ratios that can be formed from the two counts gives 144%: averted admissions divided by observed admissions. That is the relative excess: without vaccination, admissions would have been 144% higher than observed, or 2.44 times as many. The Methods define the quantity as a prevented fraction, and the Abstract and Results describe it as a share of admissions averted or prevented. That share has the counterfactual total as its denominator and is 59.0%. For a prevented fraction of 144%, observed admissions would have to be negative (−9,687).
Figure
Figure 1. Share of critical care admissions prevented by vaccination. Orange: the value reported in the Abstract, Results and Supplementary Appendix (144%). Blue: the prevented fraction recomputed from the counts printed beside it, (22,017 − 9,020) ÷ 22,017 = 59.0%. The shaded region beyond 100% cannot be reached by a prevented fraction. The axis starts at zero.
Figure 2. The two counts reported by the paper, on a linear scale from zero. The hatched segment is the 12,997 admissions averted (expected minus observed). Dividing it by the expected total gives the prevented fraction, 59.0% (blue). Dividing it by the observed total gives 144.1% (orange), the figure the paper reports as the share prevented.
Why it matters
- A headline number is impossible as worded. The 144% appears in the abstract Findings and in the Results, and is repeated in the Supplementary Appendix and the three translated abstracts. A reader who takes it at face value reads a share of admissions prevented that is about 2.4 times the correct 59%. A journal-news summary published in June 2025 already reported the result as "an estimated 144% reduction attributable to vaccination".
- The finding concerns a secondary objective. The counterfactual estimate is not part of the forecasting validation that is the paper's main aim. The model calibration and predictive validity (ρ = 0.85, 0.46, 0.60 and 0.68), the likelihood-ratio test for the vaccination term (p = 0.013), the point scores, the occupancy projections and the Interpretation are unaffected.
- The direction and the counts survive. The counts 22,017 and 9020 are printed next to the percentage, so the correct value can be recovered. Supplementary Table 5 uses 12,997 averted admissions, which equals 22,017 − 9,020. The economic estimates (about $2.9 billion CAD averted in the base case) and the Discussion's statement that the programme "effectively paid for itself" rest on that count, not on the percentage. Corrected, the result is still large: vaccination prevented about three in five of the admissions expected without it.
Explanations considered
- A different definition of the percentage. Not supported. The only definition in the paper is the Methods' "prevented fraction", obtained "by comparing model predicted critical care admissions with and without vaccination" and citing Zhou. A prevented fraction is (expected − observed) ÷ expected and is bounded by 100%. No footnote, legend or supplement defines the 144% differently; the Appendix also calls it "a prevented fraction of 144%".
- A correctly computed relative excess, described in other words. This explains the number but not the wording. (22,017 − 9,020) ÷ 9,020 = 144.1%, so "admissions would have been 144% higher without vaccination" would be correct. The Abstract instead calls it a percentage of "averted admissions", and the Results say vaccination "prevented 144% of critical care admissions". Both describe a share of admissions, which cannot exceed 100%.
- Model-predicted rather than observed admissions as the base. Not able to explain it. The Methods compare model predictions with and without vaccination, while the text quotes the observed total (9020). Whichever with-vaccination total is used, the share of counterfactual admissions prevented lies between 0% and 100%. A model-predicted total close to the observed 9020 gives a value near 59%.
- Different populations, periods or units for the two counts. Not supported. Both are counts of critical care admissions in Ontario over the same study period, and they are given together in the same parenthesis. The Appendix dates the counterfactual to March 2020 to August 2022, while the main text gives the study period as March 2020 to September 2022. That small difference cannot turn a share into 144%.
- A conversion or typesetting artifact. Ruled out. The same 144% appears in the PMC and Europe PMC text (the publisher's JATS deposit), in the typeset Appendix PDF, and in the three translated abstracts, each time next to 22,017 and 9,020.
- A correction already issued. None found (see Status).
- Side observations, not part of the main finding. (1) The Appendix text and the Supplementary Figure 11 caption give 12,977 averted admissions, while Supplementary Table 5 and 22,017 − 9,020 give 12,997, a difference of 20. (2) Supplementary Table 5 lists a critical care case fatality of 35.4% and 4549 deaths averted. 12,997 × 0.354 = 4,601, whereas 4549 equals 12,997 × 0.350. Neither difference affects the main finding.
Status
- Version of record. The publisher's pages (doi.org redirect to linkinghub.elsevier.com, thelancet.com and sciencedirect.com) returned only a redirect stub or HTTP 403 to automated requests. I checked the publisher-supplied full text in two archived copies: the PMC article page and the Europe PMC full-text XML, which is the publisher's JATS deposit. Both carry the Abstract and Results sentences quoted above. Every main-text quotation was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. I downloaded the Europe PMC supplementary bundle for PMC12173695: mmc1.pdf (Supplementary Appendix, 35 pages), mmc2.docx, mmc3.docx and mmc4.docx (French, Portuguese and Spanish abstracts), and figure files gr1 to gr4. The Appendix repeats "a prevented fraction of 144% (22,017 expected vs. 9,020 observed)" and gives no other definition, so it confirms rather than resolves the discrepancy. The translated abstracts give the same 144%.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices, and the Crossref record has empty update-to and updated-by fields. The PubPeer API returned no record (HTTP 404) for the DOI, the PubPeer search page refused automated access (HTTP 403), and a web search for the title together with "PubPeer", "correction" or "erratum" found no thread or notice. The medRxiv preprint (doi:10.1101/2025.01.21.25320912) could not be opened (HTTP 403) and was not checked. - Sources. PMC article PMC12173695; Europe PMC full-text XML and supplementary files for PMC12173695; Crossref; Europe PMC; PubPeer; EMJ news item "Updated Score Improves Prediction of COVID-19 Critical Care Demand" (emjreviews.com, June 2025); https://doi.org/10.1016/j.lana.2025.101143.
Suggested correction
In the Abstract (Findings), replace
Averted admissions attributable to vaccination were estimated at 144% (22,017 expected vs. 9020 observed).
with
Vaccination was estimated to have averted 12,997 critical care admissions, a prevented fraction of 59% (22,017 expected without vaccination vs. 9020 observed).
In the Results (last paragraph), replace "vaccination prevented 144% of critical care admissions" with "vaccination prevented 59% of the critical care admissions expected without vaccination (12,997 averted)". In the Supplementary Appendix (p. 9 and the Supplementary Figure 11 caption), change "a prevented fraction of 144%" to "a prevented fraction of 59%", and make the translated abstracts match. If the authors prefer to keep the 144% figure, it can be stated as a relative excess: "without vaccination, critical care admissions would have been an estimated 144% higher (2.4 times as many)". The authors may also wish to reconcile the averted count in the Appendix text and the Figure 11 caption (12,977) with Supplementary Table 5 (12,997), and the 35.4% case fatality with the 4549 deaths averted shown in the same table.
Recomputation script (python recompute.py.txt)
"""Recompute the counterfactual vaccine-impact percentage of PMC12173695
(Lancet Reg Health Am 2025, doi:10.1016/j.lana.2025.101143,
"Derivation and validation of a point-based forecasting tool for SARS-CoV-2 critical
care occupancy: a population-based modeling study").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC and from its Supplementary Appendix:
Abstract (Findings): "Averted admissions attributable to vaccination were estimated at
144% (22,017 expected vs. 9020 observed)."
Results, paragraph 5: "vaccination prevented 144% of critical care admissions during the
study period (22,017 admissions expected vs. 9020 observed)."
Supplementary Appendix: "approximately 12,977 admissions" (counterfactual section and
Supplementary Figure 11 caption); Supplementary Table 5: "Critical care admissions
averted 12997", "Critical care case fatality 35.4%", "Critical care deaths averted 4549".
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
EXPECTED = 22017 # counterfactual admissions, vaccine coefficient set to zero
OBSERVED = 9020 # observed admissions (Appendix: 9,020 individuals met criteria)
REPORTED_PCT = 144 # "144%" in Abstract, Results, Appendix, translated abstracts
APPX_AVERTED_TEXT = 12977 # Appendix text and Suppl. Fig. 11 caption
APPX_AVERTED_TABLE = 12997 # Supplementary Table 5
APPX_CFR = 0.354 # Supplementary Table 5
APPX_DEATHS_AVERTED = 4549 # Supplementary Table 5
# ---------------------------------------------------------------- 1. the arithmetic
averted = EXPECTED - OBSERVED
pf = averted / EXPECTED # prevented fraction (bounded by 1)
rel_excess = averted / OBSERVED # relative excess of counterfactual over observed
ratio = EXPECTED / OBSERVED
print("=== 1. Counterfactual arithmetic from the paper's own counts ===")
print(f"expected (no vaccination) : {EXPECTED:,}")
print(f"observed : {OBSERVED:,}")
print(f"averted = expected - observed : {averted:,}")
print(f"averted / expected (prevented fraction): {pf:.4f} = {pf:.1%}")
print(f"averted / observed (relative excess) : {rel_excess:.4f} = {rel_excess:.1%}")
print(f"expected / observed : {ratio:.4f}")
print(f"observed / expected : {OBSERVED/EXPECTED:.4f} = {OBSERVED/EXPECTED:.1%}")
# ---------------------------------------------------------------- 2. which formula gives 144%
print("\n=== 2. Which formula reproduces the reported 144% ===")
cands = {
"(E - O) / E [prevented fraction]": pf,
"(E - O) / O [relative excess]": rel_excess,
"E / O": ratio,
"O / E": OBSERVED / EXPECTED,
}
for k, v in cands.items():
flag = " <-- matches 144%" if round(v * 100) == REPORTED_PCT else ""
print(f"{k:36s}: {v*100:7.1f}%{flag}")
# ---------------------------------------------------------------- 3. impossibility
print("\n=== 3. What a 144% prevented fraction would imply ===")
pf_rep = REPORTED_PCT / 100
implied_obs = EXPECTED * (1 - pf_rep)
implied_averted = EXPECTED * pf_rep
print(f"observed = expected x (1 - 1.44) : {implied_obs:,.2f} (negative: impossible)")
print(f"averted = expected x 1.44 : {implied_averted:,.0f} (> expected {EXPECTED:,})")
# ---------------------------------------------------------------- 4. supplement cross-checks
print("\n=== 4. Supplementary Appendix cross-checks ===")
print(f"Table 5 averted {APPX_AVERTED_TABLE:,} == E - O ? {APPX_AVERTED_TABLE == averted}")
print(f"Appendix text averted {APPX_AVERTED_TEXT:,} == E - O ? {APPX_AVERTED_TEXT == averted} "
f"(difference {averted - APPX_AVERTED_TEXT})")
print(f"12,977 / 22,017 = {APPX_AVERTED_TEXT/EXPECTED:.1%}; 12,977 / 9,020 = {APPX_AVERTED_TEXT/OBSERVED:.1%}")
print(f"deaths averted at CFR 35.4%: {APPX_AVERTED_TABLE*APPX_CFR:,.1f}; at 35.0%: "
f"{APPX_AVERTED_TABLE*0.35:,.1f}; printed {APPX_DEATHS_AVERTED:,} = "
f"{APPX_DEATHS_AVERTED/APPX_AVERTED_TABLE:.4f} of averted admissions")
# ---------------------------------------------------------------- figures
BG, FG, FG2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 11, "axes.edgecolor": FG2, "axes.labelcolor": FG, "xtick.color": FG2,
"ytick.color": FG2, "text.color": FG, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "font.family": "DejaVu Sans",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# Figure 1: the percentage on one axis with the 100% ceiling of a prevented fraction
fig, ax = plt.subplots(figsize=(7.2, 2.9))
ax.axvspan(100, 160, color="#efeeea", zorder=0)
ax.axvline(100, color=FG2, lw=1, zorder=1)
ax.text(101.5, 1.72, "a prevented fraction cannot exceed 100%", color=FG2, fontsize=10,
va="center", ha="left")
rows = [
(1.0, REPORTED_PCT, ORANGE, f"Reported as share prevented: {REPORTED_PCT}%",
"Abstract, Results, Appendix"),
(0.0, pf * 100, BLUE, f"Recomputed prevented fraction: {pf*100:.1f}%",
f"(22,017 − 9,020) / 22,017 = {averted:,} / {EXPECTED:,}"),
]
for y, x, c, lab, sub in rows:
ax.plot([0, x], [y, y], color=c, lw=2, zorder=2)
ax.plot(x, y, "o", ms=10, color=c, zorder=3)
tx = x - 2 if x > 100 else x + 3
ha = "right" if x > 100 else "left"
ax.text(tx, y + 0.17, lab, color=c, fontsize=11, ha=ha, va="bottom", fontweight="bold")
ax.text(tx, y - 0.17, sub, color=FG2, fontsize=10, ha=ha, va="top")
ax.set_xlim(0, 160)
ax.set_ylim(-0.6, 1.85)
ax.set_yticks([])
ax.set_xticks(range(0, 161, 20))
ax.set_xticklabels([f"{t}%" for t in range(0, 161, 20)])
ax.set_xlabel("Share of critical care admissions prevented by vaccination")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: the counts and the two possible denominators
fig, ax = plt.subplots(figsize=(7.2, 3.1))
H = 0.5
ax.barh(1, EXPECTED, height=H, color="#d9d8d3", edgecolor=FG2, lw=1)
ax.barh(0, OBSERVED, height=H, color="#d9d8d3", edgecolor=FG2, lw=1)
ax.barh(1, averted, left=OBSERVED, height=H, color="none", edgecolor=FG, lw=2, hatch="///")
ax.text(EXPECTED + 300, 1, f"{EXPECTED:,}", va="center", fontsize=11)
ax.text(OBSERVED + 300, 0, f"{OBSERVED:,}", va="center", fontsize=11)
ax.text(OBSERVED + averted / 2, 1 + H / 2 + 0.06, f"averted = {averted:,}", ha="center",
va="bottom", fontsize=11)
ax.text(OBSERVED / 2, 1, f"{OBSERVED:,}\n(= observed)", ha="center", va="center", fontsize=10, color=FG2)
ax.set_yticks([1, 0])
ax.set_yticklabels(["Expected without\nvaccination", "Observed"])
ax.set_xlim(0, 40000)
ax.set_xticks(range(0, 40001, 10000))
ax.set_ylim(-0.75, 1.75)
ax.set_xlabel("Critical care admissions over the study period (count)")
ax.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(lambda v, _: f"{int(v):,}"))
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.text(23200, 0.30, f"averted ÷ expected = {pf:.1%}\nprevented fraction (Methods)",
color=BLUE, fontsize=10, va="center", fontweight="bold")
ax.text(23200, -0.35, f"averted ÷ observed = {rel_excess:.1%}\nreported as \"prevented 144%\"",
color=ORANGE, fontsize=10, va="center", fontweight="bold")
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract reports lower Ct values in vaccinated participants; the paper's data and text show higher
Lancet Reg Health Am (2025) · doi:10.1016/j.lana.2025.101309 · PMC12703870
Summary
The Findings section of the Abstract states that vaccinated participants had "significantly lower RT-qPCR Ct values" than placebo recipients. The Results report the opposite: mean Ct was higher in the vaccinated group for both serotypes, by 5.0 cycles for DENV-1 (27.2 vs 22.2) and 2.9 cycles for DENV-2 (28.2 vs 25.3), and the Research in context panel, the Discussion, Fig. 2a/2i and Supplementary Table 6 all agree with the Results. Because the paper defines Ct as inversely related to the amount of viral nucleic acid, the Abstract's wording implies more viral RNA in vaccinated participants, which contradicts both the data and the conclusion drawn in the same sentence ("the vaccine could be reducing viral replication"). The error is confined to one word in the Abstract, but it reverses the direction of the paper's main virological finding in the part of the paper most often read on its own.
What the paper reports
Abstract, Findings:
Despite the limited statistical power due to the low number of samples, we observed significantly lower RT-qPCR Ct values in the vaccinated group, particularly for the DENV-1 cases, suggesting that the vaccine could be reducing viral replication.
Research in context, "Added value of this study":
Additionally, we observed higher RT-qPCR Ct values in the vaccinated group compared to the placebo group, indicating a lower viral load among vaccinated participants and suggesting that the Butantan-DV is initially effective in reducing viral loads.
Results, "Characterization of RT-qPCR cycle threshold values":
RT-qPCR cycle threshold (Ct) values indirectly measure DENV genomic load in samples, with an inverse relationship to the input target nucleic acid. Our analysis included a total of 365 DENV-positive samples, with an overall mean Ct of 23.7 (SD: 4.85) for DENV-1 (n = 161) and 26.5 (SD: 4.67) for DENV-2 (n = 205). When comparing RT-qPCR Ct values between the Pb and Vx groups for each serotype, a statistically significant difference was observed (p < 0.001). The Vx group exhibited a higher mean Ct value than the Pb group. Specifically, the mean Ct for DENV-2 was 28.2 (SD: 4.18) in the Vx group compared to 25.3 (SD: 4.65) in the Pb group. This trend was even stronger for DENV-1, with the mean Ct being 27.2 (SD: 3.98) in the Vx group compared to 22.2 (SD: 4.40) in the Pb group (Fig. 2a and i) (Supplementary Table S6).
Discussion, fourth paragraph:
The higher Ct values observed in the vaccinated group, therefore, suggest that Butantan-DV-induced immunity may confer protection not only by preventing infection, but also by mitigating disease severity through the suppression of viral replication.
Results, "Sample characterization":
There were 234 (64.1%) participants included in the Pb group and 131 (35.8%) participants in the Vx group confirmed for DENV.
Supplementary Table 6 ("Comparative analysis of cycle threshold (Ct) values for DENV-1 and DENV-2 between vaccinated and placebo groups"; columns: N positive Placebo | Butantan-DV | mean Ct Placebo | Butantan-DV | Δ mean | t-statistic | p-value | Cohen's d):
DENV-1 | 113 | 48 | 22.2 | 27.2 | 5.00 | 10.87 | < 0.001 | 1.16
DENV-2 | 122 | 83 | 25.3 | 28.2 | 2.90 | 6.21 | < 0.001 | 0.86
Its footnote reads:
The p-value and t-statistic were calculated using the Welch's t-test for unequal variances.
What the paper's own numbers show
Method. I took the group means and SDs from the Results text and the group sizes from Supplementary Table 6. I computed the vaccinated-minus-placebo difference in mean Ct for each serotype and checked whether rounding of the printed means (to 0.1) could change its sign. To rule out swapped group labels in the body, I checked that the n-weighted combination of the two group means reproduces the overall mean the Results report for each serotype, and compared that with the combination obtained if the labels were swapped. I then recomputed a Welch t-test, its 95% CI and Cohen's d from the summary statistics. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Direction, all serotypes | Abstract, Findings | "significantly lower RT-qPCR Ct values in the vaccinated group" | Opposite to every number below |
| DENV-1 mean Ct, Vx vs Pb | Results; Suppl. Table 6 | 27.2 (SD 3.98) vs 22.2 (SD 4.40); Δ 5.00 | Vx − Pb = +5.0 cycles (+4.9 to +5.1 allowing for rounding): vaccinated higher |
| DENV-2 mean Ct, Vx vs Pb | Results; Suppl. Table 6 | 28.2 (SD 4.18) vs 25.3 (SD 4.65); Δ 2.90 | Vx − Pb = +2.9 cycles (+2.8 to +3.0): vaccinated higher |
| DENV-1 overall mean and SD | Results | 23.7 (SD 4.85), n = 161 | With Table S6 n (48 Vx, 113 Pb): 23.69 (SD 4.84). Labels swapped: 25.71 |
| DENV-2 overall mean and SD | Results | 26.5 (SD 4.67), n = 205 | With Table S6 n (83 Vx, 122 Pb): 26.47 (SD 4.68). Labels swapped: 27.03 |
| Vaccinated share implied by overall means | Results | 131 Vx of 365 | 0.300 × 161 + 0.414 × 205 ≈ 133.1 Vx, close to the 131 in Table S6 and the Results |
| DENV-1 difference, 95% CI and test | Results "p < 0.001"; Table S6 | p < 0.001 | Welch 95% CI +3.6 to +6.4; t = 7.06 (df 97.4), p = 2.4 × 10⁻¹⁰ |
| DENV-2 difference, 95% CI and test | Results "p < 0.001"; Table S6 | p < 0.001 | Welch 95% CI +1.7 to +4.1; t = 4.66 (df 187.9), p = 6.0 × 10⁻⁶ |
| Meaning of the direction | Results | Ct has "an inverse relationship to the input target nucleic acid" | Higher Ct in Vx = less viral RNA in Vx, as stated in Research in context and the Discussion |
The body of the paper is internally consistent: the group means, overall means, group sizes, Fig. 2a and 2i (boxplots in which the vaccinated group sits above placebo for both serotypes) and Supplementary Table 6 all describe higher Ct in vaccinated participants. Neither CI comes near zero, so the reversed direction in the Abstract cannot be a matter of rounding or of a borderline result.
Four smaller inconsistencies turned up during the check. None of them affects the direction or the significance of the difference:
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Welch t-statistic | Suppl. Table 6 | 10.87 (DENV-1), 6.21 (DENV-2) | 7.06 and 4.66 from the printed means, SDs and n; p stays below 0.001 |
| Cohen's d, DENV-2 | Suppl. Table 6 | 0.86 | 0.65 with the pooled SD (4.47); DENV-1 reproduces (1.17 vs 1.16) |
| Group counts | Suppl. Table 6 vs Results | 113 + 122 = 235 placebo; 161 + 205 = 366 serotype results | Results give 234 placebo and 365 samples (difference of one) |
| Test used for the Ct comparison | Methods vs Suppl. Table 6 | Methods and Fig. 2 legend: Wilcoxon rank-sum; Table S6: Welch t-test | Both tests are reported as p < 0.001 |
Figure
Figure 1. The two statements of the vaccinated-vs-placebo Ct comparison, quoted from the Abstract (Findings) and from the Results ("Characterization of RT-qPCR cycle threshold values"). Orange outlines mark text and values as reported in the paper; blue text is the difference recomputed from the Results means. The Abstract states that Ct was lower in the vaccinated group; the Results, which describe the same 365 samples, state and show that it was higher, by 5.0 cycles for DENV-1 and 2.9 cycles for DENV-2. The other places in the paper that give a direction (Research in context, Discussion, Fig. 2a/2i, Supplementary Table 6) agree with the Results.
Figure 2. Difference in mean RT-qPCR Ct, vaccinated minus placebo, on one axis of Ct cycles. Orange diamonds are the differences printed in Supplementary Table 6 (5.00 and 2.90). Blue circles and lines are the differences recomputed from the Results means and SDs with the Table S6 group sizes, with Welch 95% confidence intervals (+3.6 to +6.4 and +1.7 to +4.1). The vertical line marks no difference. The shaded half is the direction the Abstract states (vaccinated Ct lower); both intervals lie entirely in the other half.
Why it matters
The Ct comparison is the only viral-load finding in the Abstract; the other findings there describe the sample, lineages and selection. Many readers, indexing services and secondary summaries rely on the Abstract alone. A reader who takes the Abstract's "significantly lower RT-qPCR Ct values in the vaccinated group" at face value would conclude that vaccinated participants with breakthrough infections carried more viral RNA than unvaccinated ones, which is the opposite of what the paper found and the opposite of the protective interpretation the paper draws.
The sentence also contradicts itself. Under the paper's own definition, lower Ct means more viral nucleic acid, so "lower Ct" cannot support "the vaccine could be reducing viral replication". A reader who knows the Ct scale will notice the conflict; a reader who does not may repeat either half. Press coverage of the study (for example Agência FAPESP) reported reduced viral load in vaccinated participants, which matches the body of the paper, so I found no sign that the reversed wording has been repeated, but the Abstract is the text that remains attached to the article in PubMed, Europe PMC and the publisher's site.
The error does not affect the size, significance or interpretation of the result in the body of the paper, nor its main phylogenetic conclusions (no lineage linked to breakthrough infections, no positive selection, similar intra-host mutation rates), which do not depend on Ct.
Explanations considered
- The Abstract refers to a different metric, such as viral load or ΔCt. This does not account for the text. The Abstract names "RT-qPCR Ct values" explicitly, and "lower viral load" is the phrase the paper uses elsewhere for the same finding (Research in context).
- The Abstract summarizes medians rather than means. This does not account for it. The Methods compare medians with Wilcoxon and Kruskal-Wallis tests, and the boxplots in Fig. 2a and 2i show the vaccinated group's box and median above the placebo group's for both serotypes, so medians and means point the same way.
- The Abstract summarizes a different subgroup or time window. This does not account for it. The Abstract and the Results both describe the same 365 DENV-positive samples. The analyses by days after vaccination (Fig. 2e–h and 2m–p) report no correlation and are not what the Abstract summarizes; in Fig. 2e and 2m the vaccinated curve also lies above the placebo curve over most of the range.
- The group labels are swapped in the body, and the Abstract is right. This is excluded by the paper's own numbers. With the Table S6 group sizes, the n-weighted group means reproduce the reported overall means (23.69 vs 23.7; 26.47 vs 26.5), whereas swapped labels would give 25.71 and 27.03. Fig. 2a/2i, Table S6, the Research in context panel and the Discussion all carry the same labelling.
- The Abstract uses an inverted convention for the Ct scale. This does not account for it. The paper states that Ct has "an inverse relationship to the input target nucleic acid", and the Abstract's own conclusion (reduced viral replication) only follows from higher Ct.
- A conversion or typesetting artifact in the archived text. This is excluded. The publisher's typeset PDF (Elsevier, 18 pages) carries the same word "lower" in the Summary on page 1 and "higher" on pages 2, 9 and 15, as do the PMC and Europe PMC full texts.
The remaining explanation consistent with all the evidence is that the word "lower" in the Abstract should read "higher".
Status
- Version of record. I checked the publisher's typeset PDF from the PMC Open Access dataset (PMC12703870.1; PDF metadata: creator Elsevier, "The Lancet Regional Health - Americas, 53 (2026) 101309", created 2025-11-29). It reads the same as the PMC-archived text in every passage quoted above. The publisher's web pages (thelancet.com, sciencedirect.com) refused automated access. Every quote above was confirmed verbatim against the PMC-archived text with
src/show.py, except the Supplementary Table 6 rows, which were read from the supplement file. - Supplements. The only supplement, mmc1.docx (from the Europe PMC supplementary-files bundle), was opened and its Supplementary Table 6 extracted; it is quoted above. I also viewed the publisher's Fig. 2 image (gr2.jpg) to confirm the direction of panels a, e, i and m; the figure is not reproduced here.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, expression of concern or retraction: Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment links (PMID 41403717). The PubPeer API was blocked by a Cloudflare challenge, and a web search found no PubPeer entry for the DOI. A web search for the article found only press coverage.
- Sources. PMC article PMC12703870; PMC Open Access PDF (s3://pmc-oa-opendata/PMC12703870.1/PMC12703870.1.pdf); Europe PMC REST API (full text, supplementary files, metadata); Crossref; PubPeer; Agência FAPESP coverage.
Suggested correction
- In the Abstract (Findings), change "we observed significantly lower RT-qPCR Ct values in the vaccinated group" to "we observed significantly higher RT-qPCR Ct values (indicating lower viral load) in the vaccinated group". No other text, table or figure needs to change for this point.
- Minor, for Supplementary Table 6: check the Welch t-statistics (7.06 and 4.66 follow from the printed means, SDs and n, against 10.87 and 6.21 printed) and the DENV-2 Cohen's d (0.65 vs 0.86); reconcile the placebo count (113 + 122 = 235) with the 234 placebo participants and 365 samples in the Results; and state which test (Wilcoxon rank-sum, per the Methods, or Welch t, per the table) gives the reported p < 0.001.
Recomputation script (python recompute.py.txt)
"""Recompute the vaccinated-vs-placebo RT-qPCR Ct comparison of PMC12703870
(Lancet Reg Health Am, doi:10.1016/j.lana.2025.101309).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in
writeup.md and writes, next to this file:
fig1.svg/png, fig2.svg/png (the figures used in writeup.md)
candidates/cand_A..E.svg/png (five alternative figure designs for review)
Inputs are copied from the paper as archived in PMC and from its only supplement
(mmc1.docx, Supplementary Table 6):
Results, "Characterization of RT-qPCR cycle threshold values": group means/SDs,
overall means/SDs and serotype n
Results, "Sample characterization": 234 Pb and 131 Vx participants (365 in all)
Supplementary Table 6: group n, mean difference, Welch t, Cohen's d
"""
import math
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Patch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ------------------------------------------------------------------ inputs (from the paper)
# Results text: mean (SD)
REP = {
"DENV-1": dict(n=161, overall=(23.7, 4.85), vx=(27.2, 3.98), pb=(22.2, 4.40)),
"DENV-2": dict(n=205, overall=(26.5, 4.67), vx=(28.2, 4.18), pb=(25.3, 4.65)),
}
# Supplementary Table 6: N positive (Pb, Vx), delta mean, t (Welch), Cohen's d
S6 = {
"DENV-1": dict(n_pb=113, n_vx=48, delta=5.00, t=10.87, d=1.16),
"DENV-2": dict(n_pb=122, n_vx=83, delta=2.90, t=6.21, d=0.86),
}
N_PB_TOTAL, N_VX_TOTAL, N_TOTAL = 234, 131, 365 # Results, "Sample characterization"
# ------------------------------------------------------------------ style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
GRID = "#e4e3df"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# ------------------------------------------------------------------ 1. direction and size
print("1. Direction of the group difference (Results text)")
R = {}
for s, r in REP.items():
mv, sv = r["vx"]
mp, sp = r["pb"]
diff = mv - mp
# rounding to one decimal moves each mean by at most 0.05
lo, hi = diff - 0.1, diff + 0.1
R[s] = dict(diff=diff)
print(f" {s}: Vx {mv} - Pb {mp} = {diff:+.1f} Ct cycles "
f"(range allowed by rounding {lo:+.1f} to {hi:+.1f}) -> vaccinated "
f"{'HIGHER' if lo > 0 else 'LOWER/unclear'}")
# ------------------------------------------------------------------ 2. group sizes and consistency
print("\n2. Group sizes: implied by the overall mean vs Supplementary Table 6")
implied_vx_total = 0.0
for s, r in REP.items():
mo, so = r["overall"]
mv, sv = r["vx"]
mp, sp = r["pb"]
w = (mo - mp) / (mv - mp)
implied_vx_total += w * r["n"]
nv, npb = S6[s]["n_vx"], S6[s]["n_pb"]
n = nv + npb
m = (nv * mv + npb * mp) / n
ss = (nv - 1) * sv ** 2 + (npb - 1) * sp ** 2 + nv * (mv - m) ** 2 + npb * (mp - m) ** 2
sd = math.sqrt(ss / (n - 1))
R[s].update(n_vx=nv, n_pb=npb, pooled_mean=m, pooled_sd=sd)
print(f" {s}: overall mean {mo} implies Vx share {w:.3f} -> n_Vx ~ {w * r['n']:.1f}, "
f"n_Pb ~ {r['n'] - w * r['n']:.1f}")
print(f" Table S6 n: Vx {nv}, Pb {npb} (sum {n}; Results n = {r['n']}); "
f"n-weighted mean {m:.2f} (reported {mo}), pooled SD {sd:.2f} (reported {so})")
print(f" if the group labels were swapped: n-weighted mean "
f"{(nv * mp + npb * mv) / n:.2f} (reported {mo})")
vx6 = sum(S6[s]["n_vx"] for s in S6)
pb6 = sum(S6[s]["n_pb"] for s in S6)
print(f" Vx implied by overall means ~ {implied_vx_total:.1f}; Table S6 Vx total {vx6} "
f"(Results {N_VX_TOTAL}); Table S6 Pb total {pb6} (Results {N_PB_TOTAL}); "
f"serotype n {REP['DENV-1']['n']} + {REP['DENV-2']['n']} = "
f"{REP['DENV-1']['n'] + REP['DENV-2']['n']} (Results {N_TOTAL} samples)")
# ------------------------------------------------------------------ 3. tests from summary stats
print("\n3. Welch t-test from the reported means, SDs and Table S6 n (two-sided)")
for s, r in REP.items():
mv, sv = r["vx"]
mp, sp = r["pb"]
nv, npb = R[s]["n_vx"], R[s]["n_pb"]
t, p = stats.ttest_ind_from_stats(mv, sv, nv, mp, sp, npb, equal_var=False)
se = math.sqrt(sv ** 2 / nv + sp ** 2 / npb)
df = se ** 4 / ((sv ** 2 / nv) ** 2 / (nv - 1) + (sp ** 2 / npb) ** 2 / (npb - 1))
q = stats.t.ppf(0.975, df)
ci = (R[s]["diff"] - q * se, R[s]["diff"] + q * se)
sp_pool = math.sqrt(((nv - 1) * sv ** 2 + (npb - 1) * sp ** 2) / (nv + npb - 2))
d = R[s]["diff"] / sp_pool
R[s].update(t=t, p=p, df=df, ci=ci, d=d)
print(f" {s}: diff {R[s]['diff']:+.2f}, SE {se:.3f}, Welch df {df:.1f}, "
f"95% CI {ci[0]:+.2f} to {ci[1]:+.2f}; t = {t:.2f}, p = {p:.1e}")
print(f" Table S6: delta {S6[s]['delta']:.2f}, t {S6[s]['t']:.2f}, d {S6[s]['d']:.2f}; "
f"recomputed Cohen's d (pooled SD {sp_pool:.2f}) = {d:.2f}")
print(f" t needed to reproduce Table S6 t with this difference: SE would be "
f"{R[s]['diff'] / S6[s]['t']:.3f} (vs {se:.3f})")
print("\n4. What the direction means on the paper's own scale")
print(" Results: Ct values have 'an inverse relationship to the input target nucleic acid'.")
print(" Higher Ct in Vx -> less viral RNA in Vx -> consistent with 'reducing viral replication'.")
print(" 'Lower Ct in Vx' (Abstract) would mean more viral RNA in Vx, contradicting the same")
print(" sentence's conclusion, the Results numbers, Research in context and the Discussion.")
# ------------------------------------------------------------------ shared text (verbatim)
Q_ABS = ("we observed significantly lower RT-qPCR Ct values in the vaccinated group, "
"particularly for the DENV-1 cases, suggesting that the vaccine could be "
"reducing viral replication")
Q_ADD = ("we observed higher RT-qPCR Ct values in the vaccinated group compared to the "
"placebo group, indicating a lower viral load among vaccinated participants")
Q_RES = "The Vx group exhibited a higher mean Ct value than the Pb group."
Q_RES1 = ("the mean Ct being 27.2 (SD: 3.98) in the Vx group compared to 22.2 (SD: 4.40) "
"in the Pb group")
Q_RES2 = "the mean Ct for DENV-2 was 28.2 (SD: 4.18) in the Vx group compared to 25.3 (SD: 4.65) in the Pb group"
Q_DIS = "The higher Ct values observed in the vaccinated group"
Q_INV = "with an inverse relationship to the input target nucleic acid"
d1, d2 = R["DENV-1"]["diff"], R["DENV-2"]["diff"]
RECOMP = f"Vx − Pb mean Ct: {d1:+.1f} cycles (DENV-1), {d2:+.1f} cycles (DENV-2)"
def wrap(s, n):
return "\n".join(textwrap.wrap(s, n, break_on_hyphens=False))
def panel(ax, x, y, w, h, edge, lw=1.6, face="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
lw=lw, ec=edge, fc=face, transform=ax.transAxes))
def legend_text_kinds(ax, loc="lower center", y=-0.02):
h = [Patch(fc="white", ec=ORANGE, lw=2, label="Text or values as reported in the paper"),
Patch(fc="white", ec=BLUE, lw=2, label="Recomputed from the paper's own numbers")]
ax.legend(handles=h, loc=loc, bbox_to_anchor=(0.5, y), ncol=2, frameon=False, fontsize=10)
# ------------------------------------------------------------------ design A: two text panels
def design_a(path):
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0, 0, 1, 1])
ax.axis("off")
# left: Abstract
panel(ax, 0.02, 0.25, 0.42, 0.67, ORANGE)
ax.text(0.04, 0.88, "Abstract, Findings", fontsize=11.5, weight="bold", va="top")
ax.text(0.04, 0.80, "States: vaccinated Ct LOWER", fontsize=10.5, color=INK2, va="top")
ax.text(0.04, 0.72, "\u201c\u2026" + wrap(Q_ABS, 38) + "\u201d", fontsize=10, va="top",
linespacing=1.35)
ax.text(0.04, 0.30, "(The Abstract gives no Ct values.)", fontsize=10, color=INK2)
# right: Results
panel(ax, 0.56, 0.25, 0.42, 0.67, ORANGE)
ax.text(0.58, 0.88, "Results, Ct paragraph", fontsize=11.5, weight="bold", va="top")
ax.text(0.58, 0.80, "States: vaccinated Ct HIGHER", fontsize=10.5, color=INK2, va="top")
ax.text(0.58, 0.72, "\u201c" + wrap(Q_RES, 38) + "\u201d", fontsize=10, va="top",
linespacing=1.35)
ax.text(0.58, 0.60, "Mean Ct, Vx vs Pb:\nDENV-1 27.2 vs 22.2\nDENV-2 28.2 vs 25.3\n"
"p < 0.001 for each serotype", fontsize=10, va="top", linespacing=1.45)
ax.text(0.58, 0.395, f"Recomputed Vx \u2212 Pb:\n{d1:+.1f} cycles (DENV-1)\n"
f"{d2:+.1f} cycles (DENV-2)", fontsize=10, color=BLUE, weight="bold", va="top",
linespacing=1.35)
# connector
ax.annotate("", xy=(0.555, 0.61), xytext=(0.445, 0.61),
arrowprops=dict(arrowstyle="<->", color=INK, lw=2))
ax.text(0.50, 0.66, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
weight="bold")
ax.text(0.50, 0.56, "same 365\nsamples", ha="center", va="top", fontsize=10, color=INK2)
ax.text(0.02, 0.205, "Research in context (Added value), Discussion, Fig. 2a/2i and "
"Supplementary Table 6 also give the higher-Ct\ndirection. Results: Ct has "
"\u201can inverse relationship to the input target nucleic acid\u201d, so higher "
"Ct = less viral RNA.", fontsize=10, color=INK2, va="top", linespacing=1.4)
legend_text_kinds(ax, y=-0.01)
save(fig, path)
# ------------------------------------------------------------------ design B: four-location tally
def design_b(path):
rows = [
("Abstract, Findings", "“…significantly lower RT-qPCR Ct values in the vaccinated group…”", "LOWER", ORANGE),
("Research in context,\nAdded value", "“…higher RT-qPCR Ct values in the vaccinated group compared to the placebo group…”", "HIGHER", ORANGE),
("Results, Ct paragraph", "“" + Q_RES + "”", "HIGHER", ORANGE),
("Discussion, para. 4", "“" + Q_DIS + "…”", "HIGHER", ORANGE),
("Paper's means,\nrecomputed", RECOMP, "HIGHER", BLUE),
]
fig = plt.figure(figsize=(W, 4.4))
ax = fig.add_axes([0, 0, 1, 1])
ax.axis("off")
ax.text(0.02, 0.965, "Where the paper states the direction", fontsize=10.5, color=INK2, va="top")
ax.text(0.97, 0.965, "Vaccinated Ct vs placebo", fontsize=10.5, color=INK2, va="top", ha="right")
top, hgt = 0.89, 0.148
for i, (loc, txt, dirn, col) in enumerate(rows):
y = top - i * hgt
is_abs = i == 0
panel(ax, 0.02, y - hgt + 0.02, 0.96, hgt - 0.025, col, lw=2.6 if is_abs else 1.4)
ax.text(0.04, y - hgt / 2 + 0.007, loc, fontsize=10.5, weight="bold", va="center",
linespacing=1.2)
ax.text(0.305, y - hgt / 2 + 0.007, wrap(txt, 46), fontsize=10, va="center",
color=BLUE if col == BLUE else INK, linespacing=1.3)
arrow = "▼" if dirn == "LOWER" else "▲"
ax.text(0.955, y - hgt / 2 + 0.007, f"{arrow} {dirn}", fontsize=11, weight="bold",
ha="right", va="center")
ax.text(0.02, 0.135, "Only the Abstract gives “lower”; every other location, "
"and the numbers, give “higher” (higher Ct = less viral RNA).",
fontsize=10, color=INK2, va="top")
legend_text_kinds(ax, y=-0.005)
save(fig, path)
# ------------------------------------------------------------------ design C: means +/- SD on the Ct scale
def design_c(path):
fig, ax = plt.subplots(figsize=(W, 4.2))
xs = {"DENV-1": 0, "DENV-2": 1}
off = 0.16
for s, r in REP.items():
x0 = xs[s]
for k, dx, mk in (("pb", -off, "o"), ("vx", off, "s")):
m, sd = r[k]
ax.errorbar(x0 + dx, m, yerr=sd, fmt=mk, ms=9, color=ORANGE, mfc=ORANGE,
mec=BG, mew=1.5, elinewidth=2, capsize=0, zorder=3)
ax.text(x0 + dx + 0.05, m, f"{'Pb' if k == 'pb' else 'Vx'} {m}", va="center",
fontsize=10, color=INK)
top = r["vx"][0] + r["vx"][1] + 1.2
ax.plot([x0 - off, x0 - off, x0 + off, x0 + off],
[r["pb"][0] + r["pb"][1] + 0.6, top, top, r["vx"][0] + r["vx"][1] + 0.6],
color=BLUE, lw=2)
ax.text(x0, top + 0.5, f"Vx − Pb = {R[s]['diff']:+.1f}", ha="center",
va="bottom", color=BLUE, weight="bold", fontsize=10.5)
ax.set_xticks([0, 1], ["DENV-1 (n = 161)", "DENV-2 (n = 205)"], fontsize=10.5, color=INK)
ax.set_xlim(-0.6, 1.6)
ax.set_ylim(0, 40)
ax.set_ylabel("RT-qPCR Ct (mean ± SD)\nhigher Ct = less viral RNA")
ax.yaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right"):
ax.spines[sp].set_visible(False)
ax.text(-0.55, 4.2, "Abstract: “significantly lower RT-qPCR Ct values in the "
"vaccinated group”\n— the reported means show the opposite",
fontsize=10, color=INK, va="center")
h = [Line2D([], [], marker="o", ls="none", color=ORANGE, ms=9, label="Placebo (Pb), reported"),
Line2D([], [], marker="s", ls="none", color=ORANGE, ms=9, label="Vaccinated (Vx), reported"),
Line2D([], [], color=BLUE, lw=2, label="Difference recomputed")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.1), ncol=3,
frameon=False, fontsize=10)
save(fig, path)
# ------------------------------------------------------------------ design D: difference with CI, null at 0
def design_d(path):
fig, ax = plt.subplots(figsize=(W, 3.3))
ys = {"DENV-1": 1, "DENV-2": 0}
ax.axvspan(-7, 0, color="#f1f0ec", zorder=0)
ax.axvline(0, color=INK2, lw=1, zorder=1)
ax.text(-3.5, 1.62, "Direction stated in the Abstract:\nvaccinated Ct lower",
ha="center", va="top", fontsize=10, color=INK2)
ax.text(3.5, 1.62, "Direction in Results, Added value,\nDiscussion: vaccinated Ct higher",
ha="center", va="top", fontsize=10, color=INK2)
for s in REP:
y = ys[s]
lo, hi = R[s]["ci"]
ax.plot([lo, hi], [y - 0.1] * 2, color=BLUE, lw=2, zorder=2)
ax.plot(R[s]["diff"], y - 0.1, "o", color=BLUE, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(hi + 0.15, y - 0.1, f"{R[s]['diff']:+.1f} ({lo:+.1f} to {hi:+.1f})",
va="center", fontsize=10, color=INK)
ax.plot(S6[s]["delta"], y + 0.12, "D", color=ORANGE, ms=8, mec=BG, mew=1.5, zorder=3)
ax.text(S6[s]["delta"] + 0.25, y + 0.12, f"{S6[s]['delta']:+.2f}", va="center",
fontsize=10, color=INK)
ax.set_yticks([1, 0], ["DENV-1\n(Vx 48, Pb 113)", "DENV-2\n(Vx 83, Pb 122)"],
fontsize=10.5, color=INK)
ax.set_xlim(-7, 7)
ax.set_ylim(-0.5, 1.7)
ax.set_xlabel("Vaccinated − placebo mean RT-qPCR Ct (cycles)")
ax.xaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
h = [Line2D([], [], marker="D", ls="none", color=ORANGE, ms=8,
label="Reported difference (Suppl. Table 6)"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=9,
label="Recomputed: Results means, Welch 95% CI")]
ax.legend(handles=h, loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=2,
frameon=False, fontsize=10)
save(fig, path)
# ------------------------------------------------------------------ design E: logic chain
def design_e(path):
fig = plt.figure(figsize=(W, 3.8))
ax = fig.add_axes([0, 0, 1, 1])
ax.axis("off")
xs = [0.02, 0.355, 0.69]
bw = 0.29
def row(y, label, boxes, cols, links):
ax.text(0.02, y + 0.255, label, fontsize=11, weight="bold", va="bottom")
for x, txt, c in zip(xs, boxes, cols):
panel(ax, x, y, bw, 0.24, c)
ax.text(x + 0.012, y + 0.12, txt, fontsize=10, va="center", linespacing=1.3,
color=BLUE if c == BLUE else INK)
for i, (sym, ok) in enumerate(links):
xa, xb = xs[i] + bw, xs[i + 1]
ax.annotate("", xy=(xb - 0.004, y + 0.12), xytext=(xa + 0.004, y + 0.12),
arrowprops=dict(arrowstyle="->", color=INK, lw=2))
ax.text((xa + xb) / 2, y + 0.155, sym, ha="center", va="bottom", fontsize=12,
weight="bold")
row(0.60, "Abstract, Findings",
["Premise:\n“significantly lower\nRT-qPCR Ct values in\nthe vaccinated group”",
"Paper's definition\n(Results): Ct inverse to\nnucleic acid, so lower\nCt = more viral RNA",
"Conclusion:\n“the vaccine could be\nreducing viral\nreplication”"],
[ORANGE, ORANGE, ORANGE], [("", True), ("≠", False)])
ax.text(0.6675, 0.585, "does not follow", ha="center", va="top", fontsize=10,
color=INK2)
row(0.16, "Results, Added value, Discussion; recomputed from the Results means",
["Recomputed Vx \u2212 Pb\nmean Ct: " + f"{d1:+.1f} (DENV-1),\n{d2:+.1f} (DENV-2), so\nvaccinated Ct higher",
"Same definition:\nhigher Ct =\nless viral RNA",
"Same conclusion:\nsuppression of viral\nreplication"],
[BLUE, ORANGE, ORANGE], [("", True), ("=", True)])
ax.text(0.6675, 0.145, "follows", ha="center", va="top", fontsize=10, color=INK2)
legend_text_kinds(ax, y=-0.05)
save(fig, path)
design_a(os.path.join(CAND, "cand_A_text_panels"))
design_b(os.path.join(CAND, "cand_B_location_tally"))
design_c(os.path.join(CAND, "cand_C_means_sd"))
design_d(os.path.join(CAND, "cand_D_difference_ci"))
design_e(os.path.join(CAND, "cand_E_logic_chain"))
# provisional choice for writeup.md, pending review of the candidates
design_a(os.path.join(HERE, "fig1"))
design_d(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)
Abstract says VR Ocean lowered anxiety versus all conditions; Results report no difference from 2D controls
Lancet Reg Health Am (2026) · doi:10.1016/j.lana.2026.101385 · PMC12930066
Summary
The abstract states that the immersive "VR Ocean" condition improved five outcomes, situational anxiety among them, "relative to all other conditions (all p < 0.05)", which for each outcome means six significant pairwise comparisons. For anxiety, the paper's own Results report a significant difference from only three of the six comparators (VR Opera, VR Pink Noise, working memory task) and none from the three non-immersive 2D controls ("all p > 0.14" in the repeated-measures analysis; "No significant difference" in the mixed model, which the Methods call the primary analysis), with no main effect of immersion (p = 0.58). The same sentence also overstates pain unpleasantness, where VR Ocean did not differ from VR Opera (p = 0.198 and 0.19). The primary outcome, heat pain tolerance, is unaffected, but the abstract misstates a secondary result in a way that bears on the study's own question of immersion versus content.
What the paper reports
All quotations were checked verbatim against the PMC-archived text.
Abstract, Findings:
VR Ocean significantly increased heat pain tolerance (Cohen's d = 1.60), reduced pain intensity and unpleasantness, improved mood, and reduced situational anxiety relative to all other conditions (all p < 0.05).
Research in context, Added value of this study:
We found that a calming, nature based VR environment (VR Ocean) produced the greatest improvements in pain tolerance, pain intensity, unpleasantness, mood, and anxiety.
Results, "VR effects on mood and situational anxiety", second paragraph (repeated-measures ANOVA, then the mixed model):
Condition also affected situational anxiety (F (6, 330) = 29.01, p < 0.001; Fig. 3d, Cohen's d = 0.232, 95% CI = [-0.121, 0.586]), with VR Ocean reducing anxiety more than VR Opera (p = 0.024), VR Pink Noise (p < 0.001), and the Working Memory Task (p < 0.001). No differences were observed compared with 2D control conditions (all p > 0.14). A two-way ANOVA revealed a main effect of sound type (F (2, 108) = 7.98, p < 0.001), but not immersion ([F (1, 108) = 0.31, p = 0.582]), and a significant interaction (F (2, 108) = 13.06, p < 0.001).
VR Ocean reduced anxiety considerably (VR Opera mean difference = -0.21, 95% CI -0.38, -0.04, p = 0.014), VR Pink noise mean difference = -0.35, 95% CI -0.51, -0.19, p < 0.001), and Working Memory mean difference = -0.48, 95% CI -0.67, -0.29, p < 0.001). No significant difference was observed in the 2D controls. In the model excluding the Working Memory Task, there was a main effect of Music type (F (2, 61.0) = 7.9, p = 0.001), but not Immersion (F (1, 61.0) = 0.31, p = 0.58), and a significant interaction (F (2, 61.0) = 12.6, p < 0.001). Within VR, Ocean < Opera (p = 0.005) and Ocean < Pink (p < 0.001) for anxiety; within 2D, sound types did not differ (all p > 0.70).
Results, "VR effects on pain intensity and unpleasantness", third paragraph:
Similarly, condition had a significant effect on pain unpleasantness (F (6, 330) = 8.88, p < 0.001; Fig. 3b), with VR Ocean outperforming most conditions except VR Opera (p = 0.198; all other p < 0.05).
In the mixed effect model, condition also affected unpleasantness [F (6, 59.5) = 4.62, p < 0.001]. VR Ocean exceeded most conditions (VR Opera mean difference = -0.18, 95% CI -0.45, 0.09, p = 0.19);
Discussion, first paragraph:
Ocean sound also produced the largest reductions in pain unpleasantness and anxiety, and improvements in mood and users' enjoyment.
Fig. 3 legend:
For each outcome, VR conditions (blue bars) consistently produced greater improvements than their 2D counterparts (gray bars). VR Ocean led to the most robust reductions in pain ratings and anxiety, and the greatest improvements in mood and the highest enjoyment.
What the paper's own numbers show
Method. The design has seven conditions, so "relative to all other conditions (all p < 0.05)" means six significant pairwise contrasts for each outcome named. For each of the five outcomes in the abstract sentence, I counted the VR Ocean contrasts that the Results call significant. I did this for both of the paper's analyses: the repeated-measures (RM) ANOVA post hoc tests and the linear mixed model (MM). In the MM, a contrast counts as significant when its 95% CI excludes zero. For every MM contrast printed with a CI, I back-calculated a two-sided p (SE = (upper − lower)/3.92, z = estimate/SE), with t(61) as a sensitivity check. I also recomputed the p-values of the reported F statistics and the p implied by the anxiety Cohen's d and its CI. No participant-level data are public, so this is a check of the paper against itself. All numbers come from recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Anxiety: VR Ocean contrasts significant (RM ANOVA) | Results, mood and anxiety | abstract: all 6 (all p < 0.05) | 3 of 6 (VR Opera 0.024, VR Pink Noise < 0.001, WMT < 0.001) |
| Anxiety: VR Ocean vs 2D Ocean, 2D Opera, 2D Pink Noise (RM ANOVA) | same paragraph | "all p > 0.14" | 0 of 3 significant |
| Anxiety: VR Ocean vs the three 2D controls (MM) | same paragraph | "No significant difference" | 0 of 3 significant; no estimates printed |
| Anxiety: VR Ocean vs VR Opera (MM), −0.21 (−0.38 to −0.04) | same paragraph | p = 0.014 | p = 0.0155 (t(61): 0.0163) |
| Anxiety: VR Ocean vs VR Pink Noise (MM), −0.35 (−0.51 to −0.19) | same paragraph | p < 0.001 | p = 1.8 × 10⁻⁵ |
| Anxiety: VR Ocean vs WMT (MM), −0.48 (−0.67 to −0.29) | same paragraph | p < 0.001 | p = 7.4 × 10⁻⁷ |
| Anxiety effect size, d = 0.232 (95% CI −0.121 to 0.586) | same paragraph | reported alongside the condition effect | CI includes 0; implied p = 0.198 |
| Anxiety, immersion main effect, two-way ANOVA F(1,108) = 0.31 | same paragraph | p = 0.582 | p = 0.579 |
| Anxiety, immersion main effect, MM F(1,61.0) = 0.31 | same paragraph | p = 0.58 | p = 0.580 |
| Anxiety, immersion × sound interaction F(2,108) = 13.06 | same paragraph | p < 0.001 | p = 8.3 × 10⁻⁶ |
| Unpleasantness: VR Ocean contrasts significant (RM ANOVA) | Results, pain intensity and unpleasantness | abstract: all 6 (all p < 0.05) | 5 of 6 (not VR Opera, p = 0.198) |
| Unpleasantness: VR Ocean vs VR Opera (MM), −0.18 (−0.45 to 0.09) | same section | p = 0.19 | p = 0.191 (t(61): 0.188) |
| Heat pain tolerance, pain intensity, mood: contrasts significant | Results | all 6 | 6 of 6 in both analyses (no discrepancy) |
| MM contrasts with an exact printed p (6 rows): best-fitting Bonferroni family size k | Results, all outcomes | Methods: "adjusted for multiple comparisons (Bonferroni)" | k = 1 (no adjustment) fits best in 6 of 6 rows |
| Pain intensity: VR Ocean vs VR Opera (MM), −0.88 (−1.44 to −0.33) | Results, pain intensity | p < 0.001 | p = 0.0019 (t(61): 0.0024); minor, still significant |
| Pain intensity: VR Ocean vs VR Pink Noise (MM), −0.44 (−0.77 to −0.11) | same paragraph | p = 0.006 | p = 0.009; minor, still significant |
Anxiety and unpleasantness are the only outcomes where the abstract's "all other conditions" fails. Elsewhere the recomputed values agree with what is printed to within rounding, apart from two small pain-intensity discrepancies that leave both contrasts significant. The abstract sentence is the statement that disagrees with the paper's own statistics.
Figure
Figure 1. The abstract (top, orange border) puts situational anxiety among the outcomes on which VR Ocean beat "all other conditions (all p < 0.05)", that is, 6 of 6 comparisons. The anxiety paragraph of the Results (bottom) names three significant comparisons in each of the paper's two analyses and states that the remaining three, against the 2D controls, were not significant: "all p > 0.14" after the repeated-measures (RM) ANOVA, and "No significant difference was observed in the 2D controls" in the mixed model. Both analyses also report no main effect of immersion on anxiety. The blue connector gives the count made here from those printed results. All text in the panels is quoted or paraphrased from the paper; no number is estimated.
Figure 2. Scope of the overstatement. Each row is one of the five outcomes in the abstract sentence, each column one of VR Ocean's six comparators (WMT, working memory task). A filled circle means the Results report that comparison as p < 0.05; an open circle means they report it as not significant, with the printed p below. The blue column is the number of significant comparisons counted from the Results; the orange column is what the abstract's "all other conditions (all p < 0.05)" implies. The RM ANOVA post hoc results are drawn; the mixed model gives the same pattern and counts. Heat pain tolerance, pain intensity and mood agree with the abstract; unpleasantness (5 of 6) and anxiety (3 of 6) do not.
Why it matters
- Abstract and summary panel. The overstatement appears in the abstract's Findings and is repeated in the "Research in context" panel, which says VR Ocean "produced the greatest improvements in … anxiety". Many readers see only these two parts of the paper. Taken at face value, they report a significant anxiety benefit of immersive VR Ocean over the identical content in 2D, which the Results do not find.
- It bears on the study's own question. The study set out to "disentangle the roles of immersion and emotional engagement". For anxiety, the paper's own analyses find no immersion main effect (p = 0.58 in both models). VR Ocean did not differ from 2D Ocean, and the 2D conditions did not differ from one another (p > 0.70). The Fig. 3 legend says that for "each outcome" VR "consistently produced greater improvements than their 2D counterparts". The anxiety analysis in the text does not support that statement.
- Unpleasantness. The same abstract sentence also overstates the unpleasantness result, because VR Ocean did not differ from VR Opera.
- What stands. The overstatement does not change the primary outcome. Heat pain tolerance shows six of six significant contrasts, with mixed-model CIs that all exclude zero. Pain intensity and mood also stand. So do the mediation results, which find no mediation by anxiety. The paper's central interpretation is that mood and enjoyment, not anxiety, carry the effect, and that interpretation does not rest on the anxiety contrasts. The correction needed is therefore to the wording of a secondary result, not to the paper's main conclusion.
Explanations considered
- "Relative to all other conditions" modifies only the last item. Under the narrowest reading, the phrase attaches only to "reduced situational anxiety", and that is the outcome where it fails most clearly (3 of 6). Under the widest reading, it covers all five outcomes, and then it also fails for unpleasantness (5 of 6). Neither reading makes the sentence agree with the Results.
- A different analysis supports the abstract. The paper analyses anxiety three ways: a one-way RM ANOVA with post hoc tests, a two-way immersion × sound ANOVA, and a mixed model (described in Methods as the primary analysis). None of them reports VR Ocean as better than the 2D controls. The RM ANOVA gives "all p > 0.14". The two-way model shows no immersion effect and no sound differences within 2D. The mixed model states "No significant difference was observed in the 2D controls". The comparison with healthy controls uses a different sample and does not test VR Ocean against the 2D controls.
- Multiplicity adjustment. The Methods say pairwise contrasts were Bonferroni-adjusted. If the RM "p > 0.14" values were adjusted, unadjusted p-values could in principle fall below 0.05, and the abstract might have used them. Two things count against this as an explanation. First, the paper's stated inference uses its printed values, and the abstract does not mention unadjusted p. Second, the printed mixed-model CI and p pairs fit no adjustment. For each contrast printed with an exact p,
recompute.py.txtback-calculates p from the CI with no adjustment (k = 1) and with a Bonferroni family of k = 6 or 21, either multiplying the p by k or treating the CI as adjusted too. k = 1 is the closest fit in all six such rows: unpleasantness vs VR Opera gives 0.191 unadjusted against 0.47 (k = 6) and 0.90 (k = 21) under the adjusted-CI reading, with 0.19 printed; anxiety vs VR Opera gives 0.0155 unadjusted against 0.0067 and 0.0037, with 0.014 printed. On that reading, the mixed model found the 2D anxiety contrasts non-significant without any multiplicity penalty, so a multiplicity adjustment is unlikely to explain the gap between the abstract and the Results. - The omnibus test. The condition effect on anxiety is highly significant (F(6,330) = 29.01, p = 7 × 10⁻²⁸). An omnibus test shows only that the seven conditions are not all equal. It says nothing about any particular pairwise contrast, so it cannot support "all p < 0.05" for each comparison.
- A text-conversion artefact. The passages are prose, not table cells. The PMC HTML, the Europe PMC full-text XML (where the symbols are encoded as
<and>entities) and the local PMC copy all agree. No superscript, minus sign or comparison symbol was lost. - The figure supports the abstract. The publisher's high-resolution Fig. 3, panel d (situational anxiety), marks the VR Ocean vs 2D Ocean bracket as significant (***). This does not resolve the conflict. The figure disagrees with its own Results text ("all p > 0.14"), and it shows no VR Ocean vs 2D Opera or VR Ocean vs 2D Pink Noise comparison. So even the figure does not support "all other conditions". In the same panel, the VR Pink Noise vs 2D Pink Noise bracket is also marked *** with the VR bar higher (more anxiety). That contradicts the legend's statement that VR "consistently produced greater improvements than their 2D counterparts". Which of the figure and the text is correct for 2D Ocean cannot be decided from the published material. Either way, the abstract claim fails for at least two of the three 2D controls.
Status
- Version of record. The publisher's pages (doi.org → linkinghub.elsevier.com, thelancet.com, sciencedirect.com) returned only a redirect stub or HTTP 403 to automated requests. Three copies of the published text agree on every passage quoted here: the live PMC page, the Europe PMC full-text XML, and the PMC-archived text read with
src/show.py. Fig. 3 was checked in the publisher's own high-resolution file (gr3_lrg.jpg, from the Elsevier CDN, ars.els-cdn.com). - Supplements. Both supplementary files were opened:
mmc1.docx(extended methods, CONSORT flow, supplementary figures) andmmc2.docx(captions).mmc1.docxfrom the Europe PMC bundle is byte-identical to the Elsevier CDN copy. Neither file gives the VR Ocean vs 2D anxiety contrasts or any other pairwise anxiety statistics. The data sharing statement says the data are available "upon request", so there is no public source data. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has empty update-to and updated-by fields, and the PubMed record (PMID 41743617) has no CommentsCorrections entries. The PubPeer API returned no record (HTTP 404) for the DOI, and web searches for the title together with "PubPeer", "correction" or "erratum" (repeated on 2026-10-24) found no thread or notice. - Sources. PMC article PMC12930066; Europe PMC full-text XML and supplementary files for PMC12930066; Elsevier CDN copies of Fig. 3 and
mmc1.docx; Crossref; PubMed; PubPeer; https://doi.org/10.1016/j.lana.2026.101385.
Suggested correction
In the abstract's Findings, replace
reduced pain intensity and unpleasantness, improved mood, and reduced situational anxiety relative to all other conditions (all p < 0.05).
with wording that matches the Results, for example:
reduced pain intensity and improved mood relative to all other conditions (all p < 0.05), reduced pain unpleasantness relative to all conditions except VR Opera, and reduced situational anxiety relative to VR Opera, VR Pink Noise and the working memory task, but not relative to the 2D controls.
The "Research in context" sentence and the Fig. 3 legend ("VR conditions … consistently produced greater improvements than their 2D counterparts") would need the same qualification for anxiety. The authors may also wish to:
- reconcile the significance brackets in Fig. 3d (VR Ocean vs 2D Ocean, VR Pink Noise vs 2D Pink Noise, both marked ***) with the text ("all p > 0.14"; no immersion main effect);
- state which pairwise p-values and CIs are Bonferroni-adjusted;
- say what contrast the anxiety Cohen's d (0.232, 95% CI −0.121 to 0.586) refers to.
As a separate point, the within-VR heat-pain-tolerance contrasts ("mean difference = 0.88, 95% CI 0.329–1.437 … 0.44, 95% CI 0.112–0.765") repeat the pain-intensity mixed-model contrasts in magnitude (−0.88, −1.44 to −0.33; −0.44, −0.77 to −0.11). They are also on a different scale from the tolerance differences of 7.98 to 11.92 reported in the paragraph before. The authors may wish to check them.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the numbers in the write-up for PMC12930066 and regenerate its figures.
Paper: "A critical role of affective content in the analgesic effect of virtual
reality: a cross-sectional within-subject study", Lancet Reg Health Am 2026,
doi:10.1016/j.lana.2026.101385.
Every input below is copied from the paper's Abstract and Results (PMC version).
No raw data are public, so nothing here re-estimates a model: the script counts
which of VR Ocean's six pairwise comparisons the Results report as p < 0.05,
recomputes p-values from the printed F statistics, and back-calculates p-values
from the printed mixed-model 95% CIs as a consistency check.
Run: python recompute.py.txt
Figures are written next to this file as fig1.svg/png and fig2.svg/png.
"""
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
ALPHA = 0.05 # Methods: "All statistical significances were set at p < 0.05."
# ------------------------------------------------------------------ inputs
COMPARATORS = ["VR Opera", "VR Pink Noise", "2D Ocean", "2D Opera",
"2D Pink Noise", "WMT"]
# p-values for VR Ocean vs each comparator, as printed in the Results.
# RM = 7-condition repeated-measures ANOVA post hoc; MM = 7-condition mixed model.
# Strings keep the paper's inequality signs. "all<0.05" etc. mark statements that
# cover several comparisons at once ("all p < 0.05", "all other p < 0.05").
RESULTS = {
"Heat pain tolerance (primary)": {
"RM": {c: "<0.05" for c in COMPARATORS}, # "p < 0.05 for all comparisons"
"MM": {c: "CI excl. 0" for c in COMPARATORS}, # "95% CIs not crossing 0"
},
"Pain intensity": {
"RM": {c: "<0.05" for c in COMPARATORS}, # "all p < 0.05"
"MM": {"VR Opera": "<0.001", "VR Pink Noise": "0.006", "2D Ocean": "0.003",
"2D Opera": "<0.001", "2D Pink Noise": "0.002", "WMT": "<0.001"},
},
"Pain unpleasantness": {
"RM": {"VR Opera": "0.198", **{c: "<0.05" for c in COMPARATORS[1:]}},
"MM": {"VR Opera": "0.19", "VR Pink Noise": "<0.001", "2D Ocean": "0.005",
"2D Opera": "<0.001", "2D Pink Noise": "<0.001", "WMT": "<0.001"},
},
"Mood": {
"RM": {c: "<0.001" for c in COMPARATORS}, # "all p < 0.001"
"MM": {c: "<0.001" for c in COMPARATORS},
},
"Situational anxiety": {
"RM": {"VR Opera": "0.024", "VR Pink Noise": "<0.001", "WMT": "<0.001",
"2D Ocean": ">0.14", "2D Opera": ">0.14", "2D Pink Noise": ">0.14"},
# MM: "No significant difference was observed in the 2D controls."
"MM": {"VR Opera": "0.014", "VR Pink Noise": "<0.001", "WMT": "<0.001",
"2D Ocean": "n.s.", "2D Opera": "n.s.", "2D Pink Noise": "n.s."},
},
}
def significant(p):
"""True if the printed p (or statement) means p < ALPHA."""
if p.startswith("<"):
return float(p[1:]) <= ALPHA
if p.startswith(">") or p == "n.s.":
return False
if p == "CI excl. 0":
return True
return float(p) < ALPHA
def p_from_ci(est, lo, hi):
"""Two-sided normal-approximation p from an estimate and its 95% CI."""
se = (hi - lo) / (2 * 1.959964)
return 2 * stats.norm.sf(abs(est / se))
def p_from_ci_range(est, lo, hi, half=0.005):
"""Range of p consistent with est, lo, hi each rounded to 2 decimals."""
ps = []
for de in (-half, half):
for dl in (-half, half):
for dh in (-half, half):
ps.append(p_from_ci(est + de, lo + dl, hi + dh))
return min(ps), max(ps)
def main():
print("PMC12930066: abstract 'relative to all other conditions (all p < 0.05)'\n")
# 1. Count significant comparisons per outcome and model
print("1. VR Ocean vs each of 6 comparators, as reported in the Results")
counts = {}
for outcome, models in RESULTS.items():
for m in ("RM", "MM"):
k = sum(significant(models[m][c]) for c in COMPARATORS)
counts[(outcome, m)] = k
rm, mm = counts[(outcome, "RM")], counts[(outcome, "MM")]
fails = [c for c in COMPARATORS if not significant(models["RM"][c])]
print(f" {outcome:30s} RM {rm}/6 MM {mm}/6 "
f"not significant: {', '.join(fails) if fails else 'none'}")
tot = sum(counts[(o, 'RM')] for o in RESULTS)
print(f" Abstract sentence covers 5 outcomes x 6 comparators = 30 comparisons; "
f"Results report {tot}/30 as p < 0.05 (RM), "
f"{sum(counts[(o, 'MM')] for o in RESULTS)}/30 (MM)")
print(" Anxiety, per comparator (RM / MM):")
for c in COMPARATORS:
a = RESULTS["Situational anxiety"]
print(f" {c:14s} {a['RM'][c]:7s} / {a['MM'][c]:6s} "
f"significant={significant(a['RM'][c])}")
# 2. p-values from the printed F statistics (anxiety)
print("\n2. Anxiety: p recomputed from printed F statistics")
for lab, F, d1, d2, rep in [
("RM condition", 29.01, 6, 330, "<0.001"),
("RM sound type", 7.98, 2, 108, "<0.001"),
("RM immersion", 0.31, 1, 108, "0.582"),
("RM interaction", 13.06, 2, 108, "<0.001"),
("MM condition", 5.21, 6, 59.7, "<0.001"),
("MM music type", 7.9, 2, 61.0, "0.001"),
("MM immersion", 0.31, 1, 61.0, "0.58"),
("MM interaction", 12.6, 2, 61.0, "<0.001")]:
print(f" {lab:15s} F({d1},{d2}) = {F:5.2f}: p = {stats.f.sf(F, d1, d2):.3g}"
f" (reported {rep})")
# 3. Mixed-model mean differences: p back-calculated from the 95% CI
print("\n3. Mixed model: p back-calculated from printed 95% CI (normal approx.)")
for lab, est, lo, hi, rep in [
("anxiety vs VR Opera", -0.21, -0.38, -0.04, "0.014"),
("anxiety vs VR Pink Noise", -0.35, -0.51, -0.19, "<0.001"),
("anxiety vs WMT", -0.48, -0.67, -0.29, "<0.001"),
("unpleasantness vs VR Opera", -0.18, -0.45, 0.09, "0.19")]:
p = p_from_ci(est, lo, hi)
a, b = p_from_ci_range(est, lo, hi)
pt = 2 * stats.t.sf(abs(est) / ((hi - lo) / (2 * stats.t.isf(0.025, 61))), 61)
print(f" {lab:27s} p = {p:.3g} (range under rounding {a:.3g} to {b:.3g};"
f" t(61): {pt:.3g}); reported {rep}")
# 4. Cohen's d for anxiety
d, lo, hi = 0.232, -0.121, 0.586
se = (hi - lo) / (2 * 1.959964)
print(f"\n4. Anxiety Cohen's d = {d}, 95% CI [{lo}, {hi}]: includes 0 = {lo < 0 < hi};"
f" implied two-sided p = {2 * stats.norm.sf(d / se):.3f}")
print(f" For comparison, unpleasantness d = 0.37, CI [0.01, 0.72]: includes 0 = "
f"{0.01 < 0 < 0.72}")
# 5. Are the printed mixed-model p-values Bonferroni-adjusted?
# Every VR Ocean contrast printed with an estimate and 95% CI.
print("\n5. Mixed-model contrasts: printed p vs p implied by the printed CI")
print(" k = number of contrasts in a Bonferroni family (1 = no adjustment).")
print(" (a) CI unadjusted, p multiplied by k; (b) CI and p both adjusted for k.")
fits = []
for o, c, est, lo, hi, rep in MM_CONTRASTS:
row = {}
for k in (1, 6, 21):
se_a = (hi - lo) / (2 * stats.norm.isf(0.025))
se_b = (hi - lo) / (2 * stats.norm.isf(0.05 / (2 * k)))
row[("a", k)] = min(1.0, k * 2 * stats.norm.sf(abs(est / se_a)))
row[("b", k)] = min(1.0, k * 2 * stats.norm.sf(abs(est / se_b)))
se_t = (hi - lo) / (2 * stats.t.isf(0.025, 61))
p_t = 2 * stats.t.sf(abs(est / se_t), 61)
best = ""
if rep[0] not in "<>":
r = float(rep)
best_key = min(row, key=lambda key: abs(np.log(row[key] / r)))
best = f" closest fit: {best_key[0]}, k={best_key[1]}"
fits.append(best_key[1])
flag = ""
if rep.startswith("<") and row[("a", 1)] >= float(rep[1:]):
flag = " <- printed bound not met by CI-implied p"
print(f" {o:15s} vs {c:14s} {est:7.3f} [{lo:7.3f}, {hi:7.3f}] reported {rep:6s}"
f" k=1: {row[('a', 1)]:.2g} (t61 {p_t:.2g}); k=6: a {row[('a', 6)]:.2g}"
f" b {row[('b', 6)]:.2g}; k=21: a {row[('a', 21)]:.2g} b {row[('b', 21)]:.2g}"
f"{best}{flag}")
print(f" Rows with an exact printed p: {len(fits)}; closest fit is k=1 in "
f"{sum(k == 1 for k in fits)} of them")
# 6. Within-VR heat-pain-tolerance contrasts vs pain-intensity contrasts
print("\n6. Within-VR tolerance contrasts (factorial MM) vs pain-intensity contrasts (7-cond. MM)")
tol = [("Ocean vs Opera", 0.88, 0.329, 1.437), ("Ocean vs Pink", 0.44, 0.112, 0.765)]
inten = [("VR Opera", -0.88, -1.44, -0.33), ("VR Pink Noise", -0.44, -0.77, -0.11)]
for (a, e1, l1, h1), (b, e2, l2, h2) in zip(tol, inten):
print(f" tolerance {a}: {e1} [{l1}, {h1}] intensity vs {b}: {e2} [{l2}, {h2}]"
f" same magnitude: {abs(e1) == abs(e2)}")
tol_md = [c[2] for c in MM_CONTRASTS if c[0] == "tolerance"]
print(f" 7-condition tolerance mean differences range {min(abs(x) for x in tol_md)}"
f" to {max(abs(x) for x in tol_md)} units")
make_fig1(counts)
make_fig2(counts)
print("\nFigures written to", HERE)
# Mixed-model (7-condition LMM) contrasts VR Ocean minus comparator, as printed.
MM_CONTRASTS = [
("tolerance", "VR Opera", -7.984, -15.56, -0.402, "<0.05"),
("tolerance", "VR Pink Noise", -9.83, -16.86, -2.81, "<0.05"),
("tolerance", "2D Ocean", -8.09, -15.60, -0.59, "<0.05"),
("tolerance", "2D Opera", -10.70, -18.81, -2.59, "<0.05"),
("tolerance", "2D Pink Noise", -9.77, -18.37, -1.17, "<0.05"),
("tolerance", "WMT", -11.918, -20.887, -2.949, "<0.05"),
("intensity", "VR Opera", -0.88, -1.44, -0.33, "<0.001"),
("intensity", "VR Pink Noise", -0.44, -0.77, -0.11, "0.006"),
("intensity", "2D Ocean", -0.63, -1.05, -0.21, "0.003"),
("intensity", "2D Opera", -0.95, -1.50, -0.40, "<0.001"),
("intensity", "2D Pink Noise", -0.71, -1.17, -0.25, "0.002"),
("intensity", "WMT", -1.12, -1.68, -0.56, "<0.001"),
("unpleasantness", "VR Opera", -0.18, -0.45, 0.09, "0.19"),
("unpleasantness", "VR Pink Noise", -0.48, -0.76, -0.20, "<0.001"),
("unpleasantness", "2D Ocean", -0.40, -0.68, -0.12, "0.005"),
("unpleasantness", "2D Opera", -0.58, -0.89, -0.27, "<0.001"),
("unpleasantness", "2D Pink Noise", -0.46, -0.73, -0.19, "<0.001"),
("unpleasantness", "WMT", -0.72, -1.06, -0.38, "<0.001"),
("mood", "VR Opera", -0.62, -0.90, -0.34, "<0.001"),
("mood", "VR Pink Noise", -0.84, -1.12, -0.56, "<0.001"),
("mood", "2D Ocean", -0.41, -0.60, -0.22, "<0.001"),
("mood", "2D Opera", -0.93, -1.25, -0.61, "<0.001"),
("mood", "2D Pink Noise", -1.05, -1.36, -0.74, "<0.001"),
("mood", "WMT", -1.20, -1.55, -0.85, "<0.001"),
("anxiety", "VR Opera", -0.21, -0.38, -0.04, "0.014"),
("anxiety", "VR Pink Noise", -0.35, -0.51, -0.19, "<0.001"),
("anxiety", "WMT", -0.48, -0.67, -0.29, "<0.001"),
]
# ------------------------------------------------------------------ figures
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834"
BLUE = "#2a78d6"
GRID = "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
FIG_W, FIG_H = 7.2, 6.05
LINE = 0.205 / FIG_H # line height for 10 pt text, in figure fraction
PAD = 0.14 / FIG_H
def panel(ax, x, ytop, w, title, lines, edge, chars):
"""Draw a text panel from ytop downwards; return its bottom y."""
y = ytop - PAD
ax.text(x + 0.022, y, title, ha="left", va="top", fontsize=11, fontweight="bold",
color=INK)
y -= LINE * 1.45
for txt, color, bold in lines:
for ln in textwrap.wrap(txt, width=chars):
ax.text(x + 0.022, y, ln, ha="left", va="top", fontsize=10, color=color,
fontweight="bold" if bold else "normal")
y -= LINE
y -= LINE * 0.3
ybot = y - PAD * 0.4
ax.add_patch(FancyBboxPatch((x, ybot), w, ytop - ybot,
boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=edge, lw=2, zorder=0))
return ybot
def make_fig1(counts):
fig = plt.figure(figsize=(FIG_W, FIG_H))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
top_bot = panel(ax, 0.03, 0.975, 0.94, "Abstract, Findings (as printed)", [
("“VR Ocean significantly increased heat pain tolerance (Cohen's d = 1.60), "
"reduced pain intensity and unpleasantness, improved mood, and reduced "
"situational anxiety relative to all other conditions (all p < 0.05).”", INK, False),
("For anxiety this means: 6 of 6 comparisons p < 0.05", ORANGE, True),
], ORANGE, chars=84)
lt = top_bot - 1.3 / FIG_H
common = dict(edge=INK2, chars=36)
b1 = panel(ax, 0.03, lt, 0.455, "Results: RM ANOVA, post hoc", [
("VR Ocean lower anxiety than", INK2, False),
(" VR Opera p = 0.024", INK, False),
(" VR Pink Noise p < 0.001", INK, False),
(" Working memory task p < 0.001", INK, False),
("“No differences were observed compared with 2D control conditions "
"(all p > 0.14).”", INK, True),
("Immersion main effect: F(1,108) = 0.31, p = 0.582", INK2, False),
], **common)
b2 = panel(ax, 0.515, lt, 0.455, "Results: mixed model (primary)", [
("VR Ocean lower anxiety than", INK2, False),
(" VR Opera p = 0.014", INK, False),
(" VR Pink Noise p < 0.001", INK, False),
(" Working memory task p < 0.001", INK, False),
("“No significant difference was observed in the 2D controls.”", INK, True),
("Immersion main effect: F(1,61) = 0.31, p = 0.58", INK2, False),
], **common)
k_rm = counts[("Situational anxiety", "RM")]
k_mm = counts[("Situational anxiety", "MM")]
ymid = (top_bot + lt) / 2
ax.annotate("", xy=(0.5, top_bot - 0.005), xytext=(0.5, ymid + 0.045),
arrowprops=dict(arrowstyle="-|>", lw=2, color=INK2, mutation_scale=14))
for xe in (0.2575, 0.7425):
ax.annotate("", xy=(xe, lt + 0.004), xytext=(0.5 + (xe - 0.5) * 0.35, ymid - 0.045),
arrowprops=dict(arrowstyle="-|>", lw=2, color=INK2, mutation_scale=14))
ax.text(0.5, ymid,
f"Conflict: Results count {k_rm} of 6 (ANOVA) and {k_mm} of 6 (mixed model);\n"
"none of the three 2D controls differs from VR Ocean",
ha="center", va="center", fontsize=10.5, color=BLUE, fontweight="bold",
bbox=dict(boxstyle="round,pad=0.4", fc=BG, ec=BLUE, lw=2))
ax.text(0.03, min(b1, b2) - 0.02,
"Orange: claim as printed in the abstract. Blue: count made here from the "
"Results' own p-values.", ha="left", va="top", fontsize=10, color=INK2)
save(fig, "fig1")
def make_fig2(counts):
outcomes = list(RESULTS)
ylabels = ["Heat pain tolerance\n(primary)", "Pain intensity", "Pain unpleasantness",
"Mood", "Situational anxiety"]
xlabels = ["VR\nOpera", "VR\nPink", "2D\nOcean", "2D\nOpera", "2D\nPink", "WMT"]
fig, ax = plt.subplots(figsize=(7.2, 4.5))
fig.subplots_adjust(left=0.245, right=0.995, top=0.76, bottom=0.14)
ny, nx = len(outcomes), len(COMPARATORS)
for i, o in enumerate(outcomes):
y = ny - 1 - i
bounds = []
for j, c in enumerate(COMPARATORS):
p = RESULTS[o]["RM"][c]
if p.startswith(">"):
bounds.append((j, p))
if significant(p):
ax.plot(j, y, "o", ms=11, mfc=INK2, mec=INK2)
else:
ax.plot(j, y, "o", ms=11, mfc="white", mec=INK, mew=2)
if not p.startswith(">"):
ax.text(j, y - 0.38, "p = " + p, ha="center", va="center",
fontsize=10, color=INK)
if bounds:
ax.text(np.mean([b[0] for b in bounds]), y - 0.38,
f"all p > {bounds[0][1][1:]}", ha="center", va="center",
fontsize=10, color=INK)
k = counts[(o, "RM")]
ax.text(nx + 0.0, y, f"{k}/6", ha="center", va="center", fontsize=11,
fontweight="bold", color=BLUE)
ax.text(nx + 1.15, y, "6/6", ha="center", va="center", fontsize=11,
fontweight="bold", color=ORANGE)
ax.text(nx + 0.0, ny - 0.45, "Results\ncount", ha="center", va="bottom",
fontsize=10, color=BLUE, fontweight="bold")
ax.text(nx + 1.15, ny - 0.45, "Abstract\nclaim", ha="center", va="bottom",
fontsize=10, color=ORANGE, fontweight="bold")
ax.axvline(nx - 0.6, color=GRID, lw=1)
ax.set_xticks(range(nx))
ax.set_xticklabels(xlabels, fontsize=10)
ax.xaxis.tick_top()
ax.set_yticks(range(ny))
ax.set_yticklabels(ylabels[::-1], fontsize=10.5, color=INK)
ax.set_xlim(-0.55, nx + 1.7)
ax.set_ylim(-0.75, ny - 0.5)
ax.tick_params(length=0)
for s in ax.spines.values():
s.set_visible(False)
for y in range(ny):
ax.plot([-0.4, nx - 0.6], [y, y], color=GRID, lw=0.8, zorder=0)
fig.text(0.02, 0.975, "VR Ocean compared with each other condition, as reported in the "
"Results\n(RM ANOVA post hoc; the mixed model gives the same counts)", ha="left", va="top", fontsize=10,
color=INK2)
handles = [Line2D([], [], marker="o", ls="", ms=10, mfc=INK2, mec=INK2,
label="p < 0.05 in Results"),
Line2D([], [], marker="o", ls="", ms=10, mfc="white", mec=INK, mew=2,
label="not significant in Results")]
fig.legend(handles=handles, loc="lower left", bbox_to_anchor=(0.24, 0.0),
frameon=False, fontsize=10, ncol=2)
save(fig, "fig2")
if __name__ == "__main__":
main()
Abstract reports the severe-obesity odds ratio from the worsening-dyspnoea model instead of the improvement model
Lancet Reg Health Eur (2023) · doi:10.1016/j.lanepe.2023.100635 · PMC10145209
Summary
The abstract reports that hospitalised COVID-19 patients who were not severely obese had higher odds of improvement in dyspnoea between 5 months and 1 year, with an odds ratio of 4.19 (95% CI 2.14–8.19). Six of the seven abstract estimates are exact reciprocals of the primary-outcome model in Table 2, which gives 0.36 (0.18–0.72) for severe obesity. Its reciprocal is 2.78 (1.39–5.56). The printed 4.19 (2.14–8.19) is, digit for digit, the Table 3 estimate for severe obesity in a different model: the exploratory outcome of developing or worsening dyspnoea, fitted in a different analysis sample (n = 1151 rather than 990). The direction and significance are unchanged. The abstract still overstates the primary-outcome effect by about 51% and places its lower confidence limit at 2.14 instead of 1.39. Readers, data extractors and meta-analysts often take estimates from the abstract.
What the paper reports
Abstract, Findings:
We observed higher odds of improvement between 5-month and 1-year follow-up among PHOSP-COVID participants who were younger (odds ratio 1.02 per year, 95% CI 1.01–1.03), male (1.54, 1.16–2.04), neither obese nor severely obese (1.82, 1.06–3.13 and 4.19, 2.14–8.19, respectively), had no pre-existing anxiety or depression (1.56, 1.09–2.22) or cardiovascular disease (1.33, 1.00–1.79), and shorter hospital admission (1.01 per day, 1.00–1.02).
Methods, Outcomes, first paragraph (the primary outcome):
Specifically, the primary outcome was participant-reported improvement in dyspnoea between 5-month and 1-year follow-up, defined as a reduction (at least by 1 point) in the MRC Dyspnoea score between the two timepoints.
Methods, Outcomes, second paragraph (the exploratory outcome):
Therefore, the exploratory outcome variable we defined based on the 1-year and baseline MRC Dyspnoea scores was long-term post-COVID-19 dyspnoea (i.e., worsening dyspnoea over 1 year).
The analysis of this exploratory outcome aimed to answer a different question
Table 2 caption, column headers, BMI rows and footer (PHOSP-COVID n = 990 for the first three columns, COVIDENCE UK n = 3309 for the last two):
Associations between participant characteristics and improvement of dyspnoea symptoms from 5 months to 1 year.
| Univariable | Multivariable (including hospitalisation variables) | Multivariable (excluding hospitalisation variables) | Univariable | Multivariable
30 to <40 (obese) | 0.57 (0.34–0.95) | 0.55 (0.32–0.94) | 0.58 (0.34–0.99) | 0.65 (0.54–0.80) | 0.62 (0.50–0.75)
≥40 (severely obese) | 0.43 (0.22–0.84) | 0.36 (0.18–0.72) | 0.38 (0.19–0.75) | 0.70 (0.48–1.01) | 0.51 (0.34–0.75)Data are odds ratio (95% CI), with worse or persistent dyspnoea as the reference category.
Table 3 caption, BMI rows and footer (same column order; PHOSP-COVID n = 1151):
Associations between participants characteristics and development or worsening of dyspnoea symptoms at 1 year.
30 to <40 (obese) | 2.00 (1.26–3.15) | 2.22 (1.35–3.64) | 2.20 (1.35–3.57) | 2.34 (2.02–2.72) | 3.46 (2.95–4.06)
≥40 (severely obese) | 3.14 (1.70–5.79) | 4.19 (2.14–8.19) | 4.23 (2.18–8.23) | 3.01 (2.20–4.13) | 5.98 (4.24–8.43)Data are odds ratio (95% CI), with stable or improved dyspnoea as the reference category.
Results, third paragraph (the only place the text cites the improvement estimates for BMI; it gives no number):
We observed reduced odds of improved dyspnoea from 5 months to 1 year after discharge among patients hospitalised with COVID-19 who were older, female, obese or severely obese, and with pre-existing depression or anxiety (Table 2; Fig. 2).
What the paper's own numbers show
Method. In the abstract each determinant is stated at its favourable level ("younger", "male", "neither obese nor severely obese", "no ... cardiovascular disease"). Table 2 states the adverse level against the reference. So each abstract figure should be the Table 2 odds ratio with the comparison reversed: OR becomes 1/OR, and a CI (l, u) becomes (1/u, 1/l). I applied this to every column of Table 2 to find the one the abstract uses, and compared each abstract figure with Table 3 as well. As an external check of the Table 2 severe-obesity row, I also recomputed the crude odds ratio from the Table 1 counts. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Age, per year younger | Abstract vs Table 2 (multivariable incl. hosp.) | 1.02 (1.01–1.03) | 1/0.98 (0.97–0.99) = 1.02 (1.01–1.03): matches |
| Male vs female | Abstract vs Table 2 | 1.54 (1.16–2.04) | 1/0.65 (0.49–0.86) = 1.54 (1.16–2.04): matches |
| Not obese vs obese (BMI 30 to <40) | Abstract vs Table 2 | 1.82 (1.06–3.13) | 1/0.55 (0.32–0.94) = 1.82 (1.06–3.125): matches |
| No depression or anxiety | Abstract vs Table 2 | 1.56 (1.09–2.22) | 1/0.64 (0.45–0.92) = 1.56 (1.09–2.22): matches |
| No cardiovascular disease | Abstract vs Table 2 | 1.33 (1.00–1.79) | 1/0.75 (0.56–1.00) = 1.33 (1.00–1.79): matches |
| Shorter stay, per day | Abstract vs Table 2 | 1.01 (1.00–1.02) | 1/0.99 (0.98–1.00) = 1.01 (1.00–1.02): matches |
| Not severely obese vs BMI ≥40 | Abstract vs Table 2 | 4.19 (2.14–8.19) | 1/0.36 (0.18–0.72) = 2.78 (1.39–5.56): does not match |
| Severely obese, same model, other outcome | Table 3, multivariable incl. hosp. | 4.19 (2.14–8.19) | identical to the abstract figure |
| Ratio of abstract to recomputed point estimate | — | — | 4.19 / 2.778 = 1.51 (+51%) |
| Table 2 OR that would invert to 4.19 | Table 2, severely obese, multivariable incl. hosp. | 0.36 | 1/4.19 = 0.239; any value that rounds to 0.36 inverts to 2.74–2.82 |
| Severely obese, other Table 2 columns, inverted | Table 2 | 0.43 (0.22–0.84); 0.38 (0.19–0.75) | 2.33 (1.19–4.55); 2.63 (1.33–5.26) |
| Severely obese, appendix sensitivity analyses, inverted | Appendix Table S1 | 0.35 (0.17–0.72); 0.35 (0.17–0.70); 0.37 (0.14–0.93); 0.28 (0.13–0.60); 0.34 (0.17–0.69) | 2.86 (1.39–5.88); 2.86 (1.43–5.88); 2.70 (1.08–7.14); 3.57 (1.67–7.69); 2.94 (1.45–5.88) |
| Crude OR, severely obese vs BMI <25, improvement | Table 1 counts (improved/worse-or-persistent: 25/61 vs 34/36; the 692 participants with BMI recorded) | Table 2 univariable 0.43 (0.22–0.84) | (25/61)/(34/36) = 0.434 (Wald CI 0.224–0.840): matches |
| Internal consistency of the Table 2 cell | Table 2 | 0.36 (0.18–0.72) | geometric CI midpoint 0.360 |
| Ratio of severe-obese to obese estimates in the abstract | Abstract | 4.19 / 1.82 = 2.30 | 2.778 / 1.818 = 1.53 |
The abstract and Table 2 use one convention throughout: six of the seven abstract estimates are reciprocals of the PHOSP "Multivariable (including hospitalisation variables)" column, to the printed precision. The seventh, for severe obesity, cannot be produced from any improvement-outcome estimate in the paper or appendix: the ten candidates invert to between 1.43 and 3.57, and none to 4.19 (2.14–8.19). Outside the abstract, the same three numbers appear only once in the paper, in the Table 3 cell for severe obesity in the multivariable model including hospitalisation variables. That model has a different outcome, a different reference category ("stable or improved dyspnoea") and a different PHOSP sample (n = 1151). The Table 3 odds ratio is greater than 1 because it describes the adverse outcome, so it reads plausibly in the abstract's favourable-level phrasing. It is still not the improvement estimate. The Table 2 cell itself is supported by its neighbouring models, by all five sensitivity analyses, and by a crude odds ratio recomputed from the Table 1 counts.
Figure
Figure 1. Where the two BMI estimates in the abstract's Findings come from. Top panel: the abstract excerpt, with its printed values in orange. Bottom left: Table 2 (primary outcome, improvement; PHOSP-COVID n = 990; multivariable model including hospitalisation variables), with the printed odds ratios in black and their reciprocals, recomputed for the abstract's favourable-level comparison, in blue. Bottom right: Table 3 (exploratory outcome, development or worsening; n = 1151; same model specification). The obese estimate, 1.82 (1.06–3.13), is the reciprocal of Table 2's 0.55 (0.32–0.94). The severe-obesity estimate should be the reciprocal of Table 2's 0.36 (0.18–0.72), which is 2.78 (1.39–5.56). The printed 4.19 (2.14–8.19) matches the Table 3 severe-obesity cell exactly.
Figure 2. All seven odds ratios in the abstract's Findings (orange circles, as printed) next to the reciprocals of the corresponding Table 2 estimates (blue squares; PHOSP multivariable model including hospitalisation variables), with 95% CIs, on one log-scaled axis. The vertical line marks OR = 1. For six determinants the two intervals coincide to the printed precision. For severe obesity the printed estimate, 4.19 (2.14–8.19), lies well to the right of the value implied by Table 2, 2.78 (1.39–5.56). Both intervals exclude 1. The figure shows that the error is confined to one figure and that the convention the abstract uses is not in doubt.
Why it matters
- A primary-outcome effect size in the abstract is wrong. Severe obesity is one of the determinants the abstract names for the primary outcome. The estimate printed for it is about 1.5 times the value the paper's own primary model gives, and its lower confidence limit is printed as 2.14 instead of 1.39.
- It distorts the obesity gradient. As printed, the severe-obesity effect is 2.3 times the obesity effect (4.19 vs 1.82). The paper's Table 2 implies a ratio of about 1.5 (2.78 vs 1.82), with heavily overlapping confidence intervals. A reader of the abstract would infer a much steeper dose-response with BMI than the improvement analysis shows.
- Abstracts feed evidence synthesis. Systematic reviewers, guideline writers and data extractors often take estimates from abstracts. A pooled analysis of severe obesity and post-COVID-19 dyspnoea recovery that used this abstract would enter a value from a different outcome and sample.
- What does not change. Direction and statistical significance are the same with either value, so the abstract's qualitative statement ("neither obese nor severely obese" is associated with higher odds of improvement) and the Interpretation hold. Tables 2 and 3, the Results text, the Discussion and the forest plot (Fig. 2) are not affected. The underlying regressions are not in question; only the number carried into the abstract is.
Explanations considered
- A different column of Table 2. The univariable estimate 0.43 inverts to 2.33 (1.19–4.55). The model excluding hospitalisation variables, 0.38, inverts to 2.63 (1.33–5.26). The COVIDENCE UK columns invert to 1.43 and 1.96. None gives 4.19. The six correctly reproduced abstract figures (including length of hospital stay, which appears only in the model including hospitalisation variables) fix the column used as the PHOSP multivariable model including hospitalisation variables.
- A sensitivity analysis. The five severe-obesity estimates in appendix Table S1 invert to 2.70–3.57. None gives 4.19 (2.14–8.19), and the string "4.19 (2.14–8.19)" does not occur in the appendix.
- Rounding. Any odds ratio that rounds to 0.36 inverts to between 2.74 and 2.82. To invert to 4.19, the Table 2 value would have to be about 0.24.
- An error in Table 2 rather than in the abstract. Unlikely. The Table 2 cell is internally consistent (the geometric midpoint of 0.18–0.72 is 0.360). It sits close to the neighbouring models (0.43 univariable, 0.38 excluding hospitalisation variables) and to all five sensitivity analyses (0.28–0.37). Its univariable neighbour, 0.43 (0.22–0.84), is reproduced to the printed precision by the crude odds ratio from the Table 1 counts. The forest plot (Fig. 2, left panel, improvement) also shows the PHOSP severe-obesity estimate below 1, in line with Table 2.
- The abstract figure is meant to describe the exploratory outcome. Not consistent with the text. The abstract sentence is about "higher odds of improvement between 5-month and 1-year follow-up", and the other six figures in the same sentence, including the obese figure next to it, come from Table 2. Table 3 uses a different outcome, reference category and sample (n = 1151, the 1188 participants with recalled pre-COVID-19 scores minus 37 with a score of 5). Its obese estimate, 2.22 (1.35–3.64), is not the abstract's 1.82, so the abstract does not report Table 3 consistently either.
- A conversion or archiving artefact. Ruled out. The same values appear in the current PMC version of the article (PMC10145209.2), in the Europe PMC full text, and in the publisher's typeset PDF (version of record).
- Another legitimate reading of "neither obese nor severely obese". The phrase could be read as a comparison against the BMI <25 reference group rather than a literal reversal. Either way, the abstract's obese figure equals 1/0.55 from Table 2, so the severe-obesity figure on the same reading would be 1/0.36 = 2.78.
Status
- Version of record. Checked. The publisher's typeset PDF (Elsevier/thelancet.com footer, 19 pages; obtained from the Oxford ORA repository record labelled "Version of record", ora.ox.ac.uk) has the same abstract sentence and the same Table 2 and Table 3 rows. The publisher's own pages (thelancet.com, sciencedirect.com) refused automated requests (HTTP 403). PMC lists two versions of the article (PMC10145209.1 and .2). The current version, .2 (PMC OAI full text), and the Europe PMC full-text XML carry the same values. Version .1 could not be retrieved separately.
- Supplement. Checked. Appendix (mmc1.docx, from the Europe PMC supplementary files) Table S1, "Sensitivity analyses for predicting improvement of dyspnoea symptoms from the 5-month to 1-year visits of PHOSP study", gives severe-obesity improvement odds ratios of 0.28–0.37, none of which inverts to 4.19. The appendix's only "4.19" is an unrelated MRC-score entry. Fig. 2 shows the estimate of about 4.2 only in its development-or-worsening panel.
- Corrections and retractions. None found as of 2026-10-24: Crossref lists no update for the DOI, PMC and Europe PMC list no correction or retraction notice linked to the article (PMID 37261214), and a web search for an erratum returned none.
- PubPeer. No thread found in a web search for the DOI or title. PubPeer's own search API answered "Human verification required" and could not be queried automatically, so a thread cannot be fully ruled out.
- Sources. PMC10145209 (current version .2) and the PMC ID converter, Europe PMC full text, metadata and supplementary files, Crossref (no works with an
updatesrelation to the DOI), ORA version-of-record PDF, web searches. All as of 2026-10-24.
Suggested correction
In the abstract, Findings, replace
neither obese nor severely obese (1.82, 1.06–3.13 and 4.19, 2.14–8.19, respectively)
with
neither obese nor severely obese (1.82, 1.06–3.13 and 2.78, 1.39–5.56, respectively)
Here 2.78 (1.39–5.56) is the reciprocal of the Table 2 estimate 0.36 (0.18–0.72), computed from the rounded published values. The authors may prefer to recompute it from the unrounded model output. No change is needed to the tables, the Results text, the Discussion or the conclusions.
Recomputation script (python recompute.py.txt)
#!/usr/bin/env python3
"""Recompute the numbers in the write-up for PMC10145209 and regenerate its figures.
Paper: "Determinants of recovery from post-COVID-19 dyspnoea: analysis of UK prospective
cohorts of hospitalised COVID-19 patients and community-based controls",
Lancet Reg Health Eur 2023, doi:10.1016/j.lanepe.2023.100635.
Every input below is copied from the paper (abstract, Tables 1-3) or its appendix
(Table S1). Run: python recompute.py.txt
Figures are written next to this file as fig1.svg/png and fig2.svg/png.
"""
import math
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyArrowPatch, FancyBboxPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (as printed)
# Abstract, Findings: odds of IMPROVEMENT, stated for the favourable level
# (younger, male, not obese, no depression/anxiety, no CVD, shorter stay).
abstract = {
"Age (per year younger)": (1.02, 1.01, 1.03),
"Male vs female": (1.54, 1.16, 2.04),
"Not obese vs obese (30 to <40)": (1.82, 1.06, 3.13),
"Not severely obese vs BMI ≥40": (4.19, 2.14, 8.19),
"No depression/anxiety": (1.56, 1.09, 2.22),
"No cardiovascular disease": (1.33, 1.00, 1.79),
"Shorter stay (per day)": (1.01, 1.00, 1.02),
}
# Table 2 (improvement, primary outcome, PHOSP n = 990), column
# "Multivariable (including hospitalisation variables)", adverse level vs reference.
t2_incl = {
"Age (per year younger)": (0.98, 0.97, 0.99), # age per year
"Male vs female": (0.65, 0.49, 0.86), # female vs male
"Not obese vs obese (30 to <40)": (0.55, 0.32, 0.94), # obese vs BMI <25
"Not severely obese vs BMI ≥40": (0.36, 0.18, 0.72), # severely obese vs BMI <25
"No depression/anxiety": (0.64, 0.45, 0.92),
"No cardiovascular disease": (0.75, 0.56, 1.00),
"Shorter stay (per day)": (0.99, 0.98, 1.00), # length of stay per day
}
# Table 2, severely obese row, all five columns
t2_sev_row = {
"PHOSP univariable": (0.43, 0.22, 0.84),
"PHOSP multivariable incl. hospitalisation": (0.36, 0.18, 0.72),
"PHOSP multivariable excl. hospitalisation": (0.38, 0.19, 0.75),
"COVIDENCE univariable": (0.70, 0.48, 1.01),
"COVIDENCE multivariable": (0.51, 0.34, 0.75),
}
# Appendix Table S1 (PHOSP sensitivity analyses, improvement outcome), severely obese row
s1_sev = {
"S1 fully adjusted (n=990)": (0.35, 0.17, 0.72),
"S1 complete case (n=692)": (0.35, 0.17, 0.70),
"S1 multiple imputation (n=990)": (0.37, 0.14, 0.93),
"S1 excl. improved at 5 m (n=797)": (0.28, 0.13, 0.60),
"S1 adj. calendar date (n=984)": (0.34, 0.17, 0.69),
}
# Table 3 (development or worsening, exploratory outcome, PHOSP n = 1151),
# column "Multivariable (including hospitalisation variables)"
t3_incl = {
"Not obese vs obese (30 to <40)": (2.22, 1.35, 3.64),
"Not severely obese vs BMI ≥40": (4.19, 2.14, 8.19),
"No depression/anxiety": (1.52, 1.05, 2.20),
"Shorter stay (per day)": (1.02, 1.01, 1.03),
"Age (per year younger)": (1.01, 1.00, 1.03),
"Male vs female": (2.28, 1.73, 2.99),
}
# Table 3, severely obese row, all five columns
t3_sev_row = [(3.14, 1.70, 5.79), (4.19, 2.14, 8.19), (4.23, 2.18, 8.23),
(3.01, 2.20, 4.13), (5.98, 4.24, 8.43)]
# Table 1 (PHOSP, improvement analysis): BMI counts, improved / worse-or-persistent
t1 = {"<25": (34, 36), "≥40": (25, 61)}
def inv(t):
"""Reverse the comparison: OR -> 1/OR, CI (l, u) -> (1/u, 1/l)."""
o, l, u = t
return (1 / o, 1 / u, 1 / l)
def fmt(t, d=2):
return f"{t[0]:.{d}f} ({t[1]:.{d}f}–{t[2]:.{d}f})"
def r2(t):
return tuple(round(x + 1e-12, 2) for x in t)
def same(a, b, tol=0.0051):
return all(abs(x - y) <= tol for x, y in zip(a, b))
# ---------------------------------------------------------------- 1. abstract vs 1/Table 2
print("1. Abstract figures vs reciprocal of Table 2 (improvement; PHOSP multivariable incl. hosp.)")
print(f" {'quantity':36s} {'abstract':20s} {'1/Table 2':22s} {'Table 3 (worsening)':20s} match")
recomputed = {}
for k, a in abstract.items():
r = inv(t2_incl[k])
recomputed[k] = r
t3 = t3_incl.get(k)
m2 = same(r, a)
m3 = t3 is not None and tuple(t3) == a
print(f" {k:36s} {fmt(a):20s} {fmt(r, 3):22s} {fmt(t3) if t3 else '-':20s} "
f"1/T2={'yes' if m2 else 'NO '} =T3={'yes' if m3 else 'no'}")
o = inv(t2_incl["Not obese vs obese (30 to <40)"])
print(f" note: obese upper limit 1/0.32 = {o[2]:.4f}, a rounding tie, so 3.13 is consistent")
n_match = sum(same(recomputed[k], abstract[k]) for k in abstract)
print(f" abstract figures equal to 1/Table 2: {n_match} of {len(abstract)}")
sev = "Not severely obese vs BMI ≥40"
sev_r = recomputed[sev]
print(f"\n2. Severe obesity: abstract {fmt(abstract[sev])}; 1/Table 2 = {fmt(sev_r)} "
f"(exact {sev_r[0]:.4f}, {sev_r[1]:.4f}–{sev_r[2]:.4f})")
print(f" Table 3 multivariable incl. hosp. (worsening outcome): {fmt(t3_incl[sev])} "
f"-> identical to abstract: {tuple(t3_incl[sev]) == abstract[sev]}")
print(f" ratio abstract/recomputed point estimate: {abstract[sev][0] / sev_r[0]:.3f} "
f"(+{100 * (abstract[sev][0] / sev_r[0] - 1):.0f}%)")
print(f" Table 2 OR that would invert to 4.19: 1/4.19 = {1 / 4.19:.3f} (printed: 0.36)")
print(f" any OR that rounds to 0.36 (0.355-0.365) inverts to {1 / 0.365:.2f}-{1 / 0.355:.2f}")
# internal consistency of the two source cells
for nm, (est, l, u) in [("Table 2 sev. 0.36 (0.18-0.72)", t2_incl[sev]),
("Table 3 sev. 4.19 (2.14-8.19)", t3_incl[sev])]:
print(f" {nm}: geometric CI midpoint {math.sqrt(l * u):.3f}, "
f"SE(log OR) {(math.log(u) - math.log(l)) / (2 * 1.96):.3f}")
# ---------------------------------------------------------------- 3. every candidate source
print("\n3. Every improvement-outcome estimate for severe obesity in the paper and appendix, inverted")
cands = {}
for k, t in list(t2_sev_row.items()) + list(s1_sev.items()):
cands[k] = inv(t)
print(f" {k:44s} {fmt(t):18s} -> {fmt(inv(t))} equals 4.19 (2.14-8.19): "
f"{same(r2(inv(t)), abstract[sev])}")
print(" Table 3 severely obese row (worsening outcome): "
+ "; ".join(fmt(t) for t in t3_sev_row))
# ---------------------------------------------------------------- 4. crude check from Table 1
a, b = t1["≥40"]
c, d = t1["<25"]
crude = (a / b) / (c / d)
se = math.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
lo, hi = crude * math.exp(-1.96 * se), crude * math.exp(1.96 * se)
print(f"\n4. Crude OR from Table 1 counts (severely obese 25/61 vs BMI<25 34/36): "
f"{crude:.3f} ({lo:.3f}–{hi:.3f}); Table 2 univariable 0.43 (0.22–0.84)")
print(f" inverted crude OR: {1 / crude:.2f} ({1 / hi:.2f}–{1 / lo:.2f})")
# ---------------------------------------------------------------- 5. obese vs severely obese gradient
ob_a = abstract["Not obese vs obese (30 to <40)"][0]
print(f"\n5. Severe-obese / obese ratio of the abstract's two BMI estimates: "
f"abstract {abstract[sev][0] / ob_a:.2f}; corrected {sev_r[0] / o[0]:.2f}")
print(f" lower CI bound: abstract 2.14 vs corrected {sev_r[1]:.2f}")
# ---------------------------------------------------------------- figures
BG, INK, MUTED = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK, "ytick.color": INK,
"axes.edgecolor": MUTED, "figure.facecolor": BG, "axes.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# ---- Figure 1: abstract vs Table 2 vs Table 3 text panels
fig = plt.figure(figsize=(7.2, 5.6), dpi=200) # measure text at the PNG resolution
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def panel(x, y, w, h, title, sub):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=1.2",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(x + 2, y + h - 3.2, title, fontsize=11.5, weight="bold", va="top")
ax.text(x + 2, y + h - 8.4, sub, fontsize=10, color=MUTED, va="top")
# Abstract panel
panel(3, 73, 94, 24, "Abstract, Findings",
"odds of improvement from 5 months to 1 year (PHOSP-COVID, n = 990)")
ax.text(5, 82.5, "\u201c\u2026 neither obese nor severely obese", fontsize=11, va="center")
# lay out the excerpt piece by piece, using measured text widths so spacing is natural
renderer = fig.canvas.get_renderer()
xpos = 5.0
spans = []
SP = 0.9 # width of one word space, in panel units (text extents exclude spaces)
for piece, kw, gap in [("(", {}, 0.15),
("1.82, 1.06\u20133.13", {"color": ORANGE, "weight": "bold"}, SP),
("and", {}, SP),
("4.19, 2.14\u20138.19", {"color": ORANGE, "weight": "bold"}, 0.15),
(", respectively) \u2026\u201d", {}, 0)]:
t = ax.text(xpos, 77, piece, fontsize=11, va="center", **kw)
bb = t.get_window_extent(renderer=renderer).transformed(ax.transData.inverted())
spans.append((bb.x0, bb.x1))
xpos = bb.x1 + gap
ob_x = (spans[1][0] + spans[1][1]) / 2 # centre of "1.82, ..."
sv_x0, sv_x1 = spans[3] # extent of "4.19, ..."
# Table 2 panel
panel(3, 6, 46, 42, "Table 2: improvement",
"primary outcome, n = 990\nmultivariable incl. hospitalisation")
ax.text(5, 28.5, "obese (30 to <40)", fontsize=10.5, color=MUTED)
ax.text(5, 24.5, "0.55 (0.32\u20130.94)", fontsize=10.5)
# upper limit 1/0.32 = 3.125 exactly, a rounding tie; shown as 3.13, as in the abstract
ax.text(5, 20.5, f"1/OR = {o[0]:.2f} ({o[1]:.2f}\u2013{round(o[2] + 1e-9, 2):.2f})", fontsize=10.5, color=BLUE, weight="bold")
ax.text(5, 15, "severely obese (\u226540)", fontsize=10.5, color=MUTED)
ax.text(5, 11, "0.36 (0.18\u20130.72)", fontsize=10.5)
ax.text(5, 7.4, f"1/OR = {fmt(sev_r)}", fontsize=10.5, color=BLUE, weight="bold")
# Table 3 panel
panel(51, 6, 46, 42, "Table 3: worsening",
"exploratory outcome, n = 1151\nmultivariable incl. hospitalisation")
ax.text(53, 28.5, "obese (30 to <40)", fontsize=10.5, color=MUTED)
ax.text(53, 24.5, "2.22 (1.35\u20133.64)", fontsize=10.5)
ax.text(53, 15, "severely obese (\u226540)", fontsize=10.5, color=MUTED)
ax.text(53, 11, "4.19 (2.14\u20138.19)", fontsize=10.5, weight="bold", color=ORANGE)
ax.text(53, 7.4, "same digits as the abstract", fontsize=10, color=MUTED, style="italic")
def arrow(p0, p1, color, ls="-", rad=0.0):
ax.add_patch(FancyArrowPatch(p0, p1, arrowstyle="-|>", mutation_scale=14, lw=2,
color=color, linestyle=ls, shrinkA=0, shrinkB=0,
connectionstyle=f"arc3,rad={rad}"))
arrow((ob_x, 74), (ob_x, 48.6), BLUE)
ax.text(ob_x + 1.5, 63, "matches", fontsize=10, color=BLUE, weight="bold")
ax.text(ob_x + 1.5, 59, "= 1/0.55", fontsize=10, color=BLUE)
sv_mid = (sv_x0 + sv_x1) / 2
arrow((sv_mid - 3, 74), (40, 48.6), MUTED, ls="--")
ax.text(sv_mid - 0.5, 63, "should be", fontsize=10, color=MUTED, weight="bold", ha="left")
ax.text(sv_mid - 0.5, 59, f"1/0.36 = {sev_r[0]:.2f}", fontsize=10, color=MUTED, ha="left")
arrow((sv_x1 - 1, 74), (72, 48.6), ORANGE, rad=-0.25)
ax.text(72, 63, "identical to Table 3,", fontsize=10, color=ORANGE, weight="bold")
ax.text(72, 59, "a different outcome", fontsize=10, color=ORANGE, weight="bold")
ax.legend(handles=[Line2D([], [], color=ORANGE, lw=0, marker="s", ms=8, label="as printed in the abstract"),
Line2D([], [], color=BLUE, lw=0, marker="s", ms=8,
label="recomputed from the paper's own Table 2")],
loc="lower center", bbox_to_anchor=(0.5, -0.005), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# ---- Figure 2: all seven abstract figures vs 1/Table 2, one log axis
keys = list(abstract)
fig, ax = plt.subplots(figsize=(7.2, 4.4))
fig.subplots_adjust(left=0.36, right=0.97, top=0.92, bottom=0.20)
for i, k in enumerate(keys):
y = len(keys) - 1 - i
a = abstract[k]
r = recomputed[k]
ax.plot([a[1], a[2]], [y + 0.15] * 2, color=ORANGE, lw=2, solid_capstyle="butt")
ax.plot(a[0], y + 0.15, "o", color=ORANGE, ms=8)
ax.plot([r[1], r[2]], [y - 0.15] * 2, color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(r[0], y - 0.15, "s", color=BLUE, ms=8)
if k == sev:
ax.annotate(f"abstract {a[0]:.2f}", (a[2], y + 0.15), xytext=(6, 0),
textcoords="offset points", va="center", fontsize=10, color=ORANGE)
ax.annotate(f"1/Table 2: {r[0]:.2f}", (r[2], y - 0.15), xytext=(6, 0),
textcoords="offset points", va="center", fontsize=10, color=BLUE)
ax.axvline(1, color=MUTED, lw=1)
ax.set_xscale("log")
ax.set_xlim(0.8, 20)
ticks = [1, 1.5, 2, 3, 5, 8]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.minorticks_off()
ax.set_yticks(range(len(keys)))
ax.set_yticklabels(list(reversed(keys)))
ax.set_ylim(-0.7, len(keys) - 0.3)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("odds ratio for improvement (log scale)")
ax.set_title("Six of seven abstract estimates are reciprocals of Table 2; severe obesity is not",
fontsize=11, loc="left", x=-0.52)
ax.legend(handles=[Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="abstract, as printed"),
Line2D([], [], color=BLUE, marker="s", ms=8, lw=2,
label="1/OR from Table 2 (multivariable incl. hosp.)")],
loc="upper center", bbox_to_anchor=(0.30, -0.15), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nfigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Abstract reverses the direction of the cerebellar IDH3A protein difference reported in Table 3B
Mol Psychiatry (2016) · doi:10.1038/mp.2015.217 · PMC5078854
Summary
The abstract states that protein expression of IDH3A in the cerebellum of patients with bipolar disorder (BD) "was higher than that of controls". The paper's own Table 3B gives the opposite: 0.804 ± 0.390 in BD against 1.227 ± 0.543 in controls (n = 15 per group). That is 0.423 units, or 34.5%, lower in BD, and the Results and Discussion both describe it as lower. Recomputing from the table's summary statistics reproduces its reported tests to rounding (ANOVA F(3, 56) = 5.73 against the printed 5.74, P = 0.0017; Dunnett P = 0.0172 for BD vs controls, printed 0.017), so the table and Results are internally consistent and only the abstract's direction word is wrong. The abstract is the most widely read and indexed part of the paper, and as printed it makes the protein result appear to contradict the lower IDH3A and IDH3B mRNA reported in the preceding sentence of the same abstract.
What the paper reports
Abstract:
Moreover, protein expression of IDH3A in the cerebellum from BD patients was higher than that of controls.
Abstract, preceding sentence (the mRNA result the protein finding is presented alongside):
Furthermore, gene expression of two subtypes (IDH3A and IDH3B) of isocitrate dehydrogenase (IDH) in the dorsolateral prefrontal cortex from BD patients was significantly lower than that of controls
Results, "Protein expression of IDH3A and IDH3B in the cerebellum and parietal cortex":
Results showed that protein levels of IDH3A in the cerebellum from BD (P=0.017), major depressive disorder (P=0.006) and schizophrenia (P=0.001) were significantly lower than in controls (Table 3B).
Table 3B, cerebellum row (columns: Controls n=15 | BD n=15 | MDD n=15 | Schizophrenia n=15 | One-way ANOVA):
IDH3A | 1.227±0.543 | 0.804±0.390* | 0.746±0.356** | 0.660±0.303** | F(3, 56)=5.74, P=0.002
Table 3B footer:
The data are the mean±s.d.*P<0.05, **P<0.01 (vs control, Dunnett's test).
Discussion, first paragraph:
Furthermore, protein levels of IDH3A in the cerebellar tissue from BD patients were lower than those in controls.
Discussion, second paragraph:
We did find reduced expression of IDH3A protein in the cerebellum in BD and other psychiatric disorders such as major depressive disorder and schizophrenia.
Taken all together, it is likely that decreased activity of IDH3A in the brain may play a crucial role in the pathogenesis of BD
The conclusions that draw on the finding. Abstract:
These findings suggest that abnormality in the metabolism of isocitrate by IDH3A in the mitochondria plays a key role in the pathogenesis of BD, supporting the mitochondrial dysfunction hypothesis of BD. Therefore, IDH3 in the citric acid cycle could potentially be a novel therapeutic target for BD.
Conclusions:
Our study provides evidence for abnormality in the metabolism of isocitrate by IDH3A in the pathogenesis of BD.
What the paper's own numbers show
Method. I took the group means and standard deviations printed in Table 3B (n = 15 per group, four groups) and Table 3A (dorsolateral prefrontal cortex, DLPFC, mRNA; controls n = 34, BD n = 35). For each Table 3B row I rebuilt the one-way ANOVA from the summary statistics (between-group and pooled within-group sums of squares, df 3 and 56). For the cerebellar IDH3A row I computed each patient group's difference from controls, the pooled-variance t statistic on 56 df, and the exact two-sided Dunnett P for three comparisons against a common control with equal n (correlation 0.5), by numerical integration rather than simulation. For all seven rows of Table 3A I ran Student and Welch two-sample t tests from the summary statistics, and found the range of P values each test can give when every printed mean and SD is allowed to be off by up to 0.0005 (their 3-decimal rounding). I also applied the Benjamini–Hochberg adjustment over the seven genes to the printed P values, to check the table's false discovery rate (FDR) column. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Direction of cerebellar IDH3A protein, BD vs controls | Abstract | "higher than that of controls" | lower: BD − controls = 0.804 − 1.227 = −0.423 (−34.5%; ratio 0.655) |
| Direction of cerebellar IDH3A protein, BD vs controls | Results; Discussion ¶1 and ¶2 | "significantly lower"; "lower"; "reduced" | lower (agrees with Table 3B) |
| Cerebellar IDH3A protein, controls vs BD | Table 3B | 1.227 ± 0.543 vs 0.804 ± 0.390* | as printed; pooled t(56) = −2.84 |
| One-way ANOVA, cerebellar IDH3A | Table 3B | F(3, 56) = 5.74, P = 0.002 | F(3, 56) = 5.73, P = 0.0017 (rounding of the printed means and SDs allows F = 5.70 to 5.77) |
| Dunnett, BD vs controls, cerebellar IDH3A | Results; Table 3B (*) | P = 0.017 | t = −2.840, P = 0.0172 |
| Dunnett, MDD vs controls, cerebellar IDH3A | Results; Table 3B (**) | P = 0.006 | t = −3.229, P = 0.0058 |
| Dunnett, schizophrenia vs controls, cerebellar IDH3A | Results; Table 3B (**) | P = 0.001 | t = −3.807, P = 0.0010 |
| One-way ANOVA, cerebellar IDH3B | Table 3B | F(3, 56) = 2.30, P = 0.087 | F = 2.30, P = 0.0869; BD − controls = −0.181 |
| One-way ANOVA, parietal IDH3A | Table 3B | F(3, 56) = 2.58, P = 0.063 | F = 2.58, P = 0.0628; BD − controls = −0.279 |
| One-way ANOVA, parietal IDH3B | Table 3B | F(3, 56) = 0.63, P = 0.597 | F = 0.63, P = 0.5959; BD − controls = −0.117 |
| DLPFC IDH3A mRNA, controls vs BD | Table 3A; Abstract ("lower") | 1.466 ± 0.832 vs 1.017 ± 0.416, P = 0.007 | BD − controls = −0.449 (−30.6%); Student P = 0.0058, Welch P = 0.0069 |
| DLPFC IDH3B mRNA, controls vs BD | Table 3A; Abstract ("lower") | 1.198 ± 0.420 vs 0.957 ± 0.287, P = 0.007 | BD − controls = −0.241 (−20.1%); Student P = 0.0069, Welch P = 0.0074 |
| DLPFC mRNA, other Table 3A genes (unpaired t-test) | Table 3A | ACO1 0.178; ACO2 0.487; IDH1 0.755; IDH2 0.279; IDH3G 0.422 | Student P: ACO1 0.175; ACO2 0.478; IDH1 0.758; IDH2 0.277; IDH3G 0.419. Each printed P lies inside the range that 3-decimal rounding of the means and SDs allows (for example ACO2 0.459 to 0.497) |
| FDR column, Table 3A | Table 3A | 0.414; 0.568; 0.755; 0.488; 0.026; 0.026; 0.568 (ACO1 … IDH3G) | Benjamini–Hochberg of the printed P values: 0.415; 0.568; 0.755; 0.488; 0.025; 0.025; 0.568 (printed P ± 0.0005 gives 0.023 to 0.026 for IDH3A and IDH3B) |
| Cerebellar IDH3A protein, BD vs the other patient groups | Table 3B | — | BD − MDD = +0.058; BD − schizophrenia = +0.144 (BD is higher only relative to these groups, not to controls) |
All four one-way ANOVAs and the three Dunnett P values in Table 3B are reproduced to rounding from the printed summary statistics. In Table 3A, the printed t-test P values for IDH3A and IDH3B match the Welch test (0.0069 and 0.0074); the Student test gives 0.0058 for IDH3A, which does not round to the printed 0.007 even allowing for rounding of the inputs. For the other five genes the recomputed P values differ from the printed ones by up to 0.009 (ACO2: 0.478 against 0.487), but each printed value lies within the range that the 3-decimal rounding of the printed means and SDs allows, so these differences say nothing either way. The FDR column matches a Benjamini–Hochberg adjustment of the printed P values to rounding. None of this bears on the error: the only statement that disagrees with the numbers is the direction word in the abstract.
Figure
Figure 1. The two places in the paper that describe the same quantity: cerebellar IDH3A protein, BD vs controls, Stanley post-mortem samples, n = 15 per group. Left: the abstract sentence, with its direction word in orange (reported). Right: the Results sentence (black) and the Table 3B means (orange, reported), with the difference recomputed from those means in blue (BD − controls = −0.423, 34.5% lower). The connector marks the conflict. The Discussion, not shown in a panel, agrees with the Results. All text is quoted verbatim; all numbers come from Table 3B or recompute.py.txt.
Figure 2. BD/control ratio of means for every IDH3A and IDH3B measurement in Tables 3A and 3B, on a log scale with 1 (no difference) as the reference line. Points and intervals (blue) are recomputed from the printed means and standard deviations; intervals are approximate 95% confidence intervals for the ratio by the delta method on the log scale and are shown only to convey precision, not as the paper's own analysis. The right-hand labels give the direction the abstract states for the three measurements it mentions; only the cerebellar IDH3A protein label (orange) conflicts with its point, which lies at 0.655 (95% CI 0.470 to 0.914). The figure shows that, read correctly, the protein result points the same way as the two mRNA results the abstract cites beside it.
Why it matters
- The error sits in the abstract. Abstracts are what indexing services, citation screens and most readers see. A reader of the abstract alone is told that IDH3A protein goes up in BD while IDH3A and IDH3B mRNA go down, which reads as mixed or contradictory evidence. The body of the paper reports a consistent deficit.
- It reverses the direction of a supporting result, not its size or significance. The size (−34.5%) and the significance (Dunnett P = 0.017) are correct in the body. The paper's headline findings (higher CSF isocitrate in BD; lower IDH3A/IDH3B mRNA in DLPFC) do not depend on this sentence, and the corrected direction fits the paper's argument better than the printed one.
- Direction matters for the paper's stated implications. The Discussion's inference that "decreased activity of IDH3A in the brain may play a crucial role", and the suggestion that IDH3 is "a novel therapeutic target", both rest on a deficit. An abstract that reports an excess points any reader who uses it, for example in a review or meta-analysis of post-mortem mitochondrial markers, the wrong way.
- Scope. The finding is not specific to BD: the same comparison shows lower cerebellar IDH3A in major depressive disorder and schizophrenia too (−39.2% and −46.2%), as the Results and Discussion say. That is a matter of interpretation and is not affected by the error.
Explanations considered
- A different tissue, measure or comparison. The abstract sentence names the same quantity as Table 3B and the Results: IDH3A protein, cerebellum, BD vs controls. No other cerebellar IDH3A measurement appears in the paper. Does not account for it.
- A comparison with the other patient groups rather than with controls. BD is higher than MDD (+0.058) and schizophrenia (+0.144) in Table 3B, but the abstract explicitly says "than that of controls", and these differences were not tested or reported. Does not account for it.
- An inverted ratio or normalization convention (for example, values expressed as control/BD). The table, the Results and the Discussion all read the same values as lower in BD, so the paper itself uses the convention under which BD is lower. Does not account for it.
- A text-conversion artifact in the archived copy. The publisher's version of record, the PMC page and the Europe PMC XML all carry the word "higher" in the abstract and the same Table 3B values. Does not account for it.
- A supplement that changes the result. The Supplementary Information describes the western blot and qPCR methods and gives no direction of effect; Supplementary Table S4 holds sample demographics only. Nothing in the supplements could make "higher" correct, since the main text and Table 3B give the direction plainly. Does not account for it.
- An error in the table rather than the abstract. The Results, both Discussion paragraphs and the significance markers all agree with the table, and the table's ANOVA and Dunnett P values are reproduced to rounding from its own means and SDs. The table is internally consistent; the abstract is the outlier.
- A published correction. None was found (see Status).
Status
As of 2026-10-23:
- Version of record checked. The publisher's article page (reached from doi.org) carries the same abstract sentence ("was higher than that of controls"), the same Results and Discussion wording ("significantly lower", "lower"), and the same Table 3B cerebellum IDH3A row and Dunnett footnote. The PMC and Europe PMC full texts match. The discrepancy is in the published article, not introduced by conversion.
- Supplements checked. Supplementary Information (DOC) and Supplementary Table S4 were examined; they report methods and demographics and do not resolve the conflict.
- Corrections. No correction, erratum or retraction was found: Crossref has no update relations (its relation field lists only an F1000 recommendation), and PMC and Europe PMC list no correction notices. Europe PMC links one "Comment in" item (PMID 29155801, Mol Psychiatry 2018;23:792–793, doi:10.1038/mp.2017.149), a letter rather than a correction; according to the version-of-record check it describes the paper as reporting reduced cerebellar IDH3A protein, which matches the body of the paper.
- PubPeer. The PubPeer API could not be queried (it required human verification), so PubPeer was not checked directly. Web searches found no PubPeer thread and no Retraction Watch coverage.
- Sources.
src/show.py PMC5078854(PMC archived text, tables and abstract);src/known.py 10.1038/mp.2015.217(Crossref, PMC and Europe PMC notices: none); https://doi.org/10.1038/mp.2015.217 and the publisher's Table 3B page; Europe PMC full-text XML; the Supplementary Information file.
Suggested correction
In the abstract, replace
Moreover, protein expression of IDH3A in the cerebellum from BD patients was higher than that of controls.
with
Moreover, protein expression of IDH3A in the cerebellum from BD patients was lower than that of controls.
No change is needed to Table 3B, the Results, the Discussion or the Conclusions, which already report the direction correctly. Optionally, the corrected sentence could add the effect and test (0.804 ± 0.390 vs 1.227 ± 0.543; Dunnett P = 0.017) and note that the reduction was also seen in major depressive disorder and schizophrenia, as the Results state.
Recomputation script (python recompute.py.txt)
# Recomputation for PMC5078854 (Mol Psychiatry 2016, doi:10.1038/mp.2015.217)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1/fig2
# (SVG + PNG at 200 dpi) in this file's own folder. Inputs are the summary
# statistics printed in Table 3A and Table 3B of the paper; nothing else is
# assumed.
# python recompute.py.txt --candidates DIR
# additionally renders the five candidate figure designs (A to E) into DIR, for
# choosing a design; they are not part of the write-up.
import os
import sys
import itertools
import numpy as np
from scipy import stats, integrate
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = sys.argv[sys.argv.index("--candidates") + 1] if "--candidates" in sys.argv else None
# ----------------------------------------------------------------------------
# 1. Inputs from the paper
# ----------------------------------------------------------------------------
GROUPS = ["Controls", "BD", "MDD", "Schizophrenia"]
N_PER_GROUP = 15
TABLE3B = { # (mean, s.d.) per group, n=15 each; reported one-way ANOVA (F, P)
"Cerebellum IDH3A": ([(1.227, 0.543), (0.804, 0.390), (0.746, 0.356), (0.660, 0.303)], (5.74, 0.002)),
"Cerebellum IDH3B": ([(1.020, 0.883), (0.839, 0.373), (0.605, 0.243), (0.610, 0.255)], (2.30, 0.087)),
"Parietal IDH3A": ([(0.951, 0.527), (0.672, 0.310), (0.648, 0.435), (0.564, 0.307)], (2.58, 0.063)),
"Parietal IDH3B": ([(0.738, 0.311), (0.621, 0.330), (0.760, 0.560), (0.602, 0.300)], (0.63, 0.597)),
}
DUNNETT_REPORTED = {"BD": 0.017, "MDD": 0.006, "Schizophrenia": 0.001} # Results text
TABLE3A = { # DLPFC mRNA: control mean, sd, n, BD mean, sd, n, reported unpaired t-test P
"DLPFC IDH3A mRNA": (1.466, 0.832, 34, 1.017, 0.416, 35, 0.007),
"DLPFC IDH3B mRNA": (1.198, 0.420, 34, 0.957, 0.287, 35, 0.007),
}
# All seven rows of Table 3A: (control mean, sd, BD mean, sd, reported t-test P, reported FDR P)
TABLE3A_ALL = {
"ACO1": (1.083, 0.205, 1.171, 0.315, 0.178, 0.414),
"ACO2": (0.922, 0.149, 0.897, 0.142, 0.487, 0.568),
"IDH1": (1.208, 0.337, 1.233, 0.333, 0.755, 0.755),
"IDH2": (1.316, 0.544, 1.186, 0.437, 0.279, 0.488),
"IDH3A": (1.466, 0.832, 1.017, 0.416, 0.007, 0.026),
"IDH3B": (1.198, 0.420, 0.957, 0.287, 0.007, 0.026),
"IDH3G": (1.084, 0.174, 1.049, 0.183, 0.422, 0.568),
}
def dunnett_two_sided_p(t, k, df):
"""Exact two-sided Dunnett P for k treatment-vs-control contrasts with equal n
(correlation 0.5). With U0, Ui iid N(0,1) and S^2 ~ chi2_df/df, the contrast
statistics are (Ui-U0)/(sqrt2*S); conditioning on U0=u and S=s makes them
independent. P(max|T| < c) = E[ prod_i (Phi(u + sqrt2*c*s) - Phi(u - sqrt2*c*s)) ]."""
c = abs(t)
def inner(s):
f = lambda u: stats.norm.pdf(u) * (stats.norm.cdf(u + np.sqrt(2) * c * s)
- stats.norm.cdf(u - np.sqrt(2) * c * s)) ** k
return integrate.quad(f, -10, 10, epsabs=1e-12)[0]
# density of S = sqrt(chi2_df / df)
dens = lambda s: stats.chi.pdf(s * np.sqrt(df), df) * np.sqrt(df)
lo, hi = stats.chi.ppf([1e-12, 1 - 1e-12], df) / np.sqrt(df)
cover = integrate.quad(lambda s: dens(s) * inner(s), lo, hi, epsabs=1e-12, limit=200)[0]
return 1 - cover
print("=" * 78)
print("Table 3B: one-way ANOVA from summary statistics (n=15 per group, df 3, 56)")
print("=" * 78)
RES = {}
for name, (rows, (F_rep, P_rep)) in TABLE3B.items():
m = np.array([a for a, b in rows]); s = np.array([b for a, b in rows])
K = len(m); N = N_PER_GROUP * K
gm = m.mean()
ssb = N_PER_GROUP * ((m - gm) ** 2).sum()
ssw = ((N_PER_GROUP - 1) * s ** 2).sum()
msw = ssw / (N - K)
F = (ssb / (K - 1)) / msw
P = stats.f.sf(F, K - 1, N - K)
RES[name] = dict(m=m, s=s, msw=msw, F=F, P=P)
print(f"{name:18s} F(3,56) = {F:.2f}, P = {P:.4f} [reported F(3, 56)={F_rep}, P={P_rep}]")
Fs = []
for dm in itertools.product([-5e-4, 5e-4], repeat=K):
for ds in itertools.product([-5e-4, 5e-4], repeat=K):
mm = m + np.array(dm); ss = s + np.array(ds)
Fs.append((N_PER_GROUP * ((mm - mm.mean()) ** 2).sum() / (K - 1))
/ (((N_PER_GROUP - 1) * ss ** 2).sum() / (N - K)))
print(f" F range under 3-decimal rounding of means and SDs: {min(Fs):.3f} to {max(Fs):.3f}")
for i in range(1, K):
d = m[i] - m[0]
t = d / np.sqrt(msw * 2 / N_PER_GROUP)
p_unadj = 2 * stats.t.sf(abs(t), N - K)
print(f" {GROUPS[i]:13s} - Controls = {d:+.3f} ({100 * (m[i] / m[0] - 1):+.1f}%; "
f"ratio {m[i] / m[0]:.3f}), pooled t(56) = {t:+.2f}, unadjusted two-sided P = {p_unadj:.4f}")
print()
print("Cerebellar IDH3A: Dunnett two-sided P (3 comparisons vs controls, df 56)")
r = RES["Cerebellum IDH3A"]
DUN = {}
for i, g in enumerate(GROUPS[1:], start=1):
t = (r["m"][i] - r["m"][0]) / np.sqrt(r["msw"] * 2 / N_PER_GROUP)
p = dunnett_two_sided_p(t, 3, 56)
DUN[g] = (t, p)
print(f" {g:13s} t = {t:+.3f}, Dunnett P = {p:.4f} [reported P={DUNNETT_REPORTED[g]}]")
bd_ctrl = r["m"][1] - r["m"][0]
bd_ratio = r["m"][1] / r["m"][0]
print()
print(f"Direction, cerebellar IDH3A protein, BD vs controls: BD - controls = {bd_ctrl:+.3f} "
f"({100 * (bd_ratio - 1):+.1f}%), ratio {bd_ratio:.3f} -> BD is "
f"{'LOWER' if bd_ctrl < 0 else 'HIGHER'} than controls")
print(f"For contrast, BD vs the other patient groups: BD - MDD = {r['m'][1] - r['m'][2]:+.3f}, "
f"BD - schizophrenia = {r['m'][1] - r['m'][3]:+.3f}")
print()
print("=" * 78)
print("Table 3A: DLPFC mRNA, controls (n=34) vs BD (n=35)")
print("=" * 78)
MRNA = {}
for name, (m1, s1, n1, m2, s2, n2, prep) in TABLE3A.items():
st = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2)
we = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2, equal_var=False)
MRNA[name] = (m1, s1, n1, m2, s2, n2)
print(f"{name:18s} BD - controls = {m2 - m1:+.3f} ({100 * (m2 / m1 - 1):+.1f}%; ratio {m2 / m1:.3f}); "
f"Student P = {st.pvalue:.4f}, Welch P = {we.pvalue:.4f} [reported P={prep}]")
print()
print("All seven Table 3A rows (n=34 vs 35). P recomputed from the printed means/SDs;")
print("'range' = smallest to largest P obtainable when each printed mean and SD may be")
print("off by up to 0.0005 from 3-decimal rounding (extremes over the 16 corners);")
print("'fits' = the range overlaps the interval the printed 3-decimal P stands for.")
def p_t(m1, s1, m2, s2, welch):
return stats.ttest_ind_from_stats(m1, s1, 34, m2, s2, 35, equal_var=not welch).pvalue
def p_range(m1, s1, m2, s2, welch):
ps = [p_t(m1 + a, s1 + b, m2 + c, s2 + d, welch)
for a, b, c, d in itertools.product([-5e-4, 5e-4], repeat=4)]
return min(ps), max(ps)
P_REP = []
for g, (m1, s1, m2, s2, prep, frep) in TABLE3A_ALL.items():
ps, pw = p_t(m1, s1, m2, s2, False), p_t(m1, s1, m2, s2, True)
(sl, sh), (wl, wh) = p_range(m1, s1, m2, s2, False), p_range(m1, s1, m2, s2, True)
fits = lambda lo, hi: "fits" if (lo < prep + 5e-4 and prep - 5e-4 <= hi) else "does NOT fit"
P_REP.append(prep)
print(f" {g:6s} Student P = {ps:.4f} (range {sl:.4f}-{sh:.4f}, {fits(sl, sh)}); "
f"Welch P = {pw:.4f} (range {wl:.4f}-{wh:.4f}, {fits(wl, wh)}) [reported P={prep}]")
def bh(p):
p = np.asarray(p, float); n = len(p); o = np.argsort(p)
adj = np.minimum.accumulate((p[o] * n / np.arange(1, n + 1))[::-1])[::-1]
out = np.empty(n); out[o] = np.minimum(adj, 1); return out
print()
print("Table 3A FDR column: Benjamini-Hochberg over the 7 genes")
fdr_rep = bh(P_REP)
fdr_lo = bh(np.maximum(np.array(P_REP) - 5e-4, 0)); fdr_hi = bh(np.array(P_REP) + 5e-4)
for i, (g, row) in enumerate(TABLE3A_ALL.items()):
print(f" {g:6s} BH of printed P = {fdr_rep[i]:.3f} (printed P +/- 0.0005 gives {fdr_lo[i]:.4f} to "
f"{fdr_hi[i]:.4f}) [reported FDR P={row[5]}]")
# BD/control ratios with approximate 95% CIs (delta method on the log ratio),
# used only for the ratio figure (candidate D / fig2).
print()
print("BD/control ratios, approximate 95% CI (delta method on log ratio, normal approx.)")
RATIOS = []
def ratio_ci(m1, s1, n1, m2, s2, n2):
lr = np.log(m2 / m1)
se = np.sqrt(s1 ** 2 / (n1 * m1 ** 2) + s2 ** 2 / (n2 * m2 ** 2))
return m2 / m1, np.exp(lr - 1.96 * se), np.exp(lr + 1.96 * se)
for name, (m1, s1, n1, m2, s2, n2) in MRNA.items():
RATIOS.append((name, *ratio_ci(m1, s1, n1, m2, s2, n2)))
for name in ["Cerebellum IDH3A", "Cerebellum IDH3B", "Parietal IDH3A", "Parietal IDH3B"]:
m = RES[name]["m"]; s = RES[name]["s"]
RATIOS.append((name + " protein", *ratio_ci(m[0], s[0], 15, m[1], s[1], 15)))
for name, est, lo, hi in RATIOS:
print(f" {name:26s} ratio {est:.3f} (95% CI {lo:.3f} to {hi:.3f})")
# ----------------------------------------------------------------------------
# 2. Figures
# ----------------------------------------------------------------------------
BG = "#fcfcfb"; INK = "#0b0b0b"; INK2 = "#52514e"; ORANGE = "#eb6834"; BLUE = "#2a78d6"
GRID = "#e4e3df"; PANEL = "#f3f2ef"; EDGE = "#c9c7c1"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
W = 7.2
ABSTRACT_Q = ("Moreover, protein expression of IDH3A in the cerebellum from BD patients "
"was higher than that of controls.")
RESULTS_Q = ("Results showed that protein levels of IDH3A in the cerebellum from BD (P=0.017), "
"major depressive disorder (P=0.006) and schizophrenia (P=0.001) were "
"significantly lower than in controls (Table 3B).")
DISC_Q = ("Furthermore, protein levels of IDH3A in the cerebellar tissue from BD patients "
"were lower than those in controls.")
TABLE_ROW = "IDH3A | 1.227±0.543 | 0.804±0.390* | 0.746±0.356** | 0.660±0.303** | F(3, 56)=5.74, P=0.002"
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def rich_text(ax, x, y, runs, width, size=10.5, line=1.45):
"""Draw word-wrapped text made of (text, color, weight) runs in axes coords,
starting at the top-left (x, y). Returns the y of the next line."""
fig = ax.figure
rend = fig.canvas.get_renderer()
inv = ax.transAxes.inverted()
probe = ax.text(0, 0, "Ag", fontsize=size)
bb = probe.get_window_extent(rend); probe.remove()
lh = (inv.transform((0, bb.height)) - inv.transform((0, 0)))[1] * line
sp_t = ax.text(0, 0, "a a", fontsize=size); sp_a = ax.text(0, 0, "aa", fontsize=size)
sw = sp_t.get_window_extent(rend).width - sp_a.get_window_extent(rend).width
sp_t.remove(); sp_a.remove()
sw = 0.8 * (inv.transform((sw, 0)) - inv.transform((0, 0)))[0]
cx, cy = x, y
for text, color, weight in runs:
for word in text.split(" "):
if word == "":
continue
t = ax.text(cx, cy, word, color=color, fontweight=weight, fontsize=size,
va="top", ha="left", transform=ax.transAxes)
wb = t.get_window_extent(rend)
ww = (inv.transform((wb.width, 0)) - inv.transform((0, 0)))[0]
if cx + ww > x + width and cx > x:
cx, cy = x, cy - lh
t.set_position((cx, cy))
cx += ww + sw
return cy - lh
def panel(ax, x, y, w, h, fc=PANEL, ec=EDGE, lw=1.0):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
transform=ax.transAxes, fc=fc, ec=ec, lw=lw))
def tag(ax, x, y, text, color):
ax.text(x, y, text, transform=ax.transAxes, fontsize=10, color="white", fontweight="bold",
va="center", ha="left",
bbox=dict(boxstyle="round,pad=0.3", fc=color, ec="none"))
# ---- Candidate A (fig1): two text panels joined by a connector -------------
def design_A(path):
fig = plt.figure(figsize=(W, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.03, 0.955, "Same quantity, opposite direction:\ncerebellar IDH3A protein, BD vs controls (n=15 per group)",
fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
# left panel: abstract
panel(ax, 0.03, 0.30, 0.38, 0.55)
ax.text(0.05, 0.81, "Abstract", fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
rich_text(ax, 0.05, 0.73, [("“Moreover, protein expression of IDH3A in the cerebellum from BD patients was", INK, "normal"),
("higher", ORANGE, "bold"), ("than that of controls.”", INK, "normal")], 0.34)
tag(ax, 0.05, 0.36, "reported: BD higher", ORANGE)
# right panel: results + table
panel(ax, 0.52, 0.30, 0.45, 0.55)
ax.text(0.54, 0.81, "Results and Table 3B", fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
yy = rich_text(ax, 0.54, 0.73, [("“protein levels of IDH3A in the cerebellum from BD (P=0.017) [...] were significantly", INK, "normal"),
("lower", INK, "bold"), ("than in controls (Table 3B).”", INK, "normal")], 0.41)
ax.text(0.54, yy - 0.015, "Table 3B, cerebellum IDH3A (mean ± s.d.):", fontsize=10, color=INK2,
transform=ax.transAxes, va="top")
ax.text(0.54, yy - 0.075, "controls 1.227 ± 0.543", fontsize=10, color=ORANGE, fontweight="normal",
transform=ax.transAxes, va="top")
ax.text(0.775, yy - 0.075, "BD 0.804 ± 0.390*", fontsize=10, color=ORANGE, fontweight="normal",
transform=ax.transAxes, va="top")
tag(ax, 0.54, 0.36, f"recomputed: BD − controls = {bd_ctrl:+.3f}", BLUE)
# connector
arr = FancyArrowPatch((0.415, 0.575), (0.515, 0.575), transform=ax.transAxes,
arrowstyle="<|-|>", mutation_scale=14, lw=2, color=INK2)
ax.add_patch(arr)
ax.text(0.465, 0.61, "conflict", ha="center", fontsize=10, color=INK2, transform=ax.transAxes,
fontweight="bold")
# footer
rich_text(ax, 0.03, 0.22, [("The Discussion also states the protein levels “were lower than those in controls”. "
f"Recomputed from Table 3B: BD is {100 * (1 - bd_ratio):.1f}% below controls, pooled t(56) = {DUN['BD'][0]:.2f}, Dunnett P = {DUN['BD'][1]:.3f} "
"(reported P=0.017). Only the abstract gives the direction as higher.", INK2, "normal")],
0.94, size=10)
save(fig, path)
# ---- Candidate B: statement map across the paper ---------------------------
def design_B(path):
rows = [
("Abstract", "“...was higher than that of controls.”", "higher", ORANGE),
("Results", "“...BD (P=0.017) ... significantly lower than in controls”", "lower", ORANGE),
("Table 3B", "controls 1.227±0.543 BD 0.804±0.390*", "lower", ORANGE),
("Discussion ¶1", "“...from BD patients were lower than those in controls.”", "lower", ORANGE),
("Discussion ¶2", "“We did find reduced expression of IDH3A protein ...”", "lower", ORANGE),
("Recomputed", f"BD − controls = {bd_ctrl:+.3f} ({100 * (bd_ratio - 1):+.1f}%), Dunnett P = {DUN['BD'][1]:.3f}", "lower", BLUE),
]
fig = plt.figure(figsize=(W, 4.3))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
ax.text(0.03, 0.955, "Direction of cerebellar IDH3A protein, BD vs controls, in each part of the paper",
fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
ax.text(0.03, 0.885, "orange = stated in the paper; blue = recomputed from Table 3B", fontsize=10,
color=INK2, transform=ax.transAxes, va="top")
top = 0.80; step = 0.118
for i, (loc, txt, d, col) in enumerate(rows):
y = top - i * step
if loc == "Abstract":
panel(ax, 0.02, y - 0.05, 0.96, 0.10, fc="#fbe9e1", ec=ORANGE, lw=1.5)
elif loc == "Recomputed":
ax.plot([0.03, 0.97], [y + 0.062, y + 0.062], color=EDGE, lw=1, transform=ax.transAxes)
ax.text(0.04, y, loc, fontsize=10.5, fontweight="bold", va="center", transform=ax.transAxes)
ax.text(0.20, y, txt, fontsize=10, va="center", transform=ax.transAxes, color=INK)
mk = "^" if d == "higher" else "v"
ax.plot([0.85], [y], marker=mk, ms=11, color=col, transform=ax.transAxes, ls="none")
ax.text(0.875, y, d, fontsize=10.5, va="center", transform=ax.transAxes, color=INK, fontweight="bold")
ax.text(0.04, 0.06, "The abstract is the only place that states the direction as higher.", fontsize=10,
color=INK2, transform=ax.transAxes)
save(fig, path)
# ---- Candidate C: group means ± s.d. with the abstract claim annotated ------
def design_C(path):
m = RES["Cerebellum IDH3A"]["m"]; s = RES["Cerebellum IDH3A"]["s"]
fig, ax = plt.subplots(figsize=(W, 4.4))
fig.subplots_adjust(left=0.11, right=0.97, top=0.86, bottom=0.12)
x = np.arange(4)
ax.errorbar(x, m, yerr=s, fmt="o", ms=9, color=ORANGE, ecolor=ORANGE, elinewidth=2, capsize=0,
label="Table 3B: mean ± s.d. (reported)", zorder=3)
ax.axhline(m[0], color=INK2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.text(3.45, m[0] + 0.03, "control mean", fontsize=10, color=INK2, ha="right")
# recomputed difference bracket
ax.annotate("", xy=(1.18, m[1]), xytext=(1.18, m[0]),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2, mutation_scale=14))
ax.text(1.25, (m[0] + m[1]) / 2, f"recomputed:\n{bd_ctrl:+.3f} ({100 * (bd_ratio - 1):+.1f}%)\n"
f"Dunnett P = {DUN['BD'][1]:.3f}", color=INK, fontsize=10, va="center")
ax.plot([], [], color=BLUE, lw=2, label="BD − controls, recomputed")
# abstract claim
ax.annotate("", xy=(0.82, m[0] + 0.42), xytext=(0.82, m[0] + 0.05),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, ls="--", mutation_scale=14))
ax.text(0.90, m[0] + 0.30, "abstract (reported):\nBD “higher than\nthat of controls”", color=INK,
fontsize=10, ha="left", va="center")
ax.set_xticks(x, ["Controls", "BD", "MDD", "Schizophrenia"])
ax.set_xlim(-0.5, 3.5); ax.set_ylim(0, 2.0)
ax.set_ylabel("IDH3A protein, cerebellum\n(relative expression)")
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right"]:
ax.spines[sp].set_visible(False)
ax.legend(loc="upper right", bbox_to_anchor=(1.0, 1.02), frameon=False, fontsize=10)
fig.suptitle("Cerebellar IDH3A protein is lower in BD than in controls (n=15 per group)",
x=0.03, ha="left", fontsize=11, fontweight="bold")
save(fig, path)
# ---- Candidate D (fig2): BD/control ratios on a log axis --------------------
def design_D(path):
order = ["DLPFC IDH3A mRNA", "DLPFC IDH3B mRNA", "Cerebellum IDH3A protein",
"Cerebellum IDH3B protein", "Parietal IDH3A protein", "Parietal IDH3B protein"]
abstract_note = {"DLPFC IDH3A mRNA": "abstract: lower", "DLPFC IDH3B mRNA": "abstract: lower",
"Cerebellum IDH3A protein": "abstract: higher"}
d = {n: (e, l, h) for n, e, l, h in RATIOS}
fig, ax = plt.subplots(figsize=(W, 4.3))
fig.subplots_adjust(left=0.30, right=0.76, top=0.80, bottom=0.14)
ys = np.arange(len(order))[::-1]
ax.axvline(1, color=INK2, lw=1)
for y, n in zip(ys, order):
e, l, h = d[n]
ax.plot([l, h], [y, y], color=BLUE, lw=2, solid_capstyle="round")
ax.plot([e], [y], "o", ms=9, color=BLUE, mec=BG, mew=1.5, zorder=3)
if n in abstract_note:
note = abstract_note[n]
ax.text(1.62, y, note, va="center", fontsize=10,
color=ORANGE if "higher" in note else INK2, fontweight="bold" if "higher" in note else "normal",
transform=ax.get_yaxis_transform() if False else ax.transData, clip_on=False)
ax.set_xscale("log")
ticks = [0.4, 0.5, 0.6, 0.8, 1.0, 1.25, 1.5]
ax.set_xticks(ticks, [f"{t:g}" for t in ticks]); ax.minorticks_off()
ax.set_xlim(0.38, 1.55)
ax.set_yticks(ys, order)
ax.set_xlabel("BD / control ratio of means (log scale); 1 = no difference")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for sp in ["top", "right", "left"]:
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.text(0.40, len(order) - 0.35, "← lower in BD", fontsize=10, color=INK2)
ax.text(1.04, len(order) - 0.35, "higher in BD →", fontsize=10, color=INK2)
ax.plot([], [], "o-", color=BLUE, lw=2, ms=8, label="recomputed from\nTables 3A/3B\n(approx. 95% CI)")
ax.legend(loc="lower left", bbox_to_anchor=(1.0, -0.02), frameon=False, fontsize=10,
handlelength=1.6, borderaxespad=0)
fig.suptitle("BD/control ratios for IDH3A and IDH3B, Tables 3A and 3B:\nall point estimates below 1", x=0.03, ha="left", y=0.985,
fontsize=11, fontweight="bold")
save(fig, path)
# ---- Candidate E: printed vs supported sentence + number line --------------
def design_E(path):
m = RES["Cerebellum IDH3A"]["m"]; s = RES["Cerebellum IDH3A"]["s"]
fig = plt.figure(figsize=(W, 4.3))
ax = fig.add_axes([0, 0.50, 1, 0.50]); ax.axis("off")
ax.text(0.03, 0.92, "The abstract sentence as printed, and as Table 3B supports it",
fontsize=11, fontweight="bold", transform=ax.transAxes, va="top")
ax.text(0.03, 0.66, "Printed", fontsize=10.5, color=INK2, transform=ax.transAxes, va="top")
rich_text(ax, 0.17, 0.66, [("“protein expression of IDH3A in the cerebellum from BD patients was", INK, "normal"),
("higher", ORANGE, "bold"), ("than that of controls.”", INK, "normal")], 0.80)
ax.text(0.03, 0.28, "Supported", fontsize=10.5, color=INK2, transform=ax.transAxes, va="top")
rich_text(ax, 0.17, 0.28, [("protein expression of IDH3A in the cerebellum from BD patients was", INK, "normal"),
("lower", BLUE, "bold"), ("than that of controls.", INK, "normal")], 0.80)
bx = fig.add_axes([0.17, 0.20, 0.55, 0.25])
for y, (mm, ss, lab) in zip([2, 0], [(m[0], s[0], "Controls"), (m[1], s[1], "BD")]):
bx.plot([mm - ss, mm + ss], [y, y], color=ORANGE, lw=2)
bx.plot([mm], [y], "o", ms=9, color=ORANGE)
bx.text(1.02, y, f"{lab} {mm:.3f} ± {ss:.3f}", va="center", fontsize=10,
transform=bx.get_yaxis_transform(), clip_on=False)
bx.annotate("", xy=(m[1], 1), xytext=(m[0], 1),
arrowprops=dict(arrowstyle="-|>", color=BLUE, lw=2, mutation_scale=14))
bx.text(m[0] + 0.04, 1, f"recomputed {bd_ctrl:+.3f} ({100 * (bd_ratio - 1):+.1f}%)", va="center",
fontsize=10, color=INK)
bx.set_ylim(-0.6, 2.6); bx.set_xlim(0, 2.0); bx.set_yticks([])
bx.set_xlabel("Table 3B cerebellar IDH3A protein,\nmean ± s.d. (reported), n=15 per group", fontsize=10)
bx.grid(axis="x", color=GRID, lw=0.8); bx.set_axisbelow(True)
for sp in ["top", "right", "left"]:
bx.spines[sp].set_visible(False)
save(fig, path)
design_A(os.path.join(HERE, "fig1"))
design_D(os.path.join(HERE, "fig2"))
print()
print("Figures written: fig1.svg/png (design A), fig2.svg/png (design D)")
if CAND:
os.makedirs(CAND, exist_ok=True)
for nm, fn in zip("ABCDE", [design_A, design_B, design_C, design_D, design_E]):
fn(os.path.join(CAND, f"candidate_{nm}"))
print(f"Candidate designs written to {CAND}: candidate_A..E.svg/png")
Re-positive rate's 95% confidence interval excludes its own point estimate in abstract and Results
Nat Commun (2020) · doi:10.1038/s41467-020-19802-w · PMC7679396
Summary
The paper reports that 107 of 34,424 recovered COVID-19 patients tested positive again in Wuhan's city-wide screening, a re-positive rate of 0.31%, with a 95% confidence interval of 0.423–0.574%. The interval lies entirely above the point estimate, which none of the standard two-sided intervals (exact, score, Wald, Agresti–Coull, Jeffreys) does for 107/34,424. Applied to 107/34,424, the paper's stated method (Clopper–Pearson exact) gives 0.255–0.375%. The same method reproduces exactly all 27 other intervals in Tables 1–2 and the antibody text, and also the two single- and mixed-testing intervals once their unprinted counts are inferred. The point estimate and the paper's conclusions are unaffected, but the wrong interval is printed in the abstract. Anyone who extracts it, for example for a meta-analysis of re-positivity after recovery, gets a range whose lower bound lies above the correct upper bound, centred near 0.50% instead of 0.31%.
What the paper reports
Abstract:
107 of 34,424 previously recovered COVID-19 patients tested positive again (re-positive rate 0.31%, 95% CI 0.423–0.574%).
Results, second paragraph (screening results):
Of the 34,424 participants with a history of COVID-19, 107 tested positive again, giving a repositive rate of 0.310% (95% CI 0.423–0.574%).
Results, first paragraph (the denominators):
Of the 9899,828 participants, 9,865,404 had no previous diagnosis of COVID-19, and 34,424 were recovered COVID-19 patients.
Methods, "Statistical analysis" (the stated method, which covers both the asymptomatic and the re-positive rates):
Detection rate of asymptomatic positive or repositive cases was calculated by dividing the number of individuals with a positive result of nucleic acid testing by the number of participants tested. Because of extremely low detection rates, we calculated 95% confidence intervals of estimated proportions using Pearson–Klopper exact method, implemented through R package “binom” version 1.1-1…
Discussion, third paragraph (the statement that rests on this estimate):
There was a low repositive rate in recovered COVID-19 patients in Wuhan.
What the paper's own numbers show
Method. I took the count (107) and denominator (34,424) printed in the abstract and the Results and computed the proportion and its two-sided 95% Clopper–Pearson interval from beta quantiles. This is the interval that R's binom package returns for method = "exact". For comparison I also computed the Wilson, Wald, Agresti–Coull, Jeffreys and Poisson-exact intervals. To confirm that I was applying the paper's method the way the authors did, I applied the same calculation to every other interval in the paper whose count and denominator are printed: the 22 rows of Table 1 (asymptomatic positives per 10,000 by sex, age and district), the four rows of Table 2, the IgG-positive proportion in the Results (antibody results, % of 300) and the asymptomatic rate in the second paragraph of the Results. The paper has two further intervals, the asymptomatic rates for single and mixed testing in the fourth paragraph of the Results, for which only the share of samples (76.7% and 23.3%) is printed, not the counts. For these I took the denominators as 76.7% and 23.3% of 9,899,828 and searched for the counts whose exact interval matches the printed one. Finally I searched for the count, denominator and confidence level that would produce the published interval, and compared the counts in the publisher's Source Data workbook with the printed Tables 1 and 2. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Re-positive rate, 107/34,424 | Abstract; Results para 2 | 0.31% (abstract), 0.310% (Results) | 0.3108%, which rounds to 0.311%; the Results' 0.310% is the truncated value, a negligible difference |
| 95% CI, Clopper–Pearson (stated method) | Abstract; Results para 2 | 0.423–0.574% | 0.255–0.375% (0.2548–0.3755%) |
| 95% CI, other standard methods | — | — | Wilson 0.257–0.375%; Wald 0.252–0.370%; Agresti–Coull 0.257–0.376%; Jeffreys 0.256–0.374%; Poisson exact 0.255–0.376% |
| Does the reported CI contain the point estimate? | — | — | No; 0.311% is below the reported lower bound of 0.423% |
| Gap between the intervals | — | — | Reported lower bound 0.423% exceeds the recomputed upper bound 0.375% by 0.048 points; the intervals do not overlap |
| Midpoint of the interval | — | 0.4985% (midpoint of 0.423–0.574%) | 0.311%; the reported midpoint is 1.60 times the point estimate |
| Width / implied standard error | — | 0.151 points; implied SE 0.0385 points | 0.121 points; binomial SE 0.0300 points |
| Reported bound ÷ recomputed bound | — | — | lower 1.66, upper 1.53 |
| Count x out of 34,424 whose exact CI rounds to 0.423–0.574% | reverse search | — | only x = 170 (0.4225–0.5737%; rate 0.494%) |
| Denominator n for which 107/n gives 0.423–0.574% | reverse search, n = 10,000–60,000 | — | none; the closest (n = 22,514) gives 0.390–0.574% |
| Other confidence levels for 107/34,424 | reverse search | — | 90%: 0.263–0.365%; 99%: 0.239–0.397%; 99.9%: 0.221–0.422%. Even the 99.9% upper bound is below 0.423% |
| Table 1, 22 rates per 10,000 with CIs | Table 1 | e.g. Total 0.303 (0.270–0.339); Male 0.256 (0.214–0.303); Wuchang 0.807 (0.633–1.015); East Lake Scenic 0.000 (0.000–0.430) | all 22 reproduce exactly with Clopper–Pearson |
| Antibody proportions, % of 300 | Table 2; Results para 9 | 53.7 (47.8–59.4); 36.7 (31.2–42.4); 9.7 (6.6–13.6); 0.0 (0.0–1.2); IgG+ 63.3 (57.6–68.8) | all 5 reproduce exactly from the printed counts (161, 110, 29, 0; IgG+ 190) |
| Asymptomatic rates, single and mixed testing (counts not printed) | Results para 4 | single 0.321 (0.282–0.364)/10,000; mixed 0.243 (0.183–0.315)/10,000 | with denominators of about 7,593,168 (76.7% of 9,899,828) and 2,306,660 (23.3%), only the counts 244 and 56 reproduce the printed rate and interval, and 244 + 56 = 300, the total of asymptomatic positives. Both intervals are consistent with Clopper–Pearson |
| Asymptomatic rate, denominator 9,865,404 | Results para 2 ("participants without a history of COVID-19") | 0.303 (0.270–0.339)/10,000 | 0.304 (0.271–0.341)/10,000; the printed figures match 300/9,899,828 (all participants), as in Table 1 |
| Denominators | Results para 1; Table 1; Table 2 | 9,865,404 + 34,424; total 9,899,828 | sum 9,899,828; sex, age and district totals also sum to 9,899,828 and district cases to 300; printed antibody counts sum to 300, with 161 + 29 = 190 IgG-positive |
| Antibody counts in the publisher's Source Data workbook | Source Data, sheet "Raw data from Table 2" | printed Table 2: 161, 110, 29, 0; Results: 190 IgG-positive | the sheet gives 157, 110, 28, 5 (also summing to 300), hence 157 + 28 = 185 IgG-positive, 61.7% (55.9–67.2%) instead of 63.3% (57.6–68.8%). The sheet "Raw data from Table 1" matches the printed Table 1 in all 21 rows |
The stated method reproduces every other interval in Tables 1–2 and the antibody text to the last printed digit, and the two single/mixed intervals are reproduced by inferred counts that add up to the paper's 300 asymptomatic positives, so the method itself is not in doubt. The count, the denominator and the point estimate are mutually consistent. Only the re-positive interval cannot be obtained from the printed inputs, under the stated method or any other standard one.
There is one minor, separate observation. The Results attach the asymptomatic rate to the 9,865,404 participants without a COVID-19 history, but the printed rate and interval are those for all 9,899,828 participants (Table 1, Total). The difference is in the third decimal and changes nothing. A second separate observation concerns the antibody counts: the publisher's Source Data workbook gives different Table 2 counts from the printed table (row above). This does not bear on the re-positive interval.
Figure
Figure 1. The re-positive rate among recovered patients on one percentage axis, starting at zero. The orange row is the reported point estimate and 95% CI (identical in the Abstract and the Results). The blue rows are intervals recomputed from the paper's own count and denominator, 107/34,424: Clopper–Pearson (the method the Methods name), Wilson score and Wald. The dashed vertical line marks the point estimate, 0.311%. Every recomputed interval contains it and spans about 0.25–0.38%. The reported interval, 0.423–0.574%, lies wholly to the right of its own point estimate and does not overlap any of the recomputed intervals.
Figure 2. Each reported confidence bound in the paper, divided by the Clopper–Pearson bound recomputed from that row's count and denominator and rounded to the same number of decimals (log scale; 1 means identical). The counts are printed in the paper except for the single- and mixed-testing rates, where they are inferred (244 and 56; see the table). Within each row, hollow circles (lower bounds) are drawn just above the row and filled diamonds (upper bounds) just below it, so both stay visible when their ratios are equal; lower bounds of zero are omitted. All 22 Table 1 intervals, all 5 antibody intervals (from the printed Table 2 counts) and both single/mixed intervals give a ratio of exactly 1.000. The Results-text asymptomatic rate, checked against the 9,865,404 denominator the text names, gives 0.996 and 0.994 (it matches the 9,899,828 total exactly). The re-positive interval (orange) gives 1.66 for the lower bound and 1.53 for the upper bound. It is the only interval that the stated method does not reproduce with any of the paper's denominators.
Why it matters
- The interval is in the abstract. The abstract is the part of the paper most often read, indexed and quoted. As printed, it puts the plausible range of re-positivity at 0.42–0.57%. The correct range is 0.26–0.38%, and the two do not overlap.
- Evidence synthesis. Systematic reviews and meta-analyses of SARS-CoV-2 re-positivity or re-detection after recovery commonly extract a proportion with its CI, or back-calculate a standard error from the CI. Using the published interval would place this study's estimate near 0.50% (the interval midpoint) instead of 0.31%, with a standard error about 1.28 times too large (0.0385 vs 0.0300 percentage points). The study's large denominator would still give it notable weight, so the shifted centre (about 0.50%) would pull a pooled estimate upward, while its inverse-variance weight would be about 0.61 of the correct weight, that is, understated by about 40%.
- Internal coherence. A reader who sees an interval that excludes its own estimate cannot tell from the paper which number is wrong: the count, the denominator, the point estimate or the interval. The arithmetic shows that the count, denominator and point estimate agree with one another and that the interval is the outlier.
- What is not affected. The headline result, the asymptomatic detection rate of 0.303 per 10,000 (0.270–0.339), reproduces exactly, as do all of Table 1 and, with inferred counts, the single- and mixed-testing rates. The Table 2 intervals and the 63.3% IgG-positive figure reproduce exactly from the printed counts; the Source Data workbook's different counts (157, 110, 28, 5) would give 185 IgG-positive (61.7%), which does not change the paper's reading that most asymptomatic positives had antibodies. The Discussion's statement that the re-positive rate was low holds with the correct interval, whose upper bound (0.375%) is in fact lower than the published lower bound. The conclusion that post-lockdown prevalence in Wuhan was very low is unchanged.
Explanations considered
- A different interval method. This does not account for the numbers. Clopper–Pearson, Wilson, Wald, Agresti–Coull, Jeffreys and Poisson-exact intervals for 107/34,424 all lie within 0.252–0.376%. For these data the standard two-sided intervals (exact, score, Wald, Agresti–Coull, Jeffreys, Poisson exact) all contain the sample proportion, so none of the standard methods gives an interval wholly above 0.311%.
- A different confidence level. This does not account for the numbers. Changing the level changes the width, not the centre: the 90%, 99% and 99.9% exact intervals are 0.263–0.365%, 0.239–0.397% and 0.221–0.422%, all of which contain 0.311%.
- A different denominator. This does not account for the numbers. No denominator between 10,000 and 60,000 gives 107/n an exact interval that rounds to 0.423–0.574%. A smaller denominator would also change the point estimate, which the paper prints consistently as 0.31%/0.310%.
- A different count. Arithmetically, the exact 95% interval for 170/34,424 (0.494%) is 0.4225–0.5737%, which rounds to the published bounds, and no other count does. The number 170 does not appear in the archived text, and the published material does not show where the interval came from. The match is an arithmetic observation only.
- A conversion or typesetting error. This is excluded. The publisher's HTML (published 20 November 2020), the PMC archive and the Europe PMC full-text XML all carry the same interval in both the Abstract and the Results. The medRxiv preprint (10.1101/2020.06.29.20142554, version 1, 30 June 2020) already reported a relapse rate of 0.310% with the same 95% CI of 0.423–0.574%, so the interval was present before peer review.
- A different definition of "re-positive" or a subgroup. This does not account for the numbers. The Methods define re-positive cases as recovered patients who tested positive again in the screening. The abstract and Results tie the interval directly to "107 of 34,424", and the paper reports no other re-positive count or subgroup.
Status
- Version of record. The publisher's page (nature.com/articles/s41467-020-19802-w, published 20 November 2020) has the same Abstract and Results text as the PMC archive. Its change history has one entry, dated 17 December 2020: the Peer Review file had been omitted and was added. That update did not change the interval. Every quote above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. All four electronic supplementary files were checked. The Supplementary Information covers protocols, case definitions and the Ct values of asymptomatic cases, and has nothing on re-positive cases. The Peer Review File does not mention the re-positive interval. The Source Data workbook covers Tables 1–2 and Figures 1–3 but has no counts of recovered or re-positive patients. Its Table 1 sheet matches the printed Table 1, but its Table 2 sheet gives antibody counts of 157 (IgM−/IgG+), 110 (IgM−/IgG−), 28 (IgM+/IgG+) and 5 (IgM+/IgG−), which differ from the printed 161, 110, 29 and 0 and imply 185 IgG-positive cases rather than the 190 in the Results, although both sets sum to 300. The re-positive interval is unaffected. The Reporting Summary is a standard form. None of the supplements resolves the discrepancy.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI (its relation field links only the medRxiv preprint), and PMC and Europe PMC list no correction notices. The publisher's page shows no correction notice. A reader comment on the publisher's page dated 29 December 2020 pointed out that the point estimate lies outside the confidence interval; commenting is now closed, and no reply or correction followed in the record. The PubPeer API refused automated access (HTTP 403), and a web search found no PubPeer thread for the article.
- Sources. PMC article PMC7679396; publisher page https://www.nature.com/articles/s41467-020-19802-w and its electronic supplementary material; Europe PMC full-text XML; medRxiv 10.1101/2020.06.29.20142554 v1; Crossref; Europe PMC; PubPeer.
Suggested correction
- Replace the re-positive interval in the Abstract and in the second paragraph of the Results. If 107 of 34,424 is correct, the text would read "re-positive rate 0.31%, 95% CI 0.25–0.38%" in the Abstract and "repositive rate of 0.311% (95% CI 0.255–0.375%)" in the Results, using the Clopper–Pearson method stated in the Methods. (107/34,424 = 0.3108%; the printed 0.310% is the truncated value, and 0.311% is the rounded one.) If the count or denominator is the figure in error instead, correct it and recompute both the rate and the interval.
- Minor: in the second paragraph of the Results, either give the asymptomatic rate for the 9,865,404 participants without a COVID-19 history (0.304 per 10,000, 95% CI 0.271–0.341), or state that 0.303 (0.270–0.339) refers to all 9,899,828 participants, as in Table 1. Separately, "9899,828" in the first paragraph of the Results should read "9,899,828".
- Minor and separate: the antibody counts in the Source Data workbook (157, 110, 28, 5) differ from the printed Table 2 (161, 110, 29, 0) and from the 190 IgG-positive cases in the Results. Whichever set is in error should be corrected so that the Source Data and the printed table agree; if the Source Data counts are the correct ones, Table 2 and the 63.3% (57.6–68.8%) IgG-positive figure in the Results would change, the latter to 61.7% (55.9–67.2%).
Recomputation script (python recompute.py.txt)
"""Recompute the confidence intervals of PMC7679396
(Nat Commun 2020, doi:10.1038/s41467-020-19802-w).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract and Results paragraph 2: 107 re-positive of 34,424 recovered patients,
"0.31%" / "0.310% (95% CI 0.423-0.574%)"; 300 asymptomatic positives,
"0.303 (95% CI 0.270-0.339)/10,000"
Results paragraph 1: 9,899,828 tested = 9,865,404 without and 34,424 with prior COVID-19
Table 1: counts, cases and detection rate per 10,000 (95% CI) by sex, age and district
Table 2 and Results paragraph 9: antibody results in the 300 asymptomatic cases
Results paragraph 4: single and mixed testing for 76.7% and 23.3% of samples, asymptomatic
rates "0.321 (95% CI 0.282-0.364)/10,000 and 0.243 (95% CI 0.183-0.315)/10,000".
The counts behind these two rates are not printed; they are inferred below.
Methods: 95% CIs by the "Pearson-Klopper exact method" (R package binom), i.e. the
Clopper-Pearson interval, computed here with the beta quantile function.
"""
import math
import os
import numpy as np
from scipy.stats import beta, chi2, norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# ---------------------------------------------------------------- inputs (from the paper)
X_RE, N_RE = 107, 34424 # re-positive cases / recovered patients
REP_RE_PT_ABS, REP_RE_PT_RES = 0.31, 0.310 # % (abstract, Results)
REP_RE_CI = (0.423, 0.574) # % (abstract and Results)
N_TOTAL, N_NOHIST, N_HIST = 9899828, 9865404, 34424
X_ASYM = 300
REP_ASYM = (0.303, 0.270, 0.339) # per 10,000 (abstract, Results, Table 1 Total)
# Table 1: label, participants, asymptomatic positives, reported rate (lo, hi) per 10,000
TABLE1 = [
("Total", 9899828, 300, (0.303, 0.270, 0.339)),
("Male", 5162960, 132, (0.256, 0.214, 0.303)),
("Female", 4736868, 168, (0.355, 0.303, 0.413)),
("Age <=17", 969014, 12, (0.124, 0.064, 0.216)),
("Age 18-44", 4448230, 104, (0.234, 0.191, 0.283)),
("Age 45-59", 2492943, 96, (0.385, 0.312, 0.470)),
("Age >=60", 1989641, 88, (0.442, 0.355, 0.545)),
("Wuchang", 904636, 73, (0.807, 0.633, 1.015)),
("Qingshan", 414312, 23, (0.555, 0.352, 0.833)),
("Qiaokou", 583440, 32, (0.548, 0.375, 0.774)),
("Hanyang", 717429, 29, (0.404, 0.271, 0.581)),
("Jianghan", 524224, 19, (0.362, 0.218, 0.566)),
("Hongshan", 1103079, 35, (0.317, 0.221, 0.441)),
("East Lake High-tech", 782987, 19, (0.243, 0.146, 0.379)),
("Jiangan", 800440, 19, (0.237, 0.143, 0.371)),
("Caidian", 503595, 11, (0.218, 0.109, 0.391)),
("Jiangxia", 671248, 14, (0.209, 0.114, 0.350)),
("Huangpi", 979920, 14, (0.143, 0.078, 0.240)),
("Hannan", 417022, 4, (0.096, 0.026, 0.246)),
("Dongxihu", 777204, 5, (0.064, 0.021, 0.150)),
("Xinzhou", 634408, 3, (0.047, 0.010, 0.138)),
("East Lake Scenic", 85884, 0, (0.000, 0.000, 0.430)),
]
# Table 2 and Results paragraph 9: label, count of 300, reported % (lo, hi)
TABLE2 = [
("IgM- IgG+", 161, (53.7, 47.8, 59.4)),
("IgM- IgG-", 110, (36.7, 31.2, 42.4)),
("IgM+ IgG+", 29, (9.7, 6.6, 13.6)),
("IgM+ IgG-", 0, (0.0, 0.0, 1.2)),
("IgG+ (text)", 190, (63.3, 57.6, 68.8)),
]
# Results paragraph 4: label, share of samples, reported rate (lo, hi) per 10,000
SINGLE_MIXED = [
("Single testing", 0.767, (0.321, 0.282, 0.364)),
("Mixed testing", 0.233, (0.243, 0.183, 0.315)),
]
# ---------------------------------------------------------------- interval methods
def clopper_pearson(x, n, a=0.05):
lo = 0.0 if x == 0 else beta.ppf(a / 2, x, n - x + 1)
hi = 1.0 if x == n else beta.ppf(1 - a / 2, x + 1, n - x)
return lo, hi
def wilson(x, n, z=Z):
p = x / n
d = 1 + z * z / n
c = (p + z * z / (2 * n)) / d
h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
return c - h, c + h
def wald(x, n, z=Z):
p = x / n
se = math.sqrt(p * (1 - p) / n)
return p - z * se, p + z * se
def agresti_coull(x, n, z=Z):
nt = n + z * z
pt = (x + z * z / 2) / nt
se = math.sqrt(pt * (1 - pt) / nt)
return pt - z * se, pt + z * se
def jeffreys(x, n, a=0.05):
return beta.ppf(a / 2, x + 0.5, n - x + 0.5), beta.ppf(1 - a / 2, x + 0.5, n - x + 0.5)
def poisson_exact(x, n, a=0.05):
lo = 0.0 if x == 0 else chi2.ppf(a / 2, 2 * x) / 2
hi = chi2.ppf(1 - a / 2, 2 * x + 2) / 2
return lo / n, hi / n
def pct(v):
return 100.0 * v
# ---------------------------------------------------------------- 1. the re-positive rate
print("=" * 78)
print("1. Re-positive rate among recovered patients (Abstract; Results paragraph 2)")
print("=" * 78)
p_re = X_RE / N_RE
print(f"Point estimate 107/34,424 = {pct(p_re):.4f}% (reported {REP_RE_PT_ABS}% abstract, "
f"{REP_RE_PT_RES:.3f}% Results)")
print(f"Reported 95% CI: {REP_RE_CI[0]:.3f}-{REP_RE_CI[1]:.3f}%")
print(f"Does the reported CI contain the point estimate? {REP_RE_CI[0] <= pct(p_re) <= REP_RE_CI[1]}")
methods = [
("Clopper-Pearson exact (stated method)", clopper_pearson),
("Wilson score", wilson),
("Wald (normal approximation)", wald),
("Agresti-Coull", agresti_coull),
("Jeffreys", jeffreys),
("Poisson exact (count 107 as Poisson)", poisson_exact),
]
RE_CI = {}
for name, f in methods:
lo, hi = f(X_RE, N_RE)
RE_CI[name] = (lo, hi)
print(f" {name:40s} {pct(lo):.4f}-{pct(hi):.4f}% -> {pct(lo):.3f}-{pct(hi):.3f}%")
cp_lo, cp_hi = RE_CI["Clopper-Pearson exact (stated method)"]
print(f"Width: reported {REP_RE_CI[1] - REP_RE_CI[0]:.3f} points; Clopper-Pearson "
f"{pct(cp_hi - cp_lo):.3f} points")
print(f"Reported lower bound {REP_RE_CI[0]}% minus recomputed upper bound {pct(cp_hi):.3f}% = "
f"{REP_RE_CI[0] - pct(cp_hi):.3f} points (no overlap)")
mid = (REP_RE_CI[0] + REP_RE_CI[1]) / 2
print(f"Midpoint of reported CI: {mid:.4f}%; ratio to point estimate {mid / pct(p_re):.2f}")
print(f"Reported bounds / Clopper-Pearson bounds: lower {REP_RE_CI[0] / pct(cp_lo):.2f}, "
f"upper {REP_RE_CI[1] / pct(cp_hi):.2f}")
se_from_reported = (REP_RE_CI[1] - REP_RE_CI[0]) / (2 * Z)
se_true = pct(math.sqrt(p_re * (1 - p_re) / N_RE))
print(f"SE back-calculated from reported CI: {se_from_reported:.4f} points; binomial SE "
f"{se_true:.4f} points (ratio {se_from_reported / se_true:.2f})")
print(f"Inverse-variance weight with the back-calculated SE, relative to the correct weight: "
f"{(se_true / se_from_reported) ** 2:.2f}")
print("\nReverse search 1: which count x out of 34,424 gives a Clopper-Pearson CI that rounds "
"to the reported one?")
match_x = []
for x in range(1, 400):
lo, hi = clopper_pearson(x, N_RE)
if round(pct(lo), 3) == REP_RE_CI[0] and round(pct(hi), 3) == REP_RE_CI[1]:
match_x.append(x)
print(f" x = {x}: {pct(lo):.4f}-{pct(hi):.4f}% (rate {pct(x / N_RE):.3f}%)")
X_MATCH = match_x[0] if match_x else None
for x in (169, 171):
lo, hi = clopper_pearson(x, N_RE)
print(f" neighbour x = {x}: {pct(lo):.3f}-{pct(hi):.3f}%")
print("\nReverse search 2: which denominator n gives 107/n a Clopper-Pearson CI that rounds "
"to the reported one?")
hits = []
for n in range(10000, 60001):
lo, hi = clopper_pearson(X_RE, n)
if round(pct(lo), 3) == REP_RE_CI[0] and round(pct(hi), 3) == REP_RE_CI[1]:
hits.append(n)
print(f" denominators matching both bounds: {len(hits)}")
best = min(range(10000, 60001),
key=lambda n: sum(abs(pct(b) - r) for b, r in zip(clopper_pearson(X_RE, n), REP_RE_CI)))
lo, hi = clopper_pearson(X_RE, best)
print(f" closest: n = {best}: {pct(lo):.3f}-{pct(hi):.3f}% (lower bound cannot be matched)")
print("\nReverse search 3: other alpha levels for 107/34,424 (Clopper-Pearson)")
for conf in (0.90, 0.99, 0.999):
lo, hi = clopper_pearson(X_RE, N_RE, a=1 - conf)
print(f" {int(conf * 1000) / 10:g}% CI: {pct(lo):.3f}-{pct(hi):.3f}%")
print(" Every standard two-sided interval for 107/34,424 contains 0.311%; none lies above it.")
# ---------------------------------------------------------------- 2. the method elsewhere
print()
print("=" * 78)
print("2. The stated method applied to the paper's other intervals")
print("=" * 78)
ROWS = [] # (group, label, rep_lo, rep_hi, cp_lo, cp_hi) on the paper's own scale
def show(group, label, x, n, rep, scale, nd):
lo, hi = clopper_pearson(x, n)
est, lo, hi = scale * x / n, scale * lo, scale * hi
ok = (round(est, nd), round(lo, nd), round(hi, nd)) == tuple(round(v, nd) for v in rep)
fmt = f"{{:.{nd}f}}"
s = f"{fmt.format(est)} ({fmt.format(lo)}-{fmt.format(hi)})"
r = f"{fmt.format(rep[0])} ({fmt.format(rep[1])}-{fmt.format(rep[2])})"
print(f" {label:22s} {x:>4d}/{n:<9,d} recomputed {s:24s} reported {r:24s} "
f"{'match' if ok else 'DIFFERS'}")
ROWS.append((group, label, rep[1], rep[2], lo, hi))
return ok
print("Table 1, per 10,000:")
n_ok1 = sum(show("Table 1", lab, x, n, rep, 1e4, 3) for lab, n, x, rep in TABLE1)
print(f" -> {n_ok1} of {len(TABLE1)} Table 1 rows reproduce exactly")
print("Table 2 and Results paragraph 9, % of 300:")
n_ok2 = sum(show("Table 2", lab, x, 300, rep, 100, 1) for lab, x, rep in TABLE2)
print(f" -> {n_ok2} of {len(TABLE2)} antibody intervals reproduce exactly")
print("Results paragraph 2, asymptomatic rate with the 'no history' denominator 9,865,404:")
show("Results", "300/9,865,404", X_ASYM, N_NOHIST, REP_ASYM, 1e4, 3)
print("Results paragraph 4, single and mixed testing, per 10,000 (counts not printed; inferred):")
print(" Denominator taken as share x 9,899,828; the count is the one whose exact CI matches.")
SM_X = []
for lab, share, rp in SINGLE_MIXED:
n_sm = round(share * N_TOTAL)
xs = [x for x in range(0, X_ASYM + 1)
if (round(1e4 * x / n_sm, 3), *(round(1e4 * v, 3) for v in clopper_pearson(x, n_sm)))
== rp]
print(f" {lab}: n ~ {n_sm:,}; counts reproducing {rp[0]:.3f} ({rp[1]:.3f}-{rp[2]:.3f}): {xs}")
# robustness: shares that round to the printed percentage
for sh in (share - 0.0003, share + 0.0002):
n2 = round(sh * N_TOTAL)
xs2 = [x for x in range(0, X_ASYM + 1)
if (round(1e4 * x / n2, 3), *(round(1e4 * v, 3) for v in clopper_pearson(x, n2)))
== rp]
print(f" share {100 * sh:.2f}% (n = {n2:,}): matching counts {xs2}")
SM_X.append(xs[0])
show("Single/mixed", f"{lab} {xs[0]}/~{n_sm:,}", xs[0], n_sm, rp, 1e4, 3)
print(f" Inferred counts {SM_X[0]} + {SM_X[1]} = {sum(SM_X)} (total asymptomatic positives "
f"{X_ASYM})")
print("Re-positive rate, % (for comparison):")
show("Re-positive", "107/34,424", X_RE, N_RE, (REP_RE_PT_RES, *REP_RE_CI), 100, 3)
print("\nDenominator checks:")
print(f" 9,865,404 + 34,424 = {N_NOHIST + N_HIST:,} (reported total 9,899,828)")
print(f" Male + Female = {5162960 + 4736868:,}")
print(f" Sum of age groups = {969014 + 4448230 + 2492943 + 1989641:,}")
print(f" Sum of districts = {sum(r[1] for r in TABLE1[7:]):,}; "
f"cases = {sum(r[2] for r in TABLE1[7:])}")
print(f" Antibody counts 161+110+29+0 = {161 + 110 + 29 + 0}; IgG+ 161+29 = {161 + 29}")
# ---------------------------------------------------------------- 3. Source Data workbook
# Counts copied from the publisher's Source Data workbook (41467_2020_19802_MOESM4_ESM.xlsx),
# sheets "Raw data from Table 1" and "Raw data from Table 2".
SD_TABLE1 = {
"Male": (5162960, 132), "Female": (4736868, 168),
"Age <=17": (969014, 12), "Age 18-44": (4448230, 104), "Age 45-59": (2492943, 96),
"Age >=60": (1989641, 88), "Wuchang": (904636, 73), "Qingshan": (414312, 23),
"Qiaokou": (583440, 32), "Hanyang": (717429, 29), "Jianghan": (524224, 19),
"Hongshan": (1103079, 35), "East Lake High-tech": (782987, 19), "Jiangan": (800440, 19),
"Caidian": (503595, 11), "Jiangxia": (671248, 14), "Huangpi": (979920, 14),
"Hannan": (417022, 4), "Dongxihu": (777204, 5), "Xinzhou": (634408, 3),
"East Lake Scenic": (85884, 0),
}
SD_TABLE2 = [("IgM- IgG+", 157), ("IgM- IgG-", 110), ("IgM+ IgG+", 28), ("IgM+ IgG-", 5)]
print()
print("=" * 78)
print("3. Publisher's Source Data workbook compared with the printed tables")
print("=" * 78)
t1 = {lab: (n, x) for lab, n, x, _ in TABLE1}
same1 = sum(t1[k] == v for k, v in SD_TABLE1.items())
print(f"Table 1 sheet: {same1} of {len(SD_TABLE1)} rows identical to the printed Table 1 "
f"(the sheet has no Total row)")
print("Table 2 sheet vs printed Table 2 (counts of 300; % with Clopper-Pearson 95% CI):")
for (lab, xsd), (_, xp, rep) in zip(SD_TABLE2, TABLE2):
lo, hi = clopper_pearson(xsd, 300)
print(f" {lab:10s} Source Data {xsd:>3d} -> {100 * xsd / 300:.1f} ({100 * lo:.1f}-{100 * hi:.1f})"
f" printed {xp:>3d} -> {rep[0]:.1f} ({rep[1]:.1f}-{rep[2]:.1f})"
f" {'same' if xsd == xp else 'DIFFERS'}")
sd_sum = sum(x for _, x in SD_TABLE2)
sd_igg = SD_TABLE2[0][1] + SD_TABLE2[2][1]
lo, hi = clopper_pearson(sd_igg, 300)
print(f" Source Data sum = {sd_sum}; IgG+ = {SD_TABLE2[0][1]}+{SD_TABLE2[2][1]} = {sd_igg} -> "
f"{100 * sd_igg / 300:.1f}% ({100 * lo:.1f}-{100 * hi:.1f}); printed IgG+ 190 -> 63.3% "
f"(57.6-68.8)")
sd_igm = SD_TABLE2[2][1] + SD_TABLE2[3][1]
print(f" IgM+ = {sd_igm} in the Source Data vs {29 + 0} in the printed table")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"axes.facecolor": BG, "figure.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "axes.spines.top": False, "axes.spines.right": False,
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# Figure 1: reported vs recomputed interval for 107/34,424 on one % axis
rows = [
("Reported\n(Abstract; Results)", REP_RE_PT_RES, REP_RE_CI, ORANGE, "o"),
("Recomputed: Clopper–Pearson\n(the stated method)", pct(p_re),
tuple(pct(v) for v in RE_CI["Clopper-Pearson exact (stated method)"]), BLUE, "o"),
("Recomputed: Wilson score", pct(p_re), tuple(pct(v) for v in RE_CI["Wilson score"]), BLUE, "s"),
("Recomputed: Wald", pct(p_re),
tuple(pct(v) for v in RE_CI["Wald (normal approximation)"]), BLUE, "D"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ys = np.arange(len(rows))[::-1].astype(float)
for (lab, est, (lo, hi), col, mk), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
ax.plot([lo, lo], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot([hi, hi], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot(est, y, marker=mk, ms=9, color=col, mec=col, zorder=4)
ax.text(hi + 0.012, y, f"{lo:.3f}–{hi:.3f}%", va="center", color=col, fontsize=10.5)
if col == ORANGE:
ax.annotate(f"reported point estimate {est:.3f}%\nlies outside its own interval",
xy=(est + 0.008, y - 0.06), xytext=(0.395, y - 0.47), va="center",
color=ORANGE, fontsize=10, arrowprops=dict(arrowstyle="-", color=ORANGE, lw=1))
ax.axvline(pct(p_re), color=SEC, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(pct(p_re), len(rows) - 0.45, f"107/34,424 = {pct(p_re):.3f}%", ha="center",
va="bottom", color=SEC, fontsize=10)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
for t, r in zip(ax.get_yticklabels(), rows):
t.set_color(r[3])
ax.set_xlim(0, 0.70)
ax.set_ylim(-0.6, len(rows) - 0.1)
ax.set_xlabel("Re-positive rate, % (point estimate and 95% CI)")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: reported bound / recomputed Clopper-Pearson bound for every interval in the paper
# (the two single/mixed-testing rates use the inferred counts above)
fig, ax = plt.subplots(figsize=(7.2, 4.2))
groups = [("Table 1 (22 rates per 10,000)", "Table 1"),
("Table 2 and Results\n(5 antibody %, printed counts)", "Table 2"),
("Results: asymptomatic rate,\ndenominator 9,865,404", "Results"),
("Results: single / mixed testing\n(2 rates, counts inferred)", "Single/mixed"),
("Re-positive rate, 107/34,424", "Re-positive")]
ys = np.arange(len(groups))[::-1].astype(float)
rng = np.random.default_rng(1)
for (glab, gkey), y in zip(groups, ys):
sub = [r for r in ROWS if r[0] == gkey]
col = ORANGE if gkey == "Re-positive" else BLUE
for _, _, rlo, rhi, clo, chi in sub:
jit = 0.0 # identical ratios within a group would overlap anyway
if clo > 0: # lower bounds drawn above the row centre, upper bounds below it
ax.plot(rlo / round(clo, 3 if gkey != "Table 2" else 1), y + 0.14 + jit, "o", ms=8,
mfc="none", mec=col, mew=1.6, zorder=3)
ax.plot(rhi / round(chi, 3 if gkey != "Table 2" else 1), y - 0.14 + jit, "D", ms=8,
color=col, alpha=0.85, zorder=3)
re = [r for r in ROWS if r[0] == "Re-positive"][0]
ax.text(re[3] / round(re[5], 3), ys[-1] - 0.3,
f"upper {REP_RE_CI[1] / round(pct(cp_hi), 3):.2f}", ha="center", va="top",
color=ORANGE, fontsize=10)
ax.text(re[2] / round(re[4], 3), ys[-1] + 0.3,
f"lower {REP_RE_CI[0] / round(pct(cp_lo), 3):.2f}", ha="center", va="bottom",
color=ORANGE, fontsize=10)
ax.axvline(1.0, color=SEC, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.85, 2.0)
ax.set_xticks([0.9, 1.0, 1.25, 1.5, 2.0])
ax.set_xticklabels(["0.9", "1", "1.25", "1.5", "2"])
for y, txt in zip(ys[:4], ["all 22 match (ratio 1.000)",
"all 5 match (ratio 1.000)",
"0.996 and 0.994",
"both match (ratio 1.000)"]):
ax.text(1.06, y, txt, va="center", color=BLUE, fontsize=10)
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels([g[0] for g in groups])
ax.set_ylim(-0.6, len(groups) - 0.4)
ax.set_xlabel("Reported ÷ recomputed Clopper–Pearson bound (log scale)")
ax.xaxis.grid(True, color=GRID, lw=0.8, zorder=0)
ax.tick_params(axis="y", length=0)
h1 = ax.plot([], [], "o", ms=8, mfc="none", mec=SEC, mew=1.6, label="lower bound")[0]
h2 = ax.plot([], [], "D", ms=8, color=SEC, label="upper bound")[0]
ax.legend(handles=[h1, h2], loc="lower right", bbox_to_anchor=(1.0, 1.0), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nRatios plotted in Figure 2 (reported / recomputed, both rounded as printed):")
for g, lab, rlo, rhi, clo, chi in ROWS:
nd = 1 if g == "Table 2" else 3
rl = "n/a" if round(clo, nd) == 0 else f"{rlo / round(clo, nd):.3f}"
print(f" {g:12s} {lab:30s} lower {rl:>6s} upper {rhi / round(chi, nd):.3f}")
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Abstract says serum-treated human islets came from T2DM donors; the donor table lists none
Nat Metab (2024) · doi:10.1038/s42255-024-01130-8 · PMC12005094
Summary
The abstract's only human result says that "human islets from male and female donors with T2DM" showed fewer senescence markers after treatment with serum collected after an exercise training programme. The paper's own donor table (Extended Data Table 1) lists 17 islet donors, and all 17 are marked T2D = "NO", including the 11 used for serum treatment. In the Results, the Methods and Extended Data Table 2, T2DM status belongs to the people who gave the serum, so the experiment exposed islets from non-diabetic cadaveric donors to serum from exercise-trained people with T2DM. The abstract, the Results subsection heading and the Fig. 8 title instead describe an experiment on T2DM islets; no measured value changes, but the design behind the paper's main translational claim is misstated.
What the paper reports
The abstract (the only human result it states):
"Remarkably, human islets from male and female donors with T2DM treated with serum collected after a 10-week endurance exercise training programme showed a significant decrease in the levels of senescence markers." (Abstract)
The same framing appears in a heading and a figure title:
"Postexercise serum reduces senescence markers in T2DM human islets" (Results, subsection heading)
"Postexercise serum reduces senescence markers in T2DM islets." (Fig. 8 title)
The body text describes the design differently. Diabetes is attributed to the serum donors, and the islets come from separate cadaveric donors:
"To test whether exercise training affects islet senescence in humans, we obtained serum from five individuals with T2DM who participated in a moderate-intensity exercise training programme. [...] Serum was matched by sex, age and body mass index (BMI) (Extended Data Table 2) to pancreatic islets from cadaveric donors, received from the Integrated Islet Distribution Program (IIDP) (Extended Data Table 1)." (Results, "Postexercise serum reduces senescence markers in T2DM human islets", para 1)
"Human islet donors were matched by sex, age and BMI to serum donors (individuals with or without T2DM who underwent a moderate-intensity aerobic training programme)." (Methods, "Serum treatment in vitro")
"Notably, serum obtained from exercise-trained people with T2DM decreased the expression of senescence markers in human cadaveric islets, translating these findings to human physiology." (Introduction, last paragraph)
"Details of donors are given in Extended Data Table 1." (Methods, "Human islets")
Extended Data Table 1, "Donor information for human pancreatic samples" (columns: Age (years) | T2D | BMI | Applications), has 17 rows. Every T2D cell reads "NO". Eleven rows list "Serum treatment" among the applications. Two examples:
"54 | NO | 24.5 | Serum treatment, WB, FACS, qPCR"
"39 | NO | 32.4 | Serum treatment, WB"
(Extended Data Table 1, first and eleventh rows)
Elsewhere, the paper describes its human islets as non-diabetic:
"we exposed cadaveric human islets from donors without T2DM to glucagon in vitro for 4 days" (Results, "Glucagon activates AMPK in mouse and human islets", para 4)
"human (nondiabetic, 37–62 years old, male and female, n = 8 for qPCR and n = 11 for β-gal expression) islets" (Fig. 3 legend, panels i–k)
What the paper's own numbers show
Method. Both Extended Data tables were transcribed from the PMC archive into recompute.py.txt (the same values appear in the version-of-record table images). The script counts the T2D column for all islet donors and for the subset whose Applications column includes "Serum treatment". It also checks the table sizes against the counts given in the text.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| T2D status of the serum-treated islet donors | Abstract | "donors with T2DM" | 0 of 11 serum-treatment donors have T2D = YES (Extended Data Table 1) |
| Islet donors listed | Extended Data Table 1 | 17 rows | 17 |
| Islet donors with T2D = YES | Extended Data Table 1, T2D column | all "NO" | 0 of 17 |
| Islet donors used for serum treatment | Extended Data Table 1, Applications column | 11 rows list "Serum treatment" | 11 (ages 22–62); 0 with T2D |
| Serum volunteers listed | Extended Data Table 2 vs Methods ("seven female and six male participants") | 13 | 13 (consistent) |
| Serum volunteers with T2D = YES | Extended Data Table 2 vs Results ("serum from five individuals with T2DM") | 5 used, per the text | 10 of 13 listed; the paper does not say which 5 of the 10 supplied serum for Fig. 8 |
| Islet preparations | Methods ("male (17) versus female (3) islets") vs Extended Data Table 1 | 17 + 3 = 20 | 17 rows (side discrepancy) |
The serum-treatment rows in Extended Data Table 1 fall into two groups. Five rows are marked "Serum treatment, WB, FACS, qPCR", matching the five T2DM serum donors in the Results. Six rows are marked "Serum treatment, WB", with ages 22–57, which matches the age range in the Extended Data Fig. 5 legend for the non-T2D serum experiment ("Samples from individuals ranging from 22 to 57 years old, male, n=6/group"). On this reading the legend's n = 6 counts islet preparations; the legend itself does not say so. Every islet row in both groups is T2D = "NO". The split does not change the finding, but it is consistent with the Fig. 8 islets being five of the non-diabetic donors listed in Extended Data Table 1.
Figure
Figure 1. On the left (orange) is the wording reported in the abstract and repeated in the Results subsection heading and the Fig. 8 title. On the right (blue) are counts recomputed from the paper's Extended Data Tables 1 and 2. The abstract says the islets came from donors with T2DM. Extended Data Table 1 lists 17 islet donors, none with T2D, including the 11 used for serum treatment. T2D = YES appears only among the serum volunteers in Extended Data Table 2 (10 of 13), and the Results text attributes T2DM to the serum donors ("serum from five individuals with T2DM"; the paper does not say which five of the ten were used). All counts come from recompute.py.txt.
Figure 2. Each circle is one row of the paper's donor tables. Filled circles have T2D = YES and open circles have T2D = NO, as recomputed from the tables. The top row holds the 11 islet donors whose Applications column includes "Serum treatment", the middle row the 6 other islet donors, and the bottom row the 13 serum volunteers. Every islet donor is non-diabetic. The orange line gives the abstract's description of the islet donors for comparison.
Why it matters
The abstract sentence is the paper's only stated human result and its translational headline ("Remarkably, ..."). Taken at face value, it says exercise-conditioned serum reduced senescence markers in islets from people who already had T2DM, that is, in diabetic beta cells where senescence is the proposed therapeutic target. The experiment the paper actually documents is different: non-diabetic cadaveric islets cultured with serum from exercise-trained people with T2DM. These two designs answer different questions. The first tests whether diabetic islet tissue responds. The second tests whether circulating factors from trained diabetic individuals act on healthy islets.
No reported value, effect direction or P value changes, and the narrower conclusion (post-exercise serum from people with T2DM lowered senescence markers in cultured human islets) is unaffected. Three statements are framed in terms of people who already have T2DM, and the abstract's wording presents the human experiment as direct evidence for them: the abstract's closing claim of "promising therapeutic implications for T2DM", the Discussion's statement that the findings bear on "mechanisms that can restore β-cell function in individuals with T2DM", and the criterion stated in the Results subsection "Senolytic effects of exercise training" that exercise "must decrease the percentage of senescent cells in populations in which β-cell failure has already developed, such as in older individuals and those with insulin resistance or overt T2DM". These statements also draw on the mouse experiments (that criterion is introduced for the INK-ATTAC mouse model), so they do not depend on the human experiment alone. Because the abstract, the subsection heading and the Fig. 8 title all use the same wording, a reader sees the correct design only in the Results paragraph, the Methods and the Extended Data tables.
Explanations considered
Misplaced modifier or shorthand. The wording probably means "human islets treated with serum from male and female donors with T2DM". The Reporting Summary supports this: "Serum was obtained from male and female volunteers with established type 2 diabetes ... A total of 5 subjects were used in the manuscript, being 2 females and 3 males". The phrase "male and female ... with established type 2 diabetes" there describes the serum donors. This explains how the wording arose but does not make the published sentence accurate. Read normally, "human islets from male and female donors with T2DM" says the islet donors had T2DM, and the heading and Fig. 8 title ("T2DM islets") say the same.
Swapped tables. The footers of the two Extended Data tables appear swapped. Table 1 (islet donors) ends "Human serum information.", and Table 2 (serum volunteers) ends "IIDP cadaveric human islets information." If the tables themselves were swapped, the islets would be the 10 T2D = YES rows and the abstract would be right. The table contents rule this out: - Table 1 is captioned "Donor information for human pancreatic samples" and has an Applications column (serum treatment, WB, FACS, qPCR) that only makes sense for islet preparations. - Table 2 is captioned "Volunteer information for serum from moderate intensity training program", has subject IDs, and has 13 rows, matching the Methods' "seven female and six male participants". - The Methods say "Details of donors are given in Extended Data Table 1" under "Human islets". - The Results text attributes T2DM to the serum donors. - Fig. 3, Fig. 4 and the Results describe the human islets as non-diabetic.
The swapped footers are a separate, minor labelling error that adds to the confusion.
Conversion or archiving artifact. Ruled out. The conflict is categorical (YES/NO cells, the words "with T2DM"), not numeric. The version of record on nature.com has the same abstract sentence and the same Fig. 8 title. The version-of-record images of Extended Data Tables 1 and 2 have the same T2D columns (17 of 17 NO; 10 of 13 YES) and the same swapped footers.
Some islet donors had T2DM but were not listed. Nothing in the paper supports this. The Methods point to Extended Data Table 1 for donor details, the table has a T2D column, and no row is YES. The Methods do count 20 islet preparations ("male (17) versus female (3)") against 17 table rows. That gap is a separate inconsistency, but no passage anywhere in the paper says any islet donor had T2DM.
Open points about the serum donors (not the main finding). The Results say serum came from "five individuals with T2DM" and the Reporting Summary says "A total of 5 subjects were used in the manuscript", while Extended Data Table 2 lists 10 volunteers with T2D = YES. This need not be a contradiction: the Methods say serum samples were received from 13 participants, so the table may list every volunteer while only five T2DM sera were used for Fig. 8. The paper does not say which five. Similarly, the Extended Data Fig. 5 legend ("n=6/group") does not say whether n counts islet preparations or serum donors; six islet preparations treated with serum from the 3 non-T2D volunteers in Extended Data Table 2 would fit. Neither point affects the T2D status of the islet donors.
Status
As of 2026-10-23:
- Version of record. The nature.com article page, reached through doi.org, was fetched again on 2026-10-23. It carries the same abstract sentence ("human islets from male and female donors with T2DM treated with serum collected after a 10-week endurance exercise training programme...") and the same Fig. 8 title ("Postexercise serum reduces senescence markers in T2DM islets."). The page has no "Author Correction", "Change history" or "This article has been updated" notice. An earlier audit stage viewed the version-of-record images of Extended Data Tables 1 and 2 and found them identical to the PMC text (17 of 17 islet donors NO; 10 of 13 serum volunteers YES; same swapped footers). The full Results and Methods text on nature.com is paywalled, so those passages were checked in the PMC copy, which is the NIH author manuscript of the article.
- Supplements. The Reporting Summary (42255_2024_1130_MOESM2_ESM.pdf) was read on 2026-10-23. Page 2, "Reporting on sex and gender", reads: "Serum was obtained from male and female volunteers with established type 2 diabetes, no criteria was used to exclude sex. A total of 5 subjects were used in the manuscript, being 2 females and 3 males." It attributes diabetes to the serum volunteers and says nothing about diabetic islet donors. The Source Data workbook for Fig. 8 (42255_2024_1130_MOESM10_ESM.xlsx) has columns Pre, Post, dA1c, dP21CIP1, dP16INK4A and dGlucagon, most sheets holding five data rows, and no donor diabetes status. An earlier audit stage found that the Supplementary Information PDF contains only a gating-strategy figure.
- Corrections.
src/known.py(run 2026-10-23) lists no Crossref updates and no PMC or Europe PMC correction or retraction notices. A Crossreffilter=updates:query for the DOI returns 0 results. - PubPeer. PubPeer's site and API returned HTTP 403 on 2026-10-23, so PubPeer could not be checked directly. An earlier audit stage searched the web for the title and DOI together with PubPeer, correction and erratum and found no relevant thread or notice; no new web search could be run on 2026-10-23.
Sources: PMC12005094 (PubMed Central archive, read with src/show.py); https://www.nature.com/articles/s42255-024-01130-8; Reporting Summary https://static-content.springer.com/esm/art%3A10.1038%2Fs42255-024-01130-8/MediaObjects/42255_2024_1130_MOESM2_ESM.pdf; Source Data Fig. 8 (42255_2024_1130_MOESM10_ESM.xlsx, same location); https://api.crossref.org/works?filter=updates:10.1038/s42255-024-01130-8.
Suggested correction
Reword the abstract sentence so that T2DM describes the serum donors, for example: "Remarkably, human islets from non-diabetic donors treated with serum collected from male and female individuals with T2DM after a 10-week endurance exercise training programme showed a significant decrease in the levels of senescence markers." Make the same change to the Results subsection heading and the Fig. 8 title (e.g. "Postexercise serum from individuals with T2DM reduces senescence markers in human islets"). In the same correction, the authors could also:
- swap the footers of Extended Data Tables 1 and 2;
- identify which 5 of the 10 T2D = YES volunteers in Extended Data Table 2 supplied serum for Fig. 8, and state what n counts in the Extended Data Fig. 5 legend;
- reconcile the number of islet preparations (20 in the Methods, 17 in Extended Data Table 1).
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC12005094 and regenerate fig1/fig2.
Run: python recompute.py.txt (writes fig1.svg/png and fig2.svg/png next to this file)
Inputs are transcribed from the article as archived in PubMed Central (PMC12005094):
Extended Data Table 1 ("Donor information for human pancreatic samples") and
Extended Data Table 2 ("Volunteer information for serum from moderate intensity
training program"). The same values appear in the version-of-record table images.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# Extended Data Table 1: (age, T2D, BMI, applications), in table order
EDT1 = [
(54, "NO", 24.5, "Serum treatment, WB, FACS, qPCR"),
(45, "NO", 21.9, "Serum treatment, WB, FACS, qPCR"),
(62, "NO", 23, "Serum treatment, WB, FACS, qPCR"),
(58, "NO", 58, "Serum treatment, WB, FACS, qPCR"),
(53, "NO", 31.20, "Serum treatment, WB, FACS, qPCR"),
(53, "NO", 33.6, "Serum treatment, WB"),
(22, "NO", 31.79, "Serum treatment, WB"),
(57, "NO", 30.9, "Serum treatment, WB"),
(52, "NO", 31.6, "Serum treatment, WB"),
(52, "NO", 37.5, "Serum treatment, WB"),
(39, "NO", 32.4, "Serum treatment, WB"),
(49, "NO", 34.8, "FACS, qPCR"),
(37, "NO", 28.4, "FACS, qPCR"),
(56, "NO", 32.8, "FACS, qPCR"),
(42, "NO", 23.5, "FACS, qPCR"),
(39, "NO", 31.1, "FACS, qPCR"),
(58, "NO", 31.8, "FACS, qPCR"),
]
# Extended Data Table 2: (subject id, age, BMI, T2D), in table order
EDT2 = [
(1, 48, 26.6, "YES"), (3, 51, 34.1, "YES"), (4, 55, 30.1, "YES"), (8, 55, 25, "YES"),
(9, 54, 25.7, "YES"), (2, 41, 30.3, "YES"), (5, 41, 33.6, "YES"), (6, 50, 32.3, "YES"),
(7, 46, 33.1, "YES"), (10, 52, 28, "YES"), (11, 53, 33.6, "NO"), (12, 27, 30.2, "NO"),
(13, 54, 31.7, "NO"),
]
# ---------------------------------------------------------------- recomputation
n_isl = len(EDT1)
isl_t2d = sum(r[1] == "YES" for r in EDT1)
serum_isl = [r for r in EDT1 if "Serum treatment" in r[3]]
serum_isl_t2d = sum(r[1] == "YES" for r in serum_isl)
grpA = [r for r in serum_isl if "qPCR" in r[3]] # Serum treatment, WB, FACS, qPCR
grpB = [r for r in serum_isl if "qPCR" not in r[3]] # Serum treatment, WB
n_ser = len(EDT2)
ser_t2d = sum(r[3] == "YES" for r in EDT2)
ser_no_ages = sorted(r[1] for r in EDT2 if r[3] == "NO")
print("== Islet donors, Extended Data Table 1 ==")
print(f"rows listed: {n_isl}")
print(f"T2D = YES: {isl_t2d} of {n_isl}")
print(f"rows whose Applications include 'Serum treatment': {len(serum_isl)}")
print(f" of these T2D = YES: {serum_isl_t2d} of {len(serum_isl)}")
print(f" ages: {min(r[0] for r in serum_isl)}-{max(r[0] for r in serum_isl)}")
print(f" 'Serum treatment, WB, FACS, qPCR' rows: {len(grpA)} (ages {sorted(r[0] for r in grpA)})")
print(f" 'Serum treatment, WB' rows: {len(grpB)} (ages {min(r[0] for r in grpB)}-{max(r[0] for r in grpB)})")
print()
print("== Serum volunteers, Extended Data Table 2 ==")
print(f"rows listed: {n_ser}")
print(f"T2D = YES: {ser_t2d} of {n_ser}")
print(f"T2D = NO: {n_ser - ser_t2d} (ages {ser_no_ages})")
print()
print("== Cross-checks against the text ==")
print(f"Methods 'seven female and six male participants' = {7 + 6}; equals EDT2 rows: {7 + 6 == n_ser}")
print(f"Results 'serum from five individuals with T2DM' = 5; EDT2 T2D = YES rows listed: {ser_t2d} "
f"(the paper does not say which 5 of the {ser_t2d} supplied serum for Fig. 8)")
print(f"Methods 'male (17) versus female (3) islets' = {17 + 3}; EDT1 rows: {n_isl}")
print(f"Ext. Data Fig. 5 legend 'n=6/group', '22 to 57 years old'; EDT1 'Serum treatment, WB' rows: "
f"{len(grpB)}, ages {min(r[0] for r in grpB)}-{max(r[0] for r in grpB)}; EDT2 non-T2D volunteers: "
f"{n_ser - ser_t2d} (the legend does not say whether n counts islet preparations or serum donors)")
print()
print(f"Verdict: islet donors with T2D in EDT1 = {isl_t2d}; the abstract's 'human islets from male and "
f"female donors with T2DM' is not supported by the paper's own donor table.")
# ---------------------------------------------------------------- figure style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# ---------------------------------------------------------------- fig1: text-vs-table diagram
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")
def panel(x, y, w, h, color, title, sub, body):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=color, lw=2))
ax.text(x + 0.14, y + h - 0.16, title, ha="left", va="top", fontsize=11, weight="bold", color=color)
ax.text(x + 0.14, y + h - 0.44, sub, ha="left", va="top", fontsize=10, color=INK2)
ax.text(x + 0.14, y + h - 0.80, body, ha="left", va="top", fontsize=10, color=INK, linespacing=1.4)
panel(0.15, 1.55, 3.15, 2.95, ORANGE, "As reported (abstract)",
"Repeated in a heading and Fig. 8 title",
"\u201chuman islets from male and\nfemale donors with T2DM\ntreated with serum collected\nafter a 10-week endurance\nexercise training programme\u201d\n\nResults heading: \u201c\u2026 in T2DM\nhuman islets\u201d\nFig. 8 title: \u201c\u2026 in T2DM islets\u201d")
panel(3.9, 1.55, 3.15, 2.95, BLUE, "The paper's own tables",
"Extended Data Tables 1 and 2",
f"Islet donors (Table 1): {n_isl} listed\n"
f" with T2D = YES: {isl_t2d} of {n_isl}\n"
f" used for serum treatment: {len(serum_isl)}\n"
f" of those with T2D: {serum_isl_t2d} of {len(serum_isl)}\n\n"
f"Serum volunteers (Table 2): {n_ser}\n"
f" with T2D = YES: {ser_t2d} of {n_ser}\n"
"Results: \u201cserum from five\nindividuals with T2DM\u201d\n"
"(which 5 is not stated)")
# conflict box joined to both panels
ax.add_patch(FancyBboxPatch((1.0, 0.22), 5.2, 0.86, boxstyle="round,pad=0.02,rounding_size=0.08",
fc=BG, ec=INK2, lw=1.2, ls="--"))
ax.text(3.6, 0.83, "Conflict: who had T2DM?", ha="center", va="center", fontsize=11, weight="bold", color=INK)
ax.text(3.6, 0.47, "Abstract: the islet donors. Tables and Results: only the serum donors;\n"
f"all {n_isl} islet donors are T2D = NO.", ha="center", va="center", fontsize=10, color=INK)
for x0 in (1.725, 5.475):
ax.add_patch(FancyArrowPatch((x0, 1.53), (3.6 + (x0 - 3.6) * 0.55, 1.10), arrowstyle="-|>",
mutation_scale=13, lw=2, color=INK2, connectionstyle="arc3,rad=0"))
save(fig, "fig1")
# ---------------------------------------------------------------- fig2: donor-level dot strip
fig, ax = plt.subplots(figsize=(7.2, 3.0))
fig.subplots_adjust(left=0.34, right=0.95, top=0.80, bottom=0.24)
rows = []
# row 0: islet donors used for serum treatment; row 1: other islet donors; row 2: serum volunteers
for i, r in enumerate(serum_isl):
rows.append((i, 2, r[1] == "YES"))
for i, r in enumerate([r for r in EDT1 if "Serum treatment" not in r[3]]):
rows.append((i, 1, r[1] == "YES"))
for i, r in enumerate(EDT2):
rows.append((i, 0, r[3] == "YES"))
for x, y, yes in rows:
ax.plot(x, y, "o", ms=11, mec=BLUE, mew=2, mfc=BLUE if yes else BG)
ax.set_yticks([0, 1, 2])
ax.set_yticklabels([f"Serum volunteers\n(Ext. Data Table 2, n={n_ser})",
f"Islet donors, other uses\n(Ext. Data Table 1, n={n_isl - len(serum_isl)})",
f"Islet donors, serum treatment\n(Ext. Data Table 1, n={len(serum_isl)})"])
ax.set_xlim(-0.7, max(len(serum_isl), n_ser) - 0.3); ax.set_ylim(-0.6, 2.6)
ax.set_xticks([]); ax.tick_params(axis="y", length=0, colors=INK)
for s in ax.spines.values():
s.set_visible(False)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.text(len(serum_isl) - 0.3, 2, f" {serum_isl_t2d} of {len(serum_isl)} T2D", va="center", fontsize=10, color=BLUE)
ax.text(n_isl - len(serum_isl) - 0.3, 1, f" {isl_t2d} of {n_isl - len(serum_isl)} T2D", va="center", fontsize=10, color=BLUE)
ax.annotate(f"{ser_t2d} of {n_ser} T2D", xy=(ser_t2d - 1, 0), xytext=(ser_t2d - 1, -0.52),
ha="center", va="center", fontsize=10, color=BLUE)
handles = [Line2D([], [], ls="", marker="o", ms=9, mec=BLUE, mew=2, mfc=BLUE,
label="T2D = YES (recomputed from tables)"),
Line2D([], [], ls="", marker="o", ms=9, mec=BLUE, mew=2, mfc=BG, label="T2D = NO")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(-0.45, 1.02), ncol=2, frameon=False, fontsize=10)
fig.text(0.02, 0.10, "As reported in the abstract: the islets were \u201cfrom male and female donors with T2DM\u201d.",
fontsize=10, color=ORANGE, ha="left")
fig.text(0.02, 0.025, "Each circle is one table row; fill shows the T2D column of that row.", fontsize=10, color=INK2, ha="left")
save(fig, "fig2")
print("\nfigures written: fig1.svg, fig1.png, fig2.svg, fig2.png")
Abstract gives an onset age and female share that contradict the paper's own Table 1
Neurology (2025) · doi:10.1212/WNL.0000000000214316 · PMC12615010
Summary
The abstract describes the 140 autopsy-confirmed patients with multiple system atrophy (MSA) as having a "mean age at onset 62.3 ± 8.9 years" and as "44% female". Table 1 gives the same 140 patients a mean age at onset of 57.4 ± 8.5 years and a male:female split of 59:81, which is 57.9% female. The table checks out from several directions. Its subgroup counts sum to 59 men and 81 women in both of its groupings, the size-weighted mean of its subgroup onset ages is 57.40 years, and every one of its 11 subgroup mean onset ages is 59.7 years or less, so the overall mean cannot be 62.3. The paper's inferential results are not affected. However, the abstract, which is the part most often read and indexed on its own, makes the cohort look 4.9 years older at onset than it is and shows men, not women, as the majority.
What the paper reports
All quotations were checked verbatim against the PMC-archived text with src/show.py.
Abstract, Results:
Among 140 patients (mean age at onset 62.3 ± 8.9 years; 44% female), 81 (58%) initially presented without dysautonomia (57 with parkinsonism only, 17 with cerebellar syndrome only, 7 with both).
Table 1 ("Demographic Data"), All patients column:
M:F, n [%] | 59:81 [42:58]
Age at onset, y, mean ± SEM [range; median] | 57.4 ± 8.5 [35–80; 57.5]
Age of death, y, mean ± SEM [range; median] | 64.8 ± 7.7 [40–90; 64.5]
Disease duration, y, mean ± SEM [range; median] | 7.4 ± 3 [2–20; 7]
Table 1, M:F row by phenotype at onset (A, P, C, P+A, C+A, P+C, P+C+A):
16:11a [59:41] | 20:37a [35:65] | 11:6a [65:35] | 5:14 [26:74] | 2:2 [50:50] | 3:4 [37:63] | 2:7 [22:78]
Results, "Characteristics of Patients and of Clinical Phenotypes at Disease Onset":
We included a total of 140 autopsy-confirmed patients with MSA who had sufficient clinical data regarding disease progression in the analysis. The demographic characteristics and basic parameters of the study population are summarized in Table 1.
A significant association was found between sex and phenotype A, P, and C at onset, χ2(2, N = 101) = 7.015, p = 0.030, indicating a higher prevalence of P in women (n = 20 m vs n = 37 f) and of C (n = 11 m vs n = 6 f) and A (16 m vs 11 f) in men.
Methods, "Patients and Clinical Data":
The cohort has been described in previously published work, comprising 144 patients with a pathologic diagnosis of MSA.
Methods, "Statistical Analysis":
Demographic data were presented as means ± SD, ranges, and medians.
What the paper's own numbers show
Method. I transcribed the All patients column of Table 1 and every subgroup column: 7 groups by phenotype at onset and 4 groups by phenotype at final visit, each a partition of the 140 patients. I summed the subgroup sex counts. I pooled the subgroup onset means weighted by subgroup size, and pooled the spreads as SDs (within-group plus between-group variance). As a further check, I compared onset age with mean age at death minus mean disease duration. I also recomputed the Results-text chi-square from its counts, to confirm that the table's sex counts are the ones the analysis used. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Mean age at onset, all 140 | Abstract, Results | 62.3 ± 8.9 y | 57.40 ± 8.49 y (n-weighted pool of the 7 onset groups) |
| Mean age at onset, all 140 | Table 1, All patients | 57.4 ± 8.5 | 57.40 (onset groups); 57.51 (final-visit groups) |
| Highest subgroup mean onset age | Table 1, 11 subgroup columns | 55.9 to 59.7 | maximum 59.7 (final P+C, n = 12); a pooled mean cannot exceed it |
| Age at death − disease duration | Table 1, All patients | 64.8 and 7.4 | 64.8 − 7.4 = 57.4; n-weighted onset groups: 64.80 − 7.44 = 57.37 |
| Female share | Abstract, Results | 44% | 81/140 = 57.9% |
| Sex counts | Table 1, M:F | 59:81 [42:58] | onset groups sum to M 59, F 81; final-visit groups sum to M 59, F 81; M + F = n in every column |
| 44% of 140 | derived | — | 61.6 women (61/140 = 43.6%, 62/140 = 44.3%); matches no count in the paper |
| Male share | Table 1 | 42% | 59/140 = 42.1% (so 44% is not the male share either) |
| Sex × onset phenotype (A, P, C) | Results text | χ²(2, N = 101) = 7.015, p = 0.030 | 7.015, p = 0.030 from counts 16/11, 20/37, 11/6, which equal the Table 1 counts |
| Differences, abstract vs table | — | — | onset age +4.9 y; female share −13.9 percentage points |
Table 1 is internally consistent and agrees with the Results text. Its subgroup counts, subgroup means and overall column fit together to the precision of the printed figures. The pooled SD of 8.49 even reproduces the overall spread of 8.5. The abstract's two demographic figures match nothing in the paper. The 62.3-year mean is 2.6 years above the highest of the 11 subgroup means. Taken against the table's mean age at death of 64.8, it would leave about 2.5 years from onset to death, while the table's mean disease duration is 7.4 years. The 44% female share would mean 61.6 women, which is not a whole number of patients. It is not the male share (42.1%) either.
Figure
Figure 1. The two conflicting passages side by side. Left (orange): the abstract's description of the cohort, quoted verbatim, which gives an onset age of 62.3 years and 44% female. Right (blue): the Table 1 All patients cells, quoted with the row label shortened, and the values recomputed from the table's own subgroup columns, which give an n-weighted onset age of 57.40 years and 81/140 = 57.9% female. Both passages describe the same 140 patients. The abstract's onset age is 4.9 years higher than the table's, and its female share is 13.9 percentage points lower, which turns a female-majority cohort into an apparent male-majority one. The 44% also does not equal the table's male share (42.1%).
Figure 2. Left: mean age at onset in each Table 1 subgroup (open circles, labelled with subgroup size), shown for the 7 groups by phenotype at onset and the 4 groups by phenotype at final visit. Below them are the n-weighted pooled mean recomputed from the onset groups (blue circle, 57.40) and the abstract's overall mean (orange diamond, 62.3). The dotted line marks the highest subgroup mean (59.7). A weighted mean of the subgroup means cannot exceed it, so 62.3 cannot be the mean of these patients. Right: number of patients by sex. The open bar is the 59 men and the blue bar is the 81 women, both summed from the Table 1 subgroups. The orange bar is the number of women implied by the abstract's 44% of 140, which is 61.6. The x-axis starts at 0 on the right panel. On the left it spans 50 to 65 years, so the gap between 57.4 and 62.3 is shown at true scale against the spread of the subgroup means.
Why it matters
- The abstract misdescribes the cohort. Onset age and sex balance are the basic facts a reader uses to judge whom the findings apply to. The abstract puts the cohort's mean onset 4.9 years later than the body does (62.3 vs 57.4). It also reverses the sex majority, with 44% female against 57.9%. Neither error can be caught from the abstract alone, and the abstract is what databases index and what readers, reviewers and evidence syntheses most often extract.
- The abstract's sex figure conflicts with a reported finding. The paper reports a significant link between sex and onset phenotype, with parkinsonian onset more common in women (20 men vs 37 women). In the table, that sits within a female-majority cohort. A reader of the abstract would place it in a male-majority one.
- The inferential results are not affected. The tested findings are the 12 motor-only patients (9%), their longer disease duration against matched patients (8.1 ± 2.1 vs 6.3 ± 2.6 years, p = 0.035), the 51% who developed all 3 symptom complexes, and the faster progression after cerebellar onset (2.0 vs 3.4 years). None of them uses the cohort's summary onset age or sex share. The matching used each patient's own sex and onset age. Every other number in the abstract Results agrees with the body: 81 (58%), 57/17/7, 12 (9%), 51% and the durations. The error is confined to the two demographic figures.
Explanations considered
- A different population, such as the 144-patient parent cohort. This does not account for the numbers. The abstract says "Among 140 patients", the same N as Table 1 and the Results. The Methods mention the 144-patient cohort only as previously described work. That cohort's own publication (Mov Disord 2024, PMC12403858) reports an M:F of 61:83 [42.4:57.6] and an onset age of 57.42 ± 8.43. Those figures are close to Table 1 here and far from 62.3 years and 44% female.
- A different age variable. This does not account for the numbers. The paper reports age at onset (57.4), age at death (64.8) and disease duration (7.4) for the cohort, and nothing else. 62.3 is none of these and no simple combination of them. The one natural combination, death minus duration, gives 57.4 and so supports the table.
- Sex labels swapped. This does not account for the numbers. If 44% were the male share it would mean 61.6 men. The table has 59 men (42.1%) and 81 women. The Results text and the chi-square, which recomputes exactly to 7.015 from the table's counts, use the same counts.
- Table 1 is the one in error. This is not supported. The subgroup counts sum to the totals in two independent groupings, and M + F equals n in every column. The pooled subgroup means reproduce the overall 57.4 (57.40 and 57.51), and the pooled SD reproduces 8.5 (8.49). Every subgroup mean is 59.7 or below, so no arrangement of these subgroups can give an overall mean of 62.3.
- Rounding or a different denominator. This does not account for the numbers. No denominator the paper reports, whether 140, 101 (the A/P/C chi-square sample) or 99 (the Levodopa analysis), yields 44% female from the counts given. For example, the A, P and C groups together have 54 women of 101 (53.5%).
- A text-conversion artifact. Ruled out. The typeset publisher PDF prints the same abstract sentence and the same Table 1 cells as plain text (see Status). So do the Europe PMC XML and the PubMed abstract.
- Side observations, not part of the main finding. (1) Table 1 labels the spreads "mean ± SEM", but the Methods say "means ± SD". The values are SDs: the pooled subgroup SD is 8.49, while an SEM for n = 140 would be about 0.72. (2) The Results text says "Fifty patients (36%) showed only P+A", while Table 1 gives 49 [35]. With 50, the four final-visit groups would total 141, so the table's 49 is the consistent figure. Neither observation affects any conclusion.
Status
- Version of record. The publisher's site (neurology.org) returned HTTP 403 to automated requests for the article page, the PDF and the supplement. I therefore read two repository copies of the publisher's typeset 8-page PDF, both stamped as downloaded from neurology.org, with the publisher's page footer e214316 and the same creation date (2025-11-13). One is the LMU Munich repository (epub.ub.uni-muenchen.de/130571) and the other the DZNE repository (pub.dzne.de/record/282298). Both print the abstract sentence and the Table 1 cells exactly as quoted above. The Europe PMC full-text XML and the PubMed abstract (PMID 41232058) carry the same text.
- Supplements. The supplement (eTable 1, a sample overview by centre; eTable 2, symptom categories and documentation) is hosted only on neurology.org and could not be retrieved. PMC lists no supplement, and the Europe PMC supplementary bundle contains only the Figure 1 image. The Methods describe neither eTable as reporting cohort-level onset age or sex, and the cohort figures are already given in main Table 1. The data are available on request only.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or published comment.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices, and the Crossref record has no update-to or updated-by entries. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search of the title together with "PubPeer", "correction" and "erratum" found no thread or notice for this article. - Sources. PMC article PMC12615010 (via
src/show.py); Europe PMC full-text XML and supplementary files; PubMed PMID 41232058; typeset PDFs at epub.ub.uni-muenchen.de/130571 and pub.dzne.de/record/282298; the parent-cohort paper PMC12403858; Crossref; PubPeer; https://doi.org/10.1212/WNL.0000000000214316.
Suggested correction
In the abstract Results, replace
Among 140 patients (mean age at onset 62.3 ± 8.9 years; 44% female)
with
Among 140 patients (mean age at onset 57.4 ± 8.5 years; 58% female)
so that it matches Table 1 (M:F 59:81; age at onset 57.4 ± 8.5 years). If 62.3 ± 8.9 years and 44% female describe some other group, the abstract should name that group. The authors may also wish to relabel the Table 1 spreads from "SEM" to "SD", in line with the Methods and with the values. They may also wish to reconcile "Fifty patients (36%)" for the P+A group in the Results text with the 49 [35] in Table 1.
Recomputation script (python recompute.py.txt)
"""Recompute the cohort demographics of PMC12615010 (Neurology 2025,
doi:10.1212/WNL.0000000000214316) from the paper's own Table 1, and
regenerate fig1.svg/png and fig2.svg/png in this script's folder.
Run: python recompute.py.txt
All inputs are transcribed from Table 1, the abstract and the Results text.
"""
import os
import numpy as np
from scipy.stats import chi2_contingency
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
# Abstract, Results: "Among 140 patients (mean age at onset 62.3 ± 8.9 years; 44% female)"
ABS_N, ABS_ONSET, ABS_ONSET_SD, ABS_FEMALE_PCT = 140, 62.3, 8.9, 44.0
# Table 1, All patients column
T_N, T_M, T_F = 140, 59, 81
T_ONSET, T_ONSET_SPREAD = 57.4, 8.5
T_DEATH, T_DUR = 64.8, 7.4
# Table 1, phenotype at onset: A, P, C, P+A, C+A, P+C, P+C+A
on_lab = ["A", "P", "C", "P+A", "C+A", "P+C", "P+C+A"]
on_n = np.array([27, 57, 17, 19, 4, 7, 9])
on_m = np.array([16, 20, 11, 5, 2, 3, 2])
on_f = np.array([11, 37, 6, 14, 2, 4, 7])
on_age = np.array([56.9, 56.9, 57.2, 59.3, 56.5, 56.9, 59.2])
on_sd = np.array([7.1, 10, 7.8, 9, 6, 4.8, 6.1])
on_death = np.array([64.4, 65.4, 64.9, 64.4, 61.3, 64.1, 65.0])
on_dur = np.array([7.5, 8.5, 7.7, 5.2, 4.8, 7.3, 6.0])
# Table 1, phenotype at final visit: P+A, C+A, P+C, P+C+A
fv_lab = ["P+A", "C+A", "P+C", "P+C+A"]
fv_n = np.array([49, 7, 12, 72])
fv_m = np.array([18, 6, 7, 28])
fv_f = np.array([31, 1, 5, 44])
fv_age = np.array([57.5, 55.9, 59.7, 57.3])
fv_sd = np.array([10.8, 8.1, 6.9, 6.9])
# Results text: sex by onset phenotype A, P, C (m, f)
res_counts = np.array([[16, 20, 11], [11, 37, 6]])
def pooled(n, mean, sd):
gm = (n * mean).sum() / n.sum()
var = ((n - 1) * sd ** 2 + n * (mean - gm) ** 2).sum() / (n.sum() - 1)
return gm, np.sqrt(var)
# ---------------------------------------------------------------- checks
print("== Sex")
print("onset groups: n sum = %d, M sum = %d, F sum = %d, M+F == n in every group: %s"
% (on_n.sum(), on_m.sum(), on_f.sum(), bool((on_m + on_f == on_n).all())))
print("final groups: n sum = %d, M sum = %d, F sum = %d, M+F == n in every group: %s"
% (fv_n.sum(), fv_m.sum(), fv_f.sum(), bool((fv_m + fv_f == fv_n).all())))
fem_pct = 100 * T_F / T_N
male_pct = 100 * T_M / T_N
print("female share 81/140 = %.1f%%; male share 59/140 = %.1f%%" % (fem_pct, male_pct))
print("abstract 44%% female of 140 = %.1f women (not a whole number)" % (ABS_FEMALE_PCT / 100 * ABS_N))
for k in (61, 62):
print(" %d/140 = %.1f%%" % (k, 100 * k / 140))
print("difference in female share: %.1f percentage points" % (fem_pct - ABS_FEMALE_PCT))
apc_f = on_f[:3].sum(); apc_n = on_n[:3].sum()
print("women in A+P+C onset groups: %d/%d = %.1f%%" % (apc_f, apc_n, 100 * apc_f / apc_n))
for d in (140, 101, 99):
print(" 44%% of %d = %.2f" % (d, 0.44 * d))
print("\n== Age at onset")
g_on, s_on = pooled(on_n, on_age, on_sd)
g_fv, s_fv = pooled(fv_n, fv_age, fv_sd)
print("n-weighted mean of 7 onset groups = %.2f, pooled SD = %.2f" % (g_on, s_on))
print("n-weighted mean of 4 final-visit groups = %.2f, pooled SD = %.2f" % (g_fv, s_fv))
allmeans = np.concatenate([on_age, fv_age])
print("highest of the 11 subgroup mean onset ages = %.1f (abstract overall mean 62.3)" % allmeans.max())
print("Table 1 mean age at death - mean duration = %.1f - %.1f = %.1f" % (T_DEATH, T_DUR, T_DEATH - T_DUR))
wd = (on_n * on_death).sum() / on_n.sum(); wu = (on_n * on_dur).sum() / on_n.sum()
print("n-weighted onset-group death age = %.2f, duration = %.2f, difference = %.2f" % (wd, wu, wd - wu))
print("abstract onset 62.3 implies mean time to death %.1f y vs Table 1 duration 7.4 y" % (T_DEATH - ABS_ONSET))
print("abstract 62.3 - Table 1 57.4 = %.1f y" % (ABS_ONSET - T_ONSET))
print("SEM implied by SD 8.5 and n = 140: %.2f (Table 1 labels 8.5 'SEM')" % (T_ONSET_SPREAD / np.sqrt(T_N)))
print("\n== Chi-square, sex x onset phenotype A/P/C (Results text counts)")
print("Results text counts match Table 1 groups A, P, C:",
bool((res_counts[0] == on_m[:3]).all() and (res_counts[1] == on_f[:3]).all()))
chi, p, dof, _ = chi2_contingency(res_counts, correction=False)
print("chi2(%d, N = %d) = %.3f, p = %.3f (reported 7.015, p = 0.030)" % (dof, res_counts.sum(), chi, p))
print("\n== Other abstract numbers")
print("without dysautonomia at onset: 57 + 17 + 7 = %d, %d/140 = %.1f%%" % (57 + 17 + 7, 81, 100 * 81 / 140))
print("motor-only at final visit: 12/140 = %.1f%%" % (100 * 12 / 140))
print("all 3 complexes at final visit: 72/140 = %.1f%%" % (100 * 72 / 140))
print("Results text 'Fifty patients (36%%) ... P+A' vs Table 1 49 [35]: 49/140 = %.1f%%, 50/140 = %.1f%%;"
" final-visit n with 50 = %d" % (100 * 49 / 140, 100 * 50 / 140, 50 + 7 + 12 + 72))
# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, "%s.%s" % (name, ext)), dpi=200,
facecolor=BG, bbox_inches="tight", pad_inches=0.15)
plt.close(fig)
# ---------------------------------------------------------------- fig1
fig = plt.figure(figsize=(7.2, 4.1), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 60); ax.axis("off")
def panel(x, y, w, h, edge, title, lines):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=edge, lw=2))
ax.text(x + 1.5, y + h - 3.2, title, fontsize=11.5, weight="bold", color=edge, va="top")
for i, (t, kw) in enumerate(lines):
ax.text(x + 1.5, y + h - 10.5 - i * 5.6, t, va="top", **kw)
panel(1.5, 15, 41, 39, ORANGE, "Abstract, Results",
[("“Among 140 patients", dict(fontsize=10.5, style="italic")),
("(mean age at onset 62.3 ± 8.9 years;", dict(fontsize=10.5, style="italic")),
("44% female)”", dict(fontsize=10.5, style="italic")),
("Values reported in the abstract", dict(fontsize=10, color=INK2)),
("Onset age 62.3 y · 44% female", dict(fontsize=11, weight="bold", color=ORANGE))])
panel(57.5, 15, 41, 39, BLUE, "Table 1, All patients (n = 140)",
[("“M:F, n [%] 59:81 [42:58]”", dict(fontsize=10.5, style="italic")),
("“Age at onset ... 57.4 ± 8.5”", dict(fontsize=10.5, style="italic")),
("Recomputed from its subgroups", dict(fontsize=10, color=INK2)),
("n-weighted onset = %.2f y" % g_on, dict(fontsize=11, weight="bold", color=BLUE)),
("81/140 = %.1f%% female" % fem_pct, dict(fontsize=11, weight="bold", color=BLUE))])
ax.add_patch(FancyArrowPatch((44.5, 34), (55.5, 34), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=INK))
ax.text(50, 39.5, "same 140\npatients", ha="center", va="bottom", fontsize=10, color=INK2)
ax.text(50, 9.5, "Conflict: onset age differs by %.1f years; female share differs by %.1f points "
"(the majority sex flips)" % (ABS_ONSET - T_ONSET, fem_pct - ABS_FEMALE_PCT),
ha="center", va="center", fontsize=10.5, weight="bold")
ax.text(50, 4, "44%% is also not the male share (59/140 = %.1f%%); 44%% of 140 = %.1f patients"
% (male_pct, ABS_FEMALE_PCT / 100 * ABS_N), ha="center", va="center", fontsize=10, color=INK2)
save(fig, "fig1")
# ---------------------------------------------------------------- fig2
fig, (a1, a2) = plt.subplots(1, 2, figsize=(7.2, 3.9), facecolor=BG,
gridspec_kw={"width_ratios": [1.25, 1], "wspace": 0.45})
for a in (a1, a2):
a.set_facecolor(BG)
a.grid(axis="x", color=GRID, lw=0.8); a.set_axisbelow(True)
for s in ("top", "right", "left"):
a.spines[s].set_visible(False)
a.tick_params(axis="y", length=0)
# left: onset ages
labs = ["onset " + l for l in on_lab] + ["final " + l for l in fv_lab]
ns = np.concatenate([on_n, fv_n])
rows = [14, 13, 12, 11, 10, 9, 8, 6, 5, 4, 3] # onset groups, gap, final-visit groups
a1.scatter(allmeans, rows, s=48, facecolors="none", edgecolors=INK2, lw=1.5, zorder=3,
label="Table 1 subgroup means")
a1.scatter([g_on], [1], s=80, color=BLUE, zorder=4, label="Recomputed pooled mean")
a1.scatter([ABS_ONSET], [0], s=80, color=ORANGE, marker="D", zorder=4, label="Abstract")
a1.text(g_on + 0.6, 1, "%.2f" % g_on, va="center", color=BLUE, fontsize=10)
a1.text(ABS_ONSET + 0.6, 0, "62.3", va="center", ha="left", color=ORANGE, fontsize=10)
a1.axvline(allmeans.max(), color=INK2, lw=1, ls=":")
a1.text(allmeans.max() - 0.2, 14.6, "highest subgroup\nmean %.1f" % allmeans.max(), fontsize=10, color=INK2, va="bottom", ha="right")
a1.set_yticks(rows + [1, 0])
a1.set_ylim(-0.7, 16.6)
a1.set_yticklabels(["%s (n=%d)" % (l, k) for l, k in zip(labs, ns)] +
["pooled, n=140", "abstract"], fontsize=10)
a1.set_xlim(50, 65); a1.set_xlabel("Mean age at onset, years")
a1.set_title("Onset age: every subgroup\nmean is below 62.3", fontsize=10.5, loc="left")
# right: female count
a2.barh([1], [T_F], color=BLUE, height=0.5)
a2.barh([0], [ABS_FEMALE_PCT / 100 * ABS_N], color=ORANGE, height=0.5)
a2.barh([2], [T_M], color="none", edgecolor=INK2, lw=1.5, height=0.5)
a2.text(T_F + 2, 1, "81 (%.1f%%)" % fem_pct, va="center", color=BLUE, fontsize=10)
a2.text(ABS_FEMALE_PCT / 100 * ABS_N + 2, 0, "%.1f (44%%)" % (ABS_FEMALE_PCT / 100 * ABS_N),
va="center", color=ORANGE, fontsize=10)
a2.text(T_M + 2, 2, "59 (%.1f%%)" % male_pct, va="center", color=INK2, fontsize=10)
a2.set_yticks([2, 1, 0])
a2.set_yticklabels(["men, Table 1", "women, Table 1\nsubgroup sums", "women implied\nby abstract"],
fontsize=10)
a2.set_xlim(0, 140); a2.set_xlabel("Patients (of 140)")
a2.set_title("Sex: the table sums to\n81 women", fontsize=10.5, loc="left")
h, l = a1.get_legend_handles_labels()
fig.legend(h, l, loc="lower center", ncol=3, frameon=False, fontsize=10, bbox_to_anchor=(0.5, -0.14))
save(fig, "fig2")
print("\nfigures written to", HERE)
Abstract reports all headache-free subgroups at p < 0.001; Results report p = 0.126 and 0.090
Neurology (2026) · doi:10.1212/WNL.0000000000218152 · PMC13312941
Summary
This registry study of 588 participants with migraine and 147 matched healthy controls (HCs) states in its Abstract that a sensitivity analysis restricted to headache-free participants showed lower plasma CGRP than controls "across all migraine subgroups" (all p < 0.001). The paper's own Results say that, in the same analysis, controls "did not differ significantly" from migraine with aura (p = 0.126) or chronic migraine (p = 0.090). These values are about 100 times the Abstract's bound and above 0.05, and undoing the Bonferroni correction still leaves 0.042 and 0.030. The main matched-sample finding (125 vs 151 pmol/L, p < 0.001) stands, but the Results do not support the robustness claim that the Abstract and Discussion build on this analysis.
What the paper reports
Abstract, Results:
Sensitivity analyses restricted to headache-free participants confirmed the principal findings, demonstrating lower plasma CGRP concentrations across all migraine subgroups compared with controls (all p < 0.001).
Results, "Plasma CGRP Concentrations", second paragraph:
Sensitivity analyses restricted to headache-free participants produced results largely consistent with the main analysis (Figure 2). Median plasma CGRP concentrations in HCs did not differ significantly from those in participants with migraine with aura (151 [118–199] vs 143 [86–179] pmol/L; p = 0.126) or chronic migraine (151 [118–199] vs 142 [90–178] pmol/L; p = 0.090).
Results, "Plasma CGRP Concentrations", first paragraph (the main, unrestricted analysis):
Notably, all migraine subgroups exhibited significantly lower CGRP levels compared with HCs (all p ≤ 0.001).
Discussion, first paragraph:
Furthermore, preventive treatment use, including onabotulinumtoxinA specifically, was unrelated to plasma CGRP concentrations, and sensitivity analyses restricted to headache-free participants confirmed robustness of the findings.
Methods, "Statistical Analysis":
CGRP levels were compared between participants with migraine and HCs in the matched sample using the Mann-Whitney U test. As a sensitivity analysis, group comparisons were repeated in the subgroup of participants who were headache-free at the time of blood sampling.
Significance was set at a 2-sided p value of 0.05. The Bonferroni method was applied to correct for p value multiple pairwise comparisons within each subgroup analysis for the primary and secondary outcome analyses, while exploratory analyses were corrected using the Benjamini-Hochberg procedure.
Table 2, footnote a:
Mann-Whitney U tests, with p value corrected for multiple testing using the Bonferroni procedure.
Table 2 rows used below (matched sample): Healthy controls, n = 147, median 151 (118–199); Migraine with aura, n = 172, 125 (65–180); Chronic migraine, n = 380, 128 (66–179); Headache-free (interictal), n = 210, 133 (81–177), with "Interictal vs HCs" p = 0.001. The sizes of the headache-free migraine-with-aura and chronic-migraine subgroups are not reported.
Figure 2 legend (headache-free participants):
Statistically significant differences after Bonferroni correction were assessed using the Mann-Whitney U test; **p < 0.01 and ***p < 0.001.
What the paper's own numbers show
Method. First, I compared the p values that the Results give for the headache-free analysis directly with the Abstract's bound (p < 0.001) and with the paper's significance level (two-sided 0.05). Second, Table 2 lays out three pairwise comparisons per subgroup analysis (for example MA vs MO, MA vs HCs, MO vs HCs), and footnote a says the p values are Bonferroni-corrected. So I divided each reported p by 3 (and by 2 as an alternative) to get the uncorrected value, and halved it again for a one-sided test. Third, as an approximate order-of-magnitude check only, I built a deterministic pseudo-sample for each group whose quantiles match the reported median and IQR (split-normal quantile function). I then ran a two-sided Mann-Whitney U test and applied the ×3 correction. I calibrated this model on the Table 2 comparisons whose p values are printed. The headache-free subgroup sizes are not reported, so I estimated them from each subtype's share of the matched sample (MA 172/588, CM 380/588) applied to the 210 headache-free participants. I varied the estimate from 0.7× to 1.3×, and also tried the extreme n = 210, as if every headache-free participant were in the subgroup. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Headache-free sensitivity analysis, all migraine subgroups vs HC | Abstract, Results | all p < 0.001 | contradicted by the paper's own Results: p = 0.126 (MA) and 0.090 (CM) |
| Headache-free MA vs HC | Results, Plasma CGRP Concentrations ¶2 | 143 [86–179] vs 151 [118–199]; p = 0.126 | median difference −8 pmol/L; 0.126 is 126 × 0.001 and > 0.05 |
| Headache-free CM vs HC | Results, Plasma CGRP Concentrations ¶2 | 142 [90–178] vs 151 [118–199]; p = 0.090 | median difference −9 pmol/L; 0.090 is 90 × 0.001 and > 0.05 |
| Uncorrected p, headache-free MA | derived (Table 2 footnote a) | 0.126 (corrected) | 0.042 (÷3), 0.063 (÷2); one-sided 0.021; none < 0.001 |
| Uncorrected p, headache-free CM | derived (Table 2 footnote a) | 0.090 (corrected) | 0.030 (÷3), 0.045 (÷2); one-sided 0.015; none < 0.001 |
| Headache-free (all) vs HC | Table 2, "Interictal vs HCs" | p = 0.001 | does not meet "p < 0.001" either (minor) |
| Model calibration: all migraine vs HC | Table 2 | p < 0.001 | model p = 1.5 × 10⁻⁷ |
| Model calibration: MA, CM, EM, MO, ictal vs HC | Table 2 | each p < 0.001 | model p = 1.4 × 10⁻⁴, 1.1 × 10⁻⁵, 4.0 × 10⁻⁷, 2.6 × 10⁻⁷, 4.3 × 10⁻⁹ |
| Model calibration: headache-free (interictal) vs HC | Table 2 | p = 0.001 | model p = 0.0004 (the model, if anything, overstates significance) |
| Model: headache-free MA vs HC | approx., n not reported | p = 0.126 | n = 43–79: p = 0.045–0.149; n = 210: p = 0.005; smallest over n = 10–210: 0.0047 |
| Model: headache-free CM vs HC | approx., n not reported | p = 0.090 | n = 95–177: p = 0.008–0.031; n = 210: p = 0.005; smallest over n = 10–210: 0.0052 |
The contradiction does not depend on the model. It comes from the two numbers the paper prints, 0.126 and 0.090, set against the abstract's "all p < 0.001". The model only shows that the reported medians and IQRs could not have given p < 0.001 for these two comparisons at any subgroup size up to the full 210 headache-free participants. It also shows that the Results values are of the order the summary statistics imply. For chronic migraine the model gives somewhat smaller p values than reported at the expected subgroup size (0.014 at n = 136 vs 0.090). That is within what a quantile-matched approximation can resolve, and it does not bear on the 0.001 bound.
Figure
Figure 1. Short excerpts from the Abstract (left) and from the Results paragraph on the headache-free sensitivity analysis (right), with the conflicting values in orange as printed. Both passages describe the same analysis against the same 147 controls, whose median (151 pmol/L, IQR 118–199) is identical in both. The Abstract gives "all p < 0.001" for every migraine subgroup. The Results give p = 0.126 for migraine with aura and p = 0.090 for chronic migraine and say these did not differ significantly from controls. The blue line gives the values recomputed by undoing the ×3 Bonferroni correction, 0.042 and 0.030, both still above 0.001.
Figure 2. p values for headache-free migraine with aura (MA) and chronic migraine (CM) vs healthy controls, on a log axis with dashed reference lines at 0.001 and 0.05. The shaded band is the region the Abstract claims (p < 0.001). Orange dots are the Bonferroni-corrected p values reported in the Results. Open blue circles are the uncorrected values implied by a ×3 correction. Blue lines are approximate corrected p values recomputed from the reported medians and IQRs for subgroup sizes 0.7–1.3 times the expected size. The blue tick is the extreme case in which all 210 headache-free participants are in the subgroup. No reported or recomputed value enters the region the Abstract describes.
Why it matters
The Abstract uses the headache-free analysis to say the main result holds "across all migraine subgroups" at p < 0.001. The first Discussion paragraph repeats it as having "confirmed robustness of the findings". That claim addresses an obvious concern about the main result: participants sampled during a headache might have different CGRP levels from those sampled between attacks. The Results show that, once restricted to headache-free participants, two of the four migraine subgroups (with aura and chronic) were not significantly different from controls. A reader of the Abstract alone would conclude the opposite.
The error does not overturn the paper's headline finding. In the full matched sample, migraine was lower than controls (125 [68–173] vs 151 [118–199] pmol/L, p < 0.001), and every subgroup was lower than controls at p ≤ 0.001 (Table 2). The between-subtype comparisons (all p > 0.05) and the conclusion that plasma CGRP is not elevated in migraine are also unaffected. The headache-free subgroups were smaller, so the non-significant results for MA and CM may reflect lower power, as the Discussion itself acknowledges ("Some subgroup comparisons could still have lacked statistical power"). The medians still point in the same direction. Either way, the Abstract's "all p < 0.001" is not what the paper found. The claim that subgroup-level results are robust to excluding participants with headache at sampling holds only in part.
Explanations considered
- The Abstract and Results describe different analyses. This does not account for the numbers. Both passages say "restricted to headache-free participants", and the Methods describe a single such sensitivity analysis. Both use the same controls (median 151 [118–199] in each).
- The Abstract carried over the main-analysis result. This is a plausible origin, but it does not resolve the conflict. In the unrestricted analysis the Results do say "all p ≤ 0.001" (Table 2). The Abstract sentence explicitly attributes "all p < 0.001" to the headache-free analysis, and even in the main analysis one comparison (interictal vs HCs) is p = 0.001, not < 0.001.
- Corrected vs uncorrected p values. This does not account for the numbers. Footnote a says the p values are Bonferroni-corrected. Undoing a ×3 correction gives 0.042 and 0.030, and undoing a ×2 correction gives 0.063 and 0.045, all far above 0.001.
- One-sided testing. This does not account for the numbers. The Methods specify two-sided tests. Even a one-sided uncorrected p would be 0.021 and 0.015.
- Typesetting or conversion error in the Results p values. This is excluded. The Results values are plain decimals in running text, and the prose itself says the groups "did not differ significantly". The same values appear in the PMC page, in Europe PMC's publisher XML and in PubMed (see Status).
- The Results p values are the misprint, and the Abstract is right. This is unlikely. The approximate reconstruction from the reported medians and IQRs cannot produce p < 0.001 for either comparison at any subgroup size up to 210. The same model reproduces all seven Table 2 comparisons that are significant. The Results paragraph calls the sensitivity findings only "largely consistent", and the Figure 2 legend defines a "**p < 0.01" tier. Both fit some headache-free comparisons not reaching p < 0.001, although the Figure 2 legend is worded the same as Figure 1's, so it is weak evidence.
- A correction notice or supplement resolves it. No correction was found, and the supplement could not be opened (see Status). By the paper's own description, eTables 4 and 5 cover the outlier-inclusion sensitivity analysis, not the headache-free one.
Status
- Version of record. The publisher pages at neurology.org (article HTML and PDF) returned HTTP 403 to automated access, so the publisher's own rendering could not be read. Four other copies carry identical wording and values: the live PMC article page; the Europe PMC full-text XML, where the Abstract encodes the bound as "(all
p < 0.001)" and the Results as "p = 0.126" and "p = 0.090"; the PubMed record (PMID 42284536); and the institutional research-portal abstract (research.regionh.dk). Every quotation above was confirmed verbatim against the PMC-archived text withsrc/show.py. - Supplements. The article cites eTables 1–7 in an online supplement. The JATS XML has no supplementary-material links, the Europe PMC supplementary-files endpoint returned 404, and the publisher supplement URL returned 403. The Figure 2 image (headache-free comparisons with significance stars) could not be downloaded. The Data Availability statement says data are available from the corresponding author on reasonable request, so no public data allow the tests to be rerun.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI (published 2026-07-14), and PMC and Europe PMC list no notices. A web search for corrections or comments on the article found none. The PubPeer search API returned HTTP 403, so PubPeer could not be checked directly.
- Sources. PMC article PMC13312941; Europe PMC full-text XML for PMC13312941; PubMed 42284536; research.regionh.dk publication page; Crossref metadata for 10.1212/WNL.0000000000218152.
Suggested correction
- Revise the Abstract's sensitivity-analysis sentence to match the Results. For example: "Sensitivity analyses restricted to headache-free participants showed lower median plasma CGRP in all migraine subgroups than in controls; the differences were not statistically significant for migraine with aura (p = 0.126) or chronic migraine (p = 0.090)." Give the p values for the other headache-free comparisons as computed.
- Revise the Discussion's statement that the headache-free analysis "confirmed robustness of the findings" so that it notes the two non-significant subgroup comparisons.
- Report the sizes of the headache-free subgroups (MA, MO, CM, EM) and all headache-free p values, in the text or in a supplementary table, so that Figure 2 can be read alongside them.
- Minor: the main-analysis interictal vs HCs comparison is p = 0.001 (Table 2), so any summary of the main analysis should read "all p ≤ 0.001", as the Results already do, not "p < 0.001".
Recomputation script (python recompute.py.txt)
"""Recompute the headache-free sensitivity analysis of PMC13312941
(Neurology 2026, doi:10.1212/WNL.0000000000218152, "Plasma CGRP Levels in Migraine").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs copied from the paper as archived in PMC:
Abstract, Results: headache-free sensitivity analysis, all migraine subgroups vs controls "all p < 0.001"
Results, Plasma CGRP Concentrations p1 (headache-free analysis):
HC 151 [118-199] vs migraine with aura 143 [86-179] pmol/L, p = 0.126
HC 151 [118-199] vs chronic migraine 142 [90-178] pmol/L, p = 0.090
Table 2 (matched sample; footnote a: Mann-Whitney U, Bonferroni-corrected p):
HC n=147 151 (118-199); migraine n=588 125 (68-173); CM n=380 128 (66-179); EM n=208 118 (71-167);
MA n=172 125 (65-180); MO n=416 124 (72-171); ictal n=265 116 (64-163);
headache-free (interictal) n=210 133 (81-177); interictal vs HCs p = 0.001; all other subgroup vs HCs p < 0.001
The sizes of the headache-free MA and CM subgroups are not reported.
Method.
1. Direct comparison of the reported p values with the abstract's bound (p < 0.001) and with alpha = 0.05.
2. Undo the correction: with 3 pairwise comparisons per subgroup analysis (Table 2 layout), the
uncorrected p is the corrected p / 3; a one-sided p is half of that.
3. Approximate reconstruction: build a deterministic pseudo-sample for each group whose quantiles match the
reported median and IQR (split-normal quantile function), and run a two-sided Mann-Whitney U test
(asymptotic). This is only an order-of-magnitude check; it is calibrated against the Table 2 comparisons
whose p values are known.
"""
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
ORANGE, BLUE, INK, INK2, BG, GRID = "#eb6834", "#2a78d6", "#0b0b0b", "#52514e", "#fcfcfb", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2})
# ------------------------------------------------------------------ inputs (from the paper)
HC = (118, 151, 199, 147) # q1, median, q3, n
MAIN = [("All migraine", (68, 125, 173, 588), "<0.001"), ("Chronic (CM)", (66, 128, 179, 380), "<0.001"),
("Episodic (EM)", (71, 118, 167, 208), "<0.001"), ("With aura (MA)", (65, 125, 180, 172), "<0.001"),
("Without aura (MO)", (72, 124, 171, 416), "<0.001"), ("Ictal", (64, 116, 163, 265), "<0.001"),
("Headache-free (interictal)", (81, 133, 177, 210), "0.001")]
HF = {"MA": dict(iqr=(86, 143, 179), p=0.126, share=172 / 588),
"CM": dict(iqr=(90, 142, 178), p=0.090, share=380 / 588)}
N_HF = 210
K = 3 # pairwise comparisons per subgroup analysis (e.g. MA vs MO, MA vs HC, MO vs HC)
ABSTRACT_BOUND = 0.001
def pseudo(q1, med, q3, n):
pr = (np.arange(n) + 0.5) / n
z = stats.norm.ppf(pr)
z75 = stats.norm.ppf(0.75)
x = np.where(z < 0, med + z * (med - q1) / z75, med + z * (q3 - med) / z75)
return np.clip(x, 1, None)
def mwu_p(a, b):
return stats.mannwhitneyu(a, b, alternative="two-sided", method="asymptotic").pvalue
hc = pseudo(*HC)
print("=== 1. Reported headache-free p values vs the abstract's bound ===")
for k, d in HF.items():
p = d["p"]
print(f" HF {k} vs HC: reported p = {p:.3f}; p < 0.05? {p < 0.05}; p < 0.001? {p < ABSTRACT_BOUND}; "
f"ratio to 0.001 = {p / ABSTRACT_BOUND:.0f}x; median difference = {d['iqr'][1] - HC[1]} pmol/L")
print(f" Headache-free (all) vs HC (Table 2 'Interictal vs HCs'): p = 0.001; p < 0.001? {0.001 < 0.001}")
print("\n=== 2. Undoing the Bonferroni correction and one-sidedness ===")
UNCORR = {}
for k, d in HF.items():
u = d["p"] / K
UNCORR[k] = u
print(f" HF {k}: corrected {d['p']:.3f} -> uncorrected (/{K}) {u:.3f}; (/2) {d['p'] / 2:.3f}; "
f"one-sided uncorrected {u / 2:.3f}; any < 0.001? {min(u / 2, d['p'] / 2) < 0.001}")
print("\n=== 3a. Pseudo-sample check of the method on Table 2 (Bonferroni x3) ===")
for lab, g, rep in MAIN:
p = min(1, K * mwu_p(pseudo(*g), hc))
print(f" {lab:27s} vs HC: model p = {p:.1e} reported {rep}")
print("\n=== 3b. Pseudo-sample p for the headache-free subgroups (n not reported) ===")
MODEL = {}
for k, d in HF.items():
n_exp = int(round(N_HF * d["share"]))
ns = sorted({int(round(n_exp * f)) for f in (0.7, 1.0, 1.3)} | {N_HF})
MODEL[k] = {}
for n in ns:
p = min(1, K * mwu_p(pseudo(*d["iqr"], n), hc))
MODEL[k][n] = p
tag = " (every headache-free participant)" if n == N_HF else ""
print(f" HF {k} n = {n:3d}{tag}: model corrected p = {p:.3f}")
print(f" HF {k}: n expected from matched-sample share {d['share']:.3f} x {N_HF} = {n_exp}")
grid = np.arange(10, N_HF + 1)
for k, d in HF.items():
pm = np.array([min(1, K * mwu_p(pseudo(*d["iqr"], n), hc)) for n in grid])
print(f" HF {k}: smallest model corrected p for any n in 10..{N_HF} = {pm.min():.4f} (at n = {grid[pm.argmin()]}); "
f"ever < 0.001? {bool((pm < 0.001).any())}; n where model p first <= reported {d['p']}: "
f"{grid[np.argmax(pm <= d['p'])]}")
p_int = min(1, K * mwu_p(pseudo(81, 133, 177, N_HF), hc))
print(f" Calibration: model gives interictal (n=210) p = {p_int:.4f} vs reported 0.001, "
f"i.e. the model, if anything, overstates significance")
# ------------------------------------------------------------------ figure 1: text contradiction
def wrap(s, w):
return "\n".join(textwrap.wrap(s, w))
fig = plt.figure(figsize=(7.2, 4.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.3); ax.axis("off"); ax.set_facecolor(BG)
panels = [
(0.2, "Abstract, Results",
"Sensitivity analyses restricted to headache-free participants confirmed the principal findings, "
"demonstrating lower plasma CGRP concentrations across all migraine subgroups compared with controls",
"(all p < 0.001)"),
(3.75, "Results, headache-free analysis",
"Sensitivity analyses restricted to headache-free participants ... Median plasma CGRP concentrations in "
"HCs did not differ significantly from those in participants with",
"migraine with aura ... p = 0.126\nchronic migraine ... p = 0.090"),
]
W, H, Y0 = 3.25, 2.55, 1.15
for x0, head, body, key in panels:
ax.add_patch(FancyBboxPatch((x0, Y0), W, H, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec="#c9c8c3", lw=1.2))
ax.text(x0 + 0.15, Y0 + H - 0.2, head, fontsize=10.5, fontweight="bold", va="top", color=INK)
ax.text(x0 + 0.15, Y0 + H - 0.55, wrap(body, 43), fontsize=10, va="top", color=INK2, linespacing=1.35)
ax.text(x0 + 0.15, Y0 + 0.2, key, fontsize=11.5, fontweight="bold", va="bottom", color=ORANGE,
linespacing=1.3)
ax.text(7.0, 4.12, "Same analysis, same 147 controls (HC median 151 in both)", ha="right",
fontsize=10.5, color=INK)
ax.add_patch(FancyArrowPatch((2.6, 1.0), (4.6, 1.0), connectionstyle="arc3,rad=0.35",
arrowstyle="<|-|>", mutation_scale=14, lw=2, color=INK))
ax.text(3.6, 0.55, "Conflict: 0.126 and 0.090 are not < 0.001, nor even < 0.05.", ha="center", va="center",
fontsize=10.5, color=INK)
ax.text(3.6, 0.22, f"Recomputed: undoing the x{K} Bonferroni correction gives {UNCORR['MA']:.3f} and "
f"{UNCORR['CM']:.3f}, still > 0.001.", ha="center", va="center", fontsize=10.5, color=BLUE)
ax.text(0.2, 4.12, "Orange: as printed in the paper", ha="left", fontsize=10, color=ORANGE)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ------------------------------------------------------------------ figure 2: p values on one log axis
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
ax.set_facecolor(BG)
ax.set_xscale("log")
ax.set_xlim(1e-4, 1)
ax.axvspan(1e-4, ABSTRACT_BOUND, color=ORANGE, alpha=0.10, lw=0)
ax.text(1.15e-4, 1.55, "Abstract:\nall p < 0.001", color=ORANGE, fontsize=10, va="top", fontweight="bold")
for v, lab in ((0.001, "0.001"), (0.05, "0.05")):
ax.axvline(v, color=INK2, lw=0.8, ls="--")
rows = [("Headache-free MA vs HC", "MA"), ("Headache-free CM vs HC", "CM")]
for i, (lab, k) in enumerate(rows):
y = 1 - i
ns = sorted(MODEL[k])
lo_n, hi_n = ns[0], ns[-2]
pr = [MODEL[k][n] for n in ns[:-1]]
ax.plot([min(pr), max(pr)], [y - 0.18] * 2, color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(MODEL[k][N_HF], y - 0.18, marker="|", ms=12, mew=2, color=BLUE)
ax.plot(HF[k]["p"], y + 0.12, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(UNCORR[k], y + 0.12, "o", ms=9, mfc="white", mec=BLUE, mew=2, zorder=3)
ax.text(HF[k]["p"] * 1.18, y + 0.12, f"{HF[k]['p']:.3f}", color=ORANGE, va="center", fontsize=10)
ax.text(UNCORR[k] / 1.2, y + 0.12, f"{UNCORR[k]:.3f}", color=BLUE, va="center", ha="right", fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
ax.text(MODEL[k][N_HF] / 1.15, y - 0.18, f"n = {N_HF}: {MODEL[k][N_HF]:.3f}", color=BLUE, va="center",
ha="right", fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
ax.text(max(pr) * 1.15, y - 0.18, f"n = {lo_n}-{hi_n}:\n{min(pr):.3f}-{max(pr):.3f}", color=BLUE,
va="center", ha="left", fontsize=10, bbox=dict(fc=BG, ec="none", pad=1))
ax.set_yticks([1, 0]); ax.set_yticklabels([r[0] for r in rows], fontsize=10.5, color=INK)
ax.set_ylim(-0.55, 1.65)
ax.set_xlabel("p value (log scale; dashed lines at 0.001 and 0.05)")
ax.grid(axis="x", color=GRID, lw=0.6, which="major")
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="Reported in Results (Bonferroni-corrected)"),
Line2D([], [], marker="o", ls="", ms=9, mfc="white", mec=BLUE, mew=2, label=f"Recomputed: uncorrected (p / {K})"),
Line2D([], [], color=BLUE, lw=2, marker="|", ms=10, mew=2,
label="Recomputed: approx. corrected p from median/IQR (line: subgroup n\n0.7-1.3x the expected size; tick: n = 210, all headache-free participants)")]
fig.tight_layout(rect=(0, 0.25, 1, 1))
fig.legend(handles=handles, loc="lower center", fontsize=10, frameon=False, ncol=1, bbox_to_anchor=(0.5, 0.0))
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nwrote fig1.svg/png and fig2.svg/png")
NRSF gain stated as 299% more binding sites; Table 1 counts give 199% more
Nucleic Acids Res (2008) · doi:10.1093/nar/gkn488 · PMC2532738
Summary
The paper says that its method, SISSRs, found "32, 299 and 78% more" binding sites than previous analyses of the same CTCF, NRSF and STAT1 datasets. It makes this statement in the abstract, the introduction and the Results, and the Discussion repeats it as the range "32–299%". Table 1 gives the counts behind the NRSF figure: 5813 sites from SISSRs and 1946 from the previous approach. That is (5813 − 1946)/1946 = 198.7% more, or 2.99 times as many. "299" is the new count expressed as a percentage of the old one (298.7%), while the CTCF and STAT1 figures in the same sentence are correct "% more" values (32.3% and 77.9%). The NRSF gain is therefore overstated by 100 percentage points in the paper's headline sensitivity claim. SISSRs still finds about three times as many NRSF sites, not about four times as many.
What the paper reports
Abstract:
We identified 26 814, 5813 and 73 956 binding sites for CTCF, NRSF and STAT1 proteins, respectively, which is 32, 299 and 78% more than that inferred previously for the respective proteins.
Introduction, second paragraph:
Using SISSRs, we identified a total of 26 814, 5813 and 73 956 binding sites for CTCF, NRSF and STAT1, respectively, which is 32, 299 and 78% more than that inferred previously for the respective proteins (3–5).
Results, section "ChIP-Seq datasets and binding sites":
Using an FDR threshold of 10-3, we identified a total of 26 814, 5813 and 73 956 binding sites for CTCF, NRSF and STAT1 proteins, respectively. The number of sites identified in this study is 32, 299 and 78% more than what was previously identified from the same datasets for the respective proteins (3–5) (Table 1).
Discussion, first paragraph:
SISSRs’ ability to identify binding sites with high resolution helped it achieve unprecedented sensitivity and specificity, as evidenced by its ability to identify 32–299% more binding sites than that by previous approaches using the same dataset (3–5).
Table 1 ("Data and results summary"), two of its rows:
| CTCF | NRSF | STAT1 | |
|---|---|---|---|
| Binding sites identified by SISSRs | 26 814 | 5813 | 73 956 |
| Binding sites identified by previous approaches | 20 262 | 1946 | 41 582 |
What the paper's own numbers show
Method. For each protein I took the two counts in Table 1 and computed the increase as (SISSRs − previous)/previous × 100, which is what "X% more than" means. I also computed the ratio SISSRs/previous × 100, the new count as a percentage of the old one, to see which convention each reported figure matches. As a check on the table, I compared the SISSRs counts with the subset sizes the paper gives elsewhere, and the previous NRSF count with the source the paper cites for it. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| CTCF, % more sites (26 814 vs 20 262) | Abstract; Introduction; Results | 32% | 32.3% (matches) |
| NRSF, % more sites (5813 vs 1946) | Abstract; Introduction; Results | 299% | 198.7% (rounds to 199%) |
| STAT1, % more sites (73 956 vs 41 582) | Abstract; Introduction; Results | 78% | 77.9% (matches) |
| NRSF, SISSRs count as % of previous count | — | — | 298.7% (ratio 2.987) |
| CTCF and STAT1, same ratio as % | — | — | 132.3% and 177.9% (do not match 32 and 78) |
| Range of improvement | Discussion, paragraph 1 | 32–299% | 32–199% |
| Fold change implied by "299% more" | — | 3.99 | Table 1 gives 2.987 |
| Previous NRSF count that would make "299% more" correct | — | 1946 (Table 1) | 1456.9 needed |
| SISSRs NRSF count that would make "299% more" correct | — | 5813 (Table 1) | 7765 needed |
| Top 20% of NRSF sites | Methods; Results, "Accuracy" | 1160 | 20% of 5813 = 1163 (consistent) |
| 10% of NRSF and STAT1 sites | Methods, "Motif analysis" | 581 and 7396 | 581.3 and 7395.6 (consistent) |
The CTCF and STAT1 figures fit "% more" and not "% of". The NRSF figure fits "% of" and not "% more". The subset sizes in Methods and Results are consistent with the SISSRs counts in Table 1, so the NRSF count of 5813 is used consistently across the paper. (The CTCF and STAT1 subsets of 2622 and 3825 are defined by a tag-count cut-off and are approximately, not exactly, 10% and 5% of the sites; they do not bear on the NRSF figure.)
Figure
Figure 1. Percentage increase in binding sites found by SISSRs over previous approaches, for each of the three proteins. Each row is labelled with the two Table 1 counts it is computed from (SISSRs vs previous approaches). Open orange circles are the values the paper reports in the abstract, introduction and Results; filled blue dots are the values recomputed from Table 1 as (SISSRs − previous)/previous, by recompute.py.txt. For CTCF and STAT1 the two coincide to within 0.4 percentage points. For NRSF the reported 299% lies 100 percentage points above the recomputed 198.7%; 299 matches 5813/1946 × 100 = 298.7%, the new count as a percentage of the old one. The axis starts at zero.
Why it matters
- It is one of three headline numbers in the abstract. The site counts and the percentage gains are the paper's main quantitative evidence for the sensitivity of SISSRs. The Discussion uses the "32–299%" range as the evidence for "unprecedented sensitivity and specificity". With the NRSF value corrected, the range is 32–199%.
- "299% more" implies a gain about one-third larger than the data show. 299% more means about 4.0 times as many sites; Table 1 gives 2.99 times as many. A reader comparing ChIP-Seq peak callers, or citing this paper for the size of SISSRs' gain on the NRSF dataset, would overstate it.
- The conclusion holds. SISSRs still finds about three times as many NRSF sites as the previous analysis of the same data, and the CTCF and STAT1 figures are correct. The paper's other findings (motif support for the sites, tag density as an indicator of binding affinity, and core residues in NRSF and CTCF sites) do not depend on this percentage. Table 1 prints the raw counts, so a careful reader can recompute the value.
Explanations considered
- A different convention ("299% of" rather than "299% more"). It accounts for the NRSF number but not for the sentence. 5813/1946 = 298.7%, so 299 is the ratio written as a percentage. But the same sentence gives 32 for CTCF and 78 for STAT1, which are "% more" values; as ratios they would be 132 and 178. No single convention fits all three numbers, and the text says "more than" each time.
- A different baseline count. Not supported. Table 1 gives 1946 previous NRSF sites, and the earlier NRSF ChIP-Seq study that the paper cites for it (Science 2007, doi:10.1126/science.1141319) reports NRSF binding "to 1946 locations in the human genome" in its abstract. "299% more" would require a previous count of about 1457 or a SISSRs count of about 7765. Neither appears in the paper.
- A different threshold or subset. Not supported. The Results give all three counts at the same FDR threshold of 10^-3 and point to Table 1 for the comparison. No footnote, legend or other passage gives a different NRSF count, and the Methods subset sizes (1160 and 581) are consistent with 5813.
- A conversion or typesetting artifact. Not possible here. The publisher's version of record, the PMC archive and the Europe PMC full-text XML all carry "299" in the abstract, introduction and Results and "32–299%" in the Discussion, with 5813 and 1946 in the NRSF column of Table 1.
- A correction already issued. None found (see Status).
Status
- Version of record. The DOI resolves to the publisher's page (academic.oup.com/nar/article/36/16/5221/6334562). Its abstract, Table 1 and Discussion carry the same numbers as the PMC archive: "32, 299 and 78% more", SISSRs row 26 814 / 5813 / 73 956, previous-approaches row 20 262 / 1946 / 41 582, and "32–299% more". The Europe PMC full-text XML matches as well. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The supplementary archive (Supplementary Figures 1–4, as PowerPoint files) covers fragment-length estimation, the CTCF motif against tag density, and the lengths, resolution and motif counts of binding regions from previous studies. It gives no alternative site counts and does not resolve the discrepancy.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations and no PMC or Europe PMC notices, and a Crossref query for works that update this DOI returns none. The Europe PMC article record (PMID 18684996) has an empty comments-and-corrections list, and the PubMed record lists no erratum. The publisher and PMC pages show no notice. A web search for the title with "erratum", "correction" or "PubPeer" found no notice or thread. Two checks could not be completed: PubPeer returned HTTP 403 to automated requests, and the Europe PMC search endpoint returned 503 (service unavailable). - Sources. https://doi.org/10.1093/nar/gkn488 (publisher version of record); PMC article PMC2532738; Europe PMC full-text XML and supplementary files for PMC2532738; Crossref; Europe PMC record MED/18684996; PubMed; PubPeer; abstract of doi:10.1126/science.1141319 (PubMed 17540862) for the previous NRSF count.
Suggested correction
In the abstract, the introduction (second paragraph) and the Results ("ChIP-Seq datasets and binding sites"), replace
32, 299 and 78% more
with
32, 199 and 78% more
and in the Discussion (first paragraph), replace
32–299% more binding sites
with
32–199% more binding sites
Alternatively, the NRSF result could be stated as "about three times as many binding sites as previously identified (5813 vs 1946)". Table 1 needs no change.
Recomputation script (python recompute.py.txt)
"""Recompute the "% more binding sites" figures in PMC2532738
(Nucleic Acids Res 2008, doi:10.1093/nar/gkn488).
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg and fig1.png next to this file.
Inputs are copied from the paper as archived in PubMed Central:
Table 1, row "Binding sites identified by SISSRs": 26 814 | 5813 | 73 956
Table 1, row "Binding sites identified by previous approaches": 20 262 | 1946 | 41 582
Abstract / Introduction / Results: "32, 299 and 78% more"
Discussion: "32–299% more binding sites"
Methods / Results (motif analysis): top 10% of CTCF sites (2622), top 20% of NRSF
sites (1160), top 5% of STAT1 sites (3825); "X=10% would have selected only 581
binding sites for NRSF but 7396 binding sites for STAT1".
External check: the earlier NRSF ChIP-Seq study (Science 2007,
doi:10.1126/science.1141319) reports NRSF binding "to 1946 locations" in its abstract.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
PROTEINS = ["CTCF", "NRSF", "STAT1"]
SISSRS = {"CTCF": 26814, "NRSF": 5813, "STAT1": 73956} # Table 1
PREVIOUS = {"CTCF": 20262, "NRSF": 1946, "STAT1": 41582} # Table 1
REPORTED_MORE = {"CTCF": 32, "NRSF": 299, "STAT1": 78} # text, "% more"
MEME = {"CTCF": (0.10, 2622), "NRSF": (0.20, 1160), "STAT1": (0.05, 3825)}
TEN_PCT = {"NRSF": 581, "STAT1": 7396}
ORANGE, BLUE = "#eb6834", "#2a78d6"
INK, INK2, BG, GRID = "#0b0b0b", "#52514e", "#fcfcfb", "#e4e3df"
def main():
print("1. Percent more sites, recomputed from Table 1")
more, ratio = {}, {}
for k in PROTEINS:
ratio[k] = SISSRS[k] / PREVIOUS[k]
more[k] = 100 * (SISSRS[k] - PREVIOUS[k]) / PREVIOUS[k]
print(f" {k:5s} {SISSRS[k]:6d} vs {PREVIOUS[k]:6d}: ratio {ratio[k]:.4f} | "
f"% more {more[k]:.2f} (rounds to {round(more[k])}) | "
f"new as % of old {100*ratio[k]:.2f} | reported {REPORTED_MORE[k]}% more | "
f"difference {REPORTED_MORE[k]-more[k]:+.2f} pp")
print("2. What '299% more' would require")
need_prev = SISSRS["NRSF"] / (1 + 299 / 100)
need_new = PREVIOUS["NRSF"] * (1 + 299 / 100)
print(f" previous NRSF count for 299% more over 5813: {need_prev:.1f}")
print(f" SISSRs NRSF count for 299% more over 1946: {need_new:.0f}")
print(f" '299% more' implies a fold change of {1+299/100:.2f}; Table 1 gives {ratio['NRSF']:.3f}")
print("3. Each figure under each convention")
for k in PROTEINS:
fits_more = round(more[k]) == REPORTED_MORE[k]
fits_of = round(100 * ratio[k]) == REPORTED_MORE[k]
print(f" {k:5s} reported {REPORTED_MORE[k]:3d}: matches '% more' {fits_more}; "
f"matches '% of' {fits_of}")
print("4. Discussion range '32-299%'")
lo, hi = min(more.values()), max(more.values())
print(f" recomputed range: {lo:.1f}-{hi:.1f}% -> {round(lo)}-{round(hi)}%")
print("5. Side checks that use the same SISSRs counts")
for k in PROTEINS:
f, n = MEME[k]
print(f" {k:5s} {f:.0%} of {SISSRS[k]} = {f*SISSRS[k]:.0f}; stated {n} "
f"(sites with a tag-count cut-off, = {100*n/SISSRS[k]:.2f}% of all sites)")
for k, n in TEN_PCT.items():
print(f" {k:5s} 10% of {SISSRS[k]} = {0.1*SISSRS[k]:.1f}; stated {n}")
make_figure(more)
def make_figure(more):
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"svg.fonttype": "none"})
fig, ax = plt.subplots(figsize=(7.2, 3.5), facecolor=BG)
ax.set_facecolor(BG)
ys = {"CTCF": 2, "NRSF": 1, "STAT1": 0}
for k in PROTEINS:
y = ys[k]
ax.plot([0, 330], [y, y], color=GRID, lw=1, zorder=0)
ax.plot(REPORTED_MORE[k], y, "o", ms=15, mfc="none", mec=ORANGE, mew=2.2, zorder=3)
ax.plot(more[k], y, "o", ms=8.5, color=BLUE, zorder=4)
if k != "NRSF":
ax.annotate(f"reported {REPORTED_MORE[k]}% recomputed {more[k]:.1f}%",
(max(REPORTED_MORE[k], more[k]), y), xytext=(14, 0),
textcoords="offset points", va="center", color=INK2, fontsize=10)
y = ys["NRSF"]
ax.annotate("", xy=(more["NRSF"] + 5, y), xytext=(REPORTED_MORE["NRSF"] - 7, y),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.2), zorder=2)
gap = REPORTED_MORE["NRSF"] - more["NRSF"]
ax.text((more["NRSF"] + REPORTED_MORE["NRSF"]) / 2, y + 0.17,
f"{gap:.0f} percentage points", ha="center", va="bottom", color=INK, fontsize=10)
ax.text(more["NRSF"], y - 0.2, f"recomputed\n{more['NRSF']:.1f}% more",
ha="center", va="top", color=INK, fontsize=10)
ax.text(REPORTED_MORE["NRSF"], y - 0.2,
f"reported\n{REPORTED_MORE['NRSF']}% more",
ha="center", va="top", color=INK, fontsize=10)
ax.set_yticks([ys[k] for k in PROTEINS])
ax.set_yticklabels([f"{k}\n{SISSRS[k]:,} vs {PREVIOUS[k]:,}" for k in PROTEINS],
color=INK, fontsize=10.5)
ax.set_ylim(-0.6, 2.75)
ax.set_xlim(0, 330)
ax.set_xticks(range(0, 301, 50))
ax.tick_params(axis="x", colors=INK2, labelsize=10)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Binding sites found by SISSRs, % more than previous approaches",
color=INK2, fontsize=10.5)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ax.spines.values():
s.set_visible(False)
ax.spines["bottom"].set_visible(True)
ax.spines["bottom"].set_color(INK2)
h1, = ax.plot([], [], "o", ms=12, mfc="none", mec=ORANGE, mew=2.2,
label="Reported in the text (abstract, introduction, results)")
h2, = ax.plot([], [], "o", ms=8.5, color=BLUE,
label="Recomputed from Table 1: (SISSRs − previous) / previous")
ax.legend(handles=[h1, h2], loc="lower left", bbox_to_anchor=(-0.02, 1.0),
frameon=False, fontsize=10, labelcolor=INK, ncol=1,
borderaxespad=0.2, handletextpad=0.4)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("wrote fig1.svg, fig1.png")
if __name__ == "__main__":
main()
Abstract says PZ DNA has greater twist; the paper's tables and Results show it 12° lower
Nucleic Acids Res (2017) · doi:10.1093/nar/gkx144 · PMC5397145
Summary
The abstract says that in 50 μs MD simulations the duplex with six consecutive P:Z pairs shows "greater average values of stagger, slide, rise, twist and h-rise" than a G:C control. The paper's own data show the opposite for twist. In Table 3, mean twist is 21.6 ± 15.2° for PZ/PZ and 33.6 ± 4.3° for GC/GC, so PZ is 12.0° lower. H-twist is 22.1 versus 34.3°, or 12.2° lower. The Results text agrees with the tables ("about 10° smaller"), and so do the crystal values in Table 2 and every step of the substituted region in the supplement. The abstract is the only place that gives this direction. It is also the part most readers and indexing services pick up, so a reader could come away believing that PZ-rich DNA is more twisted than natural DNA, when the paper's results show less twist and a partly A-like helix.
What the paper reports
Abstract:
In these simulations, the PZ-containing DNA duplex exhibits a significantly wider major groove and greater average values of stagger, slide, rise, twist and h-rise than observed for a ‘control’ oligonucleotide in which P:Z nucleobase pairs are replaced by G:C.
Results and Discussion, "Time convergence and statistics of structural parameters in the MD simulations", paragraph 2:
On the other hand, the mean values for twist and h-twist are about 10° smaller on average for PZ/PZ than GC/GC pairs, with a greater variance (Table 3).
The same paragraph, on stagger and slide:
with the exception of stagger, which is twice as large for the PZ than the GC helix
the average ‘slide’ value is three times larger for the oligonucleotide containing six consecutive P:Z nucleobase pairs compared with that of the control oligonucleotide
The same section, paragraph 3:
In contrast to GC, distributions for PZ are bimodal for slide, twist and h-twist. The values associated with the larger peak in each case are within the range of values reported for A-form.
Table 3, "Helix parameters calculated from the MD trajectories of the PZ and GC-containing DNA duplexes" (excerpt, mean ± SD):
PZ/PZ | … | Twist (°) 21.6 ± 15.2
GC/GC | … | Twist (°) 33.6 ± 4.3
PZ/PZ | … | H-Twist (°) 22.1 ± 16.4
GC/GC | … | H-Twist (°) 34.3 ± 4.2
Table 3, footnotes a and b:
Precision bounds are based on a 95% confidence interval with respect to the standard error of the mean (<0.08 Å and 1° for lengths and angles, respectively).
We have arbitrarily chosen to list only GC/GC and PZ/PZ dinucleotide steps; values for GG/CC and PP/ZZ are given in Supplementary Tables S5–S8.
Table 2, "Comparison of experimental and computational PZ and GC Helix parameters" (central step):
H-Twist (°) | PZ Exp. 29.4 | PZ Comp. 22.1 | GC Exp. 45.0 | GC Comp. 34.3
What the paper's own numbers show
Method. I took the eight PZ/PZ and GC/GC parameters in Table 3 that bear on the abstract sentence and computed PZ minus GC, both as signed values and as magnitudes, so that a magnitude reading of "greater" is tested as well. I then checked the same comparison in Table 2 (MD and crystal), and at each of the seven dinucleotide steps of the substituted region in Supplementary Tables S6 (twist) and S7 (h-twist), matching each PZ step to the GC step at the same position. All arithmetic is in recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Direction of twist, PZ vs GC | Abstract | "greater average values of … twist" | the tables show the opposite (rows below) |
| Twist, PZ/PZ vs GC/GC (MD) | Table 3 | 21.6 ± 15.2° vs 33.6 ± 4.3° | PZ − GC = −12.0° (lower, signed or by magnitude) |
| H-twist, PZ/PZ vs GC/GC (MD) | Table 3; Table 2 "Comp." | 22.1 ± 16.4° vs 34.3 ± 4.2° | −12.2° (lower) |
| H-twist, central step (crystal) | Table 2 "Exp." | 29.4° vs 45.0° | −15.6° (lower) |
| Size of the twist difference in the text | Results, paragraph 2 | "about 10° smaller" | 12.0° (twist) and 12.2° (h-twist); same direction as the tables |
| Twist, all seven steps of the substituted region | Tables S6 | e.g. PP/ZZ 22.6 vs GG/CC 30.0; ZA/TP 28.8 vs CA/TG 30.3 | −1.5, −3.8, −7.4, −12.0, −6.6, −3.8, −1.5°; PZ lower at 7 of 7 steps; mean −5.2° |
| H-twist, all seven steps | Table S7 | e.g. PP/ZZ 24.6 vs GG/CC 31.1 | −2.5, −3.4, −6.5, −12.2, −5.7, −3.5, −2.4°; PZ lower at 7 of 7; mean −5.2° |
| Twist difference against the stated precision | Table 3 footnote a (1° for angles) | — | 12 times the precision bound; 1.07 pooled SDs (pooled SD 11.17°) |
| Twist spread | Table 3 | SD 15.2° vs 4.3° | variance 12.5 times larger for PZ |
| Stagger | Table 3; Abstract; Results "twice as large" | −0.19 vs −0.08 Å | magnitude larger (ratio 2.38); signed value lower (−0.11 Å) |
| Slide | Table 3; Abstract; Results "three times larger" | −2.86 vs −0.78 Å | magnitude larger (ratio 3.67); signed value lower (−2.08 Å) |
| Rise | Table 3; Abstract | 3.29 vs 3.19 Å | +0.10 Å (greater) |
| H-rise | Table 3; Abstract | 3.11 vs 3.08 Å | +0.03 Å (greater) |
| Major groove width | Table 3; Abstract | 26.8 vs 21.0 Å | +5.8 Å (wider) |
Of the five parameters the abstract calls "greater", rise and h-rise are greater as signed values, and stagger and slide are greater in magnitude, since both are more negative for PZ. Twist is the only one that is lower under either reading. Twist values are all positive, so the sign question does not arise for it.
Figure
Figure 1. The two places where the paper states the direction of the PZ versus GC twist difference. Left (orange): the abstract, which lists twist among the parameters with "greater average values" for the PZ duplex. Right: the Results sentence and the Table 3 and Table 2 values for the same comparison, as printed. The blue line at the bottom of the right panel gives the differences recomputed from those table values: twist −12.0° and h-twist −12.2° (PZ minus GC). The text excerpts are quoted verbatim; the table rows are shortened to the twist and h-twist columns.
Figure 2. Differences between PZ and GC twist (filled circles, solid line, Supplementary Table S6) and h-twist (open squares, dashed line, Supplementary Table S7), recomputed for each dinucleotide step of the substituted region, with the PZ step matched to the GC step at the same position. The horizontal line marks no difference. The shaded band above it is where the values would lie if the abstract's "greater twist" were right. All fourteen differences are negative, from −1.5° at the flanking TP/ZA and ZA/TP steps to −12.0° (twist) and −12.2° (h-twist) at the central PZ/PZ step reported in Table 3. So no choice of step or average within the paper's data gives a greater twist for PZ.
Why it matters
- The abstract states the wrong direction for a reported structural result. Twist is one of the five helical parameters the abstract singles out. A reader who relies on the abstract, or on abstract-based summaries and databases, would conclude that PZ DNA is more twisted than natural DNA. The paper's data show lower twist for PZ, by about 12° at the central step and by 1.5 to 12° at every substituted step.
- The corrected direction fits the paper's own interpretation. The paper describes PZ DNA as having "properties associated with both B- and A-form DNA" (section title) and reports that the larger peak of the bimodal twist distribution lies "within the range of values reported for A-form". A-form DNA has lower twist than B-form. Lower twist for PZ therefore supports the paper's reading, and the abstract's "greater twist" points the other way.
- The difference is not marginal. The 12.0° gap is 12 times the 1° precision bound the paper gives for angles and about one pooled SD of the twist distributions.
- The analysis and the main conclusions are unaffected. The simulations, the tables, the Results text, the wider major groove, the larger slide, the A/B-form classification and the proposed nitro-group stacking mechanism do not depend on the abstract's wording. The error is confined to one word in one sentence of the abstract. Researchers working on expanded genetic alphabets (P:Z is part of the AEGIS system) and on force fields for non-natural bases are the readers most likely to take the parameter trends from the abstract. The paper has been cited 18 times (Crossref count supplied with the evidence file).
Explanations considered
- "Greater" meant in magnitude. This explains stagger and slide, which are larger in magnitude but more negative for PZ. It does not explain twist: twist is positive for both duplexes, and PZ is lower whether the values are compared signed or by magnitude (21.6 vs 33.6°).
- "Greater" refers to the spread, not the mean. PZ twist does vary much more (SD 15.2 vs 4.3°, a 12.5-fold variance). But the abstract says "greater average values", and the Results treat mean and spread as separate statements ("about 10° smaller on average … with a greater variance"). So the paper itself does not use "greater" to mean "more variable".
- A different dinucleotide step or a different average. Table 3 lists only the PZ/PZ and GC/GC steps, and footnote b points to the supplement for the others. Supplementary Tables S6 and S7 give twist and h-twist at all seven steps of the substituted region, and PZ is lower at every one of them (Figure 2). The seven-step means are also lower for PZ (twist 25.4 vs 30.7°, h-twist 27.0 vs 32.1°). The histograms in Figure 2 of the paper, which pool the PZ/PZ and PP/ZZ steps, have their larger PZ peak in the A-form range, which is again lower twist.
- Crystal versus simulation. The abstract sentence is about the simulations. Even so, the crystal values in Table 2 point the same way (h-twist 29.4 vs 45.0°, −15.6°). The paper notes that "the helical twist values are larger for both in the crystal structures", which compares crystal with MD rather than PZ with GC.
- Incomplete convergence. The 12° gap is 12 times the stated precision bound, and the Results text, which describes the same averages, gives the direction the tables give.
- A text-conversion artifact. Ruled out. The publisher's page, the PMC page, the Europe PMC JATS XML and the PubMed abstract (PMID 28334863) all carry the same abstract wording, and the twist values sit under the Twist (°) and H-Twist (°) headers in the XML.
- A correction already issued. None found (see Status).
- Side observations, not part of the finding. The Results text's "about 10°" is 12.0° and 12.2° in the tables, and its "three times larger" slide is a ratio of 3.67; both are approximations in the correct direction. Supplementary Table S6 gives the PZ/PZ twist SD as 15.3° and tilt as 0.1°, where Table 3 has 15.2° and 0.2°. None of these bears on the direction of twist.
Status
- Version of record. The publisher's page (https://doi.org/10.1093/nar/gkx144, which resolves to academic.oup.com) has the same abstract sentence, the same Results sentence and the same Table 3 values as the PMC copy. The PMC page and the Europe PMC full-text XML match it word for word and number for number. The PubMed abstract (PMID 28334863) has the same wording.
- Supplements. The only supplement,
gkx144_Supp.pdf(54 pages, from the Europe PMC supplementary bundle), was read in full for the step and helix parameters. Tables S6 and S7 give twist and h-twist at every step of the substituted region. PZ is lower at all of them, so the supplement does not support "greater twist" either. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.py(re-run 2026-10-24) returns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by fields, the PubMed record has no CommentsCorrections entries, and the publisher's page shows no notice. Searching Retraction Watch for "gkx144" returned nothing, and a web search for the title together with correction, erratum or PubPeer found nothing. PubPeer could not be queried directly, because its search requires an interactive human-verification step, and the Europe PMC search service returned 503 errors during the check. - Sources. PMC article PMC5397145; Europe PMC full-text XML and supplementary files for PMC5397145;
gkx144_Supp.pdf; https://doi.org/10.1093/nar/gkx144; Crossref; PubMed (PMID 28334863); Retraction Watch database; web search.
Suggested correction
In the abstract, replace
greater average values of stagger, slide, rise, twist and h-rise than observed for a ‘control’ oligonucleotide
with
larger magnitudes of stagger and slide, greater average rise and h-rise, and lower average twist and h-twist than observed for a ‘control’ oligonucleotide
This keeps the abstract consistent with Table 3 and with the Results text. If the authors prefer a shorter change, removing "twist" from the list of "greater" parameters and adding "and smaller twist" would correct the direction. The Results text could also give the twist difference as about 12° instead of "about 10°", to match Table 3 (12.0° for twist, 12.2° for h-twist).
Recomputation script (python recompute.py.txt)
"""Recompute the direction of the twist difference in PMC5397145
(Nucleic Acids Res 2017, doi:10.1093/nar/gkx144, "Consecutive non-natural PZ nucleobase
pairs in DNA impact helical structure as seen in 50 us molecular dynamics simulations").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC and from its only supplement
(gkx144_Supp.pdf, Europe PMC supplementary bundle):
Abstract: "greater average values of stagger, slide, rise, twist and h-rise than
observed for a 'control' oligonucleotide".
Results, "Time convergence ...", paragraph 1: "the mean values for twist and h-twist
are about 10° smaller on average for PZ/PZ than GC/GC pairs, with a greater
variance (Table 3)"; stagger "twice as large"; slide "three times larger".
Table 3 (MD, mean ± SD over the trajectory; PZ/PZ and GC/GC steps).
Table 2 (central step, crystal "Exp." and MD "Comp." values).
Supplementary Tables S6 (step parameters) and S7 (helix parameters): twist and
h-twist at each of the seven steps of the substituted region.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs: Table 3
# name: (PZ mean, PZ SD, GC mean, GC SD)
T3 = {
"Stagger (Å)": (-0.19, 0.42, -0.08, 0.37),
"Slide (Å)": (-2.86, 1.32, -0.78, 0.57),
"Rise (Å)": (3.29, 0.32, 3.19, 0.30),
"Twist (°)": (21.6, 15.2, 33.6, 4.3),
"H-Rise (Å)": (3.11, 1.15, 3.08, 0.36),
"H-Twist (°)": (22.1, 16.4, 34.3, 4.2),
"Major groove (Å)": (26.8, 1.7, 21.0, 2.4),
"Minor groove (Å)": (13.4, 1.3, 12.9, 1.1),
}
ABSTRACT_GREATER = ["Stagger (Å)", "Slide (Å)", "Rise (Å)", "Twist (°)", "H-Rise (Å)"]
# ---------------------------------------------------------------- inputs: Table 2
T2_HTWIST = {"PZ Exp.": 29.4, "PZ Comp.": 22.1, "GC Exp.": 45.0, "GC Comp.": 34.3}
# ---------------------------------------------------------------- inputs: Tables S6, S7
# seven steps of the substituted region, PZ step vs the matching GC step
STEPS = [("TP/ZA", "TG/CA"), ("PP/ZZ", "GG/CC"), ("PP/ZZ", "GG/CC"), ("PZ/PZ", "GC/GC"),
("ZZ/PP", "CC/GG"), ("ZZ/PP", "CC/GG"), ("ZA/TP", "CA/TG")]
S6_TWIST_PZ = np.array([28.8, 26.4, 22.6, 21.6, 23.4, 26.4, 28.8])
S6_TWIST_GC = np.array([30.3, 30.2, 30.0, 33.6, 30.0, 30.2, 30.3])
S7_HTWIST_PZ = np.array([30.3, 28.0, 24.6, 22.1, 25.4, 27.9, 30.4])
S7_HTWIST_GC = np.array([32.8, 31.4, 31.1, 34.3, 31.1, 31.4, 32.8])
PRECISION_ANGLE = 1.0 # Table 3 footnote a: 95% CI bound for angles, degrees
# ---------------------------------------------------------------- style
W = 7.2
BG = "#fcfcfb"
INK = "#0b0b0b"
INK2 = "#52514e"
ORANGE = "#eb6834"
BLUE = "#2a78d6"
GRID = "#e4e3df"
PANEL = "#f4f3f0"
plt.rcParams.update({
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"text.color": INK, "axes.labelcolor": INK, "xtick.color": INK, "ytick.color": INK,
"axes.edgecolor": INK2, "font.size": 10.5, "font.family": "DejaVu Sans",
"svg.fonttype": "none",
})
def report():
print("Abstract: PZ has 'greater average values of stagger, slide, rise, twist and h-rise'")
print(f"{'parameter':18s}{'PZ':>8s}{'GC':>8s}{'PZ-GC':>8s}{'|PZ|-|GC|':>11s}"
f" signed PZ>GC |PZ|>|GC| in abstract list")
for k, (p, ps, g, gs) in T3.items():
print(f"{k:18s}{p:8.2f}{g:8.2f}{p - g:8.2f}{abs(p) - abs(g):11.2f} "
f"{str(p > g):12s} {str(abs(p) > abs(g)):9s} {k in ABSTRACT_GREATER}")
print()
tw, ht = T3["Twist (°)"], T3["H-Twist (°)"]
d_tw, d_ht = tw[0] - tw[2], ht[0] - ht[2]
print(f"Twist PZ/PZ - GC/GC (Table 3): {d_tw:.1f} deg")
print(f"H-twist PZ/PZ - GC/GC (Table 3): {d_ht:.1f} deg")
print(f"Results text says 'about 10 deg smaller': actual {-d_tw:.1f} and {-d_ht:.1f} deg")
print(f"Twist difference / Table 3 precision bound ({PRECISION_ANGLE:.0f} deg): "
f"{abs(d_tw) / PRECISION_ANGLE:.0f}x")
pooled = np.sqrt((tw[1] ** 2 + tw[3] ** 2) / 2)
print(f"Twist pooled SD sqrt((15.2^2+4.3^2)/2) = {pooled:.2f} deg; "
f"|difference| / pooled SD = {abs(d_tw) / pooled:.2f}")
print(f"Twist variance ratio PZ/GC (SD^2): {tw[1] ** 2 / tw[3] ** 2:.1f}")
print(f"Stagger magnitude ratio |PZ|/|GC|: {0.19 / 0.08:.2f} (text: 'twice as large')")
print(f"Slide magnitude ratio |PZ|/|GC|: {2.86 / 0.78:.2f} (text: 'three times larger')")
print()
print(f"Table 2 H-twist, MD (Comp.): PZ {T2_HTWIST['PZ Comp.']} vs GC "
f"{T2_HTWIST['GC Comp.']}: {T2_HTWIST['PZ Comp.'] - T2_HTWIST['GC Comp.']:.1f} deg")
print(f"Table 2 H-twist, crystal (Exp.): PZ {T2_HTWIST['PZ Exp.']} vs GC "
f"{T2_HTWIST['GC Exp.']}: {T2_HTWIST['PZ Exp.'] - T2_HTWIST['GC Exp.']:.1f} deg")
print()
dt = S6_TWIST_PZ - S6_TWIST_GC
dh = S7_HTWIST_PZ - S7_HTWIST_GC
print("Supplementary Tables S6/S7, step-matched PZ - GC (deg):")
for (ps, gs), a, b in zip(STEPS, dt, dh):
print(f" {ps} vs {gs}: twist {a:+.1f} h-twist {b:+.1f}")
print(f" mean over 7 steps: twist {dt.mean():+.1f}, h-twist {dh.mean():+.1f}")
print(f" steps with PZ lower: twist {(dt < 0).sum()}/7, h-twist {(dh < 0).sum()}/7")
print(f" 7-step mean twist: PZ {S6_TWIST_PZ.mean():.1f}, GC {S6_TWIST_GC.mean():.1f}")
print(f" 7-step mean h-twist: PZ {S7_HTWIST_PZ.mean():.1f}, GC {S7_HTWIST_GC.mean():.1f}")
return d_tw, d_ht, dt, dh
def m(v):
"""Signed number with a true minus sign."""
return f"{v:+.1f}".replace("-", "−")
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200)
plt.close(fig)
def textbox(ax, x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc=PANEL, ec=edge, lw=1.5, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, fontsize=10.5,
fontweight="bold", va="top", color=INK)
ax.text(x + 0.02, y + h - 0.13, body, transform=ax.transAxes, fontsize=10,
va="top", color=INK, linespacing=1.45)
def fig1(d_tw, d_ht):
fig = plt.figure(figsize=(W, 4.3))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
ax.text(0.03, 0.965, "The abstract and the paper's own tables give opposite directions "
"for twist", fontsize=11.5, fontweight="bold", va="top", transform=ax.transAxes)
# left panel: abstract
textbox(ax, 0.03, 0.16, 0.35, 0.72, "Abstract",
"“…the PZ-containing DNA duplex\n"
"exhibits a significantly wider\n"
"major groove and greater\n"
"average values of stagger,\n"
"slide, rise, twist and h-rise\n"
"than observed for a ‘control’\n"
"oligonucleotide…”", edge=ORANGE)
ax.text(0.05, 0.19, "▲ twist: PZ greater", transform=ax.transAxes,
fontsize=10.5, fontweight="bold", color=ORANGE, va="bottom")
# right panel: results text and tables
textbox(ax, 0.51, 0.16, 0.46, 0.72, "Results text, Table 3 and Table 2",
"Results: “…the mean values for twist\n"
"and h-twist are about 10° smaller on\n"
"average for PZ/PZ than GC/GC pairs…”\n"
"Table 3 (MD), PZ/PZ vs GC/GC:\n"
" Twist 21.6 ± 15.2° vs 33.6 ± 4.3°\n"
" H-Twist 22.1 ± 16.4° vs 34.3 ± 4.2°\n"
"Table 2 (crystal) H-Twist: 29.4° vs 45.0°",
edge=INK2)
ax.text(0.53, 0.245, f"PZ − GC, recomputed: twist {m(d_tw)}°,", transform=ax.transAxes,
fontsize=10.5, fontweight="bold", color=BLUE, va="bottom")
ax.text(0.53, 0.19, f"h-twist {m(d_ht)}° ▼ PZ lower", transform=ax.transAxes,
fontsize=10.5, fontweight="bold", color=BLUE, va="bottom")
ax.add_patch(FancyArrowPatch((0.385, 0.52), (0.505, 0.52), transform=ax.transAxes,
arrowstyle="<->", mutation_scale=14, lw=2, color=INK2))
ax.text(0.445, 0.565, "opposite\ndirection", ha="center", va="bottom", fontsize=10.5,
transform=ax.transAxes, linespacing=1.2)
ax.text(0.445, 0.475, "same\nPZ vs GC\ncomparison", ha="center", va="top", fontsize=10,
color=INK2, transform=ax.transAxes, linespacing=1.2)
ax.text(0.03, 0.04, "Orange: the abstract's statement. Grey box: text and table values "
"as printed.\nBlue: PZ − GC differences recomputed from those table values.",
fontsize=10, color=INK2, transform=ax.transAxes, va="bottom", linespacing=1.3)
save(fig, "fig1")
def fig2(dt, dh):
fig, ax = plt.subplots(figsize=(W, 3.7))
fig.subplots_adjust(left=0.10, right=0.97, top=0.79, bottom=0.17)
x = np.arange(len(STEPS))
ax.axhspan(0, 6, color=ORANGE, alpha=0.08, lw=0)
ax.axhline(0, color=INK2, lw=1)
ax.text(6.35, 0.6, "Abstract: PZ twist greater (values would lie here)", ha="right",
va="bottom", fontsize=10, color=ORANGE, fontweight="bold")
ax.plot(x - 0.08, dt, color=BLUE, lw=2, marker="o", ms=8, label="twist (Table S6)")
ax.plot(x + 0.08, dh, color=BLUE, lw=2, ls="--", marker="s", ms=8, mfc=BG, mew=2,
label="h-twist (Table S7)")
ax.set_xticks(x)
ax.set_xticklabels([f"{p}\nvs {g}" for p, g in STEPS], fontsize=10)
ax.set_xlim(-0.5, 6.5)
ax.set_ylim(-14, 6)
ax.set_ylabel("PZ minus GC, degrees")
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("right", "top"):
ax.spines[s].set_visible(False)
ax.annotate(f"central step: {m(dt[3])}° (Table 3 value)", xy=(3 - 0.08, dt[3]),
xytext=(3.6, -12.6), fontsize=10, color=INK, va="center",
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.legend(loc="lower left", frameon=False, fontsize=10, title="recomputed PZ − GC",
title_fontsize=10)
fig.text(0.03, 0.95, "PZ twist is lower than GC twist at every step of the substituted "
"region", fontsize=11.5, fontweight="bold", va="top")
fig.text(0.03, 0.885, "Step-matched differences, PZ step minus the matching GC step, "
"Supplementary Tables S6 and S7", fontsize=10, color=INK2, va="top")
save(fig, "fig2")
if __name__ == "__main__":
d_tw, d_ht, dt, dh = report()
fig1(d_tw, d_ht)
fig2(dt, dh)
print("\nwrote fig1.svg/png and fig2.svg/png to", HERE)
VirtualSweet's reported cross-validation accuracy is impossible given its sensitivity and specificity; supplement shows best-fold values
Nucleic Acids Res (2021) · doi:10.1093/nar/gkab292 · PMC8262722
Summary
Table 1 reports 10-fold cross-validation results for the VirtualSweet model as accuracy 0.88, sensitivity 0.97 and specificity 0.96, and the text repeats the 88%. Accuracy is a prevalence-weighted average of sensitivity and specificity, so these three values cannot describe the same evaluation: with sensitivity 0.97 and specificity 0.96, accuracy must lie between 0.955 and 0.975 even allowing for rounding, and must be at least 0.920 even if the metrics are averages over folds. The paper's supplementary file S2 shows where the row comes from. Each Table 1 cross-validation entry matches the value S2 labels "best" for that metric, and these "best" values cannot all come from one fold. S2 gives the 10-fold mean VirtualSweet accuracy as 0.765 ± 0.114, not 0.88, so the cross-validation performance stated in Table 1, the text and the Conclusions is higher than the cross-validation average the paper's own supplement reports.
What the paper reports
Table 1, "Performance statistics for the VirtualTaste models applied to cross-validation and external validation sets" (excerpt, as archived in PMC):
VirtualTaste models | VirtualTaste models | VirtualSweet | VirtualBitter | VirtualSour
Data sampling method | Data sampling method | SMOTETC | SMOTE VDM | AugRandOS
Cross-validation | Prediction accuracy | 0.88 | 0.94 | 0.98
| Sensitivity | 0.97 | 0.94 | 0.94
| Specificity | 0.96 | 0.92 | 0.97
| ROC–AUC | 0.99 | 0.97 | 0.97
| F-measure | 0.87 | 0.94 | 0.98
External validation | Prediction accuracy | 0.89 | 0.90 | 0.97
| Sensitivity | 0.86 | 0.88 | 0.80
| Specificity | 0.92 | 0.97 | 0.99
| ROC-AUC | 0.95 | 0.96 | 0.99
| F-measure | 0.88 | 0.88 | 0.84
Materials and Methods, "VirtualTaste prediction models", VirtualSweet subsection:
The model achieved a prediction accuracy of 88% on 10-fold cross-validation and 89% on external validation. The AUC–ROC values of cross-validation and external validation are 0.95 and 0.99, respectively (Table 1).
Materials and Methods, "Performance evaluation of the VirtualTaste models", first paragraph:
Each model was validated using 8 different sampling methods for 10-fold cross-validation (22). The data was split into 10-fold, keeping the ratio of active and inactive data constant as published in our previous work (22).
Same section, definition of the metrics:
Prediction Accuracy is defined as the ability of a model to correctly predict the total number of activities and inactives. Sensitivity is the trained model's ability to correctly predict the positive (taste) class. Specificity is defined as the trained model's ability to correctly predict the negative (non-taste) class.
Same section, second paragraph:
The performance statistics both on cross-validation as well as external validation for the top- performing VirtualTaste models are summarized in Table 1. The models achieved a prediction accuracy of 88% and above on both, cross-validation and external validation. The specificity and sensitivity of the VirtualTaste models are balanced and scored 90% and above, except for the sensitivity of VirtualSweet (86%), VirtualBitter (88%) and VirtualSour (80%) on the external validation set.
Materials and Methods, "VirtualTaste prediction models", first paragraph:
The standard deviation and average of the accuracy and other parameters loss are computed (See Supplementary data).
Abstract:
The VirtualTaste models achieved an overall accuracy of 90% and an AUC of 0.98 in 10-fold cross-validation and in an independent test set.
Conclusions and Future Updates, first paragraph:
The models achieved a prediction accuracy of 88% and above both on cross-validation and external validation.
Supplementary file S2.xlsx (Sheet1), VirtualSweet block, row "SMOTETC", internal cross-validation cells (int. ROC-AUC, int. Acc, int. Sens, int. Spec, int. F-Measure):
0.833+/-0.124 | best:0.990 · 0.765+/-0.114 | best:0.877 · 0.740+/-0.185 | best:0.977 · 0.791+/-0.165 | best:0.963 · 0.751+/-0.129 | best:0.870
What the paper's own numbers show
Method. For a single binary evaluation with a share p of positives, accuracy = p × sensitivity + (1 − p) × specificity, so accuracy must lie between sensitivity and specificity for every p. I applied this check to all six rows of Table 1, widening every bound by ±0.005 for two-decimal rounding. For the case where the three metrics are each averaged over the 10 folds, I used the per-fold inequality accuracy ≥ min(sensitivity, specificity) ≥ sensitivity + specificity − 1, which carries over to the averages. I then compared each Table 1 entry with the supplementary file S2.xlsx for the sampling method Table 1 names (SMOTETC, SMOTEVDM and AugRandOS). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| VirtualSweet CV accuracy | Table 1; VirtualSweet subsection ("88%") | 0.88 | must lie in 0.960 to 0.970 for any prevalence (0.955 to 0.975 with rounding) given sensitivity 0.97 and specificity 0.96 |
| Prevalence needed for accuracy 0.88 | derived | — | p = (0.88 − 0.96) ÷ (0.97 − 0.96) = −8.0, which is impossible |
| Gap after allowing for rounding | derived | ≤ 0.885 | lowest possible accuracy 0.955; gap 0.070 |
| Lower bound if metrics are fold averages | derived | — | mean accuracy ≥ 0.965 + 0.955 − 1 = 0.920 |
| Balanced accuracy | derived | — | (0.97 + 0.96) ÷ 2 = 0.965 |
| VirtualSweet CV F-measure | Table 1 | 0.87 | fits sensitivity 0.97 and specificity 0.96 only at a positive share of 0.133, where accuracy is 0.961; at p = 0.5 F1 is 0.965 |
| Misclassified training compounds (n = 1608) | Datasets section | accuracy 0.88 → 193 | specificity and sensitivity ≥ 0.96 → at most 64 |
| Other five rows (CV Bitter, CV Sour, external Sweet, Bitter, Sour) | Table 1 | accuracy 0.94, 0.98, 0.89, 0.90, 0.97 | each within its sensitivity–specificity range; CV Sour (0.98 vs 0.94 and 0.97) only at the rounding edge (0.975) |
| VirtualSweet CV entries vs S2 "best" | Table 1 vs S2, SMOTETC | 0.88, 0.97, 0.96, 0.99, 0.87 | S2 "best": 0.877, 0.977, 0.963, 0.990, 0.870 (accuracy, sensitivity, specificity, AUC, F) |
| Can the S2 "best" values come from one fold? | S2, SMOTETC | — | No: with sensitivity 0.977 and specificity 0.963 a single fold's accuracy lies in 0.963 to 0.977, not 0.877 |
| VirtualSweet CV, 10-fold mean ± SD | S2, SMOTETC | — | accuracy 0.765 ± 0.114, sensitivity 0.740 ± 0.185, specificity 0.791 ± 0.165, AUC 0.833 ± 0.124, F 0.751 ± 0.129; the mean accuracy lies between the mean sensitivity and specificity, so these are mutually consistent |
| VirtualBitter and VirtualSour CV accuracy | Table 1 vs S2 | 0.94; 0.98 | S2 "best" 0.946 and 0.978; 10-fold means 0.845 ± 0.068 and 0.954 ± 0.092 |
| External-validation rows | Table 1 vs S2 | all 15 cells | all agree with the S2 external values to within 0.01, except VirtualBitter specificity (below) |
| VirtualSweet ROC-AUC, CV / external | text vs Table 1 vs S2 | text 0.95 / 0.99; Table 1 0.99 / 0.95 | S2: CV best 0.990 (mean 0.833), external 0.951; the text has the two values the other way round |
Table 1 thus reproduces the S2 "best" values for cross-validation and the single S2 values for external validation. For VirtualSweet, the "best" entries cannot all come from the same fold, because a fold with sensitivity 0.977 and specificity 0.963 cannot have accuracy 0.877. They must have been selected metric by metric. That explains why the printed row is impossible as a single evaluation. It also means the Table 1 "Cross-validation" row is not a cross-validation estimate in the usual sense, which is the average over folds. S2 gives that average for VirtualSweet as accuracy 0.765, which is 0.115 (about one fold SD) below the 0.88 in Table 1 and the text. S2 does not define "best", and it is not always the largest fold value: in two S2 rows the "best" value is below the mean (VirtualSour randOS accuracy, best 0.970 against mean 0.977, and, at the third decimal, VirtualSour AugRandOS sensitivity, 0.94 against 0.941).
The two-decimal Table 1 values follow the S2 values sometimes by rounding (0.877 to 0.88) and sometimes by truncation (sensitivity 0.977 printed as 0.97; VirtualBitter accuracy 0.946 printed as 0.94).
Figure
Figure 1. The VirtualSweet 10-fold cross-validation row of Table 1. Orange points are the printed sensitivity (0.97), specificity (0.96) and accuracy (0.88). The blue bar is the range that accuracy must take, for any share of sweet compounds and allowing ±0.005 for rounding: 0.955 to 0.975. The dashed blue line marks the weaker bound that still holds if accuracy, sensitivity and specificity are each averaged over the 10 folds: accuracy ≥ 0.920. The printed 0.88 is outside both.
Figure 2. Internal 10-fold cross-validation for the three models, using the sampling method named in Table 1. Orange points are Table 1's "Cross-validation" entries. Open diamonds are the values supplementary file S2 labels "best". Open grey circles with bars are the S2 10-fold mean ± SD (the VirtualSour bar is cut at 1.0). Every Table 1 entry sits at the S2 "best" value, not at the mean. For VirtualSweet the means (accuracy 0.765, sensitivity 0.740, specificity 0.791, AUC 0.833, F-measure 0.751) are 0.115 to 0.230 below the printed values.
Why it matters
- A results table contains an impossible combination. As printed, the VirtualSweet cross-validation row cannot be a single set of results. A reader cannot tell from the article which of accuracy, sensitivity or specificity to trust. The supplement answers this: none of them is the cross-validation average.
- Cross-validation performance is overstated relative to the paper's own supplement. Table 1 and the text present the best value of each metric, picked separately, as the "10-fold cross-validation" result. The S2 averages are lower for all five VirtualSweet and all five VirtualBitter metrics, and for VirtualSour accuracy and F-measure: accuracy 0.765 instead of 0.88 (VirtualSweet), 0.845 instead of 0.94 (VirtualBitter) and 0.954 instead of 0.98 (VirtualSour). VirtualSour sensitivity and specificity are essentially unchanged. A best-of-ten value, picked for each metric separately, is an optimistic summary of a cross-validation run, and the VirtualSweet folds vary widely (accuracy SD 0.114).
- Summary statements rest on these numbers. "88% and above on both, cross-validation and external validation" (Methods and Conclusions) does not hold for the S2 cross-validation means of VirtualSweet (0.765) and VirtualBitter (0.845). Nor does "The specificity and sensitivity ... are balanced and scored 90% and above" for VirtualSweet sensitivity and specificity (0.740 and 0.791) or VirtualBitter specificity (0.792). The abstract's "overall accuracy of 90% and an AUC of 0.98 in 10-fold cross-validation and in an independent test set" compares with S2 cross-validation means across the three models of 0.855 for accuracy and 0.914 for AUC. The article does not say how its 90% and 0.98 were computed; the six Table 1 accuracies average 0.927 and the six AUCs 0.972.
- What is not affected. The external-validation results, which match S2 and are internally consistent for all three models, are the better guide to how the server performs on new compounds: accuracy 0.89, 0.90 and 0.97 for sweet, bitter and sour. The web server itself, the paper's main contribution, is unaffected. The finding concerns how internal-validation performance is reported.
Explanations considered
- Rounding. Does not explain it. Sensitivity of at least 0.965 and specificity of at least 0.955 force accuracy to at least 0.955. The printed 0.88 stands for at most 0.885.
- Class imbalance or SMOTETC oversampling. Does not explain it. Accuracy, sensitivity and specificity from one confusion matrix satisfy accuracy = p × sensitivity + (1 − p) × specificity for every share p. Oversampling changes p, not the identity. Solving for p gives −8.
- Metrics averaged over the 10 folds instead of pooled. Does not explain it. Each fold has accuracy ≥ sensitivity + specificity − 1, so the mean accuracy is at least 0.920. The supplement's actual fold means (0.765, 0.740, 0.791) are consistent with each other, but they are not the values in Table 1.
- Balanced accuracy or another definition of accuracy. Does not explain it. The Methods define accuracy as the share of all actives and inactives predicted correctly. Balanced accuracy would be 0.965.
- A typo in the single cell 0.88. Unlikely. The 88% appears three times in the text, the F-measure of 0.87 sits near it, and S2 gives "best" accuracy 0.877. The cell is a faithful copy of S2. What makes the row impossible is that its entries were taken from different folds.
- The Table 1 values are labelled correctly and S2 is a different analysis. Not supported. The sampling methods match, the external values agree to within 0.01 in 14 of 15 cells, and all 15 cross-validation cells but two match an S2 "best" value. The Methods refer the reader to the supplementary data for the "standard deviation and average".
- A conversion or typesetting artifact. Ruled out. The publisher's page, the PMC page and the PMC archive give the same values, and the 88% also appears in the prose.
- Separate inconsistencies, not part of the main finding. (1) The VirtualSweet text gives cross-validation and external AUC as 0.95 and 0.99, while Table 1 and S2 give 0.99 (best fold) and 0.95. (2) VirtualBitter specificity reads 0.92 for cross-validation and 0.97 for external validation in Table 1, while S2 gives a cross-validation best of 0.973 and an external value of 0.919, which looks like a swap of the two cells. (3) VirtualSour cross-validation AUC is 0.97 in Table 1 and in the text, while S2 gives a mean of 0.996 and a best of 0.998.
Status
- Version of record. The publisher's page (https://doi.org/10.1093/nar/gkab292, resolving to academic.oup.com/nar/article/49/W1/W679/6255693) shows the same Table 1 values, the same "88% on 10-fold cross-validation" sentence and the same AUC wording as the PMC page and the PMC archive. Every quotation from the article was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The supplementary archive
gkab292_supplemental_files.zip(retrieved through the Europe PMC supplementary-files endpoint) containsS2.xlsxandsupplementary_data(S1).docx. S1 gives the data-set sizes (VirtualSweet 1608 training, 403 test) and a comparison with other servers. S2 gives the internal 10-fold results as "mean+/-SD | best:x" for every sampling method, and single external values. The S2 values used here are transcribed intorecompute.py.txtand were checked cell by cell against the spreadsheet (25 rows, no mismatch). S2 resolves where the impossible row comes from, as described above. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations and no PMC or Europe PMC notices. The Crossref record has no update-to or updated-by entries. The PubMed record (PMID 33905509) has no CommentsCorrections entries, and a PubMed search for the title with "erratum" or "correction" returned no hits. Neither the publisher's page nor the PMC page shows a notice. A Retraction Watch site search and a web search found nothing. Two checks could not be completed: the Europe PMC REST search returned HTTP 503 on repeated attempts, and PubPeer refused automated access (HTTP 403), so the absence of a PubPeer thread is not confirmed. - Sources. PMC article PMC8262722; publisher's page at doi:10.1093/nar/gkab292; supplementary files S1 and S2 (Europe PMC); Crossref; PubMed; Retraction Watch; PubPeer (not reachable).
Suggested correction
In Table 1, either (a) replace the cross-validation rows with the 10-fold mean ± SD from S2. For VirtualSweet that is accuracy 0.77 ± 0.11, sensitivity 0.74 ± 0.19, specificity 0.79 ± 0.17, ROC-AUC 0.83 ± 0.12 and F-measure 0.75 ± 0.13. For VirtualBitter it is 0.85, 0.90, 0.79, 0.91 and 0.86; for VirtualSour 0.95, 0.94, 0.97, 1.00 (0.996) and 0.94. Or (b) keep the current values but relabel the rows as the best single-fold value of each metric, chosen separately, and state that the values in one column do not come from the same fold. Option (a) gives the conventional cross-validation estimate.
Correspondingly, in the VirtualSweet subsection, replace "a prediction accuracy of 88% on 10-fold cross-validation" with the mean (77%), and correct "The AUC–ROC values of cross-validation and external validation are 0.95 and 0.99, respectively" to the Table 1/S2 values (cross-validation 0.83 as a mean, or 0.99 as the best fold; external 0.95). Revise the "88% and above" and "balanced and scored 90% and above" sentences in the Methods and Conclusions, and the abstract's "overall accuracy of 90% and an AUC of 0.98 in 10-fold cross-validation", to match whichever summary is adopted. The authors may also wish to check the VirtualBitter specificity cells (Table 1: 0.92 cross-validation, 0.97 external; S2: 0.973 best, 0.919 external) and the VirtualSour cross-validation AUC (Table 1 and text 0.97; S2 0.996 mean, 0.998 best).
Recomputation script (python recompute.py.txt)
"""Recompute the consistency checks for Table 1 of PMC8262722
(Nucleic Acids Res 2021, doi:10.1093/nar/gkab292,
"VirtualTaste: a web server for the prediction of organoleptic properties of chemical compounds").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper:
Table 1 (all 30 cells), as archived in PMC and on the publisher's page.
Text, VirtualSweet subsection: CV accuracy 88%, external 89%; "AUC-ROC values of
cross-validation and external validation are 0.95 and 0.99, respectively".
Datasets: VirtualSweet training set 1608 molecules.
Supplementary file S2.xlsx (Sheet1, gkab292_supplemental_files.zip): internal 10-fold CV
results given as "mean+/-SD | best:x" and single external values, for the three sampling
methods that Table 1 names (VirtualSweet SMOTETC, VirtualBitter SMOTEVDM, VirtualSour
AugRandOS), plus every other row of the sheet for the "best >= mean" check.
"""
import os
import re
import numpy as np
from scipy.optimize import brentq
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import Patch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
METRICS = ["acc", "sens", "spec", "auc", "f"]
NAMES = {"acc": "Accuracy", "sens": "Sensitivity", "spec": "Specificity",
"auc": "ROC-AUC", "f": "F-measure"}
# ------------------------------------------------------------------ Table 1 (paper)
T1 = {
("Sweet", "CV"): dict(acc=0.88, sens=0.97, spec=0.96, auc=0.99, f=0.87),
("Bitter", "CV"): dict(acc=0.94, sens=0.94, spec=0.92, auc=0.97, f=0.94),
("Sour", "CV"): dict(acc=0.98, sens=0.94, spec=0.97, auc=0.97, f=0.98),
("Sweet", "Ext"): dict(acc=0.89, sens=0.86, spec=0.92, auc=0.95, f=0.88),
("Bitter", "Ext"): dict(acc=0.90, sens=0.88, spec=0.97, auc=0.96, f=0.88),
("Sour", "Ext"): dict(acc=0.97, sens=0.80, spec=0.99, auc=0.99, f=0.84),
}
TEXT_SWEET_AUC = dict(CV=0.95, Ext=0.99) # VirtualSweet subsection
N_SWEET_TRAIN = 1608
H = 0.005 # half-width of two-decimal rounding
# ------------------------------------------------------------------ S2.xlsx (supplement)
# internal CV: (mean, SD, best); external: single value. Columns int./ex. for each metric.
S2_RAW = {
"Sweet": {
"no_sampling": "0.817+/-0.134 | best:0.995;0.952;0.747+/-0.124 | best:0.876;0.878;0.707+/-0.225 | best:1.000;0.884;0.786+/-0.170 | best:0.963;0.873;0.719+/-0.161 | best:0.874;0.877",
"RandOS": "0.825+/-0.137 | best:0.992;0.951;0.767+/-0.126 | best:0.877;0.876;0.733+/-0.209 | best:1.000;0.879;0.802+/-0.162 | best:0.951;0.873;0.747+/-0.154 | best:0.876;0.874",
"AugRandOS": "0.823+/-0.139 | best:0.990;0.950;0.751+/-0.133 | best:0.877;0.873;0.717+/-0.213 | best:1.000;0.879;0.785+/-0.166 | best:0.963;0.868;0.730+/-0.162 | best:0.866;0.872",
"RandUS": "0.864+/-0.107 | best:0.990;0.956;0.787+/-0.115 | best:0.943;0.893;0.690+/-0.237 | best:0.988;0.879;0.883+/-0.027 | best:0.925;0.907;0.740+/-0.173 | best:0.946;0.890",
"AugRandUS": "0.815+/-0.142 | best:0.983;0.953;0.749+/-0.136 | best:0.912;0.881;0.702+/-0.236 | best:0.988;0.874;0.796+/-0.182 | best:0.962;0.887;0.719+/-0.176 | best:0.917;0.878",
"SMOTETC": "0.833+/-0.124 | best:0.990;0.951;0.765+/-0.114 | best:0.877;0.893;0.740+/-0.185 | best:0.977;0.864;0.791+/-0.165 | best:0.963;0.922;0.751+/-0.129 | best:0.870;0.888",
"SMOTEVDM": "0.830+/-0.127 | best:0.993;0.951;0.760+/-0.121 | best:0.870;0.873;0.732+/-0.188 | best:1.000;0.869;0.789+/-0.164 | best:0.951;0.877;0.745+/-0.138 | best:0.871;0.871",
},
"Bitter": {
"nosampling": "0.876+/-0.084 | best:0.962;0.961;0.783+/-0.078 | best:0.907;0.920;0.745+/-0.148 | best:0.926;0.876;0.811+/-0.109 | best:0.973;0.952;0.740+/-0.105 | best:0.882;0.902",
"augRandUS": "0.893+/-0.059 | best:0.984;0.949;0.816+/-0.071 | best:0.945;0.885;0.849+/-0.103 | best:0.963;0.927;0.783+/-0.116 | best:0.964;0.855;0.821+/-0.074 | best:0.944;0.873",
"randUS": "0.912+/-0.044 | best:0.970;0.959;0.825+/-0.078 | best:0.909;0.895;0.796+/-0.151 | best:0.944;0.905;0.854+/-0.047 | best:0.907;0.887;0.812+/-0.106 | best:0.911;0.879",
"augRandOS": "0.919+/-0.082 | best:0.985;0.959;0.842+/-0.081 | best:0.953;0.901;0.887+/-0.094 | best:0.973;0.869;0.798+/-0.117 | best:0.973;0.925;0.849+/-0.078 | best:0.952;0.881",
"randOS": "0.918+/-0.074 | best:0.990;0.954;0.838+/-0.079 | best:0.946;0.907;0.894+/-0.085 | best:0.986;0.876;0.783+/-0.114 | best:0.959;0.930;0.847+/-0.075 | best:0.945;0.889",
"kMedoids1": "0.918+/-0.032 | best:0.953;0.960;0.831+/-0.064 | best:0.898;0.904;0.807+/-0.141 | best:0.926;0.898;0.856+/-0.029 | best:0.891;0.909;0.820+/-0.096 | best:0.901;0.888",
"kMedoids2": "0.882+/-0.082 | best:0.975;0.959;0.803+/-0.084 | best:0.918;0.901;0.814+/-0.126 | best:0.963;0.912;0.792+/-0.121 | best:0.964;0.892;0.803+/-0.093 | best:0.914;0.887",
"SMOTETC": "0.914+/-0.063 | best:0.978;0.956;0.846+/-0.067 | best:0.939;0.892;0.899+/-0.051 | best:0.973;0.876;0.793+/-0.122 | best:0.959;0.903;0.856+/-0.058 | best:0.939;0.873",
"SMOTEVDM": "0.912+/-0.055 | best:0.975;0.956;0.845+/-0.068 | best:0.946;0.901;0.897+/-0.039 | best:0.946;0.876;0.792+/-0.124 | best:0.973;0.919;0.855+/-0.057 | best:0.946;0.882",
},
"Sour": {
"nosampling": "0.945+/-0.046 | best:1.000;0.994;0.943+/-0.074 | best:0.992;0.977;0.624+/-0.148 | best:0.889;0.800;0.967+/-0.082 | best:1.000;0.992;0.672+/-0.175 | best:0.941;0.842",
"augRandUS": "1.000+/-0.000 | best:1.000;0.986;1.000+/-0.000 | best:1.000;0.308;1.000+/-0.000 | best:1.000;1.000;1.000+/-0.000 | best:1.000;0.252;1.000+/-0.000 | best:1.000;0.179",
"randUS": "0.957+/-0.040 | best:1.000;0.976;0.868+/-0.063 | best:0.941;0.865;0.882+/-0.114 | best:1.000;1.000;0.858+/-0.126 | best:1.000;0.854;0.870+/-0.063 | best:0.941;0.526",
"augRandOS": "0.996+/-0.012 | best:0.998;0.994;0.954+/-0.092 | best:0.978;0.977;0.941+/-0.177 | best:0.94;0.800;0.968+/-0.080 | best:0.97;0.992;0.944+/-0.125 | best:0.98;0.842",
"randOS": "0.998+/-0.004 | best:1.000;0.991;0.977+/-0.048 | best:0.970;0.970;1.000+/-0.000 | best:1.000;0.800;0.954+/-0.096 | best:1.000;0.984;0.980+/-0.041 | best:1.000;0.800",
"kMedoids1": "0.929+/-0.060 | best:1.000;0.984;0.829+/-0.118 | best:1.000;0.910;0.854+/-0.121 | best:1.000;0.900;0.801+/-0.196 | best:1.000;0.911;0.837+/-0.108 | best:1.000;0.600",
"kMedoids2": "0.899+/-0.064 | best:0.986;0.987;0.789+/-0.106 | best:0.941;0.902;0.843+/-0.132 | best:1.000;0.900;0.738+/-0.221 | best:1.000;0.902;0.802+/-0.089 | best:0.941;0.581",
"SMOTETC": "0.999+/-0.002 | best:1.000;0.991;0.971+/-0.049 | best:1.000;0.970;0.992+/-0.024 | best:1.000;0.800;0.950+/-0.099 | best:1.000;0.984;0.974+/-0.042 | best:1.000;0.800",
"SMOTEVDM": "0.999+/-0.002 | best:1.000;0.989;0.969+/-0.049 | best:1.000;0.970;0.990+/-0.027 | best:1.000;0.800;0.947+/-0.098 | best:1.000;0.984;0.971+/-0.042 | best:1.000;0.800",
},
}
S2_ORDER = ["auc", "acc", "sens", "spec", "f"] # S2 column order (Bal-Acc column omitted)
CHOSEN = {"Sweet": "SMOTETC", "Bitter": "SMOTEVDM", "Sour": "augRandOS"} # Table 1 row 2
def parse(raw):
cells = raw.split(";")
out = {}
for i, m in enumerate(S2_ORDER):
mean, sd, best = map(float, re.match(r"([\d.]+)\+/-([\d.]+) \| best:([\d.]+)", cells[2 * i]).groups())
out[m] = dict(mean=mean, sd=sd, best=best, ext=float(cells[2 * i + 1]))
return out
S2 = {mod: {k: parse(v) for k, v in rows.items()} for mod, rows in S2_RAW.items()}
S2["Sour"]["AugRandOS"] = S2["Sour"]["augRandOS"]
def r2(x):
return float(f"{x:.2f}")
# ------------------------------------------------------------------ 1. Table 1 rows: acc within [sens, spec]?
print("=== 1. Table 1: accuracy must lie between sensitivity and specificity (acc = p*sens + (1-p)*spec) ===")
print(f"{'model':7s}{'set':4s} acc sens spec allowed acc (rounding +-0.005) consistent")
for (m, s), v in T1.items():
lo = min(v["sens"], v["spec"]) - H
hi = max(v["sens"], v["spec"]) + H
ok = (v["acc"] + H >= lo) and (v["acc"] - H <= hi)
print(f"{m:7s}{s:4s}{v['acc']:.2f} {v['sens']:.2f} {v['spec']:.2f} [{lo:.3f}, {hi:.3f}] {ok}")
se, sp, acc = T1[("Sweet", "CV")]["sens"], T1[("Sweet", "CV")]["spec"], T1[("Sweet", "CV")]["acc"]
p = np.linspace(0, 1, 10001)
a = p * se + (1 - p) * sp
print(f"\nVirtualSweet CV: accuracy over prevalence p in [0,1] ranges {a.min():.3f} to {a.max():.3f}")
print(f" prevalence needed for accuracy 0.88: p = (0.88-{sp})/({se}-{sp}) = {(acc - sp) / (se - sp):.1f}")
print(f" with rounding: lowest possible accuracy = min(sens,spec) - 0.005 = {min(se, sp) - H:.3f}; "
f"highest value that rounds to 0.88 = {acc + H:.3f}; gap = {min(se, sp) - H - (acc + H):.3f}")
fold_lb = (se - H) + (sp - H) - 1
print(f" fold-averaged metrics: per fold acc_k >= sens_k + spec_k - 1, so mean acc >= {se - H:.3f} + {sp - H:.3f} - 1 = {fold_lb:.3f}")
print(f" balanced accuracy (sens+spec)/2 = {(se + sp) / 2:.3f}")
def f1(pp, se=se, sp=sp):
tp = pp * se
fp = (1 - pp) * (1 - sp)
prec = tp / (tp + fp)
return 2 * prec * se / (prec + se)
p_f = brentq(lambda x: f1(x) - 0.87, 1e-4, 0.99)
print(f" F1 = 0.87 with sens {se}/spec {sp} needs prevalence p = {p_f:.3f}; accuracy at that p = {p_f * se + (1 - p_f) * sp:.3f}")
print(f" F1 at p = 0.5 = {f1(0.5):.3f}")
print(f" training set n = {N_SWEET_TRAIN}: accuracy 0.88 -> {round(N_SWEET_TRAIN * (1 - acc))} misclassified; "
f"sens/spec >= 0.96 -> at most {round(N_SWEET_TRAIN * (1 - sp))} misclassified")
sour = T1[("Sour", "CV")]
print(f"VirtualSour CV: accuracy {sour['acc']} vs upper bound max(sens,spec)+0.005 = {max(sour['sens'], sour['spec']) + H:.3f}; "
f"lowest value that rounds to 0.98 = {sour['acc'] - H:.3f} (consistent only at the rounding edge)")
# ------------------------------------------------------------------ 2. Table 1 vs S2
print("\n=== 2. Table 1 vs supplement S2.xlsx for the sampling method Table 1 names ===")
print(f"{'model':7s}{'metric':12s}{'T1 CV':>6s}{'S2 best':>8s}{'S2 mean+-SD':>14s} | {'T1 Ext':>6s}{'S2 ext':>7s}")
for mod, meth in CHOSEN.items():
d = S2[mod][meth]
for m in METRICS:
print(f"{mod:7s}{NAMES[m]:12s}{T1[(mod, 'CV')][m]:6.2f}{d[m]['best']:8.3f}"
f"{d[m]['mean']:8.3f}+-{d[m]['sd']:.3f} | {T1[(mod, 'Ext')][m]:6.2f}{d[m]['ext']:7.3f}")
print()
print("Cells where Table 1 differs from the corresponding S2 value by more than 0.01:")
for mod, meth in CHOSEN.items():
d = S2[mod][meth]
for m in METRICS:
for setname, key in (("CV", "best"), ("Ext", "ext")):
t, s = T1[(mod, setname)][m], d[m][key]
if abs(t - s) > 0.01:
print(f" {mod} {setname} {NAMES[m]}: Table 1 {t:.2f}, S2 {key} {s:.3f} "
f"(S2 CV best {d[m]['best']:.3f}, S2 CV mean {d[m]['mean']:.3f}, S2 ext {d[m]['ext']:.3f})")
sw = S2["Sweet"]["SMOTETC"]
print("\nVirtualSweet SMOTETC 'best' values as one fold: sens best {:.3f}, spec best {:.3f} -> a single fold would have "
"accuracy in [{:.3f}, {:.3f}], but acc best = {:.3f}; so the 'best' values are not from one fold".format(
sw["sens"]["best"], sw["spec"]["best"], min(sw["sens"]["best"], sw["spec"]["best"]),
max(sw["sens"]["best"], sw["spec"]["best"]), sw["acc"]["best"]))
print("VirtualSweet SMOTETC 10-fold means: acc {:.3f} lies within [sens {:.3f}, spec {:.3f}] -> consistent; "
"implied share of actives about {:.2f}".format(
sw["acc"]["mean"], sw["sens"]["mean"], sw["spec"]["mean"],
(sw["spec"]["mean"] - sw["acc"]["mean"]) / (sw["spec"]["mean"] - sw["sens"]["mean"])))
print(f"VirtualSweet: Table 1 CV accuracy 0.88 vs S2 10-fold mean {sw['acc']['mean']:.3f} "
f"(difference {0.88 - sw['acc']['mean']:.3f}, {(0.88 - sw['acc']['mean']) / sw['acc']['sd']:.2f} fold SDs)")
print(f"VirtualSweet AUC: text CV {TEXT_SWEET_AUC['CV']} / Ext {TEXT_SWEET_AUC['Ext']}; Table 1 CV "
f"{T1[('Sweet', 'CV')]['auc']} / Ext {T1[('Sweet', 'Ext')]['auc']}; S2 CV best {sw['auc']['best']:.3f}, "
f"CV mean {sw['auc']['mean']:.3f}, Ext {sw['auc']['ext']:.3f}")
print("\n10-fold means for the three Table 1 models (S2):")
for m in METRICS:
vals = [S2[mod][CHOSEN[mod]][m]["mean"] for mod in CHOSEN]
print(f" {NAMES[m]:12s} Sweet {vals[0]:.3f} Bitter {vals[1]:.3f} Sour {vals[2]:.3f} | mean of three {np.mean(vals):.3f}")
cv_acc_t1 = [T1[(m, 'CV')]['acc'] for m in CHOSEN]
cv_auc_t1 = [T1[(m, 'CV')]['auc'] for m in CHOSEN]
ex_acc_t1 = [T1[(m, 'Ext')]['acc'] for m in CHOSEN]
ex_auc_t1 = [T1[(m, 'Ext')]['auc'] for m in CHOSEN]
print(f"Table 1 means: CV acc {np.mean(cv_acc_t1):.3f}, CV AUC {np.mean(cv_auc_t1):.3f}, Ext acc {np.mean(ex_acc_t1):.3f}, "
f"Ext AUC {np.mean(ex_auc_t1):.3f}; all six acc {np.mean(cv_acc_t1 + ex_acc_t1):.3f}, all six AUC {np.mean(cv_auc_t1 + ex_auc_t1):.3f}")
low = [(mod, NAMES[m], S2[mod][CHOSEN[mod]][m]["mean"]) for mod in CHOSEN for m in ("sens", "spec")
if S2[mod][CHOSEN[mod]][m]["mean"] < 0.90]
print("10-fold mean sensitivity/specificity below 0.90 (text: 'balanced and scored 90% and above'):",
", ".join(f"{a} {b} {c:.3f}" for a, b, c in low))
print("\nS2 rows where a 'best' value is below its 10-fold mean (so 'best' is not always the fold maximum):")
for mod, rows in S2_RAW.items():
for meth in rows:
d = S2[mod][meth]
for m in METRICS:
if d[m]["best"] < d[m]["mean"]:
print(f" {mod} {meth} {NAMES[m]}: best {d[m]['best']:.3f} < mean {d[m]['mean']:.3f}")
# ------------------------------------------------------------------ figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GRAY = "#eb6834", "#2a78d6", "#e4e3df", "#8a8984"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT, "ytick.color": TXT,
"axes.edgecolor": SEC, "svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# ---- Figure 1: the impossible row
fig, ax = plt.subplots(figsize=(7.2, 3.3), facecolor=BG)
style(ax)
rows = ["Sensitivity", "Specificity", "Accuracy"]
y = {r: i for i, r in enumerate(rows[::-1])}
ax.scatter([se], [y["Sensitivity"]], s=90, color=ORANGE, zorder=4)
ax.scatter([sp], [y["Specificity"]], s=90, color=ORANGE, zorder=4)
ax.scatter([acc], [y["Accuracy"]], s=90, color=ORANGE, zorder=4)
ax.annotate(f"{se:.2f}", (se, y["Sensitivity"]), xytext=(0, 9), textcoords="offset points", ha="center", color=ORANGE)
ax.annotate(f"{sp:.2f}", (sp, y["Specificity"]), xytext=(0, 9), textcoords="offset points", ha="center", color=ORANGE)
ax.annotate(f"reported {acc:.2f}", (acc, y["Accuracy"]), xytext=(0, 9), textcoords="offset points", ha="center", color=ORANGE)
# allowed interval for accuracy (any prevalence, with rounding)
lo, hi = min(se, sp) - H, max(se, sp) + H
ax.plot([lo, hi], [y["Accuracy"]] * 2, color=BLUE, lw=6, solid_capstyle="butt", zorder=3)
ax.annotate(f"possible: {lo:.3f}–{hi:.3f}", (hi, y["Accuracy"]), xytext=(4, -20), textcoords="offset points",
ha="right", color=BLUE)
ax.plot([fold_lb, 1.0], [y["Accuracy"] - 0.62] * 2, color=BLUE, lw=2, ls=(0, (4, 2)), zorder=3)
ax.scatter([fold_lb], [y["Accuracy"] - 0.62], marker="|", s=120, color=BLUE, lw=2, zorder=3)
ax.annotate(f"≥ {fold_lb:.3f} even if averaged over folds", (fold_lb, y["Accuracy"] - 0.62), xytext=(-6, 0),
textcoords="offset points", ha="right", va="center", color=BLUE)
ax.set_yticks(list(y.values()))
ax.set_yticklabels(list(y.keys()))
ax.set_ylim(-0.9, 2.6)
ax.set_xlim(0.5, 1.0)
ax.set_xlabel("Value (VirtualSweet, Table 1, 10-fold cross-validation row)")
leg = [Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="Reported in Table 1"),
Line2D([], [], color=BLUE, lw=6, label="Accuracy implied by Table 1's sensitivity and specificity")]
ax.legend(handles=leg, loc="upper left", frameon=False, fontsize=10)
fig.tight_layout()
fig.savefig(os.path.join(HERE, "fig1.svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200, facecolor=BG)
plt.close(fig)
# ---- Figure 2: Table 1 CV row vs supplement S2 (mean +- SD and "best")
fig, ax = plt.subplots(figsize=(7.2, 4.8), facecolor=BG)
style(ax)
items = [("Sweet", m) for m in METRICS] + [("Bitter", "acc"), ("Sour", "acc")]
SHORT = {"acc": "accuracy", "sens": "sensitivity", "spec": "specificity", "auc": "ROC-AUC", "f": "F-measure"}
labels = [f"Sweet: {SHORT[m]}" for m in METRICS] + ["Bitter: accuracy", "Sour: accuracy"]
ys = np.arange(len(items))[::-1].astype(float)
ys[-2:] -= 0.5
for (mod, m), yy in zip(items, ys):
d = S2[mod][CHOSEN[mod]][m]
ax.plot([max(d["mean"] - d["sd"], 0.4), min(d["mean"] + d["sd"], 1.0)], [yy - 0.12] * 2, color=SEC, lw=2, zorder=2)
ax.scatter([d["mean"]], [yy - 0.12], s=70, facecolor=BG, edgecolor=SEC, lw=2, zorder=3)
ax.scatter([d["best"]], [yy - 0.12], s=80, marker="D", facecolor=BG, edgecolor=TXT, lw=1.5, zorder=3)
ax.scatter([T1[(mod, "CV")][m]], [yy + 0.14], s=80, color=ORANGE, zorder=4)
ax.annotate(f"{d['mean']:.3f}", (d["mean"], yy - 0.12), xytext=(0, -14), textcoords="offset points",
ha="center", color=SEC, fontsize=10)
ax.set_yticks(ys)
ax.set_yticklabels(labels)
ax.set_xlim(0.4, 1.0)
ax.set_ylim(ys.min() - 0.8, ys.max() + 0.5)
ax.set_xlabel("Value, internal 10-fold cross-validation (sampling method named in Table 1)")
leg = [Line2D([], [], marker="o", ls="", color=ORANGE, ms=9, label="Table 1, 'Cross-validation'"),
Line2D([], [], marker="D", ls="", mfc=BG, mec=TXT, mew=1.5, ms=8, label="Supplement S2, 'best'"),
Line2D([], [], marker="o", ls="-", color=SEC, mfc=BG, mec=SEC, mew=2, ms=8, lw=2,
label="Supplement S2, 10-fold mean ± SD")]
ax.legend(handles=leg, loc="lower center", bbox_to_anchor=(0.35, 1.0), frameon=False, fontsize=10, ncol=2,
handletextpad=0.4, columnspacing=1.0)
fig.tight_layout()
fig.savefig(os.path.join(HERE, "fig2.svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, "fig2.png"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract gives the headline smoking-initiation CI as 0.92–1.61; the body gives 0.92–1.16
PLoS Biol (2020) · doi:10.1371/journal.pbio.3000973 · PMC7728259
Summary
This two-sample Mendelian randomization study estimates the effect of genetically predicted smoking on multiple sclerosis (MS) risk. Its abstract gives the main smoking-initiation estimate as OR 1.03, 95% CI 0.92–1.61, but the Results and the Discussion give the same inverse-variance weighted (IVW) estimate as OR 1.03, 95% CI 0.92 to 1.16. On the log-odds scale, an OR of 1.03 with a lower limit of 0.92 implies an upper limit of 1.153, which matches 1.16. An interval of 0.92–1.61 would be centred on 1.22, and its upper arm would be 3.95 times as long as its lower arm, whereas every other interval in the paper is symmetric to within rounding. The published peer-review history shows that a reviewer raised this value and the response to reviewers called it a typo for 1.16, yet the version of record still prints 1.61. The null conclusion is unchanged, but the abstract interval is about 2.4 times too wide on the log scale. It also contains the observational OR of 1.5 that the paper's own Discussion says the MR result contradicts.
What the paper reports
Abstract:
There was no clear evidence for an effect of smoking on the risk of developing MS (smoking initiation: odds ratio [OR] 1.03, 95% confidence interval [CI] 0.92–1.61; lifetime smoking: OR 1.10, 95% CI 0.87–1.40). These findings suggest that smoking does not have a detrimental consequence on MS susceptibility.
Results, "Smoking initiation", first sentence:
The inverse-variance weighted (IVW) MR estimate (odds ratio [OR] 1.03, 95% confidence interval [CI] 0.92 to 1.16) revealed no strong evidence for a causal effect of the genetic risk of smoking initiation on the incidence of MS (Fig 2).
Results, "Smoking initiation", the sensitivity estimates:
the consistent MR–Egger estimate (OR 1.13, 95% CI 0.67 to 1.91)
gave a similar causal estimate (OR 1.05, 95% CI 0.93 to 1.17)
The MR-PRESSO outlier corrected causal estimate was 1.040 (95% CI 1.040 to 1.041).
Results, "Lifetime smoking":
estimate (OR 1.10, 95% CI 0.87 to 1.40) revealed no strong evidence for a causal effect of the genetic risk of lifetime smoking
which still overlapped the null (OR 1.34, 95% CI 0.49 to 3.65)
Discussion:
Although a small effect cannot be entirely excluded, the relatively narrow CIs, particularly for smoking initiation (0.92 to 1.16), make a clinically relevant effect less likely.
This contradicts previously reported observational studies that show an association with MS risk among smokers, compared to nonsmokers, of a meta-analysed effect estimate OR of 1.5
Methods, "Mendelian randomization analyses":
For smoking initiation, the ORs are expressed per unit increase in log odds of ever smoking regularly (smoking initiation)
Peer-review history published with the article (Decision Letter 1, reviewer comment, from the Europe PMC full-text XML):
Is the confidence interval for smoking cessation in the abstract correct? 1.03 (0.92, 1.61) - either the lower or upper bound seem incorrect.
Response to reviewers (attachment pbio.3000973.s002, "Response to reviewers .pdf", dated 2 Oct 2020):
Response: Thank you for pointing out this typo. The confidence intervals should instead read (0.92, 1.16). We have corrected this in the abstract of the revised manuscript.
The reviewer wrote "smoking cessation", but the only abstract interval matching "1.03 (0.92, 1.61)" is the smoking-initiation estimate, and the response treats it as that one.
What the paper's own numbers show
Method. MR estimates such as IVW, MR-Egger and MR-RAPS are computed on the log-odds scale, and their 95% intervals are exp(b ± 1.96·SE). Three things follow. ln(OR) should sit at the midpoint of ln(lower) and ln(upper). The two arms, ln(OR/lower) and ln(upper/OR), should imply the same SE, so their ratio should be close to 1. And the OR and the lower limit together fix the upper limit at exp(2·ln OR − ln lower). I applied these checks to every interval the text reports with an OR. MR-PRESSO is left out because its lower arm (1.040 to 1.040) is zero at the printed precision. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Smoking initiation IVW, upper 95% limit | Abstract | 1.61 | 1.153, implied by OR 1.03 and lower limit 0.92 |
| Smoking initiation IVW, upper 95% limit | Results; Discussion | 1.16 | 1.153, consistent |
| Implied upper limit over rounding of OR (1.025–1.035) and lower limit (0.915–0.925) | derived | 1.16 / 1.61 | 1.136–1.171: contains 1.16, excludes 1.61 |
| Log-midpoint of 0.92–1.61 | Abstract | OR 1.03 | 1.217 |
| Log-midpoint of 0.92–1.16 | Results; Discussion | OR 1.03 | 1.033 |
| OR at which 0.92–1.61 would be symmetric (over rounding of the lower limit) | derived | OR 1.03 | 1.214–1.220 |
| Upper arm ÷ lower arm, 0.92–1.61 | Abstract | — | 3.95 |
| Upper arm ÷ lower arm, 0.92–1.16 | Results; Discussion | — | 1.05 |
| Upper arm ÷ lower arm, MR-Egger 1.13 (0.67–1.91) | Results | — | 1.00 (log-midpoint 1.131) |
| Upper arm ÷ lower arm, MR-RAPS 1.05 (0.93–1.17) | Results | — | 0.89 (log-midpoint 1.043) |
| Upper arm ÷ lower arm, lifetime IVW 1.10 (0.87–1.40) | Abstract; Results | — | 1.03 (log-midpoint 1.104) |
| Upper arm ÷ lower arm, lifetime MR-Egger 1.34 (0.49–3.65) | Results | — | 1.00 (log-midpoint 1.337) |
| SE of log OR implied by the interval | Abstract (0.92–1.61) vs Results (0.92–1.16) | — | 0.1428 vs 0.0591, a ratio of 2.41 |
| Observational OR 1.5 inside the interval? | Discussion; Abstract vs Results | — | inside 0.92–1.61; outside 0.92–1.16 |
| OR 1 inside the interval? | both | — | yes for both |
| Approximate Wald P for OR 1.03 with SE 0.0591 | derived | not reported | 0.62 |
Every interval in the body is symmetric on the log scale to within rounding (arm ratios 0.89–1.05), as is the lifetime-smoking interval in the abstract. The abstract's smoking-initiation interval is the only exception. Its OR and lower limit match the Results exactly, and its upper limit has the same digits as the Results' 1.16 in reverse order.
Figure
Figure 1. The smoking-initiation IVW estimate on a log odds-ratio axis. Orange: the two versions printed in the paper. The Abstract prints OR 1.03 (0.92–1.61), and the Results and Discussion print OR 1.03 (0.92–1.16). Blue: the interval implied by the printed OR 1.03 and lower limit 0.92 under a symmetric interval on the log scale, with an upper limit of 1.153. The open diamond marks the log-scale midpoint of the Abstract interval (1.22), far from the printed OR. The solid line marks OR = 1. The dashed line marks the observational meta-analysed OR of 1.5 that the Discussion compares against. It lies inside the Abstract interval and outside the Results interval.
Figure 2. For each odds ratio and 95% CI printed in the text, the ratio of the upper arm to the lower arm on the log scale, ln(upper/OR) ÷ ln(OR/lower). A Wald interval gives 1. The six intervals from the Results and the Abstract's lifetime-smoking interval fall between 0.89 and 1.05, which is what rounding to two decimals produces. The Abstract's smoking-initiation interval, 0.92–1.61, gives 3.95.
Why it matters
The abstract is where most readers meet this result, and the 0.92–1.61 interval tells a different story from 0.92–1.16 in three ways.
- Precision. The body's interval implies a log-scale SE of about 0.059 and rules out an effect larger than about 16% per unit increase in the log odds of smoking initiation. The abstract's interval implies an SE of about 0.143, 2.4 times larger, and is compatible with effects of up to 61%.
- The paper's own argument. The Discussion argues that "the relatively narrow CIs, particularly for smoking initiation (0.92 to 1.16), make a clinically relevant effect less likely". The abstract's interval does not support that argument. Its upper limit is wide enough that a reader of the abstract could reasonably conclude a sizeable harmful effect has not been excluded, and the abstract's conclusion that "smoking does not have a detrimental consequence on MS susceptibility" then rests on a weaker interval than the analysis produced.
- Comparison with observational studies. The Discussion contrasts the MR result with an observational meta-analysed OR of 1.5. The body's interval excludes 1.5, while the abstract's includes it. The exposures are on different scales (per unit log odds of smoking initiation in the MR analysis, ever versus never smoking in the observational studies), so this comparison is approximate. It is still the comparison the paper invites.
The direction and statistical significance do not change. The point estimate is 1.03 in both places, and both intervals contain 1. Figures, tables, the lifetime-smoking estimate and the analyses themselves are unaffected. The error is in how one number in the abstract is reported.
Explanations considered
- A different estimator or sensitivity analysis. This does not account for the numbers. The abstract presents the figure as the smoking-initiation estimate, and its OR (1.03) and lower limit (0.92) are identical to the Results IVW estimate. No other smoking-initiation estimate in the text has an upper limit of 1.61: MR-Egger is 0.67–1.91, MR-RAPS 0.93–1.17 and MR-PRESSO 1.040–1.041. Supplementary Table B (MR estimates, S1 Data) does not contain 1.61 either.
- A different exposure. This does not account for the numbers. The lifetime-smoking estimate is reported separately in the same abstract sentence, and it matches the Results (0.87–1.40 in both).
- Rounding. This does not account for the numbers. Over the full rounding range of the OR (1.025–1.035) and the lower limit (0.915–0.925), the implied upper limit is 1.136–1.171. That range contains 1.16, but not 1.61.
- An asymmetric interval method. This is unlikely to account for the numbers. The paper reports IVW estimates from the TwoSampleMR package, whose intervals are symmetric on the log scale. Every other interval in the paper is symmetric to within rounding (arm ratios 0.89–1.05). To fit 0.92–1.61, the upper arm would have to be 3.95 times the lower arm for this one estimate only.
- A conversion or archiving error. This is excluded. The PMC archive copy, the Europe PMC full-text XML, the publisher's HTML page and the publisher's PDF all print 0.92–1.61 in the abstract and 0.92 to 1.16 in the Results and Discussion. The medRxiv preprint (10.1101/2020.06.24.20138834, version 1, 2020-06-24) also prints 0.92-1.61 in its abstract.
- The abstract value is the intended one and the body is wrong. This is excluded by the paper's own numbers and by the peer-review record. Only 1.16 is consistent with OR 1.03 and lower limit 0.92. The response to reviewers states that the interval "should instead read (0.92, 1.16)".
The digits of 1.61 are those of 1.16 in reverse order, which fits a transcription slip. The response to reviewers describes the value as a typo. No further explanation of the cause is offered here.
Status
- Version of record. I checked the publisher's HTML (journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3000973, fetched 2026-10-24), which gives the abstract as 0.92–1.61 and the Results and Discussion as 0.92 to 1.16. The version-of-record check for this write-up also extracted the publisher's printable PDF (13 pages), which carries the same strings. The Europe PMC full-text XML and the PMC archive copy match. Every quotation from the article above was confirmed verbatim against the PMC archive text with
src/show.py. The peer-review quotations come from the Europe PMC XML (sub-article pbio.3000973.r002, Decision Letter 1) and from the response attachment pbio.3000973.s002 (7 pages), which I downloaded from PLOS and read. - Peer-review record. A reviewer queried "1.03 (0.92, 1.61)" in Decision Letter 1 (dated 19 Aug 2020). The response dated 2 Oct 2020 calls it a typo for (0.92, 1.16) and says it was "corrected ... in the abstract of the revised manuscript". The published abstract still reads 0.92–1.61. The same response says the Discussion sentence citing 0.92-1.16 was added in that revision.
- Supplements. S1 Data (pbio.3000973.s001, DOCX: Figs A–H, Tables A–H) does not report the headline IVW interval, and neither 1.61 nor 1.16 appears in it. Its only 0.92 is a SNP allele frequency. Attachment s002 is the response to reviewers described above. The XML gives s003 the same filename and file size, so it was not read separately.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyfound one Crossref update, a "new_version" record dated 2020-12-10 that points to the article's own DOI (the update from uncorrected proof to final version). It found no PMC or Europe PMC notices. The PLOS article page has no correction notice, and its Reader Comments page shows "Reader Comments (0)". A web search for a correction or erratum returned only the article and its mirrors. The PubPeer API refused automated access (HTTP 403, "Human verification required"), so I could not check PubPeer. - Sources. PMC7728259 (PMC archive text); Europe PMC full-text XML for PMC7728259, including the peer-review sub-articles; journals.plos.org article page, Reader Comments page and supplementary files s001 and s002; Crossref; the medRxiv API record for 10.1101/2020.06.24.20138834.
Suggested correction
- In the Abstract, change "95% confidence interval [CI] 0.92–1.61" to "95% confidence interval [CI] 0.92–1.16" for smoking initiation. This is the correction the response to reviewers says was intended.
- No change is needed to the Results, the Discussion, the figures, the lifetime-smoking estimate or the conclusions as stated in the body. They already use 0.92 to 1.16.
Recomputation script (python recompute.py.txt)
"""Recompute the smoking-initiation confidence interval checks for PMC7728259
(PLoS Biol 2020, doi:10.1371/journal.pbio.3000973,
"Little evidence for an effect of smoking on multiple sclerosis risk: A Mendelian Randomization study").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs, copied from the paper as archived in PMC (all ORs on the paper's own scale):
Abstract: smoking initiation OR 1.03, 95% CI 0.92-1.61; lifetime smoking OR 1.10, 95% CI 0.87-1.40
Results, Smoking init.: IVW OR 1.03, 95% CI 0.92 to 1.16; MR-Egger OR 1.13 (0.67 to 1.91);
MR-RAPS OR 1.05 (0.93 to 1.17); MR-PRESSO 1.040 (1.040 to 1.041)
Results, Lifetime: IVW OR 1.10 (0.87 to 1.40); MR-Egger OR 1.34 (0.49 to 3.65)
Discussion: "particularly for smoking initiation (0.92 to 1.16)"; observational meta-analysed OR 1.5
Method: a 95% Wald interval on the log-OR scale is exp(b +/- 1.96 SE). So ln(OR) should sit at the
midpoint of ln(lower) and ln(upper), each arm should imply the same SE, and given OR and the lower
limit the upper limit is implied: exp(2 ln OR - ln lower).
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
import matplotlib.ticker as mticker # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.isf(0.025)
# (key, label, OR, lower, upper, location)
ROWS = [
("si_abs", "Smoking initiation, IVW", 1.03, 0.92, 1.61, "Abstract"),
("si_res", "Smoking initiation, IVW", 1.03, 0.92, 1.16, "Results; Discussion"),
("si_egg", "Smoking initiation, MR-Egger", 1.13, 0.67, 1.91, "Results"),
("si_raps", "Smoking initiation, MR-RAPS", 1.05, 0.93, 1.17, "Results"),
("lt_ivw", "Lifetime smoking, IVW", 1.10, 0.87, 1.40, "Abstract; Results"),
("lt_egg", "Lifetime smoking, MR-Egger", 1.34, 0.49, 3.65, "Results"),
]
OBS_OR = 1.5 # Discussion: observational meta-analysed OR
def check(orr, lo, hi):
mid = math.exp((math.log(lo) + math.log(hi)) / 2)
se_lo = (math.log(orr) - math.log(lo)) / Z
se_hi = (math.log(hi) - math.log(orr)) / Z
se_ci = (math.log(hi) - math.log(lo)) / (2 * Z)
hi_impl = math.exp(2 * math.log(orr) - math.log(lo))
return dict(mid=mid, se_lo=se_lo, se_hi=se_hi, ratio=se_hi / se_lo, se_ci=se_ci, hi_impl=hi_impl)
R = {}
print("=== 1. Log-symmetry of every interval reported with an OR in the text ===")
print(f" {'interval':45s} {'log-midpoint':>12s} {'SE lower arm':>12s} {'SE upper arm':>12s} {'ratio':>6s} {'implied upper':>13s}")
for key, lab, orr, lo, hi, loc in ROWS:
c = check(orr, lo, hi)
R[key] = dict(lab=lab, orr=orr, lo=lo, hi=hi, loc=loc, **c)
print(f" {lab + ' ' + str(orr) + ' (' + str(lo) + '-' + str(hi) + ')':45s} {c['mid']:12.3f} "
f"{c['se_lo']:12.4f} {c['se_hi']:12.4f} {c['ratio']:6.2f} {c['hi_impl']:13.3f} [{loc}]")
print(" MR-PRESSO 1.040 (1.040-1.041) is left out: its lower arm is zero at the printed precision.")
others = [R[k]["ratio"] for k in R if k != "si_abs"]
print(f" arm SE ratio, all intervals except the abstract's: {min(others):.2f}-{max(others):.2f}; "
f"abstract: {R['si_abs']['ratio']:.2f}")
a, b = R["si_abs"], R["si_res"]
print("\n=== 2. The headline smoking-initiation IVW estimate ===")
print(f" upper limit implied by OR 1.03 and lower 0.92: {b['hi_impl']:.3f} (Results prints 1.16; Abstract prints 1.61)")
print(f" log-midpoint of 0.92-1.16: {b['mid']:.3f}; of 0.92-1.61: {a['mid']:.3f} (reported OR 1.03)")
print(f" SE(log OR) from 0.92-1.16: {b['se_ci']:.4f}; from 0.92-1.61: {a['se_ci']:.4f}; "
f"ratio {a['se_ci'] / b['se_ci']:.2f}")
print(f" width on log scale: 0.92-1.16 {math.log(1.16 / 0.92):.3f}; 0.92-1.61 {math.log(1.61 / 0.92):.3f}")
print(f" '1.16' with its two decimals reversed: 1.{'16'[::-1]}")
print(f" observational OR {OBS_OR} inside 0.92-1.16: {0.92 <= OBS_OR <= 1.16}; inside 0.92-1.61: {0.92 <= OBS_OR <= 1.61}")
print(f" OR 1 inside both intervals: {0.92 <= 1 <= 1.16 and 0.92 <= 1 <= 1.61}")
p_res = 2 * stats.norm.sf(abs(math.log(1.03)) / b["se_ci"])
print(f" approximate Wald P for OR 1.03 with SE {b['se_ci']:.4f}: {p_res:.2f}")
print("\n=== 3. Rounding: range of the implied upper limit ===")
his = [math.exp(2 * math.log(o) - math.log(l))
for o in np.linspace(1.025, 1.035, 21) for l in np.linspace(0.915, 0.925, 21)]
print(f" OR 1.025-1.035, lower 0.915-0.925: implied upper {min(his):.3f}-{max(his):.3f}")
print(f" 1.16 inside that range: {min(his) <= 1.16 <= max(his)}; "
f"1.61 inside: {min(his) <= 1.61 <= max(his)}")
ors_needed = [math.sqrt(l * 1.61) for l in (0.915, 0.925)]
print(f" OR that 0.92-1.61 would need to be log-symmetric: {ors_needed[0]:.3f}-{ors_needed[1]:.3f} (reported 1.03)")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": TXT, "axes.edgecolor": SEC, "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"xtick.labelsize": 10, "ytick.labelsize": 10.5,
})
LW = 1.5 # 1.5 pt = 2 px at 96 dpi
MS = 9
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
def clean(ax):
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
# ---- Fig 1: the three versions of the headline interval on one log-OR axis ----
fig, ax = plt.subplots(figsize=(7.2, 4.0))
rows = [
(2, ORANGE, 1.03, 0.92, 1.61, "Abstract (as printed)", f"1.03 (0.92–1.61)"),
(1, ORANGE, 1.03, 0.92, 1.16, "Results and Discussion\n(as printed)", f"1.03 (0.92–1.16)"),
(0, BLUE, 1.03, 0.92, b["hi_impl"], "Implied by OR 1.03 and\nlower limit 0.92", f"upper limit {b['hi_impl']:.3f}"),
]
for y, col, orr, lo, hi, lab, txt in rows:
ax.plot([lo, hi], [y, y], color=col, lw=LW * 1.4, solid_capstyle="butt", zorder=3)
for x in (lo, hi):
ax.plot([x, x], [y - 0.09, y + 0.09], color=col, lw=LW, zorder=3)
ax.plot(orr, y, "o", ms=MS, color=col, zorder=4)
ax.text(hi * 1.02, y, txt, va="center", ha="left", fontsize=10, color=TXT, zorder=6,
bbox=dict(boxstyle="square,pad=0.15", fc=BG, ec="none"))
# log-midpoint of the abstract interval
ax.plot(a["mid"], 2, marker="D", ms=8, mfc="none", mec=ORANGE, mew=1.8, ls="", zorder=5)
ax.annotate(f"midpoint of 0.92–1.61\non the log scale = {a['mid']:.2f}", (a["mid"], 2), xytext=(0, 14),
textcoords="offset points", ha="center", va="bottom", fontsize=10, color=SEC)
ax.axvline(1.0, color=SEC, lw=0.8, zorder=1)
ax.text(0.99, -0.62, "OR = 1", ha="right", va="center", fontsize=10, color=SEC)
ax.axvline(OBS_OR, color=SEC, lw=0.8, ls=(0, (3, 2)), zorder=1)
ax.text(OBS_OR * 1.01, -0.62, "observational\nOR 1.5", ha="left", va="center", fontsize=10, color=SEC)
ax.set_xscale("log")
ax.set_xlim(0.8, 2.3)
ax.set_ylim(-0.85, 2.75)
ticks = [0.8, 0.9, 1.0, 1.2, 1.4, 1.6, 2.0]
ax.xaxis.set_major_locator(mticker.FixedLocator(ticks))
ax.xaxis.set_major_formatter(mticker.FixedFormatter([f"{t:g}" for t in ticks]))
ax.xaxis.set_minor_formatter(mticker.NullFormatter())
ax.set_yticks([0, 1, 2])
ax.set_yticklabels([r[5] for r in sorted(rows, key=lambda r: r[0])])
ax.set_xlabel("Odds ratio for MS, smoking initiation, IVW (log scale)")
clean(ax)
ax.legend(handles=[Line2D([], [], marker="o", color=ORANGE, lw=LW, ms=MS, label="reported in the paper"),
Line2D([], [], marker="o", color=BLUE, lw=LW, ms=MS, label="recomputed from the paper's OR and lower limit")],
loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig1")
# ---- Fig 2: arm SE ratio for every interval in the paper ----
order = ["si_abs", "si_res", "si_egg", "si_raps", "lt_ivw", "lt_egg"]
fig, ax = plt.subplots(figsize=(7.2, 4.1))
for i, k in enumerate(order):
r = R[k]
y = len(order) - 1 - i
col = BLUE
ax.plot(r["ratio"], y, "o", ms=MS, color=col, zorder=4)
ax.text(r["ratio"] + (0.12 if k != "si_abs" else -0.12), y, f"{r['ratio']:.2f}", va="center",
ha="left" if k != "si_abs" else "right", fontsize=10, color=TXT)
ax.axvline(1.0, color=SEC, lw=0.8, zorder=1)
ax.text(1.0, len(order) - 0.45, "1 = symmetric on the log scale", ha="left", va="bottom", fontsize=10, color=SEC)
ax.set_yticks(range(len(order)))
ax.set_yticklabels([f"{R[k]['lab']} {R[k]['orr']:.2f} ({R[k]['lo']:.2f}–{R[k]['hi']:.2f})\n[{R[k]['loc']}]"
for k in reversed(order)], fontsize=10)
ax.set_xlim(0, 4.5)
ax.set_ylim(-0.6, len(order) - 0.1)
ax.set_xlabel("ln(upper / OR) ÷ ln(OR / lower)")
clean(ax)
ax.annotate(f"abstract: upper arm {R['si_abs']['ratio']:.2f} times\nthe lower arm", (R["si_abs"]["ratio"], len(order) - 1), xytext=(0, -14),
textcoords="offset points", ha="right", va="top", fontsize=10, color=SEC)
ax.legend(handles=[Line2D([], [], marker="o", ls="", ms=MS, color=BLUE,
label="ratio recomputed from the OR and 95% CI printed in the paper")],
loc="lower center", bbox_to_anchor=(0.5, 0.0), bbox_transform=fig.transFigure, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0.07, 1, 1))
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Abstract reports WGS spanned 49 (98%) of 48 P-VNTRs; the body gives 92% and 45
PLoS Comput Biol (2025) · doi:10.1371/journal.pcbi.1012885 · PMC11975116
Summary
The paper compares targeted and whole-genome (WGS) PacBio HiFi sequencing for genotyping a curated set of 48 phenotype-associated VNTRs (P-VNTRs). The Abstract says the WGS data spanned "49 (98%) of P-VNTRs with at least 15 spanning reads". That count cannot be right, because 49 of 48 is 102.1%. The body gives two other values for the same quantity: 92% (Results 2.5.2 and the Discussion) and "45 VNTRs (out of 48)" (Discussion), which is 93.75%, not 92%. The best-supported figure from the paper's own text is 45 of 48 (about 94%). The Abstract's 98% therefore overstates WGS coverage of the P-VNTR set by roughly 4 to 6 percentage points. The paper's comparison with targeted sequencing (4 of 48) is unaffected.
What the paper reports
In contrast, the 40X WGS HiFi dataset spanned 98% of all VNTRs and 49 (98%) of P-VNTRs with at least 15 spanning reads, albeit with lower coverage.
— Abstract
Similar to G-VNTRs, whole genome sequencing covered 92% of P-VNTRs with median spanning coverage ≥ 15 spanning reads in 28 samples (Fig 3b). CACNA1C, DUX4 and ABCA7 VNTRs were not well-covered likely due to VNTR length.
— Results, section 2.5.2
In contrast, whole genome sequencing was unbiased with at least 15 spanning reads for 98% of G-VNTRs and 92% of the P-VNTRs. For P-VNTRs, the gap in spanning reads between the two sequencing methods was even more stark, with only 4 VNTRs in targeted sequencing and 45 VNTRs (out of 48) in whole genome sequencing supported by at least 15 spanning reads.
— Discussion, paragraph 1
The size of the P-VNTR set is stated, and adds up to 48, in several places:
We curated a set of 10 , 787 gene-proximal (G-)VNTRs, and 48 phenotype-associated (P-)VNTRs of interest.
— Abstract (the spacing in "10 , 787" is as archived)
The P-VNTRs includes 39 pathogenic and 9 non-pathogenic phenotype-associated loci.
— Fig 3 legend, panel (a)
However, 19 (39%) of 48 P-VNTRs had high repeat-unit count (RC) variation, … Another 21 had low RC variation, while the remaining 8 VNTRs associated with phenotypes based not on length but on the presence of specific motifs …
— Results, section 2.5
Only 4 VNTRs of the 48 P-VNTRs were spanned by ≥ 15 targeted spanning reads.
— Results, section 2.5.1
What the paper's own numbers show
Method: take the denominator the paper states (48 P-VNTRs, confirmed by 39 + 9 and by 19 + 21 + 8). Convert each reported count to a percentage, and each reported percentage to the count it implies over 48. Then compare these with the count implied by the three loci that Results 2.5.2 names as not well covered. All arithmetic is in recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Size of the P-VNTR set | Abstract; Results 2; Fig 3a legend; Results 2.5; Discussion | 48 | 39 + 9 = 48; 19 + 21 + 8 = 48 |
| P-VNTRs spanned by ≥ 15 WGS reads (count) | Abstract | 49 | 49/48 = 102.08%, which exceeds the whole set |
| Same quantity (percentage) | Abstract | 98% | 0.98 × 48 = 47.04, i.e. 47/48 = 97.92%. This is neither 49 nor any count in the body |
| Same quantity (percentage) | Results 2.5.2; Discussion | 92% | 0.92 × 48 = 44.16, i.e. 44/48 = 91.67% |
| Loci named as not well covered by WGS | Results 2.5.2 | CACNA1C, DUX4, ABCA7 | 48 − 3 = 45, i.e. 45/48 = 93.75% (rounds to 94%) |
| Same quantity (count) | Discussion | 45 (out of 48) | 45/48 = 93.75%. The same paragraph says 92% |
| Abstract overstatement, in percentage points | Abstract vs body | — | 98 − 93.75 = 4.25 (vs 45/48); 98 − 91.67 = 6.33 (vs 44/48) |
| P-VNTRs spanned by ≥ 15 targeted reads | Abstract; Results 2.5.1; Discussion | 4 (8%) | 4/48 = 8.33%, consistent |
| High-RC P-VNTRs | Results 2.5 | 19 (39%) | 19/48 = 39.58% (would round to 40%), a minor rounding slip |
| Low-RC + motif P-VNTRs | Results 2.5 | 29 (60% of 48) | 29/48 = 60.42%, consistent |
The paper's statements give three different counts for one quantity: 49 (Abstract), about 44 (the 92% in Results and Discussion) and 45 (Discussion). Only 45 agrees with the three loci the Results names as poorly covered. No count over 48 can produce both "49" and "98%".
Figure
Figure 1. Three passages of the paper, quoted verbatim (orange boxes), report WGS coverage of the same 48 P-VNTRs at the same threshold (at least 15 spanning reads). The blue connectors give the arithmetic from the paper's own numbers. The Abstract's "49" is larger than the set (49/48 = 102.1%), and its 98% would correspond to 47 loci. Results gives 92% (about 44 loci) but names three loci as not well covered, which implies 45. The Discussion gives both 92% and 45 of 48 (93.75%). The set size of 48 is stated and adds up consistently throughout the paper.
Figure 2. Each reported value expressed as a count out of 48 (orange circles; the 92% statements are shown as 0.92 × 48 = 44.2), set against 45, the count left after removing the three loci that Results 2.5.2 names as not well covered (blue diamonds). The dashed line marks the whole set of 48, and the shaded region holds counts larger than the set. Only the Abstract's value falls there. The axis starts at zero.
Why it matters
The Abstract is the part of the paper most readers see and cite, and it states a P-VNTR count that is larger than the P-VNTR set. Its 98% looks plausible on its face, so a reader who does not check it against the stated set size of 48 would take WGS coverage of the clinically relevant loci to be about 4 to 6 points higher than the body supports. The 98% also matches the G-VNTR figure in the same sentence, which makes WGS look equally complete for both sets. The body itself says otherwise: three named loci (CACNA1C, DUX4, ABCA7) were not well covered, largely because of their length.
The error does not change the direction of the paper's main comparison. Every value the body supports (44 or 45 of 48, 92% to 94%) remains far above targeted sequencing (4 of 48, 8.3%). The paper's conclusion that WGS is more effective for broad VNTR genotyping stands. The G-VNTR figure (98% of 10,787) is reported consistently and is not affected. The problem is an impossible count, and an inflated percentage, in the Abstract for a secondary result. There is also a smaller internal inconsistency (92% vs 45/48) in the body.
Explanations considered
- A different denominator. The paper states 48 P-VNTRs everywhere, including in "45 VNTRs (out of 48)", and its subgroup counts (39 + 9; 19 + 21 + 8) both add up to 48. S2 Table has 49 rows but only 48 unique coordinates: one locus (MUC6) appears twice, and the table flags it as a duplicate entry because it is linked to both a pathogenic and a non-pathogenic phenotype. With 49 as the denominator, 92% would fit 45/49 (91.84%) and 98% would fit 48/49 (97.96%). But the abstract's "49" would then mean 49/49 = 100%, not 98%. So no single denominator reconciles "49 (98%)", and the paper never describes the set as 49 loci.
- A different coverage rule. The Abstract says "at least 15 spanning reads". Results says "median spanning coverage ≥ 15 spanning reads in 28 samples". Under any rule, at most 48 loci can meet the threshold, so 49 is impossible whichever definition was used. The Discussion uses the same threshold and the same WGS cohort, and its two adjacent figures (92% and 45 of 48) still disagree.
- Exclusions. CACNA1C is excluded only from the allele plot (Fig 3c), not from the coverage count. Results 2.5.2 counts it among the three loci that were not well covered.
- Rounding. 45/48 = 93.75% rounds to 94%, not 92%. 92% matches 44/48 (91.67%), but the text names exactly three poorly covered loci, not four. Rounding cannot turn 49 of 48 into a valid figure.
- Text conversion or archiving artifact. These are plain integers and percentages in running text, with no symbols, superscripts or table cells involved. The same wording appears in the publisher's HTML, the Europe PMC full-text XML and the PMC archive.
- Per-locus data. Fig 3b (a heatmap of spanning reads per locus and sample) holds the underlying data, but it cannot be read reliably as numbers. S2 Table lists coordinates and categories but not WGS read counts. The true count therefore cannot be settled beyond what the text implies, and the text's three named loci point to 45.
Status
- Version of record: checked on 2026-10-23. The publisher's HTML at journals.plos.org (reached through doi.org) has all three passages word for word, as do the Europe PMC full-text XML and the PMC archive. Every quotation above was checked verbatim against the archived PMC text.
- Supplements: S1 Table (PDF: genes, phenotypes, references), S2 Table (TSV: coordinates, category, pathogenicity), S3–S5 Tables (adapters, read-length summaries) and S1–S4 Figs are listed. S2 Table was downloaded and parsed: 49 rows, 48 unique coordinates and 1 flagged duplicate (MUC6). None of the supplements gives per-locus WGS spanning-read counts, so they do not resolve the discrepancy.
- Corrections: none found. Crossref shows no update-to or updated-by relations. PMC and Europe PMC have no correction, erratum, expression of concern or retraction notice (a Europe PMC correction query returned 0 hits). The PLOS article page shows no correction notice, and its comments tab shows "Reader Comments (0)".
- PubPeer: not confirmed. The PubPeer search API and search page returned a human-verification challenge (HTTP 403) on 2026-10-23. A web search found no PubPeer entry for the article.
- Sources: https://doi.org/10.1371/journal.pcbi.1012885; Europe PMC record PMID 40193344; PMC11975116; S2 Table, doi:10.1371/journal.pcbi.1012885.s002.
Status as of 2026-10-23.
Suggested correction
Replace "49 (98%) of P-VNTRs" in the Abstract with the count and percentage supported by the per-locus data behind Fig 3b, over the stated set of 48 P-VNTRs. If the three loci named in Results 2.5.2 are the only ones below the threshold, this would read "45 (94%) of P-VNTRs". Change the "92%" in Results 2.5.2 and in the Discussion to match (for example "94%", or "45 of 48"), so that the Discussion's "92%" and "45 VNTRs (out of 48)" agree. If the coverage rule in the Abstract ("at least 15 spanning reads") differs from the one in Results ("median spanning coverage ≥ 15 … in 28 samples"), state one rule and use it throughout. Optionally, also change "19 (39%)" in Results 2.5 to "19 (40%)".
Recomputation script (python recompute.py.txt)
# Recompute every number in the PMC11975116 write-up and regenerate its figures.
# Usage: python recompute.py.txt (needs numpy + matplotlib)
# Every input below is copied from the article text (doi:10.1371/journal.pcbi.1012885).
# Figures are written next to this file: fig1.*, fig2.* and five design candidates in candidates/.
import os
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, Rectangle
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (paper)
N_PATH, N_NONPATH = 39, 9 # Fig 3a legend
N_HIGH, N_LOW, N_MOTIF = 19, 21, 8 # Results 2.5
N_STATED = 48 # Abstract, Results 2 intro, 2.5.1, Discussion ("out of 48")
ABS_COUNT, ABS_PCT = 49, 98 # Abstract: "49 (98%) of P-VNTRs"
RES_PCT = 92 # Results 2.5.2 and Discussion: "92% of (the) P-VNTRs"
DIS_COUNT = 45 # Discussion: "45 VNTRs (out of 48)"
NAMED_POOR = ["CACNA1C", "DUX4", "ABCA7"] # Results 2.5.2: "not well-covered"
TGT_COUNT, TGT_PCT = 4, 8 # Abstract / Results 2.5.1 / Discussion
S2_ROWS, S2_UNIQUE = 49, 48 # S2 Table: 49 rows, 48 unique coordinates (MUC6 listed twice)
# ---------------------------------------------------------------- arithmetic
def pct(a, b):
return 100.0 * a / b
print("== Size of the P-VNTR set")
print(f"pathogenic + non-pathogenic = {N_PATH} + {N_NONPATH} = {N_PATH + N_NONPATH}")
print(f"high-RC + low-RC + motif = {N_HIGH} + {N_LOW} + {N_MOTIF} = {N_HIGH + N_LOW + N_MOTIF}")
assert N_PATH + N_NONPATH == N_HIGH + N_LOW + N_MOTIF == N_STATED
N = N_STATED
print("\n== Abstract: '49 (98%)'")
print(f"49 / 48 = {pct(ABS_COUNT, N):.2f}% (exceeds 100%)")
print(f"98% of 48 = {ABS_PCT/100*N:.2f} loci; nearest count 47 -> 47/48 = {pct(47, N):.2f}%")
print("\n== Results 2.5.2 / Discussion: '92%'")
print(f"92% of 48 = {RES_PCT/100*N:.2f} loci; nearest count 44 -> 44/48 = {pct(44, N):.2f}%")
implied = N - len(NAMED_POOR)
print(f"48 - {len(NAMED_POOR)} named poorly covered loci ({', '.join(NAMED_POOR)}) = {implied}")
print(f"{implied}/48 = {pct(implied, N):.2f}% (rounds to {round(pct(implied, N))}%)")
print("\n== Discussion: '45 VNTRs (out of 48)'")
print(f"45/48 = {pct(DIS_COUNT, N):.2f}% (the same paragraph states 92%)")
print("\n== Counts that would round to each stated percentage (denominator 48)")
for k in range(43, 49):
print(f" {k}/48 = {pct(k, N):6.2f}%")
print("\n== Size of the abstract's overstatement (percentage points)")
print(f"98 - 93.75 = {ABS_PCT - pct(45, N):.2f} pp ; 98 - 91.67 = {ABS_PCT - pct(44, N):.2f} pp")
print("\n== Targeted sequencing (context)")
print(f"4/48 = {pct(TGT_COUNT, N):.2f}% (stated 8%)")
print("\n== Other P-VNTR percentages in Results 2.5 / 2.5.1")
print(f"19/48 = {pct(19, N):.2f}% (stated 39%); 29/48 = {pct(29, N):.2f}% (stated 60%); "
f"23/48 = {pct(23, N):.2f}% (stated 48%)")
print("\n== Alternative denominator (49 rows in S2 Table)")
for k in (45, 48, 49):
print(f" {k}/49 = {pct(k, S2_ROWS):.2f}%")
# ---------------------------------------------------------------- style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
W = 7.2
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"axes.spines.top": False, "axes.spines.right": False, "svg.fonttype": "none",
"lines.linewidth": 2,
})
LW = 1.5 # points; matplotlib 1.5 pt = 2 px at 96 dpi
def save(fig, *paths):
for p in paths:
fig.savefig(p + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(p + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def wrap(s, w):
return "\n".join(textwrap.wrap(s, w))
Q_ABS = ("“…the 40X WGS HiFi dataset spanned 98% of all VNTRs and "
"49 (98%) of P-VNTRs with at least 15 spanning reads…”")
Q_RES = ("“…whole genome sequencing covered 92% of P-VNTRs with median spanning "
"coverage ≥ 15 spanning reads in 28 samples (Fig 3b). CACNA1C, DUX4 and ABCA7 "
"VNTRs were not well-covered…”")
Q_DIS = ("“…at least 15 spanning reads for 98% of G-VNTRs and 92% of the P-VNTRs. "
"… 45 VNTRs (out of 48) in whole genome sequencing supported by at least "
"15 spanning reads.”")
# ---------------------------------------------------------------- candidate A: text panels
def cand_A():
fig = plt.figure(figsize=(W, 5.7))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 84); ax.axis("off")
def panel(x, y, w, h, title, quote, wid):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="white", ec=ORANGE, lw=LW))
ax.text(x + 1.5, y + h - 1.6, title, fontsize=11, fontweight="bold", va="top")
ax.text(x + 1.5, y + h - 6.2, wrap(quote, wid), fontsize=10, va="top", color=INK,
linespacing=1.35)
panel(2, 67, 96, 15, "Abstract", Q_ABS, 82)
panel(2, 18, 43, 29, "Results 2.5.2", Q_RES, 35)
panel(55, 18, 43, 29, "Discussion, paragraph 1", Q_DIS, 35)
arw = dict(arrowstyle="<->", color=BLUE, lw=LW)
ax.annotate("", xy=(14, 47.8), xytext=(14, 66.2), arrowprops=arw)
ax.text(16, 57, "49 of 48 = 102.1%, an impossible share\n98% of 48 \u2248 47 loci;"
" Results implies 44\u201345", va="center", fontsize=10, color=BLUE)
ax.annotate("", xy=(86, 47.8), xytext=(86, 66.2), arrowprops=arw)
ax.text(84, 57, "98% vs 92%\n49 vs 45 loci", va="center", ha="right", fontsize=10,
color=BLUE)
ax.annotate("", xy=(54.2, 32), xytext=(45.8, 32), arrowprops=arw)
ax.text(50, 12.5, "Results vs Discussion: 92% \u2260 45/48 = 93.75%; "
"48 \u2212 3 named loci = 45", ha="center", fontsize=10, color=BLUE)
ax.text(2, 6.3, "Set size stated throughout: 48 P-VNTRs = 39 pathogenic + 9 non-pathogenic "
"= 19 high-RC + 21 low-RC + 8 motif", fontsize=10, color=INK2)
ax.add_patch(Rectangle((2, 0.8), 2.6, 2.4, fc="white", ec=ORANGE, lw=LW))
ax.text(5.5, 2.0, "text as printed in the paper", va="center", fontsize=10, color=INK2)
ax.plot([42, 46], [2.0, 2.0], color=BLUE, lw=LW)
ax.text(47, 2.0, "recomputed from the paper\u2019s own numbers", va="center",
fontsize=10, color=INK2)
return fig
# ---------------------------------------------------------------- candidate B: count dot plot
ROWS = [ # label, reported count (or None), reported pct, recomputed count, recomputed note
("Abstract\n“49 (98%)”", ABS_COUNT, ABS_PCT),
("Results 2.5.2\n“92%”", RES_PCT / 100 * N, RES_PCT),
("Discussion\n“92%”", RES_PCT / 100 * N, RES_PCT),
("Discussion\n“45 (out of 48)”", DIS_COUNT, pct(DIS_COUNT, N)),
]
def cand_B():
fig, ax = plt.subplots(figsize=(W, 3.6))
ys = list(range(len(ROWS)))[::-1]
ax.axvline(N, color=INK2, lw=1, ls=(0, (4, 3)))
ax.text(N - 0.4, len(ROWS) - 0.45, "all 48 P-VNTRs", color=INK2, fontsize=10, va="bottom",
ha="right")
ax.axvspan(N, 52, color=GRID, alpha=0.6, lw=0)
ax.text(50, -0.75, "more loci\nthan exist", ha="center", va="center", color=INK2,
fontsize=10)
for y, (lab, cnt, _) in zip(ys, ROWS):
ax.plot([implied, cnt], [y, y], color=GRID, lw=3, zorder=1)
ax.scatter(cnt, y, s=170, color=ORANGE, zorder=2, edgecolor="white", linewidth=0.8)
ax.scatter(implied, y, s=70, color=BLUE, zorder=3, marker="D", edgecolor="white",
linewidth=0.8)
txt = f"{cnt:.0f}" if float(cnt).is_integer() else f"{cnt:.1f}"
ax.text(cnt + (0.5 if cnt >= implied else -0.5), y + 0.2, txt, color=ORANGE,
fontsize=10, ha="left" if cnt >= implied else "right")
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in ROWS], fontsize=10)
ax.set_xlim(0, 52); ax.set_ylim(-1.2, len(ROWS) - 0.2)
ax.set_xlabel("P-VNTRs spanned by ≥ 15 WGS reads (count out of 48)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.scatter([], [], s=120, color=ORANGE, label="reported (92% shown as 0.92 × 48 = 44.2)")
ax.scatter([], [], s=90, color=BLUE, marker="D",
label="48 − 3 named poorly covered loci = 45")
ax.legend(loc="lower left", frameon=False, fontsize=10, bbox_to_anchor=(0, -0.02))
return fig
# ---------------------------------------------------------------- candidate C: calculation flow
def cand_C():
fig = plt.figure(figsize=(W, 3.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 48); ax.axis("off")
def box(x, y, w, h, s, ec, fs=10):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1",
fc="white", ec=ec, lw=LW))
ax.text(x + w / 2, y + h / 2, s, ha="center", va="center", fontsize=fs)
def arrow(x0, y0, x1, y1, c=INK2):
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=c, lw=LW))
box(1, 30, 22, 13, "P-VNTR set\n39 + 9 = 48\n19 + 21 + 8 = 48", BLUE)
box(28, 30, 28, 13, "minus 3 named as\nnot well-covered\n(CACNA1C, DUX4, ABCA7)", BLUE)
box(61, 30, 15, 13, "45 loci", BLUE, 11)
box(83, 30, 16, 13, "45/48\n= 93.75%", BLUE, 11)
arrow(23.6, 36.5, 27.4, 36.5); arrow(56.6, 36.5, 60.4, 36.5); arrow(76.6, 36.5, 82.4, 36.5)
ax.text(1, 24.5, "As printed:", fontsize=10.5, fontweight="bold", color=INK)
box(1, 9, 30, 12, "Abstract “49 (98%)”\n49/48 = 102.1%\n98% of 48 ≈ 47",
ORANGE)
box(35, 9, 30, 12, "Results / Discussion “92%”\n0.92 × 48 = 44.2\n"
"→ 44/48 = 91.67%", ORANGE)
box(69, 9, 30, 12, "Discussion “45 (out of 48)”\n45/48 = 93.75%\n"
"(matches the 3 named loci)", ORANGE)
ax.add_patch(Rectangle((1, 1.5), 2.6, 2.4, fc="white", ec=BLUE, lw=LW))
ax.text(4.5, 2.7, "derived from the paper’s own counts", va="center", fontsize=10,
color=INK2)
ax.add_patch(Rectangle((48, 1.5), 2.6, 2.4, fc="white", ec=ORANGE, lw=LW))
ax.text(51.5, 2.7, "value as reported, with its arithmetic", va="center", fontsize=10,
color=INK2)
return fig
# ---------------------------------------------------------------- candidate D: percentage dumbbell
def cand_D():
rows = [
("Abstract: “49 (98%)”", ABS_PCT, pct(ABS_COUNT, N), "49/48"),
("Results 2.5.2: “92%”", RES_PCT, pct(implied, N), "45/48 (3 loci named)"),
("Discussion: “92%” …\n“45 VNTRs (out of 48)”", RES_PCT,
pct(DIS_COUNT, N), "45/48"),
("Targeted (context): “4 (8%)”", TGT_PCT, pct(TGT_COUNT, N), "4/48"),
]
fig, ax = plt.subplots(figsize=(W, 3.4))
ys = list(range(len(rows)))[::-1]
ax.axvline(100, color=INK2, lw=1, ls=(0, (4, 3)))
ax.text(99.3, len(rows) - 0.55, "100%", color=INK2, ha="right", fontsize=10)
for y, (lab, rep, rec, note) in zip(ys, rows):
ax.plot([rep, rec], [y, y], color=GRID, lw=3, zorder=1)
ax.scatter(rep, y, s=90, color=ORANGE, zorder=3)
ax.scatter(rec, y, s=90, color=BLUE, marker="D", zorder=3)
lo, hi = min(rep, rec), max(rep, rec)
ax.text(lo - 2, y, f"{rep:g}%", color=ORANGE, ha="right", va="center", fontsize=10)
ax.text(hi + 2, y, f"{note} = {rec:.2f}%", color=BLUE, va="center", fontsize=10)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 106); ax.set_ylim(-0.5, len(rows) - 0.4); ax.set_xticks(range(0, 101, 20))
ax.set_xlabel("Share of the 48 P-VNTRs spanned by ≥ 15 reads (%)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.scatter([], [], s=90, color=ORANGE, label="percentage as printed")
ax.scatter([], [], s=90, color=BLUE, marker="D",
label="percentage implied by the paper’s own count")
ax.legend(loc="upper center", bbox_to_anchor=(0.45, -0.22), ncol=2, frameon=False,
fontsize=10)
return fig
# ---------------------------------------------------------------- candidate E: unit grid of 48 loci
def cand_E():
fig = plt.figure(figsize=(W, 3.0))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 40); ax.axis("off")
cols, size, gap, x0, y0 = 16, 4.2, 0.9, 2, 30
for i in range(N):
r, c = divmod(i, cols)
x, y = x0 + c * (size + gap), y0 - r * (size + gap)
poor = i >= N - len(NAMED_POOR)
ax.add_patch(Rectangle((x, y), size, size, fc="white" if poor else BLUE,
ec=BLUE, lw=LW))
if i == N - 2:
ax.text(x + size / 2, y - 0.8, ", ".join(NAMED_POOR), fontsize=10,
ha="center", va="top", color=INK2)
# the abstract's 49th locus
xg = x0 + cols * (size + gap) + 2.5
yg = y0 - 2 * (size + gap)
ax.add_patch(Rectangle((xg, yg), size, size, fc="none", ec=ORANGE, lw=LW, ls=(0, (3, 2))))
ax.text(xg + size + 1.2, yg + size / 2,
"Abstract: “49 (98%)”\nwould need a 49th locus\n(49/48 = 102.1%)",
fontsize=10, color=ORANGE, va="center")
ax.text(x0, 37.5, "The 48 P-VNTRs (39 + 9 = 19 + 21 + 8 = 48), one square per locus",
fontsize=10.5, fontweight="bold")
ax.add_patch(Rectangle((x0, 7.2), 2.8, 2.8, fc=BLUE, ec=BLUE, lw=LW))
ax.text(x0 + 4, 8.6, "45 not named as poorly covered: 45/48 = 93.75%", fontsize=10,
va="center", color=INK2)
ax.add_patch(Rectangle((x0, 2.2), 2.8, 2.8, fc="white", ec=BLUE, lw=LW))
ax.text(x0 + 4, 3.6, "3 named as “not well-covered” in Results 2.5.2", fontsize=10,
va="center", color=INK2)
ax.text(60, 8.6, "Results and Discussion: “92%” = 0.92 × 48 = 44.2",
fontsize=10, color=ORANGE, va="center")
ax.text(60, 3.6, "orange = as printed; blue = recomputed", fontsize=10, color=INK2,
va="center")
return fig
designs = {"A_text_panels": cand_A, "B_count_dots": cand_B, "C_calc_flow": cand_C,
"D_percent_dumbbell": cand_D, "E_unit_grid": cand_E}
for name, fn in designs.items():
save(fn(), os.path.join(CAND, name))
# Provisional picks for the write-up (swap after the design review):
save(cand_A(), os.path.join(HERE, "fig1"))
save(cand_B(), os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE)
Abstract gives Padj = 0.010 for a primary macrolide comparison; Results give 0.19 (Table 2: 0.187)
PLoS Med (2024) · doi:10.1371/journal.pmed.1004386 · PMC11073710
Summary
This MORDOR sub-study compares gut macrolide resistance determinants across four azithromycin treatment histories. For the comparison of 4.5 years of treatment (group 1) with 2.5 years (group 2), the abstract reports "fold change 0.80, 95% CI 0.50 to 1.00, Padj = 0.010". The Results report the same comparison as "0.80-fold, 95% CI: 0.52 to 1.00, Padj = 0.19", and Table 2(A) gives its min-P adjusted p-value as 0.187. That value rounds to 0.19, not to 0.010, and a 95% interval that reaches 1.00 is not compatible with an adjusted p of 0.010. The paper's own wording ("no significant increase") and its conclusion are consistent with the correct value, 0.19. But anyone who extracts the abstract's number gets a nominally significant result, apparently a significant decrease, for the comparison that underlies the conclusion that "resistance may plateau".
What the paper reports
Abstract, "Methods and findings" (Padj is printed with a subscript in the original):
However, there was no significant increase in macrolide resistance in communities treated 4.5 years (group 1) compared to just the most recent 2.5 years (group 2) (fold change 0.80, 95% CI 0.50 to 1.00, Padj = 0.010), or between communities that had been treated for 3 years in the past (group 3) versus just 1 year in the past (group 4) (fold change 1.00, 95% CI 0.78 to 2.35, Padj = 0.52).
Results, "Primary outcome", the same comparison:
Finally, we did not find a statistically significant difference in macrolide resistance determinants between 4.5 years of azithromycin treatment (group 1) compared to 2.5 years of treatment (group 2) (0.80-fold, 95% CI: 0.52 to 1.00, Padj = 0.19, Wilcoxon rank-sum, min-P adjustment) or 3 years of treatment (group 3) (1.60-fold, 95% CI: 1.00 to 3.08, Padj = 0.06, Wilcoxon rank-sum, min-P adjustment).
Results, "Primary outcome", group 1 vs group 4:
In particular, 4.5 years of azithromycin treatments (group 1) resulted in a 2.23-fold (95% CI: 1.20 to 5.17) increase in macrolide resistance determinants compared to 1 year of azithromycin treatment only during year 3 of the study (Padj = 0.01, Wilcoxon rank-sum (i.e., without covariate adjustment), min-P adjustment).
Table 2(A), rows 1 vs. 2 and 1 vs. 4. The columns are Groups | Treatment | Aminoglycosides | Cationic antimicrobial peptides | Macrolides | Metronidazole | Multi drug resistance | Tetracyclines | Trimethoprim | Beta-lactams:
1 vs. 2 | AA-AA-AA-AA-A vs.PP-PP-AA-AA-A | 0.218 | 0.918 | 0.187 | 0.255 | 0.945 | 0.140 | 0.112 | 0.408
1 vs. 4 | AA-AA-AA-AA-A vs.PP-PP-AA-PP-P | 0.218 | 0.791 | 0.014 | 0.884 | 0.848 | 0.927 | 0.730 | 0.957
Table 2 footnote (A):
The P-values derived from the min-P procedure that were prespecified in the Statistical Analysis Plan are presented in this table.
Methods, "Statistical methods > Primary outcome":
Pairwise Wilcoxon rank-sum tests for macrolide resistance determinants between each of the treatment history groups, accounting for multiple comparisons by means of the Westfall–Young min-P method [20], which are reported as adjusted P-values.
Methods, "Statistical methods > Secondary outcomes", on how fold changes and CIs were obtained:
Fold changes were estimated by transforming normalized read counts using the transformation log(x + epsilon) where epsilon = 0.001 (qualitative results are not sensitive to choice of small epsilon) and performing analysis using the unadjusted confidence interval from the Wilcoxon rank-sum test.
Specifically, the Hodges–Lehmann estimator was computed on the transformed scale, and the inverse transformation was used to yield an approximate ratio.
The statements that rest on this comparison. Abstract, "Conclusions":
resistance may plateau after 2 to 3 years of treatment
Author summary:
Longer periods of treatment with azithromycin were not associated with increased macrolide resistance
Discussion, first paragraph:
However, we were unable to find a difference between communities treated for 4.5 years versus just 2.5 years. Thus resistance in the gut may plateau after several MDAs, rather than continue to increase until it saturates the age-group [22].
What the paper's own numbers show
Method. I matched every macrolide p-value printed in the abstract and Results to its cell in Table 2(A) and checked that each text value is the table value rounded to the printed number of decimals. I then asked whether the abstract's 0.010 could come from any other reading of the same comparison, and listed the Table 2(A) cells closest to 0.010. Finally, I used the printed 95% CI to bound the unadjusted p-value for group 1 vs group 2. The interval is the unadjusted Wilcoxon/Hodges–Lehmann interval on the log scale, and its upper bound is 1.00, the null ratio. By test inversion that corresponds to an unadjusted two-sided p of about 0.05 or more. For a rough numerical check I also approximated the standard error on the log scale from the lower half-width, the upper half-width and the full width of the interval. The community-level data are not published, so the permutation test itself cannot be redone. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Macrolides, 1 vs 2, min-P Padj | Table 2(A) | 0.187 | source value |
| Macrolides, 1 vs 2, Padj | Results | 0.19 | 0.187 to 2 dp = 0.19, consistent |
| Macrolides, 1 vs 2, Padj | Abstract | 0.010 | does not match: the Table value is 18.7 times larger; 0.187 cannot round to 0.010 |
| Significance at 0.05 of the same comparison | Abstract vs Results / Table 2(A) | 0.010 vs 0.19 / 0.187 | 0.010 < 0.05; 0.187 > 0.05. The two classifications differ |
| Macrolides, 1 vs 3, Padj | Results / Table 2(A) | 0.06 / 0.063 | consistent |
| Macrolides, 1 vs 4, Padj | Results / Table 2(A) | 0.01 / 0.014 | consistent |
| Macrolides, 2 vs 3, Padj | Results / Table 2(A) | 0.003 / 0.003 | consistent |
| Macrolides, 2 vs 4, Padj | Results / Table 2(A) | < 0.001 / < 0.001 | consistent |
| Macrolides, 3 vs 4, Padj | Abstract / Table 2(A) | 0.52 / 0.517 | consistent; the abstract's only other Padj (3 vs 4) matches Table 2(A) |
| Table 2(A) cells closest to 0.010 | Table 2(A) | — | Aminoglycosides 2 vs 4 = 0.011; Macrolides 1 vs 4 = 0.014; Macrolides 2 vs 3 = 0.003. Macrolides 1 vs 2 is 0.187 |
| 1 vs 2 CI on the log scale | Results / Abstract | 0.52 to 1.00 / 0.50 to 1.00 | −0.654 to 0.000 / −0.693 to 0.000; both reach the null (log ratio 0) |
| Approximate unadjusted p implied by the Results CI (normal approximation, log scale) | derived | — | 0.18 from the full width; 0.05 to 0.31 depending on which half-width sets the SE |
| Same, from the Abstract CI | derived | — | 0.21 from the full width; 0.05 to 0.35 |
| One-sided reading of the Table value | derived | — | 0.187 / 2 = 0.094, still not 0.010 |
| 1 vs 2 CI lower bound | Abstract vs Results | 0.50 vs 0.52 | differ by 0.02; cannot be resolved without the data |
| Groups 1+2 vs 3+4 CI lower bound | Abstract vs Results | 1.5 vs 1.49 | 1.49 to 1 dp = 1.5, rounding only |
Two points make 0.010 impossible for this comparison, not merely inconsistent. First, a Westfall–Young min-P adjusted p-value can never be smaller than the unadjusted p-value for the same hypothesis. Second, the unadjusted p implied by an interval that reaches 1.00 is about 0.05 or more. Even if the printed 1.00 were rounded up from a bound just below 1 (for example 0.995), the unadjusted p would be only slightly below 0.05, far above 0.010. Every other macrolide Padj in the text matches Table 2(A). The abstract's 0.010 is the only exception, and it sits close to the 1 vs 4 value (0.014, reported in the Results as 0.01).
Figure
Figure 1. Short excerpts for the macrolide comparison of group 1 (4.5 years of azithromycin) with group 2 (2.5 years). The left panel is the abstract; the right panel is the Results text and the Table 2(A) macrolide cell for row "1 vs. 2". Orange values are printed in the paper. The blue lines below are checks derived from the paper's numbers by recompute.py.txt. The Results value is the Table 2(A) value rounded to two decimals. The abstract's 0.010 is 18.7 times smaller than the table value and cannot be reconciled with a 95% CI whose upper bound is 1.00. Read literally, with a fold change of 0.80, it would mean a significant decrease, which the abstract does not report and the Results deny ("did not find a statistically significant difference").
Figure 2. P-values for the group 1 vs group 2 macrolide comparison on one log-scale axis; the dashed line marks 0.05. Orange circles are the three printed values: the abstract (0.010), the Results (0.19) and Table 2(A) (0.187). Blue diamonds are approximate unadjusted two-sided p-values recomputed from each printed 95% CI (normal approximation on the log scale, SE from the full width); the blue bars span the values obtained when the SE is taken from either half-width instead. A min-P adjusted p cannot be smaller than the unadjusted p, and an upper CI bound of 1.00 puts the unadjusted p at about 0.05 or more (the left end of the blue bars). The Results and Table 2(A) values are above that floor; the abstract's 0.010 is well below it.
Why it matters
The comparison of group 1 with group 2 is part of the prespecified primary outcome. It is the evidence behind the abstract's conclusion that "resistance may plateau after 2 to 3 years of treatment", the Author summary's statement that "Longer periods of treatment with azithromycin were not associated with increased macrolide resistance", and the corresponding sentence of the Discussion. With the correct value (Padj = 0.19, Table 2(A) 0.187) these statements are supported as written. The analysis, the body text and the table are consistent with each other, and the paper's conclusions do not change.
The error matters because it sits in the abstract, which is the part most often read on its own, indexed and extracted into reviews and meta-analyses. Read literally, the abstract pairs a fold change of 0.80 and an upper bound of 1.00 with Padj = 0.010. That reads as a statistically significant decrease in macrolide resistance with longer treatment, which the data do not show. The abstract does not report a significant decrease, and the Results deny any significant difference ("did not find a statistically significant difference"). A reader who takes the number at face value would put this comparison in the wrong significance class. The 0.50 vs 0.52 mismatch in the CI lower bound is a separate, minor inconsistency and does not affect any interpretation.
Explanations considered
- A different test or adjustment. This does not account for the numbers. The abstract and the Results both label the value Padj. The Methods define Padj as the Westfall–Young min-P adjusted pairwise Wilcoxon p-value. Table 2(A) prints that value as 0.187.
- The abstract quotes the unadjusted p. This does not account for the numbers. The paper does not print an unadjusted p for this comparison. The printed CI is the unadjusted Wilcoxon/Hodges–Lehmann interval, and its upper bound of 1.00 corresponds to an unadjusted p of about 0.05 or more (roughly 0.18–0.21 by normal approximation with the SE from the full width; 0.05–0.35 across SE choices), not 0.010.
- A one-sided test. This does not account for the numbers. Nothing in the paper describes a one-sided test, and halving 0.187 gives 0.094.
- The FDR (Benjamini–Hochberg) row of Table 2(B). This does not account for the numbers. That row is "NA" for macrolides.
- A different outcome class, population or time point. This does not account for the numbers. The abstract and the Results describe the same groups, the same outcome and the same fold change (0.80). The value 0.010 does not appear in the macrolide 1 vs 2 cell. The nearest Table 2(A) values are aminoglycosides 2 vs 4 (0.011) and macrolides 1 vs 4 (0.014, reported as Padj = 0.01 in the Results).
- A conversion or typesetting artifact in PMC. This is excluded. The publisher's PDF, the publisher's HTML page, PMC and the Europe PMC full-text XML all print the same abstract (0.50 to 1.00, Padj = 0.010) and the same Results (0.52 to 1.00, Padj = 0.19). No minus sign, superscript or decimal separator is involved, and the Table 2(A) column order was checked against the header in the PDF.
- The abstract value is right and the table and Results are wrong. This is not plausible. 0.010 cannot be an adjusted p for an interval that reaches the null. The table and the Results agree with each other, and the Results state that the difference is not statistically significant.
- The abstract claims a significant difference. This is not the case. The abstract says "no significant increase" and reports no significant decrease. The error is the printed number: read literally, 0.010 with a fold change of 0.80 would mean a significant decrease, which the abstract does not report and which conflicts with the Results ("did not find a statistically significant difference") and Table 2(A).
- The CI lower bound (0.50 vs 0.52). This is unexplained. It is a 0.02 difference between two printed versions of the same interval; which one is correct cannot be determined without the data. It does not affect the conclusion.
Status
- Version of record. The publisher's PDF (journals.plos.org printable file, 16 pages, text extracted) and the PLOS HTML article page carry the same abstract text (CI 0.50 to 1.00, Padj = 0.010) and the same Results text (CI 0.52 to 1.00, Padj = 0.19) as the PMC archive. In the PDF's Table 2(A), the macrolide value for "1 vs. 2" is 0.187 and for "1 vs. 4" is 0.014. On the HTML page, Table 2 is an image. The Europe PMC full-text XML matches. Every quote above was confirmed verbatim against the PMC-archived text.
- Supplements. S1 Appendix (pmed.1004386.s001.pdf, 178 pages) holds community-level summary statistics by group and class (Table A), a trial schematic, the group list, the protocols and the statistical analysis plans. It has no pairwise fold changes, CIs or p-values, so it does not resolve the discrepancy. S1 CONSORT Checklist is not relevant. The data availability statement points only to raw sequences (SRA PRJNA1066963). No community-level analysis dataset is published, so the min-P p-value cannot be recomputed.
- Corrections and comments. As of 2026-10-23 I found no correction, erratum, retraction or expression of concern. Crossref lists no update relations for the DOI, and its relation records list only the article's own components. PMC and Europe PMC (PMID 38709718) list no correction or comment links. The PLOS article page shows no correction notice. A web search for a correction notice citing the DOI returned nothing relevant. PubPeer could not be checked: both the PubPeer API and its search page returned a Cloudflare block (HTTP 403) to automated access.
- Sources. PMC article PMC11073710; publisher page and PDF at journals.plos.org (doi:10.1371/journal.pmed.1004386); S1 Appendix (pmed.1004386.s001.pdf); Europe PMC full-text XML; Crossref; PubPeer (not reachable).
Suggested correction
- In the abstract ("Methods and findings"), replace "Padj = 0.010" for group 1 vs group 2 with the Table 2(A) value, "Padj = 0.19" (0.187), after confirming it against the original analysis output.
- Reconcile the lower confidence limit for the same comparison: the abstract gives 0.50 and the Results give 0.52. Use whichever the analysis output supports in both places.
- Optionally, harmonize "1.5" (abstract) with "1.49" (Results) for the lower confidence limit of the groups 1+2 vs 3+4 comparison.
No change to the Results, Table 2, the Author summary, the Discussion or the conclusions appears to be needed.
Recomputation script (python recompute.py.txt)
"""Recompute the group 1 vs group 2 macrolide comparison of PMC11073710
(PLoS Med 2024, doi:10.1371/journal.pmed.1004386).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
All inputs are copied from the article as archived in PubMed Central (identical in the
publisher's PDF and HTML): the Abstract, the Results ("Primary outcome") and Table 2(A).
No raw data are used; the community-level data needed to redo the min-P permutation
test are not published.
"""
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (from the paper)
# Table 2(A): Westfall-Young min-P adjusted pairwise Wilcoxon p-values (100,000 permutations).
CLASSES = ["Aminoglycosides", "Cationic antimicrobial peptides", "Macrolides", "Metronidazole",
"Multi drug resistance", "Tetracyclines", "Trimethoprim", "Beta-lactams"]
TABLE2A = { # "<0.001" stored as 0.001 with a flag
"1 vs. 2": [0.218, 0.918, 0.187, 0.255, 0.945, 0.140, 0.112, 0.408],
"1 vs. 3": [0.379, 0.935, 0.063, 0.957, 0.291, 0.908, 0.690, 0.957],
"1 vs. 4": [0.218, 0.791, 0.014, 0.884, 0.848, 0.927, 0.730, 0.957],
"2 vs. 3": [0.055, 0.980, 0.003, 0.689, 0.530, 0.157, 0.690, 0.515],
"2 vs. 4": [0.011, 0.973, "<0.001", 0.840, 0.945, 0.193, 0.611, 0.508],
"3 vs. 4": [0.533, 0.980, 0.517, 0.957, 0.652, 0.908, 0.927, 0.959],
}
MAC = CLASSES.index("Macrolides")
# Macrolide comparisons as printed in the text: (fold change, CI low, CI high, Padj as printed)
RESULTS = { # Results, "Primary outcome"
"1 vs. 4": (2.23, 1.20, 5.17, "0.01"),
"2 vs. 3": (2.14, 1.38, 3.64, "0.003"),
"2 vs. 4": (3.12, 1.78, 7.31, "<0.001"),
"1 vs. 2": (0.80, 0.52, 1.00, "0.19"),
"1 vs. 3": (1.60, 1.00, 3.08, "0.06"),
}
ABSTRACT = { # Abstract, "Methods and findings"
"1 vs. 2": (0.80, 0.50, 1.00, "0.010"),
"3 vs. 4": (1.00, 0.78, 2.35, "0.52"),
}
ALPHA = 0.05
# ------------------------------------------------------------------ 1. text vs table
print("1. Macrolide comparisons: text vs Table 2(A) (min-P adjusted P)")
for pair in TABLE2A:
t = TABLE2A[pair][MAC]
r = RESULTS.get(pair, (None,) * 4)[3]
a = ABSTRACT.get(pair, (None,) * 4)[3]
if isinstance(t, str):
print(f" {pair}: Table {t}; Results {r}; Abstract {a}")
continue
parts = [f" {pair}: Table {t:.3f}"]
for lab, v in (("Results", r), ("Abstract", a)):
if v is None:
continue
dec = len(v.split(".")[1]) # decimals as printed in the text
rounded = round(t, dec)
ok = abs(rounded - float(v)) < 1e-9
parts.append(f"{lab} {v} (Table rounded to {dec} dp = {rounded:.{dec}f}: "
f"{'consistent' if ok else 'DOES NOT MATCH'})")
print("; ".join(parts))
t12 = TABLE2A["1 vs. 2"][MAC]
a12 = float(ABSTRACT["1 vs. 2"][3])
r12 = float(RESULTS["1 vs. 2"][3])
print(f" Abstract 1 vs. 2 Padj {a12:.3f} vs Table {t12:.3f}: ratio Table/Abstract = {t12 / a12:.1f}")
print(f" Significant at {ALPHA}? Abstract {a12} -> {a12 < ALPHA}; Results {r12} -> {r12 < ALPHA};"
f" Table {t12} -> {t12 < ALPHA}")
# ------------------------------------------------------------------ 2. nearest Table 2(A) entries to 0.010
print("\n2. Table 2(A) entries closest to the abstract's 0.010")
cells = []
for pair, row in TABLE2A.items():
for cls, v in zip(CLASSES, row):
if not isinstance(v, str):
cells.append((abs(v - a12), pair, cls, v))
for d, pair, cls, v in sorted(cells)[:3]:
print(f" {cls}, {pair}: {v:.3f} (|difference| = {d:.3f})")
print(f" Macrolides 1 vs. 2 itself: {t12:.3f} (|difference| = {abs(t12 - a12):.3f})")
# ------------------------------------------------------------------ 3. CI vs p-value
print("\n3. What the 95% CI implies about the unadjusted p-value (fold change 0.80)")
est = RESULTS["1 vs. 2"][0]
print(f" log(0.80) = {np.log(est):.3f}; log(upper bound 1.00) = {np.log(1.00):.3f}")
print(" The CI is the (unadjusted) Hodges-Lehmann/Wilcoxon interval. An interval from test inversion")
print(" that reaches the null (ratio 1, log 0) corresponds to an unadjusted two-sided p of about 0.05")
print(" or more; an unadjusted p of 0.010 would require an upper bound clearly below 1.")
APPROX = {}
for src, (e, lo, hi, _) in (("Results CI 0.52-1.00", RESULTS["1 vs. 2"]),
("Abstract CI 0.50-1.00", ABSTRACT["1 vs. 2"])):
res = {}
for lab, se in (("lower half-width", (np.log(e) - np.log(lo)) / 1.96),
("upper half-width", (np.log(hi) - np.log(e)) / 1.96),
("full width", (np.log(hi) - np.log(lo)) / (2 * 1.96))):
z = abs(np.log(e)) / se
p = 2 * norm.sf(z)
res[lab] = p
print(f" {src}, SE from {lab}: SE = {se:.3f}, z = {z:.2f}, approx. unadjusted p = {p:.3f}")
APPROX[src] = res
lo_p = min(min(r.values()) for r in APPROX.values())
hi_p = max(max(r.values()) for r in APPROX.values())
print(f" Range of normal-approximation p-values: {lo_p:.3f} to {hi_p:.3f}")
print(" A Westfall-Young min-P adjusted p is never smaller than the unadjusted p for the same")
print(f" comparison, so Padj = {a12:.3f} is below every value compatible with the printed CI.")
print(f" Halving the Table value (one-sided reading): {t12 / 2:.4f}, still not 0.010")
# ------------------------------------------------------------------ 4. minor CI mismatches
print("\n4. Other abstract-vs-Results numbers")
print(f" 1 vs. 2 CI lower bound: Abstract 0.50 vs Results 0.52, difference {0.52 - 0.50:.2f}")
print(f" Groups 1+2 vs 3+4 CI lower bound: Abstract 1.5 vs Results 1.49 -> 1.49 to 1 dp = {round(1.49, 1)}")
print(f" 1 vs. 2 CI on log scale (Results): {np.log(0.52):.3f} to {np.log(1.00):.3f}; "
f"(Abstract): {np.log(0.50):.3f} to 0.000")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 1: abstract vs Results / Table 2 text panels
fig = plt.figure(figsize=(7.2, 5.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(-1.0, 4.3)
ax.axis("off")
ax.set_facecolor(BG)
ax.text(0.2, 4.05, "Macrolide resistance determinants, group 1 (4.5 y azithromycin) vs group 2 (2.5 y)",
fontsize=10.5, weight="bold", va="center")
for x0, title in ((0.2, "Abstract (Methods and findings)"), (3.75, "Results and Table 2(A)")):
ax.add_patch(FancyBboxPatch((x0, 1.35), 3.25, 2.35, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec="#c9c8c3", lw=1))
ax.text(x0 + 0.15, 3.5, title, fontsize=10, weight="bold", color=INK2, va="center")
L = 0.35
ax.text(L, 3.12, "\u201cno significant increase in", fontsize=10, va="center")
ax.text(L, 2.87, "macrolide resistance\u201d", fontsize=10, va="center")
ax.text(L, 2.5, "fold change 0.80,", fontsize=10, va="center")
ax.text(L, 2.25, "95% CI 0.50 to 1.00,", fontsize=10, va="center")
ax.text(L, 1.75, f"Padj = {a12:.3f}", fontsize=13, weight="bold", color=ORANGE, va="center")
R = 3.9
ax.text(R, 3.12, "\u201cdid not find a statistically", fontsize=10, va="center")
ax.text(R, 2.87, "significant difference\u201d", fontsize=10, va="center")
ax.text(R, 2.5, "0.80-fold, 95% CI: 0.52 to 1.00,", fontsize=10, va="center")
ax.text(R, 2.25, f"Padj = {r12:.2f}", fontsize=13, weight="bold", color=ORANGE, va="center")
ax.text(R, 1.75, f"Table 2(A), Macrolides, 1 vs. 2: {t12:.3f}", fontsize=10.5, weight="bold",
color=ORANGE, va="center")
ax.add_patch(FancyArrowPatch((1.75, 1.75), (3.85, 1.75), arrowstyle="<|-|>", mutation_scale=12,
color=INK2, lw=1.5))
ax.text(2.6, 1.93, "conflicting Padj", fontsize=10, color=INK2, ha="center",
va="center", style="italic")
ax.text(0.2, 0.98, "Same comparison, same Padj definition (Westfall\u2013Young min-P, Wilcoxon rank-sum):",
fontsize=10, color=INK, va="center")
lines = [
f"round({t12:.3f}, 2) = {round(t12, 2):.2f}: matches the Results; it cannot give {a12:.3f} "
f"(Table value is {t12 / a12:.1f}\u00d7 larger)",
f"Read literally (fold change 0.80), {a12:.3f} would mean a significant decrease; the Results",
"deny one: \u201cdid not find a statistically significant difference\u201d",
"An upper CI bound of 1.00 implies an unadjusted p of about 0.05 or more, and a",
"min-P adjusted p cannot be smaller than the unadjusted p for the same comparison",
f"Nearest macrolide entry in Table 2(A): 1 vs. 4 = {TABLE2A['1 vs. 4'][MAC]:.3f} "
f"(Results: Padj = 0.01)",
]
for i, s in enumerate(lines):
ax.text(0.35, 0.72 - 0.25 * i, s, fontsize=10, color=BLUE, va="center")
ax.text(7.0, -0.95, "orange = as printed in the paper; blue = checks derived from the paper's numbers",
fontsize=10, color=INK2, ha="right", va="bottom")
save(fig, "fig1")
# Figure 2: p-values for the same comparison on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
rows = [
("Abstract: Padj", a12, None, ORANGE, "o", f"{a12:.3f}"),
("Results: Padj", r12, None, ORANGE, "o", f"{r12:.2f}"),
("Table 2(A): min-P Padj", t12, None, ORANGE, "o", f"{t12:.3f}"),
("Unadjusted p implied by\nResults CI 0.52–1.00", APPROX["Results CI 0.52-1.00"]["full width"],
(min(APPROX["Results CI 0.52-1.00"].values()), max(APPROX["Results CI 0.52-1.00"].values())),
BLUE, "D", None),
("Unadjusted p implied by\nAbstract CI 0.50–1.00", APPROX["Abstract CI 0.50-1.00"]["full width"],
(min(APPROX["Abstract CI 0.50-1.00"].values()), max(APPROX["Abstract CI 0.50-1.00"].values())),
BLUE, "D", None),
]
ys = np.arange(len(rows))[::-1]
for y, (lab, v, rng, c, m, txt) in zip(ys, rows):
if rng:
ax.plot(rng, [y, y], color=c, lw=2, solid_capstyle="round", zorder=2)
txt = f"{v:.2f} (range {rng[0]:.2f}–{rng[1]:.2f})"
ax.plot(v, y, marker=m, ms=9, color=c, mec=BG, mew=1.5, zorder=3, ls="none")
if rng:
ax.text(v, y + 0.32, txt, ha="center", va="center", fontsize=10, color=INK)
else:
ax.text(v * 1.18, y, txt, va="center", fontsize=10, color=INK)
ax.axvline(0.05, color=INK2, lw=0.8, ls="--", zorder=1)
ax.text(0.05 * 0.93, ys[0] + 0.55, "0.05", ha="right", va="center", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.004, 1.0)
ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_xticks([0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1.0])
ax.set_xticklabels(["0.005", "0.01", "0.02", "0.05", "0.1", "0.2", "0.5", "1"])
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xlabel("p-value, macrolides, group 1 vs group 2 (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#bdbcb7")
ax.tick_params(axis="y", length=0)
ax.plot([], [], "o", color=ORANGE, ms=9, label="as printed in the paper")
ax.plot([], [], "D", color=BLUE, ms=9, label="recomputed from the printed CI (normal approx.)")
ax.legend(loc="lower left", bbox_to_anchor=(-0.45, 1.02), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nFigures written: fig1.svg, fig1.png, fig2.svg, fig2.png")
Abstract gives the 14-week odds ratio as 2.51; Table 2 counts give 5.02
PLoS Med (2019) · doi:10.1371/journal.pmed.1002832 · PMC6588214
Summary
The abstract of this randomized trial reports the odds ratio for initiating family planning by 14 weeks postpartum, voucher versus control, as 2.51 (CI 1.31–4.79, P < 0.001), whereas Table 2 reports 5.02 [2.27–11.10] for the same comparison. Table 2's own counts, 149 of 158 (voucher) and 122 of 159 (control), give the crude odds ratio the Methods specify, (149 × 37) / (9 × 122) = 5.02; no 2 × 2 table with these group sizes reproduces 2.51 (1.31–4.79), and that interval excludes 5.02. The abstract therefore understates a secondary-outcome effect by a factor of two, although direction, significance, the primary outcome and the conclusions are unchanged. Separately, the abstract, Results and Discussion print the primary outcome's upper confidence limit as 13.97, where Table 2 prints 18.97 and the counts support about 18–19.
What the paper reports
Methods, "Sample size and statistical analysis":
We compared dichotomous outcomes between study groups by estimating crude odds ratios (ORs) with 95% confidence intervals, and testing for differences between the two groups. We estimated P-values with chi-squared testing using a level of significance of 0.05.
Abstract, "Methods and findings":
By 8 weeks postpartum, family planning was initiated in 144 (91%) participants in the voucher group and 83 (52%) participants in the control group (odds ratio [OR] 9.42; CI 4.67–13.97, P < 0.001). We also found high family planning uptake rates for both groups, with higher rates among the intervention group at 12 weeks (OR 5.66; CI 2.65–12.12, P < 0.001), 14 weeks (OR 2.51; CI 1.31–4.79, P < 0.001), 16 weeks (OR 4.02; CI 1.66–9.77, P = 0.001), and 20 weeks (OR 3.65; CI 1.40–9.47, P = 0.004) postpartum.
Table 2, "Primary and secondary outcomes by treatment group" (columns: Outcomes | Routine Care (n = 159) | FP Voucher (n = 158) | OR (95% CI) | P-Value), rows under "Initiation of FP":
8 weeks postpartum | 83 (52.2%) | 144 (91.4%) | 9.42 [4.67–18.97] | <0.001
12 weeks | 115 (72.3%) | 148 (93.7%) | 5.66 [2.65–12.12] | <0.001
14 weeks | 122 (76.7%) | 149 (94.3%) | 5.02 [2.27–11.10] | <0.001
16 weeks | 134 (84.3%) | 151 (95.6%) | 4.02 [1.66–9.77] | 0.001
20 weeks | 139 (87.4%) | 152 (96.2%) | 3.65 [1.40–9.47] | 0.004
Results, second paragraph:
By 8 weeks postpartum (primary outcome), family planning was initiated in 144 (91%) participants in the family planning voucher group and 83 (52%) participants in the standard of care group (OR 9.42; CI 4.67–13.97; Table 2). Contraceptive use rates continued to increase between 8 and 20 weeks and remained consistently significantly higher among the intervention group.
Discussion, first paragraph:
We found that over 90% of women supported through this family planning voucher accessed family planning by 8 weeks postpartum, which corresponds to 9.42 (CI 4.67–13.97) increased odds of initiating family planning by 8 weeks postpartum.
The 14-week odds ratio is not given as a number anywhere else in the text. The Author summary and Discussion refer to the 12- to 20-week results only qualitatively.
What the paper's own numbers show
Method. For each time point I took the Table 2 counts and built the 2 × 2 table: voucher initiators out of 158 against control initiators out of 159. I then computed the crude (sample) odds ratio, the Woolf (log-Wald) 95% interval, the exact conditional interval, and the Pearson chi-squared P value without continuity correction, which is the test the Methods name. For each interval the paper prints, I also back-calculated the standard error of log OR it implies, its geometric midpoint and the Wald P value. Finally I searched every 2 × 2 table with group sizes 158 and 159 for one that reproduces the abstract's 14-week figures. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| 14-week OR | Abstract | 2.51 (CI 1.31–4.79), P < 0.001 | 5.02 = (149 × 37) / (9 × 122) = 5513 / 1098 |
| 14-week OR | Table 2 | 5.02 [2.27–11.10], P < 0.001 | 5.02; Woolf 2.33–10.81; exact 2.26–12.25; χ² = 19.73, P = 8.9 × 10-6 |
| Ratio of the two 14-week ORs | Table 2 vs Abstract | 5.02 / 2.51 | 2.000 |
| Is 5.02 inside the Abstract's CI? | Abstract | 1.31–4.79 | No |
| Wald P implied by the Abstract's own OR and CI | Abstract | P < 0.001 | SE(log OR) 0.331, z = 2.78, P = 0.0054 |
| 2 × 2 tables (n = 158, 159) giving 2.51 with Woolf CI 1.31–4.79 | — | — | none |
| Nearest tables with OR ≈ 2.5 | — | — | need voucher 141 (not 149) or control 138 (not 122); these contradict the printed 94.3% and 76.7% |
| Other effect measures from the 14-week counts | — | — | risk ratio 1.229; inverse OR 0.199; risk difference 17.6 points; none is 2.51 |
| 8-week OR (primary) | Abstract, Results, Discussion | 9.42 (CI 4.67–13.97) | 9.42; Woolf 5.01–17.70; exact 4.87–19.07 |
| 8-week OR (primary) | Table 2 | 9.42 [4.67–18.97] | as above; 18.97 is the log-symmetric mirror of 4.67 around 9.42 (9.42² / 4.67 = 19.00) |
| Geometric midpoint of the 8-week interval | Abstract vs Table 2 | 4.67–13.97 vs 4.67–18.97 | 8.08 vs 9.41 (point estimate 9.42) |
| 12-, 16-, 20-week ORs | Abstract and Table 2 (identical) | 5.66 [2.65–12.12]; 4.02 [1.66–9.77]; 3.65 [1.40–9.47] | 5.66 (2.73–11.73), P = 4.3 × 10-7; 4.02 (1.69–9.60), P = 0.0008; 3.65 (1.42–9.34), P = 0.0044 |
| ORs from 8 to 20 weeks | Abstract | 9.42, 5.66, 2.51, 4.02, 3.65 | 9.42, 5.66, 5.02, 4.02, 3.65 (strictly decreasing) |
| 8-week voucher percentage | Table 2 | 144 (91.4%) | 144 / 158 = 91.1% (Abstract and Results say 91%) |
At all five time points the Table 2 odds ratios reproduce exactly from the Table 2 counts. So do the P values, to their printed precision. The Table 2 intervals are centred on their odds ratios on the log scale (geometric midpoints 9.41, 5.67, 5.02, 4.03, 3.64). They are slightly wider than the Woolf intervals, with implied standard errors 1.02 to 1.11 times the Woolf values. I could not identify the exact interval procedure used. These small differences are not part of the finding. The abstract's 14-week line is the only one of the five whose point estimate disagrees with the counts. The 8-week upper limit printed in the text is the only one of these initiation intervals that is not symmetric on the log scale around its estimate: it sits 0.70 log units below 9.42 but only 0.39 above.
Figure
Figure 1. The odds ratio for initiating family planning by 14 weeks, voucher versus control, on a log scale. Orange rows are the values printed in the paper. Filled: the abstract, 2.51 (1.31–4.79). Open: Table 2, 5.02 (2.27–11.10). Blue rows are recomputed from the Table 2 counts, 149/158 versus 122/159. Filled: the crude odds ratio with Woolf interval, 5.02 (2.33–10.81). Open: the conditional maximum-likelihood estimate with exact interval, 5.00 (2.26–12.25). The dashed blue line marks the recomputed 5.02, which falls outside the abstract's interval. The solid grey line marks no effect (OR = 1). The log scale is the natural one for ratios and does not exaggerate the gap: the abstract value is half the recomputed one.
Figure 2. Odds ratios with 95% intervals at each time point, log scale. Orange: as printed in the abstract. Blue: recomputed from the Table 2 counts (Woolf interval). At 12, 16 and 20 weeks the two agree, apart from small differences in interval method. At 14 weeks the abstract's estimate is half the recomputed one, and it breaks an otherwise steady decline from 9.42 to 3.65 as uptake rises in the control arm. At 8 weeks the point estimates agree, but the abstract's upper limit, 13.97, falls short of the recomputed 17.70 and of the 18.97 in Table 2. Labels in bold mark the abstract values that differ from Table 2.
Why it matters
- The abstract misstates an effect size. Initiation by 14 weeks is a prespecified secondary outcome. The abstract is the most widely read and most often indexed part of the paper, and it is the only place the 14-week odds ratio appears in the text. A reader of the abstract alone sees a 14-week effect of about 2.5, with an interval that excludes the value the data give, about 5.0.
- It distorts the time course. The abstract's sequence 9.42, 5.66, 2.51, 4.02, 3.65 suggests the effect dips at 14 weeks and then recovers. The counts show a steady decline (9.42, 5.66, 5.02, 4.02, 3.65) as the control arm catches up. A meta-analyst who takes the 14-week estimate from the abstract would enter a value too small by half, with a standard error too small as well (0.331 rather than 0.391).
- The abstract's 14-week line is internally inconsistent. Its interval implies P ≈ 0.005, not the P < 0.001 printed beside it. The printed P matches the Table 2 result instead (χ² P = 8.9 × 10-6).
- The primary-outcome interval is misprinted in three places. The abstract, Results and Discussion give 4.67–13.97 for the 8-week odds ratio. Table 2, which the Results sentence cites, gives 4.67–18.97, and the counts support an upper limit of about 18–19. The point estimate and its significance are unaffected, but the stated precision of the main result is overstated at the upper end.
- What is not affected. The direction and statistical significance of every comparison, the primary-outcome estimate (9.42), the 12-, 16- and 20-week results, and the paper's conclusions are unchanged. The correct 14-week value strengthens rather than weakens the reported benefit.
Explanations considered
- A text-conversion or typesetting artifact. Ruled out. The publisher's HTML abstract and typeset PDF (p. 2 abstract, p. 13 Table 2) carry the same values. So do the Europe PMC full-text XML, where each number sits in its own table cell, and the PMC copy. The two values differ by exactly a factor of two, not by a dropped digit, decimal point or sign.
- A different interval method. Cannot explain it. The interval method changes the limits, not the point estimate. Every standard method applied to these counts (Woolf, exact, conditional MLE) gives an estimate of 5.0, with intervals from about 2.3 to 11–12.
- A different effect measure. Not supported. From the same counts the risk ratio is 1.23, the inverse odds ratio 0.199 and the risk difference 17.6 percentage points. None equals 2.51.
- An adjusted estimate. Not supported by the paper. The Methods mention multivariable logistic regression to assess confounding, but the paper reports no adjusted 14-week estimate and does not label the abstract value as adjusted. The other four odds ratios in the same abstract sentence are identical to the crude Table 2 values, and the abstract's "P < 0.001" agrees with the crude test, not with its own interval.
- A subgroup. Not supported. The only subgroup in Table 2 is women who wanted no pregnancy in the next 2 years (n = 72 per arm, as implied by its counts and percentages). Its odds ratios are 8.31 at 8 weeks and 4.01 at 6 months, and it has no 14-week row.
- An error in the Table 2 counts rather than the abstract. Not supported. The counts match their own percentages (149/158 = 94.3%, 122/159 = 76.7%) and give exactly the odds ratio and P value printed beside them. To obtain an odds ratio near 2.5 the voucher count would have to be 141 instead of 149, or the control count 138 instead of 122. Either would contradict the printed percentages and put the 14-week count out of order with its neighbours (148 at 12 weeks, 151 at 16 weeks; 115 and 134 in the control arm). No 2 × 2 table with these group sizes reproduces 2.51 (1.31–4.79). The abstract interval is also not simply the Table 2 interval halved: 2.27/1.31 = 1.73 and 11.10/4.79 = 2.32. I could not identify how the abstract figures were produced.
- A correction already issued. None found (see Status).
- The 8-week upper limit 13.97. It differs from Table 2's 18.97 in a single digit. Table 2's value is supported by three things: the counts (Woolf upper limit 17.70, exact 19.07), the log-scale symmetry of the other four initiation intervals, and the Results sentence's own citation of Table 2. The printed 13.97 makes the interval asymmetric around 9.42 (geometric midpoint 8.08).
- Side observation. Table 2 prints the 8-week voucher percentage as 91.4%; 144/158 is 91.1%. The abstract and Results round it to 91%. This affects no conclusion.
Status
- Version of record. The DOI resolves to the PLOS Medicine article page (journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1002832). Three copies were checked: the HTML abstract, the publisher's 20-page printable PDF (text extracted with PyMuPDF; abstract on p. 2, Results p. 9, Discussion p. 10, Table 2 p. 13), and the Europe PMC full-text XML. All three carry the values quoted here: 2.51 (1.31–4.79) in the abstract, 5.02 [2.27–11.10] in Table 2, and 4.67–13.97 in the text against 4.67–18.97 in Table 2. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. S1 Text (counseling standard operating procedure), S2 Text (study protocol), S3 Text (follow-up tool) and S4 Text (CONSORT checklist). None contains outcome or source data. The deciding counts are already in Table 2.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by entries and an empty relation field. The PMC record lists no related notices, and the PLOS article page shows no correction notice and "Reader Comments (0)". The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title with "PubPeer" found no thread. - Sources. PMC article PMC6588214; https://doi.org/10.1371/journal.pmed.1002832 (PLOS Medicine HTML and printable PDF); Europe PMC full-text XML; Crossref; Europe PMC; PubPeer; trial registration NCT02964169.
Suggested correction
In the abstract, replace
14 weeks (OR 2.51; CI 1.31–4.79, P < 0.001)
with
14 weeks (OR 5.02; CI 2.27–11.10, P < 0.001)
so that it matches Table 2 and the counts 149/158 versus 122/159.
In the abstract, the second paragraph of the Results and the first paragraph of the Discussion, replace the 8-week interval "CI 4.67–13.97" with "CI 4.67–18.97", as in Table 2. The authors may also wish to change the 8-week voucher percentage in Table 2 from 91.4% to 91.1% (144/158), and to state which interval method Table 2 uses, since its limits differ slightly from the Woolf intervals for the same counts.
Recomputation script (python recompute.py.txt)
"""Recompute the odds ratios of Table 2 of PMC6588214
(PLoS Med 2019, doi:10.1371/journal.pmed.1002832,
"Provision of family planning vouchers and early initiation of postpartum contraceptive use
among women living with HIV in southwestern Uganda: A randomized controlled trial").
Run: python recompute.py.txt
Needs numpy, scipy (>= 1.10, for scipy.stats.contingency.odds_ratio) and matplotlib.
Prints every recomputed number used in writeup.md and writes fig1.svg/png and
fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Table 2, "Initiation of FP" rows: Routine Care (n = 159) and FP Voucher (n = 158) counts,
the printed OR [95% CI] and P-value for 8, 12, 14, 16 and 20 weeks.
Abstract (Methods and findings): OR; CI; P for the same five time points.
Methods: "crude odds ratios (ORs) with 95% confidence intervals" and "P-values with
chi-squared testing".
"""
import os
import numpy as np
from scipy import stats
from scipy.stats.contingency import odds_ratio
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
N_CTRL, N_VOUCH = 159, 158
# week: (control events, voucher events, Table 2 OR, lo, hi, Table 2 P, Abstract OR, lo, hi, Abstract P)
ROWS = {
8: (83, 144, 9.42, 4.67, 18.97, "<0.001", 9.42, 4.67, 13.97, "< 0.001"),
12: (115, 148, 5.66, 2.65, 12.12, "<0.001", 5.66, 2.65, 12.12, "< 0.001"),
14: (122, 149, 5.02, 2.27, 11.10, "<0.001", 2.51, 1.31, 4.79, "< 0.001"),
16: (134, 151, 4.02, 1.66, 9.77, "0.001", 4.02, 1.66, 9.77, "= 0.001"),
20: (139, 152, 3.65, 1.40, 9.47, "0.004", 3.65, 1.40, 9.47, "= 0.004"),
}
TABLE2_PCT = {8: (52.2, 91.4), 12: (72.3, 93.7), 14: (76.7, 94.3), 16: (84.3, 95.6), 20: (87.4, 96.2)}
def crude(a, n1, c, n0):
"""a events of n1 (voucher), c events of n0 (control): sample OR, Woolf CI, Pearson chi2 P, Fisher P."""
b, d = n1 - a, n0 - c
OR = (a * d) / (b * c)
se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
lo, hi = np.exp(np.log(OR) - Z * se), np.exp(np.log(OR) + Z * se)
chi2, p, _, _ = stats.chi2_contingency([[a, b], [c, d]], correction=False)
pf = stats.fisher_exact([[a, b], [c, d]])[1]
r = odds_ratio([[a, b], [c, d]], kind="conditional")
ci = r.confidence_interval(0.95)
return dict(OR=OR, se=se, lo=lo, hi=hi, chi2=chi2, p=p, pf=pf,
cmle=r.statistic, elo=ci.low, ehi=ci.high,
rr=(a / n1) / (c / n0))
def implied(OR, lo, hi):
"""SE(log OR), geometric midpoint and Wald P implied by a printed OR and CI."""
se = (np.log(hi) - np.log(lo)) / (2 * Z)
mid = np.sqrt(lo * hi)
z = np.log(OR) / se
return se, mid, 2 * stats.norm.sf(abs(z)), z
R = {}
print("=== 1. Crude ORs recomputed from the Table 2 counts (voucher n = 158, control n = 159) ===")
print(f"{'week':>4s} {'ctrl':>9s} {'vouch':>9s} {'OR':>6s} {'Woolf 95% CI':>15s} {'exact 95% CI':>15s} "
f"{'chi2':>6s} {'chi2 P':>9s} {'Fisher P':>9s} | Table 2")
for w, (c, a, tor, tlo, thi, tp, *_rest) in ROWS.items():
r = crude(a, N_VOUCH, c, N_CTRL)
R[w] = r
print(f"{w:4d} {c:4d} {100*c/N_CTRL:4.1f}% {a:4d} {100*a/N_VOUCH:4.1f}% {r['OR']:6.2f} "
f"[{r['lo']:5.2f}–{r['hi']:5.2f}] [{r['elo']:5.2f}–{r['ehi']:5.2f}] {r['chi2']:6.2f} "
f"{r['p']:9.2e} {r['pf']:9.2e} | {tor:.2f} [{tlo:.2f}–{thi:.2f}] P {tp}")
print()
print("=== 2. The 14-week row in detail ===")
c, a = ROWS[14][0], ROWS[14][1]
r = R[14]
print(f"voucher {a}/{N_VOUCH} (non-initiators {N_VOUCH-a}); control {c}/{N_CTRL} (non-initiators {N_CTRL-c})")
print(f"OR = ({a} x {N_CTRL-c}) / ({N_VOUCH-a} x {c}) = {a*(N_CTRL-c)} / {(N_VOUCH-a)*c} = {r['OR']:.4f}")
print(f"SE(log OR) = {r['se']:.3f}; Woolf CI {r['lo']:.2f}–{r['hi']:.2f}; exact (conditional MLE {r['cmle']:.2f}) CI {r['elo']:.2f}–{r['ehi']:.2f}")
print(f"Pearson chi2 = {r['chi2']:.2f}, P = {r['p']:.1e}")
print(f"Table 2 OR / Abstract OR = 5.02 / 2.51 = {5.02/2.51:.3f}; 5.02 / 2 = {5.02/2:.2f}")
se_a, mid_a, p_a, z_a = implied(2.51, 1.31, 4.79)
se_t, mid_t, p_t, z_t = implied(5.02, 2.27, 11.10)
print(f"Abstract 2.51 (1.31–4.79): geometric midpoint {mid_a:.3f}, implied SE {se_a:.3f}, z {z_a:.2f}, Wald P {p_a:.4f}")
print(f"Table 2 5.02 (2.27–11.10): geometric midpoint {mid_t:.3f}, implied SE {se_t:.3f}, z {z_t:.2f}, Wald P {p_t:.1e}")
print(f"Does the abstract CI contain the recomputed OR {r['OR']:.2f}? {1.31 <= r['OR'] <= 4.79}")
print(f"Table 2 CI limits / Abstract CI limits: 2.27/1.31 = {2.27/1.31:.2f}, 11.10/4.79 = {11.10/4.79:.2f} (not a uniform halving)")
print(f"Subgroup 'no pregnancy in next 2 years', 8 wk: 38/0.528 = {38/0.528:.1f}, 65/0.903 = {65/0.903:.1f} -> n = 72 per arm")
print(f"Other quantities from the same counts: risk ratio {r['rr']:.3f}; inverse OR {1/r['OR']:.3f}; "
f"risk difference {100*(a/N_VOUCH - c/N_CTRL):.1f} percentage points")
print()
print("=== 3. Can any 2x2 table with n = 158 and n = 159 give 2.51 (1.31–4.79)? ===")
hits = []
near = []
for aa in range(1, N_VOUCH):
for cc in range(1, N_CTRL):
b, d = N_VOUCH - aa, N_CTRL - cc
OR = aa * d / (b * cc)
se = np.sqrt(1 / aa + 1 / b + 1 / cc + 1 / d)
lo, hi = np.exp(np.log(OR) - Z * se), np.exp(np.log(OR) + Z * se)
if round(OR, 2) == 2.51 and abs(lo - 1.31) <= 0.01 and abs(hi - 4.79) <= 0.01:
hits.append((aa, cc))
if round(OR, 2) == 2.51:
near.append((aa, cc, round(lo, 2), round(hi, 2)))
print(f"tables reproducing OR 2.51 with Woolf CI 1.31–4.79 (±0.01): {hits if hits else 'none'}")
print(f"tables with OR rounding to 2.51 (any CI): {len(near)}; their voucher counts range "
f"{min(n[0] for n in near)}–{max(n[0] for n in near)}, control counts {min(n[1] for n in near)}–{max(n[1] for n in near)}")
print(" of these, the ones keeping voucher = 149 or control = 122:",
[n for n in near if n[0] == 149 or n[1] == 122] or "none")
for aa in range(130, N_VOUCH):
OR = aa * (N_CTRL - 122) / ((N_VOUCH - aa) * 122)
if 2.3 <= OR <= 2.7:
print(f" control fixed at 122: voucher {aa} gives OR {OR:.2f}")
for cc in range(90, N_CTRL):
OR = 149 * (N_CTRL - cc) / ((N_VOUCH - 149) * cc)
if 2.3 <= OR <= 2.7:
print(f" voucher fixed at 149: control {cc} gives OR {OR:.2f}")
print()
print("=== 4. Every Abstract OR/CI/P against the recomputed values ===")
for w, (c, a, tor, tlo, thi, tp, aor, alo, ahi, ap) in ROWS.items():
se_i, mid_i, p_i, _ = implied(aor, alo, ahi)
r = R[w]
print(f"{w:2d} wk Abstract {aor:.2f} ({alo:.2f}–{ahi:.2f}) P {ap}: midpoint {mid_i:.2f}, implied Wald P {p_i:.1e} | "
f"recomputed OR {r['OR']:.2f}, Woolf {r['lo']:.2f}–{r['hi']:.2f}, chi2 P {r['p']:.1e}; "
f"Abstract = Table 2? {(aor, alo, ahi) == (tor, tlo, thi)}")
print()
print("=== 5. The 8-week (primary outcome) upper limit ===")
for lab, (o, lo, hi) in [("Abstract/Results/Discussion", (9.42, 4.67, 13.97)), ("Table 2", (9.42, 4.67, 18.97))]:
se_i, mid_i, p_i, _ = implied(o, lo, hi)
print(f"{lab:28s} 9.42 ({lo}–{hi}): geometric midpoint {mid_i:.2f}; log distance OR->lo {np.log(o/lo):.3f}, OR->hi {np.log(hi/o):.3f}")
r = R[8]
print(f"recomputed: OR {r['OR']:.2f}, Woolf {r['lo']:.2f}–{r['hi']:.2f}, exact {r['elo']:.2f}–{r['ehi']:.2f}")
print(f"upper limit mirroring 4.67 on the log scale around 9.42: {9.42**2/4.67:.2f}")
print()
print("=== 6. Table 2 intervals: symmetric on the log scale, wider than Woolf (procedure not identified) ===")
for w, (c, a, tor, tlo, thi, *_r) in ROWS.items():
se_i, mid_i, _, _ = implied(tor, tlo, thi)
print(f"{w:2d} wk Table 2 midpoint {mid_i:.2f} (OR {tor}); implied SE {se_i:.3f} vs Woolf SE {R[w]['se']:.3f} "
f"(ratio {se_i/R[w]['se']:.2f})")
print()
print("=== 7. Table 2 percentages ===")
for w, (c, a, *_r) in ROWS.items():
pc, pv = TABLE2_PCT[w]
print(f"{w:2d} wk control {c}/{N_CTRL} = {100*c/N_CTRL:.1f}% (printed {pc}); voucher {a}/{N_VOUCH} = {100*a/N_VOUCH:.1f}% (printed {pv})")
print()
print("=== 8. Time course of the OR ===")
print("Abstract :", [ROWS[w][6] for w in ROWS])
print("Recomputed:", [round(R[w]['OR'], 2) for w in ROWS])
ors = [R[w]['OR'] for w in ROWS]
print("recomputed sequence strictly decreasing?", all(x > y for x, y in zip(ors, ors[1:])))
ab = [ROWS[w][6] for w in ROWS]
print("abstract sequence strictly decreasing?", all(x > y for x, y in zip(ab, ab[1:])))
# ---------------------------------------------------------------- figures
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
TICKS = [0.5, 1, 2, 5, 10, 20]
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
def style_log_x(ax, lo, hi):
ax.set_xscale("log")
ax.set_xlim(lo, hi)
ax.set_xticks([t for t in TICKS if lo <= t <= hi])
ax.set_xticklabels([f"{t:g}" for t in TICKS if lo <= t <= hi])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
# ---- fig1: the 14-week odds ratio, four versions on one log axis
r14 = R[14]
items = [
("Abstract", 2.51, 1.31, 4.79, ORANGE, True, "2.51 (1.31–4.79)"),
("Table 2", 5.02, 2.27, 11.10, ORANGE, False, "5.02 (2.27–11.10)"),
("Recomputed, Woolf", r14["OR"], r14["lo"], r14["hi"], BLUE, True,
f"{r14['OR']:.2f} ({r14['lo']:.2f}–{r14['hi']:.2f})"),
("Recomputed, exact", r14["cmle"], r14["elo"], r14["ehi"], BLUE, False,
f"{r14['cmle']:.2f} ({r14['elo']:.2f}–{r14['ehi']:.2f})"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.3))
style_log_x(ax, 0.5, 30)
ys = np.arange(len(items))[::-1].astype(float)
for (lab, o, lo, hi, col, filled, txt), y in zip(items, ys):
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(o, y, "o", ms=9, color=col, mfc=col if filled else BG, mec=col, mew=2, zorder=3)
ax.text(hi * 1.12, y, txt, ha="left", va="center", fontsize=10, color=col)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(1.04, ys[0] + 0.55, "no effect (OR = 1)", ha="left", va="center", fontsize=10, color=INK2)
ax.axvline(r14["OR"], color=BLUE, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.set_yticks(ys)
ax.set_yticklabels([i[0] for i in items])
ax.set_ylim(-0.6, len(items) - 0.2)
ax.set_xlabel("Odds ratio for FP initiation by 14 weeks, voucher vs control (log scale)")
handles = [Line2D([], [], marker="o", color=ORANGE, lw=2, ms=8, label="reported in the paper"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=8, label="recomputed from Table 2 counts 149/158 vs 122/159")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.42, -0.22), ncol=2, frameon=False, fontsize=10,
handletextpad=0.4, columnspacing=1.2)
save(fig, "fig1")
# ---- fig2: time course, abstract vs recomputed, ORs with CIs on one log axis (x = OR, rows = weeks)
fig, ax = plt.subplots(figsize=(7.2, 4.0))
style_log_x(ax, 0.8, 30)
weeks = list(ROWS)
base = np.arange(len(weeks))[::-1].astype(float)
off = 0.17
for w, y in zip(weeks, base):
c, a, tor, tlo, thi, tp, aor, alo, ahi, ap = ROWS[w]
r = R[w]
flag = (aor, alo, ahi) != (tor, tlo, thi)
ax.plot([alo, ahi], [y + off] * 2, color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(aor, y + off, "o", ms=8, color=ORANGE, zorder=3)
ax.plot([r["lo"], r["hi"]], [y - off] * 2, color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot(r["OR"], y - off, "o", ms=8, color=BLUE, zorder=3)
ax.text(21, y + off, f"{aor:.2f} ({alo:.2f}–{ahi:.2f})", ha="left", va="center", fontsize=10,
color=ORANGE, fontweight="bold" if flag else "normal")
ax.text(21, y - off, f"{r['OR']:.2f} ({r['lo']:.2f}–{r['hi']:.2f})", ha="left", va="center", fontsize=10, color=BLUE)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.set_yticks(base)
ax.set_yticklabels([f"{w} weeks" + (" (primary)" if w == 8 else "") for w in weeks])
ax.set_ylim(-0.6, len(weeks) - 0.3)
ax.set_xlim(0.8, 60)
ax.set_xticks([1, 2, 5, 10, 20])
ax.set_xticklabels(["1", "2", "5", "10", "20"])
ax.set_xlabel("Odds ratio for FP initiation, voucher vs control (log scale)")
handles = [Line2D([], [], marker="o", color=ORANGE, lw=2, ms=8, label="Abstract (bold where it differs from Table 2)"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=8, label="recomputed from Table 2 counts (Woolf CI)")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.42, -0.16), ncol=1, frameon=False, fontsize=10)
save(fig, "fig2")
print()
print("wrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
The abstract says RNF213 loss lowers IFN-β-induced ISG expression, but Fig 2A shows no difference
PLoS Pathog (2025) · doi:10.1371/journal.ppat.1013333 · PMC12251097
Summary
The abstract states that RNF213 deficiency "decreases the expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation and Acinetobacter baumannii infection." For the IFN-β condition the Results say the opposite: the IFN-β-induced increases in Cxcl9 and Cxcl10 "were comparable between WT and Rnf213–/– MSFs," and all 8 IFN-β comparisons of Cxcl9/Cxcl10 in Fig 2A (two cell types, two genes, two time points) are labelled "ns." Bar heights read off the published Fig 2A give a knockout-to-wild-type ratio of about 0.91–1.31 (geometric mean 1.04) under IFN-β, with the knockout numerically higher in half of the comparisons, versus about 0.30–0.63 (geometric mean 0.44, all 16 significant) under A. baumannii or LPS. This matters because the paper uses the intact IFN-β response as its evidence that RNF213 acts upstream of the IFN-α/β receptor (IFNAR), so the abstract sentence points to the opposite location in the pathway from the one the paper argues for.
What the paper reports
In the abstract:
RNF213 deficiency impairs type I interferon (IFN-I) production and decreases the expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation and Acinetobacter baumannii infection.
— Abstract
In the body, the same experiment (Fig 2A) is described as follows:
The increases in the expression of Cxcl9 and Cxcl10 induced by A. baumannii infection and LPS stimulation were much lower in Rnf213–/– MSFs than in WT MSFs (Fig 2A).
However, the increases in the expression of Cxcl9 and Cxcl10 induced by IFN-β treatment were comparable between WT and Rnf213–/– MSFs (Fig 2A).
Taken together, our results indicate that RNF213 positively regulates IFN-I signaling possibly by mediating events upstream of IFNAR binding.
— Results, section "IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs"
RNF213 deficiency inhibited TBK1 phosphorylation and IFN-I production in response to LPS stimulation and A. baumannii infection but not in response to IFN-β treatment, suggesting that RNF213 positively regulates IFN-I signaling by acting on targets upstream of IFNAR (Figs 2D, 2E, S3C and S3D).
— Results, section "RNF213 targets TRAF3", first sentence
The cited figure is described in its legend as:
qRT-PCR analysis of Rnf213, Cxcl9 and Cxcl10 in WT and Rnf213–/– MSFs and BMECs without treatment or infected with A. baumannii (50 MOI); or stimulated with LPS (200 ng/mL) and IFN-β (2 ng/mL) for indicated times (n = 4 technical replicates; 3 independent experiments).
— Fig 2 legend, panel A
Each quotation was checked character for character against the article as archived in PubMed Central (src/show.py PMC12251097). Each one occurs exactly once; recompute.py.txt reruns that check.
What the paper's own numbers show
Method. The claim is about direction (a decrease versus no change), so the check has two parts.
- The significance label printed above each wild-type/knockout pair in Fig 2A was transcribed exactly, for every stimulated Cxcl9 and Cxcl10 comparison (3 stimuli × 2 cell types × 2 genes × 2 time points = 24). The legend states these labels come from two-sided Student's t-tests without multiple-comparison correction.
- To show the size of each difference, the article's Data Availability statement reports no numeric source data, so the bar heights (gene/Hprt) were measured on the publisher's large Fig 2 image. For each panel a straight line was fit through the printed y-axis tick marks (largest residual 0.2% of the axis maximum), and the top edge of each outlined bar was read against it. Where the replicate dots cover a bar's top, the value is bracketed between the top of the bar outline and the middle of the dot cluster; that range is carried through to the ratio. These readings are approximate. The conclusion does not rest on them: the printed "ns" labels and the Results text already state there is no difference under IFN-β.
recompute.py.txt prints every value below and redraws both figures.
| Quantity | Location | Reported | Recomputed from the paper's own data |
|---|---|---|---|
| Effect of RNF213 deficiency on ISG expression after IFN-β | Abstract | "decreases the expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation" | Not supported: no IFN-β ISG comparison in the paper shows a decrease (rows below) |
| Same effect, as described in the body | Results, Fig 2A paragraph | "comparable between WT and Rnf213–/– MSFs" | Agrees with Fig 2A |
| Cxcl9/Cxcl10 after IFN-β, MSFs + BMECs, 3 h and 6 h (8 comparisons) | Fig 2A, IFN-β row | All 8 labelled "ns" | Knockout ÷ WT ≈ 0.91–1.31; geometric mean 1.04; knockout numerically higher in 4 of 8 |
| MSF Cxcl10, 6 h, IFN-β | Fig 2A | "ns" | Knockout ÷ WT ≈ 1.31 (≈ 0.61 vs ≈ 0.47; knockout higher) |
| Cxcl9/Cxcl10 after A. baumannii, MSFs + BMECs, 3 h and 6 h (8 comparisons) | Fig 2A, A. baumannii row | All *, ** or *** | Knockout ÷ WT ≈ 0.35–0.63; geometric mean 0.44 |
| Cxcl9/Cxcl10 after LPS, MSFs + BMECs, 6 h and 12 h (8 comparisons) | Fig 2A, LPS row | All ** or *** | Knockout ÷ WT ≈ 0.30–0.57; geometric mean 0.44 |
| A. baumannii and LPS together (16 comparisons) | Fig 2A | 16 of 16 significant | Knockout ÷ WT ≈ 0.30–0.63 (largest upper bound of a reading range 0.72); geometric mean 0.44 |
| IFN-I production / TBK1 phosphorylation after IFN-β | Results, "RNF213 targets TRAF3" | "but not in response to IFN-β treatment" | No panel cited there (Figs 2D, 2E, S3C–D) includes IFN-β-stimulated cells; their legends list only untreated, A. baumannii and LPS conditions |
So under bacterial infection or LPS, knockout cells induce roughly 40–45% as much Cxcl9/Cxcl10 as wild-type cells (geometric mean 0.44 for both stimuli), and every one of the 16 comparisons is marked significant. Under IFN-β the two genotypes are indistinguishable: the ratios scatter on both sides of 1 (geometric mean 1.04; knockout numerically higher in 4 of 8), and all 8 are marked "ns." The paper's body reports exactly this pattern. Only the abstract sentence departs from it.
Figure
Figure 1. The conflicting statements, side by side. Left (orange): the abstract sentence, with the IFN-β ISG clause highlighted. Right (blue): the Fig 2A Results sentence, the first sentence of "RNF213 targets TRAF3" (which concerns TBK1 phosphorylation and IFN-I production and cites Figs 2D, 2E, S3C–D), and a count of the significance labels in the IFN-β row of Fig 2A. The abstract's ISG clause and the Fig 2A sentence describe the same experiment and disagree only on the IFN-β condition; on A. baumannii and LPS the abstract and Results agree.
Figure 2. Size of the knockout effect for each stimulus in Fig 2A. Each filled blue dot is the ratio Rnf213–/– ÷ WT of mean Cxcl9 or Cxcl10 mRNA (gene/Hprt) for one cell type, gene and time point, from bar heights measured on the published Fig 2A; the short blue line is the range left by the measurement where replicate dots cover a bar top. Open blue diamonds are the per-stimulus geometric means. The orange label at right is the significance label printed in Fig 2A for that pair. The x-axis is logarithmic and the vertical line at 1 marks no difference. The 8 IFN-β ratios cluster around 1 (geometric mean 1.04) and all are "ns"; the 16 A. baumannii/LPS ratios sit between about 0.30 and 0.63 (geometric mean 0.44 for each stimulus) and all are *, ** or ***. The two clusters are far apart relative to the reading uncertainty.
Why it matters
- The abstract conflicts with the paper's own mechanistic inference. The body argues that RNF213 acts upstream of IFNAR, at TRAF3, precisely because knockout cells respond normally to exogenous IFN-β while producing less IFN-I after bacterial or LPS stimulation ("suggesting that RNF213 positively regulates IFN-I signaling by acting on targets upstream of IFNAR"). If RNF213 deficiency also reduced ISG induction by IFN-β, as the abstract states, RNF213 would be required downstream of the receptor, and this experiment would no longer support the upstream placement. A reader who sees only the abstract, as in indexing services, gets the mechanism wrong on this point.
- The main conclusions are unaffected and are in fact supported by the correct reading. RNF213 promotes K27-linked polyubiquitination of TRAF3 at K160, drives IFN-I production, and maintains blood–brain barrier integrity during A. baumannii infection. The title, author summary, figures, Results and Discussion all agree with the Results passages quoted above. The error is confined to one abstract sentence. The internal rescue experiment (Fig 4C), in which IFN-β restores Claudin-5/Occludin/Pecam1 expression in Rnf213–/– BMECs, likewise requires an intact IFN-β response and so agrees with the body, not the abstract.
- No numeric estimate is affected. The problem is a wrong statement of direction for one condition, not a miscalculated value.
Explanations considered
- "In response to IFN-β stimulation" modifies only "IFN-I production," not the ISG clause. This does not resolve the conflict. The paper never measures IFN-I production after IFN-β stimulation (the ELISA panels Fig 2B–C cover only A. baumannii and LPS), and IFN-β is itself a type I interferon, so "IFN-I production in response to IFN-β" is not a coherent separate readout. Under the parsing that attaches the phrase to ISG expression, it contradicts Fig 2A; under the other parsing, the Results already say IFN-I signaling is unaffected by IFN-β. Neither reading matches the body.
- The abstract refers to a different cell type or experiment. The only experiment in which Rnf213–/– cells are stimulated with IFN-β and ISGs are measured is Fig 2A, which covers both MSFs and BMECs; all 8 Cxcl9/Cxcl10 comparisons are "ns." Fig 1B–C uses IFN-β to induce Rnf213 in WT versus Ifnar–/– cells, a different genotype contrast. Fig 4C uses IFN-β pretreatment of BMECs, and the Results/Discussion describe the outcome as a rescue ("reversed by treatment with IFN-β" / "rescued by treatment with IFN-β"), which is the opposite of a reduced response. None of the eight supplementary figure legends reports IFN-β-stimulated ISG data in knockout versus WT cells.
- The Rnf213 transcript itself. In the IFN-β row of Fig 2A the Rnf213 panels are "***" because the gene is deleted in the knockout. Rnf213 is IFN-inducible (Fig 1B–C), but it is the targeted gene, not an ISG readout of the knockout's downstream signalling capacity, and neither the abstract nor the Results presents it as one.
- Low statistical power (absence of evidence). One could argue that "ns" with n = 4 technical replicates does not exclude a small decrease. But the abstract asserts a decrease, which "ns" does not support either way, and the point estimates show no tendency toward one: the geometric-mean ratio is 1.04 and the knockout is numerically higher in half the comparisons. With the same design the same figure detects 2- to 3-fold decreases under A. baumannii and LPS.
- Conversion or typesetting artifact. Ruled out. The same wording appears in the PMC-archived text, the Europe PMC full-text XML and the publisher's HTML version of record (labelled "Version 2").
- Already corrected or discussed. No correction, erratum, expression of concern or retraction was found (see Status).
None of these accounts for the conflict. The most plausible benign reading is that the abstract meant "LPS" where it wrote "IFN-β," since the Results support a decrease under LPS. That is a conjecture about wording and does not change the fact that the abstract as published contradicts the Results.
Status
As of 2026-10-23:
- Version of record: checked. The publisher's article page (journals.plos.org, reached through doi.org) offers "Version 2 / Version 1 / Uncorrected Proof." The current version carries all quoted passages verbatim: the abstract sentence, both Fig 2A Results sentences ("much lower" and "comparable"), the "but not in response to IFN-β treatment" sentence, and the Fig 4C "reversed by treatment with IFN-β" sentence. The same passages appear once each in the Europe PMC full-text XML.
- Figure data: Fig 2A has no numeric source data (Data Availability: all data are within the manuscript and its supporting files). The published Fig 2 image was measured for Fig 2 of this write-up, and the significance labels were transcribed from it. Every stimulated Cxcl9/Cxcl10 comparison was included.
- Supplements: the legends of all supporting items (S1–S8 Fig, S1–S2 Table) were read. None reports IFN-β-stimulated ISG data in Rnf213–/– versus WT cells. S3C–D, cited by the "RNF213 targets TRAF3" sentence, covers only untreated, A. baumannii and LPS conditions.
- Corrections and notices:
src/known.pyreturns only a publisher "New version" record dated 2025-07-11, which is PLOS's routine versioning (Uncorrected Proof → Version 1 → Version 2), not a correction. PubMed Central and Europe PMC list no notices. The publisher's page shows no correction, erratum, expression of concern or retraction. - Comments and PubPeer: the PLOS reader-comments page shows "Reader Comments (0)." PubPeer could not be queried directly (its search and API returned HTTP 403 / Cloudflare), and the one available web search for the title plus "pubpeer" could not run because the session's search budget was exhausted, so a PubPeer thread cannot be positively ruled out.
- Peer review: the open peer-review history published with the article was read (Europe PMC XML sub-articles r001–r004) along with the author response attachment. In Decision Letter 1, Reviewer #3 proposed the wording that became the "RNF213 targets TRAF3" sentence, including "but not in response to IFN-β treatment," and the authors adopted it. In the author response, the authors themselves wrote that "RNF213 deficiency did not affect the expression of ISGs during IFN-β stimulation (Fig 2)," which agrees with the Results and not the abstract. No review comment or author response addresses the abstract's IFN-β wording.
- Sources: PMC12251097 (
src/show.py);src/known.py 10.1371/journal.ppat.1013333; https://doi.org/10.1371/journal.ppat.1013333; the PLOS article, figure-image and reader-comments pages; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC12251097/fullTextXML (including peer-review sub-articles and the author response supplementary file).
Suggested correction
Restrict the IFN-β clause in the abstract to what the data show. For example:
RNF213 deficiency impairs type I interferon (IFN-I) production and decreases the expression of interferon-stimulated genes (ISGs) in response to Acinetobacter baumannii infection and LPS stimulation, whereas ISG induction by IFN-β is unaffected.
Alternatively, delete "IFN-β stimulation and" from the sentence. Separately, the first sentence of "RNF213 targets TRAF3" states that TBK1 phosphorylation and IFN-I production were unaffected after IFN-β treatment, yet none of the panels it cites (Figs 2D, 2E, S3C, S3D) includes IFN-β-treated samples; Fig 2A holds only Cxcl9/Cxcl10/Rnf213 mRNA after IFN-β, not TBK1 or IFN-I readouts. That sentence could restrict its IFN-β clause to ISG induction and cite Fig 2A (for example, "…but did not affect Cxcl9 and Cxcl10 induction by IFN-β treatment (Fig 2A)"), or drop the IFN-β clause for the TBK1/IFN-I readouts, for which no IFN-β-treated samples are shown.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC12251097 (PLoS Pathog 2025, doi:10.1371/journal.ppat.1013333)
# Abstract vs Results / Fig 2A: does RNF213 deficiency lower ISG expression after IFN-beta?
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs numpy and matplotlib.
#
# Inputs
# * Quotes: copied verbatim from the article as archived in PubMed Central (PMC12251097).
# When run inside the /home/ubuntu/papers repo, the script re-checks each quote against
# the archived full text with src/show.py (skipped elsewhere).
# * Fig 2A: the article gives no numeric source data (Data Availability: all data are in
# the manuscript and its supporting files). The pixel coordinates below were measured on
# the publisher's large Fig 2 image
# https://journals.plos.org/plospathogens/article/figure/image?size=large&id=10.1371/journal.ppat.1013333.g002
# (2232 x 2576 px), by locating the y-axis tick marks and the top edge of each outlined
# bar (blue = WT, red = Rnf213-/-). Where the replicate dots hide a bar's top edge, the
# top is bracketed between the middle of the dot cluster and the highest visible pixel of
# the bar outline; otherwise +/-1 px. Tick values and significance labels are transcribed
# exactly as printed. All 24 stimulated Cxcl9/Cxcl10 comparisons are included (3 stimuli
# x 2 cell types x 2 genes x 2 time points); untreated (0 h) bars are not used.
# The finding does not rest on these measurements: the printed 'ns' labels and the
# Results text say the same thing. The measurements only show the size of each difference.
import math, os, statistics, subprocess
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ quotes
QUOTES = {
"abstract": ("Abstract",
"RNF213 deficiency impairs type I interferon (IFN-I) production and decreases the "
"expression of interferon-stimulated genes (ISGs) in response to IFN-β stimulation "
"and Acinetobacter baumannii infection."),
"results_down": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
"The increases in the expression of Cxcl9 and Cxcl10 induced by A. baumannii infection "
"and LPS stimulation were much lower in Rnf213–/– MSFs than in WT MSFs (Fig 2A)."),
"results_ifnb": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
"However, the increases in the expression of Cxcl9 and Cxcl10 induced by IFN-β treatment "
"were comparable between WT and Rnf213–/– MSFs (Fig 2A)."),
"results_bmec": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
"Similar patterns of A. baumannii infection- and LPS stimulation-induced Cxcl9 and "
"Cxcl10 expression were observed in BMECs (Fig 2A)."),
"results_upstream": ("Results, 'IFN-I signaling is impaired in Rnf213–/– MSFs and BMECs'",
"Taken together, our results indicate that RNF213 positively regulates IFN-I signaling "
"possibly by mediating events upstream of IFNAR binding."),
"results_traf3": ("Results, 'RNF213 targets TRAF3', first sentence",
"RNF213 deficiency inhibited TBK1 phosphorylation and IFN-I production in response to LPS "
"stimulation and A. baumannii infection but not in response to IFN-β treatment, suggesting "
"that RNF213 positively regulates IFN-I signaling by acting on targets upstream of IFNAR "
"(Figs 2D, 2E, S3C and S3D)."),
"fig2_legend": ("Fig 2 legend, panel A",
"qRT-PCR analysis of Rnf213, Cxcl9 and Cxcl10 in WT and Rnf213–/– MSFs and BMECs without "
"treatment or infected with A. baumannii (50 MOI); or stimulated with LPS (200 ng/mL) and "
"IFN-β (2 ng/mL) for indicated times (n = 4 technical replicates; 3 independent experiments)."),
"fig4c": ("Results, 'RNF213 modulates blood–brain barrier integrity via IFN-I signaling'",
"Notably, the reduced expression of Claudin-5, Occludin, and Pecam1 in Rnf213–/– BMECs "
"was reversed by treatment with IFN-β (Fig 4C)."),
}
def check_quotes():
repo, py = "/home/ubuntu/papers", "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
print("Quote check: src/show.py not available here; skipped.")
return
try:
full = subprocess.run([py, "src/show.py", "PMC12251097", "--full"], cwd=repo,
capture_output=True, text=True, timeout=300).stdout
except Exception as e:
print(f"Quote check: could not load archived text ({e}); skipped.")
return
norm = lambda s: " ".join(s.split())
full_n = norm(full)
print("Quote check against the PMC-archived full text (occurrences):")
for key, (loc, q) in QUOTES.items():
print(f" {key:17s} {full_n.count(norm(q))} ({loc})")
# ------------------------------------------------------------------ Fig 2A pixel data
# Panel: (stimulus, cell, gene) -> y-axis ticks [(pixel y, printed value)], and per time point:
# (time, WT top-edge pixel range, KO top-edge pixel range, printed significance label)
PANELS = [
("A. baumannii", "MSF", "Cxcl9", [(150.5, .18), (217.5, .12), (285.0, .06), (352.5, 0)],
[("3 h", (296, 296), (320, 320), "***"), ("6 h", (195, 195), (297, 297), "***")]),
("A. baumannii", "MSF", "Cxcl10", [(153.5, 4), (203.5, 3), (253.5, 2), (303.5, 1), (353.5, 0)],
[("3 h", (295, 303), (319, 319), "***"), ("6 h", (190, 190), (277, 277), "***")]),
("A. baumannii", "BMEC", "Cxcl9", [(152.5, .18), (220.5, .12), (287.5, .06), (355.0, 0)],
[("3 h", (262, 262), (321.5, 324), "**"), ("6 h", (205, 205), (302.5, 304), "***")]),
("A. baumannii", "BMEC", "Cxcl10", [(150.5, 1.5), (218.5, 1.0), (285.5, .5), (353.0, 0)],
[("3 h", (271, 271), (307, 307), "*"), ("6 h", (177, 177), (292, 292), "***")]),
("LPS", "MSF", "Cxcl9", [(435.5, .20), (486.0, .15), (536.5, .10), (587.0, .05), (637.5, 0)],
[("6 h", (553, 553), (586.5, 593), "**"), ("12 h", (492, 492), (594, 594), "**")]),
("LPS", "MSF", "Cxcl10", [(435.5, 4), (485.5, 3), (535.5, 2), (585.5, 1), (635.0, 0)],
[("6 h", (498, 498), (567, 567), "**"), ("12 h", (564, 564), (614, 614), "**")]),
("LPS", "BMEC", "Cxcl9", [(434.0, .24), (501.5, .16), (569.0, .08), (636.5, 0)],
[("6 h", (562, 562), (595, 596), "**"), ("12 h", (480, 480), (564, 564), "***")]),
("LPS", "BMEC", "Cxcl10", [(434.5, 1.8), (501.5, 1.2), (568.5, .6), (636.0, 0)],
[("6 h", (548.5, 558), (595.5, 596), "**"), ("12 h", (485, 485), (568, 568), "**")]),
("IFN-β", "MSF", "Cxcl9", [(738.5, .04), (789.0, .03), (839.5, .02), (890.0, .01), (940.5, 0)],
[("3 h", (843, 844), (826, 826), "ns"), ("6 h", (782, 785), (774, 774), "ns")]),
("IFN-β", "MSF", "Cxcl10", [(739.5, 2), (790.5, 1.5), (840.5, 1), (891.5, .5), (941.5, 0)],
[("3 h", (763.5, 767), (779, 779), "ns"), ("6 h", (894, 895), (878, 882), "ns")]),
("IFN-β", "BMEC", "Cxcl9", [(739.5, .06), (806.5, .04), (874.5, .02), (941.5, 0)],
[("3 h", (865, 865), (853, 861), "ns"), ("6 h", (764.5, 769), (777, 777), "ns")]),
("IFN-β", "BMEC", "Cxcl10", [(734.5, 2.4), (803.5, 1.6), (871.5, .8), (939.5, 0)],
[("3 h", (878, 878), (879, 879), "ns"), ("6 h", (767, 767), (782, 782), "ns")]),
]
PAD = 1.0 # px added on each side of every top-edge range (line width / anti-aliasing)
def calib(ticks):
"""Least-squares line value = a + b * pixel_y through the printed tick marks."""
ys = [t[0] for t in ticks]; vs = [t[1] for t in ticks]
my, mv = statistics.fmean(ys), statistics.fmean(vs)
b = sum((y - my) * (v - mv) for y, v in zip(ys, vs)) / sum((y - my) ** 2 for y in ys)
a = mv - b * my
resid = max(abs(a + b * y - v) for y, v in zip(ys, vs))
return a, b, resid
def measure():
out, worst = [], 0.0
for stim, cell, gene, ticks, bars in PANELS:
a, b, resid = calib(ticks)
worst = max(worst, resid / max(t[1] for t in ticks))
val = lambda y: a + b * y # b < 0: smaller pixel y = taller bar
for t, (w0, w1), (k0, k1), lab in bars:
wt_mid, ko_mid = val((w0 + w1) / 2), val((k0 + k1) / 2)
wt_hi, wt_lo = val(w0 - PAD), val(w1 + PAD)
ko_hi, ko_lo = val(k0 - PAD), val(k1 + PAD)
out.append(dict(stim=stim, cell=cell, gene=gene, time=t, label=lab,
wt=wt_mid, ko=ko_mid, r=ko_mid / wt_mid,
r_lo=ko_lo / wt_hi, r_hi=ko_hi / wt_lo))
print(f"\nAxis calibration: largest tick residual = {worst:.2%} of axis maximum")
return out
def summarise(rows, stim):
sel = [d for d in rows if d["stim"] == stim]
print(f"\n{stim}: {len(sel)} Cxcl9/Cxcl10 comparisons in Fig 2A (Rnf213-/- / WT)")
for d in sel:
print(f" {d['cell']:4s} {d['gene']:6s} {d['time']:>4s} WT {d['wt']:.3g} KO {d['ko']:.3g} "
f"ratio {d['r']:.2f} [{d['r_lo']:.2f}-{d['r_hi']:.2f}] printed: {d['label']}")
rs = [d["r"] for d in sel]
gm = statistics.geometric_mean(rs)
n_ns = sum(d["label"] == "ns" for d in sel)
n_above = sum(r > 1 for r in rs)
n_cover1 = sum(d["r_lo"] <= 1 <= d["r_hi"] for d in sel)
print(f" ratio range {min(rs):.2f}-{max(rs):.2f}; geometric mean {gm:.2f}; "
f"printed 'ns' {n_ns}/{len(sel)}; KO above WT in {n_above}/{len(sel)}; "
f"measurement interval includes 1 in {n_cover1}/{len(sel)}")
return sel, gm
# ------------------------------------------------------------------ style
BG, TXT, TXT2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": TXT2, "ytick.color": TXT2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
print(f"wrote {stem}.svg and {stem}.png")
# ------------------------------------------------------------------ figure 1
def fig1(n_ns, n_tot):
fig = plt.figure(figsize=(7.2, 4.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 7.2); ax.set_ylim(0, 4.6); ax.axis("off")
def panel(x, w, title, sub, body, accent):
ax.add_patch(FancyBboxPatch((x, 0.92), w, 3.55, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=accent, lw=1.5))
ax.text(x + 0.14, 4.33, title, fontsize=11.5, fontweight="bold", va="top", color=TXT)
ax.text(x + 0.14, 4.03, sub, fontsize=10, va="top", color=accent)
yy = 3.68
for line, style in body:
ax.text(x + 0.14, yy, line, fontsize=10, va="top",
color={"n": TXT, "b": accent, "s": TXT2}[style],
fontweight="bold" if style == "b" else "normal",
fontstyle="italic" if style == "s" else "normal")
yy -= 0.235
left = [("“RNF213 deficiency impairs type I", "n"),
("interferon (IFN-I) production and", "n"),
("decreases the expression of", "b"),
("interferon-stimulated genes (ISGs)", "b"),
("in response to IFN-β stimulation", "b"),
("and Acinetobacter baumannii", "n"),
("infection.”", "n"),
("", "n"), ("", "n"), ("", "n"),
("States: ISGs lower in Rnf213–/–", "s"),
("cells after IFN-β.", "s")]
right = [("Fig 2A paragraph:", "s"),
("“…Cxcl9 and Cxcl10 induced by IFN-β", "n"),
("treatment were comparable", "b"),
("between WT and Rnf213–/– MSFs …”", "b"),
("“RNF213 targets TRAF3”:", "s"),
("“…inhibited TBK1 phosphorylation and", "n"),
("IFN-I production … but not in", "b"),
("response to IFN-β treatment …”", "b"),
("", "n"),
("Fig 2A, IFN-β row (MSFs and BMECs):", "s"),
(f"{n_ns} of {n_tot} Cxcl9/Cxcl10 comparisons", "n"),
("printed “ns”.", "n")]
panel(0.15, 3.05, "Abstract", "reported claim", left, ORANGE)
panel(4.0, 3.05, "Results and Fig 2A", "the paper's own text and labels", right, BLUE)
ax.add_patch(FancyArrowPatch((3.24, 3.0), (3.96, 3.0), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT))
ax.text(3.6, 3.3, "conflict", ha="center", fontsize=10, fontweight="bold", color=TXT)
ax.text(3.6, 2.73, "IFN-β\nclause\nonly", ha="center", va="top", fontsize=10, color=TXT2)
ax.text(0.15, 0.47,
"After A. baumannii infection or LPS, abstract and Results agree: Fig 2A shows lower Cxcl9/\n"
"Cxcl10 in Rnf213–/– cells (16 of 16 comparisons marked *, ** or ***). The conflict is the\n"
"IFN-β condition, the only one in which Rnf213–/– cells were given IFN-β and ISGs measured.",
fontsize=10, color=TXT2, va="center")
save(fig, "fig1")
# ------------------------------------------------------------------ figure 2
def fig2(groups):
rows = []
for gname, sel, gm in groups:
rows.append(("header", gname, gm))
rows += [("pt", d, None) for d in sel]
n = len(rows)
fig, ax = plt.subplots(figsize=(7.2, 8.4))
fig.subplots_adjust(left=0.25, right=0.85, top=0.90, bottom=0.09)
ax.set_xscale("log")
ax.set_xlim(0.15, 2.0)
ax.set_ylim(n - 0.4, -0.8)
ax.axvline(1.0, color=TXT2, lw=0.8, zorder=1)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ticks = [0.2, 0.3, 0.5, 0.7, 1.0, 1.5, 2.0]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(TXT2)
ax.tick_params(axis="y", length=0)
ylabels, first = [], True
for i, (kind, d, gm) in enumerate(rows):
if kind == "header":
ylabels.append("")
ax.text(-0.02, i, d, transform=ax.get_yaxis_transform(), ha="right", va="center",
fontsize=10.5, fontweight="bold", color=TXT)
ax.plot([gm], [i], marker="D", ms=9, mfc="white", mec=BLUE, mew=2, ls="none", zorder=3,
label="geometric mean of the 8 ratios (recomputed)" if d == "IFN-β" else None)
ax.text(gm * 1.07, i, f"{gm:.2f}", va="center", fontsize=10, color=BLUE)
else:
ylabels.append(f"{d['cell']} {d['gene']} {d['time']}")
ax.plot([d["r_lo"], d["r_hi"]], [i, i], color=BLUE, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([d["r"]], [i], marker="o", ms=8, color=BLUE, ls="none", zorder=3,
label="Rnf213–/– ÷ WT, bar means measured on Fig 2A\n(line: measurement range; recomputed)"
if first else None)
first = False
ax.text(1.03, i, d["label"], transform=ax.get_yaxis_transform(), va="center", ha="left",
fontsize=10.5, fontweight="bold", color=ORANGE)
ax.set_yticks(range(n)); ax.set_yticklabels(ylabels, fontsize=10, color=TXT)
ax.text(1.03, -0.75, "label printed\nin Fig 2A\n(reported)", transform=ax.get_yaxis_transform(),
va="bottom", ha="left", fontsize=10, color=ORANGE)
ax.set_xlabel("Rnf213–/– ÷ WT, mean Cxcl9 or Cxcl10 mRNA (gene/Hprt), log scale\n"
"← lower in Rnf213–/– 1 = no difference higher in Rnf213–/– →",
labelpad=6, color=TXT)
fig.legend(loc="upper left", bbox_to_anchor=(0.02, 0.997), ncol=1, frameon=False, fontsize=10,
handletextpad=0.4)
save(fig, "fig2")
if __name__ == "__main__":
check_quotes()
rows = measure()
groups = []
for stim in ("IFN-β", "A. baumannii", "LPS"):
sel, gm = summarise(rows, stim)
groups.append((stim, sel, gm))
infl = [d for d in rows if d["stim"] != "IFN-β"]
rs = [d["r"] for d in infl]
print(f"\nA. baumannii + LPS together: ratio range {min(rs):.2f}-{max(rs):.2f}; geometric mean "
f"{statistics.geometric_mean(rs):.2f}; largest upper bound {max(d['r_hi'] for d in infl):.2f}; "
f"printed *, ** or ***: {sum(d['label'] != 'ns' for d in infl)}/{len(infl)}")
ifnb = groups[0][1]
print(f"IFN-β: smallest lower bound {min(d['r_lo'] for d in ifnb):.2f}, "
f"largest upper bound {max(d['r_hi'] for d in ifnb):.2f}")
fig1(sum(d["label"] == "ns" for d in ifnb), len(ifnb))
fig2(groups)
Abstract reports 84% of ST2621 isolates as clinical; the Results count gives 98 of 164 (60%)
PLoS Pathog (2026) · doi:10.1371/journal.ppat.1013859 · PMC12810914
Summary
The abstract says that most isolates of the dominant clone ST2621 came from adult patients (59%) and from clinical samples (84%). The Results section, describing the same 164 ST2621 isolates, gives 95 adult (58%) and 98 clinical (60%) isolates, and the isolate-level S1 Table reproduces those counts exactly. An 84% clinical share would need 137 or 138 clinical ST2621 isolates. That is incompatible with the Results count, and also with the paper's own totals: Table 1 lists 186 clinical isolates in the whole collection, which leaves room for at most 135 clinical ST2621 isolates. The direction of the finding (most ST2621 isolates were clinical) is unchanged, but the abstract overstates the clinical share by 24 percentage points in the sentence that describes the clone.
What the paper reports
All quotes are from the article as archived in PubMed Central and were checked verbatim.
Abstract:
Sequence type (ST) 2621 (164, 40%) and ST39 (164, 40%) predominated with several minor STs making up the remainder. The majority of ST2621 associated with blaOXA-181 were isolated from adult patients (59%) and clinical samples (84%).
Results, first MLST paragraph:
ST2621 and ST39 were overwhelmingly dominant in the collection (S1 Table), represented by 164 isolates (39.6%) each.
Results, "Characteristics of ST2621":
blaOXA-181 was the predominant carbapenemase gene associated with this ST, being detected in all isolates, with one isolate having an additional blaNDM–1 gene.
The majority of the ST2621 isolates were from adult patients (95, 58%) and from clinical samples (98, 60%).
Results, carbapenemase genes (counts used for the upper bound below):
The predominant genes were blaOXA-181 (194, 46.9%) and blaNDM-1 (191, 46.2%).
Interestingly, most isolates with blaNDM-1 were from carriage samples (146, 72.6%), whereas those with blaOXA-181 were mostly clinical isolates (125, 64.4%) (Fig 1C).
Results, ST48 paragraph:
All ST48 isolates were blaOXA-181 producers, except for the first ST48 isolate from 2019, which produced blaOXA-48 (Fig 2). This ST was predominantly in adult patients (16, 84.2%) and from clinical samples (15, 79.0%).
Results, "Characteristics of ST39":
The majority of the ST39 isolates were obtained from neonatal (92, 56.1%) and paediatric (64, 39.0%) patients and mostly in carriage samples (128, 78.0%).
Table 1, "Source" rows (total isolates n = 778 | stored isolates n = 413):
Clinical | 357 (45.9) | 186 (45.0)
Carriage | 421 (54.1) | 227 (55.0)
Discussion:
In contrast, ST2621 was more associated with adult patients and clinical samples. Given that limited CRE colonization screening was undertaken in the adult wards, this almost certainly presents an underestimation of the true spread, and the propensity to cause more disease cannot be quantified sufficiently.
What the paper's own numbers show
Method. I divided the Results counts for ST2621 by the stated clone size (164) and found which integer counts out of 164 round to the abstract's percentages. I then bounded the number of clinical ST2621 isolates in two ways that use only counts printed in the text: (A) the 186 clinical isolates in Table 1, minus the clinical isolates of ST39 (164 − 128 carriage = 36) and ST48 (15); and (B) the 125 clinical blaOXA-181 isolates, since the Results say all ST2621 isolates carry blaOXA-181, minus at least 14 clinical blaOXA-181 isolates of ST48 (15 clinical ST48 isolates, of which at most one is the single blaOXA-48 isolate). Finally, I recounted the isolate-level rows of S1 Table (supplementary file s006, sheet "metadata_PRJEB63349"), stripping trailing spaces from six cells. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| ST2621 from clinical samples | Abstract | 84% | 98/164 = 59.8%; 84% of 164 needs 137 or 138 isolates |
| ST2621 from clinical samples | Results, "Characteristics of ST2621" | 98, 60% | 98/164 = 59.8% (consistent) |
| ST2621 from adult patients | Abstract | 59% | 95/164 = 57.9%; 59% needs 96 or 97 isolates |
| ST2621 from adult patients | Results, "Characteristics of ST2621" | 95, 58% | 95/164 = 57.9% (consistent) |
| Ceiling A on clinical ST2621 | Table 1; ST39 and ST48 paragraphs | 186 clinical in all | 186 − 36 − 15 = 135, so at most 135/164 = 82.3%; 137 + 36 + 15 = 188 and 138 + 36 + 15 = 189, both above 186 |
| Ceiling B on clinical ST2621 | blaOXA-181 and ST48 paragraphs | 125 clinical blaOXA-181 isolates | at most 125/164 = 76.2%; after subtracting ST48, at most 111/164 = 67.7% |
| ST2621 counts in S1 Table | S1 Table, isolate rows | — | n = 164: 95 adult (57.9%), 54 paediatric, 15 neonate; 98 clinical (59.8%), 66 screening (40.2%) |
| ST2621 with OXA-181 in the class D column | S1 Table | — | n = 147: 89 adult (60.5%), 92 clinical (62.6%) |
| Clinical share among ST2621 adults | S1 Table cross-tabulation | — | 78/95 = 82.1% (74/89 = 83.1% in the OXA-181 subset) |
| Adult share among ST2621 clinical isolates | S1 Table cross-tabulation | — | 78/98 = 79.6% |
| ST48 adults | Results, ST48 paragraph | 16, 84.2% | 16/19 = 84.2% (the same rounded value as the abstract's 84%) |
| ST39 carriage, neonatal, paediatric | Abstract vs Results, "Characteristics of ST39" | 76%, 57%, 38% vs 128 (78.0%), 92 (56.1%), 64 (39.0%) | 128/164 = 78.0%, 92/164 = 56.1%, 64/164 = 39.0% |
The Results percentages agree with their own counts and with the supplement. The abstract's 84% does not match any ST2621 quantity in the paper or the supplement, and it is also larger than the most clinical ST2621 isolates the paper's totals can hold (ceiling A, 82.3%). Ceiling A uses only the Table 1 total and the per-ST counts in the text, not the gene data. S1 Table has 187 clinical rows in the whole collection rather than 186. Using 187 in ceiling A gives 136/164 = 82.9%, still below 84%.
Figure
Figure 1. The two conflicting sentences, quoted verbatim. Left: the abstract's percentages, as printed (orange). Right: the percentages computed from the counts printed in the Results sentence (blue): 98 of 164 clinical (59.8%) and 95 of 164 adult (57.9%). The clinical shares differ by 24.2 percentage points. An 84% share would need 137 or 138 clinical isolates, while the Results give 98 and the Table 1 totals leave room for at most 135.
Figure 2. Share of the 164 ST2621 isolates that came from clinical samples, on a 0–100% axis. The orange dot is the abstract value (84%; dashed reference line). The blue dot is the Results count (98/164 = 59.8%). The blue bars show the largest values the paper's other counts permit: 135/164 = 82.3% from the Table 1 total of 186 clinical isolates minus the clinical isolates of ST39 (36) and ST48 (15), and 111/164 = 67.7% from the 125 clinical blaOXA-181 isolates, if, as the Results state, every ST2621 isolate carries blaOXA-181, minus at least 14 clinical blaOXA-181 isolates of ST48. The abstract value lies above both ceilings.
Why it matters
- It changes the description of the clone in the abstract. The abstract sentence is the paper's only summary of who carries ST2621. At 84% clinical, about 1 in 6 ST2621 isolates would have come from carriage screening. The paper's counts give 66 of 164, about 4 in 10. The clone's sample mix is closer to an even split than the abstract suggests.
- It sharpens a contrast the Discussion draws. The Discussion says ST2621 "was more associated with adult patients and clinical samples" than ST39, and raises a possible "propensity to cause more disease". The direction still holds with the correct figure: 60% of ST2621 isolates were clinical, against 36/164 = 22.0% of ST39 isolates. The gap, though, is smaller than the abstract implies (about 38 rather than about 62 percentage points). The paper itself notes that adult wards had limited screening, which would also raise the clinical share of an adult-associated clone.
- The main conclusions are not affected. The paper's headline findings stand without this figure: ST2621 and ST39 dominate the collection, they are distinct from the international lineages, and carbapenemase genes were probably acquired locally. No statistical test depends on it. The adult share (59% in the abstract vs 57.9% computed) differs only slightly.
- Readers are likely to quote the abstract value. The 84% appears only in the abstract, which is the part most readers, indexers and citing papers take numbers from.
Explanations considered
- A different population ("ST2621 associated with blaOXA-181"). The abstract's qualifier could mean a subset rather than all 164 isolates. The Results say blaOXA-181 was "detected in all isolates", so the subset would be all 164 and the share 59.8%. The supplement lists fewer: the S1 metadata sheet has OXA-181 in the class D column for 147 ST2621 isolates, and its "ARG summary" sheet gives 143 (87.2%). In the 147-isolate subset the clinical share is 92/147 = 62.6%. More generally, no subset of 143 or more ST2621 isolates can exceed 98/143 = 68.5% clinical. To reach 84%, a subset would have to drop at least 48 of the 66 screening isolates while keeping nearly every clinical one. This does not explain the number. (The difference between "all isolates" in the text and 143 or 147 in the supplement is a separate, smaller inconsistency. It does not affect the clinical-share comparison.)
- A different denominator or a conditional share. The nearest conditional shares in S1 Table are clinical among ST2621 adults, 78/95 = 82.1%, and clinical among adults in the OXA-181 subset, 74/89 = 83.1%. Both round to 82% or 83%, not 84%, and neither matches the sentence, which gives the adult and clinical shares side by side as fractions of the same group. Adults among clinical isolates is 78/98 = 79.6%. None of these gives 84%.
- Excluding the isolate with an additional blaNDM-1. Dropping one isolate gives 97/163 = 59.5% or 98/163 = 60.1%. It does not explain the number.
- Rounding or a typesetting slip. Rounding cannot turn 59.8% into 84%. Rounding could explain 58% vs 59% only if the adult count were 96 or 97, but the Results and the supplement both give 95. The 59% remains a small unexplained difference.
- Version or conversion differences. The publisher's HTML, the publisher's PDF, the PMC record and the Europe PMC full-text XML all carry the same abstract and Results wording, so this is not an artifact of conversion to PMC.
- A value from a neighbouring paragraph. The ST48 paragraph gives 16/19 = 84.2% adults, the same rounded value. The paper gives no information on how the abstract figure was derived, so whether the two are related cannot be determined. The correction below does not depend on it.
- Minor ST39 figures. The abstract's ST39 percentages (76% carriage, 57% neonatal, 38% paediatric) also differ by 1–2 points from the Results (78.0%, 56.1%, 39.0%). S1 Table has 165 ST39 rows against 164 in the text, because two rows share one laboratory number and one sequencing accession. These are small discrepancies and do not bear on the ST2621 figure.
Status
As of 2026-10-24:
- Version of record. The wording is the same in the publisher's article page (journals.plos.org, published 16 January 2026), the publisher's 18-page PDF, the PMC record and the Europe PMC full-text XML.
- Supplement. S1 Table (ppat.1013859.s006.xlsx, downloaded from the PLOS supplementary link) was recounted at isolate level. It supports the Results figures (95 adult, 98 clinical of 164), not the abstract. S1–S5 Figs (phylogenies and plasmid figures) were not used, because they do not give counts that bear on this question.
- Corrections. No correction, erratum, expression of concern or retraction was found. Crossref has no update records (empty
update-to,updated-byandrelation). PMC and Europe PMC list no notices (src/known.py). The PLOS article page shows no notice and no reader comments. - PubPeer. Web searches found no PubPeer thread or other public discussion. The PubPeer API and the Europe PMC search endpoint returned errors during these checks, so those two sources could not be queried directly.
Suggested correction
In the abstract, replace
The majority of ST2621 associated with blaOXA-181 were isolated from adult patients (59%) and clinical samples (84%).
with wording consistent with the Results and S1 Table, for example:
The majority of ST2621 isolates, which carried blaOXA-181, were from adult patients (95/164, 58%) and clinical samples (98/164, 60%).
The authors may also wish to:
- reconcile "detected in all isolates" for blaOXA-181 in ST2621 with the 147 (metadata sheet) and 143 (ARG summary sheet) isolates listed in S1 Table;
- align the abstract's ST39 percentages (76%, 57%, 38%) with the Results (78.0%, 56.1%, 39.0%);
- remove the duplicated ST39 row from S1 Table.
Recomputation script (python recompute.py.txt)
"""Recompute the ST2621 sample-source and population shares of PMC12810914
(PLoS Pathog 2026, doi:10.1371/journal.ppat.1013859, "Carbapenem resistant
Klebsiella pneumoniae isolates at a tertiary hospital in Cape Town, South Africa,
are dominated by specific local clones rather than previously described
international lineages").
Run: python recompute.py.txt [path/to/ppat.1013859.s006.xlsx]
Needs matplotlib; openpyxl for the optional S1 Table part. Prints every
recomputed number used in writeup.md and writes fig1.svg/png and fig2.svg/png
next to this file. The figures use only numbers printed in the article text.
If no path is given, the script tries to download S1 Table (supplementary file
s006) from the publisher into a temporary folder. If that fails, the S1 Table
part is skipped and the text-only part still runs.
Inputs copied from the article as archived in PMC:
Abstract: ST2621 (164, 40%) and ST39 (164, 40%); "The majority of ST2621 associated
with blaOXA-181 were isolated from adult patients (59%) and clinical samples (84%)";
ST39 "carriage isolates (76%) from neonatal (57%) and paediatric (38%) patients".
Results, MLST: 413 isolates; ST2621 and ST39 "represented by 164 isolates (39.6%) each";
ST48 (19, 4.6%).
Results, ST48: "predominantly in adult patients (16, 84.2%) and from clinical samples
(15, 79.0%)"; all ST48 isolates blaOXA-181 except the first, which had blaOXA-48.
Results, carbapenemase genes: blaOXA-181 in 194 isolates, of which 125 (64.4%) clinical.
Results, Characteristics of ST39: neonatal (92, 56.1%), paediatric (64, 39.0%),
carriage (128, 78.0%).
Results, Characteristics of ST2621: blaOXA-181 "detected in all isolates";
"adult patients (95, 58%) and ... clinical samples (98, 60%)".
Table 1, stored isolates (n = 413): Clinical 186 (45.0), Carriage 227 (55.0).
"""
import collections
import math
import os
import sys
import tempfile
import urllib.request
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (article text)
N_TOTAL = 413
N_2621 = 164
ABS_ADULT_PCT, ABS_CLIN_PCT = 59, 84 # Abstract
RES_ADULT_N, RES_ADULT_PCT = 95, 58 # Results, Characteristics of ST2621
RES_CLIN_N, RES_CLIN_PCT = 98, 60
N_39, ST39_CARRIAGE_N, ST39_NEO_N, ST39_PAED_N = 164, 128, 92, 64
ABS_ST39 = dict(carriage=76, neonatal=57, paediatric=38)
N_48, ST48_ADULT_N, ST48_CLIN_N = 19, 16, 15
OXA181_N, OXA181_CLIN_N = 194, 125
TABLE1_CLINICAL, TABLE1_CARRIAGE = 186, 227
S1_URL = ("https://journals.plos.org/plospathogens/article/file?"
"type=supplementary&id=10.1371/journal.ppat.1013859.s006")
def pct(a, b):
return 100.0 * a / b
def counts_rounding_to(p, n):
"""Integer counts k with round-half-up(100k/n) == p."""
lo, hi = (p - 0.5) * n / 100, (p + 0.5) * n / 100
return [k for k in range(math.ceil(lo), math.floor(hi) + 1) if 100 * k / n < p + 0.5]
print("== 1. Text-only recomputation ==")
print(f"ST2621 share of collection: {N_2621}/{N_TOTAL} = {pct(N_2621, N_TOTAL):.1f}%")
print(f"Results adult: {RES_ADULT_N}/{N_2621} = {pct(RES_ADULT_N, N_2621):.2f}% (printed {RES_ADULT_PCT}%)")
print(f"Results clinical: {RES_CLIN_N}/{N_2621} = {pct(RES_CLIN_N, N_2621):.2f}% (printed {RES_CLIN_PCT}%)")
print(f"Results screening (164 - 98): {N_2621 - RES_CLIN_N}/{N_2621} = {pct(N_2621 - RES_CLIN_N, N_2621):.1f}%")
ka = counts_rounding_to(ABS_ADULT_PCT, N_2621)
kc = counts_rounding_to(ABS_CLIN_PCT, N_2621)
print(f"Abstract {ABS_ADULT_PCT}% adult of 164 needs {ka} isolates "
f"({', '.join(f'{k}/164 = {pct(k, N_2621):.1f}%' for k in ka)})")
print(f"Abstract {ABS_CLIN_PCT}% clinical of 164 needs {kc} isolates "
f"({', '.join(f'{k}/164 = {pct(k, N_2621):.1f}%' for k in kc)})")
print(f"Abstract-minus-Results gap, clinical: {ABS_CLIN_PCT} - {pct(RES_CLIN_N, N_2621):.1f} = "
f"{ABS_CLIN_PCT - pct(RES_CLIN_N, N_2621):.1f} percentage points")
print(f"Abstract screening implied: {100 - ABS_CLIN_PCT}% (about 1 in {100 / (100 - ABS_CLIN_PCT):.1f}); "
f"Results screening {pct(N_2621 - RES_CLIN_N, N_2621):.1f}%")
# Ceiling 1: Table 1 total clinical isolates minus clinical isolates of ST39 and ST48
st39_clin = N_39 - ST39_CARRIAGE_N
ceil1 = TABLE1_CLINICAL - st39_clin - ST48_CLIN_N
print(f"\nCeiling A (Table 1): {TABLE1_CLINICAL} clinical stored isolates - ST39 clinical "
f"({N_39} - {ST39_CARRIAGE_N} = {st39_clin}) - ST48 clinical {ST48_CLIN_N} = {ceil1} "
f"-> ST2621 clinical <= {ceil1}/164 = {pct(ceil1, N_2621):.1f}%")
print(f" check: {max(kc)} + {st39_clin} + {ST48_CLIN_N} = {max(kc) + st39_clin + ST48_CLIN_N}; "
f"{min(kc)} + {st39_clin} + {ST48_CLIN_N} = {min(kc) + st39_clin + ST48_CLIN_N} vs {TABLE1_CLINICAL}")
print(f" with Results count: {RES_CLIN_N} + {st39_clin} + {ST48_CLIN_N} = "
f"{RES_CLIN_N + st39_clin + ST48_CLIN_N} <= {TABLE1_CLINICAL}")
# Ceiling 2: if all ST2621 carry blaOXA-181 (Results text), their clinical isolates are among
# the 125 clinical blaOXA-181 isolates; ST48 contributes at least 15 - 1 = 14 of those.
ceil2a = OXA181_CLIN_N
ceil2b = OXA181_CLIN_N - (ST48_CLIN_N - 1)
print(f"Ceiling B (blaOXA-181, if all ST2621 carry it): <= {ceil2a}/164 = {pct(ceil2a, N_2621):.1f}%; "
f"minus at least {ST48_CLIN_N - 1} clinical blaOXA-181 ST48 isolates: <= {ceil2b}/164 = "
f"{pct(ceil2b, N_2621):.1f}%")
print(f"blaOXA-181 clinical share as printed: {OXA181_CLIN_N}/{OXA181_N} = {pct(OXA181_CLIN_N, OXA181_N):.1f}%")
print(f"Table 1 clinical share: {TABLE1_CLINICAL}/{N_TOTAL} = {pct(TABLE1_CLINICAL, N_TOTAL):.1f}%; "
f"{TABLE1_CLINICAL} + {TABLE1_CARRIAGE} = {TABLE1_CLINICAL + TABLE1_CARRIAGE}")
# A subset of the ST2621 isolates (e.g. 'associated with blaOXA-181') reaching 84% clinical
max_screen = math.floor(RES_CLIN_N * 0.16 / 0.84 + 1e-9)
print(f"\nSubset needed for >= 83.5% clinical: at most {max_screen} of the 66 screening isolates kept "
f"if all 98 clinical are kept (so at least {N_2621 - RES_CLIN_N - max_screen} dropped); "
f"subset size <= {RES_CLIN_N + max_screen}")
for m in (143, 147, 163):
print(f" any subset of {m} ST2621 isolates: clinical share <= {RES_CLIN_N}/{m} = {pct(RES_CLIN_N, m):.1f}%")
print(f" dropping the one blaNDM-1 co-carrier: 97/163 = {pct(97, 163):.1f}% or 98/163 = {pct(98, 163):.1f}%")
print("\nOther figures in the same paragraphs")
print(f"ST48 adult {ST48_ADULT_N}/{N_48} = {pct(ST48_ADULT_N, N_48):.1f}%; "
f"ST48 clinical {ST48_CLIN_N}/{N_48} = {pct(ST48_CLIN_N, N_48):.1f}%")
print(f"ST39 carriage {ST39_CARRIAGE_N}/{N_39} = {pct(ST39_CARRIAGE_N, N_39):.1f}% (abstract {ABS_ST39['carriage']}%); "
f"neonatal {ST39_NEO_N}/{N_39} = {pct(ST39_NEO_N, N_39):.1f}% (abstract {ABS_ST39['neonatal']}%); "
f"paediatric {ST39_PAED_N}/{N_39} = {pct(ST39_PAED_N, N_39):.1f}% (abstract {ABS_ST39['paediatric']}%)")
print(f"ST39 clinical {st39_clin}/{N_39} = {pct(st39_clin, N_39):.1f}%")
print(f"ST2621 minus ST39 clinical share: Results {pct(RES_CLIN_N, N_2621) - pct(st39_clin, N_39):.1f} pp; "
f"abstract {ABS_CLIN_PCT - pct(st39_clin, N_39):.1f} pp")
print(f"Ceiling A with the S1 Table total of 187 clinical rows: 187 - {st39_clin} - {ST48_CLIN_N} = "
f"{187 - st39_clin - ST48_CLIN_N} -> {pct(187 - st39_clin - ST48_CLIN_N, N_2621):.1f}%")
# ------------------------------------------------------------------ optional: S1 Table
xlsx = sys.argv[1] if len(sys.argv) > 1 else None
if xlsx is None:
try:
tmpdir = tempfile.mkdtemp(prefix="s1table_")
xlsx = os.path.join(tmpdir, "ppat.1013859.s006.xlsx")
req = urllib.request.Request(S1_URL, headers={"User-Agent": "Mozilla/5.0"})
with urllib.request.urlopen(req, timeout=60) as r, open(xlsx, "wb") as f:
f.write(r.read())
except Exception as e: # noqa: BLE001
print(f"\n[S1 Table download failed: {e}; skipping supplement part]")
xlsx = None
if xlsx:
try:
import openpyxl
wb = openpyxl.load_workbook(xlsx, read_only=True)
except Exception as e: # noqa: BLE001
print(f"\n[S1 Table could not be opened: {e}; skipping supplement part]")
wb = None
if wb is not None:
print("\n== 2. S1 Table (supplementary file s006), sheet", wb.sheetnames[0], "==")
all_rows = list(wb.worksheets[0].iter_rows(values_only=True))
header = [str(h).strip() if h else "" for h in all_rows[1]]
col = {h: i for i, h in enumerate(header)}
rows = [r for r in all_rows[2:] if r[0]]
s = lambda v: str(v).strip() if v is not None else "" # noqa: E731 (trailing spaces exist)
ST, POP, SRC, CLD, CLB = (col["ST"], col["population"], col["source"],
col["class D carbapenemase"], col["class B carbapenemases"])
print(f"isolate rows: {len(rows)}")
trailing = sum(1 for r in rows if r[POP] != s(r[POP]) or r[SRC] != s(r[SRC]))
print(f"rows with a trailing space in population or source: {trailing}")
print("all isolates, source:", dict(collections.Counter(s(r[SRC]) for r in rows)))
print("all isolates, population:", dict(collections.Counter(s(r[POP]) for r in rows)))
print("ST counts (top 3):", collections.Counter(s(r[ST]) for r in rows).most_common(3))
def describe(label, sub):
n = len(sub)
pop = collections.Counter(s(r[POP]) for r in sub)
src = collections.Counter(s(r[SRC]) for r in sub)
print(f"{label}: n={n}; adult {pop['Adult']}/{n} = {pct(pop['Adult'], n):.1f}%; "
f"clinical {src['Clinical']}/{n} = {pct(src['Clinical'], n):.1f}%; "
f"population {dict(pop)}; source {dict(src)}")
return pop, src
st2621 = [r for r in rows if s(r[ST]) == "2621"]
describe("ST2621", st2621)
x = collections.Counter((s(r[POP]), s(r[SRC])) for r in st2621)
print("ST2621 population x source:", dict(x))
ad = sum(v for (p, _), v in x.items() if p == "Adult")
print(f" clinical among ST2621 adults: {x[('Adult', 'Clinical')]}/{ad} = "
f"{pct(x[('Adult', 'Clinical')], ad):.1f}%")
print(f" adults among ST2621 clinical: {x[('Adult', 'Clinical')]}/{RES_CLIN_N} = "
f"{pct(x[('Adult', 'Clinical')], RES_CLIN_N):.1f}%")
print("ST2621 class D carbapenemase column:", dict(collections.Counter(s(r[CLD]) for r in st2621)))
print("ST2621 class B carbapenemase column:", dict(collections.Counter(s(r[CLB]) for r in st2621)))
ox = [r for r in st2621 if s(r[CLD]) == "OXA-181"]
pop, _ = describe("ST2621 with OXA-181 in class D column", ox)
xo = collections.Counter((s(r[POP]), s(r[SRC])) for r in ox)
print(f" clinical among adults in that subset: {xo[('Adult', 'Clinical')]}/{pop['Adult']} = "
f"{pct(xo[('Adult', 'Clinical')], pop['Adult']):.1f}%")
describe("ST39", [r for r in rows if s(r[ST]) == "39"])
describe("ST48", [r for r in rows if s(r[ST]) == "48"])
# ARG summary sheet, blaOXA-181 row
for r in wb.worksheets[1].iter_rows(values_only=True):
if r[1] and str(r[1]).strip() == "blaOXA-181":
print(f"ARG summary sheet, blaOXA-181 row: ST2621 column = {r[3]}")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5,
"svg.fonttype": "none", "figure.facecolor": BG, "axes.facecolor": BG})
res_clin_pct = pct(RES_CLIN_N, N_2621)
res_adult_pct = pct(RES_ADULT_N, N_2621)
# Figure 1: the two conflicting excerpts
fig = plt.figure(figsize=(7.2, 4.35))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2)
ax.set_ylim(0, 4.35)
ax.axis("off")
def panel(x0, title, lines, items, colour, tag):
w, h, y0 = 3.15, 3.05, 0.95
ax.add_patch(FancyBboxPatch((x0, y0), w, h, boxstyle="round,pad=0,rounding_size=0.08",
fc="white", ec="#c9c7c1", lw=1.0))
ax.text(x0 + 0.15, y0 + h - 0.22, title, fontsize=10.5 if len(title) > 20 else 11.5, weight="bold", color=INK, va="top")
for i, ln in enumerate(lines):
ax.text(x0 + 0.15, y0 + h - 0.62 - 0.235 * i, ln, fontsize=10, color=INK2, va="top",
style="italic")
for i, (lab, val) in enumerate(items):
yy = y0 + 0.95 - 0.5 * i
ax.text(x0 + 0.15, yy, lab, fontsize=10.5, color=INK, va="center")
ax.text(x0 + w - 0.15, yy, val, fontsize=13, weight="bold", color=colour, va="center",
ha="right")
ax.text(x0 + w - 0.15, y0 + 1.38, tag, fontsize=10, color=colour, ha="right", va="center")
return y0 + 0.95 # y of the "clinical samples" row
yc = panel(0.2, "Abstract",
["“The majority of ST2621 associated", "with blaOXA-181 were isolated from",
"adult patients (59%) and clinical", "samples (84%).”"],
[("clinical samples", f"{ABS_CLIN_PCT}%"), ("adult patients", f"{ABS_ADULT_PCT}%")],
ORANGE, "reported in abstract")
panel(3.85, "Results: ‘Characteristics of ST2621’",
["“The majority of the ST2621 isolates", "were from adult patients (95, 58%)",
"and from clinical samples (98, 60%).”"],
[("clinical: 98 of 164", f"{res_clin_pct:.1f}%"), ("adult: 95 of 164", f"{res_adult_pct:.1f}%")],
BLUE, "recomputed from the Results counts")
ax.add_patch(FancyArrowPatch((3.38, yc), (3.82, yc), arrowstyle="<->", mutation_scale=12,
lw=2, color=INK))
ax.text(3.6, 0.6, f"Same clone, same 164 isolates: clinical share differs by "
f"{ABS_CLIN_PCT - res_clin_pct:.1f} percentage points.", ha="center", fontsize=10.5,
color=INK, weight="bold")
ax.text(3.6, 0.38, f"84% of 164 would need {min(kc)}–{max(kc)} clinical isolates. The Results count "
f"is {RES_CLIN_N},\nand the Table 1 totals leave room for at most {ceil1}.", va="top",
ha="center", fontsize=10, color=INK2)
fig.savefig(os.path.join(HERE, "fig1.svg"))
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200)
plt.close(fig)
# Figure 2: clinical share of ST2621, reported vs. what the paper's counts allow
rows2 = [
("Abstract", ABS_CLIN_PCT, ORANGE, "o", f"{ABS_CLIN_PCT}% (reported)"),
("Results count, 98 of 164", res_clin_pct, BLUE, "o", f"{res_clin_pct:.1f}% (recomputed)"),
(f"Ceiling from Table 1:\n{TABLE1_CLINICAL} clinical − {st39_clin} ST39 − {ST48_CLIN_N} ST48",
pct(ceil1, N_2621), BLUE, "|", f"≤ {pct(ceil1, N_2621):.1f}% ({ceil1} of 164)"),
(f"Ceiling if all ST2621 carry\nblaOXA-181: {OXA181_CLIN_N} − {ST48_CLIN_N - 1} ST48",
pct(ceil2b, N_2621), BLUE, "|", f"≤ {pct(ceil2b, N_2621):.1f}% ({ceil2b} of 164)"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.3))
fig.subplots_adjust(left=0.36, right=0.97, top=0.86, bottom=0.17)
ys = list(range(len(rows2)))[::-1]
for y, (lab, v, c, m, txt) in zip(ys, rows2):
if m == "|":
ax.plot([0, v], [y, y], color=c, lw=2, solid_capstyle="butt")
ax.plot([v], [y], marker="|", markersize=14, mew=2.5, color=c)
ax.text(v - 1.2, y + 0.2, txt, ha="right", va="bottom", fontsize=10, color=INK)
else:
ax.plot([v], [y], marker="o", markersize=10, color=c, mec=BG, mew=1.5, zorder=3)
ax.text(v + 2.0 if v < 80 else v - 2.0, y, txt, ha="left" if v < 80 else "right",
va="center", fontsize=10, color=INK)
ax.axvline(ABS_CLIN_PCT, color=ORANGE, lw=1, ls=(0, (3, 3)), zorder=1)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows2], fontsize=10, color=INK)
ax.set_xlim(0, 100)
ax.set_ylim(-0.6, len(rows2) - 0.4)
ax.set_xticks(range(0, 101, 20))
ax.set_xticklabels([f"{t}%" for t in range(0, 101, 20)], color=INK2)
ax.set_xlabel("ST2621 isolates from clinical samples (share of 164)", color=INK2)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color("#b5b3ad")
ax.tick_params(axis="y", length=0)
ax.tick_params(axis="x", color="#b5b3ad", labelcolor=INK2)
fig.text(0.02, 0.95, "Orange: value printed in the abstract. Blue: values computed from the paper's own counts.",
fontsize=10, color=INK2, va="center")
fig.savefig(os.path.join(HERE, "fig2.svg"))
fig.savefig(os.path.join(HERE, "fig2.png"), dpi=200)
plt.close(fig)
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
The abstract reports increased heat sensitivity, but the paper's own data show increased heat resistance
PLoS Pathog (2018) · doi:10.1371/journal.ppat.1007004 · PMC5927469
Summary
The abstract states that loss of the outer spore protein CsA "increased spore sensitivity to heat, sodium hydroxide and hydrochloric acid," grouping heat with the two chemicals. For heat this reverses the direction of the paper's own result. The Results report that the csA-mutant spores had a 15% increase in colony-forming efficiency at 75 °C (p = 0.0286), and, by the paper's stated definition of spore viability, a higher post-treatment colony-forming efficiency means more spores survive — that is, greater resistance, i.e. less sensitivity. The Discussion agrees, stating the mutant spores "were more resistant to heat." The two chemical treatments are decreases in colony-forming efficiency (−51% and −29%), so "increased sensitivity" is correct for them; only the heat term is reversed. The error is confined to a secondary characterization phenotype and the body of the paper carries the correct direction, but the reversed claim sits in the abstract and is echoed in the Discussion's closing summary.
What the paper reports
The absence of CsA reduced sporulation levels and increased spore sensitivity to heat, sodium hydroxide and hydrochloric acid.
— Abstract
Spores of the csA mutant showed an 15% increase in colony forming efficiency when incubated at 75°C (Fig 4A), a 51% decrease in colony forming efficiency when incubated with hydrochloric acid (Fig 4B) and a 29% decrease in colony forming efficiency when incubated with sodium hydroxide (Fig 4C) when compared to wild-type spores (p = 0.0286).
— Results, "The csA and csB mutations do not affect the colony forming efficiency"
Although csA mutant spores had greater sensitivity to hydrochloric acid, they were more resistant to heat and did not show an increased sensitivity to ethanol in comparison to wild-type spores (Fig 4A and 4B, S5C Fig).
— Discussion
CsA is required for wild-type levels of spore resistance to heat, sodium hydroxide and hydrochloric acid, while CsB is required for normal levels of spore adherence to cervical cells.
— Discussion (closing summary; "resistance to heat" here implies the mutant was less resistant, again reversed)
Percentage spore viability (the ability of spores to form colonies) following treatment was calculated as follows: (log10 CFU ml-1 after plating spores on HIS agar post treatment/log10 CFU ml-1 after plating untreated spores on HIS agar) X 100.
— Materials and methods, "Spore resistance" (the paper's own outcome definition)
What the paper's own numbers show
Method. This is a sign/direction error, not an arithmetic slip, so there is no hidden quantity to recompute from raw data (no numerical CFU values for Fig 4 are deposited). The check is a mechanical mapping from the paper's stated outcome definition to the implied direction of each effect: by "percentage spore viability = (log₁₀ CFU after treatment ÷ log₁₀ CFU untreated) × 100," a higher post-treatment colony-forming efficiency (CFE) means more spores survive the treatment = more resistant = less sensitive, and a lower CFE means the opposite. Applying this to the three reported CFE changes gives the direction each should carry, which is then compared with the abstract. Separately, the single reported p-value is checked against the stated n = 4 per group design. Every value below is printed by recompute.py.txt.
| Quantity | Location | Reported | Direction implied by the paper's own definition |
|---|---|---|---|
| csA mutant heat (75 °C), CFE vs wild-type | Results; Fig 4A | +15% (increase) | more survival → more resistant = LESS sensitive — opposite to the abstract's "increased sensitivity to heat" |
| csA mutant hydrochloric acid, CFE vs wild-type | Results; Fig 4B | −51% (decrease) | less survival → more sensitive — abstract agrees (no contradiction) |
| csA mutant sodium hydroxide, CFE vs wild-type | Results; Fig 4C | −29% (decrease) | less survival → more sensitive — abstract agrees (no contradiction) |
| Heat direction, Discussion body | Discussion | "more resistant to heat" | = LESS sensitive; consistent with the Results, both disagree with the abstract |
| Reported p-value | Results; Fig 4 legend ("average of four independent experiments"); Methods (Mann-Whitney) | p = 0.0286 | 2 ÷ C(8,4) = 2/70 = 0.0286, the smallest exact two-sided rank-test p for n = 4 vs 4; internally consistent and not the error |
The body of the paper (Results, Fig 4 legend's neutral "altered ... resistance," and the Discussion sentence "they were more resistant to heat") is internally consistent on the direction of all three treatments. The abstract sentence is the single outlier, and the Discussion's closing one-line summary repeats it by listing heat among the resistances for which "CsA is required."
Figure
Figure 1. The direction conflict for heat. Left (orange): the abstract groups heat with sodium hydroxide and hydrochloric acid as treatments to which loss of CsA raised sensitivity. Right (blue): the Results report a +15% colony-forming efficiency at 75 °C and the Discussion calls the mutant spores "more resistant to heat." By the paper's own viability definition a higher post-treatment colony-forming efficiency is more survival, hence greater resistance and less sensitivity, so the two descriptions of heat point in opposite directions. The lower strip applies the same mapping to each treatment: the abstract's "increased sensitivity" matches the data for the two chemicals (both are efficiency decreases) but contradicts it for heat (an efficiency increase).
Figure 2. The three reported effects on one axis. The csA-mutant-versus-wild-type change in colony-forming efficiency is plotted from zero (no change). Heat alone falls on the positive side (more survival = less sensitive); the two chemicals fall on the negative side (less survival = more sensitive). The abstract's blanket "increased spore sensitivity" is correct for the two chemicals but has the wrong sign for heat.
Why it matters
Spore resistance to heat and chemicals is one of the characterization assays supporting the paper's claim that CsA contributes to spore integrity. The reversed heat term appears in the abstract — a high-exposure, headline location — and is repeated in the Discussion's closing summary, so a reader who takes the abstract at face value would conclude that CsA normally protects spores against heat, the opposite of what the data show (the mutant was more heat-resistant). The magnitude (15%) and the statistical significance are reported correctly in the Results; only the direction word is wrong, and only for heat. The paper's headline conclusions — that the outer spore proteins maintain spore structure, and that CsB loss affects gastrointestinal colonization and clearance — do not depend on the heat direction and are untouched. This is therefore a wrong secondary result rather than a wrong key result or conclusion.
Explanations considered
- A different definition of the measured quantity. The Materials and methods define percentage viability explicitly as post-treatment over untreated colony-forming ability. Under that definition a higher value is unambiguously more survival, so a +15% change is less sensitivity. No alternative reading of the stated definition makes +15% into increased sensitivity. Does not rescue the abstract.
- The log-ratio endpoint behaves non-monotonically. The endpoint is a monotone increasing function of post-treatment CFU for fixed untreated CFU, so a reported increase still means more survivors. Does not rescue the abstract.
- "Increased sensitivity" refers to the mutation process, not the phenotype. The Discussion states plainly "they were more resistant to heat," fixing the intended phenotype direction and matching the Results. The abstract and the Discussion's closing summary are the outliers. Does not rescue the abstract.
- A statistical artifact (the +15% is noise). The paper reports the heat comparison as significant at its own threshold (p = 0.0286, the minimum attainable for 4 vs 4), and the Discussion treats the heat result as real ("more resistant to heat"). The dispute is the sign of the reported, significant effect, not its reality. Does not rescue the abstract.
- A conversion or typesetting artifact in the archived text. The same wording is present in the version of record (see Status), so this is not an archiving artifact.
Status
- Version of record. The contradiction is present verbatim in the version of record: the abstract, Results and Discussion quotes above were confirmed in the PMC archive (
src/show.py PMC5927469). The independent re-derivation supplied with this case also confirmed the identical wording against the PLOS Pathogens HTML article and the Europe PMC full-text XML. - Supplements. Fig 4 (the heat/HCl/NaOH resistance figure) is supplied only as the rendered figure; no underlying numerical CFU values are deposited for it, and the related S5 figure is an image. There is no source-data file that could resolve the heat value independently; none is needed, since the dispute is a direction stated in the text.
- Corrections and comments. As of 2026-10-23, no correction, erratum, expression of concern or retraction is on record.
src/known.pyand the Crossref record show only a self-referential "new version" relation dated 2018-04-30 (the publication itself). Europe PMC returns the single article record (PMID 29668758) withcommentCorrectionList: None. PubPeer could not be queried in this session (the API and search page returned Cloudflare/HTTP 403 blocks); no PubPeer thread is known. - Sources. PMC5927469 (local archive via
src/show.py); Crossref and PMC/Europe PMC notice check viasrc/known.pyand the Europe PMC REST API.
Suggested correction
Correct the heat term in the abstract so that it is not grouped with the two chemicals — for example, "loss of CsA reduced sporulation levels, increased spore sensitivity to sodium hydroxide and hydrochloric acid, and altered (increased) resistance to heat" — and revise the Discussion's closing summary accordingly so that it no longer implies the mutant was less heat-resistant. The Results text, the Fig 4 legend and the Discussion sentence "they were more resistant to heat" already carry the correct direction and need no change.
Recomputation script (python recompute.py.txt)
"""Recompute / verify the heat-sensitivity direction in PMC5927469
(PLoS Pathog 2018, doi:10.1371/journal.ppat.1007004).
Run: python recompute.py.txt
Needs matplotlib (and the standard library). Prints every number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
The dispute is a SIGN/DIRECTION error, not an arithmetic slip, so the "recompute"
is (a) a mechanical mapping from the paper's own outcome definition to the implied
direction of each effect, and (b) an exact check that the single reported p-value is
consistent with the stated n = 4 per group design. All inputs are copied from the
paper as archived in PubMed Central (src/show.py PMC5927469).
Paper's own outcome definition (Materials and methods, "Spore resistance"):
percentage spore viability after a treatment =
(log10 CFU/ml after treatment / log10 CFU/ml untreated) x 100
-> a HIGHER colony-forming efficiency (CFE) after a treatment means MORE spores
survive it => MORE resistant => LESS sensitive.
-> a LOWER CFE after a treatment means fewer survive => LESS resistant
=> MORE sensitive.
"""
import os
from math import comb
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (from the paper)
# csA mutant vs wild-type, % change in colony-forming efficiency after each treatment
# (Results, "The csA and csB mutations do not affect the colony forming efficiency"; Fig 4)
REPORTED_CFE_CHANGE = { # treatment : % change in CFE for csA mutant vs WT
"heat (75 C)": +15,
"hydrochloric acid": -51,
"sodium hydroxide": -29,
}
# What the abstract says about each treatment. The abstract groups all three together:
# "increased spore sensitivity to heat, sodium hydroxide and hydrochloric acid"
ABSTRACT_SAYS = {t: "MORE sensitive" for t in REPORTED_CFE_CHANGE}
# What the Discussion body says about each treatment (Discussion #p4).
DISCUSSION_SAYS = {
"heat (75 C)": "MORE resistant / LESS sensitive", # "they were more resistant to heat"
"hydrochloric acid": "MORE sensitive", # "greater sensitivity to hydrochloric acid"
"sodium hydroxide": "MORE sensitive", # "increased sensitivity to sodium hydroxide"
}
REPORTED_P = 0.0286 # Results, single p-value for the csA vs WT comparisons
N_PER_GROUP = 4 # Fig 4 legend: "the average of four independent experiments";
# Methods: Mann-Whitney test
def cfe_change_to_sensitivity(pct_change):
"""Map a csA-vs-WT % change in post-treatment CFE to a sensitivity direction,
using the paper's own definition of percentage spore viability."""
if pct_change > 0:
return "LESS sensitive (= MORE resistant)"
if pct_change < 0:
return "MORE sensitive (= LESS resistant)"
return "no change"
# ---------------------------------------------------------------- 1. direction check
print("=== 1. Direction implied by the paper's own outcome definition, per treatment ===")
print(f"{'treatment':20s} {'reported CFE':>12s} {'implied direction':34s} "
f"{'abstract says':16s} {'discussion says':26s} abstract vs data")
print("-" * 118)
contradictions = []
for t, chg in REPORTED_CFE_CHANGE.items():
implied = cfe_change_to_sensitivity(chg)
implied_more = implied.startswith("MORE sensitive")
abstract_more = ABSTRACT_SAYS[t].startswith("MORE")
agree = (implied_more == abstract_more)
if not agree:
contradictions.append(t)
verdict = "agree" if agree else "CONTRADICTION"
print(f"{t:20s} {chg:+11d}% {implied:34s} {ABSTRACT_SAYS[t]:16s} "
f"{DISCUSSION_SAYS[t]:26s} {verdict}")
print()
print(f"Treatments where the abstract contradicts the paper's own CFE data: {contradictions}")
print(f"Results (+15% CFE at 75 C) implies: {cfe_change_to_sensitivity(+15)}")
print(f"Discussion body says for heat: {DISCUSSION_SAYS['heat (75 C)']}")
print("Results and Discussion agree with each other on heat: True")
print("Abstract agrees with them on heat: False "
"(abstract groups heat under 'increased sensitivity')")
print("Abstract agrees on the two chemicals (HCl, NaOH): True "
"(both are CFE decreases = increased sensitivity)")
# ---------------------------------------------------------------- 2. p-value sanity check
print("\n=== 2. Is the single reported p = 0.0286 consistent with n = 4 per group? ===")
total = comb(2 * N_PER_GROUP, N_PER_GROUP) # C(8,4)
smallest_two_sided = 2.0 / total
print(f"n1 = n2 = {N_PER_GROUP}: number of ways to split 8 ranks = C(8,4) = {total}")
print(f"smallest attainable exact two-sided Mann-Whitney p = 2/{total} = {smallest_two_sided:.4f}")
print(f"reported p = {REPORTED_P} matches 2/{total} = {smallest_two_sided:.4f}: "
f"{abs(smallest_two_sided - REPORTED_P) < 5e-4}")
print("=> the p-value is exactly the minimum a 4-vs-4 rank test can give; it is internally "
"consistent and is NOT the error.")
# ---------------------------------------------------------------- figures
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
GREEN_OK, PANEL = "#3a8a4f", "#f4f3ef"
import matplotlib # noqa: E402
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"))
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200)
plt.close(fig)
# ---- Figure 1: two-panel abstract-vs-body contradiction diagram -----------------
fig, ax = plt.subplots(figsize=(7.2, 5.6))
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def panel(x, y, w, h, title, title_col, body, foot, foot_col):
box = FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.6,rounding_size=2.5",
linewidth=1.6, edgecolor=title_col, facecolor=PANEL, zorder=1)
ax.add_patch(box)
ax.text(x + w / 2, y + h - 3.2, title, ha="center", va="top",
fontsize=12, fontweight="bold", color=title_col)
ax.text(x + w / 2, y + h - 11, body, ha="center", va="top",
fontsize=9.3, color=TXT, linespacing=1.32)
ax.text(x + w / 2, y + 3.5, foot, ha="center", va="bottom",
fontsize=9.1, color=foot_col, fontweight="bold", linespacing=1.25)
panel(3, 60, 43, 37, "Abstract", ORANGE,
'"The absence of CsA reduced\n'
'sporulation levels and increased\n'
'spore sensitivity to heat, sodium\n'
'hydroxide and hydrochloric acid."',
"→ heat grouped with the chemicals\nas MORE sensitive", ORANGE)
panel(54, 60, 43, 37, "Results + Discussion", BLUE,
'Results / Fig 4A: csA mutant had a\n'
'"15% increase in colony forming\n'
'efficiency when incubated at 75 C"\n'
'Discussion: "they were more\n'
'resistant to heat"',
"→ heat: MORE resistant\n= LESS sensitive", BLUE)
# connector naming the conflict
arr = FancyArrowPatch((46, 78), (54, 78), arrowstyle="<|-|>", mutation_scale=18,
linewidth=2, color="#b03020", zorder=3)
ax.add_patch(arr)
ax.text(50, 73, "heat\ndirection\nreversed", ha="center", va="top",
fontsize=9.5, fontweight="bold", color="#b03020", linespacing=1.15)
# bottom strip: the three treatments and whether abstract matches the data
ax.text(50, 52, "By the paper’s own definition "
"(viability = log₁₀ CFU after ÷ log₁₀ CFU untreated × 100), a higher\n"
"post-treatment CFE means more survival = more resistant = less sensitive:",
ha="center", va="top", fontsize=9.0, color=SEC, linespacing=1.3)
cells = [
("heat (75 C)", "+15% CFE", "less sensitive", "“more sensitive”", False),
("hydrochloric acid", "−51% CFE", "more sensitive", "“more sensitive”", True),
("sodium hydroxide", "−29% CFE", "more sensitive", "“more sensitive”", True),
]
cx = [17, 50, 83]
for i, (t, cfe, implied, says, ok) in enumerate(cells):
col = GREEN_OK if ok else "#b03020"
mark = "✓ abstract matches" if ok else "✗ abstract contradicts"
ax.text(cx[i], 38, t, ha="center", fontsize=9.6, fontweight="bold", color=TXT)
ax.text(cx[i], 31, f"data: {cfe} → {implied}", ha="center", fontsize=8.7, color=BLUE)
ax.text(cx[i], 25.5, f"abstract: {says}", ha="center", fontsize=8.7, color=ORANGE)
ax.text(cx[i], 18, mark, ha="center", fontsize=9.2, fontweight="bold", color=col)
legend = [Line2D([0], [0], color=ORANGE, lw=6, label="paper’s abstract claim"),
Line2D([0], [0], color=BLUE, lw=6, label="direction from the paper’s own numbers")]
ax.legend(handles=legend, loc="lower center", bbox_to_anchor=(0.5, 0.0),
ncol=2, frameon=False, fontsize=9, handlelength=1.4)
fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.04)
save(fig, "fig1")
# ---- Figure 2: per-treatment CFE change, heat as the sign outlier ---------------
fig, ax = plt.subplots(figsize=(7.2, 3.8))
treatments = list(REPORTED_CFE_CHANGE.keys())
vals = [REPORTED_CFE_CHANGE[t] for t in treatments]
ypos = list(range(len(treatments)))[::-1]
# shade the two half-planes and label what each side means
ax.axvspan(-80, 0, color="#f3e6df", zorder=0)
ax.axvspan(0, 60, color="#e3edf6", zorder=0)
ax.text(-40, 2.62, "CFE decrease → fewer survive\n= MORE sensitive",
ha="center", va="center", fontsize=8.6, color="#9a3412")
ax.text(30, 2.62, "CFE increase → more survive\n= LESS sensitive",
ha="center", va="center", fontsize=8.6, color="#1e4e79")
ax.axvline(0, color=SEC, lw=1.4, zorder=2)
for y, t, v in zip(ypos, treatments, vals):
ax.plot([0, v], [y, y], color="#b7b6b1", lw=2, zorder=1)
ax.plot(v, y, "o", ms=13, color=ORANGE, zorder=3)
ax.text(v + (4 if v > 0 else -4), y + 0.17, f"{v:+d}%", va="center",
ha="left" if v > 0 else "right", fontsize=11, fontweight="bold", color=TXT)
agree = (v < 0) # abstract says "increased sensitivity" for all three
if agree:
ax.text(v - 4, y - 0.26, "✓ abstract agrees", va="center", ha="right",
fontsize=8.6, fontweight="bold", color=GREEN_OK)
else:
ax.text(58, y - 0.26, "✗ abstract says “more sensitive” — contradicts",
va="center", ha="right", fontsize=8.6, fontweight="bold", color="#b03020")
ax.set_yticks(ypos)
ax.set_yticklabels(treatments, fontsize=10.5)
ax.set_ylim(-0.6, 3.1)
ax.set_xlim(-80, 60)
ax.set_xlabel("csA mutant vs wild-type: % change in colony-forming efficiency after treatment "
"(Results, Fig 4)", fontsize=9.3)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.set_title("Only heat is a CFE increase; the abstract groups all three as "
"“increased sensitivity”", fontsize=10, color=TXT, pad=8)
fig.tight_layout()
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png to", HERE)
Abstract says ALT was lower in COVID-19 patients than controls; Table 1 shows it higher
PLoS Pathog (2021) · doi:10.1371/journal.ppat.1009420 · PMC8026060
Summary
This retrospective study compares 681 COVID-19 patients with 369 normal controls, split by sex and age. The second sentence of the Abstract says that, compared with age-matched controls, non-comorbid patients in all four age and sex groups had lower lymphocyte counts and lower alanine aminotransferase (ALT). The paper's own Table 1 shows that median ALT was higher in patients than in their matched controls in all four groups: by 10.0 and 2.0 IU/L in young males and young females (P < 0.0001 and P = 0.0008), and by 8.0 and 1.8 IU/L in old males and old females (not significant). The share of people with ALT above the upper normal limit was also higher among patients in every group. The Results text and the Discussion agree with the table, so the error is confined to one Abstract sentence. That sentence reverses the direction of a liver-function finding and conflicts with the paper's own conclusion about liver damage.
What the paper reports
Abstract, second sentence:
Compared to the age-matched NCs, both young and old male and female non-comorbid COVID-19 patients had lower lymphocyte counts and alanine aminotransferase (ALT) concentration, and only young male and female patients had lower neutrophil counts.
Results, "Laboratory findings of male and female patients with COVID-19", first paragraph:
Compared to the young and old normal controls (NCs), both age-matched male and female COVID-19 patients had more cases with leukocyte and lymphocyte counts below the normal lower limit (NLL) and ALT concentrations above the normal upper limit (NUL).
Discussion, first paragraph:
In this retrospective study, we found that SARS-CoV-2 infection decreases blood neutrophil and lymphocyte counts in both male and female patients and damages liver function.
Results, "Baseline characteristics" (the population of the Table 1 patient columns):
Of 681 patients, 438 (223 males and 215 females) suffered from COVID-19 without comorbidities. Of these 438 patients, 285 were young (65.0%, 146 males and 139 females) and 153 were old (35.0%, 77 males and 76 females).
Table 1, ALT rows. Columns are normal controls (young males, young females, old males, old females), then patients in the same order, then P-values e to l:
ALT, IU/L | M: 5–40 F: 5–35 | 17.0 (13.0–25.0) | 11.0 (9.0–14.0)aaaa | 21.0(17.8–26.0) | 17.0(14.0–21.8) | 27.0 (17.1–42.0) | 13.0 (11.0–21.0)cccc | 29.0 (21.0–50.5) | 18.8 (14.0–27.5)dd | >0.999 | <0.0001 | >0.999 | 0.011 | <0.0001 | 0.308 | 0.0008 | >0.999
Male >40 IU/L; Female >35 IU/L | | 12/160 (7.5) | 2/103 (1.9) | 2/50(4.0) | 2/56(3.6) | 37/143 (25.9) | 11/137 (8.0)cccc | 26/76 (34.2) | 12/76 (15.8)dd | 0.588 | 0.923 | 0.195 | 0.080 | <0.0001 | <0.0001 | 0.039 | 0.024
Table 1 footer, which defines the matched patient-versus-control P values:
P-valuei for comparison of the young normal males vs. young male patients with COVID-19.P-valuej for comparison of the old normal males vs. old male patients with COVID-19.P-valuek for comparison of the young normal females vs. young female patients with COVID-19.P-valuel for comparison of the old normal females vs. old female patients with COVID-19.
What the paper's own numbers show
Method. Table 1 prints median ALT (IQR) for each of the eight columns, and the footer says which P value belongs to each patient-versus-control comparison. For each age and sex group I subtracted the control median from the patient median and read off the paper's matched P value (i, j, k or l). For the row giving ALT above the upper normal limit, I recomputed the two proportions and tested each 2×2 table with Fisher's exact test, Yates-corrected χ² and uncorrected χ². The footer says the paper used one of these, "as appropriate". As controls, I made the same comparison for the lymphocyte and neutrophil halves of the Abstract sentence, and I checked that the Table 1 patients are the non-comorbid population the sentence describes. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Direction of patient vs control ALT | Abstract, sentence 2 | lower in all four groups | higher in 4 of 4 groups, lower in 0 of 4 |
| Median ALT, young males | Table 1, ALT row; P-value i | 17.0 vs 27.0 IU/L; P < 0.0001 | patients +10.0 IU/L (ratio 1.59), significant |
| Median ALT, young females | Table 1, ALT row; P-value k | 11.0 vs 13.0 IU/L; P = 0.0008 | patients +2.0 IU/L (ratio 1.18), significant |
| Median ALT, old males | Table 1, ALT row; P-value j | 21.0 vs 29.0 IU/L; P = 0.308 | patients +8.0 IU/L (ratio 1.38), not significant |
| Median ALT, old females | Table 1, ALT row; P-value l | 17.0 vs 18.8 IU/L; P > 0.999 | patients +1.8 IU/L (ratio 1.11), not significant |
| ALT above upper normal limit, young males | Table 1, ">40 / >35 IU/L" row | 12/160 (7.5) vs 37/143 (25.9); P < 0.0001 | +18.4 percentage points; Fisher P = 1.7×10⁻⁵ |
| ALT above upper normal limit, young females | same row | 2/103 (1.9) vs 11/137 (8.0); P = 0.039 | +6.1 points; uncorrected χ² P = 0.039 (matches); Fisher P = 0.046 |
| ALT above upper normal limit, old males | same row | 2/50 (4.0) vs 26/76 (34.2); P < 0.0001 | +30.2 points; Fisher P = 4.5×10⁻⁵ |
| ALT above upper normal limit, old females | same row | 2/56 (3.6) vs 12/76 (15.8); P = 0.024 | +12.2 points; uncorrected χ² P = 0.024 (matches); Fisher P = 0.042 |
| Lymphocyte count, same sentence (control) | Table 1, lymphocyte row | 1.9 / 1.8 / 1.9 / 1.9 vs 1.3 / 1.3 / 1.2 / 1.2 ×10⁹/L; all P < 0.0001 | lower in 4 of 4 groups: consistent with the Abstract |
| Neutrophil count, same sentence (control) | Table 1, neutrophil row | lower only in young groups (P 0.009 and < 0.0001); old groups P > 0.9999 | consistent with "only young male and female patients had lower neutrophil counts" |
| Table 1 patient population | Results, Baseline characteristics; Table 1 header | 146 + 139 + 77 + 76 non-comorbid patients | 438 = 681 − 243: the non-comorbid population the Abstract sentence names |
Every patient group has a higher median ALT than its matched control group, and a larger share of patients than controls has ALT above the upper normal limit. The increase in medians is statistically significant in both young groups. It is not significant in either old group, but it points the same way. No group shows lower ALT. The other two claims in the same Abstract sentence, on lymphocytes and neutrophils, match Table 1, so the comparison method and the column reading reproduce the paper wherever the Abstract is right.
One detail of the table can mislead a reader. The "cccc" mark on the young-female patient median (13.0) means a difference from young male patients. The young-female patient-versus-control P value is P-value k = 0.0008. Either way, the comparison is significant and shows patients higher.
Figure
Figure 1. The left panel is the Abstract sentence as printed, which gives the direction of the patient-versus-control ALT comparison as lower (orange). The right panel holds the Table 1 values for the same comparison. Each row gives the matched control and patient medians in IU/L, the patient-minus-control difference recomputed from them (blue) and the paper's own P value for that comparison (footnotes i, k, j and l). Below the rows are the reported shares with ALT above the upper normal limit, control then patient, in the same group order. In all four groups the patient value exceeds the control value, which is the opposite of the direction the Abstract states.
Figure 2. For each group, the patient-minus-control difference in median ALT recomputed from Table 1 (blue dots), with zero as the reference line. The shaded region left of zero is where the Abstract's statement places every group (patients lower). The column on the right gives the paper's matched P values. All four differences are positive, and the two young-group differences are statistically significant by the paper's own tests.
Why it matters
The Abstract is the part of a paper that most readers see, and it is what indexing services and secondary summaries carry. Taken at face value, the sentence says that SARS-CoV-2 infection goes with lower ALT, which would mean less hepatocellular injury in patients than in healthy controls. The paper's data show the reverse: higher median ALT, significant in the young groups, and a share of patients 3.4 to 8.6 times that of controls of patients with ALT above the upper normal limit (25.9% vs 7.5%, 8.0% vs 1.9%, 34.2% vs 4.0% and 15.8% vs 3.6%). The sentence also contradicts the paper's own conclusions. The Discussion says SARS-CoV-2 infection "damages liver function", and the Abstract's last sentences describe associations with "liver damage". A reader who compares the Abstract with those conclusions meets a contradiction that the body of the paper resolves in favour of higher ALT.
The paper's main conclusions are not affected. The sex and age effects on ALT rest on comparisons among patients and on the Table 2 regressions (ALT on female sex β = −0.281, P < 0.0001; on age β = 0.114, P = 0.002), not on the control comparison. The lymphocyte and neutrophil statements in the same sentence are supported by Table 1.
Explanations considered
- Misreading of the column order. The table header and the footer both fix the order: controls young male, young female, old male, old female, then patients in the same order. The footer gives an explicit P value (i, j, k, l) for each matched pair. With that pairing, all four differences are positive. The same reading reproduces the lymphocyte and neutrophil halves of the sentence exactly, so the columns are being read correctly.
- A different population. The Abstract sentence refers to "non-comorbid COVID-19 patients". The Table 1 patient columns sum to 146 + 139 + 77 + 76 = 438, the non-comorbid count given in the Results (681 − 243). The ALT denominators (143, 137, 76, 76) are within those groups, so some patients lacked an ALT value, but the population is the same. Supplementary S1 Table covers all 681 patients and has no control columns, so it cannot be the source of a different patient-versus-control direction.
- A different ALT measure. Table 1 has two ALT rows, the median concentration and the share above the upper normal limit. Both show patients higher in all four groups. There is no row for ALT below a lower limit. The Results do mention ALT "below the NLL", but only when comparing female with male patients, not patients with controls.
- "Lower" applying only to lymphocytes. As printed, "lower" governs both "lymphocyte counts and alanine aminotransferase (ALT) concentration". Even read as loosely as possible, the sentence gives no direction for ALT that the table supports, and the Results sentence on the same comparison says patients had more cases above the upper normal limit.
- Only the non-significant groups. The Abstract says "both young and old". In the two old groups the median difference is not significant, but it is positive. In the two young groups it is significant and positive. No subset of groups shows lower ALT.
- Conversion or version artifact. The PMC text, the Europe PMC full-text XML, the publisher's HTML (Version 2) and the publisher's PDF carry the same Abstract sentence and the same Table 1 ALT rows. The first online version was labelled an uncorrected proof, and the current version shows no correction notice.
- Choice of test for the above-limit proportions. The reported P values for young and old females (0.039, 0.024) match uncorrected χ². Fisher's exact test gives 0.046 and 0.042, so whichever test the paper used, the conclusion is the same. This affects only the strength of those two comparisons, not their direction.
None of these explanations accounts for the Abstract's direction. All of the paper's own numbers on this comparison point the other way.
Status
- Version of record. The publisher's article page (journals.plos.org, reached via doi.org) shows Version 2, with Version 1 labelled "Uncorrected Proof", published March 26, 2021. Its Abstract carries the sentence quoted above. The publisher PDF (16 pages, created 2021-04-01) and the Europe PMC full-text XML carry the same sentence, and their Table 1 ALT rows match the PMC table cell for cell. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. There are six: S1 to S6 Tables. S1 Table (sex- and age-specific characteristics of COVID-19 patients on admission, DOCX) covers only the 681 patients and has no normal-control columns. S2 to S6 Tables are the supporting data for Fig 1A to 1E. None bears on the patient-versus-control ALT direction.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns only a Crossref "new_version" self-relation dated 2021-04-07, which fits the move from uncorrected proof to final version, and no PMC or Europe PMC notices. The publisher page shows no correction notice and no reader comment count. PubPeer's search API refused automated access (HTTP 403), so PubPeer could not be queried directly. A web search for the title together with "PubPeer" found no thread for this article. - Sources. PMC article PMC8026060; Europe PMC full-text XML for PMC8026060; publisher HTML and PDF at journals.plos.org (doi:10.1371/journal.ppat.1009420); S1 Table (DOCX); Crossref; Europe PMC; PubPeer (not reachable).
Suggested correction
- Revise the Abstract's second sentence so that the ALT direction matches Table 1, for example: "Compared to the age-matched NCs, both young and old male and female non-comorbid COVID-19 patients had lower lymphocyte counts and more frequent ALT concentrations above the normal upper limit, with significantly higher median ALT in young male and female patients, and only young male and female patients had lower neutrophil counts."
- Optionally, state in the Table 1 footer which test produced each P value in the ">40 / >35 IU/L" row. The two female values match uncorrected χ² rather than Yates-corrected χ² or Fisher's exact test.
Recomputation script (python recompute.py.txt)
"""Recompute the patient-versus-control ALT comparison of PMC8026060
(PLoS Pathog 2021, doi:10.1371/journal.ppat.1009420,
"Sex- and age-specific clinical and immunological features of coronavirus disease 2019").
Run: python recompute.py.txt
Needs scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from Table 1 of the paper as archived in PMC (identical in the publisher PDF).
Column order in Table 1: normal controls (NC) young M, young F, old M, old F; then non-comorbid
COVID-19 patients in the same order. Footnotes: P-value i = young NC males vs young male patients,
j = old males, k = young females, l = old females.
ALT, IU/L (median, IQR):
NC 17.0 (13.0-25.0) | 11.0 (9.0-14.0) | 21.0 (17.8-26.0) | 17.0 (14.0-21.8)
Pt 27.0 (17.1-42.0) | 13.0 (11.0-21.0) | 29.0 (21.0-50.5) | 18.8 (14.0-27.5)
P-values e..l: >0.999 | <0.0001 | >0.999 | 0.011 | <0.0001 | 0.308 | 0.0008 | >0.999
ALT above upper normal limit (M >40, F >35 IU/L), n/N (%):
NC 12/160 (7.5) | 2/103 (1.9) | 2/50 (4.0) | 2/56 (3.6)
Pt 37/143 (25.9) | 11/137 (8.0) | 26/76 (34.2) | 12/76 (15.8)
P-values e..l: 0.588 | 0.923 | 0.195 | 0.080 | <0.0001 | <0.0001 | 0.039 | 0.024
Lymphocyte count, x10^9/L (median): NC 1.9 | 1.8 | 1.9 | 1.9 ; Pt 1.3 | 1.3 | 1.2 | 1.2 ; P i..l all <0.0001
Neutrophil count, x10^9/L (median): NC 3.5 | 3.5 | 3.4 | 3.3 ; Pt 3 | 2.7 | 3.4 | 3.1 ;
P i..l: 0.009 | >0.9999 | <0.0001 | >0.9999 (i = young M, j = old M, k = young F, l = old F)
Group sizes (Table 1 header): NC 160 | 103 | 50 | 56 ; Pt 146 | 139 | 77 | 76
Results, Baseline characteristics: 681 patients, 438 without comorbidities.
"""
import os
from scipy.stats import chi2_contingency, fisher_exact
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyArrowPatch, FancyBboxPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#d9d8d4"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "svg.fonttype": "none",
"text.color": TXT, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC})
# ---------------------------------------------------------------- inputs (Table 1)
GROUPS = ["Young males", "Young females", "Old males", "Old females"]
# same order as Table 1 columns; footnote letter of the matched NC-vs-patient P value
PLETTER = {"Young males": "i", "Young females": "k", "Old males": "j", "Old females": "l"}
ALT_NC = {"Young males": 17.0, "Young females": 11.0, "Old males": 21.0, "Old females": 17.0}
ALT_PT = {"Young males": 27.0, "Young females": 13.0, "Old males": 29.0, "Old females": 18.8}
ALT_P = {"i": "<0.0001", "j": "0.308", "k": "0.0008", "l": ">0.999"}
ULN_NC = {"Young males": (12, 160), "Young females": (2, 103), "Old males": (2, 50), "Old females": (2, 56)}
ULN_PT = {"Young males": (37, 143), "Young females": (11, 137), "Old males": (26, 76), "Old females": (12, 76)}
ULN_P = {"i": "<0.0001", "j": "<0.0001", "k": "0.039", "l": "0.024"}
LYM_NC = {"Young males": 1.9, "Young females": 1.8, "Old males": 1.9, "Old females": 1.9}
LYM_PT = {"Young males": 1.3, "Young females": 1.3, "Old males": 1.2, "Old females": 1.2}
NEU_NC = {"Young males": 3.5, "Young females": 3.5, "Old males": 3.4, "Old females": 3.3}
NEU_PT = {"Young males": 3.0, "Young females": 2.7, "Old males": 3.4, "Old females": 3.1}
NEU_P = {"i": "0.009", "j": ">0.9999", "k": "<0.0001", "l": ">0.9999"}
N_NC = {"Young males": 160, "Young females": 103, "Old males": 50, "Old females": 56}
N_PT = {"Young males": 146, "Young females": 139, "Old males": 77, "Old females": 76}
# ---------------------------------------------------------------- 1. median ALT
print("=== 1. Median ALT, non-comorbid patients vs age- and sex-matched normal controls (Table 1) ===")
DIFF = {}
for g in GROUPS:
d = ALT_PT[g] - ALT_NC[g]
DIFF[g] = d
L = PLETTER[g]
print(f"{g:14s} NC {ALT_NC[g]:5.1f} patients {ALT_PT[g]:5.1f} patient - control = {d:+5.1f} IU/L "
f"ratio {ALT_PT[g] / ALT_NC[g]:.2f} direction {'HIGHER' if d > 0 else 'lower'} in patients "
f"reported P-value {L} = {ALT_P[L]}")
n_hi = sum(DIFF[g] > 0 for g in GROUPS)
print(f"Groups with patient median ALT higher than controls: {n_hi}/4; lower: {4 - n_hi}/4")
print("Of these, significant at P < 0.05 by the paper's own P values: young males (i <0.0001) and young females (k 0.0008)")
print("Note: the 'cccc' mark on the young-female patient median refers to the comparison with young male patients, "
"not with young female controls.")
# ---------------------------------------------------------------- 2. ALT above upper normal limit
print("\n=== 2. ALT above the upper normal limit (M >40, F >35 IU/L), controls vs patients ===")
PP = {}
for g in GROUPS:
(a, n1), (b, n2) = ULN_NC[g], ULN_PT[g]
t = [[a, n1 - a], [b, n2 - b]]
pf = fisher_exact(t)[1]
py = chi2_contingency(t, correction=True)[1]
pu = chi2_contingency(t, correction=False)[1]
p1, p2 = 100 * a / n1, 100 * b / n2
PP[g] = (p1, p2)
L = PLETTER[g]
print(f"{g:14s} NC {a}/{n1} = {p1:4.1f}% patients {b}/{n2} = {p2:4.1f}% difference {p2 - p1:+5.1f} points "
f"Fisher P = {pf:.2g} Yates chi2 P = {py:.2g} uncorrected chi2 P = {pu:.2g} reported P-value {L} = {ULN_P[L]}")
print("Higher in patients in", sum(PP[g][1] > PP[g][0] for g in GROUPS), "of 4 groups")
print("Ratio patient share / control share:", ", ".join(f"{g} {PP[g][1] / PP[g][0]:.1f}x" for g in GROUPS))
print("ALT denominators among patients vs group size:",
{g: f"{ULN_PT[g][1]}/{N_PT[g]}" for g in GROUPS})
# ---------------------------------------------------------------- 3. the rest of the abstract sentence
print("\n=== 3. Other parts of the same abstract sentence (for contrast) ===")
for g in GROUPS:
L = PLETTER[g]
print(f"{g:14s} lymphocytes NC {LYM_NC[g]} -> patients {LYM_PT[g]} ({'lower' if LYM_PT[g] < LYM_NC[g] else 'not lower'}, P <0.0001); "
f"neutrophils NC {NEU_NC[g]} -> patients {NEU_PT[g]} "
f"({'lower' if NEU_PT[g] < NEU_NC[g] else 'not lower'}, reported P-value {L} = {NEU_P[L]})")
print("Lymphocytes lower in", sum(LYM_PT[g] < LYM_NC[g] for g in GROUPS), "of 4 groups (consistent with the abstract)")
print("Neutrophils significantly lower only in young males (0.009) and young females (<0.0001) "
"(consistent with 'only young male and female patients had lower neutrophil counts')")
# ---------------------------------------------------------------- 4. population
print("\n=== 4. Population of the Table 1 patient columns ===")
s = sum(N_PT.values())
print(f"Patients: {' + '.join(str(N_PT[g]) for g in GROUPS)} = {s}; 681 - 243 = {681 - 243}; "
f"non-comorbid patients stated in Results = 438 -> {'match' if s == 438 else 'MISMATCH'}")
print(f"Controls: {' + '.join(str(N_NC[g]) for g in GROUPS)} = {sum(N_NC.values())} (abstract: 369 NCs)")
# ---------------------------------------------------------------- figure 1: abstract vs Table 1
def panel(ax, x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.006,rounding_size=0.012",
fc="white", ec="#bdbcb8", lw=1.0, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.045, title, fontsize=11, fontweight="bold", va="top", transform=ax.transAxes)
fig = plt.figure(figsize=(7.2, 4.3), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_axis_off()
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
# left panel: abstract
lx, ly, lw, lh = 0.02, 0.15, 0.31, 0.82
panel(ax, lx, ly, lw, lh, "Abstract, 2nd sentence")
quote = ("\u201cCompared to the age-\nmatched NCs, both young\nand old male and female\nnon-comorbid COVID-19\n"
"patients had lower\nlymphocyte counts and\nalanine aminotransferase\n(ALT) concentration, \u2026\u201d")
ax.text(lx + 0.02, ly + lh - 0.12, quote, fontsize=10, color=SEC, va="top", linespacing=1.35, transform=ax.transAxes)
ax.text(lx + 0.02, ly + 0.17, "Stated direction:", fontsize=10, color=SEC, va="top", transform=ax.transAxes)
ax.text(lx + 0.02, ly + 0.11, "ALT LOWER in\npatients, all 4 groups", fontsize=11, color=ORANGE,
fontweight="bold", va="top", transform=ax.transAxes)
# right panel: Table 1
rx, ry, rw, rh = 0.45, 0.15, 0.53, 0.82
panel(ax, rx, ry, rw, rh, "Table 1, ALT (median, IU/L)")
cg, cm, cd, cp = rx + 0.02, rx + 0.235, rx + 0.375, rx + 0.51
ytop = ry + rh - 0.12
ax.text(cg, ytop, "Group", fontsize=10, color=SEC, va="top", transform=ax.transAxes)
ax.text(cm, ytop, "Control\u2192patient\n(reported)", fontsize=10, color=SEC, va="top", ha="center",
transform=ax.transAxes)
ax.text(cd, ytop, "Difference\n(recomputed)", fontsize=10, color=BLUE, va="top", ha="center", transform=ax.transAxes)
ax.text(cp, ytop, "P", fontsize=10, color=SEC, va="top", ha="right", transform=ax.transAxes)
rowy = [ytop - 0.13 - i * 0.065 for i in range(4)]
for g, yy in zip(GROUPS, rowy):
L = PLETTER[g]
ax.text(cg, yy, g, fontsize=10, va="top", transform=ax.transAxes)
ax.text(cm, yy, f"{ALT_NC[g]:.1f}\u2192{ALT_PT[g]:.1f}", fontsize=10, va="top", ha="center", transform=ax.transAxes)
ax.text(cd, yy, f"{DIFF[g]:+.1f}", fontsize=10.5, va="top", ha="center", color=BLUE, fontweight="bold",
transform=ax.transAxes)
ax.text(cp, yy, ALT_P[L], fontsize=10, va="top", ha="right", color=SEC, transform=ax.transAxes)
ax.text(cg, rowy[-1] - 0.085, "Share above upper normal limit,\ncontrol\u2192patient (reported):\n" +
", ".join(f"{PP[g][0]:.1f}\u2192{PP[g][1]:.1f}%" for g in GROUPS),
fontsize=10, color=SEC, va="top", linespacing=1.35, transform=ax.transAxes)
ax.text(cg, ry + 0.05, f"ALT HIGHER in patients, {n_hi}/4 groups", fontsize=11, color=BLUE,
fontweight="bold", va="center", transform=ax.transAxes)
# connector
cx = (lx + lw + rx) / 2
arr = FancyArrowPatch((rx - 0.005, 0.55), (lx + lw + 0.005, 0.55), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT, transform=ax.transAxes)
ax.add_patch(arr)
ax.text(cx, 0.60, "opposite\ndirection", fontsize=10, ha="center", va="bottom", fontweight="bold",
transform=ax.transAxes)
ax.text(cx, 0.50, f"0/4 lower\n{n_hi}/4 higher", fontsize=10, ha="center", va="top", color=SEC,
transform=ax.transAxes)
ax.text(0.02, 0.075, "Orange: the abstract's wording as reported. Blue: patient minus control, recomputed.",
fontsize=10, color=SEC, va="center", transform=ax.transAxes)
ax.text(0.02, 0.03, "P: the paper's own control-vs-patient P values (Table 1 footnotes i, k, j, l).",
fontsize=10, color=SEC, va="center", transform=ax.transAxes)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ---------------------------------------------------------------- figure 2: direction of each difference
fig, ax = plt.subplots(figsize=(7.2, 3.4), facecolor=BG)
ax.set_facecolor(BG)
ys = list(range(len(GROUPS)))[::-1]
xmin, xmax = -12, 14
ax.axvspan(xmin, 0, color=ORANGE, alpha=0.08, lw=0)
ax.axvline(0, color=SEC, lw=1)
ax.text(xmin + 0.4, 3.55, "Abstract (as reported): every group\nshould fall here (patients lower)", fontsize=10,
color=ORANGE, va="center", ha="left", fontweight="bold")
for g, y in zip(GROUPS, ys):
L = PLETTER[g]
ax.plot([0, DIFF[g]], [y, y], color=BLUE, lw=2, solid_capstyle="butt")
ax.plot(DIFF[g], y, "o", ms=9, color=BLUE, zorder=3)
ax.text(DIFF[g] + 0.6, y, f"{DIFF[g]:+.1f}", fontsize=10, va="center", color=BLUE, fontweight="bold")
ax.text(1.03, y, ALT_P[L], fontsize=10, va="center", ha="left", color=SEC,
transform=ax.get_yaxis_transform())
ax.text(1.03, 3.75, "Reported P", fontsize=10, va="center", ha="left", color=SEC, transform=ax.get_yaxis_transform())
ax.text(0.4, -0.75, "Table 1, recomputed: patients higher", fontsize=10, color=BLUE, ha="left",
va="center", fontweight="bold")
ax.set_yticks(ys)
ax.set_yticklabels(GROUPS, fontsize=10, color=TXT)
ax.set_xlim(xmin, xmax)
ax.set_ylim(-1.1, 4.1)
ax.set_xlabel("Median ALT, patients minus matched controls (IU/L)", fontsize=10)
ax.grid(axis="x", color=GRID, lw=0.6)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
fig.tight_layout()
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract reports phantom-sensation scores as phantom range of motion in degrees
PNAS (2021) · doi:10.1073/pnas.2019555118 · PMC7936324
Summary
The abstract states that subjects with an agonist–antagonist myoneural interface below-knee amputation (AB, n = 15) reported "significantly greater phantom range of motion" than subjects with a traditional below-knee amputation (TB, n = 7), quantified as "AB: 12.47 ± 2.41, TB: 10.14 ± 1.45 degrees". In the body of the paper, 12.47 ± 2.41 and 10.14 ± 1.9 are the group means of the phantom sensation score, a unitless sum of 1–3 grades over foot regions. The per-subject scores in Fig. 1B reproduce these values exactly (recomputed 12.47 ± 2.42 and 10.14 ± 1.95). Phantom range of motion is a different measure: the paper reports it as a percentage of the contralateral limb, separately for each direction, and never as a single value in degrees. The abstract therefore attaches another outcome's numbers, a unit and a TB standard deviation (1.45) that none of the paper's data reproduce to one of its headline comparisons. Readers of the abstract would take the difference as about 2.3 degrees of phantom motion. It is in fact a 23% higher sensation score.
What the paper reports
The abstract:
"AB subjects reported significantly greater phantom range of motion postamputation (AB: 12.47 ± 2.41, TB: 10.14 ± 1.45 degrees) when compared to TB subjects (P < 0.05)." (Abstract)
The Results give the same means, with a different label and a different TB SD:
"AB subjects reported significantly higher phantom sensations (12.47 ± 2.41, P = 0.03) compared to TB subjects (10.14 ± 1.9) (P < 0.05, Student’s t test)." (Results, "Phantom Sensations and Pain")
The Methods define the phantom sensation score as a grade per foot region:
"Subjects were then specifically asked to relate the level of perception of their lateral and medial malleoli, heel pad, arch of the foot, balls of the footpad, and toes. The more thorough and tangible the phantom limb felt to subjects, the greater the phantom sensation scored. Responses were independently graded on a 1 to 3 scale" (Methods, "Phantom Limb Sensation")
Fig. 1B is a per-subject table. It has five region columns (Malleoli, Heelpad, Arch of foot, Balls of feet, Toes), a "Phantom Sensation Score" column and a "Phantom Pain Score" column, with summary rows:
"Average Std Dev 12.47 +/- 2.4" (AB) and "Average Std Dev 10.14 +/- 1.9" (TB) (Fig. 1B, Phantom Sensation Score column; transcribed from the version-of-record PDF)
"Phantom sensations in the AB cohort (n = 15) are significantly greater than those in the TB cohort (n = 7) (P < 0.05, Student’s t test)." (Fig. 1 legend, panel B)
Phantom range of motion (ROM) is measured and reported differently:
"Data were processed through custom scripts in MATLAB (Mathworks) to identify the maximum amplitude for each movement and normalized as a percentage of the amplitude of the contralateral limb." (Methods, "ROM Percepts")
"The demonstrated ROM of the phantom limb in each direction (PF, DF, IN, EV) were normalized as a percentage of the unaffected joints’ ROM (Fig. 2C)." (Results, "ROM Percepts")
"Further, AB subjects experienced significantly greater ROM than TB subjects following amputation for PF, DF, and EV (P < 0.05, Student’s two-tailed t test)." (Results, "ROM Percepts")
What the paper's own numbers show
Method. The 22 per-subject phantom sensation scores and pain scores were transcribed from Fig. 1B of the publisher's PDF into recompute.py.txt:
- AB scores: 14, 11, 8, 13, 15, 14, 14, 8, 13, 14, 10, 14, 14, 15, 10.
- TB scores: 8, 8, 11, 12, 13, 9, 10.
The script computes each group's mean, its sample SD (n − 1), its population SD (n) and its SEM, and runs Student’s and Welch's two-sample t tests on the raw scores and on the reported summary statistics. It also derives the possible range of the score from the number of graded regions. The Methods, the Results ("ROM Percepts") and the Fig. 2 legend were searched for any phantom ROM value expressed in degrees.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| AB value in the "phantom range of motion" sentence | Abstract | 12.47 ± 2.41 degrees | Equals the AB phantom sensation score: Fig. 1B mean 12.467 (187/15), SD 2.416 |
| TB value in the "phantom range of motion" sentence | Abstract | 10.14 ± 1.45 degrees | Mean equals the TB phantom sensation score: Fig. 1B mean 10.143 (71/7). SD is 1.952, not 1.45 |
| AB phantom sensation score | Results; Fig. 1B | 12.47 ± 2.41 (Fig. 1B: ± 2.4) | 12.467 ± 2.416 |
| TB phantom sensation score | Results; Fig. 1B | 10.14 ± 1.9 | 10.143 ± 1.952 |
| TB dispersion statistics that could produce 1.45 | Abstract | 1.45 | Sample SD 1.95, population SD 1.81, SEM 0.74: none equals 1.45 |
| Possible range of the phantom sensation score | Methods; Fig. 1B | 1–3 per region; 5 region columns | 5–15, with no unit (6–18 if the two malleoli were graded separately). Observed scores run from 8 to 15, and 2 AB subjects score 15 |
| Phantom ROM, the measure the abstract names | Methods and Results, "ROM Percepts"; Fig. 2C | % of the contralateral limb, per direction (PF, DF, IN, EV) | No pooled value and no value in degrees appears anywhere in the paper |
| AB − TB difference as stated in the abstract | Abstract | 12.47 − 10.14 "degrees" | 2.33 score points; ratio of the means 1.230 (23% higher score) |
| Phantom sensation test, raw Fig. 1B scores | Results | P = 0.03 (Student’s t test) | Student t = 2.220, P = 0.038; Welch t = 2.405, P = 0.030 |
| Phantom sensation test, summary statistics with TB SD 1.9 | Results | P = 0.03 | Student t = 2.243, P = 0.036; Welch P = 0.027 |
| Phantom sensation test, summary statistics with TB SD 1.45 | Abstract | P < 0.05 | Student t = 2.349, P = 0.029; Welch P = 0.011 |
Whichever SD is used, the phantom-sensation comparison remains significant at P < 0.05. (The reported P = 0.03 matches Welch's test on the raw scores, 0.030. Student's test, which the paper names, gives 0.038. Both are below 0.05.) The discrepancy lies in what the abstract's numbers are said to measure, in their unit and in the TB SD. It does not affect whether the comparison is significant.
Other observations (outside the main finding). The pain values in the text do not match Fig. 1B either. The pain score column of Fig. 1B gives AB 3.467 ± 4.96 and TB 9.286 ± 10.23 (printed summary rows: 3.46 ± 4.9 and 9.28 ± 10.2). The Results give 12.25 ± 4.9 and 17.29 ± 10.22: the SDs agree with Fig. 1B but the means do not. The abstract gives the AB pain SD as 5.37 rather than 4.9. Fig. 1B lists 7 AB subjects with a pain score of 0, and the Results say 6 of the 15 "indicated zero pain in all categories". From the reported summary statistics, the AB versus TB pain comparison gives Student P = 0.128 (SD 4.9) or 0.141 (SD 5.37), against the reported P = 0.08. The paper does not describe enough of how the pain score was built to resolve these differences, so they are listed here only for completeness.
Figure
Figure 1. The abstract sentence (top, orange outline) gives "12.47 ± 2.41" and "10.14 ± 1.45 degrees" as phantom range of motion. The same means appear in the Results (bottom left) as phantom sensation scores, which are sums of 1–3 grades over the five foot regions of Fig. 1B. The possible range is 5–15 and the score has no unit. The TB SD there is 1.9, and the Fig. 1B per-subject scores give 1.95 (blue, recomputed in recompute.py.txt). Phantom range of motion, the measure the abstract names (bottom right), is reported only as a percentage of the contralateral limb, separately for each direction, and never in degrees. Orange text marks values printed in the paper, blue text marks values recomputed from the paper's own per-subject data.
Figure 2. Dispersion of the seven TB phantom sensation scores listed in Fig. 1B (8, 8, 11, 12, 13, 9, 10). Orange circles: SDs printed in the abstract (1.45) and in the Results and Fig. 1B (1.9). Blue diamonds: the sample SD (1.95), population SD (1.81) and SEM (0.74) recomputed from the per-subject scores. The dashed line marks the abstract's 1.45. None of the recomputed statistics equals it, so the abstract's SD cannot be explained as a different summary of the same data.
Why it matters
The sentence is the abstract's only quantitative statement about phantom perception, and the paper's Significance statement and Discussion present phantom perception as a main benefit of the procedure. Read as written, the sentence tells the reader that AB subjects could perceive about 2.3 degrees more phantom joint motion than TB subjects. That would be a very small absolute range for whole-ankle motion. It would also conflict with the paper's own ROM method, which expresses ROM as a percentage of the unaffected limb. The numbers actually describe something else: AB subjects' summed phantom sensation grades were about 23% higher than TB subjects'. The abstract also gives a TB SD (1.45) smaller than the one in the Results (1.9) and the one the data support (1.95), which makes the TB group look less variable than it was.
The direction and statistical significance of both underlying findings are unaffected. In the Results, AB subjects have higher phantom sensation scores (P < 0.05 under every recomputation above), and significantly greater phantom ROM in three of the four directions (PF, DF, EV). The error is in how the abstract reports these findings. It merges two outcomes into one sentence, gives the magnitude of one under the name and a unit of the other, and leaves the actual ROM result (percent of contralateral ROM by direction) unquantified in the abstract. Anyone extracting effect sizes from the abstract, for example in a systematic review or meta-analysis of phantom limb outcomes after amputation, would record a degree-valued ROM difference that the study did not measure.
Explanations considered
- A separate pooled phantom-ROM measure in degrees exists. It does not. The Methods ("ROM Percepts") normalize goniometer amplitudes "as a percentage of the amplitude of the contralateral limb". The Results ("ROM Percepts") and Fig. 2C report ROM separately for PF, DF, IN and EV as a percentage of the unaffected joint. The paper gives no pooled mean ± SD for ROM and no value in degrees. The values 12.47 and 10.14 occur only in the phantom-sensation sentence of the Results and in Fig. 1B, apart from the abstract.
- The numbers coincide by chance. Both means match to two decimals, and so does the AB SD (2.41). Fig. 1B's per-subject phantom sensation scores reproduce them exactly (187/15 = 12.467, 71/7 = 10.143). Two measures agreeing by chance on three statistics is not a plausible explanation.
- "Degrees" is used loosely to mean "levels" or "grades". This would explain the unit but not the label. The sentence still names range of motion rather than sensation, and in this paper range of motion is a percentage. The TB SD of 1.45 would also remain unexplained.
- 1.45 is a different dispersion statistic (population SD, SEM) for the same data. The seven TB scores give a sample SD of 1.95, a population SD of 1.81 and an SEM of 0.74. None equals 1.45 (Fig. 2), and the AB SD in the same sentence (2.41) is the ordinary sample SD.
- A different time point or subgroup. Both the abstract and the Results describe the postamputation AB versus TB comparison with n = 15 and n = 7. The paper reports no other phantom-sensation or ROM summary with these means.
- A conversion or typesetting artifact in PMC. Ruled out: the publisher's typeset PDF (cover sheet: "Final published version") has the identical abstract and Results wording, and Europe PMC's full-text XML matches.
- The supplement holds the degree values. The SI Appendix (7 pages) covers methods figures, Fig. S4 (pain scores before and after surgery), Fig. S5 (correlations of fascicle strain with ROM and with phantom sensation) and Fig. S6 (F-test P values). It gives no group phantom-ROM values in degrees and no TB SD of 1.45. The five supplementary movies show ultrasound and fascicle dynamics and contain no summary numbers.
Status
- Version of record: checked. pnas.org returned HTTP 403 to automated access. The publisher's own typeset PDF was read from MIT DSpace (hdl.handle.net/1721.1/134133, marked "Version: Final published version"). Its abstract (p. 1) and Results (p. 3) wording is identical to the PMC archive, and Fig. 1B (p. 3 of 8) supplied the per-subject scores used here.
- Supplementary material: SI Appendix PDF checked (see above). Movies S1–S5 not opened, as they contain no tabulated values.
- Corrections: none found. Crossref lists no update relations, and PMC and Europe PMC list no correction, erratum, retraction or expression of concern linked to the article.
- PubPeer: not checked. pubpeer.com refused automated requests (HTTP 403 / human-verification challenge), and a web search for PubPeer threads could not be run in this session. Whether the issue has been raised there is therefore unknown.
- Sources: PMC7936324 (PubMed Central archive); doi:10.1073/pnas.2019555118 (publisher PDF via MIT DSpace); Crossref and Europe PMC metadata.
- As of: 2026-10-23.
Suggested correction
In the abstract, replace
"AB subjects reported significantly greater phantom range of motion postamputation (AB: 12.47 ± 2.41, TB: 10.14 ± 1.45 degrees) when compared to TB subjects (P < 0.05)."
with a sentence that keeps the two outcomes separate and reports the values given in the Results and Fig. 1B, for example:
"AB subjects reported significantly higher phantom sensation scores (AB: 12.47 ± 2.41, TB: 10.14 ± 1.9; P < 0.05) and significantly greater phantom range of motion, as a percentage of the unaffected limb, for plantarflexion, dorsiflexion and eversion (P < 0.05) when compared to TB subjects."
If the authors intended a different TB SD or a phantom-ROM summary in degrees, the source of 1.45 and of any degree values should be stated, since neither can be derived from the data presented. Separately, the authors could clarify how the text pain values (12.25 ± 4.9 and 17.29 ± 10.22, with 5.37 in the abstract) relate to the Fig. 1B pain scores (3.46 ± 4.9 and 9.28 ± 10.2).
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC7936324 and regenerate fig1/fig2.
Run: python recompute.py.txt (writes fig1.svg/png and fig2.svg/png next to this file)
Inputs are transcribed from the article (PNAS 2021, doi:10.1073/pnas.2019555118):
- Abstract and Results text, as archived in PubMed Central (PMC7936324) and identical
in the publisher's typeset PDF ("Final published version").
- Fig. 1B, the per-subject table of Phantom Sensation Score and Phantom Pain Score
(15 AB rows, 7 TB rows, in printed order), read from the version-of-record PDF.
"""
import os
import textwrap
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch, Patch
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
# Fig. 1B, "Phantom Sensation Score" column
PS_AB = np.array([14, 11, 8, 13, 15, 14, 14, 8, 13, 14, 10, 14, 14, 15, 10])
PS_TB = np.array([8, 8, 11, 12, 13, 9, 10])
# Fig. 1B, "Phantom Pain Score" column
PAIN_AB = np.array([0, 2, 0, 1, 5, 0, 10, 15, 12, 3, 0, 0, 0, 0, 4])
PAIN_TB = np.array([0, 5, 0, 11, 11, 8, 30])
# Fig. 1B region columns: Malleoli, Heelpad, Arch of foot, Balls of feet, Toes
N_REGION_COLS = 5
GRADE_MIN, GRADE_MAX = 1, 3 # Methods: "graded on a 1 to 3 scale"
# Reported values
ABS = dict(ab_m=12.47, ab_s=2.41, tb_m=10.14, tb_s=1.45) # Abstract, "degrees"
RES = dict(ab_m=12.47, ab_s=2.41, tb_m=10.14, tb_s=1.9, p=0.03) # Results, phantom sensation
FIG1B = dict(ab_m=12.47, ab_s=2.4, tb_m=10.14, tb_s=1.9,
pain_ab_m=3.46, pain_ab_s=4.9, pain_tb_m=9.28, pain_tb_s=10.2)
PAIN_RES = dict(ab_m=12.25, ab_s=4.9, tb_m=17.29, tb_s=10.22, p=0.08)
PAIN_ABS_AB_S = 5.37
nA, nT = len(PS_AB), len(PS_TB)
def desc(x):
return x.mean(), x.std(ddof=1), x.std(ddof=0), x.std(ddof=1) / np.sqrt(len(x))
print("== 1. Scale of the phantom sensation score ==")
lo, hi = N_REGION_COLS * GRADE_MIN, N_REGION_COLS * GRADE_MAX
print(f"Fig. 1B region columns: {N_REGION_COLS}; graded {GRADE_MIN}-{GRADE_MAX} -> possible total {lo}-{hi}")
print(f"(if lateral and medial malleoli were graded separately, 6 regions -> {6*GRADE_MIN}-{6*GRADE_MAX})")
print(f"Observed Fig. 1B scores: AB {PS_AB.min()}-{PS_AB.max()}, TB {PS_TB.min()}-{PS_TB.max()}")
print(f"AB subjects at the {hi}-point maximum: {(PS_AB == hi).sum()}")
print()
print("== 2. Phantom sensation scores recomputed from Fig. 1B ==")
mA, sA, pA, seA = desc(PS_AB)
mT, sT, pT, seT = desc(PS_TB)
print(f"AB: n={nA}, sum={PS_AB.sum()}, mean={mA:.3f}, SD(n-1)={sA:.3f}, SD(n)={pA:.3f}, SEM={seA:.3f}")
print(f"TB: n={nT}, sum={PS_TB.sum()}, mean={mT:.3f}, SD(n-1)={sT:.3f}, SD(n)={pT:.3f}, SEM={seT:.3f}")
print(f"Reported Results/Fig. 1B: AB {RES['ab_m']} +/- {RES['ab_s']}, TB {RES['tb_m']} +/- {RES['tb_s']}")
print(f"Reported Abstract ('degrees'): AB {ABS['ab_m']} +/- {ABS['ab_s']}, TB {ABS['tb_m']} +/- {ABS['tb_s']}")
cands = {"SD (n-1)": sT, "SD (n)": pT, "SEM": seT}
print("TB dispersion candidates vs abstract 1.45:",
", ".join(f"{k} {v:.2f}" for k, v in cands.items()),
"-> any equal to 1.45 after rounding:", any(round(v, 2) == 1.45 for v in cands.values()))
print(f"Abstract AB-TB difference: {ABS['ab_m'] - ABS['tb_m']:.2f} (abstract unit: degrees); "
f"as a ratio of score means: {ABS['ab_m'] / ABS['tb_m']:.3f} "
f"({100 * (ABS['ab_m'] / ABS['tb_m'] - 1):.0f}% higher score)")
print()
print("== 3. Two-sample t tests ==")
def tt(m1, s1, n1, m2, s2, n2, eq):
r = stats.ttest_ind_from_stats(m1, s1, n1, m2, s2, n2, equal_var=eq)
return r.statistic, r.pvalue
for lab, s in [("Results TB SD 1.9", RES["tb_s"]), ("Abstract TB SD 1.45", ABS["tb_s"])]:
t, p = tt(12.47, 2.41, nA, 10.14, s, nT, True)
tw, pw = tt(12.47, 2.41, nA, 10.14, s, nT, False)
print(f"Summary stats, {lab}: Student t={t:.3f} p={p:.3f}; Welch t={tw:.3f} p={pw:.3f}")
r_s = stats.ttest_ind(PS_AB, PS_TB, equal_var=True)
r_w = stats.ttest_ind(PS_AB, PS_TB, equal_var=False)
print(f"Fig. 1B per-subject scores: Student t={r_s.statistic:.3f} p={r_s.pvalue:.3f}; "
f"Welch t={r_w.statistic:.3f} p={r_w.pvalue:.3f} (reported P = {RES['p']}, 'Student's t test')")
print()
print("== 4. Side observations: pain scores (not part of the main finding) ==")
pmA, psA, _, _ = desc(PAIN_AB)
pmT, psT, _, _ = desc(PAIN_TB)
print(f"Fig. 1B pain, recomputed: AB mean={pmA:.3f} SD={psA:.2f}; TB mean={pmT:.3f} SD={psT:.2f}")
print(f"Fig. 1B pain summary rows: AB {FIG1B['pain_ab_m']} +/- {FIG1B['pain_ab_s']}; "
f"TB {FIG1B['pain_tb_m']} +/- {FIG1B['pain_tb_s']}")
print(f"Results pain: AB {PAIN_RES['ab_m']} +/- {PAIN_RES['ab_s']}; TB {PAIN_RES['tb_m']} +/- {PAIN_RES['tb_s']}; "
f"Abstract AB SD {PAIN_ABS_AB_S}")
print(f"AB subjects with a Fig. 1B pain score of 0: {(PAIN_AB == 0).sum()} of {nA} "
f"(Results: '6 of the 15 subjects ... indicated zero pain in all categories')")
for lab, s in [("Results AB SD 4.9", 4.9), ("Abstract AB SD 5.37", 5.37)]:
t, p = tt(12.25, s, nA, 17.29, 10.22, nT, True)
print(f"Pain, summary stats, {lab}: Student t={t:.3f} p={p:.3f} (reported P = {PAIN_RES['p']})")
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
def box(ax, x, y, w, h, title, body, width_chars, edge=INK2):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.004,rounding_size=0.012",
fc="white", ec=edge, lw=1.2, transform=ax.transAxes))
ax.text(x + 0.015, y + h - 0.022, title, fontsize=10, fontweight="bold", va="top",
transform=ax.transAxes)
ax.text(x + 0.015, y + h - 0.075, "\n".join(textwrap.wrap(body, width_chars)),
fontsize=10, va="top", color=INK, transform=ax.transAxes, linespacing=1.35)
# ---- Figure 1: abstract vs body text diagram
fig = plt.figure(figsize=(7.2, 5.6), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_axis_off(); ax.set_facecolor(BG)
box(ax, 0.03, 0.80, 0.94, 0.17, "Abstract",
"\u201cAB subjects reported significantly greater phantom range of motion postamputation "
"(AB: 12.47 \u00b1 2.41, TB: 10.14 \u00b1 1.45 degrees) when compared to TB subjects (P < 0.05).\u201d",
92, edge=ORANGE)
box(ax, 0.03, 0.15, 0.455, 0.285, "Results: Phantom Sensations and Pain",
"\u201cAB subjects reported significantly higher phantom sensations (12.47 \u00b1 2.41, P = 0.03) "
"compared to TB subjects (10.14 \u00b1 1.9)\u201d. Fig. 1B lists the per-subject scores: "
"five foot regions, each graded 1\u20133 (Methods).",
40)
box(ax, 0.515, 0.15, 0.455, 0.285, "Methods / Results: ROM Percepts",
"Phantom ROM was \u201cnormalized as a percentage of the amplitude of the contralateral limb\u201d "
"and reported per direction (PF, DF, IN, EV; Fig. 2C). No pooled phantom ROM in degrees "
"appears anywhere in the paper.",
40)
for x in (0.10, 0.585):
ax.add_patch(FancyArrowPatch((x, 0.79), (x, 0.445), transform=ax.transAxes,
arrowstyle="-|>", mutation_scale=14, lw=2, color=INK2))
def note(x, y, s, **kw):
ax.text(x, y, s, fontsize=10, va="top", transform=ax.transAxes, linespacing=1.35, **kw)
note(0.12, 0.76, "Same numbers, different measure", fontweight="bold")
note(0.12, 0.72, "Unitless score, possible range 5\u201315\n(Fig. 1B scores span 8\u201315)")
note(0.12, 0.635, "TB SD 1.45 abstract (reported)", color=ORANGE, fontweight="bold")
note(0.12, 0.595, "TB SD 1.9 Results (reported)", color=ORANGE, fontweight="bold")
note(0.12, 0.555, f"TB SD {sT:.2f} Fig. 1B (recomputed)", color=BLUE, fontweight="bold")
note(0.605, 0.76, "The measure the abstract names", fontweight="bold")
note(0.605, 0.72, "% of the contralateral limb,\nper direction; never pooled\nand never given in degrees")
ax.legend(handles=[Patch(fc="white", ec=ORANGE, lw=1.5, label="Text containing the conflicting statement"),
Line2D([], [], color=ORANGE, lw=0, marker="s", ms=9, label="Value reported in the paper"),
Line2D([], [], color=BLUE, lw=0, marker="s", ms=9,
label="Recomputed from the paper's Fig. 1B per-subject scores")],
loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=1, frameon=False, fontsize=10)
save(fig, "fig1")
# ---- Figure 2: TB dispersion, abstract value vs everything the Fig. 1B data give
rows = [
("Abstract TB SD", ABS["tb_s"], ORANGE, "o", "1.45"),
("Results / Fig. 1B TB SD", RES["tb_s"], ORANGE, "o", "1.9"),
("Sample SD (n−1), Fig. 1B scores", sT, BLUE, "D", f"{sT:.2f}"),
("Population SD (n), Fig. 1B scores", pT, BLUE, "D", f"{pT:.2f}"),
("SEM, Fig. 1B scores", seT, BLUE, "D", f"{seT:.2f}"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.3), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1]
for y, (lab, v, c, m, txt) in zip(ys, rows):
ax.plot([0, v], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(v, y, marker=m, ms=10, color=c, zorder=3, lw=0)
ax.text(v + 0.08, y, txt, va="center", fontsize=10, color=c, fontweight="bold")
ax.axvline(ABS["tb_s"], color=ORANGE, lw=1, ls=(0, (3, 3)), zorder=0)
ax.set_yticks(ys, [r[0] for r in rows], fontsize=10)
ax.set_xlim(0, 2.4)
ax.set_ylim(-0.6, len(rows) - 0.4)
ax.set_xlabel("Dispersion of the TB phantom-sensation score (score points), n = 7")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(INK2)
ax.tick_params(axis="y", length=0)
ax.legend(handles=[Line2D([], [], color=ORANGE, marker="o", ms=9, lw=0, label="Reported in the paper"),
Line2D([], [], color=BLUE, marker="D", ms=9, lw=0,
label="Recomputed from Fig. 1B (TB scores 8, 8, 11, 12, 13, 9, 10)")],
loc="upper center", bbox_to_anchor=(0.35, -0.22), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg/png and fig2.svg/png")
Abstract reverses the direction of the white-light calcium responses shown in Results and Fig. 4B
Proc Natl Acad Sci U S A (2018) · doi:10.1073/pnas.1802592115 · PMC6217433
Summary
The paper reports that a single bistable opsin, parapinopsin, gives zebrafish pineal photoreceptors opposite calcium responses to UV and visible light. In one experiment (Fig. 4B), white light was shifted toward UV or toward visible wavelengths. The abstract says these shifts "respectively increased vs. decreased" the calcium signal. The Results paragraph, the Fig. 4 legend, the transient responses in the plotted Fig. 4B traces, the 405-nm and 588-nm two-photon experiments, and the paper's proposed mechanism all give the opposite: UV-shifted light lowered calcium and visible-shifted light raised it. The existence of opponency and its dependence on parapinopsin are unaffected. But the abstract, the part most readers and indexers rely on, states the sign of the naturalistic-light result backwards, and nothing in the abstract lets a reader notice.
What the paper reports
Abstract:
Changes in the spectral composition of white light toward enhanced UV or visible wavelengths respectively increased vs. decreased calcium signal in parapinopsin-sufficient but not parapinopsin-deficient photoreceptors.
Results, section "Color Opponency Based on Photoequilibrium of Parapinopsin Two States", white-light paragraph:
We found decreased and increased calcium levels following exposure to UV and visible light-mixed white light, respectively, in WT and PT-/- but not in PP-/- and PP-/-/PT-/- (Fig. 4B).
Fig. 4 legend, panel B:
Note that the calcium levels decreased and increased transiently in response to exposure to UV and visible light-mixed white lights, respectively, in WT and PT-/-.
Results, section "Antagonistic Chromatic Responses in a Single Pineal Photoreceptor Cell" (two-photon imaging, Fig. 1C):
stimulation with 405-nm light caused a decrease in fluorescence intensity.
In contrast, stimulation with 588-nm light elevated the fluorescence intensity, demonstrating the antagonistic effect of UV and visible light on the calcium in individual PP cells to UV and visible light (Fig. 1C).
Results, mechanism paragraph (Fig. 3):
The 405-nm light stimulus increases the short-wavelength component of incident light and thus increases the fractional concentration of the signaling active photoproduct from that produced by the imaging light alone, leading to hyperpolarization (Fig. 3 B and D, calcium decrease).
Conversely, because the photoproduct is more sensitive to longer wavelengths, the 588-nm light stimulus decreases its fractional concentration, reducing the pool of signaling-active opsin, resulting in termination of hyperpolarization and increased calcium levels (Fig. 3 C and D, calcium increase).
Discussion, first paragraph:
with lights biased toward UV driving greater hyperpolarization than those biased toward visible light.
Fig. 4 legend, panels A and B (stimulus intensities and sample sizes):
The light intensities (360–780 nm) of white light, UV-mixed white light, visible light-mixed white light, and direct sunlight were ∼9.2 × 1014, ∼9.3 × 1014, ∼9.9 × 1014, and 1.1 × 1017 photons per cm2·s. The light intensities of UV and visible LED lights were ∼1.3 × 1013 and ∼7.6 × 1013 photons per cm2·s, respectively.
Changes in calcium level of PP cells in WT (red open circle; n = 24), PT-/- fish (green closed circle; n = 23), PP-/- fish (purple open triangle; n = 17), and PP-/-/PT-/- fish (black closed triangle; n = 16)
(The exponents read "1014", "1013" and "1017" in the archived text. They are superscripts lost in conversion: 10^14, 10^13, 10^17.)
What the paper's own numbers show
This is a check of direction, not of arithmetic. Each sentence that uses "respectively" pairs the i-th stimulus with the i-th direction word. recompute.py.txt pulls both word orders out of the quoted sentences with a regular expression, builds the stimulus-to-direction mapping for each sentence, and compares the mappings. For the Fig. 4B traces, the direction of the WT and PT-/- deflections at the UV-mixed and visible-mixed bars, and the later course of the traces, were read by eye from the published figure image. No values were measured from it. The script also checks every quotation above against the PMC archive and does the arithmetic on the Fig. 4A intensities.
| Quantity | Location | Reported | Recomputed from the paper's body |
|---|---|---|---|
| Calcium response to UV-enhanced white light | Abstract | increased | decreased |
| Calcium response to visible-enhanced white light | Abstract | decreased | increased |
| Calcium response to UV- / visible-mixed white light | Results, white-light paragraph (Fig. 4B) | decreased / increased | decreased / increased (agrees) |
| Calcium response to UV- / visible-mixed white light | Fig. 4 legend, panel B | decreased / increased | decreased / increased (agrees) |
| Transient deflection of WT and PT-/- traces at UV- / visible-mixed bars | Fig. 4B, plotted data | downward / upward | decreased / increased (agrees; for the later rebound see Explanations, item 4) |
| Fluorescence response to 405-nm / 588-nm light (two-photon) | Results, Fig. 1C | decrease / elevated | decreased / increased (agrees) |
| Predicted calcium change for 405-nm / 588-nm light | Results, mechanism (Fig. 3) | calcium decrease / calcium increase | decreased / increased (agrees) |
| UV-biased vs visible-biased light | Discussion | greater hyperpolarization for UV-biased | lower calcium under UV-biased light (agrees) |
| Body passages with UV → decrease and visible → increase | all of the above except the abstract | — | 5 of 5, plus the Discussion's comparison |
| White + UV LED; white + visible LED (photons per cm²·s) | Fig. 4 legend A | ~9.3 × 10^14; ~9.9 × 10^14 | 9.33 × 10^14 (LED adds 1.4%); 9.96 × 10^14 (LED adds 8.3%). Both agree with the legend within the rounding of its approximate (~) inputs; white light is itself ~9.2 × 10^14 |
| n per genotype, Fig. 4B | Fig. 4 legend B | WT 24, PT-/- 23, PP-/- 17, PP-/-/PT-/- 16 | unaffected by the direction issue |
The abstract gives UV → increase and visible → decrease. Every other passage that states a direction gives UV → decrease and visible → increase. Both added LEDs raise the total photon flux above the same white background (by 1.4% and 8.3%), so the two stimuli differ in spectral composition, not in the sign of the intensity change. The opposite responses are therefore spectral, as the paper argues, and the direction of each is fixed by the body of the paper.
Figure
Figure 1. The left panel quotes the abstract and the right panel quotes the Results sentence on Fig. 4B. Under each quotation is the stimulus-to-direction mapping its "respectively" construction implies. Orange marks the abstract as published. Blue marks what the paper's own body reports. Both directions are reversed between the two panels. The Fig. 4 legend, the transient responses in the plotted Fig. 4B traces, the 405/588-nm experiments and the mechanism all match the right panel. Every word is from the paper or from the output of recompute.py.txt.
Figure 2. For each passage of the paper that states a direction, this grid gives the stated calcium change for UV-shifted light and for visible-shifted light. The abstract (orange, top row) is the only passage with UV → increase and visible → decrease. Six body passages (blue) agree with each other, including the plotted Fig. 4B traces and the mechanism. The Discussion row is a comparison between the two lights rather than a change from baseline.
[Internal, to be removed with candidates/ before sending: five candidate figure designs are in candidates/ for selection. candidate_1 = Figure 1, candidate_2 = Figure 2, candidate_3 = a sign dot plot on one axis, candidate_4 = the "respectively" pairing, candidate_5 = the paper's mechanism chain against the abstract's outcome.]
Why it matters
- The abstract states the headline naturalistic-light result backwards. The Results present Fig. 4B as a test of whether color opponency is observed "under natural light-like conditions"; the abstract reports its outcome. A reader of the abstract alone learns that UV-enriched light raises calcium in parapinopsin-expressing cells, the opposite of what was measured.
- It contradicts the paper's own mechanism. In the paper's model, UV raises the fraction of signaling-active photoproduct, which drives hyperpolarization and a calcium decrease. The abstract's wording implies the reverse. The same holds for the earlier pineal ganglion-cell recordings the Introduction cites, in which firing is "suppressed by UV and enhanced by visible light", although that is a different quantity.
- The error goes further than the paper. PubMed, Europe PMC and Semantic Scholar carry the same abstract. Citing authors, reviewers and model builders who take the sign from the abstract would carry the reversal forward.
- What is not affected. The measured data, the Results, the figure, the sample sizes and the paper's main claim are not affected: opposite-sign responses that disappear without parapinopsin. Only the direction stated in the abstract is wrong.
Explanations considered
- The abstract describes a different quantity. If the abstract meant hyperpolarization, photoproduct amount or ganglion-cell firing, the sign could legitimately differ. It says "calcium signal", the same quantity as the Results and legend (GCaMP6s ΔF/F). This does not account for the conflict.
- The abstract describes a different experiment or population. "Toward enhanced UV or visible wavelengths" matches the UV-mixed and visible-mixed white light of Fig. 4A/B. "Parapinopsin-sufficient but not parapinopsin-deficient" matches WT and PT-/- versus PP-/- and PP-/-/PT-/-. The only experiments that shift the spectrum of white light are Fig. 4B and its repeats at other intensities and durations in SI Appendix Fig. S7. This does not account for the conflict.
- "Respectively" can be read the other way. The abstract lists "UV or visible" and then "increased vs. decreased". The only grammatical pairing is UV with increased. The Results and legend use the same construction with the verbs in the opposite order. This does not account for the conflict.
- The abstract refers to a different time point or phase, such as what follows the stimulus. The plotted traces do show a later change. In WT and PT-/-, calcium dips at the UV-mixed bar, then rebounds above the pre-stimulus baseline and stays elevated until the visible-mixed bar. Taken alone, that rebound has the sign of the UV half of the abstract (UV → increased). It does not account for the abstract, for three reasons. (a) The Results and the Fig. 4 legend name the transient decrease as the response to UV-mixed light: "the calcium levels decreased and increased transiently in response to exposure to UV and visible light-mixed white lights, respectively". The legend adds that the relaxation of these responses was "not observed as a result of time-lapse recoding with low time resolution". (b) The Discussion describes "slow elevation of calcium levels after these levels were greatly decreased" under continuous blue or xenon white light (SI Appendix, Figs. S2 and S6). It relates this to "the kinetics of signaling-active photoproduct", which "could constitute a mechanism of light adaptation", and not to the spectral content of a stimulus. Applying that to the rebound in Fig. 4B is this write-up's inference, but it is the only account of such a rise that the paper offers. (c) The visible half of the abstract (visible → decreased) has no counterpart in the figure. After the visible-mixed peak, the traces return to about the level they had just before the visible-mixed bar, and no decrease follows. So even read as a later phase, the abstract's full mapping is not shown. This does not account for the conflict.
- The difference reflects total intensity rather than spectrum. Both mixed lights add photons (1.4% and 8.3% above the white light), so intensity cannot give opposite signs, and it does not bear on which sign is which. This does not account for the conflict.
- One Discussion sentence uses "increase". The Discussion speaks of "maximum changes in increase of calcium level in PP cells (10–20 s; SI Appendix, Fig. S7)". It is ambiguous and does not say which stimulus produced the increase. The SI text speaks only of "calcium decrease profiles in Fig. S7 (~20 s for reaching the maximum)" and does not name a stimulus; Fig. S7 repeats the UV- and visible-mixed stimuli, so linking that phrase to the UV-mixed responses is an inference. Neither passage supports the abstract's mapping.
- Text-conversion artifact. The conflict is in words, not numbers, and the same abstract wording appears in the Europe PMC XML, PubMed and Semantic Scholar. A lost minus sign, superscript or table cell cannot produce it. This does not account for the conflict.
- Already corrected. No correction or erratum was found (see Status).
None of these accounts for the reversal. The abstract's direction words for the Fig. 4B experiment are in the opposite order to the rest of the paper.
Status
As of 2026-10-23:
- Version of record: checked. The PNAS article page (reached through doi.org) and the PMC HTML page refused automated access (HTTP 403 and a bot check), so the publisher's HTML could not be read directly. The publisher-supplied text was read in the Europe PMC full-text JATS XML for PMC6217433, in the PubMed record (PMID 30322939) and in Semantic Scholar. All three carry the abstract sentence word for word, and the XML carries the Results and legend sentences quoted above. The PMC archive used here (
src/show.py) gives the same passages. - Figure: the published Fig. 4 image (pnas.1802592115fig04.jpg, from the PMC / Europe PMC package) was inspected. The WT and PT-/- traces drop at the UV-mixed bar, then rebound above the pre-stimulus baseline and stay elevated until the visible-mixed bar. They peak at the visible-mixed bar and then return to about their level just before it. The PP-/- and PP-/-/PT-/- traces stay flat. The figure is not reproduced here.
- Supplements: the SI Appendix (pnas.1802592115.sapp.pdf, 18 pages) was read. The Fig. S6 legend refers the UV- and visible-mixed responses to Fig. 4B. The Fig. S7 legend describes repeats at other LED intensities and durations. The SI text speaks of "calcium decrease profiles in Fig. S7" and says that more signaling-active photoproduct "would facilitate depolarizing responses to visible-light stimuli". All of this agrees with the Results, not the abstract. No source data are deposited.
- Corrections and notices:
src/known.pyreturns no Crossref updates, PMC notices or Europe PMC notices. The Crossref record has no update-to or updated-by relation. No correction, erratum, expression of concern or retraction was found. - PubPeer: a web search for the title together with "pubpeer" returned no PubPeer page. The PubPeer site and API refused automated queries (HTTP 403, Cloudflare), so a thread there cannot be ruled out.
- Sources: PMC6217433 (
src/show.py);src/known.py 10.1073/pnas.1802592115; https://doi.org/10.1073/pnas.1802592115; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC6217433/fullTextXML; https://www.ebi.ac.uk/europepmc/webservices/rest/PMC6217433/supplementaryFiles; PubMed PMID 30322939.
Suggested correction
In the Abstract, replace
Changes in the spectral composition of white light toward enhanced UV or visible wavelengths respectively increased vs. decreased calcium signal in parapinopsin-sufficient but not parapinopsin-deficient photoreceptors.
with
Changes in the spectral composition of white light toward enhanced UV or visible wavelengths respectively decreased vs. increased calcium signal in parapinopsin-sufficient but not parapinopsin-deficient photoreceptors.
This makes the abstract agree with the Results, the Fig. 4 legend, the plotted Fig. 4B data and the paper's proposed mechanism. No other part of the paper needs to change.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC6217433 (Proc Natl Acad Sci U S A 2018, doi:10.1073/pnas.1802592115)
# Abstract vs Results / Fig. 4B: direction of the calcium response to UV- and visible-enriched white light
#
# Run: python recompute.py.txt
# Prints every number and direction used in writeup.md and regenerates, in the folder that
# holds this file:
# fig1.svg / fig1.png two text panels: Abstract vs Results + Fig. 4 legend
# fig2.svg / fig2.png direction of the calcium change in every passage that states one
# candidates/candidate_{1..5}.svg/.png five candidate figure designs (1 = fig1, 2 = fig2), for selection
# Needs matplotlib (numpy only for the arithmetic).
#
# Inputs
# * Quotes: copied verbatim from the article as archived in PubMed Central (PMC6217433).
# When run inside the /home/ubuntu/papers repo, the script re-checks each quote against the
# archived full text with src/show.py --grep (skipped elsewhere).
# * Light intensities: Fig. 4 legend (A).
# * Fig. 4B plotted traces: the direction of the WT and PT-/- deflections was read by eye from
# the published figure image (pnas.1802592115fig04.jpg in the PMC / Europe PMC package).
# No values are measured from it and the image is not reproduced. The later course of the
# traces (rebound above baseline after the UV-mixed dip) was read the same way.
import os, re, subprocess, sys, textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": TXT,
"ytick.color": TXT, "svg.fonttype": "none"})
# ------------------------------------------------------------------ quotes
Q = {
"abstract": ("Abstract",
"Changes in the spectral composition of white light toward enhanced UV or visible wavelengths "
"respectively increased vs. decreased calcium signal in parapinopsin-sufficient but not "
"parapinopsin-deficient photoreceptors."),
"results4b": ("Results, 'Color Opponency Based on Photoequilibrium of Parapinopsin Two States', white-light paragraph",
"We found decreased and increased calcium levels following exposure to UV and visible "
"light-mixed white light, respectively, in WT and PT-/- but not in PP-/- and PP-/-/PT-/- (Fig. 4B)."),
"legend4b": ("Fig. 4 legend, panel B",
"Note that the calcium levels decreased and increased transiently in response to exposure to "
"UV and visible light-mixed white lights, respectively, in WT and PT-/-."),
"fig1c_405": ("Results, 'Antagonistic Chromatic Responses in a Single Pineal Photoreceptor Cell'",
"stimulation with 405-nm light caused a decrease in fluorescence intensity."),
"fig1c_588": ("Results, 'Antagonistic Chromatic Responses in a Single Pineal Photoreceptor Cell'",
"In contrast, stimulation with 588-nm light elevated the fluorescence intensity, demonstrating "
"the antagonistic effect of UV and visible light on the calcium in individual PP cells to UV "
"and visible light (Fig. 1C)."),
"mech_405": ("Results, mechanism paragraph (Fig. 3)",
"The 405-nm light stimulus increases the short-wavelength component of incident light and thus "
"increases the fractional concentration of the signaling active photoproduct from that produced "
"by the imaging light alone, leading to hyperpolarization (Fig. 3 B and D, calcium decrease)."),
"mech_588": ("Results, mechanism paragraph (Fig. 3)",
"Conversely, because the photoproduct is more sensitive to longer wavelengths, the 588-nm light "
"stimulus decreases its fractional concentration, reducing the pool of signaling-active opsin, "
"resulting in termination of hyperpolarization and increased calcium levels (Fig. 3 C and D, "
"calcium increase)."),
"discussion": ("Discussion, first paragraph",
"with lights biased toward UV driving greater hyperpolarization than those biased toward visible light."),
"intro": ("Introduction, second paragraph",
"with firing of these neurons suppressed by UV and enhanced by visible light"),
"legend4a": ("Fig. 4 legend, panel A",
"The light intensities (360–780 nm) of white light, UV-mixed white light, visible light-mixed "
"white light, and direct sunlight were ∼9.2 × 1014, ∼9.3 × 1014, ∼9.9 × 1014, and 1.1 × 1017 "
"photons per cm2·s. The light intensities of UV and visible LED lights were ∼1.3 × 1013 and "
"∼7.6 × 1013 photons per cm2·s, respectively."),
"legend4b_n": ("Fig. 4 legend, panel B",
"Changes in calcium level of PP cells in WT (red open circle; n = 24), PT-/- fish (green closed "
"circle; n = 23), PP-/- fish (purple open triangle; n = 17), and PP-/-/PT-/- fish (black closed "
"triangle; n = 16)"),
}
def norm(s):
return re.sub(r"\s+", " ", s).strip()
def check_quotes():
repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
show = os.path.join(repo, "src", "show.py")
if not os.path.exists(show):
print("[quote check skipped: src/show.py not found]")
return
py = "/home/ubuntu/papers-data/venv/bin/python"
py = py if os.path.exists(py) else sys.executable
print("=== Verbatim quote check against the PMC archive (src/show.py --grep) ===")
for k, (loc, q) in Q.items():
probe = q[:60]
out = subprocess.run([py, show, "PMC6217433", "--grep", probe], cwd=repo,
capture_output=True, text=True).stdout
ok = norm(q) in norm(out)
print(f" {'OK ' if ok else 'MISS'} {k:11s} ({loc})")
check_quotes()
# ------------------------------------------------------------------ direction mapping
# 'respectively' pairs the i-th stimulus with the i-th direction word; both orders are
# extracted from the quoted sentences themselves.
def dirword(w):
w = w.lower()
return "increase" if w.startswith("increas") else "decrease"
m = re.search(r"toward enhanced (\w+) or (\w+) wavelengths respectively (\w+) vs\. (\w+) calcium",
Q["abstract"][1])
ABS = {m.group(1): dirword(m.group(3)), m.group(2): dirword(m.group(4))}
m = re.search(r"found (\w+) and (\w+) calcium levels following exposure to (\w+) and (\w+) light-mixed",
Q["results4b"][1])
RES = {m.group(3): dirword(m.group(1)), m.group(4): dirword(m.group(2))}
m = re.search(r"calcium levels (\w+) and (\w+) transiently in response to exposure to (\w+) and (\w+) light-mixed",
Q["legend4b"][1])
LEG = {m.group(3): dirword(m.group(1)), m.group(4): dirword(m.group(2))}
M405 = "decrease" if "caused a decrease" in Q["fig1c_405"][1] else "?"
M588 = "increase" if "588-nm light elevated" in Q["fig1c_588"][1] else "?"
MECH405 = "decrease" if "calcium decrease" in Q["mech_405"][1] else "?"
MECH588 = "increase" if "calcium increase" in Q["mech_588"][1] else "?"
DISC = ("lower", "higher") if "biased toward UV driving greater hyperpolarization" in Q["discussion"][1] else ("?", "?")
FIG4B_IMAGE = ("decrease", "increase") # transient deflection at each bar, read by eye from the published Fig. 4B image (WT, PT-/-)
FIG4B_LATER = { # later course of the WT and PT-/- traces, read by eye from the same image
"after UV-mixed dip": "rebound above the pre-stimulus baseline, elevated until the visible-mixed bar",
"after visible-mixed peak": "back to about the level just before the visible-mixed bar; no decrease follows",
}
print("\n=== Direction of the calcium change stated for each stimulus ===")
print(f" Abstract UV-enhanced: {ABS['UV']:8s} visible-enhanced: {ABS['visible']}")
print(f" Results, Fig. 4B paragraph UV-mixed: {RES['UV']:8s} visible-mixed: {RES['visible']}")
print(f" Fig. 4 legend (B) UV-mixed: {LEG['UV']:8s} visible-mixed: {LEG['visible']}")
print(f" Fig. 4B traces (published image) UV-mixed: {FIG4B_IMAGE[0]:8s} visible-mixed: {FIG4B_IMAGE[1]}")
print(f" Results, Fig. 1C (two-photon) 405 nm: {M405:8s} 588 nm: {M588}")
print(f" Results, mechanism (Fig. 3) 405 nm: {MECH405:8s} 588 nm: {MECH588}")
print(f" Discussion (relative) UV-biased: {DISC[0]:8s} visible-biased: {DISC[1]} (calcium, via 'greater hyperpolarization')")
for k in ("UV", "visible"):
print(f" {k:8s} abstract == Results? {ABS[k] == RES[k]} Results == legend? {RES[k] == LEG[k]}")
body = [(RES["UV"], RES["visible"]), (LEG["UV"], LEG["visible"]), FIG4B_IMAGE, (M405, M588), (MECH405, MECH588)]
n_agree = sum(1 for b in body if b == ("decrease", "increase"))
print(f" Body statements with UV -> decrease and visible -> increase: {n_agree} of {len(body)}"
f" (plus the Discussion's relative statement)")
print(f" Abstract matches the body: {(ABS['UV'], ABS['visible']) == ('decrease', 'increase')}")
print(" Fig. 4B, later course of the WT and PT-/- traces (published image, by eye):")
for k, v in FIG4B_LATER.items():
print(f" {k:25s} {v}")
print(" -> the UV half of the abstract (UV -> increased) has a later counterpart (the rebound);")
print(" the visible half (visible -> decreased) has none, so the abstract's full mapping is not shown.")
# ------------------------------------------------------------------ light intensities (Fig. 4 legend A)
print("\n=== Fig. 4A light intensities (photons per cm2·s, 360-780 nm) ===")
white, uv_mix, vis_mix = 9.2e14, 9.3e14, 9.9e14
uv_led, vis_led = 1.3e13, 7.6e13
for lab, led, rep in (("UV-mixed", uv_led, uv_mix), ("visible-mixed", vis_led, vis_mix)):
tot = white + led
print(f" {lab:14s} white + LED = {tot:.3g} (reported ~{rep:.2g}); LED adds {led / white:.1%} to the white light")
# rounding check: each '~' value is taken as exact to its last printed digit (half a unit either way)
def rng(v, step):
return v - step / 2, v + step / 2
w_lo, w_hi = rng(white, 0.1e14)
for lab, led, rep in (("UV-mixed", uv_led, uv_mix), ("visible-mixed", vis_led, vis_mix)):
l_lo, l_hi = rng(led, 0.1e13)
r_lo, r_hi = rng(rep, 0.1e14)
lo, hi = w_lo + l_lo, w_hi + l_hi
print(f" {lab:14s} sum within rounding: {lo:.3e}-{hi:.3e}; reported ~{rep:.2g} covers {r_lo:.3e}-{r_hi:.3e};"
f" overlap: {lo <= r_hi and r_lo <= hi}")
print(" Both mixed lights add photons to the same white background (total intensity rises in both),")
print(" so the two stimuli differ in spectral composition, not in the sign of the intensity change.")
print("\n=== Fig. 4B sample sizes (legend; not affected by the direction issue) ===")
print(" WT 24, PT-/- 23, PP-/- 17, PP-/-/PT-/- 16; total", 24 + 23 + 17 + 16)
# ------------------------------------------------------------------ figure helpers
def newfig(h):
fig = plt.figure(figsize=(7.2, h), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 7.2); ax.set_ylim(0, h); ax.axis("off"); ax.set_facecolor(BG)
return fig, ax
def save(fig, *names):
for n in names:
fig.savefig(n + ".svg", facecolor=BG)
fig.savefig(n + ".png", dpi=200, facecolor=BG)
plt.close(fig)
def panel(ax, x, y, w, h, color, title, sub, body, width_chars, fs=10.5):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.08",
fc="white", ec=color, lw=2))
ax.add_patch(FancyBboxPatch((x, y + h - 0.08), w, 0.08, boxstyle="square,pad=0", fc=color, ec=color))
ax.text(x + 0.15, y + h - 0.27, title, fontsize=12, fontweight="bold", color=color, va="top")
ax.text(x + 0.15, y + h - 0.55, sub, fontsize=10, color=SEC, va="top")
ax.text(x + 0.15, y + h - 0.85, textwrap.fill(body, width_chars), fontsize=fs, va="top",
linespacing=1.35)
# ------------------------------------------------------------------ candidate 1 / fig1: two text panels
def cand1():
fig, ax = newfig(4.4)
ax.text(0.15, 4.25, "Abstract and Results give opposite directions for the Fig. 4B experiment",
fontsize=11.5, fontweight="bold", va="top")
panel(ax, 0.15, 0.95, 3.25, 3.0, ORANGE, "Abstract", "as published",
"“…white light toward enhanced UV or visible wavelengths respectively "
"increased vs. decreased calcium signal in parapinopsin-sufficient … photoreceptors.”", 36)
panel(ax, 3.8, 0.95, 3.25, 3.0, BLUE, "Results + Fig. 4 legend", "what the paper's body reports",
"“We found decreased and increased calcium levels following exposure to UV and visible "
"light-mixed white light, respectively, in WT and PT-/- … (Fig. 4B).”", 36)
# mapping rows inside panels
for x0, col, uvd, visd in ((0.3, ORANGE, ABS["UV"], ABS["visible"]), (3.95, BLUE, RES["UV"], RES["visible"])):
for yy, lab, dd in ((1.55, "UV-enriched", uvd), (1.2, "visible-enriched", visd)):
ax.text(x0, yy, lab, fontsize=10, color=col, fontweight="bold")
ax.text(x0 + 1.34, yy, f"→ calcium {dd}d", fontsize=10, color=col, fontweight="bold")
ax.add_patch(FancyArrowPatch((3.42, 1.48), (3.78, 1.48), arrowstyle="<->", mutation_scale=14,
color=SEC, lw=1.5))
ax.text(3.6, 0.72, "Conflict: both directions are reversed.", ha="center", fontsize=10.5,
fontweight="bold", color=TXT)
ax.text(3.6, 0.42, "The transient Fig. 4B responses, the 405/588-nm experiments (Fig. 1C) and the mechanism (Fig. 3)",
ha="center", fontsize=10, color=SEC)
ax.text(3.6, 0.17, "agree with the Results: UV-enriched light lowers calcium, visible-enriched light raises it.",
ha="center", fontsize=10, color=SEC)
return fig
save(cand1(), os.path.join(HERE, "fig1"), os.path.join(CAND, "candidate_1"))
# ------------------------------------------------------------------ candidate 2 / fig2: direction grid
ROWS = [
("Abstract", "enhanced UV / enhanced visible white light", ABS["UV"], ABS["visible"], ORANGE),
("Results, Fig. 4B paragraph", "UV- / visible-mixed white light", RES["UV"], RES["visible"], BLUE),
("Fig. 4 legend, panel B", "UV- / visible-mixed white light", LEG["UV"], LEG["visible"], BLUE),
("Fig. 4B traces (WT, PT-/-)", "transient, from the published image", FIG4B_IMAGE[0], FIG4B_IMAGE[1], BLUE),
("Results, Fig. 1C", "405-nm / 588-nm stimulus, two-photon", M405, M588, BLUE),
("Results, mechanism (Fig. 3)", "405-nm / 588-nm stimulus", MECH405, MECH588, BLUE),
("Discussion", "UV-biased / visible-biased light", "lower*", "higher*", BLUE),
]
SYM = {"decrease": "▼ decrease", "increase": "▲ increase", "lower*": "▼ lower*", "higher*": "▲ higher*"}
def cand2():
h = 0.95 + 0.48 * len(ROWS) + 0.75
fig, ax = newfig(h)
top = h - 0.25
ax.text(0.15, top, "Direction of the calcium change in every passage that states one", fontsize=11.5,
fontweight="bold", va="top")
ax.text(3.95, top - 0.5, "UV-shifted light", ha="center", fontsize=10.5, fontweight="bold")
ax.text(5.9, top - 0.5, "visible-shifted light", ha="center", fontsize=10.5, fontweight="bold")
y = top - 0.95
for i, (src, stim, d1, d2, col) in enumerate(ROWS):
if i == 0:
ax.add_patch(FancyBboxPatch((0.1, y - 0.2), 7.0, 0.44, boxstyle="round,pad=0.01,rounding_size=0.05",
fc="#fdf0ea", ec=ORANGE, lw=2))
else:
ax.plot([0.15, 7.05], [y + 0.24, y + 0.24], color=GRID, lw=1)
ax.text(0.2, y + 0.05, src, fontsize=10.5, fontweight="bold" if i == 0 else "normal", va="center", color=TXT)
ax.text(0.2, y - 0.13, stim, fontsize=10, color=SEC, va="center")
ax.text(3.95, y, SYM[d1], ha="center", va="center", fontsize=11, color=col, fontweight="bold")
ax.text(5.9, y, SYM[d2], ha="center", va="center", fontsize=11, color=col, fontweight="bold")
y -= 0.48
ax.text(0.15, y + 0.05, "Orange: the abstract as published. Blue: the paper's own body (text, legend, figure, mechanism).",
fontsize=10, color=SEC, va="top")
ax.text(0.15, y - 0.22, "* A comparison: UV-biased light drives “greater hyperpolarization”, i.e. lower calcium.",
fontsize=10, color=SEC, va="top")
return fig
save(cand2(), os.path.join(HERE, "fig2"), os.path.join(CAND, "candidate_2"))
# ------------------------------------------------------------------ candidate 3: sign dot plot on one axis
def cand3():
fig = plt.figure(figsize=(7.2, 3.9), facecolor=BG)
ax = fig.add_axes([0.14, 0.2, 0.52, 0.66]); ax.set_facecolor(BG)
sign = {"increase": 1, "decrease": -1}
body_src = [("Results text", RES), ("Fig. 4 legend", LEG),
("Fig. 4B traces", {"UV": FIG4B_IMAGE[0], "visible": FIG4B_IMAGE[1]}),
("Fig. 1C 405/588 nm", {"UV": M405, "visible": M588}),
("Fig. 3 mechanism", {"UV": MECH405, "visible": MECH588})]
for xi, k in enumerate(("UV", "visible")):
ax.plot(xi - 0.18, sign[ABS[k]], "o", ms=11, color=ORANGE, zorder=3)
for j, (_, d) in enumerate(body_src):
ax.plot(xi + 0.04 + 0.11 * j, sign[d[k]], "s", ms=8, color=BLUE, zorder=3)
ax.axhline(0, color=SEC, lw=0.8)
ax.set_xlim(-0.55, 1.85); ax.set_ylim(-1.6, 1.6)
ax.set_xticks([0.1, 1.1]); ax.set_xticklabels(["UV-shifted light", "visible-shifted light"])
ax.set_yticks([-1, 0, 1]); ax.set_yticklabels(["calcium\ndecrease", "no change", "calcium\nincrease"])
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.text(0.02, 0.95, "Sign of the calcium response: abstract vs the paper's body", fontsize=11.5, fontweight="bold")
fig.text(0.69, 0.74, "● Abstract, as published", color=ORANGE, fontsize=10.5, fontweight="bold")
fig.text(0.69, 0.66, "■ Paper's body,\n 5 passages:", color=BLUE, fontsize=10.5,
fontweight="bold", va="top")
fig.text(0.71, 0.53, "\n".join(n for n, _ in body_src), color=SEC, fontsize=10, va="top", linespacing=1.4)
fig.text(0.02, 0.04, "Only the sign is plotted; the paper states directions, not a single summary value.",
fontsize=10, color=SEC)
return fig
save(cand3(), os.path.join(CAND, "candidate_3"))
# ------------------------------------------------------------------ candidate 4: 'respectively' pairing
def token(ax, x, y, s, col, w=1.55):
ax.add_patch(FancyBboxPatch((x - w / 2, y - 0.16), w, 0.32, boxstyle="round,pad=0.02,rounding_size=0.06",
fc="white", ec=col, lw=2))
ax.text(x, y, s, ha="center", va="center", fontsize=10.5, color=TXT)
def cand4():
fig, ax = newfig(3.9)
ax.text(0.15, 3.75, "How “respectively” pairs each light with a direction", fontsize=11.5,
fontweight="bold", va="top")
for x0, col, title, stims, verbs, mp in (
(0.25, ORANGE, "Abstract", ("enhanced UV", "enhanced visible"), ("increased", "decreased"), ABS),
(3.85, BLUE, "Results (Fig. 4B)", ("UV-mixed", "visible-mixed"), ("decreased", "increased"), RES)):
ax.text(x0, 3.25, title, fontsize=12, fontweight="bold", color=col)
ax.text(x0, 2.98, "word order in the sentence", fontsize=10, color=SEC)
for i in range(2):
xs = x0 + 0.85 + i * 1.65
token(ax, xs, 2.5, stims[i], col)
token(ax, xs, 1.55, verbs[i], col)
ax.add_patch(FancyArrowPatch((xs, 2.32), (xs, 1.73), arrowstyle="-|>", mutation_scale=12, color=col, lw=2))
ax.text(xs + 0.08, 2.02, f"{i + 1}st" if i == 0 else "2nd", fontsize=10, color=SEC)
ax.text(x0, 1.0, f"UV → calcium {mp['UV']}", fontsize=10.5, color=col, fontweight="bold")
ax.text(x0, 0.72, f"visible → calcium {mp['visible']}", fontsize=10.5, color=col, fontweight="bold")
ax.plot([3.6, 3.6], [0.5, 3.4], color=GRID, lw=1)
ax.text(3.6, 0.2, "Same pairing rule, opposite verb order: the two sentences assign opposite directions to each light.",
ha="center", fontsize=10, color=SEC)
return fig
save(cand4(), os.path.join(CAND, "candidate_4"))
# ------------------------------------------------------------------ candidate 5: mechanism chain
def box(ax, x, y, s, col, w=1.45, h=0.62, fc="white"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.02,rounding_size=0.06", fc=fc, ec=col, lw=2))
ax.text(x + w / 2, y + h / 2, s, ha="center", va="center", fontsize=10, color=TXT, linespacing=1.25)
def cand5():
fig, ax = newfig(4.3)
ax.text(0.15, 4.15, "The paper's mechanism predicts the Results' direction, not the abstract's",
fontsize=11.5, fontweight="bold", va="top")
ax.text(0.15, 3.82, "Chain from the Results (mechanism paragraph, Fig. 3, 405/588 nm), generalized in the Discussion", fontsize=10,
color=SEC, va="top")
rows = ((2.75, "UV-shifted\nwhite light", "more active\nphotoproduct", "hyperpolarization", "calcium\ndecreases",
f"calcium\n{ABS['UV']}d"),
(1.45, "visible-shifted\nwhite light", "less active\nphotoproduct", "hyperpolarization\nends",
"calcium\nincreases", f"calcium\n{ABS['visible']}d"))
for y, a, b, c, d, e in rows:
xs = [0.15, 1.6, 3.05, 4.5]
for x, s in zip(xs, (a, b, c, d)):
box(ax, x, y, s, BLUE if x == 4.5 else SEC, w=1.3, fc="#eef4fc" if x == 4.5 else "white")
for x in xs[:-1]:
ax.add_patch(FancyArrowPatch((x + 1.32, y + 0.31), (x + 1.44, y + 0.31), arrowstyle="-|>",
mutation_scale=12, color=SEC, lw=1.5))
box(ax, 6.1, y, e, ORANGE, w=0.98, fc="#fdf0ea")
ax.text(5.95, y + 0.31, "≠", ha="center", va="center", fontsize=16, color=TXT, fontweight="bold")
ax.text(5.15, 3.5, "Results / Fig. 4B", ha="center", fontsize=10, color=BLUE, fontweight="bold")
ax.text(6.59, 3.5, "Abstract", ha="center", fontsize=10, color=ORANGE, fontweight="bold")
ax.text(0.15, 1.0, "Blue: the outcome the paper's body reports, which follows from its mechanism.", fontsize=10, color=SEC)
ax.text(0.15, 0.72, "Orange: the outcome the abstract states, which reverses it for both lights.", fontsize=10, color=SEC)
ax.text(0.15, 0.44, "Same two lights, same calcium indicator (GCaMP6s), same parapinopsin-sufficient cells (WT, PT-/-).",
fontsize=10, color=SEC)
return fig
save(cand5(), os.path.join(CAND, "candidate_5"))
print("\nFigures written to", HERE, "and", CAND)
Reported P < 0.0001 for a light-versus-dark coiling difference is inconsistent with the paper's counts
Sci Adv (2020) · doi:10.1126/sciadv.abb8930 · PMC10662364
Summary
The Results report that offspring of a single N. pachyderma parent coiled sinistrally more often in the 12-hour light flask than in the 24-hour dark flask: 87.5% (n = 9) against 50% (n = 12), with P < 0.0001 by a proportion z test. The counts that fit the paper's own percentages and totals are 7 of 8 and 6 of 12. They add up to the stated 13 of 20 sinistral offspring, and no count out of 9 gives 87.5%. A two-proportion z test on these counts gives z = 1.72 and two-sided P = 0.085, and Fisher's exact test gives P = 0.158. None of the standard variants I tried comes close to 0.0001. The Discussion relies on this contrast when it suggests that light conditions, or the pH and O2 changes that come with them, could contribute to coiling direction. That inference loses its statistical support, but the paper's main observations of asexual reproduction and morphological variability among clones do not depend on it.
What the paper reports
Results, "Morphology of parent and offspring", second paragraph:
Coiling direction was ascertained in 20 offspring, 13 of which coiled sinistrally (65%). There were differences in coiling direction proportion between the 12-hour light and 24-hour dark treatments (see Materials and Methods): 87.5% were sinistral (n = 9) in the 12-hour light treatment (flask “O”), and 50% (n = 12) in the 24-hour dark treatment (flask “D”) (P < 0.0001, by a proportion z test).
Materials and Methods, third paragraph:
No shells we recovered from flask “A,” eight offspring and the parent were recovered from flask “O,” and 17 offspring from flask “D.”
Discussion, "Phenotypic variation in a genetically similar population", second paragraph:
The relative abundance of shells coiling in the nondominant direction was greater in the 24-hour dark treatment (50%) than in the 12-hour light treatment (12.5%), indicating that factors such as light conditions or associated changes in pH and O2 could contribute to coiling direction.
What the paper's own numbers show
Method. I first found the whole-number counts consistent with each printed percentage, its printed n, the Methods recovery counts and the stated total of 13 sinistral out of 20. I then ran the two-group test the paper names (a two-proportion z test with pooled standard error, two-sided) on those counts, along with the common variants: unpooled standard error, continuity correction, one-sided, Fisher's exact test, Boschloo's exact test, and the same tests with the sinistral parent counted as a ninth shell in flask O. Last, I searched for any calculation that yields P < 0.0001. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Flask O, sinistral | Results | 87.5% (n = 9) | No count out of 9 gives 87.5% (87.5% × 9 = 7.875; 7/9 = 77.8%, 8/9 = 88.9%). 7/8 = 87.5%, and Methods report 8 offspring (plus the parent) from flask O |
| Flask D, sinistral | Results | 50% (n = 12) | 6/12 = 50%; 12 of the 17 offspring recovered from flask D |
| All offspring scored | Results | 13 of 20 sinistral (65%) | 7 + 6 = 13, 8 + 12 = 20, 13/20 = 65%. With n = 9 the total would be 21 |
| Nondominant (dextral) coiling | Discussion | 12.5% vs 50% | 1/8 = 12.5%, 6/12 = 50%: the same contrast as above |
| Pooled two-proportion z test, 7/8 vs 6/12 | Results | P < 0.0001 | pooled p = 0.65, SE = 0.2177, z = 1.723, two-sided P = 0.085 (one-sided 0.043) |
| Same, continuity corrected | Results | P < 0.0001 | z = 1.244, P = 0.213 (equal to chi-square with Yates correction) |
| Unpooled (Wald) z test | Results | P < 0.0001 | z = 2.019, P = 0.044 |
| Fisher's exact test | Results | P < 0.0001 | P = 0.158 (odds ratio 7.0) |
| Boschloo's exact test | Results | P < 0.0001 | P = 0.119 |
| Parent counted, 8/9 vs 6/12 | Results | P < 0.0001 | pooled z = 1.871, P = 0.061; unpooled z = 2.181, P = 0.029; Fisher P = 0.159 |
| 95% intervals | (derived) | — | Wilson: flask O 52.9% to 97.8%, flask D 25.4% to 74.6%; Newcombe for the 37.5-point difference: −5.0 to 64.2 points, which includes 0 |
| Equal group size needed for P < 0.0001 at 87.5% vs 50% | (derived) | n = 9 and n = 12 | 47 per group (pooled z test) |
| One-sample z test, flask D against flask O's estimate | (candidate explanation) | P < 0.0001 | 6/12 against a fixed 0.875: z = −3.93, P = 8.6e-5 (exact binomial P = 0.0018), but this is not a two-group comparison |
| Percentages entered as counts out of 100 | (candidate explanation) | P < 0.0001 | 87.5/100 vs 50/100: z = 5.72, P = 1.1e-8, but this is not a valid test of these data |
Figure
Figure 1. Two-sided P values for the light-versus-dark comparison of sinistral coiling, on a logarithmic axis. The orange bar and arrow mark the paper's reported value, P < 0.0001, which is an upper bound. The filled blue dots are P values recomputed from the counts that fit the paper's percentages and totals (7 of 8 versus 6 of 12, or 8 of 9 if the sinistral parent is counted). They range from 0.044 to 0.213 and lie near or above the dashed 0.05 line, more than 400 times the reported bound. The two hollow blue dots are the only calculations found that give a value below 0.0001: a one-sample z test of flask D's 6 of 12 against flask O's 87.5% treated as a fixed value (P = 8.6e-5), and the two percentages treated as counts out of 100 (P = 1.1e-8). They are shown only for comparison; neither is a valid two-group comparison of these data. All values come from recompute.py.txt.
Why it matters
The P value attaches to a secondary comparison, and the direction of the observed difference stays as reported: 1 of 8 dextral shells in the light flask against 6 of 12 in the dark flask. What changes is the strength of the evidence. As printed, P < 0.0001 tells the reader that a difference this large is very unlikely to arise by chance between two groups that coil alike. With 8 and 12 shells, the correct pooled z test gives P = 0.085, and the 95% interval for the difference runs from about −5 to 64 percentage points. The data are compatible with no effect of the treatment on coiling.
The Discussion uses this contrast to suggest that "light conditions or associated changes in pH and O2 could contribute to coiling direction". That is presented as a possibility, but readers may take the printed P < 0.0001 as strong support for it. The paper notes that the two species are "generally distinguished by shell coiling direction", so a claimed environmental effect on coiling bears on how field specimens are assigned to species. The comparison involves one parent's clonal offspring in two flasks, so even a significant result would not separate light from any other difference between the flasks, and the recomputed P values (0.03 to 0.21, two-sided) leave it without strong statistical support.
The paper's headline findings do not rest on this test: that the parent reproduced asexually, that its clonal offspring spanned four morphotypes, and that offspring coiled in both directions (13 of 20 sinistral). The title and abstract do not report the light-versus-dark P value, and the paper has no tables.
Explanations considered
- Typing error in n. The printed n = 9 is impossible for 87.5% and inconsistent with the total of 20. Methods report "eight offspring and the parent" from flask O, so n = 9 may count the parent while 87.5% uses the 8 offspring. This explains the denominator but not the P value: with either 8 or 9 shells, the tests give P between 0.029 and 0.213.
- A different variant of the z test. Pooled or unpooled standard error, with or without continuity correction, one- or two-sided: the smallest P across all these, with either denominator, is 0.0146 (one-sided, unpooled, parent counted). None reaches 0.0001.
- A different two-group test. Fisher's exact test (P = 0.158), Boschloo's exact test (P = 0.119) and chi-square with Yates correction (P = 0.213) give larger P values than the z test. Chi-square without correction is the same test as the pooled z test and gives the same P = 0.085.
- More shells than the text states. P < 0.0001 at these percentages would need about 47 shells per group. The Results say coiling was ascertained in 20 offspring, and Methods report only 8 offspring from flask O, so the scored samples cannot be that large.
- Comparison against a fixed reference value. Testing flask D's 6 of 12 against a fixed proportion of 0.875 with a one-sample z test gives z = −3.93 and P = 8.6e-5, which would be reported as P < 0.0001 and fits the wording "by a proportion z test". That treats flask O's estimate, based on 8 shells, as a known constant; the exact binomial version gives P = 0.0018. Testing flask O's 7 of 8 against 0.5 gives one-sample z test P = 0.034 (exact binomial P = 0.070). Neither is the two-group comparison the text describes.
- Comparison with the population rate of nondominant coiling. The Results state that natural populations contain 1 to 3% nondominant coilers. Against 3%, flask D's 6 of 12 (one-sided exact P = 5.8e-7) and all offspring's 7 of 20 (P = 1.2e-6) are significant, but flask O's 1 of 8 is not (P = 0.22). This is a different question from the one the sentence reports, which is the difference between treatments.
- Percentages entered as counts. A two-proportion z test on 87.5 of 100 versus 50 of 100 gives z = 5.72 and P = 1.1e-8, which would be reported as P < 0.0001. It is not a valid test, because the sample sizes are 8 and 12, not 100.
- Which calculation, if any. Two calculations found give P < 0.0001: the one-sample z test of flask D against flask O's estimate (P = 8.6e-5; exact binomial P = 0.0018) and the percentages read as counts out of 100 (P = 1.1e-8). Neither is a valid comparison of the two flasks, and no two-group test on the paper's counts comes close. The source of the printed value cannot be established from the paper.
- Conversion or typesetting artifact. Ruled out (see Status): the publisher's typeset PDF, the Europe PMC XML and the PMC text all print the same digits, with a real less-than sign and no superscripts or minus signs.
Status
As of 2026-10-23:
- Version of record. The doi.org and science.org pages were not reachable by script (HTTP 403), so the text was checked in the publisher's typeset PDF as mirrored on NSF PAR (par.nsf.gov/servlets/purl/10197656; 8 pages, header "Sci. Adv. 2020; 6 : eabb8930 10 July 2020", published 10 July 2020), in the Europe PMC full-text XML, and in the PMC archived text. All three carry the same sentences and numbers quoted above.
- Supplement. The Europe PMC supplementary archive holds abb8930_SM.pdf (captions for Movies S1 to S3 only) and three movie files. It contains no counts per flask and no statistics. The data statement says additional data may be requested from the authors. The supplement does not resolve the discrepancy.
- Corrections and notices. No correction, erratum, expression of concern or retraction was found in Crossref (no update-to, updated-by or relation entries), Crossref Labs (which carries Retraction Watch data), PubMed Central or Europe PMC.
- PubPeer. The PubPeer publication lookup for the DOI returned no comments, and a web search found no PubPeer thread.
Suggested correction
In Results, "Morphology of parent and offspring", second paragraph, give the flask O denominator that matches 87.5% (n = 8 offspring), or, if n = 9 includes the parent, state that and give the matching percentage (88.9%). Replace "P < 0.0001, by a proportion z test" with the value for the counts actually compared, for example "7 of 8 versus 6 of 12; P = 0.085, two-sided two-proportion z test (Fisher's exact test, P = 0.16)", and state that the difference is not statistically significant. In the Discussion, "Phenotypic variation in a genetically similar population", second paragraph, the sentence on light conditions and coiling direction could be reworded to present the 12.5% versus 50% contrast as a small-sample observation from two flasks that was not statistically significant.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- re-derivation for PMC10662364 (Sci Adv 2020, doi:10.1126/sciadv.abb8930)
# Run: python recompute.py.txt
# Prints every recomputed number used in writeup.md and regenerates fig1.svg/.png
# plus five candidate figure designs (candidates/fig1_A..E) in this script's folder.
#
# Inputs, all taken from the article text:
# Results: "Coiling direction was ascertained in 20 offspring, 13 of which coiled sinistrally (65%)."
# Results: "87.5% were sinistral (n = 9) in the 12-hour light treatment (flask "O"), and 50% (n = 12)
# in the 24-hour dark treatment (flask "D") (P < 0.0001, by a proportion z test)."
# Methods: "eight offspring and the parent were recovered from flask "O," and 17 offspring from flask "D.""
# Discussion: nondominant coiling 50% (dark) vs 12.5% (light).
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- 1. counts
print("== 1. Which counts fit the printed percentages")
for k in range(0, 10):
print(f" {k}/9 = {k/9:.4f}", "<- 87.5%?" if abs(k/9 - 0.875) < 1e-9 else "")
print(f" 87.5% of 9 = {0.875*9:.3f} (not a whole number)")
print(f" 7/8 = {7/8:.4f}; 8/9 = {8/9:.4f}; 7/9 = {7/9:.4f}; 6/12 = {6/12:.4f}")
print(f" offspring total 8 + 12 = {8+12}; sinistral 7 + 6 = {7+6}; 13/20 = {13/20:.4f} (paper: 13 of 20, 65%)")
print(f" with n = 9 the total would be 9 + 12 = {9+12}, not 20")
print(f" nondominant (dextral): 1/8 = {1/8:.4f}, 6/12 = {6/12:.4f} (paper: 12.5% and 50%)")
print(f" flask D: 12 scored for coiling of 17 offspring recovered (Methods)")
xO, nO, xD, nD = 7, 8, 6, 12
# ---------------------------------------------------------------- 2. tests
def z2(x1, n1, x2, n2, pooled=True, cc=False):
p1, p2 = x1 / n1, x2 / n2
if pooled:
p = (x1 + x2) / (n1 + n2)
se = np.sqrt(p * (1 - p) * (1 / n1 + 1 / n2))
else:
se = np.sqrt(p1 * (1 - p1) / n1 + p2 * (1 - p2) / n2)
d = abs(p1 - p2)
if cc:
d = max(0.0, d - 0.5 * (1 / n1 + 1 / n2))
z = d / se
return z, 2 * stats.norm.sf(z), se
print("\n== 2. Two-group tests, flask O vs flask D")
p_pool = (xO + xD) / (nO + nD)
z, p, se = z2(xO, nO, xD, nD)
print(f" pooled p = {p_pool:.3f}; SE = sqrt({p_pool:.2f}*{1-p_pool:.2f}*(1/8+1/12)) = {se:.4f}")
print(f" difference = {7/8 - 6/12:.3f}")
R = {}
for lab, (a, b, c, d) in {"7/8 vs 6/12": (7, 8, 6, 12), "8/9 vs 6/12 (parent counted)": (8, 9, 6, 12)}.items():
for pooled in (True, False):
for cc in (False, True):
z, p, _ = z2(a, b, c, d, pooled, cc)
R[(lab, pooled, cc)] = (z, p)
print(f" {lab:30s} {'pooled ' if pooled else 'unpooled'} {'Yates' if cc else 'no cc':5s}"
f" z = {z:.3f} two-sided P = {p:.4f} one-sided P = {p/2:.4f}")
for a, b, c, d in [(7, 8, 6, 12), (8, 9, 6, 12)]:
tab = [[a, b - a], [c, d - c]]
fe = stats.fisher_exact(tab)
chy = stats.chi2_contingency(tab, correction=True)
chn = stats.chi2_contingency(tab, correction=False)
bo = stats.boschloo_exact(tab, alternative="two-sided")
R[("fisher", a, b)] = fe.pvalue
R[("yates", a, b)] = chy[1]
R[("boschloo", a, b)] = bo.pvalue
print(f" {a}/{b} vs {c}/{d}: Fisher exact P = {fe.pvalue:.4f} (OR = {fe.statistic:.1f}); "
f"chi-square Yates = {chy[0]:.3f}, P = {chy[1]:.4f}; no Yates = {chn[0]:.3f}, P = {chn[1]:.4f}; "
f"Boschloo exact P = {bo.pvalue:.4f}")
print(f" (uncorrected chi-square P equals the pooled z test P: "
f"{chn[1]:.4f} vs {z2(a, b, c, d)[1]:.4f})")
# Wilson intervals and Newcombe hybrid-score interval for the difference
def wilson(x, n, conf=0.95):
zc = stats.norm.ppf(1 - (1 - conf) / 2)
ph = x / n
den = 1 + zc ** 2 / n
cen = (ph + zc ** 2 / (2 * n)) / den
half = zc * np.sqrt(ph * (1 - ph) / n + zc ** 2 / (4 * n * n)) / den
return cen - half, cen + half
lO, uO = wilson(xO, nO)
lD, uD = wilson(xD, nD)
dd = xO / nO - xD / nD
dl = dd - np.sqrt((xO / nO - lO) ** 2 + (uD - xD / nD) ** 2)
du = dd + np.sqrt((uO - xO / nO) ** 2 + (xD / nD - lD) ** 2)
print(f" Wilson 95% CI flask O 7/8: {lO*100:.1f}% to {uO*100:.1f}%")
print(f" Wilson 95% CI flask D 6/12: {lD*100:.1f}% to {uD*100:.1f}%")
print(f" Newcombe 95% CI for the difference 37.5 points: {dl*100:.1f} to {du*100:.1f} points (includes 0)")
# ---------------------------------------------------------------- 3. routes to P < 0.0001
print("\n== 3. Calculations that would give P < 0.0001")
z_pc, p_pc, _ = z2(87.5, 100, 50, 100)
print(f" percentages treated as counts out of 100 (87.5/100 vs 50/100): z = {z_pc:.2f}, P = {p_pc:.1e}")
z1 = (0.5 - 0.875) / np.sqrt(0.875 * 0.125 / 12)
p1s = 2 * stats.norm.sf(abs(z1))
pb = stats.binomtest(6, 12, 0.875).pvalue
print(f" one-sample z, 6/12 against a fixed 0.875: z = {z1:.2f}, P = {p1s:.1e}; exact binomial P = {pb:.4f}")
z1b = (0.875 - 0.5) / np.sqrt(0.5 * 0.5 / 8)
p1b = 2 * stats.norm.sf(z1b)
pbb = stats.binomtest(7, 8, 0.5).pvalue
print(f" one-sample z, 7/8 against a fixed 0.5: z = {z1b:.2f}, P = {p1b:.4f}; exact binomial P = {pbb:.4f}")
print(" (neither one-sample route is a two-group comparison: each treats one flask's estimate as a known constant)")
need = None
curve_n = np.arange(4, 121)
curve_p = []
for n in curve_n:
zz, pp, _ = z2(0.875 * n, n, 0.5 * n, n)
curve_p.append(pp)
if need is None and pp < 1e-4:
need = n
curve_p = np.array(curve_p)
print(f" equal groups needed for pooled-z P < 1e-4 at 87.5% vs 50%: n = {need} per group "
f"(P = {z2(0.875*need, need, 0.5*need, need)[1]:.1e}); at n = {need-1}: "
f"P = {z2(0.875*(need-1), need-1, 0.5*(need-1), need-1)[1]:.1e}")
print(f" at n = 8 and 12 (actual): P = {R[('7/8 vs 6/12', True, False)][1]:.3f}")
print(" (different question) nondominant coiling against the 3% population upper bound, exact binomial, one-sided:")
for x, n, lab in [(1, 8, "flask O"), (6, 12, "flask D"), (7, 20, "all offspring")]:
print(f" {lab}: {x}/{n}, P = {stats.binomtest(x, n, 0.03, alternative='greater').pvalue:.2e}")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "axes.edgecolor": INK2, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2, "text.color": INK, "axes.linewidth": 0.8,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
"lines.linewidth": 1.5, "lines.markersize": 8,
})
W = 7.2
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def clean(ax, xgrid=True, ygrid=False):
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.grid(False)
if xgrid:
ax.xaxis.grid(True, color=GRID, lw=0.8)
if ygrid:
ax.yaxis.grid(True, color=GRID, lw=0.8)
ax.set_axisbelow(True)
# ---- Candidate A: P values on one log axis
def fmt_small(p):
m, e = f"{p:.1e}".split("e")
return f"P = {m}e{int(e)}"
def fig_A(path):
rows = [
("Reported in the paper\n(\"proportion z test\")", None, "rep"),
("Pooled two-proportion z test", R[("7/8 vs 6/12", True, False)][1], "ok"),
("Unpooled (Wald) z test", R[("7/8 vs 6/12", False, False)][1], "ok"),
("Pooled z test, parent counted (8/9)", R[("8/9 vs 6/12 (parent counted)", True, False)][1], "ok"),
("Fisher exact test", R[("fisher", 7, 8)], "ok"),
("Chi-square with Yates correction", R[("yates", 7, 8)], "ok"),
("One-sample z test, 6/12 against a\nfixed 0.875 (not a two-group test)", p1s, "bad"),
("Percentages entered as counts\nout of 100 (not a valid test)", p_pc, "bad"),
]
fig, ax = plt.subplots(figsize=(W, 5.3))
clean(ax)
ax.set_xscale("log")
ax.set_xlim(1e-9, 1)
ax.axvline(0.05, color=INK2, lw=0.8, ls="--", zorder=1)
ys = np.arange(len(rows))[::-1]
for y, (lab, p, kind) in zip(ys, rows):
if kind == "rep":
ax.annotate("", xy=(3e-9, y), xytext=(1e-4, y),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=14))
ax.plot([1e-4], [y], marker="|", ms=14, mew=2, color=ORANGE)
ax.text(1e-6, y + 0.18, "P < 0.0001", color=INK, va="bottom", ha="center", fontsize=10.5)
elif kind == "ok":
ax.plot([p], [y], "o", color=BLUE, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(p * 1.9, y, f"P = {p:.3f}", va="center", ha="left", fontsize=10.5, color=INK)
else:
ax.plot([p], [y], "o", mfc=BG, mec=BLUE, mew=2, ms=9, zorder=3)
ax.text(p * 2.6, y, fmt_small(p), va="center", ha="left", fontsize=10.5, color=INK2)
ax.text(0.05, -0.55, " 0.05", color=INK2, ha="left", va="bottom", fontsize=10)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10.5)
ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_xticks([1e-8, 1e-6, 1e-4, 1e-2, 1])
ax.set_xticklabels(["10⁻⁸", "10⁻⁶", "10⁻⁴", "0.01", "1"])
ax.set_xlabel("Two-sided P value, log scale; flask O vs flask D")
h = [Line2D([], [], color=ORANGE, marker="|", ms=12, mew=2, lw=2, label="Reported (upper bound)"),
Line2D([], [], color=BLUE, marker="o", ls="none", ms=8, label="Recomputed from paper's counts"),
Line2D([], [], mfc=BG, mec=BLUE, marker="o", ls="none", ms=8, mew=2, label="Recomputed, invalid")]
fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=3, frameon=False,
fontsize=10, handletextpad=0.4, columnspacing=1.2)
fig.tight_layout(rect=(0, 0.07, 1, 1))
save(fig, path)
# ---- Candidate B: proportions with 95% CIs and the difference, one percentage axis
def fig_B(path):
fig, ax = plt.subplots(figsize=(W, 4.0))
clean(ax)
rows = [("Flask O, 12-h light\n7 of 8 sinistral", 87.5, (lO * 100, uO * 100)),
("Flask D, 24-h dark\n6 of 12 sinistral", 50.0, (lD * 100, uD * 100)),
("Difference, O minus D\n(percentage points)", 37.5, (dl * 100, du * 100))]
ys = [2, 1, 0]
for y, (lab, v, (lo, hi)) in zip(ys, rows):
ax.plot([lo, hi], [y, y], color=BLUE, lw=2, solid_capstyle="round")
ax.plot([v], [y], "o", color=ORANGE if y > 0 else BLUE, ms=9, mec=BG, mew=1.5, zorder=3)
ax.text(hi + 2, y, f"{v:.1f} [{lo:.1f} to {hi:.1f}]", va="center", fontsize=10.5)
ax.axvline(0, color=INK2, lw=0.8)
ax.text(1, -0.55, "no difference", color=INK2, fontsize=10, va="center")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.set_xlim(-20, 130)
ax.set_xticks([-20, 0, 20, 40, 60, 80, 100])
ax.set_ylim(-0.8, 2.5)
ax.set_xlabel("Percent sinistral (rows 1-2); difference in percentage points (row 3)")
h = [Line2D([], [], color=ORANGE, marker="o", ls="none", ms=8, label="Reported percentage"),
Line2D([], [], color=BLUE, marker="o", lw=2, ms=8, label="Recomputed difference and 95% intervals")]
fig.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
fig.suptitle("Paper: P < 0.0001. The 95%% interval for the difference includes 0\n"
"(pooled z test P = %.3f; Wilson intervals, Newcombe interval for the difference)"
% R[("7/8 vs 6/12", True, False)][1], fontsize=10.5, x=0.02, ha="left", color=INK)
fig.tight_layout(rect=(0, 0.08, 1, 1))
save(fig, path)
# ---- Candidate C: calculation-flow diagram
def fig_C(path):
fig, ax = plt.subplots(figsize=(W, 5.8))
ax.set_xlim(0, 100)
ax.set_ylim(0, 100)
ax.axis("off")
def box(x, y, w, h, title, body, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.3,rounding_size=1.5",
fc=BG, ec=edge, lw=1.5))
ax.text(x + 1.5, y + h - 2.5, title, fontsize=10, color=INK2, va="top", fontweight="bold")
ax.text(x + 1.5, y + h - (15 if "\n" in title else 10), body, fontsize=10, color=INK, va="top", linespacing=1.4)
def arrow(x0, y0, x1, y1):
ax.add_patch(FancyArrowPatch((x0, y0), (x1, y1), arrowstyle="-|>", mutation_scale=14, color=INK2, lw=1.5))
pz = R[("7/8 vs 6/12", True, False)]
box(1, 54, 29, 42, "1 PRINTED",
"Flask O: 87.5%\n sinistral, n = 9\nFlask D: 50%\n sinistral, n = 12\nAll: 13 of 20 (65%)", ORANGE)
box(35.5, 54, 29, 42, "2 FITTING COUNTS",
f"87.5% x 9 = {0.875*9:.3f}\n (not whole)\nO: 7 of 8\n (Methods: 8\n offspring + parent)\nD: 6 of 12\n7+6 = 13, 8+12 = 20", BLUE)
box(70, 54, 29, 42, "3 RECOMPUTED",
f"Pooled z = {pz[0]:.2f}\n P = {pz[1]:.3f}\nFisher exact\n P = {R[('fisher', 7, 8)]:.3f}\n"
f"Parent counted\n (8/9), P = {R[('8/9 vs 6/12 (parent counted)', True, False)][1]:.3f}", BLUE)
box(1, 6, 29, 38, "4 PRINTED", "P < 0.0001,\n\"by a proportion\n z test\"", ORANGE)
box(35.5, 6, 63.5, 38, "5 ROUTES TO P < 0.0001 (NEITHER VALID)",
f"a) One-sample z test, 6/12 against a fixed 0.875:\n"
f" z = {z1:.2f}, {fmt_small(p1s)} (exact binomial P = {pb:.4f})\n"
f"b) Percentages entered as counts out of 100:\n"
f" 87.5/100 vs 50/100: z = {z_pc:.2f}, {fmt_small(p_pc)}\n"
f"A valid z test would need about {need} shells per group.", BLUE)
arrow(30.6, 75, 35.0, 75)
arrow(65.1, 75, 69.5, 75)
arrow(30.6, 25, 35.0, 25)
ax.plot([15.5, 15.5], [45, 53], color=INK2, lw=0.8, ls=":")
ax.text(16.5, 49, "same sentence", color=INK2, fontsize=10, va="center")
h = [Line2D([], [], color=ORANGE, lw=2, label="Box holds values printed in the paper"),
Line2D([], [], color=BLUE, lw=2, label="Box holds values recomputed from them")]
ax.legend(handles=h, loc="lower center", bbox_to_anchor=(0.5, -0.09), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.08)
save(fig, path)
# ---- Candidate D: P value against group size at 87.5% vs 50%
def fig_D(path):
fig, ax = plt.subplots(figsize=(W, 3.9))
clean(ax, xgrid=False, ygrid=True)
ax.set_yscale("log")
ax.plot(curve_n, curve_p, color=BLUE, lw=2, label="Pooled z test, 87.5% vs 50%, n per group")
pa = R[("7/8 vs 6/12", True, False)][1]
neq = 2 / (1 / 8 + 1 / 12)
ax.plot([neq], [pa], "o", color=BLUE, ms=9, mec=BG, mew=1.5)
ax.annotate(f"Actual groups, 8 and 12 (plotted at their\nequivalent equal size, {neq:.1f}): P = {pa:.3f}",
xy=(neq, pa), xytext=(18, 0.2), fontsize=10.5,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
pn = z2(0.875 * need, need, 0.5 * need, need)[1]
ax.plot([need], [pn], "o", color=BLUE, ms=9, mec=BG, mew=1.5)
ax.annotate(f"P < 0.0001 first reached\nat n = {need} per group", xy=(need, pn), xytext=(need + 10, 3e-3),
fontsize=10.5, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
p100 = z2(87.5, 100, 50, 100)[1]
ax.plot([100], [p100], "o", mfc=BG, mec=BLUE, mew=2, ms=9)
ax.annotate("n = 100: percentages\nread as counts (invalid)", xy=(100, p100), xytext=(72, 4e-7), fontsize=10.5,
color=INK2, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.fill_between([0, 125], 1e-10, 1e-4, color=ORANGE, alpha=0.10, lw=0)
ax.axhline(1e-4, color=ORANGE, lw=1.5)
ax.text(122, 1.4e-4, "Reported: P < 0.0001 (shaded region)", color=INK, ha="right", va="bottom", fontsize=10.5)
ax.axhline(0.05, color=INK2, lw=0.8, ls="--")
ax.text(122, 0.058, "0.05", color=INK2, ha="right", va="bottom", fontsize=10)
ax.set_xlim(0, 125)
ax.set_ylim(1e-9, 1)
ax.set_yticks([1e-8, 1e-6, 1e-4, 1e-2, 1])
ax.set_yticklabels(["10⁻⁸", "10⁻⁶", "10⁻⁴", "0.01", "1"])
ax.set_xlabel("Shells scored per treatment group (equal groups)")
ax.set_ylabel("Two-sided P value (log scale)")
h = [Line2D([], [], color=BLUE, lw=2, marker="o", ms=8, label="Recomputed (paper's percentages, varying n)"),
Line2D([], [], color=ORANGE, lw=1.5, label="Reported threshold")]
ax.legend(handles=h, loc="lower left", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# ---- Candidate E: every scored shell as a marker
def fig_E(path):
fig, ax = plt.subplots(figsize=(W, 3.6))
ax.set_xlim(0, 30)
ax.set_ylim(-2.3, 3.4)
ax.axis("off")
x0, dx = 7.6, 0.78
rows = [(2.0, "Flask O, 12-h light", 7, 1, "7 of 8 = 87.5%\n(printed: 87.5%, n = 9)"),
(0.6, "Flask D, 24-h dark", 6, 6, "6 of 12 = 50%\n(printed: 50%, n = 12)")]
for y, lab, s_, d_, txt in rows:
ax.text(0.2, y, lab, ha="left", va="center", fontsize=10.5)
for i in range(s_ + d_):
x = x0 + i * dx
if i < s_:
ax.plot([x], [y], "o", ms=11, color=INK2)
else:
ax.plot([x], [y], "o", ms=11, mfc=BG, mec=INK2, mew=1.8)
ax.text(x0 + 12 * dx + 0.4, y, txt, va="center", fontsize=10.5, color=INK, linespacing=1.3)
ax.plot([x0 + 8 * dx], [2.0], "s", ms=11, mfc=BG, mec=INK2, mew=1.2, ls=(0, (2, 1)))
ax.text(x0 + 8 * dx, 2.5, "9th?", ha="center", fontsize=10, color=INK2)
ax.plot([0.5], [-0.75], "s", ms=10, color=ORANGE)
ax.text(1.2, -0.75, "Printed: P < 0.0001, \"by a proportion z test\"", fontsize=10.5, color=INK, va="center")
ax.plot([0.5], [-1.5], "o", ms=10, color=BLUE)
ax.text(1.2, -1.5, f"Recomputed from these counts: pooled z test P = {R[('7/8 vs 6/12', True, False)][1]:.3f};"
f" Fisher exact P = {R[('fisher', 7, 8)]:.3f}", fontsize=10.5, color=INK, va="center")
h = [Line2D([], [], marker="o", ls="none", ms=10, color=INK2, label="sinistral offspring"),
Line2D([], [], marker="o", ls="none", ms=10, mfc=BG, mec=INK2, mew=1.8, label="dextral offspring"),
Line2D([], [], marker="s", ls="none", ms=10, mfc=BG, mec=INK2, mew=1.2,
label="shell implied by n = 9 (the parent?)")]
ax.legend(handles=h, loc="upper left", bbox_to_anchor=(0.0, 1.04), ncol=3, frameon=False, fontsize=10,
handletextpad=0.3, columnspacing=1.0)
fig.tight_layout()
save(fig, path)
for name, fn in zip("ABCDE", (fig_A, fig_B, fig_C, fig_D, fig_E)):
fn(os.path.join(CAND, f"fig1_{name}"))
fig_A(os.path.join(HERE, "fig1")) # provisional choice; swap after design review
# contact sheet of the five candidates, for the design review
import matplotlib.image as mpimg
titles = {"A": "P values on one log axis (dot plot)", "B": "Proportions and difference with 95% intervals",
"C": "Calculation-flow diagram", "D": "P value vs group size at 87.5% vs 50%",
"E": "Every scored shell as a marker"}
imgs = {k: mpimg.imread(os.path.join(CAND, f"fig1_{k}.png")) for k in "ABCDE"}
hs = [imgs[k].shape[0] / imgs[k].shape[1] for k in "ABCDE"]
fig = plt.figure(figsize=(W, W * (sum(hs) + 5 * 0.06)))
gs = fig.add_gridspec(5, 1, height_ratios=hs, hspace=0.12)
for i, k in enumerate("ABCDE"):
ax = fig.add_subplot(gs[i])
ax.imshow(imgs[k])
ax.axis("off")
ax.set_title(f"{k} {titles[k]}", loc="left", fontsize=11, fontweight="bold", color=INK)
fig.savefig(os.path.join(CAND, "contact_sheet.png"), dpi=120, bbox_inches="tight")
plt.close(fig)
print("\nFigures written: fig1.svg/.png (= candidate A) and candidates/fig1_A..E .svg/.png")
Abstract says the mAb reduces neutrophil apoptosis; the body and Fig. 3 show an increase
Sci Adv (2021) · doi:10.1126/sciadv.abd1453 · PMC11057782
Summary
The abstract says the agonist anti-ChemR23 antibody "reduces neutrophil apoptosis at the site of inflammation". Every other statement in the paper about this effect says the opposite. The Results say the antibody "accelerated neutrophil apoptotic cell death", with more Caspase-3/7 activation and annexin V staining. The safety paragraph's premise is that it "accelerates neutrophil apoptosis". The Discussion reports "increased apoptosis of remaining neutrophils" at the inflammatory site. The Fig. 3 F–H data also show more apoptotic or dead neutrophils with the antibody than with the isotype control, in vitro and in mouse air-pouch exudates. The data and the headline conclusions are not affected. But the abstract, which is the part most readers and text-mining tools see, states one of the paper's two cellular mechanisms backwards.
What the paper reports
Abstract, fourth sentence:
We also identified an anti-ChemR23 agonist antibody that induces receptor signaling, promotes macrophage efferocytosis, and reduces neutrophil apoptosis at the site of inflammation.
Abstract, first sentence. This sentence sets out the effects that proresolving agonists are expected to have:
Resolution of inflammation is elicited by proresolving lipids, which activate GPCRs to induce neutrophil apoptosis, reduce neutrophil tissue recruitment, and promote macrophage efferocytosis.
Results, "Agonist anti-ChemR23 mAb as new proresolving therapeutic modality", paragraph 1:
In addition, agonist anti-ChemR23 mAb internalization accelerated neutrophil apoptotic cell death (Fig. 3G and fig. S10C), as revealed by an increase in Caspase-3/7 activation and annexin V staining (Fig. 3F and fig. S10E).
Mechanistically, this accelerated neutrophil apoptosis was associated with increased ROS production (fig. S10D).
Results, "ChemR23 activation with agonist mAb triggers resolution of acute and chronic intestinal inflammation", paragraph 4 (the safety assessment):
Given that anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G) and is a human immunoglobulin G1 (hIgG1) isotype with potential cell cytotoxicity (55), we assessed its in vivo effects on the peripheral immune cell population.
Discussion, paragraph 3:
We have identified an anti-human ChemR23 mAb with similar receptor signaling functions as RvE1, including in promoting macrophage resolutive polarization as well as inflammatory neutrophil migration and apoptosis.
However, as previously reported for RvE1 (33, 34), agonist anti-ChemR23 mAb significantly shortened the Ri, as revealed by an accelerated clearing of leukocytes from the inflammatory site, increased apoptosis of remaining neutrophils, and faster recovery from colitis.
Fig. 3 legend, panels F–H:
(F) Percentage of Caspase-3/7 human neutrophils determined by microcopy after 10 hours of culture with isotype hIgG1 or anti-ChemR23 mAb at 10 μg/ml (n = 3). (G) Percentage of dead human neutrophils determined using Live and Dead assay after 24 hours of culture with isotype hIgG1 or anti-ChemR23 mAb at 10 μg/ml (n = 8). (H) Neutrophils absolute number (left) and percentage of dead neutrophils (right) measured in mouse exudates of TNF-induced inflammation air pouch model at indicated time points.
The paper's framing of why apoptosis matters. Introduction, paragraph 2:
In IBD, delayed neutrophil apoptosis and neutrophil-mediated damage are associated with the development of UC and CD (22–26).
Discussion, paragraph 2:
Appropriate neutrophil apoptosis is fundamental for inducing resolution of inflammation
What the paper's own numbers show
Method. The discrepancy is one of direction, not arithmetic, so the check is a census. I extracted from the archived full text every sentence that mentions neutrophil apoptosis or death. I kept the sentences that state the direction of the antibody's effect, and recorded the word that sets the direction in each. I then compared that direction with the direction of the three data panels that measure it (Fig. 3F, G and the right side of H). The paper gives no numeric values for these panels. The values below were read by eye from the published figure image, are approximate, and are used only to show direction and rough size. recompute.py.txt holds all the quotes and confirms that each one appears verbatim in the PMC text (9 of 9). It prints the tally and the approximate readings, and draws the figure.
| Quantity | Location | Reported | Recomputed / observed from the paper's own content |
|---|---|---|---|
| Direction of the mAb's effect on neutrophil apoptosis | Abstract, 4th sentence | "reduces neutrophil apoptosis at the site of inflammation" (decrease) | Opposite to every statement and data panel below |
| Neutrophil apoptotic death in vitro | Results, para 1 of the mAb section (Fig. 3G, fig. S10C) | "accelerated neutrophil apoptotic cell death" | Increase |
| Caspase-3/7 activation and annexin V | Same sentence (Fig. 3F, fig. S10E) | "an increase in Caspase-3/7 activation and annexin V staining" | Increase |
| Mechanism | Results, next sentence (fig. S10D) | "this accelerated neutrophil apoptosis was associated with increased ROS production" | Increase |
| Premise of the safety assessment | Results, colitis section, para 4 | "anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G)" | Increase |
| Summary of in vitro activity | Discussion, para 3 | "promoting … inflammatory neutrophil migration and apoptosis" | Increase |
| Effect at the inflammatory site in vivo | Discussion, para 3 | "increased apoptosis of remaining neutrophils" | Increase |
| Caspase-3/7+ human neutrophils, 10 h (n = 3) | Fig. 3F | Marked * (P < 0.05) | About 45% with hIgG1 vs about 85% with the mAb (read by eye): higher with the mAb |
| Dead human neutrophils, Live/Dead, 24 h (n = 8) | Fig. 3G | Marked ** (P < 0.01) | About 20% vs about 55% (read by eye): higher with the mAb |
| Dead neutrophils in air-pouch exudate (n = 4 to 5 per group) | Fig. 3H, right | Marked * at 24 h and ** at 48 h | Higher with the mAb at both times |
| Tally of directional text statements | Whole article | — | 1 decrease (abstract only) vs 5 increase (Results 3, Discussion 2); 0 statements in the body say decrease |
| Tally of data panels | Fig. 3F, G, H right | — | 3 of 3 show more apoptosis or death with the mAb; 0 of 3 show less |
Figure
Figure 1. The left panel (orange) shows the abstract's sentence as published, which states a decrease in neutrophil apoptosis. Below it, in gray, is the abstract's own first sentence, which says proresolving GPCR agonists "induce neutrophil apoptosis". The right panel (blue) shows short verbatim excerpts from the Results and the Discussion, plus a description of Fig. 3F–H read from the published image. All of them state or show an increase. The connector marks the conflict: both panels describe the same experiments, but in opposite directions. Of the five body statements counted by recompute.py.txt, four are drawn here. The fifth, the Discussion's "promoting … inflammatory neutrophil migration and apoptosis", is listed in the table above. No values are plotted; the paper's own figure is not reproduced.
Why it matters
The abstract names two cellular mechanisms for the antibody: macrophage efferocytosis and an effect on neutrophil apoptosis. It gives the direction of the second one backwards. The paper argues that accelerated neutrophil apoptosis followed by efferocytosis is how resolution happens ("Appropriate neutrophil apoptosis is fundamental for inducing resolution of inflammation"), and that delayed apoptosis is part of IBD pathology. Read on its own, the abstract therefore describes a mechanism that runs against the paper's own model.
The direction also matters for safety. The Results say the antibody was screened for immune-cell depletion and neutropenia because it "accelerates neutrophil apoptosis" and is a cytotoxicity-capable hIgG1. An abstract that says it reduces apoptosis hides the reason for that check. Readers who see only the abstract (PubMed, indexing services, literature-mining tools that extract direction-of-effect relations) could record ChemR23 agonism as anti-apoptotic for neutrophils. That contradicts both this paper's data and the resolvin E1 literature it cites.
The data, the statistics and the headline conclusions are not affected. These are the title's "reduces tissue neutrophil accumulation", the faster resolution in the air-pouch model, and the reduced lesions, fibrosis and tumors in the colitis models. Only the wording of one abstract sentence is wrong.
Explanations considered
- Conversion or archiving error in PMC. Ruled out. The same wording appears in the publisher-deposited JATS XML held by Europe PMC (vol 7, issue 14, eabd1453, published 2 April 2021), in the PubMed abstract (PMID 33811066, checked on 2026-10-23) and in the institutional repository record (HAL, hal-03462811).
- The qualifier "at the site of inflammation" refers to a different measurement from the in vitro assays. This does not reconcile the two. The only in vivo measurement of neutrophil death at an inflammatory site is the percentage of dead neutrophils in air-pouch exudates (Fig. 3H, right). It is higher with the antibody at 24 h and 48 h, and the Discussion describes it as "increased apoptosis of remaining neutrophils" at the inflammatory site.
- "Reduces neutrophil apoptosis" means fewer apoptotic neutrophils remain because macrophages clear them faster. The paper reports no measurement showing fewer apoptotic neutrophils, and the exudate data show a higher fraction of dead neutrophils. Faster clearance would also be a statement about efferocytosis, which the same sentence already names separately. This reading is not supported.
- A different cell population or disease context. The only patient-derived neutrophil data (fig. S9, ANCA vasculitis) concern transmigration, not apoptosis. No sentence in the article reports reduced apoptosis in any population. The supplementary PDF (figs. S10C–E) could not be downloaded, but the main-text Results describe those panels as showing an increase.
- The intended phrase was "reduces neutrophil accumulation" (as in the title) or "induces neutrophil apoptosis" (as in the abstract's first sentence). Either would agree with the body. This explains how the sentence could be fixed, but it does not change what the published sentence says.
- The data, rather than the abstract, are wrong. Nothing points that way. Three separate assays in vitro (Caspase-3/7, annexin V, Live/Dead) and one in vivo all go the same way, with significance markers, and the text describes each one consistently.
A smaller wording problem sits in the same part of the paper. The Discussion's "promoting … inflammatory neutrophil migration and apoptosis" also reads as promoting migration, whereas the Results report that the antibody "significantly reduced human neutrophil transendothelial migration". This is noted for completeness and is not part of the main finding.
Status
- Version of record. The publisher page (science.org, reached via doi.org) returned HTTP 403 to automated access, so it was not read directly. The wording was confirmed in four independent copies: the publisher-deposited JATS XML in Europe PMC, the PMC text (via
src/show.py), the PubMed record (PMID 33811066) and the HAL repository record. All four have the abstract sentence as quoted. - Supplement. The supplementary PDF (sciadv.abd1453 SM, which holds figs. S10C–E) could not be retrieved: the PMC links returned 404 and the publisher link returned 403. The main Fig. 3 image (abd1453-f3.jpg, from the Europe PMC supplementary-files bundle) was viewed. Panels F, G and H go in the direction the Results describe. The discrepancy is in the abstract's wording, so the supplement would not resolve it either way.
- Corrections and comments. As of 2026-10-23, none found. Crossref lists no update relations for the DOI (
src/known.py). PMC and Europe PMC list no correction, erratum, retraction or expression of concern. The PubMed record has no comments or corrections links. - PubPeer. PubPeer's search page and API returned HTTP 403 (a Cloudflare challenge) to automated queries, so PubPeer could not be checked directly. A web search for the title together with "PubPeer" found no thread, only the article, repository records and a company announcement. Absence of a PubPeer thread is therefore not fully confirmed.
- Sources. PMC11057782; Europe PMC full-text XML and supplementary files for PMC11057782; PubMed 33811066; Crossref metadata for 10.1126/sciadv.abd1453; HAL hal-03462811.
Suggested correction
In the abstract, replace
…promotes macrophage efferocytosis, and reduces neutrophil apoptosis at the site of inflammation.
with wording that matches the Results and Discussion, for example
…promotes macrophage efferocytosis, and induces neutrophil apoptosis and reduces neutrophil accumulation at the site of inflammation.
The data, figures and statistics need no change. Optionally, the Discussion sentence "promoting macrophage resolutive polarization as well as inflammatory neutrophil migration and apoptosis" could be reworded to "promoting macrophage resolutive polarization and neutrophil apoptosis while reducing inflammatory neutrophil migration", which matches the transmigration result.
Recomputation script (python recompute.py.txt)
"""Direction-of-effect check for PMC11057782
(Sci Adv 2021;7(14):eabd1453, doi:10.1126/sciadv.abd1453).
Run: python recompute.py.txt
Needs matplotlib. Prints every number used in writeup.md and writes fig1.svg and
fig1.png next to this file.
The error is in the text, not the arithmetic. The abstract says the agonist
anti-ChemR23 mAb "reduces neutrophil apoptosis at the site of inflammation"; every
statement in the body, and the direction of the Fig. 3 F-H data, says it increases
(accelerates) it. This script lists every sentence in the article that states the
direction of the mAb's effect on neutrophil apoptosis or death, records the word
that sets the direction (classified by hand, and checked to occur in the quote),
and tallies them.
Quotes are copied verbatim from the article as archived in PubMed Central. If this
file sits in the papers repository, the script also re-checks each quote against
the archived text with src/show.py (skipped silently elsewhere).
The Fig. 3 values below were read by eye from the published figure image
(abd1453-f3.jpg); the paper prints no numeric values for these panels. They are
approximate and are used only for direction and rough size, never for a test.
"""
import os
import subprocess
import textwrap
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
PMCID = "PMC11057782"
# ------------------------------------------------------------------ statements
# (location, verbatim quote, word(s) that set the direction, direction)
# direction: +1 = increase / accelerate, -1 = decrease, 0 = context (not about the mAb)
STATEMENTS = [
("Abstract, 4th sentence",
"We also identified an anti-ChemR23 agonist antibody that induces receptor signaling, "
"promotes macrophage efferocytosis, and reduces neutrophil apoptosis at the site of inflammation.",
"reduces neutrophil apoptosis", -1),
("Results, 'Agonist anti-ChemR23 mAb as new proresolving therapeutic modality', para 1",
"In addition, agonist anti-ChemR23 mAb internalization accelerated neutrophil apoptotic cell death "
"(Fig. 3G and fig. S10C), as revealed by an increase in Caspase-3/7 activation and annexin V "
"staining (Fig. 3F and fig. S10E).",
"accelerated neutrophil apoptotic cell death", +1),
("Results, same paragraph, next sentence",
"Mechanistically, this accelerated neutrophil apoptosis was associated with increased ROS "
"production (fig. S10D).",
"accelerated neutrophil apoptosis", +1),
("Results, 'ChemR23 activation with agonist mAb triggers resolution...', para 4",
"Given that anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G) and is a human "
"immunoglobulin G1 (hIgG1) isotype with potential cell cytotoxicity (55), we assessed its in vivo "
"effects on the peripheral immune cell population.",
"accelerates neutrophil apoptosis", +1),
("Discussion, para 3 (in vitro summary)",
"We have identified an anti-human ChemR23 mAb with similar receptor signaling functions as RvE1, "
"including in promoting macrophage resolutive polarization as well as inflammatory neutrophil "
"migration and apoptosis.",
"promoting", +1),
("Discussion, para 3 (in vivo, inflammatory site)",
"However, as previously reported for RvE1 (33, 34), agonist anti-ChemR23 mAb significantly "
"shortened the Ri, as revealed by an accelerated clearing of leukocytes from the inflammatory "
"site, increased apoptosis of remaining neutrophils, and faster recovery from colitis.",
"increased apoptosis of remaining neutrophils", +1),
]
# Context sentences: what the paper says proresolving agonists are expected to do.
CONTEXT = [
("Abstract, 1st sentence",
"Resolution of inflammation is elicited by proresolving lipids, which activate GPCRs to induce "
"neutrophil apoptosis, reduce neutrophil tissue recruitment, and promote macrophage efferocytosis.",
"induce neutrophil apoptosis"),
("Introduction, para 2",
"In IBD, delayed neutrophil apoptosis and neutrophil-mediated damage are associated with the "
"development of UC and CD (22–26).",
"delayed neutrophil apoptosis"),
("Discussion, para 2",
"Appropriate neutrophil apoptosis is fundamental for inducing resolution of inflammation",
"Appropriate neutrophil apoptosis is fundamental"),
]
# Fig. 3 data panels (legend text is in the paper; values read by eye from the image).
# (panel, what is measured, n, control approx %, mAb approx %, significance marker)
FIG3 = [
("F", "Caspase-3/7+ human neutrophils, 10 h in vitro", "n = 3", 45, 85, "*"),
("G", "Dead human neutrophils (Live/Dead), 24 h in vitro", "n = 8", 20, 55, "**"),
("H right", "Dead neutrophils in mouse air-pouch exudate, 24 h and 48 h", "n = 4 to 5 per group",
None, None, "* at 24 h, ** at 48 h"),
]
def check_against_archive():
"""Re-check every quote against the PMC text when the repository tool is present."""
repo = os.path.abspath(os.path.join(HERE, "..", "..", ".."))
show = os.path.join(repo, "src", "show.py")
py = "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(show) and os.path.exists(py)):
return None
try:
full = subprocess.run([py, show, PMCID, "--full"], cwd=repo, capture_output=True,
text=True, timeout=120).stdout
except Exception:
return None
norm = " ".join(full.split())
out = []
for loc, q, *_ in STATEMENTS + CONTEXT:
out.append((loc, " ".join(q.split()) in norm))
return out
def main():
print(f"{PMCID}: statements on the direction of the mAb's effect on neutrophil apoptosis\n")
for loc, q, key, d in STATEMENTS:
assert key in q, (loc, key)
lab = {+1: "INCREASE", -1: "DECREASE"}[d]
print(f" {lab:8s} | {loc}\n governing words: '{key}'")
n_dec = sum(1 for s in STATEMENTS if s[3] == -1)
n_inc = sum(1 for s in STATEMENTS if s[3] == +1)
dec_locs = [s[0] for s in STATEMENTS if s[3] == -1]
print(f"\nText statements saying DECREASE: {n_dec} ({'; '.join(dec_locs)})")
print(f"Text statements saying INCREASE: {n_inc} (all in Results or Discussion)")
print(f"Body statements saying DECREASE: {sum(1 for s in STATEMENTS[1:] if s[3] == -1)}")
print("\nContext statements (direction proresolving agonists are expected to act):")
for loc, q, key in CONTEXT:
assert key in q
print(f" {loc}: '{key}'")
print("\nFig. 3 data panels (values read by eye from the image; approximate):")
n_up = 0
for p, what, n, c, m, sig in FIG3:
if c is not None:
print(f" Fig. 3{p}: {what} ({n}): hIgG1 ~{c}% vs anti-ChemR23 ~{m}% "
f"(difference ~{m - c} points, {m / c:.1f}x), marked {sig}")
else:
print(f" Fig. 3{p}: {what} ({n}): higher with anti-ChemR23 than hIgG1, marked {sig}")
n_up += 1
print(f"Fig. 3 panels in which apoptosis/death is higher with the mAb: {n_up} of {len(FIG3)}")
print(f"Fig. 3 panels in which it is lower: 0 of {len(FIG3)}")
chk = check_against_archive()
if chk is not None:
ok = sum(1 for _, v in chk if v)
print(f"\nQuotes found verbatim in the archived PMC text: {ok} of {len(chk)}")
for loc, v in chk:
if not v:
print(f" NOT FOUND: {loc}")
make_fig1()
# ------------------------------------------------------------------ figure
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#d9d8d4"
def wrap(s, width):
return "\n".join(textwrap.wrap(s, width))
def make_fig1():
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10, "text.color": TXT,
"svg.fonttype": "none", "figure.facecolor": BG, "savefig.facecolor": BG})
W, H = 7.2, 6.0
fig = plt.figure(figsize=(W, H))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, W)
ax.set_ylim(0, H)
ax.axis("off")
LH = 0.158 # line height in inches at 10 pt
def box(x0, y0, x1, y1, col):
ax.add_patch(FancyBboxPatch((x0, y0), x1 - x0, y1 - y0, boxstyle="round,pad=0,rounding_size=0.06",
fc="white", ec=col, lw=1.5))
def put(x, y, s, n_lines=None, **kw):
ax.text(x, y, s, va="top", ha="left", linespacing=1.25, **kw)
k = s.count("\n") + 1 if n_lines is None else n_lines
return y - k * LH * 1.0
ax.text(0.12, H - 0.14, "Direction of the mAb's effect on neutrophil apoptosis, as stated in the paper",
fontsize=11.5, fontweight="bold", va="top")
# ---- left: abstract
lx0, lx1, top, bot = 0.12, 2.55, H - 0.52, 0.86
box(lx0, bot, lx1, top, ORANGE)
y = top - 0.12
y = put(lx0 + 0.12, y, "ABSTRACT", fontsize=10.5, fontweight="bold", color=ORANGE)
y = put(lx0 + 0.12, y, "4th sentence (as published)", fontsize=10, color=SEC) - 0.06
s = STATEMENTS[0][1]
y = put(lx0 + 0.12, y, wrap("“" + s + "”", 28), fontsize=10)
y -= 0.14
y = put(lx0 + 0.12, y, "Direction stated:", fontsize=10, color=SEC)
y = put(lx0 + 0.12, y, "↓ DECREASE", fontsize=12, fontweight="bold", color=ORANGE) - 0.30
ax.plot([lx0 + 0.12, lx1 - 0.12], [y + 0.12, y + 0.12], color=GRID, lw=1)
y = put(lx0 + 0.12, y, "Same abstract, 1st sentence:", fontsize=10, color=SEC)
y = put(lx0 + 0.12, y, wrap("“…proresolving lipids, which activate GPCRs to induce "
"neutrophil apoptosis…”", 28), fontsize=10, color=SEC)
# ---- right: body
rx0, rx1 = 3.45, 7.08
box(rx0, bot, rx1, top, BLUE)
y = top - 0.12
y = put(rx0 + 0.12, y, "RESULTS, DISCUSSION, FIG. 3", fontsize=10.5, fontweight="bold", color=BLUE) - 0.06
items = [
("Results (Fig. 3F, G; in vitro)",
"“accelerated neutrophil apoptotic cell death … an increase in Caspase-3/7 activation "
"and annexin V staining”"),
("Results (mechanism)",
"“this accelerated neutrophil apoptosis was associated with increased ROS production”"),
("Results (safety, colitis section)",
"“anti-ChemR23 mAb accelerates neutrophil apoptosis (Fig. 3, F and G)”"),
("Discussion (in vivo, inflammatory site)",
"“an accelerated clearing of leukocytes from the inflammatory site, increased apoptosis "
"of remaining neutrophils”"),
("Fig. 3F, G, H (data, read from the image)",
"Caspase-3/7+ and dead neutrophils higher with the mAb than with hIgG1 in vitro, and dead "
"neutrophils higher in air-pouch exudate at 24 h and 48 h"),
]
for loc, txt in items:
y = put(rx0 + 0.12, y, loc, fontsize=10, color=SEC)
y = put(rx0 + 0.12, y, wrap(txt, 43), fontsize=10) - 0.08
y -= 0.04
y = put(rx0 + 0.12, y, "Direction stated:", fontsize=10, color=SEC)
n_inc = sum(1 for s in STATEMENTS if s[3] == +1)
y = put(rx0 + 0.12, y, "↑ INCREASE", fontsize=12, fontweight="bold", color=BLUE)
put(rx0 + 0.12, y - 0.04, f"all {n_inc} body statements; all 3 data panels",
fontsize=10, fontweight="bold", color=BLUE)
# ---- connector
ymid = top - 1.55
arr = FancyArrowPatch((lx1 + 0.04, ymid), (rx0 - 0.04, ymid), arrowstyle="<|-|>", mutation_scale=14,
lw=2, color=TXT)
ax.add_patch(arr)
ax.text((lx1 + rx0) / 2, ymid + 0.12, "opposite\ndirection", ha="center", va="bottom",
fontsize=10.5, fontweight="bold")
ax.text((lx1 + rx0) / 2, ymid - 0.12, "same\ndata\n(Fig. 3F–H)", ha="center", va="top",
fontsize=10, color=SEC, linespacing=1.2)
ax.text(0.12, 0.70, "Orange: the abstract's wording as published.", fontsize=10, color=SEC, va="top")
ax.text(0.12, 0.50, "Blue: what the Results, the Discussion and the paper's own Fig. 3 data show.",
fontsize=10, color=SEC, va="top")
ax.text(0.12, 0.30, "Quotes verbatim from the PMC archive copy (PMC11057782); "
"… marks omitted words.", fontsize=10, color=SEC, va="top")
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)
print("\nwrote fig1.svg and fig1.png")
if __name__ == "__main__":
main()
Abstract and Introduction state the reverse of the paper's own Schwann cell depletion result
Sci Adv (2023) · doi:10.1126/sciadv.add6995 · PMC9891701
Summary
The Abstract states that "Depletion of pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors", and the Introduction repeats it. (TASc are tumor-associated nonmyelinating Schwann cells.) The paper's only pancreas-specific depletion is intrapancreatic AAV8-GFAP-DTA in KPC mice. For that experiment the Results report improved survival (Fig. 2C; log-rank; n = 8, 12 and 13 mice), fewer PanIN 2/3 lesions and inhibited tumor growth (Fig. 2D–G). The Fig. 2 title reads "Depletion of TASc attenuates pancreatic cancer tumorigenesis". The main text contains 17 statements (14 sentences, one Results subsection heading and two figure titles) that state a direction for the effect of depleting, inhibiting or targeting TASc (or TASc-expressed PVT1) on tumors. Fifteen say depletion is antitumor. Only the Abstract and Introduction sentences say it is protumor. The data, statistics and the paper's central thesis are unaffected. Still, the Abstract, the paper's own summary of its findings, reverses the direction of its headline in vivo result, and no correction has been issued.
What the paper reports
The two sentences that state the depletion result as protumor:
Depletion of pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors.
Abstract, sentence 4
Depletion of TASc promoted the tumorigenesis of PDAC.
Introduction, final (third) paragraph
The same experiment as described in the Results and in the Fig. 2 legend:
TASc promotes PDAC tumorigenesis
Results, subsection heading
KPC mice given AAV8-GFAP-DTA exhibited improved survival time compared to control group (Fig. 2C). Upon AAV8-GFAP-DTA–mediated Schwann cell depletion, KPC pancreases exhibited increased percentage of PanIN 1, reduced PanIN 2/3, and inhibited tumor growth (Fig. 2, D to G).
Results, "TASc promotes PDAC tumorigenesis", paragraph 1
Depletion of TASc attenuates pancreatic cancer tumorigenesis.
Fig. 2 title
Log-rank test; n = 8, 12, and 13 mice, respectively.
Fig. 2 legend, panel C (WT, AAV8-GFAP-Blank and AAV8-GFAP-DTA KPC mice)
Error bars, SDs; n = 12 and 13 animals, respectively; Student’s t test.
Fig. 2 legend, panel G (tumor volumes at 22 weeks)
The paper's own summary of the add-back experiment in the same subsection points the same way:
Together, these results suggested that the presence of TASc in PDAC facilitates tumorigenesis and resistance to immunotherapy.
Results, "TASc promotes PDAC tumorigenesis", paragraph 2, last sentence
So do the pharmacological depletion with fluorocitrate (FC), the Discussion, and a later sentence of the same Abstract:
Furthermore, depletion of TASc in PDAC may profoundly diminish tumor growth and improve the infiltration and cytotoxicity of CD8+ T cells.
Results, "Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy", last sentence
Depletion of TASc-expressed Pvt1 or TASc using FC restored the immunoresistant metabolic microenvironment and improved the antitumor effects of immune checkpoint inhibitors.
Discussion, paragraph 2
Furthermore, depletion of TASc using a small-molecule inhibitor effectively sensitized PDAC to immunotherapy, signifying the important roles of TASc in PDAC immune resistance.
Abstract, last sentence
Each quotation was checked character for character against the article as archived in PubMed Central (src/show.py PMC9891701 --grep), and the key sentences also against the publisher's typeset PDF (see Status).
What the paper's own numbers show
Method. No quantity can be recomputed, because the error is in the stated direction of an effect. The tumor volumes and survival curves appear only as plotted data in Fig. 2, and the text and figure title state their direction explicitly. recompute.py.txt therefore holds a hand-curated list of every statement (sentence, Results subsection heading or figure title) outside Materials and Methods that states a direction for the effect of depleting, inhibiting or targeting TASc, or TASc-expressed PVT1, on tumor growth, tumorigenesis, survival or response to immunotherapy. To build it, every sentence of the Abstract, Introduction, Results and Discussion, every Results subsection heading and every figure title was read. A statement was included when its subject is the removal or inhibition of TASc (by AAV8-GFAP-DTA, by fluorocitrate, by "a small-molecule" agent, or by targeting in general) or of TASc-expressed PVT1, and it states a direction for one of those outcomes. Statements left out were: those about the presence, rather than the removal, of TASc or PVT1 (such as the heading "TASc promotes PDAC tumorigenesis"); deletions of PVT1 nucleotides, RAF1 or TDO2; immune-cell or TASc-abundance readouts only; the cytotoxicity, safety or specificity of FC; and sentences calling TASc a therapeutic target without stating a tumor outcome. The script checks each listed statement verbatim against the archived text and confirms its section, labels each with the direction it states, and tallies them. Run with the publisher's PDF and supplement as text (instructions in the script header), it also checks those. It prints every number below.
| Quantity | Location | Reported | Recomputed from the paper's own text |
|---|---|---|---|
| Effect of depleting pancreatic TASc on tumorigenesis | Abstract, sentence 4 | "promoted the tumorigenesis" | Opposite: depletion attenuates tumorigenesis (Results; Fig. 2 title; fig. S4 title) |
| Same | Introduction, final paragraph | "promoted the tumorigenesis" | Opposite, as above |
| Survival after AAV8-GFAP-DTA depletion | Results; Fig. 2C (log-rank; n = 8 WT, 12 Blank, 13 DTA) | "improved survival time compared to control group" | Depletion is antitumor |
| PanIN grade and tumor growth after depletion | Results; Fig. 2D–G (Fig. 2F n = 8, 12, 13; Fig. 2G n = 12 vs 13) | "reduced PanIN 2/3, and inhibited tumor growth" | Depletion is antitumor |
| Fig. 2 title | Fig. 2 legend | "Depletion of TASc attenuates pancreatic cancer tumorigenesis." | Depletion is antitumor |
| Directional statements on depleting, inhibiting or targeting TASc or TASc-expressed PVT1 | Main text (Abstract, Introduction, Results incl. subsection headings, Discussion, figure titles) | — | 17 statements (14 sentences, 1 heading, 2 figure titles): 2 protumor (Abstract, Introduction), 15 antitumor |
| Statements about the AAV8-GFAP-DTA experiment only | Main text | — | 5 statements: 2 protumor (Abstract, Introduction), 3 antitumor (Results ×2, Fig. 2 title) |
| Same, including the supplementary figure titles | Main text + figs. S4, S13, S16 | — | 20 statements, 2 protumor, 18 antitumor (19 and 17 if the fig. S16 schematic is left out); DTA experiment 6 statements, 2 protumor, 4 antitumor |
| Fluorocitrate or small-molecule statements | Main text | — | 7, all antitumor |
| PVT1-loss statements | Main text | — | 3, all antitumor |
| Statements naming both PVT1 loss and FC | Main text (Discussion) | — | 2, both antitumor |
| Sections that contain a protumor statement | Main text | — | Abstract and Introduction only |
| Supplement sentences of the form "deplet… promot… tumorigen…" | Supplementary Materials PDF | — | 0 |
The Abstract itself contains three depletion statements in opposite directions. Sentence 4 says depletion is protumor. The PVT1 sentence and the last sentence say depletion is antitumor.
A minor side observation that does not affect the finding: the Fig. 2G legend names three groups (WT, AAV8-GFAP-Blank, AAV8-GFAP-DTA) but gives n for only two ("n = 12 and 13 animals").
Figure
Figure 1. The left panel (orange) shows the two sentences, in the Abstract and the Introduction, that state the depletion result as protumor. The right panel (blue) shows what the paper reports for the same experiment, intrapancreatic AAV8-GFAP-DTA in KPC mice: the Results phrases for Fig. 2C and Fig. 2D–G, with sample sizes from the Fig. 2 legend, and the Fig. 2 title. The connector marks the conflict: one experiment, two opposite directions. All text is quoted from the paper. None of the paper's figures or data are reproduced.
Figure 2. Each marker is one of the 17 main-text statements that state a direction for the effect of depleting, inhibiting or targeting TASc, or TASc-expressed PVT1, on tumors (inclusion rule under Method). Columns run in reading order from the Abstract to the Discussion, followed by the two figure titles. The Results FC column includes the subsection heading "Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy". The upper row holds statements that depletion promotes tumorigenesis. The lower row holds statements that it inhibits tumor growth or improves outcome. Marker shape gives the intervention: circles for AAV8-GFAP-DTA, squares for PVT1 loss, diamonds for fluorocitrate, triangles for sentences that name both PVT1 and FC. Markers in the same cell are stacked. Only the two orange markers, in the Abstract and the Introduction, sit in the upper row. Both concern the DTA experiment, whose three other statements all sit in the lower row. The supplementary titles of figs. S4, S13 and S16 (not drawn) would all sit in the lower row. Directions and locations come from recompute.py.txt.
Why it matters
- A headline result is stated in reverse in the Abstract. The Abstract sentence reports the paper's main in vivo test of function: whether removing TASc from the pancreas changes tumor development. Read literally, it says TASc restrain PDAC. That contradicts the Results heading ("TASc promotes PDAC tumorigenesis"), the Fig. 2 title, and the paper's therapeutic argument for depleting TASc with fluorocitrate plus anti–PD-1.
- Readers of the Abstract alone get the wrong direction. Many readers, indexing services, text-mining pipelines and automated summaries rely on the Abstract. Any of them could record "depletion of Schwann cells promotes pancreatic cancer". The Introduction repeats the sentence, so both places that summarize the findings carry the error. Crossref lists 68 citing works.
- The science itself is not affected. No reported value, test or P value changes, and the Results, figure legends and Discussion consistently give the antitumor direction. The same Abstract goes on to say that depleting TASc with a small molecule sensitized PDAC to immunotherapy, so a careful reader would notice the conflict. The error is in the wording of two summary sentences, not in the data or analysis.
Explanations considered
- The sentences refer to a different experiment or setting. This does not hold. "Pancreatic-specific" depletion matches only the intrapancreatic AAV8-GFAP-DTA delivery in KPC mice (Fig. 2, fig. S4). Every other statement about that experiment, in the Results, the Fig. 2 title and the fig. S4 title, reports an antitumor effect. The paper reports no subgroup, time point or model in which depletion accelerated tumors.
- "Tumorigenesis" is used in an unusual sense. This does not hold. The paper uses the word in its ordinary sense throughout, for example "TASc promotes PDAC tumorigenesis" and "Combining FC with the anti–PD-1 antibody significantly suppressed PDAC tumorigenesis".
- The increase in PanIN 1 lesions counts as "promoted". This does not hold. After depletion, the percentage of low-grade PanIN 1 rose while the share of high-grade PanIN 2/3 fell. The paper reports this shift in the same sentence as "inhibited tumor growth" and summarizes the panel under the title "Depletion of TASc attenuates pancreatic cancer tumorigenesis".
- The intended subject was TASc, not their depletion. "TASc promoted the tumorigenesis of PDAC", as in the Results heading, would be consistent with the data. So would "Depletion of TASc attenuated tumorigenesis", as in the Fig. 2 title. This explains how the wording could have arisen. It does not change the fact that the printed sentences state the opposite of the data.
- Conversion or archiving artifact. Ruled out. The publisher's typeset PDF in the PMC Open Access dataset (version 1, the only version; marked as the publisher's version, not an author manuscript) contains both sentences word for word. So does the Europe PMC full text. The conflict is in words ("promoted" against "attenuates"), not in symbols, signs or table cells that a conversion could alter.
- The supplement resolves it. It does not. The fig. S4 title ("Depletion of TASc inhibits PDAC tumorigenesis."), the fig. S13 title ("FC treatment sensitizes PDAC to immunotherapy.") and the title of the fig. S16 schematic ("Targeting tumor-associated non-myelinating Schwann cells improves anti-tumor immunity of immunotherapy-refractory pancreatic cancer.") all support the Results. The supplement contains no sentence saying depletion promotes tumorigenesis.
None of these explanations reconciles the Abstract and Introduction sentences with the paper's results.
Status
As of 2026-10-23:
- Version of record. The publisher's typeset PDF (16 pages, created 2023-01-26) held in the PMC Open Access dataset (
s3://pmc-oa-opendata/PMC9891701.1/;is_manuscript: false,is_retracted: false) was converted to text and searched. It contains the Abstract and Introduction sentences, the Results sentences and the Fig. 2 title as quoted. In the PDF's text layer the survival sentence is interrupted by a figure caption at a column break, and its text continues unchanged after the caption. The live PMC page and the Europe PMC full-text XML carry the same wording. The publisher's own page (doi.org redirects to science.org) returned HTTP 403 to automated access, so it could not be checked directly. - Supplements. The Supplementary Materials PDF (
sciadv.add6995_sm.pdf: methods and figs. S1 to S16) was converted to text and searched for depletion, DTA and FC statements. The fig. S4, fig. S13 and fig. S16 titles agree with the Results direction. The tables archive (Tables S1 to S7) was not opened, because the finding concerns the direction of a stated result, not a tabulated value. - Corrections. Crossref lists no update for the DOI: the record has no update-to field, and a filter for works that update it returned 0 results. PMC and Europe PMC list no correction, erratum, expression of concern or retraction (
src/known.py). The Europe PMC record (PMID 36724291) has no comment/correction list. It shows a revision date of 2024-05-08 without a stated reason. The PMC copy still carries the "promoted" wording. - PubPeer and the web. The PubPeer API returned HTTP 403, so PubPeer could not be queried directly. Earlier web searches by DOI and by title, combined with correction, erratum or PubPeer, found no notice or discussion of this point. A PubPeer comment therefore cannot be ruled out completely.
- Sources: PMC9891701 (via
src/show.py);src/known.py 10.1126/sciadv.add6995; PMC Open Access dataset files PMC9891701.1.pdf, PMC9891701.1.json and sciadv.add6995_sm.pdf; https://api.crossref.org/works/10.1126/sciadv.add6995 and https://api.crossref.org/works?filter=updates:10.1126/sciadv.add6995; Europe PMC REST record for DOI 10.1126/sciadv.add6995; https://doi.org/10.1126/sciadv.add6995 (HTTP 403); https://pubpeer.com/api/search/?q=10.1126/sciadv.add6995 (HTTP 403).
Suggested correction
Reverse the direction in the two sentences so that they match the Results, the Fig. 2 title and the fig. S4 title:
- Abstract: "Depletion of pancreatic-specific TASc attenuated the tumorigenesis of PDAC tumors." Alternatively: "Pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors, and their depletion attenuated it."
- Introduction, final paragraph: "Depletion of TASc attenuated the tumorigenesis of PDAC."
No change to the data, figures, statistics or conclusions is needed. The Fig. 2G legend could also give the WT group size, since it currently gives n for two of its three groups.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC9891701 (Sci Adv 2023, doi:10.1126/sciadv.add6995)
# Abstract and Introduction vs Results / Fig. 2: what does depleting tumor-associated
# nonmyelinating Schwann cells (TASc) do to PDAC tumorigenesis?
#
# Run: python recompute.py.txt
# Prints every number used in writeup.md and regenerates fig1.{svg,png} and fig2.{svg,png}
# in the folder that holds this file. Needs matplotlib.
#
# The finding is about the direction of an effect stated in words; there is no numeric
# value to recompute. The script therefore does three things:
# 1. holds a hand-curated list of every statement (sentence, Results subsection heading or
# figure title; Materials and Methods excluded) that states a direction for the effect of
# depleting, inhibiting or targeting TASc, or TASc-expressed PVT1, on tumor growth,
# tumorigenesis, survival or response to immunotherapy, each copied verbatim from the
# article as archived in PubMed Central and labelled with the direction it states.
# The list was built by reading every sentence of the Abstract, Introduction, Results and
# Discussion, every Results subsection heading and every figure title (see INCLUSION below);
# 2. if run inside the /home/ubuntu/papers repo, re-checks each sentence against the
# archived full text (src/show.py) and confirms it sits in the stated section;
# skipped silently elsewhere;
# 3. tallies the directions and draws the two figures.
# Group sizes (n) are transcribed from the Fig. 2 legend.
#
# Optional checks of the publisher's typeset PDF and the Supplementary Materials PDF, both
# held in the PMC Open Access dataset (version 1 is the only version):
# aws s3 cp --no-sign-request s3://pmc-oa-opendata/PMC9891701.1/PMC9891701.1.pdf ./
# aws s3 cp --no-sign-request s3://pmc-oa-opendata/PMC9891701.1/sciadv.add6995_sm.pdf ./
# pdftotext PMC9891701.1.pdf vor.txt ; pdftotext sciadv.add6995_sm.pdf sm.txt
# VOR_TXT=./vor.txt SUPP_TXT=./sm.txt python recompute.py.txt
# Without those variables the two checks are reported as "not run", not as passed.
import os, re, subprocess
from collections import Counter, OrderedDict
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
HERE = os.path.dirname(os.path.abspath(__file__))
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
# ------------------------------------------------------------------ statements
# INCLUSION: subject = depleting / inhibiting / targeting TASc (by AAV8-GFAP-DTA, fluorocitrate
# or "a small-molecule" agent) or TASc-expressed PVT1; predicate = a direction for tumor growth,
# tumorigenesis, survival or response to immunotherapy. Left out: statements about the
# presence (not removal) of TASc or PVT1 (see CONVERSE), deletion of PVT1 nucleotides, RAF1 or
# TDO2, immune-cell or TASc-abundance readouts only, cytotoxicity, safety or specificity of FC,
# and statements that TASc are a therapeutic target without a stated tumor outcome.
# key: (section as archived, direction, experiment, verbatim statement)
# direction "PRO" = depletion promotes tumorigenesis (protumor)
# direction "ANTI" = depletion inhibits tumor growth / improves outcome (antitumor)
# experiment: "DTA" = AAV8-GFAP-DTA depletion of Schwann cells in KPC mice (Fig. 2)
# "PVT1" = loss of PVT1 in TASc (Fig. 6), "FC" = fluorocitrate (Fig. 7)
STATEMENTS = OrderedDict([
("abs_pro", ("ABSTRACT", "PRO", "DTA",
"Depletion of pancreatic-specific TASc promoted the tumorigenesis of PDAC tumors.")),
("abs_pvt1", ("ABSTRACT", "ANTI", "PVT1",
"Depletion of TASc-expressed PVT1 suppressed PDAC tumor growth.")),
("abs_fc", ("ABSTRACT", "ANTI", "FC",
"Furthermore, depletion of TASc using a small-molecule inhibitor effectively sensitized "
"PDAC to immunotherapy, signifying the important roles of TASc in PDAC immune resistance.")),
("intro_pro", ("INTRODUCTION", "PRO", "DTA",
"Depletion of TASc promoted the tumorigenesis of PDAC.")),
("intro_pvt1", ("INTRODUCTION", "ANTI", "PVT1",
"Depletion of TASc-expressed PVT1 suppressed PDAC tumor growth.")),
("intro_fc", ("INTRODUCTION", "ANTI", "FC",
"Furthermore, using a small-molecule compound to inhibit TASc in vivo notably improved "
"the efficacy of immune checkpoint inhibitors.")),
("res_dta_surv", ("RESULTS > TASc promotes PDAC tumorigenesis", "ANTI", "DTA",
"KPC mice given AAV8-GFAP-DTA exhibited improved survival time compared to control group "
"(Fig. 2C).")),
("res_dta_growth", ("RESULTS > TASc promotes PDAC tumorigenesis", "ANTI", "DTA",
"Upon AAV8-GFAP-DTA–mediated Schwann cell depletion, KPC pancreases exhibited increased "
"percentage of PanIN 1, reduced PanIN 2/3, and inhibited tumor growth (Fig. 2, D to G).")),
("res_pvt1", ("RESULTS > TASc-expressed PVT1 facilitates PDAC tumorigenesis", "ANTI", "PVT1",
"on the contrary, depletion of Pvt1 in Schwann cells inhibited tumor growth (Fig. 6, A and B).")),
("res_fc_heading", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"ANTI", "FC",
"Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy")), # subsection heading
("res_fc_retraction", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"ANTI", "FC",
"Treatment with FC and control antibodies led to retraction of PDAC (Fig. 7, A to C).")),
("res_fc_combo", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"ANTI", "FC",
"Combining FC with the anti–PD-1 antibody significantly suppressed PDAC tumorigenesis in "
"comparison with anti–PD-1 treatment alone (Fig. 7, A to C, and fig. S13A).")),
("res_fc_summary", ("RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"ANTI", "FC",
"Furthermore, depletion of TASc in PDAC may profoundly diminish tumor growth and improve the "
"infiltration and cytotoxicity of CD8+ T cells.")),
("disc_restored", ("DISCUSSION", "ANTI", "PVT1+FC",
"Depletion of TASc-expressed Pvt1 or TASc using FC restored the immunoresistant metabolic "
"microenvironment and improved the antitumor effects of immune checkpoint inhibitors.")),
("disc_therapy", ("DISCUSSION", "ANTI", "PVT1+FC",
"Considering the critical role of TASc in immunosuppressive microenvironments, depleting TASc "
"or TASc-expressed PVT1 in PDAC may be a promising therapeutic modality.")),
("fig2_title", ("Fig. 2 legend", "ANTI", "DTA",
"Depletion of TASc attenuates pancreatic cancer tumorigenesis.")),
("fig7_title", ("Fig. 7 legend", "ANTI", "FC",
"TASc inhibition sensitizes PDAC to immunotherapy.")),
])
KIND = {"res_fc_heading": "heading", "fig2_title": "figure title", "fig7_title": "figure title"}
# Statements about the presence (not depletion) of TASc; these imply that depletion should
# reduce tumorigenesis. Shown for context only, not part of the tally.
CONVERSE = OrderedDict([
("res_heading", ("RESULTS > TASc promotes PDAC tumorigenesis", "Results subsection heading",
"TASc promotes PDAC tumorigenesis")),
("res_s5", ("RESULTS > TASc promotes PDAC tumorigenesis", "Results, same subsection, paragraph 2",
"Together, these results suggested that the presence of TASc in PDAC facilitates "
"tumorigenesis and resistance to immunotherapy.")),
])
# Supplementary Materials (sciadv.add6995_sm.pdf), figure titles that state a direction for
# depletion/inhibition/targeting of TASc. Counted separately from the main-text tally.
# fig. S16 is a schematic (graphic illustration) of the whole model; its title is counted too.
SUPP_TITLES = OrderedDict([
("figS4_title", ("fig. S4 title", "ANTI", "DTA", "Depletion of TASc inhibits PDAC tumorigenesis.")),
("figS13_title", ("fig. S13 title", "ANTI", "FC", "FC treatment sensitizes PDAC to immunotherapy.")),
("figS16_title", ("fig. S16 title (schematic)", "ANTI", "TASc targeting",
"Targeting tumor-associated non-myelinating Schwann cells improves anti-tumor immunity of "
"immunotherapy-refractory pancreatic cancer.")),
])
# Fig. 2 legend, panels C, F and G (transcribed)
N_FIG2C = {"WT": 8, "AAV8-GFAP-Blank": 12, "AAV8-GFAP-DTA": 13} # "n = 8, 12, and 13 mice"
N_FIG2G = {"AAV8-GFAP-Blank": 12, "AAV8-GFAP-DTA": 13} # "n = 12 and 13 animals"
FIG2_LEGEND_SNIPPETS = [
"Log-rank test; n = 8, 12, and 13 mice, respectively.",
"Error bars, SDs; n = 12 and 13 animals, respectively; Student’s t test.",
]
# ------------------------------------------------------------------ verification
def load_archived_text():
repo = "/home/ubuntu/papers"
py = "/home/ubuntu/papers-data/venv/bin/python"
if not (os.path.exists(os.path.join(repo, "src/show.py")) and os.path.exists(py)):
return None
try:
out = subprocess.run([py, "src/show.py", "PMC9891701", "--full"], cwd=repo,
capture_output=True, text=True, timeout=120).stdout
except Exception:
return None
return out or None
def sections(txt):
"""Map section label -> text, using the --full layout of src/show.py."""
secs, cur = {}, None
for line in txt.splitlines():
if line.startswith("ABSTRACT:"):
cur = "ABSTRACT"; line = line[len("ABSTRACT:"):]
elif line.startswith("## "):
cur = line[3:].strip(); secs[cur] = secs.get(cur, "") + cur + "\n"; continue # heading kept
else:
m = re.match(r"\[Fig\. (\d+)\.\]", line)
if m:
cur = f"Fig. {m.group(1)} legend"
if cur:
secs[cur] = secs.get(cur, "") + line + "\n"
return secs
def verify(txt):
secs = sections(txt)
ok = True
print("Verbatim check against the archived full text (src/show.py PMC9891701 --full):")
for key, (sec, *_rest, s) in list(STATEMENTS.items()) + [
(k, (v[0], v[2])) for k, v in CONVERSE.items()]:
total = txt.count(s)
in_sec = secs.get(sec, "").count(s)
flag = "ok" if in_sec >= 1 else "MISSING"
ok &= in_sec >= 1
print(f" [{flag}] {key:18s} found {total}x in full text, {in_sec}x in '{sec[:45]}'")
leg2 = secs.get("Fig. 2 legend", "")
for snip in FIG2_LEGEND_SNIPPETS:
c = leg2.count(snip)
ok &= c == 1
print(f" [{'ok' if c == 1 else 'MISSING'}] Fig. 2 legend: {snip}")
print(" all checks passed" if ok else " SOME CHECKS FAILED")
return ok
def norm_pdf_text(t):
t = re.sub(r"-\n", "-", t) # keep hyphens split across lines
return re.sub(r"\s+", " ", t)
def verify_pdfs():
vor, supp = os.environ.get("VOR_TXT"), os.environ.get("SUPP_TXT")
print("\nPublisher PDF (version of record) check:", end=" ")
if vor and os.path.exists(vor):
t = norm_pdf_text(open(vor, encoding="utf-8", errors="replace").read())
print()
for key in ("abs_pro", "intro_pro", "res_dta_surv", "res_dta_growth", "fig2_title"):
q = STATEMENTS[key][3]
n_full = t.count(q)
if n_full:
print(f" {key:18s} found {n_full}x")
else: # the PDF text layer can interleave a figure caption inside a sentence
tail = q[q.index(" ", 12) + 1:]
print(f" {key:18s} found 0x as one string; tail '{tail[:40]}...' found "
f"{t.count(tail)}x (sentence interrupted by a figure caption in the PDF text layer)")
else:
print("not run (set VOR_TXT)")
print("Supplementary Materials check:", end=" ")
if supp and os.path.exists(supp):
t = norm_pdf_text(open(supp, encoding="utf-8", errors="replace").read())
print()
for key, (loc, d, exp, q) in SUPP_TITLES.items():
print(f" {key:18s} found {t.count(q)}x ({loc}: {q})")
n_pro = len(re.findall(r"(?i)deplet[^.]{0,80}promot[^.]{0,40}tumorigen", t))
print(f" sentences of the form 'deplet... promot... tumorigen...' in supplement: {n_pro}")
else:
print("not run (set SUPP_TXT)")
# ------------------------------------------------------------------ tally
def tally():
print("\nDirection stated for the effect of depleting/inhibiting/targeting TASc or TASc-expressed PVT1:")
for key, (sec, d, exp, s) in STATEMENTS.items():
lab = "depletion PROMOTES tumor" if d == "PRO" else "depletion INHIBITS tumor"
print(f" {key:18s} {sec[:40]:40s} {KIND.get(key, 'sentence'):12s} {exp:8s} {lab}")
c = Counter(d for _, d, _, _ in STATEMENTS.values())
n = len(STATEMENTS)
kinds = Counter(KIND.get(k, "sentence") for k in STATEMENTS)
print(f"\nTotal directional statements (main text): {n} ({dict(kinds)})")
print(f" depletion promotes tumorigenesis: {c['PRO']} "
f"({', '.join(k for k, v in STATEMENTS.items() if v[1] == 'PRO')})")
print(f" depletion inhibits tumor / improves outcome: {c['ANTI']}")
dta = [(k, v) for k, v in STATEMENTS.items() if v[2] == "DTA"]
cd = Counter(v[1] for _, v in dta)
print(f"Statements about the AAV8-GFAP-DTA depletion experiment itself: {len(dta)} "
f"({cd['PRO']} promotes [{', '.join(k for k, v in dta if v[1] == 'PRO')}], "
f"{cd['ANTI']} inhibits [{', '.join(k for k, v in dta if v[1] == 'ANTI')}])")
allst = list(STATEMENTS.values()) + list(SUPP_TITLES.values())
ca = Counter(v[1] for v in allst)
dta_all = [v for v in allst if v[2] == "DTA"]
cda = Counter(v[1] for v in dta_all)
print(f"Adding the supplementary figure titles ({', '.join(v[0] for v in SUPP_TITLES.values())}): "
f"{len(allst)} statements, {ca['PRO']} promote, {ca['ANTI']} inhibit; "
f"DTA experiment {len(dta_all)} statements, {cda['PRO']} promote, {cda['ANTI']} inhibit")
nos16 = [v for k, v in SUPP_TITLES.items() if k != "figS16_title"]
cn = Counter(v[1] for v in list(STATEMENTS.values()) + nos16)
print(f" (without the fig. S16 schematic title: {n + len(nos16)} statements, "
f"{cn['PRO']} promote, {cn['ANTI']} inhibit)")
fc = [v for v in STATEMENTS.values() if v[2] == "FC"]
print(f"Fluorocitrate/small-molecule statements (main text): {len(fc)}, all "
f"{'antitumor' if all(v[1] == 'ANTI' for v in fc) else 'MIXED'}")
pvt = [v for v in STATEMENTS.values() if v[2] == "PVT1"]
print(f"PVT1-loss statements (main text): {len(pvt)}, all "
f"{'antitumor' if all(v[1] == 'ANTI' for v in pvt) else 'MIXED'}")
both = [v for v in STATEMENTS.values() if v[2] == "PVT1+FC"]
print(f"Statements naming both PVT1 loss and FC (main text): {len(both)}, all "
f"{'antitumor' if all(v[1] == 'ANTI' for v in both) else 'MIXED'}")
pro_secs = sorted({v[0] for v in STATEMENTS.values() if v[1] == "PRO"})
print(f"Sections containing a 'promotes' statement: {pro_secs}")
print(f"\nFig. 2C group sizes (log-rank): {N_FIG2C}; total mice = {sum(N_FIG2C.values())}")
print(f"Fig. 2G group sizes (t test, tumor volume at 22 weeks): {N_FIG2G}; "
f"total = {sum(N_FIG2G.values())}")
return c, dta
# ------------------------------------------------------------------ figures
def style():
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.edgecolor": SEC, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "path", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG,
})
def panel(ax, x, y, w, h, color, title, sub, body):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
lw=2, ec=color, fc="white"))
ax.text(x + 0.02, y + h - 0.035, title, fontsize=12, fontweight="bold", color=color, va="top")
ax.text(x + 0.02, y + h - 0.095, sub, fontsize=10, color=SEC, va="top", style="italic")
yy = y + h - 0.165
for item in body:
txt, kw = item if isinstance(item, tuple) else (item, {})
ax.text(x + 0.02, yy, txt, fontsize=kw.get("size", 10.5), color=kw.get("color", TXT),
fontweight=kw.get("weight", "normal"), va="top", linespacing=1.3)
yy -= kw.get("step", 0.12)
def fig1(n2c, n2g):
fig = plt.figure(figsize=(7.2, 5.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.text(0.03, 0.98, "Effect of TASc depletion on PDAC: two directions for one experiment",
fontsize=11.5, fontweight="bold", va="top")
panel(ax, 0.03, 0.255, 0.41, 0.66, ORANGE,
"Abstract and Introduction",
"summary of the depletion result",
[("Abstract, sentence 4:", {"color": SEC, "size": 10, "step": 0.045}),
("“Depletion of pancreatic-specific\nTASc promoted the tumorigenesis\nof PDAC tumors.”",
{"step": 0.17}),
("Introduction, final paragraph:", {"color": SEC, "size": 10, "step": 0.045}),
("“Depletion of TASc promoted the\ntumorigenesis of PDAC.”", {"step": 0.12}),
("Stated: depletion is protumor", {"color": ORANGE, "weight": "bold",
"size": 10.5})])
panel(ax, 0.56, 0.255, 0.41, 0.66, BLUE,
"Results and Fig. 2",
"AAV8-GFAP-DTA depletion in KPC mice",
[("Fig. 2C (log-rank; n = %d, %d, %d):" % tuple(n2c.values()),
{"color": SEC, "size": 10, "step": 0.045}),
("“improved survival time compared\nto control group”", {"step": 0.095}),
("Fig. 2D–G (volumes: n = %d vs %d):" % tuple(n2g.values()),
{"color": SEC, "size": 10, "step": 0.045}),
("“reduced PanIN 2/3, and inhibited\ntumor growth”", {"step": 0.095}),
("Fig. 2 title:", {"color": SEC, "size": 10, "step": 0.045}),
("“Depletion of TASc attenuates\npancreatic cancer tumorigenesis.”", {"step": 0.11}),
("Shown: depletion is antitumor", {"color": BLUE, "weight": "bold",
"size": 10.5})])
arr = FancyArrowPatch((0.448, 0.585), (0.552, 0.585), arrowstyle="<|-|>", mutation_scale=16,
lw=2, color=TXT)
ax.add_patch(arr)
ax.text(0.5, 0.615, "opposite\ndirection", ha="center", va="bottom", fontsize=10,
fontweight="bold", linespacing=1.15)
ax.text(0.03, 0.205,
"The paper's only pancreas-specific depletion of TASc is this experiment: AAV8-GFAP-DTA\n"
"delivered into the pancreas of KPC mice, compared with AAV8-GFAP-Blank controls.",
fontsize=10, color=TXT, va="top", linespacing=1.35)
ax.text(0.03, 0.10,
"Elsewhere the paper agrees with the right panel: Results heading “TASc promotes PDAC\n"
"tumorigenesis”; PVT1 loss in TASc and fluorocitrate depletion both reduce tumor growth.",
fontsize=10, color=SEC, va="top", linespacing=1.35)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)
def fig2():
order = ["ABSTRACT", "INTRODUCTION",
"RESULTS > TASc promotes PDAC tumorigenesis",
"RESULTS > TASc-expressed PVT1 facilitates PDAC tumorigenesis",
"RESULTS > Targeting TASc by fluorocitrate sensitizes PDAC to immunotherapy",
"DISCUSSION", "Fig. 2 legend", "Fig. 7 legend"]
labels = ["Abstract", "Intro-\nduction", "Results\nDTA\n(Fig. 2)", "Results\nPVT1\n(Fig. 6)",
"Results\nFC\n(Fig. 7)", "Discus-\nsion", "Fig. 2\ntitle", "Fig. 7\ntitle"]
expmark = {"DTA": "o", "PVT1": "s", "FC": "D", "PVT1+FC": "^"}
fig, ax = plt.subplots(figsize=(7.2, 3.8))
fig.subplots_adjust(left=0.17, right=0.98, top=0.83, bottom=0.27)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
counts = Counter()
for key, (sec, d, exp, _) in STATEMENTS.items():
xi = order.index(sec)
yi = 1 if d == "PRO" else 0
off = counts[(xi, yi)]; counts[(xi, yi)] += 1
col = ORANGE if d == "PRO" else BLUE
ax.plot(xi, yi + 0.13 * off - 0.0, marker=expmark[exp], ms=10, color=col,
mec="white", mew=1, ls="none", zorder=3)
ax.set_xticks(range(len(order))); ax.set_xticklabels(labels, fontsize=10)
ax.set_yticks([0, 1])
ax.set_yticklabels(["depletion\ninhibits\ntumor", "depletion\npromotes\ntumor"], fontsize=10)
ax.tick_params(axis="y", length=0)
ax.set_ylim(-0.35, 1.45); ax.set_xlim(-0.6, len(order) - 0.4)
c = Counter(v[1] for v in STATEMENTS.values())
ax.text(-0.5, 1.3, f"{c['PRO']} statements (orange): the Abstract and Introduction sentences",
color=ORANGE, fontsize=10, fontweight="bold", va="center")
ax.text(1.6, 0.62, f"{c['ANTI']} statements (blue), in every section",
color=BLUE, fontsize=10, fontweight="bold", va="center")
# marker legend for experiment
from matplotlib.lines import Line2D
h = [Line2D([], [], marker=m, color=SEC, ls="none", ms=8, label=l) for l, m in
[("DTA depletion", "o"), ("PVT1 loss", "s"), ("fluorocitrate", "D"), ("PVT1 or FC", "^")]]
ax.legend(handles=h, ncol=4, loc="lower left", bbox_to_anchor=(-0.2, 1.02), frameon=False,
fontsize=10, handletextpad=0.3, columnspacing=1.0)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
plt.close(fig)
if __name__ == "__main__":
txt = load_archived_text()
if txt:
verify(txt)
else:
print("(archived text not available here; verbatim check skipped)")
verify_pdfs()
tally()
style()
fig1(N_FIG2C, N_FIG2G)
fig2()
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Adjusted odds ratio for discharge to rehabilitation, 2.45, lies outside its printed 95% CI of 0.81–1.25
Stroke (2020) · doi:10.1161/STROKEAHA.120.031217 · PMC7678670
Summary
The abstract, the Results text and Table 3 all give the adjusted odds ratio for discharge to rehabilitation among patients with stroke, with versus without COVID-19, as 2.45 (95% CI 0.81–1.25), P = 0.103. The interval does not contain its own estimate: its geometric midpoint, where a logistic-regression Wald interval puts the estimate, is 1.006. The printed estimate, lower limit and P value are consistent with each other and imply an upper limit of about 7.2–7.4, matching the width of the paper's unadjusted interval (0.75–6.38). The result stays non-significant and in the same direction, and the primary and mortality results are unaffected, but the printed interval understates the standard error about fivefold.
What the paper reports
Abstract, Results:
In patients presenting with stroke, concurrent infection with severe acute respiratory syndrome-coronavirus 2 (SARS-CoV-2) was associated with higher case-fatality (odds ratio, 10.50 [95% CI, 3.54–31.18]) and a trend towards increased occurrence of discharge to rehabilitation (odds ratio, 2.45 [95% CI, 0.81–1.25]).
Results, section "Discharge to Rehabilitation for Ischemic Stroke":
Overall, 60% (1511/2513) of patients with stroke with COVID-19 were discharged to rehabilitation, compared with 40.7% (9074/22 295) of patients with stroke without SARS-CoV-2 infection. COVID-19 was associated with a trend towards an increased frequency of discharge to rehabilitation (OR, 2.19 [95% CI, 0.75–6.38]) in unadjusted analysis. Similarly, using a logistic regression with facility fixed effects, we identified that SARS-CoV-2 infection was associated with a trend towards an increased frequency of discharge to rehabilitation (OR, 2.45 [95% CI, 0.81–1.25]), compared with noninfected individuals, albeit nonsignificant (Table 3).
Table 3, "Models Examining the Association of COVID-19 With Unfavorable Outcomes in Patients With Acute Ischemic Stroke". PMC archives this table only as a typeset image, so the values below were transcribed from that image:
Unadjusted analysis: Mortality 9.68 (3.55–26.4), P <0.001; Discharge to rehabilitation 2.19 (0.75–6.38), P 0.143
Logistic regression†: Mortality 10.50 (3.54–31.18), P <0.001; Discharge to rehabilitation 2.45 (0.81–1.25), P 0.103
†Hospital ID was used as a fixed-effects variable.
Methods, "Statistical Analysis":
For our secondary outcomes, given the limited df (relatively small number of observations), we used a step-wise logistic regression model controlling for age, sex, hypertension, hypercholesterolemia, diabetes, and peripheral vascular disease.
All results are based on 2-sided tests, and the level of statistical significance was set at 0.05.
Discussion, first paragraph:
Patients with stroke with concurrent SARS-CoV-2 infection demonstrated increased case-fatality and a trend more discharge to rehabilitation.
Conclusions:
Patients with stroke with concurrent SARS-CoV-2 infection demonstrated increased case-fatality and discharge to rehabilitation.
What the paper's own numbers show
Method. A logistic-regression Wald interval is exp(β ± 1.96·SE). It is symmetric on the log scale, so the odds ratio equals the geometric mean of the two bounds and OR² = lower × upper. I applied this check to all 20 odds ratios in Tables 2 and 3. I then rebuilt the adjusted rehabilitation interval in two ways. The first keeps the printed OR and lower bound. The second keeps the printed OR and P value. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Does the adjusted rehabilitation CI contain its estimate? | Abstract; Results; Table 3 | OR 2.45, CI 0.81–1.25 | No: 2.45 is 1.20 above the upper limit. An OR that rounds to 2.45 is at least 2.445, and an upper limit that rounds to 1.25 is below 1.255, so rounding cannot account for the gap |
| Geometric midpoint of the printed bounds | same | (should equal 2.45) | √(0.81 × 1.25) = 1.006 |
| Upper limit implied by OR 2.45 and lower limit 0.81 | Table 3 | 1.25 | 2.45² / 0.81 = 7.41; SE 0.565; implied P = 0.113 |
| 95% CI implied by OR 2.45 and printed P = 0.103 | Table 3 | 0.81–1.25 | SE 0.550 → 0.83–7.19 |
| 95% CI if the adjusted model had the unadjusted model's SE (0.546) | Table 3 | 0.81–1.25 | 0.84–7.15 |
| Unadjusted rehabilitation OR | Results; Table 3 | 2.19 (0.75–6.38), P = 0.143 | midpoint 2.187; implied P 0.151 (consistent) |
| Unadjusted mortality OR | Results; Table 3 | 9.68 (3.55–26.4) | midpoint 9.681 (consistent) |
| Adjusted mortality OR | Abstract; Results; Table 3 | 10.50 (3.54–31.18) | midpoint 10.506 (consistent) |
| Stroke, ischemic stroke, TIA and MI ORs, 16 rows | Results; Table 2 | e.g. 0.25 (0.16–0.40); MI 0.87 (0.68–1.12), P 0.264 | all contain their estimates; midpoints match the ORs to within rounding except TIA mixed-effects (below); MI implied P values 0.274, 0.379, 0.340 vs printed 0.264, 0.382, 0.341 (MI inverse-weighted: see below) |
| Crude stroke OR | Results, "COVID-19 and Ischemic Stroke" | 0.35 (0.23–0.55) | 22/2491 vs 544/21 751 → 0.353 (0.23–0.54) |
The adjusted rehabilitation entry is the only one of the 20 checked odds ratios whose interval excludes its estimate. The two reconstructions agree closely (upper limit 7.19 vs 7.41; lower limit 0.83 vs printed 0.81), so the printed lower limit and P value fit OR 2.45, and the upper limit 1.25 is the number that does not. The reconstructed width also matches the unadjusted model's interval (0.75–6.38), as one would expect for two models fitted to the same 566 patients with few events. The exact correct upper limit cannot be fixed without the model output. A value of 7.25, which differs from 1.25 in one digit, would give a midpoint of 2.42 and an implied P of 0.109, but this is only one possibility.
Side observations (not part of the main finding).
- The abstract gives "566 (0.2%)" patients with acute ischemic stroke among 24 808 discharges. 566 / 24 808 = 2.28%.
- The rehabilitation paragraph gives 60% (1511/2513) and 40.7% (9074/22 295) "of patients with stroke". The case-fatality paragraph gives 31.8% (799/2513) and 4.6% (1026/22 295). In each case the denominators are the totals of all COVID-19-positive and COVID-19-negative discharges, not the 22 and 544 stroke patients, and every numerator is larger than the matching stroke count. The percentages fit stroke patients. For example, 7/22 = 31.8% and 25/544 = 4.6% give an unadjusted mortality OR of 9.69. Among the 15 and 519 survivors, 9/15 = 60.0% and 211/519 = 40.7% give an unadjusted rehabilitation OR of 2.19, with Wald P = 0.143 and CI 0.77–6.24. These counts are one consistent reconstruction, not the paper's data. Taken literally, the printed counts 1511/2513 vs 9074/22 295 would give OR 2.20 (2.02–2.39), an interval far narrower than the one reported.
- Table 2 and the Sensitivity Analysis give the mixed-effects TIA-only OR as 0.14 (0.10–0.38). The interval contains the estimate, but its geometric midpoint is 0.195, so it is not symmetric on the log scale the way the other rows are. Its conclusion (P < 0.001) is not in question.
- Table 2 gives the inverse-weighted MI OR as 0.84 (0.75–0.94) with P = 0.025. Across every value that rounds to the printed estimate and bounds, that interval implies a Wald P between 0.0009 and 0.0055. Both are below 0.05, so the conclusion for this row does not change.
Figure
Figure 1. Odds ratios for discharge to rehabilitation, COVID-19 vs no COVID-19, among patients with ischemic stroke, with 95% CIs on a log scale. The thin vertical line marks OR = 1. Orange: the values printed in the paper. The unadjusted model is 2.19 (0.75–6.38). The adjusted model is printed as 2.45 (0.81–1.25): the filled dot is the printed estimate, the bar is the printed interval, and the open circle is the interval's geometric midpoint (1.01), where a Wald interval puts its estimate. Blue: the adjusted interval rebuilt from the paper's own numbers. The first is rebuilt from OR 2.45 and the lower limit 0.81 by log symmetry (0.81–7.41, implied P = 0.113). The second is rebuilt from OR 2.45 and the printed P = 0.103 (0.83–7.19). Both rebuilt intervals have about the same width as the unadjusted one and both include 1.
Figure 2. The standard error of log(OR) implied by each 95% CI in Table 3, computed as (ln upper − ln lower) / 3.92. Orange: intervals as printed. Blue: the two reconstructed adjusted rehabilitation intervals from Figure 1. Of the four models fitted to the same 566 stroke patients (22 with COVID-19), three imply SEs between 0.512 and 0.555; the printed adjusted rehabilitation interval implies 0.111. The reconstructed intervals imply 0.565 and 0.550, in line with the other three. The axis starts at zero.
Why it matters
- A value in the abstract is impossible as printed. The pair 2.45 (0.81–1.25) appears identically in the abstract, the Results and Table 3. A reader cannot tell from the paper which of the three numbers is wrong.
- The printed interval misstates precision. Taken at face value, 0.81–1.25 describes a fairly precise estimate close to no effect. The reconstructed interval, roughly 0.8 to 7.2–7.4, is wide and fits a comparison with only 22 COVID-19-positive stroke patients. Anyone pooling this estimate in a meta-analysis, or deriving an SE from the printed bounds, would get an SE of 0.111 instead of about 0.55–0.57, giving this study about 25 times its proper inverse-variance weight (24.7 to 26.0).
- The paper's qualitative statements on this outcome do not change. Both the printed and the reconstructed intervals include 1, and the printed P = 0.103 is above the paper's 0.05 threshold, so the abstract's "trend" wording holds. The Conclusions sentence, which says that patients "demonstrated increased case-fatality and discharge to rehabilitation", drops the word "trend". That sentence overstates a non-significant result whichever interval is correct, independent of this error.
- The main results are not affected. The primary finding (adjusted OR 0.25, 0.16–0.40, for stroke with COVID-19) and the case-fatality results (9.68 and 10.50) have internally consistent intervals and do not depend on this value.
Explanations considered
- Text-conversion or typesetting artifact in the archive. Ruled out. The same three numbers appear in the abstract, in the Results text and in the typeset image of Table 3 deposited by the publisher. No minus signs, superscripts, decimal commas or column shifts are involved.
- Rounding. Ruled out. Rounding moves a value by at most 0.005, and the gap between the estimate and the upper limit is 1.20.
- An interval method that is not symmetric on the log scale (profile likelihood, exact, bootstrap). Ruled out. In an ordinary logistic fit these methods give intervals that contain the estimate, and the printed interval does not. Of the paper's other 19 intervals, 18 are symmetric on the log scale to within rounding, so they appear to be Wald intervals.
- The interval belongs to a different model or quantity (for example, the sensitivity model with tPA and thrombectomy, or a risk difference). This does not fit the paper's own numbers. The Methods say the sensitivity models "did not change" direction and are "not reported any further". The printed P = 0.103 fits OR 2.45 with an SE of about 0.55, not the interval 0.81–1.25, which implies an SE of 0.111 and a midpoint of 1.006. An OR of 1.006 with that SE would give P = 0.96, not 0.103.
- The estimate 2.45 is the wrong number and the interval is right. Unlikely. In that case both the estimate and P = 0.103 would be wrong, since the printed interval implies P = 0.96 at its own midpoint of 1.006. The printed lower limit, the P value and the unadjusted model all agree with an OR near 2.4 and an interval about as wide as 0.8–7.3. A single mistyped upper limit accounts for every printed number.
- The supplement or a later correction gives the correct value. No. The Data Supplement contains no secondary-outcome models, and no correction was found (see Status).
Status
- Version of record. The publisher's site (ahajournals.org, full text and PDF) refused automated access (HTTP 403). The text checked is the publisher-deposited version in PubMed Central and Europe PMC (PMC7678670), including the typeset Table 3 image (str-51-3570-g004). That image was opened directly and shows "2.45 (0.81–1.25)" with P = 0.103. The abstract and the Results carry the same values.
- Supplement. The Data Supplement (str-51-3570-s001.pdf, 5 pages, from the Europe PMC supplementary-files bundle) contains Table A-I (ICD-10 definitions), Figure A-I (flowchart), Figure A-II (forest plot for the stroke-occurrence model) and Figure A-III (weekly scatterplot). It has no secondary-outcome models and does not resolve the discrepancy. No source data are deposited. The Methods say data "will be made available by contacting the corresponding author".
- Corrections and notices. None found. Crossref has no update-to or updated-by records and no relations for the DOI. PMC and Europe PMC list no correction, erratum, expression of concern or retraction for this article. The PMC record does link to an expression of concern and a retraction, but both concern a different article that this paper cites (N Engl J Med, doi:10.1056/NEJMoa2007621). They are not notices about this paper.
- PubPeer and the web. PubPeer's search endpoint refused automated access (HTTP 403). Web searches for the title and the DOI together with PubPeer, correction and erratum found no PubPeer thread or published correction. No preprint version with these results was found.
- Checked as of 2026-10-24.
Suggested correction
The adjusted odds ratio for discharge to rehabilitation should be reported with its correct 95% confidence interval in the abstract, in the Results ("Discharge to Rehabilitation for Ischemic Stroke") and in Table 3. If the printed estimate (2.45), lower limit (0.81) and P value (0.103) are right, the upper limit should be about 7.2–7.4, not 1.25. The exact value should be taken from the original model output. If another of the three numbers is the one in error, all three should be checked against that output. The Conclusions sentence "Patients with stroke with concurrent SARS-CoV-2 infection demonstrated increased case-fatality and discharge to rehabilitation" could be aligned with the abstract and Discussion, which call the rehabilitation result a non-significant trend. Separately, the denominators in the rehabilitation and case-fatality paragraphs (2513 and 22 295) and the abstract's "566 (0.2%)" (566/24 808 = 2.3%) could be checked.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC7678670 write-up and regenerate fig1.svg / fig1.png.
Paper: "Ischemic Stroke Occurs Less Frequently in Patients With COVID-19",
Stroke 2020;51:3570-3576, doi:10.1161/STROKEAHA.120.031217 (PMC7678670).
All inputs below are copied from the abstract, Results text and Table 2 / Table 3 of the paper.
Run: python recompute.py.txt
"""
import os
import numpy as np
from scipy.stats import norm
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975) # 1.95996
def wald_from_ci(o, lo, hi):
"""Geometric midpoint, SE(log OR) and implied two-sided Wald P of a printed OR (95% CI)."""
gm = np.sqrt(lo * hi)
se = (np.log(hi) - np.log(lo)) / (2 * Z)
p = 2 * norm.sf(abs(np.log(o)) / se)
return gm, se, p
def or_2x2(a, b, c, d):
"""Odds ratio (a/b)/(c/d) with Woolf (Wald) 95% CI and two-sided Wald P."""
o = (a / b) / (c / d)
se = np.sqrt(1 / a + 1 / b + 1 / c + 1 / d)
p = 2 * norm.sf(abs(np.log(o)) / se)
return o, o * np.exp(-Z * se), o * np.exp(Z * se), se, p
print("=" * 78)
print("1. The adjusted OR for discharge to rehabilitation (Abstract, Results, Table 3)")
print("=" * 78)
o, lo, hi, p_rep = 2.45, 0.81, 1.25, 0.103
print(f"Printed: OR {o} (95% CI {lo}-{hi}), P = {p_rep}")
print(f"Estimate inside its own interval? {lo <= o <= hi} (2.45 - 1.25 = {o - hi:.2f})")
print(f"Smallest OR that rounds to 2.45 = 2.445; largest upper bound that rounds to 1.25 < 1.255")
gm = np.sqrt(lo * hi)
print(f"Geometric midpoint of printed bounds sqrt(0.81*1.25) = {gm:.3f}")
print(f"Ratio of OR to that midpoint = {o / gm:.2f}")
print(f"Printed interval width on log scale: ln(1.25/0.81) = {np.log(hi / lo):.3f}"
f" -> SE {np.log(hi / lo) / (2 * Z):.3f}")
se_pr = np.log(hi / lo) / (2 * Z)
print(f"P implied for the printed interval's own midpoint ({gm:.3f}) with SE {se_pr:.3f}: "
f"{2 * norm.sf(np.log(gm) / se_pr):.2f}")
# Reconstruction A: keep OR and lower bound, use log symmetry of a Wald interval
hi_A = o ** 2 / lo
se_A = (np.log(o) - np.log(lo)) / Z
p_A = 2 * norm.sf(np.log(o) / se_A)
print(f"\nA. Keep OR 2.45 and lower 0.81 (log-symmetric): upper = 2.45^2/0.81 = {hi_A:.2f}; "
f"SE = {se_A:.3f}; implied P = {p_A:.3f}")
# Reconstruction B: keep OR and the printed P = 0.103
se_B = np.log(o) / norm.isf(p_rep / 2)
lo_B, hi_B = o * np.exp(-Z * se_B), o * np.exp(Z * se_B)
print(f"B. Keep OR 2.45 and P 0.103: SE = {se_B:.3f}; 95% CI = {lo_B:.2f}-{hi_B:.2f}")
# Reconstruction C: borrow SE of the unadjusted model
gm_u, se_u, p_u = wald_from_ci(2.19, 0.75, 6.38)
lo_C, hi_C = o * np.exp(-Z * se_u), o * np.exp(Z * se_u)
print(f"C. Use the unadjusted model's SE ({se_u:.3f}): 95% CI = {lo_C:.2f}-{hi_C:.2f}")
# A one-character candidate (1.25 -> 7.25)
gm_725, se_725, p_725 = wald_from_ci(o, lo, 7.25)
print(f"Candidate upper 7.25: geometric midpoint {gm_725:.2f}, implied P {p_725:.3f}")
print(f"All reconstructions contain 1: {min(lo, lo_B, lo_C) < 1}")
print(f"Inverse-variance weight ratio, printed SE vs reconstructed SE: "
f"(0.550/{se_pr:.3f})^2 = {(se_B / se_pr) ** 2:.1f}, (0.565/{se_pr:.3f})^2 = {(se_A / se_pr) ** 2:.1f}")
print("\n" + "=" * 78)
print("2. The other ORs in Tables 2 and 3: are they consistent with their own intervals?")
print("=" * 78)
rows = [
("Table 2 stroke/TIA unadjusted", 0.35, 0.23, 0.55, "<0.001"),
("Table 2 stroke/TIA mixed-effects", 0.25, 0.16, 0.40, "<0.001"),
("Table 2 stroke/TIA PS-matched", 0.24, 0.15, 0.38, "<0.001"),
("Table 2 stroke/TIA inverse-weighted", 0.33, 0.28, 0.39, "<0.001"),
("Table 2 ischemic stroke unadjusted", 0.40, 0.24, 0.64, "<0.001"),
("Table 2 ischemic stroke mixed-effects", 0.29, 0.18, 0.48, "<0.001"),
("Table 2 ischemic stroke PS-matched", 0.28, 0.17, 0.46, "<0.001"),
("Table 2 ischemic stroke inverse-weighted", 0.37, 0.30, 0.45, "<0.001"),
("Table 2 TIA unadjusted", 0.20, 0.07, 0.54, "<0.001"),
("Table 2 TIA mixed-effects", 0.14, 0.10, 0.38, "<0.001"),
("Table 2 TIA PS-matched", 0.12, 0.04, 0.33, "<0.001"),
("Table 2 TIA inverse-weighted", 0.18, 0.13, 0.27, "<0.001"),
("Table 2 MI unadjusted", 0.87, 0.68, 1.12, "0.264"),
("Table 2 MI mixed-effects", 0.89, 0.69, 1.16, "0.382"),
("Table 2 MI PS-matched", 0.88, 0.68, 1.15, "0.341"),
("Table 2 MI inverse-weighted", 0.84, 0.75, 0.94, "0.025"),
("Table 3 mortality unadjusted", 9.68, 3.55, 26.4, "<0.001"),
("Table 3 mortality adjusted", 10.50, 3.54, 31.18, "<0.001"),
("Table 3 rehabilitation unadjusted", 2.19, 0.75, 6.38, "0.143"),
("Table 3 rehabilitation adjusted", 2.45, 0.81, 1.25, "0.103"),
]
for lbl, oo, l, h, pr in rows:
g, s, pp = wald_from_ci(oo, l, h)
inside = l <= oo <= h
print(f"{lbl:40s} OR {oo:5.2f} ({l}-{h}) inside={str(inside):5s} "
f"geo-mid {g:6.3f} implied P {pp:.3f} printed P {pr}")
print(f"Rows checked: {len(rows)}; rows whose interval excludes the estimate: "
f"{[r[0] for r in rows if not (r[2] <= r[1] <= r[3])]}")
# MI inverse-weighted: printed P 0.025 vs P implied by its interval, over every rounding of the printed values
ps = [wald_from_ci(oo, l, h)[2] for oo in (0.835, 0.845) for l in (0.745, 0.755) for h in (0.935, 0.945)]
print(f"MI inverse-weighted, implied P over the rounding range of 0.84 (0.75-0.94): "
f"{min(ps):.4f}-{max(ps):.4f} [printed 0.025]")
print("\n" + "=" * 78)
print("3. Counts behind the secondary outcomes (side observations)")
print("=" * 78)
N, ncov = 24808, 2513
nneg = N - ncov
print(f"24808 - 2513 = {nneg}")
print(f"Abstract '566 (0.2%)': 566/24808 = {566 / N * 100:.2f}%")
print(f"2513/24808 = {ncov / N * 100:.1f}%; 22/2513 = {22 / ncov * 100:.2f}%; 544/22295 = {544 / nneg * 100:.2f}%")
o1, l1, h1, _, _ = or_2x2(22, ncov - 22, 544, nneg - 544)
print(f"Crude stroke OR 22/2491 vs 544/21751 = {o1:.3f} ({l1:.2f}-{h1:.2f}) [printed 0.35 (0.23-0.55)]")
print(f"Printed rehab fractions: 1511/2513 = {1511 / 2513 * 100:.1f}%, 9074/22295 = {9074 / nneg * 100:.1f}%")
print(f"Printed death fractions: 799/2513 = {799 / 2513 * 100:.1f}%, 1026/22295 = {1026 / nneg * 100:.1f}%")
print("Stroke patients: 22 with COVID-19, 544 without; 1511 > 22 and 9074 > 544.")
oc, lc, hc, _, pc = or_2x2(1511, 2513 - 1511, 9074, nneg - 9074)
print(f"OR from the printed rehab counts 1511/2513 vs 9074/22295 = {oc:.2f} ({lc:.2f}-{hc:.2f}) [printed unadjusted 2.19 (0.75-6.38)]")
# Deaths among stroke patients that reproduce the printed percentages
d1, d0 = 7, 25
print(f"\nDeaths {d1}/22 = {d1 / 22 * 100:.1f}% and {d0}/544 = {d0 / 544 * 100:.1f}% [printed 31.8% and 4.6%]")
om, lm, hm, sem, pm = or_2x2(d1, 22 - d1, d0, 544 - d0)
print(f" unadjusted OR = {om:.2f} ({lm:.2f}-{hm:.2f}), P = {pm:.2g} [printed 9.68 (3.55-26.4), P<0.001]")
# Rehabilitation among survivors that reproduces the printed percentages
s1, s0 = 22 - d1, 544 - d0
r1, r0 = 9, 211
print(f"Survivors {s1} and {s0}; rehabilitation {r1}/{s1} = {r1 / s1 * 100:.1f}% and {r0}/{s0} = {r0 / s0 * 100:.1f}%"
f" [printed 60% and 40.7%]")
orh, lrh, hrh, serh, prh = or_2x2(r1, s1 - r1, r0, s0 - r0)
print(f" unadjusted OR = {orh:.2f} ({lrh:.2f}-{hrh:.2f}), SE {serh:.3f}, P = {prh:.3f}"
f" [printed 2.19 (0.75-6.38), P=0.143]")
print(f" printed unadjusted interval: geo-mid {gm_u:.3f}, SE {se_u:.3f}, implied P {p_u:.3f}")
# Same percentages applied to all stroke patients (not only survivors), for comparison
r1b, r0b = round(0.60 * 22), round(0.407 * 544)
ob, lb, hb, _, pb = or_2x2(r1b, 22 - r1b, r0b, 544 - r0b)
print(f"Rehabilitation as share of all stroke patients {r1b}/22 vs {r0b}/544: OR {ob:.2f} ({lb:.2f}-{hb:.2f}), P {pb:.3f}")
# ---------------------------------------------------------------- figure
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "none",
"text.color": TXT, "axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC})
rowsF = [
# y-label, OR, lo, hi, kind, value text
("Unadjusted\n(printed)", 2.19, 0.75, 6.38, "rep", "2.19 (0.75–6.38)\nP = 0.143"),
("Adjusted\n(printed)", 2.45, 0.81, 1.25, "rep", "2.45 (0.81–1.25)\nP = 0.103"),
("Adjusted, rebuilt from\nOR and lower bound", 2.45, 0.81, hi_A, "rec",
f"2.45 (0.81–{hi_A:.2f})\nP = {p_A:.3f}"),
("Adjusted, rebuilt from\nOR and P = 0.103", 2.45, lo_B, hi_B, "rec",
f"2.45 ({lo_B:.2f}–{hi_B:.2f})\nP = 0.103"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
fig.subplots_adjust(left=0.235, right=0.775, top=0.93, bottom=0.24)
ax.set_facecolor(BG)
ys = np.arange(len(rowsF))[::-1].astype(float)
for (lbl, oo, l, h, kind, ann), y in zip(rowsF, ys):
col = ORANGE if kind == "rep" else BLUE
ax.plot([l, h], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
for b in (l, h):
ax.plot([b, b], [y - 0.13, y + 0.13], color=col, lw=2, zorder=3)
ax.plot(oo, y, "o", ms=9, color=col, mec=col, zorder=4)
ax.text(1.03, y, ann, transform=ax.get_yaxis_transform(), va="center", ha="left",
fontsize=10, color=TXT)
y_adj = ys[1]
# dotted gap between the printed upper bound and the printed estimate
ax.plot([1.25, 2.45], [y_adj, y_adj], color=SEC, lw=1, ls=":", zorder=2)
ax.text(np.sqrt(1.25 * 2.45), y_adj + 0.2, "estimate lies outside\nits own interval",
fontsize=10, color=SEC, ha="center", va="bottom")
# geometric midpoint of the printed interval
ax.plot(gm, y_adj, "o", ms=9, mfc=BG, mec=ORANGE, mew=2, zorder=5)
ax.text(gm, y_adj - 0.22, f"interval midpoint {gm:.2f}", fontsize=10, color=SEC, ha="center", va="top")
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
ax.text(1, ys[0] + 0.45, "OR = 1", fontsize=10, color=SEC, ha="center", va="bottom")
ax.set_xscale("log")
ax.set_xlim(0.5, 10)
ax.set_ylim(-0.55, ys[0] + 0.8)
ticks = [0.5, 1, 2, 5, 10]
ax.set_xticks(ticks)
ax.set_xticklabels([str(t) for t in ticks])
ax.xaxis.set_minor_formatter(matplotlib.ticker.NullFormatter())
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rowsF], color=TXT, fontsize=10)
ax.set_xlabel("Odds ratio, discharge to rehabilitation (log scale)")
ax.grid(axis="x", which="major", color=GRID, lw=0.8, zorder=0)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
ax.text(1.03, ys[0] + 0.45, "OR (95% CI)", transform=ax.get_yaxis_transform(), fontsize=10,
color=SEC, ha="left", va="bottom")
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="Reported in the paper"),
Line2D([], [], color=BLUE, marker="o", ms=8, lw=2, label="Recomputed from the paper's own numbers")]
fig.legend(handles=handles, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
fig.savefig(os.path.join(HERE, "fig1.svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, "fig1.png"), dpi=200, facecolor=BG)
print("\nwrote fig1.svg and fig1.png")
# ---------------------------------------------------------------- figure 2: implied SEs
print("\nSE of log(OR) implied by each Table 3 interval, (ln upper - ln lower) / 3.92:")
se_rows = [
("Mortality, unadjusted", wald_from_ci(9.68, 3.55, 26.4)[1], "rep"),
("Mortality, adjusted", wald_from_ci(10.50, 3.54, 31.18)[1], "rep"),
("Rehabilitation, unadjusted", se_u, "rep"),
("Rehabilitation, adjusted, as printed", se_pr, "rep"),
("Rehabilitation, adjusted, recomputed\nfrom OR 2.45 and lower 0.81", se_A, "rec"),
("Rehabilitation, adjusted, recomputed\nfrom OR 2.45 and P = 0.103", se_B, "rec"),
]
for lbl, se_, _ in se_rows:
print(f" {lbl.replace(chr(10), ' '):55s} {se_:.3f}")
fig, ax = plt.subplots(figsize=(7.2, 3.4), facecolor=BG)
fig.subplots_adjust(left=0.40, right=0.97, top=0.97, bottom=0.27)
ax.set_facecolor(BG)
ys = np.arange(len(se_rows))[::-1].astype(float)
for (lbl, se_, kind), y in zip(se_rows, ys):
col = ORANGE if kind == "rep" else BLUE
ax.plot([0, se_], [y, y], color=col, lw=2, alpha=0.35, zorder=2)
ax.plot(se_, y, "o", ms=9, color=col, zorder=3)
ax.text(se_ + 0.02, y, f"{se_:.3f}", va="center", ha="left", fontsize=10, color=TXT)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in se_rows], fontsize=10, color=TXT)
ax.set_xlim(0, 0.7)
ax.set_ylim(-0.5, ys[0] + 0.5)
ax.set_xlabel("Implied SE of log(OR) = (ln upper − ln lower) / 3.92")
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=0, label="From an interval printed in Table 3"),
Line2D([], [], color=BLUE, marker="o", ms=8, lw=0, label="From a recomputed interval")]
fig.legend(handles=handles, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
fig.savefig(os.path.join(HERE, "fig2.svg"), facecolor=BG)
fig.savefig(os.path.join(HERE, "fig2.png"), dpi=200, facecolor=BG)
print("wrote fig2.svg and fig2.png")
Binomial p for 12 of 17 complexes is 0.072, not the reported 0.0245
eLife (2024) · doi:10.7554/eLife.93333 · PMC10942632
Summary
The paper reports that 12 of 17 protein complexes had an ROC-AUC above 0.5 and calls this "a significant excess (binomial test, p=0.0245)" against a null expectation of one half. The exact one-sided binomial p-value for 12 or more successes out of 17 at p = 0.5 is 9402/131072 = 0.0717, and the two-sided value is 0.143, so neither is significant at 0.05. The reported 0.0245 equals P(X ≥ 13), the upper tail starting one success above the observed count. The second binomial test in the same paragraph shows the same pattern: 0.0012 is reported, while P(X ≥ 4 | n = 17, p = 0.05) is 0.0088. That result remains significant. The correction removes one of the paper's three complex-level arguments for a detectable physical-interaction signal in ERC, and the corrected value is now consistent with the paper's own non-significant global permutation test (p = 0.08). The headline conclusion in the title and abstract does not depend on this test.
What the paper reports
The first binomial test, its stated null, and the second binomial test:
"Of the 17 complexes, 12 had an ROC-AUC > 0.5. Since the random expectation would be that half would exceed 0.5, this is a significant excess (binomial test, p=0.0245). Moreover, five complexes had AUCs > 0.7 (Figure 4). These results indicate that physically interacting domains tend to have a higher ERC than nonphysically interacting domains in those complexes. Likewise, four complexes had their physically interacting domains ranked significantly higher than noninteracting domains at an alpha of 5% (one-tailed Mann–Whitney U test), which is also a significant excess of complexes overall (binomial test, p=0.0012)." (Results, "ERC does not distinguish physical interactions from nonphysical interactions within a given protein complex", para 4, which opens "Of the 17 complexes, 12 had an ROC-AUC > 0.5")
The count is restated in the figure legend:
"Of the 17 complexes, 12 have an ROC-AUC > 0.5. The SEC23/24 complex (bright green) has the highest ROC-AUC at 1, and the NUP84 complex (marigold) has the lowest AUC of 0.247." (Figure 4 legend)
The global test on the same AUCs:
"We compared the true average ROC-AUC from the 17 complexes (0.596) to the permuted distribution of average ROC-AUCs, which resulted in a permutation p-value of 0.08." (Results, same subsection, para 5, which opens "We then applied a third test for a general pattern")
The Discussion relies on the 12-of-17 result:
"Looking across all complexes in our study, a proportion higher than random chance showed elevated ERC between interacting domains; this result was reflected in 12 of the 17 complexes with ROC-AUCs over 0.5 and the 4 complexes with individually significant rankings." (Discussion, para 2, which opens "We found that compensatory coevolution due to physical interaction contributes to ERC")
The abstract states the overall conclusion:
"We found no generalizable relationship between physical interaction and high ERC, although a few complexes ranked physical interactions higher than nonphysical interactions." (Abstract)
What the paper's own numbers show
Method. Under the null stated in the text, the number X of complexes with AUC > 0.5 is Binomial(n = 17, p = 0.5). The exact one-sided p-value for the observed 12 is P(X ≥ 12), which includes the observed outcome. It was computed by summing binomial coefficients exactly and checked with scipy.stats.binomtest. The same was done for the second test, X ~ Binomial(17, 0.05) with 4 observed. As an input check, the 17 per-complex AUCs printed in the version-of-record Figure 4 image were transcribed into recompute.py.txt. (The PMC text legend gives only the highest and lowest values.) They give 12 of 17 above 0.5, no values exactly at 0.5, and a mean of 0.596, matching the text. All numbers below are printed by recompute.py.txt when it is run. Paragraph numbers in this write-up are one-based, counted from the first paragraph of the named subsection, and each is given with its opening words.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Complexes with ROC-AUC > 0.5 | Results para 4 ("Of the 17 complexes, 12 had…"); Fig. 4 legend; Discussion para 2 | 12 of 17 | 12 of 17 (from the Fig. 4 AUCs) |
| Mean ROC-AUC | Results para 5 ("We then applied a third test…") | 0.596 | 0.596 (from the Fig. 4 AUCs) |
| Binomial p, 12 of 17 at p = 0.5, one-sided P(X ≥ 12) | Results para 4 | 0.0245 | 9402/131072 = 0.0717 |
| Same test, exact two-sided | — | — | 0.1435 |
| P(X ≥ 13) = P(X > 12) | — | — | 3214/131072 = 0.02452 (equals the reported value) |
| Smallest count out of 17 with one-sided p < 0.05 | — | — | 13 |
| Binomial p, 4 of 17 significant at α = 0.05, one-sided P(X ≥ 4) | Results para 4 | 0.0012 | 0.0088 (still < 0.05) |
| P(X ≥ 5) = P(X > 4) at p = 0.05 | — | — | 0.00116 (equals the reported value) |
| Global permutation test on mean AUC | Results para 5 | 0.08 | not recomputable (needs per-complex ranked lists); consistent with the corrected 0.072 |
| Complexes with AUC > 0.7 (side note) | Results para 4 | "five" | 6 in the Fig. 4 AUCs (1.00, 0.821, 0.736, 0.723, 0.714, 0.713) |
| Complexes with physical > nonphysical mean ERC (side note) | Results para 2 ("We split the proteins…") vs Table 1 | 12 of 17 | 14 of 17 in Table 1 |
Both reported binomial p-values equal the upper tail that starts one success above the observed count. That tail excludes the observed outcome, so it is not the p-value of the observation. For 12 of 17, the excluded term alone is P(X = 12) = 6188/131072 = 0.0472.
Figure
Figure 1. Reported p-values (orange) and exact binomial p-values recomputed from the paper's counts (blue), on one log-scaled axis; the dashed line marks α = 0.05. Top row: 12 of 17 complexes with ROC-AUC > 0.5 under the paper's stated null of one half. The reported 0.0245 falls below 0.05. The exact one-sided p (0.0717, filled blue) and two-sided p (0.143, open blue diamond) are both above it. Bottom row: 4 of 17 complexes individually significant at α = 0.05. The reported 0.0012 and the recomputed 0.0088 are both below 0.05, so only the size of that p-value changes. All values come from the paper's text or from recompute.py.txt.
Figure 2. The null distribution of the number of complexes with AUC > 0.5 out of 17, Binomial(17, 0.5). The blue bars (12 to 17) form the upper tail that includes the observed count, and they sum to the correct one-sided p = 0.0717 (blue bracket). The reported 0.0245 equals the sum over 13 to 17 only (orange bracket), leaving out the observed outcome, whose probability is 0.0472. This figure shows why a one-step shift in the tail boundary roughly triples the p-value here.
Why it matters
Results paragraphs 4 and 5 of the subsection use three complex-level tests to argue that physical coevolution contributes a detectable signal to ERC: the 12-of-17 sign test, the count of individually significant complexes, and the global permutation test. As reported, the first two are significant and the third is not, and the Discussion says the evidence "was tempered by a global permutation test". After correction, the 12-of-17 test (p = 0.072) is not significant either, so two of the three complex-wide tests fail to reach 0.05. The corrected value is in line with the permutation p of 0.08 rather than in tension with it.
Two statements depend on the 12-of-17 result as written. The Results paragraph calls it "a significant excess", and the Discussion's "a proportion higher than random chance ... this result was reflected in 12 of the 17 complexes with ROC-AUCs over 0.5" cites it as evidence. At α = 0.05 the 12-of-17 result is not a proportion higher than chance. The Discussion's broader claim of a "non-negligible signal from physical interaction" is weakened but keeps some support. The 4-of-17 count remains significant (corrected p = 0.0088), and five complexes are reported as significant in the separate proportional-rank analysis.
The headline conclusion is that ERC does not reliably predict physical interactions ("no generalizable relationship"). The error does not undermine it. If anything the corrected value makes the evidence for a general physical-interaction signal weaker, which fits that conclusion. The error is not in the title or the abstract.
Explanations considered
The count is 13, not 12. Ruled out. The text, the Figure 4 legend and the Discussion all say 12 of 17. The 17 AUCs printed in the version-of-record Figure 4 image have exactly 12 above 0.5, and their mean (0.596) matches the value in the text, so the transcription is consistent.
A tie at AUC = 0.5 was dropped (n = 16). Does not account for it. No Figure 4 AUC equals 0.5 (the closest is SMC5/6 at 0.504). Even with n = 16, P(X ≥ 12 | 16, 0.5) = 0.0384, not 0.0245.
A two-sided test. Does not account for it. The two-sided exact p is 0.143, larger than the one-sided value.
A different null proportion. Ruled out by the text, which states the expectation as "half would exceed 0.5".
A convention that defines the test as strictly greater than the observed count. No such definition appears in the Methods, which describe only the per-complex Mann–Whitney tests and the permutations. P(X > k) is not a valid p-value for an observation of k, because it excludes the observed outcome. The value 0.0245 is what an upper-tail call returns when the observed count itself is passed as the cutoff, for example R's pbinom(12, 17, 0.5, lower.tail = FALSE), which returns P(X > 12). The correct call passes 11. R was not run here; recompute.py.txt evaluates the equivalent upper-tail function, scipy.stats.binom.sf, with both cutoffs (0.02452 for 12, 0.07173 for 11). The second test reproduces the same shift exactly (0.00116 vs 0.0088), which points to one consistent computational step rather than two separate typos.
A conversion or archiving artifact. Ruled out. The eLife version-of-record web page, the version-of-record PDF, the PMC page, the Europe PMC full-text XML and the reviewed preprint v2 all give the same numbers. The only markup difference is in the second test: in the Europe PMC XML its equals sign is wrapped in italic markup, while the eLife web page displays it as "p = 0.0012". The value is the same.
The analysis used different data. This cannot be settled from the paper and does not change the finding as published. The data deposit (see Status) contains a per-complex spreadsheet. An earlier audit stage recomputed AUCs from it and found values that differ slightly from Figure 4 (for example NUP84 0.260 vs 0.247), with 14 of 17 above 0.5. Table 1 also shows physical > nonphysical mean ERC in 14 of 17 complexes, where the text says 12. A count of 14 of 17 would give P(X ≥ 14) = 0.0064. But every statement in the paper, and the published Figure 4, reports 12 of 17, and the p-value is evaluated here for that reported count. If a different count was intended, the text, the figure and the Discussion would all need to change.
Status
As of 2026-10-23:
- Version of record. The eLife version of record (https://elifesciences.org/articles/93333, published 2024-02-28) was fetched again on 2026-10-23. It still reads "binomial test, p=0.0245" and, for the second test, "binomial test, p = 0.0012" (typeset with an italic equals sign), and it shows no correction notice. An earlier audit stage found the same wording in the version-of-record PDF (elife-93333-v1.pdf), the Europe PMC full-text XML and the reviewed preprint v2 PDF ("Binomial test, P = 0.0245"). That stage also read the 17 AUCs from the Figure 4 image in the version-of-record PDF.
- Supplements and data. The version of record has no standalone supplementary or source-data files, only two figure supplements to Figure 2. The data availability statement points to the repository github.com/nclark-lab/erc and cites an archived copy on Software Heritage; Methods ("Calculating ERC") names the physical_interaction_paper subfolder of that repository as the location of the pipeline. An earlier audit stage opened all_complexes.xlsx (17 sheets of domain-pair ftERC scores with physical-interaction labels) and run_domain.R. The R script draws the ROC curves but contains no binomial test code, so the computation of the reported p-values could not be inspected. As noted above, AUCs recomputed from the spreadsheet do not exactly reproduce Figure 4.
- Corrections.
src/known.py(run 2026-10-23) lists no Crossref updates and no PMC or Europe PMC correction or retraction notices. - PubPeer. PubPeer's API returned HTTP 404 and its search page returned a bot-check page on 2026-10-23, so PubPeer could not be queried directly. A web search on 2026-10-23 for the title together with "pubpeer" returned only the article's own pages (PMC, bioRxiv, eLife) and no PubPeer thread.
Sources: PMC10942632 (PubMed Central archive, read with src/show.py); https://elifesciences.org/articles/93333; https://cdn.elifesciences.org/articles/93333/elife-93333-v1.pdf; reviewed preprint and reviews at https://elifesciences.org/reviewed-preprints/93333/reviews; preprint doi:10.1101/2023.10.18.562970; data deposit https://github.com/nclark-lab/erc (subfolder physical_interaction_paper, named in Methods, "Calculating ERC").
Suggested correction
In Results, "ERC does not distinguish physical interactions from nonphysical interactions within a given protein complex", para 4 (opening "Of the 17 complexes, 12 had an ROC-AUC > 0.5"), replace "this is a significant excess (binomial test, p=0.0245)" with "this excess is not statistically significant (one-sided exact binomial test, p=0.072)". Replace "(binomial test, p=0.0012)" with "(one-sided exact binomial test, p=0.0088)". In the Discussion, para 2 (opening "We found that compensatory coevolution due to physical interaction contributes to ERC"), revise "a proportion higher than random chance showed elevated ERC between interacting domains; this result was reflected in 12 of the 17 complexes with ROC-AUCs over 0.5 and the 4 complexes with individually significant rankings" so that only the 4 individually significant complexes are cited as an excess over chance, and note that the 12-of-17 count is not significant. The authors could also:
- state the test sidedness and the exact tail used in the Methods;
- reconcile "five complexes had AUCs > 0.7" (Results para 4) with the six values above 0.7 in Figure 4;
- reconcile "12 had higher average ERC for the physically interacting" (Results para 2, opening "We split the proteins") with Table 1, where 14 of 17 complexes show this;
- confirm which data underlie Figure 4, since the deposited spreadsheet gives slightly different AUCs.
Recomputation script (python recompute.py.txt)
"""Recompute every number in writeup.md for PMC10942632 and regenerate fig1/fig2.
Run: python recompute.py.txt (writes fig1.svg/png and fig2.svg/png next to this file)
Inputs come from the article (eLife 2024, doi:10.7554/eLife.93333):
* Results, "ERC does not distinguish physical interactions ...", para 4
(opens "Of the 17 complexes, 12 had an ROC-AUC > 0.5"):
12 of 17 complexes with ROC-AUC > 0.5, "binomial test, p=0.0245";
4 of 17 complexes significant at alpha = 5%, "binomial test, p=0.0012".
* Figure 4 (version-of-record image): the 17 per-complex ROC-AUCs listed below.
* Same subsection, para 5 (opens "We then applied a third test"):
mean ROC-AUC 0.596, global permutation p = 0.08.
* Same subsection, para 2 (opens "We split the proteins"): "12 had higher
average ERC" for physically interacting domains.
* Table 1: mean ERC of physically vs nonphysically interacting domains.
Paragraph numbers are one-based, counted from the first paragraph of the subsection.
"""
import os
from math import comb
from fractions import Fraction
from scipy.stats import binom, binomtest
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
HERE = os.path.dirname(os.path.abspath(__file__))
N = 17
# ------------------------------------------------------------ reported values
REP_P1 = 0.0245 # 12 of 17 AUC > 0.5, null proportion 0.5
REP_P2 = 0.0012 # 4 of 17 significant, null proportion 0.05
REP_PERM = 0.08 # global permutation test on mean AUC (not recomputable here)
# Figure 4 (VOR image legend), per-complex ROC-AUC
AUC = {"SEC23/24": 1.00, "ESCRT-I": 0.821, "Exportin": 0.736, "CUL8": 0.723,
"GET4_GET5": 0.714, "SWI_SNF": 0.713, "MCM": 0.666, "TREX": 0.600,
"ORC": 0.565, "PAN1": 0.543, "COMA": 0.540, "SMC5/6": 0.504,
"EXOCYST": 0.480, "ATG17/31/29": 0.478, "F1 ATPase": 0.467,
"EIF3": 0.341, "NUP84": 0.247}
# Table 1: (physically interacting domain ERC, nonphysically interacting domain ERC)
TABLE1 = {"EIF3": (8.852, 11.125), "MCM": (10.283, 8.000), "NUP84": (1.541, 4.411),
"ORC": (5.250, 4.035), "PAN1": (4.963, 4.716), "SMC5-6": (5.153, 4.110),
"TREX": (4.157, 4.129), "EXOCYST": (6.221, 5.974), "COMA": (3.051, 2.869),
"SWI/SNF": (4.121, 1.783), "CUL8": (2.611, 1.576), "GET4-GET5": (4.569, 3.302),
"ATG17-31-29": (3.444, 1.584), "ESCRT-I": (4.041, 2.402),
"F1 ATPase": (4.883, 5.654), "SEC23/24": (10.530, 7.754),
"Exportin": (8.871, 6.326)}
# ------------------------------------------------------------ Figure 4 checks
k_auc = sum(v > 0.5 for v in AUC.values())
print("== Inputs from Figure 4 ==")
print(f"complexes listed: {len(AUC)}")
print(f"AUC > 0.5: {k_auc} of {len(AUC)} (text: 12 of 17)")
print(f"AUC = 0.5 exactly (ties): {sum(v == 0.5 for v in AUC.values())}")
print(f"mean AUC: {sum(AUC.values())/len(AUC):.3f} (text: 0.596)")
print(f"AUC > 0.7: {sum(v > 0.7 for v in AUC.values())} (text: 'five complexes had AUCs > 0.7')")
print()
# ------------------------------------------------------------ test 1: 12 of 17, p0 = 0.5
k1 = 12
terms = [comb(N, k) for k in range(k1, N + 1)]
p_ge12 = Fraction(sum(terms), 2 ** N)
p_ge13 = Fraction(sum(comb(N, k) for k in range(k1 + 1, N + 1)), 2 ** N)
two_sided = binomtest(k1, N, 0.5).pvalue
print("== Test 1: 12 of 17 complexes with AUC > 0.5, null proportion 0.5 ==")
print(f"C(17,k), k=12..17: {terms}; sum = {sum(terms)}; 2^17 = {2**N}")
print(f"one-sided P(X>=12) = {sum(terms)}/{2**N} = {float(p_ge12):.4f}"
f" (scipy binomtest greater: {binomtest(k1, N, 0.5, alternative='greater').pvalue:.4f})")
print(f"two-sided exact p = {two_sided:.4f}")
print(f"P(X>=13) = P(X>12) = {sum(terms[1:])}/{2**N} = {float(p_ge13):.5f}"
f" (reported: {REP_P1})")
print(f"P(X=12), the observed outcome left out of P(X>12) = {comb(N, 12)}/{2**N} = "
f"{binom.pmf(12, N, 0.5):.4f}")
# R's pbinom(q, n, p, lower.tail = FALSE) returns P(X > q), i.e. scipy's binom.sf(q, n, p)
print(f"upper tail with the observed count as cutoff, P(X>12) "
f"[R: pbinom(12, 17, 0.5, lower.tail = FALSE)] = {binom.sf(12, N, 0.5):.5f}")
print(f"upper tail with cutoff 11, P(X>11) = P(X>=12) "
f"[R: pbinom(11, 17, 0.5, lower.tail = FALSE)] = {binom.sf(11, N, 0.5):.5f}")
print(f"tie excluded, P(X>=12 | n=16) = {binom.sf(11, 16, 0.5):.4f}")
print(f"smallest k out of 17 with P(X>=k) < 0.05: "
f"{min(k for k in range(N + 1) if binom.sf(k - 1, N, 0.5) < 0.05)}")
print()
# ------------------------------------------------------------ test 2: 4 of 17, p0 = 0.05
k2 = 4
p2_ge4 = binom.sf(k2 - 1, N, 0.05)
p2_ge5 = binom.sf(k2, N, 0.05)
print("== Test 2: 4 of 17 complexes significant at alpha 0.05, null proportion 0.05 ==")
print(f"one-sided P(X>=4) = {p2_ge4:.4f}")
print(f"P(X>=5) = P(X>4) = {p2_ge5:.5f} (reported: {REP_P2})")
print()
# ------------------------------------------------------------ side check: Table 1
k_t1 = sum(a > b for a, b in TABLE1.values())
print("== Side check: Table 1 ==")
print(f"physical ERC > nonphysical ERC: {k_t1} of {len(TABLE1)} (text: 12 of 17)")
print(f"for reference, a count of 14 of 17 would give P(X>=14 | 17, 0.5) = "
f"{sum(comb(N, k) for k in range(14, N + 1))}/{2**N} = {binom.sf(13, N, 0.5):.4f}")
print()
print(f"Global permutation test (reported, not recomputable): p = {REP_PERM}")
# ------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK2, "axes.edgecolor": INK2,
"svg.fonttype": "none", "font.family": "DejaVu Sans"})
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, **kw)
plt.close(fig)
# Figure 1: reported vs recomputed p-values, one log axis, alpha = 0.05 reference
fig, ax = plt.subplots(figsize=(7.2, 3.5), facecolor=BG)
ax.set_facecolor(BG)
rows = [("12 of 17 complexes with ROC-AUC > 0.5\n(null proportion 0.5)",
REP_P1, float(p_ge12), two_sided),
("4 of 17 complexes significant at α = 0.05\n(null proportion 0.05)",
REP_P2, p2_ge4, None)]
ys = [1, 0]
for y, (lab, rep, rec, rec2) in zip(ys, rows):
hi = max(rep, rec, rec2 or 0)
ax.plot([min(rep, rec), hi], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(rep, y, "o", ms=10, color=ORANGE, zorder=3)
ax.plot(rec, y, "o", ms=10, color=BLUE, zorder=3)
ax.annotate(f"reported {rep}", (rep, y), xytext=(0, 11), textcoords="offset points",
ha="center", color=INK, fontsize=10)
ax.annotate(f"P(X≥{12 if y else 4}) = {rec:.4f}", (rec, y), xytext=(0, -18),
textcoords="offset points", ha="center", color=INK, fontsize=10)
if rec2 is not None:
ax.plot(rec2, y, "D", ms=8, mfc=BG, mec=BLUE, mew=2, zorder=3)
ax.annotate(f"two-sided\n{rec2:.3f}", (rec2, y), xytext=(10, 0),
textcoords="offset points", ha="left", va="center", color=INK,
fontsize=10)
ax.plot([0.05, 0.05], [-0.6, 1.5], color=INK2, lw=1, ls="--", zorder=0)
ax.text(0.05, 1.55, "α = 0.05", ha="center", va="bottom", color=INK2, fontsize=10)
ax.set_xscale("log")
ax.set_xlim(5e-4, 0.8)
ax.set_ylim(-0.6, 1.75)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=INK)
ax.set_xticks([0.001, 0.003, 0.01, 0.03, 0.1, 0.3])
ax.set_xticklabels(["0.001", "0.003", "0.01", "0.03", "0.1", "0.3"])
ax.minorticks_off()
ax.set_xlabel("binomial p-value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], marker="o", ls="", ms=9, color=ORANGE, label="reported in paper"),
Line2D([], [], marker="o", ls="", ms=9, color=BLUE,
label="recomputed, one-sided"),
Line2D([], [], marker="D", ls="", ms=8, mfc=BG, mec=BLUE, mew=2,
label="recomputed, two-sided")]
ax.legend(handles=handles, loc="upper center", bbox_to_anchor=(0.22, -0.25), ncol=3,
frameon=False, fontsize=10, handletextpad=0.3, columnspacing=1.2)
fig.tight_layout()
save(fig, "fig1")
# Figure 2: Binomial(17, 0.5) distribution; which tail each p-value sums
fig, ax = plt.subplots(figsize=(7.2, 3.6), facecolor=BG)
ax.set_facecolor(BG)
ks = list(range(N + 1))
pm = [binom.pmf(k, N, 0.5) for k in ks]
cols = ["#c9c8c3" if k < 12 else BLUE for k in ks]
ax.bar(ks, pm, width=0.8, color=cols, edgecolor=BG, lw=2, zorder=2)
ax.bar([12], [pm[12]], width=0.8, color=BLUE, edgecolor=BG, lw=2, zorder=2)
ytop = max(pm)
yb1, yb2 = pm[12] + 0.035, pm[12] + 0.07
ax.plot([12.6, 17.4], [yb1, yb1], color=ORANGE, lw=2)
for x in (12.6, 17.4):
ax.plot([x, x], [yb1 - 0.006, yb1], color=ORANGE, lw=2)
ax.text(15, yb1 + 0.004, f"reported p = P(X≥13) = {float(p_ge13):.4f}", ha="center",
va="bottom", fontsize=10, color=INK)
ax.plot([11.6, 17.4], [yb2, yb2], color=BLUE, lw=2)
for x in (11.6, 17.4):
ax.plot([x, x], [yb2 - 0.006, yb2], color=BLUE, lw=2)
ax.text(14.5, yb2 + 0.004, f"correct p = P(X≥12) = {float(p_ge12):.4f}", ha="center",
va="bottom", fontsize=10, color=INK)
ax.annotate(f"observed: 12 of 17\nP(X=12) = {binom.pmf(12, N, 0.5):.4f}",
(12, pm[12]), xytext=(9.0, pm[12] + 0.02), ha="right", fontsize=10,
color=INK, arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.set_xticks(ks)
ax.set_xlabel("number of complexes with ROC-AUC > 0.5 (out of 17)")
ax.set_ylabel("probability under null (p = 0.5)")
ax.set_ylim(0, ytop * 1.08)
ax.set_xlim(-0.7, 17.7)
ax.grid(axis="y", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, "fig2")
print("\nfigures written:", ", ".join(f"fig{i}.{e}" for i in (1, 2) for e in ("svg", "png")))
Abstract attaches the most- and least-sitting case counts to the wrong exposure groups
BMJ (2011) · doi:10.1136/bmj.d3867 · PMC3266839
Summary
The abstract illustrates the primary association with raw counts: "41/104 720 v 16/14 565 cases in most inactive v least inactive". In Table 2, 41/104 720 belongs to score 2, the group that sat least and the reference group, and 16/14 565 belongs to score 6, the group that sat most. The abstract therefore attaches each count to the opposite extreme. Read as printed, the counts give a crude rate ratio of 0.36 for most versus least inactive, which suggests a protective pattern. With Table 2's labels the ratio is 2.81, which agrees with the hazard ratios of 2.73, 2.68 and 2.34 that the paper reports. The hazard ratios, the trend test and the conclusion are not affected; the error is in the labelling of one parenthetical in the abstract's primary result.
What the paper reports
Abstract, Results:
There was an association between time of sitting and risk of idiopathic pulmonary embolism (41/104 720 v 16/14 565 cases in most inactive v least inactive in combined data; P<0.001 for trend). The risk of pulmonary embolism was more than twofold in women who spent the most time sitting compared with those who spent the least time sitting (multivariable hazard ratio 2.34, 95% confidence interval 1.30 to 4.20).
Methods, Exposures:
Thus, each woman was assigned to one of five categories of time sitting, with point values ranging from 2 (low time sitting in 1988 and 1990) to 6 (high time sitting in 1988 and 1990).
Methods, Statistical analysis:
For all analyses of physical inactivity, we used the group with the lowest amount of time sitting as the reference group.
Table 2, header and first row ("Amount of time sitting* (score 2=lowest, 6=highest), combined data from 1988 and 1990"; columns 2, 3, 4, 5, 6):
No of cases/person years | 41/104 720 | 53/175 518 | 104/209 002 | 54/88 228 | 16/14 565 | —
Table 2, hazard ratio rows (score 2 is the reference, 1.00):
Age adjusted HR (95% CI) | 1.00 | 0.79 (0.52 to 1.19) | 1.31 (0.90 to 1.88) | 1.62 (1.08 to 2.44) | 2.73 (1.53 to 4.88) | 1.29 (1.15 to 1.45), <0.001
Multivariable HR (95% CI)‡ | 1.00 | 0.79 (0.53 to 1.19) | 1.29 (0.89 to 1.85) | 1.51 (1.01 to 2.28) | 2.34 (1.30 to 4.20) | 1.25 (1.12 to 1.40), <0.001
Results, second paragraph:
For women with the most physical inactivity, the risk of pulmonary embolism was more than twice that of women with the least physical inactivity (multivariable hazard ratio 2.68, 1.50 to 4.79).
What the paper's own numbers show
Method. I took the cases and person-years for each sitting score from Table 2 and the number of participants per score from Table 1. I checked that the totals match the text, used the Table 1 sitting-hours distribution to confirm which way the score runs, and computed crude incidence rates per 100 000 person-years. I then computed the crude rate ratio for most versus least sitting twice: once with Table 2's group labels and once with the labels the abstract gives. A crude rate ratio is not adjusted, so it need not equal the Cox hazard ratios, but it should point the same way and be of similar size. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Total cases | Abstract; Results; Table 2 | 268 | 41 + 53 + 104 + 54 + 16 = 268 |
| Total participants | Abstract; Table 1 | 69 950 | 12 451 + 20 718 + 24 549 + 10 403 + 1829 = 69 950 |
| Direction of the score | Methods; Table 1 (1988 sitting) | 2 = low, 6 = high sitting | score 2: 100% sat <10 h/week; score 6: 100% sat ≥41 h/week |
| Score 2 (least sitting, reference) | Table 2, column 2 | 41/104 720 | 39.2 per 100 000 person-years |
| Score 6 (most sitting) | Table 2, column 6 | 16/14 565 | 109.9 per 100 000 person-years |
| Group labelled "most inactive" | Abstract | 41/104 720 | equals Table 2 score 2, the least-sitting group |
| Group labelled "least inactive" | Abstract | 16/14 565 | equals Table 2 score 6, the most-sitting group |
| Crude ratio, most v least, abstract's labels | Abstract | (implied) | 0.36 (39.2 ÷ 109.9) |
| Crude ratio, most v least, Table 2 labels | Table 2 | — | 2.81 (109.9 ÷ 39.2; approximate 95% CI 1.57 to 5.00) |
| HR, score 6 v score 2 | Table 2; Results; Abstract | 2.73 (age adjusted); 2.68 and 2.34 (multivariable) | consistent with 2.81, not with 0.36 |
The crude rates by score (39.2, 30.2, 49.8, 61.2, 109.9 per 100 000 person-years) follow the same pattern as the age-adjusted hazard ratios (1.00, 0.79, 1.31, 1.62, 2.73). Follow-up per woman is similar across scores (7.96 to 8.51 person-years), so person-time does not distort the comparison. The two counts in the abstract are correct numbers; only the group names attached to them are reversed.
Figure
Figure 1. Left panel: the parenthetical in the abstract's Results, with each count (orange, as printed) and the group name the abstract gives it. Right panel: the same two counts in Table 2, with the column they occupy, the group the paper's Methods and Table 2 define for that column, and the reported hazard ratio. The crude rates (blue) are computed from the printed counts. The connectors join identical counts: 41/104 720 is "most inactive" in the abstract but the least-sitting reference group in Table 2, and 16/14 565 is "least inactive" in the abstract but the most-sitting group in Table 2. Taken as printed, the abstract's comparison gives a crude most-versus-least ratio of 0.36; with Table 2's labels it is 2.81.
Figure 2. Most versus least sitting on one log axis, with a thin reference line at 1 (no difference). Orange squares with bars are the three hazard ratios (95% CI) for score 6 versus score 2 reported in Table 2, the Results and the abstract. Blue circles are crude rate ratios computed from the Table 2 counts: the open circle uses the abstract's labels (0.36), the filled circle uses Table 2's labels (2.81). Only the filled circle lies within the reported confidence intervals; the open circle lies on the other side of the null. The log scale is used because the quantities are ratios; it shows 0.36 and 2.81 as equally far from 1, which they are (one is the reciprocal of the other).
Why it matters
- The abstract's own illustration points the wrong way. The parenthetical is the abstract's only raw-data support for the primary finding. As printed, it says the most inactive women had 41 cases in 104 720 person-years and the least inactive 16 in 14 565, that is, a lower rate in the most inactive group. The next sentence then states a hazard ratio of 2.34 for most versus least sitting. A reader who checks the counts finds them contradicting the estimate they are meant to support.
- Abstract-only readers are the most exposed. The same sentence appears in the PubMed abstract. Clinicians, guideline writers and journalists who read only the abstract cannot resolve the conflict without Table 2.
- The conclusion is not affected. The hazard ratios (2.73, 2.68, 2.34), the trend tests (P<0.001), the Results text, the key points box ("more than twofold in women who spent the most time sitting") and the Conclusions all state the correct direction, and the corrected crude ratio of 2.81 agrees with them. No estimate changes in direction, size or significance.
Explanations considered
- The score runs the other way (2 = most sitting). Ruled out. The Methods define 2 as "low time sitting in 1988 and 1990" and 6 as "high time sitting in 1988 and 1990"; the Table 2 header says "score 2=lowest, 6=highest" and the Table 1 header "2=lowest, 6=highest"; and Table 1 shows that 100% of score 2 sat less than 10 hours a week in 1988 while 100% of score 6 sat 41 hours or more. The Methods also name the lowest-sitting group as the reference, and score 2 carries HR 1.00.
- "Inactive" refers to the secondary exposure (physical activity in MET). Ruled out. The sentence is about "time of sitting", and the counts in Table 3 for the physical activity analysis are different numbers. Both counts quoted in the abstract occur only in Table 2's first row.
- A different model, subset or population. Ruled out. The counts are raw cases and person-years from the combined 1988/1990 data (n = 69 950), and the five columns add up to the 268 cases stated in the abstract and Results.
- The phrase "X v Y in A v B" could be read in another order. No reading resolves it. Read in parallel, the first count goes with "most inactive", and that count is the reference (least sitting) group. No other pairing of the two counts with the two labels is supported by the sentence's structure.
- The crude ratio cannot be compared with an adjusted hazard ratio. It need not match exactly, but it should agree in direction. With Table 2's labels the crude ratio is 2.81, close to the age-adjusted HR of 2.73; with the abstract's labels it is 0.36, on the opposite side of the null from every hazard ratio the paper reports.
- A text-conversion artifact in the PMC copy. Ruled out. The publisher's typeset PDF has the same abstract sentence word for word. In that PDF, Table 2 puts 41/104 720 and HR 1.00 under column "2" and 16/14 565 and HRs 2.73 and 2.34 under column "6", judged by the positions of the words on the page. Plain text extraction prints the columns of that table in reverse order, but that is only reading order; the column positions are unambiguous. The Europe PMC full-text XML, the PMC page and the PubMed abstract agree.
- A correction already issued. None found (see Status).
Status
- Version of record. The publisher's pages on bmj.com (through doi.org, the article page, its PDF and its rapid responses) returned HTTP 403 to automated requests, and the Europe PMC and PMC PDF renders were also unavailable (403 and a CAPTCHA). I therefore checked the publisher's typeset BMJ PDF as deposited in Harvard's DASH repository, whose page footers read "BMJ 2011;342:d3867 doi: 10.1136/bmj.d3867 (Published 4 July 2011)". It has the abstract sentence word for word, and Table 2 (PDF page 10) has the same column assignment as the PMC text. The Europe PMC full-text XML, the PMC HTML page and the PubMed abstract (PMID 21727169) agree. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. Europe PMC lists no supplementary files for PMC3266839, its full-text XML has no supplementary-material element, and neither the PMC page nor the typeset PDF mentions web extras. A data supplement on bmj.com could not be excluded because the site refused access, but none is needed: the main text fixes the direction of the score, and Table 2 matches it.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations and no PMC or Europe PMC notices; the Crossref record has empty update-to and updated-by fields. The PubMed CommentsCorrections list holds only two "CommentIn" links, the accompanying BMJ editorial (d3865, PMID 21727168) and a 2012 comment in Clin J Sport Med (PMID 22388344); neither is an erratum. PubPeer returned no record (HTTP 404) to an automated DOI query and its search page refused access (403); a web search for the title together with "PubPeer" found no thread for this article. The BMJ rapid responses could not be read (403). - Sources. PMC article PMC3266839; Europe PMC full-text XML and supplementary-files endpoint for PMC3266839; typeset BMJ PDF on Harvard DASH; PubMed (PMID 21727169); Crossref; PubPeer; https://doi.org/10.1136/bmj.d3867.
Suggested correction
In the abstract's Results, replace
(41/104 720 v 16/14 565 cases in most inactive v least inactive in combined data; P<0.001 for trend)
with
(16/14 565 v 41/104 720 cases in most inactive v least inactive in combined data; P<0.001 for trend)
so that the counts match Table 2: score 6 (most sitting) 16 cases in 14 565 person-years, about 110 per 100 000 person-years, and score 2 (least sitting) 41 cases in 104 720 person-years, about 39 per 100 000 person-years. Keeping the order of the counts and swapping the labels ("in least inactive v most inactive") would be an equivalent fix. No other part of the paper needs to change.
Recomputation script (python recompute.py.txt)
"""Recompute the abstract's raw-count comparison in PMC3266839
(BMJ 2011;342:d3867, doi:10.1136/bmj.d3867,
"Physical inactivity and idiopathic pulmonary embolism in women: prospective study").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract, Results: "(41/104 720 v 16/14 565 cases in most inactive v least inactive in
combined data; P<0.001 for trend)" and "multivariable hazard ratio 2.34, 95% confidence
interval 1.30 to 4.20"; "268 cases".
Methods, Exposures: score 2 = "low time sitting in 1988 and 1990", 6 = "high time sitting
in 1988 and 1990". Methods, Statistical analysis: lowest sitting group is the reference.
Table 1: number of participants per score; sitting-hours distribution per score.
Table 2 (score 2=lowest, 6=highest): cases/person-years and hazard ratios per score.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
import matplotlib.ticker # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs
SCORES = [2, 3, 4, 5, 6]
CASES = {2: 41, 3: 53, 4: 104, 5: 54, 6: 16} # Table 2 row 1
PY = {2: 104720, 3: 175518, 4: 209002, 5: 88228, 6: 14565} # Table 2 row 1
N_PART = {2: 12451, 3: 20718, 4: 24549, 5: 10403, 6: 1829} # Table 1 row 1
# Table 1: % sitting <10 h/week and >=41 h/week in 1988, by score
SIT88_LT10 = {2: 100, 3: 36, 4: 2, 5: 0, 6: 0}
SIT88_GE41 = {2: 0, 3: 0, 4: 17, 5: 88, 6: 100}
# Table 2 hazard ratios for score 6 vs score 2 (reference)
HR = {
"Age adjusted": (2.73, 1.53, 4.88),
"Multivariable (Results text)": (2.68, 1.50, 4.79),
"Multivariable, + BMI etc. (Abstract)": (2.34, 1.30, 4.20),
}
TOTAL_CASES_TEXT = 268
TOTAL_N_TEXT = 69950
# Abstract labelling: first pair = "most inactive", second pair = "least inactive"
ABS_MOST = (41, 104720)
ABS_LEAST = (16, 14565)
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, TXT, TXT2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({"font.size": 10, "font.family": "DejaVu Sans",
"svg.fonttype": "none", "axes.edgecolor": TXT2,
"text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": TXT2, "ytick.color": TXT2})
# ---------------------------------------------------------------- 1. totals
print("=== 1. Totals ===")
tc = sum(CASES.values())
tn = sum(N_PART.values())
print(f"sum of Table 2 cases = {tc} (text: {TOTAL_CASES_TEXT})")
print(f"sum of Table 1 participants = {tn} (text/abstract: {TOTAL_N_TEXT})")
print(f"sum of Table 2 person-years = {sum(PY.values())}")
# ---------------------------------------------------------------- 2. direction of the score
print("\n=== 2. Direction of the score (Table 1, 1988 sitting hours) ===")
for s in SCORES:
print(f"score {s}: {SIT88_LT10[s]:3d}% sat <10 h/week, {SIT88_GE41[s]:3d}% sat >=41 h/week")
# ---------------------------------------------------------------- 3. crude rates
print("\n=== 3. Crude rates from Table 2 ===")
RATE = {}
for s in SCORES:
RATE[s] = CASES[s] / PY[s] * 1e5
print(f"score {s}: {CASES[s]}/{PY[s]} = {RATE[s]:.2f} per 100 000 PY; "
f"PY per woman = {PY[s] / N_PART[s]:.2f}")
rr_table = RATE[6] / RATE[2]
# exact Poisson-based (log-normal) 95% CI for the crude rate ratio, for context only
se_log = np.sqrt(1 / CASES[6] + 1 / CASES[2])
lo, hi = np.exp(np.log(rr_table) - 1.96 * se_log), np.exp(np.log(rr_table) + 1.96 * se_log)
print(f"crude rate ratio, score 6 (most sitting) / score 2 (least sitting) = {rr_table:.3f} "
f"(approx. 95% CI {lo:.2f} to {hi:.2f})")
for k, (h, l, u) in HR.items():
print(f" reported HR score 6 vs 2, {k}: {h} ({l} to {u})")
# ---------------------------------------------------------------- 4. abstract labelling
print("\n=== 4. Ratio implied by the abstract's labels ===")
r_most = ABS_MOST[0] / ABS_MOST[1] * 1e5
r_least = ABS_LEAST[0] / ABS_LEAST[1] * 1e5
rr_abs = r_most / r_least
print(f"'most inactive' as printed: {ABS_MOST[0]}/{ABS_MOST[1]} = {r_most:.2f} per 100 000 PY "
f"(matches Table 2 score {[s for s in SCORES if (CASES[s], PY[s]) == ABS_MOST][0]})")
print(f"'least inactive' as printed: {ABS_LEAST[0]}/{ABS_LEAST[1]} = {r_least:.2f} per 100 000 PY "
f"(matches Table 2 score {[s for s in SCORES if (CASES[s], PY[s]) == ABS_LEAST][0]})")
print(f"most/least ratio under the abstract's labels = {rr_abs:.3f}")
print(f"most/least ratio under Table 2's labels = {rr_table:.3f} (= 1/{rr_abs:.3f})")
# ---------------------------------------------------------------- figure 1: text-panel diagram
fig = plt.figure(figsize=(7.2, 4.5), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
ax.set_facecolor(BG)
def panel(x0, x1, title, sub):
ax.add_patch(FancyBboxPatch((x0, 0.12), x1 - x0, 0.74,
boxstyle="round,pad=0.004,rounding_size=0.012",
fc="white", ec=TXT2, lw=1.0))
ax.text(x0 + 0.02, 0.815, title, fontsize=11.5, fontweight="bold", va="center")
ax.text(x0 + 0.02, 0.755, sub, fontsize=10, color=TXT2, va="center")
LX0, LX1, RX0, RX1 = 0.03, 0.40, 0.58, 0.97
panel(LX0, LX1, "Abstract, Results", "as printed")
panel(RX0, RX1, "Table 2", "score 2 = lowest, 6 = highest sitting")
YA, YB = 0.56, 0.28
rows = [
(YA, "41/104 720", "labelled “most inactive”",
"score 2: least sitting", "(reference, HR 1.00)", RATE[2]),
(YB, "16/14 565", "labelled “least inactive”",
"score 6: most sitting", "(MV HR 2.34, 1.30 to 4.20)", RATE[6]),
]
for y, cnt, alab, tlab1, tlab2, rate in rows:
ax.text(LX0 + 0.02, y + 0.05, cnt + " cases/PY", fontsize=11, color=ORANGE,
fontweight="bold", va="center")
ax.text(LX0 + 0.02, y - 0.02, alab, fontsize=10.5, va="center")
ax.text(RX0 + 0.02, y + 0.05, cnt + " cases/PY", fontsize=11, color=ORANGE,
fontweight="bold", va="center")
ax.text(RX0 + 0.02, y - 0.02, tlab1, fontsize=10.5, va="center")
ax.text(RX0 + 0.02, y - 0.075, tlab2, fontsize=10, color=TXT2, va="center")
ax.text(RX0 + 0.02, y - 0.13 if y == YB else y - 0.13,
f"crude rate {rate:.1f} per 100 000 PY", fontsize=10, color=BLUE, va="center")
arr = FancyArrowPatch((LX1 + 0.005, y + 0.015), (RX0 - 0.005, y + 0.015),
arrowstyle="<->", mutation_scale=12, lw=2, color=TXT2)
ax.add_patch(arr)
ax.text((LX1 + RX0) / 2, y + 0.06, "same count", fontsize=10, ha="center", color=TXT2)
ax.text((LX1 + RX0) / 2, y - 0.035, "opposite\ngroup", fontsize=10, ha="center",
va="top", fontweight="bold")
ax.text(LX0 + 0.02, 0.165,
f"As printed: most v least\ninactive = {rr_abs:.2f} (crude)", fontsize=10,
color=BLUE, va="center")
ax.text(0.5, 0.94, "The abstract attaches the two extreme-group counts to the wrong groups",
fontsize=12, fontweight="bold", ha="center", va="center")
ax.text(0.03, 0.06,
"Orange: counts as printed in the paper. Blue: computed from those counts.\n"
f"With Table 2's labels the crude most v least ratio is {rr_table:.2f}. PY = person-years.",
fontsize=10, color=TXT2, va="center")
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ---------------------------------------------------------------- figure 2: ratio on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.8), facecolor=BG)
ax.set_facecolor(BG)
items = [
("Crude rate ratio, abstract's labels\n(most v least inactive, as printed)", rr_abs, None, None, BLUE, "o", "none"),
("Crude rate ratio, Table 2 labels\n(score 6 v score 2)", rr_table, None, None, BLUE, "o", BLUE),
("Age-adjusted HR (Table 2)", *HR["Age adjusted"], ORANGE, "s", ORANGE),
("Multivariable HR (Table 2, Results)", *HR["Multivariable (Results text)"], ORANGE, "s", ORANGE),
("Multivariable HR + BMI etc.\n(Table 2, Abstract)", *HR["Multivariable, + BMI etc. (Abstract)"], ORANGE, "s", ORANGE),
]
ys = np.arange(len(items))[::-1]
for y, (lab, v, l, u, c, m, fc) in zip(ys, items):
if l is not None:
ax.plot([l, u], [y, y], color=c, lw=2, solid_capstyle="butt")
ax.plot(v, y, marker=m, ms=9, mec=c, mfc=fc, mew=2, ls="none")
ax.text(v * (1.09 if l is None else 1.0), y + (0 if l is None else 0.36),
f"{v:.2f}", color=c, fontsize=10, va="center",
ha="left" if l is None else "center")
ax.axvline(1, color=TXT2, lw=0.8, zorder=0)
ax.text(1.04, len(items) - 0.35, "no difference", fontsize=10, color=TXT2, ha="left")
ax.set_xscale("log")
ax.set_xlim(0.25, 6)
ax.set_xticks([0.25, 0.5, 1, 2, 4])
ax.set_xticklabels(["0.25", "0.5", "1", "2", "4"])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys)
ax.set_yticklabels([it[0] for it in items], fontsize=10)
ax.set_ylim(-0.6, len(items) - 0.05)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for sp in ("top", "right", "left"):
ax.spines[sp].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Most v least sitting (log scale)", fontsize=10)
fig.text(0.02, 0.135,
"Orange squares: hazard ratios with 95% CI as reported (score 6 v score 2).\n"
"Blue circles: crude rate ratios computed from the Table 2 counts\n"
"(open = with the abstract's labels; filled = with Table 2's labels).",
fontsize=10, color=TXT2, va="top")
fig.subplots_adjust(left=0.36, right=0.97, top=0.97, bottom=0.30)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print("\nwrote fig1.svg/png and fig2.svg/png in", HERE)
Abstract's 95% CI for the borderline-use socioeconomic odds ratio (0.45–0.60) differs from Table 5's 0.32–0.81
BMJ (2012) · doi:10.1136/bmj.e1805 · PMC3328544
Summary
The abstract's worked example says that, at medical schools using the UKCAT only in borderline cases, applicants from socioeconomic class 4 or 5 had lower odds of a conditional offer: odds ratio 0.51, 95% confidence interval 0.45 to 0.60. Table 5, which the abstract draws on, gives the same odds ratio with a much wider interval, 0.51 (0.32 to 0.81), P = 0.004. The table's odds ratio, interval and P value agree with each other. The abstract's interval is 3.2 times narrower on the log scale, and its bounds are exactly those Table 5 prints for a different predictor in a different model (state-school attendance at factor-use schools, 0.52, 0.45 to 0.60). The direction and significance of the finding do not change, but the abstract overstates the precision of its headline socioeconomic estimate about 3.2-fold.
What the paper reports
Abstract, Results:
For example, a candidate from a non-professional background was much less likely to receive a conditional offer of a place compared with an applicant from a higher social class when applying to an institution using the test only in borderline cases (odds ratio 0.51, 95% confidence interval 0.45 to 0.60). No such effect was observed for such candidates applying to medical schools using the threshold approach (1.27, 0.84 to 1.91). These differences were generally reflected in the interactions observed when the analysis was repeated, pooling the data.
Table 5 (multilevel multiple logistic regression, outcome a conditional offer), block "UKCAT scores for borderline cases (n=7558)†" (columns: odds ratio (95% CI), P value, ICC):
Socioeconomic class 4 or 5‡ | 0.51 (0.32 to 0.81) | 0.004 | 0.23
Table 5, block "UKCAT scores as factor for admissions (n=8163)†":
Attendance at state (non-grammar) school | 0.52 (0.45 to 0.60) | <0.001 | 0.12
Socioeconomic class 4 or 5‡ | 0.62 (0.42 to 0.91) | 0.01 | 0.12
Table 5, block "UKCAT scores as threshold (n=6564)†":
Socioeconomic class 4 or 5‡ | 1.27 (0.84 to 1.91) | 0.3 | 0.13
Results, "Multilevel multiple logistic regression: applications" (the same state-school interval cited in the text):
Nevertheless this finding was consistent with the results from the separate models for the prediction of conditional offer provision (table 5) in that medical schools using UKCAT as a factor in the admissions process showed a relatively high degree of disadvantage for state school students (0.52, 0.45 to 0.60, P<0.001) even after controlling for the effects of other, potentially confounding, variables and interactions.
Methods, definition of the group:
As in previous research on widening participation we classified those who gave themselves a socioeconomic classification rating of four or more as being from a non-professional background.
Other estimates for the same predictor, checked as possible sources (Table 3, univariate, conditional offer, borderline / factor / threshold; Table 6, unconditional offer, borderline block; Table 7, pooled conditional-offer model):
Socioeconomic class 4 or 5‡ | 0.31 (0.19 to 0.50) | <0.001 | 0.34 | | 0.51 (0.40 to 0.65) | <0.001 | 0.38 | | 0.50 (0.33 to 0.76) | 0.001 | 0.49
Socioeconomic class 4 or 5‡ | 0.61 (0.39 to 0.95) | 0.03 | <0.001
Socioeconomic class*-borderline use interaction | 0.52 (0.29 to 0.91) | 0.02 | NA | NS
What the paper's own numbers show
Method. A 95% Wald interval for a logistic-regression odds ratio is exp(b ± 1.96·SE), where b is the log odds ratio. The standard error can be recovered from the printed limits as SE = (ln upper − ln lower) / (2 × 1.96), the point estimate should sit at the geometric midpoint √(lower × upper), and the two-sided P value is 2·(1 − Φ(|ln OR| / SE)). I applied this to the abstract interval, to the Table 5 row it summarises, and to every other socioeconomic estimate in the paper that could have been its source. I also checked the full rounding range of the printed values and every other odds ratio quoted in the abstract. All numbers below are printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Borderline use, SES 4/5, conditional offer | Abstract, Results | 0.51 (0.45 to 0.60); no P given | log width 0.288, SE 0.073, midpoint 0.520; implied P ≈ 4.5 × 10⁻²⁰ |
| Same estimate | Table 5, borderline block | 0.51 (0.32 to 0.81), P = 0.004 | log width 0.929, SE 0.237, midpoint 0.509, z = −2.84, P = 0.0045: consistent |
| Table 5 P over rounding range of OR and limits | derived | P = 0.004 | 0.0032 to 0.0062: includes 0.004 |
| Interval implied by Table 5's OR 0.51 and P = 0.004 | derived | 0.32 to 0.81 | 0.322 to 0.807 (0.320 to 0.812 for P = 0.0045) |
| Width of Table 5 interval relative to abstract | derived | — | 3.23 times as wide on the log scale |
| State (non-grammar) school, factor use | Table 5, factor block; Results text | 0.52 (0.45 to 0.60), P < 0.001 | same bounds as the abstract; midpoint 0.520 |
| Abstract interval midpoint, rounding range of limits | derived | 0.45 to 0.60 | 0.515 to 0.525 (fits 0.52 best; does not by itself exclude 0.51) |
| SES 4/5, factor use | Table 5, factor block | 0.62 (0.42 to 0.91), P = 0.01 | SE 0.197, P = 0.015; not the abstract interval |
| SES 4/5, threshold use (the abstract's comparator) | Abstract; Table 5, threshold block | 1.27 (0.84 to 1.91), P = 0.3 | identical in abstract and table; SE 0.210, P = 0.25 |
| SES 4/5, univariate, borderline / factor | Table 3 | 0.31 (0.19 to 0.50) / 0.51 (0.40 to 0.65) | neither matches 0.45 to 0.60 |
| SES 4/5, unconditional offer, borderline | Table 6 | 0.61 (0.39 to 0.95) | does not match |
| SES × borderline-use interaction, pooled | Table 7 | 0.52 (0.29 to 0.91), P = 0.02 | SE 0.292; does not match |
| Smallest SE possible with about 431 low-SES applications (5.7% of 7558, assuming the Table 1 share holds in the model sample) | Table 1; Table 5 n | — | √(4/431) = 0.096 unadjusted, before covariates and clustering; the abstract's 0.073 is below it, Table 5's 0.237 is above it |
| Low-SES applications needed for SE 0.073 | derived | — | at least 743 (9.8% of 7558) |
| Borderline vs threshold contrast, SES 4/5 (ratio 0.51/1.27) | Abstract; Table 5 | "No such effect ... threshold approach" | ratio 0.40; with Table 5 SEs: 95% CI 0.22 to 0.75, z = −2.88, P = 0.0039; with the abstract SE: CI 0.26 to 0.62, P = 4 × 10⁻⁵ |
| Other abstract ORs: 1.74 (1.25 to 2.41), 1.60 (0.97 to 2.62), 5.19 (2.02 to 13.33), 2.15 (1.03 to 4.48) | Abstract; Tables 11 and 12 | as in abstract | all identical to the tables |
| Abstract OR 3.57 (1.03 to 12.39), low-SES entrants | Abstract; Table 12 | 0.28 (0.08 to 0.97), borderline v factor | reciprocal 3.57 (1.03 to 12.5): consistent within rounding |
Table 5's socioeconomic row is internally consistent: its interval is centred on its odds ratio and reproduces its P value. The abstract's interval is not consistent with that row: it implies a standard error less than a third as large, and a P value about 10¹⁷ times smaller (about 9 × 10¹⁶) than the 0.004 printed in Table 5. Its bounds, 0.45 and 0.60, are exactly those of the state-school row in the factor-use block of the same table, whose odds ratio, 0.52, sits closer to the interval's geometric midpoint. A rough count-based check points the same way: if the Table 1 share applies, with about 431 low-SES applications in the borderline-use model, no adjusted estimate could have a log-odds-ratio SE as small as 0.073. Every other odds ratio and interval quoted in the abstract matches its table.
Figure
Figure 1. Odds ratios for a conditional offer with 95% confidence intervals on one logarithmic axis; the vertical line marks an odds ratio of 1. Orange: values printed in the paper (filled for socioeconomic class 4 or 5, hollow for the state-school predictor). Blue: the interval recomputed from Table 5's odds ratio 0.51 and P = 0.004, which reproduces Table 5's printed interval. The abstract interval (top row) is about a third as wide as the Table 5 interval for the same estimate (second row). The dotted guides show that its bounds coincide with those of state-school attendance in the factor-use model (fourth row). The threshold-use estimate (bottom row) is the comparator the abstract quotes, and it matches Table 5.
Figure 2. The abstract sentence (left) and the three Table 5 rows (right) that hold its numbers. The odds ratio 0.51 and the comparator 1.27 (0.84 to 1.91) match the socioeconomic rows of the borderline-use and threshold-use models. The interval 0.45 to 0.60 does not match the borderline-use socioeconomic row, which gives 0.32 to 0.81. It is identical to the interval for state-school attendance in the factor-use model.
Why it matters
The sentence is the abstract's only worked example of the paper's main claim, that disadvantage for under-represented groups depends on how schools use the UKCAT. Readers who see only the abstract are told that the socioeconomic disadvantage at borderline-use schools is estimated precisely, with a reduction in odds of 40% to 55%. The paper's own model gives a reduction of 19% to 68%, with P = 0.004 instead of the P of about 10⁻²⁰ that the abstract interval implies. The estimate rests on a small group: about 5.5% of applicants reported a non-professional background, and the Discussion itself says that because of these small numbers the study may have lacked power to detect differences between categories of medical school.
The error does not reverse any conclusion. Both intervals exclude 1, so the direction and statistical significance of the borderline-use result are unchanged. The contrast with threshold-use schools (1.27, 0.84 to 1.91) also survives with the correct interval: the ratio of the two odds ratios is 0.40 (95% CI 0.22 to 0.75, P ≈ 0.004 if the two models are treated as independent), and the pooled interaction in Table 7 gives 0.52 (0.29 to 0.91, P = 0.02). The main conclusions, the "What this study adds" box and the tables are unaffected. The affected reader is anyone who quotes or meta-analyses the abstract figure, or who judges from it how precisely the socioeconomic effect is known. Such a reader would take the uncertainty to be about a third of what the data support.
Explanations considered
- Conversion or typesetting error in the archived text. This does not account for the numbers. The PMC text and the publisher-deposited JATS XML in Europe PMC both carry "0.45 to 0.60" in the abstract and "0.51 (0.32 to 0.81)" in the correct Table 5 cell, in the borderline-use block. No minus signs, superscripts or decimal commas are involved. The publisher's own HTML and PDF on bmj.com could not be fetched (see Status).
- Rounding. This does not account for the numbers. Over the full rounding range of the printed odds ratio and limits, the Table 5 row gives P between 0.0032 and 0.0062, which includes the printed 0.004. The abstract interval gives P between 10⁻²³ and 10⁻¹⁷. A rounding change cannot turn a 0.93-wide log interval into a 0.29-wide one.
- A different model for the same predictor. This does not account for the numbers. None of the paper's other socioeconomic estimates has the interval 0.45 to 0.60: the factor-use model in Table 5 (0.42 to 0.91), the univariate models in Table 3 (0.19 to 0.50 for borderline use, 0.40 to 0.65 for factor use, 0.33 to 0.76 for threshold use), the unconditional-offer model in Table 6 (0.39 to 0.95), and the pooled interaction in Table 7 (0.29 to 0.91). The abstract's comparator (1.27, 0.84 to 1.91) matches Table 5 exactly, which identifies Table 5 as the source.
- The multiple-imputation sensitivity analysis. This is very unlikely to account for the numbers. Those results are not in the paper; they were placed on an author-hosted web page that could not be retrieved. The abstract SE is 3.23 times smaller than Table 5's, which would need about 10.4 times as many observations. Imputing the missing data could at most bring in the rest of the borderline-use sample (Table 1 lists 10 086, against 7558 applications in the model), which would shrink the SE only to about 0.205. The Results ("Missing data analysis") also say the imputed odds ratios differed from the complete-case ones by around 5% to 10%, and by 12% on average in the one exception, and the abstract point estimate is the complete-case value, 0.51.
- A different confidence level. This does not account for the numbers. With Table 5's standard error, 0.45 to 0.60 would be about a 46% interval, and the abstract labels it 95%.
- Copying of the adjacent state-school interval. This fits all the numbers, but the paper cannot confirm it. The bounds are identical to the factor-use state-school row of Table 5, which the Results text also cites as "(0.52, 0.45 to 0.60, P<0.001)". The abstract interval's geometric midpoint (0.520) matches that row's odds ratio, while 0.51 sits only at the edge of its rounding range.
Status
- Version of record. bmj.com returned HTTP 403 to every automated request: the DOI redirect, the article page, the full-text PDF and the rapid-responses page. Three PDF mirrors (bmj.com, the Durham repository and the Europe PMC render) returned bot-check pages instead of PDFs. I therefore checked the text in the publisher-deposited full text: the PMC article page (pmc.ncbi.nlm.nih.gov/articles/PMC3328544) and the Europe PMC full-text XML for PMC3328544. Both carry the abstract sentence and the Table 5 cells quoted above, with identical values. Every quotation above was confirmed verbatim against the PMC-archived text with
src/show.py. - Supplements. The journal published no data supplement. The Europe PMC supplementary-files package for PMC3328544 contains only the two article figures, and the XML has no supplementary-material element. The Data sharing statement points to an author-hosted page with further multiple-imputation details, which returned HTTP 403. The applicant-level data are not public, so the model cannot be refitted. Because Table 5 is internally consistent, the correct interval can be identified without them.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction notices. A web search for corrections or comments on the article found none. The PubPeer API refused automated access (HTTP 403), so PubPeer could not be checked directly. BMJ rapid responses could not be read (HTTP 403).
- Sources. PMC article PMC3328544; Europe PMC full-text XML and supplementary-files endpoint for PMC3328544; Crossref metadata for 10.1136/bmj.e1805; PMC and Europe PMC correction notices; a web search.
Suggested correction
- In the abstract's Results, replace "(odds ratio 0.51, 95% confidence interval 0.45 to 0.60)" with "(odds ratio 0.51, 95% confidence interval 0.32 to 0.81)", the interval given for this estimate in Table 5 (P = 0.004).
- No other change appears to be needed: Table 5, the Results text and the other abstract odds ratios agree with each other. If the multiple-imputation estimate was meant instead, report it with its own interval and say so in the abstract.
Recomputation script (python recompute.py.txt)
"""Recompute the abstract's borderline-use socioeconomic odds ratio interval in PMC3328544
(BMJ 2012, doi:10.1136/bmj.e1805, "Widening access to UK medical education for
under-represented socioeconomic groups: modelling the impact of the UKCAT in the 2009 cohort").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract, Results: borderline use, socioeconomic class 4/5, conditional offer: 0.51 (0.45 to 0.60)
threshold use, same predictor: 1.27 (0.84 to 1.91)
Table 5 (conditional offer, separate multilevel models):
borderline (n=7558): SES class 4 or 5 0.51 (0.32 to 0.81) P=0.004
factor (n=8163): state (non-grammar) school 0.52 (0.45 to 0.60) P<0.001
factor (n=8163): SES class 4 or 5 0.62 (0.42 to 0.91) P=0.01
threshold (n=6564): SES class 4 or 5 1.27 (0.84 to 1.91) P=0.3
Table 3 (univariate, conditional offer), SES 4/5: borderline 0.31 (0.19 to 0.50); factor 0.51 (0.40 to 0.65)
Table 6 (unconditional offer), borderline, SES 4/5: 0.61 (0.39 to 0.95) P=0.03
Table 7 (pooled, threshold base), SES class x borderline use interaction: 0.52 (0.29 to 0.91) P=0.02
Table 1: low socioeconomic status, applicants to borderline-use schools: 5.7%
Results text: state school, factor use: (0.52, 0.45 to 0.60, P<0.001)
Abstract other ORs: 1.74 (1.25 to 2.41), 1.60 (0.97 to 2.62), 5.19 (2.02 to 13.33),
2.15 (1.03 to 4.48), 3.57 (1.03 to 12.39)
Tables 11/12: 1.74 (1.25 to 2.41); 1.60 (0.97 to 2.62); 5.19 (2.02 to 13.33); 2.15 (1.03 to 4.48);
SES entrants, borderline v factor 0.28 (0.08 to 0.97)
Method: a 95% Wald interval for a logistic-regression odds ratio is exp(b +/- 1.96 SE), so
SE = (ln hi - ln lo) / (2 * 1.96), the point estimate sits at the geometric midpoint sqrt(lo * hi),
z = ln(OR) / SE and the two-sided P = 2 * (1 - Phi(|z|)).
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.isf(0.025)
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"svg.fonttype": "none", "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG})
def info(orr, lo, hi):
se = (math.log(hi) - math.log(lo)) / (2 * Z)
z = math.log(orr) / se
return dict(width=math.log(hi / lo), se=se, mid=math.sqrt(lo * hi), z=z,
p=2 * stats.norm.sf(abs(z)))
ROWS = [ # label, OR, lo, hi, reported P, location
("Abstract: borderline, SES 4/5", 0.51, 0.45, 0.60, "not given", "Abstract, Results"),
("Table 5 borderline: SES 4/5", 0.51, 0.32, 0.81, "0.004", "Table 5, borderline block"),
("Table 5 factor: state (non-grammar) school", 0.52, 0.45, 0.60, "<0.001", "Table 5, factor block; Results"),
("Table 5 factor: SES 4/5", 0.62, 0.42, 0.91, "0.01", "Table 5, factor block"),
("Table 5 threshold: SES 4/5", 1.27, 0.84, 1.91, "0.3", "Table 5, threshold block; Abstract"),
("Table 3 univariate borderline: SES 4/5", 0.31, 0.19, 0.50, "<0.001", "Table 3"),
("Table 3 univariate factor: SES 4/5", 0.51, 0.40, 0.65, "<0.001", "Table 3"),
("Table 6 unconditional, borderline: SES 4/5", 0.61, 0.39, 0.95, "0.03", "Table 6"),
("Table 7 pooled: SES x borderline interaction", 0.52, 0.29, 0.91, "0.02", "Table 7"),
]
print("=== 1. Log-scale width, SE, geometric midpoint and Wald P for each candidate interval ===")
R = {}
for lab, orr, lo, hi, prep, loc in ROWS:
r = info(orr, lo, hi)
R[lab] = r
print(f"{lab:46s} {orr:.2f} ({lo:.2f} to {hi:.2f}) width={r['width']:.3f} SE={r['se']:.4f} "
f"midpoint={r['mid']:.3f} z={r['z']:+.2f} P={r['p']:.2g} [reported P {prep}; {loc}]")
a, t5 = R["Abstract: borderline, SES 4/5"], R["Table 5 borderline: SES 4/5"]
print("\n=== 2. Abstract interval vs Table 5 interval for the same estimate ===")
print(f"width ratio Table 5 / abstract: {t5['width'] / a['width']:.2f}")
print(f"SE ratio Table 5 / abstract: {t5['se'] / a['se']:.2f}")
print(f"P implied by abstract interval around OR 0.51: {2 * stats.norm.sf(abs(math.log(0.51) / a['se'])):.2g}")
print(f"abstract interval midpoint {a['mid']:.3f} vs abstract OR 0.51; vs state-school OR 0.52")
print(f"Table 5 P 0.004 / abstract-implied P: {0.004 / a['p']:.1e}")
print(f"Table 5 interval midpoint {t5['mid']:.3f} vs Table 5 OR 0.51")
print("\n=== 3. Interval implied by Table 5's OR 0.51 and P = 0.004 ===")
for p in (0.004, 0.0035, 0.0045):
se = abs(math.log(0.51)) / stats.norm.isf(p / 2)
print(f"P={p}: SE={se:.4f} 95% CI {math.exp(math.log(0.51) - Z * se):.3f} to "
f"{math.exp(math.log(0.51) + Z * se):.3f}")
se_p = abs(math.log(0.51)) / stats.norm.isf(0.002)
CI_FROM_P = (math.exp(math.log(0.51) - Z * se_p), math.exp(math.log(0.51) + Z * se_p))
# rounding range for the Table 5 numbers
ps = []
for orr in np.linspace(0.505, 0.515, 11):
for lo in np.linspace(0.315, 0.325, 11):
for hi in np.linspace(0.805, 0.815, 11):
ps.append(info(orr, lo, hi)["p"])
print(f"Table 5 Wald P over the rounding range of OR and limits: {min(ps):.4f} to {max(ps):.4f}")
ps = []
for orr in np.linspace(0.505, 0.515, 11):
for lo in np.linspace(0.445, 0.455, 11):
for hi in np.linspace(0.595, 0.605, 11):
ps.append(info(orr, lo, hi)["p"])
print(f"abstract-interval Wald P over the same rounding range: {min(ps):.2g} to {max(ps):.2g}")
print("\n=== 4. Rough lower bound on SE from the number of low-SES applications ===")
n_bord, share = 7558, 0.057
n_low = n_bord * share
se_min = math.sqrt(4 / n_low) # 1/a + 1/b is smallest when offers and non-offers split 50:50
print(f"about {n_low:.0f} low-SES applications ({share:.1%} of {n_bord}); minimum unadjusted SE "
f"sqrt(4/{n_low:.0f}) = {se_min:.3f} (abstract interval implies {a['se']:.3f}; Table 5 {t5['se']:.3f})")
n_need = 4 / a["se"] ** 2
print(f"SE {a['se']:.3f} would need at least {n_need:.0f} low-SES applications ({n_need / n_bord:.1%} of {n_bord})")
print("\n=== 5. Borderline vs threshold contrast for SES 4/5 (separate models, independent samples) ===")
th = R["Table 5 threshold: SES 4/5"]
ratio = 0.51 / 1.27
for name, s in (("Table 5 interval", t5["se"]), ("abstract interval", a["se"])):
sed = math.sqrt(s ** 2 + th["se"] ** 2)
zc = math.log(ratio) / sed
print(f"{name:18s}: ratio {ratio:.2f}, SE of log ratio {sed:.3f} "
f"(95% CI {math.exp(math.log(ratio) - Z * sed):.2f} to {math.exp(math.log(ratio) + Z * sed):.2f}), "
f"z={zc:.2f}, P={2 * stats.norm.sf(abs(zc)):.2g}")
ti = R["Table 7 pooled: SES x borderline interaction"]
print(f"Table 7 pooled interaction 0.52 (0.29 to 0.91): SE {ti['se']:.3f}, z={ti['z']:.2f}, P={ti['p']:.2g}")
print("\n=== 6. Other abstract odds ratios against the tables ===")
for ab, tb in [("1.74 (1.25 to 2.41)", "Table 11: 1.74 (1.25 to 2.41)"),
("1.60 (0.97 to 2.62)", "Table 12: 1.60 (0.97 to 2.62)"),
("5.19 (2.02 to 13.33)", "Table 12: 5.19 (2.02 to 13.33)"),
("2.15 (1.03 to 4.48)", "Table 11: 2.15 (1.03 to 4.48)")]:
print(f"abstract {ab:22s} <-> {tb} identical: {ab == tb.split(': ')[1]}")
print(f"abstract 3.57 (1.03 to 12.39) <-> reciprocal of Table 12 0.28 (0.08 to 0.97): "
f"{1 / 0.28:.2f} ({1 / 0.97:.2f} to {1 / 0.08:.1f}); upper 12.39 = 1/{1 / 12.39:.4f}, within rounding of 0.08")
print("\n=== 7. Explanations: larger sample (imputation) or a different confidence level ===")
k = (t5["se"] / a["se"]) ** 2
print(f"(SE Table 5 / SE abstract)^2 = {k:.1f}: the abstract SE would need about {k:.1f} times as many observations")
print(f"Table 5 SE scaled from n=7558 to the 10 086 listed for borderline-use schools in Table 1: "
f"{t5['se'] * math.sqrt(7558 / 10086):.3f}")
zc = (math.log(0.60) - math.log(0.45)) / 2 / t5["se"]
print(f"with the Table 5 SE, 0.45 to 0.60 would be a {2 * stats.norm.cdf(zc) - 1:.0%} interval (z = {zc:.2f}), "
f"not 95%")
print(f"geometric midpoint of 0.45-0.60 = {a['mid']:.4f}; over the rounding range of the limits "
f"{math.sqrt(0.445 * 0.595):.4f} to {math.sqrt(0.455 * 0.605):.4f} (so it does not by itself exclude 0.51)")
# ------------------------------------------------------------------------------ figure 1
fig, ax = plt.subplots(figsize=(7.2, 4.3))
rows = [ # y label, OR, lo, hi, colour, filled, right-hand note
("Abstract\nborderline use, SES 4/5", 0.51, 0.45, 0.60, ORANGE, True,
"0.51 (0.45–0.60)\nimplies SE 0.073, P ≈ 5e-20"),
("Table 5\nborderline use, SES 4/5", 0.51, 0.32, 0.81, ORANGE, True,
"0.51 (0.32–0.81), P = 0.004\nimplies SE 0.237, P = 0.0045"),
("Recomputed from\nTable 5 OR and P", 0.51, CI_FROM_P[0], CI_FROM_P[1], BLUE, True,
f"{CI_FROM_P[0]:.2f}–{CI_FROM_P[1]:.2f}"),
("Table 5\nfactor use, state school", 0.52, 0.45, 0.60, ORANGE, False,
"0.52 (0.45–0.60)\nsame bounds as the abstract"),
("Table 5\nthreshold use, SES 4/5", 1.27, 0.84, 1.91, ORANGE, True,
"1.27 (0.84–1.91)\nthe abstract's comparator"),
]
ys = np.arange(len(rows))[::-1]
ax.axvline(1, color=SEC, lw=0.8, zorder=1)
for y in ys:
ax.axhline(y, color=GRID, lw=0.6, zorder=0)
for (lab, orr, lo, hi, col, filled, note), y in zip(rows, ys):
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
for b in (lo, hi):
ax.plot([b, b], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot(orr, y, "o", ms=9, color=col, mfc=col if filled else BG, mew=2, zorder=4)
ax.text(1.03, y, note, va="center", ha="left", fontsize=10, transform=ax.get_yaxis_transform(),
color=TXT if col == ORANGE else BLUE)
# dotted guides showing that the abstract interval and the state-school interval share both bounds
for b in (0.45, 0.60):
ax.plot([b, b], [ys[3], ys[0]], color=SEC, lw=0.9, ls=(0, (2, 2)), zorder=2)
ax.set_xscale("log")
ax.set_xlim(0.28, 2.2)
ax.set_xticks([0.32, 0.45, 0.6, 0.81, 1, 1.27, 1.91])
ax.set_xticklabels(["0.32", "0.45", "0.60", "0.81", "1", "1.27", "1.91"], fontsize=10)
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], fontsize=10, color=TXT)
ax.set_ylim(-0.7, len(rows) - 0.3)
ax.set_xlabel("Odds ratio for a conditional offer (95% CI), log scale\nvertical line at 1 = no difference")
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="reported"),
Line2D([], [], color=ORANGE, marker="o", ms=8, mfc=BG, mew=2, lw=2,
label="reported, other predictor"),
Line2D([], [], color=BLUE, marker="o", ms=8, lw=2, label="recomputed from the paper's numbers")]
fig.legend(handles=handles, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=3, frameon=False,
fontsize=10, handlelength=2.0, columnspacing=1.0, handletextpad=0.5)
fig.subplots_adjust(left=0.255, right=0.63, top=0.97, bottom=0.22)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), **kw)
plt.close(fig)
# ------------------------------------------------------------------------------ figure 2
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ax.set_xlim(0, 100)
ax.set_ylim(0, 57.5)
ax.axis("off")
fig.subplots_adjust(left=0.01, right=0.99, top=0.99, bottom=0.01)
def panel(x, y, w, h, title):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.4,rounding_size=1.2",
fc="#ffffff", ec=SEC, lw=1))
ax.text(x + 1.5, y + h - 2.5, title, fontsize=10.5, fontweight="bold", va="top")
panel(1, 6, 36, 50, "Abstract, Results")
ax.text(2.5, 48, "“...when applying to an\ninstitution using the test\nonly in borderline cases",
fontsize=10, va="top", color=TXT, linespacing=1.35)
ax.text(2.5, 34.5, "(odds ratio 0.51,", fontsize=10.5, va="center", color=ORANGE, fontweight="bold")
ax.text(2.5, 29.0, "95% confidence interval", fontsize=10, va="center", color=TXT)
ax.text(2.5, 24.5, "0.45 to 0.60).", fontsize=10.5, va="center", color=ORANGE, fontweight="bold")
ax.text(2.5, 17.5, "...threshold approach", fontsize=10, va="center", color=TXT)
ax.text(2.5, 13.0, "(1.27, 0.84 to 1.91).”", fontsize=10.5, va="center", color=ORANGE, fontweight="bold")
panel(58, 6, 41, 50, "Table 5 (conditional offer)")
blocks = [(45.5, "Borderline use (n=7558)", "SES class 4 or 5", "0.51 (0.32 to 0.81), P=0.004"),
(31.5, "Factor use (n=8163)", "State (non-grammar) school", "0.52 (0.45 to 0.60), P<0.001"),
(17.5, "Threshold use (n=6564)", "SES class 4 or 5", "1.27 (0.84 to 1.91), P=0.3")]
for y, blk, row, val in blocks:
ax.text(59.5, y, blk, fontsize=10, va="center", color=SEC)
ax.text(59.5, y - 4.0, row, fontsize=10, va="center", color=TXT)
ax.text(59.5, y - 8.0, val, fontsize=10.5, va="center", color=ORANGE, fontweight="bold")
def link(y0, y1, text, ty, col):
ax.add_patch(FancyArrowPatch((39.0, y0), (57.3, y1), arrowstyle="-|>", mutation_scale=12, color=col,
lw=1.5, shrinkA=0, shrinkB=0))
ax.text(47.6, ty, text, fontsize=10, ha="center", va="center", color=col, linespacing=1.2)
link(34.5, 37.5, "same OR,\nCI differs", 41.5, TXT)
link(24.5, 23.5, "identical CI,\nother predictor", 29.0, TXT)
link(13.0, 9.5, "matches", 14.8, SEC)
ax.text(1, 1.5, "Orange: values as printed in the paper. Arrows: where each abstract value is in Table 5.",
fontsize=10, color=SEC, va="center")
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), **kw)
plt.close(fig)
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract and Results print the 180-day taste-recovery estimate above its own 95% confidence interval
BMJ (2022) · doi:10.1136/bmj-2021-069503 · PMC9326326
Summary
The abstract and the Results give taste recovery at 180 days as "98.0% (92.2% to 95.5%)". The upper bound is 2.5 percentage points below the point estimate (2.40 points even at the rounding limits), so the interval cannot be a confidence interval for that estimate. The paper's other 11 recovery intervals (smell and taste at 30 to 180 days, main text and supplementary Table S8) are all symmetric on the log(1 − recovery) scale. Applied to 98.0% and 92.2%, that pattern gives an upper bound of 99.49%, and supplementary Table S8 prints this row as "98.0 (92.2 to 99.5)", so the main-text "95.5" looks like a one-digit slip for 99.5. The point estimate and every conclusion are unaffected, but the interval as printed in the abstract is impossible, and the August 2022 correction did not change it.
What the paper reports
The PMC text flattens superscripts (I2, τ2 stand for I², τ²); wording is otherwise verbatim.
Abstract, Results:
At 30, 60, 90, and 180 days, respectively, 74.1% (95% confidence interval 64.0% to 81.3%), 85.8% (77.6% to 90.9%), 90.0% (83.3% to 94.0%), and 95.7% (89.5% to 98.3%) of patients recovered their sense of smell (I2=0.0-77.2%, τ2=0.006-0.050) and 78.8% (70.5% to 84.7%), 87.7% (82.0% to 91.6%), 90.3% (83.5% to 94.3%), and 98.0% (92.2% to 95.5%) recovered their sense of taste (range of I2=0.0-72.1%, τ2=0.000-0.015).
Results, "Reconstructed IPD meta-analyses and cure models > Recovery of taste":
Recovery of taste (fig 2) occurred in 78.8% (95% confidence interval 70.5% to 84.7%) of patients at 30 days, 87.7% (82.0% to 91.6%) at 60 days, 90.3% (83.5% to 94.3%) at 90 days, and 98.0% (92.2% to 95.5%) at 180 days (I2=16%, τ2=0.000-0.015), with median recovery time of 12.4 days (95% confidence interval 10.3 to 16.3 days).
Results, "Recovery of smell" (for comparison, same method):
From the random effects distribution-free summary survival curves, 74.1% (95% confidence interval 64.0% to 81.3%) of participants recovered their sense of smell at 30 days, 85.8% (77.6% to 90.9%) at 60 days, 90.0% (83.3% to 94.0%) at 90 days, and 95.7% (89.5% to 98.3%) at 180 days
Methods, statistical analysis:
In the primary analysis, we used a non-parametric two stage random effects model to retrieve a distribution-free summary survival curve.
Figure 2 legend (how the confidence band of the summary curve was obtained):
translucent blue band represents 95% confidence intervals (CIs) obtained by extension of Greenwood’s formula with the delta method
Discussion, first paragraph, and the "What this study adds" box repeat the estimate as 98%:
In this meta-analysis of time-to-event data from 3699 patients in 18 studies, an estimated 74%, 86%, 90%, and 96% of patients self-reported smell recovery and 79%, 88%, 90%, and 98% self-reported taste recovery at 30, 60, 90, and 180 days, respectively.
Meta-analyses of reconstructed time-to-event individual patient data (IPD) showed that at 30, 60, 90, and 180 days, respectively, about 74%, 86%, 90%, and 96% of patients self-reported smell recovery and 79%, 88%, 90%, and 98% of patients self-reported taste recovery, among patients who remained alive at the given time
Supplementary Table S8 (GRADE evaluation of pooled IPD outcomes, supplement p. 43), taste recovery rows, as extracted from the PDF:
Day 30 78.8 (70.5 to 84.7) · Day 60 82.0 (82.0 to 91.6) · Day 90 90.3 (83.5 to 94.3) · Day 120 92.2 (85.9 to 95.7) · Day 150 97.1 (92.8 to 98.8) · Day 180 98.0 (92.2 to 99.5)
What the paper's own numbers show
Method. First, I checked whether each printed interval contains its estimate, including at the limits of one-decimal rounding. Second, I worked out on which scale the paper's intervals are symmetric. The summary curve is a survival curve S(t) = 1 − recovery, and its band comes from Greenwood's formula with the delta method, so a log-transformed interval for S is the natural candidate. For each interval I computed the half-widths a = log(Supper/S) and b = log(S/Slower), where the recovery interval (L, U) maps to S in (1 − U, 1 − L), and took their ratio b/a, which is 1 for an exactly symmetric interval. I used the 8 main-text intervals and the 4 extra time points (120 and 150 days) that only Table S8 gives. Third, I used that symmetry to recover the upper bound implied by the printed estimate and lower bound. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Does the taste 180-day CI contain its estimate? | Abstract; Results, "Recovery of taste" | 98.0% (92.2% to 95.5%) | No. The estimate exceeds the upper bound by 2.5 points; by at least 2.40 points at the rounding limits (97.95 vs 95.55) |
| Do the other 11 CIs contain their estimates? | Abstract; Results; Table S8 | smell and taste at 30, 60, 90, 120, 150, 180 days | Yes, all of them |
| b/a ratio, log(1 − recovery) scale, the other 11 intervals | derived | — | 0.97 to 1.04 (smell 0.99, 0.98, 1.00, 1.00, 1.03, 1.04; taste 0.99, 1.00, 1.00, 1.01, 0.97) |
| b/a ratio, taste 180 days, as printed | derived | — | −0.60 (b is negative: the "upper" bound is on the wrong side of the estimate) |
| Upper bound implied by 98.0 and 92.2, log(1 − recovery) symmetry | derived | 95.5% | 99.49% (S = 0.020, Supper = 0.078, Slower = 0.020²/0.078 = 0.00513); 99.46% to 99.52% over the rounding ranges of the inputs |
| Same, logit symmetry | derived | 95.5% | 99.51% |
| Taste 180-day upper bound in the supplement | Table S8, p. 43 | 99.5 | b/a = 1.02 with 99.5, in line with the other 11 intervals |
| Estimate at which 92.2–95.5 would be centred | derived | 98.0% | 94.08% (log(1 − recovery) midpoint); 94.06% (logit midpoint). This conflicts with 98.0 in the abstract, Results and Table S8 and with 98% in the Discussion and "What this study adds" |
The printed 95.5 and the value the other intervals imply, 99.5, differ in a single digit. The supplement prints 99.5 in the same position. The estimate 98.0 and the lower bound 92.2 appear identically in the main text and the supplement, and together with 99.5 they form an interval with the same shape as the paper's other intervals.
Figure
Figure 1. Taste recovery (%) with 95% CIs, on one linear axis from 68% to 102%; the dotted line marks 100%. Grey rows are the paper's intervals at 30, 60 and 90 days (abstract and Results) and at 120 and 150 days (Table S8 only), shown for context. The orange row is the 180-day interval printed in the abstract and Results, 98.0% (92.2% to 95.5%): the estimate (dot) lies to the right of the whole interval. The blue row keeps the printed estimate and lower bound and replaces the upper bound with the value implied by log(1 − recovery) symmetry, 99.49% (recompute.py.txt, section 3). The open orange diamond is the upper bound printed in supplementary Table S8, 99.5%.
Figure 2. For each of the paper's 12 recovery intervals (smell and taste at 30 to 180 days), the ratio b/a of the upper to the lower half-width on the log(1 − recovery) scale; 1 (dotted line) means exactly symmetric. Filled circles are intervals given in the main text, open circles those given only in Table S8. All 11 intervals other than taste at 180 days lie between 0.97 and 1.04. The printed taste 180-day interval gives −0.60 (orange), which no transformation can produce. Replacing its upper bound by the Table S8 value 99.5 gives 1.02 (blue diamond). This figure shows why the recomputed bound in Figure 1 is grounded in the paper's own method rather than assumed.
Why it matters
- The interval is impossible as printed, and it is in the abstract. A reader who takes the 180-day taste result from the abstract or the Results gets a confidence interval that excludes its own estimate. Any secondary use of the interval (for example, recomputing a standard error for a later meta-analysis or a GRADE imprecision judgement) would be wrong. Taken at face value, 92.2–95.5 would imply an estimate of about 94.1%, not 98.0%.
- The correct precision is much wider on the upper side than printed. With 99.5 as the upper bound, the interval runs from 92.2% to 99.5%. Table S8 notes that the 180-day taste estimate rests on a single study ("τ2 = N.A. (single study)"), which fits this wide interval.
- The estimate and the conclusions are not affected. The 98.0% estimate is the same in the abstract, the Results, the Discussion, "What this study adds" and Table S8. The paper's primary outcomes, as it states ("The primary outcomes were the proportions of patients remaining with smell or taste dysfunction."), are reported from the cure models as 5.6% (2.7% to 11.0%) for smell and 4.4% (1.2% to 14.6%) for taste. Those estimates, the worldwide projections of about 15 million and 12 million people, and the prognostic-factor odds ratios do not depend on this interval. Only the stated precision of one descriptive time point is wrong.
Explanations considered
- A different quantity, model or population for the interval. Not supported. In both places the interval directly follows the 98.0% estimate in a list of four estimate-and-CI pairs that share one sentence and one method (the random-effects distribution-free summary survival curve). No footnote or legend redefines it, and the other three pairs in each list are internally consistent.
- An asymmetric or back-transformed interval. Cannot explain it. The paper's intervals are asymmetric on the percentage scale but almost exactly symmetric on the log(1 − recovery) scale (b/a 0.97 to 1.04), as expected for a log-transformed Greenwood/delta-method interval. A monotone back-transformation keeps an estimate inside its interval, so no such transformation can put 98.0 above an upper bound of 95.5 (b/a = −0.60).
- Rounding. Cannot explain it. The smallest estimate that rounds to 98.0 (97.95) exceeds the largest upper bound that rounds to 95.5 (95.55) by 2.40 points.
- The point estimate, not the bound, is wrong. Not supported. 98.0 appears in the abstract, the Results and Table S8, and 98% appears in the Discussion and "What this study adds". An interval of 92.2–95.5 would centre on about 94.1%, a value the paper gives nowhere.
- The lower bound is wrong. Not supported. 92.2 appears identically in the main text and Table S8, and with 98.0 it implies an upper bound (99.49%) that matches the Table S8 value of 99.5.
- A text-conversion or typesetting artifact. Ruled out. The publisher's pages returned HTTP 403 to automated requests, so I could not read the typeset BMJ page. In the archived version of record, the figure is plain inline text (not a table cell) in two separate places, the abstract and the Results, and it reads "98.0% (92.2% to 95.5%)" in the PMC HTML page, the Europe PMC full-text XML and the PMC text read with
src/show.py. No minus signs, superscripts or special characters are involved. - A correction already issued. No. The one correction (see Status) does not concern this interval.
- Related reporting slips, not part of the main finding. (1) Table S8 gives taste recovery at 60 days as "82.0 (82.0 to 91.6)", with the estimate equal to its lower bound; the main text gives 87.7, which with 82.0–91.6 is log-symmetric (b/a = 1.00). (2) The 95% prediction interval for persistent taste dysfunction is "0.0% to 49.0%" in the abstract but "0.2% to 49.0%" in the Results; Table S8 gives "PI: 0.002 to 0.490". (3) The I² for the taste cure fraction is "I2=67%" in the abstract and Table S8 but "I2=0.0-72.1%" in the Results, the same range the abstract attaches to the taste recovery curves. None of these changes any conclusion.
Status
- Version of record. The BMJ article page, full-text PDF and rapid-responses page (reached through doi.org) returned HTTP 403 to automated requests. The archived version of record was checked in two independent renderings: the PMC article page (https://pmc.ncbi.nlm.nih.gov/articles/PMC9326326/) and the Europe PMC full-text XML. Both carry "98.0% (92.2% to 95.5%)" in the abstract and in the Results. Every quotation from the main text above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements. The single supplementary file,
tanb069503.ww.pdf(61 pages), was obtained from the Europe PMC supplementary-files bundle for PMC9326326. Its Table S8 (p. 43) prints the 180-day taste row as "98.0 (92.2 to 99.5)" from "1498 (10 studies)", and its smell rows match the main text. The supplement resolves the error: the main-text upper bound should read 99.5%. - Corrections and notices. One correction exists: BMJ 2022;378:o1939, doi:10.1136/bmj.o1939 (PMC9361052), dated 9 August 2022, found through the Crossref updated-by relation,
src/known.pyand Europe PMC. Its text, read in full via Europe PMC, only inserts the phrase "with initial dysfunction" into the abstract's sentence on cure-model estimates, the fourth point of "What this study adds" and the second paragraph of the Discussion. It does not mention the 180-day taste interval. PMC metadata also links an expression of concern (PMC9691145); its Europe PMC record is titled "Expression of concern: 6-month consequences of COVID-19 in patients discharged from hospital: a cohort study" (Lancet, doi:10.1016/s0140-6736(22)02370-4), so it concerns a primary study, not this paper. No retraction was found. - PubPeer and comments. The PubPeer API required human verification and returned no data to an automated DOI query, so PubPeer could not be checked directly. A web search for the title with "PubPeer" found no thread, and a web search for the exact string "98.0% (92.2% to 95.5%)" found no discussion of the interval. The BMJ rapid responses could not be read (HTTP 403).
- Sources (as of 2026-10-24). PMC article PMC9326326; Europe PMC full-text XML and supplementary files for PMC9326326; supplement
tanb069503.ww.pdf; correction PMC9361052 (doi:10.1136/bmj.o1939); Europe PMC record PMC9691145; Crossref; PubPeer; https://doi.org/10.1136/bmj-2021-069503.
Suggested correction
In the Results section of the abstract and in the Results paragraph "Recovery of taste", replace
98.0% (92.2% to 95.5%)
with
98.0% (92.2% to 99.5%)
matching supplementary Table S8, after the authors confirm the value from their model output. They may also wish to correct, in Table S8, the 60-day taste estimate from 82.0 to 87.7, and to reconcile the lower limit of the 95% prediction interval for persistent taste dysfunction (0.0% in the abstract, 0.2% in the Results and Table S8) and the I² reported for the taste cure fraction (67% in the abstract and Table S8, 0.0-72.1% in the Results).
Recomputation script (python recompute.py.txt)
"""Recompute the consistency checks for the 180-day taste-recovery interval in PMC9326326
(BMJ 2022, doi:10.1136/bmj-2021-069503, "Prognosis and persistence of smell and taste
dysfunction in patients with covid-19: meta-analysis with parametric cure modelling of
recovery curves").
Run: python recompute.py.txt
Needs numpy and matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract (Results) and Results "Recovery of smell" / "Recovery of taste": recovery at
30, 60, 90 and 180 days with 95% CIs (percent).
Supplementary Table S8 (tanb069503.ww.pdf, p. 43): the same quantities at 30-180 days in
30-day steps, including 120 and 150 days, which the main text does not give.
"""
import os
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs (percent)
# (outcome, day, estimate, lower, upper, source)
MAIN = [
("smell", 30, 74.1, 64.0, 81.3, "main text"),
("smell", 60, 85.8, 77.6, 90.9, "main text"),
("smell", 90, 90.0, 83.3, 94.0, "main text"),
("smell", 180, 95.7, 89.5, 98.3, "main text"),
("taste", 30, 78.8, 70.5, 84.7, "main text"),
("taste", 60, 87.7, 82.0, 91.6, "main text"),
("taste", 90, 90.3, 83.5, 94.3, "main text"),
("taste", 180, 98.0, 92.2, 95.5, "main text"),
]
# Supplementary Table S8 rows not in the main text, plus its 180-day taste row
SUPP = [
("smell", 120, 92.5, 85.6, 96.1, "Table S8"),
("smell", 150, 95.1, 88.3, 98.0, "Table S8"),
("taste", 120, 92.2, 85.9, 95.7, "Table S8"),
("taste", 150, 97.1, 92.8, 98.8, "Table S8"),
]
S8_TASTE180 = (98.0, 92.2, 99.5) # Table S8: "98.0 (92.2 to 99.5)"
S8_TASTE60 = (82.0, 82.0, 91.6) # Table S8: "82.0 (82.0 to 91.6)" (main text: 87.7)
def logit(p):
return np.log(p / (1 - p))
def expit(x):
return 1 / (1 + np.exp(-x))
def logS_halfwidths(e, lo, hi):
"""Half-widths of the interval on the log(1 - recovery) scale.
Survival S = 1 - recovery; the recovery CI (lo, hi) maps to S in (1-hi, 1-lo).
Returns (log(S_upper / S), log(S / S_lower)) and their ratio."""
S, S_up, S_low = 1 - e / 100, 1 - lo / 100, 1 - hi / 100
a = np.log(S_up) - np.log(S)
b = np.log(S) - np.log(S_low)
return a, b, b / a
# ------------------------------------------------------------------ 1. containment
print("=== 1. Does each reported interval contain its estimate? ===")
for o, d, e, lo, hi, src in MAIN + SUPP:
print(f"{o} {d:3d} d {e:5.1f} ({lo:5.1f} to {hi:5.1f}) [{src:9s}] contains estimate: {lo <= e <= hi}")
e, lo, hi = 98.0, 92.2, 95.5
print(f"\nTaste 180 d: estimate - upper bound = {e - hi:.1f} percentage points")
print(f"At the rounding limits: smallest estimate 97.95 - largest upper 95.55 = {97.95 - 95.55:.2f} points")
# ------------------------------------------------------------------ 2. scale of the interval
print("\n=== 2. Symmetry of the intervals on the log(1 - recovery) scale ===")
print("(a = log(S_upper/S), b = log(S/S_lower), S = 1 - recovery; a log-transformed CI gives b/a = 1)")
ratios = []
for o, d, e_, lo_, hi_, src in sorted(MAIN + SUPP, key=lambda r: (r[0], r[1])):
a, b, r = logS_halfwidths(e_, lo_, hi_)
ratios.append((o, d, src, r))
print(f"{o} {d:3d} d [{src:9s}] a={a:6.3f} b={b:7.3f} b/a={r:6.2f}")
ok = [r for o, d, s, r in ratios if not (o == "taste" and d == 180)]
print(f"The 11 intervals other than taste 180 d: b/a from {min(ok):.2f} to {max(ok):.2f}")
a, b, r = logS_halfwidths(*S8_TASTE180)
print(f"Taste 180 d with Table S8 upper bound 99.5: a={a:.3f} b={b:.3f} b/a={r:.2f}")
print("\n--- same check on the logit scale (for comparison) ---")
for o, d, e_, lo_, hi_, src in MAIN:
p, l, h = e_ / 100, lo_ / 100, hi_ / 100
a_, b_ = logit(p) - logit(l), logit(h) - logit(p)
print(f"{o} {d:3d} d below={a_:6.3f} above={b_:7.3f} ratio={b_ / a_:6.2f}")
# ------------------------------------------------------------------ 3. implied upper bound
print("\n=== 3. Upper bound implied by the estimate (98.0) and lower bound (92.2) ===")
S, S_up = 0.020, 0.078
S_low = S * S / S_up
up_logS = 100 * (1 - S_low)
print(f"log(1-recovery) symmetry: S_lower = {S_low:.5f} -> upper bound {up_logS:.2f}%")
up_logit = 100 * expit(2 * logit(0.98) - logit(0.922))
print(f"logit symmetry: upper bound {up_logit:.2f}%")
# sensitivity to rounding of the printed estimate and lower bound
vals = []
for e_ in (97.95, 98.05):
for lo_ in (92.15, 92.25):
s, su = 1 - e_ / 100, 1 - lo_ / 100
vals.append(100 * (1 - s * s / su))
print(f"Over the rounding ranges (98.0 +/- 0.05, 92.2 +/- 0.05): {min(vals):.2f}% to {max(vals):.2f}%")
print(f"Table S8 prints 99.5; the main text prints 95.5 (one digit differs)")
# ------------------------------------------------------------------ 4. alternative: estimate wrong
print("\n=== 4. If 92.2-95.5 were the correct interval, which estimate would it centre on? ===")
mid_logS = 100 * (1 - np.sqrt((1 - 0.922) * (1 - 0.955)))
mid_logit = 100 * expit((logit(0.922) + logit(0.955)) / 2)
print(f"log(1-recovery) midpoint: {mid_logS:.2f}% logit midpoint: {mid_logit:.2f}%")
print("The paper gives 98.0% (abstract, Results, Table S8) and 98% (Discussion, 'What this study adds').")
# ------------------------------------------------------------------ 5. side observation
print("\n=== 5. Side observation: Table S8 taste 60 d ===")
print(f"Table S8 prints {S8_TASTE60[0]} ({S8_TASTE60[1]} to {S8_TASTE60[2]}); main text 87.7 (82.0 to 91.6)")
a, b, r = logS_halfwidths(87.7, 82.0, 91.6)
print(f"With 87.7 the interval is log-symmetric (b/a={r:.2f}); with 82.0 the estimate equals its lower bound")
# ------------------------------------------------------------------ figures
BG, TXT, SEC, ORANGE, BLUE, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# ---- fig1: taste recovery estimates with intervals, one axis (percent recovered)
taste = sorted([r for r in MAIN + SUPP if r[0] == "taste" and r[1] != 180], key=lambda r: r[1])
fig, ax = plt.subplots(figsize=(7.2, 4.8))
rows = [] # (label, est, lo, hi, color, style)
for o, d, e_, lo_, hi_, src in taste:
lab = f"Day {d}" + (" (Table S8)" if src == "Table S8" else "")
rows.append((lab, e_, lo_, hi_, SEC, "context"))
rows.append(("Day 180, as printed\n(abstract, Results)", 98.0, 92.2, 95.5, ORANGE, "reported"))
rows.append(("Day 180, upper bound\nrecomputed from 98.0, 92.2", 98.0, 92.2, up_logS, BLUE, "recomputed"))
ys = np.arange(len(rows))[::-1]
for y, (lab, e_, lo_, hi_, c, kind) in zip(ys, rows):
ax.plot([lo_, hi_], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=2)
for xb in (lo_, hi_):
ax.plot([xb, xb], [y - 0.13, y + 0.13], color=c, lw=2, zorder=2)
ax.plot(e_, y, "o", ms=9, color=c, mec=c, zorder=3)
if kind == "reported":
ax.text(69.0, y, "estimate 98.0 lies 2.5 points\nabove its upper bound 95.5",
ha="left", va="center", color=ORANGE, fontsize=10)
for xv in (lo_, hi_):
ax.text(xv, y - 0.3, f"{xv:.1f}", ha="center", va="center", color=ORANGE, fontsize=10)
ax.text(e_, y + 0.3, f"{e_:.1f}", ha="center", va="center", color=ORANGE, fontsize=10)
if kind == "recomputed":
ax.text(lo_, y - 0.3, f"{lo_:.1f}", ha="center", va="center", color=BLUE, fontsize=10)
ax.text(hi_ - 0.2, y - 0.3, f"{hi_:.2f}", ha="center", va="center", color=BLUE, fontsize=10)
ax.plot(99.5, y + 0.3, "D", ms=8, mfc="none", mec=ORANGE, mew=1.6, zorder=3)
ax.text(98.9, y + 0.3, "Table S8 prints 99.5", ha="right", va="center",
color=ORANGE, fontsize=10)
ax.axvline(100, color=SEC, lw=0.8, ls=(0, (2, 2)), zorder=1)
ax.text(100.2, ys[0] + 0.15, "100%", color=SEC, fontsize=10, va="center", ha="left")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
for t, r in zip(ax.get_yticklabels(), rows):
t.set_color({"reported": ORANGE, "recomputed": BLUE}.get(r[5], SEC))
ax.set_xlim(68, 102)
ax.set_ylim(-0.7, len(rows) - 0.4)
ax.set_xlabel("Patients who recovered their sense of taste (%)")
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
fig.tight_layout(rect=(0, 0, 1, 0.94))
fig.text(0.01, 0.975, "Taste recovery with 95% CIs: the printed 180-day interval excludes its estimate",
ha="left", va="top", fontsize=11.5, color=TXT)
save(fig, "fig1")
# ---- fig2: log(1 - recovery) half-width ratio for every interval
fig, ax = plt.subplots(figsize=(7.2, 3.9))
order = sorted(MAIN + SUPP, key=lambda r: (r[0], r[1]))
labels, xs = [], []
for i, (o, d, e_, lo_, hi_, src) in enumerate(order):
_, _, r = logS_halfwidths(e_, lo_, hi_)
is_bad = (o == "taste" and d == 180)
c = ORANGE if is_bad else SEC
ax.plot(i, r, "o", ms=9, color=c, zorder=3, mfc=c if src == "main text" else BG, mec=c, mew=1.6)
labels.append(f"{o}\n{d} d")
if is_bad:
ax.annotate(f"as printed (upper 95.5): {r:.2f}", xy=(i, r), xytext=(i - 0.3, r + 0.05),
ha="right", va="center", color=ORANGE, fontsize=10)
_, _, r2 = logS_halfwidths(*S8_TASTE180)
ax.plot(i, r2, "D", ms=9, color=BLUE, zorder=3)
ax.annotate(f"with upper 99.5\n(Table S8): {r2:.2f}", xy=(i, r2), xytext=(i - 0.3, r2 - 0.42),
ha="right", va="center", color=BLUE, fontsize=10)
ax.axhline(1, color=SEC, lw=0.8, ls=(0, (2, 2)), zorder=1)
ax.text(-0.45, 1.07, "1 = symmetric", color=SEC, fontsize=10, ha="left")
ax.axhline(0, color=GRID, lw=0.8)
ax.set_xticks(range(len(order)))
ax.set_xticklabels(labels, fontsize=10)
ax.set_xlim(-0.6, len(order) - 0.4)
ax.set_ylim(-0.85, 1.35)
ax.set_ylabel("Upper ÷ lower half-width\non log(1 − recovery) scale")
ax.grid(axis="y", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.text(len(order) - 0.45, 1.28, "filled: main text open: Table S8 only", ha="right",
va="center", color=SEC, fontsize=10)
ax.set_title("Every other interval is symmetric on the log(1 − recovery) scale",
loc="left", fontsize=11.5, color=TXT)
fig.tight_layout()
save(fig, "fig2")
print("\nWrote fig1.svg, fig1.png, fig2.svg, fig2.png to", HERE)
Abstract reports the deductive-reasoning test as F(2,387), an error df larger than the whole sample
Brain (2025) · doi:10.1093/brain/awaf062 · PMC12073978
Summary
The Abstract reports the omnibus group comparison on the Deductive Reasoning Test (DRT) as F(2,387) = 18.491, P < 0.001. The Results report the same test as F(2,142) = 7.73, P < 0.001. The Results value fits the paper's own numbers exactly: Table 2 lists 48 + 55 + 43 = 146 participants, and an age-adjusted ANCOVA across three groups leaves 146 − 4 = 142 error degrees of freedom. An error df of 387 would need 391 participants, more than the 328 people recruited in total. Significance and direction are the same under either value, and the effect size each tuple implies is similar (partial η² 0.087 and 0.098). But the Abstract makes the evidence for the deductive-reasoning difference look as strong as that for the analogical-reasoning difference (F = 18.93) and implies a sample more than twice the size of the one analysed.
What the paper reports
ART and DRT performance was significantly impaired in patients with frontal lesions [ART: F(2,238) = 18.93; P < 0.001; Frontal group worse than Posterior group and healthy controls, both P < 0.001; DRT: F(2,387) = 18.491; P < 0.001; Frontal group worse than healthy controls, P < 0.01].
— Abstract
There was a significant difference between the Frontal, Posterior and HC groups for overall DRT performance [F(2,142) = 7.73; P < 0.001; see Table 2].
— Results, "Deductive reasoning task", first paragraph
HC(n = 48) | Frontal(n = 55) | Posterior(n = 43) | Left frontal(n = 24) | Right frontal(n = 31) | Left posterior(n = 19) | Right posterior(n = 24)
— Table 2, "Performance on the Deductive Reasoning Task", column headers. The table footer reads: "Groups are compared using analysis of covariance (ANCOVA), controlling for age."
Frontal, Posterior and HC groups were compared on demographic variables, using ANOVA and Fisher’s exact test, and performance on neuropsychological tests, ART and DRT, using analysis of covariance (ANCOVA), controlling for age.
— Methods, "Behavioural analysis"
Behavioural analyses were conducted on the 176 patients with lesions falling within frontal or posterior areas and HCs using SPSS version 29 (https://www.ibm.com/spss).
— Methods, "Behavioural analysis"
Two-hundred and forty-seven patients with unilateral, focal lesions were prospectively recruited from the National Hospital for Neurology and Neurosurgery
Eighty-one HCs, with no neurological or psychiatric history, were recruited to match patients as closely as possible for age, gender, years of education and National Adult Reading Test (NART) scores.
— Methods, "Participants"
What the paper's own numbers show
Method. In a one-way ANCOVA with k = 3 groups and one covariate (age), the F test for the group effect has k − 1 = 2 numerator df and N − k − 1 = N − 4 error df. The group sizes printed in the table headers therefore fix the error df. The same rule was checked against every other F test in the Results whose groups are tabulated. The age-adjusted F itself cannot be rebuilt from summary tables, because that needs the covariance of age with the scores. As a reference point only, the unadjusted one-way ANOVA F was computed from each table's n, means and SDs. Every number below is printed by recompute.py.txt (Python, numpy/scipy).
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| DRT sample size | Table 2 header | HC 48, Frontal 55, Posterior 43 | N = 146; subgroups agree (24 + 31 = 55; 19 + 24 = 43) |
| DRT error df, ANCOVA with age | Results | 142 | 146 − 4 = 142 (matches) |
| DRT error df | Abstract | 387 | would need N = 391 (ANCOVA) or 390 (no covariate) |
| Largest possible sample | Methods, Participants | 247 patients + 81 HCs | 328 in total, so a three-group ANCOVA on all of them has error df 324 |
| Behavioural analysis set | Methods, Behavioural analysis | 176 patients + HCs | 176 + 81 = 257 |
| P for F(2,142) = 7.73 | Results | P < 0.001 | 6.5 × 10⁻⁴ |
| P for F(2,387) = 18.491 | Abstract | P < 0.001 | 2.1 × 10⁻⁸ |
| Partial η² implied by each tuple | Results / Abstract | — | 0.098 / 0.087 |
| Ratio of the two DRT F values | Abstract vs Results | 18.491 vs 7.73 | 2.39 |
| Unadjusted DRT F from Table 2 n, means, SDs | Table 2 | — | F(2,143) = 6.80, P = 0.0015 (Results F is 1.14 times this; Abstract F is 2.72 times) |
| Control: ART sample and error df | Table 1; Abstract; Results | n 72 + 100 + 70; df 238 | N = 242; 242 − 4 = 238 (matches) |
| Control: unadjusted ART F from Table 1 | Table 1 | F(2,238) = 18.93 | F(2,239) = 17.36 (reported is 1.09 times this) |
| Control: DRT indeterminate / determinate | Results | F(2,142) = 7.93 / 4.23 | unadjusted 7.18 / 3.81 (ratios 1.10 / 1.11); df 142 matches |
| Control: ART progression / odd-one-out | Results | F(2,238) = 15.20 / 11.74 | unadjusted 14.16 / 11.13 (ratios 1.07 / 1.06); df 238 matches |
| Control: pooled Frontal and Posterior DRT means | Table 2 | 71.67 / 79.37 | 71.68 / 79.37 from the hemisphere subgroups |
| Other error dfs in Results (RAPM, S fluency, Stroop) | Results | 184, 151, 150 | imply N = 188, 155, 154, all ≤ 257 |
Every ART and DRT F test in the Results has error df = N − 4 for its table, and each of the six tabulated ANCOVA Fs lies 6–14% above the unadjusted F from the same table. The Abstract's DRT tuple is the only one that fits neither pattern.
Figure
Figure 1. The two printed versions of the DRT omnibus test. Left: the Abstract's F(2,387) = 18.491 (orange). Under the ANCOVA described in Methods (three groups, age as covariate) an error df of 387 corresponds to N = 391 (gray, derived from the Abstract's df). Right: the Results' F(2,142) = 7.73 and the Table 2 group sizes (black, as printed), from which N = 146 and error df 146 − 4 = 142 are recomputed (blue). The study recruited 328 participants in total (247 patients and 81 healthy controls), so no analysis described in the paper could have 387 error df.
Figure 2. Reported error df (orange circles) and the error df recomputed as N − 4 from the group sizes in Table 1 or Table 2 (blue diamonds) for every ART and DRT omnibus F test in the paper. The seven further ART tests (progression, odd-one-out, intra- and cross-dimensional and others) all report df 238 and are drawn as one row. The dotted line marks 324, the error df of a three-group ANCOVA with age on all 328 recruited participants. The Abstract's DRT value is the only one that departs from N − 4, and it lies beyond that ceiling.
Why it matters
The Abstract gives the DRT omnibus test as one of two headline behavioural results, set beside the ART result as evidence that "ART and DRT performance was significantly impaired in patients with frontal lesions". Read from the Abstract, the evidence for the two group differences looks about equally strong (F = 18.93 and F = 18.491, both with P near 2 × 10⁻⁸). By the paper's own Results, the evidence for the DRT difference is clearly weaker (F = 7.73, P = 6.5 × 10⁻⁴), and it rests on 146 participants, not the roughly 391 that the printed df implies. The DRT effect size is smaller than the ART one under either tuple (partial η² 0.087 or 0.098, against 0.137 for the ART).
The stated conclusions do not change. Both tuples give P < 0.001, the Frontal group is worse than healthy controls under either, and the implied partial η² is similar (0.087 and 0.098). The paper's main findings — the right-versus-left frontal differences on the ART and on DRT indeterminate questions, and the lesion-deficit network mapping — rest on other analyses.
The readers most likely to be affected are those who take test statistics or sample sizes from abstracts, such as reviewers judging sample size and precision, or meta-analysts. A meta-analyst who converts the Abstract tuple gets a slightly smaller partial η² (0.087 instead of 0.098) and a much larger N (391 instead of 146), which would give the DRT result far more weight than its sample supports. The Abstract also presents the DRT as a promising clinical test. For a reader weighing that claim, the Abstract overstates the strength of evidence and the sample size behind the DRT result, while the effect size it implies is close to the Results value.
Explanations considered
- A different or larger analysis population. This does not account for the numbers. Methods state that behavioural analyses used the 176 frontal and posterior patients and the HCs, at most 257 people. Even a three-group ANCOVA on all 247 recruited patients and all 81 HCs together (328) would have an error df of 324, not 387.
- An analysis without the age covariate. This does not account for the numbers. Dropping the covariate raises the error df by one (390 needed instead of 391), which still exceeds the recruited sample.
- Repeated measures, or pooling determinate and indeterminate questions. This does not account for the numbers. The paper analyses the two question types in separate ANCOVAs, each with df 142. Pooling 146 participants' two scores would give 292 observations and an error df of 288, not 387, and between-subject effects in a repeated-measures model would keep df 142.
- A typing slip in the df alone. This does not account for the numbers. The F value also differs (18.491 against 7.73), and it is reported to three decimals where every other F in the paper has two.
- Age adjustment raising F. This does not account for the numbers on its own terms. The same paper reports the age-adjusted F for this test as 7.73. Across all six tabulated tests the reported ANCOVA F is 1.06–1.14 times the unadjusted F, while the Abstract's 18.491 would be 2.72 times.
- A conversion or typesetting artefact in the archived copy. This was ruled out. The publisher's version of record and the Europe PMC full-text XML give the same Abstract text ("F(2,387) = 18.491") and the same Results text ("F(2,142) = 7.73") as the PMC archive.
- An analysis reported only in the supplement. This was ruled out. The supplementary PDF contains task descriptions, lesion methods and two supplementary tables (demographics and background tests). It reports no DRT omnibus statistic and no group sizes that could give an error df of 387.
The Abstract's 18.491 is numerically close to the ART statistic, 18.93. That is an observation only; the paper's numbers do not show how the Abstract value arose.
Status
As of 2026-10-23:
- Version of record: checked. doi.org resolves to the publisher page (Brain 148(5):1757). Its Abstract reads "DRT: F(2,387) = 18.491; P < 0.001" and its Results read "F(2,142) = 7.73; P < 0.001". The Europe PMC JATS XML and the local PMC archive match, and the Table 2 header in the XML gives n = 48, 55 and 43. The PMC HTML page returned a CAPTCHA, so Europe PMC was used in its place.
- Supplement: checked. The supplementary bundle from Europe PMC holds the same supplementary PDF (14 pages) linked from the publisher page. Its text was searched for "387", "18.49", "F(2", "DRT" and "Deductive". It does not resolve the discrepancy (see above).
- Corrections: none found. Crossref lists no updates (no update-to or updated-by entries, and no Retraction Watch entry in the Crossref labs record), and PMC and Europe PMC list no notices, checked with
src/known.pyand the Crossref API. Europe PMC (PMID 40233941) links only to a scientific commentary in the same issue (doi:10.1093/brain/awaf117), which quotes no F statistics and mentions no error. The publisher page shows no correction notice. - PubPeer: the PubPeer API returned HTTP 403 to an automated query. A web search for the title with "PubPeer", "correction" or "erratum" found no thread, erratum or published comment. A manual search at pubpeer.com is still advisable.
Sources: doi.org/10.1093/brain/awaf062 · publisher article page · PMC12073978 · Europe PMC record · Crossref works record · scientific commentary
Suggested correction
The authors could check the DRT omnibus ANCOVA in the Abstract against the original output and align it with the Results. If the Results value is correct, the Abstract would read "DRT: F(2,142) = 7.73; P < 0.001; Frontal group worse than healthy controls, P < 0.01". If 18.491 comes from a different analysis, the Abstract should say which analysis it is and how many participants it included, since no analysis described in the paper has an error df of 387.
Recomputation script (python recompute.py.txt)
"""Recompute the Deductive Reasoning Test (DRT) omnibus statistics of PMC12073978
(Brain 2025, doi:10.1093/brain/awaf062) and regenerate the figures.
Run: python recompute.py.txt
Needs: numpy, scipy, matplotlib.
Prints every recomputed number used in writeup.md. Writes fig1.svg/.png and
fig2.svg/.png into the folder that holds this file, plus five candidate figure
designs (cand_A ... cand_E, .svg/.png) into ./candidates/.
All inputs are copied from the paper (Abstract; Methods, Participants and
Behavioural analysis; Results; Tables 1 and 2).
Model stated in Methods: one-way ANCOVA across three groups (Frontal, Posterior,
healthy controls) with age as the single covariate, so
group df = k - 1 = 2, error df = N - k - 1 = N - 4.
The age-adjusted F cannot be rebuilt from the tables (it needs the age-score
covariance), so the unadjusted one-way ANOVA F is computed from the printed
group n, means and SDs as a reference point only.
"""
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
# ---------------------------------------------------------------- inputs
N_PATIENTS_RECRUITED = 247 # Methods, Participants
N_PATIENTS_FP = 176 # frontal 102 + posterior 74
N_HC = 81 # Methods, Participants
DRT_N = {"HC": 48, "Frontal": 55, "Posterior": 43} # Table 2 header
DRT_SUB = {"LF": 24, "RF": 31, "LP": 19, "RP": 24}
ART_N = {"HC": 72, "Frontal": 100, "Posterior": 70} # Table 1 header
ABSTRACT_DRT = (2, 387, 18.491) # Abstract
RESULTS_DRT = (2, 142, 7.73) # Results, Deductive reasoning task, p0
ART_REPORTED = (2, 238, 18.93) # Abstract and Results
# (label, ns, means, sds, reported F, reported df2, location)
TESTS = [
("DRT overall", list(DRT_N.values()), [82.79, 71.67, 79.37], [15.22, 15.90, 16.03], 7.73, 142, "Results"),
("DRT indeterminate", list(DRT_N.values()), [76.06, 55.89, 70.56], [25.21, 30.53, 27.60], 7.93, 142, "Results"),
("DRT determinate", list(DRT_N.values()), [84.79, 76.69, 82.14], [15.42, 15.12, 15.17], 4.23, 142, "Results"),
("ART overall", list(ART_N.values()), [79.40, 67.79, 77.38], [11.93, 15.51, 13.51], 18.93, 238, "Abstract; Results"),
("ART progression", list(ART_N.values()), [83.22, 71.94, 81.19], [12.86, 16.60, 14.52], 15.20, 238, "Results"),
("ART odd-one-out", list(ART_N.values()), [75.58, 63.71, 73.57], [15.09, 20.06, 17.08], 11.74, 238, "Results"),
]
def anova_from_summary(ns, means, sds):
ns, m, s = (np.asarray(x, float) for x in (ns, means, sds))
N, k = ns.sum(), len(ns)
gm = (ns * m).sum() / N
ssb = (ns * (m - gm) ** 2).sum()
ssw = ((ns - 1) * s ** 2).sum()
F = (ssb / (k - 1)) / (ssw / (N - k))
return int(N), k, F, stats.f.sf(F, k - 1, N - k)
def p_of(F, d1, d2):
return stats.f.sf(F, d1, d2)
def eta_p2(F, d1, d2):
return F * d1 / (F * d1 + d2)
# ---------------------------------------------------------------- numbers
print("== Sample sizes ==")
n_drt = sum(DRT_N.values())
n_art = sum(ART_N.values())
print(f"DRT analysed (Table 2): 48 + 55 + 43 = {n_drt}")
print(f" subgroup check: LF + RF = {DRT_SUB['LF'] + DRT_SUB['RF']} (Frontal 55); "
f"LP + RP = {DRT_SUB['LP'] + DRT_SUB['RP']} (Posterior 43)")
print(f"ART analysed (Table 1): 72 + 100 + 70 = {n_art}")
max_behav = N_PATIENTS_FP + N_HC
all_recruited = N_PATIENTS_RECRUITED + N_HC
print(f"Behavioural analysis set (176 F/P patients + 81 HC) = {max_behav}")
print(f"All recruited (247 patients + 81 HC) = {all_recruited}")
print("\n== Error df under the stated ANCOVA (N - 3 groups - 1 covariate) ==")
print(f"DRT: {n_drt} - 4 = {n_drt - 4} (Results reports 142)")
print(f"ART: {n_art} - 4 = {n_art - 4} (Abstract and Results report 238)")
n_implied_abs = ABSTRACT_DRT[1] + 4
print(f"N implied by abstract F(2,387): ANCOVA {n_implied_abs}; ANOVA without covariate {ABSTRACT_DRT[1] + 3}")
print(f" excess over DRT sample: {n_implied_abs - n_drt}; over all recruited: {n_implied_abs - all_recruited}")
print(f"N implied by Results F(2,142): {RESULTS_DRT[1] + 4}")
print("Within-subject pooling check: 146 participants x 2 question types = "
f"{2 * n_drt} observations -> error df {2 * n_drt - 4}")
print("\n== P values and partial eta-squared of the reported tuples ==")
for lab, (d1, d2, F) in [("Abstract DRT", ABSTRACT_DRT), ("Results DRT", RESULTS_DRT), ("ART", ART_REPORTED)]:
print(f"{lab:13s} F({d1},{d2}) = {F}: P = {p_of(F, d1, d2):.2e}, partial eta^2 = {eta_p2(F, d1, d2):.3f}")
print(f"Abstract DRT F / Results DRT F = {ABSTRACT_DRT[2] / RESULTS_DRT[2]:.2f}")
print(f"Abstract DRT F vs ART F: {ABSTRACT_DRT[2]} vs {ART_REPORTED[2]} (difference {ART_REPORTED[2] - ABSTRACT_DRT[2]:.3f})")
print("\n== Unadjusted one-way ANOVA from the tables' n, mean, SD (reference only) ==")
UNADJ = {}
for lab, ns, m, s, Frep, d2rep, loc in TESTS:
N, k, F, p = anova_from_summary(ns, m, s)
UNADJ[lab] = F
print(f"{lab:18s} N={N} unadjusted F(2,{N - k}) = {F:5.2f}, P = {p:.2e} | "
f"reported ANCOVA F(2,{d2rep}) = {Frep} ({loc}); ratio reported/unadjusted = {Frep / F:.2f}; "
f"expected error df N-4 = {N - 4}")
print(f"Abstract DRT F / unadjusted DRT F = {ABSTRACT_DRT[2] / UNADJ['DRT overall']:.2f}")
print("\n== Pooled group means from the hemisphere subgroups (Table 2) ==")
print(f"DRT Frontal: (24*75.88 + 31*68.42)/55 = {(24 * 75.88 + 31 * 68.42) / 55:.2f} (reported 71.67)")
print(f"DRT Posterior: (19*77.05 + 24*81.21)/43 = {(19 * 77.05 + 24 * 81.21) / 43:.2f} (reported 79.37)")
# Other error dfs reported in Results (implied N under the same model)
OTHER_DF = {"RAPM": 184, "S fluency": 151, "Stroop": 150}
print("\n== Other reported error dfs -> implied N (N = df + 4) ==")
for k_, v in OTHER_DF.items():
print(f"{k_}: df {v} -> N {v + 4} (<= {max_behav})")
# ---------------------------------------------------------------- figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": TXT,
"axes.edgecolor": SEC, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "none", "lines.linewidth": 2,
})
W = 7.2
def save(fig, path_noext):
os.makedirs(os.path.dirname(path_noext), exist_ok=True)
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def style_axis(ax):
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def panel(ax, x, y, w, h, title, lines, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.012,rounding_size=0.02",
fc="white", ec=edge, lw=1.6))
ax.text(x + 0.03, y + h - 0.06, title, fontsize=11.5, weight="bold", va="top")
yy = y + h - 0.17
for txt, col, wt in lines:
ax.text(x + 0.03, yy, txt, fontsize=10.5, color=col, weight=wt, va="top", family="DejaVu Sans")
yy -= 0.085
# --- Candidate A: two text panels joined by an annotated connector
def cand_A(path):
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
panel(ax, 0.03, 0.30, 0.40, 0.62, "Abstract", [
("“DRT: F(2,387) = 18.491;", ORANGE, "bold"),
("P < 0.001”", ORANGE, "bold"),
("", TXT, "normal"),
("error df 387 implies", SEC, "normal"),
(f"N = 387 + 4 = {n_implied_abs} participants", SEC, "normal"),
], ORANGE)
panel(ax, 0.57, 0.30, 0.40, 0.62, "Results / Table 2", [
("“F(2,142) = 7.73; P < 0.001”", TXT, "bold"),
("Table 2: n = 48 + 55 + 43", TXT, "normal"),
("", TXT, "normal"),
(f"recomputed: N = {n_drt}", BLUE, "bold"),
(f"error df = {n_drt} − 4 = {n_drt - 4} ✓", BLUE, "bold"),
], BLUE)
ax.annotate("", xy=(0.565, 0.61), xytext=(0.435, 0.61),
arrowprops=dict(arrowstyle="<->", color=TXT, lw=2))
ax.text(0.50, 0.66, "same test", ha="center", fontsize=10, color=SEC)
ax.text(0.50, 0.185, f"Conflict: the abstract's df needs {n_implied_abs} participants; the DRT sample is {n_drt},\n"
f"and only {all_recruited} people (247 patients + 81 controls) were recruited in total.",
ha="center", va="center", fontsize=10.5)
ax.text(0.03, 0.025, "Orange: printed in the Abstract. Black: printed in the Results and Table 2.\n"
"Blue: recomputed from the paper's group sizes. Gray: derived from the Abstract's df.",
fontsize=10, color=SEC, va="bottom")
save(fig, path)
# --- Candidate B: participants on one axis
def cand_B(path):
rows = [
("Implied by abstract F(2,387)", n_implied_abs, ORANGE, "o", "reported df + 4"),
("All recruited (247 + 81)", all_recruited, SEC, "s", "Methods"),
("Behavioural set (176 + 81)", max_behav, SEC, "s", "Methods"),
("Implied by Results F(2,142)", RESULTS_DRT[1] + 4, TXT, "o", "reported df + 4"),
("DRT groups, Table 2 (48+55+43)", n_drt, BLUE, "D", "recomputed sum"),
]
fig, ax = plt.subplots(figsize=(W, 3.6))
for i, (lab, v, c, mk, note) in enumerate(rows[::-1]):
ax.hlines(i, 0, v, color=GRID, lw=2)
ax.plot(v, i, mk, color=c, ms=10, mfc=c if mk != "o" or c == ORANGE else "white", mew=2)
ax.text(v + 8, i, f"{v}", va="center", fontsize=10.5, color=c, weight="bold")
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows[::-1]])
ax.axvline(all_recruited, color=SEC, lw=1, ls=":")
ax.text(all_recruited - 4, len(rows) - 0.45, "everyone recruited", ha="right", fontsize=10, color=SEC)
ax.set_xlim(0, 440); ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_xlabel("Participants")
style_axis(ax)
fig.text(0.01, 0.01, "Orange: from the abstract's df. Blue: recomputed from Table 2.\nGray: counts in Methods. Black open circle: from the Results df.",
fontsize=10, color=SEC)
fig.tight_layout(rect=(0, 0.11, 1, 1))
save(fig, path)
# --- Candidate C: calculation flow
def cand_C(path):
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 1); ax.set_ylim(0, 1); ax.axis("off")
def box(x, y, w, h, t, ec, col=TXT, wt="normal"):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0.008,rounding_size=0.015",
fc="white", ec=ec, lw=1.6))
ax.text(x + w / 2, y + h / 2, t, ha="center", va="center", fontsize=10.5, color=col, weight=wt)
def arrow(x0, x1, y, t=None):
ax.annotate("", xy=(x1, y), xytext=(x0, y), arrowprops=dict(arrowstyle="->", color=TXT, lw=2))
if t:
ax.text((x0 + x1) / 2, y + 0.11, t, ha="center", va="bottom", fontsize=10, color=SEC)
# columns: x, width
cols = [(0.02, 0.21), (0.30, 0.15), (0.53, 0.15), (0.76, 0.22)]
ya, yb, h = 0.60, 0.17, 0.19
ax.text(0.02, 0.92, "From the paper's group sizes (Table 2) to the Results", fontsize=11, weight="bold")
box(*[cols[0][0], ya, cols[0][1], h], "Table 2 groups\n48 + 55 + 43", SEC)
box(cols[1][0], ya, cols[1][1], h, f"N = {n_drt}", BLUE, BLUE, "bold")
box(cols[2][0], ya, cols[2][1], h, f"error df\n= {n_drt - 4}", BLUE, BLUE, "bold")
box(cols[3][0], ya, cols[3][1], h, "Results:\nF(2,142) = 7.73 ✓", TXT, TXT, "bold")
arrow(0.235, 0.295, ya + h / 2, "sum")
arrow(0.455, 0.525, ya + h / 2, "− 4")
arrow(0.685, 0.755, ya + h / 2)
ax.text(0.02, 0.49, "From the abstract's df back to a sample size", fontsize=11, weight="bold")
box(cols[3][0], yb, cols[3][1], h, "Abstract:\nF(2,387) = 18.491", ORANGE, ORANGE, "bold")
box(cols[2][0], yb, cols[2][1], h, "error df\n= 387", ORANGE, ORANGE, "bold")
box(cols[1][0], yb, cols[1][1], h, f"N = {n_implied_abs}", ORANGE, ORANGE, "bold")
box(cols[0][0], yb, cols[0][1], h, f"more than all\n{all_recruited} recruited ✗", ORANGE, TXT)
arrow(0.755, 0.685, yb + h / 2)
arrow(0.525, 0.455, yb + h / 2, "+ 4")
arrow(0.295, 0.235, yb + h / 2)
ax.text(0.02, 0.04, "− 4 = three groups and one age covariate (ANCOVA, Methods). Blue: recomputed from\n"
"the paper's group sizes. Orange: printed in the abstract or derived from it.",
fontsize=10, color=SEC, va="bottom")
save(fig, path)
# --- Candidate D: F statistics, reported vs unadjusted reference
def cand_D(path):
rows = [("DRT overall (Abstract)", ABSTRACT_DRT[2], UNADJ["DRT overall"], True)]
for lab, ns, m, s, Frep, d2, loc in TESTS:
rows.append((f"{lab} ({loc.replace('; ', ', ')})", Frep, UNADJ[lab], False))
rows = rows[::-1]
fig, ax = plt.subplots(figsize=(W, 4.3))
for i, (lab, Frep, Fu, hl) in enumerate(rows):
ax.hlines(i, min(Frep, Fu), max(Frep, Fu), color=GRID, lw=2)
ax.plot(Fu, i, "D", color=BLUE, ms=8)
ax.plot(Frep, i, "s" if hl else "o", color=ORANGE, ms=9)
if hl:
ax.text((Frep + Fu) / 2, i + 0.17, f"{Frep} = {Frep / Fu:.1f} × the unadjusted F of {Fu:.2f}",
ha="center", va="bottom", fontsize=10, color=ORANGE, weight="bold")
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows])
for t in ax.get_yticklabels():
if t.get_text().startswith("DRT overall (Abstract"):
t.set_weight("bold")
ax.set_xlim(0, 25); ax.set_ylim(-0.5, len(rows) - 0.15)
ax.set_xlabel("F statistic for the group effect")
style_axis(ax)
ax.plot([], [], "s", color=ORANGE, ms=9, label="reported F (Abstract; no stated analysis gives df 387)")
ax.plot([], [], "o", color=ORANGE, ms=9, label="reported F (Results, ANCOVA with age)")
ax.plot([], [], "D", color=BLUE, ms=8, label="unadjusted F recomputed from table n, mean, SD")
ax.legend(loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=1, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
# --- Candidate E: error df audit, reported vs N - 4
def cand_E(path):
rows = [
("DRT overall (Abstract)", 387, n_drt - 4),
("DRT overall (Results)", 142, n_drt - 4),
("DRT indeterminate (Results)", 142, n_drt - 4),
("DRT determinate (Results)", 142, n_drt - 4),
("ART overall (Abstract, Results)", 238, n_art - 4),
("ART, 7 further tests (Results)", 238, n_art - 4),
][::-1]
fig, ax = plt.subplots(figsize=(W, 3.5))
for i, (lab, rep, exp) in enumerate(rows):
ax.hlines(i, min(rep, exp), max(rep, exp), color=GRID, lw=2)
ax.plot(exp, i, "D", color=BLUE, ms=11)
ax.plot(rep, i, "o", color=ORANGE, ms=8)
if rep != exp:
ax.text(exp + 12, i - 0.3, f"387 vs {exp}", ha="left", va="center",
fontsize=10, color=TXT, weight="bold")
lim = all_recruited - 4
ax.axvline(lim, color=SEC, lw=1, ls=":")
ax.text(lim - 9, 2.5, f"largest error df for a\nthree-group ANCOVA on all\n{all_recruited} recruited: {lim}",
ha="right", va="center", fontsize=10, color=SEC)
ax.set_yticks(range(len(rows))); ax.set_yticklabels([r[0] for r in rows])
ax.set_xlim(0, 420); ax.set_ylim(-0.5, len(rows) - 0.5)
ax.set_xlabel("Error degrees of freedom")
style_axis(ax)
ax.plot([], [], "o", color=ORANGE, ms=8, label="reported error df")
ax.plot([], [], "D", color=BLUE, ms=11, label="N − 4 recomputed from the table's group sizes")
ax.legend(loc="upper center", bbox_to_anchor=(0.35, -0.22), ncol=2, frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
cand_A(os.path.join(CAND, "cand_A_text_panels"))
cand_B(os.path.join(CAND, "cand_B_participants_axis"))
cand_C(os.path.join(CAND, "cand_C_calculation_flow"))
cand_D(os.path.join(CAND, "cand_D_F_statistics"))
cand_E(os.path.join(CAND, "cand_E_error_df_audit"))
# Provisional choice for the write-up until a design is picked: A as fig1, E as fig2.
cand_A(os.path.join(HERE, "fig1"))
cand_E(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)
Results text swaps the two condition means for tic frequency, reversing the reported reduction
Curr Biol (2020) · doi:10.1016/j.cub.2020.04.044 · PMC7308804
Summary
In study 3 (16 people with Tourette syndrome), the Results say that rhythmic median nerve stimulation (MNS) "significantly reduced" tic frequency. The same sentence then gives mean tic counts of 87.6 ± 71.4 for no stimulation and 126.3 ± 94.5 for rhythmic MNS, which as printed is a 44.2% increase. The processed data the paper deposits on OSF give exactly these two values with the conditions the other way round: 87.625 ± 71.41 under MNS and 126.25 ± 94.49 without stimulation, a 30.6% reduction. The direction stated in the abstract, highlights, Figure 5 and Discussion is therefore supported, and the error is a label swap in one sentence. Recomputing the tests from the same deposited data turns up a separate problem: the printed t values (2.36, 2.41, 1.83) cannot be reproduced (1.99, 2.05, 1.41). The tic-frequency and tic-intensity effects reach p < 0.05 only under the one-tailed tests the Methods specify.
What the paper reports
Results, "Effect of Rhythmic MNS on the Occurrence of Tics and the Urge-to-Tic", second paragraph:
Quantitative analysis revealed that compared with comparable time periods of no stimulation, rhythmic MNS significantly reduced both tic frequency (i.e., total number of tics recorded over a total period of 4 min) and tic intensity (i.e., the independently, blind, rated intensity of each tic) (means for tic frequency: no stimulation = 87.6 ± 71.4, rhythmic MNS = 126.3 ± 94.5; t(15) = 2.36, p = 0.03, Figure 5 left; means for tic intensity: no stimulation = 3.0 ± 0.6, rhythmic MNS = 2.8 ± 0.6; t(15) = 2.41, p = 0.03, Figure 5 middle).
The same paragraph, next sentence:
quantitative analysis of the continuous self-estimated urge-to-tic data showed an overall reduction in the self-estimated urge-to-tic during MNS of ∼33%, the difference between rhythmic MNS and no stimulation was only marginally significant (means for self-estimated tic-to-tic intensity: no stimulation = 41.3 ± 31.7, rhythmic MNS = 29.1 ± 20.7; t(15) = 1.83, p = 0.09, Figure 5 right).
The same paragraph, on the correlation:
These analyses revealed a positive correlation (r = 0.75, R2 = 0.56, F = 12.74, p = 0.005), indicating that a reduction in tic frequency was associated with a reduction in self-estimated urge-to-tic experiences
Abstract:
it nonetheless leads to a large reduction in tic frequency and tic intensity in individuals with TS.
Highlights:
Mu-band entrainment significantly reduces motor and vocal tics in Tourette syndrome
Discussion, fourth paragraph:
This quantitative analysis clearly demonstrated that both tic frequency and tic intensity were significantly reduced by rhythmic 10 Hz MNS compared with no stimulation
STAR Methods, "Quantification and Statistical Analysis":
In all four studies, statistical analyses were performed using one-tailed paired-samples t tests.
STAR Methods, "Data and Code Availability":
Processed data of tic frequencies, tic intensities, and urge intensities are available on OSF (https://osf.io/zw93x/).
What the paper's own numbers show
Method. First I took the printed means, SDs and t values at face value and worked out the direction of change, the relative change, and the p values that t(15) implies. Then I opened the processed data the paper deposits on OSF (Study3_data.xlsx, dated 2020-04-17). It has one row per participant (n = 16) and three outcome columns, TICS, INTENSITY and SLIDER, each split into "Rhythmic" and "No Stimulation". From it I recomputed the condition means and SDs, the paired t tests (one- and two-tailed), a Wilcoxon signed-rank test, and the correlation reported in the same paragraph. The means and SDs I recomputed equal the AVERAGE and SD rows stored in the spreadsheet itself. recompute.py.txt holds the outcome values only, without participant codes or demographic columns, and prints every number below.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| Tic frequency, no stimulation | Results, study 3, para 2 | 87.6 ± 71.4 | 126.25 ± 94.49 (OSF data, n = 16) |
| Tic frequency, rhythmic MNS | same | 126.3 ± 94.5 | 87.625 ± 71.41 (OSF data) |
| Change, MNS minus no stimulation | same ("significantly reduced") | +38.7 tics as printed, a 44.2% increase | −38.6 tics, a 30.6% reduction (OSF data; also (126.3 − 87.6)/126.3 = 30.6% with the printed labels exchanged) |
| Medians, no stimulation vs MNS | OSF "median" row | — | 105 vs 67.5; fewer tics under MNS in 12 of 16 participants |
| Tic intensity, no stim vs MNS | Results, study 3, para 2 | 3.0 ± 0.6 vs 2.8 ± 0.6 | 2.97 ± 0.56 vs 2.76 ± 0.56 (consistent; labels correct) |
| Urge-to-tic, no stim vs MNS | Results, study 3, para 2 | 41.3 ± 31.7 vs 29.1 ± 20.7, df 15 | 43.06 ± 31.67 vs 29.10 ± 20.73, n = 13 (3 participants have no slider data); reduction 32.4% (text: "∼33%") |
| p from the printed t, df 15 | Results, study 3, para 2 | p = 0.03, 0.03, 0.09 | two-tailed 0.032, 0.029, 0.087; one-tailed 0.016, 0.015, 0.044 |
| Paired t, tic frequency | same | t(15) = 2.36 | t(15) = 1.99; one-tailed p = 0.032, two-tailed p = 0.065 |
| Paired t, tic intensity | same | t(15) = 2.41 | t(15) = 2.05; one-tailed p = 0.029, two-tailed p = 0.058 |
| Paired t, urge-to-tic | same | t(15) = 1.83 | t(12) = 1.41; one-tailed p = 0.092, two-tailed p = 0.184 |
| Wilcoxon signed-rank, tic frequency | not reported | — | one-sided p = 0.022, two-sided p = 0.044 |
| SD of paired differences, tic frequency | implied by printed t | 65.6 (= 38.7 × √16 / 2.36) | 77.5 |
| Correlation of reductions, tics vs urge | Results, study 3, para 2 | r = 0.75, R² = 0.56, F = 12.74, p = 0.005 | r = 0.748, R² = 0.560, F(1,10) = 12.74, p = 0.0051 (proportional reductions; n = 12, as one no-stimulation urge value is 0) |
Three points follow from the table.
- The tic-frequency means are attached to the wrong conditions. The printed pairs 87.6 ± 71.4 and 126.3 ± 94.5 match the deposited "Rhythmic" (87.625 ± 71.41) and "No Stimulation" (126.25 ± 94.49) values to every printed digit, but with the labels exchanged. Pairing cannot reverse the sign of a mean difference, so the printed values describe an increase and cannot support "significantly reduced". The tic-intensity and urge values carry the correct labels.
- The deposited file is the dataset the paper analysed. It reproduces the printed correlation exactly (r = 0.75, R² = 0.56, F = 12.74, p = 0.005), the "∼33%" urge reduction (32.4%), and the printed SDs of all six condition columns.
- The printed t values do not follow from the deposited data. Paired t tests on the deposited data give 1.99, 2.05 and 1.41, against the printed 2.36, 2.41 and 1.83. The printed p values (0.03, 0.03, 0.09) match the one-tailed p values of the recomputed tests (0.032, 0.029, 0.092), which is what the Methods specify. The same printed p values also happen to match the two-tailed p values of the printed t (0.032, 0.029, 0.087). As a numerical observation, and not as an account of how they arose, each printed t equals, to two decimals, the t whose two-tailed p equals the one-tailed p of the deposited data (2.36 and 2.41 at df 15, 1.83 at df 12). For the urge measure the printed degrees of freedom (15) also differ from the 12 that the 13 complete pairs allow, and the printed no-stimulation mean (41.3) differs from the deposited 43.06, although the SD (31.7) matches.
Figure
Figure 1. Mean tic count per participant (total over the 4 min of video analysed), drawn as an arrow from the no-stimulation condition (open circle) to the rhythmic MNS condition (filled square). Top row (orange): the values printed in the Results text, which put MNS 38.7 tics, or 44.2%, above no stimulation. Bottom row (blue): the same quantities recomputed from the processed data deposited on OSF (n = 16), which give the same two numbers with the conditions exchanged, so MNS is 38.6 tics, or 30.6%, below no stimulation. Labels show mean ± SD. The axis starts at zero.
Figure 2. Paired t statistics for the three study 3 outcomes: as printed in the Results (orange), and recomputed from the deposited data with a paired t test (blue). The dashed and dotted lines mark the one-tailed and two-tailed p = 0.05 critical values at df 15. The urge-to-tic data have 13 complete pairs (df 12), with critical values 1.78 and 2.18. Recomputed, the tic-frequency and tic-intensity effects clear only the one-tailed threshold, which is the test the Methods specify. The axis starts at zero, the null value.
Why it matters
- The headline efficacy number is printed backwards. Tic frequency is the study's main clinical outcome, and it is the basis for the abstract's "large reduction in tic frequency". A reader of the Results paragraph, or anyone extracting the means for a meta-analysis or a power calculation, gets a 44% increase under MNS instead of a 31% reduction. A pooled estimate built from the printed numbers would carry the wrong sign.
- The conclusion itself stands. The deposited data, the Figure 5 legend ("Shown on the top left are data for tic frequency as measured by the total number of tics recorded in each condition (p < 0.05)") and every summary statement agree on a reduction. Correcting the labels restores agreement within the paper.
- The strength of the evidence is weaker than the printed t values suggest. Recomputed from the deposited data, the tic-frequency and tic-intensity effects are significant only one-tailed (p = 0.032 and 0.029). Two-tailed they are not (0.065 and 0.058). A nonparametric test on tic frequency gives a two-sided p of 0.044. The one-tailed choice is stated in the Methods, so the printed p values are consistent with the stated analysis, but the printed t values are 1.18, 1.18 and 1.30 times the recomputed ones. This is separate from the label swap and does not change any stated conclusion. Readers weighing the result may still want to know about it.
Explanations considered
- A different measure, in which more counts mean fewer tics. Not supported. The sentence defines tic frequency as the "total number of tics recorded over a total period of 4 min". A larger count is more tics.
- A different subset or time window. Not supported. The Methods restrict the video analysis to the last 40 s of each 60 s epoch and say this "did not change the effects observed". Both the text and the deposited file have n = 16, and the deposited values match the printed means and SDs exactly except for the labels.
- The deposited data, not the text, carry the wrong labels. Not supported. With the deposited labels, the direction agrees with the paper's own Figure 5 tic-frequency panel (the no-stimulation column lies higher), with its legend, abstract, highlights, Discussion and Conclusion, and with the correctly labelled intensity and urge columns. The deposited file also reproduces the printed correlation exactly.
- A text-conversion artifact. Ruled out. The values are plain decimals in running text, and the Europe PMC full-text XML and the PMC HTML carry the same sentence. No superscript, sign or table structure is involved that a conversion could scramble.
- Pairing or a signed t statistic. Cannot explain it. In a paired design the mean of the differences equals the difference of the means, so no pairing structure turns 87.6 → 126.3 into a reduction. The printed t is unsigned.
- For the t-value discrepancy: a one-tailed test, or a different sidedness. Sidedness cannot explain it, because it changes p and not t. The printed p values agree with one-tailed tests on the deposited data, but the printed t values do not match the paired tests on the deposited data. Unpaired Student and Welch tests give even smaller values (1.30, 1.03 and 1.33), so a different test type does not reproduce them either. I do not attempt to explain how the printed t values arose.
- A correction already issued. None found (see Status).
Status
- Version of record. The publisher's pages (doi.org → linkinghub, ScienceDirect, cell.com) returned only redirects or HTTP 403 to automated requests. The publisher-supplied full text in the Europe PMC full-text XML for PMC7308804 and in the live PMC HTML gives the sentence exactly as quoted. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. - Supplements and data. The OSF project cited in the paper (osf.io/zw93x) lists Study1_script.m, Study2_data.xlsx, Study2_scripts.zip, Study3_data.xlsx, Study4_data.zip and Study4_scripts.zip.
Study3_data.xlsxholds the study 3 tic data used above, and the other files concern studies 1, 2 and 4. I did not examine the in-article supplements (Videos S1–S3, Table S1, Figure S2). They are described as video clips and a regression on symptom severity, and none of them reports the condition means. A copy of the accepted manuscript in an institutional repository returned HTTP 403 and was not checked. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. Crossref's only relation is a has-review link to an F1000 recommendation. PubMed (PMID 32502412) lists only a "Comment in" entry, a commentary in Curr Biol 2020;30(12):R698–R700 that is not a correction. The DOI does not appear in the Retraction Watch dataset. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC7308804; Europe PMC full-text XML; OSF project zw93x (
Study3_data.xlsx); Crossref; PubMed; Retraction Watch dataset; PubPeer; https://doi.org/10.1016/j.cub.2020.04.044.
Suggested correction
In the Results, section "Effect of Rhythmic MNS on the Occurrence of Tics and the Urge-to-Tic", replace
means for tic frequency: no stimulation = 87.6 ± 71.4, rhythmic MNS = 126.3 ± 94.5
with
means for tic frequency: no stimulation = 126.3 ± 94.5, rhythmic MNS = 87.6 ± 71.4
The authors may also wish to check the three t statistics in the same paragraph against the deposited data. Paired tests on Study3_data.xlsx give t(15) = 1.99 for tic frequency, t(15) = 2.05 for tic intensity and t(12) = 1.41 for the urge-to-tic (13 complete pairs), with one-tailed p = 0.032, 0.029 and 0.092. They may also wish to check the urge-to-tic no-stimulation mean (printed 41.3; deposited 43.06). Stating "one-tailed" next to each p value in the Results would help readers.
Recomputation script (python recompute.py.txt)
"""Recompute the study 3 tic statistics of PMC7308804
(Curr Biol 2020, doi:10.1016/j.cub.2020.04.044,
"Entraining Movement-Related Brain Oscillations to Suppress Tics in Tourette Syndrome").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Inputs
1. Printed values, Results, "Effect of Rhythmic MNS on the Occurrence of Tics and the
Urge-to-Tic", paragraph 2 (PMC text):
tic frequency: no stimulation = 87.6 ± 71.4, rhythmic MNS = 126.3 ± 94.5; t(15) = 2.36, p = 0.03
tic intensity: no stimulation = 3.0 ± 0.6, rhythmic MNS = 2.8 ± 0.6; t(15) = 2.41, p = 0.03
urge-to-tic: no stimulation = 41.3 ± 31.7, rhythmic MNS = 29.1 ± 20.7; t(15) = 1.83, p = 0.09
correlation: r = 0.75, R2 = 0.56, F = 12.74, p = 0.005
STAR Methods: "one-tailed paired-samples t tests".
2. The processed data the paper deposits on OSF (https://osf.io/zw93x/, file Study3_data.xlsx,
dated 2020-04-17): the three outcome columns (TICS, INTENSITY, SLIDER), each given for
"Rhythmic" and "No Stimulation", for the 16 participants. Only these outcome values are copied
here, in the file's row order; participant codes and all demographic and clinical columns are
left out. NaN marks the three participants with no slider data.
"""
import os
from decimal import Decimal, ROUND_HALF_UP
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
nan = np.nan
# ------------------------------------------------------------------ inputs: printed values
N_PRINTED, DF_PRINTED = 16, 15
PRINTED = {
# outcome: (no-stim mean, SD, MNS mean, SD, t, p)
"tic frequency": (87.6, 71.4, 126.3, 94.5, 2.36, 0.03),
"tic intensity": (3.0, 0.6, 2.8, 0.6, 2.41, 0.03),
"urge-to-tic": (41.3, 31.7, 29.1, 20.7, 1.83, 0.09),
}
PRINTED_CORR = dict(r=0.75, R2=0.56, F=12.74, p=0.005)
PRINTED_URGE_REDUCTION = 33 # "~33%"
# ------------------------------------------------------------------ inputs: OSF Study3_data.xlsx
TICS_MNS = np.array([60, 54, 38, 31, 75, 118, 113, 71, 108, 17, 276, 104, 37, 16, 64, 220], float)
TICS_NOS = np.array([63, 331, 100, 67, 130, 185, 62, 31, 235, 36, 279, 132, 25, 36, 110, 198], float)
INT_MNS = np.array([2.9930555555, 2.907638889, 1.76470588233333, 2.75833333333333, 3.001986902,
3.542010437, 3.196882508, 2.574810606, 2.44052429033333, 2.52,
2.99287878783333, 3.8375942685, 2.544973545, 2.9444444445,
2.52708333333333, 1.6732457295])
INT_NOS = np.array([2.92045206966667, 3.35852982116667, 2.6316964285, 3.321937322,
3.06606256816667, 3.44125274483333, 3.63888888883333, 2.32638888875,
2.924112924, 3.5646825398, 2.80345276016667, 3.93328266916667,
2.3604497355, 2.95833333325, 2.34821428566667, 1.87769106816667])
URGE_MNS = np.array([5.36370711557095, 9.27817841133405, 15.9897185752179, 13.0797002402398,
11.403426768897, 21.8305113274183, nan, 52.1664050033929, 31.2108605273619,
35.8041821614369, nan, 43.3763878464795, 72.5291528683101, nan,
51.8761550779321, 14.4005025754646])
URGE_NOS = np.array([5.35918941689058, 95.8511226704295, 22.2928708035175, 62.2512884006415,
21.7665095556081, 0.0, nan, 32.4668400628702, 77.2752151575435,
57.9591716488115, nan, 90.3065169281539, 27.2497823278225, nan,
50.1136011739528, 16.8818022855321])
# summary rows printed inside the xlsx itself (AVERAGE / SD), for a cross-check
XLSX_ROWS = {"tic frequency": (87.625, 71.41230052775688, 126.25, 94.49338601193207),
"tic intensity": (2.7637605320104157, 0.5569204767883725,
2.9672142529770844, 0.5625773165696307),
"urge-to-tic": (29.100683730696613, 20.730524438404533,
43.0595315716749, 31.67219898264484)}
DATA = {"tic frequency": (TICS_NOS, TICS_MNS),
"tic intensity": (INT_NOS, INT_MNS),
"urge-to-tic": (URGE_NOS, URGE_MNS)}
def tp(t, df):
return stats.t.sf(abs(t), df), 2 * stats.t.sf(abs(t), df)
# ------------------------------------------------------------------ 1. printed numbers, as printed
print("=== 1. Printed values taken at face value (n = 16, df = 15) ===")
for k, (m0, s0, m1, s1, t, p) in PRINTED.items():
d = m1 - m0
sdd = abs(d) * np.sqrt(N_PRINTED) / t
r = (s0**2 + s1**2 - sdd**2) / (2 * s0 * s1)
p1, p2 = tp(t, DF_PRINTED)
print(f"{k:14s} no stim {m0:6.1f} MNS {m1:6.1f} MNS - no stim = {d:+6.2f} "
f"({100*d/m0:+.1f}% of no stim)")
print(f"{'':14s} printed t = {t}: one-tailed p = {p1:.4f}, two-tailed p = {p2:.4f} "
f"(printed p = {p}); implied SD of paired differences = {sdd:.2f}, implied r = {r:.2f}")
m0, _, m1, _, _, _ = PRINTED["tic frequency"]
print(f"Tic frequency with the two printed means exchanged: (126.3 - 87.6)/126.3 = "
f"{100*(m1-m0)/m1:.1f}% reduction")
print(f"Urge, from printed means: (41.3 - 29.1)/41.3 = {100*(41.3-29.1)/41.3:.1f}% "
f"(text: ~{PRINTED_URGE_REDUCTION}%)")
# ------------------------------------------------------------------ 2. deposited data
print("\n=== 2. Deposited data (OSF Study3_data.xlsx) ===")
REC = {}
for k, (nos, mns) in DATA.items():
ok = ~np.isnan(nos) & ~np.isnan(mns)
a, b = nos[ok], mns[ok]
n = ok.sum()
res = stats.ttest_rel(a, b)
t = res.statistic
p1, p2 = tp(t, n - 1)
w2 = stats.wilcoxon(a, b).pvalue
w1 = stats.wilcoxon(a, b, alternative="greater").pvalue
REC[k] = dict(n=n, m0=a.mean(), s0=a.std(ddof=1), m1=b.mean(), s1=b.std(ddof=1),
t=t, p1=p1, p2=p2, sdd=(a - b).std(ddof=1), r=np.corrcoef(a, b)[0, 1],
fewer=int((b < a).sum()), w1=w1, w2=w2,
med0=np.median(a), med1=np.median(b))
R = REC[k]
x = XLSX_ROWS[k]
assert np.allclose([R["m1"], R["s1"], R["m0"], R["s0"]], x), k
print(f"{k:14s} n = {n}: no stim {R['m0']:.3f} ± {R['s0']:.3f} (median {R['med0']:.2f}); "
f"MNS {R['m1']:.3f} ± {R['s1']:.3f} (median {R['med1']:.2f})")
print(f"{'':14s} MNS - no stim = {R['m1']-R['m0']:+.3f} "
f"({100*(R['m1']-R['m0'])/R['m0']:+.1f}% of no stim); "
f"fewer under MNS in {R['fewer']} of {n}")
print(f"{'':14s} paired t({n-1}) = {t:.2f}; one-tailed p = {p1:.4f}, two-tailed p = {p2:.4f}; "
f"SD of differences = {R['sdd']:.2f}; r between conditions = {R['r']:.2f}")
print(f"{'':14s} Wilcoxon signed-rank: one-sided p = {w1:.4f}, two-sided p = {w2:.4f}")
ti = stats.ttest_ind(a, b).statistic
tw = stats.ttest_ind(a, b, equal_var=False).statistic
print(f"{'':14s} unpaired t (Student) = {ti:.2f}, Welch t = {tw:.2f}; "
f"printed t / paired t = {PRINTED[k][4]/t:.3f}")
# t value whose two-tailed p equals the one-tailed p of the deposited data
print(f"{'':14s} t with two-tailed p = {p1:.4f}: df {n-1}: {stats.t.isf(p1/2, n-1):.2f}; "
f"df 15: {stats.t.isf(p1/2, 15):.2f} (printed t = {PRINTED[k][4]})")
print("(the xlsx AVERAGE and SD rows equal these means and SDs: assertion passed)")
for df in (15, 12):
print(f"critical t, df {df}: one-tailed 0.05 = {stats.t.isf(0.05, df):.3f}, "
f"two-tailed 0.05 = {stats.t.isf(0.025, df):.3f}")
# ------------------------------------------------------------------ 3. correlation
print("\n=== 3. Correlation of MNS-induced reductions (tic frequency vs urge) ===")
ok = ~np.isnan(URGE_NOS) & ~np.isnan(URGE_MNS)
dt_raw = (TICS_NOS - TICS_MNS)[ok]
du_raw = (URGE_NOS - URGE_MNS)[ok]
r_raw, p_raw = stats.pearsonr(dt_raw, du_raw)
print(f"raw differences, n = {ok.sum()}: r = {r_raw:.3f}, p = {p_raw:.4f}")
with np.errstate(divide="ignore", invalid="ignore"):
dt_rel = ((TICS_NOS - TICS_MNS) / TICS_NOS)[ok]
du_rel = ((URGE_NOS - URGE_MNS) / URGE_NOS)[ok]
fin = np.isfinite(du_rel)
nr = fin.sum()
r_rel, p_rel = stats.pearsonr(dt_rel[fin], du_rel[fin])
F_rel = r_rel**2 / (1 - r_rel**2) * (nr - 2)
print(f"proportional reductions (one no-stim urge value is 0, so n = {nr}): r = {r_rel:.3f}, "
f"R2 = {r_rel**2:.3f}, F(1,{nr-2}) = {F_rel:.2f}, p = {p_rel:.4f} (printed {PRINTED_CORR})")
# ------------------------------------------------------------------ figures
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": TXT,
"axes.labelcolor": TXT, "xtick.color": SEC, "ytick.color": SEC,
"axes.edgecolor": SEC, "svg.fonttype": "none"})
def save(fig, name):
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# ---- Figure 1: tic-frequency means, printed vs deposited, as arrows from no stim to MNS
fig, ax = plt.subplots(figsize=(7.2, 3.3))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
P = PRINTED["tic frequency"]
D = REC["tic frequency"]
rows = [
(1.0, "Results text, as printed", ORA, (P[0], P[1]), (P[2], P[3])),
(0.0, "Deposited data (OSF), n = 16", BLU, (D["m0"], D["s0"]), (D["m1"], D["s1"])),
]
def r1(v): # round half up to one decimal, as the paper prints
return f"{Decimal(repr(float(v))).quantize(Decimal('0.1'), rounding=ROUND_HALF_UP)}"
for y, lab, col, (a, sa), (b, sb) in rows:
ax.annotate("", xy=(b, y), xytext=(a, y),
arrowprops=dict(arrowstyle="-|>", color=col, lw=2, mutation_scale=16,
shrinkA=6, shrinkB=7))
ax.plot(a, y, "o", ms=10, mfc=BG, mec=col, mew=2, zorder=3)
ax.plot(b, y, "s", ms=10, mfc=col, mec=col, zorder=3)
for x, sd, name in ((a, sa, "No stimulation"), (b, sb, "Rhythmic MNS")):
left = x == min(a, b)
ax.text(x + (-5 if left else 5), y, f"{name}\n{r1(x)} ± {r1(sd)}",
ha="right" if left else "left", va="center", color=TXT, fontsize=10)
pct = 100 * (b - a) / a
word = "increase" if b > a else "reduction"
ax.text((a + b) / 2, y - 0.2, f"{b-a:+.1f} tics ({pct:+.1f}%, {word})".replace("-", "\u2212"),
ha="center", va="top", color=col, fontsize=10.5, fontweight="bold")
ax.set_yticks([1.0, 0.0])
ax.set_yticklabels([r[1] for r in rows], fontsize=10.5)
for lbl, r in zip(ax.get_yticklabels(), rows):
lbl.set_color(r[2])
ax.set_ylim(-0.5, 1.35)
ax.set_xlim(0, 180)
ax.set_xticks(range(0, 181, 20))
ax.set_xlabel("Mean tics per participant (total over the 4 min analysed)", color=TXT)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.tight_layout()
save(fig, "fig1")
# ---- Figure 2: paired t statistics, printed vs recomputed from the deposited data
fig, ax = plt.subplots(figsize=(7.2, 3.8))
fig.patch.set_facecolor(BG)
ax.set_facecolor(BG)
names = ["tic frequency", "tic intensity", "urge-to-tic"]
ys = [2, 1, 0]
c1, c2 = stats.t.isf(0.05, 15), stats.t.isf(0.025, 15)
ax.axvline(0, color=SEC, lw=0.8)
ax.axvline(c1, color=SEC, lw=1, ls=(0, (4, 3)))
ax.axvline(c2, color=SEC, lw=1, ls=(0, (1, 2)))
ax.text(c1 - 0.03, 3.15, f"one-tailed p = 0.05\n(t = {c1:.2f}, df 15)", ha="right", va="top",
fontsize=10, color=SEC)
ax.text(c2 + 0.03, 3.15, f"two-tailed p = 0.05\n(t = {c2:.2f}, df 15)", ha="left", va="top",
fontsize=10, color=SEC)
for y, k in zip(ys, names):
tr, tc = PRINTED[k][4], REC[k]["t"]
ax.plot([tc, tr], [y, y], color=GRID, lw=2, zorder=1)
ax.plot(tr, y, "o", ms=9, color=ORA, zorder=3, label="Printed in Results" if y == 2 else None)
ax.plot(tc, y, "o", ms=9, color=BLU, zorder=3,
label="Recomputed from deposited data" if y == 2 else None)
ax.text(tr, y + 0.16, f"{tr:.2f}", color=ORA, fontsize=10, va="bottom", ha="center")
ax.text(tc, y - 0.16, f"{tc:.2f}", color=BLU, fontsize=10, va="top", ha="center")
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=2, frameon=False, fontsize=10,
handletextpad=0.4)
ax.set_yticks(ys)
ax.set_yticklabels(["tic frequency", "tic intensity", "urge-to-tic\n(deposited n = 13)"],
fontsize=10.5)
ax.set_ylim(-0.6, 3.2)
ax.set_xlim(0, 3.0)
ax.set_xlabel("Paired t statistic, no stimulation minus rhythmic MNS", color=TXT)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
fig.tight_layout()
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png")
Abstract calls a MELD–referral odds ratio of 1.13 "lower referral odds"
JAMA Netw Open (2026) · doi:10.1001/jamanetworkopen.2026.2567 · PMC13003376
Summary
The study follows 325 patients hospitalized with severe alcohol-associated hepatitis at 5 US transplant centers and models referral for liver transplant evaluation. The abstract states that "higher MELD scores were associated with lower referral odds" and gives, in the same sentence, an odds ratio of 1.13 (95% CI 1.07–1.18; P < .001). An odds ratio above 1, with a confidence interval wholly above 1, means that higher MELD scores go with higher odds of referral. The Results text, Table 2, Table 1 (mean MELD 32.3 in referred vs 27.4 in non-referred patients) and Supplement 1 eTable 3 all show that direction. All the numbers agree with each other. The error is a single direction word in the abstract, which reverses the stated direction of the association for one of the study's primary outcomes.
What the paper reports
Abstract, Results:
In multivariable analysis, higher MELD scores were associated with lower referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001).
Results, "Factors Associated With Referral for Transplant Evaluation", paragraph 1:
In multivariable logistic regression, adjusting for age, sex, race, ethnicity, educational level, insurance, marital status, living with children, MELD score, ADI, and site of care, a higher MELD score was associated with increased odds of referral (odds ratio [OR], 1.13 [95% CI, 1.07-1.18]; P < .001) (Table 2).
Table 2 (Multivariable Logistic Regression Analyses at Each Step in the Liver Transplant Cascade), referral column (n = 299):
MELD score (≤40)c | 1.13 (1.07-1.18) | <.001
Table 2, footnote c (the same definition appears in Table 1 and in the Methods):
Scores range from 6 to 40, with higher scores indicating severe liver disease.
Results, "Patient Characteristics", paragraph 3, and Table 1:
Compared with those who were not referred, patients who were referred for LT evaluation had significantly higher mean (SD) MELD scores (32.3 [7.7] vs 27.4 [6.9]; P < .001)
Discussion, paragraph 2:
As expected, the MELD score was associated with the transplant cascade.
Supplement 1, eTable 3 (referral model with an insurance-by-site interaction), row "Meld_cap": estimate 0.1432, standard error 0.0284, Wald χ² 25.4024, P < .0001.
All quotes from the article were checked verbatim against the PMC-archived text with src/show.py PMC13003376 --grep. The eTable 3 values were read from the supplement PDF.
What the paper's own numbers show
Method. In a logistic model, OR = exp(β). An OR above 1 means a positive coefficient: the odds of the outcome (here, referral) rise as the covariate rises. I converted the printed OR and CI bounds to the log scale and took the standard error from the CI width to get a z statistic. I compared the abstract's numbers with the Results text and Table 2. I computed the OR that a "lower odds" statement would need, which is the reciprocal OR per one-point lower MELD. I converted the eTable 3 coefficient to an OR with a Wald CI. As a descriptive check, I compared the Table 1 MELD means by referral status with a Welch t test, and approximated the crude logistic slope with the normal-discriminant formula (difference in means divided by pooled variance). That approximation is unadjusted and assumes normal MELD distributions with equal variances. It is used only to check the sign. All calculations are in recompute.py.txt.
| Quantity | Location | Reported | Recomputed |
|---|---|---|---|
| MELD → referral, adjusted OR | Abstract, Results | 1.13 (1.07–1.18), P < .001, described as "lower referral odds" | ln OR = +0.122; ln CI bounds +0.068 and +0.166, both > 0; z = 4.90, p ≈ 1.0 × 10⁻⁶: higher odds |
| MELD → referral, adjusted OR | Results, paragraph 1 | 1.13 (1.07–1.18), P < .001, "increased odds" | Same numbers as the abstract; direction word consistent with them |
| MELD → referral, adjusted OR | Table 2, referral column | 1.13 (1.07–1.18), P < .001 | Geometric CI midpoint 1.124, consistent with 1.13 |
| Size of the effect | derived | — | +13% odds per MELD point (CI +7% to +18%); 1.13¹⁰ = 3.39 (1.97–5.23) per 10 points |
| OR that "lower odds" would require | derived | not reported anywhere | 1/1.13 = 0.885 (0.847–0.935), per 1-point lower MELD |
| Sensitivity referral model | Supplement 1, eTable 3, Meld_cap | β = 0.1432, SE 0.0284, χ² 25.40, P < .0001 | (β/SE)² = 25.42; OR = 1.154 (1.091–1.220): higher odds |
| Mean MELD, referred vs not referred | Table 1; Results, Patient Characteristics | 32.3 (7.7) vs 27.4 (6.9), P < .001; SMD 0.68 | Difference +4.9 points; Welch t = 5.75 (df 228), p = 2.9 × 10⁻⁸; SMD 0.67 |
| Crude OR per MELD point (approximation) | from Table 1 | — | Pooled SD 7.21; slope 0.0944; OR ≈ 1.099 (> 1) |
| Referral rate | Abstract; Results | 120 (36.9%) | 120/325 = 36.9% |
Every number in the abstract sentence matches the Results and Table 2. The adjusted estimates, the sensitivity model and the descriptive data all point the same way: patients with higher MELD scores were more likely to be referred. Only the word "lower" in the abstract disagrees.
Figure
Figure 1. On the left is the abstract's sentence and on the right the Results text and Table 2 row. Both are short excerpts, and both printed estimates are shown in orange: OR 1.13 (95% CI 1.07–1.18), P < .001. The double arrow marks the conflict: the numbers are identical, but the abstract says "lower" and the Results say "increased". The blue box gives values recomputed from the printed numbers. The log OR and both log CI bounds are positive (z = 4.9), which contradicts "lower" and supports "increased". The Table 1 means (32.3 in referred vs 27.4 in non-referred patients) point the same way.
Figure 2. Odds ratios for referral per 1-point higher MELD score, on a log axis with a thin reference line at OR = 1. Orange: the estimate as printed in the abstract and in the Results and Table 2. It is identical in both places and lies entirely to the right of 1. Blue: values recomputed from the paper's own numbers. These are the eTable 3 sensitivity model, exp(0.1432 ± 1.96 × 0.0284) = 1.154 (1.091–1.220), and the crude normal-discriminant approximation from the Table 1 means (1.099, point estimate only), both above 1. The hollow blue marker is the reciprocal, 1/1.13 = 0.885 (0.847–0.935). This is the estimate a "lower referral odds" statement would require. The paper does not report it.
Why it matters
The abstract is the part of a paper that most readers see, and the part that indexing services, evidence summaries and citing articles most often take statements from. Referral is one of the study's primary outcomes, and MELD is one of its two named exposures. As printed, the abstract sentence tells a reader that sicker patients were less likely to be referred for transplant evaluation. That would be a clinically surprising result, and it is the opposite of what the paper's analysis shows. A reader who knows how to read an odds ratio can see that the sentence contradicts itself. A reader who takes in only the words, or a tool that extracts the claim from the text, would record the reversed direction.
The consequences are limited. No number is wrong, and the significance and size of the estimate are as printed: each additional MELD point goes with about 13% higher adjusted odds of referral. The title, Key Points and abstract Conclusions do not mention the MELD main effect. The Results, Table 2, Table 1, the Discussion and the supplement are all consistent with each other. The paper's headline finding is unaffected. That finding is the MELD × ADI interaction at intermediate MELD scores (P for interaction = .01) and the referral probabilities by ADI band, and neither depends on this sentence. The ADI, wait-listing, transplant and mortality results are also unaffected.
Explanations considered
- A different model or population in the abstract. The abstract says "in multivariable analysis" and gives the same OR, CI and P value as the multivariable referral model in the Results and the referral column of Table 2 (n = 299). No other referral model with a MELD estimate of 1.13 appears in the paper or the supplement. This does not explain the wording.
- MELD coded so that the OR is per lower score. The Methods, Table 1 and Table 2 (footnote c) define MELD as a score from 6 to 40 "with higher scores indicating severe liver disease". The Results describe the same OR of 1.13 as "increased odds" with higher MELD. If the abstract meant an OR per point of lower MELD, it would report 0.885 (0.847–0.935), not 1.13. This does not explain the wording.
- A different outcome (non-referral, or mortality). The sentence is explicitly about "referral odds". Under a non-referral coding the OR would be 0.885. The 180-day mortality OR for MELD (1.07 [1.03–1.12]) is also above 1 and is a different number. This does not explain the wording.
- The adjusted association could differ in sign from the crude one. It could in principle, but it does not here. The adjusted OR printed in the sentence is itself above 1. The crude data (Table 1, referred patients' MELD 4.9 points higher) and the eTable 3 sensitivity model agree in sign. This does not explain the wording.
- A text-conversion artifact. The error is a word, not a sign, superscript or decimal mark. The publisher's full-text page, the Europe PMC full-text XML and the local PMC archive all carry the same wording. This does not explain the wording.
- Rounding, one-sided tests or another stated convention. None of these can change the direction of an OR whose whole CI lies above 1, and the paper states no convention that would. This does not explain the wording.
The evidence fits a single wrong direction word in the abstract's Results paragraph. Nothing in the paper shows how it arose.
Status
As of 2026-10-23:
- Version of record. doi.org resolves to the publisher's full-text page (jamanetwork.com/journals/jamanetworkopen/fullarticle/2846714, published online March 19, 2026), which was read in this workflow. Its abstract Results has the same sentence, "higher MELD scores were associated with lower referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001)". Its body Results has "a higher MELD score was associated with increased odds of referral". Its Table 2 MELD row gives 1.13 (1.07-1.18), P < .001. The Europe PMC full-text XML and the PMC archive have identical text. The direct PMC web page returned only a bot-verification screen.
- Supplements. The Europe PMC supplementary-files archive was downloaded. It contains Supplement 1 (jamanetwopen-e262567-s001.pdf, eTables 1–4), Supplement 2 (Data Sharing Statement) and the figure images. Supplement 1 was read. eTable 3 gives the MELD (Meld_cap) coefficient as +0.1432 (SE 0.0284, P < .0001), which is a positive association with referral. eTable 1 gives mean MELD 29.2 overall and 32.3 among referred patients. Nothing in the supplement supports "lower" odds. The supplement agrees with the body text.
- Corrections. None found.
src/known.pyreturned no Crossref updates and no PMC or Europe PMC notices. The Crossref works record has no update-to or updated-by relation. Europe PMC (PMID 41854611) lists no comment or correction. The publisher page shows no correction notice and no posted comments. A web search for the title together with "correction", "erratum" or "PubPeer" (repeated for this write-up) found no notice, and a Retraction Watch site search found nothing. - PubPeer. The PubPeer search returned HTTP 403 and could not be queried directly. Web searches found no PubPeer thread for this article.
Sources: publisher full text; PMC13003376; Europe PMC REST full-text XML and supplementary files for PMC13003376; Crossref REST API for doi:10.1001/jamanetworkopen.2026.2567.
Suggested correction
Abstract, Results. Replace "In multivariable analysis, higher MELD scores were associated with lower referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001)." with:
In multivariable analysis, higher MELD scores were associated with higher referral odds (odds ratio, 1.13 [95% CI, 1.07-1.18]; P < .001).
The numbers stay as printed, and they match the Results text, Table 2 and Supplement 1. No other part of the paper needs to change.
Recomputation script (python recompute.py.txt)
"""Recompute every number in the PMC13003376 write-up and regenerate fig1/fig2.
Run: python recompute.py.txt
All inputs are values printed in the article (Abstract, Results, Tables 1-2)
or in Supplement 1 (eTable 3). Nothing is fitted to patient-level data.
"""
import math
import os
import numpy as np
from scipy import stats
HERE = os.path.dirname(os.path.abspath(__file__))
Z = stats.norm.ppf(0.975)
def se_log(lo, hi):
"""SE of log(OR) implied by a 95% CI."""
return (math.log(hi) - math.log(lo)) / (2 * Z)
# ------------------------------------------------------------------ inputs
OR, LO, HI = 1.13, 1.07, 1.18 # Abstract = Results p0 = Table 2 (referral, MELD <=40)
N_REFMODEL = 299 # Table 2 referral column header
N_ALL, N_REF = 325, 120 # Abstract / Results
M1, S1, N1 = 32.3, 7.7, 120 # Table 1: with referral
M0, S0, N0 = 27.4, 6.9, 205 # Table 1: without referral
B_E3, SE_E3, CHI_E3 = 0.1432, 0.0284, 25.4024 # Supplement 1 eTable 3, Meld_cap
print("=== 1. Direction implied by the reported OR (Abstract, Results, Table 2) ===")
b = math.log(OR); se = se_log(LO, HI); z = b / se
print(f" log OR = ln(1.13) = {b:+.3f}")
print(f" log CI bounds: ln(1.07) = {math.log(LO):+.3f}, ln(1.18) = {math.log(HI):+.3f} (both > 0: {math.log(LO) > 0})")
print(f" SE(log OR) from CI = {se:.4f}; z = {z:.2f}; two-sided p = {2*stats.norm.sf(abs(z)):.1e} (reported P < .001)")
print(f" geometric CI midpoint = {math.sqrt(LO*HI):.3f} (reported OR {OR})")
print(f" percent change in odds per +1 MELD point: {100*(OR-1):.0f}% (CI {100*(LO-1):.0f}% to {100*(HI-1):.0f}%)")
print(f" OR per +10 MELD points = 1.13^10 = {OR**10:.2f} (CI {LO**10:.2f} to {HI**10:.2f})")
print("\n=== 2. What 'lower referral odds' would require (reciprocal, not reported) ===")
print(f" OR per 1-point LOWER MELD = 1/1.13 = {1/OR:.3f} (CI {1/HI:.3f} to {1/LO:.3f})")
print("\n=== 3. Supplement 1 eTable 3 (sensitivity referral model, insurance x site interaction) ===")
print(f" Meld_cap coefficient {B_E3:+.4f}, SE {SE_E3:.4f}")
print(f" Wald chi2 = (b/SE)^2 = {(B_E3/SE_E3)**2:.2f} (printed {CHI_E3})")
lo3, hi3 = math.exp(B_E3 - Z*SE_E3), math.exp(B_E3 + Z*SE_E3)
OR3 = math.exp(B_E3)
print(f" OR = exp(b) = {OR3:.3f} (Wald 95% CI {lo3:.3f} to {hi3:.3f})")
print("\n=== 4. Descriptive cross-check, Table 1 MELD by referral status ===")
diff = M1 - M0
t = diff / math.sqrt(S1**2/N1 + S0**2/N0)
dfw = (S1**2/N1 + S0**2/N0)**2 / ((S1**2/N1)**2/(N1-1) + (S0**2/N0)**2/(N0-1))
print(f" mean difference = {M1} - {M0} = {diff:.1f} MELD points (referred higher)")
print(f" Welch t = {t:.2f}, df = {dfw:.0f}, p = {2*stats.t.sf(t, dfw):.1e} (reported P < .001)")
sp2 = ((N1-1)*S1**2 + (N0-1)*S0**2) / (N1+N0-2)
smd = diff / math.sqrt((S1**2 + S0**2)/2)
print(f" SMD (average-variance) = {smd:.2f} (Table 1 SMD 0.68)")
beta_crude = diff / sp2
OR_crude = math.exp(beta_crude)
print(f" pooled SD = {math.sqrt(sp2):.2f}; normal-discriminant crude logistic slope = {beta_crude:.4f}"
f" -> crude OR per MELD point ~ {OR_crude:.3f} (approximation; > 1)")
print("\n=== 5. Counts ===")
print(f" referral rate = {N_REF}/{N_ALL} = {100*N_REF/N_ALL:.1f}% (reported 36.9%)")
print(f" referral model n (Table 2 header) = {N_REFMODEL}")
# ------------------------------------------------------------------ figures
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch
from matplotlib.lines import Line2D
BG, TXT, SEC, ORA, BLU, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": SEC, "ytick.color": SEC, "font.family": "DejaVu Sans",
"svg.fonttype": "none"})
def save(fig, name):
fig.savefig(os.path.join(HERE, f"{name}.svg"), facecolor=BG, bbox_inches="tight")
fig.savefig(os.path.join(HERE, f"{name}.png"), facecolor=BG, bbox_inches="tight", dpi=200)
plt.close(fig)
# ---- Figure 1: abstract vs Results/Table 2 text diagram
fig = plt.figure(figsize=(7.2, 5.0), facecolor=BG)
ax = fig.add_axes([0, 0, 1, 1]); ax.set_xlim(0, 100); ax.set_ylim(0, 70); ax.axis("off")
def panel(x, title, lines, word, after, numbers):
ax.add_patch(FancyBboxPatch((x, 30), 38, 34, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="white", ec=SEC, lw=1.2))
ax.text(x + 2, 61.5, title, fontsize=11.5, weight="bold", va="top")
ax.text(x + 2, 55.5, lines, fontsize=10.5, style="italic", va="top", linespacing=1.35)
ax.text(x + 2, 44.6, word, fontsize=12, weight="bold", color=TXT, va="center",
bbox=dict(boxstyle="round,pad=0.25", fc="#fde6dc", ec=ORA, lw=1.5))
ax.text(x + 2 + 1.45*len(word) + 2.2, 44.6, after, fontsize=10.5, style="italic", va="center")
ax.text(x + 2, 37.3, numbers, fontsize=11, color=ORA, weight="bold", va="center", linespacing=1.3)
ax.text(x + 2, 32.4, "as printed in the paper", fontsize=10, color=SEC, va="center")
panel(2, "Abstract, Results",
"“In multivariable analysis, higher\nMELD scores were associated with",
"lower", "referral odds”", "OR 1.13 (95% CI 1.07–1.18);\nP < .001")
panel(60, "Results text and Table 2",
"“a higher MELD score was\nassociated with",
"increased", "odds of referral”", "OR 1.13 (95% CI 1.07–1.18);\nP < .001")
ax.add_patch(FancyArrowPatch((41.0, 44.6), (59.5, 44.6), arrowstyle="<|-|>", mutation_scale=13,
lw=2, color=TXT, shrinkA=1, shrinkB=1))
ax.text(50.5, 53.5, "Conflict:\nsame OR,\nCI and P;\nopposite\ndirection", ha="center", va="center",
fontsize=10, linespacing=1.15)
# recomputed box
ax.add_patch(FancyBboxPatch((14, 4), 72, 18, boxstyle="round,pad=0.4,rounding_size=1.5",
fc="#e8f0fb", ec=BLU, lw=2))
ax.text(50, 19.6, "Recomputed from the printed OR and CI", ha="center", va="top", fontsize=11,
weight="bold", color=BLU)
ax.text(50, 14.6,
f"ln(1.13) = {b:+.3f}; ln(1.07) = {math.log(LO):+.3f}, ln(1.18) = {math.log(HI):+.3f} (all > 0)\n"
f"z = {z:.1f}: each higher MELD point goes with HIGHER referral odds\n"
f"Table 1 agrees: mean MELD {M1} referred vs {M0} not referred",
ha="center", va="top", fontsize=10.2, linespacing=1.4)
ax.add_patch(FancyArrowPatch((28, 23.6), (22, 29.6), arrowstyle="-|>", mutation_scale=12,
lw=1.5, color=BLU))
ax.add_patch(FancyArrowPatch((72, 23.6), (78, 29.6), arrowstyle="-|>", mutation_scale=12,
lw=1.5, color=BLU))
ax.text(23, 26.6, "contradicts “lower”", fontsize=10, color=SEC, va="center", ha="right")
ax.text(77, 26.6, "supports “increased”", fontsize=10, color=SEC, va="center", ha="left")
leg = [Line2D([], [], color=ORA, lw=0, marker="s", ms=9, label="Reported in the paper"),
Line2D([], [], color=BLU, lw=0, marker="s", ms=9, label="Recomputed from the paper's numbers")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.5, 1.045), ncol=2, frameon=False,
fontsize=10)
save(fig, "fig1")
# ---- Figure 2: dot-and-interval plot on a log axis
rows = [
("Abstract (says “lower”)", OR, LO, HI, ORA, "o", True),
("Results and Table 2 (“increased”)", OR, LO, HI, ORA, "o", True),
("eTable 3 sensitivity model", OR3, lo3, hi3, BLU, "s", True),
("Crude, from Table 1 means", OR_crude, None, None, BLU, "D", True),
("1/1.13: what “lower” would need\n(not reported)", 1/OR, 1/HI, 1/LO, BLU, "o", False),
]
fig, ax = plt.subplots(figsize=(7.2, 4.3), facecolor=BG)
ax.set_facecolor(BG)
ys = np.arange(len(rows))[::-1].astype(float)
for (lab, o, lo, hi, col, mk, filled), y in zip(rows, ys):
if lo is not None:
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt")
txt = f"{o:.3f} ({lo:.3f}–{hi:.3f})" if col == BLU else f"{o:.2f} ({lo:.2f}–{hi:.2f})"
else:
txt = f"{o:.3f} (point only)"
ax.plot(o, y, marker=mk, ms=9, color=col, mfc=col if filled else BG, mew=2, lw=0)
ax.text(o, y + 0.33, txt, va="center", ha="center", fontsize=10, color=col)
ax.axvline(1.0, color=SEC, lw=0.9)
ax.set_xscale("log")
ax.set_xlim(0.80, 1.30)
ticks = [0.8, 0.9, 1.0, 1.1, 1.2, 1.3]
ax.set_xticks(ticks); ax.set_xticklabels([f"{t:.1f}" for t in ticks])
ax.xaxis.set_minor_locator(matplotlib.ticker.NullLocator())
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.tick_params(axis="y", length=0)
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ["top", "right", "left"]:
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.set_xlabel("Odds ratio for referral per 1-point higher MELD score (log scale)")
ax.text(0.985, ys[0] + 0.8, "← lower odds", ha="right", va="center", fontsize=10, color=SEC)
ax.text(1.015, ys[0] + 0.8, "higher odds →", ha="left", va="center", fontsize=10, color=SEC)
ax.set_ylim(-0.5, ys[0] + 1.1)
leg = [Line2D([], [], color=ORA, lw=2, marker="o", ms=8, label="Reported in the paper"),
Line2D([], [], color=BLU, lw=2, marker="s", ms=8, label="Recomputed from the paper's numbers")]
ax.legend(handles=leg, loc="upper center", bbox_to_anchor=(0.35, -0.2), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nFigures written: fig1.svg/png, fig2.svg/png")
Abstract reports P < 0.0001 for 3/170 vs 3/172 serious adverse events; the table gives P > 0.999
Nat Med (2025) · doi:10.1038/s41591-025-03674-y · PMC12092238
Summary
The abstract of this randomized trial (SUPPRESS-AF) says that procedure-related serious adverse events did not differ between PVI plus low-voltage-area ablation and PVI alone, and gives "1.7% versus 1.8%, P < 0.0001". The paper's own Extended Data Table 4 gives the counts behind those percentages, 3 of 170 and 3 of 172, and prints P > 0.999 for the same comparison with the test the Methods name (two-sided Fisher's exact test). Recomputed from those counts, Fisher's exact test gives P = 1.000. With six events in total, no split between the arms could give a two-sided Fisher P below 0.014. The words "no difference" are correct and the table is correct. The P value in the abstract is not, and a reader of the abstract alone sees a highly significant safety difference that the data do not show.
What the paper reports
Abstract:
There was no difference in the procedure-related serious adverse events between the two groups (1.7% versus 1.8%, P < 0.0001).
Extended Data Table 4, "Safety endpoints". This table is an image in the archived article; the text below was read from the publisher's image file (41591_2025_3674_Tab4_ESM.jpg). Columns: PVI+LVA-ABL n=170 | PVI-alone n=172 | p:
Procedure-related serious adverse events, n (%) | 3 (1.7) | 3 (1.8) | >0.999
Extended Data Table 4, footnote:
p for comparison between groups using the two-sided Fisher's exact test.
Methods, "Statistical analysis":
Safety endpoints were analyzed using Fisher's exact test. All tests were conducted at a two-sided α level of 0.05.
Results, "Safety endpoints":
There was no difference in procedure-related serious adverse events between the two groups. Case presentations of procedure-related serious adverse events are shown in Extended Data Table 6.
Discussion, fourth paragraph:
Serious, procedure-related, adverse event rates were not increased with the addition of LVA ablation.
Results, "Patients and follow-up" (the safety population):
A total of 342 patients were randomly assigned to either the LVA-ABL group (170 patients) or the PVI-alone group (172 patients) and safety outcomes were assessed.
Extended Data Table 6 (an image, read from 41591_2025_3674_Tab6_ESM.jpg) describes six cases: in PVI-alone, one cardiac tamponade, one stroke and one major bleeding; in PVI+LVA-ABL, one esophageal fistula and two major bleedings.
What the paper's own numbers show
Method. I took the counts and denominators from Extended Data Table 4 and checked them against the case list in Extended Data Table 6. I ran a two-sided Fisher's exact test on the 2×2 table [[3, 167], [3, 169]], the test that the Methods and the table footnote name. As robustness checks I also ran one-sided Fisher tests, chi-square tests with and without continuity correction, and Fisher's test with the efficacy-population denominator (171) in place of 172. Because Fisher's test conditions on the total number of events, I listed every possible split of the six events and its P value, to find the smallest P these margins allow. I also found how many events the LVA arm would need, against 3/172, for P < 0.0001. Finally I recomputed the other rows of Extended Data Table 4 with the same test to confirm that the method reproduces the paper's printed values. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Serious AEs, PVI+LVA-ABL | Abstract; Ext. Data Table 4 | 3 (1.7%) of 170 | 3/170 = 1.76% |
| Serious AEs, PVI-alone | Abstract; Ext. Data Table 4 | 3 (1.8%) of 172 | 3/172 = 1.74% (3/171 = 1.75%) |
| Case count | Ext. Data Table 6 | 6 cases | 3 PVI+LVA-ABL, 3 PVI-alone |
| P, serious AEs | Abstract | P < 0.0001 | Fisher two-sided P = 1.000 |
| P, serious AEs | Ext. Data Table 4 | >0.999 | Fisher two-sided P = 1.000 (odds ratio 1.012) |
| P with denominator 171 | robustness | – | 1.000 |
| One-sided Fisher P | robustness | – | 0.652 (greater) / 0.663 (less) |
| Chi-square P | robustness | – | 1.000 (Yates); 0.988 (uncorrected) |
| Risk difference | robustness | – | 0.02 percentage points (95% CI −2.76 to 2.80) |
| Lowest two-sided Fisher P over all splits of 6 events | robustness | – | 0.014 (all six in PVI-alone); 0.030 (all six in PVI+LVA-ABL) |
| LVA-arm events needed for P < 0.0001 vs 3/172 | robustness | – | 21/170 (12.4%), P = 8.7×10⁻⁵ |
| Bleeding events, 6 vs 2 | Ext. Data Table 4 | P = 0.173 | P = 0.1727 |
| Death from any cause, 2 vs 2 | Ext. Data Table 4 | >0.999 | P = 1.000 |
| Periprocedural AEs, 11 vs 4 | Ext. Data Table 4; Discussion | P = 0.069; OR 2.91 (0.91–9.31) | P = 0.0692; OR 2.91 (Woolf CI 0.91–9.31) |
| Symptomatic stroke, 4 vs 2 | Ext. Data Table 4 | 2 (1.7%), P = 0.723 | 2/172 = 1.16%, P = 0.447; with 3/172: 1.74%, P = 0.723 |
The same procedure reproduces the printed P values for bleeding, death and periprocedural events, and the table's own >0.999 for serious adverse events. Only the abstract's value disagrees.
Two smaller points, which do not bear on the main finding. First, 3/170 = 1.76% and 3/172 = 1.74%, so ordinary rounding would give 1.8% and 1.7%; the printed 1.7% and 1.8% appear in swapped order, in both the abstract and the table. Second, in the symptomatic stroke row the PVI-alone entry "2 (1.7)" is internally inconsistent: 2/172 is 1.2%, and Fisher's test on 4 vs 2 gives P = 0.447, whereas 3/172 gives both the printed 1.7% and the printed P = 0.723. The count may be 3 rather than 2; the paper does not say which entry is correct.
Figure
Figure 1. The two places the paper reports the comparison of procedure-related serious adverse events. The left panel quotes the abstract and the right panel reproduces the relevant row of Extended Data Table 4: the same percentages, the same counts and a footnote naming the two-sided Fisher exact test. The printed P values (orange) are P < 0.0001 and P > 0.999. The blue box gives the two-sided Fisher exact P recomputed from the table's counts, 3/170 vs 3/172, which is 1.000 and agrees with the table.
Figure 2. Given the arm sizes (170 and 172) and six events in total, these are the seven possible splits of the events. Bar heights are the probability of each split if the arms do not differ (hypergeometric distribution), and each bar is labelled with its two-sided Fisher exact P. The reported split, 3 vs 3, is the most likely split and has P = 1.000. The most extreme splits give P = 0.030 and 0.014, so no arrangement of six events in these arms could yield the P < 0.0001 printed in the abstract (orange text). All values are computed in recompute.py.txt.
Why it matters
The abstract is the part of the paper most readers see, and it is what indexing services and summaries reproduce. Read alone, "P < 0.0001" signals a highly significant difference in serious procedural harm between the two ablation strategies, the opposite of what the counts show and of what the same sentence says in words. A reader who notices the clash cannot tell from the abstract which part is wrong.
The effect on the paper's conclusions is small. The Results, Discussion and Extended Data Table 4 all report no difference, the table's P value is correct, and the primary efficacy result (61% vs 50% freedom from AF or AT, log-rank P = 0.127) does not depend on this value. The correct P (about 1.0) leaves the paper's safety reading unchanged: adding LVA ablation did not increase procedure-related serious adverse events in this trial. The trial is small for this endpoint, however (six events in total, risk-difference 95% CI about −2.8 to +2.8 percentage points), and the Discussion itself notes a tendency toward more periprocedural adverse events with LVA ablation (11 vs 4, P = 0.069). That point is separate and is not affected by this error.
Explanations considered
- Text conversion or typesetting artifact. Not supported. The same "P < 0.0001" appears in the publisher's PDF (PMC open-access copy, created by Springer on 2025-05-16) and in the Europe PMC full-text XML, where the "<" is an explicit
<entity between thin spaces. It is not a lost ">" sign read from an image, a mis-set superscript or a shifted table cell. - A different population, time point or event definition. Not supported. The abstract's percentages (1.7% and 1.8%) are exactly those printed in Extended Data Table 4 for procedure-related serious adverse events, and the case list in Extended Data Table 6 gives the same 3 vs 3. Using the efficacy denominator (171) instead of 172 also gives P = 1.000.
- A different test. Not supported. The Methods state that safety endpoints were analyzed with Fisher's exact test at a two-sided α of 0.05, and the table footnote says the same. One-sided Fisher tests (0.652, 0.663) and chi-square tests (0.988, 1.000) give values near 1. With these margins no split of the six events yields a two-sided Fisher P below 0.014.
- A non-inferiority or equivalence test. Not supported. A one-sided test against a margin could produce a small P for two near-identical rates (for example, a hypothetical margin of 5 percentage points would give P ≈ 2.3×10⁻⁴, and 3 points P ≈ 0.018), but the paper describes no non-inferiority or equivalence analysis and no safety margin, the trial was designed for superiority on the primary efficacy endpoint, and the sentence frames the result as "no difference". No margin is stated that would reproduce exactly "P < 0.0001".
- A correct value of a different comparison. No other comparison in the archived text is reported as P < 0.0001 (the smallest printed values, for procedural times and energy in Table 2, are given as <0.001), so there is no nearby value the sentence could have taken by mistake that can be identified. The cause of the discrepancy cannot be determined from the paper; what can be determined is that the printed value is incompatible with the printed counts and stated test.
Status
As of 2026-10-23:
- Version of record. The error is in the version of record. The nature.com article page could not be fetched automatically (it redirected to a login page), so I checked the publisher's PDF from the PMC open-access collection (PMC12092238.1.pdf, metadata creator "Springer", subject "Nature Medicine, doi:10.1038/s41591-025-03674-y") and the Europe PMC full-text XML. Both contain "(1.7% versus 1.8%, P < 0.0001)".
- Supplements. Extended Data Tables 4 (safety endpoints), 5 (periprocedural adverse events) and 6 (case descriptions of procedure-related serious adverse events) were read from the publisher's image files on media.springernature.com; the PMC text has captions only for these tables. Extended Data Tables 4 and 6 resolve the question: they give 3 vs 3 events and P > 0.999. The Supplementary Information PDF (protocol and statistical analysis plan) could not be downloaded in a readable form; it is not needed to settle the discrepancy.
- Corrections and comments. No correction, erratum, retraction or expression of concern was found. Crossref lists no update relations for the DOI, and PMC and Europe PMC list no correction or comment notices. A web search for an author correction to this article returned none.
- PubPeer. The PubPeer API returned no publication record for the DOI, and the PubPeer search page refused automated access (HTTP 403), so PubPeer could not be checked directly. A web search for the title together with "PubPeer" found no thread. Absence of a PubPeer discussion is therefore not fully confirmed.
- Sources. PMC article PMC12092238; publisher PDF (PMC open-access copy); Europe PMC full-text XML; Extended Data Table images 41591_2025_3674_Tab4_ESM.jpg, Tab5_ESM.jpg and Tab6_ESM.jpg (media.springernature.com); Crossref; Europe PMC; PubPeer.
Suggested correction
- In the abstract, replace "(1.7% versus 1.8%, P < 0.0001)" with the value in Extended Data Table 4, for example "(3/170 (1.8%) versus 3/172 (1.7%), P > 0.999)".
- Check the order of the percentages for this row in the abstract and in Extended Data Table 4: 3/170 rounds to 1.8% and 3/172 to 1.7%.
- Check the symptomatic stroke row of Extended Data Table 4: the PVI-alone entry "2 (1.7)" with P = 0.723 is consistent with 3 events, not 2. Correct either the count or the percentage and P value.
- Optional: in the Discussion, the periprocedural adverse event percentages are labelled "95% CI" ("11 (95% CI = 6.5%) versus 4 (95% CI = 2.3%) patients"); they are proportions, not confidence intervals.
Recomputation script (python recompute.py.txt)
"""Recompute the procedure-related serious adverse event P value of PMC12092238
(Nat Med 2025, doi:10.1038/s41591-025-03674-y, SUPPRESS-AF).
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes, next to this file:
fig1.svg/png (abstract vs Extended Data Table 4, text-panel diagram)
fig2.svg/png (all possible splits of the six events and their Fisher P values)
candidates/cand_A..cand_E.svg/png (five candidate designs for review; A = fig1, E = fig2)
Inputs are copied from the paper:
Abstract: "(1.7% versus 1.8%, P < 0.0001)"
Extended Data Table 4 (image 41591_2025_3674_Tab4_ESM.jpg), safety population
PVI+LVA-ABL n = 170, PVI-alone n = 172, P by two-sided Fisher's exact test
Extended Data Table 6: six case descriptions (3 PVI+LVA-ABL, 3 PVI-alone)
Discussion: OR = 2.91 (95% CI 0.91-9.31) for periprocedural adverse events
"""
import math
import os
import numpy as np
from scipy import stats
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
N_LVA, N_PVI = 170, 172 # Extended Data Table 4 headers (safety population)
N_PVI_EFFICACY = 171 # abstract / efficacy population
SAE = (3, 3) # Procedure-related serious AEs, Extended Data Table 4
SAE_PRINTED_PCT = (1.7, 1.8) # Abstract and Extended Data Table 4
ABSTRACT_P = "P < 0.0001"
TABLE_P = ">0.999"
# Other Extended Data Table 4 rows: (a, b, printed % a, printed % b, printed P)
ROWS = {
"Bleeding events": (6, 2, 3.5, 1.2, "0.173"),
"Symptomatic stroke": (4, 2, 2.4, 1.7, "0.723"),
"Death from any cause": (2, 2, 1.2, 1.2, ">0.999"),
"Periprocedural adverse events": (11, 4, 6.5, 2.3, "0.069"),
"Procedure-related serious AEs": (3, 3, 1.7, 1.8, ">0.999"),
}
# Extended Data Table 6 case list
TABLE6 = [("PVI-alone", "Cardiac tamponade"), ("PVI-alone", "Stroke"),
("PVI+LVA-ABL", "Esophageal fistula"), ("PVI+LVA-ABL", "Major bleeding"),
("PVI-alone", "Major bleeding"), ("PVI+LVA-ABL", "Major bleeding")]
# ---------------------------------------------------------------- style
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG,
"svg.fonttype": "none", "lines.linewidth": 1.5,
})
LW = 1.5 # 1.5 pt = 2 px at 96 px/in
MS = 8 # marker size in points (>= 8 px)
def fisher2(a, n1, b, n2, alt="two-sided"):
return stats.fisher_exact([[a, n1 - a], [b, n2 - b]], alternative=alt)[1]
def save(fig, path_noext):
fig.savefig(path_noext + ".svg")
fig.savefig(path_noext + ".png", dpi=200)
plt.close(fig)
def cp_ci(k, n, a=0.05):
lo = stats.beta.ppf(a / 2, k, n - k + 1) if k > 0 else 0.0
hi = stats.beta.ppf(1 - a / 2, k + 1, n - k) if k < n else 1.0
return lo, hi
# ---------------------------------------------------------------- 1. the serious-AE comparison
a, b = SAE
print("=== 1. Procedure-related serious AEs, Extended Data Table 4 ===")
print(f"PVI+LVA-ABL {a}/{N_LVA} = {100*a/N_LVA:.2f}% (printed {SAE_PRINTED_PCT[0]})")
print(f"PVI-alone {b}/{N_PVI} = {100*b/N_PVI:.2f}% (printed {SAE_PRINTED_PCT[1]})")
print(f"PVI-alone with efficacy denominator {b}/{N_PVI_EFFICACY} = {100*b/N_PVI_EFFICACY:.2f}%")
t = np.array([[a, N_LVA - a], [b, N_PVI - b]])
or_, p2 = stats.fisher_exact(t)
P_FISHER = p2
print(f"Fisher exact two-sided P = {p2:.4f} (odds ratio {or_:.3f}) reported: abstract {ABSTRACT_P}, table {TABLE_P}")
print(f"Fisher exact two-sided P with n = 171 = {fisher2(a, N_LVA, b, N_PVI_EFFICACY):.4f}")
P_GT = fisher2(a, N_LVA, b, N_PVI, "greater")
P_LT = fisher2(a, N_LVA, b, N_PVI, "less")
print(f"Fisher one-sided P: greater = {P_GT:.3f}, less = {P_LT:.3f}")
P_YATES = stats.chi2_contingency(t)[1]
P_CHI = stats.chi2_contingency(t, correction=False)[1]
print(f"chi-square P: Yates = {P_YATES:.3f}, uncorrected = {P_CHI:.4f}")
pA, pB = a / N_LVA, b / N_PVI
d = pA - pB
se = math.sqrt(pA * (1 - pA) / N_LVA + pB * (1 - pB) / N_PVI)
print(f"risk difference = {100*d:.3f} pp, Wald 95% CI {100*(d-1.96*se):.2f} to {100*(d+1.96*se):.2f} pp")
for m in (0.03, 0.05): # hypothetical non-inferiority margins (the paper describes none)
print(f"hypothetical one-sided non-inferiority test, margin {100*m:.0f} pp: P = {stats.norm.sf((m-d)/se):.1e}")
CI_LVA, CI_PVI = cp_ci(a, N_LVA), cp_ci(b, N_PVI)
print(f"Clopper-Pearson 95% CI: LVA {100*CI_LVA[0]:.2f}-{100*CI_LVA[1]:.2f}%, PVI {100*CI_PVI[0]:.2f}-{100*CI_PVI[1]:.2f}%")
cnt = {"PVI+LVA-ABL": 0, "PVI-alone": 0}
for g, _ in TABLE6:
cnt[g] += 1
print(f"Extended Data Table 6 case count: {cnt}")
# ---------------------------------------------------------------- 2. every split of the six events
print("\n=== 2. Every possible split of the six events (margins fixed: 170, 172, 6) ===")
TOT = a + b
hg = stats.hypergeom(N_LVA + N_PVI, TOT, N_LVA) # population, successes, draws
SPLIT = []
for k in range(TOT + 1):
pk = hg.pmf(k)
pf = fisher2(k, N_LVA, TOT - k, N_PVI)
SPLIT.append((k, pk, pf))
print(f" {k} vs {TOT-k}: probability under no difference = {pk:.4f}, two-sided Fisher P = {pf:.4f}")
PMIN = min(s[2] for s in SPLIT)
print(f"smallest attainable two-sided Fisher P with six events = {PMIN:.4f}")
# ---------------------------------------------------------------- 3. what would P < 0.0001 require
print("\n=== 3. Events needed in the LVA arm (vs 3/172) for two-sided Fisher P < 0.0001 ===")
CURVE = [(x, fisher2(x, N_LVA, b, N_PVI)) for x in range(0, 31)]
X_NEED = next(x for x, p in CURVE if x > b and p < 1e-4)
print(f" {X_NEED}/{N_LVA} = {100*X_NEED/N_LVA:.1f}% (P = {dict(CURVE)[X_NEED]:.2e}); "
f"{X_NEED-1}/{N_LVA}: P = {dict(CURVE)[X_NEED-1]:.2e}")
X05 = next(x for x, p in CURVE if x > b and p < 0.05)
print(f" first count with P < 0.05: {X05}/{N_LVA} (P = {dict(CURVE)[X05]:.4f})")
# ---------------------------------------------------------------- 4. other rows of Extended Data Table 4
print("\n=== 4. Other rows of Extended Data Table 4 ===")
for name, (x, y, ra, rb, rp) in ROWS.items():
print(f" {name:31s} {x}/{N_LVA} = {100*x/N_LVA:.2f}% (printed {ra}); "
f"{y}/{N_PVI} = {100*y/N_PVI:.2f}% (printed {rb}); Fisher P = {fisher2(x, N_LVA, y, N_PVI):.4f} (printed {rp})")
print(f" stroke row if PVI-alone count were 3: 3/172 = {300/172:.2f}%, Fisher P (4 vs 3) = {fisher2(4, N_LVA, 3, N_PVI):.3f}")
o = (11 * (N_PVI - 4)) / ((N_LVA - 11) * 4)
s = math.sqrt(1/11 + 1/(N_LVA-11) + 1/4 + 1/(N_PVI-4))
print(f" periprocedural AEs OR = {o:.2f}, Woolf 95% CI {math.exp(math.log(o)-1.96*s):.2f}-{math.exp(math.log(o)+1.96*s):.2f} (Discussion: 2.91, 0.91-9.31)")
# ================================================================= figures
def panel(ax, x, y, w, h, title, edge):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.015",
fc="white", ec=edge, lw=LW, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, fontsize=11,
fontweight="bold", va="top", color=INK)
def fig_A(path):
"""Text panels: abstract vs Extended Data Table 4, with recomputed value."""
fig = plt.figure(figsize=(7.2, 4.5))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
r = fig.canvas.get_renderer()
# left: abstract
panel(ax, 0.03, 0.42, 0.44, 0.54, "Abstract", INK2)
ax.text(0.05, 0.82, "\u201cThere was no difference in the\nprocedure-related serious adverse\n"
"events between the two groups\n(1.7% versus 1.8%,", transform=ax.transAxes,
fontsize=10.5, va="top", color=INK, linespacing=1.35)
tp = ax.text(0.05, 0.575, "P < 0.0001", transform=ax.transAxes, fontsize=13, fontweight="bold",
color=ORANGE, va="center")
bb = tp.get_window_extent(r).transformed(ax.transAxes.inverted())
ax.text(bb.x1 + 0.003, 0.575, ").\u201d", transform=ax.transAxes, fontsize=10.5, va="center")
# right: Extended Data Table 4
panel(ax, 0.53, 0.42, 0.44, 0.54, "Extended Data Table 4", INK2)
ax.text(0.55, 0.82, "Procedure-related serious\nadverse events, n (%)", transform=ax.transAxes,
fontsize=10.5, va="top", linespacing=1.35)
for xx, lab, val in [(0.55, "PVI+LVA-ABL\nn = 170", "3 (1.7)"), (0.705, "PVI-alone\nn = 172", "3 (1.8)")]:
ax.text(xx, 0.68, lab, transform=ax.transAxes, fontsize=10, color=INK2, va="top")
ax.text(xx, 0.565, val, transform=ax.transAxes, fontsize=10.5, va="center")
ax.text(0.855, 0.68, "P", transform=ax.transAxes, fontsize=10, color=INK2, va="top")
ax.text(0.855, 0.565, ">0.999", transform=ax.transAxes, fontsize=13, fontweight="bold",
color=ORANGE, va="center")
ax.text(0.55, 0.465, "footnote: two-sided Fisher\u2019s exact test", transform=ax.transAxes,
fontsize=10, color=INK2, va="center")
# connector between the two P values, below the panels
arr = FancyArrowPatch((0.13, 0.415), (0.90, 0.415), transform=ax.transAxes,
connectionstyle="arc3,rad=0.16", arrowstyle="<->", mutation_scale=12,
lw=LW, color=INK2)
ax.add_patch(arr)
ax.text(0.5, 0.275, "Conflict: same counts and same comparison, but P < 0.0001 vs P > 0.999",
transform=ax.transAxes, ha="center", fontsize=10.5, color=INK, fontweight="bold")
# recomputed
ax.add_patch(FancyBboxPatch((0.08, 0.075), 0.84, 0.15, boxstyle="round,pad=0,rounding_size=0.015",
fc="white", ec=BLUE, lw=LW, transform=ax.transAxes))
ax.text(0.5, 0.185, "Recomputed from the table\u2019s counts", transform=ax.transAxes,
ha="center", fontsize=10, color=INK2, va="center")
ax.text(0.5, 0.118, f"Fisher\u2019s exact test, 3/170 vs 3/172: two-sided P = {P_FISHER:.3f}",
transform=ax.transAxes, ha="center", fontsize=11, color=BLUE, fontweight="bold", va="center")
ax.text(0.5, 0.025, "Orange: values printed in the paper. Blue: recomputed from the paper\u2019s counts.",
transform=ax.transAxes, ha="center", fontsize=10, color=INK2)
save(fig, path)
def fig_B(path):
"""P values on one log axis."""
rows = [
("Abstract (printed)", 1e-4, ORANGE, "lt", "P < 0.0001"),
("Ext. Data Table 4 (printed)", 0.999, ORANGE, "gt", "P > 0.999"),
("Fisher, two-sided (stated test)", P_FISHER, BLUE, "", f"{P_FISHER:.3f}"),
("Chi-square, uncorrected", P_CHI, BLUE, "", f"{P_CHI:.3f}"),
("Fisher, one-sided (each direction)", min(P_GT, P_LT), BLUE, "", f"{P_GT:.3f} / {P_LT:.3f}"),
("Lowest P any split of 6 events allows", PMIN, BLUE, "", f"{PMIN:.3f}"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.6))
ys = np.arange(len(rows))[::-1]
for y, (lab, v, c, kind, txt) in zip(ys, rows):
mk = "o" if c == BLUE else "D"
if lab.startswith("Fisher, one-sided"):
ax.plot([P_LT, P_GT], [y, y], color=c, lw=LW)
ax.plot([P_GT, P_LT], [y, y], mk, color=c, ms=MS, mec="white", mew=1)
else:
ax.plot(v, y, mk, color=c, ms=MS, mec="white", mew=1)
if kind == "lt":
ax.annotate("", xy=(1.6e-6, y), xytext=(1e-4, y),
arrowprops=dict(arrowstyle="->", color=c, lw=LW))
ax.text(1.03, y, txt, color=INK, fontsize=10, ha="left", va="center",
transform=ax.get_yaxis_transform())
ax.axvline(0.05, color=INK2, lw=0.8, ls=(0, (3, 3)))
ax.text(0.05 * 1.12, len(rows) - 0.45, "α = 0.05", fontsize=10, color=INK2)
ax.set_xscale("log"); ax.set_xlim(1e-6, 2.2); ax.set_ylim(-0.6, len(rows) - 0.2)
ax.set_yticks(ys); ax.set_yticklabels([r[0] for r in rows], fontsize=10)
ax.set_xlabel("P value (log scale)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "D", color=ORANGE, ms=MS, label="printed in the paper")
h2, = ax.plot([], [], "o", color=BLUE, ms=MS, label="recomputed from the paper\u2019s counts")
fig.legend(handles=[h1, h2], loc="upper center", ncol=2, frameon=False, fontsize=10)
fig.tight_layout(rect=(0, 0, 0.87, 0.92))
save(fig, path)
def fig_C(path):
"""Fisher P as a function of the LVA-arm count."""
xs = np.array([c[0] for c in CURVE]); ps = np.array([c[1] for c in CURVE])
fig, ax = plt.subplots(figsize=(7.2, 3.8))
ax.plot(xs, ps, color=BLUE, lw=LW, marker="o", ms=5, mec="white", mew=0.6,
label="two-sided Fisher P, k/170 vs 3/172 (recomputed)")
ax.axhline(0.05, color=INK2, lw=0.8, ls=(0, (3, 3)))
ax.text(30.3, 0.05, "0.05", va="center", fontsize=10, color=INK2)
ax.axhline(1e-4, color=ORANGE, lw=LW, ls=(0, (5, 2)), label="abstract: P < 0.0001 (printed)")
ax.text(30.3, 1e-4, "0.0001", va="center", fontsize=10, color=INK2)
ax.plot(3, P_FISHER, "o", color=BLUE, ms=11, mec=INK, mew=1.2, zorder=5)
ax.annotate(f"reported counts 3 vs 3:\nP = {P_FISHER:.3f}", (3, P_FISHER), xytext=(0.6, 2e-3),
fontsize=10, arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
pn = dict(CURVE)[X_NEED]
ax.plot(X_NEED, pn, "o", color=BLUE, ms=11, mec=INK, mew=1.2, zorder=5)
ax.annotate(f"{X_NEED}/170 ({100*X_NEED/N_LVA:.1f}%) would be\nneeded for P < 0.0001",
(X_NEED, pn), xytext=(22.5, 3e-3), fontsize=10,
arrowprops=dict(arrowstyle="-", color=INK2, lw=0.8))
ax.set_yscale("log"); ax.set_ylim(1e-8, 2.5); ax.set_xlim(-0.5, 30)
ax.set_xlabel("Events in the PVI+LVA-ABL arm (of 170), PVI-alone fixed at 3 of 172")
ax.set_ylabel("Two-sided P (log scale)")
ax.grid(color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right"):
ax.spines[s].set_visible(False)
ax.legend(loc="lower left", frameon=False, fontsize=10)
fig.tight_layout()
save(fig, path)
def fig_D(path):
"""Per-arm event rates with exact 95% CIs."""
fig, ax = plt.subplots(figsize=(7.2, 2.9))
arms = [("PVI+LVA-ABL (3/170)", a / N_LVA, CI_LVA, SAE_PRINTED_PCT[0]),
("PVI-alone (3/172)", b / N_PVI, CI_PVI, SAE_PRINTED_PCT[1])]
for y, (lab, p, ci, pr) in zip([1, 0], arms):
ax.plot([100 * ci[0], 100 * ci[1]], [y, y], color=BLUE, lw=LW)
ax.plot(100 * p, y, "o", color=BLUE, ms=MS, mec="white", mew=1)
ax.plot(pr, y + 0.22, "D", color=ORANGE, ms=MS, mec="white", mew=1)
ax.text(100 * ci[1] + 0.15, y, f"{100*p:.2f}% (95% CI {100*ci[0]:.2f}–{100*ci[1]:.2f})",
va="center", fontsize=10)
ax.text(pr + 0.15, y + 0.22, f"printed {pr}%", va="center", fontsize=10, color=INK2)
ax.set_yticks([1, 0]); ax.set_yticklabels([r[0] for r in arms])
ax.set_xlim(0, 9.5); ax.set_ylim(-1.0, 1.6)
ax.set_xlabel("Procedure-related serious adverse events, % of patients")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
h1, = ax.plot([], [], "D", color=ORANGE, ms=MS, label="printed in the paper")
h2, = ax.plot([], [], "o-", color=BLUE, ms=MS, lw=LW, label="recomputed, exact 95% CI")
ax.legend(handles=[h1, h2], loc="lower right", ncol=2, frameon=False, fontsize=10)
ax.set_title(f"Abstract prints P < 0.0001; these counts give Fisher P = {P_FISHER:.3f}",
fontsize=11, loc="left")
fig.tight_layout()
save(fig, path)
def fig_E(path):
"""All seven possible splits of six events, with their two-sided Fisher P."""
ks = [s[0] for s in SPLIT]; pk = [s[1] for s in SPLIT]; pf = [s[2] for s in SPLIT]
fig, ax = plt.subplots(figsize=(7.2, 3.8))
cols = [BLUE if k == a else "#bcd3ee" for k in ks]
ax.bar(ks, pk, width=0.72, color=cols, edgecolor=BG, lw=2)
for k, p, f in zip(ks, pk, pf):
ax.text(k, p + 0.008, f"P = {f:.3f}", ha="center", fontsize=10, color=INK)
ax.text(k, -0.045, f"{k} vs {TOT-k}", ha="center", fontsize=10, color=INK2)
ax.text(a, pk[a] + 0.035, "reported split", ha="center", fontsize=10, color=BLUE, fontweight="bold")
ax.text(-0.5, 0.35, "Abstract: P < 0.0001 (printed)", ha="left", va="top", fontsize=10.5,
color=ORANGE, fontweight="bold")
ax.text(-0.5, 0.315, f"No split of six events can give a\ntwo-sided Fisher P below {PMIN:.3f}",
ha="left", va="top", fontsize=10, color=INK)
ax.set_xticks([]); ax.set_ylim(-0.07, 0.37); ax.set_yticks(np.arange(0, 0.36, 0.05)); ax.set_xlim(-0.6, 6.6)
ax.axhline(0, color=INK2, lw=0.8)
ax.set_ylabel("Probability if the arms do not differ")
ax.set_xlabel("Split of the six events (PVI+LVA-ABL vs PVI-alone); bar labels: two-sided Fisher P", labelpad=4)
ax.grid(axis="y", color=GRID, lw=0.8); ax.set_axisbelow(True)
ax.spines["left"].set_bounds(0, 0.35)
for s in ("top", "right", "bottom"):
ax.spines[s].set_visible(False)
fig.tight_layout()
save(fig, path)
for name, fn in [("cand_A", fig_A), ("cand_B", fig_B), ("cand_C", fig_C), ("cand_D", fig_D), ("cand_E", fig_E)]:
fn(os.path.join(CAND, name))
fig_A(os.path.join(HERE, "fig1"))
fig_E(os.path.join(HERE, "fig2"))
print("\nwrote fig1, fig2 and candidates/cand_A..E (.svg, .png) in", HERE)
Abstract reports 347 participants completing both evaluations; the paper's body and group counts give 281
Neurology (2021) · doi:10.1212/WNL.0000000000011089 · PMC7884983
Summary
The abstract says that "a total of 347 participants completed both time points of evaluation", meaning both the 1-year and the 5-year clinical evaluations. The body gives a different number three times: the Methods say 347 completed the 1-year visit, 348 the 5-year visit and 281 both; the Statistical Analysis section gives the longitudinal n as 281; and the Results group counts for both visits add up to 80 + 136 + 39 + 26 = 281. So the abstract's 347 is the 1-year completer count, and it overstates the paired cohort behind the abstract's 1-to-5-year decline percentages by 66 participants (23.5%) and obscures that 19.0% of the 1-year completers had no 5-year evaluation. The analyses themselves use the correct sample sizes, so no reported effect, percentage or conclusion changes.
What the paper reports
All quotations were checked verbatim against the PMC-archived text.
Abstract, Methods:
One-year and 5-year clinical evaluations included identical batteries for neurobehavioral, psychiatric, and cognitive outcomes. A total of 347 participants completed both time points of evaluation. Cross-sectional and longitudinal comparisons were assessed.
Abstract, Results (these are the longitudinal estimates the "both time points" cohort supports):
Both combat concussion groups also showed clinically significant decline from 1- to 5-year evaluation (66%–76% neurobehavioral NRS; 41%–54% psychiatric CAPS by group). Both control groups fared better but a subset also had clinically significant decline (37%–50% neurobehavioral NRS; 9%–25% psychiatric CAPS by group).
Methods, second paragraph:
Through these efforts, 575 participants have been prospectively enrolled and assessed at the acute (0–7 days, median 4, study 4) and subacute time points (0–30 days, median 7–9, study 2–3; 0–90 days, median 14, study 1), 347 of whom completed further clinical examination at 1 year and 348 at 5 years postinjury, with 281 completing both follow-up evaluations.
Methods, Statistical Analysis:
Five-year cross-sectional analysis considered the entire cohort that completed this follow-up evaluation (n = 348) while longitudinal analysis only considered those who completed both the 1-year and 5-year evaluations (n = 281).
Results, first paragraph:
At 5-year follow-up, 109 nonblast controls and 170 patients with concussive blast TBI, as well as 41 blast controls and 28 patients with nonblast TBI, completed evaluation, of which 80 nonblast controls, 136 patients with concussive blast TBI, 39 blast controls, and 26 patients with nonblast TBI completed both 1- and 5-year follow-up.
Figure 1 (CONSORT diagram, an image in the article) has a box labelled "Total completed 1-year follow-up (n = 347)" and another labelled "Total completed 5-year follow-up (n = 348)". It gives no count of participants with both visits. Table 1 (also an image) gives the 5-year group sizes in its column headers: nonblast CTL n = 109, blast-exposed CTL n = 41, concussive blast TBI n = 170, nonblast TBI n = 28.
What the paper's own numbers show
Method. I added up the group counts the paper gives for each cohort and compared each sum with the stated total and with the abstract. For the 1-year cohort I used the four study boxes in Figure 1. I also checked the logical bound that the number seen at both visits cannot exceed the number seen at either one, overall and within each group. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Completed both 1- and 5-year evaluations | Abstract, Methods | 347 | 281 (Results 80 + 136 + 39 + 26) |
| Completed both, as stated in the body | Methods p2; Statistical Analysis | 281; n = 281 | 281 (same sum) |
| Completed 1-year evaluation | Methods p2; Figure 1 | 347; n = 347 | 347 (Figure 1 boxes 65 + 32 + 178 + 72) |
| Completed 5-year evaluation | Methods p2; Statistical Analysis; Figure 1 | 348; n = 348; n = 348 | 348 (Results and Table 1: 109 + 170 + 41 + 28) |
| Abstract minus both-visit n | Abstract vs body | — | 347 − 281 = 66 (23.5% of 281) |
| Largest possible both-visit n | derived | — | min(347, 348) = 347, reached only if every 1-year completer also had a 5-year visit |
| 1-year completers with no 5-year visit | derived | — | 347 − 281 = 66 (19.0% of 347); by group 23 / 34 / 6 / 3, sum 66 |
| 5-year completers with no 1-year visit | derived | — | 348 − 281 = 67 |
| Seen at 1 year, 5 years or both | derived | — | 347 + 348 − 281 = 414 of 575 enrolled |
| Both-visit n within each group ≤ 1-year and 5-year n | Figure 1; Results | 80, 136, 39, 26 | holds for all four groups (1-year: 103, 170, 45, 29; 5-year: 109, 170, 41, 28) |
Group order in the last two rows: nonblast controls, blast TBI, blast controls, nonblast TBI.
The body agrees with itself everywhere. The 1-year, 5-year and both-visit totals each match the sum of their components, and the per-group counts satisfy the bound. The abstract is the only place where the both-visit cohort is given as 347, and 347 is exactly the 1-year total. A both-visit count of 347 would mean that every 1-year completer also completed the 5-year visit. That contradicts the 281 stated in the Methods, which implies that 66 participants were seen at 1 year only.
Figure
Figure 1. On the left is the abstract sentence that gives the cohort with both evaluations, with its count of 347 (orange, as reported). On the right are the three body passages that give the same cohort: the Methods study-population paragraph, the Statistical Analysis section and the first Results paragraph. The blue line is the sum of the Results group counts (80 + 136 + 39 + 26 = 281), recomputed here. The orange connectors mark the conflict: 347 in the abstract against 281 in the body, a difference of 66 participants (+23.5%). The grey connector shows that 347 is the number the Methods give for the 1-year visit alone. All quotations are verbatim, with "..." marking omitted text.
Figure 2. Cohort sizes as reported in the paper (filled orange dots) and as recomputed from the paper's own components (open blue circles), on an axis starting at zero. For the 1-year, 5-year and both-visit cohorts given in the body, the reported totals equal the sums of their parts (Figure 1 study boxes for the 1-year cohort; Results group counts for the others). The abstract's count of participants with both visits (347) sits 66 above the both-visit sum (281), at the same value as the 1-year total.
Why it matters
- The abstract misstates the size of the paired cohort. The abstract's Results give 1-to-5-year decline percentages for each group, and those percentages come from the participants seen at both visits. A reader of the abstract would take that cohort to be 347, not 281, about 23% larger than it is.
- Attrition is not visible from the abstract. With 347 "both", the abstract implies that nobody seen at 1 year was lost by year 5. In fact 66 of the 347 1-year completers (19.0%) were not in the paired cohort. Readers judging attrition bias in the longitudinal estimates need that number.
- Who could be misled. Mainly systematic reviewers and meta-analysts who take sample sizes from abstracts and could overweight the longitudinal estimates or misjudge loss to follow-up, and researchers planning follow-up cohorts. Crossref lists 17 citing works.
- What is not affected. The cross-sectional 5-year analysis used n = 348 and the longitudinal analysis n = 281, as the Statistical Analysis section states. No reported effect size, confidence interval, p value or percentage depends on the abstract's 347, and the direction and size of the findings are unchanged. The error is confined to one descriptive sentence in the abstract.
Explanations considered
- "Both time points" means a different pair of visits, such as acute and 5-year. Not supported. The sentence directly before it in the abstract names the "One-year and 5-year clinical evaluations", and the body defines the both-visit (longitudinal) cohort as the 1-year plus 5-year completers, n = 281.
- 347 is the analysed cross-sectional cohort. Not supported. The Statistical Analysis section gives the 5-year cross-sectional cohort as n = 348, which matches the Results and Table 1 group sizes (109 + 170 + 41 + 28 = 348).
- 347 is the 1-year cohort. This fits the number, but not the abstract's wording. The Methods say "347 of whom completed further clinical examination at 1 year", and the Figure 1 study boxes for the 1-year visit add up to 347. That is a count for one time point. It is not the number of participants seen at both.
- A different analysis set, population or exclusion rule. Not supported. No footnote, legend, table or Methods passage gives 347 as a both-visit count or defines an analysis set of that size. The per-group counts with both visits are given only in the Results, and they sum to 281.
- The longitudinal percentages could identify the denominator. They cannot, so they are not used as evidence here. For example, the nonblast controls' 72% / 23% / 5% (unchanged / better / worse) fits n = 80 (58 / 18 / 4) but also fits n = 109 (79 / 25 / 5).
- A text-conversion artifact. Ruled out. The number is plain text with no markup. The PMC page, the article PDF in the PMC open-access collection, the Europe PMC full-text XML and the PubMed abstract (PMID 33177226) all carry "A total of 347 participants completed both time points of evaluation."
- A correction already issued. None found (see Status).
- The 5-year counts are consistent across Figure 1, Table 1 and the Results. The 5-year total of 348 and the Table 1 group sizes include the 6 participants that Figure 1 lists as deceased before the 5-year visit (Study 1: 4 blast TBI; Study 3: 1 blast control, 1 blast TBI). The Study 2 death is marked as having occurred after the 5-year visit and is one of that study's 31. With these, the figure and table agree: blast TBI 165 + 5 = 170, blast controls 40 + 1 = 41, all groups 342 + 6 = 348.
Status
- Version of record. The publisher's page (neurology.org, reached through doi.org) returned HTTP 403 to automated requests. The check therefore rests on the publisher-deposited copies: the PMC article, the typeset article PDF in the PMC open-access collection (s3://pmc-oa-opendata/PMC7884983.1/PMC7884983.1.pdf), the Europe PMC full-text JATS XML and the PubMed abstract. All carry the passages quoted above; the PDF has the same abstract sentence ("A total of 347 participants completed both time points of evaluation") and the same Methods sentence ending "with 281 completing both follow-up evaluations". I read the Figure 1 (CONSORT) and Table 1 images from the PMC open-access copy.
- Supplements. None are attached. The JATS has no supplementary-material element, and the PMC page lists only Figures 1–4, Tables 1–2 and an author appendix table. The Data Availability statement says the data are available under data use agreements, so no public source data exist to check against. The paper's own counts are enough to settle the question.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has empty update-to, updated-by and relation fields. The PubMed record has no CommentsCorrections entries, and a Crossref title search under the journal's prefix found only the original article. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC7884983, including the Figure 1 and Table 1 images; the article PDF in the PMC open-access collection; Europe PMC full-text XML for PMC7884983; PubMed record 33177226; Crossref; Europe PMC; PubPeer; https://doi.org/10.1212/WNL.0000000000011089.
Suggested correction
In the abstract (Methods), replace
A total of 347 participants completed both time points of evaluation.
with
A total of 281 participants completed both time points of evaluation.
The authors might instead give all three counts, as the Methods do: "347 participants completed the 1-year evaluation, 348 the 5-year evaluation, and 281 both." That wording also shows readers of the abstract how much attrition there was between visits.
Recomputation script (python recompute.py.txt)
"""Recompute the sample sizes reported in PMC7884983
(Neurology 2021, doi:10.1212/WNL.0000000000011089,
"Comparison of Clinical Outcomes 1 and 5 Years Post-Injury Following Combat Concussion").
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract (Methods): "A total of 347 participants completed both time points of evaluation."
Methods, 2nd paragraph: "575 participants ... 347 of whom completed further clinical examination
at 1 year and 348 at 5 years postinjury, with 281 completing both follow-up evaluations."
Methods, Statistical Analysis: cross-sectional n = 348; longitudinal n = 281.
Results, 1st paragraph: 5-year completers 109 / 170 / 41 / 28 and both-visit completers
80 / 136 / 39 / 26 (nonblast controls / blast TBI / blast controls / nonblast TBI).
Table 1 (image) column headers: n = 109 / 41 / 170 / 28.
Figure 1 (CONSORT image): 1-year study totals 65 / 32 / 178 / 72 with group counts;
5-year study totals 57 / 31 / 164 / 96 with group counts and deaths (Study 2's death
occurred after its 5-year visit).
Results, longitudinal global outcome: nonblast controls 72% unchanged, 23% better, 5% worse.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
GROUPS = ["nonblast CTL", "blast TBI", "blast CTL", "nonblast TBI"]
# ---------------------------------------------------------------- inputs
ABSTRACT_BOTH = 347 # Abstract, Methods
METHODS_ENROLLED = 575 # Methods p2
METHODS_1Y, METHODS_5Y, METHODS_BOTH = 347, 348, 281 # Methods p2
STAT_CROSS, STAT_LONG = 348, 281 # Statistical Analysis
RES_5Y = {"nonblast CTL": 109, "blast TBI": 170, "blast CTL": 41, "nonblast TBI": 28} # Results p1
RES_BOTH = {"nonblast CTL": 80, "blast TBI": 136, "blast CTL": 39, "nonblast TBI": 26} # Results p1
TABLE1_N = {"nonblast CTL": 109, "blast CTL": 41, "blast TBI": 170, "nonblast TBI": 28} # Table 1 headers
# Figure 1, 1-year boxes: study total and group counts
FIG1_1Y = {
"Study 1": (65, {"blast CTL": 18, "blast TBI": 47}),
"Study 2": (32, {"blast TBI": 32}),
"Study 3": (178, {"nonblast CTL": 69, "blast CTL": 27, "nonblast TBI": 29, "blast TBI": 53}),
"Study 4": (72, {"nonblast CTL": 34, "blast TBI": 38}),
}
FIG1_1Y_TOTAL = 347
# Figure 1, 5-year boxes: study total, listed group counts, deaths before the 5-year visit by group.
# Study 1 box is headed "all blast TBI" for its 4 deaths; Study 3 lists "1 blast CTL, 1 blast TBI".
# Study 2 lists "Deceased (1) (made it to 5yr F/U)": that participant is one of its 31 and is not
# an additional death, so its pre-visit deaths are 0.
FIG1_5Y = {
"Study 1": (57, {"blast CTL": 18, "blast TBI": 35}, {"blast TBI": 4}),
"Study 2": (31, {"blast TBI": 31}, {}),
"Study 3": (164, {"nonblast CTL": 62, "blast CTL": 22, "nonblast TBI": 28, "blast TBI": 50},
{"blast CTL": 1, "blast TBI": 1}),
"Study 4": (96, {"nonblast CTL": 47, "blast TBI": 49}, {}),
}
FIG1_5Y_TOTAL = 348
GOSE_NBC = (72, 23, 5) # % unchanged, better, worse, nonblast controls
# ---------------------------------------------------------------- 1. totals
print("=== 1. Totals stated in the paper vs sums of the paper's own components ===")
s5 = sum(RES_5Y.values())
sb = sum(RES_BOTH.values())
print(f"Results 5-year group counts: {' + '.join(str(RES_5Y[g]) for g in GROUPS)} = {s5}"
f" (Methods 5-year n = {METHODS_5Y}; Statistical Analysis cross-sectional n = {STAT_CROSS})")
print(f"Table 1 column n: {' + '.join(str(v) for v in TABLE1_N.values())} = {sum(TABLE1_N.values())}")
print(f"Results both-visit group counts: {' + '.join(str(RES_BOTH[g]) for g in GROUPS)} = {sb}"
f" (Methods both n = {METHODS_BOTH}; Statistical Analysis longitudinal n = {STAT_LONG})")
f1 = sum(v[0] for v in FIG1_1Y.values())
print(f"Figure 1 1-year study totals: {' + '.join(str(v[0]) for v in FIG1_1Y.values())} = {f1}"
f" (Figure 1 label n = {FIG1_1Y_TOTAL}; Methods 1-year n = {METHODS_1Y})")
for s, (tot, g, *_rest) in FIG1_1Y.items():
assert sum(g.values()) == tot, s
print(" each 1-year study box: group counts sum to the box total -> True")
f5 = sum(v[0] for v in FIG1_5Y.values())
print(f"Figure 1 5-year study totals: {' + '.join(str(v[0]) for v in FIG1_5Y.values())} = {f5}"
f" (Figure 1 label n = {FIG1_5Y_TOTAL})")
print()
print("=== 2. The abstract's 'completed both' n ===")
gap = ABSTRACT_BOTH - sb
print(f"Abstract 'completed both' = {ABSTRACT_BOTH}; equals Methods 1-year n? {ABSTRACT_BOTH == METHODS_1Y}")
print(f"Abstract minus both-visit n = {ABSTRACT_BOTH} - {sb} = {gap}")
print(f"Relative overstatement = {gap}/{sb} = {100*gap/sb:.1f}%")
print(f"Upper bound on 'both' = min(1-year, 5-year) = min({METHODS_1Y}, {METHODS_5Y}) = {min(METHODS_1Y, METHODS_5Y)}")
print(f" 'both' = {ABSTRACT_BOTH} would require all {METHODS_1Y} 1-year completers to have a 5-year visit")
only1 = METHODS_1Y - METHODS_BOTH
only5 = METHODS_5Y - METHODS_BOTH
union = METHODS_1Y + METHODS_5Y - METHODS_BOTH
print(f"1-year only = {METHODS_1Y} - {METHODS_BOTH} = {only1} ({100*only1/METHODS_1Y:.1f}% of 1-year completers)")
print(f"5-year only = {METHODS_5Y} - {METHODS_BOTH} = {only5}")
print(f"Seen at 1 year or 5 years (union) = {union} of {METHODS_ENROLLED} enrolled")
print(f"Paired cohort as share of 1-year completers = {METHODS_BOTH}/{METHODS_1Y} = {100*METHODS_BOTH/METHODS_1Y:.1f}%")
print()
print("=== 3. Per-group check: both-visit count cannot exceed either visit's count ===")
fig1_1y_group = {g: 0 for g in GROUPS}
for _tot, g, *_ in FIG1_1Y.values():
for k, v in g.items():
fig1_1y_group[k] += v
for g in GROUPS:
one, five, both = fig1_1y_group[g], RES_5Y[g], RES_BOTH[g]
print(f" {g:13s} 1-year (Fig 1) {one:4d} | 5-year (Results) {five:4d} | both (Results) {both:4d}"
f" | both <= min? {both <= min(one, five)} | 1-year only {one - both:3d}")
print(f" sum of Figure 1 1-year group counts = {sum(fig1_1y_group.values())}")
print(f" sum of '1-year only' by group = {sum(fig1_1y_group[g] - RES_BOTH[g] for g in GROUPS)}")
print()
print("=== 4. Figure 1 5-year boxes reconcile with Table 1 and the Results ===")
fig1_5y_listed = {g: 0 for g in GROUPS}
fig1_5y_dead = {g: 0 for g in GROUPS}
for st, (tot, g, d) in FIG1_5Y.items():
for k, v in g.items():
fig1_5y_listed[k] += v
for k, v in d.items():
fig1_5y_dead[k] += v
lv, nd = sum(g.values()), sum(d.values())
print(f" {st}: box total {tot} | listed group counts {lv} | deaths before 5-year visit {nd}"
f" | {lv} + {nd} = {lv + nd} | equals box total? {tot == lv + nd}")
listed = sum(fig1_5y_listed.values())
dead = sum(fig1_5y_dead.values())
print(f"Listed group counts {listed} + deaths before the 5-year visit {dead} = {listed + dead}"
f" (Figure 1 label n = {FIG1_5Y_TOTAL}; Methods 5-year n = {METHODS_5Y})")
for g in GROUPS:
tot = fig1_5y_listed[g] + fig1_5y_dead[g]
print(f" {g:13s} Figure 1 listed {fig1_5y_listed[g]:4d} + deaths {fig1_5y_dead[g]} = {tot:4d}"
f" | Table 1 / Results = {RES_5Y[g]:4d} | agree? {tot == RES_5Y[g]}")
print(" -> Figure 1, Table 1 and the Results use the same 5-year counts (deaths before the visit included)")
print()
print("=== 5. Do the longitudinal GOS-E percentages identify the denominator? ===")
def splits(n, pcts):
out = []
for a in range(n + 1):
for b in range(n + 1 - a):
c = n - a - b
if all(abs(100 * k / n - p) <= 0.5 for k, p in zip((a, b, c), pcts)):
out.append((a, b, c))
return out
for n in (RES_BOTH["nonblast CTL"], RES_5Y["nonblast CTL"]):
print(f" nonblast CTL 72/23/5% with n = {n}: integer splits {splits(n, GOSE_NBC)}")
print(" -> both denominators fit, so the percentages cannot discriminate and are not used as evidence")
# ---------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.family": "DejaVu Sans", "font.size": 10,
"svg.fonttype": "none", "figure.facecolor": BG,
"axes.facecolor": BG, "savefig.facecolor": BG})
def save(fig, name):
fig.savefig(os.path.join(HERE, name + ".svg"))
fig.savefig(os.path.join(HERE, name + ".png"), dpi=200)
plt.close(fig)
# ---- Figure 1: abstract vs body text panels
fig = plt.figure(figsize=(7.2, 5.2))
ax = fig.add_axes([0, 0, 1, 1])
ax.set_xlim(0, 1)
ax.set_ylim(0, 1)
ax.axis("off")
def panel(x0, y0, w, h):
ax.add_patch(FancyBboxPatch((x0, y0), w, h, boxstyle="round,pad=0.006,rounding_size=0.012",
fc="white", ec="#c9c8c3", lw=1.0))
# left panel: abstract
LX, LW = 0.02, 0.35
panel(LX, 0.34, LW, 0.52)
ax.text(LX + 0.015, 0.835, "Abstract (Methods)", fontsize=11, weight="bold", color=INK, va="top")
ax.text(LX + 0.015, 0.775,
"“One-year and 5-year clinical\nevaluations included identical\nbatteries for neurobehavioral,\n"
"psychiatric, and cognitive\noutcomes. A total of 347\nparticipants completed both\n"
"time points of evaluation.”",
fontsize=10, color=INK2, va="top", linespacing=1.35)
ax.text(LX + 0.015, 0.40, f"{ABSTRACT_BOTH}", fontsize=22, weight="bold", color=ORANGE, va="center")
ax.text(LX + 0.12, 0.40, "completed both\n(reported, abstract)", fontsize=10, color=ORANGE, va="center")
# right panels: body
RX, RW = 0.565, 0.42
boxes = [
(0.70, 0.245, "Methods, 2nd paragraph",
"“...347 of whom completed further\nclinical examination at 1 year and\n348 at 5 years postinjury, with 281\ncompleting both follow-up\nevaluations.”"),
(0.48, 0.19, "Methods, Statistical Analysis",
"“...longitudinal analysis only\nconsidered those who completed both\nthe 1-year and 5-year evaluations\n(n = 281).”"),
(0.155, 0.28, "Results, 1st paragraph",
"“...80 nonblast controls, 136 patients\nwith concussive blast TBI, 39 blast\ncontrols, and 26 patients with\nnonblast TBI completed both 1- and\n5-year follow-up.”"),
]
for y0, h, title, body in boxes:
panel(RX, y0, RW, h)
ax.text(RX + 0.012, y0 + h - 0.02, title, fontsize=10, weight="bold", color=INK, va="top")
ax.text(RX + 0.012, y0 + h - 0.058, body, fontsize=10, color=INK2, va="top", linespacing=1.25)
ax.text(RX + 0.012, 0.18, f"80+136+39+26 = {sb} (recomputed)", fontsize=10, weight="bold",
color=BLUE, va="center")
def arrow(p0, p1, color, rad=0.0):
ax.add_patch(FancyArrowPatch(p0, p1, arrowstyle="-|>", mutation_scale=12, lw=2, color=color,
connectionstyle=f"arc3,rad={rad}"))
start = (LX + LW + 0.008, 0.43)
arrow(start, (RX - 0.008, 0.85), INK2, rad=0.2)
ax.text(0.468, 0.905, "same number as\nthe 1-year count\n(347 at 1 year)", fontsize=10,
color=INK2, ha="center", va="center")
arrow(start, (RX - 0.008, 0.575), ORANGE, rad=0.0)
arrow(start, (RX - 0.008, 0.25), ORANGE, rad=-0.1)
ax.text(0.468, 0.235, "conflict:\n347 vs 281\ncompleted both\n(66 more,\n+23.5%)", fontsize=10,
weight="bold", color=INK, ha="center", va="center")
ax.text(0.02, 0.06,
"Orange: value reported in the abstract. Blue: sum recomputed from the\n"
"paper's own group counts. Quotes are verbatim from the PMC-archived\n"
"article; “...” marks omitted text.",
fontsize=10, color=INK2, va="center", linespacing=1.3)
save(fig, "fig1")
# ---- Figure 2: dot comparison of reported vs recomputed counts
rows = [
("Completed 1-year visit", METHODS_1Y, f1, "Methods; Figure 1 boxes 65+32+178+72"),
("Completed 5-year visit", METHODS_5Y, s5, "Methods; Results 109+170+41+28"),
("Completed both (body)", METHODS_BOTH, sb, "Methods; Results 80+136+39+26"),
("Completed both (abstract)", ABSTRACT_BOTH, sb, "Abstract vs Results 80+136+39+26"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.3))
fig.subplots_adjust(left=0.27, right=0.97, top=0.86, bottom=0.17)
ys = list(range(len(rows)))[::-1]
for y, (lab, rep, rec, src) in zip(ys, rows):
if rep != rec:
ax.annotate("", xy=(rec + 3, y), xytext=(rep - 3, y),
arrowprops=dict(arrowstyle="-", lw=2, color=INK2))
ax.text((rep + rec) / 2, y + 0.2, f"{rep - rec:+d}", ha="center", va="bottom", fontsize=10,
weight="bold", color=INK)
ax.plot(rep, y, "o", ms=10, mfc=ORANGE, mec=ORANGE, zorder=3)
ax.plot(rec, y, "o", ms=10, mfc="none", mec=BLUE, mew=2, zorder=4)
ax.text(10, y - 0.28, src, fontsize=10, color=INK2, va="center")
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], color=INK)
ax.set_xlim(0, 420)
ax.set_ylim(-0.6, len(rows) - 0.3)
ax.set_xlabel("Participants (n)", color=INK)
ax.grid(axis="x", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#b5b4af")
ax.tick_params(colors=INK2, length=0)
handles = [Line2D([], [], ls="none", marker="o", ms=10, mfc=ORANGE, mec=ORANGE,
label="reported in the paper"),
Line2D([], [], ls="none", marker="o", ms=10, mfc="none", mec=BLUE, mew=2,
label="recomputed from the paper's own components")]
ax.legend(handles=handles, loc="lower left", bbox_to_anchor=(-0.36, 1.0), ncol=2, frameon=False,
fontsize=10, handletextpad=0.3, columnspacing=1.5)
save(fig, "fig2")
print()
print("figures written: fig1.svg, fig1.png, fig2.svg, fig2.png")
Abstract's 95% interval for Madagascar vaccination access (78–81%) excludes its 77% estimate; Table 3 gives 76–78%
PLoS Med (2011) · doi:10.1371/journal.pmed.1001110 · PMC3201935
Summary
The abstract gives the proportion of Madagascar's population ever covered by any measles vaccination activity (the model parameter ρ) as "77% (95% CI: 78, 81)". The interval does not contain its own point estimate. The paper reports its estimates as posterior medians with 95% intervals taken from posterior quantiles, and with that method the median always lies inside the interval. Table 3 reports the same parameter as 77% (95% CI: 76, 78), and the Results' complementary figure, 23% (95% CI: 24, 22) never accessible, converts back to exactly 76 to 78. The abstract interval is therefore inconsistent with the rest of the paper. The point estimate, the ranking of the three countries and the conclusions are unaffected, but anyone who takes the uncertainty from the abstract gets an interval shifted upward by 2 to 3 points.
What the paper reports
Abstract, Methods and Findings:
We estimate that 93% (95% CI: 91, 94) of the population in Ghana was ever covered by any measles vaccination activity, 77% (95% CI: 78, 81) in Madagascar, and 69% (95% CI: 67, 70) in Sierra Leone.
Table 3, "Estimated proportion of the population accessible by any vaccination activity (ρ), within-activity inefficiency (ψ), and routine vaccination opportunity rate (λ) for Ghana, Madagascar, and Sierra Leone." (columns Country | ρ | ψ | λ):
Ghana | 93% (95% CI: 91, 94) | 0.03 (95% CI: 0.02, 0.04) | 0.66 (95% CI: 0.54, 0.82)
Madagascar | 77% (95% CI: 76, 78) | 0.34 (95% CI: 0.28, 0.41) | 0.60 (95% CI: 0.49, 0.73)
Sierra Leone | 69% (95% CI: 67, 70) | 0.33 (95% CI: 0.31, 0.39) | 0.47 (95% CI: 0.38, 0.61)
Results, first paragraph:
We estimate that 7% (95% CI: 6, 9) of children in Ghana, 23% (95% CI: 24, 22) of children in Madagascar, and 31% (95% CI: 33, 30) of children in Sierra Leone were never accessible by routine measles vaccination or campaigns (Table 3).
Methods, "Vaccination probability and coverage", defining ρ:
Let us assume that there exists some population of size 1-ρ that never has a chance of being vaccinated in any activity, which we term the inaccessible population.
Methods, "Application to Measles Vaccination", on how estimates and intervals were obtained:
Reported parameter estimates are the medians of the posterior distributions (posteriors were normally distributed on the scale used in estimation), and 95% confidence intervals from quantiles of the posterior distribution.
Editors' Summary, "What Did the Researchers Do and Find?" (relevant to a side note below):
The researchers estimate that 93%, 77%, and 65% of the target populations in Ghana, Madagascar, and Sierra Leone, respectively, were ever covered by any vaccination activity
What the paper's own numbers show
Method. For each country I checked whether each printed interval contains its point estimate, compared the abstract with Table 3, and converted the Results' "never accessible" figures (1 − ρ) back to ρ by subtracting each value and bound from 100. The posterior samples are not provided, so the interval cannot be re-estimated from data; the checks use only numbers printed in the paper. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Madagascar ρ: does the interval contain the estimate? | Abstract | 77% (95% CI: 78, 81) | No. 77 is 1 point below the lower bound 78 |
| Madagascar ρ: does the interval contain the estimate? | Table 3 | 77% (95% CI: 76, 78) | Yes |
| Madagascar ρ: abstract vs Table 3 | Abstract; Table 3 | (78, 81) vs (76, 78) | Not identical; widths 3 and 2 points |
| Madagascar ρ implied by 1 − ρ | Results, paragraph 1 | 23% (95% CI: 24, 22) never accessible | 100 − 23 = 77, bounds 100 − 24 = 76 and 100 − 22 = 78: 77% (76, 78), identical to Table 3 |
| Never accessible implied by the abstract interval | Abstract vs Results | 23% (95% CI: 24, 22) | The abstract interval would imply 23% (19, 22), which also excludes 23 |
| Ghana ρ: abstract, Table 3, 100 − Results | all three | 93 (91, 94); 93 (91, 94); 7 (6, 9) | All three give 93% (91, 94) |
| Sierra Leone ρ: abstract, Table 3, 100 − Results | all three | 69 (67, 70); 69 (67, 70); 31 (33, 30) | All three give 69% (67, 70) |
| Sierra Leone ρ, point estimate (side note) | Editors' Summary | 65% | Abstract, Results and Table 3 give 69% |
For Ghana and Sierra Leone the abstract, Table 3 and the Results agree exactly. For Madagascar, Table 3 and the Results agree exactly, and the abstract interval is the only one of the nine printed intervals for ρ or 1 − ρ that excludes its estimate. A posterior median lies between the posterior's 2.5% and 97.5% quantiles by definition, so under the method the paper describes, "77% (95% CI: 78, 81)" cannot be a correct pair.
Figure
Figure 1. Madagascar's estimated proportion ever accessible to measles vaccination, ρ, with its 95% interval, as printed in three places. Orange rows are printed in the paper: the abstract (circle, 77% with 78 to 81) and Table 3 (square, 77% with 76 to 78). The blue row (diamond) is recomputed: 100 minus the Results' "never accessible" estimate of 23% (24, 22), which gives 77% (76, 78). The dashed vertical line marks the point estimate, 77%, which is the same in all three places. Only the abstract interval fails to contain it. The axis spans 73 to 86% so that one-point differences are visible; there is no null value for this quantity.
Why it matters
- The abstract is where the headline uncertainty is read. The abstract reports one ρ estimate per country, and this is the only Madagascar interval it gives. Someone who takes values from the abstract alone, for example for a review of coverage estimates or to set a prior on the accessible population in a measles transmission model, would record 78 to 81%, which is 2 to 3 points higher at each bound than the 76 to 78% in Table 3 and lies entirely above the point estimate.
- The point estimate and conclusions are unaffected. The 77% estimate is the same in the abstract, the Editors' Summary, the Results and Table 3, and Table S1 gives the base-model ρ for Madagascar as 0.77. With the correct interval, Madagascar (76, 78) still does not overlap Ghana (91, 94) or Sierra Leone (67, 70), so the ordering of the three countries holds. About 23% of Madagascar's children were estimated never to be reachable by any measles vaccination activity. No significance statement or comparison in the paper depends on the abstract interval.
- The error is detectable but easy to miss. The interval is visibly impossible to a reader who checks it against the estimate, and the correct values are in Table 3. Readers who copy numbers without checking containment, or automated extraction from abstracts, would not catch it. The paper has been cited about 62 times.
Explanations considered
- A different quantity or population. The abstract sentence and Table 3 both describe ρ, the proportion "ever covered" or "accessible by any vaccination activity", for the same Madagascar DHS 2008–2009 sample (9,747 children in Table 2), the same model and the same 95% level. Nothing in the paper indicates a different definition, time point, subgroup or adjustment. No Madagascar estimate anywhere in the paper has a 78 to 81 interval. Does not account for it.
- An asymmetric or one-sided interval. The Methods state that intervals are posterior quantiles around the posterior median. Any quantile-based interval, symmetric or not, contains the median. An interval that excludes its own estimate cannot be explained by asymmetry. Does not account for it.
- Rounding. Rounding to whole percentages is monotone: if the unrounded median is at least the unrounded 2.5% quantile, the rounded median is at least the rounded lower bound. The abstract has the estimate 1 point below the lower bound. Does not account for it.
- A different model specification in the abstract. Table S1 compares model variants and gives Madagascar's base-model ρ as 0.77 (point estimate only). The paper does not report an alternative model whose Madagascar interval is 78 to 81, and the abstract sentence pairs 77% with the interval as a single estimate. Not supported by anything in the paper.
- A PMC conversion or typesetting artifact. The publisher's HTML page, the publisher's PDF, the PMC and Europe PMC full text, and the French and English abstracts in the supplementary file Alternative Language Abstract S1 all print "77% (95% CI: 78, 81)". Does not account for it; the text is in the version of record.
- The Results bounds are in reversed order. The Results print 1 − ρ intervals as (24, 22) and (33, 30), with the upper bound first. This follows from subtracting Table 3's bounds in order from 100 and is itself evidence that the Results values were derived from Table 3's (76, 78). It does not create an inconsistency.
The most economical reading is a transcription slip in the abstract interval, with 76, 78 intended. The paper does not say how the abstract figure was produced, so this is an inference from the paper's other numbers.
Status
As of 2026-10-24:
- Version of record. The publisher PDF (journals.plos.org printable version) and the publisher HTML page both give the abstract as "77% (95% CI: 78, 81) in Madagascar". The PDF's Table 3 gives "77% (95% CI: 76, 78)" and its Results give "23% (95% CI: 24, 22)". The publisher HTML shows Table 3 only as an image. PMC and Europe PMC full text match the PDF.
- Supplements. The publisher lists eight supplementary files. Alternative Language Abstract S1 (French abstract, with the English abstract) repeats "77% (95% CI: 78, 81)". Table S1 gives Madagascar's base-model ρ as 0.77 without an interval. Text S1 contains derivations with no Madagascar values. No source data or posterior samples are provided. The supplements do not resolve the discrepancy.
- Corrections. No correction, erratum, retraction or expression of concern was found. Crossref lists no updates and no relations; PMC and Europe PMC list no correction notices; the PLOS article page shows no correction and "Reader Comments (0)". Web searches found no correction notice or coverage.
- PubPeer. The PubPeer API returned errors (404, then a Cloudflare challenge), and a web search found no PubPeer thread for this DOI; the absence of a thread could not be confirmed directly.
- Sources. Article as archived in PMC (PMC3201935); publisher page https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1001110 and its PDF and supplementary files; Crossref and Europe PMC records for doi:10.1371/journal.pmed.1001110.
Suggested correction
In the Abstract, Methods and Findings, replace "77% (95% CI: 78, 81) in Madagascar" with "77% (95% CI: 76, 78) in Madagascar", matching Table 3 and the Results. The source of the interval cannot be confirmed without the posterior samples, so the authors may wish to confirm it from their MCMC output.
Separately, the Editors' Summary gives Sierra Leone's proportion ever covered as 65%, while the abstract, Results and Table 3 give 69% (95% CI: 67, 70). If 69% is correct, "93%, 77%, and 65%" should read "93%, 77%, and 69%".
Recomputation script (python recompute.py.txt)
"""Recompute the consistency checks for PMC3201935
(PLoS Med 2011, doi:10.1371/journal.pmed.1001110,
"Measuring the Performance of Vaccination Programs Using Cross-Sectional Surveys:
A Likelihood Framework and Retrospective Analysis").
Run: python recompute.py.txt
Needs matplotlib. Prints every recomputed number used in writeup.md and writes
fig1.svg / fig1.png next to this file.
Inputs are copied from the paper as archived in PMC (identical in the publisher PDF):
Abstract, Methods and Findings: "93% (95% CI: 91, 94) ... Ghana ... 77% (95% CI: 78, 81)
in Madagascar, and 69% (95% CI: 67, 70) in Sierra Leone"
Table 3, column rho: Ghana 93% (91, 94); Madagascar 77% (76, 78); Sierra Leone 69% (67, 70)
Results, paragraph 1 ("never accessible", i.e. 1 - rho): Ghana 7% (6, 9);
Madagascar 23% (24, 22); Sierra Leone 31% (33, 30)
Editors' Summary: "93%, 77%, and 65%" ever covered (Ghana, Madagascar, Sierra Leone)
Methods: estimates are posterior medians; 95% CIs are posterior quantiles.
"""
import os
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ---------------------------------------------------------------- inputs (percent)
ABSTRACT = {"Ghana": (93, 91, 94), "Madagascar": (77, 78, 81), "Sierra Leone": (69, 67, 70)}
TABLE3 = {"Ghana": (93, 91, 94), "Madagascar": (77, 76, 78), "Sierra Leone": (69, 67, 70)}
NEVER = {"Ghana": (7, 6, 9), "Madagascar": (23, 24, 22), "Sierra Leone": (31, 33, 30)} # as printed
EDITORS = {"Ghana": 93, "Madagascar": 77, "Sierra Leone": 65}
def inside(p, lo, hi):
return min(lo, hi) <= p <= max(lo, hi)
# ---------------------------------------------------------------- 1. containment
print("=== 1. Does each interval contain its own point estimate? ===")
for src, d in (("Abstract", ABSTRACT), ("Table 3", TABLE3)):
for c, (p, lo, hi) in d.items():
print(f"{src:9s} {c:13s} {p}% ({lo}, {hi}) contains point: {inside(p, lo, hi)}")
p, lo, hi = ABSTRACT["Madagascar"]
print(f"Madagascar abstract: point {p} lies {lo - p} point below the lower bound {lo}")
print(f"Interval widths, Madagascar: abstract {hi - lo} points, Table 3 "
f"{TABLE3['Madagascar'][2] - TABLE3['Madagascar'][1]} points")
# ---------------------------------------------------------------- 2. abstract vs Table 3
print("\n=== 2. Abstract vs Table 3 (same quantity rho) ===")
for c in ABSTRACT:
print(f"{c:13s} abstract {ABSTRACT[c]} Table 3 {TABLE3[c]} identical: {ABSTRACT[c] == TABLE3[c]}")
# ---------------------------------------------------------------- 3. complement from Results
print("\n=== 3. rho implied by the Results 'never accessible' figures (100 - value) ===")
IMPLIED = {}
for c, (p, a, b) in NEVER.items():
q = (100 - p, min(100 - a, 100 - b), max(100 - a, 100 - b))
IMPLIED[c] = q
print(f"{c:13s} never {p}% ({a}, {b}) -> rho {q[0]}% ({q[1]}, {q[2]}); "
f"equals Table 3: {q == TABLE3[c]}; equals abstract: {q == ABSTRACT[c]}")
ab = ABSTRACT["Madagascar"]
print(f"Complement the abstract interval would imply for 'never accessible': "
f"{100 - ab[0]}% ({100 - ab[2]}, {100 - ab[1]}) vs Results 23% (22, 24)")
# ---------------------------------------------------------------- 4. side note
print("\n=== 4. Editors' Summary point estimates vs Table 3 ===")
for c in EDITORS:
print(f"{c:13s} Editors' Summary {EDITORS[c]}% Table 3 {TABLE3[c][0]}% match: {EDITORS[c] == TABLE3[c][0]}")
# ---------------------------------------------------------------- figure
BG, TXT, SEC = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "svg.fonttype": "none"})
rows = [
("Abstract (printed)", ABSTRACT["Madagascar"], ORANGE, "o", "77% (95% CI: 78, 81)"),
("Table 3 (printed)", TABLE3["Madagascar"], ORANGE, "s", "77% (95% CI: 76, 78)"),
("100 − Results “never accessible”\n23% (24, 22) (recomputed)", IMPLIED["Madagascar"], BLUE, "D",
"77% (76, 78)"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.4), facecolor=BG)
ax.set_facecolor(BG)
ys = [2, 1, 0]
for y, (lab, (pt, lo, hi), col, mk, txt) in zip(ys, rows):
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
for e in (lo, hi):
ax.plot([e, e], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
ax.plot(pt, y, marker=mk, ms=9, color=col, mec=col, zorder=4)
ax.text(max(hi, pt) + 0.4, y, txt, va="center", ha="left", color=col, fontsize=10.5)
pt = ABSTRACT["Madagascar"][0]
ax.axvline(pt, color=SEC, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(pt, 2.6, "point estimate 77%", ha="center", va="bottom", color=SEC, fontsize=10)
ax.text(79.5, 1.72, "the interval 78\u201381 excludes its own estimate", ha="center", va="top",
color=TXT, fontsize=10)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows], color=TXT)
ax.set_ylim(-0.5, 3.0)
ax.set_xlim(73, 86)
ax.set_xticks(range(73, 87, 1))
ax.set_xticklabels([str(v) if v in (75, 76, 77, 78, 80, 81, 85) else "" for v in range(73, 87)])
ax.set_xlabel("Madagascar, \u03c1: % ever accessible to measles vaccination",
color=TXT)
ax.grid(axis="x", color="#e4e3df", lw=0.6, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(SEC)
ax.tick_params(axis="both", colors=SEC, labelcolor=TXT, length=3)
ax.tick_params(axis="y", length=0)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg and fig1.png")
Two p-values for the recent-adversity estimates disagree with their own confidence intervals and forest plot
Psychol Med (2026) · doi:10.1017/S0033291726104632 · PMC13161794
Summary
The abstract and Results give two adjusted odds ratios for recent adversity from the joint-exposure model, which the paper names as its primary inferential model: aOR 1.91 (95% CI 1.47–2.49) without parental support, with p = .011, and aOR 1.76 (95% CI 1.17–2.65) with parental support, with p < .001. Recomputed from each printed aOR and CI, the p-values are about 1.5 × 10−6 and .0067. The paper's own forest plot for the same model prints "< .001" and "0.007", which agrees with the recomputation and not with the text. Both estimates stay significant at .05, so no conclusion is reversed. But the abstract presents the weaker of the two associations as the better established one.
What the paper reports
Abstract, Results:
Elevated odds from recent adversities (aOR = 1.91, 95% CI 1.47–2.49, p = .011) attenuated by parental support (aOR = 1.76, 95% CI 1.17–2.65, p < .001).
Results, "Temporal effects of adverse life events and parental support buffer", second paragraph (final model):
Compared to adolescents with no recent adversity and no parental support, those who experienced recent adversity but had no parental support had significantly higher PE odds (aOR = 1.91, 95% CI 1.47–2.49, p = .011). Among those with high parental support, recent adversity remained associated with elevated odds of PEs compared with those with no recent adversity and high parental support, although the magnitude of association was modestly attenuated (aOR = 1.76, 95% CI 1.17–2.65, p < .001).
Results, "Multivariable logistic regression modeling", first paragraph:
The joint exposure model demonstrated slightly improved explanatory value and is presented as the primary inferential model.
Methods, "Statistical analysis":
Adjusted odds ratios (aORs) with 95% confidence intervals were reported, and robust standard errors were applied.
Discussion, "Temporal effect of adversity and parental support buffer":
However, the magnitude of association was modestly attenuated in the presence of parental support (aOR 1.76 vs 1.91), suggesting partial buffering.
Forest plot of the joint-exposure model (an image, so not checked with the text tool; read from both the PMC and the publisher copies). The rows are "Adversities within 30 days and no parental support", printed as 1.92 (1.47 - 2.50), p < .001, and "Adversities within 30 days with parental support", printed as 1.76 (1.17 - 2.65), p 0.007.
What the paper's own numbers show
Method. For a logistic-regression aOR with a 95% Wald CI, SE(ln aOR) = (ln U − ln L) / (2 × 1.96), z = ln(aOR) / SE, and the two-sided p = 2 × (1 − Φ(|z|)). The paper used robust standard errors. The same robust SE gives both the CI and the p-value, so the check needs no other input. To allow for the two-decimal rounding of the CI, I also computed p for every CI whose bounds round to the printed ones. As a calibration, I applied the same method to eleven other estimates printed in the text. Their printed p-values agree with the recomputed ones to rounding, apart from one smaller side issue noted below. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| p, recent adversity without parental support | Abstract; Results | aOR 1.91 (1.47–2.49), p = .011 | SE 0.134, z = 4.81, p = 1.5 × 10−6. The range over CI rounding is 1.1 to 1.8 × 10−6. |
| p, recent adversity with parental support | Abstract; Results | aOR 1.76 (1.17–2.65), p < .001 | SE 0.209, z = 2.71, p = .0067. The range over CI rounding is .0062 to .0072. |
| Same two estimates in the joint-model forest plot | Forest plot image | 1.92 (1.47–2.50), p < .001; 1.76 (1.17–2.65), p = 0.007 | z = 4.82, p = 1.5 × 10−6; z = 2.71, p = .0067. Both agree with the plot. |
| CI that p = .011 would need, given aOR 1.91 | derived | 1.47–2.49 | SE 0.254 (1.89 times the printed CI's SE); 95% CI 1.16–3.15 |
| Widest CI that p < .001 allows, given aOR 1.76 | derived | 1.17–2.65 | SE below 0.172; 95% CI no wider than 1.26–2.46 |
| Recent adversity, main-effects model | Results | 1.86 (1.49–2.34), p < .001 | z = 5.39, p = 7 × 10−8. Consistent. |
| Undisclosed gender, SDQ ≤ 16 | Abstract; Results | 4.36 (1.50–12.66), p = .007 | p = .0068. Consistent. |
| Home safety, final model | Abstract; Results | 0.81 (0.66–0.99), p = .038 | p = .040 to .042. Consistent within CI rounding. |
| Financial hardship, final model | Results | 1.38 (1.07–1.79), p = .013 | p = .013 to .014. Consistent. |
| Low wellbeing, high school support | Results | 0.77 (0.61–0.98), p = .031 | p = .031 to .033. Consistent. |
| Side issue: high wellbeing, low school support | Results | 0.73 (0.57–0.95), p = .008 | p = .016 to .019. The forest plot prints 0.74 (0.57–0.96), p = .019. |
The log-scale midpoint of each printed CI matches its aOR: the geometric means are 1.913 and 1.761. So each aOR and its CI form a consistent pair. Only the p-values disagree with them, and they disagree in opposite directions. The printed p = .011 understates the evidence for the no-support estimate by a factor of about 7,000. The printed p < .001 overstates the evidence for the supported estimate: its own CI gives about .007. No CI that rounds to 1.17–2.65 can produce p < .001, because that p would need a lower bound of at least 1.26.
Figure
Figure 1. Two-sided p-values on a log scale, with stronger evidence to the right. Orange squares show the p-values printed in the abstract and Results. The arrow marks that "p < .001" is an upper bound. Blue circles show the p-values recomputed from the printed aOR and 95% CI with a log-scale Wald test. The short blue line is the range over all CIs that round to the printed bounds; for the second estimate it is narrower than the marker. Dashed lines mark p = .05 and p = .001. For the estimate without parental support, the printed .011 lies more than three orders of magnitude from the recomputed 1.5 × 10−6. For the estimate with parental support, the printed "< .001" lies on the wrong side of .001 from the recomputed .0067. The paper's own forest plot prints p < .001 and p = 0.007 for these two rows, in line with the blue values.
Figure 2. Adjusted odds ratios on a log scale. The thin vertical line is aOR = 1. Orange shows the printed aOR and 95% CI. Blue shows the 95% CI that the printed p-value would require, with the aOR held fixed. For p = .011, the interval around 1.91 would have to be 1.16–3.15, nearly twice as wide on the log scale as the printed 1.47–2.49. For p < .001, the interval around 1.76 could be no wider than 1.26–2.46, but the printed interval is 1.17–2.65. The printed CIs therefore cannot carry the printed p-values.
Why it matters
- The error is in the abstract and in the paper's primary model. The two estimates support the parental-support "buffer" message that the title ("Mitigating") and the Discussion highlight. A reader of the abstract alone sees p = .011 for the unbuffered association and p < .001 for the buffered one. That ordering is the reverse of what the paper's own intervals and forest plot show.
- No significance decision changes. Both associations are significant at .05 whichever p-values are used. The aORs and CIs, which carry the effect sizes, agree with each other, and they agree with the forest plot to within 0.01. The paper's conclusions that recent adversity is associated with PEs with or without parental support still hold. The no-support association is in fact better supported than the text states.
- The buffering claim rests on the point estimates, not on these p-values. The Discussion compares aOR 1.76 with 1.91. Neither p-value tests that difference. The two CIs overlap almost entirely, and the paper reports no test of the difference. The corrected p-values therefore neither add to nor take away from the buffering interpretation. They only remove an apparent sign that the supported group's estimate is the more firmly established one.
- Readers may reuse the numbers. Meta-analysts and readers who take p-values from abstracts would record the wrong strength of evidence for both estimates.
Explanations considered
- Rounding of the printed CI or aOR. Ruled out. Over every CI that rounds to 1.47–2.49, p ranges from 1.1 to 1.8 × 10−6. Over every CI that rounds to 1.17–2.65, p ranges from .0062 to .0072. Neither range comes near the printed value.
- Robust (sandwich) standard errors. Ruled out. The robust SE determines the CI as well as the p-value, so a Wald p from the printed robust CI already reflects it.
- Clustering by school or a t reference distribution. Ruled out. A t distribution with 3,700 df gives p = 1.5 × 10−6. Even with 39 df, as if the 40 schools were the units, p = 1.4 × 10−5. Any adjustment of this kind would also have widened the printed CI.
- A CI at another level. Ruled out. If 1.47–2.49 were a 90% CI, p would be 5 × 10−5. If it were a 99% CI, p would be even smaller. No conventional level yields .011.
- A Wald test on the odds-ratio scale, or a one-sided test. Ruled out. A linear-scale Wald test gives p = 4.7 × 10−4, and a one-sided log-scale test gives 7 × 10−7. Neither gives .011, and neither would explain the second estimate.
- The p-values swapped between the two estimates. Does not fully account for it. Exchanging them would make the first estimate correct ("< .001"), but the second would then read .011, while its CI gives .0067 and the forest plot prints 0.007.
- The value taken from the main-effects model. Ruled out. The main-effects estimate for recent adversity is 1.86 (1.49–2.34), printed and recomputed as p < .001.
- A text-conversion artifact. Ruled out. These values sit in running text, not in a table, with italic p and en-dash CIs. The publisher's HTML page, the Europe PMC JATS XML and the PMC archive all carry the same characters. A lost minus sign, a misread superscript or a shifted table cell cannot produce them.
- The text is right and the figure is wrong. Not supported. The forest-plot p-values agree with the CIs printed in both the text and the figure. The text p-values agree with neither.
- Side observations, not part of the main finding. (1) In both the PMC and the publisher copies, the image placed under the caption for Figure 1 ("main-effect" model) shows the joint-exposure estimates. The image under the caption for Figure 2 ("joint exposure" model) shows the main-effects estimates. The captions and images appear to be exchanged. (2) Several text aORs differ from the plotted values by 0.01, for example 1.91 vs 1.92 and 4.47 vs 4.48. (3) The forest plot lists a single reference row ("No adversities within 30 days and no parental support"), with "No adversities within 30 days with parental support" at 0.99 (0.83–1.19). The text, however, describes the 1.76 as relative to "no recent adversity and high parental support". If the 1.76 is against the common reference, the within-support contrast is 1.76 / 0.99 = 1.78; the conclusions do not change. (4) The text gives 0.73 (0.57–0.95), p = .008, for high wellbeing with low school support, but that CI implies p ≈ .016 to .019. The plotted 0.74 (0.57–0.96) is printed with p = .019 and recomputes to about .023.
Status
- Version of record. As of 2026-10-24, the publisher's page on Cambridge Core (reached through doi.org) carries the same values in the abstract and in the Results paragraph quoted above, as do the Europe PMC full-text XML and the PMC archive. Every quotation above was checked verbatim against the PMC-archived text with
src/show.py. The forest-plot values were read from both forest-plot images in the PMC copy, and the same values appear in the publisher's high-resolution image. - Supplements. The article has no supplementary material. The JATS XML has no supplementary-material element, and the Cambridge Core page data list none. The forest plots are the only other place where these estimates appear, and they resolve the discrepancy as described above.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref updates, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by field and an empty relation, and the Cambridge Core page data list empty corrections and retractions. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC13161794 and its two figure images; Europe PMC full-text XML; Cambridge Core article page via https://doi.org/10.1017/S0033291726104632; Crossref; PubPeer.
Suggested correction
In the abstract, replace
Elevated odds from recent adversities (aOR = 1.91, 95% CI 1.47–2.49, p = .011) attenuated by parental support (aOR = 1.76, 95% CI 1.17–2.65, p < .001).
with
Elevated odds from recent adversities (aOR = 1.91, 95% CI 1.47–2.49, p < .001) attenuated by parental support (aOR = 1.76, 95% CI 1.17–2.65, p = .007).
and make the same two changes in the second paragraph of the Results section "Temporal effects of adverse life events and parental support buffer". These are the values in the paper's own forest plot for the joint-exposure model. The authors may also wish to check:
- the p-value for high wellbeing with low school support (text p = .008; the CI implies about .016 to .019; the plot gives .019);
- the reference category stated for the 1.76 estimate;
- the 0.01 differences between the text and plotted aORs;
- the apparent exchange of the two forest-plot images between the Figure 1 and Figure 2 captions.
Recomputation script (python recompute.py.txt)
"""Recompute the p-values of the recent-adversity x parental-support estimates in PMC13161794
(Psychol Med 2026, doi:10.1017/S0033291726104632,
"Safe and supported: Mitigating psychotic experiences in Irish adolescents - A population-based study").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md and
writes fig1.svg/png and fig2.svg/png next to this file.
Method: Wald test on the log-odds scale, the standard construction for a logistic-regression
aOR and its 95% CI. SE = (ln U - ln L) / (2 x 1.959964); z = ln(aOR) / SE; p = 2 x (1 - Phi(|z|)).
The paper states that robust standard errors were used; the same robust SE produces both the
CI and the p-value, so it cancels out of this check.
Inputs are copied from the paper as archived in PMC (abstract, Results text) and from the
forest plot of the joint-exposure model (the image archived as S0033291726104632_fig1.jpg).
"""
import os
import numpy as np
from scipy.stats import norm, t
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z975 = norm.isf(0.025)
def wald(o, l, u, centre="or"):
"""SE, z, two-sided p for an odds ratio o with 95% CI (l, u)."""
se = (np.log(u) - np.log(l)) / (2 * Z975)
est = np.log(o) if centre == "or" else (np.log(l) + np.log(u)) / 2
z = est / se
return se, z, 2 * norm.sf(abs(z))
def p_window(l, u, fn):
"""min or max p over every CI whose bounds round to (l, u) at 2 dp, centred on the CI."""
vals = []
for ll in np.linspace(l - 0.005, l + 0.005, 41):
for uu in np.linspace(u - 0.005, u + 0.005, 41):
vals.append(wald(np.sqrt(ll * uu), ll, uu)[2])
return fn(vals)
def fmt_p(p):
return f"{p:.2e}" if p < 1e-3 else f"{p:.4f}"
# ------------------------------------------------------------- 1. the two estimates at issue
print("=== 1. Joint-exposure model, recent adversity (past 30 days) x parental support ===")
TEXT = { # abstract and Results text (identical in both places)
"A": ("Recent adversity, no parental support", 1.91, 1.47, 2.49, ".011", 0.011),
"B": ("Recent adversity, with parental support", 1.76, 1.17, 2.65, "< .001", 0.001),
}
FIG = { # forest plot of the joint-exposure model
"A": (1.92, 1.47, 2.50, "< .001"),
"B": (1.76, 1.17, 2.65, ".007"),
}
R = {}
for k, (lab, o, l, u, prep, pnum) in TEXT.items():
se, z, p = wald(o, l, u)
se_m, z_m, p_m = wald(o, l, u, centre="mid")
pmin, pmax = p_window(l, u, min), p_window(l, u, max)
fo, fl, fu, fp = FIG[k]
_, fz, fpp = wald(fo, fl, fu)
R[k] = dict(lab=lab, o=o, l=l, u=u, prep=prep, pnum=pnum, se=se, z=z, p=p,
pmin=pmin, pmax=pmax, fig=(fo, fl, fu, fp), fz=fz, fp=fpp)
print(f"{k}. {lab}")
print(f" text: aOR {o} (95% CI {l}-{u}), reported p {prep}")
print(f" ln aOR = {np.log(o):.3f}; ln CI = {np.log(l):.3f} to {np.log(u):.3f}; "
f"CI midpoint on log scale = {(np.log(l)+np.log(u))/2:.3f} (geometric mean {np.sqrt(l*u):.3f})")
print(f" SE(ln aOR) = {se:.3f}; z = {z:.2f}; p = {fmt_p(p)} "
f"(centred on CI midpoint: z = {z_m:.2f}, p = {fmt_p(p_m)})")
print(f" p over all CIs that round to {l}-{u}: {fmt_p(pmin)} to {fmt_p(pmax)}")
print(f" figure: aOR {fo:.2f} ({fl:.2f}-{fu:.2f}), printed p {fp}; recomputed z = {fz:.2f}, p = {fmt_p(fpp)}")
# ------------------------------------------------------------- 2. what the printed p would require
print("\n=== 2. CI that the printed p-value would require, holding the aOR fixed ===")
A, B = R["A"], R["B"]
se_req_A = np.log(A["o"]) / norm.isf(0.011 / 2)
ci_req_A = np.exp(np.log(A["o"]) + np.array([-1, 1]) * Z975 * se_req_A)
se_req_B = np.log(B["o"]) / norm.isf(0.001 / 2) # p < .001 requires SE below this
ci_req_B = np.exp(np.log(B["o"]) + np.array([-1, 1]) * Z975 * se_req_B)
print(f"A: p = .011 with aOR 1.91 needs SE = {se_req_A:.3f} ({se_req_A/A['se']:.2f} x the SE implied by the "
f"printed CI) and a 95% CI of {ci_req_A[0]:.2f}-{ci_req_A[1]:.2f} (printed: 1.47-2.49)")
print(f" width on log scale: required {np.log(ci_req_A[1]/ci_req_A[0]):.3f} vs printed "
f"{np.log(2.49/1.47):.3f} (ratio {np.log(ci_req_A[1]/ci_req_A[0])/np.log(2.49/1.47):.2f})")
print(f"B: p < .001 with aOR 1.76 needs SE < {se_req_B:.3f} and a 95% CI no wider than "
f"{ci_req_B[0]:.2f}-{ci_req_B[1]:.2f} (printed: 1.17-2.65, SE {B['se']:.3f})")
print(f" the printed lower bound 1.17 is below the 1.25 that a 95% CI with p = .001 would have: "
f"{ci_req_B[0]:.3f}")
# ------------------------------------------------------------- 3. alternative conventions
print("\n=== 3. Alternative conventions that might reconcile p = .011 with 1.91 (1.47-2.49) ===")
o, l, u = A["o"], A["l"], A["u"]
se_t = (np.log(u) - np.log(l)) / (2 * t.isf(0.025, 3700))
print(f"t reference, df 3700 (n = 3,728 minus 28 terms): p = {fmt_p(2*t.sf(np.log(o)/se_t, 3700))}")
se_t40 = (np.log(u) - np.log(l)) / (2 * t.isf(0.025, 39))
print(f"t reference, df 39 (40 schools as clusters): p = {fmt_p(2*t.sf(np.log(o)/se_t40, 39))}")
for lvl in (0.90, 0.99, 0.999):
zz = norm.isf((1 - lvl) / 2)
s = (np.log(u) - np.log(l)) / (2 * zz)
print(f"if the interval were a {lvl:.1%} CI: p = {fmt_p(2*norm.sf(np.log(o)/s))}")
se_lin = (u - l) / (2 * Z975)
zl = (o - 1) / se_lin
print(f"Wald test on the OR (linear) scale: z = {zl:.2f}, p = {fmt_p(2*norm.sf(zl))}")
print(f"one-sided p on the log scale: p = {fmt_p(norm.sf(A['z']))}")
# swap hypothesis
print(f"if the two printed p-values were exchanged: A would read p = .011 -> '< .001' (recomputed "
f"{fmt_p(A['p'])}, consistent); B would read '< .001' -> '.011' (recomputed {B['p']:.4f}, rounds to .007, not .011)")
# ------------------------------------------------------------- 4. calibration on other estimates
print("\n=== 4. The same method on other estimates printed in the text (calibration) ===")
OTHER = [
("Ethnic minority, final model (abstract)", 1.59, 1.26, 2.02, "< .001"),
("Undisclosed gender, SDQ <= 16 (abstract)", 4.36, 1.50, 12.66, ".007"),
("Home safety, final model (abstract)", 0.81, 0.66, 0.99, ".038"),
("School safety, final model (abstract)", 0.69, 0.58, 0.83, "< .001"),
("Cumulative adversity per event (abstract)", 1.12, 1.07, 1.17, "< .001"),
("Recent adversity, main-effects model", 1.86, 1.49, 2.34, "< .001"),
("Female, main-effects model", 0.85, 0.72, 1.00, ".050"),
("Low wellbeing, high school support", 0.77, 0.61, 0.98, ".031"),
("High wellbeing and adult support", 0.75, 0.58, 0.96, ".025"),
("Financial hardship, final model", 1.38, 1.07, 1.79, ".013"),
("North Dublin, final model", 1.24, 1.07, 1.48, ".005"),
("High wellbeing, low school support (side issue)", 0.73, 0.57, 0.95, ".008"),
]
print(f"{'estimate':50s} {'aOR (95% CI)':>20s} {'SE':>6s} {'z':>6s} {'p recomputed':>13s} {'p(mid)':>9s} printed")
for lab, o_, l_, u_, pr in OTHER:
se_, z_, p_ = wald(o_, l_, u_)
_, _, pm_ = wald(o_, l_, u_, centre="mid")
print(f"{lab:50s} {o_:5.2f} ({l_:.2f}-{u_:.2f}) {se_:6.3f} {z_:6.2f} {fmt_p(p_):>13s} {fmt_p(pm_):>9s} {pr}")
_, _, pw = wald(0.74, 0.57, 0.96)
_, _, pwm = wald(0.74, 0.57, 0.96, centre="mid")
print(f"figure value for the side-issue row: 0.74 (0.57-0.96), printed p .019; recomputed {pw:.4f} (midpoint {pwm:.4f})")
print(f"North Dublin: geometric mean of CI = {np.sqrt(1.07*1.48):.3f} (figure prints 1.26)")
print("Every printed p above except the side-issue row agrees with the recomputed p (centred on the aOR or on")
print("the CI midpoint) to rounding; the side-issue row differs by a factor of about two.")
# ------------------------------------------------------------- 5. reference category note
print("\n=== 5. Reference category of estimate B ===")
print("Figure lists one reference row ('no adversities within 30 days and no parental support') and")
print("'no adversities within 30 days with parental support' = 0.99 (0.83-1.19).")
print(f"Within-support contrast if 1.76 is against the common reference: 1.76 / 0.99 = {1.76/0.99:.2f}")
# ------------------------------------------------------------- figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID = "#eb6834", "#2a78d6", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"axes.edgecolor": INK2, "svg.fonttype": "none"})
def style(ax):
ax.set_facecolor(BG)
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
def save(fig, name):
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"{name}.{ext}"), facecolor=BG, bbox_inches="tight", **kw)
plt.close(fig)
LBOX = dict(boxstyle="square,pad=0.15", facecolor=BG, edgecolor="none")
def sci(p):
m, e = f"{p:.1e}".split("e")
sup = str.maketrans("-0123456789", "\u207b\u2070\u00b9\u00b2\u00b3\u2074\u2075\u2076\u2077\u2078\u2079")
return f"{m} \u00d7 10{str(int(e)).translate(sup)}"
# Figure 1: p-values on one log axis
fig, ax = plt.subplots(figsize=(7.2, 3.3), facecolor=BG)
style(ax)
rows = [("B", 0.0), ("A", 1.0)]
ylabels = {"A": "Recent adversity,\nno parental support\naOR 1.91 (1.47–2.49)",
"B": "Recent adversity,\nwith parental support\naOR 1.76 (1.17–2.65)"}
for k, y in rows:
r = R[k]
# recomputed: point plus range over CI rounding
ax.plot([r["pmin"], r["pmax"]], [y - 0.13] * 2, color=BLUE, lw=2, zorder=3, solid_capstyle="butt")
ax.plot(r["p"], y - 0.13, "o", color=BLUE, ms=9, zorder=4)
ptxt = sci(r["p"]) if r["p"] < 1e-3 else f"{r['p']:.4f}"
ax.annotate(f"recomputed from CI: p = {ptxt}",
(r["p"], y - 0.13), xytext=(0, -13), textcoords="offset points",
ha="center", va="top", fontsize=10, color=BLUE, bbox=LBOX, zorder=6)
# reported in text
if r["prep"].startswith("<"):
ax.annotate("", xy=(2.2e-4, y + 0.13), xytext=(1e-3, y + 0.13),
arrowprops=dict(arrowstyle="-|>", color=ORANGE, lw=2, mutation_scale=14), zorder=4)
ax.plot(1e-3, y + 0.13, "s", color=ORANGE, ms=9, zorder=5)
txt = "printed in text: p < .001"
else:
ax.plot(r["pnum"], y + 0.13, "s", color=ORANGE, ms=9, zorder=5)
txt = f"printed in text: p = {r['prep']}"
ax.annotate(txt, (r["pnum"], y + 0.13), xytext=(0, 9), textcoords="offset points",
ha="center", va="bottom", fontsize=10, color=ORANGE, bbox=LBOX, zorder=6)
for thr, lab in ((0.05, ".05"), (0.001, ".001")):
ax.axvline(thr, color=INK2, lw=0.8, ls=(0, (3, 3)), zorder=1)
ax.text(thr, 1.62, f"p = {lab}", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(3e-8, 0.3)
ax.set_ylim(-0.6, 1.6)
ax.set_yticks([y for _, y in rows])
ax.set_yticklabels([ylabels[k] for k, _ in rows], fontsize=10)
ax.invert_xaxis()
ax.set_xlabel("Two-sided p-value (log scale; stronger evidence to the right)")
leg = [Line2D([], [], marker="s", color=ORANGE, lw=0, ms=9, label="Printed in abstract and Results"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=9,
label="Recomputed from the printed aOR and 95% CI\n(line: range over CI rounding)")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0, -0.28), ncol=2, frameon=False, fontsize=10)
save(fig, "fig1")
# Figure 2: printed CI vs CI the printed p would require, on one log OR axis
fig, ax = plt.subplots(figsize=(7.2, 3.1), facecolor=BG)
style(ax)
items = [ # (y, label, est, lo, hi, color, marker)
(3.0, "printed 95% CI", A["o"], A["l"], A["u"], ORANGE, "s"),
(2.4, "95% CI that p = .011 would need", A["o"], ci_req_A[0], ci_req_A[1], BLUE, "o"),
(1.0, "printed 95% CI", B["o"], B["l"], B["u"], ORANGE, "s"),
(0.4, "widest 95% CI that p < .001 allows", B["o"], ci_req_B[0], ci_req_B[1], BLUE, "o"),
]
for y, lab, e, lo, hi, c, m in items:
ax.plot([lo, hi], [y, y], color=c, lw=2, zorder=3)
ax.plot(e, y, m, color=c, ms=9, zorder=4)
ax.text(hi * 1.03, y, f"{lab}: {lo:.2f}–{hi:.2f}", va="center", ha="left", fontsize=10, color=c)
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(1, 3.55, "aOR = 1 (no association)", ha="center", va="bottom", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(0.8, 9)
ticks = [0.8, 1, 1.5, 2, 3, 4, 6]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{x:g}" for x in ticks])
ax.minorticks_off()
ax.set_ylim(0, 3.5)
ax.set_yticks([2.7, 0.7])
ax.set_yticklabels(["Recent adversity,\nno parental support\naOR 1.91, printed p = .011",
"Recent adversity,\nwith parental support\naOR 1.76, printed p < .001"], fontsize=10)
ax.set_xlabel("Adjusted odds ratio (log scale)")
leg = [Line2D([], [], marker="s", color=ORANGE, lw=2, ms=9, label="Printed in abstract and Results"),
Line2D([], [], marker="o", color=BLUE, lw=2, ms=9, label="Implied by the printed p-value (same aOR)")]
ax.legend(handles=leg, loc="upper left", bbox_to_anchor=(0, -0.22), ncol=2, frameon=False, fontsize=10)
save(fig, "fig2")
print("\nwrote fig1.svg, fig1.png, fig2.svg, fig2.png")
Abstract and Results print an employment-disruption confidence interval that excludes its own odds ratio
Psychol Med (2021) · doi:10.1017/S0033291721004293 · PMC8523967
Summary
The abstract reports that serious psychological distress (SPD) before the pandemic predicted employment disruption with "OR 4.5, 95% CI 1.4–3.8", and the Results text gives the same estimate as "OR 4.45, 95% CI 1.44–3.75". In both places the upper confidence limit is below the odds ratio, which no 95% interval for an estimate can be. Table 2 reports the same contrast from the same model as 4.5 (1.4–13.8). On the log scale the Results-text OR and lower limit imply an upper limit of 4.45² / 1.44 = 13.75, so the printed upper limits appear to have lost a leading "1". The direction and statistical significance of the association are unchanged (p ≈ 0.01). However, the abstract understates the upper limit by a factor of about 3.7, which matters to anyone who takes the interval from the abstract.
What the paper reports
Every quotation below was checked verbatim against the PMC-archived text with src/show.py.
Abstract, Results:
SPD at T1 predicted disruption of employment (OR 4.5, 95% CI 1.4–3.8) and health care (OR 3.2, 95% CI 1.4–7.1) at T2.
Results, second paragraph (multinomial regression):
After adjustment for gender, age, race/ethnicity, and household income, SPD at T1 was associated with higher odds of employment disruption among those employed at T2 (OR 4.45, 95% CI 1.44–3.75) and with health care disruption among those with health conditions at T2 (OR 3.15, 95% CI 1.41–7.05). The association between SPD at T1 and disruption of childcare at T1 among those with children in the household was elevated but did not reach statistical significance (OR 2.16, 95% CI 0.58–8.09).
Table 2, "Association of pre-pandemic sociodemographic characteristics and serious psychological distress with experience of stressors early in the pandemic", row "High v. low". The columns are employment (not working; working, no job loss = reference; working with job loss), health care and childcare:
High v. low | 1.4 | (0.7–2.8) | 1 | 4.5 | (1.4–13.8) | 0.5 | (0.2–1.2) | 1 | 3.2 | (1.4–7.1) | 2.4 | (1.0–5.9) | 1 | 2.2 | (0.6–8.1)
Table 2 footer:
OR, odds ratio; NH, Non-Hispanic. ORs estimated in multinomial logistic regression models.
Methods, statistical analysis:
All analyses were conducted using survey procedures in SAS or Stata to account for the ALP sample design and weights.
What the paper's own numbers show
Method. The models are survey-weighted logistic and multinomial logistic regressions. For these, the 95% CI is the Wald interval exp(b ± 1.96·SE), which is symmetric on the log scale. Two consequences can be checked from the printed numbers alone. First, the OR must lie inside its interval. Second, it must equal the geometric midpoint √(L·U), up to rounding. Equivalently, U = OR² / L. I applied both checks to every odds ratio with a 95% CI in the abstract and the Results text, and to the three "High v. low" intervals in Table 2. That is 14 intervals in all. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| Employment disruption, SPD high v. low | Abstract | OR 4.5, 95% CI 1.4–3.8 | upper limit < OR; √(1.4 × 3.8) = 2.31, not 4.5 |
| Employment disruption, SPD high v. low | Results text | OR 4.45, 95% CI 1.44–3.75 | upper limit < OR; √(1.44 × 3.75) = 2.32, not 4.45 |
| Employment disruption, SPD high v. low | Table 2 | 4.5 (1.4–13.8) | contains OR; √(1.4 × 13.8) = 4.40 (consistent given one-decimal rounding) |
| Upper limit implied by the text's OR and L | derived | — | 4.45² / 1.44 = 13.75 (13.8 at one decimal, as in Table 2) |
| Check of the implied interval 1.44–13.75 | derived | — | √(1.44 × 13.75) = 4.450; log-SE 0.576 on both sides |
| Wald test implied by OR 4.45, L 1.44 | derived | — | z = 2.59, two-sided p = 0.0095 (from Table 2 alone: p = 0.0100) |
| Health care disruption | Abstract / Results / Table 2 | 3.2 (1.4–7.1) / 3.15 (1.41–7.05) / 3.2 (1.4–7.1) | consistent; √(L·U) = 3.153 |
| Childcare disruption | Results / Table 2 | 2.16 (0.58–8.09) / 2.2 (0.6–8.1) | consistent; √(L·U) = 2.166 / 2.205 |
| SPD at T3: the six Results-text ORs (serious and moderate T1 distress, job change, care disrupted, income < $35K, age 20–39) | Results text, Table 3 paragraph | e.g. 10.24 (4.5–23.3), 3.21 (1.4–7.6) | all consistent; √(L·U) / OR between 0.968 and 1.016 |
| Correct upper limit ÷ printed upper limit | Abstract and Results | 3.8 and 3.75 | 13.8 / 3.8 = 3.63; 13.75 / 3.75 = 3.67 |
The text interval cannot be rescued by assuming that the OR rather than the interval is wrong. Taken alone, 1.44–3.75 implies OR 2.32 and log-SE 0.244. But the OR is printed as 4.5 in the abstract and in Table 2, and as 4.45 in the text, and Table 2's interval agrees with it. The other 12 intervals all satisfy √(L·U) ≈ OR to within 3.2%. The two employment intervals in the abstract and Results text are the only exceptions, with ratios of 0.51 and 0.52 (Figure 2). Prefixing a "1" to each printed upper limit (3.75 → 13.75, 3.8 → 13.8) restores consistency exactly.
Figure
Figure 1. The odds ratio for employment disruption (working with job loss v. working, no job loss) in people with high v. low distress at T1, on a log scale. Orange rows are printed in the paper: the abstract (OR 4.5, CI 1.4–3.8), the Results text (OR 4.45, CI 1.44–3.75) and Table 2 (OR 4.5, CI 1.4–13.8). In the first two the point estimate lies to the right of the interval's upper limit. The blue row is the interval recomputed from the Results-text OR and lower limit under the log-symmetric Wald construction (U = OR² / L = 13.75). It coincides with Table 2. The thin vertical line marks OR = 1. All versions exclude 1, so significance is unaffected; what differs is the upper limit.
Figure 2. For each odds ratio with a 95% CI in the abstract and Results text, and for the three "High v. low" intervals in Table 2, the geometric midpoint of the printed interval divided by the printed OR (log scale; 1 means the interval is centred on its OR as a Wald interval should be). Twelve intervals lie between 0.968 and 1.016, which is what rounding produces. The abstract and Results-text intervals for employment disruption (open markers) lie at 0.51 and 0.52. This shows that the paper's intervals otherwise follow the log-symmetric convention used for the recomputation, and that the discrepancy is confined to this one estimate as printed in the text.
Why it matters
- The interval as printed is impossible, and it sits in the abstract. An estimate cannot lie outside its own 95% CI. A reader of the abstract alone cannot tell which number is wrong, the OR or the interval, without turning to Table 2.
- The precision of the association is overstated. The correct interval, about 1.4 to 13.8, is compatible with anything from a 1.4-fold to an almost 14-fold increase in the odds of job change or loss. The printed 1.4–3.8 suggests a much narrower range: on the log scale the correct interval is 2.36 times wider, and its upper limit is about 3.7 times higher.
- Evidence synthesis is the most exposed use. A meta-analyst who back-calculates a standard error from the abstract's interval would get log-SE 0.244 from its width, against 0.576 from the OR and lower limit. That would give the study far too much weight, or the entry would have to be discarded as unusable.
- The paper's conclusions are not affected. The direction of the association, its significance (p ≈ 0.01) and the point estimate are unchanged. The Discussion's qualitative statement that people with pre-pandemic SPD "were more likely than other people who were employed at the beginning of the pandemic to experience a job change" still holds. The headline T3 results (OR 10.2 for T1 SPD, 3.2 for employment disruption, 3.3 for health care disruption) and the policy conclusion are separate estimates, and their intervals are consistent.
Explanations considered
- A different model, population or contrast in the text than in Table 2. Not supported. The text describes the adjusted multinomial model of Table 2 (adjusted for gender, age, race/ethnicity and household income), among those employed, for SPD high v. low. That is the Table 2 contrast. The two other text intervals in the same sentence and paragraph (health care 3.15, 1.41–7.05; childcare 2.16, 0.58–8.09) are the Table 2 values at two decimals, so the text reports the same model.
- A non-Wald or asymmetric interval (profile likelihood, bootstrap, design-based). Not able to explain it. Any of these may be somewhat asymmetric, but none produces a 95% interval that excludes its own point estimate. The other 12 intervals in the paper are log-symmetric to within rounding.
- The OR, not the interval, is wrong. Not supported. The OR is printed three times (4.5, 4.45, 4.5), and Table 2's interval is centred on it. The text interval would imply OR 2.32, which appears nowhere in the paper.
- Rounding. Not able to explain it. Rounding changes the second decimal, not the leading digit. Rounding does account for the small gap between 13.75 (from the text's two-decimal values) and 14.46 (from 4.5² / 1.4 at one decimal); Table 2's 13.8 is 13.75 rounded.
- A text-conversion artifact in the PMC copy. Ruled out. The publisher's HTML on Cambridge Core and the typeset publisher PDF print the same numbers: 1.4–3.8 in the abstract, 1.44–3.75 in the Results and 1.4–13.8 in Table 2. Europe PMC's full-text XML matches them as well.
- Side observation, not part of the main finding. The Results sentence on childcare says "disruption of childcare at T1", whereas the stressor is measured at T2 elsewhere in the paper (abstract, Methods, Table 2 title). This is a wording slip with no numerical consequence.
Status
- Version of record. doi.org resolves to the Cambridge Core article page. The Cambridge Core HTML and the publisher PDF (9 pages; Psychological Medicine vol. 53, pp. 2418–2426, first published online 11 October 2021) print the quoted passages with the same numbers, including Table 2's "4.5 (1.4–13.8)". The Europe PMC full-text XML and the PMC page also match.
- Supplements. None exist. The Cambridge Core record lists no supplementary material, Europe PMC marks the record as having none, and the JATS XML has no supplementary-material element. Supplements therefore cannot resolve the discrepancy; Table 2 in the main text already does.
- Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturned no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has no update-to or updated-by entry, and a Crossref query for updates to this DOI returned none. The Cambridge Core page data lists no corrections, retractions or addenda. The PubMed record (PMID 34629132) has no CommentsCorrections entry. A web search for a correction, erratum or PubPeer discussion found nothing for this article. PubPeer itself could not be queried directly, because its site and API refused automated requests (HTTP 403). - Sources. PMC article PMC8523967; Europe PMC full-text XML for PMC8523967; Cambridge Core article page and publisher PDF via https://doi.org/10.1017/S0033291721004293; Crossref; PubMed; Europe PMC; web search.
Suggested correction
In the abstract, replace
SPD at T1 predicted disruption of employment (OR 4.5, 95% CI 1.4–3.8)
with
SPD at T1 predicted disruption of employment (OR 4.5, 95% CI 1.4–13.8)
and in the second paragraph of the Results, replace "(OR 4.45, 95% CI 1.44–3.75)" with "(OR 4.45, 95% CI 1.44–13.75)", or with the unrounded upper limit from the fitted model. These values follow from Table 2 and from the text's own OR and lower limit. The authors may also wish to change "disruption of childcare at T1" in the same paragraph to "at T2".
Recomputation script (python recompute.py.txt)
"""Recompute the employment-disruption confidence interval of PMC8523967
(Psychol Med, doi:10.1017/S0033291721004293, "A longitudinal study of predictors of
serious psychological distress during COVID-19 pandemic").
Run: python recompute.py.txt
Needs numpy, scipy and matplotlib. Prints every recomputed number used in writeup.md
and writes fig1.svg/png and fig2.svg/png next to this file.
Inputs are copied from the paper as archived in PMC:
Abstract: "disruption of employment (OR 4.5, 95% CI 1.4–3.8)" and the other abstract ORs.
Results, paragraph 2: "(OR 4.45, 95% CI 1.44–3.75)", "(OR 3.15, 95% CI 1.41–7.05)",
"(OR 2.16, 95% CI 0.58–8.09)".
Table 2, row "High v. low": employment 4.5 (1.4–13.8), health care 3.2 (1.4–7.1),
childcare 2.2 (0.6–8.1).
Results, Table 3 paragraph: ORs at two decimals with their CIs.
Methods: survey-weighted logistic / multinomial logistic regression (SAS or Stata), for
which the reported 95% CI is a Wald interval exp(b +/- 1.96*SE), symmetric on the log scale.
"""
import math
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.lines import Line2D # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
# (label, location, OR, lower, upper) -- all printed in the paper
ROWS = [
("Employment disruption", "Abstract", 4.5, 1.4, 3.8),
("Employment disruption", "Results text", 4.45, 1.44, 3.75),
("Employment disruption", "Table 2", 4.5, 1.4, 13.8),
("Health care disruption", "Abstract", 3.2, 1.4, 7.1),
("Health care disruption", "Results text", 3.15, 1.41, 7.05),
("Health care disruption", "Table 2", 3.2, 1.4, 7.1),
("Childcare disruption", "Results text", 2.16, 0.58, 8.09),
("Childcare disruption", "Table 2", 2.2, 0.6, 8.1),
("T3: serious distress at T1", "Results text", 10.24, 4.5, 23.3),
("T3: moderate distress at T1", "Results text", 4.55, 2.0, 9.7),
("T3: job change/loss at T2", "Results text", 3.21, 1.4, 7.6),
("T3: care disrupted at T2", "Results text", 3.32, 1.5, 7.2),
("T3: income <$35K at T1", "Results text", 2.64, 1.1, 6.4),
("T3: age 20-39", "Results text", 2.17, 1.0, 4.8),
]
def describe(orr, lo, hi):
gm = math.sqrt(lo * hi)
se_lo = (math.log(orr) - math.log(lo)) / Z
se_hi = (math.log(hi) - math.log(orr)) / Z
return gm, se_lo, se_hi
print("=== 1. Every OR with a 95% CI in the abstract, Results text and Table 2 'High v. low' row ===")
print(f"{'quantity':30s} {'location':13s} {'OR':>6s} {'CI':>14s} {'contains':>8s} {'sqrt(LU)':>9s} "
f"{'sqrt(LU)/OR':>11s} {'SE_low':>7s} {'SE_up':>7s}")
RATIOS = []
for lab, loc, orr, lo, hi in ROWS:
gm, sl, su = describe(orr, lo, hi)
RATIOS.append((lab, loc, gm / orr))
print(f"{lab:30s} {loc:13s} {orr:6.2f} {f'{lo}-{hi}':>14s} {str(lo <= orr <= hi):>8s} {gm:9.3f} "
f"{gm/orr:11.3f} {sl:7.3f} {su:7.3f}")
print()
print("=== 2. Employment disruption: the interval implied by the paper's own numbers ===")
OR_T, L_T, U_T = 4.45, 1.44, 3.75 # Results text
OR_A, L_A, U_A = 4.5, 1.4, 3.8 # Abstract
OR_2, L_2, U_2 = 4.5, 1.4, 13.8 # Table 2
se_t = (math.log(OR_T) - math.log(L_T)) / Z
U_impl = OR_T ** 2 / L_T
print(f"log-SE from the text's OR and lower bound: (ln {OR_T} - ln {L_T}) / {Z:.4f} = {se_t:.4f}")
print(f"implied upper bound U = OR^2 / L = {OR_T}^2 / {L_T} = {U_impl:.3f} -> rounds to {U_impl:.2f} (2 dp), "
f"{round(U_impl, 1)} (1 dp)")
print(f"Table 2 prints 13.8; text prints {U_T}; abstract prints {U_A}")
print(f"check: sqrt({L_T} * 13.75) = {math.sqrt(L_T*13.75):.4f}; sqrt({L_2} * {U_2}) = {math.sqrt(L_2*U_2):.3f}")
print(f"OR implied by the printed text interval: sqrt({L_T} * {U_T}) = {math.sqrt(L_T*U_T):.3f}")
print(f"OR implied by the printed abstract interval: sqrt({L_A} * {U_A}) = {math.sqrt(L_A*U_A):.3f}")
print(f"implied upper from abstract-precision values: {OR_A}^2 / {L_A} = {OR_A**2/L_A:.2f} "
f"(rounding of OR and L to 1 dp explains the gap from 13.8)")
print(f"upper bound ratio, correct / printed: 13.75 / 3.75 = {13.75/3.75:.3f}; 13.8 / 3.8 = {13.8/3.8:.3f}")
print(f"log-width ratio: ln(13.75/1.44) / ln(3.75/1.44) = {math.log(13.75/1.44)/math.log(3.75/1.44):.3f}")
z_t = math.log(OR_T) / se_t
p_t = 2 * norm.sf(z_t)
print(f"Wald z = ln {OR_T} / {se_t:.4f} = {z_t:.3f}; two-sided p = {p_t:.4f}")
se_2 = (math.log(U_2) - math.log(L_2)) / (2 * Z)
z_2 = math.log(OR_2) / se_2
print(f"from Table 2 alone: SE = ln({U_2}/{L_2}) / (2*{Z:.2f}) = {se_2:.4f}; z = {z_2:.3f}; p = {2*norm.sf(z_2):.4f}")
# an SE that would make the printed text interval a valid symmetric interval would need OR = 2.32
se_print = (math.log(U_T) - math.log(L_T)) / (2 * Z)
print(f"log-SE implied by the printed text interval taken alone: {se_print:.4f} "
f"({se_t/se_print:.2f} times smaller than the SE implied by OR and lower bound)")
# the single-digit hypothesis: prefixing '1' to each printed upper bound
for printed, lo, orr in [(U_T, L_T, OR_T), (U_A, L_A, OR_A)]:
fixed = float("1" + str(printed))
print(f"prefix '1' to {printed}: {fixed}; sqrt({lo}*{fixed}) = {math.sqrt(lo*fixed):.3f} vs OR {orr}")
print()
print("=== 3. Ratio sqrt(L*U)/OR across the paper's 14 printed intervals ===")
others = [r for (lab, loc, r) in RATIOS if not (lab == "Employment disruption" and loc != "Table 2")]
print(f"other 12 intervals: min {min(others):.3f}, max {max(others):.3f}")
for lab, loc, r in RATIOS:
if lab == "Employment disruption":
print(f" {lab} / {loc}: {r:.3f}")
# ---------------------------------------------------------------- figures
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({
"font.size": 10.5, "font.family": "DejaVu Sans", "text.color": INK, "axes.labelcolor": INK,
"xtick.color": INK2, "ytick.color": INK, "axes.edgecolor": INK2, "svg.fonttype": "none",
})
def style(ax):
ax.set_facecolor(BG)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", which="both", color=GRID, lw=0.8)
ax.set_axisbelow(True)
ax.tick_params(axis="y", length=0)
# Figure 1: dot-and-interval, employment disruption only
f1 = [
("Abstract", OR_A, L_A, U_A, ORANGE, f"OR {OR_A}, CI {L_A}–{U_A}"),
("Results text", OR_T, L_T, U_T, ORANGE, f"OR {OR_T}, CI {L_T}–{U_T}"),
("Table 2", OR_2, L_2, U_2, ORANGE, f"OR {OR_2}, CI {L_2}–{U_2}"),
("Recomputed from\nResults-text OR and L", OR_T, L_T, U_impl, BLUE,
f"OR {OR_T}, CI {L_T}–{U_impl:.2f}"),
]
fig, ax = plt.subplots(figsize=(7.2, 3.5), facecolor=BG)
style(ax)
ys = np.arange(len(f1))[::-1]
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
ax.text(1.0, len(f1) - 0.35, "OR = 1 ", ha="right", va="center", color=INK2, fontsize=10)
for y, (lab, o, lo, hi, c, txt) in zip(ys, f1):
ax.plot([lo, hi], [y, y], color=c, lw=2, solid_capstyle="butt", zorder=2)
ax.plot([lo, lo], [y - 0.12, y + 0.12], color=c, lw=2)
ax.plot([hi, hi], [y - 0.12, y + 0.12], color=c, lw=2)
ax.plot(o, y, "o", ms=9, color=c, mec=BG, mew=1.2, zorder=3)
ax.text(27, y, txt, va="center", ha="left", fontsize=10, color=INK)
if hi < o:
ax.text(math.sqrt(lo * hi), y + 0.3, f"upper limit {hi} < OR {o}", fontsize=10,
color=INK2, ha="center", va="center")
ax.set_xscale("log")
ax.set_xlim(0.7, 26)
ax.set_xticks([1, 2, 4, 8, 16])
ax.set_xticklabels(["1", "2", "4", "8", "16"])
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in f1])
ax.set_ylim(-0.6, len(f1) - 0.1)
ax.set_xlabel("Odds ratio, employment disruption, SPD at T1 high v. low (log scale)")
leg = [Line2D([0], [0], color=ORANGE, lw=2, marker="o", ms=8, label="Printed in the paper"),
Line2D([0], [0], color=BLUE, lw=2, marker="o", ms=8,
label="Recomputed: U = OR² / L (log-symmetric Wald interval)")]
fig.legend(handles=leg, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False,
fontsize=10)
fig.subplots_adjust(left=0.24, right=0.70, top=0.95, bottom=0.27)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
# Figure 2: sqrt(L*U)/OR for every interval (recomputed); the two employment intervals stand out
fig, ax = plt.subplots(figsize=(7.2, 4.6), facecolor=BG)
style(ax)
labels = [f"{lab} · {loc}" for lab, loc, _ in RATIOS]
vals = [r for _, _, r in RATIOS]
ys = np.arange(len(vals))[::-1]
ax.axvline(1, color=INK2, lw=0.8, zorder=1)
for y, v, (lab, loc, _) in zip(ys, vals, RATIOS):
bad = lab == "Employment disruption" and loc != "Table 2"
ax.plot(v, y, "o", ms=8, color=BLUE, mfc=BLUE if not bad else BG, mew=2, zorder=3)
if bad:
ax.text(v * 1.05, y, f"{v:.2f} (interval excludes its OR)", va="center", fontsize=10, color=INK,
bbox=dict(facecolor=BG, edgecolor="none", pad=1.5), zorder=4)
ax.set_xscale("log")
ax.set_xlim(0.4, 1.6)
ax.set_xticks([0.5, 0.75, 1, 1.25, 1.5])
ax.set_xticklabels(["0.5", "0.75", "1", "1.25", "1.5"])
ax.minorticks_off()
ax.set_yticks(ys)
ax.set_yticklabels(labels, fontsize=10)
ax.set_xlabel("√(L × U) ÷ OR, recomputed (log scale; 1 = consistent)")
leg = [Line2D([0], [0], color=BLUE, lw=0, marker="o", ms=8, label="Consistent with a log-symmetric CI"),
Line2D([0], [0], color=BLUE, lw=0, marker="o", ms=8, mfc=BG, mew=2,
label="Inconsistent: interval excludes its OR")]
fig.legend(handles=leg, loc="lower center", bbox_to_anchor=(0.5, 0.0), ncol=2, frameon=False, fontsize=10)
fig.subplots_adjust(left=0.47, right=0.97, top=0.98, bottom=0.2)
for ext in ("svg", "png"):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), dpi=200, facecolor=BG)
plt.close(fig)
print()
print("wrote fig1.svg, fig1.png, fig2.svg, fig2.png in", HERE)
Abstract's adjusted hazard ratio 1.30 lies outside its printed 95% CI; Table 2 gives 0.66–2.56
Stroke (2023) · doi:10.1161/STROKEAHA.123.043336 · PMC10519293
Summary
The abstract reports the multivariable hazard ratio for the primary outcome (any stroke or TIA, patients with new DWI lesions vs without) as "1.30 [95% CI, 0.10–1.02]". A point estimate cannot lie outside its own confidence interval, and no rounding closes the gap: the interval is centred on 0.32 on the log scale. The paper's own Table 2 gives the same estimate as 1.30 (0.66–2.56), P=0.454, and that interval is internally consistent: it is centred on 1.30 and implies P≈0.45. The error is confined to the abstract's interval. The point estimate, its non-significance and the paper's conclusion are unchanged, but the abstract's headline adjusted result is impossible as printed and, read at face value, points in the protective direction.
What the paper reports
Abstract, Results:
Uni- and multivariable regression analyses did not show significant differences (hazard ratio, 1.50 [95% CI, 0.76–2.94] and hazard ratio, 1.30 [95% CI, 0.10–1.02], respectively).
Abstract, Methods (definition of the outcome these hazard ratios refer to):
The primary composite clinical outcome was the time to any stroke or transient ischemic attack during follow-up.
Methods, Statistical Analysis:
Cox proportional hazard regression models were used to calculate the hazard ratio (HR) with 95% CI for the association between DWI presence and cerebrovascular events during follow-up.
Models were adjusted for the demographic or clinical variables that showed an association of P<0.20 with the determinant (DWI presence) as well as with the outcome of interest (Table S1).
Results, first paragraph:
The binary uni- and multivariable cox regression analyses revealed no significant difference in recurrent cerebrovascular events between DWI+ patients and DWI- patients (Table 2).
Table 2, "HRs of Outcome Events During Follow-Up Depending on New Postoperative DWI Lesions", first column ("Stroke and TIA in any territory"), block "DWI+ (n=51) vs DWI– (n=111)". Table 2 is typeset as an image in the archived article (str-54-2562-g002.jpg), so these cells were read from that image rather than from a text conversion:
Unadjusted | 1.50 (0.76–2.94), P=0.245
Adjusted | 1.30 (0.66–2.56), P=0.454*
*Corrected for age and systolic blood pressure.
Abstract, Conclusions:
Based on our outcome analysis within the ICSS magnetic resonance imaging substudy, DWI lesions following carotid revascularization did not seem to have a relationship with long-term stroke risk.
What the paper's own numbers show
Method. Cox model confidence intervals are Wald intervals on the log-hazard scale, so a correctly printed 95% CI is symmetric around ln(HR). From each printed HR and CI I computed: whether the HR lies inside the interval; the geometric centre √(lower × upper), which should equal the HR up to rounding; the log-asymmetry [ln(upper) − ln(HR)] − [ln(HR) − ln(lower)], which should be about 0; the standard error (ln upper − ln lower)/(2 × 1.96); and the two-sided P implied by ln(HR)/SE, for comparison with the printed P. I applied the same checks to every DWI+ vs DWI− cell of Table 2 and to the Fine-Gray sensitivity analysis in Supplemental Table S2, so that the abstract figure can be judged against the paper's other estimates. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Multivariable HR, any stroke/TIA | Abstract, Results | 1.30 [95% CI 0.10–1.02] | 1.30 is outside the interval; geometric centre 0.32; log-asymmetry −2.81 (expected ≈ 0) |
| Rounding margin | derived | — | smallest HR that rounds to 1.30 is 1.295; largest upper limit that rounds to 1.02 is 1.025; gap 0.270 |
| Adjusted HR, any stroke/TIA (age, systolic BP) | Table 2 | 1.30 (0.66–2.56), P=0.454 | centre 1.30; log-asymmetry 0.00; SE 0.346; P=0.448 |
| 95% CI implied by HR 1.30 with the Table 2 SE | derived | 0.10–1.02 (abstract) | 0.66–2.56, identical to Table 2 |
| Univariable HR, any stroke/TIA | Abstract; Table 2 | 1.50 (0.76–2.94), P=0.245 | centre 1.49; log-asymmetry −0.01; SE 0.345; P=0.240 |
| Log-width of the CI | Abstract vs Table 2 | 0.10–1.02 vs 0.76–2.94 and 0.66–2.56 | 2.32 vs 1.35 and 1.36: the abstract interval is 1.72 times as wide as the univariable one |
| Unadjusted / adjusted, ipsilateral stroke/TIA | Table 2 | 1.91 (0.81–4.49), P=0.138; 1.25 (0.52–2.98), P=0.619 | centres 1.91 and 1.24; P=0.139 and 0.616 |
| Unadjusted, stroke in any territory | Table 2 | 0.69 (0.26–1.84), P=0.456 | centre 0.69; P=0.457 |
| Unadjusted / adjusted, ipsilateral stroke | Table 2 | 0.88 (0.24–3.25), P=0.845; 0.78 (0.21–2.93), P=0.713 | centres 0.88 and 0.78; P=0.848 and 0.712 |
| Adjusted, stroke in any territory (side observation) | Table 2 | 0.70 (0.29–1.51), P=0.493 | centre 0.66; log-asymmetry −0.11; P=0.397 from the CI. A smaller inconsistency, separate from this report |
| Fine-Gray unadjusted / adjusted, any stroke/TIA | Supplemental Table S2 | 1.35 (0.68–2.66); 1.16 (0.57–2.36) | centres 1.34 and 1.16; both consistent; neither is 0.10–1.02 |
Apart from the abstract's multivariable interval, every estimate of this outcome in the paper passes the checks to within rounding. Its adjusted interval in Table 2 has almost exactly the univariable width (1.36 vs 1.35 on the log scale), as expected when adjusting for two covariates in a cohort of 162.
Figure
Hazard ratios for the primary outcome (any stroke or TIA, DWI+ vs DWI−) on one log-scaled axis. Orange: the HR (dot) and 95% CI (bar) as printed in the paper. Blue diamonds: the geometric centre of each printed interval, recomputed in recompute.py.txt. For a Cox model's Wald interval it should coincide with the HR. The dashed line marks HR = 1 (no difference). The abstract's multivariable row is the only one where the centre (0.32) and the printed HR (1.30) disagree, and its HR sits to the right of the upper limit. The Table 2 adjusted row and the univariable row are self-consistent. The abstract's interval is also wider, not narrower, than the univariable one.
Why it matters
The abstract's multivariable hazard ratio is the adjusted result for the primary outcome, and most readers and indexing services see only the abstract. As printed, the pair is impossible. Taken at face value, the interval 0.10–1.02 describes an estimate near 0.32 (a protective association) whose upper limit only just reaches 1. The study's actual adjusted estimate is 1.30 (0.66–2.56): imprecise, not significant, and leaning towards a higher hazard in DWI+ patients. The upper limit of the correct interval, about a 2.6-fold hazard, also bears on how the conclusion "did not seem to have a relationship with long-term stroke risk" should be read: the data do not exclude a clinically relevant increase. That caveat applies whichever interval is printed, but the misprinted one hides it.
Neither the direction of the main conclusion nor its significance depends on the error. Both the printed and the correct intervals include 1, the point estimate is the same, and the correct value is printed in Table 2 of the same article. A reader who checks Table 2 can recover the right value. A reader of the abstract alone, or of a database record built from it, cannot.
Explanations considered
- Rounding. Not enough. Rounding moves each number by at most 0.005, and the HR exceeds the upper limit by at least 0.270 (
recompute.py.txt, section 2). - A different kind of interval (profile likelihood, bootstrap). Not consistent with the paper. The Methods describe Cox models with 95% CIs, and the paper applies bootstrapping only to the DWI-count analyses (Table 2, "Bootstrap" row). Profile-likelihood and bootstrap intervals for a maximum-likelihood estimate would still be expected to contain the estimate. An interval that lies entirely on one side of it (log-asymmetry −2.81) is not what either method produces. The Table 2 interval for the same model is a symmetric Wald interval.
- Reversed reference group (DWI− vs DWI+). Does not reproduce the numbers. Inverting the Table 2 adjusted result gives 0.77 (0.39–1.52), and inverting the univariable result gives 0.67 (0.34–1.32). Neither is 0.10–1.02.
- Limits reported on the log scale. No. The Table 2 adjusted limits on the log scale are −0.42 and 0.94.
- A different model (competing-risk sensitivity analysis). No. Supplemental Table S2 gives Fine-Gray estimates of 1.35 (0.68–2.66) unadjusted and 1.16 (0.57–2.36) adjusted for this outcome. Both are consistent, and neither matches.
- A different cell of the paper. I found no 0.10–1.02 pair in Table 2 or in Supplemental Tables S1–S3. Table S1 contains the upper limit 1.02 in other rows, for example "0.42 (0.18, 1.02)" for ipsilateral stenosis, but no matching pair. The source of the abstract's interval cannot be determined from the paper.
- A text-conversion or archiving artifact. Ruled out. The same text appears in the PMC article page and in the Europe PMC full-text XML, both of which come from the publisher's own JATS deposit, and the en-dash formatting is the same as in the neighbouring, correct interval.
What remains is that the abstract's confidence limits were transcribed incorrectly. The paper does not show how this happened.
Status
As of 2026-10-24:
- Version of record. The abstract text was checked in the PMC article page (pmc.ncbi.nlm.nih.gov/articles/PMC10519293) and in the Europe PMC full-text XML. Both read "hazard ratio, 1.30 [95% CI, 0.10–1.02]". The publisher's pages at ahajournals.org (HTML and PDF) returned HTTP 403, so the PMC deposit of the publisher's XML stands in for the version of record. Table 2 was read from the publisher's typeset image (str-54-2562-g002.jpg) downloaded from the PMC CDN.
- Supplements. The supplementary files were downloaded through the Europe PMC supplementaryFiles endpoint: the STROBE checklist (str-54-2562-s001.pdf) and the Supplemental Material (str-54-2562-s002.pdf, Tables S1–S3 and Figures S1–S2). Table S2 gives consistent Fine-Gray estimates (above). Nothing in the supplement contains 0.10–1.02.
- Corrections and retractions. None found. The Crossref record has no update-to or updated-by entries and an empty relation field. Crossref and the PMC and Europe PMC notice checks (
src/known.py) returned nothing. The PMC article page shows no notice. A web search for the title with "correction", "erratum" and "PubPeer" returned only the article and news coverage of it. Europe PMC search returned HTTP 503 ("Search service is temporarily unavailable") on every attempt. - PubPeer. The search page returned HTTP 403, so PubPeer comments could not be checked directly.
- Sources: PMC article; Crossref record; Europe PMC supplementary files; DOI.
Suggested correction
In the Abstract, Results, replace "hazard ratio, 1.30 [95% CI, 0.10–1.02]" with "hazard ratio, 1.30 [95% CI, 0.66–2.56]", the value given for the same adjusted model in Table 2 (adjusted for age and systolic blood pressure; P=0.454). The authors may also wish to check the Table 2 cell for adjusted stroke in any territory, "0.70 (0.29–1.51), P=0.493". Its interval is centred on 0.66, and the P value it implies (0.40) differs from the one printed, which suggests that one of those numbers may also need correcting.
Recomputation script (python recompute.py.txt)
# Recompute every number in writeup.md for PMC10519293 and regenerate fig1.svg / fig1.png.
# Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
# All inputs are copied from the article as archived in PubMed Central (abstract text,
# Table 2 image str-54-2562-g002.jpg, Supplemental Table S2 in str-54-2562-s002.pdf).
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z = norm.ppf(0.975)
def wald(hr, lo, hi):
"""Back-calculate from a printed ratio and 95% CI, assuming a Wald interval on the log scale."""
se = (np.log(hi) - np.log(lo)) / (2 * Z)
centre = np.sqrt(lo * hi) # geometric centre of the interval
asym = (np.log(hi) - np.log(hr)) - (np.log(hr) - np.log(lo)) # 0 for a log-symmetric CI
p = 2 * norm.sf(abs(np.log(hr) / se))
return dict(se=se, centre=centre, asym=asym, p=p, logwidth=np.log(hi / lo),
inside=lo <= hr <= hi)
def show(label, hr, lo, hi, p_rep=None):
r = wald(hr, lo, hi)
print(f"{label:52s} {hr:.2f} ({lo:.2f}-{hi:.2f}) inside={str(r['inside']):5s} "
f"centre={r['centre']:.2f} log-asym={r['asym']:+.2f} log-width={r['logwidth']:.2f} "
f"SE={r['se']:.3f} P(from CI)={r['p']:.3f} P(reported)={p_rep}")
return r
print("== 1. Abstract, Results ==")
abs_uni = show("Abstract: univariable HR, stroke/TIA any territory", 1.50, 0.76, 2.94)
abs_multi = show("Abstract: multivariable HR, stroke/TIA any territory", 1.30, 0.10, 1.02)
print("\n== 2. Rounding cannot put 1.30 inside 0.10-1.02 ==")
print(f"smallest HR that rounds to 1.30: 1.295; largest upper limit that rounds to 1.02: 1.025;"
f" gap = {1.295 - 1.025:.3f}")
print("\n== 3. Table 2 (image), DWI+ (n=51) vs DWI- (n=111) ==")
table2 = [
("T2 unadjusted, stroke/TIA any territory", 1.50, 0.76, 2.94, 0.245),
("T2 adjusted*, stroke/TIA any territory", 1.30, 0.66, 2.56, 0.454),
("T2 unadjusted, ipsilateral stroke/TIA", 1.91, 0.81, 4.49, 0.138),
("T2 adjusted, ipsilateral stroke/TIA", 1.25, 0.52, 2.98, 0.619),
("T2 unadjusted, stroke any territory", 0.69, 0.26, 1.84, 0.456),
("T2 adjusted, stroke any territory", 0.70, 0.29, 1.51, 0.493),
("T2 unadjusted, ipsilateral stroke", 0.88, 0.24, 3.25, 0.845),
("T2 adjusted, ipsilateral stroke", 0.78, 0.21, 2.93, 0.713),
]
res = {row[0]: show(*row) for row in table2}
t2_adj = res["T2 adjusted*, stroke/TIA any territory"]
print("(*) Table 2 footnote: corrected for age and systolic blood pressure.")
p_side = 2 * norm.sf(abs(np.log(0.70) / res["T2 adjusted, stroke any territory"]["se"]))
print(f"side note, adjusted stroke any territory: P implied by 0.70 with the CI's SE = {p_side:.3f}")
print("\n== 4. What the abstract's multivariable interval would have to be ==")
ci_implied = np.exp(np.log(1.30) + np.array([-Z, Z]) * t2_adj["se"])
print(f"1.30 with the Table 2 adjusted SE ({t2_adj['se']:.3f}) -> 95% CI "
f"{ci_implied[0]:.2f}-{ci_implied[1]:.2f}")
ci_uni_se = np.exp(np.log(1.30) + np.array([-Z, Z]) * abs_uni["se"])
print(f"1.30 with the univariable SE ({abs_uni['se']:.3f}) -> 95% CI "
f"{ci_uni_se[0]:.2f}-{ci_uni_se[1]:.2f}")
print(f"log-width: abstract multivariable {abs_multi['logwidth']:.2f}; univariable "
f"{abs_uni['logwidth']:.2f}; Table 2 adjusted {t2_adj['logwidth']:.2f}; ratio "
f"{abs_multi['logwidth'] / abs_uni['logwidth']:.2f}x the univariable width")
print("\n== 5. Innocent explanations: does any transformation give 0.10-1.02? ==")
print(f"inverted reference group (DWI- vs DWI+): HR {1/1.30:.2f}, CI {1/2.56:.2f}-{1/0.66:.2f}")
print(f"inverted univariable: HR {1/1.50:.2f}, CI {1/2.94:.2f}-{1/0.76:.2f}")
print(f"Table 2 adjusted limits on the log scale: ln(0.66)={np.log(0.66):.2f}, "
f"ln(2.56)={np.log(2.56):.2f} (neither is 0.10 or 1.02)")
show("Table S2 Fine-Gray unadjusted, stroke/TIA any", 1.35, 0.68, 2.66)
show("Table S2 Fine-Gray adjusted, stroke/TIA any", 1.16, 0.57, 2.36)
print("any printed 0.10-1.02 pair in Table 2 or Table S2: no (checked by eye; see writeup)")
# ---------------------------------------------------------------- figure 1
ORANGE, BLUE = "#eb6834", "#2a78d6"
BG, TXT, TXT2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
plt.rcParams.update({"font.size": 10.5, "text.color": TXT, "axes.labelcolor": TXT,
"xtick.color": TXT2, "ytick.color": TXT, "svg.fonttype": "none"})
rows = [ # label, hr, lo, hi
("Abstract, multivariable\nHR 1.30 [95% CI 0.10–1.02]", 1.30, 0.10, 1.02),
("Table 2, adjusted\nHR 1.30 (0.66–2.56)", 1.30, 0.66, 2.56),
("Abstract and Table 2, univariable\nHR 1.50 (0.76–2.94)", 1.50, 0.76, 2.94),
]
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
ys = [2, 1, 0]
for (lab, hr, lo, hi), y in zip(rows, ys):
c = wald(hr, lo, hi)["centre"]
ax.plot([lo, hi], [y, y], color=ORANGE, lw=2, solid_capstyle="butt", zorder=2)
for e in (lo, hi):
ax.plot([e, e], [y - 0.08, y + 0.08], color=ORANGE, lw=2, zorder=2)
ax.plot(hr, y, "o", ms=9, color=ORANGE, zorder=3)
ax.plot(c, y - 0.26, "D", ms=8, mfc="white", mec=BLUE, mew=2, zorder=3)
if y == ys[0]:
ax.text(c * 1.17, y - 0.26, f"centre of this interval: {c:.2f}", color=BLUE,
ha="left", va="center", fontsize=10, zorder=4,
bbox=dict(facecolor=BG, edgecolor="none", pad=1.5))
else:
ax.text(c, y - 0.44, f"{c:.2f}", color=BLUE, ha="center", va="top", fontsize=10)
# annotate the inconsistency
y0 = ys[0]
ax.text(1.30 * 1.12, y0 + 0.02, "1.30 lies outside\nits own interval", color=TXT,
fontsize=10, ha="left", va="center")
ax.axvline(1.0, color=TXT2, lw=1, ls=(0, (4, 3)), zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.07, 6)
ticks = [0.1, 0.2, 0.5, 1, 2, 5]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.set_ylim(-0.75, 2.45)
ax.set_yticks(ys)
ax.set_yticklabels([r[0] for r in rows])
ax.grid(axis="x", color=GRID, lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color(TXT2)
ax.tick_params(axis="y", length=0)
ax.set_xlabel("Hazard ratio, any stroke or TIA, DWI+ vs DWI− (log scale)")
h1, = ax.plot([], [], "o-", color=ORANGE, lw=2, ms=9, label="Printed in the paper: HR and 95% CI")
h2, = ax.plot([], [], "D", ms=8, mfc="white", mec=BLUE, mew=2,
label="Recomputed: geometric centre of the printed CI")
ax.legend(handles=[h1, h2], loc="lower left", bbox_to_anchor=(-0.01, 1.0), ncol=1,
frameon=False, fontsize=10, handlelength=1.6)
fig.tight_layout()
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nwrote fig1.svg and fig1.png in", HERE)
Abstract reports 389 animals; the Methods list 116, and 389 sums per-context subject counts
eLife (2023) · doi:10.7554/eLife.87008 · PMC10547472
Summary
The abstract says facial behavior was coded "in 389 animals". The Methods list the study subjects by species and sex, and they add up to 116 adults (32 rhesus, 46 Barbary, 38 crested macaques). Table 2 agrees with the Methods: no per-context subject count is larger than the Methods headcount for its species. The figure 389 is exactly the sum of Table 2's "N subjects" column over all twelve species-by-context rows, so it counts each animal once for every context it was coded in. The study's analyses do not use this number and its conclusions are unaffected, but anyone who reads only the abstract will take the sample to be 3.35 times as large as it is.
What the paper reports
Abstract:
We coded facial behavior in >3000 social interactions across three social contexts (aggressive, submissive, affiliative) in 389 animals, using the Facial Action Coding System for macaques (MaqFACS).
Methods, "Study subjects and data collection", first paragraph:
Behavioral data and video recordings were collected on one adult male and 31 adult female rhesus macaques (M. mulatta), on 18 adult male and 28 adult female Barbary macaques (M. sylvanus), and 17 adult male and 21 adult female crested macaques (M. nigra).
Methods, "Facial behavior and social context coding":
We coded facial behavior of adult individuals but included their interactions with any other group member regardless of age or sex.
Table 2 shows the number of social interactions per species and context from which FACS codes were made.
Table 2, "Total number of social interactions per species and social context that were MaqFACS coded":
Species Context N interactions N subjects Duration (s) Rhesus Affiliative 193 29 1197 Rhesus Aggressive 413 32 2050 Rhesus Submissive 318 31 1262 Rhesus Unclear 121 30 802 Barbary Affiliative 683 43 4897 Barbary Aggressive 585 44 2128 Barbary Submissive 529 34 1890 Barbary Unclear 603 45 3500 Crested Affiliative 241 35 1918 Crested Aggressive 62 23 284 Crested Submissive 25 18 115 Crested Unclear 107 25 684
Discussion, on the comparison that limits inference:
Finally, effectively, our comparison is limited to three species which is a small sample.
What the paper's own numbers show
Method. I added the Methods counts by species and sex to get the number of distinct adults. For each species I summed the four Table 2 "N subjects" cells and found the largest of them. I then compared the sums with the abstract's 389, and the largest cell with the Methods headcount. As a separate check I summed the Table 2 interaction counts against the abstract's ">3000". Last, I counted animals in the authors' public data file (the file the paper's data availability statement points to): distinct animals per species, and the number of contexts each animal was coded in. Every number below is printed by recompute.py.txt. The data-file rows need the file itself: run python recompute.py.txt --data <path or URL>, where the URL is maqfacs.txt at the root of the GitHub repository the paper cites (Methods, "Specificity bias correction"), in raw form https://raw.githubusercontent.com/<owner>/macaque-facial-complexity/main/maqfacs.txt with <owner> taken from that citation. Its sha256 is printed and quoted under Status.
| Quantity | Location | Reported | Recomputed from the paper's numbers |
|---|---|---|---|
| Animals in the study | Abstract | 389 animals | 116 distinct adults (Methods) |
| Rhesus adults | Methods | 1 male + 31 female | 32 |
| Barbary adults | Methods | 18 male + 28 female | 46 |
| Crested adults | Methods | 17 male + 21 female | 38 |
| Total distinct adults | Methods | not stated | 32 + 46 + 38 = 116 |
| Rhesus, sum of N subjects over four contexts | Table 2 | 29, 32, 31, 30 | 122 (3.81 × the 32 adults) |
| Barbary, sum of N subjects over four contexts | Table 2 | 43, 44, 34, 45 | 166 (3.61 × the 46 adults) |
| Crested, sum of N subjects over four contexts | Table 2 | 35, 23, 18, 25 | 101 (2.66 × the 38 adults) |
| Sum of all 12 N-subjects cells | Table 2 | — | 122 + 166 + 101 = 389, identical to the abstract |
| Same sum without the Unclear rows | Table 2 | — | 289, so 389 is not a three-context total |
| Largest per-context N subjects, by species | Table 2 | 32 / 45 / 35 | at most the Methods headcount (32 / 46 / 38), as it must be |
| Abstract count relative to headcount | Abstract vs Methods | 389 | 389 / 116 = 3.35; 273 more animals than were studied |
| Interactions, three named contexts | Abstract vs Table 2 | >3000 | 924 + 1797 + 328 = 3049 (3880 with Unclear); consistent |
| Distinct animals in the authors' data file | Data file maqfacs.txt |
— | 32 rhesus, 46 Barbary, 40 crested = 118. Crested: 17 male, 22 female, 1 with no sex recorded (Methods: 17 male, 21 female). Rhesus and Barbary match the Methods by sex |
| Per-context distinct subjects in the data file | Data file | — | Reproduces all 12 Table 2 N-subjects cells and durations (to 0.5 s) once the file's "agonistic" label is merged into "unclear"; they sum to 389 |
| Contexts per animal in the data file | Data file | — | Rhesus: 27 animals in 4 contexts, 4 in 3, 1 in 2. Barbary: 29, 16, 1. Crested: 10 in 4, 13 in 3, 5 in 2, 12 in 1. Sum over animals of contexts = 389 |
The arithmetic leaves one reading. The abstract's 389 is the total of the "N subjects" column, a column that counts each animal once per context. Taken as a headcount, the paper's own Methods and Table 2 give 116 adults (118 animals in the published data file).
Figure
Figure 1. On the left is the abstract's sentence giving the sample as 389 animals (orange: the value the paper reports). On the right is the Methods list of adults by species and sex, which sums to 116 (blue: recomputed from the paper's own counts). The lower panel gives Table 2's four per-context "N subjects" values for each species. Each value is no larger than the Methods headcount for its species, which is consistent with each one counting distinct animals, and together they add up to 122 + 166 + 101 = 389. The authors' data file settles the reading: it reproduces all 12 cells, and the number of contexts per animal sums to 389 over 118 animals. The abstract's figure is therefore the number of subject-context pairs, not the number of animals. All values are taken from the paper or printed by recompute.py.txt.
Figure 2. For each species, the open circles are the four Table 2 per-context subject counts, stacked vertically (affiliative, aggressive, submissive, unclear, top to bottom) so that close values such as 29 to 32 do not hide one another. The blue vertical line is the Methods headcount (32, 46, 38) and the orange circle is the sum of the four context counts (122, 166, 101; together 389). Every context count lies at or below the headcount, which is consistent with each one counting distinct animals (the data-file check confirms it). Their sum is 2.7 to 3.8 times the headcount (3.81 rhesus, 3.61 Barbary, 2.66 crested) because most animals appear in more than one context. The axis starts at zero.
Why it matters
The error is in the abstract, the part of a paper that is read, indexed and copied most widely. A reader, reviewer or meta-analyst who takes the sample size from the abstract or from a database built from it will record N = 389. The correct figure is about 116 (118 in the published data file), so the study would look 3.35 times as large as it is. That matters most when studies are weighed or compared by number of individuals, and in particular for the rhesus sample: 32 animals from one breeding group, 31 of them female, a limitation the Methods acknowledge.
The inferential results are not affected. Entropy, context specificity and the random-forest classifier are computed from 500 ms time blocks within interactions, and the species comparison rests on three species, as the Discussion states. The headcount enters none of these calculations, so every reported estimate and its direction stand. The abstract's other count, ">3000 social interactions", is consistent with Table 2 (3049 in the three named contexts).
Explanations considered
- Interaction partners counted as animals. The Methods say partners could be "any other group member regardless of age or sex", so more than 116 individuals may appear in the videos. But the abstract says facial behavior was coded "in 389 animals", and the Methods say facial behavior was coded only for adults. The paper gives no group sizes or partner counts that add up to 389, so this does not account for the number.
- Another population or time point. Each species has one sample in the Methods: one rhesus breeding group, one Barbary group and two wild crested groups. Table 2's per-context subject counts never exceed the Methods headcounts. Nothing in the paper describes a larger sample.
- A three-context total. The abstract mentions three contexts. Without the Unclear rows the N-subjects sum is 289, not 389, so the figure is not a three-context total either. Only the full twelve-cell sum matches.
- A text-conversion or version artifact. The values are plain integers, and the PMC archive, Europe PMC, the eLife version of record and all five bioRxiv preprint versions carry the same abstract sentence and the same Methods counts. The authors' data file reproduces every Table 2 subject count and duration. So this is not an artifact of conversion or of one version.
- A small difference in the crested count. The data file has 40 distinct crested animals (17 male, 22 female, 1 with no sex recorded) and the Methods list 38 (17 male + 21 female). It does not matter here: on any version the headcount is 116 to 118, not 389.
None of these explains the number. Adding up the per-context subject counts reproduces it exactly.
Status
As of 2026-10-23:
- Version of record. I checked the eLife version of record (published 2023-10-03,
status: vorin the eLife article API). Its abstract has the same sentence ("… in 389 animals …"), and its Methods counts and Table 2 match the PMC archive. Europe PMC's full text and all five bioRxiv preprint versions (2022-12-07 to 2023-08-16) also say "389 animals". - Supplements. Supplementary file 1 lists the Action Units and Action Descriptors, and the other file is the MDAR reporting checklist. Neither gives subject counts. The public data file
maqfacs.txt(sha2566e68e6e1…a5f, 41 454 rows of 500 ms blocks) gives 118 distinct animals and reproduces Table 2. - Corrections. Crossref update metadata, PMC and Europe PMC list no correction, erratum, expression of concern or retraction (
src/known.py). - PubPeer. A web search found no PubPeer thread for the DOI or the title. PubPeer's own search refused automated access (HTTP 403, human verification required), so I could not query it directly.
- Sources. PMC10547472 (text and Table 2); api.elifesciences.org/articles/87008; Europe PMC full-text XML; bioRxiv doi:10.1101/2022.12.07.519469; Crossref.
Suggested correction
Abstract, change "in 389 animals" to "in 116 adult animals" (or the authors' deduplicated count, 118 if the data file's 40 crested individuals are the correct figure). If a per-context total is wanted, it could be stated as such, for example: "116 adults (389 subject-context combinations)". Reconciling the crested count, 38 in the Methods and 40 in the data file, would make the Methods and the data file agree.
Recomputation script (python recompute.py.txt)
"""Recompute the sample size stated in the abstract of PMC10547472
(eLife 2023;12:RP87008, doi:10.7554/eLife.87008).
Run: python recompute.py.txt (paper's own numbers + figures)
python recompute.py.txt --data <path or URL> (also check the authors' data file)
The data file is maqfacs.txt at the root of the GitHub repository whose URL the paper
gives in Methods, "Specificity bias correction" (https://github.com/<owner>/macaque-facial-complexity);
its raw URL is https://raw.githubusercontent.com/<owner>/macaque-facial-complexity/main/maqfacs.txt.
Needs numpy and matplotlib; pandas only for the optional data-file check.
Prints every recomputed number used in writeup.md and writes, next to this file:
fig1.svg/png, fig2.svg/png (figures used in writeup.md)
candidates/cand_A..cand_E.svg/png (five alternative designs for fig1)
Inputs are copied from the paper as archived in PubMed Central (identical in the
eLife version of record):
Abstract: ">3000 social interactions ... in 389 animals"
Methods, "Study subjects and data collection": adults per species and sex
Table 2: N interactions, N subjects and duration per species x context
Optional: the authors' data file maqfacs.txt (file at the root of the public GitHub
repository given in the paper's data availability statement; sha256 printed below),
used only to count distinct animals and contexts per animal. Candidate E needs it.
"""
import os
import sys
import urllib.request
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, Rectangle # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
CAND = os.path.join(HERE, "candidates")
os.makedirs(CAND, exist_ok=True)
# ---------------------------------------------------------------- inputs (from the paper)
ABSTRACT_ANIMALS = 389
ABSTRACT_INTERACTIONS_FLOOR = 3000 # ">3000"
SPECIES = ("Rhesus", "Barbary", "Crested")
METHODS = {"Rhesus": (1, 31), "Barbary": (18, 28), "Crested": (17, 21)} # (male, female) adults
CONTEXTS = ("Affiliative", "Aggressive", "Submissive", "Unclear")
T2 = { # Table 2: (N interactions, N subjects, duration s)
"Rhesus": {"Affiliative": (193, 29, 1197), "Aggressive": (413, 32, 2050),
"Submissive": (318, 31, 1262), "Unclear": (121, 30, 802)},
"Barbary": {"Affiliative": (683, 43, 4897), "Aggressive": (585, 44, 2128),
"Submissive": (529, 34, 1890), "Unclear": (603, 45, 3500)},
"Crested": {"Affiliative": (241, 35, 1918), "Aggressive": (62, 23, 284),
"Submissive": (25, 18, 115), "Unclear": (107, 25, 684)},
}
# ---------------------------------------------------------------- style
BG, INK, INK2, GRID = "#fcfcfb", "#0b0b0b", "#52514e", "#e4e3df"
ORANGE, BLUE, NEUTRAL = "#eb6834", "#2a78d6", "#b9b7b1"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2,
"ytick.color": INK2, "figure.facecolor": BG, "axes.facecolor": BG,
"savefig.facecolor": BG, "svg.fonttype": "path", "lines.linewidth": 2,
})
W = 7.2
def save(fig, path_noext):
fig.savefig(path_noext + ".svg", bbox_inches="tight", pad_inches=0.12)
fig.savefig(path_noext + ".png", dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# ---------------------------------------------------------------- recomputation
print("== Methods: adults per species ==")
adults = {}
for sp in SPECIES:
m, f = METHODS[sp]
adults[sp] = m + f
print(f"{sp:8s} {m} male + {f} female = {adults[sp]}")
total_adults = sum(adults.values())
print(f"Total distinct adults (Methods): {total_adults}")
print("\n== Table 2: N subjects per context ==")
subj_sum, subj_max = {}, {}
for sp in SPECIES:
vals = [T2[sp][c][1] for c in CONTEXTS]
subj_sum[sp] = sum(vals)
subj_max[sp] = max(vals)
print(f"{sp:8s} {' + '.join(map(str, vals))} = {subj_sum[sp]}; max {subj_max[sp]}"
f" <= Methods {adults[sp]}: {subj_max[sp] <= adults[sp]};"
f" sum / adults = {subj_sum[sp] / adults[sp]:.2f}")
total_subj = sum(subj_sum.values())
print(f"Sum over all 12 species x context cells: "
f"{' + '.join(str(subj_sum[s]) for s in SPECIES)} = {total_subj}")
print(f"Equals the abstract's {ABSTRACT_ANIMALS}: {total_subj == ABSTRACT_ANIMALS}")
no_unclear = sum(T2[s][c][1] for s in SPECIES for c in CONTEXTS if c != "Unclear")
print(f"Sum excluding the Unclear rows: {no_unclear}")
print(f"Ratio {ABSTRACT_ANIMALS} / {total_adults} = {ABSTRACT_ANIMALS / total_adults:.2f}")
print(f"Overstatement: {ABSTRACT_ANIMALS - total_adults} animals")
print("\n== Table 2: interactions ==")
ni_all = sum(T2[s][c][0] for s in SPECIES for c in CONTEXTS)
ni_3 = sum(T2[s][c][0] for s in SPECIES for c in CONTEXTS if c != "Unclear")
for sp in SPECIES:
print(f"{sp:8s} three named contexts {sum(T2[sp][c][0] for c in CONTEXTS[:3])};"
f" all four {sum(T2[sp][c][0] for c in CONTEXTS)}")
print(f"Three named contexts: {ni_3} (> {ABSTRACT_INTERACTIONS_FLOOR}: "
f"{ni_3 > ABSTRACT_INTERACTIONS_FLOOR}); all four contexts: {ni_all}")
# ---------------------------------------------------------------- optional: authors' data file
data = None
DATA_SRC = None
if "--data" in sys.argv:
i = sys.argv.index("--data")
if i + 1 < len(sys.argv) and not sys.argv[i + 1].startswith("--"):
DATA_SRC = sys.argv[i + 1]
else:
print("\n[--data needs an argument: a path or URL to maqfacs.txt]")
if DATA_SRC is None:
print("\n[data-file check not run: pass --data <path or URL to maqfacs.txt>]")
else:
try:
import hashlib
import io
import pandas as pd
if DATA_SRC.startswith("http"):
raw = urllib.request.urlopen(DATA_SRC, timeout=60).read()
else:
with open(DATA_SRC, "rb") as fh:
raw = fh.read()
print("\n== Authors' data file maqfacs.txt ==")
print(f"sha256 {hashlib.sha256(raw).hexdigest()} ({len(raw)} bytes)")
df = pd.read_csv(io.BytesIO(raw), sep="\t")
print(f"rows (500 ms blocks): {len(df)}")
print("context labels:", {k: int(v) for k, v in df["context"].value_counts().items()})
# Table 2's 'Unclear' row = the file's 'unclear' + 'agonistic' labels
df["ctx"] = df["context"].replace({"agonistic": "unclear"})
df["species"] = df["species"].str.capitalize()
ids = df.drop_duplicates(["species", "subject"])
n_animals = ids.groupby("species").size()
sex = ids.groupby(["species", "sex"], dropna=False).size()
print("distinct animals per species:", {k: int(v) for k, v in n_animals.items()}, "total", int(n_animals.sum()))
print("by sex (NaN = sex missing):")
print(sex.to_string())
g = df.groupby(["species", "ctx"]).agg(nsub=("subject", "nunique"), blocks=("block", "size"))
g["dur_s"] = g["blocks"] / 2
print("per species x context: distinct subjects, duration (s) = blocks / 2")
ok = True
for sp in SPECIES:
for c in CONTEXTS:
r = g.loc[(sp, c.lower())]
t = T2[sp][c]
match = int(r.nsub) == t[1] and abs(r.dur_s - t[2]) <= 0.5
ok &= match
print(f" {sp:8s} {c:12s} N subjects {int(r.nsub):3d} (Table 2 {t[1]:3d})"
f" duration {r.dur_s:7.1f} (Table 2 {t[2]:5d}) match: {match}")
print(f"All 12 N-subjects cells and durations reproduced (to 0.5 s): {ok}")
print(f"Sum of per-context distinct subjects in the data: {int(g.nsub.sum())}")
k = df.groupby(["species", "subject"])["ctx"].nunique()
dist = k.groupby(level=0).value_counts().unstack(fill_value=0)
print("animals by number of contexts they were coded in:")
print(dist.to_string())
print(f"sum over animals of contexts = {int(k.sum())}; distinct animals = {len(k)}")
data = {sp: sorted(k.loc[sp].tolist(), reverse=True) for sp in SPECIES}
except Exception as e: # network or pandas missing: figures from the paper still build
print(f"\n[data-file check skipped: {e!r}]")
# ---------------------------------------------------------------- figure helpers
def text_panel(ax, x, y, w, h, title, body, edge, title_color=INK):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=edge, lw=2, transform=ax.transAxes))
ax.text(x + 0.02, y + h - 0.045, title, transform=ax.transAxes, ha="left", va="top",
fontsize=10.5, color=title_color, fontweight="bold")
ax.text(x + 0.02, y + h - 0.15, body, transform=ax.transAxes, ha="left", va="top",
fontsize=10, color=INK, linespacing=1.45)
# ---- Candidate A: two text panels joined by an annotated connector (Abstract vs Methods/Table 2)
def cand_A(path):
fig = plt.figure(figsize=(W, 4.4))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off")
text_panel(ax, 0.01, 0.40, 0.38, 0.58, "Abstract (reported)",
"“We coded facial behavior in\n>3000 social interactions across\n"
"three social contexts (aggressive,\nsubmissive, affiliative) in\n"
"389 animals”", ORANGE)
rows = "\n".join(f"{sp:<8s} {METHODS[sp][0]:>2d} M + {METHODS[sp][1]:>2d} F = {adults[sp]:>3d}"
for sp in SPECIES)
text_panel(ax, 0.55, 0.40, 0.44, 0.58, "Methods, Study subjects",
rows + f"\n{'':23s}───\n{'Adults':<21s}= {total_adults:>3d}",
BLUE)
ax.texts[-1].set_family("DejaVu Sans Mono")
ax.annotate("", xy=(0.55, 0.67), xytext=(0.39, 0.67), xycoords="axes fraction",
arrowprops=dict(arrowstyle="<->", color=INK2, lw=2))
ax.text(0.47, 0.76, "conflict", ha="center", va="bottom", fontsize=10.5, color=INK,
fontweight="bold", transform=ax.transAxes)
ax.text(0.47, 0.58, f"389 vs {total_adults}\n= {ABSTRACT_ANIMALS / total_adults:.2f}×",
ha="center", va="top", fontsize=10, color=INK2, transform=ax.transAxes)
t2 = "\n".join(f"{sp:<8s} {' + '.join(f'{T2[sp][c][1]:2d}' for c in CONTEXTS)} = {subj_sum[sp]}"
for sp in SPECIES)
ax.add_patch(FancyBboxPatch((0.01, 0.01), 0.98, 0.33, boxstyle="round,pad=0,rounding_size=0.012",
fc="white", ec=NEUTRAL, lw=2, transform=ax.transAxes))
ax.text(0.03, 0.31, "Table 2, ‘N subjects’ per context\n(affil. + aggr. + subm. + unclear)",
fontsize=10.5, fontweight="bold", va="top", transform=ax.transAxes, linespacing=1.35)
ax.text(0.03, 0.17, t2, fontsize=10, va="top", transform=ax.transAxes, family="DejaVu Sans Mono",
linespacing=1.35)
ax.text(0.50, 0.29, f"122 + 166 + 101 = {total_subj}, the abstract's figure.\n"
"Each cell is ≤ the Methods headcount for its\nspecies; the authors' data file confirms "
"that the\nsum counts each animal once per context\nit was coded in (up to four times).",
fontsize=10, color=INK2, va="top", transform=ax.transAxes, linespacing=1.45)
save(fig, path)
# ---- Candidate B: two stacked bars on one count axis, segmented by species
def cand_B(path):
fig, ax = plt.subplots(figsize=(W, 2.9))
bars = [("Abstract: “389 animals”\n(= sum of Table 2 N subjects)", subj_sum, ORANGE),
("Methods: adults enrolled\n(distinct animals; same order)", adults, BLUE)]
for i, (lab, vals, col) in enumerate(bars):
y, left = 1 - i, 0
for sp in SPECIES:
v = vals[sp]
ax.barh(y, v - 2, left=left + 1, height=0.52, color=col, edgecolor="none")
ax.text(left + v / 2, y, f"{v}", ha="center", va="center", fontsize=10.5,
color="white", fontweight="bold")
if i == 0:
ax.text(left + v / 2, y - 0.34, sp, ha="center", va="top", fontsize=10, color=INK2)
left += v
ax.text(left + 6, y, f"{left}", ha="left", va="center", fontsize=11, fontweight="bold")
ax.set_yticks([1, 0], [b[0] for b in bars], fontsize=10)
ax.set_xlim(0, 420); ax.set_ylim(-0.75, 1.45)
ax.set_xlabel("Count (starts at zero)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.set_title(f"The abstract's count is {ABSTRACT_ANIMALS / total_adults:.2f}× the number of "
"animals listed in the Methods", fontsize=11, loc="left", color=INK)
save(fig, path)
# ---- Candidate C: per-species dot plot, contexts vs headcount vs sum
# Context circles are stacked vertically (affiliative, aggressive, submissive, unclear,
# top to bottom) so that close values such as 29-32 or 43-45 do not hide one another.
OFFS = (0.33, 0.11, -0.11, -0.33)
def cand_C(path):
fig, ax = plt.subplots(figsize=(W, 3.6))
for i, sp in enumerate(SPECIES):
y = 2 - i
vals = [T2[sp][c][1] for c in CONTEXTS]
ax.plot([0, subj_sum[sp]], [y, y], color=GRID, lw=2, zorder=1)
ax.scatter(vals, [y + o for o in OFFS], s=60, facecolor="white", edgecolor=INK2, lw=1.5,
zorder=3, label="Table 2 N subjects, one circle per context\n"
"(stacked: affiliative, aggressive, submissive, unclear)" if i == 0 else None)
ax.plot([adults[sp]] * 2, [y - 0.44, y + 0.44], color=BLUE, lw=2.5, zorder=4,
solid_capstyle="butt",
label="Methods: adults (distinct animals)" if i == 0 else None)
ax.scatter([subj_sum[sp]], [y], s=110, marker="o", color=ORANGE, zorder=4, ec=BG, lw=1.5,
label="Sum of the four context counts (adds to the abstract's 389)" if i == 0 else None)
ax.text(adults[sp] + 2, y + 0.30, f"{adults[sp]}", ha="left", va="center", fontsize=10,
color=BLUE, fontweight="bold")
ax.text(subj_sum[sp], y + 0.22, f"{subj_sum[sp]}", ha="center", va="bottom", fontsize=10,
color=INK)
ax.set_yticks([2, 1, 0], SPECIES)
ax.set_xlim(0, 180); ax.set_ylim(-0.55, 2.6)
ax.set_xlabel("Number of animals (starts at zero)")
ax.grid(axis="x", color=GRID, lw=0.8); ax.set_axisbelow(True)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.tick_params(axis="y", length=0)
ax.legend(loc="upper center", bbox_to_anchor=(0.5, -0.22), ncol=1, frameon=False, fontsize=10)
ax.set_title("Every context count fits under the headcount; their sum does not",
fontsize=11, loc="left")
save(fig, path)
# ---- Candidate D: calculation-flow diagram, two parallel chains
def cand_D(path):
fig = plt.figure(figsize=(W, 3.6))
ax = fig.add_axes([0, 0, 1, 1]); ax.axis("off"); ax.set_xlim(0, 100); ax.set_ylim(1, 60)
def box(x, y, w, h, txt, ec, bold=False, fs=10):
ax.add_patch(FancyBboxPatch((x, y), w, h, boxstyle="round,pad=0,rounding_size=0.8",
fc="white", ec=ec, lw=2))
ax.text(x + w / 2, y + h / 2, txt, ha="center", va="center", fontsize=fs,
fontweight="bold" if bold else "normal")
def arrow(x0, y0, x1, y1):
ax.annotate("", xy=(x1, y1), xytext=(x0, y0),
arrowprops=dict(arrowstyle="-|>", color=INK2, lw=1.6, mutation_scale=12))
ax.text(1, 57, "Table 2: N subjects per species × context (12 cells)", fontsize=10.5,
fontweight="bold", va="top")
ax.text(1, 27, "Methods: adults per species and sex", fontsize=10.5, fontweight="bold", va="top")
for i, sp in enumerate(SPECIES):
y = 44 - i * 7
box(1, y, 40, 5.5, f"{sp}: " + " + ".join(str(T2[sp][c][1]) for c in CONTEXTS), NEUTRAL)
arrow(41.5, y + 2.75, 49.5, y + 2.75)
box(50, y, 12, 5.5, str(subj_sum[sp]), NEUTRAL)
arrow(62.5, y + 2.75, 72.5, 39.75)
y2 = 15 - i * 7
box(1, y2, 40, 5.5, f"{sp}: {METHODS[sp][0]} M + {METHODS[sp][1]} F", NEUTRAL)
arrow(41.5, y2 + 2.75, 49.5, y2 + 2.75)
box(50, y2, 12, 5.5, str(adults[sp]), NEUTRAL)
arrow(62.5, y2 + 2.75, 72.5, 10.75)
box(73, 36, 26, 7.5, f"{total_subj}\nsubject-context pairs", ORANGE, bold=True)
box(73, 7, 26, 7.5, f"{total_adults}\ndistinct adults", BLUE, bold=True)
ax.text(86, 47, "Abstract: “in 389 animals” (reported)", ha="center", fontsize=10,
color=INK)
ax.text(86, 3.2, "Methods headcount (recomputed)", ha="center", fontsize=10, color=INK2)
ax.annotate("", xy=(86, 15.2), xytext=(86, 35.3),
arrowprops=dict(arrowstyle="<->", color=INK2, lw=1.6))
ax.text(87.5, 25.2, f"{ABSTRACT_ANIMALS / total_adults:.2f}×", fontsize=10.5,
fontweight="bold", va="center")
save(fig, path)
# ---- Candidate E: unit chart from the authors' data file, one column per animal
def cand_E(path):
if data is None:
print("[candidate E skipped: needs the authors' data file]")
return False
fig = plt.figure(figsize=(W, 3.3))
ax = fig.add_axes([0.0, 0.14, 1.0, 0.68])
cs, gap = 1.0, 0.18
for i, sp in enumerate(SPECIES):
y0 = (2 - i) * 6.2
for j, k in enumerate(data[sp]):
for r in range(k):
ax.add_patch(Rectangle((j * cs + gap / 2, y0 + r * cs + gap / 2), cs - gap, cs - gap,
fc=ORANGE, ec="none"))
n = len(data[sp])
ax.plot([0, n * cs], [y0 - 0.35, y0 - 0.35], color=BLUE, lw=2.5)
ax.text(-1, y0 + 2, sp, ha="right", va="center", fontsize=10.5)
ax.text(n * cs + 1, y0 + 2.6, f"{sum(data[sp])} squares", ha="left", va="center",
fontsize=10, color=INK)
ax.text(n * cs + 1, y0 + 1.0, f"{n} animals", ha="left", va="center", fontsize=10,
color=INK2)
ax.set_xlim(-10, 62); ax.set_ylim(-1.0, 16.6); ax.axis("off")
fig.text(0.02, 0.97, "Authors' data file: each column is one animal (blue rule spans the animals);\n"
"each orange square is one context that animal was coded in (1–4).", fontsize=10,
va="top", linespacing=1.4)
tot_sq = sum(sum(v) for v in data.values())
tot_an = sum(len(v) for v in data.values())
fig.text(0.02, 0.03, f"Total: {tot_an} animals; {tot_sq} squares, which is the abstract's "
"“389 animals”", fontsize=10.5, fontweight="bold", va="bottom")
save(fig, path)
return True
builders = {"A": cand_A, "B": cand_B, "C": cand_C, "D": cand_D, "E": cand_E}
for key, fn in builders.items():
fn(os.path.join(CAND, f"cand_{key}"))
# Figures used in writeup.md (provisional choice: A as fig1, C as fig2)
cand_A(os.path.join(HERE, "fig1"))
cand_C(os.path.join(HERE, "fig2"))
print("\nFigures written to", HERE, "and", CAND)
Abstract states a 60-fold affinity difference; the paper's reported dissociation constants imply about 2-fold
eLife (2021) · doi:10.7554/eLife.48840 · PMC8257255
Summary
The abstract says that FasL binds DR5 with an affinity "approximately 60-fold lower than the affinity between TRAIL and DR5". The only affinity values in the paper are two surface plasmon resonance (SPR) dissociation constants in the Results: KD = 1.23 × 10−12 M for DR5–FasL and 6.01 × 10−13 M for DR5–TRAIL. Their ratio is 2.05, so the paper's own numbers support a roughly 2-fold difference, 29 times smaller than the abstract states. No simple slip in either number gives 60, and nothing in the deposited SPR source data gives 60 either. The abstract is where most readers learn how strong the newly proposed FasL–DR5 interaction is, so the stated magnitude matters. The direction of the comparison (FasL binds more weakly than TRAIL) is the same in both places.
What the paper reports
The PMC text flattens superscripts ("10-12"). Below they are written as superscripts, as in the eLife version of record and the Europe PMC JATS markup. The wording is otherwise verbatim.
Abstract:
Subsequent cellular and biochemical analyses revealed that DR5 interacted specifically with recombinant FasL–Fc protein, although the strength of this interaction was approximately 60-fold lower than the affinity between TRAIL and DR5.
Results, section "DR5 is a Fas-independent receptor for sFasL that promotes arthritis":
Furthermore, specific binding of DR5 to FasL was confirmed by surface plasmon resonance (KD: 1.23 × 10−12 M for DR5–FasL versus 6.01 × 10−13 M for DR5–TRAIL; Figure 1—figure supplement 4F,G)
Legend of Figure 1—figure supplement 4:
(F, G) Surface plasmon resonance assays for DR5–FasL (F) and DR5–TRAIL (G) interactions.
Materials and methods, "Surface Plasmon resonance":
To determine equilibrium affinity measurements, recombinant DR5 (R and D systems: 631-T2-100) was coupled with a CM5 sensor chip using 10 mM sodium acetate (pH 5.5), and recombinant FasL or TRAIL (R and D Systems: 126-FL-010) was passed over the chip at a flow rate of 30 μL/min for 120 s.
BIA evaluation software version 3.1 (GE Healthcare) was used for to analyze the data.
No other KD, affinity constant or fold-difference in affinity appears anywhere in the article text, figure legends or tables.
What the paper's own numbers show
Method. A lower affinity means a larger KD, so the fold by which FasL–DR5 affinity is lower than TRAIL–DR5 affinity is KD(FasL) / KD(TRAIL). I computed this ratio from the two printed values, allowed for rounding of their last digit, and worked out what either KD would have to be for the ratio to be 60. I then checked whether a single slip in a mantissa or exponent could turn the printed values into a 60-fold ratio. Finally I set the printed KDs against the analyte concentrations listed in the eLife source data for panels 4F and 4G. Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| KD, DR5–FasL | Results; Fig. 1—fig. suppl. 4F | 1.23 × 10−12 M | (input) |
| KD, DR5–TRAIL | Results; Fig. 1—fig. suppl. 4G | 6.01 × 10−13 M | (input) |
| Fold lower affinity, FasL vs TRAIL | Abstract | approximately 60-fold | 2.05 (1.23 × 10−12 / 6.01 × 10−13 = 2.0466; 2.037 to 2.057 allowing for rounding) |
| Abstract value ÷ implied value | derived | — | 60 / 2.0466 = 29.3 |
| KD(FasL) that a 60-fold ratio would need, if KD(TRAIL) is as printed | derived | — | 3.61 × 10−11 M (29.3 × the printed value) |
| KD(TRAIL) that a 60-fold ratio would need, if KD(FasL) is as printed | derived | — | 2.05 × 10−14 M (29.3 × below the printed value) |
| Ratios under single-digit or single-exponent slips | derived | — | 0.205, 10, 20.5 or 205; none near 60 |
| Analyte concentrations injected | Figure 1 source data 1, panels 4F and 4G | FasL 125, 250, 500, 1000 nM; TRAIL 15.625 to 250 nM | lowest concentration ÷ printed KD: 1.0 × 105 (FasL), 2.6 × 104 (TRAIL) |
The two printed constants share the same unit (M), so the ratio is dimensionless and does not depend on unit conventions. It also does not depend on whether one speaks of "affinity" as 1/KD: the TRAIL-to-FasL affinity ratio is still 2.05, and FasL's affinity is 0.489 times TRAIL's. To reach 60 one of the two constants would have to be about 29 times different from its printed value, and no change of one digit or one power of ten in either does that.
Figure
Figure 1. Top: the two passages that make the same comparison. Orange text marks values as printed in the paper; blue marks the ratio computed from them. Bottom: the same comparison on a log fold axis, with the thin line at 1 for equal affinity. The ratio of the printed KDs is 2.05 (blue circle). The abstract states approximately 60 (orange diamond). The two differ by a factor of 29. A log axis is used because the quantity is a ratio; it shows the distance between the two values at its true proportion.
Figure 2. Concentration on a log axis. Orange diamonds: the KDs printed in the Results. Blue open circles: the KD each interaction would need for the abstract's 60-fold figure to hold if the other KD is as printed (3.61 × 10−11 M for FasL; 2.05 × 10−14 M for TRAIL). Grey bars with tick marks: the analyte concentrations injected, taken from the eLife source data for Figure 1—figure supplement 4F and 4G. A 60-fold difference would need one constant to move by a factor of 29. The figure also shows that both printed KDs lie four to five orders of magnitude below the lowest concentration injected. This bears on how well the KDs are determined (see Explanations considered).
Why it matters
- A headline characterization of the paper's central new interaction is not supported by its own numbers. The abstract gives readers one quantitative statement about how strongly FasL binds its proposed new receptor, and that statement is 29 times larger than the ratio of the constants the paper reports. A reader of the abstract would conclude that FasL is a much weaker DR5 ligand than TRAIL. A reader of the Results would conclude that the two bind DR5 about equally well (a 2-fold difference, both in the picomolar range).
- The magnitude bears on interpretation. Whether sFasL can act on DR5 at the concentrations present in inflamed joints, and whether TRAIL would compete with it, depends on how much weaker FasL binds. The Discussion notes that "differences in CX3CL1 production may be attributable to differences in binding affinity". A 60-fold and a 2-fold difference support different readings of that sentence.
- What is not affected. The direction of the comparison is the same in the abstract and the Results. The flow-cytometry, knockdown, knockout, immunoprecipitation and in vivo arthritis experiments do not use either KD, and their conclusions do not depend on the size of the affinity difference.
- Reach. Crossref lists 16 citing works as of 2026-10-24.
Explanations considered
- Direction or definition of "affinity". Does not account for it. Lower affinity means larger KD, and the FasL KD is indeed the larger one, so the abstract's direction is right. Whether one divides KDs or affinities (1/KD), the magnitude is 2.05, not 60.
- Unit mismatch. Ruled out. Both constants are printed in M in one sentence.
- A typographical slip in one constant. Not able to explain it. Changing one exponent by one gives 20.5 or 205, the mantissa ratio alone is 0.205, and the exponent difference alone is 10. A 60-fold ratio would need KD(FasL) ≈ 3.61 × 10−11 M or KD(TRAIL) ≈ 2.05 × 10−14 M, neither of which is one keystroke from the printed values.
- A conversion or rendering artifact. Ruled out. The eLife version of record prints "1.23 × 10−12 M" and "6.01 × 10−13 M" with superscript exponents and true minus signs. The Europe PMC JATS XML has the same markup. The PMC display "10-12" is only flattened superscripts.
- The 60-fold figure comes from a different comparison. Possible, but it does not rescue the sentence as published. In the eLife peer-review record (decision letter and author response, published with the article), the editor suggested abstract wording "along the lines of" the following: "Subsequent cellular and biochemical analysis revealed that DR5 can specifically interact with recombinant FasL-Fc protein, albeit with an approximately 60-fold lower affinity than Fas." The revised abstract in the author response changed the comparator: "albeit with an approximately 60-fold lower affinity than interaction between TRAIL and DR5." The number stayed at 60 while the comparator changed from Fas to TRAIL–DR5. The paper reports no FasL–Fas affinity, and no KD appears anywhere in the review record. So a FasL–Fas comparison cannot be checked from the paper, and the printed sentence ties 60-fold to TRAIL–DR5, for which the paper's constants give 2.05.
- A different quantity from the SPR data. Checked; none gives about 60. In the eLife source data for panels 4F and 4G, a global 1:1 binding model fitted to the software-fitted sensorgrams gives association rate constants ka ≈ 7.4 × 103 M−1s−1 for FasL and ≈ 7.3 × 104 M−1s−1 for TRAIL, a ratio of 9.8. (The FasL fit reproduces those curves to 0.00025 RU; the TRAIL fit is approximate, with a residual of 60 RU on a 2557 RU signal.) The peak responses at equal concentrations differ by factors of 2722 (125 nM) and 1390 (250 nM). The geometric-mean ratio of the injected concentrations is 5.66.
- The printed KDs themselves are imprecise. This is a separate observation. It does not explain the discrepancy, but it bears on how to correct it. Both printed KDs lie 2.6 × 104 to 1.0 × 105 times below the lowest concentration injected. At such concentrations the surface is effectively saturated at equilibrium (predicted occupancy at least 0.99996), so the equilibrium responses carry no information on KD. A picomolar KD would then rest on the dissociation rate. In the software-fitted curves of the source data, the dissociation phase is flat: the response at the end of dissociation is 1.00000 of its starting value for FasL over 240 s and 0.99993 to 0.99998 for TRAIL over 360 s. For FasL, fixing the dissociation rate anywhere from 10−12 to 10−6 s−1 fits those curves equally well (rms 0.00025 to 0.00026 RU), which corresponds to KD values from 1.3 × 10−16 to 1.3 × 10−10 M. The printed FasL KD together with the fitted ka implies kd ≈ 9.1 × 10−9 s−1, or about 2 × 10−6 of the bound complex dissociating during the 240 s window. The deposited data are therefore consistent with the printed FasL KD but do not determine it, and the same appears to hold for TRAIL. So the 2.05 ratio is what the paper reports, not necessarily the true affinity ratio. A smaller discrepancy: the Methods give a 120 s injection for both analytes, while the TRAIL sensorgrams in the source data peak at 100 s.
- A correction already issued. None found (see Status).
Status
- Version of record. The eLife version of record (elifesciences.org/articles/48840, Version of Record dated 5 July 2021) has the same abstract sentence and the same KD values as the PMC archive and the Europe PMC full-text XML. Every quotation from the article was checked verbatim against the PMC-archived text with
src/show.py. The peer-review quotations were checked against the decision letter and author response in the Europe PMC XML (sub-articles sa1 and sa2). - Supplements. Figure 1 source data 1 (
elife-48840-fig1-data1-v1.xlsx, cdn.elifesciences.org), sheet "Figure 1 - figure supplement 4", holds the raw and fitted sensorgrams for panels 4F (sFasL, 125 to 1000 nM) and 4G (TRAIL, 15.625 to 250 nM). It contains no KD table and no fold-difference.recompute.py.txtreads it (section 4) and prints every source-data number used above. Nothing in it reconciles 60-fold with the printed constants. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, retraction or expression of concern.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The Crossref record has empty update-to and updated-by fields; its relations list only the GEO dataset GSE110343 and the two peer-review documents. The eLife article page shows no correction notice. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC8257255; eLife version of record https://elifesciences.org/articles/48840 and its peer-review record; Europe PMC full-text XML; Figure 1 source data 1; Crossref; Europe PMC; PubPeer; https://doi.org/10.7554/eLife.48840.
Suggested correction
The authors should say which measurement supports the abstract's fold-difference, then make the abstract and the Results agree. If the SPR constants in the Results are the intended basis, the abstract sentence could read:
Subsequent cellular and biochemical analyses revealed that DR5 interacted specifically with recombinant FasL–Fc protein, although the strength of this interaction was approximately 2-fold lower than the affinity between TRAIL and DR5.
If the 60-fold figure comes from another measurement or comparison (for example FasL–Fas, as in the wording suggested during review), that measurement and its value should be reported in the Results and the abstract's comparator corrected to match. Because the deposited sensorgrams show essentially no dissociation and the printed picomolar KDs lie 104 to 105 times below the lowest concentration injected, the authors may also wish to report ka, kd and their uncertainties, or the range of KD that the data can support. They may also wish to correct the injection time given in the Methods for TRAIL (100 s in the source data, 120 s in the text).
Recomputation script (python recompute.py.txt)
"""Recompute the affinity comparison in PMC8257255
(eLife 2021;10:e48840, doi:10.7554/eLife.48840,
"Soluble Fas ligand drives autoantibody-induced arthritis by binding to DR5/TRAIL-R2").
Run: python recompute.py.txt (sections 1-3 and both figures; numpy + matplotlib)
python recompute.py.txt --no-source-data (skip section 4)
Section 4 also needs scipy and openpyxl. It reads the eLife source data file
elife-48840-fig1-data1-v1.xlsx: it uses a copy next to this script if there is one,
otherwise it downloads it from cdn.elifesciences.org. If neither works it is skipped.
Writes fig1.svg/png and fig2.svg/png next to this file.
Inputs copied from the paper as archived in PMC (identical in the eLife version of record):
Abstract: "the strength of this interaction was approximately 60-fold lower than the
affinity between TRAIL and DR5"
Results: "KD: 1.23 x 10^-12 M for DR5-FasL versus 6.01 x 10^-13 M for DR5-TRAIL;
Figure 1-figure supplement 4F,G"
Source data (Figure 1 source data 1, sheet "Figure 1 - figure supplement 4"):
panel 4F sFasL 125, 250, 500, 1000 nM; panel 4G TRAIL 15.625 ... 250 nM.
"""
import os
import sys
import math
for _v in ("OMP_NUM_THREADS", "OPENBLAS_NUM_THREADS", "MKL_NUM_THREADS"):
os.environ.setdefault(_v, "2") # small fits; do not spread over every core
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt # noqa: E402
from matplotlib.patches import FancyBboxPatch, FancyArrowPatch # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
# ------------------------------------------------------------------ inputs
KD_FASL = 1.23e-12 # M, DR5-FasL (Results)
KD_TRAIL = 6.01e-13 # M, DR5-TRAIL (Results)
FOLD_ABSTRACT = 60 # "approximately 60-fold lower" (Abstract)
C_FASL_nM = [125, 250, 500, 1000] # source data, panel 4F
C_TRAIL_nM = [15.625, 31.25, 62.5, 125, 250] # source data, panel 4G
def sci(x, d=2):
e = int(math.floor(math.log10(abs(x))))
return f"{x / 10**e:.{d}f} × 10^{e}"
# ------------------------------------------------------------------ 1. the ratio
print("=== 1. Fold difference implied by the reported KDs ===")
ratio = KD_FASL / KD_TRAIL
lo = 1.225e-12 / 6.015e-13
hi = 1.235e-12 / 6.005e-13
print(f"KD(DR5-FasL) = {sci(KD_FASL)} M")
print(f"KD(DR5-TRAIL) = {sci(KD_TRAIL)} M")
print(f"KD(FasL) / KD(TRAIL) = {ratio:.4f} -> FasL affinity {ratio:.2f}-fold lower than TRAIL")
print(f" range allowed by rounding of the printed values: {lo:.3f} to {hi:.3f}")
print(f" equivalently, FasL affinity / TRAIL affinity = {1 / ratio:.3f}")
print(f"Abstract: ~{FOLD_ABSTRACT}-fold. {FOLD_ABSTRACT} / {ratio:.4f} = {FOLD_ABSTRACT / ratio:.1f}")
print("\n=== 2. What a 60-fold difference would require ===")
kd_fasl_req = FOLD_ABSTRACT * KD_TRAIL
kd_trail_req = KD_FASL / FOLD_ABSTRACT
print(f"KD(FasL) needed if KD(TRAIL) is as printed: 60 × {sci(KD_TRAIL)} = {sci(kd_fasl_req)} M "
f"({kd_fasl_req / KD_FASL:.1f} × the printed FasL KD)")
print(f"KD(TRAIL) needed if KD(FasL) is as printed: {sci(KD_FASL)} / 60 = {sci(kd_trail_req)} M "
f"({KD_TRAIL / kd_trail_req:.1f} × smaller than the printed TRAIL KD)")
print("Single-digit / single-exponent slips of the printed numbers:")
cands = {
"mantissa ratio 1.23 / 6.01": 1.23 / 6.01,
"exponent difference only, 10^1": 10.0,
"FasL exponent -11 instead of -12": 1.23e-11 / KD_TRAIL,
"FasL exponent -10 instead of -12": 1.23e-10 / KD_TRAIL,
"TRAIL exponent -14 instead of -13": KD_FASL / 6.01e-14,
"TRAIL exponent -15 instead of -13": KD_FASL / 6.01e-15,
}
for k, v in cands.items():
print(f" {k:36s} {v:9.3f}")
print(" -> none is close to 60")
print("\n=== 3. Printed KDs relative to the concentrations injected (source data) ===")
for lab, kd, cs in [("FasL", KD_FASL, C_FASL_nM), ("TRAIL", KD_TRAIL, C_TRAIL_nM)]:
cmin, cmax = min(cs) * 1e-9, max(cs) * 1e-9
occ = [c * 1e-9 / (c * 1e-9 + kd) for c in cs]
print(f"{lab}: {min(cs)}-{max(cs)} nM injected; lowest concentration / KD = {cmin / kd:.3g}; "
f"equilibrium occupancy C/(C+KD) at the lowest concentration = {occ[0]:.6f}")
gm = math.prod(C_FASL_nM) ** (1 / 4) / math.prod(C_TRAIL_nM) ** (1 / 5)
print(f"geometric-mean concentration ratio FasL/TRAIL = {gm:.2f}")
# ------------------------------------------------------------------ 4. source data (optional)
def source_data():
print("\n=== 4. Source data, Figure 1-figure supplement 4F,G (sensorgrams) ===")
try:
import openpyxl
from scipy.optimize import least_squares
except ImportError as e:
print(" skipped (needs openpyxl and scipy):", e)
return
path = os.path.join(HERE, "elife-48840-fig1-data1-v1.xlsx")
if not os.path.exists(path):
import tempfile
import urllib.request
url = "https://cdn.elifesciences.org/articles/48840/elife-48840-fig1-data1-v1.xlsx"
path = os.path.join(tempfile.mkdtemp(), "fig1data.xlsx")
try:
urllib.request.urlretrieve(url, path)
except Exception as e: # noqa: BLE001
print(" skipped (could not download source data):", e)
return
wb = openpyxl.load_workbook(path, read_only=True, data_only=True)
rows = list(wb["Figure 1 - figure supplement 4"].iter_rows(values_only=True))
start_f = next(i for i, r in enumerate(rows) if r[0] == "Figure 1 - figure supplement 4F")
start_g = next(i for i, r in enumerate(rows) if r[0] == "Figure 1 - figure supplement 4G")
def block(a, b, n):
hdr = rows[a + 1][2:2 + 2 * n]
t, Y = [], []
for r in rows[a + 2:b]:
if isinstance(r[1], (int, float)):
t.append(r[1])
Y.append([np.nan if v is None else v for v in r[2:2 + 2 * n]])
Y = np.array(Y, float)
return hdr, np.array(t, float), Y[:, 0::2], Y[:, 1::2]
hf, tf, rawf, fitf = block(start_f, start_g, 4)
ht, tt, rawt, fitt = block(start_g, len(rows), 5)
print(" 4F columns:", ", ".join(hf))
print(" 4G columns:", ", ".join(ht))
res = {}
for lab, t, raw, fit, cs in [("FasL", tf, rawf, fitf, C_FASL_nM), ("TRAIL", tt, rawt, fitt, C_TRAIL_nM)]:
k_last = len(cs) - 1
m = ~np.isnan(fit[:, k_last])
tinj = t[m][np.argmax(fit[m, k_last])]
print(f" {lab}: time {t.min():.0f} to {t.max():.0f} s; fitted curves peak at t = {tinj:.0f} s "
f"(end of injection); dissociation window {t.max() - tinj:.0f} s")
for k, c in enumerate(cs):
mk = ~np.isnan(fit[:, k])
f_t, f_y = t[mk], fit[mk, k]
y0 = f_y[np.argmin(np.abs(f_t - (tinj + 1)))]
print(f" {c:>8g} nM: peak of fitted curve {f_y.max():8.2f} RU; fitted response at end of "
f"dissociation / at start = {f_y[-1] / y0:.5f}")
res[lab] = (t, fit, cs, round(tinj))
rF = np.nanmax(fitf[:, 3])
rT = np.nanmax(fitt[:, 4])
print(f" peak fitted response: FasL 1000 nM {rF:.2f} RU; TRAIL 250 nM {rT:.2f} RU")
for c in (125, 250):
a = np.nanmax(fitt[:, C_TRAIL_nM.index(c)]) / np.nanmax(fitf[:, C_FASL_nM.index(c)])
print(f" TRAIL/FasL peak fitted response at equal concentration, {c} nM: {a:.0f}")
def model(lka, lkd, rmax, ri, t, cs, tinj):
ka, kd = 10 ** lka, 10 ** lkd
out = []
for k, c in enumerate(cs):
C = c * 1e-9
kobs = ka * C + kd
req = rmax * ka * C / kobs
R = np.where(t < 0, 0, req * (1 - np.exp(-kobs * np.clip(t, 0, tinj))))
R0 = req * (1 - np.exp(-kobs * tinj))
with np.errstate(over="ignore", invalid="ignore"):
R = np.where(t > tinj, R0 * np.exp(-kd * (t - tinj)), R)
R = R + np.where((t >= 0) & (t <= tinj), ri[k], 0) # bulk refractive-index step
out.append(R)
return np.array(out).T
print(" Global 1:1 Langmuir fit (with a bulk step per curve) to the software-fitted curves:")
fits = {}
for lab, (t, fit, cs, tinj) in res.items():
m = (np.abs(t - tinj) > 0.3) & (np.abs(t) > 0.3)
T, Y = t[m], fit[m]
sel = ~np.isnan(Y)
n = len(cs)
best = None
for lka in [3, 4, 5, 6]:
for lkd in [-12, -9, -6, -4]:
r = least_squares(lambda p: (model(p[0], p[1], p[2], p[3:], T, cs, tinj) - Y)[sel],
[lka, lkd, np.nanmax(Y) * 2] + [0] * n,
bounds=([0, -20, 0] + [-1e4] * n, [10, 1, 1e7] + [1e4] * n))
if best is None or r.cost < best.cost:
best = r
ka = 10 ** best.x[0]
rms = np.sqrt(2 * best.cost / sel.sum())
fits[lab] = ka
print(f" {lab}: ka = {ka:.2g} /M/s, Rmax = {best.x[2]:.3g} RU, rms residual = {rms:.2g} RU "
f"(max response {np.nanmax(Y):.0f} RU)")
print(f" ka(TRAIL) / ka(FasL) = {fits['TRAIL'] / fits['FasL']:.1f}")
t, fit, cs, tinj = res["FasL"]
m = (np.abs(t - tinj) > 0.3) & (np.abs(t) > 0.3)
T, Y = t[m], fit[m]
print(" FasL profile: kd fixed, other parameters refitted")
for lkd in [-12, -10, -8, -6]:
r = least_squares(lambda p: (model(p[0], lkd, p[1], p[2:], T, cs, tinj) - Y).ravel(),
[3.9, 7] + [0] * 4, bounds=([0, 0] + [-1e4] * 4, [10, 1e7] + [1e4] * 4))
print(f" kd = 1e{lkd} /s: KD = {10 ** lkd / 10 ** r.x[0]:.2g} M, rms = {np.sqrt(2 * r.cost / Y.size):.5f} RU")
kd_impl = KD_FASL * fits["FasL"]
print(f" kd implied by the printed FasL KD and the fitted ka: {kd_impl:.2g} /s; "
f"fraction dissociated in {t.max() - tinj:.0f} s = {1 - math.exp(-kd_impl * (t.max() - tinj)):.1g}")
if "--no-source-data" not in sys.argv:
source_data()
# ------------------------------------------------------------------ figures
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE, GRID, GREY = "#eb6834", "#2a78d6", "#e4e3df", "#b9b8b3"
plt.rcParams.update({
"font.family": "DejaVu Sans", "font.size": 10.5, "text.color": INK,
"axes.edgecolor": INK2, "axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK2,
"figure.facecolor": BG, "axes.facecolor": BG, "savefig.facecolor": BG, "svg.fonttype": "none",
})
def save(fig, stem):
fig.savefig(os.path.join(HERE, stem + ".svg"), bbox_inches="tight", pad_inches=0.12)
fig.savefig(os.path.join(HERE, stem + ".png"), dpi=200, bbox_inches="tight", pad_inches=0.12)
plt.close(fig)
# ---- Figure 1: abstract vs Results, plus the fold comparison on a log axis
fig = plt.figure(figsize=(7.2, 4.4))
axT = fig.add_axes([0.01, 0.36, 0.98, 0.64])
axT.set_xlim(0, 1)
axT.set_ylim(0, 1)
axT.axis("off")
def panel(x, w, title, lines, colors):
axT.add_patch(FancyBboxPatch((x, 0.30), w, 0.62, boxstyle="round,pad=0.012,rounding_size=0.02",
fc="white", ec=GREY, lw=1.2))
axT.text(x + 0.02, 0.86, title, fontsize=10.5, fontweight="bold", va="top", color=INK)
y = 0.72
for s, c in zip(lines, colors):
axT.text(x + 0.02, y, s, fontsize=10, va="top", color=c,
fontweight="bold" if c != INK else "normal")
y -= 0.095
panel(0.01, 0.455, "Abstract",
["“...DR5 interacted specifically with", "recombinant FasL–Fc protein, although", "the strength of this interaction was",
"approximately 60-fold lower than the", "affinity between TRAIL and DR5.”"],
[INK, INK, INK, ORANGE, INK])
panel(0.53, 0.45, "Results (SPR, Fig. 1—fig. suppl. 4F,G)",
["“KD: 1.23 × 10⁻¹² M for DR5–FasL", "versus 6.01 × 10⁻¹³ M for DR5–TRAIL”", "",
f"1.23 × 10⁻¹² / 6.01 × 10⁻¹³ = {ratio:.2f}", "(computed from the two printed KDs)"],
[ORANGE, ORANGE, INK, BLUE, INK2])
axT.add_patch(FancyArrowPatch((0.468, 0.60), (0.518, 0.60), arrowstyle="<|-|>", mutation_scale=12,
lw=2, color=INK2))
axT.text(0.5, 0.15, f"Same comparison, two magnitudes: ~{FOLD_ABSTRACT}-fold stated vs {ratio:.2f}-fold implied "
f"(a factor of {FOLD_ABSTRACT / ratio:.0f} apart)", ha="center", va="center", fontsize=10.5, color=INK)
ax = fig.add_axes([0.06, 0.10, 0.90, 0.22])
ax.set_xscale("log")
ax.set_xlim(0.7, 150)
ax.set_ylim(-0.6, 0.6)
ax.set_yticks([])
for s in ("left", "right", "top"):
ax.spines[s].set_visible(False)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.axvline(1, color=INK2, lw=0.8)
ax.text(1.03, 0.45, "1 = equal affinity", fontsize=10, color=INK2, va="center")
ax.plot([ratio], [0], "o", ms=10, color=BLUE, zorder=3)
ax.annotate(f"{ratio:.2f}-fold: ratio of the printed KDs", (ratio, 0), (ratio, -0.42), fontsize=10,
color=BLUE, va="center", ha="left")
ax.plot([FOLD_ABSTRACT], [0], "D", ms=9, color=ORANGE, zorder=3)
ax.annotate("~60-fold: abstract", (FOLD_ABSTRACT, 0), (FOLD_ABSTRACT, 0.42), fontsize=10,
color=ORANGE, va="center", ha="center")
ax.plot([ratio, FOLD_ABSTRACT], [0, 0], color=GREY, lw=2, zorder=1)
ax.set_xticks([1, 2, 5, 10, 20, 50, 100])
ax.set_xticklabels(["1", "2", "5", "10", "20", "50", "100"])
ax.set_xlabel("How many fold lower the DR5–FasL affinity is than DR5–TRAIL (KD ratio, log scale)")
save(fig, "fig1")
# ---- Figure 2: printed KDs, the KDs a 60-fold difference would need, and the injected concentrations
fig, ax = plt.subplots(figsize=(7.2, 3.3))
ax.set_xscale("log")
ax.set_xlim(5e-15, 5e-6)
ax.set_ylim(-0.7, 1.75)
ax.grid(axis="x", which="major", color=GRID, lw=0.8)
ax.set_axisbelow(True)
for s in ("left", "right", "top"):
ax.spines[s].set_visible(False)
ax.set_yticks([1, 0])
ax.set_yticklabels(["DR5–FasL", "DR5–TRAIL"], fontsize=10.5, color=INK)
ax.tick_params(axis="y", length=0)
for y, cs in [(1, C_FASL_nM), (0, C_TRAIL_nM)]:
c = np.array(cs) * 1e-9
ax.plot([c.min(), c.max()], [y, y], color=GREY, lw=8, solid_capstyle="butt", zorder=1)
ax.plot(c, [y] * len(c), "|", ms=12, mew=2, color=INK2, zorder=2)
ax.text(np.sqrt(c.min() * c.max()), y + 0.22, f"injected {cs[0]:g}–{cs[-1]:g} nM", ha="center",
fontsize=10, color=INK2)
ax.plot([KD_FASL], [1], "D", ms=9, color=ORANGE, zorder=3)
ax.plot([KD_TRAIL], [0], "D", ms=9, color=ORANGE, zorder=3)
ax.text(KD_FASL, 1.22, "printed KD\n1.23 × 10⁻¹² M", ha="center", fontsize=10, color=ORANGE)
ax.text(KD_TRAIL * 1.6, 0.22, "printed KD\n6.01 × 10⁻¹³ M", ha="left", fontsize=10, color=ORANGE)
ax.plot([kd_fasl_req], [1], "o", ms=9, mfc="white", mew=2, color=BLUE, zorder=3)
ax.text(kd_fasl_req * 1.5, 0.98, "needed for 60-fold\nif TRAIL KD is right\n3.61 × 10⁻¹¹ M", ha="left",
va="top", fontsize=10, color=BLUE)
ax.plot([kd_trail_req], [0], "o", ms=9, mfc="white", mew=2, color=BLUE, zorder=3)
ax.text(kd_trail_req, -0.25, "needed for 60-fold\nif FasL KD is right\n2.05 × 10⁻¹⁴ M", ha="left",
va="top", fontsize=10, color=BLUE)
ax.set_xlabel("Concentration (M, log scale)")
ax.set_xticks([1e-14, 1e-13, 1e-12, 1e-11, 1e-10, 1e-9, 1e-8, 1e-7, 1e-6])
ax.set_xticklabels(["10 fM", "", "1 pM", "", "", "1 nM", "", "", "1 µM"])
save(fig, "fig2")
print("\nWrote fig1.svg/png and fig2.svg/png in", HERE)
Main IGF-1 hospitalization odds ratio 0.62 is printed with a 95% CI that excludes it
eLife (2022) · doi:10.7554/eLife.79720 · PMC9576268
Summary
The paper's primary (random-effects IVW) estimate for genetically predicted IGF-1 and COVID-19 hospitalization is given as OR 0.62 (95% CI 0.25–0.51, p = 0.018) in the Abstract, the Results and Table 2. That interval cannot belong to that estimate, because its upper limit (0.51) is below the point estimate (0.62). On the log scale the printed interval is centred on 0.36, and it would imply p ≈ 1.5 × 10-8, not 0.018. The paper's own OR and p-value imply a 95% CI of about 0.42–0.92. The hospitalization forest plot in the paper's Figure 2 (the panel titled "A. MR analysis for IGF-1 and COVID-19 hospitalization"; "the forest plot" below) draws the IVW whisker at that interval, and the interval is close to the paper's MR-PRESSO (0.43–0.88) and BMI-adjusted IVW (0.41–0.90) intervals. The OR, the p-value and the stated conclusions are unaffected, but the precision printed for a headline result in the Abstract is wrong and overstates how far the data exclude small effects.
What the paper reports
Abstract, Methods part:
Random-effects inverse variance weighted (IVW) MR approach was used as the primary MR method and the weighted median, MR-Egger, and MR Pleiotropy RESidual Sum and Outlier (MR-PRESSO) test were conducted as sensitivity analyses.
Abstract, Results part, and repeated word for word in the first paragraph of Results:
For one standard deviation increase in genetically predicted IGF-1 levels, the odds ratio was 0.77 (95% confidence interval [CI], 0.61–0.97, p=0.027) for COVID-19 susceptibility, 0.62 (95% CI: 0.25–0.51, p=0.018) for COVID-19 hospitalization, and 0.85 (95% CI: 0.52–1.38, p=0.513) for COVID-19 severity.
Table 2 ("Sex hormones, SHBG, IGF-1, and COVID-19 outcomes in Mendelian randomization (MR) analyses"), IGF-1 IVW row. Columns, in each outcome block: SNPs | OR (95% CI) | p Effect | p Heterogeneity | p Intercept, for susceptibility, hospitalization and severity:
IGF-1 | IVW | 16 | 0.77 (0.61, 0.97) | 0.027 | 0.175 | – | 16 | 0.62 (0.25, 0.51) | 0.018 | 0.715 | – | 18 | 0.85 (0.52, 1.38) | 0.513 | 0.601 | -
Table 2, IGF-1 MR-PRESSO row (in all three MR-PRESSO cells the upper limit is printed first):
IGF-1 | MR-PRESSO | 16 | 0.77 (0.98, 0.61) | – | – | – | 16 | 0.62 (0.88, 0.43) | – | – | – | 18 | 0.85 (1.34, 0.54) | - | - | -
Results, first paragraph:
Associations of IGF-1 levels with COVID-19 susceptibility and hospitalization were not statistically significant after Bonferroni correction, albeit showing a nominal significance at p<0.05.
Results, second paragraph (Table 6, BMI-associated SNPs removed):
As shown in Table 6, after removing SNPs associated with BMI, we found similar associations of genetically predicted IGF-1 levels with the risk of COVID-19 susceptibility (OR = 0.76; 95% CI: 0.60–0.96, p=0.021), hospitalization (OR = 0.61; 95% CI: 0.41–0.90, p=0.014) …
Discussion, last paragraph (the Abstract's Conclusions open with the same sentence, without "In conclusion,"):
In conclusion, our study indicated that genetically predicted high IGF-1 levels were associated with decrease the risk of COVID-19 susceptibility and hospitalization, but these associations did not survive the Bonferroni correction of multiple testing.
What the paper's own numbers show
Method. IVW Mendelian-randomization estimates are log odds ratios with a normal (Wald) standard error, so the reported OR is exp(β), the 95% CI is exp(β ± 1.96·SE) and p = 2Φ(−|β|/SE). Two checks follow. (1) The geometric centre √(lower × upper) of a printed CI must equal the OR, apart from rounding, and the CI's width fixes SE and hence p. (2) The OR and p fix SE = |ln OR| / Φ-1(1 − p/2), and hence the CI. I applied both checks to the hospitalization cell, to the other five IGF-1 IVW cells of Table 2 (main analysis) and Table 6 (BMI-associated SNPs removed), and to the neighbouring IGF-1 estimates. To see whether rounding could matter, I let OR range over [0.615, 0.625) and p over [0.0175, 0.0185). Finally, I read the vector coordinates of the forest plot (the panel "A. MR analysis for IGF-1 and COVID-19 hospitalization" in the paper's Figure 2, page 8 of the publisher's v2 PDF) and converted the drawn whiskers to ORs using the plot's own linear axis ticks (0, 1, 2, 3). Every number below is printed by recompute.py.txt.
| Quantity | Location | Reported | Recomputed from the paper's own numbers |
|---|---|---|---|
| IGF-1 → hospitalization, IVW | Abstract; Results para. 1; Table 2 | OR 0.62 (95% CI 0.25–0.51), p = 0.018 | 0.62 lies outside 0.25–0.51 |
| Centre of the printed interval | same | expected to equal 0.62 | √(0.25 × 0.51) = 0.357 |
| p implied by the printed interval | same | 0.018 | SE(log OR) 0.182; p = 1.5 × 10-8 |
| 95% CI implied by OR 0.62 and p = 0.018 | same | 0.25–0.51 | z = 2.366, SE(log OR) = 0.202, CI 0.417–0.921 |
| Same, allowing for rounding of OR and p | same | — | lower limit 0.410–0.424, upper limit 0.918–0.924; no rounding gives an upper limit below 0.515 |
| IVW whisker drawn in the forest plot | Forest plot (label "0.62(0.25,0.51)") | — | drawn from 0.420 to 0.920, box at 0.620 |
| Calibration of that reading: MR-Egger bar | Forest plot (label "0.72(0.37,1.38)") | 0.72 (0.37, 1.38) | drawn 0.370–1.380, box at 0.720 |
| Weighted-median bar | Forest plot (label "0.75(0.44,1.28)") | 0.75 (0.44, 1.28) | drawn 0.450–1.240, box at 0.750: matches Table 6's BMI-adjusted weighted median, 0.75 (0.45, 1.24), not the label or Table 2 |
| IGF-1 → hospitalization, MR-PRESSO | Table 2 | 0.62 (0.88, 0.43) | contains 0.62; centre 0.615 |
| IGF-1 → hospitalization, IVW without BMI SNPs | Table 6; Results para. 2 | 0.61 (0.41–0.90), p = 0.014 | CI from OR and p: 0.41–0.90; p from CI: 0.013 |
| IGF-1 → susceptibility, IVW | Abstract; Table 2 | 0.77 (0.61–0.97), p = 0.027 | CI from OR and p: 0.61–0.97; p from CI: 0.027 |
| IGF-1 → severity, IVW | Abstract; Table 2 | 0.85 (0.52–1.38), p = 0.513 | CI from OR and p: 0.52–1.38; p from CI: 0.505 |
The other five IGF-1 IVW cells of Tables 2 and 6 pass both checks to rounding precision: in each, the printed p lies inside the range of p-values implied by every rounding of the printed limits. So for these tables the Wald method fits how the paper's intervals were produced. Only the main-analysis hospitalization IVW cell fails, and it fails both checks by a wide margin. Its OR and p agree with each other and point to a CI of about 0.42–0.92. Three other items in the paper are consistent with that interval: the IVW whisker drawn in the forest plot (0.42–0.92, with the same axis reading reproducing the MR-Egger bar exactly), the MR-PRESSO interval for the same 16 SNPs (0.43–0.88), and the IVW interval with one BMI-associated SNP removed (0.41–0.90). The last two come from largely the same data as the IVW estimate, so they are not independent confirmations. No other cell in the paper prints the pair (0.25, 0.51).
Figure
Figure 1. IGF-1 and COVID-19 hospitalization: odds ratio per 1-SD higher genetically predicted IGF-1, with 95% CI, on a log scale. The vertical line marks OR = 1. Orange: values printed in the paper. The top row is the primary IVW estimate as printed in the Abstract, the Results and Table 2, 0.62 (0.25–0.51); the short tick marks the interval's geometric centre, 0.36, which an interval for 0.62 would have at 0.62. The bottom two orange rows are the paper's MR-PRESSO interval for the same analysis (Table 2) and its IVW interval after removing BMI-associated SNPs (Table 6). Blue: the 95% CI recomputed from the printed OR 0.62 and p = 0.018, 0.42–0.92. Grey open square: the IVW whisker as actually drawn in the paper's hospitalization forest plot (Figure 2, panel "A. MR analysis for IGF-1 and COVID-19 hospitalization"), measured from the PDF's vector coordinates, 0.42–0.92. All numbers are from the paper or from recompute.py.txt.
Figure 2. For each IGF-1 IVW estimate in Table 2 (main analysis) and Table 6 (BMI-associated SNPs removed), the printed two-sided p-value (orange dots) and the p-value implied by the printed 95% CI (blue open diamonds), on a log scale. The vertical line marks p = 0.05. In five cells the two agree to rounding (for example 0.027 printed and 0.027 implied for susceptibility). For main-analysis hospitalization the printed interval implies p = 1.5 × 10-8, against a printed p of 0.018. Among the IGF-1 IVW cells of Tables 2 and 6, the error is confined to that one interval.
Why it matters
- It is the primary estimate for one of the two outcomes named in the title, and it is in the Abstract. The IVW method is declared the primary analysis, and the hospitalization association is one of the two the title calls protective. Readers who see only the Abstract, and meta-analysts who extract the interval from it or from Table 2, get an impossible interval.
- Taken literally, the printed interval overstates the precision and the strength of the effect. The printed limits would say the data exclude any odds reduction smaller than 49% (upper limit 0.51) and are compatible with a reduction of up to 75%. The interval implied by the paper's own OR and p (0.42–0.92) is wider and reaches much closer to the null: it is compatible with a reduction as small as 8%. Anyone who back-calculates a standard error from the printed limits for pooling or power calculations gets SE(log OR) 0.18 around a centre of 0.36, not 0.20 around 0.62.
- The paper's conclusions do not change. The OR (0.62), the direction, the nominal significance (p = 0.018) and the statement that the association does not survive Bonferroni correction all rest on the OR and p, which agree with each other and with the corrected interval. The sensitivity analyses the paper cites in support are unaffected.
Explanations considered
- Upper and lower limits printed in reverse order. Does not explain it. The paper does print the upper limit first in its MR-PRESSO cells, for example "0.62 (0.88, 0.43)", but those intervals still contain their estimates. Here 0.25 < 0.51 is in normal order, and in either order neither limit reaches 0.62.
- A different confidence level, a one-sided interval, or a different scale. Ruled out. Any interval for an estimate, at any level and whether one- or two-sided, contains that estimate. The limits are not on the log-OR scale (ln 0.62 = −0.478). And a symmetric log-scale interval of the printed width around 0.62 would still have to straddle 0.62.
- The interval belongs to another model, outcome or row. Not supported. The OR, the interval, the p-value and the 16 SNPs sit in the same cell of Table 2, and the Abstract and the Results repeat the same triple. No other cell of Table 2 or Table 6 prints (0.25, 0.51). The paper's forest plot draws the IVW whisker at 0.42–0.92 while labelling it "0.62(0.25,0.51)", so the plotted whisker and the printed label disagree.
- Rounding. Ruled out. Over every OR and p that round to 0.62 and 0.018, the implied upper limit stays between 0.918 and 0.924. An upper limit that rounds to 0.51 is below 0.515, which is below any OR that rounds to 0.62.
- A conversion or typesetting artifact in the PMC copy. Ruled out. The same "0.25–0.51" appears in the publisher's typeset PDF (version 2: page 1, Abstract; page 5, Results; page 7, Table 2; and the forest-plot label on page 8), on the eLife article page, and in the PMC and Europe PMC full text.
- A correction already issued. None found (see Status).
- Side observation on the forest plot, not part of the main finding. In the forest plot the MR-PRESSO row is labelled "0.62(0.60,0.64)" with p 0.016 and drawn from 0.60 to 0.64, while Table 2 gives the MR-PRESSO hospitalization interval as 0.62 (0.88, 0.43). The weighted-median bar is drawn at 0.45–1.24 against a label of 0.44–1.28; the drawn bar equals Table 6's BMI-adjusted weighted-median hospitalization interval, 0.75 (0.45, 1.24), not Table 2's 0.75 (0.44, 1.28). So the plot is not a clean copy of a single analysis. Its IVW whisker (0.42–0.92) matches neither Table 6's IVW interval (0.41–0.90) nor the printed label, and it does match the back-calculation from the OR and p, so it is consistent with the 0.42–0.92 interval but does not decide it. None of this affects the main finding, but the authors may wish to reconcile the figure with Tables 2 and 6.
- Side observation on Table 3, not part of the main finding. In Table 3 (10 SNPs in the IGF-1 gene region; the same values are quoted in Results para. 1), two of the three IVW p-values do not match their intervals. Hospitalization 0.90 (0.74, 1.10) is printed with p = 0.645, but the interval implies p = 0.309 (0.286–0.333 over every rounding of the limits). Severity 1.01 (0.82, 1.24) is printed with p = 0.415, but the interval implies p = 0.937 (0.899–0.976). Susceptibility 0.99 (0.91, 1.07), p = 0.777, passes (implied 0.747, range 0.657–0.841). This sensitivity analysis uses correlated SNPs with a linkage-disequilibrium matrix, but it is still reported as an OR with a Wald-type 95% CI and p, so the check applies. These rows are null estimates unrelated to the main finding and do not affect it; the authors may wish to check them as well.
Status
- Version of record. eLife has published two versions (v1, 2022-10-17; v2, 2023-03-29). Both versions in the eLife API and the eLife article page carry the same "0.25–0.51" in the Abstract, the Results and Table 2, and a v1-to-v2 comparison shows no change to any result. I checked the v2 typeset PDF (
elife-79720-v2.pdf, downloaded from the eLife CDN) directly: the text on pages 1, 5 and 7 and the forest-plot label on page 8 read as quoted above, and the forest-plot coordinates used here come from page 8 of that file. The PMC and Europe PMC full texts agree. Every quotation above was checked verbatim against the PMC-archived text withsrc/show.py. - Supplements. Supplementary file 1 (
elife-79720-supp1-v2.docx) lists the genetic instruments: per-SNP associations with the exposures, including IGF-1. It contains no SNP–outcome associations and no MR result tables, and the article lists no source-data files, so the supplement cannot settle the interval. Recomputing the IVW estimate from scratch would need the COVID-19 Host Genetics Initiative round-5 summary statistics, which were not used here. The forest plot is the only other rendering of the IVW result in the paper. Its IVW whisker is consistent with 0.42–0.92, but because its weighted-median bar matches Table 6 rather than its own label, it cannot be treated as decisive source output. - Corrections and comments. As of 2026-10-24 I found no correction, erratum, expression of concern or retraction.
src/known.pyreturns no Crossref update relations, no PMC notices and no Europe PMC notices. The PubPeer API returned no record (HTTP 404) for the DOI, and a web search for the title together with "PubPeer" found no thread. - Sources. PMC article PMC9576268; eLife article page https://elifesciences.org/articles/79720 and its v1 and v2 API records; publisher PDF
elife-79720-v2.pdf; Supplementary file 1; Europe PMC full text; Crossref; PubPeer; https://doi.org/10.7554/eLife.79720.
Suggested correction
In the Abstract, the Results (first paragraph) and Table 2 (IGF-1, IVW, hospitalization), the interval "0.25–0.51" / "(0.25, 0.51)" should be replaced by the interval from the original IVW output. From the printed OR and p-value that interval is about 0.42–0.92, which is also where the paper's forest plot draws the IVW whisker. The exact limits should come from the authors' analysis output, not from this back-calculation. The label "0.62(0.25,0.51)" in the forest plot should be corrected to match. The authors may also wish to check the forest plot's MR-PRESSO label "0.62(0.60,0.64)" against Table 2's "0.62 (0.88, 0.43)", its weighted-median bar (drawn at Table 6's 0.45–1.24) against its label, and the Table 3 IVW p-values for hospitalization and severity against their intervals. They could also print every interval in the paper lower limit first.
Recomputation script (python recompute.py.txt)
# recompute.py.txt -- PMC9576268 (eLife 2022;11:e79720, doi:10.7554/eLife.79720)
# Checks the IVW odds ratio for IGF-1 -> COVID-19 hospitalization against its
# printed 95% CI, using only numbers printed in the paper.
# Run: python recompute.py.txt (needs numpy, scipy, matplotlib)
# Writes fig1 and fig2 (.svg, .png) next to this file.
import os
import numpy as np
from scipy.stats import norm
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
HERE = os.path.dirname(os.path.abspath(__file__))
Z95 = norm.isf(0.025) # 1.959964
def ci_from_or_p(orr, p):
"""Wald 95% CI on the log scale implied by a point OR and a two-sided p."""
z = norm.isf(p / 2)
se = abs(np.log(orr)) / z
return z, se, orr * np.exp(-Z95 * se), orr * np.exp(Z95 * se)
def from_ci(lo, hi):
"""Geometric centre, SE(log OR) and two-sided p implied by a 95% CI."""
lo, hi = min(lo, hi), max(lo, hi)
centre = np.sqrt(lo * hi)
se = (np.log(hi) - np.log(lo)) / (2 * Z95)
p = 2 * norm.sf(abs(np.log(centre)) / se)
return centre, se, p
print("== 1. Printed IVW hospitalization result (Abstract, Results p0, Table 2) ==")
OR, LO, HI, P = 0.62, 0.25, 0.51, 0.018
print(f"OR {OR}, printed 95% CI {LO}-{HI}, p {P}")
print(f"Point estimate inside printed CI: {LO <= OR <= HI}")
c, se, pc = from_ci(LO, HI)
print(f"Printed CI: geometric centre {c:.3f}, SE(logOR) {se:.4f}, implied p {pc:.2e}")
print(f"log OR = {np.log(OR):.4f}; log limits = {np.log(LO):.4f}, {np.log(HI):.4f}")
z, se_p, lo_r, hi_r = ci_from_or_p(OR, P)
print(f"From OR {OR} and p {P}: z {z:.3f}, SE(logOR) {se_p:.4f}, 95% CI {lo_r:.3f}-{hi_r:.3f}")
print("\n== 2. Rounding envelope (OR in [0.615, 0.625), p in [0.0175, 0.0185)) ==")
los, his = [], []
for o in np.linspace(0.615, 0.62499, 41):
for pp in np.linspace(0.0175, 0.01849, 41):
_, _, a, b = ci_from_or_p(o, pp)
los.append(a); his.append(b)
print(f"Lower limit range {min(los):.3f}-{max(los):.3f}; upper limit range {min(his):.3f}-{max(his):.3f}")
print(f"Largest possible upper limit < 0.515 ? {max(his) < 0.515}; smallest upper limit {min(his):.3f}")
print("\n== 3. Neighbouring estimates in the paper (internal consistency) ==")
rows = [
("IGF-1 susceptibility IVW (Table 2)", 0.77, 0.61, 0.97, 0.027),
("IGF-1 severity IVW (Table 2)", 0.85, 0.52, 1.38, 0.513),
("IGF-1 hospitalization MR-PRESSO (Table 2, printed 0.88, 0.43)", 0.62, 0.43, 0.88, None),
("IGF-1 hospitalization IVW, BMI SNPs removed (Table 6)", 0.61, 0.41, 0.90, 0.014),
("IGF-1 hospitalization MR-PRESSO, BMI SNPs removed (Table 6)", 0.61, 0.43, 0.86, None),
("IGF-1 hospitalization MR-Egger (Table 2)", 0.72, 0.37, 1.38, 0.336),
("IGF-1 hospitalization weighted median (Table 2)", 0.75, 0.44, 1.28, 0.294),
]
for lbl, o, lo, hi, pp in rows:
c, se, pc = from_ci(lo, hi)
s = f"{lbl}: OR {o} CI {lo}-{hi}; contains OR {lo <= o <= hi}; centre {c:.3f}; p from CI {pc:.4f}"
if pp is not None:
_, _, a, b = ci_from_or_p(o, pp)
s += f"; printed p {pp}; CI from OR & p {a:.2f}-{b:.2f}"
print(s)
print("\n== 4. Interval drawn in the paper's hospitalization forest plot (Figure 2, panel \"A. MR analysis for IGF-1 and COVID-19 hospitalization\") ==")
# Vector coordinates (PDF points) read with PyMuPDF from page 8 of the publisher PDF
# elife-79720-v2.pdf. The OR axis is linear: tick 0 at x=311.09, tick 1 at x=320.37,
# tick 2 at x=329.66, tick 3 at x=338.94.
x0, x1 = 311.09, 320.37
unit = x1 - x0
def to_or(x):
return (x - x0) / unit
bars = { # label printed in the figure : (whisker left, whisker right, box left, box right)
"MR Egger, label 0.72(0.37,1.38)": (314.52, 323.90, 316.42, 319.13),
"Weighted median, label 0.75(0.44,1.28)": (315.27, 322.60, 316.70, 319.40),
"IVW, label 0.62(0.25,0.51)": (314.99, 319.63, 315.49, 318.20),
"MR-PRESSO, label 0.62(0.60,0.64)": (316.66, 317.03, 315.49, 318.20),
}
drawn = {}
for k, (wl, wr, bl, br) in bars.items():
drawn[k] = (to_or(wl), to_or(wr), to_or((bl + br) / 2))
print(f"{k}: drawn whisker {to_or(wl):.3f}-{to_or(wr):.3f}, box centre {to_or((bl+br)/2):.3f}")
iv_lo, iv_hi, iv_c = drawn["IVW, label 0.62(0.25,0.51)"]
# ---------------------------------------------------------------- figure 1
BG, INK, INK2 = "#fcfcfb", "#0b0b0b", "#52514e"
ORANGE, BLUE = "#eb6834", "#2a78d6"
plt.rcParams.update({"font.size": 10.5, "font.family": "DejaVu Sans",
"svg.fonttype": "none", "text.color": INK,
"axes.labelcolor": INK, "xtick.color": INK2, "ytick.color": INK})
items = [ # (label, OR, lo, hi, colour, kind)
("IVW, as printed\n(Abstract, Results, Table 2)", OR, LO, HI, ORANGE, "printed"),
("IVW, recomputed from\nOR 0.62 and p = 0.018", OR, lo_r, hi_r, BLUE, "recomputed"),
("IVW, whisker as drawn in\nthe paper's forest plot", iv_c, iv_lo, iv_hi, INK2, "drawn"),
("MR-PRESSO, as printed\n(Table 2)", 0.62, 0.43, 0.88, ORANGE, "printed"),
("IVW without BMI SNPs,\nas printed (Table 6)", 0.61, 0.41, 0.90, ORANGE, "printed"),
]
fig, ax = plt.subplots(figsize=(7.2, 4.9), facecolor=BG)
ax.set_facecolor(BG)
n = len(items)
for i, (lbl, o, lo, hi, col, kind) in enumerate(items):
y = n - 1 - i
ax.plot([lo, hi], [y, y], color=col, lw=2, solid_capstyle="butt", zorder=3)
for xb in (lo, hi):
ax.plot([xb, xb], [y - 0.12, y + 0.12], color=col, lw=2, zorder=3)
mk = "o" if kind != "drawn" else "s"
face = col if kind != "drawn" else BG
ax.plot([o], [y], marker=mk, ms=9, color=col, mfc=face, mew=2, zorder=4, ls="none")
ax.text(1.29, y, f"{o:.2f} ({lo:.2f}–{hi:.2f})", va="center", ha="left",
fontsize=10.5, color=INK, transform=ax.transData)
# centre of the printed interval
c_bad = np.sqrt(LO * HI)
ax.plot([c_bad], [n - 1], marker="|", ms=14, mew=2, color=ORANGE, zorder=4)
ax.annotate(f"centre of printed\ninterval {c_bad:.2f}", xy=(c_bad, n - 1), xytext=(c_bad, n - 1 + 0.55),
ha="center", va="bottom", fontsize=10, color=INK2,
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.annotate("point estimate 0.62 lies\noutside its own interval", xy=(OR, n - 1), xytext=(0.53, n - 1 + 0.55),
ha="left", va="bottom", fontsize=10, color=INK2,
arrowprops=dict(arrowstyle="-", color=INK2, lw=1))
ax.axvline(1.0, color=INK2, lw=0.8, zorder=1)
ax.set_xscale("log")
ax.set_xlim(0.2, 1.25)
ticks = [0.2, 0.3, 0.4, 0.5, 0.62, 0.8, 1.0]
ax.set_xticks(ticks)
ax.set_xticklabels([f"{t:g}" for t in ticks])
ax.minorticks_off()
ax.set_ylim(-0.6, n - 1 + 1.35)
ax.set_yticks(range(n))
ax.set_yticklabels([it[0] for it in items][::-1], fontsize=10)
ax.set_xlabel("Odds ratio for COVID-19 hospitalization per 1-SD higher\ngenetically predicted IGF-1 (log scale; vertical line: OR = 1)", fontsize=10)
ax.grid(axis="x", color="#e4e3df", lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#b9b8b3")
ax.tick_params(axis="y", length=0)
ax.text(1.29, n - 1 + 0.75, "OR (95% CI)", ha="left", va="center", fontsize=10.5, color=INK, fontweight="bold")
from matplotlib.lines import Line2D
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, lw=2, label="reported in the paper"),
Line2D([], [], color=BLUE, marker="o", ms=8, lw=2, label="recomputed from the paper's OR and p"),
Line2D([], [], color=INK2, marker="s", mfc=BG, ms=8, lw=2, mew=2, label="measured from the paper's forest plot")]
fig.legend(handles=handles, loc="lower center", ncol=2, frameon=False, fontsize=9.5,
bbox_to_anchor=(0.5, 0.0), handlelength=2.2, columnspacing=1.2)
fig.subplots_adjust(left=0.30, right=0.79, top=0.97, bottom=0.27)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig1.{ext}"), facecolor=BG, **kw)
print("\nWrote fig1.svg and fig1.png")
# ---------------------------------------------------------------- figure 2
print("\n== 5. Printed p vs p implied by the printed 95% CI, all IGF-1 IVW cells ==")
cells = [ # label, OR, lo, hi, printed p (Table 2 and Table 6, IVW rows)
("Susceptibility, main (Table 2)", 0.77, 0.61, 0.97, 0.027),
("Hospitalization, main (Table 2)", 0.62, 0.25, 0.51, 0.018),
("Severity, main (Table 2)", 0.85, 0.52, 1.38, 0.513),
("Susceptibility, BMI SNPs removed (Table 6)", 0.76, 0.60, 0.96, 0.021),
("Hospitalization, BMI SNPs removed (Table 6)", 0.61, 0.41, 0.90, 0.014),
("Severity, BMI SNPs removed (Table 6)", 0.84, 0.52, 1.38, 0.497),
]
pimp = []
for lbl, o, lo, hi, pp in cells:
c, se, pc = from_ci(lo, hi)
pimp.append(pc)
ps = [from_ci(a, b)[2] for a in np.linspace(lo - 0.005, lo + 0.00499, 41)
for b in np.linspace(hi - 0.005, hi + 0.00499, 41)]
print(f"{lbl}: printed p {pp}; p implied by printed CI {pc:.3g} "
f"(range over rounding of the limits {min(ps):.3g}-{max(ps):.3g}; printed p inside {min(ps) <= pp <= max(ps)}); "
f"CI centre {c:.3f} vs OR {o}")
fig, ax = plt.subplots(figsize=(7.2, 3.9), facecolor=BG)
ax.set_facecolor(BG)
m = len(cells)
for i, ((lbl, o, lo, hi, pp), pc) in enumerate(zip(cells, pimp)):
y = m - 1 - i
if abs(np.log10(pc) - np.log10(pp)) > 0.3:
ax.plot([pc, pp], [y, y], color=INK2, lw=1.2, zorder=2)
ax.text(pc, y - 0.32, f"{pc:.1e}", ha="center", va="top", fontsize=10, color=INK)
ax.text(pp, y - 0.32, f"{pp}", ha="center", va="top", fontsize=10, color=INK)
ax.plot([pc], [y], marker="D", ms=11, mfc="none", mec=BLUE, mew=2, ls="none", zorder=3)
ax.plot([pp], [y], marker="o", ms=8, color=ORANGE, ls="none", zorder=4)
ax.axvline(0.05, color=INK2, lw=0.8, zorder=1)
ax.text(0.055, -0.55, "p = 0.05", ha="left", va="bottom", fontsize=10, color=INK2)
ax.set_xscale("log")
ax.set_xlim(3e-10, 1.5)
ax.set_xticks([1e-8, 1e-6, 1e-4, 1e-2, 1])
ax.set_xticklabels(["10$^{-8}$", "10$^{-6}$", "10$^{-4}$", "0.01", "1"])
ax.minorticks_off()
ax.set_ylim(-0.7, m - 0.4)
ax.set_yticks(range(m))
ax.set_yticklabels([c[0].replace("BMI SNPs removed", "BMI SNPs out") for c in cells][::-1], fontsize=10)
ax.set_xlabel("Two-sided p-value, IGF-1 IVW estimates (log scale)", fontsize=10)
ax.grid(axis="x", color="#e4e3df", lw=0.8, zorder=0)
for s in ("top", "right", "left"):
ax.spines[s].set_visible(False)
ax.spines["bottom"].set_color("#b9b8b3")
ax.tick_params(axis="y", length=0)
handles = [Line2D([], [], color=ORANGE, marker="o", ms=8, ls="none", label="p printed in the paper"),
Line2D([], [], color=BLUE, marker="D", mfc="none", mew=2, ms=10, ls="none",
label="p recomputed from the printed 95% CI")]
fig.legend(handles=handles, loc="upper center", ncol=2, frameon=False, fontsize=10, bbox_to_anchor=(0.5, 1.0))
fig.subplots_adjust(left=0.40, right=0.97, top=0.86, bottom=0.15)
for ext, kw in (("svg", {}), ("png", {"dpi": 200})):
fig.savefig(os.path.join(HERE, f"fig2.{ext}"), facecolor=BG, **kw)
print("\nWrote fig2.svg and fig2.png")
# ---------------------------------------------------------------- side check
print("\n== 6. Side check: Table 3 IVW rows (10 IGF-1 gene-region SNPs; also quoted in Results para. 1) ==")
t3 = [ # label, OR, lo, hi, printed p
("Susceptibility, gene region (Table 3)", 0.99, 0.91, 1.07, 0.777),
("Hospitalization, gene region (Table 3)", 0.90, 0.74, 1.10, 0.645),
("Severity, gene region (Table 3)", 1.01, 0.82, 1.24, 0.415),
]
for lbl, o, lo, hi, pp in t3:
c, se, pc = from_ci(lo, hi)
ps = [from_ci(a, b)[2] for a in np.linspace(lo - 0.005, lo + 0.00499, 41)
for b in np.linspace(hi - 0.005, hi + 0.00499, 41)]
print(f"{lbl}: printed p {pp}; p implied by printed CI {pc:.3f} "
f"(range over rounding of the limits {min(ps):.3f}-{max(ps):.3f}); "
f"printed p inside range {min(ps) <= pp <= max(ps)}; CI centre {c:.3f} vs OR {o}")
Registered primary outcome, door-to-balloon time, was replaced after completion; main text omits this
JAMA Netw Open (2024) · doi:10.1001/jamanetworkopen.2024.3729 · PMC10980970 · NCT05415085
Summary
This single-center randomized trial compared culprit lesion PCI before complete coronary angiography with complete angiography first in STEMI. The first ClinicalTrials.gov version of NCT05415085 (v0, 2022-06-08) registered one primary outcome, "Reduction in door-to-balloon time". The second and current version (v1, 2023-10-17) replaced it with "Needle-to-balloon time of 10 minutes or less". v1 was posted 474 days after the registered primary completion date (2022-06-30) and 164 days before publication. The paper reports needle-to-balloon time of 10 minutes or less as "the primary outcome of this study". Its main text does not say that the primary outcome was changed, and it describes the trial as "prospectively registered", although the record was first submitted about 13 months after enrollment began. The only disclosure is one undated sentence in the protocol supplement. The originally registered outcome appears in a single Results sentence as a non-significant difference (61.6 vs 73 minutes; P = .12), and the paper does not identify it as the registered primary.
What was registered
ClinicalTrials.gov NCT05415085 has two versions. The record history lists the modules each version changed, and its lastUpdateVersions field shows that the primary outcomes were last changed in v1 and the secondary outcomes in v0, so the secondary outcomes never changed.
v0, first version. First submitted 2022-04-27, QC 2022-06-08, first posted 2022-06-10. Status RECRUITING; study start 2021-05-01 (actual); primary completion 2022-06-30 (estimated); enrollment 250 (estimated). Primary outcome:
Reduction in door-to-balloon time | Change in door-to-balloon time in minutes | time frame: through study completion, an average of 1 year
The brief summary in v0 reads: "The aim of this study is to assess the impact of culprit-first versus culprit-last percutaneous coronary intervention on the door to balloon time and clinical outcomes in patients with ST-elevation myocardial infarction." The keywords include "door-to-balloon time".
v1, current version. Submitted and QC 2023-10-17, posted 2023-10-19. Changed modules: Study Status, Study Design, Outcome Measures, Contacts/Locations. Status COMPLETED; primary completion 2022-06-30 (actual); study completion 2023-04-13 (actual); enrollment 106 (actual). Primary outcome:
Needle-to-balloon time of 10 minutes or less | Change in needle-to-balloon time in minutes | time frame: through study completion, an average of 1 year
The brief summary and keywords were not changed: the current record still gives the aim as the impact "on the door to balloon time and clinical outcomes" and still lists "door-to-balloon time" as a keyword.
Secondary outcomes, identical in v0 and v1:
Need for hemodynamic (mechanical/medical) support during PCI | Number of patients that needed mechanical support during PCI
Need for respiratory support during PCI | Number of patients that needed respiratory support during PCI
Rate of failed PCI | As determined by the operator
All three have the time frame "through study completion, an average of 1 year".
What the paper reports
Abstract, "Main Outcomes and Measures":
A needle-to-balloon time of 10 minutes or less.
Methods, "Study Protocol and End Point":
The primary outcome of this study was a needle-to-balloon time of 10 minutes or less. Secondary outcomes included the need for hemodynamic support (ie, mechanical and/or medical) during PCI, the need for invasive or noninvasive ventilation during PCI, and the rate of failed PCI.
Methods, "Power Calculation":
Based on previously published data, the primary outcome of a needle-to-balloon time of 10 minutes or less was used for sample size calculation.
Methods, "Study Design and Population", the only statement about the registration:
This study was prospectively registered in Clinical Trials.gov (NCT05415085).
The same paragraph gives the enrollment period: "The study included patients with a diagnosis of STEMI who underwent primary PCI between April 1, 2021, and August 31, 2022."
Results, "Primary and Secondary Outcomes", which holds the paper's only result for door-to-balloon time:
The primary outcome of a needle-to-balloon time of 10 minutes or less was achieved in 46 of the 90 patients (51.1%) who underwent culprit lesion PCI before CAG and in 18 of the 94 patients (19.1%) who underwent CAG followed by culprit lesion PCI (odds ratio, 4.4 [95% CI, 2.2-9.1]; P < .001).
There was no significant difference in the mean (SD) door-to-balloon time (61.6 [45.8] vs 73 [59.8] minutes; P = .12).
The main text contains neither "amend" nor "changed". Door-to-balloon time appears in it only as background (Introduction and Discussion, citing earlier studies) and in the Results sentence above. The Abstract, Key Points and Conclusions do not mention it.
Supplement 1 ("Study Protocol and Statistical Analysis Plan", jamanetwopen-e243729-s001.pdf), section "3. Endpoints", is the only place the change is mentioned:
a. Primary endpoint: o Reduction in Door-to-Balloon (D2B) time Protocol amendment: The investigators ask the IRB for a study protocol amendment due to reconsideration of study protocol and primary outcome: o Revised primary outcome: Needle-to-balloon time within 10 minutes.
The protocol text carries no date for the amendment or for its approval. Its sample-size paragraph uses the revised outcome, in the same words as the paper's Power Calculation section. Its "Study Cohort" section gives an "Estimated number of 300 patients (~150 each group)". The case report form appended to it is labelled "February 2021" and "Ver 1.4". That label belongs to the form; it does not date the amendment.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Door-to-balloon time ("Reduction in door-to-balloon time"; change in minutes) | Sole primary in v0 (2022-06-08); removed in v1 (2023-10-17) | Not named as an outcome. One Results sentence: mean 61.6 vs 73 min, P = .12. Not in the Abstract, Key Points or Conclusions | demoted |
| Needle-to-balloon time of 10 minutes or less | Absent from v0; sole primary in v1 (2023-10-17, after primary completion) | Primary outcome: 46/90 (51.1%) vs 18/94 (19.1%), OR 4.4 (95% CI 2.2-9.1). Also the basis of the power calculation | matched to v1; added relative to v0 |
| Needle-to-balloon time in minutes (the v1 description, "Change in needle-to-balloon time in minutes") | Description text of the v1 primary | Reported as mean 11.4 vs 17.3 min, P < .001, not labelled primary | matched (wording difference: the registry title is a threshold, its description is continuous) |
| Hemodynamic support during PCI | Secondary, v0 and v1 | Secondary | matched |
| Respiratory support during PCI | Secondary, v0 and v1 | Secondary ("invasive or noninvasive ventilation") | matched |
| Rate of failed PCI | Secondary, v0 and v1 | Secondary | matched |
The main text gives no account of the change. A reader of the article alone would see a primary outcome that matches the current registry record and a statement that the trial was prospectively registered.
Figures
Figure 1. Registry events (blue, above the axis) and paper events (orange, below the axis) on one calendar axis. The paper's enrollment period (April 1, 2021 to August 31, 2022) is the orange bar. The light-grey band marks the period after the registered study start, and the darker band the period after the registered primary completion date (2022-06-30). The record was first submitted on 2022-04-27, 361 days after the registered start and 391 days after the paper's enrollment start, and its first version (v0, QC 2022-06-08) named door-to-balloon time as the primary outcome. The circled event, v1 on 2023-10-17, replaced the primary outcome with needle-to-balloon time of 10 minutes or less and changed enrollment from 250 (estimated) to 106 (actual). It falls after primary completion, after study completion (2023-04-13) and 164 days before publication on 2024-03-29.
Figure 2. Left column: the primary outcome in each registry version, quoted from the record (v0, 2022-06-08; v1, 2023-10-17). Right column: how the paper treats each one. The v1 outcome matches the paper's primary outcome, which carries the headline result (51.1% vs 19.1%; OR 4.4). The v0 outcome, door-to-balloon time, is not named as an outcome in the paper. Its only result is one Results sentence reporting no significant difference (61.6 vs 73 minutes; P = .12). The connector labels state the relation (REPLACED, DEMOTED, MATCHED). The bottom panel summarises the disclosure: none in the main text, one undated sentence in Supplement 1. The secondary outcomes are the same in both registry versions and in the paper.
Why it matters
A registered primary outcome lets readers judge whether the outcome that carries a trial's conclusions was chosen before the results were known. Here the first registered primary outcome, door-to-balloon time, is the one the trial's registry summary still names as its aim. It was replaced in the registry on 2023-10-17, after the registered primary completion date and after study completion. The paper's headline findings rest entirely on the replacement outcome: the Key Points finding (51.1% vs 19.1%), the abstract's odds ratio of 4.4, and the conclusion that culprit lesion PCI before complete CAG "resulted in shorter reperfusion times".
For the originally registered primary outcome the paper reports a mean (SD) door-to-balloon time of 61.6 (45.8) vs 73 (59.8) minutes, "no significant difference", P = .12. That result appears once, in the Results text. The Abstract, Key Points and Conclusions do not report it, and nothing in the paper tells the reader that it was the registered primary outcome.
The registry does not show when the decision to change the outcome was made. The protocol supplement records that an amendment was requested but gives no date. Without that date, a reader cannot tell whether the change came before enrollment, during it, or after outcome data were available. Each of these calls for a different reading of the result, and CONSORT 2010 item 6b asks trial reports to state any changes to trial outcomes after the trial commenced, with reasons.
The registration timing adds to this. The record was first submitted on 2022-04-27, 391 days after the paper's enrollment start (April 1, 2021) and 361 days after the registered start (2021-05-01). Under the ICMJE and WHO definition, which requires registration before the first participant is enrolled, the registration was retrospective. The paper's statement that the study was "prospectively registered" therefore does not match the record. The door-to-balloon primary was itself first recorded after about a year of enrollment.
Explanations considered
- Paper role (secondary analysis or a different paper for the registered outcome). Does not account for it. This is the main report of the trial. It cites only NCT05415085 and presents its outcome as "the primary outcome of this study". No other publication from the trial is cited.
- Disclosure. Partly addresses it, in a supplement only. Supplement 1 states that an amendment of the primary outcome was requested, "due to reconsideration of study protocol and primary outcome". The main text, which most readers and indexing services see, does not mention it. The supplement gives no date for the amendment or its approval, no reason beyond the quoted phrase, and no indication of whether outcome data had been seen. The main text's statement "This study was prospectively registered" does not reflect the change or the registration date.
- A protocol amendment approved before enrollment, with a late registry update. Possible, and it would be a legitimate reason for the change. The published material cannot confirm it. The protocol supplement does not date the amendment. The first registry version, submitted in April 2022 after about a year of enrollment, still named door-to-balloon time, and the registry was updated only after the trial was completed. If the amendment came before enrollment, the paper could say so with the IRB approval date.
- Wording differences (the two outcomes are the same measure described differently). Does not account for it. Door-to-balloon time runs from arrival at the hospital to first balloon inflation; needle-to-balloon time runs from the first arterial puncture to first balloon inflation (the paper defines both time points in Methods, "Data Collection"). They are different intervals, and the paper reports them separately, with different results (P = .12 vs P < .001). A smaller wording point concerns v1 itself: its title names a threshold ("10 minutes or less") while its description reads "Change in needle-to-balloon time in minutes". The paper's primary analysis uses the threshold, so it matches the v1 title.
- Correction or registry note. None found (see Status). The registry record has no note explaining the change, and the brief summary still states the door-to-balloon aim.
Separately, three other registry-paper differences are not part of the outcome change and are noted for completeness only. The registry gives enrollment as 106 (actual), while the paper reports 214 randomized in Results (106 and 108 per arm) and 216 in the Abstract. 106 is the number the paper gives for the culprit-lesion-first arm. The registry lists masking as "DOUBLE" (participant, care provider), while the paper describes an "open-label" trial. The registered primary completion date (2022-06-30) precedes the end of enrollment reported in the paper (August 31, 2022).
Status
As of 2026-10-23:
- Registry history. ClinicalTrials.gov API v2 current record, fetched 2026-10-23 (versionHolder 2026-10-22): last update submitted 2023-10-17, posted 2023-10-19. The version history (2 versions) and the archived v0 record were retrieved on 2026-10-22 from the internal history routes (
/api/int/studies/NCT05415085?history=trueand/api/int/studies/NCT05415085/history/0). The same history endpoints returned HTTP 403 when retried on 2026-10-23, and the history tab has no static content. The v1 content is identical to the current record. - Publisher version. The publisher's page (jamanetwork.com/journals/jamanetworkopen/fullarticle/2816961) carries the same registration statement and primary-outcome sentence as the PMC archive and shows no correction, erratum or editorial notice. Its comments section shows no comments. All paper quotes above were checked verbatim against the PMC-archived text. The supplement quotes were checked against the text extracted from Supplement 1 (jamanetwopen-e243729-s001.pdf).
- Corrections. Crossref lists no update relations for the DOI. PMC and Europe PMC list no correction, retraction or comment notices. A web search found no letter, comment or correction about the trial.
- PubPeer. Could not be checked: both the API and the search page refused automated access (HTTP 403).
- Sources. PMC10980970 and its Supplement 1; https://clinicaltrials.gov/study/NCT05415085 (current record and history); Crossref; Europe PMC; the publisher's article page.
Suggested correction
- In Methods, state that the primary outcome registered at ClinicalTrials.gov on 2022-06-08 was door-to-balloon time, that it was changed to needle-to-balloon time of 10 minutes or less, and give the date of the protocol amendment and its IRB approval, the reason, and whether the change was made before or after any outcome data were seen. The registry update date (2023-10-17) should also be stated.
- Report door-to-balloon time as the originally registered primary outcome, with its effect estimate and confidence interval, in the Results and in the Abstract.
- Replace "This study was prospectively registered" with the registration dates: first submitted 2022-04-27 and first posted 2022-06-10, after enrollment began on April 1, 2021.
- Minor: reconcile the number randomized (216 in the Abstract, 214 in Results). Ask the sponsor to update the registry's enrollment figure, masking field and brief summary so that they agree with the trial as reported.
Paper reports dose-limiting toxicity as primary endpoint; the only registered primary outcome is response rate
Cell Rep Med (2026) · doi:10.1016/j.xcrm.2026.102967 · PMC13522775 · NCT05459441
Summary
The ClinicalTrials.gov record for this trial has had one version (2022-07-12), and its sole primary outcome is "Clinical response rate", the percentage of patients with complete or partial response, at 24 months. The published report states that "The primary endpoint was the incidence of dose-limiting toxicities (DLTs)". DLT incidence appears nowhere in the registry, and the paper places radiographic response among "secondary or exploratory endpoints", reported as best overall response rather than at 24 months. The paper cites the registration number but does not mention that the registered primary outcome differs, and it gives no protocol amendment or reason for the difference. The paper reports that "There were no dose-limiting toxicities observed during dose escalation". The registered primary outcome is also reported, in Table 3 and outside the abstract: 2 of 17 efficacy-evaluable patients (11.8%) had a partial response and none had a complete response.
What was registered
The registry history (ClinicalTrials.gov record-history service, checked 2026-10-22 and 2026-10-23) lists exactly one version, version 0, dated 2022-07-12. The service reports outcomesUpdateCount: 0, and the current public record (API v2) has the same outcomes. The record was first submitted on 2022-06-24 and first posted on 2022-07-15. Its last update was posted on 2022-07-15. It gives the study start as 2022-01-01 (actual), so the trial was registered 174 days after it started, and no version exists from before the start date. The phase is "Early Phase 1". The brief title begins "Efficacy and Safety Evaluation of Percutaneous Ommaya Capsule Injection of Autologous Bi-dimensional Specific T Cells…".
| Version (date) | Type | Measure | Time frame | Description (verbatim) |
|---|---|---|---|---|
| v0 (2022-07-12) | Primary | "Clinical response rate" | "24 months" | "The percentage of patients with CR and PR in the total number of patients in the same period" |
| v0 (2022-07-12) | Secondary | "Overall survival (OS)" | "24 months" | "Frome the starting date of the enrollment until the date of the first documented disease progression or the date of the death from any cause,whichever comes first" |
| v0 (2022-07-12) | Secondary | "Safty(adverse events)" | "12 months" | "Number of participants with treatment-related adverse events as assessed by CTCAE v5.0" |
The other registered fields were never updated. The estimated primary completion date is 2023-06-30 and the estimated completion date is 2023-12-31. Enrolment is "30 (ESTIMATED)". The overall status is now "UNKNOWN", with last known status "RECRUITING" and status verified 2022-02. "Dose-limiting toxicity" does not appear anywhere in the record. The record's submission-tracking data show a results submission dated 2026-08-06, released and withdrawn the same day, and another released on 2026-08-08. As of the check, no results are posted and the submitted content is not public.
What the paper reports
The primary endpoint, from STAR Methods, "Objectives and endpoints":
The primary objective of this study was to evaluate the safety and tolerability of intracranial administration of non-chimeric autologous expanded T cells combined with pemetrexed. The primary endpoint was the incidence of dose-limiting toxicities (DLTs), assessed during a prespecified DLT evaluation window defined as inside the intracranial treatment cycle (once weekly for 5 weeks).
Secondary or exploratory endpoints included overall survival (OS), progression-free survival (PFS), and disease control rate (DCR).
Tumor response was determined based on the best overall response from treatment initiation until disease progression or last adequate tumor assessment.
The DLT definition, from STAR Methods, "Dose-escalation design":
DLT was defined as the occurrence of ≥1 treatment-related death or ≥3 unanticipated grade ≥3 treatment-related adverse events within the predefined DLT evaluation period.
From the Introduction:
The primary objective was to assess safety and tolerability, with comprehensive monitoring of both systemic and intracranial AEs.
Clinical outcomes, including radiographic response, progression-free survival (PFS), and overall survival (OS), were assessed as secondary or exploratory endpoints to characterize potential signals of therapeutic activity.
The result for this primary endpoint, from Results, "Safety of SCOPE-T treatment":
There were no dose-limiting toxicities observed during dose escalation; therefore, the highest dose tested (5 × 108 RAK cells) was selected for dose expansion and designated as the recommended phase 2 dose (RP2D).
(The "108" is 10^8 as rendered in the PMC text.) The response results appear in Results under the heading "Exploratory radiographic and clinical assessments", and in Table 3 ("Summary of clinical responses in the PPS and EES"):
Objective response rate | 2 (10.5%) | 2 (11.8%)
Benefit beyond imaging evaluationa | 19 (100.0%) | 17 (100.0%)
Clinical benefit rate | 19 (100.0%) | 17 (100.0%)
(The trailing "a" marks a footnote listing clinical and laboratory improvements.) The Results attribute clinical benefit to RANO assessment: "Using RANO-BM and RANO-LM criteria, all 17 patients in the EES demonstrated clinical benefit."
The sample size is discussed in STAR Methods, "Study design and patients" ("The selected sample size (n = 25) is consistent with typical phase 1/1b trials designed to evaluate safety and tolerability."), and in "Statistical analysis" ("No formal hypothesis testing or power calculation was prespecified.").
The paper refers to the registration only by its NCT number (Abstract, Introduction) and in the line in STAR Methods, "Clinical trial registration":
This trial was registered with https://clinicaltrials.gov/study/NCT05459441.
The paper never mentions that response rate was the registered primary outcome, or any protocol amendment or change of endpoints. It does not give a registration date. Apart from the per-protocol analysis set (defined in STAR Methods, "Analysis populations", as "patients who completed at least one treatment cycle without major protocol deviations"), the study protocol is referred to only in the Table 4 footer ("OS and PFS: According to protocol, data cutoff as of July 6, 2024."). The same footer adds "OS∗: Updated data follow-up as of May 01, 2026." for a separate survival column. Neither a protocol nor a statistical analysis plan is published with the article: the supplement is Document S1 (Figures S1–S9, Tables S1–S3), Data S1 (xlsx) and Document S2 (the article plus supplement).
Every quotation above was checked character for character against the article as archived in PubMed Central (src/show.py PMC13522775 --grep), and each occurs exactly once. The endpoint sentences, the registration line and the Table 3 rows also appear unchanged in the publisher's typeset PDF (Document S2). make_figures.py.txt reruns the PMC check, and the registry check against API v2.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Clinical response rate (CR + PR / total patients) | Primary, 24 months (v0, 2022-07-12) | Radiographic response among "secondary or exploratory endpoints". Not in the named list of secondary/exploratory endpoints (OS, PFS, DCR). Reported as best overall response in Table 3: ORR 2/19 (10.5%) in the PPS and 2/17 (11.8%) in the EES | Demoted; time point changed (best overall response, not 24 months) |
| Incidence of dose-limiting toxicities | Not registered at any level (v0) | Primary endpoint, with a 5-week DLT window | Added |
| Treatment-related adverse events (CTCAE v5.0) | Secondary, 12 months (v0) | Reported (Table 2; Table S3; abstract: "no treatment-related adverse events greater than grade 2"), but not listed as an endpoint. No time frame stated | Reported; not named an endpoint; time frame not stated |
| Overall survival | Secondary, 24 months (v0). The registered description reads like a progression-or-death definition | Secondary or exploratory: Kaplan-Meier median and 1-year rate | Matched by name; 24-month time frame not used |
| Progression-free survival | Not registered (v0) | Secondary or exploratory | Added |
| Disease control rate (CR + PR + SD) | Not registered (v0) | Secondary or exploratory. Table 3 has no row named DCR, but lists PR and SD counts and a "Clinical benefit rate" | Added |
| Enrolment (context) | 30, estimated (v0) | 25 enrolled; "The selected sample size (n = 25)…" | Differs; not explained |
Figures
Figure 1. Timeline of the trial and its registration. Blue circles are registry events: first submission (2022-06-24), version 0 (2022-07-12; the only version), first posting (2022-07-15) and the registered estimated primary completion date (2023-06-30, never updated), all on the axis. The results submissions (2026-08-06 and 2026-08-08; not posted as of 2026-10-23) share one blue circle, drawn on its stem above the version bar because on the axis it would sit under the publication marker one to three days earlier. Orange squares are trial and paper events taken from the article: study start and first enrolment (2022-01-01), last enrolment (2024-03-30), data cutoff (2024-07-06), journal submission (2025-11-28) and online publication (2026-08-05). The shaded band marks the 174 days between study start and registration. The blue bar shows that version 0 has been in force from 2022-07-12 to the check date (2026-10-23). No registry version changed the primary outcome, either before or after primary completion. The difference arises between the registry and the paper, not within the registry. The registered primary completion date is shown only as the registry's own, never-updated estimate; it is not a measure of reporting time (enrolment ended 2024-03-30 and the paper's data cutoff was 2024-07-06).
Figure 2. Registered outcomes compared with the outcomes in the paper. The left column (blue) shows the outcomes in version 0 (2022-07-12) with their registered type and time frame. The right column (orange) shows how the paper designates and reports each one. Every connector carries a text label. DLT incidence is labelled ADDED: it is the paper's primary endpoint and is absent from the registry. Its result box quotes the paper: no DLTs were observed during dose escalation. The registered primary, clinical response rate, is labelled DEMOTED and time point changed: the paper treats it as secondary or exploratory and reports it as best overall response, with the result from Table 3 (ORR 2/17, 11.8%, exact 95% CI 1.5–36.4%, in the EES; 2/19, 10.5%, in the PPS; no CR). The registered adverse-event outcome is labelled REPORTED. Overall survival is labelled MATCHED by name. PFS and DCR are labelled ADDED. The footnote gives the response rate over all 25 enrolled patients (2/25 = 8%, exact 95% CI 1.0–26.0%), computed here from Tables 3–4 because the registered description uses "the total number of patients" as its denominator; the paper does not report this figure. Exact intervals are Clopper-Pearson, the method the paper states for categorical variables.
Why it matters
- The registered primary outcome is reported, but it is framed as exploratory and left out of the abstract. The CR + PR rate is 2/17 (11.8%; exact 95% CI 1.5–36.4%) in the efficacy-evaluable set and 2/19 (10.5%) in the per-protocol set, with no complete responses. Over all 25 enrolled patients, the denominator the registered description implies, it would be 2/25 (8%). Of the two partial responses, one patient (Pt5) went from SD to PR and the other (Pt6) from PR to SD (Table 4). No response threshold was prespecified and no hypothesis was tested, so the result is not formally positive or negative. The abstract does not report the objective response rate.
- The abstract's efficacy statements rest on outcomes that were not registered. The abstract says "Exploratory radiographic and clinical assessments suggest disease stabilization in evaluable patients" and concludes that the findings "indicate preliminary signals of therapeutic activity". A Highlight reads "Preliminary disease control observed in a small early-phase cohort". These rely on disease control (CR + PR + SD) and on clinical benefit: Table 3 gives a "Clinical benefit rate" and a "Benefit beyond imaging evaluation" row, each 17/17 (100.0%) in the EES, and the Results attribute clinical benefit in all 17 to RANO-BM/RANO-LM assessment. Neither disease control nor clinical benefit appears in the registry.
- The headline safety conclusion rests on an unregistered primary endpoint. The abstract's main finding is safety and tolerability ("no treatment-related adverse events greater than grade 2"). The registry lists safety only as a secondary outcome, defined as a count of participants with treatment-related AEs at 12 months. The DLT endpoint the paper calls primary (result: "There were no dose-limiting toxicities observed during dose escalation"), and its "prespecified" 5-week window, cannot be checked against any public prespecified document. The safety data themselves are reported in full: Table 2 lists all AEs by grade for the 25 patients.
- The registered time frame is not used and is not addressed. The registry specifies response at 24 months. The protocol data cutoff for OS and PFS (2024-07-06, Table 4 footer) came about 3 months after the last enrolment (2024-03-30), so no patient enrolled after early July 2022 could have had 24 months of follow-up at that cutoff. The Table 4 footer also gives "OS∗: Updated data follow-up as of May 01, 2026.", so longer follow-up exists for some patients, but that update covers survival only: no response status at 24 months is reported. The paper reports best overall response instead and does not discuss the registered time frame.
- Readers comparing paper and registry get conflicting accounts. Anyone who follows the cited registration link finds response rate as the only primary outcome and no DLT endpoint. Systematic reviewers and outcome-reporting audits would code this as an undisclosed change of primary outcome.
On severity: safety as the primary endpoint is conventional for a first-in-human phase 1 study, the paper repeatedly calls its efficacy findings exploratory, and the registered primary result appears in Table 3. The safety conclusion therefore does not depend on the discrepancy. What is missing is any statement that the registered primary outcome was different.
Explanations considered
- Is this a secondary analysis or a partial report? No. The paper is the primary report of the trial. It covers all 25 patients enrolled between 1 January 2022 and 30 March 2024, at both sites, with safety, response and survival. The registry has a single primary outcome, so it cannot have been split across publications. The other NCT numbers in the paper (NCT02208362, NCT04185038, NCT04196413) belong to other trials and are cited to justify sample size and dosing.
- Does the paper disclose the change? No. Its only registration statement is "This trial was registered with https://clinicaltrials.gov/study/NCT05459441." No passage refers to the registered outcomes, an amendment, a change of endpoints or a reason for one. Searches of the text and the supplement for "primary outcome", "amend", "registered" and "response rate" found nothing relevant. The only "amend" hit is an acknowledgment of help with "amendments to enhance the manuscript".
- Was there a protocol amendment? The Table 4 footer refers to a protocol ("According to protocol, data cutoff as of July 6, 2024.") and the per-protocol set is defined by the absence of "major protocol deviations", but no protocol, amendment history or statistical analysis plan is published. The registry was never updated after 2022-07-12, so it records no amendment either. A change in the protocol may have happened, but there is no public record of one.
- Are these the same outcome under different wording? No. CR + PR response rate measures efficacy. DLT incidence ("≥1 treatment-related death or ≥3 unanticipated grade ≥3 treatment-related adverse events" within the DLT window) measures safety. The registered safety outcome, the number of participants with treatment-related AEs (CTCAE v5.0) at 12 months, is a secondary outcome. It is also a different measure from DLT incidence, with a different time window.
- Is a safety primary endpoint standard for phase 1? Yes. It is a legitimate reason a team might name DLT incidence as primary, and the paper's design (dose escalation guided by DLTs) is consistent with it. It does not account for the registry stating otherwise, though. The registered title begins "Efficacy and Safety Evaluation", and the registered primary outcome is efficacy. The paper does not explain the difference.
- Is the registry simply out of date? The record has not been updated since July 2022: the status is "UNKNOWN", and the completion dates and enrolment are still estimates. A stale record could explain why the registry was never aligned with a later protocol. It does not make the change disclosed. The registry is the only public prespecification, and the paper calls its DLT window "prespecified" without saying that the registry lists a different primary outcome.
- Did the registration timing play a part? The trial was registered 174 days after it started, so even the registered outcomes were entered after enrolment had begun. The paper neither claims prospective registration nor gives a registration date. This affects how the registration can be interpreted, but it does not explain the outcome difference: the registry and the paper still disagree.
- Is it a conversion or typesetting artifact? No. The same endpoint sentences and the same registration line appear in the PMC-archived text and in the publisher's typeset PDF (Document S2).
None of these accounts for the absence of disclosure. The issue documented here is that the difference is not acknowledged. It may well have had a legitimate reason.
Status
As of 2026-10-23:
- Registry history: checked. The ClinicalTrials.gov record-history service (
/api/int/studies/NCT05459441?history=trueand/history/0) lists one version (v0, 2022-07-12), withoutcomesUpdateCount: 0and the original primary and secondary outcomes unchanged. The current API v2 record matches v0. The website history tab (?tab=history) returns only a JavaScript shell through WebFetch. The internal routes return 403 to a cold client but 200 within a session that first calls API v2, which is how they were read. Results were submitted on 2026-08-06 and 2026-08-08 and are not posted. - Publisher version: checked through the typeset article in Document S2 (Europe PMC supplementary files: "Cell Reports Medicine 7, 102967, August 18, 2026"; Received 28 November 2025, Revised 25 March 2026, Accepted 7 July 2026, Published 5 August 2026). The primary-endpoint sentence, the secondary/exploratory sentences, the registration line and Table 3 match the PMC text. The publisher's HTML pages (cell.com, ScienceDirect) returned 403. The DOI resolves to Elsevier's linking hub.
- Corrections and notices: none found.
src/known.pyreturns no Crossref updates, no PMC notices and no Europe PMC notices, and Crossref has no update-to relation. A web search found no erratum or correction. - PubPeer: could not be checked. Its search page and API returned 403 (human-verification wall), so a PubPeer thread cannot be ruled out. COMPare does not apply: it covered 2015–2016 trials in five journals.
- Sources: https://clinicaltrials.gov/api/v2/studies/NCT05459441; https://clinicaltrials.gov/api/int/studies/NCT05459441?history=true; https://clinicaltrials.gov/api/int/studies/NCT05459441/history/0; PMC13522775 (
src/show.py, full text and Tables 2–4); https://www.ebi.ac.uk/europepmc/webservices/rest/PMC13522775/supplementaryFiles (mmc1.pdf, mmc2.xlsx, mmc3.pdf);src/known.py 10.1016/j.xcrm.2026.102967 --pmcid PMC13522775; https://doi.org/10.1016/j.xcrm.2026.102967.
Suggested correction
- A correction notice, or an added paragraph in "Objectives and endpoints". It should state that the ClinicalTrials.gov record (NCT05459441, registered 2022-06-24, after the 2022-01-01 start) lists "Clinical response rate" (CR + PR, 24 months) as the sole primary outcome and does not list DLT incidence. It should also give when and why the primary endpoint became DLT incidence (protocol version and date), or state that the registration was inaccurate from the outset.
- The protocol and statistical analysis plan, with version dates, added as supplementary material, so that the "prespecified" DLT window and the endpoint hierarchy can be checked.
- The registered primary outcome reported as registered, or a statement of why it cannot be. That means CR + PR over all enrolled patients (2/25 from the paper's Tables 3–4) alongside the EES and PPS figures, and the response status at 24 months (for instance from the May 2026 follow-up the paper already uses for survival), or a note that follow-up did not allow it. The abstract could give the objective response rate next to the disease-stabilization statement.
- The ClinicalTrials.gov record updated: outcome measures (with the change recorded in the registry's history), actual enrolment (25), actual primary completion and completion dates, and recruitment status. The results submitted in August 2026, not posted as of 2026-10-23, would let readers check the registered outcomes once they are public.
Registered 6-week re-intervention count reported instead as a 6-month total procedure count, without disclosure
Gut (2018) · doi:10.1136/gutjnl-2017-315335 · PMC6582745 · NCT02685865
Summary
This paper is the primary report of a single-centre randomised trial comparing lumen-apposing metal stents (LAMS) with plastic stents for EUS-guided drainage of walled-off necrosis (WON). In every ClinicalTrials.gov version (v0, 2016-02-18, to v7, 2019-02-08) the primary outcome has a 6-week time frame, and its description defines the counted events as re-interventions: repeat EUS-guided drainage, direct endoscopic necrosectomy (DEN) or a percutaneous catheter "due to inadequate treatment response". v0 to v3 title it "The number of re-interventions performed between metal and plastic stent groups to achieve treatment success". v4 (2017-09-13) changed "re-interventions" to "interventions" in the measure title and in the first sentence of the description, but left the definition, the hypothesis and the time frame unchanged; v4 to v7, including the current record, carry this wording. The paper's primary outcome is instead the "total number of procedures needed to achieve treatment success at 6-month follow-up". This count also includes the index procedure, ERCP, enteral feeding tubes and procedures for adverse events, so it differs from the registered outcome in both time point and construct. The paper does not mention the difference, and it reports no per-patient result or between-group test for the registered outcome.
What was registered
The record has eight versions, v0 to v7 (indices from the ClinicalTrials.gov history API). The record was first submitted on 2015-11-24. The earliest version, v0, is dated 2016-02-18 (first posted 2016-02-19), six days after the actual study start of 2016-02-12. v0 gave the start only as "2016-02", and from v3 (2017-04-21) the start is "2016-02-12" (actual). No version exists on or before the study start.
v0, 2016-02-18 (status Not yet recruiting; estimated enrolment 62). One primary outcome:
Measure: "The number of re-interventions performed between metal and plastic stent groups to achieve treatment success" (time frame: "6 weeks")
Description: "The primary objective of this study is to compare the number of re-interventions performed between metal and plastic stent groups to achieve treatment success at 6 week follow-up. Re-intervention is defined as the need for repeat EUS-guided drainage of WON using the same stent type and/or DEN and/or insertion of percutaneous catheter due to inadequate treatment response following initial WON drainage. As FCSEMS have larger diameter than plastic stents, the investigators hypothesize that the no. of re-interventions performed will be lower in the FCSEMS group compared to the plastic stent group."
The registered secondary outcome for treatment success was:
"Treatment success" (time frame: "6 week"): "Decrease in the size of WON to <33% of the original size and resolution of symptoms at 6-week outpatient follow-up (from day of hospital discharge) WITHOUT the need for cross-over to minimally invasive surgical debridement"
The other registered secondaries were technical success (day 0), treatment failure (6 months), recurrence (6 months), procedure duration (day 0), adverse events (6 months), duration of hospitalisation (6 months) and cost analysis (6 months).
v1 (2016-04-12), v2 (2017-02-13), v3 (2017-04-21). No change to the outcome measures. v1 changed the Study Identification module (brief title from "Trial Comparing FCSEMS and Plastic Stents" to "Comparison of FCSEMS and Plastic Stents") and the status, description, conditions and contacts modules; v2 and v3 changed the status module (status and dates) only.
v4, 2017-09-13 (status Active, not recruiting; actual enrolment 60; primary completion "2017-07-10", actual). The module labels include "Outcome Measures". The primary measure title became:
"The number of interventions performed between metal and plastic stent groups to achieve treatment success" (time frame: "6 weeks")
In the first sentence of the description, "re-interventions" became "interventions" ("...to compare the number of interventions performed between metal and plastic stent groups to achieve treatment success at 6 week follow-up."). The rest of the description was not changed: the definition still begins "Re-intervention is defined as the need for repeat EUS-guided drainage of WON using the same stent type and/or DEN and/or insertion of percutaneous catheter due to inadequate treatment response following initial WON drainage", and the hypothesis still refers to "the no. of re-interventions". The secondary treatment-success description became "Resolution of WON on CT scan and resolution of symptoms at 6-week outpatient follow-up (from day of hospital discharge) WITHOUT the need for cross-over to minimally invasive surgical debridement", still at "6 week". v4 also made wording edits to two other secondary-outcome descriptions, with no change to their time frames: treatment failure gained the words "or underlying diseaseor underlying disease." (as written), and procedure duration now starts when the "endoscope [is] inserted into the gastrointestinal lumen to perform transmural drainage" instead of "inserted into the oropharynx". This edit was made after the actual primary completion date then registered (2017-07-10; v6 later changed it to 2018-07-10) and nine days before the manuscript was received (2017-09-22).
v5, 2018-05-30. This is the last version before online publication (2018-06-01). The outcomes are the same as in v4.
v6, 2018-07-31, and v7, 2019-02-08. No outcome change. v6 changed the actual primary completion date from 2017-07-10 to 2018-07-10. v7 set the status to Completed. The current record (fetched from /api/v2/studies/NCT02685865 on 2026-10-23; last update posted 2019-02-11) matches v7, and it still gives the primary time frame as "6 weeks".
No version gives a 6-month time frame for the primary outcome. None defines the primary outcome as a count that includes the index procedure, ERCP, enteral feeding tubes or procedures for adverse events.
What the paper reports
The primary outcome, Abstract (Design):
"Primary outcome was comparing total number of procedures to achieve treatment success defined as symptom relief in conjunction with WON resolution on CT at 6 months."
Introduction, final paragraph:
"The primary objective of this randomised trial was to evaluate the effectiveness of LAMS by comparing the total number of procedures required to achieve treatment success between patients undergoing LAMS and plastic stent placement at 6-month follow-up."
Methods > Outcome measures:
"The main outcome measure was to compare the total number of procedures needed to achieve treatment success at 6-month follow-up."
Methods > Definitions:
"The total number of procedures required to achieve treatment success included the index endoscopic intervention, endoscopic reinterventions, percutaneous drain placement, ERCP, enteral feeding tube placement and any procedures performed for the management of adverse events or disease-specific events in each cohort during the 6-month follow-up period."
"Treatment success was defined as resolution of WON on CT scan in association with clinical resolution of symptoms at 6-month follow-up."
Methods > Sample size calculation:
"A two-sided sample size calculation was performed to detect a difference of one in the total number of procedures needed to achieve treatment success (assuming one for LAMS and two for plastic stents)"
Methods > Patient follow-up. This is the only role the 6-week visit has in the paper:
"On initial outpatient follow-up at 6 weeks postintervention, all patients underwent CT to assess treatment response prior to stent removal and an ERCP to assess for pancreatic duct integrity."
The primary result, Abstract (Results) and Table 3 ("Total no. of procedures for treatment success": mean 2.8 (SD 1.2) vs 3.2 (SD 1.5); median 2 vs 3; range 2–7 vs 2–7; p=0.192):
"There was no significant difference in total number of procedures performed (median 2 (range 2–7) LAMS vs 3 (range 2–7) plastic, p=0.192)"
The paper's statements about the registration and the protocol:
- The only registration statement is at the end of the Abstract: "Trial registration number NCT02685865." The words "registered" and "registration" (outside that line) do not appear in the text. The registry's hyphenated term "re-intervention" does not appear either, but "reinterventions" does, five times: as one component of the total procedure count (Methods > Definitions, "endoscopic reinterventions"), in the 3- and 6-month follow-up interviews ("disease-related reinterventions and readmissions") and three times in the Discussion. It is never reported as a separate outcome.
- The only protocol amendment described is in Results > LAMS cohort: "Therefore, the safety board recommended an amendment to the study protocol whereby a CT was obtained at 3 weeks in the LAMS cohort followed by stent removal if the WON had resolved." The Abstract also mentions it: "Interim audit resulted in protocol amendment where CT scan was obtained at 3 weeks postintervention followed by LAMS removal if WON had resolved."
- The only limitation about the primary outcome is in Discussion, limitations paragraph: "Four, we did not report outcomes from a patient perspective and the primary outcome was not adjudicated in a blinded fashion."
- Searches of the archived text for "registered", "primary outcome", "amend", "changed", "protocol", "6 week", "6-week" and "reintervention" found no statement about a change to the primary outcome's definition or time point.
The online supplement (gutjnl-2017-315335supp001.pdf, retrieved from the Europe PMC supplementary-files archive) has four tables: costs; "Procedures performed to achieve treatment success"; Poisson regression; and linear regression. Supplementary Table 2 lists procedure totals by type (LAMS vs plastic): "Repeat EUS-guided drainage 2 / 5", "Endoscopic necrosectomy 4 / 6", "Percutaneous drain placement 2 / 5", "Enteral feeding tube placement 7 / 13", "ERCP for pancreatic duct integrity 31 / 29" and "Management of adverse events 14 / 3". The supplement says nothing about the registration or any outcome change.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Number of re-interventions (repeat EUS drainage with the same stent type, DEN or percutaneous catheter, for inadequate response) to treatment success | Primary in v0–v3 (2016-02-18 to 2017-04-21). From v4 (2017-09-13) the title says "interventions", but the description keeps the re-intervention definition (through v7, 2019-02-08) | Not an outcome. The Results text and Supp. Table 2 give only procedure totals by type (LAMS 2+4+2 = 8; plastic 5+6+5 = 16), with no per-patient summary and no test | not reported |
| Time point of the primary outcome | "6 weeks" in every version, v0–v7; description "at 6 week follow-up" | "at 6-month follow-up" (Abstract, Introduction, Methods) | time point changed |
| Total number of procedures to treatment success (index procedure, endoscopic reinterventions, percutaneous drains, ERCP, enteral feeding tubes, procedures for adverse or disease-specific events) | In no version | Primary outcome and basis of the sample size calculation; median 2 vs 3, p=0.192 | added (as primary) |
| Treatment success | Secondary, "6 week"; v0: "<33% of the original size" plus symptoms; v4 (2017-09-13): "Resolution of WON on CT scan" plus symptoms | Secondary, "at 6-month follow-up"; 29 (93.5%) vs 28 (96.6%), p=0.999; also the end point of the primary count | time point changed |
| Technical success, procedure duration, adverse events, length of stay, costs, recurrence, treatment failure | Secondary, v0–v7 (day 0 or 6 months) | Secondary outcomes or definitions in Methods; reported in Table 3 and the text | matched |
Figures
Figure 1. Timeline of NCT02685865 and the paper. Blue marks are registry events (above the axis). Filled circles are the first submission (2015-11-24), v0 (2016-02-18, six days after the actual study start of 2016-02-12) and v4. The diamond is the registered actual primary completion (2017-07-10). Open squares are registry versions that did not change the outcomes (v1–v3, v5, v6, v7); v5 (2018-05-30) sits beside the publication marker. The ringed marker is v4 (2017-09-13), the only version that edited the outcome measures. It falls after the actual primary completion date then registered (2017-07-10; v6 later changed it to 2018-07-10) and nine days before the manuscript was received. Orange marks are the paper (below the axis): the randomisation period it reports (February 2016 to March 2017), receipt (2017-09-22; its axis marker sits under the v4 marker nine days before it, and its stem leads down to the label), online publication (2018-06-01) and print (2019-07). The paper's 6-month primary outcome appears at no point in the registry. v6 (2018-07-31), after publication, moved the actual primary completion date to 2018-07-10. Source: make_figures.py.txt.
Figure 2. On the left, in blue, are the registered outcomes. On the right, in orange, are what the paper reports. Each connector carries a label for its status. Top row: the registered primary (re-interventions to treatment success at 6 weeks) was replaced by a 6-month total procedure count. That count adds the index procedure, ERCP, feeding tubes and procedures for adverse or disease-specific events. Its result was median 2 vs 3, p=0.192, with a minimum of 2 in each arm. The v4 title edit is noted in grey. Middle row (dashed outline: an outcome with no reported result): for the registered primary as defined, the paper gives only procedure totals for its three components (8 vs 16), with no per-patient summary, no test and no 6-week split. Bottom row: the registered secondary, treatment success at 6 weeks, is reported at 6 months. The footer lists the other procedure types in the paper's count (Supplementary Table 2) and the paper's only registration statement and only protocol amendment.
Why it matters
The paper reports no result for the registered primary outcome. The registered measure is a per-patient count of re-interventions done for inadequate treatment response, reaching treatment success at 6 weeks, and the paper gives no between-group comparison of it at any time point. The closest data are procedure totals for the three registered components: 8 in the LAMS arm (2 extra transmural tracts, 4 necrosectomies, 2 percutaneous drains) against 16 in the plastic arm (5, 6 and 5). These come from Results > LAMS cohort and Plastic stent cohort ("Additional procedures undertaken due to suboptimal treatment response at 72 hours (for ongoing SIRS) included creation of additional transmural tracts (n=2), direct endoscopic necrosectomy (n=4) and percutaneous drainage catheter placement (n=2)" and "...(n=5), direct endoscopic necrosectomy (n=6) and percutaneous drainage catheter placement (n=5)") and Supplementary Table 2. These are event totals, not patient-level data. They are not split at 6 weeks and are not tested, so the registered comparison cannot be reconstructed from the paper.
The substituted outcome is built differently in ways that bear on its result:
- Shared fixed components. Every patient's count includes the index procedure. Supplementary Table 2 lists "ERCP for pancreatic duct integrity" as 31 and 29, one per randomised patient, in line with the protocolised 6-week ERCP. Both arms therefore have a floor of 2 procedures, and the observed minimum is 2 in each arm (Table 3). Components that every patient receives add the same amount to both arms and shrink the relative contrast between them.
- Procedures for adverse events. The count includes "any procedures performed for the management of adverse events". Supplementary Table 2 gives 14 such procedures in the LAMS arm and 3 in the plastic arm. These were coil embolisations, endoscopies for bleeding or buried stents, ERCP for biliary stricture, retrieval of migrated stents and removal of a PEG-J tube. The registered outcome did not count them, and in the published count they add to the LAMS total.
- Different window. The registered window ended at 6 weeks. The published window runs to 6 months and takes in readmissions and later procedures.
So the paper's null primary result (p=0.192) and its "non-superiority" headline concern a different end point from the one registered. The totals for the registered components are lower in the LAMS arm (8 vs 16), but no test is given and these are not per-patient data. Whether the registered comparison would have been null or not cannot be judged from what is published. The registered hypothesis itself ("the no. of re-interventions performed will be lower in the FCSEMS group") is never tested as stated.
Two further points make the published primary harder to interpret:
- Sample-size assumptions. The power calculation assumes means of "one for LAMS and two for plastic stents" in "the total number of procedures". Under the published definition, every patient has at least two procedures (index plus ERCP), so an assumed mean of one in the LAMS arm is not possible. The paper does not explain how the assumptions relate to the definition.
- Totals that do not add up. The procedure totals in Supplementary Table 2, plus one index procedure per patient, come to 91 (LAMS) and 90 (plastic). These correspond to means of 2.94 and 3.10, against the reported 2.8 and 3.2 in Table 3, so the per-patient composition of the published count cannot be checked from the supplement either.
This is not a case where a null registered result became a positive headline: the published primary is itself null, and both outcomes are procedure counts. But the paper presents the 6-month total count as the trial's primary outcome. Readers, reviewers and meta-analysts who take it at face value would not know that the registered primary, measured at 6 weeks and restricted to re-interventions, is not reported.
Explanations considered
- Paper role. This is the trial's primary report: 60 patients were randomised, matching the registered actual enrolment of 60, and the paper gives full efficacy, safety and cost results with a sample-size calculation for the primary outcome. It is not a secondary or follow-up analysis that could legitimately use a different end point. This does not explain the discrepancy.
- Disclosure. None. The registration is cited only as "Trial registration number NCT02685865." The text searches listed above found no statement that the primary outcome, its time point or its counting rules differ from the registration. The supplement contains only results tables. Not disclosed.
- Protocol amendments. The paper describes one amendment, recommended by the safety board: "a CT was obtained at 3 weeks in the LAMS cohort followed by stent removal if the WON had resolved." It concerns imaging and stent removal in one arm. It does not concern the primary outcome's definition, its time point or which procedures are counted. A legitimate amendment to the outcome could exist, but the paper cites no published protocol or statistical analysis plan, the registry has no posted documents, and the change is not described. A real but unreported amendment would explain why the outcome differs, but not why the paper does not say so.
- Registry edit in v4. The v4 edit (2017-09-13) changed the title from "re-interventions" to "interventions". It could be read as a move towards a broader count. It did not, however, change the time frame (still 6 weeks), and the description still defines the counted events as re-interventions for inadequate response. The registry never described the paper's 6-month count. The edit was made after the then-registered actual primary completion date (2017-07-10; changed to 2018-07-10 in v6) and is not mentioned in the paper either.
- Wording differences. The registry's "metal" and "FCSEMS" arm is the paper's LAMS. The registry lists a "Hot Axios Fully covered self-expandable metal stent", and the paper describes the Hot AXIOS LAMS as "fully covered self-expandable metal stents", so the device is the same. "Number of procedures" and "number of re-interventions" could look like loose synonyms. They are not, because the paper explicitly adds the index procedure, ERCP, feeding tubes and adverse-event procedures. Six weeks and six months cannot be a wording difference.
- The 6-week visit as the assessment point. One reading is that the registry's "6 weeks" refers to the first outpatient CT visit, which still exists in the paper. But the registry ties both the count and treatment success to "6 week follow-up". The paper defines treatment success, which closes the count, "at 6-month follow-up", and it counts procedures "during the 6-month follow-up period". The paper's 6-week visit is the CT and ERCP and stent-removal visit, not the end point of the primary outcome.
Status
As of 2026-10-23:
- Registry history. All eight versions of NCT02685865 were reviewed: v0 2016-02-18, v1 2016-04-12, v2 2017-02-13, v3 2017-04-21, v4 2017-09-13, v5 2018-05-30, v6 2018-07-31 and v7 2019-02-08. The source was the ClinicalTrials.gov history endpoints (
/api/int/studies/NCT02685865/historyand/history/<version>), from copies retrieved for this project on 2026-10-22 and 2026-10-23; the live history endpoint returned HTTP 403 on 2026-10-23. The current record (/api/v2/studies/NCT02685865, fetched 2026-10-23) matches v7. No results are posted to the registry. - Publisher version. The text checked is the PMC archive of the version of record: Gut 2019;68(7):1200–1209, received 2017-09-22, revised 2018-04-09, accepted 2018-05-02, epub 2018-06-01. The online supplement (Supplementary Tables 1–4) was read from the Europe PMC supplementary-files archive. The paper cites no published protocol or statistical analysis plan.
- Corrections. No correction, erratum, expression of concern or retraction was found in Crossref (no
update-torelation), in PMC notices or in Europe PMC notices. - Letters. Europe PMC lists five "Comment in" items (PMIDs 30030302, 30301769, 30368455, 30389713 and 30448773, the last an authors' reply). Only one was read, doi:10.1136/gutjnl-2018-317777 (PMID 30389713), from an institutional repository copy. It discusses the primary outcome "total number of procedures needed to achieve treatment success at 6-month follow-up" but does not raise the registration. The authors' reply (PMID 30448773, doi:10.1136/gutjnl-2018-317848, "Authors reply: treatment strategy for necrotising pancreatitis", Gut 2020;69:189–190) is the most likely place for any post-publication explanation of the outcome definition, but it could not be accessed: Europe PMC has no full text, and the free publisher PDF that Unpaywall lists, the article page and an alternate BMJ host all returned HTTP 403 on 2026-10-23. A search-engine excerpt of it covers only the drainage algorithm (multigate drainage, CT at 72 hours), so whether it mentions the outcome definition or the registration is unknown. The other three letters were not read; none is open access in Europe PMC.
- PubPeer. The PubPeer API and site returned a bot-verification page (HTTP 403), and a web search found no PubPeer thread for this article. Whether it has PubPeer comments is unconfirmed.
- Sources. ClinicalTrials.gov record and history tab; PMC6582745; Crossref metadata; Europe PMC.
Suggested correction
The authors and the journal could add a statement to Methods > Outcome measures, along these lines:
"The primary outcome registered at ClinicalTrials.gov (NCT02685865; first submitted 24 November 2015) was the number of re-interventions (repeat EUS-guided drainage using the same stent type, direct endoscopic necrosectomy, or percutaneous catheter insertion for inadequate treatment response) needed to achieve treatment success at 6-week follow-up. The primary outcome reported here, the total number of procedures (including the index procedure, ERCP, enteral feeding tubes and procedures for adverse events) needed to achieve treatment success at 6-month follow-up, differs from the registered outcome in its definition and time point. [Date of and reason for the change.]"
The correction could also report the registered primary outcome as prespecified: the per-patient number of re-interventions to treatment success at 6 weeks, by arm, with the planned between-group test, or say why it cannot be computed. Treatment success at 6 weeks, the registered secondary, could be reported too. If a protocol amendment or statistical analysis plan set up the 6-month total count, making that document and its date available would let readers check when the end point was fixed relative to unblinded data. The registry record could be updated to give the outcome that was analysed, with a note on when it changed.
Trial report names an unregistered primary outcome and omits registered primary HIV risk behavior
JAMA Intern Med (2014) · doi:10.1001/jamainternmed.2014.2556 · PMC4811188 · NCT00987961
Summary
ClinicalTrials.gov record NCT00987961 has listed the same two primary outcomes, "opioid use" and "HIV risk behavior", in every version from the first (2009-09-30) to the last (2015-11-10). The published primary report calls two outcomes its "prespecified primary outcome variables": entry into buprenorphine treatment and length of illicit opioid use. Illicit opioid use matches the registered "opioid use" outcome. Entry into treatment, which gives the abstract its headline result (72.2% vs 11.9%), is not a registered primary or secondary outcome, and the registered primary outcome HIV risk behavior is not reported or mentioned anywhere in the paper. The paper cites the registration only by its identifier and does not mention these differences.
What was registered
The registry record has 11 versions (v0 to v10). The ClinicalTrials.gov history summary reports primaryOutcomesSame: true, secondaryOutcomesSame: true, outcomesUpdateCount: 0 and lastUpdateVersions.primaryOutcomes: 0. The outcomes were therefore set in v0 and never edited. Versions v1 to v10 changed only the Study Status, Sponsor/Collaborators, Contacts/Locations or Study Design modules.
| Version | Submitted | Primary outcomes (registry wording) | Time frame |
|---|---|---|---|
| v0 (first version) | 2009-09-30 (first posted 2009-10-01) | "opioid use"; "HIV risk behavior" | "baseline, 1-month, 3-months, 6-months" (both) |
| v1-v6 | 2010-03-23 to 2013-07-08 | unchanged | unchanged |
| v7 (last before the print issue) | 2014-07-16 | unchanged | unchanged |
| v8-v10 (after publication) | 2015-05-21 to 2015-11-10 | unchanged | unchanged |
The registered secondary outcomes, also unchanged since v0, are "reduction in injection-related medical conditions" and "reduction in emergency department and hospital utilization", with the same time frame.
The descriptive fields say the same thing. The Detailed Description ends: "Primary outcomes are opioid use and HIV risk behavior." The Brief Summary says the study "tests whether starting Suboxone (buprenorphine) during a medical hospitalization, and then providing an appointment (a "link") for after discharge to maintenance buprenorphine in an outpatient setting will reduce HIV risk behavior in individuals who inject opioids." "HIV risk behavior" is also one of the record's three keywords, in v0 and in the current version.
Timing: the registry gives the study start as 2009-09. The paper says patients were identified "From August 1, 2009", so the first registration came about two months after recruitment began. No registry version exists from before enrollment started, but the outcomes in v0 were entered during recruitment, years before the data were complete. Primary completion is given as 2014-03 (actual); v0 estimated 2013-12.
What the paper reports
The paper's primary-outcome definition (Methods, "Primary Outcome Variables"):
The prespecified primary outcome variables were entry into buprenorphine treatment (in-person intake [anytime between study enrollment and 6 months after enrollment]) at the hospital-associated OAT program, confirmed by OAT electronic medical record review, and length of illicit opioid use (number of days of reported opioid use in the 30 days before the 1-, 3-, and 6-month interviews using a standard 30-day time-line follow-back method).
The Introduction (final paragraph) sets out the same two goals, treatment entry first:
This study had 2 primary goals. We sought to determine whether offering hospitalized, opioid-dependent patients initiation and linkage to office-based opioid addiction treatment with buprenorphine would facilitate entry into and increase persistence in buprenorphine treatment. We then examined whether this treatment initiation and linkage would decrease illicit opioid use at 6 months.
The Abstract, Main Outcomes and Measures:
Entry and sustained engagement with buprenorphine OAT at 1, 3, and 6 months (medical record verified) and prior 30-day use of illicit opioids (self-report).
Secondary outcomes (Methods, "Secondary Outcome Variables"): "Secondary outcome variables included time to entry into the buprenorphine program (days to in-person intake at the hospital-associated OAT program, confirmed by electronic medical record review) and OAT days".
The sample-size statement (Methods, Statistical Analysis) concerns opioid use only: "Specifically, the study was powered (1 – β > .8) to detect a 33% between-group difference in the rate of opioid use during follow-up (days of use per 30 follow-up days)." The Discussion's limitations paragraph refers to a single primary outcome: "our primary outcome measure (past 30-day illicit opioid use) also depended on self-report".
On the registration, the paper's only statement is the back-matter line "TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT00987961". Searches of the archived text found no occurrence of "HIV" (the only related mention is "human immunodeficiency virus disease", once as background in the Introduction and once as a discharge diagnosis in Results), no occurrence of "risk behavior", "registered" or "amend", and no use of "protocol" except for the treatment protocol. The paper cites no trial protocol, statistical analysis plan or supplement (Europe PMC: no supplementary files).
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Opioid use / illicit opioid use | Primary: "opioid use", "baseline, 1-month, 3-months, 6-months" (v0, 2009-09-30; unchanged to v10) | Primary: days of illicit opioid use in the 30 days before the 1-, 3- and 6-month interviews; IRR 0.60 (95% CI 0.46-0.73) | matched |
| HIV risk behavior | Primary: "HIV risk behavior", "baseline, 1-month, 3-months, 6-months" (v0, 2009-09-30; unchanged to v10) | Not named, defined or reported | not reported |
| Entry into buprenorphine treatment | Not registered as primary or secondary in any version | "Prespecified" primary: in-person intake at the hospital-associated OAT program by 6 months, record-verified; 52/72 (72.2%) vs 8/67 (11.9%) | added |
| Injection-related medical conditions | Secondary (v0, 2009-09-30; unchanged) | Not reported as an outcome | not reported |
| Emergency department and hospital utilization | Secondary (v0, 2009-09-30; unchanged) | Not reported as an outcome; the Discussion cites a prior study linking injection drug use to such use | not reported |
| Time to treatment entry; OAT days | Not registered | Secondary outcomes | added |
Figures
Figure 1. Timeline of the ClinicalTrials.gov record and the trial. The filled blue marker is v0 (2009-09-30), the only version that defined the outcomes. The hollow markers are v1 to v10, none of which edited the primary or secondary outcomes, before or after the registry's actual primary completion date (2014-03). The orange lane shows dates from the paper: recruitment from 2009-08-01 to 2012-10-31 (so v0 came about two months after recruitment began), presentation of the results at two meetings in April and June 2013, and the print issue of 2014-08-01. The Crossref DOI record was created on 2014-07-01; the online-first date could not be confirmed from the publisher. Study start and primary completion are month-level registry dates, drawn mid-month.
Figure 2. Registered primary outcomes (left; v0, 2009-09-30, unchanged in all later versions) against the outcomes the paper calls its prespecified primary outcomes (right; Methods, "Primary Outcome Variables"). Registered "opioid use" matches the paper's days of illicit opioid use, with the same 1-, 3- and 6-month time points; the paper reports IRR 0.60 (95% CI 0.46-0.73) for it. Registered "HIV risk behavior" has no counterpart in the paper. Entry into buprenorphine treatment appears as a primary outcome in the paper but in no version of the registration, and it carries the abstract's headline result. The note at the foot covers secondary outcomes: neither registered secondary is reported, and the paper's two secondaries were not registered.
Why it matters
The abstract's first and largest result is for an outcome the registration does not list: "linkage participants were more likely to enter buprenorphine OAT than those in the detoxification group (52 [72.2%] vs 8 [11.9%], P < .001)". The abstract conclusion that linkage "is an effective means for engaging medically hospitalized patients who are not seeking addiction treatment" rests on this outcome. Calling it "prespecified" leads readers to treat it as a confirmatory primary result, but the public record does not show that it was prespecified as an outcome at all. Entry into the linked program is also an outcome that follows closely from the intervention itself (the linkage arm was given an intake appointment and a bridging prescription), which makes the choice of primary outcome especially relevant to how the result is read.
For the registered primary outcomes, the paper reports one and not the other:
- Opioid use (registered primary): reported, favourable to linkage. "On the basis of complete case analysis, the estimated rate of illicit opioid use in those in the linkage group was approximately 40% lower (IRR, 0.60; 95% CI, 0.46-0.73; P < .01) than for participants randomized to the detoxification group." Worst-case substitution (IRR 0.73, 95% CI 0.65-0.83) and last observation carried forward (IRR 0.72, 95% CI 0.64-0.81) gave similar results. The second abstract conclusion ("reduces illicit opioid use 6 months after hospitalization") therefore rests on a registered primary outcome. Two caveats come from the paper itself. The abstract describes the IRR as applying to "the 30 days before the 6-month interview", while Results present it as a rate over the whole follow-up. And the outcome is self-reported, with 59.0% follow-up at 6 months that differed by arm ("The linkage group had a significantly higher rate of follow-up at 6 months (χ2 = 6.75, P = .009).").
- HIV risk behavior (registered primary): not reported. The paper gives no result, does not mention it as an outcome, and does not point to another report. Its direction and size cannot be judged from the paper. The registry's Brief Summary presents HIV risk behavior as the outcome the study tests, so its absence changes what the trial can be said to have shown.
The registered secondary outcomes (injection-related medical conditions; emergency department and hospital use) are also not reported. The Discussion suggests the reduction in opioid use "is likely to translate into improved health outcomes", and the Conclusions say future work "should evaluate whether decreased illicit opioid use and increased use of OAT in the 6 months after hospital discharge could have health benefits and prevent subsequent hospitalizations". Emergency department and hospital use was, however, a registered secondary outcome of this trial, with a time frame of baseline to 6 months.
The paper does acknowledge multiplicity in general terms ("Given multiple outcomes, readers concerned about overall type I error rates may consider P < .01 a conservative standard to evaluate statistical significance."), but a reader cannot tell from it how many outcomes were planned or which ones were dropped.
Explanations considered
- Paper role. This is the primary report of the randomized trial: it reports the full randomized sample (139 randomized; 67 detoxification, 72 linkage), and the registry links it as the trial's results publication. The only other publication the registry links (PMID 24045030, Addict Behav 2013) is, by its title, an analysis of days of heroin use and self-reported health among hospitalized heroin users, published before the registry's primary completion date; it is not a report of HIV risk behavior outcomes. The paper does not say HIV risk behavior is being reported elsewhere. This does not account for the discrepancy.
- Disclosure. The paper does not mention the registered outcomes, HIV risk behavior as a planned outcome, an amendment, or any difference from the registration. It calls its primary outcomes "prespecified" without saying where they were prespecified. No protocol or analysis plan is cited or supplied. Nothing in the paper accounts for the discrepancy.
- Protocol amendments. The registry history shows no outcome edit in any of the 11 versions (outcomesUpdateCount 0; primary outcomes last updated in v0), including the versions submitted after primary completion and after publication. An amendment that was made in a protocol but never entered in the registry or the paper cannot be excluded, but it is not documented anywhere available to readers. It may well have legitimate reasons; the issue documented here is that it is not disclosed.
- Wording differences. "Opioid use" and "length of illicit opioid use" are the same construct at the same time points, so that pair is treated as matched. Entry into buprenorphine treatment cannot be read as a rewording of "HIV risk behavior", or of either registered secondary outcome. The paper's "persistence in"/"sustained engagement" in treatment likewise has no registered counterpart.
- Time-point differences. None for opioid use: both use 1, 3 and 6 months (the registry adds baseline). Time points do not explain the other differences.
- Registration timing. v0 was submitted about two months after recruitment began. That makes v0 a registration made during recruitment rather than before it, but it is still the only public statement of the trial's outcomes and long predates the results. It does not explain why the paper's primary outcomes differ from it.
Status
As of 2026-10-23:
- Registry history. The ClinicalTrials.gov version list and history summary (11 versions, v0 to v10, 2009-09-30 to 2015-11-10; primaryOutcomesSame true, outcomesUpdateCount 0) were retrieved from the ClinicalTrials.gov history API on 2026-10-22, together with the full v0 record. Versions v6 (2013-07-08, the last before the likely online-first date) and v7 (2014-07-16, the last before the print issue) were fetched individually in an earlier check of this record and list the same two primary outcomes. On 2026-10-23 the history endpoints returned HTTP 403. The current record (v2 API, fetched 2026-10-23, last updated 2015-11-10) still lists "opioid use" and "HIV risk behavior" as the primary outcomes. An independent re-check, which could not reach the history endpoints, found an Internet Archive snapshot of the record dated 2011-10-19 with the same primary and secondary outcomes.
- Publisher version. The journal page (jamanetwork.com, by DOI and full-article URL) returned 403 or a CAPTCHA, so the quoted text could not be checked against the publisher's version of record, and no publisher correction notice could be looked for there. The quotes above are from the PubMed Central copy (released 2016-03-29), which appears to be the NIH-deposited author manuscript.
- Corrections. No corrections, retractions, expressions of concern or update notices were found in PubMed Central, Crossref (no update-to or relation entries) or Europe PMC. Europe PMC links one item, PMID 24978013 (JAMA Intern Med 2014;174(8):1377-8), as "Comment in"; that is a commentary, not a correction.
- PubPeer. The search returned a human-verification block, so no PubPeer result could be obtained.
- COMPare. Not applicable: it covered trials published in 2015-16 in five journals, not including JAMA Internal Medicine.
Sources: ClinicalTrials.gov NCT00987961 (current record, version history and v0, v6, v7 records); PMC4811188 as archived in PubMed Central; Crossref and Europe PMC metadata for doi:10.1001/jamainternmed.2014.2556.
Suggested correction
A correction or clarification notice could:
- State the primary and secondary outcomes as registered at ClinicalTrials.gov (NCT00987961, first submitted 2009-09-30 and unchanged since): opioid use and HIV risk behavior at baseline, 1, 3 and 6 months; secondary outcomes injection-related medical conditions and emergency department and hospital use.
- Say when and why entry into buprenorphine treatment became a primary outcome and HIV risk behavior was dropped, and point to the document (protocol, grant application or analysis plan) in which the outcomes described as "prespecified" were set out, ideally making that document available.
- Report the results for HIV risk behavior and for the registered secondary outcomes, or say where they are reported or why they cannot be.
- Label entry into buprenorphine treatment in the abstract and Methods as an outcome not listed in the trial registration, and make the abstract's description of the illicit opioid use IRR match the Results.
The registry record could also be updated with a note describing the outcomes as analysed, so that readers of either source can see the relationship between them.
Registered primary outcome is overall survival; the final report uses melanoma-specific survival without comment
N Engl J Med (2014) · doi:10.1056/NEJMoa1310460 · PMC4058881 · NCT00275496
Summary
Every version of the ClinicalTrials.gov record for MSLT-I (NCT00275496), from the first on 2006-01-10 to the current one dated 2015-09-01, gives one primary outcome: whether sentinel-node biopsy with selective lymphadenectomy "will effectively prolong overall survival", with a 10-year time frame added in 2009. The final trial report states instead that "The primary end point was melanoma-specific survival (survival until death from melanoma)." The paper does not mention the registered outcome or explain the difference, and it reports no overall-survival estimate, hazard ratio or curve. The only all-cause mortality information in the paper is a set of crude death counts in the Fig. 1B flow diagram. The trial was registered about 12 years after it started, so the registry entry may not reflect the original protocol. Even so, the registered primary outcome had been public for eight years before publication and the paper does not reconcile the two.
What was registered
ClinicalTrials.gov has 10 versions of the record. All of them postdate the study start (1993-11) and the end of enrollment (2002). The primary outcome measure text is the same, character for character, in all 10:
To determine whether wide excision of the primary with intraoperative lymphatic mapping (LM) followed by selective lymphadenectomy will effectively prolong overall survival compared to wide excision of the primary melanoma alone.
| Version | Date | Primary outcome | What changed in Outcome Measures |
|---|---|---|---|
| v0 | 2006-01-10 (first submitted; first posted 2006-01-12) | text above, no time frame | first registration |
| v1 | 2006-01-20 | unchanged | secondary only: "Sentinel node metastases in WEX arm; Incidence of nodal metastases" became "Incidence of Sentinel Node Metastases (biopys) vs clinical metastases (observation)" |
| v2, v3 | 2006-11-29, 2007-11-02 | unchanged | none (Study Status, References) |
| v4 | 2009-04-30 | unchanged; time frame "10 years" added | "10 years" added to primary and secondary; "biopys" corrected to "biopsy" |
| v5, v6 | 2011-11-07, 2012-07-30 | unchanged | none (v6 records primary completion 2012-06, actual) |
| v7 | 2012-11-19 (last version before publication) | unchanged, "10 years" | none (Study Status, References) |
| v8, v9 | 2014-05-13, 2015-09-01 | unchanged, "10 years" | none (Study Status only) |
The registered secondary outcomes in v4 to v9 are: "Disease-free survival; Incidence, timing, and anatomic distribution of distant metastases; Morbidity of procedures; Significance of TA90 levels; Incidence of Sentinel Node Metastases (biopsy) vs clinical metastases (observation); Accuracy of LM" (time frame "10 years"). Melanoma-specific survival does not appear as a primary or secondary outcome in any version.
Other registry dates: study start 1993-11; primary completion 2012-06 (actual); last update posted 2015-09-02. The current record lists this paper (PMID 24521106) as a "derived" citation and three "result" citations: the 2006 third interim report (N Engl J Med 2006 Sep 28, PMID 17005948); a 2010 Ann Surg Oncol paper on morbidity and length of stay after early versus delayed completion lymphadenectomy (PMID 20614193); and a 2012 Ann Surg Oncol paper on metastasectomy for distant metastases in MSLT-I patients (PMID 22648554). The two Ann Surg Oncol citations first appear in v7. The registry records masking as None (open label) in every version.
What the paper reports
The primary end point, in Methods, Statistical Analysis (first paragraph):
The primary end point was melanoma-specific survival (survival until death from melanoma). The secondary end points, described previously,11,12 included disease-free survival, survival with tumor-positive or tumor-negative sentinel nodes, and the incidence of sentinel-node metastases, as compared with the incidence of clinically detected nodal metastases.
References 11 and 12 are the 2006 third interim report in the same journal and a 1999 Ann Surg report on the accuracy of lymphatic mapping (PMC1420894). Neither is a protocol or a statistical analysis plan.
The registration appears once, at the end of the abstract:
(Funded by the National Cancer Institute, National Institutes of Health, and the Australia and New Zealand Melanoma Trials Group; ClinicalTrials.gov number, NCT00275496.)
The main results for the paper's own primary end point (abstract; Results, Survival Rates):
No significant treatment-related difference in the 10-year melanoma-specific survival rate was seen in the overall study population
hazard ratio for death from melanoma in the biopsy group, 0.84; 95% confidence interval [CI], 0.64 to 1.09; P = 0.18
This is for patients with intermediate-thickness melanomas (1.20 to 3.50 mm), the primary study group.
The analysis methods name only melanoma-specific and disease-free survival (Methods, Statistical Analysis, third and fourth paragraphs):
The Kaplan–Meier method was used to estimate mean 5-year and 10-year rates of melanoma-specific and disease-free survival.
Comparisons of melanoma-specific and disease-free survival were based on data from patients who underwent their assigned treatment.
The Discussion (fourth paragraph) refers to survival without saying which kind:
We did not see a significant survival advantage for all patients with intermediate-thickness melanomas.
The design changes the paper does disclose are a larger sample size ("After the second interim analysis, the sample size for the intermediate-thickness group was increased to 1200 patients") and the latent-subgroup method ("In post hoc analyses, a latent-subgroup analytic method was developed"). Both are in Methods, Statistical Analysis.
What the paper does not contain: searches of the full text as archived in PMC (abstract, body, table, figure legends) for "overall survival", "all-cause", "any cause" and "amend" find nothing. Every use of "protocol" refers to the per-protocol analysis. The one use of "registered" refers to enrollment ("Enrollment closed in 2002, after 2001 patients had been registered."). The Supplementary Appendix archived with the PMC manuscript (16 pages) has no overall-survival analysis and no statement about end points or protocol amendments. Its survival analyses are all melanoma-specific, disease-free or distant disease-free.
All quotations above were checked verbatim with src/show.py PMC4058881 --grep, and each full quotation occurs exactly once in the archived text. The Fig. 3 legend repeats the definition "survival until death from melanoma". Registry quotations were checked against the archived version records and the current API v2 record.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Overall survival (death from any cause), 10 years | Primary in every version, v0 2006-01-10 to v9 2015-09-01 (time frame "10 years" from v4 2009-04-30; v7 2012-11-19 is the last version before publication) | Not named as an end point. No estimate, hazard ratio or curve. Fig. 1B gives only crude counts of deaths by cause | not reported |
| Melanoma-specific survival, 10 years | In no version, as either primary or secondary | "The primary end point was melanoma-specific survival" | added (as primary) |
| Time frame of the primary outcome | 10 years (from v4 2009-04-30) | 10-year rates | matched |
| Disease-free survival, 10 years | Secondary, all versions (time frame "10 years" from v4 2009-04-30) | Secondary. The abstract's conclusions cite it first as a treatment benefit | matched |
| Incidence of sentinel-node metastases vs clinical metastases | Secondary, v1 onward | Secondary ("the incidence of sentinel-node metastases, as compared with the incidence of clinically detected nodal metastases") | matched |
| Distant metastases (incidence, timing, distribution) | Secondary, all versions | Reported as distant disease-free survival in the node-positive subgroup (Results; Fig. S2) | matched (as distant disease-free survival) |
| Survival with tumor-positive or tumor-negative sentinel nodes | In no version | Secondary | added (as secondary) |
The core finding is two rows: the registered primary outcome (overall survival) is not reported, and a different measure (melanoma-specific survival) is reported as the primary end point. Neither the substitution nor the reason for it is stated in the paper.
Figures
Figure 1. Timeline of the trial and its registry record. The trial started in 1993-11 (registry; the paper says it "commenced in 1994") and enrolled 2001 patients until 2002. It was first registered on 2006-01-10, about 12 years after the start. Every registry version therefore postdates the study start and the end of enrollment. Diamonds mark the versions that created or edited the Outcome Measures module: v0 (primary outcome overall survival), v1 (secondary wording only) and v4 (time frame "10 years" added to the unchanged primary). Open circles mark versions that changed other modules. The hatched band, bounded by the dashed line, is the period after primary completion (2012-06). The four versions in it did not touch outcomes: v6 (2012-07-30, Study Status; status set to Completed and primary completion 2012-06 recorded as actual), v7 (2012-11-19, Study Status and References; the last version before publication), and v8 (2014-05-13) and v9 (2015-09-01), Study Status only. The final report (2014-02-13) names melanoma-specific survival as its primary end point. Sources: ClinicalTrials.gov record history; the paper as archived in PMC; the registry's reference to the 2006 interim report.
Figure 2. Registered primary outcome versus the primary end point in the paper. Left, in blue: the registry's primary outcome as of v7 (2012-11-19), the last version before publication, worded the same since v0 (2006-01-10). Right, in orange: the paper's stated primary end point. The two use the same 10-year time frame but measure different things: overall survival counts deaths from any cause, while melanoma-specific survival counts only deaths from melanoma. The paper gives no overall-survival result. The nearest data are the crude counts of deaths by cause in the Fig. 1B flow diagram, which are not time-to-event estimates and do not account for losses to follow-up. Melanoma-specific survival appears in no registry version. Its result in the primary study group is null (hazard ratio 0.84, 95% CI 0.64 to 1.09, P = 0.18). Disease-free survival, a registered secondary outcome, is reported as a secondary end point.
Why it matters
Overall survival and melanoma-specific survival differ in more than wording. Melanoma-specific survival treats deaths from other causes as censored, so a treatment effect on non-melanoma mortality cannot appear in it. It also depends on cause-of-death attribution, which may not be blinded in an open surgical trial (the registry records masking: None in every version). For a comparison of an extra surgical procedure (sentinel-node biopsy, with immediate completion lymphadenectomy when positive) against observation, all-cause mortality is the outcome the registry named, and it captures any harm or benefit regardless of cause.
What the paper reports for the registered primary outcome: no estimate. No overall-survival rate, hazard ratio, P value or Kaplan–Meier curve appears in the abstract, main text, Table 1, figure legends or the archived Supplementary Appendix. The only related information is in the Fig. 1B flow diagram (image), which lists deaths by cause among patients included in follow-up. For intermediate-thickness melanomas: biopsy group 133 died from melanoma and 47 from other causes, 180 of 805 in all (22.4%); observation group 103 and 25, 128 of 522 (24.5%). For thick melanomas: biopsy 68 and 19, 87 of 185 (47.0%); observation 43 and 9, 52 of 126 (41.3%). These are crude proportions with unequal losses to follow-up (120 and 68 in the intermediate groups; 18 and 9 in the thick groups). They are not a substitute for the registered time-to-event analysis, and this write-up draws no inference about treatment effect from them. They show that the paper did record deaths from other causes, so an overall-survival analysis was possible from the trial data.
The abstract's positive conclusions rest on outcomes other than the registered primary: disease-free survival (a registered secondary), melanoma-specific and distant disease-free survival in the node-positive subgroup (a comparison the paper itself says raises "concern about ascertainment bias"), and a post hoc latent-subgroup analysis. The paper's own primary end point is stated openly as null in the whole primary study group. A reader comparing the paper with the registry cannot tell from the paper whether the registered overall-survival comparison was done or what it showed.
Explanations considered
- Paper role. This is the final report of the main trial: the title says "Final Trial Report", and the paper reports 10-year results for the primary study group and names its own primary end point. It is the report in which the registered 10-year primary outcome would be expected. It is not a secondary analysis or a subgroup paper. Does not account for it.
- Disclosure in the paper. The registry number appears only in the abstract's funding line. The paper does not mention overall survival, the registered outcome, or any difference from the registry. The secondary end points are said to be "described previously" (refs 11 and 12), but the paper gives no comparable reference or explanation for its primary end point. Not disclosed.
- Protocol amendment. The paper discloses two design changes, the sample-size increase after the second interim analysis and the post hoc latent-subgroup method, but no change to the primary end point. No protocol or statistical analysis plan is cited or provided. The archived Supplementary Appendix contains none. The registry's primary outcome never changed across 10 versions, so there is no registry-side amendment either. No amendment documented in either source.
- Wording differences. "Overall survival" and "melanoma-specific survival" are standard, distinct end points, and the paper defines its own narrowly as "survival until death from melanoma". The registry entry is phrased as an objective ("To determine whether…"), not a formal measure, but "overall survival" is explicit in it. Cannot be read as a rewording of the same outcome.
- Time point. Both are 10-year outcomes (registry time frame "10 years" from v4; paper reports 10-year rates). No time-point issue.
- Retrospective registration. The trial was registered in 2006, after enrollment had closed in 2002. The registry entry may therefore not match the original protocol, and there is no pre-start version to compare. The abstract of the 2006 third interim report already reported melanoma-specific and disease-free survival. I could not read that report's full text to see whether it names a primary end point. This could explain why the investigators used melanoma-specific survival. It does not explain why the final report leaves a public registration's primary outcome unmentioned and unreported. Mitigates the significance of the change, not the non-disclosure.
- Outcomes split across papers. The registry lists a single primary outcome. Results for thin melanomas are described as "exploratory" and "not reported on in this article", but that concerns a cohort, not the overall-survival end point. The registry links three MSLT-I "result" publications besides this paper. One is the 2006 interim report; neither of the other two reports 10-year overall survival by randomized group: the 2010 Ann Surg Oncol paper (PMID 20614193) compares morbidity and length of stay after early versus delayed lymphadenectomy, and the 2012 Ann Surg Oncol paper (PMID 22648554) compares survival after a stage IV diagnosis by treatment received (surgery versus systemic therapy alone). Both full texts were read in PMC (PMC2970739, PMC3405182). I did not find a separate MSLT-I publication reporting 10-year overall survival as the primary outcome. Does not account for it on present evidence.
Status
As of 2026-10-23:
- Registry history: all 10 versions read through the ClinicalTrials.gov internal history API (
/api/int/studies/NCT00275496?history=trueand/history/0to/history/9; the version list and v0 cached 2026-10-22, v1 to v9 cached 2026-10-23; these endpoints now return HTTP 403 from this machine, so the cached copies were used) and the current record through API v2 (re-fetched 2026-10-23). The primary outcome is unchanged in every version, and the current record still lists overall survival. - Publisher version: the text checked is the NIH author manuscript archived in PMC (PMC4058881), including the Fig. 1 flow-diagram image and the Supplementary Appendix PDF archived with it. The publisher's page at nejm.org returned HTTP 403, so the version of record there was not compared directly.
- Corrections:
src/known.pyfound no Crossref update-to records, no PMC notices and no Europe PMC correction or retraction notices for 10.1056/NEJMoa1310460. The reference list's erratum note belongs to the 2006 interim report (ref 11), not this paper. Europe PMC links published comments (an editorial in the same journal, N Engl J Med 2014;370:663-4, correspondence, and comments in other journals) but no erratum. - Correspondence: the May 2014 letters and authors' reply (N Engl J Med 2014;370(22):2148-50, doi:10.1056/NEJMc1402989). The authors' reply is archived in PMC (PMC4522311) and was read in full. It answers points about recurrence prevention, the nodal-metastasis curves, subgroup offsetting and earlier trials, and gives a whole-population distant disease-free survival result that is not in the paper (hazard ratio 0.89, P = 0.34). It does not mention overall survival, the registration or the choice of primary end point. The letters themselves (PMIDs 24869727 and 24869728) are not in PMC and nejm.org returned HTTP 403, so they were not read; what the reply answers suggests their topics, but whether any letter raised overall survival is unknown.
- PubPeer: the PubPeer API returned HTTP 403 (Cloudflare challenge), so PubPeer was not checked.
- Sources: ClinicalTrials.gov NCT00275496 (current record and version history); the article as archived in PubMed Central; Europe PMC records for this paper and for the 2006 and 2005 MSLT-I reports; the authors' reply to the May 2014 correspondence (PMC4522311); the two registry-linked Ann Surg Oncol papers (PMC2970739, PMC3405182); Crossref.
Suggested correction
The authors and journal could add a correction or a note to the article that:
- States that the ClinicalTrials.gov record (NCT00275496, first registered 2006-01-10) lists overall survival at 10 years as the primary outcome, and that the paper's primary end point is melanoma-specific survival.
- Says when and why the primary end point was set as melanoma-specific survival. If the original pre-2006 protocol named melanoma-specific survival, the correction could say so and make that protocol available.
- Reports the registered primary outcome: 10-year overall survival by randomized group in the intermediate-thickness (primary study) group, with the Kaplan–Meier estimates, hazard ratio, 95% CI and P value, analyzed the same way as the melanoma-specific survival results (and by intention to treat).
- If the registry entry does not reflect the protocol, the sponsor could update ClinicalTrials.gov with a dated note explaining the difference, so the record and the publication agree.
HTC uptake, the trial's first primary outcome, appears in no version of its registration
PLoS Med (2014) · doi:10.1371/journal.pmed.1001768 · PMC4267810 · NCT01459120
Summary
This cluster-randomized trial compared home-based HIV testing and counseling (HB-HTC) with mobile clinic HTC (MC-HTC) in rural Lesotho. The paper names uptake of HTC as its first primary outcome, states its main hypothesis in terms of uptake, bases its sample size on uptake, and leads the abstract results with uptake (92.5% vs 86.7%, aOR 2.06, p = 0.011). None of the four versions of ClinicalTrials.gov record NCT01459120 lists HTC uptake as a primary or a secondary outcome, and the registered outcomes never changed between versions. The paper has a section on deviations from the registration, but it discloses only that a registered cluster-level outcome was dropped, and it describes uptake as one of the endpoints that "were kept", which the record does not bear out.
What was registered
ClinicalTrials.gov holds four versions of NCT01459120. The record history lists them as v0 (submitted 2011-10-18, QC 2011-10-21, first posted 2011-10-25), v1 (2011-10-31; modules changed: Study Status, Contacts/Locations; the start date went from 2011-11 to 2011-10), v2 (2011-12-09; Study Status, Contacts/Locations) and v3 (2012-02-08, posted 2012-02-09; Study Status only; status COMPLETED; primary completion 2012-02, actual). The history summary reports primaryOutcomesSame: true, secondaryOutcomesSame: true and an outcomes update count of 0. I compared the outcomes module of v0, v1 and the current record (v3) field by field. They differ only in HTML character encoding (for example HIV/AIDS for HIV/AIDS) and line breaks. Enrollment is 1800 (ESTIMATED) in every version.
The five registered primary outcomes, identical in v0 (2011-10-18) through v3 (2012-02-08), are quoted here from the current record:
- "Proportion of newly tested HIV-positive clients over all clients tested for HIV during the campaigns". No time frame is given.
- "Proportion of clients newly tested HIV-positive who enrolled into HIV/AIDS care within one month over all clients newly tested HIV-positive during the campaigns". Time frame: "4 weeks after tested positive".
- "Overall number newly tested HIV-positive and enrolled into chronic HIV/AIDS-care at each facility". Time frame: "4 weeks after campaign". The description says this compares the facility-level total "Irrespective if these patients were tested during one of the campaigns or during routine activities at the facility" and the total "from the 5 villages where campaigns were held as compared to the five villages where no campaigns were held."
- "Absolute number of newly tested HIV-positive clients". No time frame is given.
- "Absolute number of clients newly tested HIV-positive who enrolled into HIV/AIDS care". Time frame: "4 weeks after tested HIV-positive".
The ten registered secondary outcomes cover CD4 count, WHO stage, four tuberculosis screening and sputum outcomes with their absolute-number counterparts, "Proportion of first-time HIV-testers among all clients accessing the testing services" and "Demographic characteristics of clients accessing the voluntary counseling and testing services". No primary or secondary outcome measures acceptance of testing among the people offered it. In outcome 1, "all clients tested" serves only as the denominator for HIV positivity.
What the paper reports
Methods, "Study Hypothesis, Design, and Setting":
The study's main hypothesis was that the HB-HTC approach would result in a higher uptake of HTC than the MC-HTC approach among individuals accessing the multi-disease campaigns.
The same section dates the trial "from 17 October to 25 November 2011".
Methods, "Outcomes at the Individual Level":
The trial had three primary and five secondary outcomes. The first primary outcome was the number of persons taking up HTC.
Uptake was defined as acceptance and completion of HTC with pre-test counseling, HIV test, and post-test counseling.
The second primary outcome was the proportion of participants newly tested HIV-positive among participants who took up HTC.
The third primary outcome was linkage to care within 28 d after a positive HIV test among participants who tested HIV-positive during the campaign.
Methods, "Sample Size and Statistical Analysis":
we estimated that HTC uptake would be about 65% in the MC-HTC arm
Abstract, "Methods and Findings":
The trial was powered to test the hypothesis of higher HTC uptake in HB-HTC campaigns than in MC-HTC campaigns.
Overall HTC uptake was higher in the HB-HTC group than in the MC-HTC group (92.5% versus 86.7%; adjusted odds ratio [aOR]: 2.06; 95% CI: 1.18–3.60; p = 0. 011)
(The space in "0. 011" is in the archived text.)
Methods, "Deviations from Initial Trial Registration", which is the paper's only statement about the registration:
Aggregate data from clusters on numbers of persons newly tested HIV-positive through routine facility-based HTC during the 28 d following the study campaigns were initially registered as one of the primary outcomes. While finalizing the internal study protocol, cluster data were dropped as a primary outcome because the trial was designed and powered to measure differences in outcomes at the individual level and not at the cluster level. Only individual endpoints were kept as primary outcomes—uptake of HTC, participants newly detected HIV-positive, and subsequent enrollment in HIV care—as these endpoints represent the three major steps of the HTC cascade. Nevertheless, description of cluster data was maintained and is reported in this paper.
Table 2 ("Primary outcomes") reports, for HB-HTC vs MC-HTC: uptake of HTC overall 92.5% (1,083/1,171) vs 86.7% (1,207/1,392), aOR 2.06 (1.18–3.60), p = 0.011; participants newly detected HIV-positive 3.6% (39/1,083) vs 6.2% (75/1,207), aOR 0.64 (0.48–0.86), p = 0.002; linkage to care 25.6% (10/39) vs 25.3% (19/75), aOR 0.99 (0.35–2.79), p = 0.978.
The supporting information includes Text S2, "Trial protocol", an undated internal protocol for a study "to be held from October to December 2011". Its outcome table lists "Proportion and absolute number of clients accepting HTC" first among the primary outcomes, and its sample-size section reads "Expected difference in HTC-uptake between study-arms (1st primary outcome): 80% (door-to-door) versus 65% (pitso)". The protocol does not mention the registry record.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| Uptake of HTC (acceptance and completion of testing among eligible attendees) | Not registered, as primary or secondary, in v0 (2011-10-18), v1, v2 or v3 (2012-02-08) | First primary outcome; main hypothesis; basis of the sample size; first result in the abstract (aOR 2.06, p = 0.011) | added (not disclosed) |
| Proportion newly HIV-positive among those tested | Primary 1, v0–v3 | Second primary outcome: 3.6% vs 6.2%, aOR 0.64, p = 0.002 | matched |
| Proportion newly HIV-positive enrolled in care within one month / 4 weeks | Primary 2, v0–v3 | Third primary outcome, "within 28 d": 25.6% vs 25.3%, aOR 0.99, p = 0.978 | matched (4 weeks = 28 d) |
| Facility-level number newly HIV-positive and enrolled in care, 4 weeks after campaign | Primary 3, v0–v3 | Dropped as primary; reported descriptively (49 newly HIV-positive at facilities in 28 d; 7 from control villages); no arm comparison; facility-level enrolment not reported | demoted (disclosed) |
| Absolute number newly HIV-positive | Primary 4, v0–v3 | Counts 39 vs 75 shown within the second primary outcome; not named as a primary outcome | demoted (reported as count, not disclosed) |
| Absolute number newly HIV-positive enrolled in care, 4 weeks | Primary 5, v0–v3 | Counts 10 vs 19 shown within the third primary outcome; not named as a primary outcome | demoted (reported as count, not disclosed) |
Figures
Figure 1. Timeline of the trial. Squares are ClinicalTrials.gov versions; circles are dates from the paper and its Text S2 protocol. Clusters were randomised at a meeting "held on October the 3rd" (Text S2; its campaign timetable runs from 17 October 2011), and campaigns ran from 17 October to 25 November 2011. The record was first submitted on 18 October 2011, one day after the campaigns began. All four versions therefore came after the study started, and v3 (8 February 2012, status COMPLETED) falls in the month of primary completion. No version changed any outcome; the versions changed only status, dates and locations. The record was not updated again before publication on 16 December 2014 (the axis is broken between March 2012 and December 2014).
Figure 2. Registered primary outcomes (left, blue; v0 of 18 October 2011, unchanged through v3 of 8 February 2012) and the primary outcomes reported in the paper (right, orange), with each registered item's fate labelled. The paper's first primary outcome, HTC uptake, has no registered counterpart in any version (ADDED). The two registered proportions match the paper's second and third primary outcomes (MATCHED), whose results are shown in the boxes, including the null linkage result. The registered absolute numbers appear only as counts (COUNT ONLY). The cluster-level outcome was dropped as primary and reported descriptively (DEMOTED), which is the only change the paper discloses.
Why it matters
Uptake is the outcome on which the paper's design and headline rest. The main hypothesis, the power calculation and the first result in the abstract all concern uptake, and the abstract's conclusion that "HB-HTC was better in reaching children, individuals who had never tested before, and men" draws on it. A reader who checks the registration to see whether the main comparison was pre-specified finds no uptake outcome at all. The deviations section says uptake was "kept" as a primary outcome, which reads as if uptake had been registered.
The uptake result is also sensitive to how it is analysed. The paper reports that "This overall difference was due to a much higher uptake in the age group <12 y in the HB-HTC group (Table 2)", where uptake was 87.5% vs 58.7% (aOR 4.91). It also reports that "Among participants ≥12 y of age, uptake tended to be lower in the HB-HTC group compared to the MC-HTC group (Table 2)": 94.7% vs 97.1% at 12–24 y (aOR 0.59, p = 0.322) and 93.9% vs 96.1% at ≥25 y (aOR 0.61, p = 0.117). Without a registered definition, a reader cannot tell from the registry which population and analysis were intended to carry the primary comparison.
For the registered primary outcomes the paper does report results, in full. The registered first primary (proportion newly HIV-positive among those tested) favoured the comparator: 3.6% (39/1,083) with HB-HTC vs 6.2% (75/1,207) with MC-HTC, aOR 0.64 (0.48–0.86), p = 0.002. The registered second primary (linkage to care within one month) was null: 25.6% (10/39) vs 25.3% (19/75), aOR 0.99 (0.35–2.79), p = 0.978. The abstract states both. The registered facility-level outcome was not compared between arms, which the paper discloses. So no registered primary outcome is hidden. What the registry does not support is the paper's framing of uptake as the pre-specified, powered primary outcome. On the two registered individual-level primary outcomes, the intervention arm was not better.
Explanations considered
- Paper role. This is the main report of the trial, not a secondary or exploratory analysis. Its outcomes should be compared with the registered primary outcomes.
- Disclosure. The "Deviations from Initial Trial Registration" section discloses one change: the cluster-level outcome was dropped. It does not say that uptake was absent from the registration, or that it was added, and its wording ("were kept") implies the opposite. The paper does not disclose that the registered absolute-number outcomes are not treated as separate primary outcomes, though the counts are reported. The disclosure therefore does not account for the main discrepancy.
- Protocol amendments and registry edits. The registry history rules out a change in the record: the outcomes module is the same in all four versions, so uptake was never registered and then removed. The paper attributes its outcome set to "finalizing the internal study protocol". Text S2, the internal protocol, does list uptake as the first primary outcome and powers on it, so uptake was pre-specified there. But Text S2 is undated (it speaks of a study "to be held from October to December 2011") and does not say whether it was written before or after the registration, and the registration was never brought into line with it. A pre-specified internal protocol is a legitimate reason for the outcome set the paper uses; the issue is that the paper does not tell readers that the public registration differs from it.
- Wording differences. No registered outcome measures uptake under other words. Registered primary 1 uses "all clients tested" only as a denominator for HIV positivity. The registered secondary outcome "Proportion of first-time HIV-testers among all clients accessing the testing services" is a different quantity, and the paper reports it separately as its second secondary outcome. "Demographic characteristics of clients accessing the voluntary counseling and testing services" is descriptive and also does not measure acceptance of testing.
- Time point. The registered "within one month" / "4 weeks" for linkage and the paper's "within 28 d" are equivalent, so the matched outcomes are not affected.
- Registration timing. The record was first submitted on 18 October 2011, the day after campaigns began on 17 October and about two weeks after randomisation (Text S2). This is a separate, minor timing point; it does not explain the absence of uptake, since no later version added it.
- Enrollment. The registry's estimated enrollment of 1800 was never updated; the paper analysed 2,563 eligible participants from 3,197 attendees. This is a minor, separate point and does not bear on the outcome discrepancy.
Status
As of 2026-10-23:
- Registry history. Four versions, v0 to v3 (2011-10-21 QC through 2012-02-08). The version list, v0 and v1 were retrieved from the ClinicalTrials.gov history endpoints on 2026-10-22, and higher version numbers returned no version. On 2026-10-23 those endpoints returned HTTP 403, so v2 and v3 were not re-fetched; v3 is the current record, retrieved through the v2 API on 2026-10-23. The outcomes in v0, v1 and the current record are identical apart from character encoding, and the history summary reports no outcome update in any version.
- Publisher version. The PLoS Medicine article page (published 16 December 2014; received 13 September 2013; accepted 3 November 2014) links no correction, erratum, expression of concern or retraction. Its supporting information includes Text S1 (CONSORT table), Text S2 (trial protocol) and Table S1 (costs).
- Corrections. No Crossref update, PMC notice or Europe PMC notice was found (
src/known.py). - PubPeer. The PubPeer browser-extension lookup for the DOI returned no comments. An earlier web search found no thread.
- Sources. PMC4267810 as archived in PubMed Central; Text S2 from the publisher's supporting information; ClinicalTrials.gov NCT01459120 current record and record history; Crossref, PMC and Europe PMC notice searches; PubPeer.
Suggested correction
A correction to the article could state that HTC uptake, reported as the first primary outcome and the basis of the sample-size calculation, was not listed as a primary or secondary outcome in any version of ClinicalTrials.gov record NCT01459120. It could say that uptake was specified as a primary outcome in the internal protocol (Text S2), and give that protocol's date relative to the registration (18 October 2011) and the start of the campaigns (17 October 2011) if it is known. The sentence in "Deviations from Initial Trial Registration" that says uptake was "kept" as a primary outcome could be revised to say that uptake was added relative to the registration. The same section could note that registered primary outcomes 4 and 5 (absolute numbers) are reported as counts rather than as separate primary outcomes. The sponsor could also add a note to the registry record, or post results there, that points readers to the outcome set actually analysed.
The paper calls store sales primary; the registered primary, women's diet quality, is reported as secondary
PLoS Med (2021) · doi:10.1371/journal.pmed.1003729 · PMC8423266 · NCT03518151
Summary
ClinicalTrials.gov NCT03518151 (WRAPPED1) has a single version, submitted on 2018-04-25 (658 days after the registered study start and 360 days after primary completion), which names one primary outcome, women's dietary quality score from a 20-item food frequency questionnaire (FFQ) at 3 months, and lists weekly store sales of fresh fruit and vegetables and of confectionery over 12 weeks as secondary outcomes. The published primary report reverses that order: it names "store sales (primary outcome)", calls women's and children's diets "secondary outcomes", and reports every outcome at 3 and 6 months. The paper cites the registration three times and never mentions that its outcome order differs from the registered one. The registered primary outcome is reported in Table 2 (+0.29 SD, P = 0.04, adjusted for the Index of Multiple Deprivation, IMD; +0.26 SD, P = 0.06, fully adjusted), but the abstract gives no numbers for it.
What was registered
The registry record history lists a single version. It was retrieved through the ClinicalTrials.gov history endpoint (/api/int/studies/NCT03518151/history/0) and checked against the current public API v2 record, which matches it.
| Registry version | Date | Primary outcome | Secondary outcomes |
|---|---|---|---|
| v0 (the only version) | submitted 2018-04-25; first posted 2018-05-08; last update posted 2018-05-08 | Dietary quality score, 3-month follow-up | Child's dietary quality score (3 mo); women's daily fruit and vegetable portions (3 mo); women's weekly confectionery intake (3 mo); weekly sales of fresh fruit and vegetables (12 wk); weekly sales of confectionery (12 wk) |
The registered primary outcome reads, verbatim:
Dietary quality score — "The primary outcome is women's dietary quality measured using a 20-item food frequency questionnaire (FFQ) to create a standardised dietary quality score (validated against 100-item FFQ and red cell folate)." Time frame: "3 month follow-up post intervention commencement"
The two registered store-sales outcomes read:
Weekly sales of fresh fruit and vegetables (secondary) — "Total weekly sales of fresh fruit and vegetables collected via electronic sales data". Time frame: "12 week period following intervention commencement"
Weekly sales of confectionery (secondary) — "Total weekly sales of confectionery collected via electronic sales data". Time frame: "12 week period following intervention commencement"
Registry dates (all marked "actual"): study start 2016-07-06; primary completion 2017-04-30; study completion 2017-11-30. Enrollment: 150 (actual). Because v0 is the only version, the registry primary outcome was never changed. The record was created after data collection for the primary outcome had ended. The paper agrees: it says the study "took place between April 2016 and March 2017" and that "Recruitment occurred in 3 waves between July and September 2016."
What the paper reports
The paper names store sales as its primary outcome in two places:
It aims to assess whether creating a healthier layout in discount supermarkets in England improves the healthiness of store sales (primary outcome) and the purchasing and dietary behaviours of women customers aged 18 to 45 years (secondary outcomes) after 3 and 6 months.
— Introduction, final paragraph
The data collected included 9 months continuous store sale transaction data (primary outcome) and participant loyalty card data (secondary outcome) provided by the collaborating supermarket to cover 3 time periods
In order to obtain an understanding of intervention effects on household members’ diets (secondary outcomes), interview-administered telephone questionnaires obtained information about participants’ diets and diets of their children aged 2 to 6 years (where applicable) at 3 time points: baseline (time 1) and 3 (time 2) and 6 months (time 3) following intervention commencement.
— Section 2.4, Outcome measures
The store-sales analysis covers four product groups: "Weekly sales per store of fresh fruits and vegetables, frozen vegetables, confectionery, and intervention checkout items" (Section 2.6). The abstract's quantitative results are store-sales and loyalty-card purchasing results. Its first concluding sentence is "Healthier supermarket layouts can improve the nutrition profile of store sales and likely improve household purchasing and dietary quality."
The paper mentions the registration in three places, each time without comment on its content:
The trial registration number is NCT03518151.
— Abstract, Methods and findings
ClinicalTrials.gov NCT03518151 (pre-results)
— Abstract, "Trial registration" line
it was registered with ClinicalTrials.gov (NCT03518151, pre-results).
— Section 2.1, Study design and setting
The paper describes itself as "a pilot study with a prospective matched controlled cluster design" (Section 2.1). The article text (excluding the reference list) contains none of the strings "retrospectiv", "amend", "changed", "protocol", "analysis plan", "prespecified" or "pre-specified". The word "deviat" occurs only in "standard deviation". The one statement about decisions made after data collection concerns analysis methods, not which outcome is primary:
The distributions of the data were unknown in advance of study commencement because this was a pilot study. The research team therefore made decisions about the statistical analyses following (i) completion of initial analyses to determine the shape of the outcome data; and (ii) consultation with the literature to identify appropriate statistical methods for the various outcome variables.
— Section 2.6, Statistical analysis
The discrepancy
| Outcome | Registered (v0, 2018-04-25) | In the paper | Status |
|---|---|---|---|
| Women's dietary quality score (20-item FFQ, standardised) | Primary, 3-month follow-up | Secondary ("household members’ diets (secondary outcomes)"); reported at 3 and 6 months (Results 3.4, Table 2) | demoted; time point changed (6 months added) |
| Weekly sales of fresh fruit and vegetables | Secondary, 12-week period after intervention start | Primary ("store sales (primary outcome)"); interrupted time series at intervention, 3 and 6 months (Results 3.2) | added as primary; time point changed |
| Weekly sales of confectionery | Secondary, 12-week period after intervention start | Primary (same); at intervention, 3 and 6 months | added as primary; time point changed |
| Frozen-vegetable sales; intervention checkout-item sales | Not registered | Part of the store-sales (primary) analysis (Sections 2.6, 3.2) | added |
| Child's dietary quality score | Secondary, 3 months | Secondary, 3 and 6 months (Table 2) | matched; time point changed (6 months added) |
| Women's daily fruit and vegetable portions | Secondary, 3 months | Secondary, 3 and 6 months (Table 2) | matched; time point changed (6 months added) |
| Women's weekly confectionery intake | Secondary, 3 months | Secondary, 3 and 6 months (Table 2) | matched; time point changed (6 months added) |
| Loyalty-card purchasing (fruit and vegetables, confectionery, other categories) | Not registered | "participant loyalty card data (secondary outcome)" (Sections 2.4, 3.3) | added |
| Economic outcomes (grocery spend, total store sales) | Not registered | Reported (Section 3.5) | added |
Every registered outcome is reported, so nothing is missing. The difference is in the outcome hierarchy. The only registered primary outcome is presented as secondary. Two registered secondary outcomes, together with two unregistered product groups, become the primary outcome, and a 6-month time point is added throughout.
Figures
Figure 1. Trial timeline. Blue items are registry dates; orange items come from the paper or the publisher's article page. The only registry version (v0) was submitted on 2018-04-25. That is 658 days after the registered study start (2016-07-06) and 360 days after the registered primary completion (2017-04-30). The orange bars show the study period (April 2016 to March 2017) and the recruitment window (July to September 2016) as the paper reports them. No later registry version exists, so the primary outcome was never changed in the registry. The paper (received 2020-10-15, accepted 2021-07-12, published 2021-09-07) appeared while v0 was still the only version.
Figure 2. The registered outcome hierarchy (left, ClinicalTrials.gov v0, 2018-04-25) compared with the hierarchy in the paper (right). Each connector is labelled with what changed. The sole registered primary outcome, women's dietary quality score at 3 months, is reported as a secondary outcome (DEMOTED), and a 6-month time point is added. The registered secondary weekly sales outcomes become the paper's primary outcome (PROMOTED, in the table above "added as primary"), and the time frame changes from 12 weeks to 3 and 6 months. Loyalty-card purchasing, frozen-vegetable and checkout-item sales, and economic outcomes appear only in the paper (ADDED). The bottom panel gives the paper's results for the registered primary outcome (Table 2), with the 3-month headline store-sales results for comparison, and notes that the paper discloses nothing about the difference.
Why it matters
A primary outcome named in advance limits the choices that can be made after the data are seen: which outcome is the main test, which time point counts, and what the abstract leads with. Here the only registration was created after primary completion. The paper's hierarchy then differs from that registration and the paper does not say so. A reader who opens the cited NCT number finds a different primary outcome from the one the paper names, and the paper gives no explanation.
The two hierarchies lead to different headlines. The abstract's numerical results and its first conclusion are about store sales. Fruit and vegetable sales rose by +1.71 SD (95% CI 0.45, 2.96; P = 0.01) at 3 months and +2.42 SD (0.22, 4.62; P = 0.03) at 6 months. Confectionery sales fell by −1.05 SD (−1.98, −0.12; P = 0.03) at 3 months and −1.37 SD (−2.95, 0.22; P = 0.09) at 6 months (Abstract; Results 3.2). The registered primary outcome is reported in Results 3.4 and Table 2:
| Women's diet quality score, intervention vs control, change from baseline | Beta (SD) | 95% CI | P | n |
|---|---|---|---|---|
| 3 months, adjusted for IMD (registered time point) | 0.29 | 0.01, 0.57 | 0.04 | 138 |
| 3 months, adjusted for IMD and DAG-selected confounders | 0.26 | −0.01, 0.54 | 0.06 | 132 |
| 6 months, adjusted for IMD (not a registered time point) | 0.10 | −0.16, 0.36 | 0.45 | 125 |
| 6 months, fully adjusted | 0.10 | −0.17, 0.37 | 0.48 | 119 |
Table 2 labels the IMD-only model "Unadjusted"; its footnote says "* Unadjusted = adjusted for IMD (to control for similarities between pairs of stores)." The Results text reports the 3-month IMD-adjusted estimate first: "After adjustment for IMD, diet quality score at 3 months follow-up increased from baseline by 0.29 SDs ((95% CI 0.01, 0.57), P = 0.04) among intervention women compared to those shopping in control stores."
So the registered primary outcome points the same way as the headline sales results. Its 95% CI excludes zero only in the IMD-adjusted model, and the 6-month estimate is close to zero. The abstract summarises it only as "Changes in dietary variables were predominantly in the expected direction for health benefit." The Discussion calls the diet findings "more equivocal". The paper also says it is "a pilot study, and, therefore, not powered to detect an effect" (Abstract). If the registered order had been kept, the abstract would have led with a borderline 3-month dietary effect rather than with store-sales effects. Readers and systematic reviewers should know which of these the trial was registered to test.
Explanations considered
- Paper role (secondary analysis or companion paper). This does not explain the difference. This is the main outcomes report of WRAPPED1: it reports store sales, loyalty-card purchasing and all four registered dietary outcomes, and its sample (n = 150; 62 intervention, 88 control) matches the registered enrollment of 150. It cites no other report of the registered primary outcome. Reference 59 (BMJ Open 2020;10:e036758) is the protocol of the separate WRAPPED natural-experiment study and is cited only under "Directions for future research".
- Disclosure in the paper. None found. The NCT number appears three times with no comment on its content. No passage compares the reported outcomes with the registered ones, and none says that registration happened after the study. The limitations paragraph covers the lack of randomisation, being "underpowered", and implementation that was outside the team's control, but not the outcome hierarchy. The statement in Section 2.6 about decisions made after initial analyses covers the choice of statistical methods, not the choice of primary outcome. The only supplement (S1 File) contains a flow diagram, DAGs and store-pair time series; it includes no protocol, statistical analysis plan or note on outcome changes.
- Protocol amendment. Not applicable in the registry, which has a single version that was never amended. The paper cites no protocol or statistical analysis plan for this pilot, so no protocol-level designation could be checked.
- Different wording for the same outcome. Ruled out. An individual FFQ-based dietary quality score and aggregate store sales from electronic transaction data are different measures. The registry itself lists both, the first as primary and the second as secondary.
- Pilot status. The paper calls the study a pilot, which might explain a flexible approach to analysis. However, the registration was created after data collection and still names one primary outcome. Pilot status does not explain why the paper names a different primary outcome without saying so.
- Time points. The paper explains its 0–3 and 3–6 month windows (Section 2.4: habit formation, sustained change), but never compares them with the registered 3-month and 12-week time frames. This adds a second undisclosed difference and does not explain the swap of primary and secondary outcomes.
There may be legitimate reasons for the paper's hierarchy, for example the store-level sales data being the most complete data source. This write-up does not assess them. The issue documented is only that the difference from the registered primary outcome is not disclosed.
Status
As of 2026-10-23:
- Registry history: one version only (v0, submitted 2018-04-25, posted 2018-05-08; last update submitted 2018-04-25 and posted 2018-05-08). Retrieved through the ClinicalTrials.gov history endpoints (
/api/int/studies/NCT03518151?history=true,/history/0) and the public API v2 record, which agree. - Publisher version: PLoS Med 18(9): e1003729, received 2020-10-15, accepted 2021-07-12, published 2021-09-07. The PMC archive copy (PMC8423266) was used for all quotations, and each quotation was checked verbatim with
src/show.py PMC8423266 --grep. The publisher's article page shows no correction, expression of concern or retraction; the only matches for those terms are links to the journal's policy pages. The PLOS peer-review history page could not be retrieved (HTTP 404). - Corrections and notices: none in Crossref (no update records), PMC or Europe PMC (
src/known.py). - PubPeer: not checked. Both the search page and the API returned HTTP 403 (bot challenge).
- Sources: ClinicalTrials.gov NCT03518151 (v2 API and record-history endpoints); PMC8423266 full text; journals.plos.org article page for doi:10.1371/journal.pmed.1003729; Crossref, PMC and Europe PMC notice searches. The figures are drawn by
make_figures.py.txtfrom the dates and outcomes listed in that file.
Suggested correction
A correction or clarification would make the outcome hierarchy match the registry, or explain why it does not. It could do three things:
- State in the Methods (Sections 2.1 and 2.4) and in the abstract's Trial registration line that the trial was registered on 2018-04-25, after data collection, and that the registration names women's dietary quality score (20-item FFQ) at 3 months as the sole primary outcome, with weekly store sales of fresh fruit and vegetables and of confectionery over 12 weeks as secondary outcomes.
- Either present the registered primary outcome as primary, or say that store sales were designated primary in the paper, when this was decided and why. Also state that the 6-month time point and the frozen-vegetable, checkout-item, loyalty-card and economic outcomes were not in the registration.
- Give the registered primary result in the abstract with its estimate, CI and P value: 3 months, +0.29 SD (0.01, 0.57), P = 0.04 adjusted for IMD; +0.26 SD (−0.01, 0.54), P = 0.06 fully adjusted.
Paper says trial was registered prospectively; registry record was first submitted 33 months after study start
Circulation (2021) · doi:10.1161/CIRCULATIONAHA.121.054117 · PMC8849598 · NCT03476785
Summary
The Methods state that "The trial was registered prospectively on ClinicalTrials.gov (NCT03476785)." The ClinicalTrials.gov record gives the study start as 2015-06-01 (type ACTUAL). It also shows that the record was first submitted on 2018-03-01 and first posted on 2018-03-26, 33 months (1004 days) after that start, while its status was still "Recruiting". The record does describe this trial: its final enrollment (56, ACTUAL) equals the 56 participants the paper says were randomized. The outcome reported in the paper (the LV stiffness constant at 1 year) matches the registered primary outcome in every registry version, so the outcomes were not switched. However, no version of the record existed before enrollment began, so the registry cannot show that this outcome was fixed in advance. The paper does not mention the late registration, the registry edit of 2019-04-16 that made the primary outcome description more specific and deleted one secondary outcome, or the fact that one registered secondary outcome, pulse wave velocity, is not reported.
What was registered
The ClinicalTrials.gov record history has three versions. None of them is dated on or before the study start.
| Version | Date (submitted / posted) | Status | Primary outcome (measure; time frame; description) | Secondary outcomes |
|---|---|---|---|---|
| v0 | QC 2018-03-16; first submitted 2018-03-01; posted 2018-03-26 | Recruiting; enrollment 53 (estimated) | "Left ventricular stiffness"; "1 year"; "Change in end diastolic ventricular pressure and volume curve" | "Aerobic fitness": "Change in peak aerobic capacity assessed by VO2"; "Vascular stiffness": "Change in pulse wave velocity"; "Diastolic function": "Change in left ventricular diastolic function measured by echocardiography" (all at "1 year") |
| v1 | 2019-04-16 / posted 2019-04-18 (modules changed: Study Status, Outcome Measures, Contacts/Locations) | Active, not recruiting | "Left ventricular stiffness"; "1 year"; "Change in ventricular stiffness after 1 year of exercise training measured as the stiffness constant of the diastolic limb of the left ventricular pressure-volume curve" | "Aerobic fitness": "Change in peak aerobic capacity (VO2) after 1 year of exercise training"; "Vascular stiffness": "Change in pulse wave velocity after 1 year of exercise training". The "Diastolic function" outcome was deleted. |
| v2 (current) | 2019-10-10 / posted 2019-10-14 (modules changed: Study Status, Study Design) | Completed; enrollment 56 (actual) | Same as v1 | Same as v1 |
Key registry dates, all fields of the record: - Study start: 2015-06-01, ACTUAL, in every version. - Primary completion: 2019-07-02, ACTUAL (v2). In v0 and v1 it was given as 2019-06-01, ESTIMATED.
The only outcome edit, v1 on 2019-04-16, came after recruitment had closed and 77 days before primary completion. The primary outcome measure and time frame stayed the same. The new description names a specific quantity, the stiffness constant of the diastolic pressure-volume curve, where v0 described the end-diastolic pressure-volume curve in general.
What the paper reports
The PMC copy is the NIH author manuscript (NIHMS1731529).
Registration statement. Methods, "Participant Population and Study Design", second paragraph:
The trial was registered prospectively on ClinicalTrials.gov (NCT03476785).
The abstract ends with:
Clinical Trial Registration: https://clinicaltrials.gov/ct2/show/NCT03476785
Main hypothesis and endpoint. The paper never uses the words "primary outcome" or "primary endpoint". The abstract, "Background":
In a prospective, randomized-controlled trial, we hypothesized that 1-year prolonged exercise training would reduce LV myocardial stiffness in patients with LVH.
Methods, "Statistical Analysis", which has the only other use of "primary":
The primary analysis included all participants who completed the 1-year follow-up.
Result for the stiffness endpoint. Abstract, "Results":
LV myocardial stiffness was reduced (right and downward shift in the EDPVR; pre LV myocardial stiffness 0.062±0.020 to post 0.031±0.009), whereas there was no significant change in controls (pre 0.061±0.033 to post 0.066±0.031, interaction P=0.001).
Results, "LV pressure-volume relationship":
1-year of exercise training significantly reduced LV chamber and myocardial stiffness constants (LV chamber stiffness: from 0.060±0.031 to 0.042±0.025; LV myocardial stiffness: from 0.062±0.020 to 0.031±0.009), with no significant changes in the control group (LV chamber stiffness: from 0.041±0.016 to 0.049±0.020; LV myocardial stiffness: from 0.061±0.033 to 0.066±0.031; Figure 4A: interaction P value=0.015 and Figure 4B: interaction P value=0.023).
Fitness. Results, "Effect of Exercise Training on Oxygen Uptake and LVEDV", which matches Table 2 (V̇O2 max, mL/min/kg, interaction P 0.0004):
The exercise training program resulted in a significant increase in V̇O2 max in the exercise group (from 26.0±5.3 to 31.3±5.8 mL/min/kg, P<0.0001).
Enrollment. Methods, "Participant Population and Study Design":
Fifty-six participants with LVH and elevated biomarkers were enrolled (signed a consent form) and randomized, comprising 18 controls and 38 exercisers.
The paper gives no recruitment or enrollment dates. Searches of the full text for "2015", "2016", "2017" and "2018" outside the references return nothing. Searches for "pulse wave" and "PWV" also return nothing.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| LV stiffness at 1 year (primary) | v0 2018-03-16: "Change in end diastolic ventricular pressure and volume curve"; v1 2019-04-16 and v2 2019-10-10: "...stiffness constant of the diastolic limb of the left ventricular pressure-volume curve" | LV myocardial stiffness constant "a" from the EDPVR (transmural pressure), pre vs 1 year; the main hypothesis and the headline result | matched |
| Same registered primary (alternative reading) | as above | LV chamber stiffness constant (PCWP-based EDPVR), pre vs 1 year | matched (the registry wording also fits this measure) |
| Aerobic fitness, peak VO2, 1 year (secondary) | v0, v1, v2 | V̇O2max, pre vs 1 year (Table 2, Figure 2) | matched |
| Vascular stiffness, pulse wave velocity, 1 year (secondary) | v0, v1, v2 | Not in the main text, tables or figure legends | not reported |
| Diastolic function by echocardiography, 1 year (secondary) | v0 only; deleted in v1 (2019-04-16) | Not reported. 3D echo is used only for LVEDV. | not reported (removed from the registry before primary completion) |
| LVEDV, Starling curves, preload-recruitable stroke work | Not registered | Reported (Figures 3, 5, 6), not labelled primary | added |
| Timing of registration (not an outcome) | First submitted 2018-03-01; study start 2015-06-01 (ACTUAL) | "registered prospectively" | contradicted by the registry; not disclosed |
Figures
Figure 1. Trial timeline from the ClinicalTrials.gov record history and the paper. The study start (2015-06-01, ACTUAL) comes 1004 days before the record was first submitted (2018-03-01; posted 2018-03-26, status "Recruiting"). No registry record exists for the shaded grey interval. The only version that changed the primary outcome text (v1, 2019-04-16, bold) is dated after the study start and after recruitment closed, and 77 days before primary completion (2019-07-02, ACTUAL). Version 2 (2019-10-10, open marker) changed only the status and enrollment fields. The paper's epub (2021-09-20) describes the registration as prospective.
Figure 2. Registered outcomes by registry version (left, blue) against outcomes reported in the paper (right, orange). The label on each connector names the relation. Both wordings of the registered primary outcome, LV stiffness at 1 year, correspond to the stiffness constants the paper reports. The myocardial constant is the paper's headline result and the chamber constant also fits the registry wording. The paper's result for the registered primary outcome is shown in the diagram (exercise 0.062 → 0.031; control 0.061 → 0.066). Its group × time interaction P value is 0.001 in the Abstract and 0.023 in the Results and Figure 4B. Of the registered secondary outcomes, peak VO2 is reported, pulse wave velocity is not, and echocardiographic diastolic function was deleted from the registry in v1 and is not reported. The bottom strip states the registration-timing discrepancy, which is the main finding.
Why it matters
The paper reports the registered primary outcome, and the result is positive. In completers (exercise n=20, control n=11), the LV myocardial stiffness constant fell from 0.062±0.020 to 0.031±0.009 with exercise and changed from 0.061±0.033 to 0.066±0.031 in controls. The LV chamber stiffness constant fell from 0.060±0.031 to 0.042±0.025 with exercise and changed from 0.041±0.016 to 0.049±0.020 in controls (interaction P=0.015). Nothing here suggests that a different primary outcome was substituted.
The problem is what the word "prospectively" tells a reader. Under the ICMJE recommendations, a trial counts as prospectively registered only if it was registered at or before the first participant's enrollment. Readers and systematic reviewers use that statement to decide whether the registry entry can be taken as evidence that the primary outcome, time point and analysis were fixed before any outcome data existed. Here the registry gives a study start of 2015-06-01, and its first record dates from March 2018. Primary completion was 2019-07-02 and each participant was followed for one year, so the last participant must have started the intervention by about July 2018. Of the roughly 37-month window in which participants could have been enrolled, at least 33 months came before the record existed. v0 still listed the trial as "Recruiting", so some participants may have been enrolled after registration. The paper gives no enrollment dates, so the exact split cannot be determined.
Three further points bear on how the registry can be used to check this paper: - The primary-outcome edit. The v1 change of 2019-04-16 was made after recruitment closed and before primary completion. By then most participants' 1-year data had probably been collected. The edit made the outcome more specific without changing the construct, and its new wording matches what the paper reports. It is still a post-recruitment change to the registered primary outcome, and the paper does not mention it. - Ambiguous wording. The registry says "ventricular stiffness ... stiffness constant", which could mean the chamber constant (PCWP-based) or the myocardial constant (transmural-pressure-based). The paper features the myocardial constant. Both constants show a significant interaction, so the conclusion does not depend on which one was intended. - Inconsistent P values. The abstract gives the interaction P for myocardial stiffness as 0.001. The Results and Figure 4B give 0.023 for what reads as the same comparison. Both are below 0.05, but a reader cannot tell which is correct.
The registered secondary outcome pulse wave velocity is not reported anywhere in the main text. The registered secondary outcome echocardiographic diastolic function was removed from the registry in 2019 and is also not reported. The paper explains neither omission.
Explanations considered
- Paper role. This is the primary report of the trial, not a secondary or ancillary analysis. Registered final enrollment (56, ACTUAL) equals the paper's "Fifty-six participants ... enrolled ... and randomized". The registry describes an exercise arm and a yoga arm, matching the paper's yoga-based attention control. The paper cites no other registration. Because the record describes this trial, the registration date applies to this trial's enrollment. The registry's broader title ("Prevention of Cardiovascular Stiffening With Aging and Hypertensive Heart Disease") does not change that.
- Could the study start date be wrong or refer to something other than enrollment? The start date is marked ACTUAL and is the same in all three versions. Even if it were too early, enrollment of 56 participants must have begun well before March 2018. The text reports 3,597 candidates screened, and the last participant must have started the 1-year intervention by about July 2018 to fit the ACTUAL primary completion of 2019-07-02. A March 2018 registration cannot have preceded the first enrollment unless nearly all 56 participants were enrolled in roughly four months, and nothing in the paper or the registry suggests that.
- Disclosure. The paper contains no statement that the registration was retrospective, no registration date and no reason for late registration. Its only statement about timing is the claim that registration was prospective. Searches for "registered", "protocol", "amend", "primary outcome" and "changed" found no further disclosure.
- Protocol amendments. The paper cites no published protocol or statistical analysis plan and mentions no amendments. The registry's v1 edit to the outcome measures is not mentioned. The registry also states that outcomes assessors were masked. The text says LVEDV was analysed "by an experienced cardiologist who was blinded to filling pressures". These statements are consistent with each other, but neither replaces prospective registration as public evidence of what was pre-specified.
- Wording differences. "Change in end diastolic ventricular pressure and volume curve" (v0) and "stiffness constant of the diastolic limb of the left ventricular pressure-volume curve" (v1/v2) both describe the EDPVR stiffness constant the paper reports. That accounts for the outcome match. It does not account for the registration-timing statement.
- A loose use of "prospectively". The authors might have meant that the trial was designed and conducted prospectively, which the paper also says elsewhere ("This study was a prospective randomized controlled trial"). The sentence in question, however, is specifically about registration on ClinicalTrials.gov, and in that context "prospectively" has a standard meaning that the registry dates contradict.
- Regulatory context. Whether a legal registration deadline applied to this behavioral trial was not assessed. It does not bear on whether the published statement is accurate.
Status
Checked as of 2026-10-23.
- Registry history. All three versions of NCT03476785 (v0 2018-03-16, v1 2019-04-16, v2 2019-10-10) were read from the ClinicalTrials.gov history API (/api/int/studies/NCT03476785/history/{0,1,2}), retrieved 2026-10-22/23. The current record (/api/v2/studies/NCT03476785) is identical to v2. On 2026-10-23 the history endpoint returned HTTP 403 when rechecked. The first-submitted date (2018-03-01) and study start (2015-06-01, ACTUAL) also appear in the current record, so the central finding does not depend on the history endpoint.
- Publisher version. The publisher page (ahajournals.org) returned HTTP 403, and a mirrored PDF on a university site refused the connection. The PMC text is the NIH author manuscript. The publisher-supplied PubMed abstract (PMID 34543068) has the same outcome values and the same abstract P value (interaction P=0.001), and ends "Registration: URL: https://www.clinicaltrials.gov; Unique identifier: NCT03476785." The Methods sentence "registered prospectively" could not be checked against the version of record. The online Data Supplement is not in PMC and was not checked. It could contain pulse wave velocity data, but the main text does not refer to any.
- Corrections. Crossref, PMC and Europe PMC list no correction, erratum, expression of concern or retraction (checked with src/known.py). A Letter to the Editor (doi:10.1161/CIRCULATIONAHA.121.058145, PMC8966638) raises left atrial size, atrial fibrillation, baseline BMI and follow-up length, not registration. The authors' published response was not read.
- PubPeer. The PubPeer API was blocked by Cloudflare (HTTP 403). A web search found no PubPeer thread for the DOI or title. The absence of comments is therefore not confirmed directly.
- Sources. The paper: PMC8849598, via src/show.py. The registry: ClinicalTrials.gov NCT03476785, current record and history versions 0 to 2. PubMed 34543068. Crossref, PMC and Europe PMC notices.
Suggested correction
- Replace "The trial was registered prospectively on ClinicalTrials.gov (NCT03476785)" with a statement such as: "The trial was registered retrospectively on ClinicalTrials.gov (NCT03476785; first submitted 1 March 2018), after enrollment began in June 2015." Give the reason for the late registration and, if known, how many participants were enrolled before it.
- State that the registered description of the primary outcome was revised on 16 April 2019, after recruitment closed and before primary completion, to "the stiffness constant of the diastolic limb of the left ventricular pressure-volume curve". Also state that the secondary outcome of echocardiographic diastolic function was removed at the same time.
- Specify which stiffness constant, myocardial (transmural pressure) or chamber (PCWP), is the pre-specified primary outcome. Reconcile the interaction P value for LV myocardial stiffness: 0.001 in the Abstract against 0.023 in the Results and Figure 4B.
- Report the registered secondary outcome, pulse wave velocity at 1 year, or explain why it is not reported.
Trial report omits one of two registered primary outcomes and does not disclose the omission
Lancet Reg Health Am (2025) · doi:10.1016/j.lana.2025.101339 · PMC12795688 · NCT04316221
Summary
ClinicalTrials.gov NCT04316221 has listed two primary outcomes since its first version (2020-03-18): the Caregiver Reported Child Development Instruments (CREDI) and the Guide for Monitoring Child Development (GMCD), both with a 6-month time frame. The Outcome Measures section was never edited in any of the record's six versions. The published protocol lists both instruments as primary in its abstract and data-collection schedule (Figure 7), although the defining sentence of its Primary Outcomes subsection names only the CREDI. The paper, which presents itself as the trial's primary report, names a single primary outcome (the CREDI global development score). It does not mention the GMCD anywhere, and it does not say that a registered primary outcome was dropped or why. The reported CREDI result is null. No GMCD result is reported.
What was registered
The record history (ClinicalTrials.gov history API, versions 0 to 5; the web history tab numbers the same versions 1 to 6) shows these versions and changed modules:
| Version (API) | Date | Modules changed | Primary outcomes |
|---|---|---|---|
| 0 | 2020-03-18 | first version (submitted 2020-03-16, posted 2020-03-20) | CREDI; GMCD |
| 1 | 2021-01-29 | Study Status (estimated start moved from 2020-06-01 to 2021-04-01) | unchanged |
| 2 | 2021-07-07 | Study Status (start 2021-04-01 marked actual; recruiting), Contacts/Locations | unchanged |
| 3 | 2023-01-30 | Study Status | unchanged |
| 4 | 2023-05-19 | Study Status, Study Design (enrollment changed to actual), Contacts/Locations | unchanged |
| 5 | 2026-03-12 | Study Status | unchanged |
The history summary reports "primaryOutcomesSame": true, "secondaryOutcomesSame": true and "outcomesUpdateCount": 0. The full records for versions 0, 1, 2, 4 and 5 were retrieved, and their primary-outcome entries are identical.
- Version 0 (2020-03-18), the first registration, gave an estimated start of 2020-06-01. Version 1 (2021-01-29) changed the estimated start to 2021-04-01 and was the version in force at the registered study start (2021-04-01) and when enrollment began (2021-05-28, per the paper); version 2, which marked the start as actual, was not posted until 2021-07-07. Version 1 has the same primary outcomes as version 0, word for word:
Caregiver Reported Child Development Instruments (CREDI) · time frame: 6 months · "The long form CREDI uses caregiver report and captures five domains: motor, language, cognition, social-emotional, and mental health. The CREDI is designed specifically to measure normal development within populations and has been tested in high, middle and low income countries The CREDI included a Guatemalan sample in the validation"
Guide for Monitoring Child Development (GMCD) · time frame: 6 months · "The GMCD has been used in low- and middle-income countries in children ages 0-24 months and focuses on seven areas: expressive language, receptive language, fine motor, gross motor, relating, play, and self-help." - Version 4 (2023-05-19), the last version before primary completion (2023-09-30) and before online publication (2025-12-20): the same two primary outcomes, with identical wording. - Version 5 (2026-03-12, after publication; current record): the same two primary outcomes. The record has no posted results (
hasResults: false).
The registered secondary outcomes, all at 6 months, are "changes in infant hemoglobin", "weight-for-age", "length-for-age", "Head circumference" and "Infant diet quality" (WHO Infant and Young Child Feeding indicators).
Published protocol (J Acad Nutr Diet 2022;122:432–444, doi:10.1016/j.jand.2021.03.015, online 2021-05-04; also the paper's supplement mmc2.pdf). The protocol is internally inconsistent about the GMCD. Its abstract lists both instruments as primary:
"Primary outcomes for this study are changes in global development scores, assessed using the Guide for Monitoring Global Development and the Caregiver Reported Child Development Instruments."
The body's "Primary Outcomes" subsection, however, opens with a defining sentence that names only the CREDI:
"Primary outcomes for this study are differences in global development scores between study arms, assessed using the Caregiver Reported Early Child Development Instruments (CREDI) long form at baseline (ie, infants aged 6 to 9 months) and endline."
The same subsection then describes the GMCD: "The Guide for Monitoring Child Development is extensively used alongside the CREDI in low- and middle-income countries in children aged 0 to 24 months", and "Together the Guide for Monitoring Child Development and the CREDI ... provide complementary information about child development." Figure 7 of the protocol ("Data collection description and schedule") lists the GMCD under "Primary outcome" and schedules it at endline in both groups. Under "Protocol Amendments" the protocol says: "Any modification or addition will be communicated by updating the information on the www.clinicaltrials.gov website."
Because the protocol can be read either way, the main benchmark here is the registry, which the protocol itself names as the place where changes are recorded and which lists the CREDI and the GMCD as primary outcomes in all six versions.
What the paper reports
The paper states a single primary outcome:
"The primary outcome was the overall global development score from the Caregiver Reported Early Childhood Development Instruments (CREDI) long form, which has been validated in Guatemala and piloted previously by our team."
— Methods, "Sociodemographic variables and outcomes", paragraph 2
"We hypothesized that infants receiving a single whole egg per day and local standard nutrition care during the complementary feeding period would have improved global development (primary outcome) and growth, anemia status, and diet quality (secondary outcomes) relative to infants who received only local standard nutrition care."
— Introduction, final paragraph, which opens "We report primary findings from the Saqmolo’ Project"
"Mixed linear or logistic regression models were used to estimate between-group differences in primary (global development score) and secondary (growth, anemia status, and diet quality) outcomes."
— Abstract, Methods
Its statements about the registration, protocol and analysis plan are:
"This trial was preregistered with ClinicalTrials.gov (NCT04316221). A detailed trial protocol was published previously."
— Methods, first paragraph, which also says "The study followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guideline."
"A detailed statistical analysis plan was made publicly available prior to data analysis."
— Methods, "Statistical analysis", first sentence (ref 51)
The archived PMC text, including the tables, figure legends and back matter, has no match for "GMCD" or "Guide for Monitoring". The supplementary tables (mmc1.docx) do not mention it either. The text contains no "amend" and no "deviat". The only departure from plan the paper describes concerns adherence monitoring: "More intensive weekly monitoring of self-reported egg consumption by telephone was planned but proved infeasible due to geographical limitations in cellular signal coverage."
The cited analysis plan (Harvard Dataverse doi:10.7910/DVN/1OSZOC, file "Saqmolo data analysis plan 1.19.2024.pdf", published 2024-01-23, after primary completion on 2023-09-30) says: "The primary endpoint for this study is differences between groups in child global development scores, as assessed using the Caregiver Reported Child Development Instruments (CREDI)." It does not mention the GMCD, and it does not say that the primary outcomes were changed from the registration or the protocol.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| CREDI long form, global development score, 6 months | Primary; v0 2020-03-18 to v5 2026-03-12, unchanged | Sole primary outcome. Global raw scaled score at endline (6-month intervention), with domain scores as sensitivity analyses | matched |
| Guide for Monitoring Child Development (GMCD), 6 months | Primary; v0 2020-03-18 to v5 2026-03-12, unchanged. The published protocol (2021) lists it as primary in its abstract and data-collection schedule (Figure 7); its body's defining sentence names only the CREDI | Not mentioned anywhere: not as a primary, secondary or exploratory outcome, and not in the limitations | not reported |
| Infant hemoglobin, 6 months | Secondary; v0 to v5 | Secondary: altitude-adjusted hemoglobin and anemia | matched |
| Weight-for-age, length-for-age, head circumference, 6 months | Secondary; v0 to v5 | Secondary: z scores plus underweight, stunting and low head circumference. Weight-for-length z, which the protocol lists, is also reported | matched |
| Infant diet quality (WHO IYCF indicators), 6 months | Secondary; v0 to v5 | Secondary: minimum dietary diversity, minimum meal frequency, minimum adequate diet | matched |
The randomized sample (1200) matches the registered enrollment (1200, actual).
Figures
Figure 1. Timeline of the registration and publications. The upper lane shows the six ClinicalTrials.gov versions (blue; circles are versions before the registered study start, squares are versions after it). The darker band runs from the registered study start (2021-04-01) to primary completion (2023-09-30), and the lighter band covers the period after primary completion. No version changed the primary outcomes, so the figure has no outcome-change markers: every version lists the CREDI and the GMCD. The lower lane shows the registered start and primary-completion dates, the trial dates given in the paper (orange bar, 2021-05-28 to 2023-08-03), the published protocol (2021-05-04; CREDI and GMCD as primary in its abstract and Figure 7), the analysis plan deposited after primary completion (2024-01-23, CREDI only) and the paper (2025-12-20, CREDI only). The one version dated after primary completion, v5 (2026-03-12), also follows the paper's publication and still lists both primary outcomes.
Figure 2. Registered primary outcomes and the primary outcome reported. Left (blue): the two primary outcomes registered in v0 (2020-03-18), unchanged through v5 (2026-03-12). Right (orange): the primary outcome the paper reports, with its adjusted result (β −0.08 points, 95% CI −0.22 to 0.06, p = 0.26, Table 4). The CREDI connector is labelled "MATCHED" (same instrument, same 6-month time frame). The GMCD connector, labelled "NOT REPORTED", ends at a dashed box stating that the paper names the GMCD nowhere in the text, tables or supplementary tables and gives no reason. The note at the bottom summarises the protocol (both instruments primary in its abstract and Figure 7, only the CREDI in its body's defining sentence) and the analysis plan, which lists only the CREDI and states no change.
Why it matters
- Half of the registered primary set is missing without explanation. The registry has always listed two primary development instruments, and the paper reports one. A reader who checks the paper against the registration, as CONSORT item 6b invites ("any changes to trial outcomes after the trial commenced, with reasons"), finds the GMCD missing, with no statement of whether it was collected, analysed or dropped, or when.
- The registered primary that is reported is null. In the adjusted endline model the global CREDI score did not differ between groups ("β [SE] -0.08 [0.07] points [95% CI -0.22 to 0.06]; p = 0.26"; Results, "Modeling results for primary outcome"). The change-score sensitivity analysis and the four CREDI domain scores also showed no difference, and no prespecified effect modifier interacted significantly. Endline medians were 47.7 in both groups. The CREDI result is fully reported and the omission does not affect it.
- The development conclusion rests on one of the two registered instruments. The abstract concludes that egg provision "improved diet quality but did not benefit development, growth, or anemia status among Maya infants". For development, that claim rests on the CREDI alone. The paper itself names the CREDI's mode of administration as a limitation ("The CREDI is reported by caregivers and may be subject to social desirability bias") and suggests it may explain a contrast with another trial ("This might be because of our use of the CREDI, which relies on caregiver report, versus the BSID, which uses direct assessment by a trained observer"). The protocol says the GMCD and the CREDI "provide complementary information about child development". A GMCD result, in either direction, would bear directly on how far the null development finding generalises across instruments. Readers of the paper cannot tell that such a result was planned.
- Readers relying on the analysis plan would not see the change either. The plan the paper cites was deposited after primary completion and names only the CREDI. Neither the plan nor the paper notes that it differs from the registration and from the protocol's abstract and Figure 7.
Explanations considered
- The paper is a secondary or partial report. This does not apply. The paper says "We report primary findings from the Saqmolo’ Project". It reports the registered secondary outcomes alongside the primary outcome and does not point to a separate report. A Europe PMC search for "Saqmolo" (2026-10-23) returns only this paper, the protocol (J Acad Nutr Diet 2022), a protocol abstract (Curr Dev Nutr 2021) and an egg-composition study (Food Sci Nutr 2023). None of them reports GMCD results.
- The change is disclosed. It is not. The paper's only statements about the registration and plans are that the trial "was preregistered", that a protocol "was published previously" and that an analysis plan "was made publicly available prior to data analysis". None of them mentions the GMCD or a change in primary outcomes. The limitations section does not address it either.
- A documented protocol amendment explains it. None was found. The registry's Outcome Measures module was never edited in six versions (2020-03-18 to 2026-03-12), even though the protocol commits to communicating outcome changes "by updating the information on the www.clinicaltrials.gov website". The analysis plan (version 1.19.2024) lists only the CREDI, so a narrowing appears there. But the plan does not say that it departs from the registered or protocol primary outcomes, gives no reason, and is dated after primary completion. Citing the plan therefore does not, by itself, disclose the change to the reader. Whether the GMCD was administered at all is not stated in any of these documents.
- The two names refer to the same outcome. They do not. The CREDI and the GMCD are separate instruments. The registry lists them as two primary outcomes with different descriptions (caregiver-report CREDI with five domains; GMCD with seven areas), and the protocol cites them separately and schedules them separately (Figure 7). The protocol abstract's "Guide for Monitoring Global Development" is a naming variant of the same GMCD: the protocol's Outcomes section and Figure 7 call it "Guide for Monitoring Child Development". The paper's single primary outcome is explicitly the CREDI score.
- Time-point or metric differences. These are not at issue. The CREDI is analysed at the 6-month endline, consistent with the registered time frame. The difference concerns which outcomes are reported, not when they were measured.
- A later correction resolves it. None was found (see Status).
- The protocol already made the CREDI the sole primary outcome. Partly. The defining sentence of the protocol's Primary Outcomes subsection names only the CREDI, while its abstract and Figure 7 list the GMCD as primary as well, so the protocol is ambiguous. The registry, which the protocol designates for recording any modification, lists both as primary in every version, and the paper cites the registration. The protocol's ambiguity therefore does not disclose to the paper's readers that a registered primary outcome is not reported.
The documents do not show why the GMCD is absent, and this write-up makes no inference about the reason. A change of outcomes can have legitimate reasons, such as feasibility, instrument availability or field constraints. The issue documented here is that the omission is not disclosed.
Status
As of 2026-10-23:
- Registry history. The ClinicalTrials.gov history list endpoint (
/api/int/studies/NCT04316221/history) and the version records 0, 1, 2, 4 and 5 (/history/<version>, retrieved through the study's history page in a browser because direct requests returned HTTP 403) all list both primary outcomes. The current record (API v2, last update posted 2026-03-13) also lists both. No results are posted. - Publisher version. The text quoted here is the PMC archive of the article (Lancet Reg Health Am vol 54, February 2026, article 101339; online 2025-12-20). The publisher pages (thelancet.com, sciencedirect.com) returned HTTP 403. The Elsevier article API returned metadata only (cover date 2026-02-28, open access), so the wording was not compared against the publisher's own page.
- Corrections. No correction, erratum or retraction notice was found in PMC, Crossref (no
update-toorrelationentries) or Europe PMC. - PubPeer. Not checked. The site returned a Cloudflare verification challenge.
- Sources. ClinicalTrials.gov NCT04316221 (current record and history); PMC12795688 (full text and supplements mmc1.docx and mmc2.pdf, the latter the published protocol, via Europe PMC); the protocol, J Acad Nutr Diet 2022;122:432–444 (doi:10.1016/j.jand.2021.03.015, PMID 33962902); the statistical analysis plan, Harvard Dataverse doi:10.7910/DVN/1OSZOC.
Suggested correction
A correction to the article that:
- states that ClinicalTrials.gov NCT04316221 (and the abstract and Figure 7 of the published protocol) listed two primary outcomes at 6 months, the CREDI and the Guide for Monitoring Child Development (GMCD), and that the article reports only the CREDI;
- says whether GMCD data were collected and, if not, when and why the decision was made;
- if the GMCD was collected, reports the between-group result with the same models used for the CREDI (or a reason it cannot be reported), and says when the decision to make the CREDI the sole primary outcome was taken relative to the analysis plan (version 1.19.2024) and to the start of outcome analysis, as CONSORT item 6b asks for any change to trial outcomes;
- qualifies the abstract's conclusion that the intervention "did not benefit development" as resting on the caregiver-reported CREDI only.
Separately, the ClinicalTrials.gov record could be updated to reflect the outcomes actually analysed, with the change and its reason noted, as the protocol's amendment section anticipates.
HIV care trial report redefines registered retention outcome and omits hair-sample adherence without disclosure
Lancet Reg Health Am (2022) · doi:10.1016/j.lana.2022.100307 · PMC9536254 · NCT02846350
Summary
NCT02846350 (COPA2) registered five primary outcomes on 2016-07-22, before enrolment began in November 2016. No later registry version changed them. The paper is the trial's main outcomes report. It discloses and explains two departures: viral load was analysed as a 0.50-log reduction instead of suppression, and medication persistence was not presented. It does not disclose two others. Retention in care is reported as whether the participant completed the study visit at each time point, whereas the registry and the published protocol define it as the number of missed clinic appointments in the previous 6 months. The registered primary outcome "Medication adherence: Hair sample" is not reported and is never mentioned. All the randomised comparisons that are reported are null, and the paper says so in its abstract. Its positive interpretation rests on indirect (mediation) effects that it labels exploratory.
What was registered
ClinicalTrials.gov NCT02846350 was first submitted on 2016-07-01. Version 0 passed QC on 2016-07-22 with status "Not yet recruiting" and was first posted on 2016-07-27. The registry gives the study start as 2016-07 (actual) and primary completion as 2020-11-30 (actual). The paper says baseline visits ran from November 2016 to March 2018.
Version 0 lists five primary outcomes and no secondary outcomes. The wording is quoted from the record, with escape characters removed:
| # | Measure | Description | Time frame |
|---|---|---|---|
| 1 | "HIV viral Load (VL)" | "VL will be assessed by blood sampling. Viral suppression (HIV VL <200 copies/mL) will be the primary dichotomous outcome; a secondary dichotomization will be made among detectable patients with VL <1000 and those with VL >1000." | "Less than 6 months detectable VL" |
| 2 | "Retention in care" | "Defined as the number of missed clinic appointments (i.e., "no shows" not cancelled in advance by patients or clinic staff) in the 6 months preceding assessment." | "2 years" |
| 3 | "Medication persistence" | "Medication persistence is the time to treatment discontinuation, with a permissible gap of < 30 days, participants will be considered to have discontinued the regimen if a therapy gap = > 30 days occurs." | "2 years" |
| 4 | "Medication adherence: Hair sample" | "Adherence by hair sample to estimate an average concentration of ARV medication exposure" | "2 years" |
| 5 | "Medication adherence: Self-report" | "Proportion of self-reported adherent participants over time" | "2 years" |
The record history has 13 versions, v0 (2016-07-22) to v12 (2022-06-22). None of them edits the Outcome Measures module. The history summary gives outcomesUpdateCount = 0, lastUpdateVersions.primaryOutcomes = 0 and originalData.primaryOutcomesSame = true. Version 12 was submitted the day before the paper went online (2022-06-23). Its five primary outcomes match version 0 word for word, and so does the current record (last update posted 2022-06-29). The later versions change only Study Status, Contacts/Locations, Sponsor/Collaborators, Study Design, Study Identification, Oversight, Conditions and the Document Section.
The published protocol (Trials 2018, doi:10.1186/s13063-018-2758-5, PMC6056946; reference 17 of the paper) defines the outcomes the same way. On retention:
"In this study, retention in care will be defined as the number of missed clinic appointments (i.e., "no shows" not canceled in advance by patients or clinic staff) in the 6 months preceding assessment, including that timepoint's visit with the COPA2 physician. The primary retention analysis will utilize a dichotomized indicator 0/1 of any missed visits vs. no missed visits"
On hair samples and self-report:
"Participants will provide two hair samples, at the 12-month and 24-month assessments."
"Hair sample will estimate an average concentration of ARV medication exposure over 6 weeks with cut-off values used as categorical variables (12 months and 24 months)"
"Self-report will reflect percentage adherent over the last 30 days (Visual Analog Scale; VAS)"
The protocol's analysis plan (Hypothesis 1.2) treats self-reported adherence as continuous. It "will use a linear mixed model ... to examine percentage adherence in the past 30 days as a continuous outcome".
What the paper reports
The paper states its objective (Methods, "Objectives and design"):
"The primary objective of the study was to evaluate the impact of a physician-delivered, MI patient-oriented treatment program on ART adherence, HIV viral load, retention in care, and self-efficacy among PWH disengaged from HIV care."
It cites the registration in the same paragraph ("was registered at clinicaltrials.gov, NCT02846350") and in the abstract. Its only statement about departures from plan is in Methods, "Measures of main study outcomes":
"Per the original protocol, ART adherence, HIV viral suppression, retention in care, and medication persistence were the primary outcomes of the study. At the end of the data collection period, changes to how HIV viral load was evaluated as a primary outcome had to be made given that viral suppression at baseline was a constant (see description below). Self-efficacy and patient-provider communication were the MI physician targets in the original protocol, as well as the theorised mechanisms by which the intervention was intended to work. Contrary to the study protocol, in the current study, we do not present medication persistence given issues related to pharmacy pickup data medical abstraction."
On viral load (Methods, "HIV RNA Viral Load"):
"For analyses, condition differences were analysed as a clinically significant reduction in HIV viral load from one-time point to the next, 0·50log. Though we originally intended to analyse HIV RNA viral load as the proportion of participants who were virally suppressed in each condition, the complete absence of participants who were virally suppressed at baseline precluded such analyses; that is, viral suppression at baseline was a constant at baseline."
The Discussion repeats that this "precluded the analysis of viral suppression as an outcome, as originally proposed in the protocol."
On retention, the definition is given in Methods, "Retention in HIV Care", and repeated in Results, "Retention":
"Retention in HIV care was calculated as to whether a participant completed a study clinical visit at any given time point."
"Retention was evaluated at 6 months, 12 months, 18, and 24 months as to whether the participant completed a follow-up visit."
On self-reported adherence (Methods, "Self-reported antiretroviral adherence"):
"Self-reported adherence was assessed every six months at all time points using a 10-point Visual Analogue Scale (VAS), with 0 representing none and 10 representing 100% (perfect adherence)."
The analysis covers five outcomes (Methods, "Data analyses"):
"To test the effect of the intervention on ART adherence, HIV viral load, CD4+ count, retention, and self-efficacy, mixed modelling was used"
"As such, indirect effects were examined as exploratory analyses using bivariate parallel latent growth curve (LGC) models"
A full-text search of the PMC version finds no occurrence of "hair". Hair-sample adherence is not mentioned in the text, the tables or the figure legends. No passage explains the retention definition or relates it to the registered count of missed clinic appointments. The only supplementary item listed is "Supplementary materials Image, application 1".
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| HIV viral load | Primary. "Viral suppression (HIV VL <200 copies/mL) will be the primary dichotomous outcome" (v0, 2016-07-22; unchanged to v12, 2022-06-22) | Continuous viral load, with condition differences as a "clinically significant reduction ... 0·50log". Condition p = 0.757 | Not reported as registered. The change is disclosed, with a reason |
| Retention in care | Primary. "the number of missed clinic appointments ... in the 6 months preceding assessment" (v0, 2016-07-22; unchanged to v12) | "whether a participant completed a study clinical visit at any given time point". Condition p = 0.999 | Not reported as registered. The measure is redefined and the change is not disclosed |
| Medication adherence: hair sample | Primary. "Adherence by hair sample to estimate an average concentration of ARV medication exposure" (v0; unchanged to v12) | Absent: no result and no mention | Not reported, not disclosed |
| Medication adherence: self-report | Primary. "Proportion of self-reported adherent participants over time" (v0; unchanged to v12) | Mean VAS score (0–10) in a mixed model. Condition p = 0.075 | Matches the published protocol (continuous VAS). Differs from the registry wording only (minor) |
| Medication persistence | Primary. "time to treatment discontinuation ... gap = > 30 days" (v0; unchanged to v12) | Not presented ("Contrary to the study protocol ...") | Not reported. Disclosed, with a reason |
| CD4+ count | Not registered (the registry lists no secondary outcomes) | Analysed alongside the main outcomes. Condition p = 0.584 | Added |
| Self-efficacy (HIV-ASES) | Not registered. The protocol describes it as an MI patient target or mediator | Named in the "primary objective" and analysed as a main outcome. Condition p = 0.096 | Added. The paper notes its role in the original protocol |
The registry time frame for viral load, "Less than 6 months detectable VL", is unclear. The paper assesses viral load at 6, 12, 18 and 24 months, which matches the protocol, so no time-point change is claimed here.
Figures
Figure 1. Timeline of NCT02846350. The top lane shows the 13 registry versions. Only v0 (2016-07-22, filled marker) set the primary outcomes. Hollow markers are versions that left the Outcome Measures module untouched, including the three versions (v10 to v12) after primary completion on 2020-11-30 (dashed line; shaded area after it). The middle lane shows the registry study start (2016-07) and the baseline and follow-up visit periods given in the paper (November 2016 to March 2018, follow-up to May 2020). The bottom lane shows the published protocol (2018-07-24) and the paper (online 2022-06-23, issue September 2022). The registered outcomes were in place before enrolment and never changed, so the differences in the paper did not come from a registry update. Sources: the ClinicalTrials.gov history endpoint, the PMC article and Europe PMC.
Figure 2. The five registered primary outcomes (left, blue; v0 wording, identical through v12) and the outcomes reported in the paper (right, orange). Each connector is labelled with its status. Solid connectors join a registered outcome to the measure the paper reports in its place. Dashed connectors ending in "×" mark registered outcomes with no result in the paper. Two departures are disclosed in the paper: viral load and persistence. Two are not: retention, which is redefined, and hair-sample adherence, which is absent. The self-report adherence metric matches the published protocol and differs only from the registry wording. CD4+ count and self-efficacy are added. The p values in the orange boxes are the paper's longitudinal mixed-model tests for the main effect of condition and for time × condition. The footer gives the paper's indirect and direct LGC effects. All values are quoted from the Results section.
Why it matters
Results for the registered primary outcomes. For every registered primary outcome that the paper reports, the randomised comparison is null:
- Viral load, analysed as a continuous measure (a disclosed change): condition F(1,572) = 0·10, p = 0·757; time × condition p = 0·660; LGC direct effect B = 0·13, p = 0·322.
- Retention, analysed as study-visit completion (an undisclosed change): "condition (Wald X2 = 15.36 = 0·01, p = 0·999)" (as printed); interaction p = 0·750; direct effect B = 0·48, p = 0·131.
- Self-reported adherence, as a mean VAS score: condition F(1,858) = 3·18, p = 0·075; interaction p = 0·666; direct effect B = -0·20, p = 0·388.
The abstract states these null findings plainly: "Using mixed models, the intervention had no effect on ART adherence over time by condition on HIV RNA viral load, CD4+ count retention, or self-efficacy." So the departures did not turn a null primary result into a positive one.
Where the positive conclusion comes from. The abstract's Interpretation reads: "These findings suggest that physician-delivered MI may enhance the patient-provider relationship, self-efficacy, and ART adherence, and reduced HIV viral load in patients disengaged from HIV care." This rests on indirect effects through the patient–provider relationship: adherence B = 0·188, p = 0·009; viral load B = -0·095, p = 0·027; self-efficacy B = 0·063, p = 0·001. The paper calls these "exploratory analyses". The published protocol did plan exploratory mediation analyses, with "Patient-level skills (self-efficacy, motivation, patient-provider relationship) as potential mediators". Those were planned for retention, adherence and viral suppression, though, not for self-efficacy as an outcome. None of these analyses appears in the registry.
Retention. The registered measure counts no-shows at clinic appointments over the 6 months before each assessment; the protocol dichotomises it as any versus none. The reported measure records whether the participant attended the study visit itself. Study visits were "combined with regularly occurring HIV clinical care visits" and participants "were compensated with US $10 per study visit for travel". As a result, the reported measure largely tracks completion of study follow-up, which the paper also reports as attrition (for example, 25·6% did not complete the 12-month assessment). It cannot show missed clinical appointments between study visits, and those are the behaviour a re-engagement intervention targets. The Discussion itself suggests the null retention result "may have been related to the fact that participants were recruited and incentivized by study staff, given that study and clinical visits were often scheduled on the same day". A reader comparing the paper with the registration or the protocol is not told that the measure differs.
Hair-sample adherence. This was the only objective (drug-level) adherence measure among the registered primary outcomes. The protocol scheduled collection at 12 and 24 months. The paper's adherence findings, including the indirect effect on adherence, rest entirely on self-report. A reader cannot tell from the paper whether hair samples were collected, analysed or are still unreported. Leaving out a registered objective measure without comment is the kind of selective outcome reporting that registration exists to reveal.
Explanations considered
- Paper role. The paper describes the main trial results: all four registered constructs, n = 360, two-year follow-up, and the registration cited in both the abstract and the Methods. The paper does not point to any other report of the hair-sample or missed-appointment outcomes; the word "hair" does not occur in it. Being a secondary or partial report therefore does not explain the omissions.
- Disclosure. The paper discloses the viral-load change (reason: no participant was suppressed at baseline) and the omission of medication persistence (reason: problems abstracting pharmacy pickup data). These two departures are not at issue here. Its list of protocol primary outcomes ("ART adherence, HIV viral suppression, retention in care, and medication persistence") folds the two adherence measures into one. It does not mention hair samples. It does not say that retention is measured differently from the protocol. On the viral-load change, readers may note that the eligibility criteria required "at least one HIV viral load of >500 copies/mL in the prior 3 months", and that the protocol compared suppression "at 6, 12, 18, and 24 months post baseline". Because the paper discloses this change, it is noted only for completeness.
- Registry or protocol amendments. None of the 13 registry versions changes an outcome. The registry's Document Section holds only an informed consent form (2018-10-22); no protocol or SAP is uploaded. The paper cites no protocol amendment or SAP, and none is uploaded to the registry. No amendment accounts for the retention definition or the missing hair-sample outcome.
- Wording differences. For self-reported adherence, the registry's "proportion of self-reported adherent participants" differs from the paper's mean VAS score. However, the published protocol specifies VAS percentage adherence as a continuous outcome in a linear mixed model, and the paper follows that. This is treated as a registry wording issue, not an outcome switch. For retention, the difference is not one of wording. A count of missed clinic appointments over 6 months, taken from clinic records, and a binary indicator of attending the study visit measure different things. The paper does not present them as equivalent.
- Data unavailable. The paper gives a data-availability reason for persistence. It gives none for retention or hair samples. The consent form "included a release for medical record abstraction of treatment history and laboratory results", so clinic appointment records may have been available, but the paper does not say whether they were. The issue documented here is that neither departure is disclosed, whatever the reason for it.
Status
As of 2026-10-23:
- Registry history. The ClinicalTrials.gov history (
/api/int/studies/NCT02846350/history) and versions 0 and 12 (/history/0,/history/12) were retrieved and compared with the current record (/api/v2/studies/NCT02846350). After normalising whitespace and HTML escapes, the primary outcomes are identical in all three. The registry has no secondary outcomes and no posted results (hasResults = false). - Publisher version. The article pages on thelancet.com returned HTTP 403, so the version of record was not inspected directly. This write-up uses the PMC archived text (PMC9536254). Europe PMC gives the first publication date as 2022-06-23 (Lancet Reg Health Am, vol 13, September 2022, article 100307).
- Corrections. No Crossref update relations, PMC notices or Europe PMC comment/correction links were found for 10.1016/j.lana.2022.100307.
- PubPeer. The PubPeer search page returned 403 (bot challenge) and the API returned 404 for this DOI. A web search found no PubPeer thread. The PubPeer check is therefore incomplete.
- Sources. ClinicalTrials.gov NCT02846350 and its history; the paper (PMC9536254); the published protocol (PMC6056946); Europe PMC record; Crossref metadata.
Suggested correction
The journal and authors may wish to consider a correction or clarification notice that does the following:
- States that retention was registered, and specified in the protocol, as the number of missed clinic appointments in the 6 months before each assessment (analysed as any versus none). It should say that the paper instead reports whether the study visit was completed, and why. If clinic appointment records are available, it should report the registered retention measure.
- States whether hair samples were collected at 12 and 24 months as planned, and whether they were analysed. It should either report the registered hair-sample adherence outcome or explain why it is not reported. If the outcome is reported elsewhere, it should cite that report.
- Lists the registered primary outcomes in full: five, with the two adherence measures shown separately. Against each it should give the measure actually reported or the reason none is reported, and it should mark CD4+ count and self-efficacy as outcomes not registered as primary.
- Optionally notes that the registry's "proportion of self-reported adherent participants" was operationalised as the protocol's continuous VAS measure.
The trial team could also post results to ClinicalTrials.gov for the registered outcomes, and update the registry entry with any outcome changes and their dates.
Trial registered five months into enrollment and safety primary added later; paper says "prospectively registered"
Circulation (2017) · doi:10.1161/CIRCULATIONAHA.116.026493 · PMC5459585 · NCT01858363
Summary
The paper is the primary 12-month report of a randomized trial comparing a low-dose paclitaxel-coated balloon (DCB) with uncoated angioplasty (PTA). It states that "The study was prospectively registered". The ClinicalTrials.gov record was first submitted on 2013-05-08, but the paper reports that randomization began in December 2012 and the registry gives the study start as 2012-11. In the first registered version the only primary outcome was 12-month primary patency. The primary safety end point, one of the paper's two primary end points, was added as a registered primary outcome on 2014-02-12, about 14 months after randomization began and before primary completion. Both primary end points reported in the paper match the registration in force at publication, and neither the late registration nor the added primary outcome is mentioned in the paper.
What was registered
Five registry versions exist. They are numbered 1-5 here, in date order (indices 0-4 in the ClinicalTrials.gov history API). Every version gives the study start as "2012-11", and every version gives the first submission date as 2013-05-08.
Version 1, 2013-05-20 (first submitted 2013-05-08, first posted 2013-05-21; status Recruiting; estimated enrollment 360). This is the earliest version, and no version exists on or before the study start. It has one primary outcome:
Primary outcome: "Primary Patency at 12 months post-procedure" (time frame: "12 months")
Secondary outcome: "Major adverse event rate" (time frame: "1, 6, 12, 24 months")
No safety outcome is registered as primary.
Version 2, 2014-02-12 (status Recruiting). The module labels for this version include "Outcome Measures". It has two primary outcomes and no secondary outcomes:
"Primary Efficacy Endpoint" (time frame "12 months"); description: "Primary Patency at 12 months post-procedure"
"Primary Safety Endpoint" (time frame "30-Days and 12-Months"); description: "Freedom from device and procedure related target limb major amputation or death through 30-days post-procedure and clinically-driven target lesion revascularization at 12-month"
The secondary outcome "Major adverse event rate" was deleted in this version.
Version 3, 2015-06-25 and version 4, 2016-10-05. The outcomes are unchanged from version 2. Version 4, which changed only the Study Status and IPDSharing modules, records primary completion as "2016-03" (actual). It is the last version before the paper was received (2016-11-21) and published online (2017-04-19).
Version 5, 2021-10-11 (results first posted 2021-10-13; after publication). The primary outcomes were reworded for results reporting. The measures and time points did not change; the patency description adds the definition used in the paper (PSVR ≤2.5 and freedom from CD-TLR), which versions 1-4 did not spell out:
"Patency at 12-months" (12 months): "The primary efficacy endpoint was patency at 12 months post-procedure defined as the absence of target lesion restenosis determined by duplex ultrasound peak systolic velocity ratio (PSVR) ≤2.5 and freedom from clinically-driven target lesion revascularization (CD-TLR)."
"Percentage of Patients With Freedom From Device and Procedure Related Target Limb Major Amputation or Death Through 30-days Post-procedure and Clinically-driven Target Lesion Revascularization at 12-month" (30-Days and 12-Months)
The posted results (83.9% vs 60.6% for patency; 94.1% vs 83.3% for safety) agree with the paper. Actual enrollment is given as 294.
What the paper reports
The primary end points, from the Abstract (Methods):
"The primary safety end point was a composite of freedom from device- and procedure-related death through 30 days after the procedure and freedom from target limb major amputation and clinically driven target lesion revascularization through 12 months after the procedure. The primary effectiveness end point was primary patency at 12 months."
And from Methods > Outcomes:
"The primary effectiveness end point was primary patency (per lesion) through 12 months, defined as the absence of target lesion restenosis (duplex ultrasound–determined peak systolic velocity ratio ≤2.5) and freedom from CD-TLR."
The registration statement (Methods > Study Design, para 1):
"The study was prospectively registered (http://www.clinicaltrials.gov; unique identifier: NCT01858363)."
The enrollment period (Results > Patient Enrollment and Follow-Up, para 1):
"Between December 2012 and April 2015, 294 patients were randomized to DCB (222 patients, 254 lesions) or PTA (72 patients, 79 lesions) at 18 centers in Germany and Austria"
The paper's statements about prespecification are in Methods > Statistical Analysis. Para 1 sets out the two primary hypotheses (noninferiority with a 5% margin, then superiority, for safety; superiority for patency) and says of the safety hypothesis:
"A priori, if noninferiority was met and the lower 95% confidence limit was >0%, superiority would be claimed."
Para 2 says:
"All analyses were prespecified in a statistical analysis plan."
Neither statement gives a date, or says how the primary safety hypothesis relates to the registry entry of 2014-02-12 that first listed the safety composite as a primary outcome. No protocol amendment, no change to the primary end points and no registration date is mentioned anywhere in the text. The word "protocol" appears only in a sentence about the antiplatelet regimen (Methods > Study Device and Procedure). The Limitations paragraph does not mention registration.
The discrepancy
| Outcome | Registered (version, date) | In the paper | Status |
|---|---|---|---|
| 12-month primary patency | Primary in v1 (2013-05-20) as "Primary Patency at 12 months post-procedure"; primary "Primary Efficacy Endpoint" in v2-v4 (2014-02-12 to 2016-10-05) | Primary effectiveness end point: primary patency through 12 months (PSVR ≤2.5 and freedom from CD-TLR) | matched |
| Safety composite (device/procedure-related death through 30 days; target limb major amputation and CD-TLR through 12 months) | Absent from v1. Primary "Primary Safety Endpoint" from v2 (2014-02-12), during enrollment, before primary completion | Primary safety end point | added (in v2, about 14 months after randomization began; matches v2-v4; change not disclosed) |
| Major adverse event rate | Secondary in v1 only (1, 6, 12, 24 months); deleted in v2 (2014-02-12) | Reported as "Additional outcomes" (MAEs), not as a primary end point | not a primary outcome in either source; no primary-outcome status change |
| Registration timing | First submitted 2013-05-08 (posted 2013-05-21); study start 2012-11 in every version | "prospectively registered"; randomization "Between December 2012 and April 2015" | not prospective: registered roughly 5 months after the first randomization (about 6 months after the registry start date); not disclosed |
The safety composite has a minor wording difference. In the registry wording for v2-v4 ("Freedom from device and procedure related target limb major amputation or death through 30-days post-procedure and clinically-driven target lesion revascularization at 12-month"), the 30-day window could be read as applying to amputation as well as death. The paper puts amputation in the 12-month window. The version 5 description ("freedom from device and procedure-related death through 30 days post-procedure and freedom from target limb major amputation and clinically-driven target lesion revascularization (CD-TLR) through 12 months") matches the paper exactly. The wording also gives the same measure and time points, so this is classed as matched.
Figures
Figure 1. Timeline of NCT01858363 and the paper. Blue squares are registry versions: filled squares mark versions that changed the primary outcomes, and open squares mark versions that did not. Grey diamonds mark the registry study start (2012-11) and primary completion (2016-03). The orange bar is the randomization period reported in the paper (December 2012 to April 2015). Its hatched part is the stretch before the record was first submitted (2013-05-08). The orange circle marks online publication (2017-04-19). The safety composite was added as a primary outcome in version 2 (2014-02-12), after the study start and during enrollment, but before primary completion. The shaded area is the period after primary completion (axis broken between 2017 and 2021). Version 4 (2016-10-05) falls in it but changed only Study Status and IPDSharing. The only outcome change in that period is version 5 (2021-10-11), which reworded the primaries for results posting. All dates come from the registry history and the paper. Source: make_figures.py.txt.
Figure 2. On the left, in blue, are the registered outcomes across versions 1-4 (2013-05-20 to 2016-10-05), including the registered secondary outcome and the registration date. On the right, in orange, are the outcomes as reported in the paper, with the paper's results. Each connector carries a label for its status. The 12-month patency outcome was registered as primary from the first version and matches the paper's primary effectiveness end point. The safety composite was not in version 1. It was added as a primary in version 2 (2014-02-12) and matches the paper's primary safety end point. The registered secondary "Major adverse event rate" (dashed boxes) was deleted in version 2. The paper reports MAEs as an additional outcome, so no primary outcome was dropped or demoted. The bottom row sets the registration date beside the paper's "prospectively registered" statement.
Why it matters
The paper reports a result for each registered primary outcome, and both results are positive. For primary patency at 12 months, the paper reports "83.9% (188 of 224) for DCB and 60.6% (40 of 66) for PTA, for a difference of 23.3% (95% confidence interval, 10.6%–36.1%; P<0.001)" (Results > Effectiveness Outcomes). For the primary safety composite, it reports "94.1% (193 of 205) with DCB and 83.3% (50 of 60) with PTA, for a difference of 10.8% (95% confidence interval, 0.9%–23.0%)", meeting both noninferiority and superiority (Results > Safety Outcomes). The abstract concludes: "Superiority with a low-dose DCB for femoropopliteal interventions was demonstrated over PTA for both the safety and effectiveness end points."
The registry and the paper report the same outcomes, so this is not a case of a switched primary outcome. The issue is what the registry can and cannot vouch for:
- Registration was not prospective. Under the ICMJE definition, prospective registration means registration before the first patient is enrolled. Here the record was first submitted after randomization had started. The paper's own dates place that start in December 2012, and the registry places the study start in 2012-11. The paper's statement that the study "was prospectively registered" therefore does not agree with the registry or with the paper's own enrollment dates. Registration came early in a randomization period of about 29 months and well before primary completion. The registry cannot show that the first registered primary outcome was fixed before enrollment, but it does show that it was fixed before most patients were randomized.
- One of the two primary end points was registered during enrollment. The safety composite, which supports half of the abstract's conclusion of "superiority ... for both the safety and effectiveness end points", first appears as a registered primary outcome on 2014-02-12, about 14 months after randomization began and roughly 14 months before it ended. The paper does not date the primary safety hypothesis or its noninferiority margin. It states that the superiority rule was set "A priori" and that analyses were "prespecified in a statistical analysis plan", but gives no date for either. A reader of the paper cannot tell that, by the registry's account, this end point became primary while the trial was under way. The 12-month data were not complete at the time (primary completion 2016-03).
- Readers and evidence syntheses rely on the registration statement. Reviewers who use "prospectively registered" as a marker of low risk of selective reporting could score this trial differently if they had the registry dates.
Explanations considered
- Paper role. The paper is the primary 12-month report of the trial (Methods > Patient Follow-Up: "the primary effectiveness and safety results at 12 months of follow-up presented here"). It is not a secondary or post hoc analysis, so the registered primary outcomes apply to it in full. This does not explain the discrepancy.
- Disclosure. The paper contains no statement about the registration date, about any amendment of the primary end points, or about a registry update. The only registration statement calls the registration "prospective". A search of the archived text for the NCT number, "registered", "protocol", "amend", "primary outcome", "primary end", "changed" and "December 2012" found nothing that discloses either point. The online-only Data Supplement could not be retrieved (the publisher returned HTTP 403). The paper cites it only for eligibility criteria (Table I) and the list of centers, so a disclosure there is unlikely but not ruled out. Not disclosed.
- Protocol amendments. The addition of the safety composite in version 2 may reflect a protocol amendment, for example to set up a noninferiority safety hypothesis. A change made during enrollment and before outcome data were complete can be legitimate. But no protocol, amendment history or statistical analysis plan is published, cited, or posted to the registry (the registry record has no document section). Version 2 also revised other modules (Study Identification, Study Status, Oversight, Study Design, Contacts/Locations). A legitimate amendment would account for why the registry changed, but not for why the paper is silent about it and calls the registration prospective.
- Start date revised later. This is ruled out. Version 1 already gave the study start as 2012-11 with status Recruiting, and the paper independently dates the first randomization to December 2012. The late registration does not come from a start date edited afterwards.
- Earlier registration elsewhere. None was found. The paper cites only NCT01858363, and the registry record lists no other identifiers that predate it.
- Wording differences. Version 2 relabelled the patency outcome as "Primary Efficacy Endpoint" and moved the original wording into the description. Version 5 reworded both primaries for results posting and added the paper's operational definition of patency (PSVR ≤2.5 and freedom from CD-TLR). These changes did not alter the measure or the time point, and they do not account for the discrepancy. The ambiguous 30-day window in the v2-v4 safety wording is a drafting difference, and the version 5 text resolves it in line with the paper.
- "Prospective" referring to the design. The abstract uses "prospective" for the trial design ("This was a prospective, randomized, multicenter, single-blinded trial"), which is accurate. The Methods sentence applies the word to the registration itself ("was prospectively registered"), so this reading does not resolve the statement.
Status
As of 2026-10-23:
- Registry history. Five versions of NCT01858363 were reviewed: 2013-05-20, 2014-02-12, 2015-06-25, 2016-10-05 and 2021-10-11. The source was the ClinicalTrials.gov history endpoint (
/api/int/studies/NCT01858363/history/<version>), from copies retrieved earlier for this project; the live endpoint returned HTTP 403 on 2026-10-23. The current record (/api/v2/studies/NCT01858363, last update posted 2021-10-13) agrees with version 5. - Publisher version. The text checked is the PMC archive of the version of record, Circulation 2017;135(23):2227-2236 (received 2016-11-21, accepted 2017-03-29, online 2017-04-19, issue 2017-06-06). The online-only Data Supplement was not accessible (HTTP 403).
- Corrections. No correction, erratum, expression of concern or retraction was found in Crossref (no
update-toentries), in PMC notices or in Europe PMC notices. Europe PMC lists only a "Comment in" editorial (doi:10.1161/CIRCULATIONAHA.117.028308), which is not a correction. - PubPeer. The PubPeer search page was blocked by a bot check, and a web search found no PubPeer thread for this article. Whether it has PubPeer comments is unconfirmed.
- Sources. ClinicalTrials.gov record and its history tab; PMC5459585; Crossref metadata; Europe PMC.
Suggested correction
The authors and the journal could correct the registration statement in Methods > Study Design and add the registry history. Suggested wording:
"The study was registered at http://www.clinicaltrials.gov (unique identifier: NCT01858363) on May 8, 2013, after enrollment had begun (first randomization December 2012). The initial registration listed primary patency at 12 months as the primary outcome. The primary safety end point was added to the registered primary outcomes on February 12, 2014, during enrollment and before the 12-month follow-up was complete."
If a protocol amendment set up the safety end point and its noninferiority margin, the correction could also give the amendment's date and rationale and say when the statistical analysis plan was finalized. Making the protocol and the plan available, for example as supplementary material, would let readers check that both primary hypotheses were fixed before outcome data were unblinded. The reported results do not need to change.
Not written up after the deeper check (9)
These earlier-confirmed findings were dropped because the deeper check found a fixing correction, a disclosed or legitimate change, a version-of-record difference, or a failed blind re-check.
- PMC4108080 (Cell 2010): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC12883204 (Lancet Oncol 2026): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC11450708 (Ann Intern Med 2024): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC4854752 (Lancet Haematol 2015): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC4551397 (Ann Intern Med 2015): re-derivation confirmed; the error is only in the PMC copy (an author manuscript); the publisher's version of record is correct
- PMC7144257 (Eur Respir J 2020): re-derivation confirmed; version of record yes; correction: The error is real, and the authors have published a correction for it. Erratum: Eur Respir J 2020;56(3):2050524, doi:10.1183/13993003.50524-2020 (PMID 32973076, PMC8410236, Europe PMC pubType 'Publish; gate: The Author Correction (Eur Respir J 2020;56(3):2050524, doi:10.1183/13993003.50524-2020, PMC8410236) fixes this exact error. I read the correction text on PMC. It replaces the 95% CIs in both the Abstract and Table 4 and leaves the ORs and p-values unchanged. The corrected rows are: age >=65, 3.765
- PMC11802928 (Commun Biol 2025): re-derivation uncertain; version of record yes
- PMC12366704 (Sci Adv 2025): re-derivation refuted; version of record yes
- PMC12980234 (JAMA Netw Open 2026): blind re-check refuted; paper role primary_report